Commit Graph

26 Commits

Author SHA1 Message Date
vvzvlad
551f975886 fix(collab): use '-' instead of ':' in agent page-history jobId
BullMQ rejects custom job IDs containing ':' (Redis key separator),
throwing "Custom Id cannot contain :" inside the onStoreDocument hook
for every agent edit. This broke agent-driven page saves (MCP
create_page runs as actor='agent') with HTTP 400.

Switch the agent dedup suffix from `${page.id}:agent` to
`${page.id}-agent`. The jobId is only used as a BullMQ dedup key and is
never parsed by the history processor; page.id is a UUID, so the
hyphenated id cannot collide with a human job whose id is a bare page.id.
2026-06-17 17:38:32 +03:00
vvzvlad
0a9788e89a feat(collab): separate agent edits from human edits in page history
Page-history snapshots are debounced/coalesced (one per 1–5 min window,
jobId=page.id). A human edit followed by an agent edit in the same window
collapsed into a single snapshot, losing both the pre-agent human state and
a deterministic record of the agent's result.

Two provenance-aware boundaries now bracket an agent intervention:
- Before: on a user->agent transition, onStoreDocument synchronously pins the
  current (pre-agent) human content as its own history version tagged 'user',
  inside the page-write transaction, before the agent overwrites it.
- After: agent stores enqueue an immediate (delay 0), source-keyed history job
  (jobId=`${pageId}:agent`) so the agent's result snapshots deterministically
  as 'agent' and a later human edit (jobId=page.id) cannot coalesce/retag it.

Also add an `id desc` tie-break to findPageLastHistory so "last history" stays
deterministic when two snapshots share a created_at, consistent with
findPageHistoryByPageId.

Known trade-offs (Variant 1): the delay-0 worker re-reads the row, leaving a
millisecond mis-tag window; multiple agent edits in one turn may yield multiple
versions. The reverse agent->human boundary is intentionally out of scope.
2026-06-17 06:40:28 +03:00
vvzvlad
683da7a4c5 feat(ai-chat): per-user AI agent backend — LLM config, read-only agent, provenance schema
WIP checkpoint of the gitmost AI-chat backend (plan stages A + B1 + B3a).
The agent acts under the requesting user's JWT (Docmost CASL enforces page
access); the external service-account /mcp endpoint is untouched.

LLM provider config (A2-A4):
- integrations/crypto: AES-256-GCM SecretBoxService (key derived from APP_SECRET,
  per-record salt/iv; clear error on rotation instead of crashing).
- ai_provider_credentials table/repo/types: encrypted API key stored outside
  workspace settings/baseFields, write-only (never returned by any endpoint).
- integrations/ai: per-workspace AI SDK v6 provider driver (openai/gemini/ollama),
  admin-gated GET(masked)/PATCH(write-only key)/Test endpoints; settings.ai.provider
  holds non-secret config incl. systemPrompt. Removed unused AI_* env getters (DB is
  the single source of truth).

Chat module (A1, A5-A8):
- ai_chats/ai_chat_messages repos (workspace-scoped, soft-delete, tsv never selected).
- core/ai-chat: CRUD + POST /ai-chat/stream (Fastify hijack + AI SDK v6
  pipeUIMessageStreamToResponse, abort on disconnect, persist user/assistant msgs).
- Agent loop: streamText + stepCountIs(8); read tools searchPages/getPage via a
  per-request DocmostClient over loopback REST under the user's minted access token.
- Gate settings.ai.chat (+ 503 when provider unconfigured); buildSystemPrompt with a
  non-removable safety/anti-prompt-injection framework. Per-user rate limit.

Per-user auth (B1):
- @docmost/mcp DocmostClient gains an additive getToken variant (carry a user JWT,
  re-fetch on 401) and exports DocmostClient; the email/password service-account path
  (external /mcp, stdio) is unchanged.

Agent-edit provenance backbone (B3a):
- Migration: pages/page_history (last_updated_source, last_updated_ai_chat_id) and
  comments (created_source, ai_chat_id, resolved_source).
- Signed actor/aiChatId claim in the collab token; onAuthenticate propagates it,
  onStoreDocument writes it with a sticky agent marker, saveHistory copies it.

Migrations auto-run on boot (additive). Write tools, frontend, RAG and external MCP
servers are not in this checkpoint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 01:36:41 +03:00
Philip Okugbe
33895b0607 bug fixes (#2250)
* util

* fix page position collation

* support fixed toolbar in templates editor

* date localization

* fix clipped emoji in templates editor

* fix page updated time object

* fix flickers

* fix: remove redundant breadcrumb from destination modal
2026-05-28 16:20:37 +01:00
Philip Okugbe
de60aa7e61 feat: synced blocks (transclusion) (#2163)
* feat: synced blocks (transclusion)

* fix:remove name

* make placeholders smaller

* feat: enforce strict transclusion schema

* fix: scope synced blocks to workspace, gate unsync on edit permission

* fix collab module error
2026-05-08 13:23:16 +01:00
Philip Okugbe
879aa2c3d8 feat: page update notifications (#2074)
* feat: watchers notification and email preferences

* fix: email copy

* digests

* clean up

* fix

* clean up

* move backlinks queue-up to history processor

* fix

* fix keys

* feat: group notifications

* filter

* adjust email digest window
2026-03-31 16:03:59 +01:00
Philip Okugbe
05b3c65b0f feat: notifications (#1947)
* feat: notifications
* feat: watchers

* improvements

* handle page move for watchers

* make watchers non-blocking

* more
2026-02-14 20:00:38 -08:00
Philip Okugbe
18b5781522 feat(API): page content update and retrieval (#1937)
* feat: page content update and retrieval output

* import module

* refactor naming
* support prepend

* rename contentOperation -> operation

* dry

* add yjs utils
2026-02-12 11:13:47 -08:00
Philip Okugbe
0f02261ee6 feat: page version history improvements (#1925)
* Refactor: use queue for page history

* feat: save multiple version contributors

* display contributor avatars in history list

* fix interval
2026-02-09 18:25:35 -08:00
Philip Okugbe
9fb16bc842 feat(EE): AI vector search (#1691)
* WIP

* AI module - init

* WIP

* sync

* WIP

* refactor naming

* new columns

* sync

* sync

* fix search bug

* stream response

* WIP

* feat embeddings sync

* refine

* Add workspaceId to page events

* refine

* WIP

* add translation string

* sync

* reset ai answer on query change

* hide AI search in cloud

* capture streaming error

* sync
2025-12-01 11:50:25 +00:00
Philip Okugbe
6d024fc3de feat: bulk page imports (#1219)
* refactor imports - WIP

* Add readstream

* WIP

* fix attachmentId render

* fix attachmentId render

* turndown video tag

* feat: add stream upload support and improve file handling

- Add stream upload functionality to storage drivers\n- Improve ZIP file extraction with better encoding handling\n- Fix attachment ID rendering issues\n- Add AWS S3 upload stream support\n- Update dependencies for better compatibility

* WIP

* notion formatter

* move embed parser to editor-ext package

* import embeds

* utility files

* cleanup

* Switch from happy-dom to cheerio
* Refine code

* WIP

* bug fixes and UI

* sync

* WIP

* sync

* keep import modal mounted

* Show modal during upload

* WIP

* WIP
2025-06-09 04:29:27 +01:00
Philip Okugbe
e8847bd9cd fix: handle unhandled exceptions (#1116)
* Handle unhandled exceptions
* cleanup
2025-04-29 23:29:00 +01:00
Philip Okugbe
64f0531093 feat: keep track of page contributors (#959)
* WIP

* feat: store and retrieve page contributors
2025-04-04 13:03:57 +01:00
Philip Okugbe
e209aaa272 feat: internal page links and mentions (#604)
* Work on mentions

* fix: properly parse page slug

* fix editor suggestion bugs

* mentions must start with whitespace

* add icon to page mention render

* feat: backlinks - WIP

* UI - WIP

* permissions check
* use FTS for page suggestion

* cleanup

* WIP

* page title fallback

* feat: handle internal link paste

* link styling

* WIP

* Switch back to LIKE operator for search suggestion

* WIP
* scope to workspaceId
* still create link for pages not found

* select necessary columns

* cleanups
2025-02-14 15:36:44 +00:00
Philipinho
1f4bd129a8 editor improvements
* add callout, youtube embed, image, video, table, detail, math
* fix attachments module
* other fixes
2024-06-20 14:57:00 +01:00
Philipinho
d4eefa48a8 restructure directories
* set log level based on env
2024-06-09 15:57:52 +01:00
Philipinho
2afbede8ce fix page history generation 2024-06-07 16:23:23 +01:00
Philipinho
886d9591fa frontend permissions
* rework backend workspace permissions
2024-06-03 02:54:12 +01:00
Philipinho
9c7c2f1163 updates and fixes
* seo friendly urls
* custom client serve-static module
* database fixes
* fix recent pages
* other fixes
2024-05-18 03:19:42 +01:00
Philipinho
a2768e7d30 collaboration module cleanup x2 2024-05-04 17:21:44 +01:00
Philipinho
6a57e3edb2 Make colloboration module not dependant on page module
* add unloadImmediately to collab
2024-04-22 23:40:50 +01:00
Philipinho
c18c9ae02b Refactoring
* replace TypeORM with Kysely query builder
* refactor migrations
* other changes and fixes
2024-03-29 01:46:11 +00:00
Philipinho
cdcb4e87d0 * Make JwtGuard better
* Fix auth decorators
2024-02-28 03:07:24 +00:00
Philipinho
a0ec2f30ca feat: search 2024-01-30 00:14:21 +01:00
Philipinho
9a8b605f70 * add new tiptap editor extension monorepo package
* move tiptap packages to main package.json
* add tiptap extensions schema to collaborative backend
* add basic README
2024-01-14 23:05:41 +01:00
Philipinho
093e634c0b switch to nx monorepo 2024-01-09 18:58:26 +01:00