Commit Graph

51 Commits

Author SHA1 Message Date
claude code agent 227 3b80285d57 fix(#260): open MCP collab docs by canonical UUID (slugId doc-name split)
Real root cause of the silent MCP edit loss: the web editor always opens the
collaboration document by the page UUID (`page.${page.id}`), but the MCP
opened it by the agent-supplied id — usually a slugId — so `page.${pageId}`
became `page.<slugId>`. For one DB page that is TWO independent Yjs documents;
both persist to the same `pages` row (findById/updatePage resolve id or
slugId), so the human tab's debounced store overwrites the agent edit
(last-store-wins) — gone after reload, never shown live. The slugId doc also
made the server's transclusion sync + embedding reindex throw Postgres 22P02.

Fix:
- MCP (primary): resolvePageId(pageId) returns the canonical UUID — a UUID
  short-circuits with no network call, a slugId resolves once via getPageRaw
  and is cached both ways. Every collab-write path (mutatePageContent /
  updatePageContentRealtime / replacePageContent and the mutate/replace/
  unlocked seams) now opens by the resolved UUID, so the MCP and the editor
  share ONE Yjs doc. replaceImage's whole-operation page lock also keys on the
  UUID so it serializes against the other (now-UUID-keyed) writes.
- Server (defense + kills the 22P02 noise): onStoreDocument passes the resolved
  page.id — not the raw doc-name id — to syncTransclusion, the embedding queue,
  the mention-notification job, addContributors, and the in-tx history read.
  Content store and the empty-guard are untouched.

Tests: a new MCP test stands up a real Hocuspocus server and asserts a slugId
input opens `page.<uuid>` (never `page.<slugId>`), with UUID short-circuit and
single-resolve caching; the server spec asserts the side-effects receive the
UUID for a `page.<slugId>` doc. closes #260

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 10:04:49 +03:00
claude code agent 227 cce539e8e2 fix(collab): hoist intentional-clear consume out of the store retry loop (#251)
The store-side empty-guard consumed the per-document intentional-clear flag
INSIDE the bounded retry loop. consumeIntentionalClear always deletes the
in-memory Map entry, but a tx rollback cannot un-delete it: attempt 1
consumed the flag then updatePage threw a transient error and rolled back;
attempt 2 re-read the page non-empty, saw the flag gone, and the empty-guard
silently BLOCKED the write — dropping the user's deliberate clear and
defeating the retry guarantee for clears.

Hoist the decision out of the loop (like consumeContributors /
consumeAgentTouched): consume once into `allowIntentionalClear` before the
`for`, and only read that boolean on the empty-over-non-empty branch. The
single hoisted consume still drops a pending flag for a non-empty store
(the "cleared then retyped" case), since every store consumes regardless of
incoming emptiness.

Add a regression test: arm via the real onStateless transport, updatePage
throws once then succeeds, assert it is called twice and the retry writes the
empty doc (the clear survives). It fails on the old consume-in-loop ordering
(updatePage called once) and passes after the hoist.

Document the known fail-safe limitation near the TTL constant: if document
ownership transfers / a node crashes between the stateless signal and the
debounced store, the in-memory flag is lost and the clear is silently not
applied (the doc reloads non-empty) — fail-safe, content is never destroyed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 04:17:41 +03:00
claude code agent 227 3fdb1e05a4 feat(collab): persist a deliberate page clear via an intentional-clear signal (#251)
The #248 store-side empty-guard (onStoreDocument) unconditionally refuses to
overwrite non-empty persisted content with an empty document, because a
momentarily-empty live Y.Doc is indistinguishable from a real clear at the
store layer. That correctly blocks glitches/bad-merges, but also blocks a user
who genuinely wants to empty a page. This re-introduces a WORKING, narrow,
non-spoofable exception (the dead context.intentionalClear hatch #248 removed
never had a real channel).

Definition of an intentional clear (client, IntentionalClear editor extension):
a LOCAL user transaction (docChanged, NOT a remote y-sync change — filtered via
isChangeOrigin) that reduces a non-empty doc to the empty single-paragraph
shape. This is exactly the select-all + Delete/Backspace keystroke path.

Transport (option b — hocuspocus stateless message): on that transition the
client sends a `{type:'intentional-clear'}` stateless message. The server
(PersistenceExtension.onStateless) records a short-lived (TTL 60s > 45s
maxDebounce), single-use "pending clear" flag keyed by the connection's
document. The next debounced onStoreDocument consumes it on the empty-guard
branch to let that one empty write through.

Why this is the right channel and non-spoofable:
- Yjs transaction origin/metadata does not survive to the server store; awareness
  is per-connection and racy. A stateless message ties the signal to a specific
  clear, survives the debounce, and rides the authenticated connection.
- The document is taken from the connection, never the payload, so a client
  cannot target another page.
- The flag is read ONLY on the empty-over-non-empty branch, so the worst a forged
  signal can do is clear a page the connection may already edit; it can never
  force or alter a non-empty write. Read-only connections cannot arm it. Every
  non-empty store drops a pending flag, so "cleared then retyped" leaves nothing
  usable; the flag is single-use and TTL-bounded.

NOTE: #248 is not yet on develop, so the empty-guard block is included here as
the foundation this exception extends. If #248 lands first this rebases cleanly
(the guard logic is identical; the #251-unique additions are the exception,
onStateless, the pending-flag state, and the client extension).

Tests:
- Server (real transport path, not a hand-poke): onStateless sets the flag with
  the exact client payload, then the debounced onStoreDocument persists the empty
  doc; plus single-use consumption, read-only rejection, non-empty-store drops
  the flag, and the unchanged #248 guard tests (empty-over-non-empty blocked,
  empty-over-empty allowed).
- Client: a real Editor + the actual selectAll+deleteSelection command emits the
  signal; typing / non-emptying edits / already-empty docs do not.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 04:06:39 +03:00
claude code agent 227 2d36641f28 test(coverage): add regression tests for issues #192, #206, #204
Additive test coverage across server, editor-ext, client and mcp.

#192 — AiChatService.stream integration (Section 3, against real Postgres):
- new apps/server/test/integration/ai-chat-stream.int-spec.ts drives the real
  streamText through a seeded ai/test MockLanguageModelV3 and a real Node
  ServerResponse, covering: onError persists an assistant error record
  (status 'error' + partial answer + provider cause in metadata); external MCP
  client closed exactly once on BOTH onFinish and onError; anti-tamper —
  history is rebuilt from the DB transcript, not from body.messages.

#206 — red-team findings (most already fixed+tested in #212):
- mdrt-2 (UNFIXED, data loss): turndown.dataloss.test.ts documents that
  pageBreak / transclusionReference / mention are silently dropped on Markdown
  export (characterization + it.fails for the desired survive-export contract).
- persist-6 (UNFIXED, data loss): persistence-store.spec.ts adds an it.failing
  documenting that a momentarily-empty live doc overwrites non-empty content
  (left unfixed — a store-side empty-guard is a behaviour change).

#204 — test-strategy plan, highest-priority subset:
- Phase 1: mcp-clients.lease.spec.ts covers the external MCP client
  lease/refcount/eviction lifecycle (leak / premature-close / double-close).
- Phase 2 data-integrity pure functions: editor-ext table-utils
  (transpose/moveRow/convert round-trip) and math tokenizer false-positive
  guard; client emoji-menu (+ it.fails for the unguarded localStorage
  JSON.parse bug), sort-cells, normalizeTableColumnWidths; mcp htmlEmbed/
  pageBreak markdown data-loss + footnote-diff; server export
  getInternalLinkPageName extensionless-path bug — FIXED (small/clear) + tested.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 06:15:55 +03:00
claude code agent 227 770ba70541 fix(collab): retry transient store failures so autosave edits aren't lost (#206)
persist-1: onStoreDocument wrapped the page write in a try/catch that only
logged and swallowed the error, then resolved "successfully". hocuspocus
destroys/unloads the in-memory Y.Doc right after the hook resolves (the only
copy of the latest edit), so a transient DB error (deadlock, serialization
failure, dropped connection) silently lost the edit. Worse, the post-store
branch ran on the partially-assigned `page`, broadcasting a phantom
"page.updated" and enqueueing a history snapshot for content never written.

Wrap the write in a small bounded retry (3 attempts) so the save is
re-attempted while we still hold the doc, and clear `page` on failure so the
success-only side effects never report a save that didn't happen.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 06:05:28 +03:00
claude_code 683b9d5de2 fix(provenance): address #143 review — page-stamp tests, confine is_agent, doc fixes
Resolves the open items from the latest PR #143 code review:

- test(page): cover the four agentSourceFields stamp sites (create, update,
  movePage, movePageToSpace) with agent + normal-user payload assertions;
  add findById({ includeIsAgent: true }) wiring guards to the JWT and collab
  auth-seam specs so a future drop of the option is caught.
- fix(privacy): drop `isAgent` from UserRepo.baseFields and gate it behind a
  new opt-in `findById({ includeIsAgent })`, requested only by the two auth
  seams that derive provenance — stops the flag leaking via the workspace
  member list and generic user payloads.
- docs: correct the agentSourceFields JSDoc and the two UPDATE-site comments
  to distinguish INSERT (omitted column → DB default 'user') from UPDATE
  (omitted column → existing value kept, Kysely writes only present keys).
- style(page): collapse three stray double blank lines left by an earlier edit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 02:04:23 +03:00
claude code agent 227 7705d44fc6 fix(provenance): address #143 re-review — shared resolver + decoupled badge
Architecture & design:
- Arch A: introduce resolveProvenance() as the single source of truth for
  deriving a write's actor/aiChatId from the SIGNED identity, and wire it into
  BOTH transport seams — the REST jwt.strategy and the collab
  authentication.extension. Previously the collab seam derived actor from the
  token claim alone and ignored user.isAgent, so a flagged service account's
  page-content edits over the websocket persisted as lastUpdatedSource='user',
  drifting from REST. The seams now share one resolver and can't diverge.
- Arch B: drop AiAgentBadge's page-history coupling. The generic ui/ badge no
  longer imports historyAtoms; it exposes an onActivate callback fired after the
  deep-link, and the history row passes onActivate to close its own modal.

Suggestions/warnings:
- S1: soften the jwt.strategy provenance comment (applies to every REST write).
- S2/suggestion-3: drop the redundant comment-list-item null-aiChatId test
  (covered by ai-agent-badge.test.tsx).
- S3: de-duplicate jwt.strategy.spec test #3 (the no-claim→'user' half
  duplicated test #2); keep only the signed actor='agent' claim assertion.
- W2: add keyboard-activation tests for the badge (Enter/Space, unrelated key).
- W3: flip the design doc status to "реализовано (#143)".

Tests:
- new auth-provenance.decorator.spec.ts unit-tests resolveProvenance +
  agentSourceFields.
- new collab-seam test: is_agent user with no claim → actor='agent'
  (Arch A regression guard).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-24 00:27:03 +03:00
claude code agent 227 0647faefcd chore(comments): address PR #143 review (operator doc, badge test, dedup, type)
- [warn 1] Document the is_agent operator setup so it survives plan deletion:
  added an AI-agent block to .env.example (use a DEDICATED account, set is_agent
  via SQL, never flag a human/shared account) + a CHANGELOG "Added" entry.
- [warn 2] Test the badge deep-link side effects: ai-agent-badge.test.tsx now
  renders inside an explicit jotai store, clicks the badge, and asserts the
  active chat id, window-open, cleared draft, closed history modal, AND that
  stopPropagation keeps a parent onClick from firing.
- [suggestion 3] Hoist the window.matchMedia stub into vitest.setup.ts and drop
  the duplicated beforeAll block from the three test files (ai-agent-badge,
  comment-list-item, role-cards).
- [suggestion 4] Merge the two near-duplicate "non-clickable" cases via it.each.
- [follow-up 6] Introduce a single ProvenanceSource = 'user' | 'agent' type in
  jwt-payload.ts and reference it from AuthProvenanceData, JwtPayload/
  JwtCollabPayload, and resolveSource() — so a typo can't slip through as a bare
  string. (Server auth chain; client IComment mirroring left as a follow-up.)

Follow-up 5 (shared agentSourceFields write-stamp helper) is deferred as the
review marked it — the 6 REST sites use varied shapes (create-spread vs
resolve-conditional-null vs page move), so it's a separate focused refactor.

Tests: client badge/comment/role-cards suites 11/11 pass; server auth+comment
suites 62 pass; typecheck clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 23:56:26 +03:00
claude_code 3d4ad664b3 test(refactor-tail): extract pure cores + cover collab/share/ai-chat/client gate
Batches 6-9: behaviour-preserving extractions of testable pure cores plus the
tests they unblock, and a fix for the broken client test environment.
Full suites green: server 113 suites / 1117 + 1 todo, client 30 files / 338.

client (R0 infra):
- vitest.setup.ts: in-memory localStorage/sessionStorage Storage stub wired via
  setupFiles. Unblocks menu-items.gating.test.ts (was 9 failing) -> client suite
  fully green. + menu-items.suggestions.test.ts (getSuggestionItems filter/sort).

share:
- extract buildShareMetaHtml (share-seo.util.ts) from the SEO controller; tests
  for reflected-XSS escaping in <title>/og/twitter meta, noindex, truncation;
  extractPageSlugId; updateAttachmentAttr; prepareContentForShare comment-strip
  (anonymous-viewer metadata-leak guard).

ai-chat (security extractions):
- selectAccessibleHits: CASL post-filter for semantic search (restricted page in
  an accessible space must NOT leak to the agent).
- validateResolvedAddresses: SSRF connect-time guard (block if ANY resolved
  address is private).
- resolveAudioFormat: mime whitelist (dead `?? 'webm'` fallback dropped, set
  unchanged). + mcp-servers toView header-leak guard, MCP tool namespacing.

collaboration (data-loss area):
- extract computeHistoryJob (pins the "agent delay MUST stay 0" invariant) and
  resolveSource. Integration: onAuthenticate read-only matrix (collab auth
  bypass), HistoryProcessor (contributor restore on save failure), onStoreDocument
  Approach-A boundary snapshot (human revision pinned before agent overwrite).

Reviewed (APPROVE WITH SUGGESTIONS): extractions behaviour-preserving, security
tests mutation-resistant.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 19:10:27 +03:00
claude_code 81823fce1e feat(html-embed): sandbox the embed block; split trusted trackers into an admin field
Convert the htmlEmbed node from same-origin raw-HTML execution to a sandboxed
iframe (sandbox="allow-scripts allow-popups allow-forms", no allow-same-origin,
srcdoc) with postMessage auto-resize (validated by event.source) and an optional
manual height attr. The block now runs in an opaque origin and cannot reach the
viewer's cookies/session/API, so it is safe for any member.

Because the block is now harmless, remove the entire admin/role gating apparatus:
drop htmlEmbedAllowed/canAuthorHtmlEmbed/stripDisallowedHtmlEmbedNodes/
collectHtmlEmbedSources and every role-based strip on the write paths (collab
REST/MCP + socket, page create/duplicate, import x2, transclusion unsync), along
with the now-unused WorkspaceRepo/UserRepo injections and the PageService.create
callerRole param. Keep one strip: prepareContentForShare still removes htmlEmbed
on the anonymous public-share read path when the workspace master toggle is OFF.

The workspace settings.htmlEmbed toggle is now a plain feature switch (gates the
slash-menu and share rendering); when ON the block is available to all members.

Add settings.trackerHead: an admin-only raw HTML/JS analytics snippet injected
verbatim into the <head> of public share pages only (ShareSeoController), for
trackers that genuinely need same-origin. Admin-gated via the existing CASL
Manage/Settings ability; never injected into the authenticated app shell.

Closes security-review findings #1, #2, #4, #5, #10 (and #3 as a security issue).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 02:48:41 +03:00
claude_code 19ae6a0efa Merge pull request 'feat(editor): page templates — live whole-page embed (MVP)' (#17) from feat/page-templates into develop
Develop / build (push) Has been cancelled
2026-06-20 20:34:44 +03:00
claude code agent 227 8fcce6a674 feat(html-embed): per-workspace feature toggle, default OFF
The admin-only raw HTML/JS embed is a deliberate stored-XSS surface, so gate the
whole feature behind a workspace toggle that is OFF by default; it only works
when a workspace admin explicitly enables it.

- settings.htmlEmbed (boolean, default false) + workspace-update field htmlEmbed,
  persisted via WorkspaceRepo.updateSetting with an audit diff. Flipping it is
  admin-only (same Manage Settings CASL as other workspace toggles).
- New gate htmlEmbedAllowed(featureEnabled, role) = featureEnabled && admin/owner.
  All 7 server write paths (create, duplicate, collab onStoreDocument, REST/MCP/AI
  updatePageContent, single + zip import, transclusion unsync) now read the
  workspace's settings.htmlEmbed and strip unless (toggle ON AND admin). OFF
  (default, or a failed/empty workspace lookup) strips htmlEmbed for EVERYONE
  including admins -> existing embeds are cleaned up on next save, none persist.
- Client (defense-in-depth): the /html slash item is hidden unless toggle ON +
  admin; the NodeView executes nothing and shows a 'disabled in this workspace'
  placeholder when OFF; an admin Switch in Workspace Settings -> General with a
  description of the behavior.
- docs/html-embed-admin.md documents the toggle + admin-only + fail-closed
  coedit (a non-admin save strips an admin's embed) + execution semantics.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 19:28:39 +03:00
claude code agent 227 caac5c7f36 test(html-embed): exercise the REAL admin-gate write paths + import round-trip
Release-cycle test audit: the strip boundary was tested only via a stand-in
helper re-implemented in the spec, so a deleted/misplaced guard kept CI green
(the missing create() guard was proof). Replace it with tests against real code:
- persistence.extension.onStoreDocument: real ydoc from a rich doc (columns/
  table/mention/htmlEmbed) -> non-admin strip removes only htmlEmbed, every other
  node preserved (data-loss guard); admin keeps; empty fragment no-throw.
- collaboration.handler.updatePageContent: real path, user?.role gate, decoded
  ydoc embed-free for non-admin, kept for admin.
- transclusion unsync: member stripped, admin preserved.
- editor-ext gains a vitest setup (was zero tests) + a markdown round-trip:
  the <!--html-embed:BASE64--> marker -> htmlEmbed node with decoded source, and
  hasHtmlEmbedNode matches it — pinning the marked/turndown shape the import
  strip relies on. tsconfig now excludes specs from the shipped dist.
- Fail-closed identity: source-pinned contracts that the gate keys on
  fileTask.creatorId (zip) / request userId (single) / callerRole (create) /
  authUser.role (duplicate), and missing-user -> strip (services can't load under
  jest's ESM graph; helpers replay the exact predicate).
Adds the verified-safe ^src/ jest moduleNameMapper (identical fail set).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 14:52:29 +03:00
claude code agent 227 39ae89264d feat(editor): page templates - live whole-page embed (MVP)
Embed another page's LIVE content into a host page (it updates when the source
changes, not a static copy). A page can be flagged a template for discovery in
the picker; any accessible page can be embedded.

Server:
- migrations: pages.is_template (+ partial index) and page_template_references
  (whole-page back-refs); db.d.ts/entity types hand-merged (db.d.ts is curated).
- POST /pages/toggle-template (CASL Edit) flips is_template; is_template is
  returned by findById + the sidebar tree select so the tree menu label
  reflects state. Search suggestions gain an onlyTemplates filter for the picker.
- POST /pages/template/lookup ({sourcePageIds[]}, <=50): returns each accessible
  source's {title, icon, slugId, content, sourceUpdatedAt} with comment marks
  stripped (same access path as transclusion: filterViewerAccessiblePageIds;
  inaccessible -> no_access, missing -> not_found; error path -> not_found, never
  raw content).
- reference sync (collectPageEmbedsFromPmJson + syncPageTemplateReferences) on
  the Yjs save hook; duplicatePage remaps pageEmbed.sourcePageId + inserts refs.
  Known MVP gap: REST content updates don't resync refs (lookup uses in-doc ids).

Client:
- pageEmbed node (editor-ext, registered in BOTH client + server schemas);
  read-only NodeView with a batching lookup; '/Embed page' slash + template
  picker (self-embed prevented); 'Make/Unset template' in the tree node menu.
- Cycle guard: an ancestry-chain context + depth cap (5) render a 'circular
  embed' placeholder instead of recursing.
- Public shares show a placeholder (no public lookup in MVP).

MVP excludes (follow-ups): public-share lookup, unsync->static copy, server-side
expansion for export/RAG, MCP schema mirror, point-in-time snapshots.

Implements docs/page-templates-plan.md (MVP, variant A).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 10:05:00 +03:00
claude code agent 227 bd28dbfe2b feat(editor): admin-only raw HTML/CSS/JS embed node
Adds an htmlEmbed block node that renders and executes raw HTML/CSS/JS in the
wiki origin (e.g. an analytics tracker) — the owner-chosen variant C. Because
this is stored-XSS by design, only workspace admins/owners may get such a node
persisted; everyone executes it when reading.

- Node (editor-ext): htmlEmbed atom/isolating block; source stored base64 in
  data-source for lossless HTML<->JSON round-trip. renderHTML emits only the
  encoded marker (never inlines raw markup), so generateHTML/export/search are
  not themselves injection vectors. Registered in BOTH client extensions and
  server tiptapExtensions. Markdown round-trip via an <!--html-embed:b64-->
  comment (turndown) + a marked rule.
- Client NodeView: injects source and re-creates <script> elements so they
  actually run; edit modal; renders in read-only/share too. Slash item is
  admin-gated (adminOnly filtered by the user's workspace role).
- SERVER ENFORCEMENT (the real control — UI gating alone is insufficient):
  stripHtmlEmbedNodes() removes htmlEmbed from any document persisted by a
  non-admin, applied at every write path that introduces content from an
  untrusted author: collab onStoreDocument, REST/MCP/AI updatePageContent,
  single-file import, zip/multi-file import, page duplication, and transclusion
  unsync. Page restore introduces no new content. Public share/readonly viewers
  render fetched (already-stripped) content and do NOT open a collab socket, so
  the only residual is a transient broadcast window to concurrent authenticated
  editors (documented).

Implements docs/arbitrary-html-embed-plan.md (variant C).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 08:54:54 +03:00
vvzvlad 551f975886 fix(collab): use '-' instead of ':' in agent page-history jobId
BullMQ rejects custom job IDs containing ':' (Redis key separator),
throwing "Custom Id cannot contain :" inside the onStoreDocument hook
for every agent edit. This broke agent-driven page saves (MCP
create_page runs as actor='agent') with HTTP 400.

Switch the agent dedup suffix from `${page.id}:agent` to
`${page.id}-agent`. The jobId is only used as a BullMQ dedup key and is
never parsed by the history processor; page.id is a UUID, so the
hyphenated id cannot collide with a human job whose id is a bare page.id.
2026-06-17 17:38:32 +03:00
vvzvlad 0a9788e89a feat(collab): separate agent edits from human edits in page history
Page-history snapshots are debounced/coalesced (one per 1–5 min window,
jobId=page.id). A human edit followed by an agent edit in the same window
collapsed into a single snapshot, losing both the pre-agent human state and
a deterministic record of the agent's result.

Two provenance-aware boundaries now bracket an agent intervention:
- Before: on a user->agent transition, onStoreDocument synchronously pins the
  current (pre-agent) human content as its own history version tagged 'user',
  inside the page-write transaction, before the agent overwrites it.
- After: agent stores enqueue an immediate (delay 0), source-keyed history job
  (jobId=`${pageId}:agent`) so the agent's result snapshots deterministically
  as 'agent' and a later human edit (jobId=page.id) cannot coalesce/retag it.

Also add an `id desc` tie-break to findPageLastHistory so "last history" stays
deterministic when two snapshots share a created_at, consistent with
findPageHistoryByPageId.

Known trade-offs (Variant 1): the delay-0 worker re-reads the row, leaving a
millisecond mis-tag window; multiple agent edits in one turn may yield multiple
versions. The reverse agent->human boundary is intentionally out of scope.
2026-06-17 06:40:28 +03:00
vvzvlad 683da7a4c5 feat(ai-chat): per-user AI agent backend — LLM config, read-only agent, provenance schema
WIP checkpoint of the gitmost AI-chat backend (plan stages A + B1 + B3a).
The agent acts under the requesting user's JWT (Docmost CASL enforces page
access); the external service-account /mcp endpoint is untouched.

LLM provider config (A2-A4):
- integrations/crypto: AES-256-GCM SecretBoxService (key derived from APP_SECRET,
  per-record salt/iv; clear error on rotation instead of crashing).
- ai_provider_credentials table/repo/types: encrypted API key stored outside
  workspace settings/baseFields, write-only (never returned by any endpoint).
- integrations/ai: per-workspace AI SDK v6 provider driver (openai/gemini/ollama),
  admin-gated GET(masked)/PATCH(write-only key)/Test endpoints; settings.ai.provider
  holds non-secret config incl. systemPrompt. Removed unused AI_* env getters (DB is
  the single source of truth).

Chat module (A1, A5-A8):
- ai_chats/ai_chat_messages repos (workspace-scoped, soft-delete, tsv never selected).
- core/ai-chat: CRUD + POST /ai-chat/stream (Fastify hijack + AI SDK v6
  pipeUIMessageStreamToResponse, abort on disconnect, persist user/assistant msgs).
- Agent loop: streamText + stepCountIs(8); read tools searchPages/getPage via a
  per-request DocmostClient over loopback REST under the user's minted access token.
- Gate settings.ai.chat (+ 503 when provider unconfigured); buildSystemPrompt with a
  non-removable safety/anti-prompt-injection framework. Per-user rate limit.

Per-user auth (B1):
- @docmost/mcp DocmostClient gains an additive getToken variant (carry a user JWT,
  re-fetch on 401) and exports DocmostClient; the email/password service-account path
  (external /mcp, stdio) is unchanged.

Agent-edit provenance backbone (B3a):
- Migration: pages/page_history (last_updated_source, last_updated_ai_chat_id) and
  comments (created_source, ai_chat_id, resolved_source).
- Signed actor/aiChatId claim in the collab token; onAuthenticate propagates it,
  onStoreDocument writes it with a sticky agent marker, saveHistory copies it.

Migrations auto-run on boot (additive). Write tools, frontend, RAG and external MCP
servers are not in this checkpoint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 01:36:41 +03:00
Philip Okugbe 33895b0607 bug fixes (#2250)
* util

* fix page position collation

* support fixed toolbar in templates editor

* date localization

* fix clipped emoji in templates editor

* fix page updated time object

* fix flickers

* fix: remove redundant breadcrumb from destination modal
2026-05-28 16:20:37 +01:00
Philip Okugbe de60aa7e61 feat: synced blocks (transclusion) (#2163)
* feat: synced blocks (transclusion)

* fix:remove name

* make placeholders smaller

* feat: enforce strict transclusion schema

* fix: scope synced blocks to workspace, gate unsync on edit permission

* fix collab module error
2026-05-08 13:23:16 +01:00
Philip Okugbe 879aa2c3d8 feat: page update notifications (#2074)
* feat: watchers notification and email preferences

* fix: email copy

* digests

* clean up

* fix

* clean up

* move backlinks queue-up to history processor

* fix

* fix keys

* feat: group notifications

* filter

* adjust email digest window
2026-03-31 16:03:59 +01:00
Philipinho 721651e2e2 feat: user deactivation 2026-03-02 19:05:10 +00:00
Philip Okugbe 59e945562d feat(ee): page-level access/permissions (#1971)
* Add page_hierarchy table

* feat(ee): page-level permissions

* pagination

* rename migration
fixes

* fix

* tabs

* fix theme

* cleanup

* sync

* page permissions notification
* other fixes

* sharing disbled

* fix column nodes

* toggle error handling
2026-02-26 19:49:10 +00:00
Philip Okugbe 05b3c65b0f feat: notifications (#1947)
* feat: notifications
* feat: watchers

* improvements

* handle page move for watchers

* make watchers non-blocking

* more
2026-02-14 20:00:38 -08:00
Philip Okugbe 18b5781522 feat(API): page content update and retrieval (#1937)
* feat: page content update and retrieval output

* import module

* refactor naming
* support prepend

* rename contentOperation -> operation

* dry

* add yjs utils
2026-02-12 11:13:47 -08:00
Philip Okugbe 0f02261ee6 feat: page version history improvements (#1925)
* Refactor: use queue for page history

* feat: save multiple version contributors

* display contributor avatars in history list

* fix interval
2026-02-09 18:25:35 -08:00
Philip Okugbe 60501de992 fix: missing logs on OnApplicationBootstrap hook (#1882)
* - fix: set default Nest logger and bufferLogs to false for pino compatibility
- handle redis error event

* fix collab server logging too
2026-01-29 09:25:23 +00:00
Philip Okugbe 74e915546b feat: collab redis extension with server affinity (#1873)
* feat(collab): better redis extension
* move types to own file
* debug logging
* fix: graceful collab shutdown
* rename default prefix
* pass wsAdapter to gateway
* expose event handler
* unique collab serverId generation
* uninstall @hocuspocus/extension-redis package
* expose more functions
* sync with latest
* cleanup
* fastify router options
* cleanup type
2026-01-27 17:05:05 +00:00
Philip Okugbe 657fdf8cb7 feat: Tiptap V3 migration (#1854)
* Tiptap3 migration - WIP

* fix collaboration

* remove unused code

* fix flicker

* disable duplicate extensions

* update tiptap version

* Switch to useEditorState
- Set shouldRerenderOnTransaction to false

* fix editable state

* add tippyoptions for reference

* merge main

* tiptap 3.6.1

* fix bubble menu

* fix converter

* fix menus

* fix collaboration caret css

* fix: Set `isInitialized` to force immediate react node view rendering

* feat: Migrate tippy.js menus to Floating UI

* feat: Update collaboration connection for HocusPocus v3

* fix: Connect/disconnect websocketProvider

* cleanup

* cleanup

* feat: Improved placeholder and upload handling for images

* feat: Improved placeholder and upload handling for videos

* refactor: Image node and view clean-up

* feat: Improved placeholder and upload handling for attachments

* fix: Video view styles

* fix: Transaction handling on asset upload

* fix: Use imageDimensionsFromStream

* feat: Multiple file upload, improved placeholders, local previews

* fix: Drag & drop, paste upload

* fix: Allow media as attachment

* * add skeleton pulse animation
* add translation strings
* fix attachment view responsiveness

* fix collab connection status display

* Tiptap v3.17.0

* fix suggestion menu exit bug

* fix search shortcut

* fix history editor css

* tiptap 3.17.1

---------

Co-authored-by: Arek Nawo <areknawo@areknawo.com>
2026-01-24 20:41:08 +00:00
Philip Okugbe 9fb16bc842 feat(EE): AI vector search (#1691)
* WIP

* AI module - init

* WIP

* sync

* WIP

* refactor naming

* new columns

* sync

* sync

* fix search bug

* stream response

* WIP

* feat embeddings sync

* refine

* Add workspaceId to page events

* refine

* WIP

* add translation string

* sync

* reset ai answer on query change

* hide AI search in cloud

* capture streaming error

* sync
2025-12-01 11:50:25 +00:00
Philip Okugbe e51a93221c more checks for collab auth token (#1345) 2025-07-14 10:35:03 +01:00
Philip Okugbe 6d024fc3de feat: bulk page imports (#1219)
* refactor imports - WIP

* Add readstream

* WIP

* fix attachmentId render

* fix attachmentId render

* turndown video tag

* feat: add stream upload support and improve file handling

- Add stream upload functionality to storage drivers\n- Improve ZIP file extraction with better encoding handling\n- Fix attachment ID rendering issues\n- Add AWS S3 upload stream support\n- Update dependencies for better compatibility

* WIP

* notion formatter

* move embed parser to editor-ext package

* import embeds

* utility files

* cleanup

* Switch from happy-dom to cheerio
* Refine code

* WIP

* bug fixes and UI

* sync

* WIP

* sync

* keep import modal mounted

* Show modal during upload

* WIP

* WIP
2025-06-09 04:29:27 +01:00
Philip Okugbe e8847bd9cd fix: handle unhandled exceptions (#1116)
* Handle unhandled exceptions
* cleanup
2025-04-29 23:29:00 +01:00
Philip Okugbe 64f0531093 feat: keep track of page contributors (#959)
* WIP

* feat: store and retrieve page contributors
2025-04-04 13:03:57 +01:00
Philip Okugbe fd36076ae7 feat: disconnect collab websocket on idle tabs (#848)
* disconnect real-time collab if user is idle
* log yjs document disconnect and unload in dev mode
* no longer set editor to read-only mode on collab websocket disconnection
* treat delayed collab websocket "connecting" state as disconnected
* increase maxDebounce to 45 seconds
* add reset handle to useIdle hook
2025-03-08 18:16:23 +00:00
Philip Okugbe b81c9ee10c feat: cloud and ee (#805)
* stripe init
git submodules for enterprise modules

* * Cloud billing UI - WIP
* Proxy websockets in dev mode
* Separate workspace login and creation for cloud
* Other fixes

* feat: billing (cloud)

* * add domain service
* prepare links from workspace hostname

* WIP

* Add exchange token generation
* Validate JWT token type during verification

* domain service

* add SkipTransform decorator

* * updates (server)
* add new packages
* new sso migration file

* WIP

* Fix hostname generation

* WIP

* WIP

* Reduce input error font-size
* set max password length

* jwt package

* license page - WIP

* * License management UI
* Move license key store to db

* add reflector

* SSO enforcement

* * Add default plan
* Add usePlan hook

* * Fix auth container margin in mobile
* Redirect login and home to select page in cloud

* update .gitignore

* Default to yearly

* * Trial messaging
* Handle ended trials

* Don't set to readonly on collab disconnect (Cloud)

* Refine trial (UI)
* Fix bug caused by using jotai optics atom in AppHeader component

* configurable database maximum pool

* Close SSO form on save

* wip

* sync

* Only show sign-in in cloud

* exclude base api part from workspaceId check

* close db connection beforeApplicationShutdown

* Add health/live endpoint

* clear cookie on hostname change

* reset currentUser atom

* Change text

* return 401 if workspace does not match

* feat: show user workspace list in cloud login page

* sync

* Add home path

* Prefetch to speed up queries

* * Add robots.txt
* Disallow login and forgot password routes

* wildcard user-agent

* Fix space query cache

* fix

* fix

* use space uuid for recent pages

* prefetch billing plans

* enhance license page

* sync
2025-03-06 13:38:37 +00:00
Philip Okugbe e209aaa272 feat: internal page links and mentions (#604)
* Work on mentions

* fix: properly parse page slug

* fix editor suggestion bugs

* mentions must start with whitespace

* add icon to page mention render

* feat: backlinks - WIP

* UI - WIP

* permissions check
* use FTS for page suggestion

* cleanup

* WIP

* page title fallback

* feat: handle internal link paste

* link styling

* WIP

* Switch back to LIKE operator for search suggestion

* WIP
* scope to workspaceId
* still create link for pages not found

* select necessary columns

* cleanups
2025-02-14 15:36:44 +00:00
Philip Okugbe 990612793f refactor: switch to HttpOnly cookie (#660)
* Switch to httpOnly cookie
* create endpoint to retrieve temporary collaboration token

* cleanups
2025-01-22 22:11:11 +00:00
Philipinho 1f4bd129a8 editor improvements
* add callout, youtube embed, image, video, table, detail, math
* fix attachments module
* other fixes
2024-06-20 14:57:00 +01:00
Philipinho d4eefa48a8 restructure directories
* set log level based on env
2024-06-09 15:57:52 +01:00
Philipinho 2afbede8ce fix page history generation 2024-06-07 16:23:23 +01:00
Philipinho 886d9591fa frontend permissions
* rework backend workspace permissions
2024-06-03 02:54:12 +01:00
Philipinho 9c7c2f1163 updates and fixes
* seo friendly urls
* custom client serve-static module
* database fixes
* fix recent pages
* other fixes
2024-05-18 03:19:42 +01:00
Philipinho a2768e7d30 collaboration module cleanup x2 2024-05-04 17:21:44 +01:00
Philipinho 6a57e3edb2 Make colloboration module not dependant on page module
* add unloadImmediately to collab
2024-04-22 23:40:50 +01:00
Philipinho 28ec542ed6 server: page permissions 2024-04-22 02:25:03 +01:00
Philipinho c18c9ae02b Refactoring
* replace TypeORM with Kysely query builder
* refactor migrations
* other changes and fixes
2024-03-29 01:46:11 +00:00
Philipinho cdcb4e87d0 * Make JwtGuard better
* Fix auth decorators
2024-02-28 03:07:24 +00:00
Philipinho a0ec2f30ca feat: search 2024-01-30 00:14:21 +01:00
Philipinho 9a8b605f70 * add new tiptap editor extension monorepo package
* move tiptap packages to main package.json
* add tiptap extensions schema to collaborative backend
* add basic README
2024-01-14 23:05:41 +01:00