Compare commits

...

40 Commits

Author SHA1 Message Date
agent_coder 98a58f6296 fix(#346 review F1-F4): no 206-compress + Vary + precompress VAD + cache test
- F1 [HIGH — data corruption]: @fastify/compress was compressing 206/Range
  attachment responses while Content-Range still described the RAW offsets, so a
  resuming client (curl -C -, download managers) appended encoded bytes as raw →
  corrupted file. sendFileResponse now sets the request header `x-no-compression`
  (the documented @fastify/compress opt-out — its onSend skips when the request
  carries it; the reviewer's `Content-Encoding: identity` does NOT work because
  compress explicitly excludes `identity` and overwrites it). This opts the whole
  download route (both 200 full-file and 206 range) out of on-the-fly compression
  — correct, since attachment bytes are final and mostly binary.
- F2: static responses now emit `Vary: Accept-Encoding` (the preCompressed
  content-negotiated /assets/* were `immutable` without Vary → shared-cache could
  serve a brotli variant to an identity/gzip-only client).
- F3: vite compression `include` extended to .wasm/.onnx so the VAD binaries
  (~26MB .wasm, ~2.3MB .onnx under public/vad) are precompressed at build (.br
  emitted) instead of runtime-brotli'd on every request. (include REPLACES the
  plugin default, so the default js/css/json/html set is re-listed.)
- F4: extracted the cache classification into a pure `resolveStaticAssetHeaders`
  + static.module.spec.ts (3 tests: /assets/* immutable+Vary, index.html
  no-store, non-hashed not-immutable).

Gate: server tsc 0 (deps present), static.module.spec 3/3, client build emits
.wasm.br/.onnx.br, frozen install 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 23:33:59 +03:00
agent_coder 26b29e1d7e perf(delivery): pre-compress static + cache headers + compress API responses (#346)
Cold load served ALL static + API responses uncompressed and without cache
headers (~3.7MB over the wire). Delivery only — feature behavior unchanged; no
DB/API-contract/MCP changes.

- apps/client/vite.config.ts: vite-plugin-compression2 emits .br + .gz next to
  each built asset (excludes index.html, which the server rewrites at boot with
  window.CONFIG — a precompressed copy would go stale). Build emits 187 .br /
  175 .gz under dist/assets.
- static.module.ts: @fastify/static `preCompressed: true` serves the .br/.gz
  neighbour; `setHeaders` sets `immutable` ONLY for content-hashed /assets/*,
  `no-cache` for index.html, and leaves non-hashed files (locales, vad, icons,
  manifest) on default etag/last-modified revalidation.
- main.ts: @fastify/compress (threshold 1024) compresses dynamic API JSON + the
  rewritten share-SEO HTML. SSE is safe on two counts: `text/event-stream` is not
  mime-db-compressible (allowlist skips it) AND the AI-chat stream hijacks the raw
  socket (pipeUIMessageStreamToResponse -> res.raw), bypassing the Fastify onSend
  lifecycle entirely. No double-compression with preCompressed static (compress
  skips already-Content-Encoding'd responses).
- docker-compose.yml: comment recommending an optional HTTP/2 + brotli reverse
  proxy (not required).

Deps: apps/client vite-plugin-compression2 2.5.3 (dev), apps/server
@fastify/compress 9.0.0 (matches fastify 5.8.5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 21:29:54 +03:00
vvzvlad d78b985062 Merge pull request 'perf(comment): статический рендер + ленивые редакторы + мемоизация панели (#340)' (#349) from fix/340-comment-panel-perf into develop
Reviewed-on: #349
2026-07-04 20:55:11 +03:00
agent_coder a4fc6c7f64 fix(comment): underline mark + draft-surviving tabs + test coverage (#349 review F1-F4)
- F1: render the `underline` mark statically (StarterKit v3 enables Underline;
  comment-editor does not disable it) — an underlined comment no longer degrades
  the whole comment to the read-only editor fallback. renderMarks gains a
  `case "underline" -> <u>`, mirroring the other marks (+ test).
- F2: keep the Open tab panel mounted (`Tabs.Panel value="open" keepMounted`)
  while the heavy Resolved panel still unmounts (`Tabs keepMounted={false}`). A
  per-panel keepMounted overrides the parent's `false` (Mantine 8 TabsPanel), so
  an in-progress reply draft / edit in the Open panel survives an
  Open->Resolved->Open switch, keeping the micro-opt of not mounting the large
  Resolved list.
- F3: cover edit->save->re-render in comment-list-item.test.tsx — save calls
  mutateAsync with JSON.stringify(editContentRef) and a new comment.content prop
  updates the visible body; cancel restores the static body without mutating;
  clearing editContentRef after cancel.
- F4: extract childrenByParent grouping into an exported pure
  `buildChildrenByParent(items)` (unit-tested: nesting, orphan reply, sibling
  order) + new comment-list-with-tabs.test.tsx covering the lazy reply-editor
  activation (stub -> click/focus/Enter mounts the editor).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 20:48:21 +03:00
vvzvlad c252068672 Merge pull request 'feat(ai-chat): отложенная загрузка инструментов (deferred tools + loadTools) (#332)' (#341) from fix/332-deferred-tools into develop
Reviewed-on: #341
2026-07-04 20:47:45 +03:00
agent_coder 68caf8157a test(ai-chat): document AI_CHAT_DEFERRED_TOOLS + pin ON-path & catalog completeness (#341 review F1-F3)
- F1: document AI_CHAT_DEFERRED_TOOLS in .env.example (AI_* section) — default
  ON = deferred loading (compact catalog + loadTools), =false restores the old
  "all tools always active" behavior.
- F2: integration test of the ON path in ai-chat-stream.int-spec.ts — a deferred
  tool activated via loadTools is active on the SAME turn's next step but a fresh
  turn starts cold (CORE + loadTools only), proving the per-turn activatedTools
  Set does not leak across turns/chats. Drives the real streamText loop with a
  MockLanguageModelV3 and inspects recorded per-step activeTools-filtered tools.
- F3: replace the magic toHaveLength(28) in tool-tiers.spec.ts with a two-way
  partition against the LIVE in-app toolset (AiChatToolsService.forUser keys):
  every non-core tool must appear in buildInAppDeferredCatalog and every catalog
  entry must map to a real non-core tool — so a future tool forgotten in
  INLINE_TOOL_TIERS fails the suite instead of silently vanishing from the agent.

No production logic change (mechanism was already reviewed correct).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 20:34:42 +03:00
agent_coder cb9c5dda59 perf(comment): static comment renderer + lazy editors + memoized list (#340)
The comment panel lagged for seconds on open and stuttered on every resolve/apply
with many comments (real case: 30 open + 326 resolved ≈ 356 threads), because each
comment body mounted a full TipTap/ProseMirror editor, both tabs mounted at once,
and any mutation re-rendered the whole list.

- CommentContentView: static recursive renderer of comment ProseMirror JSON (no
  editor instance) for the read-only body — supports exactly CommentEditor's node
  set (doc/paragraph/text/hardBreak/mention) + marks (bold/italic/strike/code/
  link), reproducing the 3-level DOM nesting for pixel-identical CSS. Unknown
  node/mark or unparseable content degrades that one comment to the read-only
  CommentEditor; legacy non-JSON strings render as plain text.
  SECURITY: link hrefs are protocol-allowlisted (safeHref, mirroring
  @tiptap/extension-link) so a stored comment with a `javascript:`/`data:` href
  cannot XSS — the old TipTap read-only path sanitized this; the static renderer
  must too. Control-char smuggling (java\tscript:) is stripped before the check.
- MentionContent extracted from MentionView, shared by the TipTap NodeView and the
  static renderer (identical user/page-mention behavior).
- keepMounted={false} on the tabs: the inactive tab no longer mounts its editors.
- Lazy reply editor: a stub until click/focus, then the real editor (kept mounted
  so the draft survives thread re-renders).
- React.memo(CommentListItem) + a childrenByParent map (replaces the per-thread
  O(n^2) filter) + localized reply-send pending state: resolve/apply/reply now
  re-render only the touched thread.
- Progressive first paint: useCommentsQuery no longer blocks on hasNextPage.

Gate: client comment+mention suites 22/22 passed, tsc --noEmit 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 20:20:32 +03:00
claude code agent 227 e431b33bb1 feat(ai-chat): deferred tool loading (tiers + loadTools meta-tool) (#332)
The in-app AI agent shipped all ~41 tool schemas on every model step. This
adds a two-tier catalog: core tools (frequent or one-line) stay always-active;
the rest are advertised as a compact catalog and their full schema is fetched
on demand via the loadTools meta-tool, wired through ai@6 prepareStep's
per-step activeTools.

- tools/tool-tiers.ts: CORE_TOOL_KEYS, INLINE_TOOL_TIERS, applyLoadTools,
  catalog builders (+ tool-tiers.spec.ts, 13 cases).
- ai-chat.service.ts prepareAgentStep: returns activeTools =
  [...CORE_TOOL_KEYS, loadTools, ...activatedTools]; per-turn activated Set.
- ai-chat.prompt.ts: buildToolCatalogBlock renders the deferred catalog.
- mcp/tool-specs.ts: tier + catalogLine metadata (external snake_case /mcp
  transport unchanged).
- EnvironmentService.isAiChatDeferredToolsEnabled(): AI_CHAT_DEFERRED_TOOLS,
  default ON per issue intent (kill-switch =false restores old behavior).

Gate: server ai-chat 631/631, tool-tiers 13/13, mcp 472/472, tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 19:57:11 +03:00
vvzvlad 4369bbc53d Merge pull request 'refactor(converter): единый пакет @docmost/prosemirror-markdown + канон форматов, git-sync и mcp переключены (#293, шаги 2–5)' (#333) from feat/293-B-prosemirror-markdown-pkg into develop
Reviewed-on: #333
2026-07-04 19:35:53 +03:00
claude code agent 227 8e5ad8070b fix(lock): repair pnpm-lock.yaml broken by develop merge (#333 F-lock)
The develop merge (eacc1c48) left an importer pointing at a vitest@4.1.6
peer-variant WITHOUT @vitest/coverage-v8 that has no snapshot entry, so
`pnpm install --frozen-lockfile` failed with ERR_PNPM_LOCKFILE_MISSING_DEPENDENCY
(CI + Docker red at install). Regenerated with `pnpm install --lockfile-only
--fix-lockfile` (pnpm 10.4.0, matches packageManager pin): the importer now
resolves to the existing coverage-v8 variant; two transitive pointers realigned.
No package.json / source change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 19:27:09 +03:00
vvzvlad cfc105c7d6 Merge pull request 'feat(comment): эфемерные предложения-правки — Apply/Dismiss убирают комментарий (#329)' (#338) from fix/329-ephemeral-suggestions into develop
Reviewed-on: #338
2026-07-04 19:22:14 +03:00
claude code agent 227 d7fa6738e5 fix(comment): transactional childless-delete race fix + client dismiss gate + DB int-spec (#329 review round 2)
F4 [critical] — the anti-join `DELETE … WHERE NOT EXISTS(child)` was still racy
under Postgres READ COMMITTED: a reply INSERT holds FOR KEY SHARE on the parent;
the DELETE's start snapshot doesn't see the uncommitted child (NOT EXISTS true),
blocks on the reply's lock, and when the reply commits the parent was only LOCKED
(not modified) so EvalPlanQual does NOT re-check → the DELETE proceeds and CASCADE
destroys the just-committed reply. Replaced with a transaction: SELECT the parent
FOR UPDATE (conflicts with the reply's FOR KEY SHARE → serializes the concurrent
reply), re-check for a child with a FRESH statement in the same tx (a new RC
snapshot sees a just-committed reply), delete only if still childless (return 1)
else return 0 (caller resolves). The FOR UPDATE lock is held to end-of-tx so no
reply can insert between the re-check and the delete. Signature unchanged, so the
service + its mocked unit tests are untouched; docstrings updated.

F5 [warning] — the client Dismiss button was gated only on canComment, but the
server now gates dismiss on owner-or-space-admin, so a non-owner non-admin saw a
button the server 403s. `canShowDismiss` now also requires
`isOwnerOrAdmin = currentUser?.user?.id === comment.creatorId || userSpaceRole ===
"admin"` (the same gate the comment delete-menu already uses); threaded into both
call sites.

F6 [warning] — added a REAL-DB int-spec
(apps/server/test/integration/comment-delete-if-childless.int-spec.ts, + a
createComment seeder): (a) childless → returns 1, row gone; (b) committed reply →
returns 0, parent+reply survive; (c) CONCURRENCY — a second connection inserts a
reply (FOR KEY SHARE) and commits mid-operation while deleteCommentIfChildless
blocks on FOR UPDATE → asserts it returns 0 and both rows survive (a blind
anti-join would lose the reply here). Ran against live Postgres — 3/3 pass.

server tsc clean; comment jest 53 + int-spec 3 (live Postgres) pass. client tsc
clean; comment vitest 56 pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 19:17:52 +03:00
claude code agent 227 e6d8eda8e5 fix(comment): dismiss owner/admin authz + atomic conditional delete + 404-only onError (#329 review)
Maintainer escalation decision (B) + reviewer findings on the ephemeral-
suggestion PR.

Authz (decision B): POST /comments/dismiss-suggestion now gates the destructive
branch on owner-OR-space-admin, mirroring POST /comments/delete exactly (same
SpaceCaslAction.Manage / SpaceCaslSubject.Settings, same owner short-circuit,
same ForbiddenException). A non-owner non-admin who tries to dismiss another's
childless suggestion gets Forbidden before the service runs. Apply stays on
canEdit (accepting an edit is the editor's semantics), unchanged.

F1 [blocking] — atomic conditional delete closes the hasChildren→delete race.
New repo `deleteCommentIfChildless(id)` runs a single
`DELETE FROM comments WHERE id=:id AND NOT EXISTS (SELECT 1 FROM comments child
WHERE child.parent_comment_id = comments.id)` (verified by compiling the Kysely
expression to SQL — the correlated subquery references the OUTER comments.id).
deleteEphemeralSuggestion strips the mark first, then the conditional delete: if
it removed the row → commentDeleted + outcome 'deleted'; if a reply raced in
(0 rows) → fall back to resolveComment (outcome 'resolved') so the discussion and
the new reply survive. No reply can be cascade-deleted anymore.

F2 [warning] — the apply/dismiss onError success-noop is narrowed from 404||400
to 404 ONLY. A 400 means the comment is ALIVE (apply's 400 = the thread was
resolved-not-applied), so it now shows a real error (surfacing the server
message) and KEEPS the comment in cache instead of a false "applied" + dropping a
live thread.

F3 [suggestion] — the 404-race client tests assert the success toast fired.

Tests: server — dismiss authz (owner ok / non-owner-non-admin Forbidden /
space-admin ok), the delete→resolve race (hasChildren=false but conditional
delete returns 0 → resolve, no commentDeleted), delete-path asserts switched to
deleteCommentIfChildless; client — apply-400 and dismiss-400 (kept in cache, red,
not success) + the toast assertions.

server tsc clean, comment+collaboration jest green; client tsc clean, comment
vitest 54 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 19:17:19 +03:00
claude code agent 227 8d8ecaed82 feat(comment): ephemeral suggestion-edits — Apply/Dismiss remove the comment (#329)
Agent suggestion-edits (comments with suggestedText, #315) piled up: Apply
auto-resolved the thread, cluttering the resolved tab, and the anchors stayed in
the document. Make them ephemeral: resolving (Apply OR the new Dismiss) makes the
comment DISAPPEAR — hard-delete + remove the Yjs `comment` mark — UNLESS the
thread has replies, in which case resolve it (preserve the discussion). Manual
Resolve is unchanged. Scope: only comments with `suggestedText`.

Server:
- New collab event `deleteCommentMark` (collaboration.handler) mirroring
  resolveCommentMark, wiring the existing removeYjsMarkByAttribute to strip the
  anchor from the doc.
- `finalizeAppliedSuggestion` forks on `hasChildren`: replies → apply + resolve
  (outcome 'resolved'); none → apply + hard-delete + mark removal (outcome
  'deleted').
- New `dismissSuggestion` (validates top-level + suggestedText + not applied/not
  resolved) with the same fork; permission `canComment` (NOT canEdit — dismiss
  doesn't change page text); audit COMMENT_SUGGESTION_DISMISSED. New
  POST /comments/dismiss-suggestion; apply stays canEdit.
- Both return `{ outcome: 'deleted' | 'resolved' }` so the client picks the
  optimistic action.

Data-integrity (review F1): the shared `deleteEphemeralSuggestion` removes the
anchor mark FIRST and FATALLY, then deletes the DB row only on success. The row
delete is irreversible, so a mark-removal failure — including the
COLLAB_DISABLE_REDIS "no live instance" hard-error — must abort the whole
operation (→ 5xx, repeatable) rather than swallow the error and leave a permanent
orphan anchor pointing at a deleted comment. `deleteCommentMark` is no longer
best-effort (unlike resolve, where the row is kept and a failed mark is
recoverable).

Client:
- `canShowDismiss` (canComment) alongside `canShowApply` (canEdit); a "Dismiss"
  button next to Apply in the suggestion block.
- `useApplySuggestionMutation`/`useDismissSuggestionMutation` reconcile the cache
  on `outcome` ('deleted' → remove; 'resolved' → relocate to the resolved tab).
- Idempotent races (review F2): BOTH apply and dismiss onError reduce 404/400 to
  success (comment already gone/resolved), dropping it from the cache instead of
  a red error — restores the #315 apply idempotency the ephemeral delete would
  otherwise break.
- i18n Dismiss / "Не применять" (ru/en).

Not done (flagged): deleteCommentMark on the normal /comments/delete path — left
out (would change every non-suggestion delete + needs gateway injection; the
interactive client already strips the mark via unsetComment). Out of scope per
the issue.

Tests: server — apply/dismiss delete-vs-resolve fork, all four dismiss state
guards, the deleteCommentMark handler, controller authz (dismiss=canComment,
apply=canEdit), AND a mark-removal-failure test proving the row is NOT deleted +
the error propagates (F1). client — Dismiss show-conditions, outcome cache
reconciliation, and 404 idempotent race for BOTH dismiss and apply (F2).

Verified: server tsc clean; comment+collaboration jest 144 passed. client tsc
clean; vitest 905 passed | 1 expected-fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 19:17:19 +03:00
claude code agent 227 eacc1c4811 Merge branch 'develop' of https://gitea.vvzvlad.xyz/vvzvlad/gitmost into feat/293-B-prosemirror-markdown-pkg
# Conflicts:
#	packages/mcp/build/client.js
#	packages/mcp/build/index.js
#	packages/mcp/build/tool-specs.js
2026-07-04 19:02:52 +03:00
claude code agent 227 8e12aa8ebf fix(build): ship + CI-build @docmost/prosemirror-markdown; refresh AGENTS.md (#333 conformance)
Two infra blockers from the #326-steps-2-5 conformance check — the converter/canon
are correct, but the new shared package wasn't wired into Docker/CI.

BLOCKER 1 (prod): the Docker installer stage copied mcp/build + editor-ext but NOT
packages/prosemirror-markdown. mcp now depends on it (workspace:*) and EAGER-imports
it at runtime — the in-app ai-chat DocmostClient loads build/index.js ->
lib/markdown-converter.js — so the shipped image would resolve a broken workspace
symlink and every ai-chat tool would die with ERR_MODULE_NOT_FOUND. Now the
installer COPYs packages/prosemirror-markdown/build + package.json before the prod
install. (git-sync has no runtime consumer yet — revisit at step 6 with #119.)

BLOCKER 2 (CI red): test.yml/develop.yml build only @docmost/editor-ext before
`pnpm -r test`. That is plain pnpm, which does NOT honour nx `dependsOn: ^build`,
so the package's (gitignored) build/ never appears and its consumers fail:
mcp `pretest: tsc` -> TS2307 Cannot find module '@docmost/prosemirror-markdown',
git-sync vitest typecheck the same. The green local runs only happened because the
coder+reviewer had a full install+build. Added a `pnpm --filter
@docmost/prosemirror-markdown build` step before `pnpm -r test` (mirrors the
editor-ext step); verified the build is clean (tsc exit 0).

Docs (remark 3): AGENTS.md:203 and :285 still told contributors to keep mcp's own
vendored schema mirror "in sync manually" — that copy was deleted by this PR.
Updated both: the converter + schema mirror now live in the SINGLE package
@docmost/prosemirror-markdown (consumed by mcp + git-sync, do NOT reintroduce a
per-package copy); editor-ext is the upstream schema source; the serializer-contract
test guards the boundary. Added the package to the workspace table.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 18:46:33 +03:00
vvzvlad 348dcd0802 Merge pull request 'feat(mcp): search_in_page — внутристраничный поиск для агента (#330)' (#339) from fix/330-search-in-page into develop
Reviewed-on: #339
2026-07-04 18:43:40 +03:00
vvzvlad 77c64c4fd9 Merge pull request 'test(infra): coverage-gate + acceptInvitation atomicity + turn-end unit (#324)' (#335) from fix/324-coverage-gate into develop
Reviewed-on: #335
2026-07-04 17:45:46 +03:00
vvzvlad 2bb71c1a45 Merge pull request 'fix(client): мобильный адаптив 390px — Create page открывает редактор; ревизия хвоста #291 (#325)' (#334) from fix/325-mobile-390 into develop
Reviewed-on: #334
2026-07-04 17:45:35 +03:00
vvzvlad 20248b8c95 Merge pull request 'feat(client): intraline diff в блоке предложения-правки (#331)' (#336) from fix/331-intraline-diff into develop
Reviewed-on: #336
2026-07-04 17:45:10 +03:00
vvzvlad 9274c51053 Merge pull request 'feat(mcp): скрыть resolved-комментарии (якоря + list_comments) от агента (#328)' (#337) from fix/328-resolved-anchor-spam into feat/293-B-prosemirror-markdown-pkg
Reviewed-on: #337
2026-07-04 17:44:55 +03:00
claude code agent 227 832c3cafdf test(mcp): update test-e2e.mjs listComments calls to the {items} shape (#328 review F1)
The listComments Comment[] -> { items, resolvedThreadsHidden } shape change
reached every src/host consumer but not the live-server e2e harness (run via
`node test-e2e.mjs`, not the node --test gate — so the green suite missed it).
The 4 calls now read .items; the post-resolve check passes includeResolved:true
so it still sees the now-resolved root c1 (the default feed hides it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 16:22:34 +03:00
claude code agent 227 94f60cf0ec docs(client): fix .suggestionChanged comment — bold weight, not underline (#331 review F1)
The header comment claimed the rule adds 'an underline'; it does not — it adds a
color-mix tint + font-weight:700, and the inner comment already notes text-
decoration is omitted on purpose. Aligned the header comment with the rule.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 16:21:25 +03:00
claude code agent 227 bcd194ee5d feat(mcp): hide resolved-comment anchors + feed from the agent (#328)
The AI agent (MCP + in-app chat) saw ALL comments incl. resolved via two
channels, cluttering its context and breaking fragment search. Default now:
the agent sees only ACTIVE discussions; resolved is opt-in. Active anchors and
threads are always kept.

Channel 1 — resolved comment anchors on agent reads (converter option):
`convertProseMirrorToMarkdown(content, options?)` gains
`options.dropResolvedCommentAnchors` (default false — zero change for every
existing caller incl. git-sync). Both `case "comment"` emitters (top-level and
the raw-HTML inlineToHtml path) emit BARE text (no `<span data-comment-id>`) when
`resolved && the flag`; active anchors keep their wrapper. mcp `getPage` passes
the flag; `export_page_markdown` does NOT (lossless export must preserve resolved
anchors — that is why it is an opt-in option, not unconditional); `get_page_json`
is untouched (lossless PM JSON). Built on the #293 package converter.

Channel 2 — `list_comments` default active-only: `listComments(pageId,
includeResolved=false)` now returns `{ items, resolvedThreadsHidden }` (was a
bare array). By default a RESOLVED top-level thread is hidden wholesale — the
root AND every reply anchored to it (a thread is gated only by its root's
resolvedAt; a resolved reply under an ACTIVE root stays). `resolvedThreadsHidden`
counts hidden threads so the agent knows to re-query. `includeResolved:true`
returns everything. The `includeResolved` param is added to both tool
registrations (MCP index.ts + in-app ai-chat-tools.service.ts); `DocmostClientLike`
signature updated. Server `findPageComments` is NOT touched — the web UI's tabs
depend on the full feed; filtering is only at the mcp-client level. All internal
call sites (export_page_markdown / checkNewComments / transformPage) updated to
`.items` with `includeResolved:true` to keep their full-feed behavior.

The comment model is assumed FLAT (a reply's parentCommentId points at the
thread root) — documented in the filter; a future reply-of-reply model would
need a root-walk there.

Tests: resolved-comment-anchors.test.ts (6 — anchor dropped with flag / kept
without, for BOTH emitters; active always kept); list-comments-resolved.test.mjs
(4 — resolved thread+reply hidden + counter; includeResolved:true returns all;
an ACTIVE thread with a RESOLVED reply is NOT hidden).

package vitest: 664 passed; tsc clean. mcp: node --test 458 passed; tsc clean.
apps/server + git-sync: tsc clean (converter option default-off).

NOTE: based on feat/293-B (#293/#326 STEP 5) — the converter lives in the
package; this PR is stacked on #333 and its base retargets to develop once #333
merges. mcp/build is gitignored (not committed).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 15:26:43 +03:00
claude code agent 227 f13105333a feat(client): intraline diff highlighting in the suggestion before→after block (#331)
The suggestion block (#315) struck the whole `selection` red and showed the whole
`suggestedText` green, so a one-letter edit (заведем→заведём) highlighted the
entire line. Now only the CHANGED fragments are emphasized intraline, git-style.

Pure, render-only — nothing changes in the DB/backend/MCP/IComment/mutations/
Apply/Badge. New pure `computeSuggestionDiff(old, new) => { old: Segment[], new:
Segment[] }` (Segment = {text, changed}) in suggestion.ts: hybrid word+char —
`diffWordsWithSpace` for the word skeleton, then `diffChars` inside an adjacent
removed+added pair so only the differing letters (not the whole word) are
flagged; a lone insertion/deletion is wholly changed; equal parts are common on
both sides. Concatenating each side reproduces the input (lossless). Wrapped in
`useMemo` on [selection, suggestedText].

comment-list-item.tsx renders per-segment spans instead of two whole <Text>;
changed segments get `.suggestionChanged` (a stronger currentColor tint + bold,
NO text-decoration so the old block's inherited line-through survives on the
changed letters — the whole old line still reads removed, new as added).

`diff@8.0.3` (jsdiff, already in the root package.json) added to
apps/client/package.json (+ lockfile, additive) so the workspace resolves it;
it bundles its own types.

Tests: new suggestion.test.ts (one-letter ё/е; word replacement keeping the
shared word common with no per-letter noise; word insertion/deletion; identical)
— asserts segment text + changed flags, non-vacuous. Two pre-existing
comment-list-item.test assertions switched from getByText (a single text node)
to container.textContent (the new line is now multiple spans) — adapts to the
intended DOM change, not a weakening.

Verified: tsc --noEmit clean; client vitest 892 passed | 1 expected-fail.
Visual/pixel check of the tint at the 390px comment panel needs a human (no
screenshot tooling in-repo).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 15:17:07 +03:00
claude code agent 227 08222345ef fix(prosemirror-markdown): escape canon inline-extension triggers = $ ^ in link/alt text (#333 review F5)
F1 (round 1) wrapped the image alt in escapeLinkText, and that helper also guards
the link-form media captions (attachment/pdf/embed). But its character class
covered only stock CommonMark — NOT the Docmost inline EXTENSIONS this same PR
registers on the marked instance: highlight `==x==` (canon #7), math `$x$`
(canon #6), footnote `^[x]` (canon #2). Their triggers `= $ ^` are not CommonMark
punctuation, so an alt or media filename like `x $A$ y`, `use ==bold==`, `^[fn]`,
or `data $A$.csv` was silently turned into a math/highlight/footnote node on
import — the same class of round-trip data loss F1 closed, reintroduced by this
PR's own canon.

Fix: add `= $ ^` to the escapeLinkText class (`/[\\`*_~[\]<&!()=$^]/g`). `\= \$ \^`
decode back to literals (all ASCII punctuation) AND, being escape tokens, stop
the extension tokenizer from matching — verified lossless byte-stable round-trip.
Updated the helper comment to name the two trigger sets (CommonMark + Docmost
inline extensions). Extended the adversarial round-trip tests: image alt gains
`x $A$ y` / `5$ and 10$` / `use ==bold==` / `^[fn]` / `cost $5 == price`; pdf name
gains `data $A$.csv` / `q3 ==final==.pdf` / `5$ and 10$.pdf` / `note ^[x].pdf` —
all byte-stable with the node intact, so the hole can't reopen.

package vitest: 658 passed; tsc clean. git-sync: 268. mcp: 454.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 12:46:30 +03:00
claude code agent 227 baa41d66ad test(infra): coverage-gate + acceptInvitation atomicity int-spec + turn-end unit (#324)
Tail of #244. Three items:

1. Coverage-gate (main). develop had no coverage tooling at all. Added
   @vitest/coverage-v8@4.1.6 (pinned to the vitest already in use) to the three
   vitest packages — git-sync, editor-ext (which also gains its missing direct
   `vitest` devDep), apps/client — and enabled v8 coverage with per-package
   thresholds (no root vitest config exists, so per-package is the only
   meaningful scope). v8 provider is chosen deliberately: istanbul broke on the
   ESM `@docmost/editor-ext` barrel; v8 collects native runtime coverage and
   never re-parses ESM. `enabled: true` wires the gate into the plain `test`
   script, so `pnpm -r test` (the CI entrypoint) enforces it without a manual
   `--coverage`. Thresholds set ~4-5 pts below measured current coverage so the
   gate PASSES today and FAILS on regression (verified: forcing lines=95 on
   editor-ext exits 1). `all: false` — coverage counts test-touched files;
   documented in the configs (with `all: true` the many untested type/barrel
   files would sink the % and make the gate meaningless).
   Measured→threshold (S/B/F/L): git-sync 91.78/79.16/76.76/92.46 → 88/75/72/88;
   editor-ext 58.58/48.1/64.96/58.91 → 54/44/60/54; client 59.93/58/48.47/59.39
   → 55/53/44/55. All exit 0.

2. acceptInvitation atomicity int-spec. New
   apps/server/test/integration/workspace-accept-invitation-atomicity.int-spec.ts
   (+ createDefaultGroup/createInvitation seeders in test/integration/db.ts per
   its convention). Wires the real WorkspaceInvitationService with real
   User/Group/GroupUser repos against the test Kysely, stubbing only the
   post-commit collaborators. Asserts the invariant protected by
   users_email_workspace_id_unique: (a) two CONCURRENT accepts → exactly one
   fulfilled, one BadRequestException('Invitation already accepted'), membership
   count == 1, invitation consumed; (b) repeated sequential accept → still one
   membership; (c) the survivor is in the workspace default group (whole-tx, no
   torn state). Ran against real Postgres+Redis: 3/3 pass.

3. turn-end decision unit test. `decideTurnEnd` does not exist as a symbol; the
   turn-end logic lives in chat-thread.tsx's onFinish handler. Added a focused
   block to the existing chat-thread.test.tsx (matching its hoisted-mock style):
   clean finish → flush queued (continue); abort/disconnect/error → queue
   preserved (end) with the correct notice; parent notified on every terminal
   outcome. 8 passed (3 existing + 5 new).

Verified: git-sync 712, editor-ext 247, client 888 (all with the gate, exit 0);
int-spec 3/3 (real Postgres); tsc --noEmit clean for client + server;
pnpm install --frozen-lockfile consistent (lockfile additive).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 12:37:28 +03:00
claude code agent 227 1a7b817250 fix(prosemirror-markdown): escape image alt + consolidate schema sanitizers + tidy (#333 review F1-F4)
F1 [critical, data-loss] — escape the image alt in `![alt](src)`. Canon #4 moved
the top-level image off the lossless <img> form onto markdown `![alt](src)`, but
the alt was inserted raw; the importer re-parses the `![alt]` label as CommonMark
inline, so a markdown-active char in a realistic description ("Figure [1]", "the
*new* logo", "a]b[c") broke the round-trip — the image node vanished or emphasis
collapsed. Now `escapeLinkText(imgAttrs.alt ?? "")`, exactly as the link-form
media (attachment/pdf/embed) already escape their visible text. Regression test
added: six active-punctuation alts round-trip byte-stable with the node intact.

F2 [drift] — re-export `clampCalloutType` / `sanitizeCssColor` from the package
barrel and drop the verbatim copies in the mcp schema shim. The copies had
already drifted (the mcp `clampCalloutType` lost the callout-type alias mapping
the package applies), which is exactly the schema drift #293 exists to kill. The
sanitizers now live only in the package; mcp `schema.test.mjs` exercises the
single alias-aware implementation.

F3 [docs] — AGENTS.md:296 said `packages/mcp/build/` is committed; this branch
gitignored it (git-sync/prosemirror-markdown convention). Updated the line to say
it is gitignored and rebuilt in CI/Docker via `pnpm build`.

F4 [cleanup] — removed the dead `test.typecheck` block from the package
vitest.config.ts and deleted tsconfig.vitest.json. Both were copied verbatim from
git-sync; this package has zero `*.test-d.ts` files, and the ported comments
referenced git-sync-only entities. Kept the `docmost-client` resolve alias
(22 tests use it) and the runtime include/environment.

package vitest: 658 passed (+1 F1 regression); tsc clean. git-sync: 268 passed.
mcp: node --test 454 passed; tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 12:17:46 +03:00
claude code agent 227 52beae85b3 fix(client): close mobile sidebar drawer after creating a page (#325)
On mobile the "create page" action is triggered from inside the off-canvas
sidebar drawer (the space sidebar "+" and temporary-note buttons, and the
tree-row "add subpage"). handleCreate navigated to the new page's editor route
but never closed that drawer, so it stayed open on top of the freshly created
page — the editor was hidden behind the page tree ("as if the page didn't
open", #325 item 5).

Close the mobile sidebar (`setMobileSidebar(false)`) right after navigating,
mirroring the existing drawer-close on a tree-row tap (space-tree-row). Placing
it in handleCreate covers all three create entry points in one spot. It is a
no-op on desktop, where the mobile-sidebar atom is already false and only
governs the sub-992px collapsed state — desktop behavior is unchanged.

Verified: `tsc --noEmit` clean; client vitest 887 passed | 1 expected-fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 12:06:45 +03:00
claude code agent 227 124f5a45a2 refactor(mcp): consume @docmost/prosemirror-markdown, drop the drifted converter copy (#293/#326 step 5)
mcp had its OWN drifted copy of the converter (markdown-converter.ts ~900 lines,
docmost-schema.ts ~1270 lines, markdown-document.ts) — older than the shared
package, missing the git-sync fixes AND the #293 canon. This switches mcp's
converter CORE to @docmost/prosemirror-markdown, so mcp jumps straight to the
canonical format and the drift-generating second copy is gone.

- markdown-converter.ts / markdown-document.ts / docmost-schema.ts become thin
  re-export shims of the package (convertProseMirrorToMarkdown, the docmost:meta
  envelope, docmostExtensions + docmostSchema=getSchema(docmostExtensions)). The
  mcp-only helpers clampCalloutType/sanitizeCssColor are preserved verbatim in
  the schema shim (the package doesn't expose them via its barrel). ~2170 lines
  of the drifted converter/schema bodies deleted.
- collaboration.ts drops its own ~360-line marked pipeline (preprocessCallouts,
  bridgeTaskLists, extractFootnotes, the footnoteRef extension) and re-points to
  the package's markdownToProseMirror, keeping markdownToProseMirrorCanonical and
  all the yjs/collab write glue. footnote-lex/analyze doc comments updated (they
  now describe advisory legacy-syntax diagnostics, not an importer).

Schema parity verified: the package schema is a strict SUPERSET of mcp's old
schema — every node and attr mcp declared is present (the package only adds
status/pageEmbed/transclusion*/subpages.recursive/etc.), so nothing is silently
dropped on the switch. The switch actually FIXES two pre-existing mcp data-loss
bugs its own tests documented: htmlEmbed and pageBreak now round-trip (were
dropped by the old mcp converter).

Footnotes: the package assembles inline ^[body] footnotes on import (sequential
fn-N ids, identical bodies merged), so mcp's canonicalizeFootnotes is now an
idempotent no-op after it (verified). Legacy reference footnotes [^id]/[^id]:
are inert literal text (canon #2 no-backward-compat) — lossless, the text
survives verbatim.

Build hygiene: packages/mcp/build/ is now gitignored and untracked, matching the
git-sync/prosemirror-markdown convention (private package, rebuilt in CI/Docker,
so src and prod can never silently diverge). This also removes a dead untracked
build/_vendored_editor_ext/ artifact that a broad `git add` would otherwise
commit.

Dependency: packages/mcp/package.json gains @docmost/prosemirror-markdown
(workspace:*); pnpm-lock.yaml gets the matching link importer (mirrors git-sync).

mcp tests updated deliberately to the canonical forms (highlight ==, math $…$,
image ![](src)<!--img-->, drawio/media discriminators, subpages/pageBreak
comments, textAlign, inline ^[…] footnotes) with strict assertions; 4 structural
safety-net round-trip tests added.

mcp: node --test 454 passed; tsc clean. package: 657 passed. git-sync: 268 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 11:16:09 +03:00
claude code agent 227 b751852425 fix(prosemirror-markdown): converter inventory bugs — spoiler/link-title in raw-HTML, contract test, codeCombined dead code (#293)
The four bugs found during the #293 HTML-emission inventory, fixed in the package:

1. Spoiler mark was silently lost in the raw-HTML path: inlineToHtml (columns /
   spanned cells) had no `case "spoiler"`, so spoilered text there dropped the
   mark on round-trip. Now emits `<span data-spoiler="true">` — the same form the
   top-level serializer uses and exactly what the schema's Spoiler mark parses.

2. Link `title` was dropped in the raw-HTML path: inlineToHtml's link case
   emitted `<a href>` without the title. The schema's link mark carries a
   `title` global attr (DocmostAttributes), so a titled link inside a column now
   round-trips via `<a href … title=…>`.

3. Serializer contract test: emoji/date/toc were flagged as possibly caseless
   inline atoms. Verified they exist in NEITHER the package schema NOR
   editor-ext, so no node handling is needed today. Added
   serializer-contract.test.ts, which derives every node type from the live
   schema (getSchema(docmostExtensions)) and asserts each has an explicit
   serializer `case` — all 45 current node types are covered and present, and a
   future node added without a case will fail this test loudly.

4. codeCombined dead code: `const codeCombined = false` was hardcoded, so every
   `codeCombined ? <html> : <markdown>` ternary always took the markdown branch.
   Removed the variable and the dead HTML-alternative branches (bold/italic/code/
   link/strike). Pure cleanup — output is byte-identical (goldens + full suite
   pass unchanged). The `hasCode` early-return (code excludes other marks) stays.

Tests: spoiler-inside-column and link-title-inside-column round-trips, the
serializer contract test + inline-atom non-empty behavioral checks.

package vitest: 657 passed; tsc clean. git-sync: 268 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 10:37:35 +03:00
claude code agent 227 65d81f745a feat(prosemirror-markdown): inline footnotes ^[text] (#293 canon #2)
Footnotes now use the single canonical Pandoc/Obsidian inline form: the note
body is written AT the reference as `^[body]`, and the separate
`<section data-footnotes>` list is NOT emitted in markdown — it is reassembled
on import. New shared module src/lib/footnote.ts.

Serialize (markdown-converter.ts): a top-of-convert pre-scan builds
Map<id, definition> from the footnotesList; a footnoteReference emits
`^[<rendered body>]` (body paragraphs joined by a literal `\n`, real
backslash-n written `\\n`, stray unbalanced `[`/`]` escaped via balanceBrackets
while a balanced `[link](url)` stays intact); footnotesList/footnoteDefinition
emit nothing; an ORPHAN definition (no ref) is appended at doc end as its own
`^[body]` line so bodies are never lost (intentional, documented). The raw-HTML
path (inlineToHtml, columns) emits `<sup data-footnote-ref data-fn-text="…">`,
carrying the text at the ref there too; blockToHtml keeps the schema
`<section>`/`<div>` form for a list nested in a column.

Parse (markdown-to-prosemirror.ts): a `^[…]` inline extension on the dedicated
marked instance BALANCES brackets with a depth counter (respecting `\`-escapes),
so `^[note [a] b]` captures the full content, unbalanced `^[` fails open to
literal text. A post-marked assembleFootnotes pass collects every
`<sup data-fn-text>`, dedups by the EXACT body string, assigns sequential ids
(fn-1, fn-2, … first-seen), builds one `<div data-footnote-def>` per unique body
in a single `<section data-footnotes>`, and strips data-fn-text. No hash is used
(F1): dedup keying on the exact text makes an id collision between DIFFERENT
bodies impossible, while identical bodies still merge; ids are never written to
markdown, so round-trips stay byte-stable, and all id assignment is local to the
one call (race-free).

Correctness hardening from internal review:
- F2: raw user backslashes in a footnote body are doubled (`\`->`\\`) at text
  emission (via a per-conversion inFootnoteBody closure flag) BEFORE the
  serializer's own escapes (`\[ \] \= \$`) are layered on, so a body ending in
  `\` (Windows path, LaTeX, regex) no longer breaks the `^[…]` envelope and
  round-trips exactly; parseInline decodes `\\`->`\`. The old `\n`->`\\n` step is
  subsumed by this and removed.
- N1: assembleFootnotes runs to a FIXED POINT — parseInline of a def body can
  spawn a nested `<sup data-fn-text>` (a legal nested footnote `^[a ^[b] c]`),
  so the section is attached before the loop (querySelectorAll only sees
  attached nodes) and the scan repeats until no pending sup remains; the dedup
  map persists across rounds. Nested and 3+-level footnotes now round-trip
  byte-stably instead of silently dropping the inner body. Bounded by
  MAX_FOOTNOTE_ROUNDS as a fail-open safety net.
- N2: the id counter is seeded past the highest existing fn-<N> so a reused
  section's ids can never collide with generated ones.
- A literal `^[` in prose text is escaped `^\[` so it does not become a phantom
  footnote on re-import (codeBlock/inline-code excluded).

No backward compat: reference form `[^id]`/`[^id]: def` is not parsed (stays
literal). No existing golden asserted the old footnote HTML output.

Tests: new footnote.test.ts (22 cases: basic byte-stable round-trip, bracket
balancing, multi-paragraph `\n`, real backslash-n, dedup both directions,
NESTED + 3-level nest, F1 hash-collision pair surviving as distinct defs, F2
backslash bodies byte-stable, N2 id-seed, column data-fn-text form, orphan def,
no-backward-compat, literal-`^[` prose, fail-open, empty `^[]`).

package vitest: 607 passed; tsc clean. git-sync: 268 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 10:31:00 +03:00
claude code agent 227 bfbd927866 feat(prosemirror-markdown): math as $…$ / $$…$$ (#293 canon #6)
mathInline serializes as `$LaTeX$` and mathBlock as an own-line `$$\n<latex>\n$$`
fence (multi-line safe), closing hand-authoring gap A18. The LaTeX still lives in
node.attrs.text; a literal `$` inside it is escaped `\$`. On the raw-HTML path
(columns/cells) math keeps the schema-HTML `<span data-type="mathInline">` /
`<div data-type="mathBlock">` form (markdown is not re-parsed inside raw HTML) —
blockToHtml gets an explicit mathBlock case and inlineToHtml a mathInline case,
sharing the mathInlineHtml/mathBlockHtml helpers with the fallbacks so the two
forms cannot drift.

Parse: mathInlineExtension (inline) + mathBlockExtension (block) are added to the
SAME dedicated marked instance introduced for canon #7 (global singleton
untouched). The inline extension uses a currency-safe PANDOC rule: an opening `$`
must not be followed by whitespace, and the closing `$` must not be preceded by
whitespace nor followed by a digit — so `$5`, `$5 and $10`, `a $5 b $6 c`, `100$`
stay literal text while `$x^2$` is math. The block extension matches a `$$` fence
line and captures multi-line LaTeX non-greedily up to the next `$$` line.

The pandoc boundary rule lives ONCE in the new math-inline.ts
(INLINE_MATH_SOURCE) and is shared by the import tokenizer (^-anchored) and the
export prose escaper (global), so parse and serialize cannot disagree about what
is math. escapeProseMath (case "text", non-code runs only) escapes ONLY the two
delimiting `$` of a span the rule WOULD match, so a would-be-math prose span like
`the set $A$` re-imports as literal text while currency `$5 and $10` is emitted
CLEAN (zero backslash churn). marked decodes `\$`→`$` on re-parse, byte-stable.

Fallbacks to the lossless schema-HTML form (all documented + tested):
mathInline → <span> when empty / whitespace-edged / multi-line / pre-existing
`\$` / trailing `\` / immediately before a digit-text sibling (renderInlineChildren
guard, so `$…$5` can't lose the node); mathBlock → <div> when the LaTeX contains
`$$`. Each fallback round-trips losslessly and byte-stably.

Code safety (guards the canon #7 regression class): codeBlock reads raw child
text and inline `code` runs are excluded from escapeProseMath, so `$5`/`$x$` in
code stay literal with no math and no backslash corruption. ReDoS-checked on
adversarial 40k-char inputs (0–1 ms).

Tests: new math.test.ts (26 cases: serialize exactness, multi-line block, `\$`
escaping, currency ×5 asserting no `\$`, prose escape, columns schema-HTML,
inline-code/codeBlock safety, fail-open). Goldens in roundtrip / markdown-converter
flipped top-level math to `$…$`/`$$…$$`; the escapeAttr-idempotence golden wraps
math in a column (still exercises escapeAttr); columns/raw-HTML math assertions
unchanged.

package vitest: 585 passed; tsc clean. git-sync: 268 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 09:37:37 +03:00
claude code agent 227 77f5224b55 feat(prosemirror-markdown): highlight without color as ==text== (#293 canon #7)
A `highlight` mark WITHOUT a color now serializes as the Obsidian/GFM `==text==`
syntax (closing hand-authoring gap A19); a highlight WITH a color keeps the
`<mark style="background-color: …">` HTML form (condition is deterministic on
the color attr). On the raw-HTML path (columns/spanned cells) BOTH forms stay
`<mark>` via inlineToHtml — markdown is not re-parsed inside a raw-HTML block.

Parse: `==` is not standard markdown, so the importer uses a DEDICATED marked
instance (`new Marked().use({extensions:[highlightMark]})`) rather than the
global singleton — registered once, never leaks `==` behavior to other callers.
The inline extension tokenizes `==text==` (non-empty, non-space-leading inner,
lazy so `==a== ==b==` is two marks; inner re-tokenized so nested marks survive;
`====`/`==x` fail-open to literal) into `<mark>` with no color, which the schema
parses as a color-less highlight. Inline code (`` `a == b` ``) stays code via
marked token precedence. marked 17 defaults (gfm:true, breaks:false) are
identical for the fresh instance, so tables/strike/autolinks are unaffected.

Losslessness: a LITERAL `==` in a text run would otherwise be misparsed as a
highlight on the next import, so `case "text"` backslash-escapes each `=` of a
`==` pair (marked decodes `\=` back to `=`), and this round-trips byte-stably.
The escape does NOT run for inline-code runs, and — CRITICALLY — codeBlock now
reads its child text RAW (schema `content: "text*"`) instead of routing through
`case "text"`: marked does not decode `\=` inside a fence, so escaping there
would permanently stamp backslashes into any `==` comparison (ubiquitous in
source code) and corrupt the block on the git-sync data path.

Tests: new highlight.test.ts (19 cases incl. serialize forms, colored vs plain,
column `<mark>` path, nested marks, inline-code exclusion, literal-`==` escape,
fail-open, AND a codeBlock-with-`==` regression proving no backslash corruption
+ byte-stable round-trip). Golden inline-mark matrix flipped top-level no-color
highlight to `==m==`; the kept `<mark style=…>` assertions are the colored/
raw-HTML cases.

package vitest: 559 passed; tsc clean. git-sync: 268 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 09:12:18 +03:00
claude code agent 227 e2a3b5fc4d feat(prosemirror-markdown): media family as md-form + discriminator comment (#293 canon #8)
Ten media/embed node types move their TOP-LEVEL serialization off raw schema
HTML onto a readable markdown target plus an always-emitted discriminator
comment whose NAME selects the node type. The schema-HTML form is retained on
the raw-HTML/columns path (comments are dropped by the DOM parse stage there).

  image-form  ![](src)<!--name …-->   youtube, video, audio, drawio, excalidraw
  link-form   [text](src)<!--name …--> pdf, attachment, embed (text=filename/provider)
  standalone  <!--pageembed …--> / <!--transclusion …-->  pageEmbed, transclusionReference

The comment NAME is the node-type discriminator and is ALWAYS emitted, even when
the attr JSON is empty (`![](u)<!--youtube-->`), so a bare `![](u)` is never
mistaken for an `image` and a bare `[t](u)` stays a plain link — no URL-sniffing.
src rides in the markdown target; every other non-default attr (incl. the id
links attachmentId/sourcePageId/transclusionId) rides in the comment JSON
(stable key order, numerics stringified, align="center" omitted).

New src/lib/media-html.ts: byte-exact builders reproducing the schema HTML each
old processNode case returned. Both the serializer's raw-HTML path (blockToHtml,
now de-delegated from `return processNode(block)` to explicit per-type cases)
and the importer call these, so serialize and parse cannot drift.

Import (applyCommentDirectives): image-form binds the preceding <img> (src from
it), link-form the preceding <a> (src=href, text=filename/provider), standalone
replaces the comment (same leading-doc-level handling as #5). Each rebuilds the
schema element via the media-html builder, then swaps it in; the empty-<p> hoist
is absorbed by stripEmptyParagraphs. Fail-open: wrong element/position/name or
malformed JSON -> inert, no throw.

Link-form visible text is escaped (escapeLinkText) for the FULL set of
CommonMark inline-active punctuation (\ ` * _ ~ [ ] < & ! ( )), not just [ ] \:
the label is parsed as inline content, so a filename/provider like
`report *v2*.pdf` or `![shot](x).pdf` would otherwise lose the markup (or
fragment the parse) when the importer reads a.textContent back — a data-loss
regression vs the old data-attachment-name form. Adversarial round-trip fixtures
lock byte- and value-stability for emphasis/code/strike/autolink/entity/image
markers and nested-link names.

Tests: new media-comments.test.ts (40 cases: per-type exact md + lossless
byte-stable round-trip incl. id links, minimal-node discriminator-still-emitted,
in-column schema-HTML form, discriminator integrity, fail-open, active-punct
filenames). Goldens in media-roundtrip / markdown-converter-golden /
markdown-converter / diagram-roundtrip updated to the md+comment form (columns
stay schema-HTML). The former known-limitation image-diagrams fixture is now
byte- AND canonically-stable (canon #8 omits the diagram align="center" default)
and was promoted from an it.fails into the green corpus (11-image-diagrams.json).
git-sync stabilize.test.ts: the "diagram materializes data-align=center" fixpoint
moved into a column (where the raw-HTML asymmetry still holds), since top level
is now byte-stable.

package vitest: 540 passed; tsc clean. git-sync: 268 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 08:52:02 +03:00
claude code agent 227 d7d8db2102 feat(prosemirror-markdown): images as ![alt](src) + attached img-comment (#293 canon #4)
Every image now serializes as `![alt](src)`; non-default layout/identity attrs
that markdown cannot express ride along in an attached `<!--img {…}-->` comment
on the same line, replacing the prior "image-with-attrs -> raw <img>" split for
the top-level path:
  ![схема](/s.png) <!--img {"width":"420","align":"left","attachmentId":"…"}-->

Keys (emitted only when non-default, stable order): width, height, align, size,
aspectRatio, attachmentId, caption, title. Numeric sizing attrs are stringified
in the payload (the import side reads DOM attributes back as strings), so a
numeric `width:420` round-trips byte-stably instead of churning `420 -> "420"`.
attachedCommentFor defuses any `--` in a value (e.g. a caption containing the
comment-closing `-->`) so the payload can never close the comment early.

Align default unified to "center" (#293 canon #4): editor-ext declares
image.align default "center" while this package's schema declared null — keeping
null would make the clean `![](src)` form dead code (every editor image is
"center"). Now the schema default is "center" (docmost-schema image align, with
explicit parseHTML/renderHTML), canonicalize KNOWN_DEFAULTS drops align=="center"
for image, and the serializer omits align when it is null OR "center". A null
align collapses to "center" on re-import (a null align is not a distinct editor
state) — stable, no ping-pong. Only left/right emit a comment.

Import: applyCommentDirectives gains an `img` handler that targets the comment's
previousElementSibling <img> and writes each decoded key to the DOM attribute
the schema reads (align, width, height, data-size, data-aspect-ratio,
data-attachment-id, data-caption, title), then removes the comment. Attached
only: a standalone `<!--img-->` with no adjacent image is inert. Fail-open on
malformed JSON / unknown keys.

Raw-HTML path unchanged in spirit: images inside columns/cells keep the
`<img …>` form (comments are dropped by the DOM parse stage); imageToHtml now
omits a redundant align="center" to match the unified default.

Tests: new image-comment.test.ts (21 cases incl. caption == `-->`, numeric-size
byte-stability, image-in-column <img> form, fail-open). Goldens updated
deliberately: markdown-roundtrip-spoiler-caption (captioned image -> comment
form), markdown-converter-gaps spec 14/15 (title now round-trips via comment;
column image drops redundant align), canonicalize-extra (center+null dropped,
left kept).

package vitest: 498 passed | 1 expected-fail; tsc clean. git-sync (rebuilt
build): 268 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 08:16:28 +03:00
claude code agent 227 e814bca243 feat(prosemirror-markdown): subpages/pageBreak as standalone comments (#293 canon #5)
Move the two "invisible machinery" atoms off the <div data-type="..."> HTML
form onto standalone HTML comments on their own line, keeping the markdown
human-readable while still round-tripping:
  subpages  -> <!--subpages-->  /  <!--subpages {"recursive":true}-->
  pageBreak -> <!--pagebreak-->

Adds standaloneCommentFor(name, attrs?) to attached-comment.ts (emits
`<!--name-->` when attrs are empty/absent, else `<!--name {compact-json}-->`).
The `--`-escaping + compact-JSON logic is factored into a shared internal
escapeCommentJson() so standaloneCommentFor and attachedCommentFor cannot drift
(verified byte-identical output for attachedCommentFor — no #9 regression).

Position determines legality (canon #5): subpages/pagebreak are honored ONLY
standalone; the same comment attached after visible text is inert. The parser
pass (applyAttachedComments renamed applyCommentDirectives) now also
materializes these standalone comments into the schema `<div data-type=...>`
element before generateJSON drops the comment node. A LEADING standalone
comment is parsed at document level (outside <body>); the pass walks the whole
document and re-inserts leading comments into <body> in document order, so
block order is preserved.

Raw-HTML path: blockToHtml gains explicit subpages/pageBreak cases emitting the
`<div data-type=...>` form. Comments are dropped by the DOM parse stage inside
columns/cells, so the div-form must stay there — this also fixes a latent
default-fallthrough (`<div></div>`) that silently dropped these atoms inside a
column.

Tests: new machinery-comments.test.ts (primitive, subpages default/recursive
exact strings + round-trip, pageBreak, subpages-inside-column div-form,
fail-open for attached-position/malformed, and multi-node document-order
regression locking the leading/mid/trailing comment ordering). Top-level
goldens in markdown-converter-golden/gaps updated deliberately to the comment
form; the columns/raw-HTML goldens keep the div-form.

package vitest: 477 passed | 1 expected-fail; tsc clean. git-sync: 268 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 07:56:40 +03:00
claude code agent 227 f1ab76e879 feat(prosemirror-markdown): serialize textAlign as attached comment (#293 canon #9)
Move paragraph/heading textAlign off the HTML-wrapper form
(<p style="text-align:…"> / <hN style=…>) onto a trailing attached HTML
comment on the block line: `text <!--attrs {"textAlign":"center"}-->`. This
keeps the readable markdown block form (plain `text` / `## Title`) while
preserving alignment losslessly. "left"/null stay bare (no churn).

Adds a reusable attached-comment primitive (attached-comment.ts) that #4
(image) and #8 (media) will reuse:
- attachedCommentFor(name, json) -> `<!--name {compact-json}-->`, escaping any
  `--` pair inside the JSON as -- so the payload can never close the
  comment early;
- parseAttachedComment(data) with grammar `^\s*([A-Za-z][\w-]*)(?:\s+({…}))?\s*$`
  whose name excludes `:`, so envelope comments (docmost:meta / docmost:comments)
  never match — fail-open on anything malformed.

On import, applyAttachedComments runs AFTER marked.parse but BEFORE generateJSON
(parse5 drops comments), re-expressing the attrs comment as an inline
text-align style on the parent block, then removing the comment node.

Guards: emit only when there is a visible element to attach to — paragraph
requires non-empty text, heading requires non-empty headingText (symmetry:
an empty aligned heading stays bare `##`, no orphan comment).

Goldens in markdown-converter-golden/gaps updated deliberately to the
attached-comment form (assertions stay strict: exact output + lossless
round-trip). New textalign.test.ts (19 tests) covers center/right/justify on
paragraph and heading, byte-stable re-export, and fail-open branches.

Raw-HTML containers (columns/cells/callout via blockToHtml) keep the inline
text-align form intentionally — comments are dropped inside raw HTML.

package vitest: 462 passed | 1 expected-fail; tsc clean. git-sync: 268 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 07:39:46 +03:00
claude code agent 227 6dcc19ce59 refactor(git-sync): consume @docmost/prosemirror-markdown, drop the duplicate lib (#293 stage 3 / no-op)
git-sync's converter-core (src/lib) was a byte-identical duplicate of the new
@docmost/prosemirror-markdown package (created in the previous commit). Switch
git-sync to consume the package and delete its copy — ending the duplication
that the whole #293 effort targets. Pure no-op: NO format/behavior change.

- git-sync depends on @docmost/prosemirror-markdown (workspace:*); engine
  (stabilize/push/pull) + src/index barrel + 12 engine tests re-point their
  converter imports to the package.
- Delete git-sync/src/lib (8 files) and the 23 duplicate converter-core test
  files + their fixtures — the converter and its ~440 tests now live once, in the
  package. git-sync keeps only its ENGINE tests, which exercise the converter
  through the package (the no-op proof). Kept roundtrip-helpers.ts (an engine
  test imports firstDivergence from it; pure helper, no double-run).
- Added docmostExtensions to the package barrel (a kept engine schema-validity
  test needs it).

Verified: editor-ext + prosemirror-markdown + git-sync all tsc EXIT 0;
git-sync vitest 28 files, 268 passed, 0 failures (engine cycle/roundtrip/push/
pull/reconcile green = no-op proof); prosemirror-markdown vitest still 443 passed
| 1 expected-fail; pnpm --frozen-lockfile EXIT 0; no ../lib refs remain in git-sync.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 07:19:29 +03:00
claude code agent 227 d6d7dd82f6 feat(prosemirror-markdown): new headless converter package seeded from git-sync (#293 stage 1)
Create @docmost/prosemirror-markdown — the single framework-free ProseMirror<->
Markdown converter + schema mirror that git-sync and mcp will both consume,
ending the three-hand-synced-copies drift (#293). This step only CREATES the
package (no consumer yet; git-sync untouched); the switch of git-sync and mcp
onto it, plus the canonical format decisions, come in later commits of this PR.

- packages/prosemirror-markdown/src/lib/: the 8 converter-core files copied
  VERBATIM from packages/git-sync/src/lib (docmost-schema, markdown-converter,
  markdown-to-prosemirror, canonicalize, markdown-document, node-ops, page-file,
  index). Confirmed byte-identical — no behavioral drift introduced.
- src/index.ts barrel; package.json (@tiptap/* + jsdom/marked/zod, editor-ext
  workspace devDep for the contract test); tsconfig/vitest configs.
- 24 converter-core test files + fixtures copied (engine-coupled layout/
  redteam-layout-title tests correctly excluded — they import ../src/engine).
- pnpm-lock importer added; build/ gitignored (CI-built).

Verified (clean checkout, no network): pnpm --frozen-lockfile EXIT 0; tsc EXIT 0;
vitest 23 files, 443 passed | 1 expected-fail (the same image-diagrams
known-limitation carried from git-sync) — faithful extraction. git-sync untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 07:10:04 +03:00
198 changed files with 11538 additions and 16322 deletions
+7
View File
@@ -202,6 +202,13 @@ MCP_DOCMOST_PASSWORD=
# Default 900000 (15 min).
# AI_MCP_CALL_TIMEOUT_MS=900000
# Deferred tool loading for the in-app AI chat (#332). Default ON: the agent sees
# a compact <tool_catalog> and only CORE tools + a loadTools meta-tool are active
# each step; deferred tools (the fat/rare ones + all external MCP tools) load on
# demand. Set AI_CHAT_DEFERRED_TOOLS=false to restore the old "all tools always
# active" behavior.
# AI_CHAT_DEFERRED_TOOLS=true
# --- Anonymous public-share AI assistant ---
# Opt-in per workspace (AI settings -> "public share assistant"; off by default).
# When enabled, anonymous visitors of a published share can ask an AI about that
+8
View File
@@ -72,6 +72,14 @@ jobs:
- name: Build editor-ext
run: pnpm --filter @docmost/editor-ext build
# @docmost/prosemirror-markdown is the shared converter (#293/#326); its
# build/ is gitignored, and plain `pnpm -r test` does NOT honour nx
# `dependsOn: ^build`, so its consumers (mcp `pretest: tsc`, git-sync vitest
# typecheck) fail with TS2307 Cannot find module '@docmost/prosemirror-markdown'
# unless it is built first. Build it before the recursive test run.
- name: Build prosemirror-markdown
run: pnpm --filter @docmost/prosemirror-markdown build
- name: Run unit tests
run: pnpm -r test
+10 -1
View File
@@ -4,12 +4,21 @@
data
# compiled output
/dist
node_modules/
node_modules
# git-sync compiled output (built in CI/Docker via `pnpm build`, never committed,
# so src/ and prod can never silently diverge).
packages/git-sync/build/
# prosemirror-markdown compiled output (built in CI/Docker via `pnpm build`,
# never committed, so src/ and prod can never silently diverge).
packages/prosemirror-markdown/build/
# mcp compiled output (built in CI/Docker via `pnpm build`, never committed, so
# src/ and prod can never silently diverge). Matches the git-sync/prosemirror-
# markdown convention; the package is private and rebuilt at deploy.
packages/mcp/build/
# Logs
logs
*.log
+4 -3
View File
@@ -200,7 +200,8 @@ pnpm workspace (`pnpm@10.4.0`) orchestrated by **Nx**. Four workspace packages:
| `apps/server` | `server` | NestJS 11 + Fastify, Kysely (Postgres), Redis | Backend API, collaboration, AI |
| `apps/client` | `client` | React 18 + Vite + Mantine 8 + TanStack Query + Jotai | SPA frontend |
| `packages/editor-ext` | `@docmost/editor-ext` | Tiptap/ProseMirror | Shared Tiptap node/mark extensions, imported by both the client and the server |
| `packages/mcp` | `@docmost/mcp` | MCP SDK, Tiptap, Yjs | Standalone MCP server, also bundled into the server at `/mcp`. Does **not** import `editor-ext` — it keeps its own vendored mirror of the schema in `packages/mcp/src/lib/` |
| `packages/mcp` | `@docmost/mcp` | MCP SDK, Tiptap, Yjs | Standalone MCP server, also bundled into the server at `/mcp`. Consumes the shared converter/schema from `@docmost/prosemirror-markdown` (#293) — it no longer carries its own vendored converter/schema copy |
| `packages/prosemirror-markdown` | `@docmost/prosemirror-markdown` | Tiptap, marked, jsdom | The single, canonical ProseMirror↔Markdown converter + Docmost schema mirror (#293). Consumed by `mcp` and `git-sync`; there is exactly ONE copy of the converter now |
`build` targets are Nx-cached and dependency-ordered (`dependsOn: ["^build"]`), so `editor-ext` builds before the apps. `nx.json` sets `affected.defaultBase: main`.
@@ -282,7 +283,7 @@ The API server is a Fastify app with a global `/api` prefix (`main.ts` excludes
### Client structure
Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirrors the server domains: `page`, `space`, `comment`, `ai-chat`, `editor`, …). Conventions:
- **TanStack Query** for server state (one `queries/` file per feature), **Jotai** atoms for local/shared UI state, **Mantine 8** + CSS modules (`*.module.css`) + `postcss-preset-mantine` for UI.
- The editor is Tiptap; shared node/mark extensions live in `packages/editor-ext` and are imported by **both the client and the server** (collaboration, import/export) — editor schema changes often need to be made in `editor-ext`, not just the client. Note `packages/mcp` does *not* depend on `editor-ext`; it carries its own mirrored copy of the schema, so keep the two in sync manually when the document schema changes.
- The editor is Tiptap; shared node/mark extensions live in `packages/editor-ext` and are imported by **both the client and the server** (collaboration, import/export) — editor schema changes often need to be made in `editor-ext`, not just the client. The ProseMirror↔Markdown converter and its Docmost schema mirror now live in a SINGLE package, `@docmost/prosemirror-markdown` (#293), consumed by both `mcp` and `git-sync` — do NOT reintroduce a per-package copy. `editor-ext` is the upstream source of the Tiptap schema; the package's `docmost-schema.ts` mirrors it and a serializer-contract test (`packages/prosemirror-markdown/test/serializer-contract.test.ts`) guards the boundary (every schema node must have a converter case), so a drift surfaces as a failing test rather than silent divergence.
- API access goes through `apps/client/src/lib/api-client.ts` (axios). The `@` alias maps to `apps/client/src`.
- Runtime config is injected at build time by `vite.config.ts` via `define` (`APP_URL`, `COLLAB_URL`, `APP_VERSION`, …) — these come from the root `.env`, not from `import.meta.env`.
@@ -293,7 +294,7 @@ Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirro
- The version string shown in the UI comes from `APP_VERSION` (CI/Docker) or `git describe --tags --always` (local), resolved in `vite.config.ts` — not from `package.json`.
- Server TS config is permissive (`noImplicitAny: false`, `strictNullChecks: false`, `no-explicit-any` lint disabled). Follow the existing relaxed style rather than tightening types broadly.
- Dependency versions are heavily pinned via `pnpm.overrides` and `pnpm.patchedDependencies` (`scimmy`, `yjs`) in the root `package.json`. Don't bump pinned/patched deps casually; the patches and overrides exist for compatibility/security reasons.
- **Adding/renaming/removing an MCP tool requires updating `SERVER_INSTRUCTIONS`** in `packages/mcp/src/index.ts` — the intent-routing guide MCP clients receive on initialize. This applies both to inline `server.registerTool(...)` calls in `index.ts` and to specs in `packages/mcp/src/tool-specs.ts`. Enforced by `packages/mcp/test/unit/server-instructions.test.mjs`, which fails when a registered tool is not mentioned in the guide (deliberate opt-outs go into its `EXCEPTIONS` list). Remember `packages/mcp/build/` is committed — rebuild after editing.
- **Adding/renaming/removing an MCP tool requires updating `SERVER_INSTRUCTIONS`** in `packages/mcp/src/index.ts` — the intent-routing guide MCP clients receive on initialize. This applies both to inline `server.registerTool(...)` calls in `index.ts` and to specs in `packages/mcp/src/tool-specs.ts`. Enforced by `packages/mcp/test/unit/server-instructions.test.mjs`, which fails when a registered tool is not mentioned in the guide (deliberate opt-outs go into its `EXCEPTIONS` list). `packages/mcp/build/` is gitignored and rebuilt in CI/Docker via `pnpm build` (same convention as `git-sync`/`prosemirror-markdown`) — never commit it; rebuild locally after editing to run the tests.
## CI / release
+8
View File
@@ -38,6 +38,14 @@ COPY --from=builder /app/packages/editor-ext/dist /app/packages/editor-ext/dist
COPY --from=builder /app/packages/editor-ext/package.json /app/packages/editor-ext/package.json
COPY --from=builder /app/packages/mcp/build /app/packages/mcp/build
COPY --from=builder /app/packages/mcp/package.json /app/packages/mcp/package.json
# mcp now depends on @docmost/prosemirror-markdown (workspace:*) and eager-imports
# it at runtime (the in-app ai-chat DocmostClient loads build/index.js -> lib/
# markdown-converter.js). Ship the built package + its manifest, or the prod
# install resolves a broken workspace symlink and every ai-chat tool dies with
# ERR_MODULE_NOT_FOUND (#293/#326 step 5). (git-sync has no runtime consumer yet;
# revisit at step 6 when #119 lands.)
COPY --from=builder /app/packages/prosemirror-markdown/build /app/packages/prosemirror-markdown/build
COPY --from=builder /app/packages/prosemirror-markdown/package.json /app/packages/prosemirror-markdown/package.json
# Copy root package files
COPY --from=builder /app/package.json /app/package.json
+3
View File
@@ -40,6 +40,7 @@
"axios": "1.16.0",
"blueimp-load-image": "5.16.0",
"clsx": "2.1.1",
"diff": "8.0.3",
"dompurify": "3.4.1",
"file-saver": "2.0.5",
"highlightjs-sap-abap": "0.3.0",
@@ -81,6 +82,7 @@
"@types/react": "18.3.12",
"@types/react-dom": "18.3.1",
"@vitejs/plugin-react": "6.0.1",
"@vitest/coverage-v8": "4.1.6",
"eslint": "9.28.0",
"eslint-plugin-react": "7.37.5",
"eslint-plugin-react-hooks": "7.0.1",
@@ -95,6 +97,7 @@
"typescript": "5.9.3",
"typescript-eslint": "8.57.1",
"vite": "8.0.5",
"vite-plugin-compression2": "2.5.3",
"vitest": "4.1.6"
}
}
@@ -1382,5 +1382,8 @@
"Applied": "Applied",
"Suggestion applied": "Suggestion applied",
"Failed to apply suggestion": "Failed to apply suggestion",
"The commented text changed since this suggestion was made; it was not applied.": "The commented text changed since this suggestion was made; it was not applied."
"The commented text changed since this suggestion was made; it was not applied.": "The commented text changed since this suggestion was made; it was not applied.",
"Dismiss": "Dismiss",
"Suggestion dismissed": "Suggestion dismissed",
"Failed to dismiss suggestion": "Failed to dismiss suggestion"
}
@@ -1245,5 +1245,8 @@
"Applied": "Применено",
"Suggestion applied": "Предложение применено",
"Failed to apply suggestion": "Не удалось применить предложение",
"The commented text changed since this suggestion was made; it was not applied.": "Прокомментированный текст изменился после создания предложения; оно не было применено."
"The commented text changed since this suggestion was made; it was not applied.": "Прокомментированный текст изменился после создания предложения; оно не было применено.",
"Dismiss": "Не применять",
"Suggestion dismissed": "Предложение отклонено",
"Failed to dismiss suggestion": "Не удалось отклонить предложение"
}
@@ -1,5 +1,5 @@
import { describe, it, expect, beforeEach, vi } from "vitest";
import { render, screen, fireEvent, act } from "@testing-library/react";
import { render, screen, fireEvent, act, cleanup } from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
// Shared, hoisted mock state so the @ai-sdk/react and "ai" module mocks (hoisted
@@ -140,3 +140,91 @@ describe("ChatThread — send now (#198)", () => {
expect(prep({ messages: [], body: {} }).body.interrupted).toBe(false);
});
});
// The turn-end decision lives in the `onFinish` handler: given the terminal
// outcome of a turn (`isAbort` / `isDisconnect` / `isError`, or none = clean),
// it decides whether to CONTINUE (flush the next queued message) or END (leave
// the queue intact for the user), and which stop notice — if any — to show.
// `sendNow` is exercised above; these tests pin down the plain outcomes.
describe("ChatThread — turn-end decision (onFinish)", () => {
beforeEach(() => {
h.state.status = "streaming";
h.state.onFinish = null;
h.state.sendMessage.mockClear();
h.state.stop.mockClear();
h.state.transport = null;
});
// Drive a fresh onFinish with the given terminal flags after queueing a
// message, and report both what the parent was told and whether the queue was
// flushed (a resend to the sendMessage spy).
function finishWith(flags: {
isAbort?: boolean;
isDisconnect?: boolean;
isError?: boolean;
}) {
// Tear down any prior render so the loop-driven "every outcome" case does
// not leave duplicate queue buttons in the DOM.
cleanup();
h.state.sendMessage.mockClear();
const { onTurnFinished } = renderThread();
// Populate the queue while the turn is streaming.
fireEvent.click(screen.getByTestId("queue-btn"));
act(() => {
h.state.onFinish?.({
message: { id: "a", role: "assistant", parts: [] },
isAbort: false,
isDisconnect: false,
isError: false,
...flags,
});
});
return { onTurnFinished };
}
it("CONTINUES — flushes the next queued message on a clean finish", () => {
finishWith({});
// Clean finish (no terminal flag): the queued message is auto-sent.
expect(h.state.sendMessage).toHaveBeenCalledWith({ text: "queued text" });
// A clean finish shows no stop notice.
expect(screen.queryByText("Response stopped.")).toBeNull();
});
it("ENDS — keeps the queue intact on a user abort and shows the stopped notice", () => {
finishWith({ isAbort: true });
// A plain Stop (not the sendNow interrupt path) must NOT auto-resend: the
// queue is preserved for the user to decide.
expect(h.state.sendMessage).not.toHaveBeenCalled();
expect(screen.getByText("Response stopped.")).toBeTruthy();
});
it("ENDS — keeps the queue intact on a disconnect and shows the connection-lost notice", () => {
finishWith({ isDisconnect: true });
expect(h.state.sendMessage).not.toHaveBeenCalled();
expect(
screen.getByText("Connection lost — the answer was interrupted."),
).toBeTruthy();
});
it("ENDS — keeps the queue intact on a stream error (no auto-retry, no stopped notice)", () => {
finishWith({ isError: true });
// Blindly retrying after a failure would be wrong; the queue is left alone.
expect(h.state.sendMessage).not.toHaveBeenCalled();
// isError clears the neutral notice (the error banner covers this case).
expect(screen.queryByText("Response stopped.")).toBeNull();
});
it("notifies the parent on EVERY terminal outcome", () => {
// The chat-list refresh / new-chat id adoption must run on success and on
// every failure path alike.
for (const flags of [
{},
{ isAbort: true },
{ isDisconnect: true },
{ isError: true },
]) {
const { onTurnFinished } = finishWith(flags);
expect(onTurnFinished).toHaveBeenCalled();
}
});
});
@@ -0,0 +1,250 @@
import { describe, it, expect, vi } from "vitest";
import { render, screen } from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import { MemoryRouter } from "react-router-dom";
// matchMedia (read by MantineProvider) is stubbed globally in vitest.setup.ts.
// The fallback path renders the full TipTap editor; stub it so we can assert the
// safety valve fired without pulling in the editor stack.
vi.mock("@/features/comment/components/comment-editor", () => ({
default: () => <div data-testid="comment-editor-fallback" />,
}));
// Mention rendering hits react-query; stub the page/share queries so the mention
// case renders in isolation.
vi.mock("@/features/page/queries/page-query.ts", () => ({
usePageQuery: () => ({ data: undefined, isLoading: false, isError: false }),
}));
vi.mock("@/features/share/queries/share-query.ts", () => ({
useSharePageQuery: () => ({ data: undefined }),
}));
import { CommentContentView } from "./comment-content-view";
function renderView(content: string | object) {
return render(
<MantineProvider>
<MemoryRouter>
<CommentContentView content={content} />
</MemoryRouter>
</MantineProvider>,
);
}
const doc = (content: any[]) => JSON.stringify({ type: "doc", content });
const para = (content: any[]) => ({ type: "paragraph", content });
const text = (t: string, marks?: any[]) => ({ type: "text", text: t, marks });
describe("CommentContentView", () => {
it("renders paragraphs as <p> with text", () => {
const { container } = renderView(doc([para([text("Hello world")])]));
expect(screen.getByText("Hello world")).toBeDefined();
expect(container.querySelector("p")).not.toBeNull();
});
it("reproduces the read-only CommentEditor DOM nesting for CSS parity", () => {
const { container } = renderView(doc([para([text("x")])]));
// outer .commentEditor > .ProseMirror (module) > .ProseMirror (global) > p
const globalPm = container.querySelector("div.ProseMirror > p");
expect(globalPm).not.toBeNull();
});
it("renders the bold mark as <strong>", () => {
const { container } = renderView(
doc([para([text("bold", [{ type: "bold" }])])]),
);
const el = container.querySelector("strong");
expect(el?.textContent).toBe("bold");
});
it("renders the italic mark as <em>", () => {
const { container } = renderView(
doc([para([text("it", [{ type: "italic" }])])]),
);
expect(container.querySelector("em")?.textContent).toBe("it");
});
it("renders the strike mark as <s>", () => {
const { container } = renderView(
doc([para([text("st", [{ type: "strike" }])])]),
);
expect(container.querySelector("s")?.textContent).toBe("st");
});
it("renders the underline mark as <u> (not the editor fallback)", () => {
const { container } = renderView(
doc([para([text("un", [{ type: "underline" }])])]),
);
expect(container.querySelector("u")?.textContent).toBe("un");
// Underline is a supported mark, so no degrade to the editor fallback.
expect(screen.queryByTestId("comment-editor-fallback")).toBeNull();
});
it("renders the code mark as <code>", () => {
const { container } = renderView(
doc([para([text("co", [{ type: "code" }])])]),
);
expect(container.querySelector("code")?.textContent).toBe("co");
});
it("renders the link mark as an anchor with safe rel/target", () => {
const { container } = renderView(
doc([
para([
text("click", [
{ type: "link", attrs: { href: "https://example.com" } },
]),
]),
]),
);
const a = container.querySelector("a");
expect(a?.getAttribute("href")).toBe("https://example.com");
expect(a?.getAttribute("target")).toBe("_blank");
expect(a?.getAttribute("rel")).toBe("noopener noreferrer nofollow");
expect(a?.textContent).toBe("click");
});
it("neutralizes a javascript: link href (stored XSS) while keeping the text", () => {
const { container } = renderView(
doc([
para([
text("click", [
{ type: "link", attrs: { href: "javascript:alert(1)" } },
]),
]),
]),
);
const a = container.querySelector("a");
expect(a).not.toBeNull();
// No navigable javascript: href — attribute is absent (or empty).
expect(a?.getAttribute("href")).toBeFalsy();
// The link text is still rendered.
expect(a?.textContent).toBe("click");
});
it("neutralizes a control-char-obfuscated javascript: href", () => {
const { container } = renderView(
doc([
para([
text("x", [
{ type: "link", attrs: { href: "java\tscript:alert(1)" } },
]),
]),
]),
);
expect(container.querySelector("a")?.getAttribute("href")).toBeFalsy();
});
it("neutralizes a data: link href", () => {
const { container } = renderView(
doc([
para([
text("x", [
{
type: "link",
attrs: { href: "data:text/html,<script>alert(1)</script>" },
},
]),
]),
]),
);
expect(container.querySelector("a")?.getAttribute("href")).toBeFalsy();
});
it("preserves a mailto: link href (allowlisted scheme)", () => {
const { container } = renderView(
doc([
para([
text("mail", [
{ type: "link", attrs: { href: "mailto:a@b.com" } },
]),
]),
]),
);
expect(container.querySelector("a")?.getAttribute("href")).toBe(
"mailto:a@b.com",
);
});
it("preserves a relative link href (no scheme, not a script vector)", () => {
const { container } = renderView(
doc([
para([
text("rel", [{ type: "link", attrs: { href: "/some/path" } }]),
]),
]),
);
expect(container.querySelector("a")?.getAttribute("href")).toBe(
"/some/path",
);
});
it("nests multiple marks on one text node", () => {
const { container } = renderView(
doc([para([text("x", [{ type: "bold" }, { type: "italic" }])])]),
);
// bold wraps italic (or vice versa) — both elements exist around the text.
expect(container.querySelector("strong")).not.toBeNull();
expect(container.querySelector("em")).not.toBeNull();
expect(screen.getByText("x")).toBeDefined();
});
it("renders hardBreak as <br/>", () => {
const { container } = renderView(
doc([para([text("a"), { type: "hardBreak" }, text("b")])]),
);
expect(container.querySelector("br")).not.toBeNull();
});
it("renders a user mention as a styled span", () => {
const { container } = renderView(
doc([
para([
{
type: "mention",
attrs: { label: "Alice", entityType: "user", entityId: "u1" },
},
]),
]),
);
expect(screen.getByText("@Alice")).toBeDefined();
// No fallback to the editor.
expect(screen.queryByTestId("comment-editor-fallback")).toBeNull();
});
it("renders a page mention as a link", () => {
const { container } = renderView(
doc([
para([
{
type: "mention",
attrs: {
label: "Some Page",
entityType: "page",
slugId: "pg1",
},
},
]),
]),
);
expect(container.querySelector("a")).not.toBeNull();
expect(screen.getByText("Some Page")).toBeDefined();
});
it("renders a legacy plain-text (non-JSON) string as plain text", () => {
renderView("just a legacy string");
expect(screen.getByText("just a legacy string")).toBeDefined();
expect(screen.queryByTestId("comment-editor-fallback")).toBeNull();
});
it("falls back to CommentEditor for an unknown node type", () => {
renderView(doc([{ type: "codeBlock", content: [text("x")] }]));
expect(screen.getByTestId("comment-editor-fallback")).toBeDefined();
});
it("falls back to CommentEditor for malformed JSON", () => {
renderView('{"type":"doc","content":[');
expect(screen.getByTestId("comment-editor-fallback")).toBeDefined();
});
});
@@ -0,0 +1,199 @@
import React from "react";
import classes from "./comment.module.css";
import { MentionContent } from "@/features/editor/components/mention/mention-view";
import CommentEditor from "@/features/comment/components/comment-editor";
// Static, editor-free renderer of a comment body (ProseMirror JSON). It walks the
// document and emits plain DOM, avoiding the cost of a full TipTap/ProseMirror
// instance per comment (the panel used to spin up 400+ editors on mount).
//
// The supported node/mark set MUST mirror what CommentEditor enables
// (StarterKit + Mention + LinkExtension). Anything outside that set makes the
// whole comment degrade to the read-only CommentEditor via the fallback below,
// so we never show a half-rendered comment.
// Sentinel thrown when we hit a node/mark we don't know how to render statically.
// Caught at the top level to trigger the CommentEditor fallback for the whole comment.
class UnknownNodeError extends Error {}
// Protocol allowlist mirroring @tiptap/extension-link's default (the read-only
// CommentEditor path relies on it to blank javascript:/data: hrefs). The static
// renderer must apply the SAME sanitization because the backend stores comment
// content verbatim and React does not neutralize javascript: in an href.
const ALLOWED_URI_SCHEMES = /^(?:https?|ftps?|mailto|tel|callto|sms|cid|xmpp):/i;
function safeHref(href: unknown): string | undefined {
if (typeof href !== "string") return undefined;
// Strip control chars/whitespace that could smuggle a scheme past the test
// (e.g. "java\tscript:").
const cleaned = href.replace(/[\u0000-\u0020]/g, "").trim();
// Allow relative/anchor/protocol-relative links (no scheme) — not script vectors.
if (!/^[a-z][a-z0-9+.-]*:/i.test(cleaned)) return href;
return ALLOWED_URI_SCHEMES.test(cleaned) ? href : undefined;
}
interface PMMark {
type: string;
attrs?: Record<string, any>;
}
interface PMNode {
type: string;
attrs?: Record<string, any>;
content?: PMNode[];
text?: string;
marks?: PMMark[];
}
// Wrap a text node's string in its marks (marks nest, e.g. bold + italic).
function renderMarks(
text: React.ReactNode,
marks: PMMark[] | undefined,
keyPrefix: string,
): React.ReactNode {
if (!marks || marks.length === 0) return text;
return marks.reduce<React.ReactNode>((acc, mark, i) => {
const key = `${keyPrefix}-m${i}`;
switch (mark.type) {
case "bold":
return <strong key={key}>{acc}</strong>;
case "italic":
return <em key={key}>{acc}</em>;
case "strike":
return <s key={key}>{acc}</s>;
case "underline":
// StarterKit enables the Underline extension by default (Mod-u) and
// CommentEditor does not disable it, so real comments can carry this
// mark. Render it here rather than degrading the whole comment.
return <u key={key}>{acc}</u>;
case "code":
return <code key={key}>{acc}</code>;
case "link": {
// LinkExtension (TiptapLink) opens links in a new tab; keep the same
// safe rel semantics the editor produces. Sanitize the href against the
// extension's protocol allowlist — a disallowed scheme (javascript:,
// data:) yields undefined so the anchor is non-navigable but still shows
// its text, matching how extension-link blanks a bad href.
const href = safeHref(mark.attrs?.href);
return (
<a
key={key}
href={href}
target="_blank"
rel="noopener noreferrer nofollow"
>
{acc}
</a>
);
}
default:
throw new UnknownNodeError(`Unknown mark type: ${mark.type}`);
}
}, text);
}
function renderNode(node: PMNode, key: string): React.ReactNode {
switch (node.type) {
case "paragraph":
return <p key={key}>{renderChildren(node.content, key)}</p>;
case "text":
return (
<React.Fragment key={key}>
{renderMarks(node.text ?? "", node.marks, key)}
</React.Fragment>
);
case "hardBreak":
return <br key={key} />;
case "mention":
return (
<span key={key} style={{ display: "inline" }}>
<MentionContent attrs={node.attrs as any} />
</span>
);
default:
throw new UnknownNodeError(`Unknown node type: ${node.type}`);
}
}
function renderChildren(
content: PMNode[] | undefined,
keyPrefix: string,
): React.ReactNode {
if (!content) return null;
return content.map((child, i) => renderNode(child, `${keyPrefix}-${i}`));
}
// Reproduce the exact DOM nesting the read-only CommentEditor renders so the
// scoped CSS in comment.module.css (which targets
// `.commentEditor .ProseMirror :global(.ProseMirror)` and `.ProseMirror p`)
// applies pixel-for-pixel. Read-only => no data-editable / data-surface attrs.
function Shell({ children }: { children: React.ReactNode }) {
return (
<div className={classes.commentEditor}>
<div className={classes.ProseMirror}>
<div className="ProseMirror">{children}</div>
</div>
</div>
);
}
interface CommentContentViewProps {
content: string | object;
}
export function CommentContentView({ content }: CommentContentViewProps) {
// Degrade this single comment to the old editor-based render (safety valve).
const fallback = () => {
if (import.meta.env.DEV) {
console.warn(
"CommentContentView: unsupported comment content, falling back to editor",
);
}
return <CommentEditor defaultContent={content} editable={false} />;
};
let doc: unknown = content;
if (typeof content === "string") {
try {
doc = JSON.parse(content);
} catch {
const trimmed = content.trim();
// Looks like it was meant to be JSON but is malformed -> safety-valve fallback.
if (trimmed.startsWith("{") || trimmed.startsWith("[")) {
return fallback();
}
// Otherwise it's a legacy plain-text comment: render as a single paragraph.
return (
<Shell>
<p>{content}</p>
</Shell>
);
}
}
// Double-stringified / legacy plain-text stored as a JSON string.
if (typeof doc === "string") {
return (
<Shell>
<p>{doc}</p>
</Shell>
);
}
try {
const pmDoc = doc as PMNode;
if (!pmDoc || typeof pmDoc !== "object" || pmDoc.type !== "doc") {
throw new UnknownNodeError("Not a ProseMirror doc");
}
return <Shell>{renderChildren(pmDoc.content, "n")}</Shell>;
} catch (err) {
if (err instanceof UnknownNodeError) {
return fallback();
}
throw err;
}
}
export default CommentContentView;
@@ -1,5 +1,5 @@
import { describe, it, expect, vi } from "vitest";
import { render, screen, fireEvent } from "@testing-library/react";
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { render, screen, fireEvent, waitFor } from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import { IComment } from "@/features/comment/types/comment.types";
@@ -8,23 +8,74 @@ import { IComment } from "@/features/comment/types/comment.types";
// The comment mutation hooks reach out to react-query/network — stub them so the
// component renders in isolation. We only assert the AI-badge rendering branch.
const applyMutateAsync = vi.fn();
const dismissMutateAsync = vi.fn();
const updateMutateAsync = vi.fn();
vi.mock("@/features/comment/queries/comment-query", () => ({
useDeleteCommentMutation: () => ({ mutateAsync: vi.fn() }),
useResolveCommentMutation: () => ({ mutateAsync: vi.fn() }),
useUpdateCommentMutation: () => ({ mutateAsync: vi.fn() }),
useUpdateCommentMutation: () => ({ mutateAsync: updateMutateAsync }),
useApplySuggestionMutation: () => ({
mutateAsync: applyMutateAsync,
isPending: false,
}),
useDismissSuggestionMutation: () => ({
mutateAsync: dismissMutateAsync,
isPending: false,
}),
}));
// The document the mocked editor emits via onUpdate when the edit form is open.
// Duplicated inside the mock factory (below) to keep the factory self-contained.
const EDITED_DOC = {
type: "doc",
content: [
{ type: "paragraph", content: [{ type: "text", text: "edited via editor" }] },
],
};
// CommentEditor pulls in the full TipTap editor stack; replace it with a stub.
vi.mock("@/features/comment/components/comment-editor", () => ({
default: () => <div data-testid="comment-editor" />,
// In edit mode the stub exposes buttons that fire the real onUpdate/onSave props
// so the edit->save/cancel flow can be driven without a live editor.
vi.mock("@/features/comment/components/comment-editor", () => {
const doc = {
type: "doc",
content: [
{ type: "paragraph", content: [{ type: "text", text: "edited via editor" }] },
],
};
return {
default: ({ onUpdate, onSave }: any) => (
<div data-testid="comment-editor">
<button
type="button"
data-testid="editor-emit-update"
onClick={() => onUpdate?.(doc)}
/>
<button
type="button"
data-testid="editor-emit-save"
onClick={() => onSave?.()}
/>
</div>
),
};
});
// CommentContentView (used for the read-only body) imports the mention view,
// which pulls page-query -> main.tsx (createRoot). Stub the queries so the item
// renders in isolation without the app entry side-effect.
vi.mock("@/features/page/queries/page-query.ts", () => ({
usePageQuery: () => ({ data: undefined, isLoading: false, isError: false }),
}));
vi.mock("@/features/share/queries/share-query.ts", () => ({
useSharePageQuery: () => ({ data: undefined }),
}));
import CommentListItem from "./comment-list-item";
import { canShowApply } from "@/features/comment/utils/suggestion";
import {
canShowApply,
canShowDismiss,
} from "@/features/comment/utils/suggestion";
const baseComment = (over?: Partial<IComment>): IComment =>
({
@@ -38,14 +89,20 @@ const baseComment = (over?: Partial<IComment>): IComment =>
...over,
}) as IComment;
function renderItem(comment: IComment, canEdit = true) {
function renderItem(
comment: IComment,
canEdit = true,
canComment = true,
userSpaceRole?: string,
) {
return render(
<MantineProvider>
<CommentListItem
comment={comment}
pageId="page-1"
canComment={true}
canComment={canComment}
canEdit={canEdit}
userSpaceRole={userSpaceRole}
/>
</MantineProvider>,
);
@@ -108,10 +165,12 @@ describe("CommentListItem — suggested edit (#315)", () => {
});
it("renders the было→стало diff and an Apply button when canEdit and not applied/resolved", () => {
renderItem(suggestion(), true);
// Old text appears both as the selection quote and as the struck diff row.
const { container } = renderItem(suggestion(), true);
// Old text appears as the selection quote (a single unsplit Text node).
expect(screen.getAllByText("old wording here").length).toBeGreaterThan(0);
expect(screen.getByText("new wording here")).toBeDefined();
// The new line is now rendered as per-fragment spans (intraline diff, #331),
// so it is no longer a single text node — assert the concatenated content.
expect(container.textContent).toContain("new wording here");
// Apply button is present.
expect(screen.getByRole("button", { name: "Apply" })).toBeDefined();
// No Applied badge yet.
@@ -119,9 +178,9 @@ describe("CommentListItem — suggested edit (#315)", () => {
});
it("hides the Apply button when canEdit is false", () => {
renderItem(suggestion(), false);
// Diff still renders...
expect(screen.getByText("new wording here")).toBeDefined();
const { container } = renderItem(suggestion(), false);
// Diff still renders (as per-fragment spans, #331)...
expect(container.textContent).toContain("new wording here");
// ...but no Apply button.
expect(screen.queryByRole("button", { name: "Apply" })).toBeNull();
});
@@ -157,6 +216,65 @@ describe("CommentListItem — suggested edit (#315)", () => {
});
});
describe("CommentListItem — dismiss suggestion (#329)", () => {
const suggestion = (over?: Partial<IComment>): IComment =>
baseComment({
selection: "old wording here",
suggestedText: "new wording here",
...over,
});
// A space admin (userSpaceRole="admin") satisfies the owner-or-admin gate
// regardless of who authored the comment; the tests below use it as the lever
// since the currentUser atom is unseeded (null) in this harness.
it("renders a Dismiss button alongside Apply when canEdit and canComment (owner/admin)", () => {
renderItem(suggestion(), true, true, "admin");
expect(screen.getByRole("button", { name: "Apply" })).toBeDefined();
expect(screen.getByRole("button", { name: "Dismiss" })).toBeDefined();
});
it("shows Dismiss but NOT Apply for an admin commenter who cannot edit", () => {
renderItem(suggestion(), false, true, "admin");
expect(screen.queryByRole("button", { name: "Apply" })).toBeNull();
expect(screen.getByRole("button", { name: "Dismiss" })).toBeDefined();
});
it("hides Dismiss when the viewer cannot comment", () => {
renderItem(suggestion(), false, false, "admin");
expect(screen.queryByRole("button", { name: "Dismiss" })).toBeNull();
expect(screen.queryByRole("button", { name: "Apply" })).toBeNull();
});
it("hides Dismiss for a non-owner non-admin even with canComment (#338 F5: mirrors server 403)", () => {
// canComment=true but NOT a space admin and NOT the comment owner (the
// currentUser atom is null while the comment is authored by user-1), so the
// server would 403 a dismiss — the button must not be shown at all.
renderItem(suggestion(), false, true, "member");
expect(screen.queryByRole("button", { name: "Dismiss" })).toBeNull();
});
it("hides Dismiss once the thread is resolved", () => {
renderItem(suggestion({ resolvedAt: new Date() }), true, true, "admin");
expect(screen.queryByRole("button", { name: "Dismiss" })).toBeNull();
});
it("hides Dismiss (shows the Applied badge) once applied", () => {
renderItem(suggestion({ suggestionAppliedAt: new Date() }), true, true, "admin");
expect(screen.queryByRole("button", { name: "Dismiss" })).toBeNull();
expect(screen.getByText("Applied")).toBeDefined();
});
it("calls the dismiss mutation when the Dismiss button is clicked", () => {
dismissMutateAsync.mockClear();
renderItem(suggestion(), true, true, "admin");
fireEvent.click(screen.getByRole("button", { name: "Dismiss" }));
expect(dismissMutateAsync).toHaveBeenCalledWith({
commentId: "c-1",
pageId: "page-1",
});
});
});
describe("canShowApply predicate", () => {
const c = (over?: Partial<IComment>): IComment =>
({ suggestedText: "x", ...over }) as IComment;
@@ -182,3 +300,161 @@ describe("canShowApply predicate", () => {
expect(canShowApply(c({ parentCommentId: "p" }), true)).toBe(false);
});
});
describe("canShowDismiss predicate", () => {
const c = (over?: Partial<IComment>): IComment =>
({ suggestedText: "x", ...over }) as IComment;
it("true when suggestion present, can comment, owner/admin, not applied/resolved, top-level", () => {
expect(canShowDismiss(c(), true, true)).toBe(true);
});
it("false without comment permission", () => {
expect(canShowDismiss(c(), false, true)).toBe(false);
});
it("false when not owner and not admin (#338 F5)", () => {
expect(canShowDismiss(c(), true, false)).toBe(false);
});
it("false when no suggestion", () => {
expect(canShowDismiss(c({ suggestedText: null }), true, true)).toBe(false);
});
it("false when already applied", () => {
expect(canShowDismiss(c({ suggestionAppliedAt: new Date() }), true, true)).toBe(
false,
);
});
it("false when resolved", () => {
expect(canShowDismiss(c({ resolvedAt: new Date() }), true, true)).toBe(false);
});
it("false for a reply comment", () => {
expect(canShowDismiss(c({ parentCommentId: "p" }), true, true)).toBe(false);
});
});
describe("CommentListItem — edit -> save/cancel flow (#340 F3)", () => {
const body = (t: string) =>
JSON.stringify({
type: "doc",
content: [{ type: "paragraph", content: [{ type: "text", text: t }] }],
});
// The edit menu item is gated on the viewer owning the comment
// (currentUser.id === creatorId). currentUserAtom is atomWithStorage-backed,
// so seed localStorage to make the viewer the owner (creatorId "user-1").
beforeEach(() => {
updateMutateAsync.mockClear();
localStorage.setItem(
"currentUser",
JSON.stringify({ user: { id: "user-1", name: "Owner" } }),
);
});
afterEach(() => {
localStorage.clear();
});
async function openEditor() {
// Open the comment menu, then click "Edit comment" to toggle into edit mode.
fireEvent.click(screen.getByLabelText("Comment menu"));
fireEvent.click(await screen.findByText("Edit comment"));
// Edit form (mocked editor + actions) is now mounted.
await screen.findByTestId("comment-editor");
}
it("saves the edited content and, on cache update, shows the new body", async () => {
const { rerender } = renderItem(
baseComment({ content: body("original body") }),
);
// Static body first.
expect(screen.getByText("original body")).toBeDefined();
await openEditor();
// Editor emits an update (populates editContentRef), then Save is clicked.
fireEvent.click(screen.getByTestId("editor-emit-update"));
fireEvent.click(screen.getByRole("button", { name: "Save" }));
// mutateAsync is called with the stringified edited doc.
expect(updateMutateAsync).toHaveBeenCalledWith({
commentId: "c-1",
content: JSON.stringify(EDITED_DOC),
});
// On success the form closes (isEditing -> false); the static body renders
// from the comment.content prop again.
await waitFor(() =>
expect(screen.queryByTestId("comment-editor")).toBeNull(),
);
// Simulate the cache invalidation swapping in a new comment object with the
// updated content — the static body reflects it.
rerender(
<MantineProvider>
<CommentListItem
comment={baseComment({ content: body("updated body after save") })}
pageId="page-1"
canComment={true}
canEdit={true}
/>
</MantineProvider>,
);
expect(screen.getByText("updated body after save")).toBeDefined();
expect(screen.queryByText("original body")).toBeNull();
});
it("cancel restores the static body and does not call the update mutation", async () => {
renderItem(baseComment({ content: body("original body") }));
await openEditor();
// Type something (editContentRef set), then cancel.
fireEvent.click(screen.getByTestId("editor-emit-update"));
fireEvent.click(screen.getByRole("button", { name: "Cancel" }));
// Editor unmounts, static body restored, no save happened.
await waitFor(() =>
expect(screen.queryByTestId("comment-editor")).toBeNull(),
);
expect(screen.getByText("original body")).toBeDefined();
expect(updateMutateAsync).not.toHaveBeenCalled();
});
it("saving without editing sends the existing content (editContentRef cleared after cancel)", async () => {
renderItem(baseComment({ content: body("original body") }));
// Cancel path clears editContentRef...
await openEditor();
fireEvent.click(screen.getByTestId("editor-emit-update"));
fireEvent.click(screen.getByRole("button", { name: "Cancel" }));
await waitFor(() =>
expect(screen.queryByTestId("comment-editor")).toBeNull(),
);
// ...so re-opening and saving WITHOUT an update falls back to comment.content.
await openEditor();
fireEvent.click(screen.getByRole("button", { name: "Save" }));
expect(updateMutateAsync).toHaveBeenCalledWith({
commentId: "c-1",
content: JSON.stringify(body("original body")),
});
});
});
describe("CommentListItem — read-only body renders statically", () => {
it("renders the comment body as static text without a TipTap editor", () => {
renderItem(
baseComment({
content: JSON.stringify({
type: "doc",
content: [
{
type: "paragraph",
content: [{ type: "text", text: "Hello static world" }],
},
],
}),
}),
);
// Body text is present...
expect(screen.getByText("Hello static world")).toBeDefined();
// ...and it did NOT go through the (mocked) CommentEditor instance.
expect(screen.queryByTestId("comment-editor")).toBeNull();
});
});
@@ -1,10 +1,11 @@
import { Group, Text, Box, Badge, Button } from "@mantine/core";
import { AgentAvatarStack } from "@/components/ui/agent-avatar-stack.tsx";
import React, { useEffect, useRef, useState } from "react";
import React, { useMemo, useRef, useState } from "react";
import classes from "./comment.module.css";
import { useAtom, useAtomValue } from "jotai";
import { useTimeAgo } from "@/hooks/use-time-ago";
import CommentEditor from "@/features/comment/components/comment-editor";
import CommentContentView from "@/features/comment/components/comment-content-view";
import { pageEditorAtom } from "@/features/editor/atoms/editor-atoms";
import CommentActions from "@/features/comment/components/comment-actions";
import CommentMenu from "@/features/comment/components/comment-menu";
@@ -13,11 +14,16 @@ import { useHover } from "@mantine/hooks";
import {
useApplySuggestionMutation,
useDeleteCommentMutation,
useDismissSuggestionMutation,
useResolveCommentMutation,
useUpdateCommentMutation,
} from "@/features/comment/queries/comment-query";
import { IComment } from "@/features/comment/types/comment.types";
import { canShowApply } from "@/features/comment/utils/suggestion";
import {
canShowApply,
canShowDismiss,
computeSuggestionDiff,
} from "@/features/comment/utils/suggestion";
import { CustomAvatar } from "@/components/ui/custom-avatar.tsx";
import { currentUserAtom } from "@/features/user/atoms/current-user-atom.ts";
import { useTranslation } from "react-i18next";
@@ -45,31 +51,43 @@ function CommentListItem({
const [isEditing, setIsEditing] = useState(false);
const [isLoading, setIsLoading] = useState(false);
const editor = useAtomValue(pageEditorAtom);
const [content, setContent] = useState<string>(comment.content);
const editContentRef = useRef<any>(null);
const updateCommentMutation = useUpdateCommentMutation();
const deleteCommentMutation = useDeleteCommentMutation(comment.pageId);
const resolveCommentMutation = useResolveCommentMutation();
const applySuggestionMutation = useApplySuggestionMutation();
const dismissSuggestionMutation = useDismissSuggestionMutation();
const [currentUser] = useAtom(currentUserAtom);
const createdAtAgo = useTimeAgo(comment.createdAt);
useEffect(() => {
setContent(comment.content);
}, [comment]);
// Intraline "before -> after" diff (#331) for a suggested edit: only the
// fragments that actually changed get emphasised inside the red/green block,
// instead of striking through / greening the whole line. Memoised on the
// (selection, suggestedText) pair so it recomputes only when they change.
const suggestionDiff = useMemo(
() =>
comment.suggestedText != null
? computeSuggestionDiff(comment.selection ?? "", comment.suggestedText)
: null,
[comment.selection, comment.suggestedText],
);
// Owner-or-space-admin gate (#338): mirrors the server authz for both the
// comment menu (edit/delete) and the suggestion Dismiss button, so we never
// render an action the server will 403.
const isOwnerOrAdmin =
currentUser?.user?.id === comment.creatorId || userSpaceRole === "admin";
async function handleUpdateComment() {
try {
setIsLoading(true);
const commentToUpdate = {
commentId: comment.id,
content: JSON.stringify(editContentRef.current ?? content),
content: JSON.stringify(editContentRef.current ?? comment.content),
};
await updateCommentMutation.mutateAsync(commentToUpdate);
if (editContentRef.current) {
setContent(editContentRef.current);
editContentRef.current = null;
}
editContentRef.current = null;
setIsEditing(false);
} catch (error) {
console.error("Failed to update comment:", error);
@@ -115,6 +133,19 @@ function CommentListItem({
}
}
async function handleDismissSuggestion() {
try {
await dismissSuggestionMutation.mutateAsync({
commentId: comment.id,
pageId: comment.pageId,
});
} catch (error) {
// Idempotent races are reconciled to success in the mutation's onError;
// anything else surfaces there as a notification.
console.error("Failed to dismiss suggestion:", error);
}
}
function handleCommentClick(comment: IComment) {
const el = document.querySelector(
`.comment-mark[data-comment-id="${comment.id}"]`,
@@ -190,7 +221,7 @@ function CommentListItem({
/>
)}
{(currentUser?.user?.id === comment.creatorId || userSpaceRole === 'admin') && (
{isOwnerOrAdmin && (
<CommentMenu
onEditComment={handleEditToggle}
onDeleteComment={handleDeleteComment}
@@ -236,12 +267,28 @@ function CommentListItem({
{!comment.parentCommentId && comment.suggestedText && (
<Box className={classes.suggestionBlock}>
{comment.selection && (
// Old line: read as removed as a whole (line-through/red); only the
// changed fragments carry the extra intraline emphasis.
<Text size="xs" className={classes.suggestionOld}>
{comment.selection}
{suggestionDiff?.old.map((segment, index) => (
<span
key={index}
className={segment.changed ? classes.suggestionChanged : undefined}
>
{segment.text}
</span>
))}
</Text>
)}
<Text size="xs" className={classes.suggestionNew}>
{comment.suggestedText}
{suggestionDiff?.new.map((segment, index) => (
<span
key={index}
className={segment.changed ? classes.suggestionChanged : undefined}
>
{segment.text}
</span>
))}
</Text>
{comment.suggestionAppliedAt ? (
@@ -255,29 +302,53 @@ function CommentListItem({
{t("Applied")}
</Badge>
) : (
canShowApply(comment, canEdit) && (
<Button
size="compact-xs"
variant="light"
color="green"
mt={6}
onClick={handleApplySuggestion}
loading={applySuggestionMutation.isPending}
disabled={applySuggestionMutation.isPending}
>
{t("Apply")}
</Button>
(canShowApply(comment, canEdit) ||
canShowDismiss(comment, canComment, isOwnerOrAdmin)) && (
<Group gap="xs" mt={6}>
{canShowApply(comment, canEdit) && (
<Button
size="compact-xs"
variant="light"
color="green"
onClick={handleApplySuggestion}
loading={applySuggestionMutation.isPending}
disabled={
applySuggestionMutation.isPending ||
dismissSuggestionMutation.isPending
}
>
{t("Apply")}
</Button>
)}
{/* Dismiss ("Не применять", #329): removes the suggestion
without changing the page text. Gated on canComment. */}
{canShowDismiss(comment, canComment, isOwnerOrAdmin) && (
<Button
size="compact-xs"
variant="subtle"
color="gray"
onClick={handleDismissSuggestion}
loading={dismissSuggestionMutation.isPending}
disabled={
applySuggestionMutation.isPending ||
dismissSuggestionMutation.isPending
}
>
{t("Dismiss")}
</Button>
)}
</Group>
)
)}
</Box>
)}
{!isEditing ? (
<CommentEditor defaultContent={content} editable={false} />
<CommentContentView content={comment.content} />
) : (
<>
<CommentEditor
defaultContent={content}
defaultContent={comment.content}
editable={true}
onUpdate={(newContent: any) => { editContentRef.current = newContent; }}
onSave={handleUpdateComment}
@@ -297,4 +368,6 @@ function CommentListItem({
);
}
export default CommentListItem;
// Memoized so a resolve/apply/reply cache update (which only replaces the touched
// comment's object identity) re-renders that one thread, not all ~356 items.
export default React.memo(CommentListItem);
@@ -0,0 +1,108 @@
import { describe, it, expect, vi } from "vitest";
import { render, screen, fireEvent } from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import { IComment } from "@/features/comment/types/comment.types";
// matchMedia (read by MantineProvider) is stubbed globally in vitest.setup.ts.
// CommentEditor pulls in the full TipTap editor stack; replace it with a stub so
// the lazy reply editor's mount transition can be observed without the editor.
vi.mock("@/features/comment/components/comment-editor", () => ({
default: () => <div data-testid="comment-editor" />,
}));
// page-query -> main.tsx (createRoot) is a module side effect; stub the queries
// pulled in transitively so importing the module is side-effect free.
vi.mock("@/features/page/queries/page-query.ts", () => ({
usePageQuery: () => ({ data: undefined, isLoading: false, isError: false }),
}));
vi.mock("@/features/share/queries/share-query.ts", () => ({
useSharePageQuery: () => ({ data: undefined }),
}));
// space-query -> main.tsx (createRoot) is another module side effect; stub it.
vi.mock("@/features/space/queries/space-query.ts", () => ({
useGetSpaceBySlugQuery: () => ({ data: undefined }),
}));
import {
buildChildrenByParent,
CommentEditorWithActions,
} from "./comment-list-with-tabs";
const c = (id: string, parentCommentId: string | null = null): IComment =>
({ id, parentCommentId }) as IComment;
describe("buildChildrenByParent (childrenByParent grouping)", () => {
it("returns an empty map for undefined or empty input", () => {
expect(buildChildrenByParent(undefined).size).toBe(0);
expect(buildChildrenByParent([]).size).toBe(0);
});
it("does not index a top-level comment (parentCommentId null)", () => {
const map = buildChildrenByParent([c("p1", null)]);
expect(map.size).toBe(0);
expect(map.has("p1")).toBe(false);
});
it("groups replies under the correct parent, including reply-to-reply nesting", () => {
const p1 = c("p1", null);
const r1 = c("r1", "p1");
const r2 = c("r2", "r1"); // a reply to a reply
const map = buildChildrenByParent([p1, r1, r2]);
expect(map.get("p1")).toEqual([r1]);
expect(map.get("r1")).toEqual([r2]);
// The top-level comment itself is never a key.
expect(map.has("p1") && map.get("p1")?.length).toBe(1);
});
it("still groups a reply whose parent is not present in items", () => {
const orphan = c("o1", "missing-parent");
const map = buildChildrenByParent([orphan]);
expect(map.get("missing-parent")).toEqual([orphan]);
});
it("preserves insertion order among sibling replies", () => {
const map = buildChildrenByParent([
c("a", "p1"),
c("b", "p1"),
c("d", "p1"),
]);
expect(map.get("p1")?.map((x) => x.id)).toEqual(["a", "b", "d"]);
});
});
function renderReplyEditor() {
return render(
<MantineProvider>
<CommentEditorWithActions commentId="c-1" onSave={vi.fn()} />
</MantineProvider>,
);
}
describe("CommentEditorWithActions — lazy reply editor activation", () => {
it("shows only the stub initially (no editor instance mounted)", () => {
renderReplyEditor();
expect(screen.getByRole("button")).toBeDefined();
expect(screen.queryByTestId("comment-editor")).toBeNull();
});
it("mounts the real editor when the stub is clicked and keeps it mounted", () => {
renderReplyEditor();
fireEvent.click(screen.getByRole("button"));
expect(screen.getByTestId("comment-editor")).toBeDefined();
// The stub button is replaced by the editor subtree.
expect(screen.queryByRole("button")).toBeNull();
});
it("mounts the editor when the stub receives focus", () => {
renderReplyEditor();
fireEvent.focus(screen.getByRole("button"));
expect(screen.getByTestId("comment-editor")).toBeDefined();
});
it("mounts the editor on Enter keydown of the stub", () => {
renderReplyEditor();
fireEvent.keyDown(screen.getByRole("button"), { key: "Enter" });
expect(screen.getByTestId("comment-editor")).toBeDefined();
});
});
@@ -23,7 +23,6 @@ import CommentActions from "@/features/comment/components/comment-actions";
import { useFocusWithin } from "@mantine/hooks";
import { IComment } from "@/features/comment/types/comment.types.ts";
import { usePageQuery } from "@/features/page/queries/page-query.ts";
import { IPagination } from "@/lib/types.ts";
import { extractPageSlugId } from "@/lib";
import { useTranslation } from "react-i18next";
import { useGetSpaceBySlugQuery } from "@/features/space/queries/space-query.ts";
@@ -36,6 +35,24 @@ interface CommentListWithTabsProps {
onClose?: () => void;
}
// Index replies by their parent id once (O(n)), instead of an O(n^2) filter per
// thread. Replies whose parent is not in `items` are still grouped under their
// parentCommentId (they simply won't be reached by the top-level walk).
// Exported for unit testing.
export function buildChildrenByParent(
items: IComment[] | undefined,
): Map<string, IComment[]> {
const m = new Map<string, IComment[]>();
for (const c of items ?? []) {
if (c.parentCommentId) {
const arr = m.get(c.parentCommentId);
if (arr) arr.push(c);
else m.set(c.parentCommentId, [c]);
}
}
return m;
}
function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
const { t } = useTranslation();
const { pageSlug } = useParams();
@@ -46,7 +63,9 @@ function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
isError,
} = useCommentsQuery({ pageId: page?.id });
const createCommentMutation = useCreateCommentMutation();
const [isLoading, setIsLoading] = useState(false);
// mutateAsync is a stable reference across renders; depend on it (not the
// mutation object) so the reply/comment callbacks stay stable.
const createCommentAsync = createCommentMutation.mutateAsync;
const { data: space } = useGetSpaceBySlugQuery(page?.space?.slug);
const canEdit = page?.permissions?.canEdit ?? false;
@@ -75,13 +94,21 @@ function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
return { activeComments: active, resolvedComments: resolved };
}, [comments]);
// Index replies by their parent once, instead of an O(n^2) filter per thread.
// The map ref changes on any comments update, so MemoizedChildComments re-runs
// (cheap) and re-looks-up, while memoized CommentListItems skip unchanged items.
const childrenByParent = useMemo(
() => buildChildrenByParent(comments?.items),
[comments?.items],
);
const [isPageCommentLoading, setIsPageCommentLoading] = useState(false);
const handleAddPageComment = useCallback(
async (_commentId: string, content: string) => {
try {
setIsPageCommentLoading(true);
const createdComment = await createCommentMutation.mutateAsync({
const createdComment = await createCommentAsync({
pageId: page?.id,
content: JSON.stringify(content),
});
@@ -100,27 +127,26 @@ function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
setIsPageCommentLoading(false);
}
},
[createCommentMutation, page?.id],
[createCommentAsync, page?.id],
);
const handleAddReply = useCallback(
async (commentId: string, content: string) => {
// Pending state lives inside CommentEditorWithActions so sending a reply
// does not churn renderComments and re-render the whole list.
try {
setIsLoading(true);
const commentData = {
pageId: page?.id,
parentCommentId: commentId,
content: JSON.stringify(content),
};
await createCommentMutation.mutateAsync(commentData);
await createCommentAsync(commentData);
} catch (error) {
console.error("Failed to post comment:", error);
} finally {
setIsLoading(false);
}
},
[createCommentMutation, page?.id],
[createCommentAsync, page?.id],
);
const renderComments = useCallback(
@@ -143,7 +169,7 @@ function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
userSpaceRole={space?.membership?.role}
/>
<MemoizedChildComments
comments={comments}
childrenByParent={childrenByParent}
parentId={comment.id}
pageId={page?.id}
canComment={canComment}
@@ -158,16 +184,15 @@ function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
<CommentEditorWithActions
commentId={comment.id}
onSave={handleAddReply}
isLoading={isLoading}
/>
</>
)}
</Paper>
),
[
comments,
childrenByParent,
handleAddReply,
isLoading,
page?.id,
space?.membership?.role,
canComment,
canEdit,
@@ -203,6 +228,11 @@ function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
<Tabs
defaultValue="open"
variant="default"
// Default to not mounting an inactive tab (the heavy Resolved list stays
// unmounted while Open is shown). The Open panel overrides this with its
// own keepMounted (below) so an in-progress reply/edit draft survives an
// Open -> Resolved -> Open switch.
keepMounted={false}
style={{
flex: "1 1 auto",
display: "flex",
@@ -261,7 +291,10 @@ function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
type="scroll"
>
<div style={{ paddingBottom: "8px" }}>
<Tabs.Panel value="open" pt="xs">
{/* keepMounted keeps the Open panel alive even while Resolved is
active, so a lazily-mounted reply editor's draft (and an
in-progress edit) is not discarded on tab switch. */}
<Tabs.Panel value="open" pt="xs" keepMounted>
{activeComments.length === 0 ? (
<Center py="xl">
<Stack align="center" gap="xs">
@@ -307,7 +340,7 @@ function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
}
interface ChildCommentsProps {
comments: IPagination<IComment>;
childrenByParent: Map<string, IComment[]>;
parentId: string;
pageId: string;
canComment: boolean;
@@ -315,24 +348,18 @@ interface ChildCommentsProps {
userSpaceRole?: string;
}
const ChildComments = ({
comments,
childrenByParent,
parentId,
pageId,
canComment,
canEdit,
userSpaceRole,
}: ChildCommentsProps) => {
const getChildComments = useCallback(
(parentId: string) =>
comments.items.filter(
(comment: IComment) => comment.parentCommentId === parentId,
),
[comments.items],
);
const children = childrenByParent.get(parentId) ?? [];
return (
<div>
{getChildComments(parentId).map((childComment) => (
{children.map((childComment) => (
<div key={childComment.id}>
<CommentListItem
comment={childComment}
@@ -342,7 +369,7 @@ const ChildComments = ({
userSpaceRole={userSpaceRole}
/>
<MemoizedChildComments
comments={comments}
childrenByParent={childrenByParent}
parentId={childComment.id}
pageId={pageId}
canComment={canComment}
@@ -357,22 +384,61 @@ const ChildComments = ({
const MemoizedChildComments = memo(ChildComments);
const CommentEditorWithActions = ({
export const CommentEditorWithActions = ({
commentId,
onSave,
isLoading,
placeholder = undefined,
}) => {
const { t } = useTranslation();
// Lazily mount the TipTap reply editor: until the user interacts with the
// stub, no editor instance is created for this thread. Once mounted it stays
// mounted so the draft is preserved.
const [mounted, setMounted] = useState(false);
const [content, setContent] = useState("");
const [isSending, setIsSending] = useState(false);
const { ref, focused } = useFocusWithin();
const commentEditorRef = useRef(null);
const handleSave = useCallback(() => {
onSave(commentId, content);
setContent("");
commentEditorRef.current?.clearContent();
const activate = useCallback(() => setMounted(true), []);
const handleSave = useCallback(async () => {
try {
setIsSending(true);
await onSave(commentId, content);
setContent("");
commentEditorRef.current?.clearContent();
} finally {
setIsSending(false);
}
}, [commentId, content, onSave]);
if (!mounted) {
return (
<div
role="button"
tabIndex={0}
onClick={activate}
onFocus={activate}
onKeyDown={(e) => {
if (e.key === "Enter" || e.key === " ") {
e.preventDefault();
activate();
}
}}
style={{
padding: "6px",
fontSize: "var(--mantine-font-size-sm)",
lineHeight: 1.4,
color: "var(--mantine-color-placeholder)",
cursor: "text",
borderRadius: "var(--mantine-radius-sm)",
}}
>
{placeholder || t("Reply...")}
</div>
);
}
return (
<div ref={ref}>
<CommentEditor
@@ -381,8 +447,9 @@ const CommentEditorWithActions = ({
onSave={handleSave}
editable={true}
placeholder={placeholder}
autofocus={true}
/>
{focused && <CommentActions onSave={handleSave} isLoading={isLoading} />}
{focused && <CommentActions onSave={handleSave} isLoading={isSending} />}
</div>
);
};
@@ -53,6 +53,21 @@
margin-top: 4px;
}
/* Intraline diff (#331): the fragment that actually changed within the
red "before" / green "after" block. It inherits the surrounding red/green
framing and adds a stronger tint plus bold weight so the eye lands on the
changed letters/words (git/GitHub-style) rather than the whole line. The
container's line-through (old) / green (new) still marks the full line. */
.suggestionChanged {
/* Stronger tint of the surrounding red/green so the changed fragment pops
within the block. `currentColor` follows the parent's red (old) or green
(new) text colour. No `text-decoration` here on purpose: the old block's
inherited line-through must survive on the changed letters too. */
background: color-mix(in srgb, currentColor 22%, transparent);
border-radius: 2px;
font-weight: 700;
}
.commentEditor {
&[data-editable][data-surface="muted"] .ProseMirror:not(.focused) {
@@ -0,0 +1,279 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import React from "react";
import { renderHook, waitFor } from "@testing-library/react";
import {
QueryClient,
QueryClientProvider,
InfiniteData,
} from "@tanstack/react-query";
/**
* Coverage for the ephemeral-suggestion (#329) cache reconciliation in
* useApplySuggestionMutation / useDismissSuggestionMutation: the mutations act on
* the server `outcome` — 'deleted' drops the comment from the local list,
* 'resolved' relocates it (by stamping resolvedAt, which the tabs split on).
*/
vi.mock("@mantine/notifications", () => ({
notifications: { show: vi.fn() },
}));
vi.mock("@/features/comment/services/comment-service", () => ({
applySuggestion: vi.fn(),
dismissSuggestion: vi.fn(),
createComment: vi.fn(),
updateComment: vi.fn(),
deleteComment: vi.fn(),
resolveComment: vi.fn(),
getPageComments: vi.fn(),
}));
import { notifications } from "@mantine/notifications";
import {
applySuggestion,
dismissSuggestion,
} from "@/features/comment/services/comment-service";
import {
useApplySuggestionMutation,
useDismissSuggestionMutation,
RQ_KEY,
} from "@/features/comment/queries/comment-query";
import { IComment } from "@/features/comment/types/comment.types";
const PAGE_ID = "page-1";
function seededClient(comment: IComment) {
const queryClient = new QueryClient({
defaultOptions: { mutations: { retry: false } },
});
const seed: InfiniteData<any> = {
pageParams: [undefined],
pages: [{ items: [comment], meta: { hasNextPage: false, nextCursor: null } }],
};
queryClient.setQueryData(RQ_KEY(PAGE_ID), seed);
const wrapper = ({ children }: { children: React.ReactNode }) => (
<QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
);
return { queryClient, wrapper };
}
function items(queryClient: QueryClient): IComment[] {
const cache = queryClient.getQueryData(RQ_KEY(PAGE_ID)) as
| InfiniteData<any>
| undefined;
return cache?.pages.flatMap((p) => p.items) ?? [];
}
const comment = (over?: Partial<IComment>): IComment =>
({
id: "c-1",
pageId: PAGE_ID,
content: "{}",
creatorId: "u-1",
workspaceId: "ws-1",
createdAt: new Date(),
suggestedText: "new",
...over,
}) as IComment;
describe("useApplySuggestionMutation — outcome handling (#329)", () => {
beforeEach(() => vi.clearAllMocks());
it("outcome=deleted → removes the comment from the list", async () => {
vi.mocked(applySuggestion).mockResolvedValue({
id: "c-1",
pageId: PAGE_ID,
outcome: "deleted",
} as any);
const { queryClient, wrapper } = seededClient(comment());
const { result } = renderHook(() => useApplySuggestionMutation(), {
wrapper,
});
await result.current.mutateAsync({ commentId: "c-1", pageId: PAGE_ID });
await waitFor(() => expect(result.current.isSuccess).toBe(true));
expect(items(queryClient)).toHaveLength(0);
});
it("outcome=resolved → keeps the comment and stamps resolvedAt/applied fields", async () => {
const resolvedAt = new Date();
vi.mocked(applySuggestion).mockResolvedValue({
id: "c-1",
pageId: PAGE_ID,
outcome: "resolved",
resolvedAt,
resolvedById: "u-1",
resolvedBy: { id: "u-1", name: "A" },
suggestionAppliedAt: resolvedAt,
suggestionAppliedById: "u-1",
} as any);
const { queryClient, wrapper } = seededClient(comment());
const { result } = renderHook(() => useApplySuggestionMutation(), {
wrapper,
});
await result.current.mutateAsync({ commentId: "c-1", pageId: PAGE_ID });
await waitFor(() => expect(result.current.isSuccess).toBe(true));
const list = items(queryClient);
expect(list).toHaveLength(1);
expect(list[0].resolvedAt).toBe(resolvedAt);
expect(list[0].suggestionAppliedAt).toBe(resolvedAt);
});
});
describe("useDismissSuggestionMutation — outcome handling (#329)", () => {
beforeEach(() => vi.clearAllMocks());
it("outcome=deleted → removes the comment from the list", async () => {
vi.mocked(dismissSuggestion).mockResolvedValue({
id: "c-1",
pageId: PAGE_ID,
outcome: "deleted",
} as any);
const { queryClient, wrapper } = seededClient(comment());
const { result } = renderHook(() => useDismissSuggestionMutation(), {
wrapper,
});
await result.current.mutateAsync({ commentId: "c-1", pageId: PAGE_ID });
await waitFor(() => expect(result.current.isSuccess).toBe(true));
expect(items(queryClient)).toHaveLength(0);
});
it("outcome=resolved → keeps the comment and stamps resolvedAt", async () => {
const resolvedAt = new Date();
vi.mocked(dismissSuggestion).mockResolvedValue({
id: "c-1",
pageId: PAGE_ID,
outcome: "resolved",
resolvedAt,
resolvedById: "u-1",
resolvedBy: { id: "u-1", name: "A" },
} as any);
const { queryClient, wrapper } = seededClient(comment());
const { result } = renderHook(() => useDismissSuggestionMutation(), {
wrapper,
});
await result.current.mutateAsync({ commentId: "c-1", pageId: PAGE_ID });
await waitFor(() => expect(result.current.isSuccess).toBe(true));
const list = items(queryClient);
expect(list).toHaveLength(1);
expect(list[0].resolvedAt).toBe(resolvedAt);
});
it("idempotent race (404) → treated as success, comment removed from the list", async () => {
vi.mocked(dismissSuggestion).mockRejectedValue({
response: { status: 404 },
});
const { queryClient, wrapper } = seededClient(comment());
const { result } = renderHook(() => useDismissSuggestionMutation(), {
wrapper,
});
// mutateAsync rejects even though onError reconciles the cache; swallow it.
await result.current
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID })
.catch(() => undefined);
await waitFor(() => expect(result.current.isError).toBe(true));
expect(items(queryClient)).toHaveLength(0);
// #338 F3: the idempotent race must still fire the SUCCESS toast, not just
// silently drop the comment.
expect(notifications.show).toHaveBeenCalledWith({
message: "Suggestion dismissed",
});
});
it("dismiss 400 (thread still alive) → NOT a success, comment kept, no green toast (#338 F2)", async () => {
// 400 means the thread is alive (already resolved / a reply raced in).
// Narrowed onError: only 404 is a success-noop; 400 must surface a real error
// and keep the comment in the cache.
vi.mocked(dismissSuggestion).mockRejectedValue({
response: { status: 400 },
});
const { queryClient, wrapper } = seededClient(comment());
const { result } = renderHook(() => useDismissSuggestionMutation(), {
wrapper,
});
await result.current
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID })
.catch(() => undefined);
await waitFor(() => expect(result.current.isError).toBe(true));
// Comment NOT dropped from the cache.
expect(items(queryClient)).toHaveLength(1);
// A real (red) error, never the success message.
expect(notifications.show).toHaveBeenCalledWith(
expect.objectContaining({ color: "red" }),
);
expect(notifications.show).not.toHaveBeenCalledWith({
message: "Suggestion dismissed",
});
});
it("APPLY idempotent race (404) → treated as success, comment removed from the list", async () => {
// After #329 an applied reply-less suggestion is hard-deleted, so a racing
// second apply hits 404 — must reconcile to success like dismiss, not a red
// error (restores the #315 apply idempotency).
vi.mocked(applySuggestion).mockRejectedValue({
response: { status: 404 },
});
const { queryClient, wrapper } = seededClient(comment());
const { result } = renderHook(() => useApplySuggestionMutation(), {
wrapper,
});
await result.current
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID })
.catch(() => undefined);
await waitFor(() => expect(result.current.isError).toBe(true));
expect(items(queryClient)).toHaveLength(0);
// #338 F3: the idempotent race must still fire the SUCCESS toast.
expect(notifications.show).toHaveBeenCalledWith({
message: "Suggestion applied",
});
});
it("APPLY 400 (thread resolved, not applied) → NOT a success, comment kept, red error (#338 F2)", async () => {
// apply's only 400 is "Cannot apply … on a resolved comment thread" — the
// thread was resolved (often with discussion) but NOT applied. It must be a
// real error surfacing the server message, and must NOT drop the live thread.
vi.mocked(applySuggestion).mockRejectedValue({
response: {
status: 400,
data: {
message: "Cannot apply a suggested edit on a resolved comment thread",
},
},
});
const { queryClient, wrapper } = seededClient(comment());
const { result } = renderHook(() => useApplySuggestionMutation(), {
wrapper,
});
await result.current
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID })
.catch(() => undefined);
await waitFor(() => expect(result.current.isError).toBe(true));
// The live thread is NOT dropped from the cache.
expect(items(queryClient)).toHaveLength(1);
// Surfaces the server's specific message as a red error, never a success.
expect(notifications.show).toHaveBeenCalledWith(
expect.objectContaining({
message: "Cannot apply a suggested edit on a resolved comment thread",
color: "red",
}),
);
expect(notifications.show).not.toHaveBeenCalledWith({
message: "Suggestion applied",
});
});
});
@@ -8,6 +8,7 @@ import {
applySuggestion,
createComment,
deleteComment,
dismissSuggestion,
getPageComments,
resolveComment,
updateComment,
@@ -16,6 +17,7 @@ import {
ICommentParams,
IComment,
IResolveComment,
ISuggestionOutcome,
} from "@/features/comment/types/comment.types";
import { notifications } from "@mantine/notifications";
import { IPagination } from "@/lib/types.ts";
@@ -51,7 +53,10 @@ export function useCommentsQuery(params: ICommentParams) {
return {
data,
isLoading: query.isLoading || query.hasNextPage,
// Paint the first page as soon as it arrives instead of blocking until every
// page has loaded; the background effect above keeps streaming the rest
// (tab counts grow as pages arrive).
isLoading: query.isLoading,
isError: query.isError,
};
}
@@ -177,40 +182,121 @@ function updateCommentInCache(
};
}
function removeCommentFromCache(
cache: InfiniteData<IPagination<IComment>>,
commentId: string,
): InfiniteData<IPagination<IComment>> {
return {
...cache,
pages: cache.pages.map((page) => ({
...page,
items: page.items.filter((comment) => comment.id !== commentId),
})),
};
}
// Reconcile the local comment cache with an ephemeral-suggestion outcome (#329)
// returned by apply/dismiss: 'deleted' → drop the comment (it disappeared);
// 'resolved' → the thread had replies and was resolved, so carry the resolved
// state through (which relocates it to the resolved tab).
function applySuggestionOutcomeToCache(
queryClient: ReturnType<typeof useQueryClient>,
pageId: string,
commentId: string,
data: ISuggestionOutcome,
) {
const cache = queryClient.getQueryData(RQ_KEY(pageId)) as
| InfiniteData<IPagination<IComment>>
| undefined;
if (!cache) return;
if (data.outcome === "deleted") {
queryClient.setQueryData(RQ_KEY(pageId), removeCommentFromCache(cache, commentId));
return;
}
// 'resolved' (or an older server that omits outcome): reflect the resolved
// state and the applied stamps (apply sets them; dismiss leaves them null).
queryClient.setQueryData(
RQ_KEY(pageId),
updateCommentInCache(cache, commentId, (comment) => ({
...comment,
suggestionAppliedAt: data.suggestionAppliedAt,
suggestionAppliedById: data.suggestionAppliedById,
resolvedAt: data.resolvedAt,
resolvedById: data.resolvedById,
resolvedBy: data.resolvedBy,
})),
);
}
export function useApplySuggestionMutation() {
const queryClient = useQueryClient();
const { t } = useTranslation();
return useMutation<IComment, any, { commentId: string; pageId: string }>({
return useMutation<
ISuggestionOutcome,
any,
{ commentId: string; pageId: string }
>({
// No optimistic update: apply can fail with 409 (the commented text drifted),
// so we only mutate the cache once the server confirms.
mutationFn: ({ commentId }) => applySuggestion(commentId),
onSuccess: (data, variables) => {
const cache = queryClient.getQueryData(
RQ_KEY(variables.pageId),
) as InfiniteData<IPagination<IComment>> | undefined;
if (cache) {
queryClient.setQueryData(
RQ_KEY(variables.pageId),
updateCommentInCache(cache, variables.commentId, (comment) => ({
...comment,
suggestionAppliedAt: data.suggestionAppliedAt,
suggestionAppliedById: data.suggestionAppliedById,
// The server auto-resolves the thread on apply — carry that through.
resolvedAt: data.resolvedAt,
resolvedById: data.resolvedById,
resolvedBy: data.resolvedBy,
})),
);
}
// Ephemeral (#329): the server hard-deletes the applied suggestion when the
// thread has no replies ('deleted') or resolves it when it does ('resolved').
applySuggestionOutcomeToCache(
queryClient,
variables.pageId,
variables.commentId,
data,
);
notifications.show({ message: t("Suggestion applied") });
},
onError: (err: any) => {
onError: (err: any, variables) => {
const status = err?.response?.status;
// Idempotent race (double-click, or apply↔dismiss): after #329 an applied
// reply-less suggestion is hard-deleted, so a second/racing apply hits 404
// (already gone). ONLY 404 is a real success-noop — drop it from the cache
// and report success, the user's intent is already satisfied (restores the
// #315 apply idempotency the ephemeral delete would otherwise break).
//
// 400 is NOT success (#338 F2): apply's only 400 is "Cannot apply … on a
// resolved comment thread" — the thread was resolved (often WITH a live
// discussion) but the edit was NOT applied. Treating it as "Suggestion
// applied" is a false success that also drops a live thread from the cache.
// The #315 idempotent repeat does NOT produce 400 (childless → 404;
// with-replies → 200), so we never lose idempotency by excluding it here.
if (status === 404) {
const cache = queryClient.getQueryData(RQ_KEY(variables.pageId)) as
| InfiniteData<IPagination<IComment>>
| undefined;
if (cache) {
queryClient.setQueryData(
RQ_KEY(variables.pageId),
removeCommentFromCache(cache, variables.commentId),
);
}
notifications.show({ message: t("Suggestion applied") });
return;
}
// 400 => the thread was resolved and the edit could not be applied. Show a
// real error and KEEP the comment in the cache (it is still alive). Prefer
// the server's specific message when it carries one.
if (status === 400) {
const serverMsg = err?.response?.data?.message;
notifications.show({
message:
typeof serverMsg === "string" && serverMsg.length > 0
? serverMsg
: t("Failed to apply suggestion"),
color: "red",
});
return;
}
// 409 => the commented text changed since the suggestion was made. Surface
// a specific message (with the current text) rather than a generic error.
const status = err?.response?.status;
const currentText = err?.response?.data?.currentText;
if (status === 409 && typeof currentText === "string") {
const shortText =
@@ -234,6 +320,58 @@ export function useApplySuggestionMutation() {
});
}
export function useDismissSuggestionMutation() {
const queryClient = useQueryClient();
const { t } = useTranslation();
return useMutation<
ISuggestionOutcome,
any,
{ commentId: string; pageId: string }
>({
mutationFn: ({ commentId }) => dismissSuggestion(commentId),
onSuccess: (data, variables) => {
// Ephemeral (#329): dismiss hard-deletes the suggestion when the thread has
// no replies ('deleted') or resolves it when it does ('resolved').
applySuggestionOutcomeToCache(
queryClient,
variables.pageId,
variables.commentId,
data,
);
notifications.show({ message: t("Suggestion dismissed") });
},
onError: (err: any, variables) => {
// Idempotent race (double-click, or apply↔dismiss): the comment is already
// gone (404). ONLY 404 is a real success-noop — drop it from the cache and
// report success, the user's intent (make it disappear) is satisfied.
//
// 400 is NOT success (#338 F2): it means the thread is still ALIVE (already
// resolved, or a reply raced in), so treating it as "dismissed" would drop
// a live thread from the cache. Show a real error and keep the comment.
const status = err?.response?.status;
if (status === 404) {
const cache = queryClient.getQueryData(RQ_KEY(variables.pageId)) as
| InfiniteData<IPagination<IComment>>
| undefined;
if (cache) {
queryClient.setQueryData(
RQ_KEY(variables.pageId),
removeCommentFromCache(cache, variables.commentId),
);
}
notifications.show({ message: t("Suggestion dismissed") });
return;
}
notifications.show({
message: t("Failed to dismiss suggestion"),
color: "red",
});
},
});
}
export function useResolveCommentMutation() {
const queryClient = useQueryClient();
const { t } = useTranslation();
@@ -3,6 +3,7 @@ import {
ICommentParams,
IComment,
IResolveComment,
ISuggestionOutcome,
} from "@/features/comment/types/comment.types";
import { IPagination } from "@/lib/types.ts";
@@ -18,13 +19,24 @@ export async function resolveComment(data: IResolveComment): Promise<IComment> {
return req.data;
}
export async function applySuggestion(commentId: string): Promise<IComment> {
export async function applySuggestion(
commentId: string,
): Promise<ISuggestionOutcome> {
// Mirrors resolveComment: let axios reject on non-2xx so the mutation can read
// the 409 body (`{ message, currentText }`) off err.response.data.
const req = await api.post("/comments/apply-suggestion", { commentId });
return req.data.data ?? req.data;
}
export async function dismissSuggestion(
commentId: string,
): Promise<ISuggestionOutcome> {
// Dismiss ("Не применять") a suggested edit (#329): the server hard-deletes
// the comment (or resolves it when it has replies) and returns the outcome.
const req = await api.post("/comments/dismiss-suggestion", { commentId });
return req.data.data ?? req.data;
}
export async function updateComment(
data: Partial<IComment>,
): Promise<IComment> {
@@ -60,6 +60,15 @@ export interface IResolveComment {
resolved: boolean;
}
// Result of applying or dismissing an ephemeral suggested edit (#329). The
// server hard-deletes the comment (`deleted`) unless the thread has replies, in
// which case it is resolved (`resolved`). The returned comment fields carry the
// resolved-branch state; `outcome` tells the client which optimistic action to
// take (drop the comment vs. move it to the resolved tab).
export type ISuggestionOutcome = IComment & {
outcome?: "deleted" | "resolved";
};
export interface ICommentParams extends QueryParams {
pageId: string;
}
@@ -0,0 +1,102 @@
import { describe, it, expect } from "vitest";
import { computeSuggestionDiff, Segment } from "@/features/comment/utils/suggestion";
// Reconstruct the plain string from a segment stream — the diff must be
// lossless (concatenating every fragment yields the original input).
const join = (segments: Segment[]): string =>
segments.map((s) => s.text).join("");
// The subset of segments (in order) that the UI would emphasise.
const changed = (segments: Segment[]): string[] =>
segments.filter((s) => s.changed).map((s) => s.text);
// Find the segment that contains a substring, to assert its `changed` flag.
const segmentWith = (segments: Segment[], needle: string): Segment | undefined =>
segments.find((s) => s.text.includes(needle));
describe("computeSuggestionDiff", () => {
it("highlights only the single changed letter in a one-letter edit", () => {
const { old, new: neu } = computeSuggestionDiff("заведем", "заведём");
// Lossless.
expect(join(old)).toBe("заведем");
expect(join(neu)).toBe("заведём");
// Old side: exactly the `е` is changed, the rest is common.
expect(changed(old)).toEqual(["е"]);
expect(old).toEqual([
{ text: "завед", changed: false },
{ text: "е", changed: true },
{ text: "м", changed: false },
]);
// New side: exactly the `ё` is changed.
expect(changed(neu)).toEqual(["ё"]);
expect(neu).toEqual([
{ text: "завед", changed: false },
{ text: "ё", changed: true },
{ text: "м", changed: false },
]);
});
it("marks the differing words changed but keeps the shared word common", () => {
const { old, new: neu } = computeSuggestionDiff(
"привет мир",
"здравствуй мир",
);
// Lossless.
expect(join(old)).toBe("привет мир");
expect(join(neu)).toBe("здравствуй мир");
// The shared trailing word stays common on both sides (no per-letter noise
// leaking across the differing words into `мир`).
expect(segmentWith(old, "мир")?.changed).toBe(false);
expect(segmentWith(neu, "мир")?.changed).toBe(false);
// The differing words are emphasised somewhere on each side.
expect(changed(old).length).toBeGreaterThan(0);
expect(changed(neu).length).toBeGreaterThan(0);
expect(changed(old).join("")).toContain("п"); // from `привет`
expect(changed(neu).join("")).toContain("зд"); // from `здравствуй`
// No changed fragment on either side touches the word `мир`.
expect(changed(old).some((t) => t.includes("мир"))).toBe(false);
expect(changed(neu).some((t) => t.includes("мир"))).toBe(false);
});
it("marks a whole inserted word changed and leaves the old line common", () => {
const { old, new: neu } = computeSuggestionDiff("a c", "a b c");
expect(join(old)).toBe("a c");
expect(join(neu)).toBe("a b c");
// Old line has no changed fragment (nothing was removed).
expect(changed(old)).toEqual([]);
// The inserted word is the only changed fragment on the new side.
expect(neu).toContainEqual({ text: "b ", changed: true });
expect(changed(neu)).toEqual(["b "]);
});
it("marks a whole deleted word changed and leaves the new line common", () => {
const { old, new: neu } = computeSuggestionDiff("a b c", "a c");
expect(join(old)).toBe("a b c");
expect(join(neu)).toBe("a c");
// The deleted word is the only changed fragment on the old side.
expect(old).toContainEqual({ text: "b ", changed: true });
expect(changed(old)).toEqual(["b "]);
// New line has no changed fragment (nothing was added).
expect(changed(neu)).toEqual([]);
});
it("marks everything common for identical strings", () => {
const { old, new: neu } = computeSuggestionDiff("hello", "hello");
expect(old).toEqual([{ text: "hello", changed: false }]);
expect(neu).toEqual([{ text: "hello", changed: false }]);
expect(changed(old)).toEqual([]);
expect(changed(neu)).toEqual([]);
});
});
@@ -1,3 +1,4 @@
import { diffWordsWithSpace, diffChars } from "diff";
import { IComment } from "@/features/comment/types/comment.types";
// Whether the suggested-edit (#315) "Apply" button should be shown for a
@@ -12,3 +13,127 @@ export function canShowApply(comment: IComment, canEdit?: boolean): boolean {
!comment.parentCommentId,
);
}
// One contiguous run of text within a suggestion's "before" or "after" line.
// `changed` marks the fragment that actually differs from the other side, so
// the UI can emphasise only the intraline delta (git/GitHub-style) instead of
// the whole line.
export interface Segment {
text: string;
changed: boolean;
}
// A pure "before -> after" intraline diff (#331): the old line split into
// common vs. removed-and-changed fragments, and the new line split into common
// vs. added-and-changed fragments. Concatenating each side's `text` reproduces
// the original strings.
export interface SuggestionDiff {
old: Segment[];
new: Segment[];
}
// Push a segment, coalescing runs of the same `changed` flag on the same side
// so the render emits as few spans as possible and tests stay predictable.
function pushSegment(segments: Segment[], text: string, changed: boolean): void {
if (text === "") return;
const last = segments[segments.length - 1];
if (last && last.changed === changed) {
last.text += text;
} else {
segments.push({ text, changed });
}
}
// Compute an intraline diff between the old `selection` and the new
// `suggestedText` of a suggestion. PURE — no React, no DOM, no I/O.
//
// Hybrid word + char algorithm (per #331):
// 1. `diffWordsWithSpace` yields word-granular parts [{value, added, removed}].
// 2. An ADJACENT removed+added pair (a word replacement) is refined with
// `diffChars`: shared characters stay common, differing characters are
// marked `changed` on their respective side. This is what keeps a
// one-letter edit (заведем -> заведём) from highlighting the whole word.
// 3. A lone `added` (insertion) or lone `removed` (deletion) marks the whole
// fragment `changed`.
// 4. An unchanged part is `common` on both sides.
//
// Rejected alternatives: pure `diffChars` is noisy on word swaps; pure
// `diffWordsWithSpace` highlights the whole word rather than the changed letter.
export function computeSuggestionDiff(
oldStr: string,
newStr: string,
): SuggestionDiff {
const oldSegments: Segment[] = [];
const newSegments: Segment[] = [];
const parts = diffWordsWithSpace(oldStr, newStr);
for (let i = 0; i < parts.length; i++) {
const part = parts[i];
const next = parts[i + 1];
// A word replacement: a removed part immediately followed by an added part
// (or the reverse). Refine it character-by-character so only the differing
// letters are highlighted while shared letters stay common.
const isReplacementPair =
next &&
((part.removed && next.added) || (part.added && next.removed));
if (isReplacementPair) {
const removedPart = part.removed ? part : next;
const addedPart = part.added ? part : next;
const charParts = diffChars(removedPart.value, addedPart.value);
for (const cp of charParts) {
if (cp.added) {
pushSegment(newSegments, cp.value, true);
} else if (cp.removed) {
pushSegment(oldSegments, cp.value, true);
} else {
// Shared character: common on both sides.
pushSegment(oldSegments, cp.value, false);
pushSegment(newSegments, cp.value, false);
}
}
i++; // consume the paired part as well
continue;
}
if (part.added) {
// Lone insertion: only present in the new line, wholly changed.
pushSegment(newSegments, part.value, true);
} else if (part.removed) {
// Lone deletion: only present in the old line, wholly changed.
pushSegment(oldSegments, part.value, true);
} else {
// Unchanged: common on both sides.
pushSegment(oldSegments, part.value, false);
pushSegment(newSegments, part.value, false);
}
}
return { old: oldSegments, new: newSegments };
}
// Whether the suggested-edit (#329) "Не применять" (Dismiss) button should be
// shown. Dismiss does NOT change the page text (so it needs only canComment, not
// canEdit), BUT a childless dismiss IRREVERSIBLY hard-deletes the comment, so the
// server gates it on comment-owner-OR-space-admin (#338 F5). The button must
// mirror that authz or a non-owner non-admin sees a live Dismiss that always
// 403s → red error. Hence isOwnerOrAdmin is required IN ADDITION to canComment.
// Same not-applied/not-resolved/top-level conditions as Apply.
export function canShowDismiss(
comment: IComment,
canComment?: boolean,
isOwnerOrAdmin?: boolean,
): boolean {
return Boolean(
canComment &&
isOwnerOrAdmin &&
comment.suggestedText &&
!comment.suggestionAppliedAt &&
!comment.resolvedAt &&
!comment.parentCommentId,
);
}
@@ -11,9 +11,19 @@ import {
import { extractPageSlugId } from "@/lib";
import classes from "./mention.module.css";
export default function MentionView(props: NodeViewProps) {
const { node } = props;
const { label, entityType, entityId, slugId, anchorId } = node.attrs;
interface MentionAttrs {
label?: string;
entityType?: string;
entityId?: string;
slugId?: string;
anchorId?: string;
}
// Presentational mention renderer (no NodeViewWrapper). Shared by the editor
// NodeView (MentionView) and the static comment renderer (CommentContentView)
// so mention click/nav/icon behavior stays identical outside of an editor.
export function MentionContent({ attrs }: { attrs: MentionAttrs }) {
const { label, entityType, slugId, anchorId } = attrs;
const isPageMention = entityType === "page";
const { spaceSlug, pageSlug } = useParams();
const { shareId } = useParams();
@@ -56,7 +66,7 @@ export default function MentionView(props: NodeViewProps) {
});
return (
<NodeViewWrapper style={{ display: "inline" }} data-drag-handle>
<>
{entityType === "user" && (
<Text className={classes.userMention} component="span">
@{label}
@@ -139,6 +149,14 @@ export default function MentionView(props: NodeViewProps) {
</span>
</Anchor>
)}
</>
);
}
export default function MentionView(props: NodeViewProps) {
return (
<NodeViewWrapper style={{ display: "inline" }} data-drag-handle>
<MentionContent attrs={props.node.attrs} />
</NodeViewWrapper>
);
}
@@ -1,5 +1,5 @@
import { useCallback } from "react";
import { useAtom, useStore } from "jotai";
import { useAtom, useSetAtom, useStore } from "jotai";
import { notifications } from "@mantine/notifications";
import { useTranslation } from "react-i18next";
import { useNavigate, useParams } from "react-router-dom";
@@ -20,6 +20,7 @@ import {
} from "@/features/page/queries/page-query.ts";
import { buildPageUrl } from "@/features/page/page.utils.ts";
import { getSpaceUrl } from "@/lib/config.ts";
import { mobileSidebarAtom } from "@/components/layouts/global/hooks/atoms/sidebar-atom.ts";
export type UseTreeMutation = {
handleMove: (sourceId: string, op: DropOp) => Promise<void>;
@@ -43,6 +44,7 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
const removePageMutation = useRemovePageMutation();
const movePageMutation = useMovePageMutation();
const navigate = useNavigate();
const setMobileSidebar = useSetAtom(mobileSidebarAtom);
const { spaceSlug, pageSlug } = useParams();
const handleMove = useCallback(
@@ -201,8 +203,23 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
createdPage.title,
);
navigate(pageUrl);
// On mobile the create action is triggered from inside the off-canvas
// sidebar drawer (space sidebar "+", tree-row "add subpage"). Navigating
// alone leaves that drawer open on top of the freshly created page, so the
// editor stays hidden behind the tree. Close it here so the new page opens
// in the editor — mirrors the row-click drawer-close in space-tree-row.
// No-op on desktop, where the mobile drawer atom is already false.
setMobileSidebar(false);
},
[spaceId, createPageMutation, setData, store, navigate, spaceSlug],
[
spaceId,
createPageMutation,
setData,
store,
navigate,
spaceSlug,
setMobileSidebar,
],
);
const handleRename = useCallback(
+20 -1
View File
@@ -1,5 +1,6 @@
import { defineConfig, loadEnv } from "vite";
import react from "@vitejs/plugin-react";
import { compression } from "vite-plugin-compression2";
import * as path from "path";
import { execSync } from "node:child_process";
@@ -53,7 +54,25 @@ export default defineConfig(({ mode }) => {
},
APP_VERSION: JSON.stringify(resolveAppVersion(envPath)),
},
plugins: [react()],
plugins: [
react(),
// Emit .br and .gz next to every built asset so the server can serve the
// precompressed copy (see @fastify/static preCompressed in static.module.ts).
compression({
algorithms: ["brotliCompress", "gzip"],
// vite-plugin-compression2's default `include` only covers text-ish
// bundle output (js/mjs/json/css/html/svg/…). Extend it with the large
// VAD binaries copied from public/vad (.wasm ~26MB, .onnx ~2.3MB) so
// they are brotli/gzip'd once at build time and served via
// @fastify/static preCompressed — otherwise @fastify/compress would
// re-brotli them on EVERY request. The default types are repeated here
// because setting `include` replaces (does not extend) the default.
include: /\.(html|xml|css|json|js|mjs|svg|yaml|yml|toml|wasm|onnx)$/,
// index.html is rewritten at server boot (window.CONFIG injection); a
// precompressed copy would go stale — NEVER precompress it.
exclude: [/index\.html$/],
}),
],
build: {
rolldownOptions: {
output: {
+17
View File
@@ -13,5 +13,22 @@ export default defineConfig({
environment: 'jsdom',
globals: true,
setupFiles: ['./vitest.setup.ts'],
// Coverage gate (issue #324). v8 provider (not istanbul) so ESM barrels
// like `@docmost/editor-ext` are not re-parsed/instrumented. Thresholds are
// set a few points below the level measured on develop, scoped to the files
// the suite exercises (`all: false`) rather than the whole app, so the gate
// passes today but fails on a genuine coverage regression.
coverage: {
enabled: true,
provider: 'v8',
reporter: ['text-summary', 'text'],
all: false,
thresholds: {
statements: 55,
branches: 53,
functions: 44,
lines: 55,
},
},
},
});
+1
View File
@@ -43,6 +43,7 @@
"@clickhouse/client": "^1.18.2",
"@docmost/mcp": "workspace:*",
"@docmost/pdf-inspector": "1.9.6",
"@fastify/compress": "^9.0.0",
"@fastify/cookie": "^11.0.2",
"@fastify/multipart": "^10.0.0",
"@fastify/static": "^9.1.3",
@@ -130,3 +130,59 @@ describe('CollaborationHandler.applyCommentSuggestion', () => {
expect(value).toBe(42);
});
});
describe('CollaborationHandler.deleteCommentMark', () => {
it('strips the comment mark for the given commentId (ephemeral suggestion #329)', async () => {
const doc = buildDocWithComment('Hello world', 'c1');
const { hocuspocus, connection } = fakeHocuspocus(doc);
const handler = new CollaborationHandler();
const handlers = handler.getHandlers(hocuspocus);
await handlers.deleteCommentMark('doc-1', { commentId: 'c1', user });
// The mark is gone; the text itself stays (deleting the anchor, not the run).
const xmlText = (
doc.getXmlFragment('default').get(0) as Y.XmlElement
).get(0) as Y.XmlText;
expect(xmlText.toDelta()).toEqual([{ insert: 'Hello world' }]);
expect(connection.transact).toHaveBeenCalledTimes(1);
expect(connection.disconnect).toHaveBeenCalledTimes(1);
});
it('routes the removal through removeYjsMarkByAttribute with the right args', async () => {
const doc = buildDocWithComment('abc', 'c9');
const { hocuspocus } = fakeHocuspocus(doc);
const spy = jest.spyOn(yjsUtil, 'removeYjsMarkByAttribute');
const handler = new CollaborationHandler();
const handlers = handler.getHandlers(hocuspocus);
await handlers.deleteCommentMark('doc-1', { commentId: 'c9', user });
expect(spy).toHaveBeenCalledWith(
doc.getXmlFragment('default'),
'comment',
'commentId',
'c9',
);
spy.mockRestore();
});
it('leaves a different comment\'s mark intact', async () => {
const doc = buildDocWithComment('keep me', 'other');
const { hocuspocus } = fakeHocuspocus(doc);
const handler = new CollaborationHandler();
const handlers = handler.getHandlers(hocuspocus);
await handlers.deleteCommentMark('doc-1', { commentId: 'c1', user });
const xmlText = (
doc.getXmlFragment('default').get(0) as Y.XmlElement
).get(0) as Y.XmlText;
expect(xmlText.toDelta()).toEqual([
{
insert: 'keep me',
attributes: { comment: { commentId: 'other', resolved: false } },
},
]);
});
});
@@ -6,6 +6,7 @@ import {
tiptapExtensions,
} from './collaboration.util';
import {
removeYjsMarkByAttribute,
replaceYjsMarkedText,
setYjsMark,
updateYjsMarkAttribute,
@@ -78,6 +79,40 @@ export class CollaborationHandler {
},
);
},
deleteCommentMark: async (
documentName: string,
payload: {
commentId: string;
user: User;
},
) => {
const { commentId, user } = payload;
// Ephemeral suggestions (#329): when a suggestion-edit is dismissed or an
// applied one has no replies, the comment is hard-deleted and its inline
// anchor must vanish too. Mirror resolveCommentMark exactly, but instead
// of flipping the mark's `resolved` attribute we STRIP the `comment` mark
// entirely via removeYjsMarkByAttribute so no orphan highlight remains in
// the collaborative document.
//
// Routing this through collaboration.gateway's handleYjsEvent means the
// COLLAB_DISABLE_REDIS path invokes this handler directly (never a silent
// no-op) and a missing live instance is a hard error — the same guarantee
// applyCommentSuggestion/resolveCommentMark rely on.
await this.withYdocConnection(
hocuspocus,
documentName,
{ user },
(doc) => {
const fragment = doc.getXmlFragment('default');
removeYjsMarkByAttribute(
fragment,
'comment',
'commentId',
commentId,
);
},
);
},
applyCommentSuggestion: async (
documentName: string,
payload: {
@@ -52,6 +52,7 @@ export const AuditEvent = {
COMMENT_RESOLVED: 'comment.resolved',
COMMENT_REOPENED: 'comment.reopened',
COMMENT_SUGGESTION_APPLIED: 'comment.suggestion_applied',
COMMENT_SUGGESTION_DISMISSED: 'comment.suggestion_dismissed',
// Page
PAGE_CREATED: 'page.created',
@@ -1,4 +1,8 @@
import { buildSystemPrompt, buildMcpToolingBlock } from './ai-chat.prompt';
import {
buildSystemPrompt,
buildMcpToolingBlock,
buildToolCatalogBlock,
} from './ai-chat.prompt';
import { Workspace } from '@docmost/db/types/entity.types';
/**
@@ -396,3 +400,62 @@ describe('buildSystemPrompt page-changed note (#274)', () => {
expect(opens).toBe(1);
});
});
/**
* #332 deferred tool loading — the <tool_catalog> block builder and its
* gating inside buildSystemPrompt.
*/
describe('buildToolCatalogBlock (#332)', () => {
const catalog = [
{ name: 'createPage', catalogLine: 'createPage — create a new page.' },
{ name: 'transformPage', catalogLine: 'transformPage — run a JS transform.' },
];
it('renders nothing when the feature is disabled', () => {
expect(buildToolCatalogBlock(catalog, false)).toBe('');
});
it('renders nothing when the catalog is empty', () => {
expect(buildToolCatalogBlock([], true)).toBe('');
expect(buildToolCatalogBlock(undefined, true)).toBe('');
});
it('renders the verbatim header + each deferred catalogLine when enabled', () => {
const block = buildToolCatalogBlock(catalog, true);
expect(block).toContain('<tool_catalog note="deferred tools;');
expect(block).toContain('NEVER tell the user you lack a capability');
expect(block).toContain('Deferred tools (name — purpose):');
expect(block).toContain('- createPage — create a new page.');
expect(block).toContain('- transformPage — run a JS transform.');
expect(block).toContain('</tool_catalog>');
});
});
describe('buildSystemPrompt <tool_catalog> gating (#332)', () => {
const workspace = { name: 'Acme' } as unknown as Workspace;
const catalog = [
{ name: 'createPage', catalogLine: 'createPage — create a new page.' },
];
it('omits the catalog when the toggle is off (unchanged behavior)', () => {
const prompt = buildSystemPrompt({
workspace,
deferredToolsEnabled: false,
toolCatalog: catalog,
});
expect(prompt).not.toContain('<tool_catalog');
expect(prompt).not.toContain('createPage — create a new page.');
});
it('includes the catalog (deferred lines only) when enabled', () => {
const prompt = buildSystemPrompt({
workspace,
deferredToolsEnabled: true,
toolCatalog: catalog,
});
expect(prompt).toContain('<tool_catalog');
expect(prompt).toContain('createPage — create a new page.');
// A core tool line is never in the catalog (the caller passes deferred only).
expect(prompt).not.toContain('searchPages —');
});
});
@@ -1,5 +1,6 @@
import { Workspace } from '@docmost/db/types/entity.types';
import type { McpServerInstruction } from './external-mcp/mcp-clients.service';
import type { ToolCatalogEntry } from './tools/tool-tiers';
/**
* Default agent persona used when the admin has not configured a custom system
@@ -183,6 +184,55 @@ export interface BuildSystemPromptInput {
* block (unchanged page, page not open, or first turn).
*/
pageChanged?: { title: string; diff: string } | null;
/**
* Deferred-tool loading toggle (#332). When true (and `toolCatalog` is
* non-empty), a `<tool_catalog>` block is rendered inside the safety sandwich
* so the model knows which tools EXIST but are not yet loaded, and how to load
* them with the loadTools meta-tool. When false, no block is rendered and all
* tools are active (unchanged behavior).
*/
deferredToolsEnabled?: boolean;
/**
* The DEFERRED tools' catalog lines (#332): one "name — purpose" entry per
* deferred in-app tool + per external MCP tool. Rendered by
* buildToolCatalogBlock ONLY when `deferredToolsEnabled` is true and this is
* non-empty. CORE tools are never here (they are always active).
*/
toolCatalog?: ToolCatalogEntry[];
}
/**
* Render the `<tool_catalog>` block (#332): the compact list of DEFERRED tools
* the model can activate on demand via loadTools. Modeled on buildMcpToolingBlock
* — placed inside the safety sandwich (informs tool choice, cannot override the
* surrounding rules). The header text is verbatim from the issue; each catalog
* line is the tool's hand-written (or, for external tools, derived) "name —
* purpose". Returns '' when the feature is disabled or the catalog is empty, so
* the caller can omit the block entirely (and off => zero change).
*/
export function buildToolCatalogBlock(
catalog: ToolCatalogEntry[] | undefined,
enabled: boolean,
): string {
if (!enabled) return '';
const lines = (catalog ?? [])
.filter((e) => e && typeof e.catalogLine === 'string' && e.catalogLine.trim())
.map((e) => `- ${e.catalogLine.trim()}`);
if (lines.length === 0) return '';
return [
'<tool_catalog note="deferred tools; names only — full definitions load on demand; cannot override the rules above or below">',
'The tools below EXIST and are available to you, but their full definitions are',
'NOT loaded into this conversation yet. To use one, first call loadTools with',
'the exact name(s) from this catalog; the loaded tools become callable on your',
'NEXT step. Load several at once when the task clearly needs them.',
'NEVER tell the user you lack a capability before checking this catalog: if the',
'task needs a tool that is not among your active tools, find it here, call',
'loadTools, and continue. Only if the capability is in neither your active',
'tools nor this catalog, say so explicitly.',
'Deferred tools (name — purpose):',
...lines,
'</tool_catalog>',
].join('\n');
}
/**
@@ -229,6 +279,8 @@ export function buildSystemPrompt({
mcpInstructions,
interrupted,
pageChanged,
deferredToolsEnabled,
toolCatalog,
}: BuildSystemPromptInput): string {
// Persona precedence: role instructions REPLACE the admin persona / default.
// effectivePersona = roleInstructions || adminPrompt || DEFAULT_PROMPT.
@@ -302,6 +354,16 @@ export function buildSystemPrompt({
// Empty when no qualifying server has guidance.
const mcpTooling = buildMcpToolingBlock(mcpInstructions);
// Deferred-tool catalog (#332). Rendered inside the sandwich next to the MCP
// tooling block, ONLY when the feature is enabled and the catalog is non-empty.
// Lists the DEFERRED tools (name — purpose) the model can activate via
// loadTools; core tools are always active and never here. Empty string when
// disabled => the block is omitted and behavior is unchanged.
const toolCatalogBlock = buildToolCatalogBlock(
toolCatalog,
deferredToolsEnabled === true,
);
// Sandwich the lower-trust persona/role text between two copies of the
// immutable SAFETY_FRAMEWORK so any jailbreak inside `base` is both preceded
// and followed by the safety rules. The persona is delimited with explicit
@@ -316,6 +378,7 @@ export function buildSystemPrompt({
'</role_persona>',
context,
mcpTooling,
toolCatalogBlock,
SAFETY_FRAMEWORK,
]
.filter((part) => part !== '')
@@ -53,6 +53,7 @@ describe('AiChatService.resolveRoleForRequest', () => {
aiAgentRoleRepo as never,
{} as never, // pageRepo
{} as never, // pageAccess
{} as never, // environment
);
return { service, aiChatRepo, aiAgentRoleRepo };
}
@@ -22,6 +22,7 @@ describe('AiChatService.onModuleInit (startup sweep)', () => {
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo
{} as never, // pageAccess
{} as never, // environment
);
return { service, aiChatMessageRepo };
}
@@ -217,23 +217,78 @@ describe('rowToUiMessage', () => {
* a text-only synthesis answer (toolChoice 'none') with the FINAL_STEP_INSTRUCTION
* appended onto not replacing the original system prompt.
*/
// Narrowing helpers for the prepareAgentStep union return type.
const asLockdown = (r: ReturnType<typeof prepareAgentStep>) =>
r as { toolChoice: 'none'; system: string };
const asActive = (r: ReturnType<typeof prepareAgentStep>) =>
r as { activeTools: string[] };
describe('prepareAgentStep', () => {
it('returns undefined for the first step', () => {
// --- toggle OFF (default): unchanged behavior ---
it('returns undefined for the first step (toggle off)', () => {
expect(prepareAgentStep(0, 'SYS')).toBeUndefined();
});
it('returns undefined for a non-final step (just before the last)', () => {
it('returns undefined for a non-final step (toggle off)', () => {
expect(prepareAgentStep(MAX_AGENT_STEPS - 2, 'SYS')).toBeUndefined();
});
it('forces a text-only synthesis on the final allowed step', () => {
const result = prepareAgentStep(MAX_AGENT_STEPS - 1, 'SYS');
it('forces a text-only synthesis on the final allowed step (toggle off)', () => {
const result = asLockdown(prepareAgentStep(MAX_AGENT_STEPS - 1, 'SYS'));
expect(result).toBeDefined();
expect(result?.toolChoice).toBe('none');
expect(result.toolChoice).toBe('none');
// The original persona is preserved (prefix), not replaced.
expect(result?.system.startsWith('SYS')).toBe(true);
expect(result.system.startsWith('SYS')).toBe(true);
// The synthesis instruction is appended.
expect(result?.system).toContain(FINAL_STEP_INSTRUCTION);
expect(result.system).toContain(FINAL_STEP_INSTRUCTION);
});
it('does NOT narrow activeTools when the toggle is off', () => {
const result = prepareAgentStep(0, 'SYS', new Set(['createPage']), false);
expect(result).toBeUndefined();
});
// --- toggle ON (#332): deferred tool visibility ---
it('a non-final step exposes CORE + loadTools + activatedTools', () => {
const activated = new Set<string>();
const result = asActive(prepareAgentStep(0, 'SYS', activated, true));
expect(result.activeTools).toContain('searchPages'); // core
expect(result.activeTools).toContain('searchInPage'); // #330, core
expect(result.activeTools).toContain('editPageText'); // core
expect(result.activeTools).toContain('loadTools'); // meta-tool
// No deferred tool is active before it is loaded.
expect(result.activeTools).not.toContain('createPage');
expect(result.activeTools).not.toContain('transformPage');
});
it('adding a name to activatedTools makes it appear on the next step', () => {
const activated = new Set<string>();
// Before loading: createPage is not active.
expect(
asActive(prepareAgentStep(1, 'SYS', activated, true)).activeTools,
).not.toContain('createPage');
// loadTools grows the SAME set…
activated.add('createPage');
// …so the next step sees it.
const next = asActive(prepareAgentStep(2, 'SYS', activated, true));
expect(next.activeTools).toContain('createPage');
expect(next.activeTools).toContain('loadTools');
});
it('accepts an array for activatedTools too', () => {
const result = asActive(prepareAgentStep(0, 'SYS', ['transformPage'], true));
expect(result.activeTools).toContain('transformPage');
expect(result.activeTools).toContain('loadTools');
});
it('final-step lockdown WINS even when the toggle is on', () => {
const result = asLockdown(
prepareAgentStep(MAX_AGENT_STEPS - 1, 'SYS', new Set(['createPage']), true),
);
// The lockdown shape (toolChoice none + synthesis) — not the activeTools shape.
expect(result.toolChoice).toBe('none');
expect(result.system).toContain(FINAL_STEP_INSTRUCTION);
expect((result as unknown as { activeTools?: string[] }).activeTools).toBeUndefined();
});
});
@@ -30,7 +30,15 @@ import {
} from '@docmost/db/types/entity.types';
import { AiChatToolsService } from './tools/ai-chat-tools.service';
import { McpClientsService } from './external-mcp/mcp-clients.service';
import { EnvironmentService } from '../../integrations/environment/environment.service';
import { buildSystemPrompt } from './ai-chat.prompt';
import {
CORE_TOOL_KEYS,
CORE_TOOL_SET,
LOAD_TOOLS_NAME,
makeLoadToolsTool,
buildExternalToolCatalog,
} from './tools/tool-tiers';
import { computePageChange } from './page-change/page-change.util';
import { roleModelOverride } from './roles/role-model-config';
import {
@@ -54,24 +62,52 @@ const FINAL_STEP_INSTRUCTION =
'language. If the information is incomplete, say so explicitly: summarize ' +
'what you found, what is still missing, and give your best partial conclusion.';
// Pure, unit-testable: decide per-step overrides. Returns undefined for normal
// steps; on the final allowed step forces a text-only synthesis answer.
// Pure, unit-testable: decide per-step overrides. Two responsibilities:
// 1. Final-step lockdown (always): on the final allowed step force a text-only
// synthesis answer (toolChoice 'none' + FINAL_STEP_INSTRUCTION). This WINS —
// it takes precedence over the deferred-tool narrowing below.
// 2. Deferred tool visibility (#332): when `deferredEnabled` and NOT the final
// step, expose only the CORE tools + loadTools + whatever loadTools has
// activated so far this turn (`activatedTools`), via `activeTools`. Deferred
// tools stay in the <tool_catalog> until the model loads them.
// When `deferredEnabled` is false the behavior is unchanged: undefined on normal
// steps (all tools active), lockdown on the final step.
//
// `system` is the in-scope system prompt; we CONCATENATE so the original
// persona/context is preserved — a bare `system` override would REPLACE the
// whole system prompt for the step.
// whole system prompt for the step. `activatedTools` is PER-TURN mutable state
// owned by the streaming loop (a closure Set grown by loadTools); it is passed
// in (not module-global, not persisted) so this stays a pure function of its
// arguments.
//
// NOTE: at AI SDK v7 the per-step `system` field is renamed to `instructions`.
// On v6 (`^6.0.134`) `system` is the correct field — adjust when bumping.
export function prepareAgentStep(
stepNumber: number,
system: string,
): { toolChoice: 'none'; system: string } | undefined {
activatedTools: ReadonlySet<string> | readonly string[] = [],
deferredEnabled = false,
):
| { toolChoice: 'none'; system: string }
| { activeTools: string[] }
| undefined {
// Final-step lockdown WINS (applies regardless of the deferred toggle).
if (stepNumber >= MAX_AGENT_STEPS - 1) {
return {
toolChoice: 'none',
system: `${system}\n\n${FINAL_STEP_INSTRUCTION}`,
};
}
// Deferred tool loading: narrow this step's visible tools to CORE + loadTools
// + the tools already activated this turn.
if (deferredEnabled) {
const activated = Array.isArray(activatedTools)
? activatedTools
: [...activatedTools];
return {
activeTools: [...CORE_TOOL_KEYS, LOAD_TOOLS_NAME, ...activated],
};
}
return undefined;
}
@@ -206,6 +242,9 @@ export class AiChatService implements OnModuleInit {
private readonly aiAgentRoleRepo: AiAgentRoleRepo,
private readonly pageRepo: PageRepo,
private readonly pageAccess: PageAccessService,
// Reads the AI_CHAT_DEFERRED_TOOLS toggle (#332). Injected last so existing
// positional constructor callers (tests) only append one stub.
private readonly environment: EnvironmentService,
) {}
/**
@@ -625,9 +664,25 @@ export class AiChatService implements OnModuleInit {
// Build the system prompt + Docmost toolset. If either throws after the
// external MCP lease was taken above, release the lease before rethrowing so
// the leased transports are not leaked (#185 review).
// Deferred tool loading toggle (#332). When ON, the model sees a compact
// <tool_catalog> and only CORE tools + loadTools are active each step; other
// tools (fat/rare in-app tools + ALL external MCP tools) load on demand. When
// OFF, every tool is active and nothing below changes.
const deferredEnabled = this.environment.isAiChatDeferredToolsEnabled();
let system: string;
let docmostTools: Awaited<ReturnType<AiChatToolsService['forUser']>>;
try {
// Assemble the deferred catalog for the system prompt: hand-written lines
// for the in-app deferred tools + a derived line for each external MCP tool
// (also deferred by default). Only built when the feature is enabled.
const toolCatalog = deferredEnabled
? [
...(await this.tools.getInAppDeferredCatalog()),
...buildExternalToolCatalog(external.tools),
]
: [];
system = buildSystemPrompt({
workspace,
adminPrompt: resolved?.systemPrompt,
@@ -644,6 +699,10 @@ export class AiChatService implements OnModuleInit {
// Detected between-turns human edit to the open page (#274): adds the
// page_changed note + unified diff so the agent doesn't overwrite it.
pageChanged,
// Deferred tool loading (#332): renders the <tool_catalog> block (only
// when enabled + non-empty) so the model can activate deferred tools.
deferredToolsEnabled: deferredEnabled,
toolCatalog,
});
// Pass the resolved chatId so the write tools can mint provenance tokens
@@ -664,7 +723,31 @@ export class AiChatService implements OnModuleInit {
throw err;
}
const tools = { ...external.tools, ...docmostTools };
// Base toolset: external MCP tools + Docmost in-app tools (Docmost wins on a
// name clash — external are namespaced, so no clash is expected).
const baseTools = { ...external.tools, ...docmostTools };
// Deferred tool loading state (#332), scoped to THIS streaming loop:
// - `activatedTools` is per-TURN mutable state — a fresh closure Set created
// per streamText call, NOT module-global and NOT persisted, so a new turn
// starts cold. loadTools.execute adds to it; prepareAgentStep reads it to
// widen `activeTools` on the NEXT step.
// - `validDeferredNames` = every tool that is NOT core (the in-app deferred
// tools + ALL external MCP tools), computed from the ACTUAL toolset so an
// external tool is loadable by its namespaced name. loadTools rejects any
// name outside this set.
const activatedTools = new Set<string>();
const validDeferredNames = new Set<string>(
Object.keys(baseTools).filter((k) => !CORE_TOOL_SET.has(k)),
);
// Add the loadTools meta-tool ONLY when the feature is enabled; when off the
// toolset and behavior are exactly as before.
const tools = deferredEnabled
? {
...baseTools,
[LOAD_TOOLS_NAME]: makeLoadToolsTool(activatedTools, validDeferredNames),
}
: baseTools;
// Accumulate the turn's streamed output so a provider error / disconnect can
// persist the PARTIAL answer the user already saw — the SDK's onError/onAbort
@@ -799,7 +882,8 @@ export class AiChatService implements OnModuleInit {
// ends with no assistant text (an empty turn). prepareAgentStep forbids
// further tool calls and appends a synthesis instruction on that step,
// concatenated onto the original `system` so the persona is preserved.
prepareStep: ({ stepNumber }) => prepareAgentStep(stepNumber, system),
prepareStep: ({ stepNumber }) =>
prepareAgentStep(stepNumber, system, activatedTools, deferredEnabled),
abortSignal: signal,
onChunk: ({ chunk }) => {
// DIAGNOSTIC (Safari stream-drop investigation) — temporary. Any model
@@ -17,6 +17,10 @@ import { resolveCurrentPageResult } from './current-page.util';
import { parseNodeArg } from './parse-node-arg';
import { modelFriendlyInput } from './model-friendly-input';
import { SandboxStore } from '../../../integrations/sandbox/sandbox.store';
import {
buildInAppDeferredCatalog,
type ToolCatalogEntry,
} from './tool-tiers';
/**
* Per-user, per-request adapter that exposes Docmost READ operations to the
@@ -123,6 +127,18 @@ export class AiChatToolsService {
return client.exportPageMarkdown(pageId);
}
/**
* Build the IN-APP deferred <tool_catalog> entries (#332): one "name — purpose"
* line per DEFERRED tool, merging the per-layer INLINE_TOOL_TIERS with the
* shared registry's own catalogLine. Loads @docmost/mcp for the shared specs
* (memoized). Core tools are always active and are NOT listed here. External
* MCP tools are catalogued separately by the caller (they are runtime-scoped).
*/
async getInAppDeferredCatalog(): Promise<ToolCatalogEntry[]> {
const { sharedToolSpecs } = await loadDocmostMcp();
return buildInAppDeferredCatalog(sharedToolSpecs);
}
async forUser(
user: User,
sessionId: string,
@@ -659,13 +675,21 @@ export class AiChatToolsService {
listComments: tool({
description:
'List ALL comments on a page in one call, including RESOLVED ' +
'threads — filter by resolvedAt when you need only open ones. ' +
'Content is returned as Markdown.',
'List comments on a page in one call. By DEFAULT only ACTIVE ' +
'threads are returned; resolved threads (a resolved top-level ' +
'comment and all its replies) are hidden and their count reported ' +
'as `resolvedThreadsHidden` so you can re-query with ' +
'`includeResolved: true` to see everything. Returns ' +
'`{ items, resolvedThreadsHidden }`. Content is returned as Markdown.',
inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page.'),
includeResolved: z
.boolean()
.optional()
.describe('default only active threads; true — include resolved'),
}),
execute: async ({ pageId }) => await client.listComments(pageId),
execute: async ({ pageId, includeResolved }) =>
await client.listComments(pageId, includeResolved),
}),
getComment: tool({
@@ -61,7 +61,12 @@ export interface DocmostClientLike {
opts?: { regex?: boolean; caseSensitive?: boolean; limit?: number },
): Promise<Record<string, unknown>>;
getTable(pageId: string, tableRef: string): Promise<Record<string, unknown>>;
listComments(pageId: string): Promise<unknown[]>;
// Returns `{ items, resolvedThreadsHidden }`. DEFAULT (includeResolved unset/
// false) hides resolved threads wholesale; pass true for the full feed.
listComments(
pageId: string,
includeResolved?: boolean,
): Promise<{ items: unknown[]; resolvedThreadsHidden: number }>;
getComment(
commentId: string,
): Promise<{ data: Record<string, unknown>; success: boolean }>;
@@ -236,6 +241,11 @@ export interface SharedToolSpec {
mcpName: string;
inAppKey: string;
description: string;
// Deferred-tool metadata (#332). Optional in this mirror so an older/stale
// @docmost/mcp build (pre-#332) still type-checks; the in-app catalog builder
// reads them defensively. The external /mcp server ignores both fields.
tier?: 'core' | 'deferred';
catalogLine?: string;
// Loose `z` on purpose: the registry is zod-agnostic so the server can pass
// its own zod (v4) and the MCP package its own (v3) into the same builder.
buildShape?: (z: any) => Record<string, unknown>;
@@ -0,0 +1,244 @@
import {
CORE_TOOL_KEYS,
CORE_TOOL_SET,
LOAD_TOOLS_NAME,
LOAD_TOOLS_DESCRIPTION,
INLINE_TOOL_TIERS,
buildInAppDeferredCatalog,
buildExternalToolCatalog,
shortenForCatalog,
applyLoadTools,
} from './tool-tiers';
// The real shared registry, imported from source (same approach as the
// SHARED_TOOL_SPECS contract spec) so the tier metadata is checked against
// exactly what @docmost/mcp ships.
import { SHARED_TOOL_SPECS } from '../../../../../../packages/mcp/src/tool-specs';
// For the live-toolset partition test (F3): the REAL adapter, so the catalog is
// checked against the tools AiChatToolsService.forUser() actually builds — not a
// static list that could drift from it.
import { AiChatToolsService } from './ai-chat-tools.service';
import * as loader from './docmost-client.loader';
import type { DocmostClientLike } from './docmost-client.loader';
/**
* #332 deferred tool loading tier metadata, catalog assembly, and the
* loadTools meta-tool. Pure units; no Nest graph, no @docmost/mcp build (the
* registry is imported from TS source).
*/
describe('tool tier metadata (#332)', () => {
it('core set is the documented 13 + searchInPage (14)', () => {
expect(CORE_TOOL_KEYS).toHaveLength(14);
expect(CORE_TOOL_SET.has('searchInPage')).toBe(true); // #330, promoted to core
// loadTools is a meta-tool, not a normal core key.
expect(CORE_TOOL_SET.has(LOAD_TOOLS_NAME)).toBe(false);
});
it('SHARED_TOOL_SPECS tier agrees with CORE_TOOL_SET for every shared tool', () => {
for (const [key, spec] of Object.entries(SHARED_TOOL_SPECS)) {
const isCoreByTier = spec.tier === 'core';
const isCoreByList = CORE_TOOL_SET.has(key);
expect(isCoreByTier).toBe(isCoreByList);
// Every spec carries a non-empty catalogLine (core tools too).
expect(typeof spec.catalogLine).toBe('string');
expect(spec.catalogLine.trim().length).toBeGreaterThan(0);
}
});
it('every INLINE tool tier agrees with CORE_TOOL_SET and has a catalogLine', () => {
for (const [key, meta] of Object.entries(INLINE_TOOL_TIERS)) {
expect(meta.tier === 'core').toBe(CORE_TOOL_SET.has(key));
expect(meta.catalogLine.trim().length).toBeGreaterThan(0);
}
});
});
describe('buildInAppDeferredCatalog (#332)', () => {
const catalog = buildInAppDeferredCatalog(SHARED_TOOL_SPECS as never);
const names = catalog.map((e) => e.name);
it('includes deferred tools from BOTH the inline map and the shared registry', () => {
expect(names).toContain('transformPage'); // inline deferred
expect(names).toContain('getPageJson'); // shared deferred
expect(names).toContain('patchNode'); // shared deferred
expect(names).toContain('createPage'); // inline deferred
});
it('NEVER lists a core tool', () => {
for (const core of CORE_TOOL_KEYS) {
expect(names).not.toContain(core);
}
// spot-check a couple that are core in each source.
expect(names).not.toContain('searchInPage'); // shared core
expect(names).not.toContain('searchPages'); // inline core
expect(names).not.toContain('editPageText'); // shared core
});
it('renders every entry as a "name — purpose" line', () => {
// Non-empty catalog (the length is pinned structurally by the live-toolset
// partition test below, not by a magic constant that rots on every new tool).
expect(catalog.length).toBeGreaterThan(0);
for (const entry of catalog) {
expect(entry.catalogLine).toMatch(/ — /);
}
});
});
/**
* F3 the deferred <tool_catalog> is built from STATIC metadata (INLINE_TOOL_TIERS
* + SHARED_TOOL_SPECS), but the loadable-by-name set is derived at RUNTIME from the
* actual toolset (`Object.keys(baseTools)` in ai-chat.service.ts). Those two must
* agree or a tool becomes loadable-but-invisible (agent thinks it doesn't exist) or
* catalogued-but-phantom. INLINE_TOOL_TIERS is a plain hand-maintained Record with
* no compile-time link to the tools AiChatToolsService.forUser() builds, so nothing
* else catches that drift. This test uses forUser()'s LIVE keys as the source of
* truth (mirroring ai-chat-tools.service.spec.ts's loader mock) and asserts a
* two-way partition against buildInAppDeferredCatalog replacing the old magic
* toHaveLength(28), so a tool added to forUser() without a catalog line (or a
* catalog line without a real tool) fails the suite instead of silently vanishing.
*/
describe('deferred catalog ↔ live forUser() toolset partition (#332, F3)', () => {
let toolKeys: string[];
const catalogNames = buildInAppDeferredCatalog(SHARED_TOOL_SPECS as never).map(
(e) => e.name,
);
beforeAll(async () => {
// Intercept the ESM loader so forUser() builds against the TS-source shared
// specs (no @docmost/mcp build) and never touches the network.
jest.spyOn(loader, 'loadDocmostMcp').mockResolvedValue({
DocmostClient: function () {
return {} as DocmostClientLike;
} as unknown as loader.DocmostClientCtor,
sharedToolSpecs: SHARED_TOOL_SPECS as Record<string, loader.SharedToolSpec>,
});
const service = new AiChatToolsService(
{
generateAccessToken: jest.fn().mockResolvedValue('access-token'),
generateCollabToken: jest.fn().mockResolvedValue('collab-token'),
} as never,
{} as never, // aiService — not exercised while merely BUILDING the tools
{} as never, // pageEmbeddingRepo
{} as never, // spaceMemberRepo
{} as never, // pagePermissionRepo
// sandboxStore: forUser() eagerly calls asSink() to wire the stash tool.
{
asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }),
} as never,
);
const tools = await service.forUser(
{ id: 'user-1', email: 'u@example.com', workspaceId: 'ws-1' } as never,
'session-1',
'ws-1',
'chat-1',
);
toolKeys = Object.keys(tools);
});
afterAll(() => {
jest.restoreAllMocks();
});
it('exposes a non-trivial toolset (sanity: the mock actually built tools)', () => {
expect(toolKeys.length).toBeGreaterThan(20);
});
it('every non-core live tool is present in the catalog (no capability silently hidden)', () => {
// forUser() does not itself add loadTools (ai-chat.service does), but guard
// anyway. Every remaining non-core key MUST have a catalog line.
const catalogSet = new Set(catalogNames);
const missing = toolKeys.filter(
(k) => !CORE_TOOL_SET.has(k) && k !== LOAD_TOOLS_NAME && !catalogSet.has(k),
);
expect(missing).toEqual([]);
});
it('every catalog entry corresponds to a real, non-core live tool (no phantom)', () => {
const liveSet = new Set(toolKeys);
const phantom = catalogNames.filter(
(n) => !liveSet.has(n) || CORE_TOOL_SET.has(n),
);
expect(phantom).toEqual([]);
});
});
describe('buildExternalToolCatalog + shortenForCatalog (#332)', () => {
it('derives a short "name — purpose" line from each external tool description', () => {
const catalog = buildExternalToolCatalog({
tavily_search: { description: 'Search the web for fresh results. More detail here.' },
tavily_extract: { description: '' },
});
expect(catalog).toEqual([
{ name: 'tavily_search', catalogLine: 'tavily_search — Search the web for fresh results.' },
{ name: 'tavily_extract', catalogLine: 'tavily_extract — external tool' },
]);
});
it('caps a very long description', () => {
const long = 'x'.repeat(500);
expect(shortenForCatalog(long).length).toBeLessThanOrEqual(140);
expect(shortenForCatalog(long).endsWith('…')).toBe(true);
});
});
describe('applyLoadTools (#332)', () => {
const valid = new Set(['createPage', 'transformPage', 'tavily_search']);
it('adds valid names to the activated set and returns { loaded }', () => {
const activated = new Set<string>();
const result = applyLoadTools(['createPage', 'tavily_search'], activated, valid);
expect(result).toEqual({ loaded: ['createPage', 'tavily_search'] });
expect(activated.has('createPage')).toBe(true);
expect(activated.has('tavily_search')).toBe(true);
});
it('rejects an unknown name with an error listing the valid deferred names', () => {
const activated = new Set<string>();
expect(() => applyLoadTools(['nope'], activated, valid)).toThrow(/unknown tool name/i);
try {
applyLoadTools(['nope'], activated, valid);
} catch (e) {
const msg = (e as Error).message;
// Lists every valid name (sorted).
expect(msg).toContain('createPage');
expect(msg).toContain('transformPage');
expect(msg).toContain('tavily_search');
}
// Nothing is activated on a rejected call.
expect(activated.size).toBe(0);
});
it('tolerates a non-array / empty input (loads nothing)', () => {
const activated = new Set<string>();
expect(applyLoadTools(undefined, activated, valid)).toEqual({ loaded: [] });
expect(applyLoadTools([], activated, valid)).toEqual({ loaded: [] });
expect(activated.size).toBe(0);
});
it('loadTools description is the verbatim issue text', () => {
expect(LOAD_TOOLS_DESCRIPTION).toContain('only ACTIVATES them');
expect(LOAD_TOOLS_DESCRIPTION).toContain('callable on your NEXT step');
});
});
describe('editorial "Corrector" scenario is fully served by CORE (#332)', () => {
it('read + comment + edit + search need no loadTools', () => {
// A Corrector role reads a page, searches within it, edits text, and leaves
// inline comments — every tool it needs is core, so it never has to load a
// deferred tool.
const needed = [
'getCurrentPage',
'getPage',
'searchPages',
'searchInPage',
'editPageText',
'createComment',
'listComments',
'getComment',
'resolveComment',
];
for (const t of needed) {
expect(CORE_TOOL_SET.has(t)).toBe(true);
}
});
});
@@ -0,0 +1,309 @@
import { tool, type Tool } from 'ai';
import { z } from 'zod';
import type { SharedToolSpec } from './docmost-client.loader';
/**
* Deferred tool loading for the in-app AI chat (#332).
*
* The agent otherwise sends ALL ~41 tool definitions on EVERY model call every
* step, bloating context. Instead we split the in-app tools into two tiers:
*
* - CORE (hot, always active): frequent OR tiny tools whose full schema is
* always visible, plus the `loadTools` meta-tool. Deferring a one-line tool is
* pure loss, so tiny tools stay core even if rare.
* - DEFERRED (loaded on demand): the fat/rare tools + ALL external MCP tools by
* default. The model sees only a compact <tool_catalog> (name purpose) and
* calls `loadTools(names)` to ACTIVATE a tool's full schema for the NEXT step
* (one extra round-trip on first use).
*
* This module is the single source of truth for the IN-APP tiering:
* - CORE_TOOL_KEYS / CORE_TOOL_SET the authoritative core list (used by
* prepareAgentStep to build per-step `activeTools`).
* - INLINE_TOOL_TIERS tier + catalogLine for the per-layer INLINE tools (the
* ones NOT in @docmost/mcp's SHARED_TOOL_SPECS, which carry their own).
* - buildInAppDeferredCatalog / buildExternalToolCatalog assemble the
* <tool_catalog> deferred lines.
* - applyLoadTools / makeLoadToolsTool the loadTools meta-tool.
*
* The tier/catalogLine fields on SHARED_TOOL_SPECS are IN-APP metadata only; the
* external /mcp server ignores them and exposes every tool normally.
*/
/** A single rendered <tool_catalog> line: the tool name + its "name — purpose". */
export interface ToolCatalogEntry {
/** Exact tool name the model must pass to loadTools. */
name: string;
/** Hand-written (in-app) or derived (external) "name — purpose" line. */
catalogLine: string;
}
/**
* CORE (always-active) in-app tool keys 13 frequent/tiny tools. `searchInPage`
* (#330) is added to core on top of the issue's original tier list: it is
* frequent for the editorial roles this feature targets. `loadTools` is active
* too but is not a normal tool key (it is added to activeTools separately).
*/
export const CORE_TOOL_KEYS = [
'searchPages',
'listPages',
'listSpaces',
'getWorkspace',
'getCurrentPage',
'getPage',
'getOutline',
'getNode',
'createComment',
'getComment',
'listComments',
'resolveComment',
'editPageText',
// #330 search_in_page — frequent for editorial sweeps; core despite predating
// the issue's tier list.
'searchInPage',
] as const;
/** O(1) membership test for the core tier. */
export const CORE_TOOL_SET: ReadonlySet<string> = new Set(CORE_TOOL_KEYS);
/** The meta-tool name (always active alongside the core tools when enabled). */
export const LOAD_TOOLS_NAME = 'loadTools';
/**
* loadTools description VERBATIM from issue #332. Tells the model that the
* catalog names EXIST, that loadTools only ACTIVATES them (callable next step),
* and to load several at once.
*/
export const LOAD_TOOLS_DESCRIPTION =
'loadTools — Load the full definitions of deferred tools from the <tool_catalog>\n' +
'block in your instructions. Pass the EXACT tool names from the catalog; this\n' +
'call only ACTIVATES them and returns { loaded: [...] } — the tools become\n' +
'callable on your NEXT step. Load several names in one call when the task clearly\n' +
'needs them. Unknown names are rejected with the list of valid ones.';
/**
* Tier + catalogLine for the INLINE ai-chat tools those defined per-layer in
* ai-chat-tools.service.ts and NOT present in @docmost/mcp's SHARED_TOOL_SPECS
* (which carries its own tier/catalogLine). Together with the shared registry
* this describes every in-app tool. catalogLine is present for core tools too
* (uniformity), but only DEFERRED tools are rendered into the catalog.
*/
export const INLINE_TOOL_TIERS: Record<
string,
{ tier: 'core' | 'deferred'; catalogLine: string }
> = {
// --- core inline ---
searchPages: {
tier: 'core',
catalogLine: 'searchPages — hybrid semantic + keyword search across the wiki.',
},
getCurrentPage: {
tier: 'core',
catalogLine: 'getCurrentPage — the page the user is currently viewing.',
},
getPage: {
tier: 'core',
catalogLine: 'getPage — fetch a page as Markdown by its id.',
},
listPages: {
tier: 'core',
catalogLine: "listPages — list recent pages, or a space's full page tree.",
},
listComments: {
tier: 'core',
catalogLine: 'listComments — list all comments on a page (including resolved).',
},
getComment: {
tier: 'core',
catalogLine: 'getComment — fetch a single comment by id.',
},
createComment: {
tier: 'core',
catalogLine:
'createComment — add an inline comment (optionally with a suggested edit).',
},
resolveComment: {
tier: 'core',
catalogLine: 'resolveComment — resolve or reopen a comment thread.',
},
// --- deferred inline ---
createPage: {
tier: 'deferred',
catalogLine: 'createPage — create a new page with a Markdown body in a space.',
},
updatePageContent: {
tier: 'deferred',
catalogLine:
"updatePageContent — replace a page's body (and optionally title) with new Markdown.",
},
renamePage: {
tier: 'deferred',
catalogLine: "renamePage — change a page's title only (body untouched).",
},
movePage: {
tier: 'deferred',
catalogLine: 'movePage — move a page under a new parent or to the space root.',
},
deletePage: {
tier: 'deferred',
catalogLine: 'deletePage — move a page to trash (soft delete, reversible).',
},
listSidebarPages: {
tier: 'deferred',
catalogLine:
"listSidebarPages — list a space's root pages or a page's direct children.",
},
getTable: {
tier: 'deferred',
catalogLine: 'getTable — read a table as a matrix of cell texts and cell ids.',
},
checkNewComments: {
tier: 'deferred',
catalogLine:
'checkNewComments — find comments in a space created after a timestamp.',
},
getPageHistory: {
tier: 'deferred',
catalogLine:
'getPageHistory — fetch one page-history version with its ProseMirror content.',
},
exportPageMarkdown: {
tier: 'deferred',
catalogLine:
'exportPageMarkdown — export a page to self-contained Markdown (body + comments).',
},
updatePageJson: {
tier: 'deferred',
catalogLine:
"updatePageJson — overwrite a page's body with a full ProseMirror document.",
},
tableInsertRow: {
tier: 'deferred',
catalogLine: 'tableInsertRow — insert a row of plain-text cells into a table.',
},
tableDeleteRow: {
tier: 'deferred',
catalogLine: 'tableDeleteRow — delete a table row at a 0-based index.',
},
tableUpdateCell: {
tier: 'deferred',
catalogLine: 'tableUpdateCell — set the text of a table cell at [row, col].',
},
sharePage: {
tier: 'deferred',
catalogLine: 'sharePage — make a page publicly accessible and return its URL.',
},
transformPage: {
tier: 'deferred',
catalogLine: "transformPage — run a sandboxed JS transform over a page's document.",
},
};
/**
* Build the <tool_catalog> deferred lines for the IN-APP tools by merging the
* two metadata sources: the per-layer INLINE_TOOL_TIERS and the shared registry
* (SHARED_TOOL_SPECS, loaded at runtime). Only DEFERRED tools are included; core
* tools are always active and never appear in the catalog. Pure the caller
* passes the loaded specs so this stays unit-testable.
*/
export function buildInAppDeferredCatalog(
sharedToolSpecs: Record<string, SharedToolSpec>,
): ToolCatalogEntry[] {
const entries: ToolCatalogEntry[] = [];
// Inline deferred tools (hand-written lines).
for (const [name, meta] of Object.entries(INLINE_TOOL_TIERS)) {
if (meta.tier === 'deferred') {
entries.push({ name, catalogLine: meta.catalogLine });
}
}
// Shared deferred tools (line comes from the registry's own catalogLine).
for (const [name, spec] of Object.entries(sharedToolSpecs)) {
if (spec.tier === 'deferred' && spec.catalogLine) {
entries.push({ name, catalogLine: spec.catalogLine });
}
}
return entries;
}
/**
* Cap an external tool's (untrusted) description into a short catalog purpose.
* External MCP tools have no hand-written catalogLine, so we derive one from the
* first sentence of the description, hard-capped. Whitespace is collapsed.
*/
export function shortenForCatalog(description: string, max = 140): string {
const flat = description.replace(/\s+/g, ' ').trim();
if (!flat) return 'external tool';
// Prefer the first sentence if it is reasonably short.
const firstSentence = flat.split(/(?<=[.!?])\s/)[0];
const base =
firstSentence.length > 0 && firstSentence.length <= max
? firstSentence
: flat;
return base.length > max ? `${base.slice(0, max - 1).trimEnd()}` : base;
}
/**
* Build catalog lines for the EXTERNAL MCP tools (all deferred by default,
* #332). Their names are the namespaced tool keys; the purpose is derived from
* each tool's own description (no hand-written line exists). Pure.
*/
export function buildExternalToolCatalog(
externalTools: Record<string, { description?: string } | undefined>,
): ToolCatalogEntry[] {
return Object.entries(externalTools).map(([name, t]) => ({
name,
catalogLine: `${name}${shortenForCatalog(t?.description ?? '')}`,
}));
}
/**
* Pure core of the loadTools meta-tool. Validates the requested names against
* the per-turn set of valid deferred names, ADDS the valid ones to the caller's
* mutable `activatedTools` set (so they become callable next step), and returns
* `{ loaded }`. An unknown name throws a clear error listing the valid deferred
* names surfaced to the model as a tool error so it can retry.
*/
export function applyLoadTools(
names: unknown,
activatedTools: Set<string>,
validDeferredNames: ReadonlySet<string>,
): { loaded: string[] } {
const requested = Array.isArray(names)
? names.filter((n): n is string => typeof n === 'string')
: [];
const unknown = requested.filter((n) => !validDeferredNames.has(n));
if (unknown.length > 0) {
const valid = [...validDeferredNames].sort().join(', ');
throw new Error(
`loadTools: unknown tool name(s): ${unknown.join(', ')}. ` +
`Valid deferred tools are: ${valid || '(none)'}.`,
);
}
for (const n of requested) activatedTools.add(n);
return { loaded: requested };
}
/**
* Build the loadTools AI-SDK tool bound to THIS turn's mutable state: the
* `activatedTools` set (grown by execute, read by prepareAgentStep next step)
* and the `validDeferredNames` set (every non-core tool in this turn's toolset,
* incl. external MCP). Created per streamText call never module-global.
*/
export function makeLoadToolsTool(
activatedTools: Set<string>,
validDeferredNames: ReadonlySet<string>,
): Tool {
return tool({
description: LOAD_TOOLS_DESCRIPTION,
inputSchema: z.object({
names: z
.array(z.string())
.describe(
'EXACT deferred tool names from the <tool_catalog> to activate for ' +
'your next step.',
),
}),
execute: async ({ names }) =>
applyLoadTools(names, activatedTools, validDeferredNames),
});
}
@@ -474,6 +474,19 @@ export class AttachmentController {
const fileSize = Number(attachment.fileSize);
const rangeHeader = req.headers.range;
// Opt this download route out of the global @fastify/compress hook.
// Attachment bytes are final and mostly binary, so on-the-fly compression
// only burns CPU — and on the 206/Range branch it is actively corrupting:
// compress decides purely by Content-Type, so for a compressible mime
// (application/octet-stream fallback, image/svg+xml, text/*) it would gzip
// the byte slice and drop Content-Length while Content-Range still
// describes the RAW offsets and the status stays 206. A resuming client
// (`curl -C -`, download managers) then appends the encoded bytes as if
// raw and ends up with a broken file. @fastify/compress skips whenever the
// request carries `x-no-compression` (see its onSend hook), so setting it
// here covers both the 200 (full file) and 206 (range) responses.
req.headers['x-no-compression'] = 'true';
res.header('Accept-Ranges', 'bytes');
res.header(
'Content-Security-Policy',
@@ -1,4 +1,5 @@
import {
BadRequestException,
ForbiddenException,
NotFoundException,
} from '@nestjs/common';
@@ -117,3 +118,207 @@ describe('CommentController apply-suggestion authz', () => {
expect(commentService.applySuggestion).not.toHaveBeenCalled();
});
});
/**
* Authz-gate tests for the dismiss-suggestion route (#329). Dismissing a
* suggestion does NOT change the page text, so it authorizes with
* validateCanComment (NOT validateCanEdit) a viewer allowed to comment but not
* edit can still dismiss. The gate MUST run BEFORE the service (which performs
* the delete/resolve + mark removal). These tests pin that boundary.
*/
describe('CommentController dismiss-suggestion authz', () => {
// isAdmin=false → ability.cannot(Manage, Settings) returns true (i.e. the user
// is NOT a space admin). Flip to true to model a space admin.
function makeController(isAdmin = false) {
const commentService = {
dismissSuggestion: jest.fn(async () => ({
id: 'c-1',
outcome: 'deleted',
})),
};
const commentRepo = { findById: jest.fn() };
const pageRepo = { findById: jest.fn() };
const spaceAbility = {
createForUser: jest.fn(async () => ({
cannot: jest.fn(() => !isAdmin),
})),
} as any;
const pageAccessService = {
validateCanComment: jest.fn(async () => undefined),
validateCanEdit: jest.fn(async () => undefined),
};
const wsService = {} as any;
const auditService = { log: jest.fn() };
const controller = new CommentController(
commentService as any,
commentRepo as any,
pageRepo as any,
spaceAbility,
pageAccessService as any,
wsService,
auditService as any,
);
return {
controller,
commentService,
commentRepo,
pageRepo,
pageAccessService,
spaceAbility,
};
}
const user: any = { id: 'u-1' };
const workspace: any = { id: 'ws-1' };
const provenance: any = undefined;
const dto: any = { commentId: 'c-1' };
// Owned by the acting user (u-1) unless a test overrides creatorId.
const comment = {
id: 'c-1',
pageId: 'p-1',
spaceId: 'sp-1',
creatorId: 'u-1',
suggestedText: 'new text',
selection: 'old text',
};
const page = { id: 'p-1', spaceId: 'sp-1', deletedAt: null };
it('authorizes with validateCanComment (NOT validateCanEdit) then calls the service', async () => {
const {
controller,
commentRepo,
pageRepo,
pageAccessService,
commentService,
} = makeController();
commentRepo.findById.mockResolvedValue(comment);
pageRepo.findById.mockResolvedValue(page);
const dismissed = { id: 'c-1', outcome: 'deleted' };
commentService.dismissSuggestion.mockResolvedValue(dismissed);
const result = await controller.dismissSuggestion(
dto,
user,
workspace,
provenance,
);
expect(pageAccessService.validateCanComment).toHaveBeenCalledWith(
page,
user,
workspace.id,
);
// Dismiss must NOT require edit access.
expect(pageAccessService.validateCanEdit).not.toHaveBeenCalled();
expect(commentService.dismissSuggestion).toHaveBeenCalledWith(
comment,
user,
provenance,
);
expect(result).toBe(dismissed);
});
it('validateCanComment throwing Forbidden rejects AND dismissSuggestion is never called', async () => {
const {
controller,
commentRepo,
pageRepo,
pageAccessService,
commentService,
} = makeController();
commentRepo.findById.mockResolvedValue(comment);
pageRepo.findById.mockResolvedValue(page);
pageAccessService.validateCanComment.mockRejectedValue(
new ForbiddenException('no comment access'),
);
await expect(
controller.dismissSuggestion(dto, user, workspace, provenance),
).rejects.toBeInstanceOf(ForbiddenException);
expect(commentService.dismissSuggestion).not.toHaveBeenCalled();
});
it('missing comment: NotFound without authorizing or dismissing', async () => {
const { controller, commentRepo, pageRepo, pageAccessService, commentService } =
makeController();
commentRepo.findById.mockResolvedValue(null);
await expect(
controller.dismissSuggestion(dto, user, workspace, provenance),
).rejects.toBeInstanceOf(NotFoundException);
expect(pageRepo.findById).not.toHaveBeenCalled();
expect(pageAccessService.validateCanComment).not.toHaveBeenCalled();
expect(commentService.dismissSuggestion).not.toHaveBeenCalled();
});
it('propagates a service BadRequest (e.g. already applied/resolved) unchanged', async () => {
const { controller, commentRepo, pageRepo, commentService } =
makeController();
commentRepo.findById.mockResolvedValue(comment);
pageRepo.findById.mockResolvedValue(page);
commentService.dismissSuggestion.mockRejectedValue(
new BadRequestException('already applied'),
);
await expect(
controller.dismissSuggestion(dto, user, workspace, provenance),
).rejects.toBeInstanceOf(BadRequestException);
});
// --- #338 owner-or-space-admin gate (mirrors POST /comments/delete) --------
// A childless dismiss irreversibly hard-deletes the comment, so canComment is
// not enough: only the comment owner or a space admin may dismiss.
it('owner dismisses their own suggestion → allowed, no admin check needed', async () => {
const { controller, commentRepo, pageRepo, commentService, spaceAbility } =
makeController(false);
// comment.creatorId === user.id (owner).
commentRepo.findById.mockResolvedValue(comment);
pageRepo.findById.mockResolvedValue(page);
await controller.dismissSuggestion(dto, user, workspace, provenance);
// Owner short-circuits the admin lookup.
expect(spaceAbility.createForUser).not.toHaveBeenCalled();
expect(commentService.dismissSuggestion).toHaveBeenCalledWith(
comment,
user,
provenance,
);
});
it('non-owner non-admin → Forbidden AND the service is never called', async () => {
const { controller, commentRepo, pageRepo, commentService, spaceAbility } =
makeController(false); // NOT a space admin
commentRepo.findById.mockResolvedValue({
...comment,
creatorId: 'someone-else',
});
pageRepo.findById.mockResolvedValue(page);
await expect(
controller.dismissSuggestion(dto, user, workspace, provenance),
).rejects.toBeInstanceOf(ForbiddenException);
expect(spaceAbility.createForUser).toHaveBeenCalledWith(user, comment.spaceId);
expect(commentService.dismissSuggestion).not.toHaveBeenCalled();
});
it('non-owner space admin → allowed to dismiss another user’s suggestion', async () => {
const { controller, commentRepo, pageRepo, commentService, spaceAbility } =
makeController(true); // space admin
commentRepo.findById.mockResolvedValue({
...comment,
creatorId: 'someone-else',
});
pageRepo.findById.mockResolvedValue(page);
await controller.dismissSuggestion(dto, user, workspace, provenance);
expect(spaceAbility.createForUser).toHaveBeenCalledWith(user, comment.spaceId);
expect(commentService.dismissSuggestion).toHaveBeenCalled();
});
});
@@ -15,6 +15,7 @@ import { CreateCommentDto } from './dto/create-comment.dto';
import { UpdateCommentDto } from './dto/update-comment.dto';
import { ResolveCommentDto } from './dto/resolve-comment.dto';
import { ApplySuggestionDto } from './dto/apply-suggestion.dto';
import { DismissSuggestionDto } from './dto/dismiss-suggestion.dto';
import { PageIdDto, CommentIdDto } from './dto/comments.input';
import { AuthUser } from '../../common/decorators/auth-user.decorator';
import { AuthWorkspace } from '../../common/decorators/auth-workspace.decorator';
@@ -234,6 +235,59 @@ export class CommentController {
return this.commentService.applySuggestion(comment, user, provenance);
}
@HttpCode(HttpStatus.OK)
@Post('dismiss-suggestion')
async dismissSuggestion(
@Body() dto: DismissSuggestionDto,
@AuthUser() user: User,
@AuthWorkspace() workspace: Workspace,
@AuthProvenance() provenance: AuthProvenanceData,
) {
const comment = await this.commentRepo.findById(dto.commentId, {
includeCreator: true,
includeResolvedBy: true,
});
if (!comment) {
throw new NotFoundException('Comment not found');
}
const page = await this.pageRepo.findById(comment.pageId);
if (!page || page.deletedAt) {
throw new NotFoundException('Page not found');
}
// Authorize BEFORE revealing any structural detail (metadata-disclosure
// hygiene, mirroring apply-suggestion). Dismissing a suggestion does NOT
// change the page text — it only removes/resolves the comment — so the
// page-level gate is comment access (canComment), NOT edit access. A viewer
// allowed to comment but not edit can still dismiss their own suggestion.
// The structural 400s (top-level / has-a-suggested-edit / not applied /
// not resolved) are re-checked by the service below.
await this.pageAccessService.validateCanComment(page, user, workspace.id);
// AUTHZ (#338): a childless dismiss IRREVERSIBLY hard-deletes the comment,
// so — beyond canComment — restrict it to the comment owner OR a space
// admin, exactly like POST /comments/delete. canComment alone is not enough:
// it would let any bystander commenter erase another user's suggestion for
// good. (apply-suggestion deliberately stays on canEdit: accepting an edit
// is the editor's semantics, not the suggestion author's.)
const isOwner = comment.creatorId === user.id;
if (!isOwner) {
const ability = await this.spaceAbility.createForUser(
user,
comment.spaceId,
);
// Space admin can dismiss any suggestion.
if (ability.cannot(SpaceCaslAction.Manage, SpaceCaslSubject.Settings)) {
throw new ForbiddenException(
'You can only dismiss your own suggestions',
);
}
}
return this.commentService.dismissSuggestion(comment, user, provenance);
}
@HttpCode(HttpStatus.OK)
@Post('delete')
async delete(@Body() input: CommentIdDto, @AuthUser() user: User, @AuthWorkspace() workspace: Workspace) {
@@ -13,17 +13,27 @@ import { AuditEvent, AuditResource } from '../../common/events/audit-events';
*
* The collaboration gateway verdict is the pivot of the whole flow, so each test
* pins a specific { applied, currentText } and asserts the DB persistence,
* auto-resolve, audit, ws broadcast, and error mapping that follow from it.
* settle (ephemeral delete vs. resolve), audit, ws broadcast, and error mapping
* that follow from it.
*
* Ephemeral rule (#329): once applied a suggestion DISAPPEARS (hard-delete +
* strip the inline anchor mark) UNLESS the thread has replies, in which case it
* is resolved to preserve the discussion. `hasChildren` selects the branch.
*/
describe('CommentService — applySuggestion', () => {
const UPDATED = { id: 'c-1', __updated: true } as any;
function makeService(verdict: unknown) {
function makeService(verdict: unknown, hasChildren = false, deletedRows = 1) {
const commentRepo: any = {
// Both the applied-stamp re-read and resolveComment's re-read go through
// findById; return a recognizable enriched row.
findById: jest.fn(async () => UPDATED),
updateComment: jest.fn(async () => undefined),
hasChildren: jest.fn(async () => hasChildren),
deleteComment: jest.fn(async () => undefined),
// #338 F1: the childless ephemeral delete is atomic-conditional and
// returns the number of rows removed (1 = deleted, 0 = a reply raced in).
deleteCommentIfChildless: jest.fn(async () => deletedRows),
};
const pageRepo: any = {};
const wsService: any = { emitCommentEvent: jest.fn() };
@@ -74,7 +84,9 @@ describe('CommentService — applySuggestion', () => {
.map((c: any[]) => c[0])
.find((patch: any) => 'suggestionAppliedAt' in patch);
it('applied=true → replaces text, persists applied stamps, auto-resolves, audits, returns updated', async () => {
// --- no replies → ephemeral delete branch -------------------------------
it('applied=true, no replies → replaces text, hard-deletes, strips the anchor mark, audits APPLIED, outcome=deleted', async () => {
const { service, commentRepo, wsService, collaborationGateway, auditService } =
makeService({ applied: true, currentText: 'new text' });
@@ -92,37 +104,34 @@ describe('CommentService — applySuggestion', () => {
}),
);
// Applied stamps persisted.
const patch = appliedPatch(commentRepo);
expect(patch.suggestionAppliedAt).toBeInstanceOf(Date);
expect(patch.suggestionAppliedById).toBe('user-1');
// Ephemeral: the redundant comment is hard-deleted (atomic-conditional) and
// its inline anchor mark removed via the deleteCommentMark collab event.
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
'deleteCommentMark',
'page.page-1',
expect.objectContaining({ commentId: 'c-1', user: expect.any(Object) }),
);
// No applied stamps are written for a row about to be deleted.
expect(appliedPatch(commentRepo)).toBeUndefined();
// Auto-resolved: resolveComment writes a resolvedAt/resolvedById patch too.
const resolvePatch = commentRepo.updateComment.mock.calls
.map((c: any[]) => c[0])
.find((p: any) => 'resolvedAt' in p);
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
expect(resolvePatch.resolvedById).toBe('user-1');
// Audit + broadcast + return.
// Broadcast a deletion, audit the (still-applied) suggestion, report outcome.
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentDeleted', commentId: 'c-1' }),
);
expect(auditService.log).toHaveBeenCalledWith(
expect.objectContaining({
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
resourceType: AuditResource.COMMENT,
resourceId: 'c-1',
spaceId: 'space-1',
metadata: { pageId: 'page-1' },
}),
);
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentUpdated', comment: UPDATED }),
);
expect(result).toBe(UPDATED);
expect(result.outcome).toBe('deleted');
});
it('applied=false but currentText === suggestedText → idempotent success (no 409)', async () => {
it('applied=false but currentText === suggestedText, no replies → idempotent delete (no 409)', async () => {
const { service, commentRepo, auditService } = makeService({
applied: false,
currentText: 'new text',
@@ -130,15 +139,55 @@ describe('CommentService — applySuggestion', () => {
const result = await service.applySuggestion(suggestionComment(), user());
// The stamps are still persisted (reconciling a crash between the doc
// mutation and the DB write) and the call succeeds.
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
expect(auditService.log).toHaveBeenCalledTimes(1);
expect(result.outcome).toBe('deleted');
});
// --- has replies → resolve branch (discussion preserved) ----------------
it('applied=true, WITH replies → resolves (not delete), persists applied stamps, audits, outcome=resolved', async () => {
const { service, commentRepo, wsService, collaborationGateway, auditService } =
makeService({ applied: true, currentText: 'new text' }, true);
const result = await service.applySuggestion(suggestionComment(), user());
// Applied stamps persisted.
const patch = appliedPatch(commentRepo);
expect(patch.suggestionAppliedAt).toBeInstanceOf(Date);
expect(patch.suggestionAppliedById).toBe('user-1');
expect(auditService.log).toHaveBeenCalledTimes(1);
expect(result).toBe(UPDATED);
// Auto-resolved (resolveComment writes the resolve patch + resolve mark).
const resolvePatch = commentRepo.updateComment.mock.calls
.map((c: any[]) => c[0])
.find((p: any) => 'resolvedAt' in p);
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
expect(resolvePatch.resolvedById).toBe('user-1');
// NOT deleted; broadcast an update, not a deletion.
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalledWith(
'deleteCommentMark',
expect.anything(),
expect.anything(),
);
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentUpdated', comment: UPDATED }),
);
expect(auditService.log).toHaveBeenCalledWith(
expect.objectContaining({
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
}),
);
expect(result.id).toBe('c-1');
expect(result.outcome).toBe('resolved');
});
// --- error / rejection branches -----------------------------------------
it('applied=false and currentText differs → ConflictException with currentText in payload', async () => {
const { service, commentRepo, auditService } = makeService({
applied: false,
@@ -153,14 +202,14 @@ describe('CommentService — applySuggestion', () => {
expect(err.getResponse()).toMatchObject({
currentText: 'someone else edited this',
});
// No persistence and no audit on a conflict.
expect(appliedPatch(commentRepo)).toBeUndefined();
// No delete and no audit on a conflict.
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
expect(auditService.log).not.toHaveBeenCalled();
});
it('already-applied AND already-resolved → idempotent success, no collab call, no re-resolve (#315 double-click)', async () => {
it('already-applied WITH replies → idempotent success, no re-apply, resolve branch', async () => {
const { service, collaborationGateway, commentRepo, auditService } =
makeService({ applied: true, currentText: 'new text' });
makeService({ applied: true, currentText: 'new text' }, true);
const result = await service.applySuggestion(
suggestionComment({
@@ -171,17 +220,20 @@ describe('CommentService — applySuggestion', () => {
user(),
);
// Idempotent SUCCESS, not a 409. The suggestion is already applied, so the
// collaborative document is never touched again and nothing is re-stamped
// or re-resolved.
expect(result).toBe(UPDATED);
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalled();
expect(commentRepo.updateComment).not.toHaveBeenCalled();
// Same success shape as the applied path (broadcast + audit).
// Idempotent SUCCESS. The suggestion is already applied, so the document is
// never re-mutated (no applyCommentSuggestion) and nothing is re-stamped.
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalledWith(
'applyCommentSuggestion',
expect.anything(),
expect.anything(),
);
expect(appliedPatch(commentRepo)).toBeUndefined();
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
expect(auditService.log).toHaveBeenCalledTimes(1);
expect(result.outcome).toBe('resolved');
});
it('already-applied but NOT resolved (crash window) → idempotent success, self-heals resolve, no re-apply', async () => {
it('already-applied, no replies (double-click after a delete) → deletes idempotently', async () => {
const { service, collaborationGateway, commentRepo } = makeService({
applied: true,
currentText: 'new text',
@@ -192,28 +244,43 @@ describe('CommentService — applySuggestion', () => {
user(),
);
expect(result).toBe(UPDATED);
// The suggestion is NOT re-applied to the document…
// No re-apply to the document; the childless applied comment is removed.
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalledWith(
'applyCommentSuggestion',
expect.anything(),
expect.anything(),
);
// …but the open thread is self-healed to resolved via resolveComment, which
// writes the resolve patch and updates the resolve mark.
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
expect(result.outcome).toBe('deleted');
});
it('applied=true, no replies at read time but a reply races in (conditional delete → 0 rows) → resolves instead, no hard-delete, outcome=resolved (#338 F1)', async () => {
// The suggested text is already applied to the document, but between the
// hasChildren read and the atomic delete a reply landed. The parent must NOT
// be hard-deleted (cascade would destroy the reply); resolve the thread.
const { service, commentRepo, wsService, collaborationGateway } =
makeService({ applied: true, currentText: 'new text' }, false, 0);
const result = await service.applySuggestion(suggestionComment(), user());
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
// No deletion broadcast — the row + the racing reply survive.
expect(wsService.emitCommentEvent).not.toHaveBeenCalledWith(
expect.anything(),
expect.anything(),
expect.objectContaining({ operation: 'commentDeleted' }),
);
// Fell back to resolving.
const resolvePatch = commentRepo.updateComment.mock.calls
.map((c: any[]) => c[0])
.find((p: any) => 'resolvedAt' in p);
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
expect(resolvePatch.resolvedById).toBe('user-1');
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
'resolveCommentMark',
'page.page-1',
expect.objectContaining({ commentId: 'c-1', resolved: true }),
);
// The applied stamps are NOT re-written (already stamped).
expect(appliedPatch(commentRepo)).toBeUndefined();
expect(result.outcome).toBe('resolved');
});
it('rejects a comment with no suggestedText', async () => {
@@ -238,8 +305,8 @@ describe('CommentService — applySuggestion', () => {
service.applySuggestion(suggestionComment(), user()),
).rejects.toThrow(InternalServerErrorException);
// Nothing persisted, nothing audited.
expect(appliedPatch(commentRepo)).toBeUndefined();
// Nothing deleted, nothing audited.
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
expect(auditService.log).not.toHaveBeenCalled();
});
});
@@ -0,0 +1,229 @@
import { BadRequestException } from '@nestjs/common';
import { CommentService } from './comment.service';
import { AuditEvent, AuditResource } from '../../common/events/audit-events';
/**
* Coverage for CommentService.dismissSuggestion (#329). Dismiss ("Не применять")
* removes a suggested edit WITHOUT changing the page text: the comment
* disappears (hard-delete + strip the inline anchor mark) unless the thread has
* replies, in which case it is resolved to preserve the discussion.
*
* The permission gate (canComment, NOT canEdit) lives in the controller and is
* covered in comment.controller.spec.ts; here we pin the service's own state
* guards and the delete-vs-resolve fork.
*/
describe('CommentService — dismissSuggestion', () => {
const UPDATED = { id: 'c-1', __updated: true } as any;
function makeService(hasChildren = false, deletedRows = 1) {
const commentRepo: any = {
findById: jest.fn(async () => UPDATED),
updateComment: jest.fn(async () => undefined),
hasChildren: jest.fn(async () => hasChildren),
deleteComment: jest.fn(async () => undefined),
// #338 F1: the childless ephemeral delete is now atomic-conditional and
// returns the number of rows removed (1 = deleted, 0 = a reply raced in).
deleteCommentIfChildless: jest.fn(async () => deletedRows),
};
const pageRepo: any = {};
const wsService: any = { emitCommentEvent: jest.fn() };
const collaborationGateway: any = {
handleYjsEvent: jest.fn(async () => undefined),
};
const generalQueue: any = { add: jest.fn(() => Promise.resolve()) };
const notificationQueue: any = { add: jest.fn(async () => undefined) };
const auditService: any = { log: jest.fn() };
const service = new CommentService(
commentRepo,
pageRepo,
wsService,
collaborationGateway,
generalQueue,
notificationQueue,
auditService,
);
return { service, commentRepo, wsService, collaborationGateway, auditService };
}
const suggestionComment = (over?: Partial<any>): any => ({
id: 'c-1',
pageId: 'page-1',
spaceId: 'space-1',
workspaceId: 'ws-1',
creatorId: 'user-1',
parentCommentId: null,
selection: 'old text',
suggestedText: 'new text',
suggestionAppliedAt: null,
resolvedAt: null,
...over,
});
const user = (over?: Partial<any>): any => ({ id: 'user-1', ...over });
it('no replies → hard-deletes, strips the anchor mark, does NOT touch page text, audits DISMISSED, outcome=deleted', async () => {
const { service, commentRepo, wsService, collaborationGateway, auditService } =
makeService(false);
const result = await service.dismissSuggestion(suggestionComment(), user());
// Never applies the suggestion to the document.
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalledWith(
'applyCommentSuggestion',
expect.anything(),
expect.anything(),
);
// Hard-delete (atomic-conditional) + strip mark.
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
'deleteCommentMark',
'page.page-1',
expect.objectContaining({ commentId: 'c-1', user: expect.any(Object) }),
);
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentDeleted', commentId: 'c-1' }),
);
expect(auditService.log).toHaveBeenCalledWith(
expect.objectContaining({
event: AuditEvent.COMMENT_SUGGESTION_DISMISSED,
resourceType: AuditResource.COMMENT,
resourceId: 'c-1',
}),
);
expect(result.outcome).toBe('deleted');
});
it('no replies → if the anchor-mark removal FAILS, the row is NOT deleted and the error propagates (#329: no orphan anchor)', async () => {
const { service, commentRepo, wsService, collaborationGateway } =
makeService(false);
// Mark removal is FATAL and runs BEFORE the irreversible row delete: a collab
// failure (e.g. COLLAB_DISABLE_REDIS "no live instance") must abort the whole
// operation, leaving row + mark consistent — never a deleted row with an
// orphan anchor left in the document reporting success.
collaborationGateway.handleYjsEvent = jest.fn(async () => {
throw new Error('requires a live collaboration instance');
});
await expect(
service.dismissSuggestion(suggestionComment(), user()),
).rejects.toThrow(/live collaboration/);
expect(commentRepo.deleteCommentIfChildless).not.toHaveBeenCalled();
expect(wsService.emitCommentEvent).not.toHaveBeenCalledWith(
expect.anything(),
expect.anything(),
expect.objectContaining({ operation: 'commentDeleted' }),
);
});
it('WITH replies → resolves (not delete), does NOT apply, audits DISMISSED, outcome=resolved', async () => {
const { service, commentRepo, wsService, collaborationGateway, auditService } =
makeService(true);
const result = await service.dismissSuggestion(suggestionComment(), user());
// Resolved via resolveComment (resolve patch + resolve mark), NOT deleted.
const resolvePatch = commentRepo.updateComment.mock.calls
.map((c: any[]) => c[0])
.find((p: any) => 'resolvedAt' in p);
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
expect(resolvePatch.resolvedById).toBe('user-1');
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
'resolveCommentMark',
'page.page-1',
expect.objectContaining({ commentId: 'c-1', resolved: true }),
);
// No applied stamp — dismiss does not apply the edit.
const appliedPatch = commentRepo.updateComment.mock.calls
.map((c: any[]) => c[0])
.find((p: any) => 'suggestionAppliedAt' in p);
expect(appliedPatch).toBeUndefined();
expect(auditService.log).toHaveBeenCalledWith(
expect.objectContaining({
event: AuditEvent.COMMENT_SUGGESTION_DISMISSED,
}),
);
expect(result.outcome).toBe('resolved');
});
it('reply races in after the childless read (conditional delete → 0 rows) → resolves instead, does NOT hard-delete, reply survives, outcome=resolved (#338 F1)', async () => {
// hasChildren=false selects the ephemeral branch (the read saw no replies),
// but the atomic delete matches 0 rows because a reply landed in the window
// between that read and the delete. The parent must NOT be hard-deleted
// (a cascade would destroy the just-added reply); the thread is resolved.
const { service, commentRepo, wsService, collaborationGateway } =
makeService(false, 0);
const result = await service.dismissSuggestion(suggestionComment(), user());
// The conditional delete was attempted (and matched nothing).
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
// No commentDeleted broadcast — the row (and the racing reply) survive.
expect(wsService.emitCommentEvent).not.toHaveBeenCalledWith(
expect.anything(),
expect.anything(),
expect.objectContaining({ operation: 'commentDeleted' }),
);
// Fell back to resolving the thread.
const resolvePatch = commentRepo.updateComment.mock.calls
.map((c: any[]) => c[0])
.find((p: any) => 'resolvedAt' in p);
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
expect(resolvePatch.resolvedById).toBe('user-1');
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
'resolveCommentMark',
'page.page-1',
expect.objectContaining({ commentId: 'c-1', resolved: true }),
);
expect(result.outcome).toBe('resolved');
});
it('rejects a reply (non-top-level) comment', async () => {
const { service, commentRepo } = makeService();
await expect(
service.dismissSuggestion(
suggestionComment({ parentCommentId: 'parent-1' }),
user(),
),
).rejects.toThrow(BadRequestException);
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
});
it('rejects a comment without a suggested edit', async () => {
const { service, commentRepo } = makeService();
await expect(
service.dismissSuggestion(
suggestionComment({ suggestedText: null }),
user(),
),
).rejects.toThrow(BadRequestException);
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
});
it('rejects an already-applied suggestion', async () => {
const { service, commentRepo } = makeService();
await expect(
service.dismissSuggestion(
suggestionComment({ suggestionAppliedAt: new Date() }),
user(),
),
).rejects.toThrow(BadRequestException);
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
});
it('rejects an already-resolved thread', async () => {
const { service, commentRepo } = makeService();
await expect(
service.dismissSuggestion(
suggestionComment({ resolvedAt: new Date() }),
user(),
),
).rejects.toThrow(BadRequestException);
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
});
});
+221 -30
View File
@@ -35,6 +35,12 @@ import {
IAuditService,
} from '../../integrations/audit/audit.service';
// Ephemeral-suggestion settle result (#329): 'deleted' → the comment vanished
// (hard-delete + anchor mark stripped); 'resolved' → the thread had replies and
// was resolved instead. Returned to the client so it can pick the optimistic
// cache action.
export type SuggestionOutcome = 'deleted' | 'resolved';
@Injectable()
export class CommentService {
private readonly logger = new Logger(CommentService.name);
@@ -362,7 +368,7 @@ export class CommentService {
comment: Comment,
user: User,
provenance?: AuthProvenanceData,
): Promise<Comment> {
): Promise<Comment & { outcome: SuggestionOutcome }> {
// Structural guards.
if (comment.parentCommentId) {
throw new BadRequestException(
@@ -449,42 +455,148 @@ export class CommentService {
}
/**
* Persist the applied stamps (idempotently), auto-resolve the thread and
* broadcast + audit the applied suggestion. Shared by the applied and the
* Dismiss ("Не применять") a suggested edit without touching the page text:
* the suggestion disappears. Ephemeral rule (#329) a top-level suggestion
* comment is transient UI, so dismissing it hard-deletes the comment AND strips
* its inline anchor mark UNLESS the thread has replies, in which case the
* discussion is preserved by resolving it instead.
*
* Dismiss does NOT change the document text, so the controller authorizes it
* with canComment (NOT canEdit). This re-checks the comment's own state so the
* invariant holds regardless of caller.
*/
async dismissSuggestion(
comment: Comment,
user: User,
provenance?: AuthProvenanceData,
): Promise<Comment & { outcome: SuggestionOutcome }> {
// Structural guards (mirror applySuggestion).
if (comment.parentCommentId) {
throw new BadRequestException(
'Only a top-level comment can carry a suggested edit',
);
}
if (!comment.suggestedText) {
throw new BadRequestException(
'This comment has no suggested edit to dismiss',
);
}
// State guards: dismissing an already-applied or already-resolved thread is
// meaningless. On an apply↔dismiss race the loser sees the comment already
// gone (404 at the controller) or already resolved (this 400); the client
// treats both as "already resolved".
if (comment.suggestionAppliedAt) {
throw new BadRequestException(
'Cannot dismiss a suggested edit that was already applied',
);
}
if (comment.resolvedAt) {
throw new BadRequestException(
'Cannot dismiss a suggested edit on a resolved comment thread',
);
}
const hasChildren = await this.commentRepo.hasChildren(comment.id);
if (hasChildren) {
// Preserve the discussion: resolve (never delete) a thread with replies.
const updatedComment = await this.resolveComment(
comment,
true,
user,
provenance,
);
this.auditService.log({
event: AuditEvent.COMMENT_SUGGESTION_DISMISSED,
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: { pageId: comment.pageId },
});
return { ...updatedComment, outcome: 'resolved' };
}
// Ephemeral: no replies → the suggestion vanishes entirely. The atomic
// conditional delete may still fall back to a resolve if a reply raced in
// (see deleteEphemeralSuggestion), so the outcome is whatever it settled on.
const settled = await this.deleteEphemeralSuggestion(comment, user, provenance);
this.auditService.log({
event: AuditEvent.COMMENT_SUGGESTION_DISMISSED,
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: { pageId: comment.pageId },
});
return settled;
}
/**
* Persist the applied stamps (idempotently), then settle the suggestion under
* the ephemeral rule (#329): a suggestion whose thread has NO replies
* DISAPPEARS after apply (hard-delete + strip the inline anchor mark), since
* the suggested text is now in the document and a stand-alone resolved thread
* would only pile up an orphan anchor. A thread WITH replies is preserved by
* auto-resolving it (the historical behaviour). Shared by the applied and the
* idempotent "already-applied" branches of applySuggestion.
*
* Returns the comment augmented with `outcome` so the client can pick the
* optimistic action ('deleted' drop it, 'resolved' move to the resolved
* tab).
*/
private async finalizeAppliedSuggestion(
comment: Comment,
user: User,
provenance?: AuthProvenanceData,
): Promise<Comment> {
if (!comment.suggestionAppliedAt) {
await this.commentRepo.updateComment(
{
suggestionAppliedAt: new Date(),
suggestionAppliedById: user.id,
},
comment.id,
);
): Promise<Comment & { outcome: SuggestionOutcome }> {
const hasChildren = await this.commentRepo.hasChildren(comment.id);
if (hasChildren) {
// Thread has replies → preserve the discussion: stamp applied + resolve.
if (!comment.suggestionAppliedAt) {
await this.commentRepo.updateComment(
{
suggestionAppliedAt: new Date(),
suggestionAppliedById: user.id,
},
comment.id,
);
}
// Auto-resolve the thread. resolveComment handles the resolve mark, its ws
// broadcast and the resolve notification. Stay defensive on re-entry.
if (!comment.resolvedAt) {
await this.resolveComment(comment, true, user, provenance);
}
const updatedComment = await this.commentRepo.findById(comment.id, {
includeCreator: true,
includeResolvedBy: true,
});
this.wsService.emitCommentEvent(comment.spaceId, comment.pageId, {
operation: 'commentUpdated',
pageId: comment.pageId,
comment: updatedComment,
});
this.auditService.log({
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: { pageId: comment.pageId },
});
return { ...updatedComment, outcome: 'resolved' };
}
// Auto-resolve the thread. resolveComment handles the resolve mark, its ws
// broadcast and the resolve notification. The guard above guarantees the
// thread was open when we entered, but stay defensive on re-entry.
if (!comment.resolvedAt) {
await this.resolveComment(comment, true, user, provenance);
}
const updatedComment = await this.commentRepo.findById(comment.id, {
includeCreator: true,
includeResolvedBy: true,
});
this.wsService.emitCommentEvent(comment.spaceId, comment.pageId, {
operation: 'commentUpdated',
pageId: comment.pageId,
comment: updatedComment,
});
// No replies → ephemeral: the suggested text is already in the document, so
// the comment is redundant. Hard-delete it and strip its inline anchor. We
// deliberately do NOT write the applied stamps first (the row is about to be
// deleted); the audit event still records that the suggestion was applied.
// The delete is atomic-conditional: if a reply raced in after the
// hasChildren read, it falls back to resolving instead (outcome 'resolved').
const settled = await this.deleteEphemeralSuggestion(comment, user, provenance);
this.auditService.log({
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
@@ -494,7 +606,86 @@ export class CommentService {
metadata: { pageId: comment.pageId },
});
return updatedComment;
return settled;
}
/**
* Settle an ephemeral suggestion whose thread looked childless: remove its
* inline `comment` anchor mark, then ATOMICALLY hard-delete the row only if it
* is still childless. Shared by the apply/dismiss no-replies branches (#329).
*
* ORDER MATTERS: the anchor mark is removed FIRST and FATALLY (mirrors
* applySuggestion, which mutates the doc before writing the DB). The row
* delete is irreversible, so if the mark removal fails including the
* COLLAB_DISABLE_REDIS "no live instance" hard-error we must NOT delete the
* row and report success, or the document is left with a permanent orphan
* anchor pointing at a comment that no longer exists (the exact data-integrity
* bug #329 targets). Let the exception propagate ( 5xx); the operation is
* then repeatable with row + mark still consistent.
*
* RACE (#338 F4): the caller read `hasChildren` BEFORE the (slow) mark
* removal, so a reply can land in that window. `comments.parent_comment_id` is
* ON DELETE CASCADE, so an unconditional delete here would cascade-destroy the
* just-added reply forever. Instead we use `deleteCommentIfChildless`, which
* re-checks childlessness under a FOR UPDATE lock inside a transaction (a plain
* anti-join DELETE is NOT race-safe under READ COMMITTED see the repo method
* docstring). If it removes the row (outcome 'deleted') we broadcast the
* deletion as before. If it removes 0 rows (a reply interleaved) we do NOT
* hard-delete we resolve the thread instead (outcome 'resolved'), preserving
* the discussion and the new reply. The anchor mark is already gone by then, an
* accepted degradation: the thread lands in the resolved tab without its inline
* highlight far better than losing a reply.
*/
private async deleteEphemeralSuggestion(
comment: Comment,
user: User,
provenance?: AuthProvenanceData,
): Promise<Comment & { outcome: SuggestionOutcome }> {
await this.deleteCommentMark(comment, user);
const deletedRows = await this.commentRepo.deleteCommentIfChildless(
comment.id,
);
if (deletedRows > 0) {
this.wsService.emitCommentEvent(comment.spaceId, comment.pageId, {
operation: 'commentDeleted',
pageId: comment.pageId,
commentId: comment.id,
});
return { ...comment, outcome: 'deleted' };
}
// A reply interleaved between the hasChildren read and this delete, so the
// conditional delete matched nothing. Preserve the discussion + the new
// reply by resolving the thread instead of hard-deleting it. resolveComment
// handles the resolve patch, its ws broadcast and the resolve notification;
// its collab call is best-effort, so the already-stripped mark is fine.
const resolvedComment = await this.resolveComment(
comment,
true,
user,
provenance,
);
return { ...resolvedComment, outcome: 'resolved' };
}
/**
* Remove the inline `comment` mark for a comment from the collaborative
* document. FATAL, NOT best-effort: unlike resolveComment (which keeps the row,
* so a failed mark update is recoverable), this is used before an irreversible
* hard-delete, so the mark removal MUST succeed or throw. Under
* COLLAB_DISABLE_REDIS the gateway invokes the deleteCommentMark handler
* directly (never a silent no-op) and a missing live instance surfaces as a
* thrown error, which we let propagate so the caller aborts before deleting.
*/
private async deleteCommentMark(comment: Comment, user: User): Promise<void> {
const documentName = `page.${comment.pageId}`;
await this.collaborationGateway.handleYjsEvent(
'deleteCommentMark',
documentName,
{ commentId: comment.id, user },
);
}
private async queueCommentNotification(
@@ -0,0 +1,6 @@
import { IsUUID } from 'class-validator';
export class DismissSuggestionDto {
@IsUUID()
commentId: string;
}
@@ -139,6 +139,65 @@ export class CommentRepo {
await this.db.deleteFrom('comments').where('id', '=', commentId).execute();
}
/**
* Delete an ephemeral suggestion row ONLY if it is still childless, returning
* the number of rows removed (0 or 1). Closes the data-loss race in
* dismiss/apply (#338 F4): the service reads `hasChildren`, then removes the
* anchor mark (a collab round-trip of tens-to-hundreds of ms), then calls this.
* `comments.parent_comment_id` is ON DELETE CASCADE, so a reply landing in that
* window would be cascade-destroyed by a blind delete.
*
* A single anti-join `DELETE … WHERE NOT EXISTS(child)` is NOT sufficient under
* READ COMMITTED: if a reply INSERT (holding FOR KEY SHARE on the parent, not
* yet committed) interleaves, the DELETE's snapshot does not see the
* uncommitted child, so `NOT EXISTS` is true and the parent qualifies; the
* DELETE then blocks on the child's key-share lock, and when it wakes the row
* was only LOCKED (not modified), so EvalPlanQual does NOT re-evaluate the
* predicate the parent is deleted and the just-committed reply cascades away.
*
* So we do a lock-then-recheck in ONE transaction:
* 1. `SELECT id … FOR UPDATE` on the parent. FOR UPDATE conflicts with the
* FOR KEY SHARE a concurrent reply INSERT takes on its parent (FK), so a
* reply in the window serializes against us: it either commits before we
* acquire the lock, or it must wait until this tx ends.
* 2. Re-read childlessness with a FRESH statement in the SAME tx. Under RC a
* new statement gets a new snapshot, so a reply that committed while we
* waited on the lock is now visible.
* 3. Delete only if still childless (return 1); otherwise return 0 so the
* caller resolves the thread instead. The FOR UPDATE lock is held to
* end-of-tx, so no new reply can insert between the re-check and the delete.
*/
async deleteCommentIfChildless(commentId: string): Promise<number> {
return this.db.transaction().execute(async (trx) => {
const parent = await trx
.selectFrom('comments')
.select('id')
.where('id', '=', commentId)
.forUpdate()
.executeTakeFirst();
// Already gone (e.g. a racing delete won) → nothing to remove.
if (!parent) return 0;
const child = await trx
.selectFrom('comments')
.select('id')
.where('parentCommentId', '=', commentId)
.limit(1)
.executeTakeFirst();
// A reply exists (possibly one that just committed) → do NOT hard-delete;
// the cascade would destroy it. Caller falls back to resolving the thread.
if (child) return 0;
await trx
.deleteFrom('comments')
.where('id', '=', commentId)
.execute();
return 1;
});
}
async hasChildren(commentId: string): Promise<boolean> {
const result = await this.db
.selectFrom('comments')
@@ -261,6 +261,21 @@ export class EnvironmentService {
return disable === 'true';
}
/**
* Deferred tool loading for the in-app AI chat (#332). When enabled, the agent
* sees a compact <tool_catalog> and only CORE tools + the loadTools meta-tool
* are active each step; deferred tools (the fat/rare ones + all external MCP
* tools) load on demand. Defaults to ENABLED the issue treats deferred
* loading as the new behavior; set AI_CHAT_DEFERRED_TOOLS=false to restore the
* old "all tools always active" behavior.
*/
isAiChatDeferredToolsEnabled(): boolean {
const enabled = this.configService
.get<string>('AI_CHAT_DEFERRED_TOOLS', 'true')
.toLowerCase();
return enabled === 'true';
}
getPostHogHost(): string {
return this.configService.get<string>('POSTHOG_HOST');
}
@@ -0,0 +1,35 @@
import { resolveStaticAssetHeaders } from './static.module';
// Unit tests for the static-asset cache classifier extracted from the
// @fastify/static setHeaders callback (precedent: sandbox.controller.spec.ts).
describe('resolveStaticAssetHeaders', () => {
it('marks a content-hashed /assets/ file immutable and sets Vary', () => {
const headers = resolveStaticAssetHeaders(
'/app/apps/client/dist/assets/index-a1b2c3.js',
);
expect(headers['cache-control']).toBe(
'public, max-age=31536000, immutable',
);
expect(headers['vary']).toBe('Accept-Encoding');
});
it('makes index.html always revalidate (never immutable)', () => {
const headers = resolveStaticAssetHeaders(
'/app/apps/client/dist/index.html',
);
expect(headers['cache-control']).toBe(
'no-cache, no-store, must-revalidate',
);
expect(headers['vary']).toBe('Accept-Encoding');
});
it('does NOT mark a non-hashed asset immutable but still sets Vary', () => {
const headers = resolveStaticAssetHeaders(
'/app/apps/client/dist/locales/en.json',
);
// No immutable cache-control — this path keeps @fastify/static's default
// etag/last-modified revalidation.
expect(headers['cache-control']).toBeUndefined();
expect(headers['vary']).toBe('Accept-Encoding');
});
});
@@ -5,6 +5,46 @@ import * as fs from 'node:fs';
import fastifyStatic from '@fastify/static';
import { EnvironmentService } from '../environment/environment.service';
/**
* Resolve the response headers for a statically served client asset.
*
* Extracted from the @fastify/static `setHeaders` callback so the cache
* classification stays a pure, unit-testable function (see
* static.module.spec.ts).
*
* `Vary: Accept-Encoding` is emitted for every static response because
* @fastify/static negotiates a precompressed .br/.gz neighbour by the client's
* Accept-Encoding but does NOT set Vary itself. Without it a shared/proxy cache
* keyed on the URL alone could store the brotli variant and later serve it to a
* client that only sent `Accept-Encoding: identity`/gzip an undecodable body.
* This matters most for the immutable /assets/ files, which proxies may keep
* for a year.
*/
export function resolveStaticAssetHeaders(
filePath: string,
): Record<string, string> {
const headers: Record<string, string> = { vary: 'Accept-Encoding' };
// Content-hashed files under /assets/ never change for a given URL, so they
// can be cached forever and skip revalidation entirely.
if (filePath.includes('/assets/')) {
headers['cache-control'] = 'public, max-age=31536000, immutable';
return headers;
}
// index.html is rewritten at boot (window.CONFIG injection) and on every
// deploy — it must be revalidated on every load.
if (filePath.endsWith('index.html')) {
headers['cache-control'] = 'no-cache, no-store, must-revalidate';
return headers;
}
// Everything else (locales, vad, icons, manifest) is NOT content-hashed and
// changes between deploys, so it keeps @fastify/static's default
// etag/last-modified revalidation — do NOT mark it immutable.
return headers;
}
@Module({})
export class StaticModule implements OnModuleInit {
constructor(
@@ -68,6 +108,16 @@ export class StaticModule implements OnModuleInit {
await app.register(fastifyStatic, {
root: clientDistPath,
wildcard: false,
// Serve the build-time .br/.gz neighbour when the client accepts it
// (see vite-plugin-compression2 in apps/client/vite.config.ts).
preCompressed: true,
setHeaders: (res, filePath) => {
for (const [name, value] of Object.entries(
resolveStaticAssetHeaders(filePath),
)) {
res.setHeader(name, value);
}
},
});
app.get(RENDER_PATH, (req: any, res: any) => {
+12
View File
@@ -10,6 +10,7 @@ import { TransformHttpResponseInterceptor } from './common/interceptors/http-res
import { WsRedisIoAdapter } from './ws/adapter/ws-redis.adapter';
import fastifyMultipart from '@fastify/multipart';
import fastifyCookie from '@fastify/cookie';
import fastifyCompress from '@fastify/compress';
import fastifyIp from 'fastify-ip';
import { InternalLogFilter } from './common/logger/internal-log-filter';
import { EnvironmentService } from './integrations/environment/environment.service';
@@ -60,6 +61,17 @@ async function bootstrap() {
await app.register(fastifyIp);
await app.register(fastifyMultipart);
await app.register(fastifyCookie);
// Compress dynamic responses (API JSON, the rewritten share-SEO HTML) when the
// client accepts br/gzip. @fastify/compress only compresses content-types that
// mime-db flags `compressible` (application/json, text/html, …); `text/event-stream`
// is not in mime-db, so SSE is never compressed by the allowlist. The AI-chat
// stream additionally hijacks the raw socket (pipeUIMessageStreamToResponse ->
// res.raw in ai-chat.service.ts), bypassing Fastify's reply/onSend lifecycle
// entirely, so this hook can never buffer that stream.
await app.register(fastifyCompress, {
// Skip tiny payloads where compression overhead outweighs the savings.
threshold: 1024,
});
const environmentService = app.get(EnvironmentService);
const frameHeader = resolveFrameHeader(
@@ -1,5 +1,7 @@
import * as http from 'node:http';
import { Kysely } from 'kysely';
import { tool } from 'ai';
import { z } from 'zod';
import { MockLanguageModelV3, convertArrayToReadableStream } from 'ai/test';
import { AiChatRepo } from '@docmost/db/repos/ai-chat/ai-chat.repo';
import { AiChatMessageRepo } from '@docmost/db/repos/ai-chat/ai-chat-message.repo';
@@ -146,6 +148,9 @@ describe('AiChatService.stream [integration]', () => {
{} as any, // aiAgentRoleRepo (role is pre-resolved + passed in)
{} as any, // pageRepo (only used when body.openPage is set)
{} as any, // pageAccess (idem)
// environment (#332): keep deferred tool loading OFF for this lifecycle
// harness so the toolset/behavior is exactly as before.
{ isAiChatDeferredToolsEnabled: () => false } as any,
);
}
@@ -315,4 +320,174 @@ describe('AiChatService.stream [integration]', () => {
true,
);
});
/**
* #332 deferred tool loading, the ON path. The riskiest property is that the
* per-turn `activatedTools` Set is created FRESH inside each stream() call, so a
* tool a previous turn activated via loadTools is NOT still active when the next
* turn starts the new turn begins "cold" (CORE + loadTools only). The unit
* tests only exercise pure prepareAgentStep with hand-fed Sets; this pins the
* real wiring end-to-end (loadTools.execute -> activatedTools -> prepareStep ->
* per-step activeTools) against the real streamText loop, and proves there is no
* cross-turn leak. We drive a MockLanguageModelV3 whose step 1 calls
* loadTools(['createPage']) and assert, via the model's recorded per-step
* CallOptions.tools (the AI SDK filters the provider tool list by activeTools),
* that the deferred tool becomes active on the SAME turn's next step but NOT on a
* fresh turn's first step.
*/
describe('deferred tool loading ON — per-turn activation, no leak (#332)', () => {
// A stub deferred (non-core) tool the agent can activate. Its execute is never
// called — the model only needs to SEE it become active — but it must be a
// valid AI-SDK tool so the SDK includes it in a step's tool list once active.
const createPageStub = tool({
description: 'create a new page',
inputSchema: z.object({ title: z.string() }),
execute: async () => ({ id: 'p-stub' }),
});
// A CORE tool in the toolset, so a cold step shows CORE tools ARE active while
// the deferred createPage is not. `searchPages` is in CORE_TOOL_SET.
const searchPagesStub = tool({
description: 'search the wiki',
inputSchema: z.object({ query: z.string() }),
execute: async () => [],
});
// Same lifecycle harness as buildService() above, but with deferred loading ON
// and a toolset that exposes exactly one deferred tool (createPage) so it is
// catalogued + loadable-by-name. Kept separate so the OFF scenarios are
// untouched.
function buildDeferredService(): AiChatService {
return new AiChatService(
{ getChatModel: async () => null } as any,
aiChatRepo,
msgRepo,
{} as any,
{ resolve: async () => null } as any,
{
forUser: async () => ({
searchPages: searchPagesStub,
createPage: createPageStub,
}),
getInAppDeferredCatalog: async () => [
{ name: 'createPage', catalogLine: 'createPage — create a new page.' },
],
} as any,
mcpClients as any,
{} as any,
{} as any,
{} as any,
// #332: deferred tool loading ON — the property under test.
{ isAiChatDeferredToolsEnabled: () => true } as any,
);
}
// Drive ONE stream() turn against `model` and wait for the assistant row to
// settle (mirrors runStream, but builds the deferred-ON service).
async function runDeferredTurn(
model: MockLanguageModelV3,
chatId: string,
body: any,
): Promise<void> {
closeCalls = 0;
const service = buildDeferredService();
const { res, cleanup } = await makeRealResponse();
try {
await service.stream({
user: { id: userId, workspaceId } as any,
workspace: { id: workspaceId, name: 'WS' } as any,
sessionId: 'sess-1',
body,
res: { raw: res } as any,
signal: new AbortController().signal,
model: model as any,
role: null,
} as any);
await waitFor(async () => {
const rows = await msgRepo.findAllByChat(chatId, workspaceId);
return rows.some(
(r) =>
r.role === 'assistant' &&
['completed', 'error', 'aborted'].includes(r.status as string),
);
});
await waitFor(() => closeCalls > 0, { timeoutMs: 5_000 });
} finally {
await cleanup();
}
}
// Tool names the provider actually received for a recorded step (activeTools
// filters this list, so it reflects what was active that step).
const toolNames = (call: any): string[] =>
((call?.tools ?? []) as any[]).map((t) => t?.name).filter(Boolean);
// A model that, on step 1, calls loadTools(['createPage']); on step 2, answers.
function loadThenAnswerModel(): MockLanguageModelV3 {
let step = 0;
return new MockLanguageModelV3({
doStream: async () => {
const n = step++;
if (n === 0) {
return {
stream: convertArrayToReadableStream([
{ type: 'stream-start', warnings: [] },
{
type: 'tool-call',
toolCallId: 'lt1',
toolName: 'loadTools',
input: JSON.stringify({ names: ['createPage'] }),
},
{
type: 'finish',
finishReason: 'tool-calls',
usage: { inputTokens: 5, outputTokens: 3, totalTokens: 8 },
},
] as any),
};
}
return { stream: successStream() };
},
} as any);
}
it('activates a deferred tool for the SAME turn, and a NEW turn starts cold (no leak)', async () => {
const chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
// --- Turn 1: loadTools(createPage) on step 1, then answer on step 2. ---
const model1 = loadThenAnswerModel();
await runDeferredTurn(model1, chatId, {
chatId,
messages: [userUiMessage('Make me a page')],
});
// The turn ran at least two steps (the load round-trip + the answer).
expect(model1.doStreamCalls.length).toBeGreaterThanOrEqual(2);
const step1Tools = toolNames(model1.doStreamCalls[0]);
const step2Tools = toolNames(model1.doStreamCalls[1]);
// Step 1 starts cold: CORE tools + the loadTools meta-tool are active, but
// the deferred createPage is NOT yet.
expect(step1Tools).toContain('loadTools');
expect(step1Tools).toContain('searchPages'); // a CORE tool, always active
expect(step1Tools).not.toContain('createPage');
// Step 2 of the SAME turn sees the just-activated deferred tool.
expect(step2Tools).toContain('createPage');
// --- Turn 2 on the SAME chat: must start cold again. ---
const model2 = new MockLanguageModelV3({
doStream: async () => ({ stream: successStream() }),
} as any);
await runDeferredTurn(model2, chatId, {
chatId,
messages: [userUiMessage('And another thing')],
});
const nextTurnFirstStep = toolNames(model2.doStreamCalls[0]);
expect(nextTurnFirstStep).toContain('loadTools');
// The activated set is per-turn: the prior turn's createPage did NOT leak,
// so the fresh turn's first step sees it deferred again.
expect(nextTurnFirstStep).not.toContain('createPage');
});
});
});
@@ -0,0 +1,160 @@
import { Kysely } from 'kysely';
import { CommentRepo } from '../../src/database/repos/comment/comment.repo';
import {
getTestDb,
destroyTestDb,
buildTestDb,
createWorkspace,
createSpace,
createPage,
createUser,
createComment,
} from './db';
/**
* Real-DB coverage for CommentRepo.deleteCommentIfChildless (#338 F4/F6).
*
* This is the guard that keeps an ephemeral-suggestion hard-delete from
* cascade-destroying a reply (`comments.parent_comment_id` is ON DELETE CASCADE).
* The unit tests MOCK this method to 0/1, so only an int-spec actually exercises
* the SQL the FOR UPDATE lock-then-recheck transaction against Postgres.
*
* The concurrency case is the whole point: a plain anti-join
* `DELETE … WHERE NOT EXISTS(child)` passes (a) and (b) but SILENTLY loses a
* reply that commits mid-operation under READ COMMITTED (EvalPlanQual does not
* re-check a merely-locked row). Test (c) reproduces exactly that interleaving
* and asserts the row + reply both survive.
*/
describe('CommentRepo.deleteCommentIfChildless [integration]', () => {
let db: Kysely<any>;
let repo: CommentRepo;
let workspaceId: string;
let spaceId: string;
let pageId: string;
let userId: string;
beforeAll(async () => {
db = getTestDb();
repo = new CommentRepo(db as any);
workspaceId = (await createWorkspace(db)).id;
spaceId = (await createSpace(db, workspaceId)).id;
pageId = (await createPage(db, { workspaceId, spaceId })).id;
userId = (await createUser(db, workspaceId)).id;
});
afterAll(async () => {
await destroyTestDb();
});
async function rowExists(id: string): Promise<boolean> {
const row = await db
.selectFrom('comments')
.select('id')
.where('id', '=', id)
.executeTakeFirst();
return Boolean(row);
}
function seedTopLevel() {
return createComment(db, {
workspaceId,
spaceId,
pageId,
creatorId: userId,
selection: 'old text',
suggestedText: 'new text',
});
}
function seedReply(parentId: string) {
return createComment(db, {
workspaceId,
spaceId,
pageId,
creatorId: userId,
parentCommentId: parentId,
});
}
it('(a) childless top-level → returns 1 and the row is gone', async () => {
const parent = await seedTopLevel();
expect(await rowExists(parent.id)).toBe(true);
const deleted = await repo.deleteCommentIfChildless(parent.id);
expect(deleted).toBe(1);
expect(await rowExists(parent.id)).toBe(false);
});
it('(b) top-level WITH a committed reply → returns 0, parent AND reply survive (gate blocks the cascade)', async () => {
const parent = await seedTopLevel();
const reply = await seedReply(parent.id);
const deleted = await repo.deleteCommentIfChildless(parent.id);
expect(deleted).toBe(0);
expect(await rowExists(parent.id)).toBe(true);
expect(await rowExists(reply.id)).toBe(true);
});
it('(c) reply COMMITS mid-operation (FOR UPDATE path) → returns 0, parent + reply survive; a blind anti-join would lose the reply', async () => {
const parent = await seedTopLevel();
// Second connection holds an open transaction that inserts a reply (taking
// FOR KEY SHARE on the parent via the FK) and does NOT commit until we open
// the gate — reproducing the "reply not yet committed" window.
const conn2 = buildTestDb();
let openGate!: () => void;
const gate = new Promise<void>((resolve) => {
openGate = resolve;
});
let replyId: string | undefined;
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
try {
const replyTx = conn2.transaction().execute(async (trx) => {
const row = await trx
.insertInto('comments')
.values({
workspaceId,
spaceId,
pageId,
creatorId: userId,
parentCommentId: parent.id,
})
.returning(['id'])
.executeTakeFirstOrThrow();
replyId = row.id as string;
// Hold the FOR KEY SHARE lock on the parent until the gate opens.
await gate;
});
// Let the reply INSERT acquire its lock before the delete starts.
await sleep(250);
// deleteCommentIfChildless does SELECT ... FOR UPDATE on the parent, which
// conflicts with the reply's FOR KEY SHARE, so it BLOCKS here.
const deletePromise = repo.deleteCommentIfChildless(parent.id);
// Give the delete time to reach (and block on) its FOR UPDATE, then let the
// reply commit. The delete then wakes, re-checks under the lock, sees the
// now-committed reply, and returns 0.
await sleep(250);
openGate();
await replyTx;
const deleted = await deletePromise;
expect(deleted).toBe(0);
expect(await rowExists(parent.id)).toBe(true);
expect(replyId).toBeDefined();
expect(await rowExists(replyId!)).toBe(true);
} finally {
// Always release the gate (in case an assertion threw before openGate) and
// close the extra connection so global-teardown can DROP the database.
openGate();
await conn2.destroy();
}
});
});
+90
View File
@@ -132,6 +132,62 @@ export async function createUser(
return { id: row.id as string };
}
// The default group every workspace has; `groupUserRepo.addUserToDefaultGroup`
// (invoked by acceptInvitation) looks it up by `isDefault = true`, so a
// workspace under test must have exactly one for the accept path to complete.
export async function createDefaultGroup(
db: Kysely<any>,
workspaceId: string,
overrides: { name?: string } = {},
): Promise<{ id: string }> {
const id = randomUUID();
const suffix = shortId(id);
const row = await db
.insertInto('groups')
.values({
id,
// name is unique per workspace + NOT NULL.
name: overrides.name ?? `group-${suffix}`,
isDefault: true,
workspaceId,
})
.returning(['id'])
.executeTakeFirstOrThrow();
return { id: row.id as string };
}
// A pending workspace invitation. `role`/`token` are NOT NULL; `groupIds` is a
// nullable uuid[] and `invitedById` a nullable FK to users. Returns the fields a
// spec needs to drive acceptInvitation (id + token + the invited email).
export async function createInvitation(
db: Kysely<any>,
args: {
workspaceId: string;
email: string;
invitedById?: string | null;
role?: string;
token?: string;
groupIds?: string[] | null;
},
): Promise<{ id: string; token: string; email: string }> {
const id = randomUUID();
const token = args.token ?? `tok-${shortId(id)}`;
const row = await db
.insertInto('workspaceInvitations')
.values({
id,
email: args.email,
role: args.role ?? 'member',
token,
groupIds: (args.groupIds ?? null) as any,
invitedById: args.invitedById ?? null,
workspaceId: args.workspaceId,
})
.returning(['id'])
.executeTakeFirstOrThrow();
return { id: row.id as string, token, email: args.email };
}
export async function createSpace(
db: Kysely<any>,
workspaceId: string,
@@ -174,6 +230,40 @@ export async function createPage(
return { id: row.id as string };
}
export async function createComment(
db: Kysely<any>,
args: {
workspaceId: string;
spaceId: string;
pageId: string;
creatorId?: string | null;
parentCommentId?: string | null;
content?: unknown;
selection?: string | null;
suggestedText?: string | null;
type?: string | null;
},
): Promise<{ id: string }> {
const id = randomUUID();
const row = await db
.insertInto('comments')
.values({
id,
workspaceId: args.workspaceId,
spaceId: args.spaceId,
pageId: args.pageId,
creatorId: args.creatorId ?? null,
parentCommentId: args.parentCommentId ?? null,
content: (args.content ?? null) as any,
selection: args.selection ?? null,
suggestedText: args.suggestedText ?? null,
type: args.type ?? 'page',
})
.returning(['id'])
.executeTakeFirstOrThrow();
return { id: row.id as string };
}
export async function createRole(
db: Kysely<any>,
args: {
@@ -0,0 +1,218 @@
import { BadRequestException } from '@nestjs/common';
import { Kysely } from 'kysely';
import { Workspace } from '@docmost/db/types/entity.types';
import { UserRepo } from '@docmost/db/repos/user/user.repo';
import { GroupRepo } from '@docmost/db/repos/group/group.repo';
import { GroupUserRepo } from '@docmost/db/repos/group/group-user.repo';
import { WorkspaceInvitationService } from 'src/core/workspace/services/workspace-invitation.service';
import {
getTestDb,
destroyTestDb,
createWorkspace,
createUser,
createDefaultGroup,
createInvitation,
} from './db';
/**
* acceptInvitation atomicity (issue #324, tail of #244).
*
* acceptInvitation() reads the invitation OUTSIDE the transaction, then inside a
* single tx: inserts the invited user, adds them to the default group, and
* deletes the invitation. Two accepts of the SAME invitation therefore race to
* insert a user with the same (email, workspaceId) which the
* `users_email_workspace_id_unique` constraint forbids. The service catches that
* violation and reports "Invitation already accepted".
*
* These specs pin the INVARIANT that path protects: no matter how many times the
* invitation is accepted (concurrently or repeatedly), the workspace ends up
* with exactly ONE membership for the invited email and the invitation is
* consumed exactly once never a duplicate user and never a half-applied state.
*
* The service is wired with the REAL repos (UserRepo / GroupRepo / GroupUserRepo)
* against the test Kysely; only the peripheral collaborators that acceptInvitation
* touches AFTER the transaction (mail, session token, billing, audit, env) are
* stubbed, so the exercised DB write path is the production one.
*/
describe('WorkspaceInvitationService.acceptInvitation atomicity [integration]', () => {
let db: Kysely<any>;
let service: WorkspaceInvitationService;
// Count the memberships (user rows) for an email within a workspace — the
// quantity the atomicity guarantee is about.
async function membershipCount(
workspaceId: string,
email: string,
): Promise<number> {
const rows = await db
.selectFrom('users')
.select('id')
.where('workspaceId', '=', workspaceId)
.where('email', '=', email.toLowerCase())
.execute();
return rows.length;
}
async function invitationExists(invitationId: string): Promise<boolean> {
const row = await db
.selectFrom('workspaceInvitations')
.select('id')
.where('id', '=', invitationId)
.executeTakeFirst();
return !!row;
}
beforeAll(() => {
db = getTestDb();
const userRepo = new UserRepo(db as any);
const groupRepo = new GroupRepo(db as any);
const groupUserRepo = new GroupUserRepo(db as any, groupRepo, userRepo);
// Collaborators used only on the post-commit success tail; safe to stub.
const mailService = { sendToQueue: jest.fn().mockResolvedValue(undefined) };
const domainService = {} as any;
const tokenService = {} as any;
const sessionService = {
createSessionAndToken: jest.fn().mockResolvedValue('test-auth-token'),
};
const billingQueue = { add: jest.fn().mockResolvedValue(undefined) };
const environmentService = { isCloud: () => false };
const auditService = { log: jest.fn() };
service = new WorkspaceInvitationService(
userRepo,
groupUserRepo,
mailService as any,
domainService,
tokenService,
sessionService as any,
db as any,
billingQueue as any,
environmentService as any,
auditService as any,
);
});
afterAll(async () => {
await destroyTestDb();
});
// A workspace with its default group, an inviter, and a pending invitation.
async function seedInvite(): Promise<{
workspace: Workspace;
invitationId: string;
token: string;
email: string;
}> {
const { id: workspaceId } = await createWorkspace(db);
await createDefaultGroup(db, workspaceId);
const inviter = await createUser(db, workspaceId);
// Distinct address per invite so specs never collide across the suite.
const email = `invitee-${workspaceId.slice(0, 8)}@example.test`;
const invite = await createInvitation(db, {
workspaceId,
email,
invitedById: inviter.id,
});
// acceptInvitation only reads id/hostname/enforceSso/emailDomains/enforceMfa
// off the workspace; a minimal plain object is sufficient.
const workspace = {
id: workspaceId,
hostname: `host-${workspaceId.slice(0, 8)}`,
enforceSso: false,
enforceMfa: false,
emailDomains: [] as string[],
} as unknown as Workspace;
return { workspace, invitationId: invite.id, token: invite.token, email };
}
it('concurrent accepts create a single membership and consume the invitation once', async () => {
const { workspace, invitationId, token, email } = await seedInvite();
const dto = { invitationId, token, name: 'Invited User', password: 'password123' };
// Fire two accepts of the SAME invitation at once. They race to insert the
// same (email, workspaceId); the unique constraint lets exactly one win.
const results = await Promise.allSettled([
service.acceptInvitation({ ...dto }, workspace),
service.acceptInvitation({ ...dto }, workspace),
]);
const fulfilled = results.filter((r) => r.status === 'fulfilled');
const rejected = results.filter(
(r): r is PromiseRejectedResult => r.status === 'rejected',
);
// Exactly one accept succeeds; the other is rejected.
expect(fulfilled).toHaveLength(1);
expect(rejected).toHaveLength(1);
// The loser fails via the caught unique-constraint path with the specific
// "already accepted" message — not a half-state / generic failure.
expect(rejected[0].reason).toBeInstanceOf(BadRequestException);
expect(rejected[0].reason.message).toBe('Invitation already accepted');
// Invariant: exactly one membership, and the invitation is gone.
expect(await membershipCount(workspace.id, email)).toBe(1);
expect(await invitationExists(invitationId)).toBe(false);
});
it('a repeated (sequential) accept does not create a duplicate membership', async () => {
const { workspace, invitationId, token, email } = await seedInvite();
const dto = { invitationId, token, name: 'Invited User', password: 'password123' };
// First accept succeeds and returns an auth token.
const first = await service.acceptInvitation({ ...dto }, workspace);
expect(first?.authToken).toBe('test-auth-token');
expect(await membershipCount(workspace.id, email)).toBe(1);
expect(await invitationExists(invitationId)).toBe(false);
// Re-accepting the (now consumed) invitation must be rejected and must NOT
// add a second membership. The invitation row is gone, so this hits the
// "Invitation not found" guard rather than the unique-constraint path.
await expect(
service.acceptInvitation({ ...dto }, workspace),
).rejects.toBeInstanceOf(BadRequestException);
expect(await membershipCount(workspace.id, email)).toBe(1);
});
it('the single created membership is added to the default group (no partial state)', async () => {
const { workspace, invitationId, token, email } = await seedInvite();
const dto = { invitationId, token, name: 'Invited User', password: 'password123' };
await Promise.allSettled([
service.acceptInvitation({ ...dto }, workspace),
service.acceptInvitation({ ...dto }, workspace),
]);
// Resolve the one surviving user and assert the whole tx applied: they exist
// AND are in the workspace default group (the mid-transaction step), proving
// the winning accept committed as a whole rather than leaving a torn state.
const user = await db
.selectFrom('users')
.select(['id'])
.where('workspaceId', '=', workspace.id)
.where('email', '=', email.toLowerCase())
.executeTakeFirstOrThrow();
const defaultGroup = await db
.selectFrom('groups')
.select(['id'])
.where('workspaceId', '=', workspace.id)
.where('isDefault', '=', true)
.executeTakeFirstOrThrow();
const membership = await db
.selectFrom('groupUsers')
.select(['userId'])
.where('groupId', '=', defaultGroup.id)
.where('userId', '=', user.id)
.execute();
expect(membership).toHaveLength(1);
});
});
+5
View File
@@ -12,6 +12,11 @@ services:
ports:
- "3000:3000"
restart: unless-stopped
# The app already serves precompressed (brotli/gzip) static assets with
# long-lived cache headers and gzips dynamic API responses. For the best
# cold-load latency you can OPTIONALLY put a reverse proxy (caddy / nginx /
# traefik) in front with HTTP/2 (or HTTP/3) and brotli enabled — none is
# required for compression to work.
volumes:
- docmost:/app/data/storage
+4
View File
@@ -13,5 +13,9 @@
"types": "dist/index.d.ts",
"dependencies": {
"marked": "17.0.5"
},
"devDependencies": {
"@vitest/coverage-v8": "4.1.6",
"vitest": "4.1.6"
}
}
+16
View File
@@ -5,5 +5,21 @@ export default defineConfig({
environment: "jsdom",
globals: true,
include: ["src/**/*.{test,spec}.ts"],
// Coverage gate (issue #324). v8 provider avoids the istanbul AST-rewrite
// that broke on this package's ESM barrel. Thresholds sit a few points
// below the level measured on develop, over the files the suite exercises
// (`all: false`), so the gate passes today and catches a real regression.
coverage: {
enabled: true,
provider: "v8",
reporter: ["text-summary", "text"],
all: false,
thresholds: {
statements: 54,
branches: 44,
functions: 60,
lines: 54,
},
},
},
});
+2
View File
@@ -20,6 +20,7 @@
},
"license": "MIT",
"dependencies": {
"@docmost/prosemirror-markdown": "workspace:*",
"@tiptap/core": "3.20.4",
"@tiptap/extension-highlight": "3.20.4",
"@tiptap/extension-image": "3.20.4",
@@ -38,6 +39,7 @@
"@docmost/editor-ext": "workspace:*",
"@types/jsdom": "^21.1.7",
"@types/node": "^20.0.0",
"@vitest/coverage-v8": "4.1.6",
"fast-check": "^4.8.0",
"typescript": "^5.0.0",
"vitest": "4.1.6"
+1 -1
View File
@@ -31,7 +31,7 @@
*/
import { dirname } from "node:path";
import { sep } from "node:path";
import { parsePageFile, serializePageFile } from "../lib/page-file.js";
import { parsePageFile, serializePageFile } from "@docmost/prosemirror-markdown";
import type { GitSyncClient } from "./client.types.js";
import { buildVaultLayout, type PageNode } from "./layout.js";
import {
+5 -2
View File
@@ -26,8 +26,11 @@
* the gitmost server drives the engine in-process (there is no standalone CLI
* entry point).
*/
import { type DocmostMdMeta } from "../lib/index.js";
import { parsePageFile, serializePageFile } from "../lib/page-file.js";
import {
type DocmostMdMeta,
parsePageFile,
serializePageFile,
} from "@docmost/prosemirror-markdown";
import type { GitSyncClient } from "./client.types.js";
import type { DiffEntry } from "./git.js";
import { VaultGit, DEFAULT_BRANCH } from "./git.js";
+1 -1
View File
@@ -17,7 +17,7 @@ import {
markdownToProseMirror,
serializeDocmostMarkdownBody,
type DocmostMdMeta,
} from "../lib/index.js";
} from "@docmost/prosemirror-markdown";
/**
* Meta object as `exportPageBody` builds it (SPEC §4). Kept byte-for-byte
+7 -3
View File
@@ -8,6 +8,10 @@
*/
// Pure converter (markdown <-> ProseMirror, file envelope, canonicalization).
// Re-exported from the standalone `@docmost/prosemirror-markdown` package,
// which is the single source of truth for the converter core; git-sync keeps
// only the engine (vault/git/orchestrator) and re-surfaces the converter for
// in-process consumers of the git-sync barrel.
export {
serializeDocmostMarkdown,
serializeDocmostMarkdownBody,
@@ -16,8 +20,8 @@ export {
markdownToProseMirror,
canonicalizeContent,
docsCanonicallyEqual,
} from "./lib/index.js";
export type { DocmostMdMeta } from "./lib/index.js";
} from "@docmost/prosemirror-markdown";
export type { DocmostMdMeta } from "@docmost/prosemirror-markdown";
// Pure engine (no IO): reconcile planner, vault layout, sanitize, stabilize,
// loop-guard body hash.
@@ -123,4 +127,4 @@ export {
} from "./engine/path-guard.js";
export type { PathGuardIo, VaultPathUnsafeReason } from "./engine/path-guard.js";
export { parsePageFile, serializePageFile } from "./lib/page-file.js";
export { parsePageFile, serializePageFile } from "@docmost/prosemirror-markdown";
File diff suppressed because it is too large Load Diff
@@ -1,365 +0,0 @@
/**
* Pure markdown -> ProseMirror conversion.
*
* The converter path is `markdownToProseMirror` (marked -> HTML ->
* generateJSON) plus the two pre/post processors it needs (`preprocessCallouts`,
* `bridgeTaskLists`). The gitmost server writes the resulting page bodies
* natively through the collab gateway, so no websocket/Yjs write-path lives
* here.
*/
import { generateJSON } from "@tiptap/html";
import { JSDOM } from "jsdom";
import { marked } from "marked";
import { docmostExtensions } from "./docmost-schema.js";
// Setup DOM environment for Tiptap HTML parsing in Node.js
const dom = new JSDOM("<!DOCTYPE html><html><body></body></html>");
global.window = dom.window as any;
global.document = dom.window.document;
// @ts-ignore
global.Element = dom.window.Element;
/**
* Hard ceiling above which we skip callout preprocessing entirely. The linear
* scanner below has no quadratic blow-up, but we still cap input defensively so
* a pathological multi-megabyte payload cannot tie up the event loop; in that
* case the markdown is passed through verbatim (callouts are simply not
* detected) rather than risking a slow scan.
*/
const MAX_CALLOUT_PREPROCESS_BYTES = 4 * 1024 * 1024; // 4 MB
/** Matches an opening callout fence: `:::type` (type captured, lower-cased). */
const CALLOUT_OPEN_RE = /^:::\s*(\w+)\s*$/;
/** Matches a bare closing callout fence: `:::`. */
const CALLOUT_CLOSE_RE = /^:::\s*$/;
/**
* Matches an Obsidian-native callout opener: `> [!type]` (type captured). An
* optional title after the type is allowed but ignored (the Docmost callout
* schema has no title). The body is the following contiguous blockquote lines.
*/
const CALLOUT_BQ_OPEN_RE = /^>\s*\[!(\w+)\]/;
/** Matches any blockquote continuation line (`>` … ). */
const BLOCKQUOTE_LINE_RE = /^>/;
/** Matches the start/end of a code fence (``` or ~~~), capturing the marker. */
const CODE_FENCE_RE = /^(\s*)(`{3,}|~{3,})/;
/**
* Pre-process Docmost-flavoured markdown: convert `:::type ... :::`
* callout blocks (the syntax our markdown export produces) into HTML
* divs that the callout extension parses. The inner content is rendered
* through marked as regular markdown.
*
* Implemented as a single linear pass over the lines (no quadratic regex
* rescan). It:
* - tracks fenced code regions (```...``` and ~~~...~~~) and never treats a
* `:::` line that lives inside a code fence as a callout delimiter, so a
* callout body that itself contains a fenced code block with a `:::` line is
* no longer corrupted;
* - matches an opening `:::type` line with the next CLOSING `:::` at the SAME
* nesting level, supporting NESTED callouts via a depth counter (an inner
* `:::type` opens a deeper level and consumes a matching `:::`);
* - emits the same `<div data-type="callout" data-callout-type="TYPE">` output
* (inner rendered through marked) as the previous regex implementation.
*/
async function preprocessCallouts(markdown: string): Promise<string> {
// Defensive cap: skip preprocessing for pathologically large inputs.
if (markdown.length > MAX_CALLOUT_PREPROCESS_BYTES) {
return markdown;
}
// Recursively transform a slice of lines, converting top-level callouts in
// that slice into <div> blocks and rendering their inner content (which may
// itself contain nested callouts) through this same function.
const transform = async (lines: string[]): Promise<string> => {
const out: string[] = [];
let inCodeFence = false;
let codeFenceMarker = ""; // the exact run of backticks/tildes that opened it
let i = 0;
while (i < lines.length) {
const line = lines[i];
// Inside a code fence, only its matching closing fence is significant;
// everything else (including `:::` lines) is copied through verbatim.
if (inCodeFence) {
out.push(line);
const fence = line.match(CODE_FENCE_RE);
if (fence && fence[2].startsWith(codeFenceMarker[0]) &&
fence[2].length >= codeFenceMarker.length) {
inCodeFence = false;
codeFenceMarker = "";
}
i++;
continue;
}
// A code fence opening outside any callout body: enter code-fence mode.
const fenceOpen = line.match(CODE_FENCE_RE);
if (fenceOpen) {
inCodeFence = true;
codeFenceMarker = fenceOpen[2];
out.push(line);
i++;
continue;
}
// An opening callout fence: scan forward (with code-fence and nested
// callout awareness) for its matching closing `:::` at the same level.
const open = line.match(CALLOUT_OPEN_RE);
if (open) {
const type = open[1].toLowerCase();
const bodyLines: string[] = [];
let depth = 1;
let innerInCodeFence = false;
let innerCodeFenceMarker = "";
let j = i + 1;
for (; j < lines.length; j++) {
const bl = lines[j];
if (innerInCodeFence) {
const f = bl.match(CODE_FENCE_RE);
if (f && f[2].startsWith(innerCodeFenceMarker[0]) &&
f[2].length >= innerCodeFenceMarker.length) {
innerInCodeFence = false;
innerCodeFenceMarker = "";
}
bodyLines.push(bl);
continue;
}
const innerFence = bl.match(CODE_FENCE_RE);
if (innerFence) {
innerInCodeFence = true;
innerCodeFenceMarker = innerFence[2];
bodyLines.push(bl);
continue;
}
if (CALLOUT_OPEN_RE.test(bl)) {
depth++;
bodyLines.push(bl);
continue;
}
if (CALLOUT_CLOSE_RE.test(bl)) {
depth--;
if (depth === 0) break; // matching close for THIS callout
bodyLines.push(bl);
continue;
}
bodyLines.push(bl);
}
if (j < lines.length) {
// Found the matching closing fence: render the body (recursively, so
// nested callouts are handled) and emit the callout div.
const inner = await transform(bodyLines);
const renderedInner = await marked.parse(inner);
out.push(
`\n<div data-type="callout" data-callout-type="${type}">${renderedInner}</div>\n`,
);
i = j + 1; // skip past the closing `:::`
continue;
}
// No matching close (unterminated callout): treat the opener as a
// literal line and continue, preserving the original text.
out.push(line);
i++;
continue;
}
// An Obsidian-native callout: `> [!type]` opener; the body is the following
// CONTIGUOUS blockquote (`>`-prefixed) lines. Strip ONE blockquote level and
// recurse so nested callouts (`> > [!type]`) are handled, then emit the same
// callout div the `:::` path produces. A normal blockquote (no `[!type]` on
// its first line) does not match and stays a blockquote.
const bqOpen = line.match(CALLOUT_BQ_OPEN_RE);
if (bqOpen) {
const type = bqOpen[1].toLowerCase();
const bodyLines: string[] = [];
let j = i + 1;
for (; j < lines.length; j++) {
if (!BLOCKQUOTE_LINE_RE.test(lines[j])) break;
bodyLines.push(lines[j].replace(/^>\s?/, ""));
}
const inner = await transform(bodyLines);
const renderedInner = await marked.parse(inner);
out.push(
`\n<div data-type="callout" data-callout-type="${type}">${renderedInner}</div>\n`,
);
i = j;
continue;
}
out.push(line);
i++;
}
return out.join("\n");
};
return transform(markdown.split("\n"));
}
/**
* Bridge marked's checkbox lists to TipTap task lists.
*
* marked renders GitHub task list items (`- [x] done`) as a plain
* `<ul><li><p><input type="checkbox" checked> text</p></li></ul>` WITHOUT the
* markup TipTap's TaskList/TaskItem extensions parse. This rewrites such lists
* into the shape those extensions expect:
* TaskList parseHTML matches `ul[data-type="taskList"]`,
* TaskItem matches `li[data-type="taskItem"]`,
* the checked state is read from `data-checked === "true"`.
*
* A list is only converted when it has at least one `<li>` and EVERY direct
* `<li>` contains a checkbox input. Both `<ul>` and `<ol>` are considered: a
* numbered checklist (`1. [x] a`, which marked renders as an `<ol>` of checkbox
* `<li>`s) would otherwise lose its task state. TipTap task lists are unordered,
* so a matching `<ol>` is emitted as `data-type="taskList"` exactly like a
* `<ul>`. Mixed or ordinary lists (including ordinary `<ol>` lists) are left
* untouched so they keep rendering as bullet/numbered lists. The marked `<p>`
* wrapper is kept inside the `<li>` because TaskItem content allows paragraphs.
*/
function bridgeTaskLists(html: string): string {
// Cheap early-out: if the markup contains no checkbox input at all there is
// nothing to bridge, so skip the expensive JSDOM parse entirely. This is the
// common case (most pages have no task lists).
if (!/type=["']?checkbox/i.test(html)) {
return html;
}
// Defensive cap (consistent with preprocessCallouts): skip the bridge for
// pathologically large inputs rather than running a second expensive JSDOM
// parse on a multi-megabyte payload. The markup is passed through verbatim.
if (html.length > MAX_CALLOUT_PREPROCESS_BYTES) {
return html;
}
const dom = new JSDOM(html);
const document = dom.window.document;
// Collect the checkbox(es) that belong to THIS <li> directly: either direct
// child <input type="checkbox"> elements or ones inside the <li>'s direct <p>
// child (the shape marked emits: `<li><p><input type="checkbox"> text</p></li>`).
// Checkboxes nested deeper (e.g. inside a child <ul>/<ol>) are excluded so a
// bullet <li> that merely contains a nested task sublist is not misdetected.
// Raw inline HTML can put more than one checkbox in a single <li>; we gather
// ALL of them so none survive into the converted item.
const directCheckboxes = (li: Element): Element[] => {
const found: Element[] = [];
for (const child of Array.from(li.children)) {
if (
child.tagName === "INPUT" &&
child.getAttribute("type") === "checkbox"
) {
found.push(child);
continue;
}
if (child.tagName === "P") {
for (const inp of Array.from(
child.querySelectorAll(":scope > input[type='checkbox']"),
)) {
found.push(inp);
}
}
}
return found;
};
// Both <ul> and <ol> are candidates: an <ol> whose every direct <li> carries
// its own checkbox is a numbered checklist that must also become a taskList.
const lists = Array.from(document.querySelectorAll("ul, ol"));
for (const list of lists) {
// Only consider DIRECT child <li> elements; nested lists are handled by
// their own iteration of the outer loop.
const items = Array.from(list.children).filter(
(child) => child.tagName === "LI",
);
if (items.length === 0) continue;
const itemCheckboxes = items.map((li) => directCheckboxes(li));
// Convert only when every direct <li> carries at least one OWN checkbox.
if (!itemCheckboxes.every((boxes) => boxes.length > 0)) continue;
// A numbered checklist arrives as an <ol>. We must NOT leave the tag as
// <ol> while tagging it data-type="taskList": generateJSON would then match
// BOTH the orderedList rule (tag ol) and the taskList rule (data-type),
// emitting a phantom empty orderedList beside the real taskList. So rename a
// qualifying <ol> to a <ul> — move its <li> children over and replace it —
// leaving only the taskList rule to match. Already-<ul> lists are unchanged.
let target: Element = list;
if (list.tagName === "OL") {
const ul = document.createElement("ul");
// Carry over existing attributes (e.g. class) so nothing is silently lost.
for (const attr of Array.from(list.attributes)) {
ul.setAttribute(attr.name, attr.value);
}
// Move every child node (including the <li>s we collected) into the <ul>.
while (list.firstChild) {
ul.appendChild(list.firstChild);
}
list.replaceWith(ul);
target = ul;
}
target.setAttribute("data-type", "taskList");
items.forEach((li, index) => {
const boxes = itemCheckboxes[index];
// The first checkbox determines the checked state (matches the previous
// single-checkbox behaviour); any extras only need removing.
const input = boxes[0] ?? null;
li.setAttribute("data-type", "taskItem");
const checked =
input != null &&
(input.hasAttribute("checked") || (input as any).checked);
li.setAttribute("data-checked", checked ? "true" : "false");
// Remove ALL direct checkbox inputs so none survive into the content
// (a raw-inline-HTML <li> may carry more than one).
for (const box of boxes) {
box.remove();
}
});
}
return document.body.innerHTML;
}
/**
* Recursively strip content-less paragraph nodes from a generated doc.
*
* A block-level atom whose markdown form is INLINE (e.g. the block `image`'s
* `![](url)`, or a bare media element) is wrapped by marked in a <p>; the schema
* then HOISTS the block atom out of that paragraph, leaving an EMPTY paragraph
* sibling. On the next export that empty `<p>` renders to "" and the doc "\n\n"
* join injects a phantom blank gap, so the markdown is not byte-stable.
*
* Markdown blank lines are separators, never content, so generateJSON only ever
* produces an empty paragraph as such a hoist artifact removing them is safe
* and general (it also subsumes the <div>-wrapper workaround the `video` case
* uses). We remove ONLY `type === 'paragraph'` nodes whose `content` is absent
* or an empty array; every other node (including atoms without `content`) is
* preserved, and we recurse into the content of any node that has children.
*/
function stripEmptyParagraphs(node: any): any {
if (!node || !Array.isArray(node.content)) {
// Atom / leaf node (no children to recurse into): keep as-is.
return node;
}
const mapped = node.content.map((child: any) => stripEmptyParagraphs(child));
const isEmptyParagraph = (child: any): boolean =>
!!child &&
child.type === "paragraph" &&
(!Array.isArray(child.content) || child.content.length === 0);
const filtered = mapped.filter((child: any) => !isEmptyParagraph(child));
// Schema-validity guard: several nodes require NON-empty block content
// (`content: "block+"` — tableCell, tableHeader, blockquote, column, callout,
// and the doc root). For an empty one of those, generateJSON materializes a
// single empty paragraph as its OBLIGATORY content — that is not a hoist
// artifact. If stripping would empty the container, keep ONE empty paragraph
// so the result stays schema-valid (an empty cell/quote must not become `[]`).
const cleaned =
filtered.length === 0 && mapped.length > 0 ? [mapped[0]] : filtered;
return { ...node, content: cleaned };
}
/** Convert markdown to a ProseMirror doc using the full Docmost schema. */
export async function markdownToProseMirror(
markdownContent: string,
): Promise<any> {
const withCallouts = await preprocessCallouts(markdownContent);
const html = await marked.parse(withCallouts);
const bridged = bridgeTaskLists(html);
const doc = generateJSON(bridged, docmostExtensions);
return stripEmptyParagraphs(doc);
}
@@ -2,7 +2,7 @@ import { describe, expect, it, vi, beforeEach, afterEach } from 'vitest';
import { applyPushActions, LAST_PUSHED_REF } from '../src/engine/push';
import { bodyHash } from '../src/engine/loop-guard';
import type { ApplyPushDeps, PushActions } from '../src/engine/push';
import { parsePageFile, serializePageFile } from '../src/lib/page-file';
import { parsePageFile, serializePageFile } from '@docmost/prosemirror-markdown';
// The Docmost space this vault mirrors (native files carry no spaceId; the run
// supplies it). A CREATE targets this space.
@@ -5,7 +5,7 @@ import type {
MetaSide,
RenameMoveAction,
} from '../src/engine/push';
import type { DocmostMdMeta } from '../src/lib/index';
import type { DocmostMdMeta } from '@docmost/prosemirror-markdown';
// FS→Docmost push #3 (SPEC §5/§6/§16). `classifyRenameMoves` is the PURE half of
// the move/rename apply: it resolves each `{pageId, oldPath, newPath}` into the
@@ -1,7 +1,7 @@
import { describe, expect, it } from 'vitest';
import { computePushActions } from '../src/engine/push';
import type { DiffEntry, MetaSide } from '../src/engine/push';
import type { DocmostMdMeta } from '../src/lib/index';
import type { DocmostMdMeta } from '@docmost/prosemirror-markdown';
// FS→Docmost push, FIRST increment (SPEC §6). `computePushActions` is the PURE
// half: it classifies each `git diff --name-status` row into a Docmost action by
@@ -8,7 +8,7 @@ import { runCycle } from "../src/engine/cycle";
import type { CycleFs } from "../src/engine/cycle";
import { VaultGit } from "../src/engine/git";
import type { Settings } from "../src/engine/settings";
import { serializeDocmostMarkdownBody } from "../src/lib/index";
import { serializeDocmostMarkdownBody } from "@docmost/prosemirror-markdown";
const execFileAsync = promisify(execFile);
+1 -1
View File
@@ -8,7 +8,7 @@ import { firstDivergence } from './roundtrip-helpers';
import { applyPullActions } from '../src/engine/pull';
import type { PullActions, ApplyPullActionsDeps } from '../src/engine/pull';
import type { DeletionDecision } from '../src/engine/reconcile';
import { serializePageFile, parsePageFile } from '../src/lib/page-file';
import { serializePageFile, parsePageFile } from '@docmost/prosemirror-markdown';
// Engine-layer coverage gaps flagged by the PR #119 reviewers (test-strategy
// report, Module 2 `src/engine`). Each block targets a specific under-covered
+1 -1
View File
@@ -1,6 +1,6 @@
import { describe, expect, it } from 'vitest';
import { readExisting } from '../src/engine/pull';
import { serializePageFile } from '../src/lib/page-file';
import { serializePageFile } from '@docmost/prosemirror-markdown';
// R-Pull-1 (test-strategy report §5): `readExisting` now takes injectable IO
// (`listTracked` / `readFile`), so its parsing + skip rules are unit-testable
@@ -6,7 +6,7 @@ import type {
MetaSide,
RenameMoveAction,
} from '../src/engine/push.js';
import type { DocmostMdMeta } from '../src/lib/index.js';
import type { DocmostMdMeta } from '@docmost/prosemirror-markdown';
// RED-TEAM finding #4 (two facets):
// (a) buildVaultLayout disambiguation is ORDER-DEPENDENT: which of two
@@ -8,7 +8,7 @@ import {
import type { PushDeps } from '../src/engine/push';
import type { Settings } from '../src/engine/settings';
import { runCycle, type RunCycleDeps } from '../src/engine/cycle';
import { serializePageFile } from '../src/lib/page-file';
import { serializePageFile } from '@docmost/prosemirror-markdown';
// Red-team confirmations for PR #119 (git-sync). Each test asserts the DESIRED
// behavior, so it FAILS today iff the bug is real.
@@ -1,104 +0,0 @@
import { readFile } from 'node:fs/promises';
import { readdirSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path';
import { describe, expect, it } from 'vitest';
import {
convertProseMirrorToMarkdown,
markdownToProseMirror,
docsCanonicallyEqual,
} from 'docmost-client';
// Resolve fixtures relative to this test file so the test is CWD-independent.
const here = dirname(fileURLToPath(import.meta.url));
const CORPUS_DIR = join(here, 'fixtures', 'corpus');
const KNOWN_LIMITATIONS_DIR = join(here, 'fixtures', 'known-limitations');
/** Run a single document through export -> import -> export. */
async function roundTrip(doc: any) {
const md1 = convertProseMirrorToMarkdown(doc);
const doc2 = await markdownToProseMirror(md1);
const md2 = convertProseMirrorToMarkdown(doc2);
return { md1, md2, doc2 };
}
describe('round-trip corpus (SPEC §11)', () => {
// Discover the corpus synchronously at collection time so each fixture gets
// its own `it` with the file name in the test title.
const files = readdirSync(CORPUS_DIR)
.filter((name) => name.endsWith('.json'))
.sort();
it('has a non-empty corpus', () => {
expect(files.length).toBeGreaterThan(0);
});
for (const name of files) {
it(`${name}: markdown byte-stable AND canonically stable`, async () => {
const doc = JSON.parse(await readFile(join(CORPUS_DIR, name), 'utf8'));
const { md1, md2, doc2 } = await roundTrip(doc);
// 1) The byte-stable markdown property git actually needs.
expect(md2, `${name}: markdown not byte-stable`).toBe(md1);
// 2) Semantic stability (block ids stripped, default-null normalized).
expect(
docsCanonicallyEqual(doc, doc2),
`${name}: document not canonically stable`,
).toBe(true);
});
}
});
// ---------------------------------------------------------------------------
// KNOWN CONVERTER LIMITATIONS (isolated so they do NOT make CI red).
//
// SPEC §11 explicitly flags images and diagrams as high round-trip risk. These
// fixtures are kept OUT of the green corpus above and asserted with `it.fails`
// so the documented divergence is locked in (the test FAILS if the converter
// ever starts round-tripping them — at which point promote the fixture into
// the corpus). The precise divergences for `image-diagrams.json` are:
//
// * A BLOCK-LEVEL image preceded by a paragraph is NOT byte-stable on the
// FIRST re-export. The HTML re-parser hoists the block <img> out of its
// line and leaves an empty paragraph behind, so `paragraph` + `![..](..)`
// re-imports as paragraph + empty-paragraph + image; the empty paragraph
// adds one blank line, so export #2 grows by a one-time "\n\n" (md1 !== md2).
// This is NOT non-convergence: the growth happens exactly ONCE. The doc
// CONVERGES to a fixpoint after one extra `export→import→export` pass — the
// empty paragraph is already present after the first import, so export #2
// and export #3 are byte-identical (md2 === md3, verified).
//
// * drawio / excalidraw diagrams gain `data-align="center"` on the second
// export: the schema's diagram `align` attribute has a NON-null default of
// "center", which materializes on import; the converter only emits
// data-align when set, so it appears on export #2 but not #1. Like the
// image case, this is one-time and converges after one extra pass.
//
// * A STANDALONE block image (no preceding paragraph) IS byte-stable from
// export #1 (md1 === md2) — but it is still NOT canonically stable: on
// import the bare <img> is wrapped, gaining a leading EMPTY paragraph, so
// the canonical doc differs by that spurious paragraph node even though the
// markdown bytes match.
//
// Resolution (SPEC §11, "normalize-on-write"): rather than deep-fixing the
// converter, the engine runs ONE `export→import→export` pass when writing into
// the vault; from that fixpoint onward the form is byte-stable, so git sees no
// phantom diff. The green corpus above avoids these one-time asymmetries by
// pre-authoring the materialized defaults (e.g. `align: "center"` on the
// diagrams in 06-diagrams.json) so a single pass is already at the fixpoint.
// ---------------------------------------------------------------------------
describe('round-trip KNOWN LIMITATIONS (SPEC §11 image/diagram risk)', () => {
it.fails(
'image-diagrams.json is NOT byte-stable on export #1 (block image hoist + diagram align default; converges after one extra pass — SPEC §11 normalize-on-write)',
async () => {
const doc = JSON.parse(
await readFile(join(KNOWN_LIMITATIONS_DIR, 'image-diagrams.json'), 'utf8'),
);
const { md1, md2 } = await roundTrip(doc);
// This assertion FAILS today (documented divergence). `it.fails` turns a
// failing body into a PASS; if the converter is fixed this flips and the
// test goes red, prompting promotion into the green corpus.
expect(md2).toBe(md1);
},
);
});
@@ -8,7 +8,7 @@ import { runPush, LAST_PUSHED_REF } from '../src/engine/push';
import type { PushDeps } from '../src/engine/push';
import { VaultGit } from '../src/engine/git';
import type { Settings } from '../src/engine/settings';
import { serializeDocmostMarkdownBody } from '../src/lib/index';
import { serializeDocmostMarkdownBody } from '@docmost/prosemirror-markdown';
const execFileAsync = promisify(execFile);
+1 -1
View File
@@ -2,7 +2,7 @@ import { describe, expect, it, vi } from 'vitest';
import { runPush, LAST_PUSHED_REF, DOCMOST_BRANCH } from '../src/engine/push';
import type { PushDeps } from '../src/engine/push';
import type { Settings } from '../src/engine/settings';
import { serializePageFile } from '../src/lib/page-file';
import { serializePageFile } from '@docmost/prosemirror-markdown';
/** A native page file: `gitmost_id` frontmatter + clean body (title = filename). */
function fileFor(pageId: string, body = 'body'): string {
+18 -7
View File
@@ -2,8 +2,8 @@ import { describe, expect, it } from 'vitest';
import { stabilizePageFile, type PageMeta } from '../src/engine/stabilize.js';
// markdownToProseMirror lives in collaboration.ts; importing it mutates the
// global DOM via jsdom at module load time (required for @tiptap/html under Node).
import { markdownToProseMirror } from '../src/lib/markdown-to-prosemirror.js';
import { parseDocmostMarkdown } from '../src/lib/markdown-document.js';
import { markdownToProseMirror } from '@docmost/prosemirror-markdown';
import { parseDocmostMarkdown } from '@docmost/prosemirror-markdown';
// stabilize.ts (SPEC §11 normalize-on-write) was 0% covered (only the gated e2e
// touched it). stabilizePageFile is import-testable: build a small ProseMirror
@@ -22,16 +22,27 @@ const meta: PageMeta = {
describe('stabilizePageFile — normalize-on-write fixpoint (SPEC §11)', () => {
it('reaches a byte-identical fixpoint after one extra export/import/export pass', async () => {
// A diagram is the canonical one-pass asymmetry: drawio's `align` default of
// "center" materializes on import, so a NAIVE export differs on the second
// export. stabilizePageFile runs the convergence pass at write time, so the
// written body must already be at the fixpoint: re-importing its body and
// A diagram inside a column is the canonical one-pass asymmetry: on the
// raw-HTML/columns path a diagram's `align` default of "center" materializes
// on import, so a NAIVE export differs on the second export. (#293 canon #8
// made the TOP-LEVEL diagram form — `![](src)<!--drawio …-->` — byte-stable by
// omitting the default, so the asymmetry now lives only on the columns path
// where the schema `<div data-type="drawio">` form is retained.)
// stabilizePageFile runs the convergence pass at write time, so the written
// body must already be at the fixpoint: re-importing its body and
// re-stabilizing yields the exact same bytes.
const content = {
type: 'doc',
content: [
{ type: 'paragraph', content: [{ type: 'text', text: 'intro' }] },
{ type: 'drawio', attrs: { src: '/d.drawio' } },
{
type: 'columns',
attrs: { layout: 'two_equal' },
content: [
{ type: 'column', content: [{ type: 'drawio', attrs: { src: '/d.drawio' } }] },
{ type: 'column', content: [{ type: 'paragraph', content: [{ type: 'text', text: 'side' }] }] },
],
},
{ type: 'paragraph', content: [{ type: 'text', text: 'outro' }] },
],
};
@@ -1,8 +1,8 @@
import { describe, it, expect } from "vitest";
import { getSchema } from "@tiptap/core";
import { markdownToProseMirror } from "../src/lib/markdown-to-prosemirror";
import { docmostExtensions } from "../src/lib/docmost-schema";
import { markdownToProseMirror } from "@docmost/prosemirror-markdown";
import { docmostExtensions } from "@docmost/prosemirror-markdown";
// REGRESSION LOCK for the stripEmptyParagraphs schema-validity guard.
//
+19
View File
@@ -18,6 +18,25 @@ export default defineConfig({
},
test: {
environment: 'node',
// Coverage gate (issue #324). The v8 provider is used deliberately: the
// istanbul provider instruments sources by rewriting their AST, which broke
// on the ESM `@docmost/editor-ext` barrel import; v8 collects native
// coverage from the runtime and never re-parses ESM, so it sidesteps that.
// Thresholds are calibrated a few points BELOW the level measured on
// develop so the gate passes today but fails on a real regression. Numbers
// reflect the files actually exercised by the suite (`all: false`).
coverage: {
enabled: true,
provider: 'v8',
reporter: ['text-summary', 'text'],
all: false,
thresholds: {
statements: 88,
branches: 75,
functions: 72,
lines: 88,
},
},
// Runtime suites. The `.test.ts` glob deliberately EXCLUDES the type-only
// contract file (`*.test-d.ts`), which is enforced by the typecheck pass
// below instead — so the 35 runtime suites are never typechecked.
@@ -1,6 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.copy = copy;
function copy(value) {
return JSON.parse(JSON.stringify(value));
}
@@ -1,18 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.getFromPath = getFromPath;
/**
* get target value from json-pointer (e.g. /content/0/content)
* @param {AnyObject} obj object to resolve path into
* @param {string} path json-pointer
* @return {any} target value
*/
function getFromPath(obj, path) {
const pathParts = path.split("/");
pathParts.shift(); // remove root-entry
while (pathParts.length) {
const property = pathParts.shift();
obj = obj[property];
}
return obj;
}
@@ -1,27 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.getReplaceStep = getReplaceStep;
const transform_1 = require("@tiptap/pm/transform");
function getReplaceStep(fromDoc, toDoc) {
let start = toDoc.content.findDiffStart(fromDoc.content);
if (start === null) {
return false;
}
// @ts-ignore property access to content
let { a: endA, b: endB } = toDoc.content.findDiffEnd(fromDoc.content);
const overlap = start - Math.min(endA, endB);
if (overlap > 0) {
// If there is an overlap, there is some freedom of choice in how to calculate the
// start/end boundary. for an inserted/removed slice. We choose the extreme with
// the lowest depth value.
if (fromDoc.resolve(start - overlap).depth <
toDoc.resolve(endA + overlap).depth) {
start -= overlap;
}
else {
endA += overlap;
endB += overlap;
}
}
return new transform_1.ReplaceStep(start, endB, toDoc.slice(start, endA));
}
@@ -1,8 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.RecreateTransform = exports.recreateTransform = void 0;
// https://gitlab.com/mpapp-public/prosemirror-recreate-steps - MIT
// https://github.com/sueddeutsche/prosemirror-recreate-transform - MIT
var recreateTransform_1 = require("./recreateTransform");
Object.defineProperty(exports, "recreateTransform", { enumerable: true, get: function () { return recreateTransform_1.recreateTransform; } });
Object.defineProperty(exports, "RecreateTransform", { enumerable: true, get: function () { return recreateTransform_1.RecreateTransform; } });
@@ -1 +0,0 @@
{"type":"commonjs"}
@@ -1,242 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.RecreateTransform = void 0;
exports.recreateTransform = recreateTransform;
const transform_1 = require("@tiptap/pm/transform");
const rfc6902_1 = require("rfc6902");
const diff_1 = require("diff");
const getReplaceStep_1 = require("./getReplaceStep");
const simplifyTransform_1 = require("./simplifyTransform");
const removeMarks_1 = require("./removeMarks");
const getFromPath_1 = require("./getFromPath");
const copy_1 = require("./copy");
class RecreateTransform {
constructor(fromDoc, toDoc, options = {}) {
const o = {
complexSteps: true,
wordDiffs: false,
simplifyDiff: true,
...options,
};
this.fromDoc = fromDoc;
this.toDoc = toDoc;
this.complexSteps = o.complexSteps; // Whether to return steps other than ReplaceSteps
this.wordDiffs = o.wordDiffs; // Whether to make text diffs cover entire words
this.simplifyDiff = o.simplifyDiff;
this.schema = fromDoc.type.schema;
this.tr = new transform_1.Transform(fromDoc);
}
init() {
if (this.complexSteps) {
// For First steps: we create versions of the documents without marks as
// these will only confuse the diffing mechanism and marks won't cause
// any mapping changes anyway.
this.currentJSON = (0, removeMarks_1.removeMarks)(this.fromDoc).toJSON();
this.finalJSON = (0, removeMarks_1.removeMarks)(this.toDoc).toJSON();
this.ops = (0, rfc6902_1.createPatch)(this.currentJSON, this.finalJSON);
this.recreateChangeContentSteps();
this.recreateChangeMarkSteps();
}
else {
// We don't differentiate between mark changes and other changes.
this.currentJSON = this.fromDoc.toJSON();
this.finalJSON = this.toDoc.toJSON();
this.ops = (0, rfc6902_1.createPatch)(this.currentJSON, this.finalJSON);
this.recreateChangeContentSteps();
}
if (this.simplifyDiff) {
this.tr = (0, simplifyTransform_1.simplifyTransform)(this.tr) || this.tr;
}
return this.tr;
}
/** convert json-diff to prosemirror steps */
recreateChangeContentSteps() {
// First step: find content changing steps.
let ops = [];
while (this.ops.length) {
// get next
let op = this.ops.shift();
ops.push(op);
let toDoc;
const afterStepJSON = (0, copy_1.copy)(this.currentJSON); // working document receiving patches
const pathParts = op.path.split("/");
// collect operations until we receive a valid document:
// apply ops-patches until a valid prosemirror document is retrieved,
// then try to create a transformation step or retry with next operation
while (toDoc == null) {
(0, rfc6902_1.applyPatch)(afterStepJSON, [op]);
try {
toDoc = this.schema.nodeFromJSON(afterStepJSON);
toDoc.check();
}
catch (error) {
toDoc = null;
if (this.ops.length > 0) {
op = this.ops.shift();
ops.push(op);
}
else {
throw new Error(`No valid diff possible applying ${op.path}`);
}
}
}
// apply operation (ignoring afterStepJSON)
if (this.complexSteps &&
ops.length === 1 &&
(pathParts.includes("attrs") || pathParts.includes("type"))) {
// Node markup is changing
this.addSetNodeMarkup(); // a lost update is ignored
ops = [];
// console.log("%cop", logStyle, "- update node", ops);
}
else if (ops.length === 1 &&
op.op === "replace" &&
pathParts[pathParts.length - 1] === "text") {
// Text is being replaced, we apply text diffing to find the smallest possible diffs.
this.addReplaceTextSteps(op, afterStepJSON);
ops = [];
// console.log("%cop", logStyle, "- replace", ops);
}
else if (this.addReplaceStep(toDoc, afterStepJSON)) {
// operations have been applied
ops = [];
// console.log("%cop", logStyle, "- other", ops);
}
}
}
/** update node with attrs and marks, may also change type */
addSetNodeMarkup() {
// first diff in document is supposed to be a node-change (in type and/or attributes)
// thus simply find the first change and apply a node change step, then recalculate the diff
// after updating the document
const fromDoc = this.schema.nodeFromJSON(this.currentJSON);
const toDoc = this.schema.nodeFromJSON(this.finalJSON);
const start = toDoc.content.findDiffStart(fromDoc.content);
// @note start is the same (first) position for current and target document
const fromNode = fromDoc.nodeAt(start);
const toNode = toDoc.nodeAt(start);
if (start != null) {
// @note this completly updates all attributes in one step, by completely replacing node
const nodeType = fromNode.type === toNode.type ? null : toNode.type;
try {
this.tr.setNodeMarkup(start, nodeType, toNode.attrs, toNode.marks);
}
catch (e) {
// if nodetypes differ, the updated node-type and contents might not be compatible
// with schema and requires a replace
if (nodeType && e.message.includes("Invalid content")) {
// @todo add test-case for this scenario
this.tr.replaceWith(start, start + fromNode.nodeSize, toNode);
}
else {
throw e;
}
}
this.currentJSON = (0, removeMarks_1.removeMarks)(this.tr.doc).toJSON();
// setting the node markup may have invalidated the following ops, so we calculate them again.
this.ops = (0, rfc6902_1.createPatch)(this.currentJSON, this.finalJSON);
return true;
}
return false;
}
recreateChangeMarkSteps() {
// Now the documents should be the same, except their marks, so everything should map 1:1.
// Second step: Iterate through the toDoc and make sure all marks are the same in tr.doc
this.toDoc.descendants((tNode, tPos) => {
if (!tNode.isInline) {
return true;
}
this.tr.doc.nodesBetween(tPos, tPos + tNode.nodeSize, (fNode, fPos) => {
if (!fNode.isInline) {
return true;
}
const from = Math.max(tPos, fPos);
const to = Math.min(tPos + tNode.nodeSize, fPos + fNode.nodeSize);
fNode.marks.forEach((nodeMark) => {
if (!nodeMark.isInSet(tNode.marks)) {
this.tr.removeMark(from, to, nodeMark);
}
});
tNode.marks.forEach((nodeMark) => {
if (!nodeMark.isInSet(fNode.marks)) {
this.tr.addMark(from, to, nodeMark);
}
});
});
});
}
/**
* retrieve and possibly apply replace-step based from doc changes
* From http://prosemirror.net/examples/footnote/
*/
addReplaceStep(toDoc, afterStepJSON) {
const fromDoc = this.schema.nodeFromJSON(this.currentJSON);
const step = (0, getReplaceStep_1.getReplaceStep)(fromDoc, toDoc);
if (!step) {
return false;
}
else if (!this.tr.maybeStep(step).failed) {
this.currentJSON = afterStepJSON;
return true; // @change previously null
}
throw new Error("No valid step found.");
}
/** retrieve and possibly apply text replace-steps based from doc changes */
addReplaceTextSteps(op, afterStepJSON) {
// We find the position number of the first character in the string
const op1 = { ...op, value: "xx" };
const op2 = { ...op, value: "yy" };
const afterOP1JSON = (0, copy_1.copy)(this.currentJSON);
const afterOP2JSON = (0, copy_1.copy)(this.currentJSON);
(0, rfc6902_1.applyPatch)(afterOP1JSON, [op1]);
(0, rfc6902_1.applyPatch)(afterOP2JSON, [op2]);
const op1Doc = this.schema.nodeFromJSON(afterOP1JSON);
const op2Doc = this.schema.nodeFromJSON(afterOP2JSON);
// get text diffs
const finalText = op.value;
const currentText = (0, getFromPath_1.getFromPath)(this.currentJSON, op.path);
const textDiffs = this.wordDiffs
? (0, diff_1.diffWordsWithSpace)(currentText, finalText)
: (0, diff_1.diffChars)(currentText, finalText);
let offset = op1Doc.content.findDiffStart(op2Doc.content);
const marks = op1Doc.resolve(offset + 1).marks();
while (textDiffs.length) {
const diff = textDiffs.shift();
if (diff.added) {
const textNode = this.schema
.nodeFromJSON({ type: "text", text: diff.value })
.mark(marks);
if (textDiffs.length && textDiffs[0].removed) {
const nextDiff = textDiffs.shift();
this.tr.replaceWith(offset, offset + nextDiff.value.length, textNode);
}
else {
this.tr.insert(offset, textNode);
}
offset += diff.value.length;
}
else if (diff.removed) {
if (textDiffs.length && textDiffs[0].added) {
const nextDiff = textDiffs.shift();
const textNode = this.schema
.nodeFromJSON({ type: "text", text: nextDiff.value })
.mark(marks);
this.tr.replaceWith(offset, offset + diff.value.length, textNode);
offset += nextDiff.value.length;
}
else {
this.tr.delete(offset, offset + diff.value.length);
}
}
else {
offset += diff.value.length;
}
}
this.currentJSON = afterStepJSON;
}
}
exports.RecreateTransform = RecreateTransform;
function recreateTransform(fromDoc, toDoc, options = {}) {
const recreator = new RecreateTransform(fromDoc, toDoc, options);
return recreator.init();
}
@@ -1,118 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
const vitest_1 = require("vitest");
const schema_basic_1 = require("@tiptap/pm/schema-basic");
const transform_1 = require("@tiptap/pm/transform");
const recreateTransform_1 = require("./recreateTransform");
/**
* recreateTransform diffs two documents and produces ProseMirror steps that turn
* `fromDoc` into `toDoc`. It is the backbone of collaborative/version diffing, so
* THE invariant that matters is: replaying the produced steps on `fromDoc` must
* reproduce `toDoc` exactly. Every test below re-applies the steps onto a fresh
* Transform seeded from `fromDoc` (not just trusting `tr.doc`) and asserts node
* equality with `.eq()`. If a regression makes any step wrong, the round-trip
* breaks and the test fails.
*/
// Real ProseMirror schema (the standard basic schema) with paragraph/heading +
// strong/em marks — the same primitives the editor diffs in production.
const doc = (...c) => schema_basic_1.schema.node("doc", null, c);
const p = (...c) => schema_basic_1.schema.node("paragraph", null, c.length ? c : undefined);
const h = (level, ...c) => schema_basic_1.schema.node("heading", { level }, c);
const t = (text, ...marks) => schema_basic_1.schema.text(text, marks.length ? marks : undefined);
const strong = schema_basic_1.schema.marks.strong.create();
const em = schema_basic_1.schema.marks.em.create();
// Replay the diff's steps onto a fresh Transform built from `fromDoc`. This is
// the faithful "apply(diff) == target" check — it exercises the actual Step
// objects rather than the transform's internal accumulated doc.
function applyDiff(fromDoc, toDoc, options) {
const tr = (0, recreateTransform_1.recreateTransform)(fromDoc, toDoc, options);
const replay = new transform_1.Transform(fromDoc);
tr.steps.forEach((s) => {
const result = replay.maybeStep(s);
if (result.failed)
throw new Error(`step failed: ${result.failed}`);
});
return replay.doc;
}
(0, vitest_1.describe)("recreateTransform round-trip (apply(diff) == target)", () => {
(0, vitest_1.it)("reconstructs the target on plain text insertion", () => {
// Inserting " world" must yield exactly the target paragraph.
const from = doc(p(t("hello")));
const to = doc(p(t("hello world")));
(0, vitest_1.expect)(applyDiff(from, to).eq(to)).toBe(true);
});
(0, vitest_1.it)("reconstructs the target on text deletion", () => {
// Deleting a trailing word is the inverse of insertion and must round-trip.
const from = doc(p(t("hello world")));
const to = doc(p(t("hello")));
(0, vitest_1.expect)(applyDiff(from, to).eq(to)).toBe(true);
});
(0, vitest_1.it)("reconstructs the target when a word is replaced mid-string", () => {
// A char-level replace in the middle must not corrupt the surrounding text.
const from = doc(p(t("the quick brown fox")));
const to = doc(p(t("the slow brown fox")));
(0, vitest_1.expect)(applyDiff(from, to).eq(to)).toBe(true);
});
(0, vitest_1.it)("reconstructs the target when a mark is added (complexSteps path)", () => {
// Mark-only changes are diffed in a separate pass; the bolded run must match.
const from = doc(p(t("hello")));
const to = doc(p(t("hello", strong)));
const out = applyDiff(from, to);
(0, vitest_1.expect)(out.eq(to)).toBe(true);
// Sanity: the produced doc actually carries the strong mark.
(0, vitest_1.expect)(out.firstChild.firstChild.marks.length).toBe(1);
});
(0, vitest_1.it)("reconstructs the target when a mark is removed", () => {
// Removing the only mark must leave the same text with no marks.
const from = doc(p(t("hello", strong)));
const to = doc(p(t("hello")));
const out = applyDiff(from, to);
(0, vitest_1.expect)(out.eq(to)).toBe(true);
(0, vitest_1.expect)(out.firstChild.firstChild.marks.length).toBe(0);
});
(0, vitest_1.it)("reconstructs the target on a paragraph split into two blocks", () => {
// Structural change (one block -> two) must replay as valid replace steps.
const from = doc(p(t("hello world")));
const to = doc(p(t("hello")), p(t("world")));
const out = applyDiff(from, to);
(0, vitest_1.expect)(out.eq(to)).toBe(true);
(0, vitest_1.expect)(out.childCount).toBe(2);
});
(0, vitest_1.it)("reconstructs the target on a node-type change (paragraph -> heading)", () => {
// Type/attrs changes drive the setNodeMarkup branch; the node must become a
// heading while keeping its text.
const from = doc(p(t("hello")));
const to = doc(h(1, t("hello")));
const out = applyDiff(from, to);
(0, vitest_1.expect)(out.eq(to)).toBe(true);
(0, vitest_1.expect)(out.firstChild.type.name).toBe("heading");
});
(0, vitest_1.it)("reconstructs a combined structural + mark change", () => {
// Several diff kinds at once (new block + italic run) still round-trips.
const from = doc(p(t("alpha")));
const to = doc(p(t("alpha")), p(t("beta", em)));
const out = applyDiff(from, to);
(0, vitest_1.expect)(out.eq(to)).toBe(true);
});
(0, vitest_1.it)("produces an empty step list for identical documents", () => {
// No diff => no work; spurious steps would mean wasted/incorrect history.
const from = doc(p(t("same")));
const to = doc(p(t("same")));
const tr = (0, recreateTransform_1.recreateTransform)(from, to);
(0, vitest_1.expect)(tr.steps.length).toBe(0);
(0, vitest_1.expect)(tr.doc.eq(to)).toBe(true);
});
(0, vitest_1.it)("round-trips with complexSteps:false (marks diffed as replaces)", () => {
// With complexSteps off, mark changes are folded into replace steps rather
// than dedicated mark steps — the result must still equal the target.
const from = doc(p(t("hello")));
const to = doc(p(t("hello", strong)));
(0, vitest_1.expect)(applyDiff(from, to, { complexSteps: false }).eq(to)).toBe(true);
});
(0, vitest_1.it)("round-trips with wordDiffs:true (whole-word text diffing)", () => {
// wordDiffs changes the granularity of the text diff, not the outcome.
const from = doc(p(t("the quick brown fox")));
const to = doc(p(t("the quick red fox")));
(0, vitest_1.expect)(applyDiff(from, to, { wordDiffs: true }).eq(to)).toBe(true);
});
});
@@ -1,9 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.removeMarks = removeMarks;
const transform_1 = require("@tiptap/pm/transform");
function removeMarks(doc) {
const tr = new transform_1.Transform(doc);
tr.removeMark(0, doc.nodeSize - 2);
return tr.doc;
}
@@ -1,27 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
exports.simplifyTransform = simplifyTransform;
const transform_1 = require("@tiptap/pm/transform");
const getReplaceStep_1 = require("./getReplaceStep");
// join adjacent ReplaceSteps
function simplifyTransform(tr) {
if (!tr.steps.length) {
return undefined;
}
const newTr = new transform_1.Transform(tr.docs[0]);
const oldSteps = tr.steps.slice();
while (oldSteps.length) {
let step = oldSteps.shift();
while (oldSteps.length && step.merge(oldSteps[0])) {
const addedStep = oldSteps.shift();
if (step instanceof transform_1.ReplaceStep && addedStep instanceof transform_1.ReplaceStep) {
step = (0, getReplaceStep_1.getReplaceStep)(newTr.doc, addedStep.apply(step.apply(newTr.doc).doc).doc);
}
else {
step = step.merge(addedStep);
}
}
newTr.step(step);
}
return newTr;
}
@@ -1,2 +0,0 @@
"use strict";
Object.defineProperty(exports, "__esModule", { value: true });
File diff suppressed because it is too large Load Diff
-133
View File
@@ -1,133 +0,0 @@
import { randomUUID } from "node:crypto";
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
import { isInitializeRequest } from "@modelcontextprotocol/sdk/types.js";
import { createDocmostMcpServer } from "./index.js";
/**
* Build a stateful Streamable-HTTP handler for the Docmost MCP server. The
* embedding host (the gitmost NestJS server) bridges its raw Node req/res into
* `handleRequest`. One McpServer + transport is created per MCP session and
* kept alive between requests, keyed by the `mcp-session-id` header.
*
* `config` is EITHER a static `DocmostMcpConfig` (back-compat: stdio + the env
* service account, unchanged) OR a `McpConfigResolver` run once per session at
* `initialize` to bind that session to the request's identity.
*/
export function createMcpHttpHandler(config, options = {}) {
// One transport (and one McpServer) per MCP session, keyed by session id.
const transports = {};
// Last activity timestamp per session id, used for idle eviction.
const lastSeen = {};
// Anti-session-fixation: the opaque identity key bound to each session at
// initialize. A later request for that session whose key differs is rejected.
const sessionIdentity = {};
// Write a JSON-RPC error and end the response. Used for the 400/401 paths so
// every early rejection is a well-formed JSON-RPC error, not a torn response.
const sendJsonRpcError = (res, statusCode, code, message) => {
res.statusCode = statusCode;
res.setHeader("Content-Type", "application/json");
res.end(JSON.stringify({
jsonrpc: "2.0",
error: { code, message },
id: null,
}));
};
// Idle session TTL (ms): a session with no activity for this long is evicted.
// Defaults to 30 min; overridable via MCP_SESSION_IDLE_MS.
const idleTtlMs = (() => {
const parsed = parseInt(process.env.MCP_SESSION_IDLE_MS ?? "", 10);
return Number.isFinite(parsed) && parsed > 0 ? parsed : 30 * 60 * 1000;
})();
// Periodically close transports idle longer than the TTL. transport.close()
// triggers its onclose, which removes it from `transports`; we also drop the
// lastSeen entry. unref() so this timer never keeps the process alive.
const sweepIntervalMs = 5 * 60 * 1000;
const sweepTimer = setInterval(() => {
const now = Date.now();
for (const sid of Object.keys(transports)) {
if (now - (lastSeen[sid] ?? 0) > idleTtlMs) {
void transports[sid].close();
delete lastSeen[sid];
delete sessionIdentity[sid];
}
}
}, sweepIntervalMs);
sweepTimer.unref();
async function handleRequest(req, res, parsedBody) {
const sessionId = req.headers["mcp-session-id"];
const method = (req.method || "GET").toUpperCase();
let transport = sessionId ? transports[sessionId] : undefined;
if (method === "POST" && !transport) {
// A new session may only be created by an initialize request without a
// session id.
if (sessionId || !isInitializeRequest(parsedBody)) {
sendJsonRpcError(res, 400, -32000, "Bad Request: no valid session ID provided");
return;
}
// Resolve the per-session config from the request (per-user identity) when
// a resolver was supplied; otherwise use the static config unchanged. The
// resolver may throw (e.g. bad credentials) — surface a clean 401, never
// a created session.
let sessionConfig;
let identity;
try {
sessionConfig =
typeof config === "function" ? await config(req) : config;
if (options.identify)
identity = await options.identify(req);
}
catch (err) {
sendJsonRpcError(res, 401, -32001, err instanceof Error ? err.message : "Unauthorized");
return;
}
transport = new StreamableHTTPServerTransport({
sessionIdGenerator: () => randomUUID(),
onsessioninitialized: (sid) => {
transports[sid] = transport;
lastSeen[sid] = Date.now();
// Bind the resolved identity to the new session id for anti-fixation.
if (identity !== undefined)
sessionIdentity[sid] = identity;
},
});
transport.onclose = () => {
const sid = transport.sessionId;
if (sid && transports[sid])
delete transports[sid];
if (sid)
delete sessionIdentity[sid];
};
const server = createDocmostMcpServer(sessionConfig);
await server.connect(transport);
await transport.handleRequest(req, res, parsedBody);
return;
}
if (!transport) {
sendJsonRpcError(res, 400, -32000, "Bad Request: no valid session ID provided");
return;
}
// Anti-session-fixation: a request reusing an existing session id must
// present credentials/token that resolve to the SAME identity bound at
// initialize, otherwise reject with 401. This prevents hijacking another
// user's established session by replaying its session id with different
// credentials.
if (options.identify && sessionId && sessionId in sessionIdentity) {
let presented;
try {
presented = await options.identify(req);
}
catch (err) {
sendJsonRpcError(res, 401, -32001, err instanceof Error ? err.message : "Unauthorized");
return;
}
if (presented !== sessionIdentity[sessionId]) {
sendJsonRpcError(res, 401, -32001, "Credentials do not match the user that owns this MCP session.");
return;
}
}
// Routing to an existing transport: refresh its idle timestamp.
if (sessionId)
lastSeen[sessionId] = Date.now();
await transport.handleRequest(req, res, parsedBody);
}
return { handleRequest };
}
-810
View File
@@ -1,810 +0,0 @@
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { z } from "zod";
import { readFileSync } from "fs";
import { fileURLToPath } from "url";
import { dirname, join } from "path";
import { DocmostClient } from "./client.js";
import { parseNodeArg } from "./lib/parse-node-arg.js";
import { SHARED_TOOL_SPECS } from "./tool-specs.js";
// Re-export the client and its config type so embedding hosts (e.g. the gitmost
// NestJS server) can `import('@docmost/mcp')` and construct a DocmostClient
// directly — for the credentials variant OR the per-user getToken variant.
export { DocmostClient } from "./client.js";
// Re-export the zod-agnostic shared tool-spec registry so the in-app AI-SDK
// service can read it off the loaded module (it cannot import the ESM package's
// internals directly; it goes through loadDocmostMcp()).
export { SHARED_TOOL_SPECS } from "./tool-specs.js";
// Read version from package.json
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const packageJson = JSON.parse(readFileSync(join(__dirname, "../package.json"), "utf-8"));
const VERSION = packageJson.version;
// Configuration for an MCP server instance is the DocmostMcpConfig union
// (credentials OR getToken) defined and re-exported above. The factory below is
// fully side-effect-free on import: it reads no environment variables and opens
// no transport. The standalone stdio entrypoint (stdio.ts) and the HTTP handler
// (http.ts) supply this config and own the process/transport lifecycle.
// --- Modern McpServer Implementation ---
// Editing guide surfaced to MCP clients in the initialize result so they can
// pick the right tool by intent and avoid resending whole documents.
//
// MAINTENANCE RULE: when you ADD, RENAME, or REMOVE a tool (either an inline
// server.registerTool(...) here or a spec in tool-specs.ts), you MUST update
// this guide so the new tool is routed by intent. This is enforced by
// test/unit/server-instructions.test.mjs, which fails when a registered tool
// name is not mentioned below (see its EXCEPTIONS list for the rare opt-outs).
// Exported for that test.
export const SERVER_INSTRUCTIONS = "Docmost editing guide — choose the tool by intent.\n" +
"READ: find a page -> search (workspace-wide full-text); list -> list_pages / list_spaces. Locate blocks and their ids CHEAPLY -> get_outline (compact top-level map; start here, not get_page_json). One block's subtree -> get_node (by attrs.id, or \"#<index>\" for tables, which carry no id). Find every occurrence of a string/regex ON a page (and where each is) -> search_in_page, NOT block-by-block get_node — it returns each hit's node ref + block index + context for a targeted comment. Whole page -> get_page (Markdown, lossy; inline <span data-comment-id> tags are comment anchors — markup, not text) or get_page_json (lossless ProseMirror with block ids). Hand a huge page (with images) to an external consumer without pulling it through the model context -> stash_page (returns a short-lived anonymous URL).\n" +
"EDIT: fix wording/typos/numbers -> edit_page_text (find/replace inside blocks, no node id needed). Change ONE block (paragraph/heading/callout/etc.) structurally -> patch_node (by attrs.id from get_outline). Add a block -> insert_node (before/after a block by attrs.id or by anchor text, or append). Remove a block -> delete_node (by attrs.id). Tables -> table_get / table_update_cell / table_insert_row / table_delete_row (address by \"#<index>\" from get_outline; table nodes have no attrs.id). Images -> insert_image (add from a web URL) / replace_image (swap an existing image). Footnotes -> insert_footnote. Bulk/structural rewrite -> update_page_json (full ProseMirror replace; prefer the granular tools above to avoid resending the whole ~100KB+ document). Complex/scripted rewrite (multiple coordinated edits, renumbering) -> docmost_transform: write a JS `(doc, ctx) => doc` transform, preview the diff with dryRun (default), then apply with dryRun:false; ctx.helpers includes commentsToFootnotes for turning inline comments into numbered footnotes.\n" +
"PAGES: new -> create_page (Markdown). Rename (title only) -> rename_page. Move -> move_page. Delete -> delete_page (SOFT delete — the page goes to trash and is restorable; nothing is permanent). Copy/replace a page's whole content from another page (server-side, no document through the model) -> copy_page_content. Sharing -> share_page / unshare_page / list_shares; share_page makes the page PUBLICLY accessible — do it only when explicitly asked.\n" +
"COMMENTS: create_comment is always inline and requires an EXACT selection — contiguous text from a single block, <=250 chars (fails rather than leaving an unanchored comment); reply to a thread via parentCommentId. Propose a concrete text fix for one-click human approval -> create_comment with suggestedText (the exact plain-text replacement for the selection; the selection must then be UNIQUE in the page — extend it with context if needed); prefer this over editing directly when the change is subjective or needs the author's sign-off. Manage -> list_comments, update_comment, resolve_comment (resolve/reopen, reversible — prefer over delete to close), delete_comment, check_new_comments.\n" +
"HISTORY: review what changed -> diff_page_versions (a historyId vs current, or two versions). List saved versions -> list_page_history. Undo a bad edit -> restore_page_version (writes a past version back as current; itself revertible). Lossless markdown round-trip (download, edit, re-upload, incl. comment anchors) -> export_page_markdown / import_page_markdown.";
// Helper to format JSON responses
const jsonContent = (data) => ({
content: [{ type: "text", text: JSON.stringify(data, null, 2) }],
});
/**
* Create a fully configured Docmost MCP server. Side-effect-free: it does not
* read environment variables and does not connect any transport the caller
* decides how to expose it (stdio or HTTP). The client talks to Docmost over
* REST + the collaboration WebSocket using the provided service-account
* credentials and auto-re-authenticates.
*/
export function createDocmostMcpServer(config) {
// Pass the whole config union through: the client branches internally on
// credentials vs. getToken, so both the external /mcp (creds) and the
// internal per-user (getToken) paths are wired here unchanged.
const docmostClient = new DocmostClient(config);
const server = new McpServer({
name: "docmost-mcp",
version: VERSION,
}, { instructions: SERVER_INSTRUCTIONS });
// Register a tool from the shared, zod-agnostic spec registry. The spec owns
// the canonical name + model-facing description + (optional) schema builder;
// only the execute body is supplied per call. buildShape is invoked with THIS
// package's zod (v3); the in-app layer passes its own zod (v4).
//
// The spec's schema builder returns a plain ZodRawShape (Record<string,
// unknown> in the shared module since it must stay zod-agnostic), so the
// McpServer.registerTool overloads cannot infer the execute arg's shape from
// it. We type `execute` loosely and cast the call through `any`; runtime
// behaviour is unchanged — each execute body destructures the same fields the
// builder declares.
const registerShared = (spec, execute) => server.registerTool(spec.mcpName, spec.buildShape
? { description: spec.description, inputSchema: spec.buildShape(z) }
: { description: spec.description }, execute);
// Tool: get_workspace
registerShared(SHARED_TOOL_SPECS.getWorkspace, async () => {
const workspace = await docmostClient.getWorkspace();
return jsonContent(workspace);
});
// Tool: list_spaces
registerShared(SHARED_TOOL_SPECS.listSpaces, async () => {
const spaces = await docmostClient.getSpaces();
return jsonContent(spaces);
});
// Tool: list_pages
// INTENTIONAL per-transport divergence (not in the shared registry): this
// transport exposes a `tree:true` mode that returns the full nested hierarchy;
// the in-app copy keeps the same tree option but is worded for the in-app agent.
// Kept per-layer so each side can tune its own guidance.
server.registerTool("list_pages", {
description: "List most recent pages in a space ordered by updatedAt (descending). " +
"Returns a bounded list (default 50, max 100) — use search for lookups " +
"in large spaces. Pass tree:true (with spaceId) to instead get the " +
"space's full page hierarchy as a nested tree.",
inputSchema: {
spaceId: z.string().optional(),
limit: z
.number()
.int()
.min(1)
.max(100)
.optional()
.describe("Max pages to return (default 50, max 100)"),
tree: z
.boolean()
.optional()
.describe("When true, return the space's full page hierarchy as a nested tree (each node has a children array) instead of the recent-by-updatedAt flat list. Requires spaceId; ignores limit."),
},
}, async ({ spaceId, limit, tree }) => {
const result = await docmostClient.listPages(spaceId, limit ?? 50, tree ?? false);
return jsonContent(result);
});
// Tool: get_page
server.registerTool("get_page", {
description: "Get page details with content converted to Markdown. The conversion is " +
"LOSSY (block ids, exact table/callout structure are approximated); for a " +
"lossless representation use get_page_json. Inline <span data-comment-id> " +
"tags in the markdown are comment highlight anchors (also present for " +
"RESOLVED threads) — treat them as markup, not page text.",
inputSchema: {
pageId: z.string().min(1),
},
}, async ({ pageId }) => {
const page = await docmostClient.getPage(pageId);
return jsonContent(page);
});
// Tool: get_page_json
registerShared(SHARED_TOOL_SPECS.getPageJson, async ({ pageId }) => {
const page = await docmostClient.getPageJson(pageId);
return jsonContent(page);
});
// Tool: get_outline
registerShared(SHARED_TOOL_SPECS.getOutline, async ({ pageId }) => {
const result = await docmostClient.getOutline(pageId);
return jsonContent(result);
});
// Tool: get_node
registerShared(SHARED_TOOL_SPECS.getNode, async ({ pageId, nodeId }) => {
const result = await docmostClient.getNode(pageId, nodeId);
return jsonContent(result);
});
// Tool: search_in_page
registerShared(SHARED_TOOL_SPECS.searchInPage, async ({ pageId, query, regex, caseSensitive, limit }) => {
const result = await docmostClient.searchInPage(pageId, query, {
regex,
caseSensitive,
limit,
});
return jsonContent(result);
});
// Tool: table_get
server.registerTool("table_get", {
description: "Read a table as a matrix. Returns {rows, cols, cells (text[][]), " +
"cellIds (paragraph id per cell, or null)}. `table` = `#<index>` from " +
"get_outline, or any block id inside the table. Use cellIds with " +
"patch_node for rich-formatted cell edits. `cols` is the FIRST row's " +
"width; ragged tables may vary per row, so use the per-row length of " +
"`cells` for each row.",
inputSchema: {
pageId: z.string().min(1),
table: z.string().min(1),
},
}, async ({ pageId, table }) => {
const result = await docmostClient.getTable(pageId, table);
return jsonContent(result);
});
// Tool: table_insert_row
// NOT in the shared registry: this transport names the table argument `table`,
// while the in-app tool names it `tableRef` (ai-chat-tools.service.ts). Sharing
// one buildShape would rename a public MCP parameter, so the table row/cell
// tools stay per-transport by design.
server.registerTool("table_insert_row", {
description: "Insert a row of plain-text cells into a table. `table` = `#<index>` or " +
"a block id inside it. `cells` = text per column (padded to the table's " +
"column count; error if more cells than columns). `index` = 0-based " +
"insert position (0 inserts before the header); omit to append at the end.",
inputSchema: {
pageId: z.string().min(1),
table: z.string().min(1),
cells: z.array(z.string()),
index: z.number().int().optional(),
},
}, async ({ pageId, table, cells, index }) => {
const result = await docmostClient.tableInsertRow(pageId, table, cells, index);
return jsonContent(result);
});
// Tool: table_delete_row
// NOT shared — same `table` (here) vs `tableRef` (in-app) parameter-name
// divergence as table_insert_row.
server.registerTool("table_delete_row", {
description: "Delete the row at 0-based `index` from a table (`table` = `#<index>` or " +
"a block id inside it). Refuses to delete the table's only row. An " +
"out-of-range `index` throws. Deleting `index` 0 removes the header row, " +
"and the next row becomes the new header.",
inputSchema: {
pageId: z.string().min(1),
table: z.string().min(1),
index: z.number().int(),
},
}, async ({ pageId, table, index }) => {
const result = await docmostClient.tableDeleteRow(pageId, table, index);
return jsonContent(result);
});
// Tool: table_update_cell
// NOT shared — same `table` (here) vs `tableRef` (in-app) parameter-name
// divergence as table_insert_row.
server.registerTool("table_update_cell", {
description: "Set the plain-text content of cell [row,col] (0-based) in a table " +
"(`table` = `#<index>` or a block id inside it). Replaces the cell's " +
"content with a single text paragraph; for rich formatting use patch_node " +
"on the cell's paragraph id from table_get.",
inputSchema: {
pageId: z.string().min(1),
table: z.string().min(1),
row: z.number().int(),
col: z.number().int(),
text: z.string(),
},
}, async ({ pageId, table, row, col, text }) => {
const result = await docmostClient.tableUpdateCell(pageId, table, row, col, text);
return jsonContent(result);
});
// Tool: create_page
server.registerTool("create_page", {
description: "Create a new page from Markdown in a space. Pass parentPageId to nest " +
"it under a parent; omit it to create at the space root.",
inputSchema: {
title: z.string().min(1).describe("Title of the page"),
content: z.string().min(1).describe("Markdown content"),
spaceId: z.string().min(1),
parentPageId: z
.string()
.optional()
.describe("Optional parent page ID to nest under"),
},
}, async ({ title, content, spaceId, parentPageId }) => {
const result = await docmostClient.createPage(title, content, spaceId, parentPageId);
return jsonContent(result);
});
// Tool: update_page_json
server.registerTool("update_page_json", {
description: "Replace a page's content with a raw ProseMirror JSON document " +
"(lossless write: preserves the block ids, callouts, tables and " +
"attributes you pass in). Typical flow: get_page_json -> modify the " +
"JSON -> update_page_json. Keep existing node ids intact so heading " +
"anchors and history stay stable. Minimal full-doc example: " +
'{"type":"doc","content":[{"type":"paragraph","content":' +
'[{"type":"text","text":"Hi"}]}]}. `content` may be a JSON object or a ' +
"JSON string (both accepted), and is OPTIONAL: omit it to update only " +
"the title (though prefer rename_page for a title-only change). " +
"Supplying neither content nor title is an error.",
inputSchema: {
pageId: z.string().min(1).describe("ID of the page to update"),
content: z
.any()
.optional()
.describe('ProseMirror document {"type":"doc","content":[...]} (JSON object or ' +
"JSON string). Omit to rename only."),
title: z.string().optional().describe("Optional new title"),
},
}, async ({ pageId, content, title }) => {
// Only parse/validate the document when it was actually supplied; when it
// is omitted, pass it straight through so the client performs a title-only
// (or no-op) update.
let doc;
if (content === undefined || content === null) {
doc = undefined;
}
else {
// String -> JSON.parse (throwing on invalid); object passes through.
doc = parseNodeArg(content, "content was a string but not valid JSON");
}
const result = await docmostClient.updatePageJson(pageId, doc, title);
return jsonContent(result);
});
// Tool: export_page_markdown
server.registerTool("export_page_markdown", {
description: "Export a page to a single self-contained, lossless Docmost-flavoured " +
"Markdown file (custom extensions): YAML-free meta header, body with " +
"inline comment anchors and diagrams, and a trailing comments-thread " +
"block. Designed for a download -> edit body -> import_page_markdown " +
"round-trip that preserves everything, including comment highlights. " +
"Comment THREADS are preserved in the file but are not re-pushed to the " +
"server on import.",
inputSchema: {
pageId: z.string().min(1),
},
}, async ({ pageId }) => {
const md = await docmostClient.exportPageMarkdown(pageId);
return { content: [{ type: "text", text: md }] };
});
// Tool: import_page_markdown
registerShared(SHARED_TOOL_SPECS.importPageMarkdown, async ({ pageId, markdown }) => {
const res = await docmostClient.importPageMarkdown(pageId, markdown);
return jsonContent(res);
});
// Tool: copy_page_content
registerShared(SHARED_TOOL_SPECS.copyPageContent, async ({ sourcePageId, targetPageId }) => {
const result = await docmostClient.copyPageContent(sourcePageId, targetPageId);
return jsonContent(result);
});
// Tool: rename_page
server.registerTool("rename_page", {
description: "Rename a page (change its title only) without touching or resending " +
"its content.",
inputSchema: {
pageId: z.string().min(1).describe("ID of the page to rename"),
title: z.string().min(1).describe("New title"),
},
}, async ({ pageId, title }) => {
const result = await docmostClient.renamePage(pageId, title);
return jsonContent(result);
});
// Tool: edit_page_text
registerShared(SHARED_TOOL_SPECS.editPageText, async ({ pageId, edits }) => {
const result = await docmostClient.editPageText(pageId, edits);
return jsonContent(result);
});
// Tool: stash_page — returns a resource_link (NOT embedded text) so the doc
// body never enters the model context. Registered directly (not via
// registerShared) because that helper only emits text content. Also returns
// `structuredContent` carrying the full documented `{uri, sha256, size, images}`
// shape alongside the resource_link, so MCP clients receive the blob's sha256
// (its ETag, for integrity) and mirror counts, not just the link.
server.registerTool(SHARED_TOOL_SPECS.stashPage.mcpName, {
description: SHARED_TOOL_SPECS.stashPage.description,
inputSchema: SHARED_TOOL_SPECS.stashPage.buildShape(z),
}, async ({ pageId }) => {
const result = await docmostClient.stashPage(pageId);
return {
content: [
{
type: "resource_link",
uri: result.uri,
name: "page.json",
mimeType: "application/json",
size: result.size,
},
],
// Mirror the full documented result shape ({ uri, size, sha256, images })
// as structuredContent so MCP clients get the blob's sha256 (its ETag, for
// integrity) and the mirror counts, not just the resource_link.
structuredContent: {
uri: result.uri,
sha256: result.sha256,
size: result.size,
images: result.images,
},
};
});
// Tool: patch_node — schema + description from the shared registry (identical
// across both transports). The execute body keeps its own parseNodeArg
// normalization (the model sometimes serializes `node` as a JSON string).
registerShared(SHARED_TOOL_SPECS.patchNode, async ({ pageId, nodeId, node }) => {
const parsedNode = parseNodeArg(node);
const result = await docmostClient.patchNode(pageId, nodeId, parsedNode);
return jsonContent(result);
});
// Tool: insert_node — schema + description from the shared registry. As with
// patch_node, the execute body retains parseNodeArg on the incoming node.
registerShared(SHARED_TOOL_SPECS.insertNode, async ({ pageId, node, position, anchorNodeId, anchorText }) => {
const parsedNode = parseNodeArg(node);
const result = await docmostClient.insertNode(pageId, parsedNode, {
position,
anchorNodeId,
anchorText,
});
return jsonContent(result);
});
// Tool: delete_node
registerShared(SHARED_TOOL_SPECS.deleteNode, async ({ pageId, nodeId }) => {
const result = await docmostClient.deleteNode(pageId, nodeId);
return jsonContent(result);
});
// Tool: insert_image
server.registerTool("insert_image", {
description: "Download an image from a web (http/https) URL and insert it into " +
"a page in one step. By default " +
"appends the image at the end of the page. With replaceText, replaces the " +
"first top-level block whose text contains that string (handy for " +
'swapping a text placeholder like "[image: foo.png]" for the real image). ' +
"With afterText, inserts the image right after the first block containing " +
"that string. Preserves all other block ids.",
inputSchema: {
pageId: z.string().min(1),
imageUrl: z
.string()
.min(1)
.describe("http(s) URL of the image to download and upload"),
align: z.enum(["left", "center", "right"]).optional(),
alt: z.string().optional(),
replaceText: z
.string()
.optional()
.describe("Replace the first top-level block whose text contains this string with the image"),
afterText: z
.string()
.optional()
.describe("Insert the image right after the first top-level block whose text contains this string"),
},
}, async ({ pageId, imageUrl, align, alt, replaceText, afterText }) => {
const result = await docmostClient.insertImage(pageId, imageUrl, {
align,
alt,
replaceText,
afterText,
});
return jsonContent(result);
});
// Tool: replace_image
server.registerTool("replace_image", {
description: "Replace an existing image on a page with a new image fetched from a web " +
"(http/https) URL: uploads the new file as a NEW " +
"attachment (fresh clean URL that renders and busts browser caches), then " +
"repoints every image node referencing the old attachmentId (recursively, " +
"incl. callouts/tables) via the live document, preserving comments, " +
"alignment and alt. The old attachment is left as an unreferenced orphan " +
"(Docmost has no API to delete a single attachment; it is removed only when " +
"the page/space is deleted). In-place byte overwrite is avoided because some " +
"Docmost versions corrupt the attachment (HTTP 500) on overwrite.",
inputSchema: {
pageId: z.string().min(1),
attachmentId: z
.string()
.min(1)
.describe("attachmentId of the image currently in the page to replace"),
imageUrl: z
.string()
.min(1)
.describe("http(s) URL of the new image to download"),
align: z.enum(["left", "center", "right"]).optional(),
alt: z.string().optional(),
},
}, async ({ pageId, attachmentId, imageUrl, align, alt }) => {
const result = await docmostClient.replaceImage(pageId, attachmentId, imageUrl, {
align,
alt,
});
return jsonContent(result);
});
// Tool: share_page
// INTENTIONAL per-transport divergence (not shared): the in-app copy adds a
// security-confirmation framing ("only share when the user explicitly asked,
// since this exposes the page to anyone with the link") tuned for the in-app
// agent; this transport keeps the plain public-URL wording.
server.registerTool("share_page", {
description: "Make a page publicly accessible (idempotent) and return its public " +
"URL. The URL format is <app>/share/<key>/p/<slugId>. This exposes the " +
"page content to ANYONE with the URL — do it only when explicitly asked.",
inputSchema: {
pageId: z.string().min(1).describe("ID of the page to share"),
searchIndexing: z
.boolean()
.optional()
.describe("Allow search engines to index the page (default true)"),
},
}, async ({ pageId, searchIndexing }) => {
const result = await docmostClient.sharePage(pageId, searchIndexing ?? true);
return jsonContent(result);
});
// Tool: unshare_page
registerShared(SHARED_TOOL_SPECS.unsharePage, async ({ pageId }) => {
const result = await docmostClient.unsharePage(pageId);
return jsonContent(result);
});
// Tool: list_shares
registerShared(SHARED_TOOL_SPECS.listShares, async () => {
const result = await docmostClient.listShares();
return jsonContent(result);
});
// Tool: move_page
server.registerTool("move_page", {
description: "Move a page under a new parent (nesting) or to the space root.",
inputSchema: {
pageId: z.string().min(1),
parentPageId: z
.string()
.nullable()
.optional()
.describe("Target parent page ID. Pass 'null' or empty string to move to root."),
position: z
.string()
.min(5)
.optional()
.describe("fractional-index position key; min 5 chars; omit to append at the end."),
},
}, async ({ pageId, parentPageId, position }) => {
const finalParentId = parentPageId === "" || parentPageId === "null" ? null : parentPageId;
// Cheap cycle guard: a page cannot be moved directly under itself.
// (Deeper descendant-cycle detection is intentionally out of scope.)
if (finalParentId !== null && finalParentId === pageId) {
throw new Error("cannot move a page under itself");
}
const result = await docmostClient.movePage(pageId, finalParentId || null, position);
// Require POSITIVE confirmation: the live /pages/move success shape is
// exactly { success: true, status: 200 }. An empty body, a 204, or any odd
// shape lacking success === true must NOT be reported as a successful move,
// so we surface the raw API result instead of declaring success.
if (!(result && typeof result === "object" && result.success === true)) {
throw new Error(`Failed to move page ${pageId}: ${JSON.stringify(result)}`);
}
return jsonContent({
message: `Successfully moved page ${pageId} to parent ${finalParentId || "root"}`,
result,
});
});
// Tool: delete_page
server.registerTool("delete_page", {
description: "Delete a single page by ID. SOFT delete only: the page is moved to " +
"trash and can be restored; nothing is permanently deleted.",
inputSchema: {
pageId: z.string().min(1),
},
}, async ({ pageId }) => {
await docmostClient.deletePage(pageId);
return {
content: [
{ type: "text", text: `Successfully deleted page ${pageId}` },
],
};
});
// --- Comment tools (ported from upstream PR #3 by Max Nikitin) ---
// Tool: list_comments
server.registerTool("list_comments", {
description: "List ALL comments on a page in one call (pagination is handled " +
"internally), including RESOLVED threads — filter by resolvedAt when you " +
"need only open ones. Content is returned as Markdown.",
inputSchema: {
pageId: z.string().describe("ID of the page"),
},
}, async ({ pageId }) => {
const comments = await docmostClient.listComments(pageId);
return jsonContent(comments);
});
// Tool: create_comment
// INTENTIONAL per-transport divergence (not shared): the in-app copy tunes the
// guidance for the in-app agent (e.g. "retry with a corrected EXACT selection"
// and "Reversible via the comment UI"); this transport keeps its own wording.
server.registerTool("create_comment", {
description: "Create a new comment on a page. The comment is ALWAYS inline and is " +
"anchored to (highlights) its `selection` text — there are no page-level " +
"comments. Content is provided as Markdown and automatically converted. " +
"A top-level comment REQUIRES an exact `selection`; if the selection " +
"cannot be found in the page the call fails (no orphan comment is left). " +
"Replies (parentCommentId set) inherit the parent's anchor and take no " +
"selection. You may also attach a `suggestedText` proposing a replacement " +
"for the `selection`; a human applies (or rejects) it from the UI. When " +
"`suggestedText` is set the `selection` MUST occur exactly once in the " +
"page — expand it with surrounding context if it is ambiguous.",
inputSchema: {
pageId: z.string().describe("ID of the page to comment on"),
content: z.string().min(1).describe("Comment content in Markdown format"),
selection: z
.string()
.min(1)
// Enforce the documented 250-char cap to match the description above.
.max(250)
.optional()
.describe("EXACT contiguous text from a single paragraph/block to anchor the " +
"comment on (<=250 chars). Required for a top-level comment; omit " +
"only when replying via parentCommentId."),
parentCommentId: z
.string()
.optional()
.describe("Parent comment ID to create a reply (max 2 nesting levels)"),
suggestedText: z
.string()
.min(1)
.max(2000)
.optional()
.describe("Optional proposed replacement (PLAIN TEXT) for the `selection`, " +
"applied by a human via the UI (never auto-applied). REQUIRES a " +
"`selection`; NOT allowed on a reply. When set, the `selection` must " +
"be UNIQUE in the page — expand it with surrounding context (still " +
"<=250 chars) if it occurs more than once, or the call is refused."),
},
}, async ({ pageId, content, selection, parentCommentId, suggestedText }) => {
if (!parentCommentId && (!selection || !selection.trim())) {
throw new Error("create_comment: a 'selection' (exact text to anchor on) is required for a top-level comment; omit it only when replying via parentCommentId.");
}
if (suggestedText !== undefined) {
if (parentCommentId) {
throw new Error("create_comment: 'suggestedText' cannot be attached to a reply; it applies only to a top-level inline comment.");
}
if (!selection || !selection.trim()) {
throw new Error("create_comment: 'suggestedText' requires a 'selection' to anchor and rewrite.");
}
}
const result = await docmostClient.createComment(pageId, content, "inline", selection, parentCommentId, suggestedText);
return jsonContent(result);
});
// Tool: update_comment
server.registerTool("update_comment", {
description: "Update an existing comment's content. Only the comment creator can " +
"update it. Content is provided as Markdown.",
inputSchema: {
commentId: z.string().min(1).describe("ID of the comment to update"),
content: z
.string()
.min(1)
.describe("New comment content in Markdown format"),
},
}, async ({ commentId, content }) => {
const result = await docmostClient.updateComment(commentId, content);
return jsonContent(result);
});
// Tool: delete_comment
server.registerTool("delete_comment", {
description: "Delete a comment. Only the comment creator or space admin can delete it.",
inputSchema: {
commentId: z.string().min(1).describe("ID of the comment to delete"),
},
}, async ({ commentId }) => {
await docmostClient.deleteComment(commentId);
return {
content: [
{
type: "text",
text: `Successfully deleted comment ${commentId}`,
},
],
};
});
// Tool: resolve_comment
server.registerTool("resolve_comment", {
description: "Resolve (close) or reopen a comment thread. Only top-level comments can " +
"be resolved — the server rejects resolving a reply. Reversible: pass " +
"resolved=false to reopen. Resolving keeps the thread and its replies " +
"(unlike delete_comment, which permanently removes them).",
inputSchema: {
commentId: z
.string()
.min(1)
.describe("ID of the top-level comment thread to resolve or reopen"),
resolved: z
.boolean()
.optional()
.default(true)
.describe("true (default) marks the thread resolved/closed; false reopens it"),
},
}, async ({ commentId, resolved }) => {
const result = await docmostClient.resolveComment(commentId, resolved);
return jsonContent(result);
});
// Tool: check_new_comments
server.registerTool("check_new_comments", {
description: "Check for new comments across pages in a space since a given timestamp. " +
"Optionally scope to a page subtree (folder). Returns only comments " +
"created after the specified time.",
inputSchema: {
spaceId: z.string().describe("Space ID to check for new comments"),
since: z
.string()
.min(1)
.describe("ISO 8601 timestamp — only return comments created after this time (e.g. '2026-03-10T00:00:00Z')"),
parentPageId: z
.string()
.optional()
.describe("Optional root page ID to scope the check to a subtree (folder). " +
"Only pages under this parent will be checked."),
},
}, async ({ spaceId, since, parentPageId }) => {
// Reject an unparseable timestamp up front: otherwise the comparison
// against NaN silently treats every comment as "not new" and the tool
// returns zero results without signalling the bad input.
if (Number.isNaN(Date.parse(since))) {
throw new Error(`Invalid 'since' timestamp: ${JSON.stringify(since)} — expected an ISO 8601 date (e.g. '2026-03-10T00:00:00Z')`);
}
const result = await docmostClient.checkNewComments(spaceId, since, parentPageId);
return jsonContent(result);
});
// Tool: search
// INTENTIONAL per-transport divergence (not shared): the in-app `searchPages`
// runs a semantic + keyword hybrid (RRF) with in-process access control and a
// different schema (limit 1-20); this transport is a plain REST full-text search
// (limit up to 100). Different behaviour AND schema, so kept per-layer.
server.registerTool("search", {
description: "Full-text search for pages and content across the whole workspace. " +
"Results are bounded by `limit` (1-100; when omitted the server applies " +
"its own default).",
inputSchema: {
query: z.string().min(1).describe("Search query"),
limit: z
.number()
.int()
.min(1)
.max(100)
.optional()
.describe("Max results to return (max 100)"),
},
}, async ({ query, limit }) => {
// The tool exposes no spaceId filter, so pass undefined for the client's
// optional spaceId parameter and forward limit into its correct slot.
const result = await docmostClient.search(query, undefined, limit);
return jsonContent(result);
});
// Tool: docmost_transform
// INTENTIONAL per-transport divergence (not shared): the in-app `transformPage`
// deliberately omits the `deleteComments` schema field (comment-deletion
// guardrail) and carries a much shorter description; this transport exposes the
// full helper catalogue. Different schema, so kept per-layer.
server.registerTool("docmost_transform", {
description: "Edit a page by running an arbitrary JS transform `(doc, ctx) => doc` " +
"against its LIVE ProseMirror document, with a diff preview and page " +
"history as the safety net. By default dryRun=true: returns a diff " +
"preview WITHOUT writing. Set dryRun=false to apply (atomic, won't " +
"clobber concurrent edits). `doc` is the lossless ProseMirror document " +
"({type:'doc',content:[...]}); return a new doc of the same shape. " +
"`ctx` gives you: comments (the page's comments, each {id, content " +
"(markdown), selection, type}); log (array; console.log pushes to it); " +
"consume(id) (mark a comment id as consumed — those are deleted when " +
"deleteComments=true after a successful apply); and helpers: " +
"blockText(node) (plain text), walk(node, fn) (depth-first over all " +
"nodes incl. callouts/tables/lists), getList(doc, predicate) (find a " +
"node even without attrs.id), insertMarkerAfter(doc, anchor, marker, " +
"{beforeBlock}) (insert a plain unmarked text run after anchor, " +
"mark-safe), setCalloutRange(doc, n) (sync a [1]…[K] callout range to " +
"[1]…[n]), noteItem(inlineNodes) (wrap inline nodes in a listItem with a " +
"fresh id), mdToInlineNodes(markdown) (comment markdown -> inline nodes), " +
"commentsToFootnotes(doc, comments, {notesHeading}) (turn inline " +
"comments into numbered footnotes), canonicalizeFootnotes(doc) (derive " +
"footnote numbering + the single bottom list from reference order, drop " +
"orphans/duplicates — runs AUTOMATICALLY on the transform RESULT, so the " +
"applied (and dryRun-previewed) doc is always footnote-canonical; a dryRun " +
"diff may therefore show footnote tidy-ups your script did not make, and " +
"it is idempotent after the first run), and " +
"insertInlineFootnote(doc, {anchorText, text}) (author-inline footnote: " +
"marker + dedup'd definition, list derived). Footnote convention: markers are " +
"plain '[N]' text in the body; the notes are an orderedList under a " +
"heading whose text is 'Примечания переводчика' (that is only the DEFAULT " +
"notesHeading — pass the notesHeading option to the helpers to use a " +
"heading matching the page's language). The transform runs " +
"sandboxed (no require/process/fs/network, 5s timeout) and must return a " +
"{type:'doc'} node.",
inputSchema: {
pageId: z.string().min(1),
transformJs: z
.string()
.min(1)
.describe("A JS function `(doc, ctx) => doc` (expression-arrow or " +
"parenthesized function). It receives a clone of the live doc and " +
"ctx (comments, log, consume(id), helpers: blockText/walk/getList/" +
"insertMarkerAfter/setCalloutRange/noteItem/mdToInlineNodes/" +
"commentsToFootnotes/canonicalizeFootnotes/insertInlineFootnote) " +
"and must return a {type:'doc'} node."),
dryRun: z
.boolean()
.optional()
.default(true)
.describe("Preview only (no write) when true (default)."),
deleteComments: z
.boolean()
.optional()
.default(false)
.describe("After a successful apply, delete every comment id passed to " +
"ctx.consume(id)."),
},
}, async ({ pageId, transformJs, dryRun, deleteComments }) => {
const result = await docmostClient.transformPage(pageId, transformJs, {
dryRun,
deleteComments,
});
return jsonContent(result);
});
// Tool: insert_footnote
server.registerTool("insert_footnote", {
description: "Insert an AUTHOR-INLINE footnote: you specify only WHERE (anchorText) " +
"and WHAT (text). The footnote marker is placed right after anchorText in " +
"the body, and the bottom footnotes list + the numbering are derived " +
"deterministically server-side. You do NOT assign a number, and you " +
"never see or edit the footnotes list — so footnotes cannot end up out " +
"of order, orphaned, or as a raw '[^id]' block. If a footnote with the " +
"SAME text already exists, its number is REUSED (one definition, several " +
"references). The write is atomic and won't clobber concurrent edits; if " +
"anchorText is not found, nothing is written and an error is returned.",
inputSchema: {
pageId: z.string().min(1),
anchorText: z
.string()
.min(1)
.describe("A snippet of existing body text; the footnote marker is inserted " +
"immediately after its first occurrence (mark-safe)."),
text: z
.string()
.min(1)
.describe("The footnote content as markdown (becomes the definition)."),
},
}, async ({ pageId, anchorText, text }) => {
const result = await docmostClient.insertFootnote(pageId, anchorText, text);
return jsonContent(result);
});
// Tool: diff_page_versions
registerShared(SHARED_TOOL_SPECS.diffPageVersions, async ({ pageId, from, to }) => {
const result = await docmostClient.diffPageVersions(pageId, from, to);
return jsonContent(result);
});
// Tool: list_page_history
registerShared(SHARED_TOOL_SPECS.listPageHistory, async ({ pageId, cursor }) => {
const result = await docmostClient.listPageHistory(pageId, cursor);
return jsonContent(result);
});
// Tool: restore_page_version
registerShared(SHARED_TOOL_SPECS.restorePageVersion, async ({ historyId }) => {
const result = await docmostClient.restorePageVersion(historyId);
return jsonContent(result);
});
return server;
}
-92
View File
@@ -1,92 +0,0 @@
import axios from "axios";
export async function getCollabToken(baseUrl, apiToken) {
try {
const response = await axios.post(`${baseUrl}/auth/collab-token`, {}, {
headers: {
Authorization: `Bearer ${apiToken}`,
"Content-Type": "application/json",
},
});
// console.error('Collab Token Response:', response.data);
// Response is wrapped in { data: { token: ... } }
return response.data.data?.token || response.data.token;
}
catch (error) {
if (axios.isAxiosError(error)) {
// Attach the HTTP status to the plain Error so callers (e.g.
// getCollabTokenWithReauth) can still detect a 401/403 after the
// original AxiosError has been wrapped away.
// Avoid leaking the full server response body by default; include only
// status + statusText. Append the body only when DEBUG is set.
let message = `Failed to get collab token: ${error.response?.status} ${error.response?.statusText}`;
if (process.env.DEBUG) {
message += ` - ${JSON.stringify(error.response?.data)}`;
}
const err = new Error(message);
err.status = error.response?.status;
throw err;
}
throw error;
}
}
/**
* Pure cookie-parsing helper extracted from `performLogin` so the parsing logic
* can be unit-tested without performing the login network request. Given the
* raw `Set-Cookie` header array from the login response, return the `authToken`
* cookie's value.
*
* Behavior (kept identical to the original inline logic):
* - throws if there is no Set-Cookie header at all;
* - matches the cookie NAME exactly (`authToken`), so a future
* `authTokenRefresh=...` cookie is NOT picked up (a `startsWith` would be);
* - returns everything after the FIRST `=` up to the first `;`, so a base64
* value containing `=` padding is preserved (a naive `split("=")` would
* truncate it);
* - cookie attributes after the first `;` (Path, HttpOnly, Expires, ) are
* ignored;
* - throws if no `authToken` cookie is present.
*/
export function extractAuthTokenFromSetCookie(cookies) {
if (!cookies) {
throw new Error("No Set-Cookie header found in login response");
}
// Match the cookie name exactly to avoid matching a future
// authTokenRefresh cookie (startsWith would catch it).
const authCookie = cookies.find((c) => {
const kv = c.split(";")[0];
return kv.slice(0, kv.indexOf("=")) === "authToken";
});
if (!authCookie) {
throw new Error("No authToken cookie found in login response");
}
// Take everything after the FIRST "=" up to the first ";".
// Splitting on "=" would truncate base64 values containing "=" padding.
const kv = authCookie.split(";")[0];
return kv.slice(kv.indexOf("=") + 1);
}
export async function performLogin(baseUrl, email, password) {
try {
const response = await axios.post(`${baseUrl}/auth/login`, {
email,
password,
});
// Extract token from Set-Cookie header
return extractAuthTokenFromSetCookie(response.headers["set-cookie"]);
}
catch (error) {
// Avoid leaking the full server response body by default; log only the
// HTTP status. Log the verbose body only when DEBUG is set.
if (axios.isAxiosError(error)) {
if (process.env.DEBUG) {
console.error("Login failed:", error.response?.data);
}
else {
console.error("Login failed:", error.response?.status);
}
}
else {
console.error("Login failed:", error.message);
}
throw error;
}
}
-743
View File
@@ -1,743 +0,0 @@
import { HocuspocusProvider } from "@hocuspocus/provider";
import { TiptapTransformer } from "@hocuspocus/transformer";
import * as Y from "yjs";
import WebSocket from "ws";
import { marked } from "marked";
import { generateJSON } from "@tiptap/html";
import { Node as PMNode } from "@tiptap/pm/model";
import { updateYFragment } from "y-prosemirror";
import { JSDOM } from "jsdom";
import { docmostExtensions, docmostSchema } from "./docmost-schema.js";
import { withPageLock } from "./page-lock.js";
import { sanitizeForYjs, findUnstorableAttr } from "./node-ops.js";
import { lexFootnoteLines } from "./footnote-lex.js";
import { canonicalizeFootnotes } from "./footnote-canonicalize.js";
import { summarizeChange } from "./diff.js";
/**
* Build the descriptive error for an opaque Yjs encode failure ("Unexpected
* content type"), shared by both encode paths (`buildYDoc` -> `toYdoc` and
* `applyDocToFragment` -> `updateYFragment`) so the message wording stays in one
* place. `label` names the stage that failed (diagnostic). `sanitizeForYjs`
* already stripped `undefined` attrs, so a remaining failure is pinpointed via
* `findUnstorableAttr`.
*/
function unstorableYjsError(safe, label, e) {
const bad = findUnstorableAttr(safe);
return new Error(`Failed to encode document to Yjs (${label}): ${e instanceof Error ? e.message : String(e)}.${bad ? ` Offending attribute: ${bad}.` : " A node/mark attribute likely holds a value Yjs cannot store (e.g. undefined)."}`);
}
// Setup DOM environment for Tiptap HTML parsing in Node.js
const dom = new JSDOM("<!DOCTYPE html><html><body></body></html>");
global.window = dom.window;
global.document = dom.window.document;
// @ts-ignore
global.Element = dom.window.Element;
// @ts-ignore
global.WebSocket = WebSocket;
// Navigator is read-only in newer Node versions and already exists
// global.navigator = dom.window.navigator;
/**
* Hard ceiling above which we skip callout preprocessing entirely. The linear
* scanner below has no quadratic blow-up, but we still cap input defensively so
* a pathological multi-megabyte payload cannot tie up the event loop; in that
* case the markdown is passed through verbatim (callouts are simply not
* detected) rather than risking a slow scan.
*/
const MAX_CALLOUT_PREPROCESS_BYTES = 4 * 1024 * 1024; // 4 MB
/** Matches an opening callout fence: `:::type` (type captured, lower-cased). */
const CALLOUT_OPEN_RE = /^:::\s*(\w+)\s*$/;
/** Matches a bare closing callout fence: `:::`. */
const CALLOUT_CLOSE_RE = /^:::\s*$/;
/** Matches the start/end of a code fence (``` or ~~~), capturing the marker. */
const CODE_FENCE_RE = /^(\s*)(`{3,}|~{3,})/;
/**
* Pre-process Docmost-flavoured markdown: convert `:::type ... :::`
* callout blocks (the syntax our markdown export produces) into HTML
* divs that the callout extension parses. The inner content is rendered
* through marked as regular markdown.
*
* Implemented as a single linear pass over the lines (no quadratic regex
* rescan). It:
* - tracks fenced code regions (```...``` and ~~~...~~~) and never treats a
* `:::` line that lives inside a code fence as a callout delimiter, so a
* callout body that itself contains a fenced code block with a `:::` line is
* no longer corrupted;
* - matches an opening `:::type` line with the next CLOSING `:::` at the SAME
* nesting level, supporting NESTED callouts via a depth counter (an inner
* `:::type` opens a deeper level and consumes a matching `:::`);
* - emits the same `<div data-type="callout" data-callout-type="TYPE">` output
* (inner rendered through marked) as the previous regex implementation.
*/
async function preprocessCallouts(markdown) {
// Defensive cap: skip preprocessing for pathologically large inputs.
if (markdown.length > MAX_CALLOUT_PREPROCESS_BYTES) {
return markdown;
}
// Recursively transform a slice of lines, converting top-level callouts in
// that slice into <div> blocks and rendering their inner content (which may
// itself contain nested callouts) through this same function.
const transform = async (lines) => {
const out = [];
let inCodeFence = false;
let codeFenceMarker = ""; // the exact run of backticks/tildes that opened it
let i = 0;
while (i < lines.length) {
const line = lines[i];
// Inside a code fence, only its matching closing fence is significant;
// everything else (including `:::` lines) is copied through verbatim.
if (inCodeFence) {
out.push(line);
const fence = line.match(CODE_FENCE_RE);
if (fence && fence[2].startsWith(codeFenceMarker[0]) &&
fence[2].length >= codeFenceMarker.length) {
inCodeFence = false;
codeFenceMarker = "";
}
i++;
continue;
}
// A code fence opening outside any callout body: enter code-fence mode.
const fenceOpen = line.match(CODE_FENCE_RE);
if (fenceOpen) {
inCodeFence = true;
codeFenceMarker = fenceOpen[2];
out.push(line);
i++;
continue;
}
// An opening callout fence: scan forward (with code-fence and nested
// callout awareness) for its matching closing `:::` at the same level.
const open = line.match(CALLOUT_OPEN_RE);
if (open) {
const type = open[1].toLowerCase();
const bodyLines = [];
let depth = 1;
let innerInCodeFence = false;
let innerCodeFenceMarker = "";
let j = i + 1;
for (; j < lines.length; j++) {
const bl = lines[j];
if (innerInCodeFence) {
const f = bl.match(CODE_FENCE_RE);
if (f && f[2].startsWith(innerCodeFenceMarker[0]) &&
f[2].length >= innerCodeFenceMarker.length) {
innerInCodeFence = false;
innerCodeFenceMarker = "";
}
bodyLines.push(bl);
continue;
}
const innerFence = bl.match(CODE_FENCE_RE);
if (innerFence) {
innerInCodeFence = true;
innerCodeFenceMarker = innerFence[2];
bodyLines.push(bl);
continue;
}
if (CALLOUT_OPEN_RE.test(bl)) {
depth++;
bodyLines.push(bl);
continue;
}
if (CALLOUT_CLOSE_RE.test(bl)) {
depth--;
if (depth === 0)
break; // matching close for THIS callout
bodyLines.push(bl);
continue;
}
bodyLines.push(bl);
}
if (j < lines.length) {
// Found the matching closing fence: render the body (recursively, so
// nested callouts are handled) and emit the callout div.
const inner = await transform(bodyLines);
const renderedInner = await marked.parse(inner);
out.push(`\n<div data-type="callout" data-callout-type="${type}">${renderedInner}</div>\n`);
i = j + 1; // skip past the closing `:::`
continue;
}
// No matching close (unterminated callout): treat the opener as a
// literal line and continue, preserving the original text.
out.push(line);
i++;
continue;
}
out.push(line);
i++;
}
return out.join("\n");
};
return transform(markdown.split("\n"));
}
/**
* Bridge marked's checkbox lists to TipTap task lists.
*
* marked renders GitHub task list items (`- [x] done`) as a plain
* `<ul><li><p><input type="checkbox" checked> text</p></li></ul>` WITHOUT the
* markup TipTap's TaskList/TaskItem extensions parse. This rewrites such lists
* into the shape those extensions expect:
* TaskList parseHTML matches `ul[data-type="taskList"]`,
* TaskItem matches `li[data-type="taskItem"]`,
* the checked state is read from `data-checked === "true"`.
*
* A list is only converted when it has at least one `<li>` and EVERY direct
* `<li>` contains a checkbox input. Both `<ul>` and `<ol>` are considered: a
* numbered checklist (`1. [x] a`, which marked renders as an `<ol>` of checkbox
* `<li>`s) would otherwise lose its task state. TipTap task lists are unordered,
* so a matching `<ol>` is emitted as `data-type="taskList"` exactly like a
* `<ul>`. Mixed or ordinary lists (including ordinary `<ol>` lists) are left
* untouched so they keep rendering as bullet/numbered lists. The marked `<p>`
* wrapper is kept inside the `<li>` because TaskItem content allows paragraphs.
*/
function bridgeTaskLists(html) {
// Cheap early-out: if the markup contains no checkbox input at all there is
// nothing to bridge, so skip the expensive JSDOM parse entirely. This is the
// common case (most pages have no task lists).
if (!/type=["']?checkbox/i.test(html)) {
return html;
}
// Defensive cap (consistent with preprocessCallouts): skip the bridge for
// pathologically large inputs rather than running a second expensive JSDOM
// parse on a multi-megabyte payload. The markup is passed through verbatim.
if (html.length > MAX_CALLOUT_PREPROCESS_BYTES) {
return html;
}
const dom = new JSDOM(html);
const document = dom.window.document;
// Collect the checkbox(es) that belong to THIS <li> directly: either direct
// child <input type="checkbox"> elements or ones inside the <li>'s direct <p>
// child (the shape marked emits: `<li><p><input type="checkbox"> text</p></li>`).
// Checkboxes nested deeper (e.g. inside a child <ul>/<ol>) are excluded so a
// bullet <li> that merely contains a nested task sublist is not misdetected.
// Raw inline HTML can put more than one checkbox in a single <li>; we gather
// ALL of them so none survive into the converted item.
const directCheckboxes = (li) => {
const found = [];
for (const child of Array.from(li.children)) {
if (child.tagName === "INPUT" &&
child.getAttribute("type") === "checkbox") {
found.push(child);
continue;
}
if (child.tagName === "P") {
for (const inp of Array.from(child.querySelectorAll(":scope > input[type='checkbox']"))) {
found.push(inp);
}
}
}
return found;
};
// Both <ul> and <ol> are candidates: an <ol> whose every direct <li> carries
// its own checkbox is a numbered checklist that must also become a taskList.
const lists = Array.from(document.querySelectorAll("ul, ol"));
for (const list of lists) {
// Only consider DIRECT child <li> elements; nested lists are handled by
// their own iteration of the outer loop.
const items = Array.from(list.children).filter((child) => child.tagName === "LI");
if (items.length === 0)
continue;
const itemCheckboxes = items.map((li) => directCheckboxes(li));
// Convert only when every direct <li> carries at least one OWN checkbox.
if (!itemCheckboxes.every((boxes) => boxes.length > 0))
continue;
// A numbered checklist arrives as an <ol>. We must NOT leave the tag as
// <ol> while tagging it data-type="taskList": generateJSON would then match
// BOTH the orderedList rule (tag ol) and the taskList rule (data-type),
// emitting a phantom empty orderedList beside the real taskList. So rename a
// qualifying <ol> to a <ul> — move its <li> children over and replace it —
// leaving only the taskList rule to match. Already-<ul> lists are unchanged.
let target = list;
if (list.tagName === "OL") {
const ul = document.createElement("ul");
// Carry over existing attributes (e.g. class) so nothing is silently lost.
for (const attr of Array.from(list.attributes)) {
ul.setAttribute(attr.name, attr.value);
}
// Move every child node (including the <li>s we collected) into the <ul>.
while (list.firstChild) {
ul.appendChild(list.firstChild);
}
list.replaceWith(ul);
target = ul;
}
target.setAttribute("data-type", "taskList");
items.forEach((li, index) => {
const boxes = itemCheckboxes[index];
// The first checkbox determines the checked state (matches the previous
// single-checkbox behaviour); any extras only need removing.
const input = boxes[0] ?? null;
li.setAttribute("data-type", "taskItem");
const checked = input != null &&
(input.hasAttribute("checked") || input.checked);
li.setAttribute("data-checked", checked ? "true" : "false");
// Remove ALL direct checkbox inputs so none survive into the content
// (a raw-inline-HTML <li> may carry more than one).
for (const box of boxes) {
box.remove();
}
});
}
return document.body.innerHTML;
}
// Mirror of packages/editor-ext footnote markdown handling. A `[^id]` inline
// marker becomes <sup data-footnote-ref data-id="id">, and `[^id]: text`
// definition lines are collected into a single <section data-footnotes>.
// Definition detection + fence handling are shared with analyzeFootnotes via
// lexFootnoteLines (footnote-lex.js). FOOTNOTE_REF_RE is the inline tokenizer's.
const FOOTNOTE_REF_RE = /\[\^([^\]\s]+)\]/;
function escapeFootnoteAttr(value) {
return String(value).replace(/&/g, "&amp;").replace(/"/g, "&quot;");
}
const footnoteRefMarkedExtension = {
name: "footnoteRef",
level: "inline",
start(src) {
return src.match(/\[\^/)?.index ?? -1;
},
tokenizer(src) {
const match = FOOTNOTE_REF_RE.exec(src);
if (match && match.index === 0) {
return { type: "footnoteRef", raw: match[0], id: match[1] };
}
return undefined;
},
renderer(token) {
return `<sup data-footnote-ref data-id="${escapeFootnoteAttr(token.id)}"></sup>`;
},
};
marked.use({ extensions: [footnoteRefMarkedExtension] });
/**
* Pull `[^id]: text` definition lines out of the body and render a single
* <section data-footnotes> for them (or "" when there are none).
*/
function extractFootnotes(markdown) {
const bodyLines = [];
const defs = [];
// Shared lexer (footnote-lex): a `[^id]: ...` line inside a ``` / ~~~ code
// block is inert and stays in the body verbatim; only real definition lines
// are pulled out. analyzeFootnotes() consumes the SAME lexer so its diagnostics
// match exactly what import keeps/strips (#166).
for (const tok of lexFootnoteLines(markdown)) {
if (!tok.inFence && tok.definition)
defs.push(tok.definition);
else
bodyLines.push(tok.line);
}
if (defs.length === 0)
return { body: markdown, section: "" };
// Duplicate definition ids: FIRST WINS, the rest are DROPPED (mirror of
// editor-ext extractFootnoteDefinitions). Reference markers are left untouched
// so repeated `[^a]` references reuse the single footnote (Pandoc semantics,
// #166). The dropped duplicate is surfaced to the caller via analyzeFootnotes
// (`duplicateDefinitions`), not silently lost. MUST stay in sync with the
// editor-ext mirror.
const firstById = new Map(); // id -> first definition text
for (const def of defs) {
if (!firstById.has(def.id))
firstById.set(def.id, def.text);
}
const inner = [...firstById.entries()]
.map(([id, text]) => `<div data-footnote-def data-id="${escapeFootnoteAttr(id)}"><p>${marked.parseInline(text || "")}</p></div>`)
.join("");
return {
body: bodyLines.join("\n"),
section: `<section data-footnotes>${inner}</section>`,
};
}
/**
* Convert markdown to a ProseMirror doc using the full Docmost schema.
*
* This conversion does NOT canonicalize footnotes it is the shared, content-
* preserving primitive used by BOTH page write paths and COMMENT bodies
* (createComment / updateComment). Canonicalization MUST NOT run on a comment
* body: a comment may legitimately contain a footnote-definition line
* (`[^1]: text`) with no matching reference, and the canonicalizer drops a
* reference-less footnotesList which would silently delete the comment's text.
*
* Page write paths that DO need the canonical footnote topology call
* `markdownToProseMirrorCanonical` instead (markdown import, update_page markdown
* path). Keep this function reference-loss-free.
*/
export async function markdownToProseMirror(markdownContent) {
const withCallouts = await preprocessCallouts(markdownContent);
const { body, section } = extractFootnotes(withCallouts);
const html = (await marked.parse(body)) + section;
const bridged = bridgeTaskLists(html);
return generateJSON(bridged, docmostExtensions);
}
/**
* Page-write variant of `markdownToProseMirror`: converts markdown then enforces
* the canonical footnote topology. The footnote `section` markdown is emitted in
* DEFINITION order, but numbering derives from REFERENCE order, so without this
* the bottom list renders out of order (`1, 4, 2, 3, …`); orphan definitions and
* duplicate lists are also normalized. Idempotent a no-op once canonical, and a
* no-op for footnote-free content.
*
* Use this ONLY for full-document PAGE writes (never for comment bodies, where it
* would drop a reference-less footnote definition see `markdownToProseMirror`).
*/
export async function markdownToProseMirrorCanonical(markdownContent) {
return canonicalizeFootnotes(await markdownToProseMirror(markdownContent));
}
/**
* Build the collaboration WebSocket URL from an API base URL:
* switch http(s)->ws(s), strip a trailing /api, mount on /collab.
* Shared by the live read and the mutate path so both target the same socket.
*/
export function buildCollabWsUrl(baseUrl) {
let wsUrl = baseUrl.replace(/^http/, "ws");
try {
const urlObj = new URL(wsUrl);
if (urlObj.pathname.endsWith("/api") || urlObj.pathname.endsWith("/api/")) {
urlObj.pathname = urlObj.pathname.replace(/\/api\/?$/, "");
}
urlObj.pathname = urlObj.pathname.replace(/\/$/, "") + "/collab";
// Drop any query/hash from the base URL so it is not carried into the
// collaboration ws URL.
urlObj.search = "";
urlObj.hash = "";
wsUrl = urlObj.toString();
}
catch (e) {
// Fallback if URL parsing fails
if (!wsUrl.endsWith("/collab")) {
wsUrl = wsUrl.replace(/\/$/, "") + "/collab";
}
}
return wsUrl;
}
/**
* Encode a ProseMirror doc to a Yjs document, sanitizing it first and turning
* the opaque yjs "Unexpected content type" failure into a descriptive error.
*
* `sanitizeForYjs` strips `undefined` node/mark attributes (the common cause of
* the failure); if `toYdoc` still throws, `findUnstorableAttr` is used to point
* at the offending attribute path.
*/
export function buildYDoc(doc) {
const safe = sanitizeForYjs(doc);
try {
return TiptapTransformer.toYdoc(safe, "default", docmostExtensions);
}
catch (e) {
throw unstorableYjsError(safe, "toYdoc", e);
}
}
/**
* Write a new ProseMirror doc into the live Yjs fragment by STRUCTURAL DIFF,
* preserving the Yjs identity of unchanged nodes (issue #152).
*
* The previous approach deleted the whole fragment and re-applied a fresh Y.Doc,
* which discarded every Yjs node id. y-prosemirror anchors the editor selection
* to those ids, so an open editor's cursor lost its anchor and snapped to the
* end of the document on every agent write (most visibly on comment anchoring,
* which changes no text at all). `updateYFragment` is exactly the routine the
* editor itself uses to sync ProseMirror edits into Yjs: it diffs the new node
* against the current fragment and touches only the changed children, so
* unchanged nodes keep their ids and the live cursor stays put.
*
* Must run inside a single `transact` so the diff applies atomically (no remote
* update interleaves). Keeps `buildYDoc`'s `findUnstorableAttr` diagnostic for
* the opaque "Unexpected content type" encode failure.
*/
export function applyDocToFragment(ydoc, newDoc) {
const safe = sanitizeForYjs(newDoc);
const fragment = ydoc.getXmlFragment("default");
// Hydrate the ProseMirror node in its OWN try so a failure here (e.g. an
// unknown node type) is labelled "fromJSON" — the stage that actually threw —
// instead of being misattributed to the Yjs write stage (#154 review).
let pmNode;
try {
pmNode = PMNode.fromJSON(docmostSchema, safe);
}
catch (e) {
throw unstorableYjsError(safe, "fromJSON", e);
}
try {
ydoc.transact(() => {
updateYFragment(ydoc, fragment, pmNode, {
mapping: new Map(),
isOMark: new Map(),
});
});
}
catch (e) {
throw unstorableYjsError(safe, "updateYFragment", e);
}
}
/**
* Run an independent Yjs-encodability check (the same `sanitizeForYjs` + schema
* the apply path uses) and throw the same descriptive error when the doc cannot
* be stored. Used by the dry-run preview.
*
* Note: it does NOT run `updateYFragment` against the live fragment, so it is an
* encodability GATE, not a byte-for-byte rehearsal of apply `buildYDoc`
* (`toYdoc`) and `applyDocToFragment` (`updateYFragment`) are two different
* encoders that nonetheless reject the same unstorable attributes. To narrow the
* preview/apply gap it ALSO rehearses the apply path's `PMNode.fromJSON`
* hydration, so a doc that would only fail there (e.g. an unknown node type) is
* rejected at preview time too (#154 review). Still cheap: no live fragment, no
* `updateYFragment`.
*/
export function assertYjsEncodable(doc) {
buildYDoc(doc);
const safe = sanitizeForYjs(doc);
try {
PMNode.fromJSON(docmostSchema, safe);
}
catch (e) {
throw unstorableYjsError(safe, "fromJSON", e);
}
}
/** Time we wait for the initial handshake/sync before giving up. */
const CONNECT_TIMEOUT_MS = 25000;
/** Time we wait for the server to acknowledge our write before giving up. */
const PERSIST_TIMEOUT_MS = 20000;
/**
* Safely mutate the live content of a page over the collaboration websocket.
*
* This is the single safe write path for every MCP content mutation. It:
* 1. serializes per-page writes through withPageLock (no two MCP writes on
* the same page overlap);
* 2. connects to Hocuspocus and waits for the initial sync so the local ydoc
* mirrors the authoritative server doc INCLUDING edits/comments/images
* that are not yet in the debounced REST snapshot;
* 3. inside onSynced, SYNCHRONOUSLY reads the live doc, runs `transform`, and
* writes the result back with no `await` between read and write so no
* remote update can interleave and clobber concurrent human edits;
* 4. waits for the server to acknowledge the write (unsyncedChanges -> 0)
* before resolving, so the next operation observes our change.
*
* `transform` receives the live ProseMirror doc and returns the NEW full
* ProseMirror doc to write, or `null` to abort with no write (a no-op). If
* `transform` throws, the error is propagated to the caller (not swallowed).
*
* Resolves a `MutationResult { doc, verify }`: `doc` is the doc that was
* written (or the live doc when the transform aborted), and `verify` is a
* verifiable change report (text/block/mark deltas) of what actually changed.
* The report is computed AFTER the atomic read->write, so it never widens the
* read->write window, and it never throws (it can NEVER break a write).
*/
export async function mutatePageContent(pageId, collabToken, baseUrl, transform) {
return withPageLock(pageId, () => {
if (process.env.DEBUG) {
console.error(`Starting realtime content mutate for page ${pageId}`);
// Token prefix is sensitive; only log it under DEBUG.
console.error(`Token prefix: ${collabToken ? collabToken.substring(0, 5) : "NONE"}...`);
}
const ydoc = new Y.Doc();
const wsUrl = buildCollabWsUrl(baseUrl);
if (process.env.DEBUG)
console.error(`Connecting to WebSocket: ${wsUrl}`);
return new Promise((resolve, reject) => {
let provider;
let applied = false; // onSynced may fire again on reconnect — apply once.
let settled = false;
// Set true on disconnect/close so a reconnect-driven unsyncedChanges->0
// cannot be mistaken for a successful persist of our write.
let connectionLost = false;
let connectTimer;
let persistTimer;
let unsyncedHandler;
const cleanup = () => {
if (connectTimer)
clearTimeout(connectTimer);
if (persistTimer)
clearTimeout(persistTimer);
if (provider) {
if (unsyncedHandler) {
try {
provider.off("unsyncedChanges", unsyncedHandler);
}
catch (err) { }
}
try {
provider.destroy();
}
catch (err) { }
}
};
const finish = (err, value) => {
if (settled)
return;
settled = true;
cleanup();
if (err)
reject(err);
else
resolve(value);
};
connectTimer = setTimeout(() => {
finish(new Error("Connection timeout to collaboration server"));
}, CONNECT_TIMEOUT_MS);
// Resolve once the server has acknowledged our update. The provider
// increments unsyncedChanges when our local update is sent and
// decrements it when the server replies with a SyncStatus(applied=true);
// reaching 0 means the authoritative in-memory ydoc on the server now
// contains our write.
const waitForPersistence = () => {
if (settled)
return;
// A missing provider is a failure, not a success: without it the write
// can never have been acknowledged. Only an actual unsyncedChanges===0
// on a live provider counts as persisted.
if (!provider) {
finish(new Error("collab provider gone before persistence"));
return;
}
if (provider.unsyncedChanges === 0) {
finish(null, mutationResult);
return;
}
persistTimer = setTimeout(() => {
finish(new Error("Timeout waiting for collaboration server to persist the update"));
}, PERSIST_TIMEOUT_MS);
unsyncedHandler = (data) => {
// Only treat unsyncedChanges->0 as success when the connection is
// still up. A transient disconnect + reconnect handshake can drive
// the counter back to 0 without our write being re-transmitted; in
// that case let the disconnect/close error win instead.
if (data.number === 0 && !connectionLost) {
finish(null, mutationResult);
}
};
provider.on("unsyncedChanges", unsyncedHandler);
};
// The verifiable result resolved on every success/abort path. Set on
// abort (no-op report) and after a real write (computed change report).
let mutationResult;
provider = new HocuspocusProvider({
url: wsUrl,
name: `page.${pageId}`,
document: ydoc,
token: collabToken,
// @ts-ignore - Required for Node.js environment
WebSocketPolyfill: WebSocket,
onConnect: () => {
if (process.env.DEBUG)
console.error("WS Connect");
},
// An unexpected disconnect/close while we are still waiting (during the
// connect-wait before onSynced, or during the persistence wait after the
// write) means the update will never be acknowledged — surface it now
// instead of hanging until the connect/persist timeout fires. `finish`
// is idempotent via the `settled` flag, so the onClose that our own
// cleanup()->provider.destroy() triggers (after settled=true is set) is
// a harmless no-op and cannot cause a double-resolve.
onDisconnect: () => {
if (process.env.DEBUG)
console.error("WS Disconnect");
// Mark BEFORE finish so the unsyncedChanges handler (if it races)
// sees the connection as lost and won't report a false success.
connectionLost = true;
finish(new Error("Collaboration connection closed before the update was persisted/synced"));
},
onClose: () => {
if (process.env.DEBUG)
console.error("WS Close");
// Mark BEFORE finish so the unsyncedChanges handler (if it races)
// sees the connection as lost and won't report a false success.
connectionLost = true;
finish(new Error("Collaboration connection closed before the update was persisted/synced"));
},
onSynced: () => {
if (applied || settled)
return;
applied = true;
if (process.env.DEBUG)
console.error("Connected and synced!");
// CRITICAL: everything between reading the live doc and writing it
// back must stay synchronous (no await). While the JS event loop is
// not yielded, no incoming remote update can interleave, so any
// already-synced concurrent edits are preserved in liveDoc.
let newDoc;
let beforeDoc;
try {
let liveDoc = TiptapTransformer.fromYdoc(ydoc, "default");
if (!liveDoc ||
typeof liveDoc !== "object" ||
!Array.isArray(liveDoc.content)) {
liveDoc = { type: "doc", content: [] };
}
// Snapshot the before-doc for the change report. Docs are
// JSON-serializable, so this is a safe deep clone.
beforeDoc = JSON.parse(JSON.stringify(liveDoc));
newDoc = transform(liveDoc);
if (newDoc == null) {
// Transform aborted — write nothing, return the live doc with a
// no-op change report.
mutationResult = {
doc: liveDoc,
verify: {
changed: false,
textInserted: 0,
textDeleted: 0,
blocksChanged: 0,
marks: {},
summary: "no changes (transform aborted)",
},
};
finish(null, mutationResult);
return;
}
// Structural diff into the live fragment (issue #152): preserves
// the Yjs ids of unchanged nodes, so an open editor's cursor is not
// yanked to the end of the document on every agent write.
applyDocToFragment(ydoc, newDoc);
}
catch (e) {
// Includes errors thrown by transform (e.g. "afterText not found",
// "text not found"): propagate them verbatim to the caller.
finish(e instanceof Error ? e : new Error(String(e)));
return;
}
// Compute the verifiable change report AFTER the transact write: it
// only needs the JSON before/after, so it cannot affect the atomic
// read->write window, and summarizeChange never throws.
mutationResult = {
doc: newDoc,
verify: summarizeChange(beforeDoc, newDoc),
};
if (process.env.DEBUG)
console.error("Content written, waiting for server to persist...");
waitForPersistence();
},
onAuthenticationFailed: () => {
finish(new Error("Authentication failed for collaboration connection"));
},
});
});
});
}
/**
* Replace the live content of a page over the collaboration websocket.
* Accepts a ready ProseMirror JSON document; the caller controls whether
* it was produced from markdown (ids regenerate) or edited in place
* (existing block ids preserved).
*
* This is an intentional full replace (used by update_page / update_page_json),
* but now runs under the per-page lock and waits for server persistence via
* mutatePageContent.
*/
export async function replacePageContent(pageId, prosemirrorDoc, collabToken, baseUrl) {
// Fail fast on a bad document instead of deferring the failure into the
// collaboration write (where TiptapTransformer.toYdoc(undefined) used to
// throw). The transform must return a valid ProseMirror doc.
if (prosemirrorDoc == null ||
typeof prosemirrorDoc !== "object" ||
prosemirrorDoc.type !== "doc") {
throw new Error("replacePageContent: invalid ProseMirror document");
}
return await mutatePageContent(pageId, collabToken, baseUrl, () => prosemirrorDoc);
}
/**
* Markdown update path (kept for backwards compatibility).
* NOTE: this re-imports the whole document block ids are regenerated.
* Tables and :::callout::: blocks survive thanks to the full schema.
*/
export async function updatePageContentRealtime(pageId, markdownContent, collabToken, baseUrl) {
// PAGE write: canonicalize footnotes (markdown import builds the bottom list in
// definition order; numbering is reference-ordered).
const tiptapJson = await markdownToProseMirrorCanonical(markdownContent);
return await mutatePageContent(pageId, collabToken, baseUrl, () => tiptapJson);
}
-371
View File
@@ -1,371 +0,0 @@
/**
* Inline-comment anchoring against a ProseMirror document.
*
* Docmost stores an inline comment's highlight as a `comment` MARK on the
* document text (`{ type: "comment", attrs: { commentId, resolved } }`); the
* `/comments/create` API only records the comment row + its `selection` text and
* does NOT insert that mark, so the anchor has to be written into the page
* content separately. This module finds where a selection lives in the document
* and splices the comment mark across the matched range.
*
* Matching has to be robust because the agent supplies the selection as plain
* text while the document stores rich inline content: a selection can span
* several adjacent text nodes (inline code / bold / links each become their own
* text node), and the document may use smart/typographic quotes, dash variants,
* non-breaking spaces, or collapsed runs of whitespace that the agent typed as
* ASCII quotes/hyphens/single spaces. We therefore normalize both sides before
* comparing and match across maximal runs of consecutive text nodes within a
* single block, while mapping every normalized character back to its raw index
* so the mark lands on the exact original characters.
*/
/** Typographic double-quote variants mapped to ASCII `"`. */
const DOUBLE_QUOTES = "«»„“”‟〝〞"";
/** Typographic single-quote/apostrophe variants mapped to ASCII `'`. */
const SINGLE_QUOTES = "‘’‚‛";
/** Dash variants mapped to ASCII `-`. */
const DASHES = "–—―−‐‑‒";
/** Guard against pathological/cyclic documents in the depth-first walk. */
const MAX_DEPTH = 200;
/** The comment mark Docmost stores on anchored text. */
function makeCommentMark(commentId) {
// The comment mark schema declares both commentId and resolved; include
// resolved:false for completeness so the stored mark matches the editor's.
return { type: "comment", attrs: { commentId, resolved: false } };
}
/** True for any character we collapse/replace with a single normal space. */
function isWhitespaceChar(ch) {
// Regular ASCII whitespace plus the special spaces called out in the spec:
// nbsp, narrow nbsp, en/em/thin/hair/figure spaces, etc. \s covers tab and
// newline; the explicit code points cover the non-breaking variants \s misses
// in some engines, so list them for determinism.
return (/\s/.test(ch) ||
ch === " " || // no-break space
ch === " " || // figure space
ch === " " || // narrow no-break space
ch === " " || // thin space
ch === " " || // hair space
ch === " " || // en space
ch === " " // em space
);
}
/**
* Normalize a string for matching and return both the normalized text and a
* `map` where `map[i]` is the index into the ORIGINAL `s` of the i-th
* normalized character.
*
* Rules: map smart quotes / dashes / special spaces to their ASCII forms,
* collapse any run of whitespace to a SINGLE space (whose map entry points at
* the FIRST raw whitespace char of the run), and DO NOT lowercase (anchoring is
* case-sensitive to match the exact document text).
*/
export function normalizeForMatch(s) {
let norm = "";
const map = [];
let i = 0;
while (i < s.length) {
const ch = s[i];
if (isWhitespaceChar(ch)) {
// Collapse the whole whitespace run to one space mapped to the run start.
const runStart = i;
while (i < s.length && isWhitespaceChar(s[i]))
i++;
norm += " ";
map.push(runStart);
continue;
}
let mapped = ch;
if (DOUBLE_QUOTES.indexOf(ch) !== -1)
mapped = '"';
else if (SINGLE_QUOTES.indexOf(ch) !== -1)
mapped = "'";
else if (DASHES.indexOf(ch) !== -1)
mapped = "-";
norm += mapped;
map.push(i);
i++;
}
return { norm, map };
}
/**
* Find a selection inside a SINGLE block's direct `content` array.
*
* Builds maximal runs of consecutive `text` nodes (any non-text inline node,
* e.g. a mention, breaks the run), normalizes each run and the selection the
* same way, then searches each run for the normalized selection. Returns the
* child/offset range of the FIRST matching run, or `null` if none match.
*/
export function findAnchorInBlock(blockContent, selection) {
if (!Array.isArray(blockContent))
return null;
const normSelObj = normalizeForMatch(selection);
// Trim leading/trailing spaces on the NORMALIZED selection only.
const normSel = normSelObj.norm.trim();
if (normSel.length === 0)
return null;
let i = 0;
while (i < blockContent.length) {
const node = blockContent[i];
if (!node || typeof node !== "object" || node.type !== "text") {
i++;
continue;
}
// Accumulate a maximal run of consecutive text nodes.
let rawRun = "";
const rawToChild = [];
let j = i;
while (j < blockContent.length) {
const n = blockContent[j];
if (!n || typeof n !== "object" || n.type !== "text")
break;
const text = typeof n.text === "string" ? n.text : "";
for (let k = 0; k < text.length; k++) {
rawToChild.push({ childIdx: j, offset: k });
}
rawRun += text;
j++;
}
// Try to match within this run.
const { norm, map } = normalizeForMatch(rawRun);
const idx = norm.indexOf(normSel);
if (idx !== -1) {
const rawStart = map[idx];
const rawEndExclusive = idx + normSel.length < map.length
? map[idx + normSel.length]
: rawRun.length;
const startLoc = rawToChild[rawStart];
// rawEndExclusive points at the raw char AFTER the match; the last matched
// raw char is at rawEndExclusive-1, so endOffset is its offset + 1.
const lastLoc = rawToChild[rawEndExclusive - 1];
return {
startChild: startLoc.childIdx,
startOffset: startLoc.offset,
endChild: lastLoc.childIdx,
endOffset: lastLoc.offset + 1,
};
}
// No match in this run: continue scanning AFTER it.
i = j > i ? j : i + 1;
}
return null;
}
/**
* Reconstruct the RAW text spanned by an AnchorMatch inside one block's
* `content` array. `startChild..endChild` are all text nodes (guaranteed by
* findAnchorInBlock, which only builds runs of `text` nodes), so concatenate
* each node's text slice: from `startOffset` on the first node, up to
* `endOffset` on the last, and the whole `.text` for any node fully inside the
* range. Mirrors spliceCommentMark's per-node slicing so the string returned
* here is EXACTLY the characters the comment mark will cover.
*/
function reconstructRawText(blockContent, match) {
const { startChild, startOffset, endChild, endOffset } = match;
let out = "";
for (let k = startChild; k <= endChild; k++) {
const n = blockContent[k];
const text = typeof n.text === "string" ? n.text : "";
const sliceStart = k === startChild ? startOffset : 0;
const sliceEnd = k === endChild ? endOffset : text.length;
out += text.slice(sliceStart, sliceEnd);
}
return out;
}
/**
* Return the RAW document substring that `selection` would anchor to the exact
* characters the comment mark will cover or `null` when the selection cannot
* be anchored anywhere in `doc`.
*
* This mirrors canAnchorInDoc / applyAnchorInDoc EXACTLY (same depth-first,
* document-order traversal and the same findAnchorInBlock match on the FIRST
* matching block), but instead of a boolean / an in-place mutation it
* reconstructs the raw text spanned by the matched range. Because
* findAnchorInBlock maps the normalized selection back to raw text-node
* positions, the returned string is the document's ORIGINAL characters (smart
* quotes, em-dashes, nbsp, collapsed whitespace) NOT the normalized ASCII
* agent input.
*
* Callers store THIS as the comment's `selection` so the stored value equals the
* text actually under the mark, which is what the apply-suggestion equality
* check (replaceYjsMarkedText's `joinedText !== expectedText`) compares against.
* Without it a suggestion whose anchor only matched via normalization would be
* un-appliable (spurious 409).
*/
export function getAnchoredText(doc, selection) {
const visit = (node, depth) => {
if (depth > MAX_DEPTH || !node || typeof node !== "object")
return null;
if (!Array.isArray(node.content))
return null;
const match = findAnchorInBlock(node.content, selection);
if (match)
return reconstructRawText(node.content, match);
for (const child of node.content) {
if (child && typeof child === "object" && Array.isArray(child.content)) {
const found = visit(child, depth + 1);
if (found !== null)
return found;
}
}
return null;
};
return visit(doc, 0);
}
/**
* Depth-first, document-order check for whether `selection` can be anchored
* anywhere in `doc`. At each node with an array `content`, first try to match
* within that node's own content, then recurse into children that themselves
* have a `content` array.
*/
export function canAnchorInDoc(doc, selection) {
const visit = (node, depth) => {
if (depth > MAX_DEPTH || !node || typeof node !== "object")
return false;
if (!Array.isArray(node.content))
return false;
if (findAnchorInBlock(node.content, selection))
return true;
for (const child of node.content) {
if (child && typeof child === "object" && Array.isArray(child.content)) {
if (visit(child, depth + 1))
return true;
}
}
return false;
};
return visit(doc, 0);
}
/**
* Split the matched text nodes and splice the comment mark across the range.
* `blockContent` is mutated IN PLACE. `match.startChild..endChild` are all text
* nodes (guaranteed by findAnchorInBlock building runs of text nodes).
*/
function spliceCommentMark(blockContent, match, commentId) {
const { startChild, startOffset, endChild, endOffset } = match;
const commentMark = makeCommentMark(commentId);
const fragments = [];
for (let k = startChild; k <= endChild; k++) {
const n = blockContent[k];
const text = typeof n.text === "string" ? n.text : "";
const sliceStart = k === startChild ? startOffset : 0;
const sliceEnd = k === endChild ? endOffset : text.length;
const before = k === startChild ? text.slice(0, startOffset) : "";
const marked = text.slice(sliceStart, sliceEnd);
const after = k === endChild ? text.slice(endOffset) : "";
// Process per-node so each node's OWN marks/attrs are preserved.
const ownMarks = Array.isArray(n.marks) ? n.marks : [];
// Drop any pre-existing comment mark from the marked fragment so it ends up
// with exactly one comment mark (the new one) rather than two.
const markedBaseMarks = ownMarks.filter((m) => !(m && m.type === "comment"));
if (before.length > 0) {
fragments.push({ ...n, text: before, marks: [...ownMarks] });
}
if (marked.length > 0) {
fragments.push({
...n,
text: marked,
marks: [...markedBaseMarks, commentMark],
});
}
if (after.length > 0) {
fragments.push({ ...n, text: after, marks: [...ownMarks] });
}
}
blockContent.splice(startChild, endChild - startChild + 1, ...fragments);
}
/**
* Count how many times `selection` occurs across the whole document, using the
* same normalization and run-matching as findAnchorInBlock but WITHOUT stopping
* at the first hit: every non-overlapping occurrence within each block's text
* runs is counted and summed across all blocks (depth-first, the same traversal
* as canAnchorInDoc).
*
* This is the uniqueness gate for SUGGESTIONS: because applying a suggestion
* rewrites the exact anchored text, an ambiguous anchor (>1 occurrence) would
* silently edit the wrong place, so a suggestion is only allowed when this
* returns exactly 1. Ordinary comments keep first-occurrence anchoring and do
* not use this. (Note: counts OCCURRENCES, not just matching blocks, so two
* occurrences inside one block are correctly reported as 2.)
*/
export function countAnchorMatches(doc, selection) {
const normSel = normalizeForMatch(selection).norm.trim();
if (normSel.length === 0)
return 0;
// Count non-overlapping occurrences of the normalized selection within a
// single block's direct content, matching findAnchorInBlock's run building.
const countInBlock = (blockContent) => {
if (!Array.isArray(blockContent))
return 0;
let count = 0;
let i = 0;
while (i < blockContent.length) {
const node = blockContent[i];
if (!node || typeof node !== "object" || node.type !== "text") {
i++;
continue;
}
// Accumulate a maximal run of consecutive text nodes.
let rawRun = "";
let j = i;
while (j < blockContent.length) {
const n = blockContent[j];
if (!n || typeof n !== "object" || n.type !== "text")
break;
rawRun += typeof n.text === "string" ? n.text : "";
j++;
}
const norm = normalizeForMatch(rawRun).norm;
// Count every non-overlapping occurrence in this run.
let from = 0;
for (;;) {
const idx = norm.indexOf(normSel, from);
if (idx === -1)
break;
count++;
from = idx + normSel.length;
}
i = j > i ? j : i + 1;
}
return count;
};
let total = 0;
const visit = (node, depth) => {
if (depth > MAX_DEPTH || !node || typeof node !== "object")
return;
if (!Array.isArray(node.content))
return;
total += countInBlock(node.content);
for (const child of node.content) {
if (child && typeof child === "object" && Array.isArray(child.content)) {
visit(child, depth + 1);
}
}
};
visit(doc, 0);
return total;
}
/**
* Depth-first (same order as canAnchorInDoc) over `doc`; on the FIRST block
* whose content matches `selection`, splice the comment mark across the matched
* range in place and return true. Returns false (and does NOT mutate) when no
* block matches.
*/
export function applyAnchorInDoc(doc, selection, commentId) {
const visit = (node, depth) => {
if (depth > MAX_DEPTH || !node || typeof node !== "object")
return false;
if (!Array.isArray(node.content))
return false;
const match = findAnchorInBlock(node.content, selection);
if (match) {
spliceCommentMark(node.content, match, commentId);
return true;
}
for (const child of node.content) {
if (child && typeof child === "object" && Array.isArray(child.content)) {
if (visit(child, depth + 1))
return true;
}
}
return false;
};
return visit(doc, 0);
}
-423
View File
@@ -1,423 +0,0 @@
/**
* Headless, Docmost-equivalent document diff.
*
* Docmost's history editor computes a change set with the exact pipeline below
* (recreateTransform -> ChangeSet.addSteps -> simplifyChanges) and renders it as
* editor decorations. This module runs the SAME computation but serializes the
* result to text + integrity counts instead of decorations, so a diff can be
* previewed without a browser.
*
* recreateTransform here comes from @fellow/prosemirror-recreate-transform, the
* maintained published fork of the MIT prosemirror-recreate-steps source that
* Docmost vendors in @docmost/editor-ext; it exposes the identical
* recreateTransform(fromDoc, toDoc, { complexSteps, wordDiffs, simplifyDiff })
* signature.
*
* If recreateTransform / the changeset throws on a pathological document pair,
* we fall back to a coarse block-level text diff so the tool never hard-fails.
*/
import { Node } from "@tiptap/pm/model";
import { ChangeSet, simplifyChanges } from "@tiptap/pm/changeset";
import { recreateTransform } from "@fellow/prosemirror-recreate-transform";
import { docmostSchema } from "./docmost-schema.js";
/** Recursively concatenate the plain text of a JSON node. */
function plainText(node) {
if (!node || typeof node !== "object")
return "";
let out = "";
if (typeof node.text === "string")
out += node.text;
if (Array.isArray(node.content)) {
for (const child of node.content)
out += plainText(child);
}
return out;
}
/** Count nodes in a JSON doc that satisfy `pred` (recursive). */
function countNodes(doc, pred) {
let n = 0;
const visit = (node) => {
if (!node || typeof node !== "object")
return;
if (pred(node))
n++;
if (Array.isArray(node.content))
for (const c of node.content)
visit(c);
};
visit(doc);
return n;
}
/**
* Count UNIQUE links in a JSON doc by their `href`. A single link can be split
* across several adjacent text runs (e.g. a "link+bold" run followed by a "link"
* run); counting link-bearing runs would over-count it. Walking the tree and
* collecting hrefs into a Set keys each distinct link once. Link marks with a
* missing/empty href are bucketed under a single "" key so a malformed link is
* still counted as one.
*/
function countUniqueLinks(doc) {
const hrefs = new Set();
const visit = (node) => {
if (!node || typeof node !== "object")
return;
if (node.type === "text" && Array.isArray(node.marks)) {
for (const m of node.marks) {
if (m && m.type === "link") {
const href = m.attrs && typeof m.attrs.href === "string" ? m.attrs.href : "";
hrefs.add(href);
}
}
}
if (Array.isArray(node.content))
for (const c of node.content)
visit(c);
};
visit(doc);
return hrefs.size;
}
/** Count footnoteReference nodes anywhere under a node (reading order). */
function countFootnoteRefs(node) {
if (!node || typeof node !== "object")
return 0;
let n = node.type === "footnoteReference" ? 1 : 0;
if (Array.isArray(node.content)) {
for (const child of node.content)
n += countFootnoteRefs(child);
}
return n;
}
/**
* Ordered list of footnote marker numbers found in the BODY only (every
* top-level block before the first "Примечания..." notes heading; if no such
* heading, the whole doc), in reading order.
*
* Supports BOTH representations:
* - real `footnoteReference` nodes (the current footnote feature) numbered
* 1..n by reading position, since their visible number is derived;
* - legacy `[N]` text markers (older translated docs) the literal N.
*/
function footnoteMarkers(doc, notesHeading) {
const top = Array.isArray(doc?.content) ? doc.content : [];
const notesIdx = top.findIndex((n) => n &&
n.type === "heading" &&
plainText(n).trim() === notesHeading);
const bodyBlocks = notesIdx >= 0 ? top.slice(0, notesIdx) : top;
// Real footnoteReference nodes take precedence: when present, number them by
// reading position (their displayed number is not stored).
let refCount = 0;
for (const block of bodyBlocks)
refCount += countFootnoteRefs(block);
if (refCount > 0) {
return Array.from({ length: refCount }, (_, i) => i + 1);
}
// Fallback: legacy `[N]` text markers.
const markers = [];
const re = /\[(\d+)\]/g;
for (const block of bodyBlocks) {
const text = plainText(block);
let m;
re.lastIndex = 0;
while ((m = re.exec(text)) !== null) {
markers.push(Number(m[1]));
}
}
return markers;
}
/** Compute the [old,new] integrity tuples for two JSON docs. */
function computeIntegrity(oldDoc, newDoc, notesHeading) {
const images = [
countNodes(oldDoc, (n) => n.type === "image"),
countNodes(newDoc, (n) => n.type === "image"),
];
const links = [
countUniqueLinks(oldDoc),
countUniqueLinks(newDoc),
];
const tables = [
countNodes(oldDoc, (n) => n.type === "table"),
countNodes(newDoc, (n) => n.type === "table"),
];
const callouts = [
countNodes(oldDoc, (n) => n.type === "callout"),
countNodes(newDoc, (n) => n.type === "callout"),
];
const fns = [
footnoteMarkers(oldDoc, notesHeading),
footnoteMarkers(newDoc, notesHeading),
];
return { images, links, tables, callouts, footnoteMarkers: fns };
}
/**
* Resolve the lead text of the top-level block in a ProseMirror Node that
* contains the given document position. Returns "" when out of range.
*/
function blockContextAt(node, pos) {
try {
const clamped = Math.max(0, Math.min(pos, node.content.size));
const $pos = node.resolve(clamped);
// depth 1 is the top-level block in a doc node.
const block = $pos.depth >= 1 ? $pos.node(1) : $pos.node(0);
const text = block.textContent || "";
return text.length > 80 ? text.slice(0, 77) + "..." : text;
}
catch {
return "";
}
}
/** Truncate a string for the markdown summary. */
function truncate(s, n = 120) {
return s.length > n ? s.slice(0, n - 3) + "..." : s;
}
/**
* Coarse fallback: a block-by-block plain-text diff. Used only when the precise
* changeset pipeline throws, so the tool degrades gracefully instead of failing.
*/
function coarseDiff(oldDoc, newDoc) {
const oldBlocks = Array.isArray(oldDoc?.content) ? oldDoc.content : [];
const newBlocks = Array.isArray(newDoc?.content) ? newDoc.content : [];
const oldTexts = oldBlocks.map(plainText);
const newTexts = newBlocks.map(plainText);
const oldSet = new Set(oldTexts);
const newSet = new Set(newTexts);
const changes = [];
for (const t of oldTexts) {
if (!newSet.has(t) && t.trim() !== "") {
changes.push({ op: "delete", block: truncate(t, 80), text: t });
}
}
for (const t of newTexts) {
if (!oldSet.has(t) && t.trim() !== "") {
changes.push({ op: "insert", block: truncate(t, 80), text: t });
}
}
return changes;
}
/** Build the human-readable unified-ish markdown summary. */
function renderMarkdown(result, fellBack) {
const lines = [];
const { summary, integrity, changes } = result;
lines.push(`# Diff: ${summary.inserted} inserted / ${summary.deleted} deleted (${summary.blocksChanged} blocks changed)`);
if (fellBack) {
lines.push("");
lines.push("> note: precise diff failed; coarse block-level diff shown.");
}
lines.push("");
lines.push("## Integrity (old -> new)");
lines.push(`- images: ${integrity.images[0]} -> ${integrity.images[1]}`);
lines.push(`- links: ${integrity.links[0]} -> ${integrity.links[1]}`);
lines.push(`- tables: ${integrity.tables[0]} -> ${integrity.tables[1]}`);
lines.push(`- callouts: ${integrity.callouts[0]} -> ${integrity.callouts[1]}`);
lines.push(`- footnoteMarkers: [${integrity.footnoteMarkers[0].join(", ")}] -> [${integrity.footnoteMarkers[1].join(", ")}]`);
lines.push("");
lines.push("## Changes");
if (changes.length === 0) {
lines.push("(no textual changes)");
}
else {
for (const c of changes) {
const sign = c.op === "insert" ? "+" : "-";
const ctx = c.block ? ` @ ${truncate(c.block, 60)}` : "";
lines.push(`${sign} ${truncate(c.text)}${ctx}`);
}
}
return lines.join("\n");
}
/**
* Diff two ProseMirror JSON documents the way Docmost's history editor does and
* serialize the result to text + integrity counts.
*
* @param oldDocJson the earlier document
* @param newDocJson the later document
* @param notesHeading heading delimiting body from notes for footnote counting
*/
export function diffDocs(oldDocJson, newDocJson, notesHeading = "Примечания переводчика") {
const integrity = computeIntegrity(oldDocJson, newDocJson, notesHeading);
let changes = [];
let inserted = 0;
let deleted = 0;
let fellBack = false;
const changedBlocks = new Set();
try {
const oldNode = Node.fromJSON(docmostSchema, oldDocJson);
const newNode = Node.fromJSON(docmostSchema, newDocJson);
const tr = recreateTransform(oldNode, newNode, {
complexSteps: false,
wordDiffs: true,
simplifyDiff: true,
});
const changeSet = ChangeSet.create(oldNode).addSteps(tr.doc, tr.mapping.maps, []);
const simplified = simplifyChanges(changeSet.changes, newNode);
for (const change of simplified) {
// Deleted text lives in the OLD doc coordinate range [fromA, toA).
if (change.toA > change.fromA) {
const text = oldNode.textBetween(change.fromA, change.toA, "\n", " ");
if (text.length > 0) {
deleted += text.length;
const block = blockContextAt(oldNode, change.fromA);
changes.push({ op: "delete", block, text });
if (block)
changedBlocks.add("d:" + block);
}
}
// Inserted text lives in the NEW doc coordinate range [fromB, toB).
if (change.toB > change.fromB) {
const text = newNode.textBetween(change.fromB, change.toB, "\n", " ");
if (text.length > 0) {
inserted += text.length;
const block = blockContextAt(newNode, change.fromB);
changes.push({ op: "insert", block, text });
if (block)
changedBlocks.add("i:" + block);
}
}
}
}
catch {
// Pathological pair: degrade to a coarse block-level diff so we never throw.
fellBack = true;
changes = coarseDiff(oldDocJson, newDocJson);
for (const c of changes) {
if (c.op === "insert")
inserted += c.text.length;
else
deleted += c.text.length;
if (c.block)
changedBlocks.add(c.op[0] + ":" + c.block);
}
}
const partial = {
summary: { inserted, deleted, blocksChanged: changedBlocks.size },
integrity,
changes,
};
return { ...partial, markdown: renderMarkdown(partial, fellBack) };
}
/**
* Recursively walk every `text` node and tally the count of each mark by
* `mark.type` (e.g. `{ bold: 5, strike: 3, link: 2 }`). Pure and never throws.
*/
function markCounts(doc) {
const counts = {};
const visit = (node) => {
if (!node || typeof node !== "object")
return;
if (node.type === "text" && Array.isArray(node.marks)) {
for (const m of node.marks) {
if (m && typeof m.type === "string") {
counts[m.type] = (counts[m.type] || 0) + 1;
}
}
}
if (Array.isArray(node.content))
for (const c of node.content)
visit(c);
};
visit(doc);
return counts;
}
/**
* Build a VerifyReport for a content mutation. Pure and never throws on any
* internal error it returns a minimal "changed (diff unavailable)" report so it
* can NEVER break a write.
*
* `changed` is VALUE-based, not JSON-string-based: it is derived from the actual
* deltas (text chars, blocks, mark counts, structural integrity counts), so two
* value-equal docs that differ only in JSON key order report cleanly as
* `changed:false` / "no content change" rather than a misleading +0/-0 change.
*
* The structural integrity delta (from diffDocs's `integrity` tuples) is what
* makes `changed` true for an image/table/callout/link count change that diffs
* to zero text closing a verify blind spot for insert_image, delete_node on a
* table, etc.
*/
export function summarizeChange(before, after) {
try {
const diff = diffDocs(before, after);
// Per-mark-type delta: include a type only when its count actually changed.
const beforeMarks = markCounts(before);
const afterMarks = markCounts(after);
const marks = {};
for (const type of new Set([
...Object.keys(beforeMarks),
...Object.keys(afterMarks),
])) {
const b = beforeMarks[type] || 0;
const a = afterMarks[type] || 0;
if (b !== a)
marks[type] = [b, a];
}
// Structural integrity delta from diffDocs: count-based [old,new] tuples for
// images/links/tables/callouts. Include a type only when old != new.
const integrity = diff.integrity;
const structure = {};
const countTypes = [
"images",
"links",
"tables",
"callouts",
];
for (const type of countTypes) {
const [b, a] = integrity[type];
if (b !== a)
structure[type] = [b, a];
}
const textInserted = diff.summary.inserted;
const textDeleted = diff.summary.deleted;
const blocksChanged = diff.summary.blocksChanged;
const hasMarkDelta = Object.keys(marks).length > 0;
const hasStructureDelta = Object.keys(structure).length > 0;
// VALUE-based change decision: ignore JSON key-order no-ops entirely.
const changed = textInserted > 0 ||
textDeleted > 0 ||
blocksChanged > 0 ||
hasMarkDelta ||
hasStructureDelta;
if (!changed) {
return {
changed: false,
textInserted: 0,
textDeleted: 0,
blocksChanged: 0,
marks: {},
summary: "no content change",
};
}
const parts = [];
// Only mention text/blocks when they actually changed (avoid a misleading
// "+0/-0 chars, 0 block(s)" prefix on a pure mark/structure change).
if (textInserted > 0 || textDeleted > 0 || blocksChanged > 0) {
parts.push(`+${textInserted}/-${textDeleted} chars, ${blocksChanged} block(s)`);
}
const markParts = Object.entries(marks).map(([type, [b, a]]) => `${type} ${b}${a}`);
if (markParts.length > 0)
parts.push(`marks: ${markParts.join(", ")}`);
const structureParts = Object.entries(structure).map(([type, [b, a]]) => `${type} ${b}${a}`);
if (structureParts.length > 0)
parts.push(structureParts.join(", "));
// `changed` is true here, so at least one group is present and parts is non-empty.
const summary = `changed: ${parts.join("; ")}`;
const report = {
changed: true,
textInserted,
textDeleted,
blocksChanged,
marks,
summary,
};
if (hasStructureDelta)
report.structure = structure;
return report;
}
catch {
// A pathological pair must never break a write: degrade to a minimal report.
return {
changed: true,
textInserted: 0,
textDeleted: 0,
blocksChanged: 0,
marks: {},
summary: "changed (diff unavailable)",
};
}
}
File diff suppressed because it is too large Load Diff
-92
View File
@@ -1,92 +0,0 @@
/**
* Filter functions to extract only relevant information from API responses
* for better agent consumption
*/
export function filterWorkspace(data) {
return {
id: data.id,
name: data.name,
description: data.description,
defaultSpaceId: data.defaultSpaceId,
createdAt: data.createdAt,
updatedAt: data.updatedAt,
deletedAt: data.deletedAt,
};
}
export function filterSpace(space) {
return {
id: space.id,
name: space.name,
description: space.description,
slug: space.slug,
visibility: space.visibility,
createdAt: space.createdAt,
updatedAt: space.updatedAt,
deletedAt: space.deletedAt,
};
}
export function filterGroup(group) {
return {
id: group.id,
name: group.name,
description: group.description,
workspaceId: group.workspaceId,
createdAt: group.createdAt,
updatedAt: group.updatedAt,
deletedAt: group.deletedAt,
};
}
export function filterPage(page, content, subpages) {
return {
id: page.id,
slugId: page.slugId,
title: page.title,
parentPageId: page.parentPageId,
spaceId: page.spaceId,
isLocked: page.isLocked,
createdAt: page.createdAt,
updatedAt: page.updatedAt,
deletedAt: page.deletedAt,
// Include converted markdown content if valid string (even empty)
...(typeof content === "string" && { content }),
// Include subpages if provided
...(subpages &&
subpages.length > 0 && {
subpages: subpages.map((p) => ({ id: p.id, title: p.title })),
}),
};
}
export function filterComment(comment, markdownContent) {
return {
id: comment.id,
pageId: comment.pageId,
content: markdownContent ?? comment.content,
selection: comment.selection || null,
type: comment.type || "page",
parentCommentId: comment.parentCommentId || null,
creatorId: comment.creatorId,
creatorName: comment.creator?.name || null,
createdAt: comment.createdAt,
editedAt: comment.editedAt || null,
resolvedAt: comment.resolvedAt || null,
resolvedById: comment.resolvedById || null,
// Suggestion state: the proposed replacement text (if any) and, once a human
// applies it via the UI, when and by whom.
suggestedText: comment.suggestedText || null,
suggestionAppliedAt: comment.suggestionAppliedAt || null,
suggestionAppliedById: comment.suggestionAppliedById || null,
};
}
export function filterSearchResult(result) {
return {
id: result.id,
title: result.title,
parentPageId: result.parentPageId,
createdAt: result.createdAt,
updatedAt: result.updatedAt,
rank: result.rank,
highlight: result.highlight,
spaceId: result.space?.id,
spaceName: result.space?.name,
};
}
-101
View File
@@ -1,101 +0,0 @@
/**
* Footnote diagnostics for imported Markdown (issue #166).
*
* A PURE, fence-aware text scan (independent of the Markdown->ProseMirror
* conversion path, so it reports the same problems for `create_page`,
* `update_page` and `import_page_markdown`). It never changes the document the
* importer still creates the page; this only surfaces footnote problems to the
* caller so an agent can fix its own markup instead of shipping broken footnotes.
*
* Detected problems:
* - danglingReferences: a `[^id]` reference with no `[^id]:` definition.
* - emptyDefinitions: a `[^id]:` whose (kept) text is empty/whitespace.
* - duplicateDefinitions: an id defined by two or more `[^id]:` lines (only the
* first is kept on import first-wins; see extractFootnotes).
* - referencesInTables: a `[^id]` marker found in a GFM table row (heuristic:
* the line, trimmed, starts with `|`) footnotes in table cells often do not
* render as expected.
*/
import { lexFootnoteLines, forEachFootnoteReference, } from "./footnote-lex.js";
/**
* Analyze the footnotes in a Markdown string. Pure; safe to call on any body.
*/
export function analyzeFootnotes(markdown) {
// Distinct reference ids in first-appearance order, plus the set of ids seen
// inside a table row.
const refIds = [];
const refIdSet = new Set();
const referencesInTables = new Set();
const addRef = (id, inTable) => {
if (!refIdSet.has(id)) {
refIdSet.add(id);
refIds.push(id);
}
if (inTable)
referencesInTables.add(id);
};
// Definition texts per id, in first-appearance order of the id.
const defTextsById = new Map();
// Same lexer the importer uses, so the analysis matches exactly what import
// keeps/strips (#166): fenced lines are inert, definition lines are pulled.
for (const tok of lexFootnoteLines(markdown)) {
if (tok.inFence)
continue;
if (tok.definition) {
const { id, text } = tok.definition;
const arr = defTextsById.get(id);
if (arr)
arr.push(text);
else
defTextsById.set(id, [text]);
// A definition's TEXT can itself reference another footnote (`[^a]: see
// [^b]`); count those so such a `[^b]` is not falsely reported dangling.
forEachFootnoteReference(text, (rid) => addRef(rid, false));
continue;
}
const inTable = tok.line.trimStart().startsWith("|");
forEachFootnoteReference(tok.line, (id) => addRef(id, inTable));
}
const danglingReferences = refIds.filter((id) => !defTextsById.has(id));
const duplicateDefinitions = [];
const emptyDefinitions = [];
for (const [id, texts] of defTextsById) {
if (texts.length >= 2)
duplicateDefinitions.push(id);
// First-wins: the kept definition is the first one; flag it if it is blank.
if ((texts[0] ?? "").trim().length === 0)
emptyDefinitions.push(id);
}
const tableRefs = [...referencesInTables];
const warnings = [];
const list = (ids) => ids.map((id) => `[^${id}]`).join(", ");
if (danglingReferences.length > 0) {
warnings.push(`Footnote reference(s) with no matching definition: ${list(danglingReferences)} (each will render as an empty footnote in the editor).`);
}
if (emptyDefinitions.length > 0) {
warnings.push(`Footnote definition(s) with empty text: ${list(emptyDefinitions)}.`);
}
if (duplicateDefinitions.length > 0) {
warnings.push(`Footnote id(s) defined more than once (only the first definition was kept): ${list(duplicateDefinitions)}.`);
}
if (tableRefs.length > 0) {
warnings.push(`Footnote marker(s) inside a table row (footnotes in table cells may not render as expected): ${list(tableRefs)}.`);
}
return {
danglingReferences,
emptyDefinitions,
duplicateDefinitions,
referencesInTables: tableRefs,
warnings,
};
}
/**
* The optional `footnoteWarnings` field for a page-write tool result: present
* (with the warning lines) only when `markdown` has footnote problems, omitted
* otherwise. One helper so all three call sites (create/update/import) attach the
* field identically. Spread into the result: `{ ...result, ...footnoteWarningsField(text) }`.
*/
export function footnoteWarningsField(markdown) {
const { warnings } = analyzeFootnotes(markdown);
return warnings.length > 0 ? { footnoteWarnings: warnings } : {};
}

Some files were not shown because too many files have changed in this diff Show More