Compare commits

..

32 Commits

Author SHA1 Message Date
a
c4ed4a4855 fix(footnotes): strip bare definitions on rebuild; MCP full-doc + zip-import canonicalize tests (#228)
Review #6 (approve-with-comments) follow-ups:
1. canonicalize step 7 now strips bare footnoteDefinitions at ANY depth
   (stripFootnoteDefinitionsDeep), not just footnotesList, in BOTH copies. A
   definition hand-authored outside a list (e.g. nested in a callout via a
   raw-JSON write path) was left in place while a copy was also added to the
   rebuilt list -> duplicate, idempotent, self-perpetuating. Runs only in the
   rebuild path (after the lists are stripped); the fast-path / placement-keep
   branch is untouched. Added a shared-corpus case (bare def nested in a callout)
   to pin it in both mirrors.
2. markdown-clipboard: removed the dead top-level footnoteReference check in
   canonicalizePastedFootnotes (an inline atom is never a top-level slice child;
   only the descendants scan can find it).

Test coverage:
4. New MCP binding tests (full-doc-write-canonicalize.test.mjs): update_page_json
   and copy_page_content canonicalize the persisted full doc, asserted via a new
   `replacePage` seam (symmetric to the existing `mutatePage` seam) so no live
   collab socket is needed. Routed both writers through the seam.
5. New server spec (file-import-task.service.footnote-canonicalize.spec.ts): the
   zip-import path (processGenericImport) canonicalizes footnotes — real
   markdown->HTML->JSON via a real ImportService over a temp-dir .md file, DB trx
   stubbed to capture the persisted page content. FileImportTaskService had no
   spec before.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 01:39:25 +03:00
a
9c1f952b2f fix(footnotes): guard insert against nested/bare definitions, skip definitions-only paste, doc + reorder fixes (#228)
Must-fix:
- insertInlineFootnote could glue a footnoteReference inside an EXISTING
  definition (nested footnotesList, or a bare footnoteDefinition with no list
  wrapper), which canonicalize then dropped as an orphan — silently losing the
  definition's prose. Now: (a) the body/notes boundary is computed from the first
  top-level block that IS or CONTAINS (recursively) a footnotesList/
  footnoteDefinition, not just a top-level list; and (b) the insertNodesAfterAnchor
  core skips footnotesList/footnoteDefinition subtrees entirely (skipSubtreeTypes),
  so an anchor whose only match is inside a definition -> inserted:false (clean
  abort, no write). Added tests: nested-definition, bare-definition, and
  body-before-nested-list-still-inserts.
- editor-ext footnote-canonicalize header listed `markdownToProseMirror` among the
  canonicalizing MCP paths; it is the NON-canonicalizing primitive. Replaced with
  `markdownToProseMirrorCanonical` (+ note that the plain primitive is for comment
  bodies) and added copy_page_content.
- Client paste: canonicalizePastedFootnotes now skips a definitions-ONLY paste
  (no footnoteReference anywhere) — canonicalizing it would strip the
  reference-less list and yield an EMPTY paste. Added a test.

Suggestions:
- docmost_transform now runs validateDocStructure/validateDocUrls on the RAW
  transform output BEFORE canonicalizeFootnotes (mirrors updatePageJson), so a
  too-deep doc gives the intended max-depth error instead of a stack overflow.
- docmost_transform tool description now states the RESULT is footnote-canonical
  (dryRun diff may show tidy-ups; idempotent after first run).
- insertFootnote: dropped the dead `result ? … : undefined` ternaries and the
  `as any` casts (result is always set by the time we return; the not-found path
  throws and aborts mutatePage). `const r = result!;`.

Tests / architecture:
- Added a LIVE-plugin golden case: the real footnoteSyncPlugin leaves a list with
  non-empty content after it in place, and canonicalize agrees (placement parity
  is now a driven property, not a hand-set expected).
- Added generateFootnoteId uuidv7 shape + uniqueness test.
- Item 9: added the ENFORCEMENT-RULE comments at the server parseProsemirrorContent
  and the MCP canonicalizer header (any NEW full-doc persist path MUST canonicalize;
  fragments/append/prepend and comment bodies MUST NOT). Kept per-call-site over a
  brittle grep CI test (the replace-vs-fragment + comment-vs-page nuance makes a
  single wrapper unsafe).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 23:40:28 +03:00
a
3fd66b4245 fix(footnotes): don't canonicalize comment bodies (data loss); canonicalize only page write paths (#228)
Must-fix (REAL DATA LOSS):
- markdownToProseMirror is reused for COMMENT bodies (createComment/updateComment).
  It unconditionally canonicalized, so a comment carrying a standalone footnote
  definition ([^1]: text with no matching reference) had its whole footnotesList
  stripped (referenceIds.length===0 -> stripFootnotesListsDeep) — the text
  vanished. Fix: markdownToProseMirror no longer canonicalizes (content-preserving
  primitive); a new markdownToProseMirrorCanonical wraps it for the PAGE write
  paths (markdown import via importPageMarkdown, update_page markdown via
  updatePageContentRealtime). Comment callers keep the non-canonicalizing
  primitive. Updated the now-false header comment and added create/update-comment
  inline notes. Added collaboration tests: comment path PRESERVES a reference-less
  definition; page path still drops it AND still reorders real footnotes. Updated
  the page-import canonicalization test to use the canonical variant.

Suggestions / architecture:
- #2: collapsed transforms.footnoteDefinition onto the shared
  makeFootnoteDefinition factory (adds only the inner paragraph block id); kept
  the dependency direction transforms -> footnote-authoring (no circular import,
  mirror stays pure).
- #3: confirmed docmost_transform auto-canonicalization is documented (inline
  comment, tool description, CHANGELOG) — no code change.
- #4: copyPageContent is a FULL-document write (replacePageContent of a
  type:"doc"); added a defensive canonicalizeFootnotes pass (no-op on
  already-canonical source).
- CHANGELOG entry refined to list the FULL-document write paths (incl.
  copy_page_content) and to state canonicalization is NOT applied to comment
  bodies.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 22:17:15 +03:00
a
a77a0bc92b fix(footnotes): re-review #232 — refuse footnoteRef into codeBlock/definition, deep-strip nested lists, docs + cross-copy guard (#228)
Must-fix:
- REAL BUG: insertInlineFootnote could splice a footnoteReference (inline atom)
  into a codeBlock or an existing footnoteDefinition, persisting a schema-invalid
  doc (insert_footnote skips validateDocStructure). Now the search is bounded to
  the BODY (before the first footnotesList) and the insertNodesAfterAnchor core
  refuses textblocks that can't hold the atom (codeBlock); when the only match is
  in such a place the insert returns inserted:false and the write aborts cleanly.
  Reachable via docmost_transform too. Added codeBlock / definition / fall-through
  tests.
- Fixed the deepEqualJson doc comment in both copies: arrays are order-SENSITIVE
  (correctness depends on it), only object keys are order-insensitive.
- README.ru.md MCP tool count 38 -> 39 (lines 36/47/63), matching README.md/AGENTS.
- CHANGELOG [Unreleased] Added entry for insert_footnote + server-side footnote
  canonicalization on non-editor write paths (#228).

Suggestions:
- canonicalize step 5/7 now strips footnotesList at ANY depth (both copies), so a
  schema-valid list nested in a callout/blockquote can't leave duplicate defs.
- Exclude the test-only footnote-corpus.ts fixture from the editor-ext build
  (tsconfig), so it no longer ships in dist/.
- Removed the duplicate manual canonicalize cases from the MCP unit test (the
  shared corpus covers them via full deepEqual); kept idempotence + immutability.
- insertInlineFootnote dedup key now keys off the inline array directly
  (footnoteContentKey({ content: inline })) instead of a throwaway node.

Tests / architecture:
- New client-wrapper test (#9): overrides a small mutatePage seam to assert the
  not-found path throws and persists NOTHING, and the success path shapes
  footnoteId/reused/message/verify and writes the right content. Fixed the
  misleading comment in footnote-write.test.mjs.
- B: cross-copy corpus parity guard test (loads both corpora, asserts deep-equal)
  so a typo in one copy can't pass both suites green.
- A: declined — the full-vs-fragment decision lives at the call site, so a
  prepareDocForPersist wrapper would be a bare alias for canonicalizeFootnotes;
  kept the existing per-call-site comments instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 21:41:10 +03:00
a
07ebd8c63e fix(footnotes): address PR #232 review — fragment-safe canonicalization, plugin placement parity, dead-code removal (#228)
Must-fix:
- Move canonicalizeFootnotes OUT of parseProsemirrorContent. It now runs only
  on FULL writes (createPage, updatePageContent operation==='replace'), never on
  an append/prepend fragment (a fragment would lose definition-only footnotes or
  synthesize a bogus empty list). Add a server binding spec.
- Match the live plugin's list PLACEMENT: a single already-canonical
  footnotesList is left exactly where it sits (the plugin never repositions a
  sole correct list), so the first write no longer reorders content that follows
  the list. Applied to BOTH the editor-ext copy and the MCP mirror; pinned by a
  shared golden corpus case with content after the list.
- Fix MCP tool count 38 -> 39 (README x3, AGENTS.md) and the transformJs param
  help (add canonicalizeFootnotes/insertInlineFootnote).

Simplifications:
- Remove the dead duplicate re-id mechanism (deriveFootnoteId/suffix/occurrence)
  from the PURE canonicalizer in both copies — references are never renamed, so
  the derived ids were never requested; first-wins-drop is the real behaviour.
  This also makes the editor-ext footnote-util note about "no cross-package copy"
  true again.
- Remove the sentinel round-trip in insertInlineFootnote: a generalized
  insertNodesAfterAnchor core inserts the footnoteReference node directly.
- Drop the redundant per-definition deep clone in step 4 (shallow id-normalizing
  copy; out is already deep-cloned).

Docs / architecture:
- Correct the editor-ext copy's "It exists because…" header to its real
  consumers (server import, page.service create/update, client paste).
- Note markdownToProseMirror reuse for create/update comment in collaboration.ts.
- A: shared golden JSON corpus exercised by BOTH the editor-ext copy and the MCP
  mirror (footnote-corpus.ts / .mjs) so "the two copies behave identically" is
  checkable.
- C: split the MCP canonicalizer into a pure mirror + footnote-authoring.ts.
- B: import services persist via a different path, so left one-line consolidation
  comments at the call sites rather than folding (does not fall out cleanly).

Tests: insertFootnote wrapper guards + docmost_transform dryRun auto-canonicalize
(MCP mock), page.service create/update + append/prepend binding (server jest),
shared corpus incl. nested-container reference.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 20:23:16 +03:00
a
fa929c9e86 fix(footnotes): canonicalize footnotes on server import + markdown paste (#228)
The footnote canonicalizer was wired into the MCP and editor-ext write paths
but NOT into the server's user-facing markdown/HTML import paths, so importing
or pasting markdown with out-of-order, reused, or orphan footnotes did not
canonicalize -- the exact trigger bug #228 fixes was still reproduced on
import. markdownToHtml -> htmlToJson builds ProseMirror JSON directly and never
runs the editor's footnoteSyncPlugin, and that plugin does not reorder an
existing list, so the stored footnotes kept the source's physical definition
order, retained orphans, and did not collapse reused references.

Wire canonicalizeFootnotes (already exported from @docmost/editor-ext) into
every server markdown/HTML -> page-JSON seam, before persisting:
  - ImportService.importPage (REST single-file .md/.html import)
  - FileImportTaskService (zip import worker)
  - PageService.parseProsemirrorContent (API createPage / updatePageContent)

Also hook the client markdown paste: handlePaste applies a manual transaction
(returns true), bypassing transformPasted/footnoteSyncPlugin, so a pasted
out-of-order markdown footnote block would persist out of order.
canonicalizePastedFootnotes reorders a self-contained pasted block (one that
carries its own footnotesList) to reference order, deduped and orphan-free; it
is deliberately scoped to whole-block pastes so a reference-only paste that
reuses a footnote already defined in the target doc is left untouched.

canonicalizeFootnotes is pure, idempotent and shape-safe (a doc with no
footnotes is unchanged), so it is safe on every write path.

Residual: when a pasted block merges into a doc that already has footnotes,
ordering relative to the pre-existing footnotes is still governed by the live
sync plugin (which does not reorder across the boundary).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 17:10:41 +03:00
claude code agent 227
30cb9d293c feat(footnotes): inline authoring + deterministic server-side canonicalization
Make footnotes author-inline: the agent/tool inserts a footnote at its point
of use (anchor + text) and the numbering plus the bottom list are DERIVED
deterministically server-side. The agent has no access to footnotesList and
cannot desync — out-of-order lists, orphan definitions, and raw trailing
[^id] blocks become structurally impossible.

editor-ext:
- canonicalizeFootnotes(docJSON) -> docJSON: a pure, EditorView-free port of
  footnoteSyncPlugin's end-state. Distinct reference ids in document order are
  the source of truth; exactly one trailing footnotesList holds one definition
  per referenced id in reference order (reusing the existing node or
  synthesizing an empty one); orphans dropped; duplicate definitions resolved
  deterministically (first wins, never lost); idempotent.
- Unit tests + a golden parity suite: on every editor-reachable steady state
  the live footnoteSyncPlugin's JSON is a canonicalize no-op (byte-for-byte
  parity), and the canonicalizer additionally repairs the out-of-order list a
  non-editor write produces.

mcp:
- footnote-canonicalize.ts: behavioural mirror of the editor-ext canonicalizer
  (the MCP package is intentionally decoupled from the editor barrel, like
  footnote-lex/docmost-schema), plus footnoteContentKey for content dedup.
- Auto-canonicalize on EVERY write path: markdownToProseMirror (fixes import
  ordering), update_page_json, and after every docmost_transform. Idempotent,
  so it is a no-op when footnotes are already canonical.
- insert_footnote tool + insertInlineFootnote: anchor + markdown text -> a
  mark-safe footnoteReference and a content-dedup'd definition; the list and
  numbering are derived. Same-content footnotes reuse one number/definition.
- canonicalizeFootnotes + insertInlineFootnote exposed as docmost_transform
  sandbox helpers.

Tests: editor-ext 157 green; MCP 325 green; server + client tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 06:35:25 +03:00
claude_code
904f7b4303 fix(agent-roles): bump proofreader v3 + guard against content edits without a version bump
The proofreader role content was changed (STYLE SHEET block removed) without
bumping its catalog version, so clients never saw an update. Bump proofreader
2 -> 3, and add a content-hash guard so this can't happen silently again.

- index.json: proofreader version 2 -> 3
- scripts/check.mjs: new content-hash guard. A scripts/content-hashes.json lock
  maps slug -> { version, hash } (sha256 over emoji/autoStart/name/description/
  instructions/launchMessage across all languages). check.mjs now fails when a
  role's content changed without bumping its version; the new --update-hashes
  (alias --fix) refreshes the lock but refuses to write when a bump is missing.
- check.mjs: also require every index.json role to carry a finite numeric
  version (matches the server's catalog validation), with defense-in-depth so a
  missing version can't bypass the bump guard.
- scripts/content-hashes.json: new lock artifact (not part of the served catalog).
- README.md: document the guard, the lockfile, --update-hashes, and the
  prune-then-readd limitation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 05:18:39 +03:00
claude_code
cac84dec9b refactor(ai-roles): make catalog URL a per-branch image default, drop local-fs source
The agent-roles catalog source is no longer hardcoded in app code and no longer
supports a local filesystem directory. The provider fetches only from an
http(s):// base URL read at runtime from AI_AGENT_ROLES_CATALOG_URL; an empty or
non-http value yields a 502 (catalog unavailable). The image ships a per-branch
default for that URL (set in CI), still overridable at runtime via the env var.

- provider: drop readLocal + node:fs/node:path; readRelative requires http(s)
  and 502s otherwise; remote fetch/streaming-cap/SSRF guards unchanged.
- environment.service: keep AI_AGENT_ROLES_CATALOG_URL (default ''); comment
  reflects the per-branch build-time default that is runtime-overridable.
- Dockerfile: add ARG+ENV AI_AGENT_ROLES_CATALOG_URL in the installer stage as
  the image default.
- CI: develop.yml builds with the develop raw URL; release.yml defines the main
  raw URL once in workflow env and references it from both build steps.
- tests: replace local-fixture tests with remote-mock happy/malformed bundle
  tests and a non-http => 502 case; path-traversal block uses an https source.
- docs: update .env.example, CHANGELOG (#222), agent-roles-catalog/README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 03:54:43 +03:00
claude_code
90dd8f1481 Merge branch 'develop' of https://gitea.vvzvlad.xyz/vvzvlad/gitmost into develop 2026-06-27 03:54:24 +03:00
39113c9dbf Merge pull request 'fix(share): custom address edit renames in place instead of duplicating (#226)' (#227) from fix/share-alias-rename into develop
Reviewed-on: #227
2026-06-27 03:53:31 +03:00
claude_code
1367070468 refactor(agent-roles): drop style-sheet duties from copyeditor role
Remove the STYLE SHEET / СТАЙЛ-ШИТ section from the copyeditor
(proofreader) role and clean up all dangling references to it in both
the ru and en editorial bundles:
- description: drop "maintains a style sheet" / "ведёт стайл-шит"
- instructions: remove the STYLE SHEET block
- instructions: drop "record it in the style sheet" mentions in the
  WHAT YOU DO and WHEN UNSURE sections

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 03:46:03 +03:00
claude code agent 227
767ac9e7e2 fix(share): guard alias swap/rename against concurrent-delete race; share unique-violation helpers
Address PR #227 re-review (comment 2193).

- Stability: `updatePageId`/`updateAlias` now `executeTakeFirstOrThrow`, so a row
  reaped by a concurrent `removeAlias` between the read and the UPDATE (READ
  COMMITTED) raises `NoResultError` instead of returning `undefined`. The service
  maps that to a retryable `ConflictException` (`ALIAS_PAGE_RACE`) rather than a
  200-without-alias (swap) or a generic 400 from `undefined.id` (rename). Tests
  cover both branches.
- Simplification: drop the redundant secondary "unexpected unique index" warn and
  the now-unused `UNIQUE_ALIAS_INDEX` const (the constraint name is already logged
  unconditionally; both index branches still distinguish "Alias already taken" vs
  ALIAS_PAGE_RACE).
- Architecture: extract `isUniqueViolation`/`violatedConstraint` into
  database/utils.ts; adopt them in the share-alias service and favorite.repo
  (the bare `23505` check). ai-agent-roles (#222) is on a separate unmerged branch
  and should adopt them after #227 merges (noted at the helpers). Helper unit test
  added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 03:33:33 +03:00
claude_code
2a4ef9267e refactor(ai-roles): bake catalog URL at image build, drop local-fs source
The agent-roles catalog source is no longer hardcoded in app code and no
longer supports a local filesystem directory. The provider now fetches only
from an http(s):// base URL read from AI_AGENT_ROLES_CATALOG_URL; an empty or
non-http value yields a 502 (catalog unavailable). The default URL is baked
into the Docker image at build time and set per branch in CI.

- provider: drop readLocal + node:fs/node:path; readRelative requires http(s)
  and 502s otherwise; remote fetch/streaming-cap/SSRF guards unchanged.
- environment.service: keep AI_AGENT_ROLES_CATALOG_URL (default ''); comment
  updated to reflect build-time injection, remote-only.
- Dockerfile: add ARG+ENV AI_AGENT_ROLES_CATALOG_URL in the installer stage.
- CI: develop.yml builds with the develop raw URL; release.yml (both build
  steps) with the main raw URL.
- tests: replace local-fixture tests with remote-mock happy/malformed bundle
  tests and a non-http => 502 case; path-traversal block uses an https source.
- docs: update .env.example, CHANGELOG (#222), agent-roles-catalog/README.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 03:32:48 +03:00
claude code agent 227
309719abc6 fix(share): show reassign hint instead of dead-end error for a taken custom address
The share modal flagged a custom address already owned by another page with a
red "This address is already in use" error driven by the availability probe.
That reads as terminal even though Save actually triggers the server's
409 `ALIAS_REASSIGN_REQUIRED` and opens the "Move custom address?" confirm
modal that retargets the address to the current page — so the reassign path was
hidden behind what looked like a hard stop.

Replace the red error with an informational description hint ("This address is
in use. Saving will move it to this page.") and keep Save enabled, so the
existing confirm-reassign flow is discoverable. Renaming to a FREE name was
already correct (the probe returns available -> no error -> server renames the
single row in place); this only changes the taken-name presentation.

Verified end-to-end in a real browser against a live stand on this branch:
- A (free rename `test`->`test2`): 200, same alias row renamed in place, link
  becomes `/l/test2`, no error, exactly one DB row for the page.
- B (`test2` owned by another page): hint shown (no dead-end error), Save ->
  409 ALIAS_REASSIGN_REQUIRED -> "Move custom address?" modal -> confirm -> 200,
  the single row retargets, one row each.
- C (same-name re-save): Save disabled (no-op); first-time set inserts.

Add a client component test covering both branches (taken name -> hint not
error + Save enabled; 409 -> reassign modal -> confirm sends confirmReassign).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 03:24:00 +03:00
claude_code
3511301331 Merge branch 'develop' of https://gitea.vvzvlad.xyz/vvzvlad/gitmost into develop 2026-06-27 03:12:27 +03:00
claude_code
b65ca6d7dd chore(agent-roles-catalog): merge copy-editor into proofreader, refresh editorial roles
Merge the copy-editor (📐) and proofreader (🧹 "Корректор") editorial roles
into a single role. Keep slug `proofreader`, drop slug `copy-editor`, and set
the merged role's emoji to 📐.

- index.json: remove copy-editor; bump structural-editor, line-editor,
  fact-checker, proofreader to version 2 (narrator unchanged); update editorial
  bundle description (ru/en).
- bundles/editorial/{ru,en}.json: delete copy-editor; refresh emoji/name/
  description/instructions of structural-editor, line-editor, fact-checker and
  the merged proofreader verbatim from gitmost-agenty-ru.md / gitmost-agents-en.md;
  preserve autoStart and launchMessage; leave narrator untouched.
- README.md: drop copy-editor from the editorial role list.

Validated with scripts/check.mjs (OK).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-27 03:12:14 +03:00
4a3819373d Merge pull request 'feat(ai-chat): auto-open last chat bound to the document (#191)' (#209) from feat/191-chat-doc-binding into develop
Reviewed-on: #209
2026-06-27 02:56:31 +03:00
claude code agent 227
e682bbccd1 fix(share): order swap delete-before-update and distinguish unique violations
Addresses review on PR #227.

- setAlias confirmed-reassign branch: DELETE the target page's existing
  alias row(s) BEFORE retargeting `byName` onto the page, instead of after.
  The new partial unique index `(workspace_id, page_id)` is non-deferrable
  and checked at each statement, so retargeting first momentarily left two
  rows for the page -> immediate 23505 -> rolled-back tx surfaced as a
  misleading "Alias already taken" (regressing a previously-working swap onto
  a page that already had its own alias). The reordered branch needs no
  trailing self-heal. JSDoc updated to describe the real ordering.

- catch block: the postgres@3.x driver exposes the violated index as
  `err.constraint_name` (with `.constraint` as a fallback). Map
  `share_aliases_workspace_id_alias_unique` -> "Alias already taken" and the
  new `share_aliases_workspace_id_page_id_unique` -> a distinct ALIAS_PAGE_RACE
  outcome (a concurrent same-page write, not a name clash). Always log the
  constraint name on any 23505 so the race is diagnosable.

- migration 20260627T120000: document that the dedup DELETE is intended,
  irreversible data loss (old duplicate `/l/<old>` links start 404ing after
  upgrade; `down()` cannot restore the rows). Same note added to CHANGELOG
  [Unreleased] Fixed.

Tests:
- integration: confirmed reassign onto a page that ALREADY has its own alias
  (RED before the reorder); migration up() dedup scoping across pages and a
  second workspace; mid-transaction error -> BadRequest with clean rollback.
- unit: constraint_name distinguishing (alias index, page_id index, fallback
  `.constraint`, no-info default) and non-unique error -> BadRequest; retarget
  test now asserts delete-before-update order.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:52:33 +03:00
claude code agent 227
9d2bec8eb8 fix(share): keep exactly one custom address per page on alias edit (#226)
Editing an existing share alias (e.g. slug `te` -> `ted`) failed to update
the displayed `/l/<alias>` link: `setAlias()` looked the requested slug up by
name and, if free, INSERTed a brand-new row, leaving the page with multiple
alias rows. The modal then read via `findByPageId().executeTakeFirst()` with no
`ORDER BY`, so Postgres returned an arbitrary (in practice the oldest, stale)
row. Every edit also spawned an orphan row that kept a live `/l/<old>` link
forever. Regression of #205.

Enforce the invariant "a page has EXACTLY ONE custom address":
- `setAlias()` now resolves the page's current alias row and RENAMES it in
  place when the requested name is free (insert only when the page has none),
  keeps the same-name no-op and the cross-page 409 `ALIAS_REASSIGN_REQUIRED`
  + confirmed-retarget flow, and after any successful write DELETEs all other
  alias rows for the page (self-heal). Runs in one transaction so the page is
  never transiently empty or duplicated.
- repo: add `updateAlias` (rename) and `deleteOthersForPage`; make
  `findByPageId` deterministic with `ORDER BY created_at DESC, id DESC`.
- migration: dedup existing rows (keep newest per page) + a PARTIAL unique
  index `(workspace_id, page_id) WHERE page_id IS NOT NULL` so dangling
  aliases still coexist while live ones are one-per-page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:51:51 +03:00
b6630deb32 Merge pull request 'feat(ai-roles): импортируемый мультиязычный каталог ролей агента' (#222) from feature/agent-roles-catalog into develop
Reviewed-on: #222
2026-06-27 02:39:27 +03:00
claude code agent 227
7ef98a663b Address PR #222 review: import-mutation notification tests + redirect-SSRF hardening
ITEM 1: cover useImportAiRolesFromCatalogMutation onSuccess notifications.
Add import-from-catalog-message.test.tsx (twin of update-from-catalog-message)
asserting the always-shown summary (errors:[]) and the additional red
"Failed to import N role(s)" notification when result.errors is non-empty.

ITEM 2: pass redirect:'error' to the remote catalog fetch in fetchRemote so a
compromised-but-trusted upstream cannot 3xx the fetch into the internal network
(redirect-SSRF). Add provider specs asserting the option is passed and that a
redirect rejection maps to BadGatewayException.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 02:36:28 +03:00
109ab10fc5 Merge pull request 'fix(temporary-notes): tree clock marker updates without reload + mobile-friendly full-width create buttons' (#225) from fix/temporary-notes-ui into develop
Reviewed-on: #225
2026-06-27 01:39:10 +03:00
claude code agent 227
2b7c861f78 Address PR #222 re-review: fix source-uniqueness detection + coverage/cleanups
MUST-FIX
- isSourceUniqueViolation read the wrong error field: kysely-postgres-js
  (postgres@3.4.8) puts the violated constraint on `constraint_name`, not
  node-postgres' `.constraint`, so a concurrent same-slug+language import's
  23505 was never recognized as a source-collision and surfaced a false
  "name already exists" error. Now read `constraint_name` (with `.constraint`
  as a fallback for other drivers). Fix the faked test fixture (it built the
  error with the same wrong `.constraint` field, masking the bug): it now
  uses `constraint_name`, so the test genuinely exercises the skip path and
  FAILS against the unfixed code.
- Extract the catalog modal's role-state computation into a pure
  `catalogRoleInstallState(role, workspaceRoles, language)` helper (mirrors
  role-launch.ts) and cover it with vitest: import / installed / update /
  same-slug-different-language.

SUGGESTIONS
- Restore IAiRoleUpdateFromCatalogResult as a discriminated union mirroring
  the server; narrow the consumer via `"reason" in result` (the boolean
  discriminant does not narrow under strictNullChecks:false).
- README: add a "How it's served" section documenting AI_AGENT_ROLES_CATALOG_URL
  (remote http(s) base / local path / empty => in-repo folder).
- check.mjs: drop the redundant `const key = slug` alias.
- Cover the reason->message mapping in useUpdateAiRoleFromCatalogMutation
  (4 branches) via renderHook with a mocked service.
- Cover importFromCatalog "bundle not in index" => BadGateway.
- Cover updateFromCatalog "slug in index but missing in bundle file" =>
  not-in-catalog.

ARCHITECTURE
- Extract the shared catalog read prefix: a private `loadBundleById`
  (fetchIndex -> meta -> fetchBundle -> versionMap) reused by getCatalogBundle
  and importFromCatalog, and a `catalogRoleContentFields` mapper shared by the
  import insert and update patch. The three orchestrations and their distinct
  write paths stay separate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 01:01:29 +03:00
claude code agent 227
d181b5c4ff test(temporary-notes): cover the create race-guard, broadcast deadline + cache patch; unify page->tree-node mappers
Address review comment 2159 on the temporary-notes UI work.

Tests:
- tree-model: cover handleCreate's race-guard temporaryExpiresAt patch — (a)
  server node inserted WITHOUT a deadline + create response carries one => node
  gains the deadline; (b) node already has a deadline => not overwritten, prev
  returned by reference.
- ws-tree.service.spec: broadcastPageCreated now asserts the deadline is carried
  when present and pinned to null (`?? null`) when absent.
- page-embed-query (new spec): syncTemporaryExpiresInCache patches the in-tree
  node's temporaryExpiresAt, and leaves the atom value at the same reference when
  the id is absent from the loaded tree (no write).

Refactor (closes the drift bug-class at the root):
- Client: extract one canonical pageToTreeNode(page, overrides) mapper in
  tree/utils and route buildTree, handleCreate's optimistic insert, the restore
  mutation and the duplicate handler through it. Restore stays permanent (server
  nulls temporaryExpiresAt) and duplicate stays permanent (server arms no timer)
  — both now reflect the server without a reload, where before they dropped the
  field entirely.
- Server: extract one toTreeNodeSnapshot(page) helper called by both the
  PAGE_CREATED event enrichment (page.repo) and the addTreeNode broadcast
  (ws-tree.service), so the optional temporaryExpiresAt can't drift between the
  two literals.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:58:40 +03:00
claude code agent 227
12ff76fb89 fix(temporary-notes): live sidebar clock marker + stacked mobile create buttons
Issue 1 — the sidebar tree's temporary-note clock marker did not appear/
disappear until a page reload when a note's temporary state changed.

- Make/unmake permanent from the page header menu and the in-page banner went
  through syncTemporaryExpiresInCache(), which patched the page query cache but
  never touched treeDataAtom, so the sidebar node kept its stale
  temporaryExpiresAt. Patch the tree node there too (via jotai's default store),
  so the marker updates without a reload.
- Creating a note as temporary showed no marker until reload: the create flow's
  cache write (invalidateOnCreatePage) omitted temporaryExpiresAt, so the tree
  rebuild (buildTree -> mergeRootTrees) overwrote the optimistic/socket node's
  marker with undefined. Carry temporaryExpiresAt in that cached entry.
- Thread temporaryExpiresAt through the server addTreeNode broadcast (PAGE_CREATED
  snapshot -> TreeNodeSnapshot -> broadcastPageCreated) so OTHER clients watching
  the space also render the marker immediately, and harden handleCreate's
  idempotency guard to patch the deadline if the broadcast won the insert race.

Issue 2 — the home and space-overview "New note" / "New temporary note" buttons
sat side-by-side and the temporary label clipped on narrow mobile widths. Lay
them out full-width, stacked vertically, and tint the temporary button orange
(matching the clock marker + banner) while the regular one stays neutral gray.

Tests: extend tree-socket-reducers.test.ts (addTreeNode carries
temporaryExpiresAt). Verified live with Playwright: marker appears on create and
toggles both ways with no reload; mobile buttons are stacked, full-width,
unclipped, and differently colored.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 00:29:19 +03:00
claude code agent 227
26ca19f89e agent-roles: concurrency-safe catalog import + unified source validator
Item 1 (concurrency-safe import): add a partial UNIQUE index on
(workspace_id, source->>'slug', source->>'language') WHERE source IS NOT NULL
AND deleted_at IS NULL, so two concurrent imports of the same bundle can no
longer create duplicate roles for one catalog slug+language. The in-memory
installedKeys snapshot cannot see a sibling request's writes; the index is the
backstop. importFromCatalog now catches the 23505 from THIS index (keyed off
the constraint name) and treats it as "already installed" -> skip, batch
continues. A 23505 from the name-uniqueness index keeps its existing friendly
per-role error behavior (distinguished by constraint name; an indeterminate
23505 falls back to that path, so no regression).

Item 2 (single source validator): strengthen parseSource into THE single form
validator for the source jsonb column -> returns a fully-valid RoleSource | null
(slug/language non-empty strings, version a number). The service's weaker
roleSource is removed and both layers share the RoleSource type (defined in the
db entity.types module both already import AiAgentRole from, so no import
cycle). normalizeRow / the read path now only ever yield a valid RoleSource or
null; a malformed stored source normalizes to null (tolerated by the service).

Tests: parseSource null for {} / {slug:123} / {slug:'a'} / empty-string keys /
string version, typed value for a full valid shape; service test that a
source-uniqueness 23505 is skipped (not errored) and the batch continues.
Verified the partial index rejects a duplicate source-not-null row but allows
two source-NULL rows, and the migration up/down run cleanly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 23:40:25 +03:00
claude code agent 227
50e79275e1 Address review on agent-roles catalog: changelog, docs, BadGateway on body-read abort
- CHANGELOG: document the importable multilingual agent-roles catalog under
  [Unreleased] (browse/import/update, 4 new endpoints, source column, the new
  AI_AGENT_ROLES_CATALOG_URL env var) (#222).
- Fix importFromCatalog docstring: a role is skipped only on source.slug AND
  source.language; another language of the same slug still imports.
- Provider: map a timeout/abort (or any failure) during the response-BODY read
  to a logged BadGatewayException, so a slow/dripping source yields a 502, not a
  generic 500. Existing too-large BadGateway cases are rethrown as-is.
- Service: inject a Nest Logger and log the root cause (with workspaceId/
  bundleId/slug) on a non-23505 insert error during import.
- Modal: hoist the duplicated i18n base-subtag into a single baseLang const.
- Tests: AbortError body-read -> BadGateway; null-body text() fallback (under
  and over cap); invalid-JSON and malformed-index BadGateway; non-23505 import
  error -> generic message + logged root cause.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 23:15:45 +03:00
claude code agent 227
8be8279809 Address PR #222 review: migration order, provider logging, catalog tests
- Rename catalog-source migration 20260626T120000 -> T150000 so it sorts
  after develop's latest migration (T140000-page-temporary-notes); the old
  timestamp predated ai-chat-message-status/share-aliases and tripped
  Kysely's #ensureMigrationsInOrder, aborting server boot.
- Provider: inject a Nest Logger and log the real cause (incl. response
  status) in the parseJson / readLocal / fetchRemote catch blocks, and
  propagate a useful cause into the BadGatewayException message; add a
  shortError helper (robust to jest's realm-shifted Error-likes).
- Provider: replace the manual Uint8Array assembly with
  Buffer.concat(chunks).toString('utf8'); keep the streaming size cap.
- Controller spec: add admin-gate coverage for the 4 catalog routes
  (catalog/catalogBundle/import/updateFromCatalog) - non-admin Forbidden +
  service untouched, admin delegates with the right args.
- Service spec: add getCatalog/getCatalogBundle tests covering the
  localized() three-tier fallback, the sorted language union, the
  missing-bundle BadGateway, and the role-version default.
- Provider spec: add remote fetch-rejects and non-ok (503) error branches.
- Service: drop the dead Date.now() tail in freeName (now an explicit
  unreachable throw) and extract a shared isUniqueViolation() predicate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 22:36:26 +03:00
claude_code
19f84ca0e7 feat(ai-roles): add importable, multilingual agent roles catalog
Admins can browse a curated catalog of agent roles, import roles/bundles
into a workspace, and update an imported role when the catalog ships a
newer version.

Catalog: a set of JSON files (index.json manifest + bundles/<id>/<lang>.json)
served from a local folder (dev) or a remote http(s) base URL via
AI_AGENT_ROLES_CATALOG_URL. Seeded with the existing 7 RU roles (editorial +
research bundles) plus EN translations.

Server:
- migration: nullable jsonb `source` column on ai_agent_roles
  ({ slug, language, version }; null => manually created)
- catalog provider: remote fetch with timeout + streaming size cap, or local
  read; ^[a-z0-9-]+$ segment guard against path-traversal/SSRF
- admin endpoints: catalog, catalog/bundle, import, update-from-catalog
- import/update match by slug+language; update preserves `enabled`

Client:
- catalog modal with language selector and Import/Installed/Update states
- "Import from catalog" button + empty-state CTA in the roles settings panel
- en-US/ru-RU strings

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 22:36:26 +03:00
claude code agent 227
c64d7f315e fix(ai-chat): open chat window before resolving the bound chat (#191)
Address PR #209 review.

- use-open-ai-chat.ts: call setWindowOpen(true) before awaiting
  getBoundChat so the header button feels instant on slow connections;
  the chat switch (setActiveChatId/setDraft/setSelectedRoleId) is applied
  after the round-trip resolves. Also drop the redundant no-op
  setWindowOpen(true) in the already-open branch (bare early return).
- CHANGELOG.md: document the header AI-chat button auto-opening the
  latest chat bound to the current document under [Unreleased]/Added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 21:02:15 +03:00
claude code agent 227
7a7aa79eab feat(ai-chat): auto-open last chat bound to the document (#191)
On opening the floating AI-chat window from the header on a document page,
auto-open the LAST chat bound to that document. Binding reuses the existing
ai_chats.page_id (no migration): the bound chat is the requesting user's
most-recent non-deleted chat created on that page, so a new chat on the page
becomes the bound one for free. Resolution happens only on a genuine
closed -> open transition; the provenance badge deep-link is untouched.

Server: AiChatRepo.findLatestByPage + POST /ai-chat/bound-chat (BoundChatDto),
both read-only and owner/workspace-scoped.
Client: getBoundChat service + useOpenAiChatForCurrentPage hook wired into the
app-header entry point (fail-soft to a fresh chat; draft/role cleared only on a
real switch).
Tests: repo scoping/ordering, controller wiring, and hook behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 21:01:38 +03:00
109 changed files with 11283 additions and 229 deletions

View File

@@ -132,6 +132,14 @@ MCP_DOCMOST_PASSWORD=
# NEVER set is_agent on a human or shared account — every action by that account
# (including normal human edits) would then be mis-attributed as AI.
# Agent-roles catalog source: an http(s):// base URL to the catalog's raw files
# (the server appends /index.json and /bundles/<id>/<lang>.json). This value is
# baked into the Docker image at build time per branch (see the Dockerfile ARG
# AI_AGENT_ROLES_CATALOG_URL and the CI build-args). Set it here only to point a
# local/non-Docker run at a catalog; if unset, the "import role from catalog"
# admin feature is unavailable. Local-filesystem sources are no longer supported.
# AI_AGENT_ROLES_CATALOG_URL=
# Per-embedding-call timeout in milliseconds for the RAG indexer.
# A slow/hung embeddings endpoint fails after this and the batch continues.
# AI_EMBEDDING_TIMEOUT_MS=120000

View File

@@ -52,6 +52,7 @@ jobs:
platforms: linux/amd64
build-args: |
APP_VERSION=${{ steps.version.outputs.value }}
AI_AGENT_ROLES_CATALOG_URL=https://raw.githubusercontent.com/vvzvlad/gitmost/develop/agent-roles-catalog
push: true
tags: ${{ env.IMAGE }}:develop
cache-from: type=gha,scope=develop-amd64

View File

@@ -17,6 +17,7 @@ permissions:
env:
VERSION: ${{ inputs.version || github.ref_name }}
IMAGE: ghcr.io/vvzvlad/gitmost
AI_AGENT_ROLES_CATALOG_URL: https://raw.githubusercontent.com/vvzvlad/gitmost/main/agent-roles-catalog
jobs:
# Run the reusable test suite first so a failing test blocks the image build.
@@ -57,6 +58,7 @@ jobs:
platforms: ${{ matrix.platform }}
build-args: |
APP_VERSION=${{ env.VERSION }}
AI_AGENT_ROLES_CATALOG_URL=${{ env.AI_AGENT_ROLES_CATALOG_URL }}
outputs: type=image,name=${{ env.IMAGE }},push-by-digest=true,name-canonical=true,push=true
cache-from: type=gha,scope=${{ matrix.suffix }}
cache-to: type=gha,scope=${{ matrix.suffix }},mode=max,ignore-error=true
@@ -85,6 +87,7 @@ jobs:
platforms: ${{ matrix.platform }}
build-args: |
APP_VERSION=${{ env.VERSION }}
AI_AGENT_ROLES_CATALOG_URL=${{ env.AI_AGENT_ROLES_CATALOG_URL }}
push: false
tags: |
${{ env.IMAGE }}:latest

View File

@@ -254,7 +254,7 @@ The API server is a Fastify app with a global `/api` prefix (`main.ts` excludes
- **Redis** backs caching, the BullMQ queues, the WebSocket Socket.IO adapter, and collaboration sync.
### The two AI subsystems (the main fork additions)
1. **Embedded MCP server** (`integrations/mcp/` + `packages/mcp`). The standalone `@docmost/mcp` server (38 agent-native tools: per-block patch/insert/delete by id, scripted `(doc)=>doc` transforms with dry-run diff, table editing, version diff/restore, comments, images, shares) is bundled and served over HTTP at `/mcp`. It writes through Docmost's real-time-collaboration layer so concurrent human edits aren't clobbered. Each request authenticates **per-user** via the `Authorization` header — either HTTP Basic (`base64(email:password)`, the user's own Docmost login, validated through `AuthService`) or a Bearer access JWT (the user's `authToken`) — and the session acts under that user's permissions. `MCP_DOCMOST_EMAIL` / `MCP_DOCMOST_PASSWORD` are an **optional service-account fallback**, used only when a request carries neither Basic nor Bearer credentials (back-compat for CI/scripts). An admin enables MCP with a workspace toggle (Workspace settings → AI). Optionally protected by a shared `MCP_TOKEN`: when set, every `/mcp` request must carry a matching `X-MCP-Token` header (its own header, separate from `Authorization`, which now carries the per-user Basic/Bearer credentials). Note: this changed from the older `Authorization: Bearer <MCP_TOKEN>` scheme — see `.env.example` and the CHANGELOG Breaking Changes entry.
1. **Embedded MCP server** (`integrations/mcp/` + `packages/mcp`). The standalone `@docmost/mcp` server (39 agent-native tools: per-block patch/insert/delete by id, scripted `(doc)=>doc` transforms with dry-run diff, table editing, version diff/restore, comments, images, shares) is bundled and served over HTTP at `/mcp`. It writes through Docmost's real-time-collaboration layer so concurrent human edits aren't clobbered. Each request authenticates **per-user** via the `Authorization` header — either HTTP Basic (`base64(email:password)`, the user's own Docmost login, validated through `AuthService`) or a Bearer access JWT (the user's `authToken`) — and the session acts under that user's permissions. `MCP_DOCMOST_EMAIL` / `MCP_DOCMOST_PASSWORD` are an **optional service-account fallback**, used only when a request carries neither Basic nor Bearer credentials (back-compat for CI/scripts). An admin enables MCP with a workspace toggle (Workspace settings → AI). Optionally protected by a shared `MCP_TOKEN`: when set, every `/mcp` request must carry a matching `X-MCP-Token` header (its own header, separate from `Authorization`, which now carries the per-user Basic/Bearer credentials). Note: this changed from the older `Authorization: Bearer <MCP_TOKEN>` scheme — see `.env.example` and the CHANGELOG Breaking Changes entry.
2. **AI agent chat** (`core/ai-chat/` server + `apps/client/src/features/ai-chat/` client). A built-in agent over the wiki using the Vercel **AI SDK** (`ai`, `@ai-sdk/*`) against any OpenAI-compatible provider configured per workspace (`integrations/ai/` — credentials encrypted at rest via `integrations/crypto`, stored in `ai_provider_credentials`). Key pieces:
- `core/ai-chat/tools/` — the agent's ~40 read+write tools. Every tool runs under the **calling user's** CASL permissions via a per-user loopback access token (`docmost-client.loader.ts`), so the agent can never exceed what the user could do. Only **reversible** operations are exposed (page history + trash; no permanent delete). Agent edits get an "AI agent" provenance badge in page history (`20260616T130000-agent-provenance` migration).
- `core/ai-chat/embedding/` — RAG indexer + a BullMQ consumer on `AI_QUEUE` that embeds pages into `page_embeddings` (vector search), complementing Postgres full-text search. Pages are (re)indexed on edit; `AI_EMBEDDING_TIMEOUT_MS` bounds a hung embeddings endpoint.

View File

@@ -28,6 +28,37 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
answer was cut off and builds on it instead of restarting; the rest of the
queue still flushes normally afterward. (#198)
- **Importable multilingual agent-roles catalog.** Admins can browse a curated
catalog of agent roles, grouped into bundles and offered in several languages,
and import the ones they want into the workspace (with skip-or-rename handling
for name collisions); the same role in a different language imports as a
separate install. An imported role remembers its catalog origin and offers a
one-click update when the catalog ships a newer revision. Backed by four new
admin endpoints — `POST /ai-chat/roles/catalog` (browse bundles),
`/catalog/bundle` (read one bundle's roles), `/import`, and
`/update-from-catalog` — and a new `source` column linking a role to its
catalog slug/language/version. The catalog source is configured via the
`AI_AGENT_ROLES_CATALOG_URL` env var — an `http(s)://` base URL to the
catalog's raw files; the image ships a per-branch default baked in CI, and it
can be overridden at runtime via the env var (see `.env.example`). (#222)
- **Author footnotes inline from an agent, and deterministic server-side footnote
canonicalization on every non-editor write path.** A new MCP `insert_footnote`
tool places a footnote at a body anchor by content only — the agent supplies
WHERE (anchor text) and WHAT (markdown); the number and the bottom
`footnotesList` are derived server-side, so an agent can never assign a number,
edit the list, or desync, and a same-content note reuses one definition. Under
the hood, the editor's footnote-integrity invariant (one trailing list,
numbering by first reference, no orphans/duplicates, no raw `[^id]`) is now
enforced as a pure `canonicalizeFootnotes(doc)` on the FULL-document write paths
that bypass the editor's plugins: server markdown/HTML import, `PageService`
create and full-document (`replace`) updates, the client markdown paste, and the
MCP markdown page-import / `update_page` (markdown) / `update_page_json` /
`docmost_transform` / `insert_footnote` / `copy_page_content` paths. It is
idempotent (a no-op once canonical) and is deliberately NOT applied to
append/prepend fragments, nor to COMMENT bodies — a comment may legitimately
contain a standalone footnote definition, which canonicalization would drop.
(#228)
### Fixed
- **A shared page now keeps EXACTLY ONE custom address (`/l/:alias`).** Editing a
@@ -41,6 +72,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
page), so any previously-live duplicate `/l/<old>` link begins returning the
generic 404 after upgrade — intended, but not undoable by `down()`. (#226,
#227)
- **Typing a custom address already used by another page no longer looks like a
dead end.** The share modal previously flagged such a name with a red "This
address is already in use" error, hiding the fact that saving offers to MOVE
the address to the current page. The field now shows an informational hint —
"This address is in use. Saving will move it to this page." — and keeps Save
enabled, so the existing reassign-confirm flow (`409 ALIAS_REASSIGN_REQUIRED`
"Move custom address?") is discoverable instead of reading as terminal. (#227)
## [0.94.0] - 2026-06-26
@@ -120,6 +158,13 @@ per-workspace rolling-day token budget.
applies it through the existing `/pages/update` route — reflecting it in the
title field and broadcasting to other clients. Gated by the `settings.ai.generative`
flag and throttled per user. (#199)
- **AI chat: header button auto-opens the chat bound to the current document.**
Clicking the AI-chat button in the header while viewing a page now reopens the
latest chat tied to that document instead of whatever chat was last active,
reusing the existing `ai_chats.page_id` provenance (no migration). The newest
chat you created on the page wins; with no bound chat — or off a page, or if
the lookup fails — it falls soft to a fresh chat and keeps the current
selection otherwise. (#191)
### Changed

View File

@@ -23,6 +23,11 @@ RUN apt-get update \
WORKDIR /app
# Agent-roles catalog base URL: per-branch default set at build time (CI);
# overridable at runtime via the AI_AGENT_ROLES_CATALOG_URL env var.
ARG AI_AGENT_ROLES_CATALOG_URL=""
ENV AI_AGENT_ROLES_CATALOG_URL=$AI_AGENT_ROLES_CATALOG_URL
# Copy apps
COPY --from=builder /app/apps/server/dist /app/apps/server/dist
COPY --from=builder /app/apps/client/dist /app/apps/client/dist

View File

@@ -34,7 +34,7 @@ The goal of the fork is a **100% open, AGPL-only build with no Enterprise-Editio
| --- | --- |
| **EE code removed** | Stripped all client and server Enterprise-Edition code; ships as a clean community/AGPL build with no license checks. |
| **Comment resolution** | Re-implemented from scratch as a community feature (resolve / re-open with Open/Resolved tabs). No EE code reused, available to anyone who can comment. |
| **Embedded MCP server** | A community MCP server (`@docmost/mcp`, 38 tools) is served over HTTP at `/mcp` — no enterprise license required. Replaces the removed license-gated EE MCP. |
| **Embedded MCP server** | A community MCP server (`@docmost/mcp`, 39 tools) is served over HTTP at `/mcp` — no enterprise license required. Replaces the removed license-gated EE MCP. |
| **AI agent chat** | Built-in AI agent chat over your wiki, written from scratch as a community feature — no enterprise license. The agent reads and edits pages on your behalf (scoped to your permissions), with full-text + vector (RAG) search and optional web access via external MCP servers. |
| **Rebranding** | App logo / name changed from *Docmost* to *Gitmost*. |
| **Compact page tree** | Default page-tree indentation reduced from 16px to 8px per nesting level. |
@@ -44,7 +44,7 @@ The goal of the fork is a **100% open, AGPL-only build with no Enterprise-Editio
### Embedded MCP server
Gitmost has **our own MCP server** — [docmost-mcp](https://github.com/vvzvlad/docmost-mcp),
which we wrote — **built directly into the app** and served at `/mcp`. It exposes **38
which we wrote — **built directly into the app** and served at `/mcp`. It exposes **39
agent-native tools**: surgical per-block edits (patch / insert / delete by id),
structure-preserving find/replace, scripted `(doc) => doc` transforms with a dry-run diff,
structured table editing, version history with diff / restore, comments, images and share
@@ -60,7 +60,7 @@ every little fix. And it needs no enterprise license.
| | **Gitmost `/mcp` (our docmost-mcp)** | Docmost's built-in MCP |
| --- | :---: | :---: |
| **Enterprise license** | Not required | Required |
| **Tools** | 38, agent-native | Coarse (read Markdown, page CRUD, replace whole page) |
| **Tools** | 39, agent-native | Coarse (read Markdown, page CRUD, replace whole page) |
| **Per-block edits / find-replace / scripted transforms** | ✅ | — |
| **Structured table editing, version diff / restore** | ✅ | — |
| **Comments, images, share links** | ✅ | — |

View File

@@ -33,7 +33,7 @@
| --- | --- |
| **Удалён EE-код** | Вырезан весь код Enterprise-редакции на клиенте и сервере; это чистая community/AGPL-сборка без лицензионных проверок. |
| **Резолв комментариев** | Переписан с нуля как community-функция (резолв / переоткрытие с вкладками «Открытые» / «Решённые»). EE-код не используется, доступно любому, кто может комментировать. |
| **Встроенный MCP-сервер** | Community MCP-сервер (`@docmost/mcp`, 38 инструментов) отдаётся по HTTP на `/mcp` — без enterprise-лицензии. Заменяет удалённый лицензируемый EE MCP. |
| **Встроенный MCP-сервер** | Community MCP-сервер (`@docmost/mcp`, 39 инструментов) отдаётся по HTTP на `/mcp` — без enterprise-лицензии. Заменяет удалённый лицензируемый EE MCP. |
| **Чат с AI-агентом** | Встроенный чат с AI-агентом по содержимому вики, написанный с нуля как community-функция — без enterprise-лицензии. Агент читает и редактирует страницы от вашего имени (в рамках ваших прав), с полнотекстовым + векторным (RAG) поиском и опциональным доступом в интернет через внешние MCP-серверы. |
| **Ребрендинг** | Логотип / название приложения изменены с *Docmost* на *Gitmost*. |
| **Компактное дерево страниц** | Отступ дерева страниц по умолчанию уменьшен с 16px до 8px на уровень вложенности. |
@@ -44,7 +44,7 @@
В Gitmost есть **наш собственный MCP-сервер** — [docmost-mcp](https://github.com/vvzvlad/docmost-mcp),
который мы написали сами, — **встроенный прямо в приложение** и доступный на `/mcp`. Он даёт
**38 agent-native инструментов**: точечное редактирование по блокам (patch / insert / delete
**39 agent-native инструментов**: точечное редактирование по блокам (patch / insert / delete
по id), find/replace с сохранением структуры, скриптовые трансформации `(doc) => doc` с
предпросмотром диффа, структурное редактирование таблиц, история версий с диффом /
восстановлением, комментарии, изображения и ссылки на шаринг — всё применяется через слой
@@ -60,7 +60,7 @@ real-time-коллаборации Docmost, поэтому запись нико
| | **`/mcp` в Gitmost (наш docmost-mcp)** | Родной MCP у Docmost |
| --- | :---: | :---: |
| **Enterprise-лицензия** | Не нужна | Нужна |
| **Инструменты** | 38, agent-native | Примитивные (Markdown, CRUD страниц, замена целиком) |
| **Инструменты** | 39, agent-native | Примитивные (Markdown, CRUD страниц, замена целиком) |
| **Правки по блокам / find-replace / скриптовые трансформации** | ✅ | — |
| **Структурное редактирование таблиц, дифф / восстановление версий** | ✅ | — |
| **Комментарии, изображения, ссылки на шаринг** | ✅ | — |

View File

@@ -0,0 +1,193 @@
# Agent roles catalog
This directory is **data, not application code**. It holds the content of an
"agent roles catalog": reusable agent role definitions (system prompts plus a
little metadata), grouped into bundles and translated into one or more
languages. A separate server reads these files and serves them; nothing here is
executable application logic except the validation script.
## File layout
```
agent-roles-catalog/
index.json # the catalog manifest: bundles, languages, role versions
bundles/
<bundle-id>/
<lang>.json # one file per declared language (e.g. ru.json, en.json)
scripts/
check.mjs # validates the catalog (no dependencies)
content-hashes.json # check artifact: per-role content-hash lock (NOT served)
package.json # defines the `check` script
README.md
```
Currently shipped bundles:
- `editorial` — the editorial suite (structural-editor, line-editor,
fact-checker, proofreader, narrator), languages `ru`, `en`.
- `research` — a single `researcher` role, languages `ru`, `en`.
## How it's served
The server does not bundle this data; it reads it at request time from a single
configured location, the `AI_AGENT_ROLES_CATALOG_URL` env var
(`EnvironmentService.getAiAgentRolesCatalogSource()`), an `http(s)://` base URL
to the catalog's raw files. The server fetches `<base>/index.json` for the
manifest and `<base>/bundles/<bundle-id>/<lang>.json` for each opened bundle
file (REMOTE only).
That base URL is provided as a per-branch default in the Docker image (set in
CI: a `develop` build points at the `develop` raw URL, a release build at the
`main` raw URL) and can be overridden at runtime via the
`AI_AGENT_ROLES_CATALOG_URL` env var. Local-filesystem sources are no longer
supported; if the value is unset the catalog is unavailable.
The fetched JSON is re-validated server-side (the catalog is treated as
untrusted input). See `.env.example` for the variable and the CHANGELOG for the
rollout.
## `index.json` schema
```jsonc
{
"schemaVersion": 1,
"bundles": [
{
"id": "editorial", // unique bundle id; matches bundles/<id>/
"name": { "ru": "...", "en": "..." }, // localized display name
"description": { "ru": "...", "en": "..." },
"languages": ["ru", "en"], // which <lang>.json files must exist
"roles": [
{ "slug": "structural-editor", "version": 1 }
// ...
]
}
]
}
```
`version` lives **here, in index.json**, per role. Bump it whenever a role's
content (instructions, name, description, etc.) changes, so consumers can detect
updates.
## Bundle (`<lang>.json`) schema
```jsonc
{
"schemaVersion": 1,
"language": "ru",
"roles": [
{
"slug": "structural-editor", // REQUIRED, unique across the whole catalog
"emoji": "🧱",
"name": "...", // REQUIRED, localized
"description": "...", // localized
"instructions": "...", // REQUIRED, the system prompt, localized
"autoStart": true, // whether the role starts working immediately
"launchMessage": "..." // first message sent on launch (or null)
}
]
}
```
Notes:
- `modelConfig` is intentionally absent; the server treats an absent
`modelConfig` as `null`.
- A role's `slug`, `emoji`, and `autoStart` are identical across all language
files of the same bundle. Only `name`, `description`, `instructions`, and
`launchMessage` are translated.
## Slug uniqueness
**Every `slug` must be UNIQUE ACROSS THE WHOLE CATALOG**, not just within a
bundle. A slug appears once per language file of its bundle (same slug in
`ru.json` and `en.json`), but no two different bundles may share a slug.
`scripts/check.mjs` enforces this.
## How to add things
### Add a role to an existing bundle
1. Add an entry to that bundle's `roles[]` in `index.json` with a new unique
`slug` and `version: 1`.
2. Add a role object with the same `slug` to **every** `<lang>.json` of the
bundle, translating `name`, `description`, `instructions`, and
`launchMessage`.
3. Run the check (see below).
### Add a bundle
1. Add a bundle object to `index.json` (`id`, `name`, `description`,
`languages`, `roles`).
2. Create `bundles/<id>/<lang>.json` for each declared language, with one role
object per `roles[]` entry.
3. Run the check.
### Add a language to a bundle
1. Add the language code to that bundle's `languages[]` in `index.json`.
2. Create `bundles/<id>/<lang>.json` containing every role of the bundle,
translated.
3. Run the check.
### Change a role's content
Edit the role in the relevant `<lang>.json` file(s) and **bump that role's
`version`** in `index.json`. Then run `node scripts/check.mjs --update-hashes`
to refresh the content-hash lock (`scripts/content-hashes.json`). `check.mjs`
now **fails if a role's content changed but its `version` was not bumped**, so
this step is mandatory — the lock can only be refreshed after the bump.
## Validating
From this directory:
```sh
node scripts/check.mjs # or: npm run check
```
It fails (exit code 1) if any slug is duplicated across the catalog, if a
bundle's index `roles[]` don't match the slugs present in each language file, if
a declared language file is missing, or if any role is missing a required field
(`slug`, `name`, `instructions`). It prints `OK` on success.
### Content-hash guard
`check.mjs` also guards against changing a role's content without bumping its
`version`. It keeps a lockfile, `scripts/content-hashes.json`, mapping each role
`slug` to `{ version, hash }`, where `hash` is a SHA-256 over the role's
content fields (`emoji`, `autoStart`, `name`, `description`, `instructions`,
`launchMessage`) across all of its language files, in a deterministic canonical
form. This lockfile is a **check artifact only** — the server fetches only
`index.json` and the bundle `<lang>.json` files, never this file, so it has no
effect on the served catalog or its schema.
On a normal run, for every role the check recomputes the hash and compares it
against the lock:
- content unchanged and versions agree → OK;
- content changed but `version` not bumped above the lock → **error** asking you
to bump and refresh;
- content changed and `version` bumped → **error** asking you to record it by
refreshing the lock;
- role missing from the lock, or a lock entry for a role that no longer exists →
**error** asking you to refresh.
Refresh the lock with:
```sh
node scripts/check.mjs --update-hashes # alias: --fix
```
This recomputes the lock from the current catalog, prunes entries for removed
roles, and prints what changed — but it **refuses to write** (exit 1) if any
role's content changed while its `index.json` version was not bumped, so the
version bump is always enforced first. The check also requires every
`index.json` role to carry a finite numeric `version` (the server requires the
same).
Known, accepted limitation: a deliberate prune-then-readd of a slug (remove the
role and run `--update-hashes`, then re-add it with changed content at the same
version) is **not** caught, because a brand-new slug has no lock baseline to
enforce a bump against.

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,31 @@
{
"schemaVersion": 1,
"bundles": [
{
"id": "editorial",
"name": { "ru": "Редакторский набор", "en": "Editorial suite" },
"description": {
"ru": "Полный цикл редактуры статьи: структура, стиль, корректура, факты и нарратив.",
"en": "The full article-editing cycle: structure, style, copyediting, facts, and narrative."
},
"languages": ["ru", "en"],
"roles": [
{ "slug": "structural-editor", "version": 2 },
{ "slug": "line-editor", "version": 2 },
{ "slug": "fact-checker", "version": 2 },
{ "slug": "proofreader", "version": 3 },
{ "slug": "narrator", "version": 1 }
]
},
{
"id": "research",
"name": { "ru": "Исследование", "en": "Research" },
"description": {
"ru": "Глубокое исследование темы с подготовкой отчёта.",
"en": "Deep research on a topic with a prepared report."
},
"languages": ["ru", "en"],
"roles": [ { "slug": "researcher", "version": 1 } ]
}
]
}

View File

@@ -0,0 +1,8 @@
{
"name": "agent-roles-catalog",
"private": true,
"type": "module",
"scripts": {
"check": "node scripts/check.mjs"
}
}

View File

@@ -0,0 +1,353 @@
#!/usr/bin/env node
// Validates the agent roles catalog.
// Fails (exit 1) on: duplicate slugs across the whole catalog, mismatches
// between a bundle's index roles[] and the slugs present in each language
// file, a missing declared language file, or a role missing required fields.
import { readFileSync, writeFileSync, existsSync } from "node:fs";
import { createHash } from "node:crypto";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
const __dirname = dirname(fileURLToPath(import.meta.url));
const catalogDir = join(__dirname, "..");
// `--update-hashes` (alias `--fix`) recomputes the content-hash lockfile from
// the current catalog instead of just validating against it.
const updateHashes =
process.argv.includes("--update-hashes") || process.argv.includes("--fix");
// The content-hash lockfile lives under scripts/ and is a CHECK ARTIFACT only:
// the server never fetches it, so it has zero impact on the served schema.
const lockPath = join(__dirname, "content-hashes.json");
const errors = [];
function readJson(path) {
try {
return JSON.parse(readFileSync(path, "utf8"));
} catch (err) {
errors.push(`Cannot read/parse ${path}: ${err.message}`);
return null;
}
}
const indexPath = join(catalogDir, "index.json");
if (!existsSync(indexPath)) {
console.error(`Missing index.json at ${indexPath}`);
process.exit(1);
}
const index = readJson(indexPath);
if (!index) {
for (const e of errors) console.error(e);
process.exit(1);
}
const bundles = Array.isArray(index.bundles) ? index.bundles : [];
if (bundles.length === 0) {
errors.push("index.json has no bundles[]");
}
// Track every slug seen across the whole catalog to detect duplicates.
const slugSeen = new Map(); // slug -> "bundleId/lang"
for (const bundle of bundles) {
const bundleId = bundle.id;
if (!bundleId) {
errors.push("A bundle in index.json is missing an id");
continue;
}
const indexSlugs = (bundle.roles || []).map((r) => r.slug);
// Duplicate slugs inside the bundle index roles[].
const indexSlugSet = new Set(indexSlugs);
if (indexSlugSet.size !== indexSlugs.length) {
errors.push(`Bundle "${bundleId}" index.json roles[] contains duplicate slugs`);
}
// Each index role must carry a finite numeric "version". The server requires
// this (see ai-agent-roles-catalog.provider.ts), and the content-hash guard
// below relies on it for the bump comparison, so enforce it here too.
for (const r of bundle.roles || []) {
if (typeof r.version !== "number" || !Number.isFinite(r.version)) {
errors.push(
`Bundle "${bundleId}" index.json role "${r.slug}" is missing a numeric "version"`
);
}
}
const languages = Array.isArray(bundle.languages) ? bundle.languages : [];
if (languages.length === 0) {
errors.push(`Bundle "${bundleId}" declares no languages`);
}
for (const lang of languages) {
const langPath = join(catalogDir, "bundles", bundleId, `${lang}.json`);
if (!existsSync(langPath)) {
errors.push(`Bundle "${bundleId}" declares language "${lang}" but ${langPath} is missing`);
continue;
}
const langFile = readJson(langPath);
if (!langFile) continue;
const roles = Array.isArray(langFile.roles) ? langFile.roles : [];
const fileSlugs = roles.map((r) => r && r.slug);
// (d) Required fields per role.
for (const role of roles) {
for (const field of ["slug", "name", "instructions"]) {
if (role == null || role[field] == null || role[field] === "") {
errors.push(
`Bundle "${bundleId}/${lang}" has a role missing required field "${field}" (slug=${role && role.slug})`
);
}
}
}
// (b) index roles[] must match the slugs present in each language file.
const fileSlugSet = new Set(fileSlugs);
const missingInFile = indexSlugs.filter((s) => !fileSlugSet.has(s));
const extraInFile = fileSlugs.filter((s) => !indexSlugSet.has(s));
if (missingInFile.length > 0) {
errors.push(
`Bundle "${bundleId}/${lang}" is missing roles declared in index.json: ${missingInFile.join(", ")}`
);
}
if (extraInFile.length > 0) {
errors.push(
`Bundle "${bundleId}/${lang}" has roles not declared in index.json: ${extraInFile.join(", ")}`
);
}
// (a) Duplicate slugs across the whole catalog.
for (const slug of fileSlugs) {
if (!slug) continue;
const where = `${bundleId}/${lang}`;
// Only flag duplicates across DIFFERENT bundles or files; the same slug
// is expected to appear once per language file of the same bundle.
if (slugSeen.has(slug)) {
const prev = slugSeen.get(slug);
const prevBundle = prev.split("/")[0];
if (prevBundle !== bundleId) {
errors.push(
`Slug "${slug}" is duplicated across the catalog: ${prev} and ${where}`
);
}
} else {
slugSeen.set(slug, where);
}
}
}
}
// ---------------------------------------------------------------------------
// Content-hash guard: detect "content changed without a version bump".
//
// check.mjs cannot use git history, so we maintain a lockfile
// (scripts/content-hashes.json) mapping each role slug to its recorded
// { version, hash }. On every run we recompute each role's content hash and
// compare it against the lock; a content change is only allowed once the role's
// version in index.json has been bumped and the lock refreshed.
//
// Known, accepted limitation: a deliberate prune-then-readd of a slug (remove
// the role and run --update-hashes, then re-add it with changed content at the
// same version) is NOT caught, because a brand-new slug has no lock baseline to
// enforce a bump against. We document this rather than building tombstones.
// ---------------------------------------------------------------------------
// Content fields hashed for each role, in a fixed canonical order. `slug` is
// identity (not content) and `version` lives in index.json, so neither is here.
// `modelConfig` (an OPTIONAL role field the server also serves) is intentionally
// EXCLUDED: no shipped role uses it today, and being an object it would need a
// deterministic deep canonicalization (recursive key sort) before hashing —
// otherwise JSON.stringify key-order would make the hash non-deterministic. If a
// role ever gains a `modelConfig`, add it here WITH such canonicalization so a
// change to it is still caught by the bump guard.
const CONTENT_FIELDS = [
"emoji",
"autoStart",
"name",
"description",
"instructions",
"launchMessage",
];
// Build a map of slug -> { version, langRoles: { lang: roleObject } } from the
// current catalog so we can compute hashes and read index versions.
function collectCatalogRoles() {
const out = new Map(); // slug -> { version, langRoles: Map<lang, role> }
for (const bundle of bundles) {
const bundleId = bundle.id;
if (!bundleId) continue;
const languages = Array.isArray(bundle.languages) ? bundle.languages : [];
for (const r of bundle.roles || []) {
if (!r || !r.slug) continue;
if (!out.has(r.slug)) {
out.set(r.slug, { version: r.version, langRoles: new Map() });
} else {
// Same slug declared twice in index.json roles[]; already flagged above.
out.get(r.slug).version = r.version;
}
}
for (const lang of languages) {
const langPath = join(catalogDir, "bundles", bundleId, `${lang}.json`);
if (!existsSync(langPath)) continue;
const langFile = readJson(langPath);
if (!langFile) continue;
const roles = Array.isArray(langFile.roles) ? langFile.roles : [];
for (const role of roles) {
if (!role || !role.slug) continue;
const entry = out.get(role.slug);
if (!entry) continue; // role not declared in index.json; flagged above.
entry.langRoles.set(lang, role);
}
}
}
return out;
}
// Deterministic content hash for a role: languages sorted ascending, each
// language's content fields taken in CONTENT_FIELDS order (null when absent).
function contentHash(langRoles) {
const langs = [...langRoles.keys()].sort();
const canonical = langs.map((lang) => {
const role = langRoles.get(lang);
const fields = {};
for (const field of CONTENT_FIELDS) {
fields[field] = role && role[field] != null ? role[field] : null;
}
return [lang, fields];
});
return createHash("sha256").update(JSON.stringify(canonical)).digest("hex");
}
// Compute current { version, hash } for every catalog role.
const catalogRoles = collectCatalogRoles();
const current = new Map(); // slug -> { version, hash }
for (const [slug, entry] of catalogRoles) {
current.set(slug, {
version: entry.version,
hash: contentHash(entry.langRoles),
});
}
// Load the existing lock (may be absent on first run).
let lock = {};
if (existsSync(lockPath)) {
const parsed = readJson(lockPath);
if (parsed && typeof parsed === "object") lock = parsed;
}
if (updateHashes) {
// Refresh the lock from the current catalog, but refuse to write if any role's
// content changed without its version being bumped above the existing lock.
const blockers = [];
for (const [slug, cur] of current) {
const prev = lock[slug];
if (!prev) continue; // new role; nothing to enforce a bump against.
if (cur.hash === prev.hash) continue; // content unchanged.
// Defense-in-depth: a non-numeric version must never pass the bump check via
// `undefined <= N` (which is false). The standard checks already flag a
// missing numeric version, but guard here too before comparing.
if (typeof cur.version !== "number" || !Number.isFinite(cur.version)) {
blockers.push(
`role "${slug}" content changed but its index.json "version" is missing or not numeric; set a numeric "version" before refreshing the lock`
);
} else if (cur.version <= prev.version) {
blockers.push(
`role "${slug}" content changed but its version was not bumped (still ${prev.version}); bump "version" in index.json before refreshing the lock`
);
}
}
// Still honor the standard checks before allowing a write.
if (errors.length > 0) {
console.error("Catalog check FAILED:");
for (const e of errors) console.error(` - ${e}`);
process.exit(1);
}
if (blockers.length > 0) {
console.error("Refusing to update content-hash lock:");
for (const b of blockers) console.error(` - ${b}`);
process.exit(1);
}
// Compute the change summary relative to the old lock, pruning removed slugs.
const newLock = {};
const added = [];
const changed = [];
const removed = [];
for (const [slug, cur] of [...current].sort((a, b) => a[0].localeCompare(b[0]))) {
newLock[slug] = { version: cur.version, hash: cur.hash };
const prev = lock[slug];
if (!prev) added.push(slug);
else if (prev.hash !== cur.hash || prev.version !== cur.version) changed.push(slug);
}
for (const slug of Object.keys(lock)) {
if (!current.has(slug)) removed.push(slug);
}
writeFileSync(lockPath, JSON.stringify(newLock, null, 2) + "\n");
console.log(`Wrote ${lockPath}`);
if (added.length) console.log(` added: ${added.join(", ")}`);
if (changed.length) console.log(` updated: ${changed.join(", ")}`);
if (removed.length) console.log(` pruned: ${removed.join(", ")}`);
if (!added.length && !changed.length && !removed.length) {
console.log(" (no changes; lock already up to date)");
}
console.log("OK");
process.exit(0);
}
// Normal run: validate current content against the lock.
for (const [slug, cur] of current) {
const prev = lock[slug];
if (!prev) {
errors.push(
`role "${slug}" is not recorded in the content-hash lock; run: node scripts/check.mjs --update-hashes`
);
continue;
}
if (cur.hash === prev.hash) {
// Content unchanged; the lock version must still agree with index.json.
if (cur.version !== prev.version) {
errors.push(
`role "${slug}" content is unchanged but its index.json version (${cur.version}) differs from the lock (${prev.version}); run: node scripts/check.mjs --update-hashes`
);
}
continue;
}
// Content changed.
// Defense-in-depth: treat a non-numeric version as an error before the `<=`
// comparison, so a missing version can never silently pass the bump check
// (and we avoid a misleading "version bumped to undefined" message).
if (typeof cur.version !== "number" || !Number.isFinite(cur.version)) {
errors.push(
`role "${slug}" content changed but its index.json "version" is missing or not numeric; set a numeric "version", then run: node scripts/check.mjs --update-hashes`
);
} else if (cur.version <= prev.version) {
errors.push(
`role "${slug}" content changed but its version was not bumped (still ${prev.version}); bump "version" in index.json, then run: node scripts/check.mjs --update-hashes`
);
} else {
errors.push(
`role "${slug}" content changed and version bumped to ${cur.version}; record it by running: node scripts/check.mjs --update-hashes`
);
}
}
// Lock entries for slugs that no longer exist in the catalog.
for (const slug of Object.keys(lock)) {
if (!current.has(slug)) {
errors.push(
`content-hash lock has entry for unknown role "${slug}" (no longer in the catalog); run: node scripts/check.mjs --update-hashes`
);
}
}
if (errors.length > 0) {
console.error("Catalog check FAILED:");
for (const e of errors) console.error(` - ${e}`);
process.exit(1);
}
console.log("OK");

View File

@@ -0,0 +1,26 @@
{
"fact-checker": {
"version": 2,
"hash": "d7ad1dae07d6f4321e7d40c5b36259dbf930264d748834809c4fb77294bf72e3"
},
"line-editor": {
"version": 2,
"hash": "cca324110dc6f96d2a8a239a2fb95b0ba09fad5806c9b6090a3c210ea7883ceb"
},
"narrator": {
"version": 1,
"hash": "36b38785fea6ae1c70bf6fb6b29ae5278bb86e389e61f7b9736675a589fa434c"
},
"proofreader": {
"version": 3,
"hash": "a36047c5cab837b2a727f63d4ddafc269b1fc44b90b365e770ecdb8f77e13952"
},
"researcher": {
"version": 1,
"hash": "853658fda43ddbe0a4d08f2c6e50b5116d29a2e9ccd7f46e173e65920d8f6ace"
},
"structural-editor": {
"version": 2,
"hash": "83093baa7262aef8193871a1afcf2b43b11a56fe2d00cade41355cf66d972b74"
}
}

View File

@@ -1333,6 +1333,7 @@
"A short, memorable link you can point at any shared page.": "A short, memorable link you can point at any shared page.",
"Use 2-60 lowercase letters, digits and hyphens": "Use 2-60 lowercase letters, digits and hyphens",
"This address is already in use": "This address is already in use",
"This address is in use. Saving will move it to this page.": "This address is in use. Saving will move it to this page.",
"Move custom address?": "Move custom address?",
"Move here": "Move here",
"The address \"{{alias}}\" currently points to \"{{title}}\". Move it to this page?": "The address \"{{alias}}\" currently points to \"{{title}}\". Move it to this page?",
@@ -1346,5 +1347,22 @@
"Could not generate a title": "Could not generate a title",
"AI title generation is disabled": "AI title generation is disabled",
"AI is not configured": "AI is not configured",
"Too many requests, please try again later": "Too many requests, please try again later"
"Too many requests, please try again later": "Too many requests, please try again later",
"Import from catalog": "Import from catalog",
"Browse the catalog": "Browse the catalog",
"Role catalog": "Role catalog",
"On name conflict": "On name conflict",
"Skip": "Skip",
"Import": "Import",
"Installed": "Installed",
"v{{from}} → v{{to}}": "v{{from}} → v{{to}}",
"Imported {{created}}, renamed {{renamed}}, skipped {{skipped}}": "Imported {{created}}, renamed {{renamed}}, skipped {{skipped}}",
"Failed to import {{count}} role(s)": "Failed to import {{count}} role(s)",
"The role catalog is unavailable": "The role catalog is unavailable",
"Please try again later.": "Please try again later.",
"No bundles available": "No bundles available",
"Already up to date": "Already up to date",
"Updated to the latest version": "Updated to the latest version",
"This role is no longer in the catalog": "This role is no longer in the catalog",
"This language is no longer available in the catalog": "This language is no longer available in the catalog"
}

View File

@@ -1190,6 +1190,7 @@
"A short, memorable link you can point at any shared page.": "Короткая запоминающаяся ссылка, которую можно направить на любую опубликованную страницу.",
"Use 2-60 lowercase letters, digits and hyphens": "Используйте 2–60 строчных букв, цифр и дефисов",
"This address is already in use": "Этот адрес уже занят",
"This address is in use. Saving will move it to this page.": "Этот адрес уже используется. При сохранении он будет перемещён на эту страницу.",
"Move custom address?": "Переместить пользовательский адрес?",
"Move here": "Переместить сюда",
"The address \"{{alias}}\" currently points to \"{{title}}\". Move it to this page?": "Адрес «{{alias}}» сейчас указывает на «{{title}}». Переместить его на эту страницу?",
@@ -1203,5 +1204,23 @@
"Could not generate a title": "Не удалось придумать название",
"AI title generation is disabled": "Генерация названий через AI отключена",
"AI is not configured": "AI не настроен",
"Too many requests, please try again later": "Слишком много запросов, попробуйте позже"
"Too many requests, please try again later": "Слишком много запросов, попробуйте позже",
"Import from catalog": "Импорт из каталога",
"Browse the catalog": "Открыть каталог",
"Role catalog": "Каталог ролей",
"On name conflict": "При конфликте имён",
"Skip": "Пропустить",
"Import": "Импортировать",
"Installed": "Установлено",
"v{{from}} → v{{to}}": "v{{from}} → v{{to}}",
"Imported {{created}}, renamed {{renamed}}, skipped {{skipped}}": "Импортировано: {{created}}, переименовано: {{renamed}}, пропущено: {{skipped}}",
"Failed to import {{count}} role(s)": "Не удалось импортировать ролей: {{count}}",
"The role catalog is unavailable": "Каталог ролей недоступен",
"Please try again later.": "Попробуйте позже.",
"No bundles available": "Наборы недоступны",
"No roles configured": "Роли не настроены",
"Already up to date": "Уже актуальна",
"Updated to the latest version": "Обновлено до последней версии",
"This role is no longer in the catalog": "Эта роль больше не представлена в каталоге",
"This language is no longer available in the catalog": "Этот язык больше не доступен в каталоге"
}

View File

@@ -10,12 +10,12 @@ import classes from "./app-header.module.css";
import { BrandLogo } from "@/components/ui/brand-logo";
import TopMenu from "@/components/layouts/global/top-menu.tsx";
import { Link } from "react-router-dom";
import { useAtom, useSetAtom } from "jotai";
import { useAtom } from "jotai";
import {
desktopSidebarAtom,
mobileSidebarAtom,
} from "@/components/layouts/global/hooks/atoms/sidebar-atom.ts";
import { aiChatWindowOpenAtom } from "@/features/ai-chat/atoms/ai-chat-atom.ts";
import { useOpenAiChatForCurrentPage } from "@/features/ai-chat/hooks/use-open-ai-chat.ts";
import { workspaceAtom } from "@/features/user/atoms/current-user-atom.ts";
import { useToggleSidebar } from "@/components/layouts/global/hooks/hooks/use-toggle-sidebar.ts";
import SidebarToggle from "@/components/ui/sidebar-toggle-button.tsx";
@@ -38,7 +38,9 @@ export function AppHeader() {
const toggleDesktop = useToggleSidebar(desktopSidebarAtom);
const [workspace] = useAtom(workspaceAtom);
const setAiChatWindowOpen = useSetAtom(aiChatWindowOpenAtom);
// Opening from the header auto-opens the document's bound chat (last chat
// created on the current page); off a page it keeps the current selection.
const openAiChat = useOpenAiChatForCurrentPage();
// AI chat entry point: only shown when the workspace enables it (A7 gate).
const aiChatEnabled = workspace?.settings?.ai?.chat === true;
@@ -105,7 +107,7 @@ export function AppHeader() {
color="dark"
size="sm"
aria-label={t("AI chat")}
onClick={() => setAiChatWindowOpen((v) => !v)}
onClick={openAiChat}
>
<IconMessage size={20} />
</ActionIcon>

View File

@@ -0,0 +1,135 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { renderHook, act } from "@testing-library/react";
import { Provider, createStore } from "jotai";
import type { ReactNode } from "react";
import { useOpenAiChatForCurrentPage } from "./use-open-ai-chat";
import {
activeAiChatIdAtom,
aiChatWindowOpenAtom,
aiChatDraftAtom,
selectedAiRoleIdAtom,
} from "@/features/ai-chat/atoms/ai-chat-atom.ts";
// useMatch is the only react-router-dom export the hook uses; drive its return
// per test to simulate "on a page" vs "off a page".
const useMatchMock = vi.fn();
vi.mock("react-router-dom", () => ({
useMatch: () => useMatchMock(),
}));
// The bound-chat resolver is the network boundary; stub it per test.
const getBoundChatMock = vi.fn();
vi.mock("@/features/ai-chat/services/ai-chat-service.ts", () => ({
getBoundChat: (pageId: string) => getBoundChatMock(pageId),
}));
// Put the hook on a page route by default ("doc-p1" -> page id "p1"); individual
// tests override useMatch to go off-page.
function onPage(pageSlug = "doc-p1") {
useMatchMock.mockReturnValue({ params: { pageSlug } });
}
function offPage() {
useMatchMock.mockReturnValue(null);
}
// Render the hook inside an explicit jotai store so atom side effects are
// assertable; the store is returned for setup + assertions.
function setup(seed?: (store: ReturnType<typeof createStore>) => void) {
const store = createStore();
seed?.(store);
const wrapper = ({ children }: { children: ReactNode }) => (
<Provider store={store}>{children}</Provider>
);
const { result } = renderHook(() => useOpenAiChatForCurrentPage(), { wrapper });
return { store, open: () => act(() => result.current()) };
}
describe("useOpenAiChatForCurrentPage", () => {
beforeEach(() => {
vi.clearAllMocks();
onPage();
});
it("on a page: resolves the bound chat, selects it, and opens the window", async () => {
getBoundChatMock.mockResolvedValue("bound-chat-1");
const { store, open } = setup((s) => s.set(aiChatDraftAtom, "stale draft"));
await open();
expect(getBoundChatMock).toHaveBeenCalledWith("p1");
expect(store.get(activeAiChatIdAtom)).toBe("bound-chat-1");
expect(store.get(aiChatWindowOpenAtom)).toBe(true);
expect(store.get(aiChatDraftAtom)).toBe(""); // cleared on a real switch
});
it("on a page with no bound chat: opens a fresh chat (null)", async () => {
getBoundChatMock.mockResolvedValue(null);
const { store, open } = setup((s) => s.set(activeAiChatIdAtom, "previous"));
await open();
expect(store.get(activeAiChatIdAtom)).toBeNull();
expect(store.get(aiChatWindowOpenAtom)).toBe(true);
});
it("off a page: keeps the current selection and does NOT resolve", async () => {
offPage();
const { store, open } = setup((s) => {
s.set(activeAiChatIdAtom, "keep-me");
s.set(aiChatDraftAtom, "untouched");
});
await open();
expect(getBoundChatMock).not.toHaveBeenCalled();
expect(store.get(activeAiChatIdAtom)).toBe("keep-me");
expect(store.get(aiChatDraftAtom)).toBe("untouched"); // no switch -> kept
expect(store.get(aiChatWindowOpenAtom)).toBe(true);
});
it("window already open: re-click does NOT re-resolve or switch chats", async () => {
getBoundChatMock.mockResolvedValue("would-switch");
const { store, open } = setup((s) => {
s.set(aiChatWindowOpenAtom, true);
s.set(activeAiChatIdAtom, "current");
});
await open();
expect(getBoundChatMock).not.toHaveBeenCalled();
expect(store.get(activeAiChatIdAtom)).toBe("current");
expect(store.get(aiChatWindowOpenAtom)).toBe(true);
});
it("does NOT clear the draft when the resolved chat equals the current one", async () => {
getBoundChatMock.mockResolvedValue("same");
const { store, open } = setup((s) => {
s.set(activeAiChatIdAtom, "same");
s.set(aiChatDraftAtom, "in-progress");
});
await open();
expect(store.get(aiChatDraftAtom)).toBe("in-progress"); // no switch
expect(store.get(aiChatWindowOpenAtom)).toBe(true);
});
it("fail-soft: a resolve error opens a fresh chat (null)", async () => {
getBoundChatMock.mockRejectedValue(new Error("network"));
const { store, open } = setup((s) => s.set(activeAiChatIdAtom, "previous"));
await open();
expect(store.get(activeAiChatIdAtom)).toBeNull();
expect(store.get(aiChatWindowOpenAtom)).toBe(true);
});
it("clears the picked role on a real switch", async () => {
getBoundChatMock.mockResolvedValue("bound");
const { store, open } = setup((s) => s.set(selectedAiRoleIdAtom, "role-1"));
await open();
expect(store.get(selectedAiRoleIdAtom)).toBeNull();
});
});

View File

@@ -0,0 +1,67 @@
import { useCallback } from "react";
import { useAtom, useSetAtom } from "jotai";
import { useMatch } from "react-router-dom";
import {
aiChatWindowOpenAtom,
activeAiChatIdAtom,
aiChatDraftAtom,
selectedAiRoleIdAtom,
} from "@/features/ai-chat/atoms/ai-chat-atom.ts";
import { getBoundChat } from "@/features/ai-chat/services/ai-chat-service.ts";
import { extractPageSlugId } from "@/lib";
/**
* The generic "open the AI chat" action, WITH document binding: when invoked
* while viewing a page, it resolves that page's bound chat and selects it before
* opening — so the last chat for this document re-opens by itself. With no bound
* chat (or off a page) it keeps the current selection / opens a fresh chat. Used
* by the app-header entry point; NOT by the provenance badge (which deep-links).
*/
export function useOpenAiChatForCurrentPage() {
const [windowOpen, setWindowOpen] = useAtom(aiChatWindowOpenAtom);
const [activeChatId, setActiveChatId] = useAtom(activeAiChatIdAtom);
const setDraft = useSetAtom(aiChatDraftAtom);
const setSelectedRoleId = useSetAtom(selectedAiRoleIdAtom);
// Same route-match trick the window uses: read :pageSlug from the pathname.
// AiChatWindow lives in a pathless parent layout route, so useParams() can't
// see :pageSlug — match the full path against the authenticated page route.
const match = useMatch("/s/:spaceSlug/p/:pageSlug");
const pageId = extractPageSlugId(match?.params?.pageSlug);
return useCallback(async () => {
// Re-clicks while the window is already open (incl. minimized) must NOT
// re-resolve and yank the user to another chat: resolve only on a genuine
// closed -> open transition. (`windowOpen` is already true here, so there
// is nothing to set — just bail.)
if (windowOpen) return;
// Open the window FIRST so the control feels instant: the bound-chat
// round-trip below must never gate the window appearing, or on a slow
// connection the first click reads as a hung control until the POST returns.
setWindowOpen(true);
let resolved: string | null = activeChatId; // off-a-page: keep current
if (pageId) {
try {
resolved = await getBoundChat(pageId); // null => fresh chat
} catch {
resolved = null; // fail-soft: a fresh chat is always a safe fallback
}
}
// Clear the composer draft / picked role ONLY on an actual switch, so
// reopening the same chat does not wipe an in-progress draft. Applied after
// the resolve so the window is already visible while the switch settles.
if (resolved !== activeChatId) {
setActiveChatId(resolved);
setDraft("");
setSelectedRoleId(null);
}
}, [
windowOpen,
activeChatId,
pageId,
setWindowOpen,
setActiveChatId,
setDraft,
setSelectedRoleId,
]);
}

View File

@@ -13,21 +13,40 @@ import {
deleteAiRole,
getAiChatMessages,
getAiChats,
getAiRoleCatalog,
getAiRoleCatalogBundle,
getAiRoles,
importAiRolesFromCatalog,
renameAiChat,
updateAiRole,
updateAiRoleFromCatalog,
} from "@/features/ai-chat/services/ai-chat-service.ts";
import {
IAiChat,
IAiChatMessageRow,
IAiRole,
IAiRoleCatalog,
IAiRoleCatalogBundle,
IAiRoleCreate,
IAiRoleImportPayload,
IAiRoleImportResult,
IAiRoleUpdate,
IAiRoleUpdateFromCatalogResult,
} from "@/features/ai-chat/types/ai-chat.types.ts";
import { IPagination } from "@/lib/types.ts";
export const AI_CHATS_RQ_KEY = ["ai-chats"];
export const AI_ROLES_RQ_KEY = ["ai-roles"];
// Catalog reads resolve bundle names per language, so the language is part of
// the cache key (a language switch refetches rather than reusing stale names).
export const AI_ROLE_CATALOG_RQ_KEY = (language: string) => [
"ai-role-catalog",
language,
];
export const AI_ROLE_CATALOG_BUNDLE_RQ_KEY = (
bundleId: string,
language: string,
) => ["ai-role-catalog-bundle", bundleId, language];
export const AI_CHAT_MESSAGES_RQ_KEY = (chatId: string) => [
"ai-chat-messages",
chatId,
@@ -223,3 +242,109 @@ export function useDeleteAiRoleMutation() {
},
});
}
/**
* Browse the role catalog for a language. Gated by `enabled` so the (admin-only)
* fetch runs only when the catalog modal is open. The catalog can 502 when the
* curated source is unreachable; callers handle the error state in the UI.
*/
export function useAiRoleCatalogQuery(language: string, enabled: boolean) {
return useQuery<IAiRoleCatalog, Error>({
queryKey: AI_ROLE_CATALOG_RQ_KEY(language),
queryFn: () => getAiRoleCatalog(language),
enabled,
});
}
/**
* Open one catalog bundle (role content + versions). Gated by `enabled` so the
* fetch only runs when a bundle is actually expanded.
*/
export function useAiRoleCatalogBundleQuery(
bundleId: string,
language: string,
enabled: boolean,
) {
return useQuery<IAiRoleCatalogBundle, Error>({
queryKey: AI_ROLE_CATALOG_BUNDLE_RQ_KEY(bundleId, language),
queryFn: () => getAiRoleCatalogBundle(bundleId, language),
enabled,
});
}
export function useImportAiRolesFromCatalogMutation() {
const queryClient = useQueryClient();
const { t } = useTranslation();
return useMutation<IAiRoleImportResult, Error, IAiRoleImportPayload>({
mutationFn: (payload) => importAiRolesFromCatalog(payload),
onSuccess: (result) => {
notifications.show({
message: t("Imported {{created}}, renamed {{renamed}}, skipped {{skipped}}", {
created: result.created,
renamed: result.renamed,
skipped: result.skipped,
}),
});
// Surface partial failures (e.g. unique-name races) as a red warning.
if (result.errors.length > 0) {
notifications.show({
color: "red",
message: t("Failed to import {{count}} role(s)", {
count: result.errors.length,
}),
});
}
queryClient.invalidateQueries({ queryKey: AI_ROLES_RQ_KEY });
// Imported roles can appear in the chat picker / badges.
queryClient.invalidateQueries({ queryKey: AI_CHATS_RQ_KEY });
},
onError: (error) => {
const message = error["response"]?.data?.message;
notifications.show({
message: message ?? t("Failed to update data"),
color: "red",
});
},
});
}
export function useUpdateAiRoleFromCatalogMutation() {
const queryClient = useQueryClient();
const { t } = useTranslation();
return useMutation<IAiRoleUpdateFromCatalogResult, Error, string>({
mutationFn: (id) => updateAiRoleFromCatalog(id),
onSuccess: (result) => {
// The server returns updated:false with a reason for a no-op (already
// up to date / removed from catalog / language no longer offered). Map
// each reason to a specific message instead of a generic "up to date".
// Narrow the discriminated union via `"reason" in result` (the `updated`
// boolean discriminant does not narrow under this project's
// strictNullChecks:false). Inside the branch, `reason` is the typed literal
// union, so the comparisons below are compiler-checked.
let message: string;
if (!("reason" in result)) {
message = t("Updated to the latest version");
} else if (result.reason === "not-in-catalog") {
message = t("This role is no longer in the catalog");
} else if (result.reason === "language-unavailable") {
message = t("This language is no longer available in the catalog");
} else {
// "up-to-date" (the only remaining reason).
message = t("Already up to date");
}
notifications.show({ message });
queryClient.invalidateQueries({ queryKey: AI_ROLES_RQ_KEY });
// The role badge denormalized onto the chat list may have changed.
queryClient.invalidateQueries({ queryKey: AI_CHATS_RQ_KEY });
},
onError: (error) => {
const message = error["response"]?.data?.message;
notifications.show({
message: message ?? t("Failed to update data"),
color: "red",
});
},
});
}

View File

@@ -0,0 +1,106 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import React from "react";
import { renderHook, waitFor } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { IAiRoleImportResult } from "@/features/ai-chat/types/ai-chat.types.ts";
// `useImportAiRolesFromCatalogMutation` always shows an Imported/renamed/skipped
// summary, and ADDITIONALLY a red "Failed to import N role(s)" notification when
// the result carries partial errors. These tests pin both branches via
// renderHook with a mocked service (twin precedent:
// update-from-catalog-message.test.tsx).
const notificationsShowMock = vi.fn();
vi.mock("@mantine/notifications", () => ({
notifications: { show: (opts: unknown) => notificationsShowMock(opts) },
}));
// `t` echoes the key with interpolated values so we assert against the exact
// English message strings (mirrors react-i18next's default interpolation).
vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (key: string, vars?: Record<string, unknown>) =>
vars
? key.replace(/\{\{(\w+)\}\}/g, (_m, name) => String(vars[name]))
: key,
}),
}));
vi.mock("@/features/ai-chat/services/ai-chat-service.ts", () => ({
importAiRolesFromCatalog: vi.fn(),
// Other named exports referenced by ai-chat-query.ts must exist on the mock so
// the module import resolves; they are unused by these tests.
createAiRole: vi.fn(),
deleteAiChat: vi.fn(),
deleteAiRole: vi.fn(),
getAiChatMessages: vi.fn(),
getAiChats: vi.fn(),
getAiRoleCatalog: vi.fn(),
getAiRoleCatalogBundle: vi.fn(),
getAiRoles: vi.fn(),
renameAiChat: vi.fn(),
updateAiRole: vi.fn(),
updateAiRoleFromCatalog: vi.fn(),
}));
import { importAiRolesFromCatalog } from "@/features/ai-chat/services/ai-chat-service.ts";
import { useImportAiRolesFromCatalogMutation } from "@/features/ai-chat/queries/ai-chat-query.ts";
function createWrapper() {
const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false }, mutations: { retry: false } },
});
return function Wrapper({ children }: { children: React.ReactNode }) {
return (
<QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
);
};
}
async function runMutation(result: IAiRoleImportResult) {
vi.mocked(importAiRolesFromCatalog).mockResolvedValue(result);
const { result: hook } = renderHook(
() => useImportAiRolesFromCatalogMutation(),
{ wrapper: createWrapper() },
);
hook.current.mutate({
bundleId: "general",
language: "en",
conflict: "rename",
});
await waitFor(() => expect(hook.current.isSuccess).toBe(true));
}
describe("useImportAiRolesFromCatalogMutation — success notifications", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("errors:[] -> only the summary notification (counts interpolated)", async () => {
await runMutation({ created: 3, renamed: 1, skipped: 2, errors: [] });
expect(notificationsShowMock).toHaveBeenCalledTimes(1);
expect(notificationsShowMock).toHaveBeenCalledWith({
message: "Imported 3, renamed 1, skipped 2",
});
});
it("errors.length > 0 -> summary PLUS the red failure notification", async () => {
await runMutation({
created: 1,
renamed: 0,
skipped: 0,
errors: [
{ slug: "a", message: "name taken" },
{ slug: "b", message: "name taken" },
],
});
expect(notificationsShowMock).toHaveBeenCalledTimes(2);
expect(notificationsShowMock).toHaveBeenNthCalledWith(1, {
message: "Imported 1, renamed 0, skipped 0",
});
expect(notificationsShowMock).toHaveBeenNthCalledWith(2, {
color: "red",
message: "Failed to import 2 role(s)",
});
});
});

View File

@@ -0,0 +1,100 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import React from "react";
import { renderHook, waitFor } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { IAiRoleUpdateFromCatalogResult } from "@/features/ai-chat/types/ai-chat.types.ts";
// `useUpdateAiRoleFromCatalogMutation` maps the server's discriminated result to
// a user-facing notification message. These tests pin each of the four branches
// (updated / not-in-catalog / language-unavailable / up-to-date) via renderHook
// with a mocked service (precedent: share-query.null-normalization.test.tsx).
const notificationsShowMock = vi.fn();
vi.mock("@mantine/notifications", () => ({
notifications: { show: (opts: unknown) => notificationsShowMock(opts) },
}));
// `t` echoes the key so we assert against the exact English message strings.
vi.mock("react-i18next", () => ({
useTranslation: () => ({ t: (key: string) => key }),
}));
vi.mock("@/features/ai-chat/services/ai-chat-service.ts", () => ({
updateAiRoleFromCatalog: vi.fn(),
// Other named exports referenced by ai-chat-query.ts must exist on the mock so
// the module import resolves; they are unused by these tests.
createAiRole: vi.fn(),
deleteAiChat: vi.fn(),
deleteAiRole: vi.fn(),
getAiChatMessages: vi.fn(),
getAiChats: vi.fn(),
getAiRoleCatalog: vi.fn(),
getAiRoleCatalogBundle: vi.fn(),
getAiRoles: vi.fn(),
importAiRolesFromCatalog: vi.fn(),
renameAiChat: vi.fn(),
updateAiRole: vi.fn(),
}));
import { updateAiRoleFromCatalog } from "@/features/ai-chat/services/ai-chat-service.ts";
import { useUpdateAiRoleFromCatalogMutation } from "@/features/ai-chat/queries/ai-chat-query.ts";
function createWrapper() {
const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false }, mutations: { retry: false } },
});
return function Wrapper({ children }: { children: React.ReactNode }) {
return (
<QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
);
};
}
async function runMutation(result: IAiRoleUpdateFromCatalogResult) {
vi.mocked(updateAiRoleFromCatalog).mockResolvedValue(result);
const { result: hook } = renderHook(
() => useUpdateAiRoleFromCatalogMutation(),
{ wrapper: createWrapper() },
);
hook.current.mutate("role-1");
await waitFor(() => expect(hook.current.isSuccess).toBe(true));
}
describe("useUpdateAiRoleFromCatalogMutation — reason → message", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("updated:true -> 'Updated to the latest version'", async () => {
await runMutation({
updated: true,
fromVersion: 1,
toVersion: 2,
role: { id: "role-1" } as never,
});
expect(notificationsShowMock).toHaveBeenCalledWith({
message: "Updated to the latest version",
});
});
it("not-in-catalog -> 'This role is no longer in the catalog'", async () => {
await runMutation({ updated: false, reason: "not-in-catalog" });
expect(notificationsShowMock).toHaveBeenCalledWith({
message: "This role is no longer in the catalog",
});
});
it("language-unavailable -> 'This language is no longer available in the catalog'", async () => {
await runMutation({ updated: false, reason: "language-unavailable" });
expect(notificationsShowMock).toHaveBeenCalledWith({
message: "This language is no longer available in the catalog",
});
});
it("up-to-date -> 'Already up to date'", async () => {
await runMutation({ updated: false, reason: "up-to-date" });
expect(notificationsShowMock).toHaveBeenCalledWith({
message: "Already up to date",
});
});
});

View File

@@ -6,8 +6,13 @@ import {
IAiChatMessageRow,
IAiChatMessagesParams,
IAiRole,
IAiRoleCatalog,
IAiRoleCatalogBundle,
IAiRoleCreate,
IAiRoleImportPayload,
IAiRoleImportResult,
IAiRoleUpdate,
IAiRoleUpdateFromCatalogResult,
} from "@/features/ai-chat/types/ai-chat.types.ts";
/**
@@ -37,6 +42,17 @@ export async function getAiChatMessages(
return req.data;
}
/**
* Resolve the chat bound to a document (the current user's most-recent chat
* created on that page), or null when there is none. Drives auto-open-on-page.
*/
export async function getBoundChat(pageId: string): Promise<string | null> {
const req = await api.post<{ chatId: string | null }>("/ai-chat/bound-chat", {
pageId,
});
return req.data.chatId;
}
/** Rename a chat. */
export async function renameAiChat(data: {
chatId: string;
@@ -112,3 +128,54 @@ export async function deleteAiRole(id: string): Promise<{ success: true }> {
});
return req.data;
}
/**
* Role catalog API (`/ai-chat/roles/*`, admin-only — the server enforces this).
* Browse a curated catalog, import roles/bundles into the workspace, and update
* an imported role when the catalog ships a newer version. Same `{ data }`
* unwrap convention as above.
*/
/** Browse the catalog, optionally localized to `language`. */
export async function getAiRoleCatalog(
language?: string,
): Promise<IAiRoleCatalog> {
const req = await api.post<IAiRoleCatalog>("/ai-chat/roles/catalog", {
language,
});
return req.data;
}
/** Open one catalog bundle in a language (role content + versions). */
export async function getAiRoleCatalogBundle(
bundleId: string,
language: string,
): Promise<IAiRoleCatalogBundle> {
const req = await api.post<IAiRoleCatalogBundle>(
"/ai-chat/roles/catalog/bundle",
{ bundleId, language },
);
return req.data;
}
/** Import roles from a catalog bundle into the workspace (admin). */
export async function importAiRolesFromCatalog(
payload: IAiRoleImportPayload,
): Promise<IAiRoleImportResult> {
const req = await api.post<IAiRoleImportResult>(
"/ai-chat/roles/import",
payload,
);
return req.data;
}
/** Update an already-imported role from its catalog source (admin). */
export async function updateAiRoleFromCatalog(
id: string,
): Promise<IAiRoleUpdateFromCatalogResult> {
const req = await api.post<IAiRoleUpdateFromCatalogResult>(
"/ai-chat/roles/update-from-catalog",
{ id },
);
return req.data;
}

View File

@@ -57,10 +57,79 @@ export interface IAiRole {
autoStart: boolean;
// Custom auto-start text; null/empty => the default launch message is sent.
launchMessage: string | null;
// Catalog origin of an imported role, or null for a manually-created one.
// Admin-only (present only in the admin list view); the picker view omits it.
// The admin UI compares `version` against the catalog to offer an update.
source?: { slug: string; language: string; version: number } | null;
createdAt?: string;
updatedAt?: string;
}
/** One bundle's summary in the catalog index (mirrors `getCatalog().bundles[]`). */
export interface IAiRoleCatalogBundleSummary {
id: string;
name: string;
description: string | null;
languages: string[];
roles: { slug: string; version: number }[];
}
/** The browsable catalog index (mirrors `getCatalog()`). */
export interface IAiRoleCatalog {
languages: string[];
bundles: IAiRoleCatalogBundleSummary[];
}
/** A single role inside an opened catalog bundle (localized content + version). */
export interface IAiRoleCatalogRole {
slug: string;
emoji: string | null;
name: string;
description: string | null;
instructions: string;
autoStart: boolean;
launchMessage: string | null;
version: number;
}
/** An opened catalog bundle (mirrors `getCatalogBundle()`). */
export interface IAiRoleCatalogBundle {
bundleId: string;
language: string;
roles: IAiRoleCatalogRole[];
}
/** Import payload (mirrors the server `ImportFromCatalogDto`). */
export interface IAiRoleImportPayload {
bundleId: string;
language: string;
// Omitted => import the whole bundle; otherwise only these slugs.
slugs?: string[];
conflict: "skip" | "rename";
}
/** Import result counts (mirrors `importFromCatalog()`). */
export interface IAiRoleImportResult {
created: number;
skipped: number;
renamed: number;
errors: { slug: string; message: string }[];
}
/**
* Update-from-catalog result (mirrors the server `updateFromCatalog()`). A
* discriminated union on `updated`: a no-op carries a typed `reason` the UI maps
* to a specific message; a successful update carries the version bump + new role.
* Keeping the union (not a widened `reason?: string`) lets the consumer's literal
* comparisons be compiler-checked.
*/
export type IAiRoleUpdateFromCatalogResult =
| {
updated: false;
reason: "not-in-catalog" | "up-to-date" | "language-unavailable";
}
| { updated: true; fromVersion: number; toVersion: number; role: IAiRole };
/** Admin create payload for a role. */
export interface IAiRoleCreate {
name: string;

View File

@@ -0,0 +1,107 @@
import { describe, it, expect } from "vitest";
import { catalogRoleInstallState } from "./catalog-role-install-state.ts";
import type { IAiRole } from "@/features/ai-chat/types/ai-chat.types.ts";
// Build a workspace role with a catalog source. Fields irrelevant to the
// install-state decision are filled with harmless defaults.
function installedRole(
source: { slug: string; language: string; version: number },
overrides: Partial<IAiRole> = {},
): IAiRole {
return {
id: `role-${source.slug}-${source.language}`,
name: source.slug,
emoji: null,
description: null,
enabled: true,
autoStart: true,
launchMessage: null,
source,
...overrides,
};
}
const catalogRole = { slug: "writer", version: 3 };
// Mirrors the role-launch.ts precedent: the modal's role-state computation is a
// pure function so the import/installed/update decision is testable directly.
describe("catalogRoleInstallState", () => {
it("no matching installed role -> import", () => {
const result = catalogRoleInstallState(catalogRole, [], "en");
expect(result).toEqual({ state: "import" });
});
it("same slug + language, installed version > catalog -> installed", () => {
const installed = installedRole({
slug: "writer",
language: "en",
version: 5,
});
const result = catalogRoleInstallState(catalogRole, [installed], "en");
expect(result).toEqual({ state: "installed", installed });
});
it("same slug + language, installed version == catalog -> installed", () => {
const installed = installedRole({
slug: "writer",
language: "en",
version: 3,
});
const result = catalogRoleInstallState(catalogRole, [installed], "en");
expect(result).toEqual({ state: "installed", installed });
});
it("same slug + language, installed version < catalog -> update (from/to)", () => {
const installed = installedRole({
slug: "writer",
language: "en",
version: 1,
});
const result = catalogRoleInstallState(catalogRole, [installed], "en");
expect(result).toEqual({
state: "update",
installed,
fromVersion: 1,
toVersion: 3,
});
});
it("same slug but DIFFERENT language -> import (a separate install)", () => {
// 'writer' is installed in 'ru'; browsing the 'en' catalog must offer it as a
// fresh import, not treat the ru copy as already installed.
const installed = installedRole({
slug: "writer",
language: "ru",
version: 5,
});
const result = catalogRoleInstallState(catalogRole, [installed], "en");
expect(result).toEqual({ state: "import" });
});
it("matches the right language when the same slug is installed in several", () => {
const ru = installedRole(
{ slug: "writer", language: "ru", version: 5 },
{ id: "ru-role" },
);
const en = installedRole(
{ slug: "writer", language: "en", version: 1 },
{ id: "en-role" },
);
const result = catalogRoleInstallState(catalogRole, [ru, en], "en");
expect(result).toEqual({
state: "update",
installed: en,
fromVersion: 1,
toVersion: 3,
});
});
it("ignores manually-created roles (no source) sharing the name", () => {
const manual = installedRole(
{ slug: "writer", language: "en", version: 9 },
{ source: null },
);
const result = catalogRoleInstallState(catalogRole, [manual], "en");
expect(result).toEqual({ state: "import" });
});
});

View File

@@ -0,0 +1,49 @@
import type {
IAiRole,
IAiRoleCatalogRole,
} from "@/features/ai-chat/types/ai-chat.types.ts";
/**
* The install state of a single catalog role relative to the workspace's
* existing roles. Extracted as a pure function so the catalog modal's role-state
* computation is unit-testable without mounting the component (mirrors the
* `roleLaunchMessage` precedent in role-launch.ts).
*
* A catalog role is matched to an installed role by BOTH `source.slug` and
* `source.language`: the same slug in a different language is a separate install
* (so it shows as "import", not "installed"). When matched, the installed source
* version decides the state:
* - no match -> "import"
* - matched & installed version >= catalog version -> "installed"
* - matched & installed version < catalog version -> "update" (from -> to)
*/
export type CatalogRoleInstallState =
| { state: "import" }
| { state: "installed"; installed: IAiRole }
| {
state: "update";
installed: IAiRole;
fromVersion: number;
toVersion: number;
};
export function catalogRoleInstallState(
role: Pick<IAiRoleCatalogRole, "slug" | "version">,
workspaceRoles: IAiRole[],
language: string,
): CatalogRoleInstallState {
const installed = workspaceRoles.find(
(r) => r.source?.slug === role.slug && r.source?.language === language,
);
if (!installed) return { state: "import" };
const fromVersion = installed.source?.version ?? 0;
if (fromVersion >= role.version) {
return { state: "installed", installed };
}
return {
state: "update",
installed,
fromVersion,
toVersion: role.version,
};
}

View File

@@ -0,0 +1,168 @@
import { describe, it, expect } from "vitest";
import { Editor } from "@tiptap/core";
import { Document } from "@tiptap/extension-document";
import { Paragraph } from "@tiptap/extension-paragraph";
import { Text } from "@tiptap/extension-text";
import { Node as PMNode, Fragment, Slice } from "@tiptap/pm/model";
import {
FootnoteReference,
FootnotesList,
FootnoteDefinition,
FOOTNOTE_REFERENCE_NAME,
FOOTNOTE_DEFINITION_NAME,
FOOTNOTES_LIST_NAME,
} from "@docmost/editor-ext";
import { canonicalizePastedFootnotes } from "./markdown-clipboard";
/**
* A markdown paste builds its ProseMirror fragment via DOM -> parseSlice and is
* applied with a manual transaction (handlePaste returns true), so it bypasses
* the editor's footnoteSyncPlugin — which never reorders an existing list. These
* tests pin canonicalizePastedFootnotes, the focused hook that makes a pasted
* out-of-order markdown footnote block come out canonical (issue #228).
*/
const extensions = [
Document,
Paragraph,
Text,
FootnoteReference,
FootnotesList,
FootnoteDefinition,
];
function makeSchema() {
const editor = new Editor({ extensions, content: { type: "doc", content: [] } });
const { schema } = editor;
return { editor, schema };
}
/** List footnote def ids of the (single) footnotesList in a slice, in order. */
function listIds(slice: Slice): string[] {
const out: string[] = [];
slice.content.forEach((node: PMNode) => {
if (node.type.name === FOOTNOTES_LIST_NAME) {
node.content.forEach((def: PMNode) => {
if (def.type.name === FOOTNOTE_DEFINITION_NAME) out.push(def.attrs.id);
});
}
});
return out;
}
function hasList(slice: Slice): boolean {
let found = false;
slice.content.forEach((n: PMNode) => {
if (n.type.name === FOOTNOTES_LIST_NAME) found = true;
});
return found;
}
describe("canonicalizePastedFootnotes", () => {
it("reorders a pasted block to reference order, dedups reuse, drops orphans", () => {
const { editor, schema } = makeSchema();
// Body references c, a, b (and again a => reuse); definitions a, b, c, z
// (z is an orphan) — the exact shape a markdown paste produces.
const slice = new Slice(
Fragment.fromArray([
schema.nodes.paragraph.create(null, [
schema.text("body "),
schema.nodes[FOOTNOTE_REFERENCE_NAME].create({ id: "c" }),
schema.nodes[FOOTNOTE_REFERENCE_NAME].create({ id: "a" }),
schema.nodes[FOOTNOTE_REFERENCE_NAME].create({ id: "b" }),
schema.nodes[FOOTNOTE_REFERENCE_NAME].create({ id: "a" }),
]),
schema.nodes[FOOTNOTES_LIST_NAME].create(null, [
schema.nodes[FOOTNOTE_DEFINITION_NAME].create({ id: "a" }, [
schema.nodes.paragraph.create(null, [schema.text("note A")]),
]),
schema.nodes[FOOTNOTE_DEFINITION_NAME].create({ id: "b" }, [
schema.nodes.paragraph.create(null, [schema.text("note B")]),
]),
schema.nodes[FOOTNOTE_DEFINITION_NAME].create({ id: "c" }, [
schema.nodes.paragraph.create(null, [schema.text("note C")]),
]),
schema.nodes[FOOTNOTE_DEFINITION_NAME].create({ id: "z" }, [
schema.nodes.paragraph.create(null, [schema.text("orphan")]),
]),
]),
]),
0,
0,
);
const out = canonicalizePastedFootnotes(slice, schema);
// Reference order, orphan z dropped, reused a appears once.
expect(listIds(out)).toEqual(["c", "a", "b"]);
editor.destroy();
});
it("leaves a reference-ONLY paste untouched (no synthesized definitions)", () => {
// A paste that reuses an id defined in the TARGET doc must NOT gain a
// synthesized empty definition here — it carries no footnotesList of its own.
const { editor, schema } = makeSchema();
const slice = new Slice(
Fragment.from(
schema.nodes.paragraph.create(null, [
schema.text("see "),
schema.nodes[FOOTNOTE_REFERENCE_NAME].create({ id: "a" }),
]),
),
0,
0,
);
const out = canonicalizePastedFootnotes(slice, schema);
expect(hasList(out)).toBe(false);
expect(out).toBe(slice); // returned unchanged (same reference)
editor.destroy();
});
it("leaves a definitions-ONLY paste untouched (no references -> no empty paste)", () => {
// A whole-block paste of ONLY definitions (a footnotesList with no matching
// footnoteReference anywhere in the selection). Canonicalizing it would strip
// the reference-less list -> an EMPTY paste, losing the pasted text. The hook
// must leave such a block untouched.
const { editor, schema } = makeSchema();
const slice = new Slice(
Fragment.fromArray([
schema.nodes[FOOTNOTES_LIST_NAME].create(null, [
schema.nodes[FOOTNOTE_DEFINITION_NAME].create({ id: "a" }, [
schema.nodes.paragraph.create(null, [schema.text("note A")]),
]),
schema.nodes[FOOTNOTE_DEFINITION_NAME].create({ id: "b" }, [
schema.nodes.paragraph.create(null, [schema.text("note B")]),
]),
]),
]),
0,
0,
);
const out = canonicalizePastedFootnotes(slice, schema);
expect(out).toBe(slice); // returned unchanged (same reference, content kept)
expect(listIds(out)).toEqual(["a", "b"]);
editor.destroy();
});
it("leaves an open (partial) slice untouched even if it carries a list", () => {
// An open slice (openStart/openEnd > 0) is a partial selection, not a
// standalone block, so it is returned as-is BEFORE any footnote handling.
const { editor, schema } = makeSchema();
const slice = new Slice(
Fragment.fromArray([
schema.nodes.paragraph.create(null, [
schema.nodes[FOOTNOTE_REFERENCE_NAME].create({ id: "a" }),
]),
schema.nodes[FOOTNOTES_LIST_NAME].create(null, [
schema.nodes[FOOTNOTE_DEFINITION_NAME].create({ id: "a" }, [
schema.nodes.paragraph.create(null, [schema.text("A")]),
]),
]),
]),
1,
1,
);
const out = canonicalizePastedFootnotes(slice, schema);
expect(out).toBe(slice);
editor.destroy();
});
});

View File

@@ -3,7 +3,14 @@ import { Extension } from "@tiptap/core";
import { Plugin, PluginKey, TextSelection } from "@tiptap/pm/state";
import { DOMParser, DOMSerializer, Fragment, Slice } from "@tiptap/pm/model";
import { find } from "linkifyjs";
import { markdownToHtml, htmlToMarkdown } from "@docmost/editor-ext";
import {
markdownToHtml,
htmlToMarkdown,
canonicalizeFootnotes,
FOOTNOTES_LIST_NAME,
FOOTNOTE_REFERENCE_NAME,
} from "@docmost/editor-ext";
import type { Schema } from "@tiptap/pm/model";
export const MarkdownClipboard = Extension.create({
name: "markdownClipboard",
@@ -83,12 +90,25 @@ export const MarkdownClipboard = Extension.create({
const body = elementFromString(parsed);
normalizeTableColumnWidths(body);
const contentNodes = DOMParser.fromSchema(
const parsedSlice = DOMParser.fromSchema(
this.editor.schema,
).parseSlice(body, {
preserveWhitespace: true,
});
// A markdown paste builds its ProseMirror fragment directly (DOM ->
// parseSlice), bypassing the editor's footnoteSyncPlugin, which never
// reorders an existing list. So a pasted markdown block whose footnote
// definitions are out of order (or contains orphan defs) would be
// stored out of order. Canonicalize the self-contained pasted block so
// its footnotes come out reference-ordered, deduped and orphan-free
// (issue #228). See canonicalizePastedFootnotes for why this is scoped
// to whole-block pastes that carry their own footnotesList.
const contentNodes = canonicalizePastedFootnotes(
parsedSlice,
this.editor.schema,
);
tr.replaceRange(from, to, contentNodes);
const insertEnd = tr.mapping.map(from, 1);
tr.setSelection(TextSelection.near(tr.doc.resolve(Math.max(from, insertEnd - 2)), -1));
@@ -133,6 +153,54 @@ export const MarkdownClipboard = Extension.create({
},
});
/**
* Reorder/dedup the footnotes of a SELF-CONTAINED pasted markdown block to the
* canonical invariant (the live footnoteSyncPlugin never reorders an existing
* list, so an out-of-order pasted block would otherwise persist out of order).
*
* Scoped deliberately to whole-block pastes (openStart/openEnd === 0) that carry
* their OWN footnotesList: canonicalizeFootnotes would synthesize empty
* definitions for any reference lacking a definition, which is correct for a
* standalone block but would be wrong for a reference-only paste that REUSES a
* footnote already defined in the target document — so those are left untouched
* for the paste/sync plugins to merge. Residual: when the pasted block is merged
* into a doc that already has footnotes, ordering RELATIVE to the pre-existing
* footnotes is still governed by the sync plugin (which does not reorder).
*
* Also requires at least one footnoteReference in the selection: a definitions-ONLY
* paste (`[^a]: …` with no `[^a]` reference in the same block) has no references,
* so canonicalizeFootnotes would drop the whole list and the paste would come out
* EMPTY — losing the pasted text. Such a block is left as-is for the sync plugin.
*/
export function canonicalizePastedFootnotes(slice: Slice, schema: Schema): Slice {
if (slice.openStart !== 0 || slice.openEnd !== 0) return slice;
let hasFootnotesList = false;
let hasReference = false;
slice.content.forEach((node) => {
if (node.type.name === FOOTNOTES_LIST_NAME) hasFootnotesList = true;
// footnoteReference is an inline atom, never a top-level slice child here
// (this function early-returns for open slices, so children are whole
// blocks), so it is only reachable by descending.
node.descendants((child) => {
if (child.type.name === FOOTNOTE_REFERENCE_NAME) hasReference = true;
});
});
if (!hasFootnotesList) return slice;
// No reference anywhere -> a definitions-only paste; canonicalizing would strip
// the reference-less list (empty paste). Leave it untouched.
if (!hasReference) return slice;
const content = slice.content.toJSON();
if (!Array.isArray(content)) return slice;
const canonical = canonicalizeFootnotes({ type: "doc", content }) as {
content?: unknown[];
};
const fragment = Fragment.fromJSON(schema, canonical.content ?? []);
return new Slice(fragment, 0, 0);
}
function elementFromString(value) {
// add a wrapper to preserve leading and trailing whitespace
const wrappedValue = `<body>${value}</body>`;

View File

@@ -1,4 +1,4 @@
import { Button, Group, Menu, Text } from "@mantine/core";
import { Button, Menu, Stack, Text } from "@mantine/core";
import { IconHourglass, IconPlus } from "@tabler/icons-react";
import { ReactNode } from "react";
import { useNavigate } from "react-router-dom";
@@ -21,11 +21,15 @@ function CreateNoteButton({
temporary,
label,
icon,
color,
}: {
writableSpaces: ISpace[];
temporary: boolean;
label: string;
icon: ReactNode;
// Mantine color token; lets the temporary action tint toward the warm
// orange/amber used by the clock marker + banner while "New note" stays neutral.
color: string;
}) {
const { t } = useTranslation();
const navigate = useNavigate();
@@ -54,7 +58,8 @@ function CreateNoteButton({
<Button
size="md"
variant="light"
color="gray"
color={color}
fullWidth
leftSection={icon}
loading={isPending}
onClick={() => createNote(writableSpaces[0])}
@@ -71,7 +76,8 @@ function CreateNoteButton({
<Button
size="md"
variant="light"
color="gray"
color={color}
fullWidth
leftSection={icon}
loading={isPending}
>
@@ -109,8 +115,10 @@ function CreateNoteButton({
// Prominent home-screen actions to create a new note (page). Because the home
// screen has no active space, the target space is resolved from the user's
// writable spaces: created directly when there is one, picked from a dropdown
// when there are several. Renders two equal-width buttons: a regular note and a
// temporary note (which auto-moves to Trash after the workspace lifetime).
// when there are several. Renders two full-width, vertically stacked buttons: a
// neutral regular note and an orange-tinted temporary note (which auto-moves to
// Trash after the workspace lifetime). Stacking full-width keeps the longer
// "New temporary note" label from clipping on narrow mobile widths.
export default function NewNoteButton() {
const { t } = useTranslation();
const { data } = useGetSpacesQuery({ limit: 100 });
@@ -121,19 +129,21 @@ export default function NewNoteButton() {
if (writableSpaces.length === 0) return null;
return (
<Group grow gap="sm">
<Stack gap="sm">
<CreateNoteButton
writableSpaces={writableSpaces}
temporary={false}
label={t("New note")}
icon={<IconPlus size={18} />}
color="gray"
/>
<CreateNoteButton
writableSpaces={writableSpaces}
temporary={true}
label={t("New temporary note")}
icon={<IconHourglass size={18} />}
color="orange"
/>
</Group>
</Stack>
);
}

View File

@@ -0,0 +1,69 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { getDefaultStore } from "jotai";
// Mock the app entry so importing the query module doesn't boot the whole app
// (it only needs queryClient's cache methods, which we stub here). The spies are
// declared via vi.hoisted so they exist before the hoisted vi.mock factory runs.
const { setQueryData, getQueryData, invalidateQueries } = vi.hoisted(() => ({
setQueryData: vi.fn(),
getQueryData: vi.fn(() => undefined as unknown),
invalidateQueries: vi.fn(),
}));
vi.mock("@/main.tsx", () => ({
queryClient: { setQueryData, getQueryData, invalidateQueries },
}));
import { syncTemporaryExpiresInCache } from "./page-embed-query";
import { treeDataAtom } from "@/features/page/tree/atoms/tree-data-atom.ts";
import { SpaceTreeNode } from "@/features/page/tree/types.ts";
const mkNode = (id: string, slugId: string): SpaceTreeNode =>
({
id,
slugId,
name: id,
position: "a0",
spaceId: "space-1",
parentPageId: null,
hasChildren: false,
children: [],
}) as unknown as SpaceTreeNode;
describe("syncTemporaryExpiresInCache — treeDataAtom patch", () => {
beforeEach(() => {
vi.clearAllMocks();
getQueryData.mockReturnValue(undefined);
});
it("patches the in-tree node's temporaryExpiresAt (sidebar marker updates without reload)", () => {
const store = getDefaultStore();
const tree = [mkNode("p1", "slug-1"), mkNode("p2", "slug-2")];
store.set(treeDataAtom, tree);
const deadline = "2026-07-01T00:00:00.000Z";
syncTemporaryExpiresInCache({ id: "p1", slugId: "slug-1" }, deadline);
const next = store.get(treeDataAtom);
// A new atom value was written...
expect(next).not.toBe(tree);
// ...the matching node gained the deadline...
expect(next.find((n) => n.id === "p1")?.temporaryExpiresAt).toBe(deadline);
// ...and the untouched sibling is unchanged.
expect(next.find((n) => n.id === "p2")?.temporaryExpiresAt).toBeUndefined();
});
it("leaves the atom value at the SAME reference when the id is absent from the tree (no write)", () => {
const store = getDefaultStore();
const tree = [mkNode("p1", "slug-1")];
store.set(treeDataAtom, tree);
syncTemporaryExpiresInCache(
{ id: "not-in-tree", slugId: "missing" },
"2026-07-01T00:00:00.000Z",
);
// treeModel.update is a no-op (same reference) for an unknown id, so the
// guard skips the store write entirely — same reference back.
expect(store.get(treeDataAtom)).toBe(tree);
});
});

View File

@@ -1,5 +1,6 @@
import { useMutation } from "@tanstack/react-query";
import { notifications } from "@mantine/notifications";
import { getDefaultStore } from "jotai";
import {
toggleTemplate,
toggleTemporary,
@@ -9,6 +10,9 @@ import type {
ToggleTemporaryResponse,
} from "@/features/page-embed/types/page-embed.types";
import { queryClient } from "@/main.tsx";
import { treeDataAtom } from "@/features/page/tree/atoms/tree-data-atom.ts";
import { treeModel } from "@/features/page/tree/model/tree-model";
import { SpaceTreeNode } from "@/features/page/tree/types.ts";
/**
* After toggling a note's temporary state, mirror the new deadline into the
@@ -30,6 +34,19 @@ export function syncTemporaryExpiresInCache(
});
}
}
// Patch the in-memory sidebar tree node so its temporary clock marker
// appears/disappears immediately — WITHOUT a reload. The page cache update
// above only drives the in-page banner/menu; the sidebar reads
// `temporaryExpiresAt` straight off the `treeDataAtom` node. The app uses
// jotai's default store (no <Provider>), so `getDefaultStore()` is the same
// store the sidebar's hooks read from. `treeModel.update` returns the same
// reference (a no-op) when the page isn't in the currently loaded tree.
const store = getDefaultStore();
const prevTree = store.get(treeDataAtom);
const nextTree = treeModel.update(prevTree, page.id, {
temporaryExpiresAt,
} as Partial<SpaceTreeNode>);
if (nextTree !== prevTree) store.set(treeDataAtom, nextTree);
queryClient.invalidateQueries({
predicate: (item) =>
["sidebar-pages"].includes(item.queryKey[0] as string),

View File

@@ -176,8 +176,8 @@ function PageActionMenu({ readOnly }: PageActionMenuProps) {
pageId: page.id,
temporary: next,
});
// Reflect the new deadline in the page cache so the menu label flips and
// any banner updates. The sidebar icon refreshes via its own query.
// Reflect the new deadline in the page cache (menu label + banner) AND in
// the sidebar tree node so its clock marker updates immediately, no reload.
syncTemporaryExpiresInCache(page, res.temporaryExpiresAt);
notifications.show({
message: next

View File

@@ -32,7 +32,7 @@ import {
import { notifications } from "@mantine/notifications";
import { IPagination, QueryParams } from "@/lib/types.ts";
import { queryClient } from "@/main.tsx";
import { buildTree } from "@/features/page/tree/utils";
import { buildTree, pageToTreeNode } from "@/features/page/tree/utils";
import { useEffect } from "react";
import { validate as isValidUuid } from "uuid";
import { useTranslation } from "react-i18next";
@@ -210,18 +210,15 @@ export function useRestorePageMutation() {
// Check if the page already exists in the tree (it shouldn't)
if (!treeModel.find(currentTree, restoredPage.id)) {
// Create the tree node data with hasChildren from backend
const nodeData: SpaceTreeNode = {
id: restoredPage.id,
slugId: restoredPage.slugId,
// Create the tree node data with hasChildren from backend. Routed
// through the canonical mapper so the field copy stays in lockstep with
// buildTree. The server NULLS `temporaryExpiresAt` on restore (a restored
// page is made permanent), so the mapper carries that null through and
// the node correctly shows no clock marker.
const nodeData: SpaceTreeNode = pageToTreeNode(restoredPage, {
name: restoredPage.title || "Untitled",
icon: restoredPage.icon,
position: restoredPage.position,
spaceId: restoredPage.spaceId,
parentPageId: restoredPage.parentPageId,
hasChildren: restoredPage.hasChildren || false,
children: [],
};
});
// Determine the parent and index
const parentId = restoredPage.parentPageId || null;
@@ -410,6 +407,11 @@ export function invalidateOnCreatePage(data: Partial<IPage>) {
slugId: data.slugId,
spaceId: data.spaceId,
title: data.title,
// Carry the death-timer deadline so a note created as temporary keeps its
// sidebar clock marker when the tree is rebuilt from this cached entry
// (buildTree → mergeRootTrees). Omitting it overwrote the optimistic/socket
// node's marker with `undefined`, hiding it until a reload.
temporaryExpiresAt: data.temporaryExpiresAt,
};
let queryKey: QueryKey = null;

View File

@@ -37,6 +37,7 @@ import {
} from "@/features/page-embed/queries/page-embed-query";
import { treeDataAtom } from "@/features/page/tree/atoms/tree-data-atom.ts";
import { treeModel } from "@/features/page/tree/model/tree-model";
import { pageToTreeNode } from "@/features/page/tree/utils";
import { useTreeMutation } from "@/features/page/tree/hooks/use-tree-mutation.ts";
import type { SpaceTreeNode } from "@/features/page/tree/types.ts";
import classes from "@/features/page/tree/styles/tree.module.css";
@@ -130,18 +131,14 @@ export function NodeMenu({ node, canEdit }: NodeMenuProps) {
const currentIndex = siblings?.index ?? 0;
const newIndex = currentIndex + 1;
const treeNodeData: SpaceTreeNode = {
id: duplicatedPage.id,
slugId: duplicatedPage.slugId,
name: duplicatedPage.title,
position: duplicatedPage.position,
spaceId: duplicatedPage.spaceId,
parentPageId: duplicatedPage.parentPageId,
icon: duplicatedPage.icon,
hasChildren: duplicatedPage.hasChildren,
// Routed through the canonical mapper so the field copy stays in lockstep
// with buildTree. The server does NOT arm a death timer on duplicate (the
// copy's `temporaryExpiresAt` defaults to null = permanent), so the mapper
// carries that null through and the duplicated node correctly shows no
// clock marker — matching the server without a reload.
const treeNodeData: SpaceTreeNode = pageToTreeNode(duplicatedPage, {
canEdit: true,
children: [],
};
});
setData((prev) =>
treeModel.insert(prev, parentId, treeNodeData, newIndex),

View File

@@ -9,6 +9,7 @@ import { treeModel } from "@/features/page/tree/model/tree-model";
import type { DropOp } from "@/features/page/tree/model/tree-model.types";
import { dropOpToMovePayload } from "./drop-op-to-move-payload";
import { SpaceTreeNode } from "@/features/page/tree/types.ts";
import { pageToTreeNode } from "@/features/page/tree/utils";
import { IPage } from "@/features/page/types/page.types.ts";
import {
useCreatePageMutation,
@@ -139,18 +140,15 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
throw new Error("Failed to create page");
}
const newNode: SpaceTreeNode = {
id: createdPage.id,
slugId: createdPage.slugId,
// Route through the canonical mapper so the field copy (esp.
// `temporaryExpiresAt`, which shows the temporary-note clock marker on
// optimistic insert) can't drift from buildTree. `name: ""` because a
// freshly created page is untitled; `hasChildren: false` because it has no
// children yet.
const newNode: SpaceTreeNode = pageToTreeNode(createdPage, {
name: "",
position: createdPage.position,
spaceId: createdPage.spaceId,
parentPageId: createdPage.parentPageId,
hasChildren: false,
// Show the temporary-note icon immediately on optimistic insert.
temporaryExpiresAt: createdPage.temporaryExpiresAt,
children: [],
};
});
// Read latest tree at call time. Without this, callers that mutate the
// tree (e.g. lazy-load children on expand) immediately before calling
@@ -173,7 +171,22 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
// optimistic node's id IS the real created page id (createdPage.id), so
// the ids match exactly regardless of which path runs first.
setData((prev) => {
if (treeModel.find(prev, newNode.id)) return prev;
const existing = treeModel.find(prev, newNode.id);
if (existing) {
// The server `addTreeNode` broadcast won the race and already inserted
// this node. Older broadcasts could omit `temporaryExpiresAt`, leaving
// a temporary note WITHOUT its clock marker until reload; patch it on
// from the authoritative create response so the marker shows now.
if (
newNode.temporaryExpiresAt &&
!(existing as SpaceTreeNode).temporaryExpiresAt
) {
return treeModel.update(prev, newNode.id, {
temporaryExpiresAt: newNode.temporaryExpiresAt,
} as Partial<SpaceTreeNode>);
}
return prev;
}
return treeModel.insert(prev, parentId, newNode, lastIndex);
});

View File

@@ -393,6 +393,101 @@ describe("handleCreate optimistic-insert idempotency (find-then-skip)", () => {
});
});
// handleCreate race-guard temporaryExpiresAt patch: when the server's
// addTreeNode broadcast wins the race and inserts the node BEFORE the optimistic
// updater runs, the updater must not re-insert. Two sub-branches:
// (a) the node the broadcast inserted carries NO deadline (an older broadcast
// omitted it) while the authoritative create response DOES → patch the
// deadline on so the clock marker shows now, without a reload.
// (b) the existing node ALREADY has a deadline → do NOT overwrite it; return
// `prev` by reference (a no-op write).
describe("handleCreate race-guard temporaryExpiresAt patch", () => {
type TN = TreeNode<{ name: string; temporaryExpiresAt?: string | null }>;
// Mirrors the setData updater in use-tree-mutation handleCreate.
const applyOptimisticInsert = (
tree: TN[],
parentId: string | null,
node: TN,
index: number,
): TN[] => {
const existing = treeModel.find(tree, node.id) as TN | null;
if (existing) {
if (node.temporaryExpiresAt && !existing.temporaryExpiresAt) {
return treeModel.update(tree, node.id, {
temporaryExpiresAt: node.temporaryExpiresAt,
});
}
return tree;
}
return treeModel.insert(tree, parentId, node, index);
};
const fixtureTN: TN[] = [
{ id: "a", name: "A" },
{ id: "b", name: "B" },
];
const deadline = "2026-07-01T00:00:00.000Z";
it("(a) patches temporaryExpiresAt when the existing node has none + the response carries a deadline", () => {
// Server broadcast won the race and inserted the node WITHOUT a deadline.
const afterServer = treeModel.insert(fixtureTN, null, {
id: "new",
name: "",
});
expect((treeModel.find(afterServer, "new") as TN).temporaryExpiresAt).toBe(
undefined,
);
// The authoritative create response carries the deadline.
const created: TN = { id: "new", name: "", temporaryExpiresAt: deadline };
const patched = applyOptimisticInsert(
afterServer,
null,
created,
afterServer.length,
);
// A new reference (the patch wrote) and the node now has the deadline...
expect(patched).not.toBe(afterServer);
expect((treeModel.find(patched, "new") as TN).temporaryExpiresAt).toBe(
deadline,
);
// ...and still exactly one node (no duplicate re-insert).
expect(patched.filter((n) => n.id === "new")).toHaveLength(1);
});
it("(b) does NOT overwrite an existing deadline; returns prev by reference", () => {
const existingDeadline = deadline;
// The node already exists WITH a deadline (the broadcast carried it).
const afterServer = treeModel.insert(fixtureTN, null, {
id: "new",
name: "",
temporaryExpiresAt: existingDeadline,
});
// The create response carries a DIFFERENT deadline; the guard must ignore it.
const created: TN = {
id: "new",
name: "",
temporaryExpiresAt: "2099-01-01T00:00:00.000Z",
};
const after = applyOptimisticInsert(
afterServer,
null,
created,
afterServer.length,
);
// prev returned by reference (no write) and the original deadline is kept.
expect(after).toBe(afterServer);
expect((treeModel.find(after, "new") as TN).temporaryExpiresAt).toBe(
existingDeadline,
);
});
});
// moveTreeNode socket-handler semantics: the receiver must place the moved node
// by `position` (NOT index 0) and apply the `pageData` the payload carries so a
// moved node's title/icon/chevron stay correct. This mirrors the reducer in

View File

@@ -9,26 +9,45 @@ export function sortPositionKeys(keys: any[]) {
});
}
/**
* Single canonical `IPage -> SpaceTreeNode` field mapper. Every place that
* materialises a tree node from a page (buildTree, the optimistic insert in
* handleCreate, restore, duplicate) routes through here so the field copy —
* crucially `temporaryExpiresAt` — can never silently drift between sites. The
* `overrides` cover the small per-site differences (e.g. `name: ""` for an
* optimistic create, `name: title || "Untitled"` for restore, `canEdit: true`
* for duplicate). The default `temporaryExpiresAt` comes straight off the page,
* so restore (which the server nulls) stays permanent and a temporary create
* keeps its clock marker without a reload.
*/
export function pageToTreeNode(
page: IPage,
overrides?: Partial<SpaceTreeNode>,
): SpaceTreeNode {
return {
id: page.id,
slugId: page.slugId,
name: page.title,
icon: page.icon,
position: page.position,
hasChildren: page.hasChildren,
spaceId: page.spaceId,
parentPageId: page.parentPageId,
canEdit: page.canEdit ?? page.permissions?.canEdit,
isTemplate: page.isTemplate,
temporaryExpiresAt: page.temporaryExpiresAt,
children: [],
...overrides,
};
}
export function buildTree(pages: IPage[]): SpaceTreeNode[] {
const pageMap: Record<string, SpaceTreeNode> = {};
const tree: SpaceTreeNode[] = [];
pages.forEach((page) => {
pageMap[page.id] = {
id: page.id,
slugId: page.slugId,
name: page.title,
icon: page.icon,
position: page.position,
hasChildren: page.hasChildren,
spaceId: page.spaceId,
parentPageId: page.parentPageId,
canEdit: page.canEdit ?? page.permissions?.canEdit,
isTemplate: page.isTemplate,
temporaryExpiresAt: page.temporaryExpiresAt,
children: [],
};
pageMap[page.id] = pageToTreeNode(page);
});
// Defense-in-depth: a duplicate id in `pages` would push two references to the

View File

@@ -0,0 +1,149 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { render, screen, fireEvent, waitFor } from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import type { IShareAlias } from "@/features/share/types/share.types";
// matchMedia / storage are stubbed globally in vitest.setup.ts.
// The mutation + query hooks reach react-query/network; the availability probe
// hits the API. Stub them so the section renders in isolation and we can drive
// the exact branches (taken name -> hint, 409 -> reassign modal).
const setMutateAsync = vi.fn();
let currentAlias: IShareAlias | null = null;
let availabilityResult: {
valid: boolean;
available: boolean;
currentPageId: string | null;
} = { valid: true, available: true, currentPageId: null };
vi.mock("@/features/share/queries/share-query.ts", () => ({
useShareAliasForPageQuery: () => ({ data: currentAlias }),
useSetShareAliasMutation: () => ({
mutateAsync: setMutateAsync,
isPending: false,
}),
useRemoveShareAliasMutation: () => ({
mutateAsync: vi.fn(),
isPending: false,
}),
}));
vi.mock("@/features/share/services/share-service.ts", () => ({
checkShareAliasAvailability: vi.fn(async () => availabilityResult),
}));
import ShareAliasSection from "./share-alias-section";
const aliasRow = (alias: string, pageId: string): IShareAlias => ({
id: `alias-${alias}`,
workspaceId: "ws-1",
alias,
pageId,
creatorId: "user-1",
createdAt: new Date().toISOString(),
updatedAt: new Date().toISOString(),
});
function renderSection(pageId = "page-Y") {
return render(
<MantineProvider>
<ShareAliasSection pageId={pageId} readOnly={false} />
</MantineProvider>,
);
}
describe("ShareAliasSection — taken-name handling is never a dead end", () => {
beforeEach(() => {
setMutateAsync.mockReset();
currentAlias = null;
availabilityResult = { valid: true, available: true, currentPageId: null };
});
it("shows a 'will move it here' HINT (not a terminal error) when the name belongs to another page, and keeps Save enabled", async () => {
// Page Y already owns "bee"; the user retypes a name owned by page X.
currentAlias = aliasRow("bee", "page-Y");
availabilityResult = {
valid: true,
available: false,
currentPageId: "page-X",
};
renderSection("page-Y");
const input = screen.getByPlaceholderText("my-page") as HTMLInputElement;
fireEvent.change(input, { target: { value: "test2" } });
// The reassign hint replaces the old dead-end red error.
await waitFor(
() =>
expect(
screen.getByText(
"This address is in use. Saving will move it to this page.",
),
).toBeDefined(),
{ timeout: 2000 },
);
// The old terminal "already in use" error must NOT be shown.
expect(screen.queryByText("This address is already in use")).toBeNull();
// Save stays enabled so the confirm-reassign flow can run.
const saveBtn = screen.getByRole("button", {
name: "Save",
}) as HTMLButtonElement;
expect(saveBtn.disabled).toBe(false);
});
it("opens the reassign-confirm modal on a 409 ALIAS_REASSIGN_REQUIRED (path forward, not a dead end)", async () => {
currentAlias = aliasRow("bee", "page-Y");
availabilityResult = {
valid: true,
available: false,
currentPageId: "page-X",
};
// The server rejects the un-confirmed save asking the client to confirm.
setMutateAsync.mockRejectedValueOnce({
status: 409,
response: {
status: 409,
data: {
code: "ALIAS_REASSIGN_REQUIRED",
currentPageId: "page-X",
currentPageTitle: "Alias Test Page X",
},
},
});
renderSection("page-Y");
const input = screen.getByPlaceholderText("my-page") as HTMLInputElement;
fireEvent.change(input, { target: { value: "test2" } });
const saveBtn = screen.getByRole("button", {
name: "Save",
}) as HTMLButtonElement;
await waitFor(() => expect(saveBtn.disabled).toBe(false), {
timeout: 2000,
});
fireEvent.click(saveBtn);
// First save sent WITHOUT confirmReassign.
await waitFor(() =>
expect(setMutateAsync).toHaveBeenCalledWith(
expect.objectContaining({ alias: "test2", confirmReassign: false }),
),
);
// The "Move custom address?" confirm modal must appear (the path forward).
await waitFor(() =>
expect(screen.getByText("Move custom address?")).toBeDefined(),
);
expect(screen.getByRole("button", { name: "Move here" })).toBeDefined();
// Confirming retries WITH confirmReassign: true.
setMutateAsync.mockResolvedValueOnce(aliasRow("test2", "page-Y"));
fireEvent.click(screen.getByRole("button", { name: "Move here" }));
await waitFor(() =>
expect(setMutateAsync).toHaveBeenCalledWith(
expect.objectContaining({ alias: "test2", confirmReassign: true }),
),
);
});
});

View File

@@ -120,8 +120,13 @@ export default function ShareAliasSection({
};
const showInvalid = normalized.length > 0 && !isValid;
const showTaken =
isValid && !unchanged && availability && !availability.available;
// The typed name is already in use by ANOTHER page. This is NOT a dead end:
// hitting Save triggers the server's 409 `ALIAS_REASSIGN_REQUIRED` and opens
// the "Move custom address?" confirm modal that retargets the address here.
// So surface it as an informational hint (not a terminal red error) and keep
// Save enabled, instead of looking like the address is unusable.
const reassignable =
isValid && !unchanged && !!availability && !availability.available;
// The slug prefix (e.g. "docs.example.com/l/") is static for the session.
const prefixLabel = aliasPrefixLabel();
@@ -198,9 +203,12 @@ export default function ShareAliasSection({
error={
showInvalid
? t("Use 2-60 lowercase letters, digits and hyphens")
: showTaken
? t("This address is already in use")
: undefined
: undefined
}
description={
reassignable
? t("This address is in use. Saving will move it to this page.")
: undefined
}
/>

View File

@@ -1,5 +1,5 @@
import { useState } from "react";
import { Button, Group } from "@mantine/core";
import { Button, Stack } from "@mantine/core";
import { IconHourglass, IconPlus } from "@tabler/icons-react";
import { useParams } from "react-router-dom";
import { useTranslation } from "react-i18next";
@@ -45,12 +45,16 @@ export default function SpaceCreateNoteButtons() {
.finally(() => setPending(null));
};
// Two full-width, vertically stacked buttons: a neutral regular note and an
// orange-tinted temporary note. Stacking full-width keeps the longer "New
// temporary note" label from clipping on narrow mobile widths.
return (
<Group grow gap="sm">
<Stack gap="sm">
<Button
size="md"
variant="light"
color="gray"
fullWidth
leftSection={<IconPlus size={18} />}
loading={pending === "regular"}
disabled={pending !== null}
@@ -61,7 +65,8 @@ export default function SpaceCreateNoteButtons() {
<Button
size="md"
variant="light"
color="gray"
color="orange"
fullWidth
leftSection={<IconHourglass size={18} />}
loading={pending === "temporary"}
disabled={pending !== null}
@@ -69,6 +74,6 @@ export default function SpaceCreateNoteButtons() {
>
{t("New temporary note")}
</Button>
</Group>
</Stack>
);
}

View File

@@ -323,4 +323,18 @@ describe("applyAddTreeNode", () => {
"child",
]);
});
it("carries temporaryExpiresAt onto the inserted node so the clock marker shows on create (no reload)", () => {
// A note created as temporary broadcasts addTreeNode with the death-timer
// deadline in its payload; the receiver's inserted node must keep it so
// space-tree-row renders the orange clock marker immediately.
const tree = roots();
const expiresAt = "2026-06-27T21:00:00.000Z";
const next = applyAddTreeNode(tree, {
parentId: null as unknown as string,
index: 0,
data: node("temp", { position: "a3", temporaryExpiresAt: expiresAt }),
});
expect(treeModel.find(next, "temp")?.temporaryExpiresAt).toBe(expiresAt);
});
});

View File

@@ -0,0 +1,407 @@
import { useEffect, useMemo, useState } from "react";
import {
Accordion,
Alert,
Badge,
Button,
Center,
Checkbox,
Group,
Loader,
Modal,
Radio,
Select,
Stack,
Text,
} from "@mantine/core";
import { IconAlertTriangle } from "@tabler/icons-react";
import { useTranslation } from "react-i18next";
import {
useAiRoleCatalogBundleQuery,
useAiRoleCatalogQuery,
useImportAiRolesFromCatalogMutation,
useUpdateAiRoleFromCatalogMutation,
} from "@/features/ai-chat/queries/ai-chat-query.ts";
import {
IAiRole,
IAiRoleCatalogBundleSummary,
IAiRoleCatalogRole,
} from "@/features/ai-chat/types/ai-chat.types.ts";
import { catalogRoleInstallState } from "@/features/ai-chat/utils/catalog-role-install-state.ts";
interface AiAgentRolesCatalogModalProps {
opened: boolean;
onClose: () => void;
// The current admin role list (full view, including `source`). Used to compute
// each catalog role's install state (import / installed / update available).
roles: IAiRole[];
}
/** How a name collision with an existing role is handled on import. */
type Conflict = "skip" | "rename";
/**
* Admin modal: browse the curated role catalog, import roles, and update an
* imported role when the catalog ships a newer version.
*
* Import is per-bundle (the endpoint takes a single bundleId). Each bundle's
* Accordion panel has its own "Import" button that imports only that bundle's
* checked roles — the simplest mapping to the one-bundle-per-call API and the
* clearest UX. Selection state is tracked per bundle.
*/
export default function AiAgentRolesCatalogModal({
opened,
onClose,
roles,
}: AiAgentRolesCatalogModalProps) {
const { t, i18n } = useTranslation();
// The user's i18n base subtag (e.g. "ru-RU" => "ru"); the preferred catalog
// language both when seeding and when reconciling against offered languages.
const baseLang = (i18n.language || "en").split("-")[0].toLowerCase();
// Fetch the catalog only while the modal is open. `language` drives both the
// catalog query (bundle names) and bundle reads (role content). Seed it
// synchronously from the base subtag so the first fetch already uses the
// user's language; the effect below still reconciles against the catalog's
// offered languages once they load.
const [language, setLanguage] = useState<string>(() => baseLang);
const catalogQuery = useAiRoleCatalogQuery(language || "en", opened);
// On name conflict: Skip (default) or Rename to a free " (N)" name.
const [conflict, setConflict] = useState<Conflict>("skip");
// The currently expanded bundle id (Accordion is single-open: one bundle's
// roles are fetched at a time).
const [expanded, setExpanded] = useState<string | null>(null);
// Per-bundle selected slugs (import-state roles checked for import).
const [selected, setSelected] = useState<Record<string, Set<string>>>({});
const languages = catalogQuery.data?.languages;
// Pick a sensible default language from the catalog once it loads: the i18n
// base subtag (e.g. "ru-RU" => "ru") if offered, else "en", else the first.
useEffect(() => {
if (!languages || languages.length === 0) return;
if (language && languages.includes(language)) return;
const preferred = languages.includes(baseLang)
? baseLang
: languages.includes("en")
? "en"
: languages[0];
setLanguage(preferred);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [languages]);
// Reset per-language UI state when the language changes (the bundle content,
// hence the install computations, are language-specific).
useEffect(() => {
setExpanded(null);
setSelected({});
}, [language]);
return (
<Modal
opened={opened}
onClose={onClose}
title={t("Role catalog")}
size="lg"
>
<Stack>
<Select
label={t("Language")}
data={languages ?? []}
value={language || null}
onChange={(value) => value && setLanguage(value)}
allowDeselect={false}
disabled={!languages || languages.length === 0}
comboboxProps={{ withinPortal: true }}
/>
<Radio.Group
label={t("On name conflict")}
value={conflict}
onChange={(value) => setConflict(value as Conflict)}
>
<Group mt="xs">
<Radio value="skip" label={t("Skip")} />
<Radio value="rename" label={t("Rename")} />
</Group>
</Radio.Group>
{catalogQuery.isLoading && (
<Center py="lg">
<Loader size="sm" />
</Center>
)}
{catalogQuery.isError && (
<Alert
color="red"
icon={<IconAlertTriangle size={16} />}
title={t("The role catalog is unavailable")}
>
{t("Please try again later.")}
</Alert>
)}
{catalogQuery.data && catalogQuery.data.bundles.length === 0 && (
<Text size="sm" c="dimmed">
{t("No bundles available")}
</Text>
)}
{catalogQuery.data && catalogQuery.data.bundles.length > 0 && (
<Accordion
variant="separated"
value={expanded}
onChange={setExpanded}
>
{catalogQuery.data.bundles.map((bundle) => (
<BundlePanel
key={bundle.id}
bundle={bundle}
language={language}
expanded={expanded === bundle.id}
roles={roles}
conflict={conflict}
selected={selected[bundle.id]}
onToggleSlug={(slug, checked) =>
setSelected((prev) => {
const next = new Set(prev[bundle.id] ?? []);
if (checked) next.add(slug);
else next.delete(slug);
return { ...prev, [bundle.id]: next };
})
}
onSetSelected={(slugs) =>
setSelected((prev) => ({
...prev,
[bundle.id]: new Set(slugs),
}))
}
/>
))}
</Accordion>
)}
<Group justify="flex-end" mt="sm">
<Button variant="default" onClick={onClose}>
{t("Close")}
</Button>
</Group>
</Stack>
</Modal>
);
}
interface BundlePanelProps {
bundle: IAiRoleCatalogBundleSummary;
language: string;
expanded: boolean;
roles: IAiRole[];
conflict: Conflict;
selected: Set<string> | undefined;
onToggleSlug: (slug: string, checked: boolean) => void;
onSetSelected: (slugs: string[]) => void;
}
/** One catalog bundle: its roles (fetched when expanded) + a per-bundle import. */
function BundlePanel({
bundle,
language,
expanded,
roles,
conflict,
selected,
onToggleSlug,
onSetSelected,
}: BundlePanelProps) {
const { t } = useTranslation();
// Only fetch this bundle's roles once it is actually expanded.
const bundleQuery = useAiRoleCatalogBundleQuery(
bundle.id,
language,
expanded && !!language,
);
const importMutation = useImportAiRolesFromCatalogMutation();
const updateMutation = useUpdateAiRoleFromCatalogMutation();
// Compute each catalog role's install state against the current workspace
// roles (matched by source.slug + source.language). The decision lives in the
// pure `catalogRoleInstallState` helper so it is unit-tested directly.
const computed = useMemo(() => {
const list = bundleQuery.data?.roles ?? [];
return list.map((role) => ({
role,
...catalogRoleInstallState(role, roles, language),
}));
}, [bundleQuery.data, roles, language]);
// Default-check every importable role once the bundle content arrives (unless
// the user already touched the selection for this bundle).
useEffect(() => {
if (!bundleQuery.data || selected !== undefined) return;
onSetSelected(
computed.filter((c) => c.state === "import").map((c) => c.role.slug),
);
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [bundleQuery.data]);
const importableSlugs = computed
.filter((c) => c.state === "import")
.map((c) => c.role.slug);
const checkedSlugs = importableSlugs.filter((slug) => selected?.has(slug));
function handleImport() {
importMutation.mutate({
bundleId: bundle.id,
language,
slugs: checkedSlugs,
conflict,
});
}
return (
<Accordion.Item value={bundle.id}>
<Accordion.Control>
<Stack gap={2}>
<Text fw={500}>{bundle.name}</Text>
{bundle.description && (
<Text size="xs" c="dimmed">
{bundle.description}
</Text>
)}
</Stack>
</Accordion.Control>
<Accordion.Panel>
{bundleQuery.isLoading && (
<Center py="md">
<Loader size="sm" />
</Center>
)}
{bundleQuery.isError && (
<Alert
color="red"
icon={<IconAlertTriangle size={16} />}
title={t("The role catalog is unavailable")}
>
{t("Please try again later.")}
</Alert>
)}
{bundleQuery.data && (
<Stack gap="xs">
{computed.map((entry) => (
<CatalogRoleRow
key={entry.role.slug}
role={entry.role}
state={entry.state}
checked={
entry.state === "import"
? !!selected?.has(entry.role.slug)
: false
}
onToggle={(checked) => onToggleSlug(entry.role.slug, checked)}
fromVersion={
entry.state === "update" ? entry.fromVersion : undefined
}
onUpdate={
entry.state === "update"
? () => updateMutation.mutate(entry.installed.id)
: undefined
}
updating={updateMutation.isPending}
/>
))}
<Group justify="flex-end" mt="xs">
<Button
size="xs"
onClick={handleImport}
loading={importMutation.isPending}
disabled={checkedSlugs.length === 0}
>
{t("Import")}
</Button>
</Group>
</Stack>
)}
</Accordion.Panel>
</Accordion.Item>
);
}
interface CatalogRoleRowProps {
role: IAiRoleCatalogRole;
state: "import" | "installed" | "update";
checked: boolean;
onToggle: (checked: boolean) => void;
// The installed role's current source version (only set in the "update" state).
fromVersion?: number;
onUpdate?: () => void;
updating: boolean;
}
/** A single catalog role row with its install-state affordance. */
function CatalogRoleRow({
role,
state,
checked,
onToggle,
fromVersion,
onUpdate,
updating,
}: CatalogRoleRowProps) {
const { t } = useTranslation();
return (
<Group justify="space-between" wrap="nowrap" align="flex-start">
<Group gap="xs" wrap="nowrap" align="flex-start" style={{ minWidth: 0 }}>
{state === "import" && (
<Checkbox
checked={checked}
onChange={(event) => onToggle(event.currentTarget.checked)}
aria-label={role.name}
/>
)}
<Stack gap={2} style={{ minWidth: 0 }}>
<Text fw={500} truncate>
{role.emoji ? `${role.emoji} ` : ""}
{role.name}
</Text>
{role.description && (
<Text size="xs" c="dimmed">
{role.description}
</Text>
)}
</Stack>
</Group>
<Group gap="xs" wrap="nowrap" style={{ flex: "none" }}>
{state === "installed" && (
<Badge size="sm" variant="light" color="gray">
{t("Installed")}
</Badge>
)}
{state === "update" && (
<>
<Badge size="sm" variant="light" color="blue">
{t("v{{from}} → v{{to}}", {
from: fromVersion ?? 0,
to: role.version,
})}
</Badge>
<Button size="xs" variant="light" onClick={onUpdate} loading={updating}>
{t("Update")}
</Button>
</>
)}
</Group>
</Group>
);
}

View File

@@ -13,7 +13,12 @@ import {
} from "@mantine/core";
import { useDisclosure } from "@mantine/hooks";
import { modals } from "@mantine/modals";
import { IconPencil, IconPlus, IconTrash } from "@tabler/icons-react";
import {
IconPackageImport,
IconPencil,
IconPlus,
IconTrash,
} from "@tabler/icons-react";
import { useTranslation } from "react-i18next";
import useUserRole from "@/hooks/use-user-role.tsx";
import {
@@ -23,6 +28,7 @@ import {
} from "@/features/ai-chat/queries/ai-chat-query.ts";
import { IAiRole } from "@/features/ai-chat/types/ai-chat.types.ts";
import AiAgentRoleForm from "./ai-agent-role-form.tsx";
import AiAgentRolesCatalogModal from "./ai-agent-roles-catalog-modal.tsx";
/**
* Admin section: list / add / edit / delete reusable agent roles. A role
@@ -39,6 +45,9 @@ export default function AiAgentRoles() {
const deleteMutation = useDeleteAiRoleMutation();
const [opened, { open, close }] = useDisclosure(false);
// Separate disclosure for the catalog (import/update) modal.
const [catalogOpened, { open: openCatalog, close: closeCatalog }] =
useDisclosure(false);
// The role being edited; undefined => the modal is in "create" mode.
const [editing, setEditing] = useState<IAiRole | undefined>(undefined);
@@ -86,14 +95,24 @@ export default function AiAgentRoles() {
/>
<Text fw={600}>{t("Agent roles")}</Text>
</Group>
<Button
leftSection={<IconPlus size={16} />}
variant="default"
size="xs"
onClick={openCreate}
>
{t("Add role")}
</Button>
<Group gap="xs" wrap="nowrap">
<Button
leftSection={<IconPackageImport size={16} />}
variant="default"
size="xs"
onClick={openCatalog}
>
{t("Import from catalog")}
</Button>
<Button
leftSection={<IconPlus size={16} />}
variant="default"
size="xs"
onClick={openCreate}
>
{t("Add role")}
</Button>
</Group>
</Group>
<Text size="xs" c="dimmed" mt={4}>
{t(
@@ -102,9 +121,19 @@ export default function AiAgentRoles() {
</Text>
{!isLoading && (!roles || roles.length === 0) && (
<Text size="sm" c="dimmed" mt="sm">
{t("No roles configured")}
</Text>
<Group gap="sm" mt="sm" align="center">
<Text size="sm" c="dimmed">
{t("No roles configured")}
</Text>
<Button
leftSection={<IconPackageImport size={16} />}
variant="light"
size="xs"
onClick={openCatalog}
>
{t("Browse the catalog")}
</Button>
</Group>
)}
<Stack gap="xs" mt="sm">
@@ -170,6 +199,12 @@ export default function AiAgentRoles() {
{/* Remount the form per target so its internal state re-hydrates. */}
<AiAgentRoleForm key={editing?.id ?? "new"} role={editing} onClose={close} />
</Modal>
<AiAgentRolesCatalogModal
opened={catalogOpened}
onClose={closeCatalog}
roles={roles ?? []}
/>
</Paper>
);
}

View File

@@ -0,0 +1,44 @@
import { AiChatController } from './ai-chat.controller';
import type { User, Workspace } from '@docmost/db/types/entity.types';
/**
* Wiring spec for the #191 `POST /ai-chat/bound-chat` endpoint. It must forward
* the requesting user + workspace + pageId to findLatestByPage and return the
* matched chat's id, or `{ chatId: null }` when there is none. The repo already
* scopes to the caller's OWN chats, so a foreign pageId simply yields no match
* (null) — no extra page-access check is needed. Exercised with hand-rolled
* mocks, no Nest graph and no DB.
*/
describe('AiChatController.boundChat', () => {
const user = { id: 'u1' } as User;
const workspace = { id: 'ws1' } as Workspace;
function makeController(chat: unknown) {
const aiChatRepo = {
findLatestByPage: jest.fn().mockResolvedValue(chat),
};
const controller = new AiChatController(
{} as never,
aiChatRepo as never,
{} as never,
{} as never,
);
return { controller, aiChatRepo };
}
it('returns the owned chat id and scopes the lookup to user + workspace + page', async () => {
const { controller, aiChatRepo } = makeController({
id: 'c1',
creatorId: 'u1',
});
const res = await controller.boundChat({ pageId: 'p1' }, user, workspace);
expect(aiChatRepo.findLatestByPage).toHaveBeenCalledWith('u1', 'ws1', 'p1');
expect(res).toEqual({ chatId: 'c1' });
});
it('returns { chatId: null } for a page with no owned chat (incl. foreign pageId)', async () => {
const { controller } = makeController(undefined);
const res = await controller.boundChat({ pageId: 'foreign' }, user, workspace);
expect(res).toEqual({ chatId: null });
});
});

View File

@@ -30,6 +30,7 @@ import { FileInterceptor } from '../../common/interceptors/file.interceptor';
import { AiChatService, AiChatStreamBody } from './ai-chat.service';
import { AiTranscriptionService } from './ai-transcription.service';
import {
BoundChatDto,
ChatIdDto,
ExportChatDto,
GeneratePageTitleDto,
@@ -67,6 +68,28 @@ export class AiChatController {
return this.aiChatRepo.findByCreator(user.id, workspace.id, pagination);
}
/**
* Resolve the chat bound to a document for the requesting user: the most-recent
* non-deleted chat created on that page (ai_chats.page_id). Returns
* { chatId: null } when the page has no owned chat (-> a fresh chat). No page
* access check needed: only the caller's OWN chats are matched, so a foreign
* pageId reveals nothing.
*/
@HttpCode(HttpStatus.OK)
@Post('bound-chat')
async boundChat(
@Body() dto: BoundChatDto,
@AuthUser() user: User,
@AuthWorkspace() workspace: Workspace,
): Promise<{ chatId: string | null }> {
const chat = await this.aiChatRepo.findLatestByPage(
user.id,
workspace.id,
dto.pageId,
);
return { chatId: chat?.id ?? null };
}
/** Fetch the messages of a chat (oldest first, paginated). */
@HttpCode(HttpStatus.OK)
@Post('messages')

View File

@@ -37,6 +37,12 @@ export class GetChatMessagesDto {
cursor?: string;
}
/** Resolve the chat bound to a document (the page's most-recent owned chat). */
export class BoundChatDto {
@IsString()
pageId: string;
}
/** Export a chat to Markdown (#183). `lang` localizes the few fixed
* role/tool-action labels; defaults to English server-side. */
export class ExportChatDto {

View File

@@ -39,6 +39,10 @@ describe('AiAgentRolesController admin gate', () => {
create: jest.fn().mockResolvedValue({ id: 'r1' }),
update: jest.fn().mockResolvedValue({ id: 'r1' }),
remove: jest.fn().mockResolvedValue({ success: true }),
getCatalog: jest.fn().mockResolvedValue({ languages: [], bundles: [] }),
getCatalogBundle: jest.fn().mockResolvedValue({ roles: [] }),
importFromCatalog: jest.fn().mockResolvedValue({ created: 0 }),
updateFromCatalog: jest.fn().mockResolvedValue({ updated: false }),
};
const controller = new AiAgentRolesController(
rolesService as never,
@@ -109,6 +113,90 @@ describe('AiAgentRolesController admin gate', () => {
});
});
// Catalog routes (browse + import) are ALL admin-only: a non-admin caller must
// get ForbiddenException with the service untouched; an admin delegates with
// the right arguments (import/update-from-catalog carry workspace.id).
describe('catalog routes admin gate', () => {
const catalogDto = { language: 'en' } as never;
const bundleDto = { bundleId: 'general', language: 'en' } as never;
const importDto = {
bundleId: 'general',
language: 'en',
conflict: 'skip',
} as never;
const updateDto = { id: 'r1' } as never;
describe('non-admin is rejected and the service is NOT called', () => {
it('catalog', async () => {
const { controller, rolesService } = makeController(false);
await expect(
controller.catalog(catalogDto, user, workspace),
).rejects.toBeInstanceOf(ForbiddenException);
expect(rolesService.getCatalog).not.toHaveBeenCalled();
});
it('catalog/bundle', async () => {
const { controller, rolesService } = makeController(false);
await expect(
controller.catalogBundle(bundleDto, user, workspace),
).rejects.toBeInstanceOf(ForbiddenException);
expect(rolesService.getCatalogBundle).not.toHaveBeenCalled();
});
it('import', async () => {
const { controller, rolesService } = makeController(false);
await expect(
controller.import(importDto, user, workspace),
).rejects.toBeInstanceOf(ForbiddenException);
expect(rolesService.importFromCatalog).not.toHaveBeenCalled();
});
it('update-from-catalog', async () => {
const { controller, rolesService } = makeController(false);
await expect(
controller.updateFromCatalog(updateDto, user, workspace),
).rejects.toBeInstanceOf(ForbiddenException);
expect(rolesService.updateFromCatalog).not.toHaveBeenCalled();
});
});
describe('admin delegates to the service', () => {
it('catalog passes the requested language', async () => {
const { controller, rolesService } = makeController(true);
await controller.catalog(catalogDto, user, workspace);
expect(rolesService.getCatalog).toHaveBeenCalledWith('en');
});
it('catalog/bundle passes bundleId + language', async () => {
const { controller, rolesService } = makeController(true);
await controller.catalogBundle(bundleDto, user, workspace);
expect(rolesService.getCatalogBundle).toHaveBeenCalledWith(
'general',
'en',
);
});
it('import passes workspace.id + user.id + dto', async () => {
const { controller, rolesService } = makeController(true);
await controller.import(importDto, user, workspace);
expect(rolesService.importFromCatalog).toHaveBeenCalledWith(
'ws-1',
'u1',
importDto,
);
});
it('update-from-catalog passes workspace.id + dto', async () => {
const { controller, rolesService } = makeController(true);
await controller.updateFromCatalog(updateDto, user, workspace);
expect(rolesService.updateFromCatalog).toHaveBeenCalledWith(
'ws-1',
updateDto,
);
});
});
});
describe('list (member-reachable)', () => {
it('non-admin reaches list and the service is asked for the picker view (isAdmin=false)', async () => {
const { controller, rolesService } = makeController(false);

View File

@@ -22,6 +22,12 @@ import {
CreateAgentRoleDto,
UpdateAgentRoleDto,
} from './dto/agent-role.dto';
import {
CatalogBundleDto,
CatalogQueryDto,
ImportFromCatalogDto,
UpdateFromCatalogDto,
} from './dto/agent-role-catalog.dto';
/** Path/body param for the per-role routes (update/delete). */
class AgentRoleIdDto {
@@ -113,4 +119,54 @@ export class AiAgentRolesController {
this.assertAdmin(user, workspace);
return this.rolesService.remove(workspace.id, idDto.id);
}
// --- Catalog (admin-only): browse + import + update imported roles. ---
/** Browse the curated catalog (localized to dto.language). */
@HttpCode(HttpStatus.OK)
@Post('catalog')
async catalog(
@Body() dto: CatalogQueryDto,
@AuthUser() user: User,
@AuthWorkspace() workspace: Workspace,
) {
this.assertAdmin(user, workspace);
return this.rolesService.getCatalog(dto.language);
}
/** Open one catalog bundle in a language (role content + versions). */
@HttpCode(HttpStatus.OK)
@Post('catalog/bundle')
async catalogBundle(
@Body() dto: CatalogBundleDto,
@AuthUser() user: User,
@AuthWorkspace() workspace: Workspace,
) {
this.assertAdmin(user, workspace);
return this.rolesService.getCatalogBundle(dto.bundleId, dto.language);
}
/** Import roles from a catalog bundle into the workspace. */
@HttpCode(HttpStatus.OK)
@Post('import')
async import(
@Body() dto: ImportFromCatalogDto,
@AuthUser() user: User,
@AuthWorkspace() workspace: Workspace,
) {
this.assertAdmin(user, workspace);
return this.rolesService.importFromCatalog(workspace.id, user.id, dto);
}
/** Update an already-imported role from its catalog source. */
@HttpCode(HttpStatus.OK)
@Post('update-from-catalog')
async updateFromCatalog(
@Body() dto: UpdateFromCatalogDto,
@AuthUser() user: User,
@AuthWorkspace() workspace: Workspace,
) {
this.assertAdmin(user, workspace);
return this.rolesService.updateFromCatalog(workspace.id, dto);
}
}

View File

@@ -1,16 +1,19 @@
import { Module } from '@nestjs/common';
import { AiAgentRolesController } from './ai-agent-roles.controller';
import { AiAgentRolesService } from './ai-agent-roles.service';
import { AiAgentRolesCatalogProvider } from './catalog/ai-agent-roles-catalog.provider';
/**
* Agent roles unit (v1). Admin CRUD + member-visible listing for the chat
* role picker. AiAgentRoleRepo (DatabaseModule, global) and
* WorkspaceAbilityFactory (CaslModule, global) are resolved without explicit
* imports. The stream-time role resolution + model override live in
* AiChatService / AiService; this module only hosts the management API.
* role picker, plus the admin catalog (browse/import/update). AiAgentRoleRepo
* (DatabaseModule, global), WorkspaceAbilityFactory (CaslModule, global) and
* EnvironmentService (EnvironmentModule, global — used by the catalog provider)
* are resolved without explicit imports. The stream-time role resolution +
* model override live in AiChatService / AiService; this module only hosts the
* management API.
*/
@Module({
controllers: [AiAgentRolesController],
providers: [AiAgentRolesService],
providers: [AiAgentRolesService, AiAgentRolesCatalogProvider],
})
export class AiAgentRolesModule {}

View File

@@ -1,4 +1,9 @@
import { BadRequestException, ConflictException } from '@nestjs/common';
import {
BadGatewayException,
BadRequestException,
ConflictException,
Logger,
} from '@nestjs/common';
import { AiAgentRolesService } from './ai-agent-roles.service';
import type { AiAgentRole } from '@docmost/db/types/entity.types';
import type {
@@ -27,12 +32,22 @@ describe('AiAgentRolesService guards', () => {
enabled: true,
autoStart: true,
launchMessage: null,
source: null,
createdAt: new Date(),
updatedAt: new Date(),
...over,
} as AiAgentRole;
}
// A stubbed catalog provider; the CRUD tests never reach it (they exercise
// create/update/remove/list only), so the methods just reject if hit.
function makeCatalog() {
return {
fetchIndex: jest.fn(),
fetchBundle: jest.fn(),
};
}
function makeService(opts: { existing?: AiAgentRole | undefined } = {}) {
const repo = {
findById: jest.fn().mockResolvedValue(opts.existing),
@@ -41,8 +56,9 @@ describe('AiAgentRolesService guards', () => {
softDelete: jest.fn().mockResolvedValue(undefined),
listByWorkspace: jest.fn().mockResolvedValue([]),
};
const service = new AiAgentRolesService(repo as never);
return { service, repo };
const catalog = makeCatalog();
const service = new AiAgentRolesService(repo as never, catalog as never);
return { service, repo, catalog };
}
describe('update', () => {
@@ -163,6 +179,7 @@ describe('AiAgentRolesService guards', () => {
enabled: false,
autoStart: true,
launchMessage: null,
source: null,
createdAt,
updatedAt,
});
@@ -397,7 +414,7 @@ describe('AiAgentRolesService guards', () => {
softDelete: jest.fn(),
listByWorkspace: jest.fn().mockResolvedValue(rows),
};
const service = new AiAgentRolesService(repo as never);
const service = new AiAgentRolesService(repo as never, makeCatalog() as never);
return { service, repo };
}
@@ -461,4 +478,630 @@ describe('AiAgentRolesService guards', () => {
).rejects.toBeInstanceOf(ConflictException);
});
});
// ---------------------------------------------------------------------------
// Catalog: import (skip / rename / already-installed) and update reconciliation
// against a MOCKED catalog provider + mocked repo (mirrors the CRUD style).
// ---------------------------------------------------------------------------
describe('importFromCatalog', () => {
function catalogRole(over: Record<string, unknown> = {}) {
return {
slug: 'researcher',
name: 'Researcher',
instructions: 'be a researcher',
...over,
};
}
function makeImportService(opts: {
indexRoles?: { slug: string; version: number }[];
bundleRoles?: Record<string, unknown>[];
existing?: AiAgentRole[];
}) {
const index = {
schemaVersion: 1,
bundles: [
{
id: 'general',
name: { en: 'General' },
languages: ['en'],
roles: opts.indexRoles ?? [{ slug: 'researcher', version: 3 }],
},
],
};
const bundle = {
schemaVersion: 1,
language: 'en',
roles: opts.bundleRoles ?? [catalogRole()],
};
const repo = {
findById: jest.fn(),
insert: jest.fn().mockImplementation((v) => Promise.resolve(makeRow(v))),
update: jest.fn().mockResolvedValue(undefined),
softDelete: jest.fn(),
listByWorkspace: jest.fn().mockResolvedValue(opts.existing ?? []),
};
const catalog = {
fetchIndex: jest.fn().mockResolvedValue(index),
fetchBundle: jest.fn().mockResolvedValue(bundle),
};
const service = new AiAgentRolesService(repo as never, catalog as never);
return { service, repo, catalog };
}
const dto = (over: Record<string, unknown> = {}) =>
({
bundleId: 'general',
language: 'en',
conflict: 'skip',
...over,
}) as never;
it('inserts a new role with source { slug, language, version } from the index', async () => {
const { service, repo } = makeImportService({});
const res = await service.importFromCatalog('ws-1', 'u1', dto());
expect(res).toMatchObject({ created: 1, skipped: 0, renamed: 0 });
expect(res.errors).toEqual([]);
const values = repo.insert.mock.calls[0][0];
expect(values.source).toEqual({
slug: 'researcher',
language: 'en',
version: 3,
});
expect(values.enabled).toBe(true);
});
it('already-installed catalog slug => skipped (no insert)', async () => {
const existing = [
makeRow({
id: 'r-existing',
name: 'Old researcher',
source: { slug: 'researcher', language: 'en', version: 1 } as never,
}),
];
const { service, repo } = makeImportService({ existing });
const res = await service.importFromCatalog('ws-1', 'u1', dto());
expect(res).toMatchObject({ created: 0, skipped: 1, renamed: 0 });
expect(repo.insert).not.toHaveBeenCalled();
});
it('same slug installed in a DIFFERENT language => NOT skipped (separate install)', async () => {
// Installed as `ru`; importing the `en` variant of the same slug must
// still import (dedup key is slug+language, matching the client UI).
const existing = [
makeRow({
id: 'r-ru',
name: 'Исследователь',
source: { slug: 'researcher', language: 'ru', version: 1 } as never,
}),
];
const { service, repo } = makeImportService({ existing });
const res = await service.importFromCatalog('ws-1', 'u1', dto());
expect(res).toMatchObject({ created: 1, skipped: 0, renamed: 0 });
expect(repo.insert).toHaveBeenCalledTimes(1);
expect(repo.insert.mock.calls[0][0].source).toEqual({
slug: 'researcher',
language: 'en',
version: 3,
});
});
it('name collision + conflict:skip => skipped (no insert)', async () => {
const existing = [makeRow({ id: 'r-x', name: 'Researcher' })];
const { service, repo } = makeImportService({ existing });
const res = await service.importFromCatalog(
'ws-1',
'u1',
dto({ conflict: 'skip' }),
);
expect(res).toMatchObject({ created: 0, skipped: 1, renamed: 0 });
expect(repo.insert).not.toHaveBeenCalled();
});
it('name collision + conflict:rename => inserts under " (2)"', async () => {
const existing = [makeRow({ id: 'r-x', name: 'Researcher' })];
const { service, repo } = makeImportService({ existing });
const res = await service.importFromCatalog(
'ws-1',
'u1',
dto({ conflict: 'rename' }),
);
expect(res).toMatchObject({ created: 1, skipped: 0, renamed: 1 });
expect(repo.insert.mock.calls[0][0].name).toBe('Researcher (2)');
});
it('dto.slugs filters; an unknown slug becomes an error entry', async () => {
const { service, repo } = makeImportService({
bundleRoles: [catalogRole()],
});
const res = await service.importFromCatalog(
'ws-1',
'u1',
dto({ slugs: ['researcher', 'ghost'] }),
);
expect(res.created).toBe(1);
expect(res.errors).toEqual([
{ slug: 'ghost', message: 'Role not found in catalog bundle' },
]);
expect(repo.insert).toHaveBeenCalledTimes(1);
});
it('insert unique-violation (23505) is recorded as an error, import continues', async () => {
const { service, repo } = makeImportService({
bundleRoles: [
catalogRole({ slug: 'a', name: 'A' }),
catalogRole({ slug: 'b', name: 'B' }),
],
indexRoles: [
{ slug: 'a', version: 1 },
{ slug: 'b', version: 1 },
],
});
repo.insert
.mockRejectedValueOnce({ code: '23505' })
.mockImplementationOnce((v) => Promise.resolve(makeRow(v)));
const res = await service.importFromCatalog('ws-1', 'u1', dto());
expect(res.created).toBe(1);
expect(res.errors).toEqual([
{ slug: 'a', message: 'A role with this name already exists' },
]);
});
it('source-uniqueness 23505 (concurrent import of same slug+language) => skipped, NOT an error, batch continues', async () => {
// Two parallel imports of the same bundle each build installedKeys from a
// stale snapshot, so both reach the insert for slug 'a'. The DB partial
// unique index on (workspace, source->>slug, source->>language) rejects the
// loser with a 23505 carrying the source-index constraint name. That must
// be treated as "already installed" (skip), not a per-role error, and the
// rest of the batch (slug 'b') must still import.
const { service, repo } = makeImportService({
bundleRoles: [
catalogRole({ slug: 'a', name: 'A' }),
catalogRole({ slug: 'b', name: 'B' }),
],
indexRoles: [
{ slug: 'a', version: 1 },
{ slug: 'b', version: 1 },
],
});
// The kysely-postgres-js driver surfaces the violated constraint on
// `constraint_name` (not node-postgres' `.constraint`), matching prod.
const sourceRace = Object.assign(new Error('duplicate key'), {
code: '23505',
constraint_name: 'ai_agent_roles_workspace_source_unique',
});
repo.insert
.mockRejectedValueOnce(sourceRace)
.mockImplementationOnce((v) => Promise.resolve(makeRow(v)));
const res = await service.importFromCatalog('ws-1', 'u1', dto());
// 'a' converged on the concurrent install (skip); 'b' imported; no errors.
expect(res).toMatchObject({ created: 1, skipped: 1, renamed: 0 });
expect(res.errors).toEqual([]);
// Both inserts were attempted (the batch did not abort on the 23505).
expect(repo.insert).toHaveBeenCalledTimes(2);
});
it('non-unique insert error => generic message, root cause logged, import continues', async () => {
const logSpy = jest
.spyOn(Logger.prototype, 'error')
.mockImplementation(() => undefined);
try {
const { service, repo } = makeImportService({
bundleRoles: [
catalogRole({ slug: 'a', name: 'A' }),
catalogRole({ slug: 'b', name: 'B' }),
],
indexRoles: [
{ slug: 'a', version: 1 },
{ slug: 'b', version: 1 },
],
});
// A non-23505 failure (e.g. a not-null violation) on the first insert.
const boom = Object.assign(new Error('null value in column'), {
code: '23502',
});
repo.insert
.mockRejectedValueOnce(boom)
.mockImplementationOnce((v) => Promise.resolve(makeRow(v)));
const res = await service.importFromCatalog('ws-1', 'u1', dto());
// The generic (non-409) user-facing message; the second role still imports.
expect(res.created).toBe(1);
expect(res.errors).toEqual([
{ slug: 'a', message: 'Failed to import role' },
]);
// The root cause was logged with the slug for diagnosis.
expect(logSpy).toHaveBeenCalledTimes(1);
expect(String(logSpy.mock.calls[0][0])).toContain('slug=a');
} finally {
logSpy.mockRestore();
}
});
it('bundleId absent from the index => BadGateway (no insert)', async () => {
// The requested bundle is not listed in the fetched index (a stale client
// or an index/bundle drift); the import must surface a 502 rather than
// silently doing nothing or dereferencing a missing meta.
const { service, repo } = makeImportService({});
await expect(
service.importFromCatalog('ws-1', 'u1', dto({ bundleId: 'missing' })),
).rejects.toBeInstanceOf(BadGatewayException);
expect(repo.insert).not.toHaveBeenCalled();
});
});
describe('updateFromCatalog', () => {
function makeUpdateService(opts: {
role?: AiAgentRole;
indexBundles?: unknown[];
bundleRoles?: Record<string, unknown>[];
others?: AiAgentRole[];
}) {
const index = {
schemaVersion: 1,
bundles: opts.indexBundles ?? [
{
id: 'general',
name: { en: 'General' },
languages: ['en'],
roles: [{ slug: 'researcher', version: 5 }],
},
],
};
const bundle = {
schemaVersion: 1,
language: 'en',
roles: opts.bundleRoles ?? [
{ slug: 'researcher', name: 'Researcher v5', instructions: 'new' },
],
};
const repo = {
findById: jest.fn().mockResolvedValue(opts.role),
insert: jest.fn(),
update: jest.fn().mockResolvedValue(undefined),
softDelete: jest.fn(),
listByWorkspace: jest.fn().mockResolvedValue(opts.others ?? []),
};
const catalog = {
fetchIndex: jest.fn().mockResolvedValue(index),
fetchBundle: jest.fn().mockResolvedValue(bundle),
};
const service = new AiAgentRolesService(repo as never, catalog as never);
return { service, repo, catalog };
}
const imported = (version: number, over: Partial<AiAgentRole> = {}) =>
makeRow({
id: 'r1',
name: 'Researcher',
source: { slug: 'researcher', language: 'en', version } as never,
...over,
});
it('role not imported from catalog (source null) => BadRequest', async () => {
const { service } = makeUpdateService({ role: makeRow({ source: null }) });
await expect(
service.updateFromCatalog('ws-1', { id: 'r1' } as never),
).rejects.toBeInstanceOf(BadRequestException);
});
it('role not found => BadRequest', async () => {
const { service } = makeUpdateService({ role: undefined });
await expect(
service.updateFromCatalog('ws-1', { id: 'r1' } as never),
).rejects.toBeInstanceOf(BadRequestException);
});
it('catalog version <= source.version => up-to-date (no update)', async () => {
const { service, repo } = makeUpdateService({ role: imported(5) });
const res = await service.updateFromCatalog('ws-1', { id: 'r1' } as never);
expect(res).toEqual({ updated: false, reason: 'up-to-date' });
expect(repo.update).not.toHaveBeenCalled();
});
it('slug no longer listed in any bundle => not-in-catalog', async () => {
const { service, repo } = makeUpdateService({
role: imported(1),
indexBundles: [
{
id: 'general',
name: { en: 'General' },
languages: ['en'],
roles: [{ slug: 'other', version: 9 }],
},
],
});
const res = await service.updateFromCatalog('ws-1', { id: 'r1' } as never);
expect(res).toEqual({ updated: false, reason: 'not-in-catalog' });
expect(repo.update).not.toHaveBeenCalled();
});
it('source.language no longer offered by the bundle => language-unavailable', async () => {
const { service, repo } = makeUpdateService({
role: imported(1, {
source: { slug: 'researcher', language: 'ru', version: 1 } as never,
}),
indexBundles: [
{
id: 'general',
name: { en: 'General' },
languages: ['en'],
roles: [{ slug: 'researcher', version: 5 }],
},
],
});
const res = await service.updateFromCatalog('ws-1', { id: 'r1' } as never);
expect(res).toEqual({ updated: false, reason: 'language-unavailable' });
expect(repo.update).not.toHaveBeenCalled();
});
it('newer version => updates content + bumps source.version, returns versions', async () => {
const role = imported(1);
const { service, repo } = makeUpdateService({ role });
// The post-update re-fetch returns the bumped row.
repo.findById
.mockResolvedValueOnce(role)
.mockResolvedValueOnce(
imported(5, { name: 'Researcher v5', instructions: 'new' }),
);
const res = await service.updateFromCatalog('ws-1', { id: 'r1' } as never);
expect(res).toMatchObject({
updated: true,
fromVersion: 1,
toVersion: 5,
});
const patch = repo.update.mock.calls[0][2];
expect(patch.source).toEqual({
slug: 'researcher',
language: 'en',
version: 5,
});
expect(patch.name).toBe('Researcher v5');
// enabled is never touched by an update-from-catalog.
expect('enabled' in patch).toBe(false);
});
it('slug listed in the index but missing from the bundle file => not-in-catalog', async () => {
// Index/bundle drift: the index still advertises a newer `researcher`
// (v5 > installed v1) in an offered language, but the fetched bundle file
// no longer contains that slug. The update must no-op as not-in-catalog,
// not throw or write a half-resolved role.
const { service, repo } = makeUpdateService({
role: imported(1),
bundleRoles: [
{ slug: 'someone-else', name: 'Other', instructions: 'x' },
],
});
const res = await service.updateFromCatalog('ws-1', { id: 'r1' } as never);
expect(res).toEqual({ updated: false, reason: 'not-in-catalog' });
expect(repo.update).not.toHaveBeenCalled();
});
it('new catalog name collides with another live role => keeps current name', async () => {
const role = imported(1);
const other = makeRow({ id: 'r2', name: 'Researcher v5' });
const { service, repo } = makeUpdateService({ role, others: [role, other] });
repo.findById
.mockResolvedValueOnce(role)
.mockResolvedValueOnce(imported(5));
await service.updateFromCatalog('ws-1', { id: 'r1' } as never);
// The colliding catalog name is dropped; the current name is kept.
expect(repo.update.mock.calls[0][2].name).toBe('Researcher');
});
});
// ---------------------------------------------------------------------------
// Catalog browse (getCatalog / getCatalogBundle) against a MOCKED provider.
// Covers the localized() three-tier fallback (requested lang -> en -> first ->
// null), the sorted union of bundle languages, the missing-bundle BadGateway,
// and the role-version default.
// ---------------------------------------------------------------------------
describe('getCatalog', () => {
function makeBrowseService(index: unknown) {
const repo = {
findById: jest.fn(),
insert: jest.fn(),
update: jest.fn(),
softDelete: jest.fn(),
listByWorkspace: jest.fn(),
};
const catalog = {
fetchIndex: jest.fn().mockResolvedValue(index),
fetchBundle: jest.fn(),
};
const service = new AiAgentRolesService(repo as never, catalog as never);
return { service, catalog };
}
it('returns the sorted union of every bundle language', async () => {
const { service } = makeBrowseService({
schemaVersion: 1,
bundles: [
{
id: 'a',
name: { en: 'A' },
languages: ['ru', 'en'],
roles: [],
},
{
id: 'b',
name: { en: 'B' },
languages: ['en', 'de'],
roles: [],
},
],
});
const res = await service.getCatalog('en');
expect(res.languages).toEqual(['de', 'en', 'ru']);
});
it('localized name uses the requested language when present', async () => {
const { service } = makeBrowseService({
schemaVersion: 1,
bundles: [
{
id: 'a',
name: { en: 'General', ru: 'Общие' },
description: { en: 'desc-en', ru: 'desc-ru' },
languages: ['en', 'ru'],
roles: [{ slug: 'researcher', version: 2 }],
},
],
});
const res = await service.getCatalog('ru');
expect(res.bundles[0]).toMatchObject({
id: 'a',
name: 'Общие',
description: 'desc-ru',
languages: ['en', 'ru'],
roles: [{ slug: 'researcher', version: 2 }],
});
});
it('localized name falls back to en when the requested language is missing', async () => {
const { service } = makeBrowseService({
schemaVersion: 1,
bundles: [
{
id: 'a',
name: { en: 'General', ru: 'Общие' },
languages: ['en', 'ru'],
roles: [],
},
],
});
const res = await service.getCatalog('fr');
expect(res.bundles[0].name).toBe('General');
});
it('localized name falls back to the first available locale when en is absent', async () => {
const { service } = makeBrowseService({
schemaVersion: 1,
bundles: [
{
id: 'a',
name: { ru: 'Общие', de: 'Allgemein' },
languages: ['ru', 'de'],
roles: [],
},
],
});
const res = await service.getCatalog('fr');
// Neither 'fr' nor 'en' is present -> first available value.
expect(res.bundles[0].name).toBe('Общие');
});
it('empty name map => falls back to the bundle id; absent description => null', async () => {
const { service } = makeBrowseService({
schemaVersion: 1,
bundles: [
{
id: 'a',
name: {},
languages: ['en'],
roles: [],
},
],
});
const res = await service.getCatalog('en');
expect(res.bundles[0].name).toBe('a');
expect(res.bundles[0].description).toBeNull();
});
});
describe('getCatalogBundle', () => {
function makeBundleService(opts: {
index: unknown;
bundle: unknown;
}) {
const repo = {
findById: jest.fn(),
insert: jest.fn(),
update: jest.fn(),
softDelete: jest.fn(),
listByWorkspace: jest.fn(),
};
const catalog = {
fetchIndex: jest.fn().mockResolvedValue(opts.index),
fetchBundle: jest.fn().mockResolvedValue(opts.bundle),
};
const service = new AiAgentRolesService(repo as never, catalog as never);
return { service, catalog };
}
const index = {
schemaVersion: 1,
bundles: [
{
id: 'general',
name: { en: 'General' },
languages: ['en'],
roles: [{ slug: 'researcher', version: 4 }],
},
],
};
it('missing bundle in the index => BadGateway', async () => {
const { service, catalog } = makeBundleService({
index,
bundle: { schemaVersion: 1, language: 'en', roles: [] },
});
await expect(
service.getCatalogBundle('ghost', 'en'),
).rejects.toBeInstanceOf(BadGatewayException);
expect(catalog.fetchBundle).not.toHaveBeenCalled();
});
it('maps role content with the version taken from the index', async () => {
const { service } = makeBundleService({
index,
bundle: {
schemaVersion: 1,
language: 'en',
roles: [
{
slug: 'researcher',
name: 'Researcher',
instructions: 'be a researcher',
emoji: '🔬',
autoStart: false,
launchMessage: 'go',
},
],
},
});
const res = await service.getCatalogBundle('general', 'en');
expect(res).toMatchObject({ bundleId: 'general', language: 'en' });
expect(res.roles[0]).toEqual({
slug: 'researcher',
emoji: '🔬',
name: 'Researcher',
description: null,
instructions: 'be a researcher',
autoStart: false,
launchMessage: 'go',
version: 4,
});
});
it('role absent from the index meta => version defaults to 1; autoStart defaults to true', async () => {
const { service } = makeBundleService({
index,
bundle: {
schemaVersion: 1,
language: 'en',
roles: [
{ slug: 'newcomer', name: 'Newcomer', instructions: 'hi' },
],
},
});
const res = await service.getCatalogBundle('general', 'en');
expect(res.roles[0]).toMatchObject({
slug: 'newcomer',
version: 1,
autoStart: true,
emoji: null,
launchMessage: null,
});
});
});
});

View File

@@ -1,12 +1,24 @@
import {
BadGatewayException,
BadRequestException,
ConflictException,
Injectable,
Logger,
} from '@nestjs/common';
import { AiAgentRoleRepo } from '@docmost/db/repos/ai-agent-roles/ai-agent-roles.repo';
import { AiAgentRole } from '@docmost/db/types/entity.types';
import {
AiAgentRoleRepo,
parseSource,
} from '@docmost/db/repos/ai-agent-roles/ai-agent-roles.repo';
import { AiAgentRole, RoleSource } from '@docmost/db/types/entity.types';
import { CreateAgentRoleDto, UpdateAgentRoleDto } from './dto/agent-role.dto';
import { ImportFromCatalogDto, UpdateFromCatalogDto } from './dto/agent-role-catalog.dto';
import { RoleModelConfig } from './role-model-config';
import { AiAgentRolesCatalogProvider } from './catalog/ai-agent-roles-catalog.provider';
import {
CatalogBundleFile,
CatalogBundleMeta,
CatalogRole,
} from './catalog/catalog-types';
/**
* Full (admin) view of an agent role. There are no secret columns on this table
@@ -24,6 +36,10 @@ export interface AgentRoleView {
enabled: boolean;
autoStart: boolean;
launchMessage: string | null;
// Catalog origin of an imported role, or null for a manually-created one. The
// admin UI uses `version` to offer an UPDATE when the catalog ships a newer
// revision. Admin-only (deliberately absent from AgentRolePickerView).
source: RoleSource | null;
createdAt: Date;
updatedAt: Date;
}
@@ -56,7 +72,12 @@ export interface AgentRolePickerView {
*/
@Injectable()
export class AiAgentRolesService {
constructor(private readonly repo: AiAgentRoleRepo) {}
private readonly logger = new Logger(AiAgentRolesService.name);
constructor(
private readonly repo: AiAgentRoleRepo,
private readonly catalog: AiAgentRolesCatalogProvider,
) {}
/**
* List the workspace's roles. Admins get the full view (the settings page needs
@@ -165,6 +186,316 @@ export class AiAgentRolesService {
return { success: true };
}
// -------------------------------------------------------------------------
// Catalog (admin-only). The catalog is curated, untrusted JSON fetched +
// validated by AiAgentRolesCatalogProvider; this layer resolves localized
// text and reconciles a bundle against the workspace's existing roles.
// -------------------------------------------------------------------------
/**
* Browse the catalog. Returns the union of every bundle's languages (sorted)
* plus per-bundle metadata with `name` / `description` resolved to the
* requested `language` (fallback: 'en', then the first available locale).
*/
async getCatalog(language?: string): Promise<{
languages: string[];
bundles: {
id: string;
name: string;
description: string | null;
languages: string[];
roles: { slug: string; version: number }[];
}[];
}> {
const index = await this.catalog.fetchIndex();
const languages = Array.from(
new Set(index.bundles.flatMap((b) => b.languages)),
).sort();
const bundles = index.bundles.map((b) => ({
id: b.id,
name: localized(b.name, language) ?? b.id,
description: b.description ? localized(b.description, language) : null,
languages: b.languages,
roles: b.roles.map((r) => ({ slug: r.slug, version: r.version })),
}));
return { languages, bundles };
}
/**
* Shared read prefix for the two bundle-by-id catalog paths (getCatalogBundle /
* importFromCatalog): fetch the index, resolve the requested bundle's meta
* (502 if the index does not list it), fetch its per-language file, and build
* the slug->version map from the meta. The callers keep their own response /
* write logic; only this duplicated read is factored out here.
*/
private async loadBundleById(
bundleId: string,
language: string,
): Promise<{
meta: CatalogBundleMeta;
file: CatalogBundleFile;
versions: Map<string, number>;
}> {
const index = await this.catalog.fetchIndex();
const meta = index.bundles.find((b) => b.id === bundleId);
if (!meta) {
throw new BadGatewayException('Catalog bundle not found');
}
const file = await this.catalog.fetchBundle(bundleId, language);
return { meta, file, versions: versionMap(meta) };
}
/**
* Open one bundle in a language: returns each role's content plus the version
* taken from the index (so the client can compare against an imported role's
* source.version). A missing bundle/language => BadGateway (catalog issue).
*/
async getCatalogBundle(
bundleId: string,
language: string,
): Promise<{
bundleId: string;
language: string;
roles: {
slug: string;
emoji: string | null;
name: string;
description: string | null;
instructions: string;
autoStart: boolean;
launchMessage: string | null;
version: number;
}[];
}> {
const { file, versions } = await this.loadBundleById(bundleId, language);
return {
bundleId,
language,
roles: file.roles.map((r) => ({
slug: r.slug,
emoji: r.emoji ?? null,
name: r.name,
description: r.description ?? null,
instructions: r.instructions,
autoStart: r.autoStart ?? true,
launchMessage: r.launchMessage ?? null,
version: versions.get(r.slug) ?? 1,
})),
};
}
/**
* Import a bundle's roles into the workspace. A role is "already installed"
* (and thus skipped — updates are a separate action) only when an existing
* role matches BOTH its `source.slug` AND `source.language`: this is a
* multilingual catalog, so a different language of the same slug (e.g. the
* `ru` variant of a slug already installed as `en`) is a SEPARATE install and
* still imports. A name collision with an existing role is either skipped or
* imported under a free " (N)" name, per `dto.conflict`. Inserts run
* sequentially (the repo exposes no batch insert and the volume is tiny); a
* unique-name race still surfaces as an error entry rather than aborting the
* whole import.
*/
async importFromCatalog(
workspaceId: string,
creatorId: string,
dto: ImportFromCatalogDto,
): Promise<{
created: number;
skipped: number;
renamed: number;
errors: { slug: string; message: string }[];
}> {
const { file, versions } = await this.loadBundleById(
dto.bundleId,
dto.language,
);
const errors: { slug: string; message: string }[] = [];
// Resolve the selected catalog roles (honor dto.slugs; flag unknown ones).
let selected = file.roles;
if (dto.slugs && dto.slugs.length > 0) {
const wanted = new Set(dto.slugs);
const present = new Set(file.roles.map((r) => r.slug));
for (const slug of dto.slugs) {
if (!present.has(slug)) {
errors.push({ slug, message: 'Role not found in catalog bundle' });
}
}
selected = file.roles.filter((r) => wanted.has(r.slug));
}
const existingRoles = await this.repo.listByWorkspace(workspaceId);
// Catalog roles already installed in this workspace, keyed by slug+language
// (skip; never duplicate). The key MUST match the client install-state and
// updateFromCatalog (both match by source.slug AND source.language): the
// `ru` variant of a slug already installed as `en` is a separate install.
const installedKeys = new Set(
existingRoles
.map((r) => parseSource(r.source))
.filter((s): s is RoleSource => s !== null)
.map((s) => `${s.slug}:${s.language}`),
);
// Live role names (lowercased) for collision detection. Mutated as we
// insert so two imported roles cannot both grab the same name.
const takenNames = new Set(
existingRoles.map((r) => r.name.trim().toLowerCase()),
);
let created = 0;
let skipped = 0;
let renamed = 0;
for (const role of selected) {
// Already installed from the catalog in THIS language => skip (use
// update-from-catalog). A different language of the same slug still imports.
const installKey = `${role.slug}:${dto.language}`;
if (installedKeys.has(installKey)) {
skipped++;
continue;
}
let name = role.name.trim();
let didRename = false;
if (takenNames.has(name.toLowerCase())) {
if (dto.conflict === 'skip') {
skipped++;
continue;
}
// conflict === 'rename': find a free " (N)" suffix.
name = freeName(name, takenNames);
didRename = true;
}
const version = versions.get(role.slug) ?? 1;
try {
await this.repo.insert({
workspaceId,
creatorId,
name,
...catalogRoleContentFields(role),
enabled: true,
source: { slug: role.slug, language: dto.language, version },
});
created++;
if (didRename) renamed++;
takenNames.add(name.toLowerCase());
installedKeys.add(installKey);
} catch (err) {
// A 23505 from the source-uniqueness index means a CONCURRENT import
// already installed this exact slug+language between our snapshot
// (installedKeys) and this insert: the in-process snapshot cannot see a
// sibling request's writes, so the partial unique index is the backstop.
// Outcome is identical to the snapshot-based skip above — count it as
// skipped (already installed) and continue; do NOT abort or error.
if (isSourceUniqueViolation(err)) {
skipped++;
installedKeys.add(installKey);
continue;
}
// Otherwise: a unique-NAME race (23505 on the name index) is expected and
// self-explanatory (it becomes a friendly per-role error). Any OTHER
// insert failure is unexpected, so log the root cause with enough context
// to diagnose it — the user-facing message is deliberately generic.
if (!isUniqueViolation(err)) {
this.logger.error(
`Failed to import catalog role (workspaceId=${workspaceId} bundleId=${dto.bundleId} slug=${role.slug}): ${err instanceof Error ? err.stack ?? err.message : String(err)}`,
);
}
errors.push({ slug: role.slug, message: importErrorMessage(err) });
}
}
return { created, skipped, renamed, errors };
}
/**
* Update an already-imported role from its catalog source when the catalog
* ships a newer version. Returns a discriminated result so the UI can explain
* a no-op (up-to-date / removed from catalog / language no longer offered).
* Never touches `enabled`; keeps the current name if the catalog's new name
* would collide with another role (avoiding the unique-name 409).
*/
async updateFromCatalog(
workspaceId: string,
dto: UpdateFromCatalogDto,
): Promise<
| { updated: false; reason: 'not-in-catalog' | 'up-to-date' | 'language-unavailable' }
| { updated: true; fromVersion: number; toVersion: number; role: AgentRoleView }
> {
const role = await this.repo.findById(dto.id, workspaceId);
if (!role) throw new BadRequestException('Role not found');
const source = parseSource(role.source);
if (!source || !source.slug) {
throw new BadRequestException('Role was not imported from the catalog');
}
const index = await this.catalog.fetchIndex();
// Find the bundle whose meta lists this slug, and its catalog version.
let meta: CatalogBundleMeta | undefined;
let currentVersion: number | undefined;
for (const b of index.bundles) {
const m = b.roles.find((r) => r.slug === source.slug);
if (m) {
meta = b;
currentVersion = m.version;
break;
}
}
if (!meta || currentVersion === undefined) {
return { updated: false, reason: 'not-in-catalog' };
}
if (currentVersion <= source.version) {
return { updated: false, reason: 'up-to-date' };
}
if (!meta.languages.includes(source.language)) {
return { updated: false, reason: 'language-unavailable' };
}
const file = await this.catalog.fetchBundle(meta.id, source.language);
const fresh = file.roles.find((r) => r.slug === source.slug);
if (!fresh) {
return { updated: false, reason: 'not-in-catalog' };
}
// Keep the current name when the catalog's new name would collide with
// another live role (avoids the unique-name 409). Same-name (case-insensitive)
// means "no rename needed".
const newName = fresh.name.trim();
let name = newName;
if (newName.toLowerCase() !== role.name.trim().toLowerCase()) {
const others = await this.repo.listByWorkspace(workspaceId);
const collision = others.some(
(r) =>
r.id !== role.id &&
r.name.trim().toLowerCase() === newName.toLowerCase(),
);
if (collision) name = role.name;
}
await this.repo.update(dto.id, workspaceId, {
name,
...catalogRoleContentFields(fresh),
// enabled is deliberately NOT changed.
source: {
slug: source.slug,
language: source.language,
version: currentVersion,
},
});
const updated = await this.repo.findById(dto.id, workspaceId);
if (!updated) throw new BadRequestException('Role not found');
return {
updated: true,
fromVersion: source.version,
toVersion: currentVersion,
role: this.toView(updated),
};
}
private toView(row: AiAgentRole): AgentRoleView {
return {
id: row.id,
@@ -176,6 +507,9 @@ export class AiAgentRolesService {
enabled: row.enabled,
autoStart: row.autoStart,
launchMessage: row.launchMessage ?? null,
// parseSource yields a fully-valid RoleSource | null (the row is already
// normalized; this also keeps the field type honest without a cast).
source: parseSource(row.source),
createdAt: row.createdAt,
updatedAt: row.updatedAt,
};
@@ -205,11 +539,7 @@ export class AiAgentRolesService {
* failures keep surfacing as 500s.
*/
function rethrowDuplicateName(err: unknown, name: string): never {
if (
err &&
typeof err === 'object' &&
(err as { code?: unknown }).code === '23505'
) {
if (isUniqueViolation(err)) {
throw new ConflictException(
`A role named "${name}" already exists in this workspace.`,
);
@@ -217,13 +547,120 @@ function rethrowDuplicateName(err: unknown, name: string): never {
throw err;
}
/** '' / whitespace-only / undefined => null; otherwise the trimmed value. */
function emptyToNull(value: string | undefined): string | null {
if (value === undefined) return null;
/** Whether `err` is a Postgres unique-violation (SQLSTATE 23505). */
function isUniqueViolation(err: unknown): boolean {
return (
!!err &&
typeof err === 'object' &&
(err as { code?: unknown }).code === '23505'
);
}
/**
* The partial unique index name from the
* 20260626T160000-ai-agent-roles-catalog-source-unique migration: unique on
* (workspace_id, source->>'slug', source->>'language') for catalog-imported,
* non-deleted rows. A 23505 carrying this constraint name is a source-collision
* (concurrent import of the same slug+language), distinct from a name-collision.
*/
const SOURCE_UNIQUE_CONSTRAINT = 'ai_agent_roles_workspace_source_unique';
/**
* Whether `err` is the 23505 raised by the SOURCE-uniqueness index specifically
* (vs the name-uniqueness index). The active driver (`kysely-postgres-js` over
* `postgres@3.4.8`) exposes the violated constraint name on `constraint_name`,
* so we key off that (accepting the node-postgres-style `.constraint` as a
* fallback for other drivers) — that way a source race is skipped while a name
* race still surfaces as a friendly per-role error. A 23505 with no constraint
* name (e.g. a wrapped/test error) is NOT treated as a source collision,
* preserving the existing name-race behavior.
*/
function isSourceUniqueViolation(err: unknown): boolean {
if (!isUniqueViolation(err)) return false;
const e = err as { constraint_name?: unknown; constraint?: unknown };
return (
e.constraint_name === SOURCE_UNIQUE_CONSTRAINT ||
e.constraint === SOURCE_UNIQUE_CONSTRAINT
);
}
/**
* The role-content fields shared by import (insert) and update (patch) of a
* catalog role: emoji/description/launchMessage normalized to null, model config
* normalized, autoStart defaulted. The caller adds the write-specific fields
* (`name`, `source`, and on insert `workspaceId`/`creatorId`/`enabled`).
*/
function catalogRoleContentFields(role: CatalogRole): {
emoji: string | null;
description: string | null;
instructions: string;
modelConfig: Record<string, unknown> | null;
autoStart: boolean;
launchMessage: string | null;
} {
return {
emoji: emptyToNull(role.emoji),
description: emptyToNull(role.description),
instructions: role.instructions,
modelConfig: normalizeModelConfig(role.modelConfig) as
| Record<string, unknown>
| null,
autoStart: role.autoStart ?? true,
launchMessage: emptyToNull(role.launchMessage ?? undefined),
};
}
/** '' / whitespace-only / undefined / null => null; otherwise the trimmed value. */
function emptyToNull(value: string | null | undefined): string | null {
if (value === undefined || value === null) return null;
const trimmed = value.trim();
return trimmed.length > 0 ? trimmed : null;
}
/** slug -> version map from a bundle's index metadata. */
function versionMap(meta: CatalogBundleMeta): Map<string, number> {
return new Map(meta.roles.map((r) => [r.slug, r.version]));
}
/**
* Resolve a localized value `{ en, ru, ... }` to `language`, falling back to
* 'en', then the first available locale. Returns null only for an empty map.
*/
function localized(
map: Record<string, string>,
language?: string,
): string | null {
if (language && typeof map[language] === 'string') return map[language];
if (typeof map.en === 'string') return map.en;
const first = Object.values(map)[0];
return typeof first === 'string' ? first : null;
}
/**
* Find a free display name by appending " (2)", " (3)", ... when `base` is
* already taken (case-insensitive against `taken`). Caller adds the result to
* `taken` after a successful insert.
*/
function freeName(base: string, taken: Set<string>): string {
// `taken` is finite, so within `taken.size + 2` iterations a candidate index
// is guaranteed free; the 1000 cap is a defensive upper bound far above any
// realistic per-name collision count. The throw below is therefore
// unreachable in practice and only satisfies the return-type checker.
for (let n = 2; n < 1000; n++) {
const candidate = `${base} (${n})`;
if (!taken.has(candidate.toLowerCase())) return candidate;
}
throw new BadRequestException(`Too many roles named "${base}"`);
}
/** A short, safe message for an import insert failure (409 vs other). */
function importErrorMessage(err: unknown): string {
if (isUniqueViolation(err)) {
return 'A role with this name already exists';
}
return 'Failed to import role';
}
/**
* Normalize an incoming modelConfig DTO to the persisted shape, or null when
* there is no usable override (no driver and no chatModel). The DTO's @IsIn

View File

@@ -0,0 +1,307 @@
import { BadGatewayException, BadRequestException } from '@nestjs/common';
import { AiAgentRolesCatalogProvider } from './ai-agent-roles-catalog.provider';
/**
* Provider tests against a mocked remote source (no network). They cover the
* happy read path (fetchIndex / fetchBundle), the malformed-shape rejection,
* rejection of non-http(s) sources (local sources are gone), and — most
* importantly — the `^[a-z0-9-]+$` path-traversal guard that runs BEFORE any
* path/URL is built.
*/
describe('AiAgentRolesCatalogProvider', () => {
function makeProvider(source: string) {
const env = {
getAiAgentRolesCatalogSource: () => source,
};
return new AiAgentRolesCatalogProvider(env as never);
}
it('non-http(s) source => BadGateway (local sources removed)', async () => {
for (const source of ['', '/var/lib/agent-roles-catalog', './agent-roles-catalog']) {
const provider = makeProvider(source);
await expect(provider.fetchIndex()).rejects.toBeInstanceOf(
BadGatewayException,
);
}
});
describe('remote fetch streaming size cap', () => {
const realFetch = global.fetch;
afterEach(() => {
global.fetch = realFetch;
});
/** A web ReadableStream that yields `chunks` (each a Uint8Array). */
function streamOf(chunks: Uint8Array[]): ReadableStream<Uint8Array> {
let i = 0;
return new ReadableStream<Uint8Array>({
pull(controller) {
if (i < chunks.length) controller.enqueue(chunks[i++]);
else controller.close();
},
// The provider cancels the reader on the too-large path; no-op here.
cancel() {},
});
}
/** A ReadableStream whose first read rejects (e.g. a mid-body AbortError). */
function errorStream(err: Error): ReadableStream<Uint8Array> {
return new ReadableStream<Uint8Array>({
pull() {
throw err;
},
cancel() {},
});
}
function mockResponse(opts: {
ok?: boolean;
status?: number;
headers?: Record<string, string>;
body: ReadableStream<Uint8Array> | null;
text?: string;
}): Response {
return {
ok: opts.ok ?? true,
status: opts.status ?? 200,
headers: { get: (k: string) => opts.headers?.[k.toLowerCase()] ?? null },
body: opts.body,
text: async () => opts.text ?? 'unused',
} as unknown as Response;
}
it('fetchBundle remote happy path => parses + validates', async () => {
const json = JSON.stringify({
schemaVersion: 1,
language: 'en',
roles: [
{
slug: 'researcher',
name: 'Researcher',
instructions: 'be a researcher',
},
],
});
const body = streamOf([new TextEncoder().encode(json)]);
global.fetch = jest
.fn()
.mockResolvedValue(mockResponse({ body })) as never;
const provider = makeProvider('https://catalog.example.com');
const bundle = await provider.fetchBundle('general', 'en');
expect(bundle.roles[0].slug).toBe('researcher');
});
it('fetchBundle remote malformed (role missing instructions) => BadGateway', async () => {
const json = JSON.stringify({
schemaVersion: 1,
language: 'fr',
roles: [{ slug: 'researcher', name: 'Chercheur' }],
});
const body = streamOf([new TextEncoder().encode(json)]);
global.fetch = jest
.fn()
.mockResolvedValue(mockResponse({ body })) as never;
const provider = makeProvider('https://catalog.example.com');
await expect(provider.fetchBundle('general', 'fr')).rejects.toBeInstanceOf(
BadGatewayException,
);
});
it('declared Content-Length over the cap => BadGateway before reading the body', async () => {
global.fetch = jest.fn().mockResolvedValue(
mockResponse({
headers: { 'content-length': String(2_000_000) },
body: streamOf([new Uint8Array(10)]),
}),
) as never;
const provider = makeProvider('https://catalog.example.com');
await expect(provider.fetchIndex()).rejects.toBeInstanceOf(
BadGatewayException,
);
});
it('streamed body exceeding the cap (no/under-reported Content-Length) => BadGateway', async () => {
// 1.5 MB streamed in 256 KB chunks, with no Content-Length header.
const chunks = Array.from(
{ length: 6 },
() => new Uint8Array(256 * 1024),
);
global.fetch = jest
.fn()
.mockResolvedValue(mockResponse({ body: streamOf(chunks) })) as never;
const provider = makeProvider('https://catalog.example.com');
await expect(provider.fetchIndex()).rejects.toBeInstanceOf(
BadGatewayException,
);
});
it('fetch rejects (network failure) => BadGateway (unavailable)', async () => {
global.fetch = jest
.fn()
.mockRejectedValue(new Error('ECONNREFUSED')) as never;
const provider = makeProvider('https://catalog.example.com');
await expect(provider.fetchIndex()).rejects.toBeInstanceOf(
BadGatewayException,
);
});
it('passes redirect:"error" to fetch (redirect-SSRF hardening)', async () => {
const fetchMock = jest
.fn()
.mockResolvedValue(
mockResponse({ body: streamOf([new Uint8Array(0)]) }),
);
global.fetch = fetchMock as never;
const provider = makeProvider('https://catalog.example.com');
// Body shape is irrelevant; an empty stream parses to invalid JSON and
// throws, but the fetch call (with its init) still happened.
await expect(provider.fetchIndex()).rejects.toBeDefined();
expect(fetchMock).toHaveBeenCalledWith(
expect.any(String),
expect.objectContaining({ redirect: 'error' }),
);
});
it('redirect response rejects (redirect:"error") => BadGateway', async () => {
// With redirect:"error", the platform fetch rejects on a 3xx instead of
// following it. Simulate that: the mock rejects when asked not to follow.
global.fetch = jest.fn().mockImplementation((_url, init) => {
if (init?.redirect === 'error') {
return Promise.reject(
new TypeError('fetch failed: unexpected redirect'),
);
}
return Promise.resolve(
mockResponse({ status: 302, body: null }),
);
}) as never;
const provider = makeProvider('https://catalog.example.com');
await expect(provider.fetchIndex()).rejects.toBeInstanceOf(
BadGatewayException,
);
});
it('non-ok response (503) => BadGateway carrying the status', async () => {
global.fetch = jest.fn().mockResolvedValue(
mockResponse({ ok: false, status: 503, body: null }),
) as never;
const provider = makeProvider('https://catalog.example.com');
await expect(provider.fetchIndex()).rejects.toThrow(/503/);
});
it('small streamed body parses normally (cap not hit)', async () => {
const json = JSON.stringify({
schemaVersion: 1,
bundles: [
{
id: 'general',
name: { en: 'General' },
languages: ['en'],
roles: [{ slug: 'researcher', version: 2 }],
},
],
});
const body = streamOf([new TextEncoder().encode(json)]);
global.fetch = jest
.fn()
.mockResolvedValue(mockResponse({ body })) as never;
const provider = makeProvider('https://catalog.example.com');
const index = await provider.fetchIndex();
expect(index.bundles[0].id).toBe('general');
});
it('body read aborts mid-stream (AbortError) => BadGateway (not a generic 500)', async () => {
// The 10s timer aborts the whole request; on a slow/dripping source the
// body read (reader.read()) rejects with an AbortError AFTER fetch()
// resolved. The provider must map that to BadGateway, not let it escape.
const abortErr = Object.assign(new Error('The operation was aborted'), {
name: 'AbortError',
});
global.fetch = jest
.fn()
.mockResolvedValue(mockResponse({ body: errorStream(abortErr) })) as never;
const provider = makeProvider('https://catalog.example.com');
await expect(provider.fetchIndex()).rejects.toBeInstanceOf(
BadGatewayException,
);
});
it('null body (no readable stream) => response.text() fallback parses', async () => {
const json = JSON.stringify({
schemaVersion: 1,
bundles: [
{
id: 'general',
name: { en: 'General' },
languages: ['en'],
roles: [{ slug: 'researcher', version: 2 }],
},
],
});
global.fetch = jest
.fn()
.mockResolvedValue(mockResponse({ body: null, text: json })) as never;
const provider = makeProvider('https://catalog.example.com');
const index = await provider.fetchIndex();
expect(index.bundles[0].id).toBe('general');
});
it('null body + text() over the cap => BadGateway (too large)', async () => {
const oversized = 'a'.repeat(1_000_001);
global.fetch = jest
.fn()
.mockResolvedValue(
mockResponse({ body: null, text: oversized }),
) as never;
const provider = makeProvider('https://catalog.example.com');
await expect(provider.fetchIndex()).rejects.toBeInstanceOf(
BadGatewayException,
);
});
it('invalid JSON body => BadGateway (parse failure)', async () => {
const body = streamOf([new TextEncoder().encode('{not valid json')]);
global.fetch = jest
.fn()
.mockResolvedValue(mockResponse({ body })) as never;
const provider = makeProvider('https://catalog.example.com');
await expect(provider.fetchIndex()).rejects.toBeInstanceOf(
BadGatewayException,
);
});
it('malformed index.json (valid JSON, wrong shape) => BadGateway', async () => {
// Parses as JSON but fails isCatalogIndex (schemaVersion not a number).
const body = streamOf([
new TextEncoder().encode(
JSON.stringify({ schemaVersion: 'x', bundles: [] }),
),
]);
global.fetch = jest
.fn()
.mockResolvedValue(mockResponse({ body })) as never;
const provider = makeProvider('https://catalog.example.com');
await expect(provider.fetchIndex()).rejects.toThrow(/malformed/i);
});
});
describe('path-traversal / SSRF guard (^[a-z0-9-]+$)', () => {
const bad = ['../etc', 'a/b', 'A', 'foo.bar', 'foo_bar', '', '..'];
for (const value of bad) {
it(`rejects bundleId="${value}" with BadRequest`, async () => {
const provider = makeProvider('https://catalog.example.com');
await expect(
provider.fetchBundle(value, 'en'),
).rejects.toBeInstanceOf(BadRequestException);
});
it(`rejects language="${value}" with BadRequest`, async () => {
const provider = makeProvider('https://catalog.example.com');
await expect(
provider.fetchBundle('general', value),
).rejects.toBeInstanceOf(BadRequestException);
});
}
});
});

View File

@@ -0,0 +1,311 @@
import {
BadGatewayException,
BadRequestException,
Injectable,
Logger,
} from '@nestjs/common';
import { EnvironmentService } from '../../../../integrations/environment/environment.service';
import {
CatalogBundleFile,
CatalogBundleMeta,
CatalogIndex,
CatalogRole,
} from './catalog-types';
/** Identifier shape allowed in any path/URL segment (bundleId, language). The
* ONLY characters that can appear in a fetched path — the path-traversal and
* SSRF guard. Anything else is rejected before a path/URL is built. */
const SEGMENT_RE = /^[a-z0-9-]+$/;
/** Remote fetch timeout and response-size cap. A curated catalog file is tiny;
* the cap stops a hostile/misconfigured source from streaming unbounded data. */
const FETCH_TIMEOUT_MS = 10_000;
const MAX_BYTES = 1_000_000;
/**
* Fetches + validates the agent-roles catalog from its configured source. The
* source (EnvironmentService.getAiAgentRolesCatalogSource()) is an http(s)://
* base URL — REMOTE only; local-filesystem sources are no longer supported. The
* value is baked into the Docker image at build time (set per-branch in CI).
*
* The catalog is UNTRUSTED input: every file is JSON-parsed and run through a
* hand-written type guard before any field is exposed, and every dynamic path
* segment is validated against SEGMENT_RE up front (path-traversal + SSRF).
*/
@Injectable()
export class AiAgentRolesCatalogProvider {
private readonly logger = new Logger(AiAgentRolesCatalogProvider.name);
constructor(private readonly environmentService: EnvironmentService) {}
/** Read + validate the top-level index (`index.json`). */
async fetchIndex(): Promise<CatalogIndex> {
const raw = await this.readRelative('index.json');
const parsed = this.parseJson(raw, 'index.json');
if (!isCatalogIndex(parsed)) {
throw new BadGatewayException(
'Agent roles catalog index is malformed (index.json)',
);
}
return parsed;
}
/** Read + validate one language file (`bundles/<bundleId>/<language>.json`). */
async fetchBundle(
bundleId: string,
language: string,
): Promise<CatalogBundleFile> {
// SECURITY: validate BEFORE building any path/URL (path-traversal + SSRF).
this.assertSegment(bundleId, 'bundleId');
this.assertSegment(language, 'language');
const rel = `bundles/${bundleId}/${language}.json`;
const raw = await this.readRelative(rel);
const parsed = this.parseJson(raw, rel);
if (!isCatalogBundleFile(parsed)) {
throw new BadGatewayException(
`Agent roles catalog bundle is malformed (${rel})`,
);
}
return parsed;
}
/** Reject a segment that is not a safe `[a-z0-9-]+` identifier. */
private assertSegment(value: string, field: string): void {
if (typeof value !== 'string' || !SEGMENT_RE.test(value)) {
throw new BadRequestException(`Invalid ${field}`);
}
}
/** JSON.parse with a clear BadGateway on malformed content. */
private parseJson(raw: string, rel: string): unknown {
try {
return JSON.parse(raw);
} catch (err) {
const reason = shortError(err);
this.logger.error(`Agent roles catalog JSON parse failed (${rel}): ${reason}`);
throw new BadGatewayException(
`Agent roles catalog file is not valid JSON (${rel}): ${reason}`,
);
}
}
/** Read a relative catalog path as text from the configured remote source. */
private async readRelative(rel: string): Promise<string> {
const source = this.environmentService
.getAiAgentRolesCatalogSource()
.trim();
if (!/^https?:\/\//i.test(source)) {
this.logger.error(
'Agent roles catalog source is not configured (expected an http(s):// base URL)',
);
throw new BadGatewayException(
'Agent roles catalog is unavailable: source is not configured',
);
}
return this.fetchRemote(source, rel);
}
/**
* Fetch a remote catalog file with a timeout + a STREAMING size cap. The body
* is never buffered in full before the check: we reject on a too-large
* Content-Length up front, then read the stream chunk-by-chunk and abort the
* moment the running total exceeds MAX_BYTES, so a hostile/misconfigured
* source cannot make us hold an unbounded body in memory.
*/
private async fetchRemote(base: string, rel: string): Promise<string> {
const url = `${base.replace(/\/+$/, '')}/${rel}`;
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), FETCH_TIMEOUT_MS);
try {
let response: Response;
try {
// `redirect: 'error'` hardens against redirect-SSRF: a
// compromised-but-trusted upstream cannot 3xx the fetch into the
// internal network (e.g. http://169.254.169.254/...). A redirect
// response rejects here and is mapped to BadGateway below.
response = await fetch(url, {
signal: controller.signal,
redirect: 'error',
});
} catch (err) {
const reason = shortError(err);
this.logger.error(
`Agent roles catalog remote fetch failed (${rel}): ${reason}`,
);
throw new BadGatewayException(
`Agent roles catalog is unavailable: ${reason}`,
);
}
if (!response.ok) {
this.logger.error(
`Agent roles catalog remote returned ${response.status} (${rel})`,
);
throw new BadGatewayException(
`Agent roles catalog returned ${response.status}`,
);
}
// Reject a too-large declared size before reading any body bytes.
const declared = Number(response.headers.get('content-length'));
if (Number.isFinite(declared) && declared > MAX_BYTES) {
throw new BadGatewayException('Agent roles catalog file is too large');
}
// Bound the actual read: a missing/lying Content-Length is caught here.
// The 10s timer aborts the WHOLE request, so a slow/dripping hostile
// source rejects reader.read() (or response.text()) with an AbortError
// mid-body. Map that — and any other read failure — to a logged
// BadGateway so the admin endpoint returns 502 (not a generic 500). The
// cap's own BadGateway is rethrown as-is (no double-wrap).
try {
if (response.body) {
return await readStreamCapped(response.body, MAX_BYTES);
}
// Edge: no readable stream — fall back to a buffered read + length check.
const text = await response.text();
if (text.length > MAX_BYTES) {
throw new BadGatewayException('Agent roles catalog file is too large');
}
return text;
} catch (err) {
if (err instanceof BadGatewayException) throw err;
const reason = shortError(err);
this.logger.error(
`Agent roles catalog body read failed (${rel}): ${reason}`,
);
throw new BadGatewayException(
`Agent roles catalog is unavailable: ${reason}`,
);
}
} finally {
clearTimeout(timer);
}
}
}
/**
* Read a web ReadableStream into a UTF-8 string, throwing as soon as the
* accumulated byte count exceeds `maxBytes` (the reader is cancelled so the
* underlying connection is released). Never buffers more than the cap + the
* final chunk before bailing out.
*/
async function readStreamCapped(
body: ReadableStream<Uint8Array>,
maxBytes: number,
): Promise<string> {
const reader = body.getReader();
const chunks: Uint8Array[] = [];
let total = 0;
try {
for (;;) {
const { done, value } = await reader.read();
if (done) break;
if (!value) continue;
total += value.length;
if (total > maxBytes) {
throw new BadGatewayException('Agent roles catalog file is too large');
}
chunks.push(value);
}
} finally {
// Release the stream on both the normal and the too-large/abort paths.
await reader.cancel().catch(() => undefined);
}
return Buffer.concat(chunks).toString('utf8');
}
/**
* A short, non-sensitive error string for logging/propagation: only the first
* line of the message head is kept (upstream bodies / URLs are discarded).
*/
function shortError(err: unknown): string {
let message = '';
if (typeof err === 'string') {
message = err;
} else if (
err &&
typeof err === 'object' &&
typeof (err as { message?: unknown }).message === 'string'
) {
// Read `.message` directly (works for Error instances and the realm-shifted
// Error-likes jest can hand back, where `instanceof Error` is false).
message = (err as { message: string }).message;
}
const head = (message || 'unknown error').split('\n')[0];
return head.length > 200 ? `${head.slice(0, 200)}` : head;
}
// ---------------------------------------------------------------------------
// Hand-written type guards (no zod / new deps). Each validates the exact wire
// shape declared in catalog-types.ts; anything else is rejected by the caller.
// ---------------------------------------------------------------------------
function isObject(v: unknown): v is Record<string, unknown> {
return v !== null && typeof v === 'object' && !Array.isArray(v);
}
function isStringMap(v: unknown): v is Record<string, string> {
if (!isObject(v)) return false;
return Object.values(v).every((x) => typeof x === 'string');
}
function isStringArray(v: unknown): v is string[] {
return Array.isArray(v) && v.every((x) => typeof x === 'string');
}
export function isCatalogRole(v: unknown): v is CatalogRole {
if (!isObject(v)) return false;
if (typeof v.slug !== 'string') return false;
if (typeof v.name !== 'string') return false;
if (typeof v.instructions !== 'string') return false;
if (v.emoji !== undefined && typeof v.emoji !== 'string') return false;
if (v.description !== undefined && typeof v.description !== 'string') {
return false;
}
if (v.autoStart !== undefined && typeof v.autoStart !== 'boolean') {
return false;
}
if (
v.launchMessage !== undefined &&
v.launchMessage !== null &&
typeof v.launchMessage !== 'string'
) {
return false;
}
if (
v.modelConfig !== undefined &&
v.modelConfig !== null &&
!isObject(v.modelConfig)
) {
return false;
}
return true;
}
export function isCatalogBundleFile(v: unknown): v is CatalogBundleFile {
if (!isObject(v)) return false;
if (typeof v.schemaVersion !== 'number') return false;
if (typeof v.language !== 'string') return false;
if (!Array.isArray(v.roles)) return false;
return v.roles.every(isCatalogRole);
}
function isCatalogBundleMeta(v: unknown): v is CatalogBundleMeta {
if (!isObject(v)) return false;
if (typeof v.id !== 'string') return false;
if (!isStringMap(v.name)) return false;
if (v.description !== undefined && !isStringMap(v.description)) return false;
if (!isStringArray(v.languages)) return false;
if (!Array.isArray(v.roles)) return false;
return v.roles.every(
(r) =>
isObject(r) &&
typeof r.slug === 'string' &&
typeof r.version === 'number',
);
}
export function isCatalogIndex(v: unknown): v is CatalogIndex {
if (!isObject(v)) return false;
if (typeof v.schemaVersion !== 'number') return false;
if (!Array.isArray(v.bundles)) return false;
return v.bundles.every(isCatalogBundleMeta);
}

View File

@@ -0,0 +1,47 @@
/**
* Catalog wire shapes. The catalog is curated, untrusted JSON (a GitHub repo or
* a local folder), so every shape is validated by a hand-written type guard in
* the provider before any field is used — no zod / new deps on the server.
*
* Localized fields (`name` / `description` at the bundle level) are
* `Record<language, string>` so one bundle serves many UI languages; per-role
* `name` / `description` are already language-specific (the bundle file is keyed
* by language).
*/
/** One role's content as shipped in a per-language bundle file. */
export interface CatalogRole {
slug: string;
emoji?: string;
name: string;
description?: string;
instructions: string;
autoStart?: boolean;
launchMessage?: string | null;
// Optional model override; same loose object shape as ai_agent_roles.model_config.
modelConfig?: Record<string, unknown> | null;
}
/** A single language file: `bundles/<id>/<language>.json`. */
export interface CatalogBundleFile {
schemaVersion: number;
language: string;
roles: CatalogRole[];
}
/** Bundle metadata as listed in the top-level index. Versions live here (per
* slug), so an UPDATE check needs only the index, not every language file. */
export interface CatalogBundleMeta {
id: string;
// Localized display name/description: { en: '...', ru: '...' }.
name: Record<string, string>;
description?: Record<string, string>;
languages: string[];
roles: { slug: string; version: number }[];
}
/** Top-level catalog index: `index.json`. */
export interface CatalogIndex {
schemaVersion: number;
bundles: CatalogBundleMeta[];
}

View File

@@ -0,0 +1,62 @@
import {
IsArray,
IsIn,
IsOptional,
IsString,
IsUUID,
Matches,
MaxLength,
} from 'class-validator';
/** Safe identifier shape for any catalog path segment (bundleId / language).
* Mirrors SEGMENT_RE in the catalog provider — the path-traversal/SSRF guard
* is enforced both at the API boundary (here) and in the provider. */
const SEGMENT_RE = /^[a-z0-9-]+$/;
/** Browse the catalog, optionally localized to `language` (defaults applied in
* the service: fall back to 'en', then the first available language). */
export class CatalogQueryDto {
@IsOptional()
@IsString()
@MaxLength(16)
language?: string;
}
/** Open one catalog bundle in a specific language. */
export class CatalogBundleDto {
@IsString()
@Matches(SEGMENT_RE)
bundleId: string;
@IsString()
@Matches(SEGMENT_RE)
language: string;
}
/** Import roles from a catalog bundle into the workspace. */
export class ImportFromCatalogDto {
@IsString()
@Matches(SEGMENT_RE)
bundleId: string;
@IsString()
@Matches(SEGMENT_RE)
language: string;
// Omitted => import the whole bundle; otherwise only these slugs.
@IsOptional()
@IsArray()
@IsString({ each: true })
slugs?: string[];
// How to handle a name collision with an existing (non-catalog) role:
// 'skip' leaves it; 'rename' imports under a free " (N)" name.
@IsIn(['skip', 'rename'])
conflict: 'skip' | 'rename';
}
/** Update an already-imported role from its catalog source. */
export class UpdateFromCatalogDto {
@IsUUID()
id: string;
}

View File

@@ -0,0 +1,153 @@
// Binding test for issue #228 must-fix #1 / test-coverage #12: footnote
// canonicalization moved OUT of parseProsemirrorContent and is now applied only
// on FULL-document writes (createPage, and updatePageContent with operation
// 'replace'), NEVER on an append/prepend FRAGMENT.
//
// The Yjs encode / plain-text extract are stubbed (partial module mock keeps the
// REAL canonicalizeFootnotes) and parseProsemirrorContent is spied to return the
// raw fixture, so the test isolates the canonicalize BINDING from schema/Yjs.
jest.mock('@docmost/editor-ext', () => {
const actual = jest.requireActual('@docmost/editor-ext');
return {
...actual,
createYdocFromJson: jest.fn(() => Buffer.from([])),
jsonToText: jest.fn(() => ''),
};
});
import { PageService } from './page.service';
const refNode = (id: string) => ({ type: 'footnoteReference', attrs: { id } });
const defNode = (id: string, text: string) => ({
type: 'footnoteDefinition',
attrs: { id },
content: [{ type: 'paragraph', content: [{ type: 'text', text }] }],
});
const doc = (...content: any[]) => ({ type: 'doc', content });
/** A full doc whose footnote definitions are OUT of reference order (b,a refs;
* a,b defs) — canonicalization must reorder the definitions to [b, a]. */
const outOfOrderFull = () =>
doc(
{ type: 'paragraph', content: [{ type: 'text', text: 'x' }, refNode('b'), refNode('a')] },
{ type: 'footnotesList', content: [defNode('a', 'A'), defNode('b', 'B')] },
);
/** A definition-ONLY fragment (no references): canonicalizing it would drop the
* whole footnotesList (referenceIds is empty) — i.e. LOSE the footnote. */
const defOnlyFragment = () =>
doc({ type: 'footnotesList', content: [defNode('a', 'appended note')] });
/** A reference-only fragment that REUSES an id defined elsewhere in the live
* doc: canonicalizing it would synthesize a bogus empty footnotesList/def. */
const refReuseFragment = () =>
doc({ type: 'paragraph', content: [{ type: 'text', text: 'more' }, refNode('a')] });
function listDefIds(content: any): string[] {
const list = (content.content ?? []).find((n: any) => n.type === 'footnotesList');
return (list?.content ?? [])
.filter((n: any) => n.type === 'footnoteDefinition')
.map((n: any) => n.attrs?.id);
}
function hasFootnotesList(content: any): boolean {
return (content.content ?? []).some((n: any) => n.type === 'footnotesList');
}
describe('PageService footnote canonicalization binding (#228)', () => {
function makeService() {
let insertedContent: any = null;
let yjsPayload: any = null;
const pageRepo = {
insertPage: jest.fn(async (values: any) => {
insertedContent = values.content;
return { id: 'page-id', slugId: 'slug-id' };
}),
};
const generalQueue = { add: jest.fn().mockReturnValue({ catch: jest.fn() }) };
const collaborationGateway = {
handleYjsEvent: jest.fn(async (_evt: string, _name: string, payload: any) => {
yjsPayload = payload;
}),
};
const service = new PageService(
pageRepo as any,
{} as any, // pagePermissionRepo
{} as any, // attachmentRepo
{} as any, // db
{} as any, // storageService
{} as any, // attachmentQueue
{} as any, // aiQueue
generalQueue as any,
{} as any, // eventEmitter
collaborationGateway as any,
{} as any, // watcherService
{} as any, // transclusionService
);
// Isolate the canonicalize BINDING: return the raw fixture (a deep clone so
// canonicalize never mutates the caller's object) instead of running the
// real markdown/HTML/JSON parse + schema validation.
jest
.spyOn(service as any, 'parseProsemirrorContent')
.mockImplementation(async (content: any) => structuredClone(content));
jest.spyOn(service as any, 'nextPagePosition').mockResolvedValue('a0');
return { service, getInsertedContent: () => insertedContent, getYjsPayload: () => yjsPayload };
}
it('createPage (full write) canonicalizes footnotes into reference order', async () => {
const { service, getInsertedContent } = makeService();
await service.create('user-id', 'workspace-id', {
spaceId: 'space-id',
content: outOfOrderFull(),
format: 'json',
} as any);
// Definitions reordered to reference order [b, a].
expect(listDefIds(getInsertedContent())).toEqual(['b', 'a']);
});
it("updatePageContent operation 'replace' canonicalizes footnotes", async () => {
const { service, getYjsPayload } = makeService();
await service.updatePageContent(
'page-id',
outOfOrderFull(),
'replace' as any,
'json' as any,
{ id: 'user-id' } as any,
);
expect(getYjsPayload().operation).toBe('replace');
expect(listDefIds(getYjsPayload().prosemirrorJson)).toEqual(['b', 'a']);
});
it("append of a definition-only fragment is NOT canonicalized (footnote preserved, not dropped)", async () => {
const { service, getYjsPayload } = makeService();
await service.updatePageContent(
'page-id',
defOnlyFragment(),
'append' as any,
'json' as any,
{ id: 'user-id' } as any,
);
// Canonicalizing a reference-less fragment would DROP the whole list; the
// fragment must pass through untouched so the merge keeps the definition.
expect(getYjsPayload().operation).toBe('append');
expect(hasFootnotesList(getYjsPayload().prosemirrorJson)).toBe(true);
expect(listDefIds(getYjsPayload().prosemirrorJson)).toEqual(['a']);
});
it('prepend of a reference-reuse fragment is NOT canonicalized (no synthesized garbage list)', async () => {
const { service, getYjsPayload } = makeService();
await service.updatePageContent(
'page-id',
refReuseFragment(),
'prepend' as any,
'json' as any,
{ id: 'user-id' } as any,
);
// Canonicalizing would synthesize a bogus empty footnotesList for the reused
// reference; the fragment must pass through with no list at all.
expect(getYjsPayload().operation).toBe('prepend');
expect(hasFootnotesList(getYjsPayload().prosemirrorJson)).toBe(false);
});
});

View File

@@ -52,7 +52,7 @@ import {
INTERNAL_LINK_REGEX,
extractPageSlugId,
} from '../../../integrations/export/utils';
import { markdownToHtml } from '@docmost/editor-ext';
import { markdownToHtml, canonicalizeFootnotes } from '@docmost/editor-ext';
import { WatcherService } from '../../watcher/watcher.service';
import { sql } from 'kysely';
import { TransclusionService } from '../transclusion/transclusion.service';
@@ -160,9 +160,14 @@ export class PageService {
let ydoc = undefined;
if (createPageDto?.content && createPageDto?.format) {
const prosemirrorJson = await this.parseProsemirrorContent(
createPageDto.content,
createPageDto.format,
// createPage always writes a FULL document, so canonicalize footnotes to
// the editor's invariant before persisting (issue #228). Pure + idempotent
// + shape-safe: a doc with no footnotes is returned unchanged.
const prosemirrorJson = canonicalizeFootnotes(
await this.parseProsemirrorContent(
createPageDto.content,
createPageDto.format,
),
);
content = prosemirrorJson;
@@ -343,7 +348,17 @@ export class PageService {
format: ContentFormat,
user: User,
): Promise<void> {
const prosemirrorJson = await this.parseProsemirrorContent(content, format);
let prosemirrorJson = await this.parseProsemirrorContent(content, format);
// Canonicalize footnotes ONLY for a full-document write ('replace'). For an
// append/prepend FRAGMENT, canonicalizing is semantically wrong (it would
// drop a definition-only fragment's list, or synthesize a duplicate empty
// definition for a fragment reusing an existing id) — the fragment merges
// into the live doc where the editor's footnoteSyncPlugin keeps the invariant
// (issue #228, must-fix #1).
if (operation === 'replace') {
prosemirrorJson = canonicalizeFootnotes(prosemirrorJson);
}
const documentName = `page.${pageId}`;
await this.collaborationGateway.handleYjsEvent(
@@ -1301,6 +1316,24 @@ export class PageService {
}
}
// NOTE: footnote canonicalization is intentionally NOT done here. This
// method serves BOTH full writes (createPage / updatePageContent with
// operation 'replace') AND fragment writes (append / prepend). Canonicalizing
// a FRAGMENT is semantically wrong — e.g. a definition-only fragment has no
// references, so the canonicalizer would drop its whole footnotesList (lost
// footnotes), and a fragment reusing an existing id would synthesize an empty
// duplicate definition. The canonicalizer therefore runs only at the
// FULL-DOCUMENT callers (createPage, and updatePageContent for 'replace'),
// never on a fragment (issue #228, must-fix #1).
// (Future consolidation, architecture B: the import services persist via a
// different path; folding all of these into one "prepare JSON for persist"
// helper would centralize the canonicalize call — left as follow-up.)
//
// ENFORCEMENT RULE (#228): any NEW FULL-document persist path MUST call
// `canonicalizeFootnotes(json)` before writing (see createPage and
// updatePageContent 'replace'); append/prepend FRAGMENT writes MUST NOT (it
// would drop or duplicate footnotes — that is exactly why this is per-call-site
// rather than a single wrapper here).
try {
jsonToNode(prosemirrorJson);
} catch (err) {

View File

@@ -1,4 +1,5 @@
import { BadRequestException, ConflictException } from '@nestjs/common';
import { NoResultError } from 'kysely';
import { ShareAliasService } from './share-alias.service';
/**
@@ -355,6 +356,68 @@ describe('ShareAliasService', () => {
}
});
it('maps a concurrent-delete race in the SWAP branch to a retryable 409 (not a 200-without-alias)', async () => {
const { service, shareAliasRepo } = makeService();
// Name points at another page; reassign confirmed -> swap branch.
shareAliasRepo.findByAliasAndWorkspace.mockResolvedValue({
id: 'a-1',
alias: 'foo',
pageId: 'p-other',
});
// A concurrent removeAlias deleted the row between read and UPDATE, so the
// repo's executeTakeFirstOrThrow finds 0 rows and throws NoResultError.
shareAliasRepo.updatePageId.mockRejectedValue(
new NoResultError({} as any),
);
try {
await service.setAlias({
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
alias: 'foo',
confirmReassign: true,
});
fail('expected ConflictException');
} catch (err) {
// Crucially NOT a resolved 200 carrying `undefined` as the alias.
expect(err).toBeInstanceOf(ConflictException);
expect((err as ConflictException).getResponse()).toMatchObject({
code: 'ALIAS_PAGE_RACE',
});
}
});
it('maps a concurrent-delete race in the RENAME branch to a retryable 409 (not a generic 400)', async () => {
const { service, shareAliasRepo } = makeService();
// New slug is free, but the page already owns an alias we rename in place.
shareAliasRepo.findByAliasAndWorkspace.mockResolvedValue(undefined);
shareAliasRepo.findByPageId.mockResolvedValue({
id: 'a-1',
alias: 'te',
pageId: 'p-1',
});
// The row vanished before the UPDATE; repo throws NoResultError rather
// than returning undefined (which would dereference undefined.id -> 400).
shareAliasRepo.updateAlias.mockRejectedValue(new NoResultError({} as any));
try {
await service.setAlias({
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
alias: 'ted',
});
fail('expected ConflictException');
} catch (err) {
expect(err).toBeInstanceOf(ConflictException);
expect(err).not.toBeInstanceOf(BadRequestException);
expect((err as ConflictException).getResponse()).toMatchObject({
code: 'ALIAS_PAGE_RACE',
});
}
});
it('maps a non-unique-violation db error to BadRequest (Failed to set alias)', async () => {
const { service, shareAliasRepo } = makeService();
shareAliasRepo.findByAliasAndWorkspace.mockResolvedValue(undefined);

View File

@@ -11,21 +11,21 @@ import { Page, ShareAlias } from '@docmost/db/types/entity.types';
import { isValidShareAlias, normalizeShareAlias } from './share-alias.util';
import { InjectKysely } from 'nestjs-kysely';
import { KyselyDB } from '@docmost/db/types/kysely.types';
import { executeTx } from '@docmost/db/utils';
/** Postgres unique_violation. Two unique indexes can raise it on this table. */
const PG_UNIQUE_VIOLATION = '23505';
import {
executeTx,
isUniqueViolation,
violatedConstraint,
} from '@docmost/db/utils';
import { NoResultError } from 'kysely';
/**
* Unique index names from the share_aliases migrations. The `postgres@3.x`
* driver (kysely-postgres-js) surfaces the violated constraint as
* `err.constraint_name` (NOT `.constraint`); we keep `.constraint` only as a
* defensive fallback for other drivers.
* - ALIAS: `(workspace_id, alias)` -> the vanity NAME is taken.
* Unique index name from the share_aliases migrations whose violation we map to
* a DISTINCT, non-misleading outcome:
* - PAGE_ID: partial `(workspace_id, page_id) WHERE page_id IS NOT NULL`
* -> a concurrent writer already gave THIS page an alias.
* The `(workspace_id, alias)` index (the vanity NAME being taken) needs no
* constant: it is the default "Alias already taken" mapping.
*/
const UNIQUE_ALIAS_INDEX = 'share_aliases_workspace_id_alias_unique';
const UNIQUE_PAGE_ID_INDEX = 'share_aliases_workspace_id_page_id_unique';
export interface ResolvedAliasTarget {
@@ -171,11 +171,23 @@ export class ShareAliasService {
) {
throw err;
}
// The row we read was deleted (concurrent `removeAlias`) before our UPDATE
// matched it, so `executeTakeFirstOrThrow` found no row. Surface a
// retryable conflict instead of a 200-without-alias (swap branch) or a
// generic 400 from dereferencing `undefined.id` (rename branch).
if (err instanceof NoResultError) {
this.logger.warn(
'share alias update matched no row (concurrent-delete race)',
);
throw new ConflictException({
message: 'The address changed concurrently, please retry',
code: 'ALIAS_PAGE_RACE',
});
}
// A unique index fired. Which one decides the message — always log the
// constraint so the race is diagnosable.
if (err?.code === PG_UNIQUE_VIOLATION) {
const constraint: string | undefined =
err?.constraint_name ?? err?.constraint;
if (isUniqueViolation(err)) {
const constraint = violatedConstraint(err);
this.logger.warn(
`share alias unique violation on ${constraint ?? '<unknown>'}`,
);
@@ -189,13 +201,8 @@ export class ShareAliasService {
code: 'ALIAS_PAGE_RACE',
});
}
// `(workspace_id, alias)` (UNIQUE_ALIAS_INDEX) or any other/unknown
// unique index: treat as the vanity name being claimed first.
if (constraint && constraint !== UNIQUE_ALIAS_INDEX) {
this.logger.warn(
`unexpected unique index ${constraint} mapped to "Alias already taken"`,
);
}
// `(workspace_id, alias)` or any other/unknown unique index: treat as
// the vanity name being claimed first.
throw new ConflictException({ message: 'Alias already taken' });
}
this.logger.error(err);

View File

@@ -21,6 +21,41 @@ export interface TreeNodeSnapshot {
position: string;
spaceId: string;
parentPageId: string | null;
// Death-timer deadline carried so the `addTreeNode` broadcast shows the
// temporary-note clock marker immediately on every client (incl. the author,
// whose optimistic insert can lose the race to this broadcast). null/absent =>
// permanent.
temporaryExpiresAt?: Date | string | null;
}
/**
* Single canonical builder for a `TreeNodeSnapshot` from a page-like row. Both
* the `PAGE_CREATED` event enrichment (`page.repo.insertPage`) and the
* `addTreeNode` broadcast (`WsTreeService.broadcastPageCreated`) build this same
* snapshot; routing both through here keeps the optional `temporaryExpiresAt`
* (and the `?? null` normalisation that pins a permanent note to an explicit
* null) from silently drifting between the two literals.
*/
export function toTreeNodeSnapshot(page: {
id: string;
slugId: string;
title: string | null;
icon: string | null;
position: string;
spaceId: string;
parentPageId: string | null;
temporaryExpiresAt?: Date | string | null;
}): TreeNodeSnapshot {
return {
id: page.id,
slugId: page.slugId,
title: page.title,
icon: page.icon,
position: page.position,
spaceId: page.spaceId,
parentPageId: page.parentPageId,
temporaryExpiresAt: page.temporaryExpiresAt ?? null,
};
}
export class PageEvent {

View File

@@ -0,0 +1,19 @@
import { type Kysely } from 'kysely';
export async function up(db: Kysely<any>): Promise<void> {
// `source` links an imported role back to its catalog origin
// `{ slug, language, version }`. Nullable: null => a manually-created role
// (no catalog provenance). The version lets the admin UI offer an UPDATE when
// the catalog ships a newer revision of the same slug.
await db.schema
.alterTable('ai_agent_roles')
.addColumn('source', 'jsonb', (col) => col)
.execute();
}
export async function down(db: Kysely<any>): Promise<void> {
await db.schema
.alterTable('ai_agent_roles')
.dropColumn('source')
.execute();
}

View File

@@ -0,0 +1,31 @@
import { type Kysely, sql } from 'kysely';
export async function up(db: Kysely<any>): Promise<void> {
// A catalog-imported role is uniquely identified within a workspace by its
// `source.slug` + `source.language` (a multilingual catalog: the `ru` variant
// of a slug installed as `en` is a SEPARATE install — hence both keys). The
// import path skips a slug+language already installed using an in-memory
// snapshot (installedKeys), but two CONCURRENT imports of the same bundle each
// read a stale snapshot and would both insert the same slug+language,
// duplicating the role. This partial unique index is the database-level
// backstop: the second insert gets a 23505 the service treats as
// "already installed" (skip), so the two imports converge on ONE role.
//
// Partial on `source IS NOT NULL` so MANUALLY-created roles (source NULL) are
// unconstrained — there can be many of those. Also partial on
// `deleted_at IS NULL` (like the existing name-unique index) so a soft-deleted
// role does not block re-importing the same slug+language later, matching the
// app's snapshot (listByWorkspace filters out soft-deleted rows).
await sql`
CREATE UNIQUE INDEX IF NOT EXISTS ai_agent_roles_workspace_source_unique
ON ai_agent_roles (workspace_id, (source ->> 'slug'), (source ->> 'language'))
WHERE source IS NOT NULL AND deleted_at IS NULL
`.execute(db);
}
export async function down(db: Kysely<any>): Promise<void> {
await db.schema
.dropIndex('ai_agent_roles_workspace_source_unique')
.ifExists()
.execute();
}

View File

@@ -1,4 +1,4 @@
import { AiAgentRoleRepo } from './ai-agent-roles.repo';
import { AiAgentRoleRepo, parseSource } from './ai-agent-roles.repo';
import type { KyselyDB } from '../../types/kysely.types';
/**
@@ -132,4 +132,77 @@ describe('AiAgentRoleRepo insert/update auto-start columns', () => {
expect(set2.mock.calls[0][0].launchMessage).toBeNull();
expect('autoStart' in set2.mock.calls[0][0]).toBe(false);
});
it('insert binds `source` (jsonb); update sets it only when present', async () => {
const { repo, values } = makeInsertRepo();
await repo.insert({
workspaceId: 'ws-1',
name: 'R',
instructions: 'do',
source: { slug: 'researcher', language: 'en', version: 1 },
});
// jsonbBind returns a RawBuilder for a non-empty object (not null).
expect(values.mock.calls[0][0].source).not.toBeNull();
const { repo: repo2, set } = makeUpdateRepo();
await repo2.update('r-1', 'ws-1', { name: 'X' });
expect('source' in set.mock.calls[0][0]).toBe(false);
const { repo: repo3, set: set3 } = makeUpdateRepo();
await repo3.update('r-1', 'ws-1', {
source: { slug: 's', language: 'en', version: 2 },
});
expect('source' in set3.mock.calls[0][0]).toBe(true);
});
});
/**
* parseSource is THE single form validator for the `source` jsonb column: a
* JSON-string (legacy double-encoded) is parsed; a FULLY-VALID object
* ({ slug, language, version }) passes through as a typed RoleSource; anything
* partial or wrong-shaped degrades to null (= manual role). This is the
* stricter-than-before guard that closes the drift where a weak `{}`/`{slug:123}`
* value used to be stamped as a valid source by the read path.
*/
describe('parseSource', () => {
it('parses a legacy double-encoded JSON string into the typed source', () => {
expect(
parseSource('{"slug":"researcher","language":"en","version":1}'),
).toEqual({ slug: 'researcher', language: 'en', version: 1 });
});
it('passes a fully-valid already-parsed object through', () => {
const obj = { slug: 's', language: 'en', version: 2 };
expect(parseSource(obj)).toEqual(obj);
});
it('returns the typed RoleSource (extra keys tolerated) for a valid shape', () => {
const src = parseSource({ slug: 's', language: 'ru', version: 3 });
expect(src).not.toBeNull();
// Narrowed to RoleSource: the fields are present and correctly typed.
expect(src?.slug).toBe('s');
expect(src?.language).toBe('ru');
expect(src?.version).toBe(3);
});
it('null / array / non-object / unparseable string => null', () => {
expect(parseSource(null)).toBeNull();
expect(parseSource([1, 2])).toBeNull();
expect(parseSource(42)).toBeNull();
expect(parseSource('not json')).toBeNull();
});
it('partial / wrong-typed shapes => null (no weak-but-typed-as-valid drift)', () => {
// Empty object: no slug/language/version.
expect(parseSource({})).toBeNull();
// slug present but not a string.
expect(parseSource({ slug: 123, language: 'en', version: 1 })).toBeNull();
// slug only, missing language + version.
expect(parseSource({ slug: 'a' })).toBeNull();
// empty-string slug / language are not valid catalog keys.
expect(parseSource({ slug: '', language: 'en', version: 1 })).toBeNull();
expect(parseSource({ slug: 'a', language: '', version: 1 })).toBeNull();
// version must be a number, not a numeric string.
expect(parseSource({ slug: 'a', language: 'en', version: '1' })).toBeNull();
});
});

View File

@@ -2,7 +2,7 @@ import { Injectable } from '@nestjs/common';
import { InjectKysely } from 'nestjs-kysely';
import { KyselyDB, KyselyTransaction } from '../../types/kysely.types';
import { dbOrTx, jsonbBind, parseJsonbValue } from '../../utils';
import { AiAgentRole } from '@docmost/db/types/entity.types';
import { AiAgentRole, RoleSource } from '@docmost/db/types/entity.types';
/** The jsonb shape persisted in `model_config` (loosely typed for the column). */
type ModelConfigValue = Record<string, unknown> | null;
@@ -81,6 +81,8 @@ export class AiAgentRoleRepo {
autoStart?: boolean;
// null/'' => stored as null (client default launch message).
launchMessage?: string | null;
// Catalog origin { slug, language, version } | null. null => manual role.
source?: Record<string, unknown> | null;
},
trx?: KyselyTransaction,
): Promise<AiAgentRole> {
@@ -103,6 +105,9 @@ export class AiAgentRoleRepo {
autoStart: values.autoStart ?? true,
// Empty string is treated as "no custom text" => null.
launchMessage: values.launchMessage || null,
// Same cast reason as modelConfig (see above).
// eslint-disable-next-line @typescript-eslint/no-explicit-any
source: jsonbBind(values.source) as any,
})
.returningAll()
.executeTakeFirst();
@@ -124,6 +129,8 @@ export class AiAgentRoleRepo {
autoStart?: boolean;
// undefined => unchanged; null/'' => clear to null; string => set.
launchMessage?: string | null;
// undefined => unchanged; null => clear; object => set.
source?: Record<string, unknown> | null;
},
trx?: KyselyTransaction,
): Promise<void> {
@@ -142,6 +149,9 @@ export class AiAgentRoleRepo {
// Empty string clears to null (client default launch message).
set.launchMessage = patch.launchMessage || null;
}
if (patch.source !== undefined) {
set.source = jsonbBind(patch.source);
}
await db
.updateTable('aiAgentRoles')
.set(set)
@@ -192,14 +202,46 @@ export function parseModelConfig(
);
}
/** Normalize a DB row so `modelConfig` is always an object or null. The cast
* bridges parseModelConfig's concrete `Record | null` to the column's broad
* generated `JsonValue` type (an object is a valid JsonValue at runtime). */
/**
* THE single form validator for the `source` jsonb column: parse the value read
* from the DB into a fully-valid {@link RoleSource} or null. Same legacy
* double-encoding self-heal as {@link parseModelConfig} (a JSON string is parsed
* once), then validates the FULL shape — `slug` and `language` non-empty
* strings, `version` a number. A null / corrupt / partially-shaped value (e.g.
* `{}`, `{ slug: 123 }`, `{ slug: 'a' }` missing language/version) degrades to
* null (= manually created, no catalog provenance), so a bad row never breaks
* the read path AND never stamps a half-built object as a valid `RoleSource`.
* Both the repo read-path and the service share this so the contract cannot
* drift between layers.
*/
export function parseSource(value: unknown): RoleSource | null {
return parseJsonbValue(value, isRoleSource);
}
/** Full-shape guard for a persisted `source` jsonb value (see parseSource). */
function isRoleSource(v: unknown): v is RoleSource {
if (v === null || typeof v !== 'object' || Array.isArray(v)) return false;
const obj = v as Record<string, unknown>;
return (
typeof obj.slug === 'string' &&
obj.slug.length > 0 &&
typeof obj.language === 'string' &&
obj.language.length > 0 &&
typeof obj.version === 'number'
);
}
/** Normalize a DB row so `modelConfig` and `source` are always a valid object or
* null. The casts bridge the concrete parsed types (`Record | null`,
* `RoleSource | null`) to the column's broad generated `JsonValue` type — both
* are valid JsonValues at runtime; RoleSource lacks the JsonObject index
* signature so it routes through `unknown`. */
function normalizeRow(row: AiAgentRole): AiAgentRole {
return {
...row,
modelConfig: parseModelConfig(
row.modelConfig,
) as AiAgentRole['modelConfig'],
source: parseSource(row.source) as unknown as AiAgentRole['source'],
};
}

View File

@@ -0,0 +1,85 @@
import { AiChatRepo } from './ai-chat.repo';
import type { KyselyDB } from '../../types/kysely.types';
/**
* Unit test for AiChatRepo.findLatestByPage — the "bound chat" resolver behind
* #191 (auto-open the last chat created on a document). It builds the scoping
* query, so we assert the EXACT predicates/ordering the spec mandates over a
* chainable builder mock (no live DB): user + workspace + page scope, the
* deletedAt filter, newest-by-createdAt with an id tiebreaker, limit 1. A
* live-Postgres ordering test is out of scope for this pure unit test.
*/
describe('AiChatRepo.findLatestByPage', () => {
type Recorded = {
table?: string;
wheres: Array<[string, string, unknown]>;
orderBys: Array<[string, string]>;
limit?: number;
};
function makeDb(result: unknown): { db: KyselyDB; rec: Recorded } {
const rec: Recorded = { wheres: [], orderBys: [] };
const builder: Record<string, unknown> = {};
const chain = () => builder;
builder.selectAll = chain;
builder.where = (col: string, op: string, val: unknown) => {
rec.wheres.push([col, op, val]);
return builder;
};
builder.orderBy = (col: string, dir: string) => {
rec.orderBys.push([col, dir]);
return builder;
};
builder.limit = (n: number) => {
rec.limit = n;
return builder;
};
builder.executeTakeFirst = () => Promise.resolve(result);
const db = {
selectFrom: (table: string) => {
rec.table = table;
return builder;
},
} as unknown as KyselyDB;
return { db, rec };
}
it('returns the matched chat and scopes by user + workspace + page (deletedAt null)', async () => {
const chat = { id: 'c1', creatorId: 'u1', workspaceId: 'ws1', pageId: 'p1' };
const { db, rec } = makeDb(chat);
const repo = new AiChatRepo(db);
const res = await repo.findLatestByPage('u1', 'ws1', 'p1');
expect(res).toBe(chat);
expect(rec.table).toBe('aiChats');
expect(rec.wheres).toEqual(
expect.arrayContaining([
['creatorId', '=', 'u1'],
['workspaceId', '=', 'ws1'],
['pageId', '=', 'p1'],
['deletedAt', 'is', null],
]),
);
});
it('orders newest-first by createdAt then id, limit 1', async () => {
const { db, rec } = makeDb(undefined);
const repo = new AiChatRepo(db);
await repo.findLatestByPage('u1', 'ws1', 'p1');
expect(rec.orderBys).toEqual([
['createdAt', 'desc'],
['id', 'desc'],
]);
expect(rec.limit).toBe(1);
});
it('returns undefined when the page has no owned chat', async () => {
const { db } = makeDb(undefined);
const repo = new AiChatRepo(db);
await expect(repo.findLatestByPage('u1', 'ws1', 'p1')).resolves.toBeUndefined();
});
});

View File

@@ -80,6 +80,32 @@ export class AiChatRepo {
});
}
/**
* The "bound chat" for a document: the requesting user's most recently
* created, non-deleted chat whose origin page is `pageId`. Auto-opened when
* the AI chat window is opened on that page. Newest-by-createdAt wins, so a
* chat created later on the same page supersedes earlier ones — exactly how
* "new chat -> becomes the bound one" falls out for free. Scoped to the user +
* workspace, so a foreign pageId can only ever match the caller's own chats.
*/
async findLatestByPage(
creatorId: string,
workspaceId: string,
pageId: string,
): Promise<AiChat | undefined> {
return this.db
.selectFrom('aiChats')
.selectAll('aiChats')
.where('creatorId', '=', creatorId)
.where('workspaceId', '=', workspaceId)
.where('pageId', '=', pageId)
.where('deletedAt', 'is', null)
.orderBy('createdAt', 'desc')
.orderBy('id', 'desc') // stable tiebreaker, mirrors findByCreator's cursor
.limit(1)
.executeTakeFirst();
}
async insert(
insertable: InsertableAiChat,
trx?: KyselyTransaction,

View File

@@ -7,7 +7,7 @@ import { executeWithCursorPagination } from '@docmost/db/pagination/cursor-pagin
import { jsonObjectFrom } from 'kysely/helpers/postgres';
import { ExpressionBuilder, SelectQueryBuilder, sql } from 'kysely';
import { DB } from '@docmost/db/types/db';
import { dbOrTx } from '@docmost/db/utils';
import { dbOrTx, isUniqueViolation } from '@docmost/db/utils';
export const FavoriteType = {
PAGE: 'page',
@@ -29,7 +29,8 @@ export class FavoriteRepo {
.returningAll()
.executeTakeFirst();
} catch (err: any) {
if (err?.code === '23505') return undefined;
// Idempotent favorite: a duplicate (already-favorited) is not an error.
if (isUniqueViolation(err)) return undefined;
throw err;
}
}

View File

@@ -16,7 +16,10 @@ import { jsonArrayFrom, jsonObjectFrom } from 'kysely/helpers/postgres';
import { SpaceMemberRepo } from '@docmost/db/repos/space/space-member.repo';
import { EventEmitter2 } from '@nestjs/event-emitter';
import { EventName } from '../../../common/events/event.contants';
import { TreeUpdateSnapshot } from '../../listeners/page.listener';
import {
TreeUpdateSnapshot,
toTreeNodeSnapshot,
} from '../../listeners/page.listener';
/**
* Optional extras for the PAGE_UPDATED event emitted by updatePage(s). Lets the
@@ -200,17 +203,10 @@ export class PageRepo {
this.eventEmitter.emit(EventName.PAGE_CREATED, {
pageIds: [result.id],
workspaceId: result.workspaceId,
pages: [
{
id: result.id,
slugId: result.slugId,
title: result.title,
icon: result.icon,
position: result.position,
spaceId: result.spaceId,
parentPageId: result.parentPageId,
},
],
// Built via the shared snapshot helper so the field copy (and the
// death-timer deadline that shows the sidebar clock marker without a
// reload) can't drift from the `addTreeNode` broadcast literal.
pages: [toTreeNodeSnapshot(result)],
});
return result;

View File

@@ -94,7 +94,9 @@ describe('ShareAliasRepo', () => {
return builder;
}),
returning: jest.fn(() => builder),
executeTakeFirst: jest.fn().mockResolvedValue({ id: 'a-1' }),
// Retarget uses executeTakeFirstOrThrow so a row reaped by a concurrent
// delete (0 rows matched) raises NoResultError instead of returning undefined.
executeTakeFirstOrThrow: jest.fn().mockResolvedValue({ id: 'a-1' }),
};
const db = { updateTable: jest.fn(() => builder) } as unknown as KyselyDB;
const repo = new ShareAliasRepo(db);
@@ -121,7 +123,11 @@ describe('ShareAliasRepo', () => {
return builder;
}),
returning: jest.fn(() => builder),
executeTakeFirst: jest.fn().mockResolvedValue({ id: 'a-1', alias: 'ted' }),
// Rename uses executeTakeFirstOrThrow so a row reaped by a concurrent
// delete (0 rows matched) raises NoResultError instead of returning undefined.
executeTakeFirstOrThrow: jest
.fn()
.mockResolvedValue({ id: 'a-1', alias: 'ted' }),
};
const db = { updateTable: jest.fn(() => builder) } as unknown as KyselyDB;
const repo = new ShareAliasRepo(db);

View File

@@ -92,6 +92,12 @@ export class ShareAliasRepo {
* Rename an existing alias row in place (the vanity-slug edit, e.g.
* `te` -> `ted`). Keeps the row's id/page_id/creator so the page's single
* alias pointer is preserved — only the human-readable name changes.
*
* Uses `executeTakeFirstOrThrow`: if a concurrent `delete` reaps this row
* between the service's read and this UPDATE (READ COMMITTED), the UPDATE
* matches 0 rows and kysely throws `NoResultError` rather than returning
* `undefined` for a `Promise<ShareAlias>`. The service maps that to a
* retryable conflict instead of dereferencing `undefined.id`.
*/
async updateAlias(
id: string,
@@ -105,7 +111,7 @@ export class ShareAliasRepo {
.where('id', '=', id)
.where('workspaceId', '=', workspaceId)
.returning(this.baseFields)
.executeTakeFirst();
.executeTakeFirstOrThrow();
}
/**
@@ -127,7 +133,15 @@ export class ShareAliasRepo {
.execute();
}
/** Retarget an existing alias to a new page (the "swap" operation). */
/**
* Retarget an existing alias to a new page (the "swap" operation).
*
* Uses `executeTakeFirstOrThrow`: if a concurrent `delete` reaps this row
* between the service's read and this UPDATE, the UPDATE matches 0 rows and
* kysely throws `NoResultError` instead of returning `undefined` into the 200
* response (a "success" with no alias). The service maps that to a retryable
* conflict.
*/
async updatePageId(
id: string,
pageId: string,
@@ -140,7 +154,7 @@ export class ShareAliasRepo {
.where('id', '=', id)
.where('workspaceId', '=', workspaceId)
.returning(this.baseFields)
.executeTakeFirst();
.executeTakeFirstOrThrow();
}
async delete(

View File

@@ -618,6 +618,8 @@ export interface AiAgentRoles {
autoStart: Generated<boolean>;
// Optional custom auto-start text. null/empty => client default launch message.
launchMessage: string | null;
// Catalog origin of an imported role: { slug, language, version } | null. null => manually created.
source: Json | null;
createdAt: Generated<Timestamp>;
updatedAt: Generated<Timestamp>;
deletedAt: Timestamp | null;

View File

@@ -81,6 +81,24 @@ export type UpdatableAiMcpServer = Updateable<Omit<AiMcpServersTable, 'id'>>;
// A role replaces the persona layer of the system prompt (instructions) and may
// optionally override the chat model (`modelConfig`). Soft-deletable.
export type AiAgentRole = Selectable<AiAgentRoles>;
/**
* The validated shape of the `source` jsonb column on ai_agent_roles: the
* catalog origin of an imported role. `version` lets the admin UI offer an
* UPDATE when the catalog ships a newer revision of the same slug; null `source`
* (not this type) means a manually-created role with no catalog provenance.
*
* THE single contract for that column, shared by the repo read-path
* (`parseSource`, the only form validator) and the service, so the persisted
* shape can never be validated weakly in one layer and strongly in another.
* Defined here (a leaf db-types module both already import `AiAgentRole` from) to
* avoid an import cycle between the repo and the service.
*/
export interface RoleSource {
slug: string;
language: string;
version: number;
}
export type InsertableAiAgentRole = Insertable<AiAgentRoles>;
export type UpdatableAiAgentRole = Updateable<Omit<AiAgentRoles, 'id'>>;

View File

@@ -0,0 +1,51 @@
import { isUniqueViolation, violatedConstraint } from './utils';
/**
* Unit tests for the driver-bound Postgres unique-violation helpers extracted
* from the share-alias service (and now shared with favorite.repo). They encode
* two `kysely-postgres-js` / `postgres@3.x` quirks: the SQLSTATE is the string
* `'23505'`, and the violated index name arrives as `constraint_name` (with
* `constraint` only a fallback for other drivers).
*/
describe('isUniqueViolation', () => {
it('is true for a 23505 error', () => {
expect(isUniqueViolation({ code: '23505' })).toBe(true);
});
it('is false for any other code', () => {
expect(isUniqueViolation({ code: '08006' })).toBe(false);
});
it('is false when there is no code / not an object', () => {
expect(isUniqueViolation({})).toBe(false);
expect(isUniqueViolation(null)).toBe(false);
expect(isUniqueViolation(undefined)).toBe(false);
expect(isUniqueViolation(new Error('boom'))).toBe(false);
});
});
describe('violatedConstraint', () => {
it('reads the postgres@3.x `constraint_name` field', () => {
expect(
violatedConstraint({ code: '23505', constraint_name: 'idx_a' }),
).toBe('idx_a');
});
it('falls back to `constraint` when `constraint_name` is absent', () => {
expect(violatedConstraint({ code: '23505', constraint: 'idx_b' })).toBe(
'idx_b',
);
});
it('prefers `constraint_name` over `constraint` when both are present', () => {
expect(
violatedConstraint({ constraint_name: 'idx_a', constraint: 'idx_b' }),
).toBe('idx_a');
});
it('is undefined when neither field is present', () => {
expect(violatedConstraint({ code: '23505' })).toBeUndefined();
expect(violatedConstraint(null)).toBeUndefined();
expect(violatedConstraint(undefined)).toBeUndefined();
});
});

View File

@@ -33,6 +33,35 @@ export function dbOrTx(
}
}
/** Postgres `unique_violation` SQLSTATE — raised when a write hits a UNIQUE index. */
const PG_UNIQUE_VIOLATION = '23505';
/**
* Whether `err` is a Postgres unique-violation (SQLSTATE `23505`). THE single
* check so repos/services stop re-hardcoding the magic code.
*
* NOTE (#222): `core/ai-chat/roles/ai-agent-roles.service.ts` still carries its
* own inline `23505` check on a separate, unmerged branch; it should adopt this
* helper (and {@link violatedConstraint}) after #227 lands.
*/
export function isUniqueViolation(err: unknown): boolean {
return (err as { code?: unknown } | null | undefined)?.code === PG_UNIQUE_VIOLATION;
}
/**
* The name of the UNIQUE index/constraint a `23505` error violated, or
* undefined. The `kysely-postgres-js` / `postgres@3.x` driver surfaces it as
* `err.constraint_name` (NOT `.constraint`); `.constraint` is kept only as a
* defensive fallback for other drivers.
*/
export function violatedConstraint(err: unknown): string | undefined {
const e = err as
| { constraint_name?: string; constraint?: string }
| null
| undefined;
return e?.constraint_name ?? e?.constraint;
}
/**
* Bind a JS array/object as a `jsonb` column value, working around a postgres
* driver double-encoding quirk. THE single implementation — repos that persist

View File

@@ -289,6 +289,18 @@ export class EnvironmentService {
// provider/model/key config now lives solely in workspace settings +
// ai_provider_credentials, with no env fallback. APP_SECRET stays (getAppSecret).
getAiAgentRolesCatalogSource(): string {
// Catalog location: an http(s):// base URL the catalog is fetched from.
// The image ships a per-branch default for this baked in at build time
// (Dockerfile ARG AI_AGENT_ROLES_CATALOG_URL, set per-branch in CI), but it
// is overridable at runtime via the env var (this getter returns that
// runtime value). Local-filesystem sources are no longer supported.
// Empty/unset => the catalog is unavailable (the provider returns 502).
// This is INFRA config (where the catalog lives), not provider/model
// config, so an env var is appropriate.
return this.configService.get<string>('AI_AGENT_ROLES_CATALOG_URL', '');
}
getEventStoreDriver(): string {
return this.configService
.get<string>('EVENT_STORE_DRIVER', 'postgres')

View File

@@ -0,0 +1,150 @@
// Importing FileImportTaskService transitively loads import-formatter.ts, which
// imports the ESM-only @sindresorhus/slugify package (not in jest's transform
// allowlist). slugify is irrelevant to the path under test, so it is mocked out
// to keep the module graph loadable under ts-jest (mirrors the import.service spec).
jest.mock('@sindresorhus/slugify', () => ({
__esModule: true,
default: (input: string) => String(input),
}));
// import-attachment.service.ts (loaded transitively for DI typing) imports the
// ESM-only `p-limit` / `image-dimensions`; neither is exercised on the path under
// test, so stub them so the module graph loads under ts-jest.
jest.mock('p-limit', () => ({
__esModule: true,
default: () => (fn: any) => fn(),
}));
jest.mock('image-dimensions', () => ({
__esModule: true,
imageDimensionsFromData: () => undefined,
}));
import { promises as fs } from 'fs';
import * as os from 'os';
import * as path from 'path';
import { FileImportTaskService } from './file-import-task.service';
import { ImportService } from './import.service';
/**
* Binding test for issue #228 / review #5: FileImportTaskService.processGenericImport
* is a NON-editor write path (markdownToHtml -> processHTML -> JSON, never runs
* footnoteSyncPlugin), so it canonicalizes footnotes before persisting. This pins
* that binding — the same one import.service has a spec for — which previously had
* NO spec at all.
*
* The markdown -> HTML -> ProseMirror conversion is REAL (a real ImportService,
* its createYdoc stubbed); the filesystem is a real temp dir with one .md file;
* the DB transaction is stubbed to capture the persisted page content.
*/
// Out-of-order references (c, a, b), a REUSED reference ([^a] twice), and an
// ORPHAN definition ([^z], never referenced).
const MARKDOWN = [
'# Title',
'',
'Body refs [^c] and [^a] and [^b] and again [^a].',
'',
'[^a]: note A',
'[^b]: note B',
'[^c]: note C',
'[^z]: orphan note',
].join('\n');
function footnoteListIds(content: any): string[] {
const list = (content?.content ?? []).find(
(n: any) => n.type === 'footnotesList',
);
return (list?.content ?? [])
.filter((n: any) => n.type === 'footnoteDefinition')
.map((n: any) => n.attrs?.id);
}
// A permissive chainable stub for the spaces lookup (selectFrom(...).select(...)
// .where(...).executeTakeFirst()).
function chainable(result: any): any {
const proxy: any = new Proxy(function () {}, {
get: (_t, prop) => {
if (prop === 'executeTakeFirst') return async () => result;
if (prop === 'execute') return async () => [];
return () => proxy;
},
});
return proxy;
}
describe('FileImportTaskService.processGenericImport — footnote canonicalization (#228)', () => {
it('orders footnotes by first reference, dedupes reuse, and drops orphans on zip import', async () => {
const extractDir = await fs.mkdtemp(path.join(os.tmpdir(), 'fit-canon-'));
await fs.writeFile(path.join(extractDir, 'note.md'), MARKDOWN, 'utf-8');
// Real ImportService for the html -> JSON conversion; stub the yjs encode.
const importService = new ImportService(
{} as any,
{} as any,
{} as any,
{} as any,
);
jest
.spyOn(importService as any, 'createYdoc')
.mockResolvedValue(Buffer.from([]) as any);
let captured: any = null;
const trx = {
insertInto: (table: string) => ({
values: (v: any) => {
if (table === 'pages') captured = v;
return { execute: async () => {} };
},
}),
};
const db: any = {
selectFrom: () => chainable({ slug: 'space-slug' }),
transaction: () => ({ execute: (fn: any) => fn(trx) }),
};
const importAttachmentService = {
processAttachments: async ({ html }: any) => html,
};
const backlinkRepo = { insertBacklink: jest.fn() };
const eventEmitter = { emit: jest.fn() };
const auditService = { logBatchWithContext: jest.fn() };
const pageService = { nextPagePosition: async () => 'a0' };
const service = new FileImportTaskService(
{} as any, // storageService
importService as any,
pageService as any,
backlinkRepo as any,
db,
importAttachmentService as any,
eventEmitter as any,
auditService as any,
);
const fileTask: any = {
id: 'task-1',
source: 'generic',
spaceId: 'space-1',
workspaceId: 'ws-1',
creatorId: 'user-1',
};
try {
await service.processGenericImport({ extractDir, fileTask });
expect(captured).toBeTruthy();
const content = captured.content;
// Reference order is c, a, b (NOT the markdown definition order a, b, c).
expect(footnoteListIds(content)).toEqual(['c', 'a', 'b']);
// Orphan [^z] dropped; reused [^a] collapses to one definition; one list.
expect(footnoteListIds(content)).not.toContain('z');
const lists = (content.content ?? []).filter(
(n: any) => n.type === 'footnotesList',
);
expect(lists).toHaveLength(1);
expect(footnoteListIds(content).filter((id) => id === 'a')).toHaveLength(1);
} finally {
await fs.rm(extractDir, { recursive: true, force: true });
}
});
});

View File

@@ -18,7 +18,7 @@ import { generateSlugId } from '../../../common/helpers';
import { v7 } from 'uuid';
import { generateJitteredKeyBetween } from 'fractional-indexing-jittered';
import { FileTask, InsertablePage } from '@docmost/db/types/entity.types';
import { markdownToHtml } from '@docmost/editor-ext';
import { markdownToHtml, canonicalizeFootnotes } from '@docmost/editor-ext';
import { getProsemirrorContent } from '../../../common/helpers/prosemirror/utils';
import { formatImportHtml } from '../utils/import-formatter';
import {
@@ -496,9 +496,19 @@ export class FileImportTaskService {
await this.importService.processHTML(html),
);
const { title, prosemirrorJson } =
const { title, prosemirrorJson: extractedJson } =
this.importService.extractTitleAndRemoveHeading(pmState);
// Canonicalize footnote topology on this non-editor write path
// (markdownToHtml/processHTML never runs footnoteSyncPlugin), so a
// zip-imported page's footnotes are reference-ordered, deduped, and
// orphan-free like the editor's invariant (issue #228). Pure +
// idempotent + shape-safe; a footnote-free doc is unchanged.
// (Future consolidation, architecture B: like import.service, this
// path persists directly rather than via PageService — a shared
// "prepare JSON for persist" helper would centralize this call.)
const prosemirrorJson = canonicalizeFootnotes(extractedJson);
const insertablePage: InsertablePage = {
id: page.id,
slugId: page.slugId,

View File

@@ -0,0 +1,139 @@
// Importing ImportService transitively loads import-formatter.ts, which imports
// the ESM-only @sindresorhus/slugify package (not in jest's transform
// allowlist). slugify is irrelevant to the path under test, so it is mocked out
// to keep the module graph loadable under ts-jest.
jest.mock('@sindresorhus/slugify', () => ({
__esModule: true,
default: (input: string) => String(input),
}));
import { ImportService } from './import.service';
import { canonicalizeFootnotes } from '@docmost/editor-ext';
/**
* Integration-ish test for the USER-FACING markdown import path
* (`ImportService.importPage`). It exercises the REAL markdown -> HTML -> JSON
* conversion and asserts that the stored page content has its footnotes
* canonicalized — the gap that issue #228 fixes: the import path builds
* ProseMirror JSON directly (never running the editor's footnoteSyncPlugin), so
* before this wiring the stored footnotes kept the markdown's physical
* definition order (out of order vs. references), retained orphan definitions,
* and did not collapse reused references.
*
* The DB/ydoc side-effects are stubbed: `getNewPagePosition` (DB query) and
* `createYdoc` (Yjs encode) are spied, and `pageRepo.insertPage` captures the
* persisted `content`. Everything between markdown and persistence is REAL.
*/
// Out-of-order references (c, a, b), a REUSED reference ([^a] twice -> one
// footnote), and an ORPHAN definition ([^z], never referenced).
const MARKDOWN = [
'# Title',
'',
'Body refs [^c] and [^a] and [^b] and again [^a].',
'',
'[^a]: note A',
'[^b]: note B',
'[^c]: note C',
'[^z]: orphan note',
].join('\n');
function makeFile(filename: string, contents: string) {
return {
filename,
toBuffer: async () => Buffer.from(contents),
} as any;
}
function makeService() {
let captured: any = null;
const pageRepo = {
insertPage: jest.fn(async (values: any) => {
captured = values;
return { id: 'page-id', slugId: 'slug-id' };
}),
};
const service = new ImportService(
pageRepo as any,
{} as any,
{} as any,
{} as any,
);
jest.spyOn(service as any, 'getNewPagePosition').mockResolvedValue('a0');
jest
.spyOn(service as any, 'createYdoc')
.mockResolvedValue(Buffer.from([]) as any);
return { service, pageRepo, getCaptured: () => captured };
}
/** List the footnote-definition ids of the (single) footnotesList, in order. */
function footnoteListIds(content: any): string[] {
const list = (content.content ?? []).find(
(n: any) => n.type === 'footnotesList',
);
if (!list) return [];
return (list.content ?? [])
.filter((n: any) => n.type === 'footnoteDefinition')
.map((n: any) => n.attrs?.id);
}
function definitionText(content: any, id: string): string | undefined {
const list = (content.content ?? []).find(
(n: any) => n.type === 'footnotesList',
);
const def = (list?.content ?? []).find(
(n: any) => n.type === 'footnoteDefinition' && n.attrs?.id === id,
);
return def?.content?.[0]?.content?.[0]?.text;
}
describe('ImportService.importPage — footnote canonicalization (#228)', () => {
it('orders footnotes by first reference, dedupes reuse, and drops orphans', async () => {
const { service, getCaptured } = makeService();
await service.importPage(
Promise.resolve(makeFile('note.md', MARKDOWN)),
'user-id',
'space-id',
'workspace-id',
);
const content = getCaptured().content;
expect(content).toBeTruthy();
// Reference order is c, a, b (NOT the markdown definition order a, b, c).
expect(footnoteListIds(content)).toEqual(['c', 'a', 'b']);
// Definitions preserved and attached to the right ids.
expect(definitionText(content, 'c')).toBe('note C');
expect(definitionText(content, 'a')).toBe('note A');
expect(definitionText(content, 'b')).toBe('note B');
// Orphan definition [^z] is dropped.
expect(footnoteListIds(content)).not.toContain('z');
// Reused [^a] yields exactly ONE definition, and exactly one list.
const lists = (content.content ?? []).filter(
(n: any) => n.type === 'footnotesList',
);
expect(lists).toHaveLength(1);
expect(footnoteListIds(content).filter((id) => id === 'a')).toHaveLength(1);
});
it('is idempotent: canonicalizing the stored output again is a no-op', async () => {
const { service, getCaptured } = makeService();
await service.importPage(
Promise.resolve(makeFile('note.md', MARKDOWN)),
'user-id',
'space-id',
'workspace-id',
);
const stored = getCaptured().content;
// The stored content is already canonical; running the canonicalizer a second
// time must not change it (safe to wire into every write path).
const second = canonicalizeFootnotes(stored);
expect(second).toEqual(stored);
expect(footnoteListIds(second)).toEqual(['c', 'a', 'b']);
});
});

View File

@@ -17,7 +17,7 @@ import {
import { generateJitteredKeyBetween } from 'fractional-indexing-jittered';
import { TiptapTransformer } from '@hocuspocus/transformer';
import * as Y from 'yjs';
import { markdownToHtml } from '@docmost/editor-ext';
import { markdownToHtml, canonicalizeFootnotes } from '@docmost/editor-ext';
import {
FileTaskStatus,
FileTaskType,
@@ -85,7 +85,17 @@ export class ImportService {
const extracted = this.extractTitleAndRemoveHeading(prosemirrorState);
const title = extracted.title;
const prosemirrorJson = extracted.prosemirrorJson;
// Imported markdown/HTML is built via markdownToHtml -> htmlToJson, which
// never runs the editor's footnoteSyncPlugin, so the footnote topology keeps
// the source's PHYSICAL definition order (out of order vs. references),
// retains orphan definitions, and is not deduped. Canonicalize before
// persisting so the stored page matches the editor's invariant (issue #228).
// Pure + idempotent + shape-safe: a doc with no footnotes is unchanged.
// (Future consolidation, architecture B: this import path persists directly
// via pageRepo.insertPage rather than through PageService.createPage, so the
// canonicalize call lives here; folding both into one "prepare JSON for
// persist" helper is a sensible follow-up.)
const prosemirrorJson = canonicalizeFootnotes(extracted.prosemirrorJson);
const pageTitle = title || fileName;

View File

@@ -83,6 +83,27 @@ describe('WsTreeService', () => {
);
});
it('broadcastPageCreated carries temporaryExpiresAt when the page is a temporary note', async () => {
const expiresAt = new Date('2026-07-01T00:00:00.000Z');
await service.broadcastPageCreated({ ...snapshot, temporaryExpiresAt: expiresAt });
const data =
wsService.emitTreeEvent.mock.calls[0][2].payload.data;
// The death-timer deadline reaches receivers so the clock marker renders
// immediately (incl. the author if this broadcast wins the optimistic race).
expect(data.temporaryExpiresAt).toBe(expiresAt);
});
it('broadcastPageCreated pins temporaryExpiresAt to null for a permanent page', async () => {
// Fixture omits temporaryExpiresAt; the `?? null` must send an explicit null
// (permanent) rather than undefined, so receivers clear any stale marker.
await service.broadcastPageCreated(snapshot);
const data =
wsService.emitTreeEvent.mock.calls[0][2].payload.data;
expect(data.temporaryExpiresAt).toBeNull();
});
it('broadcastPageDeleted emits deleteTreeNode with the root node only', async () => {
await service.broadcastPageDeleted({
...snapshot,

View File

@@ -5,6 +5,7 @@ import {
PageMovedEvent,
TreeNodeSnapshot,
TreeUpdateSnapshot,
toTreeNodeSnapshot,
} from '../database/listeners/page.listener';
@Injectable()
@@ -28,15 +29,16 @@ export class WsTreeService {
// Receivers place by `position` among already-loaded siblings, not by
// this absolute index (sender's loaded set differs from receivers').
index: 0,
// Built via the shared snapshot helper (same one page.repo uses to fill
// the event), then extended with the tree-only fields the client
// receiver consumes. The helper carries the death-timer deadline
// (normalised to null => permanent) so receivers — and the author, if
// this broadcast wins the race against the optimistic insert — render
// the temporary-note clock marker immediately, without it drifting from
// the event literal.
data: {
id: page.id,
slugId: page.slugId,
...toTreeNodeSnapshot(page),
name: page.title ?? '',
title: page.title,
icon: page.icon,
position: page.position,
spaceId: page.spaceId,
parentPageId: page.parentPageId,
hasChildren: false,
children: [],
},

View File

@@ -0,0 +1,371 @@
import { describe, it, expect } from 'vitest';
import { Editor, getSchema } from '@tiptap/core';
import { Document } from '@tiptap/extension-document';
import { Paragraph } from '@tiptap/extension-paragraph';
import { Text } from '@tiptap/extension-text';
import { FootnoteReference } from './footnote-reference';
import { FootnotesList } from './footnotes-list';
import { FootnoteDefinition } from './footnote-definition';
import { canonicalizeFootnotes } from './footnote-canonicalize';
import { FOOTNOTE_CORPUS } from './footnote-corpus';
import {
collectReferenceIds,
computeFootnoteNumbers,
FOOTNOTE_REFERENCE_NAME,
FOOTNOTES_LIST_NAME,
FOOTNOTE_DEFINITION_NAME,
} from './footnote-util';
import { Node as PMNode } from '@tiptap/pm/model';
const extensions = [
Document,
Paragraph,
Text,
FootnoteReference,
FootnotesList,
FootnoteDefinition,
];
const ref = (id: string) => ({ type: FOOTNOTE_REFERENCE_NAME, attrs: { id } });
const def = (id: string, text?: string) => ({
type: FOOTNOTE_DEFINITION_NAME,
attrs: { id },
content: [
text
? { type: 'paragraph', content: [{ type: 'text', text }] }
: { type: 'paragraph' },
],
});
const list = (...defs: any[]) => ({ type: FOOTNOTES_LIST_NAME, content: defs });
const para = (...inline: any[]) => ({ type: 'paragraph', content: inline });
/** Find every node of `type`, document order. */
function findAll(node: any, type: string, acc: any[] = []): any[] {
if (!node || typeof node !== 'object') return acc;
if (node.type === type) acc.push(node);
if (Array.isArray(node.content)) {
for (const c of node.content) findAll(c, type, acc);
}
return acc;
}
/** Physical id order of the definitions in the (single) footnotesList. */
function defOrder(doc: any): string[] {
return findAll(doc, FOOTNOTE_DEFINITION_NAME).map((d) => d.attrs.id);
}
const schema = getSchema(extensions);
/** Reference order (distinct, document order) computed via the shared util. */
function refOrder(doc: any): string[] {
return collectReferenceIds(PMNode.fromJSON(schema, doc));
}
describe('canonicalizeFootnotes (pure JSON)', () => {
it('orders definitions by FIRST reference (out-of-order list -> 1..N)', () => {
// References appear b, a, d, c; the bottom list is in a different (import)
// order. The canonical list must follow reference order so reading it top to
// bottom yields numbers 1..N.
const doc = {
type: 'doc',
content: [
para(
{ type: 'text', text: 'x' },
ref('b'),
ref('a'),
ref('d'),
ref('c'),
),
list(def('a', 'A'), def('c', 'C'), def('b', 'B'), def('d', 'D')),
],
};
const out = canonicalizeFootnotes(doc);
expect(defOrder(out)).toEqual(['b', 'a', 'd', 'c']);
// The physical definition order now matches reference order, so the derived
// numbers (1..N) run sequentially down the list.
expect(refOrder(out)).toEqual(['b', 'a', 'd', 'c']);
const numbers = computeFootnoteNumbers(PMNode.fromJSON(schema, out));
expect(numbers.get('b')).toBe(1);
expect(numbers.get('a')).toBe(2);
expect(numbers.get('d')).toBe(3);
expect(numbers.get('c')).toBe(4);
});
it('numbers run 1..N down the canonical list', () => {
const doc = {
type: 'doc',
content: [
para({ type: 'text', text: 'x' }, ref('b'), ref('a'), ref('c')),
list(def('a', 'A'), def('c', 'C'), def('b', 'B')),
],
};
const out = canonicalizeFootnotes(doc);
// Definition order == reference order == 1,2,3 reading down.
expect(defOrder(out)).toEqual(['b', 'a', 'c']);
});
it('drops an orphan definition (no matching reference)', () => {
const doc = {
type: 'doc',
content: [
para({ type: 'text', text: 'x' }, ref('a')),
list(def('a', 'A'), def('orphan', 'O')),
],
};
const out = canonicalizeFootnotes(doc);
expect(defOrder(out)).toEqual(['a']);
expect(findAll(out, FOOTNOTE_DEFINITION_NAME)).toHaveLength(1);
});
it('with NO references, removes the footnotesList entirely', () => {
const doc = {
type: 'doc',
content: [
para({ type: 'text', text: 'plain' }),
list(def('orphan', 'O')),
],
};
const out = canonicalizeFootnotes(doc);
expect(findAll(out, FOOTNOTES_LIST_NAME)).toHaveLength(0);
expect(findAll(out, FOOTNOTE_DEFINITION_NAME)).toHaveLength(0);
});
it('reuse: repeated references collapse to ONE definition/number', () => {
const doc = {
type: 'doc',
content: [
para(ref('d'), { type: 'text', text: ' a ' }, ref('d'), ref('d')),
list(def('d', 'shared')),
],
};
const out = canonicalizeFootnotes(doc);
// One definition; the three references keep id "d".
expect(defOrder(out)).toEqual(['d']);
expect(
findAll(out, FOOTNOTE_REFERENCE_NAME).map((r) => r.attrs.id),
).toEqual(['d', 'd', 'd']);
});
it('duplicate definitions: first wins, the rest are dropped (never resurface as orphans)', () => {
const doc = {
type: 'doc',
content: [
para({ type: 'text', text: 'x' }, ref('d')),
list(def('d', 'first'), def('d', 'second'), def('d', 'third')),
],
};
const out = canonicalizeFootnotes(doc);
const defs = findAll(out, FOOTNOTE_DEFINITION_NAME);
expect(defs.map((d) => d.attrs.id)).toEqual(['d']);
expect(defs[0].content[0].content[0].text).toBe('first');
});
it('synthesizes an empty definition for a reference that has none', () => {
const doc = {
type: 'doc',
content: [para({ type: 'text', text: 'x' }, ref('missing'))],
};
const out = canonicalizeFootnotes(doc);
expect(defOrder(out)).toEqual(['missing']);
const list0 = findAll(out, FOOTNOTES_LIST_NAME);
expect(list0).toHaveLength(1);
});
it('merges multiple footnotesList nodes into one', () => {
const doc = {
type: 'doc',
content: [
para({ type: 'text', text: 'a' }, ref('x'), ref('y')),
list(def('x', 'X')),
para({ type: 'text', text: 'tail' }),
list(def('y', 'Y')),
],
};
const out = canonicalizeFootnotes(doc);
expect(findAll(out, FOOTNOTES_LIST_NAME)).toHaveLength(1);
expect(defOrder(out)).toEqual(['x', 'y']);
});
it('places the single list before trailing empty paragraphs', () => {
const doc = {
type: 'doc',
content: [
para({ type: 'text', text: 'x' }, ref('a')),
list(def('a', 'A')),
{ type: 'paragraph' },
],
};
const out = canonicalizeFootnotes(doc);
const last = out.content[out.content.length - 1];
expect(last.type).toBe('paragraph');
expect(out.content[out.content.length - 2].type).toBe(FOOTNOTES_LIST_NAME);
});
it('is idempotent: canonicalize(canonicalize(x)) === canonicalize(x)', () => {
const doc = {
type: 'doc',
content: [
para({ type: 'text', text: 'x' }, ref('b'), ref('a')),
list(def('a', 'A'), def('b', 'B'), def('orphan', 'O')),
],
};
const once = canonicalizeFootnotes(doc);
const twice = canonicalizeFootnotes(once);
expect(twice).toEqual(once);
});
it('does not mutate its input', () => {
const doc = {
type: 'doc',
content: [
para({ type: 'text', text: 'x' }, ref('a')),
list(def('orphan', 'O')),
],
};
const snapshot = JSON.parse(JSON.stringify(doc));
canonicalizeFootnotes(doc);
expect(doc).toEqual(snapshot);
});
});
/**
* GOLDEN PARITY against the live `footnoteSyncPlugin`. The server canonicalizer
* must produce EXACTLY what the editor keeps. For every editor-reachable steady
* state (the list is already reference-ordered there), driving a real editor to
* convergence and then running `canonicalizeFootnotes` on its JSON must be a
* byte-for-byte no-op — proving the server output is identical to the editor's.
*/
describe('canonicalizeFootnotes golden parity with footnoteSyncPlugin', () => {
function makeEditor(content: any) {
return new Editor({ extensions, content });
}
/** Load `content`, fire one local edit so the sync plugin converges, return JSON. */
function pluginSteadyState(content: any): any {
const editor = makeEditor(content);
// A local doc change triggers footnoteSyncPlugin.appendTransaction.
editor.commands.insertContentAt(1, ' ');
const json = editor.state.doc.toJSON();
editor.destroy();
return json;
}
const corpus: Array<{ name: string; content: any }> = [
{
name: 'plain ref + def',
content: {
type: 'doc',
content: [para({ type: 'text', text: 'a' }, ref('x')), list(def('x', 'X'))],
},
},
{
name: 'two refs, two defs in reference order',
content: {
type: 'doc',
content: [
para({ type: 'text', text: 'a' }, ref('x'), { type: 'text', text: 'b' }, ref('y')),
list(def('x', 'X'), def('y', 'Y')),
],
},
},
{
name: 'orphan definition gets removed',
content: {
type: 'doc',
content: [para({ type: 'text', text: 'a' }, ref('x')), list(def('x', 'X'), def('orphan', 'O'))],
},
},
{
name: 'reference missing its definition (synth empty)',
content: {
type: 'doc',
content: [para({ type: 'text', text: 'a' }, ref('x'))],
},
},
{
name: 'reuse: repeated references, one definition',
content: {
type: 'doc',
content: [
para(ref('d'), { type: 'text', text: ' a ' }, ref('d'), ref('d')),
list(def('d', 'shared')),
],
},
},
{
name: 'no footnotes at all',
content: {
type: 'doc',
content: [para({ type: 'text', text: 'just text' })],
},
},
];
for (const { name, content } of corpus) {
it(`steady state is a canonicalize no-op: ${name}`, () => {
const steady = pluginSteadyState(content);
expect(canonicalizeFootnotes(steady)).toEqual(steady);
});
}
it('placement parity: the LIVE plugin leaves a list with NON-EMPTY content after it in place, and canonicalize agrees', () => {
// Drives the real footnoteSyncPlugin (not a hand-authored expected): a single
// canonical list with body content AFTER it must NOT be repositioned by the
// plugin, and the server canonicalizer must agree (step-6 placement parity).
const content = {
type: 'doc',
content: [
para({ type: 'text', text: 'a' }, ref('x')),
list(def('x', 'X')),
para({ type: 'text', text: 'epilogue' }),
],
};
const steady = pluginSteadyState(content);
// The plugin did NOT move the list to the end: a non-empty paragraph follows it.
const types = steady.content.map((n: any) => n.type);
const listPos = types.indexOf(FOOTNOTES_LIST_NAME);
expect(listPos).toBeGreaterThanOrEqual(0);
expect(listPos).toBeLessThan(types.length - 1);
const after = steady.content[listPos + 1];
expect(after.type).toBe('paragraph');
expect(JSON.stringify(after)).toContain('epilogue');
// The canonicalizer is a byte-for-byte no-op on that steady state (parity).
expect(canonicalizeFootnotes(steady)).toEqual(steady);
});
it('the canonicalizer and the editor agree on reference order and definition set', () => {
const content = {
type: 'doc',
content: [
para({ type: 'text', text: 'a' }, ref('x'), { type: 'text', text: 'b' }, ref('y')),
list(def('y', 'Y'), def('x', 'X')), // physically reversed
],
};
const steady = pluginSteadyState(content);
const canon = canonicalizeFootnotes(content);
// Same reference order and same DEFINITION SET (ids) in both, even though the
// physical list order may differ (the plugin preserves node identity, the
// canonicalizer reorders). Numbering — derived from reference order — matches.
expect(refOrder(steady)).toEqual(['x', 'y']);
expect(defOrder(canon)).toEqual(['x', 'y']);
expect(new Set(defOrder(steady))).toEqual(new Set(defOrder(canon)));
});
});
/**
* SHARED golden corpus: this editor-ext copy of `canonicalizeFootnotes` and the
* MCP mirror (`packages/mcp/src/lib/footnote-canonicalize.ts`) are BOTH run
* against the identical { input -> expected } corpus. Pinning the same expected
* outputs in both suites makes "the two pure copies behave identically" a
* checkable property without coupling the packages (architecture item A). The
* MCP mirror of these assertions lives in `test/unit/footnote-corpus.test.mjs`.
*/
describe('canonicalizeFootnotes shared golden corpus (editor-ext copy)', () => {
for (const { name, input, expected } of FOOTNOTE_CORPUS) {
it(`matches the corpus expected output: ${name}`, () => {
expect(canonicalizeFootnotes(input)).toEqual(expected);
// Idempotent on the corpus too.
expect(canonicalizeFootnotes(expected)).toEqual(expected);
});
}
});

View File

@@ -0,0 +1,272 @@
import {
FOOTNOTE_REFERENCE_NAME,
FOOTNOTES_LIST_NAME,
FOOTNOTE_DEFINITION_NAME,
} from './footnote-util';
/**
* Server-side, EditorView-free port of the footnote integrity invariant that
* `footnoteSyncPlugin` maintains in the live editor. Where the plugin is an
* `appendTransaction` that only runs inside a ProseMirror `EditorView`, this is
* a PURE function over ProseMirror JSON: `canonicalizeFootnotes(doc) -> doc`.
*
* It exists because the NON-editor write paths served by THIS copy build
* ProseMirror JSON directly (never running the editor's plugins), so the
* canonical footnote topology was never enforced on those writes. The consumers
* of this editor-ext copy are: the server markdown/HTML import
* (`markdownToHtml -> htmlToJson` in import.service / file-import-task.service),
* `PageService` create/update (`parseProsemirrorContent` for the JSON/markdown/
* HTML REST write paths), and the client markdown PASTE path
* (`markdown-clipboard.ts`). (The MCP package mirrors this canonicalizer in
* `packages/mcp/src/lib/footnote-canonicalize.ts` for its own FULL-document write
* paths — `markdownToProseMirrorCanonical` (the page markdown-import path; the
* plain `markdownToProseMirror` primitive used for COMMENT bodies does NOT
* canonicalize), `update_page_json`, `docmost_transform`, `insert_footnote`,
* `copy_page_content` — see that file's header.) All of these are the root cause
* of the symptom in the issue: footnotes rendered out of order (`1, 4, 2, 3, …`),
* a raw trailing `[^id]: …` block, and orphan definitions, all of which are
* simply the result of content written PAST the canonicalizer.
*
* The desired end-state (identical to the plugin's) is:
*
* 1. Reference ids in DOCUMENT ORDER are the single source of truth for which
* definitions exist and in what order (numbering is derived from this, see
* `computeFootnoteNumbers`). Repeated references that share an id are REUSE
* (one footnote, one number, one definition) — never re-id'd.
* 2. Exactly ONE `footnotesList`, holding one definition per referenced id in
* REFERENCE order, reusing the existing definition node (content preserved)
* or synthesizing an empty one when missing. The list sits after the last
* meaningful block (only trailing empty paragraphs may follow it).
* 3. Orphan definitions (no matching reference) are dropped.
* 4. Duplicate DEFINITIONS (two nodes sharing an id) are resolved first-wins:
* the first definition for an id is kept; later duplicates carry the SAME
* id, so they can never be referenced separately and are simply dropped.
* This matches the importer's first-wins rule ("one definition per id").
* (The LIVE editor instead re-id's a duplicate definition so a paste/collab
* merge cannot silently lose live user data; the artifacts this copy
* sanitizes are agent/import-authored, so first-wins is the right policy —
* see footnote-sync.ts `resolveCollisions`.)
* 5. Idempotent: a document that already satisfies the invariant is returned
* structurally unchanged (the existing definition/list nodes are reused
* verbatim), so re-running the canonicalizer — or running it on a write that
* the editor already canonicalized — is a no-op. This is what makes it safe
* to wire into EVERY write path without spurious mutations / git-sync churn.
*
* Divergence from the live plugin (intentional): the plugin preserves the
* PHYSICAL order of existing definition nodes to keep their Yjs/CRDT subtree
* identity stable across collaborators (numbering is decoration-derived, so the
* displayed numbers are correct regardless of physical order). This function has
* no live CRDT to protect, so when a REPAIR is needed it physically REORDERS the
* list into reference order — which is exactly the fix the out-of-order import
* needs.
*
* Placement PARITY with the plugin: when the document is already in the canonical
* single-list state, this function leaves that list EXACTLY where it sits (it
* does not move it to the end). The plugin behaves the same — it treats one
* footnotesList holding the canonical definition set as canonical regardless of
* whether content follows it (footnote-sync.ts: `primaryList` falls back to the
* last list and `noChangeNeeded` stays true). So on every editor-reachable steady
* state the two agree byte-for-byte, including when non-empty content follows the
* list; see the golden parity test and the shared corpus.
*
* Pure: deep-clones its input, never mutates the caller's object, and is
* deterministic (no `Math.random`/`Date.now`).
*/
export function canonicalizeFootnotes<T = any>(doc: T): T {
if (
doc == null ||
typeof doc !== 'object' ||
!Array.isArray((doc as any).content)
) {
return doc;
}
const out = cloneJson(doc) as any;
// 1) Distinct reference ids in document order (deep — references can live in
// callouts, tables, list items, ...). This is the ordering/numbering truth.
const referenceIds: string[] = [];
const seenRefIds = new Set<string>();
collectReferenceIds(out, referenceIds, seenRefIds);
// 2) Every definition node in document order (deep — defs normally live inside
// one or more `footnotesList` blocks, but we tolerate stray placements).
const defNodes: any[] = [];
collectDefinitions(out, defNodes);
// 3) First definition per id wins. Later duplicates carry the SAME id, so they
// can never be referenced separately and would be orphans — they are simply
// dropped (first-wins; see the file header, item 4).
const defById = new Map<string, any>();
for (const d of defNodes) {
const id = d?.attrs?.id;
if (id && !defById.has(id)) defById.set(id, d);
}
// 4) Build the ordered definition list: one per referenced id, in REFERENCE
// order, reusing the existing node (content preserved, id normalized) or
// synthesizing an empty definition. Definitions whose id is NOT referenced
// are orphans and are simply never added. The reused node is SHALLOW-copied
// (id normalized): `out` is already a deep clone and the old lists are cut,
// so a second per-definition deep clone is needless.
const orderedDefs: any[] = [];
for (const id of referenceIds) {
const existing = defById.get(id);
if (existing) {
orderedDefs.push({
...existing,
attrs: { ...(existing.attrs ?? {}), id },
});
} else {
orderedDefs.push(emptyDefinition(id));
}
}
// 5) No references -> there must be NO list at all (at any depth).
if (referenceIds.length === 0) {
stripFootnotesListsDeep(out);
return out;
}
// 6) Placement parity with the live plugin: when the document is ALREADY in the
// canonical single-list state, leave that list exactly where it sits instead
// of cutting and re-inserting it at the end. The plugin never repositions a
// sole correct list (footnote-sync.ts), so moving it here would silently
// reorder any user content that follows the list on the first write. The doc
// is in that state when there is exactly one top-level footnotesList, every
// definition in the doc is referenced (no orphans / duplicates: the def count
// equals the canonical count), and the list already holds exactly the
// canonical definitions in reference order.
const topLevelLists = out.content.filter(
(n: any) => n && n.type === FOOTNOTES_LIST_NAME,
);
if (
topLevelLists.length === 1 &&
defNodes.length === orderedDefs.length &&
deepEqualJson(topLevelLists[0].content, orderedDefs)
) {
return out;
}
// 7) Otherwise rebuild: strip every footnotesList AND every bare
// footnoteDefinition at ANY depth (collectDefinitions gathers defs
// recursively, so a list nested in a callout/blockquote — or a bare
// definition outside any list — would otherwise have its defs copied into the
// rebuilt list while the original survives in place → duplicates) and
// re-insert exactly one list after the last meaningful (non-empty paragraph)
// top-level block, so it coexists with a trailing-node empty paragraph. This
// both repairs a non-canonical doc and (in the import case) physically
// reorders the list into reference order.
stripFootnotesListsDeep(out);
stripFootnoteDefinitionsDeep(out);
const top: any[] = out.content;
let insertAt = top.length;
while (insertAt > 0 && isEmptyParagraph(top[insertAt - 1])) insertAt--;
top.splice(insertAt, 0, { type: FOOTNOTES_LIST_NAME, content: orderedDefs });
out.content = top;
return out;
}
/** Remove every `footnotesList` node at ANY depth (mutates the given clone). */
function stripFootnotesListsDeep(node: any): void {
if (!node || typeof node !== 'object' || !Array.isArray(node.content)) return;
node.content = node.content.filter(
(c: any) => !(c && c.type === FOOTNOTES_LIST_NAME),
);
for (const child of node.content) stripFootnotesListsDeep(child);
}
/**
* Remove every BARE `footnoteDefinition` node at ANY depth (mutates the given
* clone). Runs only in the rebuild path AFTER the lists are stripped, so it
* targets definitions that were sitting outside a list (e.g. hand-authored via a
* raw-JSON write path and nested in a callout); their content was already copied
* into the rebuilt list, so leaving the originals would duplicate them.
*/
function stripFootnoteDefinitionsDeep(node: any): void {
if (!node || typeof node !== 'object' || !Array.isArray(node.content)) return;
node.content = node.content.filter(
(c: any) => !(c && c.type === FOOTNOTE_DEFINITION_NAME),
);
for (const child of node.content) stripFootnoteDefinitionsDeep(child);
}
/**
* Deep equality over plain JSON: arrays are compared POSITIONALLY
* (order-SENSITIVE), object keys order-insensitively. The array order-sensitivity
* is required for correctness here — a reordered `footnotesList.content` must
* compare UNEQUAL so the canonical rebuild fires instead of leaving it in place.
*/
function deepEqualJson(a: any, b: any): boolean {
if (a === b) return true;
if (a == null || b == null || typeof a !== typeof b) return false;
if (Array.isArray(a) || Array.isArray(b)) {
if (!Array.isArray(a) || !Array.isArray(b) || a.length !== b.length) {
return false;
}
for (let i = 0; i < a.length; i++) {
if (!deepEqualJson(a[i], b[i])) return false;
}
return true;
}
if (typeof a === 'object') {
const ka = Object.keys(a);
const kb = Object.keys(b);
if (ka.length !== kb.length) return false;
for (const k of ka) {
if (!Object.prototype.hasOwnProperty.call(b, k)) return false;
if (!deepEqualJson(a[k], b[k])) return false;
}
return true;
}
return false;
}
/** A fresh empty definition node for a referenced id with no definition. */
function emptyDefinition(id: string): any {
return {
type: FOOTNOTE_DEFINITION_NAME,
attrs: { id },
content: [{ type: 'paragraph' }],
};
}
function isEmptyParagraph(node: any): boolean {
return (
!!node &&
node.type === 'paragraph' &&
(!Array.isArray(node.content) || node.content.length === 0)
);
}
/** Collect DISTINCT footnoteReference ids in document order (first appearance). */
function collectReferenceIds(
node: any,
out: string[],
seen: Set<string>,
): void {
if (!node || typeof node !== 'object') return;
if (node.type === FOOTNOTE_REFERENCE_NAME) {
const id = node?.attrs?.id;
if (id && !seen.has(id)) {
seen.add(id);
out.push(id);
}
}
if (Array.isArray(node.content)) {
for (const child of node.content) collectReferenceIds(child, out, seen);
}
}
/** Collect every footnoteDefinition node in document order. */
function collectDefinitions(node: any, out: any[]): void {
if (!node || typeof node !== 'object') return;
if (node.type === FOOTNOTE_DEFINITION_NAME) out.push(node);
if (Array.isArray(node.content)) {
for (const child of node.content) collectDefinitions(child, out);
}
}
function cloneJson<T>(v: T): T {
if (typeof structuredClone === 'function') return structuredClone(v);
return JSON.parse(JSON.stringify(v)) as T;
}

File diff suppressed because it is too large Load Diff

View File

@@ -4,3 +4,4 @@ export * from "./footnotes-list";
export * from "./footnote-definition";
export * from "./footnote-numbering";
export * from "./footnote-sync";
export * from "./footnote-canonicalize";

View File

@@ -22,5 +22,11 @@
"noFallthroughCasesInSwitch": false
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist", "src/**/*.spec.ts", "src/**/*.test.ts"]
"exclude": [
"node_modules",
"dist",
"src/**/*.spec.ts",
"src/**/*.test.ts",
"src/lib/footnote/footnote-corpus.ts"
]
}

View File

@@ -7,7 +7,7 @@ import { TiptapTransformer } from "@hocuspocus/transformer";
import * as Y from "yjs";
import WebSocket from "ws";
import { convertProseMirrorToMarkdown } from "./lib/markdown-converter.js";
import { updatePageContentRealtime, replacePageContent, markdownToProseMirror, mutatePageContent, buildCollabWsUrl, assertYjsEncodable, applyDocToFragment, } from "./lib/collaboration.js";
import { updatePageContentRealtime, replacePageContent, markdownToProseMirror, markdownToProseMirrorCanonical, mutatePageContent, buildCollabWsUrl, assertYjsEncodable, applyDocToFragment, } from "./lib/collaboration.js";
import { footnoteWarningsField } from "./lib/footnote-analyze.js";
import { buildPageTree } from "./lib/tree.js";
import { serializeDocmostMarkdown, parseDocmostMarkdown, } from "./lib/markdown-document.js";
@@ -17,7 +17,7 @@ import { applyTextEdits, } from "./lib/json-edit.js";
import { getCollabToken, performLogin } from "./lib/auth-utils.js";
import { diffDocs, summarizeChange } from "./lib/diff.js";
import { applyAnchorInDoc, canAnchorInDoc } from "./lib/comment-anchor.js";
import { blockText, walk, getList, insertMarkerAfter, setCalloutRange, noteItem, mdToInlineNodes, commentsToFootnotes, } from "./lib/transforms.js";
import { blockText, walk, getList, insertMarkerAfter, setCalloutRange, noteItem, mdToInlineNodes, commentsToFootnotes, canonicalizeFootnotes, insertInlineFootnote, } from "./lib/transforms.js";
import vm from "node:vm";
// Supported image types, kept as two lookup tables so both a local file
// extension and a remote Content-Type can be mapped to the same canonical set.
@@ -1063,10 +1063,15 @@ export class DocmostClient {
// the markdown link path (which TipTap sanitizes), raw JSON could otherwise
// inject javascript:/data: link hrefs or media srcs straight into the doc.
this.validateDocUrls(doc);
// Canonicalize footnotes (idempotent): an agent-authored JSON doc cannot
// leave footnotes out of order, orphaned, or in multiple lists — the bottom
// list + numbering are always derived from reference order. No-op when the
// footnotes are already canonical.
doc = canonicalizeFootnotes(doc);
// Write the BODY first, then the title (#159 split-brain): a failed body
// write (e.g. persist timeout) must not leave a new title over the old body.
const collabToken = await this.getCollabTokenWithReauth();
const mutation = await replacePageContent(pageId, doc, collabToken, this.apiUrl);
const mutation = await this.replacePage(pageId, doc, collabToken, this.apiUrl);
// Body persisted successfully — now it is safe to set the title.
if (title) {
await this.client.post("/pages/update", { pageId, title });
@@ -1079,6 +1084,73 @@ export class DocmostClient {
verify: mutation.verify,
};
}
/**
* AUTHOR-INLINE footnote insertion. The agent supplies only WHERE
* (`anchorText`, a snippet of body text to attach the marker after) and WHAT
* (`text`, the footnote content as markdown). Numbering and the bottom
* `footnotesList` are derived deterministically server-side
* (`insertInlineFootnote` -> `canonicalizeFootnotes`): the agent never sees,
* assigns, or edits a footnote number or the list, so it CANNOT desync.
*
* Content DEDUP: when an existing definition has the same content, its id is
* reused (one number, one definition, several references). The write is atomic
* via `mutatePageContent` (single-writer, page-locked); if the anchor text is
* not found the transform aborts with a clear error and no write happens.
*/
async insertFootnote(pageId, anchorText, text) {
await this.ensureAuthenticated();
if (!anchorText || !anchorText.trim()) {
throw new Error("insert_footnote: anchorText is required");
}
if (text == null || `${text}`.trim() === "") {
throw new Error("insert_footnote: text is required");
}
const collabToken = await this.getCollabTokenWithReauth();
let result = null;
const mutation = await this.mutatePage(pageId, collabToken, this.apiUrl, (liveDoc) => {
const r = insertInlineFootnote(liveDoc, { anchorText, text });
if (!r.inserted) {
// Abort the page-locked write by throwing: mutatePageContent does not
// persist when the transform throws, so a missing anchor leaves the
// page untouched (no partial write).
throw new Error(`insert_footnote: anchor text not found: ${JSON.stringify(anchorText.slice(0, 80))}`);
}
result = { footnoteId: r.footnoteId, reused: r.reused };
return r.doc;
});
// The not-found path throws inside the transform (aborting mutatePage), so by
// here `result` is always set.
const r = result;
return {
success: true,
modified: true,
pageId,
footnoteId: r.footnoteId,
reused: r.reused,
message: r.reused
? "Footnote inserted (reused an existing same-content definition)."
: "Footnote inserted.",
verify: mutation.verify,
};
}
/**
* Page-locked write seam over collaboration.mutatePageContent. Production just
* delegates; it exists as an overridable method so the insert_footnote wrapper
* (transform abort-on-not-found + response shaping) can be unit-tested without
* standing up a live Hocuspocus collab socket.
*/
mutatePage(pageId, collabToken, apiUrl, transform) {
return mutatePageContent(pageId, collabToken, apiUrl, transform);
}
/**
* Full-document write seam over collaboration.replacePageContent. Production
* just delegates; it exists as an overridable method so the full-doc write
* tools (update_page_json, copy_page_content) can have their footnote-
* canonicalization binding unit-tested without a live Hocuspocus collab socket.
*/
replacePage(pageId, doc, collabToken, apiUrl) {
return replacePageContent(pageId, doc, collabToken, apiUrl);
}
/**
* Export a page to a single self-contained Docmost-flavoured markdown file:
* meta block + body (with inline comment anchors + diagrams) + comment
@@ -1120,7 +1192,8 @@ export class DocmostClient {
async importPageMarkdown(pageId, fullMarkdown) {
await this.ensureAuthenticated();
const { meta, body, comments } = parseDocmostMarkdown(fullMarkdown);
const doc = await markdownToProseMirror(body);
// PAGE import: canonicalize footnotes (see markdownToProseMirrorCanonical).
const doc = await markdownToProseMirrorCanonical(body);
const collabToken = await this.getCollabTokenWithReauth();
const mutation = await replacePageContent(pageId, doc, collabToken, this.apiUrl);
// Collect distinct comment ids that actually became comment marks in the doc.
@@ -1200,13 +1273,18 @@ export class DocmostClient {
// uses, so copying never lands a javascript:/data: href/src on the target
// (parity with updatePageJson; harmless for already-stored source content).
this.validateDocUrls(content);
// Defense-in-depth (#228): this is a FULL-document write, so canonicalize
// footnotes before copying — a no-op on already-canonical source content, but
// it guarantees a copy can never propagate a non-canonical footnote topology
// to the target (parity with the other full-doc write paths).
const canonical = canonicalizeFootnotes(content);
const collabToken = await this.getCollabTokenWithReauth();
const mutation = await replacePageContent(targetPageId, content, collabToken, this.apiUrl);
const mutation = await this.replacePage(targetPageId, canonical, collabToken, this.apiUrl);
return {
success: true,
sourcePageId,
targetPageId,
copiedNodes: content.content.length,
copiedNodes: canonical.content.length,
verify: mutation.verify,
};
}
@@ -1613,7 +1691,10 @@ export class DocmostClient {
}
}
}
// Convert through the full Docmost schema (consistent with page paths)
// Convert through the full Docmost schema. Deliberately the NON-canonicalizing
// variant: a comment body may carry a footnote definition with no matching
// reference, and canonicalization would drop it (data loss). See
// markdownToProseMirror vs markdownToProseMirrorCanonical.
const jsonContent = await markdownToProseMirror(content);
const payload = {
pageId,
@@ -1701,6 +1782,7 @@ export class DocmostClient {
}
async updateComment(commentId, content) {
await this.ensureAuthenticated();
// NON-canonicalizing on purpose (comment body — see createComment).
const jsonContent = await markdownToProseMirror(content);
await this.client.post("/comments/update", {
commentId,
@@ -2422,6 +2504,8 @@ export class DocmostClient {
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
},
};
// Captured oldDoc / newDoc for the diff (set inside runTransform).
@@ -2455,16 +2539,25 @@ export class DocmostClient {
if (typeof fn !== "function") {
throw new Error("transform must evaluate to a function (doc, ctx) => doc");
}
const result = vm.runInNewContext("f(d, c)", { f: fn, d: sandbox.doc, c: ctx }, { timeout: 5000 });
if (!result ||
typeof result !== "object" ||
result.type !== "doc" ||
!Array.isArray(result.content)) {
const raw = vm.runInNewContext("f(d, c)", { f: fn, d: sandbox.doc, c: ctx }, { timeout: 5000 });
if (!raw ||
typeof raw !== "object" ||
raw.type !== "doc" ||
!Array.isArray(raw.content)) {
throw new Error('transform must return a ProseMirror doc node ({ type:"doc", content:[...] })');
}
// Validate the returned doc before it can be written.
this.validateDocStructure(result);
this.validateDocUrls(result);
// Validate the RAW transform output FIRST (structure — including the
// MAX_DEPTH guard — and URLs), mirroring updatePageJson. The canonicalizer
// recurses without a depth limiter, so validating after it would turn a
// too-deep doc into an opaque "Maximum call stack size exceeded" instead of
// the intended "nesting exceeds the maximum depth" error.
this.validateDocStructure(raw);
this.validateDocUrls(raw);
// Auto-canonicalize footnotes after the transform (idempotent): no write
// path can leave footnotes out of order / orphaned / in a raw `[^id]`
// block. In a dryRun preview this may surface footnote edits the script
// author did not write (the canonicalizer tidied them) — that is expected.
const result = canonicalizeFootnotes(raw);
newDoc = result;
return result;
};

View File

@@ -637,8 +637,15 @@ export function createDocmostMcpServer(config) {
"mark-safe), setCalloutRange(doc, n) (sync a [1]…[K] callout range to " +
"[1]…[n]), noteItem(inlineNodes) (wrap inline nodes in a listItem with a " +
"fresh id), mdToInlineNodes(markdown) (comment markdown -> inline nodes), " +
"and commentsToFootnotes(doc, comments, {notesHeading}) (turn inline " +
"comments into numbered footnotes). Footnote convention: markers are " +
"commentsToFootnotes(doc, comments, {notesHeading}) (turn inline " +
"comments into numbered footnotes), canonicalizeFootnotes(doc) (derive " +
"footnote numbering + the single bottom list from reference order, drop " +
"orphans/duplicates — runs AUTOMATICALLY on the transform RESULT, so the " +
"applied (and dryRun-previewed) doc is always footnote-canonical; a dryRun " +
"diff may therefore show footnote tidy-ups your script did not make, and " +
"it is idempotent after the first run), and " +
"insertInlineFootnote(doc, {anchorText, text}) (author-inline footnote: " +
"marker + dedup'd definition, list derived). Footnote convention: markers are " +
"plain '[N]' text in the body; the notes are an orderedList under a " +
"heading whose text is 'Примечания переводчика'. The transform runs " +
"sandboxed (no require/process/fs/network, 5s timeout) and must return a " +
@@ -652,7 +659,8 @@ export function createDocmostMcpServer(config) {
"parenthesized function). It receives a clone of the live doc and " +
"ctx (comments, log, consume(id), helpers: blockText/walk/getList/" +
"insertMarkerAfter/setCalloutRange/noteItem/mdToInlineNodes/" +
"commentsToFootnotes) and must return a {type:'doc'} node."),
"commentsToFootnotes/canonicalizeFootnotes/insertInlineFootnote) " +
"and must return a {type:'doc'} node."),
dryRun: z
.boolean()
.optional()
@@ -672,6 +680,33 @@ export function createDocmostMcpServer(config) {
});
return jsonContent(result);
});
// Tool: insert_footnote
server.registerTool("insert_footnote", {
description: "Insert an AUTHOR-INLINE footnote: you specify only WHERE (anchorText) " +
"and WHAT (text). The footnote marker is placed right after anchorText in " +
"the body, and the bottom footnotes list + the numbering are derived " +
"deterministically server-side. You do NOT assign a number, and you " +
"never see or edit the footnotes list — so footnotes cannot end up out " +
"of order, orphaned, or as a raw '[^id]' block. If a footnote with the " +
"SAME text already exists, its number is REUSED (one definition, several " +
"references). The write is atomic and won't clobber concurrent edits; if " +
"anchorText is not found, nothing is written and an error is returned.",
inputSchema: {
pageId: z.string().min(1),
anchorText: z
.string()
.min(1)
.describe("A snippet of existing body text; the footnote marker is inserted " +
"immediately after its first occurrence (mark-safe)."),
text: z
.string()
.min(1)
.describe("The footnote content as markdown (becomes the definition)."),
},
}, async ({ pageId, anchorText, text }) => {
const result = await docmostClient.insertFootnote(pageId, anchorText, text);
return jsonContent(result);
});
// Tool: diff_page_versions
registerShared(SHARED_TOOL_SPECS.diffPageVersions, async ({ pageId, from, to }) => {
const result = await docmostClient.diffPageVersions(pageId, from, to);

View File

@@ -11,6 +11,7 @@ import { docmostExtensions, docmostSchema } from "./docmost-schema.js";
import { withPageLock } from "./page-lock.js";
import { sanitizeForYjs, findUnstorableAttr } from "./node-ops.js";
import { lexFootnoteLines } from "./footnote-lex.js";
import { canonicalizeFootnotes } from "./footnote-canonicalize.js";
import { summarizeChange } from "./diff.js";
/**
* Build the descriptive error for an opaque Yjs encode failure ("Unexpected
@@ -343,7 +344,20 @@ function extractFootnotes(markdown) {
section: `<section data-footnotes>${inner}</section>`,
};
}
/** Convert markdown to a ProseMirror doc using the full Docmost schema. */
/**
* Convert markdown to a ProseMirror doc using the full Docmost schema.
*
* This conversion does NOT canonicalize footnotes — it is the shared, content-
* preserving primitive used by BOTH page write paths and COMMENT bodies
* (createComment / updateComment). Canonicalization MUST NOT run on a comment
* body: a comment may legitimately contain a footnote-definition line
* (`[^1]: text`) with no matching reference, and the canonicalizer drops a
* reference-less footnotesList — which would silently delete the comment's text.
*
* Page write paths that DO need the canonical footnote topology call
* `markdownToProseMirrorCanonical` instead (markdown import, update_page markdown
* path). Keep this function reference-loss-free.
*/
export async function markdownToProseMirror(markdownContent) {
const withCallouts = await preprocessCallouts(markdownContent);
const { body, section } = extractFootnotes(withCallouts);
@@ -351,6 +365,20 @@ export async function markdownToProseMirror(markdownContent) {
const bridged = bridgeTaskLists(html);
return generateJSON(bridged, docmostExtensions);
}
/**
* Page-write variant of `markdownToProseMirror`: converts markdown then enforces
* the canonical footnote topology. The footnote `section` markdown is emitted in
* DEFINITION order, but numbering derives from REFERENCE order, so without this
* the bottom list renders out of order (`1, 4, 2, 3, …`); orphan definitions and
* duplicate lists are also normalized. Idempotent — a no-op once canonical, and a
* no-op for footnote-free content.
*
* Use this ONLY for full-document PAGE writes (never for comment bodies, where it
* would drop a reference-less footnote definition — see `markdownToProseMirror`).
*/
export async function markdownToProseMirrorCanonical(markdownContent) {
return canonicalizeFootnotes(await markdownToProseMirror(markdownContent));
}
/**
* Build the collaboration WebSocket URL from an API base URL:
* switch http(s)->ws(s), strip a trailing /api, mount on /collab.
@@ -708,6 +736,8 @@ export async function replacePageContent(pageId, prosemirrorDoc, collabToken, ba
* Tables and :::callout::: blocks survive thanks to the full schema.
*/
export async function updatePageContentRealtime(pageId, markdownContent, collabToken, baseUrl) {
const tiptapJson = await markdownToProseMirror(markdownContent);
// PAGE write: canonicalize footnotes (markdown import builds the bottom list in
// definition order; numbering is reference-ordered).
const tiptapJson = await markdownToProseMirrorCanonical(markdownContent);
return await mutatePageContent(pageId, collabToken, baseUrl, () => tiptapJson);
}

View File

@@ -0,0 +1,88 @@
/**
* Inline-authoring helpers for footnotes (MCP).
*
* These build/identify footnote DEFINITION nodes for the author-inline tool
* (`insertInlineFootnote` in transforms.ts): a content key to de-duplicate notes
* by text, a definition-node factory, and a fresh uuidv7-style id generator.
*
* Split out of `footnote-canonicalize.ts` so that module stays a pure MIRROR of
* the editor-ext canonicalizer (compositionally symmetric to the editor-ext
* copy, which keeps its authoring helpers in `footnote-util.ts`). The pure
* canonicalizer has no dependency on these.
*/
const FOOTNOTE_DEFINITION_NAME = "footnoteDefinition";
function cloneJson(v) {
if (typeof structuredClone === "function")
return structuredClone(v);
return JSON.parse(JSON.stringify(v));
}
/**
* Normalized content key for de-duplicating footnote DEFINITIONS by their text.
*
* Two definitions with the same key are the SAME footnote — so the inline
* authoring tool reuses one id (one number, one definition, several references)
* instead of minting a second definition. Key = plaintext (whitespace-collapsed,
* trimmed) PLUS a signature of the inline mark types in order, so two notes that
* read the same but differ in formatting (one bold, one plain) are NOT merged.
* Conservative: only an exact match merges.
*/
export function footnoteContentKey(defNode) {
const parts = [];
const visit = (n) => {
if (!n || typeof n !== "object")
return;
if (n.type === "text" && typeof n.text === "string") {
const marks = Array.isArray(n.marks)
? n.marks.map((m) => m?.type).filter(Boolean).sort().join(",")
: "";
parts.push(`${n.text}${marks}`);
}
if (Array.isArray(n.content))
for (const c of n.content)
visit(c);
};
visit(defNode);
// Collapse the assembled text's whitespace and trim, keeping the mark
// signature attached so formatting differences still distinguish notes.
return parts
.join("")
.replace(/[ \t\r\n]+/g, " ")
.trim();
}
/**
* Build a footnoteDefinition node from inline ProseMirror nodes, keyed by id.
*/
export function makeFootnoteDefinition(id, inlineNodes) {
const content = Array.isArray(inlineNodes) ? cloneJson(inlineNodes) : [];
return {
type: FOOTNOTE_DEFINITION_NAME,
attrs: { id },
content: [{ type: "paragraph", content }],
};
}
/**
* Generate a uuidv7-style id (time-ordered), matching editor-ext's
* `generateFootnoteId`. Used for a genuinely-new inline footnote id.
*/
export function generateFootnoteId() {
const now = Date.now();
const timeHex = now.toString(16).padStart(12, "0");
const rand = (length) => {
let s = "";
for (let i = 0; i < length; i++)
s += Math.floor(Math.random() * 16).toString(16);
return s;
};
const versioned = "7" + rand(3);
const variantNibble = (8 + Math.floor(Math.random() * 4)).toString(16);
const variant = variantNibble + rand(3);
return (timeHex.slice(0, 8) +
"-" +
timeHex.slice(8, 12) +
"-" +
versioned +
"-" +
variant +
"-" +
rand(12));
}

View File

@@ -0,0 +1,215 @@
/**
* Server-side footnote canonicalizer (MCP mirror — PURE).
*
* `canonicalizeFootnotes(doc)` is a pure ProseMirror-JSON port of the editor's
* `footnoteSyncPlugin` end-state, identical in behaviour to
* `@docmost/editor-ext`'s `canonicalizeFootnotes`. It is mirrored here — rather
* than imported from editor-ext — for the SAME reason `footnote-lex.ts` and the
* `docmost-schema.ts` nodes are mirrored: the MCP package is deliberately
* decoupled from the browser/React-heavy editor barrel and operates on plain
* JSON. The editor-ext copy owns the golden test against the live plugin; this
* copy must stay behaviourally identical (a SHARED golden corpus, exercised by
* both test suites, pins that — see `test/unit/footnote-corpus.mjs`).
*
* This module is the pure MIRROR only. The inline-authoring helpers
* (`footnoteContentKey`, `makeFootnoteDefinition`, `generateFootnoteId`) used by
* `insertInlineFootnote` live in the sibling `footnote-authoring.ts`, so this
* file is compositionally symmetric to the editor-ext copy.
*
* Why it exists: every NON-editor write path (markdown import, update_page_json,
* docmost_transform, insert_footnote) builds ProseMirror JSON directly, so the
* editor's footnote plugins never run and the canonical topology (sequential
* numbering by first reference, one trailing list, no orphans, no raw `[^id]`)
* was never enforced. Running this at the end of every write path closes that
* gap; because it is idempotent, it is a no-op when the footnotes are already
* canonical (no spurious mutations / git-sync churn).
*
* ENFORCEMENT RULE (#228): any NEW FULL-document persist path MUST call
* `canonicalizeFootnotes(doc)` before writing — the current callers are
* `markdownToProseMirrorCanonical` (page markdown import/update; the plain
* `markdownToProseMirror` used for COMMENT bodies must NOT, or it would drop a
* reference-less definition), `update_page_json`, `docmost_transform`,
* `insert_footnote`, and `copy_page_content`. Append/prepend FRAGMENT writes MUST
* NOT canonicalize. This is deliberately per-call-site (the replace-vs-fragment
* and comment-vs-page nuances make a single naive wrapper unsafe).
*/
const FOOTNOTE_REFERENCE_NAME = "footnoteReference";
const FOOTNOTES_LIST_NAME = "footnotesList";
const FOOTNOTE_DEFINITION_NAME = "footnoteDefinition";
function cloneJson(v) {
if (typeof structuredClone === "function")
return structuredClone(v);
return JSON.parse(JSON.stringify(v));
}
function isEmptyParagraph(node) {
return (!!node &&
node.type === "paragraph" &&
(!Array.isArray(node.content) || node.content.length === 0));
}
function collectReferenceIds(node, out, seen) {
if (!node || typeof node !== "object")
return;
if (node.type === FOOTNOTE_REFERENCE_NAME) {
const id = node?.attrs?.id;
if (id && !seen.has(id)) {
seen.add(id);
out.push(id);
}
}
if (Array.isArray(node.content)) {
for (const child of node.content)
collectReferenceIds(child, out, seen);
}
}
function collectDefinitions(node, out) {
if (!node || typeof node !== "object")
return;
if (node.type === FOOTNOTE_DEFINITION_NAME)
out.push(node);
if (Array.isArray(node.content)) {
for (const child of node.content)
collectDefinitions(child, out);
}
}
function emptyDefinition(id) {
return {
type: FOOTNOTE_DEFINITION_NAME,
attrs: { id },
content: [{ type: "paragraph" }],
};
}
/**
* Deep equality over plain JSON: arrays are compared POSITIONALLY
* (order-SENSITIVE), object keys order-insensitively. The array order-sensitivity
* is required for correctness here — a reordered `footnotesList.content` must
* compare UNEQUAL so the canonical rebuild fires instead of leaving it in place.
*/
function deepEqualJson(a, b) {
if (a === b)
return true;
if (a == null || b == null || typeof a !== typeof b)
return false;
if (Array.isArray(a) || Array.isArray(b)) {
if (!Array.isArray(a) || !Array.isArray(b) || a.length !== b.length) {
return false;
}
for (let i = 0; i < a.length; i++) {
if (!deepEqualJson(a[i], b[i]))
return false;
}
return true;
}
if (typeof a === "object") {
const ka = Object.keys(a);
const kb = Object.keys(b);
if (ka.length !== kb.length)
return false;
for (const k of ka) {
if (!Object.prototype.hasOwnProperty.call(b, k))
return false;
if (!deepEqualJson(a[k], b[k]))
return false;
}
return true;
}
return false;
}
/**
* Canonicalize footnotes in a ProseMirror-JSON document. See the file header and
* the editor-ext twin for the full contract. Pure (deep-clones input,
* deterministic, idempotent).
*/
export function canonicalizeFootnotes(doc) {
if (doc == null ||
typeof doc !== "object" ||
!Array.isArray(doc.content)) {
return doc;
}
const out = cloneJson(doc);
// 1) Distinct reference ids in document order (deep — refs can live in
// callouts, tables, list items, ...). The ordering/numbering truth.
const referenceIds = [];
collectReferenceIds(out, referenceIds, new Set());
// 2) Every definition node in document order (deep).
const defNodes = [];
collectDefinitions(out, defNodes);
// 3) First definition per id wins; later duplicates carry the SAME id, so they
// cannot be referenced separately and would be orphans — they are dropped.
const defById = new Map();
for (const d of defNodes) {
const id = d?.attrs?.id;
if (id && !defById.has(id))
defById.set(id, d);
}
// 4) Build the ordered definition list: one per referenced id, in REFERENCE
// order, reusing the existing node (shallow-copied, id normalized — `out` is
// already deep-cloned and the old lists are cut) or synthesizing an empty
// one. Definitions whose id is not referenced are orphans and never added.
const orderedDefs = [];
for (const id of referenceIds) {
const existing = defById.get(id);
if (existing) {
orderedDefs.push({
...existing,
attrs: { ...(existing.attrs ?? {}), id },
});
}
else {
orderedDefs.push(emptyDefinition(id));
}
}
// 5) No references -> there must be NO list at all (at any depth).
if (referenceIds.length === 0) {
stripFootnotesListsDeep(out);
return out;
}
// 6) Placement parity with the live plugin: when the document is ALREADY in the
// canonical single-list state, leave that list exactly where it sits rather
// than cutting and re-inserting it at the end (the plugin never repositions a
// sole correct list, so moving it would silently reorder any content that
// follows the list on the first write).
const topLevelLists = out.content.filter((n) => n && n.type === FOOTNOTES_LIST_NAME);
if (topLevelLists.length === 1 &&
defNodes.length === orderedDefs.length &&
deepEqualJson(topLevelLists[0].content, orderedDefs)) {
return out;
}
// 7) Otherwise rebuild: strip every footnotesList AND every bare
// footnoteDefinition at ANY depth (collectDefinitions gathers defs
// recursively, so a list nested in a callout/blockquote — or a bare
// definition outside any list — would otherwise have its defs copied into the
// rebuilt list while the original survives in place → duplicates) and
// re-insert exactly one list after the last meaningful (non-empty paragraph)
// top-level block.
stripFootnotesListsDeep(out);
stripFootnoteDefinitionsDeep(out);
const top = out.content;
let insertAt = top.length;
while (insertAt > 0 && isEmptyParagraph(top[insertAt - 1]))
insertAt--;
top.splice(insertAt, 0, { type: FOOTNOTES_LIST_NAME, content: orderedDefs });
out.content = top;
return out;
}
/** Remove every `footnotesList` node at ANY depth (mutates the given clone). */
function stripFootnotesListsDeep(node) {
if (!node || typeof node !== "object" || !Array.isArray(node.content))
return;
node.content = node.content.filter((c) => !(c && c.type === FOOTNOTES_LIST_NAME));
for (const child of node.content)
stripFootnotesListsDeep(child);
}
/**
* Remove every BARE `footnoteDefinition` node at ANY depth (mutates the given
* clone). Runs only in the rebuild path AFTER the lists are stripped, so it
* targets definitions that were sitting outside a list (e.g. hand-authored via a
* raw-JSON write path and nested in a callout); their content was already copied
* into the rebuilt list, so leaving the originals would duplicate them.
*/
function stripFootnoteDefinitionsDeep(node) {
if (!node || typeof node !== "object" || !Array.isArray(node.content))
return;
node.content = node.content.filter((c) => !(c && c.type === FOOTNOTE_DEFINITION_NAME));
for (const child of node.content)
stripFootnoteDefinitionsDeep(child);
}

View File

@@ -14,6 +14,9 @@
* - `marks` arrays are preserved verbatim when fragments are split/reordered.
*/
import { blockPlainText } from "./node-ops.js";
import { canonicalizeFootnotes } from "./footnote-canonicalize.js";
import { footnoteContentKey, makeFootnoteDefinition, generateFootnoteId, } from "./footnote-authoring.js";
export { canonicalizeFootnotes } from "./footnote-canonicalize.js";
/** Deep-clone a JSON-serializable value without mutating the original. */
function clone(value) {
if (typeof structuredClone === "function") {
@@ -64,6 +67,36 @@ export function getList(doc, predicate) {
});
return found;
}
/**
* Textblocks that hold raw text but do NOT accept inline atom nodes. A
* `footnoteReference` is `group:"inline", atom:true`; `codeBlock` is
* `content:"text*"` (text only), so splicing a footnoteReference into it yields
* an invalid document. (paragraph/heading/detailsSummary are `inline*` and DO
* accept it; footnote definitions live inside a footnotesList which the
* footnote inserter excludes via `beforeBlock`.)
*/
const INLINE_ATOM_FORBIDDEN_BLOCKS = new Set(["codeBlock"]);
/**
* Footnote-notes subtrees the inline footnote inserter must never split into (at
* any depth): a `footnotesList` and the `footnoteDefinition`s it holds. Anchoring
* a reference inside one of these would later be dropped as an orphan by the
* canonicalizer, taking the existing definition's text with it.
*/
const FOOTNOTE_NOTES_SUBTREES = new Set([
"footnotesList",
"footnoteDefinition",
]);
/** True if `node` IS, or contains at any depth, a footnotesList/footnoteDefinition. */
function containsFootnoteNotes(node) {
if (!isObject(node))
return false;
if (FOOTNOTE_NOTES_SUBTREES.has(node.type))
return true;
if (Array.isArray(node.content)) {
return node.content.some((c) => containsFootnoteNotes(c));
}
return false;
}
/**
* Insert `marker` as a PLAIN (unmarked) text run right after the first
* occurrence of `anchor`.
@@ -83,6 +116,19 @@ export function getList(doc, predicate) {
* false when the anchor text was not found in any in-scope block.
*/
export function insertMarkerAfter(doc, anchor, marker, opts = {}) {
// A plain marker is a leading-space-padded unmarked text run.
return insertNodesAfterAnchor(doc, anchor, () => [{ type: "text", text: " " + marker }], opts);
}
/**
* Mark-safe insertion CORE: split the inline text run that holds the END of
* `anchor` (preserving the surrounding marks) and splice the nodes produced by
* `makeMiddle()` in at the split point. `insertMarkerAfter` (plain text marker)
* and `insertInlineFootnote` (a `footnoteReference` node) are both thin callers —
* the only difference is WHAT is inserted (a space-padded text run vs. a node
* that should hug the preceding word), which is exactly what `makeMiddle`
* decides. Operates on a clone; returns `{ doc, inserted }`.
*/
function insertNodesAfterAnchor(doc, anchor, makeMiddle, opts = {}) {
const out = clone(doc);
if (!isObject(out) || !Array.isArray(out.content) || !anchor) {
return { doc: out, inserted: false };
@@ -111,10 +157,25 @@ export function insertMarkerAfter(doc, anchor, marker, opts = {}) {
if (inserted || !isObject(container) || !Array.isArray(container.content)) {
return;
}
// Skip a forbidden subtree entirely (e.g. footnotesList/footnoteDefinition):
// never split into it, but keep `offset` aligned for any sibling text after
// it within this block.
if (opts.skipSubtreeTypes && opts.skipSubtreeTypes.has(container.type)) {
offset += blockPlainText(container).length;
return;
}
const inline = container.content;
// Detect whether this array is an inline array (contains text nodes).
const hasText = inline.some((n) => isObject(n) && n.type === "text");
if (hasText) {
// Refuse a textblock whose content spec cannot hold the inserted nodes
// (e.g. a codeBlock for an inline atom). Keep `offset` aligned for any
// sibling textblocks in this same block, then bail so the search falls
// through to the next candidate block.
if (opts.forbidBlockTypes && opts.forbidBlockTypes.has(container.type)) {
offset += blockPlainText(container).length;
return;
}
for (let i = 0; i < inline.length; i++) {
const n = inline[i];
const len = isObject(n) ? blockPlainText(n).length : 0;
@@ -136,8 +197,9 @@ export function insertMarkerAfter(doc, anchor, marker, opts = {}) {
if (before.length > 0) {
parts.push({ ...n, text: before, marks: [...marks] });
}
// Marker is a PLAIN run: no marks copied. Leading space separates it.
parts.push({ type: "text", text: " " + marker });
// The inserted nodes are caller-decided (a space-padded marker run,
// or a node that hugs the word). They carry no copied marks.
parts.push(...makeMiddle());
if (after.length > 0) {
parts.push({ ...n, text: after, marks: [...marks] });
}
@@ -227,14 +289,16 @@ export function noteItem(inlineNodes) {
* Wrap inline ProseMirror nodes in a real footnoteDefinition node keyed by id:
* { type:"footnoteDefinition", attrs:{id}, content:[{ type:"paragraph", content }] }
* (mirrors the editor-ext / docmost-schema FootnoteDefinition node).
*
* Built on the shared `makeFootnoteDefinition` factory (footnote-authoring.ts);
* the only extra is a fresh block id on the inner paragraph (Docmost stamps one,
* and the canonicalizer preserves attrs as-is). Single factory, one place to
* change the definition shape.
*/
export function footnoteDefinition(id, inlineNodes) {
const content = Array.isArray(inlineNodes) ? clone(inlineNodes) : [];
return {
type: "footnoteDefinition",
attrs: { id },
content: [{ type: "paragraph", attrs: { id: freshId() }, content }],
};
const node = makeFootnoteDefinition(id, inlineNodes);
node.content[0].attrs = { id: freshId() };
return node;
}
/**
* Replace every `[N]` body marker and `\u0000FN<i>\u0000` comment placeholder in
@@ -471,3 +535,97 @@ export function commentsToFootnotes(doc, comments, opts = {}) {
const synced = setCalloutRange(working, definitions.length);
return { doc: synced.doc, consumed };
}
/**
* AUTHOR-INLINE footnote insertion. The caller supplies WHERE (anchorText) and
* WHAT (markdown text); numbering and the bottom list are derived server-side by
* `canonicalizeFootnotes`. The caller never sees or edits `footnotesList`, never
* assigns a number, and cannot desync — orphans / out-of-order lists / raw
* `[^id]` markdown are structurally impossible.
*
* Content DEDUP (#3 in the issue): if an existing definition has the SAME
* normalized content key, its id is REUSED (the new reference points at it: one
* number, one definition, several references). Otherwise a fresh uuid id is
* minted and a new definition added. Conservative — only an exact content match
* merges.
*
* Mechanics: the `footnoteReference` node is inserted DIRECTLY at the anchor via
* the same mark-safe split as `insertMarkerAfter` (the shared
* `insertNodesAfterAnchor` core), so it hugs the preceding word with no text
* sentinel round-trip. The whole document is then canonicalized.
*
* Operates on a clone of `doc`. When the anchor is not found, returns the input
* unchanged with `inserted:false`.
*/
export function insertInlineFootnote(doc, opts) {
const inline = mdToInlineNodes(opts.text ?? "");
// footnoteContentKey only reads `.content`, so key off the inline array
// directly instead of building a throwaway definition node.
const key = footnoteContentKey({ content: inline });
// Content dedup: reuse an existing definition's id when its key matches.
let footnoteId = null;
let reused = false;
if (key !== "") {
walk(doc, (n) => {
if (footnoteId == null &&
isObject(n) &&
n.type === "footnoteDefinition" &&
n.attrs &&
typeof n.attrs.id === "string" &&
n.attrs.id !== "" &&
footnoteContentKey(n) === key) {
footnoteId = n.attrs.id;
reused = true;
}
});
}
if (footnoteId == null)
footnoteId = generateFootnoteId();
// Insert the footnoteReference node directly after the anchor (mark-safe
// split); it hugs the preceding word with no leading space. Two guards keep the
// inline atom out of the notes section and out of blocks that cannot hold it:
// - beforeBlock bounds the search to the BODY, before the first top-level block
// that IS or CONTAINS (at any depth) a footnotesList/footnoteDefinition — so
// a NESTED list or a bare definition also bounds the search, not just a
// top-level list;
// - skipSubtreeTypes refuses to descend into any footnotesList/footnoteDefinition
// subtree, so a reference is never glued inside an existing definition (which
// the canonicalizer would then drop as an orphan, losing that definition's
// prose); and forbidBlockTypes refuses codeBlocks (an inline atom there is a
// schema-invalid doc; insert_footnote skips validateDocStructure).
// When the only anchor match is in such a place, the insert is refused and the
// write aborts cleanly (inserted:false) instead of destroying content.
const boundaryIdx = Array.isArray(doc?.content)
? doc.content.findIndex((n) => containsFootnoteNotes(n))
: -1;
const r = insertNodesAfterAnchor(doc, (opts.anchorText ?? "").trimEnd(), () => [{ type: "footnoteReference", attrs: { id: footnoteId } }], {
...(boundaryIdx >= 0 ? { beforeBlock: boundaryIdx } : {}),
forbidBlockTypes: INLINE_ATOM_FORBIDDEN_BLOCKS,
skipSubtreeTypes: FOOTNOTE_NOTES_SUBTREES,
});
if (!r.inserted) {
return { doc: clone(doc), inserted: false, footnoteId, reused };
}
let working = r.doc;
// Add a NEW definition (canonicalize will order/place it); a reused id needs
// no new definition (the existing one is shared).
if (!reused) {
appendDefinition(working, makeFootnoteDefinition(footnoteId, inline));
}
// Derive numbering + the single bottom list deterministically.
working = canonicalizeFootnotes(working);
return { doc: working, inserted: true, footnoteId, reused };
}
/**
* Append a definition node so the canonicalizer can order/place it: into the
* first existing footnotesList, or a new trailing list when none exists.
*/
function appendDefinition(doc, defNode) {
const existingList = getList(doc, (n) => isObject(n) && n.type === "footnotesList");
if (existingList && Array.isArray(existingList.content)) {
existingList.content.push(defNode);
return;
}
if (Array.isArray(doc.content)) {
doc.content.push({ type: "footnotesList", content: [defNode] });
}
}

View File

@@ -17,6 +17,7 @@ import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
buildCollabWsUrl,
assertYjsEncodable,
@@ -60,6 +61,8 @@ import {
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
} from "./lib/transforms.js";
import vm from "node:vm";
@@ -1344,10 +1347,16 @@ export class DocmostClient {
// inject javascript:/data: link hrefs or media srcs straight into the doc.
this.validateDocUrls(doc);
// Canonicalize footnotes (idempotent): an agent-authored JSON doc cannot
// leave footnotes out of order, orphaned, or in multiple lists — the bottom
// list + numbering are always derived from reference order. No-op when the
// footnotes are already canonical.
doc = canonicalizeFootnotes(doc);
// Write the BODY first, then the title (#159 split-brain): a failed body
// write (e.g. persist timeout) must not leave a new title over the old body.
const collabToken = await this.getCollabTokenWithReauth();
const mutation = await replacePageContent(
const mutation = await this.replacePage(
pageId,
doc,
collabToken,
@@ -1368,6 +1377,95 @@ export class DocmostClient {
};
}
/**
* AUTHOR-INLINE footnote insertion. The agent supplies only WHERE
* (`anchorText`, a snippet of body text to attach the marker after) and WHAT
* (`text`, the footnote content as markdown). Numbering and the bottom
* `footnotesList` are derived deterministically server-side
* (`insertInlineFootnote` -> `canonicalizeFootnotes`): the agent never sees,
* assigns, or edits a footnote number or the list, so it CANNOT desync.
*
* Content DEDUP: when an existing definition has the same content, its id is
* reused (one number, one definition, several references). The write is atomic
* via `mutatePageContent` (single-writer, page-locked); if the anchor text is
* not found the transform aborts with a clear error and no write happens.
*/
async insertFootnote(pageId: string, anchorText: string, text: string) {
await this.ensureAuthenticated();
if (!anchorText || !anchorText.trim()) {
throw new Error("insert_footnote: anchorText is required");
}
if (text == null || `${text}`.trim() === "") {
throw new Error("insert_footnote: text is required");
}
const collabToken = await this.getCollabTokenWithReauth();
let result: { footnoteId: string; reused: boolean } | null = null;
const mutation = await this.mutatePage(
pageId,
collabToken,
this.apiUrl,
(liveDoc: any) => {
const r = insertInlineFootnote(liveDoc, { anchorText, text });
if (!r.inserted) {
// Abort the page-locked write by throwing: mutatePageContent does not
// persist when the transform throws, so a missing anchor leaves the
// page untouched (no partial write).
throw new Error(
`insert_footnote: anchor text not found: ${JSON.stringify(
anchorText.slice(0, 80),
)}`,
);
}
result = { footnoteId: r.footnoteId, reused: r.reused };
return r.doc;
},
);
// The not-found path throws inside the transform (aborting mutatePage), so by
// here `result` is always set.
const r = result!;
return {
success: true,
modified: true,
pageId,
footnoteId: r.footnoteId,
reused: r.reused,
message: r.reused
? "Footnote inserted (reused an existing same-content definition)."
: "Footnote inserted.",
verify: mutation.verify,
};
}
/**
* Page-locked write seam over collaboration.mutatePageContent. Production just
* delegates; it exists as an overridable method so the insert_footnote wrapper
* (transform abort-on-not-found + response shaping) can be unit-tested without
* standing up a live Hocuspocus collab socket.
*/
protected mutatePage(
pageId: string,
collabToken: string,
apiUrl: string,
transform: (doc: any) => any,
): Promise<{ doc?: any; verify?: any }> {
return mutatePageContent(pageId, collabToken, apiUrl, transform);
}
/**
* Full-document write seam over collaboration.replacePageContent. Production
* just delegates; it exists as an overridable method so the full-doc write
* tools (update_page_json, copy_page_content) can have their footnote-
* canonicalization binding unit-tested without a live Hocuspocus collab socket.
*/
protected replacePage(
pageId: string,
doc: any,
collabToken: string,
apiUrl: string,
): Promise<{ doc?: any; verify?: any }> {
return replacePageContent(pageId, doc, collabToken, apiUrl);
}
/**
* Export a page to a single self-contained Docmost-flavoured markdown file:
* meta block + body (with inline comment anchors + diagrams) + comment
@@ -1408,7 +1506,8 @@ export class DocmostClient {
async importPageMarkdown(pageId: string, fullMarkdown: string): Promise<any> {
await this.ensureAuthenticated();
const { meta, body, comments } = parseDocmostMarkdown(fullMarkdown);
const doc = await markdownToProseMirror(body);
// PAGE import: canonicalize footnotes (see markdownToProseMirrorCanonical).
const doc = await markdownToProseMirrorCanonical(body);
const collabToken = await this.getCollabTokenWithReauth();
const mutation = await replacePageContent(
pageId,
@@ -1503,10 +1602,16 @@ export class DocmostClient {
// (parity with updatePageJson; harmless for already-stored source content).
this.validateDocUrls(content);
// Defense-in-depth (#228): this is a FULL-document write, so canonicalize
// footnotes before copying — a no-op on already-canonical source content, but
// it guarantees a copy can never propagate a non-canonical footnote topology
// to the target (parity with the other full-doc write paths).
const canonical = canonicalizeFootnotes(content);
const collabToken = await this.getCollabTokenWithReauth();
const mutation = await replacePageContent(
const mutation = await this.replacePage(
targetPageId,
content,
canonical,
collabToken,
this.apiUrl,
);
@@ -1515,7 +1620,7 @@ export class DocmostClient {
success: true,
sourcePageId,
targetPageId,
copiedNodes: content.content.length,
copiedNodes: canonical.content.length,
verify: mutation.verify,
};
}
@@ -2033,7 +2138,10 @@ export class DocmostClient {
}
}
// Convert through the full Docmost schema (consistent with page paths)
// Convert through the full Docmost schema. Deliberately the NON-canonicalizing
// variant: a comment body may carry a footnote definition with no matching
// reference, and canonicalization would drop it (data loss). See
// markdownToProseMirror vs markdownToProseMirrorCanonical.
const jsonContent = await markdownToProseMirror(content);
const payload: Record<string, any> = {
pageId,
@@ -2136,6 +2244,7 @@ export class DocmostClient {
async updateComment(commentId: string, content: string) {
await this.ensureAuthenticated();
// NON-canonicalizing on purpose (comment body — see createComment).
const jsonContent = await markdownToProseMirror(content);
await this.client.post("/comments/update", {
commentId,
@@ -2986,6 +3095,8 @@ export class DocmostClient {
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
},
};
@@ -3022,24 +3133,33 @@ export class DocmostClient {
"transform must evaluate to a function (doc, ctx) => doc",
);
}
const result = vm.runInNewContext(
const raw = vm.runInNewContext(
"f(d, c)",
{ f: fn, d: sandbox.doc, c: ctx },
{ timeout: 5000 },
);
if (
!result ||
typeof result !== "object" ||
result.type !== "doc" ||
!Array.isArray(result.content)
!raw ||
typeof raw !== "object" ||
raw.type !== "doc" ||
!Array.isArray(raw.content)
) {
throw new Error(
'transform must return a ProseMirror doc node ({ type:"doc", content:[...] })',
);
}
// Validate the returned doc before it can be written.
this.validateDocStructure(result);
this.validateDocUrls(result);
// Validate the RAW transform output FIRST (structure — including the
// MAX_DEPTH guard — and URLs), mirroring updatePageJson. The canonicalizer
// recurses without a depth limiter, so validating after it would turn a
// too-deep doc into an opaque "Maximum call stack size exceeded" instead of
// the intended "nesting exceeds the maximum depth" error.
this.validateDocStructure(raw);
this.validateDocUrls(raw);
// Auto-canonicalize footnotes after the transform (idempotent): no write
// path can leave footnotes out of order / orphaned / in a raw `[^id]`
// block. In a dryRun preview this may surface footnote edits the script
// author did not write (the canonicalizer tidied them) — that is expected.
const result = canonicalizeFootnotes(raw);
newDoc = result;
return result;
};

View File

@@ -892,8 +892,15 @@ server.registerTool(
"mark-safe), setCalloutRange(doc, n) (sync a [1]…[K] callout range to " +
"[1]…[n]), noteItem(inlineNodes) (wrap inline nodes in a listItem with a " +
"fresh id), mdToInlineNodes(markdown) (comment markdown -> inline nodes), " +
"and commentsToFootnotes(doc, comments, {notesHeading}) (turn inline " +
"comments into numbered footnotes). Footnote convention: markers are " +
"commentsToFootnotes(doc, comments, {notesHeading}) (turn inline " +
"comments into numbered footnotes), canonicalizeFootnotes(doc) (derive " +
"footnote numbering + the single bottom list from reference order, drop " +
"orphans/duplicates — runs AUTOMATICALLY on the transform RESULT, so the " +
"applied (and dryRun-previewed) doc is always footnote-canonical; a dryRun " +
"diff may therefore show footnote tidy-ups your script did not make, and " +
"it is idempotent after the first run), and " +
"insertInlineFootnote(doc, {anchorText, text}) (author-inline footnote: " +
"marker + dedup'd definition, list derived). Footnote convention: markers are " +
"plain '[N]' text in the body; the notes are an orderedList under a " +
"heading whose text is 'Примечания переводчика'. The transform runs " +
"sandboxed (no require/process/fs/network, 5s timeout) and must return a " +
@@ -908,7 +915,8 @@ server.registerTool(
"parenthesized function). It receives a clone of the live doc and " +
"ctx (comments, log, consume(id), helpers: blockText/walk/getList/" +
"insertMarkerAfter/setCalloutRange/noteItem/mdToInlineNodes/" +
"commentsToFootnotes) and must return a {type:'doc'} node.",
"commentsToFootnotes/canonicalizeFootnotes/insertInlineFootnote) " +
"and must return a {type:'doc'} node.",
),
dryRun: z
.boolean()
@@ -934,6 +942,41 @@ server.registerTool(
},
);
// Tool: insert_footnote
server.registerTool(
"insert_footnote",
{
description:
"Insert an AUTHOR-INLINE footnote: you specify only WHERE (anchorText) " +
"and WHAT (text). The footnote marker is placed right after anchorText in " +
"the body, and the bottom footnotes list + the numbering are derived " +
"deterministically server-side. You do NOT assign a number, and you " +
"never see or edit the footnotes list — so footnotes cannot end up out " +
"of order, orphaned, or as a raw '[^id]' block. If a footnote with the " +
"SAME text already exists, its number is REUSED (one definition, several " +
"references). The write is atomic and won't clobber concurrent edits; if " +
"anchorText is not found, nothing is written and an error is returned.",
inputSchema: {
pageId: z.string().min(1),
anchorText: z
.string()
.min(1)
.describe(
"A snippet of existing body text; the footnote marker is inserted " +
"immediately after its first occurrence (mark-safe).",
),
text: z
.string()
.min(1)
.describe("The footnote content as markdown (becomes the definition)."),
},
},
async ({ pageId, anchorText, text }) => {
const result = await docmostClient.insertFootnote(pageId, anchorText, text);
return jsonContent(result);
},
);
// Tool: diff_page_versions
registerShared(
SHARED_TOOL_SPECS.diffPageVersions,

View File

@@ -11,6 +11,7 @@ import { docmostExtensions, docmostSchema } from "./docmost-schema.js";
import { withPageLock } from "./page-lock.js";
import { sanitizeForYjs, findUnstorableAttr } from "./node-ops.js";
import { lexFootnoteLines } from "./footnote-lex.js";
import { canonicalizeFootnotes } from "./footnote-canonicalize.js";
import { summarizeChange, VerifyReport } from "./diff.js";
/**
@@ -392,7 +393,20 @@ function extractFootnotes(markdown: string): {
};
}
/** Convert markdown to a ProseMirror doc using the full Docmost schema. */
/**
* Convert markdown to a ProseMirror doc using the full Docmost schema.
*
* This conversion does NOT canonicalize footnotes — it is the shared, content-
* preserving primitive used by BOTH page write paths and COMMENT bodies
* (createComment / updateComment). Canonicalization MUST NOT run on a comment
* body: a comment may legitimately contain a footnote-definition line
* (`[^1]: text`) with no matching reference, and the canonicalizer drops a
* reference-less footnotesList — which would silently delete the comment's text.
*
* Page write paths that DO need the canonical footnote topology call
* `markdownToProseMirrorCanonical` instead (markdown import, update_page markdown
* path). Keep this function reference-loss-free.
*/
export async function markdownToProseMirror(
markdownContent: string,
): Promise<any> {
@@ -403,6 +417,23 @@ export async function markdownToProseMirror(
return generateJSON(bridged, docmostExtensions);
}
/**
* Page-write variant of `markdownToProseMirror`: converts markdown then enforces
* the canonical footnote topology. The footnote `section` markdown is emitted in
* DEFINITION order, but numbering derives from REFERENCE order, so without this
* the bottom list renders out of order (`1, 4, 2, 3, …`); orphan definitions and
* duplicate lists are also normalized. Idempotent — a no-op once canonical, and a
* no-op for footnote-free content.
*
* Use this ONLY for full-document PAGE writes (never for comment bodies, where it
* would drop a reference-less footnote definition — see `markdownToProseMirror`).
*/
export async function markdownToProseMirrorCanonical(
markdownContent: string,
): Promise<any> {
return canonicalizeFootnotes(await markdownToProseMirror(markdownContent));
}
/**
* Build the collaboration WebSocket URL from an API base URL:
* switch http(s)->ws(s), strip a trailing /api, mount on /collab.
@@ -801,7 +832,9 @@ export async function updatePageContentRealtime(
collabToken: string,
baseUrl: string,
): Promise<MutationResult> {
const tiptapJson = await markdownToProseMirror(markdownContent);
// PAGE write: canonicalize footnotes (markdown import builds the bottom list in
// definition order; numbering is reference-ordered).
const tiptapJson = await markdownToProseMirrorCanonical(markdownContent);
return await mutatePageContent(
pageId,
collabToken,

View File

@@ -0,0 +1,91 @@
/**
* Inline-authoring helpers for footnotes (MCP).
*
* These build/identify footnote DEFINITION nodes for the author-inline tool
* (`insertInlineFootnote` in transforms.ts): a content key to de-duplicate notes
* by text, a definition-node factory, and a fresh uuidv7-style id generator.
*
* Split out of `footnote-canonicalize.ts` so that module stays a pure MIRROR of
* the editor-ext canonicalizer (compositionally symmetric to the editor-ext
* copy, which keeps its authoring helpers in `footnote-util.ts`). The pure
* canonicalizer has no dependency on these.
*/
const FOOTNOTE_DEFINITION_NAME = "footnoteDefinition";
function cloneJson<T>(v: T): T {
if (typeof structuredClone === "function") return structuredClone(v);
return JSON.parse(JSON.stringify(v)) as T;
}
/**
* Normalized content key for de-duplicating footnote DEFINITIONS by their text.
*
* Two definitions with the same key are the SAME footnote — so the inline
* authoring tool reuses one id (one number, one definition, several references)
* instead of minting a second definition. Key = plaintext (whitespace-collapsed,
* trimmed) PLUS a signature of the inline mark types in order, so two notes that
* read the same but differ in formatting (one bold, one plain) are NOT merged.
* Conservative: only an exact match merges.
*/
export function footnoteContentKey(defNode: any): string {
const parts: string[] = [];
const visit = (n: any): void => {
if (!n || typeof n !== "object") return;
if (n.type === "text" && typeof n.text === "string") {
const marks = Array.isArray(n.marks)
? n.marks.map((m: any) => m?.type).filter(Boolean).sort().join(",")
: "";
parts.push(`${n.text}${marks}`);
}
if (Array.isArray(n.content)) for (const c of n.content) visit(c);
};
visit(defNode);
// Collapse the assembled text's whitespace and trim, keeping the mark
// signature attached so formatting differences still distinguish notes.
return parts
.join("")
.replace(/[ \t\r\n]+/g, " ")
.trim();
}
/**
* Build a footnoteDefinition node from inline ProseMirror nodes, keyed by id.
*/
export function makeFootnoteDefinition(id: string, inlineNodes: any[]): any {
const content = Array.isArray(inlineNodes) ? cloneJson(inlineNodes) : [];
return {
type: FOOTNOTE_DEFINITION_NAME,
attrs: { id },
content: [{ type: "paragraph", content }],
};
}
/**
* Generate a uuidv7-style id (time-ordered), matching editor-ext's
* `generateFootnoteId`. Used for a genuinely-new inline footnote id.
*/
export function generateFootnoteId(): string {
const now = Date.now();
const timeHex = now.toString(16).padStart(12, "0");
const rand = (length: number) => {
let s = "";
for (let i = 0; i < length; i++)
s += Math.floor(Math.random() * 16).toString(16);
return s;
};
const versioned = "7" + rand(3);
const variantNibble = (8 + Math.floor(Math.random() * 4)).toString(16);
const variant = variantNibble + rand(3);
return (
timeHex.slice(0, 8) +
"-" +
timeHex.slice(8, 12) +
"-" +
versioned +
"-" +
variant +
"-" +
rand(12)
);
}

View File

@@ -0,0 +1,225 @@
/**
* Server-side footnote canonicalizer (MCP mirror — PURE).
*
* `canonicalizeFootnotes(doc)` is a pure ProseMirror-JSON port of the editor's
* `footnoteSyncPlugin` end-state, identical in behaviour to
* `@docmost/editor-ext`'s `canonicalizeFootnotes`. It is mirrored here — rather
* than imported from editor-ext — for the SAME reason `footnote-lex.ts` and the
* `docmost-schema.ts` nodes are mirrored: the MCP package is deliberately
* decoupled from the browser/React-heavy editor barrel and operates on plain
* JSON. The editor-ext copy owns the golden test against the live plugin; this
* copy must stay behaviourally identical (a SHARED golden corpus, exercised by
* both test suites, pins that — see `test/unit/footnote-corpus.mjs`).
*
* This module is the pure MIRROR only. The inline-authoring helpers
* (`footnoteContentKey`, `makeFootnoteDefinition`, `generateFootnoteId`) used by
* `insertInlineFootnote` live in the sibling `footnote-authoring.ts`, so this
* file is compositionally symmetric to the editor-ext copy.
*
* Why it exists: every NON-editor write path (markdown import, update_page_json,
* docmost_transform, insert_footnote) builds ProseMirror JSON directly, so the
* editor's footnote plugins never run and the canonical topology (sequential
* numbering by first reference, one trailing list, no orphans, no raw `[^id]`)
* was never enforced. Running this at the end of every write path closes that
* gap; because it is idempotent, it is a no-op when the footnotes are already
* canonical (no spurious mutations / git-sync churn).
*
* ENFORCEMENT RULE (#228): any NEW FULL-document persist path MUST call
* `canonicalizeFootnotes(doc)` before writing — the current callers are
* `markdownToProseMirrorCanonical` (page markdown import/update; the plain
* `markdownToProseMirror` used for COMMENT bodies must NOT, or it would drop a
* reference-less definition), `update_page_json`, `docmost_transform`,
* `insert_footnote`, and `copy_page_content`. Append/prepend FRAGMENT writes MUST
* NOT canonicalize. This is deliberately per-call-site (the replace-vs-fragment
* and comment-vs-page nuances make a single naive wrapper unsafe).
*/
const FOOTNOTE_REFERENCE_NAME = "footnoteReference";
const FOOTNOTES_LIST_NAME = "footnotesList";
const FOOTNOTE_DEFINITION_NAME = "footnoteDefinition";
function cloneJson<T>(v: T): T {
if (typeof structuredClone === "function") return structuredClone(v);
return JSON.parse(JSON.stringify(v)) as T;
}
function isEmptyParagraph(node: any): boolean {
return (
!!node &&
node.type === "paragraph" &&
(!Array.isArray(node.content) || node.content.length === 0)
);
}
function collectReferenceIds(node: any, out: string[], seen: Set<string>): void {
if (!node || typeof node !== "object") return;
if (node.type === FOOTNOTE_REFERENCE_NAME) {
const id = node?.attrs?.id;
if (id && !seen.has(id)) {
seen.add(id);
out.push(id);
}
}
if (Array.isArray(node.content)) {
for (const child of node.content) collectReferenceIds(child, out, seen);
}
}
function collectDefinitions(node: any, out: any[]): void {
if (!node || typeof node !== "object") return;
if (node.type === FOOTNOTE_DEFINITION_NAME) out.push(node);
if (Array.isArray(node.content)) {
for (const child of node.content) collectDefinitions(child, out);
}
}
function emptyDefinition(id: string): any {
return {
type: FOOTNOTE_DEFINITION_NAME,
attrs: { id },
content: [{ type: "paragraph" }],
};
}
/**
* Deep equality over plain JSON: arrays are compared POSITIONALLY
* (order-SENSITIVE), object keys order-insensitively. The array order-sensitivity
* is required for correctness here — a reordered `footnotesList.content` must
* compare UNEQUAL so the canonical rebuild fires instead of leaving it in place.
*/
function deepEqualJson(a: any, b: any): boolean {
if (a === b) return true;
if (a == null || b == null || typeof a !== typeof b) return false;
if (Array.isArray(a) || Array.isArray(b)) {
if (!Array.isArray(a) || !Array.isArray(b) || a.length !== b.length) {
return false;
}
for (let i = 0; i < a.length; i++) {
if (!deepEqualJson(a[i], b[i])) return false;
}
return true;
}
if (typeof a === "object") {
const ka = Object.keys(a);
const kb = Object.keys(b);
if (ka.length !== kb.length) return false;
for (const k of ka) {
if (!Object.prototype.hasOwnProperty.call(b, k)) return false;
if (!deepEqualJson(a[k], b[k])) return false;
}
return true;
}
return false;
}
/**
* Canonicalize footnotes in a ProseMirror-JSON document. See the file header and
* the editor-ext twin for the full contract. Pure (deep-clones input,
* deterministic, idempotent).
*/
export function canonicalizeFootnotes<T = any>(doc: T): T {
if (
doc == null ||
typeof doc !== "object" ||
!Array.isArray((doc as any).content)
) {
return doc;
}
const out = cloneJson(doc) as any;
// 1) Distinct reference ids in document order (deep — refs can live in
// callouts, tables, list items, ...). The ordering/numbering truth.
const referenceIds: string[] = [];
collectReferenceIds(out, referenceIds, new Set<string>());
// 2) Every definition node in document order (deep).
const defNodes: any[] = [];
collectDefinitions(out, defNodes);
// 3) First definition per id wins; later duplicates carry the SAME id, so they
// cannot be referenced separately and would be orphans — they are dropped.
const defById = new Map<string, any>();
for (const d of defNodes) {
const id = d?.attrs?.id;
if (id && !defById.has(id)) defById.set(id, d);
}
// 4) Build the ordered definition list: one per referenced id, in REFERENCE
// order, reusing the existing node (shallow-copied, id normalized — `out` is
// already deep-cloned and the old lists are cut) or synthesizing an empty
// one. Definitions whose id is not referenced are orphans and never added.
const orderedDefs: any[] = [];
for (const id of referenceIds) {
const existing = defById.get(id);
if (existing) {
orderedDefs.push({
...existing,
attrs: { ...(existing.attrs ?? {}), id },
});
} else {
orderedDefs.push(emptyDefinition(id));
}
}
// 5) No references -> there must be NO list at all (at any depth).
if (referenceIds.length === 0) {
stripFootnotesListsDeep(out);
return out;
}
// 6) Placement parity with the live plugin: when the document is ALREADY in the
// canonical single-list state, leave that list exactly where it sits rather
// than cutting and re-inserting it at the end (the plugin never repositions a
// sole correct list, so moving it would silently reorder any content that
// follows the list on the first write).
const topLevelLists = out.content.filter(
(n: any) => n && n.type === FOOTNOTES_LIST_NAME,
);
if (
topLevelLists.length === 1 &&
defNodes.length === orderedDefs.length &&
deepEqualJson(topLevelLists[0].content, orderedDefs)
) {
return out;
}
// 7) Otherwise rebuild: strip every footnotesList AND every bare
// footnoteDefinition at ANY depth (collectDefinitions gathers defs
// recursively, so a list nested in a callout/blockquote — or a bare
// definition outside any list — would otherwise have its defs copied into the
// rebuilt list while the original survives in place → duplicates) and
// re-insert exactly one list after the last meaningful (non-empty paragraph)
// top-level block.
stripFootnotesListsDeep(out);
stripFootnoteDefinitionsDeep(out);
const top: any[] = out.content;
let insertAt = top.length;
while (insertAt > 0 && isEmptyParagraph(top[insertAt - 1])) insertAt--;
top.splice(insertAt, 0, { type: FOOTNOTES_LIST_NAME, content: orderedDefs });
out.content = top;
return out;
}
/** Remove every `footnotesList` node at ANY depth (mutates the given clone). */
function stripFootnotesListsDeep(node: any): void {
if (!node || typeof node !== "object" || !Array.isArray(node.content)) return;
node.content = node.content.filter(
(c: any) => !(c && c.type === FOOTNOTES_LIST_NAME),
);
for (const child of node.content) stripFootnotesListsDeep(child);
}
/**
* Remove every BARE `footnoteDefinition` node at ANY depth (mutates the given
* clone). Runs only in the rebuild path AFTER the lists are stripped, so it
* targets definitions that were sitting outside a list (e.g. hand-authored via a
* raw-JSON write path and nested in a callout); their content was already copied
* into the rebuilt list, so leaving the originals would duplicate them.
*/
function stripFootnoteDefinitionsDeep(node: any): void {
if (!node || typeof node !== "object" || !Array.isArray(node.content)) return;
node.content = node.content.filter(
(c: any) => !(c && c.type === FOOTNOTE_DEFINITION_NAME),
);
for (const child of node.content) stripFootnoteDefinitionsDeep(child);
}

Some files were not shown because too many files have changed in this diff Show More