Compare commits

...

36 Commits

Author SHA1 Message Date
agent_coder babc42c2ff fix(#346 review F1-F4): no 206-compress + Vary + precompress VAD + cache test
- F1 [HIGH — data corruption]: @fastify/compress was compressing 206/Range
  attachment responses while Content-Range still described the RAW offsets, so a
  resuming client (curl -C -, download managers) appended encoded bytes as raw →
  corrupted file. sendFileResponse now sets the request header `x-no-compression`
  (the documented @fastify/compress opt-out — its onSend skips when the request
  carries it; the reviewer's `Content-Encoding: identity` does NOT work because
  compress explicitly excludes `identity` and overwrites it). This opts the whole
  download route (both 200 full-file and 206 range) out of on-the-fly compression
  — correct, since attachment bytes are final and mostly binary.
- F2: static responses now emit `Vary: Accept-Encoding` (the preCompressed
  content-negotiated /assets/* were `immutable` without Vary → shared-cache could
  serve a brotli variant to an identity/gzip-only client).
- F3: vite compression `include` extended to .wasm/.onnx so the VAD binaries
  (~26MB .wasm, ~2.3MB .onnx under public/vad) are precompressed at build (.br
  emitted) instead of runtime-brotli'd on every request. (include REPLACES the
  plugin default, so the default js/css/json/html set is re-listed.)
- F4: extracted the cache classification into a pure `resolveStaticAssetHeaders`
  + static.module.spec.ts (3 tests: /assets/* immutable+Vary, index.html
  no-store, non-hashed not-immutable).

Gate: server tsc 0 (deps present), static.module.spec 3/3, client build emits
.wasm.br/.onnx.br, frozen install 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 04:13:20 +03:00
agent_coder 6ee814b7f3 perf(delivery): pre-compress static + cache headers + compress API responses (#346)
Cold load served ALL static + API responses uncompressed and without cache
headers (~3.7MB over the wire). Delivery only — feature behavior unchanged; no
DB/API-contract/MCP changes.

- apps/client/vite.config.ts: vite-plugin-compression2 emits .br + .gz next to
  each built asset (excludes index.html, which the server rewrites at boot with
  window.CONFIG — a precompressed copy would go stale). Build emits 187 .br /
  175 .gz under dist/assets.
- static.module.ts: @fastify/static `preCompressed: true` serves the .br/.gz
  neighbour; `setHeaders` sets `immutable` ONLY for content-hashed /assets/*,
  `no-cache` for index.html, and leaves non-hashed files (locales, vad, icons,
  manifest) on default etag/last-modified revalidation.
- main.ts: @fastify/compress (threshold 1024) compresses dynamic API JSON + the
  rewritten share-SEO HTML. SSE is safe on two counts: `text/event-stream` is not
  mime-db-compressible (allowlist skips it) AND the AI-chat stream hijacks the raw
  socket (pipeUIMessageStreamToResponse -> res.raw), bypassing the Fastify onSend
  lifecycle entirely. No double-compression with preCompressed static (compress
  skips already-Content-Encoding'd responses).
- docker-compose.yml: comment recommending an optional HTTP/2 + brotli reverse
  proxy (not required).

Deps: apps/client vite-plugin-compression2 2.5.3 (dev), apps/server
@fastify/compress 9.0.0 (matches fastify 5.8.5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 04:13:20 +03:00
vvzvlad 3085ec1b50 Merge pull request 'fix(metrics): статика в bounded «static»-лейбл — кардинальность route (#362)' (#366) from fix/362-metrics-route-cardinality into develop
Reviewed-on: #366
2026-07-06 03:55:02 +03:00
agent_vscode 05ec9feaf9 test(int): unhang test:int — forceExit + bounded destroyTestDb (#382)
Once the ESM transform fix (8e125799) let the four previously-unparseable
int-specs run, the server integration suite stopped exiting: all 66 tests pass,
then jest prints "Jest did not exit one second after the test run has
completed." and idles until the 20-minute job timeout kills it. Separately,
ai-chat-stream.int-spec.ts failed because its afterAll (destroyTestDb) hit the
60s hook timeout — postgres.js .end() waits for in-flight queries forever, so a
leaked/stuck pooled connection hung teardown.

Pragmatic unblock (the underlying open-handle leak is tracked in #382):

- jest-integration.json: add forceExit so jest always exits after the run even
  if a suite leaves an open handle.
- db.ts: capture the singleton's raw postgres sql instance and bound the pool
  shutdown with sql.end({ timeout: 5 }) (the same bounded-end pattern already
  used in global-setup.ts) instead of Kysely.destroy(), so destroyTestDb can no
  longer hang the afterAll hook.

forceExit masks residual handle leaks rather than fixing them; the proper
investigation (run with --detectOpenHandles on a pg+redis stand, close the
leaking timers/sockets, then drop forceExit) is filed as #382.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 01:28:31 +03:00
agent_vscode 8e12579925 fix(ci): transform ESM @docmost/prosemirror-markdown in server int/e2e jest
The develop build failed in two jobs, both rooted in the ESM-only workspace
package @docmost/prosemirror-markdown (type: module, built to build/index.js
with `export * from`), which the server imports at runtime (collaboration.util,
page.service). Refactor #345 taught only the unit jest config (package.json
"jest" key) to consume it, leaving the integration and e2e configs — and the
e2e-server CI job — broken:

- `pnpm --filter server test:int` -> SyntaxError: Unexpected token 'export'
  (jest did not transform prosemirror-markdown/build/*.js).
- e2e-server job -> TS2307 Cannot find module '@docmost/prosemirror-markdown'
  (the package was never built in that job).

Mirror the proven unit config into the two failing jest configs and add the
missing build step:

- jest-integration.json / jest-e2e.json: add a babel-jest transform rule for
  `prosemirror-markdown/build/.+\.js$` (before the ts-jest rule so it wins) and
  add @docmost/prosemirror-markdown to the transformIgnorePatterns allowlist so
  the pnpm-symlinked package is transformed instead of ignored.
- develop.yml: build @docmost/prosemirror-markdown in the e2e-server job (after
  editor-ext, before migrations), like the test.yml job already does.

Verified locally: an isolated spec importing the package fails with the exact
SyntaxError under the old config and passes under the new one.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 00:36:31 +03:00
vvzvlad 20703d06c2 Merge pull request 'feat(gitmost-bridge): вставка transcript в страницу записи (#377)' (#378) from fix/377-bridge-transcript into develop
Reviewed-on: #378
2026-07-06 00:06:59 +03:00
agent_coder dab2660999 fix(gitmost-bridge): нейтрализовать сплошные thematic breaks в транскрипте
Round-1 нейтрализация ловила только spaced-разделители (`- - -`),
но пропускала сплошные `---`/`***`/`___`: на git-sync round-trip такая
строка становится horizontalRule, а он не несёт текста — строка терялась
целиком (хуже list/quote-порчи). Символ `_` вообще отсутствовал в regexе.

GITMOST_MD_BLOCK_TRIGGER_RE дополнен альтернативой на целую строку-
thematic-break `([-*_])(?:\s*\1){2,}\s*$` (3+ одинаковых `-`/`*`/`_`,
solid или через пробел); прежние группы переведены в non-capturing,
чтобы `\1` ссылался на единственную capture-группу. Обе копии regexа
(bridge и pm-markdown тест) синхронны.

Тесты: bare `---`/`***`/`___` документированы как text-losing
horizontalRule; ZWSP-нейтрализованная форма round-trip'ится параграфом
с сохранённым текстом. Кейсы падают на старом regexе.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 00:02:29 +03:00
agent_coder 751d55e9db fix(gitmost-bridge): neutralize col-0 block triggers in transcript lines + lock text-not-HTML (#378 review round 1)
Round-1 review found two issues:
- [regressions] Inserting a transcript line VERBATIM as a paragraph is unsafe on the
  git-sync doc->markdown->doc round-trip: the paragraph serializer emits text with no
  block-escape, so a line starting at col 0 with `- `/`> `/`# `/`1. `/```` ``` ````/`|`/
  `> [!info]` silently re-parses into a list/quote/heading/code/table/callout (the last
  hits the #359 callout machinery). Safe under the host contract (every line prefixed
  `You:`/`Speaker N:`, which starts with a letter) but the helper didn't enforce it, and
  leading-whitespace / stray lines exist. Fix: trim each kept line (drops the indent leak),
  and if it STILL begins with a col-0 markdown block trigger, prepend an invisible
  zero-width space (U+200B) so the trigger isn't at col 0 — the round-trip keeps it a
  paragraph. (Backslash-escape was rejected: `marked` consumes the `\` on re-import, so it
  would render visibly then vanish. ZWSP is invisible and never markdown-escaped.) The
  serializer's missing block-escape is the pre-existing root cause; this is the boundary
  defense. Prefixed transcript lines never match the trigger regex → left byte-exact.
- [test-coverage] The "inserted as TEXT, not HTML/markdown" contract wasn't locked (all
  test strings were plain alphabet). Added a test that inserts `<b>bold</b>
  <script>alert(1)</script> and *stars* and [link](x)` (with Bold/Italic/Link marks
  registered so an insertContent(html) regression would parse them) and asserts one
  verbatim text node, no marks, and getHTML() has no live tags.

Verified: apps/client tsc --noEmit 0 errors; gitmost bridge tests 4 passed; a NEW
prosemirror-markdown round-trip test (real convertProseMirrorToMarkdown ->
markdownToProseMirror) proves bare triggers corrupt while the ZWSP-neutralized form stays
a single byte-preserved paragraph and normal You:/Speaker N: lines round-trip byte-exact —
3 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 00:01:25 +03:00
agent_coder 02308012a6 feat(gitmost-bridge): insert transcript into the recording page (#377)
The native gitmost.app (stage 2) now sends a `transcript` field to
window.gitmost.createPageWithRecording, but the web bridge ignored it — the page
was created with audio only. Insert it.

- gitmost-recording.ts: add `transcript?: string` to GitmostCreatePagePayload
  (plain text, \n-separated `You:` / `Speaker N:` lines, ready to insert). Add a
  gitmostInsertTranscriptIntoEditor(editor, transcript) helper: a "Transcript"
  heading + one paragraph per non-empty line, each line inserted VERBATIM as a
  text node (never HTML → no injection), appended at doc end (below the audio).
  No-op when transcript is undefined/empty/whitespace-only/non-string.
- gitmost-global-bridge.tsx (createPageWithRecording): after the audio insert
  succeeds, call the helper inside a try/catch — best-effort, so a transcript
  failure can never turn the already-successful recording into an error (logs +
  still returns ok). Absent transcript → audio-only page, exactly as today.

DoD: recording with speech → page has audio AND a labeled transcript block;
without transcript → unchanged. Closes the web-side dependency of gitmost-app
stage 2. Verified: apps/client tsc --noEmit 0 errors; 2 unit tests
(transcript present → heading+paragraphs; undefined/""/whitespace/number/object
/null → no-op, doc unchanged).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 00:01:25 +03:00
vvzvlad 0665fcb630 Merge pull request 'fix(queue): убрать мёртвую очередь {search-queue} (#379)' (#380) from fix/379-remove-dead-search-queue into develop
Reviewed-on: #380
2026-07-05 23:44:54 +03:00
agent_coder 97bd554cb5 fix(queue): remove the dead {search-queue} — producers with no consumer (#379)
The {search-queue} BullMQ queue had producers on every page/space/workspace change
but NO consumer/@Processor — it lives in Docmost's EE edition (external search
drivers), never in this fork. Search works independently via the pages tsvector DB
trigger; the queue was pure dead weight, growing forever (1902 stuck jobs in Redis,
the first real hit of the #355 queue-growing alert — which fired correctly).

Remove the plumbing:
- the 3 producers: drop @InjectQueue(SEARCH_QUEUE) + every searchQueue.add(...) from
  page/space/workspace.listener.ts (and the isTypesense() gate that only wrapped the
  search enqueue). page.listener.handlePageUpdated did ONLY the search enqueue, so its
  @OnEvent(PAGE_UPDATED) handler is removed; the other handlers keep their aiQueue.add.
- the registration (queue.module.ts) and the metrics injection + 'search' depth-metric
  entry (metrics-bull.service.ts).
- the constants: QueueName.SEARCH_QUEUE + the 6 unused SEARCH_INDEX_* QueueJob members
  (grep-confirmed unreferenced). Shared QueueJob members (PAGE_*, SPACE_DELETED,
  WORKSPACE_DELETED — used by the AI queue / embedding/history/notification processors)
  are kept.

Search (tsvector trigger, search.service) and the PAGE_UPDATED websocket listener are
untouched. Verified: apps/server tsc --noEmit 0 errors; 6 spec suites / 52 tests green.

Ops (maintainer, out of code scope): one-time Redis cleanup
`redis-cli --scan --pattern "bull:{search-queue}:*" | xargs -r redis-cli del`, then
confirm bullmq_queue_depth{queue="search"} stays 0 (the metric no longer injects it).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 23:35:04 +03:00
vvzvlad f77a6b42de Merge pull request 'docs: how to test the application (browser E2E + out-of-band)' (#376) from docs/how-to-test into develop
Reviewed-on: #376
2026-07-05 22:40:12 +03:00
C9 Tester 134b627806 docs: add how-to-test.md (browser E2E + out-of-band) and link from AGENTS.md
Adds a testing guide covering how to verify features against a running stand:
drive the behaviour under test through the browser (not the API), verify
out-of-band in the DB/git, and the non-obvious traps. Notably the page has two
ProseMirror editors — [aria-label='Page title'] (non-collab) and
[aria-label='Page content'] (the collab body); querySelector('.ProseMirror')
returns the title, so tests must target the body editor and wait ~10s for the
hocuspocus store debounce. Links the new doc from AGENTS.md next to dev-stand.md
and adds a matching gotcha #8 to dev-stand.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 22:39:01 +03:00
vvzvlad 3267512ed9 Merge pull request 'refactor(#345): серверный экспорт/импорт markdown через @docmost/prosemirror-markdown' (#369) from refactor/345-server-converter into develop
Reviewed-on: #369
2026-07-05 20:41:30 +03:00
vvzvlad 48bd27b83c Merge pull request 'test(#351 PR 1): генеративное round-trip-тестирование конвертера — атрибутный уровень' (#373) from test/351-generative-converter into develop
Reviewed-on: #373
2026-07-05 20:40:40 +03:00
vvzvlad 265b81c93d Merge pull request 'fix(db): миграции «задним числом» из долгоживущих веток не роняют старт — CI-гейт + allowUnorderedMigrations (#363, инцидент #361)' (#365) from fix/363-migration-order into develop
Reviewed-on: #365
2026-07-05 20:40:06 +03:00
vvzvlad ed808876be Merge pull request 'fix(ai): patch ai@6.0.134 — drop O(n²) partialOutput accumulation causing heap OOM on long agent runs (#184)' (#368) from fix/ai-sdk-partial-output-oom into develop
Reviewed-on: #368
2026-07-05 20:39:51 +03:00
vvzvlad a72ddbbe86 Merge pull request 'refactor(ai-chat): единый реестр спеков инструментов — унификация tables/pages/misc/comments (#294)' (#367) from refactor/294-spec-registry-cont into develop
Reviewed-on: #367
2026-07-05 20:39:41 +03:00
agent_coder d8fc724d90 test(ai): cover the partialOutput PRESERVE branch of the ai@6.0.134 patch (#184, review F1)
The patch forks createOutputTransformStream: output==null skips partialOutput
(the OOM fix, already tested), output!=null preserves the original cumulative
accumulation. Only the skip branch was tested; the preserve branch — on which the
patch's "byte-identical when an output strategy is set" safety claim rests — had no
coverage, so a future re-port (patches are re-created via `pnpm patch` on every ai
bump) could silently route output-set calls into the skip branch and leave
partialOutput empty for object/text-output consumers, uncaught.

Add a 4th test: streamText({ ..., experimental_output: Output.text() }), drain
textStream, collect experimental_partialOutputStream, and assert it is non-empty and
cumulative (last partial == full text "Hello, world!"). Reuses the existing
makeModel() harness. Verified on the patched dist: partials are
["Hello","Hello, ","Hello, world!"]. `npx jest ai-sdk-partial-output.patch.spec.ts`
→ 4 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 19:35:14 +03:00
vvzvlad e4bfbcabaa Merge pull request 'feat(#371): редизайн модалки каталога ролей — карточки-наборы + per-role результаты импорта' (#375) from feat/371-roles-catalog into develop
Reviewed-on: #375
2026-07-05 16:43:19 +03:00
agent_coder 4c1ee50dc9 test(#351): close the mark-attr coverage hole + reclassify table spans (review round 1)
F1 [WARNING] The 'no invisible coverage hole' guard enumerated only
schema.nodes, so MARK attributes silently escaped the value-fuzz completeness
check — link.internal/target/rel/class are never fuzzed and nothing flagged it,
and a new attributed mark would slip through. Added allSchemaMarkAttrKeys() plus a
MARK_ATTR_FUZZED / MARK_ATTR_ALLOWLIST registry and two tests: every schema mark
attr must be in exactly one set (a new one turns it red), and neither set may hold
a stale row.

F2 [WARNING] The ACCEPTED annotation misclassified table colspan/rowspan as
having 'no md representation'. They DO round-trip — a spanned cell makes the
converter emit the whole table as a raw <table> with colspan/rowspan, which the
tiptap parser reads back. They are frozen only because generating a
geometrically-valid spanned table is deferred PR-2 structural work (the flat
generator hardcodes span = 1), not a markdown limit. Reclassified them as
DEFERRED-BUG (distinct from ACCEPTED) so a maintainer does not read them as an
inherent limitation; colwidth / backgroundColor(Name) stay ACCEPTED (the
raw-<table> fallback drops them).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 06:40:13 +03:00
agent_coder b8cce4f814 fix(#371): skipped role is not 'allInstalled', test the reason->action branch (review round 1)
F1 [WARNING] bundlePhase returned 'allInstalled' when a bundle's only
non-installed role was skipped (0 installed for it), so the collapsed green 'All
installed · up to date' header contradicted the open 'Installed 0 · 1 skipped'
plaque. It now returns 'mixed' whenever a skipped role is present. Fixed the test
that encoded the wrong behavior.

F2 [WARNING] The reason->action branch (name-conflict -> transient overlay +
'Rename & install'; already-installed -> informational, no button) lived only in
the component, untested. Extracted the two decisions into pure, unit-tested
helpers nameConflictSlugs() and partialOffersRename() and wired them into the
modal; both reason values are now covered.

F3 [low] Removed the unused useRef import (client eslint no-unused-vars is off, so
it shipped silently).

F4 [low] Extracted bundleCounts() as the single tally pass; bundlePhase and the
panel both derive from it instead of rescanning the roles array ~5x per render
(the same model<->component consolidation this PR is about).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 06:09:58 +03:00
agent_coder a325ddbabd feat(#371): roles catalog modal redesign — bundle cards + per-role import results
Integrates the designer-handoff Roles Catalog modal, wired to the real API; the
parent ai-agent-roles.tsx and the { opened, onClose, roles } contract are
unchanged.

- Server importFromCatalog now returns per-role lists (createdRoles /
  skippedRoles with a reason) alongside the existing counters (compat-preserving),
  so the UI can name the conflicting/installed roles.
- New pure view-model (catalog-bundle-model.ts): bundlePhase (empty | allNew |
  allInstalled | updates | mixed, ignoring the transient 'skipped'),
  installedLangForRole (same-slug-different-language hint), mapCatalogRoleToView —
  all unit-tested without mounting.
- Bundle cards with a summary status in the collapsed header (eager useQueries
  fan-out over all bundles, sharing the existing per-bundle cache keys), a single
  primary action per bundle, checkboxes + select/deselect-all, an inline result
  plaque that keeps the modal open, per-bundle and global 'Update all' request
  series with progress, and the other-language hint.
- The partial-result plaque distinguishes the skip reason: only a name-conflict
  offers 'Rename & install'; an already-installed race is informational (a rename
  re-import would just skip again and self-heal into a false success).
- All strings i18n'd (en/ru); mock handoff code (SEED/mockImport/delay) removed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 05:35:58 +03:00
agent_coder bfcee6dddc test(prosemirror-markdown): generative round-trip testing — attribute level, flat docs (#351 PR 1)
Schema-derived, property-based (fast-check) round-trip tests over flat
single-node ProseMirror documents. One test PR — src/ is untouched; the two
real bugs found are pinned as loud it.fails counterexamples, not fixed here.

- attr-arbitraries.ts: per-attribute four-state arbitraries (absent/default/
  nonDefault/degenerate), attribute list sourced from schema.nodes[t].spec.attrs;
  a documented override table supplies legal domains for constrained attrs and
  distinguishes two frozen classes explicitly — ACCEPTED limitations (no md
  representation) vs PINNED bugs (representable but dropped, tracked as
  counterexamples).
- text-arbitraries.ts: hostile text corpus (ported from the existing property
  test's supported-space guarantees).
- node-generators.ts: flat single-node generators + a completeness contract —
  every one of the schema's 45 nodes / 12 marks is either generated or listed in
  KNOWN_UNCOVERED with a reason.
- flat-roundtrip.property.test.ts: P1 (semantic round-trip via
  docsCanonicallyEqual), P2 (second-pass byte fixpoint — anti GS-EDIT-REVERT),
  P3 (totality), generator validity via schema.check(), and an explicit
  attribute-value-coverage snapshot so the not-fuzzed set can never grow silently.
- counterexamples: column.width (% dropped on parseFloat -> P2 churn) and
  orderedList.start (non-1 start renders as '1.' -> P1 loss) pinned as it.fails.

SEED=20250705, NUM_RUNS=300 per property; ~17s, no OOM (union arbitraries).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 04:38:40 +03:00
agent_coder 36b940fdb8 fix(#294 review F1-F2): test the changed execute wirings + transport-neutral descriptions
- F1: added in-app execute tests for the two wirings that ACTUALLY changed in the
  migration (the contract-parity test only checks advertised schema keys, not
  execute bodies): movePage forwards the newly-added optional `position` to
  client.movePage (and passes undefined position + null parent when omitted); the
  table trio (insert/delete/updateCell) forwards the unified `table` param
  positionally. A field destructured under the wrong name would have silently
  passed undefined to the client (execute is any-cast, tsc won't catch it).
- F2: rewrote the three migrated descriptions that hardcoded snake_case sibling
  tool names (which the in-app camelCase layer exposes under different ids,
  violating the registry's own transport-neutral-prose convention) into neutral
  prose: getPage "use get_page_json" -> "use the lossless page-JSON read tool";
  updatePageJson "get_page_json -> ... -> update_page_json" -> "read the page-JSON
  view -> modify -> write it back", "prefer rename_page" -> "prefer the rename-page
  tool"; exportPageMarkdown "import_page_markdown round-trip" -> "page-Markdown
  import round-trip" (the last was a direct regress — the in-app base said the
  camelCase importPageMarkdown). (stashPage's pre-existing get_page_json mention is
  out of scope, per the reviewer.)

Gate: mcp build 0; ai-chat-tools.service + tool-tiers (catalog-partition) pass,
incl. the 5 new execute-wiring tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 03:10:42 +03:00
agent_coder 0050ad7ebb docs(#363 review F2): update AGENTS.md migration-ordering to the new tolerant behavior
The "Migration ordering" section still described the OLD crash-loop-at-boot
behavior this PR removes ("Kysely refuses to start … rejected at boot"). Rewrote
it to the new two-layer model: the CI migration-order gate is the primary defense
(rename to a current timestamp), and the runtime now sets allowUnorderedMigrations
so the app applies a back-dated migration instead of crash-looping (with the note
that #ensureNoMissingMigrations still guards a removed applied migration, and that
migrations must stay independent since apply order can differ across instances).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 02:57:11 +03:00
agent_coder 43b11d92ab fix(#362 review F1-F3): add /icons/ prefix + honest comment + real boundary test
- F1: added '/icons/' to STATIC_PATH_PREFIXES — public/icons/ is copied verbatim
  to client/dist (a sibling of the already-included brand/vad/locales), and
  index.html references /icons/favicon-*.png on every page load, so those requests
  were getting their own route labels instead of collapsing to `static`.
- F2: corrected the comment — only /assets/ is content-hashed (unbounded per
  deploy); /vad//brand//locales//icons/ have stable names (repetitive, not
  unbounded). Either way none belong in the API-route histogram.
- F3: the negative test now exercises the trailing-slash boundary (the actual
  anti-false-collapse guard): '/assets' (no slash), '/assetsx/foo.js',
  '/iconset/x.png' must NOT collapse to `static` — cases that a buggy
  includes()/slashless-prefix impl would wrongly collapse. Plus '/icons/*' added
  to the positive it.each.

Gate: server tsc 0; metrics.spec passes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 02:38:47 +03:00
agent_coder ce70fab1df refactor(ai-chat): unify share_page into SHARED_TOOL_SPECS (#294, misc family)
Migrates share_page / sharePage into the transport-agnostic spec registry
(schema + description declared once; each transport keeps only its execute/auth):
- sharePage (deferred) -> SHARED_TOOL_SPECS; index.ts uses registerShared(),
  ai-chat uses sharedTool(); removed from INLINE_TOOL_TIERS.

Drift reconciled (documented inline): both inline copies already carried the
"only share when the user explicitly asked" security framing, so the old
"per-transport divergence" note in BOTH layers was STALE — there was no real
behavioral divergence, only wording drift. The canonical description merges the
MCP copy's URL-format + idempotency detail with the in-app copy's reversibility
note and keeps the shared security framing. pageId keeps the MCP copy's stricter
.min(1). The MCP execute keeps its own `searchIndexing ?? true` default
(per-layer, not part of the shared schema).

Intentionally NOT migrated (kept inline — genuinely divergent, as their existing
notes state):
- search / searchPages: the in-app tool is a semantic+keyword hybrid (RRF) with
  in-process access control and a tuned schema (limit 1-20); the MCP `search` is
  a plain REST full-text search (limit up to 100). Different behavior AND schema.
- docmost_transform / transformPage: the in-app tool deliberately omits the
  `deleteComments` schema field (a comment-deletion guardrail) and carries a
  shorter description. Different schema.

Gate: mcp build 0 + node --test 458/458 (page-search excluded — hangs only under
the local re2->RegExp type-shim, its source untouched), server jest 775 incl.
tool-tiers catalog-partition + shared-spec contract parity, server tsc 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 02:27:50 +03:00
agent_coder 7b4617db70 fix(#363 review F1): make the migration-order gate fail CLOSED (not open)
The CI gate — whose whole job is to BLOCK a back-dated migration — could pass
open in exactly the scenario it guards (a long branch vs a moving base, i.e. #361):

- Dropped the redundant `git fetch --depth=1`: the checkout already did
  fetch-depth:0 (full history), and the shallow graft truncated the BASE history,
  so `merge-base` (thus the three-dot `origin/base...HEAD` diff) failed when the
  base had moved ahead of the PR merge commit.
- Removed `|| true` on the diff: it swallowed that failure → `added` empty → loop
  skipped → bad=0 → gate PASS. Now `set -e` aborts the job (fail CLOSED) on any
  diff error — a gate must never pass on error.

Verified: yaml parses (jobs migration-order, test); a broken-ref diff with set -e
and no `|| true` aborts before bad=0 (fail-closed) instead of passing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 02:27:16 +03:00
agent_coder b51dae16a6 docs(mcp): mark media tools MCP-only in index.ts (#294, media family)
The media tools — insert_image, replace_image, insert_footnote — are MCP-only
by design: the in-app AI-chat agent exposes no image or footnote tools, so there
is no second layer to unify into SHARED_TOOL_SPECS. A registry spec's
tier/catalogLine are in-app metadata and the catalog-partition test forbids a
spec without a live in-app tool, so forcing them into the registry would break
the invariant. They stay per-transport (inline in index.ts).

No behavior change — documentation only (adds the rationale above each tool so a
future migrator does not re-investigate why these are not shared).

Gate: mcp tsc 0 (comment-only change).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 02:14:39 +03:00
agent_coder 39735afd73 refactor(ai-chat): unify page tools into SHARED_TOOL_SPECS (#294, pages family)
Migrates the three-layer page tools into the transport-agnostic spec registry
(schema + description declared once; each transport keeps only its execute/auth):
- getPage, listPages (core), createPage, movePage, renamePage, deletePage,
  updatePageJson, exportPageMarkdown (deferred) -> SHARED_TOOL_SPECS; index.ts
  uses registerShared(), ai-chat uses sharedTool(); removed from
  INLINE_TOOL_TIERS. Tiers preserved from CORE_TOOL_KEYS (getPage/listPages =
  core, the rest deferred).

delete_page is genuinely three-layer (in-app deletePage exists), so it IS
migrated — not MCP-only. Its H4 guardrail is preserved: the shared schema
exposes ONLY pageId, so no permanentlyDelete/forceDelete flag can reach the
client (still asserted by ai-chat-tools.service.spec.ts).

Descriptions merged (documented inline): each canonical text takes the MCP
copy's richer structural notes plus the in-app copy's reversibility framing.

Schema DRIFT reconciled (documented inline):
- createPage.content: MCP pinned .min(1) but the in-app copy left it unbounded
  and DOCUMENTS an empty body as valid ("may be empty" — creating an empty page
  to fill later is a real use). Kept the looser no-min form: create_page now also
  accepts an empty body (harmless) and no previously-valid in-app input is
  rejected. title/spaceId keep the MCP .min(1) (empty is never valid).
- movePage: MCP exposed an optional `position` (fractional-index) field the
  in-app copy lacked. Unified by KEEPING position — the in-app client already
  accepts an optional position arg, so the in-app execute now forwards it;
  optional, so no previously-valid call is rejected. `parentPageId` is nullable
  on both (real JSON null -> root); the MCP execute keeps its 'null'/'' string
  coercion as a per-layer robustness fallback.
- getPage/renamePage/updatePageJson/exportPageMarkdown/listPages: kept the MCP
  copy's stricter .min(1) on ids where the in-app copy was unbounded.

Per-transport execute logic preserved: getPage's {title,markdown} projection,
updatePageJson's JSON-string normalization, list_pages' default limit/tree, and
move_page's cycle guard + positive-confirmation check all stay in their execute
bodies.

Intentionally NOT touched: updatePageContent (Markdown-based body update; no MCP
equivalent) and getTable (name-convention divergence, see tables family) stay
inline.

Gate: mcp build 0 + node --test 458/458 (page-search excluded — hangs only under
the local re2->RegExp type-shim, its source untouched), server jest 770 incl.
tool-tiers catalog-partition + shared-spec contract parity + deletePage H4
guardrail, server tsc 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 02:13:41 +03:00
agent_vscode 9b4b38a611 fix(ai): patch ai@6.0.134 — drop O(n²) partialOutput accumulation causing heap OOM on long agent runs (#184)
Production OOM'd (JS heap 1.85 GB / 2 GB limit) during a ~20-step,
~28k-chunk autonomous agent turn. Heap snapshot analysis (memlab) showed a
single DefaultStreamTextResult retaining ~1.7 GB via the never-consumed
leftover tee() branch of its internal baseStream.

Root cause in ai@6.0.134: streamText substitutes the default text() output
strategy even when the caller passes NO `output` option. Its
createOutputTransformStream then accumulates the ENTIRE turn text and, on
EVERY text-delta, enqueues `{ part, partialOutput }` where partialOutput is
a flat snapshot of all text so far (JSON.stringify flattens the
cons-string) — O(n²) memory across the turn. Every consumer accessor tees
baseStream and keeps the second branch as the new baseStream; the final
leftover branch is never read, so its controller queue holds every chunk
(28,225 x ~164 KB in the OOM'd run) for the life of the turn.

Fix (pnpm patch on both dist/index.js and dist/index.mjs):
- pass the raw, possibly-undefined `output` option into
  createOutputTransformStream instead of defaulting to text()
- when output == null, publish each text-delta immediately without
  accumulating turn text or producing partialOutput snapshots; streaming
  granularity is unchanged, and callers that DO request an output strategy
  keep the original behavior

Our server never uses partialOutputStream / experimental_output / the
output option, so no behavior changes for us beyond memory.

Regression spec ai-sdk-partial-output.patch.spec.ts drives the real
patched SDK with MockLanguageModelV3: asserts per-delta textStream
granularity, an EMPTY experimental_partialOutputStream (tripwire — yields
one cumulative partial per delta when unpatched), and the PATCH(docmost
marker in both installed dist bundles. Also documents the patch in
AGENTS.md (must be re-created when bumping `ai`) and CHANGELOG.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 02:13:17 +03:00
agent_coder eebbe6717c refactor(ai-chat): unify table row/cell tools into SHARED_TOOL_SPECS (#294, tables family)
Migrates the three-layer table WRITE tools into the transport-agnostic spec
registry (schema + description declared once; each transport keeps only its
execute/auth):
- tableInsertRow, tableDeleteRow, tableUpdateCell -> SHARED_TOOL_SPECS;
  index.ts uses registerShared(), ai-chat uses sharedTool(); removed from
  INLINE_TOOL_TIERS (all three are deferred; not in CORE_TOOL_KEYS).

Drift reconciled (documented inline): the four table tools previously carried a
"NOT shared" note in both layers over a single parameter-NAME drift — the MCP
layer named the table reference `table`, the in-app layer `tableRef`. Unified on
the MCP name `table` (renaming the public MCP parameter would break external MCP
clients; the in-app parameter is model-facing/prompt-only and safe to rename).
The in-app execute bodies now destructure `table`. Descriptions took the MCP
copy's richer wording (documents `#<index>`, padding, header-row behavior) plus
the in-app copy's "Reversible via page history" note; both fields keep the MCP
copy's stricter .min(1) (in-app left them unbounded); sibling tool references
phrased transport-neutrally.

Intentionally NOT migrated (kept inline): table_get / getTable. Its MCP tool
name is noun-first (`table_get`) while the in-app key is verb-first (`getTable`),
which breaks the snake_case(inAppKey) naming convention the registry enforces
(shared-tool-specs.contract.spec.ts). Renaming the public MCP tool would break
external clients, so it stays per-transport — but its in-app reference param was
still aligned to `table` (was `tableRef`) for consistency with the migrated trio.

Gate: mcp tsc 0 + node --test 458/458 (page-search excluded — hangs only under
the local re2->RegExp type-shim, its source is untouched), server jest 730 incl.
tool-tiers catalog-partition + shared-spec contract parity, server tsc 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 02:06:52 +03:00
agent_coder e348433a39 refactor(ai-chat): unify comment tools into SHARED_TOOL_SPECS (#294, comments family)
Migrates the three-layer comment tools into the single transport-agnostic spec
registry (schema + model-facing description declared once; each transport keeps
only its execute/auth):
- createComment, listComments, resolveComment, checkNewComments — moved to
  SHARED_TOOL_SPECS; index.ts uses registerShared(), ai-chat uses sharedTool();
  removed from INLINE_TOOL_TIERS (tier/catalogLine now on the spec). Tiers
  preserved from CORE_TOOL_KEYS (create/list/resolve = core, check = deferred).

Intentionally NOT migrated (kept MCP-inline): update_comment / delete_comment —
they are MCP-only by design; the in-app AI-chat layer deliberately has no
updateComment/deleteComment (comment edits are irreversible / not
version-tracked), asserted by ai-chat-tools.service.spec.ts. A registry spec's
tier/catalogLine are in-app metadata and the catalog-partition test forbids a
deferred spec without a live in-app tool, so these stay per-transport.

Drift reconciled (documented inline): createComment/listComments/checkNewComments
took the more-maintained/superset description + stricter .min(1) guards.
resolveComment: `resolved` drifted (MCP optional+default(true) vs in-app
required) — kept the MCP superset, so in-app resolveComment now accepts an
omitted `resolved` (defaults to resolve) — a deliberate, backward-compatible
unification (never rejects a previously-valid input).

Gate: mcp build 0 + node --test 480/480, ai-chat 654, tool-tiers (incl. F3
catalog-partition) 16/16, server tsc 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 01:45:43 +03:00
agent_coder f759084f41 fix(metrics): collapse static-asset routes to bounded "static" label (#362)
Follow-up to #355: http_request_duration_seconds's `route` label captured raw
content-hashed asset filenames (route="/assets/index-CAbxDtto.js",
"/assets/chunk-*.js"). @fastify/static serves each file through a route whose
matched routeOptions.url IS the raw hashed path, so the label was unbounded — a
new set of names every deploy, growing the series forever (the exact cardinality
leak the API routes were protected against).

resolveRouteLabel now detects a static request by its path prefix (/assets/,
/vad/, /brand/, /locales/) FIRST and collapses it to a single `static` label
(query string stripped before the check); API routes still use the template and
404s still collapse to `unknown`. Static edge latency is already measured by
Traefik's traefik_router_request_duration_*.

Gate: server tsc 0; metrics.spec passes (added static-collapse + query-strip +
"real API route mentioning assets is NOT collapsed" cases).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 01:44:16 +03:00
agent_coder 459d636ffb fix(db): prevent the migration-order crash-loop from long-lived branches (#363, incident #361)
A long-lived branch can add a migration whose timestamped filename sorts BEFORE
migrations already applied in prod (#234's 20260627T130000-ai-chat-runs merged
after 20260704T120000-client-metrics was live). Kysely's migrator with the
default ordered setting then rejects the applied set as "corrupted migrations"
(no longer a prefix of the sorted list), throws, and the app crash-loops on boot
— exactly incident #361 (502s for ~11 min after a develop deploy). #119 and #120
(June branches) are the next such threats.

Two levels, both:
1. CI migration-order gate (a new `migration-order` job in test.yml, PR-only):
   fails the PR when an added migration sorts at/before the newest migration on
   the base branch, with an actionable message to rename it to a current
   timestamp before merge. This is the primary defense — makes back-dating
   impossible to merge accidentally.
2. `allowUnorderedMigrations: true` on BOTH Migrators (migration.service.ts
   startup auto-migrate + migrate.ts CLI): the runtime safety net — Kysely applies
   a not-yet-applied older migration instead of bricking startup, so a back-dated
   migration that bypasses the gate (manual push / hotfix branch) still boots.
   Trade-off documented inline: apply order across instances may differ from
   lexicographic, so migrations must stay independent (ours each create their own
   objects); the CI gate remains the primary line.

Verified: allowUnorderedMigrations is a valid Kysely 0.28.17 Migrator option;
server tsc clean; the gate script rejects a back-dated filename and passes a
current one. No new deps, no migration, no runtime behavior change beyond the
migrator resilience.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 01:36:57 +03:00
57 changed files with 4909 additions and 1138 deletions
+6
View File
@@ -151,6 +151,12 @@ jobs:
- name: Build editor-ext - name: Build editor-ext
run: pnpm --filter @docmost/editor-ext build run: pnpm --filter @docmost/editor-ext build
# @docmost/prosemirror-markdown is an ESM workspace package the server
# imports at runtime; its build/ is gitignored and test:e2e has no pretest
# hook, so build it before the e2e run (mirrors the test.yml job).
- name: Build prosemirror-markdown
run: pnpm --filter @docmost/prosemirror-markdown build
- name: Run migrations - name: Run migrations
run: pnpm --filter ./apps/server migration:latest run: pnpm --filter ./apps/server migration:latest
+43
View File
@@ -13,6 +13,49 @@ permissions:
contents: read contents: read
jobs: jobs:
# Guard against a long-lived branch adding a migration whose timestamped
# filename sorts BEFORE migrations already applied on the target branch (and
# thus in prod). The Kysely startup migrator rejects that as "corrupted
# migrations" and crash-loops the app on boot (incident #361). This gate fails
# the PR so the migration is renamed to a current timestamp before merge. Only
# runs for pull_request events (needs a base branch to diff against).
migration-order:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout (full history for the base-branch diff)
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Added migrations must sort after the newest on the base branch
env:
TARGET_BRANCH: ${{ github.base_ref }}
run: |
set -euo pipefail
MIG_DIR="apps/server/src/database/migrations"
# checkout above already did fetch-depth:0 (full history). Fetch the base
# WITHOUT --depth (a shallow graft would truncate the base history and
# break the merge-base when the base has moved ahead of the PR merge —
# exactly the long-branch-vs-moving-base case this gate guards, #361).
git fetch --no-tags origin "$TARGET_BRANCH"
newest_on_target=$(git ls-tree -r --name-only "origin/${TARGET_BRANCH}" "$MIG_DIR" | sort | tail -1)
# NO `|| true`: a diff failure (e.g. an unresolved merge-base) must fail
# the job CLOSED — a gate whose job is to BLOCK must never pass on error.
# `set -e` above already aborts on a non-zero diff exit.
added=$(git diff --diff-filter=A --name-only "origin/${TARGET_BRANCH}...HEAD" -- "$MIG_DIR")
bad=0
for f in $added; do
if [[ "$f" < "$newest_on_target" || "$f" == "$newest_on_target" ]]; then
echo "::error::Migration $f sorts at or before the newest on ${TARGET_BRANCH} ($newest_on_target) — rename it with a CURRENT timestamp before merge (do not change its contents). See incident #361."
bad=1
fi
done
if [ "$bad" -eq 0 ]; then
echo "Migration order OK (added migrations all sort after $newest_on_target)."
fi
exit $bad
test: test:
runs-on: ubuntu-latest runs-on: ubuntu-latest
timeout-minutes: 20 timeout-minutes: 20
+11 -2
View File
@@ -214,6 +214,12 @@ Run from the repo root unless noted. The dev workflow needs **Postgres (with the
> server, `APP_SECRET` mismatch between processes, a stale `editor-ext` white- > server, `APP_SECRET` mismatch between processes, a stale `editor-ext` white-
> screening the client, LAN exposure. See **[docs/dev-stand.md](docs/dev-stand.md)** > screening the client, LAN exposure. See **[docs/dev-stand.md](docs/dev-stand.md)**
> for the step-by-step and the traps. > for the step-by-step and the traps.
>
> **Testing the app against a stand** (browser E2E + out-of-band verification) has
> its own non-obvious traps — the page has two ProseMirror editors (only the body is
> collab-bound), a ~10s store debounce, and API-seeding the thing under test is a
> silent no-test. See **[docs/how-to-test.md](docs/how-to-test.md)** before writing
> UI tests.
```bash ```bash
pnpm install # install all workspaces (uses pnpm patches; see package.json `pnpm.patchedDependencies`) pnpm install # install all workspaces (uses pnpm patches; see package.json `pnpm.patchedDependencies`)
@@ -250,7 +256,10 @@ pnpm --filter server migration:codegen # regenerate src/databa
``` ```
Migration files live in `apps/server/src/database/migrations/` and are named `YYYYMMDDThhmmss-description.ts`. Fork-specific migrations only **add** tables (`page_embeddings`, `ai_chats`, `ai_chat_messages`, `ai_provider_credentials`, `ai_mcp_servers`, `page_template_references`) and columns (e.g. `pages.is_template`, a `NOT NULL DEFAULT false` boolean) — never drop/rewrite Docmost data. Migration files live in `apps/server/src/database/migrations/` and are named `YYYYMMDDThhmmss-description.ts`. Fork-specific migrations only **add** tables (`page_embeddings`, `ai_chats`, `ai_chat_messages`, `ai_provider_credentials`, `ai_mcp_servers`, `page_template_references`) and columns (e.g. `pages.is_template`, a `NOT NULL DEFAULT false` boolean) — never drop/rewrite Docmost data.
**Migration ordering — always check when merging branches/features.** Kysely runs migrations in **alphabetical (= timestamp) order** and refuses to start if a *new* migration sorts **before** one already applied to the DB (`corrupted migrations: ... must always have a name that comes alphabetically after the last executed migration`). When you merge a branch or land a feature, verify your migration's timestamp still sorts **after every migration that may already be applied on the target** (`/bin/ls -1 apps/server/src/database/migrations | sort | tail`). Branches developed in parallel routinely break this: a feature branch adds `…T130000-…`, `main` meanwhile ships and deploys `…T150000-…`, and after the merge the older-timestamped file is rejected at boot. **Fix = rename your migration to a timestamp after the latest one already in the target** (content unchanged — the filename is the ordering key), then rebuild so the compiled `dist/database/migrations/` picks up the new name. **Migration ordering — always check when merging branches/features.** Kysely runs migrations in **alphabetical (= timestamp) order**. A *new* migration that sorts **before** one already applied to the DB is a "back-dated" migration, which branches developed in parallel routinely produce: a feature branch adds `…T130000-…`, `develop` meanwhile ships and deploys `…T150000-…`, and after the merge the older-timestamped file has been skipped. Two layers guard this (both added for incident #361, where a back-dated migration crash-looped prod for ~11 min):
- **CI gate (primary):** the `migration-order` job in `.github/workflows/test.yml` fails a PR whose added migration sorts at/before the newest on the base branch. **So the fix is to rename your migration to a timestamp after the latest one already in the target** (`/bin/ls -1 apps/server/src/database/migrations | sort | tail`; content unchanged — the filename is the ordering key), then rebuild so the compiled `dist/database/migrations/` picks up the new name.
- **Runtime safety net:** both Migrators (`migration.service.ts` startup auto-migrate + `migrate.ts` CLI) set `allowUnorderedMigrations: true`, so the app does **not** refuse to start on an out-of-order migration — it applies the skipped older one instead of crash-looping. Kysely's `#ensureNoMissingMigrations` guard is still on (a *removed* applied migration is still an error). Because apply order can then differ from lexicographic across instances, migrations must stay **independent** (each creates its own objects) — the CI gate remains the primary line; this net only covers a gate bypass (manual push / hotfix branch).
## Architecture — the big picture ## Architecture — the big picture
@@ -294,7 +303,7 @@ Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirro
- **Errors must never be swallowed or shown as generic messages.** Every caught error MUST (1) be logged in full to the console/logger — error name, message, stack, `cause`, and (for HTTP/provider failures) the status code and response body — and (2) be surfaced to the user with a *specific, human-readable explanation of what actually went wrong*, never a bare generic string like "Something went wrong" / "Could not start recording" / "Transcription failed". Include the real reason (the underlying error/provider message) in the user-facing text. On the server, wrap third-party/provider failures with `describeProviderError` (or equivalent) and rethrow as a meaningful HTTP status + message — never let them collapse into an opaque 500. On the client, `console.error(<context>, err)` the raw error AND show the extracted reason (e.g. `err.response?.data?.message`, or the error `name: message`) in the notification. - **Errors must never be swallowed or shown as generic messages.** Every caught error MUST (1) be logged in full to the console/logger — error name, message, stack, `cause`, and (for HTTP/provider failures) the status code and response body — and (2) be surfaced to the user with a *specific, human-readable explanation of what actually went wrong*, never a bare generic string like "Something went wrong" / "Could not start recording" / "Transcription failed". Include the real reason (the underlying error/provider message) in the user-facing text. On the server, wrap third-party/provider failures with `describeProviderError` (or equivalent) and rethrow as a meaningful HTTP status + message — never let them collapse into an opaque 500. On the client, `console.error(<context>, err)` the raw error AND show the extracted reason (e.g. `err.response?.data?.message`, or the error `name: message`) in the notification.
- The version string shown in the UI comes from `APP_VERSION` (CI/Docker) or `git describe --tags --always` (local), resolved in `vite.config.ts` — not from `package.json`. - The version string shown in the UI comes from `APP_VERSION` (CI/Docker) or `git describe --tags --always` (local), resolved in `vite.config.ts` — not from `package.json`.
- Server TS config is permissive (`noImplicitAny: false`, `strictNullChecks: false`, `no-explicit-any` lint disabled). Follow the existing relaxed style rather than tightening types broadly. - Server TS config is permissive (`noImplicitAny: false`, `strictNullChecks: false`, `no-explicit-any` lint disabled). Follow the existing relaxed style rather than tightening types broadly.
- Dependency versions are heavily pinned via `pnpm.overrides` and `pnpm.patchedDependencies` (`scimmy`, `yjs`) in the root `package.json`. Don't bump pinned/patched deps casually; the patches and overrides exist for compatibility/security reasons. - Dependency versions are heavily pinned via `pnpm.overrides` and `pnpm.patchedDependencies` (`scimmy`, `yjs`, `ai`) in the root `package.json`. Don't bump pinned/patched deps casually; the patches and overrides exist for compatibility/security reasons. The `ai@6.0.134` patch disables the SDK's O(n²) cumulative `partialOutput` accumulation when no output strategy is requested (server heap OOM on long agent runs, #184; tripwire test: `apps/server/src/integrations/ai/ai-sdk-partial-output.patch.spec.ts`) — it MUST be re-created via `pnpm patch` when bumping `ai`.
- **Adding/renaming/removing an MCP tool requires updating `SERVER_INSTRUCTIONS`** in `packages/mcp/src/index.ts` — the intent-routing guide MCP clients receive on initialize. This applies both to inline `server.registerTool(...)` calls in `index.ts` and to specs in `packages/mcp/src/tool-specs.ts`. Enforced by `packages/mcp/test/unit/server-instructions.test.mjs`, which fails when a registered tool is not mentioned in the guide (deliberate opt-outs go into its `EXCEPTIONS` list). `packages/mcp/build/` is gitignored and rebuilt in CI/Docker via `pnpm build` (same convention as `git-sync`/`prosemirror-markdown`) — never commit it; rebuild locally after editing to run the tests. - **Adding/renaming/removing an MCP tool requires updating `SERVER_INSTRUCTIONS`** in `packages/mcp/src/index.ts` — the intent-routing guide MCP clients receive on initialize. This applies both to inline `server.registerTool(...)` calls in `index.ts` and to specs in `packages/mcp/src/tool-specs.ts`. Enforced by `packages/mcp/test/unit/server-instructions.test.mjs`, which fails when a registered tool is not mentioned in the guide (deliberate opt-outs go into its `EXCEPTIONS` list). `packages/mcp/build/` is gitignored and rebuilt in CI/Docker via `pnpm build` (same convention as `git-sync`/`prosemirror-markdown`) — never commit it; rebuild locally after editing to run the tests.
## CI / release ## CI / release
+8
View File
@@ -169,6 +169,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed ### Fixed
- **The server no longer runs out of heap during long autonomous agent runs.** A
new pnpm patch on `ai@6.0.134` stops the SDK from building a cumulative
snapshot of the ENTIRE turn text on every streamed text-delta when no output
strategy was requested (our server never requests one). Unpatched, those
O(n²) `partialOutput` snapshots piled up in a never-consumed internal
`tee()` branch of the stream result — a ~20-step, ~28k-chunk agent run
retained ~1.7 GB and OOM'd the 2 GB JS heap. Streaming granularity is
unchanged; the patch must be re-created if `ai` is ever bumped. (#184)
- **Internal links in exported Markdown no longer lose their visible text.** A - **Internal links in exported Markdown no longer lose their visible text.** A
link whose target page name had no file extension (e.g. a bare title) was link whose target page name had no file extension (e.g. a bare title) was
collapsed to empty text during export, producing an unclickable, label-less collapsed to empty text during export, producing an unclickable, label-less
+1
View File
@@ -98,6 +98,7 @@
"typescript": "5.9.3", "typescript": "5.9.3",
"typescript-eslint": "8.57.1", "typescript-eslint": "8.57.1",
"vite": "8.0.5", "vite": "8.0.5",
"vite-plugin-compression2": "2.5.3",
"vitest": "4.1.6" "vitest": "4.1.6"
} }
} }
@@ -1373,6 +1373,39 @@
"The role catalog is unavailable": "The role catalog is unavailable", "The role catalog is unavailable": "The role catalog is unavailable",
"Please try again later.": "Please try again later.", "Please try again later.": "Please try again later.",
"No bundles available": "No bundles available", "No bundles available": "No bundles available",
"Content": "Content",
"Content language of the roles": "Content language of the roles",
"{{count}} updates available in {{bundles}} bundles": "{{count}} updates available in {{bundles}} bundles",
"Update all ({{count}})": "Update all ({{count}})",
"Updating {{current}}/{{total}}…": "Updating {{current}}/{{total}}…",
"{{count}} roles are installed in another language. A different language installs separately and appears as new.": "{{count}} roles are installed in another language. A different language installs separately and appears as new.",
"{{count}} roles": "{{count}} roles",
"{{count}} new — none installed": "{{count}} new — none installed",
"All installed · up to date": "All installed · up to date",
"{{count}} updates · {{installed}} up to date": "{{count}} updates · {{installed}} up to date",
"{{count}} new": "{{count}} new",
"{{count}} installed": "{{count}} installed",
"{{count}} updates": "{{count}} updates",
"Install bundle": "Install bundle",
"Install {{count}} selected": "Install {{count}} selected",
"Install bundle ({{count}})": "Install bundle ({{count}})",
"{{selected}} of {{total}} selected": "{{selected}} of {{total}} selected",
"Select all": "Select all",
"Deselect all": "Deselect all",
"Skipped": "Skipped",
"v{{version}}": "v{{version}}",
"{{count}} roles installed": "{{count}} roles installed",
"{{count}} roles installed · {{renamed}} renamed": "{{count}} roles installed · {{renamed}} renamed",
"{{count}} roles updated": "{{count}} roles updated",
"Installed {{installed}} · {{skipped}} skipped": "Installed {{installed}} · {{skipped}} skipped",
"A role named \"{{name}}\" already exists in this workspace.": "A role named \"{{name}}\" already exists in this workspace.",
"\"{{name}}\" is already installed.": "\"{{name}}\" is already installed.",
"Rename & install": "Rename & install",
"Couldn’t load the catalog": "Couldn’t load the catalog",
"Check your connection and try again. Installed roles are not affected.": "Check your connection and try again. Installed roles are not affected.",
"Retry": "Retry",
"The catalog is empty": "The catalog is empty",
"No role bundles are published for this language yet. Try switching the content language.": "No role bundles are published for this language yet. Try switching the content language.",
"Already up to date": "Already up to date", "Already up to date": "Already up to date",
"Updated to the latest version": "Updated to the latest version", "Updated to the latest version": "Updated to the latest version",
"This role is no longer in the catalog": "This role is no longer in the catalog", "This role is no longer in the catalog": "This role is no longer in the catalog",
@@ -1235,6 +1235,39 @@
"The role catalog is unavailable": "Каталог ролей недоступен", "The role catalog is unavailable": "Каталог ролей недоступен",
"Please try again later.": "Попробуйте позже.", "Please try again later.": "Попробуйте позже.",
"No bundles available": "Наборы недоступны", "No bundles available": "Наборы недоступны",
"Content": "Язык контента",
"Content language of the roles": "Язык контента ролей",
"{{count}} updates available in {{bundles}} bundles": "Доступно обновлений: {{count}} в наборах: {{bundles}}",
"Update all ({{count}})": "Обновить все ({{count}})",
"Updating {{current}}/{{total}}…": "Обновление {{current}}/{{total}}…",
"{{count}} roles are installed in another language. A different language installs separately and appears as new.": "Ролей установлено на другом языке: {{count}}. Другой язык устанавливается отдельно и отображается как новый.",
"{{count}} roles": "ролей: {{count}}",
"{{count}} new — none installed": "новых: {{count}} — ничего не установлено",
"All installed · up to date": "Все установлены · актуальны",
"{{count}} updates · {{installed}} up to date": "обновлений: {{count}} · актуальны: {{installed}}",
"{{count}} new": "новых: {{count}}",
"{{count}} installed": "установлено: {{count}}",
"{{count}} updates": "обновлений: {{count}}",
"Install bundle": "Установить набор",
"Install {{count}} selected": "Установить выбранные ({{count}})",
"Install bundle ({{count}})": "Установить набор ({{count}})",
"{{selected}} of {{total}} selected": "выбрано {{selected}} из {{total}}",
"Select all": "Выбрать все",
"Deselect all": "Снять выбор",
"Skipped": "Пропущено",
"v{{version}}": "v{{version}}",
"{{count}} roles installed": "Установлено ролей: {{count}}",
"{{count}} roles installed · {{renamed}} renamed": "Установлено ролей: {{count}} · переименовано: {{renamed}}",
"{{count}} roles updated": "Обновлено ролей: {{count}}",
"Installed {{installed}} · {{skipped}} skipped": "Установлено: {{installed}} · пропущено: {{skipped}}",
"A role named \"{{name}}\" already exists in this workspace.": "Роль с именем «{{name}}» уже существует в этом рабочем пространстве.",
"\"{{name}}\" is already installed.": "«{{name}}» уже установлена.",
"Rename & install": "Переименовать и установить",
"Couldn’t load the catalog": "Не удалось загрузить каталог",
"Check your connection and try again. Installed roles are not affected.": "Проверьте подключение и попробуйте снова. Установленные роли не затронуты.",
"Retry": "Повторить",
"The catalog is empty": "Каталог пуст",
"No role bundles are published for this language yet. Try switching the content language.": "Для этого языка ещё не опубликовано ни одного набора ролей. Попробуйте сменить язык контента.",
"No roles configured": "Роли не настроены", "No roles configured": "Роли не настроены",
"Already up to date": "Уже актуальна", "Already up to date": "Уже актуальна",
"Updated to the latest version": "Обновлено до последней версии", "Updated to the latest version": "Обновлено до последней версии",
@@ -1,6 +1,7 @@
import { import {
useInfiniteQuery, useInfiniteQuery,
useMutation, useMutation,
useQueries,
useQuery, useQuery,
useQueryClient, useQueryClient,
} from "@tanstack/react-query"; } from "@tanstack/react-query";
@@ -307,6 +308,29 @@ export function useAiRoleCatalogBundleQuery(
}); });
} }
/**
* Eagerly open EVERY listed bundle's content in parallel for one language. The
* redesigned catalog shows each bundle's status summary in its COLLAPSED header,
* which needs every role's install state up front so contents can no longer be
* lazy-loaded on expand. The catalog is small, so a fan-out of `useQueries` (one
* cached read per bundle, sharing the same cache keys as
* `useAiRoleCatalogBundleQuery`) is cheap. Gated by `enabled` (modal open + a
* resolved language) so nothing fetches while the modal is closed.
*/
export function useAiRoleCatalogBundlesQueries(
bundleIds: string[],
language: string,
enabled: boolean,
) {
return useQueries({
queries: bundleIds.map((bundleId) => ({
queryKey: AI_ROLE_CATALOG_BUNDLE_RQ_KEY(bundleId, language),
queryFn: () => getAiRoleCatalogBundle(bundleId, language),
enabled: enabled && !!language,
})),
});
}
export function useImportAiRolesFromCatalogMutation() { export function useImportAiRolesFromCatalogMutation() {
const queryClient = useQueryClient(); const queryClient = useQueryClient();
const { t } = useTranslation(); const { t } = useTranslation();
@@ -77,7 +77,14 @@ describe("useImportAiRolesFromCatalogMutation — success notifications", () =>
}); });
it("errors:[] -> only the summary notification (counts interpolated)", async () => { it("errors:[] -> only the summary notification (counts interpolated)", async () => {
await runMutation({ created: 3, renamed: 1, skipped: 2, errors: [] }); await runMutation({
created: 3,
renamed: 1,
skipped: 2,
errors: [],
createdRoles: [],
skippedRoles: [],
});
expect(notificationsShowMock).toHaveBeenCalledTimes(1); expect(notificationsShowMock).toHaveBeenCalledTimes(1);
expect(notificationsShowMock).toHaveBeenCalledWith({ expect(notificationsShowMock).toHaveBeenCalledWith({
message: "Imported 3, renamed 1, skipped 2", message: "Imported 3, renamed 1, skipped 2",
@@ -93,6 +100,8 @@ describe("useImportAiRolesFromCatalogMutation — success notifications", () =>
{ slug: "a", message: "name taken" }, { slug: "a", message: "name taken" },
{ slug: "b", message: "name taken" }, { slug: "b", message: "name taken" },
], ],
createdRoles: [{ slug: "ok", name: "Ok" }],
skippedRoles: [],
}); });
expect(notificationsShowMock).toHaveBeenCalledTimes(2); expect(notificationsShowMock).toHaveBeenCalledTimes(2);
expect(notificationsShowMock).toHaveBeenNthCalledWith(1, { expect(notificationsShowMock).toHaveBeenNthCalledWith(1, {
@@ -108,12 +108,25 @@ export interface IAiRoleImportPayload {
conflict: "skip" | "rename"; conflict: "skip" | "rename";
} }
/** Import result counts (mirrors `importFromCatalog()`). */ /**
* Import result (mirrors `importFromCatalog()`). The counters (`created`,
* `skipped`, `renamed`) drive the summary notification; the per-role lists
* (`createdRoles`, `skippedRoles`) drive the redesigned catalog modal's inline
* result plaque which roles were installed (and any rename) and which were
* skipped and why (so the plaque can name the conflicting role and offer
* "Rename & install").
*/
export interface IAiRoleImportResult { export interface IAiRoleImportResult {
created: number; created: number;
skipped: number; skipped: number;
renamed: number; renamed: number;
errors: { slug: string; message: string }[]; errors: { slug: string; message: string }[];
createdRoles: { slug: string; name: string; renamedTo?: string }[];
skippedRoles: {
slug: string;
name: string;
reason: "name-conflict" | "already-installed";
}[];
} }
/** /**
@@ -0,0 +1,234 @@
import { describe, it, expect } from "vitest";
import {
bundleCounts,
bundlePhase,
installedLangForRole,
mapBundleRolesToView,
mapCatalogRoleToView,
nameConflictSlugs,
partialOffersRename,
type CatalogViewRole,
} from "./catalog-bundle-model.ts";
import type {
IAiRole,
IAiRoleCatalogRole,
} from "@/features/ai-chat/types/ai-chat.types.ts";
function installedRole(
source: { slug: string; language: string; version: number },
overrides: Partial<IAiRole> = {},
): IAiRole {
return {
id: `role-${source.slug}-${source.language}`,
name: source.slug,
emoji: null,
description: null,
enabled: true,
autoStart: true,
launchMessage: null,
source,
...overrides,
};
}
function catalogRole(
overrides: Partial<IAiRoleCatalogRole> = {},
): IAiRoleCatalogRole {
return {
slug: "writer",
emoji: "✍️",
name: "Writer",
description: "Drafts copy.",
instructions: "be a writer",
autoStart: true,
launchMessage: null,
version: 3,
...overrides,
};
}
// Build a minimal view role for bundlePhase tests.
function viewRole(status: CatalogViewRole["status"]): CatalogViewRole {
return { slug: `s-${status}`, name: status, description: "", version: 1, status };
}
describe("bundlePhase", () => {
it("empty bundle -> empty", () => {
expect(bundlePhase([])).toBe("empty");
});
it("all importable, none installed -> allNew", () => {
expect(bundlePhase([viewRole("import"), viewRole("import")])).toBe(
"allNew",
);
});
it("nothing to import or update -> allInstalled", () => {
expect(bundlePhase([viewRole("installed"), viewRole("installed")])).toBe(
"allInstalled",
);
});
it("updates present, nothing to import -> updates", () => {
expect(bundlePhase([viewRole("update"), viewRole("installed")])).toBe(
"updates",
);
});
it("import + installed (no updates) -> mixed", () => {
expect(bundlePhase([viewRole("import"), viewRole("installed")])).toBe(
"mixed",
);
});
it("import + update -> mixed", () => {
expect(bundlePhase([viewRole("import"), viewRole("update")])).toBe("mixed");
});
it("a skipped role with nothing installed -> mixed (NOT allInstalled)", () => {
// F1: a bundle whose only non-installed role was skipped has 0 installed for
// it, so the collapsed 'All installed · up to date' header would contradict
// the open 'Installed 0 · 1 skipped' plaque. It must be mixed until resolved.
expect(bundlePhase([viewRole("skipped")])).toBe("mixed");
});
it("installed + a skipped role -> mixed (partial success is not allInstalled)", () => {
expect(bundlePhase([viewRole("installed"), viewRole("skipped")])).toBe(
"mixed",
);
});
});
describe("bundleCounts", () => {
it("tallies each status once", () => {
expect(
bundleCounts([
viewRole("import"),
viewRole("import"),
viewRole("installed"),
viewRole("update"),
viewRole("skipped"),
]),
).toEqual({ importable: 2, installed: 1, update: 1, skipped: 1 });
});
});
describe("nameConflictSlugs / partialOffersRename (reason -> action)", () => {
it("only name-conflict skips become the transient overlay / offer rename", () => {
const skipped = [
{ slug: "writer", name: "Writer", reason: "name-conflict" as const },
{ slug: "editor", name: "Editor", reason: "already-installed" as const },
];
expect(nameConflictSlugs(skipped)).toEqual(["writer"]);
expect(partialOffersRename(skipped)).toBe(true);
});
it("an already-installed-only skip is informational: no overlay, no rename", () => {
const skipped = [
{ slug: "editor", name: "Editor", reason: "already-installed" as const },
];
expect(nameConflictSlugs(skipped)).toEqual([]);
expect(partialOffersRename(skipped)).toBe(false);
});
});
describe("installedLangForRole", () => {
it("returns the other language when the same slug is installed elsewhere", () => {
const roles = [installedRole({ slug: "writer", language: "ru", version: 2 })];
expect(installedLangForRole("writer", roles, "en")).toBe("ru");
});
it("returns undefined when the same slug is installed in the SAME language", () => {
const roles = [installedRole({ slug: "writer", language: "en", version: 2 })];
expect(installedLangForRole("writer", roles, "en")).toBeUndefined();
});
it("returns undefined when no install of the slug exists", () => {
expect(installedLangForRole("writer", [], "en")).toBeUndefined();
});
it("ignores manually-created roles (no source)", () => {
const roles = [
installedRole({ slug: "writer", language: "ru", version: 2 }, {
source: null,
}),
];
expect(installedLangForRole("writer", roles, "en")).toBeUndefined();
});
});
describe("mapCatalogRoleToView", () => {
it("no install -> import status, catalog version, emoji preserved", () => {
const view = mapCatalogRoleToView(catalogRole(), [], "en");
expect(view).toMatchObject({
slug: "writer",
emoji: "✍️",
name: "Writer",
description: "Drafts copy.",
status: "import",
version: 3,
});
expect(view.installedRoleId).toBeUndefined();
expect(view.installedLang).toBeUndefined();
});
it("import with the slug installed in another language -> installedLang set", () => {
const roles = [installedRole({ slug: "writer", language: "ru", version: 9 })];
const view = mapCatalogRoleToView(catalogRole(), roles, "en");
expect(view.status).toBe("import");
expect(view.installedLang).toBe("ru");
});
it("installed (up to date) -> installed status, catalog version, installedRoleId", () => {
const installed = installedRole({
slug: "writer",
language: "en",
version: 3,
});
const view = mapCatalogRoleToView(catalogRole(), [installed], "en");
expect(view).toMatchObject({
status: "installed",
version: 3,
installedRoleId: installed.id,
});
});
it("update -> version=from, newVersion=to, installedRoleId", () => {
const installed = installedRole({
slug: "writer",
language: "en",
version: 1,
});
const view = mapCatalogRoleToView(catalogRole(), [installed], "en");
expect(view).toMatchObject({
status: "update",
version: 1,
newVersion: 3,
installedRoleId: installed.id,
});
});
it("missing emoji -> emoji undefined; null description -> empty string", () => {
const view = mapCatalogRoleToView(
catalogRole({ emoji: null, description: null }),
[],
"en",
);
expect(view.emoji).toBeUndefined();
expect(view.description).toBe("");
});
});
describe("mapBundleRolesToView", () => {
it("maps a bundle's roles preserving order", () => {
const roles = [
catalogRole({ slug: "a", name: "A", version: 1 }),
catalogRole({ slug: "b", name: "B", version: 1 }),
];
const installed = [installedRole({ slug: "a", language: "en", version: 1 })];
const view = mapBundleRolesToView(roles, installed, "en");
expect(view.map((r) => r.slug)).toEqual(["a", "b"]);
expect(view[0].status).toBe("installed");
expect(view[1].status).toBe("import");
});
});
@@ -0,0 +1,206 @@
import type {
IAiRole,
IAiRoleCatalogRole,
} from "@/features/ai-chat/types/ai-chat.types.ts";
import { catalogRoleInstallState } from "@/features/ai-chat/utils/catalog-role-install-state.ts";
/**
* The redesigned catalog modal renders bundles as cards with a summary status
* (readable without expanding) and a single primary action. The per-role and
* per-bundle view model that drives that UI is derived here as PURE functions so
* the mapping, the "installed in another language" hint, and the bundle-phase
* computation are unit-testable without mounting the component (mirrors the
* `catalogRoleInstallState` precedent).
*/
/**
* A role's status in the catalog view model.
* - `import` not installed in the current content language.
* - `installed` installed and up to date.
* - `update` installed, but the catalog ships a newer version.
* - `skipped` TRANSIENT client-only status set after a conflicted import
* (a name collision under `conflict:'skip'`); never from the
* backend.
*/
export type RoleStatus = "import" | "installed" | "update" | "skipped";
/** A catalog role mapped into the modal's view model. */
export interface CatalogViewRole {
// Slug is the stable identity within a bundle; used as the row key and as the
// `slugs[]` payload for import.
slug: string;
// Optional in the catalog — the row reserves space and renders nothing when
// absent.
emoji?: string;
name: string;
description: string;
// For `installed`/`import`: the catalog version. For `update`: the installed
// (from) version, with `newVersion` holding the catalog (to) version.
version: number;
newVersion?: number;
status: RoleStatus;
// The language a same-slug role is installed under, when it differs from the
// current content language (drives the Р5 hint). Only set for `import` roles.
installedLang?: string;
// The workspace role id, present for `installed`/`update` — needed to call the
// update-from-catalog mutation.
installedRoleId?: string;
}
/**
* The summary phase of a bundle, derived from its roles' statuses. Determines
* the collapsed-header summary and the bundle's single primary action.
* - `empty` the bundle has no roles.
* - `allNew` everything is importable, nothing installed.
* - `allInstalled` everything installed & up to date; nothing else pending.
* - `updates` updates available and nothing left to import.
* - `mixed` any other combination.
*/
export type BundlePhase =
| "empty"
| "allNew"
| "allInstalled"
| "updates"
| "mixed";
/** Per-status tallies for a bundle's roles (the single source of truth). */
export interface BundleCounts {
importable: number;
installed: number;
update: number;
skipped: number;
}
/**
* Count a bundle's roles by status ONCE. Both `bundlePhase` and the panel derive
* from this, so the tally logic lives in exactly one place (no rescans / drift).
*/
export function bundleCounts(roles: CatalogViewRole[]): BundleCounts {
const counts: BundleCounts = {
importable: 0,
installed: 0,
update: 0,
skipped: 0,
};
for (const r of roles) {
if (r.status === "import") counts.importable += 1;
else if (r.status === "installed") counts.installed += 1;
else if (r.status === "update") counts.update += 1;
else if (r.status === "skipped") counts.skipped += 1;
}
return counts;
}
export function bundlePhase(roles: CatalogViewRole[]): BundlePhase {
if (roles.length === 0) return "empty";
const { importable, installed, update, skipped } = bundleCounts(roles);
// A `skipped` role is a pending post-import conflict (0 installed for it), so a
// bundle that has ANY skipped role is NOT "all installed & up to date" — that
// would make the collapsed green "up to date" header contradict the open
// panel's "Installed 0 · 1 skipped" plaque. It is `mixed` until resolved.
if (importable === 0 && update === 0 && skipped === 0) return "allInstalled";
if (update > 0 && importable === 0 && skipped === 0) return "updates";
if (importable > 0 && installed === 0 && update === 0 && skipped === 0)
return "allNew";
return "mixed";
}
/**
* The subset of a skip result that should be shown as a TRANSIENT `skipped`
* overlay in the bundle (so the row offers a re-import path). Only NAME-CONFLICT
* skips qualify: an `already-installed` skip (a concurrent-import race) has
* nothing to act on re-importing the same slug would just skip again so it
* must NOT be overlaid (else the row shows a misleading "Rename & install" that
* self-heals into a false "installed"). Pure so both reason branches are tested.
*/
export function nameConflictSlugs(
skipped: { slug: string; reason: "name-conflict" | "already-installed" }[],
): string[] {
return skipped
.filter((s) => s.reason === "name-conflict")
.map((s) => s.slug);
}
/**
* Whether a partial-import result should offer the "Rename & install" action:
* only when at least one skip is a name conflict (renameable). An
* `already-installed`-only partial is informational.
*/
export function partialOffersRename(
skipped: { reason: "name-conflict" | "already-installed" }[],
): boolean {
return skipped.some((s) => s.reason === "name-conflict");
}
/**
* For a role NOT installed in the current `language`, find a workspace role with
* the same catalog `slug` installed under a DIFFERENT language, and return that
* language. Drives the "installed in another language" hint (Р5): a different
* language of the same slug is a separate install and appears as `import`.
*/
export function installedLangForRole(
slug: string,
workspaceRoles: IAiRole[],
language: string,
): string | undefined {
const other = workspaceRoles.find(
(r) =>
r.source?.slug === slug &&
!!r.source?.language &&
r.source.language !== language,
);
return other?.source?.language;
}
/**
* Map one catalog role to the view model, computing its install status against
* the workspace roles (via `catalogRoleInstallState`) and, for importable roles,
* the other-language hint.
*/
export function mapCatalogRoleToView(
role: IAiRoleCatalogRole,
workspaceRoles: IAiRole[],
language: string,
): CatalogViewRole {
const state = catalogRoleInstallState(role, workspaceRoles, language);
const base = {
slug: role.slug,
emoji: role.emoji ?? undefined,
name: role.name,
description: role.description ?? "",
};
if (state.state === "update") {
return {
...base,
status: "update",
version: state.fromVersion,
newVersion: state.toVersion,
installedRoleId: state.installed.id,
};
}
if (state.state === "installed") {
return {
...base,
status: "installed",
version: role.version,
installedRoleId: state.installed.id,
};
}
return {
...base,
status: "import",
version: role.version,
installedLang: installedLangForRole(role.slug, workspaceRoles, language),
};
}
/**
* Map a whole bundle's catalog roles to the view model, preserving order.
*/
export function mapBundleRolesToView(
roles: IAiRoleCatalogRole[],
workspaceRoles: IAiRole[],
language: string,
): CatalogViewRole[] {
return roles.map((r) => mapCatalogRoleToView(r, workspaceRoles, language));
}
@@ -24,6 +24,7 @@ import {
GitmostListPagesResult, GitmostListPagesResult,
GitmostListSpacesResult, GitmostListSpacesResult,
gitmostDecodePayloadToFile, gitmostDecodePayloadToFile,
gitmostInsertTranscriptIntoEditor,
gitmostUploadFileToEditor, gitmostUploadFileToEditor,
} from "@/features/editor/gitmost/gitmost-recording.ts"; } from "@/features/editor/gitmost/gitmost-recording.ts";
@@ -281,6 +282,18 @@ export default function GitmostGlobalBridge() {
pageId: page.id, pageId: page.id,
}; };
} }
// Best-effort: append the transcript (heading + one paragraph per line)
// below the just-inserted audio node. The audio insert already
// succeeded, so a transcript failure must NOT turn this into an error —
// wrap it and, on any throw, log and still return ok. A missing/empty/
// non-string transcript is a no-op inside the helper (audio only).
try {
gitmostInsertTranscriptIntoEditor(editor, payload?.transcript);
} catch (err) {
console.error("[gitmost] transcript insert failed", err);
}
return { ok: true, pageId: page.id }; return { ok: true, pageId: page.id };
} catch (err: any) { } catch (err: any) {
console.error("[gitmost] createPageWithRecording failed", err); console.error("[gitmost] createPageWithRecording failed", err);
@@ -0,0 +1,150 @@
import { describe, it, expect } from "vitest";
import { Editor } from "@tiptap/core";
import { Document } from "@tiptap/extension-document";
import { Paragraph } from "@tiptap/extension-paragraph";
import { Text } from "@tiptap/extension-text";
import { Heading } from "@tiptap/extension-heading";
import { Bold } from "@tiptap/extension-bold";
import { Italic } from "@tiptap/extension-italic";
import { Link } from "@tiptap/extension-link";
import { gitmostInsertTranscriptIntoEditor } from "./gitmost-recording.ts";
const ZWSP = "​"; // U+200B, the helper's block-trigger neutralizer
/**
* #377 the web-side bridge must append the native host's transcript below the
* recording. These exercise the pure insert helper through a REAL Tiptap editor
* (Document/Paragraph/Text/Heading + Bold/Italic/Link marks so an HTML-parsing
* regression would be caught), asserting the resulting document rather than
* mocking the editor: transcript present -> "Transcript" heading + one paragraph
* per non-empty line; content is inserted as LITERAL TEXT (no HTML/markdown
* parsing); col-0 markdown block triggers are neutralized so git-sync keeps them
* paragraphs; absent/empty/non-string -> no-op.
*/
describe("gitmostInsertTranscriptIntoEditor", () => {
const makeEditor = () =>
new Editor({
// Bold/Italic/Link are registered specifically so that IF the helper ever
// regressed to inserting an HTML/markdown string (instead of a text node),
// TipTap would parse `<b>`/`*..*`/`[..](..)` into marks and the literal-
// text assertions below would fail.
extensions: [Document, Paragraph, Text, Heading, Bold, Italic, Link],
// Start from a single empty paragraph (a fresh page's baseline). The
// helper appends at the end of the doc, i.e. below existing content.
content: { type: "doc", content: [{ type: "paragraph" }] },
});
it("inserts a Transcript heading + one paragraph per non-empty line, verbatim", () => {
const editor = makeEditor();
const inserted = gitmostInsertTranscriptIntoEditor(
editor,
"You: hello there\nSpeaker 1: hi\n\nYou: bye",
);
expect(inserted).toBe(true);
const nodes = (editor.getJSON().content ?? []) as any[];
// A level-2 "Transcript" heading is present.
const heading = nodes.find((n) => n.type === "heading");
expect(heading?.attrs?.level).toBe(2);
expect(heading?.content?.[0]?.text).toBe("Transcript");
// Every non-empty transcript line becomes a paragraph, in order, verbatim;
// the blank line between them is dropped.
const texts = nodes
.filter((n) => n.type === "paragraph")
.map((n) => n.content?.[0]?.text)
.filter((t) => typeof t === "string");
expect(texts).toEqual(["You: hello there", "Speaker 1: hi", "You: bye"]);
editor.destroy();
});
it("inserts HTML + markdown metacharacters as LITERAL text (no injection / no mark parsing)", () => {
const editor = makeEditor();
const line =
"You: <b>bold</b> <script>alert(1)</script> and *stars* and [link](x)";
const inserted = gitmostInsertTranscriptIntoEditor(editor, line);
expect(inserted).toBe(true);
const paras = (editor.getJSON().content ?? []).filter(
(n: any) => n.type === "paragraph",
) as any[];
// The transcript line is exactly ONE paragraph holding a SINGLE text node
// whose text is the verbatim string — not split into bold/link/other nodes,
// not carrying any marks, not raw HTML. This FAILS if the helper switched to
// insertContent(htmlString): TipTap would then parse <b>/[link](x)/*stars*.
const content = paras[paras.length - 1].content;
expect(content).toHaveLength(1);
expect(content[0].type).toBe("text");
expect(content[0].marks ?? []).toEqual([]);
expect(content[0].text).toBe(line);
// And no bold/italic/link mark exists anywhere in the document.
const html = editor.getHTML();
expect(html).not.toMatch(/<(strong|b|em|i|a)\b/);
// The angle brackets survived as escaped entities (literal text), not a live
// <script>/<b> element.
expect(html).not.toMatch(/<script/i);
editor.destroy();
});
it("neutralizes col-0 markdown block triggers with a leading ZWSP (git-sync safety)", () => {
const editor = makeEditor();
// Trigger lines (some with a leaked indent) + a normal prefixed line.
const inserted = gitmostInsertTranscriptIntoEditor(
editor,
[
"- dash",
" > quote", // leading indent must be trimmed then neutralized
"# hash",
"1. one",
"> [!info] note",
"```js",
"---", // solid thematic break -> horizontalRule (text-losing) if unneutralized
"***",
"___",
"You: normal line",
].join("\n"),
);
expect(inserted).toBe(true);
const texts = (editor.getJSON().content ?? [])
.filter((n: any) => n.type === "paragraph")
.map((n: any) => n.content?.[0]?.text)
.filter((t: any) => typeof t === "string") as string[];
// Every block-trigger line is prefixed with the invisible ZWSP (indent
// trimmed first); the normal `You:` line is left byte-exact.
expect(texts).toEqual([
ZWSP + "- dash",
ZWSP + "> quote",
ZWSP + "# hash",
ZWSP + "1. one",
ZWSP + "> [!info] note",
ZWSP + "```js",
ZWSP + "---",
ZWSP + "***",
ZWSP + "___",
"You: normal line",
]);
editor.destroy();
});
it("is a no-op for undefined / empty / whitespace-only / non-string transcripts", () => {
for (const value of [undefined, "", " \n \n", 42, {}, null]) {
const editor = makeEditor();
const before = JSON.stringify(editor.getJSON());
const inserted = gitmostInsertTranscriptIntoEditor(editor, value as any);
expect(inserted).toBe(false);
// Document is untouched (audio-only behavior preserved).
expect(JSON.stringify(editor.getJSON())).toBe(before);
editor.destroy();
}
});
});
@@ -65,6 +65,11 @@ export interface GitmostCreatePagePayload {
base64: string; base64: string;
filename: string; filename: string;
mimeType: string; mimeType: string;
// Optional transcript for the recording: plain text, `\n`-separated, each
// line already formatted as `You: ...` / `Speaker N: ...` by the native host
// (ready to insert, no parsing needed). Omitted (no speech / no models) ->
// audio only.
transcript?: string;
} }
export interface GitmostCreatePageResult { export interface GitmostCreatePageResult {
@@ -235,6 +240,83 @@ export async function gitmostUploadFileToEditor(
} }
} }
// Zero-width space (U+200B). Prepended to a transcript line that begins with a
// markdown BLOCK trigger: it is invisible in the rendered doc but shifts the
// trigger off column 0, so the git-sync doc->markdown->doc round-trip keeps the
// line a plain paragraph (see GITMOST_MD_BLOCK_TRIGGER_RE).
const GITMOST_ZWSP = "​";
// A markdown BLOCK-level construct that, sitting at column 0 of a paragraph
// line, the git-sync markdown serializer (packages/prosemirror-markdown
// markdown-converter.ts, `case "paragraph"`) would re-parse into a NON-paragraph
// block on the doc->markdown->doc cycle. That serializer emits paragraph text
// verbatim with NO block-escape (the pre-existing root cause), so a leading
// `#`/`-`/`*`/`+`/`>`, an ordered-list `N.`/`N)`, a code fence ```/~~~, a table
// `|`, or a `> [!info]` callout opener would silently become a heading / list /
// quote / code block / table / callout. The final alternative matches a WHOLE-
// LINE thematic break — solid `---`/`***`/`___` or spaced `- - -`/`_ _ _` (3+ of
// the same `-`/`*`/`_`) — which round-trips into a `horizontalRule`; because
// that node carries NO text, an un-neutralized separator line would LOSE its
// text entirely (worse than the list/quote case). This matches a TRIMMED line's
// start; the transcript's own `You:` / `Speaker N:` prefix begins with a letter
// and never matches, so prefixed lines are left byte-exact.
const GITMOST_MD_BLOCK_TRIGGER_RE =
/^(?:#{1,6}(?:\s|$)|[-*+](?:\s|$)|>|\d+[.)](?:\s|$)|```|~~~|\||([-*_])(?:\s*\1){2,}\s*$)/;
// Append a transcript block BELOW the recording's audio node in a live editor:
// a "Transcript" heading followed by one paragraph per non-empty transcript
// line. The transcript is plain text, `\n`-separated, each line already
// formatted as `You: ...` / `Speaker N: ...` by the native host — line text is
// inserted as a TEXT node (never HTML/markdown), so there is no injection or
// mark-parsing surface. Each kept line is trimmed (drops an indent that would
// both leak into the display and, at col 0, form a markdown block trigger) and,
// if it still begins with a col-0 markdown block trigger, gets an invisible
// zero-width space prepended so the git-sync round-trip cannot turn it into a
// list/quote/heading/callout/code/table (defensive boundary against the
// serializer's missing block-escape). This is best-effort and meant to run
// AFTER the audio has already been inserted; the caller must guard against a
// throw so a transcript failure never fails the (already successful) recording.
// Returns true when a block was inserted, false when there was nothing to
// insert (transcript undefined/empty/not-a-string). A non-string value is a
// no-op, not an error.
export function gitmostInsertTranscriptIntoEditor(
editor: Editor,
transcript: unknown,
): boolean {
if (typeof transcript !== "string") return false;
const lines = transcript
.split("\n")
// Trim each line and drop blank (whitespace-only) ones.
.map((line) => line.trim())
.filter((line) => line.length > 0)
// Neutralize a col-0 markdown block trigger with an invisible ZWSP so the
// git-sync round-trip keeps the line a paragraph. Host lines (`You:` /
// `Speaker N:`) never match and stay byte-exact.
.map((line) =>
GITMOST_MD_BLOCK_TRIGGER_RE.test(line) ? GITMOST_ZWSP + line : line,
);
if (lines.length === 0) return false;
const content = [
{
type: "heading",
attrs: { level: 2 },
content: [{ type: "text", text: "Transcript" }],
},
...lines.map((line) => ({
type: "paragraph",
content: [{ type: "text", text: line }],
})),
];
// Append at the end of the document. On a freshly-created recording page the
// audio node is the last block, so the end position places the transcript
// directly below it.
const endPos = editor.state.doc.content.size;
editor.chain().focus().insertContentAt(endPos, content).run();
return true;
}
// Full insert path used by the open-page bridge (insertRecording): guard the // Full insert path used by the open-page bridge (insertRecording): guard the
// editor, validate/decode the payload, then upload. Never throws — resolves to // editor, validate/decode the payload, then upload. Never throws — resolves to
// a result code. // a result code.
+20 -1
View File
@@ -1,5 +1,6 @@
import { defineConfig, loadEnv } from "vite"; import { defineConfig, loadEnv } from "vite";
import react from "@vitejs/plugin-react"; import react from "@vitejs/plugin-react";
import { compression } from "vite-plugin-compression2";
import * as path from "path"; import * as path from "path";
import { execSync } from "node:child_process"; import { execSync } from "node:child_process";
@@ -53,7 +54,25 @@ export default defineConfig(({ mode }) => {
}, },
APP_VERSION: JSON.stringify(resolveAppVersion(envPath)), APP_VERSION: JSON.stringify(resolveAppVersion(envPath)),
}, },
plugins: [react()], plugins: [
react(),
// Emit .br and .gz next to every built asset so the server can serve the
// precompressed copy (see @fastify/static preCompressed in static.module.ts).
compression({
algorithms: ["brotliCompress", "gzip"],
// vite-plugin-compression2's default `include` only covers text-ish
// bundle output (js/mjs/json/css/html/svg/…). Extend it with the large
// VAD binaries copied from public/vad (.wasm ~26MB, .onnx ~2.3MB) so
// they are brotli/gzip'd once at build time and served via
// @fastify/static preCompressed — otherwise @fastify/compress would
// re-brotli them on EVERY request. The default types are repeated here
// because setting `include` replaces (does not extend) the default.
include: /\.(html|xml|css|json|js|mjs|svg|yaml|yml|toml|wasm|onnx)$/,
// index.html is rewritten at server boot (window.CONFIG injection); a
// precompressed copy would go stale — NEVER precompress it.
exclude: [/index\.html$/],
}),
],
build: { build: {
rolldownOptions: { rolldownOptions: {
output: { output: {
+1
View File
@@ -44,6 +44,7 @@
"@docmost/mcp": "workspace:*", "@docmost/mcp": "workspace:*",
"@docmost/pdf-inspector": "1.9.6", "@docmost/pdf-inspector": "1.9.6",
"@docmost/prosemirror-markdown": "workspace:*", "@docmost/prosemirror-markdown": "workspace:*",
"@fastify/compress": "^9.0.0",
"@fastify/cookie": "^11.0.2", "@fastify/cookie": "^11.0.2",
"@fastify/multipart": "^10.0.0", "@fastify/multipart": "^10.0.0",
"@fastify/static": "^9.1.3", "@fastify/static": "^9.1.3",
@@ -610,6 +610,63 @@ describe('AiAgentRolesService guards', () => {
expect(repo.insert.mock.calls[0][0].name).toBe('Researcher (2)'); expect(repo.insert.mock.calls[0][0].name).toBe('Researcher (2)');
}); });
it('createdRoles lists the installed role (no renamedTo when not renamed)', async () => {
const { service } = makeImportService({});
const res = await service.importFromCatalog('ws-1', 'u1', dto());
expect(res.createdRoles).toEqual([
{ slug: 'researcher', name: 'Researcher' },
]);
expect(res.skippedRoles).toEqual([]);
});
it('createdRoles carries renamedTo on a rename', async () => {
const existing = [makeRow({ id: 'r-x', name: 'Researcher' })];
const { service } = makeImportService({ existing });
const res = await service.importFromCatalog(
'ws-1',
'u1',
dto({ conflict: 'rename' }),
);
expect(res.createdRoles).toEqual([
{ slug: 'researcher', name: 'Researcher', renamedTo: 'Researcher (2)' },
]);
expect(res.skippedRoles).toEqual([]);
});
it('skippedRoles: already-installed slug carries reason "already-installed"', async () => {
const existing = [
makeRow({
id: 'r-existing',
name: 'Old researcher',
source: { slug: 'researcher', language: 'en', version: 1 } as never,
}),
];
const { service } = makeImportService({ existing });
const res = await service.importFromCatalog('ws-1', 'u1', dto());
expect(res.skippedRoles).toEqual([
{
slug: 'researcher',
name: 'Researcher',
reason: 'already-installed',
},
]);
expect(res.createdRoles).toEqual([]);
});
it('skippedRoles: a name collision under conflict:skip carries reason "name-conflict"', async () => {
const existing = [makeRow({ id: 'r-x', name: 'Researcher' })];
const { service } = makeImportService({ existing });
const res = await service.importFromCatalog(
'ws-1',
'u1',
dto({ conflict: 'skip' }),
);
expect(res.skippedRoles).toEqual([
{ slug: 'researcher', name: 'Researcher', reason: 'name-conflict' },
]);
expect(res.createdRoles).toEqual([]);
});
it('dto.slugs filters; an unknown slug becomes an error entry', async () => { it('dto.slugs filters; an unknown slug becomes an error entry', async () => {
const { service, repo } = makeImportService({ const { service, repo } = makeImportService({
bundleRoles: [catalogRole()], bundleRoles: [catalogRole()],
@@ -677,6 +734,15 @@ describe('AiAgentRolesService guards', () => {
// 'a' converged on the concurrent install (skip); 'b' imported; no errors. // 'a' converged on the concurrent install (skip); 'b' imported; no errors.
expect(res).toMatchObject({ created: 1, skipped: 1, renamed: 0 }); expect(res).toMatchObject({ created: 1, skipped: 1, renamed: 0 });
expect(res.errors).toEqual([]); expect(res.errors).toEqual([]);
// The per-role list records 'a' as an already-installed skip (the UI reads
// skippedRoles, not the counter, to render its plaque — assert the array,
// not just the count).
expect(res.skippedRoles).toContainEqual({
slug: 'a',
name: 'A',
reason: 'already-installed',
});
expect(res.createdRoles.map((r) => r.slug)).toEqual(['b']);
// Both inserts were attempted (the batch did not abort on the 23505). // Both inserts were attempted (the batch did not abort on the 23505).
expect(repo.insert).toHaveBeenCalledTimes(2); expect(repo.insert).toHaveBeenCalledTimes(2);
}); });
@@ -305,6 +305,16 @@ export class AiAgentRolesService {
skipped: number; skipped: number;
renamed: number; renamed: number;
errors: { slug: string; message: string }[]; errors: { slug: string; message: string }[];
// Per-role lists alongside the counters (kept for back-compat). The redesigned
// catalog UI needs the actual roles — which were created (and any rename) and
// which were skipped and why — to render an inline result plaque with the
// conflicting role's name and a "Rename & install" affordance.
createdRoles: { slug: string; name: string; renamedTo?: string }[];
skippedRoles: {
slug: string;
name: string;
reason: 'name-conflict' | 'already-installed';
}[];
}> { }> {
const { file, versions } = await this.loadBundleById( const { file, versions } = await this.loadBundleById(
dto.bundleId, dto.bundleId,
@@ -312,6 +322,13 @@ export class AiAgentRolesService {
); );
const errors: { slug: string; message: string }[] = []; const errors: { slug: string; message: string }[] = [];
const createdRoles: { slug: string; name: string; renamedTo?: string }[] =
[];
const skippedRoles: {
slug: string;
name: string;
reason: 'name-conflict' | 'already-installed';
}[] = [];
// Resolve the selected catalog roles (honor dto.slugs; flag unknown ones). // Resolve the selected catalog roles (honor dto.slugs; flag unknown ones).
let selected = file.roles; let selected = file.roles;
@@ -351,16 +368,27 @@ export class AiAgentRolesService {
// Already installed from the catalog in THIS language => skip (use // Already installed from the catalog in THIS language => skip (use
// update-from-catalog). A different language of the same slug still imports. // update-from-catalog). A different language of the same slug still imports.
const installKey = `${role.slug}:${dto.language}`; const installKey = `${role.slug}:${dto.language}`;
const originalName = role.name.trim();
if (installedKeys.has(installKey)) { if (installedKeys.has(installKey)) {
skipped++; skipped++;
skippedRoles.push({
slug: role.slug,
name: originalName,
reason: 'already-installed',
});
continue; continue;
} }
let name = role.name.trim(); let name = originalName;
let didRename = false; let didRename = false;
if (takenNames.has(name.toLowerCase())) { if (takenNames.has(name.toLowerCase())) {
if (dto.conflict === 'skip') { if (dto.conflict === 'skip') {
skipped++; skipped++;
skippedRoles.push({
slug: role.slug,
name: originalName,
reason: 'name-conflict',
});
continue; continue;
} }
// conflict === 'rename': find a free " (N)" suffix. // conflict === 'rename': find a free " (N)" suffix.
@@ -380,6 +408,11 @@ export class AiAgentRolesService {
}); });
created++; created++;
if (didRename) renamed++; if (didRename) renamed++;
createdRoles.push({
slug: role.slug,
name: originalName,
...(didRename ? { renamedTo: name } : {}),
});
takenNames.add(name.toLowerCase()); takenNames.add(name.toLowerCase());
installedKeys.add(installKey); installedKeys.add(installKey);
} catch (err) { } catch (err) {
@@ -391,6 +424,11 @@ export class AiAgentRolesService {
// skipped (already installed) and continue; do NOT abort or error. // skipped (already installed) and continue; do NOT abort or error.
if (isSourceUniqueViolation(err)) { if (isSourceUniqueViolation(err)) {
skipped++; skipped++;
skippedRoles.push({
slug: role.slug,
name: originalName,
reason: 'already-installed',
});
installedKeys.add(installKey); installedKeys.add(installKey);
continue; continue;
} }
@@ -407,7 +445,7 @@ export class AiAgentRolesService {
} }
} }
return { created, skipped, renamed, errors }; return { created, skipped, renamed, errors, createdRoles, skippedRoles };
} }
/** /**
@@ -539,3 +539,115 @@ describe('AiChatToolsService model-friendly input validation (#190)', () => {
expect(result.error?.message).toContain('parameter "pageId": missing (required)'); expect(result.error?.message).toContain('parameter "pageId": missing (required)');
}); });
}); });
/**
* #294 F1 the contract-parity test introspects only the ADVERTISED schema keys
* (buildShape), not the execute bodies. Most execs are unchanged pass-throughs,
* but two wirings actually CHANGED in the migration and are otherwise untested:
* - movePage now forwards the newly-added optional `position` field to the
* client (client.movePage(pageId, parentPageId, position));
* - the table trio unified its `tableRef` param to `table` and must forward it
* positionally. A field destructured under the wrong name would silently pass
* `undefined` to the client (execute is `any`-cast, so tsc won't catch it).
*/
describe('AiChatToolsService #294 changed execute wirings', () => {
const calls: Record<string, unknown[][]> = {
movePage: [],
tableInsertRow: [],
tableDeleteRow: [],
tableUpdateCell: [],
};
const fakeClient: Partial<DocmostClientLike> = {
movePage: (...args: unknown[]) => {
calls.movePage.push(args);
return Promise.resolve({ success: true });
},
tableInsertRow: (...args: unknown[]) => {
calls.tableInsertRow.push(args);
return Promise.resolve({ ok: true });
},
tableDeleteRow: (...args: unknown[]) => {
calls.tableDeleteRow.push(args);
return Promise.resolve({ ok: true });
},
tableUpdateCell: (...args: unknown[]) => {
calls.tableUpdateCell.push(args);
return Promise.resolve({ ok: true });
},
};
const tokenServiceStub = {
generateAccessToken: jest.fn().mockResolvedValue('access-token'),
generateCollabToken: jest.fn().mockResolvedValue('collab-token'),
};
let service: AiChatToolsService;
beforeEach(() => {
for (const k of Object.keys(calls)) calls[k].length = 0;
jest.spyOn(loader, 'loadDocmostMcp').mockResolvedValue(
mockLoaded(function () {
return fakeClient as DocmostClientLike;
} as unknown as loader.DocmostClientCtor),
);
service = new AiChatToolsService(
tokenServiceStub as never,
{} as never,
{} as never,
{} as never,
{} as never,
{
asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }),
} as never,
);
});
afterEach(() => jest.restoreAllMocks());
const buildTools = () =>
service.forUser(
{ id: 'user-1', email: 'u@example.com', workspaceId: 'ws-1' } as never,
'session-1',
'ws-1',
'chat-1',
);
it('movePage forwards the optional position to the client', async () => {
const tools = await buildTools();
await tools.movePage.execute(
{ pageId: 'p1', parentPageId: 'parent1', position: 'a5' } as never,
{} as never,
);
expect(calls.movePage).toEqual([['p1', 'parent1', 'a5']]);
});
it('movePage passes undefined position and null parent when omitted (unchanged behavior)', async () => {
const tools = await buildTools();
await tools.movePage.execute({ pageId: 'p2' } as never, {} as never);
expect(calls.movePage).toEqual([['p2', null, undefined]]);
});
it('tableInsertRow forwards the unified `table` param positionally', async () => {
const tools = await buildTools();
await tools.tableInsertRow.execute(
{ pageId: 'p1', table: '#0', cells: ['a', 'b'], index: 2 } as never,
{} as never,
);
expect(calls.tableInsertRow).toEqual([['p1', '#0', ['a', 'b'], 2]]);
});
it('tableDeleteRow forwards `table` positionally', async () => {
const tools = await buildTools();
await tools.tableDeleteRow.execute(
{ pageId: 'p1', table: '#0', index: 1 } as never,
{} as never,
);
expect(calls.tableDeleteRow).toEqual([['p1', '#0', 1]]);
});
it('tableUpdateCell forwards `table` positionally', async () => {
const tools = await buildTools();
await tools.tableUpdateCell.execute(
{ pageId: 'p1', table: '#0', row: 1, col: 2, text: 'x' } as never,
{} as never,
);
expect(calls.tableUpdateCell).toEqual([['p1', '#0', 1, 2, 'x']]);
});
});
@@ -316,50 +316,27 @@ export class AiChatToolsService {
execute: async () => resolveCurrentPageResult(openedPage), execute: async () => resolveCurrentPageResult(openedPage),
}), }),
getPage: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: // The execute body keeps this layer's { title, markdown } projection.
'Fetch a single page as Markdown by its page id. Returns the page ' + getPage: sharedTool(sharedToolSpecs.getPage, async ({ pageId }) => {
'title and its Markdown content. Inline <span data-comment-id> tags ' + // getPage(pageId) -> { data: filterPage(page, markdown), success }.
'in the markdown are comment highlight anchors (also present for ' + const result = await client.getPage(pageId);
'RESOLVED threads) — treat them as markup, not page text.', const data = (result?.data ?? {}) as {
inputSchema: modelFriendlyInput({ title?: string;
pageId: z.string().describe('The id (or slugId) of the page.'), content?: string;
}), };
execute: async ({ pageId }) => { return {
// getPage(pageId) -> { data: filterPage(page, markdown), success }. title: data.title ?? '',
const result = await client.getPage(pageId); markdown: typeof data.content === 'string' ? data.content : '',
const data = (result?.data ?? {}) as { };
title?: string;
content?: string;
};
return {
title: data.title ?? '',
markdown: typeof data.content === 'string' ? data.content : '',
};
},
}), }),
// --- WRITE tools (all reversible — history/trash; §6.5 / D3) --- // --- WRITE tools (all reversible — history/trash; §6.5 / D3) ---
createPage: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: createPage: sharedTool(
'Create a new page with a Markdown body in a space, optionally under ' + sharedToolSpecs.createPage,
'a parent page. Returns the new page id and title. Reversible: a page ' + async ({ title, content, spaceId, parentPageId }) => {
'can be moved to trash later.',
inputSchema: modelFriendlyInput({
title: z.string().describe('The title of the new page.'),
content: z
.string()
.describe('The page body as Markdown (may be empty).'),
spaceId: z
.string()
.describe('The id of the space to create the page in.'),
parentPageId: z
.string()
.optional()
.describe('Optional parent page id to nest the new page under.'),
}),
execute: async ({ title, content, spaceId, parentPageId }) => {
// createPage(title, content, spaceId, parentPageId?) -> // createPage(title, content, spaceId, parentPageId?) ->
// { data: filterPage(page, markdown), success }. // { data: filterPage(page, markdown), success }.
const result = await client.createPage( const result = await client.createPage(
@@ -375,7 +352,7 @@ export class AiChatToolsService {
}; };
return { id: data.id ?? data.slugId, title: data.title ?? title }; return { id: data.id ?? data.slugId, title: data.title ?? title };
}, },
}), ),
updatePageContent: tool({ updatePageContent: tool({
description: description:
@@ -399,115 +376,46 @@ export class AiChatToolsService {
}, },
}), }),
renamePage: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: renamePage: sharedTool(
"Rename a page (change its title only; the body is untouched). " + sharedToolSpecs.renamePage,
'Reversible: rename back at any time.', async ({ pageId, title }) => {
inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page to rename.'),
title: z.string().describe('The new title.'),
}),
execute: async ({ pageId, title }) => {
// renamePage(pageId, title) -> { success, pageId, title }. // renamePage(pageId, title) -> { success, pageId, title }.
await client.renamePage(pageId, title); await client.renamePage(pageId, title);
return { pageId, title }; return { pageId, title };
}, },
}), ),
movePage: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: // The shared schema adds the optional `position` field this layer lacked
'Move a page under a new parent page, or to the space root when no ' + // before; the execute now forwards it (the client already accepted it).
'parent is given. Reversible: move it back at any time.', movePage: sharedTool(
inputSchema: modelFriendlyInput({ sharedToolSpecs.movePage,
pageId: z.string().describe('The id of the page to move.'), async ({ pageId, parentPageId, position }) => {
parentPageId: z
.string()
.nullable()
.optional()
.describe(
'Target parent page id. Null/omitted moves the page to the ' +
'space root.',
),
}),
execute: async ({ pageId, parentPageId }) => {
// movePage(pageId, parentPageId, position?) -> raw move response. // movePage(pageId, parentPageId, position?) -> raw move response.
await client.movePage(pageId, parentPageId ?? null); await client.movePage(pageId, parentPageId ?? null, position);
return { pageId, parentPageId: parentPageId ?? null, moved: true }; return { pageId, parentPageId: parentPageId ?? null, moved: true };
}, },
),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// GUARDRAIL (§14 H4) preserved: the shared schema exposes ONLY pageId, so
// permanentlyDelete/forceDelete are never part of the input and can never
// be forwarded — the agent physically cannot permanently delete a page.
deletePage: sharedTool(sharedToolSpecs.deletePage, async ({ pageId }) => {
// deletePage(pageId) hits POST /pages/delete with { pageId } only,
// which is the soft-delete (trash) path on the server.
await client.deletePage(pageId);
return { pageId, trashed: true };
}), }),
deletePage: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: // This layer keeps only its own execute-side guards (require a selection
'Move a page to the trash (SOFT delete only — fully reversible; the ' + // for a top-level comment; reject suggestedText on a reply / without a
'page can be restored from trash). This NEVER permanently deletes.', // selection) — the schema+description are shared.
inputSchema: modelFriendlyInput({ createComment: sharedTool(
pageId: z.string().describe('The id of the page to move to trash.'), sharedToolSpecs.createComment,
}), async ({
// GUARDRAIL (§14 H4): the only field ever passed to the client is
// pageId. permanentlyDelete/forceDelete are not part of the schema and
// are never forwarded, so the agent physically cannot permanently
// delete a page through this tool.
execute: async ({ pageId }) => {
// deletePage(pageId) hits POST /pages/delete with { pageId } only,
// which is the soft-delete (trash) path on the server.
await client.deletePage(pageId);
return { pageId, trashed: true };
},
}),
// INTENTIONAL per-transport divergence (not shared): the description is
// tuned for the in-app agent (e.g. "retry with a corrected EXACT selection"
// and "Reversible via the comment UI"); the standalone MCP `create_comment`
// keeps its own wording. Kept per-layer.
createComment: tool({
description:
'Add an INLINE comment to a page, or reply to an existing top-level ' +
'comment (one level only — the backend rejects replies to replies). ' +
'The comment is anchored inline to the given exact `selection` text ' +
'(which gets highlighted); page-level comments are NOT supported. A ' +
"new top-level comment REQUIRES a `selection`. Replies inherit the " +
"parent's anchor and take no selection. If the call fails with a " +
'"selection not found" error, retry with a corrected EXACT selection ' +
'copied verbatim from a single paragraph/block. You may also attach a ' +
'`suggestedText` proposing a replacement for the `selection` (a human ' +
'applies it from the UI); when set, the `selection` must occur exactly ' +
'once in the page. Reversible via the comment UI.',
inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page to comment on.'),
content: z.string().describe('The comment body as Markdown.'),
selection: z
.string()
.min(1)
.max(250)
.optional()
.describe(
'EXACT contiguous text from a SINGLE paragraph/block to anchor ' +
'(highlight) the comment on (<=250 chars, avoid spanning across ' +
'formatting boundaries). Required for a new top-level comment; ' +
'omit only when replying via parentCommentId.',
),
parentCommentId: z
.string()
.optional()
.describe(
'Optional id of a TOP-LEVEL comment to reply to (one level ' +
'of replies only).',
),
suggestedText: z
.string()
.min(1)
.max(2000)
.optional()
.describe(
'Optional proposed replacement (PLAIN TEXT) for the `selection`, ' +
'applied by a human via the UI (never auto-applied). REQUIRES a ' +
'`selection`; NOT allowed on a reply. When set, the `selection` ' +
'must be UNIQUE in the page — expand it with surrounding context ' +
'(still <=250 chars) if it occurs more than once, or the call is ' +
'refused.',
),
}),
execute: async ({
pageId, pageId,
content, content,
selection, selection,
@@ -548,26 +456,17 @@ export class AiChatToolsService {
const data = (result?.data ?? {}) as { id?: string }; const data = (result?.data ?? {}) as { id?: string };
return { commentId: data.id, pageId }; return { commentId: data.id, pageId };
}, },
}), ),
resolveComment: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: resolveComment: sharedTool(
'Resolve or reopen a top-level comment thread (reversible — toggle ' + sharedToolSpecs.resolveComment,
'the resolved flag). Only top-level comments can be resolved.', async ({ commentId, resolved }) => {
inputSchema: modelFriendlyInput({
commentId: z
.string()
.describe('The id of the top-level comment to resolve/reopen.'),
resolved: z
.boolean()
.describe('true to resolve the thread, false to reopen it.'),
}),
execute: async ({ commentId, resolved }) => {
// resolveComment(commentId, resolved) -> { success, commentId, resolved }. // resolveComment(commentId, resolved) -> { success, commentId, resolved }.
await client.resolveComment(commentId, resolved); await client.resolveComment(commentId, resolved);
return { commentId, resolved }; return { commentId, resolved };
}, },
}), ),
// --- READ tools (added) --- // --- READ tools (added) ---
@@ -585,33 +484,12 @@ export class AiChatToolsService {
// hierarchy mode but is worded for the in-app agent; the standalone MCP // hierarchy mode but is worded for the in-app agent; the standalone MCP
// `list_pages` carries its own wording. Kept per-layer so each side tunes // `list_pages` carries its own wording. Kept per-layer so each side tunes
// its own guidance. // its own guidance.
listPages: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: listPages: sharedTool(
'List the most recent pages, optionally scoped to a single space. ' + sharedToolSpecs.listPages,
'Returns a bounded list (default 50, max 100). Pass tree:true (with ' + async ({ spaceId, limit, tree }) =>
"spaceId) to instead get the space's full page hierarchy as a nested tree.",
inputSchema: modelFriendlyInput({
spaceId: z
.string()
.optional()
.describe('Optional space id to scope the listing to.'),
limit: z
.number()
.int()
.min(1)
.max(100)
.optional()
.describe('Maximum number of pages (1-100).'),
tree: z
.boolean()
.optional()
.describe(
'When true, return the full page hierarchy of the given space as a nested tree (children arrays) instead of the recent-pages flat list. Requires spaceId; ignores limit.',
),
}),
execute: async ({ spaceId, limit, tree }) =>
await client.listPages(spaceId, limit, tree), await client.listPages(spaceId, limit, tree),
}), ),
listSidebarPages: tool({ listSidebarPages: tool({
description: description:
@@ -656,41 +534,34 @@ export class AiChatToolsService {
}), }),
), ),
// NOT shared (kept inline): the MCP tool name `table_get` is noun-first
// while this key is `getTable` (verb-first), breaking the
// snake_case(inAppKey) convention the shared registry enforces. Its
// reference parameter is still named `table` (was `tableRef`) so it matches
// the migrated table row/cell tools below.
getTable: tool({ getTable: tool({
description: description:
'Read a table as a matrix of cell texts (plus a parallel cellIds ' + 'Read a table as a matrix of cell texts (plus a parallel cellIds ' +
'matrix so cells can be addressed for rich edits).', 'matrix so cells can be addressed for rich edits).',
inputSchema: modelFriendlyInput({ inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page.'), pageId: z.string().describe('The id of the page.'),
tableRef: z table: z
.string() .string()
.describe( .describe(
'"#<index>" from getOutline, or a block id of any node inside ' + '"#<index>" from the page outline, or a block id of any node ' +
'the table.', 'inside the table.',
), ),
}), }),
execute: async ({ pageId, tableRef }) => execute: async ({ pageId, table }) =>
await client.getTable(pageId, tableRef), await client.getTable(pageId, table),
}), }),
listComments: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: listComments: sharedTool(
'List comments on a page in one call. By DEFAULT only ACTIVE ' + sharedToolSpecs.listComments,
'threads are returned; resolved threads (a resolved top-level ' + async ({ pageId, includeResolved }) =>
'comment and all its replies) are hidden and their count reported ' +
'as `resolvedThreadsHidden` so you can re-query with ' +
'`includeResolved: true` to see everything. Returns ' +
'`{ items, resolvedThreadsHidden }`. Content is returned as Markdown.',
inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page.'),
includeResolved: z
.boolean()
.optional()
.describe('default only active threads; true — include resolved'),
}),
execute: async ({ pageId, includeResolved }) =>
await client.listComments(pageId, includeResolved), await client.listComments(pageId, includeResolved),
}), ),
getComment: tool({ getComment: tool({
description: 'Fetch a single comment by id (content as Markdown).', description: 'Fetch a single comment by id (content as Markdown).',
@@ -700,26 +571,12 @@ export class AiChatToolsService {
execute: async ({ commentId }) => await client.getComment(commentId), execute: async ({ commentId }) => await client.getComment(commentId),
}), }),
checkNewComments: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: checkNewComments: sharedTool(
'Find new comments across a space (optionally scoped to a subtree) ' + sharedToolSpecs.checkNewComments,
'created after a given timestamp.', async ({ spaceId, since, parentPageId }) =>
inputSchema: modelFriendlyInput({
spaceId: z.string().describe('The id of the space to scan.'),
since: z
.string()
.describe('An ISO-8601 timestamp; only comments created after it.'),
parentPageId: z
.string()
.optional()
.describe(
'Optional page id to scope the scan to that page and its ' +
'descendants.',
),
}),
execute: async ({ spaceId, since, parentPageId }) =>
await client.checkNewComments(spaceId, since, parentPageId), await client.checkNewComments(spaceId, since, parentPageId),
}), ),
listShares: sharedTool( listShares: sharedTool(
sharedToolSpecs.listShares, sharedToolSpecs.listShares,
@@ -749,19 +606,14 @@ export class AiChatToolsService {
await client.diffPageVersions(pageId, from, to), await client.diffPageVersions(pageId, from, to),
), ),
exportPageMarkdown: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: exportPageMarkdown: sharedTool(
'Export a page to a single self-contained Docmost-flavoured ' + sharedToolSpecs.exportPageMarkdown,
'Markdown file (meta + body + comment threads). Lossless round-trip ' + async ({ pageId }) => {
'with importPageMarkdown.',
inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page to export.'),
}),
execute: async ({ pageId }) => {
const markdown = await client.exportPageMarkdown(pageId); const markdown = await client.exportPageMarkdown(pageId);
return { markdown }; return { markdown };
}, },
}), ),
// --- WRITE tools (added; reversible via page history/trash) --- // --- WRITE tools (added; reversible via page history/trash) ---
@@ -811,28 +663,12 @@ export class AiChatToolsService {
async ({ pageId, nodeId }) => await client.deleteNode(pageId, nodeId), async ({ pageId, nodeId }) => await client.deleteNode(pageId, nodeId),
), ),
updatePageJson: tool({ // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
description: // The execute body keeps this layer's content normalization (parity with
"Replace a page's body with a full ProseMirror document — a full " + // the standalone MCP server, index.ts update_page_json).
'overwrite — and/or update its title. Minimal example content: ' + updatePageJson: sharedTool(
'{"type":"doc","content":[{"type":"paragraph","content":' + sharedToolSpecs.updatePageJson,
'[{"type":"text","text":"Hi"}]}]}. The content arg may be a JSON ' + async ({ pageId, content, title }) => {
'object or a JSON string (both accepted). Omit content for a ' +
'title-only update. Reversible: the previous version is kept in page ' +
'history.',
inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page to update.'),
content: z
.any()
.optional()
.describe(
'Full ProseMirror doc {"type":"doc","content":[...]} (JSON ' +
'object or JSON string); omit for a title-only update.',
),
title: z.string().optional().describe('Optional new title.'),
}),
execute: async ({ pageId, content, title }) => {
// Parity with the standalone MCP server (index.ts update_page_json):
// undefined/null pass through as undefined (title-only / no-op); any // undefined/null pass through as undefined (title-only / no-op); any
// string is JSON.parsed (so an empty string "" throws, matching the // string is JSON.parsed (so an empty string "" throws, matching the
// MCP server); an object is passed through unchanged. // MCP server); an object is passed through unchanged.
@@ -845,66 +681,29 @@ export class AiChatToolsService {
} }
return await client.updatePageJson(pageId, doc, title); return await client.updatePageJson(pageId, doc, title);
}, },
}), ),
// NOT in the shared registry: this layer names the table argument // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// `tableRef`, while the standalone MCP tool names it `table` (index.ts). // The table reference parameter was unified to `table` (was `tableRef`).
// Sharing one buildShape would rename a model-facing parameter on one tableInsertRow: sharedTool(
// transport, so the table row/cell tools stay per-layer by design. sharedToolSpecs.tableInsertRow,
tableInsertRow: tool({ async ({ pageId, table, cells, index }) =>
description: await client.tableInsertRow(pageId, table, cells, index),
'Insert a row of plain-text cells into a table. Reversible via ' + ),
'page history.',
inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page.'),
tableRef: z
.string()
.describe('"#<index>" from getOutline, or a block id in the table.'),
cells: z.array(z.string()).describe('The cell texts for the row.'),
index: z
.number()
.int()
.optional()
.describe('0-based insert position (omit/out-of-range to append).'),
}),
execute: async ({ pageId, tableRef, cells, index }) =>
await client.tableInsertRow(pageId, tableRef, cells, index),
}),
// NOT shared — same `tableRef` (here) vs `table` (MCP) parameter-name // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// divergence as tableInsertRow. tableDeleteRow: sharedTool(
tableDeleteRow: tool({ sharedToolSpecs.tableDeleteRow,
description: async ({ pageId, table, index }) =>
'Delete a table row at a 0-based index. Reversible via page history.', await client.tableDeleteRow(pageId, table, index),
inputSchema: modelFriendlyInput({ ),
pageId: z.string().describe('The id of the page.'),
tableRef: z
.string()
.describe('"#<index>" from getOutline, or a block id in the table.'),
index: z.number().int().describe('0-based row index to delete.'),
}),
execute: async ({ pageId, tableRef, index }) =>
await client.tableDeleteRow(pageId, tableRef, index),
}),
// NOT shared — same `tableRef` (here) vs `table` (MCP) parameter-name // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// divergence as tableInsertRow. tableUpdateCell: sharedTool(
tableUpdateCell: tool({ sharedToolSpecs.tableUpdateCell,
description: async ({ pageId, table, row, col, text }) =>
'Set the plain-text content of a table cell at [row, col] (0-based). ' + await client.tableUpdateCell(pageId, table, row, col, text),
'Reversible via page history.', ),
inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page.'),
tableRef: z
.string()
.describe('"#<index>" from getOutline, or a block id in the table.'),
row: z.number().int().describe('0-based row index.'),
col: z.number().int().describe('0-based column index.'),
text: z.string().describe('The new cell text.'),
}),
execute: async ({ pageId, tableRef, row, col, text }) =>
await client.tableUpdateCell(pageId, tableRef, row, col, text),
}),
copyPageContent: sharedTool( copyPageContent: sharedTool(
sharedToolSpecs.copyPageContent, sharedToolSpecs.copyPageContent,
@@ -918,25 +717,14 @@ export class AiChatToolsService {
await client.importPageMarkdown(pageId, markdown), await client.importPageMarkdown(pageId, markdown),
), ),
// INTENTIONAL per-transport divergence (not shared): adds a security // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// confirmation framing ("Only share when the user explicitly asked, since // Both layers already carried the security-confirmation framing, so there
// this exposes the page to anyone with the link") for the in-app agent; the // was no real divergence to preserve — only wording drift.
// standalone MCP `share_page` keeps the plain public-URL wording. sharePage: sharedTool(
sharePage: tool({ sharedToolSpecs.sharePage,
description: async ({ pageId, searchIndexing }) =>
'Make a page PUBLICLY accessible and return its public URL. ' +
'Reversible via unsharePage. Only share when the user explicitly ' +
'asked, since this exposes the page to anyone with the link.',
inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page to share.'),
searchIndexing: z
.boolean()
.optional()
.describe('Allow public search engines to index it (default true).'),
}),
execute: async ({ pageId, searchIndexing }) =>
await client.sharePage(pageId, searchIndexing), await client.sharePage(pageId, searchIndexing),
}), ),
unsharePage: sharedTool( unsharePage: sharedTool(
sharedToolSpecs.unsharePage, sharedToolSpecs.unsharePage,
@@ -100,54 +100,26 @@ export const INLINE_TOOL_TIERS: Record<
tier: 'core', tier: 'core',
catalogLine: 'getCurrentPage — the page the user is currently viewing.', catalogLine: 'getCurrentPage — the page the user is currently viewing.',
}, },
getPage: { // NOTE: getPage and listPages moved to @docmost/mcp's SHARED_TOOL_SPECS
tier: 'core', // (#294); they carry their own tier ('core') + catalogLine there.
catalogLine: 'getPage — fetch a page as Markdown by its id.', // NOTE: createComment, listComments and resolveComment moved to
}, // @docmost/mcp's SHARED_TOOL_SPECS (#294); they carry their own tier +
listPages: { // catalogLine there. getComment stays inline (MCP-only shape divergence is
tier: 'core', // n/a — it simply has no shared spec).
catalogLine: "listPages — list recent pages, or a space's full page tree.",
},
listComments: {
tier: 'core',
catalogLine: 'listComments — list all comments on a page (including resolved).',
},
getComment: { getComment: {
tier: 'core', tier: 'core',
catalogLine: 'getComment — fetch a single comment by id.', catalogLine: 'getComment — fetch a single comment by id.',
}, },
createComment: {
tier: 'core',
catalogLine:
'createComment — add an inline comment (optionally with a suggested edit).',
},
resolveComment: {
tier: 'core',
catalogLine: 'resolveComment — resolve or reopen a comment thread.',
},
// --- deferred inline --- // --- deferred inline ---
createPage: { // NOTE: createPage, renamePage, movePage, deletePage, updatePageJson and
tier: 'deferred', // exportPageMarkdown moved to @docmost/mcp's SHARED_TOOL_SPECS (#294); they
catalogLine: 'createPage — create a new page with a Markdown body in a space.', // carry their own deferred tier + catalogLine there.
},
updatePageContent: { updatePageContent: {
tier: 'deferred', tier: 'deferred',
catalogLine: catalogLine:
"updatePageContent — replace a page's body (and optionally title) with new Markdown.", "updatePageContent — replace a page's body (and optionally title) with new Markdown.",
}, },
renamePage: {
tier: 'deferred',
catalogLine: "renamePage — change a page's title only (body untouched).",
},
movePage: {
tier: 'deferred',
catalogLine: 'movePage — move a page under a new parent or to the space root.',
},
deletePage: {
tier: 'deferred',
catalogLine: 'deletePage — move a page to trash (soft delete, reversible).',
},
listSidebarPages: { listSidebarPages: {
tier: 'deferred', tier: 'deferred',
catalogLine: catalogLine:
@@ -157,42 +129,21 @@ export const INLINE_TOOL_TIERS: Record<
tier: 'deferred', tier: 'deferred',
catalogLine: 'getTable — read a table as a matrix of cell texts and cell ids.', catalogLine: 'getTable — read a table as a matrix of cell texts and cell ids.',
}, },
checkNewComments: { // NOTE: tableInsertRow, tableDeleteRow and tableUpdateCell moved to
tier: 'deferred', // @docmost/mcp's SHARED_TOOL_SPECS (#294); they carry their own deferred tier +
catalogLine: // catalogLine there. getTable stays inline (its MCP name table_get breaks the
'checkNewComments — find comments in a space created after a timestamp.', // snake_case(inAppKey) convention, so it has no shared spec).
}, // NOTE: checkNewComments moved to @docmost/mcp's SHARED_TOOL_SPECS (#294);
// it carries its own deferred tier + catalogLine there.
getPageHistory: { getPageHistory: {
tier: 'deferred', tier: 'deferred',
catalogLine: catalogLine:
'getPageHistory — fetch one page-history version with its ProseMirror content.', 'getPageHistory — fetch one page-history version with its ProseMirror content.',
}, },
exportPageMarkdown: { // NOTE: sharePage moved to @docmost/mcp's SHARED_TOOL_SPECS (#294); it carries
tier: 'deferred', // its own deferred tier + catalogLine there. transformPage stays inline (its
catalogLine: // schema deliberately diverges — it omits the deleteComments field the MCP
'exportPageMarkdown — export a page to self-contained Markdown (body + comments).', // docmost_transform exposes, a comment-deletion guardrail).
},
updatePageJson: {
tier: 'deferred',
catalogLine:
"updatePageJson — overwrite a page's body with a full ProseMirror document.",
},
tableInsertRow: {
tier: 'deferred',
catalogLine: 'tableInsertRow — insert a row of plain-text cells into a table.',
},
tableDeleteRow: {
tier: 'deferred',
catalogLine: 'tableDeleteRow — delete a table row at a 0-based index.',
},
tableUpdateCell: {
tier: 'deferred',
catalogLine: 'tableUpdateCell — set the text of a table cell at [row, col].',
},
sharePage: {
tier: 'deferred',
catalogLine: 'sharePage — make a page publicly accessible and return its URL.',
},
transformPage: { transformPage: {
tier: 'deferred', tier: 'deferred',
catalogLine: "transformPage — run a sandboxed JS transform over a page's document.", catalogLine: "transformPage — run a sandboxed JS transform over a page's document.",
@@ -474,6 +474,19 @@ export class AttachmentController {
const fileSize = Number(attachment.fileSize); const fileSize = Number(attachment.fileSize);
const rangeHeader = req.headers.range; const rangeHeader = req.headers.range;
// Opt this download route out of the global @fastify/compress hook.
// Attachment bytes are final and mostly binary, so on-the-fly compression
// only burns CPU — and on the 206/Range branch it is actively corrupting:
// compress decides purely by Content-Type, so for a compressible mime
// (application/octet-stream fallback, image/svg+xml, text/*) it would gzip
// the byte slice and drop Content-Length while Content-Range still
// describes the RAW offsets and the status stays 206. A resuming client
// (`curl -C -`, download managers) then appends the encoded bytes as if
// raw and ends up with a broken file. @fastify/compress skips whenever the
// request carries `x-no-compression` (see its onSend hook), so setting it
// here covers both the 200 (full file) and 206 (range) responses.
req.headers['x-no-compression'] = 'true';
res.header('Accept-Ranges', 'bytes'); res.header('Accept-Ranges', 'bytes');
res.header( res.header(
'Content-Security-Policy', 'Content-Security-Policy',
@@ -4,7 +4,6 @@ import { EventName } from '../../common/events/event.contants';
import { InjectQueue } from '@nestjs/bullmq'; import { InjectQueue } from '@nestjs/bullmq';
import { QueueJob, QueueName } from '../../integrations/queue/constants'; import { QueueJob, QueueName } from '../../integrations/queue/constants';
import { Queue } from 'bullmq'; import { Queue } from 'bullmq';
import { EnvironmentService } from '../../integrations/environment/environment.service';
/** /**
* Thin snapshot of a page node carried inside domain events so the WebSocket * Thin snapshot of a page node carried inside domain events so the WebSocket
@@ -112,48 +111,24 @@ export class PageListener {
private readonly logger = new Logger(PageListener.name); private readonly logger = new Logger(PageListener.name);
constructor( constructor(
private readonly environmentService: EnvironmentService,
@InjectQueue(QueueName.SEARCH_QUEUE) private searchQueue: Queue,
@InjectQueue(QueueName.AI_QUEUE) private aiQueue: Queue, @InjectQueue(QueueName.AI_QUEUE) private aiQueue: Queue,
) {} ) {}
@OnEvent(EventName.PAGE_CREATED) @OnEvent(EventName.PAGE_CREATED)
async handlePageCreated(event: PageEvent) { async handlePageCreated(event: PageEvent) {
const { pageIds, workspaceId } = event; const { pageIds, workspaceId } = event;
if (this.isTypesense()) {
await this.searchQueue.add(QueueJob.PAGE_CREATED, {
pageIds,
});
}
await this.aiQueue.add(QueueJob.PAGE_CREATED, { pageIds, workspaceId }); await this.aiQueue.add(QueueJob.PAGE_CREATED, { pageIds, workspaceId });
} }
@OnEvent(EventName.PAGE_UPDATED)
async handlePageUpdated(event: PageEvent) {
const { pageIds } = event;
await this.searchQueue.add(QueueJob.PAGE_UPDATED, { pageIds });
}
@OnEvent(EventName.PAGE_DELETED) @OnEvent(EventName.PAGE_DELETED)
async handlePageDeleted(event: PageEvent) { async handlePageDeleted(event: PageEvent) {
const { pageIds, workspaceId } = event; const { pageIds, workspaceId } = event;
if (this.isTypesense()) {
await this.searchQueue.add(QueueJob.PAGE_DELETED, { pageIds });
}
await this.aiQueue.add(QueueJob.PAGE_DELETED, { pageIds, workspaceId }); await this.aiQueue.add(QueueJob.PAGE_DELETED, { pageIds, workspaceId });
} }
@OnEvent(EventName.PAGE_SOFT_DELETED) @OnEvent(EventName.PAGE_SOFT_DELETED)
async handlePageSoftDeleted(event: PageEvent) { async handlePageSoftDeleted(event: PageEvent) {
const { pageIds, workspaceId } = event; const { pageIds, workspaceId } = event;
if (this.isTypesense()) {
await this.searchQueue.add(QueueJob.PAGE_SOFT_DELETED, { pageIds });
}
await this.aiQueue.add(QueueJob.PAGE_SOFT_DELETED, { await this.aiQueue.add(QueueJob.PAGE_SOFT_DELETED, {
pageIds, pageIds,
workspaceId, workspaceId,
@@ -163,14 +138,6 @@ export class PageListener {
@OnEvent(EventName.PAGE_RESTORED) @OnEvent(EventName.PAGE_RESTORED)
async handlePageRestored(event: PageEvent) { async handlePageRestored(event: PageEvent) {
const { pageIds, workspaceId } = event; const { pageIds, workspaceId } = event;
if (this.isTypesense()) {
await this.searchQueue.add(QueueJob.PAGE_RESTORED, { pageIds });
}
await this.aiQueue.add(QueueJob.PAGE_RESTORED, { pageIds, workspaceId }); await this.aiQueue.add(QueueJob.PAGE_RESTORED, { pageIds, workspaceId });
} }
isTypesense(): boolean {
return this.environmentService.getSearchDriver() === 'typesense';
}
} }
@@ -4,7 +4,6 @@ import { EventName } from '../../common/events/event.contants';
import { InjectQueue } from '@nestjs/bullmq'; import { InjectQueue } from '@nestjs/bullmq';
import { QueueJob, QueueName } from '../../integrations/queue/constants'; import { QueueJob, QueueName } from '../../integrations/queue/constants';
import { Queue } from 'bullmq'; import { Queue } from 'bullmq';
import { EnvironmentService } from '../../integrations/environment/environment.service';
export class SpaceEvent { export class SpaceEvent {
spaceId: string; spaceId: string;
@@ -15,22 +14,12 @@ export class SpaceListener {
private readonly logger = new Logger(SpaceListener.name); private readonly logger = new Logger(SpaceListener.name);
constructor( constructor(
private readonly environmentService: EnvironmentService,
@InjectQueue(QueueName.SEARCH_QUEUE) private searchQueue: Queue,
@InjectQueue(QueueName.AI_QUEUE) private aiQueue: Queue, @InjectQueue(QueueName.AI_QUEUE) private aiQueue: Queue,
) {} ) {}
@OnEvent(EventName.SPACE_DELETED) @OnEvent(EventName.SPACE_DELETED)
async handleSpaceDeleted(event: SpaceEvent) { async handleSpaceDeleted(event: SpaceEvent) {
const { spaceId } = event; const { spaceId } = event;
if (this.isTypesense()) {
await this.searchQueue.add(QueueJob.SPACE_DELETED, { spaceId });
}
await this.aiQueue.add(QueueJob.SPACE_DELETED, { spaceId }); await this.aiQueue.add(QueueJob.SPACE_DELETED, { spaceId });
} }
isTypesense(): boolean {
return this.environmentService.getSearchDriver() === 'typesense';
}
} }
@@ -4,7 +4,6 @@ import { EventName } from '../../common/events/event.contants';
import { InjectQueue } from '@nestjs/bullmq'; import { InjectQueue } from '@nestjs/bullmq';
import { QueueJob, QueueName } from '../../integrations/queue/constants'; import { QueueJob, QueueName } from '../../integrations/queue/constants';
import { Queue } from 'bullmq'; import { Queue } from 'bullmq';
import { EnvironmentService } from '../../integrations/environment/environment.service';
export class WorkspaceEvent { export class WorkspaceEvent {
workspaceId: string; workspaceId: string;
@@ -15,22 +14,12 @@ export class WorkspaceListener {
private readonly logger = new Logger(WorkspaceListener.name); private readonly logger = new Logger(WorkspaceListener.name);
constructor( constructor(
private readonly environmentService: EnvironmentService,
@InjectQueue(QueueName.SEARCH_QUEUE) private searchQueue: Queue,
@InjectQueue(QueueName.AI_QUEUE) private aiQueue: Queue, @InjectQueue(QueueName.AI_QUEUE) private aiQueue: Queue,
) {} ) {}
@OnEvent(EventName.WORKSPACE_DELETED) @OnEvent(EventName.WORKSPACE_DELETED)
async handlePageDeleted(event: WorkspaceEvent) { async handlePageDeleted(event: WorkspaceEvent) {
const { workspaceId } = event; const { workspaceId } = event;
if (this.isTypesense()) {
await this.searchQueue.add(QueueJob.WORKSPACE_DELETED, { workspaceId });
}
await this.aiQueue.add(QueueJob.WORKSPACE_DELETED, { workspaceId }); await this.aiQueue.add(QueueJob.WORKSPACE_DELETED, { workspaceId });
} }
isTypesense(): boolean {
return this.environmentService.getSearchDriver() === 'typesense';
}
} }
+4
View File
@@ -24,6 +24,10 @@ const migrator = new Migrator({
path, path,
migrationFolder, migrationFolder,
}), }),
// Match the startup auto-migrator (migration.service.ts): a back-dated
// migration from a long-lived branch must be applied, not rejected as
// "corrupted migrations" (incident #361). See that file for the full rationale.
allowUnorderedMigrations: true,
}); });
run(db, migrator, migrationFolder); run(db, migrator, migrationFolder);
@@ -19,6 +19,16 @@ export class MigrationService {
path, path,
migrationFolder: path.join(__dirname, '..', 'migrations'), migrationFolder: path.join(__dirname, '..', 'migrations'),
}), }),
// A long-lived branch can add a migration whose timestamped filename sorts
// BEFORE migrations already applied in prod (e.g. #234's 20260627 landing
// after 20260704 was live). With the default (ordered) setting the startup
// migrator then sees "corrupted migrations" — the applied set is no longer a
// prefix of the sorted list — throws, and the app crash-loops on boot
// (incident #361: 502s for ~11 min). allowUnorderedMigrations runs any
// not-yet-applied migration regardless of filename order, so a back-dated
// migration is applied instead of bricking startup. A CI order-gate still
// discourages back-dating; this is the runtime safety net.
allowUnorderedMigrations: true,
}); });
const { error, results } = await migrator.migrateToLatest(); const { error, results } = await migrator.migrateToLatest();
@@ -0,0 +1,124 @@
import { readFileSync } from 'fs';
import { streamText, Output } from 'ai';
import { MockLanguageModelV3, simulateReadableStream } from 'ai/test';
/**
* Regression tests for patches/ai@6.0.134.patch (server heap OOM on long
* autonomous agent runs, #184).
*
* Unpatched ai@6.0.134 substitutes the default text() output strategy even
* when the caller passes NO `output` option. Its createOutputTransformStream
* then accumulates the ENTIRE turn text and, on EVERY text-delta, enqueues a
* flat snapshot of all text so far as `partialOutput` (O(n^2) memory). Those
* snapshots pile up in the never-consumed leftover tee() branch of
* DefaultStreamTextResult.baseStream, which is what OOM'd production during a
* ~28k-chunk agent turn. The pnpm patch skips partialOutput production
* entirely when no output strategy was requested, while keeping per-delta
* streaming granularity.
*/
describe('ai@6.0.134 pnpm patch: no partialOutput accumulation without an output strategy', () => {
const makeModel = () =>
new MockLanguageModelV3({
doStream: async () => ({
stream: simulateReadableStream({
chunks: [
{ type: 'stream-start' as const, warnings: [] },
{ type: 'text-start' as const, id: '1' },
{ type: 'text-delta' as const, id: '1', delta: 'Hello' },
{ type: 'text-delta' as const, id: '1', delta: ', ' },
{ type: 'text-delta' as const, id: '1', delta: 'world!' },
{ type: 'text-end' as const, id: '1' },
{
type: 'finish' as const,
finishReason: { unified: 'stop' as const, raw: 'stop' },
usage: {
inputTokens: {
total: 1,
noCache: undefined,
cacheRead: undefined,
cacheWrite: undefined,
},
outputTokens: { total: 1, text: 1, reasoning: undefined },
},
},
],
}),
}),
});
it('preserves per-delta streaming granularity in textStream', async () => {
const result = streamText({ model: makeModel(), prompt: 'hi' });
const deltas: string[] = [];
for await (const delta of result.textStream) {
deltas.push(delta);
}
// The patch must NOT coalesce or drop deltas: three model deltas arrive
// as three separate textStream chunks.
expect(deltas).toEqual(['Hello', ', ', 'world!']);
});
it('emits NO partialOutput values when the caller did not request an output strategy', async () => {
const result = streamText({ model: makeModel(), prompt: 'hi' });
// Fully consume the primary stream first (mirrors production usage).
for await (const _ of result.textStream) {
// drain
}
const partials: unknown[] = [];
for await (const partial of result.experimental_partialOutputStream) {
partials.push(partial);
}
// TRIPWIRE: on unpatched ai@6.0.134 the default text() output strategy
// yields one cumulative partial per text-delta here (['Hello', 'Hello, ',
// 'Hello, world!']). An empty stream proves the patch is applied and no
// cumulative snapshots are being produced (and thus none can pile up in
// the leftover internal tee branch).
expect(partials).toEqual([]);
});
it('preserves cumulative partialOutput when the caller DOES request an output strategy', async () => {
// PRESERVE-BRANCH GUARD: the patch only short-circuits partialOutput when
// `output == null`. When an output strategy IS set (here Output.text()),
// createOutputTransformStream must fall through to the ORIGINAL code path
// and keep publishing cumulative snapshots, so object/text-output consumers
// behave byte-identically to unpatched ai. A careless re-port that routed
// output-set calls into the skip branch would leave partialOutput empty and
// silently break those consumers — this test is the tripwire for that.
const result = streamText({
model: makeModel(),
prompt: 'hi',
experimental_output: Output.text(),
});
// Drain the primary stream fully and accumulate the complete output text.
let fullText = '';
for await (const delta of result.textStream) {
fullText += delta;
}
const partials: string[] = [];
for await (const partial of result.experimental_partialOutputStream) {
partials.push(partial);
}
// With a strategy set, partialOutput must be PRESERVED (non-empty) and
// cumulative: the last emitted partial equals the full accumulated text.
expect(partials.length).toBeGreaterThan(0);
expect(partials[partials.length - 1]).toBe(fullText);
expect(fullText).toBe('Hello, world!');
});
it('both installed dist builds (CJS and ESM) carry the patch marker', () => {
// Secondary guard: pins the patch to BOTH bundles the SDK ships, since
// the NestJS server consumes CJS while other tooling may load ESM.
const cjsPath = require.resolve('ai');
const mjsPath = cjsPath.replace(/index\.js$/, 'index.mjs');
expect(cjsPath).toMatch(/index\.js$/);
expect(readFileSync(cjsPath, 'utf8')).toContain('PATCH(docmost');
expect(readFileSync(mjsPath, 'utf8')).toContain('PATCH(docmost');
});
});
@@ -2,15 +2,36 @@ import { FastifyReply, FastifyRequest } from 'fastify';
import { isStreamingResponse } from './metrics.constants'; import { isStreamingResponse } from './metrics.constants';
import { observeHttp } from './metrics.registry'; import { observeHttp } from './metrics.registry';
// URL path prefixes served by @fastify/static (client build output under
// client/dist). `/assets/` holds the content-hashed bundle (index-*.js,
// chunk-*.js) — a NEW set of names every deploy, i.e. an UNBOUNDED label set
// (#362); the others (/vad/, /brand/, /locales/, /icons/ — copied verbatim from
// public/) have stable names, so they are merely repetitive per-file labels
// rather than unbounded. Either way none of these belong in the API-route
// histogram: collapse them all to one bounded `static` label. (Edge latency for
// static is already measured by Traefik's traefik_router_request_duration_*.)
const STATIC_PATH_PREFIXES = [
'/assets/',
'/vad/',
'/brand/',
'/locales/',
'/icons/',
];
/** /**
* Resolve the BOUNDED route label for an HTTP response. * Resolve the BOUNDED route label for an HTTP response.
* *
* HARD REQUIREMENT (#355): use the ROUTE TEMPLATE (`/pages/:id`), NEVER the raw * HARD REQUIREMENT (#355): use the ROUTE TEMPLATE (`/pages/:id`), NEVER a raw
* URL (`/pages/abc-123`), so label cardinality stays finite. Fastify exposes the * URL (`/pages/abc-123` or `/assets/index-CAbxDtto.js`), so label cardinality
* matched template on `req.routeOptions.url`. On 404s (no route matched) that is * stays finite. Fastify exposes the matched template on `req.routeOptions.url`,
* missing collapse to the literal `unknown`. * BUT @fastify/static serves each file through a route whose matched url is the
* raw (hashed) file path so for static assets that value is itself unbounded.
* Detect static requests by their path prefix FIRST and collapse to `static`;
* otherwise use the route template; on a 404 (no route matched) `unknown`.
*/ */
export function resolveRouteLabel(req: FastifyRequest): string { export function resolveRouteLabel(req: FastifyRequest): string {
const path = (req.url ?? '').split('?', 1)[0];
if (STATIC_PATH_PREFIXES.some((p) => path.startsWith(p))) return 'static';
const url = req.routeOptions?.url; const url = req.routeOptions?.url;
return typeof url === 'string' && url.length > 0 ? url : 'unknown'; return typeof url === 'string' && url.length > 0 ? url : 'unknown';
} }
@@ -46,7 +46,6 @@ export class MetricsBullService implements OnModuleInit, OnModuleDestroy {
@InjectQueue(QueueName.GENERAL_QUEUE) generalQueue: Queue, @InjectQueue(QueueName.GENERAL_QUEUE) generalQueue: Queue,
@InjectQueue(QueueName.BILLING_QUEUE) billingQueue: Queue, @InjectQueue(QueueName.BILLING_QUEUE) billingQueue: Queue,
@InjectQueue(QueueName.FILE_TASK_QUEUE) fileTaskQueue: Queue, @InjectQueue(QueueName.FILE_TASK_QUEUE) fileTaskQueue: Queue,
@InjectQueue(QueueName.SEARCH_QUEUE) searchQueue: Queue,
@InjectQueue(QueueName.AI_QUEUE) aiQueue: Queue, @InjectQueue(QueueName.AI_QUEUE) aiQueue: Queue,
@InjectQueue(QueueName.HISTORY_QUEUE) historyQueue: Queue, @InjectQueue(QueueName.HISTORY_QUEUE) historyQueue: Queue,
@InjectQueue(QueueName.NOTIFICATION_QUEUE) notificationQueue: Queue, @InjectQueue(QueueName.NOTIFICATION_QUEUE) notificationQueue: Queue,
@@ -58,7 +57,6 @@ export class MetricsBullService implements OnModuleInit, OnModuleDestroy {
{ label: 'general', queue: generalQueue }, { label: 'general', queue: generalQueue },
{ label: 'billing', queue: billingQueue }, { label: 'billing', queue: billingQueue },
{ label: 'file-task', queue: fileTaskQueue }, { label: 'file-task', queue: fileTaskQueue },
{ label: 'search', queue: searchQueue },
{ label: 'ai', queue: aiQueue }, { label: 'ai', queue: aiQueue },
{ label: 'history', queue: historyQueue }, { label: 'history', queue: historyQueue },
{ label: 'notification', queue: notificationQueue }, { label: 'notification', queue: notificationQueue },
@@ -25,6 +25,61 @@ describe('resolveRouteLabel (histogram route label)', () => {
const req = { url: '/x' } as unknown as FastifyRequest; const req = { url: '/x' } as unknown as FastifyRequest;
expect(resolveRouteLabel(req)).toBe('unknown'); expect(resolveRouteLabel(req)).toBe('unknown');
}); });
it.each([
'/assets/index-CAbxDtto.js',
'/assets/chunk-3OPIFGDE-CJOt9nr5.js',
'/assets/excalidraw-menu-DpsI0kFW.js',
'/vad/silero_vad_v5.onnx',
'/brand/logo.svg',
'/locales/en.json',
'/icons/app-icon-192x192.png',
])('collapses hashed/static asset %p to "static" (#362 cardinality)', (url) => {
// @fastify/static serves each file through a route whose matched url is the
// raw (hashed) file path, so routeOptions.url is itself unbounded here.
const req = {
url,
routeOptions: { url },
} as unknown as FastifyRequest;
const label = resolveRouteLabel(req);
expect(label).toBe('static');
expect(label).not.toContain('.js');
expect(label).not.toContain('index-');
});
it('strips the query string before the static-prefix check', () => {
const req = {
url: '/assets/index-CAbxDtto.js?v=2',
routeOptions: { url: '/assets/index-CAbxDtto.js' },
} as unknown as FastifyRequest;
expect(resolveRouteLabel(req)).toBe('static');
});
it('does NOT collapse a real API route that merely mentions assets', () => {
// A templated API route is kept as-is; only the static path PREFIXES collapse.
const req = {
url: '/api/pages/assets-guide',
routeOptions: { url: '/api/pages/:id' },
} as unknown as FastifyRequest;
expect(resolveRouteLabel(req)).toBe('/api/pages/:id');
});
it.each([
// The TRAILING SLASH on the prefix is the anti-false-collapse guard: a path
// that is the prefix WITHOUT its slash, or merely shares the prefix as a
// substring of a longer segment, must NOT collapse. These would collapse
// under a buggy `includes('/assets/')` / slashless-prefix impl.
'/assets',
'/assetsx/foo.js',
'/iconset/x.png',
])('does NOT collapse the prefix-boundary case %p', (url) => {
const req = {
url,
routeOptions: { url: '/some/:route' },
} as unknown as FastifyRequest;
expect(resolveRouteLabel(req)).not.toBe('static');
expect(resolveRouteLabel(req)).toBe('/some/:route');
});
}); });
describe('isStreamingResponse (SSE exclusion)', () => { describe('isStreamingResponse (SSE exclusion)', () => {
@@ -4,7 +4,6 @@ export enum QueueName {
GENERAL_QUEUE = '{general-queue}', GENERAL_QUEUE = '{general-queue}',
BILLING_QUEUE = '{billing-queue}', BILLING_QUEUE = '{billing-queue}',
FILE_TASK_QUEUE = '{file-task-queue}', FILE_TASK_QUEUE = '{file-task-queue}',
SEARCH_QUEUE = '{search-queue}',
AI_QUEUE = '{ai-queue}', AI_QUEUE = '{ai-queue}',
HISTORY_QUEUE = '{history-queue}', HISTORY_QUEUE = '{history-queue}',
NOTIFICATION_QUEUE = '{notification-queue}', NOTIFICATION_QUEUE = '{notification-queue}',
@@ -32,12 +31,6 @@ export enum QueueJob {
IMPORT_TASK = 'import-task', IMPORT_TASK = 'import-task',
EXPORT_TASK = 'export-task', EXPORT_TASK = 'export-task',
SEARCH_INDEX_PAGE = 'search-index-page',
SEARCH_INDEX_PAGES = 'search-index-pages',
SEARCH_INDEX_COMMENT = 'search-index-comment',
SEARCH_INDEX_COMMENTS = 'search-index-comments',
SEARCH_INDEX_ATTACHMENT = 'search-index-attachment',
SEARCH_INDEX_ATTACHMENTS = 'search-index-attachments',
SEARCH_REMOVE_PAGE = 'search-remove-page', SEARCH_REMOVE_PAGE = 'search-remove-page',
SEARCH_REMOVE_ASSET = 'search-remove-attachment', SEARCH_REMOVE_ASSET = 'search-remove-attachment',
SEARCH_REMOVE_FACE = 'search-remove-comment', SEARCH_REMOVE_FACE = 'search-remove-comment',
@@ -57,14 +57,6 @@ import { GeneralQueueProcessor } from './processors/general-queue.processor';
attempts: 1, attempts: 1,
}, },
}), }),
BullModule.registerQueue({
name: QueueName.SEARCH_QUEUE,
defaultJobOptions: {
removeOnComplete: true,
removeOnFail: true,
attempts: 2,
},
}),
BullModule.registerQueue({ BullModule.registerQueue({
name: QueueName.AI_QUEUE, name: QueueName.AI_QUEUE,
defaultJobOptions: { defaultJobOptions: {
@@ -0,0 +1,35 @@
import { resolveStaticAssetHeaders } from './static.module';
// Unit tests for the static-asset cache classifier extracted from the
// @fastify/static setHeaders callback (precedent: sandbox.controller.spec.ts).
describe('resolveStaticAssetHeaders', () => {
it('marks a content-hashed /assets/ file immutable and sets Vary', () => {
const headers = resolveStaticAssetHeaders(
'/app/apps/client/dist/assets/index-a1b2c3.js',
);
expect(headers['cache-control']).toBe(
'public, max-age=31536000, immutable',
);
expect(headers['vary']).toBe('Accept-Encoding');
});
it('makes index.html always revalidate (never immutable)', () => {
const headers = resolveStaticAssetHeaders(
'/app/apps/client/dist/index.html',
);
expect(headers['cache-control']).toBe(
'no-cache, no-store, must-revalidate',
);
expect(headers['vary']).toBe('Accept-Encoding');
});
it('does NOT mark a non-hashed asset immutable but still sets Vary', () => {
const headers = resolveStaticAssetHeaders(
'/app/apps/client/dist/locales/en.json',
);
// No immutable cache-control — this path keeps @fastify/static's default
// etag/last-modified revalidation.
expect(headers['cache-control']).toBeUndefined();
expect(headers['vary']).toBe('Accept-Encoding');
});
});
@@ -5,6 +5,46 @@ import * as fs from 'node:fs';
import fastifyStatic from '@fastify/static'; import fastifyStatic from '@fastify/static';
import { EnvironmentService } from '../environment/environment.service'; import { EnvironmentService } from '../environment/environment.service';
/**
* Resolve the response headers for a statically served client asset.
*
* Extracted from the @fastify/static `setHeaders` callback so the cache
* classification stays a pure, unit-testable function (see
* static.module.spec.ts).
*
* `Vary: Accept-Encoding` is emitted for every static response because
* @fastify/static negotiates a precompressed .br/.gz neighbour by the client's
* Accept-Encoding but does NOT set Vary itself. Without it a shared/proxy cache
* keyed on the URL alone could store the brotli variant and later serve it to a
* client that only sent `Accept-Encoding: identity`/gzip an undecodable body.
* This matters most for the immutable /assets/ files, which proxies may keep
* for a year.
*/
export function resolveStaticAssetHeaders(
filePath: string,
): Record<string, string> {
const headers: Record<string, string> = { vary: 'Accept-Encoding' };
// Content-hashed files under /assets/ never change for a given URL, so they
// can be cached forever and skip revalidation entirely.
if (filePath.includes('/assets/')) {
headers['cache-control'] = 'public, max-age=31536000, immutable';
return headers;
}
// index.html is rewritten at boot (window.CONFIG injection) and on every
// deploy — it must be revalidated on every load.
if (filePath.endsWith('index.html')) {
headers['cache-control'] = 'no-cache, no-store, must-revalidate';
return headers;
}
// Everything else (locales, vad, icons, manifest) is NOT content-hashed and
// changes between deploys, so it keeps @fastify/static's default
// etag/last-modified revalidation — do NOT mark it immutable.
return headers;
}
@Module({}) @Module({})
export class StaticModule implements OnModuleInit { export class StaticModule implements OnModuleInit {
constructor( constructor(
@@ -72,6 +112,16 @@ export class StaticModule implements OnModuleInit {
await app.register(fastifyStatic, { await app.register(fastifyStatic, {
root: clientDistPath, root: clientDistPath,
wildcard: false, wildcard: false,
// Serve the build-time .br/.gz neighbour when the client accepts it
// (see vite-plugin-compression2 in apps/client/vite.config.ts).
preCompressed: true,
setHeaders: (res, filePath) => {
for (const [name, value] of Object.entries(
resolveStaticAssetHeaders(filePath),
)) {
res.setHeader(name, value);
}
},
}); });
app.get(RENDER_PATH, (req: any, res: any) => { app.get(RENDER_PATH, (req: any, res: any) => {
+12
View File
@@ -10,6 +10,7 @@ import { TransformHttpResponseInterceptor } from './common/interceptors/http-res
import { WsRedisIoAdapter } from './ws/adapter/ws-redis.adapter'; import { WsRedisIoAdapter } from './ws/adapter/ws-redis.adapter';
import fastifyMultipart from '@fastify/multipart'; import fastifyMultipart from '@fastify/multipart';
import fastifyCookie from '@fastify/cookie'; import fastifyCookie from '@fastify/cookie';
import fastifyCompress from '@fastify/compress';
import fastifyIp from 'fastify-ip'; import fastifyIp from 'fastify-ip';
import { InternalLogFilter } from './common/logger/internal-log-filter'; import { InternalLogFilter } from './common/logger/internal-log-filter';
import { EnvironmentService } from './integrations/environment/environment.service'; import { EnvironmentService } from './integrations/environment/environment.service';
@@ -63,6 +64,17 @@ async function bootstrap() {
await app.register(fastifyIp); await app.register(fastifyIp);
await app.register(fastifyMultipart); await app.register(fastifyMultipart);
await app.register(fastifyCookie); await app.register(fastifyCookie);
// Compress dynamic responses (API JSON, the rewritten share-SEO HTML) when the
// client accepts br/gzip. @fastify/compress only compresses content-types that
// mime-db flags `compressible` (application/json, text/html, …); `text/event-stream`
// is not in mime-db, so SSE is never compressed by the allowlist. The AI-chat
// stream additionally hijacks the raw socket (pipeUIMessageStreamToResponse ->
// res.raw in ai-chat.service.ts), bypassing Fastify's reply/onSend lifecycle
// entirely, so this hook can never buffer that stream.
await app.register(fastifyCompress, {
// Skip tiny payloads where compression overhead outweighs the savings.
threshold: 1024,
});
const environmentService = app.get(EnvironmentService); const environmentService = app.get(EnvironmentService);
const frameHeader = resolveFrameHeader( const frameHeader = resolveFrameHeader(
+38 -18
View File
@@ -38,6 +38,24 @@ export const TEST_DATABASE_URL =
process.env.TEST_DATABASE_URL ?? process.env.TEST_DATABASE_URL ??
'postgresql://docmost:docmost_dev_pw@localhost:5432/docmost_test'; 'postgresql://docmost:docmost_dev_pw@localhost:5432/docmost_test';
// Build the raw postgres.js client (mirrors database.module.ts: max pool,
// silenced notices, bigint-as-number parsing). Kept separate so the singleton
// can hold a reference to bound its shutdown in destroyTestDb.
function buildTestSql(url: string = TEST_DATABASE_URL) {
return postgres(url, {
max: 5,
onnotice: () => {},
types: {
bigint: {
to: 20,
from: [20, 1700],
serialize: (value: number) => value.toString(),
parse: (value: string) => Number.parseInt(value),
},
},
});
}
/** /**
* Build a Kysely instance that MIRRORS the app's setup in database.module.ts: * Build a Kysely instance that MIRRORS the app's setup in database.module.ts:
* PostgresJSDialect over postgres(), CamelCasePlugin, and the bigint type * PostgresJSDialect over postgres(), CamelCasePlugin, and the bigint type
@@ -47,38 +65,40 @@ export const TEST_DATABASE_URL =
*/ */
export function buildTestDb(url: string = TEST_DATABASE_URL): Kysely<any> { export function buildTestDb(url: string = TEST_DATABASE_URL): Kysely<any> {
return new Kysely<any>({ return new Kysely<any>({
dialect: new PostgresJSDialect({ dialect: new PostgresJSDialect({ postgres: buildTestSql(url) }),
postgres: postgres(url, {
max: 5,
onnotice: () => {},
types: {
bigint: {
to: 20,
from: [20, 1700],
serialize: (value: number) => value.toString(),
parse: (value: string) => Number.parseInt(value),
},
},
}),
}),
plugins: [new CamelCasePlugin()], plugins: [new CamelCasePlugin()],
}); });
} }
let singleton: Kysely<any> | undefined; let singleton: Kysely<any> | undefined;
let singletonSql: ReturnType<typeof buildTestSql> | undefined;
/** Lazily-built shared Kysely for the test suite (one per worker; maxWorkers=1). */ /** Lazily-built shared Kysely for the test suite (one per worker; maxWorkers=1). */
export function getTestDb(): Kysely<any> { export function getTestDb(): Kysely<any> {
if (!singleton) { if (!singleton) {
singleton = buildTestDb(); singletonSql = buildTestSql();
singleton = new Kysely<any>({
dialect: new PostgresJSDialect({ postgres: singletonSql }),
plugins: [new CamelCasePlugin()],
});
} }
return singleton; return singleton;
} }
export async function destroyTestDb(): Promise<void> { export async function destroyTestDb(): Promise<void> {
if (singleton) { if (!singleton) return;
await singleton.destroy(); const sql = singletonSql;
singleton = undefined; // Clear the refs first so a hung end() cannot leave a half-closed singleton.
singleton = undefined;
singletonSql = undefined;
// postgres.js .end() waits indefinitely for in-flight queries by default; a
// leaked/stuck pooled connection would hang the afterAll hook (a 60s hook
// timeout in CI). Bound the shutdown: the { timeout } grace period lets
// active queries drain, then force-closes lingering sockets so teardown
// always completes. We close the pool directly instead of Kysely.destroy()
// (which would call sql.end() again with no timeout).
if (sql) {
await sql.end({ timeout: 5 });
} }
} }
+5 -1
View File
@@ -4,10 +4,14 @@
"testEnvironment": "node", "testEnvironment": "node",
"testRegex": ".e2e-spec.ts$", "testRegex": ".e2e-spec.ts$",
"transform": { "transform": {
"prosemirror-markdown/build/.+\\.js$": [
"babel-jest",
{ "presets": [["@babel/preset-env", { "targets": { "node": "current" } }]] }
],
"^.+\\.(t|j)sx?$": ["ts-jest", { "tsconfig": { "allowJs": true } }] "^.+\\.(t|j)sx?$": ["ts-jest", { "tsconfig": { "allowJs": true } }]
}, },
"transformIgnorePatterns": [ "transformIgnorePatterns": [
"/node_modules/(?!(\\.pnpm/)?(nanoid|uuid|image-dimensions|marked|happy-dom|lib0|@sindresorhus[+/][a-z0-9-]+|escape-string-regexp|p-limit|yocto-queue)(@|/))" "/node_modules/(?!(\\.pnpm/)?(nanoid|uuid|image-dimensions|marked|happy-dom|lib0|@sindresorhus[+/][a-z0-9-]+|escape-string-regexp|p-limit|yocto-queue|@docmost/prosemirror-markdown)(@|/))"
], ],
"moduleNameMapper": { "moduleNameMapper": {
"^@docmost/db/(.*)$": "<rootDir>/../src/database/$1", "^@docmost/db/(.*)$": "<rootDir>/../src/database/$1",
+6 -1
View File
@@ -4,14 +4,19 @@
"testRegex": ".*\\.int-spec\\.ts$", "testRegex": ".*\\.int-spec\\.ts$",
"testPathIgnorePatterns": ["/node_modules/"], "testPathIgnorePatterns": ["/node_modules/"],
"transform": { "transform": {
"prosemirror-markdown/build/.+\\.js$": [
"babel-jest",
{ "presets": [["@babel/preset-env", { "targets": { "node": "current" } }]] }
],
"^.+\\.(t|j)sx?$": "ts-jest" "^.+\\.(t|j)sx?$": "ts-jest"
}, },
"transformIgnorePatterns": [ "transformIgnorePatterns": [
"/node_modules/(?!(\\.pnpm/)?(nanoid|uuid|image-dimensions|marked|happy-dom|lib0)(@|/))" "/node_modules/(?!(\\.pnpm/)?(nanoid|uuid|image-dimensions|marked|happy-dom|lib0|@docmost/prosemirror-markdown)(@|/))"
], ],
"testEnvironment": "node", "testEnvironment": "node",
"testTimeout": 60000, "testTimeout": 60000,
"maxWorkers": 1, "maxWorkers": 1,
"forceExit": true,
"globalSetup": "<rootDir>/test/integration/global-setup.ts", "globalSetup": "<rootDir>/test/integration/global-setup.ts",
"globalTeardown": "<rootDir>/test/integration/global-teardown.ts", "globalTeardown": "<rootDir>/test/integration/global-teardown.ts",
"moduleNameMapper": { "moduleNameMapper": {
+5
View File
@@ -12,6 +12,11 @@ services:
ports: ports:
- "3000:3000" - "3000:3000"
restart: unless-stopped restart: unless-stopped
# The app already serves precompressed (brotli/gzip) static assets with
# long-lived cache headers and gzips dynamic API responses. For the best
# cold-load latency you can OPTIONALLY put a reverse proxy (caddy / nginx /
# traefik) in front with HTTP/2 (or HTTP/3) and brotli enabled — none is
# required for compression to work.
volumes: volumes:
- docmost:/app/data/storage - docmost:/app/data/storage
+9
View File
@@ -131,5 +131,14 @@ const { Client } = require("pg");
7. **Migrations don't auto-run in dev** — run `migration:latest` after every pull 7. **Migrations don't auto-run in dev** — run `migration:latest` after every pull
or branch switch. or branch switch.
8. **Automation (Playwright): type into the BODY editor, not the title.** A page has
two `.ProseMirror` editors — `[aria-label='Page title']` (non-collab) and
`[aria-label='Page content']` (the collab body). `document.querySelector('.ProseMirror')`
returns the TITLE editor, so typing there never changes body content and `mod+S`
versions nothing. Target `[aria-label='Page content']`, confirm it's collab-bound
(`el.editor.extensionManager.extensions.some(e=>e.name==='collaboration')`), and
wait ~10-12s for the store debounce before asserting `pages.content` changed. Full
testing methodology + traps: **[how-to-test.md](how-to-test.md)**.
See also the **Commands** and **Architecture → Two server processes** sections in See also the **Commands** and **Architecture → Two server processes** sections in
[`AGENTS.md`](../AGENTS.md). [`AGENTS.md`](../AGENTS.md).
+108
View File
@@ -0,0 +1,108 @@
# How to test the application (browser E2E + out-of-band)
How to actually verify a feature end-to-end against a running stand — driving the
**real app in a browser** and confirming results **out-of-band** in the DB/git, not
through the same API you're supposed to be testing. Written from real false-positives
that wasted hours (see **Traps** — read them before you write a test).
Prereq: a running stand — see **[dev-stand.md](dev-stand.md)**. Automation uses
Playwright (`pip install playwright && python -m playwright install chromium`).
## Principles
1. **Drive the behaviour under test through the browser.** The stand exists so you
exercise the real UI + realtime-collab + server path. Using `POST /api/pages/*` to
perform the action you're validating tests the API, not the app — an e2e suite can
do that. API calls are fine ONLY for one-time setup/fixtures, never for the
interaction you're asserting on.
2. **Evidence before claim.** Nothing "passes" without an artifact: a DB row, a git
diff, a screenshot looked at as an image. If you can't show it, you didn't verify it.
3. **Verify out-of-band.** Judge results from a source independent of the UI: `psql`
against the DB, a fresh `git clone` of a synced repo, a hard reload. Optimistic UI
lies about persistence.
4. **Disconfirm by default.** For each feature, actively try to prove it's broken
before concluding it works. Reload after every create/edit/save.
5. **Recon actuatability FIRST.** Before building editor tests, confirm the
interaction even works in your harness (does a typed edit reach the DB?). Skipping
this is how you ship a pile of tests that all silently exercised the wrong thing.
## The editor: two ProseMirror instances (READ THIS)
A page has **two** `.ProseMirror` editors:
| index | selector | role | collab? |
|---|---|---|---|
| 0 | `[aria-label='Page title']` | title field | **NO** (16 exts, no `collaboration`) |
| 1 | `[aria-label='Page content']` | body | **YES** (95 exts, has `collaboration`) |
`document.querySelector('.ProseMirror')` returns the **title** editor (first match).
Type there and you edit the title only — body page content never changes, so `mod+S`
"versions" unchanged content and every content test silently no-ops.
**Always target the body editor** and confirm it's collab-bound before typing:
```js
const el = document.querySelector("[aria-label='Page content']");
el.editor.extensionManager.extensions.some(e => e.name === 'collaboration'); // must be true
```
Body edits emit ~20 `/collab` websocket frames while typing and land in
`pages.content` after the **hocuspocus store debounce (~10s)** — so **wait ~12s**
before asserting persistence (checking at 6–8s is a false negative). `mod+S` (the
`save-version` stateless message) flushes immediately, so a version created right
after a settled body edit holds the typed text.
## A known-good browser flow
```
1. goto /s/<space-slug> # the "Create page" button lives in the space sidebar, not /home
2. click button[aria-label='Create page'] # fully UI-driven page creation
3. type into [aria-label='Page title'] # optional title
4. click [aria-label='Page content'] → type body text
5. wait ~12s (store debounce)
6. assert pages.content changed (psql) # out-of-band
7. mod+S / menu Save → assert page_history row (psql)
8. reload / fresh context → re-assert (persistence round-trip)
```
Auth: log in ONCE, save `storage_state.json`, reuse it across pages/agents (re-login
per run trips shared rate-limits). Cookie-based session authorizes both REST and the
collab websocket.
## Judging out-of-band
```bash
# page content / history
docker exec <db> psql -U docmost -d docmost -tAc \
"select coalesce(kind,'null'), content::text from page_history where page_id='<id>' order by created_at;"
# git-sync round-trip: clone the space repo and diff against what you pushed
git clone http://<user>:<pass>@127.0.0.1:3000/git/<spaceId>.git /tmp/x
```
`page_history.content` is full JSON — parse it, don't truncate the snippet, or a
marker check misses. For sync/async features (autosave, git-sync, idle-flush) use an
active probe: write a unique marker, wait past the debounce/poll window, re-read
out-of-band, ≥2 iterations — never conclude "broken" from a single snapshot.
## Traps (each of these produced a false result in a real run)
- **Wrong editor.** Typed into `.ProseMirror` (= title). Edits never touched body
content. → target `[aria-label='Page content']`.
- **Checked persistence too early.** Store debounce ~10s; a 6–8s check reads stale.
- **Truncated the DB snapshot** below where the test marker sits → false "content
missing".
- **API-seeded the content under test**, then "verified" the feature — that validated
the API, not the app.
- **Reused a fixed marker on a non-rebooted stand** → title/row collisions inflate
counts (`count==2`). Use a unique per-run marker (timestamp).
- **Idle/async read once** and called it "permanently broken" — it was mid-debounce.
- **Concluded env-limitation without a cross-build control.** If unsure whether a
failure is your harness or the product, run the SAME harness against a known-good
build; a divergence localizes it.
## Scope note
Some paths genuinely need a human in a real browser (rich drag-drop, native file
pickers, clipboard, and anything the harness can't actuate). Label those UNTESTED in
the report — "handled gracefully" is not "works". Keep four states distinct:
verified-working, defect, untested, env-limitation.
+2 -1
View File
@@ -96,7 +96,8 @@
"pnpm": { "pnpm": {
"patchedDependencies": { "patchedDependencies": {
"scimmy@1.3.5": "patches/scimmy@1.3.5.patch", "scimmy@1.3.5": "patches/scimmy@1.3.5.patch",
"yjs@13.6.30": "patches/yjs@13.6.30.patch" "yjs@13.6.30": "patches/yjs@13.6.30.patch",
"ai@6.0.134": "patches/ai@6.0.134.patch"
}, },
"overrides": { "overrides": {
"prosemirror-changeset": "2.4.0", "prosemirror-changeset": "2.4.0",
+83 -355
View File
@@ -118,56 +118,19 @@ export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
// transport exposes a `tree:true` mode that returns the full nested hierarchy; // transport exposes a `tree:true` mode that returns the full nested hierarchy;
// the in-app copy keeps the same tree option but is worded for the in-app agent. // the in-app copy keeps the same tree option but is worded for the in-app agent.
// Kept per-layer so each side can tune its own guidance. // Kept per-layer so each side can tune its own guidance.
server.registerTool( // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294). This
"list_pages", // transport keeps applying its own defaults (limit=50, tree=false) in execute.
{ registerShared(SHARED_TOOL_SPECS.listPages, async ({ spaceId, limit, tree }) => {
description: const result = await docmostClient.listPages(spaceId, limit ?? 50, tree ?? false);
"List most recent pages in a space ordered by updatedAt (descending). " + return jsonContent(result);
"Returns a bounded list (default 50, max 100) — use search for lookups " + });
"in large spaces. Pass tree:true (with spaceId) to instead get the " +
"space's full page hierarchy as a nested tree.",
inputSchema: {
spaceId: z.string().optional(),
limit: z
.number()
.int()
.min(1)
.max(100)
.optional()
.describe("Max pages to return (default 50, max 100)"),
tree: z
.boolean()
.optional()
.describe(
"When true, return the space's full page hierarchy as a nested tree (each node has a children array) instead of the recent-by-updatedAt flat list. Requires spaceId; ignores limit.",
),
},
},
async ({ spaceId, limit, tree }) => {
const result = await docmostClient.listPages(spaceId, limit ?? 50, tree ?? false);
return jsonContent(result);
},
);
// Tool: get_page // Tool: get_page
server.registerTool( // Schema + description now live in the shared registry (#294).
"get_page", registerShared(SHARED_TOOL_SPECS.getPage, async ({ pageId }) => {
{ const page = await docmostClient.getPage(pageId);
description: return jsonContent(page);
"Get page details with content converted to Markdown. The conversion is " + });
"LOSSY (block ids, exact table/callout structure are approximated); for a " +
"lossless representation use get_page_json. Inline <span data-comment-id> " +
"tags in the markdown are comment highlight anchors (also present for " +
"RESOLVED threads) — treat them as markup, not page text.",
inputSchema: {
pageId: z.string().min(1),
},
},
async ({ pageId }) => {
const page = await docmostClient.getPage(pageId);
return jsonContent(page);
},
);
// Tool: get_page_json // Tool: get_page_json
registerShared(SHARED_TOOL_SPECS.getPageJson, async ({ pageId }) => { registerShared(SHARED_TOOL_SPECS.getPageJson, async ({ pageId }) => {
@@ -201,6 +164,10 @@ registerShared(
); );
// Tool: table_get // Tool: table_get
// NOT in the shared registry: the MCP tool name `table_get` is noun-first while
// the in-app key is `getTable` (verb-first), breaking the snake_case(inAppKey)
// convention the shared registry enforces (shared-tool-specs.contract.spec.ts).
// Renaming the public MCP tool would break external clients, so it stays inline.
server.registerTool( server.registerTool(
"table_get", "table_get",
{ {
@@ -223,25 +190,10 @@ server.registerTool(
); );
// Tool: table_insert_row // Tool: table_insert_row
// NOT in the shared registry: this transport names the table argument `table`, // Schema + description now live in the shared registry (#294); the `table`
// while the in-app tool names it `tableRef` (ai-chat-tools.service.ts). Sharing // parameter name is the canonical one (the in-app layer was unified to it).
// one buildShape would rename a public MCP parameter, so the table row/cell registerShared(
// tools stay per-transport by design. SHARED_TOOL_SPECS.tableInsertRow,
server.registerTool(
"table_insert_row",
{
description:
"Insert a row of plain-text cells into a table. `table` = `#<index>` or " +
"a block id inside it. `cells` = text per column (padded to the table's " +
"column count; error if more cells than columns). `index` = 0-based " +
"insert position (0 inserts before the header); omit to append at the end.",
inputSchema: {
pageId: z.string().min(1),
table: z.string().min(1),
cells: z.array(z.string()),
index: z.number().int().optional(),
},
},
async ({ pageId, table, cells, index }) => { async ({ pageId, table, cells, index }) => {
const result = await docmostClient.tableInsertRow( const result = await docmostClient.tableInsertRow(
pageId, pageId,
@@ -254,22 +206,9 @@ server.registerTool(
); );
// Tool: table_delete_row // Tool: table_delete_row
// NOT shared — same `table` (here) vs `tableRef` (in-app) parameter-name // Schema + description now live in the shared registry (#294).
// divergence as table_insert_row. registerShared(
server.registerTool( SHARED_TOOL_SPECS.tableDeleteRow,
"table_delete_row",
{
description:
"Delete the row at 0-based `index` from a table (`table` = `#<index>` or " +
"a block id inside it). Refuses to delete the table's only row. An " +
"out-of-range `index` throws. Deleting `index` 0 removes the header row, " +
"and the next row becomes the new header.",
inputSchema: {
pageId: z.string().min(1),
table: z.string().min(1),
index: z.number().int(),
},
},
async ({ pageId, table, index }) => { async ({ pageId, table, index }) => {
const result = await docmostClient.tableDeleteRow(pageId, table, index); const result = await docmostClient.tableDeleteRow(pageId, table, index);
return jsonContent(result); return jsonContent(result);
@@ -277,24 +216,9 @@ server.registerTool(
); );
// Tool: table_update_cell // Tool: table_update_cell
// NOT shared — same `table` (here) vs `tableRef` (in-app) parameter-name // Schema + description now live in the shared registry (#294).
// divergence as table_insert_row. registerShared(
server.registerTool( SHARED_TOOL_SPECS.tableUpdateCell,
"table_update_cell",
{
description:
"Set the plain-text content of cell [row,col] (0-based) in a table " +
"(`table` = `#<index>` or a block id inside it). Replaces the cell's " +
"content with a single text paragraph; for rich formatting use patch_node " +
"on the cell's paragraph id from table_get.",
inputSchema: {
pageId: z.string().min(1),
table: z.string().min(1),
row: z.number().int(),
col: z.number().int(),
text: z.string(),
},
},
async ({ pageId, table, row, col, text }) => { async ({ pageId, table, row, col, text }) => {
const result = await docmostClient.tableUpdateCell( const result = await docmostClient.tableUpdateCell(
pageId, pageId,
@@ -308,22 +232,9 @@ server.registerTool(
); );
// Tool: create_page // Tool: create_page
server.registerTool( // Schema + description now live in the shared registry (#294).
"create_page", registerShared(
{ SHARED_TOOL_SPECS.createPage,
description:
"Create a new page from Markdown in a space. Pass parentPageId to nest " +
"it under a parent; omit it to create at the space root.",
inputSchema: {
title: z.string().min(1).describe("Title of the page"),
content: z.string().min(1).describe("Markdown content"),
spaceId: z.string().min(1),
parentPageId: z
.string()
.optional()
.describe("Optional parent page ID to nest under"),
},
},
async ({ title, content, spaceId, parentPageId }) => { async ({ title, content, spaceId, parentPageId }) => {
const result = await docmostClient.createPage( const result = await docmostClient.createPage(
title, title,
@@ -336,32 +247,11 @@ server.registerTool(
); );
// Tool: update_page_json // Tool: update_page_json
server.registerTool( // Schema + description now live in the shared registry (#294). The execute body
"update_page_json", // keeps this transport's content normalization (parse a JSON-string content,
{ // pass undefined/null through for a title-only/no-op update).
description: registerShared(
"Replace a page's content with a raw ProseMirror JSON document " + SHARED_TOOL_SPECS.updatePageJson,
"(lossless write: preserves the block ids, callouts, tables and " +
"attributes you pass in). Typical flow: get_page_json -> modify the " +
"JSON -> update_page_json. Keep existing node ids intact so heading " +
"anchors and history stay stable. Minimal full-doc example: " +
'{"type":"doc","content":[{"type":"paragraph","content":' +
'[{"type":"text","text":"Hi"}]}]}. `content` may be a JSON object or a ' +
"JSON string (both accepted), and is OPTIONAL: omit it to update only " +
"the title (though prefer rename_page for a title-only change). " +
"Supplying neither content nor title is an error.",
inputSchema: {
pageId: z.string().min(1).describe("ID of the page to update"),
content: z
.any()
.optional()
.describe(
'ProseMirror document {"type":"doc","content":[...]} (JSON object or ' +
"JSON string). Omit to rename only.",
),
title: z.string().optional().describe("Optional new title"),
},
},
async ({ pageId, content, title }) => { async ({ pageId, content, title }) => {
// Only parse/validate the document when it was actually supplied; when it // Only parse/validate the document when it was actually supplied; when it
// is omitted, pass it straight through so the client performs a title-only // is omitted, pass it straight through so the client performs a title-only
@@ -379,26 +269,11 @@ server.registerTool(
); );
// Tool: export_page_markdown // Tool: export_page_markdown
server.registerTool( // Schema + description now live in the shared registry (#294).
"export_page_markdown", registerShared(SHARED_TOOL_SPECS.exportPageMarkdown, async ({ pageId }) => {
{ const md = await docmostClient.exportPageMarkdown(pageId);
description: return { content: [{ type: "text" as const, text: md }] };
"Export a page to a single self-contained, lossless Docmost-flavoured " + });
"Markdown file (custom extensions): YAML-free meta header, body with " +
"inline comment anchors and diagrams, and a trailing comments-thread " +
"block. Designed for a download -> edit body -> import_page_markdown " +
"round-trip that preserves everything, including comment highlights. " +
"Comment THREADS are preserved in the file but are not re-pushed to the " +
"server on import.",
inputSchema: {
pageId: z.string().min(1),
},
},
async ({ pageId }) => {
const md = await docmostClient.exportPageMarkdown(pageId);
return { content: [{ type: "text" as const, text: md }] };
},
);
// Tool: import_page_markdown // Tool: import_page_markdown
registerShared( registerShared(
@@ -422,22 +297,11 @@ registerShared(
); );
// Tool: rename_page // Tool: rename_page
server.registerTool( // Schema + description now live in the shared registry (#294).
"rename_page", registerShared(SHARED_TOOL_SPECS.renamePage, async ({ pageId, title }) => {
{ const result = await docmostClient.renamePage(pageId, title);
description: return jsonContent(result);
"Rename a page (change its title only) without touching or resending " + });
"its content.",
inputSchema: {
pageId: z.string().min(1).describe("ID of the page to rename"),
title: z.string().min(1).describe("New title"),
},
},
async ({ pageId, title }) => {
const result = await docmostClient.renamePage(pageId, title);
return jsonContent(result);
},
);
// Tool: edit_page_text // Tool: edit_page_text
registerShared(SHARED_TOOL_SPECS.editPageText, async ({ pageId, edits }) => { registerShared(SHARED_TOOL_SPECS.editPageText, async ({ pageId, edits }) => {
@@ -516,6 +380,10 @@ registerShared(SHARED_TOOL_SPECS.deleteNode, async ({ pageId, nodeId }) => {
}); });
// Tool: insert_image // Tool: insert_image
// MCP-only by design (NOT in the shared registry): the in-app AI-chat agent
// exposes no image tools (insert/replace), so there is no second layer to unify
// — a SHARED_TOOL_SPECS entry's tier/catalogLine are in-app metadata and the
// catalog-partition test forbids a spec without a live in-app tool (#294).
server.registerTool( server.registerTool(
"insert_image", "insert_image",
{ {
@@ -561,6 +429,7 @@ server.registerTool(
); );
// Tool: replace_image // Tool: replace_image
// MCP-only by design (see insert_image): no in-app equivalent, stays inline.
server.registerTool( server.registerTool(
"replace_image", "replace_image",
{ {
@@ -603,25 +472,10 @@ server.registerTool(
); );
// Tool: share_page // Tool: share_page
// INTENTIONAL per-transport divergence (not shared): the in-app copy adds a // Schema + description now live in the shared registry (#294). The execute body
// security-confirmation framing ("only share when the user explicitly asked, // keeps this transport's own `searchIndexing ?? true` default.
// since this exposes the page to anyone with the link") tuned for the in-app registerShared(
// agent; this transport keeps the plain public-URL wording. SHARED_TOOL_SPECS.sharePage,
server.registerTool(
"share_page",
{
description:
"Make a page publicly accessible (idempotent) and return its public " +
"URL. The URL format is <app>/share/<key>/p/<slugId>. This exposes the " +
"page content to ANYONE with the URL — do it only when explicitly asked.",
inputSchema: {
pageId: z.string().min(1).describe("ID of the page to share"),
searchIndexing: z
.boolean()
.optional()
.describe("Allow search engines to index the page (default true)"),
},
},
async ({ pageId, searchIndexing }) => { async ({ pageId, searchIndexing }) => {
const result = await docmostClient.sharePage(pageId, searchIndexing ?? true); const result = await docmostClient.sharePage(pageId, searchIndexing ?? true);
return jsonContent(result); return jsonContent(result);
@@ -641,29 +495,11 @@ registerShared(SHARED_TOOL_SPECS.listShares, async () => {
}); });
// Tool: move_page // Tool: move_page
server.registerTool( // Schema + description now live in the shared registry (#294). The execute body
"move_page", // keeps this transport's cycle guard, its 'null'/'' -> null string coercion, and
{ // its positive-confirmation check on the move response.
description: registerShared(
"Move a page under a new parent (nesting) or to the space root.", SHARED_TOOL_SPECS.movePage,
inputSchema: {
pageId: z.string().min(1),
parentPageId: z
.string()
.nullable()
.optional()
.describe(
"Target parent page ID. Pass 'null' or empty string to move to root.",
),
position: z
.string()
.min(5)
.optional()
.describe(
"fractional-index position key; min 5 chars; omit to append at the end.",
),
},
},
async ({ pageId, parentPageId, position }) => { async ({ pageId, parentPageId, position }) => {
const finalParentId = const finalParentId =
parentPageId === "" || parentPageId === "null" ? null : parentPageId; parentPageId === "" || parentPageId === "null" ? null : parentPageId;
@@ -698,49 +534,22 @@ server.registerTool(
); );
// Tool: delete_page // Tool: delete_page
server.registerTool( // Schema + description now live in the shared registry (#294). The shared schema
"delete_page", // exposes ONLY pageId, so no permanent/force-delete flag can reach the client.
{ registerShared(SHARED_TOOL_SPECS.deletePage, async ({ pageId }) => {
description: await docmostClient.deletePage(pageId);
"Delete a single page by ID. SOFT delete only: the page is moved to " + return {
"trash and can be restored; nothing is permanently deleted.", content: [
inputSchema: { { type: "text" as const, text: `Successfully deleted page ${pageId}` },
pageId: z.string().min(1), ],
}, };
}, });
async ({ pageId }) => {
await docmostClient.deletePage(pageId);
return {
content: [
{ type: "text" as const, text: `Successfully deleted page ${pageId}` },
],
};
},
);
// --- Comment tools (ported from upstream PR #3 by Max Nikitin) --- // --- Comment tools (ported from upstream PR #3 by Max Nikitin) ---
// Tool: list_comments // Tool: list_comments
server.registerTool( registerShared(
"list_comments", SHARED_TOOL_SPECS.listComments,
{
description:
"List comments on a page in one call (pagination is handled " +
"internally). By DEFAULT only ACTIVE threads are returned; resolved " +
"threads (a resolved top-level comment and all its replies) are hidden " +
"and their count reported as `resolvedThreadsHidden` so you can re-query " +
"with `includeResolved: true` to see everything. Returns " +
"`{ items, resolvedThreadsHidden }`. Content is returned as Markdown.",
inputSchema: {
pageId: z.string().describe("ID of the page"),
includeResolved: z
.boolean()
.optional()
.describe(
"default only active threads; true — include resolved",
),
},
},
async ({ pageId, includeResolved }) => { async ({ pageId, includeResolved }) => {
const comments = await docmostClient.listComments(pageId, includeResolved); const comments = await docmostClient.listComments(pageId, includeResolved);
return jsonContent(comments); return jsonContent(comments);
@@ -748,55 +557,11 @@ server.registerTool(
); );
// Tool: create_comment // Tool: create_comment
// INTENTIONAL per-transport divergence (not shared): the in-app copy tunes the // Schema + description now live in the shared registry (#294). The execute body
// guidance for the in-app agent (e.g. "retry with a corrected EXACT selection" // keeps this transport's own guards (require a selection for a top-level
// and "Reversible via the comment UI"); this transport keeps its own wording. // comment; reject suggestedText on a reply / without a selection).
server.registerTool( registerShared(
"create_comment", SHARED_TOOL_SPECS.createComment,
{
description:
"Create a new comment on a page. The comment is ALWAYS inline and is " +
"anchored to (highlights) its `selection` text — there are no page-level " +
"comments. Content is provided as Markdown and automatically converted. " +
"A top-level comment REQUIRES an exact `selection`; if the selection " +
"cannot be found in the page the call fails (no orphan comment is left). " +
"Replies (parentCommentId set) inherit the parent's anchor and take no " +
"selection. You may also attach a `suggestedText` proposing a replacement " +
"for the `selection`; a human applies (or rejects) it from the UI. When " +
"`suggestedText` is set the `selection` MUST occur exactly once in the " +
"page — expand it with surrounding context if it is ambiguous.",
inputSchema: {
pageId: z.string().describe("ID of the page to comment on"),
content: z.string().min(1).describe("Comment content in Markdown format"),
selection: z
.string()
.min(1)
// Enforce the documented 250-char cap to match the description above.
.max(250)
.optional()
.describe(
"EXACT contiguous text from a single paragraph/block to anchor the " +
"comment on (<=250 chars). Required for a top-level comment; omit " +
"only when replying via parentCommentId.",
),
parentCommentId: z
.string()
.optional()
.describe("Parent comment ID to create a reply (max 2 nesting levels)"),
suggestedText: z
.string()
.min(1)
.max(2000)
.optional()
.describe(
"Optional proposed replacement (PLAIN TEXT) for the `selection`, " +
"applied by a human via the UI (never auto-applied). REQUIRES a " +
"`selection`; NOT allowed on a reply. When set, the `selection` must " +
"be UNIQUE in the page — expand it with surrounding context (still " +
"<=250 chars) if it occurs more than once, or the call is refused.",
),
},
},
async ({ pageId, content, selection, parentCommentId, suggestedText }) => { async ({ pageId, content, selection, parentCommentId, suggestedText }) => {
if (!parentCommentId && (!selection || !selection.trim())) { if (!parentCommentId && (!selection || !selection.trim())) {
throw new Error( throw new Error(
@@ -872,28 +637,9 @@ server.registerTool(
); );
// Tool: resolve_comment // Tool: resolve_comment
server.registerTool( // Schema + description now live in the shared registry (#294).
"resolve_comment", registerShared(
{ SHARED_TOOL_SPECS.resolveComment,
description:
"Resolve (close) or reopen a comment thread. Only top-level comments can " +
"be resolved — the server rejects resolving a reply. Reversible: pass " +
"resolved=false to reopen. Resolving keeps the thread and its replies " +
"(unlike delete_comment, which permanently removes them).",
inputSchema: {
commentId: z
.string()
.min(1)
.describe("ID of the top-level comment thread to resolve or reopen"),
resolved: z
.boolean()
.optional()
.default(true)
.describe(
"true (default) marks the thread resolved/closed; false reopens it",
),
},
},
async ({ commentId, resolved }) => { async ({ commentId, resolved }) => {
const result = await docmostClient.resolveComment(commentId, resolved); const result = await docmostClient.resolveComment(commentId, resolved);
return jsonContent(result); return jsonContent(result);
@@ -901,30 +647,10 @@ server.registerTool(
); );
// Tool: check_new_comments // Tool: check_new_comments
server.registerTool( // Schema + description now live in the shared registry (#294). The execute body
"check_new_comments", // keeps this transport's own guard rejecting an unparseable `since` timestamp.
{ registerShared(
description: SHARED_TOOL_SPECS.checkNewComments,
"Check for new comments across pages in a space since a given timestamp. " +
"Optionally scope to a page subtree (folder). Returns only comments " +
"created after the specified time.",
inputSchema: {
spaceId: z.string().describe("Space ID to check for new comments"),
since: z
.string()
.min(1)
.describe(
"ISO 8601 timestamp — only return comments created after this time (e.g. '2026-03-10T00:00:00Z')",
),
parentPageId: z
.string()
.optional()
.describe(
"Optional root page ID to scope the check to a subtree (folder). " +
"Only pages under this parent will be checked.",
),
},
},
async ({ spaceId, since, parentPageId }) => { async ({ spaceId, since, parentPageId }) => {
// Reject an unparseable timestamp up front: otherwise the comparison // Reject an unparseable timestamp up front: otherwise the comparison
// against NaN silently treats every comment as "not new" and the tool // against NaN silently treats every comment as "not new" and the tool
@@ -1053,6 +779,8 @@ server.registerTool(
); );
// Tool: insert_footnote // Tool: insert_footnote
// MCP-only by design (see insert_image): the in-app AI-chat agent exposes no
// footnote tool, so there is no second layer to unify — stays inline (#294).
server.registerTool( server.registerTool(
"insert_footnote", "insert_footnote",
{ {
+494
View File
@@ -316,6 +316,34 @@ export const SHARED_TOOL_SPECS = {
// --- share management --- // --- share management ---
// Unified from the per-layer inline definitions (#294). Both layers already
// carried the "only share when explicitly asked" security framing (the
// "per-transport divergence" note on the old inline copies was stale), so
// there was no real behavioral divergence to preserve — only wording drift.
sharePage: {
mcpName: 'share_page',
inAppKey: 'sharePage',
// CANONICAL: merges the MCP copy's URL-format + idempotency detail with the
// in-app copy's reversibility note; keeps the security framing both had.
description:
'Make a page PUBLICLY accessible (idempotent) and return its public URL ' +
'(format: <app>/share/<key>/p/<slugId>). This exposes the page content ' +
'to ANYONE with the URL — only share when the user explicitly asked. ' +
'Reversible: unshare it later to revoke the public URL.',
tier: 'deferred',
catalogLine: 'sharePage — make a page publicly accessible and return its URL.',
// Reconciled: MCP's stricter .min(1) on pageId kept; field descriptions from
// the in-app copy. The MCP execute keeps its own `searchIndexing ?? true`
// default (a per-layer concern, not part of the shared schema).
buildShape: (z) => ({
pageId: z.string().min(1).describe('The id of the page to share.'),
searchIndexing: z
.boolean()
.optional()
.describe('Allow public search engines to index it (default true).'),
}),
},
unsharePage: { unsharePage: {
mcpName: 'unshare_page', mcpName: 'unshare_page',
inAppKey: 'unsharePage', inAppKey: 'unsharePage',
@@ -509,4 +537,470 @@ export const SHARED_TOOL_SPECS = {
pageId: z.string().min(1), pageId: z.string().min(1),
}), }),
}, },
// --- page tools (unified from the per-layer inline definitions, #294) ---
//
// Descriptions merge both layers (the MCP copy's richer structural notes + the
// in-app copy's "Reversible via history/trash" framing where it added one).
// Field constraints keep the MCP copy's stricter .min(1) EXCEPT where the
// in-app layer deliberately allowed a looser value (documented per field).
getPage: {
mcpName: 'get_page',
inAppKey: 'getPage',
description:
'Fetch a single page as Markdown by its id. Returns the page title and ' +
'its Markdown content. The Markdown conversion is LOSSY (block ids, exact ' +
'table/callout structure are approximated); for a lossless representation ' +
'use the lossless page-JSON read tool. Inline <span data-comment-id> tags in the markdown ' +
'are comment highlight anchors (also present for RESOLVED threads) — ' +
'treat them as markup, not page text.',
tier: 'core',
catalogLine: 'getPage — fetch a page as Markdown by its id.',
// Reconciled: MCP's stricter .min(1) kept; in-app's more-informative
// "(or slugId)" describe kept.
buildShape: (z) => ({
pageId: z.string().min(1).describe('The id (or slugId) of the page.'),
}),
},
listPages: {
mcpName: 'list_pages',
inAppKey: 'listPages',
description:
'List the most recent pages (ordered by updatedAt, descending), ' +
'optionally scoped to a single space. Returns a bounded list (default ' +
'50, max 100) — use search for lookups in large spaces. Pass tree:true ' +
"(with spaceId) to instead get the space's full page hierarchy as a " +
'nested tree.',
tier: 'core',
catalogLine: "listPages — list recent pages, or a space's full page tree.",
buildShape: (z) => ({
spaceId: z
.string()
.optional()
.describe('Optional space id to scope the listing to.'),
limit: z
.number()
.int()
.min(1)
.max(100)
.optional()
.describe('Maximum number of pages (default 50, max 100).'),
tree: z
.boolean()
.optional()
.describe(
"When true, return the space's full page hierarchy as a nested tree " +
'(children arrays) instead of the recent-by-updatedAt flat list. ' +
'Requires spaceId; ignores limit.',
),
}),
},
createPage: {
mcpName: 'create_page',
inAppKey: 'createPage',
description:
'Create a new page with a Markdown body in a space, optionally under a ' +
'parent page (omit parentPageId to create at the space root). Returns ' +
'the new page id and title. Reversible: a page can be moved to trash ' +
'later.',
tier: 'deferred',
catalogLine: 'createPage — create a new page with a Markdown body in a space.',
// Reconciled schema DRIFT: the MCP copy pinned `content` to .min(1) while
// the in-app copy left it unbounded and DOCUMENTS an empty body as valid
// ("may be empty") — creating an empty page to fill in later is a real use
// case. The looser (no-min) form is kept, so create_page now also accepts an
// empty body (harmless — it creates an empty page) and no previously-valid
// in-app input is ever rejected. `title`/`spaceId` keep the MCP .min(1)
// (an empty title or space is never valid).
buildShape: (z) => ({
title: z.string().min(1).describe('The title of the new page.'),
content: z.string().describe('The page body as Markdown (may be empty).'),
spaceId: z.string().min(1).describe('The id of the space to create the page in.'),
parentPageId: z
.string()
.optional()
.describe('Optional parent page id to nest the new page under.'),
}),
},
movePage: {
mcpName: 'move_page',
inAppKey: 'movePage',
description:
'Move a page under a new parent page, or to the space root when no ' +
'parent is given. Reversible: move it back at any time.',
tier: 'deferred',
catalogLine: 'movePage — move a page under a new parent or to the space root.',
// Reconciled schema DRIFT: the MCP copy exposed a `position` field
// (fractional-index ordering) that the in-app copy lacked. Unified by
// KEEPING position (the in-app client already accepts an optional position
// arg, so the in-app execute now forwards it) — it is optional, so no
// previously-valid in-app call is rejected. `parentPageId` is `.nullable()`
// on both, so a real JSON null moves to root on either transport; the MCP
// execute additionally coerces the strings 'null'/'' to null as a robustness
// fallback (kept in its execute body, not in the shared schema).
buildShape: (z) => ({
pageId: z.string().min(1).describe('The id of the page to move.'),
parentPageId: z
.string()
.nullable()
.optional()
.describe(
'Target parent page id. Null or omitted moves the page to the space ' +
'root.',
),
position: z
.string()
.min(5)
.optional()
.describe(
'Optional fractional-index position key (min 5 chars); omit to ' +
'append at the end.',
),
}),
},
renamePage: {
mcpName: 'rename_page',
inAppKey: 'renamePage',
description:
'Rename a page (change its title only; the body is untouched, never ' +
'resent). Reversible: rename back at any time.',
tier: 'deferred',
catalogLine: "renamePage — change a page's title only (body untouched).",
buildShape: (z) => ({
pageId: z.string().min(1).describe('The id of the page to rename.'),
title: z.string().min(1).describe('The new title.'),
}),
},
deletePage: {
mcpName: 'delete_page',
inAppKey: 'deletePage',
description:
'Move a page to the trash — SOFT delete only: the page can be restored ' +
'from trash and nothing is ever permanently deleted.',
tier: 'deferred',
catalogLine: 'deletePage — move a page to trash (soft delete, reversible).',
// GUARDRAIL preserved (§14 H4): the schema exposes ONLY pageId, so a
// permanentlyDelete/forceDelete flag can never reach the client through this
// tool (asserted by ai-chat-tools.service.spec.ts).
buildShape: (z) => ({
pageId: z.string().min(1).describe('The id of the page to move to trash.'),
}),
},
updatePageJson: {
mcpName: 'update_page_json',
inAppKey: 'updatePageJson',
description:
"Replace a page's content with a raw ProseMirror JSON document (lossless " +
'write: preserves the block ids, callouts, tables and attributes you pass ' +
'in). Typical flow: read the page-JSON view -> modify the JSON -> write it back. ' +
'Keep existing node ids intact so heading anchors and history stay ' +
'stable. Minimal full-doc example: {"type":"doc","content":[{"type":' +
'"paragraph","content":[{"type":"text","text":"Hi"}]}]}. `content` may be ' +
'a JSON object or a JSON string (both accepted), and is OPTIONAL: omit it ' +
'to update only the title (though prefer the rename-page tool for a title-only ' +
'change). Supplying neither content nor title is an error. Reversible: ' +
'the previous version is kept in page history.',
tier: 'deferred',
catalogLine:
"updatePageJson — overwrite a page's body with a full ProseMirror document.",
buildShape: (z) => ({
pageId: z.string().min(1).describe('ID of the page to update'),
content: z
.any()
.optional()
.describe(
'ProseMirror document {"type":"doc","content":[...]} (JSON object or ' +
'JSON string). Omit to update only the title.',
),
title: z.string().optional().describe('Optional new title'),
}),
},
exportPageMarkdown: {
mcpName: 'export_page_markdown',
inAppKey: 'exportPageMarkdown',
// CANONICAL: the MCP copy (a strict superset of the terse in-app wording).
description:
'Export a page to a single self-contained, lossless Docmost-flavoured ' +
'Markdown file (custom extensions): YAML-free meta header, body with ' +
'inline comment anchors and diagrams, and a trailing comments-thread ' +
'block. Designed for a download -> edit body -> page-Markdown import ' +
'round-trip that preserves everything, including comment highlights. ' +
'Comment THREADS are preserved in the file but are not re-pushed to the ' +
'server on import.',
tier: 'deferred',
catalogLine:
'exportPageMarkdown — export a page to self-contained Markdown (body + comments).',
buildShape: (z) => ({
pageId: z.string().min(1).describe('The id of the page to export.'),
}),
},
// --- comment tools (unified from the per-layer inline definitions, #294) ---
//
// create_comment and resolve_comment previously carried a "per-transport
// divergence" note in BOTH layers; #294 unifies their schema + description
// here. Only the four tools that genuinely exist in BOTH layers live in the
// registry: create/list/resolve comment and check_new_comments.
//
// update_comment and delete_comment are intentionally NOT here: they exist
// ONLY on the standalone MCP server. The in-app agent deliberately exposes no
// hard comment edit/delete tool (comment edits are irreversible / not
// version-tracked; see the guardrail tests in ai-chat-tools.service.spec.ts),
// so there is nothing to unify — they stay inline in index.ts.
createComment: {
mcpName: 'create_comment',
inAppKey: 'createComment',
// CANONICAL: the in-app copy (the more-maintained one). It keeps the same
// rules as the MCP copy — inline-only, top-level requires a `selection`, no
// page-level comments, replies inherit the anchor, suggestedText must be
// unique — and adds the "retry with a corrected EXACT selection" and reply-
// to-reply-rejected guidance the MCP copy lacked. Execute-side validation
// (reject suggestedText on a reply, require a selection) stays per-layer.
description:
'Add an INLINE comment to a page, or reply to an existing top-level ' +
'comment (one level only — the backend rejects replies to replies). ' +
'The comment is anchored inline to the given exact `selection` text ' +
'(which gets highlighted); page-level comments are NOT supported. A ' +
'new top-level comment REQUIRES a `selection`. Replies inherit the ' +
"parent's anchor and take no selection. If the call fails with a " +
'"selection not found" error, retry with a corrected EXACT selection ' +
'copied verbatim from a single paragraph/block. You may also attach a ' +
'`suggestedText` proposing a replacement for the `selection` (a human ' +
'applies it from the UI); when set, the `selection` must occur exactly ' +
'once in the page. Reversible via the comment UI.',
tier: 'core',
catalogLine:
'createComment — add an inline comment (optionally with a suggested edit).',
// Reconciled schema: the field set is identical across both layers; the
// only constraint drift is `content`, which the MCP copy pinned to
// .min(1) while the in-app copy left unbounded — the stricter MCP form is
// kept (an empty comment body is never valid).
buildShape: (z) => ({
pageId: z.string().describe('The id of the page to comment on.'),
content: z.string().min(1).describe('The comment body as Markdown.'),
selection: z
.string()
.min(1)
.max(250)
.optional()
.describe(
'EXACT contiguous text from a SINGLE paragraph/block to anchor ' +
'(highlight) the comment on (<=250 chars, avoid spanning across ' +
'formatting boundaries). Required for a new top-level comment; ' +
'omit only when replying via parentCommentId.',
),
parentCommentId: z
.string()
.optional()
.describe(
'Optional id of a TOP-LEVEL comment to reply to (one level ' +
'of replies only).',
),
suggestedText: z
.string()
.min(1)
.max(2000)
.optional()
.describe(
'Optional proposed replacement (PLAIN TEXT) for the `selection`, ' +
'applied by a human via the UI (never auto-applied). REQUIRES a ' +
'`selection`; NOT allowed on a reply. When set, the `selection` ' +
'must be UNIQUE in the page — expand it with surrounding context ' +
'(still <=250 chars) if it occurs more than once, or the call is ' +
'refused.',
),
}),
},
listComments: {
mcpName: 'list_comments',
inAppKey: 'listComments',
// CANONICAL: the two copies are near-identical; the MCP copy is the
// superset (it keeps the "(pagination is handled internally)" note the
// in-app copy dropped), so it is used verbatim.
description:
'List comments on a page in one call (pagination is handled ' +
'internally). By DEFAULT only ACTIVE threads are returned; resolved ' +
'threads (a resolved top-level comment and all its replies) are hidden ' +
'and their count reported as `resolvedThreadsHidden` so you can re-query ' +
'with `includeResolved: true` to see everything. Returns ' +
'`{ items, resolvedThreadsHidden }`. Content is returned as Markdown.',
tier: 'core',
catalogLine:
'listComments — list all comments on a page (including resolved).',
buildShape: (z) => ({
pageId: z.string().describe('ID of the page'),
includeResolved: z
.boolean()
.optional()
.describe('default only active threads; true — include resolved'),
}),
},
resolveComment: {
mcpName: 'resolve_comment',
inAppKey: 'resolveComment',
// CANONICAL: the MCP copy's richer wording, minus its snake_case reference
// to `delete_comment` (a sibling tool that does NOT exist in the in-app
// layer) — rephrased transport-neutrally per the registry convention.
description:
'Resolve (close) or reopen a top-level comment thread (reversible — ' +
'pass resolved=false to reopen). Only top-level comments can be ' +
'resolved; the server rejects resolving a reply. Resolving keeps the ' +
'thread and its replies intact (it is not a deletion).',
tier: 'core',
catalogLine: 'resolveComment — resolve or reopen a comment thread.',
// Reconciled schema: `resolved` drifted — the MCP copy made it optional
// with .default(true) (resolve is the common case, documented), the in-app
// copy made it required. The MCP form is kept (a strict superset: it never
// rejects a previously-valid input and adds a sensible default), and
// commentId keeps the MCP copy's stricter .min(1).
buildShape: (z) => ({
commentId: z
.string()
.min(1)
.describe('ID of the top-level comment thread to resolve or reopen'),
resolved: z
.boolean()
.optional()
.default(true)
.describe(
'true (default) marks the thread resolved/closed; false reopens it',
),
}),
},
checkNewComments: {
mcpName: 'check_new_comments',
inAppKey: 'checkNewComments',
// CANONICAL: the MCP copy (the more detailed of the two). The MCP layer's
// execute-side guard that rejects an unparseable `since` timestamp stays in
// its execute body (per-layer logic), not in the shared schema.
description:
'Check for new comments across pages in a space since a given ' +
'timestamp. Optionally scope to a page subtree (folder). Returns only ' +
'comments created after the specified time.',
tier: 'deferred',
catalogLine:
'checkNewComments — find comments in a space created after a timestamp.',
// Reconciled schema: `since` keeps the MCP copy's stricter .min(1) (the
// in-app copy left it unbounded); field descriptions use the MCP copy's
// more detailed wording (it carries an example timestamp).
buildShape: (z) => ({
spaceId: z.string().describe('Space ID to check for new comments'),
since: z
.string()
.min(1)
.describe(
"ISO 8601 timestamp — only return comments created after this time " +
"(e.g. '2026-03-10T00:00:00Z')",
),
parentPageId: z
.string()
.optional()
.describe(
'Optional root page ID to scope the check to a subtree (folder). ' +
'Only pages under this parent will be checked.',
),
}),
},
// --- table tools (unified from the per-layer inline definitions, #294) ---
//
// These tools carried a "NOT shared" note in BOTH layers because of a single
// parameter-NAME drift: the MCP layer named the table reference `table` while
// the in-app layer named it `tableRef`. #294 reconciles that drift by unifying
// on the MCP name `table` — renaming the MCP public parameter would break
// external MCP clients, whereas the in-app parameter is model-facing
// (prompt-only) and safe to rename. The in-app execute bodies now destructure
// `table` instead of `tableRef` (nothing else changes). Descriptions take the
// MCP copy's richer wording (it documented `#<index>`, padding, header-row
// behavior) plus the in-app copy's "Reversible via page history" note; sibling
// tool references are phrased transport-neutrally.
//
// NOT here (kept inline in index.ts): table_get / getTable. Its MCP tool name
// is noun-first (`table_get`) while the in-app key is verb-first (`getTable`),
// so it breaks the snake_case(inAppKey) naming convention the registry enforces
// (shared-tool-specs.contract.spec.ts). Renaming the public MCP tool would
// break external clients, so it stays per-transport (its in-app param was still
// aligned to `table` for consistency with the migrated trio below).
tableInsertRow: {
mcpName: 'table_insert_row',
inAppKey: 'tableInsertRow',
description:
'Insert a row of plain-text cells into a table. `table` is `#<index>` ' +
'from the page outline, or a block id inside it. `cells` is the text per ' +
"column (padded to the table's column count; an error if more cells than " +
'columns). `index` is the 0-based insert position (0 inserts before the ' +
'header); omit to append at the end. Reversible via page history.',
tier: 'deferred',
catalogLine: 'tableInsertRow — insert a row of plain-text cells into a table.',
buildShape: (z) => ({
pageId: z.string().min(1).describe('The id of the page.'),
table: z
.string()
.min(1)
.describe('"#<index>" from the page outline, or a block id in the table.'),
cells: z.array(z.string()).describe('The cell texts for the row (one per column).'),
index: z
.number()
.int()
.optional()
.describe('0-based insert position (0 inserts before the header); omit to append.'),
}),
},
tableDeleteRow: {
mcpName: 'table_delete_row',
inAppKey: 'tableDeleteRow',
description:
'Delete the row at 0-based `index` from a table (`table` is `#<index>` ' +
'from the page outline, or a block id inside it). Refuses to delete the ' +
"table's only row; an out-of-range `index` throws. Deleting `index` 0 " +
'removes the header row, and the next row becomes the new header. ' +
'Reversible via page history.',
tier: 'deferred',
catalogLine: 'tableDeleteRow — delete a table row at a 0-based index.',
buildShape: (z) => ({
pageId: z.string().min(1).describe('The id of the page.'),
table: z
.string()
.min(1)
.describe('"#<index>" from the page outline, or a block id in the table.'),
index: z.number().int().describe('0-based row index to delete.'),
}),
},
tableUpdateCell: {
mcpName: 'table_update_cell',
inAppKey: 'tableUpdateCell',
description:
'Set the plain-text content of cell [row, col] (0-based) in a table ' +
'(`table` is `#<index>` from the page outline, or a block id inside it). ' +
"Replaces the cell's content with a single text paragraph; for rich " +
"formatting, patch the cell's paragraph id (obtained from reading the " +
'table) instead. Reversible via page history.',
tier: 'deferred',
catalogLine: 'tableUpdateCell — set the text of a table cell at [row, col].',
buildShape: (z) => ({
pageId: z.string().min(1).describe('The id of the page.'),
table: z
.string()
.min(1)
.describe('"#<index>" from the page outline, or a block id in the table.'),
row: z.number().int().describe('0-based row index.'),
col: z.number().int().describe('0-based column index.'),
text: z.string().describe('The new cell text.'),
}),
},
} satisfies Record<string, SharedToolSpec>; } satisfies Record<string, SharedToolSpec>;
@@ -0,0 +1,16 @@
{
"_bug": "BUG #351: a `column` whose `width` is a percentage string (e.g. \"50%\") is NOT byte-stable across export->import->export (violates P2). The `column` schema's parseHTML does `parseFloat(getAttribute('data-width'))`, which silently drops the '%' unit and returns the NUMBER 50. So the first export emits data-width=\"50%\" but the re-import stores width=50, and the second export emits data-width=\"50\": md2 !== md1, a permanent GS-EDIT-REVERT churn (every git-sync pull rewrites the column width). The editor authors column widths as percentages, so this is a real data/round-trip defect. Fix belongs in src/lib/docmost-schema.ts column.width parseHTML (preserve the unit / keep the string), which is OUT OF SCOPE for this test-only PR and must be a separate, maintainer-approved change. This flat generator therefore keeps `column.width` frozen (never generates a non-default width).",
"doc": {
"type": "doc",
"content": [
{
"type": "columns",
"attrs": { "layout": "two_equal", "widthMode": "normal" },
"content": [
{ "type": "column", "attrs": { "width": "50%" }, "content": [{ "type": "paragraph", "content": [{ "type": "text", "text": "L" }] }] },
{ "type": "column", "attrs": { "width": "50%" }, "content": [{ "type": "paragraph", "content": [{ "type": "text", "text": "R" }] }] }
]
}
]
}
}
@@ -0,0 +1,19 @@
{
"doc": {
"type": "doc",
"content": [
{
"type": "orderedList",
"attrs": { "type": null, "start": 5 },
"content": [
{
"type": "listItem",
"content": [
{ "type": "paragraph", "content": [{ "type": "text", "text": "alpha" }] }
]
}
]
}
]
}
}
@@ -0,0 +1,325 @@
/**
* Schema-DERIVED attribute-state fast-check arbitraries (#351, PR 1).
*
* This GENERALIZES the #350 stability-matrix helper (roundtrip-stability.helper.ts)
* to fast-check. Where that helper sweeps a HAND-WRITTEN 2-state matrix for one
* node spec, this module reads the attribute list straight from
* `schema.nodes[type].spec.attrs` (never a hand list) and, per attribute,
* generates over the FOUR states the issue calls for:
*
* - `absent` : the attribute is OMITTED entirely (the empty-string-vs-
* absent churn class the #350 fix targets).
* - `default` : the schema default value, authored explicitly.
* - `nonDefault` : a representative legal non-default value.
* - `degenerate` : `""` for strings, `0`/negative for numbers, the flipped
* value for booleans.
*
* Why a per-attribute override table
* Everything that CAN be derived generically from the default's runtime type is
* (booleans flip; the degenerate value follows the runtime type). But two facts
* force a small, DOCUMENTED override table:
*
* 1. CONSTRAINED domains the schema does not encode. `image.align ∈
* {left,center,right}`, `heading.level 1..6`, `callout.type
* {info,success,warning,danger}`, `columns.layout`, table-cell `align`,
* `status.color`, `orderedList.start ≥ 1`, etc. A generic "default + 1"
* would emit an ILLEGAL value, so these get an explicit legal domain.
* 2. ROUND-TRIP-safety, established EMPIRICALLY by probing the live converter
* (the classification captured in flat-roundtrip.property.test.ts). A frozen
* attribute falls into ONE of TWO explicitly-distinguished classes never a
* silent "it just doesn't round-trip":
*
* (a) ACCEPTED LIMITATION the attribute has NO markdown representation,
* so the loss is inherent to targeting markdown, not a converter
* defect. These: `paragraph`/`heading` `indent`, `callout.icon`,
* `orderedList.type` (a/A/i markers), table `colwidth` /
* `backgroundColor(Name)` (dropped by the raw-<table> fallback). Each is
* tagged `// ACCEPTED:` inline. Freezing them is correct there is
* nothing to preserve in the target format.
*
* (b) PINNED BUG the attribute IS representable in markdown but the
* converter drops it anyway (a real defect). These are NOT silently
* frozen: each is captured as a LOUD `it.fails` counterexample in
* test/fixtures/counterexamples/ + counterexamples.test.ts, and the
* freeze here only keeps the P1/P2 union green until a MAINTAINER rules
* on accept-vs-fix (the epic guardrail reserves that call). These:
* `column.width` (parseFloat drops `%`), `orderedList.start` (non-1
* start renders as `1.`). Tagged `// PINNED-BUG:` inline.
*
* (c) DEFERRED-BUG representable AND round-trips, frozen only because the
* flat generator can't yet build a valid instance. Table
* `colspan`/`rowspan` round-trip via the raw-<table> fallback, but a
* geometrically-valid spanned table is PR-2 structural work; the flat
* generator hardcodes span = 1. Tagged `// DEFERRED-BUG:` inline so a
* maintainer does not read them as an inherent limitation.
* - Several non-null-default attrs are MATERIALIZED on import but are not
* in canonicalize's KNOWN_DEFAULTS (`callout.type`, `status.color`,
* table `colspan`/`rowspan`, `columns.layout`/`widthMode`,
* `embed.width`/`height`, `heading.level`, `taskItem.checked`,
* `details.open`, `subpages.recursive`, `orderedList.start`). If left
* `absent` they re-materialize as a non-canonical default and diverge
* under P1. We mark them `always` so they are authored explicitly.
* - The documented numericstring coercion set (`width height size
* aspectRatio`) is generated as STRINGS for the media family (a stored
* number re-parses as a string), EXCEPT `embed.width/height` which the
* embed schema keeps numeric handled per-attr.
*
* Both PINNED-BUG attrs (`column.width` P2 churn, `orderedList.start` P1 loss)
* are captured as committed `it.fails` counterexamples NOT hidden here.
*/
import fc from 'fast-check';
import { getSchema } from '@tiptap/core';
import { docmostExtensions } from '../../src/lib/index.js';
import { phraseArb, letterPhraseArb, urlArb } from './text-arbitraries.js';
/** The exact ProseMirror schema the converter targets. */
export const schema = getSchema(docmostExtensions as any);
/** Sentinel: this attribute is OMITTED (the `absent` state). */
export const ABSENT = Symbol('ABSENT');
/** The documented numeric→string coercion set (issue + roundtrip-stability.helper). */
export const NUMERIC_STRING_ATTRS = ['width', 'height', 'size', 'aspectRatio'];
/** Read the schema default for every attribute of a node type. */
export function schemaAttrDefaults(type: string): Record<string, unknown> {
const specAttrs = (schema.nodes[type]?.spec?.attrs ?? {}) as Record<
string,
{ default: unknown }
>;
const out: Record<string, unknown> = {};
for (const [k, v] of Object.entries(specAttrs)) out[k] = v.default;
return out;
}
/** Attribute names for a node type, straight from the schema (never hand-listed). */
export function schemaAttrNames(type: string): string[] {
return Object.keys((schema.nodes[type]?.spec?.attrs ?? {}) as object);
}
/**
* Per-attribute policy. Everything unlisted falls back to a generic policy:
* - a BOOLEAN default is fuzzable (its non-default is the flipped value);
* - any other default is `frozen` (only `absent`/`default` are generated) so
* we never invent an unverified non-default that might not round-trip.
* Listed attrs override this with a legal `arb` domain and/or flags.
*/
interface AttrPolicy {
/** Arbitrary for the `nonDefault` state's value. */
arb?: fc.Arbitrary<unknown>;
/** Value for the `degenerate` state (fuzz mode only). Omit to skip degenerate. */
degen?: unknown;
/** Never emit `absent` — the attr must be authored (materialized default class). */
always?: boolean;
/** Never emit the schema default value (required-ish attrs like `src`). Implies always. */
noDefault?: boolean;
/** Never emit non-default/degenerate — attr has no md representation or churns. */
frozen?: boolean;
}
const num = (...xs: number[]) => fc.constantFrom(...xs);
const str = (...xs: string[]) => fc.constantFrom(...xs);
const widthStr = str('120', '320', '640');
// The documented override table, keyed `type.attr`. Every entry is grounded in
// the empirical converter probe (see flat-roundtrip.property.test.ts header).
const OVERRIDES: Record<string, AttrPolicy> = {
// ── block text containers ────────────────────────────────────────────────
// 'left' is the IMPLICIT default alignment: the converter drops it on export
// (empirically confirmed), so it never round-trips. Only center/right/justify
// carry through the `<!--attrs {textAlign}-->` comment.
'paragraph.textAlign': { arb: str('center', 'right', 'justify') },
'paragraph.indent': { frozen: true }, // ACCEPTED: no md representation
'heading.level': { always: true, arb: num(2, 3, 4, 5, 6) },
'heading.textAlign': { arb: str('center', 'right', 'justify') },
'heading.indent': { frozen: true }, // ACCEPTED: no md representation
// ── lists ────────────────────────────────────────────────────────────────
// PINNED-BUG: markdown CAN express a non-1 start ("5."), but the converter
// renders "1." and drops it -> P1 loss. See counterexamples.test.ts
// (ordered-list-start.json). Frozen only until the maintainer rules accept-vs-fix.
'orderedList.start': { always: true, frozen: true },
'orderedList.type': { frozen: true }, // ACCEPTED: a/A/i markers not expressible in GFM
'taskItem.checked': { always: true, arb: fc.constant(true) }, // boolean, default false
// ── codeBlock ────────────────────────────────────────────────────────────
'codeBlock.language': { arb: str('js', 'ts', 'python', 'go', 'rust', 'bash') },
// ── image / media (numeric→string width family) ──────────────────────────
'image.src': { noDefault: true, arb: urlArb, degen: '' },
'image.align': { arb: str('left', 'right') },
'image.alt': { arb: letterPhraseArb, degen: '' },
'image.title': { arb: letterPhraseArb },
'image.width': { arb: widthStr, degen: '' },
'image.height': { arb: widthStr, degen: '' },
'video.src': { noDefault: true, arb: urlArb, degen: '' },
'video.alt': { arb: letterPhraseArb },
'video.width': { arb: widthStr },
'video.height': { arb: widthStr },
'audio.src': { noDefault: true, arb: urlArb, degen: '' },
'youtube.src': { noDefault: true, arb: urlArb },
'pdf.src': { noDefault: true, arb: urlArb },
'pdf.name': { arb: phraseArb },
'drawio.src': { noDefault: true, arb: urlArb },
'excalidraw.src': { noDefault: true, arb: urlArb },
'attachment.url': { noDefault: true, arb: urlArb },
'attachment.name': { arb: phraseArb },
// ── callout / status ─────────────────────────────────────────────────────
'callout.type': { always: true, arb: str('success', 'warning', 'danger') },
'callout.icon': { frozen: true }, // ACCEPTED: no md representation (dropped on export)
'status.text': { noDefault: true, arb: phraseArb, degen: '' },
'status.color': { always: true, arb: str('green', 'orange', 'red', 'blue', 'yellow', 'purple') },
// ── table cells ────────────────────────────────────────────────────────────
// DEFERRED-BUG (not ACCEPTED): colspan/rowspan ARE representable and round-trip
// — a spanned cell makes the converter emit the whole table as a raw <table>
// with colspan/rowspan attrs (markdown-converter.ts tableToHtml), which the
// tiptap parser reads back. They are frozen only because generating a
// geometrically-valid spanned table is deferred STRUCTURAL work (the flat
// generator hardcodes colspan/rowspan = 1), NOT a markdown limitation.
'tableCell.colspan': { always: true, frozen: true },
'tableCell.rowspan': { always: true, frozen: true },
// ACCEPTED: colwidth / backgroundColor(Name) have no representation — the
// raw-<table> fallback (tableToHtml) drops them, so there is nothing to preserve.
'tableCell.colwidth': { frozen: true },
'tableCell.backgroundColor': { frozen: true },
'tableCell.backgroundColorName': { frozen: true },
'tableCell.align': { arb: str('left', 'center', 'right') },
'tableHeader.colspan': { always: true, frozen: true }, // DEFERRED-BUG (see tableCell.colspan)
'tableHeader.rowspan': { always: true, frozen: true }, // DEFERRED-BUG (see tableCell.rowspan)
'tableHeader.colwidth': { frozen: true }, // ACCEPTED: no representation
'tableHeader.backgroundColor': { frozen: true }, // ACCEPTED: no representation
'tableHeader.backgroundColorName': { frozen: true }, // ACCEPTED: no representation
'tableHeader.align': { arb: str('left', 'center', 'right') },
// ── details ──────────────────────────────────────────────────────────────
'details.open': { always: true, arb: fc.constant(true) }, // boolean, default false
// ── columns ──────────────────────────────────────────────────────────────
'columns.layout': { always: true, arb: str('three_equal', 'left_sidebar', 'right_sidebar') },
// widthMode round-trips via the `data-width-mode` attribute (verified P1+P2),
// so it is fuzzed, not frozen.
'columns.widthMode': { always: true, arb: str('custom') },
// PINNED-BUG: parseFloat import drops the `%` unit -> P2 churn. See
// counterexamples.test.ts (columns-column-width-percent.json).
'column.width': { frozen: true },
// ── embed (schema keeps width/height NUMERIC, not string-coerced) ─────────
'embed.src': { noDefault: true, arb: urlArb, degen: '' },
'embed.provider': { noDefault: true, arb: str('iframe', 'youtube', 'vimeo') },
'embed.width': { always: true, frozen: true },
'embed.height': { always: true, frozen: true },
// ── subpages / math / htmlEmbed ──────────────────────────────────────────
'subpages.recursive': { always: true, arb: fc.constant(true) }, // boolean, default false
'mathBlock.text': { noDefault: true, arb: str('x^2', 'a < b', '\\frac{1}{2}'), degen: '' },
'mathInline.text': { noDefault: true, arb: str('x^2', 'a < b', '\\frac{1}{2}'), degen: '' },
'htmlEmbed.source': { noDefault: true, arb: str('<b>hi</b>', '<i>x</i>', '<span>y</span>'), degen: '' },
'htmlEmbed.height': { arb: num(200, 300, 400) },
// ── footnotes / transclusion / pageEmbed / mention ───────────────────────
'footnoteDefinition.id': { noDefault: true, arb: str('fn1', 'fn2', 'note') },
'footnoteReference.id': { noDefault: true, arb: str('fn1', 'fn2', 'note') },
'pageEmbed.sourcePageId': { noDefault: true, arb: fc.uuid() },
'transclusionSource.id': { noDefault: true, arb: str('src1', 'src2') },
'transclusionReference.sourcePageId': { noDefault: true, arb: fc.uuid() },
'transclusionReference.transclusionId': { noDefault: true, arb: str('tr1', 'tr2') },
'mention.id': { noDefault: true, arb: fc.uuid() },
'mention.label': { noDefault: true, arb: phraseArb },
'mention.entityType': { noDefault: true, arb: str('user') },
'mention.entityId': { noDefault: true, arb: fc.uuid() },
};
/** Resolve the effective policy for one attribute (override merged over generic). */
function policyFor(type: string, attr: string, def: unknown): AttrPolicy {
const override = OVERRIDES[`${type}.${attr}`];
if (override) return override;
// Generic: booleans are fuzzable via their flipped value; everything else is
// frozen (only absent/default) so no unverified non-default is invented.
if (typeof def === 'boolean') return { arb: fc.constant(!def) };
return { frozen: true };
}
/**
* Whether an attribute is actually exercised at a NON-DEFAULT value (i.e. its
* policy has an `arb`, which the generic fallback does not). Used by the
* attribute-coverage snapshot test to make the generic-frozen space VISIBLE: any
* string/number attr not in OVERRIDES is silently only tested at absent/default,
* so the snapshot pins exactly which attrs are NOT value-fuzzed and forces a
* reviewer to look when a new attr lands in that invisible bucket.
*/
export function attrIsValueFuzzed(type: string, attr: string): boolean {
const def = schemaAttrDefaults(type)[attr];
return !!policyFor(type, attr, def).arb;
}
/** Every node `type.attr` in the schema (excluding the auto `id`), sorted. */
export function allSchemaAttrKeys(): string[] {
const keys: string[] = [];
for (const type of Object.keys(schema.nodes)) {
for (const attr of schemaAttrNames(type)) {
if (attr === 'id') continue;
keys.push(`${type}.${attr}`);
}
}
return keys.sort();
}
/**
* Every MARK attribute in the schema, keyed `mark:<name>.<attr>`, sorted. Marks
* are not driven by the node OVERRIDES table (they are fuzzed by the text
* generator, text-arbitraries.ts), so their value-fuzz coverage is tracked with a
* separate snapshot (see flat-roundtrip.property.test.ts) without this the
* "no invisible coverage hole" guarantee would hold for node attrs only, letting a
* new mark attr slip through unfuzzed and unallowlisted.
*/
export function allSchemaMarkAttrKeys(): string[] {
const keys: string[] = [];
for (const [name, mark] of Object.entries(schema.marks)) {
const attrs = (mark.spec?.attrs ?? {}) as Record<string, unknown>;
for (const attr of Object.keys(attrs)) keys.push(`mark:${name}.${attr}`);
}
return keys.sort();
}
export type AttrMode = 'p1' | 'fuzz';
/**
* Build an arbitrary for ONE attribute's value (or the ABSENT sentinel) across
* the states legal for `mode`:
* - p1 : absent / default / nonDefault (the round-trip-safe space).
* - fuzz : the above PLUS degenerate (P2 tolerates the one-time
* normalization; P3 only needs totality).
*/
export function attrValueArb(
type: string,
attr: string,
mode: AttrMode,
): fc.Arbitrary<unknown | typeof ABSENT> {
const def = schemaAttrDefaults(type)[attr];
const p = policyFor(type, attr, def);
const states: fc.Arbitrary<unknown | typeof ABSENT>[] = [];
if (!p.always && !p.noDefault) states.push(fc.constant(ABSENT));
if (!p.noDefault) states.push(fc.constant(def));
if (!p.frozen && p.arb) states.push(p.arb);
if (mode === 'fuzz' && !p.frozen && p.degen !== undefined) {
states.push(fc.constant(p.degen));
}
if (states.length === 0) states.push(fc.constant(def));
return fc.oneof(...states);
}
/**
* Build an arbitrary for a node's full `attrs` object over all schema attrs.
* `base` pins caller-required attrs (e.g. a concrete `src`) verbatim; any attr
* present in `base` is NOT re-generated. Omitted (ABSENT) attrs are dropped.
*/
export function nodeAttrsArb(
type: string,
mode: AttrMode,
base: Record<string, unknown> = {},
): fc.Arbitrary<Record<string, unknown>> {
const names = schemaAttrNames(type).filter((n) => !(n in base) && n !== 'id');
if (names.length === 0) return fc.constant({ ...base });
return fc
.tuple(...names.map((n) => attrValueArb(type, n, mode)))
.map((vals) => {
const attrs: Record<string, unknown> = { ...base };
names.forEach((n, i) => {
if (vals[i] !== ABSENT) attrs[n] = vals[i];
});
return attrs;
});
}
@@ -0,0 +1,74 @@
import { describe, expect, it } from 'vitest';
import { readFileSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import path from 'node:path';
import { convertProseMirrorToMarkdown } from '../../src/lib/markdown-converter.js';
import { markdownToProseMirror } from '../../src/lib/markdown-to-prosemirror.js';
import { docsCanonicallyEqual } from '../../src/lib/canonicalize.js';
// ---------------------------------------------------------------------------
// #351 committed counterexamples — REAL round-trip bugs surfaced by the flat
// generative probing (attribute level). Each is pinned here as an `it.fails`
// (vitest passes ONLY WHILE the assertion still fails), so that the day the
// underlying src/ bug is fixed, the `it.fails` starts PASSING and vitest turns
// this test RED — forcing us to delete the counterexample and (per the epic
// guardrail) tighten the generator. A bare `it.fails` would ship silent
// corruption, so every case below carries a loud `// BUG #351:` explanation.
//
// These bugs are NOT worked around by weakening any property: the offending
// attribute is kept OUT of the P1/P2 generators (documented in
// attr-arbitraries.ts), and the exact failing document lives here as the
// regression pin. FIXING the bug is a separate, maintainer-approved src/ change.
// ---------------------------------------------------------------------------
const here = path.dirname(fileURLToPath(import.meta.url));
const fixtureDir = path.resolve(here, '../fixtures/counterexamples');
function loadDoc(file: string): any {
return JSON.parse(readFileSync(path.join(fixtureDir, file), 'utf8')).doc;
}
describe('#351 counterexamples (known round-trip bugs, pinned as it.fails)', () => {
// BUG #351: a `column` with a PERCENTAGE width ("50%") is not byte-stable.
// The column schema parses `data-width` with parseFloat, dropping the '%':
// md1 = '...data-width="50%"...' (first export)
// re-import stores width = 50 (number)
// md2 = '...data-width="50"...' (second export) => md2 !== md1
// A permanent GS-EDIT-REVERT churn on every git-sync pull. The editor stores
// column widths as percentages, so this is a genuine defect. The fix is in
// src/lib/docmost-schema.ts (column.width parseHTML must preserve the unit)
// and is out of scope for this test-only PR.
it.fails('column percentage width is byte-stable (P2)', async () => {
const doc = loadDoc('columns-column-width-percent.json');
const md1 = convertProseMirrorToMarkdown(doc);
const doc2 = await markdownToProseMirror(md1);
const md2 = convertProseMirrorToMarkdown(doc2);
// This assertion currently FAILS (md2 drops the '%'), which is exactly what
// `it.fails` expects. When the schema is fixed, it will PASS and flip this
// test red — our cue to remove the pin.
expect(md2).toBe(md1);
});
// BUG #351: an `orderedList` with a non-1 `start` loses its start number.
// CommonMark CAN express this ("5." starts the list at 5), but the converter
// always emits "1." and ignores `attrs.start` (markdown-converter.ts renders
// `${index + 1}.`; the <ol> HTML path also omits `start`):
// doc.start = 5 -> md1 = "1. alpha" (start dropped on export)
// re-import stores start = 1 => docsCanonicallyEqual(rt, doc) === false
// This is a P1 (semantic round-trip) loss of the SAME class as column.width:
// representable in markdown, silently dropped by the converter. It is pinned
// here as the LOUD counterexample rather than being masked as an "accepted
// normalization" in the generator — per the epic guardrail, deciding
// accept-vs-fix for a markdown-representable loss is a MAINTAINER call, so this
// stays a visible known-bug until the maintainer rules on it. The fix would be
// in src/lib/markdown-converter.ts (emit the start number on the first item)
// and is out of scope for this test-only PR.
it.fails('ordered list start number is preserved (P1)', async () => {
const doc = loadDoc('ordered-list-start.json');
const md1 = convertProseMirrorToMarkdown(doc);
const doc2 = await markdownToProseMirror(md1);
// Currently FAILS: doc2.start === 1 while doc.start === 5. When the converter
// preserves `start`, this PASSES and flips the test red — remove the pin then.
expect(docsCanonicallyEqual(doc2, doc)).toBe(true);
});
});
@@ -0,0 +1,284 @@
import { describe, expect, it, vi } from 'vitest';
import fc from 'fast-check';
// Real converter, imported the same way the sibling property test does.
import { convertProseMirrorToMarkdown } from '../../src/lib/markdown-converter.js';
// Importing markdownToProseMirror mutates the global DOM via jsdom at module
// load (expected, required for @tiptap/html's generateJSON under Node).
import { markdownToProseMirror } from '../../src/lib/markdown-to-prosemirror.js';
import { docsCanonicallyEqual, canonicalizeContent } from '../../src/lib/index.js';
import { firstDivergence } from '../roundtrip-helpers.js';
import {
schema,
allSchemaAttrKeys,
allSchemaMarkAttrKeys,
attrIsValueFuzzed,
} from './attr-arbitraries.js';
import {
buildGenerators,
coveredTypes,
KNOWN_UNCOVERED,
} from './node-generators.js';
// ── Attribute-value coverage allowlist ──────────────────────────────────────
// The node/mark completeness contract guarantees every TYPE is generated, but
// NOT that every attribute is exercised at a NON-DEFAULT value. An attribute
// with no `arb` in attr-arbitraries.ts is only ever tested at absent/default —
// an INVISIBLE coverage hole (the reviewer's concern). This allowlist makes that
// hole EXPLICIT: it is the exact set of attrs deliberately not value-fuzzed, so
// a NEW attribute (or a newly-frozen one) that lands in this bucket flips the
// snapshot test red and forces a reviewer to classify it. Each belongs to one of:
// - internal/opaque ids & placeholders (attachmentId, slugId, placeholder,
// creatorId, anchorId) — no meaningful non-default to assert;
// - dimensions/among the media family with no standalone md form here
// (aspectRatio, size, caption, drawio/excalidraw/pdf/video/youtube w/h/align)
// — round-trip candidates deferred to a later PR, not silently dropped;
// - ACCEPTED limitations with no md representation (indent, callout.icon,
// orderedList.type, table spans/bg/colwidth);
// - PINNED bugs (column.width, orderedList.start) tracked in
// counterexamples.test.ts.
const ATTR_VALUE_FUZZ_ALLOWLIST = new Set<string>([
'attachment.attachmentId', 'attachment.mime', 'attachment.placeholder', 'attachment.size',
'audio.attachmentId', 'audio.placeholder', 'audio.size',
'callout.icon', 'column.width',
'drawio.align', 'drawio.alt', 'drawio.aspectRatio', 'drawio.attachmentId',
'drawio.height', 'drawio.size', 'drawio.title', 'drawio.width',
'embed.align', 'embed.height', 'embed.width',
'excalidraw.align', 'excalidraw.alt', 'excalidraw.aspectRatio', 'excalidraw.attachmentId',
'excalidraw.height', 'excalidraw.size', 'excalidraw.title', 'excalidraw.width',
'heading.indent',
'image.aspectRatio', 'image.attachmentId', 'image.caption', 'image.placeholder', 'image.size',
'mention.anchorId', 'mention.creatorId', 'mention.slugId',
'orderedList.start', 'orderedList.type', 'paragraph.indent',
'pdf.attachmentId', 'pdf.height', 'pdf.placeholder', 'pdf.size', 'pdf.width',
'tableCell.backgroundColor', 'tableCell.backgroundColorName', 'tableCell.colspan',
'tableCell.colwidth', 'tableCell.rowspan',
'tableHeader.backgroundColor', 'tableHeader.backgroundColorName', 'tableHeader.colspan',
'tableHeader.colwidth', 'tableHeader.rowspan',
'video.align', 'video.aspectRatio', 'video.attachmentId', 'video.placeholder', 'video.size',
'youtube.align', 'youtube.height', 'youtube.width',
]);
// ── MARK attribute-value coverage ───────────────────────────────────────────
// Marks are fuzzed by the text generator (text-arbitraries.ts markedTextRunArb),
// not the node OVERRIDES table, so their value-fuzz coverage is tracked with this
// separate registry — otherwise the "no invisible coverage hole" guarantee would
// hold for node attrs only, and a new mark attr (or a new attributed mark) would
// silently escape the fuzz set. Every schema mark attr must be in exactly one of:
// MARK_ATTR_FUZZED — actually driven at a non-default value by the generator;
// MARK_ATTR_ALLOWLIST — deliberately not value-fuzzed, with a reason.
const MARK_ATTR_FUZZED = new Set<string>([
'mark:link.href', // markedTextRunArb sets a random webUrl href
'mark:link.title', // ...and an optional letter-bearing title
'mark:highlight.color', // highlight mark carries a generated color
'mark:textStyle.color', // textStyle mark carries a generated color
'mark:comment.commentId', // comment anchor id (alphanumeric token)
'mark:comment.resolved', // comment resolved flag (rides only when true)
]);
const MARK_ATTR_ALLOWLIST = new Set<string>([
// link presentational/routing attrs: not part of the markdown link surface the
// converter emits (it round-trips href + title only), so there is no
// non-default value to assert here — a deferred concern for a link-specific
// fixture, not the flat generative pass.
'mark:link.internal',
'mark:link.target',
'mark:link.rel',
'mark:link.class',
]);
// Each run does a real convert + marked + jsdom parse (~ms). Give ample headroom
// so the suite is deterministic regardless of parallel worker load (like the
// sibling property file).
vi.setConfig({ testTimeout: 30000 });
// ---------------------------------------------------------------------------
// #351 PR 1 — GENERATIVE (property-based) round-trip over FLAT (single-node)
// documents at the ATTRIBUTE level.
//
// We assert three invariants for ANY generated valid flat document `d`
// (pmToMd = convertProseMirrorToMarkdown, mdToPm = markdownToProseMirror):
//
// P1 — semantic round-trip (nothing lost):
// docsCanonicallyEqual(await mdToPm(pmToMd(d)), d) === true
// P2 — byte fixpoint (anti "GS-EDIT-REVERT" churn):
// pmToMd(await mdToPm(pmToMd(d))) === pmToMd(d)
// P3 — totality: neither converter throws; bounded.
//
// The generators are schema-DERIVED (attribute lists come from
// schema.nodes[type].spec.attrs) and stay inside the round-trip-supported space
// proven empirically by probing the live converter (see attr-arbitraries.ts and
// text-arbitraries.ts). P1 runs over the safe attribute space; P2/P3 run over
// the wider 'fuzz' space that also injects degenerate attribute states, which
// P2 tolerates via a one-time first-pass normalization and P3 via totality only.
// ---------------------------------------------------------------------------
// Fixed seed so every failure is reproducible; fast-check also prints the
// shrunk counterexample. numRuns starts modest to keep CI under budget — the
// issue's CI target is ~300-500 per property; the nightly / PR 3 will crank
// this up further. Each property runs over the UNION (fc.oneof) of all flat
// node generators, so the runs are shared across node types (one test per
// property keeps the jsdom import cost and memory bounded — a per-generator ×
// per-property matrix is ~200 heavy tests that OOMs the worker).
const SEED = 20250705;
const NUM_RUNS = 300;
const P1_GENERATORS = buildGenerators('p1');
const FUZZ_GENERATORS = buildGenerators('fuzz');
// Union arbitraries: a single draw picks one node generator, then a document
// from it. On failure fast-check prints the shrunk counterexample doc, which
// names the offending node type directly.
const p1Union = fc.oneof(...P1_GENERATORS.map((g) => g.arb));
const fuzzUnion = fc.oneof(...FUZZ_GENERATORS.map((g) => g.arb));
async function roundTrip(doc: unknown): Promise<{ md1: string; md2: string; doc2: any }> {
const md1 = convertProseMirrorToMarkdown(doc);
const doc2 = await markdownToProseMirror(md1);
const md2 = convertProseMirrorToMarkdown(doc2);
return { md1, md2, doc2 };
}
describe('#351 flat generative round-trip — completeness contract', () => {
it('every schema node and mark is covered by a generator or explicitly allowlisted', () => {
const covered = coveredTypes();
const uncovered: string[] = [];
for (const nodeType of Object.keys(schema.nodes)) {
if (covered.has(nodeType)) continue;
if (nodeType in KNOWN_UNCOVERED) continue;
uncovered.push(`node:${nodeType}`);
}
for (const markType of Object.keys(schema.marks)) {
if (covered.has(`mark:${markType}`)) continue;
if (markType in KNOWN_UNCOVERED) continue;
uncovered.push(`mark:${markType}`);
}
// A new node/mark added to the schema with no generator AND no allowlist
// entry MUST turn this test red — that is the whole point (no silent blind
// spots).
expect(
uncovered,
`these schema types have no generator and no KNOWN_UNCOVERED reason:\n ${uncovered.join(
'\n ',
)}`,
).toEqual([]);
});
it('every KNOWN_UNCOVERED entry is a real schema type (no stale allowlist rows)', () => {
const all = new Set([...Object.keys(schema.nodes), ...Object.keys(schema.marks)]);
for (const t of Object.keys(KNOWN_UNCOVERED)) {
expect(all.has(t), `stale KNOWN_UNCOVERED entry: ${t}`).toBe(true);
}
});
it('every attribute is value-fuzzed OR explicitly allowlisted (no invisible hole)', () => {
// Makes the "generic-frozen" coverage hole VISIBLE: any schema attr not
// exercised at a non-default value must be a KNOWN entry in the allowlist.
// A new attr (or one that loses its `arb`) that falls into the not-fuzzed
// bucket without an allowlist row turns this red — no silent blind spots.
const unaccounted: string[] = [];
for (const key of allSchemaAttrKeys()) {
const i = key.indexOf('.');
const fuzzed = attrIsValueFuzzed(key.slice(0, i), key.slice(i + 1));
if (!fuzzed && !ATTR_VALUE_FUZZ_ALLOWLIST.has(key)) unaccounted.push(key);
}
expect(
unaccounted,
`these attrs are not value-fuzzed and not in ATTR_VALUE_FUZZ_ALLOWLIST:\n ${unaccounted.join(
'\n ',
)}`,
).toEqual([]);
});
it('the attribute allowlist has no stale rows (every entry is really not-fuzzed)', () => {
const notFuzzed = new Set(
allSchemaAttrKeys().filter((key) => {
const i = key.indexOf('.');
return !attrIsValueFuzzed(key.slice(0, i), key.slice(i + 1));
}),
);
for (const key of ATTR_VALUE_FUZZ_ALLOWLIST) {
expect(
notFuzzed.has(key),
`stale allowlist row (attr is now value-fuzzed, remove it): ${key}`,
).toBe(true);
}
});
it('every MARK attribute is value-fuzzed OR allowlisted (no invisible hole)', () => {
// The node guard above covers node attrs; marks are fuzzed by the text
// generator, so their coverage is tracked separately. A new mark attr (or a
// newly-attributed mark) that lands in neither set turns this red.
const unaccounted: string[] = [];
for (const key of allSchemaMarkAttrKeys()) {
if (!MARK_ATTR_FUZZED.has(key) && !MARK_ATTR_ALLOWLIST.has(key)) {
unaccounted.push(key);
}
}
expect(
unaccounted,
`these mark attrs are neither in MARK_ATTR_FUZZED nor MARK_ATTR_ALLOWLIST:\n ${unaccounted.join(
'\n ',
)}`,
).toEqual([]);
});
it('the MARK fuzz/allowlist sets have no stale rows (every entry is a real schema mark attr)', () => {
const all = new Set(allSchemaMarkAttrKeys());
for (const key of [...MARK_ATTR_FUZZED, ...MARK_ATTR_ALLOWLIST]) {
expect(all.has(key), `stale mark-attr registry row: ${key}`).toBe(true);
}
});
});
describe('#351 flat generative round-trip — properties', () => {
it('generator validity: every generated doc passes schema.check()', () => {
// A generator that emits an invalid ProseMirror document is a GENERATOR bug.
fc.assert(
fc.property(fuzzUnion, (doc) => {
schema.nodeFromJSON(doc).check(); // throws on an invalid doc
return true;
}),
{ numRuns: NUM_RUNS, seed: SEED },
);
});
it('P1 — semantic round-trip: docsCanonicallyEqual(mdToPm(pmToMd(d)), d)', async () => {
await fc.assert(
fc.asyncProperty(p1Union, async (doc) => {
const { doc2 } = await roundTrip(doc);
if (!docsCanonicallyEqual(doc2, doc)) {
// Surface the precise divergence in the failure message.
const div = firstDivergence(
JSON.parse(JSON.stringify(canonicalizeContent(doc2))),
JSON.parse(JSON.stringify(canonicalizeContent(doc))),
);
throw new Error(
`P1 divergence @ ${div?.path}: got=${JSON.stringify(div?.a)} want=${JSON.stringify(div?.b)}`,
);
}
}),
{ numRuns: NUM_RUNS, seed: SEED },
);
});
it('P2 — byte fixpoint: pmToMd(mdToPm(pmToMd(d))) === pmToMd(d)', async () => {
await fc.assert(
fc.asyncProperty(fuzzUnion, async (doc) => {
const { md1, md2 } = await roundTrip(doc);
expect(md2).toBe(md1);
}),
{ numRuns: NUM_RUNS, seed: SEED },
);
});
it('P3 — totality: neither converter throws', async () => {
await fc.assert(
fc.asyncProperty(fuzzUnion, async (doc) => {
// Throwing here fails the property; fast-check shrinks to a minimal doc.
await roundTrip(doc);
}),
{ numRuns: NUM_RUNS, seed: SEED },
);
});
});
@@ -0,0 +1,310 @@
/**
* Flat single-node document generators (#351, PR 1).
*
* For every schema node type that can stand alone, a fast-check arbitrary
* producing `{ type:'doc', content:[ <the target node> ] }` with generated attrs
* (via nodeAttrsArb) and the minimal REQUIRED immediate children the schema
* demands (a heading's inline text, a listItem's one paragraph, a table's
* minimal rows, details' summary+content, a callout's one paragraph). Kept
* FLAT: a single target node, no deep nesting nested structural generation is
* PR 2.
*
* The `mode` threads through to the attribute arbitraries:
* - 'p1' : the round-trip-safe attribute space (P1 semantic round-trip).
* - 'fuzz' : adds degenerate attribute states (P2 byte-fixpoint tolerates the
* one-time normalization; P3 only needs totality).
*
* A COMPLETENESS CONTRACT (see flat-roundtrip.property.test.ts) enumerates the
* whole schema and asserts every node/mark is EITHER produced by a generator
* here OR listed in KNOWN_UNCOVERED with a reason so a new schema type with no
* generator turns the suite RED.
*/
import fc from 'fast-check';
import { type AttrMode, nodeAttrsArb } from './attr-arbitraries.js';
import {
inlineContentArb,
headingInlineContentArb,
plainInlineContentArb,
phraseArb,
markedTextRunArb,
} from './text-arbitraries.js';
const doc = (node: any) => ({ type: 'doc', content: [node] });
const para = (content: any[]) => ({ type: 'paragraph', content });
/** A named flat-document generator. */
export interface NamedGen {
name: string;
arb: fc.Arbitrary<any>;
}
// ---------------------------------------------------------------------------
// Per-target generators, each a function of mode.
// ---------------------------------------------------------------------------
const gen = {
paragraph: (m: AttrMode) =>
fc.tuple(nodeAttrsArb('paragraph', m), inlineContentArb).map(([attrs, content]) =>
doc({ type: 'paragraph', attrs, content }),
),
heading: (m: AttrMode) =>
fc.tuple(nodeAttrsArb('heading', m), headingInlineContentArb).map(([attrs, content]) =>
doc({ type: 'heading', attrs, content }),
),
blockquote: (_m: AttrMode) =>
inlineContentArb.map((content) => doc({ type: 'blockquote', content: [para(content)] })),
bulletList: (_m: AttrMode) =>
fc
.array(inlineContentArb, { minLength: 1, maxLength: 3 })
.map((items) =>
doc({
type: 'bulletList',
content: items.map((c) => ({ type: 'listItem', content: [para(c)] })),
}),
),
orderedList: (m: AttrMode) =>
fc
.tuple(nodeAttrsArb('orderedList', m), fc.array(inlineContentArb, { minLength: 1, maxLength: 3 }))
.map(([attrs, items]) =>
doc({
type: 'orderedList',
attrs,
content: items.map((c) => ({ type: 'listItem', content: [para(c)] })),
}),
),
taskList: (m: AttrMode) =>
fc
.array(fc.tuple(nodeAttrsArb('taskItem', m), inlineContentArb), { minLength: 1, maxLength: 3 })
.map((items) =>
doc({
type: 'taskList',
content: items.map(([attrs, c]) => ({ type: 'taskItem', attrs, content: [para(c)] })),
}),
),
codeBlock: (m: AttrMode) =>
fc
.tuple(
nodeAttrsArb('codeBlock', m),
// A fenced code block always re-imports with a TRAILING NEWLINE in its
// text (empirically confirmed). Author the newline so the doc is already
// at the round-trip fixpoint (supported-space shaping, not a masked bug).
fc.array(phraseArb, { minLength: 1, maxLength: 3 }).map((lines) => lines.join('\n') + '\n'),
)
.map(([attrs, code]) =>
doc({ type: 'codeBlock', attrs, content: [{ type: 'text', text: code }] }),
),
horizontalRule: (_m: AttrMode) => fc.constant(doc({ type: 'horizontalRule' })),
pageBreak: (_m: AttrMode) => fc.constant(doc({ type: 'pageBreak' })),
image: (m: AttrMode) => nodeAttrsArb('image', m).map((attrs) => doc({ type: 'image', attrs })),
callout: (m: AttrMode) =>
fc.tuple(nodeAttrsArb('callout', m), inlineContentArb).map(([attrs, content]) =>
doc({ type: 'callout', attrs, content: [para(content)] }),
),
mathBlock: (m: AttrMode) =>
nodeAttrsArb('mathBlock', m).map((attrs) => doc({ type: 'mathBlock', attrs })),
details: (m: AttrMode) =>
fc
.tuple(nodeAttrsArb('details', m), plainInlineContentArb, inlineContentArb)
.map(([attrs, summary, body]) =>
doc({
type: 'details',
attrs,
content: [
{ type: 'detailsSummary', content: summary },
{ type: 'detailsContent', content: [para(body)] },
],
}),
),
table: (_m: AttrMode) =>
fc.integer({ min: 1, max: 3 }).chain((cols) => {
// GFM alignment is column-wide (encoded in the header separator), so a
// column's alignment must be identical on the header and every body cell,
// else the second export re-aligns and churns. Pick ONE align per column.
const alignsArb = fc.array(fc.constantFrom(undefined, 'left', 'center', 'right'), {
minLength: cols,
maxLength: cols,
});
const cell = (header: boolean, align?: string) =>
phraseArb.map((t) => ({
type: header ? 'tableHeader' : 'tableCell',
// colspan/rowspan pinned to 1 (GFM cannot express spans); optional
// column-consistent align.
attrs: { colspan: 1, rowspan: 1, ...(align ? { align } : {}) },
content: [para([{ type: 'text', text: t }])],
}));
return alignsArb.chain((aligns) => {
const headerRow = fc
.tuple(...aligns.map((a) => cell(true, a)))
.map((cells) => ({ type: 'tableRow', content: cells }));
const bodyRow = fc
.tuple(...aligns.map((a) => cell(false, a)))
.map((cells) => ({ type: 'tableRow', content: cells }));
return fc
.tuple(headerRow, fc.array(bodyRow, { minLength: 1, maxLength: 2 }))
.map(([h, body]) => doc({ type: 'table', content: [h, ...body] }));
});
}),
columns: (m: AttrMode) =>
// Couple the column count to the layout so the two stay consistent
// (two_equal/left_sidebar/right_sidebar -> 2, three_equal -> 3).
fc
.constantFrom('two_equal', 'three_equal', 'left_sidebar', 'right_sidebar')
.chain((layout) => {
const count = layout === 'three_equal' ? 3 : 2;
return fc
.tuple(
nodeAttrsArb('columns', m, { layout, widthMode: 'normal' }),
fc.array(inlineContentArb, { minLength: count, maxLength: count }),
)
.map(([attrs, bodies]) =>
doc({
type: 'columns',
attrs,
content: bodies.map((c) => ({ type: 'column', content: [para(c)] })),
}),
);
}),
subpages: (m: AttrMode) =>
nodeAttrsArb('subpages', m).map((attrs) => doc({ type: 'subpages', attrs })),
audio: (m: AttrMode) => nodeAttrsArb('audio', m).map((attrs) => doc({ type: 'audio', attrs })),
video: (m: AttrMode) => nodeAttrsArb('video', m).map((attrs) => doc({ type: 'video', attrs })),
pdf: (m: AttrMode) => nodeAttrsArb('pdf', m).map((attrs) => doc({ type: 'pdf', attrs })),
youtube: (m: AttrMode) => nodeAttrsArb('youtube', m).map((attrs) => doc({ type: 'youtube', attrs })),
embed: (m: AttrMode) => nodeAttrsArb('embed', m).map((attrs) => doc({ type: 'embed', attrs })),
drawio: (m: AttrMode) => nodeAttrsArb('drawio', m).map((attrs) => doc({ type: 'drawio', attrs })),
excalidraw: (m: AttrMode) =>
nodeAttrsArb('excalidraw', m).map((attrs) => doc({ type: 'excalidraw', attrs })),
attachment: (m: AttrMode) =>
nodeAttrsArb('attachment', m).map((attrs) => doc({ type: 'attachment', attrs })),
htmlEmbed: (m: AttrMode) =>
nodeAttrsArb('htmlEmbed', m).map((attrs) => doc({ type: 'htmlEmbed', attrs })),
pageEmbed: (m: AttrMode) =>
nodeAttrsArb('pageEmbed', m).map((attrs) => doc({ type: 'pageEmbed', attrs })),
transclusionReference: (m: AttrMode) =>
nodeAttrsArb('transclusionReference', m).map((attrs) =>
doc({ type: 'transclusionReference', attrs }),
),
transclusionSource: (m: AttrMode) =>
fc.tuple(nodeAttrsArb('transclusionSource', m), inlineContentArb).map(([attrs, content]) =>
doc({ type: 'transclusionSource', attrs, content: [para(content)] }),
),
// A footnote reference PLUS its definition (the reference has no standalone
// markdown form without its definition — see KNOWN_UNCOVERED note for the
// bare reference). Both carry the same id. The definition body uses
// headingInlineContentArb (NO hard breaks): a footnote is serialized inline as
// `^[...]`, so a hard break inside it collapses to a single space on re-parse
// (empirically confirmed) — that is the container's markdown limitation, not
// an attribute-level concern. The reference-bearing paragraph is a NORMAL
// paragraph and keeps the full inline corpus.
footnotes: (m: AttrMode) =>
fc.tuple(fc.constantFrom('fn1', 'fn2', 'note'), inlineContentArb, headingInlineContentArb).map(
([id, refText, noteBody]) => ({
type: 'doc',
content: [
para([...refText, { type: 'footnoteReference', attrs: { id } }]),
{
type: 'footnotesList',
content: [{ type: 'footnoteDefinition', attrs: { id }, content: [para(noteBody)] }],
},
],
}),
),
// ── inline targets wrapped in a paragraph ────────────────────────────────
mention: (m: AttrMode) =>
nodeAttrsArb('mention', m).map((attrs) => doc(para([{ type: 'mention', attrs }]))),
mathInline: (m: AttrMode) =>
fc.tuple(phraseArb, nodeAttrsArb('mathInline', m)).map(([t, attrs]) =>
doc(para([{ type: 'text', text: t }, { type: 'mathInline', attrs }])),
),
status: (m: AttrMode) =>
nodeAttrsArb('status', m).map((attrs) => doc(para([{ type: 'status', attrs }]))),
hardBreak: (_m: AttrMode) =>
fc.tuple(phraseArb, phraseArb).map(([a, b]) =>
doc(para([{ type: 'text', text: a }, { type: 'hardBreak' }, { type: 'text', text: b }])),
),
// ── marks: a paragraph of marked runs (covers every mark type) ───────────
marksOnText: (_m: AttrMode) =>
fc.array(markedTextRunArb, { minLength: 1, maxLength: 5 }).map((runs) => {
// Merge adjacent same-mark runs (see text-arbitraries.normalizeInline).
const out: any[] = [];
for (const r of runs) {
const prev = out[out.length - 1];
if (prev && JSON.stringify(prev.marks ?? []) === JSON.stringify(r.marks ?? [])) {
prev.text += r.text;
} else out.push({ ...r });
}
return doc(para(out));
}),
};
/** Build the full list of named generators for a given mode. */
export function buildGenerators(mode: AttrMode): NamedGen[] {
return Object.entries(gen).map(([name, f]) => ({ name, arb: f(mode) }));
}
// ---------------------------------------------------------------------------
// Completeness contract support.
// ---------------------------------------------------------------------------
/**
* Schema node/mark types deliberately NOT covered by a P1/P2 generator, each
* with a one-line reason. Excluding a type means it is kept OUT of the round-
* trip generators it does NOT weaken any property.
*
* NOTE (empirical): the candidates the issue flagged for review pageEmbed,
* subpages, transclusionSource/Reference, mention, status were PROBED against
* the live converter and DO round-trip P1/P2 with placeholder ids, so they are
* COVERED by real generators rather than allowlisted here. The allowlist below
* holds only types with no standalone flat generator by construction.
*/
export const KNOWN_UNCOVERED: Record<string, string> = {
// The root node; it is the wrapper every generated doc already is, never a
// "target" content node, so it has no standalone generator of its own.
doc: 'the document root wrapper, not a content node with a standalone generator',
};
/** Recursively collect every node type and `mark:<type>` under a tree. */
export function collectTypes(node: any, seen = new Set<string>()): Set<string> {
if (!node || typeof node !== 'object') return seen;
if (node.type) seen.add(node.type);
for (const m of node.marks ?? []) if (m?.type) seen.add(`mark:${m.type}`);
for (const c of node.content ?? []) collectTypes(c, seen);
return seen;
}
/**
* Sample every generator and return the union of node/mark types they produce.
* Deterministic (fixed seed) so the completeness contract is stable.
*/
export function coveredTypes(seed = 12345, perGen = 60): Set<string> {
const seen = new Set<string>();
for (const { arb } of buildGenerators('p1')) {
for (const sample of fc.sample(arb, { numRuns: perGen, seed })) {
collectTypes(sample, seen);
}
}
return seen;
}
@@ -0,0 +1,258 @@
/**
* Hostile inline-text corpus for the generative flat-document round-trip suite
* (#351, PR 1).
*
* These arbitraries are a DIRECT PORT of the "supported space" guardrails that
* `test/markdown-roundtrip.property.test.ts` proved empirically against the live
* converter. That file's long header documents WHY each guardrail exists; rather
* than re-derive them, we reuse the exact same shapes here so the attribute-level
* generative suite inherits the same byte-stable text space. Each guardrail is
* cited back to that file below.
*
* The corpus deliberately spans the CommonMark / canon hostile alphabet
* (`* _ [ ] ( ) { } | < > & # ! ~ = + -`), unicode / emoji / RTL, and the legal
* mark combinations on runs (including the `code` mark, which the schema's
* `excludes: "_"` makes suppress every co-occurring mark so it is never
* combined with another mark in the byte-stable space).
*/
import fc from 'fast-check';
// ---------------------------------------------------------------------------
// Words and the hostile special-character alphabet.
// (Ported from markdown-roundtrip.property.test.ts, "Inline text arbitraries".)
// ---------------------------------------------------------------------------
/** Alphanumeric "word" (no markdown-significant characters). Length 1..6. */
export const wordArb = fc
.stringMatching(/^[A-Za-z0-9]{1,6}$/)
.filter((w) => w.length > 0);
/**
* A SINGLE markdown-significant character, emitted only as an isolated,
* space-flanked token. Every char the task calls out plus a few more; each was
* verified byte-stable in this position by the sibling property test.
*
* NOTE: the backtick (`) is DELIBERATELY excluded from free-floating plain text
* (it is a code-span delimiter that re-pairs globally). It is exercised only via
* the `code` mark and code blocks see markdown-roundtrip.property.test.ts.
*/
export const specialCharArb = fc.constantFrom(
'*', '_', '[', ']', '(', ')', '{', '}', '|', '<', '>', '&', '#', '!', '~', '=', '+', '-',
);
// A pinch of unicode / emoji / RTL, always word-like (no markdown specials) so
// it stays inside the space-flanked corpus. Kept letter/emoji-bearing so it is
// never coerced to a number (see letterPhraseArb rationale).
export const unicodeWordArb = fc.constantFrom(
'café', 'naïve', 'Zürich', 'Москва', 'こんにちは', '你好', '😀', '🚀x', 'مرحبا', 'שלום',
);
/**
* A "safe special" text string: a space-joined sequence of tokens that always
* BEGINS and ENDS with an alphanumeric word, with any isolated special chars (or
* unicode words) confined to the MIDDLE, each space-flanked by words.
*
* Both boundary guarantees matter (verbatim from the sibling test):
* * Leading word: the line never opens with a block/inline trigger
* (">", "*", "-", "#", "1." ...).
* * Trailing word: adjacent text runs CONCATENATE with no separator, so a run
* ending in a bare "<" beside a run starting with a letter would form a fake
* HTML tag. Ending every run with a word keeps every special internal and
* space-flanked even after concatenation.
*/
export const safeTextArb: fc.Arbitrary<string> = fc
.tuple(
wordArb,
fc.array(fc.oneof(wordArb, specialCharArb, unicodeWordArb), {
minLength: 0,
maxLength: 3,
}),
wordArb,
)
.map(([first, middle, last]) => [first, ...middle, last].join(' '));
/**
* A plain alphanumeric phrase (1..3 words) for places where even isolated
* specials are not wanted (e.g. code-block language, mention labels, status
* text, table cells rendered on the plain-markdown path).
*/
export const phraseArb: fc.Arbitrary<string> = fc
.array(wordArb, { minLength: 1, maxLength: 3 })
.map((ws) => ws.join(' '));
/**
* A phrase guaranteed to contain at least one letter. Used for image/media alt
* text and link titles: a PURELY numeric alt/title (e.g. "0") is parsed back as
* a NUMBER and then dropped by the converter's `value || ""` coercion not
* byte-stable. A letter anywhere keeps it a string. (Ported verbatim.)
*/
export const letterPhraseArb: fc.Arbitrary<string> = fc
.tuple(
fc.stringMatching(/^[A-Za-z]{1,4}$/),
fc.array(wordArb, { minLength: 0, maxLength: 2 }),
)
.map(([head, rest]) => [head, ...rest].join(' '));
/** A paren/space-free URL — safe inside markdown link/image `(...)` syntax. */
export const urlArb: fc.Arbitrary<string> = fc
.webUrl()
.filter((u) => !/[()\s]/.test(u));
// ---------------------------------------------------------------------------
// Marked inline runs.
// (Ported from markdown-roundtrip.property.test.ts "markedTextRunArb".)
// ---------------------------------------------------------------------------
/**
* A text run with an OPTIONAL single non-code formatting mark (bold/italic/
* strike/underline/superscript/subscript/spoiler), or a SOLE `code` mark, or a
* link, or an inline comment anchor. `code` is NEVER combined with another mark
* in the byte-stable space (that combination is a documented converter
* limitation the schema's `code` mark declares `excludes: "_"`). Marks wrap
* `safeTextArb`, which stays stable even when it contains isolated specials.
*
* The mark set here is broadened past the sibling test's {bold,italic,strike}
* to also cover underline / superscript / subscript / spoiler / textStyle /
* highlight (all single, non-code marks), so the marks-on-text generator
* exercises every mark the schema declares except the deliberately-excluded
* `code`+other combination.
*/
export const markedTextRunArb: fc.Arbitrary<any> = fc.oneof(
// Plain text.
safeTextArb.map((t) => ({ type: 'text', text: t })),
// Single formatting mark (attribute-free marks).
fc
.tuple(
safeTextArb,
fc.constantFrom('bold', 'italic', 'strike', 'underline', 'superscript', 'subscript', 'spoiler'),
)
.map(([t, m]) => ({ type: 'text', text: t, marks: [{ type: m }] })),
// highlight with a color attr.
fc
.tuple(safeTextArb, fc.constantFrom('#ffcc00', '#a0e0ff', 'yellow'))
.map(([t, color]) => ({ type: 'text', text: t, marks: [{ type: 'highlight', attrs: { color } }] })),
// textStyle with a color attr.
fc
.tuple(safeTextArb, fc.constantFrom('#123456', '#ff0000', '#00aa88'))
.map(([t, color]) => ({ type: 'text', text: t, marks: [{ type: 'textStyle', attrs: { color } }] })),
// Sole code mark (backtick span). safeTextArb is backtick-free, so the span
// content cannot contain an inner backtick.
safeTextArb.map((t) => ({ type: 'text', text: t, marks: [{ type: 'code' }] })),
// Link with safe text, a paren/space-free href, optionally a letter-bearing
// title (a purely numeric title is coerced to a number and dropped).
fc
.tuple(phraseArb, urlArb, fc.option(letterPhraseArb, { nil: undefined }))
.map(([t, href, title]) => ({
type: 'text',
text: t,
marks: [{ type: 'link', attrs: title ? { href, title } : { href } }],
})),
// Inline comment anchor: a span[data-comment-id] that must survive byte-for-
// byte. commentId is an alphanumeric token; `resolved` rides only when true.
fc
.tuple(safeTextArb, fc.stringMatching(/^[A-Za-z0-9]{4,10}$/), fc.boolean())
.map(([t, commentId, resolved]) => ({
type: 'text',
text: t,
marks: [
{ type: 'comment', attrs: resolved ? { commentId, resolved: true } : { commentId } },
],
})),
);
// ---------------------------------------------------------------------------
// Inline atoms and inline-content assembly.
// (Ported from markdown-roundtrip.property.test.ts.)
// ---------------------------------------------------------------------------
/** Inline math node carrying LaTeX that includes the `a < b` the task asks for. */
export const mathInlineArb: fc.Arbitrary<any> = fc
.constantFrom('a < b', 'x^2 + y^2', 'a < b < c', '\\frac{1}{2}', 'E = mc^2')
.map((text) => ({ type: 'mathInline', attrs: { text } }));
/** Mention node; label/id/entity are plain phrases / uuids. */
export const mentionArb: fc.Arbitrary<any> = fc
.tuple(phraseArb, fc.uuid(), fc.uuid())
.map(([label, id, entityId]) => ({
type: 'mention',
attrs: { id, label, entityType: 'user', entityId },
}));
export const hardBreakArb: fc.Arbitrary<any> = fc.constant({ type: 'hardBreak' });
const sameMarks = (a: any[] | undefined, b: any[] | undefined): boolean =>
JSON.stringify(a ?? []) === JSON.stringify(b ?? []);
/**
* Canonicalize a generated inline-content array the way ProseMirror stores it,
* then trim the markdown-fragile edges. (Ported verbatim from
* markdown-roundtrip.property.test.ts "normalizeInline":)
* 1) MERGE adjacent text runs with IDENTICAL marks (the editor coalesces
* them; split same-mark runs export to ambiguous "**a****b**").
* 2) Collapse CONSECUTIVE hard breaks (two render a blank line marked eats).
* 3) Drop a TRAILING hard break (removed by the converter's .trim()).
*/
export function normalizeInline(nodes: any[]): any[] {
const out: any[] = [];
for (const node of nodes) {
const prev = out[out.length - 1];
if (node.type === 'hardBreak' && prev && prev.type === 'hardBreak') continue;
if (
node.type === 'text' &&
prev &&
prev.type === 'text' &&
sameMarks(prev.marks, node.marks)
) {
prev.text += node.text;
continue;
}
out.push(node.type === 'text' ? { ...node } : node);
}
while (out.length > 1 && out[out.length - 1].type === 'hardBreak') out.pop();
return out;
}
/**
* Inline content for a paragraph: at least one marked text run, optionally with
* inline atoms (math/mention) and hard breaks interspersed. Always starts with a
* text run so the paragraph never opens with a block trigger. (Ported.)
*/
export const inlineContentArb: fc.Arbitrary<any[]> = fc
.tuple(
markedTextRunArb,
fc.array(
fc.oneof(
{ weight: 5, arbitrary: markedTextRunArb },
{ weight: 1, arbitrary: mathInlineArb },
{ weight: 1, arbitrary: mentionArb },
{ weight: 1, arbitrary: hardBreakArb },
),
{ minLength: 0, maxLength: 4 },
),
)
.map(([first, rest]) => normalizeInline([first, ...rest]));
/**
* Inline content for a HEADING identical to a paragraph's, but WITHOUT hard
* breaks. A hard break inside an ATX heading is not byte-stable (marked splits
* the heading). (Ported.)
*/
export const headingInlineContentArb: fc.Arbitrary<any[]> = fc
.tuple(
markedTextRunArb,
fc.array(
fc.oneof(
{ weight: 5, arbitrary: markedTextRunArb },
{ weight: 1, arbitrary: mathInlineArb },
{ weight: 1, arbitrary: mentionArb },
),
{ minLength: 0, maxLength: 4 },
),
)
.map(([first, rest]) => normalizeInline([first, ...rest]));
/** Simple plain-text inline content (single run) for containers rendered on the
* raw-HTML path (table cells / column bodies) where fancy inline is undesirable. */
export const plainInlineContentArb: fc.Arbitrary<any[]> = phraseArb.map((t) => [
{ type: 'text', text: t },
]);
@@ -0,0 +1,124 @@
import { describe, expect, it } from "vitest";
// Import DIRECTLY from src (NOT the docmost-client barrel, which pulls in
// collaboration.ts and mutates global DOM at import time).
import { convertProseMirrorToMarkdown } from "../src/lib/markdown-converter.js";
import { markdownToProseMirror } from "../src/lib/markdown-to-prosemirror.js";
/**
* gitmost #377 (round-1 review, finding #1) proof, against the REAL
* converter, that the transcript-insert boundary defense survives git-sync.
*
* The web bridge (apps/client .../gitmost/gitmost-recording.ts,
* `gitmostInsertTranscriptIntoEditor`) appends each transcript line as a
* PARAGRAPH text node. The paragraph serializer here (`case "paragraph"`) emits
* that text VERBATIM with no block-escape, so a line whose text begins with a
* col-0 markdown block trigger would, on the doc -> markdown -> doc git-sync
* cycle, silently re-parse into a heading / list / quote / callout / code block.
* That missing block-escape is the pre-existing root cause; the bridge's
* boundary defense prepends an invisible zero-width space (U+200B) to a line
* that begins with such a trigger, shifting it off column 0.
*
* This test keeps a COPY of the bridge's trigger regex (the bridge is in a
* different package and can't be imported here) and asserts:
* 1. bare trigger lines DO corrupt (documents the root cause), and
* 2. the ZWSP-neutralized form round-trips as a single PARAGRAPH with the
* text byte-preserved.
*/
const ZWSP = "​"; // U+200B
// MUST stay in sync with GITMOST_MD_BLOCK_TRIGGER_RE in the client bridge.
const MD_BLOCK_TRIGGER_RE =
/^(?:#{1,6}(?:\s|$)|[-*+](?:\s|$)|>|\d+[.)](?:\s|$)|```|~~~|\||([-*_])(?:\s*\1){2,}\s*$)/;
const doc = (...nodes: any[]) => ({ type: "doc", content: nodes });
const para = (t: string) => ({
type: "paragraph",
content: [{ type: "text", text: t }],
});
const roundtrip = async (text: string) => {
const md = convertProseMirrorToMarkdown(doc(para(text)));
const back = await markdownToProseMirror(md);
return back.content as any[];
};
describe("gitmost transcript neutralization (git-sync round-trip)", () => {
// Lines that, at column 0, the serializer's missing block-escape would let
// git-sync re-parse into a non-paragraph block.
const triggerLines = [
"- dash",
"* star",
"+ plus",
"> quote",
"# hash",
"1. one",
"1) one",
"> [!info] note",
"```js",
"~~~",
// Solid + spaced thematic breaks — these re-parse into a `horizontalRule`,
// which carries NO text, so a bare separator line LOSES its text entirely
// (round-2 finding). `_` also only forms a block via this construct.
"---",
"***",
"___",
"- - -", // spaced dash break (solid form is caught by [-*+]\s too, but this is the break)
"_ _ _",
];
it("BARE trigger lines corrupt into non-paragraph blocks (root cause)", async () => {
for (const line of triggerLines) {
const blocks = await roundtrip(line);
// At least one produced block is NOT a paragraph — i.e. corruption.
const allParagraphs = blocks.every((b) => b.type === "paragraph");
expect(
allParagraphs,
`expected "${line}" to corrupt when inserted bare`,
).toBe(false);
}
});
it("BARE solid thematic breaks corrupt into a text-LOSING horizontalRule", async () => {
// The severe case: no text node survives. Documents why neutralization
// matters more here than for list/quote (where the text survived).
for (const line of ["---", "***", "___"]) {
const blocks = await roundtrip(line);
expect(blocks.map((b) => b.type)).toContain("horizontalRule");
// No block carries the original text anywhere.
const flat = JSON.stringify(blocks);
expect(flat).not.toContain(line);
}
});
it("ZWSP-neutralized trigger lines round-trip as a single paragraph, text preserved", async () => {
for (const line of triggerLines) {
// The regex must actually classify each as a trigger.
expect(MD_BLOCK_TRIGGER_RE.test(line), `regex missed "${line}"`).toBe(
true,
);
const neutralized = ZWSP + line;
const blocks = await roundtrip(neutralized);
expect(blocks).toHaveLength(1);
expect(blocks[0].type).toBe("paragraph");
// Text is byte-preserved (ZWSP + original line), so the display is the
// original line with only an invisible leading character.
expect(blocks[0].content[0].text).toBe(neutralized);
}
});
it("normal host-prefixed lines never match the trigger regex and round-trip byte-exact", async () => {
for (const line of [
"You: hello there",
"Speaker 1: - and then a dash mid-line",
"Speaker 2: 1. not a list",
]) {
expect(MD_BLOCK_TRIGGER_RE.test(line)).toBe(false);
const blocks = await roundtrip(line);
expect(blocks).toHaveLength(1);
expect(blocks[0].type).toBe("paragraph");
expect(blocks[0].content[0].text).toBe(line);
}
});
});
+68
View File
@@ -0,0 +1,68 @@
diff --git a/dist/index.js b/dist/index.js
index ae447a12f7823ec0a00837ee9f0eb809a610d5f8..a3402b2c2d021ef432cfa76e35d370073d525135 100644
--- a/dist/index.js
+++ b/dist/index.js
@@ -6578,9 +6578,19 @@ function createOutputTransformStream(output) {
controller.enqueue({ part: chunk, partialOutput: void 0 });
return;
}
- text2 += chunk.text;
textChunk += chunk.text;
textProviderMetadata = (_a21 = chunk.providerMetadata) != null ? _a21 : textProviderMetadata;
+ if (output == null) {
+ // PATCH(docmost #OOM): no output strategy requested -> publish each
+ // text-delta immediately and do NOT build cumulative partialOutput
+ // snapshots. Unpatched, the default text() output snapshots the ENTIRE
+ // accumulated turn text on every delta (O(n^2) memory) and those
+ // snapshots pile up in the never-consumed leftover tee branch of
+ // DefaultStreamTextResult.baseStream -> heap OOM on long agent turns.
+ publishTextChunk({ controller });
+ return;
+ }
+ text2 += chunk.text;
const result = await output.parsePartialOutput({ text: text2 });
if (result !== void 0) {
const currentJson = JSON.stringify(result.partial);
@@ -6959,7 +6969,7 @@ var DefaultStreamTextResult = class {
})
);
}
- this.baseStream = stream.pipeThrough(createOutputTransformStream(output != null ? output : text())).pipeThrough(eventProcessor);
+ this.baseStream = stream.pipeThrough(createOutputTransformStream(output)).pipeThrough(eventProcessor);
const { maxRetries, retry } = prepareRetries({
maxRetries: maxRetriesArg,
abortSignal
diff --git a/dist/index.mjs b/dist/index.mjs
index 663875332e3f9a9bd167c25583c515876f42951b..b840b0502c9894df983e0154805abb80e70e6331 100644
--- a/dist/index.mjs
+++ b/dist/index.mjs
@@ -6501,9 +6501,19 @@ function createOutputTransformStream(output) {
controller.enqueue({ part: chunk, partialOutput: void 0 });
return;
}
- text2 += chunk.text;
textChunk += chunk.text;
textProviderMetadata = (_a21 = chunk.providerMetadata) != null ? _a21 : textProviderMetadata;
+ if (output == null) {
+ // PATCH(docmost #OOM): no output strategy requested -> publish each
+ // text-delta immediately and do NOT build cumulative partialOutput
+ // snapshots. Unpatched, the default text() output snapshots the ENTIRE
+ // accumulated turn text on every delta (O(n^2) memory) and those
+ // snapshots pile up in the never-consumed leftover tee branch of
+ // DefaultStreamTextResult.baseStream -> heap OOM on long agent turns.
+ publishTextChunk({ controller });
+ return;
+ }
+ text2 += chunk.text;
const result = await output.parsePartialOutput({ text: text2 });
if (result !== void 0) {
const currentJson = JSON.stringify(result.partial);
@@ -6882,7 +6892,7 @@ var DefaultStreamTextResult = class {
})
);
}
- this.baseStream = stream.pipeThrough(createOutputTransformStream(output != null ? output : text())).pipeThrough(eventProcessor);
+ this.baseStream = stream.pipeThrough(createOutputTransformStream(output)).pipeThrough(eventProcessor);
const { maxRetries, retry } = prepareRetries({
maxRetries: maxRetriesArg,
abortSignal
+134 -7
View File
@@ -44,6 +44,9 @@ overrides:
ip-address: 10.1.1 ip-address: 10.1.1
patchedDependencies: patchedDependencies:
ai@6.0.134:
hash: f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9
path: patches/ai@6.0.134.patch
scimmy@1.3.5: scimmy@1.3.5:
hash: 775d80f86830b2c5dd1a250c9802c10f8fc3da3c7898373de5aa0c23993d1673 hash: 775d80f86830b2c5dd1a250c9802c10f8fc3da3c7898373de5aa0c23993d1673
path: patches/scimmy@1.3.5.patch path: patches/scimmy@1.3.5.patch
@@ -504,6 +507,9 @@ importers:
vite: vite:
specifier: 8.0.5 specifier: 8.0.5
version: 8.0.5(@types/node@22.19.1)(esbuild@0.28.0)(jiti@2.4.2)(less@4.2.0)(sugarss@5.0.1(postcss@8.5.14))(terser@5.39.0)(tsx@4.21.0)(yaml@2.8.3) version: 8.0.5(@types/node@22.19.1)(esbuild@0.28.0)(jiti@2.4.2)(less@4.2.0)(sugarss@5.0.1(postcss@8.5.14))(terser@5.39.0)(tsx@4.21.0)(yaml@2.8.3)
vite-plugin-compression2:
specifier: 2.5.3
version: 2.5.3
vitest: vitest:
specifier: 4.1.6 specifier: 4.1.6
version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@22.19.1)(@vitest/coverage-v8@4.1.6)(happy-dom@20.8.9)(jsdom@25.0.0)(vite@8.0.5(@types/node@22.19.1)(esbuild@0.28.0)(jiti@2.4.2)(less@4.2.0)(sugarss@5.0.1(postcss@8.5.14))(terser@5.39.0)(tsx@4.21.0)(yaml@2.8.3)) version: 4.1.6(@opentelemetry/api@1.9.0)(@types/node@22.19.1)(@vitest/coverage-v8@4.1.6)(happy-dom@20.8.9)(jsdom@25.0.0)(vite@8.0.5(@types/node@22.19.1)(esbuild@0.28.0)(jiti@2.4.2)(less@4.2.0)(sugarss@5.0.1(postcss@8.5.14))(terser@5.39.0)(tsx@4.21.0)(yaml@2.8.3))
@@ -546,6 +552,9 @@ importers:
'@docmost/prosemirror-markdown': '@docmost/prosemirror-markdown':
specifier: workspace:* specifier: workspace:*
version: link:../../packages/prosemirror-markdown version: link:../../packages/prosemirror-markdown
'@fastify/compress':
specifier: ^9.0.0
version: 9.0.0
'@fastify/cookie': '@fastify/cookie':
specifier: ^11.0.2 specifier: ^11.0.2
version: 11.0.2 version: 11.0.2
@@ -626,10 +635,10 @@ importers:
version: 8.3.0(socket.io-adapter@2.5.4) version: 8.3.0(socket.io-adapter@2.5.4)
ai: ai:
specifier: ^6.0.134 specifier: ^6.0.134
version: 6.0.134(zod@4.3.6) version: 6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6)
ai-sdk-ollama: ai-sdk-ollama:
specifier: ^3.8.1 specifier: ^3.8.1
version: 3.8.1(ai@6.0.134(zod@4.3.6))(zod@4.3.6) version: 3.8.1(ai@6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6))(zod@4.3.6)
bcrypt: bcrypt:
specifier: ^6.0.0 specifier: ^6.0.0
version: 6.0.0 version: 6.0.0
@@ -2703,6 +2712,9 @@ packages:
'@fastify/busboy@3.1.1': '@fastify/busboy@3.1.1':
resolution: {integrity: sha512-5DGmA8FTdB2XbDeEwc/5ZXBl6UbBAyBOOLlPuBnZ/N1SwdH9Ii+cOX3tBROlDgcTXxjOYnLMVoKk9+FXAw0CJw==} resolution: {integrity: sha512-5DGmA8FTdB2XbDeEwc/5ZXBl6UbBAyBOOLlPuBnZ/N1SwdH9Ii+cOX3tBROlDgcTXxjOYnLMVoKk9+FXAw0CJw==}
'@fastify/compress@9.0.0':
resolution: {integrity: sha512-PZRg+ut5xd/ubsGPWfoPNryoCOtEdHboIWpDieTUHov1gKdLitF8mRmT3JbqNnRbelQXSNXUsIpakAEKR6AcTQ==}
'@fastify/cookie@11.0.2': '@fastify/cookie@11.0.2':
resolution: {integrity: sha512-GWdwdGlgJxyvNv+QcKiGNevSspMQXncjMZ1J8IvuDQk0jvkzgWWZFNC2En3s+nHndZBGV8IbLwOI/sxCZw/mzA==} resolution: {integrity: sha512-GWdwdGlgJxyvNv+QcKiGNevSspMQXncjMZ1J8IvuDQk0jvkzgWWZFNC2En3s+nHndZBGV8IbLwOI/sxCZw/mzA==}
@@ -4496,6 +4508,15 @@ packages:
'@rolldown/pluginutils@1.0.0-rc.7': '@rolldown/pluginutils@1.0.0-rc.7':
resolution: {integrity: sha512-qujRfC8sFVInYSPPMLQByRh7zhwkGFS4+tyMQ83srV1qrxL4g8E2tyxVVyxd0+8QeBM1mIk9KbWxkegRr76XzA==} resolution: {integrity: sha512-qujRfC8sFVInYSPPMLQByRh7zhwkGFS4+tyMQ83srV1qrxL4g8E2tyxVVyxd0+8QeBM1mIk9KbWxkegRr76XzA==}
'@rollup/pluginutils@5.4.0':
resolution: {integrity: sha512-MfPp06CjRLfXQ3wY0R8vJDYBy/MvVcc9OulEfR0B8Iv9ko+GCNaRZ+EpJYFl27LhKsZK0o420sYCRHCjfCgeUg==}
engines: {node: '>=14.0.0'}
peerDependencies:
rollup: ^1.20.0||^2.0.0||^3.0.0||^4.0.0
peerDependenciesMeta:
rollup:
optional: true
'@selderee/plugin-htmlparser2@0.11.0': '@selderee/plugin-htmlparser2@0.11.0':
resolution: {integrity: sha512-P33hHGdldxGabLFjPPpaTxVolMrzrcegejx+0GxjrIb9Zv48D8yAIA/QTDR2dFl7Uz7urX8aX6+5bCZslr+gWQ==} resolution: {integrity: sha512-P33hHGdldxGabLFjPPpaTxVolMrzrcegejx+0GxjrIb9Zv48D8yAIA/QTDR2dFl7Uz7urX8aX6+5bCZslr+gWQ==}
@@ -5685,6 +5706,10 @@ packages:
resolution: {integrity: sha512-/XrFJgzQQQHpti1raDJC6m4ws6aNktmjBlhk8Fdlk7LwCEuDoieEJJY9OFHjfiFJFFRM2tK+Ky/IsfbbmlMu1w==} resolution: {integrity: sha512-/XrFJgzQQQHpti1raDJC6m4ws6aNktmjBlhk8Fdlk7LwCEuDoieEJJY9OFHjfiFJFFRM2tK+Ky/IsfbbmlMu1w==}
engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0} engines: {node: ^22.22.2 || ^24.15.0 || >=26.0.0}
abort-controller@3.0.0:
resolution: {integrity: sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==}
engines: {node: '>=6.5'}
abstract-logging@2.0.1: abstract-logging@2.0.1:
resolution: {integrity: sha512-2BjRTZxTPvheOvGbBslFSYOUkr+SjPtOnrLP33f+VIWLzezQpZcqVg7ja3L4dBXmzzgwT+a029jRx5PCi3JuiA==} resolution: {integrity: sha512-2BjRTZxTPvheOvGbBslFSYOUkr+SjPtOnrLP33f+VIWLzezQpZcqVg7ja3L4dBXmzzgwT+a029jRx5PCi3JuiA==}
@@ -6067,6 +6092,9 @@ packages:
buffer@5.7.1: buffer@5.7.1:
resolution: {integrity: sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==} resolution: {integrity: sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ==}
buffer@6.0.3:
resolution: {integrity: sha512-FTiCpNxtwiZZHEZbcbTIcZjERVICn9yq/pDFkTl95/AxzD1naBctN7YO68riM/gLSDY7sdrMby8hofADYuuqOA==}
bullmq@5.76.10: bullmq@5.76.10:
resolution: {integrity: sha512-LWve7SpQjYSpCP2GEsWmoyzTz2H37L8HRmSTu3YihYsTOr5kJxrfEX6aEV7m6eskEMWXSHZYTMZepX6qNaH6CQ==} resolution: {integrity: sha512-LWve7SpQjYSpCP2GEsWmoyzTz2H37L8HRmSTu3YihYsTOr5kJxrfEX6aEV7m6eskEMWXSHZYTMZepX6qNaH6CQ==}
engines: {node: '>=12.22.0'} engines: {node: '>=12.22.0'}
@@ -6822,6 +6850,9 @@ packages:
resolution: {integrity: sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==} resolution: {integrity: sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==}
engines: {node: '>= 0.4'} engines: {node: '>= 0.4'}
duplexify@3.7.1:
resolution: {integrity: sha512-07z8uv2wMyS51kKhD1KsdXJg5WQ6t93RneqRxUHnskXVtlYYkLqM0gqStQZ3pj073g687jPCHrqNfCzawLYh5g==}
ecdsa-sig-formatter@1.0.11: ecdsa-sig-formatter@1.0.11:
resolution: {integrity: sha512-nagl3RYrbNv6kQkeJIpt6NJZy8twLB/2vtz6yN9Z4vRKHN4/QZJIEbqohALSgwKdnksuY3k5Addp5lg8sVoVcQ==} resolution: {integrity: sha512-nagl3RYrbNv6kQkeJIpt6NJZy8twLB/2vtz6yN9Z4vRKHN4/QZJIEbqohALSgwKdnksuY3k5Addp5lg8sVoVcQ==}
@@ -7065,6 +7096,9 @@ packages:
resolution: {integrity: sha512-MMdARuVEQziNTeJD8DgMqmhwR11BRQ/cBP+pLtYdSTnf3MIO8fFeiINEbX36ZdNlfU/7A9f3gUw49B3oQsvwBA==} resolution: {integrity: sha512-MMdARuVEQziNTeJD8DgMqmhwR11BRQ/cBP+pLtYdSTnf3MIO8fFeiINEbX36ZdNlfU/7A9f3gUw49B3oQsvwBA==}
engines: {node: '>=4.0'} engines: {node: '>=4.0'}
estree-walker@2.0.2:
resolution: {integrity: sha512-Rfkk/Mp/DL7JVje3u18FxFujQlTNR2q6QfMSMB7AvCBx91NGj/ba3kCfza0f6dVDbw7YlRf/nDrn7pQrCCyQ/w==}
estree-walker@3.0.3: estree-walker@3.0.3:
resolution: {integrity: sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==} resolution: {integrity: sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==}
@@ -7076,6 +7110,10 @@ packages:
resolution: {integrity: sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==} resolution: {integrity: sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==}
engines: {node: '>= 0.6'} engines: {node: '>= 0.6'}
event-target-shim@5.0.1:
resolution: {integrity: sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==}
engines: {node: '>=6'}
eventemitter2@6.4.9: eventemitter2@6.4.9:
resolution: {integrity: sha512-JEPTiaOt9f04oa6NOkc4aH+nVp5I3wEjpHbIPqfgCdD5v5bUzy7xQqwcVO2aDQgOWhI28da57HksMrzK9HlRxg==} resolution: {integrity: sha512-JEPTiaOt9f04oa6NOkc4aH+nVp5I3wEjpHbIPqfgCdD5v5bUzy7xQqwcVO2aDQgOWhI28da57HksMrzK9HlRxg==}
@@ -9085,6 +9123,9 @@ packages:
peberminta@0.9.0: peberminta@0.9.0:
resolution: {integrity: sha512-XIxfHpEuSJbITd1H3EeQwpcZbTLHc+VVr8ANI9t5sit565tsI4/xK3KWTUFE2e6QiangUkh3B0jihzmGnNrRsQ==} resolution: {integrity: sha512-XIxfHpEuSJbITd1H3EeQwpcZbTLHc+VVr8ANI9t5sit565tsI4/xK3KWTUFE2e6QiangUkh3B0jihzmGnNrRsQ==}
peek-stream@1.1.3:
resolution: {integrity: sha512-FhJ+YbOSBb9/rIl2ZeE/QHEsWn7PqNYt8ARAY3kIgNGOk13g9FGyIY6JIl/xB/3TFRVoTv5as0l11weORrTekA==}
pend@1.2.0: pend@1.2.0:
resolution: {integrity: sha512-F3asv42UuXchdzt+xXqfW1OGlVBe+mxa2mqI0pg5yAHZPvFmY3Y6drSf/GQ1A86WgWEN9Kzh/WrgKa6iGcHXLg==} resolution: {integrity: sha512-F3asv42UuXchdzt+xXqfW1OGlVBe+mxa2mqI0pg5yAHZPvFmY3Y6drSf/GQ1A86WgWEN9Kzh/WrgKa6iGcHXLg==}
@@ -9330,6 +9371,10 @@ packages:
process-warning@5.0.0: process-warning@5.0.0:
resolution: {integrity: sha512-a39t9ApHNx2L4+HBnQKqxxHNs1r7KF+Intd8Q/g1bUh6q0WIp9voPXJ/x0j+ZL45KF1pJd9+q2jLIRMfvEshkA==} resolution: {integrity: sha512-a39t9ApHNx2L4+HBnQKqxxHNs1r7KF+Intd8Q/g1bUh6q0WIp9voPXJ/x0j+ZL45KF1pJd9+q2jLIRMfvEshkA==}
process@0.11.10:
resolution: {integrity: sha512-cdGef/drWFoydD1JsMzuFf8100nZl+GT+yacc2bEced5f9Rjk4z+WtFUTBu9PhOi9j/jfmBPu0mMEY4wIdAF8A==}
engines: {node: '>= 0.6.0'}
prom-client@15.1.3: prom-client@15.1.3:
resolution: {integrity: sha512-6ZiOBfCywsD4k1BN9IX0uZhF+tJkV8q8llP64G5Hajs4JOeVLPCwpPVcpXy3BwYiUGgyJzsJJQeOIv7+hDSq8g==} resolution: {integrity: sha512-6ZiOBfCywsD4k1BN9IX0uZhF+tJkV8q8llP64G5Hajs4JOeVLPCwpPVcpXy3BwYiUGgyJzsJJQeOIv7+hDSq8g==}
engines: {node: ^16 || ^18 || >=20} engines: {node: ^16 || ^18 || >=20}
@@ -9625,6 +9670,10 @@ packages:
resolution: {integrity: sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==} resolution: {integrity: sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA==}
engines: {node: '>= 6'} engines: {node: '>= 6'}
readable-stream@4.7.0:
resolution: {integrity: sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg==}
engines: {node: ^12.22.0 || ^14.17.0 || >=16.0.0}
readdirp@3.6.0: readdirp@3.6.0:
resolution: {integrity: sha512-hOS089on8RduqdbhvQ5Z37A0ESjsqz6qnRcffsMU3495FuTdqSm+7bhJ29JvIOsBDEEnan5DPu9t3To9VRlMzA==} resolution: {integrity: sha512-hOS089on8RduqdbhvQ5Z37A0ESjsqz6qnRcffsMU3495FuTdqSm+7bhJ29JvIOsBDEEnan5DPu9t3To9VRlMzA==}
engines: {node: '>=8.10.0'} engines: {node: '>=8.10.0'}
@@ -10013,6 +10062,9 @@ packages:
stream-browserify@3.0.0: stream-browserify@3.0.0:
resolution: {integrity: sha512-H73RAHsVBapbim0tU2JwwOiXUj+fikfiaoYAKHF3VJfA0pe2BCzkhAHBlLG6REzE+2WNZcxOXjK7lkso+9euLA==} resolution: {integrity: sha512-H73RAHsVBapbim0tU2JwwOiXUj+fikfiaoYAKHF3VJfA0pe2BCzkhAHBlLG6REzE+2WNZcxOXjK7lkso+9euLA==}
stream-shift@1.0.3:
resolution: {integrity: sha512-76ORR0DO1o1hlKwTbi/DM3EXWGf3ZJYO8cXX5RJwnul2DEg2oyoZyjLNoQM8WsvZiFKCRfC1O0J7iCvie3RZmQ==}
strict-event-emitter-types@2.0.0: strict-event-emitter-types@2.0.0:
resolution: {integrity: sha512-Nk/brWYpD85WlOgzw5h173aci0Teyv8YdIAEtV+N88nDB0dLlazZyJMIsN6eo1/AR61l+p6CJTG1JIyFaoNEEA==} resolution: {integrity: sha512-Nk/brWYpD85WlOgzw5h173aci0Teyv8YdIAEtV+N88nDB0dLlazZyJMIsN6eo1/AR61l+p6CJTG1JIyFaoNEEA==}
@@ -10153,6 +10205,9 @@ packages:
resolution: {integrity: sha512-g9ljZiwki/LfxmQADO3dEY1CbpmXT5Hm2fJ+QaGKwSXUylMybePR7/67YW7jOrrvjEgL1Fmz5kzyAjWVWLlucg==} resolution: {integrity: sha512-g9ljZiwki/LfxmQADO3dEY1CbpmXT5Hm2fJ+QaGKwSXUylMybePR7/67YW7jOrrvjEgL1Fmz5kzyAjWVWLlucg==}
engines: {node: '>=6'} engines: {node: '>=6'}
tar-mini@0.2.0:
resolution: {integrity: sha512-+qfUHz700DWnRutdUsxRRVZ38G1Qr27OetwaMYTdg8hcPxf46U0S1Zf76dQMWRBmusOt2ZCK5kbIaiLkoGO7WQ==}
tar-stream@2.2.0: tar-stream@2.2.0:
resolution: {integrity: sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==} resolution: {integrity: sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ==}
engines: {node: '>=6'} engines: {node: '>=6'}
@@ -10196,6 +10251,9 @@ packages:
resolution: {integrity: sha512-nt6AMGKW1p/70DF/hGBdJB57B8Tspmbp5gfJ8ilhLnt7kkr2ye7hzD6NVG8GGErk2HWF34igrL2CXmNIkzKqKw==} resolution: {integrity: sha512-nt6AMGKW1p/70DF/hGBdJB57B8Tspmbp5gfJ8ilhLnt7kkr2ye7hzD6NVG8GGErk2HWF34igrL2CXmNIkzKqKw==}
engines: {node: '>=18'} engines: {node: '>=18'}
through2@2.0.5:
resolution: {integrity: sha512-/mrRod8xqpA+IHSLyGCQ2s8SPHiCDEeQJSep1jqLYeEUClOFG2Qsh+4FU6G9VeqpZnGW/Su8LQGc4YKni5rYSQ==}
tiny-invariant@1.3.3: tiny-invariant@1.3.3:
resolution: {integrity: sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==} resolution: {integrity: sha512-+FbBPE1o9QAYvviau/qC5SE3caw21q3xkvWKBtja5vgqOWIHHJ3ioaq1VPfn/Szqctz2bU/oYeKd9/z5BL+PVg==}
@@ -10601,6 +10659,9 @@ packages:
resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==} resolution: {integrity: sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg==}
engines: {node: '>= 0.8'} engines: {node: '>= 0.8'}
vite-plugin-compression2@2.5.3:
resolution: {integrity: sha512-ItPgqQWkcnBbVw7is9OKwiZ8v6+ju9rYROl5Lp6QfQDEx/d55AwJQb/KLpsQqsU9HoigYBsZ8tK6I02UwJNvEw==}
vite@8.0.5: vite@8.0.5:
resolution: {integrity: sha512-nmu43Qvq9UopTRfMx2jOYW5l16pb3iDC1JH6yMuPkpVbzK0k+L7dfsEDH4jRgYFmsg0sTAqkojoZgzLMlwHsCQ==} resolution: {integrity: sha512-nmu43Qvq9UopTRfMx2jOYW5l16pb3iDC1JH6yMuPkpVbzK0k+L7dfsEDH4jRgYFmsg0sTAqkojoZgzLMlwHsCQ==}
engines: {node: ^20.19.0 || >=22.12.0} engines: {node: ^20.19.0 || >=22.12.0}
@@ -12940,6 +13001,15 @@ snapshots:
'@fastify/busboy@3.1.1': {} '@fastify/busboy@3.1.1': {}
'@fastify/compress@9.0.0':
dependencies:
'@fastify/accept-negotiator': 2.0.1
fastify-plugin: 5.1.0
mime-db: 1.54.0
minipass: 7.1.3
peek-stream: 1.1.3
readable-stream: 4.7.0
'@fastify/cookie@11.0.2': '@fastify/cookie@11.0.2':
dependencies: dependencies:
cookie: 1.1.1 cookie: 1.1.1
@@ -14955,6 +15025,12 @@ snapshots:
'@rolldown/pluginutils@1.0.0-rc.7': {} '@rolldown/pluginutils@1.0.0-rc.7': {}
'@rollup/pluginutils@5.4.0':
dependencies:
'@types/estree': 1.0.9
estree-walker: 2.0.2
picomatch: 4.0.4
'@selderee/plugin-htmlparser2@0.11.0': '@selderee/plugin-htmlparser2@0.11.0':
dependencies: dependencies:
domhandler: 5.0.3 domhandler: 5.0.3
@@ -16328,6 +16404,10 @@ snapshots:
abbrev@5.0.0: {} abbrev@5.0.0: {}
abort-controller@3.0.0:
dependencies:
event-target-shim: 5.0.1
abstract-logging@2.0.1: {} abstract-logging@2.0.1: {}
accepts@1.3.8: accepts@1.3.8:
@@ -16358,17 +16438,17 @@ snapshots:
agent-base@7.1.4: {} agent-base@7.1.4: {}
ai-sdk-ollama@3.8.1(ai@6.0.134(zod@4.3.6))(zod@4.3.6): ai-sdk-ollama@3.8.1(ai@6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6))(zod@4.3.6):
dependencies: dependencies:
'@ai-sdk/provider': 3.0.8 '@ai-sdk/provider': 3.0.8
'@ai-sdk/provider-utils': 4.0.21(zod@4.3.6) '@ai-sdk/provider-utils': 4.0.21(zod@4.3.6)
ai: 6.0.134(zod@4.3.6) ai: 6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6)
jsonrepair: 3.13.3 jsonrepair: 3.13.3
ollama: 0.6.3 ollama: 0.6.3
transitivePeerDependencies: transitivePeerDependencies:
- zod - zod
ai@6.0.134(zod@4.3.6): ai@6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6):
dependencies: dependencies:
'@ai-sdk/gateway': 3.0.77(zod@4.3.6) '@ai-sdk/gateway': 3.0.77(zod@4.3.6)
'@ai-sdk/provider': 3.0.8 '@ai-sdk/provider': 3.0.8
@@ -16776,6 +16856,11 @@ snapshots:
base64-js: 1.5.1 base64-js: 1.5.1
ieee754: 1.2.1 ieee754: 1.2.1
buffer@6.0.3:
dependencies:
base64-js: 1.5.1
ieee754: 1.2.1
bullmq@5.76.10: bullmq@5.76.10:
dependencies: dependencies:
cron-parser: 4.9.0 cron-parser: 4.9.0
@@ -17529,6 +17614,13 @@ snapshots:
es-errors: 1.3.0 es-errors: 1.3.0
gopd: 1.2.0 gopd: 1.2.0
duplexify@3.7.1:
dependencies:
end-of-stream: 1.4.4
inherits: 2.0.4
readable-stream: 2.3.8
stream-shift: 1.0.3
ecdsa-sig-formatter@1.0.11: ecdsa-sig-formatter@1.0.11:
dependencies: dependencies:
safe-buffer: 5.2.1 safe-buffer: 5.2.1
@@ -17955,6 +18047,8 @@ snapshots:
estraverse@5.3.0: {} estraverse@5.3.0: {}
estree-walker@2.0.2: {}
estree-walker@3.0.3: estree-walker@3.0.3:
dependencies: dependencies:
'@types/estree': 1.0.9 '@types/estree': 1.0.9
@@ -17963,6 +18057,8 @@ snapshots:
etag@1.8.1: {} etag@1.8.1: {}
event-target-shim@5.0.1: {}
eventemitter2@6.4.9: {} eventemitter2@6.4.9: {}
eventemitter3@4.0.7: {} eventemitter3@4.0.7: {}
@@ -20226,6 +20322,12 @@ snapshots:
peberminta@0.9.0: {} peberminta@0.9.0: {}
peek-stream@1.1.3:
dependencies:
buffer-from: 1.1.2
duplexify: 3.7.1
through2: 2.0.5
pend@1.2.0: {} pend@1.2.0: {}
perfect-freehand@1.2.0: {} perfect-freehand@1.2.0: {}
@@ -20497,6 +20599,8 @@ snapshots:
process-warning@5.0.0: {} process-warning@5.0.0: {}
process@0.11.10: {}
prom-client@15.1.3: prom-client@15.1.3:
dependencies: dependencies:
'@opentelemetry/api': 1.9.0 '@opentelemetry/api': 1.9.0
@@ -20918,6 +21022,14 @@ snapshots:
string_decoder: 1.3.0 string_decoder: 1.3.0
util-deprecate: 1.0.2 util-deprecate: 1.0.2
readable-stream@4.7.0:
dependencies:
abort-controller: 3.0.0
buffer: 6.0.3
events: 3.3.0
process: 0.11.10
string_decoder: 1.3.0
readdirp@3.6.0: readdirp@3.6.0:
dependencies: dependencies:
picomatch: 2.3.2 picomatch: 2.3.2
@@ -21371,6 +21483,8 @@ snapshots:
inherits: 2.0.4 inherits: 2.0.4
readable-stream: 3.6.2 readable-stream: 3.6.2
stream-shift@1.0.3: {}
strict-event-emitter-types@2.0.0: {} strict-event-emitter-types@2.0.0: {}
string-length@4.0.2: string-length@4.0.2:
@@ -21531,6 +21645,8 @@ snapshots:
tapable@2.3.0: {} tapable@2.3.0: {}
tar-mini@0.2.0: {}
tar-stream@2.2.0: tar-stream@2.2.0:
dependencies: dependencies:
bl: 4.1.0 bl: 4.1.0
@@ -21580,6 +21696,11 @@ snapshots:
throttleit@2.1.0: {} throttleit@2.1.0: {}
through2@2.0.5:
dependencies:
readable-stream: 2.3.8
xtend: 4.0.2
tiny-invariant@1.3.3: {} tiny-invariant@1.3.3: {}
tinybench@2.9.0: {} tinybench@2.9.0: {}
@@ -21984,6 +22105,13 @@ snapshots:
vary@1.1.2: {} vary@1.1.2: {}
vite-plugin-compression2@2.5.3:
dependencies:
'@rollup/pluginutils': 5.4.0
tar-mini: 0.2.0
transitivePeerDependencies:
- rollup
vite@8.0.5(@types/node@20.19.43)(esbuild@0.28.0)(jiti@2.4.2)(less@4.2.0)(sugarss@5.0.1(postcss@8.5.14))(terser@5.39.0)(tsx@4.21.0)(yaml@2.8.3): vite@8.0.5(@types/node@20.19.43)(esbuild@0.28.0)(jiti@2.4.2)(less@4.2.0)(sugarss@5.0.1(postcss@8.5.14))(terser@5.39.0)(tsx@4.21.0)(yaml@2.8.3):
dependencies: dependencies:
lightningcss: 1.32.0 lightningcss: 1.32.0
@@ -22364,8 +22492,7 @@ snapshots:
xpath@0.0.34: {} xpath@0.0.34: {}
xtend@4.0.2: xtend@4.0.2: {}
optional: true
y-indexeddb@9.0.12(yjs@13.6.30(patch_hash=1ceeb66dba1f86545c98a3ff7f5152aff9b35caf409091cef9caedb5e65c8810)): y-indexeddb@9.0.12(yjs@13.6.30(patch_hash=1ceeb66dba1f86545c98a3ff7f5152aff9b35caf409091cef9caedb5e65c8810)):
dependencies: dependencies: