Merge develop into feat/git-sync — unify converter on the branch (#293/#326 step 6a)
Per maintainer directive (#119 comment): land the canonical converter on the git-sync branch so sync is tested on the real format, NOT a dead legacy copy. #119 itself stays FROZEN (не вливается) — this only merges develop in. Resolutions (all git-sync converter conflicts → develop; engine kept as-is): - Dropped the branch's legacy `packages/git-sync/src/lib/*` converter — the converter now lives solely in `@docmost/prosemirror-markdown` (#293); the engine (pull/push/stabilize/index) only switches its imports to the package (no logic change, verified by diff). - Removed the branch's orphaned converter tests + fixtures under `packages/git-sync/test/` (their coverage moved to the package's own test suite on develop); git-sync/test now holds engine tests only. - .gitignore / Dockerfile / test.yml / AGENTS.md: unioned — build/ ignored for every package; Dockerfile COPYs both prosemirror-markdown/build (mcp+git-sync runtime) and git-sync/build (git-sync's runtime consumer lands on this branch); CI builds prosemirror-markdown before git-sync/mcp. - pnpm-lock.yaml regenerated for the merged workspace. Branch adaptations to canon (server-side tests only — converter untouched, per the guardrail that converter fixes go to the package on develop, fixtures-first): - git-sync-converter-gate.spec.ts: heading textAlign and image width/height now round-trip via the canon trailing-comment forms (#9 `<!--attrs {...}-->`, #4 `<!--img {...}-->`) instead of the old HTML-tag forms — expectations flipped to the real canon output. RESIDUAL: canon #4 does not yet carry image `align` (documented as a known divergence; fix belongs in the package on develop). - schema-attribute-contract.spec.ts: the schema mirror moved from `@docmost/git-sync/lib/docmost-schema` to `@docmost/prosemirror-markdown`; import + jest source-mapper updated. Verified: prosemirror-markdown/git-sync/mcp build clean; git-sync corpus green; server `tsc --noEmit` 0; gate + schema-attribute-contract specs 32/32.
This commit is contained in:
agent_coder
@@ -72,10 +72,19 @@ jobs:
|
||||
- name: Build editor-ext
|
||||
run: pnpm --filter @docmost/editor-ext build
|
||||
|
||||
# @docmost/prosemirror-markdown is the shared converter (#293/#326); its
|
||||
# build/ is gitignored, and plain `pnpm -r test` does NOT honour nx
|
||||
# `dependsOn: ^build`, so its consumers (mcp `pretest: tsc`, git-sync vitest
|
||||
# typecheck) fail with TS2307 Cannot find module '@docmost/prosemirror-markdown'
|
||||
# unless it is built first. Build it BEFORE git-sync/mcp (which import it).
|
||||
- name: Build prosemirror-markdown
|
||||
run: pnpm --filter @docmost/prosemirror-markdown build
|
||||
|
||||
# git-sync and mcp are no longer committed in built form (build/ is
|
||||
# gitignored), so CI must compile them: the server resolves both via their
|
||||
# built build/index.js. The server pretest also builds them, but building
|
||||
# here keeps it explicit and independent of pnpm lifecycle ordering.
|
||||
# built build/index.js (git-sync's runtime consumer lands on this branch,
|
||||
# #119). The server pretest also builds them, but building here keeps it
|
||||
# explicit and independent of pnpm lifecycle ordering.
|
||||
- name: Build git-sync and mcp
|
||||
run: pnpm --filter @docmost/git-sync build && pnpm --filter @docmost/mcp build
|
||||
|
||||
|
||||
+6
-2
@@ -6,10 +6,14 @@ data
|
||||
/dist
|
||||
/node_modules
|
||||
# workspace package node_modules (pnpm symlinks — never commit; they bake
|
||||
# machine-local store paths) and the git-sync compiled output (built in CI/Docker
|
||||
# via `pnpm build`, never committed, so src/ and prod can never silently diverge).
|
||||
# machine-local store paths).
|
||||
packages/*/node_modules/
|
||||
|
||||
# Compiled package output: build/ is gitignored for every workspace package
|
||||
# (built in CI/Docker via `pnpm build`, never committed, so src/ and prod can
|
||||
# never silently diverge). Private packages are rebuilt at deploy.
|
||||
packages/git-sync/build/
|
||||
packages/prosemirror-markdown/build/
|
||||
packages/mcp/build/
|
||||
|
||||
# Logs
|
||||
|
||||
@@ -200,8 +200,9 @@ pnpm workspace (`pnpm@10.4.0`) orchestrated by **Nx**. Five workspace packages:
|
||||
| `apps/server` | `server` | NestJS 11 + Fastify, Kysely (Postgres), Redis | Backend API, collaboration, AI |
|
||||
| `apps/client` | `client` | React 18 + Vite + Mantine 8 + TanStack Query + Jotai | SPA frontend |
|
||||
| `packages/editor-ext` | `@docmost/editor-ext` | Tiptap/ProseMirror | Shared Tiptap node/mark extensions, imported by both the client and the server |
|
||||
| `packages/mcp` | `@docmost/mcp` | MCP SDK, Tiptap, Yjs | Standalone MCP server, also bundled into the server at `/mcp`. Does **not** import `editor-ext` — it keeps its own vendored mirror of the schema in `packages/mcp/src/lib/` |
|
||||
| `packages/git-sync` | `@docmost/git-sync` | Tiptap/ProseMirror, Yjs, git | Pure ProseMirror↔Markdown converter plus the two-way Docmost↔git Markdown sync engine. Bundled into the server (loaded over the ESM bridge), built in CI and the Dockerfile. Does **not** import `editor-ext` — it keeps its own vendored mirror of the document schema (kept in sync with `editor-ext`). |
|
||||
| `packages/mcp` | `@docmost/mcp` | MCP SDK, Tiptap, Yjs | Standalone MCP server, also bundled into the server at `/mcp`. Consumes the shared converter/schema from `@docmost/prosemirror-markdown` (#293) — it no longer carries its own vendored converter/schema copy |
|
||||
| `packages/prosemirror-markdown` | `@docmost/prosemirror-markdown` | Tiptap, marked, jsdom | The single, canonical ProseMirror↔Markdown converter + Docmost schema mirror (#293). Consumed by `mcp` and `git-sync`; there is exactly ONE copy of the converter now |
|
||||
| `packages/git-sync` | `@docmost/git-sync` | Tiptap/ProseMirror, Yjs, git | The two-way Docmost↔git Markdown sync **engine** (vault layout, git orchestration, reconcile). Consumes the ProseMirror↔Markdown converter from `@docmost/prosemirror-markdown` (#293) — no longer carries its own converter copy. Bundled into the server (loaded over the ESM bridge), built in CI and the Dockerfile. |
|
||||
|
||||
`build` targets are Nx-cached and dependency-ordered (`dependsOn: ["^build"]`), so `editor-ext` builds before the apps. `nx.json` sets `affected.defaultBase: main`.
|
||||
|
||||
@@ -291,7 +292,7 @@ Two routes are mounted **outside** the `/api` prefix at the root, as raw Fastify
|
||||
### Client structure
|
||||
Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirrors the server domains: `page`, `space`, `comment`, `ai-chat`, `editor`, …). Conventions:
|
||||
- **TanStack Query** for server state (one `queries/` file per feature), **Jotai** atoms for local/shared UI state, **Mantine 8** + CSS modules (`*.module.css`) + `postcss-preset-mantine` for UI.
|
||||
- The editor is Tiptap; shared node/mark extensions live in `packages/editor-ext` and are imported by **both the client and the server** (collaboration, import/export) — editor schema changes often need to be made in `editor-ext`, not just the client. Note neither `packages/mcp` nor `packages/git-sync` depends on `editor-ext`; each carries its own mirrored copy of the schema. There are now **three** independent copies (`editor-ext` is canonical, plus `packages/mcp` and `packages/git-sync`), so keep all three in sync manually when the document schema changes.
|
||||
- The editor is Tiptap; shared node/mark extensions live in `packages/editor-ext` and are imported by **both the client and the server** (collaboration, import/export) — editor schema changes often need to be made in `editor-ext`, not just the client. The ProseMirror↔Markdown converter and its Docmost schema mirror now live in a SINGLE package, `@docmost/prosemirror-markdown` (#293), consumed by both `mcp` and `git-sync` — do NOT reintroduce a per-package copy. `editor-ext` is the upstream source of the Tiptap schema; the package's `docmost-schema.ts` mirrors it and a serializer-contract test (`packages/prosemirror-markdown/test/serializer-contract.test.ts`) guards the boundary (every schema node must have a converter case), so a drift surfaces as a failing test rather than silent divergence.
|
||||
- API access goes through `apps/client/src/lib/api-client.ts` (axios). The `@` alias maps to `apps/client/src`.
|
||||
- Runtime config is injected at build time by `vite.config.ts` via `define` (`APP_URL`, `COLLAB_URL`, `APP_VERSION`, …) — these come from the root `.env`, not from `import.meta.env`.
|
||||
|
||||
@@ -302,6 +303,7 @@ Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirro
|
||||
- The version string shown in the UI comes from `APP_VERSION` (CI/Docker) or `git describe --tags --always` (local), resolved in `vite.config.ts` — not from `package.json`.
|
||||
- Server TS config is permissive (`noImplicitAny: false`, `strictNullChecks: false`, `no-explicit-any` lint disabled). Follow the existing relaxed style rather than tightening types broadly.
|
||||
- Dependency versions are heavily pinned via `pnpm.overrides` and `pnpm.patchedDependencies` (`scimmy`, `yjs`) in the root `package.json`. Don't bump pinned/patched deps casually; the patches and overrides exist for compatibility/security reasons.
|
||||
- **Adding/renaming/removing an MCP tool requires updating `SERVER_INSTRUCTIONS`** in `packages/mcp/src/index.ts` — the intent-routing guide MCP clients receive on initialize. This applies both to inline `server.registerTool(...)` calls in `index.ts` and to specs in `packages/mcp/src/tool-specs.ts`. Enforced by `packages/mcp/test/unit/server-instructions.test.mjs`, which fails when a registered tool is not mentioned in the guide (deliberate opt-outs go into its `EXCEPTIONS` list). `packages/mcp/build/` is gitignored and rebuilt in CI/Docker via `pnpm build` (same convention as `git-sync`/`prosemirror-markdown`) — never commit it; rebuild locally after editing to run the tests.
|
||||
|
||||
## CI / release
|
||||
|
||||
|
||||
+7
-1
@@ -39,12 +39,18 @@ COPY --from=builder /app/packages/editor-ext/dist /app/packages/editor-ext/dist
|
||||
COPY --from=builder /app/packages/editor-ext/package.json /app/packages/editor-ext/package.json
|
||||
COPY --from=builder /app/packages/mcp/build /app/packages/mcp/build
|
||||
COPY --from=builder /app/packages/mcp/package.json /app/packages/mcp/package.json
|
||||
# @docmost/prosemirror-markdown is the shared converter (#293/#326). Both mcp and
|
||||
# git-sync depend on it (workspace:*) and load it at runtime, so the built package +
|
||||
# its manifest must be shipped or the prod install resolves a broken workspace
|
||||
# symlink and every consumer dies with ERR_MODULE_NOT_FOUND.
|
||||
COPY --from=builder /app/packages/prosemirror-markdown/build /app/packages/prosemirror-markdown/build
|
||||
COPY --from=builder /app/packages/prosemirror-markdown/package.json /app/packages/prosemirror-markdown/package.json
|
||||
# git-sync: the server loads @docmost/git-sync at runtime via the loader
|
||||
# (git-sync.loader.ts), which deliberately does NOT `require()` it — the package is
|
||||
# ESM-only, so the loader uses `require.resolve` + a dynamic `import()`. Without
|
||||
# these copied build artifacts that resolve/import fails and the server crashes on
|
||||
# first use. Built fresh by the builder's `pnpm build` (nx builds the package's tsc
|
||||
# `build` target).
|
||||
# `build` target). This branch (#119) is where git-sync gains its runtime consumer.
|
||||
COPY --from=builder /app/packages/git-sync/build /app/packages/git-sync/build
|
||||
COPY --from=builder /app/packages/git-sync/package.json /app/packages/git-sync/package.json
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ roles:
|
||||
Read the whole text first. Think at the level of sections and paragraphs, not sentences.
|
||||
|
||||
HOW TO LEAVE COMMENTS
|
||||
You don't edit the text yourself. For each note, select the relevant span via the MCP tool and leave a comment. Open the comment with the label `[Structure]`. Then: state the problem briefly, propose a concrete fix (move, merge, cut, add, reorder, strengthen the lead/headline), and explain why if it isn't obvious. Tag severity:
|
||||
You don't edit the text yourself. For each note, select the relevant span via the MCP tool and leave a comment. State the problem briefly, propose a concrete fix (move, merge, cut, add, reorder, strengthen the lead/headline), and explain why if it isn't obvious. Tag severity:
|
||||
- [Critical] — broken logic, the text doesn't deliver what the headline promises, a key link in the argument is missing.
|
||||
- [Major] — weak structure, a noticeable gap or redundancy, a sagging lead/headline.
|
||||
- [Minor] — an optional improvement to framing or flow.
|
||||
@@ -87,7 +87,7 @@ roles:
|
||||
- Don't rewrite the text yourself or impose your own voice. Your job is to make the author's voice livelier, not to replace it.
|
||||
|
||||
HOW TO LEAVE COMMENTS
|
||||
You don't edit the text directly. For each note, select the span via the MCP tool and leave a comment. Open the comment with the label `[Style]`. Give a concrete rephrasing, not "revise", and attach it to the comment as a suggested replacement (the `suggestedText` parameter): the exact new text for the selected fragment, plain text with no markup — the author applies it with one click. The selected fragment must occur exactly once in the text; if it isn't unique, extend the selection with surrounding context. Tag severity:
|
||||
You don't edit the text directly. For each note, select the span via the MCP tool and leave a comment. Give a concrete rephrasing, not "revise", and attach it to the comment as a suggested replacement (the `suggestedText` parameter): the exact new text for the selected fragment, plain text with no markup — the author applies it with one click. The selected fragment must occur exactly once in the text; if it isn't unique, extend the selection with surrounding context. Tag severity:
|
||||
- [Critical] — the sentence is unclear or distorts the meaning.
|
||||
- [Major] — an obvious LLM cliché, heavy bureaucratese, filler that breaks the reading.
|
||||
- [Minor] — a stylistic improvement to taste.
|
||||
@@ -128,7 +128,7 @@ roles:
|
||||
- Don't fabricate confirmations. If you can't verify, honestly mark [Unverified] or [Unverifiable].
|
||||
|
||||
HOW TO LEAVE COMMENTS
|
||||
You don't edit the text directly. For each problem claim (an error, a doubt, an unverifiable statement), select the span via the MCP tool and leave a comment; leave no comment on correct facts. Open the comment with the label `[Facts]`, then the verdict, the correction (if any), and the source. For an [Incorrect] verdict, ALWAYS attach the ready correction as a suggested replacement (the `suggestedText` parameter): since you found the correct value in the sources, propose the ready fix right away instead of merely describing the error. The replacement is the exact new text for the selected fragment, plain text with no markup; the author applies it with one click instead of retyping the fragment. The selected fragment must occur exactly once in the text; if it isn't unique, extend the selection with surrounding context. Do not attach a replacement to [Unverified], [Unverifiable], or [Opinion] verdicts. Tag severity:
|
||||
You don't edit the text directly. For each problem claim (an error, a doubt, an unverifiable statement), select the span via the MCP tool and leave a comment; leave no comment on correct facts. Give the verdict, the correction (if any), and the source. For an [Incorrect] verdict, ALWAYS attach the ready correction as a suggested replacement (the `suggestedText` parameter): since you found the correct value in the sources, propose the ready fix right away instead of merely describing the error. The replacement is the exact new text for the selected fragment, plain text with no markup; the author applies it with one click instead of retyping the fragment. The selected fragment must occur exactly once in the text; if it isn't unique, extend the selection with surrounding context. When a figure, name, term, or version to check recurs across the page, use search_in_page to find every occurrence in one call first, then place a targeted comment per hit instead of reading block by block. Do not attach a replacement to [Unverified], [Unverifiable], or [Opinion] verdicts. Tag severity:
|
||||
- [Critical] — a factual error, especially in numbers, names, or quotes, or a claim that risks misinformation.
|
||||
- [Major] — a doubtful or unconfirmed claim that needs a source.
|
||||
- [Minor] — a small correction, or false precision worth rounding or confirming.
|
||||
@@ -168,8 +168,11 @@ roles:
|
||||
- Don't verify facts — that's the Fact-checker.
|
||||
- Don't make substantive changes. Edits are minimal and mechanical.
|
||||
|
||||
HOW TO WORK
|
||||
Go through the whole text from start to finish in a single pass. Flag EVERY violation, including all repeat occurrences of the same error and minor items tagged [Minor] — don't stop at the first few or the most conspicuous. Don't summarize instead of marking up: until you've reached the end of the document, the job isn't done. One run covers the whole text, not just "the most important". For a systematic issue that recurs — straight quotes, a hyphen used as a dash, an inconsistent unit or spelling — use search_in_page to list every occurrence in one call first, then leave a targeted comment (with its replacement) on each hit, instead of scanning block by block.
|
||||
|
||||
HOW TO LEAVE COMMENTS
|
||||
You don't edit the text directly. For each fix, select the span via the MCP tool and leave a comment with the concrete correction. Attach a suggested replacement to every fix (the `suggestedText` parameter): the exact corrected text for the selected fragment, plain text with no markup — the author applies it with one click. The selected fragment must occur exactly once in the text; if it isn't unique, extend the selection with surrounding context. Do NOT leave summary notes like "throughout, replace X with Y" or "make the units/quotes/spelling consistent": such a comment can't be applied with a button. If the same error occurs in several places, walk EVERY occurrence and leave a separate targeted comment with its own replacement on each — ten targeted fixes instead of one blanket note. The only exception is a note that genuinely cannot be expressed as a replacement of a concrete fragment; leave those rare cases as an ordinary comment without a replacement. Open the comment with the label `[Copyedit]`. Tag severity:
|
||||
You don't edit the text directly. For each fix, select the span via the MCP tool and leave a comment with the concrete correction. Attach a suggested replacement to every fix (the `suggestedText` parameter): the exact corrected text for the selected fragment, plain text with no markup — the author applies it with one click. The selected fragment must occur exactly once in the text; if it isn't unique, extend the selection with surrounding context. Do NOT leave summary notes like "throughout, replace X with Y" or "make the units/quotes/spelling consistent": such a comment can't be applied with a button. If the same error occurs in several places, walk EVERY occurrence and leave a separate targeted comment with its own replacement on each — ten targeted fixes instead of one blanket note. The only exception is a note that genuinely cannot be expressed as a replacement of a concrete fragment; leave those rare cases as an ordinary comment without a replacement. Tag severity:
|
||||
- [Critical] — a grammar/spelling error or typo visible to the reader.
|
||||
- [Major] — a consistency or typography break (wrong quotes, hyphen for a dash, missing serial comma where the rest of the text has it).
|
||||
- [Minor] — optional polish.
|
||||
|
||||
@@ -34,7 +34,7 @@ roles:
|
||||
Сначала прочитай весь текст целиком. Думай на уровне разделов и абзацев, а не предложений.
|
||||
|
||||
КАК ОСТАВЛЯТЬ ЗАМЕЧАНИЯ
|
||||
Ты не редактируешь текст сам. Для каждого замечания через MCP-инструмент выдели соответствующий фрагмент и оставь к нему комментарий. Начинай комментарий с метки `[Структура]`. Дальше: коротко назови проблему, предложи конкретное решение (перенести, объединить, вырезать, добавить, переставить, усилить лид/заголовок) и при необходимости поясни, почему. Помечай важность:
|
||||
Ты не редактируешь текст сам. Для каждого замечания через MCP-инструмент выдели соответствующий фрагмент и оставь к нему комментарий. Коротко назови проблему, предложи конкретное решение (перенести, объединить, вырезать, добавить, переставить, усилить лид/заголовок) и при необходимости поясни, почему. Помечай важность:
|
||||
- [Критично] — сломана логика, текст не отвечает на заявленное в заголовке, отсутствует ключевое звено аргумента.
|
||||
- [Существенно] — слабая структура, заметный пробел или избыточность, провисающий лид/заголовок.
|
||||
- [Незначительно] — улучшение подачи или стройности, не обязательное.
|
||||
@@ -87,7 +87,7 @@ roles:
|
||||
- Не переписываешь текст сам и не навязываешь свой голос. Твоя задача — сделать авторскую интонацию живее, а не заменить собой.
|
||||
|
||||
КАК ОСТАВЛЯТЬ ЗАМЕЧАНИЯ
|
||||
Ты не редактируешь текст напрямую. Для каждого замечания через MCP-инструмент выдели фрагмент и оставь к нему комментарий. Начинай комментарий с метки `[Стиль]`. Давай конкретный вариант переформулировки, а не «переделать», и прикладывай его к комментарию как предложение-замену (параметр `suggestedText`): точный новый текст взамен выделенного фрагмента, обычным текстом без разметки — автор применит его одной кнопкой. Выделенный фрагмент должен встречаться в тексте ровно один раз; если он не уникален, расширь выделение контекстом. Помечай важность:
|
||||
Ты не редактируешь текст напрямую. Для каждого замечания через MCP-инструмент выдели фрагмент и оставь к нему комментарий. Давай конкретный вариант переформулировки, а не «переделать», и прикладывай его к комментарию как предложение-замену (параметр `suggestedText`): точный новый текст взамен выделенного фрагмента, обычным текстом без разметки — автор применит его одной кнопкой. Выделенный фрагмент должен встречаться в тексте ровно один раз; если он не уникален, расширь выделение контекстом. Помечай важность:
|
||||
- [Критично] — предложение непонятно или искажает смысл.
|
||||
- [Существенно] — явный штамп LLM, заметный канцелярит, вода, ломающая чтение.
|
||||
- [Незначительно] — стилистическое улучшение на вкус.
|
||||
@@ -128,7 +128,7 @@ roles:
|
||||
- Не выдумываешь подтверждения. Если не можешь проверить — честно ставь [Не проверено] или [Непроверяемо].
|
||||
|
||||
КАК ОСТАВЛЯТЬ ЗАМЕЧАНИЯ
|
||||
Ты не редактируешь текст напрямую. Для каждого проблемного утверждения (ошибка, сомнение, непроверяемость) через MCP-инструмент выдели фрагмент и оставь комментарий; на верные факты комментарии не оставляй. Начинай комментарий с метки `[Факты]`, затем вердикт, исправление (если нужно) и источник. К вердикту [Неверно] всегда прикладывай готовое исправление как предложение-замену (параметр `suggestedText`): раз ты нашёл по источникам верное значение — сразу предлагай готовую правку, а не только описывай ошибку. Замена — это точный новый текст взамен выделенного фрагмента, обычным текстом без разметки; автор применит её одной кнопкой, не переписывая фрагмент вручную. Выделенный фрагмент должен встречаться в тексте ровно один раз; если он не уникален, расширь выделение контекстом. К вердиктам [Не проверено], [Непроверяемо] и [Это мнение] замену не прикладывай. Помечай важность:
|
||||
Ты не редактируешь текст напрямую. Для каждого проблемного утверждения (ошибка, сомнение, непроверяемость) через MCP-инструмент выдели фрагмент и оставь комментарий; на верные факты комментарии не оставляй. В комментарии дай вердикт, исправление (если нужно) и источник. К вердикту [Неверно] всегда прикладывай готовое исправление как предложение-замену (параметр `suggestedText`): раз ты нашёл по источникам верное значение — сразу предлагай готовую правку, а не только описывай ошибку. Замена — это точный новый текст взамен выделенного фрагмента, обычным текстом без разметки; автор применит её одной кнопкой, не переписывая фрагмент вручную. Выделенный фрагмент должен встречаться в тексте ровно один раз; если он не уникален, расширь выделение контекстом. Когда проверяемая цифра, имя, термин или версия встречается по тексту несколько раз, сначала одним вызовом search_in_page найди все вхождения, а затем ставь целевой комментарий на каждое — не читая страницу поблочно. К вердиктам [Не проверено], [Непроверяемо] и [Это мнение] замену не прикладывай. Помечай важность:
|
||||
- [Критично] — фактическая ошибка, особенно в числах, именах, цитатах, или утверждение с риском дезинформации.
|
||||
- [Существенно] — сомнительное или непроверенное утверждение, требующее источника.
|
||||
- [Незначительно] — мелкое уточнение, псевдоточность, которую стоит округлить или подтвердить.
|
||||
@@ -169,8 +169,11 @@ roles:
|
||||
- Не проверяешь достоверность фактов — это фактчекер.
|
||||
- Не вносишь содержательных изменений. Правки — минимальные и механические.
|
||||
|
||||
КАК РАБОТАТЬ
|
||||
Пройди весь текст от начала до конца за один проход. Помечай КАЖДОЕ нарушение, включая все повторные вхождения одной и той же ошибки и мелочи с меткой [Незначительно], — не ограничивайся первыми несколькими или самыми заметными. Не подводи итог вместо разбора: пока не дошёл до конца документа, работа не закончена. Один прогон покрывает весь текст, а не «самое важное». Для систематической ошибки, которая повторяется — прямые кавычки, «е» вместо «ё», дефис вместо тире, неединообразная единица или написание, — сначала одним вызовом search_in_page получи все вхождения, а затем оставь на каждом целевой комментарий с заменой, вместо поблочного просмотра.
|
||||
|
||||
КАК ОСТАВЛЯТЬ ЗАМЕЧАНИЯ
|
||||
Ты не редактируешь текст напрямую. Для каждой правки через MCP-инструмент выдели фрагмент и оставь комментарий с конкретным исправлением. К каждой правке прикладывай предложение-замену (параметр `suggestedText`): точный исправленный текст взамен выделенного фрагмента, обычным текстом без разметки — автор применит его одной кнопкой. Выделенный фрагмент должен встречаться в тексте ровно один раз; если он не уникален, расширь выделение контекстом. НЕ оставляй сводных замечаний вида «во всём тексте заменить X на Y» или «привести единицы/кавычки/написание к единообразию»: такой комментарий нельзя применить кнопкой. Если одна и та же ошибка встречается в нескольких местах, обойди КАЖДОЕ вхождение и оставь на нём отдельный целевой комментарий со своей заменой — десять точечных правок вместо одной общей. Единственное исключение — замечание, которое в принципе невозможно выразить заменой конкретного фрагмента; такие редкие случаи оставляй обычным комментарием без замены. Начинай комментарий с метки `[Корректура]`. Помечай важность:
|
||||
Ты не редактируешь текст напрямую. Для каждой правки через MCP-инструмент выдели фрагмент и оставь комментарий с конкретным исправлением. К каждой правке прикладывай предложение-замену (параметр `suggestedText`): точный исправленный текст взамен выделенного фрагмента, обычным текстом без разметки — автор применит его одной кнопкой. Выделенный фрагмент должен встречаться в тексте ровно один раз; если он не уникален, расширь выделение контекстом. НЕ оставляй сводных замечаний вида «во всём тексте заменить X на Y» или «привести единицы/кавычки/написание к единообразию»: такой комментарий нельзя применить кнопкой. Если одна и та же ошибка встречается в нескольких местах, обойди КАЖДОЕ вхождение и оставь на нём отдельный целевой комментарий со своей заменой — десять точечных правок вместо одной общей. Единственное исключение — замечание, которое в принципе невозможно выразить заменой конкретного фрагмента; такие редкие случаи оставляй обычным комментарием без замены. Помечай важность:
|
||||
- [Критично] — грамматическая/орфографическая ошибка или опечатка, видимая читателю.
|
||||
- [Существенно] — нарушение единообразия или типографики (неверные кавычки, дефис вместо тире, отсутствие неразрывного пробела в критичном месте).
|
||||
- [Незначительно] — необязательная шлифовка.
|
||||
|
||||
@@ -12,13 +12,13 @@ bundles:
|
||||
- en
|
||||
roles:
|
||||
- slug: structural-editor
|
||||
version: 3
|
||||
- slug: line-editor
|
||||
version: 3
|
||||
- slug: fact-checker
|
||||
version: 4
|
||||
- slug: line-editor
|
||||
version: 4
|
||||
- slug: fact-checker
|
||||
version: 6
|
||||
- slug: proofreader
|
||||
version: 5
|
||||
version: 8
|
||||
- slug: narrator
|
||||
version: 2
|
||||
- id: research
|
||||
|
||||
@@ -1,26 +1,26 @@
|
||||
{
|
||||
"fact-checker": {
|
||||
"version": 4,
|
||||
"hash": "9160ead04d86aaa5dc7a51dd7e971c272ce0ca97cb24bf2b6ee5779deb1b19c0"
|
||||
"version": 6,
|
||||
"hash": "6bb22a9e5a5079b5cb287b5b26addbd36b9afeb7c9508287dcad9343fc53d685"
|
||||
},
|
||||
"line-editor": {
|
||||
"version": 3,
|
||||
"hash": "7f200863080799b08d5af5d1648befa0843cc5db79bb994b07baa5ad12df5123"
|
||||
"version": 4,
|
||||
"hash": "890d10f3f0bd7f2b2cfcc94463634221c557a3140e3794721748dc8d99979780"
|
||||
},
|
||||
"narrator": {
|
||||
"version": 2,
|
||||
"hash": "66fe653003b4f63ef3c3a5c5c48552fe47daeefffc16907c37c35f0e8da98851"
|
||||
},
|
||||
"proofreader": {
|
||||
"version": 5,
|
||||
"hash": "40af08c51e03c24b1986ac5cd679434e023afe31a819748966ccb0c6c62f0401"
|
||||
"version": 8,
|
||||
"hash": "cef39fed321779631ddd1077fcba53399adf0e48b301df281c71eb042610900d"
|
||||
},
|
||||
"researcher": {
|
||||
"version": 1,
|
||||
"hash": "853658fda43ddbe0a4d08f2c6e50b5116d29a2e9ccd7f46e173e65920d8f6ace"
|
||||
},
|
||||
"structural-editor": {
|
||||
"version": 3,
|
||||
"hash": "f6936e4c152c1b78980e74045658d87743f26f900c12f61fd7a45c6a0ec19425"
|
||||
"version": 4,
|
||||
"hash": "89100e0a00b88daa0d2118fd98ec1c27d06b972bfc6ec58b705553a4daed85df"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -40,6 +40,7 @@
|
||||
"axios": "1.16.0",
|
||||
"blueimp-load-image": "5.16.0",
|
||||
"clsx": "2.1.1",
|
||||
"diff": "8.0.3",
|
||||
"dompurify": "3.4.1",
|
||||
"file-saver": "2.0.5",
|
||||
"highlightjs-sap-abap": "0.3.0",
|
||||
@@ -81,6 +82,7 @@
|
||||
"@types/react": "18.3.12",
|
||||
"@types/react-dom": "18.3.1",
|
||||
"@vitejs/plugin-react": "6.0.1",
|
||||
"@vitest/coverage-v8": "4.1.6",
|
||||
"eslint": "9.28.0",
|
||||
"eslint-plugin-react": "7.37.5",
|
||||
"eslint-plugin-react-hooks": "7.0.1",
|
||||
|
||||
@@ -1390,5 +1390,8 @@
|
||||
"Applied": "Applied",
|
||||
"Suggestion applied": "Suggestion applied",
|
||||
"Failed to apply suggestion": "Failed to apply suggestion",
|
||||
"The commented text changed since this suggestion was made; it was not applied.": "The commented text changed since this suggestion was made; it was not applied."
|
||||
"The commented text changed since this suggestion was made; it was not applied.": "The commented text changed since this suggestion was made; it was not applied.",
|
||||
"Dismiss": "Dismiss",
|
||||
"Suggestion dismissed": "Suggestion dismissed",
|
||||
"Failed to dismiss suggestion": "Failed to dismiss suggestion"
|
||||
}
|
||||
|
||||
@@ -1246,5 +1246,8 @@
|
||||
"Applied": "Применено",
|
||||
"Suggestion applied": "Предложение применено",
|
||||
"Failed to apply suggestion": "Не удалось применить предложение",
|
||||
"The commented text changed since this suggestion was made; it was not applied.": "Прокомментированный текст изменился после создания предложения; оно не было применено."
|
||||
"The commented text changed since this suggestion was made; it was not applied.": "Прокомментированный текст изменился после создания предложения; оно не было применено.",
|
||||
"Dismiss": "Не применять",
|
||||
"Suggestion dismissed": "Предложение отклонено",
|
||||
"Failed to dismiss suggestion": "Не удалось отклонить предложение"
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, it, expect, beforeEach, vi } from "vitest";
|
||||
import { render, screen, fireEvent, act } from "@testing-library/react";
|
||||
import { render, screen, fireEvent, act, cleanup } from "@testing-library/react";
|
||||
import { MantineProvider } from "@mantine/core";
|
||||
|
||||
// Shared, hoisted mock state so the @ai-sdk/react and "ai" module mocks (hoisted
|
||||
@@ -140,3 +140,91 @@ describe("ChatThread — send now (#198)", () => {
|
||||
expect(prep({ messages: [], body: {} }).body.interrupted).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
// The turn-end decision lives in the `onFinish` handler: given the terminal
|
||||
// outcome of a turn (`isAbort` / `isDisconnect` / `isError`, or none = clean),
|
||||
// it decides whether to CONTINUE (flush the next queued message) or END (leave
|
||||
// the queue intact for the user), and which stop notice — if any — to show.
|
||||
// `sendNow` is exercised above; these tests pin down the plain outcomes.
|
||||
describe("ChatThread — turn-end decision (onFinish)", () => {
|
||||
beforeEach(() => {
|
||||
h.state.status = "streaming";
|
||||
h.state.onFinish = null;
|
||||
h.state.sendMessage.mockClear();
|
||||
h.state.stop.mockClear();
|
||||
h.state.transport = null;
|
||||
});
|
||||
|
||||
// Drive a fresh onFinish with the given terminal flags after queueing a
|
||||
// message, and report both what the parent was told and whether the queue was
|
||||
// flushed (a resend to the sendMessage spy).
|
||||
function finishWith(flags: {
|
||||
isAbort?: boolean;
|
||||
isDisconnect?: boolean;
|
||||
isError?: boolean;
|
||||
}) {
|
||||
// Tear down any prior render so the loop-driven "every outcome" case does
|
||||
// not leave duplicate queue buttons in the DOM.
|
||||
cleanup();
|
||||
h.state.sendMessage.mockClear();
|
||||
const { onTurnFinished } = renderThread();
|
||||
// Populate the queue while the turn is streaming.
|
||||
fireEvent.click(screen.getByTestId("queue-btn"));
|
||||
act(() => {
|
||||
h.state.onFinish?.({
|
||||
message: { id: "a", role: "assistant", parts: [] },
|
||||
isAbort: false,
|
||||
isDisconnect: false,
|
||||
isError: false,
|
||||
...flags,
|
||||
});
|
||||
});
|
||||
return { onTurnFinished };
|
||||
}
|
||||
|
||||
it("CONTINUES — flushes the next queued message on a clean finish", () => {
|
||||
finishWith({});
|
||||
// Clean finish (no terminal flag): the queued message is auto-sent.
|
||||
expect(h.state.sendMessage).toHaveBeenCalledWith({ text: "queued text" });
|
||||
// A clean finish shows no stop notice.
|
||||
expect(screen.queryByText("Response stopped.")).toBeNull();
|
||||
});
|
||||
|
||||
it("ENDS — keeps the queue intact on a user abort and shows the stopped notice", () => {
|
||||
finishWith({ isAbort: true });
|
||||
// A plain Stop (not the sendNow interrupt path) must NOT auto-resend: the
|
||||
// queue is preserved for the user to decide.
|
||||
expect(h.state.sendMessage).not.toHaveBeenCalled();
|
||||
expect(screen.getByText("Response stopped.")).toBeTruthy();
|
||||
});
|
||||
|
||||
it("ENDS — keeps the queue intact on a disconnect and shows the connection-lost notice", () => {
|
||||
finishWith({ isDisconnect: true });
|
||||
expect(h.state.sendMessage).not.toHaveBeenCalled();
|
||||
expect(
|
||||
screen.getByText("Connection lost — the answer was interrupted."),
|
||||
).toBeTruthy();
|
||||
});
|
||||
|
||||
it("ENDS — keeps the queue intact on a stream error (no auto-retry, no stopped notice)", () => {
|
||||
finishWith({ isError: true });
|
||||
// Blindly retrying after a failure would be wrong; the queue is left alone.
|
||||
expect(h.state.sendMessage).not.toHaveBeenCalled();
|
||||
// isError clears the neutral notice (the error banner covers this case).
|
||||
expect(screen.queryByText("Response stopped.")).toBeNull();
|
||||
});
|
||||
|
||||
it("notifies the parent on EVERY terminal outcome", () => {
|
||||
// The chat-list refresh / new-chat id adoption must run on success and on
|
||||
// every failure path alike.
|
||||
for (const flags of [
|
||||
{},
|
||||
{ isAbort: true },
|
||||
{ isDisconnect: true },
|
||||
{ isError: true },
|
||||
]) {
|
||||
const { onTurnFinished } = finishWith(flags);
|
||||
expect(onTurnFinished).toHaveBeenCalled();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -8,6 +8,7 @@ import { IComment } from "@/features/comment/types/comment.types";
|
||||
// The comment mutation hooks reach out to react-query/network — stub them so the
|
||||
// component renders in isolation. We only assert the AI-badge rendering branch.
|
||||
const applyMutateAsync = vi.fn();
|
||||
const dismissMutateAsync = vi.fn();
|
||||
vi.mock("@/features/comment/queries/comment-query", () => ({
|
||||
useDeleteCommentMutation: () => ({ mutateAsync: vi.fn() }),
|
||||
useResolveCommentMutation: () => ({ mutateAsync: vi.fn() }),
|
||||
@@ -16,6 +17,10 @@ vi.mock("@/features/comment/queries/comment-query", () => ({
|
||||
mutateAsync: applyMutateAsync,
|
||||
isPending: false,
|
||||
}),
|
||||
useDismissSuggestionMutation: () => ({
|
||||
mutateAsync: dismissMutateAsync,
|
||||
isPending: false,
|
||||
}),
|
||||
}));
|
||||
|
||||
// CommentEditor pulls in the full TipTap editor stack; replace it with a stub.
|
||||
@@ -24,7 +29,10 @@ vi.mock("@/features/comment/components/comment-editor", () => ({
|
||||
}));
|
||||
|
||||
import CommentListItem from "./comment-list-item";
|
||||
import { canShowApply } from "@/features/comment/utils/suggestion";
|
||||
import {
|
||||
canShowApply,
|
||||
canShowDismiss,
|
||||
} from "@/features/comment/utils/suggestion";
|
||||
|
||||
const baseComment = (over?: Partial<IComment>): IComment =>
|
||||
({
|
||||
@@ -38,14 +46,20 @@ const baseComment = (over?: Partial<IComment>): IComment =>
|
||||
...over,
|
||||
}) as IComment;
|
||||
|
||||
function renderItem(comment: IComment, canEdit = true) {
|
||||
function renderItem(
|
||||
comment: IComment,
|
||||
canEdit = true,
|
||||
canComment = true,
|
||||
userSpaceRole?: string,
|
||||
) {
|
||||
return render(
|
||||
<MantineProvider>
|
||||
<CommentListItem
|
||||
comment={comment}
|
||||
pageId="page-1"
|
||||
canComment={true}
|
||||
canComment={canComment}
|
||||
canEdit={canEdit}
|
||||
userSpaceRole={userSpaceRole}
|
||||
/>
|
||||
</MantineProvider>,
|
||||
);
|
||||
@@ -108,10 +122,12 @@ describe("CommentListItem — suggested edit (#315)", () => {
|
||||
});
|
||||
|
||||
it("renders the было→стало diff and an Apply button when canEdit and not applied/resolved", () => {
|
||||
renderItem(suggestion(), true);
|
||||
// Old text appears both as the selection quote and as the struck diff row.
|
||||
const { container } = renderItem(suggestion(), true);
|
||||
// Old text appears as the selection quote (a single unsplit Text node).
|
||||
expect(screen.getAllByText("old wording here").length).toBeGreaterThan(0);
|
||||
expect(screen.getByText("new wording here")).toBeDefined();
|
||||
// The new line is now rendered as per-fragment spans (intraline diff, #331),
|
||||
// so it is no longer a single text node — assert the concatenated content.
|
||||
expect(container.textContent).toContain("new wording here");
|
||||
// Apply button is present.
|
||||
expect(screen.getByRole("button", { name: "Apply" })).toBeDefined();
|
||||
// No Applied badge yet.
|
||||
@@ -119,9 +135,9 @@ describe("CommentListItem — suggested edit (#315)", () => {
|
||||
});
|
||||
|
||||
it("hides the Apply button when canEdit is false", () => {
|
||||
renderItem(suggestion(), false);
|
||||
// Diff still renders...
|
||||
expect(screen.getByText("new wording here")).toBeDefined();
|
||||
const { container } = renderItem(suggestion(), false);
|
||||
// Diff still renders (as per-fragment spans, #331)...
|
||||
expect(container.textContent).toContain("new wording here");
|
||||
// ...but no Apply button.
|
||||
expect(screen.queryByRole("button", { name: "Apply" })).toBeNull();
|
||||
});
|
||||
@@ -157,6 +173,65 @@ describe("CommentListItem — suggested edit (#315)", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("CommentListItem — dismiss suggestion (#329)", () => {
|
||||
const suggestion = (over?: Partial<IComment>): IComment =>
|
||||
baseComment({
|
||||
selection: "old wording here",
|
||||
suggestedText: "new wording here",
|
||||
...over,
|
||||
});
|
||||
|
||||
// A space admin (userSpaceRole="admin") satisfies the owner-or-admin gate
|
||||
// regardless of who authored the comment; the tests below use it as the lever
|
||||
// since the currentUser atom is unseeded (null) in this harness.
|
||||
it("renders a Dismiss button alongside Apply when canEdit and canComment (owner/admin)", () => {
|
||||
renderItem(suggestion(), true, true, "admin");
|
||||
expect(screen.getByRole("button", { name: "Apply" })).toBeDefined();
|
||||
expect(screen.getByRole("button", { name: "Dismiss" })).toBeDefined();
|
||||
});
|
||||
|
||||
it("shows Dismiss but NOT Apply for an admin commenter who cannot edit", () => {
|
||||
renderItem(suggestion(), false, true, "admin");
|
||||
expect(screen.queryByRole("button", { name: "Apply" })).toBeNull();
|
||||
expect(screen.getByRole("button", { name: "Dismiss" })).toBeDefined();
|
||||
});
|
||||
|
||||
it("hides Dismiss when the viewer cannot comment", () => {
|
||||
renderItem(suggestion(), false, false, "admin");
|
||||
expect(screen.queryByRole("button", { name: "Dismiss" })).toBeNull();
|
||||
expect(screen.queryByRole("button", { name: "Apply" })).toBeNull();
|
||||
});
|
||||
|
||||
it("hides Dismiss for a non-owner non-admin even with canComment (#338 F5: mirrors server 403)", () => {
|
||||
// canComment=true but NOT a space admin and NOT the comment owner (the
|
||||
// currentUser atom is null while the comment is authored by user-1), so the
|
||||
// server would 403 a dismiss — the button must not be shown at all.
|
||||
renderItem(suggestion(), false, true, "member");
|
||||
expect(screen.queryByRole("button", { name: "Dismiss" })).toBeNull();
|
||||
});
|
||||
|
||||
it("hides Dismiss once the thread is resolved", () => {
|
||||
renderItem(suggestion({ resolvedAt: new Date() }), true, true, "admin");
|
||||
expect(screen.queryByRole("button", { name: "Dismiss" })).toBeNull();
|
||||
});
|
||||
|
||||
it("hides Dismiss (shows the Applied badge) once applied", () => {
|
||||
renderItem(suggestion({ suggestionAppliedAt: new Date() }), true, true, "admin");
|
||||
expect(screen.queryByRole("button", { name: "Dismiss" })).toBeNull();
|
||||
expect(screen.getByText("Applied")).toBeDefined();
|
||||
});
|
||||
|
||||
it("calls the dismiss mutation when the Dismiss button is clicked", () => {
|
||||
dismissMutateAsync.mockClear();
|
||||
renderItem(suggestion(), true, true, "admin");
|
||||
fireEvent.click(screen.getByRole("button", { name: "Dismiss" }));
|
||||
expect(dismissMutateAsync).toHaveBeenCalledWith({
|
||||
commentId: "c-1",
|
||||
pageId: "page-1",
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("canShowApply predicate", () => {
|
||||
const c = (over?: Partial<IComment>): IComment =>
|
||||
({ suggestedText: "x", ...over }) as IComment;
|
||||
@@ -182,3 +257,32 @@ describe("canShowApply predicate", () => {
|
||||
expect(canShowApply(c({ parentCommentId: "p" }), true)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("canShowDismiss predicate", () => {
|
||||
const c = (over?: Partial<IComment>): IComment =>
|
||||
({ suggestedText: "x", ...over }) as IComment;
|
||||
|
||||
it("true when suggestion present, can comment, owner/admin, not applied/resolved, top-level", () => {
|
||||
expect(canShowDismiss(c(), true, true)).toBe(true);
|
||||
});
|
||||
it("false without comment permission", () => {
|
||||
expect(canShowDismiss(c(), false, true)).toBe(false);
|
||||
});
|
||||
it("false when not owner and not admin (#338 F5)", () => {
|
||||
expect(canShowDismiss(c(), true, false)).toBe(false);
|
||||
});
|
||||
it("false when no suggestion", () => {
|
||||
expect(canShowDismiss(c({ suggestedText: null }), true, true)).toBe(false);
|
||||
});
|
||||
it("false when already applied", () => {
|
||||
expect(canShowDismiss(c({ suggestionAppliedAt: new Date() }), true, true)).toBe(
|
||||
false,
|
||||
);
|
||||
});
|
||||
it("false when resolved", () => {
|
||||
expect(canShowDismiss(c({ resolvedAt: new Date() }), true, true)).toBe(false);
|
||||
});
|
||||
it("false for a reply comment", () => {
|
||||
expect(canShowDismiss(c({ parentCommentId: "p" }), true, true)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { Group, Text, Box, Badge, Button } from "@mantine/core";
|
||||
import { AgentAvatarStack } from "@/components/ui/agent-avatar-stack.tsx";
|
||||
import React, { useEffect, useRef, useState } from "react";
|
||||
import React, { useEffect, useMemo, useRef, useState } from "react";
|
||||
import classes from "./comment.module.css";
|
||||
import { useAtom, useAtomValue } from "jotai";
|
||||
import { useTimeAgo } from "@/hooks/use-time-ago";
|
||||
@@ -13,11 +13,16 @@ import { useHover } from "@mantine/hooks";
|
||||
import {
|
||||
useApplySuggestionMutation,
|
||||
useDeleteCommentMutation,
|
||||
useDismissSuggestionMutation,
|
||||
useResolveCommentMutation,
|
||||
useUpdateCommentMutation,
|
||||
} from "@/features/comment/queries/comment-query";
|
||||
import { IComment } from "@/features/comment/types/comment.types";
|
||||
import { canShowApply } from "@/features/comment/utils/suggestion";
|
||||
import {
|
||||
canShowApply,
|
||||
canShowDismiss,
|
||||
computeSuggestionDiff,
|
||||
} from "@/features/comment/utils/suggestion";
|
||||
import { CustomAvatar } from "@/components/ui/custom-avatar.tsx";
|
||||
import { currentUserAtom } from "@/features/user/atoms/current-user-atom.ts";
|
||||
import { useTranslation } from "react-i18next";
|
||||
@@ -51,9 +56,28 @@ function CommentListItem({
|
||||
const deleteCommentMutation = useDeleteCommentMutation(comment.pageId);
|
||||
const resolveCommentMutation = useResolveCommentMutation();
|
||||
const applySuggestionMutation = useApplySuggestionMutation();
|
||||
const dismissSuggestionMutation = useDismissSuggestionMutation();
|
||||
const [currentUser] = useAtom(currentUserAtom);
|
||||
const createdAtAgo = useTimeAgo(comment.createdAt);
|
||||
|
||||
// Intraline "before -> after" diff (#331) for a suggested edit: only the
|
||||
// fragments that actually changed get emphasised inside the red/green block,
|
||||
// instead of striking through / greening the whole line. Memoised on the
|
||||
// (selection, suggestedText) pair so it recomputes only when they change.
|
||||
const suggestionDiff = useMemo(
|
||||
() =>
|
||||
comment.suggestedText != null
|
||||
? computeSuggestionDiff(comment.selection ?? "", comment.suggestedText)
|
||||
: null,
|
||||
[comment.selection, comment.suggestedText],
|
||||
);
|
||||
|
||||
// Owner-or-space-admin gate (#338): mirrors the server authz for both the
|
||||
// comment menu (edit/delete) and the suggestion Dismiss button, so we never
|
||||
// render an action the server will 403.
|
||||
const isOwnerOrAdmin =
|
||||
currentUser?.user?.id === comment.creatorId || userSpaceRole === "admin";
|
||||
|
||||
useEffect(() => {
|
||||
setContent(comment.content);
|
||||
}, [comment]);
|
||||
@@ -115,6 +139,19 @@ function CommentListItem({
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDismissSuggestion() {
|
||||
try {
|
||||
await dismissSuggestionMutation.mutateAsync({
|
||||
commentId: comment.id,
|
||||
pageId: comment.pageId,
|
||||
});
|
||||
} catch (error) {
|
||||
// Idempotent races are reconciled to success in the mutation's onError;
|
||||
// anything else surfaces there as a notification.
|
||||
console.error("Failed to dismiss suggestion:", error);
|
||||
}
|
||||
}
|
||||
|
||||
function handleCommentClick(comment: IComment) {
|
||||
const el = document.querySelector(
|
||||
`.comment-mark[data-comment-id="${comment.id}"]`,
|
||||
@@ -190,7 +227,7 @@ function CommentListItem({
|
||||
/>
|
||||
)}
|
||||
|
||||
{(currentUser?.user?.id === comment.creatorId || userSpaceRole === 'admin') && (
|
||||
{isOwnerOrAdmin && (
|
||||
<CommentMenu
|
||||
onEditComment={handleEditToggle}
|
||||
onDeleteComment={handleDeleteComment}
|
||||
@@ -236,12 +273,28 @@ function CommentListItem({
|
||||
{!comment.parentCommentId && comment.suggestedText && (
|
||||
<Box className={classes.suggestionBlock}>
|
||||
{comment.selection && (
|
||||
// Old line: read as removed as a whole (line-through/red); only the
|
||||
// changed fragments carry the extra intraline emphasis.
|
||||
<Text size="xs" className={classes.suggestionOld}>
|
||||
{comment.selection}
|
||||
{suggestionDiff?.old.map((segment, index) => (
|
||||
<span
|
||||
key={index}
|
||||
className={segment.changed ? classes.suggestionChanged : undefined}
|
||||
>
|
||||
{segment.text}
|
||||
</span>
|
||||
))}
|
||||
</Text>
|
||||
)}
|
||||
<Text size="xs" className={classes.suggestionNew}>
|
||||
{comment.suggestedText}
|
||||
{suggestionDiff?.new.map((segment, index) => (
|
||||
<span
|
||||
key={index}
|
||||
className={segment.changed ? classes.suggestionChanged : undefined}
|
||||
>
|
||||
{segment.text}
|
||||
</span>
|
||||
))}
|
||||
</Text>
|
||||
|
||||
{comment.suggestionAppliedAt ? (
|
||||
@@ -255,18 +308,42 @@ function CommentListItem({
|
||||
{t("Applied")}
|
||||
</Badge>
|
||||
) : (
|
||||
canShowApply(comment, canEdit) && (
|
||||
<Button
|
||||
size="compact-xs"
|
||||
variant="light"
|
||||
color="green"
|
||||
mt={6}
|
||||
onClick={handleApplySuggestion}
|
||||
loading={applySuggestionMutation.isPending}
|
||||
disabled={applySuggestionMutation.isPending}
|
||||
>
|
||||
{t("Apply")}
|
||||
</Button>
|
||||
(canShowApply(comment, canEdit) ||
|
||||
canShowDismiss(comment, canComment, isOwnerOrAdmin)) && (
|
||||
<Group gap="xs" mt={6}>
|
||||
{canShowApply(comment, canEdit) && (
|
||||
<Button
|
||||
size="compact-xs"
|
||||
variant="light"
|
||||
color="green"
|
||||
onClick={handleApplySuggestion}
|
||||
loading={applySuggestionMutation.isPending}
|
||||
disabled={
|
||||
applySuggestionMutation.isPending ||
|
||||
dismissSuggestionMutation.isPending
|
||||
}
|
||||
>
|
||||
{t("Apply")}
|
||||
</Button>
|
||||
)}
|
||||
{/* Dismiss ("Не применять", #329): removes the suggestion
|
||||
without changing the page text. Gated on canComment. */}
|
||||
{canShowDismiss(comment, canComment, isOwnerOrAdmin) && (
|
||||
<Button
|
||||
size="compact-xs"
|
||||
variant="subtle"
|
||||
color="gray"
|
||||
onClick={handleDismissSuggestion}
|
||||
loading={dismissSuggestionMutation.isPending}
|
||||
disabled={
|
||||
applySuggestionMutation.isPending ||
|
||||
dismissSuggestionMutation.isPending
|
||||
}
|
||||
>
|
||||
{t("Dismiss")}
|
||||
</Button>
|
||||
)}
|
||||
</Group>
|
||||
)
|
||||
)}
|
||||
</Box>
|
||||
|
||||
@@ -53,6 +53,21 @@
|
||||
margin-top: 4px;
|
||||
}
|
||||
|
||||
/* Intraline diff (#331): the fragment that actually changed within the
|
||||
red "before" / green "after" block. It inherits the surrounding red/green
|
||||
framing and adds a stronger tint plus bold weight so the eye lands on the
|
||||
changed letters/words (git/GitHub-style) rather than the whole line. The
|
||||
container's line-through (old) / green (new) still marks the full line. */
|
||||
.suggestionChanged {
|
||||
/* Stronger tint of the surrounding red/green so the changed fragment pops
|
||||
within the block. `currentColor` follows the parent's red (old) or green
|
||||
(new) text colour. No `text-decoration` here on purpose: the old block's
|
||||
inherited line-through must survive on the changed letters too. */
|
||||
background: color-mix(in srgb, currentColor 22%, transparent);
|
||||
border-radius: 2px;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.commentEditor {
|
||||
|
||||
&[data-editable][data-surface="muted"] .ProseMirror:not(.focused) {
|
||||
|
||||
@@ -0,0 +1,279 @@
|
||||
import { describe, it, expect, vi, beforeEach } from "vitest";
|
||||
import React from "react";
|
||||
import { renderHook, waitFor } from "@testing-library/react";
|
||||
import {
|
||||
QueryClient,
|
||||
QueryClientProvider,
|
||||
InfiniteData,
|
||||
} from "@tanstack/react-query";
|
||||
|
||||
/**
|
||||
* Coverage for the ephemeral-suggestion (#329) cache reconciliation in
|
||||
* useApplySuggestionMutation / useDismissSuggestionMutation: the mutations act on
|
||||
* the server `outcome` — 'deleted' drops the comment from the local list,
|
||||
* 'resolved' relocates it (by stamping resolvedAt, which the tabs split on).
|
||||
*/
|
||||
|
||||
vi.mock("@mantine/notifications", () => ({
|
||||
notifications: { show: vi.fn() },
|
||||
}));
|
||||
|
||||
vi.mock("@/features/comment/services/comment-service", () => ({
|
||||
applySuggestion: vi.fn(),
|
||||
dismissSuggestion: vi.fn(),
|
||||
createComment: vi.fn(),
|
||||
updateComment: vi.fn(),
|
||||
deleteComment: vi.fn(),
|
||||
resolveComment: vi.fn(),
|
||||
getPageComments: vi.fn(),
|
||||
}));
|
||||
|
||||
import { notifications } from "@mantine/notifications";
|
||||
import {
|
||||
applySuggestion,
|
||||
dismissSuggestion,
|
||||
} from "@/features/comment/services/comment-service";
|
||||
import {
|
||||
useApplySuggestionMutation,
|
||||
useDismissSuggestionMutation,
|
||||
RQ_KEY,
|
||||
} from "@/features/comment/queries/comment-query";
|
||||
import { IComment } from "@/features/comment/types/comment.types";
|
||||
|
||||
const PAGE_ID = "page-1";
|
||||
|
||||
function seededClient(comment: IComment) {
|
||||
const queryClient = new QueryClient({
|
||||
defaultOptions: { mutations: { retry: false } },
|
||||
});
|
||||
const seed: InfiniteData<any> = {
|
||||
pageParams: [undefined],
|
||||
pages: [{ items: [comment], meta: { hasNextPage: false, nextCursor: null } }],
|
||||
};
|
||||
queryClient.setQueryData(RQ_KEY(PAGE_ID), seed);
|
||||
const wrapper = ({ children }: { children: React.ReactNode }) => (
|
||||
<QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
|
||||
);
|
||||
return { queryClient, wrapper };
|
||||
}
|
||||
|
||||
function items(queryClient: QueryClient): IComment[] {
|
||||
const cache = queryClient.getQueryData(RQ_KEY(PAGE_ID)) as
|
||||
| InfiniteData<any>
|
||||
| undefined;
|
||||
return cache?.pages.flatMap((p) => p.items) ?? [];
|
||||
}
|
||||
|
||||
const comment = (over?: Partial<IComment>): IComment =>
|
||||
({
|
||||
id: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
content: "{}",
|
||||
creatorId: "u-1",
|
||||
workspaceId: "ws-1",
|
||||
createdAt: new Date(),
|
||||
suggestedText: "new",
|
||||
...over,
|
||||
}) as IComment;
|
||||
|
||||
describe("useApplySuggestionMutation — outcome handling (#329)", () => {
|
||||
beforeEach(() => vi.clearAllMocks());
|
||||
|
||||
it("outcome=deleted → removes the comment from the list", async () => {
|
||||
vi.mocked(applySuggestion).mockResolvedValue({
|
||||
id: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
outcome: "deleted",
|
||||
} as any);
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
|
||||
const { result } = renderHook(() => useApplySuggestionMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
await result.current.mutateAsync({ commentId: "c-1", pageId: PAGE_ID });
|
||||
await waitFor(() => expect(result.current.isSuccess).toBe(true));
|
||||
|
||||
expect(items(queryClient)).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("outcome=resolved → keeps the comment and stamps resolvedAt/applied fields", async () => {
|
||||
const resolvedAt = new Date();
|
||||
vi.mocked(applySuggestion).mockResolvedValue({
|
||||
id: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
outcome: "resolved",
|
||||
resolvedAt,
|
||||
resolvedById: "u-1",
|
||||
resolvedBy: { id: "u-1", name: "A" },
|
||||
suggestionAppliedAt: resolvedAt,
|
||||
suggestionAppliedById: "u-1",
|
||||
} as any);
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
|
||||
const { result } = renderHook(() => useApplySuggestionMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
await result.current.mutateAsync({ commentId: "c-1", pageId: PAGE_ID });
|
||||
await waitFor(() => expect(result.current.isSuccess).toBe(true));
|
||||
|
||||
const list = items(queryClient);
|
||||
expect(list).toHaveLength(1);
|
||||
expect(list[0].resolvedAt).toBe(resolvedAt);
|
||||
expect(list[0].suggestionAppliedAt).toBe(resolvedAt);
|
||||
});
|
||||
});
|
||||
|
||||
describe("useDismissSuggestionMutation — outcome handling (#329)", () => {
|
||||
beforeEach(() => vi.clearAllMocks());
|
||||
|
||||
it("outcome=deleted → removes the comment from the list", async () => {
|
||||
vi.mocked(dismissSuggestion).mockResolvedValue({
|
||||
id: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
outcome: "deleted",
|
||||
} as any);
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
|
||||
const { result } = renderHook(() => useDismissSuggestionMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
await result.current.mutateAsync({ commentId: "c-1", pageId: PAGE_ID });
|
||||
await waitFor(() => expect(result.current.isSuccess).toBe(true));
|
||||
|
||||
expect(items(queryClient)).toHaveLength(0);
|
||||
});
|
||||
|
||||
it("outcome=resolved → keeps the comment and stamps resolvedAt", async () => {
|
||||
const resolvedAt = new Date();
|
||||
vi.mocked(dismissSuggestion).mockResolvedValue({
|
||||
id: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
outcome: "resolved",
|
||||
resolvedAt,
|
||||
resolvedById: "u-1",
|
||||
resolvedBy: { id: "u-1", name: "A" },
|
||||
} as any);
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
|
||||
const { result } = renderHook(() => useDismissSuggestionMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
await result.current.mutateAsync({ commentId: "c-1", pageId: PAGE_ID });
|
||||
await waitFor(() => expect(result.current.isSuccess).toBe(true));
|
||||
|
||||
const list = items(queryClient);
|
||||
expect(list).toHaveLength(1);
|
||||
expect(list[0].resolvedAt).toBe(resolvedAt);
|
||||
});
|
||||
|
||||
it("idempotent race (404) → treated as success, comment removed from the list", async () => {
|
||||
vi.mocked(dismissSuggestion).mockRejectedValue({
|
||||
response: { status: 404 },
|
||||
});
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
|
||||
const { result } = renderHook(() => useDismissSuggestionMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
// mutateAsync rejects even though onError reconciles the cache; swallow it.
|
||||
await result.current
|
||||
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID })
|
||||
.catch(() => undefined);
|
||||
await waitFor(() => expect(result.current.isError).toBe(true));
|
||||
|
||||
expect(items(queryClient)).toHaveLength(0);
|
||||
// #338 F3: the idempotent race must still fire the SUCCESS toast, not just
|
||||
// silently drop the comment.
|
||||
expect(notifications.show).toHaveBeenCalledWith({
|
||||
message: "Suggestion dismissed",
|
||||
});
|
||||
});
|
||||
|
||||
it("dismiss 400 (thread still alive) → NOT a success, comment kept, no green toast (#338 F2)", async () => {
|
||||
// 400 means the thread is alive (already resolved / a reply raced in).
|
||||
// Narrowed onError: only 404 is a success-noop; 400 must surface a real error
|
||||
// and keep the comment in the cache.
|
||||
vi.mocked(dismissSuggestion).mockRejectedValue({
|
||||
response: { status: 400 },
|
||||
});
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
|
||||
const { result } = renderHook(() => useDismissSuggestionMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
await result.current
|
||||
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID })
|
||||
.catch(() => undefined);
|
||||
await waitFor(() => expect(result.current.isError).toBe(true));
|
||||
|
||||
// Comment NOT dropped from the cache.
|
||||
expect(items(queryClient)).toHaveLength(1);
|
||||
// A real (red) error, never the success message.
|
||||
expect(notifications.show).toHaveBeenCalledWith(
|
||||
expect.objectContaining({ color: "red" }),
|
||||
);
|
||||
expect(notifications.show).not.toHaveBeenCalledWith({
|
||||
message: "Suggestion dismissed",
|
||||
});
|
||||
});
|
||||
|
||||
it("APPLY idempotent race (404) → treated as success, comment removed from the list", async () => {
|
||||
// After #329 an applied reply-less suggestion is hard-deleted, so a racing
|
||||
// second apply hits 404 — must reconcile to success like dismiss, not a red
|
||||
// error (restores the #315 apply idempotency).
|
||||
vi.mocked(applySuggestion).mockRejectedValue({
|
||||
response: { status: 404 },
|
||||
});
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
|
||||
const { result } = renderHook(() => useApplySuggestionMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
await result.current
|
||||
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID })
|
||||
.catch(() => undefined);
|
||||
await waitFor(() => expect(result.current.isError).toBe(true));
|
||||
|
||||
expect(items(queryClient)).toHaveLength(0);
|
||||
// #338 F3: the idempotent race must still fire the SUCCESS toast.
|
||||
expect(notifications.show).toHaveBeenCalledWith({
|
||||
message: "Suggestion applied",
|
||||
});
|
||||
});
|
||||
|
||||
it("APPLY 400 (thread resolved, not applied) → NOT a success, comment kept, red error (#338 F2)", async () => {
|
||||
// apply's only 400 is "Cannot apply … on a resolved comment thread" — the
|
||||
// thread was resolved (often with discussion) but NOT applied. It must be a
|
||||
// real error surfacing the server message, and must NOT drop the live thread.
|
||||
vi.mocked(applySuggestion).mockRejectedValue({
|
||||
response: {
|
||||
status: 400,
|
||||
data: {
|
||||
message: "Cannot apply a suggested edit on a resolved comment thread",
|
||||
},
|
||||
},
|
||||
});
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
|
||||
const { result } = renderHook(() => useApplySuggestionMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
await result.current
|
||||
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID })
|
||||
.catch(() => undefined);
|
||||
await waitFor(() => expect(result.current.isError).toBe(true));
|
||||
|
||||
// The live thread is NOT dropped from the cache.
|
||||
expect(items(queryClient)).toHaveLength(1);
|
||||
// Surfaces the server's specific message as a red error, never a success.
|
||||
expect(notifications.show).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
message: "Cannot apply a suggested edit on a resolved comment thread",
|
||||
color: "red",
|
||||
}),
|
||||
);
|
||||
expect(notifications.show).not.toHaveBeenCalledWith({
|
||||
message: "Suggestion applied",
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
applySuggestion,
|
||||
createComment,
|
||||
deleteComment,
|
||||
dismissSuggestion,
|
||||
getPageComments,
|
||||
resolveComment,
|
||||
updateComment,
|
||||
@@ -16,6 +17,7 @@ import {
|
||||
ICommentParams,
|
||||
IComment,
|
||||
IResolveComment,
|
||||
ISuggestionOutcome,
|
||||
} from "@/features/comment/types/comment.types";
|
||||
import { notifications } from "@mantine/notifications";
|
||||
import { IPagination } from "@/lib/types.ts";
|
||||
@@ -177,40 +179,121 @@ function updateCommentInCache(
|
||||
};
|
||||
}
|
||||
|
||||
function removeCommentFromCache(
|
||||
cache: InfiniteData<IPagination<IComment>>,
|
||||
commentId: string,
|
||||
): InfiniteData<IPagination<IComment>> {
|
||||
return {
|
||||
...cache,
|
||||
pages: cache.pages.map((page) => ({
|
||||
...page,
|
||||
items: page.items.filter((comment) => comment.id !== commentId),
|
||||
})),
|
||||
};
|
||||
}
|
||||
|
||||
// Reconcile the local comment cache with an ephemeral-suggestion outcome (#329)
|
||||
// returned by apply/dismiss: 'deleted' → drop the comment (it disappeared);
|
||||
// 'resolved' → the thread had replies and was resolved, so carry the resolved
|
||||
// state through (which relocates it to the resolved tab).
|
||||
function applySuggestionOutcomeToCache(
|
||||
queryClient: ReturnType<typeof useQueryClient>,
|
||||
pageId: string,
|
||||
commentId: string,
|
||||
data: ISuggestionOutcome,
|
||||
) {
|
||||
const cache = queryClient.getQueryData(RQ_KEY(pageId)) as
|
||||
| InfiniteData<IPagination<IComment>>
|
||||
| undefined;
|
||||
if (!cache) return;
|
||||
|
||||
if (data.outcome === "deleted") {
|
||||
queryClient.setQueryData(RQ_KEY(pageId), removeCommentFromCache(cache, commentId));
|
||||
return;
|
||||
}
|
||||
|
||||
// 'resolved' (or an older server that omits outcome): reflect the resolved
|
||||
// state and the applied stamps (apply sets them; dismiss leaves them null).
|
||||
queryClient.setQueryData(
|
||||
RQ_KEY(pageId),
|
||||
updateCommentInCache(cache, commentId, (comment) => ({
|
||||
...comment,
|
||||
suggestionAppliedAt: data.suggestionAppliedAt,
|
||||
suggestionAppliedById: data.suggestionAppliedById,
|
||||
resolvedAt: data.resolvedAt,
|
||||
resolvedById: data.resolvedById,
|
||||
resolvedBy: data.resolvedBy,
|
||||
})),
|
||||
);
|
||||
}
|
||||
|
||||
export function useApplySuggestionMutation() {
|
||||
const queryClient = useQueryClient();
|
||||
const { t } = useTranslation();
|
||||
|
||||
return useMutation<IComment, any, { commentId: string; pageId: string }>({
|
||||
return useMutation<
|
||||
ISuggestionOutcome,
|
||||
any,
|
||||
{ commentId: string; pageId: string }
|
||||
>({
|
||||
// No optimistic update: apply can fail with 409 (the commented text drifted),
|
||||
// so we only mutate the cache once the server confirms.
|
||||
mutationFn: ({ commentId }) => applySuggestion(commentId),
|
||||
onSuccess: (data, variables) => {
|
||||
const cache = queryClient.getQueryData(
|
||||
RQ_KEY(variables.pageId),
|
||||
) as InfiniteData<IPagination<IComment>> | undefined;
|
||||
|
||||
if (cache) {
|
||||
queryClient.setQueryData(
|
||||
RQ_KEY(variables.pageId),
|
||||
updateCommentInCache(cache, variables.commentId, (comment) => ({
|
||||
...comment,
|
||||
suggestionAppliedAt: data.suggestionAppliedAt,
|
||||
suggestionAppliedById: data.suggestionAppliedById,
|
||||
// The server auto-resolves the thread on apply — carry that through.
|
||||
resolvedAt: data.resolvedAt,
|
||||
resolvedById: data.resolvedById,
|
||||
resolvedBy: data.resolvedBy,
|
||||
})),
|
||||
);
|
||||
}
|
||||
// Ephemeral (#329): the server hard-deletes the applied suggestion when the
|
||||
// thread has no replies ('deleted') or resolves it when it does ('resolved').
|
||||
applySuggestionOutcomeToCache(
|
||||
queryClient,
|
||||
variables.pageId,
|
||||
variables.commentId,
|
||||
data,
|
||||
);
|
||||
|
||||
notifications.show({ message: t("Suggestion applied") });
|
||||
},
|
||||
onError: (err: any) => {
|
||||
onError: (err: any, variables) => {
|
||||
const status = err?.response?.status;
|
||||
// Idempotent race (double-click, or apply↔dismiss): after #329 an applied
|
||||
// reply-less suggestion is hard-deleted, so a second/racing apply hits 404
|
||||
// (already gone). ONLY 404 is a real success-noop — drop it from the cache
|
||||
// and report success, the user's intent is already satisfied (restores the
|
||||
// #315 apply idempotency the ephemeral delete would otherwise break).
|
||||
//
|
||||
// 400 is NOT success (#338 F2): apply's only 400 is "Cannot apply … on a
|
||||
// resolved comment thread" — the thread was resolved (often WITH a live
|
||||
// discussion) but the edit was NOT applied. Treating it as "Suggestion
|
||||
// applied" is a false success that also drops a live thread from the cache.
|
||||
// The #315 idempotent repeat does NOT produce 400 (childless → 404;
|
||||
// with-replies → 200), so we never lose idempotency by excluding it here.
|
||||
if (status === 404) {
|
||||
const cache = queryClient.getQueryData(RQ_KEY(variables.pageId)) as
|
||||
| InfiniteData<IPagination<IComment>>
|
||||
| undefined;
|
||||
if (cache) {
|
||||
queryClient.setQueryData(
|
||||
RQ_KEY(variables.pageId),
|
||||
removeCommentFromCache(cache, variables.commentId),
|
||||
);
|
||||
}
|
||||
notifications.show({ message: t("Suggestion applied") });
|
||||
return;
|
||||
}
|
||||
// 400 => the thread was resolved and the edit could not be applied. Show a
|
||||
// real error and KEEP the comment in the cache (it is still alive). Prefer
|
||||
// the server's specific message when it carries one.
|
||||
if (status === 400) {
|
||||
const serverMsg = err?.response?.data?.message;
|
||||
notifications.show({
|
||||
message:
|
||||
typeof serverMsg === "string" && serverMsg.length > 0
|
||||
? serverMsg
|
||||
: t("Failed to apply suggestion"),
|
||||
color: "red",
|
||||
});
|
||||
return;
|
||||
}
|
||||
// 409 => the commented text changed since the suggestion was made. Surface
|
||||
// a specific message (with the current text) rather than a generic error.
|
||||
const status = err?.response?.status;
|
||||
const currentText = err?.response?.data?.currentText;
|
||||
if (status === 409 && typeof currentText === "string") {
|
||||
const shortText =
|
||||
@@ -234,6 +317,58 @@ export function useApplySuggestionMutation() {
|
||||
});
|
||||
}
|
||||
|
||||
export function useDismissSuggestionMutation() {
|
||||
const queryClient = useQueryClient();
|
||||
const { t } = useTranslation();
|
||||
|
||||
return useMutation<
|
||||
ISuggestionOutcome,
|
||||
any,
|
||||
{ commentId: string; pageId: string }
|
||||
>({
|
||||
mutationFn: ({ commentId }) => dismissSuggestion(commentId),
|
||||
onSuccess: (data, variables) => {
|
||||
// Ephemeral (#329): dismiss hard-deletes the suggestion when the thread has
|
||||
// no replies ('deleted') or resolves it when it does ('resolved').
|
||||
applySuggestionOutcomeToCache(
|
||||
queryClient,
|
||||
variables.pageId,
|
||||
variables.commentId,
|
||||
data,
|
||||
);
|
||||
|
||||
notifications.show({ message: t("Suggestion dismissed") });
|
||||
},
|
||||
onError: (err: any, variables) => {
|
||||
// Idempotent race (double-click, or apply↔dismiss): the comment is already
|
||||
// gone (404). ONLY 404 is a real success-noop — drop it from the cache and
|
||||
// report success, the user's intent (make it disappear) is satisfied.
|
||||
//
|
||||
// 400 is NOT success (#338 F2): it means the thread is still ALIVE (already
|
||||
// resolved, or a reply raced in), so treating it as "dismissed" would drop
|
||||
// a live thread from the cache. Show a real error and keep the comment.
|
||||
const status = err?.response?.status;
|
||||
if (status === 404) {
|
||||
const cache = queryClient.getQueryData(RQ_KEY(variables.pageId)) as
|
||||
| InfiniteData<IPagination<IComment>>
|
||||
| undefined;
|
||||
if (cache) {
|
||||
queryClient.setQueryData(
|
||||
RQ_KEY(variables.pageId),
|
||||
removeCommentFromCache(cache, variables.commentId),
|
||||
);
|
||||
}
|
||||
notifications.show({ message: t("Suggestion dismissed") });
|
||||
return;
|
||||
}
|
||||
notifications.show({
|
||||
message: t("Failed to dismiss suggestion"),
|
||||
color: "red",
|
||||
});
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
export function useResolveCommentMutation() {
|
||||
const queryClient = useQueryClient();
|
||||
const { t } = useTranslation();
|
||||
|
||||
@@ -3,6 +3,7 @@ import {
|
||||
ICommentParams,
|
||||
IComment,
|
||||
IResolveComment,
|
||||
ISuggestionOutcome,
|
||||
} from "@/features/comment/types/comment.types";
|
||||
import { IPagination } from "@/lib/types.ts";
|
||||
|
||||
@@ -18,13 +19,24 @@ export async function resolveComment(data: IResolveComment): Promise<IComment> {
|
||||
return req.data;
|
||||
}
|
||||
|
||||
export async function applySuggestion(commentId: string): Promise<IComment> {
|
||||
export async function applySuggestion(
|
||||
commentId: string,
|
||||
): Promise<ISuggestionOutcome> {
|
||||
// Mirrors resolveComment: let axios reject on non-2xx so the mutation can read
|
||||
// the 409 body (`{ message, currentText }`) off err.response.data.
|
||||
const req = await api.post("/comments/apply-suggestion", { commentId });
|
||||
return req.data.data ?? req.data;
|
||||
}
|
||||
|
||||
export async function dismissSuggestion(
|
||||
commentId: string,
|
||||
): Promise<ISuggestionOutcome> {
|
||||
// Dismiss ("Не применять") a suggested edit (#329): the server hard-deletes
|
||||
// the comment (or resolves it when it has replies) and returns the outcome.
|
||||
const req = await api.post("/comments/dismiss-suggestion", { commentId });
|
||||
return req.data.data ?? req.data;
|
||||
}
|
||||
|
||||
export async function updateComment(
|
||||
data: Partial<IComment>,
|
||||
): Promise<IComment> {
|
||||
|
||||
@@ -60,6 +60,15 @@ export interface IResolveComment {
|
||||
resolved: boolean;
|
||||
}
|
||||
|
||||
// Result of applying or dismissing an ephemeral suggested edit (#329). The
|
||||
// server hard-deletes the comment (`deleted`) unless the thread has replies, in
|
||||
// which case it is resolved (`resolved`). The returned comment fields carry the
|
||||
// resolved-branch state; `outcome` tells the client which optimistic action to
|
||||
// take (drop the comment vs. move it to the resolved tab).
|
||||
export type ISuggestionOutcome = IComment & {
|
||||
outcome?: "deleted" | "resolved";
|
||||
};
|
||||
|
||||
export interface ICommentParams extends QueryParams {
|
||||
pageId: string;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,102 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { computeSuggestionDiff, Segment } from "@/features/comment/utils/suggestion";
|
||||
|
||||
// Reconstruct the plain string from a segment stream — the diff must be
|
||||
// lossless (concatenating every fragment yields the original input).
|
||||
const join = (segments: Segment[]): string =>
|
||||
segments.map((s) => s.text).join("");
|
||||
|
||||
// The subset of segments (in order) that the UI would emphasise.
|
||||
const changed = (segments: Segment[]): string[] =>
|
||||
segments.filter((s) => s.changed).map((s) => s.text);
|
||||
|
||||
// Find the segment that contains a substring, to assert its `changed` flag.
|
||||
const segmentWith = (segments: Segment[], needle: string): Segment | undefined =>
|
||||
segments.find((s) => s.text.includes(needle));
|
||||
|
||||
describe("computeSuggestionDiff", () => {
|
||||
it("highlights only the single changed letter in a one-letter edit", () => {
|
||||
const { old, new: neu } = computeSuggestionDiff("заведем", "заведём");
|
||||
|
||||
// Lossless.
|
||||
expect(join(old)).toBe("заведем");
|
||||
expect(join(neu)).toBe("заведём");
|
||||
|
||||
// Old side: exactly the `е` is changed, the rest is common.
|
||||
expect(changed(old)).toEqual(["е"]);
|
||||
expect(old).toEqual([
|
||||
{ text: "завед", changed: false },
|
||||
{ text: "е", changed: true },
|
||||
{ text: "м", changed: false },
|
||||
]);
|
||||
|
||||
// New side: exactly the `ё` is changed.
|
||||
expect(changed(neu)).toEqual(["ё"]);
|
||||
expect(neu).toEqual([
|
||||
{ text: "завед", changed: false },
|
||||
{ text: "ё", changed: true },
|
||||
{ text: "м", changed: false },
|
||||
]);
|
||||
});
|
||||
|
||||
it("marks the differing words changed but keeps the shared word common", () => {
|
||||
const { old, new: neu } = computeSuggestionDiff(
|
||||
"привет мир",
|
||||
"здравствуй мир",
|
||||
);
|
||||
|
||||
// Lossless.
|
||||
expect(join(old)).toBe("привет мир");
|
||||
expect(join(neu)).toBe("здравствуй мир");
|
||||
|
||||
// The shared trailing word stays common on both sides (no per-letter noise
|
||||
// leaking across the differing words into `мир`).
|
||||
expect(segmentWith(old, "мир")?.changed).toBe(false);
|
||||
expect(segmentWith(neu, "мир")?.changed).toBe(false);
|
||||
|
||||
// The differing words are emphasised somewhere on each side.
|
||||
expect(changed(old).length).toBeGreaterThan(0);
|
||||
expect(changed(neu).length).toBeGreaterThan(0);
|
||||
expect(changed(old).join("")).toContain("п"); // from `привет`
|
||||
expect(changed(neu).join("")).toContain("зд"); // from `здравствуй`
|
||||
|
||||
// No changed fragment on either side touches the word `мир`.
|
||||
expect(changed(old).some((t) => t.includes("мир"))).toBe(false);
|
||||
expect(changed(neu).some((t) => t.includes("мир"))).toBe(false);
|
||||
});
|
||||
|
||||
it("marks a whole inserted word changed and leaves the old line common", () => {
|
||||
const { old, new: neu } = computeSuggestionDiff("a c", "a b c");
|
||||
|
||||
expect(join(old)).toBe("a c");
|
||||
expect(join(neu)).toBe("a b c");
|
||||
|
||||
// Old line has no changed fragment (nothing was removed).
|
||||
expect(changed(old)).toEqual([]);
|
||||
// The inserted word is the only changed fragment on the new side.
|
||||
expect(neu).toContainEqual({ text: "b ", changed: true });
|
||||
expect(changed(neu)).toEqual(["b "]);
|
||||
});
|
||||
|
||||
it("marks a whole deleted word changed and leaves the new line common", () => {
|
||||
const { old, new: neu } = computeSuggestionDiff("a b c", "a c");
|
||||
|
||||
expect(join(old)).toBe("a b c");
|
||||
expect(join(neu)).toBe("a c");
|
||||
|
||||
// The deleted word is the only changed fragment on the old side.
|
||||
expect(old).toContainEqual({ text: "b ", changed: true });
|
||||
expect(changed(old)).toEqual(["b "]);
|
||||
// New line has no changed fragment (nothing was added).
|
||||
expect(changed(neu)).toEqual([]);
|
||||
});
|
||||
|
||||
it("marks everything common for identical strings", () => {
|
||||
const { old, new: neu } = computeSuggestionDiff("hello", "hello");
|
||||
|
||||
expect(old).toEqual([{ text: "hello", changed: false }]);
|
||||
expect(neu).toEqual([{ text: "hello", changed: false }]);
|
||||
expect(changed(old)).toEqual([]);
|
||||
expect(changed(neu)).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -1,3 +1,4 @@
|
||||
import { diffWordsWithSpace, diffChars } from "diff";
|
||||
import { IComment } from "@/features/comment/types/comment.types";
|
||||
|
||||
// Whether the suggested-edit (#315) "Apply" button should be shown for a
|
||||
@@ -12,3 +13,127 @@ export function canShowApply(comment: IComment, canEdit?: boolean): boolean {
|
||||
!comment.parentCommentId,
|
||||
);
|
||||
}
|
||||
|
||||
// One contiguous run of text within a suggestion's "before" or "after" line.
|
||||
// `changed` marks the fragment that actually differs from the other side, so
|
||||
// the UI can emphasise only the intraline delta (git/GitHub-style) instead of
|
||||
// the whole line.
|
||||
export interface Segment {
|
||||
text: string;
|
||||
changed: boolean;
|
||||
}
|
||||
|
||||
// A pure "before -> after" intraline diff (#331): the old line split into
|
||||
// common vs. removed-and-changed fragments, and the new line split into common
|
||||
// vs. added-and-changed fragments. Concatenating each side's `text` reproduces
|
||||
// the original strings.
|
||||
export interface SuggestionDiff {
|
||||
old: Segment[];
|
||||
new: Segment[];
|
||||
}
|
||||
|
||||
// Push a segment, coalescing runs of the same `changed` flag on the same side
|
||||
// so the render emits as few spans as possible and tests stay predictable.
|
||||
function pushSegment(segments: Segment[], text: string, changed: boolean): void {
|
||||
if (text === "") return;
|
||||
const last = segments[segments.length - 1];
|
||||
if (last && last.changed === changed) {
|
||||
last.text += text;
|
||||
} else {
|
||||
segments.push({ text, changed });
|
||||
}
|
||||
}
|
||||
|
||||
// Compute an intraline diff between the old `selection` and the new
|
||||
// `suggestedText` of a suggestion. PURE — no React, no DOM, no I/O.
|
||||
//
|
||||
// Hybrid word + char algorithm (per #331):
|
||||
// 1. `diffWordsWithSpace` yields word-granular parts [{value, added, removed}].
|
||||
// 2. An ADJACENT removed+added pair (a word replacement) is refined with
|
||||
// `diffChars`: shared characters stay common, differing characters are
|
||||
// marked `changed` on their respective side. This is what keeps a
|
||||
// one-letter edit (заведем -> заведём) from highlighting the whole word.
|
||||
// 3. A lone `added` (insertion) or lone `removed` (deletion) marks the whole
|
||||
// fragment `changed`.
|
||||
// 4. An unchanged part is `common` on both sides.
|
||||
//
|
||||
// Rejected alternatives: pure `diffChars` is noisy on word swaps; pure
|
||||
// `diffWordsWithSpace` highlights the whole word rather than the changed letter.
|
||||
export function computeSuggestionDiff(
|
||||
oldStr: string,
|
||||
newStr: string,
|
||||
): SuggestionDiff {
|
||||
const oldSegments: Segment[] = [];
|
||||
const newSegments: Segment[] = [];
|
||||
|
||||
const parts = diffWordsWithSpace(oldStr, newStr);
|
||||
|
||||
for (let i = 0; i < parts.length; i++) {
|
||||
const part = parts[i];
|
||||
const next = parts[i + 1];
|
||||
|
||||
// A word replacement: a removed part immediately followed by an added part
|
||||
// (or the reverse). Refine it character-by-character so only the differing
|
||||
// letters are highlighted while shared letters stay common.
|
||||
const isReplacementPair =
|
||||
next &&
|
||||
((part.removed && next.added) || (part.added && next.removed));
|
||||
|
||||
if (isReplacementPair) {
|
||||
const removedPart = part.removed ? part : next;
|
||||
const addedPart = part.added ? part : next;
|
||||
|
||||
const charParts = diffChars(removedPart.value, addedPart.value);
|
||||
for (const cp of charParts) {
|
||||
if (cp.added) {
|
||||
pushSegment(newSegments, cp.value, true);
|
||||
} else if (cp.removed) {
|
||||
pushSegment(oldSegments, cp.value, true);
|
||||
} else {
|
||||
// Shared character: common on both sides.
|
||||
pushSegment(oldSegments, cp.value, false);
|
||||
pushSegment(newSegments, cp.value, false);
|
||||
}
|
||||
}
|
||||
|
||||
i++; // consume the paired part as well
|
||||
continue;
|
||||
}
|
||||
|
||||
if (part.added) {
|
||||
// Lone insertion: only present in the new line, wholly changed.
|
||||
pushSegment(newSegments, part.value, true);
|
||||
} else if (part.removed) {
|
||||
// Lone deletion: only present in the old line, wholly changed.
|
||||
pushSegment(oldSegments, part.value, true);
|
||||
} else {
|
||||
// Unchanged: common on both sides.
|
||||
pushSegment(oldSegments, part.value, false);
|
||||
pushSegment(newSegments, part.value, false);
|
||||
}
|
||||
}
|
||||
|
||||
return { old: oldSegments, new: newSegments };
|
||||
}
|
||||
|
||||
// Whether the suggested-edit (#329) "Не применять" (Dismiss) button should be
|
||||
// shown. Dismiss does NOT change the page text (so it needs only canComment, not
|
||||
// canEdit), BUT a childless dismiss IRREVERSIBLY hard-deletes the comment, so the
|
||||
// server gates it on comment-owner-OR-space-admin (#338 F5). The button must
|
||||
// mirror that authz or a non-owner non-admin sees a live Dismiss that always
|
||||
// 403s → red error. Hence isOwnerOrAdmin is required IN ADDITION to canComment.
|
||||
// Same not-applied/not-resolved/top-level conditions as Apply.
|
||||
export function canShowDismiss(
|
||||
comment: IComment,
|
||||
canComment?: boolean,
|
||||
isOwnerOrAdmin?: boolean,
|
||||
): boolean {
|
||||
return Boolean(
|
||||
canComment &&
|
||||
isOwnerOrAdmin &&
|
||||
comment.suggestedText &&
|
||||
!comment.suggestionAppliedAt &&
|
||||
!comment.resolvedAt &&
|
||||
!comment.parentCommentId,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { useCallback } from "react";
|
||||
import { useAtom, useStore } from "jotai";
|
||||
import { useAtom, useSetAtom, useStore } from "jotai";
|
||||
import { notifications } from "@mantine/notifications";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import { useNavigate, useParams } from "react-router-dom";
|
||||
@@ -20,6 +20,7 @@ import {
|
||||
} from "@/features/page/queries/page-query.ts";
|
||||
import { buildPageUrl } from "@/features/page/page.utils.ts";
|
||||
import { getSpaceUrl } from "@/lib/config.ts";
|
||||
import { mobileSidebarAtom } from "@/components/layouts/global/hooks/atoms/sidebar-atom.ts";
|
||||
|
||||
export type UseTreeMutation = {
|
||||
handleMove: (sourceId: string, op: DropOp) => Promise<void>;
|
||||
@@ -43,6 +44,7 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
|
||||
const removePageMutation = useRemovePageMutation();
|
||||
const movePageMutation = useMovePageMutation();
|
||||
const navigate = useNavigate();
|
||||
const setMobileSidebar = useSetAtom(mobileSidebarAtom);
|
||||
const { spaceSlug, pageSlug } = useParams();
|
||||
|
||||
const handleMove = useCallback(
|
||||
@@ -201,8 +203,23 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
|
||||
createdPage.title,
|
||||
);
|
||||
navigate(pageUrl);
|
||||
// On mobile the create action is triggered from inside the off-canvas
|
||||
// sidebar drawer (space sidebar "+", tree-row "add subpage"). Navigating
|
||||
// alone leaves that drawer open on top of the freshly created page, so the
|
||||
// editor stays hidden behind the tree. Close it here so the new page opens
|
||||
// in the editor — mirrors the row-click drawer-close in space-tree-row.
|
||||
// No-op on desktop, where the mobile drawer atom is already false.
|
||||
setMobileSidebar(false);
|
||||
},
|
||||
[spaceId, createPageMutation, setData, store, navigate, spaceSlug],
|
||||
[
|
||||
spaceId,
|
||||
createPageMutation,
|
||||
setData,
|
||||
store,
|
||||
navigate,
|
||||
spaceSlug,
|
||||
setMobileSidebar,
|
||||
],
|
||||
);
|
||||
|
||||
const handleRename = useCallback(
|
||||
|
||||
@@ -13,5 +13,22 @@ export default defineConfig({
|
||||
environment: 'jsdom',
|
||||
globals: true,
|
||||
setupFiles: ['./vitest.setup.ts'],
|
||||
// Coverage gate (issue #324). v8 provider (not istanbul) so ESM barrels
|
||||
// like `@docmost/editor-ext` are not re-parsed/instrumented. Thresholds are
|
||||
// set a few points below the level measured on develop, scoped to the files
|
||||
// the suite exercises (`all: false`) rather than the whole app, so the gate
|
||||
// passes today but fails on a genuine coverage regression.
|
||||
coverage: {
|
||||
enabled: true,
|
||||
provider: 'v8',
|
||||
reporter: ['text-summary', 'text'],
|
||||
all: false,
|
||||
thresholds: {
|
||||
statements: 55,
|
||||
branches: 53,
|
||||
functions: 44,
|
||||
lines: 55,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
@@ -23,7 +23,7 @@
|
||||
"migration:reset": "tsx src/database/migrate.ts down-to NO_MIGRATIONS",
|
||||
"migration:codegen": "kysely-codegen --dialect=postgres --camel-case --env-file=../../.env --out-file=./src/database/types/db.d.ts",
|
||||
"lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
|
||||
"pretest": "pnpm --filter @docmost/editor-ext build && pnpm --filter @docmost/git-sync build && pnpm --filter @docmost/mcp build",
|
||||
"pretest": "pnpm --filter @docmost/editor-ext build && pnpm --filter @docmost/prosemirror-markdown build && pnpm --filter @docmost/git-sync build && pnpm --filter @docmost/mcp build",
|
||||
"test": "jest",
|
||||
"test:int": "jest --config test/jest-integration.json",
|
||||
"test:watch": "jest --watch",
|
||||
@@ -215,6 +215,7 @@
|
||||
"^src/(.*)$": "<rootDir>/$1",
|
||||
"^@docmost/git-sync$": "<rootDir>/../../../packages/git-sync/src/index.ts",
|
||||
"^@docmost/git-sync/(.*)$": "<rootDir>/../../../packages/git-sync/src/$1",
|
||||
"^@docmost/prosemirror-markdown$": "<rootDir>/../../../packages/prosemirror-markdown/src/index.ts",
|
||||
"^(\\.{1,2}/.*)\\.js$": "$1"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -130,3 +130,59 @@ describe('CollaborationHandler.applyCommentSuggestion', () => {
|
||||
expect(value).toBe(42);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CollaborationHandler.deleteCommentMark', () => {
|
||||
it('strips the comment mark for the given commentId (ephemeral suggestion #329)', async () => {
|
||||
const doc = buildDocWithComment('Hello world', 'c1');
|
||||
const { hocuspocus, connection } = fakeHocuspocus(doc);
|
||||
const handler = new CollaborationHandler();
|
||||
const handlers = handler.getHandlers(hocuspocus);
|
||||
|
||||
await handlers.deleteCommentMark('doc-1', { commentId: 'c1', user });
|
||||
|
||||
// The mark is gone; the text itself stays (deleting the anchor, not the run).
|
||||
const xmlText = (
|
||||
doc.getXmlFragment('default').get(0) as Y.XmlElement
|
||||
).get(0) as Y.XmlText;
|
||||
expect(xmlText.toDelta()).toEqual([{ insert: 'Hello world' }]);
|
||||
expect(connection.transact).toHaveBeenCalledTimes(1);
|
||||
expect(connection.disconnect).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('routes the removal through removeYjsMarkByAttribute with the right args', async () => {
|
||||
const doc = buildDocWithComment('abc', 'c9');
|
||||
const { hocuspocus } = fakeHocuspocus(doc);
|
||||
const spy = jest.spyOn(yjsUtil, 'removeYjsMarkByAttribute');
|
||||
const handler = new CollaborationHandler();
|
||||
const handlers = handler.getHandlers(hocuspocus);
|
||||
|
||||
await handlers.deleteCommentMark('doc-1', { commentId: 'c9', user });
|
||||
|
||||
expect(spy).toHaveBeenCalledWith(
|
||||
doc.getXmlFragment('default'),
|
||||
'comment',
|
||||
'commentId',
|
||||
'c9',
|
||||
);
|
||||
spy.mockRestore();
|
||||
});
|
||||
|
||||
it('leaves a different comment\'s mark intact', async () => {
|
||||
const doc = buildDocWithComment('keep me', 'other');
|
||||
const { hocuspocus } = fakeHocuspocus(doc);
|
||||
const handler = new CollaborationHandler();
|
||||
const handlers = handler.getHandlers(hocuspocus);
|
||||
|
||||
await handlers.deleteCommentMark('doc-1', { commentId: 'c1', user });
|
||||
|
||||
const xmlText = (
|
||||
doc.getXmlFragment('default').get(0) as Y.XmlElement
|
||||
).get(0) as Y.XmlText;
|
||||
expect(xmlText.toDelta()).toEqual([
|
||||
{
|
||||
insert: 'keep me',
|
||||
attributes: { comment: { commentId: 'other', resolved: false } },
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -6,6 +6,7 @@ import {
|
||||
tiptapExtensions,
|
||||
} from './collaboration.util';
|
||||
import {
|
||||
removeYjsMarkByAttribute,
|
||||
replaceYjsMarkedText,
|
||||
setYjsMark,
|
||||
updateYjsMarkAttribute,
|
||||
@@ -82,6 +83,40 @@ export class CollaborationHandler {
|
||||
},
|
||||
);
|
||||
},
|
||||
deleteCommentMark: async (
|
||||
documentName: string,
|
||||
payload: {
|
||||
commentId: string;
|
||||
user: User;
|
||||
},
|
||||
) => {
|
||||
const { commentId, user } = payload;
|
||||
// Ephemeral suggestions (#329): when a suggestion-edit is dismissed or an
|
||||
// applied one has no replies, the comment is hard-deleted and its inline
|
||||
// anchor must vanish too. Mirror resolveCommentMark exactly, but instead
|
||||
// of flipping the mark's `resolved` attribute we STRIP the `comment` mark
|
||||
// entirely via removeYjsMarkByAttribute so no orphan highlight remains in
|
||||
// the collaborative document.
|
||||
//
|
||||
// Routing this through collaboration.gateway's handleYjsEvent means the
|
||||
// COLLAB_DISABLE_REDIS path invokes this handler directly (never a silent
|
||||
// no-op) and a missing live instance is a hard error — the same guarantee
|
||||
// applyCommentSuggestion/resolveCommentMark rely on.
|
||||
await this.withYdocConnection(
|
||||
hocuspocus,
|
||||
documentName,
|
||||
{ user },
|
||||
(doc) => {
|
||||
const fragment = doc.getXmlFragment('default');
|
||||
removeYjsMarkByAttribute(
|
||||
fragment,
|
||||
'comment',
|
||||
'commentId',
|
||||
commentId,
|
||||
);
|
||||
},
|
||||
);
|
||||
},
|
||||
applyCommentSuggestion: async (
|
||||
documentName: string,
|
||||
payload: {
|
||||
|
||||
@@ -464,26 +464,23 @@ describe('git-sync converter §13.1 idempotency gate (editor-ext schema)', () =>
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// KNOWN DIVERGENCE — images (isolated so it does NOT silently weaken the gate).
|
||||
// Image layout attrs — width/height now PRESERVED by canon #4, align is a
|
||||
// RESIDUAL divergence (isolated so it does NOT silently weaken the gate).
|
||||
//
|
||||
// This is NOT a schema-name divergence: the `image` NODE itself round-trips
|
||||
// through editor-ext fine (it survives toYdoc under the real tiptapExtensions).
|
||||
// The loss is intrinsic to MARKDOWN, the on-disk transport format git-sync uses:
|
||||
// The `image` NODE round-trips through editor-ext fine. Plain markdown ``
|
||||
// has no way to express layout attrs, so the canonical converter (#293/#326
|
||||
// canon decision #4) appends a machine comment `<!--img {...}-->` carrying the
|
||||
// non-default attrs, and re-parses it on import — the same trailing-comment
|
||||
// pattern used for media/textAlign. This closes the former width/height loss.
|
||||
//
|
||||
// 1. `convertProseMirrorToMarkdown` emits a standard `` image
|
||||
// (markdown-converter.ts case "image"). Standard markdown image syntax has
|
||||
// no way to express `width` / `height` / `align`, so those attrs are
|
||||
// DROPPED on export and cannot be recovered on import.
|
||||
// 2. A block-level image is hoisted out of its line by the HTML re-parser,
|
||||
// leaving a leading EMPTY paragraph (the same block-image-hoist limitation
|
||||
// documented in packages/git-sync/test/fixtures/known-limitations).
|
||||
//
|
||||
// The gate documents the EXACT lossy shape below. If the converter is ever
|
||||
// taught to preserve image dimensions (e.g. by emitting an HTML <img> with
|
||||
// data-* attrs, as it already does for video/diagrams), these assertions flip
|
||||
// and the image fixture should be promoted into the green CORPUS above.
|
||||
// RESIDUAL GAP (must be fixed in the converter PACKAGE on develop, fixtures-first
|
||||
// — never on this branch, which no longer owns a converter copy): canon #4
|
||||
// currently serializes only `width`/`height` into `<!--img {...}-->`, NOT
|
||||
// `align`. So an image `align` is still dropped across the round trip. Locked
|
||||
// below as the exact current shape; when the package learns to carry `align`,
|
||||
// this assertion flips to `toBe('center')`.
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('git-sync converter §13.1 image dimensions preserved (was KNOWN DIVERGENCE)', () => {
|
||||
describe('git-sync converter §13.1 image width/height preserved (align: residual divergence)', () => {
|
||||
const imageDoc = doc({
|
||||
type: 'image',
|
||||
attrs: {
|
||||
@@ -494,26 +491,28 @@ describe('git-sync converter §13.1 image dimensions preserved (was KNOWN DIVERG
|
||||
},
|
||||
});
|
||||
|
||||
it('preserves width/height/align by exporting an HTML <img> (PR #119 round-trip fix)', async () => {
|
||||
it('preserves width/height via the canon `<!--img {...}-->` comment; align still drops', async () => {
|
||||
const { md, canonNormalized } = await runGate(imageDoc);
|
||||
|
||||
// A top-level image carrying layout attrs is now exported as a schema-
|
||||
// matching HTML <img> (the same path video/diagrams already use), so the
|
||||
// dimensions and alignment survive the round trip instead of collapsing to
|
||||
// bare ``.
|
||||
// Canon #4: bare `` plus a trailing `<!--img {...}-->` comment that
|
||||
// carries the non-default layout attrs (currently width/height only), so the
|
||||
// dimensions survive the round trip instead of collapsing to bare ``.
|
||||
expect(md.trim()).toBe(
|
||||
'<img src="https://example.com/pic.png" width="640" height="480" align="center">',
|
||||
' <!--img {"width":"640","height":"480"}-->',
|
||||
);
|
||||
|
||||
// The round-tripped image keeps src + the layout attrs. width/height are
|
||||
// The round-tripped image keeps src + width/height. width/height are
|
||||
// re-imported as strings (matching the video/audio/pdf string convention),
|
||||
// so assert the values rather than the JS type.
|
||||
const imgAttrs = (canonNormalized as any).content[0].attrs;
|
||||
expect((canonNormalized as any).content[0].type).toBe('image');
|
||||
expect(imgAttrs.src).toBe('https://example.com/pic.png');
|
||||
expect(imgAttrs.align).toBe('center');
|
||||
expect(String(imgAttrs.width)).toBe('640');
|
||||
expect(String(imgAttrs.height)).toBe('480');
|
||||
// RESIDUAL GAP: canon #4 does not (yet) carry `align` in `<!--img {...}-->`,
|
||||
// so the original `align: 'center'` is lost. Fix belongs in the converter
|
||||
// package on develop (fixtures-first); flip to toBe('center') once landed.
|
||||
expect(imgAttrs.align).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -534,9 +533,10 @@ describe('git-sync converter §13.1 heading text alignment round-trips', () => {
|
||||
|
||||
const { md, canonNormalized } = await runGate(alignedHeading);
|
||||
|
||||
// Export is a styled <h2> (was a lossy bare `## centered heading`).
|
||||
// Canon #9: ATX heading plus a trailing `<!--attrs {...}-->` comment carrying
|
||||
// the non-default textAlign (was a lossy bare `## centered heading`).
|
||||
expect(md.trim()).toBe(
|
||||
'<h2 style="text-align:center">centered heading</h2>',
|
||||
'## centered heading <!--attrs {"textAlign":"center"}-->',
|
||||
);
|
||||
expect(docsCanonicallyEqual(alignedHeading, canonNormalized)).toBe(true);
|
||||
});
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
import { getSchema } from '@tiptap/core';
|
||||
import { Schema } from '@tiptap/pm/model';
|
||||
import { tiptapExtensions } from './collaboration.util';
|
||||
// The vendored git-sync mirror's extension set. Imported via the subpath the
|
||||
// server jest config maps to the package SOURCE (moduleNameMapper
|
||||
// `^@docmost/git-sync/(.*)$`), so this reads the real mirror, not a build.
|
||||
import { docmostExtensions as gitSyncExtensions } from '@docmost/git-sync/lib/docmost-schema';
|
||||
// The canonical converter mirror's extension set. The schema mirror now lives in
|
||||
// the single `@docmost/prosemirror-markdown` package (#293); the server jest
|
||||
// config maps it to the package SOURCE (moduleNameMapper
|
||||
// `^@docmost/prosemirror-markdown$`), so this reads the real mirror, not a build.
|
||||
import { docmostExtensions as gitSyncExtensions } from '@docmost/prosemirror-markdown';
|
||||
|
||||
/**
|
||||
* ATTRIBUTE-LEVEL SCHEMA CONTRACT (review #293, variant A).
|
||||
|
||||
@@ -52,6 +52,7 @@ export const AuditEvent = {
|
||||
COMMENT_RESOLVED: 'comment.resolved',
|
||||
COMMENT_REOPENED: 'comment.reopened',
|
||||
COMMENT_SUGGESTION_APPLIED: 'comment.suggestion_applied',
|
||||
COMMENT_SUGGESTION_DISMISSED: 'comment.suggestion_dismissed',
|
||||
|
||||
// Page
|
||||
PAGE_CREATED: 'page.created',
|
||||
|
||||
@@ -303,7 +303,9 @@ export class AiChatToolsService {
|
||||
getPage: tool({
|
||||
description:
|
||||
'Fetch a single page as Markdown by its page id. Returns the page ' +
|
||||
'title and its Markdown content.',
|
||||
'title and its Markdown content. Inline <span data-comment-id> tags ' +
|
||||
'in the markdown are comment highlight anchors (also present for ' +
|
||||
'RESOLVED threads) — treat them as markup, not page text.',
|
||||
inputSchema: modelFriendlyInput({
|
||||
pageId: z.string().describe('The id (or slugId) of the page.'),
|
||||
}),
|
||||
@@ -628,6 +630,16 @@ export class AiChatToolsService {
|
||||
async ({ pageId, nodeId }) => await client.getNode(pageId, nodeId),
|
||||
),
|
||||
|
||||
searchInPage: sharedTool(
|
||||
sharedToolSpecs.searchInPage,
|
||||
async ({ pageId, query, regex, caseSensitive, limit }) =>
|
||||
await client.searchInPage(pageId, query, {
|
||||
regex,
|
||||
caseSensitive,
|
||||
limit,
|
||||
}),
|
||||
),
|
||||
|
||||
getTable: tool({
|
||||
description:
|
||||
'Read a table as a matrix of cell texts (plus a parallel cellIds ' +
|
||||
@@ -647,11 +659,21 @@ export class AiChatToolsService {
|
||||
|
||||
listComments: tool({
|
||||
description:
|
||||
'List all comments on a page (content as Markdown).',
|
||||
'List comments on a page in one call. By DEFAULT only ACTIVE ' +
|
||||
'threads are returned; resolved threads (a resolved top-level ' +
|
||||
'comment and all its replies) are hidden and their count reported ' +
|
||||
'as `resolvedThreadsHidden` so you can re-query with ' +
|
||||
'`includeResolved: true` to see everything. Returns ' +
|
||||
'`{ items, resolvedThreadsHidden }`. Content is returned as Markdown.',
|
||||
inputSchema: modelFriendlyInput({
|
||||
pageId: z.string().describe('The id of the page.'),
|
||||
includeResolved: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe('default only active threads; true — include resolved'),
|
||||
}),
|
||||
execute: async ({ pageId }) => await client.listComments(pageId),
|
||||
execute: async ({ pageId, includeResolved }) =>
|
||||
await client.listComments(pageId, includeResolved),
|
||||
}),
|
||||
|
||||
getComment: tool({
|
||||
|
||||
@@ -56,8 +56,18 @@ export interface DocmostClientLike {
|
||||
getOutline(pageId: string): Promise<Record<string, unknown>>;
|
||||
getPageJson(pageId: string): Promise<Record<string, unknown>>;
|
||||
getNode(pageId: string, nodeId: string): Promise<Record<string, unknown>>;
|
||||
searchInPage(
|
||||
pageId: string,
|
||||
query: string,
|
||||
opts?: { regex?: boolean; caseSensitive?: boolean; limit?: number },
|
||||
): Promise<Record<string, unknown>>;
|
||||
getTable(pageId: string, tableRef: string): Promise<Record<string, unknown>>;
|
||||
listComments(pageId: string): Promise<unknown[]>;
|
||||
// Returns `{ items, resolvedThreadsHidden }`. DEFAULT (includeResolved unset/
|
||||
// false) hides resolved threads wholesale; pass true for the full feed.
|
||||
listComments(
|
||||
pageId: string,
|
||||
includeResolved?: boolean,
|
||||
): Promise<{ items: unknown[]; resolvedThreadsHidden: number }>;
|
||||
getComment(
|
||||
commentId: string,
|
||||
): Promise<{ data: Record<string, unknown>; success: boolean }>;
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import {
|
||||
BadRequestException,
|
||||
ForbiddenException,
|
||||
NotFoundException,
|
||||
} from '@nestjs/common';
|
||||
@@ -117,3 +118,207 @@ describe('CommentController apply-suggestion authz', () => {
|
||||
expect(commentService.applySuggestion).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* Authz-gate tests for the dismiss-suggestion route (#329). Dismissing a
|
||||
* suggestion does NOT change the page text, so it authorizes with
|
||||
* validateCanComment (NOT validateCanEdit) — a viewer allowed to comment but not
|
||||
* edit can still dismiss. The gate MUST run BEFORE the service (which performs
|
||||
* the delete/resolve + mark removal). These tests pin that boundary.
|
||||
*/
|
||||
describe('CommentController dismiss-suggestion authz', () => {
|
||||
// isAdmin=false → ability.cannot(Manage, Settings) returns true (i.e. the user
|
||||
// is NOT a space admin). Flip to true to model a space admin.
|
||||
function makeController(isAdmin = false) {
|
||||
const commentService = {
|
||||
dismissSuggestion: jest.fn(async () => ({
|
||||
id: 'c-1',
|
||||
outcome: 'deleted',
|
||||
})),
|
||||
};
|
||||
const commentRepo = { findById: jest.fn() };
|
||||
const pageRepo = { findById: jest.fn() };
|
||||
const spaceAbility = {
|
||||
createForUser: jest.fn(async () => ({
|
||||
cannot: jest.fn(() => !isAdmin),
|
||||
})),
|
||||
} as any;
|
||||
const pageAccessService = {
|
||||
validateCanComment: jest.fn(async () => undefined),
|
||||
validateCanEdit: jest.fn(async () => undefined),
|
||||
};
|
||||
const wsService = {} as any;
|
||||
const auditService = { log: jest.fn() };
|
||||
|
||||
const controller = new CommentController(
|
||||
commentService as any,
|
||||
commentRepo as any,
|
||||
pageRepo as any,
|
||||
spaceAbility,
|
||||
pageAccessService as any,
|
||||
wsService,
|
||||
auditService as any,
|
||||
);
|
||||
return {
|
||||
controller,
|
||||
commentService,
|
||||
commentRepo,
|
||||
pageRepo,
|
||||
pageAccessService,
|
||||
spaceAbility,
|
||||
};
|
||||
}
|
||||
|
||||
const user: any = { id: 'u-1' };
|
||||
const workspace: any = { id: 'ws-1' };
|
||||
const provenance: any = undefined;
|
||||
const dto: any = { commentId: 'c-1' };
|
||||
// Owned by the acting user (u-1) unless a test overrides creatorId.
|
||||
const comment = {
|
||||
id: 'c-1',
|
||||
pageId: 'p-1',
|
||||
spaceId: 'sp-1',
|
||||
creatorId: 'u-1',
|
||||
suggestedText: 'new text',
|
||||
selection: 'old text',
|
||||
};
|
||||
const page = { id: 'p-1', spaceId: 'sp-1', deletedAt: null };
|
||||
|
||||
it('authorizes with validateCanComment (NOT validateCanEdit) then calls the service', async () => {
|
||||
const {
|
||||
controller,
|
||||
commentRepo,
|
||||
pageRepo,
|
||||
pageAccessService,
|
||||
commentService,
|
||||
} = makeController();
|
||||
commentRepo.findById.mockResolvedValue(comment);
|
||||
pageRepo.findById.mockResolvedValue(page);
|
||||
const dismissed = { id: 'c-1', outcome: 'deleted' };
|
||||
commentService.dismissSuggestion.mockResolvedValue(dismissed);
|
||||
|
||||
const result = await controller.dismissSuggestion(
|
||||
dto,
|
||||
user,
|
||||
workspace,
|
||||
provenance,
|
||||
);
|
||||
|
||||
expect(pageAccessService.validateCanComment).toHaveBeenCalledWith(
|
||||
page,
|
||||
user,
|
||||
workspace.id,
|
||||
);
|
||||
// Dismiss must NOT require edit access.
|
||||
expect(pageAccessService.validateCanEdit).not.toHaveBeenCalled();
|
||||
expect(commentService.dismissSuggestion).toHaveBeenCalledWith(
|
||||
comment,
|
||||
user,
|
||||
provenance,
|
||||
);
|
||||
expect(result).toBe(dismissed);
|
||||
});
|
||||
|
||||
it('validateCanComment throwing Forbidden rejects AND dismissSuggestion is never called', async () => {
|
||||
const {
|
||||
controller,
|
||||
commentRepo,
|
||||
pageRepo,
|
||||
pageAccessService,
|
||||
commentService,
|
||||
} = makeController();
|
||||
commentRepo.findById.mockResolvedValue(comment);
|
||||
pageRepo.findById.mockResolvedValue(page);
|
||||
pageAccessService.validateCanComment.mockRejectedValue(
|
||||
new ForbiddenException('no comment access'),
|
||||
);
|
||||
|
||||
await expect(
|
||||
controller.dismissSuggestion(dto, user, workspace, provenance),
|
||||
).rejects.toBeInstanceOf(ForbiddenException);
|
||||
|
||||
expect(commentService.dismissSuggestion).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('missing comment: NotFound without authorizing or dismissing', async () => {
|
||||
const { controller, commentRepo, pageRepo, pageAccessService, commentService } =
|
||||
makeController();
|
||||
commentRepo.findById.mockResolvedValue(null);
|
||||
|
||||
await expect(
|
||||
controller.dismissSuggestion(dto, user, workspace, provenance),
|
||||
).rejects.toBeInstanceOf(NotFoundException);
|
||||
|
||||
expect(pageRepo.findById).not.toHaveBeenCalled();
|
||||
expect(pageAccessService.validateCanComment).not.toHaveBeenCalled();
|
||||
expect(commentService.dismissSuggestion).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('propagates a service BadRequest (e.g. already applied/resolved) unchanged', async () => {
|
||||
const { controller, commentRepo, pageRepo, commentService } =
|
||||
makeController();
|
||||
commentRepo.findById.mockResolvedValue(comment);
|
||||
pageRepo.findById.mockResolvedValue(page);
|
||||
commentService.dismissSuggestion.mockRejectedValue(
|
||||
new BadRequestException('already applied'),
|
||||
);
|
||||
|
||||
await expect(
|
||||
controller.dismissSuggestion(dto, user, workspace, provenance),
|
||||
).rejects.toBeInstanceOf(BadRequestException);
|
||||
});
|
||||
|
||||
// --- #338 owner-or-space-admin gate (mirrors POST /comments/delete) --------
|
||||
// A childless dismiss irreversibly hard-deletes the comment, so canComment is
|
||||
// not enough: only the comment owner or a space admin may dismiss.
|
||||
|
||||
it('owner dismisses their own suggestion → allowed, no admin check needed', async () => {
|
||||
const { controller, commentRepo, pageRepo, commentService, spaceAbility } =
|
||||
makeController(false);
|
||||
// comment.creatorId === user.id (owner).
|
||||
commentRepo.findById.mockResolvedValue(comment);
|
||||
pageRepo.findById.mockResolvedValue(page);
|
||||
|
||||
await controller.dismissSuggestion(dto, user, workspace, provenance);
|
||||
|
||||
// Owner short-circuits the admin lookup.
|
||||
expect(spaceAbility.createForUser).not.toHaveBeenCalled();
|
||||
expect(commentService.dismissSuggestion).toHaveBeenCalledWith(
|
||||
comment,
|
||||
user,
|
||||
provenance,
|
||||
);
|
||||
});
|
||||
|
||||
it('non-owner non-admin → Forbidden AND the service is never called', async () => {
|
||||
const { controller, commentRepo, pageRepo, commentService, spaceAbility } =
|
||||
makeController(false); // NOT a space admin
|
||||
commentRepo.findById.mockResolvedValue({
|
||||
...comment,
|
||||
creatorId: 'someone-else',
|
||||
});
|
||||
pageRepo.findById.mockResolvedValue(page);
|
||||
|
||||
await expect(
|
||||
controller.dismissSuggestion(dto, user, workspace, provenance),
|
||||
).rejects.toBeInstanceOf(ForbiddenException);
|
||||
|
||||
expect(spaceAbility.createForUser).toHaveBeenCalledWith(user, comment.spaceId);
|
||||
expect(commentService.dismissSuggestion).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('non-owner space admin → allowed to dismiss another user’s suggestion', async () => {
|
||||
const { controller, commentRepo, pageRepo, commentService, spaceAbility } =
|
||||
makeController(true); // space admin
|
||||
commentRepo.findById.mockResolvedValue({
|
||||
...comment,
|
||||
creatorId: 'someone-else',
|
||||
});
|
||||
pageRepo.findById.mockResolvedValue(page);
|
||||
|
||||
await controller.dismissSuggestion(dto, user, workspace, provenance);
|
||||
|
||||
expect(spaceAbility.createForUser).toHaveBeenCalledWith(user, comment.spaceId);
|
||||
expect(commentService.dismissSuggestion).toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -15,6 +15,7 @@ import { CreateCommentDto } from './dto/create-comment.dto';
|
||||
import { UpdateCommentDto } from './dto/update-comment.dto';
|
||||
import { ResolveCommentDto } from './dto/resolve-comment.dto';
|
||||
import { ApplySuggestionDto } from './dto/apply-suggestion.dto';
|
||||
import { DismissSuggestionDto } from './dto/dismiss-suggestion.dto';
|
||||
import { PageIdDto, CommentIdDto } from './dto/comments.input';
|
||||
import { AuthUser } from '../../common/decorators/auth-user.decorator';
|
||||
import { AuthWorkspace } from '../../common/decorators/auth-workspace.decorator';
|
||||
@@ -234,6 +235,59 @@ export class CommentController {
|
||||
return this.commentService.applySuggestion(comment, user, provenance);
|
||||
}
|
||||
|
||||
@HttpCode(HttpStatus.OK)
|
||||
@Post('dismiss-suggestion')
|
||||
async dismissSuggestion(
|
||||
@Body() dto: DismissSuggestionDto,
|
||||
@AuthUser() user: User,
|
||||
@AuthWorkspace() workspace: Workspace,
|
||||
@AuthProvenance() provenance: AuthProvenanceData,
|
||||
) {
|
||||
const comment = await this.commentRepo.findById(dto.commentId, {
|
||||
includeCreator: true,
|
||||
includeResolvedBy: true,
|
||||
});
|
||||
if (!comment) {
|
||||
throw new NotFoundException('Comment not found');
|
||||
}
|
||||
|
||||
const page = await this.pageRepo.findById(comment.pageId);
|
||||
if (!page || page.deletedAt) {
|
||||
throw new NotFoundException('Page not found');
|
||||
}
|
||||
|
||||
// Authorize BEFORE revealing any structural detail (metadata-disclosure
|
||||
// hygiene, mirroring apply-suggestion). Dismissing a suggestion does NOT
|
||||
// change the page text — it only removes/resolves the comment — so the
|
||||
// page-level gate is comment access (canComment), NOT edit access. A viewer
|
||||
// allowed to comment but not edit can still dismiss their own suggestion.
|
||||
// The structural 400s (top-level / has-a-suggested-edit / not applied /
|
||||
// not resolved) are re-checked by the service below.
|
||||
await this.pageAccessService.validateCanComment(page, user, workspace.id);
|
||||
|
||||
// AUTHZ (#338): a childless dismiss IRREVERSIBLY hard-deletes the comment,
|
||||
// so — beyond canComment — restrict it to the comment owner OR a space
|
||||
// admin, exactly like POST /comments/delete. canComment alone is not enough:
|
||||
// it would let any bystander commenter erase another user's suggestion for
|
||||
// good. (apply-suggestion deliberately stays on canEdit: accepting an edit
|
||||
// is the editor's semantics, not the suggestion author's.)
|
||||
const isOwner = comment.creatorId === user.id;
|
||||
if (!isOwner) {
|
||||
const ability = await this.spaceAbility.createForUser(
|
||||
user,
|
||||
comment.spaceId,
|
||||
);
|
||||
// Space admin can dismiss any suggestion.
|
||||
if (ability.cannot(SpaceCaslAction.Manage, SpaceCaslSubject.Settings)) {
|
||||
throw new ForbiddenException(
|
||||
'You can only dismiss your own suggestions',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return this.commentService.dismissSuggestion(comment, user, provenance);
|
||||
}
|
||||
|
||||
@HttpCode(HttpStatus.OK)
|
||||
@Post('delete')
|
||||
async delete(@Body() input: CommentIdDto, @AuthUser() user: User, @AuthWorkspace() workspace: Workspace) {
|
||||
|
||||
@@ -13,17 +13,27 @@ import { AuditEvent, AuditResource } from '../../common/events/audit-events';
|
||||
*
|
||||
* The collaboration gateway verdict is the pivot of the whole flow, so each test
|
||||
* pins a specific { applied, currentText } and asserts the DB persistence,
|
||||
* auto-resolve, audit, ws broadcast, and error mapping that follow from it.
|
||||
* settle (ephemeral delete vs. resolve), audit, ws broadcast, and error mapping
|
||||
* that follow from it.
|
||||
*
|
||||
* Ephemeral rule (#329): once applied a suggestion DISAPPEARS (hard-delete +
|
||||
* strip the inline anchor mark) UNLESS the thread has replies, in which case it
|
||||
* is resolved to preserve the discussion. `hasChildren` selects the branch.
|
||||
*/
|
||||
describe('CommentService — applySuggestion', () => {
|
||||
const UPDATED = { id: 'c-1', __updated: true } as any;
|
||||
|
||||
function makeService(verdict: unknown) {
|
||||
function makeService(verdict: unknown, hasChildren = false, deletedRows = 1) {
|
||||
const commentRepo: any = {
|
||||
// Both the applied-stamp re-read and resolveComment's re-read go through
|
||||
// findById; return a recognizable enriched row.
|
||||
findById: jest.fn(async () => UPDATED),
|
||||
updateComment: jest.fn(async () => undefined),
|
||||
hasChildren: jest.fn(async () => hasChildren),
|
||||
deleteComment: jest.fn(async () => undefined),
|
||||
// #338 F1: the childless ephemeral delete is atomic-conditional and
|
||||
// returns the number of rows removed (1 = deleted, 0 = a reply raced in).
|
||||
deleteCommentIfChildless: jest.fn(async () => deletedRows),
|
||||
};
|
||||
const pageRepo: any = {};
|
||||
const wsService: any = { emitCommentEvent: jest.fn() };
|
||||
@@ -74,7 +84,9 @@ describe('CommentService — applySuggestion', () => {
|
||||
.map((c: any[]) => c[0])
|
||||
.find((patch: any) => 'suggestionAppliedAt' in patch);
|
||||
|
||||
it('applied=true → replaces text, persists applied stamps, auto-resolves, audits, returns updated', async () => {
|
||||
// --- no replies → ephemeral delete branch -------------------------------
|
||||
|
||||
it('applied=true, no replies → replaces text, hard-deletes, strips the anchor mark, audits APPLIED, outcome=deleted', async () => {
|
||||
const { service, commentRepo, wsService, collaborationGateway, auditService } =
|
||||
makeService({ applied: true, currentText: 'new text' });
|
||||
|
||||
@@ -92,37 +104,34 @@ describe('CommentService — applySuggestion', () => {
|
||||
}),
|
||||
);
|
||||
|
||||
// Applied stamps persisted.
|
||||
const patch = appliedPatch(commentRepo);
|
||||
expect(patch.suggestionAppliedAt).toBeInstanceOf(Date);
|
||||
expect(patch.suggestionAppliedById).toBe('user-1');
|
||||
// Ephemeral: the redundant comment is hard-deleted (atomic-conditional) and
|
||||
// its inline anchor mark removed via the deleteCommentMark collab event.
|
||||
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
|
||||
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
|
||||
'deleteCommentMark',
|
||||
'page.page-1',
|
||||
expect.objectContaining({ commentId: 'c-1', user: expect.any(Object) }),
|
||||
);
|
||||
// No applied stamps are written for a row about to be deleted.
|
||||
expect(appliedPatch(commentRepo)).toBeUndefined();
|
||||
|
||||
// Auto-resolved: resolveComment writes a resolvedAt/resolvedById patch too.
|
||||
const resolvePatch = commentRepo.updateComment.mock.calls
|
||||
.map((c: any[]) => c[0])
|
||||
.find((p: any) => 'resolvedAt' in p);
|
||||
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
|
||||
expect(resolvePatch.resolvedById).toBe('user-1');
|
||||
|
||||
// Audit + broadcast + return.
|
||||
// Broadcast a deletion, audit the (still-applied) suggestion, report outcome.
|
||||
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
|
||||
'space-1',
|
||||
'page-1',
|
||||
expect.objectContaining({ operation: 'commentDeleted', commentId: 'c-1' }),
|
||||
);
|
||||
expect(auditService.log).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
|
||||
resourceType: AuditResource.COMMENT,
|
||||
resourceId: 'c-1',
|
||||
spaceId: 'space-1',
|
||||
metadata: { pageId: 'page-1' },
|
||||
}),
|
||||
);
|
||||
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
|
||||
'space-1',
|
||||
'page-1',
|
||||
expect.objectContaining({ operation: 'commentUpdated', comment: UPDATED }),
|
||||
);
|
||||
expect(result).toBe(UPDATED);
|
||||
expect(result.outcome).toBe('deleted');
|
||||
});
|
||||
|
||||
it('applied=false but currentText === suggestedText → idempotent success (no 409)', async () => {
|
||||
it('applied=false but currentText === suggestedText, no replies → idempotent delete (no 409)', async () => {
|
||||
const { service, commentRepo, auditService } = makeService({
|
||||
applied: false,
|
||||
currentText: 'new text',
|
||||
@@ -130,15 +139,55 @@ describe('CommentService — applySuggestion', () => {
|
||||
|
||||
const result = await service.applySuggestion(suggestionComment(), user());
|
||||
|
||||
// The stamps are still persisted (reconciling a crash between the doc
|
||||
// mutation and the DB write) and the call succeeds.
|
||||
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
|
||||
expect(auditService.log).toHaveBeenCalledTimes(1);
|
||||
expect(result.outcome).toBe('deleted');
|
||||
});
|
||||
|
||||
// --- has replies → resolve branch (discussion preserved) ----------------
|
||||
|
||||
it('applied=true, WITH replies → resolves (not delete), persists applied stamps, audits, outcome=resolved', async () => {
|
||||
const { service, commentRepo, wsService, collaborationGateway, auditService } =
|
||||
makeService({ applied: true, currentText: 'new text' }, true);
|
||||
|
||||
const result = await service.applySuggestion(suggestionComment(), user());
|
||||
|
||||
// Applied stamps persisted.
|
||||
const patch = appliedPatch(commentRepo);
|
||||
expect(patch.suggestionAppliedAt).toBeInstanceOf(Date);
|
||||
expect(patch.suggestionAppliedById).toBe('user-1');
|
||||
expect(auditService.log).toHaveBeenCalledTimes(1);
|
||||
expect(result).toBe(UPDATED);
|
||||
|
||||
// Auto-resolved (resolveComment writes the resolve patch + resolve mark).
|
||||
const resolvePatch = commentRepo.updateComment.mock.calls
|
||||
.map((c: any[]) => c[0])
|
||||
.find((p: any) => 'resolvedAt' in p);
|
||||
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
|
||||
expect(resolvePatch.resolvedById).toBe('user-1');
|
||||
|
||||
// NOT deleted; broadcast an update, not a deletion.
|
||||
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
|
||||
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalledWith(
|
||||
'deleteCommentMark',
|
||||
expect.anything(),
|
||||
expect.anything(),
|
||||
);
|
||||
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
|
||||
'space-1',
|
||||
'page-1',
|
||||
expect.objectContaining({ operation: 'commentUpdated', comment: UPDATED }),
|
||||
);
|
||||
|
||||
expect(auditService.log).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
|
||||
}),
|
||||
);
|
||||
expect(result.id).toBe('c-1');
|
||||
expect(result.outcome).toBe('resolved');
|
||||
});
|
||||
|
||||
// --- error / rejection branches -----------------------------------------
|
||||
|
||||
it('applied=false and currentText differs → ConflictException with currentText in payload', async () => {
|
||||
const { service, commentRepo, auditService } = makeService({
|
||||
applied: false,
|
||||
@@ -153,14 +202,14 @@ describe('CommentService — applySuggestion', () => {
|
||||
expect(err.getResponse()).toMatchObject({
|
||||
currentText: 'someone else edited this',
|
||||
});
|
||||
// No persistence and no audit on a conflict.
|
||||
expect(appliedPatch(commentRepo)).toBeUndefined();
|
||||
// No delete and no audit on a conflict.
|
||||
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
|
||||
expect(auditService.log).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('already-applied AND already-resolved → idempotent success, no collab call, no re-resolve (#315 double-click)', async () => {
|
||||
it('already-applied WITH replies → idempotent success, no re-apply, resolve branch', async () => {
|
||||
const { service, collaborationGateway, commentRepo, auditService } =
|
||||
makeService({ applied: true, currentText: 'new text' });
|
||||
makeService({ applied: true, currentText: 'new text' }, true);
|
||||
|
||||
const result = await service.applySuggestion(
|
||||
suggestionComment({
|
||||
@@ -171,17 +220,20 @@ describe('CommentService — applySuggestion', () => {
|
||||
user(),
|
||||
);
|
||||
|
||||
// Idempotent SUCCESS, not a 409. The suggestion is already applied, so the
|
||||
// collaborative document is never touched again and nothing is re-stamped
|
||||
// or re-resolved.
|
||||
expect(result).toBe(UPDATED);
|
||||
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalled();
|
||||
expect(commentRepo.updateComment).not.toHaveBeenCalled();
|
||||
// Same success shape as the applied path (broadcast + audit).
|
||||
// Idempotent SUCCESS. The suggestion is already applied, so the document is
|
||||
// never re-mutated (no applyCommentSuggestion) and nothing is re-stamped.
|
||||
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalledWith(
|
||||
'applyCommentSuggestion',
|
||||
expect.anything(),
|
||||
expect.anything(),
|
||||
);
|
||||
expect(appliedPatch(commentRepo)).toBeUndefined();
|
||||
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
|
||||
expect(auditService.log).toHaveBeenCalledTimes(1);
|
||||
expect(result.outcome).toBe('resolved');
|
||||
});
|
||||
|
||||
it('already-applied but NOT resolved (crash window) → idempotent success, self-heals resolve, no re-apply', async () => {
|
||||
it('already-applied, no replies (double-click after a delete) → deletes idempotently', async () => {
|
||||
const { service, collaborationGateway, commentRepo } = makeService({
|
||||
applied: true,
|
||||
currentText: 'new text',
|
||||
@@ -192,28 +244,43 @@ describe('CommentService — applySuggestion', () => {
|
||||
user(),
|
||||
);
|
||||
|
||||
expect(result).toBe(UPDATED);
|
||||
|
||||
// The suggestion is NOT re-applied to the document…
|
||||
// No re-apply to the document; the childless applied comment is removed.
|
||||
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalledWith(
|
||||
'applyCommentSuggestion',
|
||||
expect.anything(),
|
||||
expect.anything(),
|
||||
);
|
||||
// …but the open thread is self-healed to resolved via resolveComment, which
|
||||
// writes the resolve patch and updates the resolve mark.
|
||||
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
|
||||
expect(result.outcome).toBe('deleted');
|
||||
});
|
||||
|
||||
it('applied=true, no replies at read time but a reply races in (conditional delete → 0 rows) → resolves instead, no hard-delete, outcome=resolved (#338 F1)', async () => {
|
||||
// The suggested text is already applied to the document, but between the
|
||||
// hasChildren read and the atomic delete a reply landed. The parent must NOT
|
||||
// be hard-deleted (cascade would destroy the reply); resolve the thread.
|
||||
const { service, commentRepo, wsService, collaborationGateway } =
|
||||
makeService({ applied: true, currentText: 'new text' }, false, 0);
|
||||
|
||||
const result = await service.applySuggestion(suggestionComment(), user());
|
||||
|
||||
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
|
||||
// No deletion broadcast — the row + the racing reply survive.
|
||||
expect(wsService.emitCommentEvent).not.toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
expect.anything(),
|
||||
expect.objectContaining({ operation: 'commentDeleted' }),
|
||||
);
|
||||
// Fell back to resolving.
|
||||
const resolvePatch = commentRepo.updateComment.mock.calls
|
||||
.map((c: any[]) => c[0])
|
||||
.find((p: any) => 'resolvedAt' in p);
|
||||
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
|
||||
expect(resolvePatch.resolvedById).toBe('user-1');
|
||||
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
|
||||
'resolveCommentMark',
|
||||
'page.page-1',
|
||||
expect.objectContaining({ commentId: 'c-1', resolved: true }),
|
||||
);
|
||||
// The applied stamps are NOT re-written (already stamped).
|
||||
expect(appliedPatch(commentRepo)).toBeUndefined();
|
||||
expect(result.outcome).toBe('resolved');
|
||||
});
|
||||
|
||||
it('rejects a comment with no suggestedText', async () => {
|
||||
@@ -238,8 +305,8 @@ describe('CommentService — applySuggestion', () => {
|
||||
service.applySuggestion(suggestionComment(), user()),
|
||||
).rejects.toThrow(InternalServerErrorException);
|
||||
|
||||
// Nothing persisted, nothing audited.
|
||||
expect(appliedPatch(commentRepo)).toBeUndefined();
|
||||
// Nothing deleted, nothing audited.
|
||||
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
|
||||
expect(auditService.log).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,229 @@
|
||||
import { BadRequestException } from '@nestjs/common';
|
||||
import { CommentService } from './comment.service';
|
||||
import { AuditEvent, AuditResource } from '../../common/events/audit-events';
|
||||
|
||||
/**
|
||||
* Coverage for CommentService.dismissSuggestion (#329). Dismiss ("Не применять")
|
||||
* removes a suggested edit WITHOUT changing the page text: the comment
|
||||
* disappears (hard-delete + strip the inline anchor mark) unless the thread has
|
||||
* replies, in which case it is resolved to preserve the discussion.
|
||||
*
|
||||
* The permission gate (canComment, NOT canEdit) lives in the controller and is
|
||||
* covered in comment.controller.spec.ts; here we pin the service's own state
|
||||
* guards and the delete-vs-resolve fork.
|
||||
*/
|
||||
describe('CommentService — dismissSuggestion', () => {
|
||||
const UPDATED = { id: 'c-1', __updated: true } as any;
|
||||
|
||||
function makeService(hasChildren = false, deletedRows = 1) {
|
||||
const commentRepo: any = {
|
||||
findById: jest.fn(async () => UPDATED),
|
||||
updateComment: jest.fn(async () => undefined),
|
||||
hasChildren: jest.fn(async () => hasChildren),
|
||||
deleteComment: jest.fn(async () => undefined),
|
||||
// #338 F1: the childless ephemeral delete is now atomic-conditional and
|
||||
// returns the number of rows removed (1 = deleted, 0 = a reply raced in).
|
||||
deleteCommentIfChildless: jest.fn(async () => deletedRows),
|
||||
};
|
||||
const pageRepo: any = {};
|
||||
const wsService: any = { emitCommentEvent: jest.fn() };
|
||||
const collaborationGateway: any = {
|
||||
handleYjsEvent: jest.fn(async () => undefined),
|
||||
};
|
||||
const generalQueue: any = { add: jest.fn(() => Promise.resolve()) };
|
||||
const notificationQueue: any = { add: jest.fn(async () => undefined) };
|
||||
const auditService: any = { log: jest.fn() };
|
||||
|
||||
const service = new CommentService(
|
||||
commentRepo,
|
||||
pageRepo,
|
||||
wsService,
|
||||
collaborationGateway,
|
||||
generalQueue,
|
||||
notificationQueue,
|
||||
auditService,
|
||||
);
|
||||
|
||||
return { service, commentRepo, wsService, collaborationGateway, auditService };
|
||||
}
|
||||
|
||||
const suggestionComment = (over?: Partial<any>): any => ({
|
||||
id: 'c-1',
|
||||
pageId: 'page-1',
|
||||
spaceId: 'space-1',
|
||||
workspaceId: 'ws-1',
|
||||
creatorId: 'user-1',
|
||||
parentCommentId: null,
|
||||
selection: 'old text',
|
||||
suggestedText: 'new text',
|
||||
suggestionAppliedAt: null,
|
||||
resolvedAt: null,
|
||||
...over,
|
||||
});
|
||||
const user = (over?: Partial<any>): any => ({ id: 'user-1', ...over });
|
||||
|
||||
it('no replies → hard-deletes, strips the anchor mark, does NOT touch page text, audits DISMISSED, outcome=deleted', async () => {
|
||||
const { service, commentRepo, wsService, collaborationGateway, auditService } =
|
||||
makeService(false);
|
||||
|
||||
const result = await service.dismissSuggestion(suggestionComment(), user());
|
||||
|
||||
// Never applies the suggestion to the document.
|
||||
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalledWith(
|
||||
'applyCommentSuggestion',
|
||||
expect.anything(),
|
||||
expect.anything(),
|
||||
);
|
||||
// Hard-delete (atomic-conditional) + strip mark.
|
||||
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
|
||||
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
|
||||
'deleteCommentMark',
|
||||
'page.page-1',
|
||||
expect.objectContaining({ commentId: 'c-1', user: expect.any(Object) }),
|
||||
);
|
||||
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
|
||||
'space-1',
|
||||
'page-1',
|
||||
expect.objectContaining({ operation: 'commentDeleted', commentId: 'c-1' }),
|
||||
);
|
||||
expect(auditService.log).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
event: AuditEvent.COMMENT_SUGGESTION_DISMISSED,
|
||||
resourceType: AuditResource.COMMENT,
|
||||
resourceId: 'c-1',
|
||||
}),
|
||||
);
|
||||
expect(result.outcome).toBe('deleted');
|
||||
});
|
||||
|
||||
it('no replies → if the anchor-mark removal FAILS, the row is NOT deleted and the error propagates (#329: no orphan anchor)', async () => {
|
||||
const { service, commentRepo, wsService, collaborationGateway } =
|
||||
makeService(false);
|
||||
// Mark removal is FATAL and runs BEFORE the irreversible row delete: a collab
|
||||
// failure (e.g. COLLAB_DISABLE_REDIS "no live instance") must abort the whole
|
||||
// operation, leaving row + mark consistent — never a deleted row with an
|
||||
// orphan anchor left in the document reporting success.
|
||||
collaborationGateway.handleYjsEvent = jest.fn(async () => {
|
||||
throw new Error('requires a live collaboration instance');
|
||||
});
|
||||
|
||||
await expect(
|
||||
service.dismissSuggestion(suggestionComment(), user()),
|
||||
).rejects.toThrow(/live collaboration/);
|
||||
|
||||
expect(commentRepo.deleteCommentIfChildless).not.toHaveBeenCalled();
|
||||
expect(wsService.emitCommentEvent).not.toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
expect.anything(),
|
||||
expect.objectContaining({ operation: 'commentDeleted' }),
|
||||
);
|
||||
});
|
||||
|
||||
it('WITH replies → resolves (not delete), does NOT apply, audits DISMISSED, outcome=resolved', async () => {
|
||||
const { service, commentRepo, wsService, collaborationGateway, auditService } =
|
||||
makeService(true);
|
||||
|
||||
const result = await service.dismissSuggestion(suggestionComment(), user());
|
||||
|
||||
// Resolved via resolveComment (resolve patch + resolve mark), NOT deleted.
|
||||
const resolvePatch = commentRepo.updateComment.mock.calls
|
||||
.map((c: any[]) => c[0])
|
||||
.find((p: any) => 'resolvedAt' in p);
|
||||
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
|
||||
expect(resolvePatch.resolvedById).toBe('user-1');
|
||||
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
|
||||
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
|
||||
'resolveCommentMark',
|
||||
'page.page-1',
|
||||
expect.objectContaining({ commentId: 'c-1', resolved: true }),
|
||||
);
|
||||
// No applied stamp — dismiss does not apply the edit.
|
||||
const appliedPatch = commentRepo.updateComment.mock.calls
|
||||
.map((c: any[]) => c[0])
|
||||
.find((p: any) => 'suggestionAppliedAt' in p);
|
||||
expect(appliedPatch).toBeUndefined();
|
||||
|
||||
expect(auditService.log).toHaveBeenCalledWith(
|
||||
expect.objectContaining({
|
||||
event: AuditEvent.COMMENT_SUGGESTION_DISMISSED,
|
||||
}),
|
||||
);
|
||||
expect(result.outcome).toBe('resolved');
|
||||
});
|
||||
|
||||
it('reply races in after the childless read (conditional delete → 0 rows) → resolves instead, does NOT hard-delete, reply survives, outcome=resolved (#338 F1)', async () => {
|
||||
// hasChildren=false selects the ephemeral branch (the read saw no replies),
|
||||
// but the atomic delete matches 0 rows because a reply landed in the window
|
||||
// between that read and the delete. The parent must NOT be hard-deleted
|
||||
// (a cascade would destroy the just-added reply); the thread is resolved.
|
||||
const { service, commentRepo, wsService, collaborationGateway } =
|
||||
makeService(false, 0);
|
||||
|
||||
const result = await service.dismissSuggestion(suggestionComment(), user());
|
||||
|
||||
// The conditional delete was attempted (and matched nothing).
|
||||
expect(commentRepo.deleteCommentIfChildless).toHaveBeenCalledWith('c-1');
|
||||
// No commentDeleted broadcast — the row (and the racing reply) survive.
|
||||
expect(wsService.emitCommentEvent).not.toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
expect.anything(),
|
||||
expect.objectContaining({ operation: 'commentDeleted' }),
|
||||
);
|
||||
// Fell back to resolving the thread.
|
||||
const resolvePatch = commentRepo.updateComment.mock.calls
|
||||
.map((c: any[]) => c[0])
|
||||
.find((p: any) => 'resolvedAt' in p);
|
||||
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
|
||||
expect(resolvePatch.resolvedById).toBe('user-1');
|
||||
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
|
||||
'resolveCommentMark',
|
||||
'page.page-1',
|
||||
expect.objectContaining({ commentId: 'c-1', resolved: true }),
|
||||
);
|
||||
expect(result.outcome).toBe('resolved');
|
||||
});
|
||||
|
||||
it('rejects a reply (non-top-level) comment', async () => {
|
||||
const { service, commentRepo } = makeService();
|
||||
await expect(
|
||||
service.dismissSuggestion(
|
||||
suggestionComment({ parentCommentId: 'parent-1' }),
|
||||
user(),
|
||||
),
|
||||
).rejects.toThrow(BadRequestException);
|
||||
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects a comment without a suggested edit', async () => {
|
||||
const { service, commentRepo } = makeService();
|
||||
await expect(
|
||||
service.dismissSuggestion(
|
||||
suggestionComment({ suggestedText: null }),
|
||||
user(),
|
||||
),
|
||||
).rejects.toThrow(BadRequestException);
|
||||
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects an already-applied suggestion', async () => {
|
||||
const { service, commentRepo } = makeService();
|
||||
await expect(
|
||||
service.dismissSuggestion(
|
||||
suggestionComment({ suggestionAppliedAt: new Date() }),
|
||||
user(),
|
||||
),
|
||||
).rejects.toThrow(BadRequestException);
|
||||
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects an already-resolved thread', async () => {
|
||||
const { service, commentRepo } = makeService();
|
||||
await expect(
|
||||
service.dismissSuggestion(
|
||||
suggestionComment({ resolvedAt: new Date() }),
|
||||
user(),
|
||||
),
|
||||
).rejects.toThrow(BadRequestException);
|
||||
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -35,6 +35,12 @@ import {
|
||||
IAuditService,
|
||||
} from '../../integrations/audit/audit.service';
|
||||
|
||||
// Ephemeral-suggestion settle result (#329): 'deleted' → the comment vanished
|
||||
// (hard-delete + anchor mark stripped); 'resolved' → the thread had replies and
|
||||
// was resolved instead. Returned to the client so it can pick the optimistic
|
||||
// cache action.
|
||||
export type SuggestionOutcome = 'deleted' | 'resolved';
|
||||
|
||||
@Injectable()
|
||||
export class CommentService {
|
||||
private readonly logger = new Logger(CommentService.name);
|
||||
@@ -362,7 +368,7 @@ export class CommentService {
|
||||
comment: Comment,
|
||||
user: User,
|
||||
provenance?: AuthProvenanceData,
|
||||
): Promise<Comment> {
|
||||
): Promise<Comment & { outcome: SuggestionOutcome }> {
|
||||
// Structural guards.
|
||||
if (comment.parentCommentId) {
|
||||
throw new BadRequestException(
|
||||
@@ -449,42 +455,148 @@ export class CommentService {
|
||||
}
|
||||
|
||||
/**
|
||||
* Persist the applied stamps (idempotently), auto-resolve the thread and
|
||||
* broadcast + audit the applied suggestion. Shared by the applied and the
|
||||
* Dismiss ("Не применять") a suggested edit without touching the page text:
|
||||
* the suggestion disappears. Ephemeral rule (#329) — a top-level suggestion
|
||||
* comment is transient UI, so dismissing it hard-deletes the comment AND strips
|
||||
* its inline anchor mark UNLESS the thread has replies, in which case the
|
||||
* discussion is preserved by resolving it instead.
|
||||
*
|
||||
* Dismiss does NOT change the document text, so the controller authorizes it
|
||||
* with canComment (NOT canEdit). This re-checks the comment's own state so the
|
||||
* invariant holds regardless of caller.
|
||||
*/
|
||||
async dismissSuggestion(
|
||||
comment: Comment,
|
||||
user: User,
|
||||
provenance?: AuthProvenanceData,
|
||||
): Promise<Comment & { outcome: SuggestionOutcome }> {
|
||||
// Structural guards (mirror applySuggestion).
|
||||
if (comment.parentCommentId) {
|
||||
throw new BadRequestException(
|
||||
'Only a top-level comment can carry a suggested edit',
|
||||
);
|
||||
}
|
||||
if (!comment.suggestedText) {
|
||||
throw new BadRequestException(
|
||||
'This comment has no suggested edit to dismiss',
|
||||
);
|
||||
}
|
||||
// State guards: dismissing an already-applied or already-resolved thread is
|
||||
// meaningless. On an apply↔dismiss race the loser sees the comment already
|
||||
// gone (404 at the controller) or already resolved (this 400); the client
|
||||
// treats both as "already resolved".
|
||||
if (comment.suggestionAppliedAt) {
|
||||
throw new BadRequestException(
|
||||
'Cannot dismiss a suggested edit that was already applied',
|
||||
);
|
||||
}
|
||||
if (comment.resolvedAt) {
|
||||
throw new BadRequestException(
|
||||
'Cannot dismiss a suggested edit on a resolved comment thread',
|
||||
);
|
||||
}
|
||||
|
||||
const hasChildren = await this.commentRepo.hasChildren(comment.id);
|
||||
|
||||
if (hasChildren) {
|
||||
// Preserve the discussion: resolve (never delete) a thread with replies.
|
||||
const updatedComment = await this.resolveComment(
|
||||
comment,
|
||||
true,
|
||||
user,
|
||||
provenance,
|
||||
);
|
||||
this.auditService.log({
|
||||
event: AuditEvent.COMMENT_SUGGESTION_DISMISSED,
|
||||
resourceType: AuditResource.COMMENT,
|
||||
resourceId: comment.id,
|
||||
spaceId: comment.spaceId,
|
||||
metadata: { pageId: comment.pageId },
|
||||
});
|
||||
return { ...updatedComment, outcome: 'resolved' };
|
||||
}
|
||||
|
||||
// Ephemeral: no replies → the suggestion vanishes entirely. The atomic
|
||||
// conditional delete may still fall back to a resolve if a reply raced in
|
||||
// (see deleteEphemeralSuggestion), so the outcome is whatever it settled on.
|
||||
const settled = await this.deleteEphemeralSuggestion(comment, user, provenance);
|
||||
this.auditService.log({
|
||||
event: AuditEvent.COMMENT_SUGGESTION_DISMISSED,
|
||||
resourceType: AuditResource.COMMENT,
|
||||
resourceId: comment.id,
|
||||
spaceId: comment.spaceId,
|
||||
metadata: { pageId: comment.pageId },
|
||||
});
|
||||
return settled;
|
||||
}
|
||||
|
||||
/**
|
||||
* Persist the applied stamps (idempotently), then settle the suggestion under
|
||||
* the ephemeral rule (#329): a suggestion whose thread has NO replies
|
||||
* DISAPPEARS after apply (hard-delete + strip the inline anchor mark), since
|
||||
* the suggested text is now in the document and a stand-alone resolved thread
|
||||
* would only pile up an orphan anchor. A thread WITH replies is preserved by
|
||||
* auto-resolving it (the historical behaviour). Shared by the applied and the
|
||||
* idempotent "already-applied" branches of applySuggestion.
|
||||
*
|
||||
* Returns the comment augmented with `outcome` so the client can pick the
|
||||
* optimistic action ('deleted' → drop it, 'resolved' → move to the resolved
|
||||
* tab).
|
||||
*/
|
||||
private async finalizeAppliedSuggestion(
|
||||
comment: Comment,
|
||||
user: User,
|
||||
provenance?: AuthProvenanceData,
|
||||
): Promise<Comment> {
|
||||
if (!comment.suggestionAppliedAt) {
|
||||
await this.commentRepo.updateComment(
|
||||
{
|
||||
suggestionAppliedAt: new Date(),
|
||||
suggestionAppliedById: user.id,
|
||||
},
|
||||
comment.id,
|
||||
);
|
||||
): Promise<Comment & { outcome: SuggestionOutcome }> {
|
||||
const hasChildren = await this.commentRepo.hasChildren(comment.id);
|
||||
|
||||
if (hasChildren) {
|
||||
// Thread has replies → preserve the discussion: stamp applied + resolve.
|
||||
if (!comment.suggestionAppliedAt) {
|
||||
await this.commentRepo.updateComment(
|
||||
{
|
||||
suggestionAppliedAt: new Date(),
|
||||
suggestionAppliedById: user.id,
|
||||
},
|
||||
comment.id,
|
||||
);
|
||||
}
|
||||
|
||||
// Auto-resolve the thread. resolveComment handles the resolve mark, its ws
|
||||
// broadcast and the resolve notification. Stay defensive on re-entry.
|
||||
if (!comment.resolvedAt) {
|
||||
await this.resolveComment(comment, true, user, provenance);
|
||||
}
|
||||
|
||||
const updatedComment = await this.commentRepo.findById(comment.id, {
|
||||
includeCreator: true,
|
||||
includeResolvedBy: true,
|
||||
});
|
||||
|
||||
this.wsService.emitCommentEvent(comment.spaceId, comment.pageId, {
|
||||
operation: 'commentUpdated',
|
||||
pageId: comment.pageId,
|
||||
comment: updatedComment,
|
||||
});
|
||||
|
||||
this.auditService.log({
|
||||
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
|
||||
resourceType: AuditResource.COMMENT,
|
||||
resourceId: comment.id,
|
||||
spaceId: comment.spaceId,
|
||||
metadata: { pageId: comment.pageId },
|
||||
});
|
||||
|
||||
return { ...updatedComment, outcome: 'resolved' };
|
||||
}
|
||||
|
||||
// Auto-resolve the thread. resolveComment handles the resolve mark, its ws
|
||||
// broadcast and the resolve notification. The guard above guarantees the
|
||||
// thread was open when we entered, but stay defensive on re-entry.
|
||||
if (!comment.resolvedAt) {
|
||||
await this.resolveComment(comment, true, user, provenance);
|
||||
}
|
||||
|
||||
const updatedComment = await this.commentRepo.findById(comment.id, {
|
||||
includeCreator: true,
|
||||
includeResolvedBy: true,
|
||||
});
|
||||
|
||||
this.wsService.emitCommentEvent(comment.spaceId, comment.pageId, {
|
||||
operation: 'commentUpdated',
|
||||
pageId: comment.pageId,
|
||||
comment: updatedComment,
|
||||
});
|
||||
// No replies → ephemeral: the suggested text is already in the document, so
|
||||
// the comment is redundant. Hard-delete it and strip its inline anchor. We
|
||||
// deliberately do NOT write the applied stamps first (the row is about to be
|
||||
// deleted); the audit event still records that the suggestion was applied.
|
||||
// The delete is atomic-conditional: if a reply raced in after the
|
||||
// hasChildren read, it falls back to resolving instead (outcome 'resolved').
|
||||
const settled = await this.deleteEphemeralSuggestion(comment, user, provenance);
|
||||
|
||||
this.auditService.log({
|
||||
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
|
||||
@@ -494,7 +606,86 @@ export class CommentService {
|
||||
metadata: { pageId: comment.pageId },
|
||||
});
|
||||
|
||||
return updatedComment;
|
||||
return settled;
|
||||
}
|
||||
|
||||
/**
|
||||
* Settle an ephemeral suggestion whose thread looked childless: remove its
|
||||
* inline `comment` anchor mark, then ATOMICALLY hard-delete the row only if it
|
||||
* is still childless. Shared by the apply/dismiss no-replies branches (#329).
|
||||
*
|
||||
* ORDER MATTERS: the anchor mark is removed FIRST and FATALLY (mirrors
|
||||
* applySuggestion, which mutates the doc before writing the DB). The row
|
||||
* delete is irreversible, so if the mark removal fails — including the
|
||||
* COLLAB_DISABLE_REDIS "no live instance" hard-error — we must NOT delete the
|
||||
* row and report success, or the document is left with a permanent orphan
|
||||
* anchor pointing at a comment that no longer exists (the exact data-integrity
|
||||
* bug #329 targets). Let the exception propagate (→ 5xx); the operation is
|
||||
* then repeatable with row + mark still consistent.
|
||||
*
|
||||
* RACE (#338 F4): the caller read `hasChildren` BEFORE the (slow) mark
|
||||
* removal, so a reply can land in that window. `comments.parent_comment_id` is
|
||||
* ON DELETE CASCADE, so an unconditional delete here would cascade-destroy the
|
||||
* just-added reply forever. Instead we use `deleteCommentIfChildless`, which
|
||||
* re-checks childlessness under a FOR UPDATE lock inside a transaction (a plain
|
||||
* anti-join DELETE is NOT race-safe under READ COMMITTED — see the repo method
|
||||
* docstring). If it removes the row (outcome 'deleted') we broadcast the
|
||||
* deletion as before. If it removes 0 rows (a reply interleaved) we do NOT
|
||||
* hard-delete — we resolve the thread instead (outcome 'resolved'), preserving
|
||||
* the discussion and the new reply. The anchor mark is already gone by then, an
|
||||
* accepted degradation: the thread lands in the resolved tab without its inline
|
||||
* highlight — far better than losing a reply.
|
||||
*/
|
||||
private async deleteEphemeralSuggestion(
|
||||
comment: Comment,
|
||||
user: User,
|
||||
provenance?: AuthProvenanceData,
|
||||
): Promise<Comment & { outcome: SuggestionOutcome }> {
|
||||
await this.deleteCommentMark(comment, user);
|
||||
|
||||
const deletedRows = await this.commentRepo.deleteCommentIfChildless(
|
||||
comment.id,
|
||||
);
|
||||
|
||||
if (deletedRows > 0) {
|
||||
this.wsService.emitCommentEvent(comment.spaceId, comment.pageId, {
|
||||
operation: 'commentDeleted',
|
||||
pageId: comment.pageId,
|
||||
commentId: comment.id,
|
||||
});
|
||||
return { ...comment, outcome: 'deleted' };
|
||||
}
|
||||
|
||||
// A reply interleaved between the hasChildren read and this delete, so the
|
||||
// conditional delete matched nothing. Preserve the discussion + the new
|
||||
// reply by resolving the thread instead of hard-deleting it. resolveComment
|
||||
// handles the resolve patch, its ws broadcast and the resolve notification;
|
||||
// its collab call is best-effort, so the already-stripped mark is fine.
|
||||
const resolvedComment = await this.resolveComment(
|
||||
comment,
|
||||
true,
|
||||
user,
|
||||
provenance,
|
||||
);
|
||||
return { ...resolvedComment, outcome: 'resolved' };
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove the inline `comment` mark for a comment from the collaborative
|
||||
* document. FATAL, NOT best-effort: unlike resolveComment (which keeps the row,
|
||||
* so a failed mark update is recoverable), this is used before an irreversible
|
||||
* hard-delete, so the mark removal MUST succeed or throw. Under
|
||||
* COLLAB_DISABLE_REDIS the gateway invokes the deleteCommentMark handler
|
||||
* directly (never a silent no-op) and a missing live instance surfaces as a
|
||||
* thrown error, which we let propagate so the caller aborts before deleting.
|
||||
*/
|
||||
private async deleteCommentMark(comment: Comment, user: User): Promise<void> {
|
||||
const documentName = `page.${comment.pageId}`;
|
||||
await this.collaborationGateway.handleYjsEvent(
|
||||
'deleteCommentMark',
|
||||
documentName,
|
||||
{ commentId: comment.id, user },
|
||||
);
|
||||
}
|
||||
|
||||
private async queueCommentNotification(
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
import { IsUUID } from 'class-validator';
|
||||
|
||||
export class DismissSuggestionDto {
|
||||
@IsUUID()
|
||||
commentId: string;
|
||||
}
|
||||
@@ -139,6 +139,65 @@ export class CommentRepo {
|
||||
await this.db.deleteFrom('comments').where('id', '=', commentId).execute();
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete an ephemeral suggestion row ONLY if it is still childless, returning
|
||||
* the number of rows removed (0 or 1). Closes the data-loss race in
|
||||
* dismiss/apply (#338 F4): the service reads `hasChildren`, then removes the
|
||||
* anchor mark (a collab round-trip of tens-to-hundreds of ms), then calls this.
|
||||
* `comments.parent_comment_id` is ON DELETE CASCADE, so a reply landing in that
|
||||
* window would be cascade-destroyed by a blind delete.
|
||||
*
|
||||
* A single anti-join `DELETE … WHERE NOT EXISTS(child)` is NOT sufficient under
|
||||
* READ COMMITTED: if a reply INSERT (holding FOR KEY SHARE on the parent, not
|
||||
* yet committed) interleaves, the DELETE's snapshot does not see the
|
||||
* uncommitted child, so `NOT EXISTS` is true and the parent qualifies; the
|
||||
* DELETE then blocks on the child's key-share lock, and when it wakes the row
|
||||
* was only LOCKED (not modified), so EvalPlanQual does NOT re-evaluate the
|
||||
* predicate → the parent is deleted and the just-committed reply cascades away.
|
||||
*
|
||||
* So we do a lock-then-recheck in ONE transaction:
|
||||
* 1. `SELECT id … FOR UPDATE` on the parent. FOR UPDATE conflicts with the
|
||||
* FOR KEY SHARE a concurrent reply INSERT takes on its parent (FK), so a
|
||||
* reply in the window serializes against us: it either commits before we
|
||||
* acquire the lock, or it must wait until this tx ends.
|
||||
* 2. Re-read childlessness with a FRESH statement in the SAME tx. Under RC a
|
||||
* new statement gets a new snapshot, so a reply that committed while we
|
||||
* waited on the lock is now visible.
|
||||
* 3. Delete only if still childless (return 1); otherwise return 0 so the
|
||||
* caller resolves the thread instead. The FOR UPDATE lock is held to
|
||||
* end-of-tx, so no new reply can insert between the re-check and the delete.
|
||||
*/
|
||||
async deleteCommentIfChildless(commentId: string): Promise<number> {
|
||||
return this.db.transaction().execute(async (trx) => {
|
||||
const parent = await trx
|
||||
.selectFrom('comments')
|
||||
.select('id')
|
||||
.where('id', '=', commentId)
|
||||
.forUpdate()
|
||||
.executeTakeFirst();
|
||||
|
||||
// Already gone (e.g. a racing delete won) → nothing to remove.
|
||||
if (!parent) return 0;
|
||||
|
||||
const child = await trx
|
||||
.selectFrom('comments')
|
||||
.select('id')
|
||||
.where('parentCommentId', '=', commentId)
|
||||
.limit(1)
|
||||
.executeTakeFirst();
|
||||
|
||||
// A reply exists (possibly one that just committed) → do NOT hard-delete;
|
||||
// the cascade would destroy it. Caller falls back to resolving the thread.
|
||||
if (child) return 0;
|
||||
|
||||
await trx
|
||||
.deleteFrom('comments')
|
||||
.where('id', '=', commentId)
|
||||
.execute();
|
||||
return 1;
|
||||
});
|
||||
}
|
||||
|
||||
async hasChildren(commentId: string): Promise<boolean> {
|
||||
const result = await this.db
|
||||
.selectFrom('comments')
|
||||
|
||||
@@ -0,0 +1,160 @@
|
||||
import { Kysely } from 'kysely';
|
||||
import { CommentRepo } from '../../src/database/repos/comment/comment.repo';
|
||||
import {
|
||||
getTestDb,
|
||||
destroyTestDb,
|
||||
buildTestDb,
|
||||
createWorkspace,
|
||||
createSpace,
|
||||
createPage,
|
||||
createUser,
|
||||
createComment,
|
||||
} from './db';
|
||||
|
||||
/**
|
||||
* Real-DB coverage for CommentRepo.deleteCommentIfChildless (#338 F4/F6).
|
||||
*
|
||||
* This is the guard that keeps an ephemeral-suggestion hard-delete from
|
||||
* cascade-destroying a reply (`comments.parent_comment_id` is ON DELETE CASCADE).
|
||||
* The unit tests MOCK this method to 0/1, so only an int-spec actually exercises
|
||||
* the SQL — the FOR UPDATE lock-then-recheck transaction — against Postgres.
|
||||
*
|
||||
* The concurrency case is the whole point: a plain anti-join
|
||||
* `DELETE … WHERE NOT EXISTS(child)` passes (a) and (b) but SILENTLY loses a
|
||||
* reply that commits mid-operation under READ COMMITTED (EvalPlanQual does not
|
||||
* re-check a merely-locked row). Test (c) reproduces exactly that interleaving
|
||||
* and asserts the row + reply both survive.
|
||||
*/
|
||||
describe('CommentRepo.deleteCommentIfChildless [integration]', () => {
|
||||
let db: Kysely<any>;
|
||||
let repo: CommentRepo;
|
||||
let workspaceId: string;
|
||||
let spaceId: string;
|
||||
let pageId: string;
|
||||
let userId: string;
|
||||
|
||||
beforeAll(async () => {
|
||||
db = getTestDb();
|
||||
repo = new CommentRepo(db as any);
|
||||
workspaceId = (await createWorkspace(db)).id;
|
||||
spaceId = (await createSpace(db, workspaceId)).id;
|
||||
pageId = (await createPage(db, { workspaceId, spaceId })).id;
|
||||
userId = (await createUser(db, workspaceId)).id;
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await destroyTestDb();
|
||||
});
|
||||
|
||||
async function rowExists(id: string): Promise<boolean> {
|
||||
const row = await db
|
||||
.selectFrom('comments')
|
||||
.select('id')
|
||||
.where('id', '=', id)
|
||||
.executeTakeFirst();
|
||||
return Boolean(row);
|
||||
}
|
||||
|
||||
function seedTopLevel() {
|
||||
return createComment(db, {
|
||||
workspaceId,
|
||||
spaceId,
|
||||
pageId,
|
||||
creatorId: userId,
|
||||
selection: 'old text',
|
||||
suggestedText: 'new text',
|
||||
});
|
||||
}
|
||||
|
||||
function seedReply(parentId: string) {
|
||||
return createComment(db, {
|
||||
workspaceId,
|
||||
spaceId,
|
||||
pageId,
|
||||
creatorId: userId,
|
||||
parentCommentId: parentId,
|
||||
});
|
||||
}
|
||||
|
||||
it('(a) childless top-level → returns 1 and the row is gone', async () => {
|
||||
const parent = await seedTopLevel();
|
||||
expect(await rowExists(parent.id)).toBe(true);
|
||||
|
||||
const deleted = await repo.deleteCommentIfChildless(parent.id);
|
||||
|
||||
expect(deleted).toBe(1);
|
||||
expect(await rowExists(parent.id)).toBe(false);
|
||||
});
|
||||
|
||||
it('(b) top-level WITH a committed reply → returns 0, parent AND reply survive (gate blocks the cascade)', async () => {
|
||||
const parent = await seedTopLevel();
|
||||
const reply = await seedReply(parent.id);
|
||||
|
||||
const deleted = await repo.deleteCommentIfChildless(parent.id);
|
||||
|
||||
expect(deleted).toBe(0);
|
||||
expect(await rowExists(parent.id)).toBe(true);
|
||||
expect(await rowExists(reply.id)).toBe(true);
|
||||
});
|
||||
|
||||
it('(c) reply COMMITS mid-operation (FOR UPDATE path) → returns 0, parent + reply survive; a blind anti-join would lose the reply', async () => {
|
||||
const parent = await seedTopLevel();
|
||||
|
||||
// Second connection holds an open transaction that inserts a reply (taking
|
||||
// FOR KEY SHARE on the parent via the FK) and does NOT commit until we open
|
||||
// the gate — reproducing the "reply not yet committed" window.
|
||||
const conn2 = buildTestDb();
|
||||
let openGate!: () => void;
|
||||
const gate = new Promise<void>((resolve) => {
|
||||
openGate = resolve;
|
||||
});
|
||||
let replyId: string | undefined;
|
||||
|
||||
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
|
||||
|
||||
try {
|
||||
const replyTx = conn2.transaction().execute(async (trx) => {
|
||||
const row = await trx
|
||||
.insertInto('comments')
|
||||
.values({
|
||||
workspaceId,
|
||||
spaceId,
|
||||
pageId,
|
||||
creatorId: userId,
|
||||
parentCommentId: parent.id,
|
||||
})
|
||||
.returning(['id'])
|
||||
.executeTakeFirstOrThrow();
|
||||
replyId = row.id as string;
|
||||
// Hold the FOR KEY SHARE lock on the parent until the gate opens.
|
||||
await gate;
|
||||
});
|
||||
|
||||
// Let the reply INSERT acquire its lock before the delete starts.
|
||||
await sleep(250);
|
||||
|
||||
// deleteCommentIfChildless does SELECT ... FOR UPDATE on the parent, which
|
||||
// conflicts with the reply's FOR KEY SHARE, so it BLOCKS here.
|
||||
const deletePromise = repo.deleteCommentIfChildless(parent.id);
|
||||
|
||||
// Give the delete time to reach (and block on) its FOR UPDATE, then let the
|
||||
// reply commit. The delete then wakes, re-checks under the lock, sees the
|
||||
// now-committed reply, and returns 0.
|
||||
await sleep(250);
|
||||
openGate();
|
||||
await replyTx;
|
||||
|
||||
const deleted = await deletePromise;
|
||||
|
||||
expect(deleted).toBe(0);
|
||||
expect(await rowExists(parent.id)).toBe(true);
|
||||
expect(replyId).toBeDefined();
|
||||
expect(await rowExists(replyId!)).toBe(true);
|
||||
} finally {
|
||||
// Always release the gate (in case an assertion threw before openGate) and
|
||||
// close the extra connection so global-teardown can DROP the database.
|
||||
openGate();
|
||||
await conn2.destroy();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -132,6 +132,62 @@ export async function createUser(
|
||||
return { id: row.id as string };
|
||||
}
|
||||
|
||||
// The default group every workspace has; `groupUserRepo.addUserToDefaultGroup`
|
||||
// (invoked by acceptInvitation) looks it up by `isDefault = true`, so a
|
||||
// workspace under test must have exactly one for the accept path to complete.
|
||||
export async function createDefaultGroup(
|
||||
db: Kysely<any>,
|
||||
workspaceId: string,
|
||||
overrides: { name?: string } = {},
|
||||
): Promise<{ id: string }> {
|
||||
const id = randomUUID();
|
||||
const suffix = shortId(id);
|
||||
const row = await db
|
||||
.insertInto('groups')
|
||||
.values({
|
||||
id,
|
||||
// name is unique per workspace + NOT NULL.
|
||||
name: overrides.name ?? `group-${suffix}`,
|
||||
isDefault: true,
|
||||
workspaceId,
|
||||
})
|
||||
.returning(['id'])
|
||||
.executeTakeFirstOrThrow();
|
||||
return { id: row.id as string };
|
||||
}
|
||||
|
||||
// A pending workspace invitation. `role`/`token` are NOT NULL; `groupIds` is a
|
||||
// nullable uuid[] and `invitedById` a nullable FK to users. Returns the fields a
|
||||
// spec needs to drive acceptInvitation (id + token + the invited email).
|
||||
export async function createInvitation(
|
||||
db: Kysely<any>,
|
||||
args: {
|
||||
workspaceId: string;
|
||||
email: string;
|
||||
invitedById?: string | null;
|
||||
role?: string;
|
||||
token?: string;
|
||||
groupIds?: string[] | null;
|
||||
},
|
||||
): Promise<{ id: string; token: string; email: string }> {
|
||||
const id = randomUUID();
|
||||
const token = args.token ?? `tok-${shortId(id)}`;
|
||||
const row = await db
|
||||
.insertInto('workspaceInvitations')
|
||||
.values({
|
||||
id,
|
||||
email: args.email,
|
||||
role: args.role ?? 'member',
|
||||
token,
|
||||
groupIds: (args.groupIds ?? null) as any,
|
||||
invitedById: args.invitedById ?? null,
|
||||
workspaceId: args.workspaceId,
|
||||
})
|
||||
.returning(['id'])
|
||||
.executeTakeFirstOrThrow();
|
||||
return { id: row.id as string, token, email: args.email };
|
||||
}
|
||||
|
||||
export async function createSpace(
|
||||
db: Kysely<any>,
|
||||
workspaceId: string,
|
||||
@@ -174,6 +230,40 @@ export async function createPage(
|
||||
return { id: row.id as string };
|
||||
}
|
||||
|
||||
export async function createComment(
|
||||
db: Kysely<any>,
|
||||
args: {
|
||||
workspaceId: string;
|
||||
spaceId: string;
|
||||
pageId: string;
|
||||
creatorId?: string | null;
|
||||
parentCommentId?: string | null;
|
||||
content?: unknown;
|
||||
selection?: string | null;
|
||||
suggestedText?: string | null;
|
||||
type?: string | null;
|
||||
},
|
||||
): Promise<{ id: string }> {
|
||||
const id = randomUUID();
|
||||
const row = await db
|
||||
.insertInto('comments')
|
||||
.values({
|
||||
id,
|
||||
workspaceId: args.workspaceId,
|
||||
spaceId: args.spaceId,
|
||||
pageId: args.pageId,
|
||||
creatorId: args.creatorId ?? null,
|
||||
parentCommentId: args.parentCommentId ?? null,
|
||||
content: (args.content ?? null) as any,
|
||||
selection: args.selection ?? null,
|
||||
suggestedText: args.suggestedText ?? null,
|
||||
type: args.type ?? 'page',
|
||||
})
|
||||
.returning(['id'])
|
||||
.executeTakeFirstOrThrow();
|
||||
return { id: row.id as string };
|
||||
}
|
||||
|
||||
export async function createRole(
|
||||
db: Kysely<any>,
|
||||
args: {
|
||||
|
||||
@@ -0,0 +1,218 @@
|
||||
import { BadRequestException } from '@nestjs/common';
|
||||
import { Kysely } from 'kysely';
|
||||
import { Workspace } from '@docmost/db/types/entity.types';
|
||||
import { UserRepo } from '@docmost/db/repos/user/user.repo';
|
||||
import { GroupRepo } from '@docmost/db/repos/group/group.repo';
|
||||
import { GroupUserRepo } from '@docmost/db/repos/group/group-user.repo';
|
||||
import { WorkspaceInvitationService } from 'src/core/workspace/services/workspace-invitation.service';
|
||||
import {
|
||||
getTestDb,
|
||||
destroyTestDb,
|
||||
createWorkspace,
|
||||
createUser,
|
||||
createDefaultGroup,
|
||||
createInvitation,
|
||||
} from './db';
|
||||
|
||||
/**
|
||||
* acceptInvitation atomicity (issue #324, tail of #244).
|
||||
*
|
||||
* acceptInvitation() reads the invitation OUTSIDE the transaction, then inside a
|
||||
* single tx: inserts the invited user, adds them to the default group, and
|
||||
* deletes the invitation. Two accepts of the SAME invitation therefore race to
|
||||
* insert a user with the same (email, workspaceId) — which the
|
||||
* `users_email_workspace_id_unique` constraint forbids. The service catches that
|
||||
* violation and reports "Invitation already accepted".
|
||||
*
|
||||
* These specs pin the INVARIANT that path protects: no matter how many times the
|
||||
* invitation is accepted (concurrently or repeatedly), the workspace ends up
|
||||
* with exactly ONE membership for the invited email and the invitation is
|
||||
* consumed exactly once — never a duplicate user and never a half-applied state.
|
||||
*
|
||||
* The service is wired with the REAL repos (UserRepo / GroupRepo / GroupUserRepo)
|
||||
* against the test Kysely; only the peripheral collaborators that acceptInvitation
|
||||
* touches AFTER the transaction (mail, session token, billing, audit, env) are
|
||||
* stubbed, so the exercised DB write path is the production one.
|
||||
*/
|
||||
describe('WorkspaceInvitationService.acceptInvitation atomicity [integration]', () => {
|
||||
let db: Kysely<any>;
|
||||
let service: WorkspaceInvitationService;
|
||||
|
||||
// Count the memberships (user rows) for an email within a workspace — the
|
||||
// quantity the atomicity guarantee is about.
|
||||
async function membershipCount(
|
||||
workspaceId: string,
|
||||
email: string,
|
||||
): Promise<number> {
|
||||
const rows = await db
|
||||
.selectFrom('users')
|
||||
.select('id')
|
||||
.where('workspaceId', '=', workspaceId)
|
||||
.where('email', '=', email.toLowerCase())
|
||||
.execute();
|
||||
return rows.length;
|
||||
}
|
||||
|
||||
async function invitationExists(invitationId: string): Promise<boolean> {
|
||||
const row = await db
|
||||
.selectFrom('workspaceInvitations')
|
||||
.select('id')
|
||||
.where('id', '=', invitationId)
|
||||
.executeTakeFirst();
|
||||
return !!row;
|
||||
}
|
||||
|
||||
beforeAll(() => {
|
||||
db = getTestDb();
|
||||
|
||||
const userRepo = new UserRepo(db as any);
|
||||
const groupRepo = new GroupRepo(db as any);
|
||||
const groupUserRepo = new GroupUserRepo(db as any, groupRepo, userRepo);
|
||||
|
||||
// Collaborators used only on the post-commit success tail; safe to stub.
|
||||
const mailService = { sendToQueue: jest.fn().mockResolvedValue(undefined) };
|
||||
const domainService = {} as any;
|
||||
const tokenService = {} as any;
|
||||
const sessionService = {
|
||||
createSessionAndToken: jest.fn().mockResolvedValue('test-auth-token'),
|
||||
};
|
||||
const billingQueue = { add: jest.fn().mockResolvedValue(undefined) };
|
||||
const environmentService = { isCloud: () => false };
|
||||
const auditService = { log: jest.fn() };
|
||||
|
||||
service = new WorkspaceInvitationService(
|
||||
userRepo,
|
||||
groupUserRepo,
|
||||
mailService as any,
|
||||
domainService,
|
||||
tokenService,
|
||||
sessionService as any,
|
||||
db as any,
|
||||
billingQueue as any,
|
||||
environmentService as any,
|
||||
auditService as any,
|
||||
);
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await destroyTestDb();
|
||||
});
|
||||
|
||||
// A workspace with its default group, an inviter, and a pending invitation.
|
||||
async function seedInvite(): Promise<{
|
||||
workspace: Workspace;
|
||||
invitationId: string;
|
||||
token: string;
|
||||
email: string;
|
||||
}> {
|
||||
const { id: workspaceId } = await createWorkspace(db);
|
||||
await createDefaultGroup(db, workspaceId);
|
||||
const inviter = await createUser(db, workspaceId);
|
||||
// Distinct address per invite so specs never collide across the suite.
|
||||
const email = `invitee-${workspaceId.slice(0, 8)}@example.test`;
|
||||
const invite = await createInvitation(db, {
|
||||
workspaceId,
|
||||
email,
|
||||
invitedById: inviter.id,
|
||||
});
|
||||
|
||||
// acceptInvitation only reads id/hostname/enforceSso/emailDomains/enforceMfa
|
||||
// off the workspace; a minimal plain object is sufficient.
|
||||
const workspace = {
|
||||
id: workspaceId,
|
||||
hostname: `host-${workspaceId.slice(0, 8)}`,
|
||||
enforceSso: false,
|
||||
enforceMfa: false,
|
||||
emailDomains: [] as string[],
|
||||
} as unknown as Workspace;
|
||||
|
||||
return { workspace, invitationId: invite.id, token: invite.token, email };
|
||||
}
|
||||
|
||||
it('concurrent accepts create a single membership and consume the invitation once', async () => {
|
||||
const { workspace, invitationId, token, email } = await seedInvite();
|
||||
|
||||
const dto = { invitationId, token, name: 'Invited User', password: 'password123' };
|
||||
|
||||
// Fire two accepts of the SAME invitation at once. They race to insert the
|
||||
// same (email, workspaceId); the unique constraint lets exactly one win.
|
||||
const results = await Promise.allSettled([
|
||||
service.acceptInvitation({ ...dto }, workspace),
|
||||
service.acceptInvitation({ ...dto }, workspace),
|
||||
]);
|
||||
|
||||
const fulfilled = results.filter((r) => r.status === 'fulfilled');
|
||||
const rejected = results.filter(
|
||||
(r): r is PromiseRejectedResult => r.status === 'rejected',
|
||||
);
|
||||
|
||||
// Exactly one accept succeeds; the other is rejected.
|
||||
expect(fulfilled).toHaveLength(1);
|
||||
expect(rejected).toHaveLength(1);
|
||||
|
||||
// The loser fails via the caught unique-constraint path with the specific
|
||||
// "already accepted" message — not a half-state / generic failure.
|
||||
expect(rejected[0].reason).toBeInstanceOf(BadRequestException);
|
||||
expect(rejected[0].reason.message).toBe('Invitation already accepted');
|
||||
|
||||
// Invariant: exactly one membership, and the invitation is gone.
|
||||
expect(await membershipCount(workspace.id, email)).toBe(1);
|
||||
expect(await invitationExists(invitationId)).toBe(false);
|
||||
});
|
||||
|
||||
it('a repeated (sequential) accept does not create a duplicate membership', async () => {
|
||||
const { workspace, invitationId, token, email } = await seedInvite();
|
||||
const dto = { invitationId, token, name: 'Invited User', password: 'password123' };
|
||||
|
||||
// First accept succeeds and returns an auth token.
|
||||
const first = await service.acceptInvitation({ ...dto }, workspace);
|
||||
expect(first?.authToken).toBe('test-auth-token');
|
||||
expect(await membershipCount(workspace.id, email)).toBe(1);
|
||||
expect(await invitationExists(invitationId)).toBe(false);
|
||||
|
||||
// Re-accepting the (now consumed) invitation must be rejected and must NOT
|
||||
// add a second membership. The invitation row is gone, so this hits the
|
||||
// "Invitation not found" guard rather than the unique-constraint path.
|
||||
await expect(
|
||||
service.acceptInvitation({ ...dto }, workspace),
|
||||
).rejects.toBeInstanceOf(BadRequestException);
|
||||
|
||||
expect(await membershipCount(workspace.id, email)).toBe(1);
|
||||
});
|
||||
|
||||
it('the single created membership is added to the default group (no partial state)', async () => {
|
||||
const { workspace, invitationId, token, email } = await seedInvite();
|
||||
const dto = { invitationId, token, name: 'Invited User', password: 'password123' };
|
||||
|
||||
await Promise.allSettled([
|
||||
service.acceptInvitation({ ...dto }, workspace),
|
||||
service.acceptInvitation({ ...dto }, workspace),
|
||||
]);
|
||||
|
||||
// Resolve the one surviving user and assert the whole tx applied: they exist
|
||||
// AND are in the workspace default group (the mid-transaction step), proving
|
||||
// the winning accept committed as a whole rather than leaving a torn state.
|
||||
const user = await db
|
||||
.selectFrom('users')
|
||||
.select(['id'])
|
||||
.where('workspaceId', '=', workspace.id)
|
||||
.where('email', '=', email.toLowerCase())
|
||||
.executeTakeFirstOrThrow();
|
||||
|
||||
const defaultGroup = await db
|
||||
.selectFrom('groups')
|
||||
.select(['id'])
|
||||
.where('workspaceId', '=', workspace.id)
|
||||
.where('isDefault', '=', true)
|
||||
.executeTakeFirstOrThrow();
|
||||
|
||||
const membership = await db
|
||||
.selectFrom('groupUsers')
|
||||
.select(['userId'])
|
||||
.where('groupId', '=', defaultGroup.id)
|
||||
.where('userId', '=', user.id)
|
||||
.execute();
|
||||
|
||||
expect(membership).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
@@ -13,5 +13,9 @@
|
||||
"types": "dist/index.d.ts",
|
||||
"dependencies": {
|
||||
"marked": "17.0.5"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@vitest/coverage-v8": "4.1.6",
|
||||
"vitest": "4.1.6"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,5 +5,21 @@ export default defineConfig({
|
||||
environment: "jsdom",
|
||||
globals: true,
|
||||
include: ["src/**/*.{test,spec}.ts"],
|
||||
// Coverage gate (issue #324). v8 provider avoids the istanbul AST-rewrite
|
||||
// that broke on this package's ESM barrel. Thresholds sit a few points
|
||||
// below the level measured on develop, over the files the suite exercises
|
||||
// (`all: false`), so the gate passes today and catches a real regression.
|
||||
coverage: {
|
||||
enabled: true,
|
||||
provider: "v8",
|
||||
reporter: ["text-summary", "text"],
|
||||
all: false,
|
||||
thresholds: {
|
||||
statements: 54,
|
||||
branches: 44,
|
||||
functions: 60,
|
||||
lines: 54,
|
||||
},
|
||||
},
|
||||
},
|
||||
});
|
||||
|
||||
@@ -20,6 +20,7 @@
|
||||
},
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@docmost/prosemirror-markdown": "workspace:*",
|
||||
"@tiptap/core": "3.20.4",
|
||||
"@tiptap/extension-highlight": "3.20.4",
|
||||
"@tiptap/extension-image": "3.20.4",
|
||||
@@ -35,8 +36,10 @@
|
||||
"zod": "4.3.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@docmost/editor-ext": "workspace:*",
|
||||
"@types/jsdom": "^21.1.7",
|
||||
"@types/node": "^20.0.0",
|
||||
"@vitest/coverage-v8": "4.1.6",
|
||||
"fast-check": "^4.8.0",
|
||||
"typescript": "^5.0.0",
|
||||
"vitest": "4.1.6"
|
||||
|
||||
@@ -31,7 +31,7 @@
|
||||
*/
|
||||
import { dirname } from "node:path";
|
||||
import { sep } from "node:path";
|
||||
import { parsePageFile, serializePageFile } from "../lib/page-file.js";
|
||||
import { parsePageFile, serializePageFile } from "@docmost/prosemirror-markdown";
|
||||
import type { GitSyncClient } from "./client.types.js";
|
||||
import { buildVaultLayout, type PageNode } from "./layout.js";
|
||||
import {
|
||||
|
||||
@@ -26,8 +26,11 @@
|
||||
* the gitmost server drives the engine in-process (there is no standalone CLI
|
||||
* entry point).
|
||||
*/
|
||||
import { type DocmostMdMeta } from "../lib/index.js";
|
||||
import { parsePageFile, serializePageFile } from "../lib/page-file.js";
|
||||
import {
|
||||
type DocmostMdMeta,
|
||||
parsePageFile,
|
||||
serializePageFile,
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
import type { GitSyncClient } from "./client.types.js";
|
||||
import type { DiffEntry } from "./git.js";
|
||||
import { VaultGit, DEFAULT_BRANCH } from "./git.js";
|
||||
|
||||
@@ -17,7 +17,7 @@ import {
|
||||
markdownToProseMirror,
|
||||
serializeDocmostMarkdownBody,
|
||||
type DocmostMdMeta,
|
||||
} from "../lib/index.js";
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
|
||||
/**
|
||||
* Meta object as `exportPageBody` builds it (SPEC §4). Kept byte-for-byte
|
||||
|
||||
@@ -8,6 +8,10 @@
|
||||
*/
|
||||
|
||||
// Pure converter (markdown <-> ProseMirror, file envelope, canonicalization).
|
||||
// Re-exported from the standalone `@docmost/prosemirror-markdown` package,
|
||||
// which is the single source of truth for the converter core; git-sync keeps
|
||||
// only the engine (vault/git/orchestrator) and re-surfaces the converter for
|
||||
// in-process consumers of the git-sync barrel.
|
||||
export {
|
||||
serializeDocmostMarkdown,
|
||||
serializeDocmostMarkdownBody,
|
||||
@@ -16,8 +20,8 @@ export {
|
||||
markdownToProseMirror,
|
||||
canonicalizeContent,
|
||||
docsCanonicallyEqual,
|
||||
} from "./lib/index.js";
|
||||
export type { DocmostMdMeta } from "./lib/index.js";
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
export type { DocmostMdMeta } from "@docmost/prosemirror-markdown";
|
||||
|
||||
// Pure engine (no IO): reconcile planner, vault layout, sanitize, stabilize,
|
||||
// loop-guard body hash.
|
||||
@@ -123,4 +127,4 @@ export {
|
||||
} from "./engine/path-guard.js";
|
||||
export type { PathGuardIo, VaultPathUnsafeReason } from "./engine/path-guard.js";
|
||||
|
||||
export { parsePageFile, serializePageFile } from "./lib/page-file.js";
|
||||
export { parsePageFile, serializePageFile } from "@docmost/prosemirror-markdown";
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,365 +0,0 @@
|
||||
/**
|
||||
* Pure markdown -> ProseMirror conversion.
|
||||
*
|
||||
* The converter path is `markdownToProseMirror` (marked -> HTML ->
|
||||
* generateJSON) plus the two pre/post processors it needs (`preprocessCallouts`,
|
||||
* `bridgeTaskLists`). The gitmost server writes the resulting page bodies
|
||||
* natively through the collab gateway, so no websocket/Yjs write-path lives
|
||||
* here.
|
||||
*/
|
||||
import { generateJSON } from "@tiptap/html";
|
||||
import { JSDOM } from "jsdom";
|
||||
import { marked } from "marked";
|
||||
import { docmostExtensions } from "./docmost-schema.js";
|
||||
|
||||
// Setup DOM environment for Tiptap HTML parsing in Node.js
|
||||
const dom = new JSDOM("<!DOCTYPE html><html><body></body></html>");
|
||||
global.window = dom.window as any;
|
||||
global.document = dom.window.document;
|
||||
// @ts-ignore
|
||||
global.Element = dom.window.Element;
|
||||
|
||||
/**
|
||||
* Hard ceiling above which we skip callout preprocessing entirely. The linear
|
||||
* scanner below has no quadratic blow-up, but we still cap input defensively so
|
||||
* a pathological multi-megabyte payload cannot tie up the event loop; in that
|
||||
* case the markdown is passed through verbatim (callouts are simply not
|
||||
* detected) rather than risking a slow scan.
|
||||
*/
|
||||
const MAX_CALLOUT_PREPROCESS_BYTES = 4 * 1024 * 1024; // 4 MB
|
||||
|
||||
/** Matches an opening callout fence: `:::type` (type captured, lower-cased). */
|
||||
const CALLOUT_OPEN_RE = /^:::\s*(\w+)\s*$/;
|
||||
/** Matches a bare closing callout fence: `:::`. */
|
||||
const CALLOUT_CLOSE_RE = /^:::\s*$/;
|
||||
/**
|
||||
* Matches an Obsidian-native callout opener: `> [!type]` (type captured). An
|
||||
* optional title after the type is allowed but ignored (the Docmost callout
|
||||
* schema has no title). The body is the following contiguous blockquote lines.
|
||||
*/
|
||||
const CALLOUT_BQ_OPEN_RE = /^>\s*\[!(\w+)\]/;
|
||||
/** Matches any blockquote continuation line (`>` … ). */
|
||||
const BLOCKQUOTE_LINE_RE = /^>/;
|
||||
/** Matches the start/end of a code fence (``` or ~~~), capturing the marker. */
|
||||
const CODE_FENCE_RE = /^(\s*)(`{3,}|~{3,})/;
|
||||
|
||||
/**
|
||||
* Pre-process Docmost-flavoured markdown: convert `:::type ... :::`
|
||||
* callout blocks (the syntax our markdown export produces) into HTML
|
||||
* divs that the callout extension parses. The inner content is rendered
|
||||
* through marked as regular markdown.
|
||||
*
|
||||
* Implemented as a single linear pass over the lines (no quadratic regex
|
||||
* rescan). It:
|
||||
* - tracks fenced code regions (```...``` and ~~~...~~~) and never treats a
|
||||
* `:::` line that lives inside a code fence as a callout delimiter, so a
|
||||
* callout body that itself contains a fenced code block with a `:::` line is
|
||||
* no longer corrupted;
|
||||
* - matches an opening `:::type` line with the next CLOSING `:::` at the SAME
|
||||
* nesting level, supporting NESTED callouts via a depth counter (an inner
|
||||
* `:::type` opens a deeper level and consumes a matching `:::`);
|
||||
* - emits the same `<div data-type="callout" data-callout-type="TYPE">` output
|
||||
* (inner rendered through marked) as the previous regex implementation.
|
||||
*/
|
||||
async function preprocessCallouts(markdown: string): Promise<string> {
|
||||
// Defensive cap: skip preprocessing for pathologically large inputs.
|
||||
if (markdown.length > MAX_CALLOUT_PREPROCESS_BYTES) {
|
||||
return markdown;
|
||||
}
|
||||
|
||||
// Recursively transform a slice of lines, converting top-level callouts in
|
||||
// that slice into <div> blocks and rendering their inner content (which may
|
||||
// itself contain nested callouts) through this same function.
|
||||
const transform = async (lines: string[]): Promise<string> => {
|
||||
const out: string[] = [];
|
||||
let inCodeFence = false;
|
||||
let codeFenceMarker = ""; // the exact run of backticks/tildes that opened it
|
||||
let i = 0;
|
||||
|
||||
while (i < lines.length) {
|
||||
const line = lines[i];
|
||||
|
||||
// Inside a code fence, only its matching closing fence is significant;
|
||||
// everything else (including `:::` lines) is copied through verbatim.
|
||||
if (inCodeFence) {
|
||||
out.push(line);
|
||||
const fence = line.match(CODE_FENCE_RE);
|
||||
if (fence && fence[2].startsWith(codeFenceMarker[0]) &&
|
||||
fence[2].length >= codeFenceMarker.length) {
|
||||
inCodeFence = false;
|
||||
codeFenceMarker = "";
|
||||
}
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// A code fence opening outside any callout body: enter code-fence mode.
|
||||
const fenceOpen = line.match(CODE_FENCE_RE);
|
||||
if (fenceOpen) {
|
||||
inCodeFence = true;
|
||||
codeFenceMarker = fenceOpen[2];
|
||||
out.push(line);
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// An opening callout fence: scan forward (with code-fence and nested
|
||||
// callout awareness) for its matching closing `:::` at the same level.
|
||||
const open = line.match(CALLOUT_OPEN_RE);
|
||||
if (open) {
|
||||
const type = open[1].toLowerCase();
|
||||
const bodyLines: string[] = [];
|
||||
let depth = 1;
|
||||
let innerInCodeFence = false;
|
||||
let innerCodeFenceMarker = "";
|
||||
let j = i + 1;
|
||||
for (; j < lines.length; j++) {
|
||||
const bl = lines[j];
|
||||
if (innerInCodeFence) {
|
||||
const f = bl.match(CODE_FENCE_RE);
|
||||
if (f && f[2].startsWith(innerCodeFenceMarker[0]) &&
|
||||
f[2].length >= innerCodeFenceMarker.length) {
|
||||
innerInCodeFence = false;
|
||||
innerCodeFenceMarker = "";
|
||||
}
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
const innerFence = bl.match(CODE_FENCE_RE);
|
||||
if (innerFence) {
|
||||
innerInCodeFence = true;
|
||||
innerCodeFenceMarker = innerFence[2];
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
if (CALLOUT_OPEN_RE.test(bl)) {
|
||||
depth++;
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
if (CALLOUT_CLOSE_RE.test(bl)) {
|
||||
depth--;
|
||||
if (depth === 0) break; // matching close for THIS callout
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
bodyLines.push(bl);
|
||||
}
|
||||
|
||||
if (j < lines.length) {
|
||||
// Found the matching closing fence: render the body (recursively, so
|
||||
// nested callouts are handled) and emit the callout div.
|
||||
const inner = await transform(bodyLines);
|
||||
const renderedInner = await marked.parse(inner);
|
||||
out.push(
|
||||
`\n<div data-type="callout" data-callout-type="${type}">${renderedInner}</div>\n`,
|
||||
);
|
||||
i = j + 1; // skip past the closing `:::`
|
||||
continue;
|
||||
}
|
||||
// No matching close (unterminated callout): treat the opener as a
|
||||
// literal line and continue, preserving the original text.
|
||||
out.push(line);
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// An Obsidian-native callout: `> [!type]` opener; the body is the following
|
||||
// CONTIGUOUS blockquote (`>`-prefixed) lines. Strip ONE blockquote level and
|
||||
// recurse so nested callouts (`> > [!type]`) are handled, then emit the same
|
||||
// callout div the `:::` path produces. A normal blockquote (no `[!type]` on
|
||||
// its first line) does not match and stays a blockquote.
|
||||
const bqOpen = line.match(CALLOUT_BQ_OPEN_RE);
|
||||
if (bqOpen) {
|
||||
const type = bqOpen[1].toLowerCase();
|
||||
const bodyLines: string[] = [];
|
||||
let j = i + 1;
|
||||
for (; j < lines.length; j++) {
|
||||
if (!BLOCKQUOTE_LINE_RE.test(lines[j])) break;
|
||||
bodyLines.push(lines[j].replace(/^>\s?/, ""));
|
||||
}
|
||||
const inner = await transform(bodyLines);
|
||||
const renderedInner = await marked.parse(inner);
|
||||
out.push(
|
||||
`\n<div data-type="callout" data-callout-type="${type}">${renderedInner}</div>\n`,
|
||||
);
|
||||
i = j;
|
||||
continue;
|
||||
}
|
||||
|
||||
out.push(line);
|
||||
i++;
|
||||
}
|
||||
|
||||
return out.join("\n");
|
||||
};
|
||||
|
||||
return transform(markdown.split("\n"));
|
||||
}
|
||||
|
||||
/**
|
||||
* Bridge marked's checkbox lists to TipTap task lists.
|
||||
*
|
||||
* marked renders GitHub task list items (`- [x] done`) as a plain
|
||||
* `<ul><li><p><input type="checkbox" checked> text</p></li></ul>` WITHOUT the
|
||||
* markup TipTap's TaskList/TaskItem extensions parse. This rewrites such lists
|
||||
* into the shape those extensions expect:
|
||||
* TaskList parseHTML matches `ul[data-type="taskList"]`,
|
||||
* TaskItem matches `li[data-type="taskItem"]`,
|
||||
* the checked state is read from `data-checked === "true"`.
|
||||
*
|
||||
* A list is only converted when it has at least one `<li>` and EVERY direct
|
||||
* `<li>` contains a checkbox input. Both `<ul>` and `<ol>` are considered: a
|
||||
* numbered checklist (`1. [x] a`, which marked renders as an `<ol>` of checkbox
|
||||
* `<li>`s) would otherwise lose its task state. TipTap task lists are unordered,
|
||||
* so a matching `<ol>` is emitted as `data-type="taskList"` exactly like a
|
||||
* `<ul>`. Mixed or ordinary lists (including ordinary `<ol>` lists) are left
|
||||
* untouched so they keep rendering as bullet/numbered lists. The marked `<p>`
|
||||
* wrapper is kept inside the `<li>` because TaskItem content allows paragraphs.
|
||||
*/
|
||||
function bridgeTaskLists(html: string): string {
|
||||
// Cheap early-out: if the markup contains no checkbox input at all there is
|
||||
// nothing to bridge, so skip the expensive JSDOM parse entirely. This is the
|
||||
// common case (most pages have no task lists).
|
||||
if (!/type=["']?checkbox/i.test(html)) {
|
||||
return html;
|
||||
}
|
||||
// Defensive cap (consistent with preprocessCallouts): skip the bridge for
|
||||
// pathologically large inputs rather than running a second expensive JSDOM
|
||||
// parse on a multi-megabyte payload. The markup is passed through verbatim.
|
||||
if (html.length > MAX_CALLOUT_PREPROCESS_BYTES) {
|
||||
return html;
|
||||
}
|
||||
const dom = new JSDOM(html);
|
||||
const document = dom.window.document;
|
||||
// Collect the checkbox(es) that belong to THIS <li> directly: either direct
|
||||
// child <input type="checkbox"> elements or ones inside the <li>'s direct <p>
|
||||
// child (the shape marked emits: `<li><p><input type="checkbox"> text</p></li>`).
|
||||
// Checkboxes nested deeper (e.g. inside a child <ul>/<ol>) are excluded so a
|
||||
// bullet <li> that merely contains a nested task sublist is not misdetected.
|
||||
// Raw inline HTML can put more than one checkbox in a single <li>; we gather
|
||||
// ALL of them so none survive into the converted item.
|
||||
const directCheckboxes = (li: Element): Element[] => {
|
||||
const found: Element[] = [];
|
||||
for (const child of Array.from(li.children)) {
|
||||
if (
|
||||
child.tagName === "INPUT" &&
|
||||
child.getAttribute("type") === "checkbox"
|
||||
) {
|
||||
found.push(child);
|
||||
continue;
|
||||
}
|
||||
if (child.tagName === "P") {
|
||||
for (const inp of Array.from(
|
||||
child.querySelectorAll(":scope > input[type='checkbox']"),
|
||||
)) {
|
||||
found.push(inp);
|
||||
}
|
||||
}
|
||||
}
|
||||
return found;
|
||||
};
|
||||
// Both <ul> and <ol> are candidates: an <ol> whose every direct <li> carries
|
||||
// its own checkbox is a numbered checklist that must also become a taskList.
|
||||
const lists = Array.from(document.querySelectorAll("ul, ol"));
|
||||
for (const list of lists) {
|
||||
// Only consider DIRECT child <li> elements; nested lists are handled by
|
||||
// their own iteration of the outer loop.
|
||||
const items = Array.from(list.children).filter(
|
||||
(child) => child.tagName === "LI",
|
||||
);
|
||||
if (items.length === 0) continue;
|
||||
const itemCheckboxes = items.map((li) => directCheckboxes(li));
|
||||
// Convert only when every direct <li> carries at least one OWN checkbox.
|
||||
if (!itemCheckboxes.every((boxes) => boxes.length > 0)) continue;
|
||||
|
||||
// A numbered checklist arrives as an <ol>. We must NOT leave the tag as
|
||||
// <ol> while tagging it data-type="taskList": generateJSON would then match
|
||||
// BOTH the orderedList rule (tag ol) and the taskList rule (data-type),
|
||||
// emitting a phantom empty orderedList beside the real taskList. So rename a
|
||||
// qualifying <ol> to a <ul> — move its <li> children over and replace it —
|
||||
// leaving only the taskList rule to match. Already-<ul> lists are unchanged.
|
||||
let target: Element = list;
|
||||
if (list.tagName === "OL") {
|
||||
const ul = document.createElement("ul");
|
||||
// Carry over existing attributes (e.g. class) so nothing is silently lost.
|
||||
for (const attr of Array.from(list.attributes)) {
|
||||
ul.setAttribute(attr.name, attr.value);
|
||||
}
|
||||
// Move every child node (including the <li>s we collected) into the <ul>.
|
||||
while (list.firstChild) {
|
||||
ul.appendChild(list.firstChild);
|
||||
}
|
||||
list.replaceWith(ul);
|
||||
target = ul;
|
||||
}
|
||||
|
||||
target.setAttribute("data-type", "taskList");
|
||||
items.forEach((li, index) => {
|
||||
const boxes = itemCheckboxes[index];
|
||||
// The first checkbox determines the checked state (matches the previous
|
||||
// single-checkbox behaviour); any extras only need removing.
|
||||
const input = boxes[0] ?? null;
|
||||
li.setAttribute("data-type", "taskItem");
|
||||
const checked =
|
||||
input != null &&
|
||||
(input.hasAttribute("checked") || (input as any).checked);
|
||||
li.setAttribute("data-checked", checked ? "true" : "false");
|
||||
// Remove ALL direct checkbox inputs so none survive into the content
|
||||
// (a raw-inline-HTML <li> may carry more than one).
|
||||
for (const box of boxes) {
|
||||
box.remove();
|
||||
}
|
||||
});
|
||||
}
|
||||
return document.body.innerHTML;
|
||||
}
|
||||
|
||||
/**
|
||||
* Recursively strip content-less paragraph nodes from a generated doc.
|
||||
*
|
||||
* A block-level atom whose markdown form is INLINE (e.g. the block `image`'s
|
||||
* ``, or a bare media element) is wrapped by marked in a <p>; the schema
|
||||
* then HOISTS the block atom out of that paragraph, leaving an EMPTY paragraph
|
||||
* sibling. On the next export that empty `<p>` renders to "" and the doc "\n\n"
|
||||
* join injects a phantom blank gap, so the markdown is not byte-stable.
|
||||
*
|
||||
* Markdown blank lines are separators, never content, so generateJSON only ever
|
||||
* produces an empty paragraph as such a hoist artifact — removing them is safe
|
||||
* and general (it also subsumes the <div>-wrapper workaround the `video` case
|
||||
* uses). We remove ONLY `type === 'paragraph'` nodes whose `content` is absent
|
||||
* or an empty array; every other node (including atoms without `content`) is
|
||||
* preserved, and we recurse into the content of any node that has children.
|
||||
*/
|
||||
function stripEmptyParagraphs(node: any): any {
|
||||
if (!node || !Array.isArray(node.content)) {
|
||||
// Atom / leaf node (no children to recurse into): keep as-is.
|
||||
return node;
|
||||
}
|
||||
const mapped = node.content.map((child: any) => stripEmptyParagraphs(child));
|
||||
const isEmptyParagraph = (child: any): boolean =>
|
||||
!!child &&
|
||||
child.type === "paragraph" &&
|
||||
(!Array.isArray(child.content) || child.content.length === 0);
|
||||
const filtered = mapped.filter((child: any) => !isEmptyParagraph(child));
|
||||
// Schema-validity guard: several nodes require NON-empty block content
|
||||
// (`content: "block+"` — tableCell, tableHeader, blockquote, column, callout,
|
||||
// and the doc root). For an empty one of those, generateJSON materializes a
|
||||
// single empty paragraph as its OBLIGATORY content — that is not a hoist
|
||||
// artifact. If stripping would empty the container, keep ONE empty paragraph
|
||||
// so the result stays schema-valid (an empty cell/quote must not become `[]`).
|
||||
const cleaned =
|
||||
filtered.length === 0 && mapped.length > 0 ? [mapped[0]] : filtered;
|
||||
return { ...node, content: cleaned };
|
||||
}
|
||||
|
||||
/** Convert markdown to a ProseMirror doc using the full Docmost schema. */
|
||||
export async function markdownToProseMirror(
|
||||
markdownContent: string,
|
||||
): Promise<any> {
|
||||
const withCallouts = await preprocessCallouts(markdownContent);
|
||||
const html = await marked.parse(withCallouts);
|
||||
const bridged = bridgeTaskLists(html);
|
||||
const doc = generateJSON(bridged, docmostExtensions);
|
||||
return stripEmptyParagraphs(doc);
|
||||
}
|
||||
@@ -2,7 +2,7 @@ import { describe, expect, it, vi, beforeEach, afterEach } from 'vitest';
|
||||
import { applyPushActions, LAST_PUSHED_REF } from '../src/engine/push';
|
||||
import { bodyHash } from '../src/engine/loop-guard';
|
||||
import type { ApplyPushDeps, PushActions } from '../src/engine/push';
|
||||
import { parsePageFile, serializePageFile } from '../src/lib/page-file';
|
||||
import { parsePageFile, serializePageFile } from '@docmost/prosemirror-markdown';
|
||||
|
||||
// The Docmost space this vault mirrors (native files carry no spaceId; the run
|
||||
// supplies it). A CREATE targets this space.
|
||||
|
||||
@@ -5,7 +5,7 @@ import type {
|
||||
MetaSide,
|
||||
RenameMoveAction,
|
||||
} from '../src/engine/push';
|
||||
import type { DocmostMdMeta } from '../src/lib/index';
|
||||
import type { DocmostMdMeta } from '@docmost/prosemirror-markdown';
|
||||
|
||||
// FS→Docmost push #3 (SPEC §5/§6/§16). `classifyRenameMoves` is the PURE half of
|
||||
// the move/rename apply: it resolves each `{pageId, oldPath, newPath}` into the
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { computePushActions } from '../src/engine/push';
|
||||
import type { DiffEntry, MetaSide } from '../src/engine/push';
|
||||
import type { DocmostMdMeta } from '../src/lib/index';
|
||||
import type { DocmostMdMeta } from '@docmost/prosemirror-markdown';
|
||||
|
||||
// FS→Docmost push, FIRST increment (SPEC §6). `computePushActions` is the PURE
|
||||
// half: it classifies each `git diff --name-status` row into a Docmost action by
|
||||
|
||||
@@ -8,7 +8,7 @@ import { runCycle } from "../src/engine/cycle";
|
||||
import type { CycleFs } from "../src/engine/cycle";
|
||||
import { VaultGit } from "../src/engine/git";
|
||||
import type { Settings } from "../src/engine/settings";
|
||||
import { serializeDocmostMarkdownBody } from "../src/lib/index";
|
||||
import { serializeDocmostMarkdownBody } from "@docmost/prosemirror-markdown";
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ import { firstDivergence } from './roundtrip-helpers';
|
||||
import { applyPullActions } from '../src/engine/pull';
|
||||
import type { PullActions, ApplyPullActionsDeps } from '../src/engine/pull';
|
||||
import type { DeletionDecision } from '../src/engine/reconcile';
|
||||
import { serializePageFile, parsePageFile } from '../src/lib/page-file';
|
||||
import { serializePageFile, parsePageFile } from '@docmost/prosemirror-markdown';
|
||||
|
||||
// Engine-layer coverage gaps flagged by the PR #119 reviewers (test-strategy
|
||||
// report, Module 2 `src/engine`). Each block targets a specific under-covered
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { readExisting } from '../src/engine/pull';
|
||||
import { serializePageFile } from '../src/lib/page-file';
|
||||
import { serializePageFile } from '@docmost/prosemirror-markdown';
|
||||
|
||||
// R-Pull-1 (test-strategy report §5): `readExisting` now takes injectable IO
|
||||
// (`listTracked` / `readFile`), so its parsing + skip rules are unit-testable
|
||||
|
||||
@@ -6,7 +6,7 @@ import type {
|
||||
MetaSide,
|
||||
RenameMoveAction,
|
||||
} from '../src/engine/push.js';
|
||||
import type { DocmostMdMeta } from '../src/lib/index.js';
|
||||
import type { DocmostMdMeta } from '@docmost/prosemirror-markdown';
|
||||
|
||||
// RED-TEAM finding #4 (two facets):
|
||||
// (a) buildVaultLayout disambiguation is ORDER-DEPENDENT: which of two
|
||||
|
||||
@@ -8,7 +8,7 @@ import {
|
||||
import type { PushDeps } from '../src/engine/push';
|
||||
import type { Settings } from '../src/engine/settings';
|
||||
import { runCycle, type RunCycleDeps } from '../src/engine/cycle';
|
||||
import { serializePageFile } from '../src/lib/page-file';
|
||||
import { serializePageFile } from '@docmost/prosemirror-markdown';
|
||||
|
||||
// Red-team confirmations for PR #119 (git-sync). Each test asserts the DESIRED
|
||||
// behavior, so it FAILS today iff the bug is real.
|
||||
|
||||
@@ -1,104 +0,0 @@
|
||||
import { readFile } from 'node:fs/promises';
|
||||
import { readdirSync } from 'node:fs';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
convertProseMirrorToMarkdown,
|
||||
markdownToProseMirror,
|
||||
docsCanonicallyEqual,
|
||||
} from 'docmost-client';
|
||||
|
||||
// Resolve fixtures relative to this test file so the test is CWD-independent.
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
const CORPUS_DIR = join(here, 'fixtures', 'corpus');
|
||||
const KNOWN_LIMITATIONS_DIR = join(here, 'fixtures', 'known-limitations');
|
||||
|
||||
/** Run a single document through export -> import -> export. */
|
||||
async function roundTrip(doc: any) {
|
||||
const md1 = convertProseMirrorToMarkdown(doc);
|
||||
const doc2 = await markdownToProseMirror(md1);
|
||||
const md2 = convertProseMirrorToMarkdown(doc2);
|
||||
return { md1, md2, doc2 };
|
||||
}
|
||||
|
||||
describe('round-trip corpus (SPEC §11)', () => {
|
||||
// Discover the corpus synchronously at collection time so each fixture gets
|
||||
// its own `it` with the file name in the test title.
|
||||
const files = readdirSync(CORPUS_DIR)
|
||||
.filter((name) => name.endsWith('.json'))
|
||||
.sort();
|
||||
|
||||
it('has a non-empty corpus', () => {
|
||||
expect(files.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
for (const name of files) {
|
||||
it(`${name}: markdown byte-stable AND canonically stable`, async () => {
|
||||
const doc = JSON.parse(await readFile(join(CORPUS_DIR, name), 'utf8'));
|
||||
const { md1, md2, doc2 } = await roundTrip(doc);
|
||||
|
||||
// 1) The byte-stable markdown property git actually needs.
|
||||
expect(md2, `${name}: markdown not byte-stable`).toBe(md1);
|
||||
// 2) Semantic stability (block ids stripped, default-null normalized).
|
||||
expect(
|
||||
docsCanonicallyEqual(doc, doc2),
|
||||
`${name}: document not canonically stable`,
|
||||
).toBe(true);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// KNOWN CONVERTER LIMITATIONS (isolated so they do NOT make CI red).
|
||||
//
|
||||
// SPEC §11 explicitly flags images and diagrams as high round-trip risk. These
|
||||
// fixtures are kept OUT of the green corpus above and asserted with `it.fails`
|
||||
// so the documented divergence is locked in (the test FAILS if the converter
|
||||
// ever starts round-tripping them — at which point promote the fixture into
|
||||
// the corpus). The precise divergences for `image-diagrams.json` are:
|
||||
//
|
||||
// * A BLOCK-LEVEL image preceded by a paragraph is NOT byte-stable on the
|
||||
// FIRST re-export. The HTML re-parser hoists the block <img> out of its
|
||||
// line and leaves an empty paragraph behind, so `paragraph` + ``
|
||||
// re-imports as paragraph + empty-paragraph + image; the empty paragraph
|
||||
// adds one blank line, so export #2 grows by a one-time "\n\n" (md1 !== md2).
|
||||
// This is NOT non-convergence: the growth happens exactly ONCE. The doc
|
||||
// CONVERGES to a fixpoint after one extra `export→import→export` pass — the
|
||||
// empty paragraph is already present after the first import, so export #2
|
||||
// and export #3 are byte-identical (md2 === md3, verified).
|
||||
//
|
||||
// * drawio / excalidraw diagrams gain `data-align="center"` on the second
|
||||
// export: the schema's diagram `align` attribute has a NON-null default of
|
||||
// "center", which materializes on import; the converter only emits
|
||||
// data-align when set, so it appears on export #2 but not #1. Like the
|
||||
// image case, this is one-time and converges after one extra pass.
|
||||
//
|
||||
// * A STANDALONE block image (no preceding paragraph) IS byte-stable from
|
||||
// export #1 (md1 === md2) — but it is still NOT canonically stable: on
|
||||
// import the bare <img> is wrapped, gaining a leading EMPTY paragraph, so
|
||||
// the canonical doc differs by that spurious paragraph node even though the
|
||||
// markdown bytes match.
|
||||
//
|
||||
// Resolution (SPEC §11, "normalize-on-write"): rather than deep-fixing the
|
||||
// converter, the engine runs ONE `export→import→export` pass when writing into
|
||||
// the vault; from that fixpoint onward the form is byte-stable, so git sees no
|
||||
// phantom diff. The green corpus above avoids these one-time asymmetries by
|
||||
// pre-authoring the materialized defaults (e.g. `align: "center"` on the
|
||||
// diagrams in 06-diagrams.json) so a single pass is already at the fixpoint.
|
||||
// ---------------------------------------------------------------------------
|
||||
describe('round-trip KNOWN LIMITATIONS (SPEC §11 image/diagram risk)', () => {
|
||||
it.fails(
|
||||
'image-diagrams.json is NOT byte-stable on export #1 (block image hoist + diagram align default; converges after one extra pass — SPEC §11 normalize-on-write)',
|
||||
async () => {
|
||||
const doc = JSON.parse(
|
||||
await readFile(join(KNOWN_LIMITATIONS_DIR, 'image-diagrams.json'), 'utf8'),
|
||||
);
|
||||
const { md1, md2 } = await roundTrip(doc);
|
||||
// This assertion FAILS today (documented divergence). `it.fails` turns a
|
||||
// failing body into a PASS; if the converter is fixed this flips and the
|
||||
// test goes red, prompting promotion into the green corpus.
|
||||
expect(md2).toBe(md1);
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -8,7 +8,7 @@ import { runPush, LAST_PUSHED_REF } from '../src/engine/push';
|
||||
import type { PushDeps } from '../src/engine/push';
|
||||
import { VaultGit } from '../src/engine/git';
|
||||
import type { Settings } from '../src/engine/settings';
|
||||
import { serializeDocmostMarkdownBody } from '../src/lib/index';
|
||||
import { serializeDocmostMarkdownBody } from '@docmost/prosemirror-markdown';
|
||||
|
||||
const execFileAsync = promisify(execFile);
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@ import { describe, expect, it, vi } from 'vitest';
|
||||
import { runPush, LAST_PUSHED_REF, DOCMOST_BRANCH } from '../src/engine/push';
|
||||
import type { PushDeps } from '../src/engine/push';
|
||||
import type { Settings } from '../src/engine/settings';
|
||||
import { serializePageFile } from '../src/lib/page-file';
|
||||
import { serializePageFile } from '@docmost/prosemirror-markdown';
|
||||
|
||||
/** A native page file: `gitmost_id` frontmatter + clean body (title = filename). */
|
||||
function fileFor(pageId: string, body = 'body'): string {
|
||||
|
||||
@@ -2,8 +2,8 @@ import { describe, expect, it } from 'vitest';
|
||||
import { stabilizePageFile, type PageMeta } from '../src/engine/stabilize.js';
|
||||
// markdownToProseMirror lives in collaboration.ts; importing it mutates the
|
||||
// global DOM via jsdom at module load time (required for @tiptap/html under Node).
|
||||
import { markdownToProseMirror } from '../src/lib/markdown-to-prosemirror.js';
|
||||
import { parseDocmostMarkdown } from '../src/lib/markdown-document.js';
|
||||
import { markdownToProseMirror } from '@docmost/prosemirror-markdown';
|
||||
import { parseDocmostMarkdown } from '@docmost/prosemirror-markdown';
|
||||
|
||||
// stabilize.ts (SPEC §11 normalize-on-write) was 0% covered (only the gated e2e
|
||||
// touched it). stabilizePageFile is import-testable: build a small ProseMirror
|
||||
@@ -22,16 +22,27 @@ const meta: PageMeta = {
|
||||
|
||||
describe('stabilizePageFile — normalize-on-write fixpoint (SPEC §11)', () => {
|
||||
it('reaches a byte-identical fixpoint after one extra export/import/export pass', async () => {
|
||||
// A diagram is the canonical one-pass asymmetry: drawio's `align` default of
|
||||
// "center" materializes on import, so a NAIVE export differs on the second
|
||||
// export. stabilizePageFile runs the convergence pass at write time, so the
|
||||
// written body must already be at the fixpoint: re-importing its body and
|
||||
// A diagram inside a column is the canonical one-pass asymmetry: on the
|
||||
// raw-HTML/columns path a diagram's `align` default of "center" materializes
|
||||
// on import, so a NAIVE export differs on the second export. (#293 canon #8
|
||||
// made the TOP-LEVEL diagram form — `<!--drawio …-->` — byte-stable by
|
||||
// omitting the default, so the asymmetry now lives only on the columns path
|
||||
// where the schema `<div data-type="drawio">` form is retained.)
|
||||
// stabilizePageFile runs the convergence pass at write time, so the written
|
||||
// body must already be at the fixpoint: re-importing its body and
|
||||
// re-stabilizing yields the exact same bytes.
|
||||
const content = {
|
||||
type: 'doc',
|
||||
content: [
|
||||
{ type: 'paragraph', content: [{ type: 'text', text: 'intro' }] },
|
||||
{ type: 'drawio', attrs: { src: '/d.drawio' } },
|
||||
{
|
||||
type: 'columns',
|
||||
attrs: { layout: 'two_equal' },
|
||||
content: [
|
||||
{ type: 'column', content: [{ type: 'drawio', attrs: { src: '/d.drawio' } }] },
|
||||
{ type: 'column', content: [{ type: 'paragraph', content: [{ type: 'text', text: 'side' }] }] },
|
||||
],
|
||||
},
|
||||
{ type: 'paragraph', content: [{ type: 'text', text: 'outro' }] },
|
||||
],
|
||||
};
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { getSchema } from "@tiptap/core";
|
||||
|
||||
import { markdownToProseMirror } from "../src/lib/markdown-to-prosemirror";
|
||||
import { docmostExtensions } from "../src/lib/docmost-schema";
|
||||
import { markdownToProseMirror } from "@docmost/prosemirror-markdown";
|
||||
import { docmostExtensions } from "@docmost/prosemirror-markdown";
|
||||
|
||||
// REGRESSION LOCK for the stripEmptyParagraphs schema-validity guard.
|
||||
//
|
||||
|
||||
@@ -18,6 +18,25 @@ export default defineConfig({
|
||||
},
|
||||
test: {
|
||||
environment: 'node',
|
||||
// Coverage gate (issue #324). The v8 provider is used deliberately: the
|
||||
// istanbul provider instruments sources by rewriting their AST, which broke
|
||||
// on the ESM `@docmost/editor-ext` barrel import; v8 collects native
|
||||
// coverage from the runtime and never re-parses ESM, so it sidesteps that.
|
||||
// Thresholds are calibrated a few points BELOW the level measured on
|
||||
// develop so the gate passes today but fails on a real regression. Numbers
|
||||
// reflect the files actually exercised by the suite (`all: false`).
|
||||
coverage: {
|
||||
enabled: true,
|
||||
provider: 'v8',
|
||||
reporter: ['text-summary', 'text'],
|
||||
all: false,
|
||||
thresholds: {
|
||||
statements: 88,
|
||||
branches: 75,
|
||||
functions: 72,
|
||||
lines: 88,
|
||||
},
|
||||
},
|
||||
// Runtime suites. The `.test.ts` glob deliberately EXCLUDES the type-only
|
||||
// contract file (`*.test-d.ts`), which is enforced by the typecheck pass
|
||||
// below instead — so the 35 runtime suites are never typechecked.
|
||||
|
||||
@@ -32,6 +32,7 @@
|
||||
"author": "Moritz Krause",
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@docmost/prosemirror-markdown": "workspace:*",
|
||||
"@fellow/prosemirror-recreate-transform": "^1.2.3",
|
||||
"@hocuspocus/provider": "^3.4.4",
|
||||
"@hocuspocus/transformer": "^3.4.4",
|
||||
@@ -51,6 +52,7 @@
|
||||
"form-data": "^4.0.0",
|
||||
"jsdom": "^27.4.0",
|
||||
"marked": "^17.0.1",
|
||||
"re2": "^1.21.0",
|
||||
"ws": "^8.19.0",
|
||||
"y-prosemirror": "1.3.7",
|
||||
"yjs": "^13.6.29",
|
||||
|
||||
@@ -47,6 +47,7 @@ import {
|
||||
deleteTableRow,
|
||||
updateTableCell,
|
||||
} from "./lib/node-ops.js";
|
||||
import { searchInDoc, SearchOptions } from "./lib/page-search.js";
|
||||
import { withPageLock } from "./lib/page-lock.js";
|
||||
import {
|
||||
applyTextEdits,
|
||||
@@ -807,8 +808,14 @@ export class DocmostClient {
|
||||
await this.ensureAuthenticated();
|
||||
const resultData = await this.getPageRaw(pageId);
|
||||
|
||||
// Agent read: hide resolved-comment anchors so the agent sees only active
|
||||
// discussions. Active anchors are kept. (The lossless export_page_markdown
|
||||
// round-trip deliberately does NOT pass this flag — resolved anchors there
|
||||
// must be preserved.)
|
||||
let content = resultData.content
|
||||
? convertProseMirrorToMarkdown(resultData.content)
|
||||
? convertProseMirrorToMarkdown(resultData.content, {
|
||||
dropResolvedCommentAnchors: true,
|
||||
})
|
||||
: "";
|
||||
|
||||
// Always fetch subpages to provide context to the agent
|
||||
@@ -1093,6 +1100,29 @@ export class DocmostClient {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Find every occurrence of `query` on a page IN MEMORY, over the plain text of
|
||||
* each text container (reusing the same `getPageRaw` fetch as the other read
|
||||
* tools) — no server search endpoint, no whole-document round-trip through the
|
||||
* model. Returns `{ total, truncated, matches }`; each match carries a ref for
|
||||
* get_node/patch_node (the `#<index>` form resolves with get_node but NOT
|
||||
* patch_node — see SearchMatch.nodeId), plus the top-level block index and a
|
||||
* short context window used to build a unique text `selection` for
|
||||
* create_comment (create_comment has no nodeId param). The pure engine
|
||||
* (`searchInDoc`) owns the traversal, glue, the RE2 ReDoS-safe regex engine
|
||||
* and the empty-query / invalid-or-unsupported-regex errors.
|
||||
*/
|
||||
async searchInPage(pageId: string, query: string, opts: SearchOptions = {}) {
|
||||
await this.ensureAuthenticated();
|
||||
const data = await this.getPageRaw(pageId);
|
||||
const result = searchInDoc(
|
||||
data.content ?? { type: "doc", content: [] },
|
||||
query,
|
||||
opts,
|
||||
);
|
||||
return { pageId, query, ...result };
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a table as a matrix. `tableRef` is `#<index>` (from get_outline) or a
|
||||
* block id of any node inside the table. Returns the cell texts plus a
|
||||
@@ -1774,7 +1804,10 @@ export class DocmostClient {
|
||||
const body = page.content ? convertProseMirrorToMarkdown(page.content) : "";
|
||||
let comments: any[] = [];
|
||||
try {
|
||||
comments = await this.listComments(pageId);
|
||||
// Lossless export: include RESOLVED threads so the export -> import
|
||||
// round-trip preserves every comment. This is exactly why the active-only
|
||||
// filter is an opt-in (default false) on listComments.
|
||||
comments = (await this.listComments(pageId, true)).items;
|
||||
} catch (e) {
|
||||
// A comments fetch failure must not lose the body; export with [] and let
|
||||
// the caller see the (empty) comments block. Log under DEBUG only.
|
||||
@@ -2343,8 +2376,21 @@ export class DocmostClient {
|
||||
}
|
||||
}
|
||||
|
||||
/** List all comments on a page (cursor-paginated), content as markdown. */
|
||||
async listComments(pageId: string) {
|
||||
/**
|
||||
* List comments on a page (cursor-paginated), content as markdown.
|
||||
*
|
||||
* DEFAULT (`includeResolved = false`) hides RESOLVED THREADS WHOLESALE so the
|
||||
* agent sees only active discussions: a top-level comment with `resolvedAt`
|
||||
* set AND every reply under it (a reply of a closed thread is part of the
|
||||
* closed thread) are dropped from `items`. `resolvedThreadsHidden` reports how
|
||||
* many resolved top-level threads were hidden so the agent can re-query with
|
||||
* `includeResolved: true` to see everything. Active threads always stay.
|
||||
*
|
||||
* Returns `{ items, resolvedThreadsHidden }` (NOT a bare array) — callers that
|
||||
* need the full feed (lossless export, transformPage, checkNewComments) pass
|
||||
* `includeResolved: true` and read `.items`.
|
||||
*/
|
||||
async listComments(pageId: string, includeResolved = false) {
|
||||
await this.ensureAuthenticated();
|
||||
let allComments: any[] = [];
|
||||
let cursor: string | null = null;
|
||||
@@ -2360,7 +2406,7 @@ export class DocmostClient {
|
||||
cursor = data.meta?.nextCursor || null;
|
||||
} while (cursor);
|
||||
|
||||
return allComments.map((comment: any) => {
|
||||
const mapped = allComments.map((comment: any) => {
|
||||
const markdown = comment.content
|
||||
? convertProseMirrorToMarkdown(
|
||||
this.parseCommentContent(comment.content),
|
||||
@@ -2368,6 +2414,31 @@ export class DocmostClient {
|
||||
: "";
|
||||
return filterComment(comment, markdown);
|
||||
});
|
||||
|
||||
if (includeResolved) {
|
||||
return { items: mapped, resolvedThreadsHidden: 0 };
|
||||
}
|
||||
|
||||
// Ids of RESOLVED top-level threads (a top-level comment has no
|
||||
// parentCommentId). A whole thread is hidden when its root is resolved.
|
||||
const resolvedRootIds = new Set(
|
||||
mapped
|
||||
.filter((c) => !c.parentCommentId && c.resolvedAt != null)
|
||||
.map((c) => c.id),
|
||||
);
|
||||
|
||||
const items = mapped.filter((c) => {
|
||||
// Hide the resolved root itself and every reply anchored to it. A reply's
|
||||
// own resolvedAt is irrelevant — its membership follows the parent thread.
|
||||
// ASSUMPTION: Docmost's comment model is FLAT — a reply's parentCommentId
|
||||
// always points at the thread ROOT (no reply-of-reply nesting), so a single
|
||||
// level of parent lookup covers a whole thread. If nested replies are ever
|
||||
// introduced, a deep reply of a resolved thread would need a root-walk here.
|
||||
if (!c.parentCommentId) return !resolvedRootIds.has(c.id);
|
||||
return !resolvedRootIds.has(c.parentCommentId);
|
||||
});
|
||||
|
||||
return { items, resolvedThreadsHidden: resolvedRootIds.size };
|
||||
}
|
||||
|
||||
async getComment(commentId: string) {
|
||||
@@ -2742,7 +2813,9 @@ export class DocmostClient {
|
||||
const results: any[] = [];
|
||||
for (const page of pagesInScope) {
|
||||
try {
|
||||
const comments = await this.listComments(page.id);
|
||||
// Full feed (incl. resolved): a "new comments since" scan reports all
|
||||
// recent activity; the active-only filter is scoped to list_comments.
|
||||
const comments = (await this.listComments(page.id, true)).items;
|
||||
const newComments = comments.filter(
|
||||
(c: any) => new Date(c.createdAt) > sinceDate,
|
||||
);
|
||||
@@ -3488,7 +3561,9 @@ export class DocmostClient {
|
||||
const deleteComments = opts.deleteComments ?? false;
|
||||
|
||||
await this.ensureAuthenticated();
|
||||
const comments = await this.listComments(pageId);
|
||||
// Full feed (incl. resolved): a page transform (e.g. comments -> footnotes)
|
||||
// must operate on every comment, so it opts into the unfiltered feed.
|
||||
const comments = (await this.listComments(pageId, true)).items;
|
||||
|
||||
// ctx handed to the sandbox. consume() records ids; helpers are the pure
|
||||
// transform primitives. log is captured from console.log inside the sandbox.
|
||||
|
||||
+58
-16
@@ -37,11 +37,20 @@ const VERSION = packageJson.version;
|
||||
|
||||
// Editing guide surfaced to MCP clients in the initialize result so they can
|
||||
// pick the right tool by intent and avoid resending whole documents.
|
||||
const SERVER_INSTRUCTIONS =
|
||||
"Docmost editing guide — choose the tool by intent: fix wording/typos/numbers (text inside blocks) -> edit_page_text (no node id needed). Change ONE block (paragraph/heading/callout/table cell/etc.) structurally -> patch_node (address by attrs.id from get_page_json). Add a block -> insert_node (before/after a block by attrs.id or by anchor text, or append). Remove a block -> delete_node (by attrs.id). Images -> insert_image (add an image from a web URL) / replace_image (swap an existing image for one from a web URL). New page -> create_page (Markdown). Bulk/structural rewrite or nodes without an id -> update_page_json (full ProseMirror replace; prefer the granular tools above to avoid resending the whole ~100KB+ document). Copy/replace a page's whole content from another page (server-side, no document through the model) -> copy_page_content. Rename a page (title only) -> rename_page. Read -> get_page (Markdown, lossy) or get_page_json (lossless ProseMirror with block ids). Comments -> create_comment (always inline; requires an EXACT selection — the contiguous text to anchor/highlight on; fails rather than leaving an unanchored comment), list_comments, update_comment, resolve_comment (resolve/reopen a thread, reversible — prefer over delete to close), delete_comment, check_new_comments. Propose a concrete text fix for one-click human approval -> create_comment with suggestedText (the exact plain-text replacement for the selection; the selection must then be UNIQUE in the page — extend it with context if needed); prefer this over editing directly when the change is subjective or needs the author's sign-off. Tip: read block ids via get_page_json, then use patch_node/insert_node/delete_node so you never resend the full document. " +
|
||||
"Complex/scripted rewrite (multiple coordinated edits, footnotes, renumbering) -> docmost_transform: write a JS `(doc, ctx) => doc` transform, preview the diff with dryRun (default), then apply with dryRun:false; ctx.helpers includes commentsToFootnotes for turning inline comments into numbered footnotes. " +
|
||||
"Review what changed -> diff_page_versions (compare a historyId to current, or two history versions). See a page's saved versions -> list_page_history. Undo a bad edit -> restore_page_version (writes a past version back as current; itself revertible). " +
|
||||
"Lossless markdown round-trip (download, edit, re-upload, incl. comment anchors) -> export_page_markdown / import_page_markdown.";
|
||||
//
|
||||
// MAINTENANCE RULE: when you ADD, RENAME, or REMOVE a tool (either an inline
|
||||
// server.registerTool(...) here or a spec in tool-specs.ts), you MUST update
|
||||
// this guide so the new tool is routed by intent. This is enforced by
|
||||
// test/unit/server-instructions.test.mjs, which fails when a registered tool
|
||||
// name is not mentioned below (see its EXCEPTIONS list for the rare opt-outs).
|
||||
// Exported for that test.
|
||||
export const SERVER_INSTRUCTIONS =
|
||||
"Docmost editing guide — choose the tool by intent.\n" +
|
||||
"READ: find a page -> search (workspace-wide full-text); list -> list_pages / list_spaces. Locate blocks and their ids CHEAPLY -> get_outline (compact top-level map; start here, not get_page_json). One block's subtree -> get_node (by attrs.id, or \"#<index>\" for tables, which carry no id). Find every occurrence of a string/regex ON a page (and where each is) -> search_in_page, NOT block-by-block get_node — it returns each hit's node ref + block index + context for a targeted comment. Whole page -> get_page (Markdown, lossy; inline <span data-comment-id> tags are comment anchors — markup, not text) or get_page_json (lossless ProseMirror with block ids). Hand a huge page (with images) to an external consumer without pulling it through the model context -> stash_page (returns a short-lived anonymous URL).\n" +
|
||||
"EDIT: fix wording/typos/numbers -> edit_page_text (find/replace inside blocks, no node id needed). Change ONE block (paragraph/heading/callout/etc.) structurally -> patch_node (by attrs.id from get_outline). Add a block -> insert_node (before/after a block by attrs.id or by anchor text, or append). Remove a block -> delete_node (by attrs.id). Tables -> table_get / table_update_cell / table_insert_row / table_delete_row (address by \"#<index>\" from get_outline; table nodes have no attrs.id). Images -> insert_image (add from a web URL) / replace_image (swap an existing image). Footnotes -> insert_footnote. Bulk/structural rewrite -> update_page_json (full ProseMirror replace; prefer the granular tools above to avoid resending the whole ~100KB+ document). Complex/scripted rewrite (multiple coordinated edits, renumbering) -> docmost_transform: write a JS `(doc, ctx) => doc` transform, preview the diff with dryRun (default), then apply with dryRun:false; ctx.helpers includes commentsToFootnotes for turning inline comments into numbered footnotes.\n" +
|
||||
"PAGES: new -> create_page (Markdown). Rename (title only) -> rename_page. Move -> move_page. Delete -> delete_page (SOFT delete — the page goes to trash and is restorable; nothing is permanent). Copy/replace a page's whole content from another page (server-side, no document through the model) -> copy_page_content. Sharing -> share_page / unshare_page / list_shares; share_page makes the page PUBLICLY accessible — do it only when explicitly asked.\n" +
|
||||
"COMMENTS: create_comment is always inline and requires an EXACT selection — contiguous text from a single block, <=250 chars (fails rather than leaving an unanchored comment); reply to a thread via parentCommentId. Propose a concrete text fix for one-click human approval -> create_comment with suggestedText (the exact plain-text replacement for the selection; the selection must then be UNIQUE in the page — extend it with context if needed); prefer this over editing directly when the change is subjective or needs the author's sign-off. Manage -> list_comments, update_comment, resolve_comment (resolve/reopen, reversible — prefer over delete to close), delete_comment, check_new_comments.\n" +
|
||||
"HISTORY: review what changed -> diff_page_versions (a historyId vs current, or two versions). List saved versions -> list_page_history. Undo a bad edit -> restore_page_version (writes a past version back as current; itself revertible). Lossless markdown round-trip (download, edit, re-upload, incl. comment anchors) -> export_page_markdown / import_page_markdown.";
|
||||
|
||||
// Helper to format JSON responses
|
||||
const jsonContent = (data: any) => ({
|
||||
@@ -147,7 +156,9 @@ server.registerTool(
|
||||
description:
|
||||
"Get page details with content converted to Markdown. The conversion is " +
|
||||
"LOSSY (block ids, exact table/callout structure are approximated); for a " +
|
||||
"lossless representation use get_page_json.",
|
||||
"lossless representation use get_page_json. Inline <span data-comment-id> " +
|
||||
"tags in the markdown are comment highlight anchors (also present for " +
|
||||
"RESOLVED threads) — treat them as markup, not page text.",
|
||||
inputSchema: {
|
||||
pageId: z.string().min(1),
|
||||
},
|
||||
@@ -176,6 +187,19 @@ registerShared(SHARED_TOOL_SPECS.getNode, async ({ pageId, nodeId }) => {
|
||||
return jsonContent(result);
|
||||
});
|
||||
|
||||
// Tool: search_in_page
|
||||
registerShared(
|
||||
SHARED_TOOL_SPECS.searchInPage,
|
||||
async ({ pageId, query, regex, caseSensitive, limit }) => {
|
||||
const result = await docmostClient.searchInPage(pageId, query, {
|
||||
regex,
|
||||
caseSensitive,
|
||||
limit,
|
||||
});
|
||||
return jsonContent(result);
|
||||
},
|
||||
);
|
||||
|
||||
// Tool: table_get
|
||||
server.registerTool(
|
||||
"table_get",
|
||||
@@ -288,7 +312,8 @@ server.registerTool(
|
||||
"create_page",
|
||||
{
|
||||
description:
|
||||
"Create a new page with content (automatically moves it to the correct hierarchy).",
|
||||
"Create a new page from Markdown in a space. Pass parentPageId to nest " +
|
||||
"it under a parent; omit it to create at the space root.",
|
||||
inputSchema: {
|
||||
title: z.string().min(1).describe("Title of the page"),
|
||||
content: z.string().min(1).describe("Markdown content"),
|
||||
@@ -587,7 +612,8 @@ server.registerTool(
|
||||
{
|
||||
description:
|
||||
"Make a page publicly accessible (idempotent) and return its public " +
|
||||
"URL. The URL format is <app>/share/<key>/p/<slugId>.",
|
||||
"URL. The URL format is <app>/share/<key>/p/<slugId>. This exposes the " +
|
||||
"page content to ANYONE with the URL — do it only when explicitly asked.",
|
||||
inputSchema: {
|
||||
pageId: z.string().min(1).describe("ID of the page to share"),
|
||||
searchIndexing: z
|
||||
@@ -619,7 +645,7 @@ server.registerTool(
|
||||
"move_page",
|
||||
{
|
||||
description:
|
||||
"Move a page to a new parent (nesting) or root. Essential for organizing pages created via 'create_page'.",
|
||||
"Move a page under a new parent (nesting) or to the space root.",
|
||||
inputSchema: {
|
||||
pageId: z.string().min(1),
|
||||
parentPageId: z
|
||||
@@ -675,7 +701,9 @@ server.registerTool(
|
||||
server.registerTool(
|
||||
"delete_page",
|
||||
{
|
||||
description: "Delete a single page by ID.",
|
||||
description:
|
||||
"Delete a single page by ID. SOFT delete only: the page is moved to " +
|
||||
"trash and can be restored; nothing is permanently deleted.",
|
||||
inputSchema: {
|
||||
pageId: z.string().min(1),
|
||||
},
|
||||
@@ -697,13 +725,24 @@ server.registerTool(
|
||||
"list_comments",
|
||||
{
|
||||
description:
|
||||
"List all comments on a page (paginated). Content is returned as Markdown.",
|
||||
"List comments on a page in one call (pagination is handled " +
|
||||
"internally). By DEFAULT only ACTIVE threads are returned; resolved " +
|
||||
"threads (a resolved top-level comment and all its replies) are hidden " +
|
||||
"and their count reported as `resolvedThreadsHidden` so you can re-query " +
|
||||
"with `includeResolved: true` to see everything. Returns " +
|
||||
"`{ items, resolvedThreadsHidden }`. Content is returned as Markdown.",
|
||||
inputSchema: {
|
||||
pageId: z.string().describe("ID of the page"),
|
||||
includeResolved: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe(
|
||||
"default only active threads; true — include resolved",
|
||||
),
|
||||
},
|
||||
},
|
||||
async ({ pageId }) => {
|
||||
const comments = await docmostClient.listComments(pageId);
|
||||
async ({ pageId, includeResolved }) => {
|
||||
const comments = await docmostClient.listComments(pageId, includeResolved);
|
||||
return jsonContent(comments);
|
||||
},
|
||||
);
|
||||
@@ -913,8 +952,9 @@ server.registerTool(
|
||||
"search",
|
||||
{
|
||||
description:
|
||||
"Search for pages and content. Results are bounded by `limit` " +
|
||||
"(default applied by the client, max 100).",
|
||||
"Full-text search for pages and content across the whole workspace. " +
|
||||
"Results are bounded by `limit` (1-100; when omitted the server applies " +
|
||||
"its own default).",
|
||||
inputSchema: {
|
||||
query: z.string().min(1).describe("Search query"),
|
||||
limit: z
|
||||
@@ -970,7 +1010,9 @@ server.registerTool(
|
||||
"insertInlineFootnote(doc, {anchorText, text}) (author-inline footnote: " +
|
||||
"marker + dedup'd definition, list derived). Footnote convention: markers are " +
|
||||
"plain '[N]' text in the body; the notes are an orderedList under a " +
|
||||
"heading whose text is 'Примечания переводчика'. The transform runs " +
|
||||
"heading whose text is 'Примечания переводчика' (that is only the DEFAULT " +
|
||||
"notesHeading — pass the notesHeading option to the helpers to use a " +
|
||||
"heading matching the page's language). The transform runs " +
|
||||
"sandboxed (no require/process/fs/network, 5s timeout) and must return a " +
|
||||
"{type:'doc'} node.",
|
||||
inputSchema: {
|
||||
|
||||
@@ -2,18 +2,24 @@ import { HocuspocusProvider } from "@hocuspocus/provider";
|
||||
import { TiptapTransformer } from "@hocuspocus/transformer";
|
||||
import * as Y from "yjs";
|
||||
import WebSocket from "ws";
|
||||
import { marked } from "marked";
|
||||
import { generateJSON } from "@tiptap/html";
|
||||
import { Node as PMNode } from "@tiptap/pm/model";
|
||||
import { updateYFragment } from "y-prosemirror";
|
||||
import { JSDOM } from "jsdom";
|
||||
// #293 STEP 5: the pure markdown -> ProseMirror import path is now owned by the
|
||||
// shared package (canonical `^[…]` footnotes, `$…$` math, `==` highlight, the
|
||||
// media-family md forms, comment-directive attrs, callouts and task lists all
|
||||
// handled there). MCP consumes it directly instead of maintaining its own
|
||||
// drifted marked pipeline; only the collab/yjs write glue and the footnote
|
||||
// canonicalization wrapper stay mcp-side.
|
||||
import { markdownToProseMirror } from "@docmost/prosemirror-markdown";
|
||||
import { docmostExtensions, docmostSchema } from "./docmost-schema.js";
|
||||
import { withPageLock } from "./page-lock.js";
|
||||
import { sanitizeForYjs, findUnstorableAttr } from "./node-ops.js";
|
||||
import { lexFootnoteLines } from "./footnote-lex.js";
|
||||
import { canonicalizeFootnotes } from "./footnote-canonicalize.js";
|
||||
import { summarizeChange, VerifyReport } from "./diff.js";
|
||||
|
||||
export { markdownToProseMirror };
|
||||
|
||||
/**
|
||||
* Build the descriptive error for an opaque Yjs encode failure ("Unexpected
|
||||
* content type"), shared by both encode paths (`buildYDoc` -> `toYdoc` and
|
||||
@@ -51,382 +57,27 @@ global.WebSocket = WebSocket;
|
||||
// global.navigator = dom.window.navigator;
|
||||
|
||||
/**
|
||||
* Hard ceiling above which we skip callout preprocessing entirely. The linear
|
||||
* scanner below has no quadratic blow-up, but we still cap input defensively so
|
||||
* a pathological multi-megabyte payload cannot tie up the event loop; in that
|
||||
* case the markdown is passed through verbatim (callouts are simply not
|
||||
* detected) rather than risking a slow scan.
|
||||
*/
|
||||
const MAX_CALLOUT_PREPROCESS_BYTES = 4 * 1024 * 1024; // 4 MB
|
||||
|
||||
/** Matches an opening callout fence: `:::type` (type captured, lower-cased). */
|
||||
const CALLOUT_OPEN_RE = /^:::\s*(\w+)\s*$/;
|
||||
/** Matches a bare closing callout fence: `:::`. */
|
||||
const CALLOUT_CLOSE_RE = /^:::\s*$/;
|
||||
/** Matches the start/end of a code fence (``` or ~~~), capturing the marker. */
|
||||
const CODE_FENCE_RE = /^(\s*)(`{3,}|~{3,})/;
|
||||
|
||||
/**
|
||||
* Pre-process Docmost-flavoured markdown: convert `:::type ... :::`
|
||||
* callout blocks (the syntax our markdown export produces) into HTML
|
||||
* divs that the callout extension parses. The inner content is rendered
|
||||
* through marked as regular markdown.
|
||||
* Page-write variant of the package's `markdownToProseMirror`: imports markdown
|
||||
* then re-runs mcp's footnote canonicalizer over the result.
|
||||
*
|
||||
* Implemented as a single linear pass over the lines (no quadratic regex
|
||||
* rescan). It:
|
||||
* - tracks fenced code regions (```...``` and ~~~...~~~) and never treats a
|
||||
* `:::` line that lives inside a code fence as a callout delimiter, so a
|
||||
* callout body that itself contains a fenced code block with a `:::` line is
|
||||
* no longer corrupted;
|
||||
* - matches an opening `:::type` line with the next CLOSING `:::` at the SAME
|
||||
* nesting level, supporting NESTED callouts via a depth counter (an inner
|
||||
* `:::type` opens a deeper level and consumes a matching `:::`);
|
||||
* - emits the same `<div data-type="callout" data-callout-type="TYPE">` output
|
||||
* (inner rendered through marked) as the previous regex implementation.
|
||||
*/
|
||||
async function preprocessCallouts(markdown: string): Promise<string> {
|
||||
// Defensive cap: skip preprocessing for pathologically large inputs.
|
||||
if (markdown.length > MAX_CALLOUT_PREPROCESS_BYTES) {
|
||||
return markdown;
|
||||
}
|
||||
|
||||
// Recursively transform a slice of lines, converting top-level callouts in
|
||||
// that slice into <div> blocks and rendering their inner content (which may
|
||||
// itself contain nested callouts) through this same function.
|
||||
const transform = async (lines: string[]): Promise<string> => {
|
||||
const out: string[] = [];
|
||||
let inCodeFence = false;
|
||||
let codeFenceMarker = ""; // the exact run of backticks/tildes that opened it
|
||||
let i = 0;
|
||||
|
||||
while (i < lines.length) {
|
||||
const line = lines[i];
|
||||
|
||||
// Inside a code fence, only its matching closing fence is significant;
|
||||
// everything else (including `:::` lines) is copied through verbatim.
|
||||
if (inCodeFence) {
|
||||
out.push(line);
|
||||
const fence = line.match(CODE_FENCE_RE);
|
||||
if (fence && fence[2].startsWith(codeFenceMarker[0]) &&
|
||||
fence[2].length >= codeFenceMarker.length) {
|
||||
inCodeFence = false;
|
||||
codeFenceMarker = "";
|
||||
}
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// A code fence opening outside any callout body: enter code-fence mode.
|
||||
const fenceOpen = line.match(CODE_FENCE_RE);
|
||||
if (fenceOpen) {
|
||||
inCodeFence = true;
|
||||
codeFenceMarker = fenceOpen[2];
|
||||
out.push(line);
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// An opening callout fence: scan forward (with code-fence and nested
|
||||
// callout awareness) for its matching closing `:::` at the same level.
|
||||
const open = line.match(CALLOUT_OPEN_RE);
|
||||
if (open) {
|
||||
const type = open[1].toLowerCase();
|
||||
const bodyLines: string[] = [];
|
||||
let depth = 1;
|
||||
let innerInCodeFence = false;
|
||||
let innerCodeFenceMarker = "";
|
||||
let j = i + 1;
|
||||
for (; j < lines.length; j++) {
|
||||
const bl = lines[j];
|
||||
if (innerInCodeFence) {
|
||||
const f = bl.match(CODE_FENCE_RE);
|
||||
if (f && f[2].startsWith(innerCodeFenceMarker[0]) &&
|
||||
f[2].length >= innerCodeFenceMarker.length) {
|
||||
innerInCodeFence = false;
|
||||
innerCodeFenceMarker = "";
|
||||
}
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
const innerFence = bl.match(CODE_FENCE_RE);
|
||||
if (innerFence) {
|
||||
innerInCodeFence = true;
|
||||
innerCodeFenceMarker = innerFence[2];
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
if (CALLOUT_OPEN_RE.test(bl)) {
|
||||
depth++;
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
if (CALLOUT_CLOSE_RE.test(bl)) {
|
||||
depth--;
|
||||
if (depth === 0) break; // matching close for THIS callout
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
bodyLines.push(bl);
|
||||
}
|
||||
|
||||
if (j < lines.length) {
|
||||
// Found the matching closing fence: render the body (recursively, so
|
||||
// nested callouts are handled) and emit the callout div.
|
||||
const inner = await transform(bodyLines);
|
||||
const renderedInner = await marked.parse(inner);
|
||||
out.push(
|
||||
`\n<div data-type="callout" data-callout-type="${type}">${renderedInner}</div>\n`,
|
||||
);
|
||||
i = j + 1; // skip past the closing `:::`
|
||||
continue;
|
||||
}
|
||||
// No matching close (unterminated callout): treat the opener as a
|
||||
// literal line and continue, preserving the original text.
|
||||
out.push(line);
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
out.push(line);
|
||||
i++;
|
||||
}
|
||||
|
||||
return out.join("\n");
|
||||
};
|
||||
|
||||
return transform(markdown.split("\n"));
|
||||
}
|
||||
|
||||
/**
|
||||
* Bridge marked's checkbox lists to TipTap task lists.
|
||||
* Footnote layering after #293 STEP 5:
|
||||
* - The package's `markdownToProseMirror` already ASSEMBLES footnotes on import
|
||||
* (canon #2): inline `^[body]` markers become the schema's
|
||||
* `footnoteReference` + a single doc-level `footnotesList`, with ids assigned
|
||||
* sequentially (`fn-1`, `fn-2`, …) in first-reference order and identical
|
||||
* bodies merged. So the import output is ALREADY in canonical footnote
|
||||
* topology.
|
||||
* - `canonicalizeFootnotes` runs AFTER as the mcp write-path invariant shared
|
||||
* with every other full-document persist path (`update_page_json`,
|
||||
* `docmost_transform`, `insert_footnote`, …). Because the package output is
|
||||
* already canonical, this layer is a no-op here (idempotent) — it exists so
|
||||
* the page-write contract is enforced uniformly regardless of how the PM doc
|
||||
* was produced, not because the import needs fixing.
|
||||
*
|
||||
* marked renders GitHub task list items (`- [x] done`) as a plain
|
||||
* `<ul><li><p><input type="checkbox" checked> text</p></li></ul>` WITHOUT the
|
||||
* markup TipTap's TaskList/TaskItem extensions parse. This rewrites such lists
|
||||
* into the shape those extensions expect:
|
||||
* TaskList parseHTML matches `ul[data-type="taskList"]`,
|
||||
* TaskItem matches `li[data-type="taskItem"]`,
|
||||
* the checked state is read from `data-checked === "true"`.
|
||||
*
|
||||
* A list is only converted when it has at least one `<li>` and EVERY direct
|
||||
* `<li>` contains a checkbox input. Both `<ul>` and `<ol>` are considered: a
|
||||
* numbered checklist (`1. [x] a`, which marked renders as an `<ol>` of checkbox
|
||||
* `<li>`s) would otherwise lose its task state. TipTap task lists are unordered,
|
||||
* so a matching `<ol>` is emitted as `data-type="taskList"` exactly like a
|
||||
* `<ul>`. Mixed or ordinary lists (including ordinary `<ol>` lists) are left
|
||||
* untouched so they keep rendering as bullet/numbered lists. The marked `<p>`
|
||||
* wrapper is kept inside the `<li>` because TaskItem content allows paragraphs.
|
||||
*/
|
||||
function bridgeTaskLists(html: string): string {
|
||||
// Cheap early-out: if the markup contains no checkbox input at all there is
|
||||
// nothing to bridge, so skip the expensive JSDOM parse entirely. This is the
|
||||
// common case (most pages have no task lists).
|
||||
if (!/type=["']?checkbox/i.test(html)) {
|
||||
return html;
|
||||
}
|
||||
// Defensive cap (consistent with preprocessCallouts): skip the bridge for
|
||||
// pathologically large inputs rather than running a second expensive JSDOM
|
||||
// parse on a multi-megabyte payload. The markup is passed through verbatim.
|
||||
if (html.length > MAX_CALLOUT_PREPROCESS_BYTES) {
|
||||
return html;
|
||||
}
|
||||
const dom = new JSDOM(html);
|
||||
const document = dom.window.document;
|
||||
// Collect the checkbox(es) that belong to THIS <li> directly: either direct
|
||||
// child <input type="checkbox"> elements or ones inside the <li>'s direct <p>
|
||||
// child (the shape marked emits: `<li><p><input type="checkbox"> text</p></li>`).
|
||||
// Checkboxes nested deeper (e.g. inside a child <ul>/<ol>) are excluded so a
|
||||
// bullet <li> that merely contains a nested task sublist is not misdetected.
|
||||
// Raw inline HTML can put more than one checkbox in a single <li>; we gather
|
||||
// ALL of them so none survive into the converted item.
|
||||
const directCheckboxes = (li: Element): Element[] => {
|
||||
const found: Element[] = [];
|
||||
for (const child of Array.from(li.children)) {
|
||||
if (
|
||||
child.tagName === "INPUT" &&
|
||||
child.getAttribute("type") === "checkbox"
|
||||
) {
|
||||
found.push(child);
|
||||
continue;
|
||||
}
|
||||
if (child.tagName === "P") {
|
||||
for (const inp of Array.from(
|
||||
child.querySelectorAll(":scope > input[type='checkbox']"),
|
||||
)) {
|
||||
found.push(inp);
|
||||
}
|
||||
}
|
||||
}
|
||||
return found;
|
||||
};
|
||||
// Both <ul> and <ol> are candidates: an <ol> whose every direct <li> carries
|
||||
// its own checkbox is a numbered checklist that must also become a taskList.
|
||||
const lists = Array.from(document.querySelectorAll("ul, ol"));
|
||||
for (const list of lists) {
|
||||
// Only consider DIRECT child <li> elements; nested lists are handled by
|
||||
// their own iteration of the outer loop.
|
||||
const items = Array.from(list.children).filter(
|
||||
(child) => child.tagName === "LI",
|
||||
);
|
||||
if (items.length === 0) continue;
|
||||
const itemCheckboxes = items.map((li) => directCheckboxes(li));
|
||||
// Convert only when every direct <li> carries at least one OWN checkbox.
|
||||
if (!itemCheckboxes.every((boxes) => boxes.length > 0)) continue;
|
||||
|
||||
// A numbered checklist arrives as an <ol>. We must NOT leave the tag as
|
||||
// <ol> while tagging it data-type="taskList": generateJSON would then match
|
||||
// BOTH the orderedList rule (tag ol) and the taskList rule (data-type),
|
||||
// emitting a phantom empty orderedList beside the real taskList. So rename a
|
||||
// qualifying <ol> to a <ul> — move its <li> children over and replace it —
|
||||
// leaving only the taskList rule to match. Already-<ul> lists are unchanged.
|
||||
let target: Element = list;
|
||||
if (list.tagName === "OL") {
|
||||
const ul = document.createElement("ul");
|
||||
// Carry over existing attributes (e.g. class) so nothing is silently lost.
|
||||
for (const attr of Array.from(list.attributes)) {
|
||||
ul.setAttribute(attr.name, attr.value);
|
||||
}
|
||||
// Move every child node (including the <li>s we collected) into the <ul>.
|
||||
while (list.firstChild) {
|
||||
ul.appendChild(list.firstChild);
|
||||
}
|
||||
list.replaceWith(ul);
|
||||
target = ul;
|
||||
}
|
||||
|
||||
target.setAttribute("data-type", "taskList");
|
||||
items.forEach((li, index) => {
|
||||
const boxes = itemCheckboxes[index];
|
||||
// The first checkbox determines the checked state (matches the previous
|
||||
// single-checkbox behaviour); any extras only need removing.
|
||||
const input = boxes[0] ?? null;
|
||||
li.setAttribute("data-type", "taskItem");
|
||||
const checked =
|
||||
input != null &&
|
||||
(input.hasAttribute("checked") || (input as any).checked);
|
||||
li.setAttribute("data-checked", checked ? "true" : "false");
|
||||
// Remove ALL direct checkbox inputs so none survive into the content
|
||||
// (a raw-inline-HTML <li> may carry more than one).
|
||||
for (const box of boxes) {
|
||||
box.remove();
|
||||
}
|
||||
});
|
||||
}
|
||||
return document.body.innerHTML;
|
||||
}
|
||||
|
||||
// Mirror of packages/editor-ext footnote markdown handling. A `[^id]` inline
|
||||
// marker becomes <sup data-footnote-ref data-id="id">, and `[^id]: text`
|
||||
// definition lines are collected into a single <section data-footnotes>.
|
||||
// Definition detection + fence handling are shared with analyzeFootnotes via
|
||||
// lexFootnoteLines (footnote-lex.js). FOOTNOTE_REF_RE is the inline tokenizer's.
|
||||
const FOOTNOTE_REF_RE = /\[\^([^\]\s]+)\]/;
|
||||
|
||||
function escapeFootnoteAttr(value: string): string {
|
||||
return String(value).replace(/&/g, "&").replace(/"/g, """);
|
||||
}
|
||||
|
||||
const footnoteRefMarkedExtension = {
|
||||
name: "footnoteRef",
|
||||
level: "inline" as const,
|
||||
start(src: string) {
|
||||
return src.match(/\[\^/)?.index ?? -1;
|
||||
},
|
||||
tokenizer(src: string) {
|
||||
const match = FOOTNOTE_REF_RE.exec(src);
|
||||
if (match && match.index === 0) {
|
||||
return { type: "footnoteRef", raw: match[0], id: match[1] };
|
||||
}
|
||||
return undefined;
|
||||
},
|
||||
renderer(token: any) {
|
||||
return `<sup data-footnote-ref data-id="${escapeFootnoteAttr(
|
||||
token.id,
|
||||
)}"></sup>`;
|
||||
},
|
||||
};
|
||||
|
||||
marked.use({ extensions: [footnoteRefMarkedExtension] });
|
||||
|
||||
/**
|
||||
* Pull `[^id]: text` definition lines out of the body and render a single
|
||||
* <section data-footnotes> for them (or "" when there are none).
|
||||
*/
|
||||
function extractFootnotes(markdown: string): {
|
||||
body: string;
|
||||
section: string;
|
||||
} {
|
||||
const bodyLines: string[] = [];
|
||||
const defs: Array<{ id: string; text: string }> = [];
|
||||
// Shared lexer (footnote-lex): a `[^id]: ...` line inside a ``` / ~~~ code
|
||||
// block is inert and stays in the body verbatim; only real definition lines
|
||||
// are pulled out. analyzeFootnotes() consumes the SAME lexer so its diagnostics
|
||||
// match exactly what import keeps/strips (#166).
|
||||
for (const tok of lexFootnoteLines(markdown)) {
|
||||
if (!tok.inFence && tok.definition) defs.push(tok.definition);
|
||||
else bodyLines.push(tok.line);
|
||||
}
|
||||
if (defs.length === 0) return { body: markdown, section: "" };
|
||||
|
||||
// Duplicate definition ids: FIRST WINS, the rest are DROPPED (mirror of
|
||||
// editor-ext extractFootnoteDefinitions). Reference markers are left untouched
|
||||
// so repeated `[^a]` references reuse the single footnote (Pandoc semantics,
|
||||
// #166). The dropped duplicate is surfaced to the caller via analyzeFootnotes
|
||||
// (`duplicateDefinitions`), not silently lost. MUST stay in sync with the
|
||||
// editor-ext mirror.
|
||||
const firstById = new Map<string, string>(); // id -> first definition text
|
||||
for (const def of defs) {
|
||||
if (!firstById.has(def.id)) firstById.set(def.id, def.text);
|
||||
}
|
||||
|
||||
const inner = [...firstById.entries()]
|
||||
.map(
|
||||
([id, text]) =>
|
||||
`<div data-footnote-def data-id="${escapeFootnoteAttr(
|
||||
id,
|
||||
)}"><p>${marked.parseInline(text || "")}</p></div>`,
|
||||
)
|
||||
.join("");
|
||||
return {
|
||||
body: bodyLines.join("\n"),
|
||||
section: `<section data-footnotes>${inner}</section>`,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert markdown to a ProseMirror doc using the full Docmost schema.
|
||||
*
|
||||
* This conversion does NOT canonicalize footnotes — it is the shared, content-
|
||||
* preserving primitive used by BOTH page write paths and COMMENT bodies
|
||||
* (createComment / updateComment). Canonicalization MUST NOT run on a comment
|
||||
* body: a comment may legitimately contain a footnote-definition line
|
||||
* (`[^1]: text`) with no matching reference, and the canonicalizer drops a
|
||||
* reference-less footnotesList — which would silently delete the comment's text.
|
||||
*
|
||||
* Page write paths that DO need the canonical footnote topology call
|
||||
* `markdownToProseMirrorCanonical` instead (markdown import, update_page markdown
|
||||
* path). Keep this function reference-loss-free.
|
||||
*/
|
||||
export async function markdownToProseMirror(
|
||||
markdownContent: string,
|
||||
): Promise<any> {
|
||||
const withCallouts = await preprocessCallouts(markdownContent);
|
||||
const { body, section } = extractFootnotes(withCallouts);
|
||||
const html = (await marked.parse(body)) + section;
|
||||
const bridged = bridgeTaskLists(html);
|
||||
return generateJSON(bridged, docmostExtensions);
|
||||
}
|
||||
|
||||
/**
|
||||
* Page-write variant of `markdownToProseMirror`: converts markdown then enforces
|
||||
* the canonical footnote topology. The footnote `section` markdown is emitted in
|
||||
* DEFINITION order, but numbering derives from REFERENCE order, so without this
|
||||
* the bottom list renders out of order (`1, 4, 2, 3, …`); orphan definitions and
|
||||
* duplicate lists are also normalized. Idempotent — a no-op once canonical, and a
|
||||
* no-op for footnote-free content.
|
||||
*
|
||||
* Use this ONLY for full-document PAGE writes (never for comment bodies, where it
|
||||
* would drop a reference-less footnote definition — see `markdownToProseMirror`).
|
||||
* Use this ONLY for full-document PAGE writes. Comment bodies call the package's
|
||||
* plain `markdownToProseMirror` (no canonicalization) — safe now because inline
|
||||
* `^[body]` footnotes carry their body at the reference point, so a comment can
|
||||
* no longer produce a reference-less footnote definition to be dropped.
|
||||
*/
|
||||
export async function markdownToProseMirrorCanonical(
|
||||
markdownContent: string,
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Footnote diagnostics for imported Markdown (issue #166).
|
||||
* Legacy footnote diagnostics for imported Markdown (issue #166).
|
||||
*
|
||||
* A PURE, fence-aware text scan (independent of the Markdown->ProseMirror
|
||||
* conversion path, so it reports the same problems for `create_page`,
|
||||
@@ -7,11 +7,18 @@
|
||||
* importer still creates the page; this only surfaces footnote problems to the
|
||||
* caller so an agent can fix its own markup instead of shipping broken footnotes.
|
||||
*
|
||||
* SCOPE after #293 STEP 5: the canonical import form is now inline `^[body]`
|
||||
* footnotes (handled by `@docmost/prosemirror-markdown`), where these problems
|
||||
* cannot arise. This scan therefore targets the LEGACY reference-style
|
||||
* (`[^id]` / `[^id]:`) markup, which is now inert on import (left as literal
|
||||
* text). The warnings remain useful as an advisory nudge when an agent still
|
||||
* authors the old syntax, but they no longer describe what the importer builds.
|
||||
*
|
||||
* Detected problems:
|
||||
* - danglingReferences: a `[^id]` reference with no `[^id]:` definition.
|
||||
* - emptyDefinitions: a `[^id]:` whose (kept) text is empty/whitespace.
|
||||
* - duplicateDefinitions: an id defined by two or more `[^id]:` lines (only the
|
||||
* first is kept on import — first-wins; see extractFootnotes).
|
||||
* first would have been kept under the old first-wins import).
|
||||
* - referencesInTables: a `[^id]` marker found in a GFM table row (heuristic:
|
||||
* the line, trimmed, starts with `|`) — footnotes in table cells often do not
|
||||
* render as expected.
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
/**
|
||||
* Shared, fence-aware line lexer for footnote markdown (MCP-internal).
|
||||
* Shared, fence-aware line lexer for legacy footnote markdown (MCP-internal).
|
||||
*
|
||||
* Both the importer (`extractFootnotes` in collaboration.ts, which strips
|
||||
* definition lines and rebuilds a footnotes section) and the diagnostics
|
||||
* (`analyzeFootnotes` in footnote-analyze.ts) must agree EXACTLY on which lines
|
||||
* are definitions and which lines are inert (inside a code fence). Sharing one
|
||||
* lexer makes "the analyzer sees what the importer leaves" a structural property
|
||||
* instead of two hand-kept copies that can drift (#166 review).
|
||||
* Since #293 STEP 5 the markdown -> ProseMirror IMPORT path lives in the shared
|
||||
* `@docmost/prosemirror-markdown` package (inline `^[body]` footnotes), so this
|
||||
* lexer no longer backs an mcp importer. It now backs ONLY the import-time
|
||||
* diagnostics (`analyzeFootnotes` in footnote-analyze.ts), which still scan the
|
||||
* raw markdown for legacy reference-style `[^id]:` definition lines and surface
|
||||
* advisory warnings (duplicate/orphan definitions) about content that is now
|
||||
* inert on import. Fence-awareness (a `[^id]:` line inside a ``` / ~~~ block is
|
||||
* NOT a definition) is the property the analyzer relies on.
|
||||
*
|
||||
* NOTE: this is deliberately NOT shared with editor-ext's
|
||||
* `extractFootnoteDefinitions` — that lives in a different package and the
|
||||
|
||||
@@ -1,903 +1,15 @@
|
||||
/**
|
||||
* Convert ProseMirror/TipTap JSON content to Markdown
|
||||
* Supports all Docmost-specific node types and extensions
|
||||
* ProseMirror -> Docmost-flavoured Markdown converter.
|
||||
*
|
||||
* #293 STEP 5: the converter CORE now lives in the shared
|
||||
* `@docmost/prosemirror-markdown` package (the canonical, lossless
|
||||
* implementation carrying every git-sync fix and the #293 canon decisions).
|
||||
* MCP consumes it directly instead of keeping its own drifted copy, so the two
|
||||
* can never diverge again. This file is a thin re-export shim kept only so the
|
||||
* many existing `./markdown-converter.js` importers (client.ts, tests) do not
|
||||
* have to move.
|
||||
*/
|
||||
export function convertProseMirrorToMarkdown(content: any): string {
|
||||
if (!content || !content.content) return "";
|
||||
|
||||
// Escape a value interpolated into an HTML double-quoted attribute value
|
||||
// (textAlign, colors, image src, math `text`, all data-* attrs, etc.). In the
|
||||
// ATTRIBUTE context only the quote that delimits the value and the ampersand
|
||||
// that starts an entity are special, so we escape ONLY & " (and ' for safety
|
||||
// when single-quoted delimiters are used). We deliberately do NOT escape < or
|
||||
// >: the HTML re-parser (parse5/jsdom via @tiptap/html) does NOT decode
|
||||
// </> back inside attribute values, so escaping them would corrupt the
|
||||
// stored data (e.g. a math node's LaTeX `a < b`) and ACCUMULATE escapes on
|
||||
// every round-trip (`a < b` -> `a < b` -> `a &lt; b`). Escaping & "
|
||||
// keeps the value inert against attribute-injection while staying idempotent.
|
||||
// NOTE: escape ONLY & and " here. The value is always wrapped in double
|
||||
// quotes, so " is the only delimiter; ' is NOT special in a double-quoted
|
||||
// value, and parse5 does not decode ' back inside attribute values, so
|
||||
// escaping ' would (like < >) corrupt the value and accumulate & on every
|
||||
// round-trip. Escaping & and " is idempotent (parse5 decodes them back).
|
||||
const escapeAttr = (value: unknown): string =>
|
||||
String(value)
|
||||
.replace(/&/g, "&")
|
||||
.replace(/"/g, """);
|
||||
|
||||
// Escape a value placed as HTML element TEXT content (between tags), where
|
||||
// <, >, and & are all significant. Used for text rendered inside raw-HTML
|
||||
// blocks (table cells / columns) so stored characters cannot inject markup.
|
||||
const escapeHtmlText = (value: unknown): string =>
|
||||
String(value)
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">");
|
||||
|
||||
// Percent-encode characters that would break out of a markdown URL target
|
||||
// (...) — whitespace/newlines and parentheses — so a stored src stays a
|
||||
// single inert token (used for image/video/youtube srcs).
|
||||
const encodeMdUrl = (value: unknown): string =>
|
||||
String(value || "")
|
||||
.replace(/\s/g, (c: string) => (c === " " ? "%20" : encodeURIComponent(c)))
|
||||
.replace(/\(/g, "%28")
|
||||
.replace(/\)/g, "%29");
|
||||
|
||||
const processNode = (node: any): string => {
|
||||
const type = node.type;
|
||||
const nodeContent = node.content || [];
|
||||
|
||||
switch (type) {
|
||||
case "doc":
|
||||
return nodeContent.map(processNode).join("\n\n");
|
||||
|
||||
case "paragraph":
|
||||
const text = nodeContent.map(processNode).join("");
|
||||
const align = node.attrs?.textAlign;
|
||||
if (align && align !== "left") {
|
||||
return `<div align="${escapeAttr(align)}">${text}</div>`;
|
||||
}
|
||||
return text || "";
|
||||
|
||||
case "heading":
|
||||
const level = node.attrs?.level || 1;
|
||||
const headingText = nodeContent.map(processNode).join("");
|
||||
return "#".repeat(level) + " " + headingText;
|
||||
|
||||
case "text":
|
||||
let textContent = node.text || "";
|
||||
// Apply marks (bold, italic, code, etc.)
|
||||
if (node.marks) {
|
||||
// Markdown code spans (`...`) cannot carry inner formatting, so when a
|
||||
// run has the `code` mark alongside ANY other mark, backtick syntax
|
||||
// would leak literal ** / []() into the code text. In that case emit
|
||||
// nested HTML (<code> innermost, the other marks wrapping it as HTML)
|
||||
// so the output is at least well-formed and re-parseable.
|
||||
//
|
||||
// NOTE: this does NOT round-trip both marks. The schema's `code` mark
|
||||
// has `excludes: "_"` (it excludes every other mark), so on import the
|
||||
// co-occurring mark is always dropped — the run comes back as `code`
|
||||
// only. We keep the emission simple and accept that the other mark is
|
||||
// lost; preserving both is impossible while `code` excludes them.
|
||||
// Only use the backtick form when `code` is the sole mark.
|
||||
const markTypes = node.marks.map((m: any) => m.type);
|
||||
const hasCode = markTypes.includes("code");
|
||||
const codeCombined = hasCode && markTypes.length > 1;
|
||||
for (const mark of node.marks) {
|
||||
switch (mark.type) {
|
||||
case "bold":
|
||||
textContent = codeCombined
|
||||
? `<strong>${textContent}</strong>`
|
||||
: `**${textContent}**`;
|
||||
break;
|
||||
case "italic":
|
||||
textContent = codeCombined
|
||||
? `<em>${textContent}</em>`
|
||||
: `*${textContent}*`;
|
||||
break;
|
||||
case "code":
|
||||
// When combined with another mark, wrap as <code> so the
|
||||
// surrounding HTML marks can nest around it; otherwise use the
|
||||
// plain backtick span.
|
||||
textContent = codeCombined
|
||||
? `<code>${textContent}</code>`
|
||||
: `\`${textContent}\``;
|
||||
break;
|
||||
case "link": {
|
||||
const href = mark.attrs?.href || "";
|
||||
const title = mark.attrs?.title;
|
||||
if (codeCombined) {
|
||||
// Emit an HTML anchor so it can wrap the nested <code>.
|
||||
const safeHref = escapeAttr(href);
|
||||
if (title) {
|
||||
textContent = `<a href="${safeHref}" title="${escapeAttr(String(title))}">${textContent}</a>`;
|
||||
} else {
|
||||
textContent = `<a href="${safeHref}">${textContent}</a>`;
|
||||
}
|
||||
} else if (title) {
|
||||
// Emit the optional markdown link title; escape an embedded
|
||||
// double-quote so it cannot terminate the title string early.
|
||||
const safeTitle = String(title).replace(/"/g, '\\"');
|
||||
textContent = `[${textContent}](${href} "${safeTitle}")`;
|
||||
} else {
|
||||
textContent = `[${textContent}](${href})`;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case "strike":
|
||||
textContent = codeCombined
|
||||
? `<s>${textContent}</s>`
|
||||
: `~~${textContent}~~`;
|
||||
break;
|
||||
case "underline":
|
||||
textContent = `<u>${textContent}</u>`;
|
||||
break;
|
||||
case "subscript":
|
||||
textContent = `<sub>${textContent}</sub>`;
|
||||
break;
|
||||
case "superscript":
|
||||
textContent = `<sup>${textContent}</sup>`;
|
||||
break;
|
||||
case "highlight": {
|
||||
// Preserve a null/empty color as a plain highlight (a bare
|
||||
// <mark> with no background-color); only emit the style when a
|
||||
// color is actually set, so a plain highlight is not forced to
|
||||
// yellow on export.
|
||||
const color = mark.attrs?.color;
|
||||
textContent = color
|
||||
? `<mark style="background-color: ${escapeAttr(color)}">${textContent}</mark>`
|
||||
: `<mark>${textContent}</mark>`;
|
||||
break;
|
||||
}
|
||||
case "textStyle":
|
||||
if (mark.attrs?.color) {
|
||||
textContent = `<span style="color: ${escapeAttr(mark.attrs.color)}">${textContent}</span>`;
|
||||
}
|
||||
break;
|
||||
case "comment": {
|
||||
// Emit the inline comment anchor so highlights round-trip. The
|
||||
// schema's Comment mark parses span[data-comment-id] (attrs
|
||||
// commentId/resolved).
|
||||
const cid = mark.attrs?.commentId;
|
||||
if (cid) {
|
||||
const resolvedAttr = mark.attrs?.resolved
|
||||
? ` data-resolved="true"`
|
||||
: "";
|
||||
textContent = `<span data-comment-id="${escapeAttr(cid)}"${resolvedAttr}>${textContent}</span>`;
|
||||
}
|
||||
break;
|
||||
}
|
||||
case "spoiler":
|
||||
// Markdown has no native spoiler syntax, so emit the same
|
||||
// lossless raw HTML the editor-ext turndown rule produces; the
|
||||
// schema's Spoiler mark parses span[data-spoiler] back on import.
|
||||
textContent = `<span data-spoiler="true">${textContent}</span>`;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
return textContent;
|
||||
|
||||
case "codeBlock":
|
||||
const language = node.attrs?.language || "";
|
||||
// Strip ALL trailing newlines so the export is idempotent: marked
|
||||
// re-adds exactly one trailing "\n" on import, so trimming only one
|
||||
// here would let the text grow by "\n" on each round-trip. Removing
|
||||
// every trailing newline makes repeated cycles stable.
|
||||
const code = nodeContent
|
||||
.map(processNode)
|
||||
.join("")
|
||||
.replace(/\n+$/, "");
|
||||
return "```" + language + "\n" + code + "\n```";
|
||||
|
||||
case "bulletList":
|
||||
return nodeContent
|
||||
.map((item: any) => processListItem(item, "-"))
|
||||
.join("\n");
|
||||
|
||||
case "orderedList":
|
||||
return nodeContent
|
||||
.map((item: any, index: number) =>
|
||||
processListItem(item, `${index + 1}.`),
|
||||
)
|
||||
.join("\n");
|
||||
|
||||
case "taskList":
|
||||
return nodeContent.map((item: any) => processTaskItem(item)).join("\n");
|
||||
|
||||
case "taskItem":
|
||||
// Delegate to the same helper used by taskList so multi-block and
|
||||
// nested task items render and indent consistently.
|
||||
return processTaskItem(node);
|
||||
|
||||
case "listItem":
|
||||
return nodeContent.map(processNode).join("\n");
|
||||
|
||||
case "blockquote":
|
||||
// Prefix EVERY line of EVERY child with "> " and separate block-level
|
||||
// children with a blank ">" line so code blocks / multi-paragraph
|
||||
// quotes round-trip correctly.
|
||||
return nodeContent
|
||||
.map((n: any) =>
|
||||
processNode(n)
|
||||
.split("\n")
|
||||
.map((line: string) => (line.length ? `> ${line}` : ">"))
|
||||
.join("\n"),
|
||||
)
|
||||
.join("\n>\n");
|
||||
|
||||
case "horizontalRule":
|
||||
return "---";
|
||||
|
||||
case "hardBreak":
|
||||
// Two trailing spaces before the newline encode a markdown hard break;
|
||||
// a bare "\n" would be reimported as a soft break and lost.
|
||||
return " \n";
|
||||
|
||||
case "image": {
|
||||
const imgAlt = node.attrs?.alt || "";
|
||||
const imgCaption = node.attrs?.caption || "";
|
||||
if (imgCaption) {
|
||||
// ![]() can't carry a caption, so (symmetric to video) emit a raw
|
||||
// <img> wrapped in a block <div>. On import marked.parse keeps the raw
|
||||
// HTML and generateJSON runs the image extension's parseHTML, which
|
||||
// restores the caption from data-caption.
|
||||
const parts: string[] = [`src="${escapeAttr(node.attrs?.src ?? "")}"`];
|
||||
if (imgAlt) parts.push(`alt="${escapeAttr(imgAlt)}"`);
|
||||
parts.push(`data-caption="${escapeAttr(imgCaption)}"`);
|
||||
return `<div><img ${parts.join(" ")}></div>`;
|
||||
}
|
||||
// Neutralize characters that could break out of the markdown image
|
||||
// URL: spaces/newlines and parentheses would terminate the (...) target
|
||||
// and let a stored src inject following markdown/HTML. Percent-encode
|
||||
// them so the URL stays a single inert token.
|
||||
const imgSrc = encodeMdUrl(node.attrs?.src);
|
||||
return ``;
|
||||
}
|
||||
|
||||
case "video": {
|
||||
// Emit the schema-matching <video> element so generateJSON rebuilds the
|
||||
// node with its attrs intact. The schema's parseHTML reads src/aria-label
|
||||
// from the standard attributes and the remaining attrs from data-*.
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [`src="${escapeAttr(attrs.src ?? "")}"`];
|
||||
if (attrs.alt) parts.push(`aria-label="${escapeAttr(attrs.alt)}"`);
|
||||
if (attrs.attachmentId)
|
||||
parts.push(
|
||||
`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`,
|
||||
);
|
||||
if (attrs.width != null)
|
||||
parts.push(`width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null)
|
||||
parts.push(`height="${escapeAttr(attrs.height)}"`);
|
||||
if (attrs.size != null)
|
||||
parts.push(`data-size="${escapeAttr(attrs.size)}"`);
|
||||
if (attrs.align)
|
||||
parts.push(`data-align="${escapeAttr(attrs.align)}"`);
|
||||
if (attrs.aspectRatio != null)
|
||||
parts.push(`data-aspect-ratio="${escapeAttr(attrs.aspectRatio)}"`);
|
||||
// Wrap in a block <div> so marked treats it as a block (a bare <video>
|
||||
// is inline-level HTML and marked wraps it in <p>, leaving a spurious
|
||||
// empty paragraph beside the hoisted block atom). The wrapper has no
|
||||
// data-type, so the schema parser ignores it and just hoists the video.
|
||||
return `<div><video ${parts.join(" ")}></video></div>`;
|
||||
}
|
||||
|
||||
case "youtube": {
|
||||
// Emit the schema-matching div[data-type="youtube"]; the schema reads
|
||||
// src from data-src and width/height/align from data-* attributes.
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [
|
||||
`data-type="youtube"`,
|
||||
`data-src="${escapeAttr(attrs.src ?? "")}"`,
|
||||
];
|
||||
if (attrs.width != null)
|
||||
parts.push(`data-width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null)
|
||||
parts.push(`data-height="${escapeAttr(attrs.height)}"`);
|
||||
if (attrs.align)
|
||||
parts.push(`data-align="${escapeAttr(attrs.align)}"`);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
case "table": {
|
||||
// A GFM pipe table cannot represent merged cells. If ANY cell carries
|
||||
// colspan>1 or rowspan>1, a pipe table would corrupt the grid on
|
||||
// re-import, so emit the WHOLE table as raw HTML <table> instead: the
|
||||
// schema's table family parseHTML (tag table/tr/td/th, with colspan/
|
||||
// rowspan read from the same-named HTML attrs and align via parseHTML)
|
||||
// round-trips it faithfully. Otherwise keep the lighter GFM pipe table.
|
||||
const tableRows: any[] = nodeContent;
|
||||
if (tableRows.length === 0) return "";
|
||||
const hasSpan = tableRows.some((row: any) =>
|
||||
(row.content || []).some(
|
||||
(cell: any) =>
|
||||
(cell.attrs?.colspan ?? 1) > 1 || (cell.attrs?.rowspan ?? 1) > 1,
|
||||
),
|
||||
);
|
||||
|
||||
if (hasSpan) {
|
||||
// Render each cell's block children to HTML (marked does NOT parse
|
||||
// markdown inside a raw HTML block, so emitting markdown here would
|
||||
// leak literal ** / `` into the cell). blockToHtml mirrors the schema
|
||||
// HTML so inner formatting re-parses into the right marks/nodes.
|
||||
const renderHtmlCell = (cell: any): string => {
|
||||
const tag = cell.type === "tableHeader" ? "th" : "td";
|
||||
const a = cell.attrs || {};
|
||||
const cellParts: string[] = [];
|
||||
if ((a.colspan ?? 1) > 1)
|
||||
cellParts.push(`colspan="${escapeAttr(a.colspan)}"`);
|
||||
if ((a.rowspan ?? 1) > 1)
|
||||
cellParts.push(`rowspan="${escapeAttr(a.rowspan)}"`);
|
||||
if (a.align) cellParts.push(`align="${escapeAttr(a.align)}"`);
|
||||
const open = cellParts.length
|
||||
? `<${tag} ${cellParts.join(" ")}>`
|
||||
: `<${tag}>`;
|
||||
const inner = (cell.content || [])
|
||||
.map((block: any) => blockToHtml(block))
|
||||
.join("");
|
||||
return `${open}${inner}</${tag}>`;
|
||||
};
|
||||
const htmlRows = tableRows
|
||||
.map(
|
||||
(row: any) =>
|
||||
`<tr>${(row.content || []).map(renderHtmlCell).join("")}</tr>`,
|
||||
)
|
||||
.join("");
|
||||
return `<table><tbody>${htmlRows}</tbody></table>`;
|
||||
}
|
||||
|
||||
// No merged cells: emit a GFM table (header row + separator) so the
|
||||
// markdown can be parsed back into a table on re-import.
|
||||
const rows = tableRows.map(processNode);
|
||||
const headerCells = tableRows[0]?.content || [];
|
||||
const columns = headerCells.length || 1;
|
||||
// Derive alignment markers (:--, :-:, --:) from each header cell.
|
||||
const markers = Array.from({ length: columns }, (_, i) => {
|
||||
const align = headerCells[i]?.attrs?.align;
|
||||
switch (align) {
|
||||
case "left":
|
||||
return ":--";
|
||||
case "center":
|
||||
return ":-:";
|
||||
case "right":
|
||||
return "--:";
|
||||
default:
|
||||
return "---";
|
||||
}
|
||||
});
|
||||
const separator = "| " + markers.join(" | ") + " |";
|
||||
return [rows[0], separator, ...rows.slice(1)].join("\n");
|
||||
}
|
||||
|
||||
case "tableRow":
|
||||
return "| " + nodeContent.map(processNode).join(" | ") + " |";
|
||||
|
||||
case "tableCell":
|
||||
case "tableHeader": {
|
||||
// Join multiple block children with a space (not "") so adjacent blocks
|
||||
// like a paragraph followed by a list don't collide into "line1- a".
|
||||
// Then collapse newlines and escape pipes so a cell containing "|" or a
|
||||
// line break cannot corrupt the surrounding GFM row.
|
||||
return nodeContent
|
||||
.map(processNode)
|
||||
.join(" ")
|
||||
.replace(/\r?\n/g, " ")
|
||||
.replace(/\|/g, "\\|");
|
||||
}
|
||||
|
||||
case "callout":
|
||||
const calloutType = node.attrs?.type || "info";
|
||||
const calloutContent = nodeContent.map(processNode).join("\n");
|
||||
return `:::${calloutType.toLowerCase()}\n${calloutContent}\n:::`;
|
||||
|
||||
case "details":
|
||||
return nodeContent.map(processNode).join("\n");
|
||||
|
||||
case "detailsSummary":
|
||||
const summaryText = nodeContent.map(processNode).join("");
|
||||
return `<details>\n<summary>${summaryText}</summary>\n`;
|
||||
|
||||
case "detailsContent":
|
||||
const detailsText = nodeContent.map(processNode).join("\n");
|
||||
return `${detailsText}\n</details>`;
|
||||
|
||||
case "mathInline": {
|
||||
// The schema's `text` attribute has no parseHTML, so TipTap's default
|
||||
// parser reads it from the `text` HTML attribute (NOT the element's text
|
||||
// content). Emit span[data-type="mathInline"] carrying the LaTeX in a
|
||||
// `text="..."` attribute so it round-trips. marked cannot parse $...$
|
||||
// back, so the previous form was lossy.
|
||||
const inlineMath = node.attrs?.text || "";
|
||||
return `<span data-type="mathInline" data-katex="true" text="${escapeAttr(inlineMath)}"></span>`;
|
||||
}
|
||||
|
||||
case "mathBlock": {
|
||||
// Same as mathInline: the LaTeX must ride in the `text` HTML attribute
|
||||
// for the schema's default parser to recover it.
|
||||
const blockMath = node.attrs?.text || "";
|
||||
return `<div data-type="mathBlock" data-katex="true" text="${escapeAttr(blockMath)}"></div>`;
|
||||
}
|
||||
|
||||
case "mention": {
|
||||
// Emit span[data-type="mention"] with the schema's data-* attributes so
|
||||
// generateJSON rebuilds the mention node instead of leaving "@label"
|
||||
// plain text that cannot re-parse.
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [`data-type="mention"`];
|
||||
if (attrs.id) parts.push(`data-id="${escapeAttr(attrs.id)}"`);
|
||||
if (attrs.label)
|
||||
parts.push(`data-label="${escapeAttr(attrs.label)}"`);
|
||||
if (attrs.entityType)
|
||||
parts.push(`data-entity-type="${escapeAttr(attrs.entityType)}"`);
|
||||
if (attrs.entityId)
|
||||
parts.push(`data-entity-id="${escapeAttr(attrs.entityId)}"`);
|
||||
if (attrs.slugId)
|
||||
parts.push(`data-slug-id="${escapeAttr(attrs.slugId)}"`);
|
||||
if (attrs.creatorId)
|
||||
parts.push(`data-creator-id="${escapeAttr(attrs.creatorId)}"`);
|
||||
if (attrs.anchorId)
|
||||
parts.push(`data-anchor-id="${escapeAttr(attrs.anchorId)}"`);
|
||||
// Keep the label as visible text content too; the schema reads attrs
|
||||
// from data-*, so the inner text is purely cosmetic and harmless.
|
||||
const mentionLabel = attrs.label || attrs.id || "";
|
||||
// The label is visible element TEXT content here (the data-* attrs above
|
||||
// carry the real values), so escape it for the text context, not attrs.
|
||||
return `<span ${parts.join(" ")}>@${escapeHtmlText(mentionLabel)}</span>`;
|
||||
}
|
||||
|
||||
case "footnoteReference": {
|
||||
// Pandoc/GFM inline marker. The number is derived (not stored), so the
|
||||
// id is the stable anchor.
|
||||
const fnId = node.attrs?.id || "";
|
||||
return fnId ? `[^${fnId}]` : "";
|
||||
}
|
||||
|
||||
case "footnotesList":
|
||||
// The container renders its definitions, each on its own `[^id]: ...`
|
||||
// line. A blank line separates the body from the notes block.
|
||||
return nodeContent.map(processNode).join("\n");
|
||||
|
||||
case "footnoteDefinition": {
|
||||
const defId = node.attrs?.id || "";
|
||||
// Collapse the definition's paragraphs into a single line; multi-line
|
||||
// footnotes are a v2 refinement.
|
||||
const defText = nodeContent
|
||||
.map(processNode)
|
||||
.join(" ")
|
||||
.replace(/\s*\n+\s*/g, " ")
|
||||
.trim();
|
||||
return defId ? `[^${defId}]: ${defText}` : "";
|
||||
}
|
||||
|
||||
case "attachment": {
|
||||
// BUG FIX: the old code read node.attrs.fileName / node.attrs.src, but
|
||||
// the schema stores name/url (plus mime/size/attachmentId). Emit the
|
||||
// schema-matching div[data-type="attachment"] with data-attachment-*
|
||||
// attrs so the node round-trips instead of degrading to a markdown link.
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [
|
||||
`data-type="attachment"`,
|
||||
`data-attachment-url="${escapeAttr(attrs.url ?? "")}"`,
|
||||
];
|
||||
if (attrs.name)
|
||||
parts.push(`data-attachment-name="${escapeAttr(attrs.name)}"`);
|
||||
if (attrs.mime)
|
||||
parts.push(`data-attachment-mime="${escapeAttr(attrs.mime)}"`);
|
||||
if (attrs.size != null)
|
||||
parts.push(`data-attachment-size="${escapeAttr(attrs.size)}"`);
|
||||
if (attrs.attachmentId)
|
||||
parts.push(
|
||||
`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`,
|
||||
);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
case "drawio":
|
||||
case "excalidraw": {
|
||||
// Emit the schema-matching div[data-type=...] carrying the diagram's
|
||||
// attrs as data-* (the schema's diagramAttributes reads src/title/alt/
|
||||
// width/height/size/aspectRatio/align/attachmentId from data-*), so the
|
||||
// diagram round-trips instead of degrading to a lossy placeholder.
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [
|
||||
`data-type="${type}"`,
|
||||
`data-src="${escapeAttr(attrs.src ?? "")}"`,
|
||||
];
|
||||
if (attrs.title != null)
|
||||
parts.push(`data-title="${escapeAttr(attrs.title)}"`);
|
||||
if (attrs.alt != null) parts.push(`data-alt="${escapeAttr(attrs.alt)}"`);
|
||||
if (attrs.width != null)
|
||||
parts.push(`data-width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null)
|
||||
parts.push(`data-height="${escapeAttr(attrs.height)}"`);
|
||||
if (attrs.size != null)
|
||||
parts.push(`data-size="${escapeAttr(attrs.size)}"`);
|
||||
if (attrs.aspectRatio != null)
|
||||
parts.push(`data-aspect-ratio="${escapeAttr(attrs.aspectRatio)}"`);
|
||||
if (attrs.align)
|
||||
parts.push(`data-align="${escapeAttr(attrs.align)}"`);
|
||||
if (attrs.attachmentId)
|
||||
parts.push(
|
||||
`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`,
|
||||
);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
case "embed": {
|
||||
// Emit the schema-matching div[data-type="embed"]; the schema reads
|
||||
// src/provider/align/width/height from data-* attributes so the node
|
||||
// (and its provider iframe info) survives the round-trip.
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [
|
||||
`data-type="embed"`,
|
||||
`data-src="${escapeAttr(attrs.src ?? "")}"`,
|
||||
`data-provider="${escapeAttr(attrs.provider ?? "")}"`,
|
||||
];
|
||||
if (attrs.align)
|
||||
parts.push(`data-align="${escapeAttr(attrs.align)}"`);
|
||||
if (attrs.width != null)
|
||||
parts.push(`data-width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null)
|
||||
parts.push(`data-height="${escapeAttr(attrs.height)}"`);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
case "audio": {
|
||||
// Emit the schema-matching <audio> element (was emitting nothing). The
|
||||
// schema reads src from src and attachmentId/size from data-*.
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [`src="${escapeAttr(attrs.src ?? "")}"`];
|
||||
if (attrs.attachmentId)
|
||||
parts.push(
|
||||
`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`,
|
||||
);
|
||||
if (attrs.size != null)
|
||||
parts.push(`data-size="${escapeAttr(attrs.size)}"`);
|
||||
// Wrap in a block <div> for the same reason as video: a bare <audio> is
|
||||
// inline-level HTML that marked would wrap in <p>.
|
||||
return `<div><audio ${parts.join(" ")}></audio></div>`;
|
||||
}
|
||||
|
||||
case "pdf": {
|
||||
// Emit the schema-matching div[data-type="pdf"] (was emitting nothing).
|
||||
// The schema reads src/width/height from standard attrs and name/
|
||||
// attachmentId/size from data-*.
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [
|
||||
`data-type="pdf"`,
|
||||
`src="${escapeAttr(attrs.src ?? "")}"`,
|
||||
];
|
||||
if (attrs.name) parts.push(`data-name="${escapeAttr(attrs.name)}"`);
|
||||
if (attrs.attachmentId)
|
||||
parts.push(
|
||||
`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`,
|
||||
);
|
||||
if (attrs.size != null)
|
||||
parts.push(`data-size="${escapeAttr(attrs.size)}"`);
|
||||
if (attrs.width != null)
|
||||
parts.push(`width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null)
|
||||
parts.push(`height="${escapeAttr(attrs.height)}"`);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
case "columns": {
|
||||
// Emit the schema-matching div[data-type="columns"] wrapper so the
|
||||
// multi-column layout survives. Without a case the children were
|
||||
// concatenated with no separator and the text merged. The schema reads
|
||||
// layout from data-layout and widthMode from data-width-mode. The whole
|
||||
// block is raw HTML, so render children via blockToHtml (NOT markdown,
|
||||
// which marked would not re-parse inside a raw HTML block).
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [`data-type="columns"`];
|
||||
if (attrs.layout)
|
||||
parts.push(`data-layout="${escapeAttr(attrs.layout)}"`);
|
||||
if (attrs.widthMode && attrs.widthMode !== "normal")
|
||||
parts.push(`data-width-mode="${escapeAttr(attrs.widthMode)}"`);
|
||||
const inner = nodeContent.map((n: any) => blockToHtml(n)).join("");
|
||||
return `<div ${parts.join(" ")}>${inner}</div>`;
|
||||
}
|
||||
|
||||
case "column": {
|
||||
// Emit the schema-matching div[data-type="column"]; the schema reads the
|
||||
// column width from data-width. Children are rendered as HTML so their
|
||||
// formatting survives inside this raw HTML block.
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [`data-type="column"`];
|
||||
if (attrs.width)
|
||||
parts.push(`data-width="${escapeAttr(attrs.width)}"`);
|
||||
const inner = nodeContent.map((n: any) => blockToHtml(n)).join("");
|
||||
return `<div ${parts.join(" ")}>${inner}</div>`;
|
||||
}
|
||||
|
||||
case "subpages":
|
||||
return "{{SUBPAGES}}";
|
||||
|
||||
default:
|
||||
// Fallback: process children
|
||||
return nodeContent.map(processNode).join("");
|
||||
}
|
||||
};
|
||||
|
||||
// Render inline content (text runs + their marks) to HTML. Used by the raw
|
||||
// HTML fallbacks (spanned tables, columns) where marked will NOT re-parse
|
||||
// markdown, so backtick/asterisk/bracket syntax would otherwise leak as
|
||||
// literal characters. Each mark is mirrored to the HTML the schema's parseHTML
|
||||
// accepts so it re-imports as the matching ProseMirror mark.
|
||||
const inlineToHtml = (inlineNodes: any[]): string =>
|
||||
(inlineNodes || [])
|
||||
.map((n: any) => {
|
||||
if (n.type === "hardBreak") return "<br>";
|
||||
if (n.type !== "text") {
|
||||
// Inline atoms (mention, mathInline) already emit schema HTML.
|
||||
return processNode(n);
|
||||
}
|
||||
let t = escapeHtmlText(n.text || "");
|
||||
for (const mark of n.marks || []) {
|
||||
switch (mark.type) {
|
||||
case "bold":
|
||||
t = `<strong>${t}</strong>`;
|
||||
break;
|
||||
case "italic":
|
||||
t = `<em>${t}</em>`;
|
||||
break;
|
||||
case "code":
|
||||
t = `<code>${t}</code>`;
|
||||
break;
|
||||
case "strike":
|
||||
t = `<s>${t}</s>`;
|
||||
break;
|
||||
case "underline":
|
||||
t = `<u>${t}</u>`;
|
||||
break;
|
||||
case "subscript":
|
||||
t = `<sub>${t}</sub>`;
|
||||
break;
|
||||
case "superscript":
|
||||
t = `<sup>${t}</sup>`;
|
||||
break;
|
||||
case "link":
|
||||
t = `<a href="${escapeAttr(mark.attrs?.href || "")}">${t}</a>`;
|
||||
break;
|
||||
case "highlight":
|
||||
t = mark.attrs?.color
|
||||
? `<mark style="background-color: ${escapeAttr(mark.attrs.color)}">${t}</mark>`
|
||||
: `<mark>${t}</mark>`;
|
||||
break;
|
||||
case "textStyle":
|
||||
if (mark.attrs?.color)
|
||||
t = `<span style="color: ${escapeAttr(mark.attrs.color)}">${t}</span>`;
|
||||
break;
|
||||
case "comment":
|
||||
// Inline comment anchor inside a raw-HTML container (columns /
|
||||
// spanned table cells), so commented text there also round-trips.
|
||||
if (mark.attrs?.commentId) {
|
||||
const r = mark.attrs?.resolved ? ` data-resolved="true"` : "";
|
||||
t = `<span data-comment-id="${escapeAttr(mark.attrs.commentId)}"${r}>${t}</span>`;
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
return t;
|
||||
})
|
||||
.join("");
|
||||
|
||||
// Emit the schema-matching <img> for an image node. Shared so the image is
|
||||
// emitted as real HTML wherever a raw-HTML container needs it (inside a column
|
||||
// or a spanned table cell), where markdown `` would NOT be re-parsed
|
||||
// and would survive as literal text. The Image extension reads src/alt from
|
||||
// the standard attributes; the Docmost extra attrs (width/height/align/size/
|
||||
// attachmentId/aspectRatio) are global attributes read from same-named DOM
|
||||
// attributes, so emit them by name.
|
||||
const imageToHtml = (node: any): string => {
|
||||
const attrs = node.attrs || {};
|
||||
const parts: string[] = [`src="${escapeAttr(attrs.src ?? "")}"`];
|
||||
if (attrs.alt) parts.push(`alt="${escapeAttr(attrs.alt)}"`);
|
||||
if (attrs.caption)
|
||||
parts.push(`data-caption="${escapeAttr(attrs.caption)}"`);
|
||||
if (attrs.title) parts.push(`title="${escapeAttr(attrs.title)}"`);
|
||||
if (attrs.width != null) parts.push(`width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null) parts.push(`height="${escapeAttr(attrs.height)}"`);
|
||||
if (attrs.align) parts.push(`align="${escapeAttr(attrs.align)}"`);
|
||||
if (attrs.size != null) parts.push(`data-size="${escapeAttr(attrs.size)}"`);
|
||||
if (attrs.attachmentId)
|
||||
parts.push(`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`);
|
||||
if (attrs.aspectRatio != null)
|
||||
parts.push(`data-aspect-ratio="${escapeAttr(attrs.aspectRatio)}"`);
|
||||
return `<img ${parts.join(" ")}>`;
|
||||
};
|
||||
|
||||
// Emit the schema-matching div[data-type="callout"] for a callout node. The
|
||||
// schema reads the banner type from data-callout-type. Children are rendered
|
||||
// as HTML so they survive inside a raw-HTML container.
|
||||
const calloutToHtml = (node: any): string => {
|
||||
const type = (node.attrs?.type || "info").toLowerCase();
|
||||
const inner = (node.content || []).map(blockToHtml).join("");
|
||||
return `<div data-type="callout" data-callout-type="${escapeAttr(type)}">${inner}</div>`;
|
||||
};
|
||||
|
||||
// Emit a schema-matching <details> tree. The schema parses <details>,
|
||||
// summary[data-type="detailsSummary"], and div[data-type="detailsContent"].
|
||||
const detailsToHtml = (node: any): string => {
|
||||
const inner = (node.content || []).map(blockToHtml).join("");
|
||||
return `<details>${inner}</details>`;
|
||||
};
|
||||
const detailsSummaryToHtml = (node: any): string =>
|
||||
`<summary data-type="detailsSummary">${inlineToHtml(node.content || [])}</summary>`;
|
||||
const detailsContentToHtml = (node: any): string => {
|
||||
const inner = (node.content || []).map(blockToHtml).join("");
|
||||
return `<div data-type="detailsContent">${inner}</div>`;
|
||||
};
|
||||
|
||||
// Emit the schema-matching taskList/taskItem HTML. bridgeTaskLists (in
|
||||
// collaboration.ts) recognizes ul[data-type="taskList"] with
|
||||
// li[data-type="taskItem"][data-checked]; emitting that directly here keeps
|
||||
// task lists inside columns/cells from degrading to literal "- [ ]" text.
|
||||
const taskListToHtml = (node: any): string => {
|
||||
const items = (node.content || [])
|
||||
.map((it: any) => {
|
||||
const checked = it.attrs?.checked ? "true" : "false";
|
||||
return `<li data-type="taskItem" data-checked="${checked}">${blockChildrenToHtml(it)}</li>`;
|
||||
})
|
||||
.join("");
|
||||
return `<ul data-type="taskList">${items}</ul>`;
|
||||
};
|
||||
|
||||
// Render a block node to HTML for the raw-HTML containers (spanned tables,
|
||||
// columns). marked does NOT re-parse markdown inside a raw-HTML block, so
|
||||
// EVERY block type that can appear inside a column or a spanned cell must be
|
||||
// emitted as schema-matching HTML here — never as markdown, or it would land
|
||||
// as literal text on re-import. Nodes whose processNode case already produces
|
||||
// schema-matching HTML (math/media/embed/attachment/nested columns/spanned
|
||||
// table) are delegated to processNode; the markdown-emitting cases
|
||||
// (image/blockquote/callout/details/hr/taskList) get explicit HTML here.
|
||||
const blockToHtml = (block: any): string => {
|
||||
const children = block.content || [];
|
||||
switch (block.type) {
|
||||
case "paragraph":
|
||||
return `<p>${inlineToHtml(children)}</p>`;
|
||||
case "heading": {
|
||||
const level = block.attrs?.level || 1;
|
||||
return `<h${level}>${inlineToHtml(children)}</h${level}>`;
|
||||
}
|
||||
case "bulletList":
|
||||
return `<ul>${children
|
||||
.map((li: any) => `<li>${blockChildrenToHtml(li)}</li>`)
|
||||
.join("")}</ul>`;
|
||||
case "orderedList":
|
||||
return `<ol>${children
|
||||
.map((li: any) => `<li>${blockChildrenToHtml(li)}</li>`)
|
||||
.join("")}</ol>`;
|
||||
case "codeBlock": {
|
||||
const lang = block.attrs?.language || "";
|
||||
// The code itself is element TEXT content (between <code> tags), so it
|
||||
// must escape < > & — NOT the attribute escaper. The language rides in
|
||||
// a class ATTRIBUTE, so it uses escapeAttr.
|
||||
const code = escapeHtmlText(
|
||||
children
|
||||
.map(processNode)
|
||||
.join("")
|
||||
.replace(/\n+$/, ""),
|
||||
);
|
||||
const cls = lang ? ` class="language-${escapeAttr(lang)}"` : "";
|
||||
return `<pre><code${cls}>${code}</code></pre>`;
|
||||
}
|
||||
case "image":
|
||||
return imageToHtml(block);
|
||||
case "blockquote":
|
||||
return `<blockquote>${children.map(blockToHtml).join("")}</blockquote>`;
|
||||
case "horizontalRule":
|
||||
return "<hr>";
|
||||
case "callout":
|
||||
return calloutToHtml(block);
|
||||
case "details":
|
||||
return detailsToHtml(block);
|
||||
case "detailsSummary":
|
||||
return detailsSummaryToHtml(block);
|
||||
case "detailsContent":
|
||||
return detailsContentToHtml(block);
|
||||
case "taskList":
|
||||
return taskListToHtml(block);
|
||||
case "taskItem":
|
||||
// A bare taskItem (outside a taskList) still needs a wrapping list so
|
||||
// the schema parses it; wrap it in a single-item taskList.
|
||||
return taskListToHtml({ content: [block] });
|
||||
// table (incl. spanned), columns/column, math, media, embed, attachment,
|
||||
// mention, etc. already emit schema-matching HTML from processNode.
|
||||
case "table":
|
||||
case "columns":
|
||||
case "column":
|
||||
case "mathBlock":
|
||||
case "video":
|
||||
case "audio":
|
||||
case "pdf":
|
||||
case "youtube":
|
||||
case "embed":
|
||||
case "attachment":
|
||||
case "drawio":
|
||||
case "excalidraw":
|
||||
return processNode(block);
|
||||
default:
|
||||
// Any still-unhandled block type: NEVER fall back to markdown inside a
|
||||
// raw-HTML block (it would become literal text). Wrap its rendered
|
||||
// children in a <div> so their content is preserved; if it has no block
|
||||
// children, render its inline content instead.
|
||||
if (children.length && children.some((c: any) => c.type !== "text")) {
|
||||
return `<div>${children.map(blockToHtml).join("")}</div>`;
|
||||
}
|
||||
return `<div>${inlineToHtml(children)}</div>`;
|
||||
}
|
||||
};
|
||||
|
||||
// Render the block children of a list item to HTML (a listItem holds block+
|
||||
// content). Mirrors processListItem but for the HTML fallback path.
|
||||
const blockChildrenToHtml = (item: any): string =>
|
||||
(item.content || []).map((b: any) => blockToHtml(b)).join("");
|
||||
|
||||
// Indent the rendered children of a list item under a marker prefix.
|
||||
// Each child block is a (possibly multi-line) string. The very first physical
|
||||
// line of the first child carries the marker (e.g. "- " or "1. "); EVERY
|
||||
// other line — the remaining lines of the first child AND all lines of every
|
||||
// subsequent child (nested lists, code blocks, extra paragraphs) — is indented
|
||||
// to align under the marker. Without indenting these continuation lines, the
|
||||
// 2nd/3rd line of a nested child collapses to column 0 and escapes the list.
|
||||
//
|
||||
// The continuation indent MUST equal the LIST marker width, which is not the
|
||||
// same as the visible prefix width:
|
||||
// - bullet "- " -> 2 columns
|
||||
// - task "- [ ] " -> marker is still "- " (the "[ ] " is content), 2
|
||||
// - ordered "1. "/"10. " -> 3/4 columns, scaling with the number's digits
|
||||
// CommonMark anchors nested content to the marker column, so an ordered item
|
||||
// indented to only 2 columns would be re-parsed as a sibling/loose content on
|
||||
// re-import. Callers therefore pass the exact indent width to use.
|
||||
const indentItemChildren = (
|
||||
childStrings: string[],
|
||||
prefix: string,
|
||||
indentWidth: number,
|
||||
): string => {
|
||||
const indent = " ".repeat(indentWidth);
|
||||
const lines: string[] = [];
|
||||
childStrings.forEach((child, childIndex) => {
|
||||
child.split("\n").forEach((line, lineIndex) => {
|
||||
if (childIndex === 0 && lineIndex === 0) {
|
||||
// First physical line of the first block gets the marker.
|
||||
lines.push(`${prefix} ${line}`);
|
||||
} else {
|
||||
// Indent every continuation line by the marker width; keep blank
|
||||
// lines blank rather than emitting trailing whitespace.
|
||||
lines.push(line.length ? `${indent}${line}` : "");
|
||||
}
|
||||
});
|
||||
});
|
||||
return lines.join("\n");
|
||||
};
|
||||
|
||||
const processListItem = (item: any, prefix: string): string => {
|
||||
const itemContent = item.content || [];
|
||||
const childStrings = itemContent.map(processNode);
|
||||
if (childStrings.length === 0) return prefix;
|
||||
// The rendered marker is `${prefix} ` (prefix + one space), so its width —
|
||||
// and thus the continuation indent — is prefix.length + 1. This is correct
|
||||
// for both bullet ("-" -> 2) and ordered ("1." -> 3, "10." -> 4) markers,
|
||||
// since for those the visible prefix IS the list marker.
|
||||
return indentItemChildren(childStrings, prefix, prefix.length + 1);
|
||||
};
|
||||
|
||||
const processTaskItem = (item: any): string => {
|
||||
const checked = item.attrs?.checked || false;
|
||||
const checkbox = checked ? "[x]" : "[ ]";
|
||||
const prefix = `- ${checkbox}`;
|
||||
const itemContent = item.content || [];
|
||||
const childStrings = itemContent.map(processNode);
|
||||
// An empty task item still needs its checkbox marker; without this guard
|
||||
// the indent below produces "" and the "- [ ]"/"- [x]" row disappears.
|
||||
if (childStrings.length === 0) return prefix;
|
||||
// The list marker for a task item is just "- " (2 columns); the "[ ] "/"[x] "
|
||||
// checkbox is item content, NOT part of the marker. So the continuation
|
||||
// indent is a fixed 2 — do NOT derive it from the wider prefix.length.
|
||||
return indentItemChildren(childStrings, prefix, 2);
|
||||
};
|
||||
|
||||
return processNode(content).trim();
|
||||
}
|
||||
export {
|
||||
convertProseMirrorToMarkdown,
|
||||
type ConvertProseMirrorToMarkdownOptions,
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
|
||||
@@ -1,136 +1,15 @@
|
||||
/**
|
||||
* Self-contained Docmost-flavoured Markdown document (custom extensions).
|
||||
* Self-contained Docmost-flavoured Markdown document envelope (`docmost:meta` /
|
||||
* `docmost:comments` blocks).
|
||||
*
|
||||
* A single `.md` file that packages everything needed to losslessly round-trip
|
||||
* a page through "download -> edit body -> re-upload":
|
||||
* - a leading `docmost:meta` block: a one-line JSON object with page identity;
|
||||
* - the Markdown body (carrying inline comment anchors and diagrams as HTML);
|
||||
* - a trailing `docmost:comments` block: a one-line JSON array of comment
|
||||
* threads.
|
||||
*
|
||||
* Both metadata blocks are HTML comments on purpose: `marked`/`generateJSON`
|
||||
* drop HTML comments, so even if the WHOLE file were ever fed straight to the
|
||||
* importer without first stripping the blocks, the metadata cannot leak into the
|
||||
* document. (A fenced ```docmost-comments``` block would WRONGLY become a
|
||||
* codeBlock node, so a fenced block is deliberately NOT used.)
|
||||
*
|
||||
* The delimiter literals may legitimately appear in the BODY too (e.g. a user
|
||||
* re-pastes an exported `.md` into a page, or a page documents this very
|
||||
* format). To stay robust, parsing treats only the FINAL, document-ending
|
||||
* `docmost:comments` block as metadata: it is the last `<!-- docmost:comments`
|
||||
* opener whose closing `-->` sits at the very end of the file. Any earlier
|
||||
* literal occurrence is left in the body untouched.
|
||||
*
|
||||
* NOTE on comments: in this version the comment THREAD records are preserved in
|
||||
* the file but are NOT pushed back to the server on import — only the inline
|
||||
* comment marks (anchors) embedded in the body are restored. Managing comment
|
||||
* records stays with the comment tools/UI.
|
||||
* #293 STEP 5: this envelope is now owned by the shared
|
||||
* `@docmost/prosemirror-markdown` package (the mcp copy was byte-identical to
|
||||
* the package's, so re-exporting is lossless). Kept as a thin shim so the
|
||||
* existing `./markdown-document.js` importers (client.ts, tests) do not move.
|
||||
*/
|
||||
|
||||
export interface DocmostMdMeta {
|
||||
version: number;
|
||||
pageId?: string;
|
||||
slugId?: string;
|
||||
title?: string;
|
||||
spaceId?: string;
|
||||
parentPageId?: string | null;
|
||||
}
|
||||
|
||||
// Match the leading meta block (allow leading whitespace). Capture group 1 is
|
||||
// the JSON text between the markers.
|
||||
const META_RE = /^\s*<!--\s*docmost:meta\s*\n([\s\S]*?)\n-->/;
|
||||
// Match a `docmost:comments` opener. Used globally to scan for the LAST opener
|
||||
// rather than end-anchoring a single regex (which would mis-capture across a
|
||||
// literal opener that appears earlier in the body).
|
||||
const COMMENTS_OPEN_RE = /<!--[ \t]*docmost:comments[ \t]*\r?\n/g;
|
||||
|
||||
/**
|
||||
* Assemble the full self-contained markdown file: meta block, body, and the
|
||||
* comments block. The meta block is always emitted; the comments block is always
|
||||
* emitted too (with `[]` when there are no comments) so the format stays uniform
|
||||
* and parsing stays simple.
|
||||
*/
|
||||
export function serializeDocmostMarkdown(
|
||||
meta: DocmostMdMeta,
|
||||
body: string,
|
||||
comments: any[],
|
||||
): string {
|
||||
const metaJson = JSON.stringify(meta);
|
||||
const commentsJson = JSON.stringify(Array.isArray(comments) ? comments : []);
|
||||
const trimmedBody = (body ?? "").trim();
|
||||
return (
|
||||
`<!-- docmost:meta\n${metaJson}\n-->\n\n` +
|
||||
`${trimmedBody}\n\n` +
|
||||
`<!-- docmost:comments\n${commentsJson}\n-->\n`
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Split a self-contained file back into its parts. Tolerant: if the meta or
|
||||
* comments block is missing (e.g. a hand-written plain-markdown file), the
|
||||
* corresponding value is returned as `null` and the whole input is treated as
|
||||
* the body. This never throws on a MISSING block; only a `JSON.parse` failure
|
||||
* inside a block that IS present is surfaced as a thrown Error with a clear
|
||||
* message. Robust to `\r\n` line endings.
|
||||
*/
|
||||
export function parseDocmostMarkdown(full: string): {
|
||||
meta: DocmostMdMeta | null;
|
||||
body: string;
|
||||
comments: any[] | null;
|
||||
} {
|
||||
// Normalize line endings so the anchored regexes work regardless of CRLF.
|
||||
const normalized = (full ?? "").replace(/\r\n/g, "\n");
|
||||
|
||||
// Extract the leading meta block (start-anchored — already unambiguous).
|
||||
let meta: DocmostMdMeta | null = null;
|
||||
let metaEnd = 0;
|
||||
const metaMatch = normalized.match(META_RE);
|
||||
if (metaMatch) {
|
||||
try {
|
||||
meta = JSON.parse(metaMatch[1]);
|
||||
} catch (e) {
|
||||
throw new Error(
|
||||
`Invalid docmost:meta JSON block: ${
|
||||
e instanceof Error ? e.message : String(e)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
// Body starts right after the matched meta block.
|
||||
metaEnd = (metaMatch.index ?? 0) + metaMatch[0].length;
|
||||
}
|
||||
|
||||
// Find the LAST `<!-- docmost:comments` opener; the real file-level block is
|
||||
// the final one whose closing `-->` ends the document. Any earlier literal
|
||||
// occurrence inside the body (e.g. a re-pasted export) is left in the body.
|
||||
let lastOpenStart = -1;
|
||||
let lastOpenEnd = -1;
|
||||
let m: RegExpExecArray | null;
|
||||
COMMENTS_OPEN_RE.lastIndex = 0;
|
||||
while ((m = COMMENTS_OPEN_RE.exec(normalized)) !== null) {
|
||||
lastOpenStart = m.index;
|
||||
lastOpenEnd = m.index + m[0].length;
|
||||
}
|
||||
|
||||
let comments: any[] | null = null;
|
||||
let bodyEnd = normalized.length;
|
||||
if (lastOpenStart !== -1) {
|
||||
const rest = normalized.slice(lastOpenEnd);
|
||||
const close = rest.match(/\r?\n-->[ \t]*\r?\n?\s*$/); // closer must end the doc
|
||||
if (close) {
|
||||
const jsonText = rest.slice(0, close.index);
|
||||
try {
|
||||
comments = JSON.parse(jsonText);
|
||||
} catch (e) {
|
||||
throw new Error(
|
||||
`Invalid docmost:comments JSON block: ${
|
||||
e instanceof Error ? e.message : String(e)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
bodyEnd = lastOpenStart; // strip from the opener to end of document
|
||||
}
|
||||
}
|
||||
|
||||
const body = normalized.slice(metaEnd, bodyEnd).trim();
|
||||
return { meta, body, comments };
|
||||
}
|
||||
export {
|
||||
serializeDocmostMarkdown,
|
||||
parseDocmostMarkdown,
|
||||
serializeDocmostMarkdownBody,
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
export type { DocmostMdMeta } from "@docmost/prosemirror-markdown";
|
||||
|
||||
@@ -0,0 +1,277 @@
|
||||
/**
|
||||
* Pure, network-free in-page search over a ProseMirror/TipTap document tree.
|
||||
*
|
||||
* `searchInDoc(doc, query, opts)` finds every occurrence of a literal substring
|
||||
* (default) or a regular expression across the page's TEXT CONTAINERS and
|
||||
* reports WHERE each match is — the container's ref (for get_node/patch_node;
|
||||
* see the SearchMatch.nodeId note for the `#<index>` caveat), the top-level
|
||||
* block index, and a short context window around the hit. It never touches the
|
||||
* network, the DB, or the schema mirror; like `comment-anchor.ts` it is
|
||||
* isolated-testable.
|
||||
*
|
||||
* REGEX ENGINE: with `regex:true` the pattern is compiled with RE2 (Google's
|
||||
* linear-time engine), NOT the JS `RegExp`. RE2 has no backtracking, so a
|
||||
* catastrophic pattern (e.g. `(a+)+$`) can never wedge the shared event loop —
|
||||
* it runs in linear time. The trade-off is that RE2 does not support the
|
||||
* backtracking-only features lookaround (`(?=…)`, `(?<=…)`) and backreferences
|
||||
* (`\1`); such a pattern is rejected up front with a clear tool error (see
|
||||
* searchInDoc) rather than being run, which is the desired behaviour — a clear
|
||||
* error the agent can fix beats a server hang.
|
||||
*
|
||||
* WHY plain text (not markdown): each container's inline text is glued into ONE
|
||||
* string via `blockPlainText`, so a match survives inline-mark boundaries
|
||||
* (bold/italic/link splits that fracture a run like "т.е." into several text
|
||||
* nodes) and comment-anchor spans never clutter the haystack.
|
||||
*
|
||||
* The SEARCH UNIT is a text container: a node whose direct children include
|
||||
* text nodes (a paragraph/heading, or the paragraph inside a table cell / list
|
||||
* item). ProseMirror keeps block vs. inline content exclusive, so a container
|
||||
* never nests another container — the walk reaches each cell/item's own text and
|
||||
* the context window is naturally scoped to that specific cell/item, not the
|
||||
* whole top-level block's glued text.
|
||||
*/
|
||||
|
||||
import RE2 from "re2";
|
||||
|
||||
import { blockPlainText } from "./node-ops.js";
|
||||
|
||||
/** An RE2 regex instance (RE2 extends `RegExp`, so it is usable as one). */
|
||||
type Re2Regex = InstanceType<typeof RE2>;
|
||||
|
||||
/** True if `value` is a non-null plain object (and not an array). */
|
||||
function isObject(value: any): value is Record<string, any> {
|
||||
return value != null && typeof value === "object" && !Array.isArray(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* A text container is a node with a `content` array holding at least one text
|
||||
* node (a child with a string `text`). These are the paragraphs/headings whose
|
||||
* glued inline text we search.
|
||||
*/
|
||||
function isTextContainer(node: any): boolean {
|
||||
return (
|
||||
isObject(node) &&
|
||||
Array.isArray(node.content) &&
|
||||
node.content.some((c: any) => isObject(c) && typeof c.text === "string")
|
||||
);
|
||||
}
|
||||
|
||||
/** Options controlling the search engine and result size. */
|
||||
export interface SearchOptions {
|
||||
/** Treat `query` as a RegExp instead of a literal substring (default false). */
|
||||
regex?: boolean;
|
||||
/** Case-sensitive matching (default false). */
|
||||
caseSensitive?: boolean;
|
||||
/** Max matches to RETURN (default 50, clamped to [1, 200]); total is unbounded. */
|
||||
limit?: number;
|
||||
}
|
||||
|
||||
/** One located occurrence. */
|
||||
export interface SearchMatch {
|
||||
/**
|
||||
* The container's ref, for addressing the block with get_node/patch_node: its
|
||||
* `attrs.id` when it has one, otherwise `#<topLevelIndex>` of the nearest
|
||||
* top-level block. Table-cell/list-item paragraphs that carry no id fall back
|
||||
* to the `#<index>` form.
|
||||
*
|
||||
* CAVEAT: the `#<index>` form is accepted by get_node (getNodeByRef resolves
|
||||
* it by top-level index) but NOT by patch_node (replaceNodeById resolves only
|
||||
* by `attrs.id`), so id-less table/cell content can be READ by this ref but
|
||||
* not PATCHED by it.
|
||||
*
|
||||
* To anchor a comment, do NOT pass this ref to create_comment — it has no
|
||||
* nodeId parameter. A top-level comment needs an exact-text `selection` that
|
||||
* occurs once on the page (it fails if the text isn't found), so build a
|
||||
* UNIQUE `selection` from before+match+after and pass THAT as create_comment's
|
||||
* `selection`.
|
||||
*/
|
||||
nodeId: string;
|
||||
/** The top-level block index (as in get_outline). */
|
||||
blockIndex: number;
|
||||
/** The container node's type (paragraph/heading/...). */
|
||||
type: string | undefined;
|
||||
/** ~40 chars of context immediately before the match (from THIS container). */
|
||||
before: string;
|
||||
/** The matched text. */
|
||||
match: string;
|
||||
/** ~40 chars of context immediately after the match (from THIS container). */
|
||||
after: string;
|
||||
}
|
||||
|
||||
/** The search result. `truncated` is true when `total > matches.length`. */
|
||||
export interface SearchResult {
|
||||
total: number;
|
||||
truncated: boolean;
|
||||
matches: SearchMatch[];
|
||||
}
|
||||
|
||||
// Result-size defaults/ceiling.
|
||||
const DEFAULT_LIMIT = 50;
|
||||
const MAX_LIMIT = 200;
|
||||
|
||||
// Context window on each side of a match.
|
||||
const CONTEXT = 40;
|
||||
|
||||
// Cheap sanity cap on the query/pattern length. ReDoS is handled structurally
|
||||
// by the RE2 engine (linear-time, no backtracking — see the module doc), so we
|
||||
// no longer truncate the per-container text: RE2 scans it in linear time and a
|
||||
// cap could silently drop real matches past it. This just rejects an absurdly
|
||||
// long pattern early with a clear error.
|
||||
const MAX_PATTERN_LENGTH = 1000;
|
||||
|
||||
/** Clamp the requested limit into [1, MAX_LIMIT], defaulting when absent. */
|
||||
function resolveLimit(limit: number | undefined): number {
|
||||
const n = typeof limit === "number" && Number.isFinite(limit) ? limit : DEFAULT_LIMIT;
|
||||
return Math.min(MAX_LIMIT, Math.max(1, Math.floor(n)));
|
||||
}
|
||||
|
||||
/**
|
||||
* Yield the [start, length] of every occurrence of the engine in `text`, in
|
||||
* order. A literal engine uses indexOf (case-folded when requested); a regex
|
||||
* engine uses a global RE2 regex (RE2 extends `RegExp`, so `.exec` advances
|
||||
* `lastIndex` exactly like the native engine). Zero-length regex matches (e.g.
|
||||
* `\b`, `a*`) are SKIPPED and lastIndex is advanced, so a pattern that can match
|
||||
* the empty string cannot flood the results or spin forever.
|
||||
*/
|
||||
function* eachMatch(
|
||||
text: string,
|
||||
query: string,
|
||||
re: Re2Regex | null,
|
||||
caseSensitive: boolean,
|
||||
): Generator<[number, number]> {
|
||||
if (re) {
|
||||
re.lastIndex = 0;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(text)) != null) {
|
||||
const len = m[0].length;
|
||||
if (len === 0) {
|
||||
// Empty match: advance past this position and do not record it.
|
||||
re.lastIndex = m.index + 1;
|
||||
continue;
|
||||
}
|
||||
yield [m.index, len];
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
// Literal engine. For case-insensitive search, fold BOTH sides only to locate
|
||||
// the indices; the reported match/context are always sliced from the original
|
||||
// text so the caller gets the real casing (needed to build a unique selection).
|
||||
const haystack = caseSensitive ? text : text.toLowerCase();
|
||||
const needle = caseSensitive ? query : query.toLowerCase();
|
||||
const len = needle.length;
|
||||
let from = 0;
|
||||
for (;;) {
|
||||
const idx = haystack.indexOf(needle, from);
|
||||
if (idx === -1) return;
|
||||
yield [idx, len];
|
||||
from = idx + len;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Search a ProseMirror document for `query` and return `{ total, truncated,
|
||||
* matches }`. `total` counts EVERY occurrence (even beyond the limit) and
|
||||
* `truncated` flags when the returned list was capped — nothing is silently
|
||||
* dropped.
|
||||
*
|
||||
* Throws a clear, model-actionable error (never a generic failure) on: an
|
||||
* empty/whitespace-only query, an over-long pattern, or — with `regex:true` — a
|
||||
* pattern RE2 rejects (invalid syntax, or the unsupported lookaround/
|
||||
* backreference features), so the agent can fix its input.
|
||||
*/
|
||||
export function searchInDoc(
|
||||
doc: any,
|
||||
query: string,
|
||||
opts: SearchOptions = {},
|
||||
): SearchResult {
|
||||
// --- edge-case guards (fail loudly so the agent can correct the call) ---
|
||||
if (typeof query !== "string" || query.trim().length === 0) {
|
||||
throw new Error(
|
||||
"search_in_page: query is empty — pass the text (or regex) to look for.",
|
||||
);
|
||||
}
|
||||
if (query.length > MAX_PATTERN_LENGTH) {
|
||||
throw new Error(
|
||||
`search_in_page: query is too long (${query.length} chars; max ${MAX_PATTERN_LENGTH}). Shorten the search text/pattern.`,
|
||||
);
|
||||
}
|
||||
|
||||
const caseSensitive = opts.caseSensitive === true;
|
||||
const limit = resolveLimit(opts.limit);
|
||||
|
||||
// Compile the pattern up front with RE2 (linear-time, ReDoS-safe) so a bad
|
||||
// pattern is a clean tool error rather than a failure deep in the traversal —
|
||||
// and so a catastrophic-backtracking pattern can never wedge the event loop.
|
||||
// RE2 throws both on syntactically invalid input AND on backtracking-only
|
||||
// features it does not implement (lookaround, backreferences); both map to the
|
||||
// same actionable error so the agent rewrites the pattern.
|
||||
let re: Re2Regex | null = null;
|
||||
if (opts.regex === true) {
|
||||
try {
|
||||
re = new RE2(query, caseSensitive ? "g" : "gi");
|
||||
} catch (e) {
|
||||
throw new Error(
|
||||
`search_in_page: invalid or unsupported regular expression: ${
|
||||
e instanceof Error ? e.message : String(e)
|
||||
} — RE2 does not support lookaround ((?=…)/(?<=…)) or backreferences (\\1); rewrite the pattern without them.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const matches: SearchMatch[] = [];
|
||||
let total = 0;
|
||||
|
||||
const topLevel =
|
||||
isObject(doc) && Array.isArray(doc.content) ? doc.content : [];
|
||||
|
||||
// Descend a top-level block, collecting matches from every text container
|
||||
// within it. blockIndex/topRef stay pinned to the enclosing top-level block.
|
||||
const descend = (node: any, blockIndex: number, topRef: string): void => {
|
||||
if (!isObject(node)) return;
|
||||
|
||||
if (isTextContainer(node)) {
|
||||
// Glue this container's inline text into one string (mark-safe). No length
|
||||
// cap: RE2 scans it in linear time (no ReDoS) and the whole document is
|
||||
// already in memory, so truncating would only risk dropping real matches
|
||||
// in a very long container.
|
||||
const text = blockPlainText(node);
|
||||
|
||||
// The container's own id addresses it verbatim in get_node/patch_node; a
|
||||
// container with no id (e.g. a table-cell paragraph) falls back to the
|
||||
// top-level block's #<index> (readable via get_node, but not patchable —
|
||||
// see the SearchMatch.nodeId note).
|
||||
const id =
|
||||
isObject(node.attrs) && typeof node.attrs.id === "string" && node.attrs.id.length > 0
|
||||
? node.attrs.id
|
||||
: topRef;
|
||||
|
||||
for (const [idx, len] of eachMatch(text, query, re, caseSensitive)) {
|
||||
total++;
|
||||
if (matches.length < limit) {
|
||||
matches.push({
|
||||
nodeId: id,
|
||||
blockIndex,
|
||||
type: node.type,
|
||||
before: text.slice(Math.max(0, idx - CONTEXT), idx),
|
||||
match: text.slice(idx, idx + len),
|
||||
after: text.slice(idx + len, idx + len + CONTEXT),
|
||||
});
|
||||
}
|
||||
}
|
||||
// A text container holds inline content only — no nested containers to
|
||||
// recurse into.
|
||||
return;
|
||||
}
|
||||
|
||||
if (Array.isArray(node.content)) {
|
||||
for (const child of node.content) descend(child, blockIndex, topRef);
|
||||
}
|
||||
};
|
||||
|
||||
for (let i = 0; i < topLevel.length; i++) {
|
||||
descend(topLevel[i], i, `#${i}`);
|
||||
}
|
||||
|
||||
return { total, truncated: total > matches.length, matches };
|
||||
}
|
||||
@@ -13,6 +13,11 @@
|
||||
// diverge on purpose (security guardrails, tuned UX, "Reversible" framing on
|
||||
// some write tools, different limits, hybrid-RRF search, etc.) stay defined
|
||||
// per-layer and are NOT represented here.
|
||||
//
|
||||
// MAINTENANCE RULE: adding, renaming, or removing a spec here (or an inline
|
||||
// registerTool in index.ts) REQUIRES updating SERVER_INSTRUCTIONS in
|
||||
// packages/mcp/src/index.ts — the intent-routing guide MCP clients receive on
|
||||
// initialize. Enforced by test/unit/server-instructions.test.mjs.
|
||||
|
||||
// Loose on purpose — see the comment above. The two zod majors expose different
|
||||
// static type surfaces, so typing this precisely would couple the registry to
|
||||
@@ -105,14 +110,68 @@ export const SHARED_TOOL_SPECS = {
|
||||
}),
|
||||
},
|
||||
|
||||
// --- in-page occurrence search (client-side, over ProseMirror plain text) ---
|
||||
|
||||
searchInPage: {
|
||||
mcpName: 'search_in_page',
|
||||
inAppKey: 'searchInPage',
|
||||
description:
|
||||
'Find every occurrence of a string (or regex) INSIDE one page and get ' +
|
||||
'WHERE each is — instead of pulling blocks one-by-one with get_node. ' +
|
||||
'Searches the plain text of each text block/cell (marks glued, so a match ' +
|
||||
'survives bold/italic/link splits; comment anchors do not interfere). ' +
|
||||
'Returns { total, truncated, matches:[{ nodeId, blockIndex, type, before, ' +
|
||||
'match, after }] }: `nodeId` is the block id (or "#<index>" for ' +
|
||||
'table/cell content) — pass it to get_node/patch_node (the "#<index>" ' +
|
||||
'form resolves with get_node but NOT patch_node, which only accepts a real ' +
|
||||
'block id). To anchor a comment, do NOT pass nodeId to create_comment (it ' +
|
||||
'has no nodeId param); build a UNIQUE text selection from before+match+' +
|
||||
'after and pass it as create_comment\'s `selection`. `blockIndex` is the ' +
|
||||
'get_outline index; `before`/`after` give ~40 chars of context to build ' +
|
||||
'that unique selection. `total` counts all ' +
|
||||
'hits and `truncated` is true when more than `limit` were found (nothing ' +
|
||||
'is silently dropped). Default is a literal, case-INSENSITIVE substring; ' +
|
||||
'set regex:true for an RE2 regular expression (linear-time, ReDoS-safe: ' +
|
||||
'char classes, word boundaries, anchors and quantifiers work; lookaround ' +
|
||||
'(?=…)/(?<=…) and backreferences \\1 are NOT supported) and ' +
|
||||
'caseSensitive:true to match case. Ideal for systematic ' +
|
||||
'editorial sweeps (unquoted "ё", straight quotes, "т.е.", stray units). An ' +
|
||||
'invalid regex or an empty query returns a clear error to fix.',
|
||||
buildShape: (z) => ({
|
||||
pageId: z.string().min(1).describe('ID of the page to search'),
|
||||
query: z
|
||||
.string()
|
||||
.min(1)
|
||||
.describe('The text to find (a literal substring, or a regex when regex:true)'),
|
||||
regex: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe(
|
||||
'Treat query as an RE2 regular expression — linear-time, ReDoS-safe; ' +
|
||||
'no lookaround or backreferences (default false).',
|
||||
),
|
||||
caseSensitive: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe('Case-sensitive matching (default false).'),
|
||||
limit: z
|
||||
.number()
|
||||
.int()
|
||||
.min(1)
|
||||
.max(200)
|
||||
.optional()
|
||||
.describe('Max matches to RETURN (default 50, max 200); total is always reported.'),
|
||||
}),
|
||||
},
|
||||
|
||||
// --- node delete ---
|
||||
|
||||
deleteNode: {
|
||||
mcpName: 'delete_node',
|
||||
inAppKey: 'deleteNode',
|
||||
description:
|
||||
'Remove a single block by its attrs.id (from the page-JSON view) WITHOUT ' +
|
||||
'resending the whole document.',
|
||||
'Remove a single block by its attrs.id (from the page outline or ' +
|
||||
'page-JSON view) WITHOUT resending the whole document.',
|
||||
buildShape: (z) => ({
|
||||
pageId: z.string().min(1),
|
||||
nodeId: z.string().min(1),
|
||||
@@ -134,7 +193,8 @@ export const SHARED_TOOL_SPECS = {
|
||||
description:
|
||||
'Replace a single content block identified by its attrs.id with a new ' +
|
||||
'ProseMirror node, WITHOUT resending the whole document; the replacement ' +
|
||||
'keeps the same node id. Get the block id from the page-JSON view, then ' +
|
||||
'keeps the same node id. Get the block id from the page outline (cheap) ' +
|
||||
'or the page-JSON view, then ' +
|
||||
'pass a ProseMirror node to put in its place. Example node: a paragraph ' +
|
||||
'{"type":"paragraph","content":[{"type":"text","text":"Hello"}]} or a ' +
|
||||
'heading {"type":"heading","attrs":{"level":2},"content":' +
|
||||
@@ -169,7 +229,8 @@ export const SHARED_TOOL_SPECS = {
|
||||
'Insert a block before/after another block (by attrs.id or anchor text) ' +
|
||||
'or append it at the end (top level). For before/after you MUST provide ' +
|
||||
'EXACTLY ONE of anchorNodeId or anchorText. Get anchor block ids from the ' +
|
||||
'page-JSON view. Avoids resending the whole document. Can also insert ' +
|
||||
'page outline or the page-JSON view. Avoids resending the whole document. ' +
|
||||
'Can also insert ' +
|
||||
'table structure: to add a tableRow, pass a tableRow node with position ' +
|
||||
'before/after and anchor INSIDE the target table — anchorNodeId of any ' +
|
||||
'block/cell in it, or anchorText matching the table; to add a ' +
|
||||
|
||||
@@ -462,7 +462,7 @@ async function main() {
|
||||
check("create_comment: markdown round-trip", c1.data.content.includes("**комментарий**"), c1.data.content);
|
||||
const reply = await client.createComment(pageId, "Ответ на комментарий.", "page", undefined, c1.data.id);
|
||||
check("create_comment: reply has parent", reply.data.parentCommentId === c1.data.id);
|
||||
const list = await client.listComments(pageId);
|
||||
const list = (await client.listComments(pageId)).items;
|
||||
check("list_comments: both visible", list.length === 2, `count=${list.length}`);
|
||||
await client.updateComment(c1.data.id, "Обновлённый текст комментария.");
|
||||
const got = await client.getComment(c1.data.id);
|
||||
@@ -472,17 +472,19 @@ async function main() {
|
||||
// resolve_comment: close the top-level thread, verify resolvedAt surfaces, then reopen
|
||||
const resolvedRes = await client.resolveComment(c1.data.id, true);
|
||||
check("resolve_comment: marks resolved", resolvedRes.success === true && resolvedRes.resolved === true);
|
||||
const listResolved = await client.listComments(pageId);
|
||||
// c1 is now resolved; the default feed hides resolved threads, so pass
|
||||
// includeResolved:true to still see it and assert its resolvedAt (#328).
|
||||
const listResolved = (await client.listComments(pageId, true)).items;
|
||||
const c1Resolved = listResolved.find((c) => c.id === c1.data.id);
|
||||
check("resolve_comment: resolvedAt set in list", !!c1Resolved?.resolvedAt, `resolvedAt=${c1Resolved?.resolvedAt}`);
|
||||
const reopenedRes = await client.resolveComment(c1.data.id, false);
|
||||
check("resolve_comment: reopen succeeds", reopenedRes.resolved === false);
|
||||
const listReopened = await client.listComments(pageId);
|
||||
const listReopened = (await client.listComments(pageId)).items;
|
||||
const c1Reopened = listReopened.find((c) => c.id === c1.data.id);
|
||||
check("resolve_comment: resolvedAt cleared on reopen", !c1Reopened?.resolvedAt, `resolvedAt=${c1Reopened?.resolvedAt}`);
|
||||
await client.deleteComment(reply.data.id);
|
||||
await client.deleteComment(c1.data.id);
|
||||
const listAfter = await client.listComments(pageId);
|
||||
const listAfter = (await client.listComments(pageId)).items;
|
||||
check("delete_comment: comments removed", listAfter.length === 0, `count=${listAfter.length}`);
|
||||
} finally {
|
||||
if (pageId) {
|
||||
|
||||
@@ -0,0 +1,160 @@
|
||||
// gitmost #328 Channel 2: DocmostClient.listComments hides RESOLVED THREADS
|
||||
// wholesale by default (a resolved top-level comment AND every reply under it),
|
||||
// returning `{ items, resolvedThreadsHidden }`. `includeResolved: true` returns
|
||||
// the full feed. These tests stand a local http.createServer in for Docmost and
|
||||
// mock the /auth/login + /comments (paginated) routes.
|
||||
import { test, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import http from "node:http";
|
||||
import { DocmostClient } from "../../build/client.js";
|
||||
|
||||
function readBody(req) {
|
||||
return new Promise((resolve) => {
|
||||
let raw = "";
|
||||
req.on("data", (c) => (raw += c));
|
||||
req.on("end", () => resolve(raw));
|
||||
});
|
||||
}
|
||||
|
||||
function startServer(handler) {
|
||||
return new Promise((resolve) => {
|
||||
const server = http.createServer(handler);
|
||||
server.listen(0, "127.0.0.1", () => {
|
||||
const { port } = server.address();
|
||||
resolve({ server, baseURL: `http://127.0.0.1:${port}/api` });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function closeServer(server) {
|
||||
return new Promise((resolve) => server.close(resolve));
|
||||
}
|
||||
|
||||
function sendJson(res, status, obj, extraHeaders = {}) {
|
||||
res.writeHead(status, { "Content-Type": "application/json", ...extraHeaders });
|
||||
res.end(JSON.stringify(obj));
|
||||
}
|
||||
|
||||
const openServers = [];
|
||||
async function spawn(handler) {
|
||||
const { server, baseURL } = await startServer(handler);
|
||||
openServers.push(server);
|
||||
return { server, baseURL };
|
||||
}
|
||||
|
||||
after(async () => {
|
||||
await Promise.all(openServers.map((s) => closeServer(s)));
|
||||
});
|
||||
|
||||
// A minimal ProseMirror comment body (a paragraph of text).
|
||||
const body = (t) => ({
|
||||
type: "doc",
|
||||
content: [{ type: "paragraph", content: [{ type: "text", text: t }] }],
|
||||
});
|
||||
|
||||
// Feed: an ACTIVE thread whose REPLY is resolved (root active, reply resolved —
|
||||
// the thread must STAY, because a thread is gated only by its ROOT's resolvedAt)
|
||||
// and a RESOLVED thread (root + reply).
|
||||
const FEED = [
|
||||
{
|
||||
id: "a",
|
||||
pageId: "page-1",
|
||||
parentCommentId: null,
|
||||
resolvedAt: null,
|
||||
createdAt: "2026-01-01T00:00:00.000Z",
|
||||
creatorId: "u1",
|
||||
content: body("active root"),
|
||||
},
|
||||
{
|
||||
id: "a1",
|
||||
pageId: "page-1",
|
||||
parentCommentId: "a",
|
||||
// A RESOLVED reply under an ACTIVE root: the thread is NOT hidden (only a
|
||||
// resolved ROOT hides a thread), so this reply survives the default filter.
|
||||
resolvedAt: "2026-02-15T00:00:00.000Z",
|
||||
createdAt: "2026-01-01T01:00:00.000Z",
|
||||
creatorId: "u1",
|
||||
content: body("resolved reply of an active thread"),
|
||||
},
|
||||
{
|
||||
id: "r",
|
||||
pageId: "page-1",
|
||||
parentCommentId: null,
|
||||
resolvedAt: "2026-02-01T00:00:00.000Z",
|
||||
createdAt: "2026-01-02T00:00:00.000Z",
|
||||
creatorId: "u1",
|
||||
content: body("resolved root"),
|
||||
},
|
||||
{
|
||||
id: "r1",
|
||||
pageId: "page-1",
|
||||
parentCommentId: "r",
|
||||
resolvedAt: null,
|
||||
createdAt: "2026-01-02T01:00:00.000Z",
|
||||
creatorId: "u1",
|
||||
content: body("resolved reply"),
|
||||
},
|
||||
];
|
||||
|
||||
function commentsServer() {
|
||||
return spawn(async (req, res) => {
|
||||
await readBody(req);
|
||||
if (req.url === "/api/auth/login") {
|
||||
sendJson(res, 200, { success: true }, {
|
||||
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (req.url === "/api/comments") {
|
||||
// Single page, no cursor.
|
||||
sendJson(res, 200, { data: { items: FEED, meta: { nextCursor: null } } });
|
||||
return;
|
||||
}
|
||||
sendJson(res, 404, { message: "not found" });
|
||||
});
|
||||
}
|
||||
|
||||
test("default hides the resolved thread (root + its reply) and counts it", async () => {
|
||||
const { baseURL } = await commentsServer();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
const result = await client.listComments("page-1");
|
||||
assert.equal(Array.isArray(result.items), true, "returns { items, ... }");
|
||||
const ids = result.items.map((c) => c.id).sort();
|
||||
assert.deepEqual(ids, ["a", "a1"], "only the active thread remains");
|
||||
assert.equal(result.resolvedThreadsHidden, 1, "one resolved thread hidden");
|
||||
});
|
||||
|
||||
test("includeResolved:true returns EVERYTHING with zero hidden", async () => {
|
||||
const { baseURL } = await commentsServer();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
const result = await client.listComments("page-1", true);
|
||||
const ids = result.items.map((c) => c.id).sort();
|
||||
assert.deepEqual(ids, ["a", "a1", "r", "r1"], "all four comments returned");
|
||||
assert.equal(result.resolvedThreadsHidden, 0, "nothing hidden with the flag");
|
||||
});
|
||||
|
||||
test("the reply of a resolved thread is hidden with the thread", async () => {
|
||||
const { baseURL } = await commentsServer();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
const result = await client.listComments("page-1");
|
||||
const ids = result.items.map((c) => c.id);
|
||||
assert.equal(ids.includes("r1"), false, "the resolved thread's reply is gone");
|
||||
assert.equal(ids.includes("r"), false, "the resolved root is gone");
|
||||
});
|
||||
|
||||
test("an ACTIVE thread whose REPLY is resolved is NOT hidden", async () => {
|
||||
// A thread is gated only by its ROOT's resolvedAt. `a1` is a resolved reply
|
||||
// under the active root `a`, so both must survive the default filter and the
|
||||
// thread must not be counted as hidden.
|
||||
const { baseURL } = await commentsServer();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
const result = await client.listComments("page-1");
|
||||
const ids = result.items.map((c) => c.id).sort();
|
||||
assert.equal(ids.includes("a"), true, "active root stays");
|
||||
assert.equal(ids.includes("a1"), true, "its resolved reply stays with the thread");
|
||||
assert.equal(result.resolvedThreadsHidden, 1, "only the resolved-root thread is hidden");
|
||||
});
|
||||
@@ -45,6 +45,7 @@ const HOST_CONTRACT_METHODS = [
|
||||
"getOutline",
|
||||
"getPageJson",
|
||||
"getNode",
|
||||
"searchInPage",
|
||||
"getTable",
|
||||
"listComments",
|
||||
"getComment",
|
||||
|
||||
@@ -127,36 +127,54 @@ test("markdownToProseMirror: an aligned GFM table maps header alignment", async
|
||||
});
|
||||
|
||||
// Comment-body data-loss guard (#228 review #4): markdownToProseMirror is reused
|
||||
// for COMMENT bodies (createComment/updateComment), so it must NOT canonicalize —
|
||||
// a comment may legitimately carry a standalone footnote definition with no
|
||||
// matching reference, and canonicalization would drop the whole list (the text
|
||||
// would vanish). The page-write variant DOES canonicalize.
|
||||
test("markdownToProseMirror (comment path) PRESERVES a reference-less footnote definition", async () => {
|
||||
// for COMMENT bodies (createComment/updateComment), so it must NOT canonicalize.
|
||||
// Under the #293 canon, footnotes are INLINE (`^[body]`), so a comment can no
|
||||
// longer carry a reference-less definition to be dropped — but the comment path
|
||||
// must still (a) leave a legacy reference-style `[^id]:` line as harmless literal
|
||||
// TEXT (never silently deleted) and (b) preserve an inline footnote it does
|
||||
// contain (no canonicalization stripping it). The page-write variant canonicalizes.
|
||||
test("markdownToProseMirror (comment path) keeps a legacy `[^id]:` line as literal text", async () => {
|
||||
// A reference-style `[^1]:` line is not canonical footnote syntax anymore, so it
|
||||
// is not parsed into a footnote node — but its TEXT must survive verbatim (no
|
||||
// data loss on the comment write path).
|
||||
const md = "A comment.\n\n[^1]: a standalone footnote definition";
|
||||
const doc = await markdownToProseMirror(md);
|
||||
const defs = findAll(doc, "footnoteDefinition");
|
||||
assert.equal(defs.length, 1, "the footnote definition must be preserved");
|
||||
assert.equal(
|
||||
findAll(doc, "footnoteDefinition").length,
|
||||
0,
|
||||
"reference-style line is not a footnote node",
|
||||
);
|
||||
assert.match(
|
||||
JSON.stringify(doc),
|
||||
/a standalone footnote definition/,
|
||||
"the definition text must survive the comment write path",
|
||||
"the text must survive the comment write path",
|
||||
);
|
||||
});
|
||||
|
||||
test("markdownToProseMirrorCanonical (page path) DROPS a reference-less footnote definition", async () => {
|
||||
// Same input through the PAGE variant: with no reference, the canonical doc has
|
||||
// no footnotesList (this is the page-side behavior the comment path must avoid).
|
||||
const md = "A page.\n\n[^1]: a standalone footnote definition";
|
||||
const doc = await markdownToProseMirrorCanonical(md);
|
||||
assert.equal(findAll(doc, "footnotesList").length, 0);
|
||||
assert.equal(findAll(doc, "footnoteDefinition").length, 0);
|
||||
test("markdownToProseMirror (comment path) PRESERVES an inline footnote (no canonicalization)", async () => {
|
||||
// An inline `^[body]` footnote in a comment imports to a real footnote node and
|
||||
// is NOT dropped: the comment path must never canonicalize away content.
|
||||
const md = "A comment.\n\n^[an inline footnote]";
|
||||
const doc = await markdownToProseMirror(md);
|
||||
assert.equal(findAll(doc, "footnoteDefinition").length, 1);
|
||||
assert.equal(findAll(doc, "footnotesList").length, 1);
|
||||
assert.match(JSON.stringify(doc), /an inline footnote/);
|
||||
});
|
||||
|
||||
test("markdownToProseMirrorCanonical still canonicalizes a real page footnote (order)", async () => {
|
||||
// Page path must STILL canonicalize: refs b,a -> definitions reorder to b,a.
|
||||
const md = "See[^b] then[^a].\n\n[^a]: alpha\n[^b]: bravo";
|
||||
test("markdownToProseMirrorCanonical (page path) yields a single reference-ordered list", async () => {
|
||||
// Page path produces the canonical footnote topology: one trailing
|
||||
// `footnotesList`, definitions in FIRST-REFERENCE order, ids assigned
|
||||
// sequentially. Inline `^[body]` footnotes carry the body at the reference
|
||||
// point, so the bottom list is inherently reference-ordered.
|
||||
const md = "See^[bravo] then^[alpha].";
|
||||
const doc = await markdownToProseMirrorCanonical(md);
|
||||
const defs = findAll(doc, "footnoteDefinition").map((d) => d.attrs.id);
|
||||
assert.deepEqual(defs, ["b", "a"]);
|
||||
const defs = findAll(doc, "footnoteDefinition");
|
||||
assert.deepEqual(
|
||||
defs.map((d) => d.attrs.id),
|
||||
["fn-1", "fn-2"],
|
||||
);
|
||||
assert.equal(findAll(doc, "footnotesList").length, 1);
|
||||
// Bodies stay in reference order (bravo referenced before alpha).
|
||||
assert.match(JSON.stringify(defs[0]), /bravo/);
|
||||
assert.match(JSON.stringify(defs[1]), /alpha/);
|
||||
});
|
||||
|
||||
@@ -210,13 +210,17 @@ test("drawio round-trips through export and import", () => {
|
||||
],
|
||||
};
|
||||
|
||||
// #293 canon #8: the media family (image/video/audio/drawio/excalidraw)
|
||||
// serializes to the markdown image form `` plus a trailing
|
||||
// discriminator comment `<!--drawio {json}-->` carrying the non-src attrs.
|
||||
const body = convertProseMirrorToMarkdown(doc);
|
||||
assert.match(body, /data-type="drawio"/);
|
||||
assert.match(body, /data-src="https:\/\/example\/diagram\.xml"/);
|
||||
assert.match(body, /!\[\]\(https:\/\/example\/diagram\.xml\)/);
|
||||
assert.match(body, /<!--drawio \{"attachmentId":"att-7"\}-->/);
|
||||
|
||||
return markdownToProseMirror(body).then((rebuilt) => {
|
||||
const diagram = find(rebuilt, "drawio");
|
||||
assert.ok(diagram, "expected a drawio node after import");
|
||||
assert.equal(diagram.attrs.src, "https://example/diagram.xml");
|
||||
assert.equal(diagram.attrs.attachmentId, "att-7");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -253,22 +253,23 @@ test("insertInlineFootnote: anchor in body BEFORE a nested list still inserts",
|
||||
assert.equal(findAll(r.doc, "footnotesList").length, 1);
|
||||
});
|
||||
|
||||
test("markdown import (page path): out-of-order definitions render as a reference-ordered list", async () => {
|
||||
// References appear b, a, c in the body; definitions are written in a, b, c
|
||||
// order (the import order). The PAGE import path (markdownToProseMirrorCanonical)
|
||||
// canonicalizes so the bottom list follows REFERENCE order — numbers read 1, 2,
|
||||
// 3 down the list. (The non-canonicalizing markdownToProseMirror, used for
|
||||
// comment bodies, would keep the import order; see collaboration.test.mjs.)
|
||||
const md = [
|
||||
"See[^b] then[^a] then[^c].",
|
||||
"",
|
||||
"[^a]: alpha",
|
||||
"[^b]: bravo",
|
||||
"[^c]: charlie",
|
||||
].join("\n");
|
||||
test("markdown import (page path): inline footnotes render as a reference-ordered list", async () => {
|
||||
// Inline `^[body]` footnotes carry their body at the reference point, so the
|
||||
// PAGE import path (markdownToProseMirrorCanonical) materializes the bottom
|
||||
// list in REFERENCE order — numbers read 1, 2, 3 down the list — with ids
|
||||
// assigned sequentially (fn-1, fn-2, fn-3).
|
||||
const md = "See^[bravo] then^[alpha] then^[charlie].";
|
||||
const json = await markdownToProseMirrorCanonical(md);
|
||||
assert.deepEqual(defIds(json), ["b", "a", "c"]);
|
||||
assert.deepEqual(defIds(json), ["fn-1", "fn-2", "fn-3"]);
|
||||
assert.equal(findAll(json, "footnotesList").length, 1);
|
||||
// Bodies materialize in reference order (bravo, alpha, charlie).
|
||||
const defsJson = JSON.stringify(findAll(json, "footnoteDefinition"));
|
||||
assert.ok(
|
||||
defsJson.indexOf("bravo") <
|
||||
defsJson.indexOf("alpha") &&
|
||||
defsJson.indexOf("alpha") < defsJson.indexOf("charlie"),
|
||||
"definitions follow reference order",
|
||||
);
|
||||
});
|
||||
|
||||
test("generateFootnoteId: valid uuidv7 shape (version 7, variant 8..b) and unique", () => {
|
||||
|
||||
@@ -49,15 +49,19 @@ const footnoteDoc = {
|
||||
],
|
||||
};
|
||||
|
||||
test("JSON -> Markdown emits pandoc footnote syntax", () => {
|
||||
test("JSON -> Markdown emits canonical inline footnote syntax (#293 canon #2)", () => {
|
||||
// Canonical markdown form is Pandoc/Obsidian INLINE footnotes: the note body is
|
||||
// written at the reference point as `^[body]`. There is NO `[^id]` reference
|
||||
// marker and NO trailing `[^id]: …` definition list; the schema id never
|
||||
// reaches markdown.
|
||||
const md = convertProseMirrorToMarkdown(footnoteDoc);
|
||||
assert.match(md, /\[\^fn1\]/);
|
||||
assert.match(md, /\[\^fn2\]/);
|
||||
assert.match(md, /\[\^fn1\]: First note\./);
|
||||
assert.match(md, /\[\^fn2\]: Second note\./);
|
||||
assert.match(md, /\^\[First note\.\]/);
|
||||
assert.match(md, /\^\[Second note\.\]/);
|
||||
assert.doesNotMatch(md, /\[\^/); // no reference-style markers
|
||||
assert.doesNotMatch(md, /^\[\^.+\]:/m); // no bottom definition lines
|
||||
});
|
||||
|
||||
test("Markdown -> JSON rebuilds footnote nodes", async () => {
|
||||
test("Markdown -> JSON rebuilds footnote nodes with sequential fn-N ids", async () => {
|
||||
const md = convertProseMirrorToMarkdown(footnoteDoc);
|
||||
const json = await markdownToProseMirror(md);
|
||||
|
||||
@@ -65,42 +69,39 @@ test("Markdown -> JSON rebuilds footnote nodes", async () => {
|
||||
const list = findAll(json, "footnotesList");
|
||||
const defs = findAll(json, "footnoteDefinition");
|
||||
|
||||
// Structure is preserved; ids are (re)assigned sequentially in first-reference
|
||||
// order by the importer (fn-1, fn-2, …) — the concrete id is never carried in
|
||||
// markdown, so it is derived on import.
|
||||
assert.equal(refs.length, 2);
|
||||
assert.deepEqual(
|
||||
refs.map((r) => r.attrs.id),
|
||||
["fn1", "fn2"],
|
||||
["fn-1", "fn-2"],
|
||||
);
|
||||
assert.equal(list.length, 1);
|
||||
assert.equal(defs.length, 2);
|
||||
assert.deepEqual(
|
||||
defs.map((d) => d.attrs.id),
|
||||
["fn1", "fn2"],
|
||||
["fn-1", "fn-2"],
|
||||
);
|
||||
});
|
||||
|
||||
test("JSON -> MD -> JSON preserves footnote ids and text", async () => {
|
||||
test("JSON -> MD -> JSON is byte-stable and preserves footnote body text", async () => {
|
||||
const md = convertProseMirrorToMarkdown(footnoteDoc);
|
||||
const json = await markdownToProseMirror(md);
|
||||
const md2 = convertProseMirrorToMarkdown(json);
|
||||
|
||||
// The second markdown serialization carries the same markers + definitions.
|
||||
assert.match(md2, /\[\^fn1\]/);
|
||||
assert.match(md2, /\[\^fn2\]/);
|
||||
assert.match(md2, /\[\^fn1\]: First note\./);
|
||||
assert.match(md2, /\[\^fn2\]: Second note\./);
|
||||
// The round trip is byte-stable (ids are not written to markdown, so the
|
||||
// concrete import id cannot perturb the output) and the bodies survive.
|
||||
assert.equal(md2, md);
|
||||
assert.match(md2, /\^\[First note\.\]/);
|
||||
assert.match(md2, /\^\[Second note\.\]/);
|
||||
});
|
||||
|
||||
test("repeated references REUSE one footnote; duplicate definitions are first-wins (#166)", async () => {
|
||||
// Reuse semantics: many `[^d]` references + several `[^d]:` definitions import
|
||||
// as ONE footnote — the references all keep id "d" (reuse), and only the FIRST
|
||||
// definition is kept (first-wins). Deterministic and stable across re-imports.
|
||||
const md = [
|
||||
"See[^d] one[^d] two[^d].",
|
||||
"",
|
||||
"[^d]: first",
|
||||
"[^d]: second",
|
||||
"[^d]: third",
|
||||
].join("\n");
|
||||
test("identical footnote bodies MERGE to one shared definition (#293 canon #2)", async () => {
|
||||
// Two references whose bodies are byte-identical import as ONE definition
|
||||
// shared by both references (dedup on the exact body text). Two DIFFERENT
|
||||
// bodies stay distinct. Deterministic and stable across re-imports.
|
||||
const md = "See^[same] and^[same], but^[other].";
|
||||
|
||||
const idsOf = async () => {
|
||||
const json = await markdownToProseMirror(md);
|
||||
@@ -120,11 +121,11 @@ test("repeated references REUSE one footnote; duplicate definitions are first-wi
|
||||
|
||||
// Stable across runs.
|
||||
assert.deepEqual(a, b);
|
||||
// Reuse: all three reference markers stay "d".
|
||||
assert.deepEqual(a.refs, ["d", "d", "d"]);
|
||||
// First-wins: a single definition "d" with the FIRST text.
|
||||
assert.deepEqual(a.defIds, ["d"]);
|
||||
assert.equal(a.defText, "first");
|
||||
// Merge: the two "same" references share fn-1; the "other" reference is fn-2.
|
||||
assert.deepEqual(a.refs, ["fn-1", "fn-1", "fn-2"]);
|
||||
// One definition per unique body, in first-reference order.
|
||||
assert.deepEqual(a.defIds, ["fn-1", "fn-2"]);
|
||||
assert.equal(a.defText, "same|other");
|
||||
});
|
||||
|
||||
test("a [^id]: line inside a fenced code block is NOT treated as a definition", async () => {
|
||||
|
||||
@@ -70,7 +70,7 @@ test("hardBreak -> trailing two-spaces+newline", () => {
|
||||
assert.equal(convertProseMirrorToMarkdown(input), "line1 \nline2");
|
||||
});
|
||||
|
||||
test("table cell with two block children joined by a space (and a pipe escaped)", () => {
|
||||
test("table cell with two block children falls back to a raw HTML table", () => {
|
||||
const input = doc({
|
||||
type: "table",
|
||||
content: [
|
||||
@@ -86,11 +86,12 @@ test("table cell with two block children joined by a space (and a pipe escaped)"
|
||||
],
|
||||
});
|
||||
|
||||
// Single-column header row + separator. The cell joins its two paragraphs
|
||||
// with a space ("a|b c") then escapes the pipe -> "a\|b c".
|
||||
// A pipe-table cell cannot represent two block children, so the canonical
|
||||
// converter emits the whole table as raw HTML (lossless) rather than lossily
|
||||
// flattening the paragraphs into one cell.
|
||||
assert.equal(
|
||||
convertProseMirrorToMarkdown(input),
|
||||
"| a\\|b c |\n| --- |",
|
||||
"<table><tbody><tr><td><p>a|b</p><p>c</p></td></tr></tbody></table>",
|
||||
);
|
||||
});
|
||||
|
||||
@@ -108,20 +109,20 @@ test("code block trailing newline trimmed", () => {
|
||||
);
|
||||
});
|
||||
|
||||
test("textAlign value: delimiting double-quote escaped (attribute-safe, idempotent; < > left literal/inert)", () => {
|
||||
test("textAlign is carried in a trailing attached-comment directive (JSON-encoded, safe)", () => {
|
||||
const input = doc({
|
||||
type: "paragraph",
|
||||
attrs: { textAlign: 'right"><b' },
|
||||
content: [text("body")],
|
||||
});
|
||||
|
||||
// Attribute values escape only & and " so the value cannot break out of the
|
||||
// quoted attribute. < and > are left literal: parse5/jsdom does NOT decode
|
||||
// </> inside attribute values, so escaping them would corrupt the value
|
||||
// and accumulate on every round-trip. The literal < > are inert inside quotes.
|
||||
// #293 canon #9: paragraph textAlign has no native markdown syntax, so it is
|
||||
// attached as a trailing `<!--attrs {json}-->` comment on the block. The value
|
||||
// is JSON-encoded, so a hostile value (`"`, `<`, `>`) is carried verbatim and
|
||||
// inert — it cannot break out of the comment.
|
||||
assert.equal(
|
||||
convertProseMirrorToMarkdown(input),
|
||||
'<div align="right"><b">body</div>',
|
||||
'body <!--attrs {"textAlign":"right\\"><b"}-->',
|
||||
);
|
||||
});
|
||||
|
||||
@@ -150,10 +151,10 @@ test("empty task item still emits its marker", () => {
|
||||
assert.equal(convertProseMirrorToMarkdown(input), "- [ ]\n- [x]");
|
||||
});
|
||||
|
||||
// Image captions (issue #221). An image WITHOUT a caption stays the lossy-free
|
||||
// ``; WITH a caption it is emitted as a raw <img data-caption>
|
||||
// wrapped in a block <div> (symmetric to video) so the round-trip md -> html ->
|
||||
// json restores the caption via the image extension's parseHTML.
|
||||
// Image captions (issue #221 / #293 canon #8). An image WITHOUT a caption stays
|
||||
// the plain ``; WITH a caption (or any other non-src attr) the extra
|
||||
// attrs ride in a trailing `<!--img {json}-->` discriminator comment on the
|
||||
// markdown image form, so the round-trip md -> json restores them.
|
||||
test("image without a caption emits plain ", () => {
|
||||
const input = doc({
|
||||
type: "image",
|
||||
@@ -162,24 +163,24 @@ test("image without a caption emits plain ", () => {
|
||||
assert.equal(convertProseMirrorToMarkdown(input), "");
|
||||
});
|
||||
|
||||
test("image with a caption emits a raw <img data-caption> in a block div", () => {
|
||||
test("image with a caption emits  plus an <!--img--> directive", () => {
|
||||
const input = doc({
|
||||
type: "image",
|
||||
attrs: { src: "/files/a.png", alt: "cat", caption: "A grey cat" },
|
||||
});
|
||||
assert.equal(
|
||||
convertProseMirrorToMarkdown(input),
|
||||
'<div><img src="/files/a.png" alt="cat" data-caption="A grey cat"></div>',
|
||||
' <!--img {"caption":"A grey cat"}-->',
|
||||
);
|
||||
});
|
||||
|
||||
test("image caption escapes & and \" in the data-caption attribute", () => {
|
||||
test("image caption is JSON-encoded in the <!--img--> directive (& and \" safe)", () => {
|
||||
const input = doc({
|
||||
type: "image",
|
||||
attrs: { src: "/files/a.png", caption: 'Tom & "Jerry"' },
|
||||
});
|
||||
assert.equal(
|
||||
convertProseMirrorToMarkdown(input),
|
||||
'<div><img src="/files/a.png" data-caption="Tom & "Jerry""></div>',
|
||||
' <!--img {"caption":"Tom & \\"Jerry\\""}-->',
|
||||
);
|
||||
});
|
||||
|
||||
@@ -55,8 +55,10 @@ test("round-trip: drawio diagram survives with src, title, dimensions, align, at
|
||||
},
|
||||
"drawio",
|
||||
);
|
||||
// The converter must emit the schema-matching div[data-type="drawio"].
|
||||
assert.match(md, /data-type="drawio"/);
|
||||
// #293 canon #8: the media family serializes to the markdown image form plus a
|
||||
// trailing discriminator comment carrying the non-src attrs.
|
||||
assert.match(md, /^!\[\]\(\/api\/files\/d\.drawio\)/);
|
||||
assert.match(md, /<!--drawio \{.*"attachmentId":"dz1".*\}-->/);
|
||||
assert.equal(found.length, 1, "drawio node must survive the round-trip");
|
||||
const a = found[0].attrs;
|
||||
assert.equal(a.src, "/api/files/d.drawio");
|
||||
@@ -123,13 +125,19 @@ test("round-trip: pdf preserves width/height (standard attrs) plus name", async
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Escaping: a src containing a double quote must survive the attribute-quoted
|
||||
// HTML emission (escapeAttr) and re-parse to the exact original value, with no
|
||||
// node loss and no HTML injection.
|
||||
// Escaping: a src containing a double quote must survive the markdown image form
|
||||
// with no node loss and no injection. In the `` link the URL is
|
||||
// normalized (a raw `"` percent-encodes to `%22`) on import — a semantically
|
||||
// equivalent, IDEMPOTENT normalization (it does not drift on further round
|
||||
// trips), not data loss.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("round-trip: a src containing a double quote is escaped and recovered intact", async () => {
|
||||
test("round-trip: a src containing a double quote is normalized (idempotent) and survives", async () => {
|
||||
const tricky = 'https://e.com/x?a="b"&c=1';
|
||||
const normalized = "https://e.com/x?a=%22b%22&c=1";
|
||||
const { found } = await roundtrip({ type: "youtube", attrs: { src: tricky } }, "youtube");
|
||||
assert.equal(found.length, 1, "node must survive a quote-bearing src");
|
||||
assert.equal(found[0].attrs.src, tricky, "the exact src is recovered");
|
||||
assert.equal(found[0].attrs.src, normalized, "the quote is percent-encoded in the URL");
|
||||
// Idempotent: a second round trip from the normalized node is byte-stable.
|
||||
const again = await roundtrip({ type: "youtube", attrs: { src: normalized } }, "youtube");
|
||||
assert.equal(again.found[0].attrs.src, normalized);
|
||||
});
|
||||
|
||||
@@ -25,19 +25,15 @@ const findAll = (node, type, acc = []) => {
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// DATA-LOSS: atom block nodes with no converter case serialize to "" and the
|
||||
// whole block disappears from markdown export.
|
||||
//
|
||||
// markdown-converter.ts has a `default` branch (~line 601) that renders a node
|
||||
// as `nodeContent.map(processNode).join("")`. For a leaf/atom node (no
|
||||
// content) that yields "" — so the node (and ALL its attributes) is dropped.
|
||||
// `htmlEmbed` and `pageBreak` are both block atoms in docmost-schema.ts with no
|
||||
// case in the converter, so they vanish on markdown export.
|
||||
//
|
||||
// These tests assert the CURRENT (buggy) behavior and name it, so that when a
|
||||
// converter case is added the failing assertion flags the test for an update.
|
||||
// #293 canon: atom block nodes with no NATIVE markdown syntax are preserved via
|
||||
// dedicated converter forms (they used to serialize to "" and vanish — the old
|
||||
// mcp converter's data-loss gap, now fixed by consuming the shared package):
|
||||
// - htmlEmbed -> a raw `<div data-type="htmlEmbed" data-source=… data-height=…>`
|
||||
// block (source base64-encoded so arbitrary HTML is inert);
|
||||
// - pageBreak -> a standalone `<!--pagebreak-->` machinery comment (#5).
|
||||
// Both survive markdown export AND a full PM -> markdown -> PM round-trip.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("DATA-LOSS: an htmlEmbed block is silently dropped from markdown export (no converter case)", () => {
|
||||
test("htmlEmbed block survives markdown export (source + height preserved)", () => {
|
||||
const input = doc(
|
||||
para(text("before")),
|
||||
{ type: "htmlEmbed", attrs: { source: "<b>hi</b>", height: 200 } },
|
||||
@@ -45,32 +41,31 @@ test("DATA-LOSS: an htmlEmbed block is silently dropped from markdown export (no
|
||||
);
|
||||
const md = convertProseMirrorToMarkdown(input);
|
||||
|
||||
// BUG: the htmlEmbed block, including its `source` and `height` attrs, is
|
||||
// gone — only the surrounding paragraphs survive. If a future fix adds an
|
||||
// htmlEmbed case, update this test to assert the block (or a placeholder)
|
||||
// survives instead.
|
||||
assert.equal(md, "before\n\n\n\nafter", "htmlEmbed currently disappears");
|
||||
assert.ok(!md.includes("<b>hi</b>"), "the embed source is NOT preserved (data-loss)");
|
||||
assert.match(md, /data-type="htmlEmbed"/);
|
||||
assert.match(md, /data-height="200"/);
|
||||
// The raw source is base64-encoded in data-source (not emitted verbatim), so
|
||||
// the surrounding markdown cannot be corrupted by hostile embed HTML.
|
||||
assert.match(md, /data-source="[^"]+"/);
|
||||
assert.ok(md.includes("before") && md.includes("after"));
|
||||
});
|
||||
|
||||
test("DATA-LOSS: an htmlEmbed does NOT round-trip (PM -> markdown -> PM loses the node)", async () => {
|
||||
test("htmlEmbed round-trips PM -> markdown -> PM (node + source recovered)", async () => {
|
||||
const input = doc(
|
||||
para(text("x")),
|
||||
{ type: "htmlEmbed", attrs: { source: "<i>raw</i>", height: 120 } },
|
||||
);
|
||||
const out = await markdownToProseMirror(convertProseMirrorToMarkdown(input));
|
||||
assert.equal(
|
||||
findAll(out, "htmlEmbed").length,
|
||||
0,
|
||||
"htmlEmbed is lost across a markdown round-trip (known data-loss gap)",
|
||||
);
|
||||
const embeds = findAll(out, "htmlEmbed");
|
||||
assert.equal(embeds.length, 1, "htmlEmbed survives the markdown round-trip");
|
||||
assert.equal(embeds[0].attrs.source, "<i>raw</i>", "source recovered intact");
|
||||
});
|
||||
|
||||
test("DATA-LOSS: a pageBreak block is silently dropped from markdown export (no converter case)", () => {
|
||||
test("pageBreak block survives markdown export and round-trips", async () => {
|
||||
const input = doc(para(text("a")), { type: "pageBreak" }, para(text("b")));
|
||||
const md = convertProseMirrorToMarkdown(input);
|
||||
// BUG: pageBreak (a block atom with no converter case) disappears.
|
||||
assert.equal(md, "a\n\n\n\nb", "pageBreak currently disappears");
|
||||
assert.match(md, /<!--pagebreak-->/);
|
||||
const out = await markdownToProseMirror(md);
|
||||
assert.equal(findAll(out, "pageBreak").length, 1);
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
@@ -0,0 +1,320 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { searchInDoc } from "../../build/lib/page-search.js";
|
||||
import { getNodeByRef } from "../../build/lib/node-ops.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Document builders. Mirror the Docmost ProseMirror shape: paragraphs/headings
|
||||
// carry an attrs.id and hold text nodes; a text node may carry marks, and
|
||||
// adjacent runs with different marks are GLUED by blockPlainText so a match can
|
||||
// straddle a mark boundary. Table cells hold id-less paragraphs.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const text = (t, marks) => (marks ? { type: "text", text: t, marks } : { type: "text", text: t });
|
||||
const para = (id, ...children) => ({ type: "paragraph", attrs: { id }, content: children });
|
||||
const heading = (id, level, t) => ({
|
||||
type: "heading",
|
||||
attrs: { id, level },
|
||||
content: [text(t)],
|
||||
});
|
||||
|
||||
function doc(...content) {
|
||||
return { type: "doc", content };
|
||||
}
|
||||
|
||||
test("literal substring: finds every occurrence with total/truncated and refs", () => {
|
||||
const d = doc(
|
||||
para("p1", text("The cat sat on the cat mat.")),
|
||||
heading("h1", 2, "Another cat here"),
|
||||
);
|
||||
const res = searchInDoc(d, "cat");
|
||||
assert.equal(res.total, 3);
|
||||
assert.equal(res.truncated, false);
|
||||
assert.equal(res.matches.length, 3);
|
||||
// First hit: paragraph p1, block index 0.
|
||||
assert.equal(res.matches[0].nodeId, "p1");
|
||||
assert.equal(res.matches[0].blockIndex, 0);
|
||||
assert.equal(res.matches[0].type, "paragraph");
|
||||
assert.equal(res.matches[0].match, "cat");
|
||||
// Third hit is in the heading (block index 1).
|
||||
assert.equal(res.matches[2].nodeId, "h1");
|
||||
assert.equal(res.matches[2].blockIndex, 1);
|
||||
assert.equal(res.matches[2].type, "heading");
|
||||
});
|
||||
|
||||
test("context windows: before/after are drawn from the SAME container", () => {
|
||||
const d = doc(para("p1", text("alpha beta gamma delta")));
|
||||
const res = searchInDoc(d, "gamma");
|
||||
assert.equal(res.matches.length, 1);
|
||||
assert.equal(res.matches[0].before, "alpha beta ");
|
||||
assert.equal(res.matches[0].match, "gamma");
|
||||
assert.equal(res.matches[0].after, " delta");
|
||||
});
|
||||
|
||||
test("context windows are bounded to ~40 chars each side", () => {
|
||||
const long = "x".repeat(100);
|
||||
const d = doc(para("p1", text(long + "NEEDLE" + long)));
|
||||
const res = searchInDoc(d, "NEEDLE");
|
||||
assert.equal(res.matches.length, 1);
|
||||
assert.equal(res.matches[0].before.length, 40);
|
||||
assert.equal(res.matches[0].after.length, 40);
|
||||
});
|
||||
|
||||
test("case-insensitive by default; caseSensitive:true narrows", () => {
|
||||
const d = doc(para("p1", text("Cat CAT cat")));
|
||||
assert.equal(searchInDoc(d, "cat").total, 3);
|
||||
assert.equal(searchInDoc(d, "cat", { caseSensitive: true }).total, 1);
|
||||
// Reported match preserves the ORIGINAL casing even under a folded search.
|
||||
const res = searchInDoc(d, "cat");
|
||||
assert.deepEqual(
|
||||
res.matches.map((m) => m.match),
|
||||
["Cat", "CAT", "cat"],
|
||||
);
|
||||
});
|
||||
|
||||
test("match survives an inline mark boundary (glued runs)", () => {
|
||||
// "т.е." is fractured across three text nodes by bold/italic marks.
|
||||
const d = doc(
|
||||
para(
|
||||
"p1",
|
||||
text("вводное слово, "),
|
||||
text("т", [{ type: "bold" }]),
|
||||
text(".", [{ type: "italic" }]),
|
||||
text("е", [{ type: "bold" }]),
|
||||
text(". дальше"),
|
||||
),
|
||||
);
|
||||
const res = searchInDoc(d, "т.е.");
|
||||
assert.equal(res.total, 1);
|
||||
assert.equal(res.matches[0].match, "т.е.");
|
||||
assert.equal(res.matches[0].nodeId, "p1");
|
||||
});
|
||||
|
||||
test("regex engine: character classes and word boundaries", () => {
|
||||
const d = doc(para("p1", text("v1 v22 version v3")));
|
||||
const res = searchInDoc(d, "\\bv\\d+\\b", { regex: true });
|
||||
assert.deepEqual(
|
||||
res.matches.map((m) => m.match),
|
||||
["v1", "v22", "v3"],
|
||||
);
|
||||
// "version" is not matched by \bv\d+\b.
|
||||
assert.equal(res.total, 3);
|
||||
});
|
||||
|
||||
test("regex is case-insensitive by default and respects caseSensitive", () => {
|
||||
const d = doc(para("p1", text("Foo foo FOO")));
|
||||
assert.equal(searchInDoc(d, "foo", { regex: true }).total, 3);
|
||||
assert.equal(
|
||||
searchInDoc(d, "foo", { regex: true, caseSensitive: true }).total,
|
||||
1,
|
||||
);
|
||||
});
|
||||
|
||||
test("regex empty/zero-length matches are skipped, not flooded", () => {
|
||||
const d = doc(para("p1", text("abc")));
|
||||
// `a*` can match the empty string at every position; we must not emit those.
|
||||
const res = searchInDoc(d, "a*", { regex: true });
|
||||
assert.equal(res.total, 1);
|
||||
assert.equal(res.matches[0].match, "a");
|
||||
});
|
||||
|
||||
test("nodeId for a table cell paragraph WITHOUT an id falls back to #<topLevelIndex>", () => {
|
||||
// A table at top-level block index 1; its cell paragraphs carry no attrs.id.
|
||||
const cellPara = (t) => ({ type: "paragraph", content: [text(t)] });
|
||||
const d = doc(
|
||||
para("intro", text("before the table")),
|
||||
{
|
||||
type: "table",
|
||||
content: [
|
||||
{
|
||||
type: "tableRow",
|
||||
content: [
|
||||
{ type: "tableCell", content: [cellPara("needle in a cell")] },
|
||||
{ type: "tableHeader", content: [cellPara("another needle")] },
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
);
|
||||
const res = searchInDoc(d, "needle");
|
||||
assert.equal(res.total, 2);
|
||||
// Both cell hits report the table's top-level #<index> (block 1) since the
|
||||
// cell paragraphs have no id.
|
||||
for (const m of res.matches) {
|
||||
assert.equal(m.nodeId, "#1");
|
||||
assert.equal(m.blockIndex, 1);
|
||||
}
|
||||
// Context is scoped to the specific cell, not the whole table's glued text.
|
||||
assert.equal(res.matches[0].after, " in a cell");
|
||||
assert.equal(res.matches[1].before, "another ");
|
||||
});
|
||||
|
||||
test("nodeId uses attrs.id when the container has one (paragraph & heading)", () => {
|
||||
const d = doc(heading("h9", 1, "heading needle"), para("p9", text("para needle")));
|
||||
const res = searchInDoc(d, "needle");
|
||||
assert.equal(res.matches[0].nodeId, "h9");
|
||||
assert.equal(res.matches[1].nodeId, "p9");
|
||||
});
|
||||
|
||||
test("limit caps the returned matches but total and truncated stay honest", () => {
|
||||
const d = doc(para("p1", text("x ".repeat(10).trim()))); // 10 'x'
|
||||
const res = searchInDoc(d, "x", { limit: 3 });
|
||||
assert.equal(res.total, 10);
|
||||
assert.equal(res.matches.length, 3);
|
||||
assert.equal(res.truncated, true);
|
||||
});
|
||||
|
||||
test("limit is clamped to the [1, 200] range", () => {
|
||||
const d = doc(para("p1", text("a".repeat(5))));
|
||||
// A limit above the ceiling still returns all 5 (< 200) without truncation.
|
||||
const hi = searchInDoc(d, "a", { limit: 9999 });
|
||||
assert.equal(hi.matches.length, 5);
|
||||
assert.equal(hi.truncated, false);
|
||||
// A non-positive limit clamps up to 1.
|
||||
const lo = searchInDoc(d, "a", { limit: 0 });
|
||||
assert.equal(lo.matches.length, 1);
|
||||
assert.equal(lo.total, 5);
|
||||
assert.equal(lo.truncated, true);
|
||||
});
|
||||
|
||||
test("invalid regex throws a clear tool error", () => {
|
||||
const d = doc(para("p1", text("hi")));
|
||||
assert.throws(
|
||||
() => searchInDoc(d, "(", { regex: true }),
|
||||
/invalid or unsupported regular expression/i,
|
||||
);
|
||||
});
|
||||
|
||||
test("RE2: a catastrophic-backtracking pattern completes FAST and correctly (no ReDoS)", () => {
|
||||
// (a+)+$ against a long run of 'a' followed by a non-'a' is the classic
|
||||
// catastrophic-backtracking case that wedges the JS RegExp engine for
|
||||
// seconds/forever. Under RE2 (linear time) it returns effectively instantly.
|
||||
const d = doc(para("p1", text("a".repeat(50_000) + "b")));
|
||||
const t0 = Date.now();
|
||||
const res = searchInDoc(d, "(a+)+$", { regex: true });
|
||||
const elapsed = Date.now() - t0;
|
||||
// No '$'-anchored all-'a' run exists (there's a trailing 'b'), so no match.
|
||||
assert.equal(res.total, 0);
|
||||
assert.equal(res.matches.length, 0);
|
||||
// Generous ceiling: the JS engine would take orders of magnitude longer.
|
||||
assert.ok(elapsed < 1000, `expected fast completion, took ${elapsed}ms`);
|
||||
});
|
||||
|
||||
test("RE2: catastrophic pattern that DOES match still completes fast and finds it", () => {
|
||||
// (a+)+b matches the whole "aaa…b"; RE2 finds it in linear time.
|
||||
const d = doc(para("p1", text("a".repeat(20_000) + "b")));
|
||||
const t0 = Date.now();
|
||||
const res = searchInDoc(d, "(a+)+b", { regex: true });
|
||||
const elapsed = Date.now() - t0;
|
||||
assert.equal(res.total, 1);
|
||||
assert.equal(res.matches[0].match, "a".repeat(20_000) + "b");
|
||||
assert.ok(elapsed < 1000, `expected fast completion, took ${elapsed}ms`);
|
||||
});
|
||||
|
||||
test("RE2: unsupported lookaround/backreference patterns yield the clear unsupported-regex error", () => {
|
||||
const d = doc(para("p1", text("hello")));
|
||||
// Lookahead / lookbehind / backreference are backtracking-only features RE2
|
||||
// rejects at compile time — a clean tool error, never a hang.
|
||||
assert.throws(
|
||||
() => searchInDoc(d, "foo(?=bar)", { regex: true }),
|
||||
/invalid or unsupported regular expression/i,
|
||||
);
|
||||
assert.throws(
|
||||
() => searchInDoc(d, "(?<=foo)bar", { regex: true }),
|
||||
/invalid or unsupported regular expression/i,
|
||||
);
|
||||
assert.throws(
|
||||
() => searchInDoc(d, "(a)\\1", { regex: true }),
|
||||
/invalid or unsupported regular expression/i,
|
||||
);
|
||||
});
|
||||
|
||||
test("F3 round-trip: every match's nodeId resolves through the REAL getNodeByRef consumer", () => {
|
||||
// A doc mixing an attrs.id paragraph and an id-less table-cell paragraph, so
|
||||
// both ref formats (block id and "#<index>") are exercised end-to-end.
|
||||
const cellPara = (t) => ({ type: "paragraph", content: [text(t)] });
|
||||
const d = doc(
|
||||
para("intro", text("find needle here")), // attrs.id ref -> "intro"
|
||||
{
|
||||
type: "table",
|
||||
content: [
|
||||
{
|
||||
type: "tableRow",
|
||||
content: [
|
||||
{ type: "tableCell", content: [cellPara("cell needle")] }, // id-less -> "#1"
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
);
|
||||
const res = searchInDoc(d, "needle");
|
||||
assert.equal(res.total, 2);
|
||||
|
||||
// Match 0: an attrs.id ref must resolve to that exact paragraph.
|
||||
assert.equal(res.matches[0].nodeId, "intro");
|
||||
const byId = getNodeByRef(d, res.matches[0].nodeId);
|
||||
assert.ok(byId, "attrs.id ref must resolve via getNodeByRef");
|
||||
assert.equal(byId.type, "paragraph");
|
||||
assert.equal(byId.node.attrs.id, "intro");
|
||||
|
||||
// Match 1: an id-less table cell falls back to the table's "#<index>", which
|
||||
// getNodeByRef resolves to the TOP-LEVEL block (the table) by index.
|
||||
assert.equal(res.matches[1].nodeId, "#1");
|
||||
const byIndex = getNodeByRef(d, res.matches[1].nodeId);
|
||||
assert.ok(byIndex, "#<index> ref must resolve via getNodeByRef");
|
||||
assert.equal(byIndex.type, "table");
|
||||
});
|
||||
|
||||
test("F4: before/after are pinned correctly at string edges (clamp not dropped)", () => {
|
||||
// Match within the first CONTEXT (40) chars of a container LONGER than
|
||||
// CONTEXT: before is only the chars that exist, never a negative-index slice.
|
||||
const head = doc(para("p1", text("ab NEEDLE" + "x".repeat(100))));
|
||||
const r1 = searchInDoc(head, "NEEDLE");
|
||||
assert.equal(r1.matches.length, 1);
|
||||
assert.equal(r1.matches[0].before, "ab ");
|
||||
assert.equal(r1.matches[0].after.length, 40); // plenty of trailing 'x'
|
||||
|
||||
// Match at index 0: before is empty.
|
||||
const atStart = doc(para("p1", text("NEEDLE tail")));
|
||||
const r2 = searchInDoc(atStart, "NEEDLE");
|
||||
assert.equal(r2.matches[0].before, "");
|
||||
assert.equal(r2.matches[0].after, " tail");
|
||||
|
||||
// Match at the container END: after is empty.
|
||||
const atEnd = doc(para("p1", text("lead NEEDLE")));
|
||||
const r3 = searchInDoc(atEnd, "NEEDLE");
|
||||
assert.equal(r3.matches[0].before, "lead ");
|
||||
assert.equal(r3.matches[0].after, "");
|
||||
});
|
||||
|
||||
test("empty or whitespace-only query is rejected", () => {
|
||||
const d = doc(para("p1", text("hi")));
|
||||
assert.throws(() => searchInDoc(d, ""), /query is empty/i);
|
||||
assert.throws(() => searchInDoc(d, " "), /query is empty/i);
|
||||
assert.throws(() => searchInDoc(d, undefined), /query is empty/i);
|
||||
});
|
||||
|
||||
test("an over-long pattern is rejected (anti-ReDoS pattern cap)", () => {
|
||||
const d = doc(para("p1", text("hi")));
|
||||
assert.throws(() => searchInDoc(d, "a".repeat(1001)), /too long/i);
|
||||
});
|
||||
|
||||
test("no matches yields an empty, non-truncated result", () => {
|
||||
const d = doc(para("p1", text("nothing to see")));
|
||||
const res = searchInDoc(d, "zebra");
|
||||
assert.deepEqual(res, { total: 0, truncated: false, matches: [] });
|
||||
});
|
||||
|
||||
test("null-safe on a missing/empty doc", () => {
|
||||
assert.deepEqual(searchInDoc(null, "x"), {
|
||||
total: 0,
|
||||
truncated: false,
|
||||
matches: [],
|
||||
});
|
||||
assert.deepEqual(searchInDoc({ type: "doc" }, "x"), {
|
||||
total: 0,
|
||||
truncated: false,
|
||||
matches: [],
|
||||
});
|
||||
});
|
||||
@@ -165,3 +165,67 @@ test("import: a colored mention span keeps the mention node", async () => {
|
||||
const out = await markdownToProseMirror('<span data-type="mention" data-id="u1" data-label="Alice" style="color: blue">@Alice</span>');
|
||||
assert.equal(findNodes(out, "mention").length, 1, "mention node must survive a colored span");
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// #293 STEP 5 canon safety net. These assert STRUCTURE/content preservation
|
||||
// (format-agnostic: the node/mark and its value survive PM -> markdown -> PM,
|
||||
// and the markdown is idempotent), NOT the exact markdown bytes — so they stay
|
||||
// valid regardless of the concrete canonical spelling. They cover the node/mark
|
||||
// types whose canonical markdown form changed in #293 (highlight-without-color,
|
||||
// textAlign, subpages, inline footnotes) and complement the existing math /
|
||||
// media / mention / column round-trips above.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("round-trip: highlight WITHOUT a color survives as a highlight mark (==)", async () => {
|
||||
const input = doc(para(text("hi", [{ type: "highlight", attrs: { color: null } }])));
|
||||
const md = convertProseMirrorToMarkdown(input);
|
||||
const out = await roundtrip(input);
|
||||
const hit = findNodes(out, "text").find(
|
||||
(n) => n.text === "hi" && (n.marks || []).some((m) => m.type === "highlight"),
|
||||
);
|
||||
assert.ok(hit, "the highlight mark must survive a color-less round-trip");
|
||||
// Idempotent markdown.
|
||||
assert.equal(convertProseMirrorToMarkdown(out), md);
|
||||
});
|
||||
|
||||
test("round-trip: paragraph textAlign survives via the attached-comment directive", async () => {
|
||||
const input = doc({
|
||||
type: "paragraph",
|
||||
attrs: { textAlign: "center" },
|
||||
content: [text("mid")],
|
||||
});
|
||||
const md = convertProseMirrorToMarkdown(input);
|
||||
const out = await roundtrip(input);
|
||||
const p = findNodes(out, "paragraph").find((n) => n.attrs && n.attrs.textAlign === "center");
|
||||
assert.ok(p, "textAlign must be restored on the paragraph");
|
||||
assert.equal(convertProseMirrorToMarkdown(out), md, "textAlign round-trip is idempotent");
|
||||
});
|
||||
|
||||
test("round-trip: subpages atom survives", async () => {
|
||||
const input = doc({ type: "subpages" });
|
||||
const out = await roundtrip(input);
|
||||
assert.equal(findNodes(out, "subpages").length, 1, "subpages node must survive");
|
||||
});
|
||||
|
||||
test("round-trip: inline footnote survives with body text (canonical structure)", async () => {
|
||||
const input = doc(
|
||||
para(text("Claim"), { type: "footnoteReference", attrs: { id: "fnA" } }),
|
||||
{
|
||||
type: "footnotesList",
|
||||
content: [
|
||||
{
|
||||
type: "footnoteDefinition",
|
||||
attrs: { id: "fnA" },
|
||||
content: [para(text("the evidence"))],
|
||||
},
|
||||
],
|
||||
},
|
||||
);
|
||||
const md = convertProseMirrorToMarkdown(input);
|
||||
const out = await roundtrip(input);
|
||||
assert.equal(findNodes(out, "footnoteReference").length, 1);
|
||||
assert.equal(findNodes(out, "footnotesList").length, 1);
|
||||
assert.equal(findNodes(out, "footnoteDefinition").length, 1);
|
||||
assert.match(JSON.stringify(out), /the evidence/, "footnote body survives");
|
||||
// Byte-stable (the schema id is never written to markdown).
|
||||
assert.equal(convertProseMirrorToMarkdown(out), md);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
// Guard: every tool the MCP server registers must be routed by intent in
|
||||
// SERVER_INSTRUCTIONS — the editing guide clients receive in the initialize
|
||||
// result. Without this, new tools silently rot out of the guide and agents
|
||||
// never learn to pick them (the guide once omitted 17 of 41 tools, including
|
||||
// get_outline, which pushed agents into fetching whole documents for block
|
||||
// ids). Tool names are extracted from the SOURCE (index.ts inline
|
||||
// registrations + tool-specs.ts shared specs) so a registration added either
|
||||
// way is caught; the guide text itself is imported from the build so the test
|
||||
// checks what actually ships.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { dirname, join } from "node:path";
|
||||
|
||||
import { SERVER_INSTRUCTIONS } from "../../build/index.js";
|
||||
|
||||
const HERE = dirname(fileURLToPath(import.meta.url));
|
||||
const SRC = join(HERE, "..", "..", "src");
|
||||
|
||||
// Tools DELIBERATELY absent from the guide. Keep this list minimal and
|
||||
// justify every entry — the default is: every tool gets routed.
|
||||
const EXCEPTIONS = new Set([
|
||||
// Trivial and self-explanatory; carries no routing decision.
|
||||
"get_workspace",
|
||||
]);
|
||||
|
||||
/**
|
||||
* Extract every registered tool name from the source. Two registration
|
||||
* mechanisms exist and both are covered:
|
||||
* - inline `server.registerTool("name", ...)` calls in index.ts;
|
||||
* - shared specs in tool-specs.ts (`mcpName: 'name'`), registered via
|
||||
* registerShared(SHARED_TOOL_SPECS.x, ...).
|
||||
*/
|
||||
function registeredToolNames() {
|
||||
const indexSrc = readFileSync(join(SRC, "index.ts"), "utf8");
|
||||
const specsSrc = readFileSync(join(SRC, "tool-specs.ts"), "utf8");
|
||||
const names = new Set();
|
||||
for (const m of indexSrc.matchAll(/registerTool\(\s*"([a-z0-9_]+)"/g)) {
|
||||
names.add(m[1]);
|
||||
}
|
||||
for (const m of specsSrc.matchAll(/mcpName:\s*['"]([a-z0-9_]+)['"]/g)) {
|
||||
names.add(m[1]);
|
||||
}
|
||||
return names;
|
||||
}
|
||||
|
||||
test("every registered tool is mentioned in SERVER_INSTRUCTIONS", () => {
|
||||
const names = registeredToolNames();
|
||||
// Sanity: if extraction regressed (regex drift), fail loudly rather than
|
||||
// vacuously passing on an empty set.
|
||||
assert.ok(
|
||||
names.size >= 40,
|
||||
`sanity: expected to extract 40+ registered tools, got ${names.size} — ` +
|
||||
"the extraction regexes in this test likely drifted from the source",
|
||||
);
|
||||
const missing = [...names]
|
||||
.filter((n) => !EXCEPTIONS.has(n))
|
||||
// \b<name>\b: `_` is a word char, so \bget_page\b does NOT match inside
|
||||
// get_page_json — a tool can't hide behind a longer sibling's mention.
|
||||
.filter((n) => !new RegExp(`\\b${n}\\b`).test(SERVER_INSTRUCTIONS))
|
||||
.sort();
|
||||
assert.deepEqual(
|
||||
missing,
|
||||
[],
|
||||
`tools missing from SERVER_INSTRUCTIONS: ${missing.join(", ")} — ` +
|
||||
"update the guide in packages/mcp/src/index.ts (see its MAINTENANCE " +
|
||||
"RULE comment), or add a justified entry to EXCEPTIONS here",
|
||||
);
|
||||
});
|
||||
|
||||
test("EXCEPTIONS entries are real registered tools", () => {
|
||||
// A stale exception (tool renamed/removed) must be cleaned up, otherwise
|
||||
// the list quietly grows past its purpose.
|
||||
const names = registeredToolNames();
|
||||
for (const name of EXCEPTIONS) {
|
||||
assert.ok(
|
||||
names.has(name),
|
||||
`EXCEPTIONS entry "${name}" is not a registered tool — remove it`,
|
||||
);
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,45 @@
|
||||
{
|
||||
"name": "@docmost/prosemirror-markdown",
|
||||
"version": "0.1.0",
|
||||
"description": "Pure ProseMirror <-> Markdown converter + schema mirror (headless, framework-free).",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "./build/index.js",
|
||||
"types": "./build/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./build/index.d.ts",
|
||||
"default": "./build/index.js"
|
||||
}
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"watch": "tsc --watch",
|
||||
"test": "vitest run",
|
||||
"test:watch": "vitest"
|
||||
},
|
||||
"license": "MIT",
|
||||
"dependencies": {
|
||||
"@tiptap/core": "3.20.4",
|
||||
"@tiptap/extension-highlight": "3.20.4",
|
||||
"@tiptap/extension-image": "3.20.4",
|
||||
"@tiptap/extension-subscript": "3.20.4",
|
||||
"@tiptap/extension-superscript": "3.20.4",
|
||||
"@tiptap/extension-task-item": "3.20.4",
|
||||
"@tiptap/extension-task-list": "3.20.4",
|
||||
"@tiptap/html": "3.20.4",
|
||||
"@tiptap/pm": "3.20.4",
|
||||
"@tiptap/starter-kit": "3.20.4",
|
||||
"jsdom": "25.0.0",
|
||||
"marked": "17.0.5",
|
||||
"zod": "4.3.6"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@docmost/editor-ext": "workspace:*",
|
||||
"@types/jsdom": "^21.1.7",
|
||||
"@types/node": "^20.0.0",
|
||||
"fast-check": "^4.8.0",
|
||||
"typescript": "^5.0.0",
|
||||
"vitest": "4.1.6"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
/**
|
||||
* Public surface of `@docmost/prosemirror-markdown`.
|
||||
*
|
||||
* A headless, framework-free ProseMirror <-> Markdown converter plus the
|
||||
* Docmost schema mirror. Everything lives under `lib/` (the converter core);
|
||||
* this top-level barrel simply re-exports that surface so the package entry is
|
||||
* the converter surface.
|
||||
*/
|
||||
export * from "./lib/index.js";
|
||||
@@ -0,0 +1,124 @@
|
||||
/**
|
||||
* Attached-comment convention (#293 canon).
|
||||
*
|
||||
* Some block-level attributes have no native markdown syntax (paragraph/heading
|
||||
* `textAlign` — #9; image/media attrs — #4/#8). Rather than HTML-wrapping the
|
||||
* whole block (the old `<div align>` / `<p style>` forms, which the maintainer
|
||||
* had to patch repeatedly and which did not round-trip cleanly), we ATTACH a
|
||||
* compact HTML comment at the END of the block's rendered line:
|
||||
*
|
||||
* Some paragraph text <!--attrs {"textAlign":"center"}-->
|
||||
*
|
||||
* The comment is invisible in any markdown renderer and is dropped by the
|
||||
* DOM/generateJSON import stage, so it can never leak into the document body.
|
||||
* The importer intercepts it BEFORE that stage (see markdown-to-prosemirror's
|
||||
* applyAttachedComments) and re-applies the encoded attributes to the node.
|
||||
*
|
||||
* This module holds the two PURE, reusable primitives of the convention so the
|
||||
* serializer, the parser, and future decisions (#4 image, #8 media) share ONE
|
||||
* implementation:
|
||||
* - `attachedCommentFor(name, json)` — build the comment string.
|
||||
* - `parseAttachedComment(data)` — parse a comment node's data back.
|
||||
*/
|
||||
|
||||
/**
|
||||
* A parsed attached comment: the leading `name` token and the decoded JSON
|
||||
* object payload (empty object when the comment carried no JSON body).
|
||||
*/
|
||||
export interface AttachedComment {
|
||||
name: string;
|
||||
attrs: Record<string, unknown>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Grammar of an attached comment's DATA (the text between `<!--` and `-->`):
|
||||
* a leading name token (`attrs`, `img`, …) optionally followed by whitespace
|
||||
* and a single JSON object. The name deliberately does NOT allow `:` so the
|
||||
* file-level envelope comments (`docmost:meta` / `docmost:comments`) never match
|
||||
* and stay inert here.
|
||||
*/
|
||||
const ATTACHED_COMMENT_RE = /^\s*([A-Za-z][\w-]*)(?:\s+(\{[\s\S]*\}))?\s*$/;
|
||||
|
||||
/**
|
||||
* Build an attached HTML comment `<!--name {compact-json}-->` for `json`.
|
||||
*
|
||||
* The JSON is emitted compactly (no spaces) via `JSON.stringify`. A string value
|
||||
* may legitimately contain two consecutive hyphens `--`, which would prematurely
|
||||
* close the HTML comment (`-->`). We defuse that WITHOUT changing the decoded
|
||||
* value: each hyphen of every `--` pair is rewritten as the JSON unicode escape
|
||||
* `-`, so `JSON.parse` on the reading side restores the exact original
|
||||
* hyphens. `--` can only occur inside a JSON string (structural JSON never
|
||||
* produces it), so a blanket replace over the stringified payload is safe.
|
||||
*/
|
||||
export function attachedCommentFor(name: string, json: object): string {
|
||||
return `<!--${name} ${escapeCommentJson(json)}-->`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Compactly stringify `json` and defuse any `--` pair so the payload can never
|
||||
* close the HTML comment early. Shared by `attachedCommentFor` (attached form)
|
||||
* and `standaloneCommentFor` (standalone form) so both stay in sync.
|
||||
*
|
||||
* A string value may legitimately contain two consecutive hyphens `--`, which
|
||||
* would prematurely close the comment (`-->`). We defuse that WITHOUT changing
|
||||
* the decoded value: each hyphen of every `--` pair is rewritten as the JSON
|
||||
* unicode escape `-`, so `JSON.parse` on the reading side restores the exact
|
||||
* original hyphens. `--` can only occur inside a JSON string (structural JSON
|
||||
* never produces it), so a blanket replace over the stringified payload is safe.
|
||||
* Scanning left-to-right and replacing each `--` handles odd runs too (`---` ->
|
||||
* two escapes + one bare `-`, still `---` after JSON.parse).
|
||||
*/
|
||||
function escapeCommentJson(json: object): string {
|
||||
return JSON.stringify(json).replace(/--/g, "\\u002d\\u002d");
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a STANDALONE machinery comment (#293 canon #5) for a block node that
|
||||
* lives on its OWN line, e.g. `<!--pagebreak-->` or `<!--subpages-->`.
|
||||
*
|
||||
* Grammar is identical to the attached form (`<!--name {JSON?}-->`), but the
|
||||
* JSON body is emitted ONLY when there are real attributes to carry:
|
||||
* - `standaloneCommentFor("pagebreak")` -> `<!--pagebreak-->`
|
||||
* - `standaloneCommentFor("subpages")` -> `<!--subpages-->`
|
||||
* - `standaloneCommentFor("subpages", {recursive:true})`
|
||||
* -> `<!--subpages {"recursive":true}-->`
|
||||
*
|
||||
* When `attrs` is undefined/null/empty-object the comment is name-only (no JSON,
|
||||
* which parses back to default attrs). Otherwise the JSON body is emitted with
|
||||
* the SAME `--`-escaping as `attachedCommentFor` (via `escapeCommentJson`), so
|
||||
* the standalone and attached encoders can never diverge.
|
||||
*/
|
||||
export function standaloneCommentFor(name: string, attrs?: object | null): string {
|
||||
if (!attrs || Object.keys(attrs).length === 0) {
|
||||
return `<!--${name}-->`;
|
||||
}
|
||||
return `<!--${name} ${escapeCommentJson(attrs)}-->`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the DATA of a comment node into `{ name, attrs }`, or `null` when it is
|
||||
* not a well-formed attached comment.
|
||||
*
|
||||
* Fail-open by design (maintainer spec): a comment whose name token is missing,
|
||||
* whose JSON body is malformed, or whose body is not a plain object returns
|
||||
* `null` so the caller ignores it and keeps default attributes. Unknown keys in
|
||||
* a valid object are preserved here and filtered by the caller.
|
||||
*/
|
||||
export function parseAttachedComment(data: string): AttachedComment | null {
|
||||
const m = ATTACHED_COMMENT_RE.exec(data);
|
||||
if (!m) return null;
|
||||
const name = m[1];
|
||||
if (m[2] === undefined) {
|
||||
// Name-only comment (no JSON body): a valid attached marker with no attrs.
|
||||
return { name, attrs: {} };
|
||||
}
|
||||
try {
|
||||
const parsed = JSON.parse(m[2]);
|
||||
if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) {
|
||||
return { name, attrs: parsed as Record<string, unknown> };
|
||||
}
|
||||
return null; // fail-open: payload is not a plain object
|
||||
} catch {
|
||||
return null; // fail-open: malformed JSON -> ignore the comment
|
||||
}
|
||||
}
|
||||
+8
-3
@@ -45,9 +45,11 @@
|
||||
* converter coercing numeric `width`/`height` to strings, which is outside
|
||||
* canonicalize's scope.
|
||||
*
|
||||
* NOTE: `image` has NO non-null align default — its `align` defaults to `null`
|
||||
* (docmost-schema.ts L174), so it is already handled by the null-drop rule and
|
||||
* is intentionally NOT listed here.
|
||||
* NOTE: `image` align now defaults to `"center"` — unified with editor-ext
|
||||
* (#293 canon #4). It is listed below so a canonical image drops `align` when
|
||||
* it equals "center" (absent ≡ default), exactly like the diagram/media nodes.
|
||||
* A null align is likewise dropped by the null-drop rule and re-imports as the
|
||||
* "center" default, so bare `` images stay canonically clean.
|
||||
*/
|
||||
const KNOWN_DEFAULTS: Record<string, Record<string, unknown>> = {
|
||||
// mark types
|
||||
@@ -62,6 +64,9 @@ const KNOWN_DEFAULTS: Record<string, Record<string, unknown>> = {
|
||||
orderedList: {
|
||||
start: 1,
|
||||
},
|
||||
image: {
|
||||
align: "center",
|
||||
},
|
||||
drawio: {
|
||||
align: "center",
|
||||
},
|
||||
+16
-1
@@ -256,7 +256,22 @@ const DocmostAttributes = Extension.create({
|
||||
{
|
||||
types: ["image"],
|
||||
attributes: {
|
||||
align: { default: null },
|
||||
// #293 canon #4: the image `align` default is unified to "center"
|
||||
// (matching editor-ext, the source of real user documents) so an
|
||||
// editor-authored image — which is always align="center" — serializes
|
||||
// as the clean `` form with NO attached comment, and only a
|
||||
// genuinely non-default alignment (left/right) emits an `<!--img-->`
|
||||
// comment. The DOM attribute name stays `align` (imageToHtml already
|
||||
// round-trips it as align="…"); only the DEFAULT value changed from
|
||||
// null to "center". parseHTML reads the `align` attribute so a bare
|
||||
// <img> with no align falls back to "center", and <img align="left">
|
||||
// reads "left".
|
||||
align: {
|
||||
default: "center",
|
||||
parseHTML: (el: HTMLElement) => el.getAttribute("align") || "center",
|
||||
renderHTML: (attrs: Record<string, any>) =>
|
||||
attrs.align && attrs.align !== "center" ? { align: attrs.align } : {},
|
||||
},
|
||||
// imageToHtml emits these Docmost-specific image attrs as data-*; map
|
||||
// them back explicitly so a top-level image (or one inside a column)
|
||||
// round-trips them. Without a parseHTML the default reads the bare
|
||||
@@ -0,0 +1,61 @@
|
||||
/**
|
||||
* #293 canon #2: inline footnotes `^[text]`.
|
||||
*
|
||||
* Shared, side-effect-free helpers used by BOTH the serializer
|
||||
* (markdown-converter.ts) and the importer (markdown-to-prosemirror.ts) so the
|
||||
* two directions cannot drift.
|
||||
*
|
||||
* The canonical markdown form is Pandoc/Obsidian inline footnotes: the note body
|
||||
* is written AT the reference point as `^[body]`; there is no separate
|
||||
* `[^id]: …` definition line and no bottom `<section>` list in the markdown. On
|
||||
* import the body is re-assembled into the schema's doc-level
|
||||
* `footnotesList`/`footnoteDefinition` so the editor sees the usual three-node
|
||||
* footnote model, while identical bodies MERGE to a single definition shared by
|
||||
* every reference. Ids are assigned by the importer's assembleFootnotes pass
|
||||
* (dedup on the EXACT body text -> sequential `fn-N`), NOT derived from a hash,
|
||||
* so two DIFFERENT bodies can never collide onto one definition (F1). The id is
|
||||
* never written to markdown (`^[body]` carries only text), so the round trip
|
||||
* stays byte-stable regardless of the concrete id.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Split an ENCODED footnote body (the inner captured between `^[` and its
|
||||
* matching `]`, or the value of a `data-fn-text` attribute) into its paragraph
|
||||
* markdown strings.
|
||||
*
|
||||
* Paragraph boundaries are the two-character literal separator `\n` (backslash +
|
||||
* n); a REAL backslash-n in the body was encoded as `\\n` (an escaped backslash
|
||||
* followed by n) by the serializer, so it must NOT split. The scan therefore
|
||||
* treats any `\<char>` as an escaped pair kept verbatim (so `\\` `n` stays a
|
||||
* literal backslash-then-n and the trailing `n` is plain), and only an
|
||||
* UNescaped `\n` is a separator. Every other backslash escape (`\=`, `\$`,
|
||||
* `\[`, …) is preserved untouched so the per-paragraph `parseInline` decodes it.
|
||||
*/
|
||||
export function splitFootnoteParagraphs(encoded: string): string[] {
|
||||
const paragraphs: string[] = [];
|
||||
let current = "";
|
||||
let i = 0;
|
||||
while (i < encoded.length) {
|
||||
const c = encoded[i];
|
||||
if (c === "\\" && i + 1 < encoded.length) {
|
||||
const next = encoded[i + 1];
|
||||
if (next === "n") {
|
||||
// Unescaped backslash-n: a paragraph separator.
|
||||
paragraphs.push(current);
|
||||
current = "";
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
// Any other escaped pair (including `\\`) is kept verbatim; consuming
|
||||
// BOTH chars is what makes an encoded real `\n` (`\\n`) safe — the `\\`
|
||||
// pair is taken here, leaving the following `n` as an ordinary literal.
|
||||
current += c + next;
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
current += c;
|
||||
i++;
|
||||
}
|
||||
paragraphs.push(current);
|
||||
return paragraphs;
|
||||
}
|
||||
@@ -16,9 +16,29 @@ export {
|
||||
export type { DocmostMdMeta } from "./markdown-document.js";
|
||||
|
||||
export { convertProseMirrorToMarkdown } from "./markdown-converter.js";
|
||||
export type { ConvertProseMirrorToMarkdownOptions } from "./markdown-converter.js";
|
||||
|
||||
export { markdownToProseMirror } from "./markdown-to-prosemirror.js";
|
||||
|
||||
// The Docmost tiptap schema mirror. Exposed so consumers (and the sync
|
||||
// engine's schema-validity regression tests) can build the exact ProseMirror
|
||||
// schema the converter targets.
|
||||
export { docmostExtensions } from "./docmost-schema.js";
|
||||
|
||||
// Schema-adjacent sanitizers used by consumers (mcp) so the single canonical,
|
||||
// alias-aware / allowlist implementations live ONLY here (no drifting copies).
|
||||
export { clampCalloutType, sanitizeCssColor } from "./docmost-schema.js";
|
||||
|
||||
// Attached-comment convention (#293 canon #9/#4/#8): the reusable primitives
|
||||
// the serializer/parser use to encode attrs that have no native markdown syntax
|
||||
// as trailing `<!--name {json}-->` comments.
|
||||
export {
|
||||
attachedCommentFor,
|
||||
standaloneCommentFor,
|
||||
parseAttachedComment,
|
||||
} from "./attached-comment.js";
|
||||
export type { AttachedComment } from "./attached-comment.js";
|
||||
|
||||
export {
|
||||
canonicalizeContent,
|
||||
docsCanonicallyEqual,
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,78 @@
|
||||
/**
|
||||
* Shared inline-math boundary rule (#293 canon #6).
|
||||
*
|
||||
* Pandoc's inline-math rule lives here because it is used in TWO directions that
|
||||
* MUST agree byte-for-byte on which `$…$` spans are math:
|
||||
*
|
||||
* - the IMPORT tokenizer (markdown-to-prosemirror.ts) that turns `$LaTeX$`
|
||||
* into a `mathInline` node, and
|
||||
* - the EXPORT escaper (markdown-converter.ts) that backslash-escapes a
|
||||
* would-be-math `$…$` span sitting in PROSE text so it re-imports as literal
|
||||
* text instead of silently materializing a phantom math node.
|
||||
*
|
||||
* Defining the rule ONCE guarantees the two directions never drift: a span the
|
||||
* tokenizer would match is EXACTLY a span the escaper neutralizes, so a prose
|
||||
* `$x$` round-trips as literal text and math `$x^2$` round-trips as math.
|
||||
*
|
||||
* The rule (currency-safe, from pandoc): an opening `$` is NOT followed by
|
||||
* whitespace; the closing `$` is NOT preceded by whitespace AND NOT immediately
|
||||
* followed by a digit; the inner run is non-empty, single-line, and may embed an
|
||||
* escaped `\$` (which never counts as the closer). Under this rule `$5`,
|
||||
* `$5 and $10`, `price is $5`, `a $5 b $6 c` all stay literal (no VALID closing
|
||||
* `$` exists — the `$` before a space-preceded amount fails the "not preceded by
|
||||
* whitespace" test, and a lone `$` has no closer), while `$x^2$` is math.
|
||||
*/
|
||||
|
||||
// Core pattern (unanchored). Escaping note for the string form:
|
||||
// \\$ -> a literal `$`
|
||||
// (?!\s) -> opening `$` NOT followed by whitespace (also forces a
|
||||
// non-empty inner: the next char must exist and be non-space)
|
||||
// (?:\\\\\\$|[^$\n])+? -> inner: shortest run of either an escaped `\$`
|
||||
// (consumed as a unit so it is never the closer) or any char
|
||||
// that is neither an unescaped `$` nor a newline
|
||||
// (?<!\s) -> the char before the closing `$` is NOT whitespace
|
||||
// \\$ -> closing `$`
|
||||
// (?![0-9]) -> closing `$` NOT immediately followed by a digit (currency)
|
||||
export const INLINE_MATH_SOURCE =
|
||||
"\\$(?!\\s)((?:\\\\\\$|[^$\\n])+?)(?<!\\s)\\$(?![0-9])";
|
||||
|
||||
/** Global matcher for the export-side prose escaper. */
|
||||
export const inlineMathGlobalRe = (): RegExp =>
|
||||
new RegExp(INLINE_MATH_SOURCE, "g");
|
||||
|
||||
/** Anchored matcher for the import-side marked tokenizer. */
|
||||
export const inlineMathAnchoredRe = (): RegExp =>
|
||||
new RegExp("^" + INLINE_MATH_SOURCE);
|
||||
|
||||
/** Decode a tokenizer-captured inner LaTeX: an escaped `\$` becomes `$`. */
|
||||
export const decodeInlineMathLatex = (inner: string): string =>
|
||||
inner.replace(/\\\$/g, "$");
|
||||
|
||||
/** Escape LaTeX for the `$…$` inline form so a literal `$` cannot close early. */
|
||||
export const encodeInlineMathLatex = (latex: string): string =>
|
||||
latex.replace(/\$/g, "\\$");
|
||||
|
||||
/**
|
||||
* Whether a `mathInline` node's LaTeX can be safely serialized as `$LaTeX$`
|
||||
* (vs. the always-lossless schema-HTML `<span>` fallback). Requires:
|
||||
* - non-empty (an empty span has no readable `$…$` form),
|
||||
* - non-whitespace edges (pandoc's opening/closing whitespace rules),
|
||||
* - single line (inline math never spans lines),
|
||||
* - no pre-existing `\$` and no trailing `\` — either would make the
|
||||
* `$`→`\$` escape ambiguous on decode (a `\\$` sequence, or an escaped
|
||||
* closing `$`), so those rare cases take the `<span>` fallback instead.
|
||||
* NOTE: a following-sibling digit (which would also break the pandoc closing
|
||||
* rule) cannot be seen from the node alone; that case is handled by the
|
||||
* serializer's inline-children pass, not here.
|
||||
*/
|
||||
export const inlineMathSerializable = (latex: string): boolean =>
|
||||
latex.length > 0 &&
|
||||
!/^\s/.test(latex) &&
|
||||
!/\s$/.test(latex) &&
|
||||
!/[\r\n]/.test(latex) &&
|
||||
!latex.includes("\\$") &&
|
||||
!/\\$/.test(latex);
|
||||
|
||||
/** Escape a value for an HTML double-quoted attribute (only & and " matter). */
|
||||
export const escapeMathAttr = (value: string): string =>
|
||||
value.replace(/&/g, "&").replace(/"/g, """);
|
||||
@@ -0,0 +1,172 @@
|
||||
/**
|
||||
* Shared schema-HTML builders for the media/discriminator family (#293 canon
|
||||
* #8).
|
||||
*
|
||||
* Canon #8 gives ten node types (youtube/video/audio/drawio/excalidraw —
|
||||
* image-form; pdf/attachment/embed — link-form; pageEmbed/transclusionReference
|
||||
* — standalone) a readable markdown TOP-LEVEL form (``/`[text](src)`/a
|
||||
* bare comment) plus a discriminator `<!--name {…}-->` comment. But TWO other
|
||||
* paths still need the RAW SCHEMA-HTML form of each node:
|
||||
*
|
||||
* 1. The serializer's raw-HTML/columns path (`blockToHtml`): a comment node is
|
||||
* dropped by the DOM parse stage that reads a raw-HTML block back, so inside
|
||||
* a column/cell these nodes MUST stay schema HTML or they vanish (data loss).
|
||||
* 2. The importer's `applyCommentDirectives`: to materialize the discriminator
|
||||
* comment it rebuilds the SAME schema element the raw-HTML path emits, then
|
||||
* swaps it in for the `<img>`/`<a>`/comment.
|
||||
*
|
||||
* Keeping these builders in ONE module means the serializer's raw-HTML path and
|
||||
* the importer's materialization can never drift: both call the same function.
|
||||
* Each builder reproduces BYTE-FOR-BYTE the schema HTML the top-level
|
||||
* `processNode` cases previously returned (so existing columns/raw-HTML goldens
|
||||
* stay green), and each output round-trips through the matching schema parseHTML
|
||||
* in docmost-schema.ts.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Escape a value interpolated into an HTML double-quoted attribute value.
|
||||
* Identical semantics to markdown-converter's `escapeAttr`: escape ONLY `&` and
|
||||
* `"` (idempotent; parse5 decodes both back). `<`/`>`/`'` are deliberately left
|
||||
* alone so values never accumulate escapes across round-trips.
|
||||
*/
|
||||
const escapeAttr = (value: unknown): string =>
|
||||
String(value).replace(/&/g, "&").replace(/"/g, """);
|
||||
|
||||
/**
|
||||
* Uploaded `<video>` player. Emits `<div><video …></video></div>`; the outer
|
||||
* `<div>` (no data-type) forces block treatment so marked does not wrap the
|
||||
* inline `<video>` in a `<p>`. Mirrors the Video schema: src/aria-label standard
|
||||
* attrs, the rest as data-*.
|
||||
*/
|
||||
export function videoToHtml(attrs: Record<string, any>): string {
|
||||
const parts: string[] = [`src="${escapeAttr(attrs.src ?? "")}"`];
|
||||
if (attrs.alt) parts.push(`aria-label="${escapeAttr(attrs.alt)}"`);
|
||||
if (attrs.attachmentId)
|
||||
parts.push(`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`);
|
||||
if (attrs.width != null) parts.push(`width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null) parts.push(`height="${escapeAttr(attrs.height)}"`);
|
||||
if (attrs.size != null) parts.push(`data-size="${escapeAttr(attrs.size)}"`);
|
||||
if (attrs.align) parts.push(`data-align="${escapeAttr(attrs.align)}"`);
|
||||
if (attrs.aspectRatio != null)
|
||||
parts.push(`data-aspect-ratio="${escapeAttr(attrs.aspectRatio)}"`);
|
||||
return `<div><video ${parts.join(" ")}></video></div>`;
|
||||
}
|
||||
|
||||
/** YouTube embed. Emits `div[data-type="youtube"]` (src via data-src). */
|
||||
export function youtubeToHtml(attrs: Record<string, any>): string {
|
||||
const parts: string[] = [
|
||||
`data-type="youtube"`,
|
||||
`data-src="${escapeAttr(attrs.src ?? "")}"`,
|
||||
];
|
||||
if (attrs.width != null)
|
||||
parts.push(`data-width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null)
|
||||
parts.push(`data-height="${escapeAttr(attrs.height)}"`);
|
||||
if (attrs.align) parts.push(`data-align="${escapeAttr(attrs.align)}"`);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
/** Uploaded `<audio>` player. Emits `<div><audio …></audio></div>`. */
|
||||
export function audioToHtml(attrs: Record<string, any>): string {
|
||||
const parts: string[] = [`src="${escapeAttr(attrs.src ?? "")}"`];
|
||||
if (attrs.attachmentId)
|
||||
parts.push(`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`);
|
||||
if (attrs.size != null) parts.push(`data-size="${escapeAttr(attrs.size)}"`);
|
||||
return `<div><audio ${parts.join(" ")}></audio></div>`;
|
||||
}
|
||||
|
||||
/**
|
||||
* draw.io / excalidraw diagram (shared diagramAttributes). Emits
|
||||
* `div[data-type="drawio"|"excalidraw"]` carrying src/title/alt/width/height/
|
||||
* size/aspectRatio/align/attachmentId as data-*.
|
||||
*/
|
||||
export function diagramToHtml(
|
||||
type: "drawio" | "excalidraw",
|
||||
attrs: Record<string, any>,
|
||||
): string {
|
||||
const parts: string[] = [
|
||||
`data-type="${type}"`,
|
||||
`data-src="${escapeAttr(attrs.src ?? "")}"`,
|
||||
];
|
||||
if (attrs.title != null) parts.push(`data-title="${escapeAttr(attrs.title)}"`);
|
||||
if (attrs.alt != null) parts.push(`data-alt="${escapeAttr(attrs.alt)}"`);
|
||||
if (attrs.width != null)
|
||||
parts.push(`data-width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null)
|
||||
parts.push(`data-height="${escapeAttr(attrs.height)}"`);
|
||||
if (attrs.size != null) parts.push(`data-size="${escapeAttr(attrs.size)}"`);
|
||||
if (attrs.aspectRatio != null)
|
||||
parts.push(`data-aspect-ratio="${escapeAttr(attrs.aspectRatio)}"`);
|
||||
if (attrs.align) parts.push(`data-align="${escapeAttr(attrs.align)}"`);
|
||||
if (attrs.attachmentId)
|
||||
parts.push(`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
/** Generic provider embed. Emits `div[data-type="embed"]` (src/provider/… data-*). */
|
||||
export function embedToHtml(attrs: Record<string, any>): string {
|
||||
const parts: string[] = [
|
||||
`data-type="embed"`,
|
||||
`data-src="${escapeAttr(attrs.src ?? "")}"`,
|
||||
`data-provider="${escapeAttr(attrs.provider ?? "")}"`,
|
||||
];
|
||||
if (attrs.align) parts.push(`data-align="${escapeAttr(attrs.align)}"`);
|
||||
if (attrs.width != null)
|
||||
parts.push(`data-width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null)
|
||||
parts.push(`data-height="${escapeAttr(attrs.height)}"`);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
/** Uploaded file attachment. Emits `div[data-type="attachment"]` (data-attachment-*). */
|
||||
export function attachmentToHtml(attrs: Record<string, any>): string {
|
||||
const parts: string[] = [
|
||||
`data-type="attachment"`,
|
||||
`data-attachment-url="${escapeAttr(attrs.url ?? "")}"`,
|
||||
];
|
||||
if (attrs.name)
|
||||
parts.push(`data-attachment-name="${escapeAttr(attrs.name)}"`);
|
||||
if (attrs.mime)
|
||||
parts.push(`data-attachment-mime="${escapeAttr(attrs.mime)}"`);
|
||||
if (attrs.size != null)
|
||||
parts.push(`data-attachment-size="${escapeAttr(attrs.size)}"`);
|
||||
if (attrs.attachmentId)
|
||||
parts.push(`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
/** Embedded PDF viewer. Emits `div[data-type="pdf"]` (src std, name/… data-*). */
|
||||
export function pdfToHtml(attrs: Record<string, any>): string {
|
||||
const parts: string[] = [
|
||||
`data-type="pdf"`,
|
||||
`src="${escapeAttr(attrs.src ?? "")}"`,
|
||||
];
|
||||
if (attrs.name) parts.push(`data-name="${escapeAttr(attrs.name)}"`);
|
||||
if (attrs.attachmentId)
|
||||
parts.push(`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`);
|
||||
if (attrs.size != null) parts.push(`data-size="${escapeAttr(attrs.size)}"`);
|
||||
if (attrs.width != null) parts.push(`width="${escapeAttr(attrs.width)}"`);
|
||||
if (attrs.height != null) parts.push(`height="${escapeAttr(attrs.height)}"`);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
/** Whole-page live embed. Emits `div[data-type="pageEmbed"]` (data-source-page-id). */
|
||||
export function pageEmbedToHtml(attrs: Record<string, any>): string {
|
||||
const parts: string[] = [`data-type="pageEmbed"`];
|
||||
if (attrs.sourcePageId)
|
||||
parts.push(`data-source-page-id="${escapeAttr(attrs.sourcePageId)}"`);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Live transclusion reference. Emits `div[data-type="transclusionReference"]`
|
||||
* (data-source-page-id + data-transclusion-id).
|
||||
*/
|
||||
export function transclusionReferenceToHtml(attrs: Record<string, any>): string {
|
||||
const parts: string[] = [`data-type="transclusionReference"`];
|
||||
if (attrs.sourcePageId)
|
||||
parts.push(`data-source-page-id="${escapeAttr(attrs.sourcePageId)}"`);
|
||||
if (attrs.transclusionId)
|
||||
parts.push(`data-transclusion-id="${escapeAttr(attrs.transclusionId)}"`);
|
||||
return `<div ${parts.join(" ")}></div>`;
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user