Review #6 (approve-with-comments) follow-ups:
1. canonicalize step 7 now strips bare footnoteDefinitions at ANY depth
(stripFootnoteDefinitionsDeep), not just footnotesList, in BOTH copies. A
definition hand-authored outside a list (e.g. nested in a callout via a
raw-JSON write path) was left in place while a copy was also added to the
rebuilt list -> duplicate, idempotent, self-perpetuating. Runs only in the
rebuild path (after the lists are stripped); the fast-path / placement-keep
branch is untouched. Added a shared-corpus case (bare def nested in a callout)
to pin it in both mirrors.
2. markdown-clipboard: removed the dead top-level footnoteReference check in
canonicalizePastedFootnotes (an inline atom is never a top-level slice child;
only the descendants scan can find it).
Test coverage:
4. New MCP binding tests (full-doc-write-canonicalize.test.mjs): update_page_json
and copy_page_content canonicalize the persisted full doc, asserted via a new
`replacePage` seam (symmetric to the existing `mutatePage` seam) so no live
collab socket is needed. Routed both writers through the seam.
5. New server spec (file-import-task.service.footnote-canonicalize.spec.ts): the
zip-import path (processGenericImport) canonicalizes footnotes — real
markdown->HTML->JSON via a real ImportService over a temp-dir .md file, DB trx
stubbed to capture the persisted page content. FileImportTaskService had no
spec before.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Must-fix:
- insertInlineFootnote could glue a footnoteReference inside an EXISTING
definition (nested footnotesList, or a bare footnoteDefinition with no list
wrapper), which canonicalize then dropped as an orphan — silently losing the
definition's prose. Now: (a) the body/notes boundary is computed from the first
top-level block that IS or CONTAINS (recursively) a footnotesList/
footnoteDefinition, not just a top-level list; and (b) the insertNodesAfterAnchor
core skips footnotesList/footnoteDefinition subtrees entirely (skipSubtreeTypes),
so an anchor whose only match is inside a definition -> inserted:false (clean
abort, no write). Added tests: nested-definition, bare-definition, and
body-before-nested-list-still-inserts.
- editor-ext footnote-canonicalize header listed `markdownToProseMirror` among the
canonicalizing MCP paths; it is the NON-canonicalizing primitive. Replaced with
`markdownToProseMirrorCanonical` (+ note that the plain primitive is for comment
bodies) and added copy_page_content.
- Client paste: canonicalizePastedFootnotes now skips a definitions-ONLY paste
(no footnoteReference anywhere) — canonicalizing it would strip the
reference-less list and yield an EMPTY paste. Added a test.
Suggestions:
- docmost_transform now runs validateDocStructure/validateDocUrls on the RAW
transform output BEFORE canonicalizeFootnotes (mirrors updatePageJson), so a
too-deep doc gives the intended max-depth error instead of a stack overflow.
- docmost_transform tool description now states the RESULT is footnote-canonical
(dryRun diff may show tidy-ups; idempotent after first run).
- insertFootnote: dropped the dead `result ? … : undefined` ternaries and the
`as any` casts (result is always set by the time we return; the not-found path
throws and aborts mutatePage). `const r = result!;`.
Tests / architecture:
- Added a LIVE-plugin golden case: the real footnoteSyncPlugin leaves a list with
non-empty content after it in place, and canonicalize agrees (placement parity
is now a driven property, not a hand-set expected).
- Added generateFootnoteId uuidv7 shape + uniqueness test.
- Item 9: added the ENFORCEMENT-RULE comments at the server parseProsemirrorContent
and the MCP canonicalizer header (any NEW full-doc persist path MUST canonicalize;
fragments/append/prepend and comment bodies MUST NOT). Kept per-call-site over a
brittle grep CI test (the replace-vs-fragment + comment-vs-page nuance makes a
single wrapper unsafe).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Approve-with-comments follow-ups (no blockers):
- callout: unify the GitHub-callout feature ticket on #192 (the callout-paste
feature the CHANGELOG already tracks); #218 is the public-share security work.
Fixed the code comment and test reference.
- export/utils.spec: pin current behavior of a leading-dot name (".gitignore" ->
"") — same bug class as #204 but unreachable via the sole caller, so document
not change.
- share.types: narrow ISharedPage to the actual /shares/page-info allowlist
(page -> Pick of id/slugId/title/icon/content; trimmed share; dropped the
spurious `extends IShare`). Verified all three consumers (shared-page,
link-view, mention-view) read only allowlist fields.
- editor-ext: extract shared CALLOUT_TYPES / normalizeCalloutType /
renderCalloutHtml into callout-common.marked.ts; both tokenizers
(`:::type` and `> [!type]`) now share the renderer + type dict while staying
separate. Eliminates the byte-identical renderer + duplicated type list.
- share.service: extract named predicate shareIdGrantsAccess(requestedShareId,
resolvedShare) for the id-or-key fast path (naming only, no control-flow
change); kept narrower than resolveReadableSharePage's id-only gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Must-fix:
- REAL BUG: insertInlineFootnote could splice a footnoteReference (inline atom)
into a codeBlock or an existing footnoteDefinition, persisting a schema-invalid
doc (insert_footnote skips validateDocStructure). Now the search is bounded to
the BODY (before the first footnotesList) and the insertNodesAfterAnchor core
refuses textblocks that can't hold the atom (codeBlock); when the only match is
in such a place the insert returns inserted:false and the write aborts cleanly.
Reachable via docmost_transform too. Added codeBlock / definition / fall-through
tests.
- Fixed the deepEqualJson doc comment in both copies: arrays are order-SENSITIVE
(correctness depends on it), only object keys are order-insensitive.
- README.ru.md MCP tool count 38 -> 39 (lines 36/47/63), matching README.md/AGENTS.
- CHANGELOG [Unreleased] Added entry for insert_footnote + server-side footnote
canonicalization on non-editor write paths (#228).
Suggestions:
- canonicalize step 5/7 now strips footnotesList at ANY depth (both copies), so a
schema-valid list nested in a callout/blockquote can't leave duplicate defs.
- Exclude the test-only footnote-corpus.ts fixture from the editor-ext build
(tsconfig), so it no longer ships in dist/.
- Removed the duplicate manual canonicalize cases from the MCP unit test (the
shared corpus covers them via full deepEqual); kept idempotence + immutability.
- insertInlineFootnote dedup key now keys off the inline array directly
(footnoteContentKey({ content: inline })) instead of a throwaway node.
Tests / architecture:
- New client-wrapper test (#9): overrides a small mutatePage seam to assert the
not-found path throws and persists NOTHING, and the success path shapes
footnoteId/reused/message/verify and writes the right content. Fixed the
misleading comment in footnote-write.test.mjs.
- B: cross-copy corpus parity guard test (loads both corpora, asserts deep-equal)
so a typo in one copy can't pass both suites green.
- A: declined — the full-vs-fragment decision lives at the call site, so a
prepareDocForPersist wrapper would be a bare alias for canonicalizeFootnotes;
kept the existing per-call-site comments instead.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Must-fix:
- Move canonicalizeFootnotes OUT of parseProsemirrorContent. It now runs only
on FULL writes (createPage, updatePageContent operation==='replace'), never on
an append/prepend fragment (a fragment would lose definition-only footnotes or
synthesize a bogus empty list). Add a server binding spec.
- Match the live plugin's list PLACEMENT: a single already-canonical
footnotesList is left exactly where it sits (the plugin never repositions a
sole correct list), so the first write no longer reorders content that follows
the list. Applied to BOTH the editor-ext copy and the MCP mirror; pinned by a
shared golden corpus case with content after the list.
- Fix MCP tool count 38 -> 39 (README x3, AGENTS.md) and the transformJs param
help (add canonicalizeFootnotes/insertInlineFootnote).
Simplifications:
- Remove the dead duplicate re-id mechanism (deriveFootnoteId/suffix/occurrence)
from the PURE canonicalizer in both copies — references are never renamed, so
the derived ids were never requested; first-wins-drop is the real behaviour.
This also makes the editor-ext footnote-util note about "no cross-package copy"
true again.
- Remove the sentinel round-trip in insertInlineFootnote: a generalized
insertNodesAfterAnchor core inserts the footnoteReference node directly.
- Drop the redundant per-definition deep clone in step 4 (shallow id-normalizing
copy; out is already deep-cloned).
Docs / architecture:
- Correct the editor-ext copy's "It exists because…" header to its real
consumers (server import, page.service create/update, client paste).
- Note markdownToProseMirror reuse for create/update comment in collaboration.ts.
- A: shared golden JSON corpus exercised by BOTH the editor-ext copy and the MCP
mirror (footnote-corpus.ts / .mjs) so "the two copies behave identically" is
checkable.
- C: split the MCP canonicalizer into a pure mirror + footnote-authoring.ts.
- B: import services persist via a different path, so left one-line consolidation
comments at the call sites rather than folding (does not fall out cleanly).
Tests: insertFootnote wrapper guards + docmost_transform dryRun auto-canonicalize
(MCP mock), page.service create/update + append/prepend binding (server jest),
shared corpus incl. nested-container reference.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make footnotes author-inline: the agent/tool inserts a footnote at its point
of use (anchor + text) and the numbering plus the bottom list are DERIVED
deterministically server-side. The agent has no access to footnotesList and
cannot desync — out-of-order lists, orphan definitions, and raw trailing
[^id] blocks become structurally impossible.
editor-ext:
- canonicalizeFootnotes(docJSON) -> docJSON: a pure, EditorView-free port of
footnoteSyncPlugin's end-state. Distinct reference ids in document order are
the source of truth; exactly one trailing footnotesList holds one definition
per referenced id in reference order (reusing the existing node or
synthesizing an empty one); orphans dropped; duplicate definitions resolved
deterministically (first wins, never lost); idempotent.
- Unit tests + a golden parity suite: on every editor-reachable steady state
the live footnoteSyncPlugin's JSON is a canonicalize no-op (byte-for-byte
parity), and the canonicalizer additionally repairs the out-of-order list a
non-editor write produces.
mcp:
- footnote-canonicalize.ts: behavioural mirror of the editor-ext canonicalizer
(the MCP package is intentionally decoupled from the editor barrel, like
footnote-lex/docmost-schema), plus footnoteContentKey for content dedup.
- Auto-canonicalize on EVERY write path: markdownToProseMirror (fixes import
ordering), update_page_json, and after every docmost_transform. Idempotent,
so it is a no-op when footnotes are already canonical.
- insert_footnote tool + insertInlineFootnote: anchor + markdown text -> a
mark-safe footnoteReference and a content-dedup'd definition; the list and
numbering are derived. Same-content footnotes reuse one number/definition.
- canonicalizeFootnotes + insertInlineFootnote exposed as docmost_transform
sandbox helpers.
Tests: editor-ext 157 green; MCP 325 green; server + client tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Additive test coverage across server, editor-ext, client and mcp.
#192 — AiChatService.stream integration (Section 3, against real Postgres):
- new apps/server/test/integration/ai-chat-stream.int-spec.ts drives the real
streamText through a seeded ai/test MockLanguageModelV3 and a real Node
ServerResponse, covering: onError persists an assistant error record
(status 'error' + partial answer + provider cause in metadata); external MCP
client closed exactly once on BOTH onFinish and onError; anti-tamper —
history is rebuilt from the DB transcript, not from body.messages.
#206 — red-team findings (most already fixed+tested in #212):
- mdrt-2 (UNFIXED, data loss): turndown.dataloss.test.ts documents that
pageBreak / transclusionReference / mention are silently dropped on Markdown
export (characterization + it.fails for the desired survive-export contract).
- persist-6 (UNFIXED, data loss): persistence-store.spec.ts adds an it.failing
documenting that a momentarily-empty live doc overwrites non-empty content
(left unfixed — a store-side empty-guard is a behaviour change).
#204 — test-strategy plan, highest-priority subset:
- Phase 1: mcp-clients.lease.spec.ts covers the external MCP client
lease/refcount/eviction lifecycle (leak / premature-close / double-close).
- Phase 2 data-integrity pure functions: editor-ext table-utils
(transpose/moveRow/convert round-trip) and math tokenizer false-positive
guard; client emoji-menu (+ it.fails for the unguarded localStorage
JSON.parse bug), sort-cells, normalizeTableColumnWidths; mcp htmlEmbed/
pageBreak markdown data-loss + footnote-diff; server export
getInternalLinkPageName extensionless-path bug — FIXED (small/clear) + tested.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Public sharing (#218):
- Bind public-share content to the requested shareId. getSharedPage now
enforces dto.shareId (forwarded from /share/:shareId/p/:slug): the page must
be reachable THROUGH that exact share (its own share, or an includeSubPages
ancestor that contains it). A forged/mismatched shareId 404s instead of
rendering off the slug alone and no longer leaks the real canonical key via
redirect. A request with no shareId keeps the legacy slug-capability path.
- Trim /shares/page-info: drop internal metadata (creatorId, spaceId,
workspaceId, contributorIds, lastUpdated*, parent/position, lock/template
flags, timestamps) from the anonymous payload.
- Default share-to-web includeSubPages to false (opt-in), so enabling a share
no longer silently exposes the whole sub-tree (#216).
Editor (#218):
- Harden the new-page pre-sync window: the body editor is kept read-only until
the collab provider is Connected and synced, so early keystrokes can't land
only in local ProseMirror and then be clobbered by the server's empty doc.
- Surface a "Connecting… (read-only)" affordance during the static phase so
input isn't silently swallowed.
Other:
- Breadcrumb: resolve from the page's own ancestor data (/pages/breadcrumbs)
instead of waiting for the lazily-built sidebar tree, so deep pages don't
render a blank breadcrumb for seconds.
- Pasting GitHub `> [!type]` callouts now converts to a callout node instead of
a literal blockquote (new marked extension wired into markdownToHtml).
Tests: editor-sync-state gate (client), getSharedPage share-binding (server),
github-callout markdown conversion (editor-ext).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A colliding transclusionSource id is deliberately NOT reassigned (its id is a
cross-reference key), while a missing id is still filled. Add coverage for both:
two sources sharing an id keep it (red if the NO_REASSIGN guard is removed), and
a source with no id gets a fresh one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
editor-pm-7: addUniqueIdsToDoc only FILLED missing ids and never deduplicated
existing ones, so a copy/paste or bulk-JSON duplicate that kept its attrs.id
produced two nodes sharing an id. MCP addressed edits (patch_node /
delete_node "before/after id") then hit the wrong node or both.
Walk the configured-type nodes in document order: the first occurrence of an
id keeps it (stable anchor), later duplicates are reassigned a fresh id.
transclusionSource ids are cross-reference keys (references resolve a source by
this id), so they are only filled-when-missing, never reassigned, to avoid
orphaning their references.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Approve-with-comments auto-review (8 axes); no blockers. Closes the two flagged
test gaps; the two forward-looking dedup suggestions (reconcileHasChildren helper;
unifying reconcileChildren/mergeRootTrees) are non-blocking architecture notes and
left for a follow-up (as with #186's forward-looking point).
1. Ambiguous-id refusal end-to-end (#159): the patch_node/delete_node guard
`if (replaced/deleted !== 1) return null` was only covered in pieces — the
replaceNodeById/deleteNodeById counts and assertUnambiguousMatch in isolation —
so loosening the guard would not have failed a test. New mock test stands up a
REAL Hocuspocus collab server seeded (via buildYDoc, same docmost extensions)
with a two-blocks-one-id document and drives the real client methods: both must
reject with /ambiguous/ AND never write to collab. Tracked via Hocuspocus
onChange (fires synchronously per update, unlike the debounced onStoreDocument)
so a clobbering write is actually observed — verified the test FAILS when the
guard is loosened to `< 1`.
2. scrollToReference zero-match bail: the branch "non-empty id but querySelectorAll
returns 0 -> matches[index] ?? matches[0] is undefined -> return false" (the real
desync: definition present, inline ref removed from the DOM) was uncovered. Added
a footnote.test.ts case: a definition for 'ghost' with no rendered ref -> false,
no scroll.
Verified: 313 mcp tests + 24 editor-ext footnote tests; prettier clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The re-review's blocking/structural points (lease leak, dup-id guard test,
body-before-title test, CHANGELOG, pg18, shared jsonb decoder) were already
addressed in commit 24264ef; this adds the 4 genuinely-new coverage requests:
- pt 6: `scrollToReference(id, index?)` exercised against a live editor DOM —
selects the index-th `sup[data-footnote-ref][data-id]` occurrence, falls back
to the first for out-of-range, returns false for an empty id (scrollIntoView
stubbed). (#168)
- pt 7: export `backlinkLabel` and pin the base-26 carry boundary
(25->z, 26->aa, 27->ab, 51->az, 52->ba). (#168)
- pt 8: integration fail-open — a PRESENT-but-corrupt tool_allowlist (jsonb
string scalar holding non-array JSON) reads back as null ("no restriction"),
covering normalizeRow's degrade branch. (#159 #172/#173)
- pt 9: getFootnoteRefCount cache invalidation — adding a `[^a]` reference bumps
the cached count 2 -> 3. (#168)
Verified: editor-ext footnote 23; client structure 7 + tsc; server int 8.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
After #166 a repeated `[^a]` is one footnote (reuse): one number, one
definition, N forward links. But the definition's ↩ only returned to the
FIRST reference. Now a definition with N references shows ↩ a b c …, each
backlink scrolling to its own occurrence (Pandoc/Wikipedia convention); a
single-reference footnote keeps the plain ↩ unchanged.
- editor-ext: `computeFootnoteRefCounts(doc)` (id -> occurrence count) cached
alongside the number map in the numbering plugin state; `getFootnoteRefCount`
getter (O(1), no per-render doc walk). `scrollToReference(id, index?)` picks
the index-th `sup[data-footnote-ref][data-id]` occurrence (document order),
falling back to the first.
- client: FootnoteDefinitionView renders one lettered link (a, b, c, … aa …)
per occurrence when refCount > 1; the chrome stays after the contentDOM so
the #146 caret invariant holds. i18n keys (ru) added.
Tests: computeFootnoteRefCounts + getFootnoteRefCount (reuse counts, unknown
id => 0); structure test gains 3 cases (N lettered links render, click jumps
to the n-th occorrence, single ref => one ↩). NOTE: the visual layout of the
backlink row needs a real browser to verify (jsdom can't); the structural and
behavioral contract is covered headless.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to the merged #166/#169. Addresses the second review pass (comment
1227):
- footnoteWarnings plumbing: extract a single `footnoteWarningsField(markdown)`
helper (footnote-analyze) and use it at all three call sites (create_page,
update_page, import_page_markdown) so the field is attached identically.
- New unit test footnote-warnings-import.test.mjs pins the contract that was
uncovered: the field is present on problems / omitted on clean input, and the
IMPORT path analyzes the BODY after the docmost:meta / docmost:comments blocks
(a footnote-like token inside those JSON blocks must NOT warn; a real body
marker must). Tested via the same pure composition the importer uses
(footnoteWarningsField(parseDocmostMarkdown(full).body)) — no collab socket
needed; a regression that analyzed fullMarkdown or skipped the body split would
now go red.
- footnote.marked.ts: correct the stale module header — it claimed "only
definitions that have a matching reference are emitted", which was never true
(orphan defs are emitted; the editor sync plugin reconciles). Now describes
first-wins + reuse + sync reconciliation.
- derive-id golden test: rename the describe from "(cross-package drift guard)"
to "(deterministic-scheme pin)" — there is no second package to drift against.
editor-ext 129, MCP 304 (+3), client+server tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- footnote-sync: remove the now-dead `refReids` (CollisionPlan field, local,
return, the 6a consumer loop) — references are never re-id'd under reuse, so it
was dead structure on the hot reconciliation path. Rewrite the stale comments
(plugin header, step 0, refOccurrences field) that still described the old
"duplicates re-id'd so both survive" model to the reuse model.
- Shared footnote lexer: new packages/mcp/src/lib/footnote-lex.ts
(lexFootnoteLines + forEachFootnoteReference). extractFootnotes (collaboration)
and analyzeFootnotes now consume the SAME fence-aware lexer, so "the analyzer
sees exactly what the importer keeps/strips" is structural, not comment-kept.
Removed the duplicated DEF_RE/fence machine from both consumers.
- Tests: new mock test for the footnoteWarnings plumbing on createPage (problems
-> field present; clean -> omitted); new paste-reuse case for TWO colliding
pasted definitions (reservation -> distinct ids). Updated the derive-id golden
test header (no MCP copy / parity test anymore).
- CHANGELOG: [Unreleased] entries for footnote reuse (Changed, supersedes 0.93.0)
and footnoteWarnings (Added).
editor-ext 129, MCP 301, server roundtrip 2; client+server tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Footnotes were strict 1:1: a repeated `[^a]` reference was treated as a
collision and re-id'd to `a__2`, and a reference with no definition synthesized
its own empty one — so an agent-authored article with reused labels produced
dozens of empty `kowiki__N` footnotes. Move to Pandoc REUSE semantics and add
non-fatal import diagnostics.
Reuse (core):
- resolveCollisions (footnote-sync): repeated references sharing an id are REUSE
(recorded once in document order, never re-id'd) — one number, one shared
definition. Only a duplicate DEFINITION is re-id'd deterministically and, with
no matching reference, dropped by the existing orphan policy (first-wins).
CollisionPlan.refReids is now always empty (harmless no-op downstream).
- extractFootnoteDefinitions (marked) and extractFootnotes (MCP): duplicate
definition ids are FIRST-WINS (keep first, drop rest); reference markers are
never rewritten. Removed the marker-rewriting and the now-dead deriveFootnoteId
mirror + helpers from the MCP path.
Import diagnostics:
- New analyzeFootnotes() (MCP): fence-aware pure scan reporting dangling
references, empty/duplicate definitions and `[^id]` markers inside table rows.
- createPage / updatePage / importPageMarkdown now attach `footnoteWarnings`
(only when non-empty) so an agent can fix its markup; the page is still created.
Paste-reuse:
- footnotePastePlugin remaps only ids the pasted slice DEFINES (a colliding
definition); a pasted lone reference to an existing id keeps it (reuse).
Tests: reuse/first-wins rewrites of footnote.test, footnote-markdown.test,
footnote.marked.orphan.test and the MCP footnotes.test; new footnote-paste.test
(editor-ext) and footnote-analyze.test (MCP). Deleted derive-id-parity.test.mjs
(the MCP no longer derives ids; editor-ext's deriveFootnoteId keeps its own
golden test). editor-ext 128, MCP 299, server roundtrip 2, client views 3,
client+server tsc clean.
Two review suggestions applied: corrected a stale "duplicated in MCP" comment and
the dangling-reference warning wording.
Note: the multi-backlink editor UI (a reused definition linking back to each of
its references) is deferred to a follow-up — this PR delivers the data-integrity
core (reuse + warnings + paste-reuse). Forward links and numbering already reuse
correctly; the backlink currently targets the first reference.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `subpages` node showed only one level of direct children. Add a `recursive`
attribute that renders the FULL descendant tree of the current page — fully
expanded, unlimited depth. Default `false`, so every previously-inserted node
stays flat (backward compatible). No backend changes: `POST /pages/tree` (via the
`getSpaceTree` wrapper) already returns the whole subtree as a flat `IPage[]`
(recursive CTE, permission-filtered); the nested tree is built on the client by
`parentPageId`.
- editor-ext `subpages.ts`: `recursive` attribute (parse/render `data-recursive`),
shared by client + server so the collab ProseMirror schema keeps the attribute.
- `getSpaceTree`: arg loosened to `{ spaceId?; pageId? }` (the endpoint accepts
either); new `useGetPageTreeQuery(pageId)` react-query hook.
- `subpages-view.tsx`: split into `FlatSubpages` (unchanged) and
`RecursiveSubpages`; `buildSubtree` assembles the nested tree (cycle/self-parent
guard, `sortPositionKeys` per level, root excluded) and a recursive `TreeNode`
renders it (16px indent per depth, soft "showing N" note past 300 — data never
capped). Shared/public context reads the already-nested shared tree, no
`/pages/tree` request.
- toggles: bubble-menu flat⇄tree button + a second slash-menu item "Page tree".
Review follow-ups folded in: invalidate `["page-tree"]` from the create / update /
move / delete cache helpers so an open recursive tree refreshes (no stale data);
mode icon made reactive on editor transactions; `t` threaded into `TreeNode`
(no per-node useTranslation); shared-subtree hook deduped to a thin alias.
editor-ext build + client `tsc --noEmit` both clean. Backend untouched.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review of #157 (Request changes) caught two blockers:
1. DEAD responsive CSS: the `@media (max-width:600px)` float-reset was added to
`image-resize.module.css`, which is imported NOWHERE — the image container's
classes come from `common/node-resize.module.css` (via buildResizeClasses).
So on mobile a floated image kept its px width + float and crushed the text,
exactly the failure the rule promised to prevent. Moved the rule to
`common/node-resize.module.css` (the module actually imported by the resize
node views); its `:global([data-image-align=...])` selectors are data-attr
based, so they work unchanged. Reverted the dead addition from the (pre-existing,
orphaned) image-resize.module.css.
2. `applyAlignment` was untested. Exported it and added `image.spec.ts` (vitest/
jsdom) covering all five align values, the data-image-align mirror, and the
floatLeft -> left reset-then-apply (the guard against a leaked float).
Switched the float writes to the canonical CSSOM `cssFloat` property (portable:
browsers + jsdom; behavior identical to the `.float` alias).
editor-ext build + client tsc clean; 6 image.spec tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds floatLeft / floatRight image alignment so text wraps beside the image,
beyond the existing block left/center/right. Ported from Forkmost PR #7 /
upstream Docmost PR #1132 (fuscodev), adapted to gitmost's imperative image
node-view (the upstream uses a React styled component; ours styles the node-view
container directly via applyAlignment).
- editor-ext image.ts: `setImageAlign` accepts `floatLeft`/`floatRight`;
`applyAlignment` resets float/padding then, for a float mode, sets
`float:left|right` + side padding on the (shrink-to-fit) container so text
flows beside it (the inner <img> already has max-width:100%). The resolved
align is mirrored onto the container as `data-image-align` for the responsive
rule. `data-align` already round-trips the value through parse/renderHTML, so
float survives serialization / collab / history with no schema change.
- image-menu.tsx: Float-left / Float-right bubble-menu buttons (IconFloatLeft/
Right) with active state.
- image-resize.module.css: on narrow screens (<=600px) a floated image collapses
to full width and drops the float (`!important`, keyed on data-image-align) —
the upstream "100% width on small screen" follow-up.
- i18n: en-US + ru-RU strings.
editor-ext build + client tsc --noEmit clean. Visual wrap behavior is best
confirmed in-browser (logic/serialization verified by build + types).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds co-located unit tests for ten targets (client → vitest *.test.ts(x),
server → jest *.spec.ts), plus minimal behavior-preserving extractions/exports
where the issue required a pure function to test:
- encode-wav: WAV header + PCM16 clamping
- editor-ext embed-provider / utils (sanitizeUrl, isInternalFileUrl) / indent
(export clampIndent)
- label.dto @Matches regex
- move-page.dto vs generateJitteredKeyBetween parity (bug locked via test.failing)
- new-note-button canCreatePage (extracted to can-create-page.ts)
- history-editor diff (extracted pure computeHistoryDiff into history-diff.ts)
- notification getTypesForTab + repo contract (direct-tab divergence locked via
test.failing)
- search buildTsQuery (extracted + sanitizes operator inputs so adversarial
queries no longer risk a to_tsquery 500)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a stable native-host JS-API on window.gitmost so the gitmost.app
WKWebView wrapper can hand a recorded audio file to the current page as an
audio block without depending on editor internals (atoms/Tiptap/Yjs).
- page-editor.tsx: register/tear down window.gitmost only while an editable
page editor is mounted (ready=true, version=1). insertRecording validates
mime/size, decodes base64 in 4-char-aligned chunks, rejects oversized
payloads before decoding (too-large), reuses the existing uploadAudioAction
pipeline, resolves machine-readable error codes
(no-editor/bad-type/too-large/insert-failed) and never throws. Cleanup is
identity-guarded so an unmounting PageEditor cannot disable a newer live
registration.
- editor-ext audio-upload: return the uploaded attachment from the upload fn
so the bridge can report success + attachmentId. Backward compatible:
existing fire-and-forget callers ignore the return value; the error path
still swallows and returns undefined (no re-throw).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract parse/renderHtmlEmbedHeight and test: '300'->300, absent->null,
'abc'->null (pins the NaN guard), '120px'->120; render 120->data-height, null/0->{}.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up fixes to the htmlEmbed-sandbox / trackerHead change:
- share-seo: inject trackerHead via a function replacer so `$`-sequences
($&, $', $`, $$) in the admin snippet are inserted literally instead of
being treated as String.replace substitution patterns; warn when the
</head> marker is absent instead of silently skipping injection.
- mcp: register a passthrough `htmlEmbed` node in the schema mirror so an
AI/MCP edit of a page containing an embed no longer throws
"Unknown node type: htmlEmbed" in TiptapTransformer.toYdoc.
- editor-ext + client: treat a non-finite `data-height` as auto (null) so a
crafted/corrupted height cannot disable auto-resize or yield a NaN iframe
height; extract a shared clampHeight helper.
- client: rename render-raw-html.{ts,test.ts} -> html-embed-sandbox.{...} and
shouldExecute -> shouldRender so the seam name matches the sandbox model.
- client: i18n the iframe title; surface the real error reason in
tracker-settings (console.error + err.response.data.message).
- docs: note hasHtmlEmbedNode is now a test-only helper; add an Unreleased
CHANGELOG entry; drop the dangling "arbitrary HTML embed" planning-doc ref.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The encode catch comment promised 'fall back to raw' but the code returns '';
returning raw source wouldn't help anyway (un-encoded markup can't be atob-decoded
downstream, so decode would yield '' regardless), and a raw value in data-source
breaks the inert-storage guarantee. '' is the correct decode-symmetric failure —
fix the misleading comment to say so. Adds a codec test for the encode-throw path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Convert the htmlEmbed node from same-origin raw-HTML execution to a sandboxed
iframe (sandbox="allow-scripts allow-popups allow-forms", no allow-same-origin,
srcdoc) with postMessage auto-resize (validated by event.source) and an optional
manual height attr. The block now runs in an opaque origin and cannot reach the
viewer's cookies/session/API, so it is safe for any member.
Because the block is now harmless, remove the entire admin/role gating apparatus:
drop htmlEmbedAllowed/canAuthorHtmlEmbed/stripDisallowedHtmlEmbedNodes/
collectHtmlEmbedSources and every role-based strip on the write paths (collab
REST/MCP + socket, page create/duplicate, import x2, transclusion unsync), along
with the now-unused WorkspaceRepo/UserRepo injections and the PageService.create
callerRole param. Keep one strip: prepareContentForShare still removes htmlEmbed
on the anonymous public-share read path when the workspace master toggle is OFF.
The workspace settings.htmlEmbed toggle is now a plain feature switch (gates the
slash-menu and share rendering); when ON the block is available to all members.
Add settings.trackerHead: an admin-only raw HTML/JS analytics snippet injected
verbatim into the <head> of public share pages only (ShareSeoController), for
trackers that genuinely need same-origin. Admin-gated via the existing CASL
Manage/Settings ability; never injected into the authenticated app shell.
Closes security-review findings #1, #2, #4, #5, #10 (and #3 as a security issue).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add ~330 tests across server (Jest), client (Vitest), editor-ext (Vitest)
and packages/mcp (node:test) for the gitmost features added since
053a9c0d: AI chat, AI agent roles, public-share assistant, MCP per-user
auth, HTML embed, page templates/embed, realtime tree, tree
expand/collapse, and the AI-settings UI.
Test-tooling fixes (prerequisite, were silently hiding coverage):
- Repair 3 page-template specs broken by the 11-arg TransclusionService
constructor; they never compiled, so template access-control / content
-leak / unsync-strip coverage was fictitious.
- Build @docmost/editor-ext before server tests via a `pretest` hook;
the stale dist omitted the new HtmlEmbed/PageEmbed exports (TS2305).
- Let jest resolve the .tsx email templates: add `tsx` to
moduleFileExtensions and widen the ts-jest transform to (t|j)sx?.
Behaviour-preserving "extract pure core" refactors that the tests drive:
- server: resolveShareAssistantRequest + uiMessageTextLength
(public-share controller), decideBasicGate + mapAuthResultToResponse
(mcp), buildErrorAssistantRecord (ai-chat), jsonbObject export (roles).
- client: render-raw-html + shouldExecute/canEdit, decide-embed-state,
page-embed picker utils, tree-socket reducers, open/close branch maps,
isEndpointConfigured/resolveKeyField; buildTreeWithChildren now treats
a permission-trimmed orphan as a root instead of crashing.
Deferred (need a test DB or HTTP harness, documented in the specs):
repo-level Postgres integration tests and the public-share XFF E2E.
Pre-existing DI/lib0-ESM suite failures are untouched and out of scope.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve conflicts at shared registration points by unioning both features
(footnotes + the already-merged html-embed / page-embed work):
- slash-menu/menu-items.ts, editor extensions.ts: keep both imports + configures
- collaboration.util.ts: register footnote nodes and pageEmbed
- editor-ext marked.utils.ts: register footnote + html-embed markdown extensions
- editor-ext package.json/tsconfig.json/vitest.config.ts: union of test config
(jsdom env for footnote DOM tests + combined test/spec include glob)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Release-cycle review found two hardening gaps:
- The sync plugin deleted+rebuilt the WHOLE footnotesList on any reorder/orphan,
replacing every definition's Yjs subtree -> a collaborator typing in a
definition could lose in-flight characters on merge. Rework to targeted,
minimal mutations: attr-only setNodeMarkup for collision re-ids, delete only
genuine orphans, insert only genuinely-missing definitions (at the list end,
not shifting existing subtrees), and consolidate multiple lists only in the
abnormal paste/merge case. An unchanged (correct id, referenced) definition is
left completely untouched. Numbering is decoration-only, so physical list order
may drift after a reorder (accepted) while displayed numbers stay correct.
Invariants preserved (reviewed + tested): one SYNC_META transaction, null when
canonical (terminates), deterministic deriveFootnoteId, remote-skip -> no
re-introduced freeze or divergence.
- computeFootnoteNumbers ran per-NodeView-render (O(n^2)/keystroke in big docs).
The numbering plugin now caches the number map in its state (computed once per
docChanged); NodeViews read it O(1) via getFootnoteNumber.
Tests: no-rebuild-on-reorder asserts unchanged definition node subtrees are
identity-preserved; isRemoteTransaction skip; enableSync:false read-only; cache
correctness. Browser re-smoke: insert (no freeze), number, persist across reload,
cascade delete all pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Release-cycle test audit: the strip boundary was tested only via a stand-in
helper re-implemented in the spec, so a deleted/misplaced guard kept CI green
(the missing create() guard was proof). Replace it with tests against real code:
- persistence.extension.onStoreDocument: real ydoc from a rich doc (columns/
table/mention/htmlEmbed) -> non-admin strip removes only htmlEmbed, every other
node preserved (data-loss guard); admin keeps; empty fragment no-throw.
- collaboration.handler.updatePageContent: real path, user?.role gate, decoded
ydoc embed-free for non-admin, kept for admin.
- transclusion unsync: member stripped, admin preserved.
- editor-ext gains a vitest setup (was zero tests) + a markdown round-trip:
the <!--html-embed:BASE64--> marker -> htmlEmbed node with decoded source, and
hasHtmlEmbedNode matches it — pinning the marked/turndown shape the import
strip relies on. tsconfig now excludes specs from the shipped dist.
- Fail-closed identity: source-pinned contracts that the gate keys on
fileTask.creatorId (zip) / request userId (single) / callerRole (create) /
authUser.role (duplicate), and missing-user -> strip (services can't load under
jest's ESM graph; helpers replay the exact predicate).
Adds the verified-safe ^src/ jest moduleNameMapper (identical fail set).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Release-cycle red-team found two same-id footnoteDefinition nodes (trivially
produced by markdown import [^d]: first / [^d]: second, or paste/duplicate)
caused silent data loss: scan() used a last-wins Map and the sync rebuild
(addToHistory:false, propagated via Yjs, un-undoable) dropped all but the last.
Fix resolves collisions so BOTH survive, with a DETERMINISTIC id scheme so
collaborators converge:
- deriveFootnoteId(originalId, occurrence, taken): the k-th (k>=2) occurrence of
id X becomes X__k, bumped with a deterministic alpha suffix only against the
doc's own id set — a pure function of document state. No Math.random/Date.now
on the sync or import paths (random uuid stays only in setFootnote, where a
single user originates a brand-new id).
- footnote-sync.resolveCollisions walks refs+defs in document order, re-ids
duplicate references via setNodeMarkup and pairs them 1:1 with definitions;
single SYNC_META-tagged transaction, returns null when canonical (terminates).
- Markdown import (footnote.marked) + MCP mirror (collaboration.ts) dedup with
the same deterministic scheme + marker rewrite; packages/mcp/build regenerated.
- Paste plugin remaps colliding pasted ids against the current doc.
Tests: two independent editors resolving the same duplicate-id doc produce
IDENTICAL ids (the cross-client determinism guard that the random version would
fail); both definitions survive the first edit; import dedup is deterministic.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds footnotes: a superscript marker in the text linked to an editable
definition in a Footnotes section at the end of the page, with auto-numbering
and a read-only hover popover. Chose the reference+definitions model (3 plain
nodes) over an inline atom with a sub-editor specifically for collaboration
safety.
editor-ext (packages/editor-ext/src/lib/footnote/):
- footnoteReference (inline atom, id), footnotesList (block, last child),
footnoteDefinition (paragraph+, id). renderHTML emits sup[data-footnote-ref]
/ section[data-footnotes] / div[data-footnote-def]; parse-rule priority makes
the empty reference win over the Superscript mark (else it is dropped on the
server save).
- numbering: a decoration-only plugin (pure function of doc order) -> every
client computes identical numbers, no document mutation, Yjs-safe.
- sync plugin: single-pass, always SYNC_META-tagged and skipping remote txns
(terminates, no loop), idempotent; canonicalizes to one trailing footnotesList
(merging duplicates), creates missing definitions, drops orphans, and
coexists with TrailingNode. Disabled in read-only.
- commands setFootnote (one tx: reference + definition at the matching index +
focus) / removeFootnote (cascade, one undo) / scrollTo*. slash /footnote.
client: superscript NodeView + floating-ui read-only popover; bottom-list and
definition NodeViews; registered in mainExtensions.
server: the three nodes registered in tiptapExtensions so collab/save/export
keep them. Round-trip regression spec guards the Superscript parse-priority.
markdown: turndown/marked round-trip to pandoc/GFM [^id] (+ a code-fence guard
so footnote-like lines inside code blocks are not extracted).
MCP mirror: schema + markdown-converter + commentsToFootnotes rewritten to real
footnote nodes + diff marker counting; NUL sentinels written as \u0000 escapes.
v2 follow-ups (per plan): definition reordering on reference move, id-collision
regeneration on paste, multiple references to one footnote.
Implements docs/footnotes-plan.md (variant B).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Embed another page's LIVE content into a host page (it updates when the source
changes, not a static copy). A page can be flagged a template for discovery in
the picker; any accessible page can be embedded.
Server:
- migrations: pages.is_template (+ partial index) and page_template_references
(whole-page back-refs); db.d.ts/entity types hand-merged (db.d.ts is curated).
- POST /pages/toggle-template (CASL Edit) flips is_template; is_template is
returned by findById + the sidebar tree select so the tree menu label
reflects state. Search suggestions gain an onlyTemplates filter for the picker.
- POST /pages/template/lookup ({sourcePageIds[]}, <=50): returns each accessible
source's {title, icon, slugId, content, sourceUpdatedAt} with comment marks
stripped (same access path as transclusion: filterViewerAccessiblePageIds;
inaccessible -> no_access, missing -> not_found; error path -> not_found, never
raw content).
- reference sync (collectPageEmbedsFromPmJson + syncPageTemplateReferences) on
the Yjs save hook; duplicatePage remaps pageEmbed.sourcePageId + inserts refs.
Known MVP gap: REST content updates don't resync refs (lookup uses in-doc ids).
Client:
- pageEmbed node (editor-ext, registered in BOTH client + server schemas);
read-only NodeView with a batching lookup; '/Embed page' slash + template
picker (self-embed prevented); 'Make/Unset template' in the tree node menu.
- Cycle guard: an ancestry-chain context + depth cap (5) render a 'circular
embed' placeholder instead of recursing.
- Public shares show a placeholder (no public lookup in MVP).
MVP excludes (follow-ups): public-share lookup, unsync->static copy, server-side
expansion for export/RAG, MCP schema mirror, point-in-time snapshots.
Implements docs/page-templates-plan.md (MVP, variant A).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an htmlEmbed block node that renders and executes raw HTML/CSS/JS in the
wiki origin (e.g. an analytics tracker) — the owner-chosen variant C. Because
this is stored-XSS by design, only workspace admins/owners may get such a node
persisted; everyone executes it when reading.
- Node (editor-ext): htmlEmbed atom/isolating block; source stored base64 in
data-source for lossless HTML<->JSON round-trip. renderHTML emits only the
encoded marker (never inlines raw markup), so generateHTML/export/search are
not themselves injection vectors. Registered in BOTH client extensions and
server tiptapExtensions. Markdown round-trip via an <!--html-embed:b64-->
comment (turndown) + a marked rule.
- Client NodeView: injects source and re-creates <script> elements so they
actually run; edit modal; renders in read-only/share too. Slash item is
admin-gated (adminOnly filtered by the user's workspace role).
- SERVER ENFORCEMENT (the real control — UI gating alone is insufficient):
stripHtmlEmbedNodes() removes htmlEmbed from any document persisted by a
non-admin, applied at every write path that introduces content from an
untrusted author: collab onStoreDocument, REST/MCP/AI updatePageContent,
single-file import, zip/multi-file import, page duplication, and transclusion
unsync. Page restore introduces no new content. Public share/readonly viewers
render fetched (already-stripped) content and do NOT open a collab socket, so
the only residual is a transient broadcast window to concurrent authenticated
editors (documented).
Implements docs/arbitrary-html-embed-plan.md (variant C).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat(editor): add alt text support for images
* feat: extend alt text support to videos and diagrams
---------
Co-authored-by: Philipinho <16838612+Philipinho@users.noreply.github.com>
* autojoiner
* fix marked
* return clipboardTextSerializer as markdown
* fix clipboardTextSerializer for single lines
* cleanup two preceeding spaces in ordered lists item
* fix extra paragraph in task list
* don't zip sinple page exports
