vvzvlad/gitmost
1
0
Fork 0
Code Issues 31 Pull Requests 15 Packages Projects Releases Activity

fix(git-sync): red-team hardening — 12 confirmed sync-breaking bugs + regression tests

Browse Source 
A 10-agent red-team pass on the two-way Docmost<->git sync surfaced 16 ranked
findings (9 others triaged out as already-defended). Wrote a reproduction test
per finding (each asserts the CORRECT behavior, so it fails on the bug), then
fixed the production code so every repro goes green. All confirmed bugs:

Round-trip data loss (markdown-converter.ts + docmost-schema.ts mirror):
- #1 editor-ext node types silently dropped on export — ported the 8 missing
  canon nodes (footnoteReference/footnotesList/footnoteDefinition, htmlEmbed,
  status, pageEmbed, transclusionSource/Reference) into the git-sync schema
  mirror and added converter cases that emit their schema-matching HTML instead
  of flattening unknown nodes to '' (this was the critical data-loss flagged in
  review #1679: footnotes/htmlEmbed lost on sync). Snapshot surface updated.
- #2 top-level image lost width/height/align/attachmentId — now emits an HTML
  <img> (like video/diagrams) when it carries layout attrs; bare images stay
  ![](src). Image node parses width/height as strings so they re-import.
- #3 code block containing a ``` fence corrupted on round-trip — outer fence is
  now widened to (longest-inner-backtick-run + 1).
- #16 deep nesting threw RangeError (page never synced) — added a depth guard
  (MAX_NODE_DEPTH=400) so the converter never overflows the stack.

Push/layout/cycle (engine):
- #4 disambiguation ' ~slugId' suffix corrupted Docmost titles + order-dependent
  layout — deterministic, order-independent sibling disambiguation; suffix is
  stripped from a path-derived title ONLY when the new name is exactly the old
  title plus the suffix (never a genuine retitle ending in ' ~token').
- #6 retry-adopt by (parent,title) clobbered the wrong duplicate-title sibling —
  ambiguous (parent,title) is no longer adopted (falls back to fresh create).
- #12 a new child under a new parent was created at ROOT — creates are ordered
  parent-before-child with an in-memory created-id map for parent resolution.
- #13 git conflict markers could reach Docmost — bodies are scanned and the
  marker lines stripped (a '=======' line is only treated as a conflict
  separator inside a <<<<<<< ... >>>>>>> block, so setext headings are safe).
- #15 a divergent `docmost` mirror was escalated by runPush but dropped by
  runCycle — RunCycleResult now forwards divergentDocmost to the orchestrator.

Server (merge / lock / provenance):
- #9 3-way merge lost a human's block edit when git inserted an adjacent block —
  finer-grained diff3 region merge (via lcs) preserves non-overlapping human
  edits; genuine same-block conflicts still resolve git-wins.
- #10 single-writer race — module-static liveLocks closes the same-process TOCTOU
  window, and a heartbeat refresh that cannot confirm the lock now aborts the
  cycle at its next write checkpoint (cooperative AbortSignal threaded through
  runCycle). Cross-process fencing tokens remain a follow-up.
- #14 sticky-agent provenance overrode an explicit actor='git-sync' write,
  blinding the listener loop-guard — resolveSource now lets an explicit actor
  win over the sticky-agent fallback (explicit agent still wins).

Verified: git-sync vitest 617 pass (+1 expected-fail), server unit jest 1541
pass, server tsc clean. A review pass over the fixes caught and corrected a
title-suffix over-strip, an inert abort signal, a document-wide conflict-marker
strip, and two leaf-atom content-holes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
claude code agent 227
2026-06-26 01:29:02 +03:00
parent b536a41ad3
commit d5079aa1d8
20 changed files with 1621 additions and 135 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/git-sync/src/lib/docmost-schema.ts
+355 -4
View File
@@ -204,11 +204,38 @@ const DocmostAttributes = Extension.create({
types: ["image"],
attributes: {
align: { default: null },
attachmentId: { default: null },
aspectRatio: { default: null },
// imageToHtml emits these Docmost-specific image attrs as data-*; map
// them back explicitly so a top-level image (or one inside a column)
// round-trips them. Without a parseHTML the default reads the bare
// attribute name (e.g. getAttribute("attachmentId") -> null) and the
// value — including the attachmentId that links the image to its
// stored file — is silently dropped on every round-trip (data loss).
attachmentId: {
default: null,
parseHTML: (el: HTMLElement) =>
el.getAttribute("data-attachment-id"),
renderHTML: (attrs: Record<string, any>) =>
attrs.attachmentId
? { "data-attachment-id": attrs.attachmentId }
: {},
},
aspectRatio: {
default: null,
parseHTML: (el: HTMLElement) =>
el.getAttribute("data-aspect-ratio"),
renderHTML: (attrs: Record<string, any>) =>
attrs.aspectRatio != null
? { "data-aspect-ratio": attrs.aspectRatio }
: {},
},
height: { default: null },
placeholder: { default: null },
size: { default: null },
size: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-size"),
renderHTML: (attrs: Record<string, any>) =>
attrs.size != null ? { "data-size": attrs.size } : {},
},
width: { default: null },
},
},
@@ -1030,6 +1057,300 @@ const PageBreak = Node.create({
},
});
/**
* Footnote feature (mirror of @docmost/editor-ext footnote, matching the MCP
* schema mirror). Three nodes connected by `id`:
* - FootnoteReference: inline atom marker in the body (<sup data-footnote-ref>);
* - FootnotesList: a single bottom container (<section data-footnotes>);
* - FootnoteDefinition: one editable note keyed by id (<div data-footnote-def>).
* The visible number is not stored; it is derived from reference order. The
* <sup> parse rule uses priority 100 so it beats the Superscript mark's <sup>
* rule (otherwise an empty reference parses as an empty superscript and drops).
*/
const FootnoteReference = Node.create({
name: "footnoteReference",
priority: 101,
group: "inline",
inline: true,
atom: true,
selectable: true,
draggable: false,
addAttributes() {
return {
id: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-id"),
renderHTML: (attrs: Record<string, any>) =>
attrs.id ? { "data-id": attrs.id } : {},
},
};
},
parseHTML() {
return [{ tag: "sup[data-footnote-ref]", priority: 100 }];
},
renderHTML({ HTMLAttributes }) {
return ["sup", { "data-footnote-ref": "", ...HTMLAttributes }];
},
});
const FootnotesList = Node.create({
name: "footnotesList",
group: "block",
content: "footnoteDefinition+",
isolating: true,
selectable: false,
defining: true,
parseHTML() {
return [{ tag: "section[data-footnotes]" }];
},
renderHTML({ HTMLAttributes }) {
return ["section", { "data-footnotes": "", ...HTMLAttributes }, 0];
},
});
const FootnoteDefinition = Node.create({
name: "footnoteDefinition",
content: "paragraph+",
defining: true,
isolating: true,
selectable: false,
addAttributes() {
return {
id: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-id"),
renderHTML: (attrs: Record<string, any>) =>
attrs.id ? { "data-id": attrs.id } : {},
},
};
},
parseHTML() {
return [{ tag: "div[data-footnote-def]" }];
},
renderHTML({ HTMLAttributes }) {
return ["div", { "data-footnote-def": "", ...HTMLAttributes }, 0];
},
});
/**
* Encode/decode the htmlEmbed `source` (arbitrary HTML/CSS/JS) to/from base64
* for the `data-source` attribute. Ported from @docmost/editor-ext so the
* markdown-converter HTML path (generateJSON via parseHTML) round-trips the
* raw source losslessly and keeps it inert while it sits in the attribute.
* `encodeURIComponent`/`decodeURIComponent` wrap btoa/atob so UTF-8 survives.
*/
export function encodeHtmlEmbedSource(source: string): string {
if (!source) return "";
try {
if (typeof btoa === "function") {
return btoa(encodeURIComponent(source));
}
return Buffer.from(encodeURIComponent(source), "utf-8").toString("base64");
} catch {
return "";
}
}
export function decodeHtmlEmbedSource(encoded: string): string {
if (!encoded) return "";
try {
if (typeof atob === "function") {
return decodeURIComponent(atob(encoded));
}
return decodeURIComponent(Buffer.from(encoded, "base64").toString("utf-8"));
} catch {
return "";
}
}
/**
* Docmost raw HTML embed. Block atom; the client renders `source` inside a
* sandboxed iframe. Mirrors the @docmost/editor-ext node — `source` rides the
* `data-source` attribute base64-encoded (this is an HTML/generateJSON path, so
* it MUST use base64 to avoid double-encoding / injection).
*/
const HtmlEmbed = Node.create({
name: "htmlEmbed",
group: "block",
inline: false,
isolating: true,
atom: true,
defining: true,
draggable: true,
addAttributes() {
return {
source: {
default: "",
parseHTML: (el: HTMLElement) =>
decodeHtmlEmbedSource(el.getAttribute("data-source") || ""),
renderHTML: (attrs: Record<string, any>) => ({
"data-source": encodeHtmlEmbedSource(attrs.source || ""),
}),
},
height: {
default: null,
parseHTML: (el: HTMLElement) => {
const v = el.getAttribute("data-height");
if (!v) return null;
const n = parseInt(v, 10);
return Number.isFinite(n) ? n : null;
},
renderHTML: (attrs: Record<string, any>) =>
attrs.height != null ? { "data-height": String(attrs.height) } : {},
},
};
},
parseHTML() {
return [{ tag: 'div[data-type="htmlEmbed"]' }];
},
renderHTML({ HTMLAttributes }) {
return ["div", { "data-type": "htmlEmbed", ...HTMLAttributes }];
},
});
/**
* Inline status pill. Mirrors @docmost/editor-ext status: the label rides in
* the element's TEXT content (not an attribute) and the color in data-color.
*/
const Status = Node.create({
name: "status",
group: "inline",
inline: true,
atom: true,
selectable: true,
draggable: true,
addAttributes() {
return {
text: {
default: "",
parseHTML: (el: HTMLElement) => el.textContent || "",
},
color: {
default: "gray",
parseHTML: (el: HTMLElement) => el.getAttribute("data-color") || "gray",
renderHTML: (attrs: Record<string, any>) => ({
"data-color": attrs.color ?? "gray",
}),
},
};
},
parseHTML() {
return [{ tag: 'span[data-type="status"]' }];
},
renderHTML({ HTMLAttributes }) {
return [
"span",
{ "data-type": "status", "data-color": HTMLAttributes["data-color"] },
`${HTMLAttributes.text ?? ""}`,
];
},
});
/**
* Whole-page live embed. Holds only a `sourcePageId` reference. Mirrors
* @docmost/editor-ext pageEmbed. Block atom.
*/
const PageEmbed = Node.create({
name: "pageEmbed",
group: "block",
atom: true,
isolating: true,
selectable: true,
draggable: true,
addAttributes() {
return {
sourcePageId: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-source-page-id"),
renderHTML: (attrs: Record<string, any>) =>
attrs.sourcePageId
? { "data-source-page-id": attrs.sourcePageId }
: {},
},
};
},
parseHTML() {
return [{ tag: 'div[data-type="pageEmbed"]' }];
},
renderHTML({ HTMLAttributes }) {
return ["div", { "data-type": "pageEmbed", ...HTMLAttributes }];
},
});
/**
* Block node types allowed inside a `transclusionSource` (mirrors
* @docmost/editor-ext transclusion constants). Excludes transclusion nodes
* (no nesting) and child-only nodes.
*/
const TRANSCLUSION_SOURCE_CONTENT_EXPRESSION =
"(paragraph | heading | blockquote | codeBlock | horizontalRule | bulletList" +
" | orderedList | taskList | image | video | audio | attachment | callout" +
" | details | embed | mathBlock | table | drawio | excalidraw | pdf" +
" | subpages | columns | youtube)+";
/** Sync-source block: editable content shared into transclusion references. */
const TransclusionSource = Node.create({
name: "transclusionSource",
group: "block",
content: TRANSCLUSION_SOURCE_CONTENT_EXPRESSION,
defining: true,
isolating: true,
addAttributes() {
return {
id: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-id"),
renderHTML: (attrs: Record<string, any>) =>
attrs.id ? { "data-id": attrs.id } : {},
},
};
},
parseHTML() {
return [{ tag: 'div[data-type="transclusionSource"]' }];
},
renderHTML({ HTMLAttributes }) {
return ["div", { "data-type": "transclusionSource", ...HTMLAttributes }, 0];
},
});
/** Live reference to a transcluded block/page. Block atom. */
const TransclusionReference = Node.create({
name: "transclusionReference",
group: "block",
atom: true,
selectable: true,
draggable: false,
addAttributes() {
return {
sourcePageId: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-source-page-id"),
renderHTML: (attrs: Record<string, any>) =>
attrs.sourcePageId
? { "data-source-page-id": attrs.sourcePageId }
: {},
},
transclusionId: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-transclusion-id"),
renderHTML: (attrs: Record<string, any>) =>
attrs.transclusionId
? { "data-transclusion-id": attrs.transclusionId }
: {},
},
};
},
parseHTML() {
return [{ tag: 'div[data-type="transclusionReference"]' }];
},
renderHTML({ HTMLAttributes }) {
return [
"div",
{ "data-type": "transclusionReference", ...HTMLAttributes },
];
},
});
/**
* Full extension list. Image is block-level (matches Docmost); the
* ProseMirror DOM parser hoists <img> found inside <p> automatically.
@@ -1041,7 +1362,29 @@ export const docmostExtensions = [
heading: {},
link: { openOnClick: false },
}),
Image.configure({ inline: false }),
// Preserve image width/height as the AUTHORED string. Without an explicit
// parseHTML the stock Image node attribute falls back to tiptap core's
// `fromString`, which coerces a numeric width like "320" into the number 320
// — changing the stored type on every markdown round-trip (Docmost stores
// these as strings, e.g. "320" or "50%", matching how video/audio/pdf are
// handled in this mirror). The node attribute is applied AFTER the global
// DocmostAttributes one, so the fix must live on the Image node itself.
Image.extend({
addAttributes() {
const parent = (this.parent?.() ?? {}) as Record<string, any>;
return {
...parent,
width: {
...parent.width,
parseHTML: (el: HTMLElement) => el.getAttribute("width"),
},
height: {
...parent.height,
parseHTML: (el: HTMLElement) => el.getAttribute("height"),
},
};
},
}).configure({ inline: false }),
TaskList,
TaskItem.configure({ nested: true }),
// Highlight stores its color unescaped and Docmost interpolates it into
@@ -1094,5 +1437,13 @@ export const docmostExtensions = [
Audio,
Pdf,
PageBreak,
FootnoteReference,
FootnotesList,
FootnoteDefinition,
HtmlEmbed,
Status,
PageEmbed,
TransclusionSource,
TransclusionReference,
DocmostAttributes,
];
packages/git-sync/src/lib/markdown-converter.ts
+157 -4
View File
@@ -1,3 +1,18 @@
import { encodeHtmlEmbedSource } from "./docmost-schema.js";
/**
* Hard cap on processNode recursion depth (see the depth guard below).
*
* Chosen well above any realistic document (the deepest legitimate nesting the
* editor can produce is far shallower) yet far below the point where the
* converter's own call stack overflows. The heaviest shape (deeply nested
* lists) costs ~5 JS frames per level and the runtime stack holds ~10k frames,
* so the measured overflow is around level ~650 (deeply nested lists); 400
* leaves a comfortable margin while still rendering pathological-but-bounded
* docs in full (the 200-level stress fixture reaches depth ~204).
*/
const MAX_NODE_DEPTH = 400;
/**
* Convert ProseMirror/TipTap JSON content to Markdown
* Supports all Docmost-specific node types and extensions
@@ -43,7 +58,34 @@ export function convertProseMirrorToMarkdown(content: any): string {
.replace(/\(/g, "%28")
.replace(/\)/g, "%29");
// Recursion depth guard. processNode is mutually recursive (directly and via
// processListItem/processTaskItem/blockToHtml), and a pathologically nested
// document (e.g. tens of thousands of nested blockquotes) would otherwise
// overflow the call stack and throw a RangeError, which would abort the sync
// and prevent the page from ever being written. We track the live nesting
// depth in a closure counter (the wrapper below) so we NEVER throw: past the
// limit we stop recursing and emit the node's own text (or nothing) instead.
// Normal documents never approach MAX_NODE_DEPTH, so their output is byte-
// identical. NOTE: the wrapper signature is (node) only — several callers use
// `.map(processNode)`, which would otherwise pass the array index as a second
// argument; the wrapper ignores extra arguments so that is harmless.
let nodeDepth = 0;
const processNode = (node: any): string => {
if (nodeDepth >= MAX_NODE_DEPTH) {
// Bail out of deeper recursion without throwing. A text node still has
// its own content worth keeping; a container at the limit collapses to
// "" (its already-too-deep subtree is dropped) rather than overflowing.
return typeof node?.text === "string" ? node.text : "";
}
nodeDepth++;
try {
return processNodeInner(node);
} finally {
nodeDepth--;
}
};
const processNodeInner = (node: any): string => {
const type = node.type;
const nodeContent = node.content || [];
@@ -182,7 +224,16 @@ export function convertProseMirrorToMarkdown(content: any): string {
.map(processNode)
.join("")
.replace(/\n+$/, "");
return "```" + language + "\n" + code + "\n```";
// CommonMark: an inner ``` run inside the code would prematurely close
// a 3-backtick fence (corrupting the block on re-import). Use an outer
// fence one backtick longer than the longest backtick run in the code
// (minimum 3) so the inner fence is always content.
const longestBacktickRun = (code.match(/`+/g) || []).reduce(
(max: number, run: string) => Math.max(max, run.length),
0,
);
const fence = "`".repeat(Math.max(3, longestBacktickRun + 1));
return fence + language + "\n" + code + "\n" + fence;
case "bulletList":
return nodeContent
@@ -228,16 +279,35 @@ export function convertProseMirrorToMarkdown(content: any): string {
// a bare "\n" would be reimported as a soft break and lost.
return " \n";
case "image":
const imgAlt = node.attrs?.alt || "";
case "image": {
const imgAttrs = node.attrs || {};
// A top-level image with layout/identity attrs beyond src/alt cannot be
// expressed by markdown `![](src)` — width/height/align/size/
// attachmentId/aspectRatio would be silently dropped on export and lost
// on re-import. Emit the SAME schema-matching <img> used inside columns
// (imageToHtml) so those attrs survive the round-trip. A bare image
// (only src/alt, optionally a title — which has no schema attr) keeps
// the lighter markdown form so existing image round-trip tests hold.
const hasLayoutAttrs =
imgAttrs.width != null ||
imgAttrs.height != null ||
imgAttrs.align ||
imgAttrs.size != null ||
imgAttrs.attachmentId ||
imgAttrs.aspectRatio != null;
if (hasLayoutAttrs) {
return imageToHtml(node);
}
const imgAlt = imgAttrs.alt || "";
// Neutralize characters that could break out of the markdown image
// URL: spaces/newlines and parentheses would terminate the (...) target
// and let a stored src inject following markdown/HTML. Percent-encode
// them so the URL stays a single inert token.
const imgSrc = encodeMdUrl(node.attrs?.src);
const imgSrc = encodeMdUrl(imgAttrs.src);
// No "caption" attribute exists in the Docmost image schema, so we do
// not emit one (the previous caption branch was dead).
return `![${imgAlt}](${imgSrc})`;
}
case "video": {
// Emit the schema-matching <video> element so generateJSON rebuilds the
@@ -581,6 +651,83 @@ export function convertProseMirrorToMarkdown(content: any): string {
case "subpages":
return "{{SUBPAGES}}";
case "status": {
// Inline status pill. The schema reads the label from the element's
// TEXT content and the color from data-color, so emit both; without a
// case this inline atom fell through to `default` and collapsed to "".
const attrs = node.attrs || {};
const statusColor = attrs.color || "gray";
return `<span data-type="status" data-color="${escapeAttr(statusColor)}">${escapeHtmlText(attrs.text ?? "")}</span>`;
}
case "htmlEmbed": {
// Block atom; the schema reads the raw source from a base64-encoded
// data-source attribute (and an optional fixed height from data-height).
// Encode with the shared helper so it decodes symmetrically on import.
const attrs = node.attrs || {};
const parts: string[] = [
`data-type="htmlEmbed"`,
`data-source="${escapeAttr(encodeHtmlEmbedSource(attrs.source ?? ""))}"`,
];
if (attrs.height != null)
parts.push(`data-height="${escapeAttr(attrs.height)}"`);
return `<div ${parts.join(" ")}></div>`;
}
case "footnoteReference": {
// Inline atom marker. The schema reads its id from data-id on a
// sup[data-footnote-ref]; the visible number is derived, not stored.
const attrs = node.attrs || {};
const idAttr = attrs.id ? ` data-id="${escapeAttr(attrs.id)}"` : "";
return `<sup data-footnote-ref${idAttr}></sup>`;
}
case "footnotesList": {
// Bottom container of footnote definitions (section[data-footnotes]).
const inner = nodeContent.map((n: any) => blockToHtml(n)).join("");
return `<section data-footnotes>${inner}</section>`;
}
case "footnoteDefinition": {
// One footnote note keyed by id (div[data-footnote-def]).
const attrs = node.attrs || {};
const idAttr = attrs.id ? ` data-id="${escapeAttr(attrs.id)}"` : "";
const inner = nodeContent.map((n: any) => blockToHtml(n)).join("");
return `<div data-footnote-def${idAttr}>${inner}</div>`;
}
case "pageEmbed": {
// Whole-page live embed; the schema reads data-source-page-id.
const attrs = node.attrs || {};
const parts: string[] = [`data-type="pageEmbed"`];
if (attrs.sourcePageId)
parts.push(`data-source-page-id="${escapeAttr(attrs.sourcePageId)}"`);
return `<div ${parts.join(" ")}></div>`;
}
case "transclusionReference": {
// Live reference to a transcluded block/page. Block atom; the schema
// reads data-source-page-id and data-transclusion-id.
const attrs = node.attrs || {};
const parts: string[] = [`data-type="transclusionReference"`];
if (attrs.sourcePageId)
parts.push(`data-source-page-id="${escapeAttr(attrs.sourcePageId)}"`);
if (attrs.transclusionId)
parts.push(
`data-transclusion-id="${escapeAttr(attrs.transclusionId)}"`,
);
return `<div ${parts.join(" ")}></div>`;
}
case "transclusionSource": {
// Sync-source container; the schema reads data-id and re-parses its
// block children, so render them as schema-matching HTML.
const attrs = node.attrs || {};
const idAttr = attrs.id ? ` data-id="${escapeAttr(attrs.id)}"` : "";
const inner = nodeContent.map((n: any) => blockToHtml(n)).join("");
return `<div data-type="transclusionSource"${idAttr}>${inner}</div>`;
}
default:
// Fallback: process children
return nodeContent.map(processNode).join("");
@@ -782,6 +929,12 @@ export function convertProseMirrorToMarkdown(content: any): string {
case "attachment":
case "drawio":
case "excalidraw":
case "htmlEmbed":
case "footnotesList":
case "footnoteDefinition":
case "pageEmbed":
case "transclusionSource":
case "transclusionReference":
return processNode(block);
default:
// Any still-unhandled block type: NEVER fall back to markdown inside a