build(git-sync): land the @docmost/git-sync package into develop, code-only (#326 step 1 / PR-A)
The git-sync converter + engine source lived only on the #119 branch; develop had just the dead compiled build/. Bring the whole package (src + ~700 tests) onto develop under CI, with NO consumer wired — git-sync stays fully inert in develop (nothing in apps/server imports it), so runtime behavior is unchanged. This unblocks #293 (extract the shared converter package from the landed source) and lets #119's functionality land LAST, already writing the canonical format (per the #326 landing order). - packages/git-sync: src (lib converter + engine) + test corpus + configs. - Remove develop's dead committed packages/git-sync/build/; gitignore it (built in CI/Docker via pnpm build, never committed — no src/build drift). - pnpm-lock.yaml: add the @docmost/git-sync importer (a missing workspace package in the lock is a CI blocker). `pnpm install --frozen-lockfile` passes. - NO server integration / loader / Dockerfile runtime changes (those come with #119 at step 6). Verified: tsc clean; vitest 711 passed | 1 expected-fail, 0 failures, 0 type errors; pnpm --frozen-lockfile EXIT 0; apps/server has no git-sync import. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
claude code agent 227
@@ -0,0 +1,247 @@
|
||||
/**
|
||||
* Semantic canonicalization of ProseMirror/TipTap documents for the round-trip
|
||||
* idempotency check (SPEC §11, "Task #0", option (b): compare a CANONICALIZED
|
||||
* form rather than raw bytes).
|
||||
*
|
||||
* `markdownToProseMirror` reconstructs schema DEFAULT attributes (e.g.
|
||||
* `indent: null` where the source omitted it) and regenerates per-block ids on
|
||||
* every import. A raw deep-equal of the source doc against the re-imported doc
|
||||
* therefore diverges even when the two are semantically identical. This module
|
||||
* normalizes a document so that two semantically-equal docs compare deep-equal
|
||||
* regardless of block ids and absent-vs-explicit-default-null attributes.
|
||||
*
|
||||
* It is a self-contained module with no external dependencies.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Known NON-NULL schema defaults that `markdownToProseMirror` materializes on
|
||||
* import, keyed by node/mark type → { attr: defaultValue }.
|
||||
*
|
||||
* Why this exists: `canonicalizeAttrs` already treats an absent attr as
|
||||
* equivalent to an explicit `null`/`undefined`. But several Docmost schema
|
||||
* attributes default to a NON-null value, so import fills them in even when the
|
||||
* source omitted them — making "attr absent" diverge from "attr at its default
|
||||
* value" under a raw deep-equal. To keep "absent ≡ explicit-default", we ALSO
|
||||
* drop any attr whose value equals its known schema default. A non-default
|
||||
* value (e.g. `orderedList.start: 5`) is NOT a default, so it is KEPT.
|
||||
*
|
||||
* Every entry below was read from `packages/docmost-client/src/lib/
|
||||
* docmost-schema.ts` (the line refs are the exact `default:` declarations) and
|
||||
* confirmed to be materialized by an export→import→export round-trip:
|
||||
* - mark `link` target / rel — DocmostAttributes + StarterKit link.
|
||||
* StarterKit's link extension defaults `target: "_blank"` and
|
||||
* `rel: "noopener noreferrer nofollow"`; both materialize on import
|
||||
* (empirically confirmed) even when the source had only `href`.
|
||||
* - mark `comment` resolved — docmost-schema.ts L213-214 (`default: false`).
|
||||
* - node `orderedList` start — provided by StarterKit's orderedList
|
||||
* (`default: 1`); materializes on import (empirically confirmed).
|
||||
* - node `drawio`/`excalidraw`/`video`/`youtube`/`embed` align — the diagram
|
||||
* attribute set and the media nodes declare `align: { default: "center" }`
|
||||
* (docmost-schema.ts L745-750 diagramAttributes; L564 video; L626 youtube;
|
||||
* L667 embed). The diagram `align` is the one the round-trip materializes
|
||||
* (docmost-schema.ts L745); the media/embed entries normalize the SAME
|
||||
* `align` default for consistency. Note: this only normalizes `align` —
|
||||
* full canonical stability of `embed` is separately limited by the
|
||||
* converter coercing numeric `width`/`height` to strings, which is outside
|
||||
* canonicalize's scope.
|
||||
*
|
||||
* NOTE: `image` has NO non-null align default — its `align` defaults to `null`
|
||||
* (docmost-schema.ts L174), so it is already handled by the null-drop rule and
|
||||
* is intentionally NOT listed here.
|
||||
*/
|
||||
const KNOWN_DEFAULTS: Record<string, Record<string, unknown>> = {
|
||||
// mark types
|
||||
link: {
|
||||
target: "_blank",
|
||||
rel: "noopener noreferrer nofollow",
|
||||
},
|
||||
comment: {
|
||||
resolved: false,
|
||||
},
|
||||
// node types
|
||||
orderedList: {
|
||||
start: 1,
|
||||
},
|
||||
drawio: {
|
||||
align: "center",
|
||||
},
|
||||
excalidraw: {
|
||||
align: "center",
|
||||
},
|
||||
video: {
|
||||
align: "center",
|
||||
},
|
||||
youtube: {
|
||||
align: "center",
|
||||
},
|
||||
embed: {
|
||||
align: "center",
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Prune an `attrs` object in place on a fresh copy: drop keys whose value is
|
||||
* `null` or `undefined` (an absent attribute and an explicit default of `null`
|
||||
* are semantically equivalent here). Optionally also drop a node-level `id`
|
||||
* (block ids are regenerated on import, SPEC §11). ALSO drop any attr whose
|
||||
* value equals the node/mark `type`'s known NON-null schema default
|
||||
* (`KNOWN_DEFAULTS`), so "attr absent" ≡ "attr at its default value" — without
|
||||
* this, the import-materialized `link.target`/`comment.resolved`/
|
||||
* `orderedList.start`/diagram `align` defaults would be a phantom diff. Every
|
||||
* non-default attribute value is KEPT (level, language, src, href, commentId,
|
||||
* width, a non-default `start`/`align`, ...).
|
||||
*
|
||||
* Returns the pruned attrs object, or `undefined` if nothing meaningful is
|
||||
* left (so the caller can drop the `attrs` key entirely: `{attrs:{}}` ≡ no
|
||||
* attrs).
|
||||
*/
|
||||
function canonicalizeAttrs(
|
||||
attrs: Record<string, unknown>,
|
||||
dropId: boolean,
|
||||
type: string | undefined,
|
||||
): Record<string, unknown> | undefined {
|
||||
const defaults = type ? KNOWN_DEFAULTS[type] : undefined;
|
||||
const out: Record<string, unknown> = {};
|
||||
// Stable key order so a JSON.stringify of the canonical form is comparable
|
||||
// regardless of the input's key order.
|
||||
for (const key of Object.keys(attrs).sort()) {
|
||||
// Block ids are regenerated on import; drop them on NODE attrs only.
|
||||
if (dropId && key === "id") continue;
|
||||
const value = attrs[key];
|
||||
// Absent ≡ explicit-default-null/undefined.
|
||||
if (value === null || value === undefined) continue;
|
||||
// Absent ≡ explicit known non-null default (e.g. link.target="_blank").
|
||||
// A non-default value (e.g. orderedList.start=5) does NOT match, so it is
|
||||
// kept. The `comment` mark's `commentId` is never a default, so it always
|
||||
// survives (SPEC §3); only its `resolved: false` default is normalized away.
|
||||
if (defaults && key in defaults && value === defaults[key]) continue;
|
||||
out[key] = value;
|
||||
}
|
||||
return Object.keys(out).length > 0 ? out : undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Return a DEEP COPY of a ProseMirror node tree, canonicalized so that two
|
||||
* semantically-equal documents compare deep-equal. Rules (applied recursively
|
||||
* to the node, its `content`, and its `marks`):
|
||||
*
|
||||
* 1. Remove node-level `attrs.id` (regenerated on import). Mark attrs are NOT
|
||||
* touched for `id` (marks carry no block id; only their meaningful attrs).
|
||||
* 2. In any `attrs` object (node OR mark) drop keys whose value is `null`/
|
||||
* `undefined` (absent ≡ explicit default null) OR equals that node/mark
|
||||
* type's known non-null schema default (absent ≡ explicit default).
|
||||
* Keep every non-default value. The type is passed into the attrs
|
||||
* normalizer so it can look up `KNOWN_DEFAULTS`.
|
||||
* 3. If an `attrs` object becomes empty after pruning, drop the `attrs` key.
|
||||
* 4. Preserve `marks` (including the `comment` mark and its `commentId` — a
|
||||
* meaningful anchor per SPEC §3; never strip it).
|
||||
* 5. Preserve `text`, `type`, and `content` order exactly.
|
||||
* 6. Never mutate the input.
|
||||
*/
|
||||
export function canonicalizeContent(node: any): any {
|
||||
if (Array.isArray(node)) {
|
||||
return node.map((child) => canonicalizeContent(child));
|
||||
}
|
||||
if (node === null || typeof node !== "object") {
|
||||
// Primitive leaf (string/number/boolean/null): returned as-is.
|
||||
return node;
|
||||
}
|
||||
|
||||
// A node is a mark when it has a `type` but never carries block `content`
|
||||
// and lives inside a `marks` array. We cannot tell from the node alone, so
|
||||
// we distinguish at the recursion site: node `attrs` drop `id`, mark `attrs`
|
||||
// do not. This is handled by passing a `dropId` flag down for the `attrs`
|
||||
// key specifically (nodes) vs the `marks[].attrs` path (marks).
|
||||
const out: Record<string, unknown> = {};
|
||||
for (const key of Object.keys(node)) {
|
||||
if (key === "attrs" && node.attrs && typeof node.attrs === "object") {
|
||||
// Node-level attrs: drop the block id, null/undefined attrs, and any
|
||||
// attr at this node type's known non-null schema default.
|
||||
const canon = canonicalizeAttrs(
|
||||
node.attrs as Record<string, unknown>,
|
||||
true,
|
||||
typeof node.type === "string" ? node.type : undefined,
|
||||
);
|
||||
if (canon !== undefined) out.attrs = canon;
|
||||
// else: drop the `attrs` key entirely (rule 3).
|
||||
} else if (key === "marks" && Array.isArray(node.marks)) {
|
||||
// Marks: keep them all (incl. comment); canonicalize their attrs but do
|
||||
// NOT drop `id` (a mark's `id` would be a meaningful attr, not a block
|
||||
// id). An empty marks array is dropped so `marks:[]` ≡ no marks.
|
||||
const marks = (node.marks as any[]).map((mark) => canonicalizeMark(mark));
|
||||
if (marks.length > 0) out.marks = marks;
|
||||
} else {
|
||||
out[key] = canonicalizeContent(node[key]);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Canonicalize a single mark: keep `type`, prune its `attrs` (null/undefined
|
||||
* AND known non-null defaults dropped, empty attrs removed) but NEVER drop a
|
||||
* mark's attribute as a "block id" — marks have no block id, only meaningful
|
||||
* attrs (href, commentId, color, level, ...). Meaningful NON-default attrs
|
||||
* survive (the `comment` mark's `commentId` is never a default, so it always
|
||||
* survives — SPEC §3); only known defaults like `link.target="_blank"`,
|
||||
* `link.rel="noopener…"` and `comment.resolved=false` are normalized away.
|
||||
*/
|
||||
function canonicalizeMark(mark: any): any {
|
||||
if (mark === null || typeof mark !== "object") return mark;
|
||||
const out: Record<string, unknown> = {};
|
||||
for (const key of Object.keys(mark)) {
|
||||
if (key === "attrs" && mark.attrs && typeof mark.attrs === "object") {
|
||||
const canon = canonicalizeAttrs(
|
||||
mark.attrs as Record<string, unknown>,
|
||||
false,
|
||||
typeof mark.type === "string" ? mark.type : undefined,
|
||||
);
|
||||
if (canon !== undefined) out.attrs = canon;
|
||||
} else {
|
||||
out[key] = canonicalizeContent(mark[key]);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Deep structural equality of two values that is key-order-insensitive.
|
||||
* Used to compare canonical forms. (`canonicalizeContent` already emits
|
||||
* `attrs` in a stable key order, but the top-level node keys preserve input
|
||||
* order, so we compare structurally rather than by string.)
|
||||
*/
|
||||
function deepEqual(a: any, b: any): boolean {
|
||||
if (a === b) return true;
|
||||
if (typeof a !== typeof b) return false;
|
||||
if (a === null || b === null) return a === b;
|
||||
if (typeof a !== "object") return false;
|
||||
|
||||
const aIsArr = Array.isArray(a);
|
||||
const bIsArr = Array.isArray(b);
|
||||
if (aIsArr !== bIsArr) return false;
|
||||
|
||||
if (aIsArr) {
|
||||
if (a.length !== b.length) return false;
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
if (!deepEqual(a[i], b[i])) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
const aKeys = Object.keys(a);
|
||||
const bKeys = Object.keys(b);
|
||||
if (aKeys.length !== bKeys.length) return false;
|
||||
for (const k of aKeys) {
|
||||
if (!Object.prototype.hasOwnProperty.call(b, k)) return false;
|
||||
if (!deepEqual(a[k], b[k])) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* True when two ProseMirror documents are semantically equal: equal after
|
||||
* canonicalization (block ids stripped, absent-vs-default-null normalized).
|
||||
*/
|
||||
export function docsCanonicallyEqual(a: any, b: any): boolean {
|
||||
return deepEqual(canonicalizeContent(a), canonicalizeContent(b));
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,26 @@
|
||||
/**
|
||||
* Public surface of the pure converter (`lib/`). This barrel re-exports the
|
||||
* PURE, IO-free pieces the sync engine needs: the self-contained markdown
|
||||
* (de)serializers, the lossless ProseMirror <-> Markdown converter, the
|
||||
* markdown -> ProseMirror import path, and semantic canonicalization for the
|
||||
* round-trip idempotency check (SPEC §11).
|
||||
*
|
||||
* There is no REST client, websocket/collab write-path, auth-utils or page-lock
|
||||
* here — the gitmost server writes natively.
|
||||
*/
|
||||
export {
|
||||
serializeDocmostMarkdown,
|
||||
parseDocmostMarkdown,
|
||||
serializeDocmostMarkdownBody,
|
||||
} from "./markdown-document.js";
|
||||
export type { DocmostMdMeta } from "./markdown-document.js";
|
||||
|
||||
export { convertProseMirrorToMarkdown } from "./markdown-converter.js";
|
||||
|
||||
export { markdownToProseMirror } from "./markdown-to-prosemirror.js";
|
||||
|
||||
export {
|
||||
canonicalizeContent,
|
||||
docsCanonicallyEqual,
|
||||
} from "./canonicalize.js";
|
||||
export { parsePageFile, serializePageFile } from "./page-file.js";
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,154 @@
|
||||
/**
|
||||
* Self-contained Docmost-flavoured Markdown document (custom extensions).
|
||||
*
|
||||
* A single `.md` file that packages everything needed to losslessly round-trip
|
||||
* a page through "download -> edit body -> re-upload":
|
||||
* - a leading `docmost:meta` block: a one-line JSON object with page identity;
|
||||
* - the Markdown body (carrying inline comment anchors and diagrams as HTML);
|
||||
* - a trailing `docmost:comments` block: a one-line JSON array of comment
|
||||
* threads.
|
||||
*
|
||||
* Both metadata blocks are HTML comments on purpose: `marked`/`generateJSON`
|
||||
* drop HTML comments, so even if the WHOLE file were ever fed straight to the
|
||||
* importer without first stripping the blocks, the metadata cannot leak into the
|
||||
* document. (A fenced ```docmost-comments``` block would WRONGLY become a
|
||||
* codeBlock node, so a fenced block is deliberately NOT used.)
|
||||
*
|
||||
* The delimiter literals may legitimately appear in the BODY too (e.g. a user
|
||||
* re-pastes an exported `.md` into a page, or a page documents this very
|
||||
* format). To stay robust, parsing treats only the FINAL, document-ending
|
||||
* `docmost:comments` block as metadata: it is the last `<!-- docmost:comments`
|
||||
* opener whose closing `-->` sits at the very end of the file. Any earlier
|
||||
* literal occurrence is left in the body untouched.
|
||||
*
|
||||
* NOTE on comments: in this version the comment THREAD records are preserved in
|
||||
* the file but are NOT pushed back to the server on import — only the inline
|
||||
* comment marks (anchors) embedded in the body are restored. Managing comment
|
||||
* records stays with the comment tools/UI.
|
||||
*/
|
||||
|
||||
export interface DocmostMdMeta {
|
||||
version: number;
|
||||
pageId?: string;
|
||||
slugId?: string;
|
||||
title?: string;
|
||||
spaceId?: string;
|
||||
parentPageId?: string | null;
|
||||
}
|
||||
|
||||
// Match the leading meta block (allow leading whitespace). Capture group 1 is
|
||||
// the JSON text between the markers.
|
||||
const META_RE = /^\s*<!--\s*docmost:meta\s*\n([\s\S]*?)\n-->/;
|
||||
// Match a `docmost:comments` opener. Used globally to scan for the LAST opener
|
||||
// rather than end-anchoring a single regex (which would mis-capture across a
|
||||
// literal opener that appears earlier in the body).
|
||||
const COMMENTS_OPEN_RE = /<!--[ \t]*docmost:comments[ \t]*\r?\n/g;
|
||||
|
||||
/**
|
||||
* Assemble the full self-contained markdown file: meta block, body, and the
|
||||
* comments block. The meta block is always emitted; the comments block is always
|
||||
* emitted too (with `[]` when there are no comments) so the format stays uniform
|
||||
* and parsing stays simple.
|
||||
*/
|
||||
export function serializeDocmostMarkdown(
|
||||
meta: DocmostMdMeta,
|
||||
body: string,
|
||||
comments: any[],
|
||||
): string {
|
||||
const metaJson = JSON.stringify(meta);
|
||||
const commentsJson = JSON.stringify(Array.isArray(comments) ? comments : []);
|
||||
const trimmedBody = (body ?? "").trim();
|
||||
return (
|
||||
`<!-- docmost:meta\n${metaJson}\n-->\n\n` +
|
||||
`${trimmedBody}\n\n` +
|
||||
`<!-- docmost:comments\n${commentsJson}\n-->\n`
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Split a self-contained file back into its parts. Tolerant: if the meta or
|
||||
* comments block is missing (e.g. a hand-written plain-markdown file), the
|
||||
* corresponding value is returned as `null` and the whole input is treated as
|
||||
* the body. This never throws on a MISSING block; only a `JSON.parse` failure
|
||||
* inside a block that IS present is surfaced as a thrown Error with a clear
|
||||
* message. Robust to `\r\n` line endings.
|
||||
*/
|
||||
export function parseDocmostMarkdown(full: string): {
|
||||
meta: DocmostMdMeta | null;
|
||||
body: string;
|
||||
comments: any[] | null;
|
||||
} {
|
||||
// Normalize line endings so the anchored regexes work regardless of CRLF.
|
||||
const normalized = (full ?? "").replace(/\r\n/g, "\n");
|
||||
|
||||
// Extract the leading meta block (start-anchored — already unambiguous).
|
||||
let meta: DocmostMdMeta | null = null;
|
||||
let metaEnd = 0;
|
||||
const metaMatch = normalized.match(META_RE);
|
||||
if (metaMatch) {
|
||||
try {
|
||||
meta = JSON.parse(metaMatch[1]);
|
||||
} catch (e) {
|
||||
throw new Error(
|
||||
`Invalid docmost:meta JSON block: ${
|
||||
e instanceof Error ? e.message : String(e)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
// Body starts right after the matched meta block.
|
||||
metaEnd = (metaMatch.index ?? 0) + metaMatch[0].length;
|
||||
}
|
||||
|
||||
// Find the LAST `<!-- docmost:comments` opener; the real file-level block is
|
||||
// the final one whose closing `-->` ends the document. Any earlier literal
|
||||
// occurrence inside the body (e.g. a re-pasted export) is left in the body.
|
||||
let lastOpenStart = -1;
|
||||
let lastOpenEnd = -1;
|
||||
let m: RegExpExecArray | null;
|
||||
COMMENTS_OPEN_RE.lastIndex = 0;
|
||||
while ((m = COMMENTS_OPEN_RE.exec(normalized)) !== null) {
|
||||
lastOpenStart = m.index;
|
||||
lastOpenEnd = m.index + m[0].length;
|
||||
}
|
||||
|
||||
let comments: any[] | null = null;
|
||||
let bodyEnd = normalized.length;
|
||||
if (lastOpenStart !== -1) {
|
||||
const rest = normalized.slice(lastOpenEnd);
|
||||
const close = rest.match(/\r?\n-->[ \t]*\r?\n?\s*$/); // closer must end the doc
|
||||
if (close) {
|
||||
const jsonText = rest.slice(0, close.index);
|
||||
try {
|
||||
comments = JSON.parse(jsonText);
|
||||
} catch (e) {
|
||||
throw new Error(
|
||||
`Invalid docmost:comments JSON block: ${
|
||||
e instanceof Error ? e.message : String(e)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
bodyEnd = lastOpenStart; // strip from the opener to end of document
|
||||
}
|
||||
}
|
||||
|
||||
const body = normalized.slice(metaEnd, bodyEnd).trim();
|
||||
return { meta, body, comments };
|
||||
}
|
||||
|
||||
/**
|
||||
* Serialize a self-contained markdown file with the meta block + body ONLY —
|
||||
* NO trailing `docmost:comments` block. The sync engine never touches
|
||||
* `/comments` (SPEC §3): the synced file carries just page identity (meta) and
|
||||
* the body, where comment threads survive only as inline `<span
|
||||
* data-comment-id>` anchor marks inside the body.
|
||||
*
|
||||
* `parseDocmostMarkdown` already tolerates a missing comments block (it returns
|
||||
* `comments: null` and treats the rest as body), so a file produced here
|
||||
* round-trips cleanly through the parser.
|
||||
*/
|
||||
export function serializeDocmostMarkdownBody(
|
||||
meta: DocmostMdMeta,
|
||||
body: string,
|
||||
): string {
|
||||
return `<!-- docmost:meta\n${JSON.stringify(meta)}\n-->\n\n${(body ?? "").trim()}\n`;
|
||||
}
|
||||
@@ -0,0 +1,365 @@
|
||||
/**
|
||||
* Pure markdown -> ProseMirror conversion.
|
||||
*
|
||||
* The converter path is `markdownToProseMirror` (marked -> HTML ->
|
||||
* generateJSON) plus the two pre/post processors it needs (`preprocessCallouts`,
|
||||
* `bridgeTaskLists`). The gitmost server writes the resulting page bodies
|
||||
* natively through the collab gateway, so no websocket/Yjs write-path lives
|
||||
* here.
|
||||
*/
|
||||
import { generateJSON } from "@tiptap/html";
|
||||
import { JSDOM } from "jsdom";
|
||||
import { marked } from "marked";
|
||||
import { docmostExtensions } from "./docmost-schema.js";
|
||||
|
||||
// Setup DOM environment for Tiptap HTML parsing in Node.js
|
||||
const dom = new JSDOM("<!DOCTYPE html><html><body></body></html>");
|
||||
global.window = dom.window as any;
|
||||
global.document = dom.window.document;
|
||||
// @ts-ignore
|
||||
global.Element = dom.window.Element;
|
||||
|
||||
/**
|
||||
* Hard ceiling above which we skip callout preprocessing entirely. The linear
|
||||
* scanner below has no quadratic blow-up, but we still cap input defensively so
|
||||
* a pathological multi-megabyte payload cannot tie up the event loop; in that
|
||||
* case the markdown is passed through verbatim (callouts are simply not
|
||||
* detected) rather than risking a slow scan.
|
||||
*/
|
||||
const MAX_CALLOUT_PREPROCESS_BYTES = 4 * 1024 * 1024; // 4 MB
|
||||
|
||||
/** Matches an opening callout fence: `:::type` (type captured, lower-cased). */
|
||||
const CALLOUT_OPEN_RE = /^:::\s*(\w+)\s*$/;
|
||||
/** Matches a bare closing callout fence: `:::`. */
|
||||
const CALLOUT_CLOSE_RE = /^:::\s*$/;
|
||||
/**
|
||||
* Matches an Obsidian-native callout opener: `> [!type]` (type captured). An
|
||||
* optional title after the type is allowed but ignored (the Docmost callout
|
||||
* schema has no title). The body is the following contiguous blockquote lines.
|
||||
*/
|
||||
const CALLOUT_BQ_OPEN_RE = /^>\s*\[!(\w+)\]/;
|
||||
/** Matches any blockquote continuation line (`>` … ). */
|
||||
const BLOCKQUOTE_LINE_RE = /^>/;
|
||||
/** Matches the start/end of a code fence (``` or ~~~), capturing the marker. */
|
||||
const CODE_FENCE_RE = /^(\s*)(`{3,}|~{3,})/;
|
||||
|
||||
/**
|
||||
* Pre-process Docmost-flavoured markdown: convert `:::type ... :::`
|
||||
* callout blocks (the syntax our markdown export produces) into HTML
|
||||
* divs that the callout extension parses. The inner content is rendered
|
||||
* through marked as regular markdown.
|
||||
*
|
||||
* Implemented as a single linear pass over the lines (no quadratic regex
|
||||
* rescan). It:
|
||||
* - tracks fenced code regions (```...``` and ~~~...~~~) and never treats a
|
||||
* `:::` line that lives inside a code fence as a callout delimiter, so a
|
||||
* callout body that itself contains a fenced code block with a `:::` line is
|
||||
* no longer corrupted;
|
||||
* - matches an opening `:::type` line with the next CLOSING `:::` at the SAME
|
||||
* nesting level, supporting NESTED callouts via a depth counter (an inner
|
||||
* `:::type` opens a deeper level and consumes a matching `:::`);
|
||||
* - emits the same `<div data-type="callout" data-callout-type="TYPE">` output
|
||||
* (inner rendered through marked) as the previous regex implementation.
|
||||
*/
|
||||
async function preprocessCallouts(markdown: string): Promise<string> {
|
||||
// Defensive cap: skip preprocessing for pathologically large inputs.
|
||||
if (markdown.length > MAX_CALLOUT_PREPROCESS_BYTES) {
|
||||
return markdown;
|
||||
}
|
||||
|
||||
// Recursively transform a slice of lines, converting top-level callouts in
|
||||
// that slice into <div> blocks and rendering their inner content (which may
|
||||
// itself contain nested callouts) through this same function.
|
||||
const transform = async (lines: string[]): Promise<string> => {
|
||||
const out: string[] = [];
|
||||
let inCodeFence = false;
|
||||
let codeFenceMarker = ""; // the exact run of backticks/tildes that opened it
|
||||
let i = 0;
|
||||
|
||||
while (i < lines.length) {
|
||||
const line = lines[i];
|
||||
|
||||
// Inside a code fence, only its matching closing fence is significant;
|
||||
// everything else (including `:::` lines) is copied through verbatim.
|
||||
if (inCodeFence) {
|
||||
out.push(line);
|
||||
const fence = line.match(CODE_FENCE_RE);
|
||||
if (fence && fence[2].startsWith(codeFenceMarker[0]) &&
|
||||
fence[2].length >= codeFenceMarker.length) {
|
||||
inCodeFence = false;
|
||||
codeFenceMarker = "";
|
||||
}
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// A code fence opening outside any callout body: enter code-fence mode.
|
||||
const fenceOpen = line.match(CODE_FENCE_RE);
|
||||
if (fenceOpen) {
|
||||
inCodeFence = true;
|
||||
codeFenceMarker = fenceOpen[2];
|
||||
out.push(line);
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// An opening callout fence: scan forward (with code-fence and nested
|
||||
// callout awareness) for its matching closing `:::` at the same level.
|
||||
const open = line.match(CALLOUT_OPEN_RE);
|
||||
if (open) {
|
||||
const type = open[1].toLowerCase();
|
||||
const bodyLines: string[] = [];
|
||||
let depth = 1;
|
||||
let innerInCodeFence = false;
|
||||
let innerCodeFenceMarker = "";
|
||||
let j = i + 1;
|
||||
for (; j < lines.length; j++) {
|
||||
const bl = lines[j];
|
||||
if (innerInCodeFence) {
|
||||
const f = bl.match(CODE_FENCE_RE);
|
||||
if (f && f[2].startsWith(innerCodeFenceMarker[0]) &&
|
||||
f[2].length >= innerCodeFenceMarker.length) {
|
||||
innerInCodeFence = false;
|
||||
innerCodeFenceMarker = "";
|
||||
}
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
const innerFence = bl.match(CODE_FENCE_RE);
|
||||
if (innerFence) {
|
||||
innerInCodeFence = true;
|
||||
innerCodeFenceMarker = innerFence[2];
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
if (CALLOUT_OPEN_RE.test(bl)) {
|
||||
depth++;
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
if (CALLOUT_CLOSE_RE.test(bl)) {
|
||||
depth--;
|
||||
if (depth === 0) break; // matching close for THIS callout
|
||||
bodyLines.push(bl);
|
||||
continue;
|
||||
}
|
||||
bodyLines.push(bl);
|
||||
}
|
||||
|
||||
if (j < lines.length) {
|
||||
// Found the matching closing fence: render the body (recursively, so
|
||||
// nested callouts are handled) and emit the callout div.
|
||||
const inner = await transform(bodyLines);
|
||||
const renderedInner = await marked.parse(inner);
|
||||
out.push(
|
||||
`\n<div data-type="callout" data-callout-type="${type}">${renderedInner}</div>\n`,
|
||||
);
|
||||
i = j + 1; // skip past the closing `:::`
|
||||
continue;
|
||||
}
|
||||
// No matching close (unterminated callout): treat the opener as a
|
||||
// literal line and continue, preserving the original text.
|
||||
out.push(line);
|
||||
i++;
|
||||
continue;
|
||||
}
|
||||
|
||||
// An Obsidian-native callout: `> [!type]` opener; the body is the following
|
||||
// CONTIGUOUS blockquote (`>`-prefixed) lines. Strip ONE blockquote level and
|
||||
// recurse so nested callouts (`> > [!type]`) are handled, then emit the same
|
||||
// callout div the `:::` path produces. A normal blockquote (no `[!type]` on
|
||||
// its first line) does not match and stays a blockquote.
|
||||
const bqOpen = line.match(CALLOUT_BQ_OPEN_RE);
|
||||
if (bqOpen) {
|
||||
const type = bqOpen[1].toLowerCase();
|
||||
const bodyLines: string[] = [];
|
||||
let j = i + 1;
|
||||
for (; j < lines.length; j++) {
|
||||
if (!BLOCKQUOTE_LINE_RE.test(lines[j])) break;
|
||||
bodyLines.push(lines[j].replace(/^>\s?/, ""));
|
||||
}
|
||||
const inner = await transform(bodyLines);
|
||||
const renderedInner = await marked.parse(inner);
|
||||
out.push(
|
||||
`\n<div data-type="callout" data-callout-type="${type}">${renderedInner}</div>\n`,
|
||||
);
|
||||
i = j;
|
||||
continue;
|
||||
}
|
||||
|
||||
out.push(line);
|
||||
i++;
|
||||
}
|
||||
|
||||
return out.join("\n");
|
||||
};
|
||||
|
||||
return transform(markdown.split("\n"));
|
||||
}
|
||||
|
||||
/**
|
||||
* Bridge marked's checkbox lists to TipTap task lists.
|
||||
*
|
||||
* marked renders GitHub task list items (`- [x] done`) as a plain
|
||||
* `<ul><li><p><input type="checkbox" checked> text</p></li></ul>` WITHOUT the
|
||||
* markup TipTap's TaskList/TaskItem extensions parse. This rewrites such lists
|
||||
* into the shape those extensions expect:
|
||||
* TaskList parseHTML matches `ul[data-type="taskList"]`,
|
||||
* TaskItem matches `li[data-type="taskItem"]`,
|
||||
* the checked state is read from `data-checked === "true"`.
|
||||
*
|
||||
* A list is only converted when it has at least one `<li>` and EVERY direct
|
||||
* `<li>` contains a checkbox input. Both `<ul>` and `<ol>` are considered: a
|
||||
* numbered checklist (`1. [x] a`, which marked renders as an `<ol>` of checkbox
|
||||
* `<li>`s) would otherwise lose its task state. TipTap task lists are unordered,
|
||||
* so a matching `<ol>` is emitted as `data-type="taskList"` exactly like a
|
||||
* `<ul>`. Mixed or ordinary lists (including ordinary `<ol>` lists) are left
|
||||
* untouched so they keep rendering as bullet/numbered lists. The marked `<p>`
|
||||
* wrapper is kept inside the `<li>` because TaskItem content allows paragraphs.
|
||||
*/
|
||||
function bridgeTaskLists(html: string): string {
|
||||
// Cheap early-out: if the markup contains no checkbox input at all there is
|
||||
// nothing to bridge, so skip the expensive JSDOM parse entirely. This is the
|
||||
// common case (most pages have no task lists).
|
||||
if (!/type=["']?checkbox/i.test(html)) {
|
||||
return html;
|
||||
}
|
||||
// Defensive cap (consistent with preprocessCallouts): skip the bridge for
|
||||
// pathologically large inputs rather than running a second expensive JSDOM
|
||||
// parse on a multi-megabyte payload. The markup is passed through verbatim.
|
||||
if (html.length > MAX_CALLOUT_PREPROCESS_BYTES) {
|
||||
return html;
|
||||
}
|
||||
const dom = new JSDOM(html);
|
||||
const document = dom.window.document;
|
||||
// Collect the checkbox(es) that belong to THIS <li> directly: either direct
|
||||
// child <input type="checkbox"> elements or ones inside the <li>'s direct <p>
|
||||
// child (the shape marked emits: `<li><p><input type="checkbox"> text</p></li>`).
|
||||
// Checkboxes nested deeper (e.g. inside a child <ul>/<ol>) are excluded so a
|
||||
// bullet <li> that merely contains a nested task sublist is not misdetected.
|
||||
// Raw inline HTML can put more than one checkbox in a single <li>; we gather
|
||||
// ALL of them so none survive into the converted item.
|
||||
const directCheckboxes = (li: Element): Element[] => {
|
||||
const found: Element[] = [];
|
||||
for (const child of Array.from(li.children)) {
|
||||
if (
|
||||
child.tagName === "INPUT" &&
|
||||
child.getAttribute("type") === "checkbox"
|
||||
) {
|
||||
found.push(child);
|
||||
continue;
|
||||
}
|
||||
if (child.tagName === "P") {
|
||||
for (const inp of Array.from(
|
||||
child.querySelectorAll(":scope > input[type='checkbox']"),
|
||||
)) {
|
||||
found.push(inp);
|
||||
}
|
||||
}
|
||||
}
|
||||
return found;
|
||||
};
|
||||
// Both <ul> and <ol> are candidates: an <ol> whose every direct <li> carries
|
||||
// its own checkbox is a numbered checklist that must also become a taskList.
|
||||
const lists = Array.from(document.querySelectorAll("ul, ol"));
|
||||
for (const list of lists) {
|
||||
// Only consider DIRECT child <li> elements; nested lists are handled by
|
||||
// their own iteration of the outer loop.
|
||||
const items = Array.from(list.children).filter(
|
||||
(child) => child.tagName === "LI",
|
||||
);
|
||||
if (items.length === 0) continue;
|
||||
const itemCheckboxes = items.map((li) => directCheckboxes(li));
|
||||
// Convert only when every direct <li> carries at least one OWN checkbox.
|
||||
if (!itemCheckboxes.every((boxes) => boxes.length > 0)) continue;
|
||||
|
||||
// A numbered checklist arrives as an <ol>. We must NOT leave the tag as
|
||||
// <ol> while tagging it data-type="taskList": generateJSON would then match
|
||||
// BOTH the orderedList rule (tag ol) and the taskList rule (data-type),
|
||||
// emitting a phantom empty orderedList beside the real taskList. So rename a
|
||||
// qualifying <ol> to a <ul> — move its <li> children over and replace it —
|
||||
// leaving only the taskList rule to match. Already-<ul> lists are unchanged.
|
||||
let target: Element = list;
|
||||
if (list.tagName === "OL") {
|
||||
const ul = document.createElement("ul");
|
||||
// Carry over existing attributes (e.g. class) so nothing is silently lost.
|
||||
for (const attr of Array.from(list.attributes)) {
|
||||
ul.setAttribute(attr.name, attr.value);
|
||||
}
|
||||
// Move every child node (including the <li>s we collected) into the <ul>.
|
||||
while (list.firstChild) {
|
||||
ul.appendChild(list.firstChild);
|
||||
}
|
||||
list.replaceWith(ul);
|
||||
target = ul;
|
||||
}
|
||||
|
||||
target.setAttribute("data-type", "taskList");
|
||||
items.forEach((li, index) => {
|
||||
const boxes = itemCheckboxes[index];
|
||||
// The first checkbox determines the checked state (matches the previous
|
||||
// single-checkbox behaviour); any extras only need removing.
|
||||
const input = boxes[0] ?? null;
|
||||
li.setAttribute("data-type", "taskItem");
|
||||
const checked =
|
||||
input != null &&
|
||||
(input.hasAttribute("checked") || (input as any).checked);
|
||||
li.setAttribute("data-checked", checked ? "true" : "false");
|
||||
// Remove ALL direct checkbox inputs so none survive into the content
|
||||
// (a raw-inline-HTML <li> may carry more than one).
|
||||
for (const box of boxes) {
|
||||
box.remove();
|
||||
}
|
||||
});
|
||||
}
|
||||
return document.body.innerHTML;
|
||||
}
|
||||
|
||||
/**
|
||||
* Recursively strip content-less paragraph nodes from a generated doc.
|
||||
*
|
||||
* A block-level atom whose markdown form is INLINE (e.g. the block `image`'s
|
||||
* ``, or a bare media element) is wrapped by marked in a <p>; the schema
|
||||
* then HOISTS the block atom out of that paragraph, leaving an EMPTY paragraph
|
||||
* sibling. On the next export that empty `<p>` renders to "" and the doc "\n\n"
|
||||
* join injects a phantom blank gap, so the markdown is not byte-stable.
|
||||
*
|
||||
* Markdown blank lines are separators, never content, so generateJSON only ever
|
||||
* produces an empty paragraph as such a hoist artifact — removing them is safe
|
||||
* and general (it also subsumes the <div>-wrapper workaround the `video` case
|
||||
* uses). We remove ONLY `type === 'paragraph'` nodes whose `content` is absent
|
||||
* or an empty array; every other node (including atoms without `content`) is
|
||||
* preserved, and we recurse into the content of any node that has children.
|
||||
*/
|
||||
function stripEmptyParagraphs(node: any): any {
|
||||
if (!node || !Array.isArray(node.content)) {
|
||||
// Atom / leaf node (no children to recurse into): keep as-is.
|
||||
return node;
|
||||
}
|
||||
const mapped = node.content.map((child: any) => stripEmptyParagraphs(child));
|
||||
const isEmptyParagraph = (child: any): boolean =>
|
||||
!!child &&
|
||||
child.type === "paragraph" &&
|
||||
(!Array.isArray(child.content) || child.content.length === 0);
|
||||
const filtered = mapped.filter((child: any) => !isEmptyParagraph(child));
|
||||
// Schema-validity guard: several nodes require NON-empty block content
|
||||
// (`content: "block+"` — tableCell, tableHeader, blockquote, column, callout,
|
||||
// and the doc root). For an empty one of those, generateJSON materializes a
|
||||
// single empty paragraph as its OBLIGATORY content — that is not a hoist
|
||||
// artifact. If stripping would empty the container, keep ONE empty paragraph
|
||||
// so the result stays schema-valid (an empty cell/quote must not become `[]`).
|
||||
const cleaned =
|
||||
filtered.length === 0 && mapped.length > 0 ? [mapped[0]] : filtered;
|
||||
return { ...node, content: cleaned };
|
||||
}
|
||||
|
||||
/** Convert markdown to a ProseMirror doc using the full Docmost schema. */
|
||||
export async function markdownToProseMirror(
|
||||
markdownContent: string,
|
||||
): Promise<any> {
|
||||
const withCallouts = await preprocessCallouts(markdownContent);
|
||||
const html = await marked.parse(withCallouts);
|
||||
const bridged = bridgeTaskLists(html);
|
||||
const doc = generateJSON(bridged, docmostExtensions);
|
||||
return stripEmptyParagraphs(doc);
|
||||
}
|
||||
@@ -0,0 +1,897 @@
|
||||
/**
|
||||
* Pure, network-free helpers for manipulating a ProseMirror/TipTap document
|
||||
* tree by node id.
|
||||
*
|
||||
* A ProseMirror node here is a plain JSON object of the shape produced by
|
||||
* Docmost: `{ type, attrs?, content?, text?, marks? }`. Children live in the
|
||||
* `content` array; a node carries a stable id in `attrs.id`. Callouts and
|
||||
* table cells hold their children in `content` just like any other block, so a
|
||||
* single recursive walk reaches them all.
|
||||
*
|
||||
* Every exported function operates on a DEEP CLONE of the input document and
|
||||
* returns the new document. The input doc and any `newNode`/`node` argument are
|
||||
* never mutated. All functions are defensively null-safe: missing/!Array
|
||||
* `content`, non-object nodes, and absent `attrs` are tolerated.
|
||||
*/
|
||||
|
||||
/** Deep-clone a JSON-serializable value without mutating the original. */
|
||||
function clone<T>(value: T): T {
|
||||
if (typeof structuredClone === "function") {
|
||||
return structuredClone(value);
|
||||
}
|
||||
// Fallback for environments without structuredClone.
|
||||
return JSON.parse(JSON.stringify(value)) as T;
|
||||
}
|
||||
|
||||
/** True if `value` is a non-null object (and not an array). */
|
||||
function isObject(value: any): value is Record<string, any> {
|
||||
return value != null && typeof value === "object" && !Array.isArray(value);
|
||||
}
|
||||
|
||||
/** True if `node` carries the given id in `node.attrs.id`. */
|
||||
function matchesId(node: any, nodeId: string): boolean {
|
||||
return isObject(node) && isObject(node.attrs) && node.attrs.id === nodeId;
|
||||
}
|
||||
|
||||
/**
|
||||
* Recursively concatenate all text contained in a node.
|
||||
*
|
||||
* Text nodes contribute their `text` string; container nodes contribute the
|
||||
* joined `blockPlainText` of their `content` children. Returns "" for nullish
|
||||
* or non-object inputs.
|
||||
*/
|
||||
export function blockPlainText(node: any): string {
|
||||
if (!isObject(node)) return "";
|
||||
let out = "";
|
||||
if (typeof node.text === "string") {
|
||||
out += node.text;
|
||||
}
|
||||
if (Array.isArray(node.content)) {
|
||||
for (const child of node.content) {
|
||||
out += blockPlainText(child);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** Truncate `text` to at most `n` chars, appending an ellipsis when cut. */
|
||||
function truncate(text: string, n: number): string {
|
||||
return text.length > n ? text.slice(0, n) + "…" : text;
|
||||
}
|
||||
|
||||
/** One compact outline entry for a single top-level block. */
|
||||
export interface OutlineEntry {
|
||||
index: number;
|
||||
type: string | undefined;
|
||||
id: string | null;
|
||||
firstText: string;
|
||||
/** Present for headings only. */
|
||||
level?: number | null;
|
||||
/** Present for tables only. */
|
||||
rows?: number;
|
||||
cols?: number;
|
||||
header?: string[];
|
||||
/** Present for list blocks only (bulletList/orderedList/taskList). */
|
||||
items?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a COMPACT outline of the TOP-LEVEL blocks of `doc` (the entries in
|
||||
* `doc.content`). Deliberately does NOT recurse into paragraphs, list items, or
|
||||
* table cells — compactness is the point; use `getNodeByRef` to drill into a
|
||||
* specific block.
|
||||
*
|
||||
* Each entry carries `{ index, type, id, firstText }`, plus type-specific
|
||||
* extras: headings add `level`; tables add `rows`/`cols` and the first row's
|
||||
* cell texts as `header`; list blocks (types ending in "List") add `items`.
|
||||
* `firstText` is the block's plain text truncated to 100 chars. Null-safe:
|
||||
* a missing or non-object doc/content yields `[]`.
|
||||
*/
|
||||
export function buildOutline(doc: any): OutlineEntry[] {
|
||||
if (!isObject(doc) || !Array.isArray(doc.content)) return [];
|
||||
|
||||
const out: OutlineEntry[] = [];
|
||||
for (let i = 0; i < doc.content.length; i++) {
|
||||
const block = doc.content[i];
|
||||
const type = isObject(block) ? block.type : undefined;
|
||||
const entry: OutlineEntry = {
|
||||
index: i,
|
||||
type,
|
||||
id: isObject(block) && isObject(block.attrs) ? block.attrs.id ?? null : null,
|
||||
firstText: truncate(blockPlainText(block), 100),
|
||||
};
|
||||
|
||||
if (type === "heading") {
|
||||
entry.level = isObject(block.attrs) ? block.attrs.level ?? null : null;
|
||||
} else if (type === "table") {
|
||||
const headerRow = block.content?.[0]?.content ?? [];
|
||||
entry.rows = block.content?.length ?? 0;
|
||||
entry.cols = block.content?.[0]?.content?.length ?? 0;
|
||||
entry.header = headerRow.map((cell: any) =>
|
||||
truncate(blockPlainText(cell), 40),
|
||||
);
|
||||
} else if (typeof type === "string" && type.endsWith("List")) {
|
||||
entry.items = block.content?.length ?? 0;
|
||||
}
|
||||
|
||||
out.push(entry);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a single node by reference and return `{ node, path, type }`, or
|
||||
* `null` when nothing matches.
|
||||
*
|
||||
* - `ref` of the form `#<n>` (e.g. `#2`) selects the TOP-LEVEL block at index
|
||||
* `n` in `doc.content`. This is the only way to address table/tableRow/
|
||||
* tableCell nodes, which carry no `attrs.id`.
|
||||
* - Otherwise `ref` is treated as a block id: the FIRST node anywhere in the
|
||||
* tree with `attrs.id === ref` is returned.
|
||||
*
|
||||
* `path` is the array of child indices from the doc root down to the node
|
||||
* (so a top-level block is `[index]`). The returned `node` is a DEEP CLONE,
|
||||
* so callers can mutate it without touching the input doc. Null-safe.
|
||||
*/
|
||||
export function getNodeByRef(
|
||||
doc: any,
|
||||
ref: string,
|
||||
): { node: any; path: number[]; type: string | undefined } | null {
|
||||
if (!isObject(doc)) return null;
|
||||
|
||||
// "#<n>": index into the top-level content array.
|
||||
const indexMatch = typeof ref === "string" ? ref.match(/^#(\d+)$/) : null;
|
||||
if (indexMatch) {
|
||||
const index = Number(indexMatch[1]);
|
||||
const block = Array.isArray(doc.content) ? doc.content[index] : undefined;
|
||||
if (!isObject(block)) return null;
|
||||
return { node: clone(block), path: [index], type: block.type };
|
||||
}
|
||||
|
||||
// Otherwise: depth-first search for the first node with attrs.id === ref.
|
||||
const search = (
|
||||
node: any,
|
||||
trail: number[],
|
||||
): { node: any; path: number[]; type: string } | null => {
|
||||
if (!isObject(node)) return null;
|
||||
if (Array.isArray(node.content)) {
|
||||
for (let i = 0; i < node.content.length; i++) {
|
||||
const child = node.content[i];
|
||||
const path = [...trail, i];
|
||||
if (matchesId(child, ref)) {
|
||||
return { node: clone(child), path, type: child.type };
|
||||
}
|
||||
const hit = search(child, path);
|
||||
if (hit != null) return hit;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
return search(doc, []);
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace EVERY node whose `attrs.id === nodeId` with a deep clone of
|
||||
* `newNode`, anywhere in the tree (including inside callouts and table cells).
|
||||
*
|
||||
* Operates on a clone of `doc`; returns `{ doc, replaced }` where `replaced`
|
||||
* is the number of nodes substituted. A fresh clone of `newNode` is used for
|
||||
* each match so they do not share references.
|
||||
*/
|
||||
export function replaceNodeById(
|
||||
doc: any,
|
||||
nodeId: string,
|
||||
newNode: any,
|
||||
): { doc: any; replaced: number } {
|
||||
const out = clone(doc);
|
||||
let replaced = 0;
|
||||
|
||||
// Walk a content array, replacing direct matches and recursing into the
|
||||
// (possibly new) children of non-matching nodes.
|
||||
const walkContent = (content: any[]): void => {
|
||||
for (let i = 0; i < content.length; i++) {
|
||||
const child = content[i];
|
||||
if (matchesId(child, nodeId)) {
|
||||
content[i] = clone(newNode);
|
||||
replaced++;
|
||||
// Do not recurse into a freshly substituted node.
|
||||
continue;
|
||||
}
|
||||
if (isObject(child) && Array.isArray(child.content)) {
|
||||
walkContent(child.content);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
if (isObject(out) && Array.isArray(out.content)) {
|
||||
walkContent(out.content);
|
||||
}
|
||||
return { doc: out, replaced };
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove EVERY node whose `attrs.id === nodeId` from its parent `content`
|
||||
* array, anywhere in the tree (recursive, including callouts and tables).
|
||||
*
|
||||
* Operates on a clone of `doc`; returns `{ doc, deleted }` where `deleted` is
|
||||
* the number of nodes removed.
|
||||
*/
|
||||
export function deleteNodeById(
|
||||
doc: any,
|
||||
nodeId: string,
|
||||
): { doc: any; deleted: number } {
|
||||
const out = clone(doc);
|
||||
let deleted = 0;
|
||||
|
||||
// Filter a content array in place, dropping matches and recursing into the
|
||||
// surviving children.
|
||||
const walkContent = (content: any[]): any[] => {
|
||||
const kept: any[] = [];
|
||||
for (const child of content) {
|
||||
if (matchesId(child, nodeId)) {
|
||||
deleted++;
|
||||
continue;
|
||||
}
|
||||
if (isObject(child) && Array.isArray(child.content)) {
|
||||
child.content = walkContent(child.content);
|
||||
}
|
||||
kept.push(child);
|
||||
}
|
||||
return kept;
|
||||
};
|
||||
|
||||
if (isObject(out) && Array.isArray(out.content)) {
|
||||
out.content = walkContent(out.content);
|
||||
}
|
||||
return { doc: out, deleted };
|
||||
}
|
||||
|
||||
/**
|
||||
* Deep-clone `doc` and strip every node/mark attribute whose value is strictly
|
||||
* `undefined`, so the result is safe to hand to Yjs (which throws an opaque
|
||||
* "Unexpected content type" when asked to store an `undefined` attribute value).
|
||||
*
|
||||
* Only `undefined` keys are removed; `null`, `false`, `0`, and `""` are all
|
||||
* legitimate JSON-storable values and are preserved. Operates on a clone and
|
||||
* returns it; the input is never mutated. Defensively null-safe like the rest
|
||||
* of the file.
|
||||
*/
|
||||
export function sanitizeForYjs(doc: any): any {
|
||||
const out = clone(doc);
|
||||
|
||||
// Drop every key whose value is strictly `undefined` from an attrs object.
|
||||
const stripUndefined = (attrs: any): void => {
|
||||
if (!isObject(attrs)) return;
|
||||
for (const key of Object.keys(attrs)) {
|
||||
if (attrs[key] === undefined) {
|
||||
delete attrs[key];
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
const walk = (node: any): void => {
|
||||
if (!isObject(node)) return;
|
||||
stripUndefined(node.attrs);
|
||||
if (Array.isArray(node.marks)) {
|
||||
for (const mark of node.marks) {
|
||||
if (isObject(mark)) stripUndefined(mark.attrs);
|
||||
}
|
||||
}
|
||||
if (Array.isArray(node.content)) {
|
||||
for (const child of node.content) {
|
||||
walk(child);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
walk(out);
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Diagnostics helper: walk the tree and return a human-readable path string for
|
||||
* the FIRST attribute value (in any `node.attrs` or `mark.attrs`) that Yjs
|
||||
* cannot store — i.e. `undefined`, a `function`, a `symbol`, or a `bigint`
|
||||
* (e.g. `content[3].content[0].attrs.indent (undefined)`). Returns `null` when
|
||||
* every attribute is storable. Null-safe.
|
||||
*/
|
||||
export function findUnstorableAttr(doc: any): string | null {
|
||||
const isUnstorable = (value: any): string | null => {
|
||||
if (value === undefined) return "undefined";
|
||||
const t = typeof value;
|
||||
if (t === "function") return "function";
|
||||
if (t === "symbol") return "symbol";
|
||||
if (t === "bigint") return "bigint";
|
||||
return null;
|
||||
};
|
||||
|
||||
// Check an attrs object; return the offending sub-path or null.
|
||||
const checkAttrs = (attrs: any, basePath: string): string | null => {
|
||||
if (!isObject(attrs)) return null;
|
||||
for (const key of Object.keys(attrs)) {
|
||||
const kind = isUnstorable(attrs[key]);
|
||||
if (kind != null) return `${basePath}.${key} (${kind})`;
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
const walk = (node: any, path: string): string | null => {
|
||||
if (!isObject(node)) return null;
|
||||
const attrHit = checkAttrs(node.attrs, `${path}.attrs`);
|
||||
if (attrHit != null) return attrHit;
|
||||
if (Array.isArray(node.marks)) {
|
||||
for (let i = 0; i < node.marks.length; i++) {
|
||||
const markHit = checkAttrs(
|
||||
node.marks[i]?.attrs,
|
||||
`${path}.marks[${i}].attrs`,
|
||||
);
|
||||
if (markHit != null) return markHit;
|
||||
}
|
||||
}
|
||||
if (Array.isArray(node.content)) {
|
||||
for (let i = 0; i < node.content.length; i++) {
|
||||
const childHit = walk(node.content[i], `${path}.content[${i}]`);
|
||||
if (childHit != null) return childHit;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
// The root doc node carries no useful index, so start the path at "doc".
|
||||
if (!isObject(doc)) return null;
|
||||
const attrHit = checkAttrs(doc.attrs, "attrs");
|
||||
if (attrHit != null) return attrHit;
|
||||
if (Array.isArray(doc.content)) {
|
||||
for (let i = 0; i < doc.content.length; i++) {
|
||||
const childHit = walk(doc.content[i], `content[${i}]`);
|
||||
if (childHit != null) return childHit;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Table structural node types and the container each must live directly inside.
|
||||
* Used by `insertNodeRelative` to splice rows/cells into the correct ancestor
|
||||
* rather than blindly into the anchor's direct parent (which would corrupt the
|
||||
* table's nesting).
|
||||
*/
|
||||
const STRUCTURAL_TYPES = new Set(["tableRow", "tableCell", "tableHeader"]);
|
||||
const REQUIRED_CONTAINER: Record<string, string> = {
|
||||
tableRow: "table",
|
||||
tableCell: "tableRow",
|
||||
tableHeader: "tableRow",
|
||||
};
|
||||
|
||||
/**
|
||||
* Locate an anchor and return its ancestor chain (from `doc` down to and
|
||||
* including the matched node). Each chain entry is `{ node, index }` where
|
||||
* `index` is the node's position inside its parent's `content` array (the root
|
||||
* doc has index -1). Returns `null` when the anchor cannot be resolved.
|
||||
*/
|
||||
function findAnchorChain(
|
||||
doc: any,
|
||||
opts: InsertOptions,
|
||||
): { node: any; index: number }[] | null {
|
||||
if (!isObject(doc)) return null;
|
||||
|
||||
// DFS by id anywhere in the tree, accumulating the path.
|
||||
if (opts.anchorNodeId != null) {
|
||||
const targetId = opts.anchorNodeId;
|
||||
const search = (
|
||||
node: any,
|
||||
index: number,
|
||||
trail: { node: any; index: number }[],
|
||||
): { node: any; index: number }[] | null => {
|
||||
if (!isObject(node)) return null;
|
||||
const here = [...trail, { node, index }];
|
||||
if (matchesId(node, targetId)) return here;
|
||||
if (Array.isArray(node.content)) {
|
||||
for (let i = 0; i < node.content.length; i++) {
|
||||
const hit = search(node.content[i], i, here);
|
||||
if (hit != null) return hit;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
};
|
||||
return search(doc, -1, []);
|
||||
}
|
||||
|
||||
// By text: only top-level blocks are scanned (same rule as the JSON path).
|
||||
if (opts.anchorText != null && Array.isArray(doc.content)) {
|
||||
for (let i = 0; i < doc.content.length; i++) {
|
||||
if (blockPlainText(doc.content[i]).includes(opts.anchorText)) {
|
||||
return [
|
||||
{ node: doc, index: -1 },
|
||||
{ node: doc.content[i], index: i },
|
||||
];
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Options controlling where `insertNodeRelative` places the new node. */
|
||||
export interface InsertOptions {
|
||||
position: "before" | "after" | "append";
|
||||
/** Resolve the anchor by node id anywhere in the tree (preferred). */
|
||||
anchorNodeId?: string;
|
||||
/** Fallback: first TOP-LEVEL block whose plain text includes this string. */
|
||||
anchorText?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Insert a deep clone of `node` relative to an anchor.
|
||||
*
|
||||
* - position "append": push the node onto the top-level `doc.content`.
|
||||
* - position "before"/"after": locate the anchor and splice the node into the
|
||||
* anchor's parent `content` array immediately before / after it.
|
||||
*
|
||||
* Anchor resolution for before/after:
|
||||
* - if `anchorNodeId` is given, find the node with `attrs.id === anchorNodeId`
|
||||
* anywhere in the tree (recursive);
|
||||
* - otherwise, if `anchorText` is given, scan only TOP-LEVEL `doc.content`
|
||||
* blocks and pick the first whose `blockPlainText` includes `anchorText`.
|
||||
*
|
||||
* Operates on a clone of `doc`; returns `{ doc, inserted }`. `inserted` is
|
||||
* false when the anchor could not be resolved (the doc is returned unchanged
|
||||
* apart from being cloned).
|
||||
*/
|
||||
export function insertNodeRelative(
|
||||
doc: any,
|
||||
node: any,
|
||||
opts: InsertOptions,
|
||||
): { doc: any; inserted: boolean } {
|
||||
const out = clone(doc);
|
||||
const fresh = clone(node);
|
||||
|
||||
// Defensive: stay null-safe like the other exports — a missing opts means
|
||||
// there is nothing actionable to do.
|
||||
if (!isObject(opts)) return { doc: out, inserted: false };
|
||||
|
||||
const isStructural = isObject(node) && STRUCTURAL_TYPES.has(node.type);
|
||||
|
||||
// "append": top-level push.
|
||||
if (opts.position === "append") {
|
||||
// Structural table nodes (tableRow/tableCell/tableHeader) cannot live at the
|
||||
// top level — appending one would produce invalid nesting.
|
||||
if (isStructural) {
|
||||
throw new Error(
|
||||
`insert_node: cannot append a ${node.type} at the top level; use ` +
|
||||
`position before/after with an anchor inside the target table`,
|
||||
);
|
||||
}
|
||||
if (isObject(out)) {
|
||||
if (!Array.isArray(out.content)) out.content = [];
|
||||
out.content.push(fresh);
|
||||
return { doc: out, inserted: true };
|
||||
}
|
||||
return { doc: out, inserted: false };
|
||||
}
|
||||
|
||||
const offset = opts.position === "after" ? 1 : 0;
|
||||
|
||||
// Structural insert (before/after a tableRow/tableCell/tableHeader): splice
|
||||
// into the nearest enclosing table/tableRow rather than the anchor's direct
|
||||
// parent, so the row/cell lands at the correct level of the table.
|
||||
if (isStructural) {
|
||||
const containerType = REQUIRED_CONTAINER[node.type];
|
||||
const chain = findAnchorChain(out, opts);
|
||||
// Anchor not resolved at all — keep the existing "anchor not found" path.
|
||||
if (chain == null) return { doc: out, inserted: false };
|
||||
|
||||
// Find the DEEPEST ancestor (including the anchor itself) of the required
|
||||
// container type.
|
||||
let containerIdx = -1;
|
||||
for (let i = chain.length - 1; i >= 0; i--) {
|
||||
if (isObject(chain[i].node) && chain[i].node.type === containerType) {
|
||||
containerIdx = i;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (containerIdx === -1) {
|
||||
throw new Error(
|
||||
`insert_node: cannot insert a ${node.type} here — the anchor is not ` +
|
||||
`inside a ${containerType}. Anchor on a cell's text or a block id ` +
|
||||
`that lives inside the target table.`,
|
||||
);
|
||||
}
|
||||
|
||||
const container = chain[containerIdx].node;
|
||||
if (!Array.isArray(container.content)) container.content = [];
|
||||
|
||||
if (containerIdx === chain.length - 1) {
|
||||
// The matched container IS the anchor node itself (e.g. anchorText
|
||||
// resolved to the table block): append/prepend within it.
|
||||
const at = opts.position === "after" ? container.content.length : 0;
|
||||
container.content.splice(at, 0, fresh);
|
||||
} else {
|
||||
// The immediate child on the path leading to the anchor is the row/cell
|
||||
// to splice next to.
|
||||
const enclosingChildIndex = chain[containerIdx + 1].index;
|
||||
container.content.splice(enclosingChildIndex + offset, 0, fresh);
|
||||
}
|
||||
return { doc: out, inserted: true };
|
||||
}
|
||||
|
||||
// Resolve by id anywhere in the tree: splice into the parent content array.
|
||||
if (opts.anchorNodeId != null) {
|
||||
let inserted = false;
|
||||
const walkContent = (content: any[]): void => {
|
||||
for (let i = 0; i < content.length; i++) {
|
||||
const child = content[i];
|
||||
if (matchesId(child, opts.anchorNodeId as string)) {
|
||||
content.splice(i + offset, 0, fresh);
|
||||
inserted = true;
|
||||
return;
|
||||
}
|
||||
if (isObject(child) && Array.isArray(child.content)) {
|
||||
walkContent(child.content);
|
||||
if (inserted) return;
|
||||
}
|
||||
}
|
||||
};
|
||||
if (isObject(out) && Array.isArray(out.content)) {
|
||||
walkContent(out.content);
|
||||
}
|
||||
return { doc: out, inserted };
|
||||
}
|
||||
|
||||
// Resolve by text: only top-level doc.content blocks are scanned.
|
||||
if (opts.anchorText != null && isObject(out) && Array.isArray(out.content)) {
|
||||
for (let i = 0; i < out.content.length; i++) {
|
||||
if (blockPlainText(out.content[i]).includes(opts.anchorText)) {
|
||||
out.content.splice(i + offset, 0, fresh);
|
||||
return { doc: out, inserted: true };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { doc: out, inserted: false };
|
||||
}
|
||||
|
||||
// ===========================================================================
|
||||
// Table editing helpers
|
||||
//
|
||||
// A Docmost table is a ProseMirror subtree with NO ids on the structural nodes:
|
||||
// table -> { type:"table", content:[tableRow...] }
|
||||
// row -> { type:"tableRow", content:[tableCell|tableHeader...] }
|
||||
// cell -> { type:"tableCell"|"tableHeader", attrs:{colspan,rowspan,colwidth},
|
||||
// content:[paragraph...] }
|
||||
// para -> { type:"paragraph", attrs:{id,indent}, content:[textNode...] }
|
||||
// Only paragraphs/headings carry an `attrs.id`, so a cell is addressed via the
|
||||
// id of the paragraph inside it. The helpers below all operate on a DEEP CLONE
|
||||
// of the input doc (via `clone`) and never mutate their inputs.
|
||||
// ===========================================================================
|
||||
|
||||
/**
|
||||
* Collect EVERY `attrs.id` present anywhere in `node` into `used`. Used to seed
|
||||
* `makeFreshId` so generated paragraph ids never collide with existing ones.
|
||||
*/
|
||||
function collectIds(node: any, used: Set<string>): void {
|
||||
if (!isObject(node)) return;
|
||||
if (isObject(node.attrs) && typeof node.attrs.id === "string") {
|
||||
used.add(node.attrs.id);
|
||||
}
|
||||
if (Array.isArray(node.content)) {
|
||||
for (const child of node.content) collectIds(child, used);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fresh-id generator: returns a random Docmost-style id (12 chars from
|
||||
* lowercase `a-z0-9`) that is not already in `used`, and records it. On the
|
||||
* rare collision the id is regenerated. Callers rely on uniqueness, not on the
|
||||
* exact string, so randomness is fine — and unlike a module-local counter it
|
||||
* needs no reset and cannot become predictable across calls.
|
||||
*/
|
||||
function makeFreshId(used: Set<string>): string {
|
||||
const alphabet = "abcdefghijklmnopqrstuvwxyz0123456789";
|
||||
let id: string;
|
||||
do {
|
||||
id = "";
|
||||
for (let i = 0; i < 12; i++) {
|
||||
id += alphabet[Math.floor(Math.random() * alphabet.length)];
|
||||
}
|
||||
} while (used.has(id) || id === "");
|
||||
used.add(id);
|
||||
return id;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a table reference against an ALREADY-CLONED doc and return the LIVE
|
||||
* table node (a reference inside `rootClone`, so the caller may mutate it) plus
|
||||
* its index path. Returns null when no table matches.
|
||||
*
|
||||
* - `#<n>`: the top-level block at index `n`, only if its `type === "table"`.
|
||||
* - otherwise: DFS for the node with `attrs.id === tableRef`, then walk UP its
|
||||
* ancestor chain to the nearest `type === "table"` ancestor.
|
||||
*/
|
||||
function locateTable(
|
||||
rootClone: any,
|
||||
tableRef: string,
|
||||
): { table: any; path: number[] } | null {
|
||||
if (!isObject(rootClone)) return null;
|
||||
|
||||
// "#<n>": index into the top-level content array; must be a table.
|
||||
const indexMatch = typeof tableRef === "string" ? tableRef.match(/^#(\d+)$/) : null;
|
||||
if (indexMatch) {
|
||||
const index = Number(indexMatch[1]);
|
||||
const block = Array.isArray(rootClone.content)
|
||||
? rootClone.content[index]
|
||||
: undefined;
|
||||
if (isObject(block) && block.type === "table") {
|
||||
return { table: block, path: [index] };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
// Otherwise: DFS for attrs.id === tableRef, tracking the ancestor chain, then
|
||||
// climb to the nearest enclosing table.
|
||||
const search = (
|
||||
node: any,
|
||||
trail: { node: any; index: number }[],
|
||||
): { table: any; path: number[] } | null => {
|
||||
if (!isObject(node)) return null;
|
||||
if (Array.isArray(node.content)) {
|
||||
for (let i = 0; i < node.content.length; i++) {
|
||||
const child = node.content[i];
|
||||
const here = [...trail, { node: child, index: i }];
|
||||
if (matchesId(child, tableRef)) {
|
||||
// Walk UP to the nearest table ancestor (including the match itself).
|
||||
for (let j = here.length - 1; j >= 0; j--) {
|
||||
if (isObject(here[j].node) && here[j].node.type === "table") {
|
||||
return {
|
||||
table: here[j].node,
|
||||
path: here.slice(0, j + 1).map((e) => e.index),
|
||||
};
|
||||
}
|
||||
}
|
||||
return null; // id found but no enclosing table
|
||||
}
|
||||
const hit = search(child, here);
|
||||
if (hit != null) return hit;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
return search(rootClone, []);
|
||||
}
|
||||
|
||||
/** Build the plain-text → single-paragraph cell content used by all writers. */
|
||||
function makeCellParagraph(id: string, text: string): any {
|
||||
return {
|
||||
type: "paragraph",
|
||||
attrs: { id, indent: 0 },
|
||||
// Empty string → a paragraph with an empty content array.
|
||||
content: text ? [{ type: "text", text }] : [],
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a table as a matrix. Returns null when `tableRef` resolves to no table.
|
||||
*
|
||||
* - `rows`/`cols`: the table's row count and the column count of its FIRST row.
|
||||
* Tables may be ragged (rows of differing length), so `cols` reflects only
|
||||
* row 0; use the per-row length of `cells`/`cellIds` for each row's actual
|
||||
* width.
|
||||
* - `cells`: `string[][]` of each cell's `blockPlainText`.
|
||||
* - `cellIds`: `(string|null)[][]` of each cell's FIRST paragraph id (or null),
|
||||
* so callers can `patch_node` a cell for rich-formatted edits.
|
||||
* - `path`: index path of the table within the doc.
|
||||
*/
|
||||
export function readTable(
|
||||
doc: any,
|
||||
tableRef: string,
|
||||
): {
|
||||
rows: number;
|
||||
cols: number;
|
||||
cells: string[][];
|
||||
cellIds: (string | null)[][];
|
||||
path: number[];
|
||||
} | null {
|
||||
const root = clone(doc);
|
||||
const located = locateTable(root, tableRef);
|
||||
if (located == null) return null;
|
||||
const { table, path } = located;
|
||||
|
||||
const rowNodes = Array.isArray(table.content) ? table.content : [];
|
||||
const rows = rowNodes.length;
|
||||
const cols = rowNodes[0]?.content?.length ?? 0;
|
||||
|
||||
const cells: string[][] = [];
|
||||
const cellIds: (string | null)[][] = [];
|
||||
for (const rowNode of rowNodes) {
|
||||
const cellNodes = Array.isArray(rowNode?.content) ? rowNode.content : [];
|
||||
const rowText: string[] = [];
|
||||
const rowIds: (string | null)[] = [];
|
||||
for (const cellNode of cellNodes) {
|
||||
rowText.push(blockPlainText(cellNode));
|
||||
// The cell's first paragraph carries the id used for patch_node.
|
||||
const firstPara = Array.isArray(cellNode?.content)
|
||||
? cellNode.content[0]
|
||||
: undefined;
|
||||
const id =
|
||||
isObject(firstPara) && isObject(firstPara.attrs)
|
||||
? firstPara.attrs.id ?? null
|
||||
: null;
|
||||
rowIds.push(id);
|
||||
}
|
||||
cells.push(rowText);
|
||||
cellIds.push(rowIds);
|
||||
}
|
||||
|
||||
return { rows, cols, cells, cellIds, path };
|
||||
}
|
||||
|
||||
/**
|
||||
* Insert a row of plain-text cells into a table. Returns `{ doc, inserted }`.
|
||||
*
|
||||
* The row is padded to the table's column count (`cells[i] ?? ""`); supplying
|
||||
* MORE cells than columns throws. Each new cell copies `colwidth` for its
|
||||
* column from the header row when present, gets a fresh-id paragraph, and a
|
||||
* `colspan:1, rowspan:1` attrs. `index` (when an integer in `[0, rows]`) splices
|
||||
* the row there; otherwise the row is appended at the end.
|
||||
*/
|
||||
export function insertTableRow(
|
||||
doc: any,
|
||||
tableRef: string,
|
||||
cells: string[],
|
||||
index?: number,
|
||||
): { doc: any; inserted: boolean } {
|
||||
const out = clone(doc);
|
||||
const located = locateTable(out, tableRef);
|
||||
if (located == null) return { doc: out, inserted: false };
|
||||
const { table } = located;
|
||||
|
||||
if (!Array.isArray(table.content)) table.content = [];
|
||||
const rows = table.content.length;
|
||||
const headerRow = table.content[0];
|
||||
const headerCells = Array.isArray(headerRow?.content) ? headerRow.content : [];
|
||||
|
||||
// Column count is the WIDEST existing row, so the guard below stays
|
||||
// meaningful for ragged tables and the new row matches the table's width.
|
||||
// Fall back to the supplied cell count only when the table has no rows.
|
||||
let colCount = 0;
|
||||
for (const r of table.content) {
|
||||
if (isObject(r) && Array.isArray(r.content)) colCount = Math.max(colCount, r.content.length);
|
||||
}
|
||||
if (colCount === 0) colCount = Array.isArray(cells) ? cells.length : 0;
|
||||
|
||||
if (Array.isArray(cells) && cells.length > colCount) {
|
||||
throw new Error(
|
||||
`table_insert_row: got ${cells.length} cell(s) but the table has ${colCount} column(s)`,
|
||||
);
|
||||
}
|
||||
|
||||
// Resolve the landing index up front so the cell-type decision and the splice
|
||||
// below agree: a valid integer in [0, rows] splices there, else we append.
|
||||
const landingIndex =
|
||||
typeof index === "number" && Number.isInteger(index) && index >= 0 && index <= rows
|
||||
? index
|
||||
: rows;
|
||||
|
||||
// Seed the id generator with every id already in the doc so the new cell
|
||||
// paragraph ids are unique within the whole document.
|
||||
const used = new Set<string>();
|
||||
collectIds(out, used);
|
||||
|
||||
const newCells: any[] = [];
|
||||
for (let i = 0; i < colCount; i++) {
|
||||
const text = (Array.isArray(cells) ? cells[i] : undefined) ?? "";
|
||||
const attrs: Record<string, any> = { colspan: 1, rowspan: 1 };
|
||||
// Copy this column's colwidth from the header row's cell when present.
|
||||
const colwidth = headerCells[i]?.attrs?.colwidth;
|
||||
if (colwidth !== undefined) attrs.colwidth = colwidth;
|
||||
// A row landing at index 0 becomes the new header row, so inherit the
|
||||
// current header cell's type per column (Docmost uses "tableHeader" there);
|
||||
// every other position is a plain data cell.
|
||||
const cellType = landingIndex === 0 ? headerCells[i]?.type ?? "tableCell" : "tableCell";
|
||||
newCells.push({
|
||||
type: cellType,
|
||||
attrs,
|
||||
content: [makeCellParagraph(makeFreshId(used), text)],
|
||||
});
|
||||
}
|
||||
|
||||
const newRow = { type: "tableRow", content: newCells };
|
||||
|
||||
// Splice at the resolved landing index (append when index was omitted/invalid).
|
||||
table.content.splice(landingIndex, 0, newRow);
|
||||
|
||||
return { doc: out, inserted: true };
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete the row at 0-based `index` from a table. Returns `{ doc, deleted }`.
|
||||
* `deleted` is false only when the table cannot be located. Throws on an
|
||||
* out-of-range index, and refuses to delete the table's only row.
|
||||
*/
|
||||
export function deleteTableRow(
|
||||
doc: any,
|
||||
tableRef: string,
|
||||
index: number,
|
||||
): { doc: any; deleted: boolean } {
|
||||
const out = clone(doc);
|
||||
const located = locateTable(out, tableRef);
|
||||
if (located == null) return { doc: out, deleted: false };
|
||||
const { table } = located;
|
||||
|
||||
if (!Array.isArray(table.content)) table.content = [];
|
||||
const rows = table.content.length;
|
||||
|
||||
if (!Number.isInteger(index) || index < 0 || index >= rows) {
|
||||
throw new Error(
|
||||
`table_delete_row: row index ${index} out of range (table has ${rows} row(s))`,
|
||||
);
|
||||
}
|
||||
if (rows <= 1) {
|
||||
throw new Error(
|
||||
"table_delete_row: refusing to delete the only row of the table",
|
||||
);
|
||||
}
|
||||
|
||||
table.content.splice(index, 1);
|
||||
return { doc: out, deleted: true };
|
||||
}
|
||||
|
||||
/**
|
||||
* Set the plain-text content of cell `[row, col]` (0-based) to `text`. Returns
|
||||
* `{ doc, updated }`; `updated` is false only when the table cannot be located.
|
||||
* Throws when `row`/`col` is out of range. The cell's own attrs (colspan/
|
||||
* rowspan/colwidth) are preserved; its content becomes a single text paragraph
|
||||
* that reuses the cell's existing first-paragraph id when present, else a fresh
|
||||
* one.
|
||||
*/
|
||||
export function updateTableCell(
|
||||
doc: any,
|
||||
tableRef: string,
|
||||
row: number,
|
||||
col: number,
|
||||
text: string,
|
||||
): { doc: any; updated: boolean } {
|
||||
const out = clone(doc);
|
||||
const located = locateTable(out, tableRef);
|
||||
if (located == null) return { doc: out, updated: false };
|
||||
const { table } = located;
|
||||
|
||||
const rowNodes = Array.isArray(table.content) ? table.content : [];
|
||||
const rows = rowNodes.length;
|
||||
const rowNode = rowNodes[row];
|
||||
const cols = isObject(rowNode) && Array.isArray(rowNode.content)
|
||||
? rowNode.content.length
|
||||
: 0;
|
||||
|
||||
if (
|
||||
!Number.isInteger(row) ||
|
||||
row < 0 ||
|
||||
row >= rows ||
|
||||
!Number.isInteger(col) ||
|
||||
col < 0 ||
|
||||
col >= cols
|
||||
) {
|
||||
throw new Error(`table_update_cell: cell [${row},${col}] out of range`);
|
||||
}
|
||||
|
||||
const cellNode = rowNode.content[col];
|
||||
// Reuse the cell's existing first-paragraph id, or mint a fresh unique one.
|
||||
const existingPara = Array.isArray(cellNode?.content)
|
||||
? cellNode.content[0]
|
||||
: undefined;
|
||||
let id =
|
||||
isObject(existingPara) && isObject(existingPara.attrs)
|
||||
? existingPara.attrs.id
|
||||
: undefined;
|
||||
if (typeof id !== "string" || id.length === 0) {
|
||||
const used = new Set<string>();
|
||||
collectIds(out, used);
|
||||
id = makeFreshId(used);
|
||||
}
|
||||
|
||||
cellNode.content = [makeCellParagraph(id, text)];
|
||||
return { doc: out, updated: true };
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
/**
|
||||
* The native-Obsidian page-file format (design: docs/backlog/git-sync-thin-meta.md).
|
||||
* A page file is CLEAN markdown with a minimal YAML frontmatter carrying ONLY the
|
||||
* page's durable identity:
|
||||
*
|
||||
* ---
|
||||
* gitmost_id: 019ef6fc-2638-7ce1-9ce3-2756ce038480
|
||||
* ---
|
||||
* <clean markdown body>
|
||||
*
|
||||
* Everything else is derived (title = filename, parentPageId = enclosing folder,
|
||||
* spaceId = the vault, updatedAt = git). `gitmost_id` (a Docmost pageId) is the
|
||||
* only non-derivable bit and travels WITH the file so identity survives any move,
|
||||
* even one git's rename detection misses. Third-party editors (Obsidian, …) see
|
||||
* clean markdown; the frontmatter is hidden in their preview.
|
||||
*
|
||||
* No backward-compat with the old `docmost:meta` format: vaults are a cache, wiped
|
||||
* and rebuilt native. A file WITHOUT a `gitmost_id` frontmatter is an un-tracked
|
||||
* (e.g. hand-written) file -> the caller ADOPTS it (creates a page, writes the id).
|
||||
*/
|
||||
|
||||
/**
|
||||
* The frontmatter key carrying the Docmost pageId. NAMESPACED (not a bare `id`)
|
||||
* so it never collides with a user's own frontmatter fields.
|
||||
*/
|
||||
export const ID_KEY = "gitmost_id";
|
||||
|
||||
/** Leading YAML frontmatter block: `---\n…\n---` at the very start of the file. */
|
||||
const FRONTMATTER_RE = /^?---\n([\s\S]*?)\n---\n?/;
|
||||
|
||||
/** The top-level `<ID_KEY>: <value>` line inside the frontmatter (quotes optional). */
|
||||
function readIdFromYaml(yaml: string): string | null {
|
||||
const re = new RegExp(`^${ID_KEY}:\\s*(.+?)\\s*$`);
|
||||
for (const line of yaml.split("\n")) {
|
||||
const m = line.match(re);
|
||||
if (m) {
|
||||
const v = m[1].trim().replace(/^["']|["']$/g, "");
|
||||
return v === "" ? null : v;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a page file into its identity (`id`) and clean markdown `body`. Tolerant:
|
||||
* a file with no frontmatter (a hand-written third-party file) returns `id: null`
|
||||
* and the whole text as the body — the caller then ADOPTS it (creates a page,
|
||||
* writes the id back).
|
||||
*
|
||||
* KNOWN LIMITATION (phase 4 — adoption, see docs/backlog/git-sync-thin-meta.md):
|
||||
* a leading frontmatter block is stripped from `body` even when it carries NO
|
||||
* `gitmost_id` but DOES carry the user's own Obsidian properties (`tags:` etc.).
|
||||
* On adoption those fields are not yet round-tripped — `serializePageFile`
|
||||
* write-back persists only `gitmost_id`. Preserving arbitrary user frontmatter
|
||||
* across the Docmost round-trip (BOTH adoption write-back AND the next pull's
|
||||
* re-serialize) is deferred to the adoption phase; until then, do NOT roll the
|
||||
* native format onto a real Obsidian vault whose notes carry properties.
|
||||
*/
|
||||
export function parsePageFile(full: string): {
|
||||
id: string | null;
|
||||
body: string;
|
||||
} {
|
||||
const text = (full ?? "").replace(/\r\n/g, "\n");
|
||||
|
||||
// Native format: a `gitmost_id` YAML frontmatter. Anything else (no frontmatter,
|
||||
// or frontmatter without the key) is an un-tracked file -> adopt.
|
||||
const fm = text.match(FRONTMATTER_RE);
|
||||
if (fm) {
|
||||
return { id: readIdFromYaml(fm[1]), body: text.slice(fm[0].length).trim() };
|
||||
}
|
||||
return { id: null, body: text.trim() };
|
||||
}
|
||||
|
||||
/**
|
||||
* Serialize a page into the thin format: `id` frontmatter + a blank line + the
|
||||
* clean body + a trailing newline. Deterministic so an unchanged page re-syncs to
|
||||
* byte-identical output (no churn — the loop-guard relies on it).
|
||||
*/
|
||||
export function serializePageFile(id: string, body: string): string {
|
||||
return `---\n${ID_KEY}: ${id}\n---\n\n${body.trim()}\n`;
|
||||
}
|
||||
Reference in New Issue
Block a user