Compare commits
18 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 14d7b21df0 | |||
| 9a8671c3af | |||
| bec2156e96 | |||
| acc705de19 | |||
| a49872444e | |||
| ffc38ea2ca | |||
| 2a951df096 | |||
| 5dc7a2703f | |||
| bfb4c8d8d0 | |||
| f794ac6d6c | |||
| e4e788f151 | |||
| 4be4a75fa3 | |||
| e6171a1810 | |||
| 2f23cf4b65 | |||
| d269bd9efe | |||
| d287c15db4 | |||
| 7cb3199d09 | |||
| e19275e96e |
@@ -124,6 +124,40 @@ MCP_DOCMOST_PASSWORD=
|
||||
# MCP_TOKEN=
|
||||
# MCP_SESSION_IDLE_MS=1800000
|
||||
#
|
||||
# --- MCP collaboration write path: concurrency + rights-staleness (#449) ------
|
||||
# MCP content writes (update_page, insert/replace nodes, comments-in-body, etc.)
|
||||
# go over the collaboration websocket and are serialized PER PAGE by an
|
||||
# in-process mutex (a module-level Map, one promise-chain per page UUID). This
|
||||
# guarantees no two MCP writes on the SAME page overlap and clobber each other.
|
||||
#
|
||||
# DEPLOY REQUIREMENT — SINGLE INSTANCE or STICKY SESSIONS. The mutex is
|
||||
# process-local. Behind a multi-replica load balancer WITHOUT sticky sessions,
|
||||
# two replicas can each "hold" the lock for the same page at the same time and
|
||||
# serialization is silently lost (concurrent full-document writes race on the
|
||||
# live Yjs fragment). Run the MCP/app as a SINGLE instance, OR pin a page's
|
||||
# traffic to one replica (sticky sessions / consistent hashing on page id). The
|
||||
# same constraint applies to the RAM-only stash_page blob store above. There is
|
||||
# deliberately no cross-process (e.g. Postgres advisory) lock yet — this is a
|
||||
# CONSCIOUS documented constraint, not an oversight (#449).
|
||||
#
|
||||
# To reduce connect-storms the write path caches ONE live collab session per
|
||||
# (wsUrl, page, token). Tunables (all optional; defaults are safe):
|
||||
# MCP_COLLAB_SESSION_IDLE_MS=60000 # idle TTL, reset per op; 0 disables cache
|
||||
# MCP_COLLAB_SESSION_MAX_ENTRIES=32 # LRU cap on cached sessions
|
||||
# MCP_COLLAB_TOKEN_TTL_MS=300000 # per-client collab-token cache (5 min)
|
||||
#
|
||||
# RIGHTS-STALENESS TRADE-OFF. A cached collab session writes under the token
|
||||
# captured at CONNECT time, and the collab-token cache reuses a token for its TTL.
|
||||
# So if a user's access to a page is REVOKED, MCP writes on an already-open
|
||||
# session may keep succeeding until the session ages out. MCP_COLLAB_SESSION_MAX_AGE_MS
|
||||
# is the HARD lifetime (checked at each acquire) that BOUNDS this window: after it,
|
||||
# the session is torn down and the next write re-auths with a fresh token, picking
|
||||
# up the revocation. Default 10 min. LOWER it to shorten the revocation lag at the
|
||||
# cost of more reconnects; RAISE it to reduce reconnects at the cost of a longer
|
||||
# stale-rights window. There is intentionally no push-based cache invalidation on
|
||||
# a rights change — this bounded window is the accepted trade-off (#449).
|
||||
# MCP_COLLAB_SESSION_MAX_AGE_MS=600000
|
||||
#
|
||||
# BLOB SANDBOX (stash_page). An in-RAM, process-local store that hands large page
|
||||
# content + images to an external consumer WITHOUT bloating the model context or
|
||||
# requiring Docmost auth. The stash_page tool serializes a page, mirrors its
|
||||
|
||||
@@ -157,6 +157,12 @@ jobs:
|
||||
- name: Build prosemirror-markdown
|
||||
run: pnpm --filter @docmost/prosemirror-markdown build
|
||||
|
||||
# docmost-client.loader.ts type-imports from @docmost/mcp (issue #446); its
|
||||
# build/ is gitignored and `test:e2e` type-checks, so build it here or tsc
|
||||
# fails with TS2307 (mirrors the e2e-mcp / mcp-server-parity jobs).
|
||||
- name: Build mcp
|
||||
run: pnpm --filter @docmost/mcp build
|
||||
|
||||
- name: Run migrations
|
||||
run: pnpm --filter ./apps/server migration:latest
|
||||
|
||||
|
||||
+111
-7
@@ -12,20 +12,108 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
- **External MCP tool names are now camelCase (all renamed).** Every tool on the
|
||||
external `/mcp` surface was renamed from `snake_case` to `camelCase`, so the
|
||||
external MCP name now matches the in-app tool name exactly (one logical tool,
|
||||
one name everywhere). For example `get_node` → `getNode`, `edit_page_text` →
|
||||
`editPageText`, `patch_node` → `patchNode`. The tools' behaviour, inputs and
|
||||
outputs are unchanged — only the names change. The single-word `search`
|
||||
keeps its name.
|
||||
|
||||
*Migration (external MCP clients only — the in-app AI agent already used these
|
||||
names and is unaffected):* update anything that refers to a tool by its
|
||||
string name — permission allowlists (`mcp__gitmost-*__get_node` →
|
||||
`mcp__gitmost-*__getNode`), saved prompts/skills, `.mcp.json` tool filters,
|
||||
and metrics dashboards that group by the `tool` label — and roll it out in
|
||||
lockstep with this deploy, because the old snake_case names stop resolving.
|
||||
Released together with the `import_page_markdown`/`update_page_markdown`
|
||||
change below so external configs break exactly once.
|
||||
|
||||
Full mapping (old → new):
|
||||
|
||||
| Old (snake_case) | New (camelCase) |
|
||||
| --- | --- |
|
||||
| `check_new_comments` | `checkNewComments` |
|
||||
| `copy_page_content` | `copyPageContent` |
|
||||
| `create_comment` | `createComment` |
|
||||
| `create_page` | `createPage` |
|
||||
| `delete_comment` | `deleteComment` |
|
||||
| `delete_node` | `deleteNode` |
|
||||
| `delete_page` | `deletePage` |
|
||||
| `diff_page_versions` | `diffPageVersions` |
|
||||
| `docmost_transform` | `docmostTransform` |
|
||||
| `drawio_create` | `drawioCreate` |
|
||||
| `drawio_get` | `drawioGet` |
|
||||
| `drawio_guide` | `drawioGuide` |
|
||||
| `drawio_shapes` | `drawioShapes` |
|
||||
| `drawio_update` | `drawioUpdate` |
|
||||
| `edit_page_text` | `editPageText` |
|
||||
| `export_page_markdown` | `exportPageMarkdown` |
|
||||
| `get_node` | `getNode` |
|
||||
| `get_outline` | `getOutline` |
|
||||
| `get_page` | `getPage` |
|
||||
| `get_page_json` | `getPageJson` |
|
||||
| `get_workspace` | `getWorkspace` |
|
||||
| `insert_footnote` | `insertFootnote` |
|
||||
| `insert_image` | `insertImage` |
|
||||
| `insert_node` | `insertNode` |
|
||||
| `list_comments` | `listComments` |
|
||||
| `list_page_history` | `listPageHistory` |
|
||||
| `list_pages` | `listPages` |
|
||||
| `list_shares` | `listShares` |
|
||||
| `list_spaces` | `listSpaces` |
|
||||
| `move_page` | `movePage` |
|
||||
| `patch_node` | `patchNode` |
|
||||
| `rename_page` | `renamePage` |
|
||||
| `replace_image` | `replaceImage` |
|
||||
| `resolve_comment` | `resolveComment` |
|
||||
| `restore_page_version` | `restorePageVersion` |
|
||||
| `search` | `search` (unchanged) |
|
||||
| `search_in_page` | `searchInPage` |
|
||||
| `share_page` | `sharePage` |
|
||||
| `stash_page` | `stashPage` |
|
||||
| `table_delete_row` | `tableDeleteRow` |
|
||||
| `table_get` | `tableGet` |
|
||||
| `table_insert_row` | `tableInsertRow` |
|
||||
| `table_update_cell` | `tableUpdateCell` |
|
||||
| `unshare_page` | `unsharePage` |
|
||||
| `update_comment` | `updateComment` |
|
||||
| `update_page_json` | `updatePageJson` |
|
||||
| `update_page_markdown` | `updatePageMarkdown` |
|
||||
|
||||
(#412)
|
||||
|
||||
- **External MCP: `import_page_markdown` removed, `update_page_markdown` added.**
|
||||
The external `/mcp` surface no longer exposes `import_page_markdown` (the
|
||||
The external `/mcp` surface no longer exposes `importPageMarkdown` (the
|
||||
round-trip parser for a self-contained *exported* Docmost-Markdown file). In
|
||||
its place it now exposes **`update_page_markdown`** — a plain-Markdown
|
||||
its place it now exposes **`updatePageMarkdown`** — a plain-Markdown
|
||||
full-body replace (`{pageId, content, title?}`) that pairs with
|
||||
`update_page_json`, re-imports the whole body (block ids regenerate) and
|
||||
`updatePageJson`, re-imports the whole body (block ids regenerate) and
|
||||
parses Docmost-flavoured markdown including `^[...]` inline footnotes.
|
||||
*Migration:* MCP clients that called `import_page_markdown` to overwrite a
|
||||
page's body from Markdown should call `update_page_markdown` instead (pass the
|
||||
*Migration:* MCP clients that called `importPageMarkdown` to overwrite a
|
||||
page's body from Markdown should call `updatePageMarkdown` instead (pass the
|
||||
markdown as `content`). Round-tripping an exported Docmost-Markdown file with
|
||||
comment anchors/diagrams is no longer available on the external MCP surface;
|
||||
export remains via `export_page_markdown`. The in-app AI agent is unaffected —
|
||||
export remains via `exportPageMarkdown`. The in-app AI agent is unaffected —
|
||||
it keeps both `importPageMarkdown` and the renamed `updatePageMarkdown` (was
|
||||
`updatePageContent`). The total MCP tool count is unchanged (−1 / +1). (#411)
|
||||
`updatePageContent`). The total MCP tool count is unchanged (−1 / +1). The
|
||||
external names shown here are the post-#412 camelCase names. (#411)
|
||||
|
||||
- **`getNode` now returns Markdown by default (was ProseMirror JSON).** The
|
||||
block-level read/write tools default to Markdown so a block round trip is
|
||||
`getNode` (markdown) → edit → `patchNode` (markdown). `getNode` now returns
|
||||
`{ …, format: "markdown", markdown }` unless you pass `format: "json"` (which
|
||||
restores the previous `{ …, node }` ProseMirror subtree); comment anchors —
|
||||
including resolved ones — are preserved in the markdown so a write-back never
|
||||
orphans a thread, and a node that cannot be a document top-level block
|
||||
(`tableRow`/`tableCell`/`tableHeader` addressed via `#<index>`) auto-falls back
|
||||
to JSON with `format: "json"` in the response. `patchNode`/`insertNode` gain a
|
||||
`markdown` input alongside `node` (provide exactly one): the markdown fragment
|
||||
may rewrite/insert several blocks at once and supports `^[...]` footnotes.
|
||||
*Migration (external MCP clients only):* a client that consumed `getNode`'s
|
||||
`node` field must now either read `markdown`, or pass `format: "json"` to keep
|
||||
the old ProseMirror-JSON output. Released together with the `#411`/`#412`
|
||||
breaking window so external configs break exactly once. (#413)
|
||||
|
||||
### Added
|
||||
|
||||
@@ -163,6 +251,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
by physical key position and matched against the commands; genuine Cyrillic
|
||||
search terms keep priority over remapped candidates, and short wrong-layout
|
||||
prefixes match by command title. (#283, #285, #287)
|
||||
- **Opt-in substring "lookup" search mode for agents.** `/api/search` gains an
|
||||
additive, opt-in mode (guarded by a new `substring` flag) that matches literal
|
||||
substrings of page titles and body text — so technical tokens the full-text
|
||||
tokenizer mangles (`backup-srv.local`, `10.0.12.5`, `WB-MGE-30D86B`) are found
|
||||
even when the FTS query is empty. It returns a location `path`, a windowed
|
||||
`snippet` and a per-response relevance `score`, supports `titleOnly` and a
|
||||
`parentPageId` subtree scope, and applies the page-level permission filter
|
||||
before the limit. The web UI never sets `substring`, so its full-text search
|
||||
behaviour is byte-for-byte unchanged. The leading-wildcard `LIKE` predicates
|
||||
are backed by GIN trigram indexes on `LOWER(f_unaccent(title))` and
|
||||
`LOWER(f_unaccent(text_content))` so lookups use a bitmap index scan instead of
|
||||
a sequential scan. (#443)
|
||||
- **MCP `search` tool returns richer, agent-oriented results.** The external MCP
|
||||
`search` response shape changes for the agent surface: each hit now carries
|
||||
`pageId` (renamed from `id`), plus `path`, `snippet` and `score`; the
|
||||
UI-oriented `spaceId`, `rank` and `highlight` fields are dropped. (#443)
|
||||
|
||||
### Changed
|
||||
|
||||
|
||||
+10
@@ -81,4 +81,14 @@ VOLUME ["/app/data/storage"]
|
||||
|
||||
EXPOSE 3000
|
||||
|
||||
# DEPLOY REQUIREMENT — SINGLE INSTANCE or STICKY SESSIONS (#449).
|
||||
# MCP content writes are serialized per page by an IN-PROCESS mutex, and the
|
||||
# stash_page blob store + cached collab sessions are RAM-only and process-local.
|
||||
# Running MULTIPLE replicas of this image behind a load balancer WITHOUT sticky
|
||||
# sessions silently breaks per-page write serialization (two replicas can lock
|
||||
# the same page at once) and makes stash_page blobs unreachable across replicas.
|
||||
# Run a SINGLE instance, or pin each page's traffic to one replica (sticky
|
||||
# sessions / consistent hashing on page id). There is deliberately no
|
||||
# cross-process lock yet — a conscious constraint. See .env.example (the "MCP
|
||||
# collaboration write path" block) and packages/mcp/README.md for details.
|
||||
CMD ["pnpm", "start"]
|
||||
|
||||
@@ -25,7 +25,7 @@ const mockLoaded = (DocmostClient: loader.DocmostClientCtor) => ({
|
||||
DocmostClient,
|
||||
sharedToolSpecs: SHARED_TOOL_SPECS as unknown as Record<string, loader.SharedToolSpec>,
|
||||
// Pure no-network draw.io helpers (#424). Type-correct stubs: these tests
|
||||
// never execute the drawio_shapes / drawio_guide tool bodies.
|
||||
// never execute the drawioShapes / drawioGuide tool bodies.
|
||||
searchShapes: (() => []) as unknown as loader.SearchShapesFn,
|
||||
getGuideSection: (() => ({
|
||||
section: 'index',
|
||||
@@ -355,23 +355,32 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
|
||||
content: [{ type: 'text', text: 'Hello' }],
|
||||
};
|
||||
|
||||
it('patchNode parses a JSON-string node and forwards it as an object', async () => {
|
||||
it('patchNode parses a JSON-string node and forwards it as { node } (object)', async () => {
|
||||
const tools = await buildTools();
|
||||
await tools.patchNode.execute(
|
||||
{ pageId: 'p1', nodeId: 'n1', node: JSON.stringify(NODE_OBJ) } as never,
|
||||
{} as never,
|
||||
);
|
||||
expect(patchNodeCalls).toHaveLength(1);
|
||||
expect(patchNodeCalls[0]).toEqual(['p1', 'n1', NODE_OBJ]);
|
||||
// #413: the 3rd arg is now the XOR input { markdown?, node? }.
|
||||
expect(patchNodeCalls[0]).toEqual([
|
||||
'p1',
|
||||
'n1',
|
||||
{ markdown: undefined, node: NODE_OBJ },
|
||||
]);
|
||||
});
|
||||
|
||||
it('patchNode passes an object node through unchanged', async () => {
|
||||
it('patchNode passes an object node through unchanged inside { node }', async () => {
|
||||
const tools = await buildTools();
|
||||
await tools.patchNode.execute(
|
||||
{ pageId: 'p1', nodeId: 'n1', node: NODE_OBJ } as never,
|
||||
{} as never,
|
||||
);
|
||||
expect(patchNodeCalls[0]).toEqual(['p1', 'n1', NODE_OBJ]);
|
||||
expect(patchNodeCalls[0]).toEqual([
|
||||
'p1',
|
||||
'n1',
|
||||
{ markdown: undefined, node: NODE_OBJ },
|
||||
]);
|
||||
});
|
||||
|
||||
it('patchNode throws the documented message on invalid JSON string', async () => {
|
||||
@@ -385,7 +394,7 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
|
||||
expect(patchNodeCalls).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('insertNode parses a JSON-string node and forwards it as an object', async () => {
|
||||
it('insertNode parses a JSON-string node and forwards it inside { node }', async () => {
|
||||
const tools = await buildTools();
|
||||
await tools.insertNode.execute(
|
||||
{
|
||||
@@ -396,9 +405,15 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
|
||||
{} as never,
|
||||
);
|
||||
expect(insertNodeCalls).toHaveLength(1);
|
||||
const [pageId, node] = insertNodeCalls[0];
|
||||
// #413: the 2nd arg is the XOR input { markdown?, node? }, the 3rd is opts.
|
||||
const [pageId, input, opts] = insertNodeCalls[0] as [
|
||||
string,
|
||||
{ markdown?: unknown; node?: unknown },
|
||||
{ position?: string },
|
||||
];
|
||||
expect(pageId).toBe('p1');
|
||||
expect(node).toEqual(NODE_OBJ);
|
||||
expect(input).toEqual({ markdown: undefined, node: NODE_OBJ });
|
||||
expect(opts.position).toBe('append');
|
||||
});
|
||||
|
||||
it('insertNode throws the documented message on invalid JSON string', async () => {
|
||||
@@ -912,7 +927,7 @@ describe('AiChatToolsService getCurrentPage selection (#388)', () => {
|
||||
});
|
||||
|
||||
/**
|
||||
* #440 review: the in-app drawio_create / drawio_update handlers must forward
|
||||
* #440 review: the in-app drawioCreate / drawioUpdate handlers must forward
|
||||
* the optional `layout:"elk"` param to the client (5th positional arg), exactly
|
||||
* like the MCP host. It was silently dropped, so ELK auto-layout worked only via
|
||||
* the standalone MCP server, not in-app. These tests pin per-host parity.
|
||||
|
||||
@@ -59,10 +59,12 @@ function __assertClientCallContract(client: DocmostClientLike): void {
|
||||
void client.getWorkspace();
|
||||
void client.getSpaces();
|
||||
void client.listPages(s, n, true);
|
||||
void client.getTree(s, s, n);
|
||||
void client.getPageContext(s);
|
||||
void client.listSidebarPages(s, s);
|
||||
void client.getOutline(s);
|
||||
void client.getPageJson(s);
|
||||
void client.getNode(s, s);
|
||||
void client.getNode(s, s, 'markdown');
|
||||
void client.searchInPage(s, s, {
|
||||
regex: true,
|
||||
caseSensitive: true,
|
||||
@@ -84,12 +86,16 @@ function __assertClientCallContract(client: DocmostClientLike): void {
|
||||
void client.movePage(s, s, s);
|
||||
void client.deletePage(s);
|
||||
void client.editPageText(s, edits);
|
||||
void client.patchNode(s, s, node);
|
||||
void client.insertNode(s, node, {
|
||||
position: 'append',
|
||||
anchorNodeId: s,
|
||||
anchorText: s,
|
||||
});
|
||||
void client.patchNode(s, s, { markdown: s, node });
|
||||
void client.insertNode(
|
||||
s,
|
||||
{ markdown: s, node },
|
||||
{
|
||||
position: 'append',
|
||||
anchorNodeId: s,
|
||||
anchorText: s,
|
||||
},
|
||||
);
|
||||
void client.deleteNode(s, s);
|
||||
void client.updatePageJson(s, node, s);
|
||||
void client.tableInsertRow(s, s, cells, n);
|
||||
@@ -117,6 +123,23 @@ function __assertClientCallContract(client: DocmostClientLike): void {
|
||||
void client.drawioGet(s, s, 'xml');
|
||||
void client.drawioCreate(s, { position: 'append', anchorNodeId: s }, s, s, 'elk');
|
||||
void client.drawioUpdate(s, s, s, s, 'elk');
|
||||
// --- draw.io high-level semantic tools (#425 stage 3) ---
|
||||
void client.drawioEditCells(s, s, [{ op: 'delete', cellId: s }], s);
|
||||
void client.drawioFromGraph(
|
||||
s,
|
||||
{ position: 'append', anchorNodeId: s },
|
||||
{ nodes: [{ id: s, label: s }] },
|
||||
'LR',
|
||||
s,
|
||||
'full',
|
||||
s,
|
||||
);
|
||||
void client.drawioFromMermaid(
|
||||
s,
|
||||
{ position: 'append', anchorNodeId: s },
|
||||
s,
|
||||
s,
|
||||
);
|
||||
// --- write (comment) ---
|
||||
void client.createComment(s, s, 'inline', s, s, s);
|
||||
void client.resolveComment(s, true);
|
||||
@@ -266,7 +289,7 @@ export class AiChatToolsService {
|
||||
// construction is shared with the page-change detection path (#274) via
|
||||
// buildDocmostClient so both go over the exact same authenticated route.
|
||||
// searchShapes / getGuideSection (#424) are the PURE, no-network helpers
|
||||
// backing drawio_shapes / drawio_guide. They are `inlineBothHosts` specs (no
|
||||
// backing drawioShapes / drawioGuide. They are `inlineBothHosts` specs (no
|
||||
// canonical execute — their catalog loader uses import.meta and can't be
|
||||
// value-imported into the zod-agnostic tool-specs.ts under the server's
|
||||
// commonjs type-check), so the shared registry loop below SKIPS them and this
|
||||
@@ -308,9 +331,10 @@ export class AiChatToolsService {
|
||||
// The in-app toolset. It starts with the tools kept INLINE here for a
|
||||
// documented per-layer reason: an intentional behaviour/schema divergence from
|
||||
// the standalone MCP surface (searchPages' hybrid RRF,
|
||||
// transformPage's guardrailed shorter schema), a
|
||||
// snake_case/camelCase naming clash the shared registry forbids (getTable vs
|
||||
// the MCP `table_get`), per-request state the registry loop cannot provide
|
||||
// transformPage's guardrailed shorter schema), a name clash the shared
|
||||
// registry forbids (in-app `getTable` verb-first vs the MCP noun-first
|
||||
// `tableGet` — the registry requires mcpName === inAppKey), per-request
|
||||
// state the registry loop cannot provide
|
||||
// (getCurrentPage reads the resolved openedPage; searchPages closes over the
|
||||
// per-request user/embedding deps), or a tool with no MCP twin
|
||||
// (listSidebarPages/getComment/getPageHistory). Every SHARED tool is then added
|
||||
@@ -477,9 +501,9 @@ export class AiChatToolsService {
|
||||
await client.listSidebarPages(spaceId, pageId),
|
||||
}),
|
||||
|
||||
// NOT shared (kept inline): the MCP tool name `table_get` is noun-first
|
||||
// while this key is `getTable` (verb-first), breaking the
|
||||
// snake_case(inAppKey) convention the shared registry enforces. Its
|
||||
// NOT shared (kept inline): the MCP tool name `tableGet` is noun-first
|
||||
// while this key is `getTable` (verb-first), so it cannot satisfy the
|
||||
// shared registry's `mcpName === inAppKey` convention (#412). Its
|
||||
// reference parameter is still named `table` (was `tableRef`) so it matches
|
||||
// the migrated table row/cell tools below.
|
||||
getTable: tool({
|
||||
@@ -522,7 +546,7 @@ export class AiChatToolsService {
|
||||
|
||||
// INTENTIONAL per-transport divergence (not shared): deliberately omits the
|
||||
// `deleteComments` schema field (comment-deletion guardrail) and carries a
|
||||
// much shorter description; the standalone MCP `docmost_transform` exposes
|
||||
// much shorter description; the standalone MCP `docmostTransform` exposes
|
||||
// the full helper catalogue. Different schema, so kept per-layer.
|
||||
transformPage: tool({
|
||||
description:
|
||||
@@ -553,7 +577,7 @@ export class AiChatToolsService {
|
||||
// WHICH mapping to run and returns its value directly (no envelope). For each
|
||||
// spec:
|
||||
// - skip `mcpOnly` specs (they belong to the standalone MCP host only);
|
||||
// - skip `inlineBothHosts` specs (drawio_shapes / drawio_guide): they carry
|
||||
// - skip `inlineBothHosts` specs (drawioShapes / drawioGuide): they carry
|
||||
// no execute and are wired INLINE just below, calling the pure helpers;
|
||||
// - use `inAppExecute` when the spec declares a DELIBERATE per-layer
|
||||
// difference (a projected result shape, a different guardrail message);
|
||||
@@ -574,7 +598,7 @@ export class AiChatToolsService {
|
||||
);
|
||||
}
|
||||
|
||||
// drawio_shapes / drawio_guide (#424): `inlineBothHosts` registry specs wired
|
||||
// drawioShapes / drawioGuide (#424): `inlineBothHosts` registry specs wired
|
||||
// here with the SAME schema+description the shared spec pins, but calling the
|
||||
// pure searchShapes / getGuideSection helpers off the loaded @docmost/mcp
|
||||
// module — they are not client methods and their catalog loader uses
|
||||
|
||||
@@ -23,6 +23,8 @@ type DocmostClientMethod =
|
||||
| 'getWorkspace'
|
||||
| 'getSpaces'
|
||||
| 'listPages'
|
||||
| 'getTree'
|
||||
| 'getPageContext'
|
||||
| 'listSidebarPages'
|
||||
| 'getOutline'
|
||||
| 'getPageJson'
|
||||
@@ -69,6 +71,10 @@ type DocmostClientMethod =
|
||||
| 'drawioGet'
|
||||
| 'drawioCreate'
|
||||
| 'drawioUpdate'
|
||||
// --- draw.io high-level semantic tools (#425 stage 3) ---
|
||||
| 'drawioEditCells'
|
||||
| 'drawioFromGraph'
|
||||
| 'drawioFromMermaid'
|
||||
// --- write (comment) ---
|
||||
| 'createComment'
|
||||
| 'resolveComment';
|
||||
@@ -146,7 +152,7 @@ export type CommentSignalTrackerFactory = (options: {
|
||||
|
||||
// Pure, no-network draw.io helpers (#424). These are plain functions on the
|
||||
// module (NOT DocmostClient methods) — the in-app AI-SDK service calls them
|
||||
// directly to wire drawio_shapes / drawio_guide, mirroring the MCP server.
|
||||
// directly to wire drawioShapes / drawioGuide, mirroring the MCP server.
|
||||
export type SearchShapesFn = (
|
||||
query: string,
|
||||
opts?: { category?: string; limit?: number },
|
||||
@@ -169,7 +175,7 @@ interface DocmostMcpModule {
|
||||
// the mocked loader in unit tests) — the stale-check below is a NO-OP when it
|
||||
// is missing, so an older build never wrongly fails startup.
|
||||
REGISTRY_STAMP?: string;
|
||||
// Pure, no-network draw.io helpers (#424) backing drawio_shapes / drawio_guide.
|
||||
// Pure, no-network draw.io helpers (#424) backing drawioShapes / drawioGuide.
|
||||
// Those two specs are `inlineBothHosts` (they stay in SHARED_TOOL_SPECS for the
|
||||
// shared contract but carry no execute — their catalog loader uses import.meta
|
||||
// and can't be value-imported into the zod-agnostic tool-specs.ts), so the
|
||||
|
||||
@@ -17,8 +17,10 @@ import { SHARED_TOOL_SPECS } from '../../../../../../packages/mcp/src/tool-specs
|
||||
* This test fails the build if a spec is added to the registry but never wired
|
||||
* in-app, if an `inAppKey` is renamed without updating the service, if the
|
||||
* description drifts between the registry and the exposed tool, if the
|
||||
* snake_case `mcpName` <-> camelCase `inAppKey` convention is broken, or if the
|
||||
* exposed tool's input-schema keys diverge from the spec's `buildShape`.
|
||||
* `mcpName === inAppKey` convention is broken (issue #412 unified the external
|
||||
* MCP tool name with the in-app key — both are the same camelCase identifier),
|
||||
* or if the exposed tool's input-schema keys diverge from the spec's
|
||||
* `buildShape`.
|
||||
*
|
||||
* It does NOT need @docmost/mcp built: the registry is imported from TS source,
|
||||
* and the ESM loader is mocked so `forUser()` never dynamically imports the
|
||||
@@ -74,10 +76,6 @@ describe('SHARED_TOOL_SPECS contract parity', () => {
|
||||
|
||||
afterAll(() => jest.restoreAllMocks());
|
||||
|
||||
// camelCase -> snake_case, matching the registry's mcpName convention.
|
||||
const toSnake = (s: string) =>
|
||||
s.replace(/[A-Z]/g, (c) => `_${c.toLowerCase()}`);
|
||||
|
||||
// Type as the (optional-buildShape) SharedToolSpec; the `satisfies` literal
|
||||
// above otherwise narrows to a union where some members lack buildShape.
|
||||
const specEntries = Object.entries(SHARED_TOOL_SPECS) as unknown as Array<
|
||||
@@ -96,8 +94,8 @@ describe('SHARED_TOOL_SPECS contract parity', () => {
|
||||
expect(spec.inAppKey).toBe(registryKey);
|
||||
});
|
||||
|
||||
it('mcpName is the snake_case form of inAppKey', () => {
|
||||
expect(spec.mcpName).toBe(toSnake(spec.inAppKey));
|
||||
it('mcpName equals inAppKey (unified camelCase name, #412)', () => {
|
||||
expect(spec.mcpName).toBe(spec.inAppKey);
|
||||
});
|
||||
|
||||
it('is exposed in-app under its inAppKey', () => {
|
||||
|
||||
@@ -36,7 +36,7 @@ describe('tool tier metadata (#332)', () => {
|
||||
});
|
||||
|
||||
it('#410 image tools are DEFERRED, footnote tool is CORE', () => {
|
||||
// insert_footnote is core (symmetric with editPageText); the image tools stay
|
||||
// insertFootnote is core (symmetric with editPageText); the image tools stay
|
||||
// deferred (rare, fat — loaded on demand). Assert both the spec tier and the
|
||||
// CORE_TOOL_SET membership so a future tier edit that desyncs them fails here.
|
||||
expect(SHARED_TOOL_SPECS.insertFootnote.tier).toBe('core');
|
||||
|
||||
@@ -60,10 +60,10 @@ export const CORE_TOOL_KEYS = [
|
||||
'listComments',
|
||||
'resolveComment',
|
||||
'editPageText',
|
||||
// #330 search_in_page — frequent for editorial sweeps; core despite predating
|
||||
// #330 searchInPage — frequent for editorial sweeps; core despite predating
|
||||
// the issue's tier list.
|
||||
'searchInPage',
|
||||
// #410 insert_footnote — core so pinpoint citations to already-written text
|
||||
// #410 insertFootnote — core so pinpoint citations to already-written text
|
||||
// don't degrade into literal `^[...]`; kept symmetric with editPageText.
|
||||
'insertFootnote',
|
||||
] as const;
|
||||
@@ -138,7 +138,7 @@ export const INLINE_TOOL_TIERS: Record<
|
||||
},
|
||||
// NOTE: tableInsertRow, tableDeleteRow and tableUpdateCell moved to
|
||||
// @docmost/mcp's SHARED_TOOL_SPECS (#294); they carry their own deferred tier +
|
||||
// catalogLine there. getTable stays inline (its MCP name table_get breaks the
|
||||
// catalogLine there. getTable stays inline (its MCP name tableGet breaks the
|
||||
// snake_case(inAppKey) convention, so it has no shared spec).
|
||||
// NOTE: checkNewComments moved to @docmost/mcp's SHARED_TOOL_SPECS (#294);
|
||||
// it carries its own deferred tier + catalogLine there.
|
||||
@@ -150,7 +150,7 @@ export const INLINE_TOOL_TIERS: Record<
|
||||
// NOTE: sharePage moved to @docmost/mcp's SHARED_TOOL_SPECS (#294); it carries
|
||||
// its own deferred tier + catalogLine there. transformPage stays inline (its
|
||||
// schema deliberately diverges — it omits the deleteComments field the MCP
|
||||
// docmost_transform exposes, a comment-deletion guardrail).
|
||||
// docmostTransform exposes, a comment-deletion guardrail).
|
||||
transformPage: {
|
||||
tier: 'deferred',
|
||||
catalogLine: "transformPage — run a sandboxed JS transform over a page's document.",
|
||||
|
||||
@@ -12,3 +12,22 @@ export class SearchResponseDto {
|
||||
updatedAt: Date;
|
||||
space: Partial<Space>;
|
||||
}
|
||||
|
||||
// Response shape for the opt-in agent-lookup mode (#443, `substring: true`).
|
||||
// Additive to the FTS response: carries the location (`path`), a windowed
|
||||
// `snippet` around the first match and a per-response sort `score`. The MCP
|
||||
// layer maps `id → pageId`; `slugId` is never exposed.
|
||||
export class SearchLookupResponseDto {
|
||||
id: string;
|
||||
slugId: string;
|
||||
title: string;
|
||||
parentPageId: string | null;
|
||||
// Ancestor titles from the space root down to the direct parent; [] for a
|
||||
// root page.
|
||||
path: string[];
|
||||
// ~300–500 chars around the first match (or a leading text window / extended
|
||||
// ts_headline fallback).
|
||||
snippet: string;
|
||||
// 0..1 float, meaningful ONLY for sorting within one response.
|
||||
score: number;
|
||||
}
|
||||
|
||||
@@ -30,6 +30,31 @@ export class SearchDTO {
|
||||
@IsOptional()
|
||||
@IsNumber()
|
||||
offset?: number;
|
||||
|
||||
// --- Opt-in agent-lookup mode (#443). ------------------------------------
|
||||
// These fields are ADDITIVE and default-off: a web client that sends none of
|
||||
// them gets byte-identical FTS behaviour and result shape. They are only read
|
||||
// by the substring/path/snippet code path in SearchService.searchPage.
|
||||
//
|
||||
// NOTE (standalone stdio vs stock upstream): stock upstream validates this DTO
|
||||
// with `whitelist: true`, so an older server silently strips these unknown
|
||||
// fields and the request degrades gracefully to the plain FTS behaviour.
|
||||
|
||||
// Enables the hybrid substring branch (title + text_content LIKE) merged with
|
||||
// the existing FTS branch, plus tiered ranking, path and windowed snippet.
|
||||
@IsOptional()
|
||||
@IsBoolean()
|
||||
substring?: boolean;
|
||||
|
||||
// Restrict the search to a page and all of its descendants (inclusive).
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
parentPageId?: string;
|
||||
|
||||
// Match titles only; do not scan text_content.
|
||||
@IsOptional()
|
||||
@IsBoolean()
|
||||
titleOnly?: boolean;
|
||||
}
|
||||
|
||||
export class SearchShareDTO extends SearchDTO {
|
||||
|
||||
@@ -60,6 +60,12 @@ export class SearchController {
|
||||
}
|
||||
}
|
||||
|
||||
// #443 graceful degradation: on EE/Typesense instances the request routes to
|
||||
// the Typesense backend, which does NOT implement the opt-in agent-lookup
|
||||
// mode. The `substring`/`parentPageId`/`titleOnly` fields are silently ignored
|
||||
// and the response carries no `path`/`snippet`/`score` and no substring/tier
|
||||
// ranking — it degrades to plain Typesense FTS. The native lookup mode below
|
||||
// is Postgres-search-driver only.
|
||||
if (this.environmentService.getSearchDriver() === 'typesense') {
|
||||
return this.searchTypesense(searchDto, {
|
||||
userId: user.id,
|
||||
|
||||
@@ -0,0 +1,95 @@
|
||||
import {
|
||||
computeLookupScore,
|
||||
escapeLikePattern,
|
||||
SearchLookupTier,
|
||||
} from './search.service';
|
||||
|
||||
/**
|
||||
* Pure-function coverage for the #443 agent-lookup helpers:
|
||||
* - escapeLikePattern: LIKE-metacharacter escaping so `%`/`_`/`\` are literals
|
||||
* (the acceptance-table requirement that a query of `%` or `_` does NOT match
|
||||
* everything);
|
||||
* - computeLookupScore: the tiered 0..1 ranking score, where a stronger tier
|
||||
* always outranks a weaker one regardless of the in-tier secondary signal.
|
||||
*
|
||||
* The DB-touching branch (substring UNION FTS, path CTE, snippet window) is
|
||||
* covered by the integration spec against the real schema.
|
||||
*/
|
||||
describe('escapeLikePattern', () => {
|
||||
it('escapes the LIKE metacharacters % _ and \\', () => {
|
||||
expect(escapeLikePattern('%')).toBe('\\%');
|
||||
expect(escapeLikePattern('_')).toBe('\\_');
|
||||
expect(escapeLikePattern('\\')).toBe('\\\\');
|
||||
});
|
||||
|
||||
it('escapes the backslash FIRST so it does not double-escape %/_', () => {
|
||||
// Input `\%` must become `\\` + `\%` = `\\\%`, not `\\%`.
|
||||
expect(escapeLikePattern('\\%')).toBe('\\\\\\%');
|
||||
});
|
||||
|
||||
it('leaves ordinary technical chars (. - / digits) untouched', () => {
|
||||
expect(escapeLikePattern('backup-srv.local')).toBe('backup-srv.local');
|
||||
expect(escapeLikePattern('10.0.12')).toBe('10.0.12');
|
||||
expect(escapeLikePattern('WB-MGE-30D86B')).toBe('WB-MGE-30D86B');
|
||||
expect(escapeLikePattern('a/b')).toBe('a/b');
|
||||
});
|
||||
|
||||
it('escapes only the metacharacters in a mixed string', () => {
|
||||
expect(escapeLikePattern('50%_off.zip')).toBe('50\\%\\_off.zip');
|
||||
});
|
||||
|
||||
it('is null/undefined-safe', () => {
|
||||
expect(escapeLikePattern(undefined as any)).toBe('');
|
||||
expect(escapeLikePattern(null as any)).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('computeLookupScore', () => {
|
||||
it('keeps every score within (0, 1]', () => {
|
||||
for (const tier of [
|
||||
SearchLookupTier.TITLE_EXACT,
|
||||
SearchLookupTier.TITLE_SUBSTRING,
|
||||
SearchLookupTier.TEXT,
|
||||
]) {
|
||||
for (const secondary of [0, 0.001, 1, 100, 1e6]) {
|
||||
const s = computeLookupScore({ tier, secondary });
|
||||
expect(s).toBeGreaterThan(0);
|
||||
expect(s).toBeLessThanOrEqual(1);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('a stronger tier ALWAYS outranks a weaker tier, whatever the secondary', () => {
|
||||
// Weak tier with a huge secondary must still lose to a strong tier with a
|
||||
// tiny secondary — tiers dominate.
|
||||
const strongLowSecondary = computeLookupScore({
|
||||
tier: SearchLookupTier.TITLE_EXACT,
|
||||
secondary: 0,
|
||||
});
|
||||
const weakHighSecondary = computeLookupScore({
|
||||
tier: SearchLookupTier.TEXT,
|
||||
secondary: 1e9,
|
||||
});
|
||||
expect(strongLowSecondary).toBeGreaterThan(weakHighSecondary);
|
||||
});
|
||||
|
||||
it('within a tier a larger secondary sorts higher', () => {
|
||||
const lo = computeLookupScore({
|
||||
tier: SearchLookupTier.TEXT,
|
||||
secondary: 0.1,
|
||||
});
|
||||
const hi = computeLookupScore({
|
||||
tier: SearchLookupTier.TEXT,
|
||||
secondary: 5,
|
||||
});
|
||||
expect(hi).toBeGreaterThan(lo);
|
||||
});
|
||||
|
||||
it('treats a negative/absent secondary as 0', () => {
|
||||
const zero = computeLookupScore({ tier: SearchLookupTier.TEXT, secondary: 0 });
|
||||
expect(computeLookupScore({ tier: SearchLookupTier.TEXT })).toBe(zero);
|
||||
expect(
|
||||
computeLookupScore({ tier: SearchLookupTier.TEXT, secondary: -5 }),
|
||||
).toBe(zero);
|
||||
});
|
||||
});
|
||||
@@ -1,6 +1,9 @@
|
||||
import { Injectable } from '@nestjs/common';
|
||||
import { SearchDTO, SearchSuggestionDTO } from './dto/search.dto';
|
||||
import { SearchResponseDto } from './dto/search-response.dto';
|
||||
import {
|
||||
SearchLookupResponseDto,
|
||||
SearchResponseDto,
|
||||
} from './dto/search-response.dto';
|
||||
import { InjectKysely } from 'nestjs-kysely';
|
||||
import { KyselyDB } from '@docmost/db/types/kysely.types';
|
||||
import { sql } from 'kysely';
|
||||
@@ -34,6 +37,53 @@ export function buildTsQuery(raw: string): string {
|
||||
return tsquery(cleaned + '*');
|
||||
}
|
||||
|
||||
// Escape the LIKE metacharacters (`%`, `_`, `\`) in a raw user query so every
|
||||
// character — including `.`, `-`, `_`, `%`, `/` — is matched LITERALLY by a
|
||||
// `col LIKE '%' || q || '%'` predicate. Without this, a query of `%` or `_`
|
||||
// would match every row (see the #443 acceptance table). The backslash is the
|
||||
// escape char (Postgres LIKE default), so it must be escaped first.
|
||||
export function escapeLikePattern(raw: string): string {
|
||||
return (raw ?? '')
|
||||
.replace(/\\/g, '\\\\')
|
||||
.replace(/%/g, '\\%')
|
||||
.replace(/_/g, '\\_');
|
||||
}
|
||||
|
||||
// Ranking tiers for the agent-lookup mode (#443), highest first. A hit's tier
|
||||
// is the strongest way it matched; ties inside a tier break on a secondary
|
||||
// signal (FTS rank, or first-match position). The numeric `score` returned to
|
||||
// the caller is derived from (tier, secondary) and is meaningful ONLY for
|
||||
// ordering within a single response.
|
||||
export enum SearchLookupTier {
|
||||
// Title equals the query, case-insensitively.
|
||||
TITLE_EXACT = 3,
|
||||
// Query is a substring of the title.
|
||||
TITLE_SUBSTRING = 2,
|
||||
// Query matched in the text (substring or FTS).
|
||||
TEXT = 1,
|
||||
}
|
||||
|
||||
export interface RankableHit {
|
||||
tier: SearchLookupTier;
|
||||
// Secondary in-tier signal, higher = better (e.g. ts_rank, or a
|
||||
// position-derived closeness score). Defaults to 0.
|
||||
secondary?: number;
|
||||
}
|
||||
|
||||
// Map (tier, secondary) → a 0..1 float used ONLY to sort one response.
|
||||
//
|
||||
// Formula: score = (tier + squash(secondary)) / (maxTier + 1), where
|
||||
// squash(x) = x / (1 + x) maps any non-negative secondary into [0, 1)
|
||||
// so a stronger tier ALWAYS outranks a weaker one regardless of the secondary
|
||||
// value, and within a tier a larger secondary sorts higher. maxTier is the top
|
||||
// enum value (TITLE_EXACT = 3), so the divisor keeps the result in (0, 1].
|
||||
export function computeLookupScore(hit: RankableHit): number {
|
||||
const maxTier = SearchLookupTier.TITLE_EXACT;
|
||||
const secondary = Math.max(0, hit.secondary ?? 0);
|
||||
const squashed = secondary / (1 + secondary);
|
||||
return (hit.tier + squashed) / (maxTier + 1);
|
||||
}
|
||||
|
||||
@Injectable()
|
||||
export class SearchService {
|
||||
constructor(
|
||||
@@ -50,12 +100,19 @@ export class SearchService {
|
||||
userId?: string;
|
||||
workspaceId: string;
|
||||
},
|
||||
): Promise<{ items: SearchResponseDto[] }> {
|
||||
): Promise<{ items: SearchResponseDto[] | SearchLookupResponseDto[] }> {
|
||||
const { query } = searchParams;
|
||||
|
||||
if (query.length < 1) {
|
||||
return { items: [] };
|
||||
}
|
||||
|
||||
// Opt-in agent-lookup mode (#443). Guarded by the `substring` flag so the
|
||||
// web-UI (which never sets it) keeps byte-identical FTS behaviour below.
|
||||
if (searchParams.substring) {
|
||||
return this.searchPageLookup(searchParams, opts);
|
||||
}
|
||||
|
||||
const searchQuery = buildTsQuery(query);
|
||||
|
||||
let queryResults = this.db
|
||||
@@ -175,6 +232,348 @@ export class SearchService {
|
||||
return { items: searchResults };
|
||||
}
|
||||
|
||||
/**
|
||||
* Agent-lookup search (#443, opt-in via `SearchDTO.substring`).
|
||||
*
|
||||
* ADDITIVE to the FTS path: runs a substring branch (title + optionally
|
||||
* text_content, LIKE with metacharacters escaped) MERGED with the existing
|
||||
* FTS branch, so technical tokens that the `english` tokenizer mangles
|
||||
* (`backup-srv.local`, `10.0.12.5`, `WB-MGE-30D86B`) are still found — even
|
||||
* when `buildTsQuery()` returns '' for a dotted/numeric query. Results carry a
|
||||
* location (`path`), a windowed `snippet` and a per-response `score`.
|
||||
*
|
||||
* The whole method is only reached when `substring: true`; the web-UI never
|
||||
* sets it, so its behaviour is unchanged.
|
||||
*/
|
||||
private async searchPageLookup(
|
||||
searchParams: SearchDTO,
|
||||
opts: { userId?: string; workspaceId: string },
|
||||
): Promise<{ items: SearchLookupResponseDto[] }> {
|
||||
const rawQuery = searchParams.query.trim();
|
||||
if (!rawQuery) {
|
||||
return { items: [] };
|
||||
}
|
||||
|
||||
const limit = Math.min(Math.max(searchParams.limit || 10, 1), 50);
|
||||
|
||||
// Normalize the query the same way as the FTS / suggest path: f_unaccent +
|
||||
// lower, done in SQL. `q` is the escaped LIKE pattern body (literal chars).
|
||||
const likeBody = escapeLikePattern(rawQuery);
|
||||
// Compare against `LOWER(f_unaccent(col))`; unaccent+lower the needle too.
|
||||
const needle = sql<string>`LOWER(f_unaccent(${rawQuery}))`;
|
||||
const likePattern = sql<string>`LOWER(f_unaccent(${'%' + likeBody + '%'}))`;
|
||||
const tsQuery = buildTsQuery(rawQuery);
|
||||
const hasTsQuery = tsQuery.length > 0;
|
||||
|
||||
// --- Resolve the space scope. ---------------------------------------------
|
||||
// Mirrors searchPage: explicit spaceId, else the authenticated user's member
|
||||
// spaces. The share path is not exposed to this opt-in mode.
|
||||
let spaceIds: string[] = [];
|
||||
if (searchParams.spaceId) {
|
||||
spaceIds = [searchParams.spaceId];
|
||||
} else if (opts.userId) {
|
||||
spaceIds = await this.spaceMemberRepo.getUserSpaceIds(opts.userId);
|
||||
} else {
|
||||
return { items: [] };
|
||||
}
|
||||
if (spaceIds.length === 0) {
|
||||
return { items: [] };
|
||||
}
|
||||
|
||||
// --- Optional parentPageId subtree scope (inclusive). ---------------------
|
||||
// Reuse the same recursive-descendants pattern used for share-scope.
|
||||
let descendantIds: string[] | null = null;
|
||||
if (searchParams.parentPageId) {
|
||||
const descendants = await this.pageRepo.getPageAndDescendants(
|
||||
searchParams.parentPageId,
|
||||
{ includeContent: false },
|
||||
);
|
||||
descendantIds = descendants.map((p: any) => p.id);
|
||||
if (descendantIds.length === 0) {
|
||||
return { items: [] };
|
||||
}
|
||||
}
|
||||
|
||||
// --- Candidate query: substring (title + text) UNION FTS. -----------------
|
||||
// We compute everything the ranker needs in SQL and pull only small columns
|
||||
// (never the whole text_content) into Node:
|
||||
// - titleExact / titleSub: tier signals
|
||||
// - textMatchPos: 1-based position of the first text match (0 = none)
|
||||
// - ftsRank: ts_rank for the FTS secondary signal (0 when no tsquery)
|
||||
// - snippet: windowed ~500 chars around the first text match, or a leading
|
||||
// text window (title-only hit), or an extended ts_headline fallback.
|
||||
const N_BEFORE = 60; // chars of context before the first match
|
||||
const SNIPPET_LEN = 500;
|
||||
|
||||
let candidates = this.db
|
||||
.selectFrom('pages')
|
||||
.select([
|
||||
'pages.id as id',
|
||||
'pages.slugId as slugId',
|
||||
'pages.title as title',
|
||||
'pages.parentPageId as parentPageId',
|
||||
// Tier signals.
|
||||
sql<boolean>`LOWER(f_unaccent(coalesce(pages.title, ''))) = ${needle}`.as(
|
||||
'titleExact',
|
||||
),
|
||||
sql<boolean>`LOWER(f_unaccent(coalesce(pages.title, ''))) LIKE ${likePattern} ESCAPE '\\'`.as(
|
||||
'titleSub',
|
||||
),
|
||||
// 1-based position of the first text match (0 = no text match).
|
||||
sql<number>`strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle})`.as(
|
||||
'textMatchPos',
|
||||
),
|
||||
// FTS secondary signal (0 when the tsquery is empty).
|
||||
hasTsQuery
|
||||
? sql<number>`ts_rank(pages.tsv, to_tsquery('english', f_unaccent(${tsQuery})))`.as(
|
||||
'ftsRank',
|
||||
)
|
||||
: sql<number>`0`.as('ftsRank'),
|
||||
// Windowed snippet, computed entirely in SQL. Priority:
|
||||
// 1. window around the first text match;
|
||||
// 2. otherwise (titleOnly: no snippet; else) a leading window of the
|
||||
// page text (title-only hit);
|
||||
// 3. otherwise an extended ts_headline for pure-FTS hits.
|
||||
//
|
||||
// #443 snippet-position fix: the match position (`strpos`) is computed in
|
||||
// the LOWER(f_unaccent(...)) space, but f_unaccent is NOT length-
|
||||
// preserving (ß→ss, æ→ae, …→..., ½→ 1/2, full-width forms), so slicing
|
||||
// the ORIGINAL text at that position was misaligned — a single expanding
|
||||
// char before the match shifted the window (or ran it past end → empty).
|
||||
// We now slice from the SAME LOWER(f_unaccent(...)) string so position
|
||||
// and slice share one coordinate space. DELIBERATE trade-off: the snippet
|
||||
// loses original case/diacritics — acceptable for an agent-facing snippet
|
||||
// (position accuracy over original-glyph fidelity). The ts_headline branch
|
||||
// matches over the ORIGINAL text itself, so it is unaffected and kept as-is.
|
||||
searchParams.titleOnly
|
||||
? sql<string>`''`.as('snippet')
|
||||
: sql<string>`
|
||||
coalesce(
|
||||
case
|
||||
when strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) > 0
|
||||
then substring(
|
||||
LOWER(f_unaccent(coalesce(pages.text_content, '')))
|
||||
from greatest(1, strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) - ${N_BEFORE})
|
||||
for ${SNIPPET_LEN}
|
||||
)
|
||||
when coalesce(pages.text_content, '') <> ''
|
||||
then substring(LOWER(f_unaccent(pages.text_content)) from 1 for 300)
|
||||
${
|
||||
hasTsQuery
|
||||
? sql`else ts_headline('english', coalesce(pages.text_content, ''), to_tsquery('english', f_unaccent(${tsQuery})), 'MinWords=25, MaxWords=40, MaxFragments=3')`
|
||||
: sql``
|
||||
}
|
||||
end,
|
||||
''
|
||||
)
|
||||
`.as('snippet'),
|
||||
])
|
||||
.where('pages.deletedAt', 'is', null)
|
||||
.where('pages.spaceId', 'in', spaceIds);
|
||||
|
||||
if (descendantIds) {
|
||||
candidates = candidates.where('pages.id', 'in', descendantIds);
|
||||
}
|
||||
|
||||
// Match predicate: title substring OR (unless titleOnly) text substring OR
|
||||
// (unless titleOnly) FTS. The substring branch runs even when the tsquery is
|
||||
// empty — that is the dotted/numeric-token case the FTS path misses.
|
||||
//
|
||||
// #443 dead-index fix: these two LIKE predicates MUST match the GIN trgm
|
||||
// index expressions EXACTLY for Postgres to use them. The indexes are on the
|
||||
// coalesce-FREE expressions `LOWER(f_unaccent(title))` (#348's
|
||||
// idx_pages_title_trgm) and `LOWER(f_unaccent(text_content))` (this PR's
|
||||
// idx_pages_text_content_trgm). A `coalesce(col,'')` wrapper here would make
|
||||
// the query expression differ from the index expression and force a Seq Scan
|
||||
// on pages for every lookup. Dropping coalesce is SEMANTICALLY EQUIVALENT:
|
||||
// `NULL LIKE '%q%'` is NULL (falsy), so a NULL title/text simply doesn't
|
||||
// match — exactly as an empty string wouldn't match `%q%`.
|
||||
candidates = candidates.where((eb) => {
|
||||
const ors = [
|
||||
eb(
|
||||
sql`LOWER(f_unaccent(pages.title))`,
|
||||
'like',
|
||||
sql`${likePattern} ESCAPE '\\'`,
|
||||
),
|
||||
];
|
||||
if (!searchParams.titleOnly) {
|
||||
ors.push(
|
||||
eb(
|
||||
sql`LOWER(f_unaccent(pages.text_content))`,
|
||||
'like',
|
||||
sql`${likePattern} ESCAPE '\\'`,
|
||||
),
|
||||
);
|
||||
if (hasTsQuery) {
|
||||
ors.push(
|
||||
sql<boolean>`pages.tsv @@ to_tsquery('english', f_unaccent(${tsQuery}))` as any,
|
||||
);
|
||||
}
|
||||
}
|
||||
return eb.or(ors);
|
||||
});
|
||||
|
||||
// Pull a generous candidate set (before permission filtering + limit).
|
||||
// Cap it so a pathological match set cannot blow up memory; 200 >> limit
|
||||
// (max 50) leaves ample headroom for the post-permission truncation.
|
||||
//
|
||||
// #443 cap-ordering fix: the 200-cap MUST be deterministic and relevance-
|
||||
// biased. Without an ORDER BY, Postgres returns an ARBITRARY 200 rows, so on
|
||||
// a broad match set (common word / short substring) a strong TITLE_EXACT hit
|
||||
// could be among the dropped rows while 200 low-tier TEXT hits fill the cap.
|
||||
// We order by the SAME SQL tier proxies the Node ranker uses — title-exact,
|
||||
// then title-substring, then fts-rank (nulls last), then earliest text-match
|
||||
// position — so the cap keeps the strongest candidates. The Node-side final
|
||||
// tier sort + slice(0, limit) below still runs and stays authoritative; this
|
||||
// ORDER BY only decides WHICH candidates survive the 200-cap.
|
||||
// NB: a BARE integer literal in ORDER BY is read by Postgres as an ordinal
|
||||
// column position (`ORDER BY 0` → "position 0 is not in select list"), so the
|
||||
// no-tsquery fallback is `0::float`, not `0`.
|
||||
const ftsRankExpr = hasTsQuery
|
||||
? sql`ts_rank(pages.tsv, to_tsquery('english', f_unaccent(${tsQuery})))`
|
||||
: sql`0::float`;
|
||||
const candidatesCapped = candidates
|
||||
// Raw-SQL ORDER BY expressions: pass the full `<expr> <dir>` as ONE arg
|
||||
// (the two-arg form treats a raw-SQL second arg as an ORDER BY position).
|
||||
.orderBy(
|
||||
sql`(LOWER(f_unaccent(coalesce(pages.title, ''))) = ${needle}) desc`,
|
||||
)
|
||||
.orderBy(
|
||||
sql`(LOWER(f_unaccent(coalesce(pages.title, ''))) LIKE ${likePattern} ESCAPE '\\') desc`,
|
||||
)
|
||||
.orderBy(sql`${ftsRankExpr} desc nulls last`)
|
||||
// Earlier text match first; strpos returns 0 for "no match", which would
|
||||
// sort BEFORE a real (>=1) position under plain ASC, so push 0 to the end.
|
||||
.orderBy(
|
||||
sql`case when strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) = 0 then 2147483647 else strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) end asc`,
|
||||
);
|
||||
|
||||
let rows: any[] = await candidatesCapped.limit(200).execute();
|
||||
|
||||
if (rows.length === 0) {
|
||||
return { items: [] };
|
||||
}
|
||||
|
||||
// --- Permissions BEFORE limit. --------------------------------------------
|
||||
// Apply the existing page-level post-filter to the MERGED set, then rank and
|
||||
// only THEN truncate to `limit` — never lose the permission filter.
|
||||
if (opts.userId) {
|
||||
const accessibleIds =
|
||||
await this.pagePermissionRepo.filterAccessiblePageIds({
|
||||
pageIds: rows.map((r) => r.id),
|
||||
userId: opts.userId,
|
||||
spaceId: searchParams.spaceId,
|
||||
workspaceId: opts.workspaceId,
|
||||
});
|
||||
const accessibleSet = new Set(accessibleIds);
|
||||
rows = rows.filter((r) => accessibleSet.has(r.id));
|
||||
}
|
||||
|
||||
if (rows.length === 0) {
|
||||
return { items: [] };
|
||||
}
|
||||
|
||||
// --- Tiered ranking + dedup. ----------------------------------------------
|
||||
// Rows are already unique by id (single pages scan), so no cross-branch
|
||||
// dedup is needed here; the tier captures the strongest match reason.
|
||||
const ranked = rows.map((r) => {
|
||||
let tier: SearchLookupTier;
|
||||
let secondary: number;
|
||||
if (r.titleExact) {
|
||||
tier = SearchLookupTier.TITLE_EXACT;
|
||||
secondary = Number(r.ftsRank) || 0;
|
||||
} else if (r.titleSub) {
|
||||
tier = SearchLookupTier.TITLE_SUBSTRING;
|
||||
secondary = Number(r.ftsRank) || 0;
|
||||
} else {
|
||||
tier = SearchLookupTier.TEXT;
|
||||
// Prefer earlier text matches; map position → closeness in (0, 1].
|
||||
const pos = Number(r.textMatchPos) || 0;
|
||||
secondary =
|
||||
pos > 0 ? 1 / (1 + (pos - 1) / 100) : Number(r.ftsRank) || 0;
|
||||
}
|
||||
return { row: r, tier, score: computeLookupScore({ tier, secondary }) };
|
||||
});
|
||||
|
||||
ranked.sort((a, b) => b.score - a.score);
|
||||
const top = ranked.slice(0, limit);
|
||||
|
||||
// --- Batch ancestor path (ONE recursive CTE, not N+1). --------------------
|
||||
const pathById = await this.buildAncestorPaths(top.map((t) => t.row.id));
|
||||
|
||||
const items: SearchLookupResponseDto[] = top.map((t) => ({
|
||||
id: t.row.id,
|
||||
slugId: t.row.slugId,
|
||||
title: t.row.title,
|
||||
parentPageId: t.row.parentPageId ?? null,
|
||||
path: pathById.get(t.row.id) ?? [],
|
||||
snippet: (t.row.snippet ?? '')
|
||||
.replace(/\r\n|\r|\n/g, ' ')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim(),
|
||||
score: t.score,
|
||||
}));
|
||||
|
||||
return { items };
|
||||
}
|
||||
|
||||
/**
|
||||
* Batch ancestor-titles helper (#443): ONE recursive CTE seeded with ALL hit
|
||||
* ids, walking UP parentPageId. Returns a map hitId → ancestor titles ordered
|
||||
* root → direct parent (the hit's own title is excluded). Root pages map to
|
||||
* an empty array. Avoids the N+1 of a per-page breadcrumb call.
|
||||
*/
|
||||
private async buildAncestorPaths(
|
||||
hitIds: string[],
|
||||
): Promise<Map<string, string[]>> {
|
||||
const result = new Map<string, string[]>();
|
||||
if (hitIds.length === 0) return result;
|
||||
|
||||
// ancestry(hit_id, page_id, title, parent_page_id, depth): seed one row per
|
||||
// hit at depth 0 (the hit itself), then walk to parents (increasing depth).
|
||||
const rows = await this.db
|
||||
.withRecursive('ancestry', (db) =>
|
||||
db
|
||||
.selectFrom('pages')
|
||||
.select([
|
||||
'pages.id as hitId',
|
||||
'pages.id as pageId',
|
||||
'pages.title as title',
|
||||
'pages.parentPageId as parentPageId',
|
||||
sql<number>`0`.as('depth'),
|
||||
])
|
||||
.where('pages.id', 'in', hitIds)
|
||||
.unionAll((exp) =>
|
||||
exp
|
||||
.selectFrom('pages as p')
|
||||
.innerJoin('ancestry as a', 'p.id', 'a.parentPageId')
|
||||
.select([
|
||||
'a.hitId as hitId',
|
||||
'p.id as pageId',
|
||||
'p.title as title',
|
||||
'p.parentPageId as parentPageId',
|
||||
sql<number>`a.depth + 1`.as('depth'),
|
||||
]),
|
||||
),
|
||||
)
|
||||
.selectFrom('ancestry')
|
||||
.select(['hitId', 'title', 'depth'])
|
||||
// depth 0 is the hit itself — excluded from the path.
|
||||
.where('depth', '>', 0)
|
||||
.orderBy('hitId')
|
||||
// Larger depth = closer to the space root. Ordering DESC gives
|
||||
// root → parent once collected.
|
||||
.orderBy('depth', 'desc')
|
||||
.execute();
|
||||
|
||||
for (const r of rows as any[]) {
|
||||
const list = result.get(r.hitId) ?? [];
|
||||
list.push(r.title);
|
||||
result.set(r.hitId, list);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
async searchSuggestions(
|
||||
suggestion: SearchSuggestionDTO,
|
||||
userId: string,
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
import { type Kysely, sql } from 'kysely';
|
||||
|
||||
/**
|
||||
* #443 — trigram indexes for the opt-in agent-lookup search mode.
|
||||
*
|
||||
* The lookup mode adds a substring branch that runs leading-wildcard
|
||||
* `LOWER(f_unaccent(col)) LIKE '%q%'` predicates on pages.title and
|
||||
* pages.text_content. A leading wildcard cannot use a b-tree index, so without a
|
||||
* GIN trigram index each such predicate is a sequential scan.
|
||||
*
|
||||
* - TITLE: the lookup-mode title predicate is `LOWER(f_unaccent(title)) LIKE
|
||||
* '%q%'` (coalesce-free, so it can use a functional index), which is IDENTICAL
|
||||
* to the one added for /search/suggest (#348). #348's perf-indexes migration
|
||||
* already created `idx_pages_title_trgm` on `(LOWER(f_unaccent(title)))
|
||||
* gin_trgm_ops`, so the title predicate is already covered — we do NOT
|
||||
* re-create that index here (it would be redundant).
|
||||
*
|
||||
* - TEXT_CONTENT: NEW. The substring branch scans text_content when the query
|
||||
* is not titleOnly. text_content is the large column, so a GIN trigram index
|
||||
* on it is the meaningful acceleration for the lookup mode. The lookup search
|
||||
* is ALWAYS space-scoped (spaceId or the user's member spaces), so on small
|
||||
* instances a per-space sequential scan is tolerable — but the index turns the
|
||||
* `%q%` text predicate into a Bitmap Index Scan and removes the only
|
||||
* unbounded-per-space cost of the feature. We add it. The trade-off is disk +
|
||||
* write amplification on page edits (GIN trigram indexes are larger and slower
|
||||
* to update than b-trees); on the small instances this fork targets that cost
|
||||
* is acceptable and the read win on agent lookups is the priority.
|
||||
*
|
||||
* DEPLOY-TIME LOCK WARNING: plain (non-CONCURRENT) CREATE INDEX — Kysely runs
|
||||
* each migration in a transaction, so CONCURRENTLY is impossible. The build takes
|
||||
* a SHARE lock that BLOCKS writes on `pages` for its duration. The text_content
|
||||
* GIN build is the slow one and can take minutes on a large tenant. For large
|
||||
* installations, run this in a maintenance window or build the index out-of-band
|
||||
* with CREATE INDEX CONCURRENTLY before deploying (then `IF NOT EXISTS` no-ops
|
||||
* here). Small/typical tenants are unaffected.
|
||||
*/
|
||||
export async function up(db: Kysely<any>): Promise<void> {
|
||||
// The title predicate is served by #348's idx_pages_title_trgm — see header.
|
||||
// Only the text_content index is introduced here.
|
||||
|
||||
// text_content trigram index. Its expression is coalesce-free —
|
||||
// `LOWER(f_unaccent(text_content))` — to EXACTLY match the coalesce-free
|
||||
// lookup-mode text substring predicate in search.service.ts, so Postgres can
|
||||
// use it (a `coalesce(...)` mismatch would silently fall back to a Seq Scan).
|
||||
await sql`
|
||||
CREATE INDEX IF NOT EXISTS idx_pages_text_content_trgm
|
||||
ON pages USING gin ((LOWER(f_unaccent(text_content))) gin_trgm_ops)
|
||||
`.execute(db);
|
||||
}
|
||||
|
||||
export async function down(db: Kysely<any>): Promise<void> {
|
||||
// Only drop the index this migration introduced. idx_pages_title_trgm is owned
|
||||
// by the #348 perf-indexes migration, so leave it for that migration's down().
|
||||
await sql`DROP INDEX IF EXISTS idx_pages_text_content_trgm`.execute(db);
|
||||
}
|
||||
@@ -143,7 +143,7 @@ export type DocmostMcpConfig = (
|
||||
buf: Buffer,
|
||||
mime: string,
|
||||
) => { uri: string; sha256: string; size: number };
|
||||
// Optional live/evict probes the package uses to keep stash_page's mirror
|
||||
// Optional live/evict probes the package uses to keep stashPage's mirror
|
||||
// counts honest under the store's FIFO eviction (mirror of the package's
|
||||
// sink type); older bindings omit them.
|
||||
has?: (uri: string) => boolean;
|
||||
|
||||
@@ -332,7 +332,7 @@ export class McpService implements OnModuleDestroy {
|
||||
// Should never happen: handle() always stashes before delegating.
|
||||
throw new UnauthorizedException('MCP authentication missing.');
|
||||
}
|
||||
// Inject the blob-sandbox sink after the auth decision so stash_page
|
||||
// Inject the blob-sandbox sink after the auth decision so stashPage
|
||||
// can store blobs in the shared in-RAM store regardless of which
|
||||
// credential variant resolved. The sink (put/has/evict + uri↔id
|
||||
// mapping) is owned by SandboxStore.asSink().
|
||||
|
||||
@@ -182,6 +182,7 @@ describe('AiChatService run-stream attach [integration]', () => {
|
||||
{
|
||||
isAiChatDeferredToolsEnabled: () => false,
|
||||
isAiChatResumableStreamEnabled: () => true,
|
||||
isAiChatFinalStepLockdownEnabled: () => false,
|
||||
} as any,
|
||||
registry,
|
||||
);
|
||||
@@ -499,6 +500,7 @@ describe('AiChatService run-stream attach [integration]', () => {
|
||||
{
|
||||
isAiChatDeferredToolsEnabled: () => false,
|
||||
isAiChatResumableStreamEnabled: () => true,
|
||||
isAiChatFinalStepLockdownEnabled: () => false,
|
||||
} as any,
|
||||
registry,
|
||||
);
|
||||
|
||||
@@ -150,7 +150,7 @@ describe('AiChatService.stream [integration]', () => {
|
||||
{} as any, // pageAccess (idem)
|
||||
// environment (#332): keep deferred tool loading OFF for this lifecycle
|
||||
// harness so the toolset/behavior is exactly as before.
|
||||
{ isAiChatDeferredToolsEnabled: () => false } as any,
|
||||
{ isAiChatDeferredToolsEnabled: () => false, isAiChatFinalStepLockdownEnabled: () => false } as any,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -378,7 +378,7 @@ describe('AiChatService.stream [integration]', () => {
|
||||
{} as any,
|
||||
{} as any,
|
||||
// #332: deferred tool loading ON — the property under test.
|
||||
{ isAiChatDeferredToolsEnabled: () => true } as any,
|
||||
{ isAiChatDeferredToolsEnabled: () => true, isAiChatFinalStepLockdownEnabled: () => false } as any,
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,123 @@
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { Kysely, sql } from 'kysely';
|
||||
import {
|
||||
getTestDb,
|
||||
destroyTestDb,
|
||||
createWorkspace,
|
||||
createSpace,
|
||||
} from './db';
|
||||
|
||||
/**
|
||||
* #443 dead-index guard — EXPLAIN on the REAL DB.
|
||||
*
|
||||
* The lookup mode's substring predicates run a leading-wildcard
|
||||
* `LOWER(f_unaccent(col)) LIKE '%q%'`. Those are only fast when Postgres uses
|
||||
* the GIN trigram indexes:
|
||||
* - idx_pages_title_trgm on (LOWER(f_unaccent(title))) [#348]
|
||||
* - idx_pages_text_content_trgm on (LOWER(f_unaccent(text_content))) [#443]
|
||||
*
|
||||
* Postgres uses a functional index ONLY when the query expression matches the
|
||||
* index expression EXACTLY. The original lookup query wrapped the columns in
|
||||
* `coalesce(col,'')`, which differs from the coalesce-FREE index expression and
|
||||
* silently forced a Seq Scan on pages for EVERY lookup (the MCP client always
|
||||
* sends substring:true). This test locks that in.
|
||||
*
|
||||
* Discriminator: `SET enable_seqscan = off` asks the planner "CAN this predicate
|
||||
* use the index at all?" — which is exactly what the coalesce bug breaks. With
|
||||
* seqscan disabled:
|
||||
* - the coalesce-FREE (fixed) predicate plans a Bitmap Index Scan on the trgm
|
||||
* index (no Seq Scan on pages);
|
||||
* - the coalesce-WRAPPED (buggy) predicate cannot use the index and falls back
|
||||
* to a Seq Scan on pages even though seqscan is disabled.
|
||||
* We assert both to prove the fix and to keep the regression from silently
|
||||
* returning.
|
||||
*/
|
||||
describe('SearchService agent-lookup EXPLAIN — trgm index is live [integration]', () => {
|
||||
let db: Kysely<any>;
|
||||
let workspaceId: string;
|
||||
let spaceId: string;
|
||||
|
||||
async function insertPage(title: string, textContent: string): Promise<void> {
|
||||
const id = randomUUID();
|
||||
await db
|
||||
.insertInto('pages')
|
||||
.values({
|
||||
id,
|
||||
slugId: `slug-${id.slice(0, 12)}`,
|
||||
title,
|
||||
textContent,
|
||||
spaceId,
|
||||
workspaceId,
|
||||
})
|
||||
.execute();
|
||||
}
|
||||
|
||||
// Run EXPLAIN (no ANALYZE — we only inspect the chosen plan) and return the
|
||||
// concatenated plan text.
|
||||
async function explain(query: string): Promise<string> {
|
||||
const rows = await sql<{ 'QUERY PLAN': string }>`EXPLAIN ${sql.raw(query)}`.execute(
|
||||
db,
|
||||
);
|
||||
return (rows.rows as any[]).map((r) => r['QUERY PLAN']).join('\n');
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
db = getTestDb();
|
||||
workspaceId = (await createWorkspace(db)).id;
|
||||
spaceId = (await createSpace(db, workspaceId)).id;
|
||||
|
||||
// Seed enough rows that a trigram index is a plausible plan. The content is
|
||||
// varied so the '%needle%' pattern is selective.
|
||||
for (let i = 0; i < 200; i++) {
|
||||
await insertPage(
|
||||
`seed-title-${i}`,
|
||||
`seed body content number ${i} lorem ipsum dolor sit amet ${i}`,
|
||||
);
|
||||
}
|
||||
await insertPage('backup-srv.local', 'the needle-token-xyz lives here');
|
||||
|
||||
// Keep the trgm indexes' stats fresh so the planner costs them correctly.
|
||||
await sql`ANALYZE pages`.execute(db);
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await destroyTestDb();
|
||||
});
|
||||
|
||||
// Force the planner to answer "can the index be used?" rather than "is it
|
||||
// cheaper than a seq scan on this size?". Restored after each test.
|
||||
beforeEach(async () => {
|
||||
await sql`SET enable_seqscan = off`.execute(db);
|
||||
});
|
||||
afterEach(async () => {
|
||||
await sql`RESET enable_seqscan`.execute(db);
|
||||
});
|
||||
|
||||
it('title predicate (coalesce-FREE, as fixed) uses idx_pages_title_trgm, not a Seq Scan', async () => {
|
||||
const plan = await explain(
|
||||
`SELECT id FROM pages WHERE LOWER(f_unaccent(title)) LIKE '%srv.local%'`,
|
||||
);
|
||||
expect(plan).toContain('idx_pages_title_trgm');
|
||||
expect(plan).not.toMatch(/Seq Scan on pages/i);
|
||||
});
|
||||
|
||||
it('text_content predicate (coalesce-FREE, as fixed) uses idx_pages_text_content_trgm, not a Seq Scan', async () => {
|
||||
const plan = await explain(
|
||||
`SELECT id FROM pages WHERE LOWER(f_unaccent(text_content)) LIKE '%needle-token%'`,
|
||||
);
|
||||
expect(plan).toContain('idx_pages_text_content_trgm');
|
||||
expect(plan).not.toMatch(/Seq Scan on pages/i);
|
||||
});
|
||||
|
||||
// Negative control: the OLD coalesce-wrapped predicate must NOT be able to use
|
||||
// the index — even with seqscan disabled it can only Seq Scan pages. If this
|
||||
// ever stops seq-scanning, the coalesce/index expressions have re-aligned and
|
||||
// the guard above is no longer meaningful.
|
||||
it('coalesce-WRAPPED text predicate (the bug) cannot use the index — falls to Seq Scan', async () => {
|
||||
const plan = await explain(
|
||||
`SELECT id FROM pages WHERE LOWER(f_unaccent(coalesce(text_content,''))) LIKE '%needle-token%'`,
|
||||
);
|
||||
expect(plan).not.toContain('idx_pages_text_content_trgm');
|
||||
expect(plan).toMatch(/Seq Scan on pages/i);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,462 @@
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { Kysely } from 'kysely';
|
||||
import { SearchService } from 'src/core/search/search.service';
|
||||
import { PageRepo } from '@docmost/db/repos/page/page.repo';
|
||||
import {
|
||||
getTestDb,
|
||||
destroyTestDb,
|
||||
createWorkspace,
|
||||
createSpace,
|
||||
} from './db';
|
||||
|
||||
/**
|
||||
* #443 — agent-lookup search mode, acceptance on the REAL DB schema.
|
||||
*
|
||||
* Exercises SearchService.searchPage(..., { substring: true }) against a
|
||||
* migrated Postgres: substring matching of technical tokens the FTS tokenizer
|
||||
* mangles (backup-srv.local, 10.0.12.5, WB-MGE-30D86B, "Теги: Docker"), the
|
||||
* populated path + snippet, parentPageId subtree scoping, titleOnly, the empty
|
||||
* result, LIKE-metacharacter escaping (`%`/`_` must NOT match everything), the
|
||||
* permission post-filter applied BEFORE the limit, and the web-UI path staying
|
||||
* on the legacy FTS shape when `substring` is absent.
|
||||
*
|
||||
* The tsv column is populated by the pages_tsvector_trigger on insert, so the
|
||||
* FTS branch is exercised too.
|
||||
*/
|
||||
describe('SearchService agent-lookup mode [integration]', () => {
|
||||
let db: Kysely<any>;
|
||||
let service: SearchService;
|
||||
let workspaceId: string;
|
||||
let spaceId: string;
|
||||
|
||||
// Direct page insert (the shared createPage seeder omits text_content /
|
||||
// parent_page_id, both of which this mode depends on). Returns the id.
|
||||
async function insertPage(args: {
|
||||
title: string;
|
||||
textContent?: string;
|
||||
parentPageId?: string | null;
|
||||
spaceId?: string;
|
||||
}): Promise<string> {
|
||||
const id = randomUUID();
|
||||
await db
|
||||
.insertInto('pages')
|
||||
.values({
|
||||
id,
|
||||
slugId: `slug-${id.slice(0, 12)}`,
|
||||
title: args.title,
|
||||
textContent: args.textContent ?? null,
|
||||
parentPageId: args.parentPageId ?? null,
|
||||
spaceId: args.spaceId ?? spaceId,
|
||||
workspaceId,
|
||||
})
|
||||
.execute();
|
||||
return id;
|
||||
}
|
||||
|
||||
// Build a SearchService wired to the real DB + a real PageRepo (only its
|
||||
// recursive-descendants method is used by this mode, and it needs only `db`),
|
||||
// with lightweight stubs for the space-membership and permission repos so a
|
||||
// test can drive scope + the permission post-filter explicitly.
|
||||
function buildService(opts?: {
|
||||
userSpaceIds?: string[];
|
||||
// ids to KEEP after the permission post-filter; undefined = keep all.
|
||||
accessibleIds?: string[];
|
||||
}): SearchService {
|
||||
const pageRepo = new PageRepo(db as any, null as any, null as any);
|
||||
const spaceMemberRepo = {
|
||||
getUserSpaceIds: async () => opts?.userSpaceIds ?? [spaceId],
|
||||
};
|
||||
const pagePermissionRepo = {
|
||||
filterAccessiblePageIds: async ({ pageIds }: { pageIds: string[] }) =>
|
||||
opts?.accessibleIds
|
||||
? pageIds.filter((id) => opts.accessibleIds!.includes(id))
|
||||
: pageIds,
|
||||
};
|
||||
return new SearchService(
|
||||
db as any,
|
||||
pageRepo as any,
|
||||
{} as any, // shareRepo — unused by the lookup path
|
||||
spaceMemberRepo as any,
|
||||
pagePermissionRepo as any,
|
||||
);
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
db = getTestDb();
|
||||
workspaceId = (await createWorkspace(db)).id;
|
||||
spaceId = (await createSpace(db, workspaceId)).id;
|
||||
service = buildService();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await destroyTestDb();
|
||||
});
|
||||
|
||||
it('finds `backup-srv.local` by the fragment `srv.local`', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'backup-srv.local',
|
||||
textContent: 'A backup server node.',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'srv.local', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
expect(items.map((i: any) => i.id)).toContain(pageId);
|
||||
const hit = items.find((i: any) => i.id === pageId);
|
||||
expect(hit.title).toBe('backup-srv.local');
|
||||
// slugId must never be part of the server response shape.
|
||||
expect('slugId' in hit).toBe(true); // server carries it; MCP strips it
|
||||
});
|
||||
|
||||
it('finds a page whose TEXT contains `10.0.12.5` by the fragment `10.0.12` (empty-tsquery case)', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'Server inventory',
|
||||
textContent: 'The backup box lives at IP: 10.0.12.5. Debian 12, backups.',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: '10.0.12', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const hit = items.find((i: any) => i.id === pageId);
|
||||
expect(hit).toBeDefined();
|
||||
// The windowed snippet must include the matched text.
|
||||
expect(hit.snippet).toContain('10.0.12.5');
|
||||
});
|
||||
|
||||
it('finds `WB-MGE-30D86B` (alphanumeric token with dashes) by title', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'WB-MGE-30D86B',
|
||||
textContent: 'Device page.',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'WB-MGE-30D86B', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const hit = items.find((i: any) => i.id === pageId);
|
||||
expect(hit).toBeDefined();
|
||||
// Exact title match → top tier (TITLE_EXACT=3) → score in [0.75, 1].
|
||||
expect(hit.score).toBeGreaterThanOrEqual(0.75);
|
||||
// And it is the top-ranked hit of its own result set.
|
||||
expect(items[0].id).toBe(pageId);
|
||||
});
|
||||
|
||||
it('finds every page whose text literally contains `Теги: Docker`', async () => {
|
||||
const a = await insertPage({
|
||||
title: 'Container host A',
|
||||
textContent: 'Some notes.\nТеги: Docker, compose\nmore.',
|
||||
});
|
||||
const b = await insertPage({
|
||||
title: 'Container host B',
|
||||
textContent: 'Prelude.\nТеги: Docker\nepilogue.',
|
||||
});
|
||||
const noise = await insertPage({
|
||||
title: 'Unrelated',
|
||||
textContent: 'Теги: Kubernetes',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'Теги: Docker', spaceId, substring: true, limit: 50 } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
expect(ids).toContain(a);
|
||||
expect(ids).toContain(b);
|
||||
expect(ids).not.toContain(noise);
|
||||
});
|
||||
|
||||
it('populates a non-empty `path` for a nested hit and `[]` for a root hit', async () => {
|
||||
const root = await insertPage({ title: 'Infrastructure' });
|
||||
const mid = await insertPage({ title: 'Datacenter A', parentPageId: root });
|
||||
const leaf = await insertPage({
|
||||
title: 'unique-nested-host',
|
||||
parentPageId: mid,
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'unique-nested-host', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const hit = items.find((i: any) => i.id === leaf);
|
||||
expect(hit.path).toEqual(['Infrastructure', 'Datacenter A']);
|
||||
|
||||
const rootHits = (await service.searchPage(
|
||||
{ query: 'Infrastructure', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
const rootHit = rootHits.items.find((i: any) => i.id === root);
|
||||
expect(rootHit.path).toEqual([]);
|
||||
});
|
||||
|
||||
it('scopes to a subtree with parentPageId (cutting off sibling branches)', async () => {
|
||||
const branchA = await insertPage({ title: 'BranchA-root' });
|
||||
const inA = await insertPage({
|
||||
title: 'scoped-target-xyz',
|
||||
parentPageId: branchA,
|
||||
});
|
||||
const branchB = await insertPage({ title: 'BranchB-root' });
|
||||
const inB = await insertPage({
|
||||
title: 'scoped-target-xyz',
|
||||
parentPageId: branchB,
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{
|
||||
query: 'scoped-target-xyz',
|
||||
spaceId,
|
||||
substring: true,
|
||||
parentPageId: branchA,
|
||||
} as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
expect(ids).toContain(inA);
|
||||
expect(ids).not.toContain(inB);
|
||||
});
|
||||
|
||||
it('includes the parent page itself in the parentPageId subtree', async () => {
|
||||
const parent = await insertPage({ title: 'self-included-parent' });
|
||||
await insertPage({ title: 'child-of-self', parentPageId: parent });
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{
|
||||
query: 'self-included-parent',
|
||||
spaceId,
|
||||
substring: true,
|
||||
parentPageId: parent,
|
||||
} as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
expect(items.map((i: any) => i.id)).toContain(parent);
|
||||
});
|
||||
|
||||
it('titleOnly does NOT match on text_content', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'Plain title',
|
||||
textContent: 'body mentions the-secret-token here',
|
||||
});
|
||||
|
||||
const withText = (await service.searchPage(
|
||||
{ query: 'the-secret-token', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
expect(withText.items.map((i: any) => i.id)).toContain(pageId);
|
||||
|
||||
const titleOnly = (await service.searchPage(
|
||||
{
|
||||
query: 'the-secret-token',
|
||||
spaceId,
|
||||
substring: true,
|
||||
titleOnly: true,
|
||||
} as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
expect(titleOnly.items.map((i: any) => i.id)).not.toContain(pageId);
|
||||
});
|
||||
|
||||
// #443 Fix #1 regression: f_unaccent is NOT length-preserving, so an
|
||||
// expanding char (ß→ss, …→...) BEFORE the match shifted the strpos position
|
||||
// relative to the ORIGINAL text and the snippet slice ran past end → empty.
|
||||
// The position and the slice now share the LOWER(f_unaccent(...)) space, so
|
||||
// the window is aligned and always contains the matched (unaccented) token.
|
||||
it('returns a populated snippet when an unaccent-EXPANDING char precedes the match', async () => {
|
||||
// 300 × `ß` (each f_unaccent-expands to `ss`) before the needle. Under the
|
||||
// old code strpos returned a position ~593 in the expanded space but the
|
||||
// slice ran over the ORIGINAL (~360 char) text → empty snippet, match lost.
|
||||
const prefix = 'ß'.repeat(300);
|
||||
const pageId = await insertPage({
|
||||
title: 'Expanding-unaccent page',
|
||||
textContent: `${prefix} needle-token-xyz trailing.`,
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'needle-token-xyz', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const hit = items.find((i: any) => i.id === pageId);
|
||||
expect(hit).toBeDefined();
|
||||
// Snippet must be non-empty AND contain the matched token (unaccented form).
|
||||
expect(hit.snippet.length).toBeGreaterThan(0);
|
||||
expect(hit.snippet).toContain('needle-token-xyz');
|
||||
});
|
||||
|
||||
// #443 Fix #2 regression: >200 matching pages for a broad substring, with
|
||||
// exactly ONE exact-title hit. Without an ORDER BY on the 200-cap the exact
|
||||
// hit could be among the arbitrarily-dropped rows; the ORDER BY keeps the
|
||||
// strongest candidates so it must survive the cap and rank at the top.
|
||||
it('keeps an exact-title hit through the 200-cap on a >200-row match set', async () => {
|
||||
const isoSpace = (await createSpace(db, workspaceId)).id;
|
||||
const svc = buildService({ userSpaceIds: [isoSpace] });
|
||||
|
||||
// 250 low-tier TEXT hits: the shared substring `capword` appears only in the
|
||||
// body, never the title, so each is a TEXT-tier match (weakest tier).
|
||||
for (let i = 0; i < 250; i++) {
|
||||
await insertPage({
|
||||
title: `filler-page-${i}`,
|
||||
textContent: `body contains capword here #${i}`,
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
}
|
||||
// Exactly one EXACT-title hit for the same query token.
|
||||
const exact = await insertPage({
|
||||
title: 'capword',
|
||||
textContent: 'unrelated body text',
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
|
||||
const { items } = (await svc.searchPage(
|
||||
{ query: 'capword', spaceId: isoSpace, substring: true, limit: 10 } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
// The exact-title hit must survive the 200-cap and appear in the top `limit`.
|
||||
expect(ids).toContain(exact);
|
||||
// And, being TITLE_EXACT, it must be the single strongest hit.
|
||||
expect(items[0].id).toBe(exact);
|
||||
});
|
||||
|
||||
// #443 Fix #3: titleOnly matches only the title, so it must not leak the page
|
||||
// body as the snippet (the old "first 300 chars of text_content" fallback).
|
||||
it('titleOnly does NOT return a text-body snippet', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'titleonly-snippet-page',
|
||||
textContent: 'SECRET-BODY-CONTENT-NOT-IN-TITLE that must not leak.',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{
|
||||
query: 'titleonly-snippet-page',
|
||||
spaceId,
|
||||
substring: true,
|
||||
titleOnly: true,
|
||||
} as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const hit = items.find((i: any) => i.id === pageId);
|
||||
expect(hit).toBeDefined();
|
||||
// The body text must not appear in the snippet; titleOnly → empty snippet.
|
||||
expect(hit.snippet).not.toContain('SECRET-BODY-CONTENT-NOT-IN-TITLE');
|
||||
expect(hit.snippet).toBe('');
|
||||
});
|
||||
|
||||
it('returns [] (not an error) for a query that matches nothing', async () => {
|
||||
const { items } = (await service.searchPage(
|
||||
{
|
||||
query: 'zzz-no-such-string-anywhere-42',
|
||||
spaceId,
|
||||
substring: true,
|
||||
} as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
expect(items).toEqual([]);
|
||||
});
|
||||
|
||||
it('a `%` query does NOT match everything (LIKE metacharacter escaped)', async () => {
|
||||
// Fresh space so we can assert on total counts without cross-test noise.
|
||||
const isoSpace = (await createSpace(db, workspaceId)).id;
|
||||
const svc = buildService({ userSpaceIds: [isoSpace] });
|
||||
await insertPage({ title: 'alpha', spaceId: isoSpace });
|
||||
await insertPage({ title: 'beta', spaceId: isoSpace });
|
||||
const literal = await insertPage({
|
||||
title: '100%-coverage',
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
|
||||
const { items } = (await svc.searchPage(
|
||||
{ query: '%', spaceId: isoSpace, substring: true, limit: 50 } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
// `%` is a literal → matches only the page that actually contains '%'.
|
||||
expect(ids).toContain(literal);
|
||||
expect(ids).not.toContain(
|
||||
items.find((i: any) => i.title === 'alpha')?.id,
|
||||
);
|
||||
expect(items.length).toBe(1);
|
||||
});
|
||||
|
||||
it('an `_` query does NOT match everything (LIKE metacharacter escaped)', async () => {
|
||||
const isoSpace = (await createSpace(db, workspaceId)).id;
|
||||
const svc = buildService({ userSpaceIds: [isoSpace] });
|
||||
await insertPage({ title: 'gamma', spaceId: isoSpace });
|
||||
const literal = await insertPage({
|
||||
title: 'snake_case_name',
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
|
||||
const { items } = (await svc.searchPage(
|
||||
{ query: '_', spaceId: isoSpace, substring: true, limit: 50 } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
expect(ids).toContain(literal);
|
||||
expect(items.length).toBe(1);
|
||||
});
|
||||
|
||||
it('applies the permission post-filter to the MERGED set BEFORE the limit', async () => {
|
||||
const isoSpace = (await createSpace(db, workspaceId)).id;
|
||||
const keep = await insertPage({
|
||||
title: 'perm-visible-target',
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
const hidden = await insertPage({
|
||||
title: 'perm-hidden-target',
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
|
||||
// Authenticated (userId set) so the permission filter runs; only `keep` is
|
||||
// accessible. limit 1 must NOT be able to select `hidden`.
|
||||
const svc = buildService({
|
||||
userSpaceIds: [isoSpace],
|
||||
accessibleIds: [keep],
|
||||
});
|
||||
const { items } = (await svc.searchPage(
|
||||
{
|
||||
query: 'perm-',
|
||||
spaceId: isoSpace,
|
||||
substring: true,
|
||||
limit: 1,
|
||||
} as any,
|
||||
{ userId: 'user-1', workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
expect(ids).toContain(keep);
|
||||
expect(ids).not.toContain(hidden);
|
||||
});
|
||||
|
||||
it('web-UI path (no `substring` flag) keeps the legacy FTS response shape', async () => {
|
||||
await insertPage({
|
||||
title: 'legacy shape page',
|
||||
textContent: 'searchable legacyword content',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'legacyword', spaceId } as any,
|
||||
{ userId: 'user-1', workspaceId },
|
||||
)) as any;
|
||||
|
||||
// Legacy hits carry rank + highlight + space, and NO path/snippet/score.
|
||||
const hit = items[0];
|
||||
expect(hit).toBeDefined();
|
||||
expect('rank' in hit).toBe(true);
|
||||
expect('highlight' in hit).toBe(true);
|
||||
expect('path' in hit).toBe(false);
|
||||
expect('snippet' in hit).toBe(false);
|
||||
expect('score' in hit).toBe(false);
|
||||
});
|
||||
});
|
||||
+115
-88
@@ -12,7 +12,7 @@ license.
|
||||
> better at *writing a small function that fixes the text* than at re-reading and
|
||||
> re-emitting a whole document. So this server is built around the way a model actually
|
||||
> wants to edit: address a block by id, run a find/replace, or hand it a
|
||||
> `(doc, ctx) => doc` transform and let it *program* the change. `docmost_transform` is
|
||||
> `(doc, ctx) => doc` transform and let it *program* the change. `docmostTransform` is
|
||||
> that interface. Other Docmost MCPs are human-shaped — they expose "open the page" and
|
||||
> "replace the page"; this one exposes the editing primitives a model is good at.
|
||||
|
||||
@@ -40,7 +40,7 @@ There are several Docmost MCPs. Here is a capability-by-capability comparison.
|
||||
| **Enterprise license required** | **No** | **Yes** | No | No | No |
|
||||
| Authentication | email + password, **auto re-auth** | API key | email + password | cookie `authToken` (copy from DevTools) | Docmost API / **direct PostgreSQL** |
|
||||
| Read page as Markdown | ✅ | ✅ | ✅ | ✅ | ✅ (read-only) |
|
||||
| **Lossless Markdown round-trip** (export / import, keeps comment anchors) | ✅ | — | — | — | — |
|
||||
| **Markdown round-trip** (export / import, keeps comment anchors) | ✅ | — | — | — | — |
|
||||
| Read **lossless ProseMirror JSON** (with block ids) | ✅ | — | — | — | — |
|
||||
| **Compact page outline** (cheap block-id lookup) | ✅ | — | — | — | — |
|
||||
| **Fetch a single block** (by id or index) | ✅ | — | — | — | — |
|
||||
@@ -69,9 +69,9 @@ There are several Docmost MCPs. Here is a capability-by-capability comparison.
|
||||
- **Token-efficient editing.** Most Docmost MCPs (and the official one) only offer
|
||||
"replace the whole page" writes — the agent must download the entire document, mutate
|
||||
it, and upload it back, paying for the full document **twice** on every tiny fix.
|
||||
This server lets the agent change exactly one block (`patch_node` / `insert_node` /
|
||||
`delete_node`), do a structure-preserving find/replace (`edit_page_text`), or copy a
|
||||
whole page server-side (`copy_page_content`) — **without the document ever passing
|
||||
This server lets the agent change exactly one block (`patchNode` / `insertNode` /
|
||||
`deleteNode`), do a structure-preserving find/replace (`editPageText`), or copy a
|
||||
whole page server-side (`copyPageContent`) — **without the document ever passing
|
||||
through the model**.
|
||||
|
||||
- **Writes that don't fight the editor.** Naive REST writes race with whatever a human
|
||||
@@ -85,12 +85,12 @@ There are several Docmost MCPs. Here is a capability-by-capability comparison.
|
||||
- **Agent-native editing model.** Human-facing servers expose "open the page" and "replace
|
||||
the page", because that mirrors how a person works. A model edits better by *programming*
|
||||
the change — addressing blocks by id, running a find/replace, or supplying a
|
||||
`(doc, ctx) => doc` transform (`docmost_transform`, with a dry-run diff before it
|
||||
`(doc, ctx) => doc` transform (`docmostTransform`, with a dry-run diff before it
|
||||
commits). This server is shaped around that, which is why it has editing primitives the
|
||||
others simply don't.
|
||||
|
||||
- **An editing safety net the others lack.** `list_page_history` → `diff_page_versions`
|
||||
→ `restore_page_version` give an agent (and you) a full view-and-undo loop. The diff
|
||||
- **An editing safety net the others lack.** `listPageHistory` → `diffPageVersions`
|
||||
→ `restorePageVersion` give an agent (and you) a full view-and-undo loop. The diff
|
||||
uses the *same* `recreateTransform → ChangeSet → simplifyChanges` pipeline Docmost's
|
||||
own history viewer uses, so what you see matches the product.
|
||||
|
||||
@@ -110,56 +110,58 @@ All 41 tools, grouped by what you'd reach for them.
|
||||
|
||||
### Exploration & retrieval
|
||||
|
||||
- **`get_workspace`** — Information about the current Docmost workspace.
|
||||
- **`list_spaces`** — All spaces in the workspace.
|
||||
- **`list_pages`** — Recent pages in a space, ordered by `updatedAt` desc (default 50,
|
||||
- **`getWorkspace`** — Information about the current Docmost workspace.
|
||||
- **`listSpaces`** — All spaces in the workspace.
|
||||
- **`listPages`** — Recent pages in a space, ordered by `updatedAt` desc (default 50,
|
||||
max 100). Use `search` for lookups in large spaces.
|
||||
- **`search`** — Full-text search across pages and content (bounded by `limit`, max 100).
|
||||
- **`get_page`** — A page's content as clean **Markdown** (convenient, but a *lossy*
|
||||
view — block ids and exact table/callout structure are approximated).
|
||||
- **`get_page_json`** — A page's **lossless ProseMirror/TipTap JSON**, including every
|
||||
- **`getPage`** — A page's content as clean **Markdown** (canonical for text; drops only
|
||||
block ids, resolved-comment anchors, and a fixed no-Markdown-representation attr set —
|
||||
table spans/colwidth/background, indent, `callout.icon`, `orderedList.type`, and link
|
||||
`internal`/`target`/`rel`/`class`; use `getPageJson` when you need those).
|
||||
- **`getPageJson`** — A page's **lossless ProseMirror/TipTap JSON**, including every
|
||||
block's `attrs.id` and the `slugId` used in URLs. This is what the per-block editing
|
||||
tools consume.
|
||||
- **`get_outline`** — A compact outline of a page's top-level blocks (`{index, type, id,
|
||||
- **`getOutline`** — A compact outline of a page's top-level blocks (`{index, type, id,
|
||||
level, firstText}`; tables add row/column counts and their header-cell texts, lists add
|
||||
item counts) **without** the document body. The cheap way to locate a section or table
|
||||
and grab its block id before
|
||||
`get_node` / `patch_node` / `insert_node`.
|
||||
- **`get_node`** — Fetch a single block's full ProseMirror subtree (lossless) without
|
||||
pulling the whole page. Address it by a block id (from `get_outline` / `get_page_json`),
|
||||
`getNode` / `patchNode` / `insertNode`.
|
||||
- **`getNode`** — Fetch a single block's full ProseMirror subtree (lossless) without
|
||||
pulling the whole page. Address it by a block id (from `getOutline` / `getPageJson`),
|
||||
or by `#<index>` for a top-level block — use the `#<index>` form for tables/rows/cells,
|
||||
which carry no id.
|
||||
|
||||
### Page lifecycle
|
||||
|
||||
- **`create_page`** — Create a page from Markdown and place it in the hierarchy (optional
|
||||
- **`createPage`** — Create a page from Markdown and place it in the hierarchy (optional
|
||||
`parentPageId`) in one call. Uses Docmost's import API for clean Markdown→ProseMirror.
|
||||
- **`rename_page`** — Change a page's title only, without touching or resending content.
|
||||
- **`move_page`** — Re-parent a page (nest it, or move to root); supports fractional-index
|
||||
- **`renamePage`** — Change a page's title only, without touching or resending content.
|
||||
- **`movePage`** — Re-parent a page (nest it, or move to root); supports fractional-index
|
||||
positioning. Returns only on a *positively confirmed* success.
|
||||
- **`delete_page`** — Delete a single page.
|
||||
- **`copy_page_content`** — Replace one page's body with a copy of another's, **entirely
|
||||
- **`deletePage`** — Delete a single page.
|
||||
- **`copyPageContent`** — Replace one page's body with a copy of another's, **entirely
|
||||
server-side** — the document never passes through the model. The target keeps its own
|
||||
title and slug (so its URL is preserved).
|
||||
|
||||
### Editing
|
||||
|
||||
- **`edit_page_text`** — Surgical find/replace inside a page's text. Preserves **all**
|
||||
- **`editPageText`** — Surgical find/replace inside a page's text. Preserves **all**
|
||||
structure: block ids, marks, links, callouts, tables. The preferred tool for fixing
|
||||
wording, typos, numbers and names.
|
||||
- **`patch_node`** — Replace a single block addressed by its `attrs.id` (from
|
||||
`get_page_json`), without resending the document.
|
||||
- **`insert_node`** — Insert a block before/after another (by `attrs.id` or anchor text),
|
||||
- **`patchNode`** — Replace a single block addressed by its `attrs.id` (from
|
||||
`getPageJson`), without resending the document.
|
||||
- **`insertNode`** — Insert a block before/after another (by `attrs.id` or anchor text),
|
||||
or append at the end.
|
||||
- **`delete_node`** — Remove a single block by its `attrs.id`.
|
||||
- **`update_page_json`** — Replace a page's entire content with a ProseMirror document
|
||||
- **`deleteNode`** — Remove a single block by its `attrs.id`.
|
||||
- **`updatePageJson`** — Replace a page's entire content with a ProseMirror document
|
||||
(bulk rewrites, or when nodes lack ids). `content` is optional — omit it to update only
|
||||
the title. Keeps the block ids you pass in, so heading anchors and history stay stable.
|
||||
- **`update_page_markdown`** — Replace a page's body (and optionally its title) with new
|
||||
- **`updatePageMarkdown`** — Replace a page's body (and optionally its title) with new
|
||||
**plain Markdown**. The whole body is re-imported (block ids regenerate — for surgical or
|
||||
id-preserving edits prefer `edit_page_text` / `patch_node` / `update_page_json`).
|
||||
id-preserving edits prefer `editPageText` / `patchNode` / `updatePageJson`).
|
||||
Docmost-flavoured markdown is parsed, including `^[...]` inline footnotes.
|
||||
- **`docmost_transform`** — The agent-native editing interface: instead of retyping a
|
||||
- **`docmostTransform`** — The agent-native editing interface: instead of retyping a
|
||||
document, the agent **writes a function that fixes it**. Edit a page by running an
|
||||
arbitrary **`(doc, ctx) => doc` JavaScript transform** against its *live* ProseMirror
|
||||
document. Runs **sandboxed**
|
||||
@@ -172,42 +174,46 @@ All 41 tools, grouped by what you'd reach for them.
|
||||
|
||||
### Tables
|
||||
|
||||
- **`table_get`** — Read a table as a matrix: `{rows, cols, cells (text[][]), cellIds}`
|
||||
- **`tableGet`** — Read a table as a matrix: `{rows, cols, cells (text[][]), cellIds}`
|
||||
(a paragraph id per cell, or `null`). Address the table by `#<index>` (from
|
||||
`get_outline`) or any block id inside it. Use `cellIds` with `patch_node` for
|
||||
`getOutline`) or any block id inside it. Use `cellIds` with `patchNode` for
|
||||
rich-formatted cell edits.
|
||||
- **`table_insert_row`** — Insert a row of plain-text cells, padded to the table's column
|
||||
- **`tableInsertRow`** — Insert a row of plain-text cells, padded to the table's column
|
||||
count (passing more cells than columns is an error). `index` is the 0-based insert
|
||||
position (0 inserts before the header); omit it to append at the end.
|
||||
- **`table_delete_row`** — Delete the row at a 0-based `index`. Refuses to delete a table's
|
||||
- **`tableDeleteRow`** — Delete the row at a 0-based `index`. Refuses to delete a table's
|
||||
only row; deleting row 0 promotes the next row to header.
|
||||
- **`table_update_cell`** — Set the plain-text content of cell `[row, col]` (0-based). For
|
||||
rich formatting, `patch_node` the cell's paragraph id from `table_get`.
|
||||
- **`tableUpdateCell`** — Set the plain-text content of cell `[row, col]` (0-based). For
|
||||
rich formatting, `patchNode` the cell's paragraph id from `tableGet`.
|
||||
|
||||
### Markdown round-trip
|
||||
|
||||
- **`export_page_markdown`** — Export a page to a single self-contained, **lossless
|
||||
Docmost-flavoured Markdown** file: a meta header, the body with inline comment anchors
|
||||
and diagrams, and a trailing comments-thread block. To replace a page's body from plain
|
||||
authoring Markdown, use `update_page_markdown`.
|
||||
- **`exportPageMarkdown`** — Export a page to a single self-contained
|
||||
**Docmost-flavoured Markdown** file: a meta header, the body with inline comment anchors
|
||||
and diagrams, and a trailing comments-thread block. The download → edit → import
|
||||
round-trip regenerates block ids and **silently drops** the no-Markdown-representation
|
||||
attr set (table merge spans/colwidth/background, indent, `callout.icon`,
|
||||
`orderedList.type`, link `internal`/`target`/`rel`/`class`); keep those in ProseMirror
|
||||
JSON if they must survive. To replace a page's body from plain authoring Markdown, use
|
||||
`updatePageMarkdown`.
|
||||
|
||||
> **Removed in this release:** `import_page_markdown` (the round-trip parser for an
|
||||
> **Removed in this release:** `importPageMarkdown` (the round-trip parser for an
|
||||
> exported Docmost-Markdown file) is **no longer exposed on the external MCP surface**.
|
||||
> To replace a page's body from Markdown, use **`update_page_markdown`** (plain Markdown
|
||||
> To replace a page's body from Markdown, use **`updatePageMarkdown`** (plain Markdown
|
||||
> body replace). See the CHANGELOG for the migration note.
|
||||
|
||||
### Images
|
||||
|
||||
- **`insert_image`** — Download an image from a web (http/https) URL and insert it in one
|
||||
- **`insertImage`** — Download an image from a web (http/https) URL and insert it in one
|
||||
step: append it, drop it in place of a text placeholder (`replaceText`), or put it after
|
||||
a given block (`afterText`). Preserves all other block ids.
|
||||
- **`replace_image`** — Swap an existing image for one fetched from a web (http/https) URL.
|
||||
- **`replaceImage`** — Swap an existing image for one fetched from a web (http/https) URL.
|
||||
Uploads the new file as a **fresh
|
||||
attachment** (clean URL that renders and busts browser caches), then re-points every
|
||||
node referencing the old attachment (recursively, including callouts/tables) via the
|
||||
live document, preserving comments, alignment and alt text. (In-place overwrite is
|
||||
deliberately avoided — some Docmost versions corrupt the attachment on overwrite.)
|
||||
- **`stash_page`** — Serialize a whole page (its full ProseMirror JSON) into an ephemeral
|
||||
- **`stashPage`** — Serialize a whole page (its full ProseMirror JSON) into an ephemeral
|
||||
in-RAM blob and return ONLY a short anonymous URL — the body never enters the model
|
||||
context, so it is the way to hand a large page (and its images) to an external consumer
|
||||
without truncation. Every internal file/image attachment is mirrored into the same
|
||||
@@ -218,35 +224,35 @@ All 41 tools, grouped by what you'd reach for them.
|
||||
|
||||
### Comments
|
||||
|
||||
- **`create_comment`** — Add a page comment, optionally **anchored inline** to an exact
|
||||
- **`createComment`** — Add a page comment, optionally **anchored inline** to an exact
|
||||
span of text (the first occurrence is wrapped in a comment mark).
|
||||
- **`list_comments`** — List a page's comments (content returned as Markdown).
|
||||
- **`update_comment`** — Edit an existing comment.
|
||||
- **`delete_comment`** — Delete a comment.
|
||||
- **`resolve_comment`** — Resolve (close) or reopen a comment thread (reversible). Only top-level
|
||||
comments can be resolved; the thread and its replies are kept, unlike `delete_comment`.
|
||||
- **`check_new_comments`** — Find comments created after a given ISO-8601 timestamp across
|
||||
- **`listComments`** — List a page's comments (content returned as Markdown).
|
||||
- **`updateComment`** — Edit an existing comment.
|
||||
- **`deleteComment`** — Delete a comment.
|
||||
- **`resolveComment`** — Resolve (close) or reopen a comment thread (reversible). Only top-level
|
||||
comments can be resolved; the thread and its replies are kept, unlike `deleteComment`.
|
||||
- **`checkNewComments`** — Find comments created after a given ISO-8601 timestamp across
|
||||
a space, optionally scoped to a page subtree — ideal for an agent that watches a doc for
|
||||
feedback.
|
||||
|
||||
### Versioning & history
|
||||
|
||||
- **`list_page_history`** — A page's saved versions (Docmost auto-snapshots on save),
|
||||
- **`listPageHistory`** — A page's saved versions (Docmost auto-snapshots on save),
|
||||
newest first, cursor-paginated. Each item's id is the `historyId`.
|
||||
- **`diff_page_versions`** — Diff two versions (or a version against the live page).
|
||||
- **`diffPageVersions`** — Diff two versions (or a version against the live page).
|
||||
Returns inserted/deleted text, integrity counts (images, links, tables, callouts,
|
||||
footnote markers), and a human-readable Markdown summary — computed with the same
|
||||
pipeline Docmost's own history viewer uses.
|
||||
- **`restore_page_version`** — Write a saved version back as the current content. Docmost
|
||||
- **`restorePageVersion`** — Write a saved version back as the current content. Docmost
|
||||
has no restore endpoint, so this creates a **new** snapshot — the restore is itself
|
||||
revertible.
|
||||
|
||||
### Sharing
|
||||
|
||||
- **`share_page`** — Make a page publicly accessible (idempotent) and return its public
|
||||
- **`sharePage`** — Make a page publicly accessible (idempotent) and return its public
|
||||
URL (`<app>/share/<key>/p/<slugId>`); optional search-engine indexing.
|
||||
- **`unshare_page`** — Revoke a page's public share.
|
||||
- **`list_shares`** — All public shares in the workspace, with titles and public URLs.
|
||||
- **`unsharePage`** — Revoke a page's public share.
|
||||
- **`listShares`** — All public shares in the workspace, with titles and public URLs.
|
||||
|
||||
---
|
||||
|
||||
@@ -255,27 +261,27 @@ All 41 tools, grouped by what you'd reach for them.
|
||||
This same guidance is also delivered at runtime via the MCP server `instructions` field,
|
||||
so capable clients steer the model automatically.
|
||||
|
||||
- **Text fixes** (wording, typos, numbers): `edit_page_text`.
|
||||
- **One block** (paragraph/heading/callout/table cell): `patch_node` / `insert_node` /
|
||||
`delete_node`, addressing the node by its `attrs.id` from `get_page_json`.
|
||||
- **Images**: `insert_image` / `replace_image`.
|
||||
- **A new page**: `create_page`.
|
||||
- **Bulk rewrite, or nodes without ids**: `update_page_json` (ProseMirror) or
|
||||
`update_page_markdown` (plain Markdown body replace).
|
||||
- **Text fixes** (wording, typos, numbers): `editPageText`.
|
||||
- **One block** (paragraph/heading/callout/table cell): `patchNode` / `insertNode` /
|
||||
`deleteNode`, addressing the node by its `attrs.id` from `getPageJson`.
|
||||
- **Images**: `insertImage` / `replaceImage`.
|
||||
- **A new page**: `createPage`.
|
||||
- **Bulk rewrite, or nodes without ids**: `updatePageJson` (ProseMirror) or
|
||||
`updatePageMarkdown` (plain Markdown body replace).
|
||||
- **Multi-step / scripted rewrite** (renumbering, footnotes, coordinated edits):
|
||||
`docmost_transform` — preview with `dryRun`, then apply.
|
||||
- **Copy a whole page's content from another page** (server-side): `copy_page_content`.
|
||||
- **Rename a page** (title only): `rename_page`.
|
||||
- **Reads**: `get_page` (Markdown) / `get_page_json` (lossless ProseMirror with ids).
|
||||
- **Review changes**: `list_page_history` → `diff_page_versions` → `restore_page_version`.
|
||||
- **Comments**: `create_comment` (with optional inline anchoring) / `list_comments` /
|
||||
`update_comment` / `resolve_comment` / `delete_comment` / `check_new_comments`.
|
||||
- **Navigate a page cheaply** (find a section/table, grab a block id): `get_outline` →
|
||||
`get_node`.
|
||||
- **Tables** (add/remove a row, set a cell): `table_get` / `table_insert_row` /
|
||||
`table_delete_row` / `table_update_cell`.
|
||||
- **Export a page as self-contained Markdown** (with comment anchors): `export_page_markdown`.
|
||||
- **Replace a page's body from Markdown**: `update_page_markdown`.
|
||||
`docmostTransform` — preview with `dryRun`, then apply.
|
||||
- **Copy a whole page's content from another page** (server-side): `copyPageContent`.
|
||||
- **Rename a page** (title only): `renamePage`.
|
||||
- **Reads**: `getPage` (Markdown) / `getPageJson` (lossless ProseMirror with ids).
|
||||
- **Review changes**: `listPageHistory` → `diffPageVersions` → `restorePageVersion`.
|
||||
- **Comments**: `createComment` (with optional inline anchoring) / `listComments` /
|
||||
`updateComment` / `resolveComment` / `deleteComment` / `checkNewComments`.
|
||||
- **Navigate a page cheaply** (find a section/table, grab a block id): `getOutline` →
|
||||
`getNode`.
|
||||
- **Tables** (add/remove a row, set a cell): `tableGet` / `tableInsertRow` /
|
||||
`tableDeleteRow` / `tableUpdateCell`.
|
||||
- **Export a page as self-contained Markdown** (with comment anchors): `exportPageMarkdown`.
|
||||
- **Replace a page's body from Markdown**: `updatePageMarkdown`.
|
||||
|
||||
---
|
||||
|
||||
@@ -287,25 +293,46 @@ so capable clients steer the model automatically.
|
||||
the debounced REST snapshot), then **reads → transforms → writes synchronously** in one
|
||||
tick so no remote update can interleave, and **waits for persistence acknowledgement**
|
||||
before returning.
|
||||
- **Per-page write serialization.** A per-`pageId` async mutex ensures two MCP writes to
|
||||
the same page never overlap; different pages never block each other.
|
||||
- **Per-page write serialization.** A per-`pageId` async mutex (keyed by the resolved
|
||||
page **UUID**, never a slugId) ensures two MCP writes to the same page never overlap;
|
||||
different pages never block each other. The lock helper fails fast if it is ever handed
|
||||
a non-UUID key, so a write path that forgot to resolve the id can never silently lock
|
||||
under a split key.
|
||||
|
||||
**Deploy requirement — single instance or sticky sessions.** This mutex is an
|
||||
in-process `Map`, and the cached collab sessions and the `stash_page` blob store are
|
||||
RAM-only and process-local. Behind a **multi-replica** load balancer **without sticky
|
||||
sessions**, two replicas can each "hold" the lock for the same page at once and per-page
|
||||
serialization is silently lost. Run the MCP/app as a **single instance**, or pin each
|
||||
page's traffic to one replica (sticky sessions / consistent hashing on the page id).
|
||||
There is deliberately no cross-process (e.g. Postgres advisory) lock yet — a conscious
|
||||
documented constraint. See the `Dockerfile` comment and the `MCP collaboration write
|
||||
path` block in `.env.example`.
|
||||
|
||||
**Rights-staleness window.** A cached collab session writes under the token captured at
|
||||
connect time (and the collab-token cache reuses a token for its TTL), so a **revoked**
|
||||
page access can lag by up to `MCP_COLLAB_SESSION_MAX_AGE_MS` (the hard session lifetime,
|
||||
default 10 min) before the next re-auth picks it up. Lower it to shorten the lag at the
|
||||
cost of more reconnects. This bounded window is an accepted trade-off; there is no
|
||||
push-based cache invalidation on a rights change.
|
||||
- **Transparent re-authentication.** Login uses email/password; expired tokens are
|
||||
refreshed automatically on the first 401/403 (covering JSON, multipart upload, and the
|
||||
collaboration-token path), with in-flight login de-duplication so a burst of calls
|
||||
triggers a single re-login.
|
||||
- **Lossless and lossy reads.** `get_page_json` returns the exact ProseMirror tree with
|
||||
block ids; `get_page` returns clean Markdown for convenience.
|
||||
- **Precise reads.** `getPageJson` returns the exact ProseMirror tree with block ids;
|
||||
`getPage` returns canonical Markdown that drops only a fixed, documented attr set.
|
||||
- **Full Docmost schema.** Markdown↔ProseMirror conversion supports callouts (including
|
||||
nested), task lists (bullet *and* numbered checklists), tables, math blocks, embeds,
|
||||
highlights, sub/superscript and more, with defensive caps against pathological input.
|
||||
- **Structured tables & lossless Markdown round-trip.** Tables can be edited as a matrix
|
||||
- **Structured tables & Markdown round-trip.** Tables can be edited as a matrix
|
||||
(read, insert/delete rows, set cells by `[row,col]`) without resending the document, and
|
||||
a page can be exported to and re-imported from a self-contained Docmost-flavoured
|
||||
Markdown file that preserves inline comment anchors and diagrams.
|
||||
Markdown file that preserves inline comment anchors and diagrams (block ids regenerate
|
||||
and a fixed no-Markdown-representation attr set is dropped — see `exportPageMarkdown`).
|
||||
- **Token-optimized responses.** API responses are filtered down to the fields agents
|
||||
actually need, and large collections (spaces, pages, comments, history) are paginated.
|
||||
- **Hardened runtime.** Global handlers keep a stray socket error from tearing down the
|
||||
stdio server; `move_page` requires a positively confirmed success; the diff engine
|
||||
stdio server; `movePage` requires a positively confirmed success; the diff engine
|
||||
falls back to a coarse block diff rather than hard-failing on a pathological document.
|
||||
|
||||
---
|
||||
@@ -363,7 +390,7 @@ npm run test:e2e
|
||||
|
||||
This project began as a fork of [MrMartiniMo/docmost-mcp](https://github.com/MrMartiniMo/docmost-mcp)
|
||||
(by Moritz Krause) and extends it substantially — adding per-block node editing,
|
||||
surgical text edits, the sandboxed `docmost_transform`, version history / diff / restore,
|
||||
surgical text edits, the sandboxed `docmostTransform`, version history / diff / restore,
|
||||
comments, image insert/replace, public sharing, server-side page copy, dual
|
||||
JSON/Markdown reads, transparent re-authentication and significant hardening. The comment
|
||||
tools were ported from upstream PR #3 by Max Nikitin. Thanks to both.
|
||||
|
||||
+118
-90
@@ -12,7 +12,7 @@
|
||||
> небольшую функцию, которая чинит текст*, чем перечитывать и заново выдавать весь
|
||||
> документ. Поэтому сервер построен вокруг того, как модели на самом деле удобно
|
||||
> редактировать: адресовать блок по id, сделать find/replace или передать трансформ
|
||||
> `(doc, ctx) => doc` и позволить модели *запрограммировать* правку. `docmost_transform` —
|
||||
> `(doc, ctx) => doc` и позволить модели *запрограммировать* правку. `docmostTransform` —
|
||||
> это и есть такой интерфейс. Другие Docmost-MCP «заточены под человека» — они дают
|
||||
> «открыть страницу» и «заменить страницу»; этот даёт примитивы редактирования, в которых
|
||||
> модель сильна.
|
||||
@@ -43,7 +43,7 @@ Docmost-MCP не сочетают:
|
||||
| **Нужна enterprise-лицензия** | **Нет** | **Да** | Нет | Нет | Нет |
|
||||
| Аутентификация | email + пароль, **авто-переавторизация** | API-ключ | email + пароль | cookie `authToken` (копировать из DevTools) | API Docmost / **напрямую PostgreSQL** |
|
||||
| Чтение страницы как Markdown | ✅ | ✅ | ✅ | ✅ | ✅ (только чтение) |
|
||||
| **Lossless Markdown round-trip** (экспорт/импорт, сохраняет якоря комментариев) | ✅ | — | — | — | — |
|
||||
| **Markdown round-trip** (экспорт/импорт, сохраняет якоря комментариев) | ✅ | — | — | — | — |
|
||||
| Чтение **lossless ProseMirror JSON** (с id блоков) | ✅ | — | — | — | — |
|
||||
| **Компактная структура страницы** (дешёвый поиск id блока) | ✅ | — | — | — | — |
|
||||
| **Получение одного блока** (по id или индексу) | ✅ | — | — | — | — |
|
||||
@@ -71,9 +71,9 @@ Docmost-MCP не сочетают:
|
||||
- **Экономия токенов при редактировании.** Большинство Docmost-MCP (и официальный)
|
||||
предлагают только запись «заменить всю страницу» — агент вынужден скачать весь документ,
|
||||
изменить и загрузить обратно, оплачивая весь документ **дважды** на каждой мелкой
|
||||
правке. Этот сервер позволяет агенту изменить ровно один блок (`patch_node` /
|
||||
`insert_node` / `delete_node`), сделать find/replace с сохранением структуры
|
||||
(`edit_page_text`) или скопировать страницу на стороне сервера (`copy_page_content`) —
|
||||
правке. Этот сервер позволяет агенту изменить ровно один блок (`patchNode` /
|
||||
`insertNode` / `deleteNode`), сделать find/replace с сохранением структуры
|
||||
(`editPageText`) или скопировать страницу на стороне сервера (`copyPageContent`) —
|
||||
**причём документ ни разу не проходит через модель**.
|
||||
|
||||
- **Записи, которые не воюют с редактором.** Наивная запись через REST конфликтует с тем,
|
||||
@@ -87,12 +87,12 @@ Docmost-MCP не сочетают:
|
||||
- **Агентоориентированная модель редактирования.** Серверы «под человека» дают «открыть
|
||||
страницу» и «заменить страницу», потому что это отражает то, как работает человек. Модель
|
||||
редактирует лучше, *программируя* правку — адресуя блоки по id, делая find/replace или
|
||||
передавая трансформ `(doc, ctx) => doc` (`docmost_transform`, с dry-run диффом перед
|
||||
передавая трансформ `(doc, ctx) => doc` (`docmostTransform`, с dry-run диффом перед
|
||||
коммитом). Этот сервер построен вокруг этого — поэтому у него есть примитивы
|
||||
редактирования, которых у остальных просто нет.
|
||||
|
||||
- **Страховка при редактировании, которой нет у других.** `list_page_history` →
|
||||
`diff_page_versions` → `restore_page_version` дают агенту (и вам) полный цикл «посмотреть
|
||||
- **Страховка при редактировании, которой нет у других.** `listPageHistory` →
|
||||
`diffPageVersions` → `restorePageVersion` дают агенту (и вам) полный цикл «посмотреть
|
||||
и откатить». Дифф использует *тот же* конвейер `recreateTransform → ChangeSet →
|
||||
simplifyChanges`, что и встроенный просмотр истории Docmost, так что результат совпадает
|
||||
с продуктом.
|
||||
@@ -113,59 +113,62 @@ Docmost-MCP не сочетают:
|
||||
|
||||
### Чтение и поиск
|
||||
|
||||
- **`get_workspace`** — Информация о текущем воркспейсе Docmost.
|
||||
- **`list_spaces`** — Все пространства воркспейса.
|
||||
- **`list_pages`** — Недавние страницы пространства, по убыванию `updatedAt` (по умолчанию
|
||||
- **`getWorkspace`** — Информация о текущем воркспейсе Docmost.
|
||||
- **`listSpaces`** — Все пространства воркспейса.
|
||||
- **`listPages`** — Недавние страницы пространства, по убыванию `updatedAt` (по умолчанию
|
||||
50, максимум 100). Для поиска в больших пространствах используйте `search`.
|
||||
- **`search`** — Полнотекстовый поиск по страницам и контенту (ограничен `limit`, максимум
|
||||
100).
|
||||
- **`get_page`** — Контент страницы как чистый **Markdown** (удобно, но это
|
||||
*lossy*-представление — id блоков и точная структура таблиц/коллаутов аппроксимируются).
|
||||
- **`get_page_json`** — **Lossless ProseMirror/TipTap JSON** страницы, включая `attrs.id`
|
||||
- **`getPage`** — Контент страницы как чистый **Markdown** (канонично для текста; теряет
|
||||
лишь id блоков, якоря разрешённых комментариев и фиксированный набор атрибутов без
|
||||
markdown-представления — спаны/colwidth/фон ячеек таблиц, отступы (indent),
|
||||
`callout.icon`, `orderedList.type` и `internal`/`target`/`rel`/`class` у ссылок;
|
||||
используйте `getPageJson`, когда они нужны).
|
||||
- **`getPageJson`** — **Lossless ProseMirror/TipTap JSON** страницы, включая `attrs.id`
|
||||
каждого блока и `slugId`, используемый в URL. Именно его потребляют инструменты
|
||||
поблочного редактирования.
|
||||
- **`get_outline`** — Компактная структура страницы из блоков верхнего уровня (`{index,
|
||||
- **`getOutline`** — Компактная структура страницы из блоков верхнего уровня (`{index,
|
||||
type, id, level, firstText}`; для таблиц добавляются число строк/столбцов и тексты ячеек
|
||||
заголовка, для списков — число пунктов) **без** тела документа. Дешёвый способ найти раздел или таблицу и получить
|
||||
id блока перед `get_node` / `patch_node` / `insert_node`.
|
||||
- **`get_node`** — Получить полное ProseMirror-поддерево одного блока (lossless), не
|
||||
вытягивая всю страницу. Адресуйте его по id блока (из `get_outline` / `get_page_json`)
|
||||
id блока перед `getNode` / `patchNode` / `insertNode`.
|
||||
- **`getNode`** — Получить полное ProseMirror-поддерево одного блока (lossless), не
|
||||
вытягивая всю страницу. Адресуйте его по id блока (из `getOutline` / `getPageJson`)
|
||||
или формой `#<index>` для блока верхнего уровня — используйте `#<index>` для
|
||||
таблиц/строк/ячеек, у которых нет id.
|
||||
|
||||
### Жизненный цикл страниц
|
||||
|
||||
- **`create_page`** — Создать страницу из Markdown и поместить в иерархию (опционально
|
||||
- **`createPage`** — Создать страницу из Markdown и поместить в иерархию (опционально
|
||||
`parentPageId`) одним вызовом. Использует import API Docmost для чистой конвертации
|
||||
Markdown→ProseMirror.
|
||||
- **`rename_page`** — Изменить только заголовок страницы, не трогая и не пересылая контент.
|
||||
- **`move_page`** — Сменить родителя страницы (вложить или вынести в корень); поддерживает
|
||||
- **`renamePage`** — Изменить только заголовок страницы, не трогая и не пересылая контент.
|
||||
- **`movePage`** — Сменить родителя страницы (вложить или вынести в корень); поддерживает
|
||||
позиционирование по fractional-index. Возвращает успех только при *положительно
|
||||
подтверждённом* результате.
|
||||
- **`delete_page`** — Удалить одну страницу.
|
||||
- **`copy_page_content`** — Заменить тело одной страницы копией тела другой, **полностью на
|
||||
- **`deletePage`** — Удалить одну страницу.
|
||||
- **`copyPageContent`** — Заменить тело одной страницы копией тела другой, **полностью на
|
||||
стороне сервера** — документ не проходит через модель. У целевой страницы сохраняются
|
||||
собственные заголовок и slug (URL не меняется).
|
||||
|
||||
### Редактирование
|
||||
|
||||
- **`edit_page_text`** — Хирургический find/replace внутри текста страницы. Сохраняет
|
||||
- **`editPageText`** — Хирургический find/replace внутри текста страницы. Сохраняет
|
||||
**всю** структуру: id блоков, marks, ссылки, коллауты, таблицы. Предпочтительный
|
||||
инструмент для правки формулировок, опечаток, чисел и имён.
|
||||
- **`patch_node`** — Заменить один блок, адресованный по `attrs.id` (из `get_page_json`),
|
||||
- **`patchNode`** — Заменить один блок, адресованный по `attrs.id` (из `getPageJson`),
|
||||
без пересылки документа.
|
||||
- **`insert_node`** — Вставить блок до/после другого (по `attrs.id` или по якорному тексту)
|
||||
- **`insertNode`** — Вставить блок до/после другого (по `attrs.id` или по якорному тексту)
|
||||
либо добавить в конец.
|
||||
- **`delete_node`** — Удалить один блок по его `attrs.id`.
|
||||
- **`update_page_json`** — Заменить весь контент страницы документом ProseMirror (массовые
|
||||
- **`deleteNode`** — Удалить один блок по его `attrs.id`.
|
||||
- **`updatePageJson`** — Заменить весь контент страницы документом ProseMirror (массовые
|
||||
перезаписи или когда у узлов нет id). `content` опционален — опустите его, чтобы изменить
|
||||
только заголовок. Сохраняет переданные id блоков, поэтому якоря заголовков и история
|
||||
остаются стабильными.
|
||||
- **`update_page_markdown`** — Заменить тело страницы (и опционально заголовок) новым
|
||||
- **`updatePageMarkdown`** — Заменить тело страницы (и опционально заголовок) новым
|
||||
**обычным Markdown**. Всё тело переимпортируется (id блоков перегенерируются — для
|
||||
хирургических правок или сохранения id используйте `edit_page_text` / `patch_node` /
|
||||
`update_page_json`). Markdown в диалекте Docmost разбирается, включая inline-сноски `^[...]`.
|
||||
- **`docmost_transform`** — Агентоориентированный интерфейс редактирования: вместо
|
||||
хирургических правок или сохранения id используйте `editPageText` / `patchNode` /
|
||||
`updatePageJson`). Markdown в диалекте Docmost разбирается, включая inline-сноски `^[...]`.
|
||||
- **`docmostTransform`** — Агентоориентированный интерфейс редактирования: вместо
|
||||
перепечатывания документа агент **пишет функцию, которая его чинит**. Редактирует
|
||||
страницу, запуская произвольный **JS-трансформ `(doc, ctx) => doc`** на её *живом*
|
||||
документе ProseMirror. Работает в **песочнице** (без `require`/`process`/`fs`/сети,
|
||||
@@ -177,42 +180,46 @@ Docmost-MCP не сочетают:
|
||||
|
||||
### Таблицы
|
||||
|
||||
- **`table_get`** — Прочитать таблицу как матрицу: `{rows, cols, cells (text[][]),
|
||||
- **`tableGet`** — Прочитать таблицу как матрицу: `{rows, cols, cells (text[][]),
|
||||
cellIds}` (id абзаца на ячейку или `null`). Адресуйте таблицу через `#<index>` (из
|
||||
`get_outline`) или любой id блока внутри неё. Используйте `cellIds` вместе с `patch_node`
|
||||
`getOutline`) или любой id блока внутри неё. Используйте `cellIds` вместе с `patchNode`
|
||||
для правок ячеек с форматированием.
|
||||
- **`table_insert_row`** — Вставить строку из текстовых ячеек, дополненную до числа
|
||||
- **`tableInsertRow`** — Вставить строку из текстовых ячеек, дополненную до числа
|
||||
столбцов таблицы (передать ячеек больше числа столбцов — ошибка). `index` — 0-based
|
||||
позиция вставки (0 вставляет перед заголовком); опустите, чтобы добавить в конец.
|
||||
- **`table_delete_row`** — Удалить строку по 0-based `index`. Отказывается удалять
|
||||
- **`tableDeleteRow`** — Удалить строку по 0-based `index`. Отказывается удалять
|
||||
единственную строку таблицы; удаление строки 0 делает заголовком следующую строку.
|
||||
- **`table_update_cell`** — Задать текстовое содержимое ячейки `[row, col]` (0-based). Для
|
||||
форматирования используйте `patch_node` по id абзаца ячейки из `table_get`.
|
||||
- **`tableUpdateCell`** — Задать текстовое содержимое ячейки `[row, col]` (0-based). Для
|
||||
форматирования используйте `patchNode` по id абзаца ячейки из `tableGet`.
|
||||
|
||||
### Markdown: экспорт и импорт
|
||||
|
||||
- **`export_page_markdown`** — Экспортировать страницу в один самодостаточный, **lossless
|
||||
Markdown в диалекте Docmost**: мета-заголовок, тело с inline-якорями комментариев и
|
||||
диаграммами и завершающий блок тредов комментариев. Чтобы заменить тело страницы из
|
||||
обычного авторского Markdown, используйте `update_page_markdown`.
|
||||
- **`exportPageMarkdown`** — Экспортировать страницу в один самодостаточный
|
||||
**Markdown в диалекте Docmost**: мета-заголовок, тело с inline-якорями комментариев и
|
||||
диаграммами и завершающий блок тредов комментариев. Round-trip скачать → отредактировать →
|
||||
импортировать перегенерирует id блоков и **молча отбрасывает** набор атрибутов без
|
||||
markdown-представления (спаны/colwidth/фон ячеек таблиц, отступы (indent), `callout.icon`,
|
||||
`orderedList.type`, `internal`/`target`/`rel`/`class` у ссылок); держите их в ProseMirror
|
||||
JSON, если они должны выжить. Чтобы заменить тело страницы из обычного авторского Markdown,
|
||||
используйте `updatePageMarkdown`.
|
||||
|
||||
> **Удалено в этом релизе:** `import_page_markdown` (парсер round-trip для
|
||||
> **Удалено в этом релизе:** `importPageMarkdown` (парсер round-trip для
|
||||
> экспортированного Docmost-Markdown-файла) **больше не отдаётся на внешней MCP-поверхности**.
|
||||
> Чтобы заменить тело страницы из Markdown, используйте **`update_page_markdown`** (замена
|
||||
> Чтобы заменить тело страницы из Markdown, используйте **`updatePageMarkdown`** (замена
|
||||
> тела обычным Markdown). См. заметку о миграции в CHANGELOG.
|
||||
|
||||
### Изображения
|
||||
|
||||
- **`insert_image`** — Загрузить локальное изображение и вставить за один шаг: добавить в
|
||||
- **`insertImage`** — Загрузить локальное изображение и вставить за один шаг: добавить в
|
||||
конец, поставить вместо текстового плейсхолдера (`replaceText`) или после заданного блока
|
||||
(`afterText`). Сохраняет id всех остальных блоков.
|
||||
- **`replace_image`** — Заменить существующее изображение. Загружает новый файл как **новое
|
||||
- **`replaceImage`** — Заменить существующее изображение. Загружает новый файл как **новое
|
||||
вложение** (чистый URL, который рендерится и сбрасывает кэш браузера), затем
|
||||
перенаправляет все узлы, ссылавшиеся на старое вложение (рекурсивно, включая
|
||||
коллауты/таблицы), через живой документ, сохраняя комментарии, выравнивание и alt-текст.
|
||||
(Перезапись «по месту» намеренно не используется — некоторые версии Docmost портят
|
||||
вложение при перезаписи.)
|
||||
- **`stash_page`** — Сериализовать страницу целиком (её полный ProseMirror JSON) в
|
||||
- **`stashPage`** — Сериализовать страницу целиком (её полный ProseMirror JSON) в
|
||||
эфемерный blob в оперативной памяти и вернуть ТОЛЬКО короткий анонимный URL — тело
|
||||
никогда не попадает в контекст модели, поэтому это способ передать большую страницу
|
||||
(вместе с её изображениями) внешнему потребителю без усечения. Каждое внутреннее
|
||||
@@ -224,35 +231,35 @@ Docmost-MCP не сочетают:
|
||||
|
||||
### Комментарии
|
||||
|
||||
- **`create_comment`** — Добавить комментарий к странице, опционально **привязав inline** к
|
||||
- **`createComment`** — Добавить комментарий к странице, опционально **привязав inline** к
|
||||
точному фрагменту текста (первое вхождение оборачивается comment-маркой).
|
||||
- **`list_comments`** — Список комментариев страницы (контент возвращается как Markdown).
|
||||
- **`update_comment`** — Изменить существующий комментарий.
|
||||
- **`delete_comment`** — Удалить комментарий.
|
||||
- **`resolve_comment`** — Закрыть (resolve) или переоткрыть тред комментария (обратимо). Resolve
|
||||
доступен только для корневых комментариев; тред и ответы сохраняются, в отличие от `delete_comment`.
|
||||
- **`check_new_comments`** — Найти комментарии, созданные после заданной метки времени
|
||||
- **`listComments`** — Список комментариев страницы (контент возвращается как Markdown).
|
||||
- **`updateComment`** — Изменить существующий комментарий.
|
||||
- **`deleteComment`** — Удалить комментарий.
|
||||
- **`resolveComment`** — Закрыть (resolve) или переоткрыть тред комментария (обратимо). Resolve
|
||||
доступен только для корневых комментариев; тред и ответы сохраняются, в отличие от `deleteComment`.
|
||||
- **`checkNewComments`** — Найти комментарии, созданные после заданной метки времени
|
||||
ISO-8601, по пространству, опционально в рамках поддерева страниц — идеально для агента,
|
||||
который следит за обратной связью в документе.
|
||||
|
||||
### Версии и история
|
||||
|
||||
- **`list_page_history`** — Сохранённые версии страницы (Docmost авто-снапшотит при каждом
|
||||
- **`listPageHistory`** — Сохранённые версии страницы (Docmost авто-снапшотит при каждом
|
||||
сохранении), новые сверху, курсорная пагинация. id каждого элемента — это `historyId`.
|
||||
- **`diff_page_versions`** — Дифф двух версий (или версии против живой страницы).
|
||||
- **`diffPageVersions`** — Дифф двух версий (или версии против живой страницы).
|
||||
Возвращает вставленный/удалённый текст, счётчики целостности (изображения, ссылки,
|
||||
таблицы, коллауты, маркеры сносок) и человекочитаемую Markdown-сводку — посчитано тем же
|
||||
конвейером, что использует встроенный просмотр истории Docmost.
|
||||
- **`restore_page_version`** — Записать сохранённую версию обратно как текущий контент. У
|
||||
- **`restorePageVersion`** — Записать сохранённую версию обратно как текущий контент. У
|
||||
Docmost нет эндпоинта восстановления, поэтому создаётся **новый** снапшот — само
|
||||
восстановление тоже обратимо.
|
||||
|
||||
### Публикация
|
||||
|
||||
- **`share_page`** — Сделать страницу публично доступной (идемпотентно) и вернуть её
|
||||
- **`sharePage`** — Сделать страницу публично доступной (идемпотентно) и вернуть её
|
||||
публичный URL (`<app>/share/<key>/p/<slugId>`); опционально индексирование поисковиками.
|
||||
- **`unshare_page`** — Отозвать публичный доступ к странице.
|
||||
- **`list_shares`** — Все публичные ссылки воркспейса с заголовками и публичными URL.
|
||||
- **`unsharePage`** — Отозвать публичный доступ к странице.
|
||||
- **`listShares`** — Все публичные ссылки воркспейса с заголовками и публичными URL.
|
||||
|
||||
---
|
||||
|
||||
@@ -261,29 +268,29 @@ Docmost-MCP не сочетают:
|
||||
Та же подсказка отдаётся в рантайме через поле `instructions` MCP-сервера, так что
|
||||
подходящие клиенты направляют модель автоматически.
|
||||
|
||||
- **Правки текста** (формулировки, опечатки, числа): `edit_page_text`.
|
||||
- **Один блок** (абзац/заголовок/коллаут/ячейка таблицы): `patch_node` / `insert_node` /
|
||||
`delete_node`, адресуя узел по его `attrs.id` из `get_page_json`.
|
||||
- **Изображения**: `insert_image` / `replace_image`.
|
||||
- **Новая страница**: `create_page`.
|
||||
- **Массовая перезапись или узлы без id**: `update_page_json` (ProseMirror) или
|
||||
`update_page_markdown` (замена тела обычным Markdown).
|
||||
- **Правки текста** (формулировки, опечатки, числа): `editPageText`.
|
||||
- **Один блок** (абзац/заголовок/коллаут/ячейка таблицы): `patchNode` / `insertNode` /
|
||||
`deleteNode`, адресуя узел по его `attrs.id` из `getPageJson`.
|
||||
- **Изображения**: `insertImage` / `replaceImage`.
|
||||
- **Новая страница**: `createPage`.
|
||||
- **Массовая перезапись или узлы без id**: `updatePageJson` (ProseMirror) или
|
||||
`updatePageMarkdown` (замена тела обычным Markdown).
|
||||
- **Многошаговая / скриптовая перезапись** (перенумерация, сноски, согласованные правки):
|
||||
`docmost_transform` — предпросмотр через `dryRun`, затем применение.
|
||||
`docmostTransform` — предпросмотр через `dryRun`, затем применение.
|
||||
- **Скопировать контент целой страницы из другой** (на стороне сервера):
|
||||
`copy_page_content`.
|
||||
- **Переименовать страницу** (только заголовок): `rename_page`.
|
||||
- **Чтение**: `get_page` (Markdown) / `get_page_json` (lossless ProseMirror с id).
|
||||
- **Просмотр изменений**: `list_page_history` → `diff_page_versions` →
|
||||
`restore_page_version`.
|
||||
- **Комментарии**: `create_comment` (с опциональной inline-привязкой) / `list_comments` /
|
||||
`update_comment` / `resolve_comment` / `delete_comment` / `check_new_comments`.
|
||||
- **Дешёвая навигация по странице** (найти раздел/таблицу, получить id блока): `get_outline`
|
||||
→ `get_node`.
|
||||
- **Таблицы** (добавить/удалить строку, задать ячейку): `table_get` / `table_insert_row` /
|
||||
`table_delete_row` / `table_update_cell`.
|
||||
- **Экспорт страницы в самодостаточный Markdown** (с якорями комментариев): `export_page_markdown`.
|
||||
- **Заменить тело страницы из Markdown**: `update_page_markdown`.
|
||||
`copyPageContent`.
|
||||
- **Переименовать страницу** (только заголовок): `renamePage`.
|
||||
- **Чтение**: `getPage` (Markdown) / `getPageJson` (lossless ProseMirror с id).
|
||||
- **Просмотр изменений**: `listPageHistory` → `diffPageVersions` →
|
||||
`restorePageVersion`.
|
||||
- **Комментарии**: `createComment` (с опциональной inline-привязкой) / `listComments` /
|
||||
`updateComment` / `resolveComment` / `deleteComment` / `checkNewComments`.
|
||||
- **Дешёвая навигация по странице** (найти раздел/таблицу, получить id блока): `getOutline`
|
||||
→ `getNode`.
|
||||
- **Таблицы** (добавить/удалить строку, задать ячейку): `tableGet` / `tableInsertRow` /
|
||||
`tableDeleteRow` / `tableUpdateCell`.
|
||||
- **Экспорт страницы в самодостаточный Markdown** (с якорями комментариев): `exportPageMarkdown`.
|
||||
- **Заменить тело страницы из Markdown**: `updatePageMarkdown`.
|
||||
|
||||
---
|
||||
|
||||
@@ -295,28 +302,49 @@ Docmost-MCP не сочетают:
|
||||
правки, которых ещё нет в дебаунс-снапшоте REST), затем **читает → трансформирует →
|
||||
пишет синхронно** в одном тике, чтобы никакое удалённое обновление не вклинилось, и
|
||||
**ждёт подтверждения сохранения** до возврата.
|
||||
- **Сериализация записи по странице.** Асинхронный мьютекс по `pageId` гарантирует, что
|
||||
две записи MCP в одну страницу никогда не пересекаются; разные страницы друг друга не
|
||||
блокируют.
|
||||
- **Сериализация записи по странице.** Асинхронный мьютекс по разрешённому **UUID**
|
||||
страницы (никогда не по slugId) гарантирует, что две записи MCP в одну страницу никогда
|
||||
не пересекаются; разные страницы друг друга не блокируют. Хелпер блокировки падает сразу
|
||||
(fail-fast), если ему передали не-UUID ключ, — путь записи, забывший разрезолвить id, не
|
||||
сможет молча взять лок под расщеплённым ключом.
|
||||
|
||||
**Требование к деплою — один инстанс или sticky-сессии.** Этот мьютекс — процесс-локальный
|
||||
`Map`, а кэш collab-сессий и хранилище `stashPage` живут только в RAM одного процесса. За
|
||||
**мультиреплика**-балансировщиком **без sticky-сессий** две реплики могут одновременно
|
||||
«держать» лок одной страницы, и сериализация по странице молча теряется. Запускайте
|
||||
MCP/приложение **одним инстансом** либо прибивайте трафик страницы к одной реплике
|
||||
(sticky-сессии / consistent hashing по id страницы). Кросс-процессной блокировки (например,
|
||||
Postgres advisory-lock) намеренно пока нет — осознанное задокументированное ограничение.
|
||||
См. комментарий в `Dockerfile` и блок `MCP collaboration write path` в `.env.example`.
|
||||
|
||||
**Окно устаревших прав.** Кэшированная collab-сессия пишет под токеном, захваченным в
|
||||
момент connect (а кэш collab-токена переиспользует токен в пределах своего TTL), поэтому
|
||||
**отозванный** доступ к странице может лагать до `MCP_COLLAB_SESSION_MAX_AGE_MS` (жёсткий
|
||||
срок жизни сессии, по умолчанию 10 мин), пока следующая переавторизация его не подхватит.
|
||||
Уменьшите значение, чтобы сократить лаг ценой большего числа переподключений. Это
|
||||
ограниченное окно — принятый trade-off; push-инвалидации кэша при смене прав нет.
|
||||
- **Прозрачная переавторизация.** Логин по email/паролю; истёкшие токены обновляются
|
||||
автоматически на первом 401/403 (покрывая JSON, multipart-загрузку и путь токена
|
||||
коллаборации), с дедупликацией параллельных логинов, так что пачка вызовов вызывает один
|
||||
повторный логин.
|
||||
- **Lossless- и lossy-чтение.** `get_page_json` возвращает точное дерево ProseMirror с id
|
||||
блоков; `get_page` возвращает чистый Markdown для удобства.
|
||||
- **Точные чтения.** `getPageJson` возвращает точное дерево ProseMirror с id блоков;
|
||||
`getPage` возвращает канонический Markdown, теряющий лишь фиксированный, документированный
|
||||
набор атрибутов.
|
||||
- **Полная схема Docmost.** Конвертация Markdown↔ProseMirror поддерживает коллауты
|
||||
(включая вложенные), списки задач (маркированные *и* нумерованные чек-листы), таблицы,
|
||||
блоки формул, эмбеды, выделение, под/надстрочный текст и прочее, с защитными лимитами
|
||||
против патологического ввода.
|
||||
- **Структурные таблицы и lossless Markdown round-trip.** Таблицы можно редактировать как
|
||||
- **Структурные таблицы и Markdown round-trip.** Таблицы можно редактировать как
|
||||
матрицу (чтение, вставка/удаление строк, задание ячеек по `[row, col]`) без пересылки
|
||||
документа, а страницу — экспортировать и заново импортировать как самодостаточный
|
||||
Markdown-файл в диалекте Docmost, сохраняющий inline-якоря комментариев и диаграммы.
|
||||
Markdown-файл в диалекте Docmost, сохраняющий inline-якоря комментариев и диаграммы
|
||||
(id блоков перегенерируются, а фиксированный набор атрибутов без markdown-представления
|
||||
отбрасывается — см. `exportPageMarkdown`).
|
||||
- **Ответы, оптимизированные по токенам.** Ответы API урезаются до полей, действительно
|
||||
нужных агентам, а большие коллекции (пространства, страницы, комментарии, история)
|
||||
пагинируются.
|
||||
- **Закалённый рантайм.** Глобальные обработчики не дают случайной ошибке сокета уронить
|
||||
stdio-сервер; `move_page` требует положительно подтверждённого успеха; движок диффа
|
||||
stdio-сервер; `movePage` требует положительно подтверждённого успеха; движок диффа
|
||||
откатывается к грубому поблочному диффу, а не падает на патологическом документе.
|
||||
|
||||
---
|
||||
@@ -376,7 +404,7 @@ npm run test:e2e
|
||||
Проект начинался как форк
|
||||
[MrMartiniMo/docmost-mcp](https://github.com/MrMartiniMo/docmost-mcp) (автор Moritz Krause)
|
||||
и существенно его расширяет — добавлены поблочное редактирование узлов, хирургические
|
||||
правки текста, песочница `docmost_transform`, история версий / дифф / восстановление,
|
||||
правки текста, песочница `docmostTransform`, история версий / дифф / восстановление,
|
||||
комментарии, вставка/замена изображений, публичные ссылки, серверное копирование страниц,
|
||||
двойное чтение JSON/Markdown, прозрачная переавторизация и значительное упрочнение.
|
||||
Инструменты комментариев портированы из upstream PR #3 от Max Nikitin. Спасибо обоим.
|
||||
|
||||
+10
-10
@@ -22,14 +22,14 @@ are debounced server-side, so the script waits ~16 s before reading back via RES
|
||||
|
||||
| # | Tool / path | What is checked | Expected |
|
||||
|---|-------------|-----------------|----------|
|
||||
| 1 | `create_page` | title with spaces, slugId returned | page created, title intact |
|
||||
| 1 | `createPage` | title with spaces, slugId returned | page created, title intact |
|
||||
| 2 | `update_page` (markdown) | headings, **bold**/*italic*/~~strike~~/`code`/link, nested bullet + ordered lists, blockquote, code block, `:::callout:::`, table | all structures survive re-import |
|
||||
| 3 | `get_page_json` | lossless ProseMirror, block ids, callout/table nodes | present (note: reads the **debounced** REST snapshot — recent collab writes may lag a few seconds) |
|
||||
| 4 | `edit_page_text` | surgical replace; block ids + marks preserved; ambiguous match rejected; missing match reported | edits applied, ids stable, errors correct |
|
||||
| 5 | `update_page_json` | full lossless write; custom block ids preserved; existing content (text edits, images, callout, table) not lost | round-trips intact |
|
||||
| 3 | `getPageJson` | lossless ProseMirror, block ids, callout/table nodes | present (note: reads the **debounced** REST snapshot — recent collab writes may lag a few seconds) |
|
||||
| 4 | `editPageText` | surgical replace; block ids + marks preserved; ambiguous match rejected; missing match reported | edits applied, ids stable, errors correct |
|
||||
| 5 | `updatePageJson` | full lossless write; custom block ids preserved; existing content (text edits, images, callout, table) not lost | round-trips intact |
|
||||
| 6 | `upload_image` | uploads attachment, returns node | src is a **clean** `/api/files/<id>/<file>` URL, served `200 image/*` |
|
||||
| 7 | `insert_image` (append / `replaceText` / `afterText`) | three placements | image lands in the right place, all other block ids preserved |
|
||||
| 8 | **`replace_image`** | swap an existing figure for new bytes; comments/align/alt preserved; **the new URL must actually serve the image** | new image renders (`200`), old node repointed |
|
||||
| 7 | `insertImage` (append / `replaceText` / `afterText`) | three placements | image lands in the right place, all other block ids preserved |
|
||||
| 8 | **`replaceImage`** | swap an existing figure for new bytes; comments/align/alt preserved; **the new URL must actually serve the image** | new image renders (`200`), old node repointed |
|
||||
|
||||
## Image-specific assertions (the recurring bug area)
|
||||
|
||||
@@ -39,7 +39,7 @@ For every uploaded/inserted/replaced image, assert at the HTTP level that the
|
||||
* `GET <src>` → `200`, `Content-Type: image/*`, body starts with the image magic
|
||||
(`89 50 4E 47` for PNG, etc.).
|
||||
* `src` does **not** contain a `?v=` query (see "Known pitfalls").
|
||||
* After `replace_image`: the returned `newAttachmentId` **differs** from the old
|
||||
* After `replaceImage`: the returned `newAttachmentId` **differs** from the old
|
||||
one (replacement uses a fresh attachment → fresh URL), and `GET <new src>` → `200`.
|
||||
* The old image node on the page is repointed to the new attachmentId.
|
||||
|
||||
@@ -64,7 +64,7 @@ broken/empty figure.
|
||||
Uploading with an existing `attachmentId` (`POST /files/upload` + `attachmentId`)
|
||||
overwrites the bytes in place. On this Docmost the attachment then returns
|
||||
**500 for every URL** (clean, `?v=`, any filename) → broken image. Therefore
|
||||
`replace_image` must upload a **new** attachment and repoint the nodes; the new
|
||||
`replaceImage` must upload a **new** attachment and repoint the nodes; the new
|
||||
id yields a new URL that both renders and busts the browser cache. The old
|
||||
attachment is left as an unreferenced orphan: Docmost exposes **no HTTP API to
|
||||
delete a single content attachment** (verified against the attachment
|
||||
@@ -80,9 +80,9 @@ broken/empty figure.
|
||||
from `?v=`. Image `src` is kept clean (`/api/files/<id>/<file>`); cache-busting
|
||||
on replace is achieved by the new attachment id.
|
||||
|
||||
3. **REST snapshot lag.** `get_page_json` reads the debounced DB snapshot, so a
|
||||
3. **REST snapshot lag.** `getPageJson` reads the debounced DB snapshot, so a
|
||||
write made moments earlier may not be visible yet. Wait (~16 s) before reading
|
||||
back, and never feed a possibly-stale snapshot straight into `update_page_json`.
|
||||
back, and never feed a possibly-stale snapshot straight into `updatePageJson`.
|
||||
|
||||
4. **Callout type narrowing (minor, open).** A `:::warning` callout is imported as
|
||||
`type: "info"` — the markdown→callout conversion does not carry non-`info`
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
{
|
||||
"$comment": "Semantic palettes for drawioFromGraph (issue #425). DATA, not code: node `kind` -> fill/stroke slot, edge `kind` -> line-style props, per preset. The `default` node palette is the issue's base table; `dark` keeps the same hues on a dark canvas with lighter strokes/font; `colorblind-safe` maps every slot onto the Okabe-Ito qualitative palette (8 colours proven distinguishable for all common colour-vision deficiencies) so no two adjacent kinds collide. `fontColor`/`fillColor`/`strokeColor` are exact draw.io values. `edgeDefault` is the fallback line style; `group` is the (always-transparent) container stroke per preset.",
|
||||
"presets": {
|
||||
"default": {
|
||||
"canvasDark": false,
|
||||
"nodes": {
|
||||
"service": { "fillColor": "#dae8fc", "strokeColor": "#6c8ebf", "fontColor": "#000000" },
|
||||
"db": { "fillColor": "#d5e8d4", "strokeColor": "#82b366", "fontColor": "#000000" },
|
||||
"queue": { "fillColor": "#fff2cc", "strokeColor": "#d6b656", "fontColor": "#000000" },
|
||||
"gateway": { "fillColor": "#ffe6cc", "strokeColor": "#d79b00", "fontColor": "#000000" },
|
||||
"error": { "fillColor": "#f8cecc", "strokeColor": "#b85450", "fontColor": "#000000" },
|
||||
"external": { "fillColor": "#f5f5f5", "strokeColor": "#666666", "fontColor": "#333333" },
|
||||
"security": { "fillColor": "#e1d5e7", "strokeColor": "#9673a6", "fontColor": "#000000" }
|
||||
},
|
||||
"edges": {
|
||||
"sync": { "props": "" },
|
||||
"async": { "props": "dashed=1;" },
|
||||
"error": { "props": "dashed=1;strokeColor=#DD344C;" }
|
||||
},
|
||||
"edgeDefault": { "strokeColor": "#333333", "fontColor": "#333333" },
|
||||
"group": { "strokeColor": "#666666", "fontColor": "#333333" }
|
||||
},
|
||||
"dark": {
|
||||
"canvasDark": true,
|
||||
"nodes": {
|
||||
"service": { "fillColor": "#1a2a44", "strokeColor": "#7ea6e0", "fontColor": "#dae8fc" },
|
||||
"db": { "fillColor": "#1f331e", "strokeColor": "#97d077", "fontColor": "#d5e8d4" },
|
||||
"queue": { "fillColor": "#3a3218", "strokeColor": "#e5c15a", "fontColor": "#fff2cc" },
|
||||
"gateway": { "fillColor": "#3a2812", "strokeColor": "#ffb570", "fontColor": "#ffe6cc" },
|
||||
"error": { "fillColor": "#3a1c1b", "strokeColor": "#e08e8b", "fontColor": "#f8cecc" },
|
||||
"external": { "fillColor": "#2b2b2b", "strokeColor": "#999999", "fontColor": "#e0e0e0" },
|
||||
"security": { "fillColor": "#2c2338", "strokeColor": "#b39ddb", "fontColor": "#e1d5e7" }
|
||||
},
|
||||
"edges": {
|
||||
"sync": { "props": "" },
|
||||
"async": { "props": "dashed=1;" },
|
||||
"error": { "props": "dashed=1;strokeColor=#ff6b6b;" }
|
||||
},
|
||||
"edgeDefault": { "strokeColor": "#cccccc", "fontColor": "#e0e0e0" },
|
||||
"group": { "strokeColor": "#aaaaaa", "fontColor": "#e0e0e0" }
|
||||
},
|
||||
"colorblind-safe": {
|
||||
"canvasDark": false,
|
||||
"okabeIto": ["#000000", "#E69F00", "#56B4E9", "#009E73", "#F0E442", "#0072B2", "#D55E00", "#CC79A7"],
|
||||
"nodes": {
|
||||
"service": { "fillColor": "#D6E9F5", "strokeColor": "#0072B2", "fontColor": "#000000" },
|
||||
"db": { "fillColor": "#D6EFE4", "strokeColor": "#009E73", "fontColor": "#000000" },
|
||||
"queue": { "fillColor": "#FCF8CC", "strokeColor": "#F0E442", "fontColor": "#000000" },
|
||||
"gateway": { "fillColor": "#FBEBD0", "strokeColor": "#E69F00", "fontColor": "#000000" },
|
||||
"error": { "fillColor": "#F7DDCC", "strokeColor": "#D55E00", "fontColor": "#000000" },
|
||||
"external": { "fillColor": "#EDEDED", "strokeColor": "#000000", "fontColor": "#000000" },
|
||||
"security": { "fillColor": "#F3DEEB", "strokeColor": "#CC79A7", "fontColor": "#000000" }
|
||||
},
|
||||
"edges": {
|
||||
"sync": { "props": "" },
|
||||
"async": { "props": "dashed=1;" },
|
||||
"error": { "props": "dashed=1;strokeColor=#D55E00;" }
|
||||
},
|
||||
"edgeDefault": { "strokeColor": "#000000", "fontColor": "#000000" },
|
||||
"group": { "strokeColor": "#000000", "fontColor": "#000000" }
|
||||
}
|
||||
}
|
||||
}
|
||||
+89
-4562
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,704 @@
|
||||
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
|
||||
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
|
||||
// changed to a mixin factory. See client/context.ts for the shared base.
|
||||
import type { GConstructor, DocmostClientContext } from "./context.js";
|
||||
import { assertFullUuid } from "./errors.js";
|
||||
import {
|
||||
filterWorkspace,
|
||||
filterSpace,
|
||||
filterPage,
|
||||
filterComment,
|
||||
filterSearchResult,
|
||||
} from "../lib/filters.js";
|
||||
import { convertProseMirrorToMarkdown } from "../lib/markdown-converter.js";
|
||||
import {
|
||||
updatePageContentRealtime,
|
||||
replacePageContent,
|
||||
markdownToProseMirror,
|
||||
markdownToProseMirrorCanonical,
|
||||
mutatePageContent,
|
||||
assertYjsEncodable,
|
||||
MutationResult,
|
||||
} from "../lib/collaboration.js";
|
||||
import {
|
||||
replaceNodeById,
|
||||
replaceNodeByIdWithMany,
|
||||
reassignCollidingBlockIds,
|
||||
deleteNodeById,
|
||||
assertUnambiguousMatch,
|
||||
insertNodeRelative,
|
||||
insertNodesRelative,
|
||||
blockPlainText,
|
||||
buildOutline,
|
||||
getNodeByRef,
|
||||
readTable,
|
||||
insertTableRow,
|
||||
deleteTableRow,
|
||||
updateTableCell,
|
||||
findInvalidNode,
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
import {
|
||||
applyAnchorInDoc,
|
||||
countAnchorMatches,
|
||||
getAnchoredText,
|
||||
resolveAnchorSelection,
|
||||
normalizeForMatch,
|
||||
} from "../lib/comment-anchor.js";
|
||||
import { closestBlockHint } from "../lib/text-normalize.js";
|
||||
import {
|
||||
blockText,
|
||||
walk,
|
||||
getList,
|
||||
insertMarkerAfter,
|
||||
setCalloutRange,
|
||||
noteItem,
|
||||
mdToInlineNodes,
|
||||
commentsToFootnotes,
|
||||
canonicalizeFootnotes,
|
||||
insertInlineFootnote,
|
||||
mergeFootnoteDefinitions,
|
||||
} from "../lib/transforms.js";
|
||||
|
||||
// Public method surface of CommentsMixin (issue #450) — a NAMED type so the factory
|
||||
// return type is expressible in the emitted .d.ts (the anonymous mixin class
|
||||
// carries the base's protected shared state, which would otherwise trip TS4094).
|
||||
// Derived from the class below; `implements ICommentsMixin` fails to compile on drift.
|
||||
export interface ICommentsMixin {
|
||||
listComments(pageId: string, includeResolved?: boolean): any;
|
||||
getComment(commentId: string): any;
|
||||
createComment(pageId: string, content: string, type?: "page" | "inline", selection?: string, parentCommentId?: string, suggestedText?: string): any;
|
||||
updateComment(commentId: string, content: string): any;
|
||||
deleteComment(commentId: string): any;
|
||||
resolveComment(commentId: string, resolved: boolean): any;
|
||||
checkNewComments(spaceId: string, since: string, parentPageId?: string): any;
|
||||
}
|
||||
|
||||
export function CommentsMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & ICommentsMixin> & TBase {
|
||||
abstract class CommentsMixin extends Base implements ICommentsMixin {
|
||||
// --- Comment methods (ported from upstream PR #3 by Max Nikitin) ---
|
||||
|
||||
/**
|
||||
* Normalize a comment's `content` into a ProseMirror doc object before
|
||||
* markdown conversion. createComment/updateComment send content as a
|
||||
* JSON.stringify(...) STRING, and the server stores it as-is, so on read it
|
||||
* comes back as a string. convertProseMirrorToMarkdown returns "" for a
|
||||
* string, so parse it first (guarded — fall back to the raw value on any
|
||||
* parse failure so a non-JSON legacy value is still handled gracefully).
|
||||
*/
|
||||
protected parseCommentContent(content: any): any {
|
||||
if (typeof content !== "string") return content;
|
||||
try {
|
||||
return JSON.parse(content);
|
||||
} catch {
|
||||
return content;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* List comments on a page (cursor-paginated), content as markdown.
|
||||
*
|
||||
* DEFAULT (`includeResolved = false`) hides RESOLVED THREADS WHOLESALE so the
|
||||
* agent sees only active discussions: a top-level comment with `resolvedAt`
|
||||
* set AND every reply under it (a reply of a closed thread is part of the
|
||||
* closed thread) are dropped from `items`. `resolvedThreadsHidden` reports how
|
||||
* many resolved top-level threads were hidden so the agent can re-query with
|
||||
* `includeResolved: true` to see everything. Active threads always stay.
|
||||
*
|
||||
* Returns `{ items, resolvedThreadsHidden }` (NOT a bare array) — callers that
|
||||
* need the full feed (lossless export, transformPage, checkNewComments) pass
|
||||
* `includeResolved: true` and read `.items`.
|
||||
*/
|
||||
async listComments(pageId: string, includeResolved = false) {
|
||||
await this.ensureAuthenticated();
|
||||
let allComments: any[] = [];
|
||||
let cursor: string | null = null;
|
||||
|
||||
// Hard ceiling + immovable-cursor guard (mirrors paginateAll): if /comments
|
||||
// ever stops advancing the cursor (the exact #442 drift scenario) this loop
|
||||
// would otherwise spin forever accumulating duplicates.
|
||||
const MAX_PAGES = 50;
|
||||
let truncated = false;
|
||||
|
||||
for (let page = 0; page < MAX_PAGES; page++) {
|
||||
const payload: Record<string, any> = { pageId, limit: 100 };
|
||||
if (cursor) payload.cursor = cursor;
|
||||
|
||||
const response = await this.client.post("/comments", payload);
|
||||
const data = response.data.data || response.data;
|
||||
const items = data.items || [];
|
||||
allComments = allComments.concat(items);
|
||||
|
||||
// Advance strictly via the server-issued cursor. A missing nextCursor or a
|
||||
// cursor identical to the one we just sent means the end (or a server that
|
||||
// ignores our pagination param) — stop instead of re-fetching page one.
|
||||
const next: string | null = data.meta?.nextCursor || null;
|
||||
if (!next || next === cursor) break;
|
||||
cursor = next;
|
||||
|
||||
// Reaching the ceiling with a still-advancing cursor means truncation.
|
||||
if (page === MAX_PAGES - 1) truncated = true;
|
||||
}
|
||||
|
||||
if (truncated) {
|
||||
console.warn(
|
||||
`listComments: comments for "${pageId}" truncated at the ${MAX_PAGES}-page cap; more pages exist on the server`,
|
||||
);
|
||||
}
|
||||
|
||||
const mapped = allComments.map((comment: any) => {
|
||||
const markdown = comment.content
|
||||
? convertProseMirrorToMarkdown(
|
||||
this.parseCommentContent(comment.content),
|
||||
)
|
||||
: "";
|
||||
return filterComment(comment, markdown);
|
||||
});
|
||||
|
||||
if (includeResolved) {
|
||||
return { items: mapped, resolvedThreadsHidden: 0 };
|
||||
}
|
||||
|
||||
// Ids of RESOLVED top-level threads (a top-level comment has no
|
||||
// parentCommentId). A whole thread is hidden when its root is resolved.
|
||||
const resolvedRootIds = new Set(
|
||||
mapped
|
||||
.filter((c) => !c.parentCommentId && c.resolvedAt != null)
|
||||
.map((c) => c.id),
|
||||
);
|
||||
|
||||
const items = mapped.filter((c) => {
|
||||
// Hide the resolved root itself and every reply anchored to it. A reply's
|
||||
// own resolvedAt is irrelevant — its membership follows the parent thread.
|
||||
// ASSUMPTION: Docmost's comment model is FLAT — a reply's parentCommentId
|
||||
// always points at the thread ROOT (no reply-of-reply nesting), so a single
|
||||
// level of parent lookup covers a whole thread. If nested replies are ever
|
||||
// introduced, a deep reply of a resolved thread would need a root-walk here.
|
||||
if (!c.parentCommentId) return !resolvedRootIds.has(c.id);
|
||||
return !resolvedRootIds.has(c.parentCommentId);
|
||||
});
|
||||
|
||||
return { items, resolvedThreadsHidden: resolvedRootIds.size };
|
||||
}
|
||||
|
||||
|
||||
async getComment(commentId: string) {
|
||||
// Fail fast (#436): reject a truncated id before any network call.
|
||||
assertFullUuid("get_comment", "commentId", commentId);
|
||||
await this.ensureAuthenticated();
|
||||
const response = await this.client.post("/comments/info", { commentId });
|
||||
const comment = response.data.data || response.data;
|
||||
const markdown = comment.content
|
||||
? convertProseMirrorToMarkdown(this.parseCommentContent(comment.content))
|
||||
: "";
|
||||
return {
|
||||
data: filterComment(comment, markdown),
|
||||
success: true,
|
||||
};
|
||||
}
|
||||
|
||||
/** Plain text of each TOP-LEVEL block of `doc`, for anchor-failure hints. */
|
||||
protected topLevelBlockTexts(doc: any): string[] {
|
||||
const content = doc && Array.isArray(doc.content) ? doc.content : [];
|
||||
return content
|
||||
.map((b: any) => blockPlainText(b))
|
||||
.filter((t: string) => t.length > 0);
|
||||
}
|
||||
|
||||
/**
|
||||
* True when per-block anchoring failed but the (normalized) selection DOES
|
||||
* appear in the blocks' joined plain text — i.e. it straddles a block
|
||||
* boundary. Blocks are joined with a newline (collapsed to one space by
|
||||
* normalizeForMatch) so a selection whose parts are separated by a paragraph
|
||||
* break still matches. Callers only reach here after single-block anchoring
|
||||
* (incl. the markdown-strip fallback) has already failed.
|
||||
*/
|
||||
protected selectionSpansMultipleBlocks(
|
||||
blockTexts: string[],
|
||||
selection: string,
|
||||
): boolean {
|
||||
const normSel = normalizeForMatch(selection).norm.trim();
|
||||
if (normSel.length === 0) return false;
|
||||
const joined = normalizeForMatch(blockTexts.join("\n")).norm;
|
||||
return joined.indexOf(normSel) !== -1;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the actionable error for a createComment anchor MISS, porting
|
||||
* editPageText's self-correction affordances: an explicit "spans multiple
|
||||
* blocks" message when the selection straddles a block boundary, otherwise a
|
||||
* "closest block text" hint quoting the block that holds the selection's
|
||||
* longest token. `live` switches the wording between the pre-check (reading the
|
||||
* persisted page) and the post-create live-anchor failure (which rolls back).
|
||||
*/
|
||||
protected anchorNotFoundError(
|
||||
doc: any,
|
||||
selection: string,
|
||||
live: boolean,
|
||||
): Error {
|
||||
const blockTexts = this.topLevelBlockTexts(doc);
|
||||
const rolled = live ? " The comment was rolled back." : "";
|
||||
if (this.selectionSpansMultipleBlocks(blockTexts, selection)) {
|
||||
return new Error(
|
||||
"createComment: the selection spans multiple blocks; anchor on a " +
|
||||
"contiguous fragment within a SINGLE paragraph/block (<=250 chars)." +
|
||||
rolled,
|
||||
);
|
||||
}
|
||||
const where = live ? "in the live document" : "in the page";
|
||||
return new Error(
|
||||
`createComment: could not find the selection text ${where} to anchor ` +
|
||||
"the comment. Provide the EXACT contiguous text from a single " +
|
||||
"paragraph/block (<=250 chars)." +
|
||||
closestBlockHint(blockTexts, selection) +
|
||||
rolled,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Create an inline comment anchored to its `selection` text, or a reply.
|
||||
*
|
||||
* Top-level comments (no `parentCommentId`) are ALWAYS inline and MUST carry a
|
||||
* `selection`: the `type` argument is kept for interface compatibility but the
|
||||
* effective type is coerced to "inline". The selection has to anchor in the
|
||||
* document; if it cannot, the comment is rolled back and an error is thrown so
|
||||
* the caller is forced to supply a proper inline selection rather than leaving
|
||||
* an orphan, unanchored comment behind. Replies (parentCommentId set) inherit
|
||||
* their parent's anchor: they take NO selection and are not anchored.
|
||||
*/
|
||||
async createComment(
|
||||
pageId: string,
|
||||
content: string,
|
||||
type: "page" | "inline" = "page",
|
||||
selection?: string,
|
||||
parentCommentId?: string,
|
||||
suggestedText?: string,
|
||||
) {
|
||||
// Fail fast (#436): a provided parent id must be a full UUID before any
|
||||
// network call. Validate only when truthy — a falsy parentCommentId means
|
||||
// "top-level comment" (mirrors the isReply computation below), not a reply.
|
||||
if (parentCommentId) {
|
||||
assertFullUuid("createComment", "parentCommentId", parentCommentId);
|
||||
}
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
const isReply = !!parentCommentId;
|
||||
const hasSuggestion =
|
||||
suggestedText !== undefined && suggestedText !== null;
|
||||
// Defense in depth mirroring the server DTO/service: a suggested edit rewrites
|
||||
// the exact anchored text, so it is only meaningful on a top-level inline
|
||||
// comment that carries a selection.
|
||||
if (hasSuggestion) {
|
||||
if (isReply) {
|
||||
throw new Error(
|
||||
"createComment: a suggested edit (suggestedText) cannot be attached to a reply; it applies only to a top-level inline comment.",
|
||||
);
|
||||
}
|
||||
if (!selection || !selection.trim()) {
|
||||
throw new Error(
|
||||
"createComment: a suggested edit (suggestedText) requires a 'selection' to anchor and rewrite.",
|
||||
);
|
||||
}
|
||||
}
|
||||
// Only top-level comments are inline-anchored, so they are stored as
|
||||
// "inline". Replies carry no inline selection, so they keep the historical
|
||||
// general ("page") type — both backward-compatible and semantically correct.
|
||||
// The `type` argument is kept for interface compatibility; createComment
|
||||
// normalizes the effective type internally, so callers may pass "inline".
|
||||
const effectiveType: "page" | "inline" = isReply ? "page" : "inline";
|
||||
if (!isReply && (!selection || !selection.trim())) {
|
||||
throw new Error(
|
||||
"createComment: an inline 'selection' (exact text to anchor on) is required for a top-level comment",
|
||||
);
|
||||
}
|
||||
|
||||
// For a SUGGESTION, the value we store as the comment's `selection` must be
|
||||
// the RAW document substring the mark lands on (typographic quotes/dashes,
|
||||
// nbsp, collapsed whitespace), NOT the agent's ASCII input. The anchor is
|
||||
// placed via normalization, so when the doc was auto-converted to
|
||||
// typographic the raw substring differs from the agent input; apply-time
|
||||
// compares the stored selection to the marked doc text STRICTLY, so storing
|
||||
// the raw substring is what makes "Apply" succeed instead of a spurious 409.
|
||||
// Captured in the pre-check below (which already reads the page) and used as
|
||||
// payload.selection. Ordinary comments keep sending the raw agent selection.
|
||||
let anchoredSelection: string | null = null;
|
||||
// Set when the anchor matched only after stripping markdown from the
|
||||
// selection (the strip fallback); surfaced as a soft warning like
|
||||
// editPageText does, so a stale-markdown selection is flagged.
|
||||
let anchorNormalized = false;
|
||||
|
||||
// For a top-level comment, fail BEFORE creating anything when the selection
|
||||
// is not present in the persisted document — this avoids leaving an orphan
|
||||
// comment + notification behind. A read failure (network) is non-fatal: the
|
||||
// live anchor step below still enforces the anchoring invariant.
|
||||
if (!isReply && selection) {
|
||||
try {
|
||||
const page = await this.getPageJson(pageId);
|
||||
if (hasSuggestion) {
|
||||
// A suggestion's anchor MUST be unambiguous: applying it rewrites the
|
||||
// exact anchored text, and ordinary anchoring silently takes the first
|
||||
// occurrence, so 0 matches -> not found and >=2 -> ambiguous, both
|
||||
// rejected BEFORE creating the comment.
|
||||
const matches = countAnchorMatches(page.content, selection);
|
||||
if (matches === 0) {
|
||||
throw this.anchorNotFoundError(page.content, selection, false);
|
||||
}
|
||||
if (matches >= 2) {
|
||||
throw new Error(
|
||||
`createComment: the suggestion's selection is ambiguous — it occurs ${matches} times in the page. ` +
|
||||
"A suggested edit must anchor to a UNIQUE location; expand the selection with surrounding context " +
|
||||
"(still <=250 chars) so it appears exactly once.",
|
||||
);
|
||||
}
|
||||
// Exactly one match: capture the RAW anchored substring to store as the
|
||||
// comment selection (so apply-time equality holds). If this returns
|
||||
// null despite countAnchorMatches===1 (shouldn't happen), fall back to
|
||||
// the raw agent selection below rather than crash.
|
||||
anchoredSelection = getAnchoredText(page.content, selection);
|
||||
anchorNormalized = resolveAnchorSelection(
|
||||
page.content,
|
||||
selection,
|
||||
).normalized;
|
||||
} else {
|
||||
const resolved = resolveAnchorSelection(page.content, selection);
|
||||
if (!resolved.found) {
|
||||
throw this.anchorNotFoundError(page.content, selection, false);
|
||||
}
|
||||
anchorNormalized = resolved.normalized;
|
||||
}
|
||||
} catch (e) {
|
||||
// Rethrow our own "not found"/"ambiguous"/"spans multiple blocks" errors;
|
||||
// swallow read/network errors so the live anchor step can still try (and
|
||||
// enforce) anchoring.
|
||||
if (
|
||||
e instanceof Error &&
|
||||
(e.message.startsWith("createComment: could not find the selection") ||
|
||||
e.message.startsWith(
|
||||
"createComment: the selection spans multiple blocks",
|
||||
) ||
|
||||
e.message.startsWith(
|
||||
"createComment: the suggestion's selection is ambiguous",
|
||||
))
|
||||
) {
|
||||
throw e;
|
||||
}
|
||||
if (process.env.DEBUG) {
|
||||
console.error(
|
||||
"Pre-check getPageJson failed; deferring to live anchor step:",
|
||||
e,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Convert through the full Docmost schema. Deliberately the NON-canonicalizing
|
||||
// variant: a comment body may carry a footnote definition with no matching
|
||||
// reference, and canonicalization would drop it (data loss). See
|
||||
// markdownToProseMirror vs markdownToProseMirrorCanonical.
|
||||
const jsonContent = await markdownToProseMirror(content);
|
||||
const payload: Record<string, any> = {
|
||||
pageId,
|
||||
content: JSON.stringify(jsonContent),
|
||||
type: effectiveType,
|
||||
};
|
||||
// For a suggestion, store the RAW anchored substring (anchoredSelection) so
|
||||
// the stored selection === the text under the mark === apply-time
|
||||
// expectedText. Ordinary comments (and the null fallback) keep the raw
|
||||
// agent selection — their selection is only display/anchor and never used
|
||||
// by apply, so their behavior is unchanged.
|
||||
if (!isReply && selection)
|
||||
payload.selection = anchoredSelection ?? selection;
|
||||
if (parentCommentId) payload.parentCommentId = parentCommentId;
|
||||
// Only a top-level inline comment (with a selection) may carry a suggestion.
|
||||
if (!isReply && selection && hasSuggestion) {
|
||||
payload.suggestedText = suggestedText;
|
||||
}
|
||||
|
||||
const response = await this.client.post("/comments/create", payload);
|
||||
const comment = response.data.data || response.data;
|
||||
const markdown = comment.content
|
||||
? convertProseMirrorToMarkdown(this.parseCommentContent(comment.content))
|
||||
: content;
|
||||
const result: any = {
|
||||
data: filterComment(comment, markdown),
|
||||
success: true,
|
||||
};
|
||||
|
||||
// Replies inherit the parent's anchor: no selection, no anchoring.
|
||||
if (isReply) {
|
||||
return result;
|
||||
}
|
||||
|
||||
// Anchor the comment in the document. The /comments/create API records the
|
||||
// comment + its `selection` text, but it does NOT insert the comment MARK
|
||||
// into the page content, so without this the inline comment has no
|
||||
// highlight/anchor and is not clickable. If anchoring fails the comment is
|
||||
// rolled back (deleted) and an error is thrown — never an orphan comment.
|
||||
const newCommentId: string = comment.id;
|
||||
// Guard: a create response without an id would mean writing a comment mark
|
||||
// with commentId: undefined and a later delete of a falsy id. We have no id
|
||||
// to roll back here (nothing was created with an id), so just fail loudly.
|
||||
if (!newCommentId) {
|
||||
throw new Error(
|
||||
"createComment: the server returned no comment id, so the comment could not be anchored",
|
||||
);
|
||||
}
|
||||
let anchored = false;
|
||||
// Set inside the transform when a suggestion's live anchor is ambiguous
|
||||
// (>=2 occurrences), so the rollback path can surface the right error.
|
||||
let ambiguousInLiveDoc = false;
|
||||
// Captured inside the transform on a not-found abort, so the rollback path
|
||||
// can surface the closest-block / spans-multiple-blocks hint built from the
|
||||
// LIVE document (the pre-check page is not in scope there).
|
||||
let liveNotFoundError: Error | null = null;
|
||||
try {
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260). The
|
||||
// /comments/create REST call above keeps the agent-supplied id.
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
// Route through the mutatePage seam (not the free function) so this
|
||||
// wrapper's uniqueness gate + rollback can be unit-tested without a live
|
||||
// Hocuspocus collab socket.
|
||||
const mutation = await this.mutatePage(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
const doc =
|
||||
liveDoc && liveDoc.type === "doc"
|
||||
? liveDoc
|
||||
: { type: "doc", content: [] };
|
||||
if (hasSuggestion) {
|
||||
// Authoritative uniqueness check against the LIVE document: a
|
||||
// suggestion must anchor to EXACTLY ONE occurrence, otherwise
|
||||
// "Apply" would rewrite the wrong/ambiguous text. If the live doc
|
||||
// no longer has exactly one occurrence (it changed since the
|
||||
// pre-check), abort so the just-created comment is rolled back
|
||||
// rather than mis-anchored to the first occurrence.
|
||||
const liveCount = countAnchorMatches(doc, selection as string);
|
||||
if (liveCount !== 1) {
|
||||
ambiguousInLiveDoc = liveCount >= 2;
|
||||
if (liveCount === 0) {
|
||||
liveNotFoundError = this.anchorNotFoundError(
|
||||
doc,
|
||||
selection as string,
|
||||
true,
|
||||
);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
}
|
||||
if (applyAnchorInDoc(doc, selection as string, newCommentId)) {
|
||||
anchored = true;
|
||||
return doc;
|
||||
}
|
||||
// Selection text not found in the LIVE document: abort the write. The
|
||||
// rollback + throw below turns this into a hard error.
|
||||
liveNotFoundError = this.anchorNotFoundError(
|
||||
doc,
|
||||
selection as string,
|
||||
true,
|
||||
);
|
||||
return null;
|
||||
},
|
||||
);
|
||||
result.verify = mutation.verify;
|
||||
} catch (e) {
|
||||
// The comment record already exists; roll it back so we never leave an
|
||||
// orphan, then rethrow the original anchoring error.
|
||||
await this.safeDeleteComment(newCommentId);
|
||||
throw e;
|
||||
}
|
||||
|
||||
if (!anchored) {
|
||||
// Mutation aborted because the selection was not found (or, for a
|
||||
// suggestion, was ambiguous) in the live document. Roll back the comment
|
||||
// and surface a hard error.
|
||||
await this.safeDeleteComment(newCommentId);
|
||||
if (ambiguousInLiveDoc) {
|
||||
throw new Error(
|
||||
"createComment: the suggestion's selection is ambiguous in the live document (multiple occurrences); the comment was rolled back. Expand the selection with surrounding context so it is unique.",
|
||||
);
|
||||
}
|
||||
throw (
|
||||
liveNotFoundError ??
|
||||
new Error(
|
||||
"createComment: failed to anchor the comment (selection not found in the live document); the comment was rolled back",
|
||||
)
|
||||
);
|
||||
}
|
||||
|
||||
// Soft warning (like editPageText): the selection only matched after
|
||||
// stripping markdown, so the caller likely quoted a styled fragment.
|
||||
if (anchorNormalized) {
|
||||
result.warning =
|
||||
"The selection matched only after stripping markdown syntax; the comment " +
|
||||
"was anchored on the document's plain text. Copy the selection verbatim " +
|
||||
"from getPage / searchInPage output to avoid this.";
|
||||
}
|
||||
|
||||
result.anchored = true;
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Best-effort rollback of a just-created comment. Swallows any delete failure
|
||||
* (logging under DEBUG) so a failed cleanup never masks the original error.
|
||||
*/
|
||||
protected async safeDeleteComment(commentId: string): Promise<void> {
|
||||
// Defense in depth: never call the delete API with a falsy id — there is
|
||||
// nothing to roll back, and deleteComment(undefined) would hit a bad route.
|
||||
if (!commentId) return;
|
||||
try {
|
||||
await this.deleteComment(commentId);
|
||||
} catch (delErr) {
|
||||
if (process.env.DEBUG) {
|
||||
console.error(
|
||||
"Failed to roll back comment after anchoring error:",
|
||||
delErr,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
async updateComment(commentId: string, content: string) {
|
||||
// Fail fast (#436): reject a truncated id before any network call.
|
||||
assertFullUuid("updateComment", "commentId", commentId);
|
||||
await this.ensureAuthenticated();
|
||||
// NON-canonicalizing on purpose (comment body — see createComment).
|
||||
const jsonContent = await markdownToProseMirror(content);
|
||||
await this.client.post("/comments/update", {
|
||||
commentId,
|
||||
content: JSON.stringify(jsonContent),
|
||||
});
|
||||
return {
|
||||
success: true,
|
||||
commentId,
|
||||
message: "Comment updated successfully.",
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
async deleteComment(commentId: string) {
|
||||
// Fail fast (#436): reject a truncated id before any network call.
|
||||
assertFullUuid("deleteComment", "commentId", commentId);
|
||||
await this.ensureAuthenticated();
|
||||
return this.client
|
||||
.post("/comments/delete", { commentId })
|
||||
.then((res) => res.data);
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve or reopen a top-level comment thread (reversible — `resolved`
|
||||
* toggles the state). Only top-level comments can be resolved; the server
|
||||
* rejects resolving a reply. Hits POST /comments/resolve.
|
||||
*/
|
||||
async resolveComment(commentId: string, resolved: boolean) {
|
||||
// Fail fast (#436): reject a truncated id before any network call.
|
||||
assertFullUuid("resolveComment", "commentId", commentId);
|
||||
await this.ensureAuthenticated();
|
||||
const response = await this.client.post("/comments/resolve", {
|
||||
commentId,
|
||||
resolved,
|
||||
});
|
||||
const comment = response.data?.data ?? response.data;
|
||||
return {
|
||||
success: true,
|
||||
commentId,
|
||||
resolved,
|
||||
comment,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Check for new comments across pages in a space (optionally scoped to a
|
||||
* subtree): pages updated after `since` are scanned and their comments
|
||||
* filtered by createdAt > since.
|
||||
*/
|
||||
async checkNewComments(
|
||||
spaceId: string,
|
||||
since: string,
|
||||
parentPageId?: string,
|
||||
) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
const sinceDate = new Date(since);
|
||||
|
||||
// Reject an unparseable `since`: comparing against an Invalid Date silently
|
||||
// yields zero new comments (every `>` against NaN is false), which would
|
||||
// mask a malformed input as "nothing new" instead of erroring.
|
||||
if (Number.isNaN(sinceDate.getTime())) {
|
||||
throw new Error(
|
||||
`checkNewComments: invalid "since" date "${since}"; expected an ISO-8601 timestamp`,
|
||||
);
|
||||
}
|
||||
|
||||
// 1. Enumerate the FULL set of pages in scope via the page tree (a complete
|
||||
// page index), NOT the bounded "/pages/recent" feed which caps at ~5000
|
||||
// recent items and silently misses comments on older pages.
|
||||
//
|
||||
// Subtree scope: when parentPageId is given, the scope is that page ITSELF
|
||||
// plus every descendant. Otherwise the scope is the whole space (all roots
|
||||
// and their descendants).
|
||||
//
|
||||
// NOTE: do NOT pre-filter by page.updatedAt — creating a comment does not
|
||||
// bump it (verified on a live server), so such a filter silently misses
|
||||
// comments on pages that were not otherwise edited. The complete tree walk
|
||||
// already restricts the scope correctly, so no recent-feed allow-list is
|
||||
// needed any more.
|
||||
//
|
||||
// The subtree scope (parentPageId given) already INCLUDES the root node
|
||||
// itself: /pages/tree seeds getPageAndDescendants with id = parentPageId, so
|
||||
// no separate getPageRaw fetch for the parent is needed.
|
||||
const { pages: pagesInScope, truncated } = await this.enumerateSpacePages(
|
||||
spaceId,
|
||||
parentPageId,
|
||||
);
|
||||
|
||||
// 2. Fetch comments for each page, keep ones created after since
|
||||
const results: any[] = [];
|
||||
for (const page of pagesInScope) {
|
||||
try {
|
||||
// Full feed (incl. resolved): a "new comments since" scan reports all
|
||||
// recent activity; the active-only filter is scoped to listComments.
|
||||
const comments = (await this.listComments(page.id, true)).items;
|
||||
const newComments = comments.filter(
|
||||
(c: any) => new Date(c.createdAt) > sinceDate,
|
||||
);
|
||||
if (newComments.length > 0) {
|
||||
results.push({
|
||||
pageId: page.id,
|
||||
pageTitle: page.title,
|
||||
comments: newComments,
|
||||
});
|
||||
}
|
||||
} catch (e: any) {
|
||||
// Skip pages with errors (e.g. deleted between calls)
|
||||
}
|
||||
}
|
||||
|
||||
const totalNewComments = results.reduce(
|
||||
(sum, r) => sum + r.comments.length,
|
||||
0,
|
||||
);
|
||||
|
||||
// `truncated` is reported by enumerateSpacePages: it is true ONLY when the
|
||||
// stdio fallback BFS hit its node cap. The primary /pages/tree path is
|
||||
// uncapped, so a space with legitimately many pages is not falsely flagged.
|
||||
return {
|
||||
since,
|
||||
scope: parentPageId ? `subtree of ${parentPageId}` : `space ${spaceId}`,
|
||||
checkedPages: pagesInScope.length,
|
||||
pagesWithNewComments: results.length,
|
||||
totalNewComments,
|
||||
truncated,
|
||||
comments: results,
|
||||
};
|
||||
}
|
||||
|
||||
// --- Image upload / embedding ---
|
||||
|
||||
/** Map a Content-Type string to a supported MIME type, or null if unsupported. */
|
||||
}
|
||||
return CommentsMixin;
|
||||
}
|
||||
@@ -0,0 +1,690 @@
|
||||
// Shared client context + core seams (issue #450). The abstract base of the
|
||||
// DocmostClient mixin chain: it owns ALL shared instance state (the axios
|
||||
// client, apiUrl, auth tokens, the resolvePageId cache, the collab-token cache,
|
||||
// the sandbox/metrics sinks) and the core HTTP/auth/pagination/write seams every
|
||||
// domain module builds on. Domain modules are mixins layered on top; the final
|
||||
// DocmostClient (client.ts) assembles them. Extracted VERBATIM from the original
|
||||
// monolith — only field/seam visibility was widened from `private` to
|
||||
// `protected` so sibling mixins can reach the shared state through `this`, and
|
||||
// the cross-module methods that live in other mixins are declared `abstract`
|
||||
// here so `this.<method>` type-checks. No behaviour changed.
|
||||
import axios, { AxiosInstance } from "axios";
|
||||
import FormData from "form-data";
|
||||
import {
|
||||
updatePageContentRealtime,
|
||||
replacePageContent,
|
||||
markdownToProseMirror,
|
||||
markdownToProseMirrorCanonical,
|
||||
mutatePageContent,
|
||||
assertYjsEncodable,
|
||||
MutationResult,
|
||||
} from "../lib/collaboration.js";
|
||||
import { acquireCollabSession } from "../lib/collab-session.js";
|
||||
import { withPageLock, isUuid } from "../lib/page-lock.js";
|
||||
import { getCollabToken, performLogin } from "../lib/auth-utils.js";
|
||||
import { formatDocmostAxiosError } from "./errors.js";
|
||||
|
||||
// A generic mixin base constructor (issue #450). Each domain mixin is a factory
|
||||
// `<T extends GConstructor<DocmostClientContext>>(Base: T) => class extends Base`
|
||||
// so the mixins compose into one prototype chain sharing this context.
|
||||
export type GConstructor<T = {}> = abstract new (...args: any[]) => T;
|
||||
|
||||
/**
|
||||
* Configuration for a DocmostClient / MCP server instance. A discriminated
|
||||
* union: either service-account credentials (email/password — the client calls
|
||||
* performLogin, powering the external /mcp HTTP endpoint and the stdio CLI) OR
|
||||
* a token getter (getToken — the client uses the returned BARE access JWT as
|
||||
* the Bearer and never calls performLogin; used for the internal per-user path).
|
||||
*
|
||||
* Both branches may ALSO carry an optional `getCollabToken` provider. When set,
|
||||
* content mutations (which go over the collaboration websocket) use the token it
|
||||
* returns INSTEAD of calling `POST /auth/collab-token`. The internal per-user
|
||||
* agent path uses this to hand the client a provenance collab token (signed
|
||||
* `actor:'agent'`+`aiChatId`), so agent content edits are attributed without a
|
||||
* spoofable client-side field. When absent the client keeps the original
|
||||
* `/auth/collab-token` path (service-account/stdio unchanged).
|
||||
*
|
||||
* Housed here (not in index.ts) so client.ts has no type dependency on index.ts;
|
||||
* index.ts re-exports it for the package's public surface.
|
||||
*/
|
||||
// Sink the stash tool writes blobs into. The host app binds this to its in-RAM
|
||||
// SandboxStore and composes the public `uri` (the package never sees the store
|
||||
// or any env). `put` returns the anonymous read URL plus integrity metadata.
|
||||
export type SandboxPut = (
|
||||
buf: Buffer,
|
||||
mime: string,
|
||||
) => { uri: string; sha256: string; size: number };
|
||||
|
||||
export type DocmostMcpConfig = { apiUrl: string } & (
|
||||
| { email: string; password: string }
|
||||
| { getToken: () => Promise<string> } // returns a BARE JWT; the client adds "Bearer "
|
||||
) & {
|
||||
// Optional collab-token provider (returns a ready collab JWT). Common to
|
||||
// both branches; see the type doc above.
|
||||
getCollabToken?: () => Promise<string>;
|
||||
// Optional blob sandbox sink. Present only where the stash tool is wired;
|
||||
// when absent, stashPage throws a clear "not configured" error. The
|
||||
// optional `has`/`evict` probes let stashPage keep its mirror counts honest
|
||||
// under the store's FIFO eviction (see stashPage); older sinks omit them.
|
||||
sandbox?: {
|
||||
put: SandboxPut;
|
||||
has?: (uri: string) => boolean;
|
||||
evict?: (uri: string) => void;
|
||||
};
|
||||
// Dependency-neutral metrics sink. When present, the client emits generic
|
||||
// (name, value, labels) samples; the HOST maps those names onto its own
|
||||
// metrics registry (the package never depends on prom-client or the server).
|
||||
// Absent in standalone/stdio mode → the client is a complete no-op here.
|
||||
onMetric?: (
|
||||
name: string,
|
||||
value: number,
|
||||
labels?: Record<string, string>,
|
||||
) => void;
|
||||
};
|
||||
|
||||
|
||||
/**
|
||||
* Collab-token cache TTL in milliseconds (issue #435). Read fresh from the
|
||||
* environment on every mint — like collab-session.ts readConfig — so tests and a
|
||||
* live rollback can change it without reloading the module.
|
||||
*
|
||||
* Why a cache at all: the live CollabSession registry (#400/#431) keys sessions
|
||||
* on (wsUrl, pageId, collabToken) for identity isolation (invariant 4). But BOTH
|
||||
* collab-token sources mint a FRESH token per mutation — the in-app provider
|
||||
* re-signs a JWT whose iat/exp (seconds) changes every second, and the external
|
||||
* MCP POSTs /auth/collab-token each call — so the token in the key changed on
|
||||
* every op and the session was almost never reused (connect-storms, 25s
|
||||
* timeouts, zombie sessions). Caching the token per-client keeps the key stable
|
||||
* across a burst of mutations so ONE session is reused.
|
||||
*
|
||||
* Default 5 min: well under the 24h collab-token lifetime AND <= the collab
|
||||
* session max-age (10 min, MCP_COLLAB_SESSION_MAX_AGE_MS), so the
|
||||
* permission-staleness window is not widened beyond what #431 already accepted.
|
||||
* The rollback knob is an EXPLICIT 0 (or a negative number): that DISABLES the
|
||||
* cache — an exact fetch-per-call legacy path, mirroring how idleMs<=0 disables
|
||||
* the session cache. Unset OR unparseable (e.g. a typo like "5min", "abc") falls
|
||||
* back to the 5-min default with the cache ON — parseInt yields NaN, which is
|
||||
* treated as "not configured", not as "disabled". So to turn the cache off you
|
||||
* must set the value to exactly 0, not to garbage.
|
||||
*/
|
||||
function readCollabTokenTtlMs(): number {
|
||||
const raw = parseInt(process.env.MCP_COLLAB_TOKEN_TTL_MS ?? "", 10);
|
||||
return Number.isFinite(raw) ? Math.max(0, raw) : 5 * 60 * 1000;
|
||||
}
|
||||
|
||||
export abstract class DocmostClientContext {
|
||||
protected client: AxiosInstance;
|
||||
protected token: string | null = null;
|
||||
protected apiUrl: string;
|
||||
// email/password are only set on the service-account (credentials) variant;
|
||||
// null on the getToken variant (where there are no credentials to log in with).
|
||||
protected email: string | null = null;
|
||||
protected password: string | null = null;
|
||||
// Per-user token provider. When set, login() calls it to obtain a BARE access
|
||||
// JWT instead of performLogin, and the 401/403 re-auth path re-calls it.
|
||||
protected getTokenFn: (() => Promise<string>) | null = null;
|
||||
// Optional collab-token provider. When set, getCollabTokenWithReauth() returns
|
||||
// its token instead of calling POST /auth/collab-token; on a 401/403 it is
|
||||
// re-invoked once. Used by the internal agent to carry signed provenance.
|
||||
protected getCollabTokenFn: (() => Promise<string>) | null = null;
|
||||
// Optional blob-sandbox sink for the stash tool. Null when not configured.
|
||||
protected sandboxPut: SandboxPut | null = null;
|
||||
// Optional probes paired with the sink. `has` lets stashPage detect a blob
|
||||
// FIFO-evicted by a LATER put in the same stash; `evict` lets it free this
|
||||
// op's image blobs if the final doc put throws. Null when the sink omits them.
|
||||
protected sandboxHas: ((uri: string) => boolean) | null = null;
|
||||
protected sandboxEvict: ((uri: string) => void) | null = null;
|
||||
// Optional dependency-neutral metrics sink (see DocmostMcpConfig.onMetric).
|
||||
// Null on the legacy positional form and whenever the host omits it → no-op.
|
||||
protected onMetricFn:
|
||||
| ((name: string, value: number, labels?: Record<string, string>) => void)
|
||||
| null = null;
|
||||
// In-flight login dedup: when the token expires, the 401 interceptor,
|
||||
// ensureAuthenticated, getCollabTokenWithReauth and the two multipart retries
|
||||
// can all call login() at once. Memoizing a single promise collapses that
|
||||
// thundering herd into ONE /auth/login request that everyone awaits.
|
||||
protected loginPromise: Promise<void> | null = null;
|
||||
// Canonical-UUID cache for resolvePageId: maps an agent-supplied slugId to the
|
||||
// page's canonical UUID, so repeated collab edits on the same page do not
|
||||
// re-fetch /pages/info. A UUID input short-circuits before this cache (see
|
||||
// resolvePageId), so only slugId->uuid entries are stored/read here.
|
||||
protected pageIdCache = new Map<string, string>();
|
||||
|
||||
// Collab-token cache (issue #435): the last minted collab token plus the
|
||||
// wall-clock time it was minted, so a burst of content mutations reuses ONE
|
||||
// token and therefore ONE live CollabSession (whose registry key includes the
|
||||
// token — #400 invariant 4). Per-instance: a DocmostClient is built per
|
||||
// user/per chat request, so a cached token can never leak across identities.
|
||||
// Reset whenever the client's identity changes (login() / this.token cleared);
|
||||
// bypassed on a forced refresh (the 401/403 reauth path). null = no token yet.
|
||||
protected collabTokenCache: { token: string; mintedAt: number } | null = null;
|
||||
|
||||
// Two construction forms:
|
||||
// - new DocmostClient(config) // discriminated union (current)
|
||||
// - new DocmostClient(baseURL, email, password) // legacy positional creds
|
||||
// The positional form is retained so existing callers/tests keep working; it
|
||||
// is exactly equivalent to the credentials branch of the object form.
|
||||
constructor(config: DocmostMcpConfig);
|
||||
constructor(baseURL: string, email: string, password: string);
|
||||
constructor(
|
||||
configOrBaseURL: DocmostMcpConfig | string,
|
||||
email?: string,
|
||||
password?: string,
|
||||
) {
|
||||
// Normalize the legacy positional form into the object union.
|
||||
const config: DocmostMcpConfig =
|
||||
typeof configOrBaseURL === "string"
|
||||
? { apiUrl: configOrBaseURL, email: email!, password: password! }
|
||||
: configOrBaseURL;
|
||||
|
||||
this.apiUrl = config.apiUrl;
|
||||
if ("getToken" in config) {
|
||||
// Token variant: carry the user's JWT via getToken; no credentials, so
|
||||
// login() must never call performLogin (there is nothing to log in with).
|
||||
this.getTokenFn = config.getToken;
|
||||
} else {
|
||||
// Service-account variant: behaves exactly as before (performLogin).
|
||||
this.email = config.email;
|
||||
this.password = config.password;
|
||||
}
|
||||
// Optional, available to both variants. When present, content mutations get
|
||||
// their collab token from here instead of POST /auth/collab-token.
|
||||
if (config.getCollabToken) {
|
||||
this.getCollabTokenFn = config.getCollabToken;
|
||||
}
|
||||
if (config.sandbox) {
|
||||
this.sandboxPut = config.sandbox.put;
|
||||
this.sandboxHas = config.sandbox.has ?? null;
|
||||
this.sandboxEvict = config.sandbox.evict ?? null;
|
||||
}
|
||||
// Legacy positional form carries no onMetric → null (complete no-op).
|
||||
this.onMetricFn = config.onMetric ?? null;
|
||||
this.client = axios.create({
|
||||
baseURL: this.apiUrl,
|
||||
// Default request timeout so a hung connection cannot wedge a per-page
|
||||
// lock or block the server indefinitely. Multipart uploads override this
|
||||
// with a longer per-request timeout.
|
||||
timeout: 30000,
|
||||
headers: {
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
});
|
||||
|
||||
// Re-authenticate transparently on a 401/403 once: the JWT authToken can
|
||||
// expire while the server is long-running, after which every cached-token
|
||||
// request would otherwise fail until a manual restart. On such a response,
|
||||
// clear the stale token, perform a fresh login, and replay the original
|
||||
// request exactly once (guarded by config._retry to avoid infinite loops;
|
||||
// the login request itself is never retried).
|
||||
this.client.interceptors.response.use(
|
||||
(response) => response,
|
||||
async (error) => {
|
||||
const config = error.config;
|
||||
const status = error.response?.status;
|
||||
const isAuthError = status === 401 || status === 403;
|
||||
const isLoginRequest =
|
||||
typeof config?.url === "string" && config.url.includes("/auth/login");
|
||||
|
||||
if (config && isAuthError && !config._retry && !isLoginRequest) {
|
||||
config._retry = true;
|
||||
// Drop the stale token + Authorization header before re-login. Also
|
||||
// clear the collab-token cache (#435): a new identity/login must not
|
||||
// keep serving a collab token minted under the old one.
|
||||
this.token = null;
|
||||
this.collabTokenCache = null;
|
||||
delete this.client.defaults.headers.common["Authorization"];
|
||||
try {
|
||||
await this.login();
|
||||
} catch (loginError) {
|
||||
// Re-login failed: surface the original error to the caller.
|
||||
return Promise.reject(error);
|
||||
}
|
||||
// Re-issue the original request with the freshly minted Bearer token.
|
||||
// Read it from the default header that login() just set, not from
|
||||
// this.token, to avoid a theoretical "Bearer null" if this.token was
|
||||
// cleared between login() resolving and this point.
|
||||
config.headers = config.headers || {};
|
||||
config.headers["Authorization"] =
|
||||
this.client.defaults.headers.common["Authorization"];
|
||||
return this.client.request(config);
|
||||
}
|
||||
|
||||
return Promise.reject(error);
|
||||
},
|
||||
);
|
||||
|
||||
// Diagnostics interceptor (issue #437). Registered AFTER the re-login
|
||||
// interceptor so a successful re-login retry (which resolves to a real
|
||||
// response) is never seen here as an error; only a genuine failure reaches
|
||||
// this rejection handler. It reformats error.message IN PLACE (see
|
||||
// formatDocmostAxiosError — kept as a mutation, not a custom Error class, so
|
||||
// the surrounding axios.isAxiosError / error.response?.status / config._retry
|
||||
// checks keep working) and re-rejects the SAME error. The _docmostFormatted
|
||||
// flag makes a re-processed retry-failure a no-op.
|
||||
this.client.interceptors.response.use(
|
||||
(response) => response,
|
||||
(error) => {
|
||||
formatDocmostAxiosError(error);
|
||||
return Promise.reject(error);
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
// --- Cross-module seams (issue #450) -----------------------------------
|
||||
// A method in one domain mixin sometimes calls a PROTECTED method owned by
|
||||
// another mixin (e.g. nodes-write -> validateDocUrls in doc-validate). Those
|
||||
// callees are `protected`, so they cannot be surfaced through the public
|
||||
// per-mixin interfaces. Declaring them here on the shared base lets `this.<m>`
|
||||
// type-check across modules. Each is a stub that is ALWAYS overridden by the
|
||||
// owning mixin (layered above this base in the chain), so the body never runs;
|
||||
// it throws only to make an impossible mis-wiring loud instead of silent.
|
||||
// (The PUBLIC cross-module callees — getPage, getPageJson, listComments,
|
||||
// deleteComment, listPageHistory — arrive via the mixins' public interfaces,
|
||||
// so they are not restated here.)
|
||||
protected enumerateSpacePages(
|
||||
_spaceId: string,
|
||||
_rootPageId?: string,
|
||||
): Promise<{ pages: any[]; truncated: boolean }> {
|
||||
throw new Error("enumerateSpacePages not wired (missing ReadMixin)");
|
||||
}
|
||||
protected validateDocUrls(_node: any, _depth?: number): void {
|
||||
throw new Error("validateDocUrls not wired (missing DocValidateMixin)");
|
||||
}
|
||||
protected validateDocStructure(_node: any, _depth?: number): void {
|
||||
throw new Error("validateDocStructure not wired (missing DocValidateMixin)");
|
||||
}
|
||||
protected assertValidNodeShape(_op: string, _node: any): void {
|
||||
throw new Error("assertValidNodeShape not wired (missing DocValidateMixin)");
|
||||
}
|
||||
protected fetchInternalFile(
|
||||
_src: string,
|
||||
): Promise<{ buffer: Buffer; mime: string }> {
|
||||
throw new Error("fetchInternalFile not wired (missing StashMixin)");
|
||||
}
|
||||
protected uploadAttachmentBuffer(
|
||||
_pageId: string,
|
||||
_buffer: Buffer,
|
||||
_fileName: string,
|
||||
_mime: string,
|
||||
): Promise<{ id: string; fileName: string; fileSize: number }> {
|
||||
throw new Error("uploadAttachmentBuffer not wired (missing MediaMixin)");
|
||||
}
|
||||
protected fetchAttachmentText(_src: string): Promise<string> {
|
||||
throw new Error("fetchAttachmentText not wired (missing MediaMixin)");
|
||||
}
|
||||
// PUBLIC cross-module callees. Declared here too (as always-overridden stubs)
|
||||
// so a mixin calling e.g. `this.getPageJson` type-checks against the base —
|
||||
// the mixin's own public interface only covers its own methods. The real
|
||||
// implementations live in ReadMixin / CommentsMixin / PagesMixin and shadow
|
||||
// these on the prototype chain.
|
||||
getPage(_pageId: string): Promise<any> {
|
||||
throw new Error("getPage not wired (missing ReadMixin)");
|
||||
}
|
||||
getPageJson(_pageId: string): Promise<any> {
|
||||
throw new Error("getPageJson not wired (missing ReadMixin)");
|
||||
}
|
||||
listComments(_pageId: string, _includeResolved?: boolean): Promise<any> {
|
||||
throw new Error("listComments not wired (missing CommentsMixin)");
|
||||
}
|
||||
deleteComment(_commentId: string): Promise<any> {
|
||||
throw new Error("deleteComment not wired (missing CommentsMixin)");
|
||||
}
|
||||
listPageHistory(_pageId: string, _cursor?: string): Promise<any> {
|
||||
throw new Error("listPageHistory not wired (missing PagesMixin)");
|
||||
}
|
||||
|
||||
/** Application base URL (API URL without the /api suffix). */
|
||||
get appUrl(): string {
|
||||
return this.apiUrl.replace(/\/api\/?$/, "");
|
||||
}
|
||||
|
||||
|
||||
async login() {
|
||||
// Reuse an in-flight login if one is already running so concurrent callers
|
||||
// share a single token fetch instead of each issuing their own.
|
||||
if (!this.loginPromise) {
|
||||
// Token variant: re-fetch a BARE JWT via getToken() (there are no
|
||||
// credentials to log in with — on a 401/403 the interceptor below calls
|
||||
// login() again, which re-invokes getToken()). Credentials variant:
|
||||
// performLogin against /auth/login exactly as before.
|
||||
const fetchToken = this.getTokenFn
|
||||
? this.getTokenFn()
|
||||
: performLogin(this.apiUrl, this.email!, this.password!);
|
||||
this.loginPromise = fetchToken
|
||||
.then((token) => {
|
||||
// Guard against an empty/invalid token (e.g. a getToken provider that
|
||||
// resolves to "" or null): without this an empty token would set a
|
||||
// literal "Authorization: Bearer null"/"Bearer " header and every
|
||||
// request would 401 with a confusing error. Fail loudly instead.
|
||||
if (typeof token !== "string" || token.length === 0) {
|
||||
throw new Error("getToken returned an empty token");
|
||||
}
|
||||
this.token = token;
|
||||
// Identity (re)established: drop any collab token minted under a
|
||||
// previous identity so the #435 cache can never outlive it.
|
||||
this.collabTokenCache = null;
|
||||
this.client.defaults.headers.common["Authorization"] =
|
||||
`Bearer ${token}`;
|
||||
})
|
||||
.finally(() => {
|
||||
this.loginPromise = null;
|
||||
});
|
||||
}
|
||||
return this.loginPromise;
|
||||
}
|
||||
|
||||
|
||||
async ensureAuthenticated() {
|
||||
if (!this.token) {
|
||||
await this.login();
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch a collaboration token, transparently re-authenticating once on a
|
||||
* 401/403. getCollabToken() uses bare axios internally, so it is NOT covered
|
||||
* by this.client's response interceptor; this helper replicates that
|
||||
* behaviour for collab-token requests: ensure a token, try once, and on an
|
||||
* expired-token auth error perform a fresh login and retry exactly once.
|
||||
*
|
||||
* Collab-token cache (issue #435): both sources — the getCollabToken provider
|
||||
* (in-app agent) AND the REST /auth/collab-token endpoint (external MCP) — mint
|
||||
* a FRESH token per call, whose string therefore changes every op. Since the
|
||||
* live CollabSession registry keys on the token string (#400/#431 invariant 4),
|
||||
* that churned the key and defeated session reuse. So we cache the last minted
|
||||
* token per-client for readCollabTokenTtlMs() and hand it back for a burst of
|
||||
* mutations, keeping the session key stable. `forceRefresh` bypasses the cache
|
||||
* (the 401/403 reauth retry uses it, so the retry cannot be handed the same
|
||||
* stale token that just failed — otherwise reauth would be a no-op). TTL 0
|
||||
* disables the cache: exact fetch-per-call legacy behaviour.
|
||||
*/
|
||||
protected async getCollabTokenWithReauth(
|
||||
forceRefresh = false,
|
||||
): Promise<string> {
|
||||
const ttl = readCollabTokenTtlMs();
|
||||
// Serve the cached collab token while it is still fresh (identity isolation
|
||||
// is preserved: the cache is a per-instance field on a client built per
|
||||
// user/per chat request, and it is cleared on every identity change).
|
||||
if (
|
||||
!forceRefresh &&
|
||||
ttl > 0 &&
|
||||
this.collabTokenCache &&
|
||||
Date.now() - this.collabTokenCache.mintedAt < ttl
|
||||
) {
|
||||
return this.collabTokenCache.token;
|
||||
}
|
||||
|
||||
// Collab-token PROVIDER path: when a getCollabToken provider was supplied
|
||||
// (the internal agent's provenance collab token), use it instead of the
|
||||
// REST /auth/collab-token endpoint. Re-invoke it once on a 401/403 (e.g. the
|
||||
// signed token expired between content mutations in a long agent turn).
|
||||
if (this.getCollabTokenFn) {
|
||||
try {
|
||||
const token = await this.getCollabTokenFn();
|
||||
if (typeof token !== "string" || token.length === 0) {
|
||||
throw new Error("getCollabToken returned an empty token");
|
||||
}
|
||||
return this.rememberCollabToken(token, ttl);
|
||||
} catch (e) {
|
||||
// On an auth error retry EXACTLY once, forcing a refresh so the retry
|
||||
// re-invokes the provider (bypassing the cache) for a genuinely fresh
|
||||
// token. `!forceRefresh` bounds it to a single retry (no loop).
|
||||
if (this.isCollabAuthError(e) && !forceRefresh) {
|
||||
return this.getCollabTokenWithReauth(true);
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
await this.ensureAuthenticated();
|
||||
try {
|
||||
const token = await getCollabToken(this.apiUrl, this.token!);
|
||||
return this.rememberCollabToken(token, ttl);
|
||||
} catch (e) {
|
||||
// getCollabToken wraps the AxiosError in a plain Error but attaches the
|
||||
// HTTP status as `.status`, so isCollabAuthError detects an auth failure
|
||||
// via either the raw AxiosError shape OR the attached status.
|
||||
if (this.isCollabAuthError(e) && !forceRefresh) {
|
||||
// Fresh login (which clears this.token AND the collab-token cache), then
|
||||
// retry exactly once with the cache bypassed via forceRefresh.
|
||||
await this.login();
|
||||
return this.getCollabTokenWithReauth(true);
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Store a freshly minted collab token in the per-client cache (issue #435) and
|
||||
* return it unchanged. No-op write when the cache is disabled (ttl<=0) or the
|
||||
* token is empty, so a disabled cache is exact fetch-per-call legacy behaviour
|
||||
* and a bad token is never cached.
|
||||
*/
|
||||
protected rememberCollabToken(token: string, ttl: number): string {
|
||||
if (ttl > 0 && typeof token === "string" && token.length > 0) {
|
||||
this.collabTokenCache = { token, mintedAt: Date.now() };
|
||||
}
|
||||
return token;
|
||||
}
|
||||
|
||||
/**
|
||||
* True when an error carries a 401/403 — either as a raw AxiosError
|
||||
* (`error.response.status`) or as the plain-Error `.status` that
|
||||
* lib/auth-utils.getCollabToken attaches after wrapping the AxiosError.
|
||||
*/
|
||||
protected isCollabAuthError(e: unknown): boolean {
|
||||
const axiosStatus = axios.isAxiosError(e) ? e.response?.status : undefined;
|
||||
const attachedStatus = (e as any)?.status;
|
||||
return (
|
||||
axiosStatus === 401 ||
|
||||
axiosStatus === 403 ||
|
||||
attachedStatus === 401 ||
|
||||
attachedStatus === 403
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Connect to the collaboration websocket, read the live doc, apply
|
||||
* `transform`, write the result, and wait for the server to persist it —
|
||||
* WITHOUT acquiring the per-page lock.
|
||||
*
|
||||
* This mirrors collaboration.mutatePageContent EXCEPT that it does not call
|
||||
* withPageLock. It exists solely so replaceImage can hold ONE withPageLock
|
||||
* across its scan -> upload -> write sequence: the per-page mutex is NOT
|
||||
* reentrant, so calling the normal (self-locking) mutatePageContent inside an
|
||||
* outer withPageLock for the same pageId would deadlock. The caller MUST hold
|
||||
* the page lock for the whole operation; this helper assumes that invariant.
|
||||
*
|
||||
* `transform` receives the live ProseMirror doc and returns the NEW full doc
|
||||
* to write, or `null` to abort with no write. Errors thrown by `transform`
|
||||
* propagate to the caller.
|
||||
*
|
||||
* Resolves a `MutationResult { doc, verify }` mirroring mutatePageContent, so
|
||||
* every content mutator (including replaceImage) can return a verifiable
|
||||
* change report. The report is computed AFTER the atomic read->write and
|
||||
* never throws.
|
||||
*/
|
||||
protected async mutateLiveContentUnlocked(
|
||||
pageId: string,
|
||||
collabToken: string,
|
||||
transform: (liveDoc: any) => any | null,
|
||||
): Promise<MutationResult> {
|
||||
// Reuse a live CollabSession for the page (issue #400) instead of opening a
|
||||
// fresh provider per op. acquireCollabSession does NOT take the per-page
|
||||
// lock — the caller (replaceImage) already holds ONE withPageLock across its
|
||||
// scan -> upload -> write sequence, and the mutex is not reentrant, so
|
||||
// taking it here would deadlock. The synchronous read->write section and the
|
||||
// unsyncedChanges/connectionLost ack logic live in CollabSession.mutate,
|
||||
// preserved verbatim from the old inline machine (incl. the #152 structural
|
||||
// diff that keeps a live editor's cursor anchored).
|
||||
const session = await acquireCollabSession(pageId, collabToken, this.apiUrl, {
|
||||
// Only the actual 25s collab connect timeout emits this — the connect-vs-
|
||||
// unload signal; the other failure paths must NOT emit it.
|
||||
onConnectTimeout: () =>
|
||||
this.onMetricFn?.("collab_connect_timeouts_total", 1),
|
||||
});
|
||||
try {
|
||||
return await session.mutate(transform);
|
||||
} catch (e) {
|
||||
// Drop the session on any failure so the next call reconnects fresh.
|
||||
session.destroy("mutate failed");
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Generic pagination handler for Docmost API endpoints
|
||||
*/
|
||||
async paginateAll<T = any>(
|
||||
endpoint: string,
|
||||
basePayload: Record<string, any> = {},
|
||||
limit: number = 100,
|
||||
): Promise<T[]> {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
const clampedLimit = Math.max(1, Math.min(100, limit));
|
||||
|
||||
// Hard ceiling on the number of pages to fetch: guards against a server
|
||||
// that returns a perpetually-true hasNextPage (which would otherwise loop
|
||||
// forever and accumulate duplicates).
|
||||
const MAX_PAGES = 50;
|
||||
|
||||
let cursor: string | undefined;
|
||||
let allItems: T[] = [];
|
||||
let truncated = false;
|
||||
|
||||
for (let page = 0; page < MAX_PAGES; page++) {
|
||||
const payload: Record<string, any> = {
|
||||
...basePayload,
|
||||
limit: clampedLimit,
|
||||
};
|
||||
if (cursor) payload.cursor = cursor;
|
||||
|
||||
const response = await this.client.post(endpoint, payload);
|
||||
|
||||
const data = response.data;
|
||||
const items = data.data?.items || data.items || [];
|
||||
const meta = data.data?.meta || data.meta;
|
||||
|
||||
allItems = allItems.concat(items);
|
||||
|
||||
// Advance strictly via the server-issued cursor. A missing nextCursor (or
|
||||
// hasNextPage false) means we reached the end. A cursor identical to the
|
||||
// one we just sent means the server did not understand our pagination
|
||||
// param — stop instead of re-fetching page one forever and duplicating.
|
||||
const next = meta?.hasNextPage ? meta?.nextCursor : null;
|
||||
if (!next || next === cursor) {
|
||||
// If the server still reports more pages but stopped issuing a usable
|
||||
// cursor at the ceiling, flag the result as truncated below.
|
||||
if (page === MAX_PAGES - 1 && meta?.hasNextPage) truncated = true;
|
||||
break;
|
||||
}
|
||||
cursor = next;
|
||||
|
||||
// Reaching the ceiling with more pages still available means the result
|
||||
// set is truncated.
|
||||
if (page === MAX_PAGES - 1) truncated = true;
|
||||
}
|
||||
|
||||
// If the loop stopped because it hit the MAX_PAGES ceiling while the server
|
||||
// still reported more results, the result set is truncated — warn so the
|
||||
// caller is not silently handed an incomplete list.
|
||||
if (truncated) {
|
||||
console.warn(
|
||||
`paginateAll: results from "${endpoint}" truncated at the ${MAX_PAGES}-page cap; more pages exist on the server`,
|
||||
);
|
||||
}
|
||||
|
||||
return allItems;
|
||||
}
|
||||
|
||||
|
||||
/** Raw page info including the ProseMirror JSON content and slugId. */
|
||||
async getPageRaw(pageId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
const response = await this.client.post("/pages/info", { pageId });
|
||||
return response.data?.data ?? response.data;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve an agent-supplied pageId to the page's CANONICAL UUID (`page.id`),
|
||||
* so every collaboration document the MCP opens is named `page.<uuid>` — the
|
||||
* SAME name the web editor always uses (`page.${page.id}`).
|
||||
*
|
||||
* The agent commonly passes a 10-char public slugId (from URLs/listings) as
|
||||
* the pageId. The web editor opens the collab doc by UUID, but the MCP used to
|
||||
* pass that slugId straight into the collab doc name (`page.<slugId>`). For one
|
||||
* DB row that produced TWO independent Yjs documents whose debounced stores
|
||||
* clobbered each other — the agent's edit was silently lost (#260).
|
||||
*
|
||||
* A UUID input short-circuits with no network round-trip. A slugId is resolved
|
||||
* once via getPageRaw and cached (both slugId->uuid and uuid->uuid), so
|
||||
* repeated edits on the same page add no extra request.
|
||||
*/
|
||||
protected async resolvePageId(pageId: string): Promise<string> {
|
||||
if (isUuid(pageId)) return pageId;
|
||||
const cached = this.pageIdCache.get(pageId);
|
||||
if (cached) return cached;
|
||||
const data = await this.getPageRaw(pageId);
|
||||
const uuid = data?.id;
|
||||
if (typeof uuid !== "string" || !uuid) {
|
||||
throw new Error(
|
||||
`Could not resolve a canonical page id for "${pageId}"`,
|
||||
);
|
||||
}
|
||||
this.pageIdCache.set(pageId, uuid);
|
||||
return uuid;
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Page-locked write seam over collaboration.mutatePageContent. Production just
|
||||
* delegates; it exists as an overridable method so the insertFootnote wrapper
|
||||
* (transform abort-on-not-found + response shaping) can be unit-tested without
|
||||
* standing up a live Hocuspocus collab socket.
|
||||
*
|
||||
* SELF-RESOLVES the pageId to the canonical UUID (issue #449, "resolve-then-
|
||||
* lock"): every write must lock and key its CollabSession by the UUID, never a
|
||||
* raw slugId (#260). resolvePageId is cached/idempotent, so a caller that
|
||||
* already resolved pays no extra round-trip; centralizing it here means a
|
||||
* caller that reaches this seam with a raw slugId still locks correctly instead
|
||||
* of silently splitting the mutex key. withPageLock also asserts the key is a
|
||||
* UUID as a hard backstop.
|
||||
*/
|
||||
protected async mutatePage(
|
||||
pageId: string,
|
||||
collabToken: string,
|
||||
apiUrl: string,
|
||||
transform: (doc: any) => any,
|
||||
): Promise<{ doc?: any; verify?: any }> {
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
return mutatePageContent(pageUuid, collabToken, apiUrl, transform);
|
||||
}
|
||||
|
||||
/**
|
||||
* Full-document write seam over collaboration.replacePageContent. Production
|
||||
* just delegates; it exists as an overridable method so the full-doc write
|
||||
* tools (updatePageJson, copyPageContent) can have their footnote-
|
||||
* canonicalization binding unit-tested without a live Hocuspocus collab socket.
|
||||
*
|
||||
* SELF-RESOLVES the pageId to the canonical UUID (issue #449, "resolve-then-
|
||||
* lock") for the same reason as mutatePage above — the lock/CollabSession key
|
||||
* is guaranteed canonical here, not left to the caller's discipline.
|
||||
*/
|
||||
protected async replacePage(
|
||||
pageId: string,
|
||||
doc: any,
|
||||
collabToken: string,
|
||||
apiUrl: string,
|
||||
): Promise<{ doc?: any; verify?: any }> {
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
return replacePageContent(pageUuid, doc, collabToken, apiUrl);
|
||||
}
|
||||
|
||||
/**
|
||||
* Export a page to a single self-contained Docmost-flavoured markdown file:
|
||||
* meta block + body (with inline comment anchors + diagrams) + comment
|
||||
* threads. Lossless round-trip target; see importPageMarkdown for the inverse.
|
||||
*/
|
||||
}
|
||||
@@ -0,0 +1,248 @@
|
||||
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
|
||||
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
|
||||
// changed to a mixin factory. See client/context.ts for the shared base.
|
||||
import type { GConstructor, DocmostClientContext } from "./context.js";
|
||||
import {
|
||||
updatePageContentRealtime,
|
||||
replacePageContent,
|
||||
markdownToProseMirror,
|
||||
markdownToProseMirrorCanonical,
|
||||
mutatePageContent,
|
||||
assertYjsEncodable,
|
||||
MutationResult,
|
||||
} from "../lib/collaboration.js";
|
||||
import {
|
||||
replaceNodeById,
|
||||
replaceNodeByIdWithMany,
|
||||
reassignCollidingBlockIds,
|
||||
deleteNodeById,
|
||||
assertUnambiguousMatch,
|
||||
insertNodeRelative,
|
||||
insertNodesRelative,
|
||||
blockPlainText,
|
||||
buildOutline,
|
||||
getNodeByRef,
|
||||
readTable,
|
||||
insertTableRow,
|
||||
deleteTableRow,
|
||||
updateTableCell,
|
||||
findInvalidNode,
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
import {
|
||||
blockText,
|
||||
walk,
|
||||
getList,
|
||||
insertMarkerAfter,
|
||||
setCalloutRange,
|
||||
noteItem,
|
||||
mdToInlineNodes,
|
||||
commentsToFootnotes,
|
||||
canonicalizeFootnotes,
|
||||
insertInlineFootnote,
|
||||
mergeFootnoteDefinitions,
|
||||
} from "../lib/transforms.js";
|
||||
|
||||
// Public method surface of DocValidateMixin (issue #450) — a NAMED type so the factory
|
||||
// return type is expressible in the emitted .d.ts (the anonymous mixin class
|
||||
// carries the base's protected shared state, which would otherwise trip TS4094).
|
||||
// Derived from the class below; `implements IDocValidateMixin` fails to compile on drift.
|
||||
export interface IDocValidateMixin {
|
||||
}
|
||||
|
||||
export function DocValidateMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IDocValidateMixin> & TBase {
|
||||
abstract class DocValidateMixin extends Base implements IDocValidateMixin {
|
||||
/**
|
||||
* Validate a URL string against a scheme allowlist for a given context.
|
||||
*
|
||||
* The markdown link path enforces safe schemes via TipTap, but the raw
|
||||
* JSON path (updatePageJson) bypasses that — so this is the sanitization
|
||||
* choke point for ProseMirror JSON written directly by the caller.
|
||||
*
|
||||
* - "link": reject javascript:, vbscript:, data: (any scheme that can
|
||||
* execute or smuggle script when the href is clicked).
|
||||
* - "src": allow only http(s):, mailto:, /api/files paths, or a
|
||||
* scheme-less relative/absolute path; reject
|
||||
* javascript:/vbscript:/data:/file:.
|
||||
*/
|
||||
protected isSafeUrl(url: unknown, context: "link" | "src"): boolean {
|
||||
if (typeof url !== "string") return false;
|
||||
const trimmed = url.trim();
|
||||
if (trimmed === "") return true; // empty href/src is harmless
|
||||
|
||||
// Extract a leading "scheme:" if present. A scheme must start with a
|
||||
// letter and contain only letters/digits/+/-/. before the colon. Strip
|
||||
// whitespace and ASCII control chars first so a tab/newline embedded in
|
||||
// the scheme cannot smuggle a dangerous scheme past the check.
|
||||
const cleaned = trimmed.replace(/[\s\x00-\x1f]+/g, "");
|
||||
const schemeMatch = /^([a-zA-Z][a-zA-Z0-9+.-]*):/.exec(cleaned);
|
||||
const scheme = schemeMatch ? schemeMatch[1].toLowerCase() : null;
|
||||
|
||||
const dangerous = new Set(["javascript", "vbscript", "data", "file"]);
|
||||
|
||||
if (context === "link") {
|
||||
if (scheme === null) return true; // relative/anchor link is fine
|
||||
// For links, data: is also blocked (can carry script payloads).
|
||||
return !new Set(["javascript", "vbscript", "data"]).has(scheme);
|
||||
}
|
||||
|
||||
// context === "src"
|
||||
if (scheme === null) return true; // relative/absolute path (incl. /api/files)
|
||||
if (dangerous.has(scheme)) return false;
|
||||
return scheme === "http" || scheme === "https" || scheme === "mailto";
|
||||
}
|
||||
|
||||
/**
|
||||
* Recursively walk a ProseMirror doc and reject any unsafe URL on a link
|
||||
* mark href or on a media node's src/url. Media nodes covered: image,
|
||||
* attachment, video, plus embed (rendered as an iframe), youtube, drawio
|
||||
* and excalidraw — all of which carry a user-controlled URL that Docmost
|
||||
* renders. Throws a clear error on the first violation. A max-depth guard
|
||||
* turns an over-deep document into a clean error instead of a RangeError
|
||||
* stack overflow.
|
||||
*/
|
||||
protected validateDocUrls(node: any, depth: number = 0): void {
|
||||
const MAX_DEPTH = 200;
|
||||
if (depth > MAX_DEPTH) {
|
||||
throw new Error(
|
||||
`document nesting exceeds the maximum depth of ${MAX_DEPTH}`,
|
||||
);
|
||||
}
|
||||
if (!node || typeof node !== "object") return;
|
||||
|
||||
// Link marks on text nodes: validate the href.
|
||||
if (Array.isArray(node.marks)) {
|
||||
for (const mark of node.marks) {
|
||||
if (mark && mark.type === "link" && mark.attrs) {
|
||||
if (!this.isSafeUrl(mark.attrs.href, "link")) {
|
||||
throw new Error(`unsafe link href rejected: "${mark.attrs.href}"`);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Media nodes: validate src/url against the stricter src allowlist.
|
||||
// embed renders as an iframe (highest risk); youtube/drawio/excalidraw
|
||||
// likewise carry a user-controlled URL Docmost renders, so they get the
|
||||
// same scheme check as image/attachment/video.
|
||||
if (
|
||||
node.type === "image" ||
|
||||
node.type === "attachment" ||
|
||||
node.type === "video" ||
|
||||
node.type === "embed" ||
|
||||
node.type === "youtube" ||
|
||||
node.type === "drawio" ||
|
||||
node.type === "excalidraw" ||
|
||||
node.type === "audio" ||
|
||||
node.type === "pdf"
|
||||
) {
|
||||
const attrs = node.attrs || {};
|
||||
for (const key of ["src", "url"]) {
|
||||
if (attrs[key] != null && !this.isSafeUrl(attrs[key], "src")) {
|
||||
throw new Error(
|
||||
`unsafe ${node.type} ${key} rejected: "${attrs[key]}"`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (Array.isArray(node.content)) {
|
||||
for (const child of node.content) {
|
||||
this.validateDocUrls(child, depth + 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Recursively validate the STRUCTURE of a ProseMirror node (reuses the
|
||||
* recursion shape of validateDocUrls). Every node must be an object with a
|
||||
* string `type`; when present, `content` must be an array, `marks` must be
|
||||
* an array of objects each with a string `type`, and a text node's `text`
|
||||
* must be a string. Throws a clear "invalid ProseMirror document" error on
|
||||
* the first violation. A max-depth guard turns an over-deep document into a
|
||||
* clean error instead of a RangeError stack overflow.
|
||||
*/
|
||||
protected validateDocStructure(node: any, depth: number = 0): void {
|
||||
const MAX_DEPTH = 200;
|
||||
if (depth > MAX_DEPTH) {
|
||||
throw new Error(
|
||||
`invalid ProseMirror document: nesting exceeds the maximum depth of ${MAX_DEPTH}`,
|
||||
);
|
||||
}
|
||||
if (!node || typeof node !== "object" || typeof node.type !== "string") {
|
||||
throw new Error(
|
||||
"invalid ProseMirror document: every node must be an object with a string `type`",
|
||||
);
|
||||
}
|
||||
if (
|
||||
"text" in node &&
|
||||
node.type === "text" &&
|
||||
typeof node.text !== "string"
|
||||
) {
|
||||
throw new Error(
|
||||
"invalid ProseMirror document: a text node must have a string `text`",
|
||||
);
|
||||
}
|
||||
if (node.marks !== undefined) {
|
||||
if (!Array.isArray(node.marks)) {
|
||||
throw new Error(
|
||||
"invalid ProseMirror document: `marks` must be an array",
|
||||
);
|
||||
}
|
||||
for (const mark of node.marks) {
|
||||
if (
|
||||
!mark ||
|
||||
typeof mark !== "object" ||
|
||||
typeof mark.type !== "string"
|
||||
) {
|
||||
throw new Error(
|
||||
"invalid ProseMirror document: every mark must be an object with a string `type`",
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (node.content !== undefined) {
|
||||
if (!Array.isArray(node.content)) {
|
||||
throw new Error(
|
||||
"invalid ProseMirror document: `content` must be an array when present",
|
||||
);
|
||||
}
|
||||
for (const child of node.content) {
|
||||
this.validateDocStructure(child, depth + 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Pre-write SHAPE gate (#409). Walk the WHOLE node tree with the shared
|
||||
* `findInvalidNode` and throw a rich, path-anchored error the instant a nested
|
||||
* node has an absent/unknown `type` (or an unknown mark) — the exact shape that
|
||||
* otherwise surfaces DEEP in the Yjs encode as the cryptic
|
||||
* `Unknown node type: undefined`, but only AFTER a collab session was opened
|
||||
* and a page lock taken. Calling this BEFORE `getCollabTokenWithReauth` /
|
||||
* `mutatePageContent` fails fast: no collab connection, no lock, deterministic
|
||||
* message. `op` names the tool for the message prefix (e.g. "patchNode").
|
||||
*
|
||||
* `findInvalidNode` derives its "known type" set from the very same
|
||||
* `docmostExtensions` the encode path uses, so a node this gate accepts is one
|
||||
* the encoder will accept too.
|
||||
*/
|
||||
protected assertValidNodeShape(op: string, node: any): void {
|
||||
const bad = findInvalidNode(node);
|
||||
if (bad) {
|
||||
throw new Error(`${op}: invalid node — ${bad.summary}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace page content with a raw ProseMirror JSON document (lossless) and/or
|
||||
* update its title. Both `doc` and `title` are optional, but at least one must
|
||||
* be supplied:
|
||||
* - `doc` provided -> validate + full-overwrite the body (and update the
|
||||
* title too when `title` is also given).
|
||||
* - `doc` omitted, `title` given -> title-only update; the body is NOT
|
||||
* touched/resent (no collab write happens).
|
||||
* - neither given -> throws (nothing to update).
|
||||
*/
|
||||
}
|
||||
return DocValidateMixin;
|
||||
}
|
||||
@@ -0,0 +1,707 @@
|
||||
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
|
||||
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
|
||||
// changed to a mixin factory. See client/context.ts for the shared base.
|
||||
import type { GConstructor, DocmostClientContext } from "./context.js";
|
||||
import { parseCells as parseDrawioCells } from "../lib/drawio-xml.js";
|
||||
import {
|
||||
replaceNodeById,
|
||||
replaceNodeByIdWithMany,
|
||||
reassignCollidingBlockIds,
|
||||
deleteNodeById,
|
||||
assertUnambiguousMatch,
|
||||
insertNodeRelative,
|
||||
insertNodesRelative,
|
||||
blockPlainText,
|
||||
buildOutline,
|
||||
getNodeByRef,
|
||||
readTable,
|
||||
insertTableRow,
|
||||
deleteTableRow,
|
||||
updateTableCell,
|
||||
findInvalidNode,
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
import {
|
||||
prepareModel,
|
||||
decodeDrawioSvg,
|
||||
buildDrawioSvg,
|
||||
mxHash,
|
||||
normalizeXml,
|
||||
countUserCells,
|
||||
} from "../lib/drawio-xml.js";
|
||||
import { renderDiagramShapes } from "../lib/drawio-preview.js";
|
||||
import { applyElkLayout } from "../lib/drawio-layout.js";
|
||||
import {
|
||||
buildFromGraph,
|
||||
type Graph,
|
||||
type LayoutMode as GraphLayoutMode,
|
||||
} from "../lib/drawio-graph.js";
|
||||
import { applyCellOps, type CellOp } from "../lib/drawio-cell-ops.js";
|
||||
import { mermaidToGraph } from "../lib/drawio-mermaid.js";
|
||||
import {
|
||||
blockText,
|
||||
walk,
|
||||
getList,
|
||||
insertMarkerAfter,
|
||||
setCalloutRange,
|
||||
noteItem,
|
||||
mdToInlineNodes,
|
||||
commentsToFootnotes,
|
||||
canonicalizeFootnotes,
|
||||
insertInlineFootnote,
|
||||
mergeFootnoteDefinitions,
|
||||
} from "../lib/transforms.js";
|
||||
|
||||
// Public method surface of DrawioMixin (issue #450) — a NAMED type so the factory
|
||||
// return type is expressible in the emitted .d.ts (the anonymous mixin class
|
||||
// carries the base's protected shared state, which would otherwise trip TS4094).
|
||||
// Derived from the class below; `implements IDrawioMixin` fails to compile on drift.
|
||||
export interface IDrawioMixin {
|
||||
drawioGet(pageId: string, node: string, format?: "xml" | "svg"): Promise<{ pageId: string; nodeId: string; format: "xml" | "svg"; content: string; meta: { attachmentId: string | null; title: string | null; width: number | null; height: number | null; cellCount: number; hash: string; }; }>;
|
||||
drawioCreate(pageId: string, where: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }, xml: string, title?: string, layout?: "elk"): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; verify?: any; }>;
|
||||
drawioUpdate(pageId: string, node: string, xml: string, baseHash: string, layout?: "elk"): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; verify?: any; }>;
|
||||
drawioEditCells(pageId: string, node: string, operations: CellOp[], baseHash: string): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; verify?: any; }>;
|
||||
drawioFromGraph(pageId: string, where: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }, graph: Graph, direction?: "LR" | "RL" | "TB" | "BT", preset?: string, layout?: GraphLayoutMode, node?: string): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; iconsResolved: number; iconsMissing: string[]; verify?: any; }>;
|
||||
drawioFromMermaid(pageId: string, where: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }, mermaid: string, preset?: string): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; iconsResolved: number; iconsMissing: string[]; verify?: any; }>;
|
||||
}
|
||||
|
||||
export function DrawioMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IDrawioMixin> & TBase {
|
||||
abstract class DrawioMixin extends Base implements IDrawioMixin {
|
||||
/**
|
||||
* Resolve a drawio node on a page by `attrs.id` or `#<index>` and return the
|
||||
* node plus its ref. Throws a clear error if the ref does not resolve to a
|
||||
* drawio node.
|
||||
*/
|
||||
protected async resolveDrawioNode(
|
||||
pageId: string,
|
||||
node: string,
|
||||
): Promise<{ node: any; ref: string }> {
|
||||
const data = await this.getPageRaw(pageId);
|
||||
const hit = getNodeByRef(
|
||||
data.content ?? { type: "doc", content: [] },
|
||||
node,
|
||||
);
|
||||
if (!hit) {
|
||||
throw new Error(
|
||||
`drawio: no node found for "${node}" on page ${pageId} (use the drawio node's attrs.id or "#<index>" from getOutline)`,
|
||||
);
|
||||
}
|
||||
if (hit.type !== "drawio") {
|
||||
throw new Error(
|
||||
`drawio: node "${node}" on page ${pageId} is a ${hit.type}, not a drawio diagram`,
|
||||
);
|
||||
}
|
||||
return { node: hit.node, ref: node };
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a drawio diagram as mxGraph XML (default) or as the raw `.drawio.svg`.
|
||||
* Runs the decode chain (base64/entity content= → drawio file → nested XML or
|
||||
* pako-inflated compressed <diagram>). The returned `hash` is the
|
||||
* optimistic-lock key for drawioUpdate.
|
||||
*/
|
||||
async drawioGet(
|
||||
pageId: string,
|
||||
node: string,
|
||||
format: "xml" | "svg" = "xml",
|
||||
): Promise<{
|
||||
pageId: string;
|
||||
nodeId: string;
|
||||
format: "xml" | "svg";
|
||||
content: string;
|
||||
meta: {
|
||||
attachmentId: string | null;
|
||||
title: string | null;
|
||||
width: number | null;
|
||||
height: number | null;
|
||||
cellCount: number;
|
||||
hash: string;
|
||||
};
|
||||
}> {
|
||||
await this.ensureAuthenticated();
|
||||
const { node: drawio } = await this.resolveDrawioNode(pageId, node);
|
||||
const attrs = drawio.attrs || {};
|
||||
const src = attrs.src;
|
||||
if (!src) {
|
||||
throw new Error(
|
||||
`drawio: node "${node}" on page ${pageId} has no src to read`,
|
||||
);
|
||||
}
|
||||
const svg = await this.fetchAttachmentText(src);
|
||||
const modelXml = decodeDrawioSvg(svg);
|
||||
const meta = {
|
||||
attachmentId: attrs.attachmentId ?? null,
|
||||
title: attrs.title ?? null,
|
||||
width: attrs.width != null ? Number(attrs.width) : null,
|
||||
height: attrs.height != null ? Number(attrs.height) : null,
|
||||
cellCount: countUserCells(modelXml),
|
||||
hash: mxHash(modelXml),
|
||||
};
|
||||
return {
|
||||
pageId,
|
||||
nodeId: attrs.id ?? node,
|
||||
format,
|
||||
content: format === "svg" ? svg : normalizeXml(modelXml),
|
||||
meta,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a drawio diagram from mxGraph XML: lint → schematic SVG preview
|
||||
* (pure TS) → build the `.drawio.svg` (createDrawioSvg contract) → create the
|
||||
* attachment → insert a `drawio` node before/after an anchor or appended.
|
||||
* `xml` is a bare `<mxGraphModel>` or a list of `<mxCell>` (the server wraps
|
||||
* it and adds the id=0/id=1 sentinels).
|
||||
*/
|
||||
async drawioCreate(
|
||||
pageId: string,
|
||||
where: {
|
||||
position: "before" | "after" | "append";
|
||||
anchorNodeId?: string;
|
||||
anchorText?: string;
|
||||
},
|
||||
xml: string,
|
||||
title?: string,
|
||||
layout?: "elk",
|
||||
): Promise<{
|
||||
success: boolean;
|
||||
nodeId: string;
|
||||
attachmentId: string;
|
||||
warnings: string[];
|
||||
verify?: any;
|
||||
}> {
|
||||
await this.ensureAuthenticated();
|
||||
if (
|
||||
!where ||
|
||||
(where.position !== "before" &&
|
||||
where.position !== "after" &&
|
||||
where.position !== "append")
|
||||
) {
|
||||
throw new Error(
|
||||
'drawioCreate: `where.position` must be one of "before", "after", "append"',
|
||||
);
|
||||
}
|
||||
if (where.position === "before" || where.position === "after") {
|
||||
const hasId =
|
||||
typeof where.anchorNodeId === "string" && where.anchorNodeId.length > 0;
|
||||
const hasText =
|
||||
typeof where.anchorText === "string" && where.anchorText.length > 0;
|
||||
if (hasId === hasText) {
|
||||
throw new Error(
|
||||
`drawioCreate: position "${where.position}" requires exactly one of anchorNodeId or anchorText`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Optional server-side ELK auto-layout: the model declares structure with
|
||||
// rough coords, ELK computes the pixels (best-effort — returns the input
|
||||
// unchanged on any layout failure).
|
||||
const laidOutXml = layout === "elk" ? await applyElkLayout(xml) : xml;
|
||||
// Pre-write pipeline (throws a structured DrawioLintError on any violation).
|
||||
const prepared = prepareModel(laidOutXml);
|
||||
const inner = renderDiagramShapes(prepared.cells, prepared.bbox);
|
||||
const diagramTitle = title || "Page-1";
|
||||
const svg = buildDrawioSvg(prepared.modelXml, inner, prepared.bbox, diagramTitle);
|
||||
|
||||
const att = await this.uploadAttachmentBuffer(
|
||||
pageId,
|
||||
Buffer.from(svg, "utf-8"),
|
||||
"diagram.drawio.svg",
|
||||
"image/svg+xml",
|
||||
);
|
||||
|
||||
// NOTE: no `id` attribute is set here. The vendored `drawio` node schema
|
||||
// (diagramAttributes) declares no `id`, so any block id would be silently
|
||||
// dropped by PMNode.fromJSON on save and the returned handle would fail to
|
||||
// resolve. The addressable handle is the node's "#<index>" (like image/table
|
||||
// nodes), computed after the insert below.
|
||||
const drawioNode: any = {
|
||||
type: "drawio",
|
||||
attrs: {
|
||||
src: `/api/files/${att.id}/${att.fileName}`,
|
||||
attachmentId: att.id,
|
||||
width: prepared.bbox.width,
|
||||
height: prepared.bbox.height,
|
||||
align: "center",
|
||||
},
|
||||
};
|
||||
if (title) drawioNode.attrs.title = title;
|
||||
// Reuse the existing URL trust boundary (rejects unsafe src schemes).
|
||||
this.validateDocUrls(drawioNode);
|
||||
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
let inserted = false;
|
||||
let insertedIndex = -1;
|
||||
const mutation = await this.mutatePage(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
inserted = false;
|
||||
insertedIndex = -1;
|
||||
const { doc: nd, inserted: ins } = insertNodeRelative(
|
||||
liveDoc,
|
||||
drawioNode,
|
||||
where,
|
||||
);
|
||||
inserted = ins;
|
||||
if (!inserted) return null; // anchor not found -> skip the write
|
||||
// Locate the freshly-inserted node to derive its "#<index>" handle. The
|
||||
// just-uploaded attachmentId is unique, so it identifies our node.
|
||||
if (Array.isArray(nd.content)) {
|
||||
insertedIndex = nd.content.findIndex(
|
||||
(b: any) =>
|
||||
b &&
|
||||
b.type === "drawio" &&
|
||||
b.attrs &&
|
||||
b.attrs.attachmentId === att.id,
|
||||
);
|
||||
}
|
||||
return nd;
|
||||
},
|
||||
);
|
||||
|
||||
if (!inserted) {
|
||||
const anchorDesc = where.anchorNodeId
|
||||
? `anchorNodeId "${where.anchorNodeId}"`
|
||||
: `anchorText "${where.anchorText}"`;
|
||||
throw new Error(
|
||||
`drawioCreate: anchor not found (${anchorDesc}) on page ${pageId}. The diagram attachment ${att.id} is now an unreferenced orphan.`,
|
||||
);
|
||||
}
|
||||
|
||||
if (insertedIndex < 0) {
|
||||
// The node was inserted nested (e.g. inside a callout/table cell via an
|
||||
// anchor), where "#<index>" — which addresses only top-level blocks —
|
||||
// cannot reference it. drawio nodes carry no persisted id, so there is no
|
||||
// stable handle for a nested diagram.
|
||||
throw new Error(
|
||||
`drawioCreate: the diagram was inserted on page ${pageId} but not as a ` +
|
||||
`top-level block, so it has no addressable "#<index>" handle. Anchor ` +
|
||||
`on a top-level block (or append) so the diagram can be re-read.`,
|
||||
);
|
||||
}
|
||||
|
||||
// The returned handle is POSITIONAL ("#<index>"): valid for the immediate
|
||||
// create -> get/update flow, but re-resolve via getOutline if the document
|
||||
// structure changes (blocks added/removed before it shift the index).
|
||||
const nodeId = `#${insertedIndex}`;
|
||||
|
||||
return {
|
||||
success: true,
|
||||
nodeId,
|
||||
attachmentId: att.id,
|
||||
warnings: prepared.warnings,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Full-replacement update of a drawio diagram. `baseHash` is MANDATORY: it is
|
||||
* compared against the hash of the diagram's CURRENT XML (from drawioGet);
|
||||
* any mismatch means a human or another agent edited the diagram after the
|
||||
* read, so the write is refused with a conflict error. On success the new
|
||||
* `.drawio.svg` is uploaded as a FRESH attachment (in-place byte overwrite is
|
||||
* avoided — some Docmost versions corrupt an attachment on overwrite, exactly
|
||||
* as replaceImage documents) and the node is repointed with new dimensions.
|
||||
*/
|
||||
async drawioUpdate(
|
||||
pageId: string,
|
||||
node: string,
|
||||
xml: string,
|
||||
baseHash: string,
|
||||
layout?: "elk",
|
||||
): Promise<{
|
||||
success: boolean;
|
||||
nodeId: string;
|
||||
attachmentId: string;
|
||||
warnings: string[];
|
||||
verify?: any;
|
||||
}> {
|
||||
await this.ensureAuthenticated();
|
||||
if (typeof baseHash !== "string" || baseHash.length === 0) {
|
||||
throw new Error(
|
||||
"drawioUpdate: baseHash is mandatory — read the diagram with drawioGet first and pass back its meta.hash",
|
||||
);
|
||||
}
|
||||
|
||||
// Resolve the node and read the CURRENT diagram to enforce the optimistic
|
||||
// lock before doing any write or upload.
|
||||
const { node: drawio, ref } = await this.resolveDrawioNode(pageId, node);
|
||||
const oldAttrs = drawio.attrs || {};
|
||||
const oldSrc = oldAttrs.src;
|
||||
// The returned handle is the caller-supplied reference. drawio nodes carry
|
||||
// no persisted id, so `ref` (an "#<index>" or a rare legacy attrs.id) is the
|
||||
// honest identifier to hand back.
|
||||
const nodeId = oldAttrs.id ?? ref;
|
||||
if (!oldSrc) {
|
||||
throw new Error(
|
||||
`drawioUpdate: node "${node}" on page ${pageId} has no src to compare against`,
|
||||
);
|
||||
}
|
||||
const currentSvg = await this.fetchAttachmentText(oldSrc);
|
||||
const currentHash = mxHash(decodeDrawioSvg(currentSvg));
|
||||
if (currentHash !== baseHash) {
|
||||
throw new Error(
|
||||
`drawioUpdate: conflict — the diagram changed since it was read ` +
|
||||
`(baseHash ${baseHash} != current ${currentHash}). Re-read it with drawioGet and retry.`,
|
||||
);
|
||||
}
|
||||
|
||||
// Optional server-side ELK auto-layout (best-effort; see drawioCreate).
|
||||
const laidOutXml = layout === "elk" ? await applyElkLayout(xml) : xml;
|
||||
// Pipeline for the new content (throws a structured DrawioLintError).
|
||||
const prepared = prepareModel(laidOutXml);
|
||||
const inner = renderDiagramShapes(prepared.cells, prepared.bbox);
|
||||
const diagramTitle = oldAttrs.title || "Page-1";
|
||||
const svg = buildDrawioSvg(prepared.modelXml, inner, prepared.bbox, diagramTitle);
|
||||
|
||||
const att = await this.uploadAttachmentBuffer(
|
||||
pageId,
|
||||
Buffer.from(svg, "utf-8"),
|
||||
"diagram.drawio.svg",
|
||||
"image/svg+xml",
|
||||
);
|
||||
const newSrc = `/api/files/${att.id}/${att.fileName}`;
|
||||
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
let repointed = 0;
|
||||
const repoint = (n: any) => {
|
||||
n.attrs = {
|
||||
...n.attrs,
|
||||
src: newSrc,
|
||||
attachmentId: att.id,
|
||||
width: prepared.bbox.width,
|
||||
height: prepared.bbox.height,
|
||||
};
|
||||
repointed++;
|
||||
};
|
||||
|
||||
const mutation = await this.mutatePage(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
repointed = 0;
|
||||
const doc =
|
||||
liveDoc && liveDoc.type === "doc"
|
||||
? liveDoc
|
||||
: { type: "doc", content: [] };
|
||||
if (!Array.isArray(doc.content)) doc.content = [];
|
||||
// Repoint ONLY the resolved node — never every node that happens to
|
||||
// share this attachmentId (a copied diagram is two nodes with one
|
||||
// attachmentId; keying on it would clobber both). Re-resolve the same
|
||||
// handle against the live doc and walk to its exact position.
|
||||
const hit = getNodeByRef(doc, ref);
|
||||
if (!hit || hit.type !== "drawio") return null; // vanished/changed -> skip
|
||||
let target: any = doc;
|
||||
for (const idx of hit.path) {
|
||||
if (!target || !Array.isArray(target.content)) {
|
||||
target = null;
|
||||
break;
|
||||
}
|
||||
target = target.content[idx];
|
||||
}
|
||||
if (!target || target.type !== "drawio") return null;
|
||||
repoint(target);
|
||||
if (repointed === 0) return null; // node vanished concurrently -> skip
|
||||
return doc;
|
||||
},
|
||||
);
|
||||
|
||||
if (repointed === 0) {
|
||||
return {
|
||||
success: true,
|
||||
nodeId,
|
||||
attachmentId: att.id,
|
||||
warnings: [
|
||||
...prepared.warnings,
|
||||
"target drawio node was removed concurrently; uploaded attachment is unreferenced",
|
||||
],
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
nodeId,
|
||||
attachmentId: att.id,
|
||||
warnings: prepared.warnings,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
// --- draw.io high-level semantic tools (issue #425) ---
|
||||
|
||||
/**
|
||||
* ID-based targeted edits of an existing drawio diagram (add / update / delete
|
||||
* cells) instead of resending the whole XML. Reads the CURRENT diagram, checks
|
||||
* the optimistic lock (`baseHash` is MANDATORY, exactly as drawioUpdate), applies
|
||||
* the operations to the parsed model (a `delete` CASCADES to container children
|
||||
* and to every edge whose source/target is deleted), then runs the SAME #423
|
||||
* pipeline as drawioUpdate (lint + quality warnings -> preview -> attachment ->
|
||||
* repoint the node). Ids are stable so diffs stay meaningful across edits.
|
||||
*/
|
||||
|
||||
// --- draw.io high-level semantic tools (issue #425) ---
|
||||
|
||||
/**
|
||||
* ID-based targeted edits of an existing drawio diagram (add / update / delete
|
||||
* cells) instead of resending the whole XML. Reads the CURRENT diagram, checks
|
||||
* the optimistic lock (`baseHash` is MANDATORY, exactly as drawioUpdate), applies
|
||||
* the operations to the parsed model (a `delete` CASCADES to container children
|
||||
* and to every edge whose source/target is deleted), then runs the SAME #423
|
||||
* pipeline as drawioUpdate (lint + quality warnings -> preview -> attachment ->
|
||||
* repoint the node). Ids are stable so diffs stay meaningful across edits.
|
||||
*/
|
||||
async drawioEditCells(
|
||||
pageId: string,
|
||||
node: string,
|
||||
operations: CellOp[],
|
||||
baseHash: string,
|
||||
): Promise<{
|
||||
success: boolean;
|
||||
nodeId: string;
|
||||
attachmentId: string;
|
||||
warnings: string[];
|
||||
verify?: any;
|
||||
}> {
|
||||
await this.ensureAuthenticated();
|
||||
if (typeof baseHash !== "string" || baseHash.length === 0) {
|
||||
throw new Error(
|
||||
"drawioEditCells: baseHash is mandatory — read the diagram with drawioGet first and pass back its meta.hash",
|
||||
);
|
||||
}
|
||||
if (!Array.isArray(operations) || operations.length === 0) {
|
||||
throw new Error(
|
||||
"drawioEditCells: operations must be a non-empty array of { op, ... }",
|
||||
);
|
||||
}
|
||||
|
||||
const { node: drawio, ref } = await this.resolveDrawioNode(pageId, node);
|
||||
const oldAttrs = drawio.attrs || {};
|
||||
const oldSrc = oldAttrs.src;
|
||||
const nodeId = oldAttrs.id ?? ref;
|
||||
if (!oldSrc) {
|
||||
throw new Error(
|
||||
`drawioEditCells: node "${node}" on page ${pageId} has no src to edit`,
|
||||
);
|
||||
}
|
||||
const currentSvg = await this.fetchAttachmentText(oldSrc);
|
||||
const currentModel = decodeDrawioSvg(currentSvg);
|
||||
const currentHash = mxHash(currentModel);
|
||||
if (currentHash !== baseHash) {
|
||||
throw new Error(
|
||||
`drawioEditCells: conflict — the diagram changed since it was read ` +
|
||||
`(baseHash ${baseHash} != current ${currentHash}). Re-read it with drawioGet and retry.`,
|
||||
);
|
||||
}
|
||||
|
||||
// Apply the operations to the parsed model, then run the standard pipeline.
|
||||
const editedModel = applyCellOps(currentModel, operations);
|
||||
const prepared = prepareModel(editedModel);
|
||||
const inner = renderDiagramShapes(prepared.cells, prepared.bbox);
|
||||
const diagramTitle = oldAttrs.title || "Page-1";
|
||||
const svg = buildDrawioSvg(prepared.modelXml, inner, prepared.bbox, diagramTitle);
|
||||
|
||||
const att = await this.uploadAttachmentBuffer(
|
||||
pageId,
|
||||
Buffer.from(svg, "utf-8"),
|
||||
"diagram.drawio.svg",
|
||||
"image/svg+xml",
|
||||
);
|
||||
const newSrc = `/api/files/${att.id}/${att.fileName}`;
|
||||
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
let repointed = 0;
|
||||
const mutation = await this.mutatePage(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
repointed = 0;
|
||||
const doc =
|
||||
liveDoc && liveDoc.type === "doc" ? liveDoc : { type: "doc", content: [] };
|
||||
if (!Array.isArray(doc.content)) doc.content = [];
|
||||
const hit = getNodeByRef(doc, ref);
|
||||
if (!hit || hit.type !== "drawio") return null;
|
||||
let target: any = doc;
|
||||
for (const idx of hit.path) {
|
||||
if (!target || !Array.isArray(target.content)) {
|
||||
target = null;
|
||||
break;
|
||||
}
|
||||
target = target.content[idx];
|
||||
}
|
||||
if (!target || target.type !== "drawio") return null;
|
||||
target.attrs = {
|
||||
...target.attrs,
|
||||
src: newSrc,
|
||||
attachmentId: att.id,
|
||||
width: prepared.bbox.width,
|
||||
height: prepared.bbox.height,
|
||||
};
|
||||
repointed++;
|
||||
return doc;
|
||||
},
|
||||
);
|
||||
|
||||
if (repointed === 0) {
|
||||
return {
|
||||
success: true,
|
||||
nodeId,
|
||||
attachmentId: att.id,
|
||||
warnings: [
|
||||
...prepared.warnings,
|
||||
"target drawio node was removed concurrently; uploaded attachment is unreferenced",
|
||||
],
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
return {
|
||||
success: true,
|
||||
nodeId,
|
||||
attachmentId: att.id,
|
||||
warnings: prepared.warnings,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The main high-level tool: build a diagram from a SEMANTIC graph (nodes with
|
||||
* a `kind`/`icon`, groups, edges) — the model never supplies coordinates or
|
||||
* style strings. The server resolves icons via the shape catalog (#424),
|
||||
* assigns palette colors from the preset, runs ELK layered layout (honouring
|
||||
* `direction` and the `layer`/`sameLayerAs`/`pinned` hints and compound groups),
|
||||
* and assembles linter-clean XML, then inserts it through the SAME create
|
||||
* pipeline as drawioCreate. `layout:"incremental"` is only meaningful when a
|
||||
* target `node` is given (it preserves that diagram's existing coordinates and
|
||||
* places only new cells); on a fresh insert it behaves like "full".
|
||||
*/
|
||||
async drawioFromGraph(
|
||||
pageId: string,
|
||||
where: {
|
||||
position: "before" | "after" | "append";
|
||||
anchorNodeId?: string;
|
||||
anchorText?: string;
|
||||
},
|
||||
graph: Graph,
|
||||
direction?: "LR" | "RL" | "TB" | "BT",
|
||||
preset?: string,
|
||||
layout?: GraphLayoutMode,
|
||||
node?: string,
|
||||
): Promise<{
|
||||
success: boolean;
|
||||
nodeId: string;
|
||||
attachmentId: string;
|
||||
warnings: string[];
|
||||
iconsResolved: number;
|
||||
iconsMissing: string[];
|
||||
verify?: any;
|
||||
}> {
|
||||
await this.ensureAuthenticated();
|
||||
// Direction/preset supplied as separate params override the graph fields so
|
||||
// both the flat tool schema and an inline graph can set them.
|
||||
const merged: Graph = {
|
||||
...graph,
|
||||
direction: direction ?? graph.direction,
|
||||
preset: preset ?? graph.preset,
|
||||
};
|
||||
const mode: GraphLayoutMode = layout ?? "full";
|
||||
|
||||
// Incremental into an EXISTING node: read its coords so ELK preserves them,
|
||||
// and keep the full existing model so incremental MERGES (never drops) any
|
||||
// cell the new graph doesn't re-list.
|
||||
let existingCoords: Map<string, { x: number; y: number }> | undefined;
|
||||
let existingModelXml: string | undefined;
|
||||
let editExisting = false;
|
||||
let baseHash: string | undefined;
|
||||
if (node && (mode === "incremental" || mode === "none")) {
|
||||
const { node: drawio } = await this.resolveDrawioNode(pageId, node);
|
||||
const src = (drawio.attrs || {}).src;
|
||||
if (src) {
|
||||
const svg = await this.fetchAttachmentText(src);
|
||||
const model = decodeDrawioSvg(svg);
|
||||
baseHash = mxHash(model);
|
||||
existingModelXml = model;
|
||||
existingCoords = new Map();
|
||||
for (const c of parseDrawioCells(model)) {
|
||||
if (c.vertex && c.geometry.x != null && c.geometry.y != null) {
|
||||
existingCoords.set(c.id, { x: c.geometry.x, y: c.geometry.y });
|
||||
}
|
||||
}
|
||||
editExisting = true;
|
||||
}
|
||||
}
|
||||
|
||||
const built = await buildFromGraph(
|
||||
merged,
|
||||
mode,
|
||||
existingCoords,
|
||||
existingModelXml,
|
||||
);
|
||||
|
||||
if (editExisting && node && baseHash) {
|
||||
// Re-target the existing diagram: replace it with the assembled model.
|
||||
const res = await this.drawioUpdate(pageId, node, built.modelXml, baseHash);
|
||||
return {
|
||||
...res,
|
||||
iconsResolved: built.iconsResolved,
|
||||
iconsMissing: built.iconsMissing,
|
||||
};
|
||||
}
|
||||
|
||||
const res = await this.drawioCreate(pageId, where, built.modelXml);
|
||||
return {
|
||||
...res,
|
||||
iconsResolved: built.iconsResolved,
|
||||
iconsMissing: built.iconsMissing,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Convert a Mermaid `flowchart` to a redactable draw.io diagram via a PURE
|
||||
* parser (no Electron / draw.io CLI): mermaid text -> graph-JSON -> the
|
||||
* drawioFromGraph pipeline. Only `flowchart`/`graph` is supported (the most
|
||||
* common wiki case); other diagram types throw a clear error so the model can
|
||||
* fall back to drawioFromGraph.
|
||||
*/
|
||||
async drawioFromMermaid(
|
||||
pageId: string,
|
||||
where: {
|
||||
position: "before" | "after" | "append";
|
||||
anchorNodeId?: string;
|
||||
anchorText?: string;
|
||||
},
|
||||
mermaid: string,
|
||||
preset?: string,
|
||||
): Promise<{
|
||||
success: boolean;
|
||||
nodeId: string;
|
||||
attachmentId: string;
|
||||
warnings: string[];
|
||||
iconsResolved: number;
|
||||
iconsMissing: string[];
|
||||
verify?: any;
|
||||
}> {
|
||||
await this.ensureAuthenticated();
|
||||
const graph = mermaidToGraph(mermaid);
|
||||
if (preset) graph.preset = preset;
|
||||
return this.drawioFromGraph(pageId, where, graph, graph.direction, graph.preset);
|
||||
}
|
||||
|
||||
// --- Page history / diff / transform ---
|
||||
|
||||
/**
|
||||
* List the saved versions (history snapshots) of a page, newest first.
|
||||
* Docmost auto-snapshots on every save. Returns one cursor-paginated page of
|
||||
* results: `{ items, nextCursor }`. The history record's id field is `id`.
|
||||
*/
|
||||
}
|
||||
return DrawioMixin;
|
||||
}
|
||||
@@ -0,0 +1,165 @@
|
||||
// Central REST error diagnostics (issues #437 + #450). SINGLE place that maps an
|
||||
// axios error to the model-facing message. Extracted verbatim from client.ts;
|
||||
// the constructor's response interceptor (see client/context.ts) routes every
|
||||
// REST call through formatDocmostAxiosError so the whole surface is uniform.
|
||||
import axios from "axios";
|
||||
|
||||
// --- Issue #437: central error diagnostics -------------------------------
|
||||
// The agent only ever sees the thrown exception's `error.message`, so a failed
|
||||
// tool must return an ACTIONABLE message (method, path, status, and the
|
||||
// server's own validation text) instead of the opaque "Request failed with
|
||||
// status code 400". These helpers + the response interceptor in the
|
||||
// constructor are the single authoritative place that text is composed.
|
||||
|
||||
// Overall cap on the composed diagnostic message so the model context stays
|
||||
// compact and a (whitelisted) server string can never blow up the text.
|
||||
const ERROR_MESSAGE_CAP = 300;
|
||||
// Only attempt to JSON.parse an arraybuffer body under this size: a larger
|
||||
// binary body is never a JSON error envelope, so parsing it just wastes memory
|
||||
// (fetchInternalFile uses responseType:"arraybuffer", so a failed file fetch
|
||||
// carries the JSON error envelope as raw bytes here).
|
||||
const ERROR_BUFFER_PARSE_CAP = 4096;
|
||||
|
||||
// Canonical 36-char UUID (8-4-4-4-12 hex). Deliberately version/variant-
|
||||
// AGNOSTIC: the ids are UUIDv7 (e.g. 019f499a-9f8c-7d68-...), so only the
|
||||
// canonical shape/length is enforced, not the version/variant nibble.
|
||||
const FULL_UUID_RE =
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
|
||||
|
||||
/**
|
||||
* Throw an actionable error BEFORE any network call when `value` is not a full
|
||||
* canonical UUID. Absorbs #436: a truncated/short comment id used to reach the
|
||||
* server and bounce back as an opaque 400/404 the agent could not self-correct;
|
||||
* failing fast here names the exact fix.
|
||||
*/
|
||||
export function assertFullUuid(
|
||||
tool: string,
|
||||
param: string,
|
||||
value: string,
|
||||
): void {
|
||||
if (typeof value !== "string" || !FULL_UUID_RE.test(value)) {
|
||||
throw new Error(
|
||||
`${tool}: '${param}' must be the FULL comment UUID (36 chars, e.g. ` +
|
||||
`019f499a-9f8c-7d68-b7be-ce100d7c6c56), got '${value}'. Copy the id ` +
|
||||
`verbatim from listComments / createComment output.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Keep ONLY the pathname of a request (no host, no query string, no fragment)
|
||||
// so the message never leaks a host or query params. Resolves a relative
|
||||
// config.url against config.baseURL, then discards everything but the path.
|
||||
function requestPath(config: any): string {
|
||||
const rawUrl = typeof config?.url === "string" ? config.url : "";
|
||||
const base =
|
||||
typeof config?.baseURL === "string" ? config.baseURL : undefined;
|
||||
try {
|
||||
// A dummy base makes an absolute config.url parse too; its host is dropped.
|
||||
return new URL(rawUrl, base ?? "http://localhost").pathname;
|
||||
} catch {
|
||||
// Malformed url: still strip any query/fragment manually.
|
||||
return rawUrl.split(/[?#]/)[0] || rawUrl;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Compose the server-facing message from `error.response.data`, using ONLY the
|
||||
* whitelisted `message`/`error` fields or the HTTP statusText. SECURITY: the
|
||||
* raw response body, headers (Authorization!) and config are NEVER read here —
|
||||
* a string/HTML body (e.g. a proxy's 502 page) is deliberately dropped in
|
||||
* favour of the statusText.
|
||||
*/
|
||||
function extractServerMessage(data: any, statusText: string): string {
|
||||
// class-validator envelope: { message: string | string[], error?: string }.
|
||||
if (
|
||||
data &&
|
||||
typeof data === "object" &&
|
||||
!Buffer.isBuffer(data) &&
|
||||
!(data instanceof ArrayBuffer)
|
||||
) {
|
||||
const msg = (data as any).message;
|
||||
if (Array.isArray(msg)) {
|
||||
const joined = msg.filter((m) => typeof m === "string").join("; ");
|
||||
if (joined) return joined;
|
||||
} else if (typeof msg === "string" && msg) {
|
||||
return msg;
|
||||
}
|
||||
const err = (data as any).error;
|
||||
if (typeof err === "string" && err) return err;
|
||||
return statusText;
|
||||
}
|
||||
|
||||
// Buffer / ArrayBuffer body: attempt a size-capped, guarded JSON.parse so a
|
||||
// failed arraybuffer fetch still surfaces the server's validation text.
|
||||
if (Buffer.isBuffer(data) || data instanceof ArrayBuffer) {
|
||||
const buf = Buffer.isBuffer(data) ? data : Buffer.from(data);
|
||||
if (buf.length > 0 && buf.length <= ERROR_BUFFER_PARSE_CAP) {
|
||||
try {
|
||||
return extractServerMessage(JSON.parse(buf.toString("utf8")), statusText);
|
||||
} catch {
|
||||
return statusText;
|
||||
}
|
||||
}
|
||||
return statusText;
|
||||
}
|
||||
|
||||
// A raw string / HTML body is never surfaced (may echo server internals).
|
||||
return statusText;
|
||||
}
|
||||
|
||||
/**
|
||||
* Reformat an AxiosError's `.message` IN PLACE into an actionable diagnostic:
|
||||
* `<METHOD> <path> failed (<status> <statusText>): <serverMessage>`
|
||||
* or, when the request never got a response:
|
||||
* `<METHOD> <path> failed: <code> (no response from server)`.
|
||||
*
|
||||
* Mutates the SAME error object (never a custom subclass) so the live
|
||||
* axios.isAxiosError / error.response?.status / config._retry checks around the
|
||||
* client keep working, and sets `_docmostFormatted` as a double-processing
|
||||
* guard. A no-op on a non-axios or already-formatted error.
|
||||
*/
|
||||
export function formatDocmostAxiosError(error: any): void {
|
||||
if (!error || error._docmostFormatted) return;
|
||||
if (!axios.isAxiosError(error)) return;
|
||||
|
||||
const config: any = error.config ?? {};
|
||||
const method =
|
||||
typeof config.method === "string" ? config.method.toUpperCase() : "";
|
||||
const methodPath = `${method} ${requestPath(config)}`.trim();
|
||||
const response = error.response;
|
||||
|
||||
let message: string;
|
||||
if (response) {
|
||||
const statusText =
|
||||
typeof response.statusText === "string" ? response.statusText : "";
|
||||
const serverMessage = extractServerMessage(response.data, statusText);
|
||||
message = `${methodPath} failed (${response.status} ${statusText}): ${serverMessage}`;
|
||||
// Full body only to stderr under DEBUG (parity with downloadImage).
|
||||
if (process.env.DEBUG) {
|
||||
console.error(
|
||||
"Docmost request failed; response body:",
|
||||
JSON.stringify(response.data),
|
||||
);
|
||||
}
|
||||
} else {
|
||||
// No response at all (ECONNREFUSED / ETIMEDOUT / ECONNRESET / DNS / timeout).
|
||||
// Use ONLY error.code, never the raw error.message: axios network messages
|
||||
// embed host:port ("connect ECONNREFUSED 127.0.0.1:3000", "getaddrinfo
|
||||
// ENOTFOUND host") and #437's invariant is that the host never reaches the
|
||||
// model-visible message. code is set for essentially every real no-response
|
||||
// error (ECONNREFUSED/ETIMEDOUT/ECONNRESET/ENOTFOUND/ECONNABORTED); the full
|
||||
// native message still goes to stderr under DEBUG.
|
||||
const reason = error.code ?? "network error";
|
||||
message = `${methodPath} failed: ${reason} (no response from server)`;
|
||||
if (process.env.DEBUG) {
|
||||
console.error("Docmost request failed; no response:", error.message);
|
||||
}
|
||||
}
|
||||
|
||||
if (message.length > ERROR_MESSAGE_CAP) {
|
||||
message = message.slice(0, ERROR_MESSAGE_CAP - 1) + "…";
|
||||
}
|
||||
|
||||
error.message = message;
|
||||
(error as any)._docmostFormatted = true;
|
||||
}
|
||||
@@ -0,0 +1,730 @@
|
||||
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
|
||||
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
|
||||
// changed to a mixin factory. See client/context.ts for the shared base.
|
||||
import type { GConstructor, DocmostClientContext } from "./context.js";
|
||||
import FormData from "form-data";
|
||||
import axios, { AxiosInstance } from "axios";
|
||||
import { basename, extname } from "path";
|
||||
import {
|
||||
updatePageContentRealtime,
|
||||
replacePageContent,
|
||||
markdownToProseMirror,
|
||||
markdownToProseMirrorCanonical,
|
||||
mutatePageContent,
|
||||
assertYjsEncodable,
|
||||
MutationResult,
|
||||
} from "../lib/collaboration.js";
|
||||
import { withPageLock, isUuid } from "../lib/page-lock.js";
|
||||
import { diffDocs, summarizeChange } from "../lib/diff.js";
|
||||
import {
|
||||
blockText,
|
||||
walk,
|
||||
getList,
|
||||
insertMarkerAfter,
|
||||
setCalloutRange,
|
||||
noteItem,
|
||||
mdToInlineNodes,
|
||||
commentsToFootnotes,
|
||||
canonicalizeFootnotes,
|
||||
insertInlineFootnote,
|
||||
mergeFootnoteDefinitions,
|
||||
} from "../lib/transforms.js";
|
||||
|
||||
// Supported image types, kept as two lookup tables so both a local file
|
||||
// extension and a remote Content-Type can be mapped to the same canonical set.
|
||||
const EXT_TO_MIME: Record<string, string> = {
|
||||
".png": "image/png",
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".gif": "image/gif",
|
||||
".webp": "image/webp",
|
||||
".svg": "image/svg+xml",
|
||||
};
|
||||
const MIME_TO_EXT: Record<string, string> = {
|
||||
"image/png": ".png",
|
||||
"image/jpeg": ".jpg",
|
||||
"image/gif": ".gif",
|
||||
"image/webp": ".webp",
|
||||
"image/svg+xml": ".svg",
|
||||
};
|
||||
|
||||
// Public method surface of MediaMixin (issue #450) — a NAMED type so the factory
|
||||
// return type is expressible in the emitted .d.ts (the anonymous mixin class
|
||||
// carries the base's protected shared state, which would otherwise trip TS4094).
|
||||
// Derived from the class below; `implements IMediaMixin` fails to compile on drift.
|
||||
export interface IMediaMixin {
|
||||
uploadImage(pageId: string, url: string): any;
|
||||
insertImage(pageId: string, url: string, opts?: { align?: "left" | "center" | "right"; alt?: string; replaceText?: string; afterText?: string; }): any;
|
||||
replaceImage(pageId: string, oldAttachmentId: string, url: string, opts?: { align?: "left" | "center" | "right"; alt?: string }): any;
|
||||
}
|
||||
|
||||
export function MediaMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IMediaMixin> & TBase {
|
||||
abstract class MediaMixin extends Base implements IMediaMixin {
|
||||
// --- Image upload / embedding ---
|
||||
|
||||
/** Map a Content-Type string to a supported MIME type, or null if unsupported. */
|
||||
protected supportedImageMime(ct: string): string | null {
|
||||
return MIME_TO_EXT[ct] ? ct : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Download a remote image from a caller-supplied URL and resolve its bytes,
|
||||
* MIME and a filename.
|
||||
*
|
||||
* SSRF / RESOURCE TRUST BOUNDARY: the URL comes from the MCP caller and is
|
||||
* fetched BY THE SERVER, so it must be guarded before and after the request.
|
||||
* The guards mirror the local-file trust boundary in uploadImage:
|
||||
* - scheme allowlist (http/https only) — rejects file:, data:, ftp:, etc.,
|
||||
* so the caller cannot use this path to read local files or other schemes;
|
||||
* - a size cap enforced both via axios maxContentLength/maxBodyLength AND a
|
||||
* post-download buffer.length re-check (defends against a missing/lying
|
||||
* Content-Length), so a huge response cannot exhaust memory;
|
||||
* - a 30s timeout. The timeout matters because replaceImage holds the
|
||||
* per-page lock across this upload, so a hung download would wedge the
|
||||
* lock for that page.
|
||||
* We deliberately do NOT block private IP ranges: the MCP caller is already
|
||||
* trusted to read arbitrary host files via the filePath path, so the marginal
|
||||
* trust granted by fetching internal URLs is comparable, and blocking would
|
||||
* break legitimate internal-image use.
|
||||
*/
|
||||
protected async fetchRemoteImage(
|
||||
url: string,
|
||||
maxBytes: number,
|
||||
): Promise<{ buffer: Buffer; mime: string; fileName: string }> {
|
||||
// Scheme allowlist first — cheapest guard, and rejects non-http(s) schemes
|
||||
// (file:, data:, ftp:, ...) before any network request is made.
|
||||
let parsed: URL;
|
||||
try {
|
||||
parsed = new URL(url);
|
||||
} catch (e: any) {
|
||||
throw new Error(`Invalid image URL "${url}": ${e.message}`);
|
||||
}
|
||||
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
|
||||
throw new Error(
|
||||
`unsupported image URL scheme "${parsed.protocol}"; only http and https are allowed`,
|
||||
);
|
||||
}
|
||||
|
||||
let response;
|
||||
try {
|
||||
response = await axios.get(url, {
|
||||
responseType: "arraybuffer",
|
||||
timeout: 30000,
|
||||
maxContentLength: maxBytes,
|
||||
maxBodyLength: maxBytes,
|
||||
headers: { Accept: "image/*" },
|
||||
});
|
||||
} catch (error) {
|
||||
// Keep the thrown message free of the raw response body (it may echo
|
||||
// server internals); surface only status/statusText. The full body is
|
||||
// logged under DEBUG for diagnostics.
|
||||
if (axios.isAxiosError(error)) {
|
||||
if (process.env.DEBUG) {
|
||||
console.error(
|
||||
"Image download failed; response body:",
|
||||
JSON.stringify(error.response?.data),
|
||||
);
|
||||
}
|
||||
throw new Error(
|
||||
`Image download failed for "${url}": ${error.response?.status ?? ""} ${error.response?.statusText ?? error.message}`.trim(),
|
||||
);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
|
||||
// axios returns an ArrayBuffer for responseType: "arraybuffer".
|
||||
const buffer = Buffer.from(response.data);
|
||||
// Re-check the size: maxContentLength relies on Content-Length, which may be
|
||||
// absent or lie, so guard against the actual byte count too.
|
||||
if (buffer.length === 0) {
|
||||
throw new Error(`Empty image response from "${url}"`);
|
||||
}
|
||||
if (buffer.length > maxBytes) {
|
||||
throw new Error(
|
||||
`Image too large: ${buffer.length} bytes exceeds the ${maxBytes}-byte cap`,
|
||||
);
|
||||
}
|
||||
|
||||
// Resolve MIME: prefer the response Content-Type (strip any "; charset=..."
|
||||
// parameter, lowercase, trim) mapped through the supported set; if the
|
||||
// header is generic/missing/unsupported, fall back to the URL path
|
||||
// extension via the existing extension->MIME logic.
|
||||
const rawCt = response.headers?.["content-type"];
|
||||
let mime: string | null = null;
|
||||
if (typeof rawCt === "string" && rawCt.length > 0) {
|
||||
const ct = rawCt.split(";")[0].trim().toLowerCase();
|
||||
mime = this.supportedImageMime(ct);
|
||||
}
|
||||
if (!mime) {
|
||||
// Fall back to the URL path extension. Use the pathname so the query
|
||||
// string never contaminates the extension lookup.
|
||||
const ext = extname(parsed.pathname).toLowerCase();
|
||||
mime = EXT_TO_MIME[ext] ?? null;
|
||||
}
|
||||
if (!mime) {
|
||||
throw new Error(
|
||||
`cannot determine supported image type for "${url}"; supported: png, jpg, jpeg, gif, webp, svg`,
|
||||
);
|
||||
}
|
||||
|
||||
// Build a filename from the URL path basename (ignore the query string),
|
||||
// defaulting to "image" when empty, and ensure it ends with the canonical
|
||||
// extension for the resolved MIME (append it when missing/mismatched).
|
||||
const canonicalExt = MIME_TO_EXT[mime];
|
||||
let fileName = basename(parsed.pathname) || "image";
|
||||
if (extname(fileName).toLowerCase() !== canonicalExt) {
|
||||
fileName += canonicalExt;
|
||||
}
|
||||
|
||||
return { buffer, mime, fileName };
|
||||
}
|
||||
|
||||
/** Build a Docmost ProseMirror image node from an uploaded attachment. */
|
||||
protected buildImageNode(
|
||||
att: { id: string; fileName: string; fileSize?: number },
|
||||
align?: "left" | "center" | "right",
|
||||
alt?: string,
|
||||
): any {
|
||||
// Clean file URL, matching Docmost's native behaviour. No cache-busting
|
||||
// query: the server serves the bare URL correctly, and replacement creates
|
||||
// a new attachment id (a new URL) which busts caches naturally.
|
||||
const src = `/api/files/${att.id}/${att.fileName}`;
|
||||
const node: any = {
|
||||
type: "image",
|
||||
attrs: {
|
||||
src,
|
||||
attachmentId: att.id,
|
||||
// Default to null when the server omits fileSize so the attr is never
|
||||
// undefined (undefined would be dropped on serialization / break the
|
||||
// ProseMirror image schema which expects size present).
|
||||
size: att.fileSize ?? null,
|
||||
align: align || "center",
|
||||
width: null,
|
||||
},
|
||||
};
|
||||
if (alt) node.attrs.alt = alt;
|
||||
return node;
|
||||
}
|
||||
|
||||
/**
|
||||
* Download a remote image from an http(s) URL and upload it as an attachment
|
||||
* of a page, returning the attachment metadata plus a ready-to-insert
|
||||
* ProseMirror image node. Local file paths are intentionally not supported:
|
||||
* the MCP caller is a remote AI with no access to this server's filesystem.
|
||||
*/
|
||||
async uploadImage(pageId: string, url: string) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
const MAX_IMAGE_BYTES = 20 * 1024 * 1024; // 20 MiB
|
||||
|
||||
// Fetch + validate the remote image (scheme allowlist, size cap, timeout).
|
||||
// See fetchRemoteImage for the SSRF / resource trust boundary.
|
||||
const fetched = await this.fetchRemoteImage(url, MAX_IMAGE_BYTES);
|
||||
const fileBuffer = fetched.buffer;
|
||||
const mime = fetched.mime;
|
||||
const fileName = fetched.fileName;
|
||||
|
||||
// Build a FRESH FormData for every send attempt. A FormData body is a
|
||||
// single-use stream that is CONSUMED on the first send, so it cannot be
|
||||
// replayed by this.client's response interceptor (replaying a consumed
|
||||
// stream fails with 'socket hang up'). Multipart re-auth is therefore done
|
||||
// here with bare axios and an explicit one-shot 401/403 retry that rebuilds
|
||||
// the body. Field order matters: text fields must precede the file part so
|
||||
// the server reads them; the server always generates a fresh attachment id.
|
||||
const buildForm = () => {
|
||||
const form = new FormData();
|
||||
form.append("pageId", pageId);
|
||||
form.append("file", fileBuffer, {
|
||||
filename: fileName,
|
||||
contentType: mime,
|
||||
});
|
||||
return form;
|
||||
};
|
||||
|
||||
// Local name distinct from the `url` parameter (the source image URL): this
|
||||
// is the /files/upload endpoint we POST the multipart body to.
|
||||
const uploadUrl = `${this.apiUrl}/files/upload`;
|
||||
let response;
|
||||
try {
|
||||
// Call buildForm() ONCE per attempt and reuse the instance for both
|
||||
// getHeaders() and the body so the Content-Type boundary matches the body.
|
||||
const form = buildForm();
|
||||
// Read the Authorization header from this.client's defaults (set by
|
||||
// login(), only ever deleted — never set to null) instead of building
|
||||
// `Bearer ${this.token}`: a concurrent JSON 401 can null this.token
|
||||
// mid-flight, which would otherwise produce a literal "Bearer null".
|
||||
// ensureAuthenticated() above guarantees login() ran, so the default
|
||||
// header exists here. A 60s timeout keeps a hung upload from wedging the
|
||||
// per-page lock (replaceImage holds withPageLock across this call).
|
||||
response = await axios.post(uploadUrl, form, {
|
||||
headers: {
|
||||
...form.getHeaders(),
|
||||
Authorization: this.client.defaults.headers.common["Authorization"],
|
||||
},
|
||||
timeout: 60000,
|
||||
});
|
||||
} catch (error) {
|
||||
// On an expired-token auth error, re-login and retry exactly once with a
|
||||
// freshly-rebuilt FormData (the previous one was already consumed).
|
||||
if (
|
||||
axios.isAxiosError(error) &&
|
||||
(error.response?.status === 401 || error.response?.status === 403)
|
||||
) {
|
||||
await this.login();
|
||||
const form2 = buildForm();
|
||||
response = await axios.post(uploadUrl, form2, {
|
||||
headers: {
|
||||
...form2.getHeaders(),
|
||||
Authorization: this.client.defaults.headers.common["Authorization"],
|
||||
},
|
||||
timeout: 60000,
|
||||
});
|
||||
} else if (axios.isAxiosError(error)) {
|
||||
// Keep the thrown message free of the raw response body (it may echo
|
||||
// request data or server internals); surface only status/statusText.
|
||||
// The full body is logged under DEBUG for diagnostics.
|
||||
if (process.env.DEBUG) {
|
||||
console.error(
|
||||
"Image upload failed; response body:",
|
||||
JSON.stringify(error.response?.data),
|
||||
);
|
||||
}
|
||||
throw new Error(
|
||||
`Image upload failed: ${error.response?.status} ${error.response?.statusText}`,
|
||||
);
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
// The attachment may arrive bare or wrapped in a { data } envelope.
|
||||
const att = response.data?.data ?? response.data;
|
||||
if (!att?.id || !att?.fileName) {
|
||||
throw new Error(
|
||||
"Unexpected /files/upload response: " + JSON.stringify(response.data),
|
||||
);
|
||||
}
|
||||
|
||||
// Some Docmost versions omit fileSize from the upload response. Fall back
|
||||
// to the fetched byte length (the bytes we just uploaded) so callers never
|
||||
// get an undefined size.
|
||||
const resolvedSize = att.fileSize ?? fileBuffer.length;
|
||||
|
||||
return {
|
||||
attachmentId: att.id,
|
||||
fileName: att.fileName,
|
||||
fileSize: resolvedSize,
|
||||
src: `/api/files/${att.id}/${att.fileName}`,
|
||||
imageNode: this.buildImageNode({ ...att, fileSize: resolvedSize }),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Upload an image from a web (http/https) URL and insert it into a page in
|
||||
* one step.
|
||||
* By default the image is appended at the end. With replaceText, the first
|
||||
* top-level block whose text contains the string is replaced; with afterText,
|
||||
* the image is inserted right after the first matching block. All other
|
||||
* block ids are preserved (only one top-level block is added or swapped).
|
||||
*/
|
||||
async insertImage(
|
||||
pageId: string,
|
||||
url: string,
|
||||
opts: {
|
||||
align?: "left" | "center" | "right";
|
||||
alt?: string;
|
||||
replaceText?: string;
|
||||
afterText?: string;
|
||||
} = {},
|
||||
) {
|
||||
const up = await this.uploadImage(pageId, url);
|
||||
// Reuse the node from uploadImage (clean /api/files/<id>/<file> src), then
|
||||
// apply align/alt onto a shallow attrs copy.
|
||||
const node: any = { ...up.imageNode, attrs: { ...up.imageNode.attrs } };
|
||||
if (opts.align) node.attrs.align = opts.align;
|
||||
if (opts.alt) node.attrs.alt = opts.alt;
|
||||
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260). The
|
||||
// uploadImage /files/upload call above keeps the agent-supplied id.
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
// Recursively collect the plain text of a top-level block.
|
||||
const blockText = (n: any): string => {
|
||||
let out = "";
|
||||
if (n.type === "text") out += n.text || "";
|
||||
for (const child of n.content || []) out += blockText(child);
|
||||
return out;
|
||||
};
|
||||
|
||||
// Insert into the LIVE synced document, not the debounced REST snapshot, so
|
||||
// concurrent edits/comments/images are preserved and parallel insertImage
|
||||
// calls (serialized by the per-page lock) each see the previous insertion.
|
||||
let placement: "replaced" | "after" | "appended" | undefined;
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
const doc =
|
||||
liveDoc && liveDoc.type === "doc"
|
||||
? liveDoc
|
||||
: { type: "doc", content: [] };
|
||||
if (!Array.isArray(doc.content)) doc.content = [];
|
||||
|
||||
if (opts.replaceText) {
|
||||
// Ambiguity guard (mirrors editPageText): count matching top-level
|
||||
// blocks first, so a non-unique fragment cannot silently replace the
|
||||
// wrong block (e.g. text that also appears inside a callout/table).
|
||||
const matches = doc.content.filter((b: any) =>
|
||||
blockText(b).includes(opts.replaceText!),
|
||||
);
|
||||
if (matches.length === 0) {
|
||||
throw new Error(`replaceText not found: "${opts.replaceText}"`);
|
||||
}
|
||||
if (matches.length > 1) {
|
||||
throw new Error(
|
||||
`replaceText "${opts.replaceText}" matches ${matches.length} blocks; use a longer unique fragment`,
|
||||
);
|
||||
}
|
||||
const idx = doc.content.findIndex((b: any) =>
|
||||
blockText(b).includes(opts.replaceText!),
|
||||
);
|
||||
// Data-loss guard: replaceText swaps the WHOLE top-level block, so if
|
||||
// the fragment only appears nested inside a container (table, callout,
|
||||
// list, blockquote) the entire structure would be destroyed. Refuse
|
||||
// when the matched block is a container rather than a leaf
|
||||
// paragraph/heading and point the caller at a safer tool.
|
||||
const CONTAINER_TYPES = new Set([
|
||||
"table",
|
||||
"callout",
|
||||
"bulletList",
|
||||
"orderedList",
|
||||
"taskList",
|
||||
"blockquote",
|
||||
]);
|
||||
const matchedBlock = doc.content[idx];
|
||||
if (matchedBlock && CONTAINER_TYPES.has(matchedBlock.type)) {
|
||||
throw new Error(
|
||||
`replaceText matched a ${matchedBlock.type} container block; replacing it would destroy the whole structure. ` +
|
||||
`Use afterText to insert near it, or updatePageJson for surgical edits.`,
|
||||
);
|
||||
}
|
||||
doc.content.splice(idx, 1, node);
|
||||
placement = "replaced";
|
||||
} else if (opts.afterText) {
|
||||
// Ambiguity guard (mirrors editPageText): refuse a non-unique fragment.
|
||||
const matches = doc.content.filter((b: any) =>
|
||||
blockText(b).includes(opts.afterText!),
|
||||
);
|
||||
if (matches.length === 0) {
|
||||
throw new Error(`afterText not found: "${opts.afterText}"`);
|
||||
}
|
||||
if (matches.length > 1) {
|
||||
throw new Error(
|
||||
`afterText "${opts.afterText}" matches ${matches.length} blocks; use a longer unique fragment`,
|
||||
);
|
||||
}
|
||||
const idx = doc.content.findIndex((b: any) =>
|
||||
blockText(b).includes(opts.afterText!),
|
||||
);
|
||||
doc.content.splice(idx + 1, 0, node);
|
||||
placement = "after";
|
||||
} else {
|
||||
doc.content.push(node);
|
||||
placement = "appended";
|
||||
}
|
||||
|
||||
return doc;
|
||||
},
|
||||
);
|
||||
|
||||
return {
|
||||
success: true,
|
||||
pageId,
|
||||
attachmentId: up.attachmentId,
|
||||
src: up.src,
|
||||
placement,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace an existing image in a page with a new image fetched from a web
|
||||
* (http/https) URL. Uploads the new file as a brand-new attachment, which
|
||||
* yields a fresh clean URL that both renders correctly and busts browser
|
||||
* caches (the URL changed). Finds every image node
|
||||
* whose attrs.attachmentId === oldAttachmentId (recursively, incl. nodes nested
|
||||
* in callouts/tables) and repoints its src/attachmentId/size, preserving
|
||||
* comments, alignment and alt. Operates on the live collab document so comments
|
||||
* and concurrent edits are preserved. Throws if no matching image is found.
|
||||
*
|
||||
* The OLD attachment is left in place as an unreferenced orphan: Docmost
|
||||
* exposes NO HTTP API to delete a single content attachment (verified against
|
||||
* the attachment controller/service and by probing the live API — deletion
|
||||
* happens only by cascade when the page, space or user is removed). This is the
|
||||
* same outcome as Docmost's own editor when an image is removed/replaced.
|
||||
* In-place byte overwrite is deliberately NOT used because some Docmost
|
||||
* versions corrupt the attachment (HTTP 500) when its bytes are overwritten.
|
||||
*/
|
||||
async replaceImage(
|
||||
pageId: string,
|
||||
oldAttachmentId: string,
|
||||
url: string,
|
||||
opts: { align?: "left" | "center" | "right"; alt?: string } = {},
|
||||
) {
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260). The
|
||||
// page lock must ALSO key on the UUID so this operation serializes against
|
||||
// other writes to the same page (mutatePageContent now locks by the resolved
|
||||
// UUID too); locking by the raw slugId here would desync the mutex key and
|
||||
// reopen the TOCTOU/orphan-attachment window the lock closes. uploadImage
|
||||
// keeps the agent-supplied id (it hits REST, not the collab doc).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
// Hold ONE per-page lock for the WHOLE operation (scan -> upload -> write).
|
||||
// Previously the scan and the write were two separate mutatePageContent
|
||||
// calls, each acquiring + releasing the lock, with the upload happening in
|
||||
// the UNLOCKED gap between them. A concurrent op could interleave there: it
|
||||
// could remove the target image so the write pass matches nothing, leaving
|
||||
// the freshly-uploaded attachment as an un-deletable orphan (Docmost has no
|
||||
// API to delete a single content attachment). Acquiring the lock once and
|
||||
// using the non-locking collab helper inside (the per-page mutex is NOT
|
||||
// reentrant, so the self-locking mutatePageContent would deadlock here)
|
||||
// closes that TOCTOU window. uploadImage hits /files/upload over plain HTTP
|
||||
// and does not touch the page lock, so it is safe to call while held.
|
||||
return withPageLock(pageUuid, async () => {
|
||||
// STEP 1: read-only live check. Scan the live document for any image node
|
||||
// matching oldAttachmentId BEFORE uploading anything, so a wrong/stale id
|
||||
// throws without ever creating an orphan attachment.
|
||||
let matchFound = false;
|
||||
const scan = (nodes: any[]) => {
|
||||
for (const node of nodes) {
|
||||
if (!node) continue;
|
||||
if (
|
||||
node.type === "image" &&
|
||||
node.attrs &&
|
||||
node.attrs.attachmentId === oldAttachmentId
|
||||
) {
|
||||
matchFound = true;
|
||||
}
|
||||
if (Array.isArray(node.content)) scan(node.content);
|
||||
}
|
||||
};
|
||||
|
||||
await this.mutateLiveContentUnlocked(pageUuid, collabToken, (liveDoc) => {
|
||||
matchFound = false; // reset per-transform (collab may retry the read).
|
||||
const doc =
|
||||
liveDoc && liveDoc.type === "doc"
|
||||
? liveDoc
|
||||
: { type: "doc", content: [] };
|
||||
if (Array.isArray(doc.content)) scan(doc.content);
|
||||
return null; // read-only: never write on the check pass.
|
||||
});
|
||||
|
||||
if (!matchFound) {
|
||||
throw new Error(
|
||||
`replaceImage: no image with attachmentId "${oldAttachmentId}" found on page ${pageId}`,
|
||||
);
|
||||
}
|
||||
|
||||
// STEP 2: a match exists — upload the new file as a FRESH attachment (new
|
||||
// id, new clean URL) and repoint every matching node in a second pass.
|
||||
// Still inside the SAME lock, so no other op can have changed the page
|
||||
// since the scan.
|
||||
const up = await this.uploadImage(pageId, url);
|
||||
|
||||
let replaced = 0;
|
||||
|
||||
// Swap the source of one image node, preserving align/alt/title/geometry.
|
||||
const repoint = (node: any) => {
|
||||
node.attrs = {
|
||||
...node.attrs,
|
||||
src: up.src,
|
||||
attachmentId: up.attachmentId,
|
||||
// Default to null when fileSize is unknown so the attr is never
|
||||
// undefined.
|
||||
size: up.fileSize ?? null,
|
||||
};
|
||||
if (opts.align) node.attrs.align = opts.align;
|
||||
if (opts.alt !== undefined) node.attrs.alt = opts.alt;
|
||||
replaced++;
|
||||
};
|
||||
|
||||
// Recursively repoint every image node (incl. ones nested in callouts/tables).
|
||||
const walk = (nodes: any[]) => {
|
||||
for (const node of nodes) {
|
||||
if (!node) continue;
|
||||
if (
|
||||
node.type === "image" &&
|
||||
node.attrs &&
|
||||
node.attrs.attachmentId === oldAttachmentId
|
||||
) {
|
||||
repoint(node);
|
||||
}
|
||||
if (Array.isArray(node.content)) walk(node.content);
|
||||
}
|
||||
};
|
||||
|
||||
const mutation = await this.mutateLiveContentUnlocked(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
(liveDoc) => {
|
||||
// Reset per-transform so collab retries recompute cleanly (no double-count).
|
||||
replaced = 0;
|
||||
const doc =
|
||||
liveDoc && liveDoc.type === "doc"
|
||||
? liveDoc
|
||||
: { type: "doc", content: [] };
|
||||
if (!Array.isArray(doc.content)) doc.content = [];
|
||||
walk(doc.content);
|
||||
if (replaced === 0) return null; // no match -> skip the write entirely
|
||||
return doc;
|
||||
},
|
||||
);
|
||||
// KNOWN LIMITATION: a same-count image SRC swap (image count unchanged, no
|
||||
// text/mark change) may still report verify.changed === false, because the
|
||||
// text+marks+integrity-count model in summarizeChange does not inspect
|
||||
// image `src`/attachmentId attributes. That is acceptable here — the
|
||||
// replace is confirmed by `replaced` below, and verify is supplementary.
|
||||
|
||||
if (replaced === 0) {
|
||||
// The pass-1 SCAN found the target (matchFound was true) and we already
|
||||
// uploaded the new attachment, but pass-2 matched nothing — a concurrent
|
||||
// editor must have removed the node between the two passes. Do NOT throw
|
||||
// here (that would leak the just-uploaded attachment AND report failure);
|
||||
// instead report success with the upload flagged as an unreferenced
|
||||
// orphan so the caller knows. (The early throw above still covers the
|
||||
// case where pass-1 finds nothing, before any upload happens.)
|
||||
return {
|
||||
success: true,
|
||||
replaced: 0,
|
||||
pageId,
|
||||
oldAttachmentId,
|
||||
newAttachmentId: up.attachmentId,
|
||||
src: up.src,
|
||||
orphanedAttachmentId: up.attachmentId,
|
||||
warning:
|
||||
"target image was removed concurrently; uploaded attachment is unreferenced",
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
pageId,
|
||||
replaced,
|
||||
oldAttachmentId,
|
||||
newAttachmentId: up.attachmentId,
|
||||
src: up.src,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
// --- draw.io diagrams (issue #423) ---
|
||||
|
||||
/**
|
||||
* Upload a ready-made byte buffer as a page attachment via the same
|
||||
* multipart /files/upload endpoint uploadImage uses. Split out as its own
|
||||
* (overridable) seam so drawioCreate/update can upload the generated
|
||||
* `.drawio.svg` without going through the URL-fetch path, and so tests can
|
||||
* stub the network. Mirrors uploadImage's fresh-FormData + one-shot 401/403
|
||||
* re-auth handling (a FormData body is single-use, so it must be rebuilt per
|
||||
* attempt).
|
||||
*/
|
||||
|
||||
// --- draw.io diagrams (issue #423) ---
|
||||
|
||||
/**
|
||||
* Upload a ready-made byte buffer as a page attachment via the same
|
||||
* multipart /files/upload endpoint uploadImage uses. Split out as its own
|
||||
* (overridable) seam so drawioCreate/update can upload the generated
|
||||
* `.drawio.svg` without going through the URL-fetch path, and so tests can
|
||||
* stub the network. Mirrors uploadImage's fresh-FormData + one-shot 401/403
|
||||
* re-auth handling (a FormData body is single-use, so it must be rebuilt per
|
||||
* attempt).
|
||||
*/
|
||||
protected async uploadAttachmentBuffer(
|
||||
pageId: string,
|
||||
buffer: Buffer,
|
||||
fileName: string,
|
||||
mime: string,
|
||||
): Promise<{ id: string; fileName: string; fileSize: number }> {
|
||||
await this.ensureAuthenticated();
|
||||
const buildForm = () => {
|
||||
const form = new FormData();
|
||||
form.append("pageId", pageId);
|
||||
form.append("file", buffer, { filename: fileName, contentType: mime });
|
||||
return form;
|
||||
};
|
||||
const uploadUrl = `${this.apiUrl}/files/upload`;
|
||||
let response;
|
||||
try {
|
||||
const form = buildForm();
|
||||
response = await axios.post(uploadUrl, form, {
|
||||
headers: {
|
||||
...form.getHeaders(),
|
||||
Authorization: this.client.defaults.headers.common["Authorization"],
|
||||
},
|
||||
timeout: 60000,
|
||||
});
|
||||
} catch (error) {
|
||||
if (
|
||||
axios.isAxiosError(error) &&
|
||||
(error.response?.status === 401 || error.response?.status === 403)
|
||||
) {
|
||||
await this.login();
|
||||
const form2 = buildForm();
|
||||
response = await axios.post(uploadUrl, form2, {
|
||||
headers: {
|
||||
...form2.getHeaders(),
|
||||
Authorization: this.client.defaults.headers.common["Authorization"],
|
||||
},
|
||||
timeout: 60000,
|
||||
});
|
||||
} else if (axios.isAxiosError(error)) {
|
||||
if (process.env.DEBUG) {
|
||||
console.error(
|
||||
"Attachment upload failed; response body:",
|
||||
JSON.stringify(error.response?.data),
|
||||
);
|
||||
}
|
||||
throw new Error(
|
||||
`Attachment upload failed: ${error.response?.status} ${error.response?.statusText}`,
|
||||
);
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
const att = response.data?.data ?? response.data;
|
||||
if (!att?.id || !att?.fileName) {
|
||||
throw new Error(
|
||||
"Unexpected /files/upload response: " + JSON.stringify(response.data),
|
||||
);
|
||||
}
|
||||
return {
|
||||
id: att.id,
|
||||
fileName: att.fileName,
|
||||
fileSize: att.fileSize ?? buffer.length,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch a stored `.drawio.svg` attachment as text. Overridable seam over
|
||||
* fetchInternalFile (the authed loopback fetch, which also rejects any
|
||||
* traversal/SSRF src) so drawioGet/update can read the current diagram and
|
||||
* tests can stub the bytes.
|
||||
*/
|
||||
protected async fetchAttachmentText(src: string): Promise<string> {
|
||||
const { buffer } = await this.fetchInternalFile(src);
|
||||
return buffer.toString("utf-8");
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a drawio node on a page by `attrs.id` or `#<index>` and return the
|
||||
* node plus its ref. Throws a clear error if the ref does not resolve to a
|
||||
* drawio node.
|
||||
*/
|
||||
}
|
||||
return MediaMixin;
|
||||
}
|
||||
@@ -0,0 +1,700 @@
|
||||
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
|
||||
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
|
||||
// changed to a mixin factory. See client/context.ts for the shared base.
|
||||
import type { GConstructor, DocmostClientContext } from "./context.js";
|
||||
import {
|
||||
updatePageContentRealtime,
|
||||
replacePageContent,
|
||||
markdownToProseMirror,
|
||||
markdownToProseMirrorCanonical,
|
||||
mutatePageContent,
|
||||
assertYjsEncodable,
|
||||
MutationResult,
|
||||
} from "../lib/collaboration.js";
|
||||
import {
|
||||
replaceNodeById,
|
||||
replaceNodeByIdWithMany,
|
||||
reassignCollidingBlockIds,
|
||||
deleteNodeById,
|
||||
assertUnambiguousMatch,
|
||||
insertNodeRelative,
|
||||
insertNodesRelative,
|
||||
blockPlainText,
|
||||
buildOutline,
|
||||
getNodeByRef,
|
||||
readTable,
|
||||
insertTableRow,
|
||||
deleteTableRow,
|
||||
updateTableCell,
|
||||
findInvalidNode,
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
import {
|
||||
importMarkdownFragment,
|
||||
canBeDocChild,
|
||||
findUnrepresentableTableAttrs,
|
||||
} from "../lib/markdown-fragment.js";
|
||||
import {
|
||||
applyTextEdits,
|
||||
TextEdit,
|
||||
TextEditResult,
|
||||
TextEditFailure,
|
||||
} from "../lib/json-edit.js";
|
||||
import {
|
||||
blockText,
|
||||
walk,
|
||||
getList,
|
||||
insertMarkerAfter,
|
||||
setCalloutRange,
|
||||
noteItem,
|
||||
mdToInlineNodes,
|
||||
commentsToFootnotes,
|
||||
canonicalizeFootnotes,
|
||||
insertInlineFootnote,
|
||||
mergeFootnoteDefinitions,
|
||||
} from "../lib/transforms.js";
|
||||
import { normalizeAndMergeFootnotes } from "../lib/footnote-normalize-merge.js";
|
||||
|
||||
// Public method surface of NodesWriteMixin (issue #450) — a NAMED type so the factory
|
||||
// return type is expressible in the emitted .d.ts (the anonymous mixin class
|
||||
// carries the base's protected shared state, which would otherwise trip TS4094).
|
||||
// Derived from the class below; `implements INodesWriteMixin` fails to compile on drift.
|
||||
export interface INodesWriteMixin {
|
||||
updatePageJson(pageId: string, doc?: any, title?: string): any;
|
||||
editPageText(pageId: string, edits: TextEdit[]): any;
|
||||
patchNode(pageId: string, nodeId: string, input: { markdown?: string; node?: any }): any;
|
||||
insertNode(pageId: string, input: { markdown?: string; node?: any }, opts: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }): any;
|
||||
deleteNode(pageId: string, nodeId: string): any;
|
||||
}
|
||||
|
||||
export function NodesWriteMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & INodesWriteMixin> & TBase {
|
||||
abstract class NodesWriteMixin extends Base implements INodesWriteMixin {
|
||||
/**
|
||||
* Replace page content with a raw ProseMirror JSON document (lossless) and/or
|
||||
* update its title. Both `doc` and `title` are optional, but at least one must
|
||||
* be supplied:
|
||||
* - `doc` provided -> validate + full-overwrite the body (and update the
|
||||
* title too when `title` is also given).
|
||||
* - `doc` omitted, `title` given -> title-only update; the body is NOT
|
||||
* touched/resent (no collab write happens).
|
||||
* - neither given -> throws (nothing to update).
|
||||
*/
|
||||
async updatePageJson(pageId: string, doc?: any, title?: string) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
// Title-only / no-op handling: when no document is supplied, do NOT write
|
||||
// the body. Update the title if one was given; otherwise there is nothing
|
||||
// to do, so fail loudly rather than silently no-op.
|
||||
if (doc == null) {
|
||||
if (!title) {
|
||||
throw new Error(
|
||||
"updatePageJson: nothing to update (provide content and/or title)",
|
||||
);
|
||||
}
|
||||
await this.client.post("/pages/update", { pageId, title });
|
||||
return {
|
||||
success: true,
|
||||
modified: true,
|
||||
message: "Page title updated (content left unchanged).",
|
||||
pageId,
|
||||
};
|
||||
}
|
||||
|
||||
// Validate the document shape before a full overwrite: a malformed doc
|
||||
// would otherwise silently corrupt the page (full-overwrite is the
|
||||
// documented behaviour; no optimistic-concurrency is applied here).
|
||||
if (
|
||||
typeof doc !== "object" ||
|
||||
doc.type !== "doc" ||
|
||||
!Array.isArray(doc.content)
|
||||
) {
|
||||
throw new Error(
|
||||
'content must be a ProseMirror document ({"type":"doc","content":[...]}) ' +
|
||||
"where content is an array of nodes each having a string `type`",
|
||||
);
|
||||
}
|
||||
|
||||
// Recurse the WHOLE document so a malformed nested node (e.g. a node with a
|
||||
// non-string type, a non-array content/marks, or a text node missing its
|
||||
// string text) is rejected up front rather than silently corrupting the
|
||||
// page on overwrite.
|
||||
this.validateDocStructure(doc);
|
||||
|
||||
// #409: beyond the string-`type` check above, reject a nested node whose
|
||||
// `type` is a string but NOT a known Docmost schema node (a typo/unknown
|
||||
// block) — the same `Unknown node type` the encoder throws — with a rich,
|
||||
// path-anchored message, still BEFORE any collab connection.
|
||||
this.assertValidNodeShape("updatePageJson", doc);
|
||||
|
||||
// Sanitize URLs before writing. This closes the JSON-path bypass: unlike
|
||||
// the markdown link path (which TipTap sanitizes), raw JSON could otherwise
|
||||
// inject javascript:/data: link hrefs or media srcs straight into the doc.
|
||||
this.validateDocUrls(doc);
|
||||
|
||||
// Canonicalize footnotes (idempotent): an agent-authored JSON doc cannot
|
||||
// leave footnotes out of order, orphaned, or in multiple lists — the bottom
|
||||
// list + numbering are always derived from reference order. No-op when the
|
||||
// footnotes are already canonical.
|
||||
// #419: normalize + merge glyph-forked definitions before canonicalizing.
|
||||
doc = normalizeAndMergeFootnotes(doc);
|
||||
doc = canonicalizeFootnotes(doc);
|
||||
|
||||
// Write the BODY first, then the title (#159 split-brain): a failed body
|
||||
// write (e.g. persist timeout) must not leave a new title over the old body.
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
const mutation = await this.replacePage(
|
||||
pageUuid,
|
||||
doc,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
);
|
||||
|
||||
// Body persisted successfully — now it is safe to set the title.
|
||||
if (title) {
|
||||
await this.client.post("/pages/update", { pageId, title });
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
modified: true,
|
||||
message: "Page content replaced from ProseMirror JSON.",
|
||||
pageId,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* AUTHOR-INLINE footnote insertion. The agent supplies only WHERE
|
||||
* (`anchorText`, a snippet of body text to attach the marker after) and WHAT
|
||||
* (`text`, the footnote content as markdown). Numbering and the bottom
|
||||
* `footnotesList` are derived deterministically server-side
|
||||
* (`insertInlineFootnote` -> `canonicalizeFootnotes`): the agent never sees,
|
||||
* assigns, or edits a footnote number or the list, so it CANNOT desync.
|
||||
*
|
||||
* Content DEDUP: when an existing definition has the same content, its id is
|
||||
* reused (one number, one definition, several references). The write is atomic
|
||||
* via `mutatePageContent` (single-writer, page-locked); if the anchor text is
|
||||
* not found the transform aborts with a clear error and no write happens.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Surgical text edits: find/replace inside text nodes of the live
|
||||
* document. Preserves all block ids, marks, callouts and tables.
|
||||
*/
|
||||
async editPageText(pageId: string, edits: TextEdit[]) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
// Apply the edits against the LIVE synced document, not the debounced REST
|
||||
// snapshot, so concurrent human edits/comments are preserved. applyTextEdits
|
||||
// records per-edit match problems in `failed` instead of throwing, and
|
||||
// applies whatever it can; we abort the write only when nothing applied.
|
||||
let results: TextEditResult[] | undefined;
|
||||
let failed: TextEditFailure[] | undefined;
|
||||
// Whether we actually wrote new content. Set inside the transform: a
|
||||
// degenerate edit (e.g. find === replace, or a batch that nets to no change)
|
||||
// can "apply" yet leave the document byte-for-byte identical, in which case
|
||||
// we must NOT write (no spurious history version) and must not claim a write
|
||||
// happened.
|
||||
let wrote = false;
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
wrote = false;
|
||||
const r = applyTextEdits(liveDoc, edits);
|
||||
results = r.results;
|
||||
failed = r.failed;
|
||||
// Nothing applied -> abort the write (mutatePageContent treats a null
|
||||
// return from the transform as "write nothing").
|
||||
if (r.results.length === 0) return null;
|
||||
// Edits "applied" but produced an identical document: skip the write so
|
||||
// no new history version is created. Stable structural comparison via
|
||||
// JSON.stringify (both docs come from the same deep-copied source, so
|
||||
// key order is stable).
|
||||
if (JSON.stringify(r.doc) === JSON.stringify(liveDoc)) return null;
|
||||
wrote = true;
|
||||
return r.doc;
|
||||
},
|
||||
);
|
||||
|
||||
if ((results?.length ?? 0) === 0 && (failed?.length ?? 0) > 0) {
|
||||
// No edit applied: surface an aggregated, actionable error so the caller
|
||||
// does not mistake a no-op for a partial success.
|
||||
throw new Error(
|
||||
"editPageText: no edits were applied (nothing written). " +
|
||||
failed!.map((f) => `"${f.find}": ${f.reason}`).join("; "),
|
||||
);
|
||||
}
|
||||
|
||||
// Edits matched but produced no content change (identical document): report
|
||||
// a successful no-op — NOT a failure — and do not falsely claim a write.
|
||||
if (!wrote) {
|
||||
return {
|
||||
success: true,
|
||||
pageId,
|
||||
applied: results,
|
||||
failed,
|
||||
message: "No changes written (edits produced identical content).",
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
const result: any = {
|
||||
success: true,
|
||||
pageId,
|
||||
applied: results,
|
||||
failed,
|
||||
message:
|
||||
(failed?.length ?? 0)
|
||||
? `Applied ${results?.length ?? 0} edit(s); ${failed!.length} failed (see failed[]). Node ids and formatting preserved.`
|
||||
: "Text edits applied (node ids and formatting preserved).",
|
||||
verify: mutation.verify,
|
||||
};
|
||||
|
||||
// If any applied edit matched only after stripping markdown (the
|
||||
// normalized fallback), warn that editPageText preserved existing marks
|
||||
// and did NOT change formatting — so a caller who intended a formatting
|
||||
// change is pointed at patchNode.
|
||||
if (results?.some((r) => r.normalized === true)) {
|
||||
result.warning =
|
||||
"Some edits matched only after stripping markdown from your find string; " +
|
||||
"editPageText preserved existing marks (it did not change bold/strike/etc.). " +
|
||||
"If you intended a formatting change, use patchNode.";
|
||||
}
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace the block whose attrs.id === nodeId. Operates on the LIVE collab
|
||||
* document so comments and concurrent edits are preserved.
|
||||
*
|
||||
* Exactly one of `input.markdown` / `input.node` (#413):
|
||||
* - `markdown` (RECOMMENDED): the block is rewritten from a canonical markdown
|
||||
* fragment. The fragment may import to N blocks (a "1 -> N" splice: rewrite a
|
||||
* whole section in one call). The FIRST resulting block INHERITS the target's
|
||||
* `attrs.id` (so an existing comment anchoring the block by id survives); the
|
||||
* rest get FRESH ids. `^[...]` footnotes in the fragment are first-class:
|
||||
* their definitions merge into the page's TAIL footnote list (content-key
|
||||
* dedup + canonicalize), same machinery insertFootnote uses. REJECTED when
|
||||
* the TARGET block carries a table-cell attribute markdown cannot represent
|
||||
* (colspan/rowspan/colwidth/background) — use the table tools or `node`.
|
||||
* - `node`: a raw ProseMirror node for precise attr/mark work. The replacement
|
||||
* keeps the target id (if `node.attrs.id` is missing it is set to nodeId).
|
||||
*
|
||||
* #159 ambiguous-id semantics are unchanged: 0 matches -> "no node"; >1 matches
|
||||
* -> "ambiguous, refused" (nothing written), on BOTH paths — the markdown path
|
||||
* runs a dry `replaceNodeById` count first, so a duplicated id never splices.
|
||||
*/
|
||||
async patchNode(
|
||||
pageId: string,
|
||||
nodeId: string,
|
||||
input: { markdown?: string; node?: any },
|
||||
) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
// XOR: exactly one of markdown / node. Both optional in the schema; the
|
||||
// runtime enforces the recommendation ("markdown for prose, node for fine
|
||||
// work") without letting an ambiguous both-or-neither call through.
|
||||
const hasMd =
|
||||
input != null &&
|
||||
typeof input.markdown === "string" &&
|
||||
input.markdown.trim() !== "";
|
||||
const hasNode = input != null && input.node != null;
|
||||
if (hasMd === hasNode) {
|
||||
throw new Error(
|
||||
"patchNode: provide exactly one of `markdown` (recommended, for prose) " +
|
||||
"or `node` (a raw ProseMirror node, for precise attr/mark work)",
|
||||
);
|
||||
}
|
||||
|
||||
if (hasMd) {
|
||||
return this.patchNodeMarkdown(pageId, nodeId, input.markdown as string);
|
||||
}
|
||||
return this.patchNodeJson(pageId, nodeId, input.node);
|
||||
}
|
||||
|
||||
/**
|
||||
* patchNode with a raw ProseMirror `node` (the pre-#413 behavior). Replaces
|
||||
* EVERY node whose attrs.id === nodeId; the swapped-in node keeps the target
|
||||
* id. #159 ambiguity refused. Split out so the markdown path can reuse the
|
||||
* shared collab/guard plumbing without a giant branch.
|
||||
*/
|
||||
protected async patchNodeJson(pageId: string, nodeId: string, node: any) {
|
||||
if (!node || typeof node !== "object" || typeof node.type !== "string") {
|
||||
throw new Error(
|
||||
"patchNode: `node` must be an object with a string `type`",
|
||||
);
|
||||
}
|
||||
// Preserve the block id WITHOUT mutating the caller's object: build a local
|
||||
// copy whose attrs.id === nodeId (so the swapped-in node keeps the id of the
|
||||
// node it replaces).
|
||||
const target = {
|
||||
...node,
|
||||
attrs: {
|
||||
...(node.attrs && typeof node.attrs === "object" ? node.attrs : {}),
|
||||
},
|
||||
};
|
||||
if (target.attrs.id == null) {
|
||||
target.attrs.id = nodeId;
|
||||
}
|
||||
|
||||
// #409: fail fast on a malformed node SHAPE (a nested child with an
|
||||
// absent/unknown `type`, e.g. a text leaf written as `{"text":"foo"}` with
|
||||
// no `"type":"text"`) BEFORE opening a collab session or taking the page
|
||||
// lock — the root-only `typeof node.type === "string"` check above never
|
||||
// sees nested children, and the encoder's `Unknown node type: undefined`
|
||||
// would otherwise only surface after the connection.
|
||||
this.assertValidNodeShape("patchNode", target);
|
||||
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
// Track the replacement count in an outer var, reset per-transform, so a
|
||||
// collab retry recomputes it cleanly (mirrors replaceImage's pattern).
|
||||
let replaced = 0;
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
replaced = 0;
|
||||
const { doc: nd, replaced: r } = replaceNodeById(
|
||||
liveDoc,
|
||||
nodeId,
|
||||
target,
|
||||
);
|
||||
replaced = r;
|
||||
// 0 matches -> skip the write. >1 matches -> the id is AMBIGUOUS: Docmost
|
||||
// duplicates block ids on copy/paste (and copyPageContent writes them
|
||||
// verbatim), so replacing "the node with id X" would silently clobber
|
||||
// EVERY duplicate (#159). Refuse: skip the write and throw below so the
|
||||
// model re-targets with a more specific anchor instead of corrupting the
|
||||
// page. Only an unambiguous single match is written.
|
||||
if (replaced !== 1) return null;
|
||||
return nd;
|
||||
},
|
||||
);
|
||||
|
||||
// 0 -> "no node"; >1 -> "ambiguous, refused" (the transform already skipped
|
||||
// the write for any count !== 1). Single shared guard (#159, #185 review).
|
||||
assertUnambiguousMatch("patchNode", "replace", replaced, nodeId, pageId);
|
||||
|
||||
return { success: true, replaced, nodeId, verify: mutation.verify };
|
||||
}
|
||||
|
||||
/**
|
||||
* patchNode with a MARKDOWN fragment (#413). Imports the fragment through the
|
||||
* canonical importer, then 1 -> N splices the resulting blocks in place of the
|
||||
* target block on the LIVE collab doc:
|
||||
* - the FIRST block inherits the target's id; the rest get FRESH ids (minted
|
||||
* by the importer/id-remap, so neighbour blocks are untouched);
|
||||
* - `^[...]` footnote definitions merge into the page's tail list;
|
||||
* - REJECTED when the target block carries a markdown-unrepresentable table
|
||||
* attr (colspan/rowspan/colwidth/background) — guarding against silent loss;
|
||||
* - #159 ambiguity is enforced by a dry `replaceNodeById` count BEFORE the
|
||||
* splice, so a duplicated id never writes.
|
||||
*/
|
||||
protected async patchNodeMarkdown(
|
||||
pageId: string,
|
||||
nodeId: string,
|
||||
markdown: string,
|
||||
) {
|
||||
// Import the fragment up front (network-free, canonical) so a bad fragment
|
||||
// fails before any collab connection or page lock.
|
||||
const { blocks, definitions } = await importMarkdownFragment(markdown);
|
||||
|
||||
// The first imported block inherits the target id; the rest keep the fresh
|
||||
// ids the importer assigned. Build the thread now so it is stable across a
|
||||
// collab retry (the transform below is pure over its inputs).
|
||||
const threaded = blocks.map((b, i) => {
|
||||
if (i !== 0) return b;
|
||||
return {
|
||||
...b,
|
||||
attrs: {
|
||||
...(b && typeof b.attrs === "object" ? b.attrs : {}),
|
||||
id: nodeId,
|
||||
},
|
||||
};
|
||||
});
|
||||
|
||||
// Shape-validate every imported block up front (parity with the JSON path):
|
||||
// the importer only emits schema nodes, but the check is cheap insurance and
|
||||
// yields the same rich #409 diagnostics if the schema ever drifts.
|
||||
for (const b of threaded) {
|
||||
this.assertValidNodeShape("patchNode", b);
|
||||
}
|
||||
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
let replaced = 0;
|
||||
let guardAttrs: string | null = null;
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
replaced = 0;
|
||||
guardAttrs = null;
|
||||
|
||||
// #159: count matches with the same recursive walk the JSON path uses;
|
||||
// only an UNAMBIGUOUS single match may write. A dry count keeps the
|
||||
// ambiguity semantics identical across both paths.
|
||||
const { replaced: count } = replaceNodeById(liveDoc, nodeId, {
|
||||
type: "paragraph",
|
||||
});
|
||||
replaced = count;
|
||||
if (count !== 1) return null;
|
||||
|
||||
// Guard against SILENT LOSS: if the target block carries a table-cell
|
||||
// attribute markdown cannot represent (colspan/rowspan/colwidth/
|
||||
// background), refuse the markdown rewrite so those attrs are not
|
||||
// dropped. Simple tables (no such attrs) rewrite fine.
|
||||
const hit = getNodeByRef(liveDoc, nodeId);
|
||||
guardAttrs = hit ? findUnrepresentableTableAttrs(hit.node) : null;
|
||||
if (guardAttrs != null) return null;
|
||||
|
||||
// Re-mint any minted block id that collides with an existing page id
|
||||
// (skip index 0: its id is intentionally the target nodeId, unique by
|
||||
// the #159 dry-count above), so the 1 -> N splice stays page-wide unique.
|
||||
reassignCollidingBlockIds(liveDoc, threaded, 0);
|
||||
|
||||
// 1 -> N splice, then merge any fragment footnote definitions into the
|
||||
// page's tail list and re-derive canonical footnote numbering.
|
||||
const { doc: spliced } = replaceNodeByIdWithMany(
|
||||
liveDoc,
|
||||
nodeId,
|
||||
threaded,
|
||||
);
|
||||
return mergeFootnoteDefinitions(spliced, definitions);
|
||||
},
|
||||
);
|
||||
|
||||
// Surface the guard rejection with an actionable message (nothing written).
|
||||
if (guardAttrs != null) {
|
||||
throw new Error(
|
||||
`patchNode: the target block has table-cell attributes markdown cannot ` +
|
||||
`represent (${guardAttrs}) — a markdown rewrite would drop them. Use ` +
|
||||
`the table tools (tableUpdateCell/tableInsertRow) or pass a raw ` +
|
||||
`ProseMirror \`node\` instead of \`markdown\`.`,
|
||||
);
|
||||
}
|
||||
|
||||
// 0 -> "no node"; >1 -> "ambiguous, refused" (the transform skipped the write
|
||||
// for any count !== 1). Shared #159 guard, identical to the JSON path.
|
||||
assertUnambiguousMatch("patchNode", "replace", replaced, nodeId, pageId);
|
||||
|
||||
return {
|
||||
success: true,
|
||||
replaced,
|
||||
nodeId,
|
||||
blocks: threaded.length,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Insert content relative to an anchor (or append it at the top level).
|
||||
* Operates on the LIVE collab document so comments and concurrent edits are
|
||||
* preserved.
|
||||
*
|
||||
* Exactly one of `input.markdown` / `input.node` (#413):
|
||||
* - `markdown` (RECOMMENDED): a canonical markdown fragment. It may import to
|
||||
* SEVERAL blocks — they are inserted IN ORDER at the anchor. `^[...]`
|
||||
* footnote definitions merge into the page's tail list (same machinery as
|
||||
* insertFootnote). Every inserted block gets a fresh id.
|
||||
* - `node`: a raw ProseMirror node for precise attr/mark work, or to insert
|
||||
* table structure (a bare tableRow/tableCell/tableHeader — NOT expressible in
|
||||
* markdown, so those stay JSON-only).
|
||||
*
|
||||
* opts.position:
|
||||
* - "append": push the content at the end of the top-level content.
|
||||
* - "before"/"after": insert as a sibling of the anchor, just before/after it.
|
||||
* Exactly one of anchorNodeId / anchorText must be given; anchorNodeId
|
||||
* locates a node anywhere by attrs.id, anchorText matches the first top-level
|
||||
* block whose plain text includes it.
|
||||
*
|
||||
* Throws if the anchor cannot be found.
|
||||
*/
|
||||
async insertNode(
|
||||
pageId: string,
|
||||
input: { markdown?: string; node?: any },
|
||||
opts: {
|
||||
position: "before" | "after" | "append";
|
||||
anchorNodeId?: string;
|
||||
anchorText?: string;
|
||||
},
|
||||
) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
// XOR: exactly one of markdown / node (both optional in the schema).
|
||||
const hasMd =
|
||||
input != null &&
|
||||
typeof input.markdown === "string" &&
|
||||
input.markdown.trim() !== "";
|
||||
const hasNode = input != null && input.node != null;
|
||||
if (hasMd === hasNode) {
|
||||
throw new Error(
|
||||
"insertNode: provide exactly one of `markdown` (recommended, for prose) " +
|
||||
"or `node` (a raw ProseMirror node, for precise attr/mark work or table structure)",
|
||||
);
|
||||
}
|
||||
|
||||
if (
|
||||
!opts ||
|
||||
(opts.position !== "before" &&
|
||||
opts.position !== "after" &&
|
||||
opts.position !== "append")
|
||||
) {
|
||||
throw new Error(
|
||||
'insertNode: `position` must be one of "before", "after", "append"',
|
||||
);
|
||||
}
|
||||
if (opts.position === "before" || opts.position === "after") {
|
||||
// before/after require EXACTLY ONE anchor (an id or a text fragment).
|
||||
const hasId =
|
||||
typeof opts.anchorNodeId === "string" && opts.anchorNodeId.length > 0;
|
||||
const hasText =
|
||||
typeof opts.anchorText === "string" && opts.anchorText.length > 0;
|
||||
if (hasId === hasText) {
|
||||
throw new Error(
|
||||
`insertNode: position "${opts.position}" requires exactly one of anchorNodeId or anchorText`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Resolve the ordered list of blocks to insert plus any footnote definitions
|
||||
// to merge. The markdown path imports canonically (so an inserted block is
|
||||
// byte-identical to the same content in a full-page import); the node path is
|
||||
// a single block with no footnote merge (raw JSON `^[...]` is not touched).
|
||||
let blocks: any[];
|
||||
let definitions: any[] = [];
|
||||
if (hasMd) {
|
||||
const frag = await importMarkdownFragment(input.markdown as string);
|
||||
blocks = frag.blocks;
|
||||
definitions = frag.definitions;
|
||||
} else {
|
||||
const node = input.node;
|
||||
if (!node || typeof node !== "object" || typeof node.type !== "string") {
|
||||
throw new Error(
|
||||
"insertNode: `node` must be an object with a string `type`",
|
||||
);
|
||||
}
|
||||
blocks = [node];
|
||||
}
|
||||
|
||||
// #409: fail fast on a malformed node SHAPE (a nested child with an
|
||||
// absent/unknown `type`) BEFORE opening a collab session or taking the page
|
||||
// lock — the root-only check above never sees nested children.
|
||||
for (const b of blocks) {
|
||||
this.assertValidNodeShape("insertNode", b);
|
||||
}
|
||||
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
// Track insertion in an outer var, reset per-transform, so a collab retry
|
||||
// recomputes it cleanly (mirrors replaceImage's pattern).
|
||||
let inserted = false;
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
inserted = false;
|
||||
// Re-mint any minted block id that collides with an existing page id
|
||||
// (all inserted blocks are fresh, no skip) so the splice stays unique.
|
||||
if (hasMd) reassignCollidingBlockIds(liveDoc, blocks);
|
||||
// Single-block node path keeps `insertNodeRelative` (it owns the
|
||||
// structural table-node splicing); the markdown path uses the array
|
||||
// splice so N blocks land in order at one anchor.
|
||||
const res = hasMd
|
||||
? insertNodesRelative(liveDoc, blocks, opts)
|
||||
: insertNodeRelative(liveDoc, blocks[0], opts);
|
||||
inserted = res.inserted;
|
||||
if (!inserted) return null; // anchor not found -> skip the write entirely
|
||||
// Merge any fragment footnote definitions into the page tail list and
|
||||
// re-derive canonical numbering (no-op when there are none).
|
||||
return mergeFootnoteDefinitions(res.doc, definitions);
|
||||
},
|
||||
);
|
||||
|
||||
if (!inserted) {
|
||||
const anchorDesc = opts.anchorNodeId
|
||||
? `anchorNodeId "${opts.anchorNodeId}"`
|
||||
: `anchorText "${opts.anchorText}"`;
|
||||
// anchorText is matched against the block's literal RENDERED plain text;
|
||||
// markdown/emoji are tolerated only as a strip-and-retry fallback, so a
|
||||
// miss usually means the text differs from what's on the page.
|
||||
const hint = opts.anchorText
|
||||
? " anchorText must be the block's literal rendered plain text (no markdown wrappers or emoji); anchorNodeId from getPageJson is more reliable."
|
||||
: "";
|
||||
throw new Error(
|
||||
`insertNode: anchor not found (${anchorDesc}) on page ${pageId}.${hint}`,
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
inserted: true,
|
||||
position: opts.position,
|
||||
blocks: blocks.length,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove EVERY node whose attrs.id === nodeId (recursively, including nodes
|
||||
* nested in callouts/tables) from its parent content array. Operates on the
|
||||
* LIVE collab document so comments and concurrent edits are preserved.
|
||||
* Throws if no node matches.
|
||||
*/
|
||||
async deleteNode(pageId: string, nodeId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
// Track the deletion count in an outer var, reset per-transform, so a
|
||||
// collab retry recomputes it cleanly (mirrors replaceImage's pattern).
|
||||
let deleted = 0;
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
deleted = 0;
|
||||
const { doc: nd, deleted: d } = deleteNodeById(liveDoc, nodeId);
|
||||
deleted = d;
|
||||
// 0 matches -> skip the write. >1 matches -> the id is AMBIGUOUS (block
|
||||
// ids are duplicated on copy/paste, #159): deleting "the node with id X"
|
||||
// would silently remove EVERY duplicate. Refuse: skip the write and throw
|
||||
// below so the model re-targets. Only an unambiguous single match is
|
||||
// deleted.
|
||||
if (deleted !== 1) return null;
|
||||
return nd;
|
||||
},
|
||||
);
|
||||
|
||||
// 0 -> "no node"; >1 -> "ambiguous, refused" (the transform already skipped
|
||||
// the write for any count !== 1). Single shared guard (#159, #185 review).
|
||||
assertUnambiguousMatch("deleteNode", "delete", deleted, nodeId, pageId);
|
||||
|
||||
return { success: true, deleted, nodeId, verify: mutation.verify };
|
||||
}
|
||||
|
||||
/** Build the public share URL for a page. */
|
||||
}
|
||||
return NodesWriteMixin;
|
||||
}
|
||||
@@ -0,0 +1,655 @@
|
||||
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
|
||||
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
|
||||
// changed to a mixin factory. See client/context.ts for the shared base.
|
||||
import type { GConstructor, DocmostClientContext } from "./context.js";
|
||||
import FormData from "form-data";
|
||||
import axios, { AxiosInstance } from "axios";
|
||||
import { convertProseMirrorToMarkdown } from "../lib/markdown-converter.js";
|
||||
import {
|
||||
updatePageContentRealtime,
|
||||
replacePageContent,
|
||||
markdownToProseMirror,
|
||||
markdownToProseMirrorCanonical,
|
||||
mutatePageContent,
|
||||
assertYjsEncodable,
|
||||
MutationResult,
|
||||
} from "../lib/collaboration.js";
|
||||
import { footnoteWarningsField } from "../lib/footnote-analyze.js";
|
||||
import {
|
||||
serializeDocmostMarkdown,
|
||||
parseDocmostMarkdown,
|
||||
} from "../lib/markdown-document.js";
|
||||
import { diffDocs, summarizeChange } from "../lib/diff.js";
|
||||
import {
|
||||
blockText,
|
||||
walk,
|
||||
getList,
|
||||
insertMarkerAfter,
|
||||
setCalloutRange,
|
||||
noteItem,
|
||||
mdToInlineNodes,
|
||||
commentsToFootnotes,
|
||||
canonicalizeFootnotes,
|
||||
insertInlineFootnote,
|
||||
mergeFootnoteDefinitions,
|
||||
} from "../lib/transforms.js";
|
||||
import { normalizeAndMergeFootnotes } from "../lib/footnote-normalize-merge.js";
|
||||
import vm from "node:vm";
|
||||
|
||||
// Public method surface of PagesMixin (issue #450) — a NAMED type so the factory
|
||||
// return type is expressible in the emitted .d.ts (the anonymous mixin class
|
||||
// carries the base's protected shared state, which would otherwise trip TS4094).
|
||||
// Derived from the class below; `implements IPagesMixin` fails to compile on drift.
|
||||
export interface IPagesMixin {
|
||||
createPage(title: string, content: string, spaceId: string, parentPageId?: string): any;
|
||||
updatePage(pageId: string, content: string, title?: string): any;
|
||||
renamePage(pageId: string, title: string): any;
|
||||
movePage(pageId: string, parentPageId: string | null, position?: string): any;
|
||||
deletePage(pageId: string): any;
|
||||
sharePage(pageId: string, searchIndexing?: boolean): any;
|
||||
listShares(): any;
|
||||
unsharePage(pageId: string): any;
|
||||
exportPageMarkdown(pageId: string): Promise<string>;
|
||||
importPageMarkdown(pageId: string, fullMarkdown: string): Promise<any>;
|
||||
copyPageContent(sourcePageId: string, targetPageId: string): any;
|
||||
listPageHistory(pageId: string, cursor?: string): any;
|
||||
getPageHistory(historyId: string): any;
|
||||
restorePageVersion(historyId: string): any;
|
||||
diffPageVersions(pageId: string, from?: string, to?: string): any;
|
||||
}
|
||||
|
||||
export function PagesMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IPagesMixin> & TBase {
|
||||
abstract class PagesMixin extends Base implements IPagesMixin {
|
||||
/**
|
||||
* Create a new page with title and content.
|
||||
* Uses the /pages/import workaround (the only endpoint accepting content),
|
||||
* then moves the page and restores the exact title: the import endpoint
|
||||
* derives the title from the FILENAME and replaces spaces with
|
||||
* underscores, so we explicitly re-set it via /pages/update afterwards.
|
||||
*/
|
||||
async createPage(
|
||||
title: string,
|
||||
content: string,
|
||||
spaceId: string,
|
||||
parentPageId?: string,
|
||||
) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
if (parentPageId) {
|
||||
try {
|
||||
await this.getPage(parentPageId);
|
||||
} catch (e) {
|
||||
throw new Error(`Parent page with ID ${parentPageId} not found.`);
|
||||
}
|
||||
}
|
||||
|
||||
// 1. Create content via Import (using multipart/form-data).
|
||||
// Build a FRESH FormData per send attempt: a FormData body is a single-use
|
||||
// stream consumed on the first send, so it cannot be replayed by
|
||||
// this.client's response interceptor (replay fails with 'socket hang up').
|
||||
// Multipart re-auth is therefore done here with bare axios and an explicit
|
||||
// one-shot 401/403 retry that rebuilds the body.
|
||||
const fileContent = Buffer.from(content, "utf-8");
|
||||
const buildForm = () => {
|
||||
const form = new FormData();
|
||||
form.append("spaceId", spaceId);
|
||||
form.append("file", fileContent, {
|
||||
filename: `${title || "import"}.md`,
|
||||
contentType: "text/markdown",
|
||||
});
|
||||
return form;
|
||||
};
|
||||
|
||||
const importUrl = `${this.apiUrl}/pages/import`;
|
||||
let response;
|
||||
try {
|
||||
// Call buildForm() ONCE per attempt and reuse the instance for both
|
||||
// getHeaders() and the body so the Content-Type boundary matches the body.
|
||||
const form = buildForm();
|
||||
// Read the Authorization header from this.client's defaults (set by
|
||||
// login(), only ever deleted — never set to null) instead of building
|
||||
// `Bearer ${this.token}`: a concurrent JSON 401 can null this.token
|
||||
// mid-flight, which would otherwise produce a literal "Bearer null".
|
||||
// ensureAuthenticated() above guarantees login() ran, so the default
|
||||
// header exists here.
|
||||
response = await axios.post(importUrl, form, {
|
||||
headers: {
|
||||
...form.getHeaders(),
|
||||
Authorization: this.client.defaults.headers.common["Authorization"],
|
||||
},
|
||||
timeout: 60000,
|
||||
});
|
||||
} catch (error) {
|
||||
// On an expired-token auth error, re-login and retry exactly once with a
|
||||
// freshly-rebuilt FormData (the previous one was already consumed).
|
||||
if (
|
||||
axios.isAxiosError(error) &&
|
||||
(error.response?.status === 401 || error.response?.status === 403)
|
||||
) {
|
||||
await this.login();
|
||||
const form2 = buildForm();
|
||||
response = await axios.post(importUrl, form2, {
|
||||
headers: {
|
||||
...form2.getHeaders(),
|
||||
Authorization: this.client.defaults.headers.common["Authorization"],
|
||||
},
|
||||
timeout: 60000,
|
||||
});
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
const newPageId = (response.data?.data ?? response.data).id;
|
||||
|
||||
// 2. Move to parent if needed
|
||||
if (parentPageId) {
|
||||
await this.movePage(newPageId, parentPageId);
|
||||
}
|
||||
|
||||
// 3. Restore the exact title (import mangles spaces into underscores)
|
||||
if (title) {
|
||||
await this.client.post("/pages/update", { pageId: newPageId, title });
|
||||
}
|
||||
|
||||
const page = await this.getPage(newPageId);
|
||||
// Surface non-fatal footnote problems (dangling refs, empty/duplicate
|
||||
// definitions, markers in tables) so the agent can fix its markup (#166).
|
||||
return { ...page, ...footnoteWarningsField(content) };
|
||||
}
|
||||
|
||||
/**
|
||||
* Update a page's content from markdown and optionally its title.
|
||||
* NOTE: full re-import — block ids regenerate. For surgical changes
|
||||
* use editPageText / updatePageJson instead.
|
||||
*/
|
||||
async updatePage(pageId: string, content: string, title?: string) {
|
||||
await this.ensureAuthenticated();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260). The
|
||||
// REST /pages/update title write below keeps the agent-supplied id (the
|
||||
// server resolves a slugId there).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
// Write the BODY first, then the title (#159 split-brain). If the collab
|
||||
// body write fails (e.g. a persist timeout), the title must be left
|
||||
// UNTOUCHED so the page never ends up with a new title over its old body.
|
||||
// A title write failing AFTER a successful body is rarer (REST is fast) and
|
||||
// leaves correct content under a stale title — the lesser inconsistency.
|
||||
let collabToken = "";
|
||||
let mutation;
|
||||
try {
|
||||
collabToken = await this.getCollabTokenWithReauth();
|
||||
mutation = await updatePageContentRealtime(
|
||||
pageUuid,
|
||||
content,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
);
|
||||
} catch (error: any) {
|
||||
// Verbose diagnostics (incl. anything that could expose a token prefix)
|
||||
// are gated behind DEBUG; the thrown Error below carries no token data.
|
||||
if (process.env.DEBUG) {
|
||||
console.error(
|
||||
"Failed to update page content via realtime collaboration:",
|
||||
error,
|
||||
);
|
||||
const tokenPreview = collabToken
|
||||
? collabToken.substring(0, 15) + "..."
|
||||
: "null";
|
||||
console.error(`Collab token preview: ${tokenPreview}`);
|
||||
}
|
||||
throw new Error(`Failed to update page content: ${error.message}`);
|
||||
}
|
||||
|
||||
// Body persisted successfully — now it is safe to set the title.
|
||||
if (title) {
|
||||
await this.client.post("/pages/update", { pageId, title });
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
modified: true,
|
||||
message: "Page updated successfully.",
|
||||
pageId: pageId,
|
||||
verify: mutation.verify,
|
||||
// Non-fatal footnote diagnostics (#166); omitted when there are none.
|
||||
...footnoteWarningsField(content),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate a URL string against a scheme allowlist for a given context.
|
||||
*
|
||||
* The markdown link path enforces safe schemes via TipTap, but the raw
|
||||
* JSON path (updatePageJson) bypasses that — so this is the sanitization
|
||||
* choke point for ProseMirror JSON written directly by the caller.
|
||||
*
|
||||
* - "link": reject javascript:, vbscript:, data: (any scheme that can
|
||||
* execute or smuggle script when the href is clicked).
|
||||
* - "src": allow only http(s):, mailto:, /api/files paths, or a
|
||||
* scheme-less relative/absolute path; reject
|
||||
* javascript:/vbscript:/data:/file:.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Rename a page (change its title only) without touching or resending its
|
||||
* content. The slug is derived from the page record, not the body, so it is
|
||||
* left intact too.
|
||||
*/
|
||||
async renamePage(pageId: string, title: string) {
|
||||
await this.ensureAuthenticated();
|
||||
await this.client.post("/pages/update", { pageId, title });
|
||||
return { success: true, pageId, title };
|
||||
}
|
||||
|
||||
/**
|
||||
* Copy the WHOLE content of one page onto another, entirely server-side: the
|
||||
* source's ProseMirror document is read and written verbatim onto the target
|
||||
* via the live collab path, so the document never passes through the model.
|
||||
*
|
||||
* Only the target's BODY is replaced — its title and slug live on the page
|
||||
* record (not in the content), so they are untouched. The source page is not
|
||||
* modified at all.
|
||||
*/
|
||||
|
||||
async movePage(
|
||||
pageId: string,
|
||||
parentPageId: string | null,
|
||||
position?: string,
|
||||
) {
|
||||
await this.ensureAuthenticated();
|
||||
// Docmost requires position >= 5 chars.
|
||||
const validPosition = position || "a00000";
|
||||
|
||||
return this.client
|
||||
.post("/pages/move", {
|
||||
pageId,
|
||||
parentPageId,
|
||||
position: validPosition,
|
||||
})
|
||||
.then((res) => res.data);
|
||||
}
|
||||
|
||||
|
||||
async deletePage(pageId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
return this.client
|
||||
.post("/pages/delete", { pageId })
|
||||
.then((res) => res.data);
|
||||
}
|
||||
|
||||
// --- Comment methods (ported from upstream PR #3 by Max Nikitin) ---
|
||||
|
||||
/**
|
||||
* Normalize a comment's `content` into a ProseMirror doc object before
|
||||
* markdown conversion. createComment/updateComment send content as a
|
||||
* JSON.stringify(...) STRING, and the server stores it as-is, so on read it
|
||||
* comes back as a string. convertProseMirrorToMarkdown returns "" for a
|
||||
* string, so parse it first (guarded — fall back to the raw value on any
|
||||
* parse failure so a non-JSON legacy value is still handled gracefully).
|
||||
*/
|
||||
|
||||
/** Share a page publicly (idempotent) and return the public URL. */
|
||||
async sharePage(pageId: string, searchIndexing: boolean = true) {
|
||||
await this.ensureAuthenticated();
|
||||
const response = await this.client.post("/shares/create", {
|
||||
pageId,
|
||||
includeSubPages: false,
|
||||
searchIndexing,
|
||||
});
|
||||
const share = response.data?.data ?? response.data;
|
||||
const slugId = share.page?.slugId || (await this.getPageRaw(pageId)).slugId;
|
||||
return {
|
||||
shareId: share.id,
|
||||
key: share.key,
|
||||
pageId: share.pageId,
|
||||
publicUrl: this.shareUrl(share.key, slugId),
|
||||
searchIndexing: share.searchIndexing,
|
||||
};
|
||||
}
|
||||
|
||||
/** List all public shares in the workspace with their URLs. */
|
||||
|
||||
/** Build the public share URL for a page. */
|
||||
protected shareUrl(shareKey: string, slugId: string): string {
|
||||
return `${this.appUrl}/share/${shareKey}/p/${slugId}`;
|
||||
}
|
||||
|
||||
/** Share a page publicly (idempotent) and return the public URL. */
|
||||
|
||||
/** List all public shares in the workspace with their URLs. */
|
||||
async listShares() {
|
||||
const shares = await this.paginateAll("/shares", {});
|
||||
return shares.map((s: any) => ({
|
||||
shareId: s.id,
|
||||
key: s.key,
|
||||
pageId: s.pageId,
|
||||
pageTitle: s.page?.title,
|
||||
publicUrl: s.page?.slugId ? this.shareUrl(s.key, s.page.slugId) : null,
|
||||
searchIndexing: s.searchIndexing,
|
||||
createdAt: s.createdAt,
|
||||
}));
|
||||
}
|
||||
|
||||
/** Remove the public share of a page. */
|
||||
async unsharePage(pageId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
const shares = await this.listShares();
|
||||
const share = shares.find((s: any) => s.pageId === pageId);
|
||||
if (!share) {
|
||||
throw new Error(`Page ${pageId} is not shared.`);
|
||||
}
|
||||
await this.client.post("/shares/delete", { shareId: share.shareId });
|
||||
return { success: true, removedShareId: share.shareId, pageId };
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Export a page to a single self-contained Docmost-flavoured markdown file:
|
||||
* meta block + body (with inline comment anchors + diagrams) + comment
|
||||
* threads. Lossless round-trip target; see importPageMarkdown for the inverse.
|
||||
*/
|
||||
async exportPageMarkdown(pageId: string): Promise<string> {
|
||||
await this.ensureAuthenticated();
|
||||
const page = await this.getPageRaw(pageId);
|
||||
const body = page.content ? convertProseMirrorToMarkdown(page.content) : "";
|
||||
let comments: any[] = [];
|
||||
try {
|
||||
// Lossless export: include RESOLVED threads so the export -> import
|
||||
// round-trip preserves every comment. This is exactly why the active-only
|
||||
// filter is an opt-in (default false) on listComments.
|
||||
comments = (await this.listComments(pageId, true)).items;
|
||||
} catch (e) {
|
||||
// A comments fetch failure must not lose the body; export with [] and let
|
||||
// the caller see the (empty) comments block. Log under DEBUG only.
|
||||
if (process.env.DEBUG) console.error("export: listComments failed", e);
|
||||
}
|
||||
const meta = {
|
||||
version: 1,
|
||||
pageId: page.id,
|
||||
slugId: page.slugId,
|
||||
title: page.title,
|
||||
spaceId: page.spaceId,
|
||||
parentPageId: page.parentPageId ?? null,
|
||||
};
|
||||
return serializeDocmostMarkdown(meta, body, comments);
|
||||
}
|
||||
|
||||
/**
|
||||
* Import a self-contained Docmost markdown file back into a page. Parses out
|
||||
* the meta + comments metadata blocks, converts the body to ProseMirror
|
||||
* (restoring comment marks + diagrams from their inline HTML), and replaces
|
||||
* the page content. Comment THREAD records are NOT written to the server in
|
||||
* this version — they are preserved in the file and the inline marks are
|
||||
* re-applied so the highlights survive; managing comment records stays with
|
||||
* the comment tools/UI.
|
||||
*/
|
||||
async importPageMarkdown(pageId: string, fullMarkdown: string): Promise<any> {
|
||||
await this.ensureAuthenticated();
|
||||
const { meta, body, comments } = parseDocmostMarkdown(fullMarkdown);
|
||||
// PAGE import: canonicalize footnotes (see markdownToProseMirrorCanonical).
|
||||
const doc = await markdownToProseMirrorCanonical(body);
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
const mutation = await replacePageContent(
|
||||
pageUuid,
|
||||
doc,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
);
|
||||
// Collect distinct comment ids that actually became comment marks in the doc.
|
||||
const collectCommentIds = (node: any, acc: Set<string>): Set<string> => {
|
||||
if (!node || typeof node !== "object") return acc;
|
||||
if (Array.isArray(node.marks)) {
|
||||
for (const mk of node.marks) {
|
||||
if (mk && mk.type === "comment" && mk.attrs?.commentId) {
|
||||
acc.add(mk.attrs.commentId);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (Array.isArray(node.content)) {
|
||||
for (const child of node.content) collectCommentIds(child, acc);
|
||||
}
|
||||
return acc;
|
||||
};
|
||||
// Count reflects the comment marks present in the written document, so an id
|
||||
// that only appears as inert text (e.g. inside a fenced code block) is not
|
||||
// counted because it never becomes a comment mark.
|
||||
const anchoredIds = collectCommentIds(doc, new Set<string>());
|
||||
const result: any = {
|
||||
success: true,
|
||||
pageId,
|
||||
anchoredCommentCount: anchoredIds.size,
|
||||
commentsInFile: Array.isArray(comments) ? comments.length : 0,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
// Warn (non-fatal) if the file was exported from a DIFFERENT page.
|
||||
if (meta?.pageId && meta.pageId !== pageId) {
|
||||
result.warning = `File was exported from page ${meta.pageId} but is being imported into ${pageId}.`;
|
||||
}
|
||||
// Non-fatal footnote diagnostics (#166), analyzed on the BODY (the part after
|
||||
// the docmost:meta / docmost:comments blocks) — so a `[^x]`-like token inside
|
||||
// those JSON blocks never produces a false warning, while real markers in the
|
||||
// body do. `body` comes from parseDocmostMarkdown(fullMarkdown) above.
|
||||
Object.assign(result, footnoteWarningsField(body));
|
||||
return result;
|
||||
}
|
||||
|
||||
/**
|
||||
* Rename a page (change its title only) without touching or resending its
|
||||
* content. The slug is derived from the page record, not the body, so it is
|
||||
* left intact too.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Copy the WHOLE content of one page onto another, entirely server-side: the
|
||||
* source's ProseMirror document is read and written verbatim onto the target
|
||||
* via the live collab path, so the document never passes through the model.
|
||||
*
|
||||
* Only the target's BODY is replaced — its title and slug live on the page
|
||||
* record (not in the content), so they are untouched. The source page is not
|
||||
* modified at all.
|
||||
*/
|
||||
async copyPageContent(sourcePageId: string, targetPageId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
// A self-copy would be a no-op overwrite; reject it explicitly so a caller
|
||||
// mistake surfaces as a clear error rather than a silent round-trip.
|
||||
if (sourcePageId === targetPageId) {
|
||||
throw new Error(
|
||||
"copyPageContent: sourcePageId and targetPageId are the same page (no-op copy)",
|
||||
);
|
||||
}
|
||||
|
||||
const source = await this.getPageRaw(sourcePageId);
|
||||
const content = source?.content;
|
||||
if (
|
||||
!content ||
|
||||
typeof content !== "object" ||
|
||||
content.type !== "doc" ||
|
||||
!Array.isArray(content.content)
|
||||
) {
|
||||
throw new Error(
|
||||
`copyPageContent: source page ${sourcePageId} has no usable ProseMirror content to copy`,
|
||||
);
|
||||
}
|
||||
|
||||
// Defense-in-depth: run the same URL-scheme sanitizer the JSON write path
|
||||
// uses, so copying never lands a javascript:/data: href/src on the target
|
||||
// (parity with updatePageJson; harmless for already-stored source content).
|
||||
this.validateDocUrls(content);
|
||||
|
||||
// Defense-in-depth (#228): this is a FULL-document write, so canonicalize
|
||||
// footnotes before copying — a no-op on already-canonical source content, but
|
||||
// it guarantees a copy can never propagate a non-canonical footnote topology
|
||||
// to the target (parity with the other full-doc write paths).
|
||||
// #419: normalize + merge glyph-forked definitions before canonicalizing.
|
||||
const canonical = canonicalizeFootnotes(normalizeAndMergeFootnotes(content));
|
||||
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the TARGET collab doc by its canonical UUID, never the slugId (#260).
|
||||
const targetUuid = await this.resolvePageId(targetPageId);
|
||||
const mutation = await this.replacePage(
|
||||
targetUuid,
|
||||
canonical,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
);
|
||||
|
||||
return {
|
||||
success: true,
|
||||
sourcePageId,
|
||||
targetPageId,
|
||||
copiedNodes: canonical.content.length,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Surgical text edits: find/replace inside text nodes of the live
|
||||
* document. Preserves all block ids, marks, callouts and tables.
|
||||
*/
|
||||
|
||||
// --- Page history / diff / transform ---
|
||||
|
||||
/**
|
||||
* List the saved versions (history snapshots) of a page, newest first.
|
||||
* Docmost auto-snapshots on every save. Returns one cursor-paginated page of
|
||||
* results: `{ items, nextCursor }`. The history record's id field is `id`.
|
||||
*/
|
||||
async listPageHistory(pageId: string, cursor?: string) {
|
||||
await this.ensureAuthenticated();
|
||||
const payload: Record<string, any> = { pageId };
|
||||
if (cursor) payload.cursor = cursor;
|
||||
const response = await this.client.post("/pages/history", payload);
|
||||
const data = response.data?.data ?? response.data;
|
||||
return {
|
||||
items: data?.items ?? [],
|
||||
nextCursor: data?.meta?.nextCursor ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch a single page-history version including its lossless ProseMirror
|
||||
* `content`. The version also carries pageId/title/createdAt.
|
||||
*/
|
||||
async getPageHistory(historyId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
const response = await this.client.post("/pages/history/info", {
|
||||
historyId,
|
||||
});
|
||||
return response.data?.data ?? response.data;
|
||||
}
|
||||
|
||||
/**
|
||||
* "Restore" a version: Docmost has NO restore endpoint, so we take the
|
||||
* version's `content` and write it as the page's current content via the live
|
||||
* collab path (which itself creates a new history snapshot). Returns the
|
||||
* affected pageId and the source historyId.
|
||||
*/
|
||||
async restorePageVersion(historyId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
const version = await this.getPageHistory(historyId);
|
||||
if (
|
||||
!version ||
|
||||
!version.pageId ||
|
||||
!version.content ||
|
||||
typeof version.content !== "object"
|
||||
) {
|
||||
throw new Error(
|
||||
`restorePageVersion: history ${historyId} has no usable content`,
|
||||
);
|
||||
}
|
||||
// Defense-in-depth: sanitize URLs in the restored content (parity with the
|
||||
// JSON write path) before writing it back.
|
||||
this.validateDocUrls(version.content);
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// version.pageId is the page entity id (already a UUID); resolvePageId
|
||||
// short-circuits a UUID with no round-trip, so this is defensive only (#260).
|
||||
const pageUuid = await this.resolvePageId(version.pageId);
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
() => version.content,
|
||||
);
|
||||
return {
|
||||
pageId: version.pageId,
|
||||
restoredFrom: historyId,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Diff two versions of a page and return a Docmost-equivalent change set.
|
||||
* `from`/`to` each resolve to a ProseMirror doc:
|
||||
* - null / undefined / "current" -> the page's CURRENT content;
|
||||
* - any other string -> that historyId's content.
|
||||
* Returns the diff plus the resolved version metadata for each side.
|
||||
*/
|
||||
async diffPageVersions(pageId: string, from?: string, to?: string) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
const isCurrent = (v?: string) => v == null || v === "" || v === "current";
|
||||
|
||||
const resolveSide = async (
|
||||
v?: string,
|
||||
): Promise<{ doc: any; meta: any }> => {
|
||||
if (isCurrent(v)) {
|
||||
const raw = await this.getPageRaw(pageId);
|
||||
return {
|
||||
doc: raw.content || { type: "doc", content: [] },
|
||||
meta: {
|
||||
kind: "current",
|
||||
pageId,
|
||||
title: raw.title,
|
||||
updatedAt: raw.updatedAt,
|
||||
},
|
||||
};
|
||||
}
|
||||
const version = await this.getPageHistory(v as string);
|
||||
return {
|
||||
doc: version.content || { type: "doc", content: [] },
|
||||
meta: {
|
||||
kind: "history",
|
||||
historyId: version.id,
|
||||
pageId: version.pageId,
|
||||
title: version.title,
|
||||
createdAt: version.createdAt,
|
||||
},
|
||||
};
|
||||
};
|
||||
|
||||
const fromSide = await resolveSide(from);
|
||||
const toSide = await resolveSide(to);
|
||||
const diff = diffDocs(fromSide.doc, toSide.doc);
|
||||
return { from: fromSide.meta, to: toSide.meta, diff };
|
||||
}
|
||||
|
||||
/**
|
||||
* Edit a page by running an arbitrary user-supplied JS transform against the
|
||||
* live document, with a diff preview + page-history safety net.
|
||||
*
|
||||
* The transform string is evaluated as `(doc, ctx) => doc` inside a node:vm
|
||||
* sandbox: it gets ONLY `{ doc, ctx, structuredClone, console }` as globals,
|
||||
* a 5s timeout, and NO access to require/process/fs/network. It must return a
|
||||
* `{ type: "doc" }` node, which is validated structurally before any write.
|
||||
*
|
||||
* `ctx` exposes:
|
||||
* - comments: the page's comments (fetched before the live read);
|
||||
* - log: an array the transform can push diagnostics to (via console.log);
|
||||
* - consume(id): mark a comment id as consumed (for deleteComments);
|
||||
* - helpers: the transforms.ts primitives + commentsToFootnotes.
|
||||
*
|
||||
* Footnote convention used by the helpers: footnote markers are plain "[N]"
|
||||
* text in the body, and the notes are an orderedList under a heading whose
|
||||
* text is "Примечания переводчика".
|
||||
*
|
||||
* dryRun (default true): read the page's current content, run the transform,
|
||||
* and return `{ pushed:false, diff, log }` WITHOUT opening the collab socket.
|
||||
* Otherwise the transform runs atomically inside mutatePageContent, optionally
|
||||
* deletes consumed comments, and returns the new historyId + diff + log.
|
||||
*/
|
||||
}
|
||||
return PagesMixin;
|
||||
}
|
||||
@@ -0,0 +1,651 @@
|
||||
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
|
||||
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
|
||||
// changed to a mixin factory. See client/context.ts for the shared base.
|
||||
import type { GConstructor, DocmostClientContext } from "./context.js";
|
||||
import axios, { AxiosInstance } from "axios";
|
||||
import {
|
||||
filterWorkspace,
|
||||
filterSpace,
|
||||
filterPage,
|
||||
filterComment,
|
||||
filterSearchResult,
|
||||
} from "../lib/filters.js";
|
||||
import { convertProseMirrorToMarkdown } from "../lib/markdown-converter.js";
|
||||
import {
|
||||
collectInternalFileNodes,
|
||||
normalizeFileUrl,
|
||||
resolveInternalFilePath,
|
||||
} from "../lib/internal-file-urls.js";
|
||||
import { buildPageTree } from "../lib/tree.js";
|
||||
import {
|
||||
replaceNodeById,
|
||||
replaceNodeByIdWithMany,
|
||||
reassignCollidingBlockIds,
|
||||
deleteNodeById,
|
||||
assertUnambiguousMatch,
|
||||
insertNodeRelative,
|
||||
insertNodesRelative,
|
||||
blockPlainText,
|
||||
buildOutline,
|
||||
getNodeByRef,
|
||||
readTable,
|
||||
insertTableRow,
|
||||
deleteTableRow,
|
||||
updateTableCell,
|
||||
findInvalidNode,
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
import {
|
||||
importMarkdownFragment,
|
||||
canBeDocChild,
|
||||
findUnrepresentableTableAttrs,
|
||||
} from "../lib/markdown-fragment.js";
|
||||
import { searchInDoc, SearchOptions } from "../lib/page-search.js";
|
||||
import {
|
||||
blockText,
|
||||
walk,
|
||||
getList,
|
||||
insertMarkerAfter,
|
||||
setCalloutRange,
|
||||
noteItem,
|
||||
mdToInlineNodes,
|
||||
commentsToFootnotes,
|
||||
canonicalizeFootnotes,
|
||||
insertInlineFootnote,
|
||||
mergeFootnoteDefinitions,
|
||||
} from "../lib/transforms.js";
|
||||
|
||||
// Public method surface of ReadMixin (issue #450) — a NAMED type so the factory
|
||||
// return type is expressible in the emitted .d.ts (the anonymous mixin class
|
||||
// carries the base's protected shared state, which would otherwise trip TS4094).
|
||||
// Derived from the class below; `implements IReadMixin` fails to compile on drift.
|
||||
export interface IReadMixin {
|
||||
getWorkspace(): any;
|
||||
getSpaces(): any;
|
||||
listPages(spaceId?: string, limit?: number, tree?: boolean): any;
|
||||
getTree(spaceId: string, rootPageId?: string, maxDepth?: number): any;
|
||||
getPageContext(pageId: string): any;
|
||||
listSidebarPages(spaceId: string, pageId?: string): any;
|
||||
getPage(pageId: string): any;
|
||||
getPageJson(pageId: string): any;
|
||||
getOutline(pageId: string): any;
|
||||
getNode(pageId: string, nodeId: string, format?: "markdown" | "json"): any;
|
||||
searchInPage(pageId: string, query: string, opts?: SearchOptions): any;
|
||||
getTable(pageId: string, tableRef: string): any;
|
||||
search(query: string, spaceId?: string, limit?: number, opts?: { parentPageId?: string; titleOnly?: boolean }): any;
|
||||
}
|
||||
|
||||
export function ReadMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IReadMixin> & TBase {
|
||||
abstract class ReadMixin extends Base implements IReadMixin {
|
||||
async getWorkspace() {
|
||||
await this.ensureAuthenticated();
|
||||
const response = await this.client.post("/workspace/info", {});
|
||||
return {
|
||||
data: filterWorkspace(response.data?.data ?? response.data),
|
||||
success: response.data.success,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
async getSpaces() {
|
||||
const spaces = await this.paginateAll("/spaces", {});
|
||||
return spaces.map((space) => filterSpace(space));
|
||||
}
|
||||
|
||||
/**
|
||||
* List pages in one of two modes.
|
||||
*
|
||||
* Default (`tree` false): most recent pages by updatedAt (descending),
|
||||
* bounded. Fetching the whole space can exceed MCP response/time limits on
|
||||
* large instances, so a single bounded page of results is returned (default
|
||||
* 50, max 100) via the `/pages/recent` feed.
|
||||
*
|
||||
* Tree (`tree` true): DEPRECATED — prefer `getTree`, which shares this exact
|
||||
* code path (a single `/pages/tree` request via `enumerateSpacePages` +
|
||||
* `buildPageTree`) but returns the compact `{pageId, title, children?,
|
||||
* hasChildren?}` shape and supports `rootPageId`/`maxDepth`. This tree mode is
|
||||
* kept for backward compatibility; it REQUIRES `spaceId` (a page tree is
|
||||
* scoped to one space) and IGNORES `limit` — the whole hierarchy is returned.
|
||||
* It fetches the tree via `enumerateSpacePages`, which on the fork server
|
||||
* resolves to a single `/pages/tree` request returning the whole
|
||||
* permission-filtered flat page set (soft-deleted pages excluded
|
||||
* server-side); the cursor-BFS in `enumerateSpacePages` is only a fallback for
|
||||
* stock upstream servers that lack `/pages/tree`.
|
||||
*/
|
||||
async listPages(spaceId?: string, limit: number = 50, tree: boolean = false) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
if (tree) {
|
||||
if (!spaceId) {
|
||||
throw new Error(
|
||||
"listPages: tree mode requires a spaceId (a page tree is scoped to one space). Pass spaceId, or omit tree to get the recent-pages list.",
|
||||
);
|
||||
}
|
||||
const { pages } = await this.enumerateSpacePages(spaceId);
|
||||
return buildPageTree(pages);
|
||||
}
|
||||
|
||||
const clampedLimit = Math.max(1, Math.min(100, limit));
|
||||
const payload: Record<string, any> = { limit: clampedLimit, page: 1 };
|
||||
if (spaceId) payload.spaceId = spaceId;
|
||||
const response = await this.client.post("/pages/recent", payload);
|
||||
const data = response.data;
|
||||
const items = data.data?.items || data.items || [];
|
||||
return items.map((page: any) => filterPage(page));
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch a space's page hierarchy (or one subtree) as a nested tree in a SINGLE
|
||||
* request — the #443 `getTree` tool. Shares its whole code path with
|
||||
* `listPages(tree:true)`: `enumerateSpacePages` issues one `POST /pages/tree`
|
||||
* (with the cursor-BFS only as a fallback for stock upstream servers that lack
|
||||
* the endpoint), then `buildPageTree` nests the flat, permission-filtered,
|
||||
* position-ordered list. No second tree fetch, no per-node BFS.
|
||||
*
|
||||
* - `rootPageId` — restrict to that page's subtree; the server seeds the CTE
|
||||
* with the page itself, so the result is exactly ONE root (the page and its
|
||||
* descendants). Omit it for the whole space.
|
||||
* - `maxDepth` — trim the response to that many levels (roots = depth 1) to
|
||||
* save tokens; the server still returns everything in one request, the cut
|
||||
* is applied in `buildPageTree` AFTER the full tree is built. A node whose
|
||||
* children were cut carries `hasChildren: true` (source of truth = the flat
|
||||
* item's server `hasChildren`) so the caller can descend with a follow-up
|
||||
* `getTree(spaceId, rootPageId=that node)` call.
|
||||
*
|
||||
* Output nodes are `{pageId, title, children?, hasChildren?}` — only the UUID
|
||||
* `pageId` is exposed (never `slugId`/`icon`/`position`). Requires `spaceId`
|
||||
* (a page tree is scoped to one space).
|
||||
*/
|
||||
async getTree(spaceId: string, rootPageId?: string, maxDepth?: number) {
|
||||
await this.ensureAuthenticated();
|
||||
if (!spaceId) {
|
||||
throw new Error(
|
||||
"getTree: spaceId is required (a page tree is scoped to one space).",
|
||||
);
|
||||
}
|
||||
const { pages } = await this.enumerateSpacePages(spaceId, rootPageId);
|
||||
return buildPageTree(pages, { shape: "getTree", maxDepth });
|
||||
}
|
||||
|
||||
/**
|
||||
* "Where am I / what's around" for a single page — the #443 `getPageContext`
|
||||
* tool. Metadata only (no page content), using exactly TWO server requests:
|
||||
*
|
||||
* 1. `POST /pages/breadcrumbs` — a recursive CTE that walks UP from the page.
|
||||
* The server returns the chain root->page order (it `.reverse()`s the
|
||||
* child-first walk before responding), INCLUDING the page itself as the
|
||||
* LAST element. So the last element is the page and everything before it
|
||||
* is the ancestor chain root->parent. This carries the page's own title
|
||||
* and spaceId, so no extra page-info fetch is needed for a UUID input.
|
||||
* 2. `listSidebarPages(spaceId, pageId)` — the page's DIRECT children,
|
||||
* cursor-paginated (a page with >20 children returns ALL of them, no
|
||||
* dupes) and in sidebar `position` order, each carrying `hasChildren`.
|
||||
*
|
||||
* The input may be a slugId (agents copy them from URLs); it is run through
|
||||
* `resolvePageId` first, exactly like the other page tools. A UUID input adds
|
||||
* no request there (short-circuit), keeping the total at two; a slugId input
|
||||
* adds one unavoidable resolve round-trip.
|
||||
*
|
||||
* INVARIANT: only the UUID `pageId` is exposed anywhere — server `id` is
|
||||
* mapped to `pageId` and `slugId` is never leaked. A nonexistent/inaccessible
|
||||
* pageId makes the server 404/403, which propagates as a clear tool error
|
||||
* (never a hollow empty object).
|
||||
*/
|
||||
async getPageContext(pageId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
// Resolve a possibly-slugId input to the canonical UUID (no round-trip for a
|
||||
// UUID). Errors here (bad/inaccessible id) propagate as a clear tool error.
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
// Request 1: the ancestor chain, root->page, page included as the LAST item.
|
||||
const response = await this.client.post("/pages/breadcrumbs", {
|
||||
pageId: pageUuid,
|
||||
});
|
||||
const chain: any[] = (response.data?.data ?? response.data) ?? [];
|
||||
if (!Array.isArray(chain) || chain.length === 0) {
|
||||
// The endpoint always includes the page itself, so an empty chain means
|
||||
// the page is gone/inaccessible — surface a clear error, not {}.
|
||||
throw new Error(`getPageContext: page "${pageId}" not found or inaccessible`);
|
||||
}
|
||||
|
||||
// Split: the last element is the page, the rest (root->parent) are the
|
||||
// breadcrumbs. A root page has no ancestors -> breadcrumbs is [].
|
||||
const self = chain[chain.length - 1];
|
||||
const ancestors = chain.slice(0, -1);
|
||||
|
||||
const page = {
|
||||
pageId: self.id,
|
||||
title: self.title,
|
||||
spaceId: self.spaceId,
|
||||
};
|
||||
const breadcrumbs = ancestors.map((n: any) => ({
|
||||
pageId: n.id,
|
||||
title: n.title,
|
||||
}));
|
||||
|
||||
// Request 2: direct children in sidebar order, each with hasChildren.
|
||||
const childItems = await this.listSidebarPages(self.spaceId, pageUuid);
|
||||
const children = childItems.map((c: any) => ({
|
||||
pageId: c.id,
|
||||
title: c.title,
|
||||
hasChildren: Boolean(c.hasChildren),
|
||||
}));
|
||||
|
||||
return { page, breadcrumbs, children };
|
||||
}
|
||||
|
||||
/**
|
||||
* List sidebar pages for a space. With no pageId the request returns the
|
||||
* space ROOT pages; with a pageId it returns the direct CHILDREN of that
|
||||
* page. pageId is therefore optional and is only included in the POST body
|
||||
* when provided (an empty/undefined pageId would otherwise change the
|
||||
* semantics on the server).
|
||||
*/
|
||||
async listSidebarPages(spaceId: string, pageId?: string) {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
// Paginate via the server-issued cursor. The server switched from OFFSET
|
||||
// (`page`) to CURSOR (`cursor`/`nextCursor`) pagination, and the global
|
||||
// ValidationPipe(whitelist:true) SILENTLY STRIPS the obsolete `page` field
|
||||
// — so the old offset loop got the SAME first page every time (with
|
||||
// hasNextPage stuck true) and dropped every child beyond the first page.
|
||||
const MAX_PAGES = 50;
|
||||
let cursor: string | undefined;
|
||||
let allItems: any[] = [];
|
||||
let truncated = false;
|
||||
|
||||
for (let i = 0; i < MAX_PAGES; i++) {
|
||||
// limit: 100 is the server-side Max; cuts request count 5x vs the default 20.
|
||||
const payload: Record<string, any> = { spaceId, limit: 100 };
|
||||
// Only send pageId when scoping to a page's children; omit it for roots.
|
||||
if (pageId) payload.pageId = pageId;
|
||||
if (cursor) payload.cursor = cursor;
|
||||
|
||||
const data = (await this.client.post("/pages/sidebar-pages", payload)).data
|
||||
?.data;
|
||||
allItems = allItems.concat(data?.items ?? []);
|
||||
|
||||
// Advance strictly via the server-issued cursor; a missing/repeated cursor
|
||||
// means the protocol drifted again — stop instead of looping on page one.
|
||||
const next = data?.meta?.hasNextPage ? data?.meta?.nextCursor : null;
|
||||
if (!next || next === cursor) break;
|
||||
cursor = next;
|
||||
|
||||
// Reaching the ceiling with more pages still available means the child
|
||||
// list is truncated (mirrors paginateAll).
|
||||
if (i === MAX_PAGES - 1) truncated = true;
|
||||
}
|
||||
|
||||
// Warn on real truncation (ceiling hit while the server still had pages) so
|
||||
// the caller is not silently handed an incomplete child list.
|
||||
if (truncated) {
|
||||
console.warn(
|
||||
`listSidebarPages: children of "${pageId ?? spaceId}" truncated at the ${MAX_PAGES}-page cap; more pages exist on the server`,
|
||||
);
|
||||
}
|
||||
|
||||
return allItems;
|
||||
}
|
||||
|
||||
/**
|
||||
* Enumerate EVERY page in a space (or in a subtree, when rootPageId is given).
|
||||
*
|
||||
* Primary path (fork server): a SINGLE `POST /pages/tree` returns the whole
|
||||
* space (or a subtree) as a flat, permission-filtered list in one request, in
|
||||
* the exact node shape buildPageTree consumes. This replaces the old
|
||||
* per-node BFS, which issued N sidebar requests and — after the server moved
|
||||
* to cursor pagination — silently lost every child past the first sidebar
|
||||
* page (the obsolete `page` param was stripped by ValidationPipe).
|
||||
*
|
||||
* The subtree variant (rootPageId given) INCLUDES the root node itself
|
||||
* (getPageAndDescendants seeds with id = rootPageId), unlike the old BFS
|
||||
* which started from the root's children.
|
||||
*
|
||||
* Fallback path (stdio mode may target STOCK upstream Docmost, which lacks
|
||||
* `/pages/tree`): on a 404/405 it falls back to the cursor-based BFS below,
|
||||
* walking direct children via the fixed cursor listSidebarPages. Safeguards:
|
||||
* a `visited` Set of page ids prevents re-processing a node (cycles /
|
||||
* duplicate references), and a hard node cap bounds pathological trees so the
|
||||
* walk always terminates.
|
||||
*
|
||||
* Returns `{ pages, truncated }`. `truncated` is true ONLY when the fallback
|
||||
* BFS stopped at its MAX_NODES cap — the primary /pages/tree path is uncapped
|
||||
* and always returns the complete set, so it never reports truncation.
|
||||
*/
|
||||
protected async enumerateSpacePages(
|
||||
spaceId: string,
|
||||
rootPageId?: string,
|
||||
): Promise<{ pages: any[]; truncated: boolean }> {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
// Single request replaces the whole BFS: /pages/tree returns the full
|
||||
// permission-filtered flat page set of a space (or a subtree) at once. This
|
||||
// path is uncapped, so it is never truncated.
|
||||
const payload = rootPageId ? { pageId: rootPageId } : { spaceId };
|
||||
try {
|
||||
const response = await this.client.post("/pages/tree", payload);
|
||||
const pages = (response.data?.data ?? response.data)?.items ?? [];
|
||||
return { pages, truncated: false };
|
||||
} catch (e: any) {
|
||||
// Only fall back when the endpoint is absent (stock upstream Docmost);
|
||||
// any other error is a genuine failure and must propagate.
|
||||
if (
|
||||
!axios.isAxiosError(e) ||
|
||||
(e.response?.status !== 404 && e.response?.status !== 405)
|
||||
) {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback: cursor-based breadth-first walk via listSidebarPages.
|
||||
const MAX_NODES = 10000;
|
||||
const result: any[] = [];
|
||||
const visited = new Set<string>();
|
||||
|
||||
// Seed with the root node itself when scoping to a subtree, so its own
|
||||
// comments aren't dropped: the primary /pages/tree seeds
|
||||
// getPageAndDescendants with id = rootPageId (root included), but
|
||||
// listSidebarPages(spaceId, rootPageId) returns only the root's CHILDREN.
|
||||
// The `visited` set below prevents a double-add if the root also appears
|
||||
// among the children. getPageRaw returns a page whose id/title/spaceId are
|
||||
// exactly what buildPageTree and checkNewComments consume.
|
||||
if (rootPageId) {
|
||||
try {
|
||||
const root = await this.getPageRaw(rootPageId);
|
||||
if (root?.id) {
|
||||
result.push(root);
|
||||
visited.add(root.id);
|
||||
}
|
||||
} catch {
|
||||
// Non-fatal: if the root can't be read, fall through to children-only.
|
||||
}
|
||||
}
|
||||
|
||||
// Seed the queue with the starting level (subtree children or roots).
|
||||
const queue: any[] = await this.listSidebarPages(spaceId, rootPageId);
|
||||
|
||||
while (queue.length > 0 && result.length < MAX_NODES) {
|
||||
const node = queue.shift();
|
||||
if (!node || typeof node !== "object" || !node.id) continue;
|
||||
|
||||
// Skip already-seen ids to guard against cycles / duplicate references.
|
||||
if (visited.has(node.id)) continue;
|
||||
visited.add(node.id);
|
||||
|
||||
result.push(node);
|
||||
|
||||
if (node.hasChildren) {
|
||||
try {
|
||||
const children = await this.listSidebarPages(spaceId, node.id);
|
||||
for (const child of children) queue.push(child);
|
||||
} catch (e: any) {
|
||||
// A failure fetching one node's children must not abort the whole
|
||||
// walk: skip this branch and keep enumerating the rest.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Truncated only when the cap was hit with the queue still non-empty (real
|
||||
// truncation, not a natural end at exactly MAX_NODES).
|
||||
return {
|
||||
pages: result,
|
||||
truncated: result.length >= MAX_NODES && queue.length > 0,
|
||||
};
|
||||
}
|
||||
|
||||
/** Raw page info including the ProseMirror JSON content and slugId. */
|
||||
|
||||
async getPage(pageId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
const resultData = await this.getPageRaw(pageId);
|
||||
|
||||
// Agent read: hide resolved-comment anchors so the agent sees only active
|
||||
// discussions. Active anchors are kept. (The lossless exportPageMarkdown
|
||||
// round-trip deliberately does NOT pass this flag — resolved anchors there
|
||||
// must be preserved.)
|
||||
let content = resultData.content
|
||||
? convertProseMirrorToMarkdown(resultData.content, {
|
||||
dropResolvedCommentAnchors: true,
|
||||
})
|
||||
: "";
|
||||
|
||||
// Always fetch subpages to provide context to the agent
|
||||
let subpages: any[] = [];
|
||||
try {
|
||||
// `pageId` may be a slugId, but the sidebar-pages endpoint requires the
|
||||
// UUID; `resultData.id` holds the resolved UUID returned by getPageRaw.
|
||||
subpages = await this.listSidebarPages(resultData.spaceId, resultData.id);
|
||||
} catch (e: any) {
|
||||
console.warn("Failed to fetch subpages:", e);
|
||||
}
|
||||
|
||||
// Resolve subpages if the placeholder exists
|
||||
if (content && content.includes("{{SUBPAGES}}")) {
|
||||
if (subpages && subpages.length > 0) {
|
||||
const list = subpages
|
||||
.map((p: any) => `- [${p.title}](page:${p.id})`)
|
||||
.join("\n");
|
||||
content = content.replace("{{SUBPAGES}}", `### Subpages\n${list}`);
|
||||
} else {
|
||||
content = content.replace("{{SUBPAGES}}", "");
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
data: filterPage(resultData, content, subpages),
|
||||
success: true,
|
||||
};
|
||||
}
|
||||
|
||||
/** Page info + raw ProseMirror JSON content (lossless representation). */
|
||||
async getPageJson(pageId: string) {
|
||||
const data = await this.getPageRaw(pageId);
|
||||
return {
|
||||
id: data.id,
|
||||
slugId: data.slugId,
|
||||
title: data.title,
|
||||
parentPageId: data.parentPageId,
|
||||
spaceId: data.spaceId,
|
||||
updatedAt: data.updatedAt,
|
||||
content: data.content || { type: "doc", content: [] },
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch an INTERNAL Docmost file (authed loopback) for sandbox mirroring.
|
||||
* `src` is normalized to `/api/files/<id>/<file>`; `this.client.baseURL`
|
||||
* already ends in `/api`, so we strip the leading `/api` and request the
|
||||
* relative path with the client's Authorization header. Returns the raw bytes
|
||||
* and the response Content-Type (mime), defaulting to octet-stream.
|
||||
*
|
||||
* The fetch is size-bounded (hard 64 MiB ceiling) purely to protect memory;
|
||||
* the authoritative per-blob cap is enforced by the sandbox `put`. The path is
|
||||
* resolved via resolveInternalFilePath, which REJECTS (throws) any traversal
|
||||
* or percent-encoded src that would let an attacker-controlled `attrs.src`
|
||||
* escape `/api/files/` and reach another internal endpoint (SSRF). That throw
|
||||
* happens before this.client.get, so a malicious src is counted as a failed
|
||||
* mirror — it never reaches the network.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Compact outline of a page's top-level blocks (no full document body).
|
||||
* Cheap way to locate sections/tables and grab block ids before drilling in
|
||||
* with getNode / patchNode / insertNode.
|
||||
*/
|
||||
async getOutline(pageId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
const data = await this.getPageRaw(pageId);
|
||||
return {
|
||||
pageId,
|
||||
slugId: data.slugId,
|
||||
title: data.title,
|
||||
outline: buildOutline(data.content ?? { type: "doc", content: [] }),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch a single block for editing by reference: a block id (headings/
|
||||
* paragraphs/callouts/images), or `#<index>` to select a top-level block by its
|
||||
* outline index (the only way to reach tables/rows/cells, which carry no id).
|
||||
*
|
||||
* `format` (#413):
|
||||
* - `"markdown"` (DEFAULT): serialize the block via the canonical converter
|
||||
* (`{type:"doc",content:[node]}` -> `convertProseMirrorToMarkdown`) — a read
|
||||
* "for editing": pair it with `patchNode({markdown})` to rewrite the block.
|
||||
* Comment anchors (`<span data-comment-id>`, INCLUDING resolved ones) are
|
||||
* NOT stripped here (unlike getPage): losing them on write-back would
|
||||
* orphan the thread. Returns `{ ..., format:"markdown", markdown }`.
|
||||
* - `"json"`: return the raw ProseMirror subtree as-is (lossless; the previous
|
||||
* default). Returns `{ ..., format:"json", node }`.
|
||||
*
|
||||
* AUTO fallback: a type that cannot be a document top-level child
|
||||
* (tableRow/tableCell/tableHeader, addressed by `#<index>`) is NOT expressible
|
||||
* as a standalone markdown document, so a `"markdown"` request for such a node
|
||||
* transparently falls back to JSON with an explicit `format:"json"` field. The
|
||||
* check derives from the schema's `doc` contentMatch, so it tracks the schema.
|
||||
*/
|
||||
async getNode(
|
||||
pageId: string,
|
||||
nodeId: string,
|
||||
format: "markdown" | "json" = "markdown",
|
||||
) {
|
||||
await this.ensureAuthenticated();
|
||||
const data = await this.getPageRaw(pageId);
|
||||
const hit = getNodeByRef(
|
||||
data.content ?? { type: "doc", content: [] },
|
||||
nodeId,
|
||||
);
|
||||
if (!hit) {
|
||||
throw new Error(
|
||||
`getNode: no node found for "${nodeId}" on page ${pageId} (use a block id from getOutline, or "#<index>" for a top-level block such as a table)`,
|
||||
);
|
||||
}
|
||||
|
||||
// JSON requested (or a non-top-level type that markdown cannot represent as a
|
||||
// standalone document): return the subtree verbatim.
|
||||
if (format === "json" || !canBeDocChild(hit.type)) {
|
||||
return {
|
||||
pageId,
|
||||
ref: nodeId,
|
||||
path: hit.path,
|
||||
type: hit.type,
|
||||
format: "json" as const,
|
||||
node: hit.node,
|
||||
};
|
||||
}
|
||||
|
||||
// Markdown: wrap the node as a one-block doc and run the canonical converter.
|
||||
// Comment anchors are DELIBERATELY preserved (converter default) so a
|
||||
// getNode(markdown) -> edit -> patchNode(markdown) round trip does not orphan
|
||||
// a comment thread; this differs from getPage, which strips them.
|
||||
const markdown = convertProseMirrorToMarkdown({
|
||||
type: "doc",
|
||||
content: [hit.node],
|
||||
});
|
||||
return {
|
||||
pageId,
|
||||
ref: nodeId,
|
||||
path: hit.path,
|
||||
type: hit.type,
|
||||
format: "markdown" as const,
|
||||
markdown,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Find every occurrence of `query` on a page IN MEMORY, over the plain text of
|
||||
* each text container (reusing the same `getPageRaw` fetch as the other read
|
||||
* tools) — no server search endpoint, no whole-document round-trip through the
|
||||
* model. Returns `{ total, truncated, matches }`; each match carries a ref for
|
||||
* getNode/patchNode (the `#<index>` form resolves with getNode but NOT
|
||||
* patchNode — see SearchMatch.nodeId), plus the top-level block index and a
|
||||
* short context window used to build a unique text `selection` for
|
||||
* createComment (createComment has no nodeId param). The pure engine
|
||||
* (`searchInDoc`) owns the traversal, glue, the RE2 ReDoS-safe regex engine
|
||||
* and the empty-query / invalid-or-unsupported-regex errors.
|
||||
*/
|
||||
async searchInPage(pageId: string, query: string, opts: SearchOptions = {}) {
|
||||
await this.ensureAuthenticated();
|
||||
const data = await this.getPageRaw(pageId);
|
||||
const result = searchInDoc(
|
||||
data.content ?? { type: "doc", content: [] },
|
||||
query,
|
||||
opts,
|
||||
);
|
||||
return { pageId, query, ...result };
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a table as a matrix. `tableRef` is `#<index>` (from getOutline) or a
|
||||
* block id of any node inside the table. Returns the cell texts plus a
|
||||
* parallel cellIds matrix (each cell's first paragraph id, or null) so a
|
||||
* caller can patchNode a cell for rich-formatted edits. Throws when no table
|
||||
* resolves for the reference.
|
||||
*/
|
||||
async getTable(pageId: string, tableRef: string) {
|
||||
await this.ensureAuthenticated();
|
||||
const data = await this.getPageRaw(pageId);
|
||||
const t = readTable(data.content ?? { type: "doc", content: [] }, tableRef);
|
||||
if (!t) {
|
||||
throw new Error(
|
||||
`tableGet: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
|
||||
);
|
||||
}
|
||||
return {
|
||||
pageId,
|
||||
table: tableRef,
|
||||
rows: t.rows,
|
||||
cols: t.cols,
|
||||
path: t.path,
|
||||
cells: t.cells,
|
||||
cellIds: t.cellIds,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Insert a row of plain-text cells into a table on the LIVE collab document.
|
||||
* `tableRef` is `#<index>` or a block id inside the target table. `cells` is
|
||||
* padded to the table's column count (more cells than columns throws); `index`
|
||||
* is a 0-based insert position (omit/out-of-range to append). Throws when no
|
||||
* table resolves for the reference.
|
||||
*/
|
||||
|
||||
async search(
|
||||
query: string,
|
||||
spaceId?: string,
|
||||
limit?: number,
|
||||
opts: { parentPageId?: string; titleOnly?: boolean } = {},
|
||||
) {
|
||||
await this.ensureAuthenticated();
|
||||
// Opt into the #443 agent-lookup mode: `substring: true` turns on the hybrid
|
||||
// substring + FTS branch that returns path + snippet + score. A stock
|
||||
// upstream server strips these unknown DTO fields (whitelist:true) and
|
||||
// silently degrades to plain FTS — see the tool-registration comment.
|
||||
const payload: Record<string, any> = {
|
||||
query,
|
||||
spaceId,
|
||||
substring: true,
|
||||
};
|
||||
if (opts.parentPageId) payload.parentPageId = opts.parentPageId;
|
||||
if (opts.titleOnly) payload.titleOnly = true;
|
||||
// Clamp an optional caller-supplied limit into the lookup range (1..50)
|
||||
// before forwarding; omit it when not provided so the server default applies.
|
||||
if (limit !== undefined) {
|
||||
payload.limit = Math.max(1, Math.min(50, limit));
|
||||
}
|
||||
const response = await this.client.post("/search", payload);
|
||||
|
||||
// Normalize both response shapes: bare array and paginated { items: [...] }
|
||||
const data = response.data?.data;
|
||||
const items = Array.isArray(data) ? data : data?.items || [];
|
||||
const filteredItems = items.map((item: any) => filterSearchResult(item));
|
||||
|
||||
return {
|
||||
items: filteredItems,
|
||||
success: response.data?.success || false,
|
||||
};
|
||||
}
|
||||
|
||||
}
|
||||
return ReadMixin;
|
||||
}
|
||||
@@ -0,0 +1,225 @@
|
||||
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
|
||||
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
|
||||
// changed to a mixin factory. See client/context.ts for the shared base.
|
||||
import type { GConstructor, DocmostClientContext } from "./context.js";
|
||||
import {
|
||||
collectInternalFileNodes,
|
||||
normalizeFileUrl,
|
||||
resolveInternalFilePath,
|
||||
} from "../lib/internal-file-urls.js";
|
||||
|
||||
// Public method surface of StashMixin (issue #450) — a NAMED type so the factory
|
||||
// return type is expressible in the emitted .d.ts (the anonymous mixin class
|
||||
// carries the base's protected shared state, which would otherwise trip TS4094).
|
||||
// Derived from the class below; `implements IStashMixin` fails to compile on drift.
|
||||
export interface IStashMixin {
|
||||
stashPage(pageId: string): Promise<{ uri: string; sha256: string; size: number; images: { mirrored: number; failed: number }; }>;
|
||||
}
|
||||
|
||||
export function StashMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IStashMixin> & TBase {
|
||||
abstract class StashMixin extends Base implements IStashMixin {
|
||||
/**
|
||||
* Fetch an INTERNAL Docmost file (authed loopback) for sandbox mirroring.
|
||||
* `src` is normalized to `/api/files/<id>/<file>`; `this.client.baseURL`
|
||||
* already ends in `/api`, so we strip the leading `/api` and request the
|
||||
* relative path with the client's Authorization header. Returns the raw bytes
|
||||
* and the response Content-Type (mime), defaulting to octet-stream.
|
||||
*
|
||||
* The fetch is size-bounded (hard 64 MiB ceiling) purely to protect memory;
|
||||
* the authoritative per-blob cap is enforced by the sandbox `put`. The path is
|
||||
* resolved via resolveInternalFilePath, which REJECTS (throws) any traversal
|
||||
* or percent-encoded src that would let an attacker-controlled `attrs.src`
|
||||
* escape `/api/files/` and reach another internal endpoint (SSRF). That throw
|
||||
* happens before this.client.get, so a malicious src is counted as a failed
|
||||
* mirror — it never reaches the network.
|
||||
*/
|
||||
protected async fetchInternalFile(
|
||||
src: string,
|
||||
): Promise<{ buffer: Buffer; mime: string }> {
|
||||
const HARD_CEILING = 64 * 1024 * 1024; // 64 MiB memory guard
|
||||
const relPath = resolveInternalFilePath(src);
|
||||
const response = await this.client.get(relPath, {
|
||||
responseType: "arraybuffer",
|
||||
timeout: 30000,
|
||||
maxContentLength: HARD_CEILING,
|
||||
maxBodyLength: HARD_CEILING,
|
||||
});
|
||||
const buffer = Buffer.from(response.data);
|
||||
if (buffer.length === 0) {
|
||||
throw new Error(`Empty file response from "${src}"`);
|
||||
}
|
||||
const rawCt = response.headers?.["content-type"];
|
||||
const mime =
|
||||
typeof rawCt === "string" && rawCt.length > 0
|
||||
? rawCt.split(";")[0].trim().toLowerCase()
|
||||
: "application/octet-stream";
|
||||
return { buffer, mime };
|
||||
}
|
||||
|
||||
/**
|
||||
* Stash a page's full content into the in-RAM blob sandbox and return ONLY a
|
||||
* short anonymous URL — the body never enters the model context (this is the
|
||||
* whole point: ~30KB+ ProseMirror docs blow the model context if passed as a
|
||||
* tool argument). Every INTERNAL file/image src (the type-agnostic criterion,
|
||||
* so drawio/excalidraw/video/file nodes are covered too) is mirrored into the
|
||||
* sandbox and its `src` rewritten to the sandbox URL, so an external consumer
|
||||
* can fetch the images anonymously. External http(s) srcs are left untouched.
|
||||
*
|
||||
* Blobs live in RAM with a short TTL and are cleared on restart — consume the
|
||||
* URLs within the TTL and one uptime. A failed image fetch never aborts the
|
||||
* doc: the original src is kept and the failure counted.
|
||||
*
|
||||
* Returns { uri, sha256, size, images:{mirrored, failed} }. `uri` and `sha256`
|
||||
* are for the document blob; `sha256` is also the blob's ETag (integrity).
|
||||
*/
|
||||
async stashPage(pageId: string): Promise<{
|
||||
uri: string;
|
||||
sha256: string;
|
||||
size: number;
|
||||
images: { mirrored: number; failed: number };
|
||||
}> {
|
||||
if (!this.sandboxPut) {
|
||||
throw new Error(
|
||||
"stashPage is unavailable: the blob sandbox is not configured on this server",
|
||||
);
|
||||
}
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
// Stash the SAME shape getPageJson returns (id/title/.../content), with a
|
||||
// deep clone so the rewrite never mutates anything shared.
|
||||
const pageJson = await this.getPageJson(pageId);
|
||||
const cloned: any = structuredClone(pageJson);
|
||||
|
||||
// Group internal-file nodes by normalized src so each unique resource is
|
||||
// fetched + stored ONCE (dedup), and every node sharing that src points at
|
||||
// the one sandbox blob. Capture each node's ORIGINAL raw src per-node:
|
||||
// dedup groups nodes whose normalized src is equal even when their raw srcs
|
||||
// differ (e.g. `/api/files/...` vs the bare `/files/...`), so on a revert we
|
||||
// must restore each node's own original value, not the group key.
|
||||
const bySrc = new Map<string, Array<{ node: any; origSrc: string }>>();
|
||||
for (const node of collectInternalFileNodes(cloned.content)) {
|
||||
const origSrc = String(node.attrs.src);
|
||||
const src = normalizeFileUrl(origSrc);
|
||||
const entry = { node, origSrc };
|
||||
const group = bySrc.get(src);
|
||||
if (group) group.push(entry);
|
||||
else bySrc.set(src, [entry]);
|
||||
}
|
||||
|
||||
let mirrored = 0;
|
||||
let failed = 0;
|
||||
// Record every successful mirror so it can be (a) reverted if its blob gets
|
||||
// FIFO-evicted by a LATER put in this same stash, and (b) freed if the final
|
||||
// doc put throws.
|
||||
const mirrors: Array<{
|
||||
uri: string;
|
||||
entries: Array<{ node: any; origSrc: string }>;
|
||||
}> = [];
|
||||
const MAX_CONCURRENCY = 5;
|
||||
const groups = [...bySrc.entries()];
|
||||
for (let i = 0; i < groups.length; i += MAX_CONCURRENCY) {
|
||||
const batch = groups.slice(i, i + MAX_CONCURRENCY);
|
||||
await Promise.all(
|
||||
batch.map(async ([src, entries]) => {
|
||||
try {
|
||||
const { buffer, mime } = await this.fetchInternalFile(src);
|
||||
// put may throw if the blob exceeds the per-blob/total caps.
|
||||
const stored = this.sandboxPut!(buffer, mime);
|
||||
for (const entry of entries) entry.node.attrs.src = stored.uri;
|
||||
mirrors.push({ uri: stored.uri, entries });
|
||||
mirrored++;
|
||||
} catch (err) {
|
||||
// One bad/oversized image (or a rejected traversal src) must not
|
||||
// abort the document. Logged unconditionally (never the blob body),
|
||||
// matching the package's ungated console.warn convention.
|
||||
failed++;
|
||||
console.warn(
|
||||
`stashPage: failed to mirror "${src}": ${
|
||||
err instanceof Error ? err.message : String(err)
|
||||
}`,
|
||||
);
|
||||
}
|
||||
}),
|
||||
);
|
||||
}
|
||||
|
||||
// Revert one mirror's nodes to their original internal srcs and re-count it
|
||||
// as failed (its blob was FIFO-evicted before the doc could reference it
|
||||
// safely).
|
||||
const revertMirror = (mirror: {
|
||||
uri: string;
|
||||
entries: Array<{ node: any; origSrc: string }>;
|
||||
}) => {
|
||||
for (const entry of mirror.entries) entry.node.attrs.src = entry.origSrc;
|
||||
mirrored--;
|
||||
failed++;
|
||||
console.warn(
|
||||
`stashPage: mirrored blob ${mirror.uri} was evicted before the doc ` +
|
||||
`could safely reference it; reverted its src and counted it as failed`,
|
||||
);
|
||||
};
|
||||
|
||||
// Pre-put reconciliation: an image put earlier in THIS stash can FIFO-evict
|
||||
// an even-earlier image of the same stash. Drop those from the live set
|
||||
// first so the first serialized doc is already mostly correct.
|
||||
let liveMirrors = mirrors;
|
||||
if (this.sandboxHas) {
|
||||
liveMirrors = [];
|
||||
for (const mirror of mirrors) {
|
||||
if (this.sandboxHas(mirror.uri)) liveMirrors.push(mirror);
|
||||
else revertMirror(mirror);
|
||||
}
|
||||
}
|
||||
|
||||
// Put the document, then reconcile against eviction caused by the doc put
|
||||
// ITSELF (the doc is newest, FIFO drops oldest = this stash's images). Each
|
||||
// iteration reverts >=1 mirror, so the loop terminates (worst case: all
|
||||
// images reverted and the doc references no sandbox image URLs).
|
||||
let stored: { uri: string; sha256: string; size: number };
|
||||
for (;;) {
|
||||
const docBuf = Buffer.from(JSON.stringify(cloned), "utf8");
|
||||
let docStored: { uri: string; sha256: string; size: number };
|
||||
try {
|
||||
docStored = this.sandboxPut(docBuf, "application/json");
|
||||
} catch (err) {
|
||||
// The doc put failed (e.g. doc exceeds the cap). Free this op's image
|
||||
// blobs instead of leaking them in RAM for the whole TTL, then
|
||||
// re-throw.
|
||||
if (this.sandboxEvict) {
|
||||
for (const mirror of liveMirrors) this.sandboxEvict(mirror.uri);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
|
||||
if (!this.sandboxHas) {
|
||||
stored = docStored;
|
||||
break;
|
||||
}
|
||||
const evictedNow = liveMirrors.filter((m) => !this.sandboxHas!(m.uri));
|
||||
if (evictedNow.length === 0) {
|
||||
stored = docStored;
|
||||
break;
|
||||
}
|
||||
// The doc we just stored references now-dead blobs. Revert those nodes,
|
||||
// drop the stale doc blob, and loop to re-serialize + re-put the
|
||||
// corrected doc.
|
||||
for (const mirror of evictedNow) revertMirror(mirror);
|
||||
liveMirrors = liveMirrors.filter((m) => this.sandboxHas!(m.uri));
|
||||
if (this.sandboxEvict) this.sandboxEvict(docStored.uri);
|
||||
}
|
||||
return {
|
||||
uri: stored.uri,
|
||||
sha256: stored.sha256,
|
||||
size: stored.size,
|
||||
images: { mirrored, failed },
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Compact outline of a page's top-level blocks (no full document body).
|
||||
* Cheap way to locate sections/tables and grab block ids before drilling in
|
||||
* with getNode / patchNode / insertNode.
|
||||
*/
|
||||
}
|
||||
return StashMixin;
|
||||
}
|
||||
@@ -0,0 +1,291 @@
|
||||
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
|
||||
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
|
||||
// changed to a mixin factory. See client/context.ts for the shared base.
|
||||
import type { GConstructor, DocmostClientContext } from "./context.js";
|
||||
import {
|
||||
updatePageContentRealtime,
|
||||
replacePageContent,
|
||||
markdownToProseMirror,
|
||||
markdownToProseMirrorCanonical,
|
||||
mutatePageContent,
|
||||
assertYjsEncodable,
|
||||
MutationResult,
|
||||
} from "../lib/collaboration.js";
|
||||
import {
|
||||
replaceNodeById,
|
||||
replaceNodeByIdWithMany,
|
||||
reassignCollidingBlockIds,
|
||||
deleteNodeById,
|
||||
assertUnambiguousMatch,
|
||||
insertNodeRelative,
|
||||
insertNodesRelative,
|
||||
blockPlainText,
|
||||
buildOutline,
|
||||
getNodeByRef,
|
||||
readTable,
|
||||
insertTableRow,
|
||||
deleteTableRow,
|
||||
updateTableCell,
|
||||
findInvalidNode,
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
import { withPageLock, isUuid } from "../lib/page-lock.js";
|
||||
import {
|
||||
blockText,
|
||||
walk,
|
||||
getList,
|
||||
insertMarkerAfter,
|
||||
setCalloutRange,
|
||||
noteItem,
|
||||
mdToInlineNodes,
|
||||
commentsToFootnotes,
|
||||
canonicalizeFootnotes,
|
||||
insertInlineFootnote,
|
||||
mergeFootnoteDefinitions,
|
||||
} from "../lib/transforms.js";
|
||||
|
||||
// Public method surface of TablesMixin (issue #450) — a NAMED type so the factory
|
||||
// return type is expressible in the emitted .d.ts (the anonymous mixin class
|
||||
// carries the base's protected shared state, which would otherwise trip TS4094).
|
||||
// Derived from the class below; `implements ITablesMixin` fails to compile on drift.
|
||||
export interface ITablesMixin {
|
||||
insertFootnote(pageId: string, anchorText: string, text: string): any;
|
||||
tableInsertRow(pageId: string, tableRef: string, cells: string[], index?: number): any;
|
||||
tableDeleteRow(pageId: string, tableRef: string, index: number): any;
|
||||
tableUpdateCell(pageId: string, tableRef: string, row: number, col: number, text: string): any;
|
||||
}
|
||||
|
||||
export function TablesMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & ITablesMixin> & TBase {
|
||||
abstract class TablesMixin extends Base implements ITablesMixin {
|
||||
/**
|
||||
* AUTHOR-INLINE footnote insertion. The agent supplies only WHERE
|
||||
* (`anchorText`, a snippet of body text to attach the marker after) and WHAT
|
||||
* (`text`, the footnote content as markdown). Numbering and the bottom
|
||||
* `footnotesList` are derived deterministically server-side
|
||||
* (`insertInlineFootnote` -> `canonicalizeFootnotes`): the agent never sees,
|
||||
* assigns, or edits a footnote number or the list, so it CANNOT desync.
|
||||
*
|
||||
* Content DEDUP: when an existing definition has the same content, its id is
|
||||
* reused (one number, one definition, several references). The write is atomic
|
||||
* via `mutatePageContent` (single-writer, page-locked); if the anchor text is
|
||||
* not found the transform aborts with a clear error and no write happens.
|
||||
*/
|
||||
async insertFootnote(pageId: string, anchorText: string, text: string) {
|
||||
await this.ensureAuthenticated();
|
||||
if (!anchorText || !anchorText.trim()) {
|
||||
throw new Error("insertFootnote: anchorText is required");
|
||||
}
|
||||
if (text == null || `${text}`.trim() === "") {
|
||||
throw new Error("insertFootnote: text is required");
|
||||
}
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
let result: { footnoteId: string; reused: boolean } | null = null;
|
||||
const mutation = await this.mutatePage(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc: any) => {
|
||||
const r = insertInlineFootnote(liveDoc, { anchorText, text });
|
||||
if (!r.inserted) {
|
||||
// Abort the page-locked write by throwing: mutatePageContent does not
|
||||
// persist when the transform throws, so a missing anchor leaves the
|
||||
// page untouched (no partial write).
|
||||
throw new Error(
|
||||
`insertFootnote: anchor text not found: ${JSON.stringify(
|
||||
anchorText.slice(0, 80),
|
||||
)}`,
|
||||
);
|
||||
}
|
||||
result = { footnoteId: r.footnoteId, reused: r.reused };
|
||||
return r.doc;
|
||||
},
|
||||
);
|
||||
// The not-found path throws inside the transform (aborting mutatePage), so by
|
||||
// here `result` is always set.
|
||||
const r = result!;
|
||||
return {
|
||||
success: true,
|
||||
modified: true,
|
||||
pageId,
|
||||
footnoteId: r.footnoteId,
|
||||
reused: r.reused,
|
||||
message: r.reused
|
||||
? "Footnote inserted (reused an existing same-content definition)."
|
||||
: "Footnote inserted.",
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Page-locked write seam over collaboration.mutatePageContent. Production just
|
||||
* delegates; it exists as an overridable method so the insertFootnote wrapper
|
||||
* (transform abort-on-not-found + response shaping) can be unit-tested without
|
||||
* standing up a live Hocuspocus collab socket.
|
||||
*
|
||||
* SELF-RESOLVES the pageId to the canonical UUID (issue #449, "resolve-then-
|
||||
* lock"): every write must lock and key its CollabSession by the UUID, never a
|
||||
* raw slugId (#260). resolvePageId is cached/idempotent, so a caller that
|
||||
* already resolved pays no extra round-trip; centralizing it here means a
|
||||
* caller that reaches this seam with a raw slugId still locks correctly instead
|
||||
* of silently splitting the mutex key. withPageLock also asserts the key is a
|
||||
* UUID as a hard backstop.
|
||||
*/
|
||||
|
||||
/**
|
||||
* Insert a row of plain-text cells into a table on the LIVE collab document.
|
||||
* `tableRef` is `#<index>` or a block id inside the target table. `cells` is
|
||||
* padded to the table's column count (more cells than columns throws); `index`
|
||||
* is a 0-based insert position (omit/out-of-range to append). Throws when no
|
||||
* table resolves for the reference.
|
||||
*/
|
||||
async tableInsertRow(
|
||||
pageId: string,
|
||||
tableRef: string,
|
||||
cells: string[],
|
||||
index?: number,
|
||||
) {
|
||||
await this.ensureAuthenticated();
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
// Track insertion in an outer var, reset per-transform, so a collab retry
|
||||
// recomputes it cleanly (mirrors insertNode's pattern).
|
||||
let inserted = false;
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
inserted = false;
|
||||
const { doc: nd, inserted: ins } = insertTableRow(
|
||||
liveDoc,
|
||||
tableRef,
|
||||
cells,
|
||||
index,
|
||||
);
|
||||
inserted = ins;
|
||||
if (!inserted) return null; // table not found -> skip the write entirely
|
||||
return nd;
|
||||
},
|
||||
);
|
||||
|
||||
if (!inserted) {
|
||||
throw new Error(
|
||||
`tableInsertRow: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
|
||||
);
|
||||
}
|
||||
return {
|
||||
success: true,
|
||||
table: tableRef,
|
||||
inserted: true,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Delete the row at 0-based `index` from a table on the LIVE collab document.
|
||||
* `tableRef` is `#<index>` or a block id inside the target table. The helper's
|
||||
* out-of-range and last-row errors propagate; a missing table throws here.
|
||||
*/
|
||||
async tableDeleteRow(pageId: string, tableRef: string, index: number) {
|
||||
await this.ensureAuthenticated();
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
let deleted = false;
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
deleted = false;
|
||||
const { doc: nd, deleted: del } = deleteTableRow(
|
||||
liveDoc,
|
||||
tableRef,
|
||||
index,
|
||||
);
|
||||
deleted = del;
|
||||
if (!deleted) return null; // table not found -> skip the write entirely
|
||||
return nd;
|
||||
},
|
||||
);
|
||||
|
||||
if (!deleted) {
|
||||
throw new Error(
|
||||
`tableDeleteRow: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
|
||||
);
|
||||
}
|
||||
return {
|
||||
success: true,
|
||||
table: tableRef,
|
||||
deleted: true,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Set the plain-text content of cell `[row, col]` (0-based) in a table on the
|
||||
* LIVE collab document, replacing the cell's content with a single text
|
||||
* paragraph (the cell's first-paragraph id is preserved). `tableRef` is
|
||||
* `#<index>` or a block id inside the target table. The helper's out-of-range
|
||||
* error propagates; a missing table throws here.
|
||||
*/
|
||||
async tableUpdateCell(
|
||||
pageId: string,
|
||||
tableRef: string,
|
||||
row: number,
|
||||
col: number,
|
||||
text: string,
|
||||
) {
|
||||
await this.ensureAuthenticated();
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
|
||||
let updated = false;
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
(liveDoc) => {
|
||||
updated = false;
|
||||
const { doc: nd, updated: upd } = updateTableCell(
|
||||
liveDoc,
|
||||
tableRef,
|
||||
row,
|
||||
col,
|
||||
text,
|
||||
);
|
||||
updated = upd;
|
||||
if (!updated) return null; // table not found -> skip the write entirely
|
||||
return nd;
|
||||
},
|
||||
);
|
||||
|
||||
if (!updated) {
|
||||
throw new Error(
|
||||
`tableUpdateCell: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
|
||||
);
|
||||
}
|
||||
return {
|
||||
success: true,
|
||||
table: tableRef,
|
||||
row,
|
||||
col,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a new page with title and content.
|
||||
* Uses the /pages/import workaround (the only endpoint accepting content),
|
||||
* then moves the page and restores the exact title: the import endpoint
|
||||
* derives the title from the FILENAME and replaces spaces with
|
||||
* underscores, so we explicitly re-set it via /pages/update afterwards.
|
||||
*/
|
||||
}
|
||||
return TablesMixin;
|
||||
}
|
||||
@@ -0,0 +1,232 @@
|
||||
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
|
||||
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
|
||||
// changed to a mixin factory. See client/context.ts for the shared base.
|
||||
import type { GConstructor, DocmostClientContext } from "./context.js";
|
||||
import {
|
||||
updatePageContentRealtime,
|
||||
replacePageContent,
|
||||
markdownToProseMirror,
|
||||
markdownToProseMirrorCanonical,
|
||||
mutatePageContent,
|
||||
assertYjsEncodable,
|
||||
MutationResult,
|
||||
} from "../lib/collaboration.js";
|
||||
import { diffDocs, summarizeChange } from "../lib/diff.js";
|
||||
import {
|
||||
blockText,
|
||||
walk,
|
||||
getList,
|
||||
insertMarkerAfter,
|
||||
setCalloutRange,
|
||||
noteItem,
|
||||
mdToInlineNodes,
|
||||
commentsToFootnotes,
|
||||
canonicalizeFootnotes,
|
||||
insertInlineFootnote,
|
||||
mergeFootnoteDefinitions,
|
||||
} from "../lib/transforms.js";
|
||||
import { normalizeAndMergeFootnotes } from "../lib/footnote-normalize-merge.js";
|
||||
import vm from "node:vm";
|
||||
|
||||
// Public method surface of TransformsMixin (issue #450) — a NAMED type so the factory
|
||||
// return type is expressible in the emitted .d.ts (the anonymous mixin class
|
||||
// carries the base's protected shared state, which would otherwise trip TS4094).
|
||||
// Derived from the class below; `implements ITransformsMixin` fails to compile on drift.
|
||||
export interface ITransformsMixin {
|
||||
transformPage(pageId: string, transformJs: string, opts?: { dryRun?: boolean; deleteComments?: boolean }): any;
|
||||
}
|
||||
|
||||
export function TransformsMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & ITransformsMixin> & TBase {
|
||||
abstract class TransformsMixin extends Base implements ITransformsMixin {
|
||||
/**
|
||||
* Edit a page by running an arbitrary user-supplied JS transform against the
|
||||
* live document, with a diff preview + page-history safety net.
|
||||
*
|
||||
* The transform string is evaluated as `(doc, ctx) => doc` inside a node:vm
|
||||
* sandbox: it gets ONLY `{ doc, ctx, structuredClone, console }` as globals,
|
||||
* a 5s timeout, and NO access to require/process/fs/network. It must return a
|
||||
* `{ type: "doc" }` node, which is validated structurally before any write.
|
||||
*
|
||||
* `ctx` exposes:
|
||||
* - comments: the page's comments (fetched before the live read);
|
||||
* - log: an array the transform can push diagnostics to (via console.log);
|
||||
* - consume(id): mark a comment id as consumed (for deleteComments);
|
||||
* - helpers: the transforms.ts primitives + commentsToFootnotes.
|
||||
*
|
||||
* Footnote convention used by the helpers: footnote markers are plain "[N]"
|
||||
* text in the body, and the notes are an orderedList under a heading whose
|
||||
* text is "Примечания переводчика".
|
||||
*
|
||||
* dryRun (default true): read the page's current content, run the transform,
|
||||
* and return `{ pushed:false, diff, log }` WITHOUT opening the collab socket.
|
||||
* Otherwise the transform runs atomically inside mutatePageContent, optionally
|
||||
* deletes consumed comments, and returns the new historyId + diff + log.
|
||||
*/
|
||||
async transformPage(
|
||||
pageId: string,
|
||||
transformJs: string,
|
||||
opts: { dryRun?: boolean; deleteComments?: boolean } = {},
|
||||
) {
|
||||
const dryRun = opts.dryRun ?? true;
|
||||
const deleteComments = opts.deleteComments ?? false;
|
||||
|
||||
await this.ensureAuthenticated();
|
||||
// Full feed (incl. resolved): a page transform (e.g. comments -> footnotes)
|
||||
// must operate on every comment, so it opts into the unfiltered feed.
|
||||
const comments = (await this.listComments(pageId, true)).items;
|
||||
|
||||
// ctx handed to the sandbox. consume() records ids; helpers are the pure
|
||||
// transform primitives. log is captured from console.log inside the sandbox.
|
||||
const ctx = {
|
||||
comments,
|
||||
log: [] as string[],
|
||||
consumed: new Set<string>(),
|
||||
consume(id: string) {
|
||||
this.consumed.add(id);
|
||||
},
|
||||
helpers: {
|
||||
blockText,
|
||||
walk,
|
||||
getList,
|
||||
insertMarkerAfter,
|
||||
setCalloutRange,
|
||||
noteItem,
|
||||
mdToInlineNodes,
|
||||
commentsToFootnotes,
|
||||
canonicalizeFootnotes,
|
||||
insertInlineFootnote,
|
||||
},
|
||||
};
|
||||
|
||||
// Captured oldDoc / newDoc for the diff (set inside runTransform).
|
||||
let oldDoc: any;
|
||||
let newDoc: any;
|
||||
|
||||
// SYNCHRONOUS transform runner — safe to call inside mutatePageContent's
|
||||
// onSynced (no await between the live read and the write).
|
||||
const runTransform = (liveDoc: any): any => {
|
||||
oldDoc = structuredClone(liveDoc);
|
||||
const sandbox: Record<string, any> = {
|
||||
doc: structuredClone(liveDoc),
|
||||
ctx,
|
||||
structuredClone,
|
||||
console: {
|
||||
log: (...a: any[]) => ctx.log.push(a.map((x) => String(x)).join(" ")),
|
||||
},
|
||||
};
|
||||
// Wrap the provided string in parentheses so both an expression-arrow
|
||||
// (`(doc, ctx) => {...}`) and a parenthesized function work. Run it in a
|
||||
// fresh context with no require/process/module so the transform cannot
|
||||
// touch fs/network/process. 5s wall-clock timeout.
|
||||
let fn: any;
|
||||
try {
|
||||
fn = vm.runInNewContext("(" + transformJs + ")", sandbox, {
|
||||
timeout: 5000,
|
||||
});
|
||||
} catch (e: any) {
|
||||
throw new Error(`transform did not compile: ${e?.message ?? e}`);
|
||||
}
|
||||
if (typeof fn !== "function") {
|
||||
throw new Error(
|
||||
"transform must evaluate to a function (doc, ctx) => doc",
|
||||
);
|
||||
}
|
||||
const raw = vm.runInNewContext(
|
||||
"f(d, c)",
|
||||
{ f: fn, d: sandbox.doc, c: ctx },
|
||||
{ timeout: 5000 },
|
||||
);
|
||||
if (
|
||||
!raw ||
|
||||
typeof raw !== "object" ||
|
||||
raw.type !== "doc" ||
|
||||
!Array.isArray(raw.content)
|
||||
) {
|
||||
throw new Error(
|
||||
'transform must return a ProseMirror doc node ({ type:"doc", content:[...] })',
|
||||
);
|
||||
}
|
||||
// Validate the RAW transform output FIRST (structure — including the
|
||||
// MAX_DEPTH guard — and URLs), mirroring updatePageJson. The canonicalizer
|
||||
// recurses without a depth limiter, so validating after it would turn a
|
||||
// too-deep doc into an opaque "Maximum call stack size exceeded" instead of
|
||||
// the intended "nesting exceeds the maximum depth" error.
|
||||
this.validateDocStructure(raw);
|
||||
this.validateDocUrls(raw);
|
||||
// Auto-canonicalize footnotes after the transform (idempotent): no write
|
||||
// path can leave footnotes out of order / orphaned / in a raw `[^id]`
|
||||
// block. In a dryRun preview this may surface footnote edits the script
|
||||
// author did not write (the canonicalizer tidied them) — that is expected.
|
||||
// #419: normalize + merge glyph-forked definitions before canonicalizing.
|
||||
const result = canonicalizeFootnotes(normalizeAndMergeFootnotes(raw));
|
||||
newDoc = result;
|
||||
return result;
|
||||
};
|
||||
|
||||
if (dryRun) {
|
||||
// Preview only: run against the current REST snapshot, never open the
|
||||
// socket. oldDoc/newDoc are captured by runTransform.
|
||||
const raw = await this.getPageRaw(pageId);
|
||||
const current = raw.content || { type: "doc", content: [] };
|
||||
runTransform(current);
|
||||
// Run an independent Yjs-encodability check (same sanitize + schema as the
|
||||
// apply path), so the preview fails with the same descriptive error when
|
||||
// the doc is not encodable instead of returning a misleadingly-green diff.
|
||||
assertYjsEncodable(newDoc);
|
||||
return {
|
||||
pushed: false,
|
||||
diff: diffDocs(oldDoc, newDoc),
|
||||
log: ctx.log,
|
||||
};
|
||||
}
|
||||
|
||||
// Apply atomically against the live doc.
|
||||
const collabToken = await this.getCollabTokenWithReauth();
|
||||
// Open the collab doc by the canonical UUID, never the slugId (#260).
|
||||
const pageUuid = await this.resolvePageId(pageId);
|
||||
const mutation = await mutatePageContent(
|
||||
pageUuid,
|
||||
collabToken,
|
||||
this.apiUrl,
|
||||
runTransform,
|
||||
);
|
||||
|
||||
// Optionally delete consumed comments (best-effort; a delete failure must
|
||||
// not undo the successful write).
|
||||
const deletedComments: string[] = [];
|
||||
if (deleteComments) {
|
||||
for (const id of ctx.consumed) {
|
||||
try {
|
||||
await this.deleteComment(id);
|
||||
deletedComments.push(id);
|
||||
} catch (e) {
|
||||
if (process.env.DEBUG) {
|
||||
console.error(`transform: failed to delete comment ${id}:`, e);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Fetch the newest historyId (Docmost snapshots on the write above).
|
||||
let historyId: string | null = null;
|
||||
try {
|
||||
const hist = await this.listPageHistory(pageId);
|
||||
historyId = hist.items?.[0]?.id ?? null;
|
||||
} catch (e) {
|
||||
if (process.env.DEBUG) {
|
||||
console.error("transform: failed to fetch history id:", e);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
pushed: true,
|
||||
historyId,
|
||||
diff: diffDocs(oldDoc, newDoc),
|
||||
deletedComments,
|
||||
log: ctx.log,
|
||||
verify: mutation.verify,
|
||||
};
|
||||
}
|
||||
}
|
||||
return TransformsMixin;
|
||||
}
|
||||
@@ -57,17 +57,14 @@ export const DEFAULT_COMMENT_SIGNAL_DEBOUNCE_MS = 20_000;
|
||||
|
||||
/**
|
||||
* Tools whose OWN result must NOT carry the signal — it would be tautological
|
||||
* (the agent is already looking at comments) and noisy. Listed in BOTH the
|
||||
* standalone MCP snake_case names AND the in-app camelCase keys so a single set
|
||||
* covers both surfaces (the signal text itself uses the camelCase `listComments`
|
||||
* per roadmap #412). `getComment` (single fetch) is intentionally NOT excluded.
|
||||
* (the agent is already looking at comments) and noisy. Since issue #412 both
|
||||
* the standalone MCP surface and the in-app agent use the same camelCase tool
|
||||
* names, so a single set of camelCase names covers both surfaces. `getComment`
|
||||
* (single fetch) is intentionally NOT excluded.
|
||||
*/
|
||||
export const COMMENT_SIGNAL_EXCLUDED_TOOLS: ReadonlySet<string> = new Set([
|
||||
"list_comments",
|
||||
"listComments",
|
||||
"check_new_comments",
|
||||
"checkNewComments",
|
||||
"create_comment",
|
||||
"createComment",
|
||||
]);
|
||||
|
||||
|
||||
+63
-30
@@ -58,7 +58,7 @@ export type {
|
||||
CommentSignalTrackerOptions,
|
||||
} from "./comment-signal.js";
|
||||
// Re-export the pure, no-network draw.io helpers (#424) so the in-app AI-SDK
|
||||
// service can wire drawio_shapes / drawio_guide off the loaded module. These are
|
||||
// service can wire drawioShapes / drawioGuide off the loaded module. These are
|
||||
// NOT client methods (no page/backend hit) — the in-app handler calls them
|
||||
// directly, mirroring how the standalone MCP server wires them here.
|
||||
export { searchShapes } from "./lib/drawio-shapes.js";
|
||||
@@ -89,7 +89,7 @@ const VERSION = packageJson.version;
|
||||
// (SHARED_TOOL_SPECS + INLINE_MCP_INVENTORY), so it can no longer drift out of
|
||||
// sync with the registered tools. Re-exported here (its old home) so existing
|
||||
// importers are unaffected; the composition lives in server-instructions.ts.
|
||||
// The drawio_shapes / drawio_guide tools (#424) stay in SHARED_TOOL_SPECS (so the
|
||||
// The drawioShapes / drawioGuide tools (#424) stay in SHARED_TOOL_SPECS (so the
|
||||
// generated <tool_inventory> picks them up from their catalogLine automatically)
|
||||
// but are flagged `inlineBothHosts` and registered inline below (their pure
|
||||
// helpers can't cross into tool-specs.ts); only the hand-written routing prose in
|
||||
@@ -286,7 +286,7 @@ export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
|
||||
// the wrapping is typed loosely and cast — runtime behaviour is unchanged.
|
||||
const registerSharedFromSpec = (spec: SharedToolSpec) => {
|
||||
if (spec.inAppOnly) return;
|
||||
// `inlineBothHosts` specs (drawio_shapes / drawio_guide) carry no execute —
|
||||
// `inlineBothHosts` specs (drawioShapes / drawioGuide) carry no execute —
|
||||
// their pure helper cannot cross into the zod-agnostic tool-specs.ts, so they
|
||||
// are registered INLINE below (searchShapes / getGuideSection). Skip them here
|
||||
// so the loop never dereferences a missing `execute`.
|
||||
@@ -316,7 +316,7 @@ export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
|
||||
}
|
||||
|
||||
// --- INLINE drawio helper tools (IN the shared registry, but inlineBothHosts) ---
|
||||
// drawio_shapes / drawio_guide (#424) live in SHARED_TOOL_SPECS (so the shared
|
||||
// drawioShapes / drawioGuide (#424) live in SHARED_TOOL_SPECS (so the shared
|
||||
// contract pins their name/description/schema across both hosts) but carry the
|
||||
// `inlineBothHosts` flag and NO execute: their pure backing helpers
|
||||
// (searchShapes / getGuideSection) cannot be value-imported into the
|
||||
@@ -356,25 +356,25 @@ export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
|
||||
|
||||
// --- INLINE tools kept per-transport (NOT in the shared registry) ---
|
||||
// Each stays inline for a documented reason: a snake_case/camelCase naming
|
||||
// clash the registry convention forbids (table_get), an intentional
|
||||
// per-transport behaviour/schema divergence (search, docmost_transform), or a
|
||||
// tool that exists ONLY on this standalone MCP surface (update_comment,
|
||||
// delete_comment — the in-app agent deliberately exposes no hard comment
|
||||
// clash the registry convention forbids (tableGet), an intentional
|
||||
// per-transport behaviour/schema divergence (search, docmostTransform), or a
|
||||
// tool that exists ONLY on this standalone MCP surface (updateComment,
|
||||
// deleteComment — the in-app agent deliberately exposes no hard comment
|
||||
// edit/delete tool).
|
||||
|
||||
// Tool: table_get
|
||||
// NOT in the shared registry: the MCP tool name `table_get` is noun-first while
|
||||
// Tool: tableGet
|
||||
// NOT in the shared registry: the MCP tool name `tableGet` is noun-first while
|
||||
// the in-app key is `getTable` (verb-first), breaking the snake_case(inAppKey)
|
||||
// convention the shared registry enforces (shared-tool-specs.contract.spec.ts).
|
||||
// Renaming the public MCP tool would break external clients, so it stays inline.
|
||||
server.registerTool(
|
||||
"table_get",
|
||||
"tableGet",
|
||||
{
|
||||
description:
|
||||
"Read a table as a matrix. Returns {rows, cols, cells (text[][]), " +
|
||||
"cellIds (paragraph id per cell, or null)}. `table` = `#<index>` from " +
|
||||
"get_outline, or any block id inside the table. Use cellIds with " +
|
||||
"patch_node for rich-formatted cell edits. `cols` is the FIRST row's " +
|
||||
"getOutline, or any block id inside the table. Use cellIds with " +
|
||||
"patchNode for rich-formatted cell edits. `cols` is the FIRST row's " +
|
||||
"width; ragged tables may vary per row, so use the per-row length of " +
|
||||
"`cells` for each row.",
|
||||
inputSchema: {
|
||||
@@ -388,9 +388,9 @@ server.registerTool(
|
||||
},
|
||||
);
|
||||
|
||||
// Tool: update_comment
|
||||
// Tool: updateComment
|
||||
server.registerTool(
|
||||
"update_comment",
|
||||
"updateComment",
|
||||
{
|
||||
description:
|
||||
"Update an existing comment's content. Only the comment creator can " +
|
||||
@@ -409,9 +409,9 @@ server.registerTool(
|
||||
},
|
||||
);
|
||||
|
||||
// Tool: delete_comment
|
||||
// Tool: deleteComment
|
||||
server.registerTool(
|
||||
"delete_comment",
|
||||
"deleteComment",
|
||||
{
|
||||
description:
|
||||
"Delete a comment. Only the comment creator or space admin can delete it.",
|
||||
@@ -435,41 +435,74 @@ server.registerTool(
|
||||
// Tool: search
|
||||
// INTENTIONAL per-transport divergence (not shared): the in-app `searchPages`
|
||||
// runs a semantic + keyword hybrid (RRF) with in-process access control and a
|
||||
// different schema (limit 1-20); this transport is a plain REST full-text search
|
||||
// (limit up to 100). Different behaviour AND schema, so kept per-layer.
|
||||
// different schema; this transport is the #443 agent-lookup search — a hybrid
|
||||
// substring + full-text search that also returns each hit's location (`path`)
|
||||
// and a windowed `snippet`, so one call answers "where is it and what's in it".
|
||||
// The in-app hybrid-RRF search is deliberately NOT touched. Different behaviour
|
||||
// AND schema, so kept per-layer.
|
||||
//
|
||||
// STANDALONE-vs-STOCK-UPSTREAM: the client sends the opt-in `substring`/
|
||||
// `parentPageId`/`titleOnly` DTO fields. A stock upstream server validates the
|
||||
// DTO with `whitelist: true` and silently strips these unknown fields, so the
|
||||
// request degrades gracefully to plain FTS (no path/snippet, current shape).
|
||||
//
|
||||
// EE/TYPESENSE DEGRADATION (#443): on an instance whose SEARCH_DRIVER is
|
||||
// `typesense`, the server routes this request to the Typesense backend, which
|
||||
// does NOT implement agent-lookup — the substring/path/snippet/tiering is
|
||||
// ignored and the response degrades to plain Typesense FTS. The rich lookup
|
||||
// shape is only produced by the native Postgres search driver.
|
||||
server.registerTool(
|
||||
"search",
|
||||
{
|
||||
description:
|
||||
"Full-text search for pages and content across the whole workspace. " +
|
||||
"Results are bounded by `limit` (1-100; when omitted the server applies " +
|
||||
"its own default).",
|
||||
"Find pages by a fragment of a technical string (hostnames, IPs, IDs " +
|
||||
"like `srv.local`, `10.0.12`, `WB-MGE-30D86B`) — one call returns each " +
|
||||
"hit's location (`path`: ancestor titles root→parent) and a `snippet` " +
|
||||
"around the first match, so you rarely need a follow-up get_page. " +
|
||||
"Matches substrings literally (dots/dashes/digits are not tokenized) as " +
|
||||
"well as full-text. Returns `{ pageId, title, path, snippet, score }` " +
|
||||
"sorted by `score` (a per-response relevance float).",
|
||||
inputSchema: {
|
||||
query: z.string().min(1).describe("Search query"),
|
||||
spaceId: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe("Restrict the search to a single space"),
|
||||
parentPageId: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
"Restrict to a page and all its descendants (the page itself included)",
|
||||
),
|
||||
titleOnly: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.describe("Match page titles only; skip page text"),
|
||||
limit: z
|
||||
.number()
|
||||
.int()
|
||||
.min(1)
|
||||
.max(100)
|
||||
.max(50)
|
||||
.optional()
|
||||
.describe("Max results to return (max 100)"),
|
||||
.describe("Max results to return (1-50, default 10)"),
|
||||
},
|
||||
},
|
||||
async ({ query, limit }) => {
|
||||
// The tool exposes no spaceId filter, so pass undefined for the client's
|
||||
// optional spaceId parameter and forward limit into its correct slot.
|
||||
const result = await docmostClient.search(query, undefined, limit);
|
||||
async ({ query, spaceId, parentPageId, titleOnly, limit }) => {
|
||||
const result = await docmostClient.search(query, spaceId, limit, {
|
||||
parentPageId,
|
||||
titleOnly,
|
||||
});
|
||||
return jsonContent(result);
|
||||
},
|
||||
);
|
||||
|
||||
// Tool: docmost_transform
|
||||
// Tool: docmostTransform
|
||||
// INTENTIONAL per-transport divergence (not shared): the in-app `transformPage`
|
||||
// deliberately omits the `deleteComments` schema field (comment-deletion
|
||||
// guardrail) and carries a much shorter description; this transport exposes the
|
||||
// full helper catalogue. Different schema, so kept per-layer.
|
||||
server.registerTool(
|
||||
"docmost_transform",
|
||||
"docmostTransform",
|
||||
{
|
||||
description:
|
||||
"Edit a page by running an arbitrary JS transform `(doc, ctx) => doc` " +
|
||||
|
||||
@@ -440,9 +440,16 @@ export class CollabSession {
|
||||
// must stay synchronous (no await). While the JS event loop is not
|
||||
// yielded, no incoming remote update can interleave, so any already-synced
|
||||
// concurrent edits are preserved in liveDoc.
|
||||
//
|
||||
// INVARIANT 1 is machine-checked: the BEGIN/END markers below delimit the
|
||||
// no-await window, and test/unit/no-await-critical-window.test.mjs scans
|
||||
// this source and FAILS if any `await` (or `for await`/`yield`) appears
|
||||
// between them. Do NOT add an await inside this block — an accidental
|
||||
// async boundary here silently reopens the clobber-live-edits race (#152).
|
||||
let newDoc: any;
|
||||
let beforeDoc: any;
|
||||
try {
|
||||
// === MUTATE-CRITICAL-WINDOW: BEGIN (no await between here and END #449) ===
|
||||
let liveDoc = TiptapTransformer.fromYdoc(this.ydoc, "default");
|
||||
if (
|
||||
!liveDoc ||
|
||||
@@ -480,6 +487,7 @@ export class CollabSession {
|
||||
// ids of unchanged nodes, so an open editor's cursor is not yanked to the
|
||||
// end of the document on every agent write.
|
||||
applyDocToFragment(this.ydoc, newDoc);
|
||||
// === MUTATE-CRITICAL-WINDOW: END (#449) ===
|
||||
} catch (e) {
|
||||
// Includes errors thrown by transform (e.g. "afterText not found",
|
||||
// "text not found"): propagate them verbatim to the caller.
|
||||
|
||||
@@ -87,8 +87,8 @@ global.WebSocket = WebSocket;
|
||||
* bodies merged. So the import output is ALREADY in canonical footnote
|
||||
* topology.
|
||||
* - `canonicalizeFootnotes` runs AFTER as the mcp write-path invariant shared
|
||||
* with every other full-document persist path (`update_page_json`,
|
||||
* `docmost_transform`, `insert_footnote`, …). Because the package output is
|
||||
* with every other full-document persist path (`updatePageJson`,
|
||||
* `docmostTransform`, `insertFootnote`, …). Because the package output is
|
||||
* already canonical, this layer is a no-op here (idempotent) — it exists so
|
||||
* the page-write contract is enforced uniformly regardless of how the PM doc
|
||||
* was produced, not because the import needs fixing.
|
||||
@@ -282,7 +282,7 @@ export async function mutatePageContent(
|
||||
* it was produced from markdown (ids regenerate) or edited in place
|
||||
* (existing block ids preserved).
|
||||
*
|
||||
* This is an intentional full replace (used by update_page / update_page_json),
|
||||
* This is an intentional full replace (used by update_page / updatePageJson),
|
||||
* but now runs under the per-page lock and waits for server persistence via
|
||||
* mutatePageContent.
|
||||
*/
|
||||
|
||||
@@ -20,7 +20,7 @@
|
||||
*
|
||||
* MARKDOWN-STRIP FALLBACK: when the agent copies a selection that still carries
|
||||
* inline markdown (`**bold**`, `` `code` ``, `[t](u)`), the raw locator will not
|
||||
* match the document's plain text. Exactly like edit_page_text's json-edit
|
||||
* match the document's plain text. Exactly like editPageText's json-edit
|
||||
* fallback, we first try the verbatim selection and, ONLY if it anchors nowhere
|
||||
* in the whole document, retry with `stripInlineMarkdown` applied. `canAnchorInDoc`,
|
||||
* `getAnchoredText` and `applyAnchorInDoc` share this decision via
|
||||
|
||||
+163
-53
@@ -14,7 +14,19 @@
|
||||
* signature.
|
||||
*
|
||||
* If recreateTransform / the changeset throws on a pathological document pair,
|
||||
* we fall back to a coarse block-level text diff so the tool never hard-fails.
|
||||
* OR the pair is too large to diff cheaply (see the size guard below), we fall
|
||||
* back to a coarse block-level text diff so the tool never hard-fails and never
|
||||
* pins the event loop.
|
||||
*
|
||||
* SIZE GUARD (issue #464 — prod CPU-DoS). recreateTransform computes its diff via
|
||||
* rfc6902.createPatch, whose array diff is O(n·m) Levenshtein per array pair and
|
||||
* whose per-run word diff is O(w²); on a large/heavily-changed doc this runs for
|
||||
* seconds-to-hours and starves the whole process (BullMQ, Redis lock renewals,
|
||||
* embeddings). It never THROWS — it just never finishes — so the try/catch below
|
||||
* cannot save us. Because diffDocs runs on EVERY in-app/MCP content edit's verify
|
||||
* report, we PRE-FLIGHT the doc size and route anything above a cheap cap straight
|
||||
* to the coarse fallback (the same shape the catch produces). Same cap+fallback
|
||||
* pattern as the ELK-layout DoS fix (#440 / c917dcc3).
|
||||
*/
|
||||
|
||||
import { Node } from "@tiptap/pm/model";
|
||||
@@ -72,6 +84,56 @@ function countNodes(doc: any, pred: (node: any) => boolean): number {
|
||||
return n;
|
||||
}
|
||||
|
||||
// --- Issue #464: pre-flight size guard for the precise diff ------------------
|
||||
// Defaults are BENCHMARK-derived on the recreateTransform(complexSteps:false,
|
||||
// wordDiffs:true, simplifyDiff:true) pipeline, chosen so the WORST case (a fully
|
||||
// re-written doc — the adversarial shape that drove the incident) keeps the
|
||||
// synchronous block under ~200ms REGARDLESS of input:
|
||||
// - 150 total nodes: worst-case pair ~176ms; the O(node²) array diff crosses
|
||||
// 200ms at ~170 nodes and then explodes super-linearly (400 nodes ~1.3s,
|
||||
// 800 ~5.5s), so cap just below the crossover.
|
||||
// - 12 KiB serialized JSON: an independent axis, because the per-run word diff
|
||||
// is O(words²) — a FEW nodes with very long text runs is dangerous even at a
|
||||
// low node count (17 nodes / ~11 KiB ~176ms, / ~14 KiB ~290ms). A node-light
|
||||
// but byte-heavy doc is still refused.
|
||||
// Either metric over its cap routes to the coarse fallback. Both are env-tunable
|
||||
// for operators who accept more CPU in exchange for exact diffs on larger docs.
|
||||
const DEFAULT_MAX_NODES = 150;
|
||||
const DEFAULT_MAX_BYTES = 12 * 1024;
|
||||
|
||||
/**
|
||||
* Read a positive-integer env override, falling back to `dflt`. Garbage / unset /
|
||||
* non-finite / non-positive all fall back (so the guard can never be accidentally
|
||||
* disabled by a malformed value). Read fresh on every call so a test / operator
|
||||
* can flip the knob without a restart.
|
||||
*/
|
||||
function readPositiveIntEnv(name: string, dflt: number): number {
|
||||
const raw = parseInt(process.env[name] ?? "", 10);
|
||||
return Number.isFinite(raw) && raw > 0 ? raw : dflt;
|
||||
}
|
||||
|
||||
/**
|
||||
* True when the pair is too large for the precise (recreateTransform) diff and
|
||||
* must degrade to the coarse fallback. Takes the MAX of the two docs on each
|
||||
* metric so an ASYMMETRIC pair (a small new doc vs a huge old doc, or vice
|
||||
* versa) — which still explodes rfc6902 — is caught. Cheap: one node walk +
|
||||
* one JSON.stringify per doc, both O(size).
|
||||
*/
|
||||
function exceedsDiffSizeGuard(oldDoc: any, newDoc: any): boolean {
|
||||
const maxNodes = readPositiveIntEnv("MCP_DIFF_MAX_NODES", DEFAULT_MAX_NODES);
|
||||
const maxBytes = readPositiveIntEnv("MCP_DIFF_MAX_BYTES", DEFAULT_MAX_BYTES);
|
||||
const nodes = Math.max(
|
||||
countNodes(oldDoc, () => true),
|
||||
countNodes(newDoc, () => true),
|
||||
);
|
||||
if (nodes > maxNodes) return true;
|
||||
const bytes = Math.max(
|
||||
JSON.stringify(oldDoc)?.length ?? 0,
|
||||
JSON.stringify(newDoc)?.length ?? 0,
|
||||
);
|
||||
return bytes > maxBytes;
|
||||
}
|
||||
|
||||
/**
|
||||
* Count UNIQUE links in a JSON doc by their `href`. A single link can be split
|
||||
* across several adjacent text runs (e.g. a "link+bold" run followed by a "link"
|
||||
@@ -226,6 +288,81 @@ function coarseDiff(oldDoc: any, newDoc: any): DiffChange[] {
|
||||
return changes;
|
||||
}
|
||||
|
||||
/** Accumulated textual changes plus their derived char/block tallies. */
|
||||
interface DiffTally {
|
||||
changes: DiffChange[];
|
||||
inserted: number;
|
||||
deleted: number;
|
||||
changedBlocks: Set<string>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Produce the coarse-fallback tally for a pair. This is the SINGLE source of the
|
||||
* `fellBack:true` result shape, shared by BOTH degrade paths in diffDocs (the
|
||||
* pre-flight size guard and the recreateTransform catch) so they behave and
|
||||
* report identically.
|
||||
*/
|
||||
function coarseDiffTally(oldDoc: any, newDoc: any): DiffTally {
|
||||
const changes = coarseDiff(oldDoc, newDoc);
|
||||
let inserted = 0;
|
||||
let deleted = 0;
|
||||
const changedBlocks = new Set<string>();
|
||||
for (const c of changes) {
|
||||
if (c.op === "insert") inserted += c.text.length;
|
||||
else deleted += c.text.length;
|
||||
if (c.block) changedBlocks.add(c.op[0] + ":" + c.block);
|
||||
}
|
||||
return { changes, inserted, deleted, changedBlocks };
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute the PRECISE tally via the recreateTransform pipeline. Callers MUST
|
||||
* gate this behind the size guard (it can block the event loop for a large pair)
|
||||
* and wrap it in try/catch (a pathological pair can throw); on either the guard
|
||||
* or a throw, use `coarseDiffTally` instead. Kept as a sibling of
|
||||
* `coarseDiffTally` so both produce the same `DiffTally` shape.
|
||||
*/
|
||||
function preciseDiffTally(oldDocJson: any, newDocJson: any): DiffTally {
|
||||
const oldNode = Node.fromJSON(docmostSchema, oldDocJson);
|
||||
const newNode = Node.fromJSON(docmostSchema, newDocJson);
|
||||
const tr = recreateTransform(oldNode, newNode, {
|
||||
complexSteps: false,
|
||||
wordDiffs: true,
|
||||
simplifyDiff: true,
|
||||
});
|
||||
const changeSet = ChangeSet.create(oldNode).addSteps(tr.doc, tr.mapping.maps, []);
|
||||
const simplified = simplifyChanges(changeSet.changes, newNode);
|
||||
|
||||
const changes: DiffChange[] = [];
|
||||
let inserted = 0;
|
||||
let deleted = 0;
|
||||
const changedBlocks = new Set<string>();
|
||||
|
||||
for (const change of simplified) {
|
||||
// Deleted text lives in the OLD doc coordinate range [fromA, toA).
|
||||
if (change.toA > change.fromA) {
|
||||
const text = oldNode.textBetween(change.fromA, change.toA, "\n", " ");
|
||||
if (text.length > 0) {
|
||||
deleted += text.length;
|
||||
const block = blockContextAt(oldNode, change.fromA);
|
||||
changes.push({ op: "delete", block, text });
|
||||
if (block) changedBlocks.add("d:" + block);
|
||||
}
|
||||
}
|
||||
// Inserted text lives in the NEW doc coordinate range [fromB, toB).
|
||||
if (change.toB > change.fromB) {
|
||||
const text = newNode.textBetween(change.fromB, change.toB, "\n", " ");
|
||||
if (text.length > 0) {
|
||||
inserted += text.length;
|
||||
const block = blockContextAt(newNode, change.fromB);
|
||||
changes.push({ op: "insert", block, text });
|
||||
if (block) changedBlocks.add("i:" + block);
|
||||
}
|
||||
}
|
||||
}
|
||||
return { changes, inserted, deleted, changedBlocks };
|
||||
}
|
||||
|
||||
/** Build the human-readable unified-ish markdown summary. */
|
||||
function renderMarkdown(
|
||||
result: Omit<DiffResult, "markdown">,
|
||||
@@ -276,66 +413,39 @@ export function diffDocs(
|
||||
newDocJson: any,
|
||||
notesHeading: string = "Примечания переводчика",
|
||||
): DiffResult {
|
||||
// computeIntegrity is cheap (linear node walks) and its counts are needed in
|
||||
// BOTH the precise and coarse paths, so it always runs first.
|
||||
const integrity = computeIntegrity(oldDocJson, newDocJson, notesHeading);
|
||||
|
||||
let changes: DiffChange[] = [];
|
||||
let inserted = 0;
|
||||
let deleted = 0;
|
||||
let fellBack = false;
|
||||
const changedBlocks = new Set<string>();
|
||||
let tally: DiffTally;
|
||||
|
||||
try {
|
||||
const oldNode = Node.fromJSON(docmostSchema, oldDocJson);
|
||||
const newNode = Node.fromJSON(docmostSchema, newDocJson);
|
||||
const tr = recreateTransform(oldNode, newNode, {
|
||||
complexSteps: false,
|
||||
wordDiffs: true,
|
||||
simplifyDiff: true,
|
||||
});
|
||||
const changeSet = ChangeSet.create(oldNode).addSteps(
|
||||
tr.doc,
|
||||
tr.mapping.maps,
|
||||
[],
|
||||
);
|
||||
const simplified = simplifyChanges(changeSet.changes, newNode);
|
||||
|
||||
for (const change of simplified) {
|
||||
// Deleted text lives in the OLD doc coordinate range [fromA, toA).
|
||||
if (change.toA > change.fromA) {
|
||||
const text = oldNode.textBetween(change.fromA, change.toA, "\n", " ");
|
||||
if (text.length > 0) {
|
||||
deleted += text.length;
|
||||
const block = blockContextAt(oldNode, change.fromA);
|
||||
changes.push({ op: "delete", block, text });
|
||||
if (block) changedBlocks.add("d:" + block);
|
||||
}
|
||||
}
|
||||
// Inserted text lives in the NEW doc coordinate range [fromB, toB).
|
||||
if (change.toB > change.fromB) {
|
||||
const text = newNode.textBetween(change.fromB, change.toB, "\n", " ");
|
||||
if (text.length > 0) {
|
||||
inserted += text.length;
|
||||
const block = blockContextAt(newNode, change.fromB);
|
||||
changes.push({ op: "insert", block, text });
|
||||
if (block) changedBlocks.add("i:" + block);
|
||||
}
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Pathological pair: degrade to a coarse block-level diff so we never throw.
|
||||
// Pre-flight size guard (#464): a too-large pair would make recreateTransform
|
||||
// block the event loop for seconds-to-hours WITHOUT throwing, so route it to
|
||||
// the coarse fallback BEFORE calling recreateTransform at all. Both this path
|
||||
// and the catch below go through coarseDiffTally for an identical `fellBack`
|
||||
// result shape.
|
||||
if (exceedsDiffSizeGuard(oldDocJson, newDocJson)) {
|
||||
fellBack = true;
|
||||
changes = coarseDiff(oldDocJson, newDocJson);
|
||||
for (const c of changes) {
|
||||
if (c.op === "insert") inserted += c.text.length;
|
||||
else deleted += c.text.length;
|
||||
if (c.block) changedBlocks.add(c.op[0] + ":" + c.block);
|
||||
tally = coarseDiffTally(oldDocJson, newDocJson);
|
||||
} else {
|
||||
try {
|
||||
tally = preciseDiffTally(oldDocJson, newDocJson);
|
||||
} catch {
|
||||
// Pathological pair: degrade to a coarse block-level diff so we never throw.
|
||||
fellBack = true;
|
||||
tally = coarseDiffTally(oldDocJson, newDocJson);
|
||||
}
|
||||
}
|
||||
|
||||
const partial: Omit<DiffResult, "markdown"> = {
|
||||
summary: { inserted, deleted, blocksChanged: changedBlocks.size },
|
||||
summary: {
|
||||
inserted: tally.inserted,
|
||||
deleted: tally.deleted,
|
||||
blocksChanged: tally.changedBlocks.size,
|
||||
},
|
||||
integrity,
|
||||
changes,
|
||||
changes: tally.changes,
|
||||
};
|
||||
return { ...partial, markdown: renderMarkdown(partial, fellBack) };
|
||||
}
|
||||
@@ -380,7 +490,7 @@ export interface VerifyReport {
|
||||
/**
|
||||
* ONLY structural integrity types whose count changed, as [before, after]
|
||||
* (images/links/tables/callouts). Surfaces structural mutations that touch
|
||||
* neither text nor marks (e.g. insert_image, deleting a table) which diffDocs
|
||||
* neither text nor marks (e.g. insertImage, deleting a table) which diffDocs
|
||||
* — being TEXT-only — would otherwise report as "no content change".
|
||||
*/
|
||||
structure?: Record<string, [number, number]>;
|
||||
@@ -400,7 +510,7 @@ export interface VerifyReport {
|
||||
*
|
||||
* The structural integrity delta (from diffDocs's `integrity` tuples) is what
|
||||
* makes `changed` true for an image/table/callout/link count change that diffs
|
||||
* to zero text — closing a verify blind spot for insert_image, delete_node on a
|
||||
* to zero text — closing a verify blind spot for insertImage, deleteNode on a
|
||||
* table, etc.
|
||||
*/
|
||||
export function summarizeChange(before: any, after: any): VerifyReport {
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
// ID-based cell operations for `drawioEditCells` (issue #425, stage 3).
|
||||
//
|
||||
// Instead of resending the whole XML (whose diff is fragile — draw.io reorders
|
||||
// attributes, and a {search,replace} text match breaks on it), the model sends
|
||||
// targeted operations keyed by cell id:
|
||||
//
|
||||
// { op: "add", xml: "<mxCell .../>" } // append a new cell
|
||||
// { op: "update", cellId: "n3", xml: "<mxCell .../>" } // replace that cell
|
||||
// { op: "delete", cellId: "n5" } // + CASCADE
|
||||
//
|
||||
// `delete` CASCADES: it removes the cell, every descendant cell whose parent
|
||||
// chain leads to it (container children), AND every edge whose source or target
|
||||
// is any deleted cell. Ids are STABLE across edits so diffs stay meaningful.
|
||||
//
|
||||
// Operations apply to the parsed DOM of the current model; the caller re-lints
|
||||
// and rebuilds the .drawio.svg through the existing #423 pipeline afterwards.
|
||||
|
||||
import { JSDOM } from "jsdom";
|
||||
|
||||
let _window: any = null;
|
||||
function xmlWindow(): any {
|
||||
if (!_window) _window = new JSDOM("").window;
|
||||
return _window;
|
||||
}
|
||||
|
||||
export type CellOp =
|
||||
| { op: "add"; xml: string }
|
||||
| { op: "update"; cellId: string; xml: string }
|
||||
| { op: "delete"; cellId: string };
|
||||
|
||||
export class CellOpsError extends Error {
|
||||
constructor(message: string) {
|
||||
super(`drawioEditCells: ${message}`);
|
||||
this.name = "CellOpsError";
|
||||
}
|
||||
}
|
||||
|
||||
// The mxGraph root sentinels. id="0" is the graph root; id="1" is the default
|
||||
// layer that parents every real cell. A delete targeting either would cascade
|
||||
// through the whole diagram body (every cell chains up to "1"), so such an op is
|
||||
// rejected outright.
|
||||
const SENTINEL_IDS = new Set(["0", "1"]);
|
||||
|
||||
/** Parse a single `<mxCell …>…</mxCell>` fragment into an element, or throw. */
|
||||
function parseCellFragment(xml: string): any {
|
||||
const parser = new (xmlWindow().DOMParser)();
|
||||
// Wrap so a self-closed or child-bearing single cell parses as one root.
|
||||
const doc = parser.parseFromString(`<root>${xml}</root>`, "application/xml");
|
||||
if (doc.getElementsByTagName("parsererror").length > 0) {
|
||||
throw new CellOpsError(`operation xml is not well-formed: ${xml.slice(0, 120)}`);
|
||||
}
|
||||
const cells = doc.getElementsByTagName("mxCell");
|
||||
if (cells.length !== 1) {
|
||||
throw new CellOpsError(
|
||||
`each add/update op must carry exactly one <mxCell> (got ${cells.length})`,
|
||||
);
|
||||
}
|
||||
return cells[0];
|
||||
}
|
||||
|
||||
/** All ids reachable as descendants of `rootId` via the parent relation. */
|
||||
function collectDescendants(
|
||||
rootId: string,
|
||||
parentOf: Map<string, string | undefined>,
|
||||
): Set<string> {
|
||||
const doomed = new Set<string>([rootId]);
|
||||
let grew = true;
|
||||
while (grew) {
|
||||
grew = false;
|
||||
for (const [id, parent] of parentOf) {
|
||||
if (!doomed.has(id) && parent != null && doomed.has(parent)) {
|
||||
doomed.add(id);
|
||||
grew = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
return doomed;
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply the operation list to a model XML string and return the new model XML.
|
||||
* Uses the DOM so attribute order / formatting is preserved for untouched cells.
|
||||
* Throws CellOpsError on an unknown target id or a malformed op fragment (so the
|
||||
* model gets a precise error and nothing is half-applied).
|
||||
*/
|
||||
export function applyCellOps(modelXml: string, ops: CellOp[]): string {
|
||||
if (!Array.isArray(ops) || ops.length === 0) {
|
||||
throw new CellOpsError("operations must be a non-empty array");
|
||||
}
|
||||
const parser = new (xmlWindow().DOMParser)();
|
||||
const doc = parser.parseFromString(modelXml, "application/xml");
|
||||
if (doc.getElementsByTagName("parsererror").length > 0) {
|
||||
throw new CellOpsError("the current diagram XML is not well-formed");
|
||||
}
|
||||
const root = doc.getElementsByTagName("root")[0];
|
||||
if (!root) throw new CellOpsError("the current diagram has no <root> element");
|
||||
|
||||
const cellEls = () => Array.from(root.getElementsByTagName("mxCell")) as any[];
|
||||
const byId = () => {
|
||||
const m = new Map<string, any>();
|
||||
for (const el of cellEls()) m.set(el.getAttribute("id") ?? "", el);
|
||||
return m;
|
||||
};
|
||||
|
||||
for (const op of ops) {
|
||||
if (op.op === "add") {
|
||||
const frag = parseCellFragment(op.xml);
|
||||
const id = frag.getAttribute("id");
|
||||
if (!id) throw new CellOpsError("an add op's <mxCell> is missing an id");
|
||||
if (byId().has(id))
|
||||
throw new CellOpsError(`add op id "${id}" already exists (use update)`);
|
||||
root.appendChild(doc.importNode(frag, true));
|
||||
} else if (op.op === "update") {
|
||||
const map = byId();
|
||||
const target = map.get(op.cellId);
|
||||
if (!target)
|
||||
throw new CellOpsError(`update target cell "${op.cellId}" does not exist`);
|
||||
const frag = parseCellFragment(op.xml);
|
||||
const newId = frag.getAttribute("id");
|
||||
if (newId && newId !== op.cellId)
|
||||
throw new CellOpsError(
|
||||
`update op cellId "${op.cellId}" != the <mxCell> id "${newId}" (ids are stable)`,
|
||||
);
|
||||
// Replace the element in place so surrounding cells are untouched.
|
||||
const imported = doc.importNode(frag, true);
|
||||
target.parentNode.replaceChild(imported, target);
|
||||
} else if (op.op === "delete") {
|
||||
// Reject a sentinel-targeted delete BEFORE collecting descendants: "0"/"1"
|
||||
// parent the entire diagram, so a cascade from either would wipe the whole
|
||||
// model body (doomed.delete("0"/"1") only spared the sentinel itself, not
|
||||
// its children).
|
||||
if (SENTINEL_IDS.has(op.cellId))
|
||||
throw new CellOpsError(
|
||||
`cannot delete sentinel cell "${op.cellId}" (the graph root/default layer)`,
|
||||
);
|
||||
const map = byId();
|
||||
if (!map.has(op.cellId))
|
||||
throw new CellOpsError(`delete target cell "${op.cellId}" does not exist`);
|
||||
// Build the parent relation over the CURRENT cells for the cascade.
|
||||
const parentOf = new Map<string, string | undefined>();
|
||||
for (const el of cellEls()) {
|
||||
parentOf.set(el.getAttribute("id") ?? "", el.getAttribute("parent") ?? undefined);
|
||||
}
|
||||
const doomed = collectDescendants(op.cellId, parentOf);
|
||||
// Cascade to edges whose source/target is any doomed cell.
|
||||
for (const el of cellEls()) {
|
||||
if (el.getAttribute("edge") !== "1") continue;
|
||||
const src = el.getAttribute("source");
|
||||
const tgt = el.getAttribute("target");
|
||||
if ((src && doomed.has(src)) || (tgt && doomed.has(tgt))) {
|
||||
doomed.add(el.getAttribute("id") ?? "");
|
||||
}
|
||||
}
|
||||
// Never delete the sentinels even if referenced by a malformed op.
|
||||
doomed.delete("0");
|
||||
doomed.delete("1");
|
||||
for (const el of cellEls()) {
|
||||
const id = el.getAttribute("id") ?? "";
|
||||
if (doomed.has(id)) el.parentNode.removeChild(el);
|
||||
}
|
||||
} else {
|
||||
throw new CellOpsError(`unknown op "${(op as any).op}"`);
|
||||
}
|
||||
}
|
||||
|
||||
const ser = new (xmlWindow().XMLSerializer)();
|
||||
return ser.serializeToString(doc.documentElement);
|
||||
}
|
||||
@@ -0,0 +1,916 @@
|
||||
// Semantic graph -> draw.io pipeline for `drawioFromGraph` (issue #425, stage 3).
|
||||
//
|
||||
// The model describes a diagram SEMANTICALLY — nodes with a `kind` and an
|
||||
// optional `icon`, groups (containers), edges with a `kind` — and NEVER sees a
|
||||
// coordinate or a style string. This module owns the whole server-side pipeline:
|
||||
//
|
||||
// 1. validateGraph — a hand-written validator (no zod dependency, so this
|
||||
// lib stays importable by client.ts without coupling to
|
||||
// a zod major) that rejects malformed graphs early.
|
||||
// 2. resolveNodeStyle — `icon` -> exact style via the shape catalog (#424);
|
||||
// an UNKNOWN icon degrades to a generic shape by `kind`
|
||||
// WITH the label (never an empty square). `kind` -> the
|
||||
// preset palette slot.
|
||||
// 3. graphToElk — graph -> ELK-JSON, honouring the layout hints
|
||||
// (`layer`/`sameLayerAs` -> layer constraints, `pinned`
|
||||
// -> a fixed node) and compound group nodes.
|
||||
// 4. assembleModel — graph + ELK coordinates -> a full mxGraphModel XML
|
||||
// that satisfies the #423 linter BY CONSTRUCTION
|
||||
// (sentinels, transparent containers, relative child
|
||||
// coords, cross-container edges parent="1", >=150px
|
||||
// gaps from ELK spacing, escaped labels).
|
||||
//
|
||||
// The `layout` mode: "full" re-lays everything; "incremental" fixes existing
|
||||
// coordinates (ELK interactive mode) and places only new nodes; "none" keeps the
|
||||
// caller-provided/prior coordinates untouched.
|
||||
|
||||
import ELK from "elkjs/lib/elk.bundled.js";
|
||||
import { JSDOM } from "jsdom";
|
||||
import {
|
||||
searchShapes,
|
||||
awsServiceStyle,
|
||||
type ShapeResult,
|
||||
} from "./drawio-shapes.js";
|
||||
import {
|
||||
getPreset,
|
||||
genericNodeStyle,
|
||||
iconNodeStyle,
|
||||
edgeStyle,
|
||||
groupStyle,
|
||||
type PresetData,
|
||||
} from "./drawio-presets.js";
|
||||
import { MIN_SHAPE_GAP } from "./drawio-xml.js";
|
||||
|
||||
// --- graph schema (plain TS + a hand validator) ----------------------------
|
||||
|
||||
export interface GraphNode {
|
||||
id: string;
|
||||
label: string;
|
||||
kind?: string;
|
||||
/** Icon reference, e.g. "aws:lambda" | "azure:cosmos" | "lambda". */
|
||||
icon?: string;
|
||||
/** Group (container) id this node belongs to. */
|
||||
group?: string;
|
||||
/** Layer hint (ELK layerChoiceConstraint): 0-based column/row index. */
|
||||
layer?: number;
|
||||
/** Put this node in the same layer as another node id. */
|
||||
sameLayerAs?: string;
|
||||
/** Fix this node at exact coordinates (an ELK fixed node). */
|
||||
pinned?: { x: number; y: number };
|
||||
}
|
||||
|
||||
export interface GraphGroup {
|
||||
id: string;
|
||||
label: string;
|
||||
kind?: string;
|
||||
/** Parent group id — lets a group nest inside another group (e.g. subnet in VPC). */
|
||||
group?: string;
|
||||
}
|
||||
|
||||
export interface GraphEdge {
|
||||
from: string;
|
||||
to: string;
|
||||
label?: string;
|
||||
kind?: string;
|
||||
}
|
||||
|
||||
export interface Graph {
|
||||
nodes: GraphNode[];
|
||||
groups?: GraphGroup[];
|
||||
edges?: GraphEdge[];
|
||||
direction?: "LR" | "RL" | "TB" | "BT";
|
||||
preset?: string;
|
||||
}
|
||||
|
||||
export type LayoutMode = "none" | "full" | "incremental";
|
||||
|
||||
/** A structured validation error (mirrors the drawio linter's shape loosely). */
|
||||
export class GraphValidationError extends Error {
|
||||
issues: string[];
|
||||
constructor(issues: string[]) {
|
||||
super(`drawioFromGraph: invalid graph — ${issues.join("; ")}`);
|
||||
this.name = "GraphValidationError";
|
||||
this.issues = issues;
|
||||
}
|
||||
}
|
||||
|
||||
const MAX_GRAPH_NODES = 500; // parity with drawio-layout's ELK_MAX_NODES.
|
||||
// Edge/group caps mirror drawio-layout's ELK_MAX_EDGES. Without an edge cap a
|
||||
// tiny node set with a huge edge list (e.g. 500 nodes / 200000 edges) passes
|
||||
// node validation, then graphToElk/runElk exhausts the heap SYNCHRONOUSLY inside
|
||||
// elk.bundled.js — before the 5s ELK timeout can fire and OUTSIDE it entirely
|
||||
// for the mapper/assembler — crashing the worker on LLM-authored input. Reject
|
||||
// the over-limit shape here, before any layout or assembly runs.
|
||||
export const MAX_GRAPH_EDGES = 1000; // parity with drawio-layout's ELK_MAX_EDGES.
|
||||
export const MAX_GRAPH_GROUPS = 500; // groups are compound ELK nodes; bound them too.
|
||||
|
||||
/**
|
||||
* Validate the graph structure BEFORE any layout/assembly so the model gets a
|
||||
* precise, actionable error instead of a corrupt diagram. Throws
|
||||
* GraphValidationError listing every problem.
|
||||
*/
|
||||
export function validateGraph(graph: Graph): void {
|
||||
const issues: string[] = [];
|
||||
if (!graph || typeof graph !== "object") {
|
||||
throw new GraphValidationError(["graph must be an object"]);
|
||||
}
|
||||
if (!Array.isArray(graph.nodes) || graph.nodes.length === 0) {
|
||||
throw new GraphValidationError(["graph.nodes must be a non-empty array"]);
|
||||
}
|
||||
// Size caps FIRST (fail fast, before touching per-element loops) so an
|
||||
// over-limit graph can never reach the layout engine and OOM the worker.
|
||||
if (graph.nodes.length > MAX_GRAPH_NODES) {
|
||||
throw new GraphValidationError([
|
||||
`graph has ${graph.nodes.length} nodes (max ${MAX_GRAPH_NODES})`,
|
||||
]);
|
||||
}
|
||||
if (Array.isArray(graph.edges) && graph.edges.length > MAX_GRAPH_EDGES) {
|
||||
throw new GraphValidationError([
|
||||
`graph has ${graph.edges.length} edges (max ${MAX_GRAPH_EDGES})`,
|
||||
]);
|
||||
}
|
||||
if (Array.isArray(graph.groups) && graph.groups.length > MAX_GRAPH_GROUPS) {
|
||||
throw new GraphValidationError([
|
||||
`graph has ${graph.groups.length} groups (max ${MAX_GRAPH_GROUPS})`,
|
||||
]);
|
||||
}
|
||||
|
||||
const nodeIds = new Set<string>();
|
||||
const groupIds = new Set<string>();
|
||||
for (const g of graph.groups ?? []) {
|
||||
if (!g.id) issues.push("a group is missing its id");
|
||||
else if (groupIds.has(g.id)) issues.push(`duplicate group id "${g.id}"`);
|
||||
groupIds.add(g.id);
|
||||
}
|
||||
for (const n of graph.nodes) {
|
||||
if (!n.id) issues.push("a node is missing its id");
|
||||
else if (nodeIds.has(n.id)) issues.push(`duplicate node id "${n.id}"`);
|
||||
else if (groupIds.has(n.id))
|
||||
issues.push(`node id "${n.id}" collides with a group id`);
|
||||
nodeIds.add(n.id);
|
||||
if (typeof n.label !== "string" || n.label === "")
|
||||
issues.push(`node "${n.id}" is missing a label`);
|
||||
if (n.group != null && !groupIds.has(n.group))
|
||||
issues.push(`node "${n.id}" references unknown group "${n.group}"`);
|
||||
if (n.pinned != null) {
|
||||
if (
|
||||
typeof n.pinned.x !== "number" ||
|
||||
typeof n.pinned.y !== "number" ||
|
||||
!Number.isFinite(n.pinned.x) ||
|
||||
!Number.isFinite(n.pinned.y)
|
||||
)
|
||||
issues.push(`node "${n.id}" has an invalid pinned {x,y}`);
|
||||
}
|
||||
if (n.layer != null && (!Number.isInteger(n.layer) || n.layer < 0))
|
||||
issues.push(`node "${n.id}" has an invalid layer (must be a >=0 integer)`);
|
||||
}
|
||||
// sameLayerAs must reference an existing node (checked after all ids known).
|
||||
for (const n of graph.nodes) {
|
||||
if (n.sameLayerAs != null && !nodeIds.has(n.sameLayerAs))
|
||||
issues.push(
|
||||
`node "${n.id}" sameLayerAs references unknown node "${n.sameLayerAs}"`,
|
||||
);
|
||||
}
|
||||
for (const e of graph.edges ?? []) {
|
||||
if (!e.from || !e.to) {
|
||||
issues.push("an edge is missing from/to");
|
||||
continue;
|
||||
}
|
||||
if (!nodeIds.has(e.from) && !groupIds.has(e.from))
|
||||
issues.push(`edge from "${e.from}" resolves to no node/group`);
|
||||
if (!nodeIds.has(e.to) && !groupIds.has(e.to))
|
||||
issues.push(`edge to "${e.to}" resolves to no node/group`);
|
||||
}
|
||||
if (issues.length > 0) throw new GraphValidationError(issues);
|
||||
}
|
||||
|
||||
// --- icon resolution -------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Resolve a node's `icon` reference to a concrete style-string + size via the
|
||||
* shape catalog. Accepts "aws:lambda", "azure:cosmos", or a bare "lambda". An
|
||||
* AWS `resIcon` name is built directly (exact service-icon template). Anything
|
||||
* else goes through searchShapes. Returns null when nothing resolves — the
|
||||
* caller then falls back to a generic shape by kind (never an empty box).
|
||||
*/
|
||||
export function resolveIcon(icon: string): ShapeResult | null {
|
||||
const raw = icon.trim();
|
||||
if (raw === "") return null;
|
||||
let provider = "";
|
||||
let name = raw;
|
||||
const colon = raw.indexOf(":");
|
||||
if (colon !== -1) {
|
||||
provider = raw.slice(0, colon).trim().toLowerCase();
|
||||
name = raw.slice(colon + 1).trim();
|
||||
}
|
||||
|
||||
if (provider === "aws") {
|
||||
// Prefer an exact resIcon match from the catalog (carries the right size and
|
||||
// any rebrand/blocklist note); if the underscore/space name doesn't hit,
|
||||
// build the canonical service-icon style directly so it is never an empty box.
|
||||
const results = searchShapes(name.replace(/_/g, " "), { limit: 5 });
|
||||
const aws4 = results.find((r) => r.style.includes("mxgraph.aws4"));
|
||||
if (aws4) return aws4;
|
||||
return {
|
||||
style: awsServiceStyle(name.replace(/\s+/g, "_")),
|
||||
w: 78,
|
||||
h: 78,
|
||||
title: name,
|
||||
type: "vertex",
|
||||
};
|
||||
}
|
||||
|
||||
// Non-AWS or bare name: fuzzy search the catalog. Take the top vertex hit,
|
||||
// but REJECT a weak match (the fuzzy scorer can prefix-match an unrelated
|
||||
// stencil, e.g. "not..." -> "Notebook"); require the hit's title to actually
|
||||
// share a meaningful token with the query, otherwise degrade to generic-by-kind.
|
||||
const q = provider ? `${provider} ${name}` : name;
|
||||
const results = searchShapes(q, { limit: 8 });
|
||||
const hit = results.find((r) => r.type !== "edge") ?? results[0];
|
||||
if (!hit) return null;
|
||||
if (!isRelevantMatch(name, hit.title)) return null;
|
||||
return hit;
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a resolved stencil is a genuine match for the requested icon name (as
|
||||
* opposed to a loose prefix hit on an unrelated shape). True if any 3+ char
|
||||
* token of the query appears in the stencil title, or vice-versa.
|
||||
*/
|
||||
function isRelevantMatch(name: string, title: string): boolean {
|
||||
const norm = (s: string) => s.toLowerCase().replace(/[^a-z0-9]+/g, " ").trim();
|
||||
const qTokens = norm(name).split(/\s+/).filter((t) => t.length >= 3);
|
||||
if (qTokens.length === 0) return true; // very short names: trust the scorer
|
||||
const t = norm(title);
|
||||
const tTokens = new Set(t.split(/\s+/));
|
||||
for (const qt of qTokens) {
|
||||
if (tTokens.has(qt)) return true;
|
||||
if (t.includes(qt)) return true;
|
||||
for (const tt of tTokens) if (tt.length >= 3 && qt.includes(tt)) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide the final style + size for a node. When `icon` resolves, use the icon
|
||||
* style (overlaid with a dark-preset font fix); otherwise a GENERIC shape by
|
||||
* `kind` carrying the label. `resolved` reports whether an icon was found (used
|
||||
* by the acceptance test that asserts no empty squares).
|
||||
*/
|
||||
export function resolveNodeStyle(
|
||||
preset: PresetData,
|
||||
node: GraphNode,
|
||||
): { style: string; w: number; h: number; iconResolved: boolean } {
|
||||
if (node.icon) {
|
||||
const shape = resolveIcon(node.icon);
|
||||
if (shape) {
|
||||
return {
|
||||
style: iconNodeStyle(preset, shape.style),
|
||||
w: shape.w,
|
||||
h: shape.h,
|
||||
iconResolved: true,
|
||||
};
|
||||
}
|
||||
}
|
||||
// Generic shape by kind, sized to the label so a long label never overflows.
|
||||
const w = Math.max(120, estimateLabelWidth(node.label) + 32);
|
||||
return { style: genericNodeStyle(preset, node.kind), w, h: 60, iconResolved: false };
|
||||
}
|
||||
|
||||
/** Rough rendered width of the longest label line at 12px (~0.6em/glyph). */
|
||||
function estimateLabelWidth(label: string): number {
|
||||
const lines = label.split(/\r?\n|
|<br\s*\/?>/i);
|
||||
let longest = 0;
|
||||
for (const l of lines) longest = Math.max(longest, l.trim().length);
|
||||
return Math.ceil(longest * 12 * 0.6);
|
||||
}
|
||||
|
||||
// --- graph -> ELK-JSON -----------------------------------------------------
|
||||
|
||||
interface ElkNode {
|
||||
id: string;
|
||||
width?: number;
|
||||
height?: number;
|
||||
x?: number;
|
||||
y?: number;
|
||||
children?: ElkNode[];
|
||||
layoutOptions?: Record<string, string>;
|
||||
}
|
||||
interface ElkEdge {
|
||||
id: string;
|
||||
sources: string[];
|
||||
targets: string[];
|
||||
}
|
||||
interface ElkGraph extends ElkNode {
|
||||
edges?: ElkEdge[];
|
||||
}
|
||||
|
||||
const ELK_DIRECTION: Record<string, string> = {
|
||||
LR: "RIGHT",
|
||||
RL: "LEFT",
|
||||
TB: "DOWN",
|
||||
BT: "UP",
|
||||
};
|
||||
|
||||
/** Sizes resolved per node id (from resolveNodeStyle), fed to the ELK mapper. */
|
||||
export interface NodeSize {
|
||||
w: number;
|
||||
h: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the ELK graph from the semantic graph + resolved node sizes. Compound
|
||||
* group nodes nest their members (a group may itself nest in another group).
|
||||
* `only` restricts the graph to a subset of node ids (used by the incremental
|
||||
* path to lay out ONLY the new nodes). Layout HINTS (`layer`/`sameLayerAs`/
|
||||
* `pinned`) are NOT encoded as ELK constraints here — ELK's constraint knobs are
|
||||
* unreliable across versions — they are enforced deterministically AFTER layout
|
||||
* by applyHints, which is exact and testable.
|
||||
*/
|
||||
export function graphToElk(
|
||||
graph: Graph,
|
||||
sizes: Map<string, NodeSize>,
|
||||
opts: { only?: Set<string> } = {},
|
||||
): ElkGraph {
|
||||
const direction = ELK_DIRECTION[graph.direction ?? "LR"] ?? "RIGHT";
|
||||
const only = opts.only;
|
||||
const include = (id: string) => !only || only.has(id);
|
||||
|
||||
const makeNode = (n: GraphNode): ElkNode => {
|
||||
const size = sizes.get(n.id) ?? { w: 140, h: 60 };
|
||||
return { id: n.id, width: size.w, height: size.h };
|
||||
};
|
||||
|
||||
// Group children nest under their group node; ungrouped nodes are roots.
|
||||
const groupNode = new Map<string, ElkNode>();
|
||||
const usedGroups = new Set<string>();
|
||||
for (const g of graph.groups ?? []) {
|
||||
const size = sizes.get(g.id) ?? { w: 200, h: 150 };
|
||||
groupNode.set(g.id, {
|
||||
id: g.id,
|
||||
width: size.w,
|
||||
height: size.h,
|
||||
children: [],
|
||||
layoutOptions: {
|
||||
"elk.algorithm": "layered",
|
||||
"elk.direction": direction,
|
||||
"elk.padding": "[top=40,left=30,bottom=30,right=30]",
|
||||
"elk.layered.spacing.nodeNodeBetweenLayers": "170",
|
||||
"elk.spacing.nodeNode": "170",
|
||||
},
|
||||
});
|
||||
}
|
||||
const roots: ElkNode[] = [];
|
||||
for (const n of graph.nodes) {
|
||||
if (!include(n.id)) continue;
|
||||
const en = makeNode(n);
|
||||
if (n.group && groupNode.has(n.group)) {
|
||||
groupNode.get(n.group)!.children!.push(en);
|
||||
usedGroups.add(n.group);
|
||||
} else {
|
||||
roots.push(en);
|
||||
}
|
||||
}
|
||||
// Nest group nodes into their parent group (a subnet inside a VPC); groups
|
||||
// with no parent group become roots. Only groups that hold an included node.
|
||||
const groupIdSet = new Set((graph.groups ?? []).map((g) => g.id));
|
||||
for (const g of graph.groups ?? []) {
|
||||
if (only && !usedGroups.has(g.id)) continue;
|
||||
const en = groupNode.get(g.id)!;
|
||||
if (g.group && groupIdSet.has(g.group) && g.group !== g.id && (!only || usedGroups.has(g.group))) {
|
||||
groupNode.get(g.group)!.children!.push(en);
|
||||
} else {
|
||||
roots.push(en);
|
||||
}
|
||||
}
|
||||
|
||||
// Edges: endpoints may be nodes or groups; INCLUDE_CHILDREN spans the nesting.
|
||||
const validIds = new Set<string>([
|
||||
...graph.nodes.filter((n) => include(n.id)).map((n) => n.id),
|
||||
...(graph.groups ?? []).map((g) => g.id),
|
||||
]);
|
||||
const edges: ElkEdge[] = [];
|
||||
(graph.edges ?? []).forEach((e, i) => {
|
||||
if (!validIds.has(e.from) || !validIds.has(e.to)) return;
|
||||
edges.push({ id: `e${i}`, sources: [e.from], targets: [e.to] });
|
||||
});
|
||||
|
||||
const rootOptions: Record<string, string> = {
|
||||
"elk.algorithm": "layered",
|
||||
"elk.direction": direction,
|
||||
"elk.hierarchyHandling": "INCLUDE_CHILDREN",
|
||||
"elk.layered.spacing.nodeNodeBetweenLayers": "170",
|
||||
"elk.spacing.nodeNode": "170",
|
||||
"elk.spacing.edgeNode": "40",
|
||||
"elk.spacing.edgeEdge": "30",
|
||||
"elk.padding": "[top=20,left=20,bottom=20,right=20]",
|
||||
};
|
||||
|
||||
return { id: "root", layoutOptions: rootOptions, children: roots, edges };
|
||||
}
|
||||
|
||||
/**
|
||||
* Enforce the layout hints DETERMINISTICALLY on ELK's output (mutates `geo`):
|
||||
* - `sameLayerAs`: snap the dependent node's LAYER-AXIS coordinate to its
|
||||
* anchor's, so the pair lands in the same layer (x for LR/RL, y for TB/BT).
|
||||
* `layer` groups nodes with the same index onto the same anchor coordinate.
|
||||
* - `pinned`: override the node's coordinate with the exact pinned {x,y}.
|
||||
* Applied only to top-level (ungrouped) nodes, whose ELK coords are absolute.
|
||||
*/
|
||||
export function applyHints(
|
||||
graph: Graph,
|
||||
geo: Map<string, { x: number; y: number; w: number; h: number }>,
|
||||
): void {
|
||||
const dir = graph.direction ?? "LR";
|
||||
const layerAxis: "x" | "y" = dir === "TB" || dir === "BT" ? "y" : "x";
|
||||
// The perpendicular (cross-layer) axis: members snapped onto one layer must be
|
||||
// spread along THIS axis so they don't stack onto the same point.
|
||||
const crossAxis: "x" | "y" = layerAxis === "x" ? "y" : "x";
|
||||
const crossSize: "w" | "h" = crossAxis === "x" ? "w" : "h";
|
||||
const grouped = new Set(
|
||||
graph.nodes.filter((n) => n.group).map((n) => n.id),
|
||||
);
|
||||
|
||||
// sameLayerAs / layer: co-assign the layer-axis coordinate.
|
||||
// Build the effective layer key per node, then pick a representative coord.
|
||||
const layerKeyOf = new Map<string, string>();
|
||||
const explicitLayer = new Map<string, number>();
|
||||
for (const n of graph.nodes) if (n.layer != null) explicitLayer.set(n.id, n.layer);
|
||||
const byId = new Map(graph.nodes.map((n) => [n.id, n]));
|
||||
const resolveKey = (n: GraphNode): string | null => {
|
||||
if (explicitLayer.has(n.id)) return `L${explicitLayer.get(n.id)}`;
|
||||
const seen = new Set<string>([n.id]);
|
||||
let cur: GraphNode | undefined = n;
|
||||
while (cur && cur.sameLayerAs != null && !seen.has(cur.sameLayerAs)) {
|
||||
seen.add(cur.sameLayerAs);
|
||||
const t = byId.get(cur.sameLayerAs);
|
||||
if (!t) break;
|
||||
if (explicitLayer.has(t.id)) return `L${explicitLayer.get(t.id)}`;
|
||||
cur = t;
|
||||
}
|
||||
// A sameLayerAs chain with no explicit layer: key on the chain's root id.
|
||||
if (n.sameLayerAs != null) {
|
||||
let root = n.id;
|
||||
const s2 = new Set<string>([n.id]);
|
||||
let c: GraphNode | undefined = n;
|
||||
while (c && c.sameLayerAs != null && !s2.has(c.sameLayerAs)) {
|
||||
s2.add(c.sameLayerAs);
|
||||
root = c.sameLayerAs;
|
||||
c = byId.get(c.sameLayerAs);
|
||||
}
|
||||
return `C${root}`;
|
||||
}
|
||||
return null;
|
||||
};
|
||||
for (const n of graph.nodes) {
|
||||
if (grouped.has(n.id)) continue; // group children are relative — skip
|
||||
const key = resolveKey(n);
|
||||
if (key) layerKeyOf.set(n.id, key);
|
||||
}
|
||||
// Group members of each layer key so we can snap AND spread them together.
|
||||
const membersOf = new Map<string, string[]>();
|
||||
for (const n of graph.nodes) {
|
||||
const key = layerKeyOf.get(n.id);
|
||||
if (key == null) continue;
|
||||
if (!geo.has(n.id)) continue;
|
||||
(membersOf.get(key) ?? membersOf.set(key, []).get(key)!).push(n.id);
|
||||
}
|
||||
// For each layer key: snap every member to the FIRST member's layer-axis coord,
|
||||
// then SPREAD them along the perpendicular (cross-layer) axis with a >=
|
||||
// MIN_SHAPE_GAP gap. Without the spread, a sameLayerAs chain whose nodes ELK
|
||||
// happened to give the same cross-axis coordinate would collapse onto one point
|
||||
// -> shape-overlap + edge-through-shape quality warnings (breaking the
|
||||
// "0 warnings by construction" guarantee for these AUTO-positioned hints). We
|
||||
// start from the members' minimum cross-axis coord and stack them with a gap
|
||||
// of MIN_SHAPE_GAP beyond each shape's cross-axis size.
|
||||
for (const [key, members] of membersOf) {
|
||||
if (members.length === 0) continue;
|
||||
// Snap layer-axis coord to the first member.
|
||||
const repCoord = geo.get(members[0])![layerAxis];
|
||||
// Preserve the members' existing relative order along the cross axis so the
|
||||
// spread stays visually stable, then re-lay them contiguously.
|
||||
const sorted = [...members].sort(
|
||||
(a, b) => geo.get(a)![crossAxis] - geo.get(b)![crossAxis],
|
||||
);
|
||||
let cursor = geo.get(sorted[0])![crossAxis];
|
||||
for (const id of sorted) {
|
||||
const g = geo.get(id)!;
|
||||
g[layerAxis] = repCoord;
|
||||
g[crossAxis] = cursor;
|
||||
cursor += g[crossSize] + MIN_SHAPE_GAP;
|
||||
}
|
||||
void key;
|
||||
}
|
||||
|
||||
// pinned: exact override (wins over any layer snap). Explicit user coordinates
|
||||
// are user intent, but CLAMP to non-negative so an out-of-bounds pin (e.g.
|
||||
// x:-500) never renders off-canvas. Two user-pinned nodes at the same point is
|
||||
// user error the server can't silently relocate — the assembler docstring
|
||||
// documents that explicit pins are user-directed and MAY warn (see #423/#425
|
||||
// acceptance: the "0 quality-warnings by construction" guarantee is for
|
||||
// AUTO-LAYOUT, not for coordinates the user pinned by hand).
|
||||
for (const n of graph.nodes) {
|
||||
if (!n.pinned) continue;
|
||||
const px = Math.max(0, n.pinned.x);
|
||||
const py = Math.max(0, n.pinned.y);
|
||||
const g = geo.get(n.id);
|
||||
if (g) {
|
||||
g.x = px;
|
||||
g.y = py;
|
||||
} else {
|
||||
const sz = { w: 140, h: 60 };
|
||||
geo.set(n.id, { x: px, y: py, w: sz.w, h: sz.h });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Incremental variant of applyHints: apply `pinned` only to NEW nodes (those
|
||||
* absent from `existing`); an existing node's coordinates are NEVER changed
|
||||
* (acceptance #3). sameLayerAs/layer snapping is intentionally skipped in the
|
||||
* incremental path — moving a new node's layer axis could still be desired, but
|
||||
* it must never move an existing cell, so we keep the incremental contract
|
||||
* simple: existing cells are frozen, new pinned nodes honour their pin.
|
||||
*/
|
||||
export function applyHintsForNew(
|
||||
graph: Graph,
|
||||
geo: Map<string, { x: number; y: number; w: number; h: number }>,
|
||||
existing: Map<string, { x: number; y: number }>,
|
||||
): void {
|
||||
for (const n of graph.nodes) {
|
||||
if (existing.has(n.id)) continue; // never move an existing cell
|
||||
if (!n.pinned) continue;
|
||||
const px = Math.max(0, n.pinned.x); // clamp out-of-bounds pins non-negative
|
||||
const py = Math.max(0, n.pinned.y);
|
||||
const g = geo.get(n.id);
|
||||
if (g) {
|
||||
g.x = px;
|
||||
g.y = py;
|
||||
} else {
|
||||
geo.set(n.id, { x: px, y: py, w: 140, h: 60 });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// --- layout runner ---------------------------------------------------------
|
||||
|
||||
const ELK_TIMEOUT_MS = 5000;
|
||||
|
||||
/**
|
||||
* Run ELK over the mapped graph and return computed geometry per id (coords are
|
||||
* parent-relative, matching mxGraph's convention for container children). On any
|
||||
* ELK failure/timeout the returned map is empty and the caller falls back to a
|
||||
* deterministic grid placement (so the write never fails on a layout hiccup).
|
||||
*/
|
||||
export async function runElk(
|
||||
elk: ElkGraph,
|
||||
): Promise<Map<string, { x: number; y: number; w: number; h: number }>> {
|
||||
const geo = new Map<string, { x: number; y: number; w: number; h: number }>();
|
||||
let timer: ReturnType<typeof setTimeout> | undefined;
|
||||
try {
|
||||
const Ctor: any = (ELK as any).default ?? ELK;
|
||||
const inst = new Ctor();
|
||||
const timeout = new Promise<never>((_, reject) => {
|
||||
timer = setTimeout(() => reject(new Error("ELK timed out")), ELK_TIMEOUT_MS);
|
||||
});
|
||||
const laid = (await Promise.race([inst.layout(elk as any), timeout])) as ElkGraph;
|
||||
const walk = (n: ElkNode) => {
|
||||
if (n.id !== "root") {
|
||||
geo.set(n.id, {
|
||||
x: Math.round(n.x ?? 0),
|
||||
y: Math.round(n.y ?? 0),
|
||||
w: Math.round(n.width ?? 140),
|
||||
h: Math.round(n.height ?? 60),
|
||||
});
|
||||
}
|
||||
for (const c of n.children ?? []) walk(c);
|
||||
};
|
||||
walk(laid);
|
||||
} catch {
|
||||
return new Map(); // best-effort: empty -> caller uses fallback grid.
|
||||
} finally {
|
||||
if (timer) clearTimeout(timer);
|
||||
}
|
||||
return geo;
|
||||
}
|
||||
|
||||
// --- XML assembler ---------------------------------------------------------
|
||||
|
||||
/** Order groups so a parent group always precedes its nested children. */
|
||||
function topoSortGroups(groups: GraphGroup[], groupIds: Set<string>): GraphGroup[] {
|
||||
const byId = new Map(groups.map((g) => [g.id, g]));
|
||||
const out: GraphGroup[] = [];
|
||||
const done = new Set<string>();
|
||||
const visit = (g: GraphGroup, stack: Set<string>) => {
|
||||
if (done.has(g.id)) return;
|
||||
if (stack.has(g.id)) return; // cycle guard
|
||||
stack.add(g.id);
|
||||
if (g.group && groupIds.has(g.group) && g.group !== g.id) {
|
||||
const parent = byId.get(g.group);
|
||||
if (parent) visit(parent, stack);
|
||||
}
|
||||
stack.delete(g.id);
|
||||
if (!done.has(g.id)) {
|
||||
done.add(g.id);
|
||||
out.push(g);
|
||||
}
|
||||
};
|
||||
for (const g of groups) visit(g, new Set());
|
||||
return out;
|
||||
}
|
||||
|
||||
function xmlEscapeAttr(s: string): string {
|
||||
return s
|
||||
.replace(/&/g, "&")
|
||||
.replace(/</g, "<")
|
||||
.replace(/>/g, ">")
|
||||
.replace(/"/g, """)
|
||||
.replace(/\r\n|\r|\n/g, "
"); // literal newline -> the linter-approved entity
|
||||
}
|
||||
|
||||
export interface AssembleResult {
|
||||
modelXml: string;
|
||||
iconsResolved: number;
|
||||
iconsMissing: string[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Assemble the final mxGraphModel XML from the graph + resolved styles/coords.
|
||||
* Guarantees BY CONSTRUCTION that the #423 linter passes:
|
||||
* - id=0 and id=1(parent=0) sentinels;
|
||||
* - each node/group is vertex="1" (containers get container=1 via the style);
|
||||
* - each edge is edge="1" with a child <mxGeometry relative="1" as="geometry"/>;
|
||||
* - group children set parent=<groupId> and RELATIVE coords; an edge between
|
||||
* two different parents is parent="1";
|
||||
* - labels are XML-escaped and any newline is 
.
|
||||
* `geo` may be empty (ELK failed) — then a deterministic grid is used so the
|
||||
* output is still valid and non-overlapping (>=170px stride).
|
||||
*
|
||||
* QUALITY-WARNING GUARANTEE: the "0 quality-warnings by construction" promise
|
||||
* holds for AUTO-LAYOUT — ELK spacing plus applyHints' cross-axis spread for the
|
||||
* server-positioned `layer`/`sameLayerAs` hints keep shapes >=MIN_SHAPE_GAP
|
||||
* apart. It does NOT extend to explicit `pinned` coordinates: those are
|
||||
* user-directed, so two nodes the user pins to the same/overlapping point are
|
||||
* user error the server honours verbatim (only clamped non-negative) and MAY
|
||||
* therefore produce a quality warning.
|
||||
*/
|
||||
export function assembleModel(
|
||||
graph: Graph,
|
||||
opts: {
|
||||
preset: PresetData;
|
||||
styles: Map<string, { style: string; w: number; h: number; iconResolved: boolean }>;
|
||||
geo: Map<string, { x: number; y: number; w: number; h: number }>;
|
||||
},
|
||||
): AssembleResult {
|
||||
const { preset, styles, geo } = opts;
|
||||
const groupIds = new Set((graph.groups ?? []).map((g) => g.id));
|
||||
const nodeById = new Map(graph.nodes.map((n) => [n.id, n]));
|
||||
|
||||
// Fallback grid when ELK produced nothing: lay ungrouped nodes on a grid with
|
||||
// a 190px stride (>150 gap). Grouped nodes/groups are placed inside their group.
|
||||
const fallback = geo.size === 0;
|
||||
const gridPos = (i: number) => ({ x: 40 + (i % 5) * 200, y: 40 + Math.floor(i / 5) * 140 });
|
||||
|
||||
const cells: string[] = ['<mxCell id="0"/>', '<mxCell id="1" parent="0"/>'];
|
||||
|
||||
// Groups first (they are parents of their members). A nested group sets
|
||||
// parent=<parentGroupId>; emit parents before children so parent-exists holds.
|
||||
let gi = 0;
|
||||
const groupGeo = new Map<string, { x: number; y: number; w: number; h: number }>();
|
||||
const orderedGroups = topoSortGroups(graph.groups ?? [], groupIds);
|
||||
for (const g of orderedGroups) {
|
||||
const gg = geo.get(g.id) ?? { ...gridPos(gi++), w: 320, h: 220 };
|
||||
groupGeo.set(g.id, gg);
|
||||
const style = groupStyle(preset);
|
||||
const gParent = g.group && groupIds.has(g.group) && g.group !== g.id ? g.group : "1";
|
||||
cells.push(
|
||||
`<mxCell id="${xmlEscapeAttr(g.id)}" value="${xmlEscapeAttr(g.label)}" style="${style}" vertex="1" parent="${xmlEscapeAttr(gParent)}">` +
|
||||
`<mxGeometry x="${gg.x}" y="${gg.y}" width="${gg.w}" height="${gg.h}" as="geometry"/></mxCell>`,
|
||||
);
|
||||
}
|
||||
|
||||
// Nodes. A grouped node's coords are RELATIVE to its group (ELK already
|
||||
// returns child coords relative to the parent; for the fallback grid we place
|
||||
// children on a small in-group grid).
|
||||
let ungrouped = (graph.groups?.length ?? 0);
|
||||
const inGroupIndex = new Map<string, number>();
|
||||
let iconsResolved = 0;
|
||||
const iconsMissing: string[] = [];
|
||||
for (const n of graph.nodes) {
|
||||
const st = styles.get(n.id)!;
|
||||
if (n.icon) {
|
||||
if (st.iconResolved) iconsResolved++;
|
||||
else iconsMissing.push(n.id);
|
||||
}
|
||||
let x: number;
|
||||
let y: number;
|
||||
const g = geo.get(n.id);
|
||||
if (g && !fallback) {
|
||||
x = g.x;
|
||||
y = g.y;
|
||||
} else if (n.group && groupIds.has(n.group)) {
|
||||
const k = inGroupIndex.get(n.group) ?? 0;
|
||||
inGroupIndex.set(n.group, k + 1);
|
||||
x = 30 + (k % 3) * 180;
|
||||
y = 40 + Math.floor(k / 3) * 120;
|
||||
} else {
|
||||
const p = gridPos(ungrouped++);
|
||||
x = p.x;
|
||||
y = p.y;
|
||||
}
|
||||
const parent = n.group && groupIds.has(n.group) ? n.group : "1";
|
||||
cells.push(
|
||||
`<mxCell id="${xmlEscapeAttr(n.id)}" value="${xmlEscapeAttr(n.label)}" style="${st.style}" vertex="1" parent="${xmlEscapeAttr(parent)}">` +
|
||||
`<mxGeometry x="${x}" y="${y}" width="${st.w}" height="${st.h}" as="geometry"/></mxCell>`,
|
||||
);
|
||||
}
|
||||
|
||||
// Edges. parent="1" whenever the two endpoints have different container
|
||||
// parents (or either is a group); otherwise the shared group id.
|
||||
(graph.edges ?? []).forEach((e, i) => {
|
||||
const style = edgeStyle(preset, e.kind);
|
||||
const fromNode = nodeById.get(e.from);
|
||||
const toNode = nodeById.get(e.to);
|
||||
const fromParent = fromNode?.group && groupIds.has(fromNode.group) ? fromNode.group : "1";
|
||||
const toParent = toNode?.group && groupIds.has(toNode.group) ? toNode.group : "1";
|
||||
const parent = fromParent === toParent ? fromParent : "1";
|
||||
const label = e.label ? ` value="${xmlEscapeAttr(e.label)}"` : "";
|
||||
cells.push(
|
||||
`<mxCell id="ge${i}"${label} style="${style}" edge="1" parent="${xmlEscapeAttr(parent)}" ` +
|
||||
`source="${xmlEscapeAttr(e.from)}" target="${xmlEscapeAttr(e.to)}">` +
|
||||
`<mxGeometry relative="1" as="geometry"/></mxCell>`,
|
||||
);
|
||||
});
|
||||
|
||||
const modelAttrs =
|
||||
'dx="0" dy="0" grid="1" gridSize="10" page="1" pageWidth="850" pageHeight="1100" adaptiveColors="auto"';
|
||||
const modelXml = `<mxGraphModel ${modelAttrs}><root>${cells.join("")}</root></mxGraphModel>`;
|
||||
return { modelXml, iconsResolved, iconsMissing };
|
||||
}
|
||||
|
||||
// --- incremental merge -----------------------------------------------------
|
||||
|
||||
let _mergeWindow: any = null;
|
||||
function mergeWindow(): any {
|
||||
if (!_mergeWindow) _mergeWindow = new JSDOM("").window;
|
||||
return _mergeWindow;
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge the freshly-assembled graph XML with the EXISTING diagram model so an
|
||||
* incremental "add a node" call never drops a hand-placed cell. `assembleModel`
|
||||
* emits ONLY the passed graph's cells; on its own it would replace the whole
|
||||
* model, wiping any existing cell the caller didn't re-list. This splices every
|
||||
* existing cell that the graph does NOT re-list (preserved verbatim: coords,
|
||||
* style, edges) into the assembled root:
|
||||
* - id in the graph -> the graph's (re-laid) cell wins (already assembled;
|
||||
* coords are frozen for existing ids via the incremental geo path);
|
||||
* - id NOT in the graph -> the existing cell is preserved verbatim;
|
||||
* - a graph node absent from the existing model -> added (offset clear).
|
||||
* The sentinels ("0"/"1") come from the assembled model and are never doubled.
|
||||
*/
|
||||
function mergeExistingCells(
|
||||
assembledXml: string,
|
||||
existingModelXml: string,
|
||||
graph: Graph,
|
||||
): string {
|
||||
const win = mergeWindow();
|
||||
const parser = new win.DOMParser();
|
||||
const existingDoc = parser.parseFromString(existingModelXml, "application/xml");
|
||||
if (existingDoc.getElementsByTagName("parsererror").length > 0) {
|
||||
// Existing model unreadable: fall back to the assembled model alone (still a
|
||||
// valid diagram — better than throwing on a corrupt prior file).
|
||||
return assembledXml;
|
||||
}
|
||||
const assembledDoc = parser.parseFromString(assembledXml, "application/xml");
|
||||
const root = assembledDoc.getElementsByTagName("root")[0];
|
||||
if (!root) return assembledXml;
|
||||
|
||||
// Ids the assembled model already emitted (graph nodes/groups/edges + sentinels).
|
||||
const assembledIds = new Set<string>();
|
||||
for (const el of Array.from(root.getElementsByTagName("mxCell")) as any[]) {
|
||||
const id = el.getAttribute("id");
|
||||
if (id) assembledIds.add(id);
|
||||
}
|
||||
// The graph's own ids: any existing cell with one of these is superseded by the
|
||||
// assembled version and must NOT be re-imported.
|
||||
const graphIds = new Set<string>([
|
||||
...graph.nodes.map((n) => n.id),
|
||||
...(graph.groups ?? []).map((g) => g.id),
|
||||
]);
|
||||
|
||||
const existingCells = Array.from(
|
||||
existingDoc.getElementsByTagName("mxCell"),
|
||||
) as any[];
|
||||
for (const el of existingCells) {
|
||||
const id = el.getAttribute("id") ?? "";
|
||||
if (id === "0" || id === "1") continue; // sentinels come from the assembled model
|
||||
if (graphIds.has(id)) continue; // graph re-lists it -> assembled version wins
|
||||
if (assembledIds.has(id)) continue; // id collision guard -> keep assembled
|
||||
root.appendChild(assembledDoc.importNode(el, true));
|
||||
assembledIds.add(id);
|
||||
}
|
||||
|
||||
const ser = new win.XMLSerializer();
|
||||
return ser.serializeToString(assembledDoc.documentElement);
|
||||
}
|
||||
|
||||
// --- top-level: graph -> mxGraphModel XML ----------------------------------
|
||||
|
||||
export interface BuildFromGraphResult {
|
||||
modelXml: string;
|
||||
iconsResolved: number;
|
||||
iconsMissing: string[];
|
||||
layout: LayoutMode;
|
||||
}
|
||||
|
||||
/**
|
||||
* The full server-side pipeline: validate -> resolve styles/icons -> map to ELK
|
||||
* -> run ELK (or fall back) -> assemble linter-clean XML. `existingCoords` is
|
||||
* supplied for `layout:"incremental"` (the coordinates of the diagram's current
|
||||
* cells, so they are preserved and only new nodes are placed). `existingModelXml`
|
||||
* is the current diagram's full model XML — in incremental mode every existing
|
||||
* cell the graph does NOT re-list is MERGED back in verbatim so a hand-placed
|
||||
* cell is never dropped (WARNING #4). Pure — no network.
|
||||
*/
|
||||
export async function buildFromGraph(
|
||||
graph: Graph,
|
||||
layout: LayoutMode = "full",
|
||||
existingCoords?: Map<string, { x: number; y: number }>,
|
||||
existingModelXml?: string,
|
||||
): Promise<BuildFromGraphResult> {
|
||||
validateGraph(graph);
|
||||
const preset = getPreset(graph.preset);
|
||||
|
||||
// Resolve every node's style + size (icon or generic-by-kind).
|
||||
const styles = new Map<
|
||||
string,
|
||||
{ style: string; w: number; h: number; iconResolved: boolean }
|
||||
>();
|
||||
const sizes = new Map<string, NodeSize>();
|
||||
for (const n of graph.nodes) {
|
||||
const s = resolveNodeStyle(preset, n);
|
||||
styles.set(n.id, s);
|
||||
sizes.set(n.id, { w: s.w, h: s.h });
|
||||
}
|
||||
// Group sizes: seed a min box; ELK computes the real size when it lays out.
|
||||
for (const g of graph.groups ?? []) sizes.set(g.id, { w: 240, h: 180 });
|
||||
|
||||
let geo = new Map<string, { x: number; y: number; w: number; h: number }>();
|
||||
|
||||
if (layout === "incremental" && existingCoords && existingCoords.size > 0) {
|
||||
// INCREMENTAL: keep every existing cell's coords VERBATIM (acceptance #3 —
|
||||
// never move a hand-arranged cell) and lay out ONLY the new nodes, then
|
||||
// offset that block clear of the existing bbox so nothing overlaps.
|
||||
for (const [id, c] of existingCoords) {
|
||||
const sz = sizes.get(id) ?? { w: 140, h: 60 };
|
||||
geo.set(id, { x: c.x, y: c.y, w: sz.w, h: sz.h });
|
||||
}
|
||||
const newIds = new Set(
|
||||
graph.nodes.filter((n) => !existingCoords.has(n.id)).map((n) => n.id),
|
||||
);
|
||||
if (newIds.size > 0) {
|
||||
const elk = graphToElk(graph, sizes, { only: newIds });
|
||||
const laid = await runElk(elk);
|
||||
// Place the new block below the existing content (a clear >=170px gap).
|
||||
let maxY = 0;
|
||||
for (const c of existingCoords.values()) maxY = Math.max(maxY, c.y);
|
||||
const offsetY = maxY + 200;
|
||||
for (const [id, g] of laid) {
|
||||
if (newIds.has(id)) geo.set(id, { ...g, y: g.y + offsetY });
|
||||
else if (!geo.has(id)) geo.set(id, g); // a new group container
|
||||
}
|
||||
}
|
||||
// Hints still apply to NEW pinned nodes only (existing ones stay put).
|
||||
applyHintsForNew(graph, geo, existingCoords);
|
||||
} else if (layout !== "none") {
|
||||
const elk = graphToElk(graph, sizes);
|
||||
geo = await runElk(elk);
|
||||
applyHints(graph, geo);
|
||||
} else if (existingCoords) {
|
||||
// layout:"none" with prior coords -> keep them verbatim.
|
||||
for (const [id, c] of existingCoords) {
|
||||
const sz = sizes.get(id) ?? { w: 140, h: 60 };
|
||||
geo.set(id, { x: c.x, y: c.y, w: sz.w, h: sz.h });
|
||||
}
|
||||
}
|
||||
|
||||
const assembled = assembleModel(graph, { preset, styles, geo });
|
||||
// In incremental mode, splice back every existing cell the graph didn't
|
||||
// re-list so an "add one node" call preserves the user's manual layout.
|
||||
let modelXml = assembled.modelXml;
|
||||
if (
|
||||
layout === "incremental" &&
|
||||
existingModelXml &&
|
||||
existingCoords &&
|
||||
existingCoords.size > 0
|
||||
) {
|
||||
modelXml = mergeExistingCells(modelXml, existingModelXml, graph);
|
||||
}
|
||||
return {
|
||||
modelXml,
|
||||
iconsResolved: assembled.iconsResolved,
|
||||
iconsMissing: assembled.iconsMissing,
|
||||
layout,
|
||||
};
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
// Progressive-disclosure authoring reference for the `drawio_guide` tool
|
||||
// Progressive-disclosure authoring reference for the `drawioGuide` tool
|
||||
// (issue #424, stage 2). The FULL draw.io authoring guide would bloat every
|
||||
// context window, so it is split into small sections the model reads on demand:
|
||||
// skeleton | layout | containers | icons-aws | icons-azure
|
||||
@@ -22,7 +22,7 @@ export const GUIDE_SECTIONS: GuideSection[] = [
|
||||
"icons-azure",
|
||||
];
|
||||
|
||||
const SKELETON = `# drawio_guide: skeleton
|
||||
const SKELETON = `# drawioGuide: skeleton
|
||||
|
||||
Canonical mxGraph skeleton. id="0" and id="1" are MANDATORY sentinels; every
|
||||
real cell has parent="1" (or a container id). Set adaptiveColors="auto" on the
|
||||
@@ -50,7 +50,7 @@ model so Docmost's dark theme adapts strokeColor/fillColor/fontColor="default".
|
||||
</mxGraphModel>
|
||||
\`\`\`
|
||||
|
||||
Three accepted inputs to drawio_create/drawio_update: a bare <mxGraphModel>, a
|
||||
Three accepted inputs to drawioCreate/drawioUpdate: a bare <mxGraphModel>, a
|
||||
full <mxfile> (decoded to its first page), or a raw list of <mxCell> (the server
|
||||
wraps it and adds the id=0/id=1 sentinels).
|
||||
|
||||
@@ -58,12 +58,12 @@ Hard rules: a cell is vertex="1" XOR edge="1" (a container/group is neither);
|
||||
every edge has a child <mxGeometry relative="1" as="geometry"/>; ids are unique;
|
||||
no XML comments; put html=1 in styles and XML-escape value (& -> &,
|
||||
< -> <); a newline in a label is 
, never a literal \\n. Don't guess
|
||||
shape=mxgraph.* names — call drawio_shapes first (a wrong name renders empty).`;
|
||||
shape=mxgraph.* names — call drawioShapes first (a wrong name renders empty).`;
|
||||
|
||||
const LAYOUT = `# drawio_guide: layout
|
||||
const LAYOUT = `# drawioGuide: layout
|
||||
|
||||
Turn "make it look good" into checkable numbers. Or pass layout:"elk" to
|
||||
drawio_create/drawio_update and the server computes coordinates for you (ELK
|
||||
drawioCreate/drawioUpdate and the server computes coordinates for you (ELK
|
||||
layered layout, honouring nested containers) — you declare structure, it places
|
||||
pixels.
|
||||
|
||||
@@ -96,7 +96,7 @@ The linter returns quality WARNINGS (bbox overlap, edge through a shape,
|
||||
edge-on-edge, gap <150px, label wider than its shape, negative/off-page coords).
|
||||
They do not block the write — fix them and retry, max 2 iterations.`;
|
||||
|
||||
const CONTAINERS = `# drawio_guide: containers
|
||||
const CONTAINERS = `# drawioGuide: containers
|
||||
|
||||
Groups/zones are TRANSPARENT containers. A coloured group fill is an instant
|
||||
"AI-generated" tell — never fill a group.
|
||||
@@ -133,10 +133,10 @@ Example (transparent zone with two children and an internal edge):
|
||||
</mxCell>
|
||||
\`\`\``;
|
||||
|
||||
const ICONS_AWS = `# drawio_guide: icons-aws
|
||||
const ICONS_AWS = `# drawioGuide: icons-aws
|
||||
|
||||
Two mutually-exclusive AWS icon patterns — mixing them is the #1 cause of empty
|
||||
boxes. Always call drawio_shapes for the exact resIcon name; do not guess.
|
||||
boxes. Always call drawioShapes for the exact resIcon name; do not guess.
|
||||
|
||||
| Level | style | strokeColor |
|
||||
|---|---|---|
|
||||
@@ -168,7 +168,7 @@ Group stencils (transparent containers): AWS Cloud group_aws_cloud_alt, VPC
|
||||
group_vpc2, Subnet group_security_group, Account group_account; subnets use
|
||||
shape=mxgraph.aws4.group;grIcon=mxgraph.aws4.group_public_subnet;.`;
|
||||
|
||||
const ICONS_AZURE = `# drawio_guide: icons-azure
|
||||
const ICONS_AZURE = `# drawioGuide: icons-azure
|
||||
|
||||
shape=mxgraph.azure2.* does NOT render in every host. Use the portable
|
||||
image-style instead:
|
||||
@@ -190,7 +190,7 @@ an absolute URL fallback for the image:
|
||||
https://raw.githubusercontent.com/jgraph/drawio/dev/src/main/webapp/img/lib/azure2/<category>/<Icon>.svg
|
||||
\`\`\`
|
||||
|
||||
Call drawio_shapes with the service name (e.g. "cosmos", "api management",
|
||||
Call drawioShapes with the service name (e.g. "cosmos", "api management",
|
||||
"front door") to get the exact image-style string and default 68x68 size.`;
|
||||
|
||||
const CONTENT: Record<GuideSection, string> = {
|
||||
@@ -216,13 +216,13 @@ export function getGuideSection(section?: string): {
|
||||
return { section: key, content: CONTENT[key], sections: GUIDE_SECTIONS };
|
||||
}
|
||||
const index =
|
||||
"# drawio_guide\n\nProgressive-disclosure draw.io authoring reference. " +
|
||||
"Call drawio_guide(section) with one of:\n" +
|
||||
"# drawioGuide\n\nProgressive-disclosure draw.io authoring reference. " +
|
||||
"Call drawioGuide(section) with one of:\n" +
|
||||
"- skeleton — canonical mxGraph XML, sentinels, the three accepted inputs, hard rules\n" +
|
||||
"- layout — spacing heuristics, edge routing, the layout:\"elk\" option, quality warnings\n" +
|
||||
"- containers — transparent groups, relative child coords, cross-container edges, swimlanes\n" +
|
||||
"- icons-aws — the service/resource icon patterns, category colors, rebrandings, blocklist\n" +
|
||||
"- icons-azure — the portable image-style paths\n\n" +
|
||||
"Also call drawio_shapes(query) for verified stencil style-strings.";
|
||||
"Also call drawioShapes(query) for verified stencil style-strings.";
|
||||
return { section: "index", content: index, sections: GUIDE_SECTIONS };
|
||||
}
|
||||
|
||||
@@ -19,7 +19,7 @@ const DEFAULT_W = 140;
|
||||
const DEFAULT_H = 60;
|
||||
|
||||
// DoS bounds for the in-process ELK layout. The mxGraph XML is LLM-supplied
|
||||
// (layout:"elk" in drawio_create/drawio_update) and elkjs runs synchronously on
|
||||
// (layout:"elk" in drawioCreate/drawioUpdate) and elkjs runs synchronously on
|
||||
// the MCP server's event loop, so an unbounded graph would block it for
|
||||
// seconds-to-minutes. A ~1MB XML (well under the stage-1 16MB cap) can carry
|
||||
// thousands of nodes. We cap the graph size and race the layout against a
|
||||
@@ -81,7 +81,7 @@ interface ElkGraph extends ElkNode {
|
||||
/**
|
||||
* Apply an ELK layered layout to a drawio input and return a full mxGraphModel
|
||||
* string with rewritten geometry. Accepts the same three input forms as
|
||||
* drawio_create (a bare model, an <mxfile>, or a <mxCell> list). Async because
|
||||
* drawioCreate (a bare model, an <mxfile>, or a <mxCell> list). Async because
|
||||
* elkjs' layout() is promise-based. On any layout failure the ORIGINAL
|
||||
* (normalized) model is returned unchanged — layout is best-effort polish, never
|
||||
* a reason to fail the write.
|
||||
|
||||
@@ -0,0 +1,347 @@
|
||||
// Pure Mermaid `flowchart` -> graph-JSON parser for `drawioFromMermaid` (issue
|
||||
// #425, stage 3, OPTIONAL). The escape clause in the issue: convert WITHOUT
|
||||
// Electron/draw.io-CLI, so a pure text parser only. It handles the common wiki
|
||||
// flowchart subset — node shapes, labelled/dashed edges, subgraphs (-> groups),
|
||||
// and the direction header — and emits a Graph the drawioFromGraph pipeline
|
||||
// renders as an EDITABLE draw.io diagram. Anything beyond flowchart (sequence /
|
||||
// class / state) throws a clear error so the model falls back to drawioFromGraph.
|
||||
//
|
||||
// DELIBERATELY NARROW: this is not a full Mermaid grammar (Mermaid's own parser
|
||||
// is a 100KB+ browser dependency). It covers `flowchart`/`graph` with the node
|
||||
// shapes and edge arrows that show up in practice; unusual syntax is skipped
|
||||
// rather than mis-parsed, and a diagram that yields no nodes throws.
|
||||
|
||||
import type { Graph, GraphNode, GraphEdge, GraphGroup } from "./drawio-graph.js";
|
||||
|
||||
export class MermaidParseError extends Error {
|
||||
constructor(message: string) {
|
||||
super(`drawioFromMermaid: ${message}`);
|
||||
this.name = "MermaidParseError";
|
||||
}
|
||||
}
|
||||
|
||||
// Input-size bounds applied BEFORE parsing. Without them a pathological mermaid
|
||||
// string (e.g. 300000 connection lines, or 20000 nested `subgraph`s) builds a
|
||||
// huge intermediate node/edge/group structure that OOM-crashes the worker — the
|
||||
// downstream validateGraph caps in drawio-graph can't help because the parser
|
||||
// exhausts the heap constructing the intermediate FIRST. These caps reject the
|
||||
// over-limit input fast, before a single line is parsed.
|
||||
const MAX_MERMAID_CHARS = 200_000; // ~200 KB of source is far beyond any real diagram.
|
||||
const MAX_MERMAID_LINES = 20_000;
|
||||
const MAX_MERMAID_GROUPS = 500; // parity with drawio-graph's MAX_GRAPH_GROUPS.
|
||||
// Per connection line, the number of chained nodes we will expand (`A-->B-->C`).
|
||||
const MAX_CHAIN_NODES = 500;
|
||||
|
||||
const DIRECTIONS: Record<string, Graph["direction"]> = {
|
||||
LR: "LR",
|
||||
RL: "RL",
|
||||
TB: "TB",
|
||||
TD: "TB",
|
||||
BT: "BT",
|
||||
};
|
||||
|
||||
/**
|
||||
* Node-shape delimiters -> a semantic `kind`. Mermaid encodes shape in the
|
||||
* bracket style; we map the common ones to the palette kinds so the diagram is
|
||||
* colored meaningfully (a decision/diamond -> queue, a database cylinder -> db,
|
||||
* a rounded/stadium -> service, a subroutine/hexagon -> gateway, default rect ->
|
||||
* service). The label text lives between the delimiters.
|
||||
*/
|
||||
interface ShapeDef {
|
||||
open: string;
|
||||
close: string;
|
||||
kind: string;
|
||||
}
|
||||
// Order matters: longer/multi-char delimiters first so "([" beats "(".
|
||||
const SHAPES: ShapeDef[] = [
|
||||
{ open: "([", close: "])", kind: "service" }, // stadium
|
||||
{ open: "[[", close: "]]", kind: "gateway" }, // subroutine
|
||||
{ open: "[(", close: ")]", kind: "db" }, // cylinder-ish / database
|
||||
{ open: "((", close: "))", kind: "external" }, // circle
|
||||
{ open: "{{", close: "}}", kind: "gateway" }, // hexagon
|
||||
{ open: "[", close: "]", kind: "service" }, // rectangle
|
||||
{ open: "(", close: ")", kind: "service" }, // rounded
|
||||
{ open: "{", close: "}", kind: "queue" }, // rhombus / decision
|
||||
{ open: ">", close: "]", kind: "external" }, // asymmetric flag
|
||||
];
|
||||
|
||||
/** Strip Mermaid label quoting/escapes and normalise whitespace. */
|
||||
function cleanLabel(raw: string): string {
|
||||
let s = raw.trim();
|
||||
if (
|
||||
(s.startsWith('"') && s.endsWith('"')) ||
|
||||
(s.startsWith("'") && s.endsWith("'"))
|
||||
) {
|
||||
s = s.slice(1, -1);
|
||||
}
|
||||
return s.replace(/<br\s*\/?>/gi, " ").replace(/\s+/g, " ").trim();
|
||||
}
|
||||
|
||||
/** A single edge-arrow spec: its regex and the resulting edge `kind`. */
|
||||
interface ArrowDef {
|
||||
re: RegExp;
|
||||
kind: string;
|
||||
}
|
||||
// Dotted arrows (`-.->`) -> async; thick (`==>`) stay sync; normal `-->`/`---`.
|
||||
// Each captures an optional `|label|` OR inline label between the two arrow
|
||||
// halves. Applied to the segment between two node tokens.
|
||||
const ARROWS: ArrowDef[] = [
|
||||
{ re: /-\.->|-\.-/, kind: "async" },
|
||||
{ re: /==>|===/, kind: "sync" },
|
||||
{ re: /-->|---/, kind: "sync" },
|
||||
];
|
||||
|
||||
interface ParsedRef {
|
||||
id: string;
|
||||
node?: GraphNode;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a single node token like `A`, `A[Label]`, `db[(Orders)]`, `d{Choose}`.
|
||||
* Returns the id and, when the token declares a shape/label, a GraphNode.
|
||||
*/
|
||||
function parseNodeToken(token: string): ParsedRef | null {
|
||||
const t = token.trim();
|
||||
if (t === "") return null;
|
||||
for (const shape of SHAPES) {
|
||||
const oi = t.indexOf(shape.open);
|
||||
if (oi <= 0) continue;
|
||||
if (!t.endsWith(shape.close)) continue;
|
||||
const id = t.slice(0, oi).trim();
|
||||
const label = cleanLabel(t.slice(oi + shape.open.length, t.length - shape.close.length));
|
||||
if (!id) return null;
|
||||
return { id, node: { id, label: label || id, kind: shape.kind } };
|
||||
}
|
||||
// Bare id (no shape declared here — may be defined elsewhere).
|
||||
if (/^[A-Za-z0-9_.-]+$/.test(t)) return { id: t };
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Split a connection line into [leftToken, arrowSegment, rightToken]. Returns
|
||||
* null if the line has no arrow. The arrow segment may embed a label as
|
||||
* `-->|text|` or `-- text -->`.
|
||||
*/
|
||||
function splitConnection(
|
||||
line: string,
|
||||
): { left: string; right: string; kind: string; label?: string } | null {
|
||||
for (const arrow of ARROWS) {
|
||||
// Find the arrow occurrence. Support a mid-arrow label: `A -- text --> B`.
|
||||
const m = arrow.re.exec(line);
|
||||
if (!m) continue;
|
||||
const idx = m.index;
|
||||
let left = line.slice(0, idx).trim();
|
||||
let rest = line.slice(idx + m[0].length).trim();
|
||||
let label: string | undefined;
|
||||
// Pipe label: `-->|HTTPS| B`.
|
||||
const pipe = /^\|([^|]*)\|\s*(.*)$/.exec(rest);
|
||||
if (pipe) {
|
||||
label = cleanLabel(pipe[1]);
|
||||
rest = pipe[2].trim();
|
||||
}
|
||||
// Mid-arrow label on the left side: `A -- text` before the arrow half.
|
||||
const midLeft = /^(.*?)\s*--\s*(.+)$/.exec(left);
|
||||
if (!label && midLeft && /-\.|--|==/.test(line.slice(0, idx))) {
|
||||
// Only treat as a label when there's clearly text after `--`.
|
||||
if (!/[\[\](){}]/.test(midLeft[2])) {
|
||||
left = midLeft[1].trim();
|
||||
label = cleanLabel(midLeft[2]);
|
||||
}
|
||||
}
|
||||
if (!left || !rest) return null;
|
||||
return { left, right: rest, kind: arrow.kind, label };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse Mermaid flowchart text into a Graph. Handles the header
|
||||
* (`flowchart LR` / `graph TD`), `subgraph <id>[title] … end` blocks (-> groups),
|
||||
* node declarations, and connection lines. Throws MermaidParseError for a
|
||||
* non-flowchart diagram or when nothing parses.
|
||||
*/
|
||||
export function mermaidToGraph(mermaid: string): Graph {
|
||||
if (typeof mermaid !== "string" || mermaid.trim() === "") {
|
||||
throw new MermaidParseError("empty mermaid input");
|
||||
}
|
||||
// Size guards FIRST — bound the raw input before building any intermediate.
|
||||
if (mermaid.length > MAX_MERMAID_CHARS) {
|
||||
throw new MermaidParseError(
|
||||
`input is ${mermaid.length} chars (max ${MAX_MERMAID_CHARS}); split the diagram or use drawioFromGraph`,
|
||||
);
|
||||
}
|
||||
const rawLines = mermaid.split(/\r?\n/);
|
||||
if (rawLines.length > MAX_MERMAID_LINES) {
|
||||
throw new MermaidParseError(
|
||||
`input has ${rawLines.length} lines (max ${MAX_MERMAID_LINES}); split the diagram or use drawioFromGraph`,
|
||||
);
|
||||
}
|
||||
const nodes = new Map<string, GraphNode>();
|
||||
const groups: GraphGroup[] = [];
|
||||
const edges: GraphEdge[] = [];
|
||||
let direction: Graph["direction"] = "LR";
|
||||
let sawHeader = false;
|
||||
|
||||
// Stack of active subgraph ids (nesting); the top is the current group.
|
||||
const groupStack: string[] = [];
|
||||
let anonGroup = 0;
|
||||
|
||||
const ensureNode = (ref: ParsedRef) => {
|
||||
const existing = nodes.get(ref.id);
|
||||
if (ref.node) {
|
||||
if (existing) {
|
||||
// Fill in a label/kind if this token declared a shape and the prior didn't.
|
||||
if (existing.label === existing.id && ref.node.label !== ref.node.id)
|
||||
existing.label = ref.node.label;
|
||||
if (!existing.kind) existing.kind = ref.node.kind;
|
||||
} else {
|
||||
nodes.set(ref.id, { ...ref.node });
|
||||
}
|
||||
} else if (!existing) {
|
||||
nodes.set(ref.id, { id: ref.id, label: ref.id, kind: "service" });
|
||||
}
|
||||
// Assign to the current subgraph if inside one and not yet grouped.
|
||||
const cur = groupStack[groupStack.length - 1];
|
||||
const n = nodes.get(ref.id)!;
|
||||
if (cur && n.group == null) n.group = cur;
|
||||
};
|
||||
|
||||
for (const raw of rawLines) {
|
||||
let line = raw.trim();
|
||||
if (line === "" || line.startsWith("%%")) continue; // blank / comment
|
||||
|
||||
// Header.
|
||||
const header = /^(flowchart|graph)\s+([A-Za-z]{2})\b/.exec(line);
|
||||
if (header) {
|
||||
sawHeader = true;
|
||||
const dir = DIRECTIONS[header[2].toUpperCase()];
|
||||
if (dir) direction = dir;
|
||||
continue;
|
||||
}
|
||||
if (/^(sequenceDiagram|classDiagram|stateDiagram|erDiagram|gantt|pie|journey)\b/.test(line)) {
|
||||
throw new MermaidParseError(
|
||||
`only 'flowchart'/'graph' is supported (got '${line.split(/\s+/)[0]}'); use drawioFromGraph instead`,
|
||||
);
|
||||
}
|
||||
|
||||
// Subgraph open: `subgraph id [Title]` or `subgraph Title`.
|
||||
const sg = /^subgraph\s+(.+)$/.exec(line);
|
||||
if (sg) {
|
||||
const spec = sg[1].trim();
|
||||
let id: string;
|
||||
let label: string;
|
||||
const bracket = /^([A-Za-z0-9_.-]+)\s*\[(.+)\]$/.exec(spec);
|
||||
if (bracket) {
|
||||
id = bracket[1];
|
||||
label = cleanLabel(bracket[2]);
|
||||
} else if (/^[A-Za-z0-9_.-]+$/.test(spec)) {
|
||||
id = spec;
|
||||
label = spec;
|
||||
} else {
|
||||
id = `sg${anonGroup++}`;
|
||||
label = cleanLabel(spec);
|
||||
}
|
||||
if (!groups.some((g) => g.id === id)) {
|
||||
if (groups.length >= MAX_MERMAID_GROUPS) {
|
||||
throw new MermaidParseError(
|
||||
`too many subgraphs (max ${MAX_MERMAID_GROUPS}); use drawioFromGraph for a diagram this large`,
|
||||
);
|
||||
}
|
||||
groups.push({ id, label, kind: "group" });
|
||||
}
|
||||
groupStack.push(id);
|
||||
continue;
|
||||
}
|
||||
if (/^end\b/.test(line)) {
|
||||
groupStack.pop();
|
||||
continue;
|
||||
}
|
||||
// `direction LR` inside a subgraph — apply to the top-level direction.
|
||||
const innerDir = /^direction\s+([A-Za-z]{2})\b/.exec(line);
|
||||
if (innerDir) {
|
||||
const dir = DIRECTIONS[innerDir[1].toUpperCase()];
|
||||
if (dir) direction = dir;
|
||||
continue;
|
||||
}
|
||||
// Style/class/click directives: ignore (no visual mapping in our palette).
|
||||
if (/^(style|classDef|class|click|linkStyle)\b/.test(line)) continue;
|
||||
|
||||
// Strip a trailing semicolon.
|
||||
if (line.endsWith(";")) line = line.slice(0, -1).trim();
|
||||
|
||||
// Connection line (possibly chained: A --> B --> C).
|
||||
const conn = splitConnection(line);
|
||||
if (conn) {
|
||||
// Handle a simple chain by re-splitting the right side.
|
||||
let leftTok = conn.left;
|
||||
let seg: typeof conn | null = conn;
|
||||
let guard = 0;
|
||||
while (seg) {
|
||||
if (guard++ >= MAX_CHAIN_NODES) {
|
||||
// Don't silently drop the tail of an over-long chain — surface it so
|
||||
// the model knows the diagram was too large rather than getting a
|
||||
// quietly-truncated result.
|
||||
throw new MermaidParseError(
|
||||
`a single connection chain exceeds ${MAX_CHAIN_NODES} nodes; split it or use drawioFromGraph`,
|
||||
);
|
||||
}
|
||||
const leftRef = parseNodeToken(leftTok);
|
||||
// The right side may itself contain another arrow (a chain).
|
||||
const nextSeg = splitConnection(seg.right);
|
||||
const rightTokenStr = nextSeg ? seg.right.slice(0, splitIndex(seg.right)) : seg.right;
|
||||
const rightRef = parseNodeToken(nextSeg ? nextSeg.left : seg.right);
|
||||
if (leftRef && rightRef) {
|
||||
ensureNode(leftRef);
|
||||
ensureNode(rightRef);
|
||||
edges.push({
|
||||
from: leftRef.id,
|
||||
to: rightRef.id,
|
||||
label: seg.label,
|
||||
kind: seg.kind,
|
||||
});
|
||||
}
|
||||
if (!nextSeg) break;
|
||||
leftTok = nextSeg.left;
|
||||
seg = nextSeg;
|
||||
void rightTokenStr;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
// Standalone node declaration `A[Label]` OR a bare member ref `C` inside a
|
||||
// subgraph (which claims that node for the current group).
|
||||
const nodeRef = parseNodeToken(line);
|
||||
if (nodeRef && (nodeRef.node || groupStack.length > 0)) {
|
||||
ensureNode(nodeRef);
|
||||
continue;
|
||||
}
|
||||
// Unknown line: skip silently (robustness over strictness).
|
||||
}
|
||||
|
||||
if (!sawHeader && nodes.size === 0) {
|
||||
throw new MermaidParseError(
|
||||
"input does not look like a mermaid flowchart (no 'flowchart'/'graph' header and no nodes)",
|
||||
);
|
||||
}
|
||||
if (nodes.size === 0) {
|
||||
throw new MermaidParseError("no nodes parsed from the flowchart");
|
||||
}
|
||||
|
||||
const graph: Graph = {
|
||||
nodes: Array.from(nodes.values()),
|
||||
direction,
|
||||
};
|
||||
if (groups.length > 0) graph.groups = groups;
|
||||
if (edges.length > 0) graph.edges = edges;
|
||||
return graph;
|
||||
}
|
||||
|
||||
/** Index of the first arrow in a segment (for chain splitting). */
|
||||
function splitIndex(s: string): number {
|
||||
let best = -1;
|
||||
for (const arrow of ARROWS) {
|
||||
const m = arrow.re.exec(s);
|
||||
if (m && (best === -1 || m.index < best)) best = m.index;
|
||||
}
|
||||
return best === -1 ? s.length : best;
|
||||
}
|
||||
@@ -0,0 +1,151 @@
|
||||
// Semantic color/line presets for the graph tools (issue #425, stage 3). The
|
||||
// PALETTE is DATA (packages/mcp/data/drawio-presets.json), not code: a node
|
||||
// `kind` maps to a { fillColor, strokeColor, fontColor } slot and an edge `kind`
|
||||
// maps to line-style props, per named preset (`default` / `dark` /
|
||||
// `colorblind-safe`). This module only loads that data and turns a slot into a
|
||||
// draw.io style fragment. The INVARIANT of the graph tools is that the model
|
||||
// never sees a style string — it names a `kind`, the server picks the slot.
|
||||
//
|
||||
// Loading mirrors drawio-shapes.ts: the JSON is read once via `import.meta.url`
|
||||
// relative to the built module. That is why this module (and drawio-graph.ts
|
||||
// which imports it) is reached ONLY through client.ts's ESM build and never
|
||||
// value-imported into the zod-agnostic tool-specs.ts (which the in-app server
|
||||
// type-checks under module:commonjs, where `import.meta` is a TS1343 error).
|
||||
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
/** A node color slot: the three draw.io color values for a `kind`. */
|
||||
export interface NodeSlot {
|
||||
fillColor: string;
|
||||
strokeColor: string;
|
||||
fontColor: string;
|
||||
}
|
||||
|
||||
/** An edge line style: the extra style props appended for an edge `kind`. */
|
||||
export interface EdgeStyle {
|
||||
props: string;
|
||||
}
|
||||
|
||||
export interface PresetData {
|
||||
canvasDark: boolean;
|
||||
okabeIto?: string[];
|
||||
nodes: Record<string, NodeSlot>;
|
||||
edges: Record<string, EdgeStyle>;
|
||||
edgeDefault: { strokeColor: string; fontColor: string };
|
||||
group: { strokeColor: string; fontColor: string };
|
||||
}
|
||||
|
||||
interface PresetsFile {
|
||||
presets: Record<string, PresetData>;
|
||||
}
|
||||
|
||||
/** The three shipped preset names. */
|
||||
export const PRESET_NAMES = ["default", "dark", "colorblind-safe"] as const;
|
||||
export type PresetName = (typeof PRESET_NAMES)[number];
|
||||
|
||||
/** Every node `kind` the base palette defines (also the generic-shape kinds). */
|
||||
export const NODE_KINDS = [
|
||||
"service",
|
||||
"db",
|
||||
"queue",
|
||||
"gateway",
|
||||
"error",
|
||||
"external",
|
||||
"security",
|
||||
] as const;
|
||||
export type NodeKind = (typeof NODE_KINDS)[number];
|
||||
|
||||
/** Edge `kind`s the palette styles; anything else falls back to `sync`. */
|
||||
export const EDGE_KINDS = ["sync", "async", "error"] as const;
|
||||
export type EdgeKind = (typeof EDGE_KINDS)[number];
|
||||
|
||||
let _presets: Record<string, PresetData> | null = null;
|
||||
|
||||
function presetsPath(): URL {
|
||||
// build/lib/drawio-presets.js -> ../../data/drawio-presets.json
|
||||
return new URL("../../data/drawio-presets.json", import.meta.url);
|
||||
}
|
||||
|
||||
/** Load + parse the bundled preset table once, then cache it. */
|
||||
export function loadPresets(): Record<string, PresetData> {
|
||||
if (_presets) return _presets;
|
||||
const json = readFileSync(presetsPath(), "utf-8");
|
||||
const parsed = JSON.parse(json) as PresetsFile;
|
||||
_presets = parsed.presets;
|
||||
return _presets;
|
||||
}
|
||||
|
||||
/** Resolve a preset by name, defaulting to `default` for an unknown name. */
|
||||
export function getPreset(name?: string): PresetData {
|
||||
const presets = loadPresets();
|
||||
if (name && presets[name]) return presets[name];
|
||||
return presets["default"];
|
||||
}
|
||||
|
||||
/** The slot for a node `kind` in a preset, falling back to `service`. */
|
||||
export function nodeSlot(preset: PresetData, kind?: string): NodeSlot {
|
||||
if (kind && preset.nodes[kind]) return preset.nodes[kind];
|
||||
return preset.nodes["service"];
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the draw.io style string for a GENERIC (no-icon) node of a given kind.
|
||||
* A rounded rectangle carrying the slot's fill/stroke/font. `whiteSpace=wrap`
|
||||
* and `html=1` let a long label wrap inside the shape (the assembler also sizes
|
||||
* the shape to the label, so the linter's label-overflow warning never fires).
|
||||
*/
|
||||
export function genericNodeStyle(preset: PresetData, kind?: string): string {
|
||||
const s = nodeSlot(preset, kind);
|
||||
return (
|
||||
`rounded=1;whiteSpace=wrap;html=1;` +
|
||||
`fillColor=${s.fillColor};strokeColor=${s.strokeColor};fontColor=${s.fontColor};`
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Overlay the preset's node slot colors onto a resolved ICON style-string
|
||||
* (from the shape catalog). An AWS/Azure icon carries its OWN mandatory
|
||||
* fill/stroke (the category color / white outline) that MUST NOT be recolored,
|
||||
* so for an icon we only ensure a readable fontColor when the preset is dark;
|
||||
* otherwise the icon style is returned verbatim. Keeping the icon's own colors
|
||||
* is deliberate: recoloring an AWS service icon breaks its category semantics.
|
||||
*/
|
||||
export function iconNodeStyle(preset: PresetData, iconStyle: string): string {
|
||||
if (!preset.canvasDark) return iconStyle;
|
||||
// On a dark canvas an icon's fontColor is usually a dark ink that vanishes;
|
||||
// append a light fontColor (icons put their label BELOW the glyph, so this
|
||||
// only affects the caption, never the glyph fill).
|
||||
if (/fontColor=/.test(iconStyle)) {
|
||||
return iconStyle.replace(/fontColor=[^;]*/, "fontColor=#e0e0e0");
|
||||
}
|
||||
return iconStyle + (iconStyle.endsWith(";") ? "" : ";") + "fontColor=#e0e0e0;";
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the draw.io style for an edge of a given `kind`. Base is an orthogonal
|
||||
* connector (edgeStyle=orthogonalEdgeStyle) with rounded corners and an open
|
||||
* arrowhead, plus the preset's default stroke/font, then the kind's extra props
|
||||
* (dashed / colored) overlaid. An unknown kind falls back to `sync` (solid).
|
||||
*/
|
||||
export function edgeStyle(preset: PresetData, kind?: string): string {
|
||||
const k = kind && preset.edges[kind] ? kind : "sync";
|
||||
const base =
|
||||
`edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=open;` +
|
||||
`strokeColor=${preset.edgeDefault.strokeColor};fontColor=${preset.edgeDefault.fontColor};`;
|
||||
return base + preset.edges[k].props;
|
||||
}
|
||||
|
||||
/**
|
||||
* Group (container) style: ALWAYS transparent (`fillColor=none;container=1;`)
|
||||
* per the spec, carrying the preset's group stroke/font. `dropTarget=1` marks it
|
||||
* a drop target in the editor; `verticalAlign=top;align=left;spacingLeft=8;` puts
|
||||
* the group label in the top-left like draw.io's own boundary containers.
|
||||
*/
|
||||
export function groupStyle(preset: PresetData): string {
|
||||
return (
|
||||
`rounded=0;whiteSpace=wrap;html=1;` +
|
||||
`fillColor=none;container=1;dropTarget=1;collapsible=0;` +
|
||||
`strokeColor=${preset.group.strokeColor};fontColor=${preset.group.fontColor};` +
|
||||
`verticalAlign=top;align=left;spacingLeft=8;spacingTop=4;`
|
||||
);
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
// Verified draw.io shape catalog for the `drawio_shapes` tool (issue #424,
|
||||
// Verified draw.io shape catalog for the `drawioShapes` tool (issue #424,
|
||||
// stage 2). This is the fix for AI-generated diagrams' #1 defect: guessed
|
||||
// `shape=mxgraph.*` names that render as EMPTY BOXES because the stencil does
|
||||
// not exist. Instead of guessing, the model queries this catalog and gets back
|
||||
|
||||
@@ -1070,7 +1070,7 @@ export function prepareModel(inputXml: string): PreparedModel {
|
||||
};
|
||||
}
|
||||
|
||||
/** Cell count of a decoded model (user cells only) — used by drawio_get meta. */
|
||||
/** Cell count of a decoded model (user cells only) — used by drawioGet meta. */
|
||||
export function countUserCells(modelXml: string): number {
|
||||
return parseCells(modelXml).filter((c) => c.id !== "0" && c.id !== "1").length;
|
||||
}
|
||||
|
||||
@@ -83,16 +83,32 @@ export function filterComment(comment: any, markdownContent?: string) {
|
||||
};
|
||||
}
|
||||
|
||||
// Map one server search hit to the MCP output contract (#443):
|
||||
// { pageId, title, path, snippet, score }
|
||||
//
|
||||
// INVARIANT: the only page identifier exposed is `pageId` (the server `id`
|
||||
// UUID). The server also carries `slugId` — it is NEVER surfaced.
|
||||
//
|
||||
// GRACEFUL DEGRADATION: against a stock upstream server the opt-in lookup DTO
|
||||
// fields are stripped, so the response is the legacy FTS shape (no path/snippet/
|
||||
// score, a `highlight` + `rank` instead). We synthesize the contract from
|
||||
// whatever is present: `snippet` falls back to the FTS `highlight`, `score` to
|
||||
// the FTS `rank`, and `path` to [] (upstream has no path). This keeps the tool
|
||||
// usable even when the server has not been upgraded.
|
||||
export function filterSearchResult(result: any) {
|
||||
return {
|
||||
id: result.id,
|
||||
pageId: result.id,
|
||||
title: result.title,
|
||||
parentPageId: result.parentPageId,
|
||||
createdAt: result.createdAt,
|
||||
updatedAt: result.updatedAt,
|
||||
rank: result.rank,
|
||||
highlight: result.highlight,
|
||||
spaceId: result.space?.id,
|
||||
spaceName: result.space?.name,
|
||||
path: Array.isArray(result.path) ? result.path : [],
|
||||
snippet:
|
||||
typeof result.snippet === "string"
|
||||
? result.snippet
|
||||
: (result.highlight ?? ""),
|
||||
score:
|
||||
typeof result.score === "number"
|
||||
? result.score
|
||||
: typeof result.rank === "number"
|
||||
? result.rank
|
||||
: 0,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -16,8 +16,8 @@
|
||||
* `insertInlineFootnote` live in `@docmost/prosemirror-markdown` (next to the
|
||||
* importer's `assembleFootnotes`, #414), so this file stays a pure mirror.
|
||||
*
|
||||
* Why it exists: every NON-editor write path (markdown import, update_page_json,
|
||||
* docmost_transform, insert_footnote) builds ProseMirror JSON directly, so the
|
||||
* Why it exists: every NON-editor write path (markdown import, updatePageJson,
|
||||
* docmostTransform, insertFootnote) builds ProseMirror JSON directly, so the
|
||||
* editor's footnote plugins never run and the canonical topology (sequential
|
||||
* numbering by first reference, one trailing list, no orphans, no raw `[^id]`)
|
||||
* was never enforced. Running this at the end of every write path closes that
|
||||
@@ -28,8 +28,8 @@
|
||||
* `canonicalizeFootnotes(doc)` before writing — the current callers are
|
||||
* `markdownToProseMirrorCanonical` (page markdown import/update; the plain
|
||||
* `markdownToProseMirror` used for COMMENT bodies must NOT, or it would drop a
|
||||
* reference-less definition), `update_page_json`, `docmost_transform`,
|
||||
* `insert_footnote`, and `copy_page_content`. Append/prepend FRAGMENT writes MUST
|
||||
* reference-less definition), `updatePageJson`, `docmostTransform`,
|
||||
* `insertFootnote`, and `copyPageContent`. Append/prepend FRAGMENT writes MUST
|
||||
* NOT canonicalize. This is deliberately per-call-site (the replace-vs-fragment
|
||||
* and comment-vs-page nuances make a single naive wrapper unsafe).
|
||||
*/
|
||||
|
||||
@@ -283,7 +283,7 @@ export function applyTextEdits(
|
||||
for (const edit of edits) {
|
||||
if (!edit.find) throw new Error("edit.find must be a non-empty string");
|
||||
|
||||
// HARD-REFUSE formatting changes. edit_page_text edits PLAIN TEXT only and
|
||||
// HARD-REFUSE formatting changes. editPageText edits PLAIN TEXT only and
|
||||
// writes the replacement verbatim, so it cannot add/remove marks. We refuse
|
||||
// only a pure formatting TOGGLE: find and replace differ ONLY by balanced
|
||||
// markdown markers (e.g. find:"~~$69~~" / replace:"$69", or find:"M5Stack" /
|
||||
@@ -304,22 +304,22 @@ export function applyTextEdits(
|
||||
failed.push({
|
||||
find: edit.find,
|
||||
reason:
|
||||
"edit_page_text edits plain text only and cannot add or remove formatting marks (bold/italic/strike/code/link); it writes the replacement as LITERAL text. This edit looks like a formatting change (markdown markers in find/replace). To change marks, read the block with get_page_json and use patch_node (or update_page_json) to set the node's marks array.",
|
||||
"editPageText edits plain text only and cannot add or remove formatting marks (bold/italic/strike/code/link); it writes the replacement as LITERAL text. This edit looks like a formatting change (markdown markers in find/replace). To change marks, read the block with getPageJson and use patchNode (or updatePageJson) to set the node's marks array.",
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
// HARD-REFUSE inline footnote tokens (#410). `^[...]` in a `replace` is
|
||||
// markdown that only becomes a real footnote when a whole markdown body is
|
||||
// written (create_page / update_page_content / import_page_markdown). Written
|
||||
// through edit_page_text it stays a LITERAL string in the text — the exact
|
||||
// written (createPage / update_page_content / importPageMarkdown). Written
|
||||
// through editPageText it stays a LITERAL string in the text — the exact
|
||||
// failure mode #410 fixes — so refuse it here (defense-in-depth) and point the
|
||||
// caller at insert_footnote, mirroring the formatting-marker refusal above.
|
||||
// caller at insertFootnote, mirroring the formatting-marker refusal above.
|
||||
if (/\^\[[\s\S]*?\]/.test(edit.replace)) {
|
||||
failed.push({
|
||||
find: edit.find,
|
||||
reason:
|
||||
"edit_page_text writes the replacement as LITERAL text, so a `^[...]` footnote token does not parse into a real footnote (it would appear verbatim in the page). To add a footnote to existing text, use insert_footnote (anchorText = where, text = the note).",
|
||||
"editPageText writes the replacement as LITERAL text, so a `^[...]` footnote token does not parse into a real footnote (it would appear verbatim in the page). To add a footnote to existing text, use insertFootnote (anchorText = where, text = the note).",
|
||||
});
|
||||
continue;
|
||||
}
|
||||
@@ -381,12 +381,12 @@ export function applyTextEdits(
|
||||
let reason: string;
|
||||
if (existsAcrossAtom) {
|
||||
reason =
|
||||
"match crosses a non-text inline node (image/break/mention); use update_page_json for structural changes.";
|
||||
"match crosses a non-text inline node (image/break/mention); use updatePageJson for structural changes.";
|
||||
} else {
|
||||
// Append a bounded "closest text" hint: find the FIRST block that
|
||||
// contains the longest whitespace-delimited token (>= 3 chars) of the
|
||||
// (stripped, then raw) locator, and quote that block's plain text. Shared
|
||||
// with create_comment via closestBlockHint so both give the same hint.
|
||||
// with createComment via closestBlockHint so both give the same hint.
|
||||
reason = "text not found in the document." + closestBlockHint(blockPlain, edit.find);
|
||||
}
|
||||
failed.push({ find: edit.find, reason });
|
||||
|
||||
@@ -0,0 +1,243 @@
|
||||
/**
|
||||
* Single-BLOCK markdown fragment support for `patch_node` / `insert_node`
|
||||
* (#413). These tools accept EITHER a raw ProseMirror `node` (fine attr/mark
|
||||
* work) OR a `markdown` string (the recommended default): a small markdown
|
||||
* fragment is run through the canonical importer, yielding the SAME topology a
|
||||
* full-page markdown import would — so a block written via markdown is
|
||||
* canonically identical to the same content imported whole (no "second canon").
|
||||
*
|
||||
* The importer produces a full `{type:"doc", content:[...blocks..., footnotesList?]}`.
|
||||
* A fragment write needs the BLOCKS separately from the footnote DEFINITIONS so
|
||||
* the caller can splice the blocks into the live document and merge the
|
||||
* definitions into the page's TAIL footnote list via the existing footnote
|
||||
* machinery (`insertInlineFootnote`'s `appendDefinition` + `canonicalizeFootnotes`).
|
||||
*
|
||||
* Footnote id-collision safety: the importer assigns sequential ids (`fn-1`,
|
||||
* `fn-2`, …) starting from 1 for EVERY fragment, so a fragment's `fn-1` would
|
||||
* collide with an existing page footnote also numbered `fn-1` — and
|
||||
* `canonicalizeFootnotes` matches references to definitions BY id, so the
|
||||
* fragment's reference would silently re-hang onto the page's unrelated
|
||||
* definition. To make the merge safe regardless of the page's current numbering,
|
||||
* every fragment footnote id is REMAPPED to a fresh uuid (via the importer's own
|
||||
* `generateFootnoteId`) across BOTH the references (inside the blocks) and the
|
||||
* definitions before either is handed back. Content-identical notes still merge
|
||||
* downstream via `normalizeAndMergeFootnotes` (content-key), and the whole doc is
|
||||
* renumbered by `canonicalizeFootnotes`, so the caller-visible numbering stays
|
||||
* canonical.
|
||||
*/
|
||||
|
||||
import { markdownToProseMirror } from "./collaboration.js";
|
||||
import { generateFootnoteId } from "@docmost/prosemirror-markdown";
|
||||
import { docmostSchema } from "./docmost-schema.js";
|
||||
|
||||
/** True if `value` is a non-null, non-array object. */
|
||||
function isObject(value: any): value is Record<string, any> {
|
||||
return value != null && typeof value === "object" && !Array.isArray(value);
|
||||
}
|
||||
|
||||
/**
|
||||
* Deep-walk `node` collecting every footnote id it uses (on `footnoteReference`
|
||||
* and `footnoteDefinition` nodes) and build a stable OLD->NEW remap, minting a
|
||||
* fresh uuid per distinct old id. The map is shared across a fragment's blocks
|
||||
* and definitions so a reference and its definition receive the SAME new id.
|
||||
*/
|
||||
function buildFootnoteIdRemap(nodes: any[]): Map<string, string> {
|
||||
const remap = new Map<string, string>();
|
||||
const visit = (node: any): void => {
|
||||
if (!isObject(node)) return;
|
||||
if (
|
||||
(node.type === "footnoteReference" ||
|
||||
node.type === "footnoteDefinition") &&
|
||||
isObject(node.attrs) &&
|
||||
typeof node.attrs.id === "string" &&
|
||||
node.attrs.id !== ""
|
||||
) {
|
||||
if (!remap.has(node.attrs.id)) {
|
||||
remap.set(node.attrs.id, generateFootnoteId());
|
||||
}
|
||||
}
|
||||
if (Array.isArray(node.content)) {
|
||||
for (const child of node.content) visit(child);
|
||||
}
|
||||
};
|
||||
for (const n of nodes) visit(n);
|
||||
return remap;
|
||||
}
|
||||
|
||||
/** Rewrite every footnote id in `node` IN PLACE using `remap` (deep). */
|
||||
function applyFootnoteIdRemap(node: any, remap: Map<string, string>): void {
|
||||
if (!isObject(node)) return;
|
||||
if (
|
||||
(node.type === "footnoteReference" || node.type === "footnoteDefinition") &&
|
||||
isObject(node.attrs) &&
|
||||
typeof node.attrs.id === "string"
|
||||
) {
|
||||
const next = remap.get(node.attrs.id);
|
||||
if (next) node.attrs.id = next;
|
||||
}
|
||||
if (Array.isArray(node.content)) {
|
||||
for (const child of node.content) applyFootnoteIdRemap(child, remap);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate a short random block id for an imported block that arrives without one
|
||||
* (the markdown importer emits `attrs.id: null`). Mirrors the mcp `freshId`
|
||||
* convention (base36 random, unique within one document). The patch path then
|
||||
* OVERWRITES the first block's id with the target id; every other block keeps the
|
||||
* fresh id minted here — so a 1 -> N section rewrite yields addressable,
|
||||
* comment-anchorable blocks rather than a run of null-id paragraphs.
|
||||
*/
|
||||
function freshBlockId(): string {
|
||||
return (
|
||||
Math.random().toString(36).slice(2, 12) +
|
||||
Math.random().toString(36).slice(2, 6)
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Assign a fresh id to every top-level block whose `attrs.id` is null/missing,
|
||||
* IN PLACE. Only the block's own id is touched (not descendants — those keep the
|
||||
* importer's structure). Ensures each imported block is independently addressable.
|
||||
*/
|
||||
function assignFreshBlockIds(blocks: any[]): void {
|
||||
for (const b of blocks) {
|
||||
if (!isObject(b)) continue;
|
||||
if (!isObject(b.attrs)) b.attrs = {};
|
||||
if (b.attrs.id == null || b.attrs.id === "") {
|
||||
b.attrs.id = freshBlockId();
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** The parsed shape of a markdown fragment: its blocks + footnote definitions. */
|
||||
export interface MarkdownFragment {
|
||||
/** Top-level blocks, in order, with the trailing `footnotesList` removed. */
|
||||
blocks: any[];
|
||||
/**
|
||||
* The `footnoteDefinition` nodes lifted from the imported `footnotesList`, with
|
||||
* ids already remapped to match the references left inside `blocks`. Empty when
|
||||
* the fragment used no footnotes.
|
||||
*/
|
||||
definitions: any[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Import a markdown fragment and return its blocks separately from its footnote
|
||||
* definitions, with all footnote ids remapped to fresh uuids (see the file
|
||||
* header). The importer's `^[body]` inline-footnote handling is used verbatim —
|
||||
* `^[...]` in the fragment is a first-class footnote, NOT rejected — so the
|
||||
* markdown path matches the full-page import exactly.
|
||||
*
|
||||
* Throws when the fragment imports to zero blocks (an empty / whitespace-only
|
||||
* markdown string is not a valid block write).
|
||||
*/
|
||||
export async function importMarkdownFragment(
|
||||
markdown: string,
|
||||
): Promise<MarkdownFragment> {
|
||||
const doc = await markdownToProseMirror(markdown);
|
||||
const content: any[] = Array.isArray(doc?.content) ? doc.content : [];
|
||||
|
||||
const blocks: any[] = [];
|
||||
const definitions: any[] = [];
|
||||
for (const node of content) {
|
||||
if (isObject(node) && node.type === "footnotesList") {
|
||||
// Lift the definitions out of the list; the list wrapper itself is
|
||||
// reconstructed on the page by the canonicalizer after the merge.
|
||||
if (Array.isArray(node.content)) {
|
||||
for (const def of node.content) {
|
||||
if (isObject(def) && def.type === "footnoteDefinition") {
|
||||
definitions.push(def);
|
||||
}
|
||||
}
|
||||
}
|
||||
continue;
|
||||
}
|
||||
blocks.push(node);
|
||||
}
|
||||
|
||||
if (blocks.length === 0) {
|
||||
throw new Error(
|
||||
"markdown fragment produced no blocks — provide non-empty markdown, or use `node` for a raw ProseMirror node",
|
||||
);
|
||||
}
|
||||
|
||||
// Remap footnote ids across BOTH blocks and definitions so a fragment `fn-1`
|
||||
// cannot collide with a page footnote of the same number.
|
||||
const remap = buildFootnoteIdRemap([...blocks, ...definitions]);
|
||||
if (remap.size > 0) {
|
||||
for (const b of blocks) applyFootnoteIdRemap(b, remap);
|
||||
for (const d of definitions) applyFootnoteIdRemap(d, remap);
|
||||
}
|
||||
|
||||
// Every top-level block needs a stable id (the importer leaves them null). The
|
||||
// patch path OVERWRITES the first block's id with the target id afterwards.
|
||||
assignFreshBlockIds(blocks);
|
||||
|
||||
return { blocks, definitions };
|
||||
}
|
||||
|
||||
/**
|
||||
* True when `type` is a valid TOP-LEVEL child of the document node per the
|
||||
* canonical schema's content model — i.e. `get_node` can serialize it to
|
||||
* markdown by wrapping it in `{type:"doc",content:[node]}`. Derived from the
|
||||
* schema's `doc` contentMatch (NOT a hand-written type list) so it tracks the
|
||||
* schema automatically: `tableRow`/`tableCell`/`tableHeader` (addressed only via
|
||||
* `#<index>`) are NOT doc children and yield false, so `get_node` auto-falls back
|
||||
* to JSON for them.
|
||||
*/
|
||||
export function canBeDocChild(type: string | undefined): boolean {
|
||||
if (typeof type !== "string") return false;
|
||||
const nodeType = docmostSchema.nodes[type];
|
||||
if (!nodeType) return false;
|
||||
return docmostSchema.nodes.doc.contentMatch.matchType(nodeType) != null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Table-cell attributes that CANNOT survive a markdown round-trip: the converter
|
||||
* emits colspan/rowspan (and align) as HTML `<table>` cell attrs, but silently
|
||||
* drops `colwidth`, `backgroundColor`, and `backgroundColorName`. A markdown
|
||||
* `patch_node` on a block that carries any of these (a merged / colored /
|
||||
* fixed-width cell) would therefore lose them — so it is REJECTED, pointing the
|
||||
* caller at the table tools or the raw-`node` JSON path. `align` is intentionally
|
||||
* absent: it round-trips as GFM alignment.
|
||||
*/
|
||||
function cellCarriesUnrepresentableAttrs(node: any): boolean {
|
||||
if (!isObject(node)) return false;
|
||||
if (node.type !== "tableCell" && node.type !== "tableHeader") return false;
|
||||
const a = isObject(node.attrs) ? node.attrs : {};
|
||||
if ((a.colspan ?? 1) > 1) return true;
|
||||
if ((a.rowspan ?? 1) > 1) return true;
|
||||
if (a.colwidth != null) return true;
|
||||
if (a.backgroundColor != null) return true;
|
||||
if (a.backgroundColorName != null) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scan a target block (the node being replaced) for any table cell carrying an
|
||||
* attribute markdown cannot represent (colspan/rowspan/colwidth/background). When
|
||||
* one is found, return a human-readable list of the offending attr NAMES so the
|
||||
* caller can build an actionable rejection message; return null when the block is
|
||||
* safe to rewrite from markdown. Deep — a colored cell nested inside a table
|
||||
* inside a callout is still caught.
|
||||
*/
|
||||
export function findUnrepresentableTableAttrs(node: any): string | null {
|
||||
const found = new Set<string>();
|
||||
const visit = (n: any): void => {
|
||||
if (!isObject(n)) return;
|
||||
if (cellCarriesUnrepresentableAttrs(n)) {
|
||||
const a = isObject(n.attrs) ? n.attrs : {};
|
||||
if ((a.colspan ?? 1) > 1) found.add("colspan");
|
||||
if ((a.rowspan ?? 1) > 1) found.add("rowspan");
|
||||
if (a.colwidth != null) found.add("colwidth");
|
||||
if (a.backgroundColor != null) found.add("backgroundColor");
|
||||
if (a.backgroundColorName != null) found.add("backgroundColorName");
|
||||
}
|
||||
if (Array.isArray(n.content)) {
|
||||
for (const child of n.content) visit(child);
|
||||
}
|
||||
};
|
||||
visit(node);
|
||||
return found.size > 0 ? Array.from(found).sort().join(", ") : null;
|
||||
}
|
||||
@@ -10,6 +10,20 @@
|
||||
|
||||
const chains = new Map<string, Promise<unknown>>();
|
||||
|
||||
// Canonical UUID shape (versions 1–8, matching the `uuid` package's `validate`
|
||||
// that the server's isValidUUID uses). This is the SINGLE source of truth for
|
||||
// "is this a canonical page UUID?" in the MCP: client.ts's resolvePageId
|
||||
// imports isUuid from here to decide whether a pageId already IS a UUID (and so
|
||||
// needs no /pages/info round-trip). page.repo.ts treats any non-UUID pageId as
|
||||
// a slugId; a 10-char nanoid slugId never contains dashes, so it can never be
|
||||
// misread as a UUID here.
|
||||
export const UUID_RE =
|
||||
/^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
|
||||
|
||||
export function isUuid(value: string): boolean {
|
||||
return typeof value === "string" && UUID_RE.test(value);
|
||||
}
|
||||
|
||||
// The returned promise carries the real result/rejection of `fn` and MUST be
|
||||
// awaited/handled by the caller; only the internal chaining tail swallows
|
||||
// errors (purely to gate ordering).
|
||||
@@ -17,6 +31,25 @@ export function withPageLock<T>(
|
||||
pageId: string,
|
||||
fn: () => Promise<T>,
|
||||
): Promise<T> {
|
||||
// STRUCTURAL INVARIANT (issue #449, "resolve-then-lock"): the mutex key MUST
|
||||
// be the canonical page UUID, never a raw slugId. The whole write path relies
|
||||
// on the lock key AND the CollabSession cache key being the resolved UUID
|
||||
// (#260) — if a future write method forgot to call resolvePageId and locked
|
||||
// under a slugId, two writes to the same page would take DIFFERENT mutex keys
|
||||
// and silently lose serialization (clobbering live human edits). This was an
|
||||
// invariant enforced only by comments/convention; assert it in CODE so the
|
||||
// violation fails fast and loud at the lock instead of corrupting data in
|
||||
// prod. The centralizing helper (mutatePageContent/replacePageContent) already
|
||||
// guards a raw-input caller, but this backstop catches ANY path.
|
||||
if (!isUuid(pageId)) {
|
||||
throw new Error(
|
||||
`withPageLock: key must be a canonical page UUID, got '${pageId}'. ` +
|
||||
`The write path must resolvePageId(pageId) BEFORE locking so the ` +
|
||||
`mutex/CollabSession cache key is the UUID (invariant "resolve-then-` +
|
||||
`lock", #260/#449). A slugId or other non-UUID key would silently lose ` +
|
||||
`per-page serialization.`,
|
||||
);
|
||||
}
|
||||
// Wait for the previous op on this page; swallow its error so a failure does
|
||||
// not poison the queue for the next caller.
|
||||
const prev = (chains.get(pageId) ?? Promise.resolve()).catch(() => {});
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
*
|
||||
* `searchInDoc(doc, query, opts)` finds every occurrence of a literal substring
|
||||
* (default) or a regular expression across the page's TEXT CONTAINERS and
|
||||
* reports WHERE each match is — the container's ref (for get_node/patch_node;
|
||||
* reports WHERE each match is — the container's ref (for getNode/patchNode;
|
||||
* see the SearchMatch.nodeId note for the `#<index>` caveat), the top-level
|
||||
* block index, and a short context window around the hit. It never touches the
|
||||
* network, the DB, or the schema mirror; like `comment-anchor.ts` it is
|
||||
@@ -69,24 +69,24 @@ export interface SearchOptions {
|
||||
/** One located occurrence. */
|
||||
export interface SearchMatch {
|
||||
/**
|
||||
* The container's ref, for addressing the block with get_node/patch_node: its
|
||||
* The container's ref, for addressing the block with getNode/patchNode: its
|
||||
* `attrs.id` when it has one, otherwise `#<topLevelIndex>` of the nearest
|
||||
* top-level block. Table-cell/list-item paragraphs that carry no id fall back
|
||||
* to the `#<index>` form.
|
||||
*
|
||||
* CAVEAT: the `#<index>` form is accepted by get_node (getNodeByRef resolves
|
||||
* it by top-level index) but NOT by patch_node (replaceNodeById resolves only
|
||||
* CAVEAT: the `#<index>` form is accepted by getNode (getNodeByRef resolves
|
||||
* it by top-level index) but NOT by patchNode (replaceNodeById resolves only
|
||||
* by `attrs.id`), so id-less table/cell content can be READ by this ref but
|
||||
* not PATCHED by it.
|
||||
*
|
||||
* To anchor a comment, do NOT pass this ref to create_comment — it has no
|
||||
* To anchor a comment, do NOT pass this ref to createComment — it has no
|
||||
* nodeId parameter. A top-level comment needs an exact-text `selection` that
|
||||
* occurs once on the page (it fails if the text isn't found), so build a
|
||||
* UNIQUE `selection` from before+match+after and pass THAT as create_comment's
|
||||
* UNIQUE `selection` from before+match+after and pass THAT as createComment's
|
||||
* `selection`.
|
||||
*/
|
||||
nodeId: string;
|
||||
/** The top-level block index (as in get_outline). */
|
||||
/** The top-level block index (as in getOutline). */
|
||||
blockIndex: number;
|
||||
/** The container node's type (paragraph/heading/...). */
|
||||
type: string | undefined;
|
||||
@@ -188,12 +188,12 @@ export function searchInDoc(
|
||||
// --- edge-case guards (fail loudly so the agent can correct the call) ---
|
||||
if (typeof query !== "string" || query.trim().length === 0) {
|
||||
throw new Error(
|
||||
"search_in_page: query is empty — pass the text (or regex) to look for.",
|
||||
"searchInPage: query is empty — pass the text (or regex) to look for.",
|
||||
);
|
||||
}
|
||||
if (query.length > MAX_PATTERN_LENGTH) {
|
||||
throw new Error(
|
||||
`search_in_page: query is too long (${query.length} chars; max ${MAX_PATTERN_LENGTH}). Shorten the search text/pattern.`,
|
||||
`searchInPage: query is too long (${query.length} chars; max ${MAX_PATTERN_LENGTH}). Shorten the search text/pattern.`,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -212,7 +212,7 @@ export function searchInDoc(
|
||||
re = new RE2(query, caseSensitive ? "g" : "gi");
|
||||
} catch (e) {
|
||||
throw new Error(
|
||||
`search_in_page: invalid or unsupported regular expression: ${
|
||||
`searchInPage: invalid or unsupported regular expression: ${
|
||||
e instanceof Error ? e.message : String(e)
|
||||
} — RE2 does not support lookaround ((?=…)/(?<=…)) or backreferences (\\1); rewrite the pattern without them.`,
|
||||
);
|
||||
@@ -237,9 +237,9 @@ export function searchInDoc(
|
||||
// in a very long container.
|
||||
const text = blockPlainText(node);
|
||||
|
||||
// The container's own id addresses it verbatim in get_node/patch_node; a
|
||||
// The container's own id addresses it verbatim in getNode/patchNode; a
|
||||
// container with no id (e.g. a table-cell paragraph) falls back to the
|
||||
// top-level block's #<index> (readable via get_node, but not patchable —
|
||||
// top-level block's #<index> (readable via getNode, but not patchable —
|
||||
// see the SearchMatch.nodeId note).
|
||||
const id =
|
||||
isObject(node.attrs) && typeof node.attrs.id === "string" && node.attrs.id.length > 0
|
||||
|
||||
@@ -117,7 +117,7 @@ export function stripInlineMarkdown(s: string): string {
|
||||
|
||||
/**
|
||||
* Build a bounded "closest text" hint for an anchor/find MISS, shared by
|
||||
* edit_page_text (json-edit) and create_comment (client) so both surface the
|
||||
* editPageText (json-edit) and createComment (client) so both surface the
|
||||
* same self-correction affordance.
|
||||
*
|
||||
* Take the longest whitespace-delimited token (>= 3 chars) of the locator
|
||||
|
||||
@@ -740,7 +740,7 @@ export function insertInlineFootnote(
|
||||
// subtree, so a reference is never glued inside an existing definition (which
|
||||
// the canonicalizer would then drop as an orphan, losing that definition's
|
||||
// prose); and forbidBlockTypes refuses codeBlocks (an inline atom there is a
|
||||
// schema-invalid doc; insert_footnote skips validateDocStructure).
|
||||
// schema-invalid doc; insertFootnote skips validateDocStructure).
|
||||
// When the only anchor match is in such a place, the insert is refused and the
|
||||
// write aborts cleanly (inserted:false) instead of destroying content.
|
||||
const boundaryIdx = Array.isArray(doc?.content)
|
||||
@@ -774,6 +774,61 @@ export function insertInlineFootnote(
|
||||
return { doc: working, inserted: true, footnoteId, reused };
|
||||
}
|
||||
|
||||
/**
|
||||
* Merge an ARRAY of footnote definitions (e.g. the definitions lifted from an
|
||||
* imported markdown FRAGMENT) into `doc`\'s footnote list, then re-derive the
|
||||
* canonical footnote topology — the SAME two-step machinery `insertInlineFootnote`
|
||||
* uses (`appendDefinition` -> `normalizeAndMergeFootnotes` -> `canonicalizeFootnotes`).
|
||||
*
|
||||
* The fragment\'s `footnoteReference` nodes are assumed to ALREADY be spliced into
|
||||
* `doc` (inside the just-inserted blocks) with ids matching these definitions, so
|
||||
* after appending the definitions the canonicalizer orders/numbers everything by
|
||||
* first-reference order, merges content-identical notes, and drops any orphan.
|
||||
* Same documented caveat as every other write path: full canonicalization drops a
|
||||
* definition no reference points at.
|
||||
*
|
||||
* NOT merely a no-op when `definitions` is empty: it still canonicalizes when
|
||||
* the (post-splice) `doc` carries footnote artifacts (a `footnotesList` or any
|
||||
* `footnoteReference`), so a splice that removed the LAST referrer of a page
|
||||
* footnote drops the now-orphaned definition — matching a full page re-import
|
||||
* (which always canonicalizes) and preserving the "canonically identical to the
|
||||
* same content imported whole" invariant. A truly footnote-free doc (no artifacts
|
||||
* and no definitions) is returned untouched — the fast path, no clone. When the
|
||||
* work runs it goes through the pure passes (which clone), so the caller\'s `doc`
|
||||
* is not mutated.
|
||||
*/
|
||||
export function mergeFootnoteDefinitions(doc: any, definitions: any[]): any {
|
||||
const defs = Array.isArray(definitions) ? definitions : [];
|
||||
// True fast path ONLY when there is nothing to merge AND nothing to canonicalize
|
||||
// away; otherwise fall through so an orphan left by a splice is still dropped.
|
||||
if (defs.length === 0 && !hasFootnoteArtifacts(doc)) return doc;
|
||||
// Clone before appending: `appendDefinition` mutates in place, and the caller
|
||||
// must not see a half-merged doc if a later pass throws.
|
||||
let working = clone(doc);
|
||||
for (const def of defs) {
|
||||
appendDefinition(working, def);
|
||||
}
|
||||
// #419: normalize + merge glyph-forked definitions before canonicalizing.
|
||||
working = normalizeAndMergeFootnotes(working);
|
||||
working = canonicalizeFootnotes(working);
|
||||
return working;
|
||||
}
|
||||
|
||||
/**
|
||||
* True if `doc`'s tree contains any `footnotesList` node OR any
|
||||
* `footnoteReference` node. Used to decide whether an empty-`definitions` merge
|
||||
* must still canonicalize (to drop an orphan a splice left behind).
|
||||
*/
|
||||
function hasFootnoteArtifacts(doc: any): boolean {
|
||||
let found = false;
|
||||
walk(doc, (n) => {
|
||||
if (isObject(n) && (n.type === "footnotesList" || n.type === "footnoteReference")) {
|
||||
found = true;
|
||||
}
|
||||
});
|
||||
return found;
|
||||
}
|
||||
|
||||
/**
|
||||
* Append a definition node so the canonicalizer can order/place it: into the
|
||||
* first existing footnotesList, or a new trailing list when none exists.
|
||||
|
||||
@@ -1,11 +1,30 @@
|
||||
/**
|
||||
* Options for `buildPageTree`. Fully OPTIONAL so the existing call form
|
||||
* `buildPageTree(nodes)` keeps its historic behaviour (lean `{id, slugId,
|
||||
* title, children?}` output, no depth cut) unchanged.
|
||||
*
|
||||
* - `shape: "getTree"` — emit the #443 `getTree` output node shape
|
||||
* `{pageId, title, children?, hasChildren?}` instead of the lean
|
||||
* `{id, slugId, title, children?}` shape. `slugId`/`icon`/`position` are
|
||||
* never exposed (INVARIANT: only the UUID `pageId` leaves the MCP layer).
|
||||
* - `maxDepth` — trim the built tree to this many levels (root nodes are
|
||||
* depth 1). Only meaningful together with `shape: "getTree"` (the lean shape
|
||||
* has no `hasChildren` to signal a cut). See the depth logic below.
|
||||
*/
|
||||
export interface BuildPageTreeOptions {
|
||||
shape?: "lean" | "getTree";
|
||||
maxDepth?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Pure tree-builder: turn a flat array of sidebar-style page nodes (as produced
|
||||
* by `enumerateSpacePages`) into a nested tree.
|
||||
*
|
||||
* Input: a flat array of nodes. Each node is expected to carry at least
|
||||
* { id, slugId, title, position, parentPageId } (extra fields are ignored).
|
||||
* { id, slugId, title, position, parentPageId } (extra fields are ignored),
|
||||
* plus a server `hasChildren` boolean used by the `getTree` shape below.
|
||||
*
|
||||
* Output: an array of ROOT nodes, each shaped as
|
||||
* Output (default / `shape: "lean"`): an array of ROOT nodes, each shaped as
|
||||
* { id, slugId, title, children? }
|
||||
* where `children` is the array of child nodes (same shape, recursively). The
|
||||
* `children` key is OMITTED entirely when a node has no children — consistent
|
||||
@@ -13,6 +32,14 @@
|
||||
* lean (nesting alone conveys the structure; parentPageId/position/hasChildren
|
||||
* are intentionally dropped from the output).
|
||||
*
|
||||
* Output (`shape: "getTree"`, the #443 tool shape): each node is
|
||||
* { pageId, title, children?, hasChildren? }
|
||||
* — the server `id` is exposed as `pageId` (never `slugId`/`icon`/`position`).
|
||||
* `children` is omitted for leaves and for nodes trimmed by `maxDepth`.
|
||||
* `hasChildren: true` is set ONLY on a node whose children exist on the server
|
||||
* (per the flat item's `hasChildren`) but were CUT by `maxDepth`; on leaves and
|
||||
* on fully-expanded interior nodes the field is omitted (see `maxDepth` below).
|
||||
*
|
||||
* Linking rule: a node is attached as a child of `parentPageId` only when that
|
||||
* parent id is actually present in the input. Otherwise — including a null /
|
||||
* undefined `parentPageId`, or a parent that was capped out of the bounded walk
|
||||
@@ -26,18 +53,42 @@
|
||||
* fractional-index ASCII keys (e.g. "a0", "a1"). Nodes with a missing/undefined
|
||||
* `position` sort last.
|
||||
*
|
||||
* maxDepth (getTree shape only): the tree is built in FULL first, then trimmed
|
||||
* on the way out. Root nodes are depth 1. `maxDepth: N` keeps nodes at depth
|
||||
* <= N and drops the `children` of any node AT depth N. A node whose children
|
||||
* were dropped this way gets `hasChildren: true` when it actually had children
|
||||
* in the flat input (source of truth = the server `hasChildren` flag), so the
|
||||
* caller knows it can descend further with a follow-up `rootPageId` call. An
|
||||
* absent/undefined `maxDepth` means no cut (whole tree). `maxDepth <= 0` is
|
||||
* treated as "no cut" (defensive; the tool schema clamps to >= 1).
|
||||
*
|
||||
* Pure: no I/O, no network, deterministic.
|
||||
*/
|
||||
export function buildPageTree(nodes: any[]): any[] {
|
||||
type OutputNode = {
|
||||
export function buildPageTree(
|
||||
nodes: any[],
|
||||
options: BuildPageTreeOptions = {},
|
||||
): any[] {
|
||||
const getTreeShape = options.shape === "getTree";
|
||||
// A finite, positive cut only; anything else means "no cut".
|
||||
const maxDepth =
|
||||
typeof options.maxDepth === "number" &&
|
||||
Number.isFinite(options.maxDepth) &&
|
||||
options.maxDepth > 0
|
||||
? Math.floor(options.maxDepth)
|
||||
: undefined;
|
||||
|
||||
type InternalNode = {
|
||||
id: string;
|
||||
// Retained internally for shaping; never all emitted at once.
|
||||
slugId: any;
|
||||
title: any;
|
||||
children?: OutputNode[];
|
||||
hasServerChildren: boolean;
|
||||
children?: InternalNode[];
|
||||
};
|
||||
|
||||
// Map id -> output node. Build the lean output shape up front.
|
||||
const byId = new Map<string, OutputNode>();
|
||||
// Map id -> internal node. Build up front; the output shape is projected at
|
||||
// the very end so the maxDepth cut can consult `hasServerChildren`.
|
||||
const byId = new Map<string, InternalNode>();
|
||||
// Preserve the original position string for sorting (kept off the output).
|
||||
const positionById = new Map<string, string | undefined>();
|
||||
|
||||
@@ -49,6 +100,7 @@ export function buildPageTree(nodes: any[]): any[] {
|
||||
id: node.id,
|
||||
slugId: node.slugId,
|
||||
title: node.title,
|
||||
hasServerChildren: node.hasChildren === true,
|
||||
});
|
||||
positionById.set(node.id, node.position);
|
||||
}
|
||||
@@ -90,5 +142,30 @@ export function buildPageTree(nodes: any[]): any[] {
|
||||
}
|
||||
|
||||
roots.sort(byPosition);
|
||||
return roots.map((id) => byId.get(id)!);
|
||||
const rootNodes = roots.map((id) => byId.get(id)!);
|
||||
|
||||
// Project the internal nodes into the requested OUTPUT shape, applying the
|
||||
// maxDepth cut for the getTree shape. `depth` is 1-based (roots = depth 1).
|
||||
const project = (node: InternalNode, depth: number): any => {
|
||||
if (getTreeShape) {
|
||||
const out: any = { pageId: node.id, title: node.title };
|
||||
const atCut = maxDepth !== undefined && depth >= maxDepth;
|
||||
if (!atCut && node.children && node.children.length > 0) {
|
||||
out.children = node.children.map((c) => project(c, depth + 1));
|
||||
} else if (atCut && node.hasServerChildren) {
|
||||
// Children exist on the server but were trimmed by maxDepth: signal it
|
||||
// so the caller can descend with a follow-up rootPageId call.
|
||||
out.hasChildren = true;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
// Lean (historic) shape: cycle-safe, no depth cut, no hasChildren.
|
||||
const out: any = { id: node.id, slugId: node.slugId, title: node.title };
|
||||
if (node.children && node.children.length > 0) {
|
||||
out.children = node.children.map((c) => project(c, depth + 1));
|
||||
}
|
||||
return out;
|
||||
};
|
||||
|
||||
return rootNodes.map((n) => project(n, 1));
|
||||
}
|
||||
|
||||
@@ -40,11 +40,11 @@ import { SHARED_TOOL_SPECS, SharedToolSpec } from "./tool-specs.js";
|
||||
*/
|
||||
export const ROUTING_PROSE =
|
||||
"Docmost editing guide — choose the tool by intent. The <tool_inventory> at the end lists every tool with a one-line purpose; the notes below are the routing hints for WHEN to reach for each.\n" +
|
||||
"READ: find a page -> search (workspace-wide full-text); list -> list_pages / list_spaces. Locate blocks and their ids CHEAPLY -> get_outline (compact top-level map; start here, not get_page_json). One block's subtree -> get_node (by attrs.id, or \"#<index>\" for tables, which carry no id). Find every occurrence of a string/regex ON a page (and where each is) -> search_in_page, NOT block-by-block get_node — it returns each hit's node ref + block index + context for a targeted comment. Whole page -> get_page (Markdown, lossy; inline <span data-comment-id> tags are comment anchors — markup, not text) or get_page_json (lossless ProseMirror with block ids). Hand a huge page (with images) to an external consumer without pulling it through the model context -> stash_page (returns a short-lived anonymous URL).\n" +
|
||||
"EDIT: fix wording/typos/numbers -> edit_page_text (find/replace inside blocks, no node id needed). Change ONE block (paragraph/heading/callout/etc.) structurally -> patch_node (by attrs.id from get_outline). Add a block -> insert_node (before/after a block by attrs.id or by anchor text, or append). Remove a block -> delete_node (by attrs.id). Tables -> table_get / table_update_cell / table_insert_row / table_delete_row (address by \"#<index>\" from get_outline; table nodes have no attrs.id). Images -> insert_image (add from a web URL) / replace_image (swap an existing image). Draw.io diagrams -> drawio_create (create from mxGraph XML and insert), drawio_get (read a diagram as mxGraph XML + a hash), drawio_update (replace a diagram; pass the hash from drawio_get as baseHash for optimistic locking); before authoring a diagram, drawio_shapes (look up verified stencil style-strings so a shape name never renders as an empty box) and drawio_guide (on-demand authoring reference: skeleton/layout/containers/icons-aws/icons-azure), and pass layout:\"elk\" to drawio_create/drawio_update to auto-place nodes. Footnotes -> insert_footnote. Bulk/structural rewrite -> update_page_json (full ProseMirror replace) or update_page_markdown (full plain-Markdown body replace, re-imported — block ids regenerate); prefer the granular tools above to avoid resending the whole ~100KB+ document. Complex/scripted rewrite (multiple coordinated edits, renumbering) -> docmost_transform: write a JS `(doc, ctx) => doc` transform, preview the diff with dryRun (default), then apply with dryRun:false; ctx.helpers includes commentsToFootnotes for turning inline comments into numbered footnotes.\n" +
|
||||
"PAGES: new -> create_page (Markdown). Rename (title only) -> rename_page. Move -> move_page. Delete -> delete_page (SOFT delete — the page goes to trash and is restorable; nothing is permanent). Copy/replace a page's whole content from another page (server-side, no document through the model) -> copy_page_content. Sharing -> share_page / unshare_page / list_shares; share_page makes the page PUBLICLY accessible — do it only when explicitly asked.\n" +
|
||||
"COMMENTS: create_comment is always inline and requires an EXACT selection — contiguous text from a single block, <=250 chars (fails rather than leaving an unanchored comment); reply to a thread via parentCommentId. Propose a concrete text fix for one-click human approval -> create_comment with suggestedText (the exact plain-text replacement for the selection; the selection must then be UNIQUE in the page — extend it with context if needed); prefer this over editing directly when the change is subjective or needs the author's sign-off. Manage -> list_comments, update_comment, resolve_comment (resolve/reopen, reversible — prefer over delete to close), delete_comment, check_new_comments.\n" +
|
||||
"HISTORY: review what changed -> diff_page_versions (a historyId vs current, or two versions). List saved versions -> list_page_history. Undo a bad edit -> restore_page_version (writes a past version back as current; itself revertible). Export a page to self-contained Docmost Markdown (with comment anchors) -> export_page_markdown.";
|
||||
"READ: find a page by a fragment of a technical string (hostname/IP/ID like srv.local, 10.0.12, WB-MGE-30D86B) -> search — hybrid substring + full-text, returns each hit's location (path: root->parent titles) and a snippet around the match, so you rarely need a follow-up getPage; scope with spaceId or parentPageId (a subtree), titleOnly to match titles only. A space's page HIERARCHY (or one subtree) -> getTree (one request, complete, `{pageId,title,children?}`; rootPageId for a subtree, maxDepth to trim depth — a trimmed node gets hasChildren:true); prefer it over listPages tree:true (deprecated). Have a pageId, need WHERE-AM-I / what's around it (its breadcrumbs + direct children, metadata only) -> getPageContext (one call; parent = last breadcrumb, [] for a root page). list -> listPages / listSpaces. Locate blocks and their ids CHEAPLY -> getOutline (compact top-level map; start here, not getPageJson). One block, for editing -> getNode (by attrs.id, or \"#<index>\" for tables, which carry no id) — returns MARKDOWN by default (comment anchors kept for safe write-back); pass format:\"json\" for the raw ProseMirror subtree. Find every occurrence of a string/regex ON a page (and where each is) -> searchInPage, NOT block-by-block getNode — it returns each hit's node ref + block index + context for a targeted comment. Whole page -> getPage (Markdown, canonical for text; drops only block ids, resolved-comment anchors, and a fixed no-md-representation attr set: table spans/colwidth/bg, indent, callout.icon, orderedList.type, link internal/target/rel/class; inline <span data-comment-id> tags are comment anchors — markup, not text) or getPageJson (full ProseMirror with block ids, for those dropped attrs). Hand a huge page (with images) to an external consumer without pulling it through the model context -> stashPage (returns a short-lived anonymous URL).\n" +
|
||||
"EDIT: fix wording/typos/numbers -> editPageText (find/replace inside blocks, no node id needed). Edit a block -> getNode(markdown) -> edit the markdown -> patchNode(markdown) (by attrs.id from getOutline; the markdown fragment may be several blocks — a 1->N section rewrite in one call, the first block keeps the id). Reach for patchNode's `node`-JSON only for fine attr/mark work; a table cell with spans/colors/fixed width -> the table tools (patchNode markdown refuses it). Add a block -> insertNode (markdown, before/after a block by attrs.id or by anchor text, or append; `node` for raw JSON or bare table structure). Remove a block -> deleteNode (by attrs.id). Tables -> tableGet / tableUpdateCell / tableInsertRow / tableDeleteRow (address by \"#<index>\" from getOutline; table nodes have no attrs.id). Images -> insertImage (add from a web URL) / replaceImage (swap an existing image). Draw.io diagrams -> PREFER the high-level semantic tools that hide coordinates/styles: drawioFromGraph (architecture/cloud/network diagrams — describe nodes/groups/edges by kind+icon, the server picks layout, colors and verified icons; hints layer/sameLayerAs/pinned and layout:full|incremental|none) and drawioFromMermaid (standard flowcharts — write Mermaid, get an editable diagram). For targeted tweaks of an existing diagram use drawioEditCells (id-based add/update/delete with cascade delete + baseHash lock). Raw mxGraph XML via drawioCreate/drawioUpdate is the escape-hatch for exotic/wireframe diagrams; drawioGet reads a diagram as mxGraph XML + a hash (pass it as baseHash to drawioUpdate/drawioEditCells for optimistic locking). Before authoring raw XML, drawioShapes (look up verified stencil style-strings so a shape name never renders as an empty box) and drawioGuide (on-demand authoring reference: skeleton/layout/containers/icons-aws/icons-azure), and pass layout:\"elk\" to drawioCreate/drawioUpdate to auto-place nodes. Footnotes -> insertFootnote. Bulk/structural rewrite -> updatePageJson (full ProseMirror replace) or updatePageMarkdown (full plain-Markdown body replace, re-imported — block ids regenerate); prefer the granular tools above to avoid resending the whole ~100KB+ document. Complex/scripted rewrite (multiple coordinated edits, renumbering) -> docmostTransform: write a JS `(doc, ctx) => doc` transform, preview the diff with dryRun (default), then apply with dryRun:false; ctx.helpers includes commentsToFootnotes for turning inline comments into numbered footnotes.\n" +
|
||||
"PAGES: new -> createPage (Markdown). Rename (title only) -> renamePage. Move -> movePage. Delete -> deletePage (SOFT delete — the page goes to trash and is restorable; nothing is permanent). Copy/replace a page's whole content from another page (server-side, no document through the model) -> copyPageContent. Sharing -> sharePage / unsharePage / listShares; sharePage makes the page PUBLICLY accessible — do it only when explicitly asked.\n" +
|
||||
"COMMENTS: createComment is always inline and requires an EXACT selection — contiguous text from a single block, <=250 chars (fails rather than leaving an unanchored comment); reply to a thread via parentCommentId. Propose a concrete text fix for one-click human approval -> createComment with suggestedText (the exact plain-text replacement for the selection; the selection must then be UNIQUE in the page — extend it with context if needed); prefer this over editing directly when the change is subjective or needs the author's sign-off. Manage -> listComments, updateComment, resolveComment (resolve/reopen, reversible — prefer over delete to close), deleteComment, checkNewComments.\n" +
|
||||
"HISTORY: review what changed -> diffPageVersions (a historyId vs current, or two versions). List saved versions -> listPageHistory. Undo a bad edit -> restorePageVersion (writes a past version back as current; itself revertible). Export a page to self-contained Docmost Markdown (with comment anchors) -> exportPageMarkdown.";
|
||||
|
||||
/**
|
||||
* A single generated inventory line: the tool's registered NAME + a one-line
|
||||
@@ -81,57 +81,62 @@ type Family = (typeof FAMILY_ORDER)[number];
|
||||
const TOOL_FAMILY: Record<string, Family> = {
|
||||
// READ
|
||||
search: "READ",
|
||||
list_pages: "READ",
|
||||
list_spaces: "READ",
|
||||
get_outline: "READ",
|
||||
get_node: "READ",
|
||||
search_in_page: "READ",
|
||||
get_page: "READ",
|
||||
get_page_json: "READ",
|
||||
get_workspace: "READ",
|
||||
stash_page: "READ",
|
||||
listPages: "READ",
|
||||
getTree: "READ",
|
||||
getPageContext: "READ",
|
||||
listSpaces: "READ",
|
||||
getOutline: "READ",
|
||||
getNode: "READ",
|
||||
searchInPage: "READ",
|
||||
getPage: "READ",
|
||||
getPageJson: "READ",
|
||||
getWorkspace: "READ",
|
||||
stashPage: "READ",
|
||||
// EDIT
|
||||
edit_page_text: "EDIT",
|
||||
patch_node: "EDIT",
|
||||
insert_node: "EDIT",
|
||||
delete_node: "EDIT",
|
||||
update_page_json: "EDIT",
|
||||
update_page_markdown: "EDIT",
|
||||
table_get: "EDIT",
|
||||
table_update_cell: "EDIT",
|
||||
table_insert_row: "EDIT",
|
||||
table_delete_row: "EDIT",
|
||||
insert_image: "EDIT",
|
||||
replace_image: "EDIT",
|
||||
insert_footnote: "EDIT",
|
||||
drawio_get: "EDIT",
|
||||
drawio_create: "EDIT",
|
||||
drawio_update: "EDIT",
|
||||
drawio_shapes: "EDIT",
|
||||
drawio_guide: "EDIT",
|
||||
docmost_transform: "EDIT",
|
||||
editPageText: "EDIT",
|
||||
patchNode: "EDIT",
|
||||
insertNode: "EDIT",
|
||||
deleteNode: "EDIT",
|
||||
updatePageJson: "EDIT",
|
||||
updatePageMarkdown: "EDIT",
|
||||
tableGet: "EDIT",
|
||||
tableUpdateCell: "EDIT",
|
||||
tableInsertRow: "EDIT",
|
||||
tableDeleteRow: "EDIT",
|
||||
insertImage: "EDIT",
|
||||
replaceImage: "EDIT",
|
||||
insertFootnote: "EDIT",
|
||||
drawioGet: "EDIT",
|
||||
drawioCreate: "EDIT",
|
||||
drawioUpdate: "EDIT",
|
||||
drawioEditCells: "EDIT",
|
||||
drawioFromGraph: "EDIT",
|
||||
drawioFromMermaid: "EDIT",
|
||||
drawioShapes: "EDIT",
|
||||
drawioGuide: "EDIT",
|
||||
docmostTransform: "EDIT",
|
||||
// PAGES
|
||||
create_page: "PAGES",
|
||||
rename_page: "PAGES",
|
||||
move_page: "PAGES",
|
||||
delete_page: "PAGES",
|
||||
copy_page_content: "PAGES",
|
||||
share_page: "PAGES",
|
||||
unshare_page: "PAGES",
|
||||
list_shares: "PAGES",
|
||||
createPage: "PAGES",
|
||||
renamePage: "PAGES",
|
||||
movePage: "PAGES",
|
||||
deletePage: "PAGES",
|
||||
copyPageContent: "PAGES",
|
||||
sharePage: "PAGES",
|
||||
unsharePage: "PAGES",
|
||||
listShares: "PAGES",
|
||||
// COMMENTS
|
||||
create_comment: "COMMENTS",
|
||||
list_comments: "COMMENTS",
|
||||
update_comment: "COMMENTS",
|
||||
resolve_comment: "COMMENTS",
|
||||
delete_comment: "COMMENTS",
|
||||
check_new_comments: "COMMENTS",
|
||||
createComment: "COMMENTS",
|
||||
listComments: "COMMENTS",
|
||||
updateComment: "COMMENTS",
|
||||
resolveComment: "COMMENTS",
|
||||
deleteComment: "COMMENTS",
|
||||
checkNewComments: "COMMENTS",
|
||||
// HISTORY
|
||||
diff_page_versions: "HISTORY",
|
||||
list_page_history: "HISTORY",
|
||||
restore_page_version: "HISTORY",
|
||||
export_page_markdown: "HISTORY",
|
||||
// import_page_markdown is now inAppOnly (#411) — it is not registered on the
|
||||
diffPageVersions: "HISTORY",
|
||||
listPageHistory: "HISTORY",
|
||||
restorePageVersion: "HISTORY",
|
||||
exportPageMarkdown: "HISTORY",
|
||||
// importPageMarkdown is now inAppOnly (#411) — it is not registered on the
|
||||
// external MCP host, so it no longer appears in the generated inventory.
|
||||
};
|
||||
|
||||
@@ -145,26 +150,26 @@ const TOOL_FAMILY: Record<string, Family> = {
|
||||
*/
|
||||
export const INLINE_MCP_INVENTORY: ToolInventoryLine[] = [
|
||||
{
|
||||
name: "table_get",
|
||||
name: "tableGet",
|
||||
purpose:
|
||||
"read a table as a matrix of cell texts + per-cell paragraph ids.",
|
||||
},
|
||||
{
|
||||
name: "search",
|
||||
purpose:
|
||||
"full-text search for pages and content across the whole workspace.",
|
||||
"find pages by a fragment of a technical string (hybrid substring + full-text); returns each hit's path and a snippet.",
|
||||
},
|
||||
{
|
||||
name: "docmost_transform",
|
||||
name: "docmostTransform",
|
||||
purpose:
|
||||
"edit a page by running a sandboxed JS `(doc, ctx) => doc` transform, with a dryRun diff preview.",
|
||||
},
|
||||
{
|
||||
name: "update_comment",
|
||||
name: "updateComment",
|
||||
purpose: "update an existing comment's content (creator only).",
|
||||
},
|
||||
{
|
||||
name: "delete_comment",
|
||||
name: "deleteComment",
|
||||
purpose: "delete a comment (creator or space admin only).",
|
||||
},
|
||||
];
|
||||
|
||||
+541
-159
File diff suppressed because it is too large
Load Diff
+76
-76
@@ -84,20 +84,20 @@ async function main() {
|
||||
let pageId = null;
|
||||
|
||||
try {
|
||||
// 1. create_page: title with spaces must survive (was: underscores bug)
|
||||
// 1. createPage: title with spaces must survive (was: underscores bug)
|
||||
const created = await client.createPage("Тест апгрейда MCP сервера", MD, spaceId);
|
||||
pageId = created.data.id;
|
||||
check("create_page: title keeps spaces", created.data.title === "Тест апгрейда MCP сервера", created.data.title);
|
||||
check("create_page: slugId exposed", typeof created.data.slugId === "string" && created.data.slugId.length > 0, created.data.slugId);
|
||||
check("createPage: title keeps spaces", created.data.title === "Тест апгрейда MCP сервера", created.data.title);
|
||||
check("createPage: slugId exposed", typeof created.data.slugId === "string" && created.data.slugId.length > 0, created.data.slugId);
|
||||
|
||||
// 2. get_page_json: raw ProseMirror with callout + table
|
||||
// 2. getPageJson: raw ProseMirror with callout + table
|
||||
const pj = await client.getPageJson(pageId);
|
||||
const types = pj.content.content.map((n) => n.type);
|
||||
check("get_page_json: callout node present", types.includes("callout"), types.join(","));
|
||||
check("get_page_json: table node present", types.includes("table"));
|
||||
check("get_page_json: slugId present", !!pj.slugId);
|
||||
check("getPageJson: callout node present", types.includes("callout"), types.join(","));
|
||||
check("getPageJson: table node present", types.includes("table"));
|
||||
check("getPageJson: slugId present", !!pj.slugId);
|
||||
|
||||
// 3. edit_page_text: surgical replace, ids preserved
|
||||
// 3. editPageText: surgical replace, ids preserved
|
||||
const idsBefore = JSON.stringify(
|
||||
pj.content.content.filter((n) => n.attrs?.id).map((n) => n.attrs.id),
|
||||
);
|
||||
@@ -105,26 +105,26 @@ async function main() {
|
||||
{ find: "БУКВОЕД", replace: "КНИГОЛЮБ" },
|
||||
{ find: "[1]", replace: "[42]" },
|
||||
]);
|
||||
check("edit_page_text: both edits applied", editRes.applied.every((e) => e.replacements === 1));
|
||||
check("editPageText: both edits applied", editRes.applied.every((e) => e.replacements === 1));
|
||||
await new Promise((r) => setTimeout(r, 16000)); // wait for server persistence
|
||||
const pj2 = await client.getPageJson(pageId);
|
||||
const text2 = JSON.stringify(pj2.content);
|
||||
check("edit_page_text: replacement visible", text2.includes("КНИГОЛЮБ") && text2.includes("[42]"));
|
||||
check("edit_page_text: old text gone", !text2.includes("БУКВОЕД"));
|
||||
check("editPageText: replacement visible", text2.includes("КНИГОЛЮБ") && text2.includes("[42]"));
|
||||
check("editPageText: old text gone", !text2.includes("БУКВОЕД"));
|
||||
const idsAfter = JSON.stringify(
|
||||
pj2.content.content.filter((n) => n.attrs?.id).map((n) => n.attrs.id),
|
||||
);
|
||||
check("edit_page_text: block ids preserved", idsBefore === idsAfter);
|
||||
check("edit_page_text: callout survived", JSON.stringify(pj2.content).includes('"callout"'));
|
||||
check("edit_page_text: table survived", pj2.content.content.some((n) => n.type === "table"));
|
||||
check("editPageText: block ids preserved", idsBefore === idsAfter);
|
||||
check("editPageText: callout survived", JSON.stringify(pj2.content).includes('"callout"'));
|
||||
check("editPageText: table survived", pj2.content.content.some((n) => n.type === "table"));
|
||||
|
||||
// 4. error reporting: ambiguous and missing finds
|
||||
let err1 = "";
|
||||
try { await client.editPageText(pageId, [{ find: "Колонка", replace: "X" }]); } catch (e) { err1 = e.message; }
|
||||
check("edit_page_text: ambiguous match rejected", err1.includes("matches"), err1);
|
||||
check("editPageText: ambiguous match rejected", err1.includes("matches"), err1);
|
||||
let err2 = "";
|
||||
try { await client.editPageText(pageId, [{ find: "НЕСУЩЕСТВУЮЩЕЕ", replace: "X" }]); } catch (e) { err2 = e.message; }
|
||||
check("edit_page_text: missing text reported", err2.includes("not found"), err2);
|
||||
check("editPageText: missing text reported", err2.includes("not found"), err2);
|
||||
|
||||
// 5. update_page (markdown): table + callout must survive the re-import
|
||||
await client.updatePage(pageId, MD + "\nДобавленный абзац.\n");
|
||||
@@ -137,21 +137,21 @@ async function main() {
|
||||
const cellText = JSON.stringify(tableNode);
|
||||
check("update_page md: table cells intact", cellText.includes("четыре") && cellText.includes("Колонка А"));
|
||||
|
||||
// 6. update_page_json: lossless write round-trip
|
||||
// 6. updatePageJson: lossless write round-trip
|
||||
pj3.content.content.push({
|
||||
type: "paragraph",
|
||||
attrs: { id: "testidjsonpush", indent: 0, textAlign: null },
|
||||
content: [{ type: "text", text: "Абзац, добавленный через update_page_json." }],
|
||||
content: [{ type: "text", text: "Абзац, добавленный через updatePageJson." }],
|
||||
});
|
||||
await client.updatePageJson(pageId, pj3.content);
|
||||
await new Promise((r) => setTimeout(r, 16000));
|
||||
const pj4 = await client.getPageJson(pageId);
|
||||
const lastNode = pj4.content.content[pj4.content.content.length - 1];
|
||||
check("update_page_json: paragraph appended", JSON.stringify(pj4.content).includes("добавленный через update_page_json"));
|
||||
check("update_page_json: custom node id preserved", lastNode.attrs?.id === "testidjsonpush", lastNode.attrs?.id);
|
||||
check("updatePageJson: paragraph appended", JSON.stringify(pj4.content).includes("добавленный через updatePageJson"));
|
||||
check("updatePageJson: custom node id preserved", lastNode.attrs?.id === "testidjsonpush", lastNode.attrs?.id);
|
||||
|
||||
// 6b. images: upload / insert / replace (clean src, fresh attachment on replace).
|
||||
// insert_image / replace_image take an http(s) URL that the SERVER fetches;
|
||||
// insertImage / replaceImage take an http(s) URL that the SERVER fetches;
|
||||
// local file paths are intentionally unsupported. The Docmost server runs on
|
||||
// the same host as this test, so serve the PNG bytes over a throwaway
|
||||
// localhost HTTP server it can reach.
|
||||
@@ -186,13 +186,13 @@ async function main() {
|
||||
validateStatus: () => true,
|
||||
});
|
||||
|
||||
// insert_image: append the first PNG, src must be clean (no ?v=) and fetchable.
|
||||
// insertImage: append the first PNG, src must be clean (no ?v=) and fetchable.
|
||||
const ins = await client.insertImage(pageId, urlA);
|
||||
check("insert_image: src has no ?v= cache-buster", !ins.src.includes("?v="), ins.src);
|
||||
check("insertImage: src has no ?v= cache-buster", !ins.src.includes("?v="), ins.src);
|
||||
const fileA = await fetchFile(ins.src);
|
||||
check("insert_image: file fetch returns 200", fileA.status === 200, `status=${fileA.status}`);
|
||||
check("insertImage: file fetch returns 200", fileA.status === 200, `status=${fileA.status}`);
|
||||
check(
|
||||
"insert_image: content-type is image/*",
|
||||
"insertImage: content-type is image/*",
|
||||
String(fileA.headers["content-type"] || "").startsWith("image/"),
|
||||
String(fileA.headers["content-type"]),
|
||||
);
|
||||
@@ -209,25 +209,25 @@ async function main() {
|
||||
};
|
||||
const imgNode = findImage(pjImg.content.content);
|
||||
const oldAttachmentId = imgNode?.attrs?.attachmentId;
|
||||
check("insert_image: image node present after persist", !!oldAttachmentId, oldAttachmentId);
|
||||
check("insertImage: image node present after persist", !!oldAttachmentId, oldAttachmentId);
|
||||
|
||||
// replace_image: must create a NEW attachment with a clean, fetchable URL.
|
||||
// replaceImage: must create a NEW attachment with a clean, fetchable URL.
|
||||
// The 200 fetch is the assertion that catches the in-place-overwrite HTTP 500 regression.
|
||||
const rep = await client.replaceImage(pageId, oldAttachmentId, urlB);
|
||||
check("replace_image: new attachment id differs from old", rep.newAttachmentId !== oldAttachmentId, `${oldAttachmentId} -> ${rep.newAttachmentId}`);
|
||||
check("replace_image: src has no ?v= cache-buster", !rep.src.includes("?v="), rep.src);
|
||||
check("replaceImage: new attachment id differs from old", rep.newAttachmentId !== oldAttachmentId, `${oldAttachmentId} -> ${rep.newAttachmentId}`);
|
||||
check("replaceImage: src has no ?v= cache-buster", !rep.src.includes("?v="), rep.src);
|
||||
const fileB = await fetchFile(rep.src);
|
||||
check("replace_image: new file fetch returns 200", fileB.status === 200, `status=${fileB.status}`);
|
||||
check("replaceImage: new file fetch returns 200", fileB.status === 200, `status=${fileB.status}`);
|
||||
check(
|
||||
"replace_image: new content-type is image/*",
|
||||
"replaceImage: new content-type is image/*",
|
||||
String(fileB.headers["content-type"] || "").startsWith("image/"),
|
||||
String(fileB.headers["content-type"]),
|
||||
);
|
||||
|
||||
await new Promise((r) => setTimeout(r, 16000));
|
||||
const pjImg2 = await client.getPageJson(pageId);
|
||||
check("replace_image: page has new attachment id", !!findImage(pjImg2.content.content, rep.newAttachmentId), rep.newAttachmentId);
|
||||
check("replace_image: old attachment id repointed away", !findImage(pjImg2.content.content, oldAttachmentId), oldAttachmentId);
|
||||
check("replaceImage: page has new attachment id", !!findImage(pjImg2.content.content, rep.newAttachmentId), rep.newAttachmentId);
|
||||
check("replaceImage: old attachment id repointed away", !findImage(pjImg2.content.content, oldAttachmentId), oldAttachmentId);
|
||||
} finally {
|
||||
imgServer.close();
|
||||
}
|
||||
@@ -275,10 +275,10 @@ async function main() {
|
||||
await client.editPageText(fid, [{ find: "PRICEMARK", replace: "$& costs $100" }]);
|
||||
await new Promise((r) => setTimeout(r, 16000));
|
||||
const ftext = JSON.stringify((await client.getPageJson(fid)).content);
|
||||
check("feature: edit_page_text inserts $-pattern literally (no $& expansion)", ftext.includes("$& costs $100") && !ftext.includes("PRICEMARK costs"));
|
||||
check("feature: editPageText inserts $-pattern literally (no $& expansion)", ftext.includes("$& costs $100") && !ftext.includes("PRICEMARK costs"));
|
||||
let badThrew = false;
|
||||
try { await client.replaceImage(fid, "00000000-0000-0000-0000-000000000000", featPng); } catch (e) { badThrew = /no image with attachmentId/.test(e.message); }
|
||||
check("feature: replace_image with unknown id throws (no orphan upload)", badThrew);
|
||||
check("feature: replaceImage with unknown id throws (no orphan upload)", badThrew);
|
||||
} finally {
|
||||
try { await client.deletePage(fid); } catch {}
|
||||
try { unlinkSync(featPng); } catch {}
|
||||
@@ -286,7 +286,7 @@ async function main() {
|
||||
}
|
||||
|
||||
// 6d. node ops: patch / insert / delete a block by id on a throwaway page.
|
||||
// Three paragraphs are written with KNOWN ids via update_page_json so the
|
||||
// Three paragraphs are written with KNOWN ids via updatePageJson so the
|
||||
// ids can be targeted directly; each op is verified via getPageJson after
|
||||
// the standard 16s persistence wait.
|
||||
{
|
||||
@@ -348,7 +348,7 @@ async function main() {
|
||||
}
|
||||
}
|
||||
|
||||
// 6e. rename_page: title-only update must leave the content untouched.
|
||||
// 6e. renamePage: title-only update must leave the content untouched.
|
||||
{
|
||||
const rp = await client.createPage("E2E rename before " + Date.now(), "Rename body marker RENAMEBODY.", spaceId);
|
||||
const rid = rp.data.id;
|
||||
@@ -357,19 +357,19 @@ async function main() {
|
||||
const beforeContent = JSON.stringify(beforeJson);
|
||||
const newTitle = "E2E rename AFTER " + Date.now();
|
||||
const rr = await client.renamePage(rid, newTitle);
|
||||
check("rename_page: returns success+title", rr.success === true && rr.title === newTitle, JSON.stringify(rr));
|
||||
check("renamePage: returns success+title", rr.success === true && rr.title === newTitle, JSON.stringify(rr));
|
||||
await new Promise((r) => setTimeout(r, 16000));
|
||||
const afterJson = await client.getPageJson(rid);
|
||||
check("rename_page: title changed", afterJson.title === newTitle, afterJson.title);
|
||||
check("rename_page: content unchanged", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("RENAMEBODY"));
|
||||
check("renamePage: title changed", afterJson.title === newTitle, afterJson.title);
|
||||
check("renamePage: content unchanged", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("RENAMEBODY"));
|
||||
const afterMd = (await client.getPage(rid)).data;
|
||||
check("rename_page: get_page reflects new title", afterMd.title === newTitle, afterMd.title);
|
||||
check("renamePage: getPage reflects new title", afterMd.title === newTitle, afterMd.title);
|
||||
} finally {
|
||||
try { await client.deletePage(rid); } catch {}
|
||||
}
|
||||
}
|
||||
|
||||
// 6f. update_page_json title-only: omitting content updates the title and
|
||||
// 6f. updatePageJson title-only: omitting content updates the title and
|
||||
// leaves the body intact; supplying neither content nor title throws.
|
||||
{
|
||||
const up = await client.createPage("E2E upj-title before " + Date.now(), "Title-only body marker UPJTITLEBODY.", spaceId);
|
||||
@@ -378,20 +378,20 @@ async function main() {
|
||||
const beforeContent = JSON.stringify((await client.getPageJson(uid)).content);
|
||||
const newTitle = "E2E upj-title AFTER " + Date.now();
|
||||
const ur = await client.updatePageJson(uid, undefined, newTitle);
|
||||
check("update_page_json title-only: succeeds", ur.success === true, JSON.stringify(ur));
|
||||
check("updatePageJson title-only: succeeds", ur.success === true, JSON.stringify(ur));
|
||||
await new Promise((r) => setTimeout(r, 16000));
|
||||
const afterJson = await client.getPageJson(uid);
|
||||
check("update_page_json title-only: title updated", afterJson.title === newTitle, afterJson.title);
|
||||
check("update_page_json title-only: content intact", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("UPJTITLEBODY"));
|
||||
check("updatePageJson title-only: title updated", afterJson.title === newTitle, afterJson.title);
|
||||
check("updatePageJson title-only: content intact", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("UPJTITLEBODY"));
|
||||
let upjErr = "";
|
||||
try { await client.updatePageJson(uid); } catch (e) { upjErr = e.message; }
|
||||
check("update_page_json: neither content nor title throws", upjErr.includes("nothing to update"), upjErr);
|
||||
check("updatePageJson: neither content nor title throws", upjErr.includes("nothing to update"), upjErr);
|
||||
} finally {
|
||||
try { await client.deletePage(uid); } catch {}
|
||||
}
|
||||
}
|
||||
|
||||
// 6g. copy_page_content: B's body becomes a copy of A's body, server-side,
|
||||
// 6g. copyPageContent: B's body becomes a copy of A's body, server-side,
|
||||
// while B's title/slugId stay put. Both pages are throwaways.
|
||||
{
|
||||
let aid = null;
|
||||
@@ -409,24 +409,24 @@ async function main() {
|
||||
const aNodeCount = aJson.content.content.length;
|
||||
|
||||
const cr = await client.copyPageContent(aid, bid);
|
||||
check("copy_page_content: returns success + node count", cr.success === true && cr.copiedNodes === aNodeCount, JSON.stringify(cr));
|
||||
check("copyPageContent: returns success + node count", cr.success === true && cr.copiedNodes === aNodeCount, JSON.stringify(cr));
|
||||
await new Promise((r) => setTimeout(r, 16000));
|
||||
|
||||
const bAfter = await client.getPageJson(bid);
|
||||
const bText = JSON.stringify(bAfter.content);
|
||||
check("copy_page_content: B now has A's marker", bText.includes("COPYSOURCE"));
|
||||
check("copy_page_content: B's old marker gone", !bText.includes("COPYTARGET"));
|
||||
check("copy_page_content: B node count equals A's", bAfter.content.content.length === aNodeCount, `${bAfter.content.content.length} vs ${aNodeCount}`);
|
||||
check("copy_page_content: B title unchanged", bAfter.title === bTitleBefore, bAfter.title);
|
||||
check("copy_page_content: B slugId unchanged", bAfter.slugId === bSlugBefore, bAfter.slugId);
|
||||
check("copyPageContent: B now has A's marker", bText.includes("COPYSOURCE"));
|
||||
check("copyPageContent: B's old marker gone", !bText.includes("COPYTARGET"));
|
||||
check("copyPageContent: B node count equals A's", bAfter.content.content.length === aNodeCount, `${bAfter.content.content.length} vs ${aNodeCount}`);
|
||||
check("copyPageContent: B title unchanged", bAfter.title === bTitleBefore, bAfter.title);
|
||||
check("copyPageContent: B slugId unchanged", bAfter.slugId === bSlugBefore, bAfter.slugId);
|
||||
|
||||
// Source must be left untouched by the copy.
|
||||
const aAfter = JSON.stringify((await client.getPageJson(aid)).content);
|
||||
check("copy_page_content: source page unchanged", aAfter === JSON.stringify(aJson.content) && aAfter.includes("COPYSOURCE"));
|
||||
check("copyPageContent: source page unchanged", aAfter === JSON.stringify(aJson.content) && aAfter.includes("COPYSOURCE"));
|
||||
|
||||
let copyErr = "";
|
||||
try { await client.copyPageContent(aid, aid); } catch (e) { copyErr = e.message; }
|
||||
check("copy_page_content: self-copy rejected", copyErr.includes("same page"), copyErr);
|
||||
check("copyPageContent: self-copy rejected", copyErr.includes("same page"), copyErr);
|
||||
} finally {
|
||||
try { if (bid) await client.deletePage(bid); } catch {}
|
||||
try { if (aid) await client.deletePage(aid); } catch {}
|
||||
@@ -435,22 +435,22 @@ async function main() {
|
||||
|
||||
// 7. shares: create (idempotent), public access, list, unshare
|
||||
const share = await client.sharePage(pageId);
|
||||
check("share_page: returns public URL", share.publicUrl?.startsWith(`${APP}/share/`), share.publicUrl);
|
||||
check("sharePage: returns public URL", share.publicUrl?.startsWith(`${APP}/share/`), share.publicUrl);
|
||||
const share2 = await client.sharePage(pageId);
|
||||
check("share_page: idempotent", share2.key === share.key);
|
||||
check("sharePage: idempotent", share2.key === share.key);
|
||||
const anon = await axios.post(`${API}/shares/page-info`, { pageId: pj4.slugId, shareId: share.key }, { validateStatus: () => true });
|
||||
check("share_page: anonymous access works", anon.status === 200);
|
||||
check("sharePage: anonymous access works", anon.status === 200);
|
||||
const shares = await client.listShares();
|
||||
check("list_shares: contains our page", shares.some((s) => s.pageId === pageId && s.publicUrl === share.publicUrl));
|
||||
check("listShares: contains our page", shares.some((s) => s.pageId === pageId && s.publicUrl === share.publicUrl));
|
||||
const un = await client.unsharePage(pageId);
|
||||
check("unshare_page: success", un.success === true);
|
||||
check("unsharePage: success", un.success === true);
|
||||
const anon2 = await axios.post(`${API}/shares/page-info`, { pageId: pj4.slugId, shareId: share.key }, { validateStatus: () => true });
|
||||
check("unshare_page: public access revoked", anon2.status !== 200, `status=${anon2.status}`);
|
||||
check("unsharePage: public access revoked", anon2.status !== 200, `status=${anon2.status}`);
|
||||
|
||||
// 8. get_page markdown round-trip sanity (table separator present)
|
||||
// 8. getPage markdown round-trip sanity (table separator present)
|
||||
const md = await client.getPage(pageId);
|
||||
check("get_page md: table separator emitted", md.data.content.includes("| --- |"), "");
|
||||
check("get_page md: callout exported as Obsidian '> [!info]'", md.data.content.includes("> [!info]"));
|
||||
check("getPage md: table separator emitted", md.data.content.includes("| --- |"), "");
|
||||
check("getPage md: callout exported as Obsidian '> [!info]'", md.data.content.includes("> [!info]"));
|
||||
|
||||
// 9. comments: create / list / reply / update / check_new / delete
|
||||
const beforeComments = new Date(Date.now() - 1000).toISOString();
|
||||
@@ -458,34 +458,34 @@ async function main() {
|
||||
// that exists in the persisted page to anchor on. "Добавленный абзац." is a
|
||||
// plain paragraph re-imported in section 5 and still present here.
|
||||
const c1 = await client.createComment(pageId, "Первый **комментарий** с [ссылкой](https://example.com).", "inline", "Добавленный абзац.");
|
||||
check("create_comment: created", !!c1.data.id, c1.data.id);
|
||||
check("create_comment: markdown round-trip", c1.data.content.includes("**комментарий**"), c1.data.content);
|
||||
check("createComment: created", !!c1.data.id, c1.data.id);
|
||||
check("createComment: markdown round-trip", c1.data.content.includes("**комментарий**"), c1.data.content);
|
||||
const reply = await client.createComment(pageId, "Ответ на комментарий.", "page", undefined, c1.data.id);
|
||||
check("create_comment: reply has parent", reply.data.parentCommentId === c1.data.id);
|
||||
check("createComment: reply has parent", reply.data.parentCommentId === c1.data.id);
|
||||
const list = (await client.listComments(pageId)).items;
|
||||
check("list_comments: both visible", list.length === 2, `count=${list.length}`);
|
||||
check("listComments: both visible", list.length === 2, `count=${list.length}`);
|
||||
await client.updateComment(c1.data.id, "Обновлённый текст комментария.");
|
||||
const got = await client.getComment(c1.data.id);
|
||||
check("update_comment + get_comment: content updated", got.data.content.includes("Обновлённый"), got.data.content);
|
||||
check("updateComment + get_comment: content updated", got.data.content.includes("Обновлённый"), got.data.content);
|
||||
const news = await client.checkNewComments(spaceId, beforeComments, pageId);
|
||||
check("check_new_comments: finds new comments in subtree", news.totalNewComments >= 2, `total=${news.totalNewComments}`);
|
||||
// resolve_comment: close the top-level thread, verify resolvedAt surfaces, then reopen
|
||||
check("checkNewComments: finds new comments in subtree", news.totalNewComments >= 2, `total=${news.totalNewComments}`);
|
||||
// resolveComment: close the top-level thread, verify resolvedAt surfaces, then reopen
|
||||
const resolvedRes = await client.resolveComment(c1.data.id, true);
|
||||
check("resolve_comment: marks resolved", resolvedRes.success === true && resolvedRes.resolved === true);
|
||||
check("resolveComment: marks resolved", resolvedRes.success === true && resolvedRes.resolved === true);
|
||||
// c1 is now resolved; the default feed hides resolved threads, so pass
|
||||
// includeResolved:true to still see it and assert its resolvedAt (#328).
|
||||
const listResolved = (await client.listComments(pageId, true)).items;
|
||||
const c1Resolved = listResolved.find((c) => c.id === c1.data.id);
|
||||
check("resolve_comment: resolvedAt set in list", !!c1Resolved?.resolvedAt, `resolvedAt=${c1Resolved?.resolvedAt}`);
|
||||
check("resolveComment: resolvedAt set in list", !!c1Resolved?.resolvedAt, `resolvedAt=${c1Resolved?.resolvedAt}`);
|
||||
const reopenedRes = await client.resolveComment(c1.data.id, false);
|
||||
check("resolve_comment: reopen succeeds", reopenedRes.resolved === false);
|
||||
check("resolveComment: reopen succeeds", reopenedRes.resolved === false);
|
||||
const listReopened = (await client.listComments(pageId)).items;
|
||||
const c1Reopened = listReopened.find((c) => c.id === c1.data.id);
|
||||
check("resolve_comment: resolvedAt cleared on reopen", !c1Reopened?.resolvedAt, `resolvedAt=${c1Reopened?.resolvedAt}`);
|
||||
check("resolveComment: resolvedAt cleared on reopen", !c1Reopened?.resolvedAt, `resolvedAt=${c1Reopened?.resolvedAt}`);
|
||||
await client.deleteComment(reply.data.id);
|
||||
await client.deleteComment(c1.data.id);
|
||||
const listAfter = (await client.listComments(pageId)).items;
|
||||
check("delete_comment: comments removed", listAfter.length === 0, `count=${listAfter.length}`);
|
||||
check("deleteComment: comments removed", listAfter.length === 0, `count=${listAfter.length}`);
|
||||
} finally {
|
||||
if (pageId) {
|
||||
await client.deletePage(pageId);
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
// Mock collab regression for the AMBIGUOUS-id refusal in patch_node / delete_node
|
||||
// Mock collab regression for the AMBIGUOUS-id refusal in patchNode / deleteNode
|
||||
// (#159, PR #185 review pt 1). When a page has TWO blocks sharing one attrs.id
|
||||
// (Docmost duplicates block ids on copy/paste), the transform's
|
||||
// `if (replaced !== 1) return null` / `if (deleted !== 1) return null` guard must
|
||||
@@ -126,18 +126,20 @@ after(async () => {
|
||||
);
|
||||
});
|
||||
|
||||
test("patch_node REFUSES an ambiguous (duplicate) id without writing to collab", async () => {
|
||||
test("patchNode REFUSES an ambiguous (duplicate) id without writing to collab", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
await assert.rejects(
|
||||
() =>
|
||||
client.patchNode("11111111-1111-4111-8111-111111111111", DUP_ID, {
|
||||
type: "paragraph",
|
||||
content: [{ type: "text", text: "replacement" }],
|
||||
node: {
|
||||
type: "paragraph",
|
||||
content: [{ type: "text", text: "replacement" }],
|
||||
},
|
||||
}),
|
||||
/ambiguous/i,
|
||||
"patch_node must reject a duplicate-id target with an 'ambiguous' error",
|
||||
"patchNode must reject a duplicate-id target with an 'ambiguous' error",
|
||||
);
|
||||
|
||||
assert.equal(
|
||||
@@ -147,14 +149,14 @@ test("patch_node REFUSES an ambiguous (duplicate) id without writing to collab",
|
||||
);
|
||||
});
|
||||
|
||||
test("delete_node REFUSES an ambiguous (duplicate) id without writing to collab", async () => {
|
||||
test("deleteNode REFUSES an ambiguous (duplicate) id without writing to collab", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
await assert.rejects(
|
||||
() => client.deleteNode("22222222-2222-4222-8222-222222222222", DUP_ID),
|
||||
/ambiguous/i,
|
||||
"delete_node must reject a duplicate-id target with an 'ambiguous' error",
|
||||
"deleteNode must reject a duplicate-id target with an 'ambiguous' error",
|
||||
);
|
||||
|
||||
assert.equal(
|
||||
|
||||
@@ -0,0 +1,272 @@
|
||||
// Contract tests for the stage-3 drawio client methods (issue #425):
|
||||
// drawioEditCells / drawioFromGraph / drawioFromMermaid. Same seam-override
|
||||
// pattern as drawio-tools.test.mjs: a DocmostClient subclass stubs the I/O seams
|
||||
// so the tool logic runs without a live Docmost / collab socket.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { DocmostClient } from "../../build/client.js";
|
||||
import {
|
||||
buildDrawioSvg,
|
||||
normalizeXml,
|
||||
mxHash,
|
||||
decodeDrawioSvg,
|
||||
parseCells,
|
||||
} from "../../build/lib/drawio-xml.js";
|
||||
|
||||
const DRAWIO_SCHEMA_ATTRS = new Set([
|
||||
"src", "title", "alt", "width", "height", "size", "aspectRatio", "align", "attachmentId",
|
||||
]);
|
||||
function applyDrawioSchemaDrop(node) {
|
||||
if (!node || typeof node !== "object") return;
|
||||
if (node.type === "drawio" && node.attrs && typeof node.attrs === "object") {
|
||||
for (const key of Object.keys(node.attrs))
|
||||
if (!DRAWIO_SCHEMA_ATTRS.has(key)) delete node.attrs[key];
|
||||
}
|
||||
if (Array.isArray(node.content)) for (const c of node.content) applyDrawioSchemaDrop(c);
|
||||
}
|
||||
|
||||
function svgFor(model, bbox = { width: 400, height: 300 }) {
|
||||
return buildDrawioSvg(normalizeXml(model), "<g/>", bbox);
|
||||
}
|
||||
|
||||
function makeClient({ pageDoc, attachmentSvg } = {}) {
|
||||
const calls = { uploads: [], mutations: [] };
|
||||
class TestClient extends DocmostClient {
|
||||
async ensureAuthenticated() {}
|
||||
async getCollabTokenWithReauth() {
|
||||
return "collab-token";
|
||||
}
|
||||
async resolvePageId(pageId) {
|
||||
return `uuid-${pageId}`;
|
||||
}
|
||||
async getPageRaw(pageId) {
|
||||
return {
|
||||
id: pageId, slugId: "s", title: "P", spaceId: "sp",
|
||||
content: pageDoc ?? { type: "doc", content: [] },
|
||||
};
|
||||
}
|
||||
async uploadAttachmentBuffer(pageId, buffer, fileName) {
|
||||
const id = `att-${calls.uploads.length + 1}`;
|
||||
calls.uploads.push({ pageId, fileName, svg: buffer.toString("utf-8") });
|
||||
return { id, fileName, fileSize: buffer.length };
|
||||
}
|
||||
async fetchAttachmentText() {
|
||||
return attachmentSvg;
|
||||
}
|
||||
mutatePage(pageId, token, apiUrl, transform) {
|
||||
const clone = structuredClone(pageDoc ?? { type: "doc", content: [] });
|
||||
const doc = transform(clone);
|
||||
if (doc) applyDrawioSchemaDrop(doc);
|
||||
calls.mutations.push({ pageId, doc });
|
||||
return Promise.resolve({ doc, verify: { changed: doc != null } });
|
||||
}
|
||||
}
|
||||
const client = new TestClient("http://127.0.0.1:1/api", "e@x.com", "pw");
|
||||
return { client, calls };
|
||||
}
|
||||
|
||||
function findDrawio(node, acc = []) {
|
||||
if (!node || typeof node !== "object") return acc;
|
||||
if (node.type === "drawio") acc.push(node);
|
||||
if (Array.isArray(node.content)) for (const c of node.content) findDrawio(c, acc);
|
||||
return acc;
|
||||
}
|
||||
|
||||
// A stored diagram: a group with two children and an edge.
|
||||
const STORED =
|
||||
"<mxGraphModel><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/>" +
|
||||
'<mxCell id="grp" value="G" style="container=1;fillColor=none;" vertex="1" parent="1">' +
|
||||
'<mxGeometry x="0" y="0" width="300" height="200" as="geometry"/></mxCell>' +
|
||||
'<mxCell id="a" value="A" style="rounded=1;" vertex="1" parent="grp">' +
|
||||
'<mxGeometry x="10" y="10" width="80" height="40" as="geometry"/></mxCell>' +
|
||||
'<mxCell id="b" value="B" style="rounded=1;" vertex="1" parent="grp">' +
|
||||
'<mxGeometry x="10" y="90" width="80" height="40" as="geometry"/></mxCell>' +
|
||||
'<mxCell id="e" style="" edge="1" parent="grp" source="a" target="b">' +
|
||||
'<mxGeometry relative="1" as="geometry"/></mxCell>' +
|
||||
"</root></mxGraphModel>";
|
||||
|
||||
function drawioPageDoc() {
|
||||
return {
|
||||
type: "doc",
|
||||
content: [
|
||||
{
|
||||
type: "drawio",
|
||||
attrs: {
|
||||
id: "d1", src: "/api/files/att-1/diagram.drawio.svg",
|
||||
attachmentId: "att-1", width: 400, height: 300,
|
||||
},
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
// --- drawioEditCells --------------------------------------------------------
|
||||
|
||||
test("drawioEditCells: applies ops and repoints the node (current baseHash)", async () => {
|
||||
const { client, calls } = makeClient({
|
||||
pageDoc: drawioPageDoc(),
|
||||
attachmentSvg: svgFor(STORED),
|
||||
});
|
||||
const baseHash = mxHash(normalizeXml(STORED));
|
||||
const res = await client.drawioEditCells(
|
||||
"page1",
|
||||
"d1",
|
||||
[
|
||||
{
|
||||
op: "update",
|
||||
cellId: "a",
|
||||
xml:
|
||||
'<mxCell id="a" value="Renamed" style="rounded=1;" vertex="1" parent="grp">' +
|
||||
'<mxGeometry x="10" y="10" width="80" height="40" as="geometry"/></mxCell>',
|
||||
},
|
||||
],
|
||||
baseHash,
|
||||
);
|
||||
assert.equal(res.success, true);
|
||||
assert.equal(calls.uploads.length, 1);
|
||||
const written = decodeDrawioSvg(calls.uploads[0].svg);
|
||||
const cells = parseCells(written);
|
||||
assert.equal(cells.find((c) => c.id === "a").value, "Renamed");
|
||||
assert.equal(cells.find((c) => c.id === "b").value, "B"); // untouched
|
||||
const n = findDrawio(calls.mutations[0].doc)[0];
|
||||
// The stub numbers uploads from 1; this edit is the first upload -> att-1.
|
||||
assert.equal(n.attrs.attachmentId, "att-1");
|
||||
});
|
||||
|
||||
test("drawioEditCells: delete of the container cascades to children + edge", async () => {
|
||||
const { client, calls } = makeClient({
|
||||
pageDoc: drawioPageDoc(),
|
||||
attachmentSvg: svgFor(STORED),
|
||||
});
|
||||
const baseHash = mxHash(normalizeXml(STORED));
|
||||
await client.drawioEditCells("page1", "d1", [{ op: "delete", cellId: "grp" }], baseHash);
|
||||
const written = decodeDrawioSvg(calls.uploads[0].svg);
|
||||
const ids = parseCells(written).filter((c) => c.id !== "0" && c.id !== "1").map((c) => c.id);
|
||||
assert.deepEqual(ids, [], "grp + a + b + edge all cascaded away");
|
||||
});
|
||||
|
||||
test("drawioEditCells: stale baseHash -> conflict, no upload", async () => {
|
||||
const { client, calls } = makeClient({
|
||||
pageDoc: drawioPageDoc(),
|
||||
attachmentSvg: svgFor(STORED),
|
||||
});
|
||||
await assert.rejects(
|
||||
() => client.drawioEditCells("page1", "d1", [{ op: "delete", cellId: "a" }], "stale"),
|
||||
/conflict/,
|
||||
);
|
||||
assert.equal(calls.uploads.length, 0);
|
||||
});
|
||||
|
||||
test("drawioEditCells: baseHash is mandatory", async () => {
|
||||
const { client } = makeClient({ pageDoc: drawioPageDoc(), attachmentSvg: svgFor(STORED) });
|
||||
await assert.rejects(
|
||||
() => client.drawioEditCells("page1", "d1", [{ op: "delete", cellId: "a" }], ""),
|
||||
/baseHash is mandatory/,
|
||||
);
|
||||
});
|
||||
|
||||
// --- drawioFromGraph --------------------------------------------------------
|
||||
|
||||
test("drawioFromGraph: builds a diagram from a graph and inserts a node", async () => {
|
||||
const pageDoc = { type: "doc", content: [{ type: "paragraph", attrs: { id: "p1" }, content: [] }] };
|
||||
const { client, calls } = makeClient({ pageDoc });
|
||||
const res = await client.drawioFromGraph(
|
||||
"page1",
|
||||
{ position: "append" },
|
||||
{
|
||||
nodes: [
|
||||
{ id: "api", label: "API", kind: "gateway", icon: "aws:api_gateway", group: "vpc" },
|
||||
{ id: "fn", label: "Handler", kind: "service", icon: "aws:lambda", group: "vpc" },
|
||||
{ id: "db", label: "Orders", kind: "db", icon: "aws:dynamodb" },
|
||||
],
|
||||
groups: [{ id: "vpc", label: "VPC" }],
|
||||
edges: [{ from: "api", to: "fn", kind: "sync" }, { from: "fn", to: "db", kind: "async" }],
|
||||
},
|
||||
"LR",
|
||||
"default",
|
||||
);
|
||||
assert.equal(res.success, true);
|
||||
assert.equal(res.nodeId, "#1");
|
||||
assert.equal(res.iconsMissing.length, 0, `unresolved: ${res.iconsMissing}`);
|
||||
assert.equal(res.iconsResolved, 3);
|
||||
// The uploaded model decodes back and carries the group + nodes.
|
||||
const written = decodeDrawioSvg(calls.uploads[0].svg);
|
||||
const cells = parseCells(written);
|
||||
assert.ok(cells.some((c) => c.id === "vpc"));
|
||||
assert.ok(cells.some((c) => c.id === "api"));
|
||||
// Group is transparent.
|
||||
const vpc = cells.find((c) => c.id === "vpc");
|
||||
assert.equal(vpc.styleMap.fillColor, "none");
|
||||
assert.equal(vpc.styleMap.container, "1");
|
||||
});
|
||||
|
||||
test("drawioFromGraph: an invalid graph throws before any upload", async () => {
|
||||
const { client, calls } = makeClient({ pageDoc: { type: "doc", content: [] } });
|
||||
await assert.rejects(
|
||||
() => client.drawioFromGraph("page1", { position: "append" }, { nodes: [] }),
|
||||
/non-empty/,
|
||||
);
|
||||
assert.equal(calls.uploads.length, 0);
|
||||
});
|
||||
|
||||
test("drawioFromGraph incremental into an existing node keeps prior coords", async () => {
|
||||
// The stored diagram has a,b at known coords; add a new node c incrementally.
|
||||
const { client, calls } = makeClient({
|
||||
pageDoc: drawioPageDoc(),
|
||||
attachmentSvg: svgFor(STORED),
|
||||
});
|
||||
const res = await client.drawioFromGraph(
|
||||
"page1",
|
||||
{ position: "append" },
|
||||
{
|
||||
nodes: [
|
||||
{ id: "a", label: "A" },
|
||||
{ id: "b", label: "B" },
|
||||
{ id: "c", label: "C new" },
|
||||
],
|
||||
edges: [{ from: "b", to: "c" }],
|
||||
},
|
||||
undefined,
|
||||
undefined,
|
||||
"incremental",
|
||||
"d1", // target the existing diagram
|
||||
);
|
||||
assert.equal(res.success, true);
|
||||
const written = decodeDrawioSvg(calls.uploads[0].svg);
|
||||
const cells = parseCells(written);
|
||||
const a = cells.find((c) => c.id === "a");
|
||||
const b = cells.find((c) => c.id === "b");
|
||||
// Existing coords preserved (the stored a/b absolute coords from STORED).
|
||||
assert.equal(a.geometry.x, 10);
|
||||
assert.equal(a.geometry.y, 10);
|
||||
assert.equal(b.geometry.x, 10);
|
||||
assert.equal(b.geometry.y, 90);
|
||||
assert.ok(cells.some((c) => c.id === "c"), "new node c added");
|
||||
});
|
||||
|
||||
// --- drawioFromMermaid ------------------------------------------------------
|
||||
|
||||
test("drawioFromMermaid: converts a flowchart and inserts a diagram", async () => {
|
||||
const pageDoc = { type: "doc", content: [{ type: "paragraph", attrs: { id: "p1" }, content: [] }] };
|
||||
const { client, calls } = makeClient({ pageDoc });
|
||||
const res = await client.drawioFromMermaid(
|
||||
"page1",
|
||||
{ position: "append" },
|
||||
"flowchart LR\n A[Start] --> B{Choose}\n B -->|yes| C[Done]\n B -->|no| D[Stop]",
|
||||
);
|
||||
assert.equal(res.success, true);
|
||||
const written = decodeDrawioSvg(calls.uploads[0].svg);
|
||||
const cells = parseCells(written);
|
||||
for (const id of ["A", "B", "C", "D"]) {
|
||||
assert.ok(cells.some((c) => c.id === id), `node ${id} present`);
|
||||
}
|
||||
});
|
||||
|
||||
test("drawioFromMermaid: a non-flowchart is rejected, no upload", async () => {
|
||||
const { client, calls } = makeClient({ pageDoc: { type: "doc", content: [] } });
|
||||
await assert.rejects(
|
||||
() => client.drawioFromMermaid("page1", { position: "append" }, "sequenceDiagram\n A->>B: x"),
|
||||
/only 'flowchart'\/'graph' is supported/,
|
||||
);
|
||||
assert.equal(calls.uploads.length, 0);
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
// Contract tests for the drawio_get / drawio_create / drawio_update client
|
||||
// Contract tests for the drawioGet / drawioCreate / drawioUpdate client
|
||||
// methods (issue #423). Follows the repo's seam-override pattern (see
|
||||
// full-doc-write-canonicalize.test.mjs): a DocmostClient subclass stubs the I/O
|
||||
// seams (auth, collab token, page read, attachment upload/fetch, the mutatePage
|
||||
@@ -114,9 +114,9 @@ function findDrawio(node, acc = []) {
|
||||
return acc;
|
||||
}
|
||||
|
||||
// --- drawio_create ---------------------------------------------------------
|
||||
// --- drawioCreate ---------------------------------------------------------
|
||||
|
||||
test("drawio_create: lints, builds the .drawio.svg, uploads and inserts a node", async () => {
|
||||
test("drawioCreate: lints, builds the .drawio.svg, uploads and inserts a node", async () => {
|
||||
const pageDoc = {
|
||||
type: "doc",
|
||||
content: [{ type: "paragraph", attrs: { id: "p1" }, content: [] }],
|
||||
@@ -148,7 +148,7 @@ test("drawio_create: lints, builds the .drawio.svg, uploads and inserts a node",
|
||||
assert.equal(n.attrs.title, "My diagram");
|
||||
});
|
||||
|
||||
test("drawio_create: a lint violation throws before any upload", async () => {
|
||||
test("drawioCreate: a lint violation throws before any upload", async () => {
|
||||
const { client, calls } = makeClient({ pageDoc: { type: "doc", content: [] } });
|
||||
// Edge with no child geometry -> edge-geometry rule.
|
||||
const bad =
|
||||
@@ -162,7 +162,7 @@ test("drawio_create: a lint violation throws before any upload", async () => {
|
||||
assert.equal(calls.uploads.length, 0, "no attachment uploaded on lint failure");
|
||||
});
|
||||
|
||||
test("drawio_create: before/after requires exactly one anchor", async () => {
|
||||
test("drawioCreate: before/after requires exactly one anchor", async () => {
|
||||
const { client } = makeClient({ pageDoc: { type: "doc", content: [] } });
|
||||
await assert.rejects(
|
||||
() => client.drawioCreate("page1", { position: "before" }, MODEL),
|
||||
@@ -170,9 +170,9 @@ test("drawio_create: before/after requires exactly one anchor", async () => {
|
||||
);
|
||||
});
|
||||
|
||||
// --- drawio_get ------------------------------------------------------------
|
||||
// --- drawioGet ------------------------------------------------------------
|
||||
|
||||
test("drawio_get: decodes the model and returns meta with a hash", async () => {
|
||||
test("drawioGet: decodes the model and returns meta with a hash", async () => {
|
||||
const pageDoc = {
|
||||
type: "doc",
|
||||
content: [
|
||||
@@ -198,7 +198,7 @@ test("drawio_get: decodes the model and returns meta with a hash", async () => {
|
||||
assert.equal(res.meta.hash, mxHash(normalizeXml(MODEL)));
|
||||
});
|
||||
|
||||
test("drawio_get: format=svg returns the raw .drawio.svg", async () => {
|
||||
test("drawioGet: format=svg returns the raw .drawio.svg", async () => {
|
||||
const svg = svgFor(MODEL);
|
||||
const pageDoc = {
|
||||
type: "doc",
|
||||
@@ -211,7 +211,7 @@ test("drawio_get: format=svg returns the raw .drawio.svg", async () => {
|
||||
assert.equal(res.content, svg);
|
||||
});
|
||||
|
||||
test("drawio_get: reads a HUMAN-saved compressed diagram losslessly (pako)", async () => {
|
||||
test("drawioGet: reads a HUMAN-saved compressed diagram losslessly (pako)", async () => {
|
||||
const pageDoc = {
|
||||
type: "doc",
|
||||
content: [
|
||||
@@ -223,7 +223,7 @@ test("drawio_get: reads a HUMAN-saved compressed diagram losslessly (pako)", asy
|
||||
assert.equal(res.content, normalizeXml(MODEL));
|
||||
});
|
||||
|
||||
// --- drawio_update ---------------------------------------------------------
|
||||
// --- drawioUpdate ---------------------------------------------------------
|
||||
|
||||
const UPDATED_MODEL =
|
||||
'<mxGraphModel><root>' +
|
||||
@@ -250,7 +250,7 @@ function updatePageDoc() {
|
||||
};
|
||||
}
|
||||
|
||||
test("drawio_update: stale baseHash -> conflict, no upload", async () => {
|
||||
test("drawioUpdate: stale baseHash -> conflict, no upload", async () => {
|
||||
const { client, calls } = makeClient({
|
||||
pageDoc: updatePageDoc(),
|
||||
attachmentSvg: svgFor(MODEL),
|
||||
@@ -262,7 +262,7 @@ test("drawio_update: stale baseHash -> conflict, no upload", async () => {
|
||||
assert.equal(calls.uploads.length, 0, "no upload on conflict");
|
||||
});
|
||||
|
||||
test("drawio_update: current baseHash -> uploads new attachment and repoints node dims", async () => {
|
||||
test("drawioUpdate: current baseHash -> uploads new attachment and repoints node dims", async () => {
|
||||
const currentHash = mxHash(normalizeXml(MODEL));
|
||||
const { client, calls } = makeClient({
|
||||
pageDoc: updatePageDoc(),
|
||||
@@ -285,7 +285,7 @@ test("drawio_update: current baseHash -> uploads new attachment and repoints nod
|
||||
assert.equal(n.attrs.id, undefined);
|
||||
});
|
||||
|
||||
test("drawio_update: baseHash is mandatory", async () => {
|
||||
test("drawioUpdate: baseHash is mandatory", async () => {
|
||||
const { client } = makeClient({ pageDoc: updatePageDoc(), attachmentSvg: svgFor(MODEL) });
|
||||
await assert.rejects(
|
||||
() => client.drawioUpdate("page1", "d1", UPDATED_MODEL, ""),
|
||||
@@ -295,7 +295,7 @@ test("drawio_update: baseHash is mandatory", async () => {
|
||||
|
||||
// --- Fix 1: the create handle must resolve on the SAVED doc (no id) ---------
|
||||
|
||||
test("drawio_create -> get/update: returned #<index> handle resolves on the saved doc (id dropped)", async () => {
|
||||
test("drawioCreate -> get/update: returned #<index> handle resolves on the saved doc (id dropped)", async () => {
|
||||
// Create appends a drawio node after the existing paragraph.
|
||||
const createDoc = {
|
||||
type: "doc",
|
||||
@@ -316,13 +316,13 @@ test("drawio_create -> get/update: returned #<index> handle resolves on the save
|
||||
const savedDoc = create.calls.mutations[0].doc;
|
||||
assert.equal(findDrawio(savedDoc)[0].attrs.id, undefined);
|
||||
|
||||
// drawio_get with the returned handle resolves the just-created node.
|
||||
// drawioGet with the returned handle resolves the just-created node.
|
||||
const getClient = makeClient({ pageDoc: savedDoc, attachmentSvg: svgFor(MODEL) });
|
||||
const got = await getClient.client.drawioGet("page1", res.nodeId, "xml");
|
||||
assert.equal(got.nodeId, res.nodeId);
|
||||
assert.equal(got.content, normalizeXml(MODEL));
|
||||
|
||||
// drawio_update with the same handle + the hash from get repoints that node.
|
||||
// drawioUpdate with the same handle + the hash from get repoints that node.
|
||||
const upClient = makeClient({ pageDoc: savedDoc, attachmentSvg: svgFor(MODEL) });
|
||||
const upd = await upClient.client.drawioUpdate(
|
||||
"page1",
|
||||
@@ -342,7 +342,7 @@ test("drawio_create -> get/update: returned #<index> handle resolves on the save
|
||||
|
||||
// --- error paths: the LLM must get a clean error, not a crash --------------
|
||||
|
||||
test("drawio_get: a bad node ref -> clean 'no node found' error", async () => {
|
||||
test("drawioGet: a bad node ref -> clean 'no node found' error", async () => {
|
||||
// Page has one paragraph; the requested ref resolves to nothing.
|
||||
const pageDoc = {
|
||||
type: "doc",
|
||||
@@ -355,7 +355,7 @@ test("drawio_get: a bad node ref -> clean 'no node found' error", async () => {
|
||||
);
|
||||
});
|
||||
|
||||
test("drawio_get: a drawio node with no src -> clean 'has no src to read' error", async () => {
|
||||
test("drawioGet: a drawio node with no src -> clean 'has no src to read' error", async () => {
|
||||
const pageDoc = {
|
||||
type: "doc",
|
||||
content: [
|
||||
@@ -370,7 +370,7 @@ test("drawio_get: a drawio node with no src -> clean 'has no src to read' error"
|
||||
);
|
||||
});
|
||||
|
||||
test("drawio_update: the resolved node is NOT a drawio node -> clean error, no upload", async () => {
|
||||
test("drawioUpdate: the resolved node is NOT a drawio node -> clean error, no upload", async () => {
|
||||
// "#0" resolves to a paragraph. The update must refuse cleanly rather than
|
||||
// crash or repoint the wrong node.
|
||||
const pageDoc = {
|
||||
@@ -386,7 +386,7 @@ test("drawio_update: the resolved node is NOT a drawio node -> clean error, no u
|
||||
assert.equal(calls.mutations.length, 0, "no write when the node is not a diagram");
|
||||
});
|
||||
|
||||
test("drawio_create: anchor not found -> clean error that reports the orphan attachment", async () => {
|
||||
test("drawioCreate: anchor not found -> clean error that reports the orphan attachment", async () => {
|
||||
// The upload happens before the mutate transform; when the anchor cannot be
|
||||
// found the write is skipped and the (now unreferenced) attachment is named
|
||||
// in the error, exactly as the code documents.
|
||||
@@ -416,7 +416,7 @@ test("drawio_create: anchor not found -> clean error that reports the orphan att
|
||||
|
||||
// --- Fix 2: update targets ONLY the resolved node --------------------------
|
||||
|
||||
test("drawio_update: repoints ONLY the addressed node, not siblings sharing an attachmentId", async () => {
|
||||
test("drawioUpdate: repoints ONLY the addressed node, not siblings sharing an attachmentId", async () => {
|
||||
// A copied diagram: two drawio nodes share one attachmentId. Updating via the
|
||||
// "#0" handle must touch node #0 only, never the sibling copy.
|
||||
const shared = {
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
// (issue #228):
|
||||
// - insertFootnote (#11): the required-argument guards reject BEFORE any write,
|
||||
// and never touch the collab/mutate path.
|
||||
// - transformPage / docmost_transform (#13): the auto-canonicalize step
|
||||
// - transformPage / docmostTransform (#13): the auto-canonicalize step
|
||||
// (`result = canonicalizeFootnotes(raw)`) runs after every transform, so a
|
||||
// transform that introduces an orphan footnote definition is silently tidied
|
||||
// away — observable as an EMPTY diff in a dryRun preview.
|
||||
@@ -10,7 +10,7 @@
|
||||
// These stand a local http.createServer in for Docmost and only exercise plain
|
||||
// HTTP routes (login / comments / pages.info), deliberately avoiding the live
|
||||
// Hocuspocus collab WebSocket: the insertFootnote guards short-circuit before it,
|
||||
// and docmost_transform's dryRun preview never opens it. The collab mutate path
|
||||
// and docmostTransform's dryRun preview never opens it. The collab mutate path
|
||||
// itself — abort-via-throw on a missing anchor with NO persisted write, and the
|
||||
// reused-vs-new response shaping — is covered in
|
||||
// test/mock/insert-footnote-wrapper.test.mjs (which overrides the mutatePage
|
||||
@@ -101,7 +101,7 @@ test("insertFootnote rejects an empty text before any write", async () => {
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// #13 docmost_transform auto-canonicalization: a transform that adds an orphan
|
||||
// #13 docmostTransform auto-canonicalization: a transform that adds an orphan
|
||||
// footnote definition produces NO net change (the canonicalizer drops it), so a
|
||||
// dryRun preview reports an empty diff. Without the auto-canonicalize step the
|
||||
// orphan would survive and the diff would be non-empty.
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// Footnote-canonicalization binding tests for the MCP FULL-document write tools
|
||||
// (issue #228, review #4): update_page_json and copy_page_content must persist a
|
||||
// (issue #228, review #4): updatePageJson and copyPageContent must persist a
|
||||
// footnote-canonical doc. These override the `replacePage` seam (symmetric to the
|
||||
// `mutatePage` seam used by the insert-footnote-wrapper test) to capture the
|
||||
// persisted doc WITHOUT a live Hocuspocus collab socket. Symmetric to the
|
||||
@@ -44,7 +44,7 @@ function makeClient(sourceDoc) {
|
||||
return { client, calls };
|
||||
}
|
||||
|
||||
test("update_page_json canonicalizes the persisted full doc (out-of-order -> reference order)", async () => {
|
||||
test("updatePageJson canonicalizes the persisted full doc (out-of-order -> reference order)", async () => {
|
||||
const { client, calls } = makeClient();
|
||||
const outOfOrder = {
|
||||
type: "doc",
|
||||
@@ -60,7 +60,7 @@ test("update_page_json canonicalizes the persisted full doc (out-of-order -> ref
|
||||
assert.equal(findAll(calls.replaced[0].doc, "footnotesList").length, 1);
|
||||
});
|
||||
|
||||
test("copy_page_content canonicalizes the persisted copy (orphan definition dropped)", async () => {
|
||||
test("copyPageContent canonicalizes the persisted copy (orphan definition dropped)", async () => {
|
||||
const sourceDoc = {
|
||||
type: "doc",
|
||||
content: [
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
// #413: getNode's markdown-default format, its JSON opt-in, the non-top-level
|
||||
// AUTO fallback to JSON, and comment-anchor preservation (incl. resolved) on the
|
||||
// markdown read. getNode only reads (getPageRaw), so a lightweight subclass that
|
||||
// stubs auth + the page fetch is enough — no collab socket needed.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { DocmostClient } from "../../build/client.js";
|
||||
|
||||
function makeClient(doc) {
|
||||
class TestClient extends DocmostClient {
|
||||
async ensureAuthenticated() {}
|
||||
async getPageRaw(pageId) {
|
||||
return { id: pageId, slugId: "s", title: "P", spaceId: "sp", content: doc };
|
||||
}
|
||||
}
|
||||
return new TestClient("http://127.0.0.1:1/api", "e@x.com", "pw");
|
||||
}
|
||||
|
||||
const P = "p1";
|
||||
|
||||
test("getNode defaults to markdown for a paragraph", async () => {
|
||||
const doc = {
|
||||
type: "doc",
|
||||
content: [
|
||||
{
|
||||
type: "paragraph",
|
||||
attrs: { id: "b1" },
|
||||
content: [{ type: "text", text: "hello world" }],
|
||||
},
|
||||
],
|
||||
};
|
||||
const res = await makeClient(doc).getNode(P, "b1");
|
||||
assert.equal(res.format, "markdown");
|
||||
assert.equal(typeof res.markdown, "string");
|
||||
assert.match(res.markdown, /hello world/);
|
||||
assert.equal(res.node, undefined, "markdown result carries no raw node");
|
||||
});
|
||||
|
||||
test("getNode format:'json' returns the raw subtree verbatim", async () => {
|
||||
const target = {
|
||||
type: "paragraph",
|
||||
attrs: { id: "b1" },
|
||||
content: [{ type: "text", text: "hello" }],
|
||||
};
|
||||
const doc = { type: "doc", content: [target] };
|
||||
const res = await makeClient(doc).getNode(P, "b1", "json");
|
||||
assert.equal(res.format, "json");
|
||||
assert.deepEqual(res.node, target);
|
||||
assert.equal(res.markdown, undefined);
|
||||
});
|
||||
|
||||
test("getNode AUTO-falls back to JSON for a non-top-level type (tableRow via #index)", async () => {
|
||||
const doc = {
|
||||
type: "doc",
|
||||
content: [
|
||||
{
|
||||
type: "table",
|
||||
content: [
|
||||
{
|
||||
type: "tableRow",
|
||||
content: [
|
||||
{
|
||||
type: "tableCell",
|
||||
attrs: { colspan: 1, rowspan: 1 },
|
||||
content: [
|
||||
{
|
||||
type: "paragraph",
|
||||
attrs: { id: "cp" },
|
||||
content: [{ type: "text", text: "x" }],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
// "#0.0"-style refs are not supported; the whole table is "#0", a row is only
|
||||
// reachable by drilling — but a tableRow IS a non-doc-child type. Address the
|
||||
// table itself as "#0": a table CAN be a doc child, so markdown is fine there.
|
||||
// To hit the fallback, address the row by walking: getNode resolves "#0" to the
|
||||
// table (doc child -> markdown). Instead we verify the schema gate directly by
|
||||
// asking for the table (markdown) and a row is exercised via the unit test on
|
||||
// canBeDocChild; here confirm a table renders as markdown.
|
||||
const tableRes = await makeClient(doc).getNode(P, "#0");
|
||||
assert.equal(tableRes.format, "markdown", "a table is a doc child -> markdown");
|
||||
|
||||
// Now build a doc whose top-level block IS a tableRow (schematically invalid but
|
||||
// exercises the getNode fallback branch): getNode("#0") resolves it and, because
|
||||
// tableRow cannot be a doc child, must fall back to JSON.
|
||||
const rowDoc = {
|
||||
type: "doc",
|
||||
content: [
|
||||
{
|
||||
type: "tableRow",
|
||||
content: [
|
||||
{
|
||||
type: "tableCell",
|
||||
attrs: { colspan: 1, rowspan: 1 },
|
||||
content: [{ type: "paragraph", content: [{ type: "text", text: "y" }] }],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
const rowRes = await makeClient(rowDoc).getNode(P, "#0");
|
||||
assert.equal(rowRes.format, "json", "a tableRow cannot be a doc child -> JSON fallback");
|
||||
assert.equal(rowRes.type, "tableRow");
|
||||
assert.ok(rowRes.node, "the JSON fallback returns the raw subtree");
|
||||
});
|
||||
|
||||
test("getNode(markdown) PRESERVES comment anchors — active and resolved", async () => {
|
||||
// A paragraph with two comment marks: one active, one resolved. get_page strips
|
||||
// resolved anchors; getNode must NOT (a read for editing/write-back).
|
||||
const doc = {
|
||||
type: "doc",
|
||||
content: [
|
||||
{
|
||||
type: "paragraph",
|
||||
attrs: { id: "b1" },
|
||||
content: [
|
||||
{ type: "text", text: "start " },
|
||||
{
|
||||
type: "text",
|
||||
text: "active",
|
||||
marks: [{ type: "comment", attrs: { commentId: "cid-active" } }],
|
||||
},
|
||||
{ type: "text", text: " mid " },
|
||||
{
|
||||
type: "text",
|
||||
text: "resolved",
|
||||
marks: [
|
||||
{
|
||||
type: "comment",
|
||||
attrs: { commentId: "cid-resolved", resolved: true },
|
||||
},
|
||||
],
|
||||
},
|
||||
{ type: "text", text: " end" },
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
const res = await makeClient(doc).getNode(P, "b1");
|
||||
assert.equal(res.format, "markdown");
|
||||
assert.match(
|
||||
res.markdown,
|
||||
/data-comment-id="cid-active"/,
|
||||
"the active comment anchor is preserved",
|
||||
);
|
||||
assert.match(
|
||||
res.markdown,
|
||||
/data-comment-id="cid-resolved"/,
|
||||
"the RESOLVED comment anchor is ALSO preserved (unlike get_page)",
|
||||
);
|
||||
});
|
||||
@@ -0,0 +1,375 @@
|
||||
// Mock-HTTP tests for DocmostClient.getPageContext — the #443 "where am I /
|
||||
// what's around" read tool. A local http.createServer stands in for Docmost
|
||||
// (same harness style as pagination-cursor.test.mjs) so everything is
|
||||
// deterministic and offline.
|
||||
//
|
||||
// Contract pinned here:
|
||||
// - Two requests: POST /pages/breadcrumbs (ancestor chain root->page, page
|
||||
// INCLUDED as the LAST element) + listSidebarPages (direct children).
|
||||
// - Split: last chain element -> `page`; the rest (root->parent) ->
|
||||
// `breadcrumbs`. A ROOT page (chain length 1) -> breadcrumbs: [].
|
||||
// - children: {pageId, title, hasChildren} in sidebar order.
|
||||
// - INVARIANT: only the UUID `pageId` is exposed, never `slugId`.
|
||||
// - A slugId input is resolved via /pages/info first (adds one request); a
|
||||
// UUID input short-circuits (stays at two requests).
|
||||
// - A bad/inaccessible pageId throws a CLEAR error, not an empty object.
|
||||
import { test, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import http from "node:http";
|
||||
import { DocmostClient } from "../../build/client.js";
|
||||
|
||||
function readBody(req) {
|
||||
return new Promise((resolve) => {
|
||||
let raw = "";
|
||||
req.on("data", (chunk) => {
|
||||
raw += chunk;
|
||||
});
|
||||
req.on("end", () => resolve(raw));
|
||||
});
|
||||
}
|
||||
|
||||
function startServer(handler) {
|
||||
return new Promise((resolve) => {
|
||||
const server = http.createServer(handler);
|
||||
server.listen(0, "127.0.0.1", () => {
|
||||
const { port } = server.address();
|
||||
resolve({ server, baseURL: `http://127.0.0.1:${port}/api` });
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function closeServer(server) {
|
||||
return new Promise((resolve) => server.close(resolve));
|
||||
}
|
||||
|
||||
function sendJson(res, status, obj, extraHeaders = {}) {
|
||||
res.writeHead(status, { "Content-Type": "application/json", ...extraHeaders });
|
||||
res.end(JSON.stringify(obj));
|
||||
}
|
||||
|
||||
const openServers = [];
|
||||
async function spawn(handler) {
|
||||
const { server, baseURL } = await startServer(handler);
|
||||
openServers.push(server);
|
||||
return { server, baseURL };
|
||||
}
|
||||
|
||||
after(async () => {
|
||||
await Promise.all(openServers.map((s) => closeServer(s)));
|
||||
});
|
||||
|
||||
function handleLogin(req, res) {
|
||||
if (req.url === "/api/auth/login") {
|
||||
sendJson(res, 200, { success: true }, {
|
||||
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
|
||||
});
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// Two real UUIDs so resolvePageId short-circuits (no /pages/info round-trip).
|
||||
const ROOT_UUID = "00000000-0000-4000-8000-000000000001";
|
||||
const MID_UUID = "00000000-0000-4000-8000-000000000002";
|
||||
const PAGE_UUID = "00000000-0000-4000-8000-000000000003";
|
||||
const CHILD_A = "00000000-0000-4000-8000-00000000000a";
|
||||
const CHILD_B = "00000000-0000-4000-8000-00000000000b";
|
||||
|
||||
// Build a breadcrumbs response as the server sends it: root->page order, page
|
||||
// LAST, wrapped in the {data,success} envelope. slugId/icon/position are present
|
||||
// on the wire (they must NOT leak into the tool output).
|
||||
function breadcrumbsEnvelope(chain) {
|
||||
return { success: true, data: chain };
|
||||
}
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// 1) 3rd-level page: page = last chain element; breadcrumbs = the two ancestors
|
||||
// root->parent; children mapped {pageId,title,hasChildren} in order; no leak;
|
||||
// exactly two requests for a UUID input.
|
||||
// -----------------------------------------------------------------------------
|
||||
test("getPageContext: 3rd-level page splits chain, maps children, no slugId leak, 2 requests", async () => {
|
||||
let breadcrumbReqs = 0;
|
||||
let sidebarReqs = 0;
|
||||
let infoReqs = 0;
|
||||
let breadcrumbBody = null;
|
||||
|
||||
const { baseURL } = await spawn(async (req, res) => {
|
||||
const raw = await readBody(req);
|
||||
if (handleLogin(req, res)) return;
|
||||
if (req.url === "/api/pages/info") {
|
||||
infoReqs++;
|
||||
sendJson(res, 404, {});
|
||||
return;
|
||||
}
|
||||
if (req.url === "/api/pages/breadcrumbs") {
|
||||
breadcrumbReqs++;
|
||||
breadcrumbBody = JSON.parse(raw || "{}");
|
||||
// root -> parent -> page (page LAST). slugId/icon/position on the wire.
|
||||
sendJson(
|
||||
res,
|
||||
200,
|
||||
breadcrumbsEnvelope([
|
||||
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", position: "a", icon: null, parentPageId: null, hasChildren: true },
|
||||
{ id: MID_UUID, slugId: "midSlug", title: "Datacenter A", spaceId: "sp1", position: "a", icon: null, parentPageId: ROOT_UUID, hasChildren: true },
|
||||
{ id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1", position: "b", icon: null, parentPageId: MID_UUID, hasChildren: true },
|
||||
]),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (req.url === "/api/pages/sidebar-pages") {
|
||||
sidebarReqs++;
|
||||
const body = JSON.parse(raw || "{}");
|
||||
assert.equal(body.pageId, PAGE_UUID, "children scoped to the page UUID");
|
||||
assert.equal(body.spaceId, "sp1", "children scoped to the page's space");
|
||||
sendJson(res, 200, {
|
||||
success: true,
|
||||
data: {
|
||||
items: [
|
||||
{ id: CHILD_A, slugId: "aSlug", title: "Servers", parentPageId: PAGE_UUID, hasChildren: true, position: "a" },
|
||||
{ id: CHILD_B, slugId: "bSlug", title: "Network", parentPageId: PAGE_UUID, hasChildren: false, position: "b" },
|
||||
],
|
||||
meta: { hasNextPage: false, nextCursor: null },
|
||||
},
|
||||
});
|
||||
return;
|
||||
}
|
||||
sendJson(res, 404, {});
|
||||
});
|
||||
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
const result = await client.getPageContext(PAGE_UUID);
|
||||
|
||||
assert.equal(infoReqs, 0, "UUID input short-circuits resolvePageId (no /pages/info)");
|
||||
assert.equal(breadcrumbReqs, 1, "exactly one breadcrumbs request");
|
||||
assert.equal(sidebarReqs, 1, "exactly one sidebar request");
|
||||
assert.deepEqual(breadcrumbBody, { pageId: PAGE_UUID }, "breadcrumbs posts the UUID");
|
||||
|
||||
// page = the LAST chain element.
|
||||
assert.deepEqual(result.page, {
|
||||
pageId: PAGE_UUID,
|
||||
title: "Rack 12",
|
||||
spaceId: "sp1",
|
||||
});
|
||||
// breadcrumbs = root->parent (the chain minus the page itself).
|
||||
assert.deepEqual(result.breadcrumbs, [
|
||||
{ pageId: ROOT_UUID, title: "Infrastructure" },
|
||||
{ pageId: MID_UUID, title: "Datacenter A" },
|
||||
]);
|
||||
// children mapped in order, hasChildren coerced to boolean.
|
||||
assert.deepEqual(result.children, [
|
||||
{ pageId: CHILD_A, title: "Servers", hasChildren: true },
|
||||
{ pageId: CHILD_B, title: "Network", hasChildren: false },
|
||||
]);
|
||||
|
||||
// No slugId anywhere in the output.
|
||||
const dump = JSON.stringify(result);
|
||||
assert.ok(!dump.includes("Slug"), "no slugId leaks into the output");
|
||||
assert.ok(!/\bslugId\b/.test(dump), "no slugId key in the output");
|
||||
});
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// 2) ROOT page: chain has ONE element (the page itself) -> breadcrumbs: [].
|
||||
// -----------------------------------------------------------------------------
|
||||
test("getPageContext: a root page has breadcrumbs: []", async () => {
|
||||
const { baseURL } = await spawn(async (req, res) => {
|
||||
const raw = await readBody(req);
|
||||
if (handleLogin(req, res)) return;
|
||||
if (req.url === "/api/pages/breadcrumbs") {
|
||||
// A root page: the CTE returns only the page itself.
|
||||
sendJson(
|
||||
res,
|
||||
200,
|
||||
breadcrumbsEnvelope([
|
||||
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", parentPageId: null, hasChildren: false },
|
||||
]),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (req.url === "/api/pages/sidebar-pages") {
|
||||
sendJson(res, 200, {
|
||||
success: true,
|
||||
data: { items: [], meta: { hasNextPage: false, nextCursor: null } },
|
||||
});
|
||||
return;
|
||||
}
|
||||
sendJson(res, 404, {});
|
||||
});
|
||||
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
const result = await client.getPageContext(ROOT_UUID);
|
||||
|
||||
assert.deepEqual(result.page, {
|
||||
pageId: ROOT_UUID,
|
||||
title: "Infrastructure",
|
||||
spaceId: "sp1",
|
||||
});
|
||||
assert.deepEqual(result.breadcrumbs, [], "root page: no ancestors");
|
||||
assert.deepEqual(result.children, [], "no children");
|
||||
});
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// 3) A slugId input is resolved via /pages/info first (one extra request), then
|
||||
// breadcrumbs/sidebar use the resolved UUID.
|
||||
// -----------------------------------------------------------------------------
|
||||
test("getPageContext: a slugId input is resolved via /pages/info", async () => {
|
||||
let infoReqs = 0;
|
||||
let infoBody = null;
|
||||
let breadcrumbBody = null;
|
||||
|
||||
const { baseURL } = await spawn(async (req, res) => {
|
||||
const raw = await readBody(req);
|
||||
if (handleLogin(req, res)) return;
|
||||
if (req.url === "/api/pages/info") {
|
||||
infoReqs++;
|
||||
infoBody = JSON.parse(raw || "{}");
|
||||
// getPageRaw: slugId -> canonical UUID.
|
||||
sendJson(res, 200, {
|
||||
success: true,
|
||||
data: { id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1" },
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (req.url === "/api/pages/breadcrumbs") {
|
||||
breadcrumbBody = JSON.parse(raw || "{}");
|
||||
sendJson(
|
||||
res,
|
||||
200,
|
||||
breadcrumbsEnvelope([
|
||||
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", parentPageId: null },
|
||||
{ id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1", parentPageId: ROOT_UUID },
|
||||
]),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (req.url === "/api/pages/sidebar-pages") {
|
||||
sendJson(res, 200, {
|
||||
success: true,
|
||||
data: { items: [], meta: { hasNextPage: false, nextCursor: null } },
|
||||
});
|
||||
return;
|
||||
}
|
||||
sendJson(res, 404, {});
|
||||
});
|
||||
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
const result = await client.getPageContext("pageSlug");
|
||||
|
||||
assert.equal(infoReqs, 1, "slugId resolved via one /pages/info");
|
||||
assert.deepEqual(infoBody, { pageId: "pageSlug" }, "resolve posts the raw slugId");
|
||||
assert.deepEqual(
|
||||
breadcrumbBody,
|
||||
{ pageId: PAGE_UUID },
|
||||
"breadcrumbs posts the RESOLVED uuid, not the slugId",
|
||||
);
|
||||
assert.equal(result.page.pageId, PAGE_UUID, "page.pageId is the UUID");
|
||||
assert.deepEqual(result.breadcrumbs, [
|
||||
{ pageId: ROOT_UUID, title: "Infrastructure" },
|
||||
]);
|
||||
});
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// 4) >20 children: cursor pagination returns ALL of them, no dupes (regression
|
||||
// on the #442 bug class — getPageContext must not re-introduce a cap).
|
||||
// -----------------------------------------------------------------------------
|
||||
test("getPageContext: a page with >20 children returns ALL of them (no cap, no dupes)", async () => {
|
||||
// 45 children spread over three cursor pages.
|
||||
const all = Array.from({ length: 45 }, (_, i) => ({
|
||||
id: `child-${i}`,
|
||||
slugId: `slug-${i}`,
|
||||
title: `Child ${i}`,
|
||||
parentPageId: PAGE_UUID,
|
||||
hasChildren: i % 2 === 0,
|
||||
}));
|
||||
const PAGES = {
|
||||
"": { items: all.slice(0, 20), nextCursor: "c1" },
|
||||
c1: { items: all.slice(20, 40), nextCursor: "c2" },
|
||||
c2: { items: all.slice(40), nextCursor: null },
|
||||
};
|
||||
|
||||
const { baseURL } = await spawn(async (req, res) => {
|
||||
const raw = await readBody(req);
|
||||
if (handleLogin(req, res)) return;
|
||||
if (req.url === "/api/pages/breadcrumbs") {
|
||||
sendJson(
|
||||
res,
|
||||
200,
|
||||
breadcrumbsEnvelope([
|
||||
{ id: PAGE_UUID, slugId: "pageSlug", title: "Big Parent", spaceId: "sp1", parentPageId: null },
|
||||
]),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (req.url === "/api/pages/sidebar-pages") {
|
||||
const body = JSON.parse(raw || "{}");
|
||||
const page = PAGES[body.cursor ?? ""] ?? { items: [], nextCursor: null };
|
||||
sendJson(res, 200, {
|
||||
success: true,
|
||||
data: {
|
||||
items: page.items,
|
||||
meta: { hasNextPage: page.nextCursor != null, nextCursor: page.nextCursor },
|
||||
},
|
||||
});
|
||||
return;
|
||||
}
|
||||
sendJson(res, 404, {});
|
||||
});
|
||||
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
const result = await client.getPageContext(PAGE_UUID);
|
||||
|
||||
assert.equal(result.children.length, 45, "all 45 children returned");
|
||||
const ids = result.children.map((c) => c.pageId);
|
||||
assert.equal(new Set(ids).size, 45, "no duplicate children");
|
||||
assert.deepEqual(ids, all.map((c) => c.id), "children in server order across cursor pages");
|
||||
assert.equal(result.children[0].hasChildren, true, "hasChildren preserved (child 0)");
|
||||
assert.equal(result.children[1].hasChildren, false, "hasChildren preserved (child 1)");
|
||||
});
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// 5) A nonexistent / inaccessible pageId -> a CLEAR error, NOT an empty object.
|
||||
// -----------------------------------------------------------------------------
|
||||
test("getPageContext: a bad/inaccessible pageId throws a clear error (not {})", async () => {
|
||||
const { baseURL } = await spawn(async (req, res) => {
|
||||
await readBody(req);
|
||||
if (handleLogin(req, res)) return;
|
||||
if (req.url === "/api/pages/breadcrumbs") {
|
||||
// Server rejects an unknown/forbidden page.
|
||||
sendJson(res, 404, { message: "Page not found" });
|
||||
return;
|
||||
}
|
||||
sendJson(res, 404, {});
|
||||
});
|
||||
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
await assert.rejects(
|
||||
() => client.getPageContext(PAGE_UUID),
|
||||
(err) => {
|
||||
assert.ok(err instanceof Error, "throws an Error");
|
||||
return true;
|
||||
},
|
||||
"a 404 from breadcrumbs propagates as a thrown error, not a hollow {}",
|
||||
);
|
||||
});
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// 6) An empty breadcrumbs chain (should never happen — the endpoint always
|
||||
// includes the page itself) is treated as not-found, not a hollow {page:...}.
|
||||
// -----------------------------------------------------------------------------
|
||||
test("getPageContext: an empty breadcrumbs chain throws (defensive)", async () => {
|
||||
const { baseURL } = await spawn(async (req, res) => {
|
||||
await readBody(req);
|
||||
if (handleLogin(req, res)) return;
|
||||
if (req.url === "/api/pages/breadcrumbs") {
|
||||
sendJson(res, 200, breadcrumbsEnvelope([]));
|
||||
return;
|
||||
}
|
||||
sendJson(res, 404, {});
|
||||
});
|
||||
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
await assert.rejects(
|
||||
() => client.getPageContext(PAGE_UUID),
|
||||
/not found or inaccessible/,
|
||||
"an empty chain is a clear error, not {}",
|
||||
);
|
||||
});
|
||||
@@ -1,6 +1,6 @@
|
||||
// Mock regression for the FAIL-FAST invalid-node validation (#409).
|
||||
//
|
||||
// A structural editor (patch_node / insert_node / update_page_json) given a doc
|
||||
// A structural editor (patchNode / insertNode / updatePageJson) given a doc
|
||||
// whose NESTED child has an absent/unknown `type` (the exact shape the Yjs
|
||||
// encoder rejects with `Unknown node type: undefined`) must throw a RICH,
|
||||
// path-anchored error BEFORE it ever opens a collab session or takes a page
|
||||
@@ -23,7 +23,7 @@ import { Hocuspocus } from "@hocuspocus/server";
|
||||
import { DocmostClient } from "../../build/client.js";
|
||||
import { buildYDoc } from "../../build/lib/collaboration.js";
|
||||
|
||||
// A minimal valid seed doc with a real block id, so the happy-path patch_node
|
||||
// A minimal valid seed doc with a real block id, so the happy-path patchNode
|
||||
// finds its target.
|
||||
const SEED_ID = "seed-para-id";
|
||||
function seedDoc() {
|
||||
@@ -130,14 +130,14 @@ const nestedUnknownTypeNode = () => ({
|
||||
content: [{ type: "paragraf", content: [{ type: "text", text: "x" }] }],
|
||||
});
|
||||
|
||||
test("patch_node fails fast on a nested typeless node — no collab connection", async () => {
|
||||
test("patchNode fails fast on a nested typeless node — no collab connection", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
await assert.rejects(
|
||||
() => client.patchNode(PAGE, SEED_ID, nestedTypelessNode()),
|
||||
() => client.patchNode(PAGE, SEED_ID, { node: nestedTypelessNode() }),
|
||||
(err) => {
|
||||
assert.match(err.message, /patch_node: invalid node/);
|
||||
assert.match(err.message, /patchNode: invalid node/);
|
||||
assert.match(err.message, /missing "type"/);
|
||||
assert.match(err.message, /content\[0\]/); // path-anchored
|
||||
return true;
|
||||
@@ -152,17 +152,21 @@ test("patch_node fails fast on a nested typeless node — no collab connection",
|
||||
assert.equal(state.changed, false, "the collab doc must never be written");
|
||||
});
|
||||
|
||||
test("insert_node fails fast on a nested UNKNOWN type — no collab connection", async () => {
|
||||
test("insertNode fails fast on a nested UNKNOWN type — no collab connection", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
await assert.rejects(
|
||||
() =>
|
||||
client.insertNode(PAGE, nestedUnknownTypeNode(), {
|
||||
position: "append",
|
||||
}),
|
||||
client.insertNode(
|
||||
PAGE,
|
||||
{ node: nestedUnknownTypeNode() },
|
||||
{
|
||||
position: "append",
|
||||
},
|
||||
),
|
||||
(err) => {
|
||||
assert.match(err.message, /insert_node: invalid node/);
|
||||
assert.match(err.message, /insertNode: invalid node/);
|
||||
assert.match(err.message, /unknown node type "paragraf"/);
|
||||
return true;
|
||||
},
|
||||
@@ -172,7 +176,7 @@ test("insert_node fails fast on a nested UNKNOWN type — no collab connection",
|
||||
assert.equal(state.changed, false);
|
||||
});
|
||||
|
||||
test("update_page_json fails fast on a nested typeless node — no collab connection", async () => {
|
||||
test("updatePageJson fails fast on a nested typeless node — no collab connection", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
@@ -184,7 +188,7 @@ test("update_page_json fails fast on a nested typeless node — no collab connec
|
||||
await assert.rejects(
|
||||
() => client.updatePageJson(PAGE, badDoc),
|
||||
(err) => {
|
||||
// update_page_json runs validateDocStructure first (string-type check),
|
||||
// updatePageJson runs validateDocStructure first (string-type check),
|
||||
// which already rejects a typeless node — so the message may come from
|
||||
// either guard, but the write must not happen.
|
||||
assert.match(err.message, /type/i);
|
||||
@@ -196,7 +200,7 @@ test("update_page_json fails fast on a nested typeless node — no collab connec
|
||||
assert.equal(state.changed, false);
|
||||
});
|
||||
|
||||
test("update_page_json fails fast on a nested UNKNOWN type name — rich #409 message", async () => {
|
||||
test("updatePageJson fails fast on a nested UNKNOWN type name — rich #409 message", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
@@ -210,7 +214,7 @@ test("update_page_json fails fast on a nested UNKNOWN type name — rich #409 me
|
||||
await assert.rejects(
|
||||
() => client.updatePageJson(PAGE, badDoc),
|
||||
(err) => {
|
||||
assert.match(err.message, /update_page_json: invalid node/);
|
||||
assert.match(err.message, /updatePageJson: invalid node/);
|
||||
assert.match(err.message, /unknown node type "paragraf"/);
|
||||
return true;
|
||||
},
|
||||
@@ -220,13 +224,15 @@ test("update_page_json fails fast on a nested UNKNOWN type name — rich #409 me
|
||||
assert.equal(state.changed, false);
|
||||
});
|
||||
|
||||
test("patch_node with a well-formed node proceeds to the collab write", async () => {
|
||||
test("patchNode with a well-formed node proceeds to the collab write", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
const result = await client.patchNode(PAGE, SEED_ID, {
|
||||
type: "paragraph",
|
||||
content: [{ type: "text", text: "replacement" }],
|
||||
node: {
|
||||
type: "paragraph",
|
||||
content: [{ type: "text", text: "replacement" }],
|
||||
},
|
||||
});
|
||||
|
||||
assert.equal(result.success, true);
|
||||
|
||||
@@ -0,0 +1,538 @@
|
||||
// Mock collab tests for the #413 MARKDOWN path of patchNode / insertNode and the
|
||||
// markdown-default getNode. These stand up a real Hocuspocus collab server seeded
|
||||
// with a chosen document (mirroring ambiguous-node-id.test.mjs), let the client
|
||||
// run its real transform against a live Y.Doc, and read the persisted result back
|
||||
// to assert on the written document.
|
||||
//
|
||||
// Coverage (issue #413):
|
||||
// - CANON CONVERGENCE: a block written via patchNode(markdown) is canonically
|
||||
// equal to the SAME content run through a full markdown import (no "second
|
||||
// canon" appears on the block-level path).
|
||||
// - id-THREAD on a 1->N splice: the first block inherits the target id, the rest
|
||||
// get fresh ids, and every NEIGHBOUR block is byte-identical before/after.
|
||||
// - XOR validation (both / neither markdown+node -> error).
|
||||
// - span/color-attr GUARD on the target block (a merged/colored cell refuses a
|
||||
// markdown patch, nothing written).
|
||||
// - `^[...]` footnote in the fragment -> a definition in the tail list + renumber.
|
||||
// - insertNode(markdown) inserts N blocks in order at the anchor.
|
||||
import { test, after } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import http from "node:http";
|
||||
import { WebSocketServer } from "ws";
|
||||
import { Hocuspocus } from "@hocuspocus/server";
|
||||
import { DocmostClient } from "../../build/client.js";
|
||||
import { buildYDoc } from "../../build/lib/collaboration.js";
|
||||
import {
|
||||
docsCanonicallyEqual,
|
||||
markdownToProseMirror,
|
||||
} from "@docmost/prosemirror-markdown";
|
||||
|
||||
const PAGE = "11111111-1111-4111-8111-111111111111";
|
||||
|
||||
// Deep JSON clone for byte-identity assertions.
|
||||
const jclone = (v) => JSON.parse(JSON.stringify(v));
|
||||
|
||||
function findAll(node, type, acc = []) {
|
||||
if (!node || typeof node !== "object") return acc;
|
||||
if (node.type === type) acc.push(node);
|
||||
if (Array.isArray(node.content))
|
||||
for (const c of node.content) findAll(c, type, acc);
|
||||
return acc;
|
||||
}
|
||||
|
||||
// Stand up an HTTP+Hocuspocus stack seeded with `seedDoc`. `state.lastDoc` holds
|
||||
// the most recently persisted document JSON (decoded from the live Y.Doc on every
|
||||
// change) so a test can inspect exactly what was written.
|
||||
async function spawnCollabStack(seedDoc) {
|
||||
const state = { changed: false, lastDoc: null };
|
||||
|
||||
const hocuspocus = new Hocuspocus({
|
||||
quiet: true,
|
||||
async onLoadDocument() {
|
||||
return buildYDoc(seedDoc);
|
||||
},
|
||||
async onChange(data) {
|
||||
state.changed = true;
|
||||
try {
|
||||
const frag = data.document.getXmlFragment("default");
|
||||
// Decode the live fragment back to JSON via the same helper the client
|
||||
// reads with — but simpler: use the yjs->json path exposed by the doc.
|
||||
state.lastDoc = fragmentToJson(frag);
|
||||
} catch {
|
||||
/* ignore decode errors in teardown races */
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
const wss = new WebSocketServer({ noServer: true });
|
||||
const server = http.createServer((req, res) => {
|
||||
let raw = "";
|
||||
req.on("data", (c) => (raw += c));
|
||||
req.on("end", () => {
|
||||
if (req.url === "/api/auth/login") {
|
||||
res.writeHead(200, {
|
||||
"Content-Type": "application/json",
|
||||
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
|
||||
});
|
||||
res.end(JSON.stringify({ success: true }));
|
||||
return;
|
||||
}
|
||||
if (req.url === "/api/auth/collab-token") {
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify({ data: { token: "collab-jwt" } }));
|
||||
return;
|
||||
}
|
||||
res.writeHead(404, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify({ message: "not found" }));
|
||||
});
|
||||
});
|
||||
|
||||
server.on("upgrade", (request, socket, head) => {
|
||||
if (!request.url || !request.url.startsWith("/collab")) {
|
||||
socket.destroy();
|
||||
return;
|
||||
}
|
||||
wss.handleUpgrade(request, socket, head, (ws) => {
|
||||
hocuspocus.handleConnection(ws, request);
|
||||
});
|
||||
});
|
||||
|
||||
const baseURL = await new Promise((resolve) => {
|
||||
server.listen(0, "127.0.0.1", () => {
|
||||
const { port } = server.address();
|
||||
resolve(`http://127.0.0.1:${port}/api`);
|
||||
});
|
||||
});
|
||||
|
||||
openStacks.push({ server, hocuspocus });
|
||||
return { state, baseURL };
|
||||
}
|
||||
|
||||
// Minimal XmlFragment -> ProseMirror JSON decode, mirroring the shape Docmost
|
||||
// stores. Reads element name as node type, attributes as attrs, and recurses into
|
||||
// children; text nodes carry their string.
|
||||
function fragmentToJson(frag) {
|
||||
const decodeNode = (el) => {
|
||||
if (el.constructor.name === "YXmlText") {
|
||||
// A yjs text node: collect the string with its formatting deltas.
|
||||
const delta = el.toDelta();
|
||||
return delta.map((d) => {
|
||||
const node = { type: "text", text: d.insert };
|
||||
if (d.attributes && Object.keys(d.attributes).length) {
|
||||
node.marks = Object.entries(d.attributes).map(([type, attrs]) =>
|
||||
attrs && typeof attrs === "object" && Object.keys(attrs).length
|
||||
? { type, attrs }
|
||||
: { type },
|
||||
);
|
||||
}
|
||||
return node;
|
||||
});
|
||||
}
|
||||
const node = { type: el.nodeName };
|
||||
const attrs = el.getAttributes();
|
||||
if (attrs && Object.keys(attrs).length) node.attrs = attrs;
|
||||
const children = [];
|
||||
for (const child of el.toArray()) {
|
||||
const decoded = decodeNode(child);
|
||||
if (Array.isArray(decoded)) children.push(...decoded);
|
||||
else children.push(decoded);
|
||||
}
|
||||
if (children.length) node.content = children;
|
||||
return node;
|
||||
};
|
||||
const content = [];
|
||||
for (const child of frag.toArray()) content.push(decodeNode(child));
|
||||
return { type: "doc", content };
|
||||
}
|
||||
|
||||
const openStacks = [];
|
||||
after(async () => {
|
||||
await Promise.all(
|
||||
openStacks.map(
|
||||
({ server, hocuspocus }) =>
|
||||
new Promise((resolve) => {
|
||||
server.close(() => {
|
||||
Promise.resolve(hocuspocus.destroy?.()).finally(resolve);
|
||||
});
|
||||
}),
|
||||
),
|
||||
);
|
||||
});
|
||||
|
||||
// A seed doc with two neighbour paragraphs around a target paragraph.
|
||||
function seed3() {
|
||||
return {
|
||||
type: "doc",
|
||||
content: [
|
||||
{
|
||||
type: "paragraph",
|
||||
attrs: { id: "before-id" },
|
||||
content: [{ type: "text", text: "before" }],
|
||||
},
|
||||
{
|
||||
type: "paragraph",
|
||||
attrs: { id: "target-id" },
|
||||
content: [{ type: "text", text: "old target" }],
|
||||
},
|
||||
{
|
||||
type: "paragraph",
|
||||
attrs: { id: "after-id" },
|
||||
content: [{ type: "text", text: "after" }],
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
test("patchNode(markdown): XOR — both markdown and node is rejected, nothing written", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack(seed3());
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
await assert.rejects(
|
||||
() =>
|
||||
client.patchNode(PAGE, "target-id", {
|
||||
markdown: "hello",
|
||||
node: { type: "paragraph" },
|
||||
}),
|
||||
/exactly one of/i,
|
||||
);
|
||||
assert.equal(state.changed, false, "no write on an XOR violation");
|
||||
});
|
||||
|
||||
test("patchNode(markdown): XOR — neither markdown nor node is rejected", async () => {
|
||||
const { baseURL } = await spawnCollabStack(seed3());
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
await assert.rejects(
|
||||
() => client.patchNode(PAGE, "target-id", {}),
|
||||
/exactly one of/i,
|
||||
);
|
||||
});
|
||||
|
||||
test("patchNode(markdown): single block keeps the id; neighbours byte-identical", async () => {
|
||||
const before = seed3();
|
||||
const { state, baseURL } = await spawnCollabStack(before);
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
const res = await client.patchNode(PAGE, "target-id", {
|
||||
markdown: "the **new** target",
|
||||
});
|
||||
assert.equal(res.success, true);
|
||||
assert.equal(res.replaced, 1);
|
||||
assert.equal(res.blocks, 1);
|
||||
|
||||
const doc = state.lastDoc;
|
||||
const paras = doc.content;
|
||||
// The rewritten block still carries the target id.
|
||||
const target = paras.find((p) => p.attrs?.id === "target-id");
|
||||
assert.ok(target, "rewritten block inherits target-id");
|
||||
assert.equal(target.content.some((n) => n.text === "new"), true);
|
||||
// Neighbours are byte-identical to the seed.
|
||||
const beforeNode = paras.find((p) => p.attrs?.id === "before-id");
|
||||
const afterNode = paras.find((p) => p.attrs?.id === "after-id");
|
||||
assert.deepEqual(beforeNode, before.content[0]);
|
||||
assert.deepEqual(afterNode, before.content[2]);
|
||||
});
|
||||
|
||||
test("patchNode(markdown): 1->N splice threads the id onto the first block; neighbours byte-identical", async () => {
|
||||
const before = seed3();
|
||||
const { state, baseURL } = await spawnCollabStack(before);
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
// Two paragraphs of markdown -> a 2-block fragment replacing one block.
|
||||
const res = await client.patchNode(PAGE, "target-id", {
|
||||
markdown: "first para\n\nsecond para",
|
||||
});
|
||||
assert.equal(res.blocks, 2);
|
||||
|
||||
const doc = state.lastDoc;
|
||||
const idx = doc.content.findIndex((p) => p.attrs?.id === "target-id");
|
||||
assert.ok(idx >= 0, "first spliced block inherits target-id");
|
||||
const first = doc.content[idx];
|
||||
const second = doc.content[idx + 1];
|
||||
assert.equal(first.content.some((n) => n.text === "first para"), true);
|
||||
assert.equal(second.content.some((n) => n.text === "second para"), true);
|
||||
// The second block has a DIFFERENT (fresh) id.
|
||||
assert.notEqual(second.attrs?.id, "target-id");
|
||||
assert.ok(second.attrs?.id, "the extra block gets a fresh id");
|
||||
// Neighbours untouched, byte-identical.
|
||||
assert.deepEqual(
|
||||
doc.content.find((p) => p.attrs?.id === "before-id"),
|
||||
before.content[0],
|
||||
);
|
||||
assert.deepEqual(
|
||||
doc.content.find((p) => p.attrs?.id === "after-id"),
|
||||
before.content[2],
|
||||
);
|
||||
});
|
||||
|
||||
test("patchNode(markdown): CANON CONVERGENCE — block equals the same content full-imported", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack(seed3());
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
const md = "a paragraph with **bold**, _italic_ and `code`";
|
||||
await client.patchNode(PAGE, "target-id", { markdown: md });
|
||||
|
||||
// The block as persisted.
|
||||
const target = state.lastDoc.content.find((p) => p.attrs?.id === "target-id");
|
||||
// The same markdown run through the full-page importer.
|
||||
const full = await markdownToProseMirror(md);
|
||||
const fullBlock = full.content[0];
|
||||
|
||||
assert.ok(
|
||||
docsCanonicallyEqual(
|
||||
{ type: "doc", content: [target] },
|
||||
{ type: "doc", content: [fullBlock] },
|
||||
),
|
||||
"a patchNode(markdown) block must be canonically equal to a full import — no second canon",
|
||||
);
|
||||
});
|
||||
|
||||
test("patchNode(markdown): a paragraph inside a merged (colspan) cell rewrites fine — the cell's span is preserved", async () => {
|
||||
// A cell paragraph carries an id and IS id-targetable; rewriting ITS content
|
||||
// from markdown replaces only the paragraph, so the cell's colspan is NOT lost
|
||||
// (the span lives on the cell, which patchNode leaves in place). This is the
|
||||
// correct behavior: no false guard, no loss. The guard's REJECTION logic (when
|
||||
// the replaced block itself carries/contains an unrepresentable span) is proven
|
||||
// by the findUnrepresentableTableAttrs unit test — that case is not reachable
|
||||
// through the id-targeting API because tables/cells carry no addressable id.
|
||||
const doc = {
|
||||
type: "doc",
|
||||
content: [
|
||||
{
|
||||
type: "table",
|
||||
content: [
|
||||
{
|
||||
type: "tableRow",
|
||||
content: [
|
||||
{
|
||||
type: "tableCell",
|
||||
attrs: { colspan: 2, rowspan: 1 },
|
||||
content: [
|
||||
{
|
||||
type: "paragraph",
|
||||
attrs: { id: "cell-para" },
|
||||
content: [{ type: "text", text: "merged" }],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
const { state, baseURL } = await spawnCollabStack(doc);
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
const res = await client.patchNode(PAGE, "cell-para", { markdown: "rewritten" });
|
||||
assert.equal(res.success, true);
|
||||
// The cell's colspan survives (the span is on the cell, not the paragraph).
|
||||
const cell = findAll(state.lastDoc, "tableCell")[0];
|
||||
assert.equal(cell.attrs.colspan, 2, "the cell's colspan is preserved");
|
||||
const para = findAll(cell, "paragraph")[0];
|
||||
assert.equal(
|
||||
(para.content || []).some((n) => n.text === "rewritten"),
|
||||
true,
|
||||
);
|
||||
});
|
||||
|
||||
test("patchNode(markdown): a `^[...]` footnote in the fragment lands in the tail list", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack(seed3());
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
await client.patchNode(PAGE, "target-id", {
|
||||
markdown: "a claim^[the supporting note]",
|
||||
});
|
||||
|
||||
const doc = state.lastDoc;
|
||||
const lists = findAll(doc, "footnotesList");
|
||||
assert.equal(lists.length, 1, "exactly one tail footnotesList");
|
||||
const defs = findAll(doc, "footnoteDefinition");
|
||||
assert.equal(defs.length, 1, "one definition for the fragment footnote");
|
||||
const refs = findAll(doc, "footnoteReference");
|
||||
assert.equal(refs.length, 1, "one reference in the body");
|
||||
// Reference and definition share an id (renumbered canonically).
|
||||
assert.equal(refs[0].attrs.id, defs[0].attrs.id);
|
||||
});
|
||||
|
||||
test("insertNode(markdown): inserts N blocks in order after the anchor", async () => {
|
||||
const before = seed3();
|
||||
const { state, baseURL } = await spawnCollabStack(before);
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
const res = await client.insertNode(
|
||||
PAGE,
|
||||
{ markdown: "new one\n\nnew two" },
|
||||
{ position: "after", anchorNodeId: "before-id" },
|
||||
);
|
||||
assert.equal(res.success, true);
|
||||
assert.equal(res.blocks, 2);
|
||||
|
||||
const texts = state.lastDoc.content.map((p) => (p.content || []).map((n) => n.text).join(""));
|
||||
// Order: before, new one, new two, target, after.
|
||||
assert.deepEqual(texts, ["before", "new one", "new two", "old target", "after"]);
|
||||
});
|
||||
|
||||
test("insertNode(markdown): XOR — both markdown and node is rejected", async () => {
|
||||
const { baseURL } = await spawnCollabStack(seed3());
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
await assert.rejects(
|
||||
() =>
|
||||
client.insertNode(
|
||||
PAGE,
|
||||
{ markdown: "x", node: { type: "paragraph" } },
|
||||
{ position: "append" },
|
||||
),
|
||||
/exactly one of/i,
|
||||
);
|
||||
});
|
||||
|
||||
// A seed page whose ONLY footnote reference lives in the target paragraph p1,
|
||||
// with a matching definition in a trailing footnotesList. Rewriting p1 with a
|
||||
// footnote-free fragment removes the last referrer -> the definition is orphaned.
|
||||
function seedOrphanFootnote() {
|
||||
return {
|
||||
type: "doc",
|
||||
content: [
|
||||
{
|
||||
type: "paragraph",
|
||||
attrs: { id: "p1" },
|
||||
content: [
|
||||
{ type: "text", text: "a claim" },
|
||||
{ type: "footnoteReference", attrs: { id: "fn-1", referenceNumber: 1 } },
|
||||
],
|
||||
},
|
||||
{
|
||||
type: "footnotesList",
|
||||
content: [
|
||||
{
|
||||
type: "footnoteDefinition",
|
||||
attrs: { id: "fn-1" },
|
||||
content: [
|
||||
{
|
||||
type: "paragraph",
|
||||
attrs: { id: "def-para" },
|
||||
content: [{ type: "text", text: "the supporting note" }],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
test("patchNode(markdown): removing the LAST footnote referrer drops the now-orphan definition (canonical convergence)", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack(seedOrphanFootnote());
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
// The fragment has NO footnotes -> definitions=[]; the splice removes the only
|
||||
// footnoteReference, leaving the tail definition orphaned. The canonicalization
|
||||
// pass (which mergeFootnoteDefinitions must still run) has to drop it.
|
||||
await client.patchNode(PAGE, "p1", { markdown: "just text" });
|
||||
|
||||
const doc = state.lastDoc;
|
||||
assert.equal(
|
||||
findAll(doc, "footnoteDefinition").length,
|
||||
0,
|
||||
"the orphaned definition is dropped",
|
||||
);
|
||||
assert.equal(
|
||||
findAll(doc, "footnotesList").length,
|
||||
0,
|
||||
"the emptied footnotesList is removed",
|
||||
);
|
||||
assert.equal(findAll(doc, "footnoteReference").length, 0, "no references remain");
|
||||
|
||||
// Convergence: the persisted result equals the SAME content imported whole.
|
||||
const full = await markdownToProseMirror("just text");
|
||||
const target = doc.content.find((p) => p.attrs?.id === "p1");
|
||||
assert.ok(
|
||||
docsCanonicallyEqual(
|
||||
{ type: "doc", content: [target] },
|
||||
{ type: "doc", content: [full.content[0]] },
|
||||
),
|
||||
"the post-splice doc is canonically identical to a full re-import",
|
||||
);
|
||||
});
|
||||
|
||||
test("patchNode(markdown): a pure-text patch on a footnote-FREE page leaves footnote topology untouched (fast path)", async () => {
|
||||
const before = seed3();
|
||||
const { state, baseURL } = await spawnCollabStack(before);
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
await client.patchNode(PAGE, "target-id", { markdown: "plain replacement" });
|
||||
|
||||
const doc = state.lastDoc;
|
||||
assert.equal(findAll(doc, "footnotesList").length, 0, "no footnotesList appears");
|
||||
assert.equal(findAll(doc, "footnoteDefinition").length, 0, "no definition appears");
|
||||
assert.equal(findAll(doc, "footnoteReference").length, 0, "no reference appears");
|
||||
// Neighbours byte-identical (the fast path does not clone/reshape the tree).
|
||||
assert.deepEqual(
|
||||
doc.content.find((p) => p.attrs?.id === "before-id"),
|
||||
before.content[0],
|
||||
);
|
||||
assert.deepEqual(
|
||||
doc.content.find((p) => p.attrs?.id === "after-id"),
|
||||
before.content[2],
|
||||
);
|
||||
});
|
||||
|
||||
test("insertNode(markdown): a footnote-free insert on a page carrying a footnote still canonicalizes (definitions empty)", async () => {
|
||||
// The page has an existing footnote (ref + tail def). Inserting a footnote-free
|
||||
// fragment keeps the reference alive, so the definition stays — but the write
|
||||
// path must still run canonicalization (definitions=[]), producing exactly one
|
||||
// tail list with the reference/definition ids in sync.
|
||||
const { state, baseURL } = await spawnCollabStack(seedOrphanFootnote());
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
const res = await client.insertNode(
|
||||
PAGE,
|
||||
{ markdown: "unrelated one\n\nunrelated two" },
|
||||
{ position: "after", anchorNodeId: "p1" },
|
||||
);
|
||||
assert.equal(res.success, true);
|
||||
|
||||
const doc = state.lastDoc;
|
||||
assert.equal(findAll(doc, "footnoteReference").length, 1, "the existing reference survives");
|
||||
assert.equal(findAll(doc, "footnotesList").length, 1, "exactly one tail list");
|
||||
const defs = findAll(doc, "footnoteDefinition");
|
||||
assert.equal(defs.length, 1, "the definition is kept (still referenced)");
|
||||
assert.equal(findAll(doc, "footnoteReference")[0].attrs.id, defs[0].attrs.id);
|
||||
});
|
||||
|
||||
// Collect every TOP-LEVEL block id in a doc (the invariant the splice dedup
|
||||
// guarantees is page-wide top-level uniqueness).
|
||||
function topLevelIds(doc) {
|
||||
return doc.content
|
||||
.map((b) => b?.attrs?.id)
|
||||
.filter((id) => id != null);
|
||||
}
|
||||
|
||||
test("patchNode(markdown): a 1->N splice yields page-wide UNIQUE top-level block ids", async () => {
|
||||
const before = seed3();
|
||||
const { state, baseURL } = await spawnCollabStack(before);
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
await client.patchNode(PAGE, "target-id", {
|
||||
markdown: "one\n\ntwo\n\nthree",
|
||||
});
|
||||
|
||||
const ids = topLevelIds(state.lastDoc);
|
||||
assert.equal(new Set(ids).size, ids.length, "all top-level block ids are unique");
|
||||
// The target id is still present (threaded onto the first block).
|
||||
assert.ok(ids.includes("target-id"), "the first block still inherits target-id");
|
||||
});
|
||||
|
||||
test("insertNode(markdown): inserting multiple blocks yields page-wide UNIQUE top-level block ids", async () => {
|
||||
const before = seed3();
|
||||
const { state, baseURL } = await spawnCollabStack(before);
|
||||
const client = new DocmostClient(baseURL, "e@x.com", "pw");
|
||||
|
||||
await client.insertNode(
|
||||
PAGE,
|
||||
{ markdown: "alpha\n\nbeta\n\ngamma" },
|
||||
{ position: "after", anchorNodeId: "before-id" },
|
||||
);
|
||||
|
||||
const ids = topLevelIds(state.lastDoc);
|
||||
assert.equal(new Set(ids).size, ids.length, "all top-level block ids are unique");
|
||||
});
|
||||
@@ -155,7 +155,7 @@ test("listSidebarPages terminates (no dups) when the server ignores the cursor",
|
||||
// -----------------------------------------------------------------------------
|
||||
// 3a) enumerateSpacePages happy path: a SINGLE /pages/tree request.
|
||||
// -----------------------------------------------------------------------------
|
||||
test("enumerateSpacePages (via list_pages tree) uses one /pages/tree request", async () => {
|
||||
test("enumerateSpacePages (via listPages tree) uses one /pages/tree request", async () => {
|
||||
let treeRequests = 0;
|
||||
let sidebarRequests = 0;
|
||||
let treeBody = null;
|
||||
@@ -184,7 +184,7 @@ test("enumerateSpacePages (via list_pages tree) uses one /pages/tree request", a
|
||||
});
|
||||
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
// list_pages tree:true -> enumerateSpacePages(spaceId) -> buildPageTree.
|
||||
// listPages tree:true -> enumerateSpacePages(spaceId) -> buildPageTree.
|
||||
const tree = await client.listPages("space-1", 50, true);
|
||||
|
||||
assert.equal(treeRequests, 1, "exactly one /pages/tree request for the space");
|
||||
@@ -375,7 +375,7 @@ test("listComments terminates (no dups) when the server ignores the cursor", asy
|
||||
});
|
||||
|
||||
// -----------------------------------------------------------------------------
|
||||
// 4) check_new_comments subtree: the root is included in scope WITHOUT a
|
||||
// 4) checkNewComments subtree: the root is included in scope WITHOUT a
|
||||
// separate getPageRaw (/pages/info) request for the parent.
|
||||
// -----------------------------------------------------------------------------
|
||||
test("checkNewComments subtree includes the root without a separate getPageRaw", async () => {
|
||||
|
||||
@@ -244,7 +244,7 @@ test("tableInsertRow with a slugId opens the collab doc by the resolved UUID (#2
|
||||
);
|
||||
});
|
||||
|
||||
test("the generic mutate (insert_footnote) with a slugId opens by the resolved UUID (#260)", async () => {
|
||||
test("the generic mutate (insertFootnote) with a slugId opens by the resolved UUID (#260)", async () => {
|
||||
const { state, baseURL } = await spawnCollabStack();
|
||||
const client = new DocmostClient(baseURL, "user@example.com", "pw");
|
||||
|
||||
@@ -254,7 +254,7 @@ test("the generic mutate (insert_footnote) with a slugId opens by the resolved U
|
||||
assert.deepEqual(
|
||||
state.docNames,
|
||||
[`page.${UUID}`],
|
||||
"insert_footnote (via the mutatePage seam) must open the collab doc by UUID",
|
||||
"insertFootnote (via the mutatePage seam) must open the collab doc by UUID",
|
||||
);
|
||||
});
|
||||
|
||||
@@ -372,7 +372,11 @@ test("replaceImage opens by the resolved UUID AND keys its page lock by that UUI
|
||||
// single flush. This proves the flush actually executes queued callbacks, so
|
||||
// probeRan === false above means "blocked", not "the flush never ran anyone".
|
||||
let freeRan = false;
|
||||
const freeDone = withPageLock(`page.free-${UUID}`, async () => {
|
||||
// A DIFFERENT canonical UUID (unrelated to the page under test). withPageLock
|
||||
// now asserts its key is a canonical UUID (#449), so the "free" probe key must
|
||||
// also be a valid — but distinct — UUID, not a synthetic label.
|
||||
const FREE_UUID = "99999999-9999-4999-8999-999999999999";
|
||||
const freeDone = withPageLock(FREE_UUID, async () => {
|
||||
freeRan = true;
|
||||
});
|
||||
await new Promise((r) => setImmediate(r));
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
// Server round-trip test for the stash_page MCP tool result shape. The in-app
|
||||
// Server round-trip test for the stashPage MCP tool result shape. The in-app
|
||||
// path returns the full documented `{ uri, size, sha256, images }` object, but
|
||||
// the MCP transport must deliver the SAME shape: a resource_link (primary
|
||||
// payload) PLUS a `structuredContent` mirror carrying sha256 + image counts.
|
||||
@@ -107,7 +107,7 @@ async function buildBaseURL() {
|
||||
});
|
||||
}
|
||||
|
||||
test("stash_page MCP tool returns a resource_link AND a structuredContent mirror", async () => {
|
||||
test("stashPage MCP tool returns a resource_link AND a structuredContent mirror", async () => {
|
||||
const baseURL = await buildBaseURL();
|
||||
const sandbox = makeSandbox();
|
||||
const server = createDocmostMcpServer({
|
||||
@@ -124,7 +124,7 @@ test("stash_page MCP tool returns a resource_link AND a structuredContent mirror
|
||||
|
||||
try {
|
||||
const res = await client.callTool({
|
||||
name: "stash_page",
|
||||
name: "stashPage",
|
||||
arguments: { pageId: "page-1" },
|
||||
});
|
||||
|
||||
|
||||
@@ -20,11 +20,11 @@ import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
|
||||
|
||||
import { createDocmostMcpServer } from "../../build/index.js";
|
||||
|
||||
// The tool we drive. get_workspace has NO input schema, so protocol-level input
|
||||
// The tool we drive. getWorkspace has NO input schema, so protocol-level input
|
||||
// validation cannot short-circuit before the handler runs — the wrapped handler
|
||||
// is guaranteed to execute (and then fail on the unreachable backend, which is
|
||||
// exactly what we want: the wrapper times in a finally on throw too).
|
||||
const TOOL_NAME = "get_workspace";
|
||||
const TOOL_NAME = "getWorkspace";
|
||||
|
||||
test("the factory's registerTool monkeypatch times a live tool call and labels it with the registration name", async () => {
|
||||
const calls = [];
|
||||
|
||||
@@ -323,7 +323,9 @@ test("MCP_COLLAB_SESSION_IDLE_MS=0 disables the cache (legacy provider-per-op)",
|
||||
});
|
||||
|
||||
test("replaceImage-shaped flow: acquire under an EXTERNAL page lock does not deadlock and reuses one session", async () => {
|
||||
const pageId = "page-lock";
|
||||
// withPageLock now asserts a canonical UUID key (#449); this flow takes the
|
||||
// real page lock (mirroring replaceImage), so the key must be a valid UUID.
|
||||
const pageId = "77777777-7777-4777-8777-777777777777";
|
||||
// Mirror replaceImage: hold ONE withPageLock across scan (read-only) + write,
|
||||
// each going through the non-locking acquireCollabSession.
|
||||
const result = await withPageLock(pageId, async () => {
|
||||
|
||||
@@ -152,7 +152,7 @@ test("tautological comment tools are excluded and never probe", async () => {
|
||||
const { comments, probeCalls, tracker } = makeWorld();
|
||||
tracker.noteWorkingPage("p1");
|
||||
comments.push({ createdAt: 9_999_999 });
|
||||
for (const name of ["listComments", "list_comments", "checkNewComments", "createComment"]) {
|
||||
for (const name of ["listComments", "listComments", "checkNewComments", "createComment"]) {
|
||||
assert.equal(await tracker.maybeSignal(name), null);
|
||||
}
|
||||
assert.equal(probeCalls.length, 0);
|
||||
@@ -188,7 +188,7 @@ function fakeTracker({ line }) {
|
||||
noteWorkingPage: (p) => events.push(["note", p]),
|
||||
advanceWatermark: () => events.push(["advance"]),
|
||||
isExcludedTool: (n) =>
|
||||
new Set(["listComments", "list_comments"]).has(n),
|
||||
new Set(["listComments", "listComments"]).has(n),
|
||||
maybeSignal: async () => line,
|
||||
};
|
||||
}
|
||||
@@ -221,7 +221,7 @@ test("withCommentSignal: appends ONE extra text element when signalled", async (
|
||||
test("withCommentSignal: excluded tool advances the watermark and does not append", async () => {
|
||||
const tracker = fakeTracker({ line: "SHOULD-NOT-APPEAR" });
|
||||
const original = { content: [{ type: "text", text: "comments" }] };
|
||||
const wrapped = withCommentSignal("list_comments", async () => original, tracker);
|
||||
const wrapped = withCommentSignal("listComments", async () => original, tracker);
|
||||
const result = await wrapped({ pageId: "p1" });
|
||||
assert.equal(result, original); // unchanged
|
||||
assert.ok(tracker.events.some((e) => e[0] === "advance"));
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
// Issue #464 — prove the size guard SKIPS the recreateTransform pipeline over
|
||||
// the cap, not merely that it returns "coarse". node:test's mock.module needs an
|
||||
// experimental flag the suite does not pass, so instead of a module spy we use a
|
||||
// deterministic BEHAVIORAL proxy that isolates the one variable — the guard:
|
||||
//
|
||||
// Same over-cap pair, run twice:
|
||||
// (a) default caps -> guard trips -> recreateTransform skipped,
|
||||
// (b) caps raised above the doc -> guard OFF -> recreateTransform DOES run.
|
||||
//
|
||||
// The only code path that differs between (a) and (b) is whether
|
||||
// recreateTransform executes. recreateTransform on this pair is O(n²) and takes
|
||||
// SECONDS; the guarded path is a linear coarse diff taking milliseconds. So a
|
||||
// large (a)≪(b) time ratio can ONLY be explained by (a) skipping the transform.
|
||||
// This asserts the skip without depending on mock.module.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { diffDocs } from "../../build/lib/diff.js";
|
||||
|
||||
const t = (text) => ({ type: "text", text });
|
||||
const para = (text) => ({ type: "paragraph", content: text ? [t(text)] : [] });
|
||||
const doc = (children) => ({ type: "doc", content: children });
|
||||
function buildDoc(n, seed) {
|
||||
return doc(
|
||||
Array.from({ length: n }, (_, i) =>
|
||||
para(Array.from({ length: 8 }, (_, w) => `${seed}${i}_${w}`).join(" ")),
|
||||
),
|
||||
);
|
||||
}
|
||||
function clearEnv() {
|
||||
delete process.env.MCP_DIFF_MAX_NODES;
|
||||
delete process.env.MCP_DIFF_MAX_BYTES;
|
||||
}
|
||||
function timed(fn) {
|
||||
const s = performance.now();
|
||||
const out = fn();
|
||||
return { out, ms: performance.now() - s };
|
||||
}
|
||||
|
||||
// A 300-para (~600-node) pair: comfortably over the 150-node default, yet small
|
||||
// enough that the un-guarded recreateTransform still FINISHES (~1-3s) so the
|
||||
// test can time the contrast without hanging.
|
||||
const OLD = buildDoc(300, "a");
|
||||
const NEW = buildDoc(300, "b");
|
||||
|
||||
test("guard skips recreateTransform over-cap (guarded run is far faster than un-guarded)", () => {
|
||||
// (a) Guarded: default caps -> should short-circuit to coarse, near-instant.
|
||||
clearEnv();
|
||||
const guarded = timed(() => diffDocs(OLD, NEW));
|
||||
assert.match(
|
||||
guarded.out.markdown,
|
||||
/coarse block-level diff/,
|
||||
"guarded run must be coarse (guard tripped)",
|
||||
);
|
||||
|
||||
// (b) Un-guarded: raise both caps above the doc so the precise path runs.
|
||||
process.env.MCP_DIFF_MAX_NODES = "1000000";
|
||||
process.env.MCP_DIFF_MAX_BYTES = "100000000";
|
||||
let unguarded;
|
||||
try {
|
||||
unguarded = timed(() => diffDocs(OLD, NEW));
|
||||
} finally {
|
||||
clearEnv();
|
||||
}
|
||||
assert.doesNotMatch(
|
||||
unguarded.out.markdown,
|
||||
/coarse block-level diff/,
|
||||
"with caps raised, the precise recreateTransform path runs",
|
||||
);
|
||||
|
||||
// The precise run executed recreateTransform (O(n²)); the guarded run did not.
|
||||
// Require a large speedup so the ONLY explanation is the skipped transform.
|
||||
assert.ok(
|
||||
guarded.ms * 5 < unguarded.ms,
|
||||
`guarded (${guarded.ms.toFixed(1)}ms) must be >=5x faster than un-guarded ` +
|
||||
`(${unguarded.ms.toFixed(1)}ms); a small gap would mean the transform still ran`,
|
||||
);
|
||||
});
|
||||
|
||||
test("guarded over-cap call stays within the ~200ms event-loop budget", () => {
|
||||
clearEnv();
|
||||
// Best-of-3 to shed GC/JIT noise; the guarded coarse path is a linear walk.
|
||||
let best = Infinity;
|
||||
for (let i = 0; i < 3; i++) best = Math.min(best, timed(() => diffDocs(OLD, NEW)).ms);
|
||||
assert.ok(best < 200, `guarded over-cap diff must be <200ms, was ${best.toFixed(1)}ms`);
|
||||
});
|
||||
@@ -0,0 +1,229 @@
|
||||
// Issue #464 — prod CPU-DoS pre-flight size guard for diffDocs.
|
||||
//
|
||||
// diffDocs synchronously calls recreateTransform (rfc6902) which is O(n·m) in
|
||||
// node count and O(w²) in per-run word count; on a large/heavily-changed doc it
|
||||
// pins the event loop for seconds-to-hours WITHOUT throwing. A pre-flight size
|
||||
// guard routes any doc over MCP_DIFF_MAX_NODES / MCP_DIFF_MAX_BYTES straight to
|
||||
// the coarse fallback (`fellBack:true`), so the sync block stays ~<200ms.
|
||||
//
|
||||
// These tests assert the BEHAVIOR of the guard (fast + coarse-mode + asymmetry +
|
||||
// env knobs). A sibling test (diff-guard-skips-recreate.test.mjs) proves
|
||||
// recreateTransform is skipped over the cap via a behavioral proxy (guarded run
|
||||
// is orders of magnitude faster than the same pair with the caps raised).
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { diffDocs } from "../../build/lib/diff.js";
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Builders
|
||||
// ---------------------------------------------------------------------------
|
||||
const t = (text) => ({ type: "text", text });
|
||||
const para = (text) => ({ type: "paragraph", content: text ? [t(text)] : [] });
|
||||
const doc = (children) => ({ type: "doc", content: children });
|
||||
|
||||
/** A doc of `n` paragraphs whose words are seeded from `seed` (fully changeable). */
|
||||
function buildDoc(n, wordsPerPara, seed) {
|
||||
const blocks = [];
|
||||
for (let i = 0; i < n; i++) {
|
||||
const words = [];
|
||||
for (let w = 0; w < wordsPerPara; w++) words.push(`${seed}${i}_${w}`);
|
||||
blocks.push(para(words.join(" ")));
|
||||
}
|
||||
return doc(blocks);
|
||||
}
|
||||
|
||||
/** Reset the env knobs to their unset default between tests. */
|
||||
function clearEnv() {
|
||||
delete process.env.MCP_DIFF_MAX_NODES;
|
||||
delete process.env.MCP_DIFF_MAX_BYTES;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Over-threshold (by node count) -> FAST + coarse mode.
|
||||
// A fully re-written 600-para doc is the worst case that drove the incident;
|
||||
// with the guard it must return in well under the ~200ms budget and in coarse
|
||||
// mode. Without the guard this single call takes multiple SECONDS.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("over-threshold doc falls back to coarse mode and returns fast", () => {
|
||||
clearEnv();
|
||||
// 600 paragraphs -> ~1200 nodes, far over the 150-node default.
|
||||
const oldDoc = buildDoc(600, 8, "a");
|
||||
const newDoc = buildDoc(600, 8, "b");
|
||||
|
||||
const start = performance.now();
|
||||
const r = diffDocs(oldDoc, newDoc);
|
||||
const elapsed = performance.now() - start;
|
||||
|
||||
// Coarse mode is signalled in the markdown note (fellBack path).
|
||||
assert.match(
|
||||
r.markdown,
|
||||
/coarse block-level diff/,
|
||||
"over-threshold pair must use the coarse fallback",
|
||||
);
|
||||
// Budget: the guard makes this near-instant. Generous 1s ceiling to avoid CI
|
||||
// flake while still being ~10x under the multi-second un-guarded cost.
|
||||
assert.ok(
|
||||
elapsed < 1000,
|
||||
`expected fast coarse fallback, took ${elapsed.toFixed(0)}ms`,
|
||||
);
|
||||
// Coarse diff still detects the wholesale change.
|
||||
assert.ok(r.summary.inserted > 0 || r.summary.deleted > 0, "reports changes");
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Under-threshold (small) doc -> precise diff, NOT coarse mode. No regression.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("under-threshold doc uses the precise diff (no fallback note)", () => {
|
||||
clearEnv();
|
||||
const oldDoc = doc([para("Hello world")]);
|
||||
const newDoc = doc([para("Hello brave world")]);
|
||||
const r = diffDocs(oldDoc, newDoc);
|
||||
|
||||
assert.doesNotMatch(
|
||||
r.markdown,
|
||||
/coarse block-level diff/,
|
||||
"a small doc must take the precise path",
|
||||
);
|
||||
// Precise word diff finds exactly the inserted word.
|
||||
const ins = r.changes.find((c) => c.op === "insert");
|
||||
assert.ok(ins && /brave/.test(ins.text), "precise diff isolates the inserted word");
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Asymmetry: a small NEW doc vs a huge OLD doc (and vice versa) still explodes
|
||||
// rfc6902, so max(old,new) must trip the guard in BOTH directions.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("asymmetric pair (huge old, tiny new) falls back to coarse", () => {
|
||||
clearEnv();
|
||||
const hugeOld = buildDoc(600, 8, "a");
|
||||
const tinyNew = doc([para("just one line")]);
|
||||
const r = diffDocs(hugeOld, tinyNew);
|
||||
assert.match(r.markdown, /coarse block-level diff/, "huge-old side must trip the guard");
|
||||
});
|
||||
|
||||
test("asymmetric pair (tiny old, huge new) falls back to coarse", () => {
|
||||
clearEnv();
|
||||
const tinyOld = doc([para("just one line")]);
|
||||
const hugeNew = buildDoc(600, 8, "b");
|
||||
const r = diffDocs(tinyOld, hugeNew);
|
||||
assert.match(r.markdown, /coarse block-level diff/, "huge-new side must trip the guard");
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Byte axis: a FEW nodes but a very large serialized size (long text runs) is
|
||||
// dangerous too (per-run word diff is O(words²)), so the byte cap must trip
|
||||
// independently of the node count.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("node-light but byte-heavy doc falls back on the byte cap", () => {
|
||||
clearEnv();
|
||||
// 5 paragraphs (~11 nodes, well under the node cap) but each a very long run,
|
||||
// pushing the serialized size far over the 12 KiB byte default.
|
||||
const bigRun = (seed) =>
|
||||
doc(
|
||||
Array.from({ length: 5 }, (_, i) =>
|
||||
para(Array.from({ length: 800 }, (_, w) => `${seed}${i}_${w}`).join(" ")),
|
||||
),
|
||||
);
|
||||
const oldDoc = bigRun("a");
|
||||
const newDoc = bigRun("b");
|
||||
// Sanity: node count is under the default node cap, so ONLY the byte cap can
|
||||
// be what trips the guard here.
|
||||
const nodeCount = (d) => {
|
||||
let n = 0;
|
||||
const v = (x) => {
|
||||
if (!x || typeof x !== "object") return;
|
||||
n++;
|
||||
if (Array.isArray(x.content)) for (const c of x.content) v(c);
|
||||
};
|
||||
v(d);
|
||||
return n;
|
||||
};
|
||||
assert.ok(nodeCount(oldDoc) < 150, "node count is under the node cap");
|
||||
assert.ok(JSON.stringify(oldDoc).length > 12 * 1024, "serialized size is over the byte cap");
|
||||
|
||||
const r = diffDocs(oldDoc, newDoc);
|
||||
assert.match(r.markdown, /coarse block-level diff/, "byte cap must trip independently");
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Env override: a very low MCP_DIFF_MAX_NODES forces fallback on a tiny doc,
|
||||
// proving the knob is read fresh and actually gates the diff.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("MCP_DIFF_MAX_NODES override forces fallback on a small doc", () => {
|
||||
clearEnv();
|
||||
const oldDoc = doc([para("Hello world")]);
|
||||
const newDoc = doc([para("Hello brave world")]);
|
||||
|
||||
// Baseline: default caps -> precise diff.
|
||||
assert.doesNotMatch(diffDocs(oldDoc, newDoc).markdown, /coarse block-level diff/);
|
||||
|
||||
// Knob set absurdly low -> even this 4-node doc trips the guard.
|
||||
process.env.MCP_DIFF_MAX_NODES = "1";
|
||||
try {
|
||||
const r = diffDocs(oldDoc, newDoc);
|
||||
assert.match(r.markdown, /coarse block-level diff/, "low node cap forces fallback");
|
||||
} finally {
|
||||
clearEnv();
|
||||
}
|
||||
});
|
||||
|
||||
test("MCP_DIFF_MAX_BYTES override forces fallback on a small doc", () => {
|
||||
clearEnv();
|
||||
const oldDoc = doc([para("Hello world")]);
|
||||
const newDoc = doc([para("Hello brave world")]);
|
||||
|
||||
process.env.MCP_DIFF_MAX_BYTES = "1";
|
||||
try {
|
||||
const r = diffDocs(oldDoc, newDoc);
|
||||
assert.match(r.markdown, /coarse block-level diff/, "low byte cap forces fallback");
|
||||
} finally {
|
||||
clearEnv();
|
||||
}
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Garbage / unset env values fall back to the DEFAULT (the guard can never be
|
||||
// accidentally disabled by a malformed knob). A small doc must still diff
|
||||
// precisely under a garbage cap.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("garbage env values fall back to the default cap (guard not disabled)", () => {
|
||||
clearEnv();
|
||||
const oldDoc = doc([para("Hello world")]);
|
||||
const newDoc = doc([para("Hello brave world")]);
|
||||
|
||||
for (const bad of ["not-a-number", "0", "-5", "", "NaN", "1e999"]) {
|
||||
process.env.MCP_DIFF_MAX_NODES = bad;
|
||||
process.env.MCP_DIFF_MAX_BYTES = bad;
|
||||
// Under the DEFAULT caps this small doc is precise (garbage did not raise
|
||||
// OR disable the cap). "1e999" -> parseInt yields 1 (finite) which is a
|
||||
// valid low cap and would fall back; exclude that from the precise check.
|
||||
const r = diffDocs(oldDoc, newDoc);
|
||||
if (bad === "1e999") {
|
||||
// parseInt("1e999",10) === 1 -> a legit low cap -> fallback. Guard active.
|
||||
assert.match(r.markdown, /coarse block-level diff/);
|
||||
} else {
|
||||
assert.doesNotMatch(
|
||||
r.markdown,
|
||||
/coarse block-level diff/,
|
||||
`garbage value ${JSON.stringify(bad)} must fall back to the default cap`,
|
||||
);
|
||||
}
|
||||
}
|
||||
clearEnv();
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// A large doc that trips the guard must still return the correct INTEGRITY
|
||||
// counts (computeIntegrity runs before the diff and is unaffected by fallback).
|
||||
// ---------------------------------------------------------------------------
|
||||
test("integrity counts are still correct on a guard-tripped (coarse) doc", () => {
|
||||
clearEnv();
|
||||
const image = { type: "image", attrs: { src: "/api/files/a.png" } };
|
||||
const oldDoc = doc([image, ...buildDoc(600, 8, "a").content]);
|
||||
const newDoc = doc([...buildDoc(600, 8, "b").content]); // image removed
|
||||
|
||||
const r = diffDocs(oldDoc, newDoc);
|
||||
assert.match(r.markdown, /coarse block-level diff/, "large pair fell back");
|
||||
assert.deepEqual(r.integrity.images, [1, 0], "integrity is computed regardless of fallback");
|
||||
});
|
||||
@@ -114,7 +114,7 @@ test("summarizeChange treats a key-order-only difference as no change", () => {
|
||||
// (v) CRITICAL: a structural change that touches no text/marks — adding an
|
||||
// image node (images 0 -> 1) — must report changed:true and surface the
|
||||
// integrity delta in structure + summary, closing the verify blind spot for
|
||||
// insert_image / delete_node on structural nodes.
|
||||
// insertImage / deleteNode on structural nodes.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("summarizeChange surfaces an image-count change (0->1)", () => {
|
||||
const before = doc(para(t("caption")));
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
// Unit tests for the drawioEditCells operations (issue #425, acceptance #4):
|
||||
// add / update / delete applied to the parsed model, with a cascade delete that
|
||||
// removes container children AND every connected edge.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { applyCellOps, CellOpsError } from "../../build/lib/drawio-cell-ops.js";
|
||||
import { parseCells } from "../../build/lib/drawio-xml.js";
|
||||
|
||||
const MODEL =
|
||||
"<mxGraphModel><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/>" +
|
||||
'<mxCell id="grp" value="G" style="container=1;fillColor=none;" vertex="1" parent="1">' +
|
||||
'<mxGeometry x="0" y="0" width="300" height="200" as="geometry"/></mxCell>' +
|
||||
'<mxCell id="c1" value="Child1" style="rounded=1;" vertex="1" parent="grp">' +
|
||||
'<mxGeometry x="10" y="10" width="80" height="40" as="geometry"/></mxCell>' +
|
||||
'<mxCell id="c2" value="Child2" style="rounded=1;" vertex="1" parent="grp">' +
|
||||
'<mxGeometry x="10" y="80" width="80" height="40" as="geometry"/></mxCell>' +
|
||||
'<mxCell id="out" value="Outside" style="rounded=1;" vertex="1" parent="1">' +
|
||||
'<mxGeometry x="400" y="10" width="80" height="40" as="geometry"/></mxCell>' +
|
||||
'<mxCell id="e1" style="" edge="1" parent="1" source="c1" target="out">' +
|
||||
'<mxGeometry relative="1" as="geometry"/></mxCell>' +
|
||||
'<mxCell id="e2" style="" edge="1" parent="1" source="out" target="c2">' +
|
||||
'<mxGeometry relative="1" as="geometry"/></mxCell>' +
|
||||
"</root></mxGraphModel>";
|
||||
|
||||
const ids = (xml) =>
|
||||
parseCells(xml)
|
||||
.filter((c) => c.id !== "0" && c.id !== "1")
|
||||
.map((c) => c.id)
|
||||
.sort();
|
||||
|
||||
test("update changes ONLY the targeted cell", () => {
|
||||
const out = applyCellOps(MODEL, [
|
||||
{
|
||||
op: "update",
|
||||
cellId: "c1",
|
||||
xml:
|
||||
'<mxCell id="c1" value="Renamed" style="rounded=1;" vertex="1" parent="grp">' +
|
||||
'<mxGeometry x="10" y="10" width="80" height="40" as="geometry"/></mxCell>',
|
||||
},
|
||||
]);
|
||||
const cells = parseCells(out);
|
||||
assert.equal(cells.find((c) => c.id === "c1").value, "Renamed");
|
||||
// Every OTHER cell is untouched.
|
||||
assert.equal(cells.find((c) => c.id === "c2").value, "Child2");
|
||||
assert.equal(cells.find((c) => c.id === "out").value, "Outside");
|
||||
assert.deepEqual(ids(out), ids(MODEL));
|
||||
});
|
||||
|
||||
test("delete of a container removes its children AND the connected edges", () => {
|
||||
const out = applyCellOps(MODEL, [{ op: "delete", cellId: "grp" }]);
|
||||
// grp + c1 + c2 gone (cascade to children); e1 (c1->out) and e2 (out->c2)
|
||||
// gone (cascade to connected edges); "out" survives.
|
||||
assert.deepEqual(ids(out), ["out"]);
|
||||
});
|
||||
|
||||
test("delete of a leaf only cascades to its connected edges, not siblings", () => {
|
||||
const out = applyCellOps(MODEL, [{ op: "delete", cellId: "c1" }]);
|
||||
// c1 gone + e1 (c1->out) gone; c2, out, grp, e2 survive.
|
||||
assert.deepEqual(ids(out), ["c2", "e2", "grp", "out"]);
|
||||
});
|
||||
|
||||
test("add appends a new cell", () => {
|
||||
const out = applyCellOps(MODEL, [
|
||||
{
|
||||
op: "add",
|
||||
xml:
|
||||
'<mxCell id="new1" value="N" style="rounded=1;" vertex="1" parent="1">' +
|
||||
'<mxGeometry x="500" y="10" width="80" height="40" as="geometry"/></mxCell>',
|
||||
},
|
||||
]);
|
||||
assert.ok(parseCells(out).some((c) => c.id === "new1"));
|
||||
});
|
||||
|
||||
test("delete never removes the sentinels", () => {
|
||||
const out = applyCellOps(MODEL, [{ op: "delete", cellId: "out" }]);
|
||||
const cells = parseCells(out);
|
||||
assert.ok(cells.some((c) => c.id === "0"));
|
||||
assert.ok(cells.some((c) => c.id === "1"));
|
||||
});
|
||||
|
||||
test("errors: unknown update/delete target, duplicate add id, id mismatch", () => {
|
||||
assert.throws(
|
||||
() => applyCellOps(MODEL, [{ op: "update", cellId: "ghost", xml: '<mxCell id="ghost"/>' }]),
|
||||
/does not exist/,
|
||||
);
|
||||
assert.throws(
|
||||
() => applyCellOps(MODEL, [{ op: "delete", cellId: "ghost" }]),
|
||||
/does not exist/,
|
||||
);
|
||||
assert.throws(
|
||||
() => applyCellOps(MODEL, [{ op: "add", xml: '<mxCell id="c1"/>' }]),
|
||||
/already exists/,
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
applyCellOps(MODEL, [
|
||||
{ op: "update", cellId: "c1", xml: '<mxCell id="c2"/>' },
|
||||
]),
|
||||
/ids are stable/,
|
||||
);
|
||||
assert.throws(() => applyCellOps(MODEL, []), CellOpsError);
|
||||
});
|
||||
|
||||
test("an add op with two cells or a missing id is rejected", () => {
|
||||
assert.throws(
|
||||
() => applyCellOps(MODEL, [{ op: "add", xml: '<mxCell id="a"/><mxCell id="b"/>' }]),
|
||||
/exactly one <mxCell>/,
|
||||
);
|
||||
assert.throws(
|
||||
() => applyCellOps(MODEL, [{ op: "add", xml: '<mxCell value="x"/>' }]),
|
||||
/missing an id/,
|
||||
);
|
||||
});
|
||||
|
||||
// --- SUGGESTION #5: sentinel cells are protected from delete ------------------
|
||||
|
||||
test("delete targeting a sentinel id is rejected (no wipe of the diagram body)", () => {
|
||||
for (const sid of ["0", "1"]) {
|
||||
assert.throws(
|
||||
() => applyCellOps(MODEL, [{ op: "delete", cellId: sid }]),
|
||||
(e) => e instanceof CellOpsError && /cannot delete sentinel cell/.test(e.message),
|
||||
);
|
||||
}
|
||||
// A normal delete still works and the sentinels remain intact.
|
||||
const out = applyCellOps(MODEL, [{ op: "delete", cellId: "out" }]);
|
||||
const remaining = parseCells(out).map((c) => c.id);
|
||||
assert.ok(remaining.includes("0") && remaining.includes("1"), "sentinels survive");
|
||||
});
|
||||
@@ -0,0 +1,380 @@
|
||||
// Unit tests for the drawioFromGraph pipeline (issue #425, stage 3): the
|
||||
// semantic graph -> ELK -> linter-clean XML assembler, plus the layout hints
|
||||
// (pinned / sameLayerAs / layer) and incremental layout. Pure — no client, no
|
||||
// network — so they run under `node --test` against the built lib.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
buildFromGraph,
|
||||
validateGraph,
|
||||
resolveNodeStyle,
|
||||
GraphValidationError,
|
||||
} from "../../build/lib/drawio-graph.js";
|
||||
import { getPreset } from "../../build/lib/drawio-presets.js";
|
||||
import {
|
||||
prepareModel,
|
||||
parseCells,
|
||||
computeQualityWarnings,
|
||||
} from "../../build/lib/drawio-xml.js";
|
||||
|
||||
function cellsOf(xml) {
|
||||
return parseCells(xml).filter((c) => c.id !== "0" && c.id !== "1");
|
||||
}
|
||||
function byId(xml) {
|
||||
const m = new Map();
|
||||
for (const c of parseCells(xml)) m.set(c.id, c);
|
||||
return m;
|
||||
}
|
||||
|
||||
// --- Acceptance #1: 15-node graph, 2 nested groups, AWS icons ---------------
|
||||
|
||||
test("from_graph: 15+ nodes, 2 nested groups, AWS icons -> 0 lint errors, 0 warnings, all icons resolve", async () => {
|
||||
const awsIcons = [
|
||||
"aws:lambda", "aws:dynamodb", "aws:api_gateway", "aws:s3", "aws:sqs",
|
||||
"aws:sns", "aws:ec2", "aws:rds", "aws:cloudfront", "aws:elasticache",
|
||||
"aws:kinesis", "aws:cognito", "aws:secrets_manager", "aws:cloudwatch",
|
||||
];
|
||||
const kinds = [
|
||||
"service", "db", "gateway", "service", "queue", "queue", "service", "db",
|
||||
"gateway", "db", "queue", "security", "security", "external",
|
||||
];
|
||||
const nodes = [];
|
||||
for (let i = 0; i < 14; i++) {
|
||||
nodes.push({
|
||||
id: "n" + i,
|
||||
label: "Node " + i,
|
||||
kind: kinds[i],
|
||||
icon: awsIcons[i],
|
||||
group: i < 6 ? "sub1" : i < 10 ? "vpc1" : undefined,
|
||||
});
|
||||
}
|
||||
nodes.push({ id: "ext1", label: "External Service", kind: "external" });
|
||||
const graph = {
|
||||
nodes,
|
||||
groups: [
|
||||
{ id: "vpc1", label: "VPC 10.0.0.0/16", kind: "vpc" },
|
||||
{ id: "sub1", label: "Private Subnet", kind: "subnet", group: "vpc1" }, // NESTED
|
||||
],
|
||||
edges: [
|
||||
{ from: "n0", to: "n1", kind: "sync" },
|
||||
{ from: "n1", to: "n2", kind: "async" },
|
||||
{ from: "n2", to: "n3" },
|
||||
{ from: "n3", to: "n7", kind: "sync" },
|
||||
{ from: "n7", to: "n8" },
|
||||
{ from: "n8", to: "ext1", kind: "error" },
|
||||
{ from: "n4", to: "n5" },
|
||||
{ from: "n10", to: "n11" },
|
||||
],
|
||||
direction: "LR",
|
||||
preset: "default",
|
||||
};
|
||||
const r = await buildFromGraph(graph, "full");
|
||||
|
||||
assert.equal(graph.nodes.length >= 15, true, "at least 15 nodes");
|
||||
// All icons resolved — NO empty squares.
|
||||
assert.equal(r.iconsMissing.length, 0, `unresolved icons: ${r.iconsMissing}`);
|
||||
assert.equal(r.iconsResolved, 14);
|
||||
|
||||
// 0 lint errors + 0 quality-warnings.
|
||||
const prepared = prepareModel(r.modelXml);
|
||||
assert.equal(prepared.warnings.length, 0, prepared.warnings.join("\n"));
|
||||
|
||||
// Groups are TRANSPARENT containers.
|
||||
const cells = byId(r.modelXml);
|
||||
for (const gid of ["vpc1", "sub1"]) {
|
||||
const g = cells.get(gid);
|
||||
assert.ok(g, `${gid} present`);
|
||||
assert.equal(g.styleMap.container, "1", `${gid} container=1`);
|
||||
assert.equal(g.styleMap.fillColor, "none", `${gid} fillColor=none`);
|
||||
assert.equal(g.styleMap.dropTarget, "1", `${gid} dropTarget=1`);
|
||||
}
|
||||
// The nested group sub1's parent IS vpc1 (nesting honoured).
|
||||
assert.equal(cells.get("sub1").parent, "vpc1");
|
||||
// A grouped node's parent is its group (relative coords).
|
||||
assert.equal(cells.get("n0").parent, "sub1");
|
||||
});
|
||||
|
||||
test("from_graph: an UNKNOWN icon degrades to a labelled generic shape (never empty)", async () => {
|
||||
const graph = {
|
||||
nodes: [
|
||||
{ id: "a", label: "Mystery", kind: "service", icon: "not:a-real-icon-xyz" },
|
||||
{ id: "b", label: "Plain", kind: "db" },
|
||||
],
|
||||
edges: [{ from: "a", to: "b" }],
|
||||
};
|
||||
const r = await buildFromGraph(graph, "full");
|
||||
const cells = byId(r.modelXml);
|
||||
// The node still carries its label and a real (non-empty) style with a fill.
|
||||
assert.equal(cells.get("a").value, "Mystery");
|
||||
assert.match(cells.get("a").style, /fillColor=/);
|
||||
// It is reported as missing so the model can see the degradation.
|
||||
assert.ok(r.iconsMissing.includes("a"));
|
||||
});
|
||||
|
||||
// --- Acceptance #2: hints -----------------------------------------------------
|
||||
|
||||
test("from_graph: a pinned node stays at its exact coordinates", async () => {
|
||||
const graph = {
|
||||
nodes: [
|
||||
{ id: "a", label: "A", pinned: { x: 40, y: 900 } },
|
||||
{ id: "b", label: "B" },
|
||||
{ id: "c", label: "C" },
|
||||
],
|
||||
edges: [{ from: "a", to: "b" }, { from: "b", to: "c" }],
|
||||
};
|
||||
const a = byId((await buildFromGraph(graph, "full")).modelXml).get("a");
|
||||
assert.equal(a.geometry.x, 40);
|
||||
assert.equal(a.geometry.y, 900);
|
||||
});
|
||||
|
||||
test("from_graph: a sameLayerAs pair lands in the same layer (equal layer-axis coord)", async () => {
|
||||
const graph = {
|
||||
nodes: [
|
||||
{ id: "x", label: "X" },
|
||||
{ id: "y", label: "Y", sameLayerAs: "x" },
|
||||
{ id: "z", label: "Z" },
|
||||
],
|
||||
edges: [{ from: "z", to: "x" }, { from: "z", to: "y" }],
|
||||
direction: "LR", // layer axis = x
|
||||
};
|
||||
const cells = byId((await buildFromGraph(graph, "full")).modelXml);
|
||||
assert.equal(cells.get("x").geometry.x, cells.get("y").geometry.x);
|
||||
});
|
||||
|
||||
test("from_graph: an explicit layer index co-aligns nodes on the layer axis", async () => {
|
||||
const graph = {
|
||||
nodes: [
|
||||
{ id: "p", label: "P", layer: 0 },
|
||||
{ id: "q", label: "Q", layer: 0 },
|
||||
{ id: "r", label: "R", layer: 1 },
|
||||
],
|
||||
edges: [{ from: "p", to: "r" }],
|
||||
direction: "LR",
|
||||
};
|
||||
const cells = byId((await buildFromGraph(graph, "full")).modelXml);
|
||||
assert.equal(cells.get("p").geometry.x, cells.get("q").geometry.x);
|
||||
});
|
||||
|
||||
test("from_graph: TB direction snaps sameLayerAs on the Y axis", async () => {
|
||||
const graph = {
|
||||
nodes: [
|
||||
{ id: "x", label: "X" },
|
||||
{ id: "y", label: "Y", sameLayerAs: "x" },
|
||||
{ id: "z", label: "Z" },
|
||||
],
|
||||
edges: [{ from: "z", to: "x" }, { from: "z", to: "y" }],
|
||||
direction: "TB", // layer axis = y
|
||||
};
|
||||
const cells = byId((await buildFromGraph(graph, "full")).modelXml);
|
||||
assert.equal(cells.get("x").geometry.y, cells.get("y").geometry.y);
|
||||
});
|
||||
|
||||
// --- Acceptance #3: incremental never moves existing cells --------------------
|
||||
|
||||
test("from_graph incremental: adding a node does NOT move existing cells", async () => {
|
||||
const existing = new Map([
|
||||
["a", { x: 100, y: 100 }],
|
||||
["b", { x: 400, y: 100 }],
|
||||
]);
|
||||
const graph = {
|
||||
nodes: [
|
||||
{ id: "a", label: "A" },
|
||||
{ id: "b", label: "B" },
|
||||
{ id: "cnew", label: "New" },
|
||||
],
|
||||
edges: [{ from: "a", to: "b" }, { from: "b", to: "cnew" }],
|
||||
};
|
||||
const cells = byId((await buildFromGraph(graph, "incremental", existing)).modelXml);
|
||||
assert.deepEqual(
|
||||
[cells.get("a").geometry.x, cells.get("a").geometry.y],
|
||||
[100, 100],
|
||||
);
|
||||
assert.deepEqual(
|
||||
[cells.get("b").geometry.x, cells.get("b").geometry.y],
|
||||
[400, 100],
|
||||
);
|
||||
// The new node was placed and does not overlap the frozen block.
|
||||
const cn = cells.get("cnew");
|
||||
assert.ok(cn.geometry.y >= 200, "new node placed clear of the existing block");
|
||||
});
|
||||
|
||||
// --- direction honoured -------------------------------------------------------
|
||||
|
||||
test("from_graph: LR vs RL flip the layout axis order", async () => {
|
||||
const mk = (dir) => ({
|
||||
nodes: [{ id: "s", label: "S" }, { id: "t", label: "T" }],
|
||||
edges: [{ from: "s", to: "t" }],
|
||||
direction: dir,
|
||||
});
|
||||
const lr = byId((await buildFromGraph(mk("LR"), "full")).modelXml);
|
||||
// In LR the target sits to the RIGHT of the source.
|
||||
assert.ok(lr.get("t").geometry.x > lr.get("s").geometry.x, "LR: t right of s");
|
||||
const rl = byId((await buildFromGraph(mk("RL"), "full")).modelXml);
|
||||
assert.ok(rl.get("t").geometry.x < rl.get("s").geometry.x, "RL: t left of s");
|
||||
});
|
||||
|
||||
// --- validation ---------------------------------------------------------------
|
||||
|
||||
test("validateGraph: rejects duplicate node ids, unknown group/edge refs", () => {
|
||||
assert.throws(
|
||||
() => validateGraph({ nodes: [{ id: "a", label: "A" }, { id: "a", label: "B" }] }),
|
||||
GraphValidationError,
|
||||
);
|
||||
assert.throws(
|
||||
() => validateGraph({ nodes: [{ id: "a", label: "A", group: "ghost" }] }),
|
||||
/unknown group/,
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
validateGraph({
|
||||
nodes: [{ id: "a", label: "A" }],
|
||||
edges: [{ from: "a", to: "ghost" }],
|
||||
}),
|
||||
/resolves to no node/,
|
||||
);
|
||||
assert.throws(() => validateGraph({ nodes: [] }), /non-empty/);
|
||||
});
|
||||
|
||||
test("resolveNodeStyle: kind maps to the preset palette slot for a generic node", () => {
|
||||
const preset = getPreset("default");
|
||||
const s = resolveNodeStyle(preset, { id: "d", label: "DB", kind: "db" });
|
||||
assert.equal(s.iconResolved, false);
|
||||
assert.match(s.style, /fillColor=#d5e8d4;strokeColor=#82b366/);
|
||||
});
|
||||
|
||||
// --- the assembled XML is always linter-clean --------------------------------
|
||||
|
||||
test("from_graph: a plain cross-container-edge graph is linter-clean", async () => {
|
||||
const graph = {
|
||||
nodes: [
|
||||
{ id: "a", label: "A", group: "g1" },
|
||||
{ id: "b", label: "B", group: "g2" },
|
||||
],
|
||||
groups: [
|
||||
{ id: "g1", label: "G1" },
|
||||
{ id: "g2", label: "G2" },
|
||||
],
|
||||
edges: [{ from: "a", to: "b", label: "x" }],
|
||||
};
|
||||
const r = await buildFromGraph(graph, "full");
|
||||
const prepared = prepareModel(r.modelXml); // throws on any lint error
|
||||
assert.equal(prepared.warnings.length, 0, prepared.warnings.join("\n"));
|
||||
// A cross-container edge is parented at the layer sentinel "1".
|
||||
const edge = cellsOf(r.modelXml).find((c) => c.edge);
|
||||
assert.equal(edge.parent, "1");
|
||||
});
|
||||
|
||||
// --- CRITICAL #1: edge / group caps reject FAST (no layout, no OOM) -----------
|
||||
|
||||
test("validateGraph: an over-limit EDGE count is rejected before any layout", () => {
|
||||
// 2 nodes, 200000 edges: passes node validation, would OOM graphToElk/runElk.
|
||||
const edges = [];
|
||||
for (let i = 0; i < 200_000; i++) edges.push({ from: "a", to: "b" });
|
||||
const graph = { nodes: [{ id: "a", label: "A" }, { id: "b", label: "B" }], edges };
|
||||
const t0 = Date.now();
|
||||
assert.throws(
|
||||
() => validateGraph(graph),
|
||||
(e) => e instanceof GraphValidationError && /200000 edges .*max 1000/.test(e.message),
|
||||
);
|
||||
assert.ok(Date.now() - t0 < 1000, "must reject in well under a second (no layout)");
|
||||
});
|
||||
|
||||
test("validateGraph: an over-limit GROUP count is rejected fast", () => {
|
||||
const groups = [];
|
||||
for (let i = 0; i < 600; i++) groups.push({ id: "g" + i, label: "G" });
|
||||
const t0 = Date.now();
|
||||
assert.throws(
|
||||
() => validateGraph({ nodes: [{ id: "a", label: "A" }], groups }),
|
||||
(e) => e instanceof GraphValidationError && /600 groups .*max 500/.test(e.message),
|
||||
);
|
||||
assert.ok(Date.now() - t0 < 1000);
|
||||
});
|
||||
|
||||
test("validateGraph: exactly-at-cap edges/groups are accepted", () => {
|
||||
const edges = [];
|
||||
for (let i = 0; i < 1000; i++) edges.push({ from: "a", to: "b" });
|
||||
assert.doesNotThrow(() =>
|
||||
validateGraph({ nodes: [{ id: "a", label: "A" }, { id: "b", label: "B" }], edges }),
|
||||
);
|
||||
});
|
||||
|
||||
// --- WARNING #3: sameLayerAs spread -> 0 quality-warnings by construction -----
|
||||
|
||||
test("from_graph: a sameLayerAs chain of 5 yields 0 quality-warnings (cross-axis spread)", async () => {
|
||||
const graph = {
|
||||
nodes: [
|
||||
{ id: "a", label: "A" },
|
||||
{ id: "b", label: "B", sameLayerAs: "a" },
|
||||
{ id: "c", label: "C", sameLayerAs: "b" },
|
||||
{ id: "d", label: "D", sameLayerAs: "c" },
|
||||
{ id: "e", label: "E", sameLayerAs: "d" },
|
||||
],
|
||||
edges: [{ from: "a", to: "b" }],
|
||||
direction: "LR",
|
||||
};
|
||||
const r = await buildFromGraph(graph, "full");
|
||||
const cells = parseCells(r.modelXml);
|
||||
const warnings = computeQualityWarnings(cells);
|
||||
assert.equal(warnings.length, 0, warnings.join("\n"));
|
||||
// The four chained dependents share one layer-axis (x) coordinate...
|
||||
const by = new Map(cells.map((c) => [c.id, c]));
|
||||
const xs = ["b", "c", "d", "e"].map((id) => by.get(id).geometry.x);
|
||||
assert.equal(new Set(xs).size, 1, "chained nodes must share the layer axis");
|
||||
// ...but are spread on the cross axis (y) with distinct coordinates.
|
||||
const ys = ["b", "c", "d", "e"].map((id) => by.get(id).geometry.y);
|
||||
assert.equal(new Set(ys).size, 4, "chained nodes must not stack on the cross axis");
|
||||
});
|
||||
|
||||
test("from_graph: pinned coords are honored verbatim; a negative pin is clamped non-negative", async () => {
|
||||
const graph = {
|
||||
nodes: [
|
||||
{ id: "p1", label: "P1", pinned: { x: 100, y: 100 } },
|
||||
// Two user-pinned nodes at (nearly) the same point: user intent, honored.
|
||||
{ id: "p2", label: "P2", pinned: { x: -500, y: 100 } }, // out-of-bounds x -> clamped to 0
|
||||
],
|
||||
direction: "LR",
|
||||
};
|
||||
const r = await buildFromGraph(graph, "full");
|
||||
const by = new Map(parseCells(r.modelXml).map((c) => [c.id, c]));
|
||||
assert.equal(by.get("p1").geometry.x, 100);
|
||||
assert.equal(by.get("p1").geometry.y, 100);
|
||||
assert.equal(by.get("p2").geometry.x, 0, "negative pin x clamped to 0");
|
||||
assert.equal(by.get("p2").geometry.y, 100, "pin y honored");
|
||||
// A pinned overlap MAY warn — that's user-directed and documented; we only
|
||||
// assert the coords are honored (the guarantee softening), not warning count.
|
||||
});
|
||||
|
||||
// --- WARNING #4: incremental MERGE preserves unlisted existing cells ----------
|
||||
|
||||
test("from_graph incremental: an existing cell not in the new graph SURVIVES the add", async () => {
|
||||
const existingModelXml =
|
||||
'<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/>' +
|
||||
'<mxCell id="manual" value="Hand Placed" style="rounded=1;" vertex="1" parent="1">' +
|
||||
'<mxGeometry x="900" y="900" width="120" height="60" as="geometry"/></mxCell>' +
|
||||
'<mxCell id="old1" value="Old One" style="rounded=1;" vertex="1" parent="1">' +
|
||||
'<mxGeometry x="40" y="40" width="120" height="60" as="geometry"/></mxCell>' +
|
||||
"</root></mxGraphModel>";
|
||||
const existingCoords = new Map([
|
||||
["manual", { x: 900, y: 900 }],
|
||||
["old1", { x: 40, y: 40 }],
|
||||
]);
|
||||
// The model sends ONLY the re-listed old1 + the new node — NOT "manual".
|
||||
const graph = {
|
||||
nodes: [
|
||||
{ id: "old1", label: "Old One" },
|
||||
{ id: "new1", label: "Added Node" },
|
||||
],
|
||||
edges: [{ from: "old1", to: "new1" }],
|
||||
direction: "LR",
|
||||
};
|
||||
const r = await buildFromGraph(graph, "incremental", existingCoords, existingModelXml);
|
||||
const by = new Map(parseCells(r.modelXml).map((c) => [c.id, c]));
|
||||
// The unlisted hand-placed cell survives, verbatim coords.
|
||||
assert.ok(by.has("manual"), "unlisted existing cell must not be dropped");
|
||||
assert.equal(by.get("manual").geometry.x, 900);
|
||||
assert.equal(by.get("manual").geometry.y, 900);
|
||||
// The re-listed existing cell keeps its frozen coords; the new node is added.
|
||||
assert.equal(by.get("old1").geometry.x, 40);
|
||||
assert.equal(by.get("old1").geometry.y, 40);
|
||||
assert.ok(by.has("new1"), "the newly added node is present");
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
// Unit tests for the drawio_guide progressive-disclosure reference (issue #424).
|
||||
// Unit tests for the drawioGuide progressive-disclosure reference (issue #424).
|
||||
// Acceptance #2: every section is returned and each is <= ~4KB so pulling one
|
||||
// does not bloat the model's context.
|
||||
import { test } from "node:test";
|
||||
|
||||
@@ -0,0 +1,114 @@
|
||||
// Unit tests for the Mermaid flowchart -> graph parser (issue #425, acceptance
|
||||
// #6, OPTIONAL). Verifies a flowchart with a branch + a subgraph parses to a
|
||||
// graph the from_graph pipeline renders as valid, editable drawio, and that a
|
||||
// non-flowchart diagram is rejected with a clear error.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { mermaidToGraph, MermaidParseError } from "../../build/lib/drawio-mermaid.js";
|
||||
import { buildFromGraph } from "../../build/lib/drawio-graph.js";
|
||||
import { prepareModel } from "../../build/lib/drawio-xml.js";
|
||||
|
||||
test("flowchart with a branch + subgraph -> a valid, editable drawio", async () => {
|
||||
const mm = `flowchart LR
|
||||
A[Start] --> B{Decision}
|
||||
B -->|yes| C[(Database)]
|
||||
B -->|no| D[End]
|
||||
subgraph backend [Backend Services]
|
||||
C
|
||||
E([Cache])
|
||||
end
|
||||
D -.-> E`;
|
||||
const graph = mermaidToGraph(mm);
|
||||
|
||||
// Direction + nodes + a group + labelled/dashed edges parsed.
|
||||
assert.equal(graph.direction, "LR");
|
||||
const nodeIds = graph.nodes.map((n) => n.id).sort();
|
||||
assert.deepEqual(nodeIds, ["A", "B", "C", "D", "E"]);
|
||||
// The decision `{}` maps to the queue palette; the `[(db)]` to db.
|
||||
assert.equal(graph.nodes.find((n) => n.id === "B").kind, "queue");
|
||||
assert.equal(graph.nodes.find((n) => n.id === "C").kind, "db");
|
||||
// The subgraph became a group and claimed its members.
|
||||
assert.equal(graph.groups.length, 1);
|
||||
assert.equal(graph.groups[0].id, "backend");
|
||||
assert.equal(graph.nodes.find((n) => n.id === "C").group, "backend");
|
||||
assert.equal(graph.nodes.find((n) => n.id === "E").group, "backend");
|
||||
// A pipe label and a dotted (async) edge.
|
||||
const yes = graph.edges.find((e) => e.from === "B" && e.to === "C");
|
||||
assert.equal(yes.label, "yes");
|
||||
const dotted = graph.edges.find((e) => e.from === "D" && e.to === "E");
|
||||
assert.equal(dotted.kind, "async");
|
||||
|
||||
// The whole thing renders linter-clean.
|
||||
const built = await buildFromGraph(graph, "full");
|
||||
const prepared = prepareModel(built.modelXml);
|
||||
assert.equal(prepared.warnings.length, 0, prepared.warnings.join("\n"));
|
||||
});
|
||||
|
||||
test("graph TD header sets a top-down direction", () => {
|
||||
const g = mermaidToGraph("graph TD\n X --> Y");
|
||||
assert.equal(g.direction, "TB");
|
||||
assert.deepEqual(g.nodes.map((n) => n.id).sort(), ["X", "Y"]);
|
||||
});
|
||||
|
||||
test("a chained connection A --> B --> C yields two edges", () => {
|
||||
const g = mermaidToGraph("flowchart LR\n A[a] --> B[b] --> C[c]");
|
||||
const pairs = g.edges.map((e) => `${e.from}->${e.to}`).sort();
|
||||
assert.deepEqual(pairs, ["A->B", "B->C"]);
|
||||
});
|
||||
|
||||
test("a non-flowchart diagram is rejected with a clear error", () => {
|
||||
assert.throws(
|
||||
() => mermaidToGraph("sequenceDiagram\n Alice->>Bob: Hi"),
|
||||
/only 'flowchart'\/'graph' is supported/,
|
||||
);
|
||||
assert.throws(() => mermaidToGraph(""), MermaidParseError);
|
||||
});
|
||||
|
||||
// --- CRITICAL #2 / NIT: input-size bounds reject FAST (no OOM) ----------------
|
||||
|
||||
test("mermaidToGraph: an over-length input is rejected before parsing (fast)", () => {
|
||||
const huge = "flowchart LR\n" + "A-->B\n".repeat(60_000); // ~360 KB > 200 KB cap
|
||||
const t0 = Date.now();
|
||||
assert.throws(
|
||||
() => mermaidToGraph(huge),
|
||||
(e) => e instanceof MermaidParseError && /max 200000/.test(e.message),
|
||||
);
|
||||
assert.ok(Date.now() - t0 < 1000, "must reject in well under a second (no parse)");
|
||||
});
|
||||
|
||||
test("mermaidToGraph: an over-line-count input is rejected fast", () => {
|
||||
const many = "flowchart LR\n" + "A\n".repeat(25_000); // > 20000 line cap
|
||||
const t0 = Date.now();
|
||||
assert.throws(
|
||||
() => mermaidToGraph(many),
|
||||
(e) => e instanceof MermaidParseError && /max 20000/.test(e.message),
|
||||
);
|
||||
assert.ok(Date.now() - t0 < 1000);
|
||||
});
|
||||
|
||||
test("mermaidToGraph: too many subgraphs is rejected", () => {
|
||||
let src = "flowchart LR\n";
|
||||
for (let i = 0; i < 600; i++) src += `subgraph s${i}\nend\n`;
|
||||
assert.throws(
|
||||
() => mermaidToGraph(src),
|
||||
(e) => e instanceof MermaidParseError && /too many subgraphs .*max 500/.test(e.message),
|
||||
);
|
||||
});
|
||||
|
||||
test("mermaidToGraph: an over-long connection chain throws (NON-silent truncation)", () => {
|
||||
const chain =
|
||||
"flowchart LR\n" +
|
||||
Array.from({ length: 600 }, (_, i) => "N" + i).join("-->");
|
||||
assert.throws(
|
||||
() => mermaidToGraph(chain),
|
||||
(e) => e instanceof MermaidParseError && /chain exceeds 500 nodes/.test(e.message),
|
||||
);
|
||||
});
|
||||
|
||||
test("mermaidToGraph: a chain of 60 nodes parses (no silent 50-node truncation)", () => {
|
||||
const chain =
|
||||
"flowchart LR\n" +
|
||||
Array.from({ length: 60 }, (_, i) => "N" + i).join("-->");
|
||||
const g = mermaidToGraph(chain);
|
||||
assert.equal(g.nodes.length, 60, "all 60 chained nodes are kept");
|
||||
});
|
||||
@@ -0,0 +1,117 @@
|
||||
// Snapshot + invariant tests for the semantic presets (issue #425, acceptance
|
||||
// #5): every node kind has a style-string per preset, and colorblind-safe uses
|
||||
// only the Okabe-Ito palette (no problematic color pairs).
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
getPreset,
|
||||
genericNodeStyle,
|
||||
edgeStyle,
|
||||
groupStyle,
|
||||
NODE_KINDS,
|
||||
EDGE_KINDS,
|
||||
PRESET_NAMES,
|
||||
} from "../../build/lib/drawio-presets.js";
|
||||
|
||||
// The Okabe-Ito qualitative palette (8 colours distinguishable under the common
|
||||
// colour-vision deficiencies). colorblind-safe MUST draw its strokes from here.
|
||||
const OKABE_ITO = [
|
||||
"#000000", "#e69f00", "#56b4e9", "#009e73",
|
||||
"#f0e442", "#0072b2", "#d55e00", "#cc79a7",
|
||||
].map((s) => s.toLowerCase());
|
||||
|
||||
// The exact base-palette fills from the issue's table (a snapshot: a change to
|
||||
// the default palette is a deliberate, reviewed edit — this catches accidents).
|
||||
const DEFAULT_FILLS = {
|
||||
service: "#dae8fc",
|
||||
db: "#d5e8d4",
|
||||
queue: "#fff2cc",
|
||||
gateway: "#ffe6cc",
|
||||
error: "#f8cecc",
|
||||
external: "#f5f5f5",
|
||||
security: "#e1d5e7",
|
||||
};
|
||||
const DEFAULT_STROKES = {
|
||||
service: "#6c8ebf",
|
||||
db: "#82b366",
|
||||
queue: "#d6b656",
|
||||
gateway: "#d79b00",
|
||||
error: "#b85450",
|
||||
external: "#666666",
|
||||
security: "#9673a6",
|
||||
};
|
||||
|
||||
test("every node kind has a style-string in every preset", () => {
|
||||
for (const p of PRESET_NAMES) {
|
||||
const preset = getPreset(p);
|
||||
for (const kind of NODE_KINDS) {
|
||||
const style = genericNodeStyle(preset, kind);
|
||||
assert.match(style, /fillColor=#[0-9a-fA-F]{6}/, `${p}/${kind} has a fill`);
|
||||
assert.match(style, /strokeColor=#[0-9a-fA-F]{6}/, `${p}/${kind} has a stroke`);
|
||||
assert.match(style, /fontColor=#[0-9a-fA-F]{6}/, `${p}/${kind} has a font`);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test("default preset matches the issue's palette snapshot", () => {
|
||||
const preset = getPreset("default");
|
||||
for (const kind of NODE_KINDS) {
|
||||
const style = genericNodeStyle(preset, kind);
|
||||
assert.ok(
|
||||
style.includes(`fillColor=${DEFAULT_FILLS[kind]}`),
|
||||
`default/${kind} fill ${DEFAULT_FILLS[kind]}`,
|
||||
);
|
||||
assert.ok(
|
||||
style.includes(`strokeColor=${DEFAULT_STROKES[kind]}`),
|
||||
`default/${kind} stroke ${DEFAULT_STROKES[kind]}`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("colorblind-safe strokes come ONLY from the Okabe-Ito palette", () => {
|
||||
const preset = getPreset("colorblind-safe");
|
||||
const usedStrokes = new Set();
|
||||
for (const kind of NODE_KINDS) {
|
||||
const stroke = preset.nodes[kind].strokeColor.toLowerCase();
|
||||
assert.ok(
|
||||
OKABE_ITO.includes(stroke),
|
||||
`colorblind-safe/${kind} stroke ${stroke} is NOT Okabe-Ito`,
|
||||
);
|
||||
usedStrokes.add(stroke);
|
||||
}
|
||||
// No two kinds share a stroke (each is a distinct, distinguishable hue) — the
|
||||
// "no problematic color pairs" acceptance: distinct Okabe-Ito hues.
|
||||
assert.equal(
|
||||
usedStrokes.size,
|
||||
NODE_KINDS.length,
|
||||
"each kind gets a distinct Okabe-Ito stroke",
|
||||
);
|
||||
});
|
||||
|
||||
test("edge kinds: sync solid, async dashed, error red-dashed", () => {
|
||||
const preset = getPreset("default");
|
||||
const sync = edgeStyle(preset, "sync");
|
||||
const async_ = edgeStyle(preset, "async");
|
||||
const error = edgeStyle(preset, "error");
|
||||
assert.doesNotMatch(sync, /dashed=1/, "sync is solid");
|
||||
assert.match(async_, /dashed=1/, "async is dashed");
|
||||
assert.match(error, /dashed=1/, "error is dashed");
|
||||
assert.match(error, /strokeColor=#DD344C/i, "error is red");
|
||||
// An unknown edge kind falls back to sync (solid).
|
||||
assert.doesNotMatch(edgeStyle(preset, "weird"), /dashed=1/);
|
||||
});
|
||||
|
||||
test("group style is always transparent (fillColor=none;container=1;dropTarget=1)", () => {
|
||||
for (const p of PRESET_NAMES) {
|
||||
const style = groupStyle(getPreset(p));
|
||||
assert.match(style, /fillColor=none/, `${p} group transparent`);
|
||||
assert.match(style, /container=1/, `${p} group is a container`);
|
||||
assert.match(style, /dropTarget=1/, `${p} group is a drop target`);
|
||||
}
|
||||
});
|
||||
|
||||
test("EDGE_KINDS / NODE_KINDS constants match the palette", () => {
|
||||
const preset = getPreset("default");
|
||||
for (const k of NODE_KINDS) assert.ok(preset.nodes[k], `node kind ${k}`);
|
||||
for (const k of EDGE_KINDS) assert.ok(preset.edges[k], `edge kind ${k}`);
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
// Unit tests for the drawio_shapes verified-stencil catalog (issue #424).
|
||||
// Unit tests for the drawioShapes verified-stencil catalog (issue #424).
|
||||
// Covers acceptance #1: a "lambda" query returns a valid mxgraph.aws4 icon with
|
||||
// the right service/resource pattern + sizes; a blocklisted stencil query
|
||||
// returns its working replacement.
|
||||
@@ -22,7 +22,7 @@ test("the bundled index loads and is the real ~10k-shape catalog", () => {
|
||||
}
|
||||
});
|
||||
|
||||
test('drawio_shapes("lambda") returns a valid mxgraph.aws4 service icon', () => {
|
||||
test('drawioShapes("lambda") returns a valid mxgraph.aws4 service icon', () => {
|
||||
const results = searchShapes("lambda", { limit: 5 });
|
||||
assert.ok(results.length > 0);
|
||||
// Acceptance #1: a valid aws4 service-level icon (resourceIcon + resIcon)
|
||||
|
||||
@@ -7,16 +7,16 @@ import assert from "node:assert/strict";
|
||||
import { SERVER_INSTRUCTIONS } from "../../build/index.js";
|
||||
import { SHARED_TOOL_SPECS } from "../../build/tool-specs.js";
|
||||
|
||||
test("drawio_shapes and drawio_guide are in the shared registry", () => {
|
||||
assert.equal(SHARED_TOOL_SPECS.drawioShapes.mcpName, "drawio_shapes");
|
||||
assert.equal(SHARED_TOOL_SPECS.drawioGuide.mcpName, "drawio_guide");
|
||||
test("drawioShapes and drawioGuide are in the shared registry", () => {
|
||||
assert.equal(SHARED_TOOL_SPECS.drawioShapes.mcpName, "drawioShapes");
|
||||
assert.equal(SHARED_TOOL_SPECS.drawioGuide.mcpName, "drawioGuide");
|
||||
// Deferred tier, matching the stage-1 drawio tools.
|
||||
assert.equal(SHARED_TOOL_SPECS.drawioShapes.tier, "deferred");
|
||||
assert.equal(SHARED_TOOL_SPECS.drawioGuide.tier, "deferred");
|
||||
});
|
||||
|
||||
test("the new tools are routed in SERVER_INSTRUCTIONS", () => {
|
||||
for (const name of ["drawio_shapes", "drawio_guide"]) {
|
||||
for (const name of ["drawioShapes", "drawioGuide"]) {
|
||||
assert.match(SERVER_INSTRUCTIONS, new RegExp(`\\b${name}\\b`), `${name} missing from guide`);
|
||||
}
|
||||
});
|
||||
@@ -26,7 +26,7 @@ test("the hard-rules block is injected into create/update descriptions", () => {
|
||||
const d = SHARED_TOOL_SPECS[key].description;
|
||||
assert.match(d, /sentinels are MANDATORY/);
|
||||
assert.match(d, /vertex="1" XOR edge="1"/);
|
||||
assert.match(d, /call drawio_shapes first/);
|
||||
assert.match(d, /call drawioShapes first/);
|
||||
assert.match(d, /adaptiveColors="auto"/);
|
||||
assert.match(d, /
/);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
// Drift guards for the stage-3 drawio tools (issue #425): the three high-level
|
||||
// tools must be in the shared registry, routed in SERVER_INSTRUCTIONS, expose
|
||||
// the right schema fields, and carry an `execute` (they call CLIENT methods, so
|
||||
// unlike drawioShapes/drawioGuide they are NOT inlineBothHosts).
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { SERVER_INSTRUCTIONS } from "../../build/index.js";
|
||||
import { SHARED_TOOL_SPECS } from "../../build/tool-specs.js";
|
||||
|
||||
const NEW = ["drawioEditCells", "drawioFromGraph", "drawioFromMermaid"];
|
||||
|
||||
test("the three stage-3 tools are in the shared registry (deferred, camelCase)", () => {
|
||||
for (const name of NEW) {
|
||||
const spec = SHARED_TOOL_SPECS[name];
|
||||
assert.ok(spec, `${name} missing from registry`);
|
||||
assert.equal(spec.mcpName, name);
|
||||
assert.equal(spec.inAppKey, name);
|
||||
assert.equal(spec.tier, "deferred");
|
||||
}
|
||||
});
|
||||
|
||||
test("the stage-3 tools carry an execute (client-backed, NOT inlineBothHosts)", () => {
|
||||
for (const name of NEW) {
|
||||
const spec = SHARED_TOOL_SPECS[name];
|
||||
assert.equal(typeof spec.execute, "function", `${name} needs an execute`);
|
||||
assert.notEqual(spec.inlineBothHosts, true, `${name} must not be inlineBothHosts`);
|
||||
}
|
||||
});
|
||||
|
||||
test("the stage-3 tools are routed in SERVER_INSTRUCTIONS", () => {
|
||||
for (const name of NEW) {
|
||||
assert.match(SERVER_INSTRUCTIONS, new RegExp(`\\b${name}\\b`), `${name} missing from guide`);
|
||||
}
|
||||
});
|
||||
|
||||
test("drawioFromGraph exposes graph + direction/preset/layout params", () => {
|
||||
const shape = SHARED_TOOL_SPECS.drawioFromGraph.buildShape(makeZodStub());
|
||||
for (const key of ["pageId", "graph", "position", "direction", "preset", "layout", "node"]) {
|
||||
assert.ok(key in shape, `drawioFromGraph missing ${key}`);
|
||||
}
|
||||
});
|
||||
|
||||
test("drawioEditCells exposes operations + baseHash", () => {
|
||||
const shape = SHARED_TOOL_SPECS.drawioEditCells.buildShape(makeZodStub());
|
||||
for (const key of ["pageId", "node", "operations", "baseHash"]) {
|
||||
assert.ok(key in shape, `drawioEditCells missing ${key}`);
|
||||
}
|
||||
});
|
||||
|
||||
test("drawioFromMermaid exposes a mermaid param", () => {
|
||||
const shape = SHARED_TOOL_SPECS.drawioFromMermaid.buildShape(makeZodStub());
|
||||
assert.ok("mermaid" in shape);
|
||||
});
|
||||
|
||||
test("the hard-rules block is injected into edit_cells (raw <mxCell> ops) but NOT from_graph/from_mermaid", () => {
|
||||
// edit_cells takes raw <mxCell> xml in add/update ops, so it surfaces the XML
|
||||
// rules. from_graph/from_mermaid NEVER expose XML to the model (the whole point
|
||||
// is the model never writes a style/coord), so the hard rules would be noise.
|
||||
assert.match(SHARED_TOOL_SPECS.drawioEditCells.description, /sentinels are MANDATORY/);
|
||||
assert.doesNotMatch(SHARED_TOOL_SPECS.drawioFromGraph.description, /sentinels are MANDATORY/);
|
||||
assert.doesNotMatch(SHARED_TOOL_SPECS.drawioFromMermaid.description, /sentinels are MANDATORY/);
|
||||
});
|
||||
|
||||
test("routing prose distinguishes from_graph (architectures) vs from_mermaid (standard)", () => {
|
||||
// The EDIT-section routing sentence must mention the semantic tools' intents.
|
||||
assert.match(SERVER_INSTRUCTIONS, /drawioFromGraph[\s\S]*architecture|architecture[\s\S]*drawioFromGraph/i);
|
||||
assert.match(SERVER_INSTRUCTIONS, /drawioFromMermaid[\s\S]*flowchart|flowchart[\s\S]*drawioFromMermaid/i);
|
||||
});
|
||||
|
||||
// Tiny zod stub (buildShape only calls string/number/enum/array/object +
|
||||
// chained min/optional/describe — all return `this`; object() returns a chain
|
||||
// too so nested schemas resolve).
|
||||
function makeZodStub() {
|
||||
const chain = new Proxy(
|
||||
{},
|
||||
{ get: (_t, p) => (p === "parse" ? () => ({}) : () => chain) },
|
||||
);
|
||||
return {
|
||||
string: () => chain,
|
||||
number: () => chain,
|
||||
enum: () => chain,
|
||||
array: () => chain,
|
||||
object: () => chain,
|
||||
};
|
||||
}
|
||||
@@ -258,7 +258,7 @@ test("a non-axios error is passed through untouched", () => {
|
||||
const GOOD_UUID = "019f499a-9f8c-7d68-b7be-ce100d7c6c56";
|
||||
|
||||
test("assertFullUuid accepts a full canonical UUID (any version nibble)", () => {
|
||||
assert.doesNotThrow(() => assertFullUuid("resolve_comment", "commentId", GOOD_UUID));
|
||||
assert.doesNotThrow(() => assertFullUuid("resolveComment", "commentId", GOOD_UUID));
|
||||
// A v4 id also passes (version/variant-agnostic).
|
||||
assert.doesNotThrow(() =>
|
||||
assertFullUuid("get_comment", "commentId", "3d5b7c1e-2f4a-4b6c-8d9e-0f1a2b3c4d5e"),
|
||||
@@ -267,10 +267,10 @@ test("assertFullUuid accepts a full canonical UUID (any version nibble)", () =>
|
||||
|
||||
test("assertFullUuid rejects a truncated prefix", () => {
|
||||
assert.throws(
|
||||
() => assertFullUuid("resolve_comment", "commentId", "019f499a"),
|
||||
() => assertFullUuid("resolveComment", "commentId", "019f499a"),
|
||||
(e) =>
|
||||
e.message.startsWith(
|
||||
"resolve_comment: 'commentId' must be the FULL comment UUID",
|
||||
"resolveComment: 'commentId' must be the FULL comment UUID",
|
||||
) &&
|
||||
e.message.includes("got '019f499a'") &&
|
||||
e.message.includes("Copy the id verbatim"),
|
||||
@@ -279,11 +279,11 @@ test("assertFullUuid rejects a truncated prefix", () => {
|
||||
|
||||
test("assertFullUuid rejects garbage and empty string", () => {
|
||||
assert.throws(
|
||||
() => assertFullUuid("delete_comment", "commentId", "not-a-uuid"),
|
||||
() => assertFullUuid("deleteComment", "commentId", "not-a-uuid"),
|
||||
/must be the FULL comment UUID.*got 'not-a-uuid'/s,
|
||||
);
|
||||
assert.throws(
|
||||
() => assertFullUuid("update_comment", "commentId", ""),
|
||||
() => assertFullUuid("updateComment", "commentId", ""),
|
||||
/must be the FULL comment UUID.*got ''/s,
|
||||
);
|
||||
});
|
||||
@@ -436,14 +436,14 @@ test("all 5 comment-id call sites reject a bad id with ZERO network traffic", as
|
||||
const client = new DocmostClient(baseURL, "u@example.com", "pw");
|
||||
const bad = "019f499a"; // truncated
|
||||
|
||||
await assert.rejects(() => client.resolveComment(bad, true), /resolve_comment: 'commentId'/);
|
||||
await assert.rejects(() => client.updateComment(bad, "hi"), /update_comment: 'commentId'/);
|
||||
await assert.rejects(() => client.deleteComment(bad), /delete_comment: 'commentId'/);
|
||||
await assert.rejects(() => client.resolveComment(bad, true), /resolveComment: 'commentId'/);
|
||||
await assert.rejects(() => client.updateComment(bad, "hi"), /updateComment: 'commentId'/);
|
||||
await assert.rejects(() => client.deleteComment(bad), /deleteComment: 'commentId'/);
|
||||
await assert.rejects(() => client.getComment(bad), /get_comment: 'commentId'/);
|
||||
// createComment validates parentCommentId only when provided.
|
||||
await assert.rejects(
|
||||
() => client.createComment("page-1", "body", "inline", "sel", bad),
|
||||
/create_comment: 'parentCommentId'/,
|
||||
/createComment: 'parentCommentId'/,
|
||||
);
|
||||
|
||||
assert.equal(requests, 0, "no request (not even /auth/login) may be issued for a bad id");
|
||||
|
||||
@@ -1,7 +1,11 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { filterComment, filterPage } from "../../build/lib/filters.js";
|
||||
import {
|
||||
filterComment,
|
||||
filterPage,
|
||||
filterSearchResult,
|
||||
} from "../../build/lib/filters.js";
|
||||
|
||||
test("filterComment includes resolvedAt/resolvedById as null when absent", () => {
|
||||
const result = filterComment({
|
||||
@@ -171,3 +175,91 @@ test("filterPage includes both content and subpages together", () => {
|
||||
assert.equal(result.content, "body");
|
||||
assert.deepEqual(result.subpages, [{ id: "s1", title: "Sub" }]);
|
||||
});
|
||||
|
||||
// --- filterSearchResult (#443 agent-lookup contract) -------------------------
|
||||
|
||||
test("filterSearchResult maps the lookup shape to {pageId,title,path,snippet,score}", () => {
|
||||
const result = filterSearchResult({
|
||||
id: "0199aa-uuid",
|
||||
slugId: "slug-secret",
|
||||
title: "backup-srv.local",
|
||||
parentPageId: "0199pp",
|
||||
path: ["Infrastructure", "Datacenter A", "Servers"],
|
||||
snippet: "…IP: 10.0.12.5. Debian 12…",
|
||||
score: 0.92,
|
||||
});
|
||||
|
||||
assert.deepEqual(result, {
|
||||
pageId: "0199aa-uuid",
|
||||
title: "backup-srv.local",
|
||||
path: ["Infrastructure", "Datacenter A", "Servers"],
|
||||
snippet: "…IP: 10.0.12.5. Debian 12…",
|
||||
score: 0.92,
|
||||
});
|
||||
});
|
||||
|
||||
test("filterSearchResult NEVER exposes slugId (pageId is the only identifier)", () => {
|
||||
const result = filterSearchResult({
|
||||
id: "uuid-1",
|
||||
slugId: "slug-1",
|
||||
title: "t",
|
||||
path: [],
|
||||
snippet: "s",
|
||||
score: 0.1,
|
||||
});
|
||||
assert.equal("slugId" in result, false);
|
||||
assert.equal("id" in result, false);
|
||||
assert.equal(result.pageId, "uuid-1");
|
||||
});
|
||||
|
||||
test("filterSearchResult root page yields path: []", () => {
|
||||
const result = filterSearchResult({
|
||||
id: "uuid-root",
|
||||
title: "Root",
|
||||
path: [],
|
||||
snippet: "s",
|
||||
score: 0.5,
|
||||
});
|
||||
assert.deepEqual(result.path, []);
|
||||
});
|
||||
|
||||
test("filterSearchResult degrades a legacy FTS hit (no lookup fields)", () => {
|
||||
// Stock upstream stripped the opt-in DTO fields → legacy shape with
|
||||
// highlight + rank and no path/snippet/score.
|
||||
const result = filterSearchResult({
|
||||
id: "uuid-legacy",
|
||||
slugId: "slug-legacy",
|
||||
title: "Legacy",
|
||||
parentPageId: null,
|
||||
rank: 0.37,
|
||||
highlight: "…matched <b>text</b>…",
|
||||
space: { id: "sp1", name: "Space" },
|
||||
});
|
||||
|
||||
assert.equal(result.pageId, "uuid-legacy");
|
||||
assert.equal(result.title, "Legacy");
|
||||
// snippet falls back to highlight, score to rank, path to [].
|
||||
assert.equal(result.snippet, "…matched <b>text</b>…");
|
||||
assert.equal(result.score, 0.37);
|
||||
assert.deepEqual(result.path, []);
|
||||
assert.equal("slugId" in result, false);
|
||||
});
|
||||
|
||||
test("filterSearchResult is null-safe on missing snippet/score/path", () => {
|
||||
const result = filterSearchResult({ id: "u", title: "t" });
|
||||
assert.equal(result.pageId, "u");
|
||||
assert.equal(result.snippet, "");
|
||||
assert.equal(result.score, 0);
|
||||
assert.deepEqual(result.path, []);
|
||||
});
|
||||
|
||||
test("filterSearchResult ignores a non-array path", () => {
|
||||
const result = filterSearchResult({
|
||||
id: "u",
|
||||
title: "t",
|
||||
path: "not-an-array",
|
||||
snippet: "s",
|
||||
score: 1,
|
||||
});
|
||||
assert.deepEqual(result.path, []);
|
||||
});
|
||||
|
||||
@@ -9,7 +9,7 @@ import {
|
||||
|
||||
// Pins the footnoteWarnings PLUMBING contract (#169 review; reduced in #414): the
|
||||
// field is present only when legacy reference-style `[^id]:` syntax is used and
|
||||
// omitted otherwise, AND `import_page_markdown` analyzes the BODY (after the
|
||||
// omitted otherwise, AND `importPageMarkdown` analyzes the BODY (after the
|
||||
// docmost:meta / docmost:comments blocks) — so a footnote-like token inside those
|
||||
// JSON blocks never warns, while a real definition in the body does.
|
||||
// importPageMarkdown does exactly `footnoteWarningsField(parseDocmostMarkdown(full).body)`
|
||||
|
||||
@@ -25,7 +25,7 @@ test("formatting-only edit (strip-toggle) is refused, not applied", () => {
|
||||
assert.equal(failed.length, 1, "one refused edit");
|
||||
assert.equal(failed[0].find, "~~x~~");
|
||||
assert.match(failed[0].reason, /cannot add or remove formatting marks/);
|
||||
assert.match(failed[0].reason, /patch_node/);
|
||||
assert.match(failed[0].reason, /patchNode/);
|
||||
// The document is untouched (the strike mark is preserved).
|
||||
assert.deepEqual(out, snapshot);
|
||||
});
|
||||
@@ -143,7 +143,7 @@ test("typo fix wrapped in markdown still applies (not refused)", () => {
|
||||
// ---------------------------------------------------------------------------
|
||||
// (iv) #410 footnote token: a `replace` containing `^[...]` is refused into
|
||||
// failed[] (it would be written as a LITERAL string, never a real footnote).
|
||||
// Nothing is applied; the reason points at insert_footnote.
|
||||
// Nothing is applied; the reason points at insertFootnote.
|
||||
// ---------------------------------------------------------------------------
|
||||
test("replace containing a `^[...]` footnote token is refused, not applied", () => {
|
||||
const input = doc(paragraph(textNode("The claim stands.")));
|
||||
@@ -156,7 +156,7 @@ test("replace containing a `^[...]` footnote token is refused, not applied", ()
|
||||
assert.equal(results.length, 0, "nothing applied");
|
||||
assert.equal(failed.length, 1, "one refused edit");
|
||||
assert.equal(failed[0].find, "The claim stands.");
|
||||
assert.match(failed[0].reason, /insert_footnote/);
|
||||
assert.match(failed[0].reason, /insertFootnote/);
|
||||
// The document is byte-for-byte untouched — no literal `^[` was written.
|
||||
assert.deepEqual(out, snapshot);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
// #413: unit tests for the markdown-fragment helpers used by patchNode/insertNode.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import {
|
||||
importMarkdownFragment,
|
||||
canBeDocChild,
|
||||
findUnrepresentableTableAttrs,
|
||||
} from "../../build/lib/markdown-fragment.js";
|
||||
|
||||
function findAll(node, type, acc = []) {
|
||||
if (!node || typeof node !== "object") return acc;
|
||||
if (node.type === type) acc.push(node);
|
||||
if (Array.isArray(node.content))
|
||||
for (const c of node.content) findAll(c, type, acc);
|
||||
return acc;
|
||||
}
|
||||
|
||||
test("importMarkdownFragment: plain markdown -> blocks, no definitions", async () => {
|
||||
const { blocks, definitions } = await importMarkdownFragment(
|
||||
"first\n\nsecond",
|
||||
);
|
||||
assert.equal(blocks.length, 2);
|
||||
assert.equal(definitions.length, 0);
|
||||
assert.equal(blocks[0].type, "paragraph");
|
||||
});
|
||||
|
||||
test("importMarkdownFragment: `^[...]` footnote -> a definition + a remapped ref", async () => {
|
||||
const { blocks, definitions } = await importMarkdownFragment(
|
||||
"a claim^[the note]",
|
||||
);
|
||||
assert.equal(definitions.length, 1);
|
||||
const refs = findAll({ type: "doc", content: blocks }, "footnoteReference");
|
||||
assert.equal(refs.length, 1);
|
||||
// The reference id must match the (remapped) definition id.
|
||||
assert.equal(refs[0].attrs.id, definitions[0].attrs.id);
|
||||
// The id is NOT the importer's sequential "fn-1" — it was remapped to a fresh
|
||||
// uuid so it cannot collide with a page footnote of the same number.
|
||||
assert.notEqual(refs[0].attrs.id, "fn-1");
|
||||
});
|
||||
|
||||
test("importMarkdownFragment: whitespace markdown imports to a single empty paragraph", async () => {
|
||||
// The importer yields one empty paragraph for whitespace-only input (not zero
|
||||
// blocks), so the fragment path returns that block. The client's XOR guard
|
||||
// (markdown.trim() !== "") is what rejects an empty-string patch up front, so
|
||||
// importMarkdownFragment never sees a truly empty string via patch/insert.
|
||||
const { blocks, definitions } = await importMarkdownFragment(" \n ");
|
||||
assert.equal(blocks.length, 1);
|
||||
assert.equal(blocks[0].type, "paragraph");
|
||||
assert.equal(definitions.length, 0);
|
||||
});
|
||||
|
||||
test("canBeDocChild: paragraph/heading/table are doc children; tableRow/cell are not", () => {
|
||||
assert.equal(canBeDocChild("paragraph"), true);
|
||||
assert.equal(canBeDocChild("heading"), true);
|
||||
assert.equal(canBeDocChild("table"), true);
|
||||
assert.equal(canBeDocChild("tableRow"), false);
|
||||
assert.equal(canBeDocChild("tableCell"), false);
|
||||
assert.equal(canBeDocChild("tableHeader"), false);
|
||||
assert.equal(canBeDocChild("text"), false);
|
||||
assert.equal(canBeDocChild(undefined), false);
|
||||
assert.equal(canBeDocChild("notARealType"), false);
|
||||
});
|
||||
|
||||
const cell = (attrs, text) => ({
|
||||
type: "tableCell",
|
||||
attrs,
|
||||
content: [{ type: "paragraph", content: [{ type: "text", text }] }],
|
||||
});
|
||||
|
||||
test("findUnrepresentableTableAttrs: null for a plain paragraph and a simple table", () => {
|
||||
assert.equal(
|
||||
findUnrepresentableTableAttrs({
|
||||
type: "paragraph",
|
||||
content: [{ type: "text", text: "x" }],
|
||||
}),
|
||||
null,
|
||||
);
|
||||
const simpleTable = {
|
||||
type: "table",
|
||||
content: [
|
||||
{
|
||||
type: "tableRow",
|
||||
content: [cell({ colspan: 1, rowspan: 1 }, "a")],
|
||||
},
|
||||
],
|
||||
};
|
||||
assert.equal(findUnrepresentableTableAttrs(simpleTable), null);
|
||||
});
|
||||
|
||||
test("findUnrepresentableTableAttrs: flags colspan/rowspan/colwidth/backgroundColor", () => {
|
||||
const mk = (attrs) => ({
|
||||
type: "table",
|
||||
content: [{ type: "tableRow", content: [cell(attrs, "a")] }],
|
||||
});
|
||||
assert.match(findUnrepresentableTableAttrs(mk({ colspan: 2 })), /colspan/);
|
||||
assert.match(findUnrepresentableTableAttrs(mk({ rowspan: 2 })), /rowspan/);
|
||||
assert.match(
|
||||
findUnrepresentableTableAttrs(mk({ colwidth: [120] })),
|
||||
/colwidth/,
|
||||
);
|
||||
assert.match(
|
||||
findUnrepresentableTableAttrs(mk({ backgroundColor: "#eee" })),
|
||||
/backgroundColor/,
|
||||
);
|
||||
});
|
||||
|
||||
test("findUnrepresentableTableAttrs: finds a span nested deep (table inside a callout)", () => {
|
||||
const doc = {
|
||||
type: "callout",
|
||||
content: [
|
||||
{
|
||||
type: "table",
|
||||
content: [
|
||||
{ type: "tableRow", content: [cell({ colspan: 3 }, "wide")] },
|
||||
],
|
||||
},
|
||||
],
|
||||
};
|
||||
assert.match(findUnrepresentableTableAttrs(doc), /colspan/);
|
||||
});
|
||||
@@ -0,0 +1,84 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { dirname, join } from "node:path";
|
||||
|
||||
// Issue #449, invariant 2 ("no-await-окно"): the atomicity of the
|
||||
// read -> transform -> write section in CollabSession.mutate depends on there
|
||||
// being NO `await` (nor any other async yield point) between
|
||||
// `TiptapTransformer.fromYdoc` and `applyDocToFragment`. Yjs applies queued
|
||||
// remote updates only when the event loop yields, so an accidental await in that
|
||||
// window would let a concurrent human edit interleave and be clobbered (#152).
|
||||
//
|
||||
// That was an invariant enforced only by a comment. This test turns a violation
|
||||
// RED: it reads the SOURCE of collab-session.ts, extracts the block delimited by
|
||||
// the machine-readable BEGIN/END markers, and asserts no async boundary appears
|
||||
// inside it. Introducing an `await` (or `for await`, or `yield`) between the
|
||||
// markers fails this test.
|
||||
|
||||
const here = dirname(fileURLToPath(import.meta.url));
|
||||
// Scan the .ts SOURCE (not the compiled .js): the markers live in the source and
|
||||
// transpilation could rewrite/erase them, so the source is the authoritative
|
||||
// artifact the human edits.
|
||||
const sourcePath = join(here, "..", "..", "src", "lib", "collab-session.ts");
|
||||
const source = readFileSync(sourcePath, "utf8");
|
||||
|
||||
const BEGIN = "=== MUTATE-CRITICAL-WINDOW: BEGIN";
|
||||
const END = "=== MUTATE-CRITICAL-WINDOW: END";
|
||||
|
||||
test("critical-window markers exist exactly once each", () => {
|
||||
const begins = source.split(BEGIN).length - 1;
|
||||
const ends = source.split(END).length - 1;
|
||||
assert.equal(
|
||||
begins,
|
||||
1,
|
||||
`expected exactly one '${BEGIN}' marker, found ${begins}`,
|
||||
);
|
||||
assert.equal(ends, 1, `expected exactly one '${END}' marker, found ${ends}`);
|
||||
});
|
||||
|
||||
test("the read->write critical window contains no async boundary (no await/yield)", () => {
|
||||
const beginIdx = source.indexOf(BEGIN);
|
||||
const endIdx = source.indexOf(END);
|
||||
assert.ok(beginIdx !== -1, "BEGIN marker not found");
|
||||
assert.ok(endIdx !== -1, "END marker not found");
|
||||
assert.ok(endIdx > beginIdx, "END marker must come after BEGIN marker");
|
||||
|
||||
// The block strictly between the two marker lines. Move past the end of the
|
||||
// BEGIN marker line so the marker comment text itself is not scanned.
|
||||
const afterBeginLine = source.indexOf("\n", beginIdx) + 1;
|
||||
const block = source.slice(afterBeginLine, endIdx);
|
||||
|
||||
// Detect any real async yield keyword as a whole word. `\bawait\b` also matches
|
||||
// inside `for await`, which is exactly what we want to forbid here.
|
||||
const forbidden = [/\bawait\b/, /\byield\b/];
|
||||
for (const re of forbidden) {
|
||||
const m = block.match(re);
|
||||
assert.equal(
|
||||
m,
|
||||
null,
|
||||
`forbidden async boundary '${m?.[0]}' found inside the no-await critical ` +
|
||||
`window of CollabSession.mutate. INVARIANT 1 (#449): the block between ` +
|
||||
`TiptapTransformer.fromYdoc and applyDocToFragment must be fully ` +
|
||||
`synchronous — an await there reopens the clobber-live-edits race (#152).`,
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
test("the critical window still spans fromYdoc -> applyDocToFragment", () => {
|
||||
// Guards the markers from drifting off the code they are meant to protect: if
|
||||
// someone moves the read/write out of the window, this catches it.
|
||||
const beginIdx = source.indexOf(BEGIN);
|
||||
const endIdx = source.indexOf(END);
|
||||
const afterBeginLine = source.indexOf("\n", beginIdx) + 1;
|
||||
const block = source.slice(afterBeginLine, endIdx);
|
||||
assert.ok(
|
||||
block.includes("TiptapTransformer.fromYdoc"),
|
||||
"critical window must contain the TiptapTransformer.fromYdoc read",
|
||||
);
|
||||
assert.ok(
|
||||
block.includes("applyDocToFragment"),
|
||||
"critical window must contain the applyDocToFragment write",
|
||||
);
|
||||
});
|
||||
@@ -486,34 +486,34 @@ test("insertNodeRelative truly-missing anchor still returns inserted:false", ()
|
||||
assert.equal(inserted, false);
|
||||
});
|
||||
|
||||
// assertUnambiguousMatch (#159, #185 review pt 2): the patch_node/delete_node
|
||||
// assertUnambiguousMatch (#159, #185 review pt 2): the patchNode/deleteNode
|
||||
// guard. Docmost duplicates block ids on copy/paste, so a write by id that
|
||||
// matches >1 node must be REFUSED (the caller already skipped the write for any
|
||||
// count !== 1; this reports the error). The duplicate COUNT itself is covered by
|
||||
// the replaceNodeById/deleteNodeById tests above (count===2 for a 2-dup doc).
|
||||
test("assertUnambiguousMatch: count 0 throws 'no node found'", () => {
|
||||
assert.throws(
|
||||
() => assertUnambiguousMatch("patch_node", "replace", 0, "n1", "p1"),
|
||||
/patch_node: no node with id "n1" found on page p1/,
|
||||
() => assertUnambiguousMatch("patchNode", "replace", 0, "n1", "p1"),
|
||||
/patchNode: no node with id "n1" found on page p1/,
|
||||
);
|
||||
});
|
||||
|
||||
test("assertUnambiguousMatch: count > 1 refuses with an 'ambiguous' error", () => {
|
||||
assert.throws(
|
||||
() => assertUnambiguousMatch("patch_node", "replace", 2, "dup", "p1"),
|
||||
() => assertUnambiguousMatch("patchNode", "replace", 2, "dup", "p1"),
|
||||
/ambiguous.*Refusing to replace all of them; nothing was changed/,
|
||||
);
|
||||
assert.throws(
|
||||
() => assertUnambiguousMatch("delete_node", "delete", 3, "dup", "p1"),
|
||||
() => assertUnambiguousMatch("deleteNode", "delete", 3, "dup", "p1"),
|
||||
/ambiguous.*Refusing to delete all of them; nothing was changed/,
|
||||
);
|
||||
});
|
||||
|
||||
test("assertUnambiguousMatch: exactly one match does NOT throw", () => {
|
||||
assert.doesNotThrow(() =>
|
||||
assertUnambiguousMatch("patch_node", "replace", 1, "n1", "p1"),
|
||||
assertUnambiguousMatch("patchNode", "replace", 1, "n1", "p1"),
|
||||
);
|
||||
assert.doesNotThrow(() =>
|
||||
assertUnambiguousMatch("delete_node", "delete", 1, "n1", "p1"),
|
||||
assertUnambiguousMatch("deleteNode", "delete", 1, "n1", "p1"),
|
||||
);
|
||||
});
|
||||
|
||||
@@ -1,13 +1,26 @@
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
|
||||
import { withPageLock } from "../../build/lib/page-lock.js";
|
||||
import { withPageLock, isUuid } from "../../build/lib/page-lock.js";
|
||||
|
||||
const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
|
||||
// withPageLock now asserts its key is a canonical UUID (#449, "resolve-then-
|
||||
// lock"), so the mechanics tests below must lock under real UUIDs, not arbitrary
|
||||
// labels. Distinct valid UUIDv7-shaped ids for the distinct-page cases.
|
||||
const U = {
|
||||
same: "11111111-1111-7111-8111-111111111111",
|
||||
ordered: "22222222-2222-7222-8222-222222222222",
|
||||
poison: "33333333-3333-7333-8333-333333333333",
|
||||
poison2: "44444444-4444-7444-8444-444444444444",
|
||||
A: "aaaaaaaa-aaaa-7aaa-8aaa-aaaaaaaaaaaa",
|
||||
B: "bbbbbbbb-bbbb-7bbb-8bbb-bbbbbbbbbbbb",
|
||||
leak: "55555555-5555-7555-8555-555555555555",
|
||||
};
|
||||
|
||||
test("two ops on the same pageId run strictly sequentially (no overlap)", async () => {
|
||||
const events = [];
|
||||
const pageId = "same-page";
|
||||
const pageId = U.same;
|
||||
|
||||
const p1 = withPageLock(pageId, async () => {
|
||||
events.push("start-1");
|
||||
@@ -33,7 +46,7 @@ test("two ops on the same pageId run strictly sequentially (no overlap)", async
|
||||
});
|
||||
|
||||
test("same pageId ordering holds for many queued ops", async () => {
|
||||
const pageId = "ordered-page";
|
||||
const pageId = U.ordered;
|
||||
const order = [];
|
||||
const active = { count: 0, maxConcurrent: 0 };
|
||||
|
||||
@@ -60,7 +73,7 @@ test("same pageId ordering holds for many queued ops", async () => {
|
||||
});
|
||||
|
||||
test("a rejecting op does not poison the chain for the same page", async () => {
|
||||
const pageId = "poison-page";
|
||||
const pageId = U.poison;
|
||||
const events = [];
|
||||
|
||||
const failing = withPageLock(pageId, async () => {
|
||||
@@ -87,7 +100,7 @@ test("a rejecting op does not poison the chain for the same page", async () => {
|
||||
});
|
||||
|
||||
test("failing op queued before a success both resolve/reject correctly", async () => {
|
||||
const pageId = "poison-page-2";
|
||||
const pageId = U.poison2;
|
||||
const order = [];
|
||||
|
||||
const failing = withPageLock(pageId, async () => {
|
||||
@@ -111,14 +124,14 @@ test("failing op queued before a success both resolve/reject correctly", async (
|
||||
test("ops on different pageIds run concurrently (overlap)", async () => {
|
||||
const events = [];
|
||||
|
||||
const pA = withPageLock("page-A", async () => {
|
||||
const pA = withPageLock(U.A, async () => {
|
||||
events.push("A-start");
|
||||
await delay(40);
|
||||
events.push("A-end");
|
||||
return "A";
|
||||
});
|
||||
|
||||
const pB = withPageLock("page-B", async () => {
|
||||
const pB = withPageLock(U.B, async () => {
|
||||
events.push("B-start");
|
||||
await delay(10);
|
||||
events.push("B-end");
|
||||
@@ -134,7 +147,7 @@ test("ops on different pageIds run concurrently (overlap)", async () => {
|
||||
});
|
||||
|
||||
test("no functional leak: many sequential ops on same page keep working", async () => {
|
||||
const pageId = "leak-page";
|
||||
const pageId = U.leak;
|
||||
|
||||
// Run a long series of fully sequential ops (each awaited before the next is
|
||||
// queued) so the internal map entry is created and dropped repeatedly.
|
||||
@@ -151,3 +164,56 @@ test("no functional leak: many sequential ops on same page keep working", async
|
||||
const final = await withPageLock(pageId, async () => "still-works");
|
||||
assert.equal(final, "still-works");
|
||||
});
|
||||
|
||||
// --- Issue #449: fail-fast on a non-canonical lock key ---------------------
|
||||
// A write method that reaches the lock path with an unresolved slugId (or any
|
||||
// non-UUID key) must fail IMMEDIATELY and LOUDLY, not lock under a split key and
|
||||
// silently lose per-page serialization. These assert withPageLock rejects such
|
||||
// a key before ever running fn.
|
||||
|
||||
test("withPageLock throws on a raw 10-char slugId (unresolved key)", () => {
|
||||
let ran = false;
|
||||
assert.throws(
|
||||
() =>
|
||||
withPageLock("p7Xk29Lm4Q", async () => {
|
||||
ran = true;
|
||||
return "should-not-run";
|
||||
}),
|
||||
/canonical page UUID|resolve-then-lock/i,
|
||||
"a slugId key must fail-fast at the lock",
|
||||
);
|
||||
// The work must NOT have started: fail-fast means no serialization was
|
||||
// silently skipped under a bad key.
|
||||
assert.equal(ran, false, "fn must not run when the key is rejected");
|
||||
});
|
||||
|
||||
test("withPageLock throws on other non-UUID keys (label, empty, non-string)", () => {
|
||||
for (const bad of ["same-page", "", "not-a-uuid", "1234"]) {
|
||||
assert.throws(
|
||||
() => withPageLock(bad, async () => "x"),
|
||||
/canonical page UUID/i,
|
||||
`expected withPageLock to reject key ${JSON.stringify(bad)}`,
|
||||
);
|
||||
}
|
||||
// A non-string key is also rejected (guards a mistyped call site).
|
||||
assert.throws(
|
||||
() => withPageLock(/** @type {any} */ (undefined), async () => "x"),
|
||||
/canonical page UUID/i,
|
||||
);
|
||||
});
|
||||
|
||||
test("withPageLock accepts a canonical UUID key (no false positive)", async () => {
|
||||
const uuid = "0192f3a4-b5c6-7d8e-9f01-23456789abcd";
|
||||
assert.equal(isUuid(uuid), true);
|
||||
const r = await withPageLock(uuid, async () => "ok");
|
||||
assert.equal(r, "ok");
|
||||
});
|
||||
|
||||
test("isUuid discriminates UUIDs from slugIds (shared predicate)", () => {
|
||||
// The predicate withPageLock asserts on is the SAME one resolvePageId uses to
|
||||
// decide whether a pageId is already a UUID (imported from page-lock).
|
||||
assert.equal(isUuid("0192f3a4-b5c6-7d8e-9f01-23456789abcd"), true);
|
||||
assert.equal(isUuid("p7Xk29Lm4Q"), false); // 10-char nanoid slugId
|
||||
assert.equal(isUuid("not-a-uuid"), false);
|
||||
assert.equal(isUuid(""), false);
|
||||
});
|
||||
|
||||
@@ -155,7 +155,7 @@ test("insertTableRow at index 0 inserts before the header and pads to 3 cells",
|
||||
test("insertTableRow throws when given more cells than columns", () => {
|
||||
assert.throws(
|
||||
() => insertTableRow(makeDoc(), "#1", ["a", "b", "c", "d"]),
|
||||
/table_insert_row: got 4 cell\(s\) but the table has 3 column\(s\)/,
|
||||
/tableInsertRow: got 4 cell\(s\) but the table has 3 column\(s\)/,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -232,7 +232,7 @@ test("insertTableRow uses the max column count across all rows (ragged table)",
|
||||
// ...but 4 cells exceed the widest row and throw.
|
||||
assert.throws(
|
||||
() => insertTableRow(makeRaggedDoc(), "#0", ["a", "b", "c", "d"]),
|
||||
/table_insert_row: got 4 cell\(s\) but the table has 3 column\(s\)/,
|
||||
/tableInsertRow: got 4 cell\(s\) but the table has 3 column\(s\)/,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -286,7 +286,7 @@ test("deleteTableRow removes the 3rd row -> rows:2", () => {
|
||||
test("deleteTableRow out-of-range index throws", () => {
|
||||
assert.throws(
|
||||
() => deleteTableRow(makeDoc(), "#1", 9),
|
||||
/table_delete_row: row index 9 out of range \(table has 3 row\(s\)\)/,
|
||||
/tableDeleteRow: row index 9 out of range \(table has 3 row\(s\)\)/,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -329,10 +329,10 @@ test("updateTableCell sets cell [1,1] to 'Z' and preserves the paragraph id", ()
|
||||
test("updateTableCell out-of-range row/col throws", () => {
|
||||
assert.throws(
|
||||
() => updateTableCell(makeDoc(), "#1", 9, 0, "x"),
|
||||
/table_update_cell: cell \[9,0\] out of range/,
|
||||
/tableUpdateCell: cell \[9,0\] out of range/,
|
||||
);
|
||||
assert.throws(
|
||||
() => updateTableCell(makeDoc(), "#1", 0, 9, "x"),
|
||||
/table_update_cell: cell \[0,9\] out of range/,
|
||||
/tableUpdateCell: cell \[0,9\] out of range/,
|
||||
);
|
||||
});
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user