Compare commits

..

15 Commits

Author SHA1 Message Date
agent_coder 14d7b21df0 refactor(mcp): распил client.ts (5206 строк) на доменные модули за тонким фасадом (#450)
client.ts был god-object'ом на 5206 строк / ~65 методов / 5 ответственностей —
любая правка рисковала всем write-path. Разнесли на доменные модули;
DocmostClient остаётся ТОНКИМ ФАСАДОМ с прежним внешним контрактом. Чистый
рефакторинг, поведение не меняется. closes #450

- Фасад client.ts (93 строки) композирует 10 миксинов над общим абстрактным
  базовым классом. Паттерн МИКСИНЫ (не context-object): тесты субклассируют
  DocmostClient и переопределяют seam'ы; единая цепочка прототипов сохраняет
  виртуальную диспетчеризацию this.<method> сквозь модули.
- client/context.ts (база): общее состояние (axios, apiUrl, токены, кэши),
  конструктор + оба интерсептора, login/ensureAuthenticated/paginateAll/
  resolvePageId/mutateLiveContentUnlocked + write-seam'ы mutatePage/replacePage.
  private→protected (внутреннее, не в публичном контракте).
- Модули (каждый ≤730): read, pages, nodes-write, media (images/attachments/
  drawio), comments, transforms, tables, stash, doc-validate. errors.ts —
  единый REST error-mapping (довершение #437; тексты сообщений без изменений).
- Внешний контракт СОХРАНЁН (доказано компиляцией с обеих сторон): 53 публичных
  метода через `implements IXMixin` (без ручного зеркала, #446); оба Pick
  (in-app loader + tool-specs) резолвятся; множество async-методов 59==59.
  Нагруженные seam'ы (replaceImage один-лок #425, self-resolve #449, единый
  error-путь) байт-идентичны; ни одного дубля публичного метода.
- zod v3→v4: investigate-only, отложено — SDK 1.29 поддерживает v4, но мажор-
  бамп трогает всю схема-поверхность; отдельным follow-up.

Стоит на #449 (#475). Тесты: mcp node --test 800/800 (== база), tsc чисто в
client/. Ни одной строки кода не потеряно; убраны 52 дублированных JSDoc-блока.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 01:04:42 +03:00
agent_coder 9a8671c3af refactor(mcp): структурные инварианты конкурентной записи — UUID-assert в page-lock, self-resolve seams, no-await-guard (#449)
Механика конкурентной записи (withPageLock → acquireCollabSession → mutate)
держалась на цепочке конвенций в комментариях. Новый write-метод без знания
правил компилировался и уходил в прод (класс #260/#152/#159). Закрепляем ключевые
инварианты кодом.

- page-lock.ts: экспортированы UUID_RE/isUuid (тот же regex, что resolvePageId,
  UUID v1–8/v7). withPageLock FAIL-FAST кидает при не-UUID ключе ДО любой работы
  (комментарий-инвариант #260/#449) — забытый resolve/slugId больше не даёт тихую
  потерю сериализации под другим ключом. client.ts импортирует isUuid оттуда
  (убран локальный дубль — resolver и assert не разъедутся).
- mutatePage/replacePage seams стали async и сами вызывают resolvePageId — ключ
  лока/кэша канонический даже если вызывающий забыл (для уже-UUID это cached
  no-op; все 7 текущих вызывающих и так резолвят). replaceImage (один внешний
  лок + mutateLiveContentUnlocked) не тронут, deadlock невозможен.
- collab-session.ts: машинно-проверяемые маркеры MUTATE-CRITICAL-WINDOW
  BEGIN/END вокруг синхронного блока fromYdoc→applyDocToFragment (INVARIANT 1).
  Тест no-await-critical-window читает исходник и краснеет на await/yield в окне
  (проверено нейтером). Случайный await больше не тихо клоббит живые правки.
- Документация осознанной позиции: single-instance/sticky-sessions —
  требование деплоя (Dockerfile + README EN/RU + .env.example), т.к. мьютекс и
  stash — per-process. Окно устаревших прав (кэш-сессия пишет под токеном
  момента connect до MCP_COLLAB_SESSION_MAX_AGE_MS=10мин) — задокументированный
  trade-off в .env.example; push-инвалидации нет (осознанно).

Тесты: page-lock fail-fast (slugId/пусто/non-string → throw; канонический UUID
принят), no-away-guard, обновлённые фикстуры на валидные UUID. #449-специфичные
37/37 зелёные; mcp tsc чисто. closes #449.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 23:51:02 +03:00
vvzvlad bec2156e96 Merge pull request 'recovery: хвост стопки breaking-окна — #413, #415, #443 (3 части), #425 в develop' (#474) from feat/443-get-page-context into develop
Reviewed-on: #474
2026-07-10 23:49:29 +03:00
vvzvlad acc705de19 Merge pull request 'feat(mcp): drawio стадия 3 — семантические тулы fromGraph/fromMermaid/editCells (#425)' (#472) from feat/425-drawio-graph into feat/443-get-page-context
Reviewed-on: #472
2026-07-10 23:45:45 +03:00
vvzvlad a49872444e Merge pull request 'refactor(mcp)!: BREAKING — все имена MCP-тулов snake_case → camelCase (#412)' (#463) from refactor/412-camelcase-tool-names into develop
Reviewed-on: #463
2026-07-10 23:44:42 +03:00
vvzvlad ffc38ea2ca Merge pull request 'fix(mcp): pre-flight size-guard в diffDocs против CPU-DoS на больших правках (#464)' (#465) from fix/464-diffdocs-cpu-guard into develop
Reviewed-on: #465
2026-07-10 23:44:20 +03:00
agent_coder 2a951df096 feat(mcp): drawio стадия 3 — семантические тулы drawioFromGraph/FromMermaid/EditCells (#425)
Сырой XML остаётся escape-hatch, но для 90% случаев модель не видит ни координат,
ни style-строк — весь класс ошибок лейаута и иконок уходит by construction: эти
решения принимает сервер, а не LLM. Стоит на #443, переиспользует конвейер #423 и
shape-index/elkjs/линтер #424.

- drawioFromGraph(pageId, where, graph, direction?, preset?, layout?): граф
  узлов/групп/связей → резолв иконок (shape-index #424; неизвестная → generic по
  kind с подписью, не пустой квадрат), стили из пресета (kind→палитра),
  elkjs-layered с compound-группами, ассемблер XML (линтер-чистый by construction:
  зазоры >=150, прозрачные контейнеры, относительные координаты детей,
  cross-container рёбра parent=1, эскейп меток). Хинты pinned/sameLayerAs/layer и
  layout none/full/incremental — детерминированным post-pass'ом (ELK-констрейнты
  оказались ненадёжны). Пресеты default/dark/colorblind-safe (Okabe-Ito) — данные.
- drawioFromMermaid(pageId, where, mermaid): чистый парсер flowchart (без
  браузера/CLI) → graph → тот же конвейер. Формы/направление/пунктир=async/
  subgraph→группы/цепочки; не-flowchart отвергает внятно.
- drawioEditCells(pageId, node, operations, baseHash): ID-based add/update/delete,
  delete каскадит на детей контейнера и связанные рёбра; baseHash обязателен
  (optimistic lock как drawioUpdate); сентинелы 0/1 от delete защищены.

DoS-границы (LLM-вход): MAX_GRAPH_EDGES=1000/GROUPS=500 в validateGraph (узлы уже
500), mermaid MAX_CHARS=200k/LINES=20k/GROUPS=500, chain 500 — все с быстрым
throw ДО лейаута/ассемблера (ассемблер и маппер вне ELK-таймаута, иначе OOM
воркера). incremental сохраняет неперечисленные существующие ячейки (mergeExisting
Cells) — «добавь узел» не стирает ручную расстановку. sameLayerAs/layer после
снапа раскладываются по перпендикулярной оси с зазором >=150 → 0 quality-warnings;
pinned — точные пользовательские координаты (clamp>=0, могут дать warning, гарантия
«0 by construction» относится к авто-лейауту).

Регистрация: 3 shared-spec на оба хоста (camelCase, execute-in-spec, inlineBoth
Hosts не понадобился), DocmostClientLike/Method += 3, contract, routing-проза
(fromGraph→архитектуры/облака, fromMermaid→стандартные flowchart, raw xml→экзотика).

Тесты: mcp node --test 782/782 (57 новых) — приёмка (15+ узлов/2 вложенные группы/
AWS-иконки→0 lint/0 warnings/иконки резолвятся, hints, incremental без сдвига,
edit_cells update/cascade/stale-baseHash, снапшоты пресетов + colorblind-safe,
mermaid ветвление+subgraph) + регрессии на DoS-границы. tsc чисто; server jest
(contract + ai-chat) 290/290. closes #425.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 23:16:50 +03:00
agent_coder 5dc7a2703f feat(mcp): getPageContext — «где я / что вокруг» по pageId одним вызовом (#443, часть 3/3)
Финальная из трёх частей #443. Есть pageId (из поиска, из ссылки) — нужно понять
местоположение и окружение. Раньше это get_page + полное дерево. Только чтение,
только метаданные (без контента). closes #443.

- client.ts getPageContext(pageId): resolvePageId (slugId→uuid, для uuid без
  round-trip) → POST /pages/breadcrumbs + listSidebarPages(spaceId, uuid). Два
  запроса для uuid-входа (spaceId берётся из ответа breadcrumbs), +1 для slugId.
  Выход {page:{pageId,title,spaceId}, breadcrumbs:[{pageId,title}] (root→parent;
  [] для корня), children:[{pageId,title,hasChildren}]}.
- Порядок breadcrumbs выверен по серверному источнику (getPageBreadCrumbs: CTE
  сидится на childPageId, идёт вверх, .reverse() → root→page; страница —
  ПОСЛЕДНИЙ элемент). page = chain[last], breadcrumbs = chain.slice(0,-1); корень
  → []. Проекция явная — наружу только pageId (UUID), slugId/icon/position не
  утекают. Пустой chain / 404 / 403 → явная ошибка, не пустой объект.
- children — прямые дети через listSidebarPages (cursor-пагинация #442/#451,
  страница с >20 детьми отдаёт всех без дублей, порядок по position, hasChildren).
- Общий реестр: getPageContext на ОБОИХ хостах через цикл спеков (mcpName===
  inAppKey==="getPageContext", camelCase); DocmostClientLike/DocmostClientMethod
  += it; compile-time contract; TOOL_FAMILY READ + routing-проза.

Тесты: get-page-context.test.mjs +6 (3-й уровень split, no-slugId-leak + 2
запроса, корень→[], slugId-resolve, >20 детей без cap/дублей, плохой id→ошибка,
пустой chain→throw), tool-specs +1. mcp node --test 725/725, tsc чисто; server
jest (contract + ai-chat) 265/265.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 22:19:25 +03:00
agent_coder bfb4c8d8d0 feat(mcp): getTree — иерархия пространства/поддерева одним вызовом (#443, часть 2/3)
Раньше единственный способ получить дерево — listPages tree:true: BFS по
sidebar-эндпоинту, десятки-сотни HTTP-вызовов и молчаливая потеря страниц
(#442). Бэкенд форка уже отдаёт всё одним POST /pages/tree (getSidebarPagesTree,
merged #442/#451), так что это MCP-сторона + общий реестр. Только чтение.

- lib/tree.ts: buildPageTree(nodes) аддитивно расширен до buildPageTree(nodes,
  options?) с {shape?: "lean"|"getTree"; maxDepth?}. Дефолт/{} — байт-идентичный
  lean {id,slugId,title,children?} (существующие вызыватели listPages tree:true
  и subtree-BFS не тронуты, есть регрессионный тест на точный набор полей).
  shape:"getTree" проецирует {pageId, title, children?, hasChildren?} (id→pageId;
  slugId/icon/position/parentPageId не утекают ни на одной глубине).
- client.ts: getTree(spaceId, rootPageId?, maxDepth?) — тот же код-путь, что
  listPages tree:true: один enumerateSpacePages(spaceId, rootPageId) (единичный
  /pages/tree + cursor-BFS фолбэк для stock upstream) → buildPageTree(pages,
  {shape:"getTree", maxDepth}). listPages tree:true помечен DEPRECATED в JSDoc
  (BFS-фолбэк на месте, поведение не тронуто).
- maxDepth/hasChildren: полное дерево строится, потом обрезается. Корни = глубина
  1; maxDepth:1 → только корни; hasChildren:true ТОЛЬКО на срезанном узле с
  серверным hasChildren (source of truth), опущен у листьев и раскрытых узлов.
  Схема тула min(1) отбраковывает 0/отрицательные.
- tool-specs.ts: shared-спек getTree (оба хоста через цикл реестра),
  DocmostClientLike Pick += getTree, listPages desc/catalogLine — про депрекацию.
  server-instructions.ts: getTree:"READ" + routing-проза. Loader
  DocmostClientMethod += getTree, compile-time client-call contract += getTree.

Циклы/self-ref безопасны: project рекурсирует только от корней, циклический
компонент недостижим из корней (нет бесконечной рекурсии). Orphan (родитель
отфильтрован пермишенами) всплывает как корень.

Тесты: tree.test.mjs +9 (nesting+порядок+no-leak, maxDepth:1/2 + hasChildren,
orphan→root, seeded rootPageId, ≤0/нефинитный = без среза, lean-байт-идентичность),
tool-specs.test.mjs (схема getTree + депрекация listPages). mcp node --test
718/718, tsc чисто; server jest (shared-tool-specs.contract + ai-chat) 263/263.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 21:58:32 +03:00
agent_coder f794ac6d6c fix(search): оживить trgm-индекс — убрать coalesce из LIKE-предикатов (#443, ревью)
Ревьюер поймал EXPLAIN'ом: lookup-query фильтровал по
LOWER(f_unaccent(coalesce(col,''))), а GIN-trgm-индексы — на coalesce-free
LOWER(f_unaccent(col)). Postgres берёт функциональный индекс только при точном
совпадении выражения, так что обёртка coalesce отключала индекс → Seq Scan по
pages на КАЖДЫЙ lookup (MCP-клиент всегда шлёт substring:true). Зелёный
int-гейт это не ловил (проверял корректность на крошечном датасете, не
использование индекса).

- search.service.ts: убрал coalesce из двух index-driving LIKE-предикатов
  (title + text_content). Семантически эквивалентно: NULL LIKE '%q%' = NULL
  (falsy), NULL-страница просто не матчится — как пустая строка не матчит %q%.
  SELECT/ORDER BY оставлены с coalesce (индекс не выбирают, Node гардит NULL).
- Миграция: убран избыточный ре-ассерт idx_pages_title_trgm — его создаёт #348
  на том же coalesce-free выражении, теперь title-предикат его использует.
  idx_pages_text_content_trgm без изменений (уже coalesce-free), поправлены
  вводящие в заблуждение комментарии. down() дропает только text-индекс.
- Новый тест search-lookup-explain.int-spec.ts (3): title→idx_pages_title_trgm,
  text→idx_pages_text_content_trgm (оба ассертят отсутствие Seq Scan on pages),
  + негативный контроль (coalesce-обёртка не может использовать индекс) против
  тихой ре-регрессии. Дискриминатор enable_seqscan=off.
- CHANGELOG: две записи (opt-in substring lookup mode + смена shape MCP search).

EXPLAIN на реальном PG: обе fixed-ветки → Bitmap Index Scan on trgm; buggy
coalesce → Seq Scan (Disabled:true). Гейт: server jest src/core/search 27/27
(16 исходных int без изменений + 3 EXPLAIN + 8 unit), mcp node --test 708/708,
tsc чисто.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 21:57:16 +03:00
agent_coder e4e788f151 feat(search): агентский lookup-режим — substring по точкам/дефисам/цифрам, path, snippet, scope (#443, часть 1/3)
MCP-сервером пользуются LLM-агенты; основной паттерн — lookup: найти страницу по
обрывку технической строки и сразу понять, где она лежит и что в ней. Раньше на
такой вопрос уходило 3–5 вызовов. Эта часть закрывает `search` (часть 1 из 3;
get_tree и get_page_context — следующими PR стопкой). Только чтение.

Серверная часть — opt-in, веб-UI байт-в-байт неизменен:
- SearchDTO: опциональные substring/parentPageId/titleOnly (default-off; без
  флагов путь FTS не тронут — guard `if (substring) return searchPageLookup`).
- searchPageLookup: один скан pages, WHERE = title LIKE '%q%' OR (если не
  titleOnly) text LIKE '%q%' OR (если tsquery непустой) tsv @@ ... . LIKE-
  метасимволы %/_/\ экранируются (escapeLikePattern, ESCAPE '\') — `%`/`_` не
  матчат всё; substring-ветка работает даже при пустом tsquery (кейс 10.0.12).
- Ранжирование тирами (TITLE_EXACT > TITLE_SUBSTRING > TEXT), вторичный сигнал
  ts_rank / позиция; score∈(0,1] только для сортировки одной выдачи (формула в
  комментарии). 200-cap упорядочен по SQL-прокси тира ДО среза (иначе Postgres
  отдаёт произвольные 200 и сильный хит мог выпасть). Пермишен-фильтр к
  merged-набору ДО limit. path — одна рекурсивная CTE на все хиты (не N+1).
- snippet оконный в SQL (~500 символов вокруг первого совпадения). Позиция и
  срез в ОДНОМ пространстве LOWER(f_unaccent(...)) — f_unaccent не length-
  preserving (ß→ss, лигатуры, …→...), иначе окно смещалось/пустело. titleOnly →
  пустой snippet. Компромисс задокументирован.
- Миграция: GIN gin_trgm_ops по LOWER(f_unaccent(text_content)); title-trgm
  индекс #348 переиспользован (IF NOT EXISTS), down() дропает только новый.

MCP: схема search (spaceId/parentPageId/titleOnly/limit 1–50, default 10),
client.search прокидывает substring:true, filterSearchResult → {pageId, title,
path, snippet, score}. Инвариант: наружу только pageId (UUID), slugId/id
никогда. Комментарии про намеренное расхождение с in-app hybrid-RRF (не тронут)
и про деградацию на stock-upstream/Typesense (substring→plain FTS, без path/
snippet).

Проверка на реальном Postgres: server integration 16/16 (вся acceptance-таблица
#443 + регрессии на смещение snippet и cap-200), server unit 27/27, mcp
node --test 708/708, tsc чисто.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 21:34:53 +03:00
agent_coder 4be4a75fa3 docs(mcp): точные описания потерь getPage/exportPageMarkdown — конвертер канонический (#415)
Описания двух тулов устарели после #345 (канонический конвертер, #293/#351):
«LOSSY…approximated» у getPage и «lossless» у exportPageMarkdown искажали
роутинг агентов и молчаливо скрывали реальную потерю данных. Четвёртый линк
breaking-окна, стоит на #413.

- tool-specs.ts: getPage — вместо «LOSSY…approximated» точный закрытый список
  потерь (canonical for text; теряет лишь id блоков, resolved-якоря
  комментариев и фиксированный набор ACCEPTED-атрибутов без markdown-
  представления: спаны/colwidth/фон ячеек таблиц, indent, callout.icon,
  orderedList.type, link internal/target/rel/class). exportPageMarkdown —
  убрано «lossless»: round-trip перегенерирует id блоков и молча отбрасывает
  тот же набор (в первую очередь merge-спаны ячеек); держать в page-JSON, если
  нужны. Список выведен из источника истины (ATTR_VALUE_FUZZ_ALLOWLIST +
  MARK_ATTR_ALLOWLIST); opaque carried-verbatim токены (attachmentId, mime,
  slugId и пр.) намеренно НЕ в списке потерь — они round-trip'ятся.
- server-instructions.ts: READ-строка ROUTING_PROSE приведена к тому же
  точному списку, без противоречия markdown-default роутингу #413.
- Исправлена вторая латентная неточность: getPage описывался как
  сохраняющий resolved-якоря — но client.ts передаёт dropResolvedCommentAnchors:
  true, resolved-якоря скрыты (getNode, наоборот, их сохраняет — другой тул).
- README пакета (EN + RU-зеркало) приведены в соответствие passage-for-passage:
  убраны безоговорочные «lossless/lossy» для Markdown round-trip; genuinely-
  lossless ссылки на raw-JSON (getPageJson/getNode) оставлены.

Логику не трогает. server-instructions.test.mjs удалён и заменён
tool-inventory.test.mjs (без пинов на старую формулировку). mcp tsc чисто,
node --test 702/702, tool-inventory 5/5.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 20:55:28 +03:00
agent_coder e6171a1810 fix(mcp): сходимость orphan-сноски + дедуп block-id в markdown-splice (#413, ревью)
Правки по внутреннему ревью #413.

1. Orphan-сноска (нарушение «no second canon»). mergeFootnoteDefinitions
   раньше делал ранний return при пустых definitions, пропуская
   canonicalizeFootnotes. Если markdown-фрагмент БЕЗ сносок заменял/удалял
   блок, бывший последним referrer'ом существующей сноски, её определение
   оставалось orphan в хвостовом списке (полный ре-импорт его бы убрал).
   Теперь fast-path (возврат doc по ссылке без clone) только когда
   defs.length===0 И !hasFootnoteArtifacts(doc); иначе клон → append (no-op
   при пустых) → normalizeAndMergeFootnotes → canonicalizeFootnotes, как при
   полном импорте. Новый предикат hasFootnoteArtifacts обходит дерево на
   любой footnotesList/footnoteReference (существующие walk/isObject).
   Идемпотентно: несвязанный plain-патч на странице со сносками не трогает
   их топологию (canonicalizeFootnotes шаг 6 возвращает как есть).

2. Block-id disjoint от страницы. Новый экспорт reassignCollidingBlockIds
   (liveDoc, blocks, skipIndex?) в prosemirror-markdown/node-ops (переиспользует
   collectIds/makeFreshId) + barrel. Зовётся перед сплайсами в client.ts:
   patch — (liveDoc, threaded, 0) (skip 0 = блок, унаследовавший id цели);
   insert — (liveDoc, blocks) без skip. used-аккумулятор ловит и коллизию со
   страницей, и внутрифрагментную. freshBlockId/freshId без изменений.

Тесты (+6, markdown-patch-insert): orphan-repro (0 defs/0 list/0 refs +
docsCanonicallyEqual полному импорту), fast-path (footnote-free patch не трогает
топологию, соседи byte-identical), insert канонизирует при пустых defs со
страничной сноской, уникальность top-level block-id для patch 1→N и insert N.
mcp node --test 702/702, pmd vitest 736, tsc чисто.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 20:38:06 +03:00
agent_coder d269bd9efe feat(mcp): markdown — формат по умолчанию для блочных getNode/patchNode/insertNode (#413)
Канонический конвертер (#293/#345/#351) зрел для блочного уровня: markdown
становится дефолтом чтения/записи ОДНОГО блока, PM JSON — опция «для тонких
работ». Новых тулов нет, поверхность не растёт. Имена уже camelCase (стоит на #412).

- getNode(pageId, nodeId, format='markdown'): дефолт markdown — обёртка
  {type:doc,content:[node]} → convertProseMirrorToMarkdown, comment-якоря
  (ВКЛЮЧАЯ resolved) сохранены (это чтение под редактирование, не getPage).
  format:'json' — сырой сабтри. Авто-фолбэк не-топ-левел типов (tableRow/Cell/
  Header по #<index>) в JSON через canBeDocChild = docmostSchema.nodes.doc.
  contentMatch.matchType (не рукописный список); поле format на каждом ответе.
- patchNode/insertNode: XOR-вход {markdown?|node?} (оба optional в схеме, XOR
  на рантайме). markdown → импорт фрагмента → 1→N сплайс: первый блок наследует
  id цели, остальные свежие; dry replaceNodeById для #159-ambiguity ДО сплайса;
  соседние блоки byte-identical. Guard findUnrepresentableTableAttrs: цель со
  span/colwidth/backgroundColor → отказ с указанием на table-тулы/node-JSON.
  insertNode — insertNodesRelative (N блоков по порядку); голый tableRow/Cell
  JSON-only.
- Сноски: ^[...] во фрагменте → канон-импортёр; importMarkdownFragment делит
  блоки от footnotesList, РЕМАПИТ id сносок фрагмента в свежие uuid (fn-1
  фрагмента не коллизит с fn-1 страницы), mergeFootnoteDefinitions добавляет
  через ту же машинерию appendDefinition→normalizeAndMergeFootnotes→
  canonicalizeFootnotes, что insertFootnote. Сырые JSON-пути не тронуты.
- node-ops: replaceNodeByIdWithMany/insertNodesRelative (сплайс массива) в
  prosemirror-markdown/node-ops (канон после #414) + barrel.
- ROUTING_PROSE (READ/EDIT), compile-time client-call contract, CHANGELOG
  (getNode default→markdown, breaking для внешних клиентов, в окне #411/#412).

Тесты: сходимость (patchNode(markdown) блок docsCanonicallyEqual полному
импорту — нет «второго канона»); id-нить 1→N (первый наследует, остальные
свежие, соседи byte-identical); XOR; getNode markdown/json/non-top-level-fallback;
сохранение comment-якорей (active+resolved); ^[...]→хвостовой список+перенумерация;
guard. mcp node --test 697/697; pmd vitest 736; tsc чисто; server jest 273.
Третий линк breaking-окна, стоит на #412 (#411→#412→ЭТОТ→#415).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 20:10:38 +03:00
agent_coder 7cb3199d09 refactor(mcp)!: BREAKING — все имена MCP-тулов snake_case → camelCase (унификация с in-app) (#412)
Один логический тул жил под двумя именами: внешний MCP snake_case
(edit_page_text), in-app camelCase (editPageText) — дублирование доков, путаница
при переносе промптов/скиллов, помеха шарингу спек (#294). Решение владельца:
единый camelCase везде, включая внешний MCP. После этого mcpName === inAppKey.

- tool-specs.ts: mcpName ВЫВЕДЕН из ключа спеки (mcpName == inAppKey) для всех
  43 shared-спек — раньше divergent snake, теперь равен ключу (проверено: mcpName
  читается только структурно — цикл регистрации, генератор <tool_inventory>,
  TOOL_FAMILY). +5 inline-регистраций (tableGet/updateComment/deleteComment/
  docmostTransform; search без изменений). Рантайм: 47 тулов, все camelCase, ноль
  подчёркиваний.
- Контракт-конвенция ИНВЕРТИРОВАНА: shared-tool-specs.contract.spec
  `mcpName === toSnake(inAppKey)` → `mcpName === inAppKey`; tool-specs.test
  и tool-inventory.test обновлены.
- ROUTING_PROSE/TOOL_FAMILY/INLINE_MCP_INVENTORY (server-instructions.ts) →
  camelCase (105 замен). ai-chat.prompt/guard уже на in-app camelCase-ключах —
  без изменений (guard прошёл). comment-signal EXCLUDED_TOOLS схлопнут с
  дублей snake+camel до camelCase.
- Некоторое неочевидное: assertUnambiguousMatch(op: "patch_node"|"delete_node")
  в prosemirror-markdown/node-ops — op интерполируется в model-facing ошибку;
  литерал-юнион + call-sites → "patchNode"|"deleteNode".
- Все snake-имена в описаниях/error-строках/комментах/тестах/доках → camelCase
  (whole-token, longest-match-first). CHANGELOG: BREAKING-таблица 46 строк +
  миграция (allowlists mcp__gitmost-*__get_node→__getNode, промпты/скиллы,
  .mcp.json, метрики по tool-label); релизится вместе с #411.
  Внутренние имена методов (PageService.updatePageContent и т.п.) НЕ тронуты —
  переименованы только ИМЕНА ТУЛОВ.

Гейт: mcp node --test 677/677; tsc -p apps/server чисто; jest ai-chat-tools.
service + shared-tool-specs.contract + tool-tiers + ai-chat.prompt +
comment-signal-inapp → 323. Второй линк breaking-окна (#411→ЭТОТ→#413→#415).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 19:10:38 +03:00
104 changed files with 13881 additions and 5365 deletions
+34
View File
@@ -124,6 +124,40 @@ MCP_DOCMOST_PASSWORD=
# MCP_TOKEN= # MCP_TOKEN=
# MCP_SESSION_IDLE_MS=1800000 # MCP_SESSION_IDLE_MS=1800000
# #
# --- MCP collaboration write path: concurrency + rights-staleness (#449) ------
# MCP content writes (update_page, insert/replace nodes, comments-in-body, etc.)
# go over the collaboration websocket and are serialized PER PAGE by an
# in-process mutex (a module-level Map, one promise-chain per page UUID). This
# guarantees no two MCP writes on the SAME page overlap and clobber each other.
#
# DEPLOY REQUIREMENT — SINGLE INSTANCE or STICKY SESSIONS. The mutex is
# process-local. Behind a multi-replica load balancer WITHOUT sticky sessions,
# two replicas can each "hold" the lock for the same page at the same time and
# serialization is silently lost (concurrent full-document writes race on the
# live Yjs fragment). Run the MCP/app as a SINGLE instance, OR pin a page's
# traffic to one replica (sticky sessions / consistent hashing on page id). The
# same constraint applies to the RAM-only stash_page blob store above. There is
# deliberately no cross-process (e.g. Postgres advisory) lock yet — this is a
# CONSCIOUS documented constraint, not an oversight (#449).
#
# To reduce connect-storms the write path caches ONE live collab session per
# (wsUrl, page, token). Tunables (all optional; defaults are safe):
# MCP_COLLAB_SESSION_IDLE_MS=60000 # idle TTL, reset per op; 0 disables cache
# MCP_COLLAB_SESSION_MAX_ENTRIES=32 # LRU cap on cached sessions
# MCP_COLLAB_TOKEN_TTL_MS=300000 # per-client collab-token cache (5 min)
#
# RIGHTS-STALENESS TRADE-OFF. A cached collab session writes under the token
# captured at CONNECT time, and the collab-token cache reuses a token for its TTL.
# So if a user's access to a page is REVOKED, MCP writes on an already-open
# session may keep succeeding until the session ages out. MCP_COLLAB_SESSION_MAX_AGE_MS
# is the HARD lifetime (checked at each acquire) that BOUNDS this window: after it,
# the session is torn down and the next write re-auths with a fresh token, picking
# up the revocation. Default 10 min. LOWER it to shorten the revocation lag at the
# cost of more reconnects; RAISE it to reduce reconnects at the cost of a longer
# stale-rights window. There is intentionally no push-based cache invalidation on
# a rights change — this bounded window is the accepted trade-off (#449).
# MCP_COLLAB_SESSION_MAX_AGE_MS=600000
#
# BLOB SANDBOX (stash_page). An in-RAM, process-local store that hands large page # BLOB SANDBOX (stash_page). An in-RAM, process-local store that hands large page
# content + images to an external consumer WITHOUT bloating the model context or # content + images to an external consumer WITHOUT bloating the model context or
# requiring Docmost auth. The stash_page tool serializes a page, mirrors its # requiring Docmost auth. The stash_page tool serializes a page, mirrors its
+111 -7
View File
@@ -12,20 +12,108 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Breaking Changes ### Breaking Changes
- **External MCP tool names are now camelCase (all renamed).** Every tool on the
external `/mcp` surface was renamed from `snake_case` to `camelCase`, so the
external MCP name now matches the in-app tool name exactly (one logical tool,
one name everywhere). For example `get_node``getNode`, `edit_page_text`
`editPageText`, `patch_node``patchNode`. The tools' behaviour, inputs and
outputs are unchanged — only the names change. The single-word `search`
keeps its name.
*Migration (external MCP clients only — the in-app AI agent already used these
names and is unaffected):* update anything that refers to a tool by its
string name — permission allowlists (`mcp__gitmost-*__get_node`
`mcp__gitmost-*__getNode`), saved prompts/skills, `.mcp.json` tool filters,
and metrics dashboards that group by the `tool` label — and roll it out in
lockstep with this deploy, because the old snake_case names stop resolving.
Released together with the `import_page_markdown`/`update_page_markdown`
change below so external configs break exactly once.
Full mapping (old → new):
| Old (snake_case) | New (camelCase) |
| --- | --- |
| `check_new_comments` | `checkNewComments` |
| `copy_page_content` | `copyPageContent` |
| `create_comment` | `createComment` |
| `create_page` | `createPage` |
| `delete_comment` | `deleteComment` |
| `delete_node` | `deleteNode` |
| `delete_page` | `deletePage` |
| `diff_page_versions` | `diffPageVersions` |
| `docmost_transform` | `docmostTransform` |
| `drawio_create` | `drawioCreate` |
| `drawio_get` | `drawioGet` |
| `drawio_guide` | `drawioGuide` |
| `drawio_shapes` | `drawioShapes` |
| `drawio_update` | `drawioUpdate` |
| `edit_page_text` | `editPageText` |
| `export_page_markdown` | `exportPageMarkdown` |
| `get_node` | `getNode` |
| `get_outline` | `getOutline` |
| `get_page` | `getPage` |
| `get_page_json` | `getPageJson` |
| `get_workspace` | `getWorkspace` |
| `insert_footnote` | `insertFootnote` |
| `insert_image` | `insertImage` |
| `insert_node` | `insertNode` |
| `list_comments` | `listComments` |
| `list_page_history` | `listPageHistory` |
| `list_pages` | `listPages` |
| `list_shares` | `listShares` |
| `list_spaces` | `listSpaces` |
| `move_page` | `movePage` |
| `patch_node` | `patchNode` |
| `rename_page` | `renamePage` |
| `replace_image` | `replaceImage` |
| `resolve_comment` | `resolveComment` |
| `restore_page_version` | `restorePageVersion` |
| `search` | `search` (unchanged) |
| `search_in_page` | `searchInPage` |
| `share_page` | `sharePage` |
| `stash_page` | `stashPage` |
| `table_delete_row` | `tableDeleteRow` |
| `table_get` | `tableGet` |
| `table_insert_row` | `tableInsertRow` |
| `table_update_cell` | `tableUpdateCell` |
| `unshare_page` | `unsharePage` |
| `update_comment` | `updateComment` |
| `update_page_json` | `updatePageJson` |
| `update_page_markdown` | `updatePageMarkdown` |
(#412)
- **External MCP: `import_page_markdown` removed, `update_page_markdown` added.** - **External MCP: `import_page_markdown` removed, `update_page_markdown` added.**
The external `/mcp` surface no longer exposes `import_page_markdown` (the The external `/mcp` surface no longer exposes `importPageMarkdown` (the
round-trip parser for a self-contained *exported* Docmost-Markdown file). In round-trip parser for a self-contained *exported* Docmost-Markdown file). In
its place it now exposes **`update_page_markdown`** — a plain-Markdown its place it now exposes **`updatePageMarkdown`** — a plain-Markdown
full-body replace (`{pageId, content, title?}`) that pairs with full-body replace (`{pageId, content, title?}`) that pairs with
`update_page_json`, re-imports the whole body (block ids regenerate) and `updatePageJson`, re-imports the whole body (block ids regenerate) and
parses Docmost-flavoured markdown including `^[...]` inline footnotes. parses Docmost-flavoured markdown including `^[...]` inline footnotes.
*Migration:* MCP clients that called `import_page_markdown` to overwrite a *Migration:* MCP clients that called `importPageMarkdown` to overwrite a
page's body from Markdown should call `update_page_markdown` instead (pass the page's body from Markdown should call `updatePageMarkdown` instead (pass the
markdown as `content`). Round-tripping an exported Docmost-Markdown file with markdown as `content`). Round-tripping an exported Docmost-Markdown file with
comment anchors/diagrams is no longer available on the external MCP surface; comment anchors/diagrams is no longer available on the external MCP surface;
export remains via `export_page_markdown`. The in-app AI agent is unaffected — export remains via `exportPageMarkdown`. The in-app AI agent is unaffected —
it keeps both `importPageMarkdown` and the renamed `updatePageMarkdown` (was it keeps both `importPageMarkdown` and the renamed `updatePageMarkdown` (was
`updatePageContent`). The total MCP tool count is unchanged (−1 / +1). (#411) `updatePageContent`). The total MCP tool count is unchanged (−1 / +1). The
external names shown here are the post-#412 camelCase names. (#411)
- **`getNode` now returns Markdown by default (was ProseMirror JSON).** The
block-level read/write tools default to Markdown so a block round trip is
`getNode` (markdown) → edit → `patchNode` (markdown). `getNode` now returns
`{ …, format: "markdown", markdown }` unless you pass `format: "json"` (which
restores the previous `{ …, node }` ProseMirror subtree); comment anchors —
including resolved ones — are preserved in the markdown so a write-back never
orphans a thread, and a node that cannot be a document top-level block
(`tableRow`/`tableCell`/`tableHeader` addressed via `#<index>`) auto-falls back
to JSON with `format: "json"` in the response. `patchNode`/`insertNode` gain a
`markdown` input alongside `node` (provide exactly one): the markdown fragment
may rewrite/insert several blocks at once and supports `^[...]` footnotes.
*Migration (external MCP clients only):* a client that consumed `getNode`'s
`node` field must now either read `markdown`, or pass `format: "json"` to keep
the old ProseMirror-JSON output. Released together with the `#411`/`#412`
breaking window so external configs break exactly once. (#413)
### Added ### Added
@@ -163,6 +251,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
by physical key position and matched against the commands; genuine Cyrillic by physical key position and matched against the commands; genuine Cyrillic
search terms keep priority over remapped candidates, and short wrong-layout search terms keep priority over remapped candidates, and short wrong-layout
prefixes match by command title. (#283, #285, #287) prefixes match by command title. (#283, #285, #287)
- **Opt-in substring "lookup" search mode for agents.** `/api/search` gains an
additive, opt-in mode (guarded by a new `substring` flag) that matches literal
substrings of page titles and body text — so technical tokens the full-text
tokenizer mangles (`backup-srv.local`, `10.0.12.5`, `WB-MGE-30D86B`) are found
even when the FTS query is empty. It returns a location `path`, a windowed
`snippet` and a per-response relevance `score`, supports `titleOnly` and a
`parentPageId` subtree scope, and applies the page-level permission filter
before the limit. The web UI never sets `substring`, so its full-text search
behaviour is byte-for-byte unchanged. The leading-wildcard `LIKE` predicates
are backed by GIN trigram indexes on `LOWER(f_unaccent(title))` and
`LOWER(f_unaccent(text_content))` so lookups use a bitmap index scan instead of
a sequential scan. (#443)
- **MCP `search` tool returns richer, agent-oriented results.** The external MCP
`search` response shape changes for the agent surface: each hit now carries
`pageId` (renamed from `id`), plus `path`, `snippet` and `score`; the
UI-oriented `spaceId`, `rank` and `highlight` fields are dropped. (#443)
### Changed ### Changed
+10
View File
@@ -81,4 +81,14 @@ VOLUME ["/app/data/storage"]
EXPOSE 3000 EXPOSE 3000
# DEPLOY REQUIREMENT — SINGLE INSTANCE or STICKY SESSIONS (#449).
# MCP content writes are serialized per page by an IN-PROCESS mutex, and the
# stash_page blob store + cached collab sessions are RAM-only and process-local.
# Running MULTIPLE replicas of this image behind a load balancer WITHOUT sticky
# sessions silently breaks per-page write serialization (two replicas can lock
# the same page at once) and makes stash_page blobs unreachable across replicas.
# Run a SINGLE instance, or pin each page's traffic to one replica (sticky
# sessions / consistent hashing on page id). There is deliberately no
# cross-process lock yet — a conscious constraint. See .env.example (the "MCP
# collaboration write path" block) and packages/mcp/README.md for details.
CMD ["pnpm", "start"] CMD ["pnpm", "start"]
@@ -25,7 +25,7 @@ const mockLoaded = (DocmostClient: loader.DocmostClientCtor) => ({
DocmostClient, DocmostClient,
sharedToolSpecs: SHARED_TOOL_SPECS as unknown as Record<string, loader.SharedToolSpec>, sharedToolSpecs: SHARED_TOOL_SPECS as unknown as Record<string, loader.SharedToolSpec>,
// Pure no-network draw.io helpers (#424). Type-correct stubs: these tests // Pure no-network draw.io helpers (#424). Type-correct stubs: these tests
// never execute the drawio_shapes / drawio_guide tool bodies. // never execute the drawioShapes / drawioGuide tool bodies.
searchShapes: (() => []) as unknown as loader.SearchShapesFn, searchShapes: (() => []) as unknown as loader.SearchShapesFn,
getGuideSection: (() => ({ getGuideSection: (() => ({
section: 'index', section: 'index',
@@ -355,23 +355,32 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
content: [{ type: 'text', text: 'Hello' }], content: [{ type: 'text', text: 'Hello' }],
}; };
it('patchNode parses a JSON-string node and forwards it as an object', async () => { it('patchNode parses a JSON-string node and forwards it as { node } (object)', async () => {
const tools = await buildTools(); const tools = await buildTools();
await tools.patchNode.execute( await tools.patchNode.execute(
{ pageId: 'p1', nodeId: 'n1', node: JSON.stringify(NODE_OBJ) } as never, { pageId: 'p1', nodeId: 'n1', node: JSON.stringify(NODE_OBJ) } as never,
{} as never, {} as never,
); );
expect(patchNodeCalls).toHaveLength(1); expect(patchNodeCalls).toHaveLength(1);
expect(patchNodeCalls[0]).toEqual(['p1', 'n1', NODE_OBJ]); // #413: the 3rd arg is now the XOR input { markdown?, node? }.
expect(patchNodeCalls[0]).toEqual([
'p1',
'n1',
{ markdown: undefined, node: NODE_OBJ },
]);
}); });
it('patchNode passes an object node through unchanged', async () => { it('patchNode passes an object node through unchanged inside { node }', async () => {
const tools = await buildTools(); const tools = await buildTools();
await tools.patchNode.execute( await tools.patchNode.execute(
{ pageId: 'p1', nodeId: 'n1', node: NODE_OBJ } as never, { pageId: 'p1', nodeId: 'n1', node: NODE_OBJ } as never,
{} as never, {} as never,
); );
expect(patchNodeCalls[0]).toEqual(['p1', 'n1', NODE_OBJ]); expect(patchNodeCalls[0]).toEqual([
'p1',
'n1',
{ markdown: undefined, node: NODE_OBJ },
]);
}); });
it('patchNode throws the documented message on invalid JSON string', async () => { it('patchNode throws the documented message on invalid JSON string', async () => {
@@ -385,7 +394,7 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
expect(patchNodeCalls).toHaveLength(0); expect(patchNodeCalls).toHaveLength(0);
}); });
it('insertNode parses a JSON-string node and forwards it as an object', async () => { it('insertNode parses a JSON-string node and forwards it inside { node }', async () => {
const tools = await buildTools(); const tools = await buildTools();
await tools.insertNode.execute( await tools.insertNode.execute(
{ {
@@ -396,9 +405,15 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
{} as never, {} as never,
); );
expect(insertNodeCalls).toHaveLength(1); expect(insertNodeCalls).toHaveLength(1);
const [pageId, node] = insertNodeCalls[0]; // #413: the 2nd arg is the XOR input { markdown?, node? }, the 3rd is opts.
const [pageId, input, opts] = insertNodeCalls[0] as [
string,
{ markdown?: unknown; node?: unknown },
{ position?: string },
];
expect(pageId).toBe('p1'); expect(pageId).toBe('p1');
expect(node).toEqual(NODE_OBJ); expect(input).toEqual({ markdown: undefined, node: NODE_OBJ });
expect(opts.position).toBe('append');
}); });
it('insertNode throws the documented message on invalid JSON string', async () => { it('insertNode throws the documented message on invalid JSON string', async () => {
@@ -912,7 +927,7 @@ describe('AiChatToolsService getCurrentPage selection (#388)', () => {
}); });
/** /**
* #440 review: the in-app drawio_create / drawio_update handlers must forward * #440 review: the in-app drawioCreate / drawioUpdate handlers must forward
* the optional `layout:"elk"` param to the client (5th positional arg), exactly * the optional `layout:"elk"` param to the client (5th positional arg), exactly
* like the MCP host. It was silently dropped, so ELK auto-layout worked only via * like the MCP host. It was silently dropped, so ELK auto-layout worked only via
* the standalone MCP server, not in-app. These tests pin per-host parity. * the standalone MCP server, not in-app. These tests pin per-host parity.
@@ -59,10 +59,12 @@ function __assertClientCallContract(client: DocmostClientLike): void {
void client.getWorkspace(); void client.getWorkspace();
void client.getSpaces(); void client.getSpaces();
void client.listPages(s, n, true); void client.listPages(s, n, true);
void client.getTree(s, s, n);
void client.getPageContext(s);
void client.listSidebarPages(s, s); void client.listSidebarPages(s, s);
void client.getOutline(s); void client.getOutline(s);
void client.getPageJson(s); void client.getPageJson(s);
void client.getNode(s, s); void client.getNode(s, s, 'markdown');
void client.searchInPage(s, s, { void client.searchInPage(s, s, {
regex: true, regex: true,
caseSensitive: true, caseSensitive: true,
@@ -84,12 +86,16 @@ function __assertClientCallContract(client: DocmostClientLike): void {
void client.movePage(s, s, s); void client.movePage(s, s, s);
void client.deletePage(s); void client.deletePage(s);
void client.editPageText(s, edits); void client.editPageText(s, edits);
void client.patchNode(s, s, node); void client.patchNode(s, s, { markdown: s, node });
void client.insertNode(s, node, { void client.insertNode(
s,
{ markdown: s, node },
{
position: 'append', position: 'append',
anchorNodeId: s, anchorNodeId: s,
anchorText: s, anchorText: s,
}); },
);
void client.deleteNode(s, s); void client.deleteNode(s, s);
void client.updatePageJson(s, node, s); void client.updatePageJson(s, node, s);
void client.tableInsertRow(s, s, cells, n); void client.tableInsertRow(s, s, cells, n);
@@ -117,6 +123,23 @@ function __assertClientCallContract(client: DocmostClientLike): void {
void client.drawioGet(s, s, 'xml'); void client.drawioGet(s, s, 'xml');
void client.drawioCreate(s, { position: 'append', anchorNodeId: s }, s, s, 'elk'); void client.drawioCreate(s, { position: 'append', anchorNodeId: s }, s, s, 'elk');
void client.drawioUpdate(s, s, s, s, 'elk'); void client.drawioUpdate(s, s, s, s, 'elk');
// --- draw.io high-level semantic tools (#425 stage 3) ---
void client.drawioEditCells(s, s, [{ op: 'delete', cellId: s }], s);
void client.drawioFromGraph(
s,
{ position: 'append', anchorNodeId: s },
{ nodes: [{ id: s, label: s }] },
'LR',
s,
'full',
s,
);
void client.drawioFromMermaid(
s,
{ position: 'append', anchorNodeId: s },
s,
s,
);
// --- write (comment) --- // --- write (comment) ---
void client.createComment(s, s, 'inline', s, s, s); void client.createComment(s, s, 'inline', s, s, s);
void client.resolveComment(s, true); void client.resolveComment(s, true);
@@ -266,7 +289,7 @@ export class AiChatToolsService {
// construction is shared with the page-change detection path (#274) via // construction is shared with the page-change detection path (#274) via
// buildDocmostClient so both go over the exact same authenticated route. // buildDocmostClient so both go over the exact same authenticated route.
// searchShapes / getGuideSection (#424) are the PURE, no-network helpers // searchShapes / getGuideSection (#424) are the PURE, no-network helpers
// backing drawio_shapes / drawio_guide. They are `inlineBothHosts` specs (no // backing drawioShapes / drawioGuide. They are `inlineBothHosts` specs (no
// canonical execute — their catalog loader uses import.meta and can't be // canonical execute — their catalog loader uses import.meta and can't be
// value-imported into the zod-agnostic tool-specs.ts under the server's // value-imported into the zod-agnostic tool-specs.ts under the server's
// commonjs type-check), so the shared registry loop below SKIPS them and this // commonjs type-check), so the shared registry loop below SKIPS them and this
@@ -308,9 +331,10 @@ export class AiChatToolsService {
// The in-app toolset. It starts with the tools kept INLINE here for a // The in-app toolset. It starts with the tools kept INLINE here for a
// documented per-layer reason: an intentional behaviour/schema divergence from // documented per-layer reason: an intentional behaviour/schema divergence from
// the standalone MCP surface (searchPages' hybrid RRF, // the standalone MCP surface (searchPages' hybrid RRF,
// transformPage's guardrailed shorter schema), a // transformPage's guardrailed shorter schema), a name clash the shared
// snake_case/camelCase naming clash the shared registry forbids (getTable vs // registry forbids (in-app `getTable` verb-first vs the MCP noun-first
// the MCP `table_get`), per-request state the registry loop cannot provide // `tableGet` — the registry requires mcpName === inAppKey), per-request
// state the registry loop cannot provide
// (getCurrentPage reads the resolved openedPage; searchPages closes over the // (getCurrentPage reads the resolved openedPage; searchPages closes over the
// per-request user/embedding deps), or a tool with no MCP twin // per-request user/embedding deps), or a tool with no MCP twin
// (listSidebarPages/getComment/getPageHistory). Every SHARED tool is then added // (listSidebarPages/getComment/getPageHistory). Every SHARED tool is then added
@@ -477,9 +501,9 @@ export class AiChatToolsService {
await client.listSidebarPages(spaceId, pageId), await client.listSidebarPages(spaceId, pageId),
}), }),
// NOT shared (kept inline): the MCP tool name `table_get` is noun-first // NOT shared (kept inline): the MCP tool name `tableGet` is noun-first
// while this key is `getTable` (verb-first), breaking the // while this key is `getTable` (verb-first), so it cannot satisfy the
// snake_case(inAppKey) convention the shared registry enforces. Its // shared registry's `mcpName === inAppKey` convention (#412). Its
// reference parameter is still named `table` (was `tableRef`) so it matches // reference parameter is still named `table` (was `tableRef`) so it matches
// the migrated table row/cell tools below. // the migrated table row/cell tools below.
getTable: tool({ getTable: tool({
@@ -522,7 +546,7 @@ export class AiChatToolsService {
// INTENTIONAL per-transport divergence (not shared): deliberately omits the // INTENTIONAL per-transport divergence (not shared): deliberately omits the
// `deleteComments` schema field (comment-deletion guardrail) and carries a // `deleteComments` schema field (comment-deletion guardrail) and carries a
// much shorter description; the standalone MCP `docmost_transform` exposes // much shorter description; the standalone MCP `docmostTransform` exposes
// the full helper catalogue. Different schema, so kept per-layer. // the full helper catalogue. Different schema, so kept per-layer.
transformPage: tool({ transformPage: tool({
description: description:
@@ -553,7 +577,7 @@ export class AiChatToolsService {
// WHICH mapping to run and returns its value directly (no envelope). For each // WHICH mapping to run and returns its value directly (no envelope). For each
// spec: // spec:
// - skip `mcpOnly` specs (they belong to the standalone MCP host only); // - skip `mcpOnly` specs (they belong to the standalone MCP host only);
// - skip `inlineBothHosts` specs (drawio_shapes / drawio_guide): they carry // - skip `inlineBothHosts` specs (drawioShapes / drawioGuide): they carry
// no execute and are wired INLINE just below, calling the pure helpers; // no execute and are wired INLINE just below, calling the pure helpers;
// - use `inAppExecute` when the spec declares a DELIBERATE per-layer // - use `inAppExecute` when the spec declares a DELIBERATE per-layer
// difference (a projected result shape, a different guardrail message); // difference (a projected result shape, a different guardrail message);
@@ -574,7 +598,7 @@ export class AiChatToolsService {
); );
} }
// drawio_shapes / drawio_guide (#424): `inlineBothHosts` registry specs wired // drawioShapes / drawioGuide (#424): `inlineBothHosts` registry specs wired
// here with the SAME schema+description the shared spec pins, but calling the // here with the SAME schema+description the shared spec pins, but calling the
// pure searchShapes / getGuideSection helpers off the loaded @docmost/mcp // pure searchShapes / getGuideSection helpers off the loaded @docmost/mcp
// module — they are not client methods and their catalog loader uses // module — they are not client methods and their catalog loader uses
@@ -23,6 +23,8 @@ type DocmostClientMethod =
| 'getWorkspace' | 'getWorkspace'
| 'getSpaces' | 'getSpaces'
| 'listPages' | 'listPages'
| 'getTree'
| 'getPageContext'
| 'listSidebarPages' | 'listSidebarPages'
| 'getOutline' | 'getOutline'
| 'getPageJson' | 'getPageJson'
@@ -69,6 +71,10 @@ type DocmostClientMethod =
| 'drawioGet' | 'drawioGet'
| 'drawioCreate' | 'drawioCreate'
| 'drawioUpdate' | 'drawioUpdate'
// --- draw.io high-level semantic tools (#425 stage 3) ---
| 'drawioEditCells'
| 'drawioFromGraph'
| 'drawioFromMermaid'
// --- write (comment) --- // --- write (comment) ---
| 'createComment' | 'createComment'
| 'resolveComment'; | 'resolveComment';
@@ -146,7 +152,7 @@ export type CommentSignalTrackerFactory = (options: {
// Pure, no-network draw.io helpers (#424). These are plain functions on the // Pure, no-network draw.io helpers (#424). These are plain functions on the
// module (NOT DocmostClient methods) — the in-app AI-SDK service calls them // module (NOT DocmostClient methods) — the in-app AI-SDK service calls them
// directly to wire drawio_shapes / drawio_guide, mirroring the MCP server. // directly to wire drawioShapes / drawioGuide, mirroring the MCP server.
export type SearchShapesFn = ( export type SearchShapesFn = (
query: string, query: string,
opts?: { category?: string; limit?: number }, opts?: { category?: string; limit?: number },
@@ -169,7 +175,7 @@ interface DocmostMcpModule {
// the mocked loader in unit tests) — the stale-check below is a NO-OP when it // the mocked loader in unit tests) — the stale-check below is a NO-OP when it
// is missing, so an older build never wrongly fails startup. // is missing, so an older build never wrongly fails startup.
REGISTRY_STAMP?: string; REGISTRY_STAMP?: string;
// Pure, no-network draw.io helpers (#424) backing drawio_shapes / drawio_guide. // Pure, no-network draw.io helpers (#424) backing drawioShapes / drawioGuide.
// Those two specs are `inlineBothHosts` (they stay in SHARED_TOOL_SPECS for the // Those two specs are `inlineBothHosts` (they stay in SHARED_TOOL_SPECS for the
// shared contract but carry no execute — their catalog loader uses import.meta // shared contract but carry no execute — their catalog loader uses import.meta
// and can't be value-imported into the zod-agnostic tool-specs.ts), so the // and can't be value-imported into the zod-agnostic tool-specs.ts), so the
@@ -17,8 +17,10 @@ import { SHARED_TOOL_SPECS } from '../../../../../../packages/mcp/src/tool-specs
* This test fails the build if a spec is added to the registry but never wired * This test fails the build if a spec is added to the registry but never wired
* in-app, if an `inAppKey` is renamed without updating the service, if the * in-app, if an `inAppKey` is renamed without updating the service, if the
* description drifts between the registry and the exposed tool, if the * description drifts between the registry and the exposed tool, if the
* snake_case `mcpName` <-> camelCase `inAppKey` convention is broken, or if the * `mcpName === inAppKey` convention is broken (issue #412 unified the external
* exposed tool's input-schema keys diverge from the spec's `buildShape`. * MCP tool name with the in-app key — both are the same camelCase identifier),
* or if the exposed tool's input-schema keys diverge from the spec's
* `buildShape`.
* *
* It does NOT need @docmost/mcp built: the registry is imported from TS source, * It does NOT need @docmost/mcp built: the registry is imported from TS source,
* and the ESM loader is mocked so `forUser()` never dynamically imports the * and the ESM loader is mocked so `forUser()` never dynamically imports the
@@ -74,10 +76,6 @@ describe('SHARED_TOOL_SPECS contract parity', () => {
afterAll(() => jest.restoreAllMocks()); afterAll(() => jest.restoreAllMocks());
// camelCase -> snake_case, matching the registry's mcpName convention.
const toSnake = (s: string) =>
s.replace(/[A-Z]/g, (c) => `_${c.toLowerCase()}`);
// Type as the (optional-buildShape) SharedToolSpec; the `satisfies` literal // Type as the (optional-buildShape) SharedToolSpec; the `satisfies` literal
// above otherwise narrows to a union where some members lack buildShape. // above otherwise narrows to a union where some members lack buildShape.
const specEntries = Object.entries(SHARED_TOOL_SPECS) as unknown as Array< const specEntries = Object.entries(SHARED_TOOL_SPECS) as unknown as Array<
@@ -96,8 +94,8 @@ describe('SHARED_TOOL_SPECS contract parity', () => {
expect(spec.inAppKey).toBe(registryKey); expect(spec.inAppKey).toBe(registryKey);
}); });
it('mcpName is the snake_case form of inAppKey', () => { it('mcpName equals inAppKey (unified camelCase name, #412)', () => {
expect(spec.mcpName).toBe(toSnake(spec.inAppKey)); expect(spec.mcpName).toBe(spec.inAppKey);
}); });
it('is exposed in-app under its inAppKey', () => { it('is exposed in-app under its inAppKey', () => {
@@ -36,7 +36,7 @@ describe('tool tier metadata (#332)', () => {
}); });
it('#410 image tools are DEFERRED, footnote tool is CORE', () => { it('#410 image tools are DEFERRED, footnote tool is CORE', () => {
// insert_footnote is core (symmetric with editPageText); the image tools stay // insertFootnote is core (symmetric with editPageText); the image tools stay
// deferred (rare, fat — loaded on demand). Assert both the spec tier and the // deferred (rare, fat — loaded on demand). Assert both the spec tier and the
// CORE_TOOL_SET membership so a future tier edit that desyncs them fails here. // CORE_TOOL_SET membership so a future tier edit that desyncs them fails here.
expect(SHARED_TOOL_SPECS.insertFootnote.tier).toBe('core'); expect(SHARED_TOOL_SPECS.insertFootnote.tier).toBe('core');
@@ -60,10 +60,10 @@ export const CORE_TOOL_KEYS = [
'listComments', 'listComments',
'resolveComment', 'resolveComment',
'editPageText', 'editPageText',
// #330 search_in_page — frequent for editorial sweeps; core despite predating // #330 searchInPage — frequent for editorial sweeps; core despite predating
// the issue's tier list. // the issue's tier list.
'searchInPage', 'searchInPage',
// #410 insert_footnote — core so pinpoint citations to already-written text // #410 insertFootnote — core so pinpoint citations to already-written text
// don't degrade into literal `^[...]`; kept symmetric with editPageText. // don't degrade into literal `^[...]`; kept symmetric with editPageText.
'insertFootnote', 'insertFootnote',
] as const; ] as const;
@@ -138,7 +138,7 @@ export const INLINE_TOOL_TIERS: Record<
}, },
// NOTE: tableInsertRow, tableDeleteRow and tableUpdateCell moved to // NOTE: tableInsertRow, tableDeleteRow and tableUpdateCell moved to
// @docmost/mcp's SHARED_TOOL_SPECS (#294); they carry their own deferred tier + // @docmost/mcp's SHARED_TOOL_SPECS (#294); they carry their own deferred tier +
// catalogLine there. getTable stays inline (its MCP name table_get breaks the // catalogLine there. getTable stays inline (its MCP name tableGet breaks the
// snake_case(inAppKey) convention, so it has no shared spec). // snake_case(inAppKey) convention, so it has no shared spec).
// NOTE: checkNewComments moved to @docmost/mcp's SHARED_TOOL_SPECS (#294); // NOTE: checkNewComments moved to @docmost/mcp's SHARED_TOOL_SPECS (#294);
// it carries its own deferred tier + catalogLine there. // it carries its own deferred tier + catalogLine there.
@@ -150,7 +150,7 @@ export const INLINE_TOOL_TIERS: Record<
// NOTE: sharePage moved to @docmost/mcp's SHARED_TOOL_SPECS (#294); it carries // NOTE: sharePage moved to @docmost/mcp's SHARED_TOOL_SPECS (#294); it carries
// its own deferred tier + catalogLine there. transformPage stays inline (its // its own deferred tier + catalogLine there. transformPage stays inline (its
// schema deliberately diverges — it omits the deleteComments field the MCP // schema deliberately diverges — it omits the deleteComments field the MCP
// docmost_transform exposes, a comment-deletion guardrail). // docmostTransform exposes, a comment-deletion guardrail).
transformPage: { transformPage: {
tier: 'deferred', tier: 'deferred',
catalogLine: "transformPage — run a sandboxed JS transform over a page's document.", catalogLine: "transformPage — run a sandboxed JS transform over a page's document.",
@@ -12,3 +12,22 @@ export class SearchResponseDto {
updatedAt: Date; updatedAt: Date;
space: Partial<Space>; space: Partial<Space>;
} }
// Response shape for the opt-in agent-lookup mode (#443, `substring: true`).
// Additive to the FTS response: carries the location (`path`), a windowed
// `snippet` around the first match and a per-response sort `score`. The MCP
// layer maps `id → pageId`; `slugId` is never exposed.
export class SearchLookupResponseDto {
id: string;
slugId: string;
title: string;
parentPageId: string | null;
// Ancestor titles from the space root down to the direct parent; [] for a
// root page.
path: string[];
// ~300–500 chars around the first match (or a leading text window / extended
// ts_headline fallback).
snippet: string;
// 0..1 float, meaningful ONLY for sorting within one response.
score: number;
}
@@ -30,6 +30,31 @@ export class SearchDTO {
@IsOptional() @IsOptional()
@IsNumber() @IsNumber()
offset?: number; offset?: number;
// --- Opt-in agent-lookup mode (#443). ------------------------------------
// These fields are ADDITIVE and default-off: a web client that sends none of
// them gets byte-identical FTS behaviour and result shape. They are only read
// by the substring/path/snippet code path in SearchService.searchPage.
//
// NOTE (standalone stdio vs stock upstream): stock upstream validates this DTO
// with `whitelist: true`, so an older server silently strips these unknown
// fields and the request degrades gracefully to the plain FTS behaviour.
// Enables the hybrid substring branch (title + text_content LIKE) merged with
// the existing FTS branch, plus tiered ranking, path and windowed snippet.
@IsOptional()
@IsBoolean()
substring?: boolean;
// Restrict the search to a page and all of its descendants (inclusive).
@IsOptional()
@IsString()
parentPageId?: string;
// Match titles only; do not scan text_content.
@IsOptional()
@IsBoolean()
titleOnly?: boolean;
} }
export class SearchShareDTO extends SearchDTO { export class SearchShareDTO extends SearchDTO {
@@ -60,6 +60,12 @@ export class SearchController {
} }
} }
// #443 graceful degradation: on EE/Typesense instances the request routes to
// the Typesense backend, which does NOT implement the opt-in agent-lookup
// mode. The `substring`/`parentPageId`/`titleOnly` fields are silently ignored
// and the response carries no `path`/`snippet`/`score` and no substring/tier
// ranking — it degrades to plain Typesense FTS. The native lookup mode below
// is Postgres-search-driver only.
if (this.environmentService.getSearchDriver() === 'typesense') { if (this.environmentService.getSearchDriver() === 'typesense') {
return this.searchTypesense(searchDto, { return this.searchTypesense(searchDto, {
userId: user.id, userId: user.id,
@@ -0,0 +1,95 @@
import {
computeLookupScore,
escapeLikePattern,
SearchLookupTier,
} from './search.service';
/**
* Pure-function coverage for the #443 agent-lookup helpers:
* - escapeLikePattern: LIKE-metacharacter escaping so `%`/`_`/`\` are literals
* (the acceptance-table requirement that a query of `%` or `_` does NOT match
* everything);
* - computeLookupScore: the tiered 0..1 ranking score, where a stronger tier
* always outranks a weaker one regardless of the in-tier secondary signal.
*
* The DB-touching branch (substring UNION FTS, path CTE, snippet window) is
* covered by the integration spec against the real schema.
*/
describe('escapeLikePattern', () => {
it('escapes the LIKE metacharacters % _ and \\', () => {
expect(escapeLikePattern('%')).toBe('\\%');
expect(escapeLikePattern('_')).toBe('\\_');
expect(escapeLikePattern('\\')).toBe('\\\\');
});
it('escapes the backslash FIRST so it does not double-escape %/_', () => {
// Input `\%` must become `\\` + `\%` = `\\\%`, not `\\%`.
expect(escapeLikePattern('\\%')).toBe('\\\\\\%');
});
it('leaves ordinary technical chars (. - / digits) untouched', () => {
expect(escapeLikePattern('backup-srv.local')).toBe('backup-srv.local');
expect(escapeLikePattern('10.0.12')).toBe('10.0.12');
expect(escapeLikePattern('WB-MGE-30D86B')).toBe('WB-MGE-30D86B');
expect(escapeLikePattern('a/b')).toBe('a/b');
});
it('escapes only the metacharacters in a mixed string', () => {
expect(escapeLikePattern('50%_off.zip')).toBe('50\\%\\_off.zip');
});
it('is null/undefined-safe', () => {
expect(escapeLikePattern(undefined as any)).toBe('');
expect(escapeLikePattern(null as any)).toBe('');
});
});
describe('computeLookupScore', () => {
it('keeps every score within (0, 1]', () => {
for (const tier of [
SearchLookupTier.TITLE_EXACT,
SearchLookupTier.TITLE_SUBSTRING,
SearchLookupTier.TEXT,
]) {
for (const secondary of [0, 0.001, 1, 100, 1e6]) {
const s = computeLookupScore({ tier, secondary });
expect(s).toBeGreaterThan(0);
expect(s).toBeLessThanOrEqual(1);
}
}
});
it('a stronger tier ALWAYS outranks a weaker tier, whatever the secondary', () => {
// Weak tier with a huge secondary must still lose to a strong tier with a
// tiny secondary — tiers dominate.
const strongLowSecondary = computeLookupScore({
tier: SearchLookupTier.TITLE_EXACT,
secondary: 0,
});
const weakHighSecondary = computeLookupScore({
tier: SearchLookupTier.TEXT,
secondary: 1e9,
});
expect(strongLowSecondary).toBeGreaterThan(weakHighSecondary);
});
it('within a tier a larger secondary sorts higher', () => {
const lo = computeLookupScore({
tier: SearchLookupTier.TEXT,
secondary: 0.1,
});
const hi = computeLookupScore({
tier: SearchLookupTier.TEXT,
secondary: 5,
});
expect(hi).toBeGreaterThan(lo);
});
it('treats a negative/absent secondary as 0', () => {
const zero = computeLookupScore({ tier: SearchLookupTier.TEXT, secondary: 0 });
expect(computeLookupScore({ tier: SearchLookupTier.TEXT })).toBe(zero);
expect(
computeLookupScore({ tier: SearchLookupTier.TEXT, secondary: -5 }),
).toBe(zero);
});
});
+401 -2
View File
@@ -1,6 +1,9 @@
import { Injectable } from '@nestjs/common'; import { Injectable } from '@nestjs/common';
import { SearchDTO, SearchSuggestionDTO } from './dto/search.dto'; import { SearchDTO, SearchSuggestionDTO } from './dto/search.dto';
import { SearchResponseDto } from './dto/search-response.dto'; import {
SearchLookupResponseDto,
SearchResponseDto,
} from './dto/search-response.dto';
import { InjectKysely } from 'nestjs-kysely'; import { InjectKysely } from 'nestjs-kysely';
import { KyselyDB } from '@docmost/db/types/kysely.types'; import { KyselyDB } from '@docmost/db/types/kysely.types';
import { sql } from 'kysely'; import { sql } from 'kysely';
@@ -34,6 +37,53 @@ export function buildTsQuery(raw: string): string {
return tsquery(cleaned + '*'); return tsquery(cleaned + '*');
} }
// Escape the LIKE metacharacters (`%`, `_`, `\`) in a raw user query so every
// character — including `.`, `-`, `_`, `%`, `/` — is matched LITERALLY by a
// `col LIKE '%' || q || '%'` predicate. Without this, a query of `%` or `_`
// would match every row (see the #443 acceptance table). The backslash is the
// escape char (Postgres LIKE default), so it must be escaped first.
export function escapeLikePattern(raw: string): string {
return (raw ?? '')
.replace(/\\/g, '\\\\')
.replace(/%/g, '\\%')
.replace(/_/g, '\\_');
}
// Ranking tiers for the agent-lookup mode (#443), highest first. A hit's tier
// is the strongest way it matched; ties inside a tier break on a secondary
// signal (FTS rank, or first-match position). The numeric `score` returned to
// the caller is derived from (tier, secondary) and is meaningful ONLY for
// ordering within a single response.
export enum SearchLookupTier {
// Title equals the query, case-insensitively.
TITLE_EXACT = 3,
// Query is a substring of the title.
TITLE_SUBSTRING = 2,
// Query matched in the text (substring or FTS).
TEXT = 1,
}
export interface RankableHit {
tier: SearchLookupTier;
// Secondary in-tier signal, higher = better (e.g. ts_rank, or a
// position-derived closeness score). Defaults to 0.
secondary?: number;
}
// Map (tier, secondary) → a 0..1 float used ONLY to sort one response.
//
// Formula: score = (tier + squash(secondary)) / (maxTier + 1), where
// squash(x) = x / (1 + x) maps any non-negative secondary into [0, 1)
// so a stronger tier ALWAYS outranks a weaker one regardless of the secondary
// value, and within a tier a larger secondary sorts higher. maxTier is the top
// enum value (TITLE_EXACT = 3), so the divisor keeps the result in (0, 1].
export function computeLookupScore(hit: RankableHit): number {
const maxTier = SearchLookupTier.TITLE_EXACT;
const secondary = Math.max(0, hit.secondary ?? 0);
const squashed = secondary / (1 + secondary);
return (hit.tier + squashed) / (maxTier + 1);
}
@Injectable() @Injectable()
export class SearchService { export class SearchService {
constructor( constructor(
@@ -50,12 +100,19 @@ export class SearchService {
userId?: string; userId?: string;
workspaceId: string; workspaceId: string;
}, },
): Promise<{ items: SearchResponseDto[] }> { ): Promise<{ items: SearchResponseDto[] | SearchLookupResponseDto[] }> {
const { query } = searchParams; const { query } = searchParams;
if (query.length < 1) { if (query.length < 1) {
return { items: [] }; return { items: [] };
} }
// Opt-in agent-lookup mode (#443). Guarded by the `substring` flag so the
// web-UI (which never sets it) keeps byte-identical FTS behaviour below.
if (searchParams.substring) {
return this.searchPageLookup(searchParams, opts);
}
const searchQuery = buildTsQuery(query); const searchQuery = buildTsQuery(query);
let queryResults = this.db let queryResults = this.db
@@ -175,6 +232,348 @@ export class SearchService {
return { items: searchResults }; return { items: searchResults };
} }
/**
* Agent-lookup search (#443, opt-in via `SearchDTO.substring`).
*
* ADDITIVE to the FTS path: runs a substring branch (title + optionally
* text_content, LIKE with metacharacters escaped) MERGED with the existing
* FTS branch, so technical tokens that the `english` tokenizer mangles
* (`backup-srv.local`, `10.0.12.5`, `WB-MGE-30D86B`) are still found even
* when `buildTsQuery()` returns '' for a dotted/numeric query. Results carry a
* location (`path`), a windowed `snippet` and a per-response `score`.
*
* The whole method is only reached when `substring: true`; the web-UI never
* sets it, so its behaviour is unchanged.
*/
private async searchPageLookup(
searchParams: SearchDTO,
opts: { userId?: string; workspaceId: string },
): Promise<{ items: SearchLookupResponseDto[] }> {
const rawQuery = searchParams.query.trim();
if (!rawQuery) {
return { items: [] };
}
const limit = Math.min(Math.max(searchParams.limit || 10, 1), 50);
// Normalize the query the same way as the FTS / suggest path: f_unaccent +
// lower, done in SQL. `q` is the escaped LIKE pattern body (literal chars).
const likeBody = escapeLikePattern(rawQuery);
// Compare against `LOWER(f_unaccent(col))`; unaccent+lower the needle too.
const needle = sql<string>`LOWER(f_unaccent(${rawQuery}))`;
const likePattern = sql<string>`LOWER(f_unaccent(${'%' + likeBody + '%'}))`;
const tsQuery = buildTsQuery(rawQuery);
const hasTsQuery = tsQuery.length > 0;
// --- Resolve the space scope. ---------------------------------------------
// Mirrors searchPage: explicit spaceId, else the authenticated user's member
// spaces. The share path is not exposed to this opt-in mode.
let spaceIds: string[] = [];
if (searchParams.spaceId) {
spaceIds = [searchParams.spaceId];
} else if (opts.userId) {
spaceIds = await this.spaceMemberRepo.getUserSpaceIds(opts.userId);
} else {
return { items: [] };
}
if (spaceIds.length === 0) {
return { items: [] };
}
// --- Optional parentPageId subtree scope (inclusive). ---------------------
// Reuse the same recursive-descendants pattern used for share-scope.
let descendantIds: string[] | null = null;
if (searchParams.parentPageId) {
const descendants = await this.pageRepo.getPageAndDescendants(
searchParams.parentPageId,
{ includeContent: false },
);
descendantIds = descendants.map((p: any) => p.id);
if (descendantIds.length === 0) {
return { items: [] };
}
}
// --- Candidate query: substring (title + text) UNION FTS. -----------------
// We compute everything the ranker needs in SQL and pull only small columns
// (never the whole text_content) into Node:
// - titleExact / titleSub: tier signals
// - textMatchPos: 1-based position of the first text match (0 = none)
// - ftsRank: ts_rank for the FTS secondary signal (0 when no tsquery)
// - snippet: windowed ~500 chars around the first text match, or a leading
// text window (title-only hit), or an extended ts_headline fallback.
const N_BEFORE = 60; // chars of context before the first match
const SNIPPET_LEN = 500;
let candidates = this.db
.selectFrom('pages')
.select([
'pages.id as id',
'pages.slugId as slugId',
'pages.title as title',
'pages.parentPageId as parentPageId',
// Tier signals.
sql<boolean>`LOWER(f_unaccent(coalesce(pages.title, ''))) = ${needle}`.as(
'titleExact',
),
sql<boolean>`LOWER(f_unaccent(coalesce(pages.title, ''))) LIKE ${likePattern} ESCAPE '\\'`.as(
'titleSub',
),
// 1-based position of the first text match (0 = no text match).
sql<number>`strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle})`.as(
'textMatchPos',
),
// FTS secondary signal (0 when the tsquery is empty).
hasTsQuery
? sql<number>`ts_rank(pages.tsv, to_tsquery('english', f_unaccent(${tsQuery})))`.as(
'ftsRank',
)
: sql<number>`0`.as('ftsRank'),
// Windowed snippet, computed entirely in SQL. Priority:
// 1. window around the first text match;
// 2. otherwise (titleOnly: no snippet; else) a leading window of the
// page text (title-only hit);
// 3. otherwise an extended ts_headline for pure-FTS hits.
//
// #443 snippet-position fix: the match position (`strpos`) is computed in
// the LOWER(f_unaccent(...)) space, but f_unaccent is NOT length-
// preserving (ß→ss, æ→ae, …→..., ½→ 1/2, full-width forms), so slicing
// the ORIGINAL text at that position was misaligned — a single expanding
// char before the match shifted the window (or ran it past end → empty).
// We now slice from the SAME LOWER(f_unaccent(...)) string so position
// and slice share one coordinate space. DELIBERATE trade-off: the snippet
// loses original case/diacritics — acceptable for an agent-facing snippet
// (position accuracy over original-glyph fidelity). The ts_headline branch
// matches over the ORIGINAL text itself, so it is unaffected and kept as-is.
searchParams.titleOnly
? sql<string>`''`.as('snippet')
: sql<string>`
coalesce(
case
when strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) > 0
then substring(
LOWER(f_unaccent(coalesce(pages.text_content, '')))
from greatest(1, strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) - ${N_BEFORE})
for ${SNIPPET_LEN}
)
when coalesce(pages.text_content, '') <> ''
then substring(LOWER(f_unaccent(pages.text_content)) from 1 for 300)
${
hasTsQuery
? sql`else ts_headline('english', coalesce(pages.text_content, ''), to_tsquery('english', f_unaccent(${tsQuery})), 'MinWords=25, MaxWords=40, MaxFragments=3')`
: sql``
}
end,
''
)
`.as('snippet'),
])
.where('pages.deletedAt', 'is', null)
.where('pages.spaceId', 'in', spaceIds);
if (descendantIds) {
candidates = candidates.where('pages.id', 'in', descendantIds);
}
// Match predicate: title substring OR (unless titleOnly) text substring OR
// (unless titleOnly) FTS. The substring branch runs even when the tsquery is
// empty — that is the dotted/numeric-token case the FTS path misses.
//
// #443 dead-index fix: these two LIKE predicates MUST match the GIN trgm
// index expressions EXACTLY for Postgres to use them. The indexes are on the
// coalesce-FREE expressions `LOWER(f_unaccent(title))` (#348's
// idx_pages_title_trgm) and `LOWER(f_unaccent(text_content))` (this PR's
// idx_pages_text_content_trgm). A `coalesce(col,'')` wrapper here would make
// the query expression differ from the index expression and force a Seq Scan
// on pages for every lookup. Dropping coalesce is SEMANTICALLY EQUIVALENT:
// `NULL LIKE '%q%'` is NULL (falsy), so a NULL title/text simply doesn't
// match — exactly as an empty string wouldn't match `%q%`.
candidates = candidates.where((eb) => {
const ors = [
eb(
sql`LOWER(f_unaccent(pages.title))`,
'like',
sql`${likePattern} ESCAPE '\\'`,
),
];
if (!searchParams.titleOnly) {
ors.push(
eb(
sql`LOWER(f_unaccent(pages.text_content))`,
'like',
sql`${likePattern} ESCAPE '\\'`,
),
);
if (hasTsQuery) {
ors.push(
sql<boolean>`pages.tsv @@ to_tsquery('english', f_unaccent(${tsQuery}))` as any,
);
}
}
return eb.or(ors);
});
// Pull a generous candidate set (before permission filtering + limit).
// Cap it so a pathological match set cannot blow up memory; 200 >> limit
// (max 50) leaves ample headroom for the post-permission truncation.
//
// #443 cap-ordering fix: the 200-cap MUST be deterministic and relevance-
// biased. Without an ORDER BY, Postgres returns an ARBITRARY 200 rows, so on
// a broad match set (common word / short substring) a strong TITLE_EXACT hit
// could be among the dropped rows while 200 low-tier TEXT hits fill the cap.
// We order by the SAME SQL tier proxies the Node ranker uses — title-exact,
// then title-substring, then fts-rank (nulls last), then earliest text-match
// position — so the cap keeps the strongest candidates. The Node-side final
// tier sort + slice(0, limit) below still runs and stays authoritative; this
// ORDER BY only decides WHICH candidates survive the 200-cap.
// NB: a BARE integer literal in ORDER BY is read by Postgres as an ordinal
// column position (`ORDER BY 0` → "position 0 is not in select list"), so the
// no-tsquery fallback is `0::float`, not `0`.
const ftsRankExpr = hasTsQuery
? sql`ts_rank(pages.tsv, to_tsquery('english', f_unaccent(${tsQuery})))`
: sql`0::float`;
const candidatesCapped = candidates
// Raw-SQL ORDER BY expressions: pass the full `<expr> <dir>` as ONE arg
// (the two-arg form treats a raw-SQL second arg as an ORDER BY position).
.orderBy(
sql`(LOWER(f_unaccent(coalesce(pages.title, ''))) = ${needle}) desc`,
)
.orderBy(
sql`(LOWER(f_unaccent(coalesce(pages.title, ''))) LIKE ${likePattern} ESCAPE '\\') desc`,
)
.orderBy(sql`${ftsRankExpr} desc nulls last`)
// Earlier text match first; strpos returns 0 for "no match", which would
// sort BEFORE a real (>=1) position under plain ASC, so push 0 to the end.
.orderBy(
sql`case when strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) = 0 then 2147483647 else strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) end asc`,
);
let rows: any[] = await candidatesCapped.limit(200).execute();
if (rows.length === 0) {
return { items: [] };
}
// --- Permissions BEFORE limit. --------------------------------------------
// Apply the existing page-level post-filter to the MERGED set, then rank and
// only THEN truncate to `limit` — never lose the permission filter.
if (opts.userId) {
const accessibleIds =
await this.pagePermissionRepo.filterAccessiblePageIds({
pageIds: rows.map((r) => r.id),
userId: opts.userId,
spaceId: searchParams.spaceId,
workspaceId: opts.workspaceId,
});
const accessibleSet = new Set(accessibleIds);
rows = rows.filter((r) => accessibleSet.has(r.id));
}
if (rows.length === 0) {
return { items: [] };
}
// --- Tiered ranking + dedup. ----------------------------------------------
// Rows are already unique by id (single pages scan), so no cross-branch
// dedup is needed here; the tier captures the strongest match reason.
const ranked = rows.map((r) => {
let tier: SearchLookupTier;
let secondary: number;
if (r.titleExact) {
tier = SearchLookupTier.TITLE_EXACT;
secondary = Number(r.ftsRank) || 0;
} else if (r.titleSub) {
tier = SearchLookupTier.TITLE_SUBSTRING;
secondary = Number(r.ftsRank) || 0;
} else {
tier = SearchLookupTier.TEXT;
// Prefer earlier text matches; map position → closeness in (0, 1].
const pos = Number(r.textMatchPos) || 0;
secondary =
pos > 0 ? 1 / (1 + (pos - 1) / 100) : Number(r.ftsRank) || 0;
}
return { row: r, tier, score: computeLookupScore({ tier, secondary }) };
});
ranked.sort((a, b) => b.score - a.score);
const top = ranked.slice(0, limit);
// --- Batch ancestor path (ONE recursive CTE, not N+1). --------------------
const pathById = await this.buildAncestorPaths(top.map((t) => t.row.id));
const items: SearchLookupResponseDto[] = top.map((t) => ({
id: t.row.id,
slugId: t.row.slugId,
title: t.row.title,
parentPageId: t.row.parentPageId ?? null,
path: pathById.get(t.row.id) ?? [],
snippet: (t.row.snippet ?? '')
.replace(/\r\n|\r|\n/g, ' ')
.replace(/\s+/g, ' ')
.trim(),
score: t.score,
}));
return { items };
}
/**
* Batch ancestor-titles helper (#443): ONE recursive CTE seeded with ALL hit
* ids, walking UP parentPageId. Returns a map hitId ancestor titles ordered
* root direct parent (the hit's own title is excluded). Root pages map to
* an empty array. Avoids the N+1 of a per-page breadcrumb call.
*/
private async buildAncestorPaths(
hitIds: string[],
): Promise<Map<string, string[]>> {
const result = new Map<string, string[]>();
if (hitIds.length === 0) return result;
// ancestry(hit_id, page_id, title, parent_page_id, depth): seed one row per
// hit at depth 0 (the hit itself), then walk to parents (increasing depth).
const rows = await this.db
.withRecursive('ancestry', (db) =>
db
.selectFrom('pages')
.select([
'pages.id as hitId',
'pages.id as pageId',
'pages.title as title',
'pages.parentPageId as parentPageId',
sql<number>`0`.as('depth'),
])
.where('pages.id', 'in', hitIds)
.unionAll((exp) =>
exp
.selectFrom('pages as p')
.innerJoin('ancestry as a', 'p.id', 'a.parentPageId')
.select([
'a.hitId as hitId',
'p.id as pageId',
'p.title as title',
'p.parentPageId as parentPageId',
sql<number>`a.depth + 1`.as('depth'),
]),
),
)
.selectFrom('ancestry')
.select(['hitId', 'title', 'depth'])
// depth 0 is the hit itself — excluded from the path.
.where('depth', '>', 0)
.orderBy('hitId')
// Larger depth = closer to the space root. Ordering DESC gives
// root → parent once collected.
.orderBy('depth', 'desc')
.execute();
for (const r of rows as any[]) {
const list = result.get(r.hitId) ?? [];
list.push(r.title);
result.set(r.hitId, list);
}
return result;
}
async searchSuggestions( async searchSuggestions(
suggestion: SearchSuggestionDTO, suggestion: SearchSuggestionDTO,
userId: string, userId: string,
@@ -0,0 +1,55 @@
import { type Kysely, sql } from 'kysely';
/**
* #443 trigram indexes for the opt-in agent-lookup search mode.
*
* The lookup mode adds a substring branch that runs leading-wildcard
* `LOWER(f_unaccent(col)) LIKE '%q%'` predicates on pages.title and
* pages.text_content. A leading wildcard cannot use a b-tree index, so without a
* GIN trigram index each such predicate is a sequential scan.
*
* - TITLE: the lookup-mode title predicate is `LOWER(f_unaccent(title)) LIKE
* '%q%'` (coalesce-free, so it can use a functional index), which is IDENTICAL
* to the one added for /search/suggest (#348). #348's perf-indexes migration
* already created `idx_pages_title_trgm` on `(LOWER(f_unaccent(title)))
* gin_trgm_ops`, so the title predicate is already covered — we do NOT
* re-create that index here (it would be redundant).
*
* - TEXT_CONTENT: NEW. The substring branch scans text_content when the query
* is not titleOnly. text_content is the large column, so a GIN trigram index
* on it is the meaningful acceleration for the lookup mode. The lookup search
* is ALWAYS space-scoped (spaceId or the user's member spaces), so on small
* instances a per-space sequential scan is tolerable but the index turns the
* `%q%` text predicate into a Bitmap Index Scan and removes the only
* unbounded-per-space cost of the feature. We add it. The trade-off is disk +
* write amplification on page edits (GIN trigram indexes are larger and slower
* to update than b-trees); on the small instances this fork targets that cost
* is acceptable and the read win on agent lookups is the priority.
*
* DEPLOY-TIME LOCK WARNING: plain (non-CONCURRENT) CREATE INDEX Kysely runs
* each migration in a transaction, so CONCURRENTLY is impossible. The build takes
* a SHARE lock that BLOCKS writes on `pages` for its duration. The text_content
* GIN build is the slow one and can take minutes on a large tenant. For large
* installations, run this in a maintenance window or build the index out-of-band
* with CREATE INDEX CONCURRENTLY before deploying (then `IF NOT EXISTS` no-ops
* here). Small/typical tenants are unaffected.
*/
export async function up(db: Kysely<any>): Promise<void> {
// The title predicate is served by #348's idx_pages_title_trgm — see header.
// Only the text_content index is introduced here.
// text_content trigram index. Its expression is coalesce-free —
// `LOWER(f_unaccent(text_content))` — to EXACTLY match the coalesce-free
// lookup-mode text substring predicate in search.service.ts, so Postgres can
// use it (a `coalesce(...)` mismatch would silently fall back to a Seq Scan).
await sql`
CREATE INDEX IF NOT EXISTS idx_pages_text_content_trgm
ON pages USING gin ((LOWER(f_unaccent(text_content))) gin_trgm_ops)
`.execute(db);
}
export async function down(db: Kysely<any>): Promise<void> {
// Only drop the index this migration introduced. idx_pages_title_trgm is owned
// by the #348 perf-indexes migration, so leave it for that migration's down().
await sql`DROP INDEX IF EXISTS idx_pages_text_content_trgm`.execute(db);
}
@@ -143,7 +143,7 @@ export type DocmostMcpConfig = (
buf: Buffer, buf: Buffer,
mime: string, mime: string,
) => { uri: string; sha256: string; size: number }; ) => { uri: string; sha256: string; size: number };
// Optional live/evict probes the package uses to keep stash_page's mirror // Optional live/evict probes the package uses to keep stashPage's mirror
// counts honest under the store's FIFO eviction (mirror of the package's // counts honest under the store's FIFO eviction (mirror of the package's
// sink type); older bindings omit them. // sink type); older bindings omit them.
has?: (uri: string) => boolean; has?: (uri: string) => boolean;
@@ -332,7 +332,7 @@ export class McpService implements OnModuleDestroy {
// Should never happen: handle() always stashes before delegating. // Should never happen: handle() always stashes before delegating.
throw new UnauthorizedException('MCP authentication missing.'); throw new UnauthorizedException('MCP authentication missing.');
} }
// Inject the blob-sandbox sink after the auth decision so stash_page // Inject the blob-sandbox sink after the auth decision so stashPage
// can store blobs in the shared in-RAM store regardless of which // can store blobs in the shared in-RAM store regardless of which
// credential variant resolved. The sink (put/has/evict + uri↔id // credential variant resolved. The sink (put/has/evict + uri↔id
// mapping) is owned by SandboxStore.asSink(). // mapping) is owned by SandboxStore.asSink().
@@ -0,0 +1,123 @@
import { randomUUID } from 'node:crypto';
import { Kysely, sql } from 'kysely';
import {
getTestDb,
destroyTestDb,
createWorkspace,
createSpace,
} from './db';
/**
* #443 dead-index guard EXPLAIN on the REAL DB.
*
* The lookup mode's substring predicates run a leading-wildcard
* `LOWER(f_unaccent(col)) LIKE '%q%'`. Those are only fast when Postgres uses
* the GIN trigram indexes:
* - idx_pages_title_trgm on (LOWER(f_unaccent(title))) [#348]
* - idx_pages_text_content_trgm on (LOWER(f_unaccent(text_content))) [#443]
*
* Postgres uses a functional index ONLY when the query expression matches the
* index expression EXACTLY. The original lookup query wrapped the columns in
* `coalesce(col,'')`, which differs from the coalesce-FREE index expression and
* silently forced a Seq Scan on pages for EVERY lookup (the MCP client always
* sends substring:true). This test locks that in.
*
* Discriminator: `SET enable_seqscan = off` asks the planner "CAN this predicate
* use the index at all?" which is exactly what the coalesce bug breaks. With
* seqscan disabled:
* - the coalesce-FREE (fixed) predicate plans a Bitmap Index Scan on the trgm
* index (no Seq Scan on pages);
* - the coalesce-WRAPPED (buggy) predicate cannot use the index and falls back
* to a Seq Scan on pages even though seqscan is disabled.
* We assert both to prove the fix and to keep the regression from silently
* returning.
*/
describe('SearchService agent-lookup EXPLAIN — trgm index is live [integration]', () => {
let db: Kysely<any>;
let workspaceId: string;
let spaceId: string;
async function insertPage(title: string, textContent: string): Promise<void> {
const id = randomUUID();
await db
.insertInto('pages')
.values({
id,
slugId: `slug-${id.slice(0, 12)}`,
title,
textContent,
spaceId,
workspaceId,
})
.execute();
}
// Run EXPLAIN (no ANALYZE — we only inspect the chosen plan) and return the
// concatenated plan text.
async function explain(query: string): Promise<string> {
const rows = await sql<{ 'QUERY PLAN': string }>`EXPLAIN ${sql.raw(query)}`.execute(
db,
);
return (rows.rows as any[]).map((r) => r['QUERY PLAN']).join('\n');
}
beforeAll(async () => {
db = getTestDb();
workspaceId = (await createWorkspace(db)).id;
spaceId = (await createSpace(db, workspaceId)).id;
// Seed enough rows that a trigram index is a plausible plan. The content is
// varied so the '%needle%' pattern is selective.
for (let i = 0; i < 200; i++) {
await insertPage(
`seed-title-${i}`,
`seed body content number ${i} lorem ipsum dolor sit amet ${i}`,
);
}
await insertPage('backup-srv.local', 'the needle-token-xyz lives here');
// Keep the trgm indexes' stats fresh so the planner costs them correctly.
await sql`ANALYZE pages`.execute(db);
});
afterAll(async () => {
await destroyTestDb();
});
// Force the planner to answer "can the index be used?" rather than "is it
// cheaper than a seq scan on this size?". Restored after each test.
beforeEach(async () => {
await sql`SET enable_seqscan = off`.execute(db);
});
afterEach(async () => {
await sql`RESET enable_seqscan`.execute(db);
});
it('title predicate (coalesce-FREE, as fixed) uses idx_pages_title_trgm, not a Seq Scan', async () => {
const plan = await explain(
`SELECT id FROM pages WHERE LOWER(f_unaccent(title)) LIKE '%srv.local%'`,
);
expect(plan).toContain('idx_pages_title_trgm');
expect(plan).not.toMatch(/Seq Scan on pages/i);
});
it('text_content predicate (coalesce-FREE, as fixed) uses idx_pages_text_content_trgm, not a Seq Scan', async () => {
const plan = await explain(
`SELECT id FROM pages WHERE LOWER(f_unaccent(text_content)) LIKE '%needle-token%'`,
);
expect(plan).toContain('idx_pages_text_content_trgm');
expect(plan).not.toMatch(/Seq Scan on pages/i);
});
// Negative control: the OLD coalesce-wrapped predicate must NOT be able to use
// the index — even with seqscan disabled it can only Seq Scan pages. If this
// ever stops seq-scanning, the coalesce/index expressions have re-aligned and
// the guard above is no longer meaningful.
it('coalesce-WRAPPED text predicate (the bug) cannot use the index — falls to Seq Scan', async () => {
const plan = await explain(
`SELECT id FROM pages WHERE LOWER(f_unaccent(coalesce(text_content,''))) LIKE '%needle-token%'`,
);
expect(plan).not.toContain('idx_pages_text_content_trgm');
expect(plan).toMatch(/Seq Scan on pages/i);
});
});
@@ -0,0 +1,462 @@
import { randomUUID } from 'node:crypto';
import { Kysely } from 'kysely';
import { SearchService } from 'src/core/search/search.service';
import { PageRepo } from '@docmost/db/repos/page/page.repo';
import {
getTestDb,
destroyTestDb,
createWorkspace,
createSpace,
} from './db';
/**
* #443 agent-lookup search mode, acceptance on the REAL DB schema.
*
* Exercises SearchService.searchPage(..., { substring: true }) against a
* migrated Postgres: substring matching of technical tokens the FTS tokenizer
* mangles (backup-srv.local, 10.0.12.5, WB-MGE-30D86B, "Теги: Docker"), the
* populated path + snippet, parentPageId subtree scoping, titleOnly, the empty
* result, LIKE-metacharacter escaping (`%`/`_` must NOT match everything), the
* permission post-filter applied BEFORE the limit, and the web-UI path staying
* on the legacy FTS shape when `substring` is absent.
*
* The tsv column is populated by the pages_tsvector_trigger on insert, so the
* FTS branch is exercised too.
*/
describe('SearchService agent-lookup mode [integration]', () => {
let db: Kysely<any>;
let service: SearchService;
let workspaceId: string;
let spaceId: string;
// Direct page insert (the shared createPage seeder omits text_content /
// parent_page_id, both of which this mode depends on). Returns the id.
async function insertPage(args: {
title: string;
textContent?: string;
parentPageId?: string | null;
spaceId?: string;
}): Promise<string> {
const id = randomUUID();
await db
.insertInto('pages')
.values({
id,
slugId: `slug-${id.slice(0, 12)}`,
title: args.title,
textContent: args.textContent ?? null,
parentPageId: args.parentPageId ?? null,
spaceId: args.spaceId ?? spaceId,
workspaceId,
})
.execute();
return id;
}
// Build a SearchService wired to the real DB + a real PageRepo (only its
// recursive-descendants method is used by this mode, and it needs only `db`),
// with lightweight stubs for the space-membership and permission repos so a
// test can drive scope + the permission post-filter explicitly.
function buildService(opts?: {
userSpaceIds?: string[];
// ids to KEEP after the permission post-filter; undefined = keep all.
accessibleIds?: string[];
}): SearchService {
const pageRepo = new PageRepo(db as any, null as any, null as any);
const spaceMemberRepo = {
getUserSpaceIds: async () => opts?.userSpaceIds ?? [spaceId],
};
const pagePermissionRepo = {
filterAccessiblePageIds: async ({ pageIds }: { pageIds: string[] }) =>
opts?.accessibleIds
? pageIds.filter((id) => opts.accessibleIds!.includes(id))
: pageIds,
};
return new SearchService(
db as any,
pageRepo as any,
{} as any, // shareRepo — unused by the lookup path
spaceMemberRepo as any,
pagePermissionRepo as any,
);
}
beforeAll(async () => {
db = getTestDb();
workspaceId = (await createWorkspace(db)).id;
spaceId = (await createSpace(db, workspaceId)).id;
service = buildService();
});
afterAll(async () => {
await destroyTestDb();
});
it('finds `backup-srv.local` by the fragment `srv.local`', async () => {
const pageId = await insertPage({
title: 'backup-srv.local',
textContent: 'A backup server node.',
});
const { items } = (await service.searchPage(
{ query: 'srv.local', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
expect(items.map((i: any) => i.id)).toContain(pageId);
const hit = items.find((i: any) => i.id === pageId);
expect(hit.title).toBe('backup-srv.local');
// slugId must never be part of the server response shape.
expect('slugId' in hit).toBe(true); // server carries it; MCP strips it
});
it('finds a page whose TEXT contains `10.0.12.5` by the fragment `10.0.12` (empty-tsquery case)', async () => {
const pageId = await insertPage({
title: 'Server inventory',
textContent: 'The backup box lives at IP: 10.0.12.5. Debian 12, backups.',
});
const { items } = (await service.searchPage(
{ query: '10.0.12', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// The windowed snippet must include the matched text.
expect(hit.snippet).toContain('10.0.12.5');
});
it('finds `WB-MGE-30D86B` (alphanumeric token with dashes) by title', async () => {
const pageId = await insertPage({
title: 'WB-MGE-30D86B',
textContent: 'Device page.',
});
const { items } = (await service.searchPage(
{ query: 'WB-MGE-30D86B', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// Exact title match → top tier (TITLE_EXACT=3) → score in [0.75, 1].
expect(hit.score).toBeGreaterThanOrEqual(0.75);
// And it is the top-ranked hit of its own result set.
expect(items[0].id).toBe(pageId);
});
it('finds every page whose text literally contains `Теги: Docker`', async () => {
const a = await insertPage({
title: 'Container host A',
textContent: 'Some notes.\nТеги: Docker, compose\nmore.',
});
const b = await insertPage({
title: 'Container host B',
textContent: 'Prelude.\nТеги: Docker\nepilogue.',
});
const noise = await insertPage({
title: 'Unrelated',
textContent: 'Теги: Kubernetes',
});
const { items } = (await service.searchPage(
{ query: 'Теги: Docker', spaceId, substring: true, limit: 50 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(a);
expect(ids).toContain(b);
expect(ids).not.toContain(noise);
});
it('populates a non-empty `path` for a nested hit and `[]` for a root hit', async () => {
const root = await insertPage({ title: 'Infrastructure' });
const mid = await insertPage({ title: 'Datacenter A', parentPageId: root });
const leaf = await insertPage({
title: 'unique-nested-host',
parentPageId: mid,
});
const { items } = (await service.searchPage(
{ query: 'unique-nested-host', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === leaf);
expect(hit.path).toEqual(['Infrastructure', 'Datacenter A']);
const rootHits = (await service.searchPage(
{ query: 'Infrastructure', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const rootHit = rootHits.items.find((i: any) => i.id === root);
expect(rootHit.path).toEqual([]);
});
it('scopes to a subtree with parentPageId (cutting off sibling branches)', async () => {
const branchA = await insertPage({ title: 'BranchA-root' });
const inA = await insertPage({
title: 'scoped-target-xyz',
parentPageId: branchA,
});
const branchB = await insertPage({ title: 'BranchB-root' });
const inB = await insertPage({
title: 'scoped-target-xyz',
parentPageId: branchB,
});
const { items } = (await service.searchPage(
{
query: 'scoped-target-xyz',
spaceId,
substring: true,
parentPageId: branchA,
} as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(inA);
expect(ids).not.toContain(inB);
});
it('includes the parent page itself in the parentPageId subtree', async () => {
const parent = await insertPage({ title: 'self-included-parent' });
await insertPage({ title: 'child-of-self', parentPageId: parent });
const { items } = (await service.searchPage(
{
query: 'self-included-parent',
spaceId,
substring: true,
parentPageId: parent,
} as any,
{ workspaceId },
)) as any;
expect(items.map((i: any) => i.id)).toContain(parent);
});
it('titleOnly does NOT match on text_content', async () => {
const pageId = await insertPage({
title: 'Plain title',
textContent: 'body mentions the-secret-token here',
});
const withText = (await service.searchPage(
{ query: 'the-secret-token', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
expect(withText.items.map((i: any) => i.id)).toContain(pageId);
const titleOnly = (await service.searchPage(
{
query: 'the-secret-token',
spaceId,
substring: true,
titleOnly: true,
} as any,
{ workspaceId },
)) as any;
expect(titleOnly.items.map((i: any) => i.id)).not.toContain(pageId);
});
// #443 Fix #1 regression: f_unaccent is NOT length-preserving, so an
// expanding char (ß→ss, …→...) BEFORE the match shifted the strpos position
// relative to the ORIGINAL text and the snippet slice ran past end → empty.
// The position and the slice now share the LOWER(f_unaccent(...)) space, so
// the window is aligned and always contains the matched (unaccented) token.
it('returns a populated snippet when an unaccent-EXPANDING char precedes the match', async () => {
// 300 × `ß` (each f_unaccent-expands to `ss`) before the needle. Under the
// old code strpos returned a position ~593 in the expanded space but the
// slice ran over the ORIGINAL (~360 char) text → empty snippet, match lost.
const prefix = 'ß'.repeat(300);
const pageId = await insertPage({
title: 'Expanding-unaccent page',
textContent: `${prefix} needle-token-xyz trailing.`,
});
const { items } = (await service.searchPage(
{ query: 'needle-token-xyz', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// Snippet must be non-empty AND contain the matched token (unaccented form).
expect(hit.snippet.length).toBeGreaterThan(0);
expect(hit.snippet).toContain('needle-token-xyz');
});
// #443 Fix #2 regression: >200 matching pages for a broad substring, with
// exactly ONE exact-title hit. Without an ORDER BY on the 200-cap the exact
// hit could be among the arbitrarily-dropped rows; the ORDER BY keeps the
// strongest candidates so it must survive the cap and rank at the top.
it('keeps an exact-title hit through the 200-cap on a >200-row match set', async () => {
const isoSpace = (await createSpace(db, workspaceId)).id;
const svc = buildService({ userSpaceIds: [isoSpace] });
// 250 low-tier TEXT hits: the shared substring `capword` appears only in the
// body, never the title, so each is a TEXT-tier match (weakest tier).
for (let i = 0; i < 250; i++) {
await insertPage({
title: `filler-page-${i}`,
textContent: `body contains capword here #${i}`,
spaceId: isoSpace,
});
}
// Exactly one EXACT-title hit for the same query token.
const exact = await insertPage({
title: 'capword',
textContent: 'unrelated body text',
spaceId: isoSpace,
});
const { items } = (await svc.searchPage(
{ query: 'capword', spaceId: isoSpace, substring: true, limit: 10 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
// The exact-title hit must survive the 200-cap and appear in the top `limit`.
expect(ids).toContain(exact);
// And, being TITLE_EXACT, it must be the single strongest hit.
expect(items[0].id).toBe(exact);
});
// #443 Fix #3: titleOnly matches only the title, so it must not leak the page
// body as the snippet (the old "first 300 chars of text_content" fallback).
it('titleOnly does NOT return a text-body snippet', async () => {
const pageId = await insertPage({
title: 'titleonly-snippet-page',
textContent: 'SECRET-BODY-CONTENT-NOT-IN-TITLE that must not leak.',
});
const { items } = (await service.searchPage(
{
query: 'titleonly-snippet-page',
spaceId,
substring: true,
titleOnly: true,
} as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// The body text must not appear in the snippet; titleOnly → empty snippet.
expect(hit.snippet).not.toContain('SECRET-BODY-CONTENT-NOT-IN-TITLE');
expect(hit.snippet).toBe('');
});
it('returns [] (not an error) for a query that matches nothing', async () => {
const { items } = (await service.searchPage(
{
query: 'zzz-no-such-string-anywhere-42',
spaceId,
substring: true,
} as any,
{ workspaceId },
)) as any;
expect(items).toEqual([]);
});
it('a `%` query does NOT match everything (LIKE metacharacter escaped)', async () => {
// Fresh space so we can assert on total counts without cross-test noise.
const isoSpace = (await createSpace(db, workspaceId)).id;
const svc = buildService({ userSpaceIds: [isoSpace] });
await insertPage({ title: 'alpha', spaceId: isoSpace });
await insertPage({ title: 'beta', spaceId: isoSpace });
const literal = await insertPage({
title: '100%-coverage',
spaceId: isoSpace,
});
const { items } = (await svc.searchPage(
{ query: '%', spaceId: isoSpace, substring: true, limit: 50 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
// `%` is a literal → matches only the page that actually contains '%'.
expect(ids).toContain(literal);
expect(ids).not.toContain(
items.find((i: any) => i.title === 'alpha')?.id,
);
expect(items.length).toBe(1);
});
it('an `_` query does NOT match everything (LIKE metacharacter escaped)', async () => {
const isoSpace = (await createSpace(db, workspaceId)).id;
const svc = buildService({ userSpaceIds: [isoSpace] });
await insertPage({ title: 'gamma', spaceId: isoSpace });
const literal = await insertPage({
title: 'snake_case_name',
spaceId: isoSpace,
});
const { items } = (await svc.searchPage(
{ query: '_', spaceId: isoSpace, substring: true, limit: 50 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(literal);
expect(items.length).toBe(1);
});
it('applies the permission post-filter to the MERGED set BEFORE the limit', async () => {
const isoSpace = (await createSpace(db, workspaceId)).id;
const keep = await insertPage({
title: 'perm-visible-target',
spaceId: isoSpace,
});
const hidden = await insertPage({
title: 'perm-hidden-target',
spaceId: isoSpace,
});
// Authenticated (userId set) so the permission filter runs; only `keep` is
// accessible. limit 1 must NOT be able to select `hidden`.
const svc = buildService({
userSpaceIds: [isoSpace],
accessibleIds: [keep],
});
const { items } = (await svc.searchPage(
{
query: 'perm-',
spaceId: isoSpace,
substring: true,
limit: 1,
} as any,
{ userId: 'user-1', workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(keep);
expect(ids).not.toContain(hidden);
});
it('web-UI path (no `substring` flag) keeps the legacy FTS response shape', async () => {
await insertPage({
title: 'legacy shape page',
textContent: 'searchable legacyword content',
});
const { items } = (await service.searchPage(
{ query: 'legacyword', spaceId } as any,
{ userId: 'user-1', workspaceId },
)) as any;
// Legacy hits carry rank + highlight + space, and NO path/snippet/score.
const hit = items[0];
expect(hit).toBeDefined();
expect('rank' in hit).toBe(true);
expect('highlight' in hit).toBe(true);
expect('path' in hit).toBe(false);
expect('snippet' in hit).toBe(false);
expect('score' in hit).toBe(false);
});
});
+115 -88
View File
@@ -12,7 +12,7 @@ license.
> better at *writing a small function that fixes the text* than at re-reading and > better at *writing a small function that fixes the text* than at re-reading and
> re-emitting a whole document. So this server is built around the way a model actually > re-emitting a whole document. So this server is built around the way a model actually
> wants to edit: address a block by id, run a find/replace, or hand it a > wants to edit: address a block by id, run a find/replace, or hand it a
> `(doc, ctx) => doc` transform and let it *program* the change. `docmost_transform` is > `(doc, ctx) => doc` transform and let it *program* the change. `docmostTransform` is
> that interface. Other Docmost MCPs are human-shaped — they expose "open the page" and > that interface. Other Docmost MCPs are human-shaped — they expose "open the page" and
> "replace the page"; this one exposes the editing primitives a model is good at. > "replace the page"; this one exposes the editing primitives a model is good at.
@@ -40,7 +40,7 @@ There are several Docmost MCPs. Here is a capability-by-capability comparison.
| **Enterprise license required** | **No** | **Yes** | No | No | No | | **Enterprise license required** | **No** | **Yes** | No | No | No |
| Authentication | email + password, **auto re-auth** | API key | email + password | cookie `authToken` (copy from DevTools) | Docmost API / **direct PostgreSQL** | | Authentication | email + password, **auto re-auth** | API key | email + password | cookie `authToken` (copy from DevTools) | Docmost API / **direct PostgreSQL** |
| Read page as Markdown | ✅ | ✅ | ✅ | ✅ | ✅ (read-only) | | Read page as Markdown | ✅ | ✅ | ✅ | ✅ | ✅ (read-only) |
| **Lossless Markdown round-trip** (export / import, keeps comment anchors) | ✅ | — | — | — | — | | **Markdown round-trip** (export / import, keeps comment anchors) | ✅ | — | — | — | — |
| Read **lossless ProseMirror JSON** (with block ids) | ✅ | — | — | — | — | | Read **lossless ProseMirror JSON** (with block ids) | ✅ | — | — | — | — |
| **Compact page outline** (cheap block-id lookup) | ✅ | — | — | — | — | | **Compact page outline** (cheap block-id lookup) | ✅ | — | — | — | — |
| **Fetch a single block** (by id or index) | ✅ | — | — | — | — | | **Fetch a single block** (by id or index) | ✅ | — | — | — | — |
@@ -69,9 +69,9 @@ There are several Docmost MCPs. Here is a capability-by-capability comparison.
- **Token-efficient editing.** Most Docmost MCPs (and the official one) only offer - **Token-efficient editing.** Most Docmost MCPs (and the official one) only offer
"replace the whole page" writes — the agent must download the entire document, mutate "replace the whole page" writes — the agent must download the entire document, mutate
it, and upload it back, paying for the full document **twice** on every tiny fix. it, and upload it back, paying for the full document **twice** on every tiny fix.
This server lets the agent change exactly one block (`patch_node` / `insert_node` / This server lets the agent change exactly one block (`patchNode` / `insertNode` /
`delete_node`), do a structure-preserving find/replace (`edit_page_text`), or copy a `deleteNode`), do a structure-preserving find/replace (`editPageText`), or copy a
whole page server-side (`copy_page_content`) — **without the document ever passing whole page server-side (`copyPageContent`) — **without the document ever passing
through the model**. through the model**.
- **Writes that don't fight the editor.** Naive REST writes race with whatever a human - **Writes that don't fight the editor.** Naive REST writes race with whatever a human
@@ -85,12 +85,12 @@ There are several Docmost MCPs. Here is a capability-by-capability comparison.
- **Agent-native editing model.** Human-facing servers expose "open the page" and "replace - **Agent-native editing model.** Human-facing servers expose "open the page" and "replace
the page", because that mirrors how a person works. A model edits better by *programming* the page", because that mirrors how a person works. A model edits better by *programming*
the change — addressing blocks by id, running a find/replace, or supplying a the change — addressing blocks by id, running a find/replace, or supplying a
`(doc, ctx) => doc` transform (`docmost_transform`, with a dry-run diff before it `(doc, ctx) => doc` transform (`docmostTransform`, with a dry-run diff before it
commits). This server is shaped around that, which is why it has editing primitives the commits). This server is shaped around that, which is why it has editing primitives the
others simply don't. others simply don't.
- **An editing safety net the others lack.** `list_page_history``diff_page_versions` - **An editing safety net the others lack.** `listPageHistory``diffPageVersions`
`restore_page_version` give an agent (and you) a full view-and-undo loop. The diff `restorePageVersion` give an agent (and you) a full view-and-undo loop. The diff
uses the *same* `recreateTransform → ChangeSet → simplifyChanges` pipeline Docmost's uses the *same* `recreateTransform → ChangeSet → simplifyChanges` pipeline Docmost's
own history viewer uses, so what you see matches the product. own history viewer uses, so what you see matches the product.
@@ -110,56 +110,58 @@ All 41 tools, grouped by what you'd reach for them.
### Exploration & retrieval ### Exploration & retrieval
- **`get_workspace`** — Information about the current Docmost workspace. - **`getWorkspace`** — Information about the current Docmost workspace.
- **`list_spaces`** — All spaces in the workspace. - **`listSpaces`** — All spaces in the workspace.
- **`list_pages`** — Recent pages in a space, ordered by `updatedAt` desc (default 50, - **`listPages`** — Recent pages in a space, ordered by `updatedAt` desc (default 50,
max 100). Use `search` for lookups in large spaces. max 100). Use `search` for lookups in large spaces.
- **`search`** — Full-text search across pages and content (bounded by `limit`, max 100). - **`search`** — Full-text search across pages and content (bounded by `limit`, max 100).
- **`get_page`** — A page's content as clean **Markdown** (convenient, but a *lossy* - **`getPage`** — A page's content as clean **Markdown** (canonical for text; drops only
view — block ids and exact table/callout structure are approximated). block ids, resolved-comment anchors, and a fixed no-Markdown-representation attr set —
- **`get_page_json`** — A page's **lossless ProseMirror/TipTap JSON**, including every table spans/colwidth/background, indent, `callout.icon`, `orderedList.type`, and link
`internal`/`target`/`rel`/`class`; use `getPageJson` when you need those).
- **`getPageJson`** — A page's **lossless ProseMirror/TipTap JSON**, including every
block's `attrs.id` and the `slugId` used in URLs. This is what the per-block editing block's `attrs.id` and the `slugId` used in URLs. This is what the per-block editing
tools consume. tools consume.
- **`get_outline`** — A compact outline of a page's top-level blocks (`{index, type, id, - **`getOutline`** — A compact outline of a page's top-level blocks (`{index, type, id,
level, firstText}`; tables add row/column counts and their header-cell texts, lists add level, firstText}`; tables add row/column counts and their header-cell texts, lists add
item counts) **without** the document body. The cheap way to locate a section or table item counts) **without** the document body. The cheap way to locate a section or table
and grab its block id before and grab its block id before
`get_node` / `patch_node` / `insert_node`. `getNode` / `patchNode` / `insertNode`.
- **`get_node`** — Fetch a single block's full ProseMirror subtree (lossless) without - **`getNode`** — Fetch a single block's full ProseMirror subtree (lossless) without
pulling the whole page. Address it by a block id (from `get_outline` / `get_page_json`), pulling the whole page. Address it by a block id (from `getOutline` / `getPageJson`),
or by `#<index>` for a top-level block — use the `#<index>` form for tables/rows/cells, or by `#<index>` for a top-level block — use the `#<index>` form for tables/rows/cells,
which carry no id. which carry no id.
### Page lifecycle ### Page lifecycle
- **`create_page`** — Create a page from Markdown and place it in the hierarchy (optional - **`createPage`** — Create a page from Markdown and place it in the hierarchy (optional
`parentPageId`) in one call. Uses Docmost's import API for clean Markdown→ProseMirror. `parentPageId`) in one call. Uses Docmost's import API for clean Markdown→ProseMirror.
- **`rename_page`** — Change a page's title only, without touching or resending content. - **`renamePage`** — Change a page's title only, without touching or resending content.
- **`move_page`** — Re-parent a page (nest it, or move to root); supports fractional-index - **`movePage`** — Re-parent a page (nest it, or move to root); supports fractional-index
positioning. Returns only on a *positively confirmed* success. positioning. Returns only on a *positively confirmed* success.
- **`delete_page`** — Delete a single page. - **`deletePage`** — Delete a single page.
- **`copy_page_content`** — Replace one page's body with a copy of another's, **entirely - **`copyPageContent`** — Replace one page's body with a copy of another's, **entirely
server-side** — the document never passes through the model. The target keeps its own server-side** — the document never passes through the model. The target keeps its own
title and slug (so its URL is preserved). title and slug (so its URL is preserved).
### Editing ### Editing
- **`edit_page_text`** — Surgical find/replace inside a page's text. Preserves **all** - **`editPageText`** — Surgical find/replace inside a page's text. Preserves **all**
structure: block ids, marks, links, callouts, tables. The preferred tool for fixing structure: block ids, marks, links, callouts, tables. The preferred tool for fixing
wording, typos, numbers and names. wording, typos, numbers and names.
- **`patch_node`** — Replace a single block addressed by its `attrs.id` (from - **`patchNode`** — Replace a single block addressed by its `attrs.id` (from
`get_page_json`), without resending the document. `getPageJson`), without resending the document.
- **`insert_node`** — Insert a block before/after another (by `attrs.id` or anchor text), - **`insertNode`** — Insert a block before/after another (by `attrs.id` or anchor text),
or append at the end. or append at the end.
- **`delete_node`** — Remove a single block by its `attrs.id`. - **`deleteNode`** — Remove a single block by its `attrs.id`.
- **`update_page_json`** — Replace a page's entire content with a ProseMirror document - **`updatePageJson`** — Replace a page's entire content with a ProseMirror document
(bulk rewrites, or when nodes lack ids). `content` is optional — omit it to update only (bulk rewrites, or when nodes lack ids). `content` is optional — omit it to update only
the title. Keeps the block ids you pass in, so heading anchors and history stay stable. the title. Keeps the block ids you pass in, so heading anchors and history stay stable.
- **`update_page_markdown`** — Replace a page's body (and optionally its title) with new - **`updatePageMarkdown`** — Replace a page's body (and optionally its title) with new
**plain Markdown**. The whole body is re-imported (block ids regenerate — for surgical or **plain Markdown**. The whole body is re-imported (block ids regenerate — for surgical or
id-preserving edits prefer `edit_page_text` / `patch_node` / `update_page_json`). id-preserving edits prefer `editPageText` / `patchNode` / `updatePageJson`).
Docmost-flavoured markdown is parsed, including `^[...]` inline footnotes. Docmost-flavoured markdown is parsed, including `^[...]` inline footnotes.
- **`docmost_transform`** — The agent-native editing interface: instead of retyping a - **`docmostTransform`** — The agent-native editing interface: instead of retyping a
document, the agent **writes a function that fixes it**. Edit a page by running an document, the agent **writes a function that fixes it**. Edit a page by running an
arbitrary **`(doc, ctx) => doc` JavaScript transform** against its *live* ProseMirror arbitrary **`(doc, ctx) => doc` JavaScript transform** against its *live* ProseMirror
document. Runs **sandboxed** document. Runs **sandboxed**
@@ -172,42 +174,46 @@ All 41 tools, grouped by what you'd reach for them.
### Tables ### Tables
- **`table_get`** — Read a table as a matrix: `{rows, cols, cells (text[][]), cellIds}` - **`tableGet`** — Read a table as a matrix: `{rows, cols, cells (text[][]), cellIds}`
(a paragraph id per cell, or `null`). Address the table by `#<index>` (from (a paragraph id per cell, or `null`). Address the table by `#<index>` (from
`get_outline`) or any block id inside it. Use `cellIds` with `patch_node` for `getOutline`) or any block id inside it. Use `cellIds` with `patchNode` for
rich-formatted cell edits. rich-formatted cell edits.
- **`table_insert_row`** — Insert a row of plain-text cells, padded to the table's column - **`tableInsertRow`** — Insert a row of plain-text cells, padded to the table's column
count (passing more cells than columns is an error). `index` is the 0-based insert count (passing more cells than columns is an error). `index` is the 0-based insert
position (0 inserts before the header); omit it to append at the end. position (0 inserts before the header); omit it to append at the end.
- **`table_delete_row`** — Delete the row at a 0-based `index`. Refuses to delete a table's - **`tableDeleteRow`** — Delete the row at a 0-based `index`. Refuses to delete a table's
only row; deleting row 0 promotes the next row to header. only row; deleting row 0 promotes the next row to header.
- **`table_update_cell`** — Set the plain-text content of cell `[row, col]` (0-based). For - **`tableUpdateCell`** — Set the plain-text content of cell `[row, col]` (0-based). For
rich formatting, `patch_node` the cell's paragraph id from `table_get`. rich formatting, `patchNode` the cell's paragraph id from `tableGet`.
### Markdown round-trip ### Markdown round-trip
- **`export_page_markdown`** — Export a page to a single self-contained, **lossless - **`exportPageMarkdown`** — Export a page to a single self-contained
Docmost-flavoured Markdown** file: a meta header, the body with inline comment anchors **Docmost-flavoured Markdown** file: a meta header, the body with inline comment anchors
and diagrams, and a trailing comments-thread block. To replace a page's body from plain and diagrams, and a trailing comments-thread block. The download → edit → import
authoring Markdown, use `update_page_markdown`. round-trip regenerates block ids and **silently drops** the no-Markdown-representation
attr set (table merge spans/colwidth/background, indent, `callout.icon`,
`orderedList.type`, link `internal`/`target`/`rel`/`class`); keep those in ProseMirror
JSON if they must survive. To replace a page's body from plain authoring Markdown, use
`updatePageMarkdown`.
> **Removed in this release:** `import_page_markdown` (the round-trip parser for an > **Removed in this release:** `importPageMarkdown` (the round-trip parser for an
> exported Docmost-Markdown file) is **no longer exposed on the external MCP surface**. > exported Docmost-Markdown file) is **no longer exposed on the external MCP surface**.
> To replace a page's body from Markdown, use **`update_page_markdown`** (plain Markdown > To replace a page's body from Markdown, use **`updatePageMarkdown`** (plain Markdown
> body replace). See the CHANGELOG for the migration note. > body replace). See the CHANGELOG for the migration note.
### Images ### Images
- **`insert_image`** — Download an image from a web (http/https) URL and insert it in one - **`insertImage`** — Download an image from a web (http/https) URL and insert it in one
step: append it, drop it in place of a text placeholder (`replaceText`), or put it after step: append it, drop it in place of a text placeholder (`replaceText`), or put it after
a given block (`afterText`). Preserves all other block ids. a given block (`afterText`). Preserves all other block ids.
- **`replace_image`** — Swap an existing image for one fetched from a web (http/https) URL. - **`replaceImage`** — Swap an existing image for one fetched from a web (http/https) URL.
Uploads the new file as a **fresh Uploads the new file as a **fresh
attachment** (clean URL that renders and busts browser caches), then re-points every attachment** (clean URL that renders and busts browser caches), then re-points every
node referencing the old attachment (recursively, including callouts/tables) via the node referencing the old attachment (recursively, including callouts/tables) via the
live document, preserving comments, alignment and alt text. (In-place overwrite is live document, preserving comments, alignment and alt text. (In-place overwrite is
deliberately avoided — some Docmost versions corrupt the attachment on overwrite.) deliberately avoided — some Docmost versions corrupt the attachment on overwrite.)
- **`stash_page`** — Serialize a whole page (its full ProseMirror JSON) into an ephemeral - **`stashPage`** — Serialize a whole page (its full ProseMirror JSON) into an ephemeral
in-RAM blob and return ONLY a short anonymous URL — the body never enters the model in-RAM blob and return ONLY a short anonymous URL — the body never enters the model
context, so it is the way to hand a large page (and its images) to an external consumer context, so it is the way to hand a large page (and its images) to an external consumer
without truncation. Every internal file/image attachment is mirrored into the same without truncation. Every internal file/image attachment is mirrored into the same
@@ -218,35 +224,35 @@ All 41 tools, grouped by what you'd reach for them.
### Comments ### Comments
- **`create_comment`** — Add a page comment, optionally **anchored inline** to an exact - **`createComment`** — Add a page comment, optionally **anchored inline** to an exact
span of text (the first occurrence is wrapped in a comment mark). span of text (the first occurrence is wrapped in a comment mark).
- **`list_comments`** — List a page's comments (content returned as Markdown). - **`listComments`** — List a page's comments (content returned as Markdown).
- **`update_comment`** — Edit an existing comment. - **`updateComment`** — Edit an existing comment.
- **`delete_comment`** — Delete a comment. - **`deleteComment`** — Delete a comment.
- **`resolve_comment`** — Resolve (close) or reopen a comment thread (reversible). Only top-level - **`resolveComment`** — Resolve (close) or reopen a comment thread (reversible). Only top-level
comments can be resolved; the thread and its replies are kept, unlike `delete_comment`. comments can be resolved; the thread and its replies are kept, unlike `deleteComment`.
- **`check_new_comments`** — Find comments created after a given ISO-8601 timestamp across - **`checkNewComments`** — Find comments created after a given ISO-8601 timestamp across
a space, optionally scoped to a page subtree — ideal for an agent that watches a doc for a space, optionally scoped to a page subtree — ideal for an agent that watches a doc for
feedback. feedback.
### Versioning & history ### Versioning & history
- **`list_page_history`** — A page's saved versions (Docmost auto-snapshots on save), - **`listPageHistory`** — A page's saved versions (Docmost auto-snapshots on save),
newest first, cursor-paginated. Each item's id is the `historyId`. newest first, cursor-paginated. Each item's id is the `historyId`.
- **`diff_page_versions`** — Diff two versions (or a version against the live page). - **`diffPageVersions`** — Diff two versions (or a version against the live page).
Returns inserted/deleted text, integrity counts (images, links, tables, callouts, Returns inserted/deleted text, integrity counts (images, links, tables, callouts,
footnote markers), and a human-readable Markdown summary — computed with the same footnote markers), and a human-readable Markdown summary — computed with the same
pipeline Docmost's own history viewer uses. pipeline Docmost's own history viewer uses.
- **`restore_page_version`** — Write a saved version back as the current content. Docmost - **`restorePageVersion`** — Write a saved version back as the current content. Docmost
has no restore endpoint, so this creates a **new** snapshot — the restore is itself has no restore endpoint, so this creates a **new** snapshot — the restore is itself
revertible. revertible.
### Sharing ### Sharing
- **`share_page`** — Make a page publicly accessible (idempotent) and return its public - **`sharePage`** — Make a page publicly accessible (idempotent) and return its public
URL (`<app>/share/<key>/p/<slugId>`); optional search-engine indexing. URL (`<app>/share/<key>/p/<slugId>`); optional search-engine indexing.
- **`unshare_page`** — Revoke a page's public share. - **`unsharePage`** — Revoke a page's public share.
- **`list_shares`** — All public shares in the workspace, with titles and public URLs. - **`listShares`** — All public shares in the workspace, with titles and public URLs.
--- ---
@@ -255,27 +261,27 @@ All 41 tools, grouped by what you'd reach for them.
This same guidance is also delivered at runtime via the MCP server `instructions` field, This same guidance is also delivered at runtime via the MCP server `instructions` field,
so capable clients steer the model automatically. so capable clients steer the model automatically.
- **Text fixes** (wording, typos, numbers): `edit_page_text`. - **Text fixes** (wording, typos, numbers): `editPageText`.
- **One block** (paragraph/heading/callout/table cell): `patch_node` / `insert_node` / - **One block** (paragraph/heading/callout/table cell): `patchNode` / `insertNode` /
`delete_node`, addressing the node by its `attrs.id` from `get_page_json`. `deleteNode`, addressing the node by its `attrs.id` from `getPageJson`.
- **Images**: `insert_image` / `replace_image`. - **Images**: `insertImage` / `replaceImage`.
- **A new page**: `create_page`. - **A new page**: `createPage`.
- **Bulk rewrite, or nodes without ids**: `update_page_json` (ProseMirror) or - **Bulk rewrite, or nodes without ids**: `updatePageJson` (ProseMirror) or
`update_page_markdown` (plain Markdown body replace). `updatePageMarkdown` (plain Markdown body replace).
- **Multi-step / scripted rewrite** (renumbering, footnotes, coordinated edits): - **Multi-step / scripted rewrite** (renumbering, footnotes, coordinated edits):
`docmost_transform` — preview with `dryRun`, then apply. `docmostTransform` — preview with `dryRun`, then apply.
- **Copy a whole page's content from another page** (server-side): `copy_page_content`. - **Copy a whole page's content from another page** (server-side): `copyPageContent`.
- **Rename a page** (title only): `rename_page`. - **Rename a page** (title only): `renamePage`.
- **Reads**: `get_page` (Markdown) / `get_page_json` (lossless ProseMirror with ids). - **Reads**: `getPage` (Markdown) / `getPageJson` (lossless ProseMirror with ids).
- **Review changes**: `list_page_history``diff_page_versions``restore_page_version`. - **Review changes**: `listPageHistory``diffPageVersions``restorePageVersion`.
- **Comments**: `create_comment` (with optional inline anchoring) / `list_comments` / - **Comments**: `createComment` (with optional inline anchoring) / `listComments` /
`update_comment` / `resolve_comment` / `delete_comment` / `check_new_comments`. `updateComment` / `resolveComment` / `deleteComment` / `checkNewComments`.
- **Navigate a page cheaply** (find a section/table, grab a block id): `get_outline` - **Navigate a page cheaply** (find a section/table, grab a block id): `getOutline`
`get_node`. `getNode`.
- **Tables** (add/remove a row, set a cell): `table_get` / `table_insert_row` / - **Tables** (add/remove a row, set a cell): `tableGet` / `tableInsertRow` /
`table_delete_row` / `table_update_cell`. `tableDeleteRow` / `tableUpdateCell`.
- **Export a page as self-contained Markdown** (with comment anchors): `export_page_markdown`. - **Export a page as self-contained Markdown** (with comment anchors): `exportPageMarkdown`.
- **Replace a page's body from Markdown**: `update_page_markdown`. - **Replace a page's body from Markdown**: `updatePageMarkdown`.
--- ---
@@ -287,25 +293,46 @@ so capable clients steer the model automatically.
the debounced REST snapshot), then **reads → transforms → writes synchronously** in one the debounced REST snapshot), then **reads → transforms → writes synchronously** in one
tick so no remote update can interleave, and **waits for persistence acknowledgement** tick so no remote update can interleave, and **waits for persistence acknowledgement**
before returning. before returning.
- **Per-page write serialization.** A per-`pageId` async mutex ensures two MCP writes to - **Per-page write serialization.** A per-`pageId` async mutex (keyed by the resolved
the same page never overlap; different pages never block each other. page **UUID**, never a slugId) ensures two MCP writes to the same page never overlap;
different pages never block each other. The lock helper fails fast if it is ever handed
a non-UUID key, so a write path that forgot to resolve the id can never silently lock
under a split key.
**Deploy requirement — single instance or sticky sessions.** This mutex is an
in-process `Map`, and the cached collab sessions and the `stash_page` blob store are
RAM-only and process-local. Behind a **multi-replica** load balancer **without sticky
sessions**, two replicas can each "hold" the lock for the same page at once and per-page
serialization is silently lost. Run the MCP/app as a **single instance**, or pin each
page's traffic to one replica (sticky sessions / consistent hashing on the page id).
There is deliberately no cross-process (e.g. Postgres advisory) lock yet — a conscious
documented constraint. See the `Dockerfile` comment and the `MCP collaboration write
path` block in `.env.example`.
**Rights-staleness window.** A cached collab session writes under the token captured at
connect time (and the collab-token cache reuses a token for its TTL), so a **revoked**
page access can lag by up to `MCP_COLLAB_SESSION_MAX_AGE_MS` (the hard session lifetime,
default 10 min) before the next re-auth picks it up. Lower it to shorten the lag at the
cost of more reconnects. This bounded window is an accepted trade-off; there is no
push-based cache invalidation on a rights change.
- **Transparent re-authentication.** Login uses email/password; expired tokens are - **Transparent re-authentication.** Login uses email/password; expired tokens are
refreshed automatically on the first 401/403 (covering JSON, multipart upload, and the refreshed automatically on the first 401/403 (covering JSON, multipart upload, and the
collaboration-token path), with in-flight login de-duplication so a burst of calls collaboration-token path), with in-flight login de-duplication so a burst of calls
triggers a single re-login. triggers a single re-login.
- **Lossless and lossy reads.** `get_page_json` returns the exact ProseMirror tree with - **Precise reads.** `getPageJson` returns the exact ProseMirror tree with block ids;
block ids; `get_page` returns clean Markdown for convenience. `getPage` returns canonical Markdown that drops only a fixed, documented attr set.
- **Full Docmost schema.** Markdown↔ProseMirror conversion supports callouts (including - **Full Docmost schema.** Markdown↔ProseMirror conversion supports callouts (including
nested), task lists (bullet *and* numbered checklists), tables, math blocks, embeds, nested), task lists (bullet *and* numbered checklists), tables, math blocks, embeds,
highlights, sub/superscript and more, with defensive caps against pathological input. highlights, sub/superscript and more, with defensive caps against pathological input.
- **Structured tables & lossless Markdown round-trip.** Tables can be edited as a matrix - **Structured tables & Markdown round-trip.** Tables can be edited as a matrix
(read, insert/delete rows, set cells by `[row,col]`) without resending the document, and (read, insert/delete rows, set cells by `[row,col]`) without resending the document, and
a page can be exported to and re-imported from a self-contained Docmost-flavoured a page can be exported to and re-imported from a self-contained Docmost-flavoured
Markdown file that preserves inline comment anchors and diagrams. Markdown file that preserves inline comment anchors and diagrams (block ids regenerate
and a fixed no-Markdown-representation attr set is dropped — see `exportPageMarkdown`).
- **Token-optimized responses.** API responses are filtered down to the fields agents - **Token-optimized responses.** API responses are filtered down to the fields agents
actually need, and large collections (spaces, pages, comments, history) are paginated. actually need, and large collections (spaces, pages, comments, history) are paginated.
- **Hardened runtime.** Global handlers keep a stray socket error from tearing down the - **Hardened runtime.** Global handlers keep a stray socket error from tearing down the
stdio server; `move_page` requires a positively confirmed success; the diff engine stdio server; `movePage` requires a positively confirmed success; the diff engine
falls back to a coarse block diff rather than hard-failing on a pathological document. falls back to a coarse block diff rather than hard-failing on a pathological document.
--- ---
@@ -363,7 +390,7 @@ npm run test:e2e
This project began as a fork of [MrMartiniMo/docmost-mcp](https://github.com/MrMartiniMo/docmost-mcp) This project began as a fork of [MrMartiniMo/docmost-mcp](https://github.com/MrMartiniMo/docmost-mcp)
(by Moritz Krause) and extends it substantially — adding per-block node editing, (by Moritz Krause) and extends it substantially — adding per-block node editing,
surgical text edits, the sandboxed `docmost_transform`, version history / diff / restore, surgical text edits, the sandboxed `docmostTransform`, version history / diff / restore,
comments, image insert/replace, public sharing, server-side page copy, dual comments, image insert/replace, public sharing, server-side page copy, dual
JSON/Markdown reads, transparent re-authentication and significant hardening. The comment JSON/Markdown reads, transparent re-authentication and significant hardening. The comment
tools were ported from upstream PR #3 by Max Nikitin. Thanks to both. tools were ported from upstream PR #3 by Max Nikitin. Thanks to both.
+118 -90
View File
@@ -12,7 +12,7 @@
> небольшую функцию, которая чинит текст*, чем перечитывать и заново выдавать весь > небольшую функцию, которая чинит текст*, чем перечитывать и заново выдавать весь
> документ. Поэтому сервер построен вокруг того, как модели на самом деле удобно > документ. Поэтому сервер построен вокруг того, как модели на самом деле удобно
> редактировать: адресовать блок по id, сделать find/replace или передать трансформ > редактировать: адресовать блок по id, сделать find/replace или передать трансформ
> `(doc, ctx) => doc` и позволить модели *запрограммировать* правку. `docmost_transform` > `(doc, ctx) => doc` и позволить модели *запрограммировать* правку. `docmostTransform`
> это и есть такой интерфейс. Другие Docmost-MCP «заточены под человека» — они дают > это и есть такой интерфейс. Другие Docmost-MCP «заточены под человека» — они дают
> «открыть страницу» и «заменить страницу»; этот даёт примитивы редактирования, в которых > «открыть страницу» и «заменить страницу»; этот даёт примитивы редактирования, в которых
> модель сильна. > модель сильна.
@@ -43,7 +43,7 @@ Docmost-MCP не сочетают:
| **Нужна enterprise-лицензия** | **Нет** | **Да** | Нет | Нет | Нет | | **Нужна enterprise-лицензия** | **Нет** | **Да** | Нет | Нет | Нет |
| Аутентификация | email + пароль, **авто-переавторизация** | API-ключ | email + пароль | cookie `authToken` (копировать из DevTools) | API Docmost / **напрямую PostgreSQL** | | Аутентификация | email + пароль, **авто-переавторизация** | API-ключ | email + пароль | cookie `authToken` (копировать из DevTools) | API Docmost / **напрямую PostgreSQL** |
| Чтение страницы как Markdown | ✅ | ✅ | ✅ | ✅ | ✅ (только чтение) | | Чтение страницы как Markdown | ✅ | ✅ | ✅ | ✅ | ✅ (только чтение) |
| **Lossless Markdown round-trip** (экспорт/импорт, сохраняет якоря комментариев) | ✅ | — | — | — | — | | **Markdown round-trip** (экспорт/импорт, сохраняет якоря комментариев) | ✅ | — | — | — | — |
| Чтение **lossless ProseMirror JSON** (с id блоков) | ✅ | — | — | — | — | | Чтение **lossless ProseMirror JSON** (с id блоков) | ✅ | — | — | — | — |
| **Компактная структура страницы** (дешёвый поиск id блока) | ✅ | — | — | — | — | | **Компактная структура страницы** (дешёвый поиск id блока) | ✅ | — | — | — | — |
| **Получение одного блока** (по id или индексу) | ✅ | — | — | — | — | | **Получение одного блока** (по id или индексу) | ✅ | — | — | — | — |
@@ -71,9 +71,9 @@ Docmost-MCP не сочетают:
- **Экономия токенов при редактировании.** Большинство Docmost-MCP (и официальный) - **Экономия токенов при редактировании.** Большинство Docmost-MCP (и официальный)
предлагают только запись «заменить всю страницу» — агент вынужден скачать весь документ, предлагают только запись «заменить всю страницу» — агент вынужден скачать весь документ,
изменить и загрузить обратно, оплачивая весь документ **дважды** на каждой мелкой изменить и загрузить обратно, оплачивая весь документ **дважды** на каждой мелкой
правке. Этот сервер позволяет агенту изменить ровно один блок (`patch_node` / правке. Этот сервер позволяет агенту изменить ровно один блок (`patchNode` /
`insert_node` / `delete_node`), сделать find/replace с сохранением структуры `insertNode` / `deleteNode`), сделать find/replace с сохранением структуры
(`edit_page_text`) или скопировать страницу на стороне сервера (`copy_page_content`) — (`editPageText`) или скопировать страницу на стороне сервера (`copyPageContent`) —
**причём документ ни разу не проходит через модель**. **причём документ ни разу не проходит через модель**.
- **Записи, которые не воюют с редактором.** Наивная запись через REST конфликтует с тем, - **Записи, которые не воюют с редактором.** Наивная запись через REST конфликтует с тем,
@@ -87,12 +87,12 @@ Docmost-MCP не сочетают:
- **Агентоориентированная модель редактирования.** Серверы «под человека» дают «открыть - **Агентоориентированная модель редактирования.** Серверы «под человека» дают «открыть
страницу» и «заменить страницу», потому что это отражает то, как работает человек. Модель страницу» и «заменить страницу», потому что это отражает то, как работает человек. Модель
редактирует лучше, *программируя* правку — адресуя блоки по id, делая find/replace или редактирует лучше, *программируя* правку — адресуя блоки по id, делая find/replace или
передавая трансформ `(doc, ctx) => doc` (`docmost_transform`, с dry-run диффом перед передавая трансформ `(doc, ctx) => doc` (`docmostTransform`, с dry-run диффом перед
коммитом). Этот сервер построен вокруг этого — поэтому у него есть примитивы коммитом). Этот сервер построен вокруг этого — поэтому у него есть примитивы
редактирования, которых у остальных просто нет. редактирования, которых у остальных просто нет.
- **Страховка при редактировании, которой нет у других.** `list_page_history` - **Страховка при редактировании, которой нет у других.** `listPageHistory`
`diff_page_versions``restore_page_version` дают агенту (и вам) полный цикл «посмотреть `diffPageVersions``restorePageVersion` дают агенту (и вам) полный цикл «посмотреть
и откатить». Дифф использует *тот же* конвейер `recreateTransform → ChangeSet → и откатить». Дифф использует *тот же* конвейер `recreateTransform → ChangeSet →
simplifyChanges`, что и встроенный просмотр истории Docmost, так что результат совпадает simplifyChanges`, что и встроенный просмотр истории Docmost, так что результат совпадает
с продуктом. с продуктом.
@@ -113,59 +113,62 @@ Docmost-MCP не сочетают:
### Чтение и поиск ### Чтение и поиск
- **`get_workspace`** — Информация о текущем воркспейсе Docmost. - **`getWorkspace`** — Информация о текущем воркспейсе Docmost.
- **`list_spaces`** — Все пространства воркспейса. - **`listSpaces`** — Все пространства воркспейса.
- **`list_pages`** — Недавние страницы пространства, по убыванию `updatedAt` (по умолчанию - **`listPages`** — Недавние страницы пространства, по убыванию `updatedAt` (по умолчанию
50, максимум 100). Для поиска в больших пространствах используйте `search`. 50, максимум 100). Для поиска в больших пространствах используйте `search`.
- **`search`** — Полнотекстовый поиск по страницам и контенту (ограничен `limit`, максимум - **`search`** — Полнотекстовый поиск по страницам и контенту (ограничен `limit`, максимум
100). 100).
- **`get_page`** — Контент страницы как чистый **Markdown** (удобно, но это - **`getPage`** — Контент страницы как чистый **Markdown** (канонично для текста; теряет
*lossy*-представление — id блоков и точная структура таблиц/коллаутов аппроксимируются). лишь id блоков, якоря разрешённых комментариев и фиксированный набор атрибутов без
- **`get_page_json`** — **Lossless ProseMirror/TipTap JSON** страницы, включая `attrs.id` markdown-представления — спаны/colwidth/фон ячеек таблиц, отступы (indent),
`callout.icon`, `orderedList.type` и `internal`/`target`/`rel`/`class` у ссылок;
используйте `getPageJson`, когда они нужны).
- **`getPageJson`** — **Lossless ProseMirror/TipTap JSON** страницы, включая `attrs.id`
каждого блока и `slugId`, используемый в URL. Именно его потребляют инструменты каждого блока и `slugId`, используемый в URL. Именно его потребляют инструменты
поблочного редактирования. поблочного редактирования.
- **`get_outline`** — Компактная структура страницы из блоков верхнего уровня (`{index, - **`getOutline`** — Компактная структура страницы из блоков верхнего уровня (`{index,
type, id, level, firstText}`; для таблиц добавляются число строк/столбцов и тексты ячеек type, id, level, firstText}`; для таблиц добавляются число строк/столбцов и тексты ячеек
заголовка, для списков — число пунктов) **без** тела документа. Дешёвый способ найти раздел или таблицу и получить заголовка, для списков — число пунктов) **без** тела документа. Дешёвый способ найти раздел или таблицу и получить
id блока перед `get_node` / `patch_node` / `insert_node`. id блока перед `getNode` / `patchNode` / `insertNode`.
- **`get_node`** — Получить полное ProseMirror-поддерево одного блока (lossless), не - **`getNode`** — Получить полное ProseMirror-поддерево одного блока (lossless), не
вытягивая всю страницу. Адресуйте его по id блока (из `get_outline` / `get_page_json`) вытягивая всю страницу. Адресуйте его по id блока (из `getOutline` / `getPageJson`)
или формой `#<index>` для блока верхнего уровня — используйте `#<index>` для или формой `#<index>` для блока верхнего уровня — используйте `#<index>` для
таблиц/строк/ячеек, у которых нет id. таблиц/строк/ячеек, у которых нет id.
### Жизненный цикл страниц ### Жизненный цикл страниц
- **`create_page`** — Создать страницу из Markdown и поместить в иерархию (опционально - **`createPage`** — Создать страницу из Markdown и поместить в иерархию (опционально
`parentPageId`) одним вызовом. Использует import API Docmost для чистой конвертации `parentPageId`) одним вызовом. Использует import API Docmost для чистой конвертации
Markdown→ProseMirror. Markdown→ProseMirror.
- **`rename_page`** — Изменить только заголовок страницы, не трогая и не пересылая контент. - **`renamePage`** — Изменить только заголовок страницы, не трогая и не пересылая контент.
- **`move_page`** — Сменить родителя страницы (вложить или вынести в корень); поддерживает - **`movePage`** — Сменить родителя страницы (вложить или вынести в корень); поддерживает
позиционирование по fractional-index. Возвращает успех только при *положительно позиционирование по fractional-index. Возвращает успех только при *положительно
подтверждённом* результате. подтверждённом* результате.
- **`delete_page`** — Удалить одну страницу. - **`deletePage`** — Удалить одну страницу.
- **`copy_page_content`** — Заменить тело одной страницы копией тела другой, **полностью на - **`copyPageContent`** — Заменить тело одной страницы копией тела другой, **полностью на
стороне сервера** — документ не проходит через модель. У целевой страницы сохраняются стороне сервера** — документ не проходит через модель. У целевой страницы сохраняются
собственные заголовок и slug (URL не меняется). собственные заголовок и slug (URL не меняется).
### Редактирование ### Редактирование
- **`edit_page_text`** — Хирургический find/replace внутри текста страницы. Сохраняет - **`editPageText`** — Хирургический find/replace внутри текста страницы. Сохраняет
**всю** структуру: id блоков, marks, ссылки, коллауты, таблицы. Предпочтительный **всю** структуру: id блоков, marks, ссылки, коллауты, таблицы. Предпочтительный
инструмент для правки формулировок, опечаток, чисел и имён. инструмент для правки формулировок, опечаток, чисел и имён.
- **`patch_node`** — Заменить один блок, адресованный по `attrs.id` (из `get_page_json`), - **`patchNode`** — Заменить один блок, адресованный по `attrs.id` (из `getPageJson`),
без пересылки документа. без пересылки документа.
- **`insert_node`** — Вставить блок до/после другого (по `attrs.id` или по якорному тексту) - **`insertNode`** — Вставить блок до/после другого (по `attrs.id` или по якорному тексту)
либо добавить в конец. либо добавить в конец.
- **`delete_node`** — Удалить один блок по его `attrs.id`. - **`deleteNode`** — Удалить один блок по его `attrs.id`.
- **`update_page_json`** — Заменить весь контент страницы документом ProseMirror (массовые - **`updatePageJson`** — Заменить весь контент страницы документом ProseMirror (массовые
перезаписи или когда у узлов нет id). `content` опционален — опустите его, чтобы изменить перезаписи или когда у узлов нет id). `content` опционален — опустите его, чтобы изменить
только заголовок. Сохраняет переданные id блоков, поэтому якоря заголовков и история только заголовок. Сохраняет переданные id блоков, поэтому якоря заголовков и история
остаются стабильными. остаются стабильными.
- **`update_page_markdown`** — Заменить тело страницы (и опционально заголовок) новым - **`updatePageMarkdown`** — Заменить тело страницы (и опционально заголовок) новым
**обычным Markdown**. Всё тело переимпортируется (id блоков перегенерируются — для **обычным Markdown**. Всё тело переимпортируется (id блоков перегенерируются — для
хирургических правок или сохранения id используйте `edit_page_text` / `patch_node` / хирургических правок или сохранения id используйте `editPageText` / `patchNode` /
`update_page_json`). Markdown в диалекте Docmost разбирается, включая inline-сноски `^[...]`. `updatePageJson`). Markdown в диалекте Docmost разбирается, включая inline-сноски `^[...]`.
- **`docmost_transform`** — Агентоориентированный интерфейс редактирования: вместо - **`docmostTransform`** — Агентоориентированный интерфейс редактирования: вместо
перепечатывания документа агент **пишет функцию, которая его чинит**. Редактирует перепечатывания документа агент **пишет функцию, которая его чинит**. Редактирует
страницу, запуская произвольный **JS-трансформ `(doc, ctx) => doc`** на её *живом* страницу, запуская произвольный **JS-трансформ `(doc, ctx) => doc`** на её *живом*
документе ProseMirror. Работает в **песочнице** (без `require`/`process`/`fs`/сети, документе ProseMirror. Работает в **песочнице** (без `require`/`process`/`fs`/сети,
@@ -177,42 +180,46 @@ Docmost-MCP не сочетают:
### Таблицы ### Таблицы
- **`table_get`** — Прочитать таблицу как матрицу: `{rows, cols, cells (text[][]), - **`tableGet`** — Прочитать таблицу как матрицу: `{rows, cols, cells (text[][]),
cellIds}` (id абзаца на ячейку или `null`). Адресуйте таблицу через `#<index>` (из cellIds}` (id абзаца на ячейку или `null`). Адресуйте таблицу через `#<index>` (из
`get_outline`) или любой id блока внутри неё. Используйте `cellIds` вместе с `patch_node` `getOutline`) или любой id блока внутри неё. Используйте `cellIds` вместе с `patchNode`
для правок ячеек с форматированием. для правок ячеек с форматированием.
- **`table_insert_row`** — Вставить строку из текстовых ячеек, дополненную до числа - **`tableInsertRow`** — Вставить строку из текстовых ячеек, дополненную до числа
столбцов таблицы (передать ячеек больше числа столбцов — ошибка). `index` — 0-based столбцов таблицы (передать ячеек больше числа столбцов — ошибка). `index` — 0-based
позиция вставки (0 вставляет перед заголовком); опустите, чтобы добавить в конец. позиция вставки (0 вставляет перед заголовком); опустите, чтобы добавить в конец.
- **`table_delete_row`** — Удалить строку по 0-based `index`. Отказывается удалять - **`tableDeleteRow`** — Удалить строку по 0-based `index`. Отказывается удалять
единственную строку таблицы; удаление строки 0 делает заголовком следующую строку. единственную строку таблицы; удаление строки 0 делает заголовком следующую строку.
- **`table_update_cell`** — Задать текстовое содержимое ячейки `[row, col]` (0-based). Для - **`tableUpdateCell`** — Задать текстовое содержимое ячейки `[row, col]` (0-based). Для
форматирования используйте `patch_node` по id абзаца ячейки из `table_get`. форматирования используйте `patchNode` по id абзаца ячейки из `tableGet`.
### Markdown: экспорт и импорт ### Markdown: экспорт и импорт
- **`export_page_markdown`** — Экспортировать страницу в один самодостаточный, **lossless - **`exportPageMarkdown`** — Экспортировать страницу в один самодостаточный
Markdown в диалекте Docmost**: мета-заголовок, тело с inline-якорями комментариев и **Markdown в диалекте Docmost**: мета-заголовок, тело с inline-якорями комментариев и
диаграммами и завершающий блок тредов комментариев. Чтобы заменить тело страницы из диаграммами и завершающий блок тредов комментариев. Round-trip скачать → отредактировать →
обычного авторского Markdown, используйте `update_page_markdown`. импортировать перегенерирует id блоков и **молча отбрасывает** набор атрибутов без
markdown-представления (спаны/colwidth/фон ячеек таблиц, отступы (indent), `callout.icon`,
`orderedList.type`, `internal`/`target`/`rel`/`class` у ссылок); держите их в ProseMirror
JSON, если они должны выжить. Чтобы заменить тело страницы из обычного авторского Markdown,
используйте `updatePageMarkdown`.
> **Удалено в этом релизе:** `import_page_markdown` (парсер round-trip для > **Удалено в этом релизе:** `importPageMarkdown` (парсер round-trip для
> экспортированного Docmost-Markdown-файла) **больше не отдаётся на внешней MCP-поверхности**. > экспортированного Docmost-Markdown-файла) **больше не отдаётся на внешней MCP-поверхности**.
> Чтобы заменить тело страницы из Markdown, используйте **`update_page_markdown`** (замена > Чтобы заменить тело страницы из Markdown, используйте **`updatePageMarkdown`** (замена
> тела обычным Markdown). См. заметку о миграции в CHANGELOG. > тела обычным Markdown). См. заметку о миграции в CHANGELOG.
### Изображения ### Изображения
- **`insert_image`** — Загрузить локальное изображение и вставить за один шаг: добавить в - **`insertImage`** — Загрузить локальное изображение и вставить за один шаг: добавить в
конец, поставить вместо текстового плейсхолдера (`replaceText`) или после заданного блока конец, поставить вместо текстового плейсхолдера (`replaceText`) или после заданного блока
(`afterText`). Сохраняет id всех остальных блоков. (`afterText`). Сохраняет id всех остальных блоков.
- **`replace_image`** — Заменить существующее изображение. Загружает новый файл как **новое - **`replaceImage`** — Заменить существующее изображение. Загружает новый файл как **новое
вложение** (чистый URL, который рендерится и сбрасывает кэш браузера), затем вложение** (чистый URL, который рендерится и сбрасывает кэш браузера), затем
перенаправляет все узлы, ссылавшиеся на старое вложение (рекурсивно, включая перенаправляет все узлы, ссылавшиеся на старое вложение (рекурсивно, включая
коллауты/таблицы), через живой документ, сохраняя комментарии, выравнивание и alt-текст. коллауты/таблицы), через живой документ, сохраняя комментарии, выравнивание и alt-текст.
(Перезапись «по месту» намеренно не используется — некоторые версии Docmost портят (Перезапись «по месту» намеренно не используется — некоторые версии Docmost портят
вложение при перезаписи.) вложение при перезаписи.)
- **`stash_page`** — Сериализовать страницу целиком (её полный ProseMirror JSON) в - **`stashPage`** — Сериализовать страницу целиком (её полный ProseMirror JSON) в
эфемерный blob в оперативной памяти и вернуть ТОЛЬКО короткий анонимный URL — тело эфемерный blob в оперативной памяти и вернуть ТОЛЬКО короткий анонимный URL — тело
никогда не попадает в контекст модели, поэтому это способ передать большую страницу никогда не попадает в контекст модели, поэтому это способ передать большую страницу
(вместе с её изображениями) внешнему потребителю без усечения. Каждое внутреннее (вместе с её изображениями) внешнему потребителю без усечения. Каждое внутреннее
@@ -224,35 +231,35 @@ Docmost-MCP не сочетают:
### Комментарии ### Комментарии
- **`create_comment`** — Добавить комментарий к странице, опционально **привязав inline** к - **`createComment`** — Добавить комментарий к странице, опционально **привязав inline** к
точному фрагменту текста (первое вхождение оборачивается comment-маркой). точному фрагменту текста (первое вхождение оборачивается comment-маркой).
- **`list_comments`** — Список комментариев страницы (контент возвращается как Markdown). - **`listComments`** — Список комментариев страницы (контент возвращается как Markdown).
- **`update_comment`** — Изменить существующий комментарий. - **`updateComment`** — Изменить существующий комментарий.
- **`delete_comment`** — Удалить комментарий. - **`deleteComment`** — Удалить комментарий.
- **`resolve_comment`** — Закрыть (resolve) или переоткрыть тред комментария (обратимо). Resolve - **`resolveComment`** — Закрыть (resolve) или переоткрыть тред комментария (обратимо). Resolve
доступен только для корневых комментариев; тред и ответы сохраняются, в отличие от `delete_comment`. доступен только для корневых комментариев; тред и ответы сохраняются, в отличие от `deleteComment`.
- **`check_new_comments`** — Найти комментарии, созданные после заданной метки времени - **`checkNewComments`** — Найти комментарии, созданные после заданной метки времени
ISO-8601, по пространству, опционально в рамках поддерева страниц — идеально для агента, ISO-8601, по пространству, опционально в рамках поддерева страниц — идеально для агента,
который следит за обратной связью в документе. который следит за обратной связью в документе.
### Версии и история ### Версии и история
- **`list_page_history`** — Сохранённые версии страницы (Docmost авто-снапшотит при каждом - **`listPageHistory`** — Сохранённые версии страницы (Docmost авто-снапшотит при каждом
сохранении), новые сверху, курсорная пагинация. id каждого элемента — это `historyId`. сохранении), новые сверху, курсорная пагинация. id каждого элемента — это `historyId`.
- **`diff_page_versions`** — Дифф двух версий (или версии против живой страницы). - **`diffPageVersions`** — Дифф двух версий (или версии против живой страницы).
Возвращает вставленный/удалённый текст, счётчики целостности (изображения, ссылки, Возвращает вставленный/удалённый текст, счётчики целостности (изображения, ссылки,
таблицы, коллауты, маркеры сносок) и человекочитаемую Markdown-сводку — посчитано тем же таблицы, коллауты, маркеры сносок) и человекочитаемую Markdown-сводку — посчитано тем же
конвейером, что использует встроенный просмотр истории Docmost. конвейером, что использует встроенный просмотр истории Docmost.
- **`restore_page_version`** — Записать сохранённую версию обратно как текущий контент. У - **`restorePageVersion`** — Записать сохранённую версию обратно как текущий контент. У
Docmost нет эндпоинта восстановления, поэтому создаётся **новый** снапшот — само Docmost нет эндпоинта восстановления, поэтому создаётся **новый** снапшот — само
восстановление тоже обратимо. восстановление тоже обратимо.
### Публикация ### Публикация
- **`share_page`** — Сделать страницу публично доступной (идемпотентно) и вернуть её - **`sharePage`** — Сделать страницу публично доступной (идемпотентно) и вернуть её
публичный URL (`<app>/share/<key>/p/<slugId>`); опционально индексирование поисковиками. публичный URL (`<app>/share/<key>/p/<slugId>`); опционально индексирование поисковиками.
- **`unshare_page`** — Отозвать публичный доступ к странице. - **`unsharePage`** — Отозвать публичный доступ к странице.
- **`list_shares`** — Все публичные ссылки воркспейса с заголовками и публичными URL. - **`listShares`** — Все публичные ссылки воркспейса с заголовками и публичными URL.
--- ---
@@ -261,29 +268,29 @@ Docmost-MCP не сочетают:
Та же подсказка отдаётся в рантайме через поле `instructions` MCP-сервера, так что Та же подсказка отдаётся в рантайме через поле `instructions` MCP-сервера, так что
подходящие клиенты направляют модель автоматически. подходящие клиенты направляют модель автоматически.
- **Правки текста** (формулировки, опечатки, числа): `edit_page_text`. - **Правки текста** (формулировки, опечатки, числа): `editPageText`.
- **Один блок** (абзац/заголовок/коллаут/ячейка таблицы): `patch_node` / `insert_node` / - **Один блок** (абзац/заголовок/коллаут/ячейка таблицы): `patchNode` / `insertNode` /
`delete_node`, адресуя узел по его `attrs.id` из `get_page_json`. `deleteNode`, адресуя узел по его `attrs.id` из `getPageJson`.
- **Изображения**: `insert_image` / `replace_image`. - **Изображения**: `insertImage` / `replaceImage`.
- **Новая страница**: `create_page`. - **Новая страница**: `createPage`.
- **Массовая перезапись или узлы без id**: `update_page_json` (ProseMirror) или - **Массовая перезапись или узлы без id**: `updatePageJson` (ProseMirror) или
`update_page_markdown` (замена тела обычным Markdown). `updatePageMarkdown` (замена тела обычным Markdown).
- **Многошаговая / скриптовая перезапись** (перенумерация, сноски, согласованные правки): - **Многошаговая / скриптовая перезапись** (перенумерация, сноски, согласованные правки):
`docmost_transform` — предпросмотр через `dryRun`, затем применение. `docmostTransform` — предпросмотр через `dryRun`, затем применение.
- **Скопировать контент целой страницы из другой** (на стороне сервера): - **Скопировать контент целой страницы из другой** (на стороне сервера):
`copy_page_content`. `copyPageContent`.
- **Переименовать страницу** (только заголовок): `rename_page`. - **Переименовать страницу** (только заголовок): `renamePage`.
- **Чтение**: `get_page` (Markdown) / `get_page_json` (lossless ProseMirror с id). - **Чтение**: `getPage` (Markdown) / `getPageJson` (lossless ProseMirror с id).
- **Просмотр изменений**: `list_page_history``diff_page_versions` - **Просмотр изменений**: `listPageHistory``diffPageVersions`
`restore_page_version`. `restorePageVersion`.
- **Комментарии**: `create_comment` (с опциональной inline-привязкой) / `list_comments` / - **Комментарии**: `createComment` (с опциональной inline-привязкой) / `listComments` /
`update_comment` / `resolve_comment` / `delete_comment` / `check_new_comments`. `updateComment` / `resolveComment` / `deleteComment` / `checkNewComments`.
- **Дешёвая навигация по странице** (найти раздел/таблицу, получить id блока): `get_outline` - **Дешёвая навигация по странице** (найти раздел/таблицу, получить id блока): `getOutline`
`get_node`. `getNode`.
- **Таблицы** (добавить/удалить строку, задать ячейку): `table_get` / `table_insert_row` / - **Таблицы** (добавить/удалить строку, задать ячейку): `tableGet` / `tableInsertRow` /
`table_delete_row` / `table_update_cell`. `tableDeleteRow` / `tableUpdateCell`.
- **Экспорт страницы в самодостаточный Markdown** (с якорями комментариев): `export_page_markdown`. - **Экспорт страницы в самодостаточный Markdown** (с якорями комментариев): `exportPageMarkdown`.
- **Заменить тело страницы из Markdown**: `update_page_markdown`. - **Заменить тело страницы из Markdown**: `updatePageMarkdown`.
--- ---
@@ -295,28 +302,49 @@ Docmost-MCP не сочетают:
правки, которых ещё нет в дебаунс-снапшоте REST), затем **читает → трансформирует → правки, которых ещё нет в дебаунс-снапшоте REST), затем **читает → трансформирует →
пишет синхронно** в одном тике, чтобы никакое удалённое обновление не вклинилось, и пишет синхронно** в одном тике, чтобы никакое удалённое обновление не вклинилось, и
**ждёт подтверждения сохранения** до возврата. **ждёт подтверждения сохранения** до возврата.
- **Сериализация записи по странице.** Асинхронный мьютекс по `pageId` гарантирует, что - **Сериализация записи по странице.** Асинхронный мьютекс по разрешённому **UUID**
две записи MCP в одну страницу никогда не пересекаются; разные страницы друг друга не страницы (никогда не по slugId) гарантирует, что две записи MCP в одну страницу никогда
блокируют. не пересекаются; разные страницы друг друга не блокируют. Хелпер блокировки падает сразу
(fail-fast), если ему передали не-UUID ключ, — путь записи, забывший разрезолвить id, не
сможет молча взять лок под расщеплённым ключом.
**Требование к деплою — один инстанс или sticky-сессии.** Этот мьютекс — процесс-локальный
`Map`, а кэш collab-сессий и хранилище `stashPage` живут только в RAM одного процесса. За
**мультиреплика**-балансировщиком **без sticky-сессий** две реплики могут одновременно
«держать» лок одной страницы, и сериализация по странице молча теряется. Запускайте
MCP/приложение **одним инстансом** либо прибивайте трафик страницы к одной реплике
(sticky-сессии / consistent hashing по id страницы). Кросс-процессной блокировки (например,
Postgres advisory-lock) намеренно пока нет — осознанное задокументированное ограничение.
См. комментарий в `Dockerfile` и блок `MCP collaboration write path` в `.env.example`.
**Окно устаревших прав.** Кэшированная collab-сессия пишет под токеном, захваченным в
момент connect (а кэш collab-токена переиспользует токен в пределах своего TTL), поэтому
**отозванный** доступ к странице может лагать до `MCP_COLLAB_SESSION_MAX_AGE_MS` (жёсткий
срок жизни сессии, по умолчанию 10 мин), пока следующая переавторизация его не подхватит.
Уменьшите значение, чтобы сократить лаг ценой большего числа переподключений. Это
ограниченное окно — принятый trade-off; push-инвалидации кэша при смене прав нет.
- **Прозрачная переавторизация.** Логин по email/паролю; истёкшие токены обновляются - **Прозрачная переавторизация.** Логин по email/паролю; истёкшие токены обновляются
автоматически на первом 401/403 (покрывая JSON, multipart-загрузку и путь токена автоматически на первом 401/403 (покрывая JSON, multipart-загрузку и путь токена
коллаборации), с дедупликацией параллельных логинов, так что пачка вызовов вызывает один коллаборации), с дедупликацией параллельных логинов, так что пачка вызовов вызывает один
повторный логин. повторный логин.
- **Lossless- и lossy-чтение.** `get_page_json` возвращает точное дерево ProseMirror с id - **Точные чтения.** `getPageJson` возвращает точное дерево ProseMirror с id блоков;
блоков; `get_page` возвращает чистый Markdown для удобства. `getPage` возвращает канонический Markdown, теряющий лишь фиксированный, документированный
набор атрибутов.
- **Полная схема Docmost.** Конвертация Markdown↔ProseMirror поддерживает коллауты - **Полная схема Docmost.** Конвертация Markdown↔ProseMirror поддерживает коллауты
(включая вложенные), списки задач (маркированные *и* нумерованные чек-листы), таблицы, (включая вложенные), списки задач (маркированные *и* нумерованные чек-листы), таблицы,
блоки формул, эмбеды, выделение, под/надстрочный текст и прочее, с защитными лимитами блоки формул, эмбеды, выделение, под/надстрочный текст и прочее, с защитными лимитами
против патологического ввода. против патологического ввода.
- **Структурные таблицы и lossless Markdown round-trip.** Таблицы можно редактировать как - **Структурные таблицы и Markdown round-trip.** Таблицы можно редактировать как
матрицу (чтение, вставка/удаление строк, задание ячеек по `[row, col]`) без пересылки матрицу (чтение, вставка/удаление строк, задание ячеек по `[row, col]`) без пересылки
документа, а страницу — экспортировать и заново импортировать как самодостаточный документа, а страницу — экспортировать и заново импортировать как самодостаточный
Markdown-файл в диалекте Docmost, сохраняющий inline-якоря комментариев и диаграммы. Markdown-файл в диалекте Docmost, сохраняющий inline-якоря комментариев и диаграммы
(id блоков перегенерируются, а фиксированный набор атрибутов без markdown-представления
отбрасывается — см. `exportPageMarkdown`).
- **Ответы, оптимизированные по токенам.** Ответы API урезаются до полей, действительно - **Ответы, оптимизированные по токенам.** Ответы API урезаются до полей, действительно
нужных агентам, а большие коллекции (пространства, страницы, комментарии, история) нужных агентам, а большие коллекции (пространства, страницы, комментарии, история)
пагинируются. пагинируются.
- **Закалённый рантайм.** Глобальные обработчики не дают случайной ошибке сокета уронить - **Закалённый рантайм.** Глобальные обработчики не дают случайной ошибке сокета уронить
stdio-сервер; `move_page` требует положительно подтверждённого успеха; движок диффа stdio-сервер; `movePage` требует положительно подтверждённого успеха; движок диффа
откатывается к грубому поблочному диффу, а не падает на патологическом документе. откатывается к грубому поблочному диффу, а не падает на патологическом документе.
--- ---
@@ -376,7 +404,7 @@ npm run test:e2e
Проект начинался как форк Проект начинался как форк
[MrMartiniMo/docmost-mcp](https://github.com/MrMartiniMo/docmost-mcp) (автор Moritz Krause) [MrMartiniMo/docmost-mcp](https://github.com/MrMartiniMo/docmost-mcp) (автор Moritz Krause)
и существенно его расширяет — добавлены поблочное редактирование узлов, хирургические и существенно его расширяет — добавлены поблочное редактирование узлов, хирургические
правки текста, песочница `docmost_transform`, история версий / дифф / восстановление, правки текста, песочница `docmostTransform`, история версий / дифф / восстановление,
комментарии, вставка/замена изображений, публичные ссылки, серверное копирование страниц, комментарии, вставка/замена изображений, публичные ссылки, серверное копирование страниц,
двойное чтение JSON/Markdown, прозрачная переавторизация и значительное упрочнение. двойное чтение JSON/Markdown, прозрачная переавторизация и значительное упрочнение.
Инструменты комментариев портированы из upstream PR #3 от Max Nikitin. Спасибо обоим. Инструменты комментариев портированы из upstream PR #3 от Max Nikitin. Спасибо обоим.
+10 -10
View File
@@ -22,14 +22,14 @@ are debounced server-side, so the script waits ~16 s before reading back via RES
| # | Tool / path | What is checked | Expected | | # | Tool / path | What is checked | Expected |
|---|-------------|-----------------|----------| |---|-------------|-----------------|----------|
| 1 | `create_page` | title with spaces, slugId returned | page created, title intact | | 1 | `createPage` | title with spaces, slugId returned | page created, title intact |
| 2 | `update_page` (markdown) | headings, **bold**/*italic*/~~strike~~/`code`/link, nested bullet + ordered lists, blockquote, code block, `:::callout:::`, table | all structures survive re-import | | 2 | `update_page` (markdown) | headings, **bold**/*italic*/~~strike~~/`code`/link, nested bullet + ordered lists, blockquote, code block, `:::callout:::`, table | all structures survive re-import |
| 3 | `get_page_json` | lossless ProseMirror, block ids, callout/table nodes | present (note: reads the **debounced** REST snapshot — recent collab writes may lag a few seconds) | | 3 | `getPageJson` | lossless ProseMirror, block ids, callout/table nodes | present (note: reads the **debounced** REST snapshot — recent collab writes may lag a few seconds) |
| 4 | `edit_page_text` | surgical replace; block ids + marks preserved; ambiguous match rejected; missing match reported | edits applied, ids stable, errors correct | | 4 | `editPageText` | surgical replace; block ids + marks preserved; ambiguous match rejected; missing match reported | edits applied, ids stable, errors correct |
| 5 | `update_page_json` | full lossless write; custom block ids preserved; existing content (text edits, images, callout, table) not lost | round-trips intact | | 5 | `updatePageJson` | full lossless write; custom block ids preserved; existing content (text edits, images, callout, table) not lost | round-trips intact |
| 6 | `upload_image` | uploads attachment, returns node | src is a **clean** `/api/files/<id>/<file>` URL, served `200 image/*` | | 6 | `upload_image` | uploads attachment, returns node | src is a **clean** `/api/files/<id>/<file>` URL, served `200 image/*` |
| 7 | `insert_image` (append / `replaceText` / `afterText`) | three placements | image lands in the right place, all other block ids preserved | | 7 | `insertImage` (append / `replaceText` / `afterText`) | three placements | image lands in the right place, all other block ids preserved |
| 8 | **`replace_image`** | swap an existing figure for new bytes; comments/align/alt preserved; **the new URL must actually serve the image** | new image renders (`200`), old node repointed | | 8 | **`replaceImage`** | swap an existing figure for new bytes; comments/align/alt preserved; **the new URL must actually serve the image** | new image renders (`200`), old node repointed |
## Image-specific assertions (the recurring bug area) ## Image-specific assertions (the recurring bug area)
@@ -39,7 +39,7 @@ For every uploaded/inserted/replaced image, assert at the HTTP level that the
* `GET <src>``200`, `Content-Type: image/*`, body starts with the image magic * `GET <src>``200`, `Content-Type: image/*`, body starts with the image magic
(`89 50 4E 47` for PNG, etc.). (`89 50 4E 47` for PNG, etc.).
* `src` does **not** contain a `?v=` query (see "Known pitfalls"). * `src` does **not** contain a `?v=` query (see "Known pitfalls").
* After `replace_image`: the returned `newAttachmentId` **differs** from the old * After `replaceImage`: the returned `newAttachmentId` **differs** from the old
one (replacement uses a fresh attachment → fresh URL), and `GET <new src>``200`. one (replacement uses a fresh attachment → fresh URL), and `GET <new src>``200`.
* The old image node on the page is repointed to the new attachmentId. * The old image node on the page is repointed to the new attachmentId.
@@ -64,7 +64,7 @@ broken/empty figure.
Uploading with an existing `attachmentId` (`POST /files/upload` + `attachmentId`) Uploading with an existing `attachmentId` (`POST /files/upload` + `attachmentId`)
overwrites the bytes in place. On this Docmost the attachment then returns overwrites the bytes in place. On this Docmost the attachment then returns
**500 for every URL** (clean, `?v=`, any filename) → broken image. Therefore **500 for every URL** (clean, `?v=`, any filename) → broken image. Therefore
`replace_image` must upload a **new** attachment and repoint the nodes; the new `replaceImage` must upload a **new** attachment and repoint the nodes; the new
id yields a new URL that both renders and busts the browser cache. The old id yields a new URL that both renders and busts the browser cache. The old
attachment is left as an unreferenced orphan: Docmost exposes **no HTTP API to attachment is left as an unreferenced orphan: Docmost exposes **no HTTP API to
delete a single content attachment** (verified against the attachment delete a single content attachment** (verified against the attachment
@@ -80,9 +80,9 @@ broken/empty figure.
from `?v=`. Image `src` is kept clean (`/api/files/<id>/<file>`); cache-busting from `?v=`. Image `src` is kept clean (`/api/files/<id>/<file>`); cache-busting
on replace is achieved by the new attachment id. on replace is achieved by the new attachment id.
3. **REST snapshot lag.** `get_page_json` reads the debounced DB snapshot, so a 3. **REST snapshot lag.** `getPageJson` reads the debounced DB snapshot, so a
write made moments earlier may not be visible yet. Wait (~16 s) before reading write made moments earlier may not be visible yet. Wait (~16 s) before reading
back, and never feed a possibly-stale snapshot straight into `update_page_json`. back, and never feed a possibly-stale snapshot straight into `updatePageJson`.
4. **Callout type narrowing (minor, open).** A `:::warning` callout is imported as 4. **Callout type narrowing (minor, open).** A `:::warning` callout is imported as
`type: "info"` — the markdown→callout conversion does not carry non-`info` `type: "info"` — the markdown→callout conversion does not carry non-`info`
+63
View File
@@ -0,0 +1,63 @@
{
"$comment": "Semantic palettes for drawioFromGraph (issue #425). DATA, not code: node `kind` -> fill/stroke slot, edge `kind` -> line-style props, per preset. The `default` node palette is the issue's base table; `dark` keeps the same hues on a dark canvas with lighter strokes/font; `colorblind-safe` maps every slot onto the Okabe-Ito qualitative palette (8 colours proven distinguishable for all common colour-vision deficiencies) so no two adjacent kinds collide. `fontColor`/`fillColor`/`strokeColor` are exact draw.io values. `edgeDefault` is the fallback line style; `group` is the (always-transparent) container stroke per preset.",
"presets": {
"default": {
"canvasDark": false,
"nodes": {
"service": { "fillColor": "#dae8fc", "strokeColor": "#6c8ebf", "fontColor": "#000000" },
"db": { "fillColor": "#d5e8d4", "strokeColor": "#82b366", "fontColor": "#000000" },
"queue": { "fillColor": "#fff2cc", "strokeColor": "#d6b656", "fontColor": "#000000" },
"gateway": { "fillColor": "#ffe6cc", "strokeColor": "#d79b00", "fontColor": "#000000" },
"error": { "fillColor": "#f8cecc", "strokeColor": "#b85450", "fontColor": "#000000" },
"external": { "fillColor": "#f5f5f5", "strokeColor": "#666666", "fontColor": "#333333" },
"security": { "fillColor": "#e1d5e7", "strokeColor": "#9673a6", "fontColor": "#000000" }
},
"edges": {
"sync": { "props": "" },
"async": { "props": "dashed=1;" },
"error": { "props": "dashed=1;strokeColor=#DD344C;" }
},
"edgeDefault": { "strokeColor": "#333333", "fontColor": "#333333" },
"group": { "strokeColor": "#666666", "fontColor": "#333333" }
},
"dark": {
"canvasDark": true,
"nodes": {
"service": { "fillColor": "#1a2a44", "strokeColor": "#7ea6e0", "fontColor": "#dae8fc" },
"db": { "fillColor": "#1f331e", "strokeColor": "#97d077", "fontColor": "#d5e8d4" },
"queue": { "fillColor": "#3a3218", "strokeColor": "#e5c15a", "fontColor": "#fff2cc" },
"gateway": { "fillColor": "#3a2812", "strokeColor": "#ffb570", "fontColor": "#ffe6cc" },
"error": { "fillColor": "#3a1c1b", "strokeColor": "#e08e8b", "fontColor": "#f8cecc" },
"external": { "fillColor": "#2b2b2b", "strokeColor": "#999999", "fontColor": "#e0e0e0" },
"security": { "fillColor": "#2c2338", "strokeColor": "#b39ddb", "fontColor": "#e1d5e7" }
},
"edges": {
"sync": { "props": "" },
"async": { "props": "dashed=1;" },
"error": { "props": "dashed=1;strokeColor=#ff6b6b;" }
},
"edgeDefault": { "strokeColor": "#cccccc", "fontColor": "#e0e0e0" },
"group": { "strokeColor": "#aaaaaa", "fontColor": "#e0e0e0" }
},
"colorblind-safe": {
"canvasDark": false,
"okabeIto": ["#000000", "#E69F00", "#56B4E9", "#009E73", "#F0E442", "#0072B2", "#D55E00", "#CC79A7"],
"nodes": {
"service": { "fillColor": "#D6E9F5", "strokeColor": "#0072B2", "fontColor": "#000000" },
"db": { "fillColor": "#D6EFE4", "strokeColor": "#009E73", "fontColor": "#000000" },
"queue": { "fillColor": "#FCF8CC", "strokeColor": "#F0E442", "fontColor": "#000000" },
"gateway": { "fillColor": "#FBEBD0", "strokeColor": "#E69F00", "fontColor": "#000000" },
"error": { "fillColor": "#F7DDCC", "strokeColor": "#D55E00", "fontColor": "#000000" },
"external": { "fillColor": "#EDEDED", "strokeColor": "#000000", "fontColor": "#000000" },
"security": { "fillColor": "#F3DEEB", "strokeColor": "#CC79A7", "fontColor": "#000000" }
},
"edges": {
"sync": { "props": "" },
"async": { "props": "dashed=1;" },
"error": { "props": "dashed=1;strokeColor=#D55E00;" }
},
"edgeDefault": { "strokeColor": "#000000", "fontColor": "#000000" },
"group": { "strokeColor": "#000000", "fontColor": "#000000" }
}
}
}
+86 -4559
View File
File diff suppressed because it is too large Load Diff
+704
View File
@@ -0,0 +1,704 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import { assertFullUuid } from "./errors.js";
import {
filterWorkspace,
filterSpace,
filterPage,
filterComment,
filterSearchResult,
} from "../lib/filters.js";
import { convertProseMirrorToMarkdown } from "../lib/markdown-converter.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import {
applyAnchorInDoc,
countAnchorMatches,
getAnchoredText,
resolveAnchorSelection,
normalizeForMatch,
} from "../lib/comment-anchor.js";
import { closestBlockHint } from "../lib/text-normalize.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Public method surface of CommentsMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements ICommentsMixin` fails to compile on drift.
export interface ICommentsMixin {
listComments(pageId: string, includeResolved?: boolean): any;
getComment(commentId: string): any;
createComment(pageId: string, content: string, type?: "page" | "inline", selection?: string, parentCommentId?: string, suggestedText?: string): any;
updateComment(commentId: string, content: string): any;
deleteComment(commentId: string): any;
resolveComment(commentId: string, resolved: boolean): any;
checkNewComments(spaceId: string, since: string, parentPageId?: string): any;
}
export function CommentsMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & ICommentsMixin> & TBase {
abstract class CommentsMixin extends Base implements ICommentsMixin {
// --- Comment methods (ported from upstream PR #3 by Max Nikitin) ---
/**
* Normalize a comment's `content` into a ProseMirror doc object before
* markdown conversion. createComment/updateComment send content as a
* JSON.stringify(...) STRING, and the server stores it as-is, so on read it
* comes back as a string. convertProseMirrorToMarkdown returns "" for a
* string, so parse it first (guarded fall back to the raw value on any
* parse failure so a non-JSON legacy value is still handled gracefully).
*/
protected parseCommentContent(content: any): any {
if (typeof content !== "string") return content;
try {
return JSON.parse(content);
} catch {
return content;
}
}
/**
* List comments on a page (cursor-paginated), content as markdown.
*
* DEFAULT (`includeResolved = false`) hides RESOLVED THREADS WHOLESALE so the
* agent sees only active discussions: a top-level comment with `resolvedAt`
* set AND every reply under it (a reply of a closed thread is part of the
* closed thread) are dropped from `items`. `resolvedThreadsHidden` reports how
* many resolved top-level threads were hidden so the agent can re-query with
* `includeResolved: true` to see everything. Active threads always stay.
*
* Returns `{ items, resolvedThreadsHidden }` (NOT a bare array) callers that
* need the full feed (lossless export, transformPage, checkNewComments) pass
* `includeResolved: true` and read `.items`.
*/
async listComments(pageId: string, includeResolved = false) {
await this.ensureAuthenticated();
let allComments: any[] = [];
let cursor: string | null = null;
// Hard ceiling + immovable-cursor guard (mirrors paginateAll): if /comments
// ever stops advancing the cursor (the exact #442 drift scenario) this loop
// would otherwise spin forever accumulating duplicates.
const MAX_PAGES = 50;
let truncated = false;
for (let page = 0; page < MAX_PAGES; page++) {
const payload: Record<string, any> = { pageId, limit: 100 };
if (cursor) payload.cursor = cursor;
const response = await this.client.post("/comments", payload);
const data = response.data.data || response.data;
const items = data.items || [];
allComments = allComments.concat(items);
// Advance strictly via the server-issued cursor. A missing nextCursor or a
// cursor identical to the one we just sent means the end (or a server that
// ignores our pagination param) — stop instead of re-fetching page one.
const next: string | null = data.meta?.nextCursor || null;
if (!next || next === cursor) break;
cursor = next;
// Reaching the ceiling with a still-advancing cursor means truncation.
if (page === MAX_PAGES - 1) truncated = true;
}
if (truncated) {
console.warn(
`listComments: comments for "${pageId}" truncated at the ${MAX_PAGES}-page cap; more pages exist on the server`,
);
}
const mapped = allComments.map((comment: any) => {
const markdown = comment.content
? convertProseMirrorToMarkdown(
this.parseCommentContent(comment.content),
)
: "";
return filterComment(comment, markdown);
});
if (includeResolved) {
return { items: mapped, resolvedThreadsHidden: 0 };
}
// Ids of RESOLVED top-level threads (a top-level comment has no
// parentCommentId). A whole thread is hidden when its root is resolved.
const resolvedRootIds = new Set(
mapped
.filter((c) => !c.parentCommentId && c.resolvedAt != null)
.map((c) => c.id),
);
const items = mapped.filter((c) => {
// Hide the resolved root itself and every reply anchored to it. A reply's
// own resolvedAt is irrelevant — its membership follows the parent thread.
// ASSUMPTION: Docmost's comment model is FLAT — a reply's parentCommentId
// always points at the thread ROOT (no reply-of-reply nesting), so a single
// level of parent lookup covers a whole thread. If nested replies are ever
// introduced, a deep reply of a resolved thread would need a root-walk here.
if (!c.parentCommentId) return !resolvedRootIds.has(c.id);
return !resolvedRootIds.has(c.parentCommentId);
});
return { items, resolvedThreadsHidden: resolvedRootIds.size };
}
async getComment(commentId: string) {
// Fail fast (#436): reject a truncated id before any network call.
assertFullUuid("get_comment", "commentId", commentId);
await this.ensureAuthenticated();
const response = await this.client.post("/comments/info", { commentId });
const comment = response.data.data || response.data;
const markdown = comment.content
? convertProseMirrorToMarkdown(this.parseCommentContent(comment.content))
: "";
return {
data: filterComment(comment, markdown),
success: true,
};
}
/** Plain text of each TOP-LEVEL block of `doc`, for anchor-failure hints. */
protected topLevelBlockTexts(doc: any): string[] {
const content = doc && Array.isArray(doc.content) ? doc.content : [];
return content
.map((b: any) => blockPlainText(b))
.filter((t: string) => t.length > 0);
}
/**
* True when per-block anchoring failed but the (normalized) selection DOES
* appear in the blocks' joined plain text i.e. it straddles a block
* boundary. Blocks are joined with a newline (collapsed to one space by
* normalizeForMatch) so a selection whose parts are separated by a paragraph
* break still matches. Callers only reach here after single-block anchoring
* (incl. the markdown-strip fallback) has already failed.
*/
protected selectionSpansMultipleBlocks(
blockTexts: string[],
selection: string,
): boolean {
const normSel = normalizeForMatch(selection).norm.trim();
if (normSel.length === 0) return false;
const joined = normalizeForMatch(blockTexts.join("\n")).norm;
return joined.indexOf(normSel) !== -1;
}
/**
* Build the actionable error for a createComment anchor MISS, porting
* editPageText's self-correction affordances: an explicit "spans multiple
* blocks" message when the selection straddles a block boundary, otherwise a
* "closest block text" hint quoting the block that holds the selection's
* longest token. `live` switches the wording between the pre-check (reading the
* persisted page) and the post-create live-anchor failure (which rolls back).
*/
protected anchorNotFoundError(
doc: any,
selection: string,
live: boolean,
): Error {
const blockTexts = this.topLevelBlockTexts(doc);
const rolled = live ? " The comment was rolled back." : "";
if (this.selectionSpansMultipleBlocks(blockTexts, selection)) {
return new Error(
"createComment: the selection spans multiple blocks; anchor on a " +
"contiguous fragment within a SINGLE paragraph/block (<=250 chars)." +
rolled,
);
}
const where = live ? "in the live document" : "in the page";
return new Error(
`createComment: could not find the selection text ${where} to anchor ` +
"the comment. Provide the EXACT contiguous text from a single " +
"paragraph/block (<=250 chars)." +
closestBlockHint(blockTexts, selection) +
rolled,
);
}
/**
* Create an inline comment anchored to its `selection` text, or a reply.
*
* Top-level comments (no `parentCommentId`) are ALWAYS inline and MUST carry a
* `selection`: the `type` argument is kept for interface compatibility but the
* effective type is coerced to "inline". The selection has to anchor in the
* document; if it cannot, the comment is rolled back and an error is thrown so
* the caller is forced to supply a proper inline selection rather than leaving
* an orphan, unanchored comment behind. Replies (parentCommentId set) inherit
* their parent's anchor: they take NO selection and are not anchored.
*/
async createComment(
pageId: string,
content: string,
type: "page" | "inline" = "page",
selection?: string,
parentCommentId?: string,
suggestedText?: string,
) {
// Fail fast (#436): a provided parent id must be a full UUID before any
// network call. Validate only when truthy — a falsy parentCommentId means
// "top-level comment" (mirrors the isReply computation below), not a reply.
if (parentCommentId) {
assertFullUuid("createComment", "parentCommentId", parentCommentId);
}
await this.ensureAuthenticated();
const isReply = !!parentCommentId;
const hasSuggestion =
suggestedText !== undefined && suggestedText !== null;
// Defense in depth mirroring the server DTO/service: a suggested edit rewrites
// the exact anchored text, so it is only meaningful on a top-level inline
// comment that carries a selection.
if (hasSuggestion) {
if (isReply) {
throw new Error(
"createComment: a suggested edit (suggestedText) cannot be attached to a reply; it applies only to a top-level inline comment.",
);
}
if (!selection || !selection.trim()) {
throw new Error(
"createComment: a suggested edit (suggestedText) requires a 'selection' to anchor and rewrite.",
);
}
}
// Only top-level comments are inline-anchored, so they are stored as
// "inline". Replies carry no inline selection, so they keep the historical
// general ("page") type — both backward-compatible and semantically correct.
// The `type` argument is kept for interface compatibility; createComment
// normalizes the effective type internally, so callers may pass "inline".
const effectiveType: "page" | "inline" = isReply ? "page" : "inline";
if (!isReply && (!selection || !selection.trim())) {
throw new Error(
"createComment: an inline 'selection' (exact text to anchor on) is required for a top-level comment",
);
}
// For a SUGGESTION, the value we store as the comment's `selection` must be
// the RAW document substring the mark lands on (typographic quotes/dashes,
// nbsp, collapsed whitespace), NOT the agent's ASCII input. The anchor is
// placed via normalization, so when the doc was auto-converted to
// typographic the raw substring differs from the agent input; apply-time
// compares the stored selection to the marked doc text STRICTLY, so storing
// the raw substring is what makes "Apply" succeed instead of a spurious 409.
// Captured in the pre-check below (which already reads the page) and used as
// payload.selection. Ordinary comments keep sending the raw agent selection.
let anchoredSelection: string | null = null;
// Set when the anchor matched only after stripping markdown from the
// selection (the strip fallback); surfaced as a soft warning like
// editPageText does, so a stale-markdown selection is flagged.
let anchorNormalized = false;
// For a top-level comment, fail BEFORE creating anything when the selection
// is not present in the persisted document — this avoids leaving an orphan
// comment + notification behind. A read failure (network) is non-fatal: the
// live anchor step below still enforces the anchoring invariant.
if (!isReply && selection) {
try {
const page = await this.getPageJson(pageId);
if (hasSuggestion) {
// A suggestion's anchor MUST be unambiguous: applying it rewrites the
// exact anchored text, and ordinary anchoring silently takes the first
// occurrence, so 0 matches -> not found and >=2 -> ambiguous, both
// rejected BEFORE creating the comment.
const matches = countAnchorMatches(page.content, selection);
if (matches === 0) {
throw this.anchorNotFoundError(page.content, selection, false);
}
if (matches >= 2) {
throw new Error(
`createComment: the suggestion's selection is ambiguous — it occurs ${matches} times in the page. ` +
"A suggested edit must anchor to a UNIQUE location; expand the selection with surrounding context " +
"(still <=250 chars) so it appears exactly once.",
);
}
// Exactly one match: capture the RAW anchored substring to store as the
// comment selection (so apply-time equality holds). If this returns
// null despite countAnchorMatches===1 (shouldn't happen), fall back to
// the raw agent selection below rather than crash.
anchoredSelection = getAnchoredText(page.content, selection);
anchorNormalized = resolveAnchorSelection(
page.content,
selection,
).normalized;
} else {
const resolved = resolveAnchorSelection(page.content, selection);
if (!resolved.found) {
throw this.anchorNotFoundError(page.content, selection, false);
}
anchorNormalized = resolved.normalized;
}
} catch (e) {
// Rethrow our own "not found"/"ambiguous"/"spans multiple blocks" errors;
// swallow read/network errors so the live anchor step can still try (and
// enforce) anchoring.
if (
e instanceof Error &&
(e.message.startsWith("createComment: could not find the selection") ||
e.message.startsWith(
"createComment: the selection spans multiple blocks",
) ||
e.message.startsWith(
"createComment: the suggestion's selection is ambiguous",
))
) {
throw e;
}
if (process.env.DEBUG) {
console.error(
"Pre-check getPageJson failed; deferring to live anchor step:",
e,
);
}
}
}
// Convert through the full Docmost schema. Deliberately the NON-canonicalizing
// variant: a comment body may carry a footnote definition with no matching
// reference, and canonicalization would drop it (data loss). See
// markdownToProseMirror vs markdownToProseMirrorCanonical.
const jsonContent = await markdownToProseMirror(content);
const payload: Record<string, any> = {
pageId,
content: JSON.stringify(jsonContent),
type: effectiveType,
};
// For a suggestion, store the RAW anchored substring (anchoredSelection) so
// the stored selection === the text under the mark === apply-time
// expectedText. Ordinary comments (and the null fallback) keep the raw
// agent selection — their selection is only display/anchor and never used
// by apply, so their behavior is unchanged.
if (!isReply && selection)
payload.selection = anchoredSelection ?? selection;
if (parentCommentId) payload.parentCommentId = parentCommentId;
// Only a top-level inline comment (with a selection) may carry a suggestion.
if (!isReply && selection && hasSuggestion) {
payload.suggestedText = suggestedText;
}
const response = await this.client.post("/comments/create", payload);
const comment = response.data.data || response.data;
const markdown = comment.content
? convertProseMirrorToMarkdown(this.parseCommentContent(comment.content))
: content;
const result: any = {
data: filterComment(comment, markdown),
success: true,
};
// Replies inherit the parent's anchor: no selection, no anchoring.
if (isReply) {
return result;
}
// Anchor the comment in the document. The /comments/create API records the
// comment + its `selection` text, but it does NOT insert the comment MARK
// into the page content, so without this the inline comment has no
// highlight/anchor and is not clickable. If anchoring fails the comment is
// rolled back (deleted) and an error is thrown — never an orphan comment.
const newCommentId: string = comment.id;
// Guard: a create response without an id would mean writing a comment mark
// with commentId: undefined and a later delete of a falsy id. We have no id
// to roll back here (nothing was created with an id), so just fail loudly.
if (!newCommentId) {
throw new Error(
"createComment: the server returned no comment id, so the comment could not be anchored",
);
}
let anchored = false;
// Set inside the transform when a suggestion's live anchor is ambiguous
// (>=2 occurrences), so the rollback path can surface the right error.
let ambiguousInLiveDoc = false;
// Captured inside the transform on a not-found abort, so the rollback path
// can surface the closest-block / spans-multiple-blocks hint built from the
// LIVE document (the pre-check page is not in scope there).
let liveNotFoundError: Error | null = null;
try {
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260). The
// /comments/create REST call above keeps the agent-supplied id.
const pageUuid = await this.resolvePageId(pageId);
// Route through the mutatePage seam (not the free function) so this
// wrapper's uniqueness gate + rollback can be unit-tested without a live
// Hocuspocus collab socket.
const mutation = await this.mutatePage(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
const doc =
liveDoc && liveDoc.type === "doc"
? liveDoc
: { type: "doc", content: [] };
if (hasSuggestion) {
// Authoritative uniqueness check against the LIVE document: a
// suggestion must anchor to EXACTLY ONE occurrence, otherwise
// "Apply" would rewrite the wrong/ambiguous text. If the live doc
// no longer has exactly one occurrence (it changed since the
// pre-check), abort so the just-created comment is rolled back
// rather than mis-anchored to the first occurrence.
const liveCount = countAnchorMatches(doc, selection as string);
if (liveCount !== 1) {
ambiguousInLiveDoc = liveCount >= 2;
if (liveCount === 0) {
liveNotFoundError = this.anchorNotFoundError(
doc,
selection as string,
true,
);
}
return null;
}
}
if (applyAnchorInDoc(doc, selection as string, newCommentId)) {
anchored = true;
return doc;
}
// Selection text not found in the LIVE document: abort the write. The
// rollback + throw below turns this into a hard error.
liveNotFoundError = this.anchorNotFoundError(
doc,
selection as string,
true,
);
return null;
},
);
result.verify = mutation.verify;
} catch (e) {
// The comment record already exists; roll it back so we never leave an
// orphan, then rethrow the original anchoring error.
await this.safeDeleteComment(newCommentId);
throw e;
}
if (!anchored) {
// Mutation aborted because the selection was not found (or, for a
// suggestion, was ambiguous) in the live document. Roll back the comment
// and surface a hard error.
await this.safeDeleteComment(newCommentId);
if (ambiguousInLiveDoc) {
throw new Error(
"createComment: the suggestion's selection is ambiguous in the live document (multiple occurrences); the comment was rolled back. Expand the selection with surrounding context so it is unique.",
);
}
throw (
liveNotFoundError ??
new Error(
"createComment: failed to anchor the comment (selection not found in the live document); the comment was rolled back",
)
);
}
// Soft warning (like editPageText): the selection only matched after
// stripping markdown, so the caller likely quoted a styled fragment.
if (anchorNormalized) {
result.warning =
"The selection matched only after stripping markdown syntax; the comment " +
"was anchored on the document's plain text. Copy the selection verbatim " +
"from getPage / searchInPage output to avoid this.";
}
result.anchored = true;
return result;
}
/**
* Best-effort rollback of a just-created comment. Swallows any delete failure
* (logging under DEBUG) so a failed cleanup never masks the original error.
*/
protected async safeDeleteComment(commentId: string): Promise<void> {
// Defense in depth: never call the delete API with a falsy id — there is
// nothing to roll back, and deleteComment(undefined) would hit a bad route.
if (!commentId) return;
try {
await this.deleteComment(commentId);
} catch (delErr) {
if (process.env.DEBUG) {
console.error(
"Failed to roll back comment after anchoring error:",
delErr,
);
}
}
}
async updateComment(commentId: string, content: string) {
// Fail fast (#436): reject a truncated id before any network call.
assertFullUuid("updateComment", "commentId", commentId);
await this.ensureAuthenticated();
// NON-canonicalizing on purpose (comment body — see createComment).
const jsonContent = await markdownToProseMirror(content);
await this.client.post("/comments/update", {
commentId,
content: JSON.stringify(jsonContent),
});
return {
success: true,
commentId,
message: "Comment updated successfully.",
};
}
async deleteComment(commentId: string) {
// Fail fast (#436): reject a truncated id before any network call.
assertFullUuid("deleteComment", "commentId", commentId);
await this.ensureAuthenticated();
return this.client
.post("/comments/delete", { commentId })
.then((res) => res.data);
}
/**
* Resolve or reopen a top-level comment thread (reversible `resolved`
* toggles the state). Only top-level comments can be resolved; the server
* rejects resolving a reply. Hits POST /comments/resolve.
*/
async resolveComment(commentId: string, resolved: boolean) {
// Fail fast (#436): reject a truncated id before any network call.
assertFullUuid("resolveComment", "commentId", commentId);
await this.ensureAuthenticated();
const response = await this.client.post("/comments/resolve", {
commentId,
resolved,
});
const comment = response.data?.data ?? response.data;
return {
success: true,
commentId,
resolved,
comment,
};
}
/**
* Check for new comments across pages in a space (optionally scoped to a
* subtree): pages updated after `since` are scanned and their comments
* filtered by createdAt > since.
*/
async checkNewComments(
spaceId: string,
since: string,
parentPageId?: string,
) {
await this.ensureAuthenticated();
const sinceDate = new Date(since);
// Reject an unparseable `since`: comparing against an Invalid Date silently
// yields zero new comments (every `>` against NaN is false), which would
// mask a malformed input as "nothing new" instead of erroring.
if (Number.isNaN(sinceDate.getTime())) {
throw new Error(
`checkNewComments: invalid "since" date "${since}"; expected an ISO-8601 timestamp`,
);
}
// 1. Enumerate the FULL set of pages in scope via the page tree (a complete
// page index), NOT the bounded "/pages/recent" feed which caps at ~5000
// recent items and silently misses comments on older pages.
//
// Subtree scope: when parentPageId is given, the scope is that page ITSELF
// plus every descendant. Otherwise the scope is the whole space (all roots
// and their descendants).
//
// NOTE: do NOT pre-filter by page.updatedAt — creating a comment does not
// bump it (verified on a live server), so such a filter silently misses
// comments on pages that were not otherwise edited. The complete tree walk
// already restricts the scope correctly, so no recent-feed allow-list is
// needed any more.
//
// The subtree scope (parentPageId given) already INCLUDES the root node
// itself: /pages/tree seeds getPageAndDescendants with id = parentPageId, so
// no separate getPageRaw fetch for the parent is needed.
const { pages: pagesInScope, truncated } = await this.enumerateSpacePages(
spaceId,
parentPageId,
);
// 2. Fetch comments for each page, keep ones created after since
const results: any[] = [];
for (const page of pagesInScope) {
try {
// Full feed (incl. resolved): a "new comments since" scan reports all
// recent activity; the active-only filter is scoped to listComments.
const comments = (await this.listComments(page.id, true)).items;
const newComments = comments.filter(
(c: any) => new Date(c.createdAt) > sinceDate,
);
if (newComments.length > 0) {
results.push({
pageId: page.id,
pageTitle: page.title,
comments: newComments,
});
}
} catch (e: any) {
// Skip pages with errors (e.g. deleted between calls)
}
}
const totalNewComments = results.reduce(
(sum, r) => sum + r.comments.length,
0,
);
// `truncated` is reported by enumerateSpacePages: it is true ONLY when the
// stdio fallback BFS hit its node cap. The primary /pages/tree path is
// uncapped, so a space with legitimately many pages is not falsely flagged.
return {
since,
scope: parentPageId ? `subtree of ${parentPageId}` : `space ${spaceId}`,
checkedPages: pagesInScope.length,
pagesWithNewComments: results.length,
totalNewComments,
truncated,
comments: results,
};
}
// --- Image upload / embedding ---
/** Map a Content-Type string to a supported MIME type, or null if unsupported. */
}
return CommentsMixin;
}
+690
View File
@@ -0,0 +1,690 @@
// Shared client context + core seams (issue #450). The abstract base of the
// DocmostClient mixin chain: it owns ALL shared instance state (the axios
// client, apiUrl, auth tokens, the resolvePageId cache, the collab-token cache,
// the sandbox/metrics sinks) and the core HTTP/auth/pagination/write seams every
// domain module builds on. Domain modules are mixins layered on top; the final
// DocmostClient (client.ts) assembles them. Extracted VERBATIM from the original
// monolith — only field/seam visibility was widened from `private` to
// `protected` so sibling mixins can reach the shared state through `this`, and
// the cross-module methods that live in other mixins are declared `abstract`
// here so `this.<method>` type-checks. No behaviour changed.
import axios, { AxiosInstance } from "axios";
import FormData from "form-data";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import { acquireCollabSession } from "../lib/collab-session.js";
import { withPageLock, isUuid } from "../lib/page-lock.js";
import { getCollabToken, performLogin } from "../lib/auth-utils.js";
import { formatDocmostAxiosError } from "./errors.js";
// A generic mixin base constructor (issue #450). Each domain mixin is a factory
// `<T extends GConstructor<DocmostClientContext>>(Base: T) => class extends Base`
// so the mixins compose into one prototype chain sharing this context.
export type GConstructor<T = {}> = abstract new (...args: any[]) => T;
/**
* Configuration for a DocmostClient / MCP server instance. A discriminated
* union: either service-account credentials (email/password the client calls
* performLogin, powering the external /mcp HTTP endpoint and the stdio CLI) OR
* a token getter (getToken the client uses the returned BARE access JWT as
* the Bearer and never calls performLogin; used for the internal per-user path).
*
* Both branches may ALSO carry an optional `getCollabToken` provider. When set,
* content mutations (which go over the collaboration websocket) use the token it
* returns INSTEAD of calling `POST /auth/collab-token`. The internal per-user
* agent path uses this to hand the client a provenance collab token (signed
* `actor:'agent'`+`aiChatId`), so agent content edits are attributed without a
* spoofable client-side field. When absent the client keeps the original
* `/auth/collab-token` path (service-account/stdio unchanged).
*
* Housed here (not in index.ts) so client.ts has no type dependency on index.ts;
* index.ts re-exports it for the package's public surface.
*/
// Sink the stash tool writes blobs into. The host app binds this to its in-RAM
// SandboxStore and composes the public `uri` (the package never sees the store
// or any env). `put` returns the anonymous read URL plus integrity metadata.
export type SandboxPut = (
buf: Buffer,
mime: string,
) => { uri: string; sha256: string; size: number };
export type DocmostMcpConfig = { apiUrl: string } & (
| { email: string; password: string }
| { getToken: () => Promise<string> } // returns a BARE JWT; the client adds "Bearer "
) & {
// Optional collab-token provider (returns a ready collab JWT). Common to
// both branches; see the type doc above.
getCollabToken?: () => Promise<string>;
// Optional blob sandbox sink. Present only where the stash tool is wired;
// when absent, stashPage throws a clear "not configured" error. The
// optional `has`/`evict` probes let stashPage keep its mirror counts honest
// under the store's FIFO eviction (see stashPage); older sinks omit them.
sandbox?: {
put: SandboxPut;
has?: (uri: string) => boolean;
evict?: (uri: string) => void;
};
// Dependency-neutral metrics sink. When present, the client emits generic
// (name, value, labels) samples; the HOST maps those names onto its own
// metrics registry (the package never depends on prom-client or the server).
// Absent in standalone/stdio mode → the client is a complete no-op here.
onMetric?: (
name: string,
value: number,
labels?: Record<string, string>,
) => void;
};
/**
* Collab-token cache TTL in milliseconds (issue #435). Read fresh from the
* environment on every mint like collab-session.ts readConfig so tests and a
* live rollback can change it without reloading the module.
*
* Why a cache at all: the live CollabSession registry (#400/#431) keys sessions
* on (wsUrl, pageId, collabToken) for identity isolation (invariant 4). But BOTH
* collab-token sources mint a FRESH token per mutation the in-app provider
* re-signs a JWT whose iat/exp (seconds) changes every second, and the external
* MCP POSTs /auth/collab-token each call so the token in the key changed on
* every op and the session was almost never reused (connect-storms, 25s
* timeouts, zombie sessions). Caching the token per-client keeps the key stable
* across a burst of mutations so ONE session is reused.
*
* Default 5 min: well under the 24h collab-token lifetime AND <= the collab
* session max-age (10 min, MCP_COLLAB_SESSION_MAX_AGE_MS), so the
* permission-staleness window is not widened beyond what #431 already accepted.
* The rollback knob is an EXPLICIT 0 (or a negative number): that DISABLES the
* cache an exact fetch-per-call legacy path, mirroring how idleMs<=0 disables
* the session cache. Unset OR unparseable (e.g. a typo like "5min", "abc") falls
* back to the 5-min default with the cache ON parseInt yields NaN, which is
* treated as "not configured", not as "disabled". So to turn the cache off you
* must set the value to exactly 0, not to garbage.
*/
function readCollabTokenTtlMs(): number {
const raw = parseInt(process.env.MCP_COLLAB_TOKEN_TTL_MS ?? "", 10);
return Number.isFinite(raw) ? Math.max(0, raw) : 5 * 60 * 1000;
}
export abstract class DocmostClientContext {
protected client: AxiosInstance;
protected token: string | null = null;
protected apiUrl: string;
// email/password are only set on the service-account (credentials) variant;
// null on the getToken variant (where there are no credentials to log in with).
protected email: string | null = null;
protected password: string | null = null;
// Per-user token provider. When set, login() calls it to obtain a BARE access
// JWT instead of performLogin, and the 401/403 re-auth path re-calls it.
protected getTokenFn: (() => Promise<string>) | null = null;
// Optional collab-token provider. When set, getCollabTokenWithReauth() returns
// its token instead of calling POST /auth/collab-token; on a 401/403 it is
// re-invoked once. Used by the internal agent to carry signed provenance.
protected getCollabTokenFn: (() => Promise<string>) | null = null;
// Optional blob-sandbox sink for the stash tool. Null when not configured.
protected sandboxPut: SandboxPut | null = null;
// Optional probes paired with the sink. `has` lets stashPage detect a blob
// FIFO-evicted by a LATER put in the same stash; `evict` lets it free this
// op's image blobs if the final doc put throws. Null when the sink omits them.
protected sandboxHas: ((uri: string) => boolean) | null = null;
protected sandboxEvict: ((uri: string) => void) | null = null;
// Optional dependency-neutral metrics sink (see DocmostMcpConfig.onMetric).
// Null on the legacy positional form and whenever the host omits it → no-op.
protected onMetricFn:
| ((name: string, value: number, labels?: Record<string, string>) => void)
| null = null;
// In-flight login dedup: when the token expires, the 401 interceptor,
// ensureAuthenticated, getCollabTokenWithReauth and the two multipart retries
// can all call login() at once. Memoizing a single promise collapses that
// thundering herd into ONE /auth/login request that everyone awaits.
protected loginPromise: Promise<void> | null = null;
// Canonical-UUID cache for resolvePageId: maps an agent-supplied slugId to the
// page's canonical UUID, so repeated collab edits on the same page do not
// re-fetch /pages/info. A UUID input short-circuits before this cache (see
// resolvePageId), so only slugId->uuid entries are stored/read here.
protected pageIdCache = new Map<string, string>();
// Collab-token cache (issue #435): the last minted collab token plus the
// wall-clock time it was minted, so a burst of content mutations reuses ONE
// token and therefore ONE live CollabSession (whose registry key includes the
// token — #400 invariant 4). Per-instance: a DocmostClient is built per
// user/per chat request, so a cached token can never leak across identities.
// Reset whenever the client's identity changes (login() / this.token cleared);
// bypassed on a forced refresh (the 401/403 reauth path). null = no token yet.
protected collabTokenCache: { token: string; mintedAt: number } | null = null;
// Two construction forms:
// - new DocmostClient(config) // discriminated union (current)
// - new DocmostClient(baseURL, email, password) // legacy positional creds
// The positional form is retained so existing callers/tests keep working; it
// is exactly equivalent to the credentials branch of the object form.
constructor(config: DocmostMcpConfig);
constructor(baseURL: string, email: string, password: string);
constructor(
configOrBaseURL: DocmostMcpConfig | string,
email?: string,
password?: string,
) {
// Normalize the legacy positional form into the object union.
const config: DocmostMcpConfig =
typeof configOrBaseURL === "string"
? { apiUrl: configOrBaseURL, email: email!, password: password! }
: configOrBaseURL;
this.apiUrl = config.apiUrl;
if ("getToken" in config) {
// Token variant: carry the user's JWT via getToken; no credentials, so
// login() must never call performLogin (there is nothing to log in with).
this.getTokenFn = config.getToken;
} else {
// Service-account variant: behaves exactly as before (performLogin).
this.email = config.email;
this.password = config.password;
}
// Optional, available to both variants. When present, content mutations get
// their collab token from here instead of POST /auth/collab-token.
if (config.getCollabToken) {
this.getCollabTokenFn = config.getCollabToken;
}
if (config.sandbox) {
this.sandboxPut = config.sandbox.put;
this.sandboxHas = config.sandbox.has ?? null;
this.sandboxEvict = config.sandbox.evict ?? null;
}
// Legacy positional form carries no onMetric → null (complete no-op).
this.onMetricFn = config.onMetric ?? null;
this.client = axios.create({
baseURL: this.apiUrl,
// Default request timeout so a hung connection cannot wedge a per-page
// lock or block the server indefinitely. Multipart uploads override this
// with a longer per-request timeout.
timeout: 30000,
headers: {
"Content-Type": "application/json",
},
});
// Re-authenticate transparently on a 401/403 once: the JWT authToken can
// expire while the server is long-running, after which every cached-token
// request would otherwise fail until a manual restart. On such a response,
// clear the stale token, perform a fresh login, and replay the original
// request exactly once (guarded by config._retry to avoid infinite loops;
// the login request itself is never retried).
this.client.interceptors.response.use(
(response) => response,
async (error) => {
const config = error.config;
const status = error.response?.status;
const isAuthError = status === 401 || status === 403;
const isLoginRequest =
typeof config?.url === "string" && config.url.includes("/auth/login");
if (config && isAuthError && !config._retry && !isLoginRequest) {
config._retry = true;
// Drop the stale token + Authorization header before re-login. Also
// clear the collab-token cache (#435): a new identity/login must not
// keep serving a collab token minted under the old one.
this.token = null;
this.collabTokenCache = null;
delete this.client.defaults.headers.common["Authorization"];
try {
await this.login();
} catch (loginError) {
// Re-login failed: surface the original error to the caller.
return Promise.reject(error);
}
// Re-issue the original request with the freshly minted Bearer token.
// Read it from the default header that login() just set, not from
// this.token, to avoid a theoretical "Bearer null" if this.token was
// cleared between login() resolving and this point.
config.headers = config.headers || {};
config.headers["Authorization"] =
this.client.defaults.headers.common["Authorization"];
return this.client.request(config);
}
return Promise.reject(error);
},
);
// Diagnostics interceptor (issue #437). Registered AFTER the re-login
// interceptor so a successful re-login retry (which resolves to a real
// response) is never seen here as an error; only a genuine failure reaches
// this rejection handler. It reformats error.message IN PLACE (see
// formatDocmostAxiosError — kept as a mutation, not a custom Error class, so
// the surrounding axios.isAxiosError / error.response?.status / config._retry
// checks keep working) and re-rejects the SAME error. The _docmostFormatted
// flag makes a re-processed retry-failure a no-op.
this.client.interceptors.response.use(
(response) => response,
(error) => {
formatDocmostAxiosError(error);
return Promise.reject(error);
},
);
}
// --- Cross-module seams (issue #450) -----------------------------------
// A method in one domain mixin sometimes calls a PROTECTED method owned by
// another mixin (e.g. nodes-write -> validateDocUrls in doc-validate). Those
// callees are `protected`, so they cannot be surfaced through the public
// per-mixin interfaces. Declaring them here on the shared base lets `this.<m>`
// type-check across modules. Each is a stub that is ALWAYS overridden by the
// owning mixin (layered above this base in the chain), so the body never runs;
// it throws only to make an impossible mis-wiring loud instead of silent.
// (The PUBLIC cross-module callees — getPage, getPageJson, listComments,
// deleteComment, listPageHistory — arrive via the mixins' public interfaces,
// so they are not restated here.)
protected enumerateSpacePages(
_spaceId: string,
_rootPageId?: string,
): Promise<{ pages: any[]; truncated: boolean }> {
throw new Error("enumerateSpacePages not wired (missing ReadMixin)");
}
protected validateDocUrls(_node: any, _depth?: number): void {
throw new Error("validateDocUrls not wired (missing DocValidateMixin)");
}
protected validateDocStructure(_node: any, _depth?: number): void {
throw new Error("validateDocStructure not wired (missing DocValidateMixin)");
}
protected assertValidNodeShape(_op: string, _node: any): void {
throw new Error("assertValidNodeShape not wired (missing DocValidateMixin)");
}
protected fetchInternalFile(
_src: string,
): Promise<{ buffer: Buffer; mime: string }> {
throw new Error("fetchInternalFile not wired (missing StashMixin)");
}
protected uploadAttachmentBuffer(
_pageId: string,
_buffer: Buffer,
_fileName: string,
_mime: string,
): Promise<{ id: string; fileName: string; fileSize: number }> {
throw new Error("uploadAttachmentBuffer not wired (missing MediaMixin)");
}
protected fetchAttachmentText(_src: string): Promise<string> {
throw new Error("fetchAttachmentText not wired (missing MediaMixin)");
}
// PUBLIC cross-module callees. Declared here too (as always-overridden stubs)
// so a mixin calling e.g. `this.getPageJson` type-checks against the base —
// the mixin's own public interface only covers its own methods. The real
// implementations live in ReadMixin / CommentsMixin / PagesMixin and shadow
// these on the prototype chain.
getPage(_pageId: string): Promise<any> {
throw new Error("getPage not wired (missing ReadMixin)");
}
getPageJson(_pageId: string): Promise<any> {
throw new Error("getPageJson not wired (missing ReadMixin)");
}
listComments(_pageId: string, _includeResolved?: boolean): Promise<any> {
throw new Error("listComments not wired (missing CommentsMixin)");
}
deleteComment(_commentId: string): Promise<any> {
throw new Error("deleteComment not wired (missing CommentsMixin)");
}
listPageHistory(_pageId: string, _cursor?: string): Promise<any> {
throw new Error("listPageHistory not wired (missing PagesMixin)");
}
/** Application base URL (API URL without the /api suffix). */
get appUrl(): string {
return this.apiUrl.replace(/\/api\/?$/, "");
}
async login() {
// Reuse an in-flight login if one is already running so concurrent callers
// share a single token fetch instead of each issuing their own.
if (!this.loginPromise) {
// Token variant: re-fetch a BARE JWT via getToken() (there are no
// credentials to log in with — on a 401/403 the interceptor below calls
// login() again, which re-invokes getToken()). Credentials variant:
// performLogin against /auth/login exactly as before.
const fetchToken = this.getTokenFn
? this.getTokenFn()
: performLogin(this.apiUrl, this.email!, this.password!);
this.loginPromise = fetchToken
.then((token) => {
// Guard against an empty/invalid token (e.g. a getToken provider that
// resolves to "" or null): without this an empty token would set a
// literal "Authorization: Bearer null"/"Bearer " header and every
// request would 401 with a confusing error. Fail loudly instead.
if (typeof token !== "string" || token.length === 0) {
throw new Error("getToken returned an empty token");
}
this.token = token;
// Identity (re)established: drop any collab token minted under a
// previous identity so the #435 cache can never outlive it.
this.collabTokenCache = null;
this.client.defaults.headers.common["Authorization"] =
`Bearer ${token}`;
})
.finally(() => {
this.loginPromise = null;
});
}
return this.loginPromise;
}
async ensureAuthenticated() {
if (!this.token) {
await this.login();
}
}
/**
* Fetch a collaboration token, transparently re-authenticating once on a
* 401/403. getCollabToken() uses bare axios internally, so it is NOT covered
* by this.client's response interceptor; this helper replicates that
* behaviour for collab-token requests: ensure a token, try once, and on an
* expired-token auth error perform a fresh login and retry exactly once.
*
* Collab-token cache (issue #435): both sources the getCollabToken provider
* (in-app agent) AND the REST /auth/collab-token endpoint (external MCP) mint
* a FRESH token per call, whose string therefore changes every op. Since the
* live CollabSession registry keys on the token string (#400/#431 invariant 4),
* that churned the key and defeated session reuse. So we cache the last minted
* token per-client for readCollabTokenTtlMs() and hand it back for a burst of
* mutations, keeping the session key stable. `forceRefresh` bypasses the cache
* (the 401/403 reauth retry uses it, so the retry cannot be handed the same
* stale token that just failed otherwise reauth would be a no-op). TTL 0
* disables the cache: exact fetch-per-call legacy behaviour.
*/
protected async getCollabTokenWithReauth(
forceRefresh = false,
): Promise<string> {
const ttl = readCollabTokenTtlMs();
// Serve the cached collab token while it is still fresh (identity isolation
// is preserved: the cache is a per-instance field on a client built per
// user/per chat request, and it is cleared on every identity change).
if (
!forceRefresh &&
ttl > 0 &&
this.collabTokenCache &&
Date.now() - this.collabTokenCache.mintedAt < ttl
) {
return this.collabTokenCache.token;
}
// Collab-token PROVIDER path: when a getCollabToken provider was supplied
// (the internal agent's provenance collab token), use it instead of the
// REST /auth/collab-token endpoint. Re-invoke it once on a 401/403 (e.g. the
// signed token expired between content mutations in a long agent turn).
if (this.getCollabTokenFn) {
try {
const token = await this.getCollabTokenFn();
if (typeof token !== "string" || token.length === 0) {
throw new Error("getCollabToken returned an empty token");
}
return this.rememberCollabToken(token, ttl);
} catch (e) {
// On an auth error retry EXACTLY once, forcing a refresh so the retry
// re-invokes the provider (bypassing the cache) for a genuinely fresh
// token. `!forceRefresh` bounds it to a single retry (no loop).
if (this.isCollabAuthError(e) && !forceRefresh) {
return this.getCollabTokenWithReauth(true);
}
throw e;
}
}
await this.ensureAuthenticated();
try {
const token = await getCollabToken(this.apiUrl, this.token!);
return this.rememberCollabToken(token, ttl);
} catch (e) {
// getCollabToken wraps the AxiosError in a plain Error but attaches the
// HTTP status as `.status`, so isCollabAuthError detects an auth failure
// via either the raw AxiosError shape OR the attached status.
if (this.isCollabAuthError(e) && !forceRefresh) {
// Fresh login (which clears this.token AND the collab-token cache), then
// retry exactly once with the cache bypassed via forceRefresh.
await this.login();
return this.getCollabTokenWithReauth(true);
}
throw e;
}
}
/**
* Store a freshly minted collab token in the per-client cache (issue #435) and
* return it unchanged. No-op write when the cache is disabled (ttl<=0) or the
* token is empty, so a disabled cache is exact fetch-per-call legacy behaviour
* and a bad token is never cached.
*/
protected rememberCollabToken(token: string, ttl: number): string {
if (ttl > 0 && typeof token === "string" && token.length > 0) {
this.collabTokenCache = { token, mintedAt: Date.now() };
}
return token;
}
/**
* True when an error carries a 401/403 either as a raw AxiosError
* (`error.response.status`) or as the plain-Error `.status` that
* lib/auth-utils.getCollabToken attaches after wrapping the AxiosError.
*/
protected isCollabAuthError(e: unknown): boolean {
const axiosStatus = axios.isAxiosError(e) ? e.response?.status : undefined;
const attachedStatus = (e as any)?.status;
return (
axiosStatus === 401 ||
axiosStatus === 403 ||
attachedStatus === 401 ||
attachedStatus === 403
);
}
/**
* Connect to the collaboration websocket, read the live doc, apply
* `transform`, write the result, and wait for the server to persist it
* WITHOUT acquiring the per-page lock.
*
* This mirrors collaboration.mutatePageContent EXCEPT that it does not call
* withPageLock. It exists solely so replaceImage can hold ONE withPageLock
* across its scan -> upload -> write sequence: the per-page mutex is NOT
* reentrant, so calling the normal (self-locking) mutatePageContent inside an
* outer withPageLock for the same pageId would deadlock. The caller MUST hold
* the page lock for the whole operation; this helper assumes that invariant.
*
* `transform` receives the live ProseMirror doc and returns the NEW full doc
* to write, or `null` to abort with no write. Errors thrown by `transform`
* propagate to the caller.
*
* Resolves a `MutationResult { doc, verify }` mirroring mutatePageContent, so
* every content mutator (including replaceImage) can return a verifiable
* change report. The report is computed AFTER the atomic read->write and
* never throws.
*/
protected async mutateLiveContentUnlocked(
pageId: string,
collabToken: string,
transform: (liveDoc: any) => any | null,
): Promise<MutationResult> {
// Reuse a live CollabSession for the page (issue #400) instead of opening a
// fresh provider per op. acquireCollabSession does NOT take the per-page
// lock — the caller (replaceImage) already holds ONE withPageLock across its
// scan -> upload -> write sequence, and the mutex is not reentrant, so
// taking it here would deadlock. The synchronous read->write section and the
// unsyncedChanges/connectionLost ack logic live in CollabSession.mutate,
// preserved verbatim from the old inline machine (incl. the #152 structural
// diff that keeps a live editor's cursor anchored).
const session = await acquireCollabSession(pageId, collabToken, this.apiUrl, {
// Only the actual 25s collab connect timeout emits this — the connect-vs-
// unload signal; the other failure paths must NOT emit it.
onConnectTimeout: () =>
this.onMetricFn?.("collab_connect_timeouts_total", 1),
});
try {
return await session.mutate(transform);
} catch (e) {
// Drop the session on any failure so the next call reconnects fresh.
session.destroy("mutate failed");
throw e;
}
}
/**
* Generic pagination handler for Docmost API endpoints
*/
async paginateAll<T = any>(
endpoint: string,
basePayload: Record<string, any> = {},
limit: number = 100,
): Promise<T[]> {
await this.ensureAuthenticated();
const clampedLimit = Math.max(1, Math.min(100, limit));
// Hard ceiling on the number of pages to fetch: guards against a server
// that returns a perpetually-true hasNextPage (which would otherwise loop
// forever and accumulate duplicates).
const MAX_PAGES = 50;
let cursor: string | undefined;
let allItems: T[] = [];
let truncated = false;
for (let page = 0; page < MAX_PAGES; page++) {
const payload: Record<string, any> = {
...basePayload,
limit: clampedLimit,
};
if (cursor) payload.cursor = cursor;
const response = await this.client.post(endpoint, payload);
const data = response.data;
const items = data.data?.items || data.items || [];
const meta = data.data?.meta || data.meta;
allItems = allItems.concat(items);
// Advance strictly via the server-issued cursor. A missing nextCursor (or
// hasNextPage false) means we reached the end. A cursor identical to the
// one we just sent means the server did not understand our pagination
// param — stop instead of re-fetching page one forever and duplicating.
const next = meta?.hasNextPage ? meta?.nextCursor : null;
if (!next || next === cursor) {
// If the server still reports more pages but stopped issuing a usable
// cursor at the ceiling, flag the result as truncated below.
if (page === MAX_PAGES - 1 && meta?.hasNextPage) truncated = true;
break;
}
cursor = next;
// Reaching the ceiling with more pages still available means the result
// set is truncated.
if (page === MAX_PAGES - 1) truncated = true;
}
// If the loop stopped because it hit the MAX_PAGES ceiling while the server
// still reported more results, the result set is truncated — warn so the
// caller is not silently handed an incomplete list.
if (truncated) {
console.warn(
`paginateAll: results from "${endpoint}" truncated at the ${MAX_PAGES}-page cap; more pages exist on the server`,
);
}
return allItems;
}
/** Raw page info including the ProseMirror JSON content and slugId. */
async getPageRaw(pageId: string) {
await this.ensureAuthenticated();
const response = await this.client.post("/pages/info", { pageId });
return response.data?.data ?? response.data;
}
/**
* Resolve an agent-supplied pageId to the page's CANONICAL UUID (`page.id`),
* so every collaboration document the MCP opens is named `page.<uuid>` the
* SAME name the web editor always uses (`page.${page.id}`).
*
* The agent commonly passes a 10-char public slugId (from URLs/listings) as
* the pageId. The web editor opens the collab doc by UUID, but the MCP used to
* pass that slugId straight into the collab doc name (`page.<slugId>`). For one
* DB row that produced TWO independent Yjs documents whose debounced stores
* clobbered each other the agent's edit was silently lost (#260).
*
* A UUID input short-circuits with no network round-trip. A slugId is resolved
* once via getPageRaw and cached (both slugId->uuid and uuid->uuid), so
* repeated edits on the same page add no extra request.
*/
protected async resolvePageId(pageId: string): Promise<string> {
if (isUuid(pageId)) return pageId;
const cached = this.pageIdCache.get(pageId);
if (cached) return cached;
const data = await this.getPageRaw(pageId);
const uuid = data?.id;
if (typeof uuid !== "string" || !uuid) {
throw new Error(
`Could not resolve a canonical page id for "${pageId}"`,
);
}
this.pageIdCache.set(pageId, uuid);
return uuid;
}
/**
* Page-locked write seam over collaboration.mutatePageContent. Production just
* delegates; it exists as an overridable method so the insertFootnote wrapper
* (transform abort-on-not-found + response shaping) can be unit-tested without
* standing up a live Hocuspocus collab socket.
*
* SELF-RESOLVES the pageId to the canonical UUID (issue #449, "resolve-then-
* lock"): every write must lock and key its CollabSession by the UUID, never a
* raw slugId (#260). resolvePageId is cached/idempotent, so a caller that
* already resolved pays no extra round-trip; centralizing it here means a
* caller that reaches this seam with a raw slugId still locks correctly instead
* of silently splitting the mutex key. withPageLock also asserts the key is a
* UUID as a hard backstop.
*/
protected async mutatePage(
pageId: string,
collabToken: string,
apiUrl: string,
transform: (doc: any) => any,
): Promise<{ doc?: any; verify?: any }> {
const pageUuid = await this.resolvePageId(pageId);
return mutatePageContent(pageUuid, collabToken, apiUrl, transform);
}
/**
* Full-document write seam over collaboration.replacePageContent. Production
* just delegates; it exists as an overridable method so the full-doc write
* tools (updatePageJson, copyPageContent) can have their footnote-
* canonicalization binding unit-tested without a live Hocuspocus collab socket.
*
* SELF-RESOLVES the pageId to the canonical UUID (issue #449, "resolve-then-
* lock") for the same reason as mutatePage above the lock/CollabSession key
* is guaranteed canonical here, not left to the caller's discipline.
*/
protected async replacePage(
pageId: string,
doc: any,
collabToken: string,
apiUrl: string,
): Promise<{ doc?: any; verify?: any }> {
const pageUuid = await this.resolvePageId(pageId);
return replacePageContent(pageUuid, doc, collabToken, apiUrl);
}
/**
* Export a page to a single self-contained Docmost-flavoured markdown file:
* meta block + body (with inline comment anchors + diagrams) + comment
* threads. Lossless round-trip target; see importPageMarkdown for the inverse.
*/
}
+248
View File
@@ -0,0 +1,248 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Public method surface of DocValidateMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IDocValidateMixin` fails to compile on drift.
export interface IDocValidateMixin {
}
export function DocValidateMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IDocValidateMixin> & TBase {
abstract class DocValidateMixin extends Base implements IDocValidateMixin {
/**
* Validate a URL string against a scheme allowlist for a given context.
*
* The markdown link path enforces safe schemes via TipTap, but the raw
* JSON path (updatePageJson) bypasses that so this is the sanitization
* choke point for ProseMirror JSON written directly by the caller.
*
* - "link": reject javascript:, vbscript:, data: (any scheme that can
* execute or smuggle script when the href is clicked).
* - "src": allow only http(s):, mailto:, /api/files paths, or a
* scheme-less relative/absolute path; reject
* javascript:/vbscript:/data:/file:.
*/
protected isSafeUrl(url: unknown, context: "link" | "src"): boolean {
if (typeof url !== "string") return false;
const trimmed = url.trim();
if (trimmed === "") return true; // empty href/src is harmless
// Extract a leading "scheme:" if present. A scheme must start with a
// letter and contain only letters/digits/+/-/. before the colon. Strip
// whitespace and ASCII control chars first so a tab/newline embedded in
// the scheme cannot smuggle a dangerous scheme past the check.
const cleaned = trimmed.replace(/[\s\x00-\x1f]+/g, "");
const schemeMatch = /^([a-zA-Z][a-zA-Z0-9+.-]*):/.exec(cleaned);
const scheme = schemeMatch ? schemeMatch[1].toLowerCase() : null;
const dangerous = new Set(["javascript", "vbscript", "data", "file"]);
if (context === "link") {
if (scheme === null) return true; // relative/anchor link is fine
// For links, data: is also blocked (can carry script payloads).
return !new Set(["javascript", "vbscript", "data"]).has(scheme);
}
// context === "src"
if (scheme === null) return true; // relative/absolute path (incl. /api/files)
if (dangerous.has(scheme)) return false;
return scheme === "http" || scheme === "https" || scheme === "mailto";
}
/**
* Recursively walk a ProseMirror doc and reject any unsafe URL on a link
* mark href or on a media node's src/url. Media nodes covered: image,
* attachment, video, plus embed (rendered as an iframe), youtube, drawio
* and excalidraw all of which carry a user-controlled URL that Docmost
* renders. Throws a clear error on the first violation. A max-depth guard
* turns an over-deep document into a clean error instead of a RangeError
* stack overflow.
*/
protected validateDocUrls(node: any, depth: number = 0): void {
const MAX_DEPTH = 200;
if (depth > MAX_DEPTH) {
throw new Error(
`document nesting exceeds the maximum depth of ${MAX_DEPTH}`,
);
}
if (!node || typeof node !== "object") return;
// Link marks on text nodes: validate the href.
if (Array.isArray(node.marks)) {
for (const mark of node.marks) {
if (mark && mark.type === "link" && mark.attrs) {
if (!this.isSafeUrl(mark.attrs.href, "link")) {
throw new Error(`unsafe link href rejected: "${mark.attrs.href}"`);
}
}
}
}
// Media nodes: validate src/url against the stricter src allowlist.
// embed renders as an iframe (highest risk); youtube/drawio/excalidraw
// likewise carry a user-controlled URL Docmost renders, so they get the
// same scheme check as image/attachment/video.
if (
node.type === "image" ||
node.type === "attachment" ||
node.type === "video" ||
node.type === "embed" ||
node.type === "youtube" ||
node.type === "drawio" ||
node.type === "excalidraw" ||
node.type === "audio" ||
node.type === "pdf"
) {
const attrs = node.attrs || {};
for (const key of ["src", "url"]) {
if (attrs[key] != null && !this.isSafeUrl(attrs[key], "src")) {
throw new Error(
`unsafe ${node.type} ${key} rejected: "${attrs[key]}"`,
);
}
}
}
if (Array.isArray(node.content)) {
for (const child of node.content) {
this.validateDocUrls(child, depth + 1);
}
}
}
/**
* Recursively validate the STRUCTURE of a ProseMirror node (reuses the
* recursion shape of validateDocUrls). Every node must be an object with a
* string `type`; when present, `content` must be an array, `marks` must be
* an array of objects each with a string `type`, and a text node's `text`
* must be a string. Throws a clear "invalid ProseMirror document" error on
* the first violation. A max-depth guard turns an over-deep document into a
* clean error instead of a RangeError stack overflow.
*/
protected validateDocStructure(node: any, depth: number = 0): void {
const MAX_DEPTH = 200;
if (depth > MAX_DEPTH) {
throw new Error(
`invalid ProseMirror document: nesting exceeds the maximum depth of ${MAX_DEPTH}`,
);
}
if (!node || typeof node !== "object" || typeof node.type !== "string") {
throw new Error(
"invalid ProseMirror document: every node must be an object with a string `type`",
);
}
if (
"text" in node &&
node.type === "text" &&
typeof node.text !== "string"
) {
throw new Error(
"invalid ProseMirror document: a text node must have a string `text`",
);
}
if (node.marks !== undefined) {
if (!Array.isArray(node.marks)) {
throw new Error(
"invalid ProseMirror document: `marks` must be an array",
);
}
for (const mark of node.marks) {
if (
!mark ||
typeof mark !== "object" ||
typeof mark.type !== "string"
) {
throw new Error(
"invalid ProseMirror document: every mark must be an object with a string `type`",
);
}
}
}
if (node.content !== undefined) {
if (!Array.isArray(node.content)) {
throw new Error(
"invalid ProseMirror document: `content` must be an array when present",
);
}
for (const child of node.content) {
this.validateDocStructure(child, depth + 1);
}
}
}
/**
* Pre-write SHAPE gate (#409). Walk the WHOLE node tree with the shared
* `findInvalidNode` and throw a rich, path-anchored error the instant a nested
* node has an absent/unknown `type` (or an unknown mark) the exact shape that
* otherwise surfaces DEEP in the Yjs encode as the cryptic
* `Unknown node type: undefined`, but only AFTER a collab session was opened
* and a page lock taken. Calling this BEFORE `getCollabTokenWithReauth` /
* `mutatePageContent` fails fast: no collab connection, no lock, deterministic
* message. `op` names the tool for the message prefix (e.g. "patchNode").
*
* `findInvalidNode` derives its "known type" set from the very same
* `docmostExtensions` the encode path uses, so a node this gate accepts is one
* the encoder will accept too.
*/
protected assertValidNodeShape(op: string, node: any): void {
const bad = findInvalidNode(node);
if (bad) {
throw new Error(`${op}: invalid node — ${bad.summary}`);
}
}
/**
* Replace page content with a raw ProseMirror JSON document (lossless) and/or
* update its title. Both `doc` and `title` are optional, but at least one must
* be supplied:
* - `doc` provided -> validate + full-overwrite the body (and update the
* title too when `title` is also given).
* - `doc` omitted, `title` given -> title-only update; the body is NOT
* touched/resent (no collab write happens).
* - neither given -> throws (nothing to update).
*/
}
return DocValidateMixin;
}
+707
View File
@@ -0,0 +1,707 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import { parseCells as parseDrawioCells } from "../lib/drawio-xml.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import {
prepareModel,
decodeDrawioSvg,
buildDrawioSvg,
mxHash,
normalizeXml,
countUserCells,
} from "../lib/drawio-xml.js";
import { renderDiagramShapes } from "../lib/drawio-preview.js";
import { applyElkLayout } from "../lib/drawio-layout.js";
import {
buildFromGraph,
type Graph,
type LayoutMode as GraphLayoutMode,
} from "../lib/drawio-graph.js";
import { applyCellOps, type CellOp } from "../lib/drawio-cell-ops.js";
import { mermaidToGraph } from "../lib/drawio-mermaid.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Public method surface of DrawioMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IDrawioMixin` fails to compile on drift.
export interface IDrawioMixin {
drawioGet(pageId: string, node: string, format?: "xml" | "svg"): Promise<{ pageId: string; nodeId: string; format: "xml" | "svg"; content: string; meta: { attachmentId: string | null; title: string | null; width: number | null; height: number | null; cellCount: number; hash: string; }; }>;
drawioCreate(pageId: string, where: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }, xml: string, title?: string, layout?: "elk"): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; verify?: any; }>;
drawioUpdate(pageId: string, node: string, xml: string, baseHash: string, layout?: "elk"): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; verify?: any; }>;
drawioEditCells(pageId: string, node: string, operations: CellOp[], baseHash: string): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; verify?: any; }>;
drawioFromGraph(pageId: string, where: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }, graph: Graph, direction?: "LR" | "RL" | "TB" | "BT", preset?: string, layout?: GraphLayoutMode, node?: string): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; iconsResolved: number; iconsMissing: string[]; verify?: any; }>;
drawioFromMermaid(pageId: string, where: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }, mermaid: string, preset?: string): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; iconsResolved: number; iconsMissing: string[]; verify?: any; }>;
}
export function DrawioMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IDrawioMixin> & TBase {
abstract class DrawioMixin extends Base implements IDrawioMixin {
/**
* Resolve a drawio node on a page by `attrs.id` or `#<index>` and return the
* node plus its ref. Throws a clear error if the ref does not resolve to a
* drawio node.
*/
protected async resolveDrawioNode(
pageId: string,
node: string,
): Promise<{ node: any; ref: string }> {
const data = await this.getPageRaw(pageId);
const hit = getNodeByRef(
data.content ?? { type: "doc", content: [] },
node,
);
if (!hit) {
throw new Error(
`drawio: no node found for "${node}" on page ${pageId} (use the drawio node's attrs.id or "#<index>" from getOutline)`,
);
}
if (hit.type !== "drawio") {
throw new Error(
`drawio: node "${node}" on page ${pageId} is a ${hit.type}, not a drawio diagram`,
);
}
return { node: hit.node, ref: node };
}
/**
* Read a drawio diagram as mxGraph XML (default) or as the raw `.drawio.svg`.
* Runs the decode chain (base64/entity content= drawio file nested XML or
* pako-inflated compressed <diagram>). The returned `hash` is the
* optimistic-lock key for drawioUpdate.
*/
async drawioGet(
pageId: string,
node: string,
format: "xml" | "svg" = "xml",
): Promise<{
pageId: string;
nodeId: string;
format: "xml" | "svg";
content: string;
meta: {
attachmentId: string | null;
title: string | null;
width: number | null;
height: number | null;
cellCount: number;
hash: string;
};
}> {
await this.ensureAuthenticated();
const { node: drawio } = await this.resolveDrawioNode(pageId, node);
const attrs = drawio.attrs || {};
const src = attrs.src;
if (!src) {
throw new Error(
`drawio: node "${node}" on page ${pageId} has no src to read`,
);
}
const svg = await this.fetchAttachmentText(src);
const modelXml = decodeDrawioSvg(svg);
const meta = {
attachmentId: attrs.attachmentId ?? null,
title: attrs.title ?? null,
width: attrs.width != null ? Number(attrs.width) : null,
height: attrs.height != null ? Number(attrs.height) : null,
cellCount: countUserCells(modelXml),
hash: mxHash(modelXml),
};
return {
pageId,
nodeId: attrs.id ?? node,
format,
content: format === "svg" ? svg : normalizeXml(modelXml),
meta,
};
}
/**
* Create a drawio diagram from mxGraph XML: lint schematic SVG preview
* (pure TS) build the `.drawio.svg` (createDrawioSvg contract) create the
* attachment insert a `drawio` node before/after an anchor or appended.
* `xml` is a bare `<mxGraphModel>` or a list of `<mxCell>` (the server wraps
* it and adds the id=0/id=1 sentinels).
*/
async drawioCreate(
pageId: string,
where: {
position: "before" | "after" | "append";
anchorNodeId?: string;
anchorText?: string;
},
xml: string,
title?: string,
layout?: "elk",
): Promise<{
success: boolean;
nodeId: string;
attachmentId: string;
warnings: string[];
verify?: any;
}> {
await this.ensureAuthenticated();
if (
!where ||
(where.position !== "before" &&
where.position !== "after" &&
where.position !== "append")
) {
throw new Error(
'drawioCreate: `where.position` must be one of "before", "after", "append"',
);
}
if (where.position === "before" || where.position === "after") {
const hasId =
typeof where.anchorNodeId === "string" && where.anchorNodeId.length > 0;
const hasText =
typeof where.anchorText === "string" && where.anchorText.length > 0;
if (hasId === hasText) {
throw new Error(
`drawioCreate: position "${where.position}" requires exactly one of anchorNodeId or anchorText`,
);
}
}
// Optional server-side ELK auto-layout: the model declares structure with
// rough coords, ELK computes the pixels (best-effort — returns the input
// unchanged on any layout failure).
const laidOutXml = layout === "elk" ? await applyElkLayout(xml) : xml;
// Pre-write pipeline (throws a structured DrawioLintError on any violation).
const prepared = prepareModel(laidOutXml);
const inner = renderDiagramShapes(prepared.cells, prepared.bbox);
const diagramTitle = title || "Page-1";
const svg = buildDrawioSvg(prepared.modelXml, inner, prepared.bbox, diagramTitle);
const att = await this.uploadAttachmentBuffer(
pageId,
Buffer.from(svg, "utf-8"),
"diagram.drawio.svg",
"image/svg+xml",
);
// NOTE: no `id` attribute is set here. The vendored `drawio` node schema
// (diagramAttributes) declares no `id`, so any block id would be silently
// dropped by PMNode.fromJSON on save and the returned handle would fail to
// resolve. The addressable handle is the node's "#<index>" (like image/table
// nodes), computed after the insert below.
const drawioNode: any = {
type: "drawio",
attrs: {
src: `/api/files/${att.id}/${att.fileName}`,
attachmentId: att.id,
width: prepared.bbox.width,
height: prepared.bbox.height,
align: "center",
},
};
if (title) drawioNode.attrs.title = title;
// Reuse the existing URL trust boundary (rejects unsafe src schemes).
this.validateDocUrls(drawioNode);
const collabToken = await this.getCollabTokenWithReauth();
const pageUuid = await this.resolvePageId(pageId);
let inserted = false;
let insertedIndex = -1;
const mutation = await this.mutatePage(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
inserted = false;
insertedIndex = -1;
const { doc: nd, inserted: ins } = insertNodeRelative(
liveDoc,
drawioNode,
where,
);
inserted = ins;
if (!inserted) return null; // anchor not found -> skip the write
// Locate the freshly-inserted node to derive its "#<index>" handle. The
// just-uploaded attachmentId is unique, so it identifies our node.
if (Array.isArray(nd.content)) {
insertedIndex = nd.content.findIndex(
(b: any) =>
b &&
b.type === "drawio" &&
b.attrs &&
b.attrs.attachmentId === att.id,
);
}
return nd;
},
);
if (!inserted) {
const anchorDesc = where.anchorNodeId
? `anchorNodeId "${where.anchorNodeId}"`
: `anchorText "${where.anchorText}"`;
throw new Error(
`drawioCreate: anchor not found (${anchorDesc}) on page ${pageId}. The diagram attachment ${att.id} is now an unreferenced orphan.`,
);
}
if (insertedIndex < 0) {
// The node was inserted nested (e.g. inside a callout/table cell via an
// anchor), where "#<index>" — which addresses only top-level blocks —
// cannot reference it. drawio nodes carry no persisted id, so there is no
// stable handle for a nested diagram.
throw new Error(
`drawioCreate: the diagram was inserted on page ${pageId} but not as a ` +
`top-level block, so it has no addressable "#<index>" handle. Anchor ` +
`on a top-level block (or append) so the diagram can be re-read.`,
);
}
// The returned handle is POSITIONAL ("#<index>"): valid for the immediate
// create -> get/update flow, but re-resolve via getOutline if the document
// structure changes (blocks added/removed before it shift the index).
const nodeId = `#${insertedIndex}`;
return {
success: true,
nodeId,
attachmentId: att.id,
warnings: prepared.warnings,
verify: mutation.verify,
};
}
/**
* Full-replacement update of a drawio diagram. `baseHash` is MANDATORY: it is
* compared against the hash of the diagram's CURRENT XML (from drawioGet);
* any mismatch means a human or another agent edited the diagram after the
* read, so the write is refused with a conflict error. On success the new
* `.drawio.svg` is uploaded as a FRESH attachment (in-place byte overwrite is
* avoided some Docmost versions corrupt an attachment on overwrite, exactly
* as replaceImage documents) and the node is repointed with new dimensions.
*/
async drawioUpdate(
pageId: string,
node: string,
xml: string,
baseHash: string,
layout?: "elk",
): Promise<{
success: boolean;
nodeId: string;
attachmentId: string;
warnings: string[];
verify?: any;
}> {
await this.ensureAuthenticated();
if (typeof baseHash !== "string" || baseHash.length === 0) {
throw new Error(
"drawioUpdate: baseHash is mandatory — read the diagram with drawioGet first and pass back its meta.hash",
);
}
// Resolve the node and read the CURRENT diagram to enforce the optimistic
// lock before doing any write or upload.
const { node: drawio, ref } = await this.resolveDrawioNode(pageId, node);
const oldAttrs = drawio.attrs || {};
const oldSrc = oldAttrs.src;
// The returned handle is the caller-supplied reference. drawio nodes carry
// no persisted id, so `ref` (an "#<index>" or a rare legacy attrs.id) is the
// honest identifier to hand back.
const nodeId = oldAttrs.id ?? ref;
if (!oldSrc) {
throw new Error(
`drawioUpdate: node "${node}" on page ${pageId} has no src to compare against`,
);
}
const currentSvg = await this.fetchAttachmentText(oldSrc);
const currentHash = mxHash(decodeDrawioSvg(currentSvg));
if (currentHash !== baseHash) {
throw new Error(
`drawioUpdate: conflict — the diagram changed since it was read ` +
`(baseHash ${baseHash} != current ${currentHash}). Re-read it with drawioGet and retry.`,
);
}
// Optional server-side ELK auto-layout (best-effort; see drawioCreate).
const laidOutXml = layout === "elk" ? await applyElkLayout(xml) : xml;
// Pipeline for the new content (throws a structured DrawioLintError).
const prepared = prepareModel(laidOutXml);
const inner = renderDiagramShapes(prepared.cells, prepared.bbox);
const diagramTitle = oldAttrs.title || "Page-1";
const svg = buildDrawioSvg(prepared.modelXml, inner, prepared.bbox, diagramTitle);
const att = await this.uploadAttachmentBuffer(
pageId,
Buffer.from(svg, "utf-8"),
"diagram.drawio.svg",
"image/svg+xml",
);
const newSrc = `/api/files/${att.id}/${att.fileName}`;
const collabToken = await this.getCollabTokenWithReauth();
const pageUuid = await this.resolvePageId(pageId);
let repointed = 0;
const repoint = (n: any) => {
n.attrs = {
...n.attrs,
src: newSrc,
attachmentId: att.id,
width: prepared.bbox.width,
height: prepared.bbox.height,
};
repointed++;
};
const mutation = await this.mutatePage(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
repointed = 0;
const doc =
liveDoc && liveDoc.type === "doc"
? liveDoc
: { type: "doc", content: [] };
if (!Array.isArray(doc.content)) doc.content = [];
// Repoint ONLY the resolved node — never every node that happens to
// share this attachmentId (a copied diagram is two nodes with one
// attachmentId; keying on it would clobber both). Re-resolve the same
// handle against the live doc and walk to its exact position.
const hit = getNodeByRef(doc, ref);
if (!hit || hit.type !== "drawio") return null; // vanished/changed -> skip
let target: any = doc;
for (const idx of hit.path) {
if (!target || !Array.isArray(target.content)) {
target = null;
break;
}
target = target.content[idx];
}
if (!target || target.type !== "drawio") return null;
repoint(target);
if (repointed === 0) return null; // node vanished concurrently -> skip
return doc;
},
);
if (repointed === 0) {
return {
success: true,
nodeId,
attachmentId: att.id,
warnings: [
...prepared.warnings,
"target drawio node was removed concurrently; uploaded attachment is unreferenced",
],
verify: mutation.verify,
};
}
return {
success: true,
nodeId,
attachmentId: att.id,
warnings: prepared.warnings,
verify: mutation.verify,
};
}
// --- draw.io high-level semantic tools (issue #425) ---
/**
* ID-based targeted edits of an existing drawio diagram (add / update / delete
* cells) instead of resending the whole XML. Reads the CURRENT diagram, checks
* the optimistic lock (`baseHash` is MANDATORY, exactly as drawioUpdate), applies
* the operations to the parsed model (a `delete` CASCADES to container children
* and to every edge whose source/target is deleted), then runs the SAME #423
* pipeline as drawioUpdate (lint + quality warnings -> preview -> attachment ->
* repoint the node). Ids are stable so diffs stay meaningful across edits.
*/
// --- draw.io high-level semantic tools (issue #425) ---
/**
* ID-based targeted edits of an existing drawio diagram (add / update / delete
* cells) instead of resending the whole XML. Reads the CURRENT diagram, checks
* the optimistic lock (`baseHash` is MANDATORY, exactly as drawioUpdate), applies
* the operations to the parsed model (a `delete` CASCADES to container children
* and to every edge whose source/target is deleted), then runs the SAME #423
* pipeline as drawioUpdate (lint + quality warnings -> preview -> attachment ->
* repoint the node). Ids are stable so diffs stay meaningful across edits.
*/
async drawioEditCells(
pageId: string,
node: string,
operations: CellOp[],
baseHash: string,
): Promise<{
success: boolean;
nodeId: string;
attachmentId: string;
warnings: string[];
verify?: any;
}> {
await this.ensureAuthenticated();
if (typeof baseHash !== "string" || baseHash.length === 0) {
throw new Error(
"drawioEditCells: baseHash is mandatory — read the diagram with drawioGet first and pass back its meta.hash",
);
}
if (!Array.isArray(operations) || operations.length === 0) {
throw new Error(
"drawioEditCells: operations must be a non-empty array of { op, ... }",
);
}
const { node: drawio, ref } = await this.resolveDrawioNode(pageId, node);
const oldAttrs = drawio.attrs || {};
const oldSrc = oldAttrs.src;
const nodeId = oldAttrs.id ?? ref;
if (!oldSrc) {
throw new Error(
`drawioEditCells: node "${node}" on page ${pageId} has no src to edit`,
);
}
const currentSvg = await this.fetchAttachmentText(oldSrc);
const currentModel = decodeDrawioSvg(currentSvg);
const currentHash = mxHash(currentModel);
if (currentHash !== baseHash) {
throw new Error(
`drawioEditCells: conflict — the diagram changed since it was read ` +
`(baseHash ${baseHash} != current ${currentHash}). Re-read it with drawioGet and retry.`,
);
}
// Apply the operations to the parsed model, then run the standard pipeline.
const editedModel = applyCellOps(currentModel, operations);
const prepared = prepareModel(editedModel);
const inner = renderDiagramShapes(prepared.cells, prepared.bbox);
const diagramTitle = oldAttrs.title || "Page-1";
const svg = buildDrawioSvg(prepared.modelXml, inner, prepared.bbox, diagramTitle);
const att = await this.uploadAttachmentBuffer(
pageId,
Buffer.from(svg, "utf-8"),
"diagram.drawio.svg",
"image/svg+xml",
);
const newSrc = `/api/files/${att.id}/${att.fileName}`;
const collabToken = await this.getCollabTokenWithReauth();
const pageUuid = await this.resolvePageId(pageId);
let repointed = 0;
const mutation = await this.mutatePage(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
repointed = 0;
const doc =
liveDoc && liveDoc.type === "doc" ? liveDoc : { type: "doc", content: [] };
if (!Array.isArray(doc.content)) doc.content = [];
const hit = getNodeByRef(doc, ref);
if (!hit || hit.type !== "drawio") return null;
let target: any = doc;
for (const idx of hit.path) {
if (!target || !Array.isArray(target.content)) {
target = null;
break;
}
target = target.content[idx];
}
if (!target || target.type !== "drawio") return null;
target.attrs = {
...target.attrs,
src: newSrc,
attachmentId: att.id,
width: prepared.bbox.width,
height: prepared.bbox.height,
};
repointed++;
return doc;
},
);
if (repointed === 0) {
return {
success: true,
nodeId,
attachmentId: att.id,
warnings: [
...prepared.warnings,
"target drawio node was removed concurrently; uploaded attachment is unreferenced",
],
verify: mutation.verify,
};
}
return {
success: true,
nodeId,
attachmentId: att.id,
warnings: prepared.warnings,
verify: mutation.verify,
};
}
/**
* The main high-level tool: build a diagram from a SEMANTIC graph (nodes with
* a `kind`/`icon`, groups, edges) the model never supplies coordinates or
* style strings. The server resolves icons via the shape catalog (#424),
* assigns palette colors from the preset, runs ELK layered layout (honouring
* `direction` and the `layer`/`sameLayerAs`/`pinned` hints and compound groups),
* and assembles linter-clean XML, then inserts it through the SAME create
* pipeline as drawioCreate. `layout:"incremental"` is only meaningful when a
* target `node` is given (it preserves that diagram's existing coordinates and
* places only new cells); on a fresh insert it behaves like "full".
*/
async drawioFromGraph(
pageId: string,
where: {
position: "before" | "after" | "append";
anchorNodeId?: string;
anchorText?: string;
},
graph: Graph,
direction?: "LR" | "RL" | "TB" | "BT",
preset?: string,
layout?: GraphLayoutMode,
node?: string,
): Promise<{
success: boolean;
nodeId: string;
attachmentId: string;
warnings: string[];
iconsResolved: number;
iconsMissing: string[];
verify?: any;
}> {
await this.ensureAuthenticated();
// Direction/preset supplied as separate params override the graph fields so
// both the flat tool schema and an inline graph can set them.
const merged: Graph = {
...graph,
direction: direction ?? graph.direction,
preset: preset ?? graph.preset,
};
const mode: GraphLayoutMode = layout ?? "full";
// Incremental into an EXISTING node: read its coords so ELK preserves them,
// and keep the full existing model so incremental MERGES (never drops) any
// cell the new graph doesn't re-list.
let existingCoords: Map<string, { x: number; y: number }> | undefined;
let existingModelXml: string | undefined;
let editExisting = false;
let baseHash: string | undefined;
if (node && (mode === "incremental" || mode === "none")) {
const { node: drawio } = await this.resolveDrawioNode(pageId, node);
const src = (drawio.attrs || {}).src;
if (src) {
const svg = await this.fetchAttachmentText(src);
const model = decodeDrawioSvg(svg);
baseHash = mxHash(model);
existingModelXml = model;
existingCoords = new Map();
for (const c of parseDrawioCells(model)) {
if (c.vertex && c.geometry.x != null && c.geometry.y != null) {
existingCoords.set(c.id, { x: c.geometry.x, y: c.geometry.y });
}
}
editExisting = true;
}
}
const built = await buildFromGraph(
merged,
mode,
existingCoords,
existingModelXml,
);
if (editExisting && node && baseHash) {
// Re-target the existing diagram: replace it with the assembled model.
const res = await this.drawioUpdate(pageId, node, built.modelXml, baseHash);
return {
...res,
iconsResolved: built.iconsResolved,
iconsMissing: built.iconsMissing,
};
}
const res = await this.drawioCreate(pageId, where, built.modelXml);
return {
...res,
iconsResolved: built.iconsResolved,
iconsMissing: built.iconsMissing,
};
}
/**
* Convert a Mermaid `flowchart` to a redactable draw.io diagram via a PURE
* parser (no Electron / draw.io CLI): mermaid text -> graph-JSON -> the
* drawioFromGraph pipeline. Only `flowchart`/`graph` is supported (the most
* common wiki case); other diagram types throw a clear error so the model can
* fall back to drawioFromGraph.
*/
async drawioFromMermaid(
pageId: string,
where: {
position: "before" | "after" | "append";
anchorNodeId?: string;
anchorText?: string;
},
mermaid: string,
preset?: string,
): Promise<{
success: boolean;
nodeId: string;
attachmentId: string;
warnings: string[];
iconsResolved: number;
iconsMissing: string[];
verify?: any;
}> {
await this.ensureAuthenticated();
const graph = mermaidToGraph(mermaid);
if (preset) graph.preset = preset;
return this.drawioFromGraph(pageId, where, graph, graph.direction, graph.preset);
}
// --- Page history / diff / transform ---
/**
* List the saved versions (history snapshots) of a page, newest first.
* Docmost auto-snapshots on every save. Returns one cursor-paginated page of
* results: `{ items, nextCursor }`. The history record's id field is `id`.
*/
}
return DrawioMixin;
}
+165
View File
@@ -0,0 +1,165 @@
// Central REST error diagnostics (issues #437 + #450). SINGLE place that maps an
// axios error to the model-facing message. Extracted verbatim from client.ts;
// the constructor's response interceptor (see client/context.ts) routes every
// REST call through formatDocmostAxiosError so the whole surface is uniform.
import axios from "axios";
// --- Issue #437: central error diagnostics -------------------------------
// The agent only ever sees the thrown exception's `error.message`, so a failed
// tool must return an ACTIONABLE message (method, path, status, and the
// server's own validation text) instead of the opaque "Request failed with
// status code 400". These helpers + the response interceptor in the
// constructor are the single authoritative place that text is composed.
// Overall cap on the composed diagnostic message so the model context stays
// compact and a (whitelisted) server string can never blow up the text.
const ERROR_MESSAGE_CAP = 300;
// Only attempt to JSON.parse an arraybuffer body under this size: a larger
// binary body is never a JSON error envelope, so parsing it just wastes memory
// (fetchInternalFile uses responseType:"arraybuffer", so a failed file fetch
// carries the JSON error envelope as raw bytes here).
const ERROR_BUFFER_PARSE_CAP = 4096;
// Canonical 36-char UUID (8-4-4-4-12 hex). Deliberately version/variant-
// AGNOSTIC: the ids are UUIDv7 (e.g. 019f499a-9f8c-7d68-...), so only the
// canonical shape/length is enforced, not the version/variant nibble.
const FULL_UUID_RE =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
/**
* Throw an actionable error BEFORE any network call when `value` is not a full
* canonical UUID. Absorbs #436: a truncated/short comment id used to reach the
* server and bounce back as an opaque 400/404 the agent could not self-correct;
* failing fast here names the exact fix.
*/
export function assertFullUuid(
tool: string,
param: string,
value: string,
): void {
if (typeof value !== "string" || !FULL_UUID_RE.test(value)) {
throw new Error(
`${tool}: '${param}' must be the FULL comment UUID (36 chars, e.g. ` +
`019f499a-9f8c-7d68-b7be-ce100d7c6c56), got '${value}'. Copy the id ` +
`verbatim from listComments / createComment output.`,
);
}
}
// Keep ONLY the pathname of a request (no host, no query string, no fragment)
// so the message never leaks a host or query params. Resolves a relative
// config.url against config.baseURL, then discards everything but the path.
function requestPath(config: any): string {
const rawUrl = typeof config?.url === "string" ? config.url : "";
const base =
typeof config?.baseURL === "string" ? config.baseURL : undefined;
try {
// A dummy base makes an absolute config.url parse too; its host is dropped.
return new URL(rawUrl, base ?? "http://localhost").pathname;
} catch {
// Malformed url: still strip any query/fragment manually.
return rawUrl.split(/[?#]/)[0] || rawUrl;
}
}
/**
* Compose the server-facing message from `error.response.data`, using ONLY the
* whitelisted `message`/`error` fields or the HTTP statusText. SECURITY: the
* raw response body, headers (Authorization!) and config are NEVER read here
* a string/HTML body (e.g. a proxy's 502 page) is deliberately dropped in
* favour of the statusText.
*/
function extractServerMessage(data: any, statusText: string): string {
// class-validator envelope: { message: string | string[], error?: string }.
if (
data &&
typeof data === "object" &&
!Buffer.isBuffer(data) &&
!(data instanceof ArrayBuffer)
) {
const msg = (data as any).message;
if (Array.isArray(msg)) {
const joined = msg.filter((m) => typeof m === "string").join("; ");
if (joined) return joined;
} else if (typeof msg === "string" && msg) {
return msg;
}
const err = (data as any).error;
if (typeof err === "string" && err) return err;
return statusText;
}
// Buffer / ArrayBuffer body: attempt a size-capped, guarded JSON.parse so a
// failed arraybuffer fetch still surfaces the server's validation text.
if (Buffer.isBuffer(data) || data instanceof ArrayBuffer) {
const buf = Buffer.isBuffer(data) ? data : Buffer.from(data);
if (buf.length > 0 && buf.length <= ERROR_BUFFER_PARSE_CAP) {
try {
return extractServerMessage(JSON.parse(buf.toString("utf8")), statusText);
} catch {
return statusText;
}
}
return statusText;
}
// A raw string / HTML body is never surfaced (may echo server internals).
return statusText;
}
/**
* Reformat an AxiosError's `.message` IN PLACE into an actionable diagnostic:
* `<METHOD> <path> failed (<status> <statusText>): <serverMessage>`
* or, when the request never got a response:
* `<METHOD> <path> failed: <code> (no response from server)`.
*
* Mutates the SAME error object (never a custom subclass) so the live
* axios.isAxiosError / error.response?.status / config._retry checks around the
* client keep working, and sets `_docmostFormatted` as a double-processing
* guard. A no-op on a non-axios or already-formatted error.
*/
export function formatDocmostAxiosError(error: any): void {
if (!error || error._docmostFormatted) return;
if (!axios.isAxiosError(error)) return;
const config: any = error.config ?? {};
const method =
typeof config.method === "string" ? config.method.toUpperCase() : "";
const methodPath = `${method} ${requestPath(config)}`.trim();
const response = error.response;
let message: string;
if (response) {
const statusText =
typeof response.statusText === "string" ? response.statusText : "";
const serverMessage = extractServerMessage(response.data, statusText);
message = `${methodPath} failed (${response.status} ${statusText}): ${serverMessage}`;
// Full body only to stderr under DEBUG (parity with downloadImage).
if (process.env.DEBUG) {
console.error(
"Docmost request failed; response body:",
JSON.stringify(response.data),
);
}
} else {
// No response at all (ECONNREFUSED / ETIMEDOUT / ECONNRESET / DNS / timeout).
// Use ONLY error.code, never the raw error.message: axios network messages
// embed host:port ("connect ECONNREFUSED 127.0.0.1:3000", "getaddrinfo
// ENOTFOUND host") and #437's invariant is that the host never reaches the
// model-visible message. code is set for essentially every real no-response
// error (ECONNREFUSED/ETIMEDOUT/ECONNRESET/ENOTFOUND/ECONNABORTED); the full
// native message still goes to stderr under DEBUG.
const reason = error.code ?? "network error";
message = `${methodPath} failed: ${reason} (no response from server)`;
if (process.env.DEBUG) {
console.error("Docmost request failed; no response:", error.message);
}
}
if (message.length > ERROR_MESSAGE_CAP) {
message = message.slice(0, ERROR_MESSAGE_CAP - 1) + "…";
}
error.message = message;
(error as any)._docmostFormatted = true;
}
+730
View File
@@ -0,0 +1,730 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import FormData from "form-data";
import axios, { AxiosInstance } from "axios";
import { basename, extname } from "path";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import { withPageLock, isUuid } from "../lib/page-lock.js";
import { diffDocs, summarizeChange } from "../lib/diff.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Supported image types, kept as two lookup tables so both a local file
// extension and a remote Content-Type can be mapped to the same canonical set.
const EXT_TO_MIME: Record<string, string> = {
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".gif": "image/gif",
".webp": "image/webp",
".svg": "image/svg+xml",
};
const MIME_TO_EXT: Record<string, string> = {
"image/png": ".png",
"image/jpeg": ".jpg",
"image/gif": ".gif",
"image/webp": ".webp",
"image/svg+xml": ".svg",
};
// Public method surface of MediaMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IMediaMixin` fails to compile on drift.
export interface IMediaMixin {
uploadImage(pageId: string, url: string): any;
insertImage(pageId: string, url: string, opts?: { align?: "left" | "center" | "right"; alt?: string; replaceText?: string; afterText?: string; }): any;
replaceImage(pageId: string, oldAttachmentId: string, url: string, opts?: { align?: "left" | "center" | "right"; alt?: string }): any;
}
export function MediaMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IMediaMixin> & TBase {
abstract class MediaMixin extends Base implements IMediaMixin {
// --- Image upload / embedding ---
/** Map a Content-Type string to a supported MIME type, or null if unsupported. */
protected supportedImageMime(ct: string): string | null {
return MIME_TO_EXT[ct] ? ct : null;
}
/**
* Download a remote image from a caller-supplied URL and resolve its bytes,
* MIME and a filename.
*
* SSRF / RESOURCE TRUST BOUNDARY: the URL comes from the MCP caller and is
* fetched BY THE SERVER, so it must be guarded before and after the request.
* The guards mirror the local-file trust boundary in uploadImage:
* - scheme allowlist (http/https only) rejects file:, data:, ftp:, etc.,
* so the caller cannot use this path to read local files or other schemes;
* - a size cap enforced both via axios maxContentLength/maxBodyLength AND a
* post-download buffer.length re-check (defends against a missing/lying
* Content-Length), so a huge response cannot exhaust memory;
* - a 30s timeout. The timeout matters because replaceImage holds the
* per-page lock across this upload, so a hung download would wedge the
* lock for that page.
* We deliberately do NOT block private IP ranges: the MCP caller is already
* trusted to read arbitrary host files via the filePath path, so the marginal
* trust granted by fetching internal URLs is comparable, and blocking would
* break legitimate internal-image use.
*/
protected async fetchRemoteImage(
url: string,
maxBytes: number,
): Promise<{ buffer: Buffer; mime: string; fileName: string }> {
// Scheme allowlist first — cheapest guard, and rejects non-http(s) schemes
// (file:, data:, ftp:, ...) before any network request is made.
let parsed: URL;
try {
parsed = new URL(url);
} catch (e: any) {
throw new Error(`Invalid image URL "${url}": ${e.message}`);
}
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
throw new Error(
`unsupported image URL scheme "${parsed.protocol}"; only http and https are allowed`,
);
}
let response;
try {
response = await axios.get(url, {
responseType: "arraybuffer",
timeout: 30000,
maxContentLength: maxBytes,
maxBodyLength: maxBytes,
headers: { Accept: "image/*" },
});
} catch (error) {
// Keep the thrown message free of the raw response body (it may echo
// server internals); surface only status/statusText. The full body is
// logged under DEBUG for diagnostics.
if (axios.isAxiosError(error)) {
if (process.env.DEBUG) {
console.error(
"Image download failed; response body:",
JSON.stringify(error.response?.data),
);
}
throw new Error(
`Image download failed for "${url}": ${error.response?.status ?? ""} ${error.response?.statusText ?? error.message}`.trim(),
);
}
throw error;
}
// axios returns an ArrayBuffer for responseType: "arraybuffer".
const buffer = Buffer.from(response.data);
// Re-check the size: maxContentLength relies on Content-Length, which may be
// absent or lie, so guard against the actual byte count too.
if (buffer.length === 0) {
throw new Error(`Empty image response from "${url}"`);
}
if (buffer.length > maxBytes) {
throw new Error(
`Image too large: ${buffer.length} bytes exceeds the ${maxBytes}-byte cap`,
);
}
// Resolve MIME: prefer the response Content-Type (strip any "; charset=..."
// parameter, lowercase, trim) mapped through the supported set; if the
// header is generic/missing/unsupported, fall back to the URL path
// extension via the existing extension->MIME logic.
const rawCt = response.headers?.["content-type"];
let mime: string | null = null;
if (typeof rawCt === "string" && rawCt.length > 0) {
const ct = rawCt.split(";")[0].trim().toLowerCase();
mime = this.supportedImageMime(ct);
}
if (!mime) {
// Fall back to the URL path extension. Use the pathname so the query
// string never contaminates the extension lookup.
const ext = extname(parsed.pathname).toLowerCase();
mime = EXT_TO_MIME[ext] ?? null;
}
if (!mime) {
throw new Error(
`cannot determine supported image type for "${url}"; supported: png, jpg, jpeg, gif, webp, svg`,
);
}
// Build a filename from the URL path basename (ignore the query string),
// defaulting to "image" when empty, and ensure it ends with the canonical
// extension for the resolved MIME (append it when missing/mismatched).
const canonicalExt = MIME_TO_EXT[mime];
let fileName = basename(parsed.pathname) || "image";
if (extname(fileName).toLowerCase() !== canonicalExt) {
fileName += canonicalExt;
}
return { buffer, mime, fileName };
}
/** Build a Docmost ProseMirror image node from an uploaded attachment. */
protected buildImageNode(
att: { id: string; fileName: string; fileSize?: number },
align?: "left" | "center" | "right",
alt?: string,
): any {
// Clean file URL, matching Docmost's native behaviour. No cache-busting
// query: the server serves the bare URL correctly, and replacement creates
// a new attachment id (a new URL) which busts caches naturally.
const src = `/api/files/${att.id}/${att.fileName}`;
const node: any = {
type: "image",
attrs: {
src,
attachmentId: att.id,
// Default to null when the server omits fileSize so the attr is never
// undefined (undefined would be dropped on serialization / break the
// ProseMirror image schema which expects size present).
size: att.fileSize ?? null,
align: align || "center",
width: null,
},
};
if (alt) node.attrs.alt = alt;
return node;
}
/**
* Download a remote image from an http(s) URL and upload it as an attachment
* of a page, returning the attachment metadata plus a ready-to-insert
* ProseMirror image node. Local file paths are intentionally not supported:
* the MCP caller is a remote AI with no access to this server's filesystem.
*/
async uploadImage(pageId: string, url: string) {
await this.ensureAuthenticated();
const MAX_IMAGE_BYTES = 20 * 1024 * 1024; // 20 MiB
// Fetch + validate the remote image (scheme allowlist, size cap, timeout).
// See fetchRemoteImage for the SSRF / resource trust boundary.
const fetched = await this.fetchRemoteImage(url, MAX_IMAGE_BYTES);
const fileBuffer = fetched.buffer;
const mime = fetched.mime;
const fileName = fetched.fileName;
// Build a FRESH FormData for every send attempt. A FormData body is a
// single-use stream that is CONSUMED on the first send, so it cannot be
// replayed by this.client's response interceptor (replaying a consumed
// stream fails with 'socket hang up'). Multipart re-auth is therefore done
// here with bare axios and an explicit one-shot 401/403 retry that rebuilds
// the body. Field order matters: text fields must precede the file part so
// the server reads them; the server always generates a fresh attachment id.
const buildForm = () => {
const form = new FormData();
form.append("pageId", pageId);
form.append("file", fileBuffer, {
filename: fileName,
contentType: mime,
});
return form;
};
// Local name distinct from the `url` parameter (the source image URL): this
// is the /files/upload endpoint we POST the multipart body to.
const uploadUrl = `${this.apiUrl}/files/upload`;
let response;
try {
// Call buildForm() ONCE per attempt and reuse the instance for both
// getHeaders() and the body so the Content-Type boundary matches the body.
const form = buildForm();
// Read the Authorization header from this.client's defaults (set by
// login(), only ever deleted — never set to null) instead of building
// `Bearer ${this.token}`: a concurrent JSON 401 can null this.token
// mid-flight, which would otherwise produce a literal "Bearer null".
// ensureAuthenticated() above guarantees login() ran, so the default
// header exists here. A 60s timeout keeps a hung upload from wedging the
// per-page lock (replaceImage holds withPageLock across this call).
response = await axios.post(uploadUrl, form, {
headers: {
...form.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} catch (error) {
// On an expired-token auth error, re-login and retry exactly once with a
// freshly-rebuilt FormData (the previous one was already consumed).
if (
axios.isAxiosError(error) &&
(error.response?.status === 401 || error.response?.status === 403)
) {
await this.login();
const form2 = buildForm();
response = await axios.post(uploadUrl, form2, {
headers: {
...form2.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} else if (axios.isAxiosError(error)) {
// Keep the thrown message free of the raw response body (it may echo
// request data or server internals); surface only status/statusText.
// The full body is logged under DEBUG for diagnostics.
if (process.env.DEBUG) {
console.error(
"Image upload failed; response body:",
JSON.stringify(error.response?.data),
);
}
throw new Error(
`Image upload failed: ${error.response?.status} ${error.response?.statusText}`,
);
} else {
throw error;
}
}
// The attachment may arrive bare or wrapped in a { data } envelope.
const att = response.data?.data ?? response.data;
if (!att?.id || !att?.fileName) {
throw new Error(
"Unexpected /files/upload response: " + JSON.stringify(response.data),
);
}
// Some Docmost versions omit fileSize from the upload response. Fall back
// to the fetched byte length (the bytes we just uploaded) so callers never
// get an undefined size.
const resolvedSize = att.fileSize ?? fileBuffer.length;
return {
attachmentId: att.id,
fileName: att.fileName,
fileSize: resolvedSize,
src: `/api/files/${att.id}/${att.fileName}`,
imageNode: this.buildImageNode({ ...att, fileSize: resolvedSize }),
};
}
/**
* Upload an image from a web (http/https) URL and insert it into a page in
* one step.
* By default the image is appended at the end. With replaceText, the first
* top-level block whose text contains the string is replaced; with afterText,
* the image is inserted right after the first matching block. All other
* block ids are preserved (only one top-level block is added or swapped).
*/
async insertImage(
pageId: string,
url: string,
opts: {
align?: "left" | "center" | "right";
alt?: string;
replaceText?: string;
afterText?: string;
} = {},
) {
const up = await this.uploadImage(pageId, url);
// Reuse the node from uploadImage (clean /api/files/<id>/<file> src), then
// apply align/alt onto a shallow attrs copy.
const node: any = { ...up.imageNode, attrs: { ...up.imageNode.attrs } };
if (opts.align) node.attrs.align = opts.align;
if (opts.alt) node.attrs.alt = opts.alt;
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260). The
// uploadImage /files/upload call above keeps the agent-supplied id.
const pageUuid = await this.resolvePageId(pageId);
// Recursively collect the plain text of a top-level block.
const blockText = (n: any): string => {
let out = "";
if (n.type === "text") out += n.text || "";
for (const child of n.content || []) out += blockText(child);
return out;
};
// Insert into the LIVE synced document, not the debounced REST snapshot, so
// concurrent edits/comments/images are preserved and parallel insertImage
// calls (serialized by the per-page lock) each see the previous insertion.
let placement: "replaced" | "after" | "appended" | undefined;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
const doc =
liveDoc && liveDoc.type === "doc"
? liveDoc
: { type: "doc", content: [] };
if (!Array.isArray(doc.content)) doc.content = [];
if (opts.replaceText) {
// Ambiguity guard (mirrors editPageText): count matching top-level
// blocks first, so a non-unique fragment cannot silently replace the
// wrong block (e.g. text that also appears inside a callout/table).
const matches = doc.content.filter((b: any) =>
blockText(b).includes(opts.replaceText!),
);
if (matches.length === 0) {
throw new Error(`replaceText not found: "${opts.replaceText}"`);
}
if (matches.length > 1) {
throw new Error(
`replaceText "${opts.replaceText}" matches ${matches.length} blocks; use a longer unique fragment`,
);
}
const idx = doc.content.findIndex((b: any) =>
blockText(b).includes(opts.replaceText!),
);
// Data-loss guard: replaceText swaps the WHOLE top-level block, so if
// the fragment only appears nested inside a container (table, callout,
// list, blockquote) the entire structure would be destroyed. Refuse
// when the matched block is a container rather than a leaf
// paragraph/heading and point the caller at a safer tool.
const CONTAINER_TYPES = new Set([
"table",
"callout",
"bulletList",
"orderedList",
"taskList",
"blockquote",
]);
const matchedBlock = doc.content[idx];
if (matchedBlock && CONTAINER_TYPES.has(matchedBlock.type)) {
throw new Error(
`replaceText matched a ${matchedBlock.type} container block; replacing it would destroy the whole structure. ` +
`Use afterText to insert near it, or updatePageJson for surgical edits.`,
);
}
doc.content.splice(idx, 1, node);
placement = "replaced";
} else if (opts.afterText) {
// Ambiguity guard (mirrors editPageText): refuse a non-unique fragment.
const matches = doc.content.filter((b: any) =>
blockText(b).includes(opts.afterText!),
);
if (matches.length === 0) {
throw new Error(`afterText not found: "${opts.afterText}"`);
}
if (matches.length > 1) {
throw new Error(
`afterText "${opts.afterText}" matches ${matches.length} blocks; use a longer unique fragment`,
);
}
const idx = doc.content.findIndex((b: any) =>
blockText(b).includes(opts.afterText!),
);
doc.content.splice(idx + 1, 0, node);
placement = "after";
} else {
doc.content.push(node);
placement = "appended";
}
return doc;
},
);
return {
success: true,
pageId,
attachmentId: up.attachmentId,
src: up.src,
placement,
verify: mutation.verify,
};
}
/**
* Replace an existing image in a page with a new image fetched from a web
* (http/https) URL. Uploads the new file as a brand-new attachment, which
* yields a fresh clean URL that both renders correctly and busts browser
* caches (the URL changed). Finds every image node
* whose attrs.attachmentId === oldAttachmentId (recursively, incl. nodes nested
* in callouts/tables) and repoints its src/attachmentId/size, preserving
* comments, alignment and alt. Operates on the live collab document so comments
* and concurrent edits are preserved. Throws if no matching image is found.
*
* The OLD attachment is left in place as an unreferenced orphan: Docmost
* exposes NO HTTP API to delete a single content attachment (verified against
* the attachment controller/service and by probing the live API deletion
* happens only by cascade when the page, space or user is removed). This is the
* same outcome as Docmost's own editor when an image is removed/replaced.
* In-place byte overwrite is deliberately NOT used because some Docmost
* versions corrupt the attachment (HTTP 500) when its bytes are overwritten.
*/
async replaceImage(
pageId: string,
oldAttachmentId: string,
url: string,
opts: { align?: "left" | "center" | "right"; alt?: string } = {},
) {
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260). The
// page lock must ALSO key on the UUID so this operation serializes against
// other writes to the same page (mutatePageContent now locks by the resolved
// UUID too); locking by the raw slugId here would desync the mutex key and
// reopen the TOCTOU/orphan-attachment window the lock closes. uploadImage
// keeps the agent-supplied id (it hits REST, not the collab doc).
const pageUuid = await this.resolvePageId(pageId);
// Hold ONE per-page lock for the WHOLE operation (scan -> upload -> write).
// Previously the scan and the write were two separate mutatePageContent
// calls, each acquiring + releasing the lock, with the upload happening in
// the UNLOCKED gap between them. A concurrent op could interleave there: it
// could remove the target image so the write pass matches nothing, leaving
// the freshly-uploaded attachment as an un-deletable orphan (Docmost has no
// API to delete a single content attachment). Acquiring the lock once and
// using the non-locking collab helper inside (the per-page mutex is NOT
// reentrant, so the self-locking mutatePageContent would deadlock here)
// closes that TOCTOU window. uploadImage hits /files/upload over plain HTTP
// and does not touch the page lock, so it is safe to call while held.
return withPageLock(pageUuid, async () => {
// STEP 1: read-only live check. Scan the live document for any image node
// matching oldAttachmentId BEFORE uploading anything, so a wrong/stale id
// throws without ever creating an orphan attachment.
let matchFound = false;
const scan = (nodes: any[]) => {
for (const node of nodes) {
if (!node) continue;
if (
node.type === "image" &&
node.attrs &&
node.attrs.attachmentId === oldAttachmentId
) {
matchFound = true;
}
if (Array.isArray(node.content)) scan(node.content);
}
};
await this.mutateLiveContentUnlocked(pageUuid, collabToken, (liveDoc) => {
matchFound = false; // reset per-transform (collab may retry the read).
const doc =
liveDoc && liveDoc.type === "doc"
? liveDoc
: { type: "doc", content: [] };
if (Array.isArray(doc.content)) scan(doc.content);
return null; // read-only: never write on the check pass.
});
if (!matchFound) {
throw new Error(
`replaceImage: no image with attachmentId "${oldAttachmentId}" found on page ${pageId}`,
);
}
// STEP 2: a match exists — upload the new file as a FRESH attachment (new
// id, new clean URL) and repoint every matching node in a second pass.
// Still inside the SAME lock, so no other op can have changed the page
// since the scan.
const up = await this.uploadImage(pageId, url);
let replaced = 0;
// Swap the source of one image node, preserving align/alt/title/geometry.
const repoint = (node: any) => {
node.attrs = {
...node.attrs,
src: up.src,
attachmentId: up.attachmentId,
// Default to null when fileSize is unknown so the attr is never
// undefined.
size: up.fileSize ?? null,
};
if (opts.align) node.attrs.align = opts.align;
if (opts.alt !== undefined) node.attrs.alt = opts.alt;
replaced++;
};
// Recursively repoint every image node (incl. ones nested in callouts/tables).
const walk = (nodes: any[]) => {
for (const node of nodes) {
if (!node) continue;
if (
node.type === "image" &&
node.attrs &&
node.attrs.attachmentId === oldAttachmentId
) {
repoint(node);
}
if (Array.isArray(node.content)) walk(node.content);
}
};
const mutation = await this.mutateLiveContentUnlocked(
pageUuid,
collabToken,
(liveDoc) => {
// Reset per-transform so collab retries recompute cleanly (no double-count).
replaced = 0;
const doc =
liveDoc && liveDoc.type === "doc"
? liveDoc
: { type: "doc", content: [] };
if (!Array.isArray(doc.content)) doc.content = [];
walk(doc.content);
if (replaced === 0) return null; // no match -> skip the write entirely
return doc;
},
);
// KNOWN LIMITATION: a same-count image SRC swap (image count unchanged, no
// text/mark change) may still report verify.changed === false, because the
// text+marks+integrity-count model in summarizeChange does not inspect
// image `src`/attachmentId attributes. That is acceptable here — the
// replace is confirmed by `replaced` below, and verify is supplementary.
if (replaced === 0) {
// The pass-1 SCAN found the target (matchFound was true) and we already
// uploaded the new attachment, but pass-2 matched nothing — a concurrent
// editor must have removed the node between the two passes. Do NOT throw
// here (that would leak the just-uploaded attachment AND report failure);
// instead report success with the upload flagged as an unreferenced
// orphan so the caller knows. (The early throw above still covers the
// case where pass-1 finds nothing, before any upload happens.)
return {
success: true,
replaced: 0,
pageId,
oldAttachmentId,
newAttachmentId: up.attachmentId,
src: up.src,
orphanedAttachmentId: up.attachmentId,
warning:
"target image was removed concurrently; uploaded attachment is unreferenced",
verify: mutation.verify,
};
}
return {
success: true,
pageId,
replaced,
oldAttachmentId,
newAttachmentId: up.attachmentId,
src: up.src,
verify: mutation.verify,
};
});
}
// --- draw.io diagrams (issue #423) ---
/**
* Upload a ready-made byte buffer as a page attachment via the same
* multipart /files/upload endpoint uploadImage uses. Split out as its own
* (overridable) seam so drawioCreate/update can upload the generated
* `.drawio.svg` without going through the URL-fetch path, and so tests can
* stub the network. Mirrors uploadImage's fresh-FormData + one-shot 401/403
* re-auth handling (a FormData body is single-use, so it must be rebuilt per
* attempt).
*/
// --- draw.io diagrams (issue #423) ---
/**
* Upload a ready-made byte buffer as a page attachment via the same
* multipart /files/upload endpoint uploadImage uses. Split out as its own
* (overridable) seam so drawioCreate/update can upload the generated
* `.drawio.svg` without going through the URL-fetch path, and so tests can
* stub the network. Mirrors uploadImage's fresh-FormData + one-shot 401/403
* re-auth handling (a FormData body is single-use, so it must be rebuilt per
* attempt).
*/
protected async uploadAttachmentBuffer(
pageId: string,
buffer: Buffer,
fileName: string,
mime: string,
): Promise<{ id: string; fileName: string; fileSize: number }> {
await this.ensureAuthenticated();
const buildForm = () => {
const form = new FormData();
form.append("pageId", pageId);
form.append("file", buffer, { filename: fileName, contentType: mime });
return form;
};
const uploadUrl = `${this.apiUrl}/files/upload`;
let response;
try {
const form = buildForm();
response = await axios.post(uploadUrl, form, {
headers: {
...form.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} catch (error) {
if (
axios.isAxiosError(error) &&
(error.response?.status === 401 || error.response?.status === 403)
) {
await this.login();
const form2 = buildForm();
response = await axios.post(uploadUrl, form2, {
headers: {
...form2.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} else if (axios.isAxiosError(error)) {
if (process.env.DEBUG) {
console.error(
"Attachment upload failed; response body:",
JSON.stringify(error.response?.data),
);
}
throw new Error(
`Attachment upload failed: ${error.response?.status} ${error.response?.statusText}`,
);
} else {
throw error;
}
}
const att = response.data?.data ?? response.data;
if (!att?.id || !att?.fileName) {
throw new Error(
"Unexpected /files/upload response: " + JSON.stringify(response.data),
);
}
return {
id: att.id,
fileName: att.fileName,
fileSize: att.fileSize ?? buffer.length,
};
}
/**
* Fetch a stored `.drawio.svg` attachment as text. Overridable seam over
* fetchInternalFile (the authed loopback fetch, which also rejects any
* traversal/SSRF src) so drawioGet/update can read the current diagram and
* tests can stub the bytes.
*/
protected async fetchAttachmentText(src: string): Promise<string> {
const { buffer } = await this.fetchInternalFile(src);
return buffer.toString("utf-8");
}
/**
* Resolve a drawio node on a page by `attrs.id` or `#<index>` and return the
* node plus its ref. Throws a clear error if the ref does not resolve to a
* drawio node.
*/
}
return MediaMixin;
}
+700
View File
@@ -0,0 +1,700 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import {
importMarkdownFragment,
canBeDocChild,
findUnrepresentableTableAttrs,
} from "../lib/markdown-fragment.js";
import {
applyTextEdits,
TextEdit,
TextEditResult,
TextEditFailure,
} from "../lib/json-edit.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
import { normalizeAndMergeFootnotes } from "../lib/footnote-normalize-merge.js";
// Public method surface of NodesWriteMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements INodesWriteMixin` fails to compile on drift.
export interface INodesWriteMixin {
updatePageJson(pageId: string, doc?: any, title?: string): any;
editPageText(pageId: string, edits: TextEdit[]): any;
patchNode(pageId: string, nodeId: string, input: { markdown?: string; node?: any }): any;
insertNode(pageId: string, input: { markdown?: string; node?: any }, opts: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }): any;
deleteNode(pageId: string, nodeId: string): any;
}
export function NodesWriteMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & INodesWriteMixin> & TBase {
abstract class NodesWriteMixin extends Base implements INodesWriteMixin {
/**
* Replace page content with a raw ProseMirror JSON document (lossless) and/or
* update its title. Both `doc` and `title` are optional, but at least one must
* be supplied:
* - `doc` provided -> validate + full-overwrite the body (and update the
* title too when `title` is also given).
* - `doc` omitted, `title` given -> title-only update; the body is NOT
* touched/resent (no collab write happens).
* - neither given -> throws (nothing to update).
*/
async updatePageJson(pageId: string, doc?: any, title?: string) {
await this.ensureAuthenticated();
// Title-only / no-op handling: when no document is supplied, do NOT write
// the body. Update the title if one was given; otherwise there is nothing
// to do, so fail loudly rather than silently no-op.
if (doc == null) {
if (!title) {
throw new Error(
"updatePageJson: nothing to update (provide content and/or title)",
);
}
await this.client.post("/pages/update", { pageId, title });
return {
success: true,
modified: true,
message: "Page title updated (content left unchanged).",
pageId,
};
}
// Validate the document shape before a full overwrite: a malformed doc
// would otherwise silently corrupt the page (full-overwrite is the
// documented behaviour; no optimistic-concurrency is applied here).
if (
typeof doc !== "object" ||
doc.type !== "doc" ||
!Array.isArray(doc.content)
) {
throw new Error(
'content must be a ProseMirror document ({"type":"doc","content":[...]}) ' +
"where content is an array of nodes each having a string `type`",
);
}
// Recurse the WHOLE document so a malformed nested node (e.g. a node with a
// non-string type, a non-array content/marks, or a text node missing its
// string text) is rejected up front rather than silently corrupting the
// page on overwrite.
this.validateDocStructure(doc);
// #409: beyond the string-`type` check above, reject a nested node whose
// `type` is a string but NOT a known Docmost schema node (a typo/unknown
// block) — the same `Unknown node type` the encoder throws — with a rich,
// path-anchored message, still BEFORE any collab connection.
this.assertValidNodeShape("updatePageJson", doc);
// Sanitize URLs before writing. This closes the JSON-path bypass: unlike
// the markdown link path (which TipTap sanitizes), raw JSON could otherwise
// inject javascript:/data: link hrefs or media srcs straight into the doc.
this.validateDocUrls(doc);
// Canonicalize footnotes (idempotent): an agent-authored JSON doc cannot
// leave footnotes out of order, orphaned, or in multiple lists — the bottom
// list + numbering are always derived from reference order. No-op when the
// footnotes are already canonical.
// #419: normalize + merge glyph-forked definitions before canonicalizing.
doc = normalizeAndMergeFootnotes(doc);
doc = canonicalizeFootnotes(doc);
// Write the BODY first, then the title (#159 split-brain): a failed body
// write (e.g. persist timeout) must not leave a new title over the old body.
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
const mutation = await this.replacePage(
pageUuid,
doc,
collabToken,
this.apiUrl,
);
// Body persisted successfully — now it is safe to set the title.
if (title) {
await this.client.post("/pages/update", { pageId, title });
}
return {
success: true,
modified: true,
message: "Page content replaced from ProseMirror JSON.",
pageId,
verify: mutation.verify,
};
}
/**
* AUTHOR-INLINE footnote insertion. The agent supplies only WHERE
* (`anchorText`, a snippet of body text to attach the marker after) and WHAT
* (`text`, the footnote content as markdown). Numbering and the bottom
* `footnotesList` are derived deterministically server-side
* (`insertInlineFootnote` -> `canonicalizeFootnotes`): the agent never sees,
* assigns, or edits a footnote number or the list, so it CANNOT desync.
*
* Content DEDUP: when an existing definition has the same content, its id is
* reused (one number, one definition, several references). The write is atomic
* via `mutatePageContent` (single-writer, page-locked); if the anchor text is
* not found the transform aborts with a clear error and no write happens.
*/
/**
* Surgical text edits: find/replace inside text nodes of the live
* document. Preserves all block ids, marks, callouts and tables.
*/
async editPageText(pageId: string, edits: TextEdit[]) {
await this.ensureAuthenticated();
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
// Apply the edits against the LIVE synced document, not the debounced REST
// snapshot, so concurrent human edits/comments are preserved. applyTextEdits
// records per-edit match problems in `failed` instead of throwing, and
// applies whatever it can; we abort the write only when nothing applied.
let results: TextEditResult[] | undefined;
let failed: TextEditFailure[] | undefined;
// Whether we actually wrote new content. Set inside the transform: a
// degenerate edit (e.g. find === replace, or a batch that nets to no change)
// can "apply" yet leave the document byte-for-byte identical, in which case
// we must NOT write (no spurious history version) and must not claim a write
// happened.
let wrote = false;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
wrote = false;
const r = applyTextEdits(liveDoc, edits);
results = r.results;
failed = r.failed;
// Nothing applied -> abort the write (mutatePageContent treats a null
// return from the transform as "write nothing").
if (r.results.length === 0) return null;
// Edits "applied" but produced an identical document: skip the write so
// no new history version is created. Stable structural comparison via
// JSON.stringify (both docs come from the same deep-copied source, so
// key order is stable).
if (JSON.stringify(r.doc) === JSON.stringify(liveDoc)) return null;
wrote = true;
return r.doc;
},
);
if ((results?.length ?? 0) === 0 && (failed?.length ?? 0) > 0) {
// No edit applied: surface an aggregated, actionable error so the caller
// does not mistake a no-op for a partial success.
throw new Error(
"editPageText: no edits were applied (nothing written). " +
failed!.map((f) => `"${f.find}": ${f.reason}`).join("; "),
);
}
// Edits matched but produced no content change (identical document): report
// a successful no-op — NOT a failure — and do not falsely claim a write.
if (!wrote) {
return {
success: true,
pageId,
applied: results,
failed,
message: "No changes written (edits produced identical content).",
verify: mutation.verify,
};
}
const result: any = {
success: true,
pageId,
applied: results,
failed,
message:
(failed?.length ?? 0)
? `Applied ${results?.length ?? 0} edit(s); ${failed!.length} failed (see failed[]). Node ids and formatting preserved.`
: "Text edits applied (node ids and formatting preserved).",
verify: mutation.verify,
};
// If any applied edit matched only after stripping markdown (the
// normalized fallback), warn that editPageText preserved existing marks
// and did NOT change formatting — so a caller who intended a formatting
// change is pointed at patchNode.
if (results?.some((r) => r.normalized === true)) {
result.warning =
"Some edits matched only after stripping markdown from your find string; " +
"editPageText preserved existing marks (it did not change bold/strike/etc.). " +
"If you intended a formatting change, use patchNode.";
}
return result;
}
/**
* Replace the block whose attrs.id === nodeId. Operates on the LIVE collab
* document so comments and concurrent edits are preserved.
*
* Exactly one of `input.markdown` / `input.node` (#413):
* - `markdown` (RECOMMENDED): the block is rewritten from a canonical markdown
* fragment. The fragment may import to N blocks (a "1 -> N" splice: rewrite a
* whole section in one call). The FIRST resulting block INHERITS the target's
* `attrs.id` (so an existing comment anchoring the block by id survives); the
* rest get FRESH ids. `^[...]` footnotes in the fragment are first-class:
* their definitions merge into the page's TAIL footnote list (content-key
* dedup + canonicalize), same machinery insertFootnote uses. REJECTED when
* the TARGET block carries a table-cell attribute markdown cannot represent
* (colspan/rowspan/colwidth/background) use the table tools or `node`.
* - `node`: a raw ProseMirror node for precise attr/mark work. The replacement
* keeps the target id (if `node.attrs.id` is missing it is set to nodeId).
*
* #159 ambiguous-id semantics are unchanged: 0 matches -> "no node"; >1 matches
* -> "ambiguous, refused" (nothing written), on BOTH paths the markdown path
* runs a dry `replaceNodeById` count first, so a duplicated id never splices.
*/
async patchNode(
pageId: string,
nodeId: string,
input: { markdown?: string; node?: any },
) {
await this.ensureAuthenticated();
// XOR: exactly one of markdown / node. Both optional in the schema; the
// runtime enforces the recommendation ("markdown for prose, node for fine
// work") without letting an ambiguous both-or-neither call through.
const hasMd =
input != null &&
typeof input.markdown === "string" &&
input.markdown.trim() !== "";
const hasNode = input != null && input.node != null;
if (hasMd === hasNode) {
throw new Error(
"patchNode: provide exactly one of `markdown` (recommended, for prose) " +
"or `node` (a raw ProseMirror node, for precise attr/mark work)",
);
}
if (hasMd) {
return this.patchNodeMarkdown(pageId, nodeId, input.markdown as string);
}
return this.patchNodeJson(pageId, nodeId, input.node);
}
/**
* patchNode with a raw ProseMirror `node` (the pre-#413 behavior). Replaces
* EVERY node whose attrs.id === nodeId; the swapped-in node keeps the target
* id. #159 ambiguity refused. Split out so the markdown path can reuse the
* shared collab/guard plumbing without a giant branch.
*/
protected async patchNodeJson(pageId: string, nodeId: string, node: any) {
if (!node || typeof node !== "object" || typeof node.type !== "string") {
throw new Error(
"patchNode: `node` must be an object with a string `type`",
);
}
// Preserve the block id WITHOUT mutating the caller's object: build a local
// copy whose attrs.id === nodeId (so the swapped-in node keeps the id of the
// node it replaces).
const target = {
...node,
attrs: {
...(node.attrs && typeof node.attrs === "object" ? node.attrs : {}),
},
};
if (target.attrs.id == null) {
target.attrs.id = nodeId;
}
// #409: fail fast on a malformed node SHAPE (a nested child with an
// absent/unknown `type`, e.g. a text leaf written as `{"text":"foo"}` with
// no `"type":"text"`) BEFORE opening a collab session or taking the page
// lock — the root-only `typeof node.type === "string"` check above never
// sees nested children, and the encoder's `Unknown node type: undefined`
// would otherwise only surface after the connection.
this.assertValidNodeShape("patchNode", target);
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
// Track the replacement count in an outer var, reset per-transform, so a
// collab retry recomputes it cleanly (mirrors replaceImage's pattern).
let replaced = 0;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
replaced = 0;
const { doc: nd, replaced: r } = replaceNodeById(
liveDoc,
nodeId,
target,
);
replaced = r;
// 0 matches -> skip the write. >1 matches -> the id is AMBIGUOUS: Docmost
// duplicates block ids on copy/paste (and copyPageContent writes them
// verbatim), so replacing "the node with id X" would silently clobber
// EVERY duplicate (#159). Refuse: skip the write and throw below so the
// model re-targets with a more specific anchor instead of corrupting the
// page. Only an unambiguous single match is written.
if (replaced !== 1) return null;
return nd;
},
);
// 0 -> "no node"; >1 -> "ambiguous, refused" (the transform already skipped
// the write for any count !== 1). Single shared guard (#159, #185 review).
assertUnambiguousMatch("patchNode", "replace", replaced, nodeId, pageId);
return { success: true, replaced, nodeId, verify: mutation.verify };
}
/**
* patchNode with a MARKDOWN fragment (#413). Imports the fragment through the
* canonical importer, then 1 -> N splices the resulting blocks in place of the
* target block on the LIVE collab doc:
* - the FIRST block inherits the target's id; the rest get FRESH ids (minted
* by the importer/id-remap, so neighbour blocks are untouched);
* - `^[...]` footnote definitions merge into the page's tail list;
* - REJECTED when the target block carries a markdown-unrepresentable table
* attr (colspan/rowspan/colwidth/background) guarding against silent loss;
* - #159 ambiguity is enforced by a dry `replaceNodeById` count BEFORE the
* splice, so a duplicated id never writes.
*/
protected async patchNodeMarkdown(
pageId: string,
nodeId: string,
markdown: string,
) {
// Import the fragment up front (network-free, canonical) so a bad fragment
// fails before any collab connection or page lock.
const { blocks, definitions } = await importMarkdownFragment(markdown);
// The first imported block inherits the target id; the rest keep the fresh
// ids the importer assigned. Build the thread now so it is stable across a
// collab retry (the transform below is pure over its inputs).
const threaded = blocks.map((b, i) => {
if (i !== 0) return b;
return {
...b,
attrs: {
...(b && typeof b.attrs === "object" ? b.attrs : {}),
id: nodeId,
},
};
});
// Shape-validate every imported block up front (parity with the JSON path):
// the importer only emits schema nodes, but the check is cheap insurance and
// yields the same rich #409 diagnostics if the schema ever drifts.
for (const b of threaded) {
this.assertValidNodeShape("patchNode", b);
}
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
let replaced = 0;
let guardAttrs: string | null = null;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
replaced = 0;
guardAttrs = null;
// #159: count matches with the same recursive walk the JSON path uses;
// only an UNAMBIGUOUS single match may write. A dry count keeps the
// ambiguity semantics identical across both paths.
const { replaced: count } = replaceNodeById(liveDoc, nodeId, {
type: "paragraph",
});
replaced = count;
if (count !== 1) return null;
// Guard against SILENT LOSS: if the target block carries a table-cell
// attribute markdown cannot represent (colspan/rowspan/colwidth/
// background), refuse the markdown rewrite so those attrs are not
// dropped. Simple tables (no such attrs) rewrite fine.
const hit = getNodeByRef(liveDoc, nodeId);
guardAttrs = hit ? findUnrepresentableTableAttrs(hit.node) : null;
if (guardAttrs != null) return null;
// Re-mint any minted block id that collides with an existing page id
// (skip index 0: its id is intentionally the target nodeId, unique by
// the #159 dry-count above), so the 1 -> N splice stays page-wide unique.
reassignCollidingBlockIds(liveDoc, threaded, 0);
// 1 -> N splice, then merge any fragment footnote definitions into the
// page's tail list and re-derive canonical footnote numbering.
const { doc: spliced } = replaceNodeByIdWithMany(
liveDoc,
nodeId,
threaded,
);
return mergeFootnoteDefinitions(spliced, definitions);
},
);
// Surface the guard rejection with an actionable message (nothing written).
if (guardAttrs != null) {
throw new Error(
`patchNode: the target block has table-cell attributes markdown cannot ` +
`represent (${guardAttrs}) — a markdown rewrite would drop them. Use ` +
`the table tools (tableUpdateCell/tableInsertRow) or pass a raw ` +
`ProseMirror \`node\` instead of \`markdown\`.`,
);
}
// 0 -> "no node"; >1 -> "ambiguous, refused" (the transform skipped the write
// for any count !== 1). Shared #159 guard, identical to the JSON path.
assertUnambiguousMatch("patchNode", "replace", replaced, nodeId, pageId);
return {
success: true,
replaced,
nodeId,
blocks: threaded.length,
verify: mutation.verify,
};
}
/**
* Insert content relative to an anchor (or append it at the top level).
* Operates on the LIVE collab document so comments and concurrent edits are
* preserved.
*
* Exactly one of `input.markdown` / `input.node` (#413):
* - `markdown` (RECOMMENDED): a canonical markdown fragment. It may import to
* SEVERAL blocks they are inserted IN ORDER at the anchor. `^[...]`
* footnote definitions merge into the page's tail list (same machinery as
* insertFootnote). Every inserted block gets a fresh id.
* - `node`: a raw ProseMirror node for precise attr/mark work, or to insert
* table structure (a bare tableRow/tableCell/tableHeader NOT expressible in
* markdown, so those stay JSON-only).
*
* opts.position:
* - "append": push the content at the end of the top-level content.
* - "before"/"after": insert as a sibling of the anchor, just before/after it.
* Exactly one of anchorNodeId / anchorText must be given; anchorNodeId
* locates a node anywhere by attrs.id, anchorText matches the first top-level
* block whose plain text includes it.
*
* Throws if the anchor cannot be found.
*/
async insertNode(
pageId: string,
input: { markdown?: string; node?: any },
opts: {
position: "before" | "after" | "append";
anchorNodeId?: string;
anchorText?: string;
},
) {
await this.ensureAuthenticated();
// XOR: exactly one of markdown / node (both optional in the schema).
const hasMd =
input != null &&
typeof input.markdown === "string" &&
input.markdown.trim() !== "";
const hasNode = input != null && input.node != null;
if (hasMd === hasNode) {
throw new Error(
"insertNode: provide exactly one of `markdown` (recommended, for prose) " +
"or `node` (a raw ProseMirror node, for precise attr/mark work or table structure)",
);
}
if (
!opts ||
(opts.position !== "before" &&
opts.position !== "after" &&
opts.position !== "append")
) {
throw new Error(
'insertNode: `position` must be one of "before", "after", "append"',
);
}
if (opts.position === "before" || opts.position === "after") {
// before/after require EXACTLY ONE anchor (an id or a text fragment).
const hasId =
typeof opts.anchorNodeId === "string" && opts.anchorNodeId.length > 0;
const hasText =
typeof opts.anchorText === "string" && opts.anchorText.length > 0;
if (hasId === hasText) {
throw new Error(
`insertNode: position "${opts.position}" requires exactly one of anchorNodeId or anchorText`,
);
}
}
// Resolve the ordered list of blocks to insert plus any footnote definitions
// to merge. The markdown path imports canonically (so an inserted block is
// byte-identical to the same content in a full-page import); the node path is
// a single block with no footnote merge (raw JSON `^[...]` is not touched).
let blocks: any[];
let definitions: any[] = [];
if (hasMd) {
const frag = await importMarkdownFragment(input.markdown as string);
blocks = frag.blocks;
definitions = frag.definitions;
} else {
const node = input.node;
if (!node || typeof node !== "object" || typeof node.type !== "string") {
throw new Error(
"insertNode: `node` must be an object with a string `type`",
);
}
blocks = [node];
}
// #409: fail fast on a malformed node SHAPE (a nested child with an
// absent/unknown `type`) BEFORE opening a collab session or taking the page
// lock — the root-only check above never sees nested children.
for (const b of blocks) {
this.assertValidNodeShape("insertNode", b);
}
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
// Track insertion in an outer var, reset per-transform, so a collab retry
// recomputes it cleanly (mirrors replaceImage's pattern).
let inserted = false;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
inserted = false;
// Re-mint any minted block id that collides with an existing page id
// (all inserted blocks are fresh, no skip) so the splice stays unique.
if (hasMd) reassignCollidingBlockIds(liveDoc, blocks);
// Single-block node path keeps `insertNodeRelative` (it owns the
// structural table-node splicing); the markdown path uses the array
// splice so N blocks land in order at one anchor.
const res = hasMd
? insertNodesRelative(liveDoc, blocks, opts)
: insertNodeRelative(liveDoc, blocks[0], opts);
inserted = res.inserted;
if (!inserted) return null; // anchor not found -> skip the write entirely
// Merge any fragment footnote definitions into the page tail list and
// re-derive canonical numbering (no-op when there are none).
return mergeFootnoteDefinitions(res.doc, definitions);
},
);
if (!inserted) {
const anchorDesc = opts.anchorNodeId
? `anchorNodeId "${opts.anchorNodeId}"`
: `anchorText "${opts.anchorText}"`;
// anchorText is matched against the block's literal RENDERED plain text;
// markdown/emoji are tolerated only as a strip-and-retry fallback, so a
// miss usually means the text differs from what's on the page.
const hint = opts.anchorText
? " anchorText must be the block's literal rendered plain text (no markdown wrappers or emoji); anchorNodeId from getPageJson is more reliable."
: "";
throw new Error(
`insertNode: anchor not found (${anchorDesc}) on page ${pageId}.${hint}`,
);
}
return {
success: true,
inserted: true,
position: opts.position,
blocks: blocks.length,
verify: mutation.verify,
};
}
/**
* Remove EVERY node whose attrs.id === nodeId (recursively, including nodes
* nested in callouts/tables) from its parent content array. Operates on the
* LIVE collab document so comments and concurrent edits are preserved.
* Throws if no node matches.
*/
async deleteNode(pageId: string, nodeId: string) {
await this.ensureAuthenticated();
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
// Track the deletion count in an outer var, reset per-transform, so a
// collab retry recomputes it cleanly (mirrors replaceImage's pattern).
let deleted = 0;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
deleted = 0;
const { doc: nd, deleted: d } = deleteNodeById(liveDoc, nodeId);
deleted = d;
// 0 matches -> skip the write. >1 matches -> the id is AMBIGUOUS (block
// ids are duplicated on copy/paste, #159): deleting "the node with id X"
// would silently remove EVERY duplicate. Refuse: skip the write and throw
// below so the model re-targets. Only an unambiguous single match is
// deleted.
if (deleted !== 1) return null;
return nd;
},
);
// 0 -> "no node"; >1 -> "ambiguous, refused" (the transform already skipped
// the write for any count !== 1). Single shared guard (#159, #185 review).
assertUnambiguousMatch("deleteNode", "delete", deleted, nodeId, pageId);
return { success: true, deleted, nodeId, verify: mutation.verify };
}
/** Build the public share URL for a page. */
}
return NodesWriteMixin;
}
+655
View File
@@ -0,0 +1,655 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import FormData from "form-data";
import axios, { AxiosInstance } from "axios";
import { convertProseMirrorToMarkdown } from "../lib/markdown-converter.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import { footnoteWarningsField } from "../lib/footnote-analyze.js";
import {
serializeDocmostMarkdown,
parseDocmostMarkdown,
} from "../lib/markdown-document.js";
import { diffDocs, summarizeChange } from "../lib/diff.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
import { normalizeAndMergeFootnotes } from "../lib/footnote-normalize-merge.js";
import vm from "node:vm";
// Public method surface of PagesMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IPagesMixin` fails to compile on drift.
export interface IPagesMixin {
createPage(title: string, content: string, spaceId: string, parentPageId?: string): any;
updatePage(pageId: string, content: string, title?: string): any;
renamePage(pageId: string, title: string): any;
movePage(pageId: string, parentPageId: string | null, position?: string): any;
deletePage(pageId: string): any;
sharePage(pageId: string, searchIndexing?: boolean): any;
listShares(): any;
unsharePage(pageId: string): any;
exportPageMarkdown(pageId: string): Promise<string>;
importPageMarkdown(pageId: string, fullMarkdown: string): Promise<any>;
copyPageContent(sourcePageId: string, targetPageId: string): any;
listPageHistory(pageId: string, cursor?: string): any;
getPageHistory(historyId: string): any;
restorePageVersion(historyId: string): any;
diffPageVersions(pageId: string, from?: string, to?: string): any;
}
export function PagesMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IPagesMixin> & TBase {
abstract class PagesMixin extends Base implements IPagesMixin {
/**
* Create a new page with title and content.
* Uses the /pages/import workaround (the only endpoint accepting content),
* then moves the page and restores the exact title: the import endpoint
* derives the title from the FILENAME and replaces spaces with
* underscores, so we explicitly re-set it via /pages/update afterwards.
*/
async createPage(
title: string,
content: string,
spaceId: string,
parentPageId?: string,
) {
await this.ensureAuthenticated();
if (parentPageId) {
try {
await this.getPage(parentPageId);
} catch (e) {
throw new Error(`Parent page with ID ${parentPageId} not found.`);
}
}
// 1. Create content via Import (using multipart/form-data).
// Build a FRESH FormData per send attempt: a FormData body is a single-use
// stream consumed on the first send, so it cannot be replayed by
// this.client's response interceptor (replay fails with 'socket hang up').
// Multipart re-auth is therefore done here with bare axios and an explicit
// one-shot 401/403 retry that rebuilds the body.
const fileContent = Buffer.from(content, "utf-8");
const buildForm = () => {
const form = new FormData();
form.append("spaceId", spaceId);
form.append("file", fileContent, {
filename: `${title || "import"}.md`,
contentType: "text/markdown",
});
return form;
};
const importUrl = `${this.apiUrl}/pages/import`;
let response;
try {
// Call buildForm() ONCE per attempt and reuse the instance for both
// getHeaders() and the body so the Content-Type boundary matches the body.
const form = buildForm();
// Read the Authorization header from this.client's defaults (set by
// login(), only ever deleted — never set to null) instead of building
// `Bearer ${this.token}`: a concurrent JSON 401 can null this.token
// mid-flight, which would otherwise produce a literal "Bearer null".
// ensureAuthenticated() above guarantees login() ran, so the default
// header exists here.
response = await axios.post(importUrl, form, {
headers: {
...form.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} catch (error) {
// On an expired-token auth error, re-login and retry exactly once with a
// freshly-rebuilt FormData (the previous one was already consumed).
if (
axios.isAxiosError(error) &&
(error.response?.status === 401 || error.response?.status === 403)
) {
await this.login();
const form2 = buildForm();
response = await axios.post(importUrl, form2, {
headers: {
...form2.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} else {
throw error;
}
}
const newPageId = (response.data?.data ?? response.data).id;
// 2. Move to parent if needed
if (parentPageId) {
await this.movePage(newPageId, parentPageId);
}
// 3. Restore the exact title (import mangles spaces into underscores)
if (title) {
await this.client.post("/pages/update", { pageId: newPageId, title });
}
const page = await this.getPage(newPageId);
// Surface non-fatal footnote problems (dangling refs, empty/duplicate
// definitions, markers in tables) so the agent can fix its markup (#166).
return { ...page, ...footnoteWarningsField(content) };
}
/**
* Update a page's content from markdown and optionally its title.
* NOTE: full re-import block ids regenerate. For surgical changes
* use editPageText / updatePageJson instead.
*/
async updatePage(pageId: string, content: string, title?: string) {
await this.ensureAuthenticated();
// Open the collab doc by the canonical UUID, never the slugId (#260). The
// REST /pages/update title write below keeps the agent-supplied id (the
// server resolves a slugId there).
const pageUuid = await this.resolvePageId(pageId);
// Write the BODY first, then the title (#159 split-brain). If the collab
// body write fails (e.g. a persist timeout), the title must be left
// UNTOUCHED so the page never ends up with a new title over its old body.
// A title write failing AFTER a successful body is rarer (REST is fast) and
// leaves correct content under a stale title — the lesser inconsistency.
let collabToken = "";
let mutation;
try {
collabToken = await this.getCollabTokenWithReauth();
mutation = await updatePageContentRealtime(
pageUuid,
content,
collabToken,
this.apiUrl,
);
} catch (error: any) {
// Verbose diagnostics (incl. anything that could expose a token prefix)
// are gated behind DEBUG; the thrown Error below carries no token data.
if (process.env.DEBUG) {
console.error(
"Failed to update page content via realtime collaboration:",
error,
);
const tokenPreview = collabToken
? collabToken.substring(0, 15) + "..."
: "null";
console.error(`Collab token preview: ${tokenPreview}`);
}
throw new Error(`Failed to update page content: ${error.message}`);
}
// Body persisted successfully — now it is safe to set the title.
if (title) {
await this.client.post("/pages/update", { pageId, title });
}
return {
success: true,
modified: true,
message: "Page updated successfully.",
pageId: pageId,
verify: mutation.verify,
// Non-fatal footnote diagnostics (#166); omitted when there are none.
...footnoteWarningsField(content),
};
}
/**
* Validate a URL string against a scheme allowlist for a given context.
*
* The markdown link path enforces safe schemes via TipTap, but the raw
* JSON path (updatePageJson) bypasses that so this is the sanitization
* choke point for ProseMirror JSON written directly by the caller.
*
* - "link": reject javascript:, vbscript:, data: (any scheme that can
* execute or smuggle script when the href is clicked).
* - "src": allow only http(s):, mailto:, /api/files paths, or a
* scheme-less relative/absolute path; reject
* javascript:/vbscript:/data:/file:.
*/
/**
* Rename a page (change its title only) without touching or resending its
* content. The slug is derived from the page record, not the body, so it is
* left intact too.
*/
async renamePage(pageId: string, title: string) {
await this.ensureAuthenticated();
await this.client.post("/pages/update", { pageId, title });
return { success: true, pageId, title };
}
/**
* Copy the WHOLE content of one page onto another, entirely server-side: the
* source's ProseMirror document is read and written verbatim onto the target
* via the live collab path, so the document never passes through the model.
*
* Only the target's BODY is replaced its title and slug live on the page
* record (not in the content), so they are untouched. The source page is not
* modified at all.
*/
async movePage(
pageId: string,
parentPageId: string | null,
position?: string,
) {
await this.ensureAuthenticated();
// Docmost requires position >= 5 chars.
const validPosition = position || "a00000";
return this.client
.post("/pages/move", {
pageId,
parentPageId,
position: validPosition,
})
.then((res) => res.data);
}
async deletePage(pageId: string) {
await this.ensureAuthenticated();
return this.client
.post("/pages/delete", { pageId })
.then((res) => res.data);
}
// --- Comment methods (ported from upstream PR #3 by Max Nikitin) ---
/**
* Normalize a comment's `content` into a ProseMirror doc object before
* markdown conversion. createComment/updateComment send content as a
* JSON.stringify(...) STRING, and the server stores it as-is, so on read it
* comes back as a string. convertProseMirrorToMarkdown returns "" for a
* string, so parse it first (guarded fall back to the raw value on any
* parse failure so a non-JSON legacy value is still handled gracefully).
*/
/** Share a page publicly (idempotent) and return the public URL. */
async sharePage(pageId: string, searchIndexing: boolean = true) {
await this.ensureAuthenticated();
const response = await this.client.post("/shares/create", {
pageId,
includeSubPages: false,
searchIndexing,
});
const share = response.data?.data ?? response.data;
const slugId = share.page?.slugId || (await this.getPageRaw(pageId)).slugId;
return {
shareId: share.id,
key: share.key,
pageId: share.pageId,
publicUrl: this.shareUrl(share.key, slugId),
searchIndexing: share.searchIndexing,
};
}
/** List all public shares in the workspace with their URLs. */
/** Build the public share URL for a page. */
protected shareUrl(shareKey: string, slugId: string): string {
return `${this.appUrl}/share/${shareKey}/p/${slugId}`;
}
/** Share a page publicly (idempotent) and return the public URL. */
/** List all public shares in the workspace with their URLs. */
async listShares() {
const shares = await this.paginateAll("/shares", {});
return shares.map((s: any) => ({
shareId: s.id,
key: s.key,
pageId: s.pageId,
pageTitle: s.page?.title,
publicUrl: s.page?.slugId ? this.shareUrl(s.key, s.page.slugId) : null,
searchIndexing: s.searchIndexing,
createdAt: s.createdAt,
}));
}
/** Remove the public share of a page. */
async unsharePage(pageId: string) {
await this.ensureAuthenticated();
const shares = await this.listShares();
const share = shares.find((s: any) => s.pageId === pageId);
if (!share) {
throw new Error(`Page ${pageId} is not shared.`);
}
await this.client.post("/shares/delete", { shareId: share.shareId });
return { success: true, removedShareId: share.shareId, pageId };
}
/**
* Export a page to a single self-contained Docmost-flavoured markdown file:
* meta block + body (with inline comment anchors + diagrams) + comment
* threads. Lossless round-trip target; see importPageMarkdown for the inverse.
*/
async exportPageMarkdown(pageId: string): Promise<string> {
await this.ensureAuthenticated();
const page = await this.getPageRaw(pageId);
const body = page.content ? convertProseMirrorToMarkdown(page.content) : "";
let comments: any[] = [];
try {
// Lossless export: include RESOLVED threads so the export -> import
// round-trip preserves every comment. This is exactly why the active-only
// filter is an opt-in (default false) on listComments.
comments = (await this.listComments(pageId, true)).items;
} catch (e) {
// A comments fetch failure must not lose the body; export with [] and let
// the caller see the (empty) comments block. Log under DEBUG only.
if (process.env.DEBUG) console.error("export: listComments failed", e);
}
const meta = {
version: 1,
pageId: page.id,
slugId: page.slugId,
title: page.title,
spaceId: page.spaceId,
parentPageId: page.parentPageId ?? null,
};
return serializeDocmostMarkdown(meta, body, comments);
}
/**
* Import a self-contained Docmost markdown file back into a page. Parses out
* the meta + comments metadata blocks, converts the body to ProseMirror
* (restoring comment marks + diagrams from their inline HTML), and replaces
* the page content. Comment THREAD records are NOT written to the server in
* this version they are preserved in the file and the inline marks are
* re-applied so the highlights survive; managing comment records stays with
* the comment tools/UI.
*/
async importPageMarkdown(pageId: string, fullMarkdown: string): Promise<any> {
await this.ensureAuthenticated();
const { meta, body, comments } = parseDocmostMarkdown(fullMarkdown);
// PAGE import: canonicalize footnotes (see markdownToProseMirrorCanonical).
const doc = await markdownToProseMirrorCanonical(body);
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
const mutation = await replacePageContent(
pageUuid,
doc,
collabToken,
this.apiUrl,
);
// Collect distinct comment ids that actually became comment marks in the doc.
const collectCommentIds = (node: any, acc: Set<string>): Set<string> => {
if (!node || typeof node !== "object") return acc;
if (Array.isArray(node.marks)) {
for (const mk of node.marks) {
if (mk && mk.type === "comment" && mk.attrs?.commentId) {
acc.add(mk.attrs.commentId);
}
}
}
if (Array.isArray(node.content)) {
for (const child of node.content) collectCommentIds(child, acc);
}
return acc;
};
// Count reflects the comment marks present in the written document, so an id
// that only appears as inert text (e.g. inside a fenced code block) is not
// counted because it never becomes a comment mark.
const anchoredIds = collectCommentIds(doc, new Set<string>());
const result: any = {
success: true,
pageId,
anchoredCommentCount: anchoredIds.size,
commentsInFile: Array.isArray(comments) ? comments.length : 0,
verify: mutation.verify,
};
// Warn (non-fatal) if the file was exported from a DIFFERENT page.
if (meta?.pageId && meta.pageId !== pageId) {
result.warning = `File was exported from page ${meta.pageId} but is being imported into ${pageId}.`;
}
// Non-fatal footnote diagnostics (#166), analyzed on the BODY (the part after
// the docmost:meta / docmost:comments blocks) — so a `[^x]`-like token inside
// those JSON blocks never produces a false warning, while real markers in the
// body do. `body` comes from parseDocmostMarkdown(fullMarkdown) above.
Object.assign(result, footnoteWarningsField(body));
return result;
}
/**
* Rename a page (change its title only) without touching or resending its
* content. The slug is derived from the page record, not the body, so it is
* left intact too.
*/
/**
* Copy the WHOLE content of one page onto another, entirely server-side: the
* source's ProseMirror document is read and written verbatim onto the target
* via the live collab path, so the document never passes through the model.
*
* Only the target's BODY is replaced its title and slug live on the page
* record (not in the content), so they are untouched. The source page is not
* modified at all.
*/
async copyPageContent(sourcePageId: string, targetPageId: string) {
await this.ensureAuthenticated();
// A self-copy would be a no-op overwrite; reject it explicitly so a caller
// mistake surfaces as a clear error rather than a silent round-trip.
if (sourcePageId === targetPageId) {
throw new Error(
"copyPageContent: sourcePageId and targetPageId are the same page (no-op copy)",
);
}
const source = await this.getPageRaw(sourcePageId);
const content = source?.content;
if (
!content ||
typeof content !== "object" ||
content.type !== "doc" ||
!Array.isArray(content.content)
) {
throw new Error(
`copyPageContent: source page ${sourcePageId} has no usable ProseMirror content to copy`,
);
}
// Defense-in-depth: run the same URL-scheme sanitizer the JSON write path
// uses, so copying never lands a javascript:/data: href/src on the target
// (parity with updatePageJson; harmless for already-stored source content).
this.validateDocUrls(content);
// Defense-in-depth (#228): this is a FULL-document write, so canonicalize
// footnotes before copying — a no-op on already-canonical source content, but
// it guarantees a copy can never propagate a non-canonical footnote topology
// to the target (parity with the other full-doc write paths).
// #419: normalize + merge glyph-forked definitions before canonicalizing.
const canonical = canonicalizeFootnotes(normalizeAndMergeFootnotes(content));
const collabToken = await this.getCollabTokenWithReauth();
// Open the TARGET collab doc by its canonical UUID, never the slugId (#260).
const targetUuid = await this.resolvePageId(targetPageId);
const mutation = await this.replacePage(
targetUuid,
canonical,
collabToken,
this.apiUrl,
);
return {
success: true,
sourcePageId,
targetPageId,
copiedNodes: canonical.content.length,
verify: mutation.verify,
};
}
/**
* Surgical text edits: find/replace inside text nodes of the live
* document. Preserves all block ids, marks, callouts and tables.
*/
// --- Page history / diff / transform ---
/**
* List the saved versions (history snapshots) of a page, newest first.
* Docmost auto-snapshots on every save. Returns one cursor-paginated page of
* results: `{ items, nextCursor }`. The history record's id field is `id`.
*/
async listPageHistory(pageId: string, cursor?: string) {
await this.ensureAuthenticated();
const payload: Record<string, any> = { pageId };
if (cursor) payload.cursor = cursor;
const response = await this.client.post("/pages/history", payload);
const data = response.data?.data ?? response.data;
return {
items: data?.items ?? [],
nextCursor: data?.meta?.nextCursor ?? null,
};
}
/**
* Fetch a single page-history version including its lossless ProseMirror
* `content`. The version also carries pageId/title/createdAt.
*/
async getPageHistory(historyId: string) {
await this.ensureAuthenticated();
const response = await this.client.post("/pages/history/info", {
historyId,
});
return response.data?.data ?? response.data;
}
/**
* "Restore" a version: Docmost has NO restore endpoint, so we take the
* version's `content` and write it as the page's current content via the live
* collab path (which itself creates a new history snapshot). Returns the
* affected pageId and the source historyId.
*/
async restorePageVersion(historyId: string) {
await this.ensureAuthenticated();
const version = await this.getPageHistory(historyId);
if (
!version ||
!version.pageId ||
!version.content ||
typeof version.content !== "object"
) {
throw new Error(
`restorePageVersion: history ${historyId} has no usable content`,
);
}
// Defense-in-depth: sanitize URLs in the restored content (parity with the
// JSON write path) before writing it back.
this.validateDocUrls(version.content);
const collabToken = await this.getCollabTokenWithReauth();
// version.pageId is the page entity id (already a UUID); resolvePageId
// short-circuits a UUID with no round-trip, so this is defensive only (#260).
const pageUuid = await this.resolvePageId(version.pageId);
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
() => version.content,
);
return {
pageId: version.pageId,
restoredFrom: historyId,
verify: mutation.verify,
};
}
/**
* Diff two versions of a page and return a Docmost-equivalent change set.
* `from`/`to` each resolve to a ProseMirror doc:
* - null / undefined / "current" -> the page's CURRENT content;
* - any other string -> that historyId's content.
* Returns the diff plus the resolved version metadata for each side.
*/
async diffPageVersions(pageId: string, from?: string, to?: string) {
await this.ensureAuthenticated();
const isCurrent = (v?: string) => v == null || v === "" || v === "current";
const resolveSide = async (
v?: string,
): Promise<{ doc: any; meta: any }> => {
if (isCurrent(v)) {
const raw = await this.getPageRaw(pageId);
return {
doc: raw.content || { type: "doc", content: [] },
meta: {
kind: "current",
pageId,
title: raw.title,
updatedAt: raw.updatedAt,
},
};
}
const version = await this.getPageHistory(v as string);
return {
doc: version.content || { type: "doc", content: [] },
meta: {
kind: "history",
historyId: version.id,
pageId: version.pageId,
title: version.title,
createdAt: version.createdAt,
},
};
};
const fromSide = await resolveSide(from);
const toSide = await resolveSide(to);
const diff = diffDocs(fromSide.doc, toSide.doc);
return { from: fromSide.meta, to: toSide.meta, diff };
}
/**
* Edit a page by running an arbitrary user-supplied JS transform against the
* live document, with a diff preview + page-history safety net.
*
* The transform string is evaluated as `(doc, ctx) => doc` inside a node:vm
* sandbox: it gets ONLY `{ doc, ctx, structuredClone, console }` as globals,
* a 5s timeout, and NO access to require/process/fs/network. It must return a
* `{ type: "doc" }` node, which is validated structurally before any write.
*
* `ctx` exposes:
* - comments: the page's comments (fetched before the live read);
* - log: an array the transform can push diagnostics to (via console.log);
* - consume(id): mark a comment id as consumed (for deleteComments);
* - helpers: the transforms.ts primitives + commentsToFootnotes.
*
* Footnote convention used by the helpers: footnote markers are plain "[N]"
* text in the body, and the notes are an orderedList under a heading whose
* text is "Примечания переводчика".
*
* dryRun (default true): read the page's current content, run the transform,
* and return `{ pushed:false, diff, log }` WITHOUT opening the collab socket.
* Otherwise the transform runs atomically inside mutatePageContent, optionally
* deletes consumed comments, and returns the new historyId + diff + log.
*/
}
return PagesMixin;
}
+651
View File
@@ -0,0 +1,651 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import axios, { AxiosInstance } from "axios";
import {
filterWorkspace,
filterSpace,
filterPage,
filterComment,
filterSearchResult,
} from "../lib/filters.js";
import { convertProseMirrorToMarkdown } from "../lib/markdown-converter.js";
import {
collectInternalFileNodes,
normalizeFileUrl,
resolveInternalFilePath,
} from "../lib/internal-file-urls.js";
import { buildPageTree } from "../lib/tree.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import {
importMarkdownFragment,
canBeDocChild,
findUnrepresentableTableAttrs,
} from "../lib/markdown-fragment.js";
import { searchInDoc, SearchOptions } from "../lib/page-search.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Public method surface of ReadMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IReadMixin` fails to compile on drift.
export interface IReadMixin {
getWorkspace(): any;
getSpaces(): any;
listPages(spaceId?: string, limit?: number, tree?: boolean): any;
getTree(spaceId: string, rootPageId?: string, maxDepth?: number): any;
getPageContext(pageId: string): any;
listSidebarPages(spaceId: string, pageId?: string): any;
getPage(pageId: string): any;
getPageJson(pageId: string): any;
getOutline(pageId: string): any;
getNode(pageId: string, nodeId: string, format?: "markdown" | "json"): any;
searchInPage(pageId: string, query: string, opts?: SearchOptions): any;
getTable(pageId: string, tableRef: string): any;
search(query: string, spaceId?: string, limit?: number, opts?: { parentPageId?: string; titleOnly?: boolean }): any;
}
export function ReadMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IReadMixin> & TBase {
abstract class ReadMixin extends Base implements IReadMixin {
async getWorkspace() {
await this.ensureAuthenticated();
const response = await this.client.post("/workspace/info", {});
return {
data: filterWorkspace(response.data?.data ?? response.data),
success: response.data.success,
};
}
async getSpaces() {
const spaces = await this.paginateAll("/spaces", {});
return spaces.map((space) => filterSpace(space));
}
/**
* List pages in one of two modes.
*
* Default (`tree` false): most recent pages by updatedAt (descending),
* bounded. Fetching the whole space can exceed MCP response/time limits on
* large instances, so a single bounded page of results is returned (default
* 50, max 100) via the `/pages/recent` feed.
*
* Tree (`tree` true): DEPRECATED prefer `getTree`, which shares this exact
* code path (a single `/pages/tree` request via `enumerateSpacePages` +
* `buildPageTree`) but returns the compact `{pageId, title, children?,
* hasChildren?}` shape and supports `rootPageId`/`maxDepth`. This tree mode is
* kept for backward compatibility; it REQUIRES `spaceId` (a page tree is
* scoped to one space) and IGNORES `limit` the whole hierarchy is returned.
* It fetches the tree via `enumerateSpacePages`, which on the fork server
* resolves to a single `/pages/tree` request returning the whole
* permission-filtered flat page set (soft-deleted pages excluded
* server-side); the cursor-BFS in `enumerateSpacePages` is only a fallback for
* stock upstream servers that lack `/pages/tree`.
*/
async listPages(spaceId?: string, limit: number = 50, tree: boolean = false) {
await this.ensureAuthenticated();
if (tree) {
if (!spaceId) {
throw new Error(
"listPages: tree mode requires a spaceId (a page tree is scoped to one space). Pass spaceId, or omit tree to get the recent-pages list.",
);
}
const { pages } = await this.enumerateSpacePages(spaceId);
return buildPageTree(pages);
}
const clampedLimit = Math.max(1, Math.min(100, limit));
const payload: Record<string, any> = { limit: clampedLimit, page: 1 };
if (spaceId) payload.spaceId = spaceId;
const response = await this.client.post("/pages/recent", payload);
const data = response.data;
const items = data.data?.items || data.items || [];
return items.map((page: any) => filterPage(page));
}
/**
* Fetch a space's page hierarchy (or one subtree) as a nested tree in a SINGLE
* request the #443 `getTree` tool. Shares its whole code path with
* `listPages(tree:true)`: `enumerateSpacePages` issues one `POST /pages/tree`
* (with the cursor-BFS only as a fallback for stock upstream servers that lack
* the endpoint), then `buildPageTree` nests the flat, permission-filtered,
* position-ordered list. No second tree fetch, no per-node BFS.
*
* - `rootPageId` restrict to that page's subtree; the server seeds the CTE
* with the page itself, so the result is exactly ONE root (the page and its
* descendants). Omit it for the whole space.
* - `maxDepth` trim the response to that many levels (roots = depth 1) to
* save tokens; the server still returns everything in one request, the cut
* is applied in `buildPageTree` AFTER the full tree is built. A node whose
* children were cut carries `hasChildren: true` (source of truth = the flat
* item's server `hasChildren`) so the caller can descend with a follow-up
* `getTree(spaceId, rootPageId=that node)` call.
*
* Output nodes are `{pageId, title, children?, hasChildren?}` only the UUID
* `pageId` is exposed (never `slugId`/`icon`/`position`). Requires `spaceId`
* (a page tree is scoped to one space).
*/
async getTree(spaceId: string, rootPageId?: string, maxDepth?: number) {
await this.ensureAuthenticated();
if (!spaceId) {
throw new Error(
"getTree: spaceId is required (a page tree is scoped to one space).",
);
}
const { pages } = await this.enumerateSpacePages(spaceId, rootPageId);
return buildPageTree(pages, { shape: "getTree", maxDepth });
}
/**
* "Where am I / what's around" for a single page the #443 `getPageContext`
* tool. Metadata only (no page content), using exactly TWO server requests:
*
* 1. `POST /pages/breadcrumbs` a recursive CTE that walks UP from the page.
* The server returns the chain root->page order (it `.reverse()`s the
* child-first walk before responding), INCLUDING the page itself as the
* LAST element. So the last element is the page and everything before it
* is the ancestor chain root->parent. This carries the page's own title
* and spaceId, so no extra page-info fetch is needed for a UUID input.
* 2. `listSidebarPages(spaceId, pageId)` the page's DIRECT children,
* cursor-paginated (a page with >20 children returns ALL of them, no
* dupes) and in sidebar `position` order, each carrying `hasChildren`.
*
* The input may be a slugId (agents copy them from URLs); it is run through
* `resolvePageId` first, exactly like the other page tools. A UUID input adds
* no request there (short-circuit), keeping the total at two; a slugId input
* adds one unavoidable resolve round-trip.
*
* INVARIANT: only the UUID `pageId` is exposed anywhere server `id` is
* mapped to `pageId` and `slugId` is never leaked. A nonexistent/inaccessible
* pageId makes the server 404/403, which propagates as a clear tool error
* (never a hollow empty object).
*/
async getPageContext(pageId: string) {
await this.ensureAuthenticated();
// Resolve a possibly-slugId input to the canonical UUID (no round-trip for a
// UUID). Errors here (bad/inaccessible id) propagate as a clear tool error.
const pageUuid = await this.resolvePageId(pageId);
// Request 1: the ancestor chain, root->page, page included as the LAST item.
const response = await this.client.post("/pages/breadcrumbs", {
pageId: pageUuid,
});
const chain: any[] = (response.data?.data ?? response.data) ?? [];
if (!Array.isArray(chain) || chain.length === 0) {
// The endpoint always includes the page itself, so an empty chain means
// the page is gone/inaccessible — surface a clear error, not {}.
throw new Error(`getPageContext: page "${pageId}" not found or inaccessible`);
}
// Split: the last element is the page, the rest (root->parent) are the
// breadcrumbs. A root page has no ancestors -> breadcrumbs is [].
const self = chain[chain.length - 1];
const ancestors = chain.slice(0, -1);
const page = {
pageId: self.id,
title: self.title,
spaceId: self.spaceId,
};
const breadcrumbs = ancestors.map((n: any) => ({
pageId: n.id,
title: n.title,
}));
// Request 2: direct children in sidebar order, each with hasChildren.
const childItems = await this.listSidebarPages(self.spaceId, pageUuid);
const children = childItems.map((c: any) => ({
pageId: c.id,
title: c.title,
hasChildren: Boolean(c.hasChildren),
}));
return { page, breadcrumbs, children };
}
/**
* List sidebar pages for a space. With no pageId the request returns the
* space ROOT pages; with a pageId it returns the direct CHILDREN of that
* page. pageId is therefore optional and is only included in the POST body
* when provided (an empty/undefined pageId would otherwise change the
* semantics on the server).
*/
async listSidebarPages(spaceId: string, pageId?: string) {
await this.ensureAuthenticated();
// Paginate via the server-issued cursor. The server switched from OFFSET
// (`page`) to CURSOR (`cursor`/`nextCursor`) pagination, and the global
// ValidationPipe(whitelist:true) SILENTLY STRIPS the obsolete `page` field
// — so the old offset loop got the SAME first page every time (with
// hasNextPage stuck true) and dropped every child beyond the first page.
const MAX_PAGES = 50;
let cursor: string | undefined;
let allItems: any[] = [];
let truncated = false;
for (let i = 0; i < MAX_PAGES; i++) {
// limit: 100 is the server-side Max; cuts request count 5x vs the default 20.
const payload: Record<string, any> = { spaceId, limit: 100 };
// Only send pageId when scoping to a page's children; omit it for roots.
if (pageId) payload.pageId = pageId;
if (cursor) payload.cursor = cursor;
const data = (await this.client.post("/pages/sidebar-pages", payload)).data
?.data;
allItems = allItems.concat(data?.items ?? []);
// Advance strictly via the server-issued cursor; a missing/repeated cursor
// means the protocol drifted again — stop instead of looping on page one.
const next = data?.meta?.hasNextPage ? data?.meta?.nextCursor : null;
if (!next || next === cursor) break;
cursor = next;
// Reaching the ceiling with more pages still available means the child
// list is truncated (mirrors paginateAll).
if (i === MAX_PAGES - 1) truncated = true;
}
// Warn on real truncation (ceiling hit while the server still had pages) so
// the caller is not silently handed an incomplete child list.
if (truncated) {
console.warn(
`listSidebarPages: children of "${pageId ?? spaceId}" truncated at the ${MAX_PAGES}-page cap; more pages exist on the server`,
);
}
return allItems;
}
/**
* Enumerate EVERY page in a space (or in a subtree, when rootPageId is given).
*
* Primary path (fork server): a SINGLE `POST /pages/tree` returns the whole
* space (or a subtree) as a flat, permission-filtered list in one request, in
* the exact node shape buildPageTree consumes. This replaces the old
* per-node BFS, which issued N sidebar requests and after the server moved
* to cursor pagination silently lost every child past the first sidebar
* page (the obsolete `page` param was stripped by ValidationPipe).
*
* The subtree variant (rootPageId given) INCLUDES the root node itself
* (getPageAndDescendants seeds with id = rootPageId), unlike the old BFS
* which started from the root's children.
*
* Fallback path (stdio mode may target STOCK upstream Docmost, which lacks
* `/pages/tree`): on a 404/405 it falls back to the cursor-based BFS below,
* walking direct children via the fixed cursor listSidebarPages. Safeguards:
* a `visited` Set of page ids prevents re-processing a node (cycles /
* duplicate references), and a hard node cap bounds pathological trees so the
* walk always terminates.
*
* Returns `{ pages, truncated }`. `truncated` is true ONLY when the fallback
* BFS stopped at its MAX_NODES cap the primary /pages/tree path is uncapped
* and always returns the complete set, so it never reports truncation.
*/
protected async enumerateSpacePages(
spaceId: string,
rootPageId?: string,
): Promise<{ pages: any[]; truncated: boolean }> {
await this.ensureAuthenticated();
// Single request replaces the whole BFS: /pages/tree returns the full
// permission-filtered flat page set of a space (or a subtree) at once. This
// path is uncapped, so it is never truncated.
const payload = rootPageId ? { pageId: rootPageId } : { spaceId };
try {
const response = await this.client.post("/pages/tree", payload);
const pages = (response.data?.data ?? response.data)?.items ?? [];
return { pages, truncated: false };
} catch (e: any) {
// Only fall back when the endpoint is absent (stock upstream Docmost);
// any other error is a genuine failure and must propagate.
if (
!axios.isAxiosError(e) ||
(e.response?.status !== 404 && e.response?.status !== 405)
) {
throw e;
}
}
// Fallback: cursor-based breadth-first walk via listSidebarPages.
const MAX_NODES = 10000;
const result: any[] = [];
const visited = new Set<string>();
// Seed with the root node itself when scoping to a subtree, so its own
// comments aren't dropped: the primary /pages/tree seeds
// getPageAndDescendants with id = rootPageId (root included), but
// listSidebarPages(spaceId, rootPageId) returns only the root's CHILDREN.
// The `visited` set below prevents a double-add if the root also appears
// among the children. getPageRaw returns a page whose id/title/spaceId are
// exactly what buildPageTree and checkNewComments consume.
if (rootPageId) {
try {
const root = await this.getPageRaw(rootPageId);
if (root?.id) {
result.push(root);
visited.add(root.id);
}
} catch {
// Non-fatal: if the root can't be read, fall through to children-only.
}
}
// Seed the queue with the starting level (subtree children or roots).
const queue: any[] = await this.listSidebarPages(spaceId, rootPageId);
while (queue.length > 0 && result.length < MAX_NODES) {
const node = queue.shift();
if (!node || typeof node !== "object" || !node.id) continue;
// Skip already-seen ids to guard against cycles / duplicate references.
if (visited.has(node.id)) continue;
visited.add(node.id);
result.push(node);
if (node.hasChildren) {
try {
const children = await this.listSidebarPages(spaceId, node.id);
for (const child of children) queue.push(child);
} catch (e: any) {
// A failure fetching one node's children must not abort the whole
// walk: skip this branch and keep enumerating the rest.
}
}
}
// Truncated only when the cap was hit with the queue still non-empty (real
// truncation, not a natural end at exactly MAX_NODES).
return {
pages: result,
truncated: result.length >= MAX_NODES && queue.length > 0,
};
}
/** Raw page info including the ProseMirror JSON content and slugId. */
async getPage(pageId: string) {
await this.ensureAuthenticated();
const resultData = await this.getPageRaw(pageId);
// Agent read: hide resolved-comment anchors so the agent sees only active
// discussions. Active anchors are kept. (The lossless exportPageMarkdown
// round-trip deliberately does NOT pass this flag — resolved anchors there
// must be preserved.)
let content = resultData.content
? convertProseMirrorToMarkdown(resultData.content, {
dropResolvedCommentAnchors: true,
})
: "";
// Always fetch subpages to provide context to the agent
let subpages: any[] = [];
try {
// `pageId` may be a slugId, but the sidebar-pages endpoint requires the
// UUID; `resultData.id` holds the resolved UUID returned by getPageRaw.
subpages = await this.listSidebarPages(resultData.spaceId, resultData.id);
} catch (e: any) {
console.warn("Failed to fetch subpages:", e);
}
// Resolve subpages if the placeholder exists
if (content && content.includes("{{SUBPAGES}}")) {
if (subpages && subpages.length > 0) {
const list = subpages
.map((p: any) => `- [${p.title}](page:${p.id})`)
.join("\n");
content = content.replace("{{SUBPAGES}}", `### Subpages\n${list}`);
} else {
content = content.replace("{{SUBPAGES}}", "");
}
}
return {
data: filterPage(resultData, content, subpages),
success: true,
};
}
/** Page info + raw ProseMirror JSON content (lossless representation). */
async getPageJson(pageId: string) {
const data = await this.getPageRaw(pageId);
return {
id: data.id,
slugId: data.slugId,
title: data.title,
parentPageId: data.parentPageId,
spaceId: data.spaceId,
updatedAt: data.updatedAt,
content: data.content || { type: "doc", content: [] },
};
}
/**
* Fetch an INTERNAL Docmost file (authed loopback) for sandbox mirroring.
* `src` is normalized to `/api/files/<id>/<file>`; `this.client.baseURL`
* already ends in `/api`, so we strip the leading `/api` and request the
* relative path with the client's Authorization header. Returns the raw bytes
* and the response Content-Type (mime), defaulting to octet-stream.
*
* The fetch is size-bounded (hard 64 MiB ceiling) purely to protect memory;
* the authoritative per-blob cap is enforced by the sandbox `put`. The path is
* resolved via resolveInternalFilePath, which REJECTS (throws) any traversal
* or percent-encoded src that would let an attacker-controlled `attrs.src`
* escape `/api/files/` and reach another internal endpoint (SSRF). That throw
* happens before this.client.get, so a malicious src is counted as a failed
* mirror it never reaches the network.
*/
/**
* Compact outline of a page's top-level blocks (no full document body).
* Cheap way to locate sections/tables and grab block ids before drilling in
* with getNode / patchNode / insertNode.
*/
async getOutline(pageId: string) {
await this.ensureAuthenticated();
const data = await this.getPageRaw(pageId);
return {
pageId,
slugId: data.slugId,
title: data.title,
outline: buildOutline(data.content ?? { type: "doc", content: [] }),
};
}
/**
* Fetch a single block for editing by reference: a block id (headings/
* paragraphs/callouts/images), or `#<index>` to select a top-level block by its
* outline index (the only way to reach tables/rows/cells, which carry no id).
*
* `format` (#413):
* - `"markdown"` (DEFAULT): serialize the block via the canonical converter
* (`{type:"doc",content:[node]}` -> `convertProseMirrorToMarkdown`) a read
* "for editing": pair it with `patchNode({markdown})` to rewrite the block.
* Comment anchors (`<span data-comment-id>`, INCLUDING resolved ones) are
* NOT stripped here (unlike getPage): losing them on write-back would
* orphan the thread. Returns `{ ..., format:"markdown", markdown }`.
* - `"json"`: return the raw ProseMirror subtree as-is (lossless; the previous
* default). Returns `{ ..., format:"json", node }`.
*
* AUTO fallback: a type that cannot be a document top-level child
* (tableRow/tableCell/tableHeader, addressed by `#<index>`) is NOT expressible
* as a standalone markdown document, so a `"markdown"` request for such a node
* transparently falls back to JSON with an explicit `format:"json"` field. The
* check derives from the schema's `doc` contentMatch, so it tracks the schema.
*/
async getNode(
pageId: string,
nodeId: string,
format: "markdown" | "json" = "markdown",
) {
await this.ensureAuthenticated();
const data = await this.getPageRaw(pageId);
const hit = getNodeByRef(
data.content ?? { type: "doc", content: [] },
nodeId,
);
if (!hit) {
throw new Error(
`getNode: no node found for "${nodeId}" on page ${pageId} (use a block id from getOutline, or "#<index>" for a top-level block such as a table)`,
);
}
// JSON requested (or a non-top-level type that markdown cannot represent as a
// standalone document): return the subtree verbatim.
if (format === "json" || !canBeDocChild(hit.type)) {
return {
pageId,
ref: nodeId,
path: hit.path,
type: hit.type,
format: "json" as const,
node: hit.node,
};
}
// Markdown: wrap the node as a one-block doc and run the canonical converter.
// Comment anchors are DELIBERATELY preserved (converter default) so a
// getNode(markdown) -> edit -> patchNode(markdown) round trip does not orphan
// a comment thread; this differs from getPage, which strips them.
const markdown = convertProseMirrorToMarkdown({
type: "doc",
content: [hit.node],
});
return {
pageId,
ref: nodeId,
path: hit.path,
type: hit.type,
format: "markdown" as const,
markdown,
};
}
/**
* Find every occurrence of `query` on a page IN MEMORY, over the plain text of
* each text container (reusing the same `getPageRaw` fetch as the other read
* tools) no server search endpoint, no whole-document round-trip through the
* model. Returns `{ total, truncated, matches }`; each match carries a ref for
* getNode/patchNode (the `#<index>` form resolves with getNode but NOT
* patchNode see SearchMatch.nodeId), plus the top-level block index and a
* short context window used to build a unique text `selection` for
* createComment (createComment has no nodeId param). The pure engine
* (`searchInDoc`) owns the traversal, glue, the RE2 ReDoS-safe regex engine
* and the empty-query / invalid-or-unsupported-regex errors.
*/
async searchInPage(pageId: string, query: string, opts: SearchOptions = {}) {
await this.ensureAuthenticated();
const data = await this.getPageRaw(pageId);
const result = searchInDoc(
data.content ?? { type: "doc", content: [] },
query,
opts,
);
return { pageId, query, ...result };
}
/**
* Read a table as a matrix. `tableRef` is `#<index>` (from getOutline) or a
* block id of any node inside the table. Returns the cell texts plus a
* parallel cellIds matrix (each cell's first paragraph id, or null) so a
* caller can patchNode a cell for rich-formatted edits. Throws when no table
* resolves for the reference.
*/
async getTable(pageId: string, tableRef: string) {
await this.ensureAuthenticated();
const data = await this.getPageRaw(pageId);
const t = readTable(data.content ?? { type: "doc", content: [] }, tableRef);
if (!t) {
throw new Error(
`tableGet: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
);
}
return {
pageId,
table: tableRef,
rows: t.rows,
cols: t.cols,
path: t.path,
cells: t.cells,
cellIds: t.cellIds,
};
}
/**
* Insert a row of plain-text cells into a table on the LIVE collab document.
* `tableRef` is `#<index>` or a block id inside the target table. `cells` is
* padded to the table's column count (more cells than columns throws); `index`
* is a 0-based insert position (omit/out-of-range to append). Throws when no
* table resolves for the reference.
*/
async search(
query: string,
spaceId?: string,
limit?: number,
opts: { parentPageId?: string; titleOnly?: boolean } = {},
) {
await this.ensureAuthenticated();
// Opt into the #443 agent-lookup mode: `substring: true` turns on the hybrid
// substring + FTS branch that returns path + snippet + score. A stock
// upstream server strips these unknown DTO fields (whitelist:true) and
// silently degrades to plain FTS — see the tool-registration comment.
const payload: Record<string, any> = {
query,
spaceId,
substring: true,
};
if (opts.parentPageId) payload.parentPageId = opts.parentPageId;
if (opts.titleOnly) payload.titleOnly = true;
// Clamp an optional caller-supplied limit into the lookup range (1..50)
// before forwarding; omit it when not provided so the server default applies.
if (limit !== undefined) {
payload.limit = Math.max(1, Math.min(50, limit));
}
const response = await this.client.post("/search", payload);
// Normalize both response shapes: bare array and paginated { items: [...] }
const data = response.data?.data;
const items = Array.isArray(data) ? data : data?.items || [];
const filteredItems = items.map((item: any) => filterSearchResult(item));
return {
items: filteredItems,
success: response.data?.success || false,
};
}
}
return ReadMixin;
}
+225
View File
@@ -0,0 +1,225 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import {
collectInternalFileNodes,
normalizeFileUrl,
resolveInternalFilePath,
} from "../lib/internal-file-urls.js";
// Public method surface of StashMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IStashMixin` fails to compile on drift.
export interface IStashMixin {
stashPage(pageId: string): Promise<{ uri: string; sha256: string; size: number; images: { mirrored: number; failed: number }; }>;
}
export function StashMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IStashMixin> & TBase {
abstract class StashMixin extends Base implements IStashMixin {
/**
* Fetch an INTERNAL Docmost file (authed loopback) for sandbox mirroring.
* `src` is normalized to `/api/files/<id>/<file>`; `this.client.baseURL`
* already ends in `/api`, so we strip the leading `/api` and request the
* relative path with the client's Authorization header. Returns the raw bytes
* and the response Content-Type (mime), defaulting to octet-stream.
*
* The fetch is size-bounded (hard 64 MiB ceiling) purely to protect memory;
* the authoritative per-blob cap is enforced by the sandbox `put`. The path is
* resolved via resolveInternalFilePath, which REJECTS (throws) any traversal
* or percent-encoded src that would let an attacker-controlled `attrs.src`
* escape `/api/files/` and reach another internal endpoint (SSRF). That throw
* happens before this.client.get, so a malicious src is counted as a failed
* mirror it never reaches the network.
*/
protected async fetchInternalFile(
src: string,
): Promise<{ buffer: Buffer; mime: string }> {
const HARD_CEILING = 64 * 1024 * 1024; // 64 MiB memory guard
const relPath = resolveInternalFilePath(src);
const response = await this.client.get(relPath, {
responseType: "arraybuffer",
timeout: 30000,
maxContentLength: HARD_CEILING,
maxBodyLength: HARD_CEILING,
});
const buffer = Buffer.from(response.data);
if (buffer.length === 0) {
throw new Error(`Empty file response from "${src}"`);
}
const rawCt = response.headers?.["content-type"];
const mime =
typeof rawCt === "string" && rawCt.length > 0
? rawCt.split(";")[0].trim().toLowerCase()
: "application/octet-stream";
return { buffer, mime };
}
/**
* Stash a page's full content into the in-RAM blob sandbox and return ONLY a
* short anonymous URL the body never enters the model context (this is the
* whole point: ~30KB+ ProseMirror docs blow the model context if passed as a
* tool argument). Every INTERNAL file/image src (the type-agnostic criterion,
* so drawio/excalidraw/video/file nodes are covered too) is mirrored into the
* sandbox and its `src` rewritten to the sandbox URL, so an external consumer
* can fetch the images anonymously. External http(s) srcs are left untouched.
*
* Blobs live in RAM with a short TTL and are cleared on restart consume the
* URLs within the TTL and one uptime. A failed image fetch never aborts the
* doc: the original src is kept and the failure counted.
*
* Returns { uri, sha256, size, images:{mirrored, failed} }. `uri` and `sha256`
* are for the document blob; `sha256` is also the blob's ETag (integrity).
*/
async stashPage(pageId: string): Promise<{
uri: string;
sha256: string;
size: number;
images: { mirrored: number; failed: number };
}> {
if (!this.sandboxPut) {
throw new Error(
"stashPage is unavailable: the blob sandbox is not configured on this server",
);
}
await this.ensureAuthenticated();
// Stash the SAME shape getPageJson returns (id/title/.../content), with a
// deep clone so the rewrite never mutates anything shared.
const pageJson = await this.getPageJson(pageId);
const cloned: any = structuredClone(pageJson);
// Group internal-file nodes by normalized src so each unique resource is
// fetched + stored ONCE (dedup), and every node sharing that src points at
// the one sandbox blob. Capture each node's ORIGINAL raw src per-node:
// dedup groups nodes whose normalized src is equal even when their raw srcs
// differ (e.g. `/api/files/...` vs the bare `/files/...`), so on a revert we
// must restore each node's own original value, not the group key.
const bySrc = new Map<string, Array<{ node: any; origSrc: string }>>();
for (const node of collectInternalFileNodes(cloned.content)) {
const origSrc = String(node.attrs.src);
const src = normalizeFileUrl(origSrc);
const entry = { node, origSrc };
const group = bySrc.get(src);
if (group) group.push(entry);
else bySrc.set(src, [entry]);
}
let mirrored = 0;
let failed = 0;
// Record every successful mirror so it can be (a) reverted if its blob gets
// FIFO-evicted by a LATER put in this same stash, and (b) freed if the final
// doc put throws.
const mirrors: Array<{
uri: string;
entries: Array<{ node: any; origSrc: string }>;
}> = [];
const MAX_CONCURRENCY = 5;
const groups = [...bySrc.entries()];
for (let i = 0; i < groups.length; i += MAX_CONCURRENCY) {
const batch = groups.slice(i, i + MAX_CONCURRENCY);
await Promise.all(
batch.map(async ([src, entries]) => {
try {
const { buffer, mime } = await this.fetchInternalFile(src);
// put may throw if the blob exceeds the per-blob/total caps.
const stored = this.sandboxPut!(buffer, mime);
for (const entry of entries) entry.node.attrs.src = stored.uri;
mirrors.push({ uri: stored.uri, entries });
mirrored++;
} catch (err) {
// One bad/oversized image (or a rejected traversal src) must not
// abort the document. Logged unconditionally (never the blob body),
// matching the package's ungated console.warn convention.
failed++;
console.warn(
`stashPage: failed to mirror "${src}": ${
err instanceof Error ? err.message : String(err)
}`,
);
}
}),
);
}
// Revert one mirror's nodes to their original internal srcs and re-count it
// as failed (its blob was FIFO-evicted before the doc could reference it
// safely).
const revertMirror = (mirror: {
uri: string;
entries: Array<{ node: any; origSrc: string }>;
}) => {
for (const entry of mirror.entries) entry.node.attrs.src = entry.origSrc;
mirrored--;
failed++;
console.warn(
`stashPage: mirrored blob ${mirror.uri} was evicted before the doc ` +
`could safely reference it; reverted its src and counted it as failed`,
);
};
// Pre-put reconciliation: an image put earlier in THIS stash can FIFO-evict
// an even-earlier image of the same stash. Drop those from the live set
// first so the first serialized doc is already mostly correct.
let liveMirrors = mirrors;
if (this.sandboxHas) {
liveMirrors = [];
for (const mirror of mirrors) {
if (this.sandboxHas(mirror.uri)) liveMirrors.push(mirror);
else revertMirror(mirror);
}
}
// Put the document, then reconcile against eviction caused by the doc put
// ITSELF (the doc is newest, FIFO drops oldest = this stash's images). Each
// iteration reverts >=1 mirror, so the loop terminates (worst case: all
// images reverted and the doc references no sandbox image URLs).
let stored: { uri: string; sha256: string; size: number };
for (;;) {
const docBuf = Buffer.from(JSON.stringify(cloned), "utf8");
let docStored: { uri: string; sha256: string; size: number };
try {
docStored = this.sandboxPut(docBuf, "application/json");
} catch (err) {
// The doc put failed (e.g. doc exceeds the cap). Free this op's image
// blobs instead of leaking them in RAM for the whole TTL, then
// re-throw.
if (this.sandboxEvict) {
for (const mirror of liveMirrors) this.sandboxEvict(mirror.uri);
}
throw err;
}
if (!this.sandboxHas) {
stored = docStored;
break;
}
const evictedNow = liveMirrors.filter((m) => !this.sandboxHas!(m.uri));
if (evictedNow.length === 0) {
stored = docStored;
break;
}
// The doc we just stored references now-dead blobs. Revert those nodes,
// drop the stale doc blob, and loop to re-serialize + re-put the
// corrected doc.
for (const mirror of evictedNow) revertMirror(mirror);
liveMirrors = liveMirrors.filter((m) => this.sandboxHas!(m.uri));
if (this.sandboxEvict) this.sandboxEvict(docStored.uri);
}
return {
uri: stored.uri,
sha256: stored.sha256,
size: stored.size,
images: { mirrored, failed },
};
}
/**
* Compact outline of a page's top-level blocks (no full document body).
* Cheap way to locate sections/tables and grab block ids before drilling in
* with getNode / patchNode / insertNode.
*/
}
return StashMixin;
}
+291
View File
@@ -0,0 +1,291 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import { withPageLock, isUuid } from "../lib/page-lock.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Public method surface of TablesMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements ITablesMixin` fails to compile on drift.
export interface ITablesMixin {
insertFootnote(pageId: string, anchorText: string, text: string): any;
tableInsertRow(pageId: string, tableRef: string, cells: string[], index?: number): any;
tableDeleteRow(pageId: string, tableRef: string, index: number): any;
tableUpdateCell(pageId: string, tableRef: string, row: number, col: number, text: string): any;
}
export function TablesMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & ITablesMixin> & TBase {
abstract class TablesMixin extends Base implements ITablesMixin {
/**
* AUTHOR-INLINE footnote insertion. The agent supplies only WHERE
* (`anchorText`, a snippet of body text to attach the marker after) and WHAT
* (`text`, the footnote content as markdown). Numbering and the bottom
* `footnotesList` are derived deterministically server-side
* (`insertInlineFootnote` -> `canonicalizeFootnotes`): the agent never sees,
* assigns, or edits a footnote number or the list, so it CANNOT desync.
*
* Content DEDUP: when an existing definition has the same content, its id is
* reused (one number, one definition, several references). The write is atomic
* via `mutatePageContent` (single-writer, page-locked); if the anchor text is
* not found the transform aborts with a clear error and no write happens.
*/
async insertFootnote(pageId: string, anchorText: string, text: string) {
await this.ensureAuthenticated();
if (!anchorText || !anchorText.trim()) {
throw new Error("insertFootnote: anchorText is required");
}
if (text == null || `${text}`.trim() === "") {
throw new Error("insertFootnote: text is required");
}
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
let result: { footnoteId: string; reused: boolean } | null = null;
const mutation = await this.mutatePage(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc: any) => {
const r = insertInlineFootnote(liveDoc, { anchorText, text });
if (!r.inserted) {
// Abort the page-locked write by throwing: mutatePageContent does not
// persist when the transform throws, so a missing anchor leaves the
// page untouched (no partial write).
throw new Error(
`insertFootnote: anchor text not found: ${JSON.stringify(
anchorText.slice(0, 80),
)}`,
);
}
result = { footnoteId: r.footnoteId, reused: r.reused };
return r.doc;
},
);
// The not-found path throws inside the transform (aborting mutatePage), so by
// here `result` is always set.
const r = result!;
return {
success: true,
modified: true,
pageId,
footnoteId: r.footnoteId,
reused: r.reused,
message: r.reused
? "Footnote inserted (reused an existing same-content definition)."
: "Footnote inserted.",
verify: mutation.verify,
};
}
/**
* Page-locked write seam over collaboration.mutatePageContent. Production just
* delegates; it exists as an overridable method so the insertFootnote wrapper
* (transform abort-on-not-found + response shaping) can be unit-tested without
* standing up a live Hocuspocus collab socket.
*
* SELF-RESOLVES the pageId to the canonical UUID (issue #449, "resolve-then-
* lock"): every write must lock and key its CollabSession by the UUID, never a
* raw slugId (#260). resolvePageId is cached/idempotent, so a caller that
* already resolved pays no extra round-trip; centralizing it here means a
* caller that reaches this seam with a raw slugId still locks correctly instead
* of silently splitting the mutex key. withPageLock also asserts the key is a
* UUID as a hard backstop.
*/
/**
* Insert a row of plain-text cells into a table on the LIVE collab document.
* `tableRef` is `#<index>` or a block id inside the target table. `cells` is
* padded to the table's column count (more cells than columns throws); `index`
* is a 0-based insert position (omit/out-of-range to append). Throws when no
* table resolves for the reference.
*/
async tableInsertRow(
pageId: string,
tableRef: string,
cells: string[],
index?: number,
) {
await this.ensureAuthenticated();
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
// Track insertion in an outer var, reset per-transform, so a collab retry
// recomputes it cleanly (mirrors insertNode's pattern).
let inserted = false;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
inserted = false;
const { doc: nd, inserted: ins } = insertTableRow(
liveDoc,
tableRef,
cells,
index,
);
inserted = ins;
if (!inserted) return null; // table not found -> skip the write entirely
return nd;
},
);
if (!inserted) {
throw new Error(
`tableInsertRow: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
);
}
return {
success: true,
table: tableRef,
inserted: true,
verify: mutation.verify,
};
}
/**
* Delete the row at 0-based `index` from a table on the LIVE collab document.
* `tableRef` is `#<index>` or a block id inside the target table. The helper's
* out-of-range and last-row errors propagate; a missing table throws here.
*/
async tableDeleteRow(pageId: string, tableRef: string, index: number) {
await this.ensureAuthenticated();
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
let deleted = false;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
deleted = false;
const { doc: nd, deleted: del } = deleteTableRow(
liveDoc,
tableRef,
index,
);
deleted = del;
if (!deleted) return null; // table not found -> skip the write entirely
return nd;
},
);
if (!deleted) {
throw new Error(
`tableDeleteRow: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
);
}
return {
success: true,
table: tableRef,
deleted: true,
verify: mutation.verify,
};
}
/**
* Set the plain-text content of cell `[row, col]` (0-based) in a table on the
* LIVE collab document, replacing the cell's content with a single text
* paragraph (the cell's first-paragraph id is preserved). `tableRef` is
* `#<index>` or a block id inside the target table. The helper's out-of-range
* error propagates; a missing table throws here.
*/
async tableUpdateCell(
pageId: string,
tableRef: string,
row: number,
col: number,
text: string,
) {
await this.ensureAuthenticated();
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
let updated = false;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
updated = false;
const { doc: nd, updated: upd } = updateTableCell(
liveDoc,
tableRef,
row,
col,
text,
);
updated = upd;
if (!updated) return null; // table not found -> skip the write entirely
return nd;
},
);
if (!updated) {
throw new Error(
`tableUpdateCell: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
);
}
return {
success: true,
table: tableRef,
row,
col,
verify: mutation.verify,
};
}
/**
* Create a new page with title and content.
* Uses the /pages/import workaround (the only endpoint accepting content),
* then moves the page and restores the exact title: the import endpoint
* derives the title from the FILENAME and replaces spaces with
* underscores, so we explicitly re-set it via /pages/update afterwards.
*/
}
return TablesMixin;
}
+232
View File
@@ -0,0 +1,232 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import { diffDocs, summarizeChange } from "../lib/diff.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
import { normalizeAndMergeFootnotes } from "../lib/footnote-normalize-merge.js";
import vm from "node:vm";
// Public method surface of TransformsMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements ITransformsMixin` fails to compile on drift.
export interface ITransformsMixin {
transformPage(pageId: string, transformJs: string, opts?: { dryRun?: boolean; deleteComments?: boolean }): any;
}
export function TransformsMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & ITransformsMixin> & TBase {
abstract class TransformsMixin extends Base implements ITransformsMixin {
/**
* Edit a page by running an arbitrary user-supplied JS transform against the
* live document, with a diff preview + page-history safety net.
*
* The transform string is evaluated as `(doc, ctx) => doc` inside a node:vm
* sandbox: it gets ONLY `{ doc, ctx, structuredClone, console }` as globals,
* a 5s timeout, and NO access to require/process/fs/network. It must return a
* `{ type: "doc" }` node, which is validated structurally before any write.
*
* `ctx` exposes:
* - comments: the page's comments (fetched before the live read);
* - log: an array the transform can push diagnostics to (via console.log);
* - consume(id): mark a comment id as consumed (for deleteComments);
* - helpers: the transforms.ts primitives + commentsToFootnotes.
*
* Footnote convention used by the helpers: footnote markers are plain "[N]"
* text in the body, and the notes are an orderedList under a heading whose
* text is "Примечания переводчика".
*
* dryRun (default true): read the page's current content, run the transform,
* and return `{ pushed:false, diff, log }` WITHOUT opening the collab socket.
* Otherwise the transform runs atomically inside mutatePageContent, optionally
* deletes consumed comments, and returns the new historyId + diff + log.
*/
async transformPage(
pageId: string,
transformJs: string,
opts: { dryRun?: boolean; deleteComments?: boolean } = {},
) {
const dryRun = opts.dryRun ?? true;
const deleteComments = opts.deleteComments ?? false;
await this.ensureAuthenticated();
// Full feed (incl. resolved): a page transform (e.g. comments -> footnotes)
// must operate on every comment, so it opts into the unfiltered feed.
const comments = (await this.listComments(pageId, true)).items;
// ctx handed to the sandbox. consume() records ids; helpers are the pure
// transform primitives. log is captured from console.log inside the sandbox.
const ctx = {
comments,
log: [] as string[],
consumed: new Set<string>(),
consume(id: string) {
this.consumed.add(id);
},
helpers: {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
},
};
// Captured oldDoc / newDoc for the diff (set inside runTransform).
let oldDoc: any;
let newDoc: any;
// SYNCHRONOUS transform runner — safe to call inside mutatePageContent's
// onSynced (no await between the live read and the write).
const runTransform = (liveDoc: any): any => {
oldDoc = structuredClone(liveDoc);
const sandbox: Record<string, any> = {
doc: structuredClone(liveDoc),
ctx,
structuredClone,
console: {
log: (...a: any[]) => ctx.log.push(a.map((x) => String(x)).join(" ")),
},
};
// Wrap the provided string in parentheses so both an expression-arrow
// (`(doc, ctx) => {...}`) and a parenthesized function work. Run it in a
// fresh context with no require/process/module so the transform cannot
// touch fs/network/process. 5s wall-clock timeout.
let fn: any;
try {
fn = vm.runInNewContext("(" + transformJs + ")", sandbox, {
timeout: 5000,
});
} catch (e: any) {
throw new Error(`transform did not compile: ${e?.message ?? e}`);
}
if (typeof fn !== "function") {
throw new Error(
"transform must evaluate to a function (doc, ctx) => doc",
);
}
const raw = vm.runInNewContext(
"f(d, c)",
{ f: fn, d: sandbox.doc, c: ctx },
{ timeout: 5000 },
);
if (
!raw ||
typeof raw !== "object" ||
raw.type !== "doc" ||
!Array.isArray(raw.content)
) {
throw new Error(
'transform must return a ProseMirror doc node ({ type:"doc", content:[...] })',
);
}
// Validate the RAW transform output FIRST (structure — including the
// MAX_DEPTH guard — and URLs), mirroring updatePageJson. The canonicalizer
// recurses without a depth limiter, so validating after it would turn a
// too-deep doc into an opaque "Maximum call stack size exceeded" instead of
// the intended "nesting exceeds the maximum depth" error.
this.validateDocStructure(raw);
this.validateDocUrls(raw);
// Auto-canonicalize footnotes after the transform (idempotent): no write
// path can leave footnotes out of order / orphaned / in a raw `[^id]`
// block. In a dryRun preview this may surface footnote edits the script
// author did not write (the canonicalizer tidied them) — that is expected.
// #419: normalize + merge glyph-forked definitions before canonicalizing.
const result = canonicalizeFootnotes(normalizeAndMergeFootnotes(raw));
newDoc = result;
return result;
};
if (dryRun) {
// Preview only: run against the current REST snapshot, never open the
// socket. oldDoc/newDoc are captured by runTransform.
const raw = await this.getPageRaw(pageId);
const current = raw.content || { type: "doc", content: [] };
runTransform(current);
// Run an independent Yjs-encodability check (same sanitize + schema as the
// apply path), so the preview fails with the same descriptive error when
// the doc is not encodable instead of returning a misleadingly-green diff.
assertYjsEncodable(newDoc);
return {
pushed: false,
diff: diffDocs(oldDoc, newDoc),
log: ctx.log,
};
}
// Apply atomically against the live doc.
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
runTransform,
);
// Optionally delete consumed comments (best-effort; a delete failure must
// not undo the successful write).
const deletedComments: string[] = [];
if (deleteComments) {
for (const id of ctx.consumed) {
try {
await this.deleteComment(id);
deletedComments.push(id);
} catch (e) {
if (process.env.DEBUG) {
console.error(`transform: failed to delete comment ${id}:`, e);
}
}
}
}
// Fetch the newest historyId (Docmost snapshots on the write above).
let historyId: string | null = null;
try {
const hist = await this.listPageHistory(pageId);
historyId = hist.items?.[0]?.id ?? null;
} catch (e) {
if (process.env.DEBUG) {
console.error("transform: failed to fetch history id:", e);
}
}
return {
pushed: true,
historyId,
diff: diffDocs(oldDoc, newDoc),
deletedComments,
log: ctx.log,
verify: mutation.verify,
};
}
}
return TransformsMixin;
}
+4 -7
View File
@@ -57,17 +57,14 @@ export const DEFAULT_COMMENT_SIGNAL_DEBOUNCE_MS = 20_000;
/** /**
* Tools whose OWN result must NOT carry the signal it would be tautological * Tools whose OWN result must NOT carry the signal it would be tautological
* (the agent is already looking at comments) and noisy. Listed in BOTH the * (the agent is already looking at comments) and noisy. Since issue #412 both
* standalone MCP snake_case names AND the in-app camelCase keys so a single set * the standalone MCP surface and the in-app agent use the same camelCase tool
* covers both surfaces (the signal text itself uses the camelCase `listComments` * names, so a single set of camelCase names covers both surfaces. `getComment`
* per roadmap #412). `getComment` (single fetch) is intentionally NOT excluded. * (single fetch) is intentionally NOT excluded.
*/ */
export const COMMENT_SIGNAL_EXCLUDED_TOOLS: ReadonlySet<string> = new Set([ export const COMMENT_SIGNAL_EXCLUDED_TOOLS: ReadonlySet<string> = new Set([
"list_comments",
"listComments", "listComments",
"check_new_comments",
"checkNewComments", "checkNewComments",
"create_comment",
"createComment", "createComment",
]); ]);
+63 -30
View File
@@ -58,7 +58,7 @@ export type {
CommentSignalTrackerOptions, CommentSignalTrackerOptions,
} from "./comment-signal.js"; } from "./comment-signal.js";
// Re-export the pure, no-network draw.io helpers (#424) so the in-app AI-SDK // Re-export the pure, no-network draw.io helpers (#424) so the in-app AI-SDK
// service can wire drawio_shapes / drawio_guide off the loaded module. These are // service can wire drawioShapes / drawioGuide off the loaded module. These are
// NOT client methods (no page/backend hit) — the in-app handler calls them // NOT client methods (no page/backend hit) — the in-app handler calls them
// directly, mirroring how the standalone MCP server wires them here. // directly, mirroring how the standalone MCP server wires them here.
export { searchShapes } from "./lib/drawio-shapes.js"; export { searchShapes } from "./lib/drawio-shapes.js";
@@ -89,7 +89,7 @@ const VERSION = packageJson.version;
// (SHARED_TOOL_SPECS + INLINE_MCP_INVENTORY), so it can no longer drift out of // (SHARED_TOOL_SPECS + INLINE_MCP_INVENTORY), so it can no longer drift out of
// sync with the registered tools. Re-exported here (its old home) so existing // sync with the registered tools. Re-exported here (its old home) so existing
// importers are unaffected; the composition lives in server-instructions.ts. // importers are unaffected; the composition lives in server-instructions.ts.
// The drawio_shapes / drawio_guide tools (#424) stay in SHARED_TOOL_SPECS (so the // The drawioShapes / drawioGuide tools (#424) stay in SHARED_TOOL_SPECS (so the
// generated <tool_inventory> picks them up from their catalogLine automatically) // generated <tool_inventory> picks them up from their catalogLine automatically)
// but are flagged `inlineBothHosts` and registered inline below (their pure // but are flagged `inlineBothHosts` and registered inline below (their pure
// helpers can't cross into tool-specs.ts); only the hand-written routing prose in // helpers can't cross into tool-specs.ts); only the hand-written routing prose in
@@ -286,7 +286,7 @@ export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
// the wrapping is typed loosely and cast — runtime behaviour is unchanged. // the wrapping is typed loosely and cast — runtime behaviour is unchanged.
const registerSharedFromSpec = (spec: SharedToolSpec) => { const registerSharedFromSpec = (spec: SharedToolSpec) => {
if (spec.inAppOnly) return; if (spec.inAppOnly) return;
// `inlineBothHosts` specs (drawio_shapes / drawio_guide) carry no execute — // `inlineBothHosts` specs (drawioShapes / drawioGuide) carry no execute —
// their pure helper cannot cross into the zod-agnostic tool-specs.ts, so they // their pure helper cannot cross into the zod-agnostic tool-specs.ts, so they
// are registered INLINE below (searchShapes / getGuideSection). Skip them here // are registered INLINE below (searchShapes / getGuideSection). Skip them here
// so the loop never dereferences a missing `execute`. // so the loop never dereferences a missing `execute`.
@@ -316,7 +316,7 @@ export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
} }
// --- INLINE drawio helper tools (IN the shared registry, but inlineBothHosts) --- // --- INLINE drawio helper tools (IN the shared registry, but inlineBothHosts) ---
// drawio_shapes / drawio_guide (#424) live in SHARED_TOOL_SPECS (so the shared // drawioShapes / drawioGuide (#424) live in SHARED_TOOL_SPECS (so the shared
// contract pins their name/description/schema across both hosts) but carry the // contract pins their name/description/schema across both hosts) but carry the
// `inlineBothHosts` flag and NO execute: their pure backing helpers // `inlineBothHosts` flag and NO execute: their pure backing helpers
// (searchShapes / getGuideSection) cannot be value-imported into the // (searchShapes / getGuideSection) cannot be value-imported into the
@@ -356,25 +356,25 @@ export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
// --- INLINE tools kept per-transport (NOT in the shared registry) --- // --- INLINE tools kept per-transport (NOT in the shared registry) ---
// Each stays inline for a documented reason: a snake_case/camelCase naming // Each stays inline for a documented reason: a snake_case/camelCase naming
// clash the registry convention forbids (table_get), an intentional // clash the registry convention forbids (tableGet), an intentional
// per-transport behaviour/schema divergence (search, docmost_transform), or a // per-transport behaviour/schema divergence (search, docmostTransform), or a
// tool that exists ONLY on this standalone MCP surface (update_comment, // tool that exists ONLY on this standalone MCP surface (updateComment,
// delete_comment — the in-app agent deliberately exposes no hard comment // deleteComment — the in-app agent deliberately exposes no hard comment
// edit/delete tool). // edit/delete tool).
// Tool: table_get // Tool: tableGet
// NOT in the shared registry: the MCP tool name `table_get` is noun-first while // NOT in the shared registry: the MCP tool name `tableGet` is noun-first while
// the in-app key is `getTable` (verb-first), breaking the snake_case(inAppKey) // the in-app key is `getTable` (verb-first), breaking the snake_case(inAppKey)
// convention the shared registry enforces (shared-tool-specs.contract.spec.ts). // convention the shared registry enforces (shared-tool-specs.contract.spec.ts).
// Renaming the public MCP tool would break external clients, so it stays inline. // Renaming the public MCP tool would break external clients, so it stays inline.
server.registerTool( server.registerTool(
"table_get", "tableGet",
{ {
description: description:
"Read a table as a matrix. Returns {rows, cols, cells (text[][]), " + "Read a table as a matrix. Returns {rows, cols, cells (text[][]), " +
"cellIds (paragraph id per cell, or null)}. `table` = `#<index>` from " + "cellIds (paragraph id per cell, or null)}. `table` = `#<index>` from " +
"get_outline, or any block id inside the table. Use cellIds with " + "getOutline, or any block id inside the table. Use cellIds with " +
"patch_node for rich-formatted cell edits. `cols` is the FIRST row's " + "patchNode for rich-formatted cell edits. `cols` is the FIRST row's " +
"width; ragged tables may vary per row, so use the per-row length of " + "width; ragged tables may vary per row, so use the per-row length of " +
"`cells` for each row.", "`cells` for each row.",
inputSchema: { inputSchema: {
@@ -388,9 +388,9 @@ server.registerTool(
}, },
); );
// Tool: update_comment // Tool: updateComment
server.registerTool( server.registerTool(
"update_comment", "updateComment",
{ {
description: description:
"Update an existing comment's content. Only the comment creator can " + "Update an existing comment's content. Only the comment creator can " +
@@ -409,9 +409,9 @@ server.registerTool(
}, },
); );
// Tool: delete_comment // Tool: deleteComment
server.registerTool( server.registerTool(
"delete_comment", "deleteComment",
{ {
description: description:
"Delete a comment. Only the comment creator or space admin can delete it.", "Delete a comment. Only the comment creator or space admin can delete it.",
@@ -435,41 +435,74 @@ server.registerTool(
// Tool: search // Tool: search
// INTENTIONAL per-transport divergence (not shared): the in-app `searchPages` // INTENTIONAL per-transport divergence (not shared): the in-app `searchPages`
// runs a semantic + keyword hybrid (RRF) with in-process access control and a // runs a semantic + keyword hybrid (RRF) with in-process access control and a
// different schema (limit 1-20); this transport is a plain REST full-text search // different schema; this transport is the #443 agent-lookup search — a hybrid
// (limit up to 100). Different behaviour AND schema, so kept per-layer. // substring + full-text search that also returns each hit's location (`path`)
// and a windowed `snippet`, so one call answers "where is it and what's in it".
// The in-app hybrid-RRF search is deliberately NOT touched. Different behaviour
// AND schema, so kept per-layer.
//
// STANDALONE-vs-STOCK-UPSTREAM: the client sends the opt-in `substring`/
// `parentPageId`/`titleOnly` DTO fields. A stock upstream server validates the
// DTO with `whitelist: true` and silently strips these unknown fields, so the
// request degrades gracefully to plain FTS (no path/snippet, current shape).
//
// EE/TYPESENSE DEGRADATION (#443): on an instance whose SEARCH_DRIVER is
// `typesense`, the server routes this request to the Typesense backend, which
// does NOT implement agent-lookup — the substring/path/snippet/tiering is
// ignored and the response degrades to plain Typesense FTS. The rich lookup
// shape is only produced by the native Postgres search driver.
server.registerTool( server.registerTool(
"search", "search",
{ {
description: description:
"Full-text search for pages and content across the whole workspace. " + "Find pages by a fragment of a technical string (hostnames, IPs, IDs " +
"Results are bounded by `limit` (1-100; when omitted the server applies " + "like `srv.local`, `10.0.12`, `WB-MGE-30D86B`) — one call returns each " +
"its own default).", "hit's location (`path`: ancestor titles root→parent) and a `snippet` " +
"around the first match, so you rarely need a follow-up get_page. " +
"Matches substrings literally (dots/dashes/digits are not tokenized) as " +
"well as full-text. Returns `{ pageId, title, path, snippet, score }` " +
"sorted by `score` (a per-response relevance float).",
inputSchema: { inputSchema: {
query: z.string().min(1).describe("Search query"), query: z.string().min(1).describe("Search query"),
spaceId: z
.string()
.optional()
.describe("Restrict the search to a single space"),
parentPageId: z
.string()
.optional()
.describe(
"Restrict to a page and all its descendants (the page itself included)",
),
titleOnly: z
.boolean()
.optional()
.describe("Match page titles only; skip page text"),
limit: z limit: z
.number() .number()
.int() .int()
.min(1) .min(1)
.max(100) .max(50)
.optional() .optional()
.describe("Max results to return (max 100)"), .describe("Max results to return (1-50, default 10)"),
}, },
}, },
async ({ query, limit }) => { async ({ query, spaceId, parentPageId, titleOnly, limit }) => {
// The tool exposes no spaceId filter, so pass undefined for the client's const result = await docmostClient.search(query, spaceId, limit, {
// optional spaceId parameter and forward limit into its correct slot. parentPageId,
const result = await docmostClient.search(query, undefined, limit); titleOnly,
});
return jsonContent(result); return jsonContent(result);
}, },
); );
// Tool: docmost_transform // Tool: docmostTransform
// INTENTIONAL per-transport divergence (not shared): the in-app `transformPage` // INTENTIONAL per-transport divergence (not shared): the in-app `transformPage`
// deliberately omits the `deleteComments` schema field (comment-deletion // deliberately omits the `deleteComments` schema field (comment-deletion
// guardrail) and carries a much shorter description; this transport exposes the // guardrail) and carries a much shorter description; this transport exposes the
// full helper catalogue. Different schema, so kept per-layer. // full helper catalogue. Different schema, so kept per-layer.
server.registerTool( server.registerTool(
"docmost_transform", "docmostTransform",
{ {
description: description:
"Edit a page by running an arbitrary JS transform `(doc, ctx) => doc` " + "Edit a page by running an arbitrary JS transform `(doc, ctx) => doc` " +
+8
View File
@@ -440,9 +440,16 @@ export class CollabSession {
// must stay synchronous (no await). While the JS event loop is not // must stay synchronous (no await). While the JS event loop is not
// yielded, no incoming remote update can interleave, so any already-synced // yielded, no incoming remote update can interleave, so any already-synced
// concurrent edits are preserved in liveDoc. // concurrent edits are preserved in liveDoc.
//
// INVARIANT 1 is machine-checked: the BEGIN/END markers below delimit the
// no-await window, and test/unit/no-await-critical-window.test.mjs scans
// this source and FAILS if any `await` (or `for await`/`yield`) appears
// between them. Do NOT add an await inside this block — an accidental
// async boundary here silently reopens the clobber-live-edits race (#152).
let newDoc: any; let newDoc: any;
let beforeDoc: any; let beforeDoc: any;
try { try {
// === MUTATE-CRITICAL-WINDOW: BEGIN (no await between here and END #449) ===
let liveDoc = TiptapTransformer.fromYdoc(this.ydoc, "default"); let liveDoc = TiptapTransformer.fromYdoc(this.ydoc, "default");
if ( if (
!liveDoc || !liveDoc ||
@@ -480,6 +487,7 @@ export class CollabSession {
// ids of unchanged nodes, so an open editor's cursor is not yanked to the // ids of unchanged nodes, so an open editor's cursor is not yanked to the
// end of the document on every agent write. // end of the document on every agent write.
applyDocToFragment(this.ydoc, newDoc); applyDocToFragment(this.ydoc, newDoc);
// === MUTATE-CRITICAL-WINDOW: END (#449) ===
} catch (e) { } catch (e) {
// Includes errors thrown by transform (e.g. "afterText not found", // Includes errors thrown by transform (e.g. "afterText not found",
// "text not found"): propagate them verbatim to the caller. // "text not found"): propagate them verbatim to the caller.
+3 -3
View File
@@ -87,8 +87,8 @@ global.WebSocket = WebSocket;
* bodies merged. So the import output is ALREADY in canonical footnote * bodies merged. So the import output is ALREADY in canonical footnote
* topology. * topology.
* - `canonicalizeFootnotes` runs AFTER as the mcp write-path invariant shared * - `canonicalizeFootnotes` runs AFTER as the mcp write-path invariant shared
* with every other full-document persist path (`update_page_json`, * with every other full-document persist path (`updatePageJson`,
* `docmost_transform`, `insert_footnote`, ). Because the package output is * `docmostTransform`, `insertFootnote`, ). Because the package output is
* already canonical, this layer is a no-op here (idempotent) it exists so * already canonical, this layer is a no-op here (idempotent) it exists so
* the page-write contract is enforced uniformly regardless of how the PM doc * the page-write contract is enforced uniformly regardless of how the PM doc
* was produced, not because the import needs fixing. * was produced, not because the import needs fixing.
@@ -282,7 +282,7 @@ export async function mutatePageContent(
* it was produced from markdown (ids regenerate) or edited in place * it was produced from markdown (ids regenerate) or edited in place
* (existing block ids preserved). * (existing block ids preserved).
* *
* This is an intentional full replace (used by update_page / update_page_json), * This is an intentional full replace (used by update_page / updatePageJson),
* but now runs under the per-page lock and waits for server persistence via * but now runs under the per-page lock and waits for server persistence via
* mutatePageContent. * mutatePageContent.
*/ */
+1 -1
View File
@@ -20,7 +20,7 @@
* *
* MARKDOWN-STRIP FALLBACK: when the agent copies a selection that still carries * MARKDOWN-STRIP FALLBACK: when the agent copies a selection that still carries
* inline markdown (`**bold**`, `` `code` ``, `[t](u)`), the raw locator will not * inline markdown (`**bold**`, `` `code` ``, `[t](u)`), the raw locator will not
* match the document's plain text. Exactly like edit_page_text's json-edit * match the document's plain text. Exactly like editPageText's json-edit
* fallback, we first try the verbatim selection and, ONLY if it anchors nowhere * fallback, we first try the verbatim selection and, ONLY if it anchors nowhere
* in the whole document, retry with `stripInlineMarkdown` applied. `canAnchorInDoc`, * in the whole document, retry with `stripInlineMarkdown` applied. `canAnchorInDoc`,
* `getAnchoredText` and `applyAnchorInDoc` share this decision via * `getAnchoredText` and `applyAnchorInDoc` share this decision via
+2 -2
View File
@@ -490,7 +490,7 @@ export interface VerifyReport {
/** /**
* ONLY structural integrity types whose count changed, as [before, after] * ONLY structural integrity types whose count changed, as [before, after]
* (images/links/tables/callouts). Surfaces structural mutations that touch * (images/links/tables/callouts). Surfaces structural mutations that touch
* neither text nor marks (e.g. insert_image, deleting a table) which diffDocs * neither text nor marks (e.g. insertImage, deleting a table) which diffDocs
* being TEXT-only would otherwise report as "no content change". * being TEXT-only would otherwise report as "no content change".
*/ */
structure?: Record<string, [number, number]>; structure?: Record<string, [number, number]>;
@@ -510,7 +510,7 @@ export interface VerifyReport {
* *
* The structural integrity delta (from diffDocs's `integrity` tuples) is what * The structural integrity delta (from diffDocs's `integrity` tuples) is what
* makes `changed` true for an image/table/callout/link count change that diffs * makes `changed` true for an image/table/callout/link count change that diffs
* to zero text closing a verify blind spot for insert_image, delete_node on a * to zero text closing a verify blind spot for insertImage, deleteNode on a
* table, etc. * table, etc.
*/ */
export function summarizeChange(before: any, after: any): VerifyReport { export function summarizeChange(before: any, after: any): VerifyReport {
+168
View File
@@ -0,0 +1,168 @@
// ID-based cell operations for `drawioEditCells` (issue #425, stage 3).
//
// Instead of resending the whole XML (whose diff is fragile — draw.io reorders
// attributes, and a {search,replace} text match breaks on it), the model sends
// targeted operations keyed by cell id:
//
// { op: "add", xml: "<mxCell .../>" } // append a new cell
// { op: "update", cellId: "n3", xml: "<mxCell .../>" } // replace that cell
// { op: "delete", cellId: "n5" } // + CASCADE
//
// `delete` CASCADES: it removes the cell, every descendant cell whose parent
// chain leads to it (container children), AND every edge whose source or target
// is any deleted cell. Ids are STABLE across edits so diffs stay meaningful.
//
// Operations apply to the parsed DOM of the current model; the caller re-lints
// and rebuilds the .drawio.svg through the existing #423 pipeline afterwards.
import { JSDOM } from "jsdom";
let _window: any = null;
function xmlWindow(): any {
if (!_window) _window = new JSDOM("").window;
return _window;
}
export type CellOp =
| { op: "add"; xml: string }
| { op: "update"; cellId: string; xml: string }
| { op: "delete"; cellId: string };
export class CellOpsError extends Error {
constructor(message: string) {
super(`drawioEditCells: ${message}`);
this.name = "CellOpsError";
}
}
// The mxGraph root sentinels. id="0" is the graph root; id="1" is the default
// layer that parents every real cell. A delete targeting either would cascade
// through the whole diagram body (every cell chains up to "1"), so such an op is
// rejected outright.
const SENTINEL_IDS = new Set(["0", "1"]);
/** Parse a single `<mxCell …>…</mxCell>` fragment into an element, or throw. */
function parseCellFragment(xml: string): any {
const parser = new (xmlWindow().DOMParser)();
// Wrap so a self-closed or child-bearing single cell parses as one root.
const doc = parser.parseFromString(`<root>${xml}</root>`, "application/xml");
if (doc.getElementsByTagName("parsererror").length > 0) {
throw new CellOpsError(`operation xml is not well-formed: ${xml.slice(0, 120)}`);
}
const cells = doc.getElementsByTagName("mxCell");
if (cells.length !== 1) {
throw new CellOpsError(
`each add/update op must carry exactly one <mxCell> (got ${cells.length})`,
);
}
return cells[0];
}
/** All ids reachable as descendants of `rootId` via the parent relation. */
function collectDescendants(
rootId: string,
parentOf: Map<string, string | undefined>,
): Set<string> {
const doomed = new Set<string>([rootId]);
let grew = true;
while (grew) {
grew = false;
for (const [id, parent] of parentOf) {
if (!doomed.has(id) && parent != null && doomed.has(parent)) {
doomed.add(id);
grew = true;
}
}
}
return doomed;
}
/**
* Apply the operation list to a model XML string and return the new model XML.
* Uses the DOM so attribute order / formatting is preserved for untouched cells.
* Throws CellOpsError on an unknown target id or a malformed op fragment (so the
* model gets a precise error and nothing is half-applied).
*/
export function applyCellOps(modelXml: string, ops: CellOp[]): string {
if (!Array.isArray(ops) || ops.length === 0) {
throw new CellOpsError("operations must be a non-empty array");
}
const parser = new (xmlWindow().DOMParser)();
const doc = parser.parseFromString(modelXml, "application/xml");
if (doc.getElementsByTagName("parsererror").length > 0) {
throw new CellOpsError("the current diagram XML is not well-formed");
}
const root = doc.getElementsByTagName("root")[0];
if (!root) throw new CellOpsError("the current diagram has no <root> element");
const cellEls = () => Array.from(root.getElementsByTagName("mxCell")) as any[];
const byId = () => {
const m = new Map<string, any>();
for (const el of cellEls()) m.set(el.getAttribute("id") ?? "", el);
return m;
};
for (const op of ops) {
if (op.op === "add") {
const frag = parseCellFragment(op.xml);
const id = frag.getAttribute("id");
if (!id) throw new CellOpsError("an add op's <mxCell> is missing an id");
if (byId().has(id))
throw new CellOpsError(`add op id "${id}" already exists (use update)`);
root.appendChild(doc.importNode(frag, true));
} else if (op.op === "update") {
const map = byId();
const target = map.get(op.cellId);
if (!target)
throw new CellOpsError(`update target cell "${op.cellId}" does not exist`);
const frag = parseCellFragment(op.xml);
const newId = frag.getAttribute("id");
if (newId && newId !== op.cellId)
throw new CellOpsError(
`update op cellId "${op.cellId}" != the <mxCell> id "${newId}" (ids are stable)`,
);
// Replace the element in place so surrounding cells are untouched.
const imported = doc.importNode(frag, true);
target.parentNode.replaceChild(imported, target);
} else if (op.op === "delete") {
// Reject a sentinel-targeted delete BEFORE collecting descendants: "0"/"1"
// parent the entire diagram, so a cascade from either would wipe the whole
// model body (doomed.delete("0"/"1") only spared the sentinel itself, not
// its children).
if (SENTINEL_IDS.has(op.cellId))
throw new CellOpsError(
`cannot delete sentinel cell "${op.cellId}" (the graph root/default layer)`,
);
const map = byId();
if (!map.has(op.cellId))
throw new CellOpsError(`delete target cell "${op.cellId}" does not exist`);
// Build the parent relation over the CURRENT cells for the cascade.
const parentOf = new Map<string, string | undefined>();
for (const el of cellEls()) {
parentOf.set(el.getAttribute("id") ?? "", el.getAttribute("parent") ?? undefined);
}
const doomed = collectDescendants(op.cellId, parentOf);
// Cascade to edges whose source/target is any doomed cell.
for (const el of cellEls()) {
if (el.getAttribute("edge") !== "1") continue;
const src = el.getAttribute("source");
const tgt = el.getAttribute("target");
if ((src && doomed.has(src)) || (tgt && doomed.has(tgt))) {
doomed.add(el.getAttribute("id") ?? "");
}
}
// Never delete the sentinels even if referenced by a malformed op.
doomed.delete("0");
doomed.delete("1");
for (const el of cellEls()) {
const id = el.getAttribute("id") ?? "";
if (doomed.has(id)) el.parentNode.removeChild(el);
}
} else {
throw new CellOpsError(`unknown op "${(op as any).op}"`);
}
}
const ser = new (xmlWindow().XMLSerializer)();
return ser.serializeToString(doc.documentElement);
}
+916
View File
@@ -0,0 +1,916 @@
// Semantic graph -> draw.io pipeline for `drawioFromGraph` (issue #425, stage 3).
//
// The model describes a diagram SEMANTICALLY — nodes with a `kind` and an
// optional `icon`, groups (containers), edges with a `kind` — and NEVER sees a
// coordinate or a style string. This module owns the whole server-side pipeline:
//
// 1. validateGraph — a hand-written validator (no zod dependency, so this
// lib stays importable by client.ts without coupling to
// a zod major) that rejects malformed graphs early.
// 2. resolveNodeStyle — `icon` -> exact style via the shape catalog (#424);
// an UNKNOWN icon degrades to a generic shape by `kind`
// WITH the label (never an empty square). `kind` -> the
// preset palette slot.
// 3. graphToElk — graph -> ELK-JSON, honouring the layout hints
// (`layer`/`sameLayerAs` -> layer constraints, `pinned`
// -> a fixed node) and compound group nodes.
// 4. assembleModel — graph + ELK coordinates -> a full mxGraphModel XML
// that satisfies the #423 linter BY CONSTRUCTION
// (sentinels, transparent containers, relative child
// coords, cross-container edges parent="1", >=150px
// gaps from ELK spacing, escaped labels).
//
// The `layout` mode: "full" re-lays everything; "incremental" fixes existing
// coordinates (ELK interactive mode) and places only new nodes; "none" keeps the
// caller-provided/prior coordinates untouched.
import ELK from "elkjs/lib/elk.bundled.js";
import { JSDOM } from "jsdom";
import {
searchShapes,
awsServiceStyle,
type ShapeResult,
} from "./drawio-shapes.js";
import {
getPreset,
genericNodeStyle,
iconNodeStyle,
edgeStyle,
groupStyle,
type PresetData,
} from "./drawio-presets.js";
import { MIN_SHAPE_GAP } from "./drawio-xml.js";
// --- graph schema (plain TS + a hand validator) ----------------------------
export interface GraphNode {
id: string;
label: string;
kind?: string;
/** Icon reference, e.g. "aws:lambda" | "azure:cosmos" | "lambda". */
icon?: string;
/** Group (container) id this node belongs to. */
group?: string;
/** Layer hint (ELK layerChoiceConstraint): 0-based column/row index. */
layer?: number;
/** Put this node in the same layer as another node id. */
sameLayerAs?: string;
/** Fix this node at exact coordinates (an ELK fixed node). */
pinned?: { x: number; y: number };
}
export interface GraphGroup {
id: string;
label: string;
kind?: string;
/** Parent group id — lets a group nest inside another group (e.g. subnet in VPC). */
group?: string;
}
export interface GraphEdge {
from: string;
to: string;
label?: string;
kind?: string;
}
export interface Graph {
nodes: GraphNode[];
groups?: GraphGroup[];
edges?: GraphEdge[];
direction?: "LR" | "RL" | "TB" | "BT";
preset?: string;
}
export type LayoutMode = "none" | "full" | "incremental";
/** A structured validation error (mirrors the drawio linter's shape loosely). */
export class GraphValidationError extends Error {
issues: string[];
constructor(issues: string[]) {
super(`drawioFromGraph: invalid graph — ${issues.join("; ")}`);
this.name = "GraphValidationError";
this.issues = issues;
}
}
const MAX_GRAPH_NODES = 500; // parity with drawio-layout's ELK_MAX_NODES.
// Edge/group caps mirror drawio-layout's ELK_MAX_EDGES. Without an edge cap a
// tiny node set with a huge edge list (e.g. 500 nodes / 200000 edges) passes
// node validation, then graphToElk/runElk exhausts the heap SYNCHRONOUSLY inside
// elk.bundled.js — before the 5s ELK timeout can fire and OUTSIDE it entirely
// for the mapper/assembler — crashing the worker on LLM-authored input. Reject
// the over-limit shape here, before any layout or assembly runs.
export const MAX_GRAPH_EDGES = 1000; // parity with drawio-layout's ELK_MAX_EDGES.
export const MAX_GRAPH_GROUPS = 500; // groups are compound ELK nodes; bound them too.
/**
* Validate the graph structure BEFORE any layout/assembly so the model gets a
* precise, actionable error instead of a corrupt diagram. Throws
* GraphValidationError listing every problem.
*/
export function validateGraph(graph: Graph): void {
const issues: string[] = [];
if (!graph || typeof graph !== "object") {
throw new GraphValidationError(["graph must be an object"]);
}
if (!Array.isArray(graph.nodes) || graph.nodes.length === 0) {
throw new GraphValidationError(["graph.nodes must be a non-empty array"]);
}
// Size caps FIRST (fail fast, before touching per-element loops) so an
// over-limit graph can never reach the layout engine and OOM the worker.
if (graph.nodes.length > MAX_GRAPH_NODES) {
throw new GraphValidationError([
`graph has ${graph.nodes.length} nodes (max ${MAX_GRAPH_NODES})`,
]);
}
if (Array.isArray(graph.edges) && graph.edges.length > MAX_GRAPH_EDGES) {
throw new GraphValidationError([
`graph has ${graph.edges.length} edges (max ${MAX_GRAPH_EDGES})`,
]);
}
if (Array.isArray(graph.groups) && graph.groups.length > MAX_GRAPH_GROUPS) {
throw new GraphValidationError([
`graph has ${graph.groups.length} groups (max ${MAX_GRAPH_GROUPS})`,
]);
}
const nodeIds = new Set<string>();
const groupIds = new Set<string>();
for (const g of graph.groups ?? []) {
if (!g.id) issues.push("a group is missing its id");
else if (groupIds.has(g.id)) issues.push(`duplicate group id "${g.id}"`);
groupIds.add(g.id);
}
for (const n of graph.nodes) {
if (!n.id) issues.push("a node is missing its id");
else if (nodeIds.has(n.id)) issues.push(`duplicate node id "${n.id}"`);
else if (groupIds.has(n.id))
issues.push(`node id "${n.id}" collides with a group id`);
nodeIds.add(n.id);
if (typeof n.label !== "string" || n.label === "")
issues.push(`node "${n.id}" is missing a label`);
if (n.group != null && !groupIds.has(n.group))
issues.push(`node "${n.id}" references unknown group "${n.group}"`);
if (n.pinned != null) {
if (
typeof n.pinned.x !== "number" ||
typeof n.pinned.y !== "number" ||
!Number.isFinite(n.pinned.x) ||
!Number.isFinite(n.pinned.y)
)
issues.push(`node "${n.id}" has an invalid pinned {x,y}`);
}
if (n.layer != null && (!Number.isInteger(n.layer) || n.layer < 0))
issues.push(`node "${n.id}" has an invalid layer (must be a >=0 integer)`);
}
// sameLayerAs must reference an existing node (checked after all ids known).
for (const n of graph.nodes) {
if (n.sameLayerAs != null && !nodeIds.has(n.sameLayerAs))
issues.push(
`node "${n.id}" sameLayerAs references unknown node "${n.sameLayerAs}"`,
);
}
for (const e of graph.edges ?? []) {
if (!e.from || !e.to) {
issues.push("an edge is missing from/to");
continue;
}
if (!nodeIds.has(e.from) && !groupIds.has(e.from))
issues.push(`edge from "${e.from}" resolves to no node/group`);
if (!nodeIds.has(e.to) && !groupIds.has(e.to))
issues.push(`edge to "${e.to}" resolves to no node/group`);
}
if (issues.length > 0) throw new GraphValidationError(issues);
}
// --- icon resolution -------------------------------------------------------
/**
* Resolve a node's `icon` reference to a concrete style-string + size via the
* shape catalog. Accepts "aws:lambda", "azure:cosmos", or a bare "lambda". An
* AWS `resIcon` name is built directly (exact service-icon template). Anything
* else goes through searchShapes. Returns null when nothing resolves the
* caller then falls back to a generic shape by kind (never an empty box).
*/
export function resolveIcon(icon: string): ShapeResult | null {
const raw = icon.trim();
if (raw === "") return null;
let provider = "";
let name = raw;
const colon = raw.indexOf(":");
if (colon !== -1) {
provider = raw.slice(0, colon).trim().toLowerCase();
name = raw.slice(colon + 1).trim();
}
if (provider === "aws") {
// Prefer an exact resIcon match from the catalog (carries the right size and
// any rebrand/blocklist note); if the underscore/space name doesn't hit,
// build the canonical service-icon style directly so it is never an empty box.
const results = searchShapes(name.replace(/_/g, " "), { limit: 5 });
const aws4 = results.find((r) => r.style.includes("mxgraph.aws4"));
if (aws4) return aws4;
return {
style: awsServiceStyle(name.replace(/\s+/g, "_")),
w: 78,
h: 78,
title: name,
type: "vertex",
};
}
// Non-AWS or bare name: fuzzy search the catalog. Take the top vertex hit,
// but REJECT a weak match (the fuzzy scorer can prefix-match an unrelated
// stencil, e.g. "not..." -> "Notebook"); require the hit's title to actually
// share a meaningful token with the query, otherwise degrade to generic-by-kind.
const q = provider ? `${provider} ${name}` : name;
const results = searchShapes(q, { limit: 8 });
const hit = results.find((r) => r.type !== "edge") ?? results[0];
if (!hit) return null;
if (!isRelevantMatch(name, hit.title)) return null;
return hit;
}
/**
* Whether a resolved stencil is a genuine match for the requested icon name (as
* opposed to a loose prefix hit on an unrelated shape). True if any 3+ char
* token of the query appears in the stencil title, or vice-versa.
*/
function isRelevantMatch(name: string, title: string): boolean {
const norm = (s: string) => s.toLowerCase().replace(/[^a-z0-9]+/g, " ").trim();
const qTokens = norm(name).split(/\s+/).filter((t) => t.length >= 3);
if (qTokens.length === 0) return true; // very short names: trust the scorer
const t = norm(title);
const tTokens = new Set(t.split(/\s+/));
for (const qt of qTokens) {
if (tTokens.has(qt)) return true;
if (t.includes(qt)) return true;
for (const tt of tTokens) if (tt.length >= 3 && qt.includes(tt)) return true;
}
return false;
}
/**
* Decide the final style + size for a node. When `icon` resolves, use the icon
* style (overlaid with a dark-preset font fix); otherwise a GENERIC shape by
* `kind` carrying the label. `resolved` reports whether an icon was found (used
* by the acceptance test that asserts no empty squares).
*/
export function resolveNodeStyle(
preset: PresetData,
node: GraphNode,
): { style: string; w: number; h: number; iconResolved: boolean } {
if (node.icon) {
const shape = resolveIcon(node.icon);
if (shape) {
return {
style: iconNodeStyle(preset, shape.style),
w: shape.w,
h: shape.h,
iconResolved: true,
};
}
}
// Generic shape by kind, sized to the label so a long label never overflows.
const w = Math.max(120, estimateLabelWidth(node.label) + 32);
return { style: genericNodeStyle(preset, node.kind), w, h: 60, iconResolved: false };
}
/** Rough rendered width of the longest label line at 12px (~0.6em/glyph). */
function estimateLabelWidth(label: string): number {
const lines = label.split(/\r?\n|&#xa;|<br\s*\/?>/i);
let longest = 0;
for (const l of lines) longest = Math.max(longest, l.trim().length);
return Math.ceil(longest * 12 * 0.6);
}
// --- graph -> ELK-JSON -----------------------------------------------------
interface ElkNode {
id: string;
width?: number;
height?: number;
x?: number;
y?: number;
children?: ElkNode[];
layoutOptions?: Record<string, string>;
}
interface ElkEdge {
id: string;
sources: string[];
targets: string[];
}
interface ElkGraph extends ElkNode {
edges?: ElkEdge[];
}
const ELK_DIRECTION: Record<string, string> = {
LR: "RIGHT",
RL: "LEFT",
TB: "DOWN",
BT: "UP",
};
/** Sizes resolved per node id (from resolveNodeStyle), fed to the ELK mapper. */
export interface NodeSize {
w: number;
h: number;
}
/**
* Build the ELK graph from the semantic graph + resolved node sizes. Compound
* group nodes nest their members (a group may itself nest in another group).
* `only` restricts the graph to a subset of node ids (used by the incremental
* path to lay out ONLY the new nodes). Layout HINTS (`layer`/`sameLayerAs`/
* `pinned`) are NOT encoded as ELK constraints here ELK's constraint knobs are
* unreliable across versions they are enforced deterministically AFTER layout
* by applyHints, which is exact and testable.
*/
export function graphToElk(
graph: Graph,
sizes: Map<string, NodeSize>,
opts: { only?: Set<string> } = {},
): ElkGraph {
const direction = ELK_DIRECTION[graph.direction ?? "LR"] ?? "RIGHT";
const only = opts.only;
const include = (id: string) => !only || only.has(id);
const makeNode = (n: GraphNode): ElkNode => {
const size = sizes.get(n.id) ?? { w: 140, h: 60 };
return { id: n.id, width: size.w, height: size.h };
};
// Group children nest under their group node; ungrouped nodes are roots.
const groupNode = new Map<string, ElkNode>();
const usedGroups = new Set<string>();
for (const g of graph.groups ?? []) {
const size = sizes.get(g.id) ?? { w: 200, h: 150 };
groupNode.set(g.id, {
id: g.id,
width: size.w,
height: size.h,
children: [],
layoutOptions: {
"elk.algorithm": "layered",
"elk.direction": direction,
"elk.padding": "[top=40,left=30,bottom=30,right=30]",
"elk.layered.spacing.nodeNodeBetweenLayers": "170",
"elk.spacing.nodeNode": "170",
},
});
}
const roots: ElkNode[] = [];
for (const n of graph.nodes) {
if (!include(n.id)) continue;
const en = makeNode(n);
if (n.group && groupNode.has(n.group)) {
groupNode.get(n.group)!.children!.push(en);
usedGroups.add(n.group);
} else {
roots.push(en);
}
}
// Nest group nodes into their parent group (a subnet inside a VPC); groups
// with no parent group become roots. Only groups that hold an included node.
const groupIdSet = new Set((graph.groups ?? []).map((g) => g.id));
for (const g of graph.groups ?? []) {
if (only && !usedGroups.has(g.id)) continue;
const en = groupNode.get(g.id)!;
if (g.group && groupIdSet.has(g.group) && g.group !== g.id && (!only || usedGroups.has(g.group))) {
groupNode.get(g.group)!.children!.push(en);
} else {
roots.push(en);
}
}
// Edges: endpoints may be nodes or groups; INCLUDE_CHILDREN spans the nesting.
const validIds = new Set<string>([
...graph.nodes.filter((n) => include(n.id)).map((n) => n.id),
...(graph.groups ?? []).map((g) => g.id),
]);
const edges: ElkEdge[] = [];
(graph.edges ?? []).forEach((e, i) => {
if (!validIds.has(e.from) || !validIds.has(e.to)) return;
edges.push({ id: `e${i}`, sources: [e.from], targets: [e.to] });
});
const rootOptions: Record<string, string> = {
"elk.algorithm": "layered",
"elk.direction": direction,
"elk.hierarchyHandling": "INCLUDE_CHILDREN",
"elk.layered.spacing.nodeNodeBetweenLayers": "170",
"elk.spacing.nodeNode": "170",
"elk.spacing.edgeNode": "40",
"elk.spacing.edgeEdge": "30",
"elk.padding": "[top=20,left=20,bottom=20,right=20]",
};
return { id: "root", layoutOptions: rootOptions, children: roots, edges };
}
/**
* Enforce the layout hints DETERMINISTICALLY on ELK's output (mutates `geo`):
* - `sameLayerAs`: snap the dependent node's LAYER-AXIS coordinate to its
* anchor's, so the pair lands in the same layer (x for LR/RL, y for TB/BT).
* `layer` groups nodes with the same index onto the same anchor coordinate.
* - `pinned`: override the node's coordinate with the exact pinned {x,y}.
* Applied only to top-level (ungrouped) nodes, whose ELK coords are absolute.
*/
export function applyHints(
graph: Graph,
geo: Map<string, { x: number; y: number; w: number; h: number }>,
): void {
const dir = graph.direction ?? "LR";
const layerAxis: "x" | "y" = dir === "TB" || dir === "BT" ? "y" : "x";
// The perpendicular (cross-layer) axis: members snapped onto one layer must be
// spread along THIS axis so they don't stack onto the same point.
const crossAxis: "x" | "y" = layerAxis === "x" ? "y" : "x";
const crossSize: "w" | "h" = crossAxis === "x" ? "w" : "h";
const grouped = new Set(
graph.nodes.filter((n) => n.group).map((n) => n.id),
);
// sameLayerAs / layer: co-assign the layer-axis coordinate.
// Build the effective layer key per node, then pick a representative coord.
const layerKeyOf = new Map<string, string>();
const explicitLayer = new Map<string, number>();
for (const n of graph.nodes) if (n.layer != null) explicitLayer.set(n.id, n.layer);
const byId = new Map(graph.nodes.map((n) => [n.id, n]));
const resolveKey = (n: GraphNode): string | null => {
if (explicitLayer.has(n.id)) return `L${explicitLayer.get(n.id)}`;
const seen = new Set<string>([n.id]);
let cur: GraphNode | undefined = n;
while (cur && cur.sameLayerAs != null && !seen.has(cur.sameLayerAs)) {
seen.add(cur.sameLayerAs);
const t = byId.get(cur.sameLayerAs);
if (!t) break;
if (explicitLayer.has(t.id)) return `L${explicitLayer.get(t.id)}`;
cur = t;
}
// A sameLayerAs chain with no explicit layer: key on the chain's root id.
if (n.sameLayerAs != null) {
let root = n.id;
const s2 = new Set<string>([n.id]);
let c: GraphNode | undefined = n;
while (c && c.sameLayerAs != null && !s2.has(c.sameLayerAs)) {
s2.add(c.sameLayerAs);
root = c.sameLayerAs;
c = byId.get(c.sameLayerAs);
}
return `C${root}`;
}
return null;
};
for (const n of graph.nodes) {
if (grouped.has(n.id)) continue; // group children are relative — skip
const key = resolveKey(n);
if (key) layerKeyOf.set(n.id, key);
}
// Group members of each layer key so we can snap AND spread them together.
const membersOf = new Map<string, string[]>();
for (const n of graph.nodes) {
const key = layerKeyOf.get(n.id);
if (key == null) continue;
if (!geo.has(n.id)) continue;
(membersOf.get(key) ?? membersOf.set(key, []).get(key)!).push(n.id);
}
// For each layer key: snap every member to the FIRST member's layer-axis coord,
// then SPREAD them along the perpendicular (cross-layer) axis with a >=
// MIN_SHAPE_GAP gap. Without the spread, a sameLayerAs chain whose nodes ELK
// happened to give the same cross-axis coordinate would collapse onto one point
// -> shape-overlap + edge-through-shape quality warnings (breaking the
// "0 warnings by construction" guarantee for these AUTO-positioned hints). We
// start from the members' minimum cross-axis coord and stack them with a gap
// of MIN_SHAPE_GAP beyond each shape's cross-axis size.
for (const [key, members] of membersOf) {
if (members.length === 0) continue;
// Snap layer-axis coord to the first member.
const repCoord = geo.get(members[0])![layerAxis];
// Preserve the members' existing relative order along the cross axis so the
// spread stays visually stable, then re-lay them contiguously.
const sorted = [...members].sort(
(a, b) => geo.get(a)![crossAxis] - geo.get(b)![crossAxis],
);
let cursor = geo.get(sorted[0])![crossAxis];
for (const id of sorted) {
const g = geo.get(id)!;
g[layerAxis] = repCoord;
g[crossAxis] = cursor;
cursor += g[crossSize] + MIN_SHAPE_GAP;
}
void key;
}
// pinned: exact override (wins over any layer snap). Explicit user coordinates
// are user intent, but CLAMP to non-negative so an out-of-bounds pin (e.g.
// x:-500) never renders off-canvas. Two user-pinned nodes at the same point is
// user error the server can't silently relocate — the assembler docstring
// documents that explicit pins are user-directed and MAY warn (see #423/#425
// acceptance: the "0 quality-warnings by construction" guarantee is for
// AUTO-LAYOUT, not for coordinates the user pinned by hand).
for (const n of graph.nodes) {
if (!n.pinned) continue;
const px = Math.max(0, n.pinned.x);
const py = Math.max(0, n.pinned.y);
const g = geo.get(n.id);
if (g) {
g.x = px;
g.y = py;
} else {
const sz = { w: 140, h: 60 };
geo.set(n.id, { x: px, y: py, w: sz.w, h: sz.h });
}
}
}
/**
* Incremental variant of applyHints: apply `pinned` only to NEW nodes (those
* absent from `existing`); an existing node's coordinates are NEVER changed
* (acceptance #3). sameLayerAs/layer snapping is intentionally skipped in the
* incremental path moving a new node's layer axis could still be desired, but
* it must never move an existing cell, so we keep the incremental contract
* simple: existing cells are frozen, new pinned nodes honour their pin.
*/
export function applyHintsForNew(
graph: Graph,
geo: Map<string, { x: number; y: number; w: number; h: number }>,
existing: Map<string, { x: number; y: number }>,
): void {
for (const n of graph.nodes) {
if (existing.has(n.id)) continue; // never move an existing cell
if (!n.pinned) continue;
const px = Math.max(0, n.pinned.x); // clamp out-of-bounds pins non-negative
const py = Math.max(0, n.pinned.y);
const g = geo.get(n.id);
if (g) {
g.x = px;
g.y = py;
} else {
geo.set(n.id, { x: px, y: py, w: 140, h: 60 });
}
}
}
// --- layout runner ---------------------------------------------------------
const ELK_TIMEOUT_MS = 5000;
/**
* Run ELK over the mapped graph and return computed geometry per id (coords are
* parent-relative, matching mxGraph's convention for container children). On any
* ELK failure/timeout the returned map is empty and the caller falls back to a
* deterministic grid placement (so the write never fails on a layout hiccup).
*/
export async function runElk(
elk: ElkGraph,
): Promise<Map<string, { x: number; y: number; w: number; h: number }>> {
const geo = new Map<string, { x: number; y: number; w: number; h: number }>();
let timer: ReturnType<typeof setTimeout> | undefined;
try {
const Ctor: any = (ELK as any).default ?? ELK;
const inst = new Ctor();
const timeout = new Promise<never>((_, reject) => {
timer = setTimeout(() => reject(new Error("ELK timed out")), ELK_TIMEOUT_MS);
});
const laid = (await Promise.race([inst.layout(elk as any), timeout])) as ElkGraph;
const walk = (n: ElkNode) => {
if (n.id !== "root") {
geo.set(n.id, {
x: Math.round(n.x ?? 0),
y: Math.round(n.y ?? 0),
w: Math.round(n.width ?? 140),
h: Math.round(n.height ?? 60),
});
}
for (const c of n.children ?? []) walk(c);
};
walk(laid);
} catch {
return new Map(); // best-effort: empty -> caller uses fallback grid.
} finally {
if (timer) clearTimeout(timer);
}
return geo;
}
// --- XML assembler ---------------------------------------------------------
/** Order groups so a parent group always precedes its nested children. */
function topoSortGroups(groups: GraphGroup[], groupIds: Set<string>): GraphGroup[] {
const byId = new Map(groups.map((g) => [g.id, g]));
const out: GraphGroup[] = [];
const done = new Set<string>();
const visit = (g: GraphGroup, stack: Set<string>) => {
if (done.has(g.id)) return;
if (stack.has(g.id)) return; // cycle guard
stack.add(g.id);
if (g.group && groupIds.has(g.group) && g.group !== g.id) {
const parent = byId.get(g.group);
if (parent) visit(parent, stack);
}
stack.delete(g.id);
if (!done.has(g.id)) {
done.add(g.id);
out.push(g);
}
};
for (const g of groups) visit(g, new Set());
return out;
}
function xmlEscapeAttr(s: string): string {
return s
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/\r\n|\r|\n/g, "&#xa;"); // literal newline -> the linter-approved entity
}
export interface AssembleResult {
modelXml: string;
iconsResolved: number;
iconsMissing: string[];
}
/**
* Assemble the final mxGraphModel XML from the graph + resolved styles/coords.
* Guarantees BY CONSTRUCTION that the #423 linter passes:
* - id=0 and id=1(parent=0) sentinels;
* - each node/group is vertex="1" (containers get container=1 via the style);
* - each edge is edge="1" with a child <mxGeometry relative="1" as="geometry"/>;
* - group children set parent=<groupId> and RELATIVE coords; an edge between
* two different parents is parent="1";
* - labels are XML-escaped and any newline is &#xa;.
* `geo` may be empty (ELK failed) then a deterministic grid is used so the
* output is still valid and non-overlapping (>=170px stride).
*
* QUALITY-WARNING GUARANTEE: the "0 quality-warnings by construction" promise
* holds for AUTO-LAYOUT ELK spacing plus applyHints' cross-axis spread for the
* server-positioned `layer`/`sameLayerAs` hints keep shapes >=MIN_SHAPE_GAP
* apart. It does NOT extend to explicit `pinned` coordinates: those are
* user-directed, so two nodes the user pins to the same/overlapping point are
* user error the server honours verbatim (only clamped non-negative) and MAY
* therefore produce a quality warning.
*/
export function assembleModel(
graph: Graph,
opts: {
preset: PresetData;
styles: Map<string, { style: string; w: number; h: number; iconResolved: boolean }>;
geo: Map<string, { x: number; y: number; w: number; h: number }>;
},
): AssembleResult {
const { preset, styles, geo } = opts;
const groupIds = new Set((graph.groups ?? []).map((g) => g.id));
const nodeById = new Map(graph.nodes.map((n) => [n.id, n]));
// Fallback grid when ELK produced nothing: lay ungrouped nodes on a grid with
// a 190px stride (>150 gap). Grouped nodes/groups are placed inside their group.
const fallback = geo.size === 0;
const gridPos = (i: number) => ({ x: 40 + (i % 5) * 200, y: 40 + Math.floor(i / 5) * 140 });
const cells: string[] = ['<mxCell id="0"/>', '<mxCell id="1" parent="0"/>'];
// Groups first (they are parents of their members). A nested group sets
// parent=<parentGroupId>; emit parents before children so parent-exists holds.
let gi = 0;
const groupGeo = new Map<string, { x: number; y: number; w: number; h: number }>();
const orderedGroups = topoSortGroups(graph.groups ?? [], groupIds);
for (const g of orderedGroups) {
const gg = geo.get(g.id) ?? { ...gridPos(gi++), w: 320, h: 220 };
groupGeo.set(g.id, gg);
const style = groupStyle(preset);
const gParent = g.group && groupIds.has(g.group) && g.group !== g.id ? g.group : "1";
cells.push(
`<mxCell id="${xmlEscapeAttr(g.id)}" value="${xmlEscapeAttr(g.label)}" style="${style}" vertex="1" parent="${xmlEscapeAttr(gParent)}">` +
`<mxGeometry x="${gg.x}" y="${gg.y}" width="${gg.w}" height="${gg.h}" as="geometry"/></mxCell>`,
);
}
// Nodes. A grouped node's coords are RELATIVE to its group (ELK already
// returns child coords relative to the parent; for the fallback grid we place
// children on a small in-group grid).
let ungrouped = (graph.groups?.length ?? 0);
const inGroupIndex = new Map<string, number>();
let iconsResolved = 0;
const iconsMissing: string[] = [];
for (const n of graph.nodes) {
const st = styles.get(n.id)!;
if (n.icon) {
if (st.iconResolved) iconsResolved++;
else iconsMissing.push(n.id);
}
let x: number;
let y: number;
const g = geo.get(n.id);
if (g && !fallback) {
x = g.x;
y = g.y;
} else if (n.group && groupIds.has(n.group)) {
const k = inGroupIndex.get(n.group) ?? 0;
inGroupIndex.set(n.group, k + 1);
x = 30 + (k % 3) * 180;
y = 40 + Math.floor(k / 3) * 120;
} else {
const p = gridPos(ungrouped++);
x = p.x;
y = p.y;
}
const parent = n.group && groupIds.has(n.group) ? n.group : "1";
cells.push(
`<mxCell id="${xmlEscapeAttr(n.id)}" value="${xmlEscapeAttr(n.label)}" style="${st.style}" vertex="1" parent="${xmlEscapeAttr(parent)}">` +
`<mxGeometry x="${x}" y="${y}" width="${st.w}" height="${st.h}" as="geometry"/></mxCell>`,
);
}
// Edges. parent="1" whenever the two endpoints have different container
// parents (or either is a group); otherwise the shared group id.
(graph.edges ?? []).forEach((e, i) => {
const style = edgeStyle(preset, e.kind);
const fromNode = nodeById.get(e.from);
const toNode = nodeById.get(e.to);
const fromParent = fromNode?.group && groupIds.has(fromNode.group) ? fromNode.group : "1";
const toParent = toNode?.group && groupIds.has(toNode.group) ? toNode.group : "1";
const parent = fromParent === toParent ? fromParent : "1";
const label = e.label ? ` value="${xmlEscapeAttr(e.label)}"` : "";
cells.push(
`<mxCell id="ge${i}"${label} style="${style}" edge="1" parent="${xmlEscapeAttr(parent)}" ` +
`source="${xmlEscapeAttr(e.from)}" target="${xmlEscapeAttr(e.to)}">` +
`<mxGeometry relative="1" as="geometry"/></mxCell>`,
);
});
const modelAttrs =
'dx="0" dy="0" grid="1" gridSize="10" page="1" pageWidth="850" pageHeight="1100" adaptiveColors="auto"';
const modelXml = `<mxGraphModel ${modelAttrs}><root>${cells.join("")}</root></mxGraphModel>`;
return { modelXml, iconsResolved, iconsMissing };
}
// --- incremental merge -----------------------------------------------------
let _mergeWindow: any = null;
function mergeWindow(): any {
if (!_mergeWindow) _mergeWindow = new JSDOM("").window;
return _mergeWindow;
}
/**
* Merge the freshly-assembled graph XML with the EXISTING diagram model so an
* incremental "add a node" call never drops a hand-placed cell. `assembleModel`
* emits ONLY the passed graph's cells; on its own it would replace the whole
* model, wiping any existing cell the caller didn't re-list. This splices every
* existing cell that the graph does NOT re-list (preserved verbatim: coords,
* style, edges) into the assembled root:
* - id in the graph -> the graph's (re-laid) cell wins (already assembled;
* coords are frozen for existing ids via the incremental geo path);
* - id NOT in the graph -> the existing cell is preserved verbatim;
* - a graph node absent from the existing model -> added (offset clear).
* The sentinels ("0"/"1") come from the assembled model and are never doubled.
*/
function mergeExistingCells(
assembledXml: string,
existingModelXml: string,
graph: Graph,
): string {
const win = mergeWindow();
const parser = new win.DOMParser();
const existingDoc = parser.parseFromString(existingModelXml, "application/xml");
if (existingDoc.getElementsByTagName("parsererror").length > 0) {
// Existing model unreadable: fall back to the assembled model alone (still a
// valid diagram — better than throwing on a corrupt prior file).
return assembledXml;
}
const assembledDoc = parser.parseFromString(assembledXml, "application/xml");
const root = assembledDoc.getElementsByTagName("root")[0];
if (!root) return assembledXml;
// Ids the assembled model already emitted (graph nodes/groups/edges + sentinels).
const assembledIds = new Set<string>();
for (const el of Array.from(root.getElementsByTagName("mxCell")) as any[]) {
const id = el.getAttribute("id");
if (id) assembledIds.add(id);
}
// The graph's own ids: any existing cell with one of these is superseded by the
// assembled version and must NOT be re-imported.
const graphIds = new Set<string>([
...graph.nodes.map((n) => n.id),
...(graph.groups ?? []).map((g) => g.id),
]);
const existingCells = Array.from(
existingDoc.getElementsByTagName("mxCell"),
) as any[];
for (const el of existingCells) {
const id = el.getAttribute("id") ?? "";
if (id === "0" || id === "1") continue; // sentinels come from the assembled model
if (graphIds.has(id)) continue; // graph re-lists it -> assembled version wins
if (assembledIds.has(id)) continue; // id collision guard -> keep assembled
root.appendChild(assembledDoc.importNode(el, true));
assembledIds.add(id);
}
const ser = new win.XMLSerializer();
return ser.serializeToString(assembledDoc.documentElement);
}
// --- top-level: graph -> mxGraphModel XML ----------------------------------
export interface BuildFromGraphResult {
modelXml: string;
iconsResolved: number;
iconsMissing: string[];
layout: LayoutMode;
}
/**
* The full server-side pipeline: validate -> resolve styles/icons -> map to ELK
* -> run ELK (or fall back) -> assemble linter-clean XML. `existingCoords` is
* supplied for `layout:"incremental"` (the coordinates of the diagram's current
* cells, so they are preserved and only new nodes are placed). `existingModelXml`
* is the current diagram's full model XML in incremental mode every existing
* cell the graph does NOT re-list is MERGED back in verbatim so a hand-placed
* cell is never dropped (WARNING #4). Pure no network.
*/
export async function buildFromGraph(
graph: Graph,
layout: LayoutMode = "full",
existingCoords?: Map<string, { x: number; y: number }>,
existingModelXml?: string,
): Promise<BuildFromGraphResult> {
validateGraph(graph);
const preset = getPreset(graph.preset);
// Resolve every node's style + size (icon or generic-by-kind).
const styles = new Map<
string,
{ style: string; w: number; h: number; iconResolved: boolean }
>();
const sizes = new Map<string, NodeSize>();
for (const n of graph.nodes) {
const s = resolveNodeStyle(preset, n);
styles.set(n.id, s);
sizes.set(n.id, { w: s.w, h: s.h });
}
// Group sizes: seed a min box; ELK computes the real size when it lays out.
for (const g of graph.groups ?? []) sizes.set(g.id, { w: 240, h: 180 });
let geo = new Map<string, { x: number; y: number; w: number; h: number }>();
if (layout === "incremental" && existingCoords && existingCoords.size > 0) {
// INCREMENTAL: keep every existing cell's coords VERBATIM (acceptance #3 —
// never move a hand-arranged cell) and lay out ONLY the new nodes, then
// offset that block clear of the existing bbox so nothing overlaps.
for (const [id, c] of existingCoords) {
const sz = sizes.get(id) ?? { w: 140, h: 60 };
geo.set(id, { x: c.x, y: c.y, w: sz.w, h: sz.h });
}
const newIds = new Set(
graph.nodes.filter((n) => !existingCoords.has(n.id)).map((n) => n.id),
);
if (newIds.size > 0) {
const elk = graphToElk(graph, sizes, { only: newIds });
const laid = await runElk(elk);
// Place the new block below the existing content (a clear >=170px gap).
let maxY = 0;
for (const c of existingCoords.values()) maxY = Math.max(maxY, c.y);
const offsetY = maxY + 200;
for (const [id, g] of laid) {
if (newIds.has(id)) geo.set(id, { ...g, y: g.y + offsetY });
else if (!geo.has(id)) geo.set(id, g); // a new group container
}
}
// Hints still apply to NEW pinned nodes only (existing ones stay put).
applyHintsForNew(graph, geo, existingCoords);
} else if (layout !== "none") {
const elk = graphToElk(graph, sizes);
geo = await runElk(elk);
applyHints(graph, geo);
} else if (existingCoords) {
// layout:"none" with prior coords -> keep them verbatim.
for (const [id, c] of existingCoords) {
const sz = sizes.get(id) ?? { w: 140, h: 60 };
geo.set(id, { x: c.x, y: c.y, w: sz.w, h: sz.h });
}
}
const assembled = assembleModel(graph, { preset, styles, geo });
// In incremental mode, splice back every existing cell the graph didn't
// re-list so an "add one node" call preserves the user's manual layout.
let modelXml = assembled.modelXml;
if (
layout === "incremental" &&
existingModelXml &&
existingCoords &&
existingCoords.size > 0
) {
modelXml = mergeExistingCells(modelXml, existingModelXml, graph);
}
return {
modelXml,
iconsResolved: assembled.iconsResolved,
iconsMissing: assembled.iconsMissing,
layout,
};
}
+14 -14
View File
@@ -1,4 +1,4 @@
// Progressive-disclosure authoring reference for the `drawio_guide` tool // Progressive-disclosure authoring reference for the `drawioGuide` tool
// (issue #424, stage 2). The FULL draw.io authoring guide would bloat every // (issue #424, stage 2). The FULL draw.io authoring guide would bloat every
// context window, so it is split into small sections the model reads on demand: // context window, so it is split into small sections the model reads on demand:
// skeleton | layout | containers | icons-aws | icons-azure // skeleton | layout | containers | icons-aws | icons-azure
@@ -22,7 +22,7 @@ export const GUIDE_SECTIONS: GuideSection[] = [
"icons-azure", "icons-azure",
]; ];
const SKELETON = `# drawio_guide: skeleton const SKELETON = `# drawioGuide: skeleton
Canonical mxGraph skeleton. id="0" and id="1" are MANDATORY sentinels; every Canonical mxGraph skeleton. id="0" and id="1" are MANDATORY sentinels; every
real cell has parent="1" (or a container id). Set adaptiveColors="auto" on the real cell has parent="1" (or a container id). Set adaptiveColors="auto" on the
@@ -50,7 +50,7 @@ model so Docmost's dark theme adapts strokeColor/fillColor/fontColor="default".
</mxGraphModel> </mxGraphModel>
\`\`\` \`\`\`
Three accepted inputs to drawio_create/drawio_update: a bare <mxGraphModel>, a Three accepted inputs to drawioCreate/drawioUpdate: a bare <mxGraphModel>, a
full <mxfile> (decoded to its first page), or a raw list of <mxCell> (the server full <mxfile> (decoded to its first page), or a raw list of <mxCell> (the server
wraps it and adds the id=0/id=1 sentinels). wraps it and adds the id=0/id=1 sentinels).
@@ -58,12 +58,12 @@ Hard rules: a cell is vertex="1" XOR edge="1" (a container/group is neither);
every edge has a child <mxGeometry relative="1" as="geometry"/>; ids are unique; every edge has a child <mxGeometry relative="1" as="geometry"/>; ids are unique;
no XML comments; put html=1 in styles and XML-escape value (& -> &amp;, no XML comments; put html=1 in styles and XML-escape value (& -> &amp;,
< -> &lt;); a newline in a label is &#xa;, never a literal \\n. Don't guess < -> &lt;); a newline in a label is &#xa;, never a literal \\n. Don't guess
shape=mxgraph.* names call drawio_shapes first (a wrong name renders empty).`; shape=mxgraph.* names call drawioShapes first (a wrong name renders empty).`;
const LAYOUT = `# drawio_guide: layout const LAYOUT = `# drawioGuide: layout
Turn "make it look good" into checkable numbers. Or pass layout:"elk" to Turn "make it look good" into checkable numbers. Or pass layout:"elk" to
drawio_create/drawio_update and the server computes coordinates for you (ELK drawioCreate/drawioUpdate and the server computes coordinates for you (ELK
layered layout, honouring nested containers) you declare structure, it places layered layout, honouring nested containers) you declare structure, it places
pixels. pixels.
@@ -96,7 +96,7 @@ The linter returns quality WARNINGS (bbox overlap, edge through a shape,
edge-on-edge, gap <150px, label wider than its shape, negative/off-page coords). edge-on-edge, gap <150px, label wider than its shape, negative/off-page coords).
They do not block the write fix them and retry, max 2 iterations.`; They do not block the write fix them and retry, max 2 iterations.`;
const CONTAINERS = `# drawio_guide: containers const CONTAINERS = `# drawioGuide: containers
Groups/zones are TRANSPARENT containers. A coloured group fill is an instant Groups/zones are TRANSPARENT containers. A coloured group fill is an instant
"AI-generated" tell never fill a group. "AI-generated" tell never fill a group.
@@ -133,10 +133,10 @@ Example (transparent zone with two children and an internal edge):
</mxCell> </mxCell>
\`\`\``; \`\`\``;
const ICONS_AWS = `# drawio_guide: icons-aws const ICONS_AWS = `# drawioGuide: icons-aws
Two mutually-exclusive AWS icon patterns mixing them is the #1 cause of empty Two mutually-exclusive AWS icon patterns mixing them is the #1 cause of empty
boxes. Always call drawio_shapes for the exact resIcon name; do not guess. boxes. Always call drawioShapes for the exact resIcon name; do not guess.
| Level | style | strokeColor | | Level | style | strokeColor |
|---|---|---| |---|---|---|
@@ -168,7 +168,7 @@ Group stencils (transparent containers): AWS Cloud group_aws_cloud_alt, VPC
group_vpc2, Subnet group_security_group, Account group_account; subnets use group_vpc2, Subnet group_security_group, Account group_account; subnets use
shape=mxgraph.aws4.group;grIcon=mxgraph.aws4.group_public_subnet;.`; shape=mxgraph.aws4.group;grIcon=mxgraph.aws4.group_public_subnet;.`;
const ICONS_AZURE = `# drawio_guide: icons-azure const ICONS_AZURE = `# drawioGuide: icons-azure
shape=mxgraph.azure2.* does NOT render in every host. Use the portable shape=mxgraph.azure2.* does NOT render in every host. Use the portable
image-style instead: image-style instead:
@@ -190,7 +190,7 @@ an absolute URL fallback for the image:
https://raw.githubusercontent.com/jgraph/drawio/dev/src/main/webapp/img/lib/azure2/<category>/<Icon>.svg https://raw.githubusercontent.com/jgraph/drawio/dev/src/main/webapp/img/lib/azure2/<category>/<Icon>.svg
\`\`\` \`\`\`
Call drawio_shapes with the service name (e.g. "cosmos", "api management", Call drawioShapes with the service name (e.g. "cosmos", "api management",
"front door") to get the exact image-style string and default 68x68 size.`; "front door") to get the exact image-style string and default 68x68 size.`;
const CONTENT: Record<GuideSection, string> = { const CONTENT: Record<GuideSection, string> = {
@@ -216,13 +216,13 @@ export function getGuideSection(section?: string): {
return { section: key, content: CONTENT[key], sections: GUIDE_SECTIONS }; return { section: key, content: CONTENT[key], sections: GUIDE_SECTIONS };
} }
const index = const index =
"# drawio_guide\n\nProgressive-disclosure draw.io authoring reference. " + "# drawioGuide\n\nProgressive-disclosure draw.io authoring reference. " +
"Call drawio_guide(section) with one of:\n" + "Call drawioGuide(section) with one of:\n" +
"- skeleton — canonical mxGraph XML, sentinels, the three accepted inputs, hard rules\n" + "- skeleton — canonical mxGraph XML, sentinels, the three accepted inputs, hard rules\n" +
"- layout — spacing heuristics, edge routing, the layout:\"elk\" option, quality warnings\n" + "- layout — spacing heuristics, edge routing, the layout:\"elk\" option, quality warnings\n" +
"- containers — transparent groups, relative child coords, cross-container edges, swimlanes\n" + "- containers — transparent groups, relative child coords, cross-container edges, swimlanes\n" +
"- icons-aws — the service/resource icon patterns, category colors, rebrandings, blocklist\n" + "- icons-aws — the service/resource icon patterns, category colors, rebrandings, blocklist\n" +
"- icons-azure — the portable image-style paths\n\n" + "- icons-azure — the portable image-style paths\n\n" +
"Also call drawio_shapes(query) for verified stencil style-strings."; "Also call drawioShapes(query) for verified stencil style-strings.";
return { section: "index", content: index, sections: GUIDE_SECTIONS }; return { section: "index", content: index, sections: GUIDE_SECTIONS };
} }
+2 -2
View File
@@ -19,7 +19,7 @@ const DEFAULT_W = 140;
const DEFAULT_H = 60; const DEFAULT_H = 60;
// DoS bounds for the in-process ELK layout. The mxGraph XML is LLM-supplied // DoS bounds for the in-process ELK layout. The mxGraph XML is LLM-supplied
// (layout:"elk" in drawio_create/drawio_update) and elkjs runs synchronously on // (layout:"elk" in drawioCreate/drawioUpdate) and elkjs runs synchronously on
// the MCP server's event loop, so an unbounded graph would block it for // the MCP server's event loop, so an unbounded graph would block it for
// seconds-to-minutes. A ~1MB XML (well under the stage-1 16MB cap) can carry // seconds-to-minutes. A ~1MB XML (well under the stage-1 16MB cap) can carry
// thousands of nodes. We cap the graph size and race the layout against a // thousands of nodes. We cap the graph size and race the layout against a
@@ -81,7 +81,7 @@ interface ElkGraph extends ElkNode {
/** /**
* Apply an ELK layered layout to a drawio input and return a full mxGraphModel * Apply an ELK layered layout to a drawio input and return a full mxGraphModel
* string with rewritten geometry. Accepts the same three input forms as * string with rewritten geometry. Accepts the same three input forms as
* drawio_create (a bare model, an <mxfile>, or a <mxCell> list). Async because * drawioCreate (a bare model, an <mxfile>, or a <mxCell> list). Async because
* elkjs' layout() is promise-based. On any layout failure the ORIGINAL * elkjs' layout() is promise-based. On any layout failure the ORIGINAL
* (normalized) model is returned unchanged layout is best-effort polish, never * (normalized) model is returned unchanged layout is best-effort polish, never
* a reason to fail the write. * a reason to fail the write.
+347
View File
@@ -0,0 +1,347 @@
// Pure Mermaid `flowchart` -> graph-JSON parser for `drawioFromMermaid` (issue
// #425, stage 3, OPTIONAL). The escape clause in the issue: convert WITHOUT
// Electron/draw.io-CLI, so a pure text parser only. It handles the common wiki
// flowchart subset — node shapes, labelled/dashed edges, subgraphs (-> groups),
// and the direction header — and emits a Graph the drawioFromGraph pipeline
// renders as an EDITABLE draw.io diagram. Anything beyond flowchart (sequence /
// class / state) throws a clear error so the model falls back to drawioFromGraph.
//
// DELIBERATELY NARROW: this is not a full Mermaid grammar (Mermaid's own parser
// is a 100KB+ browser dependency). It covers `flowchart`/`graph` with the node
// shapes and edge arrows that show up in practice; unusual syntax is skipped
// rather than mis-parsed, and a diagram that yields no nodes throws.
import type { Graph, GraphNode, GraphEdge, GraphGroup } from "./drawio-graph.js";
export class MermaidParseError extends Error {
constructor(message: string) {
super(`drawioFromMermaid: ${message}`);
this.name = "MermaidParseError";
}
}
// Input-size bounds applied BEFORE parsing. Without them a pathological mermaid
// string (e.g. 300000 connection lines, or 20000 nested `subgraph`s) builds a
// huge intermediate node/edge/group structure that OOM-crashes the worker — the
// downstream validateGraph caps in drawio-graph can't help because the parser
// exhausts the heap constructing the intermediate FIRST. These caps reject the
// over-limit input fast, before a single line is parsed.
const MAX_MERMAID_CHARS = 200_000; // ~200 KB of source is far beyond any real diagram.
const MAX_MERMAID_LINES = 20_000;
const MAX_MERMAID_GROUPS = 500; // parity with drawio-graph's MAX_GRAPH_GROUPS.
// Per connection line, the number of chained nodes we will expand (`A-->B-->C`).
const MAX_CHAIN_NODES = 500;
const DIRECTIONS: Record<string, Graph["direction"]> = {
LR: "LR",
RL: "RL",
TB: "TB",
TD: "TB",
BT: "BT",
};
/**
* Node-shape delimiters -> a semantic `kind`. Mermaid encodes shape in the
* bracket style; we map the common ones to the palette kinds so the diagram is
* colored meaningfully (a decision/diamond -> queue, a database cylinder -> db,
* a rounded/stadium -> service, a subroutine/hexagon -> gateway, default rect ->
* service). The label text lives between the delimiters.
*/
interface ShapeDef {
open: string;
close: string;
kind: string;
}
// Order matters: longer/multi-char delimiters first so "([" beats "(".
const SHAPES: ShapeDef[] = [
{ open: "([", close: "])", kind: "service" }, // stadium
{ open: "[[", close: "]]", kind: "gateway" }, // subroutine
{ open: "[(", close: ")]", kind: "db" }, // cylinder-ish / database
{ open: "((", close: "))", kind: "external" }, // circle
{ open: "{{", close: "}}", kind: "gateway" }, // hexagon
{ open: "[", close: "]", kind: "service" }, // rectangle
{ open: "(", close: ")", kind: "service" }, // rounded
{ open: "{", close: "}", kind: "queue" }, // rhombus / decision
{ open: ">", close: "]", kind: "external" }, // asymmetric flag
];
/** Strip Mermaid label quoting/escapes and normalise whitespace. */
function cleanLabel(raw: string): string {
let s = raw.trim();
if (
(s.startsWith('"') && s.endsWith('"')) ||
(s.startsWith("'") && s.endsWith("'"))
) {
s = s.slice(1, -1);
}
return s.replace(/<br\s*\/?>/gi, " ").replace(/\s+/g, " ").trim();
}
/** A single edge-arrow spec: its regex and the resulting edge `kind`. */
interface ArrowDef {
re: RegExp;
kind: string;
}
// Dotted arrows (`-.->`) -> async; thick (`==>`) stay sync; normal `-->`/`---`.
// Each captures an optional `|label|` OR inline label between the two arrow
// halves. Applied to the segment between two node tokens.
const ARROWS: ArrowDef[] = [
{ re: /-\.->|-\.-/, kind: "async" },
{ re: /==>|===/, kind: "sync" },
{ re: /-->|---/, kind: "sync" },
];
interface ParsedRef {
id: string;
node?: GraphNode;
}
/**
* Parse a single node token like `A`, `A[Label]`, `db[(Orders)]`, `d{Choose}`.
* Returns the id and, when the token declares a shape/label, a GraphNode.
*/
function parseNodeToken(token: string): ParsedRef | null {
const t = token.trim();
if (t === "") return null;
for (const shape of SHAPES) {
const oi = t.indexOf(shape.open);
if (oi <= 0) continue;
if (!t.endsWith(shape.close)) continue;
const id = t.slice(0, oi).trim();
const label = cleanLabel(t.slice(oi + shape.open.length, t.length - shape.close.length));
if (!id) return null;
return { id, node: { id, label: label || id, kind: shape.kind } };
}
// Bare id (no shape declared here — may be defined elsewhere).
if (/^[A-Za-z0-9_.-]+$/.test(t)) return { id: t };
return null;
}
/**
* Split a connection line into [leftToken, arrowSegment, rightToken]. Returns
* null if the line has no arrow. The arrow segment may embed a label as
* `-->|text|` or `-- text -->`.
*/
function splitConnection(
line: string,
): { left: string; right: string; kind: string; label?: string } | null {
for (const arrow of ARROWS) {
// Find the arrow occurrence. Support a mid-arrow label: `A -- text --> B`.
const m = arrow.re.exec(line);
if (!m) continue;
const idx = m.index;
let left = line.slice(0, idx).trim();
let rest = line.slice(idx + m[0].length).trim();
let label: string | undefined;
// Pipe label: `-->|HTTPS| B`.
const pipe = /^\|([^|]*)\|\s*(.*)$/.exec(rest);
if (pipe) {
label = cleanLabel(pipe[1]);
rest = pipe[2].trim();
}
// Mid-arrow label on the left side: `A -- text` before the arrow half.
const midLeft = /^(.*?)\s*--\s*(.+)$/.exec(left);
if (!label && midLeft && /-\.|--|==/.test(line.slice(0, idx))) {
// Only treat as a label when there's clearly text after `--`.
if (!/[\[\](){}]/.test(midLeft[2])) {
left = midLeft[1].trim();
label = cleanLabel(midLeft[2]);
}
}
if (!left || !rest) return null;
return { left, right: rest, kind: arrow.kind, label };
}
return null;
}
/**
* Parse Mermaid flowchart text into a Graph. Handles the header
* (`flowchart LR` / `graph TD`), `subgraph <id>[title] … end` blocks (-> groups),
* node declarations, and connection lines. Throws MermaidParseError for a
* non-flowchart diagram or when nothing parses.
*/
export function mermaidToGraph(mermaid: string): Graph {
if (typeof mermaid !== "string" || mermaid.trim() === "") {
throw new MermaidParseError("empty mermaid input");
}
// Size guards FIRST — bound the raw input before building any intermediate.
if (mermaid.length > MAX_MERMAID_CHARS) {
throw new MermaidParseError(
`input is ${mermaid.length} chars (max ${MAX_MERMAID_CHARS}); split the diagram or use drawioFromGraph`,
);
}
const rawLines = mermaid.split(/\r?\n/);
if (rawLines.length > MAX_MERMAID_LINES) {
throw new MermaidParseError(
`input has ${rawLines.length} lines (max ${MAX_MERMAID_LINES}); split the diagram or use drawioFromGraph`,
);
}
const nodes = new Map<string, GraphNode>();
const groups: GraphGroup[] = [];
const edges: GraphEdge[] = [];
let direction: Graph["direction"] = "LR";
let sawHeader = false;
// Stack of active subgraph ids (nesting); the top is the current group.
const groupStack: string[] = [];
let anonGroup = 0;
const ensureNode = (ref: ParsedRef) => {
const existing = nodes.get(ref.id);
if (ref.node) {
if (existing) {
// Fill in a label/kind if this token declared a shape and the prior didn't.
if (existing.label === existing.id && ref.node.label !== ref.node.id)
existing.label = ref.node.label;
if (!existing.kind) existing.kind = ref.node.kind;
} else {
nodes.set(ref.id, { ...ref.node });
}
} else if (!existing) {
nodes.set(ref.id, { id: ref.id, label: ref.id, kind: "service" });
}
// Assign to the current subgraph if inside one and not yet grouped.
const cur = groupStack[groupStack.length - 1];
const n = nodes.get(ref.id)!;
if (cur && n.group == null) n.group = cur;
};
for (const raw of rawLines) {
let line = raw.trim();
if (line === "" || line.startsWith("%%")) continue; // blank / comment
// Header.
const header = /^(flowchart|graph)\s+([A-Za-z]{2})\b/.exec(line);
if (header) {
sawHeader = true;
const dir = DIRECTIONS[header[2].toUpperCase()];
if (dir) direction = dir;
continue;
}
if (/^(sequenceDiagram|classDiagram|stateDiagram|erDiagram|gantt|pie|journey)\b/.test(line)) {
throw new MermaidParseError(
`only 'flowchart'/'graph' is supported (got '${line.split(/\s+/)[0]}'); use drawioFromGraph instead`,
);
}
// Subgraph open: `subgraph id [Title]` or `subgraph Title`.
const sg = /^subgraph\s+(.+)$/.exec(line);
if (sg) {
const spec = sg[1].trim();
let id: string;
let label: string;
const bracket = /^([A-Za-z0-9_.-]+)\s*\[(.+)\]$/.exec(spec);
if (bracket) {
id = bracket[1];
label = cleanLabel(bracket[2]);
} else if (/^[A-Za-z0-9_.-]+$/.test(spec)) {
id = spec;
label = spec;
} else {
id = `sg${anonGroup++}`;
label = cleanLabel(spec);
}
if (!groups.some((g) => g.id === id)) {
if (groups.length >= MAX_MERMAID_GROUPS) {
throw new MermaidParseError(
`too many subgraphs (max ${MAX_MERMAID_GROUPS}); use drawioFromGraph for a diagram this large`,
);
}
groups.push({ id, label, kind: "group" });
}
groupStack.push(id);
continue;
}
if (/^end\b/.test(line)) {
groupStack.pop();
continue;
}
// `direction LR` inside a subgraph — apply to the top-level direction.
const innerDir = /^direction\s+([A-Za-z]{2})\b/.exec(line);
if (innerDir) {
const dir = DIRECTIONS[innerDir[1].toUpperCase()];
if (dir) direction = dir;
continue;
}
// Style/class/click directives: ignore (no visual mapping in our palette).
if (/^(style|classDef|class|click|linkStyle)\b/.test(line)) continue;
// Strip a trailing semicolon.
if (line.endsWith(";")) line = line.slice(0, -1).trim();
// Connection line (possibly chained: A --> B --> C).
const conn = splitConnection(line);
if (conn) {
// Handle a simple chain by re-splitting the right side.
let leftTok = conn.left;
let seg: typeof conn | null = conn;
let guard = 0;
while (seg) {
if (guard++ >= MAX_CHAIN_NODES) {
// Don't silently drop the tail of an over-long chain — surface it so
// the model knows the diagram was too large rather than getting a
// quietly-truncated result.
throw new MermaidParseError(
`a single connection chain exceeds ${MAX_CHAIN_NODES} nodes; split it or use drawioFromGraph`,
);
}
const leftRef = parseNodeToken(leftTok);
// The right side may itself contain another arrow (a chain).
const nextSeg = splitConnection(seg.right);
const rightTokenStr = nextSeg ? seg.right.slice(0, splitIndex(seg.right)) : seg.right;
const rightRef = parseNodeToken(nextSeg ? nextSeg.left : seg.right);
if (leftRef && rightRef) {
ensureNode(leftRef);
ensureNode(rightRef);
edges.push({
from: leftRef.id,
to: rightRef.id,
label: seg.label,
kind: seg.kind,
});
}
if (!nextSeg) break;
leftTok = nextSeg.left;
seg = nextSeg;
void rightTokenStr;
}
continue;
}
// Standalone node declaration `A[Label]` OR a bare member ref `C` inside a
// subgraph (which claims that node for the current group).
const nodeRef = parseNodeToken(line);
if (nodeRef && (nodeRef.node || groupStack.length > 0)) {
ensureNode(nodeRef);
continue;
}
// Unknown line: skip silently (robustness over strictness).
}
if (!sawHeader && nodes.size === 0) {
throw new MermaidParseError(
"input does not look like a mermaid flowchart (no 'flowchart'/'graph' header and no nodes)",
);
}
if (nodes.size === 0) {
throw new MermaidParseError("no nodes parsed from the flowchart");
}
const graph: Graph = {
nodes: Array.from(nodes.values()),
direction,
};
if (groups.length > 0) graph.groups = groups;
if (edges.length > 0) graph.edges = edges;
return graph;
}
/** Index of the first arrow in a segment (for chain splitting). */
function splitIndex(s: string): number {
let best = -1;
for (const arrow of ARROWS) {
const m = arrow.re.exec(s);
if (m && (best === -1 || m.index < best)) best = m.index;
}
return best === -1 ? s.length : best;
}
+151
View File
@@ -0,0 +1,151 @@
// Semantic color/line presets for the graph tools (issue #425, stage 3). The
// PALETTE is DATA (packages/mcp/data/drawio-presets.json), not code: a node
// `kind` maps to a { fillColor, strokeColor, fontColor } slot and an edge `kind`
// maps to line-style props, per named preset (`default` / `dark` /
// `colorblind-safe`). This module only loads that data and turns a slot into a
// draw.io style fragment. The INVARIANT of the graph tools is that the model
// never sees a style string — it names a `kind`, the server picks the slot.
//
// Loading mirrors drawio-shapes.ts: the JSON is read once via `import.meta.url`
// relative to the built module. That is why this module (and drawio-graph.ts
// which imports it) is reached ONLY through client.ts's ESM build and never
// value-imported into the zod-agnostic tool-specs.ts (which the in-app server
// type-checks under module:commonjs, where `import.meta` is a TS1343 error).
import { readFileSync } from "node:fs";
/** A node color slot: the three draw.io color values for a `kind`. */
export interface NodeSlot {
fillColor: string;
strokeColor: string;
fontColor: string;
}
/** An edge line style: the extra style props appended for an edge `kind`. */
export interface EdgeStyle {
props: string;
}
export interface PresetData {
canvasDark: boolean;
okabeIto?: string[];
nodes: Record<string, NodeSlot>;
edges: Record<string, EdgeStyle>;
edgeDefault: { strokeColor: string; fontColor: string };
group: { strokeColor: string; fontColor: string };
}
interface PresetsFile {
presets: Record<string, PresetData>;
}
/** The three shipped preset names. */
export const PRESET_NAMES = ["default", "dark", "colorblind-safe"] as const;
export type PresetName = (typeof PRESET_NAMES)[number];
/** Every node `kind` the base palette defines (also the generic-shape kinds). */
export const NODE_KINDS = [
"service",
"db",
"queue",
"gateway",
"error",
"external",
"security",
] as const;
export type NodeKind = (typeof NODE_KINDS)[number];
/** Edge `kind`s the palette styles; anything else falls back to `sync`. */
export const EDGE_KINDS = ["sync", "async", "error"] as const;
export type EdgeKind = (typeof EDGE_KINDS)[number];
let _presets: Record<string, PresetData> | null = null;
function presetsPath(): URL {
// build/lib/drawio-presets.js -> ../../data/drawio-presets.json
return new URL("../../data/drawio-presets.json", import.meta.url);
}
/** Load + parse the bundled preset table once, then cache it. */
export function loadPresets(): Record<string, PresetData> {
if (_presets) return _presets;
const json = readFileSync(presetsPath(), "utf-8");
const parsed = JSON.parse(json) as PresetsFile;
_presets = parsed.presets;
return _presets;
}
/** Resolve a preset by name, defaulting to `default` for an unknown name. */
export function getPreset(name?: string): PresetData {
const presets = loadPresets();
if (name && presets[name]) return presets[name];
return presets["default"];
}
/** The slot for a node `kind` in a preset, falling back to `service`. */
export function nodeSlot(preset: PresetData, kind?: string): NodeSlot {
if (kind && preset.nodes[kind]) return preset.nodes[kind];
return preset.nodes["service"];
}
/**
* Build the draw.io style string for a GENERIC (no-icon) node of a given kind.
* A rounded rectangle carrying the slot's fill/stroke/font. `whiteSpace=wrap`
* and `html=1` let a long label wrap inside the shape (the assembler also sizes
* the shape to the label, so the linter's label-overflow warning never fires).
*/
export function genericNodeStyle(preset: PresetData, kind?: string): string {
const s = nodeSlot(preset, kind);
return (
`rounded=1;whiteSpace=wrap;html=1;` +
`fillColor=${s.fillColor};strokeColor=${s.strokeColor};fontColor=${s.fontColor};`
);
}
/**
* Overlay the preset's node slot colors onto a resolved ICON style-string
* (from the shape catalog). An AWS/Azure icon carries its OWN mandatory
* fill/stroke (the category color / white outline) that MUST NOT be recolored,
* so for an icon we only ensure a readable fontColor when the preset is dark;
* otherwise the icon style is returned verbatim. Keeping the icon's own colors
* is deliberate: recoloring an AWS service icon breaks its category semantics.
*/
export function iconNodeStyle(preset: PresetData, iconStyle: string): string {
if (!preset.canvasDark) return iconStyle;
// On a dark canvas an icon's fontColor is usually a dark ink that vanishes;
// append a light fontColor (icons put their label BELOW the glyph, so this
// only affects the caption, never the glyph fill).
if (/fontColor=/.test(iconStyle)) {
return iconStyle.replace(/fontColor=[^;]*/, "fontColor=#e0e0e0");
}
return iconStyle + (iconStyle.endsWith(";") ? "" : ";") + "fontColor=#e0e0e0;";
}
/**
* Build the draw.io style for an edge of a given `kind`. Base is an orthogonal
* connector (edgeStyle=orthogonalEdgeStyle) with rounded corners and an open
* arrowhead, plus the preset's default stroke/font, then the kind's extra props
* (dashed / colored) overlaid. An unknown kind falls back to `sync` (solid).
*/
export function edgeStyle(preset: PresetData, kind?: string): string {
const k = kind && preset.edges[kind] ? kind : "sync";
const base =
`edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=open;` +
`strokeColor=${preset.edgeDefault.strokeColor};fontColor=${preset.edgeDefault.fontColor};`;
return base + preset.edges[k].props;
}
/**
* Group (container) style: ALWAYS transparent (`fillColor=none;container=1;`)
* per the spec, carrying the preset's group stroke/font. `dropTarget=1` marks it
* a drop target in the editor; `verticalAlign=top;align=left;spacingLeft=8;` puts
* the group label in the top-left like draw.io's own boundary containers.
*/
export function groupStyle(preset: PresetData): string {
return (
`rounded=0;whiteSpace=wrap;html=1;` +
`fillColor=none;container=1;dropTarget=1;collapsible=0;` +
`strokeColor=${preset.group.strokeColor};fontColor=${preset.group.fontColor};` +
`verticalAlign=top;align=left;spacingLeft=8;spacingTop=4;`
);
}
+1 -1
View File
@@ -1,4 +1,4 @@
// Verified draw.io shape catalog for the `drawio_shapes` tool (issue #424, // Verified draw.io shape catalog for the `drawioShapes` tool (issue #424,
// stage 2). This is the fix for AI-generated diagrams' #1 defect: guessed // stage 2). This is the fix for AI-generated diagrams' #1 defect: guessed
// `shape=mxgraph.*` names that render as EMPTY BOXES because the stencil does // `shape=mxgraph.*` names that render as EMPTY BOXES because the stencil does
// not exist. Instead of guessing, the model queries this catalog and gets back // not exist. Instead of guessing, the model queries this catalog and gets back
+1 -1
View File
@@ -1070,7 +1070,7 @@ export function prepareModel(inputXml: string): PreparedModel {
}; };
} }
/** Cell count of a decoded model (user cells only) — used by drawio_get meta. */ /** Cell count of a decoded model (user cells only) — used by drawioGet meta. */
export function countUserCells(modelXml: string): number { export function countUserCells(modelXml: string): number {
return parseCells(modelXml).filter((c) => c.id !== "0" && c.id !== "1").length; return parseCells(modelXml).filter((c) => c.id !== "0" && c.id !== "1").length;
} }
+24 -8
View File
@@ -83,16 +83,32 @@ export function filterComment(comment: any, markdownContent?: string) {
}; };
} }
// Map one server search hit to the MCP output contract (#443):
// { pageId, title, path, snippet, score }
//
// INVARIANT: the only page identifier exposed is `pageId` (the server `id`
// UUID). The server also carries `slugId` — it is NEVER surfaced.
//
// GRACEFUL DEGRADATION: against a stock upstream server the opt-in lookup DTO
// fields are stripped, so the response is the legacy FTS shape (no path/snippet/
// score, a `highlight` + `rank` instead). We synthesize the contract from
// whatever is present: `snippet` falls back to the FTS `highlight`, `score` to
// the FTS `rank`, and `path` to [] (upstream has no path). This keeps the tool
// usable even when the server has not been upgraded.
export function filterSearchResult(result: any) { export function filterSearchResult(result: any) {
return { return {
id: result.id, pageId: result.id,
title: result.title, title: result.title,
parentPageId: result.parentPageId, path: Array.isArray(result.path) ? result.path : [],
createdAt: result.createdAt, snippet:
updatedAt: result.updatedAt, typeof result.snippet === "string"
rank: result.rank, ? result.snippet
highlight: result.highlight, : (result.highlight ?? ""),
spaceId: result.space?.id, score:
spaceName: result.space?.name, typeof result.score === "number"
? result.score
: typeof result.rank === "number"
? result.rank
: 0,
}; };
} }
@@ -16,8 +16,8 @@
* `insertInlineFootnote` live in `@docmost/prosemirror-markdown` (next to the * `insertInlineFootnote` live in `@docmost/prosemirror-markdown` (next to the
* importer's `assembleFootnotes`, #414), so this file stays a pure mirror. * importer's `assembleFootnotes`, #414), so this file stays a pure mirror.
* *
* Why it exists: every NON-editor write path (markdown import, update_page_json, * Why it exists: every NON-editor write path (markdown import, updatePageJson,
* docmost_transform, insert_footnote) builds ProseMirror JSON directly, so the * docmostTransform, insertFootnote) builds ProseMirror JSON directly, so the
* editor's footnote plugins never run and the canonical topology (sequential * editor's footnote plugins never run and the canonical topology (sequential
* numbering by first reference, one trailing list, no orphans, no raw `[^id]`) * numbering by first reference, one trailing list, no orphans, no raw `[^id]`)
* was never enforced. Running this at the end of every write path closes that * was never enforced. Running this at the end of every write path closes that
@@ -28,8 +28,8 @@
* `canonicalizeFootnotes(doc)` before writing the current callers are * `canonicalizeFootnotes(doc)` before writing the current callers are
* `markdownToProseMirrorCanonical` (page markdown import/update; the plain * `markdownToProseMirrorCanonical` (page markdown import/update; the plain
* `markdownToProseMirror` used for COMMENT bodies must NOT, or it would drop a * `markdownToProseMirror` used for COMMENT bodies must NOT, or it would drop a
* reference-less definition), `update_page_json`, `docmost_transform`, * reference-less definition), `updatePageJson`, `docmostTransform`,
* `insert_footnote`, and `copy_page_content`. Append/prepend FRAGMENT writes MUST * `insertFootnote`, and `copyPageContent`. Append/prepend FRAGMENT writes MUST
* NOT canonicalize. This is deliberately per-call-site (the replace-vs-fragment * NOT canonicalize. This is deliberately per-call-site (the replace-vs-fragment
* and comment-vs-page nuances make a single naive wrapper unsafe). * and comment-vs-page nuances make a single naive wrapper unsafe).
*/ */
+8 -8
View File
@@ -283,7 +283,7 @@ export function applyTextEdits(
for (const edit of edits) { for (const edit of edits) {
if (!edit.find) throw new Error("edit.find must be a non-empty string"); if (!edit.find) throw new Error("edit.find must be a non-empty string");
// HARD-REFUSE formatting changes. edit_page_text edits PLAIN TEXT only and // HARD-REFUSE formatting changes. editPageText edits PLAIN TEXT only and
// writes the replacement verbatim, so it cannot add/remove marks. We refuse // writes the replacement verbatim, so it cannot add/remove marks. We refuse
// only a pure formatting TOGGLE: find and replace differ ONLY by balanced // only a pure formatting TOGGLE: find and replace differ ONLY by balanced
// markdown markers (e.g. find:"~~$69~~" / replace:"$69", or find:"M5Stack" / // markdown markers (e.g. find:"~~$69~~" / replace:"$69", or find:"M5Stack" /
@@ -304,22 +304,22 @@ export function applyTextEdits(
failed.push({ failed.push({
find: edit.find, find: edit.find,
reason: reason:
"edit_page_text edits plain text only and cannot add or remove formatting marks (bold/italic/strike/code/link); it writes the replacement as LITERAL text. This edit looks like a formatting change (markdown markers in find/replace). To change marks, read the block with get_page_json and use patch_node (or update_page_json) to set the node's marks array.", "editPageText edits plain text only and cannot add or remove formatting marks (bold/italic/strike/code/link); it writes the replacement as LITERAL text. This edit looks like a formatting change (markdown markers in find/replace). To change marks, read the block with getPageJson and use patchNode (or updatePageJson) to set the node's marks array.",
}); });
continue; continue;
} }
// HARD-REFUSE inline footnote tokens (#410). `^[...]` in a `replace` is // HARD-REFUSE inline footnote tokens (#410). `^[...]` in a `replace` is
// markdown that only becomes a real footnote when a whole markdown body is // markdown that only becomes a real footnote when a whole markdown body is
// written (create_page / update_page_content / import_page_markdown). Written // written (createPage / update_page_content / importPageMarkdown). Written
// through edit_page_text it stays a LITERAL string in the text — the exact // through editPageText it stays a LITERAL string in the text — the exact
// failure mode #410 fixes — so refuse it here (defense-in-depth) and point the // failure mode #410 fixes — so refuse it here (defense-in-depth) and point the
// caller at insert_footnote, mirroring the formatting-marker refusal above. // caller at insertFootnote, mirroring the formatting-marker refusal above.
if (/\^\[[\s\S]*?\]/.test(edit.replace)) { if (/\^\[[\s\S]*?\]/.test(edit.replace)) {
failed.push({ failed.push({
find: edit.find, find: edit.find,
reason: reason:
"edit_page_text writes the replacement as LITERAL text, so a `^[...]` footnote token does not parse into a real footnote (it would appear verbatim in the page). To add a footnote to existing text, use insert_footnote (anchorText = where, text = the note).", "editPageText writes the replacement as LITERAL text, so a `^[...]` footnote token does not parse into a real footnote (it would appear verbatim in the page). To add a footnote to existing text, use insertFootnote (anchorText = where, text = the note).",
}); });
continue; continue;
} }
@@ -381,12 +381,12 @@ export function applyTextEdits(
let reason: string; let reason: string;
if (existsAcrossAtom) { if (existsAcrossAtom) {
reason = reason =
"match crosses a non-text inline node (image/break/mention); use update_page_json for structural changes."; "match crosses a non-text inline node (image/break/mention); use updatePageJson for structural changes.";
} else { } else {
// Append a bounded "closest text" hint: find the FIRST block that // Append a bounded "closest text" hint: find the FIRST block that
// contains the longest whitespace-delimited token (>= 3 chars) of the // contains the longest whitespace-delimited token (>= 3 chars) of the
// (stripped, then raw) locator, and quote that block's plain text. Shared // (stripped, then raw) locator, and quote that block's plain text. Shared
// with create_comment via closestBlockHint so both give the same hint. // with createComment via closestBlockHint so both give the same hint.
reason = "text not found in the document." + closestBlockHint(blockPlain, edit.find); reason = "text not found in the document." + closestBlockHint(blockPlain, edit.find);
} }
failed.push({ find: edit.find, reason }); failed.push({ find: edit.find, reason });
+243
View File
@@ -0,0 +1,243 @@
/**
* Single-BLOCK markdown fragment support for `patch_node` / `insert_node`
* (#413). These tools accept EITHER a raw ProseMirror `node` (fine attr/mark
* work) OR a `markdown` string (the recommended default): a small markdown
* fragment is run through the canonical importer, yielding the SAME topology a
* full-page markdown import would so a block written via markdown is
* canonically identical to the same content imported whole (no "second canon").
*
* The importer produces a full `{type:"doc", content:[...blocks..., footnotesList?]}`.
* A fragment write needs the BLOCKS separately from the footnote DEFINITIONS so
* the caller can splice the blocks into the live document and merge the
* definitions into the page's TAIL footnote list via the existing footnote
* machinery (`insertInlineFootnote`'s `appendDefinition` + `canonicalizeFootnotes`).
*
* Footnote id-collision safety: the importer assigns sequential ids (`fn-1`,
* `fn-2`, ) starting from 1 for EVERY fragment, so a fragment's `fn-1` would
* collide with an existing page footnote also numbered `fn-1` and
* `canonicalizeFootnotes` matches references to definitions BY id, so the
* fragment's reference would silently re-hang onto the page's unrelated
* definition. To make the merge safe regardless of the page's current numbering,
* every fragment footnote id is REMAPPED to a fresh uuid (via the importer's own
* `generateFootnoteId`) across BOTH the references (inside the blocks) and the
* definitions before either is handed back. Content-identical notes still merge
* downstream via `normalizeAndMergeFootnotes` (content-key), and the whole doc is
* renumbered by `canonicalizeFootnotes`, so the caller-visible numbering stays
* canonical.
*/
import { markdownToProseMirror } from "./collaboration.js";
import { generateFootnoteId } from "@docmost/prosemirror-markdown";
import { docmostSchema } from "./docmost-schema.js";
/** True if `value` is a non-null, non-array object. */
function isObject(value: any): value is Record<string, any> {
return value != null && typeof value === "object" && !Array.isArray(value);
}
/**
* Deep-walk `node` collecting every footnote id it uses (on `footnoteReference`
* and `footnoteDefinition` nodes) and build a stable OLD->NEW remap, minting a
* fresh uuid per distinct old id. The map is shared across a fragment's blocks
* and definitions so a reference and its definition receive the SAME new id.
*/
function buildFootnoteIdRemap(nodes: any[]): Map<string, string> {
const remap = new Map<string, string>();
const visit = (node: any): void => {
if (!isObject(node)) return;
if (
(node.type === "footnoteReference" ||
node.type === "footnoteDefinition") &&
isObject(node.attrs) &&
typeof node.attrs.id === "string" &&
node.attrs.id !== ""
) {
if (!remap.has(node.attrs.id)) {
remap.set(node.attrs.id, generateFootnoteId());
}
}
if (Array.isArray(node.content)) {
for (const child of node.content) visit(child);
}
};
for (const n of nodes) visit(n);
return remap;
}
/** Rewrite every footnote id in `node` IN PLACE using `remap` (deep). */
function applyFootnoteIdRemap(node: any, remap: Map<string, string>): void {
if (!isObject(node)) return;
if (
(node.type === "footnoteReference" || node.type === "footnoteDefinition") &&
isObject(node.attrs) &&
typeof node.attrs.id === "string"
) {
const next = remap.get(node.attrs.id);
if (next) node.attrs.id = next;
}
if (Array.isArray(node.content)) {
for (const child of node.content) applyFootnoteIdRemap(child, remap);
}
}
/**
* Generate a short random block id for an imported block that arrives without one
* (the markdown importer emits `attrs.id: null`). Mirrors the mcp `freshId`
* convention (base36 random, unique within one document). The patch path then
* OVERWRITES the first block's id with the target id; every other block keeps the
* fresh id minted here so a 1 -> N section rewrite yields addressable,
* comment-anchorable blocks rather than a run of null-id paragraphs.
*/
function freshBlockId(): string {
return (
Math.random().toString(36).slice(2, 12) +
Math.random().toString(36).slice(2, 6)
);
}
/**
* Assign a fresh id to every top-level block whose `attrs.id` is null/missing,
* IN PLACE. Only the block's own id is touched (not descendants those keep the
* importer's structure). Ensures each imported block is independently addressable.
*/
function assignFreshBlockIds(blocks: any[]): void {
for (const b of blocks) {
if (!isObject(b)) continue;
if (!isObject(b.attrs)) b.attrs = {};
if (b.attrs.id == null || b.attrs.id === "") {
b.attrs.id = freshBlockId();
}
}
}
/** The parsed shape of a markdown fragment: its blocks + footnote definitions. */
export interface MarkdownFragment {
/** Top-level blocks, in order, with the trailing `footnotesList` removed. */
blocks: any[];
/**
* The `footnoteDefinition` nodes lifted from the imported `footnotesList`, with
* ids already remapped to match the references left inside `blocks`. Empty when
* the fragment used no footnotes.
*/
definitions: any[];
}
/**
* Import a markdown fragment and return its blocks separately from its footnote
* definitions, with all footnote ids remapped to fresh uuids (see the file
* header). The importer's `^[body]` inline-footnote handling is used verbatim
* `^[...]` in the fragment is a first-class footnote, NOT rejected so the
* markdown path matches the full-page import exactly.
*
* Throws when the fragment imports to zero blocks (an empty / whitespace-only
* markdown string is not a valid block write).
*/
export async function importMarkdownFragment(
markdown: string,
): Promise<MarkdownFragment> {
const doc = await markdownToProseMirror(markdown);
const content: any[] = Array.isArray(doc?.content) ? doc.content : [];
const blocks: any[] = [];
const definitions: any[] = [];
for (const node of content) {
if (isObject(node) && node.type === "footnotesList") {
// Lift the definitions out of the list; the list wrapper itself is
// reconstructed on the page by the canonicalizer after the merge.
if (Array.isArray(node.content)) {
for (const def of node.content) {
if (isObject(def) && def.type === "footnoteDefinition") {
definitions.push(def);
}
}
}
continue;
}
blocks.push(node);
}
if (blocks.length === 0) {
throw new Error(
"markdown fragment produced no blocks — provide non-empty markdown, or use `node` for a raw ProseMirror node",
);
}
// Remap footnote ids across BOTH blocks and definitions so a fragment `fn-1`
// cannot collide with a page footnote of the same number.
const remap = buildFootnoteIdRemap([...blocks, ...definitions]);
if (remap.size > 0) {
for (const b of blocks) applyFootnoteIdRemap(b, remap);
for (const d of definitions) applyFootnoteIdRemap(d, remap);
}
// Every top-level block needs a stable id (the importer leaves them null). The
// patch path OVERWRITES the first block's id with the target id afterwards.
assignFreshBlockIds(blocks);
return { blocks, definitions };
}
/**
* True when `type` is a valid TOP-LEVEL child of the document node per the
* canonical schema's content model i.e. `get_node` can serialize it to
* markdown by wrapping it in `{type:"doc",content:[node]}`. Derived from the
* schema's `doc` contentMatch (NOT a hand-written type list) so it tracks the
* schema automatically: `tableRow`/`tableCell`/`tableHeader` (addressed only via
* `#<index>`) are NOT doc children and yield false, so `get_node` auto-falls back
* to JSON for them.
*/
export function canBeDocChild(type: string | undefined): boolean {
if (typeof type !== "string") return false;
const nodeType = docmostSchema.nodes[type];
if (!nodeType) return false;
return docmostSchema.nodes.doc.contentMatch.matchType(nodeType) != null;
}
/**
* Table-cell attributes that CANNOT survive a markdown round-trip: the converter
* emits colspan/rowspan (and align) as HTML `<table>` cell attrs, but silently
* drops `colwidth`, `backgroundColor`, and `backgroundColorName`. A markdown
* `patch_node` on a block that carries any of these (a merged / colored /
* fixed-width cell) would therefore lose them so it is REJECTED, pointing the
* caller at the table tools or the raw-`node` JSON path. `align` is intentionally
* absent: it round-trips as GFM alignment.
*/
function cellCarriesUnrepresentableAttrs(node: any): boolean {
if (!isObject(node)) return false;
if (node.type !== "tableCell" && node.type !== "tableHeader") return false;
const a = isObject(node.attrs) ? node.attrs : {};
if ((a.colspan ?? 1) > 1) return true;
if ((a.rowspan ?? 1) > 1) return true;
if (a.colwidth != null) return true;
if (a.backgroundColor != null) return true;
if (a.backgroundColorName != null) return true;
return false;
}
/**
* Scan a target block (the node being replaced) for any table cell carrying an
* attribute markdown cannot represent (colspan/rowspan/colwidth/background). When
* one is found, return a human-readable list of the offending attr NAMES so the
* caller can build an actionable rejection message; return null when the block is
* safe to rewrite from markdown. Deep a colored cell nested inside a table
* inside a callout is still caught.
*/
export function findUnrepresentableTableAttrs(node: any): string | null {
const found = new Set<string>();
const visit = (n: any): void => {
if (!isObject(n)) return;
if (cellCarriesUnrepresentableAttrs(n)) {
const a = isObject(n.attrs) ? n.attrs : {};
if ((a.colspan ?? 1) > 1) found.add("colspan");
if ((a.rowspan ?? 1) > 1) found.add("rowspan");
if (a.colwidth != null) found.add("colwidth");
if (a.backgroundColor != null) found.add("backgroundColor");
if (a.backgroundColorName != null) found.add("backgroundColorName");
}
if (Array.isArray(n.content)) {
for (const child of n.content) visit(child);
}
};
visit(node);
return found.size > 0 ? Array.from(found).sort().join(", ") : null;
}
+33
View File
@@ -10,6 +10,20 @@
const chains = new Map<string, Promise<unknown>>(); const chains = new Map<string, Promise<unknown>>();
// Canonical UUID shape (versions 1–8, matching the `uuid` package's `validate`
// that the server's isValidUUID uses). This is the SINGLE source of truth for
// "is this a canonical page UUID?" in the MCP: client.ts's resolvePageId
// imports isUuid from here to decide whether a pageId already IS a UUID (and so
// needs no /pages/info round-trip). page.repo.ts treats any non-UUID pageId as
// a slugId; a 10-char nanoid slugId never contains dashes, so it can never be
// misread as a UUID here.
export const UUID_RE =
/^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
export function isUuid(value: string): boolean {
return typeof value === "string" && UUID_RE.test(value);
}
// The returned promise carries the real result/rejection of `fn` and MUST be // The returned promise carries the real result/rejection of `fn` and MUST be
// awaited/handled by the caller; only the internal chaining tail swallows // awaited/handled by the caller; only the internal chaining tail swallows
// errors (purely to gate ordering). // errors (purely to gate ordering).
@@ -17,6 +31,25 @@ export function withPageLock<T>(
pageId: string, pageId: string,
fn: () => Promise<T>, fn: () => Promise<T>,
): Promise<T> { ): Promise<T> {
// STRUCTURAL INVARIANT (issue #449, "resolve-then-lock"): the mutex key MUST
// be the canonical page UUID, never a raw slugId. The whole write path relies
// on the lock key AND the CollabSession cache key being the resolved UUID
// (#260) — if a future write method forgot to call resolvePageId and locked
// under a slugId, two writes to the same page would take DIFFERENT mutex keys
// and silently lose serialization (clobbering live human edits). This was an
// invariant enforced only by comments/convention; assert it in CODE so the
// violation fails fast and loud at the lock instead of corrupting data in
// prod. The centralizing helper (mutatePageContent/replacePageContent) already
// guards a raw-input caller, but this backstop catches ANY path.
if (!isUuid(pageId)) {
throw new Error(
`withPageLock: key must be a canonical page UUID, got '${pageId}'. ` +
`The write path must resolvePageId(pageId) BEFORE locking so the ` +
`mutex/CollabSession cache key is the UUID (invariant "resolve-then-` +
`lock", #260/#449). A slugId or other non-UUID key would silently lose ` +
`per-page serialization.`,
);
}
// Wait for the previous op on this page; swallow its error so a failure does // Wait for the previous op on this page; swallow its error so a failure does
// not poison the queue for the next caller. // not poison the queue for the next caller.
const prev = (chains.get(pageId) ?? Promise.resolve()).catch(() => {}); const prev = (chains.get(pageId) ?? Promise.resolve()).catch(() => {});
+12 -12
View File
@@ -3,7 +3,7 @@
* *
* `searchInDoc(doc, query, opts)` finds every occurrence of a literal substring * `searchInDoc(doc, query, opts)` finds every occurrence of a literal substring
* (default) or a regular expression across the page's TEXT CONTAINERS and * (default) or a regular expression across the page's TEXT CONTAINERS and
* reports WHERE each match is the container's ref (for get_node/patch_node; * reports WHERE each match is the container's ref (for getNode/patchNode;
* see the SearchMatch.nodeId note for the `#<index>` caveat), the top-level * see the SearchMatch.nodeId note for the `#<index>` caveat), the top-level
* block index, and a short context window around the hit. It never touches the * block index, and a short context window around the hit. It never touches the
* network, the DB, or the schema mirror; like `comment-anchor.ts` it is * network, the DB, or the schema mirror; like `comment-anchor.ts` it is
@@ -69,24 +69,24 @@ export interface SearchOptions {
/** One located occurrence. */ /** One located occurrence. */
export interface SearchMatch { export interface SearchMatch {
/** /**
* The container's ref, for addressing the block with get_node/patch_node: its * The container's ref, for addressing the block with getNode/patchNode: its
* `attrs.id` when it has one, otherwise `#<topLevelIndex>` of the nearest * `attrs.id` when it has one, otherwise `#<topLevelIndex>` of the nearest
* top-level block. Table-cell/list-item paragraphs that carry no id fall back * top-level block. Table-cell/list-item paragraphs that carry no id fall back
* to the `#<index>` form. * to the `#<index>` form.
* *
* CAVEAT: the `#<index>` form is accepted by get_node (getNodeByRef resolves * CAVEAT: the `#<index>` form is accepted by getNode (getNodeByRef resolves
* it by top-level index) but NOT by patch_node (replaceNodeById resolves only * it by top-level index) but NOT by patchNode (replaceNodeById resolves only
* by `attrs.id`), so id-less table/cell content can be READ by this ref but * by `attrs.id`), so id-less table/cell content can be READ by this ref but
* not PATCHED by it. * not PATCHED by it.
* *
* To anchor a comment, do NOT pass this ref to create_comment it has no * To anchor a comment, do NOT pass this ref to createComment it has no
* nodeId parameter. A top-level comment needs an exact-text `selection` that * nodeId parameter. A top-level comment needs an exact-text `selection` that
* occurs once on the page (it fails if the text isn't found), so build a * occurs once on the page (it fails if the text isn't found), so build a
* UNIQUE `selection` from before+match+after and pass THAT as create_comment's * UNIQUE `selection` from before+match+after and pass THAT as createComment's
* `selection`. * `selection`.
*/ */
nodeId: string; nodeId: string;
/** The top-level block index (as in get_outline). */ /** The top-level block index (as in getOutline). */
blockIndex: number; blockIndex: number;
/** The container node's type (paragraph/heading/...). */ /** The container node's type (paragraph/heading/...). */
type: string | undefined; type: string | undefined;
@@ -188,12 +188,12 @@ export function searchInDoc(
// --- edge-case guards (fail loudly so the agent can correct the call) --- // --- edge-case guards (fail loudly so the agent can correct the call) ---
if (typeof query !== "string" || query.trim().length === 0) { if (typeof query !== "string" || query.trim().length === 0) {
throw new Error( throw new Error(
"search_in_page: query is empty — pass the text (or regex) to look for.", "searchInPage: query is empty — pass the text (or regex) to look for.",
); );
} }
if (query.length > MAX_PATTERN_LENGTH) { if (query.length > MAX_PATTERN_LENGTH) {
throw new Error( throw new Error(
`search_in_page: query is too long (${query.length} chars; max ${MAX_PATTERN_LENGTH}). Shorten the search text/pattern.`, `searchInPage: query is too long (${query.length} chars; max ${MAX_PATTERN_LENGTH}). Shorten the search text/pattern.`,
); );
} }
@@ -212,7 +212,7 @@ export function searchInDoc(
re = new RE2(query, caseSensitive ? "g" : "gi"); re = new RE2(query, caseSensitive ? "g" : "gi");
} catch (e) { } catch (e) {
throw new Error( throw new Error(
`search_in_page: invalid or unsupported regular expression: ${ `searchInPage: invalid or unsupported regular expression: ${
e instanceof Error ? e.message : String(e) e instanceof Error ? e.message : String(e)
} RE2 does not support lookaround ((?=)/(?<=)) or backreferences (\\1); rewrite the pattern without them.`, } RE2 does not support lookaround ((?=)/(?<=)) or backreferences (\\1); rewrite the pattern without them.`,
); );
@@ -237,9 +237,9 @@ export function searchInDoc(
// in a very long container. // in a very long container.
const text = blockPlainText(node); const text = blockPlainText(node);
// The container's own id addresses it verbatim in get_node/patch_node; a // The container's own id addresses it verbatim in getNode/patchNode; a
// container with no id (e.g. a table-cell paragraph) falls back to the // container with no id (e.g. a table-cell paragraph) falls back to the
// top-level block's #<index> (readable via get_node, but not patchable — // top-level block's #<index> (readable via getNode, but not patchable —
// see the SearchMatch.nodeId note). // see the SearchMatch.nodeId note).
const id = const id =
isObject(node.attrs) && typeof node.attrs.id === "string" && node.attrs.id.length > 0 isObject(node.attrs) && typeof node.attrs.id === "string" && node.attrs.id.length > 0
+1 -1
View File
@@ -117,7 +117,7 @@ export function stripInlineMarkdown(s: string): string {
/** /**
* Build a bounded "closest text" hint for an anchor/find MISS, shared by * Build a bounded "closest text" hint for an anchor/find MISS, shared by
* edit_page_text (json-edit) and create_comment (client) so both surface the * editPageText (json-edit) and createComment (client) so both surface the
* same self-correction affordance. * same self-correction affordance.
* *
* Take the longest whitespace-delimited token (>= 3 chars) of the locator * Take the longest whitespace-delimited token (>= 3 chars) of the locator
+56 -1
View File
@@ -740,7 +740,7 @@ export function insertInlineFootnote(
// subtree, so a reference is never glued inside an existing definition (which // subtree, so a reference is never glued inside an existing definition (which
// the canonicalizer would then drop as an orphan, losing that definition's // the canonicalizer would then drop as an orphan, losing that definition's
// prose); and forbidBlockTypes refuses codeBlocks (an inline atom there is a // prose); and forbidBlockTypes refuses codeBlocks (an inline atom there is a
// schema-invalid doc; insert_footnote skips validateDocStructure). // schema-invalid doc; insertFootnote skips validateDocStructure).
// When the only anchor match is in such a place, the insert is refused and the // When the only anchor match is in such a place, the insert is refused and the
// write aborts cleanly (inserted:false) instead of destroying content. // write aborts cleanly (inserted:false) instead of destroying content.
const boundaryIdx = Array.isArray(doc?.content) const boundaryIdx = Array.isArray(doc?.content)
@@ -774,6 +774,61 @@ export function insertInlineFootnote(
return { doc: working, inserted: true, footnoteId, reused }; return { doc: working, inserted: true, footnoteId, reused };
} }
/**
* Merge an ARRAY of footnote definitions (e.g. the definitions lifted from an
* imported markdown FRAGMENT) into `doc`\'s footnote list, then re-derive the
* canonical footnote topology the SAME two-step machinery `insertInlineFootnote`
* uses (`appendDefinition` -> `normalizeAndMergeFootnotes` -> `canonicalizeFootnotes`).
*
* The fragment\'s `footnoteReference` nodes are assumed to ALREADY be spliced into
* `doc` (inside the just-inserted blocks) with ids matching these definitions, so
* after appending the definitions the canonicalizer orders/numbers everything by
* first-reference order, merges content-identical notes, and drops any orphan.
* Same documented caveat as every other write path: full canonicalization drops a
* definition no reference points at.
*
* NOT merely a no-op when `definitions` is empty: it still canonicalizes when
* the (post-splice) `doc` carries footnote artifacts (a `footnotesList` or any
* `footnoteReference`), so a splice that removed the LAST referrer of a page
* footnote drops the now-orphaned definition matching a full page re-import
* (which always canonicalizes) and preserving the "canonically identical to the
* same content imported whole" invariant. A truly footnote-free doc (no artifacts
* and no definitions) is returned untouched the fast path, no clone. When the
* work runs it goes through the pure passes (which clone), so the caller\'s `doc`
* is not mutated.
*/
export function mergeFootnoteDefinitions(doc: any, definitions: any[]): any {
const defs = Array.isArray(definitions) ? definitions : [];
// True fast path ONLY when there is nothing to merge AND nothing to canonicalize
// away; otherwise fall through so an orphan left by a splice is still dropped.
if (defs.length === 0 && !hasFootnoteArtifacts(doc)) return doc;
// Clone before appending: `appendDefinition` mutates in place, and the caller
// must not see a half-merged doc if a later pass throws.
let working = clone(doc);
for (const def of defs) {
appendDefinition(working, def);
}
// #419: normalize + merge glyph-forked definitions before canonicalizing.
working = normalizeAndMergeFootnotes(working);
working = canonicalizeFootnotes(working);
return working;
}
/**
* True if `doc`'s tree contains any `footnotesList` node OR any
* `footnoteReference` node. Used to decide whether an empty-`definitions` merge
* must still canonicalize (to drop an orphan a splice left behind).
*/
function hasFootnoteArtifacts(doc: any): boolean {
let found = false;
walk(doc, (n) => {
if (isObject(n) && (n.type === "footnotesList" || n.type === "footnoteReference")) {
found = true;
}
});
return found;
}
/** /**
* Append a definition node so the canonicalizer can order/place it: into the * Append a definition node so the canonicalizer can order/place it: into the
* first existing footnotesList, or a new trailing list when none exists. * first existing footnotesList, or a new trailing list when none exists.
+85 -8
View File
@@ -1,11 +1,30 @@
/**
* Options for `buildPageTree`. Fully OPTIONAL so the existing call form
* `buildPageTree(nodes)` keeps its historic behaviour (lean `{id, slugId,
* title, children?}` output, no depth cut) unchanged.
*
* - `shape: "getTree"` emit the #443 `getTree` output node shape
* `{pageId, title, children?, hasChildren?}` instead of the lean
* `{id, slugId, title, children?}` shape. `slugId`/`icon`/`position` are
* never exposed (INVARIANT: only the UUID `pageId` leaves the MCP layer).
* - `maxDepth` trim the built tree to this many levels (root nodes are
* depth 1). Only meaningful together with `shape: "getTree"` (the lean shape
* has no `hasChildren` to signal a cut). See the depth logic below.
*/
export interface BuildPageTreeOptions {
shape?: "lean" | "getTree";
maxDepth?: number;
}
/** /**
* Pure tree-builder: turn a flat array of sidebar-style page nodes (as produced * Pure tree-builder: turn a flat array of sidebar-style page nodes (as produced
* by `enumerateSpacePages`) into a nested tree. * by `enumerateSpacePages`) into a nested tree.
* *
* Input: a flat array of nodes. Each node is expected to carry at least * Input: a flat array of nodes. Each node is expected to carry at least
* { id, slugId, title, position, parentPageId } (extra fields are ignored). * { id, slugId, title, position, parentPageId } (extra fields are ignored),
* plus a server `hasChildren` boolean used by the `getTree` shape below.
* *
* Output: an array of ROOT nodes, each shaped as * Output (default / `shape: "lean"`): an array of ROOT nodes, each shaped as
* { id, slugId, title, children? } * { id, slugId, title, children? }
* where `children` is the array of child nodes (same shape, recursively). The * where `children` is the array of child nodes (same shape, recursively). The
* `children` key is OMITTED entirely when a node has no children consistent * `children` key is OMITTED entirely when a node has no children consistent
@@ -13,6 +32,14 @@
* lean (nesting alone conveys the structure; parentPageId/position/hasChildren * lean (nesting alone conveys the structure; parentPageId/position/hasChildren
* are intentionally dropped from the output). * are intentionally dropped from the output).
* *
* Output (`shape: "getTree"`, the #443 tool shape): each node is
* { pageId, title, children?, hasChildren? }
* the server `id` is exposed as `pageId` (never `slugId`/`icon`/`position`).
* `children` is omitted for leaves and for nodes trimmed by `maxDepth`.
* `hasChildren: true` is set ONLY on a node whose children exist on the server
* (per the flat item's `hasChildren`) but were CUT by `maxDepth`; on leaves and
* on fully-expanded interior nodes the field is omitted (see `maxDepth` below).
*
* Linking rule: a node is attached as a child of `parentPageId` only when that * Linking rule: a node is attached as a child of `parentPageId` only when that
* parent id is actually present in the input. Otherwise including a null / * parent id is actually present in the input. Otherwise including a null /
* undefined `parentPageId`, or a parent that was capped out of the bounded walk * undefined `parentPageId`, or a parent that was capped out of the bounded walk
@@ -26,18 +53,42 @@
* fractional-index ASCII keys (e.g. "a0", "a1"). Nodes with a missing/undefined * fractional-index ASCII keys (e.g. "a0", "a1"). Nodes with a missing/undefined
* `position` sort last. * `position` sort last.
* *
* maxDepth (getTree shape only): the tree is built in FULL first, then trimmed
* on the way out. Root nodes are depth 1. `maxDepth: N` keeps nodes at depth
* <= N and drops the `children` of any node AT depth N. A node whose children
* were dropped this way gets `hasChildren: true` when it actually had children
* in the flat input (source of truth = the server `hasChildren` flag), so the
* caller knows it can descend further with a follow-up `rootPageId` call. An
* absent/undefined `maxDepth` means no cut (whole tree). `maxDepth <= 0` is
* treated as "no cut" (defensive; the tool schema clamps to >= 1).
*
* Pure: no I/O, no network, deterministic. * Pure: no I/O, no network, deterministic.
*/ */
export function buildPageTree(nodes: any[]): any[] { export function buildPageTree(
type OutputNode = { nodes: any[],
options: BuildPageTreeOptions = {},
): any[] {
const getTreeShape = options.shape === "getTree";
// A finite, positive cut only; anything else means "no cut".
const maxDepth =
typeof options.maxDepth === "number" &&
Number.isFinite(options.maxDepth) &&
options.maxDepth > 0
? Math.floor(options.maxDepth)
: undefined;
type InternalNode = {
id: string; id: string;
// Retained internally for shaping; never all emitted at once.
slugId: any; slugId: any;
title: any; title: any;
children?: OutputNode[]; hasServerChildren: boolean;
children?: InternalNode[];
}; };
// Map id -> output node. Build the lean output shape up front. // Map id -> internal node. Build up front; the output shape is projected at
const byId = new Map<string, OutputNode>(); // the very end so the maxDepth cut can consult `hasServerChildren`.
const byId = new Map<string, InternalNode>();
// Preserve the original position string for sorting (kept off the output). // Preserve the original position string for sorting (kept off the output).
const positionById = new Map<string, string | undefined>(); const positionById = new Map<string, string | undefined>();
@@ -49,6 +100,7 @@ export function buildPageTree(nodes: any[]): any[] {
id: node.id, id: node.id,
slugId: node.slugId, slugId: node.slugId,
title: node.title, title: node.title,
hasServerChildren: node.hasChildren === true,
}); });
positionById.set(node.id, node.position); positionById.set(node.id, node.position);
} }
@@ -90,5 +142,30 @@ export function buildPageTree(nodes: any[]): any[] {
} }
roots.sort(byPosition); roots.sort(byPosition);
return roots.map((id) => byId.get(id)!); const rootNodes = roots.map((id) => byId.get(id)!);
// Project the internal nodes into the requested OUTPUT shape, applying the
// maxDepth cut for the getTree shape. `depth` is 1-based (roots = depth 1).
const project = (node: InternalNode, depth: number): any => {
if (getTreeShape) {
const out: any = { pageId: node.id, title: node.title };
const atCut = maxDepth !== undefined && depth >= maxDepth;
if (!atCut && node.children && node.children.length > 0) {
out.children = node.children.map((c) => project(c, depth + 1));
} else if (atCut && node.hasServerChildren) {
// Children exist on the server but were trimmed by maxDepth: signal it
// so the caller can descend with a follow-up rootPageId call.
out.hasChildren = true;
}
return out;
}
// Lean (historic) shape: cycle-safe, no depth cut, no hasChildren.
const out: any = { id: node.id, slugId: node.slugId, title: node.title };
if (node.children && node.children.length > 0) {
out.children = node.children.map((c) => project(c, depth + 1));
}
return out;
};
return rootNodes.map((n) => project(n, 1));
} }
+62 -57
View File
@@ -40,11 +40,11 @@ import { SHARED_TOOL_SPECS, SharedToolSpec } from "./tool-specs.js";
*/ */
export const ROUTING_PROSE = export const ROUTING_PROSE =
"Docmost editing guide — choose the tool by intent. The <tool_inventory> at the end lists every tool with a one-line purpose; the notes below are the routing hints for WHEN to reach for each.\n" + "Docmost editing guide — choose the tool by intent. The <tool_inventory> at the end lists every tool with a one-line purpose; the notes below are the routing hints for WHEN to reach for each.\n" +
"READ: find a page -> search (workspace-wide full-text); list -> list_pages / list_spaces. Locate blocks and their ids CHEAPLY -> get_outline (compact top-level map; start here, not get_page_json). One block's subtree -> get_node (by attrs.id, or \"#<index>\" for tables, which carry no id). Find every occurrence of a string/regex ON a page (and where each is) -> search_in_page, NOT block-by-block get_node — it returns each hit's node ref + block index + context for a targeted comment. Whole page -> get_page (Markdown, lossy; inline <span data-comment-id> tags are comment anchors — markup, not text) or get_page_json (lossless ProseMirror with block ids). Hand a huge page (with images) to an external consumer without pulling it through the model context -> stash_page (returns a short-lived anonymous URL).\n" + "READ: find a page by a fragment of a technical string (hostname/IP/ID like srv.local, 10.0.12, WB-MGE-30D86B) -> search — hybrid substring + full-text, returns each hit's location (path: root->parent titles) and a snippet around the match, so you rarely need a follow-up getPage; scope with spaceId or parentPageId (a subtree), titleOnly to match titles only. A space's page HIERARCHY (or one subtree) -> getTree (one request, complete, `{pageId,title,children?}`; rootPageId for a subtree, maxDepth to trim depth — a trimmed node gets hasChildren:true); prefer it over listPages tree:true (deprecated). Have a pageId, need WHERE-AM-I / what's around it (its breadcrumbs + direct children, metadata only) -> getPageContext (one call; parent = last breadcrumb, [] for a root page). list -> listPages / listSpaces. Locate blocks and their ids CHEAPLY -> getOutline (compact top-level map; start here, not getPageJson). One block, for editing -> getNode (by attrs.id, or \"#<index>\" for tables, which carry no id) — returns MARKDOWN by default (comment anchors kept for safe write-back); pass format:\"json\" for the raw ProseMirror subtree. Find every occurrence of a string/regex ON a page (and where each is) -> searchInPage, NOT block-by-block getNode — it returns each hit's node ref + block index + context for a targeted comment. Whole page -> getPage (Markdown, canonical for text; drops only block ids, resolved-comment anchors, and a fixed no-md-representation attr set: table spans/colwidth/bg, indent, callout.icon, orderedList.type, link internal/target/rel/class; inline <span data-comment-id> tags are comment anchors — markup, not text) or getPageJson (full ProseMirror with block ids, for those dropped attrs). Hand a huge page (with images) to an external consumer without pulling it through the model context -> stashPage (returns a short-lived anonymous URL).\n" +
"EDIT: fix wording/typos/numbers -> edit_page_text (find/replace inside blocks, no node id needed). Change ONE block (paragraph/heading/callout/etc.) structurally -> patch_node (by attrs.id from get_outline). Add a block -> insert_node (before/after a block by attrs.id or by anchor text, or append). Remove a block -> delete_node (by attrs.id). Tables -> table_get / table_update_cell / table_insert_row / table_delete_row (address by \"#<index>\" from get_outline; table nodes have no attrs.id). Images -> insert_image (add from a web URL) / replace_image (swap an existing image). Draw.io diagrams -> drawio_create (create from mxGraph XML and insert), drawio_get (read a diagram as mxGraph XML + a hash), drawio_update (replace a diagram; pass the hash from drawio_get as baseHash for optimistic locking); before authoring a diagram, drawio_shapes (look up verified stencil style-strings so a shape name never renders as an empty box) and drawio_guide (on-demand authoring reference: skeleton/layout/containers/icons-aws/icons-azure), and pass layout:\"elk\" to drawio_create/drawio_update to auto-place nodes. Footnotes -> insert_footnote. Bulk/structural rewrite -> update_page_json (full ProseMirror replace) or update_page_markdown (full plain-Markdown body replace, re-imported — block ids regenerate); prefer the granular tools above to avoid resending the whole ~100KB+ document. Complex/scripted rewrite (multiple coordinated edits, renumbering) -> docmost_transform: write a JS `(doc, ctx) => doc` transform, preview the diff with dryRun (default), then apply with dryRun:false; ctx.helpers includes commentsToFootnotes for turning inline comments into numbered footnotes.\n" + "EDIT: fix wording/typos/numbers -> editPageText (find/replace inside blocks, no node id needed). Edit a block -> getNode(markdown) -> edit the markdown -> patchNode(markdown) (by attrs.id from getOutline; the markdown fragment may be several blocks — a 1->N section rewrite in one call, the first block keeps the id). Reach for patchNode's `node`-JSON only for fine attr/mark work; a table cell with spans/colors/fixed width -> the table tools (patchNode markdown refuses it). Add a block -> insertNode (markdown, before/after a block by attrs.id or by anchor text, or append; `node` for raw JSON or bare table structure). Remove a block -> deleteNode (by attrs.id). Tables -> tableGet / tableUpdateCell / tableInsertRow / tableDeleteRow (address by \"#<index>\" from getOutline; table nodes have no attrs.id). Images -> insertImage (add from a web URL) / replaceImage (swap an existing image). Draw.io diagrams -> PREFER the high-level semantic tools that hide coordinates/styles: drawioFromGraph (architecture/cloud/network diagrams — describe nodes/groups/edges by kind+icon, the server picks layout, colors and verified icons; hints layer/sameLayerAs/pinned and layout:full|incremental|none) and drawioFromMermaid (standard flowcharts — write Mermaid, get an editable diagram). For targeted tweaks of an existing diagram use drawioEditCells (id-based add/update/delete with cascade delete + baseHash lock). Raw mxGraph XML via drawioCreate/drawioUpdate is the escape-hatch for exotic/wireframe diagrams; drawioGet reads a diagram as mxGraph XML + a hash (pass it as baseHash to drawioUpdate/drawioEditCells for optimistic locking). Before authoring raw XML, drawioShapes (look up verified stencil style-strings so a shape name never renders as an empty box) and drawioGuide (on-demand authoring reference: skeleton/layout/containers/icons-aws/icons-azure), and pass layout:\"elk\" to drawioCreate/drawioUpdate to auto-place nodes. Footnotes -> insertFootnote. Bulk/structural rewrite -> updatePageJson (full ProseMirror replace) or updatePageMarkdown (full plain-Markdown body replace, re-imported — block ids regenerate); prefer the granular tools above to avoid resending the whole ~100KB+ document. Complex/scripted rewrite (multiple coordinated edits, renumbering) -> docmostTransform: write a JS `(doc, ctx) => doc` transform, preview the diff with dryRun (default), then apply with dryRun:false; ctx.helpers includes commentsToFootnotes for turning inline comments into numbered footnotes.\n" +
"PAGES: new -> create_page (Markdown). Rename (title only) -> rename_page. Move -> move_page. Delete -> delete_page (SOFT delete — the page goes to trash and is restorable; nothing is permanent). Copy/replace a page's whole content from another page (server-side, no document through the model) -> copy_page_content. Sharing -> share_page / unshare_page / list_shares; share_page makes the page PUBLICLY accessible — do it only when explicitly asked.\n" + "PAGES: new -> createPage (Markdown). Rename (title only) -> renamePage. Move -> movePage. Delete -> deletePage (SOFT delete — the page goes to trash and is restorable; nothing is permanent). Copy/replace a page's whole content from another page (server-side, no document through the model) -> copyPageContent. Sharing -> sharePage / unsharePage / listShares; sharePage makes the page PUBLICLY accessible — do it only when explicitly asked.\n" +
"COMMENTS: create_comment is always inline and requires an EXACT selection — contiguous text from a single block, <=250 chars (fails rather than leaving an unanchored comment); reply to a thread via parentCommentId. Propose a concrete text fix for one-click human approval -> create_comment with suggestedText (the exact plain-text replacement for the selection; the selection must then be UNIQUE in the page — extend it with context if needed); prefer this over editing directly when the change is subjective or needs the author's sign-off. Manage -> list_comments, update_comment, resolve_comment (resolve/reopen, reversible — prefer over delete to close), delete_comment, check_new_comments.\n" + "COMMENTS: createComment is always inline and requires an EXACT selection — contiguous text from a single block, <=250 chars (fails rather than leaving an unanchored comment); reply to a thread via parentCommentId. Propose a concrete text fix for one-click human approval -> createComment with suggestedText (the exact plain-text replacement for the selection; the selection must then be UNIQUE in the page — extend it with context if needed); prefer this over editing directly when the change is subjective or needs the author's sign-off. Manage -> listComments, updateComment, resolveComment (resolve/reopen, reversible — prefer over delete to close), deleteComment, checkNewComments.\n" +
"HISTORY: review what changed -> diff_page_versions (a historyId vs current, or two versions). List saved versions -> list_page_history. Undo a bad edit -> restore_page_version (writes a past version back as current; itself revertible). Export a page to self-contained Docmost Markdown (with comment anchors) -> export_page_markdown."; "HISTORY: review what changed -> diffPageVersions (a historyId vs current, or two versions). List saved versions -> listPageHistory. Undo a bad edit -> restorePageVersion (writes a past version back as current; itself revertible). Export a page to self-contained Docmost Markdown (with comment anchors) -> exportPageMarkdown.";
/** /**
* A single generated inventory line: the tool's registered NAME + a one-line * A single generated inventory line: the tool's registered NAME + a one-line
@@ -81,57 +81,62 @@ type Family = (typeof FAMILY_ORDER)[number];
const TOOL_FAMILY: Record<string, Family> = { const TOOL_FAMILY: Record<string, Family> = {
// READ // READ
search: "READ", search: "READ",
list_pages: "READ", listPages: "READ",
list_spaces: "READ", getTree: "READ",
get_outline: "READ", getPageContext: "READ",
get_node: "READ", listSpaces: "READ",
search_in_page: "READ", getOutline: "READ",
get_page: "READ", getNode: "READ",
get_page_json: "READ", searchInPage: "READ",
get_workspace: "READ", getPage: "READ",
stash_page: "READ", getPageJson: "READ",
getWorkspace: "READ",
stashPage: "READ",
// EDIT // EDIT
edit_page_text: "EDIT", editPageText: "EDIT",
patch_node: "EDIT", patchNode: "EDIT",
insert_node: "EDIT", insertNode: "EDIT",
delete_node: "EDIT", deleteNode: "EDIT",
update_page_json: "EDIT", updatePageJson: "EDIT",
update_page_markdown: "EDIT", updatePageMarkdown: "EDIT",
table_get: "EDIT", tableGet: "EDIT",
table_update_cell: "EDIT", tableUpdateCell: "EDIT",
table_insert_row: "EDIT", tableInsertRow: "EDIT",
table_delete_row: "EDIT", tableDeleteRow: "EDIT",
insert_image: "EDIT", insertImage: "EDIT",
replace_image: "EDIT", replaceImage: "EDIT",
insert_footnote: "EDIT", insertFootnote: "EDIT",
drawio_get: "EDIT", drawioGet: "EDIT",
drawio_create: "EDIT", drawioCreate: "EDIT",
drawio_update: "EDIT", drawioUpdate: "EDIT",
drawio_shapes: "EDIT", drawioEditCells: "EDIT",
drawio_guide: "EDIT", drawioFromGraph: "EDIT",
docmost_transform: "EDIT", drawioFromMermaid: "EDIT",
drawioShapes: "EDIT",
drawioGuide: "EDIT",
docmostTransform: "EDIT",
// PAGES // PAGES
create_page: "PAGES", createPage: "PAGES",
rename_page: "PAGES", renamePage: "PAGES",
move_page: "PAGES", movePage: "PAGES",
delete_page: "PAGES", deletePage: "PAGES",
copy_page_content: "PAGES", copyPageContent: "PAGES",
share_page: "PAGES", sharePage: "PAGES",
unshare_page: "PAGES", unsharePage: "PAGES",
list_shares: "PAGES", listShares: "PAGES",
// COMMENTS // COMMENTS
create_comment: "COMMENTS", createComment: "COMMENTS",
list_comments: "COMMENTS", listComments: "COMMENTS",
update_comment: "COMMENTS", updateComment: "COMMENTS",
resolve_comment: "COMMENTS", resolveComment: "COMMENTS",
delete_comment: "COMMENTS", deleteComment: "COMMENTS",
check_new_comments: "COMMENTS", checkNewComments: "COMMENTS",
// HISTORY // HISTORY
diff_page_versions: "HISTORY", diffPageVersions: "HISTORY",
list_page_history: "HISTORY", listPageHistory: "HISTORY",
restore_page_version: "HISTORY", restorePageVersion: "HISTORY",
export_page_markdown: "HISTORY", exportPageMarkdown: "HISTORY",
// import_page_markdown is now inAppOnly (#411) — it is not registered on the // importPageMarkdown is now inAppOnly (#411) — it is not registered on the
// external MCP host, so it no longer appears in the generated inventory. // external MCP host, so it no longer appears in the generated inventory.
}; };
@@ -145,26 +150,26 @@ const TOOL_FAMILY: Record<string, Family> = {
*/ */
export const INLINE_MCP_INVENTORY: ToolInventoryLine[] = [ export const INLINE_MCP_INVENTORY: ToolInventoryLine[] = [
{ {
name: "table_get", name: "tableGet",
purpose: purpose:
"read a table as a matrix of cell texts + per-cell paragraph ids.", "read a table as a matrix of cell texts + per-cell paragraph ids.",
}, },
{ {
name: "search", name: "search",
purpose: purpose:
"full-text search for pages and content across the whole workspace.", "find pages by a fragment of a technical string (hybrid substring + full-text); returns each hit's path and a snippet.",
}, },
{ {
name: "docmost_transform", name: "docmostTransform",
purpose: purpose:
"edit a page by running a sandboxed JS `(doc, ctx) => doc` transform, with a dryRun diff preview.", "edit a page by running a sandboxed JS `(doc, ctx) => doc` transform, with a dryRun diff preview.",
}, },
{ {
name: "update_comment", name: "updateComment",
purpose: "update an existing comment's content (creator only).", purpose: "update an existing comment's content (creator only).",
}, },
{ {
name: "delete_comment", name: "deleteComment",
purpose: "delete a comment (creator or space admin only).", purpose: "delete a comment (creator or space admin only).",
}, },
]; ];
File diff suppressed because it is too large Load Diff
+76 -76
View File
@@ -84,20 +84,20 @@ async function main() {
let pageId = null; let pageId = null;
try { try {
// 1. create_page: title with spaces must survive (was: underscores bug) // 1. createPage: title with spaces must survive (was: underscores bug)
const created = await client.createPage("Тест апгрейда MCP сервера", MD, spaceId); const created = await client.createPage("Тест апгрейда MCP сервера", MD, spaceId);
pageId = created.data.id; pageId = created.data.id;
check("create_page: title keeps spaces", created.data.title === "Тест апгрейда MCP сервера", created.data.title); check("createPage: title keeps spaces", created.data.title === "Тест апгрейда MCP сервера", created.data.title);
check("create_page: slugId exposed", typeof created.data.slugId === "string" && created.data.slugId.length > 0, created.data.slugId); check("createPage: slugId exposed", typeof created.data.slugId === "string" && created.data.slugId.length > 0, created.data.slugId);
// 2. get_page_json: raw ProseMirror with callout + table // 2. getPageJson: raw ProseMirror with callout + table
const pj = await client.getPageJson(pageId); const pj = await client.getPageJson(pageId);
const types = pj.content.content.map((n) => n.type); const types = pj.content.content.map((n) => n.type);
check("get_page_json: callout node present", types.includes("callout"), types.join(",")); check("getPageJson: callout node present", types.includes("callout"), types.join(","));
check("get_page_json: table node present", types.includes("table")); check("getPageJson: table node present", types.includes("table"));
check("get_page_json: slugId present", !!pj.slugId); check("getPageJson: slugId present", !!pj.slugId);
// 3. edit_page_text: surgical replace, ids preserved // 3. editPageText: surgical replace, ids preserved
const idsBefore = JSON.stringify( const idsBefore = JSON.stringify(
pj.content.content.filter((n) => n.attrs?.id).map((n) => n.attrs.id), pj.content.content.filter((n) => n.attrs?.id).map((n) => n.attrs.id),
); );
@@ -105,26 +105,26 @@ async function main() {
{ find: "БУКВОЕД", replace: "КНИГОЛЮБ" }, { find: "БУКВОЕД", replace: "КНИГОЛЮБ" },
{ find: "[1]", replace: "[42]" }, { find: "[1]", replace: "[42]" },
]); ]);
check("edit_page_text: both edits applied", editRes.applied.every((e) => e.replacements === 1)); check("editPageText: both edits applied", editRes.applied.every((e) => e.replacements === 1));
await new Promise((r) => setTimeout(r, 16000)); // wait for server persistence await new Promise((r) => setTimeout(r, 16000)); // wait for server persistence
const pj2 = await client.getPageJson(pageId); const pj2 = await client.getPageJson(pageId);
const text2 = JSON.stringify(pj2.content); const text2 = JSON.stringify(pj2.content);
check("edit_page_text: replacement visible", text2.includes("КНИГОЛЮБ") && text2.includes("[42]")); check("editPageText: replacement visible", text2.includes("КНИГОЛЮБ") && text2.includes("[42]"));
check("edit_page_text: old text gone", !text2.includes("БУКВОЕД")); check("editPageText: old text gone", !text2.includes("БУКВОЕД"));
const idsAfter = JSON.stringify( const idsAfter = JSON.stringify(
pj2.content.content.filter((n) => n.attrs?.id).map((n) => n.attrs.id), pj2.content.content.filter((n) => n.attrs?.id).map((n) => n.attrs.id),
); );
check("edit_page_text: block ids preserved", idsBefore === idsAfter); check("editPageText: block ids preserved", idsBefore === idsAfter);
check("edit_page_text: callout survived", JSON.stringify(pj2.content).includes('"callout"')); check("editPageText: callout survived", JSON.stringify(pj2.content).includes('"callout"'));
check("edit_page_text: table survived", pj2.content.content.some((n) => n.type === "table")); check("editPageText: table survived", pj2.content.content.some((n) => n.type === "table"));
// 4. error reporting: ambiguous and missing finds // 4. error reporting: ambiguous and missing finds
let err1 = ""; let err1 = "";
try { await client.editPageText(pageId, [{ find: "Колонка", replace: "X" }]); } catch (e) { err1 = e.message; } try { await client.editPageText(pageId, [{ find: "Колонка", replace: "X" }]); } catch (e) { err1 = e.message; }
check("edit_page_text: ambiguous match rejected", err1.includes("matches"), err1); check("editPageText: ambiguous match rejected", err1.includes("matches"), err1);
let err2 = ""; let err2 = "";
try { await client.editPageText(pageId, [{ find: "НЕСУЩЕСТВУЮЩЕЕ", replace: "X" }]); } catch (e) { err2 = e.message; } try { await client.editPageText(pageId, [{ find: "НЕСУЩЕСТВУЮЩЕЕ", replace: "X" }]); } catch (e) { err2 = e.message; }
check("edit_page_text: missing text reported", err2.includes("not found"), err2); check("editPageText: missing text reported", err2.includes("not found"), err2);
// 5. update_page (markdown): table + callout must survive the re-import // 5. update_page (markdown): table + callout must survive the re-import
await client.updatePage(pageId, MD + "\nДобавленный абзац.\n"); await client.updatePage(pageId, MD + "\nДобавленный абзац.\n");
@@ -137,21 +137,21 @@ async function main() {
const cellText = JSON.stringify(tableNode); const cellText = JSON.stringify(tableNode);
check("update_page md: table cells intact", cellText.includes("четыре") && cellText.includes("Колонка А")); check("update_page md: table cells intact", cellText.includes("четыре") && cellText.includes("Колонка А"));
// 6. update_page_json: lossless write round-trip // 6. updatePageJson: lossless write round-trip
pj3.content.content.push({ pj3.content.content.push({
type: "paragraph", type: "paragraph",
attrs: { id: "testidjsonpush", indent: 0, textAlign: null }, attrs: { id: "testidjsonpush", indent: 0, textAlign: null },
content: [{ type: "text", text: "Абзац, добавленный через update_page_json." }], content: [{ type: "text", text: "Абзац, добавленный через updatePageJson." }],
}); });
await client.updatePageJson(pageId, pj3.content); await client.updatePageJson(pageId, pj3.content);
await new Promise((r) => setTimeout(r, 16000)); await new Promise((r) => setTimeout(r, 16000));
const pj4 = await client.getPageJson(pageId); const pj4 = await client.getPageJson(pageId);
const lastNode = pj4.content.content[pj4.content.content.length - 1]; const lastNode = pj4.content.content[pj4.content.content.length - 1];
check("update_page_json: paragraph appended", JSON.stringify(pj4.content).includes("добавленный через update_page_json")); check("updatePageJson: paragraph appended", JSON.stringify(pj4.content).includes("добавленный через updatePageJson"));
check("update_page_json: custom node id preserved", lastNode.attrs?.id === "testidjsonpush", lastNode.attrs?.id); check("updatePageJson: custom node id preserved", lastNode.attrs?.id === "testidjsonpush", lastNode.attrs?.id);
// 6b. images: upload / insert / replace (clean src, fresh attachment on replace). // 6b. images: upload / insert / replace (clean src, fresh attachment on replace).
// insert_image / replace_image take an http(s) URL that the SERVER fetches; // insertImage / replaceImage take an http(s) URL that the SERVER fetches;
// local file paths are intentionally unsupported. The Docmost server runs on // local file paths are intentionally unsupported. The Docmost server runs on
// the same host as this test, so serve the PNG bytes over a throwaway // the same host as this test, so serve the PNG bytes over a throwaway
// localhost HTTP server it can reach. // localhost HTTP server it can reach.
@@ -186,13 +186,13 @@ async function main() {
validateStatus: () => true, validateStatus: () => true,
}); });
// insert_image: append the first PNG, src must be clean (no ?v=) and fetchable. // insertImage: append the first PNG, src must be clean (no ?v=) and fetchable.
const ins = await client.insertImage(pageId, urlA); const ins = await client.insertImage(pageId, urlA);
check("insert_image: src has no ?v= cache-buster", !ins.src.includes("?v="), ins.src); check("insertImage: src has no ?v= cache-buster", !ins.src.includes("?v="), ins.src);
const fileA = await fetchFile(ins.src); const fileA = await fetchFile(ins.src);
check("insert_image: file fetch returns 200", fileA.status === 200, `status=${fileA.status}`); check("insertImage: file fetch returns 200", fileA.status === 200, `status=${fileA.status}`);
check( check(
"insert_image: content-type is image/*", "insertImage: content-type is image/*",
String(fileA.headers["content-type"] || "").startsWith("image/"), String(fileA.headers["content-type"] || "").startsWith("image/"),
String(fileA.headers["content-type"]), String(fileA.headers["content-type"]),
); );
@@ -209,25 +209,25 @@ async function main() {
}; };
const imgNode = findImage(pjImg.content.content); const imgNode = findImage(pjImg.content.content);
const oldAttachmentId = imgNode?.attrs?.attachmentId; const oldAttachmentId = imgNode?.attrs?.attachmentId;
check("insert_image: image node present after persist", !!oldAttachmentId, oldAttachmentId); check("insertImage: image node present after persist", !!oldAttachmentId, oldAttachmentId);
// replace_image: must create a NEW attachment with a clean, fetchable URL. // replaceImage: must create a NEW attachment with a clean, fetchable URL.
// The 200 fetch is the assertion that catches the in-place-overwrite HTTP 500 regression. // The 200 fetch is the assertion that catches the in-place-overwrite HTTP 500 regression.
const rep = await client.replaceImage(pageId, oldAttachmentId, urlB); const rep = await client.replaceImage(pageId, oldAttachmentId, urlB);
check("replace_image: new attachment id differs from old", rep.newAttachmentId !== oldAttachmentId, `${oldAttachmentId} -> ${rep.newAttachmentId}`); check("replaceImage: new attachment id differs from old", rep.newAttachmentId !== oldAttachmentId, `${oldAttachmentId} -> ${rep.newAttachmentId}`);
check("replace_image: src has no ?v= cache-buster", !rep.src.includes("?v="), rep.src); check("replaceImage: src has no ?v= cache-buster", !rep.src.includes("?v="), rep.src);
const fileB = await fetchFile(rep.src); const fileB = await fetchFile(rep.src);
check("replace_image: new file fetch returns 200", fileB.status === 200, `status=${fileB.status}`); check("replaceImage: new file fetch returns 200", fileB.status === 200, `status=${fileB.status}`);
check( check(
"replace_image: new content-type is image/*", "replaceImage: new content-type is image/*",
String(fileB.headers["content-type"] || "").startsWith("image/"), String(fileB.headers["content-type"] || "").startsWith("image/"),
String(fileB.headers["content-type"]), String(fileB.headers["content-type"]),
); );
await new Promise((r) => setTimeout(r, 16000)); await new Promise((r) => setTimeout(r, 16000));
const pjImg2 = await client.getPageJson(pageId); const pjImg2 = await client.getPageJson(pageId);
check("replace_image: page has new attachment id", !!findImage(pjImg2.content.content, rep.newAttachmentId), rep.newAttachmentId); check("replaceImage: page has new attachment id", !!findImage(pjImg2.content.content, rep.newAttachmentId), rep.newAttachmentId);
check("replace_image: old attachment id repointed away", !findImage(pjImg2.content.content, oldAttachmentId), oldAttachmentId); check("replaceImage: old attachment id repointed away", !findImage(pjImg2.content.content, oldAttachmentId), oldAttachmentId);
} finally { } finally {
imgServer.close(); imgServer.close();
} }
@@ -275,10 +275,10 @@ async function main() {
await client.editPageText(fid, [{ find: "PRICEMARK", replace: "$& costs $100" }]); await client.editPageText(fid, [{ find: "PRICEMARK", replace: "$& costs $100" }]);
await new Promise((r) => setTimeout(r, 16000)); await new Promise((r) => setTimeout(r, 16000));
const ftext = JSON.stringify((await client.getPageJson(fid)).content); const ftext = JSON.stringify((await client.getPageJson(fid)).content);
check("feature: edit_page_text inserts $-pattern literally (no $& expansion)", ftext.includes("$& costs $100") && !ftext.includes("PRICEMARK costs")); check("feature: editPageText inserts $-pattern literally (no $& expansion)", ftext.includes("$& costs $100") && !ftext.includes("PRICEMARK costs"));
let badThrew = false; let badThrew = false;
try { await client.replaceImage(fid, "00000000-0000-0000-0000-000000000000", featPng); } catch (e) { badThrew = /no image with attachmentId/.test(e.message); } try { await client.replaceImage(fid, "00000000-0000-0000-0000-000000000000", featPng); } catch (e) { badThrew = /no image with attachmentId/.test(e.message); }
check("feature: replace_image with unknown id throws (no orphan upload)", badThrew); check("feature: replaceImage with unknown id throws (no orphan upload)", badThrew);
} finally { } finally {
try { await client.deletePage(fid); } catch {} try { await client.deletePage(fid); } catch {}
try { unlinkSync(featPng); } catch {} try { unlinkSync(featPng); } catch {}
@@ -286,7 +286,7 @@ async function main() {
} }
// 6d. node ops: patch / insert / delete a block by id on a throwaway page. // 6d. node ops: patch / insert / delete a block by id on a throwaway page.
// Three paragraphs are written with KNOWN ids via update_page_json so the // Three paragraphs are written with KNOWN ids via updatePageJson so the
// ids can be targeted directly; each op is verified via getPageJson after // ids can be targeted directly; each op is verified via getPageJson after
// the standard 16s persistence wait. // the standard 16s persistence wait.
{ {
@@ -348,7 +348,7 @@ async function main() {
} }
} }
// 6e. rename_page: title-only update must leave the content untouched. // 6e. renamePage: title-only update must leave the content untouched.
{ {
const rp = await client.createPage("E2E rename before " + Date.now(), "Rename body marker RENAMEBODY.", spaceId); const rp = await client.createPage("E2E rename before " + Date.now(), "Rename body marker RENAMEBODY.", spaceId);
const rid = rp.data.id; const rid = rp.data.id;
@@ -357,19 +357,19 @@ async function main() {
const beforeContent = JSON.stringify(beforeJson); const beforeContent = JSON.stringify(beforeJson);
const newTitle = "E2E rename AFTER " + Date.now(); const newTitle = "E2E rename AFTER " + Date.now();
const rr = await client.renamePage(rid, newTitle); const rr = await client.renamePage(rid, newTitle);
check("rename_page: returns success+title", rr.success === true && rr.title === newTitle, JSON.stringify(rr)); check("renamePage: returns success+title", rr.success === true && rr.title === newTitle, JSON.stringify(rr));
await new Promise((r) => setTimeout(r, 16000)); await new Promise((r) => setTimeout(r, 16000));
const afterJson = await client.getPageJson(rid); const afterJson = await client.getPageJson(rid);
check("rename_page: title changed", afterJson.title === newTitle, afterJson.title); check("renamePage: title changed", afterJson.title === newTitle, afterJson.title);
check("rename_page: content unchanged", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("RENAMEBODY")); check("renamePage: content unchanged", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("RENAMEBODY"));
const afterMd = (await client.getPage(rid)).data; const afterMd = (await client.getPage(rid)).data;
check("rename_page: get_page reflects new title", afterMd.title === newTitle, afterMd.title); check("renamePage: getPage reflects new title", afterMd.title === newTitle, afterMd.title);
} finally { } finally {
try { await client.deletePage(rid); } catch {} try { await client.deletePage(rid); } catch {}
} }
} }
// 6f. update_page_json title-only: omitting content updates the title and // 6f. updatePageJson title-only: omitting content updates the title and
// leaves the body intact; supplying neither content nor title throws. // leaves the body intact; supplying neither content nor title throws.
{ {
const up = await client.createPage("E2E upj-title before " + Date.now(), "Title-only body marker UPJTITLEBODY.", spaceId); const up = await client.createPage("E2E upj-title before " + Date.now(), "Title-only body marker UPJTITLEBODY.", spaceId);
@@ -378,20 +378,20 @@ async function main() {
const beforeContent = JSON.stringify((await client.getPageJson(uid)).content); const beforeContent = JSON.stringify((await client.getPageJson(uid)).content);
const newTitle = "E2E upj-title AFTER " + Date.now(); const newTitle = "E2E upj-title AFTER " + Date.now();
const ur = await client.updatePageJson(uid, undefined, newTitle); const ur = await client.updatePageJson(uid, undefined, newTitle);
check("update_page_json title-only: succeeds", ur.success === true, JSON.stringify(ur)); check("updatePageJson title-only: succeeds", ur.success === true, JSON.stringify(ur));
await new Promise((r) => setTimeout(r, 16000)); await new Promise((r) => setTimeout(r, 16000));
const afterJson = await client.getPageJson(uid); const afterJson = await client.getPageJson(uid);
check("update_page_json title-only: title updated", afterJson.title === newTitle, afterJson.title); check("updatePageJson title-only: title updated", afterJson.title === newTitle, afterJson.title);
check("update_page_json title-only: content intact", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("UPJTITLEBODY")); check("updatePageJson title-only: content intact", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("UPJTITLEBODY"));
let upjErr = ""; let upjErr = "";
try { await client.updatePageJson(uid); } catch (e) { upjErr = e.message; } try { await client.updatePageJson(uid); } catch (e) { upjErr = e.message; }
check("update_page_json: neither content nor title throws", upjErr.includes("nothing to update"), upjErr); check("updatePageJson: neither content nor title throws", upjErr.includes("nothing to update"), upjErr);
} finally { } finally {
try { await client.deletePage(uid); } catch {} try { await client.deletePage(uid); } catch {}
} }
} }
// 6g. copy_page_content: B's body becomes a copy of A's body, server-side, // 6g. copyPageContent: B's body becomes a copy of A's body, server-side,
// while B's title/slugId stay put. Both pages are throwaways. // while B's title/slugId stay put. Both pages are throwaways.
{ {
let aid = null; let aid = null;
@@ -409,24 +409,24 @@ async function main() {
const aNodeCount = aJson.content.content.length; const aNodeCount = aJson.content.content.length;
const cr = await client.copyPageContent(aid, bid); const cr = await client.copyPageContent(aid, bid);
check("copy_page_content: returns success + node count", cr.success === true && cr.copiedNodes === aNodeCount, JSON.stringify(cr)); check("copyPageContent: returns success + node count", cr.success === true && cr.copiedNodes === aNodeCount, JSON.stringify(cr));
await new Promise((r) => setTimeout(r, 16000)); await new Promise((r) => setTimeout(r, 16000));
const bAfter = await client.getPageJson(bid); const bAfter = await client.getPageJson(bid);
const bText = JSON.stringify(bAfter.content); const bText = JSON.stringify(bAfter.content);
check("copy_page_content: B now has A's marker", bText.includes("COPYSOURCE")); check("copyPageContent: B now has A's marker", bText.includes("COPYSOURCE"));
check("copy_page_content: B's old marker gone", !bText.includes("COPYTARGET")); check("copyPageContent: B's old marker gone", !bText.includes("COPYTARGET"));
check("copy_page_content: B node count equals A's", bAfter.content.content.length === aNodeCount, `${bAfter.content.content.length} vs ${aNodeCount}`); check("copyPageContent: B node count equals A's", bAfter.content.content.length === aNodeCount, `${bAfter.content.content.length} vs ${aNodeCount}`);
check("copy_page_content: B title unchanged", bAfter.title === bTitleBefore, bAfter.title); check("copyPageContent: B title unchanged", bAfter.title === bTitleBefore, bAfter.title);
check("copy_page_content: B slugId unchanged", bAfter.slugId === bSlugBefore, bAfter.slugId); check("copyPageContent: B slugId unchanged", bAfter.slugId === bSlugBefore, bAfter.slugId);
// Source must be left untouched by the copy. // Source must be left untouched by the copy.
const aAfter = JSON.stringify((await client.getPageJson(aid)).content); const aAfter = JSON.stringify((await client.getPageJson(aid)).content);
check("copy_page_content: source page unchanged", aAfter === JSON.stringify(aJson.content) && aAfter.includes("COPYSOURCE")); check("copyPageContent: source page unchanged", aAfter === JSON.stringify(aJson.content) && aAfter.includes("COPYSOURCE"));
let copyErr = ""; let copyErr = "";
try { await client.copyPageContent(aid, aid); } catch (e) { copyErr = e.message; } try { await client.copyPageContent(aid, aid); } catch (e) { copyErr = e.message; }
check("copy_page_content: self-copy rejected", copyErr.includes("same page"), copyErr); check("copyPageContent: self-copy rejected", copyErr.includes("same page"), copyErr);
} finally { } finally {
try { if (bid) await client.deletePage(bid); } catch {} try { if (bid) await client.deletePage(bid); } catch {}
try { if (aid) await client.deletePage(aid); } catch {} try { if (aid) await client.deletePage(aid); } catch {}
@@ -435,22 +435,22 @@ async function main() {
// 7. shares: create (idempotent), public access, list, unshare // 7. shares: create (idempotent), public access, list, unshare
const share = await client.sharePage(pageId); const share = await client.sharePage(pageId);
check("share_page: returns public URL", share.publicUrl?.startsWith(`${APP}/share/`), share.publicUrl); check("sharePage: returns public URL", share.publicUrl?.startsWith(`${APP}/share/`), share.publicUrl);
const share2 = await client.sharePage(pageId); const share2 = await client.sharePage(pageId);
check("share_page: idempotent", share2.key === share.key); check("sharePage: idempotent", share2.key === share.key);
const anon = await axios.post(`${API}/shares/page-info`, { pageId: pj4.slugId, shareId: share.key }, { validateStatus: () => true }); const anon = await axios.post(`${API}/shares/page-info`, { pageId: pj4.slugId, shareId: share.key }, { validateStatus: () => true });
check("share_page: anonymous access works", anon.status === 200); check("sharePage: anonymous access works", anon.status === 200);
const shares = await client.listShares(); const shares = await client.listShares();
check("list_shares: contains our page", shares.some((s) => s.pageId === pageId && s.publicUrl === share.publicUrl)); check("listShares: contains our page", shares.some((s) => s.pageId === pageId && s.publicUrl === share.publicUrl));
const un = await client.unsharePage(pageId); const un = await client.unsharePage(pageId);
check("unshare_page: success", un.success === true); check("unsharePage: success", un.success === true);
const anon2 = await axios.post(`${API}/shares/page-info`, { pageId: pj4.slugId, shareId: share.key }, { validateStatus: () => true }); const anon2 = await axios.post(`${API}/shares/page-info`, { pageId: pj4.slugId, shareId: share.key }, { validateStatus: () => true });
check("unshare_page: public access revoked", anon2.status !== 200, `status=${anon2.status}`); check("unsharePage: public access revoked", anon2.status !== 200, `status=${anon2.status}`);
// 8. get_page markdown round-trip sanity (table separator present) // 8. getPage markdown round-trip sanity (table separator present)
const md = await client.getPage(pageId); const md = await client.getPage(pageId);
check("get_page md: table separator emitted", md.data.content.includes("| --- |"), ""); check("getPage md: table separator emitted", md.data.content.includes("| --- |"), "");
check("get_page md: callout exported as Obsidian '> [!info]'", md.data.content.includes("> [!info]")); check("getPage md: callout exported as Obsidian '> [!info]'", md.data.content.includes("> [!info]"));
// 9. comments: create / list / reply / update / check_new / delete // 9. comments: create / list / reply / update / check_new / delete
const beforeComments = new Date(Date.now() - 1000).toISOString(); const beforeComments = new Date(Date.now() - 1000).toISOString();
@@ -458,34 +458,34 @@ async function main() {
// that exists in the persisted page to anchor on. "Добавленный абзац." is a // that exists in the persisted page to anchor on. "Добавленный абзац." is a
// plain paragraph re-imported in section 5 and still present here. // plain paragraph re-imported in section 5 and still present here.
const c1 = await client.createComment(pageId, "Первый **комментарий** с [ссылкой](https://example.com).", "inline", "Добавленный абзац."); const c1 = await client.createComment(pageId, "Первый **комментарий** с [ссылкой](https://example.com).", "inline", "Добавленный абзац.");
check("create_comment: created", !!c1.data.id, c1.data.id); check("createComment: created", !!c1.data.id, c1.data.id);
check("create_comment: markdown round-trip", c1.data.content.includes("**комментарий**"), c1.data.content); check("createComment: markdown round-trip", c1.data.content.includes("**комментарий**"), c1.data.content);
const reply = await client.createComment(pageId, "Ответ на комментарий.", "page", undefined, c1.data.id); const reply = await client.createComment(pageId, "Ответ на комментарий.", "page", undefined, c1.data.id);
check("create_comment: reply has parent", reply.data.parentCommentId === c1.data.id); check("createComment: reply has parent", reply.data.parentCommentId === c1.data.id);
const list = (await client.listComments(pageId)).items; const list = (await client.listComments(pageId)).items;
check("list_comments: both visible", list.length === 2, `count=${list.length}`); check("listComments: both visible", list.length === 2, `count=${list.length}`);
await client.updateComment(c1.data.id, "Обновлённый текст комментария."); await client.updateComment(c1.data.id, "Обновлённый текст комментария.");
const got = await client.getComment(c1.data.id); const got = await client.getComment(c1.data.id);
check("update_comment + get_comment: content updated", got.data.content.includes("Обновлённый"), got.data.content); check("updateComment + get_comment: content updated", got.data.content.includes("Обновлённый"), got.data.content);
const news = await client.checkNewComments(spaceId, beforeComments, pageId); const news = await client.checkNewComments(spaceId, beforeComments, pageId);
check("check_new_comments: finds new comments in subtree", news.totalNewComments >= 2, `total=${news.totalNewComments}`); check("checkNewComments: finds new comments in subtree", news.totalNewComments >= 2, `total=${news.totalNewComments}`);
// resolve_comment: close the top-level thread, verify resolvedAt surfaces, then reopen // resolveComment: close the top-level thread, verify resolvedAt surfaces, then reopen
const resolvedRes = await client.resolveComment(c1.data.id, true); const resolvedRes = await client.resolveComment(c1.data.id, true);
check("resolve_comment: marks resolved", resolvedRes.success === true && resolvedRes.resolved === true); check("resolveComment: marks resolved", resolvedRes.success === true && resolvedRes.resolved === true);
// c1 is now resolved; the default feed hides resolved threads, so pass // c1 is now resolved; the default feed hides resolved threads, so pass
// includeResolved:true to still see it and assert its resolvedAt (#328). // includeResolved:true to still see it and assert its resolvedAt (#328).
const listResolved = (await client.listComments(pageId, true)).items; const listResolved = (await client.listComments(pageId, true)).items;
const c1Resolved = listResolved.find((c) => c.id === c1.data.id); const c1Resolved = listResolved.find((c) => c.id === c1.data.id);
check("resolve_comment: resolvedAt set in list", !!c1Resolved?.resolvedAt, `resolvedAt=${c1Resolved?.resolvedAt}`); check("resolveComment: resolvedAt set in list", !!c1Resolved?.resolvedAt, `resolvedAt=${c1Resolved?.resolvedAt}`);
const reopenedRes = await client.resolveComment(c1.data.id, false); const reopenedRes = await client.resolveComment(c1.data.id, false);
check("resolve_comment: reopen succeeds", reopenedRes.resolved === false); check("resolveComment: reopen succeeds", reopenedRes.resolved === false);
const listReopened = (await client.listComments(pageId)).items; const listReopened = (await client.listComments(pageId)).items;
const c1Reopened = listReopened.find((c) => c.id === c1.data.id); const c1Reopened = listReopened.find((c) => c.id === c1.data.id);
check("resolve_comment: resolvedAt cleared on reopen", !c1Reopened?.resolvedAt, `resolvedAt=${c1Reopened?.resolvedAt}`); check("resolveComment: resolvedAt cleared on reopen", !c1Reopened?.resolvedAt, `resolvedAt=${c1Reopened?.resolvedAt}`);
await client.deleteComment(reply.data.id); await client.deleteComment(reply.data.id);
await client.deleteComment(c1.data.id); await client.deleteComment(c1.data.id);
const listAfter = (await client.listComments(pageId)).items; const listAfter = (await client.listComments(pageId)).items;
check("delete_comment: comments removed", listAfter.length === 0, `count=${listAfter.length}`); check("deleteComment: comments removed", listAfter.length === 0, `count=${listAfter.length}`);
} finally { } finally {
if (pageId) { if (pageId) {
await client.deletePage(pageId); await client.deletePage(pageId);
@@ -1,4 +1,4 @@
// Mock collab regression for the AMBIGUOUS-id refusal in patch_node / delete_node // Mock collab regression for the AMBIGUOUS-id refusal in patchNode / deleteNode
// (#159, PR #185 review pt 1). When a page has TWO blocks sharing one attrs.id // (#159, PR #185 review pt 1). When a page has TWO blocks sharing one attrs.id
// (Docmost duplicates block ids on copy/paste), the transform's // (Docmost duplicates block ids on copy/paste), the transform's
// `if (replaced !== 1) return null` / `if (deleted !== 1) return null` guard must // `if (replaced !== 1) return null` / `if (deleted !== 1) return null` guard must
@@ -126,18 +126,20 @@ after(async () => {
); );
}); });
test("patch_node REFUSES an ambiguous (duplicate) id without writing to collab", async () => { test("patchNode REFUSES an ambiguous (duplicate) id without writing to collab", async () => {
const { state, baseURL } = await spawnCollabStack(); const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects( await assert.rejects(
() => () =>
client.patchNode("11111111-1111-4111-8111-111111111111", DUP_ID, { client.patchNode("11111111-1111-4111-8111-111111111111", DUP_ID, {
node: {
type: "paragraph", type: "paragraph",
content: [{ type: "text", text: "replacement" }], content: [{ type: "text", text: "replacement" }],
},
}), }),
/ambiguous/i, /ambiguous/i,
"patch_node must reject a duplicate-id target with an 'ambiguous' error", "patchNode must reject a duplicate-id target with an 'ambiguous' error",
); );
assert.equal( assert.equal(
@@ -147,14 +149,14 @@ test("patch_node REFUSES an ambiguous (duplicate) id without writing to collab",
); );
}); });
test("delete_node REFUSES an ambiguous (duplicate) id without writing to collab", async () => { test("deleteNode REFUSES an ambiguous (duplicate) id without writing to collab", async () => {
const { state, baseURL } = await spawnCollabStack(); const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects( await assert.rejects(
() => client.deleteNode("22222222-2222-4222-8222-222222222222", DUP_ID), () => client.deleteNode("22222222-2222-4222-8222-222222222222", DUP_ID),
/ambiguous/i, /ambiguous/i,
"delete_node must reject a duplicate-id target with an 'ambiguous' error", "deleteNode must reject a duplicate-id target with an 'ambiguous' error",
); );
assert.equal( assert.equal(
@@ -0,0 +1,272 @@
// Contract tests for the stage-3 drawio client methods (issue #425):
// drawioEditCells / drawioFromGraph / drawioFromMermaid. Same seam-override
// pattern as drawio-tools.test.mjs: a DocmostClient subclass stubs the I/O seams
// so the tool logic runs without a live Docmost / collab socket.
import { test } from "node:test";
import assert from "node:assert/strict";
import { DocmostClient } from "../../build/client.js";
import {
buildDrawioSvg,
normalizeXml,
mxHash,
decodeDrawioSvg,
parseCells,
} from "../../build/lib/drawio-xml.js";
const DRAWIO_SCHEMA_ATTRS = new Set([
"src", "title", "alt", "width", "height", "size", "aspectRatio", "align", "attachmentId",
]);
function applyDrawioSchemaDrop(node) {
if (!node || typeof node !== "object") return;
if (node.type === "drawio" && node.attrs && typeof node.attrs === "object") {
for (const key of Object.keys(node.attrs))
if (!DRAWIO_SCHEMA_ATTRS.has(key)) delete node.attrs[key];
}
if (Array.isArray(node.content)) for (const c of node.content) applyDrawioSchemaDrop(c);
}
function svgFor(model, bbox = { width: 400, height: 300 }) {
return buildDrawioSvg(normalizeXml(model), "<g/>", bbox);
}
function makeClient({ pageDoc, attachmentSvg } = {}) {
const calls = { uploads: [], mutations: [] };
class TestClient extends DocmostClient {
async ensureAuthenticated() {}
async getCollabTokenWithReauth() {
return "collab-token";
}
async resolvePageId(pageId) {
return `uuid-${pageId}`;
}
async getPageRaw(pageId) {
return {
id: pageId, slugId: "s", title: "P", spaceId: "sp",
content: pageDoc ?? { type: "doc", content: [] },
};
}
async uploadAttachmentBuffer(pageId, buffer, fileName) {
const id = `att-${calls.uploads.length + 1}`;
calls.uploads.push({ pageId, fileName, svg: buffer.toString("utf-8") });
return { id, fileName, fileSize: buffer.length };
}
async fetchAttachmentText() {
return attachmentSvg;
}
mutatePage(pageId, token, apiUrl, transform) {
const clone = structuredClone(pageDoc ?? { type: "doc", content: [] });
const doc = transform(clone);
if (doc) applyDrawioSchemaDrop(doc);
calls.mutations.push({ pageId, doc });
return Promise.resolve({ doc, verify: { changed: doc != null } });
}
}
const client = new TestClient("http://127.0.0.1:1/api", "e@x.com", "pw");
return { client, calls };
}
function findDrawio(node, acc = []) {
if (!node || typeof node !== "object") return acc;
if (node.type === "drawio") acc.push(node);
if (Array.isArray(node.content)) for (const c of node.content) findDrawio(c, acc);
return acc;
}
// A stored diagram: a group with two children and an edge.
const STORED =
"<mxGraphModel><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/>" +
'<mxCell id="grp" value="G" style="container=1;fillColor=none;" vertex="1" parent="1">' +
'<mxGeometry x="0" y="0" width="300" height="200" as="geometry"/></mxCell>' +
'<mxCell id="a" value="A" style="rounded=1;" vertex="1" parent="grp">' +
'<mxGeometry x="10" y="10" width="80" height="40" as="geometry"/></mxCell>' +
'<mxCell id="b" value="B" style="rounded=1;" vertex="1" parent="grp">' +
'<mxGeometry x="10" y="90" width="80" height="40" as="geometry"/></mxCell>' +
'<mxCell id="e" style="" edge="1" parent="grp" source="a" target="b">' +
'<mxGeometry relative="1" as="geometry"/></mxCell>' +
"</root></mxGraphModel>";
function drawioPageDoc() {
return {
type: "doc",
content: [
{
type: "drawio",
attrs: {
id: "d1", src: "/api/files/att-1/diagram.drawio.svg",
attachmentId: "att-1", width: 400, height: 300,
},
},
],
};
}
// --- drawioEditCells --------------------------------------------------------
test("drawioEditCells: applies ops and repoints the node (current baseHash)", async () => {
const { client, calls } = makeClient({
pageDoc: drawioPageDoc(),
attachmentSvg: svgFor(STORED),
});
const baseHash = mxHash(normalizeXml(STORED));
const res = await client.drawioEditCells(
"page1",
"d1",
[
{
op: "update",
cellId: "a",
xml:
'<mxCell id="a" value="Renamed" style="rounded=1;" vertex="1" parent="grp">' +
'<mxGeometry x="10" y="10" width="80" height="40" as="geometry"/></mxCell>',
},
],
baseHash,
);
assert.equal(res.success, true);
assert.equal(calls.uploads.length, 1);
const written = decodeDrawioSvg(calls.uploads[0].svg);
const cells = parseCells(written);
assert.equal(cells.find((c) => c.id === "a").value, "Renamed");
assert.equal(cells.find((c) => c.id === "b").value, "B"); // untouched
const n = findDrawio(calls.mutations[0].doc)[0];
// The stub numbers uploads from 1; this edit is the first upload -> att-1.
assert.equal(n.attrs.attachmentId, "att-1");
});
test("drawioEditCells: delete of the container cascades to children + edge", async () => {
const { client, calls } = makeClient({
pageDoc: drawioPageDoc(),
attachmentSvg: svgFor(STORED),
});
const baseHash = mxHash(normalizeXml(STORED));
await client.drawioEditCells("page1", "d1", [{ op: "delete", cellId: "grp" }], baseHash);
const written = decodeDrawioSvg(calls.uploads[0].svg);
const ids = parseCells(written).filter((c) => c.id !== "0" && c.id !== "1").map((c) => c.id);
assert.deepEqual(ids, [], "grp + a + b + edge all cascaded away");
});
test("drawioEditCells: stale baseHash -> conflict, no upload", async () => {
const { client, calls } = makeClient({
pageDoc: drawioPageDoc(),
attachmentSvg: svgFor(STORED),
});
await assert.rejects(
() => client.drawioEditCells("page1", "d1", [{ op: "delete", cellId: "a" }], "stale"),
/conflict/,
);
assert.equal(calls.uploads.length, 0);
});
test("drawioEditCells: baseHash is mandatory", async () => {
const { client } = makeClient({ pageDoc: drawioPageDoc(), attachmentSvg: svgFor(STORED) });
await assert.rejects(
() => client.drawioEditCells("page1", "d1", [{ op: "delete", cellId: "a" }], ""),
/baseHash is mandatory/,
);
});
// --- drawioFromGraph --------------------------------------------------------
test("drawioFromGraph: builds a diagram from a graph and inserts a node", async () => {
const pageDoc = { type: "doc", content: [{ type: "paragraph", attrs: { id: "p1" }, content: [] }] };
const { client, calls } = makeClient({ pageDoc });
const res = await client.drawioFromGraph(
"page1",
{ position: "append" },
{
nodes: [
{ id: "api", label: "API", kind: "gateway", icon: "aws:api_gateway", group: "vpc" },
{ id: "fn", label: "Handler", kind: "service", icon: "aws:lambda", group: "vpc" },
{ id: "db", label: "Orders", kind: "db", icon: "aws:dynamodb" },
],
groups: [{ id: "vpc", label: "VPC" }],
edges: [{ from: "api", to: "fn", kind: "sync" }, { from: "fn", to: "db", kind: "async" }],
},
"LR",
"default",
);
assert.equal(res.success, true);
assert.equal(res.nodeId, "#1");
assert.equal(res.iconsMissing.length, 0, `unresolved: ${res.iconsMissing}`);
assert.equal(res.iconsResolved, 3);
// The uploaded model decodes back and carries the group + nodes.
const written = decodeDrawioSvg(calls.uploads[0].svg);
const cells = parseCells(written);
assert.ok(cells.some((c) => c.id === "vpc"));
assert.ok(cells.some((c) => c.id === "api"));
// Group is transparent.
const vpc = cells.find((c) => c.id === "vpc");
assert.equal(vpc.styleMap.fillColor, "none");
assert.equal(vpc.styleMap.container, "1");
});
test("drawioFromGraph: an invalid graph throws before any upload", async () => {
const { client, calls } = makeClient({ pageDoc: { type: "doc", content: [] } });
await assert.rejects(
() => client.drawioFromGraph("page1", { position: "append" }, { nodes: [] }),
/non-empty/,
);
assert.equal(calls.uploads.length, 0);
});
test("drawioFromGraph incremental into an existing node keeps prior coords", async () => {
// The stored diagram has a,b at known coords; add a new node c incrementally.
const { client, calls } = makeClient({
pageDoc: drawioPageDoc(),
attachmentSvg: svgFor(STORED),
});
const res = await client.drawioFromGraph(
"page1",
{ position: "append" },
{
nodes: [
{ id: "a", label: "A" },
{ id: "b", label: "B" },
{ id: "c", label: "C new" },
],
edges: [{ from: "b", to: "c" }],
},
undefined,
undefined,
"incremental",
"d1", // target the existing diagram
);
assert.equal(res.success, true);
const written = decodeDrawioSvg(calls.uploads[0].svg);
const cells = parseCells(written);
const a = cells.find((c) => c.id === "a");
const b = cells.find((c) => c.id === "b");
// Existing coords preserved (the stored a/b absolute coords from STORED).
assert.equal(a.geometry.x, 10);
assert.equal(a.geometry.y, 10);
assert.equal(b.geometry.x, 10);
assert.equal(b.geometry.y, 90);
assert.ok(cells.some((c) => c.id === "c"), "new node c added");
});
// --- drawioFromMermaid ------------------------------------------------------
test("drawioFromMermaid: converts a flowchart and inserts a diagram", async () => {
const pageDoc = { type: "doc", content: [{ type: "paragraph", attrs: { id: "p1" }, content: [] }] };
const { client, calls } = makeClient({ pageDoc });
const res = await client.drawioFromMermaid(
"page1",
{ position: "append" },
"flowchart LR\n A[Start] --> B{Choose}\n B -->|yes| C[Done]\n B -->|no| D[Stop]",
);
assert.equal(res.success, true);
const written = decodeDrawioSvg(calls.uploads[0].svg);
const cells = parseCells(written);
for (const id of ["A", "B", "C", "D"]) {
assert.ok(cells.some((c) => c.id === id), `node ${id} present`);
}
});
test("drawioFromMermaid: a non-flowchart is rejected, no upload", async () => {
const { client, calls } = makeClient({ pageDoc: { type: "doc", content: [] } });
await assert.rejects(
() => client.drawioFromMermaid("page1", { position: "append" }, "sequenceDiagram\n A->>B: x"),
/only 'flowchart'\/'graph' is supported/,
);
assert.equal(calls.uploads.length, 0);
});
+21 -21
View File
@@ -1,4 +1,4 @@
// Contract tests for the drawio_get / drawio_create / drawio_update client // Contract tests for the drawioGet / drawioCreate / drawioUpdate client
// methods (issue #423). Follows the repo's seam-override pattern (see // methods (issue #423). Follows the repo's seam-override pattern (see
// full-doc-write-canonicalize.test.mjs): a DocmostClient subclass stubs the I/O // full-doc-write-canonicalize.test.mjs): a DocmostClient subclass stubs the I/O
// seams (auth, collab token, page read, attachment upload/fetch, the mutatePage // seams (auth, collab token, page read, attachment upload/fetch, the mutatePage
@@ -114,9 +114,9 @@ function findDrawio(node, acc = []) {
return acc; return acc;
} }
// --- drawio_create --------------------------------------------------------- // --- drawioCreate ---------------------------------------------------------
test("drawio_create: lints, builds the .drawio.svg, uploads and inserts a node", async () => { test("drawioCreate: lints, builds the .drawio.svg, uploads and inserts a node", async () => {
const pageDoc = { const pageDoc = {
type: "doc", type: "doc",
content: [{ type: "paragraph", attrs: { id: "p1" }, content: [] }], content: [{ type: "paragraph", attrs: { id: "p1" }, content: [] }],
@@ -148,7 +148,7 @@ test("drawio_create: lints, builds the .drawio.svg, uploads and inserts a node",
assert.equal(n.attrs.title, "My diagram"); assert.equal(n.attrs.title, "My diagram");
}); });
test("drawio_create: a lint violation throws before any upload", async () => { test("drawioCreate: a lint violation throws before any upload", async () => {
const { client, calls } = makeClient({ pageDoc: { type: "doc", content: [] } }); const { client, calls } = makeClient({ pageDoc: { type: "doc", content: [] } });
// Edge with no child geometry -> edge-geometry rule. // Edge with no child geometry -> edge-geometry rule.
const bad = const bad =
@@ -162,7 +162,7 @@ test("drawio_create: a lint violation throws before any upload", async () => {
assert.equal(calls.uploads.length, 0, "no attachment uploaded on lint failure"); assert.equal(calls.uploads.length, 0, "no attachment uploaded on lint failure");
}); });
test("drawio_create: before/after requires exactly one anchor", async () => { test("drawioCreate: before/after requires exactly one anchor", async () => {
const { client } = makeClient({ pageDoc: { type: "doc", content: [] } }); const { client } = makeClient({ pageDoc: { type: "doc", content: [] } });
await assert.rejects( await assert.rejects(
() => client.drawioCreate("page1", { position: "before" }, MODEL), () => client.drawioCreate("page1", { position: "before" }, MODEL),
@@ -170,9 +170,9 @@ test("drawio_create: before/after requires exactly one anchor", async () => {
); );
}); });
// --- drawio_get ------------------------------------------------------------ // --- drawioGet ------------------------------------------------------------
test("drawio_get: decodes the model and returns meta with a hash", async () => { test("drawioGet: decodes the model and returns meta with a hash", async () => {
const pageDoc = { const pageDoc = {
type: "doc", type: "doc",
content: [ content: [
@@ -198,7 +198,7 @@ test("drawio_get: decodes the model and returns meta with a hash", async () => {
assert.equal(res.meta.hash, mxHash(normalizeXml(MODEL))); assert.equal(res.meta.hash, mxHash(normalizeXml(MODEL)));
}); });
test("drawio_get: format=svg returns the raw .drawio.svg", async () => { test("drawioGet: format=svg returns the raw .drawio.svg", async () => {
const svg = svgFor(MODEL); const svg = svgFor(MODEL);
const pageDoc = { const pageDoc = {
type: "doc", type: "doc",
@@ -211,7 +211,7 @@ test("drawio_get: format=svg returns the raw .drawio.svg", async () => {
assert.equal(res.content, svg); assert.equal(res.content, svg);
}); });
test("drawio_get: reads a HUMAN-saved compressed diagram losslessly (pako)", async () => { test("drawioGet: reads a HUMAN-saved compressed diagram losslessly (pako)", async () => {
const pageDoc = { const pageDoc = {
type: "doc", type: "doc",
content: [ content: [
@@ -223,7 +223,7 @@ test("drawio_get: reads a HUMAN-saved compressed diagram losslessly (pako)", asy
assert.equal(res.content, normalizeXml(MODEL)); assert.equal(res.content, normalizeXml(MODEL));
}); });
// --- drawio_update --------------------------------------------------------- // --- drawioUpdate ---------------------------------------------------------
const UPDATED_MODEL = const UPDATED_MODEL =
'<mxGraphModel><root>' + '<mxGraphModel><root>' +
@@ -250,7 +250,7 @@ function updatePageDoc() {
}; };
} }
test("drawio_update: stale baseHash -> conflict, no upload", async () => { test("drawioUpdate: stale baseHash -> conflict, no upload", async () => {
const { client, calls } = makeClient({ const { client, calls } = makeClient({
pageDoc: updatePageDoc(), pageDoc: updatePageDoc(),
attachmentSvg: svgFor(MODEL), attachmentSvg: svgFor(MODEL),
@@ -262,7 +262,7 @@ test("drawio_update: stale baseHash -> conflict, no upload", async () => {
assert.equal(calls.uploads.length, 0, "no upload on conflict"); assert.equal(calls.uploads.length, 0, "no upload on conflict");
}); });
test("drawio_update: current baseHash -> uploads new attachment and repoints node dims", async () => { test("drawioUpdate: current baseHash -> uploads new attachment and repoints node dims", async () => {
const currentHash = mxHash(normalizeXml(MODEL)); const currentHash = mxHash(normalizeXml(MODEL));
const { client, calls } = makeClient({ const { client, calls } = makeClient({
pageDoc: updatePageDoc(), pageDoc: updatePageDoc(),
@@ -285,7 +285,7 @@ test("drawio_update: current baseHash -> uploads new attachment and repoints nod
assert.equal(n.attrs.id, undefined); assert.equal(n.attrs.id, undefined);
}); });
test("drawio_update: baseHash is mandatory", async () => { test("drawioUpdate: baseHash is mandatory", async () => {
const { client } = makeClient({ pageDoc: updatePageDoc(), attachmentSvg: svgFor(MODEL) }); const { client } = makeClient({ pageDoc: updatePageDoc(), attachmentSvg: svgFor(MODEL) });
await assert.rejects( await assert.rejects(
() => client.drawioUpdate("page1", "d1", UPDATED_MODEL, ""), () => client.drawioUpdate("page1", "d1", UPDATED_MODEL, ""),
@@ -295,7 +295,7 @@ test("drawio_update: baseHash is mandatory", async () => {
// --- Fix 1: the create handle must resolve on the SAVED doc (no id) --------- // --- Fix 1: the create handle must resolve on the SAVED doc (no id) ---------
test("drawio_create -> get/update: returned #<index> handle resolves on the saved doc (id dropped)", async () => { test("drawioCreate -> get/update: returned #<index> handle resolves on the saved doc (id dropped)", async () => {
// Create appends a drawio node after the existing paragraph. // Create appends a drawio node after the existing paragraph.
const createDoc = { const createDoc = {
type: "doc", type: "doc",
@@ -316,13 +316,13 @@ test("drawio_create -> get/update: returned #<index> handle resolves on the save
const savedDoc = create.calls.mutations[0].doc; const savedDoc = create.calls.mutations[0].doc;
assert.equal(findDrawio(savedDoc)[0].attrs.id, undefined); assert.equal(findDrawio(savedDoc)[0].attrs.id, undefined);
// drawio_get with the returned handle resolves the just-created node. // drawioGet with the returned handle resolves the just-created node.
const getClient = makeClient({ pageDoc: savedDoc, attachmentSvg: svgFor(MODEL) }); const getClient = makeClient({ pageDoc: savedDoc, attachmentSvg: svgFor(MODEL) });
const got = await getClient.client.drawioGet("page1", res.nodeId, "xml"); const got = await getClient.client.drawioGet("page1", res.nodeId, "xml");
assert.equal(got.nodeId, res.nodeId); assert.equal(got.nodeId, res.nodeId);
assert.equal(got.content, normalizeXml(MODEL)); assert.equal(got.content, normalizeXml(MODEL));
// drawio_update with the same handle + the hash from get repoints that node. // drawioUpdate with the same handle + the hash from get repoints that node.
const upClient = makeClient({ pageDoc: savedDoc, attachmentSvg: svgFor(MODEL) }); const upClient = makeClient({ pageDoc: savedDoc, attachmentSvg: svgFor(MODEL) });
const upd = await upClient.client.drawioUpdate( const upd = await upClient.client.drawioUpdate(
"page1", "page1",
@@ -342,7 +342,7 @@ test("drawio_create -> get/update: returned #<index> handle resolves on the save
// --- error paths: the LLM must get a clean error, not a crash -------------- // --- error paths: the LLM must get a clean error, not a crash --------------
test("drawio_get: a bad node ref -> clean 'no node found' error", async () => { test("drawioGet: a bad node ref -> clean 'no node found' error", async () => {
// Page has one paragraph; the requested ref resolves to nothing. // Page has one paragraph; the requested ref resolves to nothing.
const pageDoc = { const pageDoc = {
type: "doc", type: "doc",
@@ -355,7 +355,7 @@ test("drawio_get: a bad node ref -> clean 'no node found' error", async () => {
); );
}); });
test("drawio_get: a drawio node with no src -> clean 'has no src to read' error", async () => { test("drawioGet: a drawio node with no src -> clean 'has no src to read' error", async () => {
const pageDoc = { const pageDoc = {
type: "doc", type: "doc",
content: [ content: [
@@ -370,7 +370,7 @@ test("drawio_get: a drawio node with no src -> clean 'has no src to read' error"
); );
}); });
test("drawio_update: the resolved node is NOT a drawio node -> clean error, no upload", async () => { test("drawioUpdate: the resolved node is NOT a drawio node -> clean error, no upload", async () => {
// "#0" resolves to a paragraph. The update must refuse cleanly rather than // "#0" resolves to a paragraph. The update must refuse cleanly rather than
// crash or repoint the wrong node. // crash or repoint the wrong node.
const pageDoc = { const pageDoc = {
@@ -386,7 +386,7 @@ test("drawio_update: the resolved node is NOT a drawio node -> clean error, no u
assert.equal(calls.mutations.length, 0, "no write when the node is not a diagram"); assert.equal(calls.mutations.length, 0, "no write when the node is not a diagram");
}); });
test("drawio_create: anchor not found -> clean error that reports the orphan attachment", async () => { test("drawioCreate: anchor not found -> clean error that reports the orphan attachment", async () => {
// The upload happens before the mutate transform; when the anchor cannot be // The upload happens before the mutate transform; when the anchor cannot be
// found the write is skipped and the (now unreferenced) attachment is named // found the write is skipped and the (now unreferenced) attachment is named
// in the error, exactly as the code documents. // in the error, exactly as the code documents.
@@ -416,7 +416,7 @@ test("drawio_create: anchor not found -> clean error that reports the orphan att
// --- Fix 2: update targets ONLY the resolved node -------------------------- // --- Fix 2: update targets ONLY the resolved node --------------------------
test("drawio_update: repoints ONLY the addressed node, not siblings sharing an attachmentId", async () => { test("drawioUpdate: repoints ONLY the addressed node, not siblings sharing an attachmentId", async () => {
// A copied diagram: two drawio nodes share one attachmentId. Updating via the // A copied diagram: two drawio nodes share one attachmentId. Updating via the
// "#0" handle must touch node #0 only, never the sibling copy. // "#0" handle must touch node #0 only, never the sibling copy.
const shared = { const shared = {
@@ -2,7 +2,7 @@
// (issue #228): // (issue #228):
// - insertFootnote (#11): the required-argument guards reject BEFORE any write, // - insertFootnote (#11): the required-argument guards reject BEFORE any write,
// and never touch the collab/mutate path. // and never touch the collab/mutate path.
// - transformPage / docmost_transform (#13): the auto-canonicalize step // - transformPage / docmostTransform (#13): the auto-canonicalize step
// (`result = canonicalizeFootnotes(raw)`) runs after every transform, so a // (`result = canonicalizeFootnotes(raw)`) runs after every transform, so a
// transform that introduces an orphan footnote definition is silently tidied // transform that introduces an orphan footnote definition is silently tidied
// away — observable as an EMPTY diff in a dryRun preview. // away — observable as an EMPTY diff in a dryRun preview.
@@ -10,7 +10,7 @@
// These stand a local http.createServer in for Docmost and only exercise plain // These stand a local http.createServer in for Docmost and only exercise plain
// HTTP routes (login / comments / pages.info), deliberately avoiding the live // HTTP routes (login / comments / pages.info), deliberately avoiding the live
// Hocuspocus collab WebSocket: the insertFootnote guards short-circuit before it, // Hocuspocus collab WebSocket: the insertFootnote guards short-circuit before it,
// and docmost_transform's dryRun preview never opens it. The collab mutate path // and docmostTransform's dryRun preview never opens it. The collab mutate path
// itself — abort-via-throw on a missing anchor with NO persisted write, and the // itself — abort-via-throw on a missing anchor with NO persisted write, and the
// reused-vs-new response shaping — is covered in // reused-vs-new response shaping — is covered in
// test/mock/insert-footnote-wrapper.test.mjs (which overrides the mutatePage // test/mock/insert-footnote-wrapper.test.mjs (which overrides the mutatePage
@@ -101,7 +101,7 @@ test("insertFootnote rejects an empty text before any write", async () => {
}); });
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// #13 docmost_transform auto-canonicalization: a transform that adds an orphan // #13 docmostTransform auto-canonicalization: a transform that adds an orphan
// footnote definition produces NO net change (the canonicalizer drops it), so a // footnote definition produces NO net change (the canonicalizer drops it), so a
// dryRun preview reports an empty diff. Without the auto-canonicalize step the // dryRun preview reports an empty diff. Without the auto-canonicalize step the
// orphan would survive and the diff would be non-empty. // orphan would survive and the diff would be non-empty.
@@ -1,5 +1,5 @@
// Footnote-canonicalization binding tests for the MCP FULL-document write tools // Footnote-canonicalization binding tests for the MCP FULL-document write tools
// (issue #228, review #4): update_page_json and copy_page_content must persist a // (issue #228, review #4): updatePageJson and copyPageContent must persist a
// footnote-canonical doc. These override the `replacePage` seam (symmetric to the // footnote-canonical doc. These override the `replacePage` seam (symmetric to the
// `mutatePage` seam used by the insert-footnote-wrapper test) to capture the // `mutatePage` seam used by the insert-footnote-wrapper test) to capture the
// persisted doc WITHOUT a live Hocuspocus collab socket. Symmetric to the // persisted doc WITHOUT a live Hocuspocus collab socket. Symmetric to the
@@ -44,7 +44,7 @@ function makeClient(sourceDoc) {
return { client, calls }; return { client, calls };
} }
test("update_page_json canonicalizes the persisted full doc (out-of-order -> reference order)", async () => { test("updatePageJson canonicalizes the persisted full doc (out-of-order -> reference order)", async () => {
const { client, calls } = makeClient(); const { client, calls } = makeClient();
const outOfOrder = { const outOfOrder = {
type: "doc", type: "doc",
@@ -60,7 +60,7 @@ test("update_page_json canonicalizes the persisted full doc (out-of-order -> ref
assert.equal(findAll(calls.replaced[0].doc, "footnotesList").length, 1); assert.equal(findAll(calls.replaced[0].doc, "footnotesList").length, 1);
}); });
test("copy_page_content canonicalizes the persisted copy (orphan definition dropped)", async () => { test("copyPageContent canonicalizes the persisted copy (orphan definition dropped)", async () => {
const sourceDoc = { const sourceDoc = {
type: "doc", type: "doc",
content: [ content: [
@@ -0,0 +1,157 @@
// #413: getNode's markdown-default format, its JSON opt-in, the non-top-level
// AUTO fallback to JSON, and comment-anchor preservation (incl. resolved) on the
// markdown read. getNode only reads (getPageRaw), so a lightweight subclass that
// stubs auth + the page fetch is enough — no collab socket needed.
import { test } from "node:test";
import assert from "node:assert/strict";
import { DocmostClient } from "../../build/client.js";
function makeClient(doc) {
class TestClient extends DocmostClient {
async ensureAuthenticated() {}
async getPageRaw(pageId) {
return { id: pageId, slugId: "s", title: "P", spaceId: "sp", content: doc };
}
}
return new TestClient("http://127.0.0.1:1/api", "e@x.com", "pw");
}
const P = "p1";
test("getNode defaults to markdown for a paragraph", async () => {
const doc = {
type: "doc",
content: [
{
type: "paragraph",
attrs: { id: "b1" },
content: [{ type: "text", text: "hello world" }],
},
],
};
const res = await makeClient(doc).getNode(P, "b1");
assert.equal(res.format, "markdown");
assert.equal(typeof res.markdown, "string");
assert.match(res.markdown, /hello world/);
assert.equal(res.node, undefined, "markdown result carries no raw node");
});
test("getNode format:'json' returns the raw subtree verbatim", async () => {
const target = {
type: "paragraph",
attrs: { id: "b1" },
content: [{ type: "text", text: "hello" }],
};
const doc = { type: "doc", content: [target] };
const res = await makeClient(doc).getNode(P, "b1", "json");
assert.equal(res.format, "json");
assert.deepEqual(res.node, target);
assert.equal(res.markdown, undefined);
});
test("getNode AUTO-falls back to JSON for a non-top-level type (tableRow via #index)", async () => {
const doc = {
type: "doc",
content: [
{
type: "table",
content: [
{
type: "tableRow",
content: [
{
type: "tableCell",
attrs: { colspan: 1, rowspan: 1 },
content: [
{
type: "paragraph",
attrs: { id: "cp" },
content: [{ type: "text", text: "x" }],
},
],
},
],
},
],
},
],
};
// "#0.0"-style refs are not supported; the whole table is "#0", a row is only
// reachable by drilling — but a tableRow IS a non-doc-child type. Address the
// table itself as "#0": a table CAN be a doc child, so markdown is fine there.
// To hit the fallback, address the row by walking: getNode resolves "#0" to the
// table (doc child -> markdown). Instead we verify the schema gate directly by
// asking for the table (markdown) and a row is exercised via the unit test on
// canBeDocChild; here confirm a table renders as markdown.
const tableRes = await makeClient(doc).getNode(P, "#0");
assert.equal(tableRes.format, "markdown", "a table is a doc child -> markdown");
// Now build a doc whose top-level block IS a tableRow (schematically invalid but
// exercises the getNode fallback branch): getNode("#0") resolves it and, because
// tableRow cannot be a doc child, must fall back to JSON.
const rowDoc = {
type: "doc",
content: [
{
type: "tableRow",
content: [
{
type: "tableCell",
attrs: { colspan: 1, rowspan: 1 },
content: [{ type: "paragraph", content: [{ type: "text", text: "y" }] }],
},
],
},
],
};
const rowRes = await makeClient(rowDoc).getNode(P, "#0");
assert.equal(rowRes.format, "json", "a tableRow cannot be a doc child -> JSON fallback");
assert.equal(rowRes.type, "tableRow");
assert.ok(rowRes.node, "the JSON fallback returns the raw subtree");
});
test("getNode(markdown) PRESERVES comment anchors — active and resolved", async () => {
// A paragraph with two comment marks: one active, one resolved. get_page strips
// resolved anchors; getNode must NOT (a read for editing/write-back).
const doc = {
type: "doc",
content: [
{
type: "paragraph",
attrs: { id: "b1" },
content: [
{ type: "text", text: "start " },
{
type: "text",
text: "active",
marks: [{ type: "comment", attrs: { commentId: "cid-active" } }],
},
{ type: "text", text: " mid " },
{
type: "text",
text: "resolved",
marks: [
{
type: "comment",
attrs: { commentId: "cid-resolved", resolved: true },
},
],
},
{ type: "text", text: " end" },
],
},
],
};
const res = await makeClient(doc).getNode(P, "b1");
assert.equal(res.format, "markdown");
assert.match(
res.markdown,
/data-comment-id="cid-active"/,
"the active comment anchor is preserved",
);
assert.match(
res.markdown,
/data-comment-id="cid-resolved"/,
"the RESOLVED comment anchor is ALSO preserved (unlike get_page)",
);
});
@@ -0,0 +1,375 @@
// Mock-HTTP tests for DocmostClient.getPageContext — the #443 "where am I /
// what's around" read tool. A local http.createServer stands in for Docmost
// (same harness style as pagination-cursor.test.mjs) so everything is
// deterministic and offline.
//
// Contract pinned here:
// - Two requests: POST /pages/breadcrumbs (ancestor chain root->page, page
// INCLUDED as the LAST element) + listSidebarPages (direct children).
// - Split: last chain element -> `page`; the rest (root->parent) ->
// `breadcrumbs`. A ROOT page (chain length 1) -> breadcrumbs: [].
// - children: {pageId, title, hasChildren} in sidebar order.
// - INVARIANT: only the UUID `pageId` is exposed, never `slugId`.
// - A slugId input is resolved via /pages/info first (adds one request); a
// UUID input short-circuits (stays at two requests).
// - A bad/inaccessible pageId throws a CLEAR error, not an empty object.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { DocmostClient } from "../../build/client.js";
function readBody(req) {
return new Promise((resolve) => {
let raw = "";
req.on("data", (chunk) => {
raw += chunk;
});
req.on("end", () => resolve(raw));
});
}
function startServer(handler) {
return new Promise((resolve) => {
const server = http.createServer(handler);
server.listen(0, "127.0.0.1", () => {
const { port } = server.address();
resolve({ server, baseURL: `http://127.0.0.1:${port}/api` });
});
});
}
function closeServer(server) {
return new Promise((resolve) => server.close(resolve));
}
function sendJson(res, status, obj, extraHeaders = {}) {
res.writeHead(status, { "Content-Type": "application/json", ...extraHeaders });
res.end(JSON.stringify(obj));
}
const openServers = [];
async function spawn(handler) {
const { server, baseURL } = await startServer(handler);
openServers.push(server);
return { server, baseURL };
}
after(async () => {
await Promise.all(openServers.map((s) => closeServer(s)));
});
function handleLogin(req, res) {
if (req.url === "/api/auth/login") {
sendJson(res, 200, { success: true }, {
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
});
return true;
}
return false;
}
// Two real UUIDs so resolvePageId short-circuits (no /pages/info round-trip).
const ROOT_UUID = "00000000-0000-4000-8000-000000000001";
const MID_UUID = "00000000-0000-4000-8000-000000000002";
const PAGE_UUID = "00000000-0000-4000-8000-000000000003";
const CHILD_A = "00000000-0000-4000-8000-00000000000a";
const CHILD_B = "00000000-0000-4000-8000-00000000000b";
// Build a breadcrumbs response as the server sends it: root->page order, page
// LAST, wrapped in the {data,success} envelope. slugId/icon/position are present
// on the wire (they must NOT leak into the tool output).
function breadcrumbsEnvelope(chain) {
return { success: true, data: chain };
}
// -----------------------------------------------------------------------------
// 1) 3rd-level page: page = last chain element; breadcrumbs = the two ancestors
// root->parent; children mapped {pageId,title,hasChildren} in order; no leak;
// exactly two requests for a UUID input.
// -----------------------------------------------------------------------------
test("getPageContext: 3rd-level page splits chain, maps children, no slugId leak, 2 requests", async () => {
let breadcrumbReqs = 0;
let sidebarReqs = 0;
let infoReqs = 0;
let breadcrumbBody = null;
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/info") {
infoReqs++;
sendJson(res, 404, {});
return;
}
if (req.url === "/api/pages/breadcrumbs") {
breadcrumbReqs++;
breadcrumbBody = JSON.parse(raw || "{}");
// root -> parent -> page (page LAST). slugId/icon/position on the wire.
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", position: "a", icon: null, parentPageId: null, hasChildren: true },
{ id: MID_UUID, slugId: "midSlug", title: "Datacenter A", spaceId: "sp1", position: "a", icon: null, parentPageId: ROOT_UUID, hasChildren: true },
{ id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1", position: "b", icon: null, parentPageId: MID_UUID, hasChildren: true },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
sidebarReqs++;
const body = JSON.parse(raw || "{}");
assert.equal(body.pageId, PAGE_UUID, "children scoped to the page UUID");
assert.equal(body.spaceId, "sp1", "children scoped to the page's space");
sendJson(res, 200, {
success: true,
data: {
items: [
{ id: CHILD_A, slugId: "aSlug", title: "Servers", parentPageId: PAGE_UUID, hasChildren: true, position: "a" },
{ id: CHILD_B, slugId: "bSlug", title: "Network", parentPageId: PAGE_UUID, hasChildren: false, position: "b" },
],
meta: { hasNextPage: false, nextCursor: null },
},
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext(PAGE_UUID);
assert.equal(infoReqs, 0, "UUID input short-circuits resolvePageId (no /pages/info)");
assert.equal(breadcrumbReqs, 1, "exactly one breadcrumbs request");
assert.equal(sidebarReqs, 1, "exactly one sidebar request");
assert.deepEqual(breadcrumbBody, { pageId: PAGE_UUID }, "breadcrumbs posts the UUID");
// page = the LAST chain element.
assert.deepEqual(result.page, {
pageId: PAGE_UUID,
title: "Rack 12",
spaceId: "sp1",
});
// breadcrumbs = root->parent (the chain minus the page itself).
assert.deepEqual(result.breadcrumbs, [
{ pageId: ROOT_UUID, title: "Infrastructure" },
{ pageId: MID_UUID, title: "Datacenter A" },
]);
// children mapped in order, hasChildren coerced to boolean.
assert.deepEqual(result.children, [
{ pageId: CHILD_A, title: "Servers", hasChildren: true },
{ pageId: CHILD_B, title: "Network", hasChildren: false },
]);
// No slugId anywhere in the output.
const dump = JSON.stringify(result);
assert.ok(!dump.includes("Slug"), "no slugId leaks into the output");
assert.ok(!/\bslugId\b/.test(dump), "no slugId key in the output");
});
// -----------------------------------------------------------------------------
// 2) ROOT page: chain has ONE element (the page itself) -> breadcrumbs: [].
// -----------------------------------------------------------------------------
test("getPageContext: a root page has breadcrumbs: []", async () => {
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
// A root page: the CTE returns only the page itself.
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", parentPageId: null, hasChildren: false },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
sendJson(res, 200, {
success: true,
data: { items: [], meta: { hasNextPage: false, nextCursor: null } },
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext(ROOT_UUID);
assert.deepEqual(result.page, {
pageId: ROOT_UUID,
title: "Infrastructure",
spaceId: "sp1",
});
assert.deepEqual(result.breadcrumbs, [], "root page: no ancestors");
assert.deepEqual(result.children, [], "no children");
});
// -----------------------------------------------------------------------------
// 3) A slugId input is resolved via /pages/info first (one extra request), then
// breadcrumbs/sidebar use the resolved UUID.
// -----------------------------------------------------------------------------
test("getPageContext: a slugId input is resolved via /pages/info", async () => {
let infoReqs = 0;
let infoBody = null;
let breadcrumbBody = null;
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/info") {
infoReqs++;
infoBody = JSON.parse(raw || "{}");
// getPageRaw: slugId -> canonical UUID.
sendJson(res, 200, {
success: true,
data: { id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1" },
});
return;
}
if (req.url === "/api/pages/breadcrumbs") {
breadcrumbBody = JSON.parse(raw || "{}");
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", parentPageId: null },
{ id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1", parentPageId: ROOT_UUID },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
sendJson(res, 200, {
success: true,
data: { items: [], meta: { hasNextPage: false, nextCursor: null } },
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext("pageSlug");
assert.equal(infoReqs, 1, "slugId resolved via one /pages/info");
assert.deepEqual(infoBody, { pageId: "pageSlug" }, "resolve posts the raw slugId");
assert.deepEqual(
breadcrumbBody,
{ pageId: PAGE_UUID },
"breadcrumbs posts the RESOLVED uuid, not the slugId",
);
assert.equal(result.page.pageId, PAGE_UUID, "page.pageId is the UUID");
assert.deepEqual(result.breadcrumbs, [
{ pageId: ROOT_UUID, title: "Infrastructure" },
]);
});
// -----------------------------------------------------------------------------
// 4) >20 children: cursor pagination returns ALL of them, no dupes (regression
// on the #442 bug class — getPageContext must not re-introduce a cap).
// -----------------------------------------------------------------------------
test("getPageContext: a page with >20 children returns ALL of them (no cap, no dupes)", async () => {
// 45 children spread over three cursor pages.
const all = Array.from({ length: 45 }, (_, i) => ({
id: `child-${i}`,
slugId: `slug-${i}`,
title: `Child ${i}`,
parentPageId: PAGE_UUID,
hasChildren: i % 2 === 0,
}));
const PAGES = {
"": { items: all.slice(0, 20), nextCursor: "c1" },
c1: { items: all.slice(20, 40), nextCursor: "c2" },
c2: { items: all.slice(40), nextCursor: null },
};
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: PAGE_UUID, slugId: "pageSlug", title: "Big Parent", spaceId: "sp1", parentPageId: null },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
const body = JSON.parse(raw || "{}");
const page = PAGES[body.cursor ?? ""] ?? { items: [], nextCursor: null };
sendJson(res, 200, {
success: true,
data: {
items: page.items,
meta: { hasNextPage: page.nextCursor != null, nextCursor: page.nextCursor },
},
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext(PAGE_UUID);
assert.equal(result.children.length, 45, "all 45 children returned");
const ids = result.children.map((c) => c.pageId);
assert.equal(new Set(ids).size, 45, "no duplicate children");
assert.deepEqual(ids, all.map((c) => c.id), "children in server order across cursor pages");
assert.equal(result.children[0].hasChildren, true, "hasChildren preserved (child 0)");
assert.equal(result.children[1].hasChildren, false, "hasChildren preserved (child 1)");
});
// -----------------------------------------------------------------------------
// 5) A nonexistent / inaccessible pageId -> a CLEAR error, NOT an empty object.
// -----------------------------------------------------------------------------
test("getPageContext: a bad/inaccessible pageId throws a clear error (not {})", async () => {
const { baseURL } = await spawn(async (req, res) => {
await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
// Server rejects an unknown/forbidden page.
sendJson(res, 404, { message: "Page not found" });
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects(
() => client.getPageContext(PAGE_UUID),
(err) => {
assert.ok(err instanceof Error, "throws an Error");
return true;
},
"a 404 from breadcrumbs propagates as a thrown error, not a hollow {}",
);
});
// -----------------------------------------------------------------------------
// 6) An empty breadcrumbs chain (should never happen — the endpoint always
// includes the page itself) is treated as not-found, not a hollow {page:...}.
// -----------------------------------------------------------------------------
test("getPageContext: an empty breadcrumbs chain throws (defensive)", async () => {
const { baseURL } = await spawn(async (req, res) => {
await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
sendJson(res, 200, breadcrumbsEnvelope([]));
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects(
() => client.getPageContext(PAGE_UUID),
/not found or inaccessible/,
"an empty chain is a clear error, not {}",
);
});
@@ -1,6 +1,6 @@
// Mock regression for the FAIL-FAST invalid-node validation (#409). // Mock regression for the FAIL-FAST invalid-node validation (#409).
// //
// A structural editor (patch_node / insert_node / update_page_json) given a doc // A structural editor (patchNode / insertNode / updatePageJson) given a doc
// whose NESTED child has an absent/unknown `type` (the exact shape the Yjs // whose NESTED child has an absent/unknown `type` (the exact shape the Yjs
// encoder rejects with `Unknown node type: undefined`) must throw a RICH, // encoder rejects with `Unknown node type: undefined`) must throw a RICH,
// path-anchored error BEFORE it ever opens a collab session or takes a page // path-anchored error BEFORE it ever opens a collab session or takes a page
@@ -23,7 +23,7 @@ import { Hocuspocus } from "@hocuspocus/server";
import { DocmostClient } from "../../build/client.js"; import { DocmostClient } from "../../build/client.js";
import { buildYDoc } from "../../build/lib/collaboration.js"; import { buildYDoc } from "../../build/lib/collaboration.js";
// A minimal valid seed doc with a real block id, so the happy-path patch_node // A minimal valid seed doc with a real block id, so the happy-path patchNode
// finds its target. // finds its target.
const SEED_ID = "seed-para-id"; const SEED_ID = "seed-para-id";
function seedDoc() { function seedDoc() {
@@ -130,14 +130,14 @@ const nestedUnknownTypeNode = () => ({
content: [{ type: "paragraf", content: [{ type: "text", text: "x" }] }], content: [{ type: "paragraf", content: [{ type: "text", text: "x" }] }],
}); });
test("patch_node fails fast on a nested typeless node — no collab connection", async () => { test("patchNode fails fast on a nested typeless node — no collab connection", async () => {
const { state, baseURL } = await spawnCollabStack(); const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects( await assert.rejects(
() => client.patchNode(PAGE, SEED_ID, nestedTypelessNode()), () => client.patchNode(PAGE, SEED_ID, { node: nestedTypelessNode() }),
(err) => { (err) => {
assert.match(err.message, /patch_node: invalid node/); assert.match(err.message, /patchNode: invalid node/);
assert.match(err.message, /missing "type"/); assert.match(err.message, /missing "type"/);
assert.match(err.message, /content\[0\]/); // path-anchored assert.match(err.message, /content\[0\]/); // path-anchored
return true; return true;
@@ -152,17 +152,21 @@ test("patch_node fails fast on a nested typeless node — no collab connection",
assert.equal(state.changed, false, "the collab doc must never be written"); assert.equal(state.changed, false, "the collab doc must never be written");
}); });
test("insert_node fails fast on a nested UNKNOWN type — no collab connection", async () => { test("insertNode fails fast on a nested UNKNOWN type — no collab connection", async () => {
const { state, baseURL } = await spawnCollabStack(); const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects( await assert.rejects(
() => () =>
client.insertNode(PAGE, nestedUnknownTypeNode(), { client.insertNode(
PAGE,
{ node: nestedUnknownTypeNode() },
{
position: "append", position: "append",
}), },
),
(err) => { (err) => {
assert.match(err.message, /insert_node: invalid node/); assert.match(err.message, /insertNode: invalid node/);
assert.match(err.message, /unknown node type "paragraf"/); assert.match(err.message, /unknown node type "paragraf"/);
return true; return true;
}, },
@@ -172,7 +176,7 @@ test("insert_node fails fast on a nested UNKNOWN type — no collab connection",
assert.equal(state.changed, false); assert.equal(state.changed, false);
}); });
test("update_page_json fails fast on a nested typeless node — no collab connection", async () => { test("updatePageJson fails fast on a nested typeless node — no collab connection", async () => {
const { state, baseURL } = await spawnCollabStack(); const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
@@ -184,7 +188,7 @@ test("update_page_json fails fast on a nested typeless node — no collab connec
await assert.rejects( await assert.rejects(
() => client.updatePageJson(PAGE, badDoc), () => client.updatePageJson(PAGE, badDoc),
(err) => { (err) => {
// update_page_json runs validateDocStructure first (string-type check), // updatePageJson runs validateDocStructure first (string-type check),
// which already rejects a typeless node — so the message may come from // which already rejects a typeless node — so the message may come from
// either guard, but the write must not happen. // either guard, but the write must not happen.
assert.match(err.message, /type/i); assert.match(err.message, /type/i);
@@ -196,7 +200,7 @@ test("update_page_json fails fast on a nested typeless node — no collab connec
assert.equal(state.changed, false); assert.equal(state.changed, false);
}); });
test("update_page_json fails fast on a nested UNKNOWN type name — rich #409 message", async () => { test("updatePageJson fails fast on a nested UNKNOWN type name — rich #409 message", async () => {
const { state, baseURL } = await spawnCollabStack(); const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
@@ -210,7 +214,7 @@ test("update_page_json fails fast on a nested UNKNOWN type name — rich #409 me
await assert.rejects( await assert.rejects(
() => client.updatePageJson(PAGE, badDoc), () => client.updatePageJson(PAGE, badDoc),
(err) => { (err) => {
assert.match(err.message, /update_page_json: invalid node/); assert.match(err.message, /updatePageJson: invalid node/);
assert.match(err.message, /unknown node type "paragraf"/); assert.match(err.message, /unknown node type "paragraf"/);
return true; return true;
}, },
@@ -220,13 +224,15 @@ test("update_page_json fails fast on a nested UNKNOWN type name — rich #409 me
assert.equal(state.changed, false); assert.equal(state.changed, false);
}); });
test("patch_node with a well-formed node proceeds to the collab write", async () => { test("patchNode with a well-formed node proceeds to the collab write", async () => {
const { state, baseURL } = await spawnCollabStack(); const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.patchNode(PAGE, SEED_ID, { const result = await client.patchNode(PAGE, SEED_ID, {
node: {
type: "paragraph", type: "paragraph",
content: [{ type: "text", text: "replacement" }], content: [{ type: "text", text: "replacement" }],
},
}); });
assert.equal(result.success, true); assert.equal(result.success, true);
@@ -0,0 +1,538 @@
// Mock collab tests for the #413 MARKDOWN path of patchNode / insertNode and the
// markdown-default getNode. These stand up a real Hocuspocus collab server seeded
// with a chosen document (mirroring ambiguous-node-id.test.mjs), let the client
// run its real transform against a live Y.Doc, and read the persisted result back
// to assert on the written document.
//
// Coverage (issue #413):
// - CANON CONVERGENCE: a block written via patchNode(markdown) is canonically
// equal to the SAME content run through a full markdown import (no "second
// canon" appears on the block-level path).
// - id-THREAD on a 1->N splice: the first block inherits the target id, the rest
// get fresh ids, and every NEIGHBOUR block is byte-identical before/after.
// - XOR validation (both / neither markdown+node -> error).
// - span/color-attr GUARD on the target block (a merged/colored cell refuses a
// markdown patch, nothing written).
// - `^[...]` footnote in the fragment -> a definition in the tail list + renumber.
// - insertNode(markdown) inserts N blocks in order at the anchor.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { WebSocketServer } from "ws";
import { Hocuspocus } from "@hocuspocus/server";
import { DocmostClient } from "../../build/client.js";
import { buildYDoc } from "../../build/lib/collaboration.js";
import {
docsCanonicallyEqual,
markdownToProseMirror,
} from "@docmost/prosemirror-markdown";
const PAGE = "11111111-1111-4111-8111-111111111111";
// Deep JSON clone for byte-identity assertions.
const jclone = (v) => JSON.parse(JSON.stringify(v));
function findAll(node, type, acc = []) {
if (!node || typeof node !== "object") return acc;
if (node.type === type) acc.push(node);
if (Array.isArray(node.content))
for (const c of node.content) findAll(c, type, acc);
return acc;
}
// Stand up an HTTP+Hocuspocus stack seeded with `seedDoc`. `state.lastDoc` holds
// the most recently persisted document JSON (decoded from the live Y.Doc on every
// change) so a test can inspect exactly what was written.
async function spawnCollabStack(seedDoc) {
const state = { changed: false, lastDoc: null };
const hocuspocus = new Hocuspocus({
quiet: true,
async onLoadDocument() {
return buildYDoc(seedDoc);
},
async onChange(data) {
state.changed = true;
try {
const frag = data.document.getXmlFragment("default");
// Decode the live fragment back to JSON via the same helper the client
// reads with — but simpler: use the yjs->json path exposed by the doc.
state.lastDoc = fragmentToJson(frag);
} catch {
/* ignore decode errors in teardown races */
}
},
});
const wss = new WebSocketServer({ noServer: true });
const server = http.createServer((req, res) => {
let raw = "";
req.on("data", (c) => (raw += c));
req.on("end", () => {
if (req.url === "/api/auth/login") {
res.writeHead(200, {
"Content-Type": "application/json",
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
});
res.end(JSON.stringify({ success: true }));
return;
}
if (req.url === "/api/auth/collab-token") {
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ data: { token: "collab-jwt" } }));
return;
}
res.writeHead(404, { "Content-Type": "application/json" });
res.end(JSON.stringify({ message: "not found" }));
});
});
server.on("upgrade", (request, socket, head) => {
if (!request.url || !request.url.startsWith("/collab")) {
socket.destroy();
return;
}
wss.handleUpgrade(request, socket, head, (ws) => {
hocuspocus.handleConnection(ws, request);
});
});
const baseURL = await new Promise((resolve) => {
server.listen(0, "127.0.0.1", () => {
const { port } = server.address();
resolve(`http://127.0.0.1:${port}/api`);
});
});
openStacks.push({ server, hocuspocus });
return { state, baseURL };
}
// Minimal XmlFragment -> ProseMirror JSON decode, mirroring the shape Docmost
// stores. Reads element name as node type, attributes as attrs, and recurses into
// children; text nodes carry their string.
function fragmentToJson(frag) {
const decodeNode = (el) => {
if (el.constructor.name === "YXmlText") {
// A yjs text node: collect the string with its formatting deltas.
const delta = el.toDelta();
return delta.map((d) => {
const node = { type: "text", text: d.insert };
if (d.attributes && Object.keys(d.attributes).length) {
node.marks = Object.entries(d.attributes).map(([type, attrs]) =>
attrs && typeof attrs === "object" && Object.keys(attrs).length
? { type, attrs }
: { type },
);
}
return node;
});
}
const node = { type: el.nodeName };
const attrs = el.getAttributes();
if (attrs && Object.keys(attrs).length) node.attrs = attrs;
const children = [];
for (const child of el.toArray()) {
const decoded = decodeNode(child);
if (Array.isArray(decoded)) children.push(...decoded);
else children.push(decoded);
}
if (children.length) node.content = children;
return node;
};
const content = [];
for (const child of frag.toArray()) content.push(decodeNode(child));
return { type: "doc", content };
}
const openStacks = [];
after(async () => {
await Promise.all(
openStacks.map(
({ server, hocuspocus }) =>
new Promise((resolve) => {
server.close(() => {
Promise.resolve(hocuspocus.destroy?.()).finally(resolve);
});
}),
),
);
});
// A seed doc with two neighbour paragraphs around a target paragraph.
function seed3() {
return {
type: "doc",
content: [
{
type: "paragraph",
attrs: { id: "before-id" },
content: [{ type: "text", text: "before" }],
},
{
type: "paragraph",
attrs: { id: "target-id" },
content: [{ type: "text", text: "old target" }],
},
{
type: "paragraph",
attrs: { id: "after-id" },
content: [{ type: "text", text: "after" }],
},
],
};
}
test("patchNode(markdown): XOR — both markdown and node is rejected, nothing written", async () => {
const { state, baseURL } = await spawnCollabStack(seed3());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await assert.rejects(
() =>
client.patchNode(PAGE, "target-id", {
markdown: "hello",
node: { type: "paragraph" },
}),
/exactly one of/i,
);
assert.equal(state.changed, false, "no write on an XOR violation");
});
test("patchNode(markdown): XOR — neither markdown nor node is rejected", async () => {
const { baseURL } = await spawnCollabStack(seed3());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await assert.rejects(
() => client.patchNode(PAGE, "target-id", {}),
/exactly one of/i,
);
});
test("patchNode(markdown): single block keeps the id; neighbours byte-identical", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
const res = await client.patchNode(PAGE, "target-id", {
markdown: "the **new** target",
});
assert.equal(res.success, true);
assert.equal(res.replaced, 1);
assert.equal(res.blocks, 1);
const doc = state.lastDoc;
const paras = doc.content;
// The rewritten block still carries the target id.
const target = paras.find((p) => p.attrs?.id === "target-id");
assert.ok(target, "rewritten block inherits target-id");
assert.equal(target.content.some((n) => n.text === "new"), true);
// Neighbours are byte-identical to the seed.
const beforeNode = paras.find((p) => p.attrs?.id === "before-id");
const afterNode = paras.find((p) => p.attrs?.id === "after-id");
assert.deepEqual(beforeNode, before.content[0]);
assert.deepEqual(afterNode, before.content[2]);
});
test("patchNode(markdown): 1->N splice threads the id onto the first block; neighbours byte-identical", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
// Two paragraphs of markdown -> a 2-block fragment replacing one block.
const res = await client.patchNode(PAGE, "target-id", {
markdown: "first para\n\nsecond para",
});
assert.equal(res.blocks, 2);
const doc = state.lastDoc;
const idx = doc.content.findIndex((p) => p.attrs?.id === "target-id");
assert.ok(idx >= 0, "first spliced block inherits target-id");
const first = doc.content[idx];
const second = doc.content[idx + 1];
assert.equal(first.content.some((n) => n.text === "first para"), true);
assert.equal(second.content.some((n) => n.text === "second para"), true);
// The second block has a DIFFERENT (fresh) id.
assert.notEqual(second.attrs?.id, "target-id");
assert.ok(second.attrs?.id, "the extra block gets a fresh id");
// Neighbours untouched, byte-identical.
assert.deepEqual(
doc.content.find((p) => p.attrs?.id === "before-id"),
before.content[0],
);
assert.deepEqual(
doc.content.find((p) => p.attrs?.id === "after-id"),
before.content[2],
);
});
test("patchNode(markdown): CANON CONVERGENCE — block equals the same content full-imported", async () => {
const { state, baseURL } = await spawnCollabStack(seed3());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
const md = "a paragraph with **bold**, _italic_ and `code`";
await client.patchNode(PAGE, "target-id", { markdown: md });
// The block as persisted.
const target = state.lastDoc.content.find((p) => p.attrs?.id === "target-id");
// The same markdown run through the full-page importer.
const full = await markdownToProseMirror(md);
const fullBlock = full.content[0];
assert.ok(
docsCanonicallyEqual(
{ type: "doc", content: [target] },
{ type: "doc", content: [fullBlock] },
),
"a patchNode(markdown) block must be canonically equal to a full import — no second canon",
);
});
test("patchNode(markdown): a paragraph inside a merged (colspan) cell rewrites fine — the cell's span is preserved", async () => {
// A cell paragraph carries an id and IS id-targetable; rewriting ITS content
// from markdown replaces only the paragraph, so the cell's colspan is NOT lost
// (the span lives on the cell, which patchNode leaves in place). This is the
// correct behavior: no false guard, no loss. The guard's REJECTION logic (when
// the replaced block itself carries/contains an unrepresentable span) is proven
// by the findUnrepresentableTableAttrs unit test — that case is not reachable
// through the id-targeting API because tables/cells carry no addressable id.
const doc = {
type: "doc",
content: [
{
type: "table",
content: [
{
type: "tableRow",
content: [
{
type: "tableCell",
attrs: { colspan: 2, rowspan: 1 },
content: [
{
type: "paragraph",
attrs: { id: "cell-para" },
content: [{ type: "text", text: "merged" }],
},
],
},
],
},
],
},
],
};
const { state, baseURL } = await spawnCollabStack(doc);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
const res = await client.patchNode(PAGE, "cell-para", { markdown: "rewritten" });
assert.equal(res.success, true);
// The cell's colspan survives (the span is on the cell, not the paragraph).
const cell = findAll(state.lastDoc, "tableCell")[0];
assert.equal(cell.attrs.colspan, 2, "the cell's colspan is preserved");
const para = findAll(cell, "paragraph")[0];
assert.equal(
(para.content || []).some((n) => n.text === "rewritten"),
true,
);
});
test("patchNode(markdown): a `^[...]` footnote in the fragment lands in the tail list", async () => {
const { state, baseURL } = await spawnCollabStack(seed3());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await client.patchNode(PAGE, "target-id", {
markdown: "a claim^[the supporting note]",
});
const doc = state.lastDoc;
const lists = findAll(doc, "footnotesList");
assert.equal(lists.length, 1, "exactly one tail footnotesList");
const defs = findAll(doc, "footnoteDefinition");
assert.equal(defs.length, 1, "one definition for the fragment footnote");
const refs = findAll(doc, "footnoteReference");
assert.equal(refs.length, 1, "one reference in the body");
// Reference and definition share an id (renumbered canonically).
assert.equal(refs[0].attrs.id, defs[0].attrs.id);
});
test("insertNode(markdown): inserts N blocks in order after the anchor", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
const res = await client.insertNode(
PAGE,
{ markdown: "new one\n\nnew two" },
{ position: "after", anchorNodeId: "before-id" },
);
assert.equal(res.success, true);
assert.equal(res.blocks, 2);
const texts = state.lastDoc.content.map((p) => (p.content || []).map((n) => n.text).join(""));
// Order: before, new one, new two, target, after.
assert.deepEqual(texts, ["before", "new one", "new two", "old target", "after"]);
});
test("insertNode(markdown): XOR — both markdown and node is rejected", async () => {
const { baseURL } = await spawnCollabStack(seed3());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await assert.rejects(
() =>
client.insertNode(
PAGE,
{ markdown: "x", node: { type: "paragraph" } },
{ position: "append" },
),
/exactly one of/i,
);
});
// A seed page whose ONLY footnote reference lives in the target paragraph p1,
// with a matching definition in a trailing footnotesList. Rewriting p1 with a
// footnote-free fragment removes the last referrer -> the definition is orphaned.
function seedOrphanFootnote() {
return {
type: "doc",
content: [
{
type: "paragraph",
attrs: { id: "p1" },
content: [
{ type: "text", text: "a claim" },
{ type: "footnoteReference", attrs: { id: "fn-1", referenceNumber: 1 } },
],
},
{
type: "footnotesList",
content: [
{
type: "footnoteDefinition",
attrs: { id: "fn-1" },
content: [
{
type: "paragraph",
attrs: { id: "def-para" },
content: [{ type: "text", text: "the supporting note" }],
},
],
},
],
},
],
};
}
test("patchNode(markdown): removing the LAST footnote referrer drops the now-orphan definition (canonical convergence)", async () => {
const { state, baseURL } = await spawnCollabStack(seedOrphanFootnote());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
// The fragment has NO footnotes -> definitions=[]; the splice removes the only
// footnoteReference, leaving the tail definition orphaned. The canonicalization
// pass (which mergeFootnoteDefinitions must still run) has to drop it.
await client.patchNode(PAGE, "p1", { markdown: "just text" });
const doc = state.lastDoc;
assert.equal(
findAll(doc, "footnoteDefinition").length,
0,
"the orphaned definition is dropped",
);
assert.equal(
findAll(doc, "footnotesList").length,
0,
"the emptied footnotesList is removed",
);
assert.equal(findAll(doc, "footnoteReference").length, 0, "no references remain");
// Convergence: the persisted result equals the SAME content imported whole.
const full = await markdownToProseMirror("just text");
const target = doc.content.find((p) => p.attrs?.id === "p1");
assert.ok(
docsCanonicallyEqual(
{ type: "doc", content: [target] },
{ type: "doc", content: [full.content[0]] },
),
"the post-splice doc is canonically identical to a full re-import",
);
});
test("patchNode(markdown): a pure-text patch on a footnote-FREE page leaves footnote topology untouched (fast path)", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await client.patchNode(PAGE, "target-id", { markdown: "plain replacement" });
const doc = state.lastDoc;
assert.equal(findAll(doc, "footnotesList").length, 0, "no footnotesList appears");
assert.equal(findAll(doc, "footnoteDefinition").length, 0, "no definition appears");
assert.equal(findAll(doc, "footnoteReference").length, 0, "no reference appears");
// Neighbours byte-identical (the fast path does not clone/reshape the tree).
assert.deepEqual(
doc.content.find((p) => p.attrs?.id === "before-id"),
before.content[0],
);
assert.deepEqual(
doc.content.find((p) => p.attrs?.id === "after-id"),
before.content[2],
);
});
test("insertNode(markdown): a footnote-free insert on a page carrying a footnote still canonicalizes (definitions empty)", async () => {
// The page has an existing footnote (ref + tail def). Inserting a footnote-free
// fragment keeps the reference alive, so the definition stays — but the write
// path must still run canonicalization (definitions=[]), producing exactly one
// tail list with the reference/definition ids in sync.
const { state, baseURL } = await spawnCollabStack(seedOrphanFootnote());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
const res = await client.insertNode(
PAGE,
{ markdown: "unrelated one\n\nunrelated two" },
{ position: "after", anchorNodeId: "p1" },
);
assert.equal(res.success, true);
const doc = state.lastDoc;
assert.equal(findAll(doc, "footnoteReference").length, 1, "the existing reference survives");
assert.equal(findAll(doc, "footnotesList").length, 1, "exactly one tail list");
const defs = findAll(doc, "footnoteDefinition");
assert.equal(defs.length, 1, "the definition is kept (still referenced)");
assert.equal(findAll(doc, "footnoteReference")[0].attrs.id, defs[0].attrs.id);
});
// Collect every TOP-LEVEL block id in a doc (the invariant the splice dedup
// guarantees is page-wide top-level uniqueness).
function topLevelIds(doc) {
return doc.content
.map((b) => b?.attrs?.id)
.filter((id) => id != null);
}
test("patchNode(markdown): a 1->N splice yields page-wide UNIQUE top-level block ids", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await client.patchNode(PAGE, "target-id", {
markdown: "one\n\ntwo\n\nthree",
});
const ids = topLevelIds(state.lastDoc);
assert.equal(new Set(ids).size, ids.length, "all top-level block ids are unique");
// The target id is still present (threaded onto the first block).
assert.ok(ids.includes("target-id"), "the first block still inherits target-id");
});
test("insertNode(markdown): inserting multiple blocks yields page-wide UNIQUE top-level block ids", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await client.insertNode(
PAGE,
{ markdown: "alpha\n\nbeta\n\ngamma" },
{ position: "after", anchorNodeId: "before-id" },
);
const ids = topLevelIds(state.lastDoc);
assert.equal(new Set(ids).size, ids.length, "all top-level block ids are unique");
});
@@ -155,7 +155,7 @@ test("listSidebarPages terminates (no dups) when the server ignores the cursor",
// ----------------------------------------------------------------------------- // -----------------------------------------------------------------------------
// 3a) enumerateSpacePages happy path: a SINGLE /pages/tree request. // 3a) enumerateSpacePages happy path: a SINGLE /pages/tree request.
// ----------------------------------------------------------------------------- // -----------------------------------------------------------------------------
test("enumerateSpacePages (via list_pages tree) uses one /pages/tree request", async () => { test("enumerateSpacePages (via listPages tree) uses one /pages/tree request", async () => {
let treeRequests = 0; let treeRequests = 0;
let sidebarRequests = 0; let sidebarRequests = 0;
let treeBody = null; let treeBody = null;
@@ -184,7 +184,7 @@ test("enumerateSpacePages (via list_pages tree) uses one /pages/tree request", a
}); });
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
// list_pages tree:true -> enumerateSpacePages(spaceId) -> buildPageTree. // listPages tree:true -> enumerateSpacePages(spaceId) -> buildPageTree.
const tree = await client.listPages("space-1", 50, true); const tree = await client.listPages("space-1", 50, true);
assert.equal(treeRequests, 1, "exactly one /pages/tree request for the space"); assert.equal(treeRequests, 1, "exactly one /pages/tree request for the space");
@@ -375,7 +375,7 @@ test("listComments terminates (no dups) when the server ignores the cursor", asy
}); });
// ----------------------------------------------------------------------------- // -----------------------------------------------------------------------------
// 4) check_new_comments subtree: the root is included in scope WITHOUT a // 4) checkNewComments subtree: the root is included in scope WITHOUT a
// separate getPageRaw (/pages/info) request for the parent. // separate getPageRaw (/pages/info) request for the parent.
// ----------------------------------------------------------------------------- // -----------------------------------------------------------------------------
test("checkNewComments subtree includes the root without a separate getPageRaw", async () => { test("checkNewComments subtree includes the root without a separate getPageRaw", async () => {
@@ -244,7 +244,7 @@ test("tableInsertRow with a slugId opens the collab doc by the resolved UUID (#2
); );
}); });
test("the generic mutate (insert_footnote) with a slugId opens by the resolved UUID (#260)", async () => { test("the generic mutate (insertFootnote) with a slugId opens by the resolved UUID (#260)", async () => {
const { state, baseURL } = await spawnCollabStack(); const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
@@ -254,7 +254,7 @@ test("the generic mutate (insert_footnote) with a slugId opens by the resolved U
assert.deepEqual( assert.deepEqual(
state.docNames, state.docNames,
[`page.${UUID}`], [`page.${UUID}`],
"insert_footnote (via the mutatePage seam) must open the collab doc by UUID", "insertFootnote (via the mutatePage seam) must open the collab doc by UUID",
); );
}); });
@@ -372,7 +372,11 @@ test("replaceImage opens by the resolved UUID AND keys its page lock by that UUI
// single flush. This proves the flush actually executes queued callbacks, so // single flush. This proves the flush actually executes queued callbacks, so
// probeRan === false above means "blocked", not "the flush never ran anyone". // probeRan === false above means "blocked", not "the flush never ran anyone".
let freeRan = false; let freeRan = false;
const freeDone = withPageLock(`page.free-${UUID}`, async () => { // A DIFFERENT canonical UUID (unrelated to the page under test). withPageLock
// now asserts its key is a canonical UUID (#449), so the "free" probe key must
// also be a valid — but distinct — UUID, not a synthetic label.
const FREE_UUID = "99999999-9999-4999-8999-999999999999";
const freeDone = withPageLock(FREE_UUID, async () => {
freeRan = true; freeRan = true;
}); });
await new Promise((r) => setImmediate(r)); await new Promise((r) => setImmediate(r));
@@ -1,4 +1,4 @@
// Server round-trip test for the stash_page MCP tool result shape. The in-app // Server round-trip test for the stashPage MCP tool result shape. The in-app
// path returns the full documented `{ uri, size, sha256, images }` object, but // path returns the full documented `{ uri, size, sha256, images }` object, but
// the MCP transport must deliver the SAME shape: a resource_link (primary // the MCP transport must deliver the SAME shape: a resource_link (primary
// payload) PLUS a `structuredContent` mirror carrying sha256 + image counts. // payload) PLUS a `structuredContent` mirror carrying sha256 + image counts.
@@ -107,7 +107,7 @@ async function buildBaseURL() {
}); });
} }
test("stash_page MCP tool returns a resource_link AND a structuredContent mirror", async () => { test("stashPage MCP tool returns a resource_link AND a structuredContent mirror", async () => {
const baseURL = await buildBaseURL(); const baseURL = await buildBaseURL();
const sandbox = makeSandbox(); const sandbox = makeSandbox();
const server = createDocmostMcpServer({ const server = createDocmostMcpServer({
@@ -124,7 +124,7 @@ test("stash_page MCP tool returns a resource_link AND a structuredContent mirror
try { try {
const res = await client.callTool({ const res = await client.callTool({
name: "stash_page", name: "stashPage",
arguments: { pageId: "page-1" }, arguments: { pageId: "page-1" },
}); });
@@ -20,11 +20,11 @@ import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
import { createDocmostMcpServer } from "../../build/index.js"; import { createDocmostMcpServer } from "../../build/index.js";
// The tool we drive. get_workspace has NO input schema, so protocol-level input // The tool we drive. getWorkspace has NO input schema, so protocol-level input
// validation cannot short-circuit before the handler runs — the wrapped handler // validation cannot short-circuit before the handler runs — the wrapped handler
// is guaranteed to execute (and then fail on the unreachable backend, which is // is guaranteed to execute (and then fail on the unreachable backend, which is
// exactly what we want: the wrapper times in a finally on throw too). // exactly what we want: the wrapper times in a finally on throw too).
const TOOL_NAME = "get_workspace"; const TOOL_NAME = "getWorkspace";
test("the factory's registerTool monkeypatch times a live tool call and labels it with the registration name", async () => { test("the factory's registerTool monkeypatch times a live tool call and labels it with the registration name", async () => {
const calls = []; const calls = [];
@@ -323,7 +323,9 @@ test("MCP_COLLAB_SESSION_IDLE_MS=0 disables the cache (legacy provider-per-op)",
}); });
test("replaceImage-shaped flow: acquire under an EXTERNAL page lock does not deadlock and reuses one session", async () => { test("replaceImage-shaped flow: acquire under an EXTERNAL page lock does not deadlock and reuses one session", async () => {
const pageId = "page-lock"; // withPageLock now asserts a canonical UUID key (#449); this flow takes the
// real page lock (mirroring replaceImage), so the key must be a valid UUID.
const pageId = "77777777-7777-4777-8777-777777777777";
// Mirror replaceImage: hold ONE withPageLock across scan (read-only) + write, // Mirror replaceImage: hold ONE withPageLock across scan (read-only) + write,
// each going through the non-locking acquireCollabSession. // each going through the non-locking acquireCollabSession.
const result = await withPageLock(pageId, async () => { const result = await withPageLock(pageId, async () => {
@@ -152,7 +152,7 @@ test("tautological comment tools are excluded and never probe", async () => {
const { comments, probeCalls, tracker } = makeWorld(); const { comments, probeCalls, tracker } = makeWorld();
tracker.noteWorkingPage("p1"); tracker.noteWorkingPage("p1");
comments.push({ createdAt: 9_999_999 }); comments.push({ createdAt: 9_999_999 });
for (const name of ["listComments", "list_comments", "checkNewComments", "createComment"]) { for (const name of ["listComments", "listComments", "checkNewComments", "createComment"]) {
assert.equal(await tracker.maybeSignal(name), null); assert.equal(await tracker.maybeSignal(name), null);
} }
assert.equal(probeCalls.length, 0); assert.equal(probeCalls.length, 0);
@@ -188,7 +188,7 @@ function fakeTracker({ line }) {
noteWorkingPage: (p) => events.push(["note", p]), noteWorkingPage: (p) => events.push(["note", p]),
advanceWatermark: () => events.push(["advance"]), advanceWatermark: () => events.push(["advance"]),
isExcludedTool: (n) => isExcludedTool: (n) =>
new Set(["listComments", "list_comments"]).has(n), new Set(["listComments", "listComments"]).has(n),
maybeSignal: async () => line, maybeSignal: async () => line,
}; };
} }
@@ -221,7 +221,7 @@ test("withCommentSignal: appends ONE extra text element when signalled", async (
test("withCommentSignal: excluded tool advances the watermark and does not append", async () => { test("withCommentSignal: excluded tool advances the watermark and does not append", async () => {
const tracker = fakeTracker({ line: "SHOULD-NOT-APPEAR" }); const tracker = fakeTracker({ line: "SHOULD-NOT-APPEAR" });
const original = { content: [{ type: "text", text: "comments" }] }; const original = { content: [{ type: "text", text: "comments" }] };
const wrapped = withCommentSignal("list_comments", async () => original, tracker); const wrapped = withCommentSignal("listComments", async () => original, tracker);
const result = await wrapped({ pageId: "p1" }); const result = await wrapped({ pageId: "p1" });
assert.equal(result, original); // unchanged assert.equal(result, original); // unchanged
assert.ok(tracker.events.some((e) => e[0] === "advance")); assert.ok(tracker.events.some((e) => e[0] === "advance"));
+1 -1
View File
@@ -114,7 +114,7 @@ test("summarizeChange treats a key-order-only difference as no change", () => {
// (v) CRITICAL: a structural change that touches no text/marks — adding an // (v) CRITICAL: a structural change that touches no text/marks — adding an
// image node (images 0 -> 1) — must report changed:true and surface the // image node (images 0 -> 1) — must report changed:true and surface the
// integrity delta in structure + summary, closing the verify blind spot for // integrity delta in structure + summary, closing the verify blind spot for
// insert_image / delete_node on structural nodes. // insertImage / deleteNode on structural nodes.
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
test("summarizeChange surfaces an image-count change (0->1)", () => { test("summarizeChange surfaces an image-count change (0->1)", () => {
const before = doc(para(t("caption"))); const before = doc(para(t("caption")));
@@ -0,0 +1,128 @@
// Unit tests for the drawioEditCells operations (issue #425, acceptance #4):
// add / update / delete applied to the parsed model, with a cascade delete that
// removes container children AND every connected edge.
import { test } from "node:test";
import assert from "node:assert/strict";
import { applyCellOps, CellOpsError } from "../../build/lib/drawio-cell-ops.js";
import { parseCells } from "../../build/lib/drawio-xml.js";
const MODEL =
"<mxGraphModel><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/>" +
'<mxCell id="grp" value="G" style="container=1;fillColor=none;" vertex="1" parent="1">' +
'<mxGeometry x="0" y="0" width="300" height="200" as="geometry"/></mxCell>' +
'<mxCell id="c1" value="Child1" style="rounded=1;" vertex="1" parent="grp">' +
'<mxGeometry x="10" y="10" width="80" height="40" as="geometry"/></mxCell>' +
'<mxCell id="c2" value="Child2" style="rounded=1;" vertex="1" parent="grp">' +
'<mxGeometry x="10" y="80" width="80" height="40" as="geometry"/></mxCell>' +
'<mxCell id="out" value="Outside" style="rounded=1;" vertex="1" parent="1">' +
'<mxGeometry x="400" y="10" width="80" height="40" as="geometry"/></mxCell>' +
'<mxCell id="e1" style="" edge="1" parent="1" source="c1" target="out">' +
'<mxGeometry relative="1" as="geometry"/></mxCell>' +
'<mxCell id="e2" style="" edge="1" parent="1" source="out" target="c2">' +
'<mxGeometry relative="1" as="geometry"/></mxCell>' +
"</root></mxGraphModel>";
const ids = (xml) =>
parseCells(xml)
.filter((c) => c.id !== "0" && c.id !== "1")
.map((c) => c.id)
.sort();
test("update changes ONLY the targeted cell", () => {
const out = applyCellOps(MODEL, [
{
op: "update",
cellId: "c1",
xml:
'<mxCell id="c1" value="Renamed" style="rounded=1;" vertex="1" parent="grp">' +
'<mxGeometry x="10" y="10" width="80" height="40" as="geometry"/></mxCell>',
},
]);
const cells = parseCells(out);
assert.equal(cells.find((c) => c.id === "c1").value, "Renamed");
// Every OTHER cell is untouched.
assert.equal(cells.find((c) => c.id === "c2").value, "Child2");
assert.equal(cells.find((c) => c.id === "out").value, "Outside");
assert.deepEqual(ids(out), ids(MODEL));
});
test("delete of a container removes its children AND the connected edges", () => {
const out = applyCellOps(MODEL, [{ op: "delete", cellId: "grp" }]);
// grp + c1 + c2 gone (cascade to children); e1 (c1->out) and e2 (out->c2)
// gone (cascade to connected edges); "out" survives.
assert.deepEqual(ids(out), ["out"]);
});
test("delete of a leaf only cascades to its connected edges, not siblings", () => {
const out = applyCellOps(MODEL, [{ op: "delete", cellId: "c1" }]);
// c1 gone + e1 (c1->out) gone; c2, out, grp, e2 survive.
assert.deepEqual(ids(out), ["c2", "e2", "grp", "out"]);
});
test("add appends a new cell", () => {
const out = applyCellOps(MODEL, [
{
op: "add",
xml:
'<mxCell id="new1" value="N" style="rounded=1;" vertex="1" parent="1">' +
'<mxGeometry x="500" y="10" width="80" height="40" as="geometry"/></mxCell>',
},
]);
assert.ok(parseCells(out).some((c) => c.id === "new1"));
});
test("delete never removes the sentinels", () => {
const out = applyCellOps(MODEL, [{ op: "delete", cellId: "out" }]);
const cells = parseCells(out);
assert.ok(cells.some((c) => c.id === "0"));
assert.ok(cells.some((c) => c.id === "1"));
});
test("errors: unknown update/delete target, duplicate add id, id mismatch", () => {
assert.throws(
() => applyCellOps(MODEL, [{ op: "update", cellId: "ghost", xml: '<mxCell id="ghost"/>' }]),
/does not exist/,
);
assert.throws(
() => applyCellOps(MODEL, [{ op: "delete", cellId: "ghost" }]),
/does not exist/,
);
assert.throws(
() => applyCellOps(MODEL, [{ op: "add", xml: '<mxCell id="c1"/>' }]),
/already exists/,
);
assert.throws(
() =>
applyCellOps(MODEL, [
{ op: "update", cellId: "c1", xml: '<mxCell id="c2"/>' },
]),
/ids are stable/,
);
assert.throws(() => applyCellOps(MODEL, []), CellOpsError);
});
test("an add op with two cells or a missing id is rejected", () => {
assert.throws(
() => applyCellOps(MODEL, [{ op: "add", xml: '<mxCell id="a"/><mxCell id="b"/>' }]),
/exactly one <mxCell>/,
);
assert.throws(
() => applyCellOps(MODEL, [{ op: "add", xml: '<mxCell value="x"/>' }]),
/missing an id/,
);
});
// --- SUGGESTION #5: sentinel cells are protected from delete ------------------
test("delete targeting a sentinel id is rejected (no wipe of the diagram body)", () => {
for (const sid of ["0", "1"]) {
assert.throws(
() => applyCellOps(MODEL, [{ op: "delete", cellId: sid }]),
(e) => e instanceof CellOpsError && /cannot delete sentinel cell/.test(e.message),
);
}
// A normal delete still works and the sentinels remain intact.
const out = applyCellOps(MODEL, [{ op: "delete", cellId: "out" }]);
const remaining = parseCells(out).map((c) => c.id);
assert.ok(remaining.includes("0") && remaining.includes("1"), "sentinels survive");
});
@@ -0,0 +1,380 @@
// Unit tests for the drawioFromGraph pipeline (issue #425, stage 3): the
// semantic graph -> ELK -> linter-clean XML assembler, plus the layout hints
// (pinned / sameLayerAs / layer) and incremental layout. Pure — no client, no
// network — so they run under `node --test` against the built lib.
import { test } from "node:test";
import assert from "node:assert/strict";
import {
buildFromGraph,
validateGraph,
resolveNodeStyle,
GraphValidationError,
} from "../../build/lib/drawio-graph.js";
import { getPreset } from "../../build/lib/drawio-presets.js";
import {
prepareModel,
parseCells,
computeQualityWarnings,
} from "../../build/lib/drawio-xml.js";
function cellsOf(xml) {
return parseCells(xml).filter((c) => c.id !== "0" && c.id !== "1");
}
function byId(xml) {
const m = new Map();
for (const c of parseCells(xml)) m.set(c.id, c);
return m;
}
// --- Acceptance #1: 15-node graph, 2 nested groups, AWS icons ---------------
test("from_graph: 15+ nodes, 2 nested groups, AWS icons -> 0 lint errors, 0 warnings, all icons resolve", async () => {
const awsIcons = [
"aws:lambda", "aws:dynamodb", "aws:api_gateway", "aws:s3", "aws:sqs",
"aws:sns", "aws:ec2", "aws:rds", "aws:cloudfront", "aws:elasticache",
"aws:kinesis", "aws:cognito", "aws:secrets_manager", "aws:cloudwatch",
];
const kinds = [
"service", "db", "gateway", "service", "queue", "queue", "service", "db",
"gateway", "db", "queue", "security", "security", "external",
];
const nodes = [];
for (let i = 0; i < 14; i++) {
nodes.push({
id: "n" + i,
label: "Node " + i,
kind: kinds[i],
icon: awsIcons[i],
group: i < 6 ? "sub1" : i < 10 ? "vpc1" : undefined,
});
}
nodes.push({ id: "ext1", label: "External Service", kind: "external" });
const graph = {
nodes,
groups: [
{ id: "vpc1", label: "VPC 10.0.0.0/16", kind: "vpc" },
{ id: "sub1", label: "Private Subnet", kind: "subnet", group: "vpc1" }, // NESTED
],
edges: [
{ from: "n0", to: "n1", kind: "sync" },
{ from: "n1", to: "n2", kind: "async" },
{ from: "n2", to: "n3" },
{ from: "n3", to: "n7", kind: "sync" },
{ from: "n7", to: "n8" },
{ from: "n8", to: "ext1", kind: "error" },
{ from: "n4", to: "n5" },
{ from: "n10", to: "n11" },
],
direction: "LR",
preset: "default",
};
const r = await buildFromGraph(graph, "full");
assert.equal(graph.nodes.length >= 15, true, "at least 15 nodes");
// All icons resolved — NO empty squares.
assert.equal(r.iconsMissing.length, 0, `unresolved icons: ${r.iconsMissing}`);
assert.equal(r.iconsResolved, 14);
// 0 lint errors + 0 quality-warnings.
const prepared = prepareModel(r.modelXml);
assert.equal(prepared.warnings.length, 0, prepared.warnings.join("\n"));
// Groups are TRANSPARENT containers.
const cells = byId(r.modelXml);
for (const gid of ["vpc1", "sub1"]) {
const g = cells.get(gid);
assert.ok(g, `${gid} present`);
assert.equal(g.styleMap.container, "1", `${gid} container=1`);
assert.equal(g.styleMap.fillColor, "none", `${gid} fillColor=none`);
assert.equal(g.styleMap.dropTarget, "1", `${gid} dropTarget=1`);
}
// The nested group sub1's parent IS vpc1 (nesting honoured).
assert.equal(cells.get("sub1").parent, "vpc1");
// A grouped node's parent is its group (relative coords).
assert.equal(cells.get("n0").parent, "sub1");
});
test("from_graph: an UNKNOWN icon degrades to a labelled generic shape (never empty)", async () => {
const graph = {
nodes: [
{ id: "a", label: "Mystery", kind: "service", icon: "not:a-real-icon-xyz" },
{ id: "b", label: "Plain", kind: "db" },
],
edges: [{ from: "a", to: "b" }],
};
const r = await buildFromGraph(graph, "full");
const cells = byId(r.modelXml);
// The node still carries its label and a real (non-empty) style with a fill.
assert.equal(cells.get("a").value, "Mystery");
assert.match(cells.get("a").style, /fillColor=/);
// It is reported as missing so the model can see the degradation.
assert.ok(r.iconsMissing.includes("a"));
});
// --- Acceptance #2: hints -----------------------------------------------------
test("from_graph: a pinned node stays at its exact coordinates", async () => {
const graph = {
nodes: [
{ id: "a", label: "A", pinned: { x: 40, y: 900 } },
{ id: "b", label: "B" },
{ id: "c", label: "C" },
],
edges: [{ from: "a", to: "b" }, { from: "b", to: "c" }],
};
const a = byId((await buildFromGraph(graph, "full")).modelXml).get("a");
assert.equal(a.geometry.x, 40);
assert.equal(a.geometry.y, 900);
});
test("from_graph: a sameLayerAs pair lands in the same layer (equal layer-axis coord)", async () => {
const graph = {
nodes: [
{ id: "x", label: "X" },
{ id: "y", label: "Y", sameLayerAs: "x" },
{ id: "z", label: "Z" },
],
edges: [{ from: "z", to: "x" }, { from: "z", to: "y" }],
direction: "LR", // layer axis = x
};
const cells = byId((await buildFromGraph(graph, "full")).modelXml);
assert.equal(cells.get("x").geometry.x, cells.get("y").geometry.x);
});
test("from_graph: an explicit layer index co-aligns nodes on the layer axis", async () => {
const graph = {
nodes: [
{ id: "p", label: "P", layer: 0 },
{ id: "q", label: "Q", layer: 0 },
{ id: "r", label: "R", layer: 1 },
],
edges: [{ from: "p", to: "r" }],
direction: "LR",
};
const cells = byId((await buildFromGraph(graph, "full")).modelXml);
assert.equal(cells.get("p").geometry.x, cells.get("q").geometry.x);
});
test("from_graph: TB direction snaps sameLayerAs on the Y axis", async () => {
const graph = {
nodes: [
{ id: "x", label: "X" },
{ id: "y", label: "Y", sameLayerAs: "x" },
{ id: "z", label: "Z" },
],
edges: [{ from: "z", to: "x" }, { from: "z", to: "y" }],
direction: "TB", // layer axis = y
};
const cells = byId((await buildFromGraph(graph, "full")).modelXml);
assert.equal(cells.get("x").geometry.y, cells.get("y").geometry.y);
});
// --- Acceptance #3: incremental never moves existing cells --------------------
test("from_graph incremental: adding a node does NOT move existing cells", async () => {
const existing = new Map([
["a", { x: 100, y: 100 }],
["b", { x: 400, y: 100 }],
]);
const graph = {
nodes: [
{ id: "a", label: "A" },
{ id: "b", label: "B" },
{ id: "cnew", label: "New" },
],
edges: [{ from: "a", to: "b" }, { from: "b", to: "cnew" }],
};
const cells = byId((await buildFromGraph(graph, "incremental", existing)).modelXml);
assert.deepEqual(
[cells.get("a").geometry.x, cells.get("a").geometry.y],
[100, 100],
);
assert.deepEqual(
[cells.get("b").geometry.x, cells.get("b").geometry.y],
[400, 100],
);
// The new node was placed and does not overlap the frozen block.
const cn = cells.get("cnew");
assert.ok(cn.geometry.y >= 200, "new node placed clear of the existing block");
});
// --- direction honoured -------------------------------------------------------
test("from_graph: LR vs RL flip the layout axis order", async () => {
const mk = (dir) => ({
nodes: [{ id: "s", label: "S" }, { id: "t", label: "T" }],
edges: [{ from: "s", to: "t" }],
direction: dir,
});
const lr = byId((await buildFromGraph(mk("LR"), "full")).modelXml);
// In LR the target sits to the RIGHT of the source.
assert.ok(lr.get("t").geometry.x > lr.get("s").geometry.x, "LR: t right of s");
const rl = byId((await buildFromGraph(mk("RL"), "full")).modelXml);
assert.ok(rl.get("t").geometry.x < rl.get("s").geometry.x, "RL: t left of s");
});
// --- validation ---------------------------------------------------------------
test("validateGraph: rejects duplicate node ids, unknown group/edge refs", () => {
assert.throws(
() => validateGraph({ nodes: [{ id: "a", label: "A" }, { id: "a", label: "B" }] }),
GraphValidationError,
);
assert.throws(
() => validateGraph({ nodes: [{ id: "a", label: "A", group: "ghost" }] }),
/unknown group/,
);
assert.throws(
() =>
validateGraph({
nodes: [{ id: "a", label: "A" }],
edges: [{ from: "a", to: "ghost" }],
}),
/resolves to no node/,
);
assert.throws(() => validateGraph({ nodes: [] }), /non-empty/);
});
test("resolveNodeStyle: kind maps to the preset palette slot for a generic node", () => {
const preset = getPreset("default");
const s = resolveNodeStyle(preset, { id: "d", label: "DB", kind: "db" });
assert.equal(s.iconResolved, false);
assert.match(s.style, /fillColor=#d5e8d4;strokeColor=#82b366/);
});
// --- the assembled XML is always linter-clean --------------------------------
test("from_graph: a plain cross-container-edge graph is linter-clean", async () => {
const graph = {
nodes: [
{ id: "a", label: "A", group: "g1" },
{ id: "b", label: "B", group: "g2" },
],
groups: [
{ id: "g1", label: "G1" },
{ id: "g2", label: "G2" },
],
edges: [{ from: "a", to: "b", label: "x" }],
};
const r = await buildFromGraph(graph, "full");
const prepared = prepareModel(r.modelXml); // throws on any lint error
assert.equal(prepared.warnings.length, 0, prepared.warnings.join("\n"));
// A cross-container edge is parented at the layer sentinel "1".
const edge = cellsOf(r.modelXml).find((c) => c.edge);
assert.equal(edge.parent, "1");
});
// --- CRITICAL #1: edge / group caps reject FAST (no layout, no OOM) -----------
test("validateGraph: an over-limit EDGE count is rejected before any layout", () => {
// 2 nodes, 200000 edges: passes node validation, would OOM graphToElk/runElk.
const edges = [];
for (let i = 0; i < 200_000; i++) edges.push({ from: "a", to: "b" });
const graph = { nodes: [{ id: "a", label: "A" }, { id: "b", label: "B" }], edges };
const t0 = Date.now();
assert.throws(
() => validateGraph(graph),
(e) => e instanceof GraphValidationError && /200000 edges .*max 1000/.test(e.message),
);
assert.ok(Date.now() - t0 < 1000, "must reject in well under a second (no layout)");
});
test("validateGraph: an over-limit GROUP count is rejected fast", () => {
const groups = [];
for (let i = 0; i < 600; i++) groups.push({ id: "g" + i, label: "G" });
const t0 = Date.now();
assert.throws(
() => validateGraph({ nodes: [{ id: "a", label: "A" }], groups }),
(e) => e instanceof GraphValidationError && /600 groups .*max 500/.test(e.message),
);
assert.ok(Date.now() - t0 < 1000);
});
test("validateGraph: exactly-at-cap edges/groups are accepted", () => {
const edges = [];
for (let i = 0; i < 1000; i++) edges.push({ from: "a", to: "b" });
assert.doesNotThrow(() =>
validateGraph({ nodes: [{ id: "a", label: "A" }, { id: "b", label: "B" }], edges }),
);
});
// --- WARNING #3: sameLayerAs spread -> 0 quality-warnings by construction -----
test("from_graph: a sameLayerAs chain of 5 yields 0 quality-warnings (cross-axis spread)", async () => {
const graph = {
nodes: [
{ id: "a", label: "A" },
{ id: "b", label: "B", sameLayerAs: "a" },
{ id: "c", label: "C", sameLayerAs: "b" },
{ id: "d", label: "D", sameLayerAs: "c" },
{ id: "e", label: "E", sameLayerAs: "d" },
],
edges: [{ from: "a", to: "b" }],
direction: "LR",
};
const r = await buildFromGraph(graph, "full");
const cells = parseCells(r.modelXml);
const warnings = computeQualityWarnings(cells);
assert.equal(warnings.length, 0, warnings.join("\n"));
// The four chained dependents share one layer-axis (x) coordinate...
const by = new Map(cells.map((c) => [c.id, c]));
const xs = ["b", "c", "d", "e"].map((id) => by.get(id).geometry.x);
assert.equal(new Set(xs).size, 1, "chained nodes must share the layer axis");
// ...but are spread on the cross axis (y) with distinct coordinates.
const ys = ["b", "c", "d", "e"].map((id) => by.get(id).geometry.y);
assert.equal(new Set(ys).size, 4, "chained nodes must not stack on the cross axis");
});
test("from_graph: pinned coords are honored verbatim; a negative pin is clamped non-negative", async () => {
const graph = {
nodes: [
{ id: "p1", label: "P1", pinned: { x: 100, y: 100 } },
// Two user-pinned nodes at (nearly) the same point: user intent, honored.
{ id: "p2", label: "P2", pinned: { x: -500, y: 100 } }, // out-of-bounds x -> clamped to 0
],
direction: "LR",
};
const r = await buildFromGraph(graph, "full");
const by = new Map(parseCells(r.modelXml).map((c) => [c.id, c]));
assert.equal(by.get("p1").geometry.x, 100);
assert.equal(by.get("p1").geometry.y, 100);
assert.equal(by.get("p2").geometry.x, 0, "negative pin x clamped to 0");
assert.equal(by.get("p2").geometry.y, 100, "pin y honored");
// A pinned overlap MAY warn — that's user-directed and documented; we only
// assert the coords are honored (the guarantee softening), not warning count.
});
// --- WARNING #4: incremental MERGE preserves unlisted existing cells ----------
test("from_graph incremental: an existing cell not in the new graph SURVIVES the add", async () => {
const existingModelXml =
'<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/>' +
'<mxCell id="manual" value="Hand Placed" style="rounded=1;" vertex="1" parent="1">' +
'<mxGeometry x="900" y="900" width="120" height="60" as="geometry"/></mxCell>' +
'<mxCell id="old1" value="Old One" style="rounded=1;" vertex="1" parent="1">' +
'<mxGeometry x="40" y="40" width="120" height="60" as="geometry"/></mxCell>' +
"</root></mxGraphModel>";
const existingCoords = new Map([
["manual", { x: 900, y: 900 }],
["old1", { x: 40, y: 40 }],
]);
// The model sends ONLY the re-listed old1 + the new node — NOT "manual".
const graph = {
nodes: [
{ id: "old1", label: "Old One" },
{ id: "new1", label: "Added Node" },
],
edges: [{ from: "old1", to: "new1" }],
direction: "LR",
};
const r = await buildFromGraph(graph, "incremental", existingCoords, existingModelXml);
const by = new Map(parseCells(r.modelXml).map((c) => [c.id, c]));
// The unlisted hand-placed cell survives, verbatim coords.
assert.ok(by.has("manual"), "unlisted existing cell must not be dropped");
assert.equal(by.get("manual").geometry.x, 900);
assert.equal(by.get("manual").geometry.y, 900);
// The re-listed existing cell keeps its frozen coords; the new node is added.
assert.equal(by.get("old1").geometry.x, 40);
assert.equal(by.get("old1").geometry.y, 40);
assert.ok(by.has("new1"), "the newly added node is present");
});
+1 -1
View File
@@ -1,4 +1,4 @@
// Unit tests for the drawio_guide progressive-disclosure reference (issue #424). // Unit tests for the drawioGuide progressive-disclosure reference (issue #424).
// Acceptance #2: every section is returned and each is <= ~4KB so pulling one // Acceptance #2: every section is returned and each is <= ~4KB so pulling one
// does not bloat the model's context. // does not bloat the model's context.
import { test } from "node:test"; import { test } from "node:test";
@@ -0,0 +1,114 @@
// Unit tests for the Mermaid flowchart -> graph parser (issue #425, acceptance
// #6, OPTIONAL). Verifies a flowchart with a branch + a subgraph parses to a
// graph the from_graph pipeline renders as valid, editable drawio, and that a
// non-flowchart diagram is rejected with a clear error.
import { test } from "node:test";
import assert from "node:assert/strict";
import { mermaidToGraph, MermaidParseError } from "../../build/lib/drawio-mermaid.js";
import { buildFromGraph } from "../../build/lib/drawio-graph.js";
import { prepareModel } from "../../build/lib/drawio-xml.js";
test("flowchart with a branch + subgraph -> a valid, editable drawio", async () => {
const mm = `flowchart LR
A[Start] --> B{Decision}
B -->|yes| C[(Database)]
B -->|no| D[End]
subgraph backend [Backend Services]
C
E([Cache])
end
D -.-> E`;
const graph = mermaidToGraph(mm);
// Direction + nodes + a group + labelled/dashed edges parsed.
assert.equal(graph.direction, "LR");
const nodeIds = graph.nodes.map((n) => n.id).sort();
assert.deepEqual(nodeIds, ["A", "B", "C", "D", "E"]);
// The decision `{}` maps to the queue palette; the `[(db)]` to db.
assert.equal(graph.nodes.find((n) => n.id === "B").kind, "queue");
assert.equal(graph.nodes.find((n) => n.id === "C").kind, "db");
// The subgraph became a group and claimed its members.
assert.equal(graph.groups.length, 1);
assert.equal(graph.groups[0].id, "backend");
assert.equal(graph.nodes.find((n) => n.id === "C").group, "backend");
assert.equal(graph.nodes.find((n) => n.id === "E").group, "backend");
// A pipe label and a dotted (async) edge.
const yes = graph.edges.find((e) => e.from === "B" && e.to === "C");
assert.equal(yes.label, "yes");
const dotted = graph.edges.find((e) => e.from === "D" && e.to === "E");
assert.equal(dotted.kind, "async");
// The whole thing renders linter-clean.
const built = await buildFromGraph(graph, "full");
const prepared = prepareModel(built.modelXml);
assert.equal(prepared.warnings.length, 0, prepared.warnings.join("\n"));
});
test("graph TD header sets a top-down direction", () => {
const g = mermaidToGraph("graph TD\n X --> Y");
assert.equal(g.direction, "TB");
assert.deepEqual(g.nodes.map((n) => n.id).sort(), ["X", "Y"]);
});
test("a chained connection A --> B --> C yields two edges", () => {
const g = mermaidToGraph("flowchart LR\n A[a] --> B[b] --> C[c]");
const pairs = g.edges.map((e) => `${e.from}->${e.to}`).sort();
assert.deepEqual(pairs, ["A->B", "B->C"]);
});
test("a non-flowchart diagram is rejected with a clear error", () => {
assert.throws(
() => mermaidToGraph("sequenceDiagram\n Alice->>Bob: Hi"),
/only 'flowchart'\/'graph' is supported/,
);
assert.throws(() => mermaidToGraph(""), MermaidParseError);
});
// --- CRITICAL #2 / NIT: input-size bounds reject FAST (no OOM) ----------------
test("mermaidToGraph: an over-length input is rejected before parsing (fast)", () => {
const huge = "flowchart LR\n" + "A-->B\n".repeat(60_000); // ~360 KB > 200 KB cap
const t0 = Date.now();
assert.throws(
() => mermaidToGraph(huge),
(e) => e instanceof MermaidParseError && /max 200000/.test(e.message),
);
assert.ok(Date.now() - t0 < 1000, "must reject in well under a second (no parse)");
});
test("mermaidToGraph: an over-line-count input is rejected fast", () => {
const many = "flowchart LR\n" + "A\n".repeat(25_000); // > 20000 line cap
const t0 = Date.now();
assert.throws(
() => mermaidToGraph(many),
(e) => e instanceof MermaidParseError && /max 20000/.test(e.message),
);
assert.ok(Date.now() - t0 < 1000);
});
test("mermaidToGraph: too many subgraphs is rejected", () => {
let src = "flowchart LR\n";
for (let i = 0; i < 600; i++) src += `subgraph s${i}\nend\n`;
assert.throws(
() => mermaidToGraph(src),
(e) => e instanceof MermaidParseError && /too many subgraphs .*max 500/.test(e.message),
);
});
test("mermaidToGraph: an over-long connection chain throws (NON-silent truncation)", () => {
const chain =
"flowchart LR\n" +
Array.from({ length: 600 }, (_, i) => "N" + i).join("-->");
assert.throws(
() => mermaidToGraph(chain),
(e) => e instanceof MermaidParseError && /chain exceeds 500 nodes/.test(e.message),
);
});
test("mermaidToGraph: a chain of 60 nodes parses (no silent 50-node truncation)", () => {
const chain =
"flowchart LR\n" +
Array.from({ length: 60 }, (_, i) => "N" + i).join("-->");
const g = mermaidToGraph(chain);
assert.equal(g.nodes.length, 60, "all 60 chained nodes are kept");
});
@@ -0,0 +1,117 @@
// Snapshot + invariant tests for the semantic presets (issue #425, acceptance
// #5): every node kind has a style-string per preset, and colorblind-safe uses
// only the Okabe-Ito palette (no problematic color pairs).
import { test } from "node:test";
import assert from "node:assert/strict";
import {
getPreset,
genericNodeStyle,
edgeStyle,
groupStyle,
NODE_KINDS,
EDGE_KINDS,
PRESET_NAMES,
} from "../../build/lib/drawio-presets.js";
// The Okabe-Ito qualitative palette (8 colours distinguishable under the common
// colour-vision deficiencies). colorblind-safe MUST draw its strokes from here.
const OKABE_ITO = [
"#000000", "#e69f00", "#56b4e9", "#009e73",
"#f0e442", "#0072b2", "#d55e00", "#cc79a7",
].map((s) => s.toLowerCase());
// The exact base-palette fills from the issue's table (a snapshot: a change to
// the default palette is a deliberate, reviewed edit — this catches accidents).
const DEFAULT_FILLS = {
service: "#dae8fc",
db: "#d5e8d4",
queue: "#fff2cc",
gateway: "#ffe6cc",
error: "#f8cecc",
external: "#f5f5f5",
security: "#e1d5e7",
};
const DEFAULT_STROKES = {
service: "#6c8ebf",
db: "#82b366",
queue: "#d6b656",
gateway: "#d79b00",
error: "#b85450",
external: "#666666",
security: "#9673a6",
};
test("every node kind has a style-string in every preset", () => {
for (const p of PRESET_NAMES) {
const preset = getPreset(p);
for (const kind of NODE_KINDS) {
const style = genericNodeStyle(preset, kind);
assert.match(style, /fillColor=#[0-9a-fA-F]{6}/, `${p}/${kind} has a fill`);
assert.match(style, /strokeColor=#[0-9a-fA-F]{6}/, `${p}/${kind} has a stroke`);
assert.match(style, /fontColor=#[0-9a-fA-F]{6}/, `${p}/${kind} has a font`);
}
}
});
test("default preset matches the issue's palette snapshot", () => {
const preset = getPreset("default");
for (const kind of NODE_KINDS) {
const style = genericNodeStyle(preset, kind);
assert.ok(
style.includes(`fillColor=${DEFAULT_FILLS[kind]}`),
`default/${kind} fill ${DEFAULT_FILLS[kind]}`,
);
assert.ok(
style.includes(`strokeColor=${DEFAULT_STROKES[kind]}`),
`default/${kind} stroke ${DEFAULT_STROKES[kind]}`,
);
}
});
test("colorblind-safe strokes come ONLY from the Okabe-Ito palette", () => {
const preset = getPreset("colorblind-safe");
const usedStrokes = new Set();
for (const kind of NODE_KINDS) {
const stroke = preset.nodes[kind].strokeColor.toLowerCase();
assert.ok(
OKABE_ITO.includes(stroke),
`colorblind-safe/${kind} stroke ${stroke} is NOT Okabe-Ito`,
);
usedStrokes.add(stroke);
}
// No two kinds share a stroke (each is a distinct, distinguishable hue) — the
// "no problematic color pairs" acceptance: distinct Okabe-Ito hues.
assert.equal(
usedStrokes.size,
NODE_KINDS.length,
"each kind gets a distinct Okabe-Ito stroke",
);
});
test("edge kinds: sync solid, async dashed, error red-dashed", () => {
const preset = getPreset("default");
const sync = edgeStyle(preset, "sync");
const async_ = edgeStyle(preset, "async");
const error = edgeStyle(preset, "error");
assert.doesNotMatch(sync, /dashed=1/, "sync is solid");
assert.match(async_, /dashed=1/, "async is dashed");
assert.match(error, /dashed=1/, "error is dashed");
assert.match(error, /strokeColor=#DD344C/i, "error is red");
// An unknown edge kind falls back to sync (solid).
assert.doesNotMatch(edgeStyle(preset, "weird"), /dashed=1/);
});
test("group style is always transparent (fillColor=none;container=1;dropTarget=1)", () => {
for (const p of PRESET_NAMES) {
const style = groupStyle(getPreset(p));
assert.match(style, /fillColor=none/, `${p} group transparent`);
assert.match(style, /container=1/, `${p} group is a container`);
assert.match(style, /dropTarget=1/, `${p} group is a drop target`);
}
});
test("EDGE_KINDS / NODE_KINDS constants match the palette", () => {
const preset = getPreset("default");
for (const k of NODE_KINDS) assert.ok(preset.nodes[k], `node kind ${k}`);
for (const k of EDGE_KINDS) assert.ok(preset.edges[k], `edge kind ${k}`);
});
@@ -1,4 +1,4 @@
// Unit tests for the drawio_shapes verified-stencil catalog (issue #424). // Unit tests for the drawioShapes verified-stencil catalog (issue #424).
// Covers acceptance #1: a "lambda" query returns a valid mxgraph.aws4 icon with // Covers acceptance #1: a "lambda" query returns a valid mxgraph.aws4 icon with
// the right service/resource pattern + sizes; a blocklisted stencil query // the right service/resource pattern + sizes; a blocklisted stencil query
// returns its working replacement. // returns its working replacement.
@@ -22,7 +22,7 @@ test("the bundled index loads and is the real ~10k-shape catalog", () => {
} }
}); });
test('drawio_shapes("lambda") returns a valid mxgraph.aws4 service icon', () => { test('drawioShapes("lambda") returns a valid mxgraph.aws4 service icon', () => {
const results = searchShapes("lambda", { limit: 5 }); const results = searchShapes("lambda", { limit: 5 });
assert.ok(results.length > 0); assert.ok(results.length > 0);
// Acceptance #1: a valid aws4 service-level icon (resourceIcon + resIcon) // Acceptance #1: a valid aws4 service-level icon (resourceIcon + resIcon)
@@ -7,16 +7,16 @@ import assert from "node:assert/strict";
import { SERVER_INSTRUCTIONS } from "../../build/index.js"; import { SERVER_INSTRUCTIONS } from "../../build/index.js";
import { SHARED_TOOL_SPECS } from "../../build/tool-specs.js"; import { SHARED_TOOL_SPECS } from "../../build/tool-specs.js";
test("drawio_shapes and drawio_guide are in the shared registry", () => { test("drawioShapes and drawioGuide are in the shared registry", () => {
assert.equal(SHARED_TOOL_SPECS.drawioShapes.mcpName, "drawio_shapes"); assert.equal(SHARED_TOOL_SPECS.drawioShapes.mcpName, "drawioShapes");
assert.equal(SHARED_TOOL_SPECS.drawioGuide.mcpName, "drawio_guide"); assert.equal(SHARED_TOOL_SPECS.drawioGuide.mcpName, "drawioGuide");
// Deferred tier, matching the stage-1 drawio tools. // Deferred tier, matching the stage-1 drawio tools.
assert.equal(SHARED_TOOL_SPECS.drawioShapes.tier, "deferred"); assert.equal(SHARED_TOOL_SPECS.drawioShapes.tier, "deferred");
assert.equal(SHARED_TOOL_SPECS.drawioGuide.tier, "deferred"); assert.equal(SHARED_TOOL_SPECS.drawioGuide.tier, "deferred");
}); });
test("the new tools are routed in SERVER_INSTRUCTIONS", () => { test("the new tools are routed in SERVER_INSTRUCTIONS", () => {
for (const name of ["drawio_shapes", "drawio_guide"]) { for (const name of ["drawioShapes", "drawioGuide"]) {
assert.match(SERVER_INSTRUCTIONS, new RegExp(`\\b${name}\\b`), `${name} missing from guide`); assert.match(SERVER_INSTRUCTIONS, new RegExp(`\\b${name}\\b`), `${name} missing from guide`);
} }
}); });
@@ -26,7 +26,7 @@ test("the hard-rules block is injected into create/update descriptions", () => {
const d = SHARED_TOOL_SPECS[key].description; const d = SHARED_TOOL_SPECS[key].description;
assert.match(d, /sentinels are MANDATORY/); assert.match(d, /sentinels are MANDATORY/);
assert.match(d, /vertex="1" XOR edge="1"/); assert.match(d, /vertex="1" XOR edge="1"/);
assert.match(d, /call drawio_shapes first/); assert.match(d, /call drawioShapes first/);
assert.match(d, /adaptiveColors="auto"/); assert.match(d, /adaptiveColors="auto"/);
assert.match(d, /&#xa;/); assert.match(d, /&#xa;/);
} }
@@ -0,0 +1,85 @@
// Drift guards for the stage-3 drawio tools (issue #425): the three high-level
// tools must be in the shared registry, routed in SERVER_INSTRUCTIONS, expose
// the right schema fields, and carry an `execute` (they call CLIENT methods, so
// unlike drawioShapes/drawioGuide they are NOT inlineBothHosts).
import { test } from "node:test";
import assert from "node:assert/strict";
import { SERVER_INSTRUCTIONS } from "../../build/index.js";
import { SHARED_TOOL_SPECS } from "../../build/tool-specs.js";
const NEW = ["drawioEditCells", "drawioFromGraph", "drawioFromMermaid"];
test("the three stage-3 tools are in the shared registry (deferred, camelCase)", () => {
for (const name of NEW) {
const spec = SHARED_TOOL_SPECS[name];
assert.ok(spec, `${name} missing from registry`);
assert.equal(spec.mcpName, name);
assert.equal(spec.inAppKey, name);
assert.equal(spec.tier, "deferred");
}
});
test("the stage-3 tools carry an execute (client-backed, NOT inlineBothHosts)", () => {
for (const name of NEW) {
const spec = SHARED_TOOL_SPECS[name];
assert.equal(typeof spec.execute, "function", `${name} needs an execute`);
assert.notEqual(spec.inlineBothHosts, true, `${name} must not be inlineBothHosts`);
}
});
test("the stage-3 tools are routed in SERVER_INSTRUCTIONS", () => {
for (const name of NEW) {
assert.match(SERVER_INSTRUCTIONS, new RegExp(`\\b${name}\\b`), `${name} missing from guide`);
}
});
test("drawioFromGraph exposes graph + direction/preset/layout params", () => {
const shape = SHARED_TOOL_SPECS.drawioFromGraph.buildShape(makeZodStub());
for (const key of ["pageId", "graph", "position", "direction", "preset", "layout", "node"]) {
assert.ok(key in shape, `drawioFromGraph missing ${key}`);
}
});
test("drawioEditCells exposes operations + baseHash", () => {
const shape = SHARED_TOOL_SPECS.drawioEditCells.buildShape(makeZodStub());
for (const key of ["pageId", "node", "operations", "baseHash"]) {
assert.ok(key in shape, `drawioEditCells missing ${key}`);
}
});
test("drawioFromMermaid exposes a mermaid param", () => {
const shape = SHARED_TOOL_SPECS.drawioFromMermaid.buildShape(makeZodStub());
assert.ok("mermaid" in shape);
});
test("the hard-rules block is injected into edit_cells (raw <mxCell> ops) but NOT from_graph/from_mermaid", () => {
// edit_cells takes raw <mxCell> xml in add/update ops, so it surfaces the XML
// rules. from_graph/from_mermaid NEVER expose XML to the model (the whole point
// is the model never writes a style/coord), so the hard rules would be noise.
assert.match(SHARED_TOOL_SPECS.drawioEditCells.description, /sentinels are MANDATORY/);
assert.doesNotMatch(SHARED_TOOL_SPECS.drawioFromGraph.description, /sentinels are MANDATORY/);
assert.doesNotMatch(SHARED_TOOL_SPECS.drawioFromMermaid.description, /sentinels are MANDATORY/);
});
test("routing prose distinguishes from_graph (architectures) vs from_mermaid (standard)", () => {
// The EDIT-section routing sentence must mention the semantic tools' intents.
assert.match(SERVER_INSTRUCTIONS, /drawioFromGraph[\s\S]*architecture|architecture[\s\S]*drawioFromGraph/i);
assert.match(SERVER_INSTRUCTIONS, /drawioFromMermaid[\s\S]*flowchart|flowchart[\s\S]*drawioFromMermaid/i);
});
// Tiny zod stub (buildShape only calls string/number/enum/array/object +
// chained min/optional/describe — all return `this`; object() returns a chain
// too so nested schemas resolve).
function makeZodStub() {
const chain = new Proxy(
{},
{ get: (_t, p) => (p === "parse" ? () => ({}) : () => chain) },
);
return {
string: () => chain,
number: () => chain,
enum: () => chain,
array: () => chain,
object: () => chain,
};
}
@@ -258,7 +258,7 @@ test("a non-axios error is passed through untouched", () => {
const GOOD_UUID = "019f499a-9f8c-7d68-b7be-ce100d7c6c56"; const GOOD_UUID = "019f499a-9f8c-7d68-b7be-ce100d7c6c56";
test("assertFullUuid accepts a full canonical UUID (any version nibble)", () => { test("assertFullUuid accepts a full canonical UUID (any version nibble)", () => {
assert.doesNotThrow(() => assertFullUuid("resolve_comment", "commentId", GOOD_UUID)); assert.doesNotThrow(() => assertFullUuid("resolveComment", "commentId", GOOD_UUID));
// A v4 id also passes (version/variant-agnostic). // A v4 id also passes (version/variant-agnostic).
assert.doesNotThrow(() => assert.doesNotThrow(() =>
assertFullUuid("get_comment", "commentId", "3d5b7c1e-2f4a-4b6c-8d9e-0f1a2b3c4d5e"), assertFullUuid("get_comment", "commentId", "3d5b7c1e-2f4a-4b6c-8d9e-0f1a2b3c4d5e"),
@@ -267,10 +267,10 @@ test("assertFullUuid accepts a full canonical UUID (any version nibble)", () =>
test("assertFullUuid rejects a truncated prefix", () => { test("assertFullUuid rejects a truncated prefix", () => {
assert.throws( assert.throws(
() => assertFullUuid("resolve_comment", "commentId", "019f499a"), () => assertFullUuid("resolveComment", "commentId", "019f499a"),
(e) => (e) =>
e.message.startsWith( e.message.startsWith(
"resolve_comment: 'commentId' must be the FULL comment UUID", "resolveComment: 'commentId' must be the FULL comment UUID",
) && ) &&
e.message.includes("got '019f499a'") && e.message.includes("got '019f499a'") &&
e.message.includes("Copy the id verbatim"), e.message.includes("Copy the id verbatim"),
@@ -279,11 +279,11 @@ test("assertFullUuid rejects a truncated prefix", () => {
test("assertFullUuid rejects garbage and empty string", () => { test("assertFullUuid rejects garbage and empty string", () => {
assert.throws( assert.throws(
() => assertFullUuid("delete_comment", "commentId", "not-a-uuid"), () => assertFullUuid("deleteComment", "commentId", "not-a-uuid"),
/must be the FULL comment UUID.*got 'not-a-uuid'/s, /must be the FULL comment UUID.*got 'not-a-uuid'/s,
); );
assert.throws( assert.throws(
() => assertFullUuid("update_comment", "commentId", ""), () => assertFullUuid("updateComment", "commentId", ""),
/must be the FULL comment UUID.*got ''/s, /must be the FULL comment UUID.*got ''/s,
); );
}); });
@@ -436,14 +436,14 @@ test("all 5 comment-id call sites reject a bad id with ZERO network traffic", as
const client = new DocmostClient(baseURL, "u@example.com", "pw"); const client = new DocmostClient(baseURL, "u@example.com", "pw");
const bad = "019f499a"; // truncated const bad = "019f499a"; // truncated
await assert.rejects(() => client.resolveComment(bad, true), /resolve_comment: 'commentId'/); await assert.rejects(() => client.resolveComment(bad, true), /resolveComment: 'commentId'/);
await assert.rejects(() => client.updateComment(bad, "hi"), /update_comment: 'commentId'/); await assert.rejects(() => client.updateComment(bad, "hi"), /updateComment: 'commentId'/);
await assert.rejects(() => client.deleteComment(bad), /delete_comment: 'commentId'/); await assert.rejects(() => client.deleteComment(bad), /deleteComment: 'commentId'/);
await assert.rejects(() => client.getComment(bad), /get_comment: 'commentId'/); await assert.rejects(() => client.getComment(bad), /get_comment: 'commentId'/);
// createComment validates parentCommentId only when provided. // createComment validates parentCommentId only when provided.
await assert.rejects( await assert.rejects(
() => client.createComment("page-1", "body", "inline", "sel", bad), () => client.createComment("page-1", "body", "inline", "sel", bad),
/create_comment: 'parentCommentId'/, /createComment: 'parentCommentId'/,
); );
assert.equal(requests, 0, "no request (not even /auth/login) may be issued for a bad id"); assert.equal(requests, 0, "no request (not even /auth/login) may be issued for a bad id");
+93 -1
View File
@@ -1,7 +1,11 @@
import { test } from "node:test"; import { test } from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { filterComment, filterPage } from "../../build/lib/filters.js"; import {
filterComment,
filterPage,
filterSearchResult,
} from "../../build/lib/filters.js";
test("filterComment includes resolvedAt/resolvedById as null when absent", () => { test("filterComment includes resolvedAt/resolvedById as null when absent", () => {
const result = filterComment({ const result = filterComment({
@@ -171,3 +175,91 @@ test("filterPage includes both content and subpages together", () => {
assert.equal(result.content, "body"); assert.equal(result.content, "body");
assert.deepEqual(result.subpages, [{ id: "s1", title: "Sub" }]); assert.deepEqual(result.subpages, [{ id: "s1", title: "Sub" }]);
}); });
// --- filterSearchResult (#443 agent-lookup contract) -------------------------
test("filterSearchResult maps the lookup shape to {pageId,title,path,snippet,score}", () => {
const result = filterSearchResult({
id: "0199aa-uuid",
slugId: "slug-secret",
title: "backup-srv.local",
parentPageId: "0199pp",
path: ["Infrastructure", "Datacenter A", "Servers"],
snippet: "…IP: 10.0.12.5. Debian 12…",
score: 0.92,
});
assert.deepEqual(result, {
pageId: "0199aa-uuid",
title: "backup-srv.local",
path: ["Infrastructure", "Datacenter A", "Servers"],
snippet: "…IP: 10.0.12.5. Debian 12…",
score: 0.92,
});
});
test("filterSearchResult NEVER exposes slugId (pageId is the only identifier)", () => {
const result = filterSearchResult({
id: "uuid-1",
slugId: "slug-1",
title: "t",
path: [],
snippet: "s",
score: 0.1,
});
assert.equal("slugId" in result, false);
assert.equal("id" in result, false);
assert.equal(result.pageId, "uuid-1");
});
test("filterSearchResult root page yields path: []", () => {
const result = filterSearchResult({
id: "uuid-root",
title: "Root",
path: [],
snippet: "s",
score: 0.5,
});
assert.deepEqual(result.path, []);
});
test("filterSearchResult degrades a legacy FTS hit (no lookup fields)", () => {
// Stock upstream stripped the opt-in DTO fields → legacy shape with
// highlight + rank and no path/snippet/score.
const result = filterSearchResult({
id: "uuid-legacy",
slugId: "slug-legacy",
title: "Legacy",
parentPageId: null,
rank: 0.37,
highlight: "…matched <b>text</b>…",
space: { id: "sp1", name: "Space" },
});
assert.equal(result.pageId, "uuid-legacy");
assert.equal(result.title, "Legacy");
// snippet falls back to highlight, score to rank, path to [].
assert.equal(result.snippet, "…matched <b>text</b>…");
assert.equal(result.score, 0.37);
assert.deepEqual(result.path, []);
assert.equal("slugId" in result, false);
});
test("filterSearchResult is null-safe on missing snippet/score/path", () => {
const result = filterSearchResult({ id: "u", title: "t" });
assert.equal(result.pageId, "u");
assert.equal(result.snippet, "");
assert.equal(result.score, 0);
assert.deepEqual(result.path, []);
});
test("filterSearchResult ignores a non-array path", () => {
const result = filterSearchResult({
id: "u",
title: "t",
path: "not-an-array",
snippet: "s",
score: 1,
});
assert.deepEqual(result.path, []);
});
@@ -9,7 +9,7 @@ import {
// Pins the footnoteWarnings PLUMBING contract (#169 review; reduced in #414): the // Pins the footnoteWarnings PLUMBING contract (#169 review; reduced in #414): the
// field is present only when legacy reference-style `[^id]:` syntax is used and // field is present only when legacy reference-style `[^id]:` syntax is used and
// omitted otherwise, AND `import_page_markdown` analyzes the BODY (after the // omitted otherwise, AND `importPageMarkdown` analyzes the BODY (after the
// docmost:meta / docmost:comments blocks) — so a footnote-like token inside those // docmost:meta / docmost:comments blocks) — so a footnote-like token inside those
// JSON blocks never warns, while a real definition in the body does. // JSON blocks never warns, while a real definition in the body does.
// importPageMarkdown does exactly `footnoteWarningsField(parseDocmostMarkdown(full).body)` // importPageMarkdown does exactly `footnoteWarningsField(parseDocmostMarkdown(full).body)`
@@ -25,7 +25,7 @@ test("formatting-only edit (strip-toggle) is refused, not applied", () => {
assert.equal(failed.length, 1, "one refused edit"); assert.equal(failed.length, 1, "one refused edit");
assert.equal(failed[0].find, "~~x~~"); assert.equal(failed[0].find, "~~x~~");
assert.match(failed[0].reason, /cannot add or remove formatting marks/); assert.match(failed[0].reason, /cannot add or remove formatting marks/);
assert.match(failed[0].reason, /patch_node/); assert.match(failed[0].reason, /patchNode/);
// The document is untouched (the strike mark is preserved). // The document is untouched (the strike mark is preserved).
assert.deepEqual(out, snapshot); assert.deepEqual(out, snapshot);
}); });
@@ -143,7 +143,7 @@ test("typo fix wrapped in markdown still applies (not refused)", () => {
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// (iv) #410 footnote token: a `replace` containing `^[...]` is refused into // (iv) #410 footnote token: a `replace` containing `^[...]` is refused into
// failed[] (it would be written as a LITERAL string, never a real footnote). // failed[] (it would be written as a LITERAL string, never a real footnote).
// Nothing is applied; the reason points at insert_footnote. // Nothing is applied; the reason points at insertFootnote.
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
test("replace containing a `^[...]` footnote token is refused, not applied", () => { test("replace containing a `^[...]` footnote token is refused, not applied", () => {
const input = doc(paragraph(textNode("The claim stands."))); const input = doc(paragraph(textNode("The claim stands.")));
@@ -156,7 +156,7 @@ test("replace containing a `^[...]` footnote token is refused, not applied", ()
assert.equal(results.length, 0, "nothing applied"); assert.equal(results.length, 0, "nothing applied");
assert.equal(failed.length, 1, "one refused edit"); assert.equal(failed.length, 1, "one refused edit");
assert.equal(failed[0].find, "The claim stands."); assert.equal(failed[0].find, "The claim stands.");
assert.match(failed[0].reason, /insert_footnote/); assert.match(failed[0].reason, /insertFootnote/);
// The document is byte-for-byte untouched — no literal `^[` was written. // The document is byte-for-byte untouched — no literal `^[` was written.
assert.deepEqual(out, snapshot); assert.deepEqual(out, snapshot);
}); });
@@ -0,0 +1,121 @@
// #413: unit tests for the markdown-fragment helpers used by patchNode/insertNode.
import { test } from "node:test";
import assert from "node:assert/strict";
import {
importMarkdownFragment,
canBeDocChild,
findUnrepresentableTableAttrs,
} from "../../build/lib/markdown-fragment.js";
function findAll(node, type, acc = []) {
if (!node || typeof node !== "object") return acc;
if (node.type === type) acc.push(node);
if (Array.isArray(node.content))
for (const c of node.content) findAll(c, type, acc);
return acc;
}
test("importMarkdownFragment: plain markdown -> blocks, no definitions", async () => {
const { blocks, definitions } = await importMarkdownFragment(
"first\n\nsecond",
);
assert.equal(blocks.length, 2);
assert.equal(definitions.length, 0);
assert.equal(blocks[0].type, "paragraph");
});
test("importMarkdownFragment: `^[...]` footnote -> a definition + a remapped ref", async () => {
const { blocks, definitions } = await importMarkdownFragment(
"a claim^[the note]",
);
assert.equal(definitions.length, 1);
const refs = findAll({ type: "doc", content: blocks }, "footnoteReference");
assert.equal(refs.length, 1);
// The reference id must match the (remapped) definition id.
assert.equal(refs[0].attrs.id, definitions[0].attrs.id);
// The id is NOT the importer's sequential "fn-1" — it was remapped to a fresh
// uuid so it cannot collide with a page footnote of the same number.
assert.notEqual(refs[0].attrs.id, "fn-1");
});
test("importMarkdownFragment: whitespace markdown imports to a single empty paragraph", async () => {
// The importer yields one empty paragraph for whitespace-only input (not zero
// blocks), so the fragment path returns that block. The client's XOR guard
// (markdown.trim() !== "") is what rejects an empty-string patch up front, so
// importMarkdownFragment never sees a truly empty string via patch/insert.
const { blocks, definitions } = await importMarkdownFragment(" \n ");
assert.equal(blocks.length, 1);
assert.equal(blocks[0].type, "paragraph");
assert.equal(definitions.length, 0);
});
test("canBeDocChild: paragraph/heading/table are doc children; tableRow/cell are not", () => {
assert.equal(canBeDocChild("paragraph"), true);
assert.equal(canBeDocChild("heading"), true);
assert.equal(canBeDocChild("table"), true);
assert.equal(canBeDocChild("tableRow"), false);
assert.equal(canBeDocChild("tableCell"), false);
assert.equal(canBeDocChild("tableHeader"), false);
assert.equal(canBeDocChild("text"), false);
assert.equal(canBeDocChild(undefined), false);
assert.equal(canBeDocChild("notARealType"), false);
});
const cell = (attrs, text) => ({
type: "tableCell",
attrs,
content: [{ type: "paragraph", content: [{ type: "text", text }] }],
});
test("findUnrepresentableTableAttrs: null for a plain paragraph and a simple table", () => {
assert.equal(
findUnrepresentableTableAttrs({
type: "paragraph",
content: [{ type: "text", text: "x" }],
}),
null,
);
const simpleTable = {
type: "table",
content: [
{
type: "tableRow",
content: [cell({ colspan: 1, rowspan: 1 }, "a")],
},
],
};
assert.equal(findUnrepresentableTableAttrs(simpleTable), null);
});
test("findUnrepresentableTableAttrs: flags colspan/rowspan/colwidth/backgroundColor", () => {
const mk = (attrs) => ({
type: "table",
content: [{ type: "tableRow", content: [cell(attrs, "a")] }],
});
assert.match(findUnrepresentableTableAttrs(mk({ colspan: 2 })), /colspan/);
assert.match(findUnrepresentableTableAttrs(mk({ rowspan: 2 })), /rowspan/);
assert.match(
findUnrepresentableTableAttrs(mk({ colwidth: [120] })),
/colwidth/,
);
assert.match(
findUnrepresentableTableAttrs(mk({ backgroundColor: "#eee" })),
/backgroundColor/,
);
});
test("findUnrepresentableTableAttrs: finds a span nested deep (table inside a callout)", () => {
const doc = {
type: "callout",
content: [
{
type: "table",
content: [
{ type: "tableRow", content: [cell({ colspan: 3 }, "wide")] },
],
},
],
};
assert.match(findUnrepresentableTableAttrs(doc), /colspan/);
});
@@ -0,0 +1,84 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
// Issue #449, invariant 2 ("no-await-окно"): the atomicity of the
// read -> transform -> write section in CollabSession.mutate depends on there
// being NO `await` (nor any other async yield point) between
// `TiptapTransformer.fromYdoc` and `applyDocToFragment`. Yjs applies queued
// remote updates only when the event loop yields, so an accidental await in that
// window would let a concurrent human edit interleave and be clobbered (#152).
//
// That was an invariant enforced only by a comment. This test turns a violation
// RED: it reads the SOURCE of collab-session.ts, extracts the block delimited by
// the machine-readable BEGIN/END markers, and asserts no async boundary appears
// inside it. Introducing an `await` (or `for await`, or `yield`) between the
// markers fails this test.
const here = dirname(fileURLToPath(import.meta.url));
// Scan the .ts SOURCE (not the compiled .js): the markers live in the source and
// transpilation could rewrite/erase them, so the source is the authoritative
// artifact the human edits.
const sourcePath = join(here, "..", "..", "src", "lib", "collab-session.ts");
const source = readFileSync(sourcePath, "utf8");
const BEGIN = "=== MUTATE-CRITICAL-WINDOW: BEGIN";
const END = "=== MUTATE-CRITICAL-WINDOW: END";
test("critical-window markers exist exactly once each", () => {
const begins = source.split(BEGIN).length - 1;
const ends = source.split(END).length - 1;
assert.equal(
begins,
1,
`expected exactly one '${BEGIN}' marker, found ${begins}`,
);
assert.equal(ends, 1, `expected exactly one '${END}' marker, found ${ends}`);
});
test("the read->write critical window contains no async boundary (no await/yield)", () => {
const beginIdx = source.indexOf(BEGIN);
const endIdx = source.indexOf(END);
assert.ok(beginIdx !== -1, "BEGIN marker not found");
assert.ok(endIdx !== -1, "END marker not found");
assert.ok(endIdx > beginIdx, "END marker must come after BEGIN marker");
// The block strictly between the two marker lines. Move past the end of the
// BEGIN marker line so the marker comment text itself is not scanned.
const afterBeginLine = source.indexOf("\n", beginIdx) + 1;
const block = source.slice(afterBeginLine, endIdx);
// Detect any real async yield keyword as a whole word. `\bawait\b` also matches
// inside `for await`, which is exactly what we want to forbid here.
const forbidden = [/\bawait\b/, /\byield\b/];
for (const re of forbidden) {
const m = block.match(re);
assert.equal(
m,
null,
`forbidden async boundary '${m?.[0]}' found inside the no-await critical ` +
`window of CollabSession.mutate. INVARIANT 1 (#449): the block between ` +
`TiptapTransformer.fromYdoc and applyDocToFragment must be fully ` +
`synchronous — an await there reopens the clobber-live-edits race (#152).`,
);
}
});
test("the critical window still spans fromYdoc -> applyDocToFragment", () => {
// Guards the markers from drifting off the code they are meant to protect: if
// someone moves the read/write out of the window, this catches it.
const beginIdx = source.indexOf(BEGIN);
const endIdx = source.indexOf(END);
const afterBeginLine = source.indexOf("\n", beginIdx) + 1;
const block = source.slice(afterBeginLine, endIdx);
assert.ok(
block.includes("TiptapTransformer.fromYdoc"),
"critical window must contain the TiptapTransformer.fromYdoc read",
);
assert.ok(
block.includes("applyDocToFragment"),
"critical window must contain the applyDocToFragment write",
);
});
+7 -7
View File
@@ -486,34 +486,34 @@ test("insertNodeRelative truly-missing anchor still returns inserted:false", ()
assert.equal(inserted, false); assert.equal(inserted, false);
}); });
// assertUnambiguousMatch (#159, #185 review pt 2): the patch_node/delete_node // assertUnambiguousMatch (#159, #185 review pt 2): the patchNode/deleteNode
// guard. Docmost duplicates block ids on copy/paste, so a write by id that // guard. Docmost duplicates block ids on copy/paste, so a write by id that
// matches >1 node must be REFUSED (the caller already skipped the write for any // matches >1 node must be REFUSED (the caller already skipped the write for any
// count !== 1; this reports the error). The duplicate COUNT itself is covered by // count !== 1; this reports the error). The duplicate COUNT itself is covered by
// the replaceNodeById/deleteNodeById tests above (count===2 for a 2-dup doc). // the replaceNodeById/deleteNodeById tests above (count===2 for a 2-dup doc).
test("assertUnambiguousMatch: count 0 throws 'no node found'", () => { test("assertUnambiguousMatch: count 0 throws 'no node found'", () => {
assert.throws( assert.throws(
() => assertUnambiguousMatch("patch_node", "replace", 0, "n1", "p1"), () => assertUnambiguousMatch("patchNode", "replace", 0, "n1", "p1"),
/patch_node: no node with id "n1" found on page p1/, /patchNode: no node with id "n1" found on page p1/,
); );
}); });
test("assertUnambiguousMatch: count > 1 refuses with an 'ambiguous' error", () => { test("assertUnambiguousMatch: count > 1 refuses with an 'ambiguous' error", () => {
assert.throws( assert.throws(
() => assertUnambiguousMatch("patch_node", "replace", 2, "dup", "p1"), () => assertUnambiguousMatch("patchNode", "replace", 2, "dup", "p1"),
/ambiguous.*Refusing to replace all of them; nothing was changed/, /ambiguous.*Refusing to replace all of them; nothing was changed/,
); );
assert.throws( assert.throws(
() => assertUnambiguousMatch("delete_node", "delete", 3, "dup", "p1"), () => assertUnambiguousMatch("deleteNode", "delete", 3, "dup", "p1"),
/ambiguous.*Refusing to delete all of them; nothing was changed/, /ambiguous.*Refusing to delete all of them; nothing was changed/,
); );
}); });
test("assertUnambiguousMatch: exactly one match does NOT throw", () => { test("assertUnambiguousMatch: exactly one match does NOT throw", () => {
assert.doesNotThrow(() => assert.doesNotThrow(() =>
assertUnambiguousMatch("patch_node", "replace", 1, "n1", "p1"), assertUnambiguousMatch("patchNode", "replace", 1, "n1", "p1"),
); );
assert.doesNotThrow(() => assert.doesNotThrow(() =>
assertUnambiguousMatch("delete_node", "delete", 1, "n1", "p1"), assertUnambiguousMatch("deleteNode", "delete", 1, "n1", "p1"),
); );
}); });
+74 -8
View File
@@ -1,13 +1,26 @@
import { test } from "node:test"; import { test } from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { withPageLock } from "../../build/lib/page-lock.js"; import { withPageLock, isUuid } from "../../build/lib/page-lock.js";
const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
// withPageLock now asserts its key is a canonical UUID (#449, "resolve-then-
// lock"), so the mechanics tests below must lock under real UUIDs, not arbitrary
// labels. Distinct valid UUIDv7-shaped ids for the distinct-page cases.
const U = {
same: "11111111-1111-7111-8111-111111111111",
ordered: "22222222-2222-7222-8222-222222222222",
poison: "33333333-3333-7333-8333-333333333333",
poison2: "44444444-4444-7444-8444-444444444444",
A: "aaaaaaaa-aaaa-7aaa-8aaa-aaaaaaaaaaaa",
B: "bbbbbbbb-bbbb-7bbb-8bbb-bbbbbbbbbbbb",
leak: "55555555-5555-7555-8555-555555555555",
};
test("two ops on the same pageId run strictly sequentially (no overlap)", async () => { test("two ops on the same pageId run strictly sequentially (no overlap)", async () => {
const events = []; const events = [];
const pageId = "same-page"; const pageId = U.same;
const p1 = withPageLock(pageId, async () => { const p1 = withPageLock(pageId, async () => {
events.push("start-1"); events.push("start-1");
@@ -33,7 +46,7 @@ test("two ops on the same pageId run strictly sequentially (no overlap)", async
}); });
test("same pageId ordering holds for many queued ops", async () => { test("same pageId ordering holds for many queued ops", async () => {
const pageId = "ordered-page"; const pageId = U.ordered;
const order = []; const order = [];
const active = { count: 0, maxConcurrent: 0 }; const active = { count: 0, maxConcurrent: 0 };
@@ -60,7 +73,7 @@ test("same pageId ordering holds for many queued ops", async () => {
}); });
test("a rejecting op does not poison the chain for the same page", async () => { test("a rejecting op does not poison the chain for the same page", async () => {
const pageId = "poison-page"; const pageId = U.poison;
const events = []; const events = [];
const failing = withPageLock(pageId, async () => { const failing = withPageLock(pageId, async () => {
@@ -87,7 +100,7 @@ test("a rejecting op does not poison the chain for the same page", async () => {
}); });
test("failing op queued before a success both resolve/reject correctly", async () => { test("failing op queued before a success both resolve/reject correctly", async () => {
const pageId = "poison-page-2"; const pageId = U.poison2;
const order = []; const order = [];
const failing = withPageLock(pageId, async () => { const failing = withPageLock(pageId, async () => {
@@ -111,14 +124,14 @@ test("failing op queued before a success both resolve/reject correctly", async (
test("ops on different pageIds run concurrently (overlap)", async () => { test("ops on different pageIds run concurrently (overlap)", async () => {
const events = []; const events = [];
const pA = withPageLock("page-A", async () => { const pA = withPageLock(U.A, async () => {
events.push("A-start"); events.push("A-start");
await delay(40); await delay(40);
events.push("A-end"); events.push("A-end");
return "A"; return "A";
}); });
const pB = withPageLock("page-B", async () => { const pB = withPageLock(U.B, async () => {
events.push("B-start"); events.push("B-start");
await delay(10); await delay(10);
events.push("B-end"); events.push("B-end");
@@ -134,7 +147,7 @@ test("ops on different pageIds run concurrently (overlap)", async () => {
}); });
test("no functional leak: many sequential ops on same page keep working", async () => { test("no functional leak: many sequential ops on same page keep working", async () => {
const pageId = "leak-page"; const pageId = U.leak;
// Run a long series of fully sequential ops (each awaited before the next is // Run a long series of fully sequential ops (each awaited before the next is
// queued) so the internal map entry is created and dropped repeatedly. // queued) so the internal map entry is created and dropped repeatedly.
@@ -151,3 +164,56 @@ test("no functional leak: many sequential ops on same page keep working", async
const final = await withPageLock(pageId, async () => "still-works"); const final = await withPageLock(pageId, async () => "still-works");
assert.equal(final, "still-works"); assert.equal(final, "still-works");
}); });
// --- Issue #449: fail-fast on a non-canonical lock key ---------------------
// A write method that reaches the lock path with an unresolved slugId (or any
// non-UUID key) must fail IMMEDIATELY and LOUDLY, not lock under a split key and
// silently lose per-page serialization. These assert withPageLock rejects such
// a key before ever running fn.
test("withPageLock throws on a raw 10-char slugId (unresolved key)", () => {
let ran = false;
assert.throws(
() =>
withPageLock("p7Xk29Lm4Q", async () => {
ran = true;
return "should-not-run";
}),
/canonical page UUID|resolve-then-lock/i,
"a slugId key must fail-fast at the lock",
);
// The work must NOT have started: fail-fast means no serialization was
// silently skipped under a bad key.
assert.equal(ran, false, "fn must not run when the key is rejected");
});
test("withPageLock throws on other non-UUID keys (label, empty, non-string)", () => {
for (const bad of ["same-page", "", "not-a-uuid", "1234"]) {
assert.throws(
() => withPageLock(bad, async () => "x"),
/canonical page UUID/i,
`expected withPageLock to reject key ${JSON.stringify(bad)}`,
);
}
// A non-string key is also rejected (guards a mistyped call site).
assert.throws(
() => withPageLock(/** @type {any} */ (undefined), async () => "x"),
/canonical page UUID/i,
);
});
test("withPageLock accepts a canonical UUID key (no false positive)", async () => {
const uuid = "0192f3a4-b5c6-7d8e-9f01-23456789abcd";
assert.equal(isUuid(uuid), true);
const r = await withPageLock(uuid, async () => "ok");
assert.equal(r, "ok");
});
test("isUuid discriminates UUIDs from slugIds (shared predicate)", () => {
// The predicate withPageLock asserts on is the SAME one resolvePageId uses to
// decide whether a pageId is already a UUID (imported from page-lock).
assert.equal(isUuid("0192f3a4-b5c6-7d8e-9f01-23456789abcd"), true);
assert.equal(isUuid("p7Xk29Lm4Q"), false); // 10-char nanoid slugId
assert.equal(isUuid("not-a-uuid"), false);
assert.equal(isUuid(""), false);
});
+5 -5
View File
@@ -155,7 +155,7 @@ test("insertTableRow at index 0 inserts before the header and pads to 3 cells",
test("insertTableRow throws when given more cells than columns", () => { test("insertTableRow throws when given more cells than columns", () => {
assert.throws( assert.throws(
() => insertTableRow(makeDoc(), "#1", ["a", "b", "c", "d"]), () => insertTableRow(makeDoc(), "#1", ["a", "b", "c", "d"]),
/table_insert_row: got 4 cell\(s\) but the table has 3 column\(s\)/, /tableInsertRow: got 4 cell\(s\) but the table has 3 column\(s\)/,
); );
}); });
@@ -232,7 +232,7 @@ test("insertTableRow uses the max column count across all rows (ragged table)",
// ...but 4 cells exceed the widest row and throw. // ...but 4 cells exceed the widest row and throw.
assert.throws( assert.throws(
() => insertTableRow(makeRaggedDoc(), "#0", ["a", "b", "c", "d"]), () => insertTableRow(makeRaggedDoc(), "#0", ["a", "b", "c", "d"]),
/table_insert_row: got 4 cell\(s\) but the table has 3 column\(s\)/, /tableInsertRow: got 4 cell\(s\) but the table has 3 column\(s\)/,
); );
}); });
@@ -286,7 +286,7 @@ test("deleteTableRow removes the 3rd row -> rows:2", () => {
test("deleteTableRow out-of-range index throws", () => { test("deleteTableRow out-of-range index throws", () => {
assert.throws( assert.throws(
() => deleteTableRow(makeDoc(), "#1", 9), () => deleteTableRow(makeDoc(), "#1", 9),
/table_delete_row: row index 9 out of range \(table has 3 row\(s\)\)/, /tableDeleteRow: row index 9 out of range \(table has 3 row\(s\)\)/,
); );
}); });
@@ -329,10 +329,10 @@ test("updateTableCell sets cell [1,1] to 'Z' and preserves the paragraph id", ()
test("updateTableCell out-of-range row/col throws", () => { test("updateTableCell out-of-range row/col throws", () => {
assert.throws( assert.throws(
() => updateTableCell(makeDoc(), "#1", 9, 0, "x"), () => updateTableCell(makeDoc(), "#1", 9, 0, "x"),
/table_update_cell: cell \[9,0\] out of range/, /tableUpdateCell: cell \[9,0\] out of range/,
); );
assert.throws( assert.throws(
() => updateTableCell(makeDoc(), "#1", 0, 9, "x"), () => updateTableCell(makeDoc(), "#1", 0, 9, "x"),
/table_update_cell: cell \[0,9\] out of range/, /tableUpdateCell: cell \[0,9\] out of range/,
); );
}); });
+15 -14
View File
@@ -35,13 +35,13 @@ function registeredToolNames() {
const indexSrc = readFileSync(join(SRC, "index.ts"), "utf8"); const indexSrc = readFileSync(join(SRC, "index.ts"), "utf8");
const specsSrc = readFileSync(join(SRC, "tool-specs.ts"), "utf8"); const specsSrc = readFileSync(join(SRC, "tool-specs.ts"), "utf8");
const names = new Set(); const names = new Set();
for (const m of indexSrc.matchAll(/registerTool\(\s*"([a-z0-9_]+)"/g)) { for (const m of indexSrc.matchAll(/registerTool\(\s*"([a-zA-Z0-9_]+)"/g)) {
names.add(m[1]); names.add(m[1]);
} }
// Each spec is one `{ ... }` block; scrape its mcpName but skip a block that // Each spec is one `{ ... }` block; scrape its mcpName but skip a block that
// carries `inAppOnly: true` (not registered on the external MCP host). // carries `inAppOnly: true` (not registered on the external MCP host).
for (const block of specsSrc.split(/\n\s{2}\w+:\s*\{/)) { for (const block of specsSrc.split(/\n\s{2}\w+:\s*\{/)) {
const nameMatch = block.match(/mcpName:\s*['"]([a-z0-9_]+)['"]/); const nameMatch = block.match(/mcpName:\s*['"]([a-zA-Z0-9_]+)['"]/);
if (!nameMatch) continue; if (!nameMatch) continue;
if (/inAppOnly:\s*true/.test(block)) continue; if (/inAppOnly:\s*true/.test(block)) continue;
names.add(nameMatch[1]); names.add(nameMatch[1]);
@@ -82,27 +82,28 @@ test("the inventory has no phantom tool (every line is a real registered tool)",
); );
}); });
// #411: the external MCP surface gains update_page_markdown and LOSES // #411: the external MCP surface gains updatePageMarkdown and LOSES
// import_page_markdown (now inAppOnly). The in-app agent still keeps // importPageMarkdown (now inAppOnly). The in-app agent still keeps
// importPageMarkdown — asserted in the server-side contract spec. // importPageMarkdown — asserted in the server-side contract spec. (#412 renamed
test("update_page_markdown is on the MCP surface; import_page_markdown is NOT", () => { // both public MCP tool names to camelCase.)
test("updatePageMarkdown is on the MCP surface; importPageMarkdown is NOT", () => {
const inventory = new Set(buildToolInventoryLines().map((l) => l.name)); const inventory = new Set(buildToolInventoryLines().map((l) => l.name));
assert.ok( assert.ok(
inventory.has("update_page_markdown"), inventory.has("updatePageMarkdown"),
"update_page_markdown should be registered on the external MCP surface", "updatePageMarkdown should be registered on the external MCP surface",
); );
assert.ok( assert.ok(
!inventory.has("import_page_markdown"), !inventory.has("importPageMarkdown"),
"import_page_markdown must be dropped from the external MCP surface (#411)", "importPageMarkdown must be dropped from the external MCP surface (#411)",
); );
// And the routing prose no longer points MCP clients at it. // And the routing prose no longer points MCP clients at it.
assert.ok( assert.ok(
!ROUTING_PROSE.includes("import_page_markdown"), !ROUTING_PROSE.includes("importPageMarkdown"),
"ROUTING_PROSE still mentions the removed import_page_markdown", "ROUTING_PROSE still mentions the removed importPageMarkdown",
); );
assert.ok( assert.ok(
ROUTING_PROSE.includes("update_page_markdown"), ROUTING_PROSE.includes("updatePageMarkdown"),
"ROUTING_PROSE should mention update_page_markdown", "ROUTING_PROSE should mention updatePageMarkdown",
); );
}); });
+120 -31
View File
@@ -22,10 +22,13 @@ test("every spec exposes mcpName + inAppKey, and the key matches inAppKey", () =
} }
}); });
test("mcpName uses snake_case and inAppKey uses camelCase", () => { // Since issue #412 the external MCP name equals the in-app key: both are the
// same camelCase identifier (mcpName === inAppKey).
test("mcpName and inAppKey are the same camelCase identifier", () => {
for (const [key, spec] of Object.entries(SHARED_TOOL_SPECS)) { for (const [key, spec] of Object.entries(SHARED_TOOL_SPECS)) {
assert.match(spec.mcpName, /^[a-z0-9]+(_[a-z0-9]+)*$/, `${key}: mcpName not snake_case`); assert.match(spec.mcpName, /^[a-z][a-zA-Z0-9]*$/, `${key}: mcpName not camelCase`);
assert.match(spec.inAppKey, /^[a-z][a-zA-Z0-9]*$/, `${key}: inAppKey not camelCase`); assert.match(spec.inAppKey, /^[a-z][a-zA-Z0-9]*$/, `${key}: inAppKey not camelCase`);
assert.equal(spec.mcpName, spec.inAppKey, `${key}: mcpName must equal inAppKey`);
} }
}); });
@@ -59,7 +62,7 @@ test("buildShape (when present) returns a usable ZodRawShape with a real zod", (
test("editPageText builder produces { pageId, edits } and drops the stale strip-and-retry claim", () => { test("editPageText builder produces { pageId, edits } and drops the stale strip-and-retry claim", () => {
const spec = SHARED_TOOL_SPECS.editPageText; const spec = SHARED_TOOL_SPECS.editPageText;
assert.equal(spec.mcpName, "edit_page_text"); assert.equal(spec.mcpName, "editPageText");
const shape = spec.buildShape(z); const shape = spec.buildShape(z);
assert.deepEqual(Object.keys(shape).sort(), ["edits", "pageId"]); assert.deepEqual(Object.keys(shape).sort(), ["edits", "pageId"]);
// A valid edits batch parses. // A valid edits batch parses.
@@ -78,49 +81,68 @@ test("editPageText builder produces { pageId, edits } and drops the stale strip-
assert.match(spec.description, /REFUSED into\s+failed\[\]/); assert.match(spec.description, /REFUSED into\s+failed\[\]/);
}); });
test("getNode builder produces exactly { pageId, nodeId }", () => { // #413: getNode gained an optional `format` (markdown default / json opt-in).
const shape = SHARED_TOOL_SPECS.getNode.buildShape(z); test("getNode builder produces { pageId, nodeId, format? } with format optional", () => {
assert.deepEqual(Object.keys(shape).sort(), ["nodeId", "pageId"]); const spec = SHARED_TOOL_SPECS.getNode;
const shape = spec.buildShape(z);
assert.deepEqual(Object.keys(shape).sort(), ["format", "nodeId", "pageId"]);
const schema = z.object(shape);
// format is optional (markdown default lives in the client).
assert.doesNotThrow(() => schema.parse({ pageId: "p1", nodeId: "n1" }));
assert.doesNotThrow(() =>
schema.parse({ pageId: "p1", nodeId: "n1", format: "json" }),
);
assert.throws(() =>
schema.parse({ pageId: "p1", nodeId: "n1", format: "yaml" }),
);
// The description advertises the markdown default and the json opt-in.
assert.match(spec.description, /markdown/i);
assert.match(spec.description, /json/i);
}); });
test("patchNode spec exists, merges BOTH descriptions, builds { pageId, nodeId, node }", () => { // #413: patchNode takes XOR { markdown | node } (both schema-optional).
test("patchNode spec exists, describes markdown+node XOR, builds { pageId, nodeId, markdown?, node? }", () => {
const spec = SHARED_TOOL_SPECS.patchNode; const spec = SHARED_TOOL_SPECS.patchNode;
assert.ok(spec, "patchNode spec missing"); assert.ok(spec, "patchNode spec missing");
assert.equal(spec.mcpName, "patch_node"); assert.equal(spec.mcpName, "patchNode");
assert.equal(spec.inAppKey, "patchNode"); assert.equal(spec.inAppKey, "patchNode");
// The canonical description must carry the key guidance from BOTH originals: // The canonical description must carry the #413 guidance.
// - MCP-only: "WITHOUT resending the whole document" + the cheaper/safer note. assert.match(spec.description, /WITHOUT/i);
// - in-app-only: "keeps the same node id" + the "Reversible ... page history" assert.match(spec.description, /EXACTLY ONE of `markdown` or `node`/);
// framing the MCP copy lacked. assert.match(spec.description, /RECOMMENDED/);
assert.match(spec.description, /WITHOUT resending the whole document/); assert.match(spec.description, /keeps the same block id/i);
assert.match(spec.description, /Cheaper and safer/);
assert.match(spec.description, /keeps the same node id/i);
assert.match(spec.description, /Reversible/i); assert.match(spec.description, /Reversible/i);
assert.match(spec.description, /page history/i); assert.match(spec.description, /page history/i);
const shape = spec.buildShape(z); const shape = spec.buildShape(z);
assert.deepEqual(Object.keys(shape).sort(), ["node", "nodeId", "pageId"]); assert.deepEqual(
// A minimal valid input parses (node accepts an arbitrary object via z.any()). Object.keys(shape).sort(),
const parsed = z.object(shape).parse({ ["markdown", "node", "nodeId", "pageId"],
);
// markdown and node are BOTH optional in the schema (XOR enforced at runtime).
const schema = z.object(shape);
const parsedMd = schema.parse({ pageId: "p1", nodeId: "n1", markdown: "hi" });
assert.equal(parsedMd.markdown, "hi");
const parsedNode = schema.parse({
pageId: "p1", pageId: "p1",
nodeId: "n1", nodeId: "n1",
node: { type: "paragraph" }, node: { type: "paragraph" },
}); });
assert.equal(parsed.pageId, "p1"); assert.equal(parsedNode.pageId, "p1");
assert.equal(parsed.nodeId, "n1"); // Neither given parses at the schema level (the client throws the XOR error).
assert.doesNotThrow(() => schema.parse({ pageId: "p1", nodeId: "n1" }));
}); });
test("insertNode spec exists, merges BOTH descriptions, builds the full anchor shape", () => { // #413: insertNode also takes XOR { markdown | node } plus the anchor shape.
test("insertNode spec exists, describes markdown+node XOR, builds the full anchor+content shape", () => {
const spec = SHARED_TOOL_SPECS.insertNode; const spec = SHARED_TOOL_SPECS.insertNode;
assert.ok(spec, "insertNode spec missing"); assert.ok(spec, "insertNode spec missing");
assert.equal(spec.mcpName, "insert_node"); assert.equal(spec.mcpName, "insertNode");
assert.equal(spec.inAppKey, "insertNode"); assert.equal(spec.inAppKey, "insertNode");
// Canonical description must keep BOTH sides' nuance:
// - in-app-only: "EXACTLY ONE of anchorNodeId or anchorText" + "Reversible".
// - MCP-only: the table-structure (tableRow/tableCell) insertion guidance.
assert.match(spec.description, /EXACTLY ONE of anchorNodeId or anchorText/); assert.match(spec.description, /EXACTLY ONE of anchorNodeId or anchorText/);
assert.match(spec.description, /EXACTLY ONE of `markdown` or `node`/);
assert.match(spec.description, /tableRow/); assert.match(spec.description, /tableRow/);
assert.match(spec.description, /append is top-level only/); assert.match(spec.description, /append is top-level only/);
assert.match(spec.description, /Reversible via page history/); assert.match(spec.description, /Reversible via page history/);
@@ -128,18 +150,85 @@ test("insertNode spec exists, merges BOTH descriptions, builds the full anchor s
const shape = spec.buildShape(z); const shape = spec.buildShape(z);
assert.deepEqual( assert.deepEqual(
Object.keys(shape).sort(), Object.keys(shape).sort(),
["anchorNodeId", "anchorText", "node", "pageId", "position"], ["anchorNodeId", "anchorText", "markdown", "node", "pageId", "position"],
); );
// before/after/append are the only accepted positions; anchors are optional. // before/after/append are the only accepted positions; markdown/node/anchors optional.
const schema = z.object(shape); const schema = z.object(shape);
assert.doesNotThrow(() =>
schema.parse({ pageId: "p1", markdown: "hi", position: "append" }),
);
assert.doesNotThrow(() => assert.doesNotThrow(() =>
schema.parse({ pageId: "p1", node: { type: "paragraph" }, position: "append" }), schema.parse({ pageId: "p1", node: { type: "paragraph" }, position: "append" }),
); );
assert.throws(() => assert.throws(() =>
schema.parse({ pageId: "p1", node: {}, position: "sideways" }), schema.parse({ pageId: "p1", markdown: "x", position: "sideways" }),
); );
}); });
// #443: getTree — a space's page hierarchy (or a subtree) in one request.
test("getTree spec exists on both hosts, builds { spaceId, rootPageId?, maxDepth? }", () => {
const spec = SHARED_TOOL_SPECS.getTree;
assert.ok(spec, "getTree spec missing");
assert.equal(spec.mcpName, "getTree");
assert.equal(spec.inAppKey, "getTree");
// Shared spec: registered on BOTH hosts.
assert.notEqual(spec.inAppOnly, true);
assert.notEqual(spec.mcpOnly, true);
const shape = spec.buildShape(z);
assert.deepEqual(Object.keys(shape).sort(), ["maxDepth", "rootPageId", "spaceId"]);
const schema = z.object(shape);
// spaceId required; rootPageId + maxDepth optional.
assert.doesNotThrow(() => schema.parse({ spaceId: "sp1" }));
assert.throws(() => schema.parse({}));
assert.doesNotThrow(() =>
schema.parse({ spaceId: "sp1", rootPageId: "p1", maxDepth: 2 }),
);
// maxDepth is an integer >= 1.
assert.throws(() => schema.parse({ spaceId: "sp1", maxDepth: 0 }));
assert.throws(() => schema.parse({ spaceId: "sp1", maxDepth: 1.5 }));
// The description advertises the output node shape, rootPageId, maxDepth, and
// steers away from the deprecated listPages tree:true.
assert.match(spec.description, /pageId/);
assert.match(spec.description, /rootPageId/);
assert.match(spec.description, /maxDepth/);
assert.match(spec.description, /hasChildren/);
assert.match(spec.description, /listPages tree:true/);
});
// #443: getPageContext — a page's breadcrumbs + direct children in one call.
test("getPageContext spec exists on both hosts, builds { pageId }", () => {
const spec = SHARED_TOOL_SPECS.getPageContext;
assert.ok(spec, "getPageContext spec missing");
assert.equal(spec.mcpName, "getPageContext");
assert.equal(spec.inAppKey, "getPageContext");
// Shared spec: registered on BOTH hosts.
assert.notEqual(spec.inAppOnly, true);
assert.notEqual(spec.mcpOnly, true);
const shape = spec.buildShape(z);
assert.deepEqual(Object.keys(shape).sort(), ["pageId"]);
const schema = z.object(shape);
// pageId required.
assert.doesNotThrow(() => schema.parse({ pageId: "p1" }));
assert.throws(() => schema.parse({}));
// The description advertises the output shape (page/breadcrumbs/children) and
// the root-page empty-breadcrumbs contract.
assert.match(spec.description, /breadcrumbs/);
assert.match(spec.description, /children/);
assert.match(spec.description, /hasChildren/);
assert.match(spec.description, /getTree/);
});
// #443: listPages tree:true is deprecated in favour of getTree.
test("listPages description deprecates tree:true and points at getTree", () => {
const spec = SHARED_TOOL_SPECS.listPages;
assert.match(spec.description, /DEPRECATED/i);
assert.match(spec.description, /getTree/);
});
test("no-arg specs (getWorkspace/listSpaces/listShares) omit buildShape", () => { test("no-arg specs (getWorkspace/listSpaces/listShares) omit buildShape", () => {
for (const key of ["getWorkspace", "listSpaces", "listShares"]) { for (const key of ["getWorkspace", "listSpaces", "listShares"]) {
assert.equal(SHARED_TOOL_SPECS[key].buildShape, undefined, `${key} should be no-arg`); assert.equal(SHARED_TOOL_SPECS[key].buildShape, undefined, `${key} should be no-arg`);
@@ -150,7 +239,7 @@ test("no-arg specs (getWorkspace/listSpaces/listShares) omit buildShape", () =>
test("updatePageMarkdown spec exists, pairs with updatePageJson, builds { pageId, content, title }", () => { test("updatePageMarkdown spec exists, pairs with updatePageJson, builds { pageId, content, title }", () => {
const spec = SHARED_TOOL_SPECS.updatePageMarkdown; const spec = SHARED_TOOL_SPECS.updatePageMarkdown;
assert.ok(spec, "updatePageMarkdown spec missing"); assert.ok(spec, "updatePageMarkdown spec missing");
assert.equal(spec.mcpName, "update_page_markdown"); assert.equal(spec.mcpName, "updatePageMarkdown");
assert.equal(spec.inAppKey, "updatePageMarkdown"); assert.equal(spec.inAppKey, "updatePageMarkdown");
// Registered on BOTH hosts (a shared spec, no inAppOnly/mcpOnly flag). // Registered on BOTH hosts (a shared spec, no inAppOnly/mcpOnly flag).
assert.notEqual(spec.inAppOnly, true); assert.notEqual(spec.inAppOnly, true);
@@ -168,13 +257,13 @@ test("updatePageMarkdown spec exists, pairs with updatePageJson, builds { pageId
assert.match(spec.description, /\^\[/); assert.match(spec.description, /\^\[/);
}); });
// #411: import_page_markdown is dropped from the EXTERNAL MCP surface but stays // #411: importPageMarkdown is dropped from the EXTERNAL MCP surface but stays
// available to the in-app agent — encoded as inAppOnly on the shared spec. // available to the in-app agent — encoded as inAppOnly on the shared spec.
test("importPageMarkdown spec is inAppOnly (removed from the external MCP surface, kept in-app)", () => { test("importPageMarkdown spec is inAppOnly (removed from the external MCP surface, kept in-app)", () => {
const spec = SHARED_TOOL_SPECS.importPageMarkdown; const spec = SHARED_TOOL_SPECS.importPageMarkdown;
assert.ok(spec, "importPageMarkdown spec missing"); assert.ok(spec, "importPageMarkdown spec missing");
assert.equal(spec.inAppOnly, true); assert.equal(spec.inAppOnly, true);
// The spec + its client method are NOT deleted — only hidden from the MCP host. // The spec + its client method are NOT deleted — only hidden from the MCP host.
assert.equal(spec.mcpName, "import_page_markdown"); assert.equal(spec.mcpName, "importPageMarkdown");
assert.equal(spec.inAppKey, "importPageMarkdown"); assert.equal(spec.inAppKey, "importPageMarkdown");
}); });
+2 -2
View File
@@ -13,7 +13,7 @@ test("times a tool and preserves the handler's return value", async () => {
const onMetric = (name, value, labels) => calls.push({ name, value, labels }); const onMetric = (name, value, labels) => calls.push({ name, value, labels });
const handler = async (arg) => ({ ok: true, echo: arg }); const handler = async (arg) => ({ ok: true, echo: arg });
const wrapped = timeToolHandler("get_page", handler, onMetric); const wrapped = timeToolHandler("getPage", handler, onMetric);
const result = await wrapped("hello"); const result = await wrapped("hello");
// Return value passes through untouched. // Return value passes through untouched.
@@ -22,7 +22,7 @@ test("times a tool and preserves the handler's return value", async () => {
// Exactly one sample, correct name/labels, numeric non-negative duration. // Exactly one sample, correct name/labels, numeric non-negative duration.
assert.equal(calls.length, 1); assert.equal(calls.length, 1);
assert.equal(calls[0].name, "mcp_tool_duration_seconds"); assert.equal(calls[0].name, "mcp_tool_duration_seconds");
assert.deepEqual(calls[0].labels, { tool: "get_page" }); assert.deepEqual(calls[0].labels, { tool: "getPage" });
assert.equal(typeof calls[0].value, "number"); assert.equal(typeof calls[0].value, "number");
assert.ok(calls[0].value >= 0, "duration must be non-negative seconds"); assert.ok(calls[0].value >= 0, "duration must be non-negative seconds");
}); });
+161
View File
@@ -137,3 +137,164 @@ test("buildPageTree output shape is lean (drops position/parentPageId/hasChildre
assert.equal("hasChildren" in node, false); assert.equal("hasChildren" in node, false);
assert.equal("spaceId" in node, false); assert.equal("spaceId" in node, false);
}); });
// ---------------------------------------------------------------------------
// #443 getTree output shape: { pageId, title, children?, hasChildren? }
// ---------------------------------------------------------------------------
// A small representative space used across the getTree tests:
// r1 (Infrastructure)
// c1 (Datacenter A)
// g1 (Servers) [leaf]
// c2 (Datacenter B) [leaf]
// r2 (Notes) [leaf]
const SAMPLE = [
{ id: "r2", slugId: "s-r2", title: "Notes", position: "a1", icon: "📝", hasChildren: false },
{ id: "r1", slugId: "s-r1", title: "Infrastructure", position: "a0", icon: "🏢", hasChildren: true },
{ id: "c2", slugId: "s-c2", title: "Datacenter B", position: "b1", parentPageId: "r1", icon: "🅱️", hasChildren: false },
{ id: "c1", slugId: "s-c1", title: "Datacenter A", position: "b0", parentPageId: "r1", icon: "🅰️", hasChildren: true },
{ id: "g1", slugId: "s-g1", title: "Servers", position: "c0", parentPageId: "c1", icon: "🖥️", hasChildren: false },
];
test("getTree shape: correct nesting + order-by-position, only {pageId,title,children?}, no leak", () => {
const tree = buildPageTree(SAMPLE, { shape: "getTree" });
// Roots sorted by position: r1 (a0) before r2 (a1).
assert.deepEqual(
tree.map((n) => n.pageId),
["r1", "r2"],
);
// r1's children sorted by position: c1 (b0) before c2 (b1).
assert.deepEqual(
tree[0].children.map((n) => n.pageId),
["c1", "c2"],
);
// Deep nesting: g1 under c1.
assert.deepEqual(
tree[0].children[0].children.map((n) => n.pageId),
["g1"],
);
// No slugId/icon/position/parentPageId/hasChildren leak on any node.
const walk = (nodes) => {
for (const n of nodes) {
assert.deepEqual(
Object.keys(n).sort(),
n.children ? ["children", "pageId", "title"] : ["pageId", "title"],
`unexpected keys on ${n.pageId}: ${Object.keys(n)}`,
);
assert.equal("slugId" in n, false);
assert.equal("icon" in n, false);
assert.equal("position" in n, false);
assert.equal("parentPageId" in n, false);
// Fully-expanded tree (no maxDepth): hasChildren never set.
assert.equal("hasChildren" in n, false);
if (n.children) walk(n.children);
}
};
walk(tree);
});
test("getTree maxDepth:1 returns roots only, each with hasChildren from the flat item", () => {
const tree = buildPageTree(SAMPLE, { shape: "getTree", maxDepth: 1 });
assert.deepEqual(
tree.map((n) => n.pageId),
["r1", "r2"],
);
// No children arrays at depth 1 when maxDepth:1.
for (const n of tree) assert.equal("children" in n, false);
// r1 has children on the server -> hasChildren:true; r2 is a leaf -> omitted.
assert.equal(tree[0].hasChildren, true);
assert.equal("hasChildren" in tree[1], false);
});
test("getTree maxDepth:2 cuts grandchildren; hasChildren only on the cut interior node", () => {
const tree = buildPageTree(SAMPLE, { shape: "getTree", maxDepth: 2 });
const r1 = tree[0];
// Depth-1 node r1 was EXPANDED (its children are present) -> no hasChildren.
assert.equal("hasChildren" in r1, false);
assert.equal(r1.children.length, 2);
const [c1, c2] = r1.children;
// c1 is at depth 2 (the cut) and has children on the server -> hasChildren:true,
// and its grandchild g1 is NOT present.
assert.equal(c1.pageId, "c1");
assert.equal("children" in c1, false);
assert.equal(c1.hasChildren, true);
// c2 is at depth 2 but is a leaf on the server -> hasChildren omitted.
assert.equal(c2.pageId, "c2");
assert.equal("children" in c2, false);
assert.equal("hasChildren" in c2, false);
// r2 is a depth-1 leaf -> no hasChildren, no children.
assert.equal("hasChildren" in tree[1], false);
});
test("getTree hasChildren is set ONLY on depth-cut nodes (not leaves, not expanded interior nodes)", () => {
// Full tree (no cut): NO node anywhere carries hasChildren.
const full = buildPageTree(SAMPLE, { shape: "getTree" });
const anyHasChildren = (nodes) =>
nodes.some((n) => "hasChildren" in n || (n.children && anyHasChildren(n.children)));
assert.equal(anyHasChildren(full), false);
});
test("getTree orphan (parent filtered out) surfaces as a root, not dropped", () => {
const tree = buildPageTree(
[
{ id: "root", slugId: "s-root", title: "Root", position: "a0", hasChildren: true },
// parentPageId points at an id NOT in the flat list (parent filtered by perms).
{ id: "orphan", slugId: "s-orphan", title: "Orphan", position: "a1", parentPageId: "gone", hasChildren: false },
],
{ shape: "getTree" },
);
assert.deepEqual(
tree.map((n) => n.pageId).sort(),
["orphan", "root"],
);
const orphan = tree.find((n) => n.pageId === "orphan");
assert.equal("children" in orphan, false);
assert.equal("hasChildren" in orphan, false);
});
test("getTree rootPageId path: a seeded single-root subtree keeps the getTree shape", () => {
// Simulate the server seeding the CTE with the subtree root c1: the flat list
// it returns contains c1 (now a root, parent absent) + its descendant g1.
const subtree = [
{ id: "c1", slugId: "s-c1", title: "Datacenter A", position: "b0", hasChildren: true },
{ id: "g1", slugId: "s-g1", title: "Servers", position: "c0", parentPageId: "c1", hasChildren: false },
];
const tree = buildPageTree(subtree, { shape: "getTree" });
assert.equal(tree.length, 1);
assert.equal(tree[0].pageId, "c1");
assert.deepEqual(
tree[0].children.map((n) => n.pageId),
["g1"],
);
assert.equal("slugId" in tree[0], false);
});
test("getTree maxDepth<=0 / non-finite is treated as no cut (whole tree)", () => {
for (const bad of [0, -3, NaN, Infinity, undefined]) {
const tree = buildPageTree(SAMPLE, { shape: "getTree", maxDepth: bad });
// Grandchild g1 present -> no cut applied.
assert.deepEqual(
tree[0].children[0].children.map((n) => n.pageId),
["g1"],
`maxDepth=${bad} should not cut`,
);
}
});
test("buildPageTree() with no options is byte-identical to the historic lean call", () => {
// Guard the existing callers: buildPageTree(pages) must be unchanged by the
// additive options param.
const withoutOpts = buildPageTree(SAMPLE);
const withEmptyOpts = buildPageTree(SAMPLE, {});
assert.deepEqual(withoutOpts, withEmptyOpts);
// And it is the lean {id,slugId,title,children?} shape, not the getTree shape.
assert.deepEqual(Object.keys(withoutOpts[0]).sort(), ["children", "id", "slugId", "title"]);
});
@@ -334,7 +334,7 @@ const DocmostAttributes = Extension.create({
* Docmost inline comment mark. Anchors a comment thread to a text range via * Docmost inline comment mark. Anchors a comment thread to a text range via
* `commentId`. Without it, any document containing comment highlights fails to * `commentId`. Without it, any document containing comment highlights fails to
* round-trip through the schema ("There is no mark type comment in this schema"), * round-trip through the schema ("There is no mark type comment in this schema"),
* which breaks update_page_json and edit_page_text on every commented page. * which breaks updatePageJson and editPageText on every commented page.
* Mirrors Docmost's @docmost/editor-ext comment mark (commentId / resolved). * Mirrors Docmost's @docmost/editor-ext comment mark (commentId / resolved).
*/ */
const Comment = Mark.create({ const Comment = Mark.create({

Some files were not shown because too many files have changed in this diff Show More