Compare commits

..

50 Commits

Author SHA1 Message Date
vvzvlad 512bcba5f3 Merge pull request 'refactor(mcp): распил client.ts (5206 строк) на доменные модули за тонким фасадом (#450)' (#478) from refactor/450-split-client into refactor/449-write-invariants
Reviewed-on: #478
2026-07-11 01:30:28 +03:00
agent_coder 14d7b21df0 refactor(mcp): распил client.ts (5206 строк) на доменные модули за тонким фасадом (#450)
client.ts был god-object'ом на 5206 строк / ~65 методов / 5 ответственностей —
любая правка рисковала всем write-path. Разнесли на доменные модули;
DocmostClient остаётся ТОНКИМ ФАСАДОМ с прежним внешним контрактом. Чистый
рефакторинг, поведение не меняется. closes #450

- Фасад client.ts (93 строки) композирует 10 миксинов над общим абстрактным
  базовым классом. Паттерн МИКСИНЫ (не context-object): тесты субклассируют
  DocmostClient и переопределяют seam'ы; единая цепочка прототипов сохраняет
  виртуальную диспетчеризацию this.<method> сквозь модули.
- client/context.ts (база): общее состояние (axios, apiUrl, токены, кэши),
  конструктор + оба интерсептора, login/ensureAuthenticated/paginateAll/
  resolvePageId/mutateLiveContentUnlocked + write-seam'ы mutatePage/replacePage.
  private→protected (внутреннее, не в публичном контракте).
- Модули (каждый ≤730): read, pages, nodes-write, media (images/attachments/
  drawio), comments, transforms, tables, stash, doc-validate. errors.ts —
  единый REST error-mapping (довершение #437; тексты сообщений без изменений).
- Внешний контракт СОХРАНЁН (доказано компиляцией с обеих сторон): 53 публичных
  метода через `implements IXMixin` (без ручного зеркала, #446); оба Pick
  (in-app loader + tool-specs) резолвятся; множество async-методов 59==59.
  Нагруженные seam'ы (replaceImage один-лок #425, self-resolve #449, единый
  error-путь) байт-идентичны; ни одного дубля публичного метода.
- zod v3→v4: investigate-only, отложено — SDK 1.29 поддерживает v4, но мажор-
  бамп трогает всю схема-поверхность; отдельным follow-up.

Стоит на #449 (#475). Тесты: mcp node --test 800/800 (== база), tsc чисто в
client/. Ни одной строки кода не потеряно; убраны 52 дублированных JSDoc-блока.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 01:04:42 +03:00
agent_coder 9a8671c3af refactor(mcp): структурные инварианты конкурентной записи — UUID-assert в page-lock, self-resolve seams, no-await-guard (#449)
Механика конкурентной записи (withPageLock → acquireCollabSession → mutate)
держалась на цепочке конвенций в комментариях. Новый write-метод без знания
правил компилировался и уходил в прод (класс #260/#152/#159). Закрепляем ключевые
инварианты кодом.

- page-lock.ts: экспортированы UUID_RE/isUuid (тот же regex, что resolvePageId,
  UUID v1–8/v7). withPageLock FAIL-FAST кидает при не-UUID ключе ДО любой работы
  (комментарий-инвариант #260/#449) — забытый resolve/slugId больше не даёт тихую
  потерю сериализации под другим ключом. client.ts импортирует isUuid оттуда
  (убран локальный дубль — resolver и assert не разъедутся).
- mutatePage/replacePage seams стали async и сами вызывают resolvePageId — ключ
  лока/кэша канонический даже если вызывающий забыл (для уже-UUID это cached
  no-op; все 7 текущих вызывающих и так резолвят). replaceImage (один внешний
  лок + mutateLiveContentUnlocked) не тронут, deadlock невозможен.
- collab-session.ts: машинно-проверяемые маркеры MUTATE-CRITICAL-WINDOW
  BEGIN/END вокруг синхронного блока fromYdoc→applyDocToFragment (INVARIANT 1).
  Тест no-await-critical-window читает исходник и краснеет на await/yield в окне
  (проверено нейтером). Случайный await больше не тихо клоббит живые правки.
- Документация осознанной позиции: single-instance/sticky-sessions —
  требование деплоя (Dockerfile + README EN/RU + .env.example), т.к. мьютекс и
  stash — per-process. Окно устаревших прав (кэш-сессия пишет под токеном
  момента connect до MCP_COLLAB_SESSION_MAX_AGE_MS=10мин) — задокументированный
  trade-off в .env.example; push-инвалидации нет (осознанно).

Тесты: page-lock fail-fast (slugId/пусто/non-string → throw; канонический UUID
принят), no-away-guard, обновлённые фикстуры на валидные UUID. #449-специфичные
37/37 зелёные; mcp tsc чисто. closes #449.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 23:51:02 +03:00
vvzvlad bec2156e96 Merge pull request 'recovery: хвост стопки breaking-окна — #413, #415, #443 (3 части), #425 в develop' (#474) from feat/443-get-page-context into develop
Reviewed-on: #474
2026-07-10 23:49:29 +03:00
vvzvlad acc705de19 Merge pull request 'feat(mcp): drawio стадия 3 — семантические тулы fromGraph/fromMermaid/editCells (#425)' (#472) from feat/425-drawio-graph into feat/443-get-page-context
Reviewed-on: #472
2026-07-10 23:45:45 +03:00
vvzvlad a49872444e Merge pull request 'refactor(mcp)!: BREAKING — все имена MCP-тулов snake_case → camelCase (#412)' (#463) from refactor/412-camelcase-tool-names into develop
Reviewed-on: #463
2026-07-10 23:44:42 +03:00
vvzvlad ffc38ea2ca Merge pull request 'fix(mcp): pre-flight size-guard в diffDocs против CPU-DoS на больших правках (#464)' (#465) from fix/464-diffdocs-cpu-guard into develop
Reviewed-on: #465
2026-07-10 23:44:20 +03:00
agent_coder 2a951df096 feat(mcp): drawio стадия 3 — семантические тулы drawioFromGraph/FromMermaid/EditCells (#425)
Сырой XML остаётся escape-hatch, но для 90% случаев модель не видит ни координат,
ни style-строк — весь класс ошибок лейаута и иконок уходит by construction: эти
решения принимает сервер, а не LLM. Стоит на #443, переиспользует конвейер #423 и
shape-index/elkjs/линтер #424.

- drawioFromGraph(pageId, where, graph, direction?, preset?, layout?): граф
  узлов/групп/связей → резолв иконок (shape-index #424; неизвестная → generic по
  kind с подписью, не пустой квадрат), стили из пресета (kind→палитра),
  elkjs-layered с compound-группами, ассемблер XML (линтер-чистый by construction:
  зазоры >=150, прозрачные контейнеры, относительные координаты детей,
  cross-container рёбра parent=1, эскейп меток). Хинты pinned/sameLayerAs/layer и
  layout none/full/incremental — детерминированным post-pass'ом (ELK-констрейнты
  оказались ненадёжны). Пресеты default/dark/colorblind-safe (Okabe-Ito) — данные.
- drawioFromMermaid(pageId, where, mermaid): чистый парсер flowchart (без
  браузера/CLI) → graph → тот же конвейер. Формы/направление/пунктир=async/
  subgraph→группы/цепочки; не-flowchart отвергает внятно.
- drawioEditCells(pageId, node, operations, baseHash): ID-based add/update/delete,
  delete каскадит на детей контейнера и связанные рёбра; baseHash обязателен
  (optimistic lock как drawioUpdate); сентинелы 0/1 от delete защищены.

DoS-границы (LLM-вход): MAX_GRAPH_EDGES=1000/GROUPS=500 в validateGraph (узлы уже
500), mermaid MAX_CHARS=200k/LINES=20k/GROUPS=500, chain 500 — все с быстрым
throw ДО лейаута/ассемблера (ассемблер и маппер вне ELK-таймаута, иначе OOM
воркера). incremental сохраняет неперечисленные существующие ячейки (mergeExisting
Cells) — «добавь узел» не стирает ручную расстановку. sameLayerAs/layer после
снапа раскладываются по перпендикулярной оси с зазором >=150 → 0 quality-warnings;
pinned — точные пользовательские координаты (clamp>=0, могут дать warning, гарантия
«0 by construction» относится к авто-лейауту).

Регистрация: 3 shared-spec на оба хоста (camelCase, execute-in-spec, inlineBoth
Hosts не понадобился), DocmostClientLike/Method += 3, contract, routing-проза
(fromGraph→архитектуры/облака, fromMermaid→стандартные flowchart, raw xml→экзотика).

Тесты: mcp node --test 782/782 (57 новых) — приёмка (15+ узлов/2 вложенные группы/
AWS-иконки→0 lint/0 warnings/иконки резолвятся, hints, incremental без сдвига,
edit_cells update/cascade/stale-baseHash, снапшоты пресетов + colorblind-safe,
mermaid ветвление+subgraph) + регрессии на DoS-границы. tsc чисто; server jest
(contract + ai-chat) 290/290. closes #425.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 23:16:50 +03:00
agent_coder 5dc7a2703f feat(mcp): getPageContext — «где я / что вокруг» по pageId одним вызовом (#443, часть 3/3)
Финальная из трёх частей #443. Есть pageId (из поиска, из ссылки) — нужно понять
местоположение и окружение. Раньше это get_page + полное дерево. Только чтение,
только метаданные (без контента). closes #443.

- client.ts getPageContext(pageId): resolvePageId (slugId→uuid, для uuid без
  round-trip) → POST /pages/breadcrumbs + listSidebarPages(spaceId, uuid). Два
  запроса для uuid-входа (spaceId берётся из ответа breadcrumbs), +1 для slugId.
  Выход {page:{pageId,title,spaceId}, breadcrumbs:[{pageId,title}] (root→parent;
  [] для корня), children:[{pageId,title,hasChildren}]}.
- Порядок breadcrumbs выверен по серверному источнику (getPageBreadCrumbs: CTE
  сидится на childPageId, идёт вверх, .reverse() → root→page; страница —
  ПОСЛЕДНИЙ элемент). page = chain[last], breadcrumbs = chain.slice(0,-1); корень
  → []. Проекция явная — наружу только pageId (UUID), slugId/icon/position не
  утекают. Пустой chain / 404 / 403 → явная ошибка, не пустой объект.
- children — прямые дети через listSidebarPages (cursor-пагинация #442/#451,
  страница с >20 детьми отдаёт всех без дублей, порядок по position, hasChildren).
- Общий реестр: getPageContext на ОБОИХ хостах через цикл спеков (mcpName===
  inAppKey==="getPageContext", camelCase); DocmostClientLike/DocmostClientMethod
  += it; compile-time contract; TOOL_FAMILY READ + routing-проза.

Тесты: get-page-context.test.mjs +6 (3-й уровень split, no-slugId-leak + 2
запроса, корень→[], slugId-resolve, >20 детей без cap/дублей, плохой id→ошибка,
пустой chain→throw), tool-specs +1. mcp node --test 725/725, tsc чисто; server
jest (contract + ai-chat) 265/265.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 22:19:25 +03:00
agent_coder bfb4c8d8d0 feat(mcp): getTree — иерархия пространства/поддерева одним вызовом (#443, часть 2/3)
Раньше единственный способ получить дерево — listPages tree:true: BFS по
sidebar-эндпоинту, десятки-сотни HTTP-вызовов и молчаливая потеря страниц
(#442). Бэкенд форка уже отдаёт всё одним POST /pages/tree (getSidebarPagesTree,
merged #442/#451), так что это MCP-сторона + общий реестр. Только чтение.

- lib/tree.ts: buildPageTree(nodes) аддитивно расширен до buildPageTree(nodes,
  options?) с {shape?: "lean"|"getTree"; maxDepth?}. Дефолт/{} — байт-идентичный
  lean {id,slugId,title,children?} (существующие вызыватели listPages tree:true
  и subtree-BFS не тронуты, есть регрессионный тест на точный набор полей).
  shape:"getTree" проецирует {pageId, title, children?, hasChildren?} (id→pageId;
  slugId/icon/position/parentPageId не утекают ни на одной глубине).
- client.ts: getTree(spaceId, rootPageId?, maxDepth?) — тот же код-путь, что
  listPages tree:true: один enumerateSpacePages(spaceId, rootPageId) (единичный
  /pages/tree + cursor-BFS фолбэк для stock upstream) → buildPageTree(pages,
  {shape:"getTree", maxDepth}). listPages tree:true помечен DEPRECATED в JSDoc
  (BFS-фолбэк на месте, поведение не тронуто).
- maxDepth/hasChildren: полное дерево строится, потом обрезается. Корни = глубина
  1; maxDepth:1 → только корни; hasChildren:true ТОЛЬКО на срезанном узле с
  серверным hasChildren (source of truth), опущен у листьев и раскрытых узлов.
  Схема тула min(1) отбраковывает 0/отрицательные.
- tool-specs.ts: shared-спек getTree (оба хоста через цикл реестра),
  DocmostClientLike Pick += getTree, listPages desc/catalogLine — про депрекацию.
  server-instructions.ts: getTree:"READ" + routing-проза. Loader
  DocmostClientMethod += getTree, compile-time client-call contract += getTree.

Циклы/self-ref безопасны: project рекурсирует только от корней, циклический
компонент недостижим из корней (нет бесконечной рекурсии). Orphan (родитель
отфильтрован пермишенами) всплывает как корень.

Тесты: tree.test.mjs +9 (nesting+порядок+no-leak, maxDepth:1/2 + hasChildren,
orphan→root, seeded rootPageId, ≤0/нефинитный = без среза, lean-байт-идентичность),
tool-specs.test.mjs (схема getTree + депрекация listPages). mcp node --test
718/718, tsc чисто; server jest (shared-tool-specs.contract + ai-chat) 263/263.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 21:58:32 +03:00
agent_coder f794ac6d6c fix(search): оживить trgm-индекс — убрать coalesce из LIKE-предикатов (#443, ревью)
Ревьюер поймал EXPLAIN'ом: lookup-query фильтровал по
LOWER(f_unaccent(coalesce(col,''))), а GIN-trgm-индексы — на coalesce-free
LOWER(f_unaccent(col)). Postgres берёт функциональный индекс только при точном
совпадении выражения, так что обёртка coalesce отключала индекс → Seq Scan по
pages на КАЖДЫЙ lookup (MCP-клиент всегда шлёт substring:true). Зелёный
int-гейт это не ловил (проверял корректность на крошечном датасете, не
использование индекса).

- search.service.ts: убрал coalesce из двух index-driving LIKE-предикатов
  (title + text_content). Семантически эквивалентно: NULL LIKE '%q%' = NULL
  (falsy), NULL-страница просто не матчится — как пустая строка не матчит %q%.
  SELECT/ORDER BY оставлены с coalesce (индекс не выбирают, Node гардит NULL).
- Миграция: убран избыточный ре-ассерт idx_pages_title_trgm — его создаёт #348
  на том же coalesce-free выражении, теперь title-предикат его использует.
  idx_pages_text_content_trgm без изменений (уже coalesce-free), поправлены
  вводящие в заблуждение комментарии. down() дропает только text-индекс.
- Новый тест search-lookup-explain.int-spec.ts (3): title→idx_pages_title_trgm,
  text→idx_pages_text_content_trgm (оба ассертят отсутствие Seq Scan on pages),
  + негативный контроль (coalesce-обёртка не может использовать индекс) против
  тихой ре-регрессии. Дискриминатор enable_seqscan=off.
- CHANGELOG: две записи (opt-in substring lookup mode + смена shape MCP search).

EXPLAIN на реальном PG: обе fixed-ветки → Bitmap Index Scan on trgm; buggy
coalesce → Seq Scan (Disabled:true). Гейт: server jest src/core/search 27/27
(16 исходных int без изменений + 3 EXPLAIN + 8 unit), mcp node --test 708/708,
tsc чисто.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 21:57:16 +03:00
agent_coder e4e788f151 feat(search): агентский lookup-режим — substring по точкам/дефисам/цифрам, path, snippet, scope (#443, часть 1/3)
MCP-сервером пользуются LLM-агенты; основной паттерн — lookup: найти страницу по
обрывку технической строки и сразу понять, где она лежит и что в ней. Раньше на
такой вопрос уходило 3–5 вызовов. Эта часть закрывает `search` (часть 1 из 3;
get_tree и get_page_context — следующими PR стопкой). Только чтение.

Серверная часть — opt-in, веб-UI байт-в-байт неизменен:
- SearchDTO: опциональные substring/parentPageId/titleOnly (default-off; без
  флагов путь FTS не тронут — guard `if (substring) return searchPageLookup`).
- searchPageLookup: один скан pages, WHERE = title LIKE '%q%' OR (если не
  titleOnly) text LIKE '%q%' OR (если tsquery непустой) tsv @@ ... . LIKE-
  метасимволы %/_/\ экранируются (escapeLikePattern, ESCAPE '\') — `%`/`_` не
  матчат всё; substring-ветка работает даже при пустом tsquery (кейс 10.0.12).
- Ранжирование тирами (TITLE_EXACT > TITLE_SUBSTRING > TEXT), вторичный сигнал
  ts_rank / позиция; score∈(0,1] только для сортировки одной выдачи (формула в
  комментарии). 200-cap упорядочен по SQL-прокси тира ДО среза (иначе Postgres
  отдаёт произвольные 200 и сильный хит мог выпасть). Пермишен-фильтр к
  merged-набору ДО limit. path — одна рекурсивная CTE на все хиты (не N+1).
- snippet оконный в SQL (~500 символов вокруг первого совпадения). Позиция и
  срез в ОДНОМ пространстве LOWER(f_unaccent(...)) — f_unaccent не length-
  preserving (ß→ss, лигатуры, …→...), иначе окно смещалось/пустело. titleOnly →
  пустой snippet. Компромисс задокументирован.
- Миграция: GIN gin_trgm_ops по LOWER(f_unaccent(text_content)); title-trgm
  индекс #348 переиспользован (IF NOT EXISTS), down() дропает только новый.

MCP: схема search (spaceId/parentPageId/titleOnly/limit 1–50, default 10),
client.search прокидывает substring:true, filterSearchResult → {pageId, title,
path, snippet, score}. Инвариант: наружу только pageId (UUID), slugId/id
никогда. Комментарии про намеренное расхождение с in-app hybrid-RRF (не тронут)
и про деградацию на stock-upstream/Typesense (substring→plain FTS, без path/
snippet).

Проверка на реальном Postgres: server integration 16/16 (вся acceptance-таблица
#443 + регрессии на смещение snippet и cap-200), server unit 27/27, mcp
node --test 708/708, tsc чисто.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 21:34:53 +03:00
agent_coder 4be4a75fa3 docs(mcp): точные описания потерь getPage/exportPageMarkdown — конвертер канонический (#415)
Описания двух тулов устарели после #345 (канонический конвертер, #293/#351):
«LOSSY…approximated» у getPage и «lossless» у exportPageMarkdown искажали
роутинг агентов и молчаливо скрывали реальную потерю данных. Четвёртый линк
breaking-окна, стоит на #413.

- tool-specs.ts: getPage — вместо «LOSSY…approximated» точный закрытый список
  потерь (canonical for text; теряет лишь id блоков, resolved-якоря
  комментариев и фиксированный набор ACCEPTED-атрибутов без markdown-
  представления: спаны/colwidth/фон ячеек таблиц, indent, callout.icon,
  orderedList.type, link internal/target/rel/class). exportPageMarkdown —
  убрано «lossless»: round-trip перегенерирует id блоков и молча отбрасывает
  тот же набор (в первую очередь merge-спаны ячеек); держать в page-JSON, если
  нужны. Список выведен из источника истины (ATTR_VALUE_FUZZ_ALLOWLIST +
  MARK_ATTR_ALLOWLIST); opaque carried-verbatim токены (attachmentId, mime,
  slugId и пр.) намеренно НЕ в списке потерь — они round-trip'ятся.
- server-instructions.ts: READ-строка ROUTING_PROSE приведена к тому же
  точному списку, без противоречия markdown-default роутингу #413.
- Исправлена вторая латентная неточность: getPage описывался как
  сохраняющий resolved-якоря — но client.ts передаёт dropResolvedCommentAnchors:
  true, resolved-якоря скрыты (getNode, наоборот, их сохраняет — другой тул).
- README пакета (EN + RU-зеркало) приведены в соответствие passage-for-passage:
  убраны безоговорочные «lossless/lossy» для Markdown round-trip; genuinely-
  lossless ссылки на raw-JSON (getPageJson/getNode) оставлены.

Логику не трогает. server-instructions.test.mjs удалён и заменён
tool-inventory.test.mjs (без пинов на старую формулировку). mcp tsc чисто,
node --test 702/702, tool-inventory 5/5.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 20:55:28 +03:00
agent_coder e6171a1810 fix(mcp): сходимость orphan-сноски + дедуп block-id в markdown-splice (#413, ревью)
Правки по внутреннему ревью #413.

1. Orphan-сноска (нарушение «no second canon»). mergeFootnoteDefinitions
   раньше делал ранний return при пустых definitions, пропуская
   canonicalizeFootnotes. Если markdown-фрагмент БЕЗ сносок заменял/удалял
   блок, бывший последним referrer'ом существующей сноски, её определение
   оставалось orphan в хвостовом списке (полный ре-импорт его бы убрал).
   Теперь fast-path (возврат doc по ссылке без clone) только когда
   defs.length===0 И !hasFootnoteArtifacts(doc); иначе клон → append (no-op
   при пустых) → normalizeAndMergeFootnotes → canonicalizeFootnotes, как при
   полном импорте. Новый предикат hasFootnoteArtifacts обходит дерево на
   любой footnotesList/footnoteReference (существующие walk/isObject).
   Идемпотентно: несвязанный plain-патч на странице со сносками не трогает
   их топологию (canonicalizeFootnotes шаг 6 возвращает как есть).

2. Block-id disjoint от страницы. Новый экспорт reassignCollidingBlockIds
   (liveDoc, blocks, skipIndex?) в prosemirror-markdown/node-ops (переиспользует
   collectIds/makeFreshId) + barrel. Зовётся перед сплайсами в client.ts:
   patch — (liveDoc, threaded, 0) (skip 0 = блок, унаследовавший id цели);
   insert — (liveDoc, blocks) без skip. used-аккумулятор ловит и коллизию со
   страницей, и внутрифрагментную. freshBlockId/freshId без изменений.

Тесты (+6, markdown-patch-insert): orphan-repro (0 defs/0 list/0 refs +
docsCanonicallyEqual полному импорту), fast-path (footnote-free patch не трогает
топологию, соседи byte-identical), insert канонизирует при пустых defs со
страничной сноской, уникальность top-level block-id для patch 1→N и insert N.
mcp node --test 702/702, pmd vitest 736, tsc чисто.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 20:38:06 +03:00
agent_coder 2f23cf4b65 fix(mcp): pre-flight size-guard в diffDocs против CPU-DoS на больших правках (#464)
Боевой инцидент: diffDocs синхронно зовёт recreateTransform (rfc6902), чей
array-diff — O(n·m) по числу нод и O(w²) по длине прогонов слов, и который
НИКОГДА не бросает. Поэтому существующий catch→coarseDiff не спасает event loop:
большая агентская правка блокирует воркер на секунды→минуты (бенч: 401 нода→1.3с,
801→5.5с, 1601→OOM; отдельная ось — байт-тяжёлый вход: 309КБ→OOM), а это душит
все остальные запросы воркера. Отсюда живой прод-хэнг.

Фикс — pre-flight size-guard ПЕРЕД дорогим вызовом:
- readPositiveIntEnv(name, dflt): парс env каждый вызов; мусор/пусто/≤0/не-finite →
  дефолт, так что guard нельзя случайно отключить.
- exceedsDiffSizeGuard(old, new): срабатывает по max(old,new) на ОБЕИХ осях —
  countNodes и JSON.stringify().length — так что асимметричная пара (крошечный
  new / огромный old и наоборот) тоже триггерит. Дёшево: один обход + один
  stringify на документ.
- Дефолты MCP_DIFF_MAX_NODES=150, MCP_DIFF_MAX_BYTES=12288 (12 КиБ),
  env-переопределяемы. Подобраны бенчмарком под бюджет ~200мс на ЛЮБОЙ форме
  входа на границе cap'а (исходные догадки тикета 3000-5000 нод / 512КБ-1МБ были
  в 20-30× завышены — пропускали бы многоминутные блоки).
- Рефактор: выделены coarseDiffTally (единый источник формы fellBack:true) и
  preciseDiffTally. Новый поток diffDocs: computeIntegrity (без изменений, для
  обоих путей) → если exceedsDiffSizeGuard → coarseDiffTally(fellBack=true) →
  иначе try preciseDiffTally / catch → coarseDiffTally. Обе деградации дают
  идентичную форму результата.

Fail-closed: единственный путь к recreateTransform — ветка else, достижимая
только когда guard НЕ сработал; огромный документ физически до transform не
доходит. computeIntegrity (корректностный сигнал) считается ВСЕГДА. Все три
call-site потребляют DiffResult как информационный preview, не гейтят запись —
coarse-фоллбэк контракт-сохраняющий.

Тесты (11): over/under-порог, обе асимметрии, байт-ось (нод-мало/байт-много),
env-override низким cap'ом, garbage-env→дефолт (guard не отключается), integrity
корректен на tripped-документе; behavioral-proxy что transform пропущен над
cap'ом (guarded ~5мс vs caps-raised ~3.3с, ≥5× + <200мс бюджет). node --test
688/688. tsc --noEmit чисто. Только packages/mcp, внешний симлинк не тронут.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 20:32:50 +03:00
agent_coder d269bd9efe feat(mcp): markdown — формат по умолчанию для блочных getNode/patchNode/insertNode (#413)
Канонический конвертер (#293/#345/#351) зрел для блочного уровня: markdown
становится дефолтом чтения/записи ОДНОГО блока, PM JSON — опция «для тонких
работ». Новых тулов нет, поверхность не растёт. Имена уже camelCase (стоит на #412).

- getNode(pageId, nodeId, format='markdown'): дефолт markdown — обёртка
  {type:doc,content:[node]} → convertProseMirrorToMarkdown, comment-якоря
  (ВКЛЮЧАЯ resolved) сохранены (это чтение под редактирование, не getPage).
  format:'json' — сырой сабтри. Авто-фолбэк не-топ-левел типов (tableRow/Cell/
  Header по #<index>) в JSON через canBeDocChild = docmostSchema.nodes.doc.
  contentMatch.matchType (не рукописный список); поле format на каждом ответе.
- patchNode/insertNode: XOR-вход {markdown?|node?} (оба optional в схеме, XOR
  на рантайме). markdown → импорт фрагмента → 1→N сплайс: первый блок наследует
  id цели, остальные свежие; dry replaceNodeById для #159-ambiguity ДО сплайса;
  соседние блоки byte-identical. Guard findUnrepresentableTableAttrs: цель со
  span/colwidth/backgroundColor → отказ с указанием на table-тулы/node-JSON.
  insertNode — insertNodesRelative (N блоков по порядку); голый tableRow/Cell
  JSON-only.
- Сноски: ^[...] во фрагменте → канон-импортёр; importMarkdownFragment делит
  блоки от footnotesList, РЕМАПИТ id сносок фрагмента в свежие uuid (fn-1
  фрагмента не коллизит с fn-1 страницы), mergeFootnoteDefinitions добавляет
  через ту же машинерию appendDefinition→normalizeAndMergeFootnotes→
  canonicalizeFootnotes, что insertFootnote. Сырые JSON-пути не тронуты.
- node-ops: replaceNodeByIdWithMany/insertNodesRelative (сплайс массива) в
  prosemirror-markdown/node-ops (канон после #414) + barrel.
- ROUTING_PROSE (READ/EDIT), compile-time client-call contract, CHANGELOG
  (getNode default→markdown, breaking для внешних клиентов, в окне #411/#412).

Тесты: сходимость (patchNode(markdown) блок docsCanonicallyEqual полному
импорту — нет «второго канона»); id-нить 1→N (первый наследует, остальные
свежие, соседи byte-identical); XOR; getNode markdown/json/non-top-level-fallback;
сохранение comment-якорей (active+resolved); ^[...]→хвостовой список+перенумерация;
guard. mcp node --test 697/697; pmd vitest 736; tsc чисто; server jest 273.
Третий линк breaking-окна, стоит на #412 (#411→#412→ЭТОТ→#415).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 20:10:38 +03:00
agent_vscode d287c15db4 fix(ci,ai-chat): восстановить зелёный CI на develop
Чинит два падения прогона Develop (#29104398390):

A. Интеграционные тесты сервера (10 failed) падали с
   `TypeError: this.environment.isAiChatFinalStepLockdownEnabled is not a function`.
   Мерж #444 добавил вызов этого метода в AiChatService.stream, но два
   интеграционных фикстура не обновили env-стаб. Добавлен
   `isAiChatFinalStepLockdownEnabled: () => false` в 4 стаба
   (ai-chat-attach.int-spec.ts, ai-chat-stream.int-spec.ts).

B. Job e2e-server не компилировал app.e2e-spec.ts:
   `TS2307: Cannot find module '@docmost/mcp'`. Рефактор #446 добавил
   тип-импорт из @docmost/mcp в docmost-client.loader.ts, но job не
   собирал этот пакет (его build/ в gitignore). Добавлен шаг
   `Build mcp` в e2e-server (по образцу e2e-mcp / mcp-server-parity).

Только тесты и CI-конфиг; продакшн-логика не менялась.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 19:42:16 +03:00
agent_coder 7cb3199d09 refactor(mcp)!: BREAKING — все имена MCP-тулов snake_case → camelCase (унификация с in-app) (#412)
Один логический тул жил под двумя именами: внешний MCP snake_case
(edit_page_text), in-app camelCase (editPageText) — дублирование доков, путаница
при переносе промптов/скиллов, помеха шарингу спек (#294). Решение владельца:
единый camelCase везде, включая внешний MCP. После этого mcpName === inAppKey.

- tool-specs.ts: mcpName ВЫВЕДЕН из ключа спеки (mcpName == inAppKey) для всех
  43 shared-спек — раньше divergent snake, теперь равен ключу (проверено: mcpName
  читается только структурно — цикл регистрации, генератор <tool_inventory>,
  TOOL_FAMILY). +5 inline-регистраций (tableGet/updateComment/deleteComment/
  docmostTransform; search без изменений). Рантайм: 47 тулов, все camelCase, ноль
  подчёркиваний.
- Контракт-конвенция ИНВЕРТИРОВАНА: shared-tool-specs.contract.spec
  `mcpName === toSnake(inAppKey)` → `mcpName === inAppKey`; tool-specs.test
  и tool-inventory.test обновлены.
- ROUTING_PROSE/TOOL_FAMILY/INLINE_MCP_INVENTORY (server-instructions.ts) →
  camelCase (105 замен). ai-chat.prompt/guard уже на in-app camelCase-ключах —
  без изменений (guard прошёл). comment-signal EXCLUDED_TOOLS схлопнут с
  дублей snake+camel до camelCase.
- Некоторое неочевидное: assertUnambiguousMatch(op: "patch_node"|"delete_node")
  в prosemirror-markdown/node-ops — op интерполируется в model-facing ошибку;
  литерал-юнион + call-sites → "patchNode"|"deleteNode".
- Все snake-имена в описаниях/error-строках/комментах/тестах/доках → camelCase
  (whole-token, longest-match-first). CHANGELOG: BREAKING-таблица 46 строк +
  миграция (allowlists mcp__gitmost-*__get_node→__getNode, промпты/скиллы,
  .mcp.json, метрики по tool-label); релизится вместе с #411.
  Внутренние имена методов (PageService.updatePageContent и т.п.) НЕ тронуты —
  переименованы только ИМЕНА ТУЛОВ.

Гейт: mcp node --test 677/677; tsc -p apps/server чисто; jest ai-chat-tools.
service + shared-tool-specs.contract + tool-tiers + ai-chat.prompt +
comment-signal-inapp → 323. Второй линк breaking-окна (#411→ЭТОТ→#413→#415).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 19:10:38 +03:00
vvzvlad e19275e96e Merge pull request 'refactor(tools): updatePageContent → updatePageMarkdown; −import_page_markdown с MCP (#411)' (#462) from refactor/411-update-page-markdown into develop
Reviewed-on: #462
2026-07-10 18:36:44 +03:00
agent_coder 3a521ada4d refactor(tools): updatePageContent → updatePageMarkdown; внешний MCP: +updatePageMarkdown, −import_page_markdown (#411)
Поверхности записи «целым телом» были несимметричны: у in-app агента полная
замена тела markdown называлась updatePageContent (имя не про формат, тогда как
парный updatePageJson — про JSON), а у внешнего MCP голого plain-body-replace
не было вовсе (только import_page_markdown — на деле парсер round-trip к
export_page_markdown, не plain-replace). Пара должна быть updatePageMarkdown /
updatePageJson.

Пост-Фаза-1б архитектура (реестр + циклы по обоим хостам):
- новая shared-спека updatePageMarkdown (mcpName update_page_markdown, inAppKey
  updatePageMarkdown, tier как у updatePageJson) с execute (client, {pageId,
  content, title}) => client.updatePage(...) — тот же путь updatePageContentRealtime
  → markdownToProseMirrorCanonical, ^[...]-сноски парсятся. Реестровый цикл
  регистрирует её на ОБОИХ хостах автоматически. Добавлен 'updatePage' в
  Pick DocmostClientLike.
- import_page_markdown убран с внешнего MCP через inAppOnly:true у спеки
  importPageMarkdown — MCP-цикл и генератор инвентаря её пропускают, in-app
  агент сохраняет importPageMarkdown; спека и client-метод НЕ удалены.
- удалён inline in-app updatePageContent tool (теперь из реестра под inAppKey
  updatePageMarkdown) + его INLINE_TOOL_TIERS-энтри.
- ROUTING_PROSE: bulk-rewrite ссылается на update_page_markdown|update_page_json;
  убрано упоминание import_page_markdown; инвентарь генерируется из catalogLine.
- лейбл-мапы chat-markdown.util (en/ru), человекочитаемые метки не тронуты.
- НЕ тронуты одноимённые внутренности: PageService.updatePageContent,
  updatePageContentRealtime, collaboration.handler — переименовано только имя тула.

Тесты: updatePageMarkdown на обеих поверхностях с идентичной схемой, forward в
client.updatePage; import_page_markdown ОТСУТСТВУЕТ на MCP, присутствует in-app;
^[...]→сноски покрыт через collaboration.test. CHANGELOG BREAKING + миграция;
README/README.ru пакета обновлены. Гейт: mcp node --test 646/646, server jest
259, tsc чисто. Первый линк breaking-окна #416 (#411→#412→#413→#415).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 18:33:20 +03:00
vvzvlad 576db3c8f9 Merge pull request 'feat(mcp): drawio стадия 2 — guide, каталог фигур, ELK-лейаут, quality-warnings (#424)' (#440) from feat/424-drawio-rules into develop
Reviewed-on: #440
2026-07-10 18:29:46 +03:00
agent_coder ddb37376a4 refactor(mcp): вписать drawio-тулы в реестровый цикл (Фаза 1б смержена) (#440)
Фаза 1б (#445/#446/#447/#448) влилась ПЕРЕД этим PR, поэтому явная проводка
drawio через registerShared/sharedTool конфликтовала с реестровыми циклами.
Фолд:
- drawioGet/Create/Update → канонический execute в спеке (это client-методы):
  execute возвращает сырой результат, цикл оборачивает jsonContent на MCP и
  отдаёт как есть in-app — байт-в-байт как старые тела, overrides не нужны.
  layout сохранён: добавлен в buildShape create/update + 5-м аргументом
  (layout as 'elk'|undefined) в обоих execute.
- drawioShapes/drawioGuide → остаются inline на ОБОИХ хостах. Гайд архитектора
  «execute в спеке с импортом searchShapes/getGuideSection в tool-specs.ts»
  оказался невозможен: drawio-shapes.ts использует import.meta.url, а
  tool-specs.ts тайпчекается из исходника под module:commonjs (contract-спека)
  → TS1343 на статический value-import, а import.meta не индиректится. Введён
  флаг спеки inlineBothHosts: спеки остаются в SHARED_TOOL_SPECS (contract
  пинит имя/описание/схему), но без execute; ОБА цикла их пропускают (добавлен
  симметричный guard в MCP-цикл), каждый хост регистрирует их inline через
  чистые хелперы — поведение байт-в-байт как до ребейза.
- docmost-client.loader: взята develop-форма DocmostClientLike = Pick<
  DocmostClient> (#446); ручное зеркало убрано, паритет layout наследуется из
  реальной сигнатуры client.
- ROUTING_PROSE: drawio intent-подсказки (shapes-first, guide, layout:elk);
  инвентарные строки убраны (генерируются из catalogLine).

Гейт: mcp node --test 674/674; server jest ai-chat-tools.service + contract
(211, все 5 drawio на обоих хостах, идентичная схема) + tool-tiers → 264;
tsc чисто; layout-passthrough тест 3/3. ELK DoS-кап и layout-фикс из round 3/4
сохранены.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 16:34:54 +03:00
agent_coder dc58974b31 fix(ai-chat): пробросить layout:elk в in-app drawioCreate/drawioUpdate — паритет с MCP (ревью #440)
In-app хендлеры drawio_create/drawio_update деструктурировали args БЕЗ layout и
не передавали его клиенту 5-м аргументом → layout:"elk" (схема его принимает —
общий buildShape) ТИХО терялся, ELK-автолейаут работал только по MCP-хосту.
Корень: ручное зеркало DocmostClientLike (loader) отстало от реального client.ts
— у его drawioCreate/drawioUpdate не было параметра layout (то, что #446 чинит
деривацией типа, но #446 ещё не влит). Добавил layout?:'elk' в обе сигнатуры
зеркала + проброс в обоих хендлерах.

Тест (пропущенный зелёным гейтом пробел — не было теста на in-app passthrough):
in-app drawioCreate/drawioUpdate с layout:'elk' → фейк-клиент получает layout
5-м позиционным аргументом; omit-кейс → undefined. Мутационно: убрать проброс
в drawioCreate → layout-create-тест краснеет.

Гейт: mcp build чисто; tsc -p apps/server без новых ошибок; jest
ai-chat-tools.service (35) + shared-tool-specs.contract + tool-tiers +
comment-signal-inapp → 273 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 16:16:19 +03:00
agent_coder c917dcc3c1 fix(mcp): ограничить ELK-лейаут (кап узлов/рёбер + таймаут) — untrusted-граф DoS (ревью #440)
applyElkLayout крутит elkjs СИНХРОННО в процессе на mxGraph-XML от LLM
(layout:'elk' в drawio_create/update) без лимита размера и без таймаута —
большой граф (тысячи узлов, ~1МБ XML проходит stage-1 cap 16МБ) блокирует
event-loop MCP-сервера на секунды-минуты. try/catch ловил только брошенную
ошибку, но не зависание.

- кап ДО построения графа: >500 узлов или >1000 рёбер → вернуть исходный XML
  (best-effort, как существующий catch); синхронный elkjs → кап и есть
  реальная защита;
- Promise.race с 5s-таймаутом (defense-in-depth на случай async-elkjs); таймер
  гасится в finally → нет утечки хендла и unhandled-rejection (проигравший
  timeout остаётся pending с погашенным таймером);
- тест: 600-узловой граф возвращается без изменений и быстро (<2s) — кап-путь.

79/79 drawio-тестов зелёные.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 16:15:06 +03:00
agent_coder eddc3b5c33 feat(ai-chat): пробросить drawio_shapes/drawio_guide in-app — восстановить SHARED_TOOL_SPECS-паритет (#424)
Стадия-1 (#434) уже была довяжена in-app в develop (f46d89ea, agent_vscode)
для CRUD-тулов; два новых чистых read-only хелпера стадии-2 остались
незаброшенными → contract-parity спека падала 6 ассертами (по 3 на
drawio_shapes/drawio_guide). В отличие от CRUD-тулов это ЧИСТЫЕ функции без
сетевого вызова, поэтому НЕ client-методы:

- реэкспорт searchShapes / getGuideSection (+ тип SearchShapesOptions) из
  entry пакета @docmost/mcp; loadDocmostMcp() пробрасывает их так же, как
  sharedToolSpecs (типы SearchShapesFn/GetGuideSectionFn);
- две записи sharedTool(...) в forUser() после drawioUpdate: drawioShapes
  повторяет серверный вызов searchShapes(query,{category,limit}) и форму
  { query, count, results }; drawioGuide — getGuideSection(section)
  (omit section -> index); голый объект без jsonContent-envelope, как у
  соседних in-app хендлеров;
- DocmostClientLike и HOST_CONTRACT_METHODS-вайтлист НЕ тронуты (это не
  методы клиента);
- три тест-мока (contract/service/tool-tiers) получили type-only no-op
  заглушки под расширенный тип loadDocmostMcp() — тела инструментов в этих
  тестах не исполняются, contract-спека реально гоняет настоящий
  SHARED_TOOL_SPECS.

Внутреннее ревью обвязки: APPROVE, 0 находок. shared-tool-specs.contract:
211/211 (было 6 падений); client-host-contract drift-guard 3/0; tsc EXIT 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 16:15:06 +03:00
agent_coder e454fe189c feat(mcp): drawio стадия 2 — правила качества, каталог фигур, guide, ELK-лейаут, warnings (#424)
Надстройка над стадией-1 (сырой mxGraph XML) — помогает агенту рисовать
корректные диаграммы без бэкенд-рендеринга:

- hard-rules в описаниях drawio_create/drawio_update (геометрия, parent-
  relative координаты, стили);
- drawio_guide (5 секций: skeleton / layout / containers / icons-aws /
  icons-azure, каждая ≤4KB) — по требованию, не раздувает контекст;
- drawio_shapes — реальный jgraph shape-index (10446 фигур, gzip 437KB,
  ленивый node:zlib gunzip) + курируемый оверлей (service-level паттерны
  AWS/Azure, note-подсказки на пустые resIcon, палитра категорий);
  ранжирование aws4>aws3; escapeRe в score (не ReDoS);
- layout:"elk" через elkjs (чистый JS, dependencies:{}) — compound-nesting,
  best-effort (на сбое ELK возвращает нормализованный вход), 73→0 warnings
  на 12-узловом графе;
- 6 типов quality-warnings в линтере (overlap, out-of-bounds, edge-cross,
  и т.п.), геометрия Liang-Barsky; warnings НИКОГДА не блокируют write.

Оба новых инструмента в SHARED_TOOL_SPECS (tier:deferred) + SERVER_
INSTRUCTIONS; drift-guards зелёные. elkjs ^0.11.1 — единственный новый
рантайм-деп; lockfile синхронизирован (--frozen-lockfile --offline EXIT 0).
data/ едет с воркспейсом (.gitignore-негация !packages/mcp/data/).

Внутренний цикл: 1 проход внутреннего ревью (APPROVE WITH SUGGESTIONS);
все 59 профильных тестов зелёные.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 16:12:37 +03:00
vvzvlad ea99d4fe63 Merge pull request 'feat(mcp): внятная диагностика ошибок тулов + fail-fast валидация comment-id (#437, #436)' (#441) from feat/437-error-diagnostics into develop
Reviewed-on: #441
2026-07-10 16:03:44 +03:00
vvzvlad 23966ce51c Merge pull request 'fix(ai-chat): «send now» во время detached-run — серверный stop + ограниченный ретрай 409 (#396)' (#456) from fix/396-sendnow-detached-run into develop
Reviewed-on: #456
2026-07-10 16:03:18 +03:00
vvzvlad a53b2f454e Merge pull request 'fix(delivery): immutable-кэш ассетов — отключить дефолтный cacheControl @fastify/static (#452)' (#455) from fix/452-immutable-cache-control into develop
Reviewed-on: #455
2026-07-10 16:03:03 +03:00
vvzvlad d219eb7525 Merge pull request 'fix(ai-chat): защита от петель агента — lockdown под тоггл + детектор деградации + бюджет шагов (#444)' (#454) from fix/444-agent-loop-guards into develop
Reviewed-on: #454
2026-07-10 16:02:48 +03:00
vvzvlad 93d244478e Merge pull request 'refactor(tools): генерировать инвентарь SERVER_INSTRUCTIONS из реестра + guard имён тулов (#448)' (#460) from refactor/448-generate-inventory into develop
Reviewed-on: #460
2026-07-10 16:02:17 +03:00
vvzvlad 791f709c18 Merge pull request 'refactor(tools): execute-маппинг в SHARED_TOOL_SPECS + автопроводка обоих хостов (#445)' (#459) from refactor/445-execute-mapping into develop
Reviewed-on: #459
2026-07-10 16:02:01 +03:00
vvzvlad f8a27cba91 Merge pull request 'refactor(mcp): вывести DocmostClientLike/SharedToolSpec из реального типа — убить ручные зеркала (#446)' (#458) from refactor/446-derive-client-types into develop
Reviewed-on: #458
2026-07-10 16:01:48 +03:00
vvzvlad 61dc9b50c1 Merge pull request 'fix(ci,mcp): REGISTRY_STAMP в билде + кросс-пакетный CI — закрыть skew build/vs/src (#447)' (#457) from fix/447-registry-stamp-ci into develop
Reviewed-on: #457
2026-07-10 16:01:38 +03:00
vvzvlad cebb1cca87 Merge pull request 'fix(mcp): pre-validate node JSON против схемы + путь битого узла (#409)' (#461) from fix/409-invalid-node-validation into develop
Reviewed-on: #461
2026-07-10 16:01:25 +03:00
vvzvlad 605c0f3dda Merge pull request 'docs(mcp): поправить устаревшую ссылку footnote-authoring.ts в комментариях' (#453) from docs/footnote-authoring-comment-cleanup into develop
Reviewed-on: #453
2026-07-10 15:49:35 +03:00
agent_coder 0f5f048ca2 fix(mcp): pre-validate node JSON против схемы + путь битого узла (#409, остаток Фазы 1)
Структурные редакторы (patchNode/insertNode/updatePageJson/transformPage) кидали
опаковый Yjs-крах на агентском JSON с вложенным узлом без/с неизвестным `type`:
«Failed to encode document to Yjs (fromJSON): Unknown node type: undefined» —
ГЛУБОКО в энкодере, уже ПОСЛЕ открытия collab-сессии, а хинт мислейблил это как
проблему атрибута. Агент ретраил вслепую (~34 краха в истории 06-17…07-07).

- findInvalidNode(doc) в prosemirror-markdown/node-ops.ts: DFS по content,
  возвращает {path, summary} первого узла с отсутствующим/не-строковым `type`
  или типом/маркой вне схемы. Множество имён — из getSchema(docmostExtensions),
  ТОГО ЖЕ, из которого энкод-путь строит docmostSchema → «известный тип»
  обходчика ровно то, что примет PMNode.fromJSON/toYdoc (сверено на 45 узлах +
  12 марках, ни ложных положительных, ни пропуска краш-типа).
- unstorableYjsError: findInvalidNode ПЕРВЫМ (node-shape крах больше не
  мислейблится как атрибут), затем findUnstorableAttr, generic-фраза последней.
- assertValidNodeShape(op, node) ДО getCollabTokenWithReauth/mutatePageContent
  в patchNode/insertNode/updatePageJson: fail-fast — collab-сессия не
  открывается, page-lock не берётся, сообщение детерминировано (mock-тест
  ассертит collabTokenFetched===false на битом пути). tableUpdateCell не тронут
  (строит абзац из plain text через makeCellParagraph, агентский JSON не глотает).
- Описания patch_node/insert_node/update_page_json: каждый узел, включая
  вложенные, несёт строковый `type` из схемы; текст-листы {"type":"text",...}.

sanitizeForYjs (стрип undefined-атрибутов) сохранён — другой класс отказа.
Внутреннее ревью: APPROVE WITH SUGGESTIONS — schema-fidelity/fail-fast/
no-false-positive/precedence подтверждены; замечания необязательны (тест
перечисления схемы, depth-guard безобиден т.к. энкодер падает раньше).
prosemirror-markdown vitest 726/726, mcp node --test 613/613.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 10:49:51 +03:00
agent_coder 4cb762b039 docs(mcp): обновить AGENTS.md + коммент под генерируемый инвентарь (ревью #460)
Две доковые правки по ревью: (1) AGENTS.md-буллет описывал ДО-#448 мир (ручная
правка SERVER_INSTRUCTIONS, enforced server-instructions.test.mjs, EXCEPTIONS) —
переписан: shared-спеки авто-обновляют генерируемый <tool_inventory>, только
inline-тул требует строки в INLINE_MCP_INVENTORY, enforced tool-inventory.test.mjs,
EXCEPTIONS больше нет; (2) коммент в server-instructions.ts называл гард окольно
('tool-specs.test.mjs's sibling test') → назван tool-inventory.test.mjs напрямую.
Единственная оставшаяся ссылка на удалённый тест устранена. Только доки/комменты.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 09:55:30 +03:00
agent_coder d0f99052cf refactor(tools): генерировать инвентарь SERVER_INSTRUCTIONS из реестра + guard имён тулов в промпте (#448)
Финальный линк Фазы 1б. Инвентарь тулов жил в 4 рукописных прозаических копиях
(SERVER_INSTRUCTIONS под regex-тестом; <tool_catalog>; имена в ai-chat.prompt.ts
без гарда; README) — роадмап #416 планировал 4 последовательных ручных правки
этого текста (#411/#412/#413/#415).

- SERVER_INSTRUCTIONS разбит (новый модуль server-instructions.ts): ROUTING_
  PROSE (рукописные intent-подсказки «когда что» — осмысленно ручные, перенесены
  ДОСЛОВНО со всеми предостережениями: <=250 у create_comment, soft-delete у
  delete_page, baseHash у drawio_update, PUBLIC у share_page) + buildToolInventory()
  — генерирует <tool_inventory> из реестра (mcpName + purpose из catalogLine,
  группировка по TOOL_FAMILY, бакет OTHER ловит незамаппленное → тул нельзя
  тихо потерять) + 5 inline MCP-only (INLINE_MCP_INVENTORY). Детерминирован
  (семейства FAMILY_ORDER, имена localeCompare). regex-тест server-instructions
  удалён; структурные гарантии — в новом tool-inventory.test.mjs (точное
  членство множества сильнее старого \b-скрейпа).
- Имена тулов в ai-chat.prompt.ts → через экспорт PROMPT_TOOL_NAMES; новый гард
  ai-chat.prompt.tool-names.spec.ts: каждое имя — реальный тул реестра, скан
  guidance-нот на camelCase-токены падает на несуществующем (escape-
  нейтрализация против ложных nThe-токенов).
- INLINE_TOOL_TIERS уже содержал ровно 8 genuinely-inline тулов (после #445) —
  сжатие не потребовалось.

Критерий: добавление/переименование спека меняет инвентарь БЕЗ правки прозы.
Внутреннее ревью: APPROVE — фактическим прогоном подтверждено, что НИ ОДИН тул
из старого SERVER_INSTRUCTIONS не выпал (диф старый-vs-новый пуст; добавился
get_workspace, раньше прятавшийся в EXCEPTIONS); проза дословна; инвентарь
полон/детерминирован/без фантомов; гард краснеет на обеих ветках провала.
613 node + 289 jest зелёные. Стоит на #445 — мержить последним в стопке 1б.

README-каталоги вне обязательного скоупа (docs-скрипт) — в чек-лист #412.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 09:21:28 +03:00
agent_coder 8c74659d91 refactor(tools): execute-маппинг в SHARED_TOOL_SPECS + автопроводка обоих хостов (#445)
Ядро Фазы 1б. Реестр (#294) шарил только метаданные (имя/схема/описание/tier),
но НЕ execute-логику — у каждого shared-тула было ДВА рукописных execute-тела
с копией маппинга аргументов (MCP registerShared в index.ts; in-app sharedTool
в ai-chat-tools.service, зеркалящий MCP-транспорт вручную). Корень
повторяющихся parity-багов (f46d89ea drawio, f8d26420 stashPage, fc9088b7
node-args): добавление одного тула = 7-9 согласованных ручных правок в двух
пакетах.

- SharedToolSpec расширен: канонический execute(client, args) (чистый JS —
  свободно пересекает zod-мажорную границу v3/v4) + оверрайды
  mcpExecute/inAppExecute/mcpOnly/inAppOnly для ОСОЗНАННЫХ per-layer различий.
  client: DocmostClientLike (Pick из #446). Канон возвращает СЫРЬЁ, каждый хост
  накладывает свой конверт (MCP jsonContent, in-app как есть); override владеет
  результатом хоста целиком.
- Оба хоста → циклы по Object.values(SHARED_TOOL_SPECS): index.ts registerShared
  39→0 (цикл), ai-chat-tools.service sharedTool ~40→1 (цикл). Добавление спека
  автоматически регистрирует тул в ОБОИХ хостах — сценарий PR #434 невозможен
  по построению.
- Осознанные различия через overrides (ни одно не сплющено к одному хосту):
  оба mcpExecute+inAppExecute — createPage/movePage/deletePage/
  exportPageMarkdown/createComment (guardrails, конверты, проекции, тексты
  ошибок); execute+inAppExecute — getPage/renamePage/resolveComment;
  execute+mcpExecute — stashPage (resource_link+structuredContent),
  checkNewComments (since-guard только на MCP).
- Оставлены inline (по делу): update_comment/delete_comment (MCP-only, in-app
  не даёт хард-правку/удаление комментов), search/transformPage (per-transport
  дивергенция — hybrid RRF / без deleteComments), table_get (noun-vs-verb
  naming clash — уедет после camelCase #412), getCurrentPage/updatePageContent/
  listSidebarPages/getComment/getPageHistory (in-app-only, per-request state).
- Guard-тесты (contract-parity, phantom-catalog) сохранены — теперь инварианты,
  не «последняя линия».

Внутреннее ревью: APPROVE, построчная BEFORE/AFTER-сверка по каждому shared-тулу
на обоих хостах — ни одного изменённого per-host поведения (порядок/дефолты
аргументов, guard'ы, конверты, проекции сохранены), множества тулов побайтово
совпадают (48 in-app, 45 MCP), кросс-zod-граница чистая (нет z. в execute),
611 mcp + 260 server тестов зелёные. Ядро Фазы 1б, стоит на #446 — мержить после.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 08:59:40 +03:00
agent_coder fe5b6ecd8c refactor(mcp): вывести DocmostClientLike/SharedToolSpec из реального типа клиента — убить ручные зеркала (#446)
Восстановленный отложенный долг #294: @docmost/mcp не отдавал .d.ts, поэтому в
сервере жили ТРИ дрейфующие ручные копии одних и тех же имён/сигнатур
(DocmostClientLike ~230 строк, копия SharedToolSpec, name-only HOST_CONTRACT_
METHODS-тест). In-app execute-тела зовут клиент ПОЗИЦИОННО, так что перестановка
параметра в client.ts доезжала до прода рантайм-ошибкой без сигнала на компиляции.

- declaration:true (+declarationMap) в packages/mcp/tsconfig.json; types-экспорт
  в package.json (exports → conditional {types, default} для . и ./http;
  require.resolve/dynamic-import резолвят default → build/index.js, рантайм не
  тронут). build/index.d.ts эмитится, реэкспортит DocmostClient + SharedToolSpec.
  Правок исходников пакета для эмита НЕ потребовалось.
- DocmostClientLike → Pick<DocmostClient, 48 методов> из type-only import
  (стёрт на компиляции, ESM/CJS-границу не задевает); ручное зеркало удалено.
- SharedToolSpec → type-only реэкспорт из пакета; ручная копия удалена.
- client-host-contract.test.mjs удалён целиком — имена И сигнатуры теперь
  проверяет tsc.
- Позиционная безопасность: never-called __assertClientCallContract(client:
  DocmostClientLike) воспроизводит каждый позиционный вызов с типизированными
  плейсхолдерами (AI-SDK стирает вход execute-замыканий в any, иначе позиционные
  вызовы не проверялись). Перестановка параметров client.ts → ошибка компиляции
  сервера ровно тут. Loose as-касты в ai-chat-tools.service не потребовали
  правок; as any не добавлялся.

Внутреннее ревью: APPROVE. Runtime resolution через conditional exports не сломан
(разобрано для прод-инсталляции, не только symlink); покрытие
__assertClientCallContract полное (48 call-sites == union == assert, сверено
программно); Pick полон; демонстрация reorder → TS2345 в assert. Единственная
находка (Promise<any> в части возвратов) предсуществующая в client.ts, вне
цели PR. Стоит на #447 (закрытие skew build/vs/src) — мержить после него.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 08:29:46 +03:00
agent_coder e24ddf6b3e test(ai-chat): покрыть safety-путь детектора деградации + границы (ревью #454)
Ревью: дизайн LGTM, но safety-фича недотестирована. Добавлено (только тесты,
прод-код не тронут):
- e2e-реакция детектора: streamText эмитит degenerate-чанки → union abortSignal
  срабатывает с 'Output degeneration detected' (отличимо от Stop) → onAbort
  пишет status:error + OUTPUT_DEGENERATION_ERROR + усечённый content, лиза MCP
  закрыта. Именно ДЕЙСТВИЕ на детект (детектит-но-не-действует = защиты нет);
- граница monochar-порога: hasPeriodicTail('x'×59)=false, ('x'×60)=true
  (мутация >=→> раньше выживала);
- empty-turn маркер (шаги исчерпаны + без текста → STEP_LIMIT_NO_ANSWER_MARKER;
  негативы на нормальный текст-ход и на исчерпание-с-текстом — гардят AND);
- различение degeneration-onAbort vs user-Stop (Stop → status:aborted, без
  error/усечения).

Мутационно: (a) >=→> роняет 60-границу; (b) нейтрализация onAbort-ветки роняет
reaction-тест; (c) нейтрализация маркера роняет empty-turn-тест. +7 тестов,
137 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 07:54:34 +03:00
agent_coder 3cba551800 fix(ai-chat): «send now» во время detached-run — авторитетный серверный stop + ограниченный ретрай 409 (#396)
В автономном режиме «Interrupt and send now» во время живого detached-run делал
только локальный stop() (abort SSE), который сервер игнорирует (run живёт по
дизайну #184/#234), поэтому onFinish→flush новый POST упирался в гейт «один
активный run на чат» → 409 A_RUN_ALREADY_ACTIVE, новый turn не стартовал.
handleStop делает правильно (доп. onServerStop), sendNow — нет. Вариант A
(клиентский, горячий путь сервера не тронут):

- sendNow в автономном режиме дополнительно зовёт onServerStop(chatId) (или
  откладывает через stopPendingRef, если chatId ещё не усыновлён — как
  handleStop) и взводит one-shot supersedeRetryRef ДО stop();
- транспорт-fetch на supersede-отправке (и только на ней) ретраит РОВНО 409 с
  body.code===A_RUN_ALREADY_ACTIVE до 4 попыток с бэкоффом 150/300/600ms;
  onServerStop гарантирует осадку run → ретрай сходится. Обычная отправка (не
  взведён флаг) падает на 409 мгновенно. isRunAlreadyActive читает
  response.clone() → тело оригинала возвращается потребителю нетронутым.
  409 всегда до записи user-строки (pre-check/beginRun раньше insert) → повтор
  POST безопасен, дублей нет.

Внутренний цикл: 2 прохода. Ревью нашло CRITICAL: supersedeRetryRef застревал
взведённым, когда sendNow взвёл, но POST не ушёл (promoted head удалён →
flushNext() false; либо wasResumed-return) — следующая обычная отправка молча
ретраила настоящий 409. Починка: разоружать флаг симметрично остальным one-shot
(в ветке !flushNext() и в isStreaming-defuse-эффекте); транспорт read-and-clear
на входе каждой отправки. Мутационно: убрать disarm → strand-тест краснеет
(4 вызова вместо 1). Легаси-режим не тронут (регресс-гард).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 07:50:04 +03:00
agent_coder 15a9eba562 fix(delivery): immutable-кэш ассетов — отключить дефолтный cacheControl @fastify/static (#452)
Хэшированные ассеты отдавали 'cache-control: public, max-age=0' вместо
'public, max-age=31536000, immutable' → повторные заходы ревалидировали каждый
ассет (десятки 304 × мобильный RTT), главный выигрыш #346 не реализовывался.
Причина: у @fastify/static опция cacheControl:true по умолчанию пишет свой
Cache-Control (из maxAge, дефолт 0) ПОСЛЕ setHeaders-колбэка, затирая immutable-
заголовок из resolveStaticAssetHeaders. Фикс — cacheControl:false, колбэк
владеет заголовком. preCompressed не конфликтовал, потому баг был только в
заголовках.

Крайние случаи проверены: locales/vad/иконки получают только vary (без
cache-control → браузер ревалидирует по etag — ок); index.html отдаётся
отдельным wildcard-роутом со своим no-cache (не затронут); preCompressed .br
получает путь с /assets/ → маппинг матчит, immutable ставится.

Тест: bare-fastify + inject() — /assets/<hashed>.js содержит immutable+
max-age=31536000, /locales/en.json — нет. Мутационно: cacheControl:true роняет
ассерт immutable. jest static.module → 5/5.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 07:30:44 +03:00
agent_coder 2c03fefa9d fix(ai-chat): защита от петель агента — lockdown под тоггл, детектор деградации, бюджет шагов (#444)
Третий класс петли (инцидент 2026-07-10): ран упёрся в 20-шаговый кап, спалив
все шаги на чтение; на 20-м шаге final-step lockdown отнял инструменты
(toolChoice:'none') посреди незаконченной работы → модель выродилась в
текст-повтор («loadTools.» ×20416, 255КБ). Пакет защит по дизайну владельца:

- MAX_AGENT_STEPS 20→50; спеки выводятся из константы (нет захардкоженных 19/20).
- Final-step lockdown под env-тогглом AI_CHAT_FINAL_STEP_LOCKDOWN (дефолт OFF,
  по образцу AI_CHAT_DEFERRED_TOOLS): OFF — инструменты доступны на всех шагах +
  мягкий финальный нудж; ON — легаси toolChoice:'none'+FINAL_STEP_INSTRUCTION.
  Спеки параметризованы по тогглу. .env.example задокументирован.
- Пустой ход (все шаги без текста, шаги исчерпаны) получает синтетический
  маркер-текст — виден в UI и реплее.
- Детектор токен-деградации в onChunk (единственная защита от болтовни, БЕЗ
  maxOutputTokens — tool-аргументы это выходные токены): чистые правила
  (≥25 одинаковых строк ИЛИ периодический хвост), при срабатывании abort через
  внутренний AbortController ∪ effectiveSignal (AbortSignal.any), финализация в
  onAbort: усечение хвоста, ai_chat_runs.error=Output degeneration detected,
  лизы MCP/снапшоты освобождаются (существующий lifecycle).
- Предупреждение о бюджете шагов на MAX-6…MAX-2 с убывающим N.
- loadTools-описание и преамбула каталога явно говорят, что CORE-тулы всегда
  активны (список из CORE_TOOL_KEYS динамически).

Внутренний цикл: 2 прохода. Ревью нашло data-loss-риск: правило периодичности
детектора ложно срабатывало на markdown-разделителях/setext-подчёркиваниях/
хвостовых пробелах (монохар-хвост p-периодичен при ЛЮБОМ p → ложный abort с
пометкой error и усечением). Починка: отдельная монохар-проверка (порог 60,
выше любого реального разделителя) + требование ≥2 различных символов в
периодическом блоке при p≥2. Реальный loadTools-цикл (период ~10) ловится.
Мутационно: 59 одинаковых — не флаг, 60 — флаг; loadTools×20416 — флаг.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 07:28:48 +03:00
agent_coder 76af4f692e docs(mcp): поправить устаревшую ссылку footnote-authoring.ts -> @docmost/prosemirror-markdown
#429 (дедуп node-ops) перенёс footnoteContentKey в @docmost/prosemirror-markdown
и удалил footnote-authoring.ts, но два docstring-комментария в
footnote-normalize-merge.ts всё ещё ссылались на старое имя файла. Только
комментарии, на сборку/поведение не влияет.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 07:26:42 +03:00
agent_coder 4809348457 test(mcp): ассертить collab-hint (pageId + transient/retry) в reject-текстах (ревью #441)
Enrichment collab-ошибок из Phase C (#437) дописывает
'(pageId …; transient — retry once; …)' к connect-timeout/connection-closed,
но reject-регекспы матчили только базовый текст → проходили и С hint, и БЕЗ
(vacuous). Ужесточил два ассерта (connection-closed, connect-timeout) до
'<база> (pageId page-1; transient' — теперь рефактор, убравший hint(), их
роняет. Мутационно: hint()->'' → ровно эти 2 теста краснеют (18->16).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 07:14:39 +03:00
agent_coder d3d32d637b fix(mcp): no-response диагностика — не отдавать сырой error.message (утечка host в модель) (внутр. ревью #437)
no-response ветка формата использовала error.code ?? error.message: при
отсутствии code axios-сообщения сетевых ошибок содержат host:port
('connect ECONNREFUSED 127.0.0.1:3000', 'getaddrinfo ENOTFOUND host'), что
нарушает инвариант #437 «host никогда не попадает в видимое модели сообщение».
Теперь только error.code (?? 'network error'); полный нативный текст уходит в
stderr под DEBUG. code проставлен фактически для всех реальных no-response
ошибок. Тест обновлён: сырое host-содержащее сообщение -> нейтральный reason,
плюс ассерт что host в сообщении отсутствует.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 07:12:37 +03:00
agent_coder 9a435201b8 feat(mcp): enrich collab connect/persist/closed error texts with pageId + retry hint (#437)
Append `(pageId <id>; transient — retry once; persistent failures mean the
collab server is unreachable/overloaded)` to the connect-timeout, persist-timeout
and connection-closed error texts in CollabSession, so the agent can self-correct
instead of blind-looping. The Yjs-encode error is left untouched (it already names
the offending attribute).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 07:12:37 +03:00
agent_coder d6827b9210 feat(mcp): actionable tool errors — central axios diagnostics + fail-fast comment-id guard (#437)
Phase A: add ONE response interceptor on DocmostClient's axios instance,
registered AFTER the re-login interceptor, that reformats a failed request's
error.message IN PLACE (never a custom Error subclass, so the live
axios.isAxiosError / error.response?.status / config._retry checks keep working)
into `<METHOD> <path> failed (<status> <statusText>): <serverMessage>`, or the
no-response variant. serverMessage is built ONLY from the whitelisted
message/error fields or statusText — raw string/HTML bodies, headers and config
never appear; an arraybuffer body is size-capped JSON.parsed; the full body goes
to stderr only under DEBUG (parity with downloadImage). A _docmostFormatted flag
guards against double-processing.

Phase B (#436): assertFullUuid throws an actionable error BEFORE any network
call at all five commentId sites (resolve/update/delete/get_comment and
create_comment's parentCommentId when provided), so a truncated id can no longer
loop as an opaque 400/404.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 07:12:37 +03:00
140 changed files with 20406 additions and 6647 deletions
+45
View File
@@ -124,6 +124,40 @@ MCP_DOCMOST_PASSWORD=
# MCP_TOKEN=
# MCP_SESSION_IDLE_MS=1800000
#
# --- MCP collaboration write path: concurrency + rights-staleness (#449) ------
# MCP content writes (update_page, insert/replace nodes, comments-in-body, etc.)
# go over the collaboration websocket and are serialized PER PAGE by an
# in-process mutex (a module-level Map, one promise-chain per page UUID). This
# guarantees no two MCP writes on the SAME page overlap and clobber each other.
#
# DEPLOY REQUIREMENT — SINGLE INSTANCE or STICKY SESSIONS. The mutex is
# process-local. Behind a multi-replica load balancer WITHOUT sticky sessions,
# two replicas can each "hold" the lock for the same page at the same time and
# serialization is silently lost (concurrent full-document writes race on the
# live Yjs fragment). Run the MCP/app as a SINGLE instance, OR pin a page's
# traffic to one replica (sticky sessions / consistent hashing on page id). The
# same constraint applies to the RAM-only stash_page blob store above. There is
# deliberately no cross-process (e.g. Postgres advisory) lock yet — this is a
# CONSCIOUS documented constraint, not an oversight (#449).
#
# To reduce connect-storms the write path caches ONE live collab session per
# (wsUrl, page, token). Tunables (all optional; defaults are safe):
# MCP_COLLAB_SESSION_IDLE_MS=60000 # idle TTL, reset per op; 0 disables cache
# MCP_COLLAB_SESSION_MAX_ENTRIES=32 # LRU cap on cached sessions
# MCP_COLLAB_TOKEN_TTL_MS=300000 # per-client collab-token cache (5 min)
#
# RIGHTS-STALENESS TRADE-OFF. A cached collab session writes under the token
# captured at CONNECT time, and the collab-token cache reuses a token for its TTL.
# So if a user's access to a page is REVOKED, MCP writes on an already-open
# session may keep succeeding until the session ages out. MCP_COLLAB_SESSION_MAX_AGE_MS
# is the HARD lifetime (checked at each acquire) that BOUNDS this window: after it,
# the session is torn down and the next write re-auths with a fresh token, picking
# up the revocation. Default 10 min. LOWER it to shorten the revocation lag at the
# cost of more reconnects; RAISE it to reduce reconnects at the cost of a longer
# stale-rights window. There is intentionally no push-based cache invalidation on
# a rights change — this bounded window is the accepted trade-off (#449).
# MCP_COLLAB_SESSION_MAX_AGE_MS=600000
#
# BLOB SANDBOX (stash_page). An in-RAM, process-local store that hands large page
# content + images to an external consumer WITHOUT bloating the model context or
# requiring Docmost auth. The stash_page tool serializes a page, mirrors its
@@ -217,6 +251,17 @@ MCP_DOCMOST_PASSWORD=
# active" behavior.
# AI_CHAT_DEFERRED_TOOLS=true
# Final-step lockdown for the in-app agent loop (#444). Default OFF. When ON
# (legacy), the LAST allowed step forces a text-only answer: the model's tools are
# stripped (toolChoice=none) and a synthesis instruction is appended. That
# tool-stripping caused a token-degeneration incident — robbed of its tools on the
# final step mid-work, the model emitted a ~255KB block repeating a single token —
# so the default is now OFF: the last step keeps its tools and gets only a SOFT
# nudge to finish with a text summary, and a token-degeneration detector is the
# universal anti-babble guard. Enable this ONLY for a model that reliably ends its
# turns with a clear text answer.
# AI_CHAT_FINAL_STEP_LOCKDOWN=false
# --- Autonomous / detached agent runs (settings.ai.autonomousRuns) ---
# Opt-in per workspace (AI settings; off by default). When on, a chat turn becomes
# a server-side RUN that survives a browser disconnect — only an explicit Stop ends
+6
View File
@@ -157,6 +157,12 @@ jobs:
- name: Build prosemirror-markdown
run: pnpm --filter @docmost/prosemirror-markdown build
# docmost-client.loader.ts type-imports from @docmost/mcp (issue #446); its
# build/ is gitignored and `test:e2e` type-checks, so build it here or tsc
# fails with TS2307 (mirrors the e2e-mcp / mcp-server-parity jobs).
- name: Build mcp
run: pnpm --filter @docmost/mcp build
- name: Run migrations
run: pnpm --filter ./apps/server migration:latest
+5
View File
@@ -2,6 +2,11 @@
.env.dev
.env.prod
data
# Exception: the committed draw.io shape catalog (issue #424) lives in a `data/`
# dir, but the bare `data` ignore above is meant for runtime state, not this
# bundled build asset. Re-include the directory and its contents.
!packages/mcp/data/
!packages/mcp/data/**
# compiled output
/dist
node_modules
+1 -1
View File
@@ -338,7 +338,7 @@ Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirro
- The version string shown in the UI comes from `APP_VERSION` (CI/Docker) or `git describe --tags --always` (local), resolved in `vite.config.ts` — not from `package.json`.
- Server TS config is permissive (`noImplicitAny: false`, `strictNullChecks: false`, `no-explicit-any` lint disabled). Follow the existing relaxed style rather than tightening types broadly.
- Dependency versions are heavily pinned via `pnpm.overrides` and `pnpm.patchedDependencies` (`scimmy`, `yjs`, `ai`) in the root `package.json`. Don't bump pinned/patched deps casually; the patches and overrides exist for compatibility/security reasons. The `ai@6.0.134` patch disables the SDK's O(n²) cumulative `partialOutput` accumulation when no output strategy is requested (server heap OOM on long agent runs, #184; tripwire test: `apps/server/src/integrations/ai/ai-sdk-partial-output.patch.spec.ts`) — it MUST be re-created via `pnpm patch` when bumping `ai`.
- **Adding/renaming/removing an MCP tool requires updating `SERVER_INSTRUCTIONS`** in `packages/mcp/src/index.ts` — the intent-routing guide MCP clients receive on initialize. This applies both to inline `server.registerTool(...)` calls in `index.ts` and to specs in `packages/mcp/src/tool-specs.ts`. Enforced by `packages/mcp/test/unit/server-instructions.test.mjs`, which fails when a registered tool is not mentioned in the guide (deliberate opt-outs go into its `EXCEPTIONS` list). `packages/mcp/build/` is gitignored and rebuilt in CI/Docker via `pnpm build` (same convention as `git-sync`/`prosemirror-markdown`) — never commit it; rebuild locally after editing to run the tests.
- **The MCP tool inventory in `SERVER_INSTRUCTIONS` is GENERATED from the registry** (`packages/mcp/src/server-instructions.ts`: `buildToolInventory()` over `SHARED_TOOL_SPECS`) and spliced into the hand-written routing prose (`ROUTING_PROSE`). So adding/renaming/removing a **shared** spec in `packages/mcp/src/tool-specs.ts` auto-updates the `<tool_inventory>` — no manual `SERVER_INSTRUCTIONS` edit needed. Only an **inline** MCP-only tool (those registered via `server.registerTool(...)` in `index.ts`, not through the registry) needs a one-line entry in `INLINE_MCP_INVENTORY`. Enforced by `packages/mcp/test/unit/tool-inventory.test.mjs`, which fails when a registered tool is missing from the generated inventory (there is no `EXCEPTIONS` opt-out anymore — every tool must appear). Update `ROUTING_PROSE` when a tool's *intent guidance* (when-to-use) changes. `packages/mcp/build/` is gitignored and rebuilt in CI/Docker via `pnpm build` (same convention as `git-sync`/`prosemirror-markdown`) — never commit it; rebuild locally after editing to run the tests.
## CI / release
+121
View File
@@ -10,6 +10,111 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Breaking Changes
- **External MCP tool names are now camelCase (all renamed).** Every tool on the
external `/mcp` surface was renamed from `snake_case` to `camelCase`, so the
external MCP name now matches the in-app tool name exactly (one logical tool,
one name everywhere). For example `get_node``getNode`, `edit_page_text`
`editPageText`, `patch_node``patchNode`. The tools' behaviour, inputs and
outputs are unchanged — only the names change. The single-word `search`
keeps its name.
*Migration (external MCP clients only — the in-app AI agent already used these
names and is unaffected):* update anything that refers to a tool by its
string name — permission allowlists (`mcp__gitmost-*__get_node`
`mcp__gitmost-*__getNode`), saved prompts/skills, `.mcp.json` tool filters,
and metrics dashboards that group by the `tool` label — and roll it out in
lockstep with this deploy, because the old snake_case names stop resolving.
Released together with the `import_page_markdown`/`update_page_markdown`
change below so external configs break exactly once.
Full mapping (old → new):
| Old (snake_case) | New (camelCase) |
| --- | --- |
| `check_new_comments` | `checkNewComments` |
| `copy_page_content` | `copyPageContent` |
| `create_comment` | `createComment` |
| `create_page` | `createPage` |
| `delete_comment` | `deleteComment` |
| `delete_node` | `deleteNode` |
| `delete_page` | `deletePage` |
| `diff_page_versions` | `diffPageVersions` |
| `docmost_transform` | `docmostTransform` |
| `drawio_create` | `drawioCreate` |
| `drawio_get` | `drawioGet` |
| `drawio_guide` | `drawioGuide` |
| `drawio_shapes` | `drawioShapes` |
| `drawio_update` | `drawioUpdate` |
| `edit_page_text` | `editPageText` |
| `export_page_markdown` | `exportPageMarkdown` |
| `get_node` | `getNode` |
| `get_outline` | `getOutline` |
| `get_page` | `getPage` |
| `get_page_json` | `getPageJson` |
| `get_workspace` | `getWorkspace` |
| `insert_footnote` | `insertFootnote` |
| `insert_image` | `insertImage` |
| `insert_node` | `insertNode` |
| `list_comments` | `listComments` |
| `list_page_history` | `listPageHistory` |
| `list_pages` | `listPages` |
| `list_shares` | `listShares` |
| `list_spaces` | `listSpaces` |
| `move_page` | `movePage` |
| `patch_node` | `patchNode` |
| `rename_page` | `renamePage` |
| `replace_image` | `replaceImage` |
| `resolve_comment` | `resolveComment` |
| `restore_page_version` | `restorePageVersion` |
| `search` | `search` (unchanged) |
| `search_in_page` | `searchInPage` |
| `share_page` | `sharePage` |
| `stash_page` | `stashPage` |
| `table_delete_row` | `tableDeleteRow` |
| `table_get` | `tableGet` |
| `table_insert_row` | `tableInsertRow` |
| `table_update_cell` | `tableUpdateCell` |
| `unshare_page` | `unsharePage` |
| `update_comment` | `updateComment` |
| `update_page_json` | `updatePageJson` |
| `update_page_markdown` | `updatePageMarkdown` |
(#412)
- **External MCP: `import_page_markdown` removed, `update_page_markdown` added.**
The external `/mcp` surface no longer exposes `importPageMarkdown` (the
round-trip parser for a self-contained *exported* Docmost-Markdown file). In
its place it now exposes **`updatePageMarkdown`** — a plain-Markdown
full-body replace (`{pageId, content, title?}`) that pairs with
`updatePageJson`, re-imports the whole body (block ids regenerate) and
parses Docmost-flavoured markdown including `^[...]` inline footnotes.
*Migration:* MCP clients that called `importPageMarkdown` to overwrite a
page's body from Markdown should call `updatePageMarkdown` instead (pass the
markdown as `content`). Round-tripping an exported Docmost-Markdown file with
comment anchors/diagrams is no longer available on the external MCP surface;
export remains via `exportPageMarkdown`. The in-app AI agent is unaffected —
it keeps both `importPageMarkdown` and the renamed `updatePageMarkdown` (was
`updatePageContent`). The total MCP tool count is unchanged (−1 / +1). The
external names shown here are the post-#412 camelCase names. (#411)
- **`getNode` now returns Markdown by default (was ProseMirror JSON).** The
block-level read/write tools default to Markdown so a block round trip is
`getNode` (markdown) → edit → `patchNode` (markdown). `getNode` now returns
`{ …, format: "markdown", markdown }` unless you pass `format: "json"` (which
restores the previous `{ …, node }` ProseMirror subtree); comment anchors —
including resolved ones — are preserved in the markdown so a write-back never
orphans a thread, and a node that cannot be a document top-level block
(`tableRow`/`tableCell`/`tableHeader` addressed via `#<index>`) auto-falls back
to JSON with `format: "json"` in the response. `patchNode`/`insertNode` gain a
`markdown` input alongside `node` (provide exactly one): the markdown fragment
may rewrite/insert several blocks at once and supports `^[...]` footnotes.
*Migration (external MCP clients only):* a client that consumed `getNode`'s
`node` field must now either read `markdown`, or pass `format: "json"` to keep
the old ProseMirror-JSON output. Released together with the `#411`/`#412`
breaking window so external configs break exactly once. (#413)
### Added
- **Place several images side by side in a row.** A new "Inline (side by
@@ -146,6 +251,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
by physical key position and matched against the commands; genuine Cyrillic
search terms keep priority over remapped candidates, and short wrong-layout
prefixes match by command title. (#283, #285, #287)
- **Opt-in substring "lookup" search mode for agents.** `/api/search` gains an
additive, opt-in mode (guarded by a new `substring` flag) that matches literal
substrings of page titles and body text — so technical tokens the full-text
tokenizer mangles (`backup-srv.local`, `10.0.12.5`, `WB-MGE-30D86B`) are found
even when the FTS query is empty. It returns a location `path`, a windowed
`snippet` and a per-response relevance `score`, supports `titleOnly` and a
`parentPageId` subtree scope, and applies the page-level permission filter
before the limit. The web UI never sets `substring`, so its full-text search
behaviour is byte-for-byte unchanged. The leading-wildcard `LIKE` predicates
are backed by GIN trigram indexes on `LOWER(f_unaccent(title))` and
`LOWER(f_unaccent(text_content))` so lookups use a bitmap index scan instead of
a sequential scan. (#443)
- **MCP `search` tool returns richer, agent-oriented results.** The external MCP
`search` response shape changes for the agent surface: each hit now carries
`pageId` (renamed from `id`), plus `path`, `snippet` and `score`; the
UI-oriented `spaceId`, `rank` and `highlight` fields are dropped. (#443)
### Changed
+10
View File
@@ -81,4 +81,14 @@ VOLUME ["/app/data/storage"]
EXPOSE 3000
# DEPLOY REQUIREMENT — SINGLE INSTANCE or STICKY SESSIONS (#449).
# MCP content writes are serialized per page by an IN-PROCESS mutex, and the
# stash_page blob store + cached collab sessions are RAM-only and process-local.
# Running MULTIPLE replicas of this image behind a load balancer WITHOUT sticky
# sessions silently breaks per-page write serialization (two replicas can lock
# the same page at once) and makes stash_page blobs unreachable across replicas.
# Run a SINGLE instance, or pin each page's traffic to one replica (sticky
# sessions / consistent hashing on page id). There is deliberately no
# cross-process lock yet — a conscious constraint. See .env.example (the "MCP
# collaboration write path" block) and packages/mcp/README.md for details.
CMD ["pnpm", "start"]
@@ -28,7 +28,10 @@ const h = vi.hoisted(() => ({
body: Record<string, unknown>;
}) => { body: Record<string, unknown> };
prepareReconnectToStreamRequest?: () => { api?: string };
fetch?: (input: unknown, init?: { method?: string }) => Promise<unknown>;
fetch?: (
input: unknown,
init?: { method?: string; body?: unknown },
) => Promise<unknown>;
},
},
}));
@@ -200,6 +203,244 @@ describe("ChatThread — send now (#198)", () => {
});
});
// #396: in autonomous mode a live sendNow must additionally request the
// AUTHORITATIVE server stop of the detached run (a local abort is only a client
// disconnect the server ignores) and arm a bounded 409 retry so the re-POST
// converges once the one-active-run slot frees. Legacy mode is unchanged.
describe("ChatThread — send now server-stop + supersede retry (#396)", () => {
beforeEach(resetState);
afterEach(cleanup);
// A settled assistant tail => no mount resume (attemptResumeRef false), so the
// "Send now" button is visible for the NEW local streaming turn while
// autonomous runs are enabled.
const settledTail = () => [
row("u1", "user", undefined, "hi"),
row("a1", "assistant", "succeeded", "done"),
];
it("autonomous: sendNow during a live stream calls onServerStop with the chat id", () => {
const { onServerStop } = renderThread({
autonomousRunsEnabled: true,
initialRows: settledTail(),
});
fireEvent.click(screen.getByTestId("queue-btn"));
fireEvent.click(screen.getByLabelText("Send now"));
expect(h.state.stop).toHaveBeenCalledTimes(1);
expect(onServerStop).toHaveBeenCalledWith("c1");
});
it("legacy (autonomous off): sendNow does NOT call onServerStop and does NOT retry the send", async () => {
const { onServerStop } = renderThread({
autonomousRunsEnabled: false,
initialRows: settledTail(),
});
fireEvent.click(screen.getByTestId("queue-btn"));
fireEvent.click(screen.getByLabelText("Send now"));
expect(onServerStop).not.toHaveBeenCalled();
// The supersede retry must NOT be armed: a POST that 409s is returned as-is
// (single fetch, no retry).
const fetchMock = vi
.fn()
.mockResolvedValue(
new Response(JSON.stringify({ code: "A_RUN_ALREADY_ACTIVE" }), {
status: 409,
}),
);
vi.stubGlobal("fetch", fetchMock);
let res!: Response;
await act(async () => {
res = (await h.state.transport!.fetch!("http://x", {
method: "POST",
body: "{}",
})) as Response;
});
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(res.status).toBe(409);
});
it("armed supersede send retries 409 A_RUN_ALREADY_ACTIVE and succeeds once the slot frees", async () => {
renderThread({ autonomousRunsEnabled: true, initialRows: settledTail() });
// Arm the retry by performing a live sendNow (autonomous branch sets the ref).
fireEvent.click(screen.getByTestId("queue-btn"));
fireEvent.click(screen.getByLabelText("Send now"));
const fetchMock = vi
.fn()
// First POST: the old detached run still holds the slot -> 409.
.mockResolvedValueOnce(
new Response(JSON.stringify({ code: "A_RUN_ALREADY_ACTIVE" }), {
status: 409,
}),
)
// Retry: the server stop settled the old run -> 200.
.mockResolvedValueOnce(new Response("ok", { status: 200 }));
vi.stubGlobal("fetch", fetchMock);
let res!: Response;
await act(async () => {
res = (await h.state.transport!.fetch!("http://x", {
method: "POST",
body: "{}",
})) as Response;
});
expect(fetchMock).toHaveBeenCalledTimes(2);
expect(res.status).toBe(200);
});
it("supersede retry is one-shot: a later send (ref cleared) does NOT retry a 409", async () => {
renderThread({ autonomousRunsEnabled: true, initialRows: settledTail() });
fireEvent.click(screen.getByTestId("queue-btn"));
fireEvent.click(screen.getByLabelText("Send now")); // arms the one-shot
// First armed send: immediately succeeds, consuming the arm.
let fetchMock = vi
.fn()
.mockResolvedValue(new Response("ok", { status: 200 }));
vi.stubGlobal("fetch", fetchMock);
await act(async () => {
await h.state.transport!.fetch!("http://x", {
method: "POST",
body: "{}",
});
});
expect(fetchMock).toHaveBeenCalledTimes(1);
// A subsequent send is NOT armed -> a 409 is returned as-is (no retry).
fetchMock = vi.fn().mockResolvedValue(
new Response(JSON.stringify({ code: "A_RUN_ALREADY_ACTIVE" }), {
status: 409,
}),
);
vi.stubGlobal("fetch", fetchMock);
let res!: Response;
await act(async () => {
res = (await h.state.transport!.fetch!("http://x", {
method: "POST",
body: "{}",
})) as Response;
});
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(res.status).toBe(409);
});
it("supersede retry is bounded: exhaustion surfaces the 409 error", async () => {
renderThread({ autonomousRunsEnabled: true, initialRows: settledTail() });
fireEvent.click(screen.getByTestId("queue-btn"));
fireEvent.click(screen.getByLabelText("Send now"));
// Every attempt 409s -> after 4 attempts the last 409 surfaces.
const fetchMock = vi.fn().mockResolvedValue(
new Response(JSON.stringify({ code: "A_RUN_ALREADY_ACTIVE" }), {
status: 409,
}),
);
vi.stubGlobal("fetch", fetchMock);
let res!: Response;
await act(async () => {
res = (await h.state.transport!.fetch!("http://x", {
method: "POST",
body: "{}",
})) as Response;
});
// 4 attempts total (1 immediate + 3 backoff retries), then give up.
expect(fetchMock).toHaveBeenCalledTimes(4);
expect(res.status).toBe(409);
});
it("armed supersede send does NOT retry a non-409 status", async () => {
renderThread({ autonomousRunsEnabled: true, initialRows: settledTail() });
fireEvent.click(screen.getByTestId("queue-btn"));
fireEvent.click(screen.getByLabelText("Send now"));
const fetchMock = vi
.fn()
.mockResolvedValue(new Response("boom", { status: 500 }));
vi.stubGlobal("fetch", fetchMock);
let res!: Response;
await act(async () => {
res = (await h.state.transport!.fetch!("http://x", {
method: "POST",
body: "{}",
})) as Response;
});
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(res.status).toBe(500);
});
// Strand-path regression: sendNow arms the supersede retry, but if the promoted
// head is removed before the abort's onFinish lands, flushNext() sends nothing
// (returns false) and NO re-POST consumes the arm. The arm must be disarmed on
// that no-send branch so the NEXT unrelated NORMAL send does not inherit it and
// silently retry a genuine 409 (e.g. a legitimate two-tab conflict) 4x instead
// of surfacing it immediately.
it("strand-path: a stranded supersede arm (flushNext no-send) does NOT retry a later normal 409", async () => {
renderThread({ autonomousRunsEnabled: true, initialRows: settledTail() });
// Arm the retry via a live autonomous sendNow (promotes the head + arms).
fireEvent.click(screen.getByTestId("queue-btn"));
fireEvent.click(screen.getByLabelText("Send now"));
// Remove the promoted head BEFORE the abort lands, so flushNext() returns
// false (no POST) and the arm would strand without the disarm fix.
fireEvent.click(screen.getByLabelText("Remove queued message"));
// The abort's onFinish now takes the flushOnAbortRef branch, calls flushNext()
// which finds an empty queue and returns false -> the no-send disarm must run.
act(() => {
h.state.onFinish?.({
message: { id: "a1", role: "assistant", parts: [] },
isAbort: true,
isDisconnect: false,
isError: false,
});
});
// No re-POST was sent (nothing to flush).
expect(h.state.sendMessage).not.toHaveBeenCalled();
// A subsequent NORMAL send that 409s must be returned as-is (exactly 1 fetch):
// the stranded arm must NOT cause the genuine 409 to be retried.
const fetchMock = vi
.fn()
.mockResolvedValue(
new Response(JSON.stringify({ code: "A_RUN_ALREADY_ACTIVE" }), {
status: 409,
}),
);
vi.stubGlobal("fetch", fetchMock);
let res!: Response;
await act(async () => {
res = (await h.state.transport!.fetch!("http://x", {
method: "POST",
body: "{}",
})) as Response;
});
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(res.status).toBe(409);
});
it("armed supersede send does NOT retry a 409 with a different (non-A_RUN_ALREADY_ACTIVE) body", async () => {
renderThread({ autonomousRunsEnabled: true, initialRows: settledTail() });
fireEvent.click(screen.getByTestId("queue-btn"));
fireEvent.click(screen.getByLabelText("Send now"));
const fetchMock = vi.fn().mockResolvedValue(
new Response(JSON.stringify({ code: "SOMETHING_ELSE" }), {
status: 409,
}),
);
vi.stubGlobal("fetch", fetchMock);
let res!: Response;
await act(async () => {
res = (await h.state.transport!.fetch!("http://x", {
method: "POST",
body: "{}",
})) as Response;
});
expect(fetchMock).toHaveBeenCalledTimes(1);
expect(res.status).toBe(409);
});
});
// #388: the editor selection is snapshotted at send time and nested inside
// openPage on the wire. The getter is read live from a ref, so each send ships a
// fresh snapshot.
@@ -70,6 +70,36 @@ const RECONNECT_MAX_ATTEMPTS = 5;
// Backoff before attempt N (1-based): 1s, 2s, 4s, 8s, 16s.
const RECONNECT_BASE_DELAY_MS = 1000;
// #396: bounded retry for the "Interrupt and send now" re-send when it races the
// authoritative server stop of the just-superseded detached run. The re-POST can
// arrive before the old run has released the one-active-run slot, so the server
// returns 409 A_RUN_ALREADY_ACTIVE. The server stop guarantees the slot frees, so
// a few short backoffs converge. 4 total attempts: attempt 1 fires immediately,
// then these are the waits BEFORE attempts 2, 3 and 4 (150ms, 300ms, 600ms). If
// all 4 attempts 409, the last 409 surfaces (the banner) — acceptable per #396.
const SUPERSEDE_RETRY_DELAYS_MS = [150, 300, 600];
// The server error code that means "another run is already active for this chat".
const A_RUN_ALREADY_ACTIVE = "A_RUN_ALREADY_ACTIVE";
/**
* #396: defensively decide whether a 409 response is the one-active-run gate
* rejection (code A_RUN_ALREADY_ACTIVE) vs. some other 409. Reads a CLONE so the
* original response body stays intact for the caller when it is returned as-is.
* Any parse failure or unexpected shape => false (do NOT retry).
*/
async function isRunAlreadyActive(response: Response): Promise<boolean> {
try {
const body = (await response.clone().json()) as unknown;
return (
typeof body === "object" &&
body !== null &&
(body as { code?: unknown }).code === A_RUN_ALREADY_ACTIVE
);
} catch {
return false;
}
}
/** The page the user is currently viewing, sent as chat context. */
export interface OpenPageContext {
id: string;
@@ -326,6 +356,26 @@ export default function ChatThread({
const flushOnAbortRef = useRef(false);
const interruptNextSendRef = useRef(false);
// #396: one-shot arm for the bounded 409 A_RUN_ALREADY_ACTIVE retry on the
// "Interrupt and send now" re-send in autonomous mode. sendNow triggers the
// authoritative server stop of the detached run, but that stop and the
// onFinish->flushNext re-POST race: the new POST can hit the one-active-run
// gate before the old detached run has settled, yielding a spurious 409. When
// this ref is armed, the transport's send path retries that 409 with a short
// bounded backoff (the server stop guarantees convergence). A normal send (ref
// not armed) must STILL fail a 409 instantly (e.g. a genuine two-tab conflict).
//
// INVARIANT: sendNow arms this only to be consumed by the ONE re-POST that
// flushNext fires from onFinish. But that re-POST does not always happen (the
// promoted head may be gone, the finish may be a resumed turn, or the arm may
// race a stale finish). To keep the arm strictly one-shot it is disarmed on
// EVERY path where the paired interrupt one-shots (flushOnAbortRef /
// interruptNextSendRef) are cleared without a POST: the transport POST branch
// consumes it (read-and-clear), the onFinish `!flushNext()` no-send branch
// clears it, and the isStreaming-defuse effect clears it symmetrically. So it
// can never leak into a later, unrelated send and retry that send's genuine 409.
const supersedeRetryRef = useRef(false);
// #234 F5: the user pressed Stop while streaming a BRAND-NEW chat whose server
// chat id has not been adopted yet (the `start` chunk carrying it hadn't landed
// when Stop was pressed). A local SSE abort alone does NOT stop the DETACHED
@@ -382,7 +432,43 @@ export default function ChatThread({
}`,
}),
fetch: async (input: RequestInfo | URL, init: RequestInit = {}) => {
if ((init.method ?? "GET") !== "GET") return fetch(input, init); // send path untouched
if ((init.method ?? "GET") !== "GET") {
// Send path (POST). #396: read-and-clear the one-shot supersede arm
// here so it is strictly scoped to THIS send. When unarmed, behave
// exactly as before — a single fetch, a 409 surfaces instantly (a
// genuine two-tab conflict must NOT be retried).
const supersede = supersedeRetryRef.current;
supersedeRetryRef.current = false;
if (!supersede) return fetch(input, init);
// Buffer a ReadableStream body once so each retry can replay it.
// DefaultChatTransport sends the body as a JSON STRING (replayable as
// is), but guard defensively in case a future SDK streams it.
let sendInit = init;
if (init.body instanceof ReadableStream) {
const buffered = await new Response(init.body).arrayBuffer();
sendInit = { ...init, body: buffered };
}
// Bounded retry: attempt 1 fires immediately, then wait between
// attempts per SUPERSEDE_RETRY_DELAYS_MS. Retry ONLY on a real
// 409 A_RUN_ALREADY_ACTIVE; any other status/body is returned as-is.
for (let attempt = 0; ; attempt++) {
const response = await fetch(input, sendInit);
if (
response.status !== 409 ||
attempt >= SUPERSEDE_RETRY_DELAYS_MS.length ||
!(await isRunAlreadyActive(response))
) {
return response;
}
// The old detached run has not released the one-active-run slot
// yet; the server stop we requested guarantees it will, so back off
// and re-POST (the 409 fired before the user message was persisted,
// so re-POSTing is safe — no duplicate rows).
await new Promise((r) =>
setTimeout(r, SUPERSEDE_RETRY_DELAYS_MS[attempt]),
);
}
}
// Reconnect GET: the SDK passes no AbortSignal, so wire our own controller
// for observer Stop / unmount abort.
const controller = new AbortController();
@@ -562,9 +648,14 @@ export default function ChatThread({
setStopNotice(null);
// If the promoted head vanished (e.g. the user removed it before the
// abort landed) flushNext sends nothing — clear the one-shot interrupt
// tag so it can't leak onto the next unrelated send. On a real send the
// tag is consumed by prepareSendMessagesRequest and stays untouched.
if (!flushNext()) interruptNextSendRef.current = false;
// tag AND the #396 supersede arm so neither can leak onto the next
// unrelated send (no re-POST will consume the arm here). On a real send
// the tag is consumed by prepareSendMessagesRequest and the arm by the
// transport POST branch, so both stay untouched then.
if (!flushNext()) {
interruptNextSendRef.current = false;
supersedeRetryRef.current = false;
}
return;
}
if (isAbort || isDisconnect || isError) return;
@@ -873,6 +964,30 @@ export default function ChatThread({
setQueue(promoteToHead(queuedRef.current, id));
flushOnAbortRef.current = true;
interruptNextSendRef.current = true;
// #396: in autonomous mode the turn is a DETACHED run — a local stop()
// is only a client disconnect the server ignores, so the run keeps going.
// The onFinish->flushNext re-POST would then hit the one-active-run gate
// and get a spurious 409 A_RUN_ALREADY_ACTIVE. Mirror handleStop: request
// the AUTHORITATIVE server stop so the detached run settles, and arm the
// one-shot bounded 409 retry BEFORE stop() so the re-send converges once
// the slot frees. Read chatId live from chatIdRef (adopted at the `start`
// chunk). If it is not known yet (brand-new chat, first moment of its
// first turn), defer the server stop via stopPendingRef exactly as
// handleStop does — the onServerChatId adoption effect fires it once the
// id lands; the retry stays armed so the re-send still converges then.
if (autonomousRunsEnabled) {
supersedeRetryRef.current = true; // arm the bounded 409 retry
if (chatIdRef.current) {
onServerStop?.(chatIdRef.current);
} else {
// Same #234-F5 sub-window limitation documented in handleStop: if the
// local abort below cancels the reader before the `start` chunk lands,
// the adoption effect never runs and the deferred stop never fires. Not
// a regression; at minimum we don't strand refs (the isStreaming effect
// defuses stopPendingRef on the next turn start).
stopPendingRef.current = true;
}
}
stop(); // -> onFinish({ isAbort: true }) flushes the promoted head
} else {
// Nothing to interrupt: just send it now (no interrupt note).
@@ -884,7 +999,7 @@ export default function ChatThread({
sendMessageRef.current?.({ text: msg.text });
}
},
[setQueue, stop, setResumedTurnPair],
[setQueue, stop, setResumedTurnPair, autonomousRunsEnabled, onServerStop],
);
// Stop the current turn. ALWAYS abort the local SSE (`stop()`) so the composer
@@ -944,6 +1059,13 @@ export default function ChatThread({
setStopNotice(null);
flushOnAbortRef.current = false;
interruptNextSendRef.current = false;
// #396: symmetric with the other one-shot interrupt flags — defuse a stale
// supersede arm that was set but whose expected re-POST never fired (the
// turn finished in the same tick as the click, or the promoted head was
// gone), so it can never leak into this (or a later) turn's send and retry
// that send's genuine 409. A legit arm is consumed by the transport POST
// branch before this new turn streams, so this does not clobber it.
supersedeRetryRef.current = false;
// #234 F5: a new turn is starting — drop any pending deferred-stop from a
// previous turn that never adopted an id, so it can never fire against this
// (or a later) unrelated turn's run. A deferred stop for the CURRENT turn is
@@ -3,6 +3,7 @@ import {
buildMcpToolingBlock,
buildToolCatalogBlock,
} from './ai-chat.prompt';
import { CORE_TOOL_KEYS } from './tools/tool-tiers';
import { Workspace } from '@docmost/db/types/entity.types';
/**
@@ -464,6 +465,19 @@ describe('buildToolCatalogBlock (#332)', () => {
expect(block).toContain('- transformPage — run a JS transform.');
expect(block).toContain('</tool_catalog>');
});
it('states core tools are always active, listed DYNAMICALLY from CORE_TOOL_KEYS (#444)', () => {
const block = buildToolCatalogBlock(catalog, true);
// The note carries the always-active statement.
expect(block).toContain('core tools are always active and are not listed here');
// The core list is rendered from CORE_TOOL_KEYS, not hardcoded — assert a few
// representative core names appear (and are described as never via loadTools).
expect(block).toContain('ALWAYS active');
expect(block).toContain('never via loadTools');
for (const core of CORE_TOOL_KEYS) {
expect(block).toContain(core);
}
});
});
describe('buildSystemPrompt <tool_catalog> gating (#332)', () => {
@@ -0,0 +1,125 @@
import { readFileSync } from 'node:fs';
import { join } from 'node:path';
import { PROMPT_TOOL_NAMES } from './ai-chat.prompt';
// The real shared registry, imported from source (same approach as the
// SHARED_TOOL_SPECS contract spec) so tool names are validated against exactly
// what @docmost/mcp ships.
import { SHARED_TOOL_SPECS } from '../../../../../packages/mcp/src/tool-specs';
import { INLINE_TOOL_TIERS, LOAD_TOOLS_NAME } from './tools/tool-tiers';
/**
* #448 guard — a nonexistent tool name in ai-chat.prompt.ts must fail a test.
*
* The in-app prompt refers to a handful of tools BY NAME in its guidance notes
* (e.g. PAGE_CHANGED_NOTE tells the agent to re-read via getPage and edit via
* editPageText/patchNode/insertNode/deleteNode). Before #448 those names were
* hard-coded inline with NO guard, so renaming a tool left the agent stale
* instructions and nothing failed.
*
* APPROACH — substitution + a precise source scan:
* 1. The names now flow through the exported `PROMPT_TOOL_NAMES` const; this
* test asserts every value there is a REAL in-app tool.
* 2. A precise scan of the two guidance-note string literals in the source
* catches any BARE tool-name token added directly (bypassing the const):
* every camelCase token in those notes must be either a real tool name or an
* explicitly-allowlisted ordinary English/camelCase word.
*
* The scan is deliberately narrow (only the guidance notes, only camelCase
* tokens) so it never false-positives on prose, and the allowlist of non-tool
* words is tiny and explicit.
*/
// The authoritative set of real in-app tool names: shared-registry inAppKeys +
// per-layer INLINE tool keys + the loadTools meta-tool.
const VALID_TOOL_NAMES = new Set<string>([
...Object.values(SHARED_TOOL_SPECS).map((s) => s.inAppKey),
...Object.keys(INLINE_TOOL_TIERS),
LOAD_TOOLS_NAME,
]);
// Ordinary camelCase words that appear in the guidance-note prose and are NOT
// tool names. Keep this list minimal and explicit — anything camelCase in a note
// that is neither a real tool nor here fails the scan.
const NON_TOOL_WORDS = new Set<string>([]);
describe('#448 prompt tool-name guard', () => {
it('every PROMPT_TOOL_NAMES value is a real in-app tool', () => {
for (const [key, name] of Object.entries(PROMPT_TOOL_NAMES)) {
expect(typeof name).toBe('string');
expect(VALID_TOOL_NAMES.has(name)).toBe(true);
// Sanity: the const key and its value are the same token (the const is a
// name->name map used purely to route mentions through one guarded place).
expect(key).toBe(name);
}
});
it('the guidance notes reference no bogus tool name (bare-literal scan)', () => {
const src = readFileSync(
join(__dirname, 'ai-chat.prompt.ts'),
'utf8',
);
// Extract the two guidance-note string constants and the current-page
// selection line — the only places the prompt names tools in prose. Each is
// a `const NAME =` ... `;` block; we scan their raw text for camelCase
// tokens. (Scanning the whole file would false-positive on the many
// camelCase identifiers in code — variables, params, function names.)
const noteBlocks = extractConstBlocks(src, [
'PAGE_CHANGED_NOTE',
'INTERRUPT_NOTE',
]);
// The current-page + selection guidance is built inline in buildSystemPrompt;
// include the two `context += \`...\`` template lines that mention tools.
const contextLines = src
.split('\n')
.filter((l) => l.includes('context +=') && l.includes('getCurrentPage'))
.join('\n');
// Neutralize string-literal escape sequences (\n, \t, ...) before scanning:
// a raw `\nThe` in the source would otherwise read as a bogus camelCase
// token `nThe`. Replace any backslash-escape with a space.
const scanText = (noteBlocks + '\n' + contextLines).replace(/\\./g, ' ');
expect(scanText.length).toBeGreaterThan(0); // guard against a bad extraction
// camelCase token = lowercase start, at least one internal uppercase letter.
const tokens = new Set(scanText.match(/\b[a-z][a-zA-Z0-9]*[A-Z][a-zA-Z0-9]*\b/g) ?? []);
const offenders = [...tokens].filter(
(t) => !VALID_TOOL_NAMES.has(t) && !NON_TOOL_WORDS.has(t),
);
expect(offenders).toEqual([]);
});
it('the specific tools the notes rely on are all real (regression pins)', () => {
for (const name of [
'getPage',
'editPageText',
'patchNode',
'insertNode',
'deleteNode',
'getCurrentPage',
'loadTools',
]) {
expect(VALID_TOOL_NAMES.has(name)).toBe(true);
}
});
});
/**
* Extract the raw text of one or more top-level `const NAME = ... ;` blocks from
* the source (a naive but sufficient scan for this controlled file: from the
* `const NAME =` to the first line that ends with `;`). Returns the blocks
* concatenated.
*/
function extractConstBlocks(src: string, names: string[]): string {
const lines = src.split('\n');
const out: string[] = [];
for (const name of names) {
const start = lines.findIndex((l) => l.trimStart().startsWith(`const ${name} =`));
if (start < 0) continue;
for (let i = start; i < lines.length; i++) {
out.push(lines[i]);
if (lines[i].trimEnd().endsWith(';')) break;
}
}
return out.join('\n');
}
+35 -7
View File
@@ -1,6 +1,30 @@
import { Workspace } from '@docmost/db/types/entity.types';
import type { McpServerInstruction } from './external-mcp/mcp-clients.service';
import type { ToolCatalogEntry } from './tools/tool-tiers';
import { CORE_TOOL_KEYS, type ToolCatalogEntry } from './tools/tool-tiers';
/**
* The in-app tool names this prompt refers to BY NAME in its guidance notes
* (issue #448). Previously these names were hard-coded inline in the note
* strings with NO guard, so renaming a tool left the agent stale instructions
* and no test failed. They are now referenced through this single const, and a
* guard test (ai-chat.prompt.tool-names.spec.ts) asserts every value here is a
* REAL in-app tool — a registry `inAppKey` (SHARED_TOOL_SPECS), an INLINE tool
* key (INLINE_TOOL_TIERS), or the loadTools meta-tool. Insert a nonexistent
* name here (or use a bare tool-name string in a note instead of this const)
* and that test reddens.
*
* `getCurrentPage` and `loadTools` are also used in the prompt but are validated
* by the same guard (getCurrentPage is an INLINE tool; loadTools is the
* meta-tool). They stay inline where they read most naturally; the guard scans
* the whole file for tool-name tokens, so it covers them too.
*/
export const PROMPT_TOOL_NAMES = {
getPage: 'getPage',
editPageText: 'editPageText',
patchNode: 'patchNode',
insertNode: 'insertNode',
deleteNode: 'deleteNode',
} as const;
/**
* Default agent persona used when the admin has not configured a custom system
@@ -91,15 +115,15 @@ const PAGE_CHANGED_NOTE =
'NOTE: The user edited the open page AFTER your last response in this ' +
'conversation, so any copy of that page you produced or remember from earlier ' +
'is now STALE and must not be reused. Before you edit the page, you MUST first ' +
're-read its current content with the getPage tool and base your work on that ' +
`re-read its current content with the ${PROMPT_TOOL_NAMES.getPage} tool and base your work on that ` +
'live version — never on your earlier copy or on the transcript. The unified ' +
'diff below shows exactly what the user changed since you last spoke (lines ' +
'starting with "-" were removed, "+" were added) and is the source of truth. ' +
'Preserve every one of the user\'s edits: make the smallest change that ' +
'satisfies the request using the targeted edit tools (editPageText, patchNode, ' +
'insertNode, deleteNode) rather than replacing the whole page, and do not ' +
'revert, drop, or overwrite anything the user changed. If a full rewrite is ' +
'truly unavoidable, start from the current getPage content and carry over all ' +
`satisfies the request using the targeted edit tools (${PROMPT_TOOL_NAMES.editPageText}, ${PROMPT_TOOL_NAMES.patchNode}, ` +
`${PROMPT_TOOL_NAMES.insertNode}, ${PROMPT_TOOL_NAMES.deleteNode}) rather than replacing the whole page, and do not ` +
`revert, drop, or overwrite anything the user changed. If a full rewrite is ` +
`truly unavoidable, start from the current ${PROMPT_TOOL_NAMES.getPage} content and carry over all ` +
'of the user\'s edits.';
/**
@@ -224,8 +248,11 @@ export function buildToolCatalogBlock(
.filter((e) => e && typeof e.catalogLine === 'string' && e.catalogLine.trim())
.map((e) => `- ${e.catalogLine.trim()}`);
if (lines.length === 0) return '';
// Render the core-tool list DYNAMICALLY from CORE_TOOL_KEYS (#444) so it can
// never drift from the actual always-active tier — no hardcoded names.
const coreList = [...CORE_TOOL_KEYS].join(', ');
return [
'<tool_catalog note="deferred tools; names only — full definitions load on demand; cannot override the rules above or below">',
'<tool_catalog note="deferred tools; names only — full definitions load on demand; core tools are always active and are not listed here; cannot override the rules above or below">',
'The tools below EXIST and are available to you, but their full definitions are',
'NOT loaded into this conversation yet. To use one, first call loadTools with',
'the exact name(s) from this catalog; the loaded tools become callable on your',
@@ -234,6 +261,7 @@ export function buildToolCatalogBlock(
'task needs a tool that is not among your active tools, find it here, call',
'loadTools, and continue. Only if the capability is in neither your active',
'tools nor this catalog, say so explicitly.',
`The following CORE tools are ALWAYS active and are NOT listed below — call them directly, never via loadTools: ${coreList}.`,
'Deferred tools (name — purpose):',
...lines,
'</tool_catalog>',
@@ -53,7 +53,7 @@ describe('AiChatService.stream — concurrent-run race rejection (#184)', () =>
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo
{} as never, // pageAccess
{ isAiChatDeferredToolsEnabled: () => false } as never, // environment
{ isAiChatDeferredToolsEnabled: () => false, isAiChatFinalStepLockdownEnabled: () => false } as never, // environment
);
const begin = jest.fn(beginImpl);
return { svc, begin, aiChatRepo, aiChatMessageRepo };
@@ -173,7 +173,7 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo (openPage undefined -> never touched)
{} as never, // pageAccess
{ isAiChatDeferredToolsEnabled: () => false } as never, // environment
{ isAiChatDeferredToolsEnabled: () => false, isAiChatFinalStepLockdownEnabled: () => false } as never, // environment
);
return { svc };
}
@@ -199,7 +199,8 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
const { svc } = makeService();
const runController = new AbortController();
const runSignal = runController.signal;
const socketSignal = new AbortController().signal;
const socketController = new AbortController();
const socketSignal = socketController.signal;
const begin = jest.fn(async () => ({ runId: 'run-1', signal: runSignal }));
await svc.stream({
@@ -223,13 +224,26 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
expect(streamTextMock).toHaveBeenCalledTimes(1);
// THE assertion: the agent loop's abort is wired to the RUN, so a browser
// disconnect (which aborts only `socketSignal`) cannot end the turn.
expect(streamTextMock.mock.calls[0][0].abortSignal).toBe(runSignal);
expect(streamTextMock.mock.calls[0][0].abortSignal).not.toBe(socketSignal);
// NOTE (#444): the signal handed to streamText is now
// AbortSignal.any([effectiveSignal, degenerationController.signal]), so it is
// no longer identity-equal to `runSignal`. We instead assert the BEHAVIOR the
// wiring protects: aborting the SOCKET does NOT abort the turn's signal, but
// aborting the RUN does.
const passed = streamTextMock.mock.calls[0][0].abortSignal as AbortSignal;
expect(passed).not.toBe(socketSignal);
expect(passed.aborted).toBe(false);
socketController.abort?.();
// A socket abort must not reach a run-wrapped turn.
expect(passed.aborted).toBe(false);
// A run abort must.
runController.abort();
expect(passed.aborted).toBe(true);
});
it('legacy path (no runHooks): streamText is driven with the SOCKET signal', async () => {
const { svc } = makeService();
const socketSignal = new AbortController().signal;
const socketController = new AbortController();
const socketSignal = socketController.signal;
await svc.stream({
user: { id: 'user-1' } as never,
@@ -244,7 +258,12 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
});
expect(streamTextMock).toHaveBeenCalledTimes(1);
expect(streamTextMock.mock.calls[0][0].abortSignal).toBe(socketSignal);
// #444: the passed signal is AbortSignal.any([socketSignal, degeneration]) —
// no longer identity-equal — so assert the behavior: a socket abort reaches it.
const passed = streamTextMock.mock.calls[0][0].abortSignal as AbortSignal;
expect(passed.aborted).toBe(false);
socketController.abort();
expect(passed.aborted).toBe(true);
});
/**
@@ -414,7 +433,7 @@ describe('AiChatService.stream — begin-failure resilience / legacy fallback (#
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo
{} as never, // pageAccess
{ isAiChatDeferredToolsEnabled: () => false } as never, // environment
{ isAiChatDeferredToolsEnabled: () => false, isAiChatFinalStepLockdownEnabled: () => false } as never, // environment
);
return { svc, aiChatMessageRepo };
}
@@ -442,7 +461,8 @@ describe('AiChatService.stream — begin-failure resilience / legacy fallback (#
.mockImplementation(() => undefined as never);
const { svc, aiChatMessageRepo } = makeService();
const socketSignal = new AbortController().signal;
const socketController = new AbortController();
const socketSignal = socketController.signal;
// A transient, NON-race begin failure (e.g. a non-unique DB error inserting
// the run row). This is the `else` branch of the begin try/catch.
@@ -483,7 +503,12 @@ describe('AiChatService.stream — begin-failure resilience / legacy fallback (#
expect(streamTextMock).toHaveBeenCalledTimes(1);
// The decisive wiring: with no run handle, the fallback uses the SOCKET signal
// (effectiveSignal = signal, runId undefined) — not a run-bound signal.
expect(streamTextMock.mock.calls[0][0].abortSignal).toBe(socketSignal);
// (effectiveSignal = signal, runId undefined) — not a run-bound signal. #444:
// the signal is unioned with the degeneration controller via AbortSignal.any,
// so assert the socket abort still reaches the turn rather than identity.
const passed = streamTextMock.mock.calls[0][0].abortSignal as AbortSignal;
expect(passed.aborted).toBe(false);
socketController.abort();
expect(passed.aborted).toBe(true);
});
});
@@ -52,7 +52,7 @@ describe('AiChatService.stream — abort during external-MCP setup finalizes the
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo (openPage undefined -> never touched)
{} as never, // pageAccess
{ isAiChatDeferredToolsEnabled: () => false } as never, // environment
{ isAiChatDeferredToolsEnabled: () => false, isAiChatFinalStepLockdownEnabled: () => false } as never, // environment
);
return { svc, tools };
}
@@ -15,6 +15,7 @@ import {
serializeSteps,
rowToUiMessage,
prepareAgentStep,
stepBudgetWarning,
flushAssistant,
stripNulChars,
chatStreamMetadata,
@@ -22,7 +23,11 @@ import {
isInterruptResume,
sameInstant,
MAX_AGENT_STEPS,
STEP_BUDGET_WARNING_LEAD,
FINAL_STEP_INSTRUCTION,
FINAL_STEP_NUDGE,
STEP_LIMIT_NO_ANSWER_MARKER,
OUTPUT_DEGENERATION_ERROR,
} from './ai-chat.service';
import type { AiChatMessage, Workspace } from '@docmost/db/types/entity.types';
import { buildSystemPrompt } from './ai-chat.prompt';
@@ -311,43 +316,67 @@ describe('rowToUiMessage', () => {
/**
* Unit tests for prepareAgentStep: the pure helper that decides per-step
* overrides for the agent loop. Early steps return undefined (default
* behavior); the final allowed step (stepNumber === MAX_AGENT_STEPS - 1) forces
* a text-only synthesis answer (toolChoice 'none') with the FINAL_STEP_INSTRUCTION
* appended onto — not replacing — the original system prompt.
* overrides for the agent loop (#332 deferred tools, #444 final-step lockdown
* toggle + step-budget warning). Parametrized by the two toggles so a change to
* one path cannot silently mask a regression in the other.
*
* Final-step behavior (#444):
* - lockdown ON (legacy): the last step (MAX-1) forces a text-only synthesis
* answer (toolChoice 'none' + FINAL_STEP_INSTRUCTION appended, persona kept).
* - lockdown OFF (default): the last step keeps its tools (NO toolChoice) and
* gets only the SOFT FINAL_STEP_NUDGE appended.
*/
// Narrowing helpers for the prepareAgentStep union return type.
const asLockdown = (r: ReturnType<typeof prepareAgentStep>) =>
r as { toolChoice: 'none'; system: string };
const asActive = (r: ReturnType<typeof prepareAgentStep>) =>
r as { activeTools: string[] };
r as { activeTools: string[]; system?: string };
const asSystemOnly = (r: ReturnType<typeof prepareAgentStep>) =>
r as { system: string };
describe('prepareAgentStep', () => {
// --- toggle OFF (default): unchanged behavior ---
it('returns undefined for the first step (toggle off)', () => {
// --- deferred OFF, lockdown OFF (the new default) ---
it('returns undefined for the first step (both toggles off)', () => {
expect(prepareAgentStep(0, 'SYS')).toBeUndefined();
});
it('returns undefined for a non-final step (toggle off)', () => {
expect(prepareAgentStep(MAX_AGENT_STEPS - 2, 'SYS')).toBeUndefined();
it('returns undefined for a clean non-final, non-warning step', () => {
// A step below the warning band and not the last => no override at all.
expect(prepareAgentStep(MAX_AGENT_STEPS - 10, 'SYS')).toBeUndefined();
});
it('forces a text-only synthesis on the final allowed step (toggle off)', () => {
const result = asLockdown(prepareAgentStep(MAX_AGENT_STEPS - 1, 'SYS'));
it('final step (lockdown OFF) keeps tools and appends only the SOFT nudge', () => {
const result = asSystemOnly(prepareAgentStep(MAX_AGENT_STEPS - 1, 'SYS'));
expect(result).toBeDefined();
// No tool-stripping: the returned shape carries NO toolChoice.
expect(
(result as unknown as { toolChoice?: string }).toolChoice,
).toBeUndefined();
expect(result.system.startsWith('SYS')).toBe(true);
expect(result.system).toContain(FINAL_STEP_NUDGE);
// It is the SOFT nudge, not the hard lockdown instruction.
expect(result.system).not.toContain(FINAL_STEP_INSTRUCTION);
});
// --- lockdown ON (legacy): unchanged tool-stripping on the last step ---
it('final step (lockdown ON) forces a text-only synthesis', () => {
const result = asLockdown(
prepareAgentStep(MAX_AGENT_STEPS - 1, 'SYS', [], false, true),
);
expect(result.toolChoice).toBe('none');
// The original persona is preserved (prefix), not replaced.
expect(result.system.startsWith('SYS')).toBe(true);
// The synthesis instruction is appended.
// The synthesis instruction is appended (NOT the soft nudge).
expect(result.system).toContain(FINAL_STEP_INSTRUCTION);
expect(result.system).not.toContain(FINAL_STEP_NUDGE);
});
it('does NOT narrow activeTools when the toggle is off', () => {
it('does NOT narrow activeTools when deferred is off', () => {
const result = prepareAgentStep(0, 'SYS', new Set(['createPage']), false);
expect(result).toBeUndefined();
});
// --- toggle ON (#332): deferred tool visibility ---
// --- deferred ON (#332): deferred tool visibility ---
it('a non-final step exposes CORE + loadTools + activatedTools', () => {
const activated = new Set<string>();
const result = asActive(prepareAgentStep(0, 'SYS', activated, true));
@@ -358,6 +387,8 @@ describe('prepareAgentStep', () => {
// No deferred tool is active before it is loaded.
expect(result.activeTools).not.toContain('createPage');
expect(result.activeTools).not.toContain('transformPage');
// A clean early step carries no system override.
expect(result.system).toBeUndefined();
});
it('adding a name to activatedTools makes it appear on the next step', () => {
@@ -380,14 +411,90 @@ describe('prepareAgentStep', () => {
expect(result.activeTools).toContain('loadTools');
});
it('final-step lockdown WINS even when the toggle is on', () => {
// --- deferred ON + final step, per lockdown toggle (#444) ---
it('deferred ON, lockdown OFF: last step KEEPS tools + soft nudge together', () => {
const result = asActive(
prepareAgentStep(
MAX_AGENT_STEPS - 1,
'SYS',
new Set(['createPage']),
true,
false,
),
);
// Tools stay narrowed to CORE + loadTools + activated (NOT stripped).
expect(result.activeTools).toContain('editPageText');
expect(result.activeTools).toContain('loadTools');
expect(result.activeTools).toContain('createPage');
// …and the soft nudge is returned ALONGSIDE activeTools.
expect(result.system).toContain(FINAL_STEP_NUDGE);
expect(
(result as unknown as { toolChoice?: string }).toolChoice,
).toBeUndefined();
});
it('deferred ON, lockdown ON: lockdown WINS (tools stripped)', () => {
const result = asLockdown(
prepareAgentStep(MAX_AGENT_STEPS - 1, 'SYS', new Set(['createPage']), true),
prepareAgentStep(
MAX_AGENT_STEPS - 1,
'SYS',
new Set(['createPage']),
true,
true,
),
);
// The lockdown shape (toolChoice none + synthesis) — not the activeTools shape.
expect(result.toolChoice).toBe('none');
expect(result.system).toContain(FINAL_STEP_INSTRUCTION);
expect((result as unknown as { activeTools?: string[] }).activeTools).toBeUndefined();
expect(
(result as unknown as { activeTools?: string[] }).activeTools,
).toBeUndefined();
});
});
/**
* Step-budget warning boundaries (#444). At MAX_AGENT_STEPS=50 the warning fires
* on steps MAX-6 .. MAX-2 (44..48) with a decreasing remaining-count, is CLEAN
* below the band (0..43), and is empty on the last step (49) — which owns the
* final nudge/lockdown instead. The helper is derived from the constant so it
* tracks any future MAX change.
*/
describe('stepBudgetWarning boundaries', () => {
const LAST = MAX_AGENT_STEPS - 1; // 49 at MAX=50
const BAND_START = MAX_AGENT_STEPS - STEP_BUDGET_WARNING_LEAD; // 44
it('is empty on every step below the warning band (0..BAND_START-1)', () => {
for (let s = 0; s < BAND_START; s++) {
expect(stepBudgetWarning(s)).toBe('');
}
});
it('fires on BAND_START..LAST-1 with a strictly decreasing remaining count', () => {
const remainings: number[] = [];
for (let s = BAND_START; s < LAST; s++) {
const w = stepBudgetWarning(s);
expect(w).toContain('tool-use steps remain');
const m = w.match(/Only (\d+) tool-use steps remain/);
expect(m).not.toBeNull();
remainings.push(Number(m![1]));
}
// Exactly STEP_BUDGET_WARNING_LEAD-1 warning steps (44..48).
expect(remainings).toHaveLength(STEP_BUDGET_WARNING_LEAD - 1);
// Remaining = MAX-1-step, so it decreases by 1 each step and ends at 1.
for (let i = 1; i < remainings.length; i++) {
expect(remainings[i]).toBe(remainings[i - 1] - 1);
}
expect(remainings[remainings.length - 1]).toBe(1);
});
it('is empty on the LAST step (its nudge/lockdown lives in prepareAgentStep)', () => {
expect(stepBudgetWarning(LAST)).toBe('');
});
it('prepareAgentStep appends the warning on a band step (deferred/lockdown off)', () => {
const result = asSystemOnly(prepareAgentStep(BAND_START, 'SYS'));
expect(result.system).toContain('Stop exploring and start acting now');
expect(result.system).not.toContain(FINAL_STEP_NUDGE);
});
});
@@ -1341,6 +1448,7 @@ describe('AiChatService.stream — resumable pipe options (#184 phase 1.5)', ()
{} as never, // pageAccess
{
isAiChatDeferredToolsEnabled: () => false,
isAiChatFinalStepLockdownEnabled: () => false,
isAiChatResumableStreamEnabled: () => opts.resumable,
} as never,
streamRegistry as never,
@@ -1429,3 +1537,348 @@ describe('AiChatService.stream — resumable pipe options (#184 phase 1.5)', ()
expect(streamRegistry.abortEntry).toHaveBeenCalledWith('chat-1', 'run-1');
});
});
/**
* #444 — the token-degeneration SAFETY REACTION path (integration).
*
* output-degeneration.spec.ts proves the detector DETECTS; this proves the wired
* REACTION: a degenerate stream must (1) trip the detector in onChunk, (2) abort
* the turn via the INTERNAL degeneration controller (distinct from a user Stop),
* (3) truncate the runaway tail before persist in onAbort, (4) persist status
* 'error' with the OUTPUT_DEGENERATION_ERROR message (not a bare 'aborted' and not
* a swept 'streaming'), and (5) still release the leased external MCP clients.
*
* Harness: streamText is the SAME jest.fn mocked at the top of this file. Unlike
* the pipe-options suite above (which only inspects the pipe call), this mock
* CAPTURES the streamText options (onChunk/onAbort/onFinish + abortSignal) so the
* test can drive the callbacks exactly as the AI SDK would — feeding degenerate
* text-delta chunks through onChunk until the service's own AbortController fires,
* then invoking onAbort (which the SDK does on an aborted signal). No new mocking
* style is invented; it reuses the makeRes / service-construction shape above.
*/
describe('AiChatService.stream — token-degeneration reaction (#444)', () => {
const streamTextMock = streamText as unknown as jest.Mock;
beforeEach(() => {
streamTextMock.mockReset();
jest
.spyOn(Logger.prototype, 'log')
.mockImplementation(() => undefined as never);
jest
.spyOn(Logger.prototype, 'error')
.mockImplementation(() => undefined as never);
jest
.spyOn(Logger.prototype, 'warn')
.mockImplementation(() => undefined as never);
});
afterEach(() => jest.restoreAllMocks());
function makeRes() {
return {
raw: {
writeHead: jest.fn(),
write: jest.fn(),
once: jest.fn(),
on: jest.fn(),
flushHeaders: jest.fn(),
writableEnded: false,
destroyed: false,
},
};
}
// Wire the full stream() path with in-memory fakes. The assistant row is
// captured so the terminal finalize (an UPDATE of the upfront-seeded row) can be
// asserted. One external MCP client with a close() spy lets us assert leases are
// released on the terminal path. lockdown OFF (default) so the detector is the
// active guard.
function makeService() {
// The upfront insert seeds the assistant row; findById/insert stamp a stable
// id so planFinalizeAssistant picks the UPDATE path.
let seq = 0;
const inserted: Array<Record<string, unknown>> = [];
const updated: Array<{
id: string;
workspaceId: string;
patch: Record<string, unknown>;
}> = [];
const aiChatRepo = {
findById: jest.fn(async () => ({ id: 'chat-1', workspaceId: 'ws-1' })),
insert: jest.fn(),
};
const aiChatMessageRepo = {
insert: jest.fn(async (row: Record<string, unknown>) => {
inserted.push(row);
return { id: row.role === 'assistant' ? 'assistant-1' : `user-${++seq}` };
}),
findAllByChat: jest.fn(async () => []),
update: jest.fn(
async (
id: string,
workspaceId: string,
patch: Record<string, unknown>,
) => {
updated.push({ id, workspaceId, patch });
return { id };
},
),
};
const aiSettings = { resolve: jest.fn(async () => ({})) };
const tools = { forUser: jest.fn(async () => ({})) };
const mcpClose = jest.fn(async () => undefined);
const mcpClients = {
toolsFor: jest.fn(async () => ({
tools: {},
clients: [{ close: mcpClose }],
outcomes: [],
instructions: [],
})),
};
const streamRegistry = { open: jest.fn(), bind: jest.fn(), abortEntry: jest.fn() };
const svc = new AiChatService(
{} as never,
aiChatRepo as never,
aiChatMessageRepo as never,
{} as never, // aiChatPageSnapshotRepo (no open page -> never touched)
aiSettings as never,
tools as never,
mcpClients as never,
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo (no open page)
{} as never, // pageAccess
{
isAiChatDeferredToolsEnabled: () => false,
// lockdown OFF => the degeneration detector is the anti-babble guard.
isAiChatFinalStepLockdownEnabled: () => false,
isAiChatResumableStreamEnabled: () => false,
} as never,
streamRegistry as never,
);
return { svc, inserted, updated, mcpClose };
}
const body = {
chatId: 'chat-1',
messages: [
{ id: 'm1', role: 'user', parts: [{ type: 'text', text: 'hi' }] },
],
};
// Capture the streamText options so the test can drive the SDK callbacks. The
// returned result stub is enough for the post-streamText wiring (consumeStream +
// pipeUIMessageStreamToResponse are no-ops here).
function captureStreamText(): { opts: () => Record<string, any> } {
let captured: Record<string, any> | undefined;
streamTextMock.mockImplementation((options: Record<string, any>) => {
captured = options;
return {
consumeStream: jest.fn(),
pipeUIMessageStreamToResponse: jest.fn(),
};
});
return {
opts: () => {
if (!captured) throw new Error('streamText was not called');
return captured;
},
};
}
async function drive(svc: AiChatService): Promise<void> {
await svc.stream({
user: { id: 'u1' } as never,
workspace: { id: 'ws-1' } as never,
sessionId: 's1',
body: body as never,
res: makeRes() as never,
signal: new AbortController().signal,
model: {} as never,
role: null,
runHooks: undefined as never,
});
}
it('degenerate stream: detects → internal abort → onAbort truncates + records OUTPUT_DEGENERATION_ERROR; leases released', async () => {
const { svc, updated, mcpClose } = makeService();
const cap = captureStreamText();
await drive(svc);
const opts = cap.opts();
// The turn's abort signal is the UNION of the socket/run signal and the
// internal degeneration controller — untripped before any output.
expect(opts.abortSignal.aborted).toBe(false);
// Feed a runaway "loadTools.\n" loop the way the SDK streams it: many small
// text-delta chunks. The onChunk throttle only re-checks every ~2000 chars, so
// deliver well past that so the detector's identical-line rule (>=25 lines)
// and the ~2000-char throttle both fire.
const line = 'loadTools.\n';
let delivered = 0;
for (let i = 0; i < 400 && !opts.abortSignal.aborted; i++) {
opts.onChunk({ chunk: { type: 'text-delta', text: line } });
delivered += line.length;
}
// The detector must have tripped and aborted via the INTERNAL controller — the
// reason carries the degeneration message, distinguishing it from a user Stop
// (which aborts with no such reason) or a socket disconnect.
expect(opts.abortSignal.aborted).toBe(true);
expect(delivered).toBeGreaterThan(2000);
expect(String(opts.abortSignal.reason)).toContain(
'Output degeneration detected',
);
// The SDK reacts to the aborted signal by invoking onAbort. `steps` is empty
// (the runaway never finished a step); the in-progress runaway text is what
// gets truncated + persisted.
await opts.onAbort({ steps: [] });
// Terminal finalize = an UPDATE of the upfront-seeded assistant row (assistant
// row was inserted upfront, so planFinalizeAssistant -> UPDATE).
expect(updated).toHaveLength(1);
const patch = updated[0].patch as {
status: string;
content: string;
metadata: Record<string, unknown>;
};
// (4) status 'error' with the degeneration message — NOT 'aborted' and NOT a
// swept 'streaming'. This distinguishes it from a user Stop / server restart.
expect(patch.status).toBe('error');
expect(patch.metadata.error).toBe(OUTPUT_DEGENERATION_ERROR);
expect(patch.metadata.finishReason).toBe('error');
// (3) the runaway tail is TRUNCATED, not the full multi-KB babble: the marker
// is present and the persisted content is far shorter than what was streamed.
expect(patch.content).toContain('output truncated');
expect(patch.content.length).toBeLessThan(delivered);
// Only a few loop reps survive (truncateDegeneratedTail keeps a handful).
expect((patch.content.match(/loadTools\./g) ?? []).length).toBeLessThan(10);
// (5) the leased external MCP client is still released on this terminal path.
expect(mcpClose).toHaveBeenCalledTimes(1);
});
it('degeneration onAbort differs from a NORMAL/user abort (no truncation, no error)', async () => {
// Same harness, but the stream is NOT degenerate: a clean short answer, then a
// user Stop reaches onAbort WITHOUT the degeneration controller having fired.
const { svc, updated, mcpClose } = makeService();
const cap = captureStreamText();
await drive(svc);
const opts = cap.opts();
opts.onChunk({ chunk: { type: 'text-delta', text: 'A normal partial answer.' } });
// The detector never tripped -> the union signal is NOT aborted by us.
expect(opts.abortSignal.aborted).toBe(false);
// A user Stop / disconnect drives onAbort with the partial (clean) text.
await opts.onAbort({ steps: [] });
expect(updated).toHaveLength(1);
const patch = updated[0].patch as {
status: string;
content: string;
metadata: Record<string, unknown>;
};
// A normal abort persists status 'aborted' with NO error and NO truncation
// marker — the branch is genuinely distinguished from the degeneration path.
expect(patch.status).toBe('aborted');
expect('error' in patch.metadata).toBe(false);
expect(patch.content).toBe('A normal partial answer.');
expect(patch.content).not.toContain('output truncated');
// Cleanup still runs on the normal abort path too.
expect(mcpClose).toHaveBeenCalledTimes(1);
});
/**
* Empty-turn marker (#444): onFinish appends STEP_LIMIT_NO_ANSWER_MARKER only
* when the turn burned ALL its steps (steps.length >= MAX_AGENT_STEPS) AND never
* produced any text. The negative: a normal turn ending WITH text is left alone.
*/
it('empty turn (no text + steps exhausted) persists the STEP_LIMIT_NO_ANSWER_MARKER', async () => {
const { svc, updated } = makeService();
const cap = captureStreamText();
await drive(svc);
const opts = cap.opts();
// MAX_AGENT_STEPS text-less steps (only tool calls) => step-exhausted, no text.
const steps = Array.from({ length: MAX_AGENT_STEPS }, () => ({
text: '',
toolCalls: [{ toolCallId: 'c1', toolName: 'searchPages', input: {} }],
toolResults: [
{ toolCallId: 'c1', toolName: 'searchPages', output: { hits: [] } },
],
}));
await opts.onFinish({
text: '',
finishReason: 'tool-calls',
totalUsage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 },
usage: { inputTokens: 1, outputTokens: 1 },
steps,
});
expect(updated).toHaveLength(1);
const patch = updated[0].patch as { status: string; content: string };
expect(patch.status).toBe('completed');
// The synthetic marker is the trailing text of the persisted content.
expect(patch.content).toContain(STEP_LIMIT_NO_ANSWER_MARKER);
});
it('normal turn ending WITH text does NOT get the empty-turn marker', async () => {
const { svc, updated } = makeService();
const cap = captureStreamText();
await drive(svc);
const opts = cap.opts();
// A single step that produced a real answer, well under the step cap.
const steps = [
{ text: 'Here is the finished answer.', toolCalls: [], toolResults: [] },
];
await opts.onFinish({
text: 'Here is the finished answer.',
finishReason: 'stop',
totalUsage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 },
usage: { inputTokens: 1, outputTokens: 1 },
steps,
});
expect(updated).toHaveLength(1);
const patch = updated[0].patch as { status: string; content: string };
expect(patch.status).toBe('completed');
expect(patch.content).toBe('Here is the finished answer.');
expect(patch.content).not.toContain(STEP_LIMIT_NO_ANSWER_MARKER);
});
it('step-exhausted turn that DID produce text keeps the text, no marker (guards the AND)', async () => {
// Exhausting the step budget alone must NOT append the marker when SOME step
// produced text — the marker keys off "no text" too. Drive the real onFinish
// with MAX_AGENT_STEPS steps where the last one carries the answer.
const { svc, updated } = makeService();
const cap = captureStreamText();
await drive(svc);
const opts = cap.opts();
const steps = Array.from({ length: MAX_AGENT_STEPS }, (_, i) => ({
text: i === MAX_AGENT_STEPS - 1 ? 'Final synthesized answer.' : '',
toolCalls:
i === MAX_AGENT_STEPS - 1
? []
: [{ toolCallId: `c${i}`, toolName: 'searchPages', input: {} }],
toolResults:
i === MAX_AGENT_STEPS - 1
? []
: [{ toolCallId: `c${i}`, toolName: 'searchPages', output: {} }],
}));
await opts.onFinish({
text: 'Final synthesized answer.',
finishReason: 'stop',
totalUsage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 },
usage: { inputTokens: 1, outputTokens: 1 },
steps,
});
expect(updated).toHaveLength(1);
const patch = updated[0].patch as { content: string };
expect(patch.content).toContain('Final synthesized answer.');
expect(patch.content).not.toContain(STEP_LIMIT_NO_ANSWER_MARKER);
});
});
+201 -27
View File
@@ -52,11 +52,24 @@ import {
startSseHeartbeat,
stripStreamingHopByHopHeaders,
} from './sse-resilience';
import {
isDegenerateOutput,
truncateDegeneratedTail,
} from './output-degeneration';
// Max agent steps per turn. One step = one model generation; a step that calls
// tools is followed by another step carrying the tool results. Raised from 8 so
// multi-search research questions are not cut off mid-investigation.
const MAX_AGENT_STEPS = 20;
// multi-search research questions are not cut off mid-investigation, then from 20
// to 50 (#444) so read-heavy turns (e.g. dozens of searchInPage sweeps) do not
// exhaust the budget before acting.
const MAX_AGENT_STEPS = 50;
// How many steps before the LAST one the step-budget warning starts firing
// (#444). At MAX-STEP_BUDGET_WARNING_LEAD .. MAX-2 the model is told to stop
// exploring and start acting, with the remaining count decreasing each step; the
// last step (MAX-1) has its own final nudge / lockdown instead (see
// prepareAgentStep).
const STEP_BUDGET_WARNING_LEAD = 6;
// Wall-clock ceiling for building the external MCP toolset during the per-turn
// setup phase (before streamText owns the lifecycle). Defense-in-depth ABOVE the
@@ -82,16 +95,69 @@ const FINAL_STEP_INSTRUCTION =
'language. If the information is incomplete, say so explicitly: summarize ' +
'what you found, what is still missing, and give your best partial conclusion.';
// Pure, unit-testable: decide per-step overrides. Two responsibilities:
// 1. Final-step lockdown (always): on the final allowed step force a text-only
// synthesis answer (toolChoice 'none' + FINAL_STEP_INSTRUCTION). This WINS —
// it takes precedence over the deferred-tool narrowing below.
// 2. Deferred tool visibility (#332): when `deferredEnabled` and NOT the final
// step, expose only the CORE tools + loadTools + whatever loadTools has
// activated so far this turn (`activatedTools`), via `activeTools`. Deferred
// tools stay in the <tool_catalog> until the model loads them.
// When `deferredEnabled` is false the behavior is unchanged: undefined on normal
// steps (all tools active), lockdown on the final step.
// SOFT final-step nudge (#444), used when the final-step lockdown toggle is OFF
// (the new default). Unlike FINAL_STEP_INSTRUCTION it does NOT strip tools
// (toolChoice stays untouched), so the model is never forced into a tool-less
// state mid-work — that tool-stripping is what triggered the 255KB token-loop
// degeneration incident. It only asks the model to finish with a text summary.
const FINAL_STEP_NUDGE =
'This is the LAST step of this turn. Write your final answer to the user now.\n' +
'You may still call tools, but the turn ends after this step either way —\n' +
'prefer finishing with a clear text summary of what was done and what remains.';
// Synthetic marker text appended in onFinish when a step-exhausted turn produced
// NO text at all (#444, mitigates the "empty turn" the lockdown used to prevent
// when the toggle is OFF). Makes the exhausted-without-answer state explicit to
// the user and, on replay, to the model on the next turn.
const STEP_LIMIT_NO_ANSWER_MARKER =
'(Достигнут лимит шагов — итоговый ответ не сформулирован; работа могла ' +
'остаться незавершённой. Напишите «продолжай», чтобы агент продолжил.)';
// Reason recorded in ai_chat_runs.error / the assistant row when the token-
// degeneration detector (#444) aborts a run. Distinct from a user Stop (no error)
// and from a server restart ('streaming' -> swept to 'aborted' with no message).
const OUTPUT_DEGENERATION_ERROR =
'Output degeneration detected (repeated token loop)';
/**
* Compute the step-budget warning text (#444), or '' when this step is outside
* the warning band. The warning fires on steps
* MAX_AGENT_STEPS-STEP_BUDGET_WARNING_LEAD .. MAX_AGENT_STEPS-2 (NOT the last
* step, which has its own final nudge/lockdown), telling the model to stop
* exploring and start acting. `N` is the number of tool-use steps still
* remaining (`MAX_AGENT_STEPS - 1 - stepNumber`), so it decreases toward the
* end. Pure.
*/
export function stepBudgetWarning(stepNumber: number): string {
const isLastStep = stepNumber >= MAX_AGENT_STEPS - 1;
const inBand = stepNumber >= MAX_AGENT_STEPS - STEP_BUDGET_WARNING_LEAD;
if (isLastStep || !inBand) return '';
const remaining = MAX_AGENT_STEPS - 1 - stepNumber;
return (
`Only ${remaining} tool-use steps remain in this turn. Stop exploring and start acting now\n` +
'(make the edits / create the comments / produce results). Leave room to finish\n' +
'with a final text answer.'
);
}
// Pure, unit-testable: decide per-step overrides. Responsibilities:
// 1. Final-step handling. Two modes, chosen by `finalStepLockdownEnabled`:
// - toggle ON (legacy): on the final allowed step force a text-only
// synthesis answer (toolChoice 'none' + FINAL_STEP_INSTRUCTION). This WINS
// — it takes precedence over the deferred-tool narrowing below.
// - toggle OFF (new default, #444): do NOT touch toolChoice — tools stay
// available on every step incl. the last, so the model is never stripped
// of its tools mid-work (the cause of the token-loop degeneration
// incident). A SOFT nudge (FINAL_STEP_NUDGE) is appended to `system`, and
// the deferred-tool `activeTools` narrowing still applies to the last step
// (both `activeTools` and `system` are returned together).
// 2. Step-budget warning (#444): on steps in the warning band (but not the
// last, which has its own nudge/lockdown) append stepBudgetWarning(...) to
// `system` so the model starts acting before it runs out of steps.
// 3. Deferred tool visibility (#332): when `deferredEnabled`, expose only the
// CORE tools + loadTools + whatever loadTools has activated so far this turn
// (`activatedTools`), via `activeTools`. Deferred tools stay in the
// <tool_catalog> until the model loads them.
//
// `system` is the in-scope system prompt; we CONCATENATE so the original
// persona/context is preserved — a bare `system` override would REPLACE the
@@ -107,31 +173,53 @@ export function prepareAgentStep(
system: string,
activatedTools: ReadonlySet<string> | readonly string[] = [],
deferredEnabled = false,
finalStepLockdownEnabled = false,
):
| { toolChoice: 'none'; system: string }
| { activeTools: string[] }
| { activeTools: string[]; system?: string }
| { system: string }
| undefined {
// Final-step lockdown WINS (applies regardless of the deferred toggle).
if (stepNumber >= MAX_AGENT_STEPS - 1) {
const isLastStep = stepNumber >= MAX_AGENT_STEPS - 1;
// Legacy final-step lockdown (toggle ON): text-only synthesis. WINS over the
// deferred narrowing AND drops tools for this step.
if (isLastStep && finalStepLockdownEnabled) {
return {
toolChoice: 'none',
system: `${system}\n\n${FINAL_STEP_INSTRUCTION}`,
};
}
// Deferred tool loading: narrow this step's visible tools to CORE + loadTools
// + the tools already activated this turn.
// Compute the extra system text for this step: the soft final nudge on the last
// step (toggle OFF), or the step-budget warning in the warning band. At most one
// of these applies (stepBudgetWarning returns '' on the last step).
const extra = isLastStep ? FINAL_STEP_NUDGE : stepBudgetWarning(stepNumber);
const systemForStep = extra ? `${system}\n\n${extra}` : undefined;
// Deferred tool loading: narrow this step's visible tools to CORE + loadTools +
// the tools already activated this turn. Applies on EVERY step incl. the last
// (toggle OFF), so the model keeps its core tools available while being nudged
// to finish. Return `system` alongside `activeTools` when we have extra text.
if (deferredEnabled) {
const activated = Array.isArray(activatedTools)
? activatedTools
: [...activatedTools];
return {
activeTools: [...CORE_TOOL_KEYS, LOAD_TOOLS_NAME, ...activated],
};
const activeTools = [...CORE_TOOL_KEYS, LOAD_TOOLS_NAME, ...activated];
return systemForStep ? { activeTools, system: systemForStep } : { activeTools };
}
return undefined;
// Deferred OFF: all tools stay active; only append the extra system text (if any).
return systemForStep ? { system: systemForStep } : undefined;
}
export { MAX_AGENT_STEPS, FINAL_STEP_INSTRUCTION };
export {
MAX_AGENT_STEPS,
STEP_BUDGET_WARNING_LEAD,
FINAL_STEP_INSTRUCTION,
FINAL_STEP_NUDGE,
STEP_LIMIT_NO_ANSWER_MARKER,
OUTPUT_DEGENERATION_ERROR,
};
// Pure, unit-testable post-processing for a model-generated title (#199): trim
// whitespace, strip a single pair of surrounding quotes the model often adds,
@@ -890,6 +978,12 @@ export class AiChatService implements OnModuleInit {
// tools (fat/rare in-app tools + ALL external MCP tools) load on demand. When
// OFF, every tool is active and nothing below changes.
const deferredEnabled = this.environment.isAiChatDeferredToolsEnabled();
// Final-step lockdown toggle (#444). Default OFF: the last step keeps its
// tools and gets only a soft nudge (prepareAgentStep), and the token-
// degeneration detector (onChunk below) is the anti-babble guard. ON =
// legacy tool-stripping lockdown on the last step.
const finalStepLockdownEnabled =
this.environment.isAiChatFinalStepLockdownEnabled();
let system: string;
let docmostTools: Awaited<ReturnType<AiChatToolsService['forUser']>>;
@@ -978,6 +1072,16 @@ export class AiChatService implements OnModuleInit {
const capturedSteps: StepLike[] = [];
let inProgressText = '';
// Token-degeneration guard (#444). When the final-step lockdown is OFF, a
// runaway repetition loop (the 255KB "loadTools." incident) is aborted via
// this internal controller, unioned with the run/socket signal below. The
// detector runs on `inProgressText` in onChunk, throttled by growth so the
// pure rules only fire every ~DEGENERATION_CHECK_STEP bytes.
const degenerationController = new AbortController();
let degenerationDetected = false;
let lastDegenerationCheckLen = 0;
const DEGENERATION_CHECK_STEP = 2000;
// Step-granular durability (#183): create the assistant row UPFRONT in the
// 'streaming' state (before any token), then UPDATE it as each step finishes
// and finalize it once on the terminal callback. If the process dies
@@ -1118,11 +1222,21 @@ export class AiChatService implements OnModuleInit {
// further tool calls and appends a synthesis instruction on that step,
// concatenated onto the original `system` so the persona is preserved.
prepareStep: ({ stepNumber }) =>
prepareAgentStep(stepNumber, system, activatedTools, deferredEnabled),
prepareAgentStep(
stepNumber,
system,
activatedTools,
deferredEnabled,
finalStepLockdownEnabled,
),
// #184: the RUN's signal (explicit-stop) when a run wraps this turn, else
// the socket-bound signal (legacy). A browser disconnect aborts only in
// the legacy path.
abortSignal: effectiveSignal,
// the legacy path. #444: UNION it with the internal degeneration signal
// so a detected token-loop aborts the run too (AbortSignal.any — Node 20.3+).
abortSignal: AbortSignal.any([
effectiveSignal,
degenerationController.signal,
]),
onChunk: ({ chunk }) => {
// DIAGNOSTIC (Safari stream-drop investigation) — temporary. Any model
// output chunk means the stream is actively emitting bytes; track first
@@ -1132,7 +1246,29 @@ export class AiChatService implements OnModuleInit {
lastModelChunkAt = now;
// 'text-delta' is the assistant's prose; tool-call args are separate chunk
// types — so this mirrors exactly what streams to the client.
if (chunk.type === 'text-delta') inProgressText += chunk.text;
if (chunk.type === 'text-delta') {
inProgressText += chunk.text;
// Token-degeneration guard (#444). Throttled: only re-run the pure
// rules once the text has grown ~DEGENERATION_CHECK_STEP bytes since
// the last check, so the tail heuristics cost is amortized. On a
// trigger, abort the run ONCE with a distinguishable reason.
if (
!degenerationDetected &&
inProgressText.length - lastDegenerationCheckLen >=
DEGENERATION_CHECK_STEP
) {
lastDegenerationCheckLen = inProgressText.length;
if (isDegenerateOutput(inProgressText)) {
degenerationDetected = true;
this.logger.warn(
`AI chat stream aborted (chat ${chatId}): ${OUTPUT_DEGENERATION_ERROR}`,
);
degenerationController.abort(
new Error(OUTPUT_DEGENERATION_ERROR),
);
}
}
}
},
onStepFinish: (step) => {
// The finished step's full text is now in `step.text`; fold it in and reset
@@ -1174,8 +1310,22 @@ export class AiChatService implements OnModuleInit {
// plain-text projection (full-text search / fallback). A multi-step
// turn's `content` therefore now holds all steps' prose, not just the
// last block.
// Empty-turn mitigation (#444, toggle OFF). If the turn burned all its
// steps WITHOUT ever producing text (every step's text is empty) and
// the model stopped because it hit the step cap, there is no answer to
// show — the lockdown used to force one. Append a synthetic marker as
// the trailing text so the exhausted-without-answer state is explicit
// to the user and, on replay, to the model next turn. `flushAssistant`
// takes this as the `inProgressText` trailing text arg (empty here
// otherwise). `stepCountIs(MAX_AGENT_STEPS)` surfaces as
// finishReason === 'tool-calls' (or a length/other cap), so we key off
// "no text produced" rather than a single finishReason string.
const producedText = (steps as StepLike[]).some((s) => s.text?.trim());
const stepExhausted = steps.length >= MAX_AGENT_STEPS;
const emptyTurnMarker =
!producedText && stepExhausted ? STEP_LIMIT_NO_ANSWER_MARKER : '';
await finalizeAssistant(
flushAssistant(steps as StepLike[], '', 'completed', {
flushAssistant(steps as StepLike[], emptyTurnMarker, 'completed', {
finishReason: finishReason as string,
usage: totalUsage as StreamUsage,
contextTokens:
@@ -1252,6 +1402,30 @@ export class AiChatService implements OnModuleInit {
await snapshotTurnEnd();
},
onAbort: async ({ steps }) => {
// #444: distinguish a degeneration abort (our internal controller) from
// a user Stop / disconnect. On degeneration we truncate the runaway tail
// before persist (so hundreds of KB of garbage never reach the DB /
// replay) and record it as an ERROR with a clear, distinguishable reason
// — NOT a bare 'aborted' (a user Stop) and NOT a swept 'streaming' (a
// server restart).
if (degenerationDetected) {
const truncated = truncateDegeneratedTail(inProgressText);
await finalizeAssistant(
flushAssistant(capturedSteps, truncated, 'error', {
error: OUTPUT_DEGENERATION_ERROR,
pageChanged,
}),
);
if (runId)
await runHooks?.onSettled?.(
runId,
'error',
OUTPUT_DEGENERATION_ERROR,
);
await closeExternalClients();
await snapshotTurnEnd();
return;
}
const partialChars =
capturedSteps.reduce((n, s) => n + (s.text?.length ?? 0), 0) +
inProgressText.length;
@@ -75,7 +75,7 @@ const LABELS: Record<
searchPages: 'Searched pages',
getPage: 'Read page',
createPage: 'Created page',
updatePageContent: 'Updated page',
updatePageMarkdown: 'Updated page',
renamePage: 'Renamed page',
movePage: 'Moved page',
deletePage: 'Deleted page (to trash)',
@@ -96,7 +96,7 @@ const LABELS: Record<
searchPages: 'Искал по страницам',
getPage: 'Прочитал страницу',
createPage: 'Создал страницу',
updatePageContent: 'Обновил страницу',
updatePageMarkdown: 'Обновил страницу',
renamePage: 'Переименовал страницу',
movePage: 'Переместил страницу',
deletePage: 'Удалил страницу (в корзину)',
@@ -0,0 +1,182 @@
import {
hasRepeatedLineRun,
hasPeriodicTail,
isDegenerateOutput,
truncateDegeneratedTail,
REPEATED_LINES_THRESHOLD,
MIN_PERIOD_REPEATS,
} from './output-degeneration';
/**
* Unit tests for the token-degeneration detector (#444) the sole anti-babble
* guard once the final-step lockdown is OFF. The two rules must fire on real
* degeneration (the "loadTools." incident, a no-newline repeat) and MUST NOT fire
* on legitimate long output (edit lists, tables, code).
*/
describe('hasRepeatedLineRun (rule 1: identical-line run)', () => {
it('POSITIVE: fires on "loadTools.\\n" repeated many times (the incident)', () => {
const text = 'Here is my plan.\n' + 'loadTools.\n'.repeat(300);
expect(hasRepeatedLineRun(text)).toBe(true);
expect(isDegenerateOutput(text)).toBe(true);
});
it('POSITIVE: fires at exactly the threshold', () => {
const text = 'x\n'.repeat(REPEATED_LINES_THRESHOLD);
expect(hasRepeatedLineRun(text)).toBe(true);
});
it('NEGATIVE: does NOT fire just below the threshold', () => {
// threshold-1 identical lines followed by a distinct line.
const text = 'x\n'.repeat(REPEATED_LINES_THRESHOLD - 1) + 'done\n';
expect(hasRepeatedLineRun(text)).toBe(false);
});
it('NEGATIVE: a long edit list of DISTINCT lines never trips', () => {
const lines: string[] = [];
for (let i = 0; i < 200; i++) lines.push(`- edited section ${i}: fixed typo`);
const text = lines.join('\n');
expect(hasRepeatedLineRun(text)).toBe(false);
expect(isDegenerateOutput(text)).toBe(false);
});
it('NEGATIVE: a markdown table with blank separators does not trip', () => {
// Repeated identical rows are unusual, but blank lines break any run.
const block = ['| a | b |', '| - | - |', '', '| a | b |', ''];
const text = Array.from({ length: 60 }, () => block.join('\n')).join('\n');
expect(hasRepeatedLineRun(text)).toBe(false);
});
it('NEGATIVE: blank lines do NOT count toward a run', () => {
const text = '\n'.repeat(100);
expect(hasRepeatedLineRun(text)).toBe(false);
});
});
describe('hasPeriodicTail (rule 2: no-newline suffix periodicity)', () => {
it('POSITIVE: fires on a single char repeated with no newlines', () => {
const text = 'answer: ' + 'a'.repeat(500);
expect(hasPeriodicTail(text)).toBe(true);
expect(isDegenerateOutput(text)).toBe(true);
});
it('POSITIVE: fires on a multi-char block repeat with no newlines', () => {
const text = 'prefix ' + 'abcdef'.repeat(100);
expect(hasPeriodicTail(text)).toBe(true);
});
it('POSITIVE: at least MIN_PERIOD_REPEATS repeats of a small block', () => {
const text = 'go'.repeat(MIN_PERIOD_REPEATS);
expect(hasPeriodicTail(text)).toBe(true);
});
it('NEGATIVE: prose does not look periodic', () => {
const text =
'The quick brown fox jumps over the lazy dog while the sun sets slowly ' +
'behind the distant mountains and the river winds through the valley below.';
expect(hasPeriodicTail(text)).toBe(false);
expect(isDegenerateOutput(text)).toBe(false);
});
it('NEGATIVE: a long code block is not flagged', () => {
const code = `
function compute(values) {
let total = 0;
for (const v of values) {
total += v * 2;
}
return total / values.length;
}
export const helper = (x) => x + 1;
const config = { retries: 3, timeout: 5000, backoff: 'exp' };
`.repeat(3);
expect(isDegenerateOutput(code)).toBe(false);
});
it('NEGATIVE: a short string well under the repeat count is safe', () => {
expect(hasPeriodicTail('ababab')).toBe(false);
});
// Regression (#444): a trivial single-char period (p===1) must NOT flag
// legitimate divider/underline/whitespace runs. These are common in real
// model output and previously false-positived at ~20 identical chars, aborting
// the run and truncating output. They must all be treated as clean.
it('NEGATIVE: a markdown horizontal rule is not flagged', () => {
const text = 'text\n' + '-'.repeat(40);
expect(hasPeriodicTail(text)).toBe(false);
expect(isDegenerateOutput(text)).toBe(false);
});
it('NEGATIVE: a setext heading underline is not flagged', () => {
const text = 'Title\n' + '='.repeat(30);
expect(hasPeriodicTail(text)).toBe(false);
expect(isDegenerateOutput(text)).toBe(false);
});
it('NEGATIVE: a box-drawing divider with no trailing newline is not flagged', () => {
const text = 'done ' + '─'.repeat(50);
expect(hasPeriodicTail(text)).toBe(false);
expect(isDegenerateOutput(text)).toBe(false);
});
it('NEGATIVE: trailing spaces are not flagged', () => {
const text = 'answer' + ' '.repeat(40);
expect(hasPeriodicTail(text)).toBe(false);
expect(isDegenerateOutput(text)).toBe(false);
});
// TRIVIAL_MIN_REPEATS boundary (#444 review). The monochar-tail branch fires at
// EXACTLY 60 identical trailing chars (`run >= TRIVIAL_MIN_REPEATS`), so 59 is
// clean and 60 trips. These pin the `>=` and MUST fail if the comparison is
// flipped to `>` (the surviving mutation). The value 60 is HARD-CODED here on
// purpose: TRIVIAL_MIN_REPEATS is a private constant and the assert must lock
// the literal boundary the reviewer named, not track a constant edit.
it('NEGATIVE: 59 identical trailing chars is one below the monochar threshold', () => {
expect(hasPeriodicTail('x'.repeat(59))).toBe(false);
expect(isDegenerateOutput('x'.repeat(59))).toBe(false);
});
it('POSITIVE: 60 identical trailing chars hits the monochar threshold exactly', () => {
// Fails if `run >= TRIVIAL_MIN_REPEATS` is mutated to `run > …`.
expect(hasPeriodicTail('x'.repeat(60))).toBe(true);
expect(isDegenerateOutput('x'.repeat(60))).toBe(true);
});
// Positive counterparts: a GENUINE single-char runaway (hundreds+ of repeats)
// and the real incident (period>=2, "loadTools." ×N) must still fire.
it('POSITIVE: a genuine single-char runaway is still flagged', () => {
const text = 'x'.repeat(5000);
expect(hasPeriodicTail(text)).toBe(true);
expect(isDegenerateOutput(text)).toBe(true);
});
it('POSITIVE: the "loadTools." incident (period>=2) is still flagged', () => {
const text = 'loadTools.'.repeat(500);
expect(hasPeriodicTail(text)).toBe(true);
expect(isDegenerateOutput(text)).toBe(true);
});
});
describe('truncateDegeneratedTail', () => {
it('collapses a repeated-line loop to a few reps + marker', () => {
const text = 'plan\n' + 'loadTools.\n'.repeat(20000);
const out = truncateDegeneratedTail(text);
expect(out.length).toBeLessThan(text.length);
expect(out).toContain('output truncated');
// Keeps the leading context and a few loop reps.
expect(out).toContain('plan');
expect((out.match(/loadTools\./g) ?? []).length).toBeLessThan(10);
});
it('collapses a no-newline periodic loop to a few blocks + marker', () => {
const text = 'answer: ' + 'xy'.repeat(50000);
const out = truncateDegeneratedTail(text);
expect(out.length).toBeLessThan(text.length);
expect(out).toContain('output truncated');
expect(out).toContain('answer:');
});
it('returns non-degenerate text unchanged (by identity)', () => {
const text = 'A perfectly normal, finished assistant answer.';
expect(truncateDegeneratedTail(text)).toBe(text);
});
});
@@ -0,0 +1,189 @@
/**
* Token-degeneration detector for the in-app agent stream (#444).
*
* When the final-step lockdown is OFF (the new default) there is no toolChoice
* override to strip the model's tools mid-work, so the anti-babble safety net is
* this detector. It watches the accumulating assistant text and, on a runaway
* repetition loop (the 255KB "loadTools." incident), aborts the run.
*
* Both rules are PURE functions of the text tail so they are cheap to run every
* few KB and are unit-testable in isolation. They operate on the TAIL only
* (`TAIL_WINDOW` chars) so the cost is bounded regardless of how long the turn is.
*/
/** How many trailing chars of the accumulated text the rules inspect. */
export const TAIL_WINDOW = 3000;
/** Rule 1 threshold: minimum consecutive identical non-empty lines to trigger. */
export const REPEATED_LINES_THRESHOLD = 25;
/** Rule 2: maximum length of a repeating block considered for periodicity. */
export const MAX_PERIOD_LEN = 150;
/** Rule 2: minimum number of consecutive block repeats to trigger. */
export const MIN_PERIOD_REPEATS = 20;
/**
* Rule 1 `REPEATED_LINES_THRESHOLD` consecutive IDENTICAL non-empty lines at
* the tail. Catches the classic newline-delimited loop ("loadTools.\n" ×N).
* Blank lines break a run (a table / list with blank separators never trips it);
* a run of ordinary distinct lines (an edit list, code) never reaches the count.
*
* NB: `REPEATED_LINES_THRESHOLD` (25) is only THIS rule's own trigger, not the
* effective floor for detecting a repeated-line loop. In practice a newline-
* delimited repeat also has a fixed period (line + '\n'), so rule 2 catches it
* via periodicity at `MIN_PERIOD_REPEATS` (20) repeats the two rules combine
* (see `isDegenerateOutput`), so the effective lower bound for a short identical
* line loop is ~20, not 25. Pure.
*/
export function hasRepeatedLineRun(
text: string,
threshold = REPEATED_LINES_THRESHOLD,
): boolean {
const tail = text.length > TAIL_WINDOW ? text.slice(-TAIL_WINDOW) : text;
const lines = tail.split('\n');
let run = 1;
let prev: string | null = null;
for (const line of lines) {
if (line.length > 0 && line === prev) {
run += 1;
if (run >= threshold) return true;
} else {
run = 1;
}
prev = line;
}
return false;
}
/**
* Rule 2 cheap suffix-periodicity check: the tail ends in
* `MIN_PERIOD_REPEATS` back-to-back repeats of a single block of length
* `MAX_PERIOD_LEN`. Catches a no-newline repeat ("abcabcabc…") the line rule
* misses. For each candidate period length p we verify the last `repeats*p`
* chars are p-periodic; we stop at the smallest p that satisfies the repeat
* count. Bounded by MAX_PERIOD_LEN × TAIL_WINDOW comparisons negligible. Pure.
*/
export function hasPeriodicTail(
text: string,
maxPeriod = MAX_PERIOD_LEN,
minRepeats = MIN_PERIOD_REPEATS,
): boolean {
const tail = text.length > TAIL_WINDOW ? text.slice(-TAIL_WINDOW) : text;
const n = tail.length;
// Not even the shortest possible loop fits in the tail.
if (n < minRepeats) return false;
// A tail of ONE repeated char (a "trivial period") is common in LEGIT output —
// markdown rules (----/====), setext underlines, box-drawing dividers,
// trailing spaces routinely produce 20–50 identical chars. Such a run is
// p-periodic for EVERY p, so it would otherwise trip the block rule at p>=2
// too, not just p===1. We therefore split the check: a monochar tail needs far
// more repeats (a real single-char babble loop produces hundreds-to-thousands;
// 60 is well above any realistic divider yet a fifth of TAIL_WINDOW), while a
// genuine multi-char block repeat (>=2 distinct chars, e.g. the "loadTools."
// incident, period ~10) keeps the normal MIN_PERIOD_REPEATS threshold.
const TRIVIAL_MIN_REPEATS = 60;
// Monochar-tail check (the trivial-period case): count the trailing run of one
// identical char and require TRIVIAL_MIN_REPEATS of them.
{
const last = tail[n - 1];
let run = 1;
for (let i = n - 2; i >= 0 && tail[i] === last; i--) run++;
if (run >= TRIVIAL_MIN_REPEATS) return true;
}
const maxP = maxPeriod;
for (let p = 2; p <= maxP; p++) {
// Verify the last (minRepeats*p) chars are p-periodic AND not monochar (a
// monochar span is the trivial case handled above, so skip it here to avoid
// re-flagging a legit divider at a composite period).
const span = minRepeats * p;
// Not enough tail to hold this many repeats of this period.
if (span > n) continue;
const start = n - span;
let periodic = true;
let multiChar = false;
for (let i = n - 1; i >= start + p; i--) {
if (tail[i] !== tail[i - p]) {
periodic = false;
break;
}
}
if (!periodic) continue;
// Confirm the block itself has >=2 distinct chars (else it's monochar).
for (let i = start + 1; i < n; i++) {
if (tail[i] !== tail[start]) {
multiChar = true;
break;
}
}
if (multiChar) return true;
}
return false;
}
/**
* Combined guard used by the stream's onChunk: true when EITHER rule fires.
* Pure the caller owns the abort side effect.
*/
export function isDegenerateOutput(text: string): boolean {
return hasRepeatedLineRun(text) || hasPeriodicTail(text);
}
/**
* Truncate a degenerated tail before persist so hundreds of KB of garbage never
* reach the DB / replay (#444). Keeps everything up to and including the FIRST
* `keepRepeats` repeats of the detected loop, then appends a short marker. If no
* loop is detected the text is returned unchanged (by identity).
*
* Implementation: find the shortest tail period (same check as hasPeriodicTail),
* keep the prefix before the loop plus `keepRepeats` copies of the block, drop
* the rest. This is best-effort cosmetic trimming; correctness does not depend on
* finding the exact minimal loop. Pure.
*/
export function truncateDegeneratedTail(
text: string,
keepRepeats = 3,
): string {
const marker = '\n…[output truncated: repeated token loop detected]';
// Try the line rule first: collapse a long run of identical lines.
const lines = text.split('\n');
let runStart = -1;
let run = 1;
for (let i = 1; i < lines.length; i++) {
if (lines[i].length > 0 && lines[i] === lines[i - 1]) {
if (run === 1) runStart = i - 1;
run += 1;
if (run >= REPEATED_LINES_THRESHOLD) {
const kept = lines.slice(0, runStart + keepRepeats).join('\n');
return kept + marker;
}
} else {
run = 1;
runStart = -1;
}
}
// Fall back to periodicity over the whole string (bounded by the same block
// length). Find the smallest period that makes the SUFFIX highly repetitive.
const n = text.length;
const maxP = Math.min(MAX_PERIOD_LEN, Math.floor(n / MIN_PERIOD_REPEATS));
for (let p = 1; p <= maxP; p++) {
// Count how many trailing p-blocks are periodic.
let reps = 1;
let i = n - 1;
for (; i >= p; i--) {
if (text[i] !== text[i - p]) break;
}
// The loop above walks over the periodic suffix; its length is (n-1 - i).
const periodicLen = n - 1 - i;
reps = Math.floor(periodicLen / p) + 1;
if (reps >= MIN_PERIOD_REPEATS) {
const loopStart = n - reps * p; // start of the fully-periodic suffix
const kept = text.slice(0, loopStart + keepRepeats * p);
return kept + marker;
}
}
return text;
}
@@ -1,6 +1,17 @@
import { AiChatToolsService } from './ai-chat-tools.service';
import * as loader from './docmost-client.loader';
import type { DocmostClientLike } from './docmost-client.loader';
// Test-double type for the loopback client. `DocmostClientLike` is now DERIVED
// from the real `DocmostClient` (issue #446), so its method RETURN types are the
// concrete client shapes. These stubs deliberately return minimal recording
// shapes (e.g. `{ ok: true }`), which no longer satisfy those concrete returns —
// so the doubles are typed with the same method NAMES but loose async returns.
// Each is still cast to `DocmostClientLike` at the (return-erased) mock site, so
// the positional-call type-safety on the PRODUCTION client is unaffected.
type FakeDocmostClient = Partial<
Record<keyof DocmostClientLike, (...args: any[]) => Promise<any>>
>;
// The real zod-agnostic shared tool-spec registry. It has no runtime deps, so
// importing the TS source directly keeps these mocks honest: the service builds
// the shared tools from exactly the specs the package ships, not a hand-stub.
@@ -12,7 +23,15 @@ import { SHARED_TOOL_SPECS } from '../../../../../../packages/mcp/src/tool-specs
// sync.
const mockLoaded = (DocmostClient: loader.DocmostClientCtor) => ({
DocmostClient,
sharedToolSpecs: SHARED_TOOL_SPECS as Record<string, loader.SharedToolSpec>,
sharedToolSpecs: SHARED_TOOL_SPECS as unknown as Record<string, loader.SharedToolSpec>,
// Pure no-network draw.io helpers (#424). Type-correct stubs: these tests
// never execute the drawioShapes / drawioGuide tool bodies.
searchShapes: (() => []) as unknown as loader.SearchShapesFn,
getGuideSection: (() => ({
section: 'index',
content: '',
sections: [],
})) as unknown as loader.GetGuideSectionFn,
});
/**
@@ -31,7 +50,7 @@ describe('AiChatToolsService deletePage guardrail (H4)', () => {
// Minimal fake DocmostClient: only the write methods the tools touch need to
// exist; deletePage records its args. No network, no ESM import.
const fakeClient: Partial<DocmostClientLike> = {
const fakeClient: FakeDocmostClient = {
deletePage: (...args: unknown[]) => {
deletePageCalls.push(args);
return Promise.resolve({ success: true });
@@ -160,7 +179,7 @@ describe('AiChatToolsService deletePage guardrail (H4)', () => {
describe('AiChatToolsService expanded toolset guardrails', () => {
// No client method is invoked here — every assertion is on tool presence /
// input schema — so an empty fake client is sufficient.
const fakeClient: Partial<DocmostClientLike> = {};
const fakeClient: FakeDocmostClient = {};
const tokenServiceStub = {
generateAccessToken: jest.fn().mockResolvedValue('access-token'),
@@ -264,8 +283,9 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
const patchNodeCalls: unknown[][] = [];
const insertNodeCalls: unknown[][] = [];
const updatePageJsonCalls: unknown[][] = [];
const updatePageCalls: unknown[][] = [];
const fakeClient: Partial<DocmostClientLike> = {
const fakeClient: FakeDocmostClient = {
patchNode: (...args: unknown[]) => {
patchNodeCalls.push(args);
return Promise.resolve({ ok: true });
@@ -278,6 +298,11 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
updatePageJsonCalls.push(args);
return Promise.resolve({ ok: true });
},
// Backs the plain-Markdown full-body-replace tool updatePageMarkdown (#411).
updatePage: (...args: unknown[]) => {
updatePageCalls.push(args);
return Promise.resolve({ success: true });
},
};
const tokenServiceStub = {
@@ -291,6 +316,7 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
patchNodeCalls.length = 0;
insertNodeCalls.length = 0;
updatePageJsonCalls.length = 0;
updatePageCalls.length = 0;
jest.spyOn(loader, 'loadDocmostMcp').mockResolvedValue(
mockLoaded(function () {
return fakeClient as DocmostClientLike;
@@ -329,23 +355,32 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
content: [{ type: 'text', text: 'Hello' }],
};
it('patchNode parses a JSON-string node and forwards it as an object', async () => {
it('patchNode parses a JSON-string node and forwards it as { node } (object)', async () => {
const tools = await buildTools();
await tools.patchNode.execute(
{ pageId: 'p1', nodeId: 'n1', node: JSON.stringify(NODE_OBJ) } as never,
{} as never,
);
expect(patchNodeCalls).toHaveLength(1);
expect(patchNodeCalls[0]).toEqual(['p1', 'n1', NODE_OBJ]);
// #413: the 3rd arg is now the XOR input { markdown?, node? }.
expect(patchNodeCalls[0]).toEqual([
'p1',
'n1',
{ markdown: undefined, node: NODE_OBJ },
]);
});
it('patchNode passes an object node through unchanged', async () => {
it('patchNode passes an object node through unchanged inside { node }', async () => {
const tools = await buildTools();
await tools.patchNode.execute(
{ pageId: 'p1', nodeId: 'n1', node: NODE_OBJ } as never,
{} as never,
);
expect(patchNodeCalls[0]).toEqual(['p1', 'n1', NODE_OBJ]);
expect(patchNodeCalls[0]).toEqual([
'p1',
'n1',
{ markdown: undefined, node: NODE_OBJ },
]);
});
it('patchNode throws the documented message on invalid JSON string', async () => {
@@ -359,7 +394,7 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
expect(patchNodeCalls).toHaveLength(0);
});
it('insertNode parses a JSON-string node and forwards it as an object', async () => {
it('insertNode parses a JSON-string node and forwards it inside { node }', async () => {
const tools = await buildTools();
await tools.insertNode.execute(
{
@@ -370,9 +405,15 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
{} as never,
);
expect(insertNodeCalls).toHaveLength(1);
const [pageId, node] = insertNodeCalls[0];
// #413: the 2nd arg is the XOR input { markdown?, node? }, the 3rd is opts.
const [pageId, input, opts] = insertNodeCalls[0] as [
string,
{ markdown?: unknown; node?: unknown },
{ position?: string },
];
expect(pageId).toBe('p1');
expect(node).toEqual(NODE_OBJ);
expect(input).toEqual({ markdown: undefined, node: NODE_OBJ });
expect(opts.position).toBe('append');
});
it('insertNode throws the documented message on invalid JSON string', async () => {
@@ -426,6 +467,54 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
).rejects.toThrow('content was a string but not valid JSON');
expect(updatePageJsonCalls).toHaveLength(0);
});
// #411: the plain-Markdown full-body-replace tool is now the shared
// `updatePageMarkdown` (was inline `updatePageContent`). It forwards to
// client.updatePage(pageId, content, title) -> updatePageContentRealtime ->
// markdownToProseMirrorCanonical, so `^[...]` footnotes materialize.
it('updatePageMarkdown forwards { pageId, content, title } to client.updatePage', async () => {
const tools = await buildTools();
await tools.updatePageMarkdown.execute(
{ pageId: 'p1', content: 'Body^[a note]', title: 'New title' } as never,
{} as never,
);
expect(updatePageCalls).toHaveLength(1);
expect(updatePageCalls[0]).toEqual(['p1', 'Body^[a note]', 'New title']);
});
it('updatePageMarkdown returns the RAW client result in-app (deliberate #411 shape change, documented on the spec)', async () => {
const tools = await buildTools();
// Registry canonical execute returns client.updatePage's result verbatim.
// The old inline tool projected to { pageId, updated }; the rename now
// surfaces the raw result (nothing reads the removed `.updated`; the raw
// shape carries footnote/verify warnings and matches the on-both-hosts
// registry convention). fakeClient.updatePage resolves { success: true }.
const result = await tools.updatePageMarkdown.execute(
{ pageId: 'p1', content: '# Hi' } as never,
{} as never,
);
expect(result).toEqual({ success: true });
});
it('updatePageMarkdown forwards title=undefined when omitted', async () => {
const tools = await buildTools();
await tools.updatePageMarkdown.execute(
{ pageId: 'p1', content: '# Hi' } as never,
{} as never,
);
expect(updatePageCalls[0]).toEqual(['p1', '# Hi', undefined]);
});
// #411 surface split: the plain-Markdown replace tool exists in-app under the
// new key; the OLD inline updatePageContent key is gone; importPageMarkdown is
// still present IN-APP (only the external MCP surface drops it — asserted in
// packages/mcp/test/unit/tool-inventory.test.mjs).
it('exposes updatePageMarkdown in-app, no legacy updatePageContent, keeps importPageMarkdown', async () => {
const tools = await buildTools();
expect(tools.updatePageMarkdown).toBeDefined();
expect((tools as Record<string, unknown>).updatePageContent).toBeUndefined();
expect(tools.importPageMarkdown).toBeDefined();
});
});
/**
@@ -439,7 +528,7 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
* getOutline) are exercised here end-to-end through forUser().
*/
describe('AiChatToolsService model-friendly input validation (#190)', () => {
const fakeClient: Partial<DocmostClientLike> = {};
const fakeClient: FakeDocmostClient = {};
const tokenServiceStub = {
generateAccessToken: jest.fn().mockResolvedValue('access-token'),
generateCollabToken: jest.fn().mockResolvedValue('collab-token'),
@@ -557,7 +646,7 @@ describe('AiChatToolsService #294 changed execute wirings', () => {
tableDeleteRow: [],
tableUpdateCell: [],
};
const fakeClient: Partial<DocmostClientLike> = {
const fakeClient: FakeDocmostClient = {
movePage: (...args: unknown[]) => {
calls.movePage.push(args);
return Promise.resolve({ success: true });
@@ -666,7 +755,7 @@ describe('AiChatToolsService #410 footnote + image tools', () => {
insertImage: [],
replaceImage: [],
};
const fakeClient: Partial<DocmostClientLike> = {
const fakeClient: FakeDocmostClient = {
insertFootnote: (...args: unknown[]) => {
calls.insertFootnote.push(args);
return Promise.resolve({ success: true, footnoteId: 'fn1', reused: false });
@@ -836,3 +925,109 @@ describe('AiChatToolsService getCurrentPage selection (#388)', () => {
);
});
});
/**
* #440 review: the in-app drawioCreate / drawioUpdate handlers must forward
* the optional `layout:"elk"` param to the client (5th positional arg), exactly
* like the MCP host. It was silently dropped, so ELK auto-layout worked only via
* the standalone MCP server, not in-app. These tests pin per-host parity.
*/
describe('AiChatToolsService drawio layout passthrough (#440)', () => {
const createCalls: unknown[][] = [];
const updateCalls: unknown[][] = [];
// FakeDocmostClient (not Partial<DocmostClientLike>): since #446 derived
// DocmostClientLike from the real client, its drawioCreate/drawioUpdate return
// the concrete result shape, so a minimal stub object would not be assignable.
// FakeDocmostClient types every method as (...args) => Promise<any>, which is
// exactly what these arg-capturing doubles need.
const fakeClient: FakeDocmostClient = {
drawioCreate: (...args: unknown[]) => {
createCalls.push(args);
return Promise.resolve({ success: true, nodeId: '#0' });
},
drawioUpdate: (...args: unknown[]) => {
updateCalls.push(args);
return Promise.resolve({ success: true, nodeId: '#0' });
},
};
const tokenServiceStub = {
generateAccessToken: jest.fn().mockResolvedValue('access-token'),
generateCollabToken: jest.fn().mockResolvedValue('collab-token'),
};
let service: AiChatToolsService;
beforeEach(() => {
createCalls.length = 0;
updateCalls.length = 0;
jest.spyOn(loader, 'loadDocmostMcp').mockResolvedValue(
mockLoaded(function () {
return fakeClient as DocmostClientLike;
} as unknown as loader.DocmostClientCtor),
);
service = new AiChatToolsService(
tokenServiceStub as never,
{} as never,
{} as never,
{} as never,
{} as never,
{
asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }),
} as never,
);
});
afterEach(() => jest.restoreAllMocks());
const buildTools = () =>
service.forUser(
{ id: 'user-1', email: 'u@example.com', workspaceId: 'ws-1' } as never,
'session-1',
'ws-1',
'chat-1',
);
it('forwards layout:"elk" to client.drawioCreate as the 5th positional arg', async () => {
const tools = await buildTools();
await tools.drawioCreate.execute(
{
pageId: 'p-1',
xml: '<mxGraphModel/>',
position: 'append',
layout: 'elk',
} as never,
{} as never,
);
expect(createCalls).toHaveLength(1);
// drawioCreate(pageId, where, xml, title, layout) — layout is args[4].
expect(createCalls[0][4]).toBe('elk');
});
it('forwards layout:"elk" to client.drawioUpdate as the 5th positional arg', async () => {
const tools = await buildTools();
await tools.drawioUpdate.execute(
{
pageId: 'p-1',
node: '#0',
xml: '<mxGraphModel/>',
baseHash: 'h',
layout: 'elk',
} as never,
{} as never,
);
expect(updateCalls).toHaveLength(1);
// drawioUpdate(pageId, node, xml, baseHash, layout) — layout is args[4].
expect(updateCalls[0][4]).toBe('elk');
});
it('omits layout (undefined 5th arg) when not requested', async () => {
const tools = await buildTools();
await tools.drawioCreate.execute(
{ pageId: 'p-1', xml: '<mxGraphModel/>', position: 'append' } as never,
{} as never,
);
expect(createCalls[0][4]).toBeUndefined();
});
});
@@ -26,6 +26,125 @@ import {
type ToolCatalogEntry,
} from './tool-tiers';
/**
* Compile-time contract (issue #446): the in-app tool `execute` closures below
* call the loopback `DocmostClient` POSITIONALLY (e.g.
* `client.drawioGet(pageId, node, format ?? 'xml')`). Those closures receive an
* AI-SDK-erased (`any`) input, so a positional call inside them is NOT checked
* against the real signature a parameter reorder/type-change in
* `packages/mcp/src/client.ts` would otherwise reach production as a runtime
* "wrong argument" tool failure with zero compile signal (the restored #294
* debt). This never-called function reproduces every positional call with
* correctly-typed placeholder arguments against the DERIVED `DocmostClientLike`
* (a `Pick` of the real `DocmostClient`), so any such reorder/rename becomes a
* SERVER COMPILE ERROR here. It emits nothing (types only) and is never invoked;
* keep each call in lockstep with the matching `execute` body below.
*/
// eslint-disable-next-line @typescript-eslint/no-unused-vars
function __assertClientCallContract(client: DocmostClientLike): void {
// Placeholders standing in for the AI-SDK-erased execute inputs. Their types
// are deliberately concrete so the positional calls are checked end-to-end.
const s = '' as string;
const n = 0 as number;
const node: unknown = null;
const edits: Array<{ find: string; replace: string; replaceAll?: boolean }> =
[];
const cells: string[] = [];
const align = undefined as 'left' | 'center' | 'right' | undefined;
// --- read ---
void client.search(s, undefined, n);
void client.getPage(s);
void client.getPageRaw(s);
void client.getWorkspace();
void client.getSpaces();
void client.listPages(s, n, true);
void client.getTree(s, s, n);
void client.getPageContext(s);
void client.listSidebarPages(s, s);
void client.getOutline(s);
void client.getPageJson(s);
void client.getNode(s, s, 'markdown');
void client.searchInPage(s, s, {
regex: true,
caseSensitive: true,
limit: n,
});
void client.getTable(s, s);
void client.listComments(s, true);
void client.getComment(s);
void client.checkNewComments(s, s, s);
void client.listShares();
void client.listPageHistory(s, s);
void client.getPageHistory(s);
void client.diffPageVersions(s, s, s);
void client.exportPageMarkdown(s);
// --- write (page) ---
void client.createPage(s, s, s, s);
void client.updatePage(s, s, s);
void client.renamePage(s, s);
void client.movePage(s, s, s);
void client.deletePage(s);
void client.editPageText(s, edits);
void client.patchNode(s, s, { markdown: s, node });
void client.insertNode(
s,
{ markdown: s, node },
{
position: 'append',
anchorNodeId: s,
anchorText: s,
},
);
void client.deleteNode(s, s);
void client.updatePageJson(s, node, s);
void client.tableInsertRow(s, s, cells, n);
void client.tableDeleteRow(s, s, n);
void client.tableUpdateCell(s, s, n, n, s);
void client.copyPageContent(s, s);
void client.importPageMarkdown(s, s);
void client.sharePage(s, true);
void client.unsharePage(s);
void client.restorePageVersion(s);
void client.transformPage(s, s, { dryRun: true });
void client.stashPage(s);
// --- write (image / footnote), in-app since #410 ---
void client.insertFootnote(s, s, s);
void client.insertImage(s, s, {
align,
alt: s,
replaceText: s,
afterText: s,
});
void client.replaceImage(s, s, s, { align, alt: s });
// --- draw.io diagrams (#423 stage 1, #424 stage 2) ---
// The 5th `layout` arg (#424) is exercised so this parity assertion fails if the
// client signature drops it — it must reach the client from the shared execute.
void client.drawioGet(s, s, 'xml');
void client.drawioCreate(s, { position: 'append', anchorNodeId: s }, s, s, 'elk');
void client.drawioUpdate(s, s, s, s, 'elk');
// --- draw.io high-level semantic tools (#425 stage 3) ---
void client.drawioEditCells(s, s, [{ op: 'delete', cellId: s }], s);
void client.drawioFromGraph(
s,
{ position: 'append', anchorNodeId: s },
{ nodes: [{ id: s, label: s }] },
'LR',
s,
'full',
s,
);
void client.drawioFromMermaid(
s,
{ position: 'append', anchorNodeId: s },
s,
s,
);
// --- write (comment) ---
void client.createComment(s, s, 'inline', s, s, s);
void client.resolveComment(s, true);
}
/**
* Per-user, per-request adapter that exposes Docmost READ operations to the
* agent as AI SDK tools (STAGE A = read only).
@@ -169,8 +288,19 @@ export class AiChatToolsService {
// provenance tokens) and load the shared tool-spec registry. Client
// construction is shared with the page-change detection path (#274) via
// buildDocmostClient so both go over the exact same authenticated route.
const { sharedToolSpecs, createCommentSignalTracker } =
await loadDocmostMcp();
// searchShapes / getGuideSection (#424) are the PURE, no-network helpers
// backing drawioShapes / drawioGuide. They are `inlineBothHosts` specs (no
// canonical execute — their catalog loader uses import.meta and can't be
// value-imported into the zod-agnostic tool-specs.ts under the server's
// commonjs type-check), so the shared registry loop below SKIPS them and this
// service wires them inline (see drawioShapes/drawioGuide entries), mirroring
// how index.ts registers them on the standalone MCP host.
const {
sharedToolSpecs,
createCommentSignalTracker,
searchShapes,
getGuideSection,
} = await loadDocmostMcp();
const client = await this.buildDocmostClient(
user,
sessionId,
@@ -198,6 +328,18 @@ export class AiChatToolsService {
execute,
});
// The in-app toolset. It starts with the tools kept INLINE here for a
// documented per-layer reason: an intentional behaviour/schema divergence from
// the standalone MCP surface (searchPages' hybrid RRF,
// transformPage's guardrailed shorter schema), a name clash the shared
// registry forbids (in-app `getTable` verb-first vs the MCP noun-first
// `tableGet` — the registry requires mcpName === inAppKey), per-request
// state the registry loop cannot provide
// (getCurrentPage reads the resolved openedPage; searchPages closes over the
// per-request user/embedding deps), or a tool with no MCP twin
// (listSidebarPages/getComment/getPageHistory). Every SHARED tool is then added
// by the registry loop below (see it), so there is exactly one arg-mapping per
// shared tool and it can never drift from the MCP host again (#445).
const tools: Record<string, Tool> = {
// INTENTIONAL per-transport divergence (not in the shared registry): this
// in-app search runs a semantic + keyword hybrid (RRF) with in-process
@@ -332,180 +474,14 @@ export class AiChatToolsService {
execute: async () => resolveCurrentPageResult(openedPage),
}),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// The execute body keeps this layer's { title, markdown } projection.
getPage: sharedTool(sharedToolSpecs.getPage, async ({ pageId }) => {
// getPage(pageId) -> { data: filterPage(page, markdown), success }.
const result = await client.getPage(pageId);
const data = (result?.data ?? {}) as {
title?: string;
content?: string;
};
return {
title: data.title ?? '',
markdown: typeof data.content === 'string' ? data.content : '',
};
}),
// --- WRITE tools (all reversible — history/trash; §6.5 / D3) ---
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
createPage: sharedTool(
sharedToolSpecs.createPage,
async ({ title, content, spaceId, parentPageId }) => {
// createPage(title, content, spaceId, parentPageId?) ->
// { data: filterPage(page, markdown), success }.
const result = await client.createPage(
title,
content ?? '',
spaceId,
parentPageId,
);
const data = (result?.data ?? {}) as {
id?: string;
slugId?: string;
title?: string;
};
return { id: data.id ?? data.slugId, title: data.title ?? title };
},
),
updatePageContent: tool({
description:
"Replace a page's body with new Markdown content (and optionally its " +
'title). Reversible: the previous version is kept in page history.',
inputSchema: modelFriendlyInput({
pageId: z.string().describe('The id of the page to update.'),
content: z.string().describe('The new page body as Markdown.'),
title: z
.string()
.optional()
.describe('Optional new title for the page.'),
}),
execute: async ({ pageId, content, title }) => {
// updatePage mutates the live collab doc -> provenance flows from the
// collab-token provider. Returns { success, modified, message, pageId }.
const result = (await client.updatePage(pageId, content, title)) as {
success?: boolean;
};
return { pageId, updated: result?.success ?? true };
},
}),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
renamePage: sharedTool(
sharedToolSpecs.renamePage,
async ({ pageId, title }) => {
// renamePage(pageId, title) -> { success, pageId, title }.
await client.renamePage(pageId, title);
return { pageId, title };
},
),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// The shared schema adds the optional `position` field this layer lacked
// before; the execute now forwards it (the client already accepted it).
movePage: sharedTool(
sharedToolSpecs.movePage,
async ({ pageId, parentPageId, position }) => {
// movePage(pageId, parentPageId, position?) -> raw move response.
await client.movePage(pageId, parentPageId ?? null, position);
return { pageId, parentPageId: parentPageId ?? null, moved: true };
},
),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// GUARDRAIL (§14 H4) preserved: the shared schema exposes ONLY pageId, so
// permanentlyDelete/forceDelete are never part of the input and can never
// be forwarded — the agent physically cannot permanently delete a page.
deletePage: sharedTool(sharedToolSpecs.deletePage, async ({ pageId }) => {
// deletePage(pageId) hits POST /pages/delete with { pageId } only,
// which is the soft-delete (trash) path on the server.
await client.deletePage(pageId);
return { pageId, trashed: true };
}),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// This layer keeps only its own execute-side guards (require a selection
// for a top-level comment; reject suggestedText on a reply / without a
// selection) — the schema+description are shared.
createComment: sharedTool(
sharedToolSpecs.createComment,
async ({
pageId,
content,
selection,
parentCommentId,
suggestedText,
}) => {
// createComment(pageId, content, type, selection?, parentCommentId?,
// suggestedText?). Top-level comments are inline and must carry a
// selection to anchor on; replies inherit the parent's anchor (no
// selection). Throwing here surfaces a tool error to the model (Vercel
// `ai` SDK) so the agent retries with a better selection — do not
// catch/suppress it.
if (!parentCommentId && (!selection || !selection.trim())) {
throw new Error(
"createComment requires a 'selection' (exact text to anchor on) for a new top-level comment.",
);
}
if (suggestedText !== undefined) {
if (parentCommentId) {
throw new Error(
"createComment: 'suggestedText' cannot be attached to a reply; it applies only to a top-level inline comment.",
);
}
if (!selection || !selection.trim()) {
throw new Error(
"createComment: 'suggestedText' requires a 'selection' to anchor and rewrite.",
);
}
}
const result = await client.createComment(
pageId,
content,
'inline',
selection,
parentCommentId,
suggestedText,
);
const data = (result?.data ?? {}) as { id?: string };
return { commentId: data.id, pageId };
},
),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
resolveComment: sharedTool(
sharedToolSpecs.resolveComment,
async ({ commentId, resolved }) => {
// resolveComment(commentId, resolved) -> { success, commentId, resolved }.
await client.resolveComment(commentId, resolved);
return { commentId, resolved };
},
),
// --- READ tools (added) ---
getWorkspace: sharedTool(
sharedToolSpecs.getWorkspace,
async () => await client.getWorkspace(),
),
listSpaces: sharedTool(
sharedToolSpecs.listSpaces,
async () => await client.getSpaces(),
),
// INTENTIONAL per-transport divergence (not shared): keeps the `tree:true`
// hierarchy mode but is worded for the in-app agent; the standalone MCP
// `list_pages` carries its own wording. Kept per-layer so each side tunes
// its own guidance.
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
listPages: sharedTool(
sharedToolSpecs.listPages,
async ({ spaceId, limit, tree }) =>
await client.listPages(spaceId, limit, tree),
),
//
// NOTE (issue #411): the plain-Markdown full-body-replace tool is no longer
// inline here — it moved to @docmost/mcp's SHARED_TOOL_SPECS as
// `updatePageMarkdown` (was inline `updatePageContent`) so it registers on
// BOTH the external MCP and the in-app agent. The registry loop below adds
// it under its inAppKey. importPageMarkdown stays a shared spec too (now
// inAppOnly — dropped from the external MCP surface, kept in-app).
listSidebarPages: tool({
description:
@@ -525,34 +501,9 @@ export class AiChatToolsService {
await client.listSidebarPages(spaceId, pageId),
}),
getOutline: sharedTool(
sharedToolSpecs.getOutline,
async ({ pageId }) => await client.getOutline(pageId),
),
getPageJson: sharedTool(
sharedToolSpecs.getPageJson,
async ({ pageId }) => await client.getPageJson(pageId),
),
getNode: sharedTool(
sharedToolSpecs.getNode,
async ({ pageId, nodeId }) => await client.getNode(pageId, nodeId),
),
searchInPage: sharedTool(
sharedToolSpecs.searchInPage,
async ({ pageId, query, regex, caseSensitive, limit }) =>
await client.searchInPage(pageId, query, {
regex,
caseSensitive,
limit,
}),
),
// NOT shared (kept inline): the MCP tool name `table_get` is noun-first
// while this key is `getTable` (verb-first), breaking the
// snake_case(inAppKey) convention the shared registry enforces. Its
// NOT shared (kept inline): the MCP tool name `tableGet` is noun-first
// while this key is `getTable` (verb-first), so it cannot satisfy the
// shared registry's `mcpName === inAppKey` convention (#412). Its
// reference parameter is still named `table` (was `tableRef`) so it matches
// the migrated table row/cell tools below.
getTable: tool({
@@ -572,13 +523,6 @@ export class AiChatToolsService {
await client.getTable(pageId, table),
}),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
listComments: sharedTool(
sharedToolSpecs.listComments,
async ({ pageId, includeResolved }) =>
await client.listComments(pageId, includeResolved),
),
getComment: tool({
description: 'Fetch a single comment by id (content as Markdown).',
inputSchema: modelFriendlyInput({
@@ -587,24 +531,6 @@ export class AiChatToolsService {
execute: async ({ commentId }) => await client.getComment(commentId),
}),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
checkNewComments: sharedTool(
sharedToolSpecs.checkNewComments,
async ({ spaceId, since, parentPageId }) =>
await client.checkNewComments(spaceId, since, parentPageId),
),
listShares: sharedTool(
sharedToolSpecs.listShares,
async () => await client.listShares(),
),
listPageHistory: sharedTool(
sharedToolSpecs.listPageHistory,
async ({ pageId, cursor }) =>
await client.listPageHistory(pageId, cursor),
),
getPageHistory: tool({
description:
'Fetch a single page-history version including its lossless ' +
@@ -616,206 +542,11 @@ export class AiChatToolsService {
await client.getPageHistory(historyId),
}),
diffPageVersions: sharedTool(
sharedToolSpecs.diffPageVersions,
async ({ pageId, from, to }) =>
await client.diffPageVersions(pageId, from, to),
),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
exportPageMarkdown: sharedTool(
sharedToolSpecs.exportPageMarkdown,
async ({ pageId }) => {
const markdown = await client.exportPageMarkdown(pageId);
return { markdown };
},
),
// --- WRITE tools (added; reversible via page history/trash) ---
editPageText: sharedTool(
sharedToolSpecs.editPageText,
async ({ pageId, edits }) => await client.editPageText(pageId, edits),
),
// Returns ONLY the short link object — never the document body — so a
// large page can be handed to an external consumer without bloating
// context.
stashPage: sharedTool(
sharedToolSpecs.stashPage,
async ({ pageId }) => await client.stashPage(pageId),
),
// Schema + description from the shared registry (identical across both
// transports). The execute body keeps its OWN parseNodeArg normalization:
// the model sometimes serializes the node as a JSON string, and we parse it
// before the client's typeof-object guard rejects it (parity with the
// standalone MCP server, index.ts patch_node).
patchNode: sharedTool(
sharedToolSpecs.patchNode,
async ({ pageId, nodeId, node }) => {
const parsedNode = parseNodeArg(node);
return await client.patchNode(pageId, nodeId, parsedNode);
},
),
// Shared registry schema + description; execute retains parseNodeArg on the
// incoming node (parity with the standalone MCP server, index.ts
// insert_node).
insertNode: sharedTool(
sharedToolSpecs.insertNode,
async ({ pageId, node, position, anchorNodeId, anchorText }) => {
const parsedNode = parseNodeArg(node);
return await client.insertNode(pageId, parsedNode, {
position,
anchorNodeId,
anchorText,
});
},
),
deleteNode: sharedTool(
sharedToolSpecs.deleteNode,
async ({ pageId, nodeId }) => await client.deleteNode(pageId, nodeId),
),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// The execute body keeps this layer's content normalization (parity with
// the standalone MCP server, index.ts update_page_json).
updatePageJson: sharedTool(
sharedToolSpecs.updatePageJson,
async ({ pageId, content, title }) => {
// undefined/null pass through as undefined (title-only / no-op); any
// string is JSON.parsed (so an empty string "" throws, matching the
// MCP server); an object is passed through unchanged.
let doc;
if (content === undefined || content === null) {
doc = undefined;
} else {
// String -> JSON.parse (throwing on invalid); object passes through.
doc = parseNodeArg(content, 'content was a string but not valid JSON');
}
return await client.updatePageJson(pageId, doc, title);
},
),
// Schema + description live in @docmost/mcp's SHARED_TOOL_SPECS (#410).
// Promoted from MCP-only so the in-app agent can attach a REAL footnote to
// already-written text instead of leaving a literal `^[...]` string.
insertFootnote: sharedTool(
sharedToolSpecs.insertFootnote,
async ({ pageId, anchorText, text }) =>
await client.insertFootnote(pageId, anchorText, text),
),
// Schema + description live in @docmost/mcp's SHARED_TOOL_SPECS (#410).
// The schema field is `imageUrl`; the client method takes it positionally.
insertImage: sharedTool(
sharedToolSpecs.insertImage,
async ({ pageId, imageUrl, align, alt, replaceText, afterText }) =>
await client.insertImage(pageId, imageUrl, {
align,
alt,
replaceText,
afterText,
}),
),
// Schema + description live in @docmost/mcp's SHARED_TOOL_SPECS (#410).
replaceImage: sharedTool(
sharedToolSpecs.replaceImage,
async ({ pageId, attachmentId, imageUrl, align, alt }) =>
await client.replaceImage(pageId, attachmentId, imageUrl, {
align,
alt,
}),
),
// Schema + description live in @docmost/mcp's SHARED_TOOL_SPECS (#423).
// meta.hash in the result is the baseHash drawioUpdate requires.
drawioGet: sharedTool(
sharedToolSpecs.drawioGet,
async ({ pageId, node, format }) =>
await client.drawioGet(pageId, node, format ?? 'xml'),
),
// Schema + description live in @docmost/mcp's SHARED_TOOL_SPECS (#423).
// The flat schema fields are regrouped into the client's `where` object.
drawioCreate: sharedTool(
sharedToolSpecs.drawioCreate,
async ({ pageId, xml, position, anchorNodeId, anchorText, title }) =>
await client.drawioCreate(
pageId,
{ position, anchorNodeId, anchorText },
xml,
title,
),
),
// Schema + description live in @docmost/mcp's SHARED_TOOL_SPECS (#423).
// baseHash is the optimistic lock: mismatch => structured conflict error.
drawioUpdate: sharedTool(
sharedToolSpecs.drawioUpdate,
async ({ pageId, node, xml, baseHash }) =>
await client.drawioUpdate(pageId, node, xml, baseHash),
),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// The table reference parameter was unified to `table` (was `tableRef`).
tableInsertRow: sharedTool(
sharedToolSpecs.tableInsertRow,
async ({ pageId, table, cells, index }) =>
await client.tableInsertRow(pageId, table, cells, index),
),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
tableDeleteRow: sharedTool(
sharedToolSpecs.tableDeleteRow,
async ({ pageId, table, index }) =>
await client.tableDeleteRow(pageId, table, index),
),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
tableUpdateCell: sharedTool(
sharedToolSpecs.tableUpdateCell,
async ({ pageId, table, row, col, text }) =>
await client.tableUpdateCell(pageId, table, row, col, text),
),
copyPageContent: sharedTool(
sharedToolSpecs.copyPageContent,
async ({ sourcePageId, targetPageId }) =>
await client.copyPageContent(sourcePageId, targetPageId),
),
importPageMarkdown: sharedTool(
sharedToolSpecs.importPageMarkdown,
async ({ pageId, markdown }) =>
await client.importPageMarkdown(pageId, markdown),
),
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
// Both layers already carried the security-confirmation framing, so there
// was no real divergence to preserve — only wording drift.
sharePage: sharedTool(
sharedToolSpecs.sharePage,
async ({ pageId, searchIndexing }) =>
await client.sharePage(pageId, searchIndexing),
),
unsharePage: sharedTool(
sharedToolSpecs.unsharePage,
async ({ pageId }) => await client.unsharePage(pageId),
),
restorePageVersion: sharedTool(
sharedToolSpecs.restorePageVersion,
async ({ historyId }) => await client.restorePageVersion(historyId),
),
// INTENTIONAL per-transport divergence (not shared): deliberately omits the
// `deleteComments` schema field (comment-deletion guardrail) and carries a
// much shorter description; the standalone MCP `docmost_transform` exposes
// much shorter description; the standalone MCP `docmostTransform` exposes
// the full helper catalogue. Different schema, so kept per-layer.
transformPage: tool({
description:
@@ -841,6 +572,51 @@ export class AiChatToolsService {
}),
};
// Add EVERY shared tool from the zod-agnostic registry in one loop (#445).
// The spec owns the canonical arg->client mapping; this host only decides
// WHICH mapping to run and returns its value directly (no envelope). For each
// spec:
// - skip `mcpOnly` specs (they belong to the standalone MCP host only);
// - skip `inlineBothHosts` specs (drawioShapes / drawioGuide): they carry
// no execute and are wired INLINE just below, calling the pure helpers;
// - use `inAppExecute` when the spec declares a DELIBERATE per-layer
// difference (a projected result shape, a different guardrail message);
// - otherwise use the canonical `execute` (raw client result, identical to
// the MCP host's before it wraps it as JSON).
// The execute receives the AI-SDK-validated, type-erased input; the spec reads
// the same fields its buildShape declares. This is the SINGLE place the in-app
// arg mapping lives — it can no longer silently drift from the MCP host.
for (const spec of Object.values(sharedToolSpecs)) {
if (spec.mcpOnly) continue;
if (spec.inlineBothHosts) continue;
const run = spec.inAppExecute ?? spec.execute;
if (!run) continue; // defensive: a shared spec always carries one of them.
tools[spec.inAppKey] = sharedTool(
spec,
(async (args) =>
run(client, args as Record<string, unknown>)) as Tool['execute'],
);
}
// drawioShapes / drawioGuide (#424): `inlineBothHosts` registry specs wired
// here with the SAME schema+description the shared spec pins, but calling the
// pure searchShapes / getGuideSection helpers off the loaded @docmost/mcp
// module — they are not client methods and their catalog loader uses
// import.meta, so they cannot live in the zod-agnostic shared execute. The raw
// result is identical to the MCP host's (which wraps it as JSON text); here
// the in-app host returns it plain, exactly like every other shared tool.
tools[sharedToolSpecs.drawioShapes.inAppKey] = sharedTool(
sharedToolSpecs.drawioShapes,
async ({ query, category, limit }) => {
const results = searchShapes(query, { category, limit });
return { query, count: results.length, results };
},
);
tools[sharedToolSpecs.drawioGuide.inAppKey] = sharedTool(
sharedToolSpecs.drawioGuide,
async ({ section }) => getGuideSection(section),
);
// Passive "new comments: N" signal (#417). PER-TURN state (forUser runs once
// per turn), so the watermark starts now and only comments a human leaves
// WHILE this turn runs are signalled — exactly the mid-turn loop; between-turn
@@ -7,6 +7,16 @@ import type {
DocmostClientLike,
CommentSignalTrackerLike,
} from './docmost-client.loader';
// Test-double type for the loopback client. `DocmostClientLike` is now DERIVED
// from the real `DocmostClient` (issue #446), so its method RETURN types are the
// concrete client shapes. These probe stubs deliberately return minimal shapes
// (e.g. `getPageRaw` yielding only `{ title }`), so the doubles use the same
// method NAMES but loose async returns; each is cast to `DocmostClientLike` at
// the (return-erased) mock site, leaving production positional-call safety intact.
type FakeDocmostClient = Partial<
Record<keyof DocmostClientLike, (...args: any[]) => Promise<any>>
>;
import { SHARED_TOOL_SPECS } from '../../../../../../packages/mcp/src/tool-specs';
// The REAL shared tracker factory, imported from source (same cross-boundary
// approach the tool-specs spec uses) so the in-app wiring is exercised against
@@ -268,15 +278,23 @@ describe('AiChatToolsService forUser + comment signal (real tracker)', () => {
// seeded at forUser time).
const future = new Date(Date.now() + 3_600_000).toISOString();
function buildService(fakeClient: Partial<DocmostClientLike>) {
function buildService(fakeClient: FakeDocmostClient) {
jest.spyOn(loader, 'loadDocmostMcp').mockResolvedValue({
DocmostClient: function () {
return fakeClient as DocmostClientLike;
} as unknown as loader.DocmostClientCtor,
sharedToolSpecs: SHARED_TOOL_SPECS as Record<string, loader.SharedToolSpec>,
sharedToolSpecs: SHARED_TOOL_SPECS as unknown as Record<string, loader.SharedToolSpec>,
// Wire the REAL factory so the in-app path is exercised end to end.
createCommentSignalTracker:
createCommentSignalTracker as unknown as loader.CommentSignalTrackerFactory,
// Pure no-network draw.io helpers (#424) — required on the loader return;
// this comment-signal test doesn't exercise them, so no-op stubs suffice.
searchShapes: (() => []) as unknown as loader.SearchShapesFn,
getGuideSection: (() => ({
section: '',
content: '',
sections: [],
})) as unknown as loader.GetGuideSectionFn,
});
return new AiChatToolsService(
tokenServiceStub as never,
@@ -317,7 +335,7 @@ describe('AiChatToolsService forUser + comment signal (real tracker)', () => {
afterEach(() => jest.restoreAllMocks());
it('emits the signal (model-only) on a non-comment tool when a new comment exists', async () => {
const fakeClient: Partial<DocmostClientLike> = {
const fakeClient: FakeDocmostClient = {
getPage: async () => ({
data: { title: 'Иранские языки', content: 'body' },
success: true,
@@ -342,7 +360,7 @@ describe('AiChatToolsService forUser + comment signal (real tracker)', () => {
});
it('does NOT add the signal to the listComments tool itself (tautological)', async () => {
const fakeClient: Partial<DocmostClientLike> = {
const fakeClient: FakeDocmostClient = {
listComments: async () => ({
items: [{ createdAt: future }],
resolvedThreadsHidden: 0,
@@ -356,7 +374,7 @@ describe('AiChatToolsService forUser + comment signal (real tracker)', () => {
});
it('no new comments => tool output is byte-identical AND the model sees no signal', async () => {
const fakeClient: Partial<DocmostClientLike> = {
const fakeClient: FakeDocmostClient = {
getPage: async () => ({
data: { title: 'T', content: 'body' },
success: true,
@@ -372,7 +390,7 @@ describe('AiChatToolsService forUser + comment signal (real tracker)', () => {
});
it('injection-safety: a malicious page title cannot forge a second signal', async () => {
const fakeClient: Partial<DocmostClientLike> = {
const fakeClient: FakeDocmostClient = {
getPage: async () => ({
data: { title: 'body-title', content: 'body' },
success: true,
@@ -2,266 +2,101 @@ import { createHash } from 'node:crypto';
import { existsSync, readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { pathToFileURL } from 'node:url';
import type { DocmostClient, SharedToolSpec } from '@docmost/mcp';
// Re-export SharedToolSpec so downstream server modules keep a single import
// path (they import it from this loader). The shape is DERIVED from the package
// entry, not re-declared here — see the import above (issue #446).
export type { SharedToolSpec } from '@docmost/mcp';
/**
* Minimal structural type for the `DocmostClient` class we consume from the
* ESM-only `@docmost/mcp` package. We only need the constructor + the read/write
* methods used by the per-user tool adapter; the full client surface lives in
* `packages/mcp/src/client.ts`. Signatures here mirror that file exactly.
*
* DRIFT GUARD: the method NAMES below are runtime-checked against the real
* `DocmostClient` by `packages/mcp/test/unit/client-host-contract.test.mjs`
* (which can import the ESM class directly). If you rename/remove a method here
* or in client.ts, that test fails so a stale mirror cannot silently ship a
* runtime "x is not a function" into an agent tool call. Keep the two in sync.
*
* STAGED PLAN full derivation `DocmostClientLike = <real DocmostClient type>`
* (issue #193, layer 3) is intentionally NOT done; it stays a hand-mirror for
* now because of two verified blockers across the ESM(mcp)/CJS(server) boundary:
* 1. `@docmost/mcp` emits NO declaration files (its tsconfig has no
* `declaration`, package.json has no `types`/types-export) and the server
* tsconfig has no path mapping for it the server only loads it via the
* runtime `import()` trick below, so there is no type to import today.
* 2. The real client methods have inferred, CONCRETE return types; the in-app
* tool adapter reads results through loose `Record<string,unknown>` returns
* + `as` casts (e.g. `(result?.data ?? {}) as { title?: string }`).
* Deriving the exact type would make those casts non-overlapping ("may be a
* mistake") and break the build, and `Partial<DocmostClientLike>` test stubs
* would have to satisfy the full concrete surface.
* To do it safely later (incrementally): (a) turn on `declaration: true` in
* packages/mcp/tsconfig.json + add a `types` export condition and commit the
* emitted `.d.ts`; (b) `import type { DocmostClient } from '@docmost/mcp'` here
* and replace this interface with a `Pick<DocmostClient, ...>` of the consumed
* methods; (c) audit every `as` cast in ai-chat-tools.service.ts against the now
* concrete return types (double-cast through `unknown` only where genuinely
* needed); (d) keep the runtime guard test as a belt-and-braces check. Until
* then the guard test above is the cheap, behaviour-neutral protection.
* The exact set of `DocmostClient` methods the per-user in-app tool adapter
* consumes. This is the AUTHORITATIVE list of the client surface the server
* depends on; the adapter calls these methods POSITIONALLY, so this set is what
* the derived type below type-checks against the real class (issue #446).
*/
export interface DocmostClientLike {
type DocmostClientMethod =
// --- read ---
search(
query: string,
spaceId?: string,
limit?: number,
): Promise<{ items: unknown[]; success: boolean }>;
getPage(
pageId: string,
): Promise<{ data: Record<string, unknown>; success: boolean }>;
// Light raw page info (`/pages/info`): title + slugId + ProseMirror content,
// WITHOUT the Markdown render / subpage expansion getPage does. Used by the
// comment-signal probe to read just the page title on a hit.
getPageRaw(pageId: string): Promise<Record<string, unknown> | null>;
getWorkspace(): Promise<{ data: Record<string, unknown>; success: boolean }>;
getSpaces(): Promise<unknown[]>;
listPages(
spaceId?: string,
limit?: number,
tree?: boolean,
): Promise<unknown[]>;
listSidebarPages(spaceId: string, pageId?: string): Promise<unknown[]>;
getOutline(pageId: string): Promise<Record<string, unknown>>;
getPageJson(pageId: string): Promise<Record<string, unknown>>;
getNode(pageId: string, nodeId: string): Promise<Record<string, unknown>>;
searchInPage(
pageId: string,
query: string,
opts?: { regex?: boolean; caseSensitive?: boolean; limit?: number },
): Promise<Record<string, unknown>>;
getTable(pageId: string, tableRef: string): Promise<Record<string, unknown>>;
// Returns `{ items, resolvedThreadsHidden }`. DEFAULT (includeResolved unset/
// false) hides resolved threads wholesale; pass true for the full feed.
listComments(
pageId: string,
includeResolved?: boolean,
): Promise<{ items: unknown[]; resolvedThreadsHidden: number }>;
getComment(
commentId: string,
): Promise<{ data: Record<string, unknown>; success: boolean }>;
checkNewComments(
spaceId: string,
since: string,
parentPageId?: string,
): Promise<unknown>;
listShares(): Promise<unknown[]>;
listPageHistory(
pageId: string,
cursor?: string,
): Promise<{ items: unknown[]; nextCursor: string | null }>;
getPageHistory(historyId: string): Promise<Record<string, unknown>>;
diffPageVersions(
pageId: string,
from?: string,
to?: string,
): Promise<Record<string, unknown>>;
exportPageMarkdown(pageId: string): Promise<string>;
| 'search'
| 'getPage'
| 'getPageRaw'
| 'getWorkspace'
| 'getSpaces'
| 'listPages'
| 'getTree'
| 'getPageContext'
| 'listSidebarPages'
| 'getOutline'
| 'getPageJson'
| 'getNode'
| 'searchInPage'
| 'getTable'
| 'listComments'
| 'getComment'
| 'checkNewComments'
| 'listShares'
| 'listPageHistory'
| 'getPageHistory'
| 'diffPageVersions'
| 'exportPageMarkdown'
// --- write (page) ---
createPage(
title: string,
content: string,
spaceId: string,
parentPageId?: string,
): Promise<{ data: Record<string, unknown>; success: boolean }>;
// Markdown content update via the collab path (carries provenance via the
// collab-token provider). Optionally also updates the title.
updatePage(
pageId: string,
content: string,
title?: string,
): Promise<Record<string, unknown>>;
// Title-only rename via REST.
renamePage(
pageId: string,
title: string,
): Promise<Record<string, unknown>>;
// Move via REST. parentPageId null => move to space root.
movePage(
pageId: string,
parentPageId: string | null,
position?: string,
): Promise<unknown>;
// SOFT delete only (POST /pages/delete with { pageId }). NEVER permanent.
deletePage(pageId: string): Promise<unknown>;
editPageText(
pageId: string,
edits: Array<{ find: string; replace: string; replaceAll?: boolean }>,
): Promise<Record<string, unknown>>;
patchNode(
pageId: string,
nodeId: string,
node: unknown,
): Promise<Record<string, unknown>>;
insertNode(
pageId: string,
node: unknown,
opts: {
position: 'before' | 'after' | 'append';
anchorNodeId?: string;
anchorText?: string;
},
): Promise<Record<string, unknown>>;
deleteNode(
pageId: string,
nodeId: string,
): Promise<Record<string, unknown>>;
updatePageJson(
pageId: string,
doc?: unknown,
title?: string,
): Promise<Record<string, unknown>>;
// Attach an author-inline footnote after the first occurrence of anchorText;
// numbering + the footnotes list are derived server-side.
insertFootnote(
pageId: string,
anchorText: string,
text: string,
): Promise<Record<string, unknown>>;
// Download a web image and insert it into the page (append, or replace/after a
// text anchor). `url` is the image http(s) URL.
insertImage(
pageId: string,
url: string,
opts?: {
align?: 'left' | 'center' | 'right';
alt?: string;
replaceText?: string;
afterText?: string;
},
): Promise<Record<string, unknown>>;
// Swap an existing image (by its attachmentId) for a new one fetched from a web
// URL, repointing every reference in the live document.
replaceImage(
pageId: string,
oldAttachmentId: string,
url: string,
opts?: { align?: 'left' | 'center' | 'right'; alt?: string },
): Promise<Record<string, unknown>>;
// --- draw.io diagrams (#423, stage 1) ---
// Read a diagram as decoded mxGraph XML (default) or the raw .drawio.svg.
// meta.hash is the optimistic-lock key drawioUpdate expects as baseHash.
drawioGet(
pageId: string,
node: string,
format?: 'xml' | 'svg',
): Promise<Record<string, unknown>>;
// Lint mxGraph XML, build the .drawio.svg attachment and insert a drawio node.
drawioCreate(
pageId: string,
where: {
position: 'before' | 'after' | 'append';
anchorNodeId?: string;
anchorText?: string;
},
xml: string,
title?: string,
): Promise<Record<string, unknown>>;
// Optimistic-locked full replacement of a diagram (baseHash from drawioGet).
drawioUpdate(
pageId: string,
node: string,
xml: string,
baseHash: string,
): Promise<Record<string, unknown>>;
tableInsertRow(
pageId: string,
tableRef: string,
cells: string[],
index?: number,
): Promise<Record<string, unknown>>;
tableDeleteRow(
pageId: string,
tableRef: string,
index: number,
): Promise<Record<string, unknown>>;
tableUpdateCell(
pageId: string,
tableRef: string,
row: number,
col: number,
text: string,
): Promise<Record<string, unknown>>;
copyPageContent(
sourcePageId: string,
targetPageId: string,
): Promise<Record<string, unknown>>;
importPageMarkdown(
pageId: string,
fullMarkdown: string,
): Promise<Record<string, unknown>>;
sharePage(
pageId: string,
searchIndexing?: boolean,
): Promise<Record<string, unknown>>;
unsharePage(pageId: string): Promise<Record<string, unknown>>;
restorePageVersion(historyId: string): Promise<Record<string, unknown>>;
// The opts type declares deleteComments? to match the real client signature,
// but the agent tool NEVER sets it (comment deletion stays unreachable).
transformPage(
pageId: string,
transformJs: string,
opts?: { dryRun?: boolean; deleteComments?: boolean },
): Promise<Record<string, unknown>>;
| 'createPage'
| 'updatePage'
| 'renamePage'
| 'movePage'
| 'deletePage'
| 'editPageText'
| 'patchNode'
| 'insertNode'
| 'deleteNode'
| 'updatePageJson'
| 'tableInsertRow'
| 'tableDeleteRow'
| 'tableUpdateCell'
| 'copyPageContent'
| 'importPageMarkdown'
| 'sharePage'
| 'unsharePage'
| 'restorePageVersion'
| 'transformPage'
| 'stashPage'
// --- write (image / footnote), in-app since #410 ---
| 'insertImage'
| 'replaceImage'
| 'insertFootnote'
// --- draw.io diagrams (#423 stage 1, #424 stage 2) ---
// DERIVED from the real DocmostClient (#446): drawioCreate/drawioUpdate carry
// the optional layout:"elk" 5th arg in the real signature, so the layout parity
// (#440) is inherited automatically — no hand-written mirror to keep in sync.
| 'drawioGet'
| 'drawioCreate'
| 'drawioUpdate'
// --- draw.io high-level semantic tools (#425 stage 3) ---
| 'drawioEditCells'
| 'drawioFromGraph'
| 'drawioFromMermaid'
// --- write (comment) ---
createComment(
pageId: string,
content: string,
type?: 'page' | 'inline',
selection?: string,
parentCommentId?: string,
suggestedText?: string,
): Promise<{ data: Record<string, unknown>; success: boolean }>;
resolveComment(
commentId: string,
resolved: boolean,
): Promise<Record<string, unknown>>;
// Serialize a page + mirror its internal images into the blob sandbox; returns
// ONLY a short anonymous URL (the body never enters the model context).
stashPage(pageId: string): Promise<{
uri: string;
sha256: string;
size: number;
images: { mirrored: number; failed: number };
}>;
}
| 'createComment'
| 'resolveComment';
/**
* The client surface the per-user tool adapter consumes, DERIVED from the real
* `DocmostClient` type in `@docmost/mcp` (issue #446, restored #294 debt). This
* replaces the former hand-mirror of ~45 method signatures.
*
* `import type` (above) is fully ERASED at compile time, so nothing is actually
* imported from the ESM-only package at runtime the server still loads the
* class through the dynamic `import()` trick in `loadDocmostMcp` below; this is
* purely a compile-time type. Deriving via `Pick` means a parameter reorder or a
* type change to any of these methods in `client.ts` now becomes a SERVER
* COMPILE ERROR at the positional call sites in ai-chat-tools.service.ts,
* instead of a silent runtime "wrong argument" failure inside an agent tool.
*
* This made the old name-only drift-guard test
* (packages/mcp/test/unit/client-host-contract.test.mjs) redundant tsc now
* enforces both names AND signatures so that test was removed.
*/
export type DocmostClientLike = Pick<DocmostClient, DocmostClientMethod>;
export type DocmostClientConfig = {
apiUrl: string;
@@ -283,32 +118,7 @@ export type DocmostClientConfig = {
};
export interface DocmostClientCtor {
new (config: DocmostClientConfig): DocmostClientLike;
}
/**
* Local hand-mirror of the `SharedToolSpec` shape exported from
* `@docmost/mcp` (packages/mcp/src/tool-specs.ts). Same approach as
* `DocmostClientLike`: we do not import the ESM package's types directly across
* the CJS/ESM boundary. The registry itself has no runtime deps, but keeping the
* type local avoids coupling the server build to the package's type surface.
*
* `buildShape` is intentionally zod-agnostic: it returns a plain ZodRawShape
* built with whatever zod namespace the caller passes (the server passes its own
* zod v4; the MCP package passes its zod v3). See the registry module comment.
*/
export interface SharedToolSpec {
mcpName: string;
inAppKey: string;
description: string;
// Deferred-tool metadata (#332). Optional in this mirror so an older/stale
// @docmost/mcp build (pre-#332) still type-checks; the in-app catalog builder
// reads them defensively. The external /mcp server ignores both fields.
tier?: 'core' | 'deferred';
catalogLine?: string;
// Loose `z` on purpose: the registry is zod-agnostic so the server can pass
// its own zod (v4) and the MCP package its own (v3) into the same builder.
buildShape?: (z: any) => Record<string, unknown>;
new (config: DocmostClientConfig): DocmostClient;
}
/**
@@ -340,6 +150,19 @@ export type CommentSignalTrackerFactory = (options: {
debounceMs?: number;
}) => CommentSignalTrackerLike;
// Pure, no-network draw.io helpers (#424). These are plain functions on the
// module (NOT DocmostClient methods) — the in-app AI-SDK service calls them
// directly to wire drawioShapes / drawioGuide, mirroring the MCP server.
export type SearchShapesFn = (
query: string,
opts?: { category?: string; limit?: number },
) => Array<Record<string, unknown>>;
export type GetGuideSectionFn = (section?: string) => {
section: string;
content: string;
sections: string[];
};
interface DocmostMcpModule {
DocmostClient: DocmostClientCtor;
SHARED_TOOL_SPECS: Record<string, SharedToolSpec>;
@@ -352,6 +175,15 @@ interface DocmostMcpModule {
// the mocked loader in unit tests) — the stale-check below is a NO-OP when it
// is missing, so an older build never wrongly fails startup.
REGISTRY_STAMP?: string;
// Pure, no-network draw.io helpers (#424) backing drawioShapes / drawioGuide.
// Those two specs are `inlineBothHosts` (they stay in SHARED_TOOL_SPECS for the
// shared contract but carry no execute — their catalog loader uses import.meta
// and can't be value-imported into the zod-agnostic tool-specs.ts), so the
// in-app service wires them INLINE off these helpers, mirroring the standalone
// MCP host. Exposed off the loaded module so the service and its test mocks can
// reach them.
searchShapes: SearchShapesFn;
getGuideSection: GetGuideSectionFn;
}
/**
@@ -415,6 +247,8 @@ export async function loadDocmostMcp(): Promise<{
DocmostClient: DocmostClientCtor;
sharedToolSpecs: Record<string, SharedToolSpec>;
createCommentSignalTracker?: CommentSignalTrackerFactory;
searchShapes: SearchShapesFn;
getGuideSection: GetGuideSectionFn;
}> {
if (!modulePromise) {
modulePromise = (async () => {
@@ -460,5 +294,8 @@ export async function loadDocmostMcp(): Promise<{
// Optional: forwarded when present so the in-app layer can build the passive
// comment signal (#417); undefined on a stale build => signal disabled.
createCommentSignalTracker: mod.createCommentSignalTracker,
// Pure no-network draw.io helpers (#424); not client methods.
searchShapes: mod.searchShapes,
getGuideSection: mod.getGuideSection,
};
}
@@ -17,8 +17,10 @@ import { SHARED_TOOL_SPECS } from '../../../../../../packages/mcp/src/tool-specs
* This test fails the build if a spec is added to the registry but never wired
* in-app, if an `inAppKey` is renamed without updating the service, if the
* description drifts between the registry and the exposed tool, if the
* snake_case `mcpName` <-> camelCase `inAppKey` convention is broken, or if the
* exposed tool's input-schema keys diverge from the spec's `buildShape`.
* `mcpName === inAppKey` convention is broken (issue #412 unified the external
* MCP tool name with the in-app key both are the same camelCase identifier),
* or if the exposed tool's input-schema keys diverge from the spec's
* `buildShape`.
*
* It does NOT need @docmost/mcp built: the registry is imported from TS source,
* and the ESM loader is mocked so `forUser()` never dynamically imports the
@@ -45,6 +47,16 @@ describe('SHARED_TOOL_SPECS contract parity', () => {
string,
loader.SharedToolSpec
>,
// Pure no-network draw.io helpers (#424). The contract test never executes
// a tool body, so type-correct stubs suffice (the real functions can't be
// imported here — drawio-shapes.ts uses import.meta, incompatible with the
// CommonJS jest transform).
searchShapes: (() => []) as unknown as loader.SearchShapesFn,
getGuideSection: (() => ({
section: 'index',
content: '',
sections: [],
})) as unknown as loader.GetGuideSectionFn,
});
const service = new AiChatToolsService(
tokenServiceStub as never,
@@ -64,13 +76,9 @@ describe('SHARED_TOOL_SPECS contract parity', () => {
afterAll(() => jest.restoreAllMocks());
// camelCase -> snake_case, matching the registry's mcpName convention.
const toSnake = (s: string) =>
s.replace(/[A-Z]/g, (c) => `_${c.toLowerCase()}`);
// Type as the (optional-buildShape) SharedToolSpec; the `satisfies` literal
// above otherwise narrows to a union where some members lack buildShape.
const specEntries = Object.entries(SHARED_TOOL_SPECS) as Array<
const specEntries = Object.entries(SHARED_TOOL_SPECS) as unknown as Array<
[string, loader.SharedToolSpec]
>;
@@ -86,8 +94,8 @@ describe('SHARED_TOOL_SPECS contract parity', () => {
expect(spec.inAppKey).toBe(registryKey);
});
it('mcpName is the snake_case form of inAppKey', () => {
expect(spec.mcpName).toBe(toSnake(spec.inAppKey));
it('mcpName equals inAppKey (unified camelCase name, #412)', () => {
expect(spec.mcpName).toBe(spec.inAppKey);
});
it('is exposed in-app under its inAppKey', () => {
@@ -36,7 +36,7 @@ describe('tool tier metadata (#332)', () => {
});
it('#410 image tools are DEFERRED, footnote tool is CORE', () => {
// insert_footnote is core (symmetric with editPageText); the image tools stay
// insertFootnote is core (symmetric with editPageText); the image tools stay
// deferred (rare, fat — loaded on demand). Assert both the spec tier and the
// CORE_TOOL_SET membership so a future tier edit that desyncs them fails here.
expect(SHARED_TOOL_SPECS.insertFootnote.tier).toBe('core');
@@ -123,7 +123,14 @@ describe('deferred catalog ↔ live forUser() toolset partition (#332, F3)', ()
DocmostClient: function () {
return {} as DocmostClientLike;
} as unknown as loader.DocmostClientCtor,
sharedToolSpecs: SHARED_TOOL_SPECS as Record<string, loader.SharedToolSpec>,
sharedToolSpecs: SHARED_TOOL_SPECS as unknown as Record<string, loader.SharedToolSpec>,
// Pure no-network draw.io helpers (#424); tool bodies are never executed here.
searchShapes: (() => []) as unknown as loader.SearchShapesFn,
getGuideSection: (() => ({
section: 'index',
content: '',
sections: [],
})) as unknown as loader.GetGuideSectionFn,
});
const service = new AiChatToolsService(
{
@@ -232,6 +239,16 @@ describe('applyLoadTools (#332)', () => {
expect(LOAD_TOOLS_DESCRIPTION).toContain('only ACTIVATES them');
expect(LOAD_TOOLS_DESCRIPTION).toContain('callable on your NEXT step');
});
it('loadTools description tells the model CORE tools are always active (#444)', () => {
expect(LOAD_TOOLS_DESCRIPTION).toContain(
'Tools NOT listed in the catalog are CORE and ALWAYS active',
);
expect(LOAD_TOOLS_DESCRIPTION).toContain('NEVER via loadTools');
// Names it out explicitly so the model doesn't loadTools a core tool.
expect(LOAD_TOOLS_DESCRIPTION).toContain('createComment');
expect(LOAD_TOOLS_DESCRIPTION).toContain('searchInPage');
});
});
describe('editorial "Corrector" scenario is fully served by CORE (#332)', () => {
@@ -60,10 +60,10 @@ export const CORE_TOOL_KEYS = [
'listComments',
'resolveComment',
'editPageText',
// #330 search_in_page — frequent for editorial sweeps; core despite predating
// #330 searchInPage — frequent for editorial sweeps; core despite predating
// the issue's tier list.
'searchInPage',
// #410 insert_footnote — core so pinpoint citations to already-written text
// #410 insertFootnote — core so pinpoint citations to already-written text
// don't degrade into literal `^[...]`; kept symmetric with editPageText.
'insertFootnote',
] as const;
@@ -84,7 +84,10 @@ export const LOAD_TOOLS_DESCRIPTION =
'block in your instructions. Pass the EXACT tool names from the catalog; this\n' +
'call only ACTIVATES them and returns { loaded: [...] } — the tools become\n' +
'callable on your NEXT step. Load several names in one call when the task clearly\n' +
'needs them. Unknown names are rejected with the list of valid ones.';
'needs them. Unknown names are rejected with the list of valid ones.\n' +
'Tools NOT listed in the catalog are CORE and ALWAYS active — call them directly,\n' +
'NEVER via loadTools (e.g. createComment, listComments, resolveComment,\n' +
'editPageText, searchInPage).';
/**
* Tier + catalogLine for the INLINE ai-chat tools those defined per-layer in
@@ -121,12 +124,9 @@ export const INLINE_TOOL_TIERS: Record<
// --- deferred inline ---
// NOTE: createPage, renamePage, movePage, deletePage, updatePageJson and
// exportPageMarkdown moved to @docmost/mcp's SHARED_TOOL_SPECS (#294); they
// carry their own deferred tier + catalogLine there.
updatePageContent: {
tier: 'deferred',
catalogLine:
"updatePageContent — replace a page's body (and optionally title) with new Markdown.",
},
// carry their own deferred tier + catalogLine there. updatePageContent moved
// there too as updatePageMarkdown (#411) — a shared registry spec now, so it
// is no longer an inline tier entry.
listSidebarPages: {
tier: 'deferred',
catalogLine:
@@ -138,7 +138,7 @@ export const INLINE_TOOL_TIERS: Record<
},
// NOTE: tableInsertRow, tableDeleteRow and tableUpdateCell moved to
// @docmost/mcp's SHARED_TOOL_SPECS (#294); they carry their own deferred tier +
// catalogLine there. getTable stays inline (its MCP name table_get breaks the
// catalogLine there. getTable stays inline (its MCP name tableGet breaks the
// snake_case(inAppKey) convention, so it has no shared spec).
// NOTE: checkNewComments moved to @docmost/mcp's SHARED_TOOL_SPECS (#294);
// it carries its own deferred tier + catalogLine there.
@@ -150,7 +150,7 @@ export const INLINE_TOOL_TIERS: Record<
// NOTE: sharePage moved to @docmost/mcp's SHARED_TOOL_SPECS (#294); it carries
// its own deferred tier + catalogLine there. transformPage stays inline (its
// schema deliberately diverges — it omits the deleteComments field the MCP
// docmost_transform exposes, a comment-deletion guardrail).
// docmostTransform exposes, a comment-deletion guardrail).
transformPage: {
tier: 'deferred',
catalogLine: "transformPage — run a sandboxed JS transform over a page's document.",
@@ -12,3 +12,22 @@ export class SearchResponseDto {
updatedAt: Date;
space: Partial<Space>;
}
// Response shape for the opt-in agent-lookup mode (#443, `substring: true`).
// Additive to the FTS response: carries the location (`path`), a windowed
// `snippet` around the first match and a per-response sort `score`. The MCP
// layer maps `id → pageId`; `slugId` is never exposed.
export class SearchLookupResponseDto {
id: string;
slugId: string;
title: string;
parentPageId: string | null;
// Ancestor titles from the space root down to the direct parent; [] for a
// root page.
path: string[];
// ~300–500 chars around the first match (or a leading text window / extended
// ts_headline fallback).
snippet: string;
// 0..1 float, meaningful ONLY for sorting within one response.
score: number;
}
@@ -30,6 +30,31 @@ export class SearchDTO {
@IsOptional()
@IsNumber()
offset?: number;
// --- Opt-in agent-lookup mode (#443). ------------------------------------
// These fields are ADDITIVE and default-off: a web client that sends none of
// them gets byte-identical FTS behaviour and result shape. They are only read
// by the substring/path/snippet code path in SearchService.searchPage.
//
// NOTE (standalone stdio vs stock upstream): stock upstream validates this DTO
// with `whitelist: true`, so an older server silently strips these unknown
// fields and the request degrades gracefully to the plain FTS behaviour.
// Enables the hybrid substring branch (title + text_content LIKE) merged with
// the existing FTS branch, plus tiered ranking, path and windowed snippet.
@IsOptional()
@IsBoolean()
substring?: boolean;
// Restrict the search to a page and all of its descendants (inclusive).
@IsOptional()
@IsString()
parentPageId?: string;
// Match titles only; do not scan text_content.
@IsOptional()
@IsBoolean()
titleOnly?: boolean;
}
export class SearchShareDTO extends SearchDTO {
@@ -60,6 +60,12 @@ export class SearchController {
}
}
// #443 graceful degradation: on EE/Typesense instances the request routes to
// the Typesense backend, which does NOT implement the opt-in agent-lookup
// mode. The `substring`/`parentPageId`/`titleOnly` fields are silently ignored
// and the response carries no `path`/`snippet`/`score` and no substring/tier
// ranking — it degrades to plain Typesense FTS. The native lookup mode below
// is Postgres-search-driver only.
if (this.environmentService.getSearchDriver() === 'typesense') {
return this.searchTypesense(searchDto, {
userId: user.id,
@@ -0,0 +1,95 @@
import {
computeLookupScore,
escapeLikePattern,
SearchLookupTier,
} from './search.service';
/**
* Pure-function coverage for the #443 agent-lookup helpers:
* - escapeLikePattern: LIKE-metacharacter escaping so `%`/`_`/`\` are literals
* (the acceptance-table requirement that a query of `%` or `_` does NOT match
* everything);
* - computeLookupScore: the tiered 0..1 ranking score, where a stronger tier
* always outranks a weaker one regardless of the in-tier secondary signal.
*
* The DB-touching branch (substring UNION FTS, path CTE, snippet window) is
* covered by the integration spec against the real schema.
*/
describe('escapeLikePattern', () => {
it('escapes the LIKE metacharacters % _ and \\', () => {
expect(escapeLikePattern('%')).toBe('\\%');
expect(escapeLikePattern('_')).toBe('\\_');
expect(escapeLikePattern('\\')).toBe('\\\\');
});
it('escapes the backslash FIRST so it does not double-escape %/_', () => {
// Input `\%` must become `\\` + `\%` = `\\\%`, not `\\%`.
expect(escapeLikePattern('\\%')).toBe('\\\\\\%');
});
it('leaves ordinary technical chars (. - / digits) untouched', () => {
expect(escapeLikePattern('backup-srv.local')).toBe('backup-srv.local');
expect(escapeLikePattern('10.0.12')).toBe('10.0.12');
expect(escapeLikePattern('WB-MGE-30D86B')).toBe('WB-MGE-30D86B');
expect(escapeLikePattern('a/b')).toBe('a/b');
});
it('escapes only the metacharacters in a mixed string', () => {
expect(escapeLikePattern('50%_off.zip')).toBe('50\\%\\_off.zip');
});
it('is null/undefined-safe', () => {
expect(escapeLikePattern(undefined as any)).toBe('');
expect(escapeLikePattern(null as any)).toBe('');
});
});
describe('computeLookupScore', () => {
it('keeps every score within (0, 1]', () => {
for (const tier of [
SearchLookupTier.TITLE_EXACT,
SearchLookupTier.TITLE_SUBSTRING,
SearchLookupTier.TEXT,
]) {
for (const secondary of [0, 0.001, 1, 100, 1e6]) {
const s = computeLookupScore({ tier, secondary });
expect(s).toBeGreaterThan(0);
expect(s).toBeLessThanOrEqual(1);
}
}
});
it('a stronger tier ALWAYS outranks a weaker tier, whatever the secondary', () => {
// Weak tier with a huge secondary must still lose to a strong tier with a
// tiny secondary — tiers dominate.
const strongLowSecondary = computeLookupScore({
tier: SearchLookupTier.TITLE_EXACT,
secondary: 0,
});
const weakHighSecondary = computeLookupScore({
tier: SearchLookupTier.TEXT,
secondary: 1e9,
});
expect(strongLowSecondary).toBeGreaterThan(weakHighSecondary);
});
it('within a tier a larger secondary sorts higher', () => {
const lo = computeLookupScore({
tier: SearchLookupTier.TEXT,
secondary: 0.1,
});
const hi = computeLookupScore({
tier: SearchLookupTier.TEXT,
secondary: 5,
});
expect(hi).toBeGreaterThan(lo);
});
it('treats a negative/absent secondary as 0', () => {
const zero = computeLookupScore({ tier: SearchLookupTier.TEXT, secondary: 0 });
expect(computeLookupScore({ tier: SearchLookupTier.TEXT })).toBe(zero);
expect(
computeLookupScore({ tier: SearchLookupTier.TEXT, secondary: -5 }),
).toBe(zero);
});
});
+401 -2
View File
@@ -1,6 +1,9 @@
import { Injectable } from '@nestjs/common';
import { SearchDTO, SearchSuggestionDTO } from './dto/search.dto';
import { SearchResponseDto } from './dto/search-response.dto';
import {
SearchLookupResponseDto,
SearchResponseDto,
} from './dto/search-response.dto';
import { InjectKysely } from 'nestjs-kysely';
import { KyselyDB } from '@docmost/db/types/kysely.types';
import { sql } from 'kysely';
@@ -34,6 +37,53 @@ export function buildTsQuery(raw: string): string {
return tsquery(cleaned + '*');
}
// Escape the LIKE metacharacters (`%`, `_`, `\`) in a raw user query so every
// character — including `.`, `-`, `_`, `%`, `/` — is matched LITERALLY by a
// `col LIKE '%' || q || '%'` predicate. Without this, a query of `%` or `_`
// would match every row (see the #443 acceptance table). The backslash is the
// escape char (Postgres LIKE default), so it must be escaped first.
export function escapeLikePattern(raw: string): string {
return (raw ?? '')
.replace(/\\/g, '\\\\')
.replace(/%/g, '\\%')
.replace(/_/g, '\\_');
}
// Ranking tiers for the agent-lookup mode (#443), highest first. A hit's tier
// is the strongest way it matched; ties inside a tier break on a secondary
// signal (FTS rank, or first-match position). The numeric `score` returned to
// the caller is derived from (tier, secondary) and is meaningful ONLY for
// ordering within a single response.
export enum SearchLookupTier {
// Title equals the query, case-insensitively.
TITLE_EXACT = 3,
// Query is a substring of the title.
TITLE_SUBSTRING = 2,
// Query matched in the text (substring or FTS).
TEXT = 1,
}
export interface RankableHit {
tier: SearchLookupTier;
// Secondary in-tier signal, higher = better (e.g. ts_rank, or a
// position-derived closeness score). Defaults to 0.
secondary?: number;
}
// Map (tier, secondary) → a 0..1 float used ONLY to sort one response.
//
// Formula: score = (tier + squash(secondary)) / (maxTier + 1), where
// squash(x) = x / (1 + x) maps any non-negative secondary into [0, 1)
// so a stronger tier ALWAYS outranks a weaker one regardless of the secondary
// value, and within a tier a larger secondary sorts higher. maxTier is the top
// enum value (TITLE_EXACT = 3), so the divisor keeps the result in (0, 1].
export function computeLookupScore(hit: RankableHit): number {
const maxTier = SearchLookupTier.TITLE_EXACT;
const secondary = Math.max(0, hit.secondary ?? 0);
const squashed = secondary / (1 + secondary);
return (hit.tier + squashed) / (maxTier + 1);
}
@Injectable()
export class SearchService {
constructor(
@@ -50,12 +100,19 @@ export class SearchService {
userId?: string;
workspaceId: string;
},
): Promise<{ items: SearchResponseDto[] }> {
): Promise<{ items: SearchResponseDto[] | SearchLookupResponseDto[] }> {
const { query } = searchParams;
if (query.length < 1) {
return { items: [] };
}
// Opt-in agent-lookup mode (#443). Guarded by the `substring` flag so the
// web-UI (which never sets it) keeps byte-identical FTS behaviour below.
if (searchParams.substring) {
return this.searchPageLookup(searchParams, opts);
}
const searchQuery = buildTsQuery(query);
let queryResults = this.db
@@ -175,6 +232,348 @@ export class SearchService {
return { items: searchResults };
}
/**
* Agent-lookup search (#443, opt-in via `SearchDTO.substring`).
*
* ADDITIVE to the FTS path: runs a substring branch (title + optionally
* text_content, LIKE with metacharacters escaped) MERGED with the existing
* FTS branch, so technical tokens that the `english` tokenizer mangles
* (`backup-srv.local`, `10.0.12.5`, `WB-MGE-30D86B`) are still found even
* when `buildTsQuery()` returns '' for a dotted/numeric query. Results carry a
* location (`path`), a windowed `snippet` and a per-response `score`.
*
* The whole method is only reached when `substring: true`; the web-UI never
* sets it, so its behaviour is unchanged.
*/
private async searchPageLookup(
searchParams: SearchDTO,
opts: { userId?: string; workspaceId: string },
): Promise<{ items: SearchLookupResponseDto[] }> {
const rawQuery = searchParams.query.trim();
if (!rawQuery) {
return { items: [] };
}
const limit = Math.min(Math.max(searchParams.limit || 10, 1), 50);
// Normalize the query the same way as the FTS / suggest path: f_unaccent +
// lower, done in SQL. `q` is the escaped LIKE pattern body (literal chars).
const likeBody = escapeLikePattern(rawQuery);
// Compare against `LOWER(f_unaccent(col))`; unaccent+lower the needle too.
const needle = sql<string>`LOWER(f_unaccent(${rawQuery}))`;
const likePattern = sql<string>`LOWER(f_unaccent(${'%' + likeBody + '%'}))`;
const tsQuery = buildTsQuery(rawQuery);
const hasTsQuery = tsQuery.length > 0;
// --- Resolve the space scope. ---------------------------------------------
// Mirrors searchPage: explicit spaceId, else the authenticated user's member
// spaces. The share path is not exposed to this opt-in mode.
let spaceIds: string[] = [];
if (searchParams.spaceId) {
spaceIds = [searchParams.spaceId];
} else if (opts.userId) {
spaceIds = await this.spaceMemberRepo.getUserSpaceIds(opts.userId);
} else {
return { items: [] };
}
if (spaceIds.length === 0) {
return { items: [] };
}
// --- Optional parentPageId subtree scope (inclusive). ---------------------
// Reuse the same recursive-descendants pattern used for share-scope.
let descendantIds: string[] | null = null;
if (searchParams.parentPageId) {
const descendants = await this.pageRepo.getPageAndDescendants(
searchParams.parentPageId,
{ includeContent: false },
);
descendantIds = descendants.map((p: any) => p.id);
if (descendantIds.length === 0) {
return { items: [] };
}
}
// --- Candidate query: substring (title + text) UNION FTS. -----------------
// We compute everything the ranker needs in SQL and pull only small columns
// (never the whole text_content) into Node:
// - titleExact / titleSub: tier signals
// - textMatchPos: 1-based position of the first text match (0 = none)
// - ftsRank: ts_rank for the FTS secondary signal (0 when no tsquery)
// - snippet: windowed ~500 chars around the first text match, or a leading
// text window (title-only hit), or an extended ts_headline fallback.
const N_BEFORE = 60; // chars of context before the first match
const SNIPPET_LEN = 500;
let candidates = this.db
.selectFrom('pages')
.select([
'pages.id as id',
'pages.slugId as slugId',
'pages.title as title',
'pages.parentPageId as parentPageId',
// Tier signals.
sql<boolean>`LOWER(f_unaccent(coalesce(pages.title, ''))) = ${needle}`.as(
'titleExact',
),
sql<boolean>`LOWER(f_unaccent(coalesce(pages.title, ''))) LIKE ${likePattern} ESCAPE '\\'`.as(
'titleSub',
),
// 1-based position of the first text match (0 = no text match).
sql<number>`strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle})`.as(
'textMatchPos',
),
// FTS secondary signal (0 when the tsquery is empty).
hasTsQuery
? sql<number>`ts_rank(pages.tsv, to_tsquery('english', f_unaccent(${tsQuery})))`.as(
'ftsRank',
)
: sql<number>`0`.as('ftsRank'),
// Windowed snippet, computed entirely in SQL. Priority:
// 1. window around the first text match;
// 2. otherwise (titleOnly: no snippet; else) a leading window of the
// page text (title-only hit);
// 3. otherwise an extended ts_headline for pure-FTS hits.
//
// #443 snippet-position fix: the match position (`strpos`) is computed in
// the LOWER(f_unaccent(...)) space, but f_unaccent is NOT length-
// preserving (ß→ss, æ→ae, …→..., ½→ 1/2, full-width forms), so slicing
// the ORIGINAL text at that position was misaligned — a single expanding
// char before the match shifted the window (or ran it past end → empty).
// We now slice from the SAME LOWER(f_unaccent(...)) string so position
// and slice share one coordinate space. DELIBERATE trade-off: the snippet
// loses original case/diacritics — acceptable for an agent-facing snippet
// (position accuracy over original-glyph fidelity). The ts_headline branch
// matches over the ORIGINAL text itself, so it is unaffected and kept as-is.
searchParams.titleOnly
? sql<string>`''`.as('snippet')
: sql<string>`
coalesce(
case
when strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) > 0
then substring(
LOWER(f_unaccent(coalesce(pages.text_content, '')))
from greatest(1, strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) - ${N_BEFORE})
for ${SNIPPET_LEN}
)
when coalesce(pages.text_content, '') <> ''
then substring(LOWER(f_unaccent(pages.text_content)) from 1 for 300)
${
hasTsQuery
? sql`else ts_headline('english', coalesce(pages.text_content, ''), to_tsquery('english', f_unaccent(${tsQuery})), 'MinWords=25, MaxWords=40, MaxFragments=3')`
: sql``
}
end,
''
)
`.as('snippet'),
])
.where('pages.deletedAt', 'is', null)
.where('pages.spaceId', 'in', spaceIds);
if (descendantIds) {
candidates = candidates.where('pages.id', 'in', descendantIds);
}
// Match predicate: title substring OR (unless titleOnly) text substring OR
// (unless titleOnly) FTS. The substring branch runs even when the tsquery is
// empty — that is the dotted/numeric-token case the FTS path misses.
//
// #443 dead-index fix: these two LIKE predicates MUST match the GIN trgm
// index expressions EXACTLY for Postgres to use them. The indexes are on the
// coalesce-FREE expressions `LOWER(f_unaccent(title))` (#348's
// idx_pages_title_trgm) and `LOWER(f_unaccent(text_content))` (this PR's
// idx_pages_text_content_trgm). A `coalesce(col,'')` wrapper here would make
// the query expression differ from the index expression and force a Seq Scan
// on pages for every lookup. Dropping coalesce is SEMANTICALLY EQUIVALENT:
// `NULL LIKE '%q%'` is NULL (falsy), so a NULL title/text simply doesn't
// match — exactly as an empty string wouldn't match `%q%`.
candidates = candidates.where((eb) => {
const ors = [
eb(
sql`LOWER(f_unaccent(pages.title))`,
'like',
sql`${likePattern} ESCAPE '\\'`,
),
];
if (!searchParams.titleOnly) {
ors.push(
eb(
sql`LOWER(f_unaccent(pages.text_content))`,
'like',
sql`${likePattern} ESCAPE '\\'`,
),
);
if (hasTsQuery) {
ors.push(
sql<boolean>`pages.tsv @@ to_tsquery('english', f_unaccent(${tsQuery}))` as any,
);
}
}
return eb.or(ors);
});
// Pull a generous candidate set (before permission filtering + limit).
// Cap it so a pathological match set cannot blow up memory; 200 >> limit
// (max 50) leaves ample headroom for the post-permission truncation.
//
// #443 cap-ordering fix: the 200-cap MUST be deterministic and relevance-
// biased. Without an ORDER BY, Postgres returns an ARBITRARY 200 rows, so on
// a broad match set (common word / short substring) a strong TITLE_EXACT hit
// could be among the dropped rows while 200 low-tier TEXT hits fill the cap.
// We order by the SAME SQL tier proxies the Node ranker uses — title-exact,
// then title-substring, then fts-rank (nulls last), then earliest text-match
// position — so the cap keeps the strongest candidates. The Node-side final
// tier sort + slice(0, limit) below still runs and stays authoritative; this
// ORDER BY only decides WHICH candidates survive the 200-cap.
// NB: a BARE integer literal in ORDER BY is read by Postgres as an ordinal
// column position (`ORDER BY 0` → "position 0 is not in select list"), so the
// no-tsquery fallback is `0::float`, not `0`.
const ftsRankExpr = hasTsQuery
? sql`ts_rank(pages.tsv, to_tsquery('english', f_unaccent(${tsQuery})))`
: sql`0::float`;
const candidatesCapped = candidates
// Raw-SQL ORDER BY expressions: pass the full `<expr> <dir>` as ONE arg
// (the two-arg form treats a raw-SQL second arg as an ORDER BY position).
.orderBy(
sql`(LOWER(f_unaccent(coalesce(pages.title, ''))) = ${needle}) desc`,
)
.orderBy(
sql`(LOWER(f_unaccent(coalesce(pages.title, ''))) LIKE ${likePattern} ESCAPE '\\') desc`,
)
.orderBy(sql`${ftsRankExpr} desc nulls last`)
// Earlier text match first; strpos returns 0 for "no match", which would
// sort BEFORE a real (>=1) position under plain ASC, so push 0 to the end.
.orderBy(
sql`case when strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) = 0 then 2147483647 else strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) end asc`,
);
let rows: any[] = await candidatesCapped.limit(200).execute();
if (rows.length === 0) {
return { items: [] };
}
// --- Permissions BEFORE limit. --------------------------------------------
// Apply the existing page-level post-filter to the MERGED set, then rank and
// only THEN truncate to `limit` — never lose the permission filter.
if (opts.userId) {
const accessibleIds =
await this.pagePermissionRepo.filterAccessiblePageIds({
pageIds: rows.map((r) => r.id),
userId: opts.userId,
spaceId: searchParams.spaceId,
workspaceId: opts.workspaceId,
});
const accessibleSet = new Set(accessibleIds);
rows = rows.filter((r) => accessibleSet.has(r.id));
}
if (rows.length === 0) {
return { items: [] };
}
// --- Tiered ranking + dedup. ----------------------------------------------
// Rows are already unique by id (single pages scan), so no cross-branch
// dedup is needed here; the tier captures the strongest match reason.
const ranked = rows.map((r) => {
let tier: SearchLookupTier;
let secondary: number;
if (r.titleExact) {
tier = SearchLookupTier.TITLE_EXACT;
secondary = Number(r.ftsRank) || 0;
} else if (r.titleSub) {
tier = SearchLookupTier.TITLE_SUBSTRING;
secondary = Number(r.ftsRank) || 0;
} else {
tier = SearchLookupTier.TEXT;
// Prefer earlier text matches; map position → closeness in (0, 1].
const pos = Number(r.textMatchPos) || 0;
secondary =
pos > 0 ? 1 / (1 + (pos - 1) / 100) : Number(r.ftsRank) || 0;
}
return { row: r, tier, score: computeLookupScore({ tier, secondary }) };
});
ranked.sort((a, b) => b.score - a.score);
const top = ranked.slice(0, limit);
// --- Batch ancestor path (ONE recursive CTE, not N+1). --------------------
const pathById = await this.buildAncestorPaths(top.map((t) => t.row.id));
const items: SearchLookupResponseDto[] = top.map((t) => ({
id: t.row.id,
slugId: t.row.slugId,
title: t.row.title,
parentPageId: t.row.parentPageId ?? null,
path: pathById.get(t.row.id) ?? [],
snippet: (t.row.snippet ?? '')
.replace(/\r\n|\r|\n/g, ' ')
.replace(/\s+/g, ' ')
.trim(),
score: t.score,
}));
return { items };
}
/**
* Batch ancestor-titles helper (#443): ONE recursive CTE seeded with ALL hit
* ids, walking UP parentPageId. Returns a map hitId ancestor titles ordered
* root direct parent (the hit's own title is excluded). Root pages map to
* an empty array. Avoids the N+1 of a per-page breadcrumb call.
*/
private async buildAncestorPaths(
hitIds: string[],
): Promise<Map<string, string[]>> {
const result = new Map<string, string[]>();
if (hitIds.length === 0) return result;
// ancestry(hit_id, page_id, title, parent_page_id, depth): seed one row per
// hit at depth 0 (the hit itself), then walk to parents (increasing depth).
const rows = await this.db
.withRecursive('ancestry', (db) =>
db
.selectFrom('pages')
.select([
'pages.id as hitId',
'pages.id as pageId',
'pages.title as title',
'pages.parentPageId as parentPageId',
sql<number>`0`.as('depth'),
])
.where('pages.id', 'in', hitIds)
.unionAll((exp) =>
exp
.selectFrom('pages as p')
.innerJoin('ancestry as a', 'p.id', 'a.parentPageId')
.select([
'a.hitId as hitId',
'p.id as pageId',
'p.title as title',
'p.parentPageId as parentPageId',
sql<number>`a.depth + 1`.as('depth'),
]),
),
)
.selectFrom('ancestry')
.select(['hitId', 'title', 'depth'])
// depth 0 is the hit itself — excluded from the path.
.where('depth', '>', 0)
.orderBy('hitId')
// Larger depth = closer to the space root. Ordering DESC gives
// root → parent once collected.
.orderBy('depth', 'desc')
.execute();
for (const r of rows as any[]) {
const list = result.get(r.hitId) ?? [];
list.push(r.title);
result.set(r.hitId, list);
}
return result;
}
async searchSuggestions(
suggestion: SearchSuggestionDTO,
userId: string,
@@ -0,0 +1,55 @@
import { type Kysely, sql } from 'kysely';
/**
* #443 trigram indexes for the opt-in agent-lookup search mode.
*
* The lookup mode adds a substring branch that runs leading-wildcard
* `LOWER(f_unaccent(col)) LIKE '%q%'` predicates on pages.title and
* pages.text_content. A leading wildcard cannot use a b-tree index, so without a
* GIN trigram index each such predicate is a sequential scan.
*
* - TITLE: the lookup-mode title predicate is `LOWER(f_unaccent(title)) LIKE
* '%q%'` (coalesce-free, so it can use a functional index), which is IDENTICAL
* to the one added for /search/suggest (#348). #348's perf-indexes migration
* already created `idx_pages_title_trgm` on `(LOWER(f_unaccent(title)))
* gin_trgm_ops`, so the title predicate is already covered — we do NOT
* re-create that index here (it would be redundant).
*
* - TEXT_CONTENT: NEW. The substring branch scans text_content when the query
* is not titleOnly. text_content is the large column, so a GIN trigram index
* on it is the meaningful acceleration for the lookup mode. The lookup search
* is ALWAYS space-scoped (spaceId or the user's member spaces), so on small
* instances a per-space sequential scan is tolerable but the index turns the
* `%q%` text predicate into a Bitmap Index Scan and removes the only
* unbounded-per-space cost of the feature. We add it. The trade-off is disk +
* write amplification on page edits (GIN trigram indexes are larger and slower
* to update than b-trees); on the small instances this fork targets that cost
* is acceptable and the read win on agent lookups is the priority.
*
* DEPLOY-TIME LOCK WARNING: plain (non-CONCURRENT) CREATE INDEX Kysely runs
* each migration in a transaction, so CONCURRENTLY is impossible. The build takes
* a SHARE lock that BLOCKS writes on `pages` for its duration. The text_content
* GIN build is the slow one and can take minutes on a large tenant. For large
* installations, run this in a maintenance window or build the index out-of-band
* with CREATE INDEX CONCURRENTLY before deploying (then `IF NOT EXISTS` no-ops
* here). Small/typical tenants are unaffected.
*/
export async function up(db: Kysely<any>): Promise<void> {
// The title predicate is served by #348's idx_pages_title_trgm — see header.
// Only the text_content index is introduced here.
// text_content trigram index. Its expression is coalesce-free —
// `LOWER(f_unaccent(text_content))` — to EXACTLY match the coalesce-free
// lookup-mode text substring predicate in search.service.ts, so Postgres can
// use it (a `coalesce(...)` mismatch would silently fall back to a Seq Scan).
await sql`
CREATE INDEX IF NOT EXISTS idx_pages_text_content_trgm
ON pages USING gin ((LOWER(f_unaccent(text_content))) gin_trgm_ops)
`.execute(db);
}
export async function down(db: Kysely<any>): Promise<void> {
// Only drop the index this migration introduced. idx_pages_title_trgm is owned
// by the #348 perf-indexes migration, so leave it for that migration's down().
await sql`DROP INDEX IF EXISTS idx_pages_text_content_trgm`.execute(db);
}
@@ -158,4 +158,27 @@ describe('EnvironmentService', () => {
).toBe('https://app.example.com');
});
});
describe('isAiChatFinalStepLockdownEnabled (#444)', () => {
const build = (val?: string) =>
new EnvironmentService({
get: (key: string, def?: string) =>
key === 'AI_CHAT_FINAL_STEP_LOCKDOWN' ? (val ?? def) : def,
} as any);
it('defaults to OFF (false) when unset — the new anti-degeneration default', () => {
expect(build(undefined).isAiChatFinalStepLockdownEnabled()).toBe(false);
});
it('is true only for the exact opt-in "true" (case-insensitive)', () => {
expect(build('true').isAiChatFinalStepLockdownEnabled()).toBe(true);
expect(build('TRUE').isAiChatFinalStepLockdownEnabled()).toBe(true);
});
it('stays OFF for any other value', () => {
expect(build('false').isAiChatFinalStepLockdownEnabled()).toBe(false);
expect(build('1').isAiChatFinalStepLockdownEnabled()).toBe(false);
expect(build('yes').isAiChatFinalStepLockdownEnabled()).toBe(false);
});
});
});
@@ -292,6 +292,24 @@ export class EnvironmentService {
return enabled === 'true';
}
/**
* Final-step lockdown for the in-app agent loop (#444). When ON (legacy), the
* LAST allowed step forces a text-only answer: tools are stripped
* (toolChoice:'none') and a synthesis instruction is appended. Defaults to OFF:
* stripping the tools mid-work triggered a token-loop degeneration incident
* (the model, robbed of its tools on the final step, emitted a 255KB block
* repeating a single token). With the toggle OFF the last step keeps its tools
* and gets only a SOFT nudge to finish with a text summary; the universal
* anti-babble guard is the token-degeneration detector instead. Enable this
* only for a model that does NOT reliably end its turns with a text answer.
*/
isAiChatFinalStepLockdownEnabled(): boolean {
const enabled = this.configService
.get<string>('AI_CHAT_FINAL_STEP_LOCKDOWN', 'false')
.toLowerCase();
return enabled === 'true';
}
/**
* Resumable SSE transport for durable agent runs (#184 phase 1.5). When
* enabled, a run tees its SSE frames into the in-memory run-stream registry so
@@ -143,7 +143,7 @@ export type DocmostMcpConfig = (
buf: Buffer,
mime: string,
) => { uri: string; sha256: string; size: number };
// Optional live/evict probes the package uses to keep stash_page's mirror
// Optional live/evict probes the package uses to keep stashPage's mirror
// counts honest under the store's FIFO eviction (mirror of the package's
// sink type); older bindings omit them.
has?: (uri: string) => boolean;
@@ -332,7 +332,7 @@ export class McpService implements OnModuleDestroy {
// Should never happen: handle() always stashes before delegating.
throw new UnauthorizedException('MCP authentication missing.');
}
// Inject the blob-sandbox sink after the auth decision so stash_page
// Inject the blob-sandbox sink after the auth decision so stashPage
// can store blobs in the shared in-RAM store regardless of which
// credential variant resolved. The sink (put/has/evict + uri↔id
// mapping) is owned by SandboxStore.asSink().
@@ -1,3 +1,8 @@
import * as fs from 'node:fs';
import * as os from 'node:os';
import { join } from 'node:path';
import Fastify, { FastifyInstance } from 'fastify';
import fastifyStatic from '@fastify/static';
import { resolveStaticAssetHeaders } from './static.module';
// Unit tests for the static-asset cache classifier extracted from the
@@ -33,3 +38,69 @@ describe('resolveStaticAssetHeaders', () => {
expect(headers['vary']).toBe('Accept-Encoding');
});
});
// Integration test proving the ACTUAL response header emitted by @fastify/static
// with the exact registration options StaticModule uses. This is the regression
// guard for #452: without `cacheControl: false`, @fastify/static writes its own
// `Cache-Control: public, max-age=0` AFTER the setHeaders callback, overwriting
// the immutable header — the /assets/ assertion below would then fail.
describe('static.module @fastify/static registration (integration)', () => {
let app: FastifyInstance;
let tmpDir: string;
beforeAll(async () => {
tmpDir = fs.mkdtempSync(join(os.tmpdir(), 'static-module-spec-'));
fs.mkdirSync(join(tmpDir, 'assets'), { recursive: true });
fs.mkdirSync(join(tmpDir, 'locales'), { recursive: true });
fs.writeFileSync(
join(tmpDir, 'assets', 'index-a1b2c3.js'),
'console.log(1);',
);
fs.writeFileSync(join(tmpDir, 'locales', 'en.json'), '{"hello":"world"}');
app = Fastify();
// Mirror StaticModule.onModuleInit's registration options exactly.
await app.register(fastifyStatic, {
root: tmpDir,
wildcard: false,
preCompressed: true,
cacheControl: false,
setHeaders: (res, filePath) => {
for (const [name, value] of Object.entries(
resolveStaticAssetHeaders(filePath),
)) {
res.setHeader(name, value);
}
},
});
await app.ready();
});
afterAll(async () => {
await app.close();
fs.rmSync(tmpDir, { recursive: true, force: true });
});
it('serves a hashed /assets/ file with an immutable, 1-year cache-control', async () => {
const res = await app.inject({
method: 'GET',
url: '/assets/index-a1b2c3.js',
});
expect(res.statusCode).toBe(200);
const cacheControl = res.headers['cache-control'];
expect(cacheControl).toContain('immutable');
expect(cacheControl).toContain('max-age=31536000');
});
it('serves a non-hashed /locales/ file WITHOUT an immutable cache-control', async () => {
const res = await app.inject({ method: 'GET', url: '/locales/en.json' });
expect(res.statusCode).toBe(200);
// resolveStaticAssetHeaders sets no cache-control here and cacheControl:false
// stops @fastify/static from adding one, so the browser revalidates by
// etag/last-modified — either an absent header or one without `immutable`.
const cacheControl = res.headers['cache-control'];
if (cacheControl !== undefined) {
expect(cacheControl).not.toContain('immutable');
}
});
});
@@ -115,6 +115,11 @@ export class StaticModule implements OnModuleInit {
// Serve the build-time .br/.gz neighbour when the client accepts it
// (see vite-plugin-compression2 in apps/client/vite.config.ts).
preCompressed: true,
// @fastify/static's default cacheControl:true writes its own
// Cache-Control (from maxAge, default 0) AFTER the setHeaders callback,
// silently overwriting the immutable header that resolveStaticAssetHeaders
// sets — disable it so setHeaders/resolveStaticAssetHeaders own the header.
cacheControl: false,
setHeaders: (res, filePath) => {
for (const [name, value] of Object.entries(
resolveStaticAssetHeaders(filePath),
@@ -182,6 +182,7 @@ describe('AiChatService run-stream attach [integration]', () => {
{
isAiChatDeferredToolsEnabled: () => false,
isAiChatResumableStreamEnabled: () => true,
isAiChatFinalStepLockdownEnabled: () => false,
} as any,
registry,
);
@@ -499,6 +500,7 @@ describe('AiChatService run-stream attach [integration]', () => {
{
isAiChatDeferredToolsEnabled: () => false,
isAiChatResumableStreamEnabled: () => true,
isAiChatFinalStepLockdownEnabled: () => false,
} as any,
registry,
);
@@ -150,7 +150,7 @@ describe('AiChatService.stream [integration]', () => {
{} as any, // pageAccess (idem)
// environment (#332): keep deferred tool loading OFF for this lifecycle
// harness so the toolset/behavior is exactly as before.
{ isAiChatDeferredToolsEnabled: () => false } as any,
{ isAiChatDeferredToolsEnabled: () => false, isAiChatFinalStepLockdownEnabled: () => false } as any,
);
}
@@ -378,7 +378,7 @@ describe('AiChatService.stream [integration]', () => {
{} as any,
{} as any,
// #332: deferred tool loading ON — the property under test.
{ isAiChatDeferredToolsEnabled: () => true } as any,
{ isAiChatDeferredToolsEnabled: () => true, isAiChatFinalStepLockdownEnabled: () => false } as any,
);
}
@@ -0,0 +1,123 @@
import { randomUUID } from 'node:crypto';
import { Kysely, sql } from 'kysely';
import {
getTestDb,
destroyTestDb,
createWorkspace,
createSpace,
} from './db';
/**
* #443 dead-index guard EXPLAIN on the REAL DB.
*
* The lookup mode's substring predicates run a leading-wildcard
* `LOWER(f_unaccent(col)) LIKE '%q%'`. Those are only fast when Postgres uses
* the GIN trigram indexes:
* - idx_pages_title_trgm on (LOWER(f_unaccent(title))) [#348]
* - idx_pages_text_content_trgm on (LOWER(f_unaccent(text_content))) [#443]
*
* Postgres uses a functional index ONLY when the query expression matches the
* index expression EXACTLY. The original lookup query wrapped the columns in
* `coalesce(col,'')`, which differs from the coalesce-FREE index expression and
* silently forced a Seq Scan on pages for EVERY lookup (the MCP client always
* sends substring:true). This test locks that in.
*
* Discriminator: `SET enable_seqscan = off` asks the planner "CAN this predicate
* use the index at all?" which is exactly what the coalesce bug breaks. With
* seqscan disabled:
* - the coalesce-FREE (fixed) predicate plans a Bitmap Index Scan on the trgm
* index (no Seq Scan on pages);
* - the coalesce-WRAPPED (buggy) predicate cannot use the index and falls back
* to a Seq Scan on pages even though seqscan is disabled.
* We assert both to prove the fix and to keep the regression from silently
* returning.
*/
describe('SearchService agent-lookup EXPLAIN — trgm index is live [integration]', () => {
let db: Kysely<any>;
let workspaceId: string;
let spaceId: string;
async function insertPage(title: string, textContent: string): Promise<void> {
const id = randomUUID();
await db
.insertInto('pages')
.values({
id,
slugId: `slug-${id.slice(0, 12)}`,
title,
textContent,
spaceId,
workspaceId,
})
.execute();
}
// Run EXPLAIN (no ANALYZE — we only inspect the chosen plan) and return the
// concatenated plan text.
async function explain(query: string): Promise<string> {
const rows = await sql<{ 'QUERY PLAN': string }>`EXPLAIN ${sql.raw(query)}`.execute(
db,
);
return (rows.rows as any[]).map((r) => r['QUERY PLAN']).join('\n');
}
beforeAll(async () => {
db = getTestDb();
workspaceId = (await createWorkspace(db)).id;
spaceId = (await createSpace(db, workspaceId)).id;
// Seed enough rows that a trigram index is a plausible plan. The content is
// varied so the '%needle%' pattern is selective.
for (let i = 0; i < 200; i++) {
await insertPage(
`seed-title-${i}`,
`seed body content number ${i} lorem ipsum dolor sit amet ${i}`,
);
}
await insertPage('backup-srv.local', 'the needle-token-xyz lives here');
// Keep the trgm indexes' stats fresh so the planner costs them correctly.
await sql`ANALYZE pages`.execute(db);
});
afterAll(async () => {
await destroyTestDb();
});
// Force the planner to answer "can the index be used?" rather than "is it
// cheaper than a seq scan on this size?". Restored after each test.
beforeEach(async () => {
await sql`SET enable_seqscan = off`.execute(db);
});
afterEach(async () => {
await sql`RESET enable_seqscan`.execute(db);
});
it('title predicate (coalesce-FREE, as fixed) uses idx_pages_title_trgm, not a Seq Scan', async () => {
const plan = await explain(
`SELECT id FROM pages WHERE LOWER(f_unaccent(title)) LIKE '%srv.local%'`,
);
expect(plan).toContain('idx_pages_title_trgm');
expect(plan).not.toMatch(/Seq Scan on pages/i);
});
it('text_content predicate (coalesce-FREE, as fixed) uses idx_pages_text_content_trgm, not a Seq Scan', async () => {
const plan = await explain(
`SELECT id FROM pages WHERE LOWER(f_unaccent(text_content)) LIKE '%needle-token%'`,
);
expect(plan).toContain('idx_pages_text_content_trgm');
expect(plan).not.toMatch(/Seq Scan on pages/i);
});
// Negative control: the OLD coalesce-wrapped predicate must NOT be able to use
// the index — even with seqscan disabled it can only Seq Scan pages. If this
// ever stops seq-scanning, the coalesce/index expressions have re-aligned and
// the guard above is no longer meaningful.
it('coalesce-WRAPPED text predicate (the bug) cannot use the index — falls to Seq Scan', async () => {
const plan = await explain(
`SELECT id FROM pages WHERE LOWER(f_unaccent(coalesce(text_content,''))) LIKE '%needle-token%'`,
);
expect(plan).not.toContain('idx_pages_text_content_trgm');
expect(plan).toMatch(/Seq Scan on pages/i);
});
});
@@ -0,0 +1,462 @@
import { randomUUID } from 'node:crypto';
import { Kysely } from 'kysely';
import { SearchService } from 'src/core/search/search.service';
import { PageRepo } from '@docmost/db/repos/page/page.repo';
import {
getTestDb,
destroyTestDb,
createWorkspace,
createSpace,
} from './db';
/**
* #443 agent-lookup search mode, acceptance on the REAL DB schema.
*
* Exercises SearchService.searchPage(..., { substring: true }) against a
* migrated Postgres: substring matching of technical tokens the FTS tokenizer
* mangles (backup-srv.local, 10.0.12.5, WB-MGE-30D86B, "Теги: Docker"), the
* populated path + snippet, parentPageId subtree scoping, titleOnly, the empty
* result, LIKE-metacharacter escaping (`%`/`_` must NOT match everything), the
* permission post-filter applied BEFORE the limit, and the web-UI path staying
* on the legacy FTS shape when `substring` is absent.
*
* The tsv column is populated by the pages_tsvector_trigger on insert, so the
* FTS branch is exercised too.
*/
describe('SearchService agent-lookup mode [integration]', () => {
let db: Kysely<any>;
let service: SearchService;
let workspaceId: string;
let spaceId: string;
// Direct page insert (the shared createPage seeder omits text_content /
// parent_page_id, both of which this mode depends on). Returns the id.
async function insertPage(args: {
title: string;
textContent?: string;
parentPageId?: string | null;
spaceId?: string;
}): Promise<string> {
const id = randomUUID();
await db
.insertInto('pages')
.values({
id,
slugId: `slug-${id.slice(0, 12)}`,
title: args.title,
textContent: args.textContent ?? null,
parentPageId: args.parentPageId ?? null,
spaceId: args.spaceId ?? spaceId,
workspaceId,
})
.execute();
return id;
}
// Build a SearchService wired to the real DB + a real PageRepo (only its
// recursive-descendants method is used by this mode, and it needs only `db`),
// with lightweight stubs for the space-membership and permission repos so a
// test can drive scope + the permission post-filter explicitly.
function buildService(opts?: {
userSpaceIds?: string[];
// ids to KEEP after the permission post-filter; undefined = keep all.
accessibleIds?: string[];
}): SearchService {
const pageRepo = new PageRepo(db as any, null as any, null as any);
const spaceMemberRepo = {
getUserSpaceIds: async () => opts?.userSpaceIds ?? [spaceId],
};
const pagePermissionRepo = {
filterAccessiblePageIds: async ({ pageIds }: { pageIds: string[] }) =>
opts?.accessibleIds
? pageIds.filter((id) => opts.accessibleIds!.includes(id))
: pageIds,
};
return new SearchService(
db as any,
pageRepo as any,
{} as any, // shareRepo — unused by the lookup path
spaceMemberRepo as any,
pagePermissionRepo as any,
);
}
beforeAll(async () => {
db = getTestDb();
workspaceId = (await createWorkspace(db)).id;
spaceId = (await createSpace(db, workspaceId)).id;
service = buildService();
});
afterAll(async () => {
await destroyTestDb();
});
it('finds `backup-srv.local` by the fragment `srv.local`', async () => {
const pageId = await insertPage({
title: 'backup-srv.local',
textContent: 'A backup server node.',
});
const { items } = (await service.searchPage(
{ query: 'srv.local', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
expect(items.map((i: any) => i.id)).toContain(pageId);
const hit = items.find((i: any) => i.id === pageId);
expect(hit.title).toBe('backup-srv.local');
// slugId must never be part of the server response shape.
expect('slugId' in hit).toBe(true); // server carries it; MCP strips it
});
it('finds a page whose TEXT contains `10.0.12.5` by the fragment `10.0.12` (empty-tsquery case)', async () => {
const pageId = await insertPage({
title: 'Server inventory',
textContent: 'The backup box lives at IP: 10.0.12.5. Debian 12, backups.',
});
const { items } = (await service.searchPage(
{ query: '10.0.12', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// The windowed snippet must include the matched text.
expect(hit.snippet).toContain('10.0.12.5');
});
it('finds `WB-MGE-30D86B` (alphanumeric token with dashes) by title', async () => {
const pageId = await insertPage({
title: 'WB-MGE-30D86B',
textContent: 'Device page.',
});
const { items } = (await service.searchPage(
{ query: 'WB-MGE-30D86B', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// Exact title match → top tier (TITLE_EXACT=3) → score in [0.75, 1].
expect(hit.score).toBeGreaterThanOrEqual(0.75);
// And it is the top-ranked hit of its own result set.
expect(items[0].id).toBe(pageId);
});
it('finds every page whose text literally contains `Теги: Docker`', async () => {
const a = await insertPage({
title: 'Container host A',
textContent: 'Some notes.\nТеги: Docker, compose\nmore.',
});
const b = await insertPage({
title: 'Container host B',
textContent: 'Prelude.\nТеги: Docker\nepilogue.',
});
const noise = await insertPage({
title: 'Unrelated',
textContent: 'Теги: Kubernetes',
});
const { items } = (await service.searchPage(
{ query: 'Теги: Docker', spaceId, substring: true, limit: 50 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(a);
expect(ids).toContain(b);
expect(ids).not.toContain(noise);
});
it('populates a non-empty `path` for a nested hit and `[]` for a root hit', async () => {
const root = await insertPage({ title: 'Infrastructure' });
const mid = await insertPage({ title: 'Datacenter A', parentPageId: root });
const leaf = await insertPage({
title: 'unique-nested-host',
parentPageId: mid,
});
const { items } = (await service.searchPage(
{ query: 'unique-nested-host', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === leaf);
expect(hit.path).toEqual(['Infrastructure', 'Datacenter A']);
const rootHits = (await service.searchPage(
{ query: 'Infrastructure', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const rootHit = rootHits.items.find((i: any) => i.id === root);
expect(rootHit.path).toEqual([]);
});
it('scopes to a subtree with parentPageId (cutting off sibling branches)', async () => {
const branchA = await insertPage({ title: 'BranchA-root' });
const inA = await insertPage({
title: 'scoped-target-xyz',
parentPageId: branchA,
});
const branchB = await insertPage({ title: 'BranchB-root' });
const inB = await insertPage({
title: 'scoped-target-xyz',
parentPageId: branchB,
});
const { items } = (await service.searchPage(
{
query: 'scoped-target-xyz',
spaceId,
substring: true,
parentPageId: branchA,
} as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(inA);
expect(ids).not.toContain(inB);
});
it('includes the parent page itself in the parentPageId subtree', async () => {
const parent = await insertPage({ title: 'self-included-parent' });
await insertPage({ title: 'child-of-self', parentPageId: parent });
const { items } = (await service.searchPage(
{
query: 'self-included-parent',
spaceId,
substring: true,
parentPageId: parent,
} as any,
{ workspaceId },
)) as any;
expect(items.map((i: any) => i.id)).toContain(parent);
});
it('titleOnly does NOT match on text_content', async () => {
const pageId = await insertPage({
title: 'Plain title',
textContent: 'body mentions the-secret-token here',
});
const withText = (await service.searchPage(
{ query: 'the-secret-token', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
expect(withText.items.map((i: any) => i.id)).toContain(pageId);
const titleOnly = (await service.searchPage(
{
query: 'the-secret-token',
spaceId,
substring: true,
titleOnly: true,
} as any,
{ workspaceId },
)) as any;
expect(titleOnly.items.map((i: any) => i.id)).not.toContain(pageId);
});
// #443 Fix #1 regression: f_unaccent is NOT length-preserving, so an
// expanding char (ß→ss, …→...) BEFORE the match shifted the strpos position
// relative to the ORIGINAL text and the snippet slice ran past end → empty.
// The position and the slice now share the LOWER(f_unaccent(...)) space, so
// the window is aligned and always contains the matched (unaccented) token.
it('returns a populated snippet when an unaccent-EXPANDING char precedes the match', async () => {
// 300 × `ß` (each f_unaccent-expands to `ss`) before the needle. Under the
// old code strpos returned a position ~593 in the expanded space but the
// slice ran over the ORIGINAL (~360 char) text → empty snippet, match lost.
const prefix = 'ß'.repeat(300);
const pageId = await insertPage({
title: 'Expanding-unaccent page',
textContent: `${prefix} needle-token-xyz trailing.`,
});
const { items } = (await service.searchPage(
{ query: 'needle-token-xyz', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// Snippet must be non-empty AND contain the matched token (unaccented form).
expect(hit.snippet.length).toBeGreaterThan(0);
expect(hit.snippet).toContain('needle-token-xyz');
});
// #443 Fix #2 regression: >200 matching pages for a broad substring, with
// exactly ONE exact-title hit. Without an ORDER BY on the 200-cap the exact
// hit could be among the arbitrarily-dropped rows; the ORDER BY keeps the
// strongest candidates so it must survive the cap and rank at the top.
it('keeps an exact-title hit through the 200-cap on a >200-row match set', async () => {
const isoSpace = (await createSpace(db, workspaceId)).id;
const svc = buildService({ userSpaceIds: [isoSpace] });
// 250 low-tier TEXT hits: the shared substring `capword` appears only in the
// body, never the title, so each is a TEXT-tier match (weakest tier).
for (let i = 0; i < 250; i++) {
await insertPage({
title: `filler-page-${i}`,
textContent: `body contains capword here #${i}`,
spaceId: isoSpace,
});
}
// Exactly one EXACT-title hit for the same query token.
const exact = await insertPage({
title: 'capword',
textContent: 'unrelated body text',
spaceId: isoSpace,
});
const { items } = (await svc.searchPage(
{ query: 'capword', spaceId: isoSpace, substring: true, limit: 10 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
// The exact-title hit must survive the 200-cap and appear in the top `limit`.
expect(ids).toContain(exact);
// And, being TITLE_EXACT, it must be the single strongest hit.
expect(items[0].id).toBe(exact);
});
// #443 Fix #3: titleOnly matches only the title, so it must not leak the page
// body as the snippet (the old "first 300 chars of text_content" fallback).
it('titleOnly does NOT return a text-body snippet', async () => {
const pageId = await insertPage({
title: 'titleonly-snippet-page',
textContent: 'SECRET-BODY-CONTENT-NOT-IN-TITLE that must not leak.',
});
const { items } = (await service.searchPage(
{
query: 'titleonly-snippet-page',
spaceId,
substring: true,
titleOnly: true,
} as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// The body text must not appear in the snippet; titleOnly → empty snippet.
expect(hit.snippet).not.toContain('SECRET-BODY-CONTENT-NOT-IN-TITLE');
expect(hit.snippet).toBe('');
});
it('returns [] (not an error) for a query that matches nothing', async () => {
const { items } = (await service.searchPage(
{
query: 'zzz-no-such-string-anywhere-42',
spaceId,
substring: true,
} as any,
{ workspaceId },
)) as any;
expect(items).toEqual([]);
});
it('a `%` query does NOT match everything (LIKE metacharacter escaped)', async () => {
// Fresh space so we can assert on total counts without cross-test noise.
const isoSpace = (await createSpace(db, workspaceId)).id;
const svc = buildService({ userSpaceIds: [isoSpace] });
await insertPage({ title: 'alpha', spaceId: isoSpace });
await insertPage({ title: 'beta', spaceId: isoSpace });
const literal = await insertPage({
title: '100%-coverage',
spaceId: isoSpace,
});
const { items } = (await svc.searchPage(
{ query: '%', spaceId: isoSpace, substring: true, limit: 50 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
// `%` is a literal → matches only the page that actually contains '%'.
expect(ids).toContain(literal);
expect(ids).not.toContain(
items.find((i: any) => i.title === 'alpha')?.id,
);
expect(items.length).toBe(1);
});
it('an `_` query does NOT match everything (LIKE metacharacter escaped)', async () => {
const isoSpace = (await createSpace(db, workspaceId)).id;
const svc = buildService({ userSpaceIds: [isoSpace] });
await insertPage({ title: 'gamma', spaceId: isoSpace });
const literal = await insertPage({
title: 'snake_case_name',
spaceId: isoSpace,
});
const { items } = (await svc.searchPage(
{ query: '_', spaceId: isoSpace, substring: true, limit: 50 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(literal);
expect(items.length).toBe(1);
});
it('applies the permission post-filter to the MERGED set BEFORE the limit', async () => {
const isoSpace = (await createSpace(db, workspaceId)).id;
const keep = await insertPage({
title: 'perm-visible-target',
spaceId: isoSpace,
});
const hidden = await insertPage({
title: 'perm-hidden-target',
spaceId: isoSpace,
});
// Authenticated (userId set) so the permission filter runs; only `keep` is
// accessible. limit 1 must NOT be able to select `hidden`.
const svc = buildService({
userSpaceIds: [isoSpace],
accessibleIds: [keep],
});
const { items } = (await svc.searchPage(
{
query: 'perm-',
spaceId: isoSpace,
substring: true,
limit: 1,
} as any,
{ userId: 'user-1', workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(keep);
expect(ids).not.toContain(hidden);
});
it('web-UI path (no `substring` flag) keeps the legacy FTS response shape', async () => {
await insertPage({
title: 'legacy shape page',
textContent: 'searchable legacyword content',
});
const { items } = (await service.searchPage(
{ query: 'legacyword', spaceId } as any,
{ userId: 'user-1', workspaceId },
)) as any;
// Legacy hits carry rank + highlight + space, and NO path/snippet/score.
const hit = items[0];
expect(hit).toBeDefined();
expect('rank' in hit).toBe(true);
expect('highlight' in hit).toBe(true);
expect('path' in hit).toBe(false);
expect('snippet' in hit).toBe(false);
expect('score' in hit).toBe(false);
});
});
+120 -88
View File
@@ -12,7 +12,7 @@ license.
> better at *writing a small function that fixes the text* than at re-reading and
> re-emitting a whole document. So this server is built around the way a model actually
> wants to edit: address a block by id, run a find/replace, or hand it a
> `(doc, ctx) => doc` transform and let it *program* the change. `docmost_transform` is
> `(doc, ctx) => doc` transform and let it *program* the change. `docmostTransform` is
> that interface. Other Docmost MCPs are human-shaped — they expose "open the page" and
> "replace the page"; this one exposes the editing primitives a model is good at.
@@ -40,7 +40,7 @@ There are several Docmost MCPs. Here is a capability-by-capability comparison.
| **Enterprise license required** | **No** | **Yes** | No | No | No |
| Authentication | email + password, **auto re-auth** | API key | email + password | cookie `authToken` (copy from DevTools) | Docmost API / **direct PostgreSQL** |
| Read page as Markdown | ✅ | ✅ | ✅ | ✅ | ✅ (read-only) |
| **Lossless Markdown round-trip** (export / import, keeps comment anchors) | ✅ | — | — | — | — |
| **Markdown round-trip** (export / import, keeps comment anchors) | ✅ | — | — | — | — |
| Read **lossless ProseMirror JSON** (with block ids) | ✅ | — | — | — | — |
| **Compact page outline** (cheap block-id lookup) | ✅ | — | — | — | — |
| **Fetch a single block** (by id or index) | ✅ | — | — | — | — |
@@ -69,9 +69,9 @@ There are several Docmost MCPs. Here is a capability-by-capability comparison.
- **Token-efficient editing.** Most Docmost MCPs (and the official one) only offer
"replace the whole page" writes — the agent must download the entire document, mutate
it, and upload it back, paying for the full document **twice** on every tiny fix.
This server lets the agent change exactly one block (`patch_node` / `insert_node` /
`delete_node`), do a structure-preserving find/replace (`edit_page_text`), or copy a
whole page server-side (`copy_page_content`) — **without the document ever passing
This server lets the agent change exactly one block (`patchNode` / `insertNode` /
`deleteNode`), do a structure-preserving find/replace (`editPageText`), or copy a
whole page server-side (`copyPageContent`) — **without the document ever passing
through the model**.
- **Writes that don't fight the editor.** Naive REST writes race with whatever a human
@@ -85,12 +85,12 @@ There are several Docmost MCPs. Here is a capability-by-capability comparison.
- **Agent-native editing model.** Human-facing servers expose "open the page" and "replace
the page", because that mirrors how a person works. A model edits better by *programming*
the change — addressing blocks by id, running a find/replace, or supplying a
`(doc, ctx) => doc` transform (`docmost_transform`, with a dry-run diff before it
`(doc, ctx) => doc` transform (`docmostTransform`, with a dry-run diff before it
commits). This server is shaped around that, which is why it has editing primitives the
others simply don't.
- **An editing safety net the others lack.** `list_page_history``diff_page_versions`
`restore_page_version` give an agent (and you) a full view-and-undo loop. The diff
- **An editing safety net the others lack.** `listPageHistory``diffPageVersions`
`restorePageVersion` give an agent (and you) a full view-and-undo loop. The diff
uses the *same* `recreateTransform → ChangeSet → simplifyChanges` pipeline Docmost's
own history viewer uses, so what you see matches the product.
@@ -110,52 +110,58 @@ All 41 tools, grouped by what you'd reach for them.
### Exploration & retrieval
- **`get_workspace`** — Information about the current Docmost workspace.
- **`list_spaces`** — All spaces in the workspace.
- **`list_pages`** — Recent pages in a space, ordered by `updatedAt` desc (default 50,
- **`getWorkspace`** — Information about the current Docmost workspace.
- **`listSpaces`** — All spaces in the workspace.
- **`listPages`** — Recent pages in a space, ordered by `updatedAt` desc (default 50,
max 100). Use `search` for lookups in large spaces.
- **`search`** — Full-text search across pages and content (bounded by `limit`, max 100).
- **`get_page`** — A page's content as clean **Markdown** (convenient, but a *lossy*
view — block ids and exact table/callout structure are approximated).
- **`get_page_json`** — A page's **lossless ProseMirror/TipTap JSON**, including every
- **`getPage`** — A page's content as clean **Markdown** (canonical for text; drops only
block ids, resolved-comment anchors, and a fixed no-Markdown-representation attr set —
table spans/colwidth/background, indent, `callout.icon`, `orderedList.type`, and link
`internal`/`target`/`rel`/`class`; use `getPageJson` when you need those).
- **`getPageJson`** — A page's **lossless ProseMirror/TipTap JSON**, including every
block's `attrs.id` and the `slugId` used in URLs. This is what the per-block editing
tools consume.
- **`get_outline`** — A compact outline of a page's top-level blocks (`{index, type, id,
- **`getOutline`** — A compact outline of a page's top-level blocks (`{index, type, id,
level, firstText}`; tables add row/column counts and their header-cell texts, lists add
item counts) **without** the document body. The cheap way to locate a section or table
and grab its block id before
`get_node` / `patch_node` / `insert_node`.
- **`get_node`** — Fetch a single block's full ProseMirror subtree (lossless) without
pulling the whole page. Address it by a block id (from `get_outline` / `get_page_json`),
`getNode` / `patchNode` / `insertNode`.
- **`getNode`** — Fetch a single block's full ProseMirror subtree (lossless) without
pulling the whole page. Address it by a block id (from `getOutline` / `getPageJson`),
or by `#<index>` for a top-level block — use the `#<index>` form for tables/rows/cells,
which carry no id.
### Page lifecycle
- **`create_page`** — Create a page from Markdown and place it in the hierarchy (optional
- **`createPage`** — Create a page from Markdown and place it in the hierarchy (optional
`parentPageId`) in one call. Uses Docmost's import API for clean Markdown→ProseMirror.
- **`rename_page`** — Change a page's title only, without touching or resending content.
- **`move_page`** — Re-parent a page (nest it, or move to root); supports fractional-index
- **`renamePage`** — Change a page's title only, without touching or resending content.
- **`movePage`** — Re-parent a page (nest it, or move to root); supports fractional-index
positioning. Returns only on a *positively confirmed* success.
- **`delete_page`** — Delete a single page.
- **`copy_page_content`** — Replace one page's body with a copy of another's, **entirely
- **`deletePage`** — Delete a single page.
- **`copyPageContent`** — Replace one page's body with a copy of another's, **entirely
server-side** — the document never passes through the model. The target keeps its own
title and slug (so its URL is preserved).
### Editing
- **`edit_page_text`** — Surgical find/replace inside a page's text. Preserves **all**
- **`editPageText`** — Surgical find/replace inside a page's text. Preserves **all**
structure: block ids, marks, links, callouts, tables. The preferred tool for fixing
wording, typos, numbers and names.
- **`patch_node`** — Replace a single block addressed by its `attrs.id` (from
`get_page_json`), without resending the document.
- **`insert_node`** — Insert a block before/after another (by `attrs.id` or anchor text),
- **`patchNode`** — Replace a single block addressed by its `attrs.id` (from
`getPageJson`), without resending the document.
- **`insertNode`** — Insert a block before/after another (by `attrs.id` or anchor text),
or append at the end.
- **`delete_node`** — Remove a single block by its `attrs.id`.
- **`update_page_json`** — Replace a page's entire content with a ProseMirror document
- **`deleteNode`** — Remove a single block by its `attrs.id`.
- **`updatePageJson`** — Replace a page's entire content with a ProseMirror document
(bulk rewrites, or when nodes lack ids). `content` is optional — omit it to update only
the title. Keeps the block ids you pass in, so heading anchors and history stay stable.
- **`docmost_transform`** — The agent-native editing interface: instead of retyping a
- **`updatePageMarkdown`** — Replace a page's body (and optionally its title) with new
**plain Markdown**. The whole body is re-imported (block ids regenerate — for surgical or
id-preserving edits prefer `editPageText` / `patchNode` / `updatePageJson`).
Docmost-flavoured markdown is parsed, including `^[...]` inline footnotes.
- **`docmostTransform`** — The agent-native editing interface: instead of retyping a
document, the agent **writes a function that fixes it**. Edit a page by running an
arbitrary **`(doc, ctx) => doc` JavaScript transform** against its *live* ProseMirror
document. Runs **sandboxed**
@@ -168,42 +174,46 @@ All 41 tools, grouped by what you'd reach for them.
### Tables
- **`table_get`** — Read a table as a matrix: `{rows, cols, cells (text[][]), cellIds}`
- **`tableGet`** — Read a table as a matrix: `{rows, cols, cells (text[][]), cellIds}`
(a paragraph id per cell, or `null`). Address the table by `#<index>` (from
`get_outline`) or any block id inside it. Use `cellIds` with `patch_node` for
`getOutline`) or any block id inside it. Use `cellIds` with `patchNode` for
rich-formatted cell edits.
- **`table_insert_row`** — Insert a row of plain-text cells, padded to the table's column
- **`tableInsertRow`** — Insert a row of plain-text cells, padded to the table's column
count (passing more cells than columns is an error). `index` is the 0-based insert
position (0 inserts before the header); omit it to append at the end.
- **`table_delete_row`** — Delete the row at a 0-based `index`. Refuses to delete a table's
- **`tableDeleteRow`** — Delete the row at a 0-based `index`. Refuses to delete a table's
only row; deleting row 0 promotes the next row to header.
- **`table_update_cell`** — Set the plain-text content of cell `[row, col]` (0-based). For
rich formatting, `patch_node` the cell's paragraph id from `table_get`.
- **`tableUpdateCell`** — Set the plain-text content of cell `[row, col]` (0-based). For
rich formatting, `patchNode` the cell's paragraph id from `tableGet`.
### Markdown round-trip
- **`export_page_markdown`** — Export a page to a single self-contained, **lossless
Docmost-flavoured Markdown** file: a meta header, the body with inline comment anchors
and diagrams, and a trailing comments-thread block. Built for a download → edit body
`import_page_markdown` round-trip that preserves everything, including comment highlights.
- **`import_page_markdown`** — Replace a page's content from a Docmost-flavoured Markdown
file produced by `export_page_markdown`, restoring comment-highlight anchors and diagrams
from their inline HTML. (Comment *threads* in the file are not re-created on the server —
only the page body and inline comment marks are written; manage threads via the comment
tools/UI.)
- **`exportPageMarkdown`** — Export a page to a single self-contained
**Docmost-flavoured Markdown** file: a meta header, the body with inline comment anchors
and diagrams, and a trailing comments-thread block. The download → edit → import
round-trip regenerates block ids and **silently drops** the no-Markdown-representation
attr set (table merge spans/colwidth/background, indent, `callout.icon`,
`orderedList.type`, link `internal`/`target`/`rel`/`class`); keep those in ProseMirror
JSON if they must survive. To replace a page's body from plain authoring Markdown, use
`updatePageMarkdown`.
> **Removed in this release:** `importPageMarkdown` (the round-trip parser for an
> exported Docmost-Markdown file) is **no longer exposed on the external MCP surface**.
> To replace a page's body from Markdown, use **`updatePageMarkdown`** (plain Markdown
> body replace). See the CHANGELOG for the migration note.
### Images
- **`insert_image`** — Download an image from a web (http/https) URL and insert it in one
- **`insertImage`** — Download an image from a web (http/https) URL and insert it in one
step: append it, drop it in place of a text placeholder (`replaceText`), or put it after
a given block (`afterText`). Preserves all other block ids.
- **`replace_image`** — Swap an existing image for one fetched from a web (http/https) URL.
- **`replaceImage`** — Swap an existing image for one fetched from a web (http/https) URL.
Uploads the new file as a **fresh
attachment** (clean URL that renders and busts browser caches), then re-points every
node referencing the old attachment (recursively, including callouts/tables) via the
live document, preserving comments, alignment and alt text. (In-place overwrite is
deliberately avoided — some Docmost versions corrupt the attachment on overwrite.)
- **`stash_page`** — Serialize a whole page (its full ProseMirror JSON) into an ephemeral
- **`stashPage`** — Serialize a whole page (its full ProseMirror JSON) into an ephemeral
in-RAM blob and return ONLY a short anonymous URL — the body never enters the model
context, so it is the way to hand a large page (and its images) to an external consumer
without truncation. Every internal file/image attachment is mirrored into the same
@@ -214,35 +224,35 @@ All 41 tools, grouped by what you'd reach for them.
### Comments
- **`create_comment`** — Add a page comment, optionally **anchored inline** to an exact
- **`createComment`** — Add a page comment, optionally **anchored inline** to an exact
span of text (the first occurrence is wrapped in a comment mark).
- **`list_comments`** — List a page's comments (content returned as Markdown).
- **`update_comment`** — Edit an existing comment.
- **`delete_comment`** — Delete a comment.
- **`resolve_comment`** — Resolve (close) or reopen a comment thread (reversible). Only top-level
comments can be resolved; the thread and its replies are kept, unlike `delete_comment`.
- **`check_new_comments`** — Find comments created after a given ISO-8601 timestamp across
- **`listComments`** — List a page's comments (content returned as Markdown).
- **`updateComment`** — Edit an existing comment.
- **`deleteComment`** — Delete a comment.
- **`resolveComment`** — Resolve (close) or reopen a comment thread (reversible). Only top-level
comments can be resolved; the thread and its replies are kept, unlike `deleteComment`.
- **`checkNewComments`** — Find comments created after a given ISO-8601 timestamp across
a space, optionally scoped to a page subtree — ideal for an agent that watches a doc for
feedback.
### Versioning & history
- **`list_page_history`** — A page's saved versions (Docmost auto-snapshots on save),
- **`listPageHistory`** — A page's saved versions (Docmost auto-snapshots on save),
newest first, cursor-paginated. Each item's id is the `historyId`.
- **`diff_page_versions`** — Diff two versions (or a version against the live page).
- **`diffPageVersions`** — Diff two versions (or a version against the live page).
Returns inserted/deleted text, integrity counts (images, links, tables, callouts,
footnote markers), and a human-readable Markdown summary — computed with the same
pipeline Docmost's own history viewer uses.
- **`restore_page_version`** — Write a saved version back as the current content. Docmost
- **`restorePageVersion`** — Write a saved version back as the current content. Docmost
has no restore endpoint, so this creates a **new** snapshot — the restore is itself
revertible.
### Sharing
- **`share_page`** — Make a page publicly accessible (idempotent) and return its public
- **`sharePage`** — Make a page publicly accessible (idempotent) and return its public
URL (`<app>/share/<key>/p/<slugId>`); optional search-engine indexing.
- **`unshare_page`** — Revoke a page's public share.
- **`list_shares`** — All public shares in the workspace, with titles and public URLs.
- **`unsharePage`** — Revoke a page's public share.
- **`listShares`** — All public shares in the workspace, with titles and public URLs.
---
@@ -251,26 +261,27 @@ All 41 tools, grouped by what you'd reach for them.
This same guidance is also delivered at runtime via the MCP server `instructions` field,
so capable clients steer the model automatically.
- **Text fixes** (wording, typos, numbers): `edit_page_text`.
- **One block** (paragraph/heading/callout/table cell): `patch_node` / `insert_node` /
`delete_node`, addressing the node by its `attrs.id` from `get_page_json`.
- **Images**: `insert_image` / `replace_image`.
- **A new page**: `create_page`.
- **Bulk rewrite, or nodes without ids**: `update_page_json`.
- **Text fixes** (wording, typos, numbers): `editPageText`.
- **One block** (paragraph/heading/callout/table cell): `patchNode` / `insertNode` /
`deleteNode`, addressing the node by its `attrs.id` from `getPageJson`.
- **Images**: `insertImage` / `replaceImage`.
- **A new page**: `createPage`.
- **Bulk rewrite, or nodes without ids**: `updatePageJson` (ProseMirror) or
`updatePageMarkdown` (plain Markdown body replace).
- **Multi-step / scripted rewrite** (renumbering, footnotes, coordinated edits):
`docmost_transform` — preview with `dryRun`, then apply.
- **Copy a whole page's content from another page** (server-side): `copy_page_content`.
- **Rename a page** (title only): `rename_page`.
- **Reads**: `get_page` (Markdown) / `get_page_json` (lossless ProseMirror with ids).
- **Review changes**: `list_page_history``diff_page_versions``restore_page_version`.
- **Comments**: `create_comment` (with optional inline anchoring) / `list_comments` /
`update_comment` / `resolve_comment` / `delete_comment` / `check_new_comments`.
- **Navigate a page cheaply** (find a section/table, grab a block id): `get_outline`
`get_node`.
- **Tables** (add/remove a row, set a cell): `table_get` / `table_insert_row` /
`table_delete_row` / `table_update_cell`.
- **Round-trip a page as Markdown** (download, edit, re-upload losslessly with comments):
`export_page_markdown` / `import_page_markdown`.
`docmostTransform` — preview with `dryRun`, then apply.
- **Copy a whole page's content from another page** (server-side): `copyPageContent`.
- **Rename a page** (title only): `renamePage`.
- **Reads**: `getPage` (Markdown) / `getPageJson` (lossless ProseMirror with ids).
- **Review changes**: `listPageHistory``diffPageVersions``restorePageVersion`.
- **Comments**: `createComment` (with optional inline anchoring) / `listComments` /
`updateComment` / `resolveComment` / `deleteComment` / `checkNewComments`.
- **Navigate a page cheaply** (find a section/table, grab a block id): `getOutline`
`getNode`.
- **Tables** (add/remove a row, set a cell): `tableGet` / `tableInsertRow` /
`tableDeleteRow` / `tableUpdateCell`.
- **Export a page as self-contained Markdown** (with comment anchors): `exportPageMarkdown`.
- **Replace a page's body from Markdown**: `updatePageMarkdown`.
---
@@ -282,25 +293,46 @@ so capable clients steer the model automatically.
the debounced REST snapshot), then **reads → transforms → writes synchronously** in one
tick so no remote update can interleave, and **waits for persistence acknowledgement**
before returning.
- **Per-page write serialization.** A per-`pageId` async mutex ensures two MCP writes to
the same page never overlap; different pages never block each other.
- **Per-page write serialization.** A per-`pageId` async mutex (keyed by the resolved
page **UUID**, never a slugId) ensures two MCP writes to the same page never overlap;
different pages never block each other. The lock helper fails fast if it is ever handed
a non-UUID key, so a write path that forgot to resolve the id can never silently lock
under a split key.
**Deploy requirement — single instance or sticky sessions.** This mutex is an
in-process `Map`, and the cached collab sessions and the `stash_page` blob store are
RAM-only and process-local. Behind a **multi-replica** load balancer **without sticky
sessions**, two replicas can each "hold" the lock for the same page at once and per-page
serialization is silently lost. Run the MCP/app as a **single instance**, or pin each
page's traffic to one replica (sticky sessions / consistent hashing on the page id).
There is deliberately no cross-process (e.g. Postgres advisory) lock yet — a conscious
documented constraint. See the `Dockerfile` comment and the `MCP collaboration write
path` block in `.env.example`.
**Rights-staleness window.** A cached collab session writes under the token captured at
connect time (and the collab-token cache reuses a token for its TTL), so a **revoked**
page access can lag by up to `MCP_COLLAB_SESSION_MAX_AGE_MS` (the hard session lifetime,
default 10 min) before the next re-auth picks it up. Lower it to shorten the lag at the
cost of more reconnects. This bounded window is an accepted trade-off; there is no
push-based cache invalidation on a rights change.
- **Transparent re-authentication.** Login uses email/password; expired tokens are
refreshed automatically on the first 401/403 (covering JSON, multipart upload, and the
collaboration-token path), with in-flight login de-duplication so a burst of calls
triggers a single re-login.
- **Lossless and lossy reads.** `get_page_json` returns the exact ProseMirror tree with
block ids; `get_page` returns clean Markdown for convenience.
- **Precise reads.** `getPageJson` returns the exact ProseMirror tree with block ids;
`getPage` returns canonical Markdown that drops only a fixed, documented attr set.
- **Full Docmost schema.** Markdown↔ProseMirror conversion supports callouts (including
nested), task lists (bullet *and* numbered checklists), tables, math blocks, embeds,
highlights, sub/superscript and more, with defensive caps against pathological input.
- **Structured tables & lossless Markdown round-trip.** Tables can be edited as a matrix
- **Structured tables & Markdown round-trip.** Tables can be edited as a matrix
(read, insert/delete rows, set cells by `[row,col]`) without resending the document, and
a page can be exported to and re-imported from a self-contained Docmost-flavoured
Markdown file that preserves inline comment anchors and diagrams.
Markdown file that preserves inline comment anchors and diagrams (block ids regenerate
and a fixed no-Markdown-representation attr set is dropped — see `exportPageMarkdown`).
- **Token-optimized responses.** API responses are filtered down to the fields agents
actually need, and large collections (spaces, pages, comments, history) are paginated.
- **Hardened runtime.** Global handlers keep a stray socket error from tearing down the
stdio server; `move_page` requires a positively confirmed success; the diff engine
stdio server; `movePage` requires a positively confirmed success; the diff engine
falls back to a coarse block diff rather than hard-failing on a pathological document.
---
@@ -358,7 +390,7 @@ npm run test:e2e
This project began as a fork of [MrMartiniMo/docmost-mcp](https://github.com/MrMartiniMo/docmost-mcp)
(by Moritz Krause) and extends it substantially — adding per-block node editing,
surgical text edits, the sandboxed `docmost_transform`, version history / diff / restore,
surgical text edits, the sandboxed `docmostTransform`, version history / diff / restore,
comments, image insert/replace, public sharing, server-side page copy, dual
JSON/Markdown reads, transparent re-authentication and significant hardening. The comment
tools were ported from upstream PR #3 by Max Nikitin. Thanks to both.
+122 -90
View File
@@ -12,7 +12,7 @@
> небольшую функцию, которая чинит текст*, чем перечитывать и заново выдавать весь
> документ. Поэтому сервер построен вокруг того, как модели на самом деле удобно
> редактировать: адресовать блок по id, сделать find/replace или передать трансформ
> `(doc, ctx) => doc` и позволить модели *запрограммировать* правку. `docmost_transform`
> `(doc, ctx) => doc` и позволить модели *запрограммировать* правку. `docmostTransform`
> это и есть такой интерфейс. Другие Docmost-MCP «заточены под человека» — они дают
> «открыть страницу» и «заменить страницу»; этот даёт примитивы редактирования, в которых
> модель сильна.
@@ -43,7 +43,7 @@ Docmost-MCP не сочетают:
| **Нужна enterprise-лицензия** | **Нет** | **Да** | Нет | Нет | Нет |
| Аутентификация | email + пароль, **авто-переавторизация** | API-ключ | email + пароль | cookie `authToken` (копировать из DevTools) | API Docmost / **напрямую PostgreSQL** |
| Чтение страницы как Markdown | ✅ | ✅ | ✅ | ✅ | ✅ (только чтение) |
| **Lossless Markdown round-trip** (экспорт/импорт, сохраняет якоря комментариев) | ✅ | — | — | — | — |
| **Markdown round-trip** (экспорт/импорт, сохраняет якоря комментариев) | ✅ | — | — | — | — |
| Чтение **lossless ProseMirror JSON** (с id блоков) | ✅ | — | — | — | — |
| **Компактная структура страницы** (дешёвый поиск id блока) | ✅ | — | — | — | — |
| **Получение одного блока** (по id или индексу) | ✅ | — | — | — | — |
@@ -71,9 +71,9 @@ Docmost-MCP не сочетают:
- **Экономия токенов при редактировании.** Большинство Docmost-MCP (и официальный)
предлагают только запись «заменить всю страницу» — агент вынужден скачать весь документ,
изменить и загрузить обратно, оплачивая весь документ **дважды** на каждой мелкой
правке. Этот сервер позволяет агенту изменить ровно один блок (`patch_node` /
`insert_node` / `delete_node`), сделать find/replace с сохранением структуры
(`edit_page_text`) или скопировать страницу на стороне сервера (`copy_page_content`) —
правке. Этот сервер позволяет агенту изменить ровно один блок (`patchNode` /
`insertNode` / `deleteNode`), сделать find/replace с сохранением структуры
(`editPageText`) или скопировать страницу на стороне сервера (`copyPageContent`) —
**причём документ ни разу не проходит через модель**.
- **Записи, которые не воюют с редактором.** Наивная запись через REST конфликтует с тем,
@@ -87,12 +87,12 @@ Docmost-MCP не сочетают:
- **Агентоориентированная модель редактирования.** Серверы «под человека» дают «открыть
страницу» и «заменить страницу», потому что это отражает то, как работает человек. Модель
редактирует лучше, *программируя* правку — адресуя блоки по id, делая find/replace или
передавая трансформ `(doc, ctx) => doc` (`docmost_transform`, с dry-run диффом перед
передавая трансформ `(doc, ctx) => doc` (`docmostTransform`, с dry-run диффом перед
коммитом). Этот сервер построен вокруг этого — поэтому у него есть примитивы
редактирования, которых у остальных просто нет.
- **Страховка при редактировании, которой нет у других.** `list_page_history`
`diff_page_versions``restore_page_version` дают агенту (и вам) полный цикл «посмотреть
- **Страховка при редактировании, которой нет у других.** `listPageHistory`
`diffPageVersions``restorePageVersion` дают агенту (и вам) полный цикл «посмотреть
и откатить». Дифф использует *тот же* конвейер `recreateTransform → ChangeSet →
simplifyChanges`, что и встроенный просмотр истории Docmost, так что результат совпадает
с продуктом.
@@ -113,55 +113,62 @@ Docmost-MCP не сочетают:
### Чтение и поиск
- **`get_workspace`** — Информация о текущем воркспейсе Docmost.
- **`list_spaces`** — Все пространства воркспейса.
- **`list_pages`** — Недавние страницы пространства, по убыванию `updatedAt` (по умолчанию
- **`getWorkspace`** — Информация о текущем воркспейсе Docmost.
- **`listSpaces`** — Все пространства воркспейса.
- **`listPages`** — Недавние страницы пространства, по убыванию `updatedAt` (по умолчанию
50, максимум 100). Для поиска в больших пространствах используйте `search`.
- **`search`** — Полнотекстовый поиск по страницам и контенту (ограничен `limit`, максимум
100).
- **`get_page`** — Контент страницы как чистый **Markdown** (удобно, но это
*lossy*-представление — id блоков и точная структура таблиц/коллаутов аппроксимируются).
- **`get_page_json`** — **Lossless ProseMirror/TipTap JSON** страницы, включая `attrs.id`
- **`getPage`** — Контент страницы как чистый **Markdown** (канонично для текста; теряет
лишь id блоков, якоря разрешённых комментариев и фиксированный набор атрибутов без
markdown-представления — спаны/colwidth/фон ячеек таблиц, отступы (indent),
`callout.icon`, `orderedList.type` и `internal`/`target`/`rel`/`class` у ссылок;
используйте `getPageJson`, когда они нужны).
- **`getPageJson`** — **Lossless ProseMirror/TipTap JSON** страницы, включая `attrs.id`
каждого блока и `slugId`, используемый в URL. Именно его потребляют инструменты
поблочного редактирования.
- **`get_outline`** — Компактная структура страницы из блоков верхнего уровня (`{index,
- **`getOutline`** — Компактная структура страницы из блоков верхнего уровня (`{index,
type, id, level, firstText}`; для таблиц добавляются число строк/столбцов и тексты ячеек
заголовка, для списков — число пунктов) **без** тела документа. Дешёвый способ найти раздел или таблицу и получить
id блока перед `get_node` / `patch_node` / `insert_node`.
- **`get_node`** — Получить полное ProseMirror-поддерево одного блока (lossless), не
вытягивая всю страницу. Адресуйте его по id блока (из `get_outline` / `get_page_json`)
id блока перед `getNode` / `patchNode` / `insertNode`.
- **`getNode`** — Получить полное ProseMirror-поддерево одного блока (lossless), не
вытягивая всю страницу. Адресуйте его по id блока (из `getOutline` / `getPageJson`)
или формой `#<index>` для блока верхнего уровня — используйте `#<index>` для
таблиц/строк/ячеек, у которых нет id.
### Жизненный цикл страниц
- **`create_page`** — Создать страницу из Markdown и поместить в иерархию (опционально
- **`createPage`** — Создать страницу из Markdown и поместить в иерархию (опционально
`parentPageId`) одним вызовом. Использует import API Docmost для чистой конвертации
Markdown→ProseMirror.
- **`rename_page`** — Изменить только заголовок страницы, не трогая и не пересылая контент.
- **`move_page`** — Сменить родителя страницы (вложить или вынести в корень); поддерживает
- **`renamePage`** — Изменить только заголовок страницы, не трогая и не пересылая контент.
- **`movePage`** — Сменить родителя страницы (вложить или вынести в корень); поддерживает
позиционирование по fractional-index. Возвращает успех только при *положительно
подтверждённом* результате.
- **`delete_page`** — Удалить одну страницу.
- **`copy_page_content`** — Заменить тело одной страницы копией тела другой, **полностью на
- **`deletePage`** — Удалить одну страницу.
- **`copyPageContent`** — Заменить тело одной страницы копией тела другой, **полностью на
стороне сервера** — документ не проходит через модель. У целевой страницы сохраняются
собственные заголовок и slug (URL не меняется).
### Редактирование
- **`edit_page_text`** — Хирургический find/replace внутри текста страницы. Сохраняет
- **`editPageText`** — Хирургический find/replace внутри текста страницы. Сохраняет
**всю** структуру: id блоков, marks, ссылки, коллауты, таблицы. Предпочтительный
инструмент для правки формулировок, опечаток, чисел и имён.
- **`patch_node`** — Заменить один блок, адресованный по `attrs.id` (из `get_page_json`),
- **`patchNode`** — Заменить один блок, адресованный по `attrs.id` (из `getPageJson`),
без пересылки документа.
- **`insert_node`** — Вставить блок до/после другого (по `attrs.id` или по якорному тексту)
- **`insertNode`** — Вставить блок до/после другого (по `attrs.id` или по якорному тексту)
либо добавить в конец.
- **`delete_node`** — Удалить один блок по его `attrs.id`.
- **`update_page_json`** — Заменить весь контент страницы документом ProseMirror (массовые
- **`deleteNode`** — Удалить один блок по его `attrs.id`.
- **`updatePageJson`** — Заменить весь контент страницы документом ProseMirror (массовые
перезаписи или когда у узлов нет id). `content` опционален — опустите его, чтобы изменить
только заголовок. Сохраняет переданные id блоков, поэтому якоря заголовков и история
остаются стабильными.
- **`docmost_transform`** — Агентоориентированный интерфейс редактирования: вместо
- **`updatePageMarkdown`** — Заменить тело страницы (и опционально заголовок) новым
**обычным Markdown**. Всё тело переимпортируется (id блоков перегенерируются — для
хирургических правок или сохранения id используйте `editPageText` / `patchNode` /
`updatePageJson`). Markdown в диалекте Docmost разбирается, включая inline-сноски `^[...]`.
- **`docmostTransform`** — Агентоориентированный интерфейс редактирования: вместо
перепечатывания документа агент **пишет функцию, которая его чинит**. Редактирует
страницу, запуская произвольный **JS-трансформ `(doc, ctx) => doc`** на её *живом*
документе ProseMirror. Работает в **песочнице** (без `require`/`process`/`fs`/сети,
@@ -173,43 +180,46 @@ Docmost-MCP не сочетают:
### Таблицы
- **`table_get`** — Прочитать таблицу как матрицу: `{rows, cols, cells (text[][]),
- **`tableGet`** — Прочитать таблицу как матрицу: `{rows, cols, cells (text[][]),
cellIds}` (id абзаца на ячейку или `null`). Адресуйте таблицу через `#<index>` (из
`get_outline`) или любой id блока внутри неё. Используйте `cellIds` вместе с `patch_node`
`getOutline`) или любой id блока внутри неё. Используйте `cellIds` вместе с `patchNode`
для правок ячеек с форматированием.
- **`table_insert_row`** — Вставить строку из текстовых ячеек, дополненную до числа
- **`tableInsertRow`** — Вставить строку из текстовых ячеек, дополненную до числа
столбцов таблицы (передать ячеек больше числа столбцов — ошибка). `index` — 0-based
позиция вставки (0 вставляет перед заголовком); опустите, чтобы добавить в конец.
- **`table_delete_row`** — Удалить строку по 0-based `index`. Отказывается удалять
- **`tableDeleteRow`** — Удалить строку по 0-based `index`. Отказывается удалять
единственную строку таблицы; удаление строки 0 делает заголовком следующую строку.
- **`table_update_cell`** — Задать текстовое содержимое ячейки `[row, col]` (0-based). Для
форматирования используйте `patch_node` по id абзаца ячейки из `table_get`.
- **`tableUpdateCell`** — Задать текстовое содержимое ячейки `[row, col]` (0-based). Для
форматирования используйте `patchNode` по id абзаца ячейки из `tableGet`.
### Markdown: экспорт и импорт
- **`export_page_markdown`** — Экспортировать страницу в один самодостаточный, **lossless
Markdown в диалекте Docmost**: мета-заголовок, тело с inline-якорями комментариев и
диаграммами и завершающий блок тредов комментариев. Рассчитан на цикл «скачать →
отредактировать тело → `import_page_markdown`», сохраняющий всё, включая выделения
комментариев.
- **`import_page_markdown`** — Заменить контент страницы из Markdown-файла в диалекте
Docmost, созданного `export_page_markdown`, восстанавливая якоря-выделения комментариев и
диаграммы из их inline-HTML. (Треды комментариев из файла не пересоздаются на сервере —
записываются только тело страницы и inline-марки комментариев; тредами управляйте через
инструменты/UI комментариев.)
- **`exportPageMarkdown`** — Экспортировать страницу в один самодостаточный
**Markdown в диалекте Docmost**: мета-заголовок, тело с inline-якорями комментариев и
диаграммами и завершающий блок тредов комментариев. Round-trip скачать → отредактировать →
импортировать перегенерирует id блоков и **молча отбрасывает** набор атрибутов без
markdown-представления (спаны/colwidth/фон ячеек таблиц, отступы (indent), `callout.icon`,
`orderedList.type`, `internal`/`target`/`rel`/`class` у ссылок); держите их в ProseMirror
JSON, если они должны выжить. Чтобы заменить тело страницы из обычного авторского Markdown,
используйте `updatePageMarkdown`.
> **Удалено в этом релизе:** `importPageMarkdown` (парсер round-trip для
> экспортированного Docmost-Markdown-файла) **больше не отдаётся на внешней MCP-поверхности**.
> Чтобы заменить тело страницы из Markdown, используйте **`updatePageMarkdown`** (замена
> тела обычным Markdown). См. заметку о миграции в CHANGELOG.
### Изображения
- **`insert_image`** — Загрузить локальное изображение и вставить за один шаг: добавить в
- **`insertImage`** — Загрузить локальное изображение и вставить за один шаг: добавить в
конец, поставить вместо текстового плейсхолдера (`replaceText`) или после заданного блока
(`afterText`). Сохраняет id всех остальных блоков.
- **`replace_image`** — Заменить существующее изображение. Загружает новый файл как **новое
- **`replaceImage`** — Заменить существующее изображение. Загружает новый файл как **новое
вложение** (чистый URL, который рендерится и сбрасывает кэш браузера), затем
перенаправляет все узлы, ссылавшиеся на старое вложение (рекурсивно, включая
коллауты/таблицы), через живой документ, сохраняя комментарии, выравнивание и alt-текст.
(Перезапись «по месту» намеренно не используется — некоторые версии Docmost портят
вложение при перезаписи.)
- **`stash_page`** — Сериализовать страницу целиком (её полный ProseMirror JSON) в
- **`stashPage`** — Сериализовать страницу целиком (её полный ProseMirror JSON) в
эфемерный blob в оперативной памяти и вернуть ТОЛЬКО короткий анонимный URL — тело
никогда не попадает в контекст модели, поэтому это способ передать большую страницу
(вместе с её изображениями) внешнему потребителю без усечения. Каждое внутреннее
@@ -221,35 +231,35 @@ Docmost-MCP не сочетают:
### Комментарии
- **`create_comment`** — Добавить комментарий к странице, опционально **привязав inline** к
- **`createComment`** — Добавить комментарий к странице, опционально **привязав inline** к
точному фрагменту текста (первое вхождение оборачивается comment-маркой).
- **`list_comments`** — Список комментариев страницы (контент возвращается как Markdown).
- **`update_comment`** — Изменить существующий комментарий.
- **`delete_comment`** — Удалить комментарий.
- **`resolve_comment`** — Закрыть (resolve) или переоткрыть тред комментария (обратимо). Resolve
доступен только для корневых комментариев; тред и ответы сохраняются, в отличие от `delete_comment`.
- **`check_new_comments`** — Найти комментарии, созданные после заданной метки времени
- **`listComments`** — Список комментариев страницы (контент возвращается как Markdown).
- **`updateComment`** — Изменить существующий комментарий.
- **`deleteComment`** — Удалить комментарий.
- **`resolveComment`** — Закрыть (resolve) или переоткрыть тред комментария (обратимо). Resolve
доступен только для корневых комментариев; тред и ответы сохраняются, в отличие от `deleteComment`.
- **`checkNewComments`** — Найти комментарии, созданные после заданной метки времени
ISO-8601, по пространству, опционально в рамках поддерева страниц — идеально для агента,
который следит за обратной связью в документе.
### Версии и история
- **`list_page_history`** — Сохранённые версии страницы (Docmost авто-снапшотит при каждом
- **`listPageHistory`** — Сохранённые версии страницы (Docmost авто-снапшотит при каждом
сохранении), новые сверху, курсорная пагинация. id каждого элемента — это `historyId`.
- **`diff_page_versions`** — Дифф двух версий (или версии против живой страницы).
- **`diffPageVersions`** — Дифф двух версий (или версии против живой страницы).
Возвращает вставленный/удалённый текст, счётчики целостности (изображения, ссылки,
таблицы, коллауты, маркеры сносок) и человекочитаемую Markdown-сводку — посчитано тем же
конвейером, что использует встроенный просмотр истории Docmost.
- **`restore_page_version`** — Записать сохранённую версию обратно как текущий контент. У
- **`restorePageVersion`** — Записать сохранённую версию обратно как текущий контент. У
Docmost нет эндпоинта восстановления, поэтому создаётся **новый** снапшот — само
восстановление тоже обратимо.
### Публикация
- **`share_page`** — Сделать страницу публично доступной (идемпотентно) и вернуть её
- **`sharePage`** — Сделать страницу публично доступной (идемпотентно) и вернуть её
публичный URL (`<app>/share/<key>/p/<slugId>`); опционально индексирование поисковиками.
- **`unshare_page`** — Отозвать публичный доступ к странице.
- **`list_shares`** — Все публичные ссылки воркспейса с заголовками и публичными URL.
- **`unsharePage`** — Отозвать публичный доступ к странице.
- **`listShares`** — Все публичные ссылки воркспейса с заголовками и публичными URL.
---
@@ -258,28 +268,29 @@ Docmost-MCP не сочетают:
Та же подсказка отдаётся в рантайме через поле `instructions` MCP-сервера, так что
подходящие клиенты направляют модель автоматически.
- **Правки текста** (формулировки, опечатки, числа): `edit_page_text`.
- **Один блок** (абзац/заголовок/коллаут/ячейка таблицы): `patch_node` / `insert_node` /
`delete_node`, адресуя узел по его `attrs.id` из `get_page_json`.
- **Изображения**: `insert_image` / `replace_image`.
- **Новая страница**: `create_page`.
- **Массовая перезапись или узлы без id**: `update_page_json`.
- **Правки текста** (формулировки, опечатки, числа): `editPageText`.
- **Один блок** (абзац/заголовок/коллаут/ячейка таблицы): `patchNode` / `insertNode` /
`deleteNode`, адресуя узел по его `attrs.id` из `getPageJson`.
- **Изображения**: `insertImage` / `replaceImage`.
- **Новая страница**: `createPage`.
- **Массовая перезапись или узлы без id**: `updatePageJson` (ProseMirror) или
`updatePageMarkdown` (замена тела обычным Markdown).
- **Многошаговая / скриптовая перезапись** (перенумерация, сноски, согласованные правки):
`docmost_transform` — предпросмотр через `dryRun`, затем применение.
`docmostTransform` — предпросмотр через `dryRun`, затем применение.
- **Скопировать контент целой страницы из другой** (на стороне сервера):
`copy_page_content`.
- **Переименовать страницу** (только заголовок): `rename_page`.
- **Чтение**: `get_page` (Markdown) / `get_page_json` (lossless ProseMirror с id).
- **Просмотр изменений**: `list_page_history``diff_page_versions`
`restore_page_version`.
- **Комментарии**: `create_comment` (с опциональной inline-привязкой) / `list_comments` /
`update_comment` / `resolve_comment` / `delete_comment` / `check_new_comments`.
- **Дешёвая навигация по странице** (найти раздел/таблицу, получить id блока): `get_outline`
`get_node`.
- **Таблицы** (добавить/удалить строку, задать ячейку): `table_get` / `table_insert_row` /
`table_delete_row` / `table_update_cell`.
- **Round-trip страницы через Markdown** (скачать, отредактировать, залить обратно без
потерь, с комментариями): `export_page_markdown` / `import_page_markdown`.
`copyPageContent`.
- **Переименовать страницу** (только заголовок): `renamePage`.
- **Чтение**: `getPage` (Markdown) / `getPageJson` (lossless ProseMirror с id).
- **Просмотр изменений**: `listPageHistory``diffPageVersions`
`restorePageVersion`.
- **Комментарии**: `createComment` (с опциональной inline-привязкой) / `listComments` /
`updateComment` / `resolveComment` / `deleteComment` / `checkNewComments`.
- **Дешёвая навигация по странице** (найти раздел/таблицу, получить id блока): `getOutline`
`getNode`.
- **Таблицы** (добавить/удалить строку, задать ячейку): `tableGet` / `tableInsertRow` /
`tableDeleteRow` / `tableUpdateCell`.
- **Экспорт страницы в самодостаточный Markdown** (с якорями комментариев): `exportPageMarkdown`.
- **Заменить тело страницы из Markdown**: `updatePageMarkdown`.
---
@@ -291,28 +302,49 @@ Docmost-MCP не сочетают:
правки, которых ещё нет в дебаунс-снапшоте REST), затем **читает → трансформирует →
пишет синхронно** в одном тике, чтобы никакое удалённое обновление не вклинилось, и
**ждёт подтверждения сохранения** до возврата.
- **Сериализация записи по странице.** Асинхронный мьютекс по `pageId` гарантирует, что
две записи MCP в одну страницу никогда не пересекаются; разные страницы друг друга не
блокируют.
- **Сериализация записи по странице.** Асинхронный мьютекс по разрешённому **UUID**
страницы (никогда не по slugId) гарантирует, что две записи MCP в одну страницу никогда
не пересекаются; разные страницы друг друга не блокируют. Хелпер блокировки падает сразу
(fail-fast), если ему передали не-UUID ключ, — путь записи, забывший разрезолвить id, не
сможет молча взять лок под расщеплённым ключом.
**Требование к деплою — один инстанс или sticky-сессии.** Этот мьютекс — процесс-локальный
`Map`, а кэш collab-сессий и хранилище `stashPage` живут только в RAM одного процесса. За
**мультиреплика**-балансировщиком **без sticky-сессий** две реплики могут одновременно
«держать» лок одной страницы, и сериализация по странице молча теряется. Запускайте
MCP/приложение **одним инстансом** либо прибивайте трафик страницы к одной реплике
(sticky-сессии / consistent hashing по id страницы). Кросс-процессной блокировки (например,
Postgres advisory-lock) намеренно пока нет — осознанное задокументированное ограничение.
См. комментарий в `Dockerfile` и блок `MCP collaboration write path` в `.env.example`.
**Окно устаревших прав.** Кэшированная collab-сессия пишет под токеном, захваченным в
момент connect (а кэш collab-токена переиспользует токен в пределах своего TTL), поэтому
**отозванный** доступ к странице может лагать до `MCP_COLLAB_SESSION_MAX_AGE_MS` (жёсткий
срок жизни сессии, по умолчанию 10 мин), пока следующая переавторизация его не подхватит.
Уменьшите значение, чтобы сократить лаг ценой большего числа переподключений. Это
ограниченное окно — принятый trade-off; push-инвалидации кэша при смене прав нет.
- **Прозрачная переавторизация.** Логин по email/паролю; истёкшие токены обновляются
автоматически на первом 401/403 (покрывая JSON, multipart-загрузку и путь токена
коллаборации), с дедупликацией параллельных логинов, так что пачка вызовов вызывает один
повторный логин.
- **Lossless- и lossy-чтение.** `get_page_json` возвращает точное дерево ProseMirror с id
блоков; `get_page` возвращает чистый Markdown для удобства.
- **Точные чтения.** `getPageJson` возвращает точное дерево ProseMirror с id блоков;
`getPage` возвращает канонический Markdown, теряющий лишь фиксированный, документированный
набор атрибутов.
- **Полная схема Docmost.** Конвертация Markdown↔ProseMirror поддерживает коллауты
(включая вложенные), списки задач (маркированные *и* нумерованные чек-листы), таблицы,
блоки формул, эмбеды, выделение, под/надстрочный текст и прочее, с защитными лимитами
против патологического ввода.
- **Структурные таблицы и lossless Markdown round-trip.** Таблицы можно редактировать как
- **Структурные таблицы и Markdown round-trip.** Таблицы можно редактировать как
матрицу (чтение, вставка/удаление строк, задание ячеек по `[row, col]`) без пересылки
документа, а страницу — экспортировать и заново импортировать как самодостаточный
Markdown-файл в диалекте Docmost, сохраняющий inline-якоря комментариев и диаграммы.
Markdown-файл в диалекте Docmost, сохраняющий inline-якоря комментариев и диаграммы
(id блоков перегенерируются, а фиксированный набор атрибутов без markdown-представления
отбрасывается — см. `exportPageMarkdown`).
- **Ответы, оптимизированные по токенам.** Ответы API урезаются до полей, действительно
нужных агентам, а большие коллекции (пространства, страницы, комментарии, история)
пагинируются.
- **Закалённый рантайм.** Глобальные обработчики не дают случайной ошибке сокета уронить
stdio-сервер; `move_page` требует положительно подтверждённого успеха; движок диффа
stdio-сервер; `movePage` требует положительно подтверждённого успеха; движок диффа
откатывается к грубому поблочному диффу, а не падает на патологическом документе.
---
@@ -372,7 +404,7 @@ npm run test:e2e
Проект начинался как форк
[MrMartiniMo/docmost-mcp](https://github.com/MrMartiniMo/docmost-mcp) (автор Moritz Krause)
и существенно его расширяет — добавлены поблочное редактирование узлов, хирургические
правки текста, песочница `docmost_transform`, история версий / дифф / восстановление,
правки текста, песочница `docmostTransform`, история версий / дифф / восстановление,
комментарии, вставка/замена изображений, публичные ссылки, серверное копирование страниц,
двойное чтение JSON/Markdown, прозрачная переавторизация и значительное упрочнение.
Инструменты комментариев портированы из upstream PR #3 от Max Nikitin. Спасибо обоим.
+10 -10
View File
@@ -22,14 +22,14 @@ are debounced server-side, so the script waits ~16 s before reading back via RES
| # | Tool / path | What is checked | Expected |
|---|-------------|-----------------|----------|
| 1 | `create_page` | title with spaces, slugId returned | page created, title intact |
| 1 | `createPage` | title with spaces, slugId returned | page created, title intact |
| 2 | `update_page` (markdown) | headings, **bold**/*italic*/~~strike~~/`code`/link, nested bullet + ordered lists, blockquote, code block, `:::callout:::`, table | all structures survive re-import |
| 3 | `get_page_json` | lossless ProseMirror, block ids, callout/table nodes | present (note: reads the **debounced** REST snapshot — recent collab writes may lag a few seconds) |
| 4 | `edit_page_text` | surgical replace; block ids + marks preserved; ambiguous match rejected; missing match reported | edits applied, ids stable, errors correct |
| 5 | `update_page_json` | full lossless write; custom block ids preserved; existing content (text edits, images, callout, table) not lost | round-trips intact |
| 3 | `getPageJson` | lossless ProseMirror, block ids, callout/table nodes | present (note: reads the **debounced** REST snapshot — recent collab writes may lag a few seconds) |
| 4 | `editPageText` | surgical replace; block ids + marks preserved; ambiguous match rejected; missing match reported | edits applied, ids stable, errors correct |
| 5 | `updatePageJson` | full lossless write; custom block ids preserved; existing content (text edits, images, callout, table) not lost | round-trips intact |
| 6 | `upload_image` | uploads attachment, returns node | src is a **clean** `/api/files/<id>/<file>` URL, served `200 image/*` |
| 7 | `insert_image` (append / `replaceText` / `afterText`) | three placements | image lands in the right place, all other block ids preserved |
| 8 | **`replace_image`** | swap an existing figure for new bytes; comments/align/alt preserved; **the new URL must actually serve the image** | new image renders (`200`), old node repointed |
| 7 | `insertImage` (append / `replaceText` / `afterText`) | three placements | image lands in the right place, all other block ids preserved |
| 8 | **`replaceImage`** | swap an existing figure for new bytes; comments/align/alt preserved; **the new URL must actually serve the image** | new image renders (`200`), old node repointed |
## Image-specific assertions (the recurring bug area)
@@ -39,7 +39,7 @@ For every uploaded/inserted/replaced image, assert at the HTTP level that the
* `GET <src>``200`, `Content-Type: image/*`, body starts with the image magic
(`89 50 4E 47` for PNG, etc.).
* `src` does **not** contain a `?v=` query (see "Known pitfalls").
* After `replace_image`: the returned `newAttachmentId` **differs** from the old
* After `replaceImage`: the returned `newAttachmentId` **differs** from the old
one (replacement uses a fresh attachment → fresh URL), and `GET <new src>``200`.
* The old image node on the page is repointed to the new attachmentId.
@@ -64,7 +64,7 @@ broken/empty figure.
Uploading with an existing `attachmentId` (`POST /files/upload` + `attachmentId`)
overwrites the bytes in place. On this Docmost the attachment then returns
**500 for every URL** (clean, `?v=`, any filename) → broken image. Therefore
`replace_image` must upload a **new** attachment and repoint the nodes; the new
`replaceImage` must upload a **new** attachment and repoint the nodes; the new
id yields a new URL that both renders and busts the browser cache. The old
attachment is left as an unreferenced orphan: Docmost exposes **no HTTP API to
delete a single content attachment** (verified against the attachment
@@ -80,9 +80,9 @@ broken/empty figure.
from `?v=`. Image `src` is kept clean (`/api/files/<id>/<file>`); cache-busting
on replace is achieved by the new attachment id.
3. **REST snapshot lag.** `get_page_json` reads the debounced DB snapshot, so a
3. **REST snapshot lag.** `getPageJson` reads the debounced DB snapshot, so a
write made moments earlier may not be visible yet. Wait (~16 s) before reading
back, and never feed a possibly-stale snapshot straight into `update_page_json`.
back, and never feed a possibly-stale snapshot straight into `updatePageJson`.
4. **Callout type narrowing (minor, open).** A `:::warning` callout is imported as
`type: "info"` — the markdown→callout conversion does not carry non-`info`
+63
View File
@@ -0,0 +1,63 @@
{
"$comment": "Semantic palettes for drawioFromGraph (issue #425). DATA, not code: node `kind` -> fill/stroke slot, edge `kind` -> line-style props, per preset. The `default` node palette is the issue's base table; `dark` keeps the same hues on a dark canvas with lighter strokes/font; `colorblind-safe` maps every slot onto the Okabe-Ito qualitative palette (8 colours proven distinguishable for all common colour-vision deficiencies) so no two adjacent kinds collide. `fontColor`/`fillColor`/`strokeColor` are exact draw.io values. `edgeDefault` is the fallback line style; `group` is the (always-transparent) container stroke per preset.",
"presets": {
"default": {
"canvasDark": false,
"nodes": {
"service": { "fillColor": "#dae8fc", "strokeColor": "#6c8ebf", "fontColor": "#000000" },
"db": { "fillColor": "#d5e8d4", "strokeColor": "#82b366", "fontColor": "#000000" },
"queue": { "fillColor": "#fff2cc", "strokeColor": "#d6b656", "fontColor": "#000000" },
"gateway": { "fillColor": "#ffe6cc", "strokeColor": "#d79b00", "fontColor": "#000000" },
"error": { "fillColor": "#f8cecc", "strokeColor": "#b85450", "fontColor": "#000000" },
"external": { "fillColor": "#f5f5f5", "strokeColor": "#666666", "fontColor": "#333333" },
"security": { "fillColor": "#e1d5e7", "strokeColor": "#9673a6", "fontColor": "#000000" }
},
"edges": {
"sync": { "props": "" },
"async": { "props": "dashed=1;" },
"error": { "props": "dashed=1;strokeColor=#DD344C;" }
},
"edgeDefault": { "strokeColor": "#333333", "fontColor": "#333333" },
"group": { "strokeColor": "#666666", "fontColor": "#333333" }
},
"dark": {
"canvasDark": true,
"nodes": {
"service": { "fillColor": "#1a2a44", "strokeColor": "#7ea6e0", "fontColor": "#dae8fc" },
"db": { "fillColor": "#1f331e", "strokeColor": "#97d077", "fontColor": "#d5e8d4" },
"queue": { "fillColor": "#3a3218", "strokeColor": "#e5c15a", "fontColor": "#fff2cc" },
"gateway": { "fillColor": "#3a2812", "strokeColor": "#ffb570", "fontColor": "#ffe6cc" },
"error": { "fillColor": "#3a1c1b", "strokeColor": "#e08e8b", "fontColor": "#f8cecc" },
"external": { "fillColor": "#2b2b2b", "strokeColor": "#999999", "fontColor": "#e0e0e0" },
"security": { "fillColor": "#2c2338", "strokeColor": "#b39ddb", "fontColor": "#e1d5e7" }
},
"edges": {
"sync": { "props": "" },
"async": { "props": "dashed=1;" },
"error": { "props": "dashed=1;strokeColor=#ff6b6b;" }
},
"edgeDefault": { "strokeColor": "#cccccc", "fontColor": "#e0e0e0" },
"group": { "strokeColor": "#aaaaaa", "fontColor": "#e0e0e0" }
},
"colorblind-safe": {
"canvasDark": false,
"okabeIto": ["#000000", "#E69F00", "#56B4E9", "#009E73", "#F0E442", "#0072B2", "#D55E00", "#CC79A7"],
"nodes": {
"service": { "fillColor": "#D6E9F5", "strokeColor": "#0072B2", "fontColor": "#000000" },
"db": { "fillColor": "#D6EFE4", "strokeColor": "#009E73", "fontColor": "#000000" },
"queue": { "fillColor": "#FCF8CC", "strokeColor": "#F0E442", "fontColor": "#000000" },
"gateway": { "fillColor": "#FBEBD0", "strokeColor": "#E69F00", "fontColor": "#000000" },
"error": { "fillColor": "#F7DDCC", "strokeColor": "#D55E00", "fontColor": "#000000" },
"external": { "fillColor": "#EDEDED", "strokeColor": "#000000", "fontColor": "#000000" },
"security": { "fillColor": "#F3DEEB", "strokeColor": "#CC79A7", "fontColor": "#000000" }
},
"edges": {
"sync": { "props": "" },
"async": { "props": "dashed=1;" },
"error": { "props": "dashed=1;strokeColor=#D55E00;" }
},
"edgeDefault": { "strokeColor": "#000000", "fontColor": "#000000" },
"group": { "strokeColor": "#000000", "fontColor": "#000000" }
}
}
}
Binary file not shown.
+10 -2
View File
@@ -5,9 +5,16 @@
"private": true,
"type": "module",
"main": "./build/index.js",
"types": "./build/index.d.ts",
"exports": {
".": "./build/index.js",
"./http": "./build/http.js"
".": {
"types": "./build/index.d.ts",
"default": "./build/index.js"
},
"./http": {
"types": "./build/http.d.ts",
"default": "./build/http.js"
}
},
"bin": {
"docmost-mcp": "./build/stdio.js"
@@ -50,6 +57,7 @@
"@tiptap/starter-kit": "3.20.4",
"@types/jsdom": "^27.0.0",
"axios": "^1.6.0",
"elkjs": "^0.11.1",
"form-data": "^4.0.0",
"jsdom": "^27.4.0",
"marked": "^17.0.1",
+89 -4322
View File
File diff suppressed because it is too large Load Diff
+704
View File
@@ -0,0 +1,704 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import { assertFullUuid } from "./errors.js";
import {
filterWorkspace,
filterSpace,
filterPage,
filterComment,
filterSearchResult,
} from "../lib/filters.js";
import { convertProseMirrorToMarkdown } from "../lib/markdown-converter.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import {
applyAnchorInDoc,
countAnchorMatches,
getAnchoredText,
resolveAnchorSelection,
normalizeForMatch,
} from "../lib/comment-anchor.js";
import { closestBlockHint } from "../lib/text-normalize.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Public method surface of CommentsMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements ICommentsMixin` fails to compile on drift.
export interface ICommentsMixin {
listComments(pageId: string, includeResolved?: boolean): any;
getComment(commentId: string): any;
createComment(pageId: string, content: string, type?: "page" | "inline", selection?: string, parentCommentId?: string, suggestedText?: string): any;
updateComment(commentId: string, content: string): any;
deleteComment(commentId: string): any;
resolveComment(commentId: string, resolved: boolean): any;
checkNewComments(spaceId: string, since: string, parentPageId?: string): any;
}
export function CommentsMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & ICommentsMixin> & TBase {
abstract class CommentsMixin extends Base implements ICommentsMixin {
// --- Comment methods (ported from upstream PR #3 by Max Nikitin) ---
/**
* Normalize a comment's `content` into a ProseMirror doc object before
* markdown conversion. createComment/updateComment send content as a
* JSON.stringify(...) STRING, and the server stores it as-is, so on read it
* comes back as a string. convertProseMirrorToMarkdown returns "" for a
* string, so parse it first (guarded fall back to the raw value on any
* parse failure so a non-JSON legacy value is still handled gracefully).
*/
protected parseCommentContent(content: any): any {
if (typeof content !== "string") return content;
try {
return JSON.parse(content);
} catch {
return content;
}
}
/**
* List comments on a page (cursor-paginated), content as markdown.
*
* DEFAULT (`includeResolved = false`) hides RESOLVED THREADS WHOLESALE so the
* agent sees only active discussions: a top-level comment with `resolvedAt`
* set AND every reply under it (a reply of a closed thread is part of the
* closed thread) are dropped from `items`. `resolvedThreadsHidden` reports how
* many resolved top-level threads were hidden so the agent can re-query with
* `includeResolved: true` to see everything. Active threads always stay.
*
* Returns `{ items, resolvedThreadsHidden }` (NOT a bare array) callers that
* need the full feed (lossless export, transformPage, checkNewComments) pass
* `includeResolved: true` and read `.items`.
*/
async listComments(pageId: string, includeResolved = false) {
await this.ensureAuthenticated();
let allComments: any[] = [];
let cursor: string | null = null;
// Hard ceiling + immovable-cursor guard (mirrors paginateAll): if /comments
// ever stops advancing the cursor (the exact #442 drift scenario) this loop
// would otherwise spin forever accumulating duplicates.
const MAX_PAGES = 50;
let truncated = false;
for (let page = 0; page < MAX_PAGES; page++) {
const payload: Record<string, any> = { pageId, limit: 100 };
if (cursor) payload.cursor = cursor;
const response = await this.client.post("/comments", payload);
const data = response.data.data || response.data;
const items = data.items || [];
allComments = allComments.concat(items);
// Advance strictly via the server-issued cursor. A missing nextCursor or a
// cursor identical to the one we just sent means the end (or a server that
// ignores our pagination param) — stop instead of re-fetching page one.
const next: string | null = data.meta?.nextCursor || null;
if (!next || next === cursor) break;
cursor = next;
// Reaching the ceiling with a still-advancing cursor means truncation.
if (page === MAX_PAGES - 1) truncated = true;
}
if (truncated) {
console.warn(
`listComments: comments for "${pageId}" truncated at the ${MAX_PAGES}-page cap; more pages exist on the server`,
);
}
const mapped = allComments.map((comment: any) => {
const markdown = comment.content
? convertProseMirrorToMarkdown(
this.parseCommentContent(comment.content),
)
: "";
return filterComment(comment, markdown);
});
if (includeResolved) {
return { items: mapped, resolvedThreadsHidden: 0 };
}
// Ids of RESOLVED top-level threads (a top-level comment has no
// parentCommentId). A whole thread is hidden when its root is resolved.
const resolvedRootIds = new Set(
mapped
.filter((c) => !c.parentCommentId && c.resolvedAt != null)
.map((c) => c.id),
);
const items = mapped.filter((c) => {
// Hide the resolved root itself and every reply anchored to it. A reply's
// own resolvedAt is irrelevant — its membership follows the parent thread.
// ASSUMPTION: Docmost's comment model is FLAT — a reply's parentCommentId
// always points at the thread ROOT (no reply-of-reply nesting), so a single
// level of parent lookup covers a whole thread. If nested replies are ever
// introduced, a deep reply of a resolved thread would need a root-walk here.
if (!c.parentCommentId) return !resolvedRootIds.has(c.id);
return !resolvedRootIds.has(c.parentCommentId);
});
return { items, resolvedThreadsHidden: resolvedRootIds.size };
}
async getComment(commentId: string) {
// Fail fast (#436): reject a truncated id before any network call.
assertFullUuid("get_comment", "commentId", commentId);
await this.ensureAuthenticated();
const response = await this.client.post("/comments/info", { commentId });
const comment = response.data.data || response.data;
const markdown = comment.content
? convertProseMirrorToMarkdown(this.parseCommentContent(comment.content))
: "";
return {
data: filterComment(comment, markdown),
success: true,
};
}
/** Plain text of each TOP-LEVEL block of `doc`, for anchor-failure hints. */
protected topLevelBlockTexts(doc: any): string[] {
const content = doc && Array.isArray(doc.content) ? doc.content : [];
return content
.map((b: any) => blockPlainText(b))
.filter((t: string) => t.length > 0);
}
/**
* True when per-block anchoring failed but the (normalized) selection DOES
* appear in the blocks' joined plain text i.e. it straddles a block
* boundary. Blocks are joined with a newline (collapsed to one space by
* normalizeForMatch) so a selection whose parts are separated by a paragraph
* break still matches. Callers only reach here after single-block anchoring
* (incl. the markdown-strip fallback) has already failed.
*/
protected selectionSpansMultipleBlocks(
blockTexts: string[],
selection: string,
): boolean {
const normSel = normalizeForMatch(selection).norm.trim();
if (normSel.length === 0) return false;
const joined = normalizeForMatch(blockTexts.join("\n")).norm;
return joined.indexOf(normSel) !== -1;
}
/**
* Build the actionable error for a createComment anchor MISS, porting
* editPageText's self-correction affordances: an explicit "spans multiple
* blocks" message when the selection straddles a block boundary, otherwise a
* "closest block text" hint quoting the block that holds the selection's
* longest token. `live` switches the wording between the pre-check (reading the
* persisted page) and the post-create live-anchor failure (which rolls back).
*/
protected anchorNotFoundError(
doc: any,
selection: string,
live: boolean,
): Error {
const blockTexts = this.topLevelBlockTexts(doc);
const rolled = live ? " The comment was rolled back." : "";
if (this.selectionSpansMultipleBlocks(blockTexts, selection)) {
return new Error(
"createComment: the selection spans multiple blocks; anchor on a " +
"contiguous fragment within a SINGLE paragraph/block (<=250 chars)." +
rolled,
);
}
const where = live ? "in the live document" : "in the page";
return new Error(
`createComment: could not find the selection text ${where} to anchor ` +
"the comment. Provide the EXACT contiguous text from a single " +
"paragraph/block (<=250 chars)." +
closestBlockHint(blockTexts, selection) +
rolled,
);
}
/**
* Create an inline comment anchored to its `selection` text, or a reply.
*
* Top-level comments (no `parentCommentId`) are ALWAYS inline and MUST carry a
* `selection`: the `type` argument is kept for interface compatibility but the
* effective type is coerced to "inline". The selection has to anchor in the
* document; if it cannot, the comment is rolled back and an error is thrown so
* the caller is forced to supply a proper inline selection rather than leaving
* an orphan, unanchored comment behind. Replies (parentCommentId set) inherit
* their parent's anchor: they take NO selection and are not anchored.
*/
async createComment(
pageId: string,
content: string,
type: "page" | "inline" = "page",
selection?: string,
parentCommentId?: string,
suggestedText?: string,
) {
// Fail fast (#436): a provided parent id must be a full UUID before any
// network call. Validate only when truthy — a falsy parentCommentId means
// "top-level comment" (mirrors the isReply computation below), not a reply.
if (parentCommentId) {
assertFullUuid("createComment", "parentCommentId", parentCommentId);
}
await this.ensureAuthenticated();
const isReply = !!parentCommentId;
const hasSuggestion =
suggestedText !== undefined && suggestedText !== null;
// Defense in depth mirroring the server DTO/service: a suggested edit rewrites
// the exact anchored text, so it is only meaningful on a top-level inline
// comment that carries a selection.
if (hasSuggestion) {
if (isReply) {
throw new Error(
"createComment: a suggested edit (suggestedText) cannot be attached to a reply; it applies only to a top-level inline comment.",
);
}
if (!selection || !selection.trim()) {
throw new Error(
"createComment: a suggested edit (suggestedText) requires a 'selection' to anchor and rewrite.",
);
}
}
// Only top-level comments are inline-anchored, so they are stored as
// "inline". Replies carry no inline selection, so they keep the historical
// general ("page") type — both backward-compatible and semantically correct.
// The `type` argument is kept for interface compatibility; createComment
// normalizes the effective type internally, so callers may pass "inline".
const effectiveType: "page" | "inline" = isReply ? "page" : "inline";
if (!isReply && (!selection || !selection.trim())) {
throw new Error(
"createComment: an inline 'selection' (exact text to anchor on) is required for a top-level comment",
);
}
// For a SUGGESTION, the value we store as the comment's `selection` must be
// the RAW document substring the mark lands on (typographic quotes/dashes,
// nbsp, collapsed whitespace), NOT the agent's ASCII input. The anchor is
// placed via normalization, so when the doc was auto-converted to
// typographic the raw substring differs from the agent input; apply-time
// compares the stored selection to the marked doc text STRICTLY, so storing
// the raw substring is what makes "Apply" succeed instead of a spurious 409.
// Captured in the pre-check below (which already reads the page) and used as
// payload.selection. Ordinary comments keep sending the raw agent selection.
let anchoredSelection: string | null = null;
// Set when the anchor matched only after stripping markdown from the
// selection (the strip fallback); surfaced as a soft warning like
// editPageText does, so a stale-markdown selection is flagged.
let anchorNormalized = false;
// For a top-level comment, fail BEFORE creating anything when the selection
// is not present in the persisted document — this avoids leaving an orphan
// comment + notification behind. A read failure (network) is non-fatal: the
// live anchor step below still enforces the anchoring invariant.
if (!isReply && selection) {
try {
const page = await this.getPageJson(pageId);
if (hasSuggestion) {
// A suggestion's anchor MUST be unambiguous: applying it rewrites the
// exact anchored text, and ordinary anchoring silently takes the first
// occurrence, so 0 matches -> not found and >=2 -> ambiguous, both
// rejected BEFORE creating the comment.
const matches = countAnchorMatches(page.content, selection);
if (matches === 0) {
throw this.anchorNotFoundError(page.content, selection, false);
}
if (matches >= 2) {
throw new Error(
`createComment: the suggestion's selection is ambiguous — it occurs ${matches} times in the page. ` +
"A suggested edit must anchor to a UNIQUE location; expand the selection with surrounding context " +
"(still <=250 chars) so it appears exactly once.",
);
}
// Exactly one match: capture the RAW anchored substring to store as the
// comment selection (so apply-time equality holds). If this returns
// null despite countAnchorMatches===1 (shouldn't happen), fall back to
// the raw agent selection below rather than crash.
anchoredSelection = getAnchoredText(page.content, selection);
anchorNormalized = resolveAnchorSelection(
page.content,
selection,
).normalized;
} else {
const resolved = resolveAnchorSelection(page.content, selection);
if (!resolved.found) {
throw this.anchorNotFoundError(page.content, selection, false);
}
anchorNormalized = resolved.normalized;
}
} catch (e) {
// Rethrow our own "not found"/"ambiguous"/"spans multiple blocks" errors;
// swallow read/network errors so the live anchor step can still try (and
// enforce) anchoring.
if (
e instanceof Error &&
(e.message.startsWith("createComment: could not find the selection") ||
e.message.startsWith(
"createComment: the selection spans multiple blocks",
) ||
e.message.startsWith(
"createComment: the suggestion's selection is ambiguous",
))
) {
throw e;
}
if (process.env.DEBUG) {
console.error(
"Pre-check getPageJson failed; deferring to live anchor step:",
e,
);
}
}
}
// Convert through the full Docmost schema. Deliberately the NON-canonicalizing
// variant: a comment body may carry a footnote definition with no matching
// reference, and canonicalization would drop it (data loss). See
// markdownToProseMirror vs markdownToProseMirrorCanonical.
const jsonContent = await markdownToProseMirror(content);
const payload: Record<string, any> = {
pageId,
content: JSON.stringify(jsonContent),
type: effectiveType,
};
// For a suggestion, store the RAW anchored substring (anchoredSelection) so
// the stored selection === the text under the mark === apply-time
// expectedText. Ordinary comments (and the null fallback) keep the raw
// agent selection — their selection is only display/anchor and never used
// by apply, so their behavior is unchanged.
if (!isReply && selection)
payload.selection = anchoredSelection ?? selection;
if (parentCommentId) payload.parentCommentId = parentCommentId;
// Only a top-level inline comment (with a selection) may carry a suggestion.
if (!isReply && selection && hasSuggestion) {
payload.suggestedText = suggestedText;
}
const response = await this.client.post("/comments/create", payload);
const comment = response.data.data || response.data;
const markdown = comment.content
? convertProseMirrorToMarkdown(this.parseCommentContent(comment.content))
: content;
const result: any = {
data: filterComment(comment, markdown),
success: true,
};
// Replies inherit the parent's anchor: no selection, no anchoring.
if (isReply) {
return result;
}
// Anchor the comment in the document. The /comments/create API records the
// comment + its `selection` text, but it does NOT insert the comment MARK
// into the page content, so without this the inline comment has no
// highlight/anchor and is not clickable. If anchoring fails the comment is
// rolled back (deleted) and an error is thrown — never an orphan comment.
const newCommentId: string = comment.id;
// Guard: a create response without an id would mean writing a comment mark
// with commentId: undefined and a later delete of a falsy id. We have no id
// to roll back here (nothing was created with an id), so just fail loudly.
if (!newCommentId) {
throw new Error(
"createComment: the server returned no comment id, so the comment could not be anchored",
);
}
let anchored = false;
// Set inside the transform when a suggestion's live anchor is ambiguous
// (>=2 occurrences), so the rollback path can surface the right error.
let ambiguousInLiveDoc = false;
// Captured inside the transform on a not-found abort, so the rollback path
// can surface the closest-block / spans-multiple-blocks hint built from the
// LIVE document (the pre-check page is not in scope there).
let liveNotFoundError: Error | null = null;
try {
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260). The
// /comments/create REST call above keeps the agent-supplied id.
const pageUuid = await this.resolvePageId(pageId);
// Route through the mutatePage seam (not the free function) so this
// wrapper's uniqueness gate + rollback can be unit-tested without a live
// Hocuspocus collab socket.
const mutation = await this.mutatePage(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
const doc =
liveDoc && liveDoc.type === "doc"
? liveDoc
: { type: "doc", content: [] };
if (hasSuggestion) {
// Authoritative uniqueness check against the LIVE document: a
// suggestion must anchor to EXACTLY ONE occurrence, otherwise
// "Apply" would rewrite the wrong/ambiguous text. If the live doc
// no longer has exactly one occurrence (it changed since the
// pre-check), abort so the just-created comment is rolled back
// rather than mis-anchored to the first occurrence.
const liveCount = countAnchorMatches(doc, selection as string);
if (liveCount !== 1) {
ambiguousInLiveDoc = liveCount >= 2;
if (liveCount === 0) {
liveNotFoundError = this.anchorNotFoundError(
doc,
selection as string,
true,
);
}
return null;
}
}
if (applyAnchorInDoc(doc, selection as string, newCommentId)) {
anchored = true;
return doc;
}
// Selection text not found in the LIVE document: abort the write. The
// rollback + throw below turns this into a hard error.
liveNotFoundError = this.anchorNotFoundError(
doc,
selection as string,
true,
);
return null;
},
);
result.verify = mutation.verify;
} catch (e) {
// The comment record already exists; roll it back so we never leave an
// orphan, then rethrow the original anchoring error.
await this.safeDeleteComment(newCommentId);
throw e;
}
if (!anchored) {
// Mutation aborted because the selection was not found (or, for a
// suggestion, was ambiguous) in the live document. Roll back the comment
// and surface a hard error.
await this.safeDeleteComment(newCommentId);
if (ambiguousInLiveDoc) {
throw new Error(
"createComment: the suggestion's selection is ambiguous in the live document (multiple occurrences); the comment was rolled back. Expand the selection with surrounding context so it is unique.",
);
}
throw (
liveNotFoundError ??
new Error(
"createComment: failed to anchor the comment (selection not found in the live document); the comment was rolled back",
)
);
}
// Soft warning (like editPageText): the selection only matched after
// stripping markdown, so the caller likely quoted a styled fragment.
if (anchorNormalized) {
result.warning =
"The selection matched only after stripping markdown syntax; the comment " +
"was anchored on the document's plain text. Copy the selection verbatim " +
"from getPage / searchInPage output to avoid this.";
}
result.anchored = true;
return result;
}
/**
* Best-effort rollback of a just-created comment. Swallows any delete failure
* (logging under DEBUG) so a failed cleanup never masks the original error.
*/
protected async safeDeleteComment(commentId: string): Promise<void> {
// Defense in depth: never call the delete API with a falsy id — there is
// nothing to roll back, and deleteComment(undefined) would hit a bad route.
if (!commentId) return;
try {
await this.deleteComment(commentId);
} catch (delErr) {
if (process.env.DEBUG) {
console.error(
"Failed to roll back comment after anchoring error:",
delErr,
);
}
}
}
async updateComment(commentId: string, content: string) {
// Fail fast (#436): reject a truncated id before any network call.
assertFullUuid("updateComment", "commentId", commentId);
await this.ensureAuthenticated();
// NON-canonicalizing on purpose (comment body — see createComment).
const jsonContent = await markdownToProseMirror(content);
await this.client.post("/comments/update", {
commentId,
content: JSON.stringify(jsonContent),
});
return {
success: true,
commentId,
message: "Comment updated successfully.",
};
}
async deleteComment(commentId: string) {
// Fail fast (#436): reject a truncated id before any network call.
assertFullUuid("deleteComment", "commentId", commentId);
await this.ensureAuthenticated();
return this.client
.post("/comments/delete", { commentId })
.then((res) => res.data);
}
/**
* Resolve or reopen a top-level comment thread (reversible `resolved`
* toggles the state). Only top-level comments can be resolved; the server
* rejects resolving a reply. Hits POST /comments/resolve.
*/
async resolveComment(commentId: string, resolved: boolean) {
// Fail fast (#436): reject a truncated id before any network call.
assertFullUuid("resolveComment", "commentId", commentId);
await this.ensureAuthenticated();
const response = await this.client.post("/comments/resolve", {
commentId,
resolved,
});
const comment = response.data?.data ?? response.data;
return {
success: true,
commentId,
resolved,
comment,
};
}
/**
* Check for new comments across pages in a space (optionally scoped to a
* subtree): pages updated after `since` are scanned and their comments
* filtered by createdAt > since.
*/
async checkNewComments(
spaceId: string,
since: string,
parentPageId?: string,
) {
await this.ensureAuthenticated();
const sinceDate = new Date(since);
// Reject an unparseable `since`: comparing against an Invalid Date silently
// yields zero new comments (every `>` against NaN is false), which would
// mask a malformed input as "nothing new" instead of erroring.
if (Number.isNaN(sinceDate.getTime())) {
throw new Error(
`checkNewComments: invalid "since" date "${since}"; expected an ISO-8601 timestamp`,
);
}
// 1. Enumerate the FULL set of pages in scope via the page tree (a complete
// page index), NOT the bounded "/pages/recent" feed which caps at ~5000
// recent items and silently misses comments on older pages.
//
// Subtree scope: when parentPageId is given, the scope is that page ITSELF
// plus every descendant. Otherwise the scope is the whole space (all roots
// and their descendants).
//
// NOTE: do NOT pre-filter by page.updatedAt — creating a comment does not
// bump it (verified on a live server), so such a filter silently misses
// comments on pages that were not otherwise edited. The complete tree walk
// already restricts the scope correctly, so no recent-feed allow-list is
// needed any more.
//
// The subtree scope (parentPageId given) already INCLUDES the root node
// itself: /pages/tree seeds getPageAndDescendants with id = parentPageId, so
// no separate getPageRaw fetch for the parent is needed.
const { pages: pagesInScope, truncated } = await this.enumerateSpacePages(
spaceId,
parentPageId,
);
// 2. Fetch comments for each page, keep ones created after since
const results: any[] = [];
for (const page of pagesInScope) {
try {
// Full feed (incl. resolved): a "new comments since" scan reports all
// recent activity; the active-only filter is scoped to listComments.
const comments = (await this.listComments(page.id, true)).items;
const newComments = comments.filter(
(c: any) => new Date(c.createdAt) > sinceDate,
);
if (newComments.length > 0) {
results.push({
pageId: page.id,
pageTitle: page.title,
comments: newComments,
});
}
} catch (e: any) {
// Skip pages with errors (e.g. deleted between calls)
}
}
const totalNewComments = results.reduce(
(sum, r) => sum + r.comments.length,
0,
);
// `truncated` is reported by enumerateSpacePages: it is true ONLY when the
// stdio fallback BFS hit its node cap. The primary /pages/tree path is
// uncapped, so a space with legitimately many pages is not falsely flagged.
return {
since,
scope: parentPageId ? `subtree of ${parentPageId}` : `space ${spaceId}`,
checkedPages: pagesInScope.length,
pagesWithNewComments: results.length,
totalNewComments,
truncated,
comments: results,
};
}
// --- Image upload / embedding ---
/** Map a Content-Type string to a supported MIME type, or null if unsupported. */
}
return CommentsMixin;
}
+690
View File
@@ -0,0 +1,690 @@
// Shared client context + core seams (issue #450). The abstract base of the
// DocmostClient mixin chain: it owns ALL shared instance state (the axios
// client, apiUrl, auth tokens, the resolvePageId cache, the collab-token cache,
// the sandbox/metrics sinks) and the core HTTP/auth/pagination/write seams every
// domain module builds on. Domain modules are mixins layered on top; the final
// DocmostClient (client.ts) assembles them. Extracted VERBATIM from the original
// monolith — only field/seam visibility was widened from `private` to
// `protected` so sibling mixins can reach the shared state through `this`, and
// the cross-module methods that live in other mixins are declared `abstract`
// here so `this.<method>` type-checks. No behaviour changed.
import axios, { AxiosInstance } from "axios";
import FormData from "form-data";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import { acquireCollabSession } from "../lib/collab-session.js";
import { withPageLock, isUuid } from "../lib/page-lock.js";
import { getCollabToken, performLogin } from "../lib/auth-utils.js";
import { formatDocmostAxiosError } from "./errors.js";
// A generic mixin base constructor (issue #450). Each domain mixin is a factory
// `<T extends GConstructor<DocmostClientContext>>(Base: T) => class extends Base`
// so the mixins compose into one prototype chain sharing this context.
export type GConstructor<T = {}> = abstract new (...args: any[]) => T;
/**
* Configuration for a DocmostClient / MCP server instance. A discriminated
* union: either service-account credentials (email/password the client calls
* performLogin, powering the external /mcp HTTP endpoint and the stdio CLI) OR
* a token getter (getToken the client uses the returned BARE access JWT as
* the Bearer and never calls performLogin; used for the internal per-user path).
*
* Both branches may ALSO carry an optional `getCollabToken` provider. When set,
* content mutations (which go over the collaboration websocket) use the token it
* returns INSTEAD of calling `POST /auth/collab-token`. The internal per-user
* agent path uses this to hand the client a provenance collab token (signed
* `actor:'agent'`+`aiChatId`), so agent content edits are attributed without a
* spoofable client-side field. When absent the client keeps the original
* `/auth/collab-token` path (service-account/stdio unchanged).
*
* Housed here (not in index.ts) so client.ts has no type dependency on index.ts;
* index.ts re-exports it for the package's public surface.
*/
// Sink the stash tool writes blobs into. The host app binds this to its in-RAM
// SandboxStore and composes the public `uri` (the package never sees the store
// or any env). `put` returns the anonymous read URL plus integrity metadata.
export type SandboxPut = (
buf: Buffer,
mime: string,
) => { uri: string; sha256: string; size: number };
export type DocmostMcpConfig = { apiUrl: string } & (
| { email: string; password: string }
| { getToken: () => Promise<string> } // returns a BARE JWT; the client adds "Bearer "
) & {
// Optional collab-token provider (returns a ready collab JWT). Common to
// both branches; see the type doc above.
getCollabToken?: () => Promise<string>;
// Optional blob sandbox sink. Present only where the stash tool is wired;
// when absent, stashPage throws a clear "not configured" error. The
// optional `has`/`evict` probes let stashPage keep its mirror counts honest
// under the store's FIFO eviction (see stashPage); older sinks omit them.
sandbox?: {
put: SandboxPut;
has?: (uri: string) => boolean;
evict?: (uri: string) => void;
};
// Dependency-neutral metrics sink. When present, the client emits generic
// (name, value, labels) samples; the HOST maps those names onto its own
// metrics registry (the package never depends on prom-client or the server).
// Absent in standalone/stdio mode → the client is a complete no-op here.
onMetric?: (
name: string,
value: number,
labels?: Record<string, string>,
) => void;
};
/**
* Collab-token cache TTL in milliseconds (issue #435). Read fresh from the
* environment on every mint like collab-session.ts readConfig so tests and a
* live rollback can change it without reloading the module.
*
* Why a cache at all: the live CollabSession registry (#400/#431) keys sessions
* on (wsUrl, pageId, collabToken) for identity isolation (invariant 4). But BOTH
* collab-token sources mint a FRESH token per mutation the in-app provider
* re-signs a JWT whose iat/exp (seconds) changes every second, and the external
* MCP POSTs /auth/collab-token each call so the token in the key changed on
* every op and the session was almost never reused (connect-storms, 25s
* timeouts, zombie sessions). Caching the token per-client keeps the key stable
* across a burst of mutations so ONE session is reused.
*
* Default 5 min: well under the 24h collab-token lifetime AND <= the collab
* session max-age (10 min, MCP_COLLAB_SESSION_MAX_AGE_MS), so the
* permission-staleness window is not widened beyond what #431 already accepted.
* The rollback knob is an EXPLICIT 0 (or a negative number): that DISABLES the
* cache an exact fetch-per-call legacy path, mirroring how idleMs<=0 disables
* the session cache. Unset OR unparseable (e.g. a typo like "5min", "abc") falls
* back to the 5-min default with the cache ON parseInt yields NaN, which is
* treated as "not configured", not as "disabled". So to turn the cache off you
* must set the value to exactly 0, not to garbage.
*/
function readCollabTokenTtlMs(): number {
const raw = parseInt(process.env.MCP_COLLAB_TOKEN_TTL_MS ?? "", 10);
return Number.isFinite(raw) ? Math.max(0, raw) : 5 * 60 * 1000;
}
export abstract class DocmostClientContext {
protected client: AxiosInstance;
protected token: string | null = null;
protected apiUrl: string;
// email/password are only set on the service-account (credentials) variant;
// null on the getToken variant (where there are no credentials to log in with).
protected email: string | null = null;
protected password: string | null = null;
// Per-user token provider. When set, login() calls it to obtain a BARE access
// JWT instead of performLogin, and the 401/403 re-auth path re-calls it.
protected getTokenFn: (() => Promise<string>) | null = null;
// Optional collab-token provider. When set, getCollabTokenWithReauth() returns
// its token instead of calling POST /auth/collab-token; on a 401/403 it is
// re-invoked once. Used by the internal agent to carry signed provenance.
protected getCollabTokenFn: (() => Promise<string>) | null = null;
// Optional blob-sandbox sink for the stash tool. Null when not configured.
protected sandboxPut: SandboxPut | null = null;
// Optional probes paired with the sink. `has` lets stashPage detect a blob
// FIFO-evicted by a LATER put in the same stash; `evict` lets it free this
// op's image blobs if the final doc put throws. Null when the sink omits them.
protected sandboxHas: ((uri: string) => boolean) | null = null;
protected sandboxEvict: ((uri: string) => void) | null = null;
// Optional dependency-neutral metrics sink (see DocmostMcpConfig.onMetric).
// Null on the legacy positional form and whenever the host omits it → no-op.
protected onMetricFn:
| ((name: string, value: number, labels?: Record<string, string>) => void)
| null = null;
// In-flight login dedup: when the token expires, the 401 interceptor,
// ensureAuthenticated, getCollabTokenWithReauth and the two multipart retries
// can all call login() at once. Memoizing a single promise collapses that
// thundering herd into ONE /auth/login request that everyone awaits.
protected loginPromise: Promise<void> | null = null;
// Canonical-UUID cache for resolvePageId: maps an agent-supplied slugId to the
// page's canonical UUID, so repeated collab edits on the same page do not
// re-fetch /pages/info. A UUID input short-circuits before this cache (see
// resolvePageId), so only slugId->uuid entries are stored/read here.
protected pageIdCache = new Map<string, string>();
// Collab-token cache (issue #435): the last minted collab token plus the
// wall-clock time it was minted, so a burst of content mutations reuses ONE
// token and therefore ONE live CollabSession (whose registry key includes the
// token — #400 invariant 4). Per-instance: a DocmostClient is built per
// user/per chat request, so a cached token can never leak across identities.
// Reset whenever the client's identity changes (login() / this.token cleared);
// bypassed on a forced refresh (the 401/403 reauth path). null = no token yet.
protected collabTokenCache: { token: string; mintedAt: number } | null = null;
// Two construction forms:
// - new DocmostClient(config) // discriminated union (current)
// - new DocmostClient(baseURL, email, password) // legacy positional creds
// The positional form is retained so existing callers/tests keep working; it
// is exactly equivalent to the credentials branch of the object form.
constructor(config: DocmostMcpConfig);
constructor(baseURL: string, email: string, password: string);
constructor(
configOrBaseURL: DocmostMcpConfig | string,
email?: string,
password?: string,
) {
// Normalize the legacy positional form into the object union.
const config: DocmostMcpConfig =
typeof configOrBaseURL === "string"
? { apiUrl: configOrBaseURL, email: email!, password: password! }
: configOrBaseURL;
this.apiUrl = config.apiUrl;
if ("getToken" in config) {
// Token variant: carry the user's JWT via getToken; no credentials, so
// login() must never call performLogin (there is nothing to log in with).
this.getTokenFn = config.getToken;
} else {
// Service-account variant: behaves exactly as before (performLogin).
this.email = config.email;
this.password = config.password;
}
// Optional, available to both variants. When present, content mutations get
// their collab token from here instead of POST /auth/collab-token.
if (config.getCollabToken) {
this.getCollabTokenFn = config.getCollabToken;
}
if (config.sandbox) {
this.sandboxPut = config.sandbox.put;
this.sandboxHas = config.sandbox.has ?? null;
this.sandboxEvict = config.sandbox.evict ?? null;
}
// Legacy positional form carries no onMetric → null (complete no-op).
this.onMetricFn = config.onMetric ?? null;
this.client = axios.create({
baseURL: this.apiUrl,
// Default request timeout so a hung connection cannot wedge a per-page
// lock or block the server indefinitely. Multipart uploads override this
// with a longer per-request timeout.
timeout: 30000,
headers: {
"Content-Type": "application/json",
},
});
// Re-authenticate transparently on a 401/403 once: the JWT authToken can
// expire while the server is long-running, after which every cached-token
// request would otherwise fail until a manual restart. On such a response,
// clear the stale token, perform a fresh login, and replay the original
// request exactly once (guarded by config._retry to avoid infinite loops;
// the login request itself is never retried).
this.client.interceptors.response.use(
(response) => response,
async (error) => {
const config = error.config;
const status = error.response?.status;
const isAuthError = status === 401 || status === 403;
const isLoginRequest =
typeof config?.url === "string" && config.url.includes("/auth/login");
if (config && isAuthError && !config._retry && !isLoginRequest) {
config._retry = true;
// Drop the stale token + Authorization header before re-login. Also
// clear the collab-token cache (#435): a new identity/login must not
// keep serving a collab token minted under the old one.
this.token = null;
this.collabTokenCache = null;
delete this.client.defaults.headers.common["Authorization"];
try {
await this.login();
} catch (loginError) {
// Re-login failed: surface the original error to the caller.
return Promise.reject(error);
}
// Re-issue the original request with the freshly minted Bearer token.
// Read it from the default header that login() just set, not from
// this.token, to avoid a theoretical "Bearer null" if this.token was
// cleared between login() resolving and this point.
config.headers = config.headers || {};
config.headers["Authorization"] =
this.client.defaults.headers.common["Authorization"];
return this.client.request(config);
}
return Promise.reject(error);
},
);
// Diagnostics interceptor (issue #437). Registered AFTER the re-login
// interceptor so a successful re-login retry (which resolves to a real
// response) is never seen here as an error; only a genuine failure reaches
// this rejection handler. It reformats error.message IN PLACE (see
// formatDocmostAxiosError — kept as a mutation, not a custom Error class, so
// the surrounding axios.isAxiosError / error.response?.status / config._retry
// checks keep working) and re-rejects the SAME error. The _docmostFormatted
// flag makes a re-processed retry-failure a no-op.
this.client.interceptors.response.use(
(response) => response,
(error) => {
formatDocmostAxiosError(error);
return Promise.reject(error);
},
);
}
// --- Cross-module seams (issue #450) -----------------------------------
// A method in one domain mixin sometimes calls a PROTECTED method owned by
// another mixin (e.g. nodes-write -> validateDocUrls in doc-validate). Those
// callees are `protected`, so they cannot be surfaced through the public
// per-mixin interfaces. Declaring them here on the shared base lets `this.<m>`
// type-check across modules. Each is a stub that is ALWAYS overridden by the
// owning mixin (layered above this base in the chain), so the body never runs;
// it throws only to make an impossible mis-wiring loud instead of silent.
// (The PUBLIC cross-module callees — getPage, getPageJson, listComments,
// deleteComment, listPageHistory — arrive via the mixins' public interfaces,
// so they are not restated here.)
protected enumerateSpacePages(
_spaceId: string,
_rootPageId?: string,
): Promise<{ pages: any[]; truncated: boolean }> {
throw new Error("enumerateSpacePages not wired (missing ReadMixin)");
}
protected validateDocUrls(_node: any, _depth?: number): void {
throw new Error("validateDocUrls not wired (missing DocValidateMixin)");
}
protected validateDocStructure(_node: any, _depth?: number): void {
throw new Error("validateDocStructure not wired (missing DocValidateMixin)");
}
protected assertValidNodeShape(_op: string, _node: any): void {
throw new Error("assertValidNodeShape not wired (missing DocValidateMixin)");
}
protected fetchInternalFile(
_src: string,
): Promise<{ buffer: Buffer; mime: string }> {
throw new Error("fetchInternalFile not wired (missing StashMixin)");
}
protected uploadAttachmentBuffer(
_pageId: string,
_buffer: Buffer,
_fileName: string,
_mime: string,
): Promise<{ id: string; fileName: string; fileSize: number }> {
throw new Error("uploadAttachmentBuffer not wired (missing MediaMixin)");
}
protected fetchAttachmentText(_src: string): Promise<string> {
throw new Error("fetchAttachmentText not wired (missing MediaMixin)");
}
// PUBLIC cross-module callees. Declared here too (as always-overridden stubs)
// so a mixin calling e.g. `this.getPageJson` type-checks against the base —
// the mixin's own public interface only covers its own methods. The real
// implementations live in ReadMixin / CommentsMixin / PagesMixin and shadow
// these on the prototype chain.
getPage(_pageId: string): Promise<any> {
throw new Error("getPage not wired (missing ReadMixin)");
}
getPageJson(_pageId: string): Promise<any> {
throw new Error("getPageJson not wired (missing ReadMixin)");
}
listComments(_pageId: string, _includeResolved?: boolean): Promise<any> {
throw new Error("listComments not wired (missing CommentsMixin)");
}
deleteComment(_commentId: string): Promise<any> {
throw new Error("deleteComment not wired (missing CommentsMixin)");
}
listPageHistory(_pageId: string, _cursor?: string): Promise<any> {
throw new Error("listPageHistory not wired (missing PagesMixin)");
}
/** Application base URL (API URL without the /api suffix). */
get appUrl(): string {
return this.apiUrl.replace(/\/api\/?$/, "");
}
async login() {
// Reuse an in-flight login if one is already running so concurrent callers
// share a single token fetch instead of each issuing their own.
if (!this.loginPromise) {
// Token variant: re-fetch a BARE JWT via getToken() (there are no
// credentials to log in with — on a 401/403 the interceptor below calls
// login() again, which re-invokes getToken()). Credentials variant:
// performLogin against /auth/login exactly as before.
const fetchToken = this.getTokenFn
? this.getTokenFn()
: performLogin(this.apiUrl, this.email!, this.password!);
this.loginPromise = fetchToken
.then((token) => {
// Guard against an empty/invalid token (e.g. a getToken provider that
// resolves to "" or null): without this an empty token would set a
// literal "Authorization: Bearer null"/"Bearer " header and every
// request would 401 with a confusing error. Fail loudly instead.
if (typeof token !== "string" || token.length === 0) {
throw new Error("getToken returned an empty token");
}
this.token = token;
// Identity (re)established: drop any collab token minted under a
// previous identity so the #435 cache can never outlive it.
this.collabTokenCache = null;
this.client.defaults.headers.common["Authorization"] =
`Bearer ${token}`;
})
.finally(() => {
this.loginPromise = null;
});
}
return this.loginPromise;
}
async ensureAuthenticated() {
if (!this.token) {
await this.login();
}
}
/**
* Fetch a collaboration token, transparently re-authenticating once on a
* 401/403. getCollabToken() uses bare axios internally, so it is NOT covered
* by this.client's response interceptor; this helper replicates that
* behaviour for collab-token requests: ensure a token, try once, and on an
* expired-token auth error perform a fresh login and retry exactly once.
*
* Collab-token cache (issue #435): both sources the getCollabToken provider
* (in-app agent) AND the REST /auth/collab-token endpoint (external MCP) mint
* a FRESH token per call, whose string therefore changes every op. Since the
* live CollabSession registry keys on the token string (#400/#431 invariant 4),
* that churned the key and defeated session reuse. So we cache the last minted
* token per-client for readCollabTokenTtlMs() and hand it back for a burst of
* mutations, keeping the session key stable. `forceRefresh` bypasses the cache
* (the 401/403 reauth retry uses it, so the retry cannot be handed the same
* stale token that just failed otherwise reauth would be a no-op). TTL 0
* disables the cache: exact fetch-per-call legacy behaviour.
*/
protected async getCollabTokenWithReauth(
forceRefresh = false,
): Promise<string> {
const ttl = readCollabTokenTtlMs();
// Serve the cached collab token while it is still fresh (identity isolation
// is preserved: the cache is a per-instance field on a client built per
// user/per chat request, and it is cleared on every identity change).
if (
!forceRefresh &&
ttl > 0 &&
this.collabTokenCache &&
Date.now() - this.collabTokenCache.mintedAt < ttl
) {
return this.collabTokenCache.token;
}
// Collab-token PROVIDER path: when a getCollabToken provider was supplied
// (the internal agent's provenance collab token), use it instead of the
// REST /auth/collab-token endpoint. Re-invoke it once on a 401/403 (e.g. the
// signed token expired between content mutations in a long agent turn).
if (this.getCollabTokenFn) {
try {
const token = await this.getCollabTokenFn();
if (typeof token !== "string" || token.length === 0) {
throw new Error("getCollabToken returned an empty token");
}
return this.rememberCollabToken(token, ttl);
} catch (e) {
// On an auth error retry EXACTLY once, forcing a refresh so the retry
// re-invokes the provider (bypassing the cache) for a genuinely fresh
// token. `!forceRefresh` bounds it to a single retry (no loop).
if (this.isCollabAuthError(e) && !forceRefresh) {
return this.getCollabTokenWithReauth(true);
}
throw e;
}
}
await this.ensureAuthenticated();
try {
const token = await getCollabToken(this.apiUrl, this.token!);
return this.rememberCollabToken(token, ttl);
} catch (e) {
// getCollabToken wraps the AxiosError in a plain Error but attaches the
// HTTP status as `.status`, so isCollabAuthError detects an auth failure
// via either the raw AxiosError shape OR the attached status.
if (this.isCollabAuthError(e) && !forceRefresh) {
// Fresh login (which clears this.token AND the collab-token cache), then
// retry exactly once with the cache bypassed via forceRefresh.
await this.login();
return this.getCollabTokenWithReauth(true);
}
throw e;
}
}
/**
* Store a freshly minted collab token in the per-client cache (issue #435) and
* return it unchanged. No-op write when the cache is disabled (ttl<=0) or the
* token is empty, so a disabled cache is exact fetch-per-call legacy behaviour
* and a bad token is never cached.
*/
protected rememberCollabToken(token: string, ttl: number): string {
if (ttl > 0 && typeof token === "string" && token.length > 0) {
this.collabTokenCache = { token, mintedAt: Date.now() };
}
return token;
}
/**
* True when an error carries a 401/403 either as a raw AxiosError
* (`error.response.status`) or as the plain-Error `.status` that
* lib/auth-utils.getCollabToken attaches after wrapping the AxiosError.
*/
protected isCollabAuthError(e: unknown): boolean {
const axiosStatus = axios.isAxiosError(e) ? e.response?.status : undefined;
const attachedStatus = (e as any)?.status;
return (
axiosStatus === 401 ||
axiosStatus === 403 ||
attachedStatus === 401 ||
attachedStatus === 403
);
}
/**
* Connect to the collaboration websocket, read the live doc, apply
* `transform`, write the result, and wait for the server to persist it
* WITHOUT acquiring the per-page lock.
*
* This mirrors collaboration.mutatePageContent EXCEPT that it does not call
* withPageLock. It exists solely so replaceImage can hold ONE withPageLock
* across its scan -> upload -> write sequence: the per-page mutex is NOT
* reentrant, so calling the normal (self-locking) mutatePageContent inside an
* outer withPageLock for the same pageId would deadlock. The caller MUST hold
* the page lock for the whole operation; this helper assumes that invariant.
*
* `transform` receives the live ProseMirror doc and returns the NEW full doc
* to write, or `null` to abort with no write. Errors thrown by `transform`
* propagate to the caller.
*
* Resolves a `MutationResult { doc, verify }` mirroring mutatePageContent, so
* every content mutator (including replaceImage) can return a verifiable
* change report. The report is computed AFTER the atomic read->write and
* never throws.
*/
protected async mutateLiveContentUnlocked(
pageId: string,
collabToken: string,
transform: (liveDoc: any) => any | null,
): Promise<MutationResult> {
// Reuse a live CollabSession for the page (issue #400) instead of opening a
// fresh provider per op. acquireCollabSession does NOT take the per-page
// lock — the caller (replaceImage) already holds ONE withPageLock across its
// scan -> upload -> write sequence, and the mutex is not reentrant, so
// taking it here would deadlock. The synchronous read->write section and the
// unsyncedChanges/connectionLost ack logic live in CollabSession.mutate,
// preserved verbatim from the old inline machine (incl. the #152 structural
// diff that keeps a live editor's cursor anchored).
const session = await acquireCollabSession(pageId, collabToken, this.apiUrl, {
// Only the actual 25s collab connect timeout emits this — the connect-vs-
// unload signal; the other failure paths must NOT emit it.
onConnectTimeout: () =>
this.onMetricFn?.("collab_connect_timeouts_total", 1),
});
try {
return await session.mutate(transform);
} catch (e) {
// Drop the session on any failure so the next call reconnects fresh.
session.destroy("mutate failed");
throw e;
}
}
/**
* Generic pagination handler for Docmost API endpoints
*/
async paginateAll<T = any>(
endpoint: string,
basePayload: Record<string, any> = {},
limit: number = 100,
): Promise<T[]> {
await this.ensureAuthenticated();
const clampedLimit = Math.max(1, Math.min(100, limit));
// Hard ceiling on the number of pages to fetch: guards against a server
// that returns a perpetually-true hasNextPage (which would otherwise loop
// forever and accumulate duplicates).
const MAX_PAGES = 50;
let cursor: string | undefined;
let allItems: T[] = [];
let truncated = false;
for (let page = 0; page < MAX_PAGES; page++) {
const payload: Record<string, any> = {
...basePayload,
limit: clampedLimit,
};
if (cursor) payload.cursor = cursor;
const response = await this.client.post(endpoint, payload);
const data = response.data;
const items = data.data?.items || data.items || [];
const meta = data.data?.meta || data.meta;
allItems = allItems.concat(items);
// Advance strictly via the server-issued cursor. A missing nextCursor (or
// hasNextPage false) means we reached the end. A cursor identical to the
// one we just sent means the server did not understand our pagination
// param — stop instead of re-fetching page one forever and duplicating.
const next = meta?.hasNextPage ? meta?.nextCursor : null;
if (!next || next === cursor) {
// If the server still reports more pages but stopped issuing a usable
// cursor at the ceiling, flag the result as truncated below.
if (page === MAX_PAGES - 1 && meta?.hasNextPage) truncated = true;
break;
}
cursor = next;
// Reaching the ceiling with more pages still available means the result
// set is truncated.
if (page === MAX_PAGES - 1) truncated = true;
}
// If the loop stopped because it hit the MAX_PAGES ceiling while the server
// still reported more results, the result set is truncated — warn so the
// caller is not silently handed an incomplete list.
if (truncated) {
console.warn(
`paginateAll: results from "${endpoint}" truncated at the ${MAX_PAGES}-page cap; more pages exist on the server`,
);
}
return allItems;
}
/** Raw page info including the ProseMirror JSON content and slugId. */
async getPageRaw(pageId: string) {
await this.ensureAuthenticated();
const response = await this.client.post("/pages/info", { pageId });
return response.data?.data ?? response.data;
}
/**
* Resolve an agent-supplied pageId to the page's CANONICAL UUID (`page.id`),
* so every collaboration document the MCP opens is named `page.<uuid>` the
* SAME name the web editor always uses (`page.${page.id}`).
*
* The agent commonly passes a 10-char public slugId (from URLs/listings) as
* the pageId. The web editor opens the collab doc by UUID, but the MCP used to
* pass that slugId straight into the collab doc name (`page.<slugId>`). For one
* DB row that produced TWO independent Yjs documents whose debounced stores
* clobbered each other the agent's edit was silently lost (#260).
*
* A UUID input short-circuits with no network round-trip. A slugId is resolved
* once via getPageRaw and cached (both slugId->uuid and uuid->uuid), so
* repeated edits on the same page add no extra request.
*/
protected async resolvePageId(pageId: string): Promise<string> {
if (isUuid(pageId)) return pageId;
const cached = this.pageIdCache.get(pageId);
if (cached) return cached;
const data = await this.getPageRaw(pageId);
const uuid = data?.id;
if (typeof uuid !== "string" || !uuid) {
throw new Error(
`Could not resolve a canonical page id for "${pageId}"`,
);
}
this.pageIdCache.set(pageId, uuid);
return uuid;
}
/**
* Page-locked write seam over collaboration.mutatePageContent. Production just
* delegates; it exists as an overridable method so the insertFootnote wrapper
* (transform abort-on-not-found + response shaping) can be unit-tested without
* standing up a live Hocuspocus collab socket.
*
* SELF-RESOLVES the pageId to the canonical UUID (issue #449, "resolve-then-
* lock"): every write must lock and key its CollabSession by the UUID, never a
* raw slugId (#260). resolvePageId is cached/idempotent, so a caller that
* already resolved pays no extra round-trip; centralizing it here means a
* caller that reaches this seam with a raw slugId still locks correctly instead
* of silently splitting the mutex key. withPageLock also asserts the key is a
* UUID as a hard backstop.
*/
protected async mutatePage(
pageId: string,
collabToken: string,
apiUrl: string,
transform: (doc: any) => any,
): Promise<{ doc?: any; verify?: any }> {
const pageUuid = await this.resolvePageId(pageId);
return mutatePageContent(pageUuid, collabToken, apiUrl, transform);
}
/**
* Full-document write seam over collaboration.replacePageContent. Production
* just delegates; it exists as an overridable method so the full-doc write
* tools (updatePageJson, copyPageContent) can have their footnote-
* canonicalization binding unit-tested without a live Hocuspocus collab socket.
*
* SELF-RESOLVES the pageId to the canonical UUID (issue #449, "resolve-then-
* lock") for the same reason as mutatePage above the lock/CollabSession key
* is guaranteed canonical here, not left to the caller's discipline.
*/
protected async replacePage(
pageId: string,
doc: any,
collabToken: string,
apiUrl: string,
): Promise<{ doc?: any; verify?: any }> {
const pageUuid = await this.resolvePageId(pageId);
return replacePageContent(pageUuid, doc, collabToken, apiUrl);
}
/**
* Export a page to a single self-contained Docmost-flavoured markdown file:
* meta block + body (with inline comment anchors + diagrams) + comment
* threads. Lossless round-trip target; see importPageMarkdown for the inverse.
*/
}
+248
View File
@@ -0,0 +1,248 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Public method surface of DocValidateMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IDocValidateMixin` fails to compile on drift.
export interface IDocValidateMixin {
}
export function DocValidateMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IDocValidateMixin> & TBase {
abstract class DocValidateMixin extends Base implements IDocValidateMixin {
/**
* Validate a URL string against a scheme allowlist for a given context.
*
* The markdown link path enforces safe schemes via TipTap, but the raw
* JSON path (updatePageJson) bypasses that so this is the sanitization
* choke point for ProseMirror JSON written directly by the caller.
*
* - "link": reject javascript:, vbscript:, data: (any scheme that can
* execute or smuggle script when the href is clicked).
* - "src": allow only http(s):, mailto:, /api/files paths, or a
* scheme-less relative/absolute path; reject
* javascript:/vbscript:/data:/file:.
*/
protected isSafeUrl(url: unknown, context: "link" | "src"): boolean {
if (typeof url !== "string") return false;
const trimmed = url.trim();
if (trimmed === "") return true; // empty href/src is harmless
// Extract a leading "scheme:" if present. A scheme must start with a
// letter and contain only letters/digits/+/-/. before the colon. Strip
// whitespace and ASCII control chars first so a tab/newline embedded in
// the scheme cannot smuggle a dangerous scheme past the check.
const cleaned = trimmed.replace(/[\s\x00-\x1f]+/g, "");
const schemeMatch = /^([a-zA-Z][a-zA-Z0-9+.-]*):/.exec(cleaned);
const scheme = schemeMatch ? schemeMatch[1].toLowerCase() : null;
const dangerous = new Set(["javascript", "vbscript", "data", "file"]);
if (context === "link") {
if (scheme === null) return true; // relative/anchor link is fine
// For links, data: is also blocked (can carry script payloads).
return !new Set(["javascript", "vbscript", "data"]).has(scheme);
}
// context === "src"
if (scheme === null) return true; // relative/absolute path (incl. /api/files)
if (dangerous.has(scheme)) return false;
return scheme === "http" || scheme === "https" || scheme === "mailto";
}
/**
* Recursively walk a ProseMirror doc and reject any unsafe URL on a link
* mark href or on a media node's src/url. Media nodes covered: image,
* attachment, video, plus embed (rendered as an iframe), youtube, drawio
* and excalidraw all of which carry a user-controlled URL that Docmost
* renders. Throws a clear error on the first violation. A max-depth guard
* turns an over-deep document into a clean error instead of a RangeError
* stack overflow.
*/
protected validateDocUrls(node: any, depth: number = 0): void {
const MAX_DEPTH = 200;
if (depth > MAX_DEPTH) {
throw new Error(
`document nesting exceeds the maximum depth of ${MAX_DEPTH}`,
);
}
if (!node || typeof node !== "object") return;
// Link marks on text nodes: validate the href.
if (Array.isArray(node.marks)) {
for (const mark of node.marks) {
if (mark && mark.type === "link" && mark.attrs) {
if (!this.isSafeUrl(mark.attrs.href, "link")) {
throw new Error(`unsafe link href rejected: "${mark.attrs.href}"`);
}
}
}
}
// Media nodes: validate src/url against the stricter src allowlist.
// embed renders as an iframe (highest risk); youtube/drawio/excalidraw
// likewise carry a user-controlled URL Docmost renders, so they get the
// same scheme check as image/attachment/video.
if (
node.type === "image" ||
node.type === "attachment" ||
node.type === "video" ||
node.type === "embed" ||
node.type === "youtube" ||
node.type === "drawio" ||
node.type === "excalidraw" ||
node.type === "audio" ||
node.type === "pdf"
) {
const attrs = node.attrs || {};
for (const key of ["src", "url"]) {
if (attrs[key] != null && !this.isSafeUrl(attrs[key], "src")) {
throw new Error(
`unsafe ${node.type} ${key} rejected: "${attrs[key]}"`,
);
}
}
}
if (Array.isArray(node.content)) {
for (const child of node.content) {
this.validateDocUrls(child, depth + 1);
}
}
}
/**
* Recursively validate the STRUCTURE of a ProseMirror node (reuses the
* recursion shape of validateDocUrls). Every node must be an object with a
* string `type`; when present, `content` must be an array, `marks` must be
* an array of objects each with a string `type`, and a text node's `text`
* must be a string. Throws a clear "invalid ProseMirror document" error on
* the first violation. A max-depth guard turns an over-deep document into a
* clean error instead of a RangeError stack overflow.
*/
protected validateDocStructure(node: any, depth: number = 0): void {
const MAX_DEPTH = 200;
if (depth > MAX_DEPTH) {
throw new Error(
`invalid ProseMirror document: nesting exceeds the maximum depth of ${MAX_DEPTH}`,
);
}
if (!node || typeof node !== "object" || typeof node.type !== "string") {
throw new Error(
"invalid ProseMirror document: every node must be an object with a string `type`",
);
}
if (
"text" in node &&
node.type === "text" &&
typeof node.text !== "string"
) {
throw new Error(
"invalid ProseMirror document: a text node must have a string `text`",
);
}
if (node.marks !== undefined) {
if (!Array.isArray(node.marks)) {
throw new Error(
"invalid ProseMirror document: `marks` must be an array",
);
}
for (const mark of node.marks) {
if (
!mark ||
typeof mark !== "object" ||
typeof mark.type !== "string"
) {
throw new Error(
"invalid ProseMirror document: every mark must be an object with a string `type`",
);
}
}
}
if (node.content !== undefined) {
if (!Array.isArray(node.content)) {
throw new Error(
"invalid ProseMirror document: `content` must be an array when present",
);
}
for (const child of node.content) {
this.validateDocStructure(child, depth + 1);
}
}
}
/**
* Pre-write SHAPE gate (#409). Walk the WHOLE node tree with the shared
* `findInvalidNode` and throw a rich, path-anchored error the instant a nested
* node has an absent/unknown `type` (or an unknown mark) the exact shape that
* otherwise surfaces DEEP in the Yjs encode as the cryptic
* `Unknown node type: undefined`, but only AFTER a collab session was opened
* and a page lock taken. Calling this BEFORE `getCollabTokenWithReauth` /
* `mutatePageContent` fails fast: no collab connection, no lock, deterministic
* message. `op` names the tool for the message prefix (e.g. "patchNode").
*
* `findInvalidNode` derives its "known type" set from the very same
* `docmostExtensions` the encode path uses, so a node this gate accepts is one
* the encoder will accept too.
*/
protected assertValidNodeShape(op: string, node: any): void {
const bad = findInvalidNode(node);
if (bad) {
throw new Error(`${op}: invalid node — ${bad.summary}`);
}
}
/**
* Replace page content with a raw ProseMirror JSON document (lossless) and/or
* update its title. Both `doc` and `title` are optional, but at least one must
* be supplied:
* - `doc` provided -> validate + full-overwrite the body (and update the
* title too when `title` is also given).
* - `doc` omitted, `title` given -> title-only update; the body is NOT
* touched/resent (no collab write happens).
* - neither given -> throws (nothing to update).
*/
}
return DocValidateMixin;
}
+707
View File
@@ -0,0 +1,707 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import { parseCells as parseDrawioCells } from "../lib/drawio-xml.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import {
prepareModel,
decodeDrawioSvg,
buildDrawioSvg,
mxHash,
normalizeXml,
countUserCells,
} from "../lib/drawio-xml.js";
import { renderDiagramShapes } from "../lib/drawio-preview.js";
import { applyElkLayout } from "../lib/drawio-layout.js";
import {
buildFromGraph,
type Graph,
type LayoutMode as GraphLayoutMode,
} from "../lib/drawio-graph.js";
import { applyCellOps, type CellOp } from "../lib/drawio-cell-ops.js";
import { mermaidToGraph } from "../lib/drawio-mermaid.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Public method surface of DrawioMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IDrawioMixin` fails to compile on drift.
export interface IDrawioMixin {
drawioGet(pageId: string, node: string, format?: "xml" | "svg"): Promise<{ pageId: string; nodeId: string; format: "xml" | "svg"; content: string; meta: { attachmentId: string | null; title: string | null; width: number | null; height: number | null; cellCount: number; hash: string; }; }>;
drawioCreate(pageId: string, where: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }, xml: string, title?: string, layout?: "elk"): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; verify?: any; }>;
drawioUpdate(pageId: string, node: string, xml: string, baseHash: string, layout?: "elk"): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; verify?: any; }>;
drawioEditCells(pageId: string, node: string, operations: CellOp[], baseHash: string): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; verify?: any; }>;
drawioFromGraph(pageId: string, where: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }, graph: Graph, direction?: "LR" | "RL" | "TB" | "BT", preset?: string, layout?: GraphLayoutMode, node?: string): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; iconsResolved: number; iconsMissing: string[]; verify?: any; }>;
drawioFromMermaid(pageId: string, where: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }, mermaid: string, preset?: string): Promise<{ success: boolean; nodeId: string; attachmentId: string; warnings: string[]; iconsResolved: number; iconsMissing: string[]; verify?: any; }>;
}
export function DrawioMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IDrawioMixin> & TBase {
abstract class DrawioMixin extends Base implements IDrawioMixin {
/**
* Resolve a drawio node on a page by `attrs.id` or `#<index>` and return the
* node plus its ref. Throws a clear error if the ref does not resolve to a
* drawio node.
*/
protected async resolveDrawioNode(
pageId: string,
node: string,
): Promise<{ node: any; ref: string }> {
const data = await this.getPageRaw(pageId);
const hit = getNodeByRef(
data.content ?? { type: "doc", content: [] },
node,
);
if (!hit) {
throw new Error(
`drawio: no node found for "${node}" on page ${pageId} (use the drawio node's attrs.id or "#<index>" from getOutline)`,
);
}
if (hit.type !== "drawio") {
throw new Error(
`drawio: node "${node}" on page ${pageId} is a ${hit.type}, not a drawio diagram`,
);
}
return { node: hit.node, ref: node };
}
/**
* Read a drawio diagram as mxGraph XML (default) or as the raw `.drawio.svg`.
* Runs the decode chain (base64/entity content= drawio file nested XML or
* pako-inflated compressed <diagram>). The returned `hash` is the
* optimistic-lock key for drawioUpdate.
*/
async drawioGet(
pageId: string,
node: string,
format: "xml" | "svg" = "xml",
): Promise<{
pageId: string;
nodeId: string;
format: "xml" | "svg";
content: string;
meta: {
attachmentId: string | null;
title: string | null;
width: number | null;
height: number | null;
cellCount: number;
hash: string;
};
}> {
await this.ensureAuthenticated();
const { node: drawio } = await this.resolveDrawioNode(pageId, node);
const attrs = drawio.attrs || {};
const src = attrs.src;
if (!src) {
throw new Error(
`drawio: node "${node}" on page ${pageId} has no src to read`,
);
}
const svg = await this.fetchAttachmentText(src);
const modelXml = decodeDrawioSvg(svg);
const meta = {
attachmentId: attrs.attachmentId ?? null,
title: attrs.title ?? null,
width: attrs.width != null ? Number(attrs.width) : null,
height: attrs.height != null ? Number(attrs.height) : null,
cellCount: countUserCells(modelXml),
hash: mxHash(modelXml),
};
return {
pageId,
nodeId: attrs.id ?? node,
format,
content: format === "svg" ? svg : normalizeXml(modelXml),
meta,
};
}
/**
* Create a drawio diagram from mxGraph XML: lint schematic SVG preview
* (pure TS) build the `.drawio.svg` (createDrawioSvg contract) create the
* attachment insert a `drawio` node before/after an anchor or appended.
* `xml` is a bare `<mxGraphModel>` or a list of `<mxCell>` (the server wraps
* it and adds the id=0/id=1 sentinels).
*/
async drawioCreate(
pageId: string,
where: {
position: "before" | "after" | "append";
anchorNodeId?: string;
anchorText?: string;
},
xml: string,
title?: string,
layout?: "elk",
): Promise<{
success: boolean;
nodeId: string;
attachmentId: string;
warnings: string[];
verify?: any;
}> {
await this.ensureAuthenticated();
if (
!where ||
(where.position !== "before" &&
where.position !== "after" &&
where.position !== "append")
) {
throw new Error(
'drawioCreate: `where.position` must be one of "before", "after", "append"',
);
}
if (where.position === "before" || where.position === "after") {
const hasId =
typeof where.anchorNodeId === "string" && where.anchorNodeId.length > 0;
const hasText =
typeof where.anchorText === "string" && where.anchorText.length > 0;
if (hasId === hasText) {
throw new Error(
`drawioCreate: position "${where.position}" requires exactly one of anchorNodeId or anchorText`,
);
}
}
// Optional server-side ELK auto-layout: the model declares structure with
// rough coords, ELK computes the pixels (best-effort — returns the input
// unchanged on any layout failure).
const laidOutXml = layout === "elk" ? await applyElkLayout(xml) : xml;
// Pre-write pipeline (throws a structured DrawioLintError on any violation).
const prepared = prepareModel(laidOutXml);
const inner = renderDiagramShapes(prepared.cells, prepared.bbox);
const diagramTitle = title || "Page-1";
const svg = buildDrawioSvg(prepared.modelXml, inner, prepared.bbox, diagramTitle);
const att = await this.uploadAttachmentBuffer(
pageId,
Buffer.from(svg, "utf-8"),
"diagram.drawio.svg",
"image/svg+xml",
);
// NOTE: no `id` attribute is set here. The vendored `drawio` node schema
// (diagramAttributes) declares no `id`, so any block id would be silently
// dropped by PMNode.fromJSON on save and the returned handle would fail to
// resolve. The addressable handle is the node's "#<index>" (like image/table
// nodes), computed after the insert below.
const drawioNode: any = {
type: "drawio",
attrs: {
src: `/api/files/${att.id}/${att.fileName}`,
attachmentId: att.id,
width: prepared.bbox.width,
height: prepared.bbox.height,
align: "center",
},
};
if (title) drawioNode.attrs.title = title;
// Reuse the existing URL trust boundary (rejects unsafe src schemes).
this.validateDocUrls(drawioNode);
const collabToken = await this.getCollabTokenWithReauth();
const pageUuid = await this.resolvePageId(pageId);
let inserted = false;
let insertedIndex = -1;
const mutation = await this.mutatePage(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
inserted = false;
insertedIndex = -1;
const { doc: nd, inserted: ins } = insertNodeRelative(
liveDoc,
drawioNode,
where,
);
inserted = ins;
if (!inserted) return null; // anchor not found -> skip the write
// Locate the freshly-inserted node to derive its "#<index>" handle. The
// just-uploaded attachmentId is unique, so it identifies our node.
if (Array.isArray(nd.content)) {
insertedIndex = nd.content.findIndex(
(b: any) =>
b &&
b.type === "drawio" &&
b.attrs &&
b.attrs.attachmentId === att.id,
);
}
return nd;
},
);
if (!inserted) {
const anchorDesc = where.anchorNodeId
? `anchorNodeId "${where.anchorNodeId}"`
: `anchorText "${where.anchorText}"`;
throw new Error(
`drawioCreate: anchor not found (${anchorDesc}) on page ${pageId}. The diagram attachment ${att.id} is now an unreferenced orphan.`,
);
}
if (insertedIndex < 0) {
// The node was inserted nested (e.g. inside a callout/table cell via an
// anchor), where "#<index>" — which addresses only top-level blocks —
// cannot reference it. drawio nodes carry no persisted id, so there is no
// stable handle for a nested diagram.
throw new Error(
`drawioCreate: the diagram was inserted on page ${pageId} but not as a ` +
`top-level block, so it has no addressable "#<index>" handle. Anchor ` +
`on a top-level block (or append) so the diagram can be re-read.`,
);
}
// The returned handle is POSITIONAL ("#<index>"): valid for the immediate
// create -> get/update flow, but re-resolve via getOutline if the document
// structure changes (blocks added/removed before it shift the index).
const nodeId = `#${insertedIndex}`;
return {
success: true,
nodeId,
attachmentId: att.id,
warnings: prepared.warnings,
verify: mutation.verify,
};
}
/**
* Full-replacement update of a drawio diagram. `baseHash` is MANDATORY: it is
* compared against the hash of the diagram's CURRENT XML (from drawioGet);
* any mismatch means a human or another agent edited the diagram after the
* read, so the write is refused with a conflict error. On success the new
* `.drawio.svg` is uploaded as a FRESH attachment (in-place byte overwrite is
* avoided some Docmost versions corrupt an attachment on overwrite, exactly
* as replaceImage documents) and the node is repointed with new dimensions.
*/
async drawioUpdate(
pageId: string,
node: string,
xml: string,
baseHash: string,
layout?: "elk",
): Promise<{
success: boolean;
nodeId: string;
attachmentId: string;
warnings: string[];
verify?: any;
}> {
await this.ensureAuthenticated();
if (typeof baseHash !== "string" || baseHash.length === 0) {
throw new Error(
"drawioUpdate: baseHash is mandatory — read the diagram with drawioGet first and pass back its meta.hash",
);
}
// Resolve the node and read the CURRENT diagram to enforce the optimistic
// lock before doing any write or upload.
const { node: drawio, ref } = await this.resolveDrawioNode(pageId, node);
const oldAttrs = drawio.attrs || {};
const oldSrc = oldAttrs.src;
// The returned handle is the caller-supplied reference. drawio nodes carry
// no persisted id, so `ref` (an "#<index>" or a rare legacy attrs.id) is the
// honest identifier to hand back.
const nodeId = oldAttrs.id ?? ref;
if (!oldSrc) {
throw new Error(
`drawioUpdate: node "${node}" on page ${pageId} has no src to compare against`,
);
}
const currentSvg = await this.fetchAttachmentText(oldSrc);
const currentHash = mxHash(decodeDrawioSvg(currentSvg));
if (currentHash !== baseHash) {
throw new Error(
`drawioUpdate: conflict — the diagram changed since it was read ` +
`(baseHash ${baseHash} != current ${currentHash}). Re-read it with drawioGet and retry.`,
);
}
// Optional server-side ELK auto-layout (best-effort; see drawioCreate).
const laidOutXml = layout === "elk" ? await applyElkLayout(xml) : xml;
// Pipeline for the new content (throws a structured DrawioLintError).
const prepared = prepareModel(laidOutXml);
const inner = renderDiagramShapes(prepared.cells, prepared.bbox);
const diagramTitle = oldAttrs.title || "Page-1";
const svg = buildDrawioSvg(prepared.modelXml, inner, prepared.bbox, diagramTitle);
const att = await this.uploadAttachmentBuffer(
pageId,
Buffer.from(svg, "utf-8"),
"diagram.drawio.svg",
"image/svg+xml",
);
const newSrc = `/api/files/${att.id}/${att.fileName}`;
const collabToken = await this.getCollabTokenWithReauth();
const pageUuid = await this.resolvePageId(pageId);
let repointed = 0;
const repoint = (n: any) => {
n.attrs = {
...n.attrs,
src: newSrc,
attachmentId: att.id,
width: prepared.bbox.width,
height: prepared.bbox.height,
};
repointed++;
};
const mutation = await this.mutatePage(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
repointed = 0;
const doc =
liveDoc && liveDoc.type === "doc"
? liveDoc
: { type: "doc", content: [] };
if (!Array.isArray(doc.content)) doc.content = [];
// Repoint ONLY the resolved node — never every node that happens to
// share this attachmentId (a copied diagram is two nodes with one
// attachmentId; keying on it would clobber both). Re-resolve the same
// handle against the live doc and walk to its exact position.
const hit = getNodeByRef(doc, ref);
if (!hit || hit.type !== "drawio") return null; // vanished/changed -> skip
let target: any = doc;
for (const idx of hit.path) {
if (!target || !Array.isArray(target.content)) {
target = null;
break;
}
target = target.content[idx];
}
if (!target || target.type !== "drawio") return null;
repoint(target);
if (repointed === 0) return null; // node vanished concurrently -> skip
return doc;
},
);
if (repointed === 0) {
return {
success: true,
nodeId,
attachmentId: att.id,
warnings: [
...prepared.warnings,
"target drawio node was removed concurrently; uploaded attachment is unreferenced",
],
verify: mutation.verify,
};
}
return {
success: true,
nodeId,
attachmentId: att.id,
warnings: prepared.warnings,
verify: mutation.verify,
};
}
// --- draw.io high-level semantic tools (issue #425) ---
/**
* ID-based targeted edits of an existing drawio diagram (add / update / delete
* cells) instead of resending the whole XML. Reads the CURRENT diagram, checks
* the optimistic lock (`baseHash` is MANDATORY, exactly as drawioUpdate), applies
* the operations to the parsed model (a `delete` CASCADES to container children
* and to every edge whose source/target is deleted), then runs the SAME #423
* pipeline as drawioUpdate (lint + quality warnings -> preview -> attachment ->
* repoint the node). Ids are stable so diffs stay meaningful across edits.
*/
// --- draw.io high-level semantic tools (issue #425) ---
/**
* ID-based targeted edits of an existing drawio diagram (add / update / delete
* cells) instead of resending the whole XML. Reads the CURRENT diagram, checks
* the optimistic lock (`baseHash` is MANDATORY, exactly as drawioUpdate), applies
* the operations to the parsed model (a `delete` CASCADES to container children
* and to every edge whose source/target is deleted), then runs the SAME #423
* pipeline as drawioUpdate (lint + quality warnings -> preview -> attachment ->
* repoint the node). Ids are stable so diffs stay meaningful across edits.
*/
async drawioEditCells(
pageId: string,
node: string,
operations: CellOp[],
baseHash: string,
): Promise<{
success: boolean;
nodeId: string;
attachmentId: string;
warnings: string[];
verify?: any;
}> {
await this.ensureAuthenticated();
if (typeof baseHash !== "string" || baseHash.length === 0) {
throw new Error(
"drawioEditCells: baseHash is mandatory — read the diagram with drawioGet first and pass back its meta.hash",
);
}
if (!Array.isArray(operations) || operations.length === 0) {
throw new Error(
"drawioEditCells: operations must be a non-empty array of { op, ... }",
);
}
const { node: drawio, ref } = await this.resolveDrawioNode(pageId, node);
const oldAttrs = drawio.attrs || {};
const oldSrc = oldAttrs.src;
const nodeId = oldAttrs.id ?? ref;
if (!oldSrc) {
throw new Error(
`drawioEditCells: node "${node}" on page ${pageId} has no src to edit`,
);
}
const currentSvg = await this.fetchAttachmentText(oldSrc);
const currentModel = decodeDrawioSvg(currentSvg);
const currentHash = mxHash(currentModel);
if (currentHash !== baseHash) {
throw new Error(
`drawioEditCells: conflict — the diagram changed since it was read ` +
`(baseHash ${baseHash} != current ${currentHash}). Re-read it with drawioGet and retry.`,
);
}
// Apply the operations to the parsed model, then run the standard pipeline.
const editedModel = applyCellOps(currentModel, operations);
const prepared = prepareModel(editedModel);
const inner = renderDiagramShapes(prepared.cells, prepared.bbox);
const diagramTitle = oldAttrs.title || "Page-1";
const svg = buildDrawioSvg(prepared.modelXml, inner, prepared.bbox, diagramTitle);
const att = await this.uploadAttachmentBuffer(
pageId,
Buffer.from(svg, "utf-8"),
"diagram.drawio.svg",
"image/svg+xml",
);
const newSrc = `/api/files/${att.id}/${att.fileName}`;
const collabToken = await this.getCollabTokenWithReauth();
const pageUuid = await this.resolvePageId(pageId);
let repointed = 0;
const mutation = await this.mutatePage(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
repointed = 0;
const doc =
liveDoc && liveDoc.type === "doc" ? liveDoc : { type: "doc", content: [] };
if (!Array.isArray(doc.content)) doc.content = [];
const hit = getNodeByRef(doc, ref);
if (!hit || hit.type !== "drawio") return null;
let target: any = doc;
for (const idx of hit.path) {
if (!target || !Array.isArray(target.content)) {
target = null;
break;
}
target = target.content[idx];
}
if (!target || target.type !== "drawio") return null;
target.attrs = {
...target.attrs,
src: newSrc,
attachmentId: att.id,
width: prepared.bbox.width,
height: prepared.bbox.height,
};
repointed++;
return doc;
},
);
if (repointed === 0) {
return {
success: true,
nodeId,
attachmentId: att.id,
warnings: [
...prepared.warnings,
"target drawio node was removed concurrently; uploaded attachment is unreferenced",
],
verify: mutation.verify,
};
}
return {
success: true,
nodeId,
attachmentId: att.id,
warnings: prepared.warnings,
verify: mutation.verify,
};
}
/**
* The main high-level tool: build a diagram from a SEMANTIC graph (nodes with
* a `kind`/`icon`, groups, edges) the model never supplies coordinates or
* style strings. The server resolves icons via the shape catalog (#424),
* assigns palette colors from the preset, runs ELK layered layout (honouring
* `direction` and the `layer`/`sameLayerAs`/`pinned` hints and compound groups),
* and assembles linter-clean XML, then inserts it through the SAME create
* pipeline as drawioCreate. `layout:"incremental"` is only meaningful when a
* target `node` is given (it preserves that diagram's existing coordinates and
* places only new cells); on a fresh insert it behaves like "full".
*/
async drawioFromGraph(
pageId: string,
where: {
position: "before" | "after" | "append";
anchorNodeId?: string;
anchorText?: string;
},
graph: Graph,
direction?: "LR" | "RL" | "TB" | "BT",
preset?: string,
layout?: GraphLayoutMode,
node?: string,
): Promise<{
success: boolean;
nodeId: string;
attachmentId: string;
warnings: string[];
iconsResolved: number;
iconsMissing: string[];
verify?: any;
}> {
await this.ensureAuthenticated();
// Direction/preset supplied as separate params override the graph fields so
// both the flat tool schema and an inline graph can set them.
const merged: Graph = {
...graph,
direction: direction ?? graph.direction,
preset: preset ?? graph.preset,
};
const mode: GraphLayoutMode = layout ?? "full";
// Incremental into an EXISTING node: read its coords so ELK preserves them,
// and keep the full existing model so incremental MERGES (never drops) any
// cell the new graph doesn't re-list.
let existingCoords: Map<string, { x: number; y: number }> | undefined;
let existingModelXml: string | undefined;
let editExisting = false;
let baseHash: string | undefined;
if (node && (mode === "incremental" || mode === "none")) {
const { node: drawio } = await this.resolveDrawioNode(pageId, node);
const src = (drawio.attrs || {}).src;
if (src) {
const svg = await this.fetchAttachmentText(src);
const model = decodeDrawioSvg(svg);
baseHash = mxHash(model);
existingModelXml = model;
existingCoords = new Map();
for (const c of parseDrawioCells(model)) {
if (c.vertex && c.geometry.x != null && c.geometry.y != null) {
existingCoords.set(c.id, { x: c.geometry.x, y: c.geometry.y });
}
}
editExisting = true;
}
}
const built = await buildFromGraph(
merged,
mode,
existingCoords,
existingModelXml,
);
if (editExisting && node && baseHash) {
// Re-target the existing diagram: replace it with the assembled model.
const res = await this.drawioUpdate(pageId, node, built.modelXml, baseHash);
return {
...res,
iconsResolved: built.iconsResolved,
iconsMissing: built.iconsMissing,
};
}
const res = await this.drawioCreate(pageId, where, built.modelXml);
return {
...res,
iconsResolved: built.iconsResolved,
iconsMissing: built.iconsMissing,
};
}
/**
* Convert a Mermaid `flowchart` to a redactable draw.io diagram via a PURE
* parser (no Electron / draw.io CLI): mermaid text -> graph-JSON -> the
* drawioFromGraph pipeline. Only `flowchart`/`graph` is supported (the most
* common wiki case); other diagram types throw a clear error so the model can
* fall back to drawioFromGraph.
*/
async drawioFromMermaid(
pageId: string,
where: {
position: "before" | "after" | "append";
anchorNodeId?: string;
anchorText?: string;
},
mermaid: string,
preset?: string,
): Promise<{
success: boolean;
nodeId: string;
attachmentId: string;
warnings: string[];
iconsResolved: number;
iconsMissing: string[];
verify?: any;
}> {
await this.ensureAuthenticated();
const graph = mermaidToGraph(mermaid);
if (preset) graph.preset = preset;
return this.drawioFromGraph(pageId, where, graph, graph.direction, graph.preset);
}
// --- Page history / diff / transform ---
/**
* List the saved versions (history snapshots) of a page, newest first.
* Docmost auto-snapshots on every save. Returns one cursor-paginated page of
* results: `{ items, nextCursor }`. The history record's id field is `id`.
*/
}
return DrawioMixin;
}
+165
View File
@@ -0,0 +1,165 @@
// Central REST error diagnostics (issues #437 + #450). SINGLE place that maps an
// axios error to the model-facing message. Extracted verbatim from client.ts;
// the constructor's response interceptor (see client/context.ts) routes every
// REST call through formatDocmostAxiosError so the whole surface is uniform.
import axios from "axios";
// --- Issue #437: central error diagnostics -------------------------------
// The agent only ever sees the thrown exception's `error.message`, so a failed
// tool must return an ACTIONABLE message (method, path, status, and the
// server's own validation text) instead of the opaque "Request failed with
// status code 400". These helpers + the response interceptor in the
// constructor are the single authoritative place that text is composed.
// Overall cap on the composed diagnostic message so the model context stays
// compact and a (whitelisted) server string can never blow up the text.
const ERROR_MESSAGE_CAP = 300;
// Only attempt to JSON.parse an arraybuffer body under this size: a larger
// binary body is never a JSON error envelope, so parsing it just wastes memory
// (fetchInternalFile uses responseType:"arraybuffer", so a failed file fetch
// carries the JSON error envelope as raw bytes here).
const ERROR_BUFFER_PARSE_CAP = 4096;
// Canonical 36-char UUID (8-4-4-4-12 hex). Deliberately version/variant-
// AGNOSTIC: the ids are UUIDv7 (e.g. 019f499a-9f8c-7d68-...), so only the
// canonical shape/length is enforced, not the version/variant nibble.
const FULL_UUID_RE =
/^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
/**
* Throw an actionable error BEFORE any network call when `value` is not a full
* canonical UUID. Absorbs #436: a truncated/short comment id used to reach the
* server and bounce back as an opaque 400/404 the agent could not self-correct;
* failing fast here names the exact fix.
*/
export function assertFullUuid(
tool: string,
param: string,
value: string,
): void {
if (typeof value !== "string" || !FULL_UUID_RE.test(value)) {
throw new Error(
`${tool}: '${param}' must be the FULL comment UUID (36 chars, e.g. ` +
`019f499a-9f8c-7d68-b7be-ce100d7c6c56), got '${value}'. Copy the id ` +
`verbatim from listComments / createComment output.`,
);
}
}
// Keep ONLY the pathname of a request (no host, no query string, no fragment)
// so the message never leaks a host or query params. Resolves a relative
// config.url against config.baseURL, then discards everything but the path.
function requestPath(config: any): string {
const rawUrl = typeof config?.url === "string" ? config.url : "";
const base =
typeof config?.baseURL === "string" ? config.baseURL : undefined;
try {
// A dummy base makes an absolute config.url parse too; its host is dropped.
return new URL(rawUrl, base ?? "http://localhost").pathname;
} catch {
// Malformed url: still strip any query/fragment manually.
return rawUrl.split(/[?#]/)[0] || rawUrl;
}
}
/**
* Compose the server-facing message from `error.response.data`, using ONLY the
* whitelisted `message`/`error` fields or the HTTP statusText. SECURITY: the
* raw response body, headers (Authorization!) and config are NEVER read here
* a string/HTML body (e.g. a proxy's 502 page) is deliberately dropped in
* favour of the statusText.
*/
function extractServerMessage(data: any, statusText: string): string {
// class-validator envelope: { message: string | string[], error?: string }.
if (
data &&
typeof data === "object" &&
!Buffer.isBuffer(data) &&
!(data instanceof ArrayBuffer)
) {
const msg = (data as any).message;
if (Array.isArray(msg)) {
const joined = msg.filter((m) => typeof m === "string").join("; ");
if (joined) return joined;
} else if (typeof msg === "string" && msg) {
return msg;
}
const err = (data as any).error;
if (typeof err === "string" && err) return err;
return statusText;
}
// Buffer / ArrayBuffer body: attempt a size-capped, guarded JSON.parse so a
// failed arraybuffer fetch still surfaces the server's validation text.
if (Buffer.isBuffer(data) || data instanceof ArrayBuffer) {
const buf = Buffer.isBuffer(data) ? data : Buffer.from(data);
if (buf.length > 0 && buf.length <= ERROR_BUFFER_PARSE_CAP) {
try {
return extractServerMessage(JSON.parse(buf.toString("utf8")), statusText);
} catch {
return statusText;
}
}
return statusText;
}
// A raw string / HTML body is never surfaced (may echo server internals).
return statusText;
}
/**
* Reformat an AxiosError's `.message` IN PLACE into an actionable diagnostic:
* `<METHOD> <path> failed (<status> <statusText>): <serverMessage>`
* or, when the request never got a response:
* `<METHOD> <path> failed: <code> (no response from server)`.
*
* Mutates the SAME error object (never a custom subclass) so the live
* axios.isAxiosError / error.response?.status / config._retry checks around the
* client keep working, and sets `_docmostFormatted` as a double-processing
* guard. A no-op on a non-axios or already-formatted error.
*/
export function formatDocmostAxiosError(error: any): void {
if (!error || error._docmostFormatted) return;
if (!axios.isAxiosError(error)) return;
const config: any = error.config ?? {};
const method =
typeof config.method === "string" ? config.method.toUpperCase() : "";
const methodPath = `${method} ${requestPath(config)}`.trim();
const response = error.response;
let message: string;
if (response) {
const statusText =
typeof response.statusText === "string" ? response.statusText : "";
const serverMessage = extractServerMessage(response.data, statusText);
message = `${methodPath} failed (${response.status} ${statusText}): ${serverMessage}`;
// Full body only to stderr under DEBUG (parity with downloadImage).
if (process.env.DEBUG) {
console.error(
"Docmost request failed; response body:",
JSON.stringify(response.data),
);
}
} else {
// No response at all (ECONNREFUSED / ETIMEDOUT / ECONNRESET / DNS / timeout).
// Use ONLY error.code, never the raw error.message: axios network messages
// embed host:port ("connect ECONNREFUSED 127.0.0.1:3000", "getaddrinfo
// ENOTFOUND host") and #437's invariant is that the host never reaches the
// model-visible message. code is set for essentially every real no-response
// error (ECONNREFUSED/ETIMEDOUT/ECONNRESET/ENOTFOUND/ECONNABORTED); the full
// native message still goes to stderr under DEBUG.
const reason = error.code ?? "network error";
message = `${methodPath} failed: ${reason} (no response from server)`;
if (process.env.DEBUG) {
console.error("Docmost request failed; no response:", error.message);
}
}
if (message.length > ERROR_MESSAGE_CAP) {
message = message.slice(0, ERROR_MESSAGE_CAP - 1) + "…";
}
error.message = message;
(error as any)._docmostFormatted = true;
}
+730
View File
@@ -0,0 +1,730 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import FormData from "form-data";
import axios, { AxiosInstance } from "axios";
import { basename, extname } from "path";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import { withPageLock, isUuid } from "../lib/page-lock.js";
import { diffDocs, summarizeChange } from "../lib/diff.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Supported image types, kept as two lookup tables so both a local file
// extension and a remote Content-Type can be mapped to the same canonical set.
const EXT_TO_MIME: Record<string, string> = {
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".gif": "image/gif",
".webp": "image/webp",
".svg": "image/svg+xml",
};
const MIME_TO_EXT: Record<string, string> = {
"image/png": ".png",
"image/jpeg": ".jpg",
"image/gif": ".gif",
"image/webp": ".webp",
"image/svg+xml": ".svg",
};
// Public method surface of MediaMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IMediaMixin` fails to compile on drift.
export interface IMediaMixin {
uploadImage(pageId: string, url: string): any;
insertImage(pageId: string, url: string, opts?: { align?: "left" | "center" | "right"; alt?: string; replaceText?: string; afterText?: string; }): any;
replaceImage(pageId: string, oldAttachmentId: string, url: string, opts?: { align?: "left" | "center" | "right"; alt?: string }): any;
}
export function MediaMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IMediaMixin> & TBase {
abstract class MediaMixin extends Base implements IMediaMixin {
// --- Image upload / embedding ---
/** Map a Content-Type string to a supported MIME type, or null if unsupported. */
protected supportedImageMime(ct: string): string | null {
return MIME_TO_EXT[ct] ? ct : null;
}
/**
* Download a remote image from a caller-supplied URL and resolve its bytes,
* MIME and a filename.
*
* SSRF / RESOURCE TRUST BOUNDARY: the URL comes from the MCP caller and is
* fetched BY THE SERVER, so it must be guarded before and after the request.
* The guards mirror the local-file trust boundary in uploadImage:
* - scheme allowlist (http/https only) rejects file:, data:, ftp:, etc.,
* so the caller cannot use this path to read local files or other schemes;
* - a size cap enforced both via axios maxContentLength/maxBodyLength AND a
* post-download buffer.length re-check (defends against a missing/lying
* Content-Length), so a huge response cannot exhaust memory;
* - a 30s timeout. The timeout matters because replaceImage holds the
* per-page lock across this upload, so a hung download would wedge the
* lock for that page.
* We deliberately do NOT block private IP ranges: the MCP caller is already
* trusted to read arbitrary host files via the filePath path, so the marginal
* trust granted by fetching internal URLs is comparable, and blocking would
* break legitimate internal-image use.
*/
protected async fetchRemoteImage(
url: string,
maxBytes: number,
): Promise<{ buffer: Buffer; mime: string; fileName: string }> {
// Scheme allowlist first — cheapest guard, and rejects non-http(s) schemes
// (file:, data:, ftp:, ...) before any network request is made.
let parsed: URL;
try {
parsed = new URL(url);
} catch (e: any) {
throw new Error(`Invalid image URL "${url}": ${e.message}`);
}
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
throw new Error(
`unsupported image URL scheme "${parsed.protocol}"; only http and https are allowed`,
);
}
let response;
try {
response = await axios.get(url, {
responseType: "arraybuffer",
timeout: 30000,
maxContentLength: maxBytes,
maxBodyLength: maxBytes,
headers: { Accept: "image/*" },
});
} catch (error) {
// Keep the thrown message free of the raw response body (it may echo
// server internals); surface only status/statusText. The full body is
// logged under DEBUG for diagnostics.
if (axios.isAxiosError(error)) {
if (process.env.DEBUG) {
console.error(
"Image download failed; response body:",
JSON.stringify(error.response?.data),
);
}
throw new Error(
`Image download failed for "${url}": ${error.response?.status ?? ""} ${error.response?.statusText ?? error.message}`.trim(),
);
}
throw error;
}
// axios returns an ArrayBuffer for responseType: "arraybuffer".
const buffer = Buffer.from(response.data);
// Re-check the size: maxContentLength relies on Content-Length, which may be
// absent or lie, so guard against the actual byte count too.
if (buffer.length === 0) {
throw new Error(`Empty image response from "${url}"`);
}
if (buffer.length > maxBytes) {
throw new Error(
`Image too large: ${buffer.length} bytes exceeds the ${maxBytes}-byte cap`,
);
}
// Resolve MIME: prefer the response Content-Type (strip any "; charset=..."
// parameter, lowercase, trim) mapped through the supported set; if the
// header is generic/missing/unsupported, fall back to the URL path
// extension via the existing extension->MIME logic.
const rawCt = response.headers?.["content-type"];
let mime: string | null = null;
if (typeof rawCt === "string" && rawCt.length > 0) {
const ct = rawCt.split(";")[0].trim().toLowerCase();
mime = this.supportedImageMime(ct);
}
if (!mime) {
// Fall back to the URL path extension. Use the pathname so the query
// string never contaminates the extension lookup.
const ext = extname(parsed.pathname).toLowerCase();
mime = EXT_TO_MIME[ext] ?? null;
}
if (!mime) {
throw new Error(
`cannot determine supported image type for "${url}"; supported: png, jpg, jpeg, gif, webp, svg`,
);
}
// Build a filename from the URL path basename (ignore the query string),
// defaulting to "image" when empty, and ensure it ends with the canonical
// extension for the resolved MIME (append it when missing/mismatched).
const canonicalExt = MIME_TO_EXT[mime];
let fileName = basename(parsed.pathname) || "image";
if (extname(fileName).toLowerCase() !== canonicalExt) {
fileName += canonicalExt;
}
return { buffer, mime, fileName };
}
/** Build a Docmost ProseMirror image node from an uploaded attachment. */
protected buildImageNode(
att: { id: string; fileName: string; fileSize?: number },
align?: "left" | "center" | "right",
alt?: string,
): any {
// Clean file URL, matching Docmost's native behaviour. No cache-busting
// query: the server serves the bare URL correctly, and replacement creates
// a new attachment id (a new URL) which busts caches naturally.
const src = `/api/files/${att.id}/${att.fileName}`;
const node: any = {
type: "image",
attrs: {
src,
attachmentId: att.id,
// Default to null when the server omits fileSize so the attr is never
// undefined (undefined would be dropped on serialization / break the
// ProseMirror image schema which expects size present).
size: att.fileSize ?? null,
align: align || "center",
width: null,
},
};
if (alt) node.attrs.alt = alt;
return node;
}
/**
* Download a remote image from an http(s) URL and upload it as an attachment
* of a page, returning the attachment metadata plus a ready-to-insert
* ProseMirror image node. Local file paths are intentionally not supported:
* the MCP caller is a remote AI with no access to this server's filesystem.
*/
async uploadImage(pageId: string, url: string) {
await this.ensureAuthenticated();
const MAX_IMAGE_BYTES = 20 * 1024 * 1024; // 20 MiB
// Fetch + validate the remote image (scheme allowlist, size cap, timeout).
// See fetchRemoteImage for the SSRF / resource trust boundary.
const fetched = await this.fetchRemoteImage(url, MAX_IMAGE_BYTES);
const fileBuffer = fetched.buffer;
const mime = fetched.mime;
const fileName = fetched.fileName;
// Build a FRESH FormData for every send attempt. A FormData body is a
// single-use stream that is CONSUMED on the first send, so it cannot be
// replayed by this.client's response interceptor (replaying a consumed
// stream fails with 'socket hang up'). Multipart re-auth is therefore done
// here with bare axios and an explicit one-shot 401/403 retry that rebuilds
// the body. Field order matters: text fields must precede the file part so
// the server reads them; the server always generates a fresh attachment id.
const buildForm = () => {
const form = new FormData();
form.append("pageId", pageId);
form.append("file", fileBuffer, {
filename: fileName,
contentType: mime,
});
return form;
};
// Local name distinct from the `url` parameter (the source image URL): this
// is the /files/upload endpoint we POST the multipart body to.
const uploadUrl = `${this.apiUrl}/files/upload`;
let response;
try {
// Call buildForm() ONCE per attempt and reuse the instance for both
// getHeaders() and the body so the Content-Type boundary matches the body.
const form = buildForm();
// Read the Authorization header from this.client's defaults (set by
// login(), only ever deleted — never set to null) instead of building
// `Bearer ${this.token}`: a concurrent JSON 401 can null this.token
// mid-flight, which would otherwise produce a literal "Bearer null".
// ensureAuthenticated() above guarantees login() ran, so the default
// header exists here. A 60s timeout keeps a hung upload from wedging the
// per-page lock (replaceImage holds withPageLock across this call).
response = await axios.post(uploadUrl, form, {
headers: {
...form.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} catch (error) {
// On an expired-token auth error, re-login and retry exactly once with a
// freshly-rebuilt FormData (the previous one was already consumed).
if (
axios.isAxiosError(error) &&
(error.response?.status === 401 || error.response?.status === 403)
) {
await this.login();
const form2 = buildForm();
response = await axios.post(uploadUrl, form2, {
headers: {
...form2.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} else if (axios.isAxiosError(error)) {
// Keep the thrown message free of the raw response body (it may echo
// request data or server internals); surface only status/statusText.
// The full body is logged under DEBUG for diagnostics.
if (process.env.DEBUG) {
console.error(
"Image upload failed; response body:",
JSON.stringify(error.response?.data),
);
}
throw new Error(
`Image upload failed: ${error.response?.status} ${error.response?.statusText}`,
);
} else {
throw error;
}
}
// The attachment may arrive bare or wrapped in a { data } envelope.
const att = response.data?.data ?? response.data;
if (!att?.id || !att?.fileName) {
throw new Error(
"Unexpected /files/upload response: " + JSON.stringify(response.data),
);
}
// Some Docmost versions omit fileSize from the upload response. Fall back
// to the fetched byte length (the bytes we just uploaded) so callers never
// get an undefined size.
const resolvedSize = att.fileSize ?? fileBuffer.length;
return {
attachmentId: att.id,
fileName: att.fileName,
fileSize: resolvedSize,
src: `/api/files/${att.id}/${att.fileName}`,
imageNode: this.buildImageNode({ ...att, fileSize: resolvedSize }),
};
}
/**
* Upload an image from a web (http/https) URL and insert it into a page in
* one step.
* By default the image is appended at the end. With replaceText, the first
* top-level block whose text contains the string is replaced; with afterText,
* the image is inserted right after the first matching block. All other
* block ids are preserved (only one top-level block is added or swapped).
*/
async insertImage(
pageId: string,
url: string,
opts: {
align?: "left" | "center" | "right";
alt?: string;
replaceText?: string;
afterText?: string;
} = {},
) {
const up = await this.uploadImage(pageId, url);
// Reuse the node from uploadImage (clean /api/files/<id>/<file> src), then
// apply align/alt onto a shallow attrs copy.
const node: any = { ...up.imageNode, attrs: { ...up.imageNode.attrs } };
if (opts.align) node.attrs.align = opts.align;
if (opts.alt) node.attrs.alt = opts.alt;
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260). The
// uploadImage /files/upload call above keeps the agent-supplied id.
const pageUuid = await this.resolvePageId(pageId);
// Recursively collect the plain text of a top-level block.
const blockText = (n: any): string => {
let out = "";
if (n.type === "text") out += n.text || "";
for (const child of n.content || []) out += blockText(child);
return out;
};
// Insert into the LIVE synced document, not the debounced REST snapshot, so
// concurrent edits/comments/images are preserved and parallel insertImage
// calls (serialized by the per-page lock) each see the previous insertion.
let placement: "replaced" | "after" | "appended" | undefined;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
const doc =
liveDoc && liveDoc.type === "doc"
? liveDoc
: { type: "doc", content: [] };
if (!Array.isArray(doc.content)) doc.content = [];
if (opts.replaceText) {
// Ambiguity guard (mirrors editPageText): count matching top-level
// blocks first, so a non-unique fragment cannot silently replace the
// wrong block (e.g. text that also appears inside a callout/table).
const matches = doc.content.filter((b: any) =>
blockText(b).includes(opts.replaceText!),
);
if (matches.length === 0) {
throw new Error(`replaceText not found: "${opts.replaceText}"`);
}
if (matches.length > 1) {
throw new Error(
`replaceText "${opts.replaceText}" matches ${matches.length} blocks; use a longer unique fragment`,
);
}
const idx = doc.content.findIndex((b: any) =>
blockText(b).includes(opts.replaceText!),
);
// Data-loss guard: replaceText swaps the WHOLE top-level block, so if
// the fragment only appears nested inside a container (table, callout,
// list, blockquote) the entire structure would be destroyed. Refuse
// when the matched block is a container rather than a leaf
// paragraph/heading and point the caller at a safer tool.
const CONTAINER_TYPES = new Set([
"table",
"callout",
"bulletList",
"orderedList",
"taskList",
"blockquote",
]);
const matchedBlock = doc.content[idx];
if (matchedBlock && CONTAINER_TYPES.has(matchedBlock.type)) {
throw new Error(
`replaceText matched a ${matchedBlock.type} container block; replacing it would destroy the whole structure. ` +
`Use afterText to insert near it, or updatePageJson for surgical edits.`,
);
}
doc.content.splice(idx, 1, node);
placement = "replaced";
} else if (opts.afterText) {
// Ambiguity guard (mirrors editPageText): refuse a non-unique fragment.
const matches = doc.content.filter((b: any) =>
blockText(b).includes(opts.afterText!),
);
if (matches.length === 0) {
throw new Error(`afterText not found: "${opts.afterText}"`);
}
if (matches.length > 1) {
throw new Error(
`afterText "${opts.afterText}" matches ${matches.length} blocks; use a longer unique fragment`,
);
}
const idx = doc.content.findIndex((b: any) =>
blockText(b).includes(opts.afterText!),
);
doc.content.splice(idx + 1, 0, node);
placement = "after";
} else {
doc.content.push(node);
placement = "appended";
}
return doc;
},
);
return {
success: true,
pageId,
attachmentId: up.attachmentId,
src: up.src,
placement,
verify: mutation.verify,
};
}
/**
* Replace an existing image in a page with a new image fetched from a web
* (http/https) URL. Uploads the new file as a brand-new attachment, which
* yields a fresh clean URL that both renders correctly and busts browser
* caches (the URL changed). Finds every image node
* whose attrs.attachmentId === oldAttachmentId (recursively, incl. nodes nested
* in callouts/tables) and repoints its src/attachmentId/size, preserving
* comments, alignment and alt. Operates on the live collab document so comments
* and concurrent edits are preserved. Throws if no matching image is found.
*
* The OLD attachment is left in place as an unreferenced orphan: Docmost
* exposes NO HTTP API to delete a single content attachment (verified against
* the attachment controller/service and by probing the live API deletion
* happens only by cascade when the page, space or user is removed). This is the
* same outcome as Docmost's own editor when an image is removed/replaced.
* In-place byte overwrite is deliberately NOT used because some Docmost
* versions corrupt the attachment (HTTP 500) when its bytes are overwritten.
*/
async replaceImage(
pageId: string,
oldAttachmentId: string,
url: string,
opts: { align?: "left" | "center" | "right"; alt?: string } = {},
) {
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260). The
// page lock must ALSO key on the UUID so this operation serializes against
// other writes to the same page (mutatePageContent now locks by the resolved
// UUID too); locking by the raw slugId here would desync the mutex key and
// reopen the TOCTOU/orphan-attachment window the lock closes. uploadImage
// keeps the agent-supplied id (it hits REST, not the collab doc).
const pageUuid = await this.resolvePageId(pageId);
// Hold ONE per-page lock for the WHOLE operation (scan -> upload -> write).
// Previously the scan and the write were two separate mutatePageContent
// calls, each acquiring + releasing the lock, with the upload happening in
// the UNLOCKED gap between them. A concurrent op could interleave there: it
// could remove the target image so the write pass matches nothing, leaving
// the freshly-uploaded attachment as an un-deletable orphan (Docmost has no
// API to delete a single content attachment). Acquiring the lock once and
// using the non-locking collab helper inside (the per-page mutex is NOT
// reentrant, so the self-locking mutatePageContent would deadlock here)
// closes that TOCTOU window. uploadImage hits /files/upload over plain HTTP
// and does not touch the page lock, so it is safe to call while held.
return withPageLock(pageUuid, async () => {
// STEP 1: read-only live check. Scan the live document for any image node
// matching oldAttachmentId BEFORE uploading anything, so a wrong/stale id
// throws without ever creating an orphan attachment.
let matchFound = false;
const scan = (nodes: any[]) => {
for (const node of nodes) {
if (!node) continue;
if (
node.type === "image" &&
node.attrs &&
node.attrs.attachmentId === oldAttachmentId
) {
matchFound = true;
}
if (Array.isArray(node.content)) scan(node.content);
}
};
await this.mutateLiveContentUnlocked(pageUuid, collabToken, (liveDoc) => {
matchFound = false; // reset per-transform (collab may retry the read).
const doc =
liveDoc && liveDoc.type === "doc"
? liveDoc
: { type: "doc", content: [] };
if (Array.isArray(doc.content)) scan(doc.content);
return null; // read-only: never write on the check pass.
});
if (!matchFound) {
throw new Error(
`replaceImage: no image with attachmentId "${oldAttachmentId}" found on page ${pageId}`,
);
}
// STEP 2: a match exists — upload the new file as a FRESH attachment (new
// id, new clean URL) and repoint every matching node in a second pass.
// Still inside the SAME lock, so no other op can have changed the page
// since the scan.
const up = await this.uploadImage(pageId, url);
let replaced = 0;
// Swap the source of one image node, preserving align/alt/title/geometry.
const repoint = (node: any) => {
node.attrs = {
...node.attrs,
src: up.src,
attachmentId: up.attachmentId,
// Default to null when fileSize is unknown so the attr is never
// undefined.
size: up.fileSize ?? null,
};
if (opts.align) node.attrs.align = opts.align;
if (opts.alt !== undefined) node.attrs.alt = opts.alt;
replaced++;
};
// Recursively repoint every image node (incl. ones nested in callouts/tables).
const walk = (nodes: any[]) => {
for (const node of nodes) {
if (!node) continue;
if (
node.type === "image" &&
node.attrs &&
node.attrs.attachmentId === oldAttachmentId
) {
repoint(node);
}
if (Array.isArray(node.content)) walk(node.content);
}
};
const mutation = await this.mutateLiveContentUnlocked(
pageUuid,
collabToken,
(liveDoc) => {
// Reset per-transform so collab retries recompute cleanly (no double-count).
replaced = 0;
const doc =
liveDoc && liveDoc.type === "doc"
? liveDoc
: { type: "doc", content: [] };
if (!Array.isArray(doc.content)) doc.content = [];
walk(doc.content);
if (replaced === 0) return null; // no match -> skip the write entirely
return doc;
},
);
// KNOWN LIMITATION: a same-count image SRC swap (image count unchanged, no
// text/mark change) may still report verify.changed === false, because the
// text+marks+integrity-count model in summarizeChange does not inspect
// image `src`/attachmentId attributes. That is acceptable here — the
// replace is confirmed by `replaced` below, and verify is supplementary.
if (replaced === 0) {
// The pass-1 SCAN found the target (matchFound was true) and we already
// uploaded the new attachment, but pass-2 matched nothing — a concurrent
// editor must have removed the node between the two passes. Do NOT throw
// here (that would leak the just-uploaded attachment AND report failure);
// instead report success with the upload flagged as an unreferenced
// orphan so the caller knows. (The early throw above still covers the
// case where pass-1 finds nothing, before any upload happens.)
return {
success: true,
replaced: 0,
pageId,
oldAttachmentId,
newAttachmentId: up.attachmentId,
src: up.src,
orphanedAttachmentId: up.attachmentId,
warning:
"target image was removed concurrently; uploaded attachment is unreferenced",
verify: mutation.verify,
};
}
return {
success: true,
pageId,
replaced,
oldAttachmentId,
newAttachmentId: up.attachmentId,
src: up.src,
verify: mutation.verify,
};
});
}
// --- draw.io diagrams (issue #423) ---
/**
* Upload a ready-made byte buffer as a page attachment via the same
* multipart /files/upload endpoint uploadImage uses. Split out as its own
* (overridable) seam so drawioCreate/update can upload the generated
* `.drawio.svg` without going through the URL-fetch path, and so tests can
* stub the network. Mirrors uploadImage's fresh-FormData + one-shot 401/403
* re-auth handling (a FormData body is single-use, so it must be rebuilt per
* attempt).
*/
// --- draw.io diagrams (issue #423) ---
/**
* Upload a ready-made byte buffer as a page attachment via the same
* multipart /files/upload endpoint uploadImage uses. Split out as its own
* (overridable) seam so drawioCreate/update can upload the generated
* `.drawio.svg` without going through the URL-fetch path, and so tests can
* stub the network. Mirrors uploadImage's fresh-FormData + one-shot 401/403
* re-auth handling (a FormData body is single-use, so it must be rebuilt per
* attempt).
*/
protected async uploadAttachmentBuffer(
pageId: string,
buffer: Buffer,
fileName: string,
mime: string,
): Promise<{ id: string; fileName: string; fileSize: number }> {
await this.ensureAuthenticated();
const buildForm = () => {
const form = new FormData();
form.append("pageId", pageId);
form.append("file", buffer, { filename: fileName, contentType: mime });
return form;
};
const uploadUrl = `${this.apiUrl}/files/upload`;
let response;
try {
const form = buildForm();
response = await axios.post(uploadUrl, form, {
headers: {
...form.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} catch (error) {
if (
axios.isAxiosError(error) &&
(error.response?.status === 401 || error.response?.status === 403)
) {
await this.login();
const form2 = buildForm();
response = await axios.post(uploadUrl, form2, {
headers: {
...form2.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} else if (axios.isAxiosError(error)) {
if (process.env.DEBUG) {
console.error(
"Attachment upload failed; response body:",
JSON.stringify(error.response?.data),
);
}
throw new Error(
`Attachment upload failed: ${error.response?.status} ${error.response?.statusText}`,
);
} else {
throw error;
}
}
const att = response.data?.data ?? response.data;
if (!att?.id || !att?.fileName) {
throw new Error(
"Unexpected /files/upload response: " + JSON.stringify(response.data),
);
}
return {
id: att.id,
fileName: att.fileName,
fileSize: att.fileSize ?? buffer.length,
};
}
/**
* Fetch a stored `.drawio.svg` attachment as text. Overridable seam over
* fetchInternalFile (the authed loopback fetch, which also rejects any
* traversal/SSRF src) so drawioGet/update can read the current diagram and
* tests can stub the bytes.
*/
protected async fetchAttachmentText(src: string): Promise<string> {
const { buffer } = await this.fetchInternalFile(src);
return buffer.toString("utf-8");
}
/**
* Resolve a drawio node on a page by `attrs.id` or `#<index>` and return the
* node plus its ref. Throws a clear error if the ref does not resolve to a
* drawio node.
*/
}
return MediaMixin;
}
+700
View File
@@ -0,0 +1,700 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import {
importMarkdownFragment,
canBeDocChild,
findUnrepresentableTableAttrs,
} from "../lib/markdown-fragment.js";
import {
applyTextEdits,
TextEdit,
TextEditResult,
TextEditFailure,
} from "../lib/json-edit.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
import { normalizeAndMergeFootnotes } from "../lib/footnote-normalize-merge.js";
// Public method surface of NodesWriteMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements INodesWriteMixin` fails to compile on drift.
export interface INodesWriteMixin {
updatePageJson(pageId: string, doc?: any, title?: string): any;
editPageText(pageId: string, edits: TextEdit[]): any;
patchNode(pageId: string, nodeId: string, input: { markdown?: string; node?: any }): any;
insertNode(pageId: string, input: { markdown?: string; node?: any }, opts: { position: "before" | "after" | "append"; anchorNodeId?: string; anchorText?: string; }): any;
deleteNode(pageId: string, nodeId: string): any;
}
export function NodesWriteMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & INodesWriteMixin> & TBase {
abstract class NodesWriteMixin extends Base implements INodesWriteMixin {
/**
* Replace page content with a raw ProseMirror JSON document (lossless) and/or
* update its title. Both `doc` and `title` are optional, but at least one must
* be supplied:
* - `doc` provided -> validate + full-overwrite the body (and update the
* title too when `title` is also given).
* - `doc` omitted, `title` given -> title-only update; the body is NOT
* touched/resent (no collab write happens).
* - neither given -> throws (nothing to update).
*/
async updatePageJson(pageId: string, doc?: any, title?: string) {
await this.ensureAuthenticated();
// Title-only / no-op handling: when no document is supplied, do NOT write
// the body. Update the title if one was given; otherwise there is nothing
// to do, so fail loudly rather than silently no-op.
if (doc == null) {
if (!title) {
throw new Error(
"updatePageJson: nothing to update (provide content and/or title)",
);
}
await this.client.post("/pages/update", { pageId, title });
return {
success: true,
modified: true,
message: "Page title updated (content left unchanged).",
pageId,
};
}
// Validate the document shape before a full overwrite: a malformed doc
// would otherwise silently corrupt the page (full-overwrite is the
// documented behaviour; no optimistic-concurrency is applied here).
if (
typeof doc !== "object" ||
doc.type !== "doc" ||
!Array.isArray(doc.content)
) {
throw new Error(
'content must be a ProseMirror document ({"type":"doc","content":[...]}) ' +
"where content is an array of nodes each having a string `type`",
);
}
// Recurse the WHOLE document so a malformed nested node (e.g. a node with a
// non-string type, a non-array content/marks, or a text node missing its
// string text) is rejected up front rather than silently corrupting the
// page on overwrite.
this.validateDocStructure(doc);
// #409: beyond the string-`type` check above, reject a nested node whose
// `type` is a string but NOT a known Docmost schema node (a typo/unknown
// block) — the same `Unknown node type` the encoder throws — with a rich,
// path-anchored message, still BEFORE any collab connection.
this.assertValidNodeShape("updatePageJson", doc);
// Sanitize URLs before writing. This closes the JSON-path bypass: unlike
// the markdown link path (which TipTap sanitizes), raw JSON could otherwise
// inject javascript:/data: link hrefs or media srcs straight into the doc.
this.validateDocUrls(doc);
// Canonicalize footnotes (idempotent): an agent-authored JSON doc cannot
// leave footnotes out of order, orphaned, or in multiple lists — the bottom
// list + numbering are always derived from reference order. No-op when the
// footnotes are already canonical.
// #419: normalize + merge glyph-forked definitions before canonicalizing.
doc = normalizeAndMergeFootnotes(doc);
doc = canonicalizeFootnotes(doc);
// Write the BODY first, then the title (#159 split-brain): a failed body
// write (e.g. persist timeout) must not leave a new title over the old body.
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
const mutation = await this.replacePage(
pageUuid,
doc,
collabToken,
this.apiUrl,
);
// Body persisted successfully — now it is safe to set the title.
if (title) {
await this.client.post("/pages/update", { pageId, title });
}
return {
success: true,
modified: true,
message: "Page content replaced from ProseMirror JSON.",
pageId,
verify: mutation.verify,
};
}
/**
* AUTHOR-INLINE footnote insertion. The agent supplies only WHERE
* (`anchorText`, a snippet of body text to attach the marker after) and WHAT
* (`text`, the footnote content as markdown). Numbering and the bottom
* `footnotesList` are derived deterministically server-side
* (`insertInlineFootnote` -> `canonicalizeFootnotes`): the agent never sees,
* assigns, or edits a footnote number or the list, so it CANNOT desync.
*
* Content DEDUP: when an existing definition has the same content, its id is
* reused (one number, one definition, several references). The write is atomic
* via `mutatePageContent` (single-writer, page-locked); if the anchor text is
* not found the transform aborts with a clear error and no write happens.
*/
/**
* Surgical text edits: find/replace inside text nodes of the live
* document. Preserves all block ids, marks, callouts and tables.
*/
async editPageText(pageId: string, edits: TextEdit[]) {
await this.ensureAuthenticated();
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
// Apply the edits against the LIVE synced document, not the debounced REST
// snapshot, so concurrent human edits/comments are preserved. applyTextEdits
// records per-edit match problems in `failed` instead of throwing, and
// applies whatever it can; we abort the write only when nothing applied.
let results: TextEditResult[] | undefined;
let failed: TextEditFailure[] | undefined;
// Whether we actually wrote new content. Set inside the transform: a
// degenerate edit (e.g. find === replace, or a batch that nets to no change)
// can "apply" yet leave the document byte-for-byte identical, in which case
// we must NOT write (no spurious history version) and must not claim a write
// happened.
let wrote = false;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
wrote = false;
const r = applyTextEdits(liveDoc, edits);
results = r.results;
failed = r.failed;
// Nothing applied -> abort the write (mutatePageContent treats a null
// return from the transform as "write nothing").
if (r.results.length === 0) return null;
// Edits "applied" but produced an identical document: skip the write so
// no new history version is created. Stable structural comparison via
// JSON.stringify (both docs come from the same deep-copied source, so
// key order is stable).
if (JSON.stringify(r.doc) === JSON.stringify(liveDoc)) return null;
wrote = true;
return r.doc;
},
);
if ((results?.length ?? 0) === 0 && (failed?.length ?? 0) > 0) {
// No edit applied: surface an aggregated, actionable error so the caller
// does not mistake a no-op for a partial success.
throw new Error(
"editPageText: no edits were applied (nothing written). " +
failed!.map((f) => `"${f.find}": ${f.reason}`).join("; "),
);
}
// Edits matched but produced no content change (identical document): report
// a successful no-op — NOT a failure — and do not falsely claim a write.
if (!wrote) {
return {
success: true,
pageId,
applied: results,
failed,
message: "No changes written (edits produced identical content).",
verify: mutation.verify,
};
}
const result: any = {
success: true,
pageId,
applied: results,
failed,
message:
(failed?.length ?? 0)
? `Applied ${results?.length ?? 0} edit(s); ${failed!.length} failed (see failed[]). Node ids and formatting preserved.`
: "Text edits applied (node ids and formatting preserved).",
verify: mutation.verify,
};
// If any applied edit matched only after stripping markdown (the
// normalized fallback), warn that editPageText preserved existing marks
// and did NOT change formatting — so a caller who intended a formatting
// change is pointed at patchNode.
if (results?.some((r) => r.normalized === true)) {
result.warning =
"Some edits matched only after stripping markdown from your find string; " +
"editPageText preserved existing marks (it did not change bold/strike/etc.). " +
"If you intended a formatting change, use patchNode.";
}
return result;
}
/**
* Replace the block whose attrs.id === nodeId. Operates on the LIVE collab
* document so comments and concurrent edits are preserved.
*
* Exactly one of `input.markdown` / `input.node` (#413):
* - `markdown` (RECOMMENDED): the block is rewritten from a canonical markdown
* fragment. The fragment may import to N blocks (a "1 -> N" splice: rewrite a
* whole section in one call). The FIRST resulting block INHERITS the target's
* `attrs.id` (so an existing comment anchoring the block by id survives); the
* rest get FRESH ids. `^[...]` footnotes in the fragment are first-class:
* their definitions merge into the page's TAIL footnote list (content-key
* dedup + canonicalize), same machinery insertFootnote uses. REJECTED when
* the TARGET block carries a table-cell attribute markdown cannot represent
* (colspan/rowspan/colwidth/background) use the table tools or `node`.
* - `node`: a raw ProseMirror node for precise attr/mark work. The replacement
* keeps the target id (if `node.attrs.id` is missing it is set to nodeId).
*
* #159 ambiguous-id semantics are unchanged: 0 matches -> "no node"; >1 matches
* -> "ambiguous, refused" (nothing written), on BOTH paths the markdown path
* runs a dry `replaceNodeById` count first, so a duplicated id never splices.
*/
async patchNode(
pageId: string,
nodeId: string,
input: { markdown?: string; node?: any },
) {
await this.ensureAuthenticated();
// XOR: exactly one of markdown / node. Both optional in the schema; the
// runtime enforces the recommendation ("markdown for prose, node for fine
// work") without letting an ambiguous both-or-neither call through.
const hasMd =
input != null &&
typeof input.markdown === "string" &&
input.markdown.trim() !== "";
const hasNode = input != null && input.node != null;
if (hasMd === hasNode) {
throw new Error(
"patchNode: provide exactly one of `markdown` (recommended, for prose) " +
"or `node` (a raw ProseMirror node, for precise attr/mark work)",
);
}
if (hasMd) {
return this.patchNodeMarkdown(pageId, nodeId, input.markdown as string);
}
return this.patchNodeJson(pageId, nodeId, input.node);
}
/**
* patchNode with a raw ProseMirror `node` (the pre-#413 behavior). Replaces
* EVERY node whose attrs.id === nodeId; the swapped-in node keeps the target
* id. #159 ambiguity refused. Split out so the markdown path can reuse the
* shared collab/guard plumbing without a giant branch.
*/
protected async patchNodeJson(pageId: string, nodeId: string, node: any) {
if (!node || typeof node !== "object" || typeof node.type !== "string") {
throw new Error(
"patchNode: `node` must be an object with a string `type`",
);
}
// Preserve the block id WITHOUT mutating the caller's object: build a local
// copy whose attrs.id === nodeId (so the swapped-in node keeps the id of the
// node it replaces).
const target = {
...node,
attrs: {
...(node.attrs && typeof node.attrs === "object" ? node.attrs : {}),
},
};
if (target.attrs.id == null) {
target.attrs.id = nodeId;
}
// #409: fail fast on a malformed node SHAPE (a nested child with an
// absent/unknown `type`, e.g. a text leaf written as `{"text":"foo"}` with
// no `"type":"text"`) BEFORE opening a collab session or taking the page
// lock — the root-only `typeof node.type === "string"` check above never
// sees nested children, and the encoder's `Unknown node type: undefined`
// would otherwise only surface after the connection.
this.assertValidNodeShape("patchNode", target);
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
// Track the replacement count in an outer var, reset per-transform, so a
// collab retry recomputes it cleanly (mirrors replaceImage's pattern).
let replaced = 0;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
replaced = 0;
const { doc: nd, replaced: r } = replaceNodeById(
liveDoc,
nodeId,
target,
);
replaced = r;
// 0 matches -> skip the write. >1 matches -> the id is AMBIGUOUS: Docmost
// duplicates block ids on copy/paste (and copyPageContent writes them
// verbatim), so replacing "the node with id X" would silently clobber
// EVERY duplicate (#159). Refuse: skip the write and throw below so the
// model re-targets with a more specific anchor instead of corrupting the
// page. Only an unambiguous single match is written.
if (replaced !== 1) return null;
return nd;
},
);
// 0 -> "no node"; >1 -> "ambiguous, refused" (the transform already skipped
// the write for any count !== 1). Single shared guard (#159, #185 review).
assertUnambiguousMatch("patchNode", "replace", replaced, nodeId, pageId);
return { success: true, replaced, nodeId, verify: mutation.verify };
}
/**
* patchNode with a MARKDOWN fragment (#413). Imports the fragment through the
* canonical importer, then 1 -> N splices the resulting blocks in place of the
* target block on the LIVE collab doc:
* - the FIRST block inherits the target's id; the rest get FRESH ids (minted
* by the importer/id-remap, so neighbour blocks are untouched);
* - `^[...]` footnote definitions merge into the page's tail list;
* - REJECTED when the target block carries a markdown-unrepresentable table
* attr (colspan/rowspan/colwidth/background) guarding against silent loss;
* - #159 ambiguity is enforced by a dry `replaceNodeById` count BEFORE the
* splice, so a duplicated id never writes.
*/
protected async patchNodeMarkdown(
pageId: string,
nodeId: string,
markdown: string,
) {
// Import the fragment up front (network-free, canonical) so a bad fragment
// fails before any collab connection or page lock.
const { blocks, definitions } = await importMarkdownFragment(markdown);
// The first imported block inherits the target id; the rest keep the fresh
// ids the importer assigned. Build the thread now so it is stable across a
// collab retry (the transform below is pure over its inputs).
const threaded = blocks.map((b, i) => {
if (i !== 0) return b;
return {
...b,
attrs: {
...(b && typeof b.attrs === "object" ? b.attrs : {}),
id: nodeId,
},
};
});
// Shape-validate every imported block up front (parity with the JSON path):
// the importer only emits schema nodes, but the check is cheap insurance and
// yields the same rich #409 diagnostics if the schema ever drifts.
for (const b of threaded) {
this.assertValidNodeShape("patchNode", b);
}
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
let replaced = 0;
let guardAttrs: string | null = null;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
replaced = 0;
guardAttrs = null;
// #159: count matches with the same recursive walk the JSON path uses;
// only an UNAMBIGUOUS single match may write. A dry count keeps the
// ambiguity semantics identical across both paths.
const { replaced: count } = replaceNodeById(liveDoc, nodeId, {
type: "paragraph",
});
replaced = count;
if (count !== 1) return null;
// Guard against SILENT LOSS: if the target block carries a table-cell
// attribute markdown cannot represent (colspan/rowspan/colwidth/
// background), refuse the markdown rewrite so those attrs are not
// dropped. Simple tables (no such attrs) rewrite fine.
const hit = getNodeByRef(liveDoc, nodeId);
guardAttrs = hit ? findUnrepresentableTableAttrs(hit.node) : null;
if (guardAttrs != null) return null;
// Re-mint any minted block id that collides with an existing page id
// (skip index 0: its id is intentionally the target nodeId, unique by
// the #159 dry-count above), so the 1 -> N splice stays page-wide unique.
reassignCollidingBlockIds(liveDoc, threaded, 0);
// 1 -> N splice, then merge any fragment footnote definitions into the
// page's tail list and re-derive canonical footnote numbering.
const { doc: spliced } = replaceNodeByIdWithMany(
liveDoc,
nodeId,
threaded,
);
return mergeFootnoteDefinitions(spliced, definitions);
},
);
// Surface the guard rejection with an actionable message (nothing written).
if (guardAttrs != null) {
throw new Error(
`patchNode: the target block has table-cell attributes markdown cannot ` +
`represent (${guardAttrs}) — a markdown rewrite would drop them. Use ` +
`the table tools (tableUpdateCell/tableInsertRow) or pass a raw ` +
`ProseMirror \`node\` instead of \`markdown\`.`,
);
}
// 0 -> "no node"; >1 -> "ambiguous, refused" (the transform skipped the write
// for any count !== 1). Shared #159 guard, identical to the JSON path.
assertUnambiguousMatch("patchNode", "replace", replaced, nodeId, pageId);
return {
success: true,
replaced,
nodeId,
blocks: threaded.length,
verify: mutation.verify,
};
}
/**
* Insert content relative to an anchor (or append it at the top level).
* Operates on the LIVE collab document so comments and concurrent edits are
* preserved.
*
* Exactly one of `input.markdown` / `input.node` (#413):
* - `markdown` (RECOMMENDED): a canonical markdown fragment. It may import to
* SEVERAL blocks they are inserted IN ORDER at the anchor. `^[...]`
* footnote definitions merge into the page's tail list (same machinery as
* insertFootnote). Every inserted block gets a fresh id.
* - `node`: a raw ProseMirror node for precise attr/mark work, or to insert
* table structure (a bare tableRow/tableCell/tableHeader NOT expressible in
* markdown, so those stay JSON-only).
*
* opts.position:
* - "append": push the content at the end of the top-level content.
* - "before"/"after": insert as a sibling of the anchor, just before/after it.
* Exactly one of anchorNodeId / anchorText must be given; anchorNodeId
* locates a node anywhere by attrs.id, anchorText matches the first top-level
* block whose plain text includes it.
*
* Throws if the anchor cannot be found.
*/
async insertNode(
pageId: string,
input: { markdown?: string; node?: any },
opts: {
position: "before" | "after" | "append";
anchorNodeId?: string;
anchorText?: string;
},
) {
await this.ensureAuthenticated();
// XOR: exactly one of markdown / node (both optional in the schema).
const hasMd =
input != null &&
typeof input.markdown === "string" &&
input.markdown.trim() !== "";
const hasNode = input != null && input.node != null;
if (hasMd === hasNode) {
throw new Error(
"insertNode: provide exactly one of `markdown` (recommended, for prose) " +
"or `node` (a raw ProseMirror node, for precise attr/mark work or table structure)",
);
}
if (
!opts ||
(opts.position !== "before" &&
opts.position !== "after" &&
opts.position !== "append")
) {
throw new Error(
'insertNode: `position` must be one of "before", "after", "append"',
);
}
if (opts.position === "before" || opts.position === "after") {
// before/after require EXACTLY ONE anchor (an id or a text fragment).
const hasId =
typeof opts.anchorNodeId === "string" && opts.anchorNodeId.length > 0;
const hasText =
typeof opts.anchorText === "string" && opts.anchorText.length > 0;
if (hasId === hasText) {
throw new Error(
`insertNode: position "${opts.position}" requires exactly one of anchorNodeId or anchorText`,
);
}
}
// Resolve the ordered list of blocks to insert plus any footnote definitions
// to merge. The markdown path imports canonically (so an inserted block is
// byte-identical to the same content in a full-page import); the node path is
// a single block with no footnote merge (raw JSON `^[...]` is not touched).
let blocks: any[];
let definitions: any[] = [];
if (hasMd) {
const frag = await importMarkdownFragment(input.markdown as string);
blocks = frag.blocks;
definitions = frag.definitions;
} else {
const node = input.node;
if (!node || typeof node !== "object" || typeof node.type !== "string") {
throw new Error(
"insertNode: `node` must be an object with a string `type`",
);
}
blocks = [node];
}
// #409: fail fast on a malformed node SHAPE (a nested child with an
// absent/unknown `type`) BEFORE opening a collab session or taking the page
// lock — the root-only check above never sees nested children.
for (const b of blocks) {
this.assertValidNodeShape("insertNode", b);
}
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
// Track insertion in an outer var, reset per-transform, so a collab retry
// recomputes it cleanly (mirrors replaceImage's pattern).
let inserted = false;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
inserted = false;
// Re-mint any minted block id that collides with an existing page id
// (all inserted blocks are fresh, no skip) so the splice stays unique.
if (hasMd) reassignCollidingBlockIds(liveDoc, blocks);
// Single-block node path keeps `insertNodeRelative` (it owns the
// structural table-node splicing); the markdown path uses the array
// splice so N blocks land in order at one anchor.
const res = hasMd
? insertNodesRelative(liveDoc, blocks, opts)
: insertNodeRelative(liveDoc, blocks[0], opts);
inserted = res.inserted;
if (!inserted) return null; // anchor not found -> skip the write entirely
// Merge any fragment footnote definitions into the page tail list and
// re-derive canonical numbering (no-op when there are none).
return mergeFootnoteDefinitions(res.doc, definitions);
},
);
if (!inserted) {
const anchorDesc = opts.anchorNodeId
? `anchorNodeId "${opts.anchorNodeId}"`
: `anchorText "${opts.anchorText}"`;
// anchorText is matched against the block's literal RENDERED plain text;
// markdown/emoji are tolerated only as a strip-and-retry fallback, so a
// miss usually means the text differs from what's on the page.
const hint = opts.anchorText
? " anchorText must be the block's literal rendered plain text (no markdown wrappers or emoji); anchorNodeId from getPageJson is more reliable."
: "";
throw new Error(
`insertNode: anchor not found (${anchorDesc}) on page ${pageId}.${hint}`,
);
}
return {
success: true,
inserted: true,
position: opts.position,
blocks: blocks.length,
verify: mutation.verify,
};
}
/**
* Remove EVERY node whose attrs.id === nodeId (recursively, including nodes
* nested in callouts/tables) from its parent content array. Operates on the
* LIVE collab document so comments and concurrent edits are preserved.
* Throws if no node matches.
*/
async deleteNode(pageId: string, nodeId: string) {
await this.ensureAuthenticated();
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
// Track the deletion count in an outer var, reset per-transform, so a
// collab retry recomputes it cleanly (mirrors replaceImage's pattern).
let deleted = 0;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
deleted = 0;
const { doc: nd, deleted: d } = deleteNodeById(liveDoc, nodeId);
deleted = d;
// 0 matches -> skip the write. >1 matches -> the id is AMBIGUOUS (block
// ids are duplicated on copy/paste, #159): deleting "the node with id X"
// would silently remove EVERY duplicate. Refuse: skip the write and throw
// below so the model re-targets. Only an unambiguous single match is
// deleted.
if (deleted !== 1) return null;
return nd;
},
);
// 0 -> "no node"; >1 -> "ambiguous, refused" (the transform already skipped
// the write for any count !== 1). Single shared guard (#159, #185 review).
assertUnambiguousMatch("deleteNode", "delete", deleted, nodeId, pageId);
return { success: true, deleted, nodeId, verify: mutation.verify };
}
/** Build the public share URL for a page. */
}
return NodesWriteMixin;
}
+655
View File
@@ -0,0 +1,655 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import FormData from "form-data";
import axios, { AxiosInstance } from "axios";
import { convertProseMirrorToMarkdown } from "../lib/markdown-converter.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import { footnoteWarningsField } from "../lib/footnote-analyze.js";
import {
serializeDocmostMarkdown,
parseDocmostMarkdown,
} from "../lib/markdown-document.js";
import { diffDocs, summarizeChange } from "../lib/diff.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
import { normalizeAndMergeFootnotes } from "../lib/footnote-normalize-merge.js";
import vm from "node:vm";
// Public method surface of PagesMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IPagesMixin` fails to compile on drift.
export interface IPagesMixin {
createPage(title: string, content: string, spaceId: string, parentPageId?: string): any;
updatePage(pageId: string, content: string, title?: string): any;
renamePage(pageId: string, title: string): any;
movePage(pageId: string, parentPageId: string | null, position?: string): any;
deletePage(pageId: string): any;
sharePage(pageId: string, searchIndexing?: boolean): any;
listShares(): any;
unsharePage(pageId: string): any;
exportPageMarkdown(pageId: string): Promise<string>;
importPageMarkdown(pageId: string, fullMarkdown: string): Promise<any>;
copyPageContent(sourcePageId: string, targetPageId: string): any;
listPageHistory(pageId: string, cursor?: string): any;
getPageHistory(historyId: string): any;
restorePageVersion(historyId: string): any;
diffPageVersions(pageId: string, from?: string, to?: string): any;
}
export function PagesMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IPagesMixin> & TBase {
abstract class PagesMixin extends Base implements IPagesMixin {
/**
* Create a new page with title and content.
* Uses the /pages/import workaround (the only endpoint accepting content),
* then moves the page and restores the exact title: the import endpoint
* derives the title from the FILENAME and replaces spaces with
* underscores, so we explicitly re-set it via /pages/update afterwards.
*/
async createPage(
title: string,
content: string,
spaceId: string,
parentPageId?: string,
) {
await this.ensureAuthenticated();
if (parentPageId) {
try {
await this.getPage(parentPageId);
} catch (e) {
throw new Error(`Parent page with ID ${parentPageId} not found.`);
}
}
// 1. Create content via Import (using multipart/form-data).
// Build a FRESH FormData per send attempt: a FormData body is a single-use
// stream consumed on the first send, so it cannot be replayed by
// this.client's response interceptor (replay fails with 'socket hang up').
// Multipart re-auth is therefore done here with bare axios and an explicit
// one-shot 401/403 retry that rebuilds the body.
const fileContent = Buffer.from(content, "utf-8");
const buildForm = () => {
const form = new FormData();
form.append("spaceId", spaceId);
form.append("file", fileContent, {
filename: `${title || "import"}.md`,
contentType: "text/markdown",
});
return form;
};
const importUrl = `${this.apiUrl}/pages/import`;
let response;
try {
// Call buildForm() ONCE per attempt and reuse the instance for both
// getHeaders() and the body so the Content-Type boundary matches the body.
const form = buildForm();
// Read the Authorization header from this.client's defaults (set by
// login(), only ever deleted — never set to null) instead of building
// `Bearer ${this.token}`: a concurrent JSON 401 can null this.token
// mid-flight, which would otherwise produce a literal "Bearer null".
// ensureAuthenticated() above guarantees login() ran, so the default
// header exists here.
response = await axios.post(importUrl, form, {
headers: {
...form.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} catch (error) {
// On an expired-token auth error, re-login and retry exactly once with a
// freshly-rebuilt FormData (the previous one was already consumed).
if (
axios.isAxiosError(error) &&
(error.response?.status === 401 || error.response?.status === 403)
) {
await this.login();
const form2 = buildForm();
response = await axios.post(importUrl, form2, {
headers: {
...form2.getHeaders(),
Authorization: this.client.defaults.headers.common["Authorization"],
},
timeout: 60000,
});
} else {
throw error;
}
}
const newPageId = (response.data?.data ?? response.data).id;
// 2. Move to parent if needed
if (parentPageId) {
await this.movePage(newPageId, parentPageId);
}
// 3. Restore the exact title (import mangles spaces into underscores)
if (title) {
await this.client.post("/pages/update", { pageId: newPageId, title });
}
const page = await this.getPage(newPageId);
// Surface non-fatal footnote problems (dangling refs, empty/duplicate
// definitions, markers in tables) so the agent can fix its markup (#166).
return { ...page, ...footnoteWarningsField(content) };
}
/**
* Update a page's content from markdown and optionally its title.
* NOTE: full re-import block ids regenerate. For surgical changes
* use editPageText / updatePageJson instead.
*/
async updatePage(pageId: string, content: string, title?: string) {
await this.ensureAuthenticated();
// Open the collab doc by the canonical UUID, never the slugId (#260). The
// REST /pages/update title write below keeps the agent-supplied id (the
// server resolves a slugId there).
const pageUuid = await this.resolvePageId(pageId);
// Write the BODY first, then the title (#159 split-brain). If the collab
// body write fails (e.g. a persist timeout), the title must be left
// UNTOUCHED so the page never ends up with a new title over its old body.
// A title write failing AFTER a successful body is rarer (REST is fast) and
// leaves correct content under a stale title — the lesser inconsistency.
let collabToken = "";
let mutation;
try {
collabToken = await this.getCollabTokenWithReauth();
mutation = await updatePageContentRealtime(
pageUuid,
content,
collabToken,
this.apiUrl,
);
} catch (error: any) {
// Verbose diagnostics (incl. anything that could expose a token prefix)
// are gated behind DEBUG; the thrown Error below carries no token data.
if (process.env.DEBUG) {
console.error(
"Failed to update page content via realtime collaboration:",
error,
);
const tokenPreview = collabToken
? collabToken.substring(0, 15) + "..."
: "null";
console.error(`Collab token preview: ${tokenPreview}`);
}
throw new Error(`Failed to update page content: ${error.message}`);
}
// Body persisted successfully — now it is safe to set the title.
if (title) {
await this.client.post("/pages/update", { pageId, title });
}
return {
success: true,
modified: true,
message: "Page updated successfully.",
pageId: pageId,
verify: mutation.verify,
// Non-fatal footnote diagnostics (#166); omitted when there are none.
...footnoteWarningsField(content),
};
}
/**
* Validate a URL string against a scheme allowlist for a given context.
*
* The markdown link path enforces safe schemes via TipTap, but the raw
* JSON path (updatePageJson) bypasses that so this is the sanitization
* choke point for ProseMirror JSON written directly by the caller.
*
* - "link": reject javascript:, vbscript:, data: (any scheme that can
* execute or smuggle script when the href is clicked).
* - "src": allow only http(s):, mailto:, /api/files paths, or a
* scheme-less relative/absolute path; reject
* javascript:/vbscript:/data:/file:.
*/
/**
* Rename a page (change its title only) without touching or resending its
* content. The slug is derived from the page record, not the body, so it is
* left intact too.
*/
async renamePage(pageId: string, title: string) {
await this.ensureAuthenticated();
await this.client.post("/pages/update", { pageId, title });
return { success: true, pageId, title };
}
/**
* Copy the WHOLE content of one page onto another, entirely server-side: the
* source's ProseMirror document is read and written verbatim onto the target
* via the live collab path, so the document never passes through the model.
*
* Only the target's BODY is replaced its title and slug live on the page
* record (not in the content), so they are untouched. The source page is not
* modified at all.
*/
async movePage(
pageId: string,
parentPageId: string | null,
position?: string,
) {
await this.ensureAuthenticated();
// Docmost requires position >= 5 chars.
const validPosition = position || "a00000";
return this.client
.post("/pages/move", {
pageId,
parentPageId,
position: validPosition,
})
.then((res) => res.data);
}
async deletePage(pageId: string) {
await this.ensureAuthenticated();
return this.client
.post("/pages/delete", { pageId })
.then((res) => res.data);
}
// --- Comment methods (ported from upstream PR #3 by Max Nikitin) ---
/**
* Normalize a comment's `content` into a ProseMirror doc object before
* markdown conversion. createComment/updateComment send content as a
* JSON.stringify(...) STRING, and the server stores it as-is, so on read it
* comes back as a string. convertProseMirrorToMarkdown returns "" for a
* string, so parse it first (guarded fall back to the raw value on any
* parse failure so a non-JSON legacy value is still handled gracefully).
*/
/** Share a page publicly (idempotent) and return the public URL. */
async sharePage(pageId: string, searchIndexing: boolean = true) {
await this.ensureAuthenticated();
const response = await this.client.post("/shares/create", {
pageId,
includeSubPages: false,
searchIndexing,
});
const share = response.data?.data ?? response.data;
const slugId = share.page?.slugId || (await this.getPageRaw(pageId)).slugId;
return {
shareId: share.id,
key: share.key,
pageId: share.pageId,
publicUrl: this.shareUrl(share.key, slugId),
searchIndexing: share.searchIndexing,
};
}
/** List all public shares in the workspace with their URLs. */
/** Build the public share URL for a page. */
protected shareUrl(shareKey: string, slugId: string): string {
return `${this.appUrl}/share/${shareKey}/p/${slugId}`;
}
/** Share a page publicly (idempotent) and return the public URL. */
/** List all public shares in the workspace with their URLs. */
async listShares() {
const shares = await this.paginateAll("/shares", {});
return shares.map((s: any) => ({
shareId: s.id,
key: s.key,
pageId: s.pageId,
pageTitle: s.page?.title,
publicUrl: s.page?.slugId ? this.shareUrl(s.key, s.page.slugId) : null,
searchIndexing: s.searchIndexing,
createdAt: s.createdAt,
}));
}
/** Remove the public share of a page. */
async unsharePage(pageId: string) {
await this.ensureAuthenticated();
const shares = await this.listShares();
const share = shares.find((s: any) => s.pageId === pageId);
if (!share) {
throw new Error(`Page ${pageId} is not shared.`);
}
await this.client.post("/shares/delete", { shareId: share.shareId });
return { success: true, removedShareId: share.shareId, pageId };
}
/**
* Export a page to a single self-contained Docmost-flavoured markdown file:
* meta block + body (with inline comment anchors + diagrams) + comment
* threads. Lossless round-trip target; see importPageMarkdown for the inverse.
*/
async exportPageMarkdown(pageId: string): Promise<string> {
await this.ensureAuthenticated();
const page = await this.getPageRaw(pageId);
const body = page.content ? convertProseMirrorToMarkdown(page.content) : "";
let comments: any[] = [];
try {
// Lossless export: include RESOLVED threads so the export -> import
// round-trip preserves every comment. This is exactly why the active-only
// filter is an opt-in (default false) on listComments.
comments = (await this.listComments(pageId, true)).items;
} catch (e) {
// A comments fetch failure must not lose the body; export with [] and let
// the caller see the (empty) comments block. Log under DEBUG only.
if (process.env.DEBUG) console.error("export: listComments failed", e);
}
const meta = {
version: 1,
pageId: page.id,
slugId: page.slugId,
title: page.title,
spaceId: page.spaceId,
parentPageId: page.parentPageId ?? null,
};
return serializeDocmostMarkdown(meta, body, comments);
}
/**
* Import a self-contained Docmost markdown file back into a page. Parses out
* the meta + comments metadata blocks, converts the body to ProseMirror
* (restoring comment marks + diagrams from their inline HTML), and replaces
* the page content. Comment THREAD records are NOT written to the server in
* this version they are preserved in the file and the inline marks are
* re-applied so the highlights survive; managing comment records stays with
* the comment tools/UI.
*/
async importPageMarkdown(pageId: string, fullMarkdown: string): Promise<any> {
await this.ensureAuthenticated();
const { meta, body, comments } = parseDocmostMarkdown(fullMarkdown);
// PAGE import: canonicalize footnotes (see markdownToProseMirrorCanonical).
const doc = await markdownToProseMirrorCanonical(body);
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
const mutation = await replacePageContent(
pageUuid,
doc,
collabToken,
this.apiUrl,
);
// Collect distinct comment ids that actually became comment marks in the doc.
const collectCommentIds = (node: any, acc: Set<string>): Set<string> => {
if (!node || typeof node !== "object") return acc;
if (Array.isArray(node.marks)) {
for (const mk of node.marks) {
if (mk && mk.type === "comment" && mk.attrs?.commentId) {
acc.add(mk.attrs.commentId);
}
}
}
if (Array.isArray(node.content)) {
for (const child of node.content) collectCommentIds(child, acc);
}
return acc;
};
// Count reflects the comment marks present in the written document, so an id
// that only appears as inert text (e.g. inside a fenced code block) is not
// counted because it never becomes a comment mark.
const anchoredIds = collectCommentIds(doc, new Set<string>());
const result: any = {
success: true,
pageId,
anchoredCommentCount: anchoredIds.size,
commentsInFile: Array.isArray(comments) ? comments.length : 0,
verify: mutation.verify,
};
// Warn (non-fatal) if the file was exported from a DIFFERENT page.
if (meta?.pageId && meta.pageId !== pageId) {
result.warning = `File was exported from page ${meta.pageId} but is being imported into ${pageId}.`;
}
// Non-fatal footnote diagnostics (#166), analyzed on the BODY (the part after
// the docmost:meta / docmost:comments blocks) — so a `[^x]`-like token inside
// those JSON blocks never produces a false warning, while real markers in the
// body do. `body` comes from parseDocmostMarkdown(fullMarkdown) above.
Object.assign(result, footnoteWarningsField(body));
return result;
}
/**
* Rename a page (change its title only) without touching or resending its
* content. The slug is derived from the page record, not the body, so it is
* left intact too.
*/
/**
* Copy the WHOLE content of one page onto another, entirely server-side: the
* source's ProseMirror document is read and written verbatim onto the target
* via the live collab path, so the document never passes through the model.
*
* Only the target's BODY is replaced its title and slug live on the page
* record (not in the content), so they are untouched. The source page is not
* modified at all.
*/
async copyPageContent(sourcePageId: string, targetPageId: string) {
await this.ensureAuthenticated();
// A self-copy would be a no-op overwrite; reject it explicitly so a caller
// mistake surfaces as a clear error rather than a silent round-trip.
if (sourcePageId === targetPageId) {
throw new Error(
"copyPageContent: sourcePageId and targetPageId are the same page (no-op copy)",
);
}
const source = await this.getPageRaw(sourcePageId);
const content = source?.content;
if (
!content ||
typeof content !== "object" ||
content.type !== "doc" ||
!Array.isArray(content.content)
) {
throw new Error(
`copyPageContent: source page ${sourcePageId} has no usable ProseMirror content to copy`,
);
}
// Defense-in-depth: run the same URL-scheme sanitizer the JSON write path
// uses, so copying never lands a javascript:/data: href/src on the target
// (parity with updatePageJson; harmless for already-stored source content).
this.validateDocUrls(content);
// Defense-in-depth (#228): this is a FULL-document write, so canonicalize
// footnotes before copying — a no-op on already-canonical source content, but
// it guarantees a copy can never propagate a non-canonical footnote topology
// to the target (parity with the other full-doc write paths).
// #419: normalize + merge glyph-forked definitions before canonicalizing.
const canonical = canonicalizeFootnotes(normalizeAndMergeFootnotes(content));
const collabToken = await this.getCollabTokenWithReauth();
// Open the TARGET collab doc by its canonical UUID, never the slugId (#260).
const targetUuid = await this.resolvePageId(targetPageId);
const mutation = await this.replacePage(
targetUuid,
canonical,
collabToken,
this.apiUrl,
);
return {
success: true,
sourcePageId,
targetPageId,
copiedNodes: canonical.content.length,
verify: mutation.verify,
};
}
/**
* Surgical text edits: find/replace inside text nodes of the live
* document. Preserves all block ids, marks, callouts and tables.
*/
// --- Page history / diff / transform ---
/**
* List the saved versions (history snapshots) of a page, newest first.
* Docmost auto-snapshots on every save. Returns one cursor-paginated page of
* results: `{ items, nextCursor }`. The history record's id field is `id`.
*/
async listPageHistory(pageId: string, cursor?: string) {
await this.ensureAuthenticated();
const payload: Record<string, any> = { pageId };
if (cursor) payload.cursor = cursor;
const response = await this.client.post("/pages/history", payload);
const data = response.data?.data ?? response.data;
return {
items: data?.items ?? [],
nextCursor: data?.meta?.nextCursor ?? null,
};
}
/**
* Fetch a single page-history version including its lossless ProseMirror
* `content`. The version also carries pageId/title/createdAt.
*/
async getPageHistory(historyId: string) {
await this.ensureAuthenticated();
const response = await this.client.post("/pages/history/info", {
historyId,
});
return response.data?.data ?? response.data;
}
/**
* "Restore" a version: Docmost has NO restore endpoint, so we take the
* version's `content` and write it as the page's current content via the live
* collab path (which itself creates a new history snapshot). Returns the
* affected pageId and the source historyId.
*/
async restorePageVersion(historyId: string) {
await this.ensureAuthenticated();
const version = await this.getPageHistory(historyId);
if (
!version ||
!version.pageId ||
!version.content ||
typeof version.content !== "object"
) {
throw new Error(
`restorePageVersion: history ${historyId} has no usable content`,
);
}
// Defense-in-depth: sanitize URLs in the restored content (parity with the
// JSON write path) before writing it back.
this.validateDocUrls(version.content);
const collabToken = await this.getCollabTokenWithReauth();
// version.pageId is the page entity id (already a UUID); resolvePageId
// short-circuits a UUID with no round-trip, so this is defensive only (#260).
const pageUuid = await this.resolvePageId(version.pageId);
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
() => version.content,
);
return {
pageId: version.pageId,
restoredFrom: historyId,
verify: mutation.verify,
};
}
/**
* Diff two versions of a page and return a Docmost-equivalent change set.
* `from`/`to` each resolve to a ProseMirror doc:
* - null / undefined / "current" -> the page's CURRENT content;
* - any other string -> that historyId's content.
* Returns the diff plus the resolved version metadata for each side.
*/
async diffPageVersions(pageId: string, from?: string, to?: string) {
await this.ensureAuthenticated();
const isCurrent = (v?: string) => v == null || v === "" || v === "current";
const resolveSide = async (
v?: string,
): Promise<{ doc: any; meta: any }> => {
if (isCurrent(v)) {
const raw = await this.getPageRaw(pageId);
return {
doc: raw.content || { type: "doc", content: [] },
meta: {
kind: "current",
pageId,
title: raw.title,
updatedAt: raw.updatedAt,
},
};
}
const version = await this.getPageHistory(v as string);
return {
doc: version.content || { type: "doc", content: [] },
meta: {
kind: "history",
historyId: version.id,
pageId: version.pageId,
title: version.title,
createdAt: version.createdAt,
},
};
};
const fromSide = await resolveSide(from);
const toSide = await resolveSide(to);
const diff = diffDocs(fromSide.doc, toSide.doc);
return { from: fromSide.meta, to: toSide.meta, diff };
}
/**
* Edit a page by running an arbitrary user-supplied JS transform against the
* live document, with a diff preview + page-history safety net.
*
* The transform string is evaluated as `(doc, ctx) => doc` inside a node:vm
* sandbox: it gets ONLY `{ doc, ctx, structuredClone, console }` as globals,
* a 5s timeout, and NO access to require/process/fs/network. It must return a
* `{ type: "doc" }` node, which is validated structurally before any write.
*
* `ctx` exposes:
* - comments: the page's comments (fetched before the live read);
* - log: an array the transform can push diagnostics to (via console.log);
* - consume(id): mark a comment id as consumed (for deleteComments);
* - helpers: the transforms.ts primitives + commentsToFootnotes.
*
* Footnote convention used by the helpers: footnote markers are plain "[N]"
* text in the body, and the notes are an orderedList under a heading whose
* text is "Примечания переводчика".
*
* dryRun (default true): read the page's current content, run the transform,
* and return `{ pushed:false, diff, log }` WITHOUT opening the collab socket.
* Otherwise the transform runs atomically inside mutatePageContent, optionally
* deletes consumed comments, and returns the new historyId + diff + log.
*/
}
return PagesMixin;
}
+651
View File
@@ -0,0 +1,651 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import axios, { AxiosInstance } from "axios";
import {
filterWorkspace,
filterSpace,
filterPage,
filterComment,
filterSearchResult,
} from "../lib/filters.js";
import { convertProseMirrorToMarkdown } from "../lib/markdown-converter.js";
import {
collectInternalFileNodes,
normalizeFileUrl,
resolveInternalFilePath,
} from "../lib/internal-file-urls.js";
import { buildPageTree } from "../lib/tree.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import {
importMarkdownFragment,
canBeDocChild,
findUnrepresentableTableAttrs,
} from "../lib/markdown-fragment.js";
import { searchInDoc, SearchOptions } from "../lib/page-search.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Public method surface of ReadMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IReadMixin` fails to compile on drift.
export interface IReadMixin {
getWorkspace(): any;
getSpaces(): any;
listPages(spaceId?: string, limit?: number, tree?: boolean): any;
getTree(spaceId: string, rootPageId?: string, maxDepth?: number): any;
getPageContext(pageId: string): any;
listSidebarPages(spaceId: string, pageId?: string): any;
getPage(pageId: string): any;
getPageJson(pageId: string): any;
getOutline(pageId: string): any;
getNode(pageId: string, nodeId: string, format?: "markdown" | "json"): any;
searchInPage(pageId: string, query: string, opts?: SearchOptions): any;
getTable(pageId: string, tableRef: string): any;
search(query: string, spaceId?: string, limit?: number, opts?: { parentPageId?: string; titleOnly?: boolean }): any;
}
export function ReadMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IReadMixin> & TBase {
abstract class ReadMixin extends Base implements IReadMixin {
async getWorkspace() {
await this.ensureAuthenticated();
const response = await this.client.post("/workspace/info", {});
return {
data: filterWorkspace(response.data?.data ?? response.data),
success: response.data.success,
};
}
async getSpaces() {
const spaces = await this.paginateAll("/spaces", {});
return spaces.map((space) => filterSpace(space));
}
/**
* List pages in one of two modes.
*
* Default (`tree` false): most recent pages by updatedAt (descending),
* bounded. Fetching the whole space can exceed MCP response/time limits on
* large instances, so a single bounded page of results is returned (default
* 50, max 100) via the `/pages/recent` feed.
*
* Tree (`tree` true): DEPRECATED prefer `getTree`, which shares this exact
* code path (a single `/pages/tree` request via `enumerateSpacePages` +
* `buildPageTree`) but returns the compact `{pageId, title, children?,
* hasChildren?}` shape and supports `rootPageId`/`maxDepth`. This tree mode is
* kept for backward compatibility; it REQUIRES `spaceId` (a page tree is
* scoped to one space) and IGNORES `limit` the whole hierarchy is returned.
* It fetches the tree via `enumerateSpacePages`, which on the fork server
* resolves to a single `/pages/tree` request returning the whole
* permission-filtered flat page set (soft-deleted pages excluded
* server-side); the cursor-BFS in `enumerateSpacePages` is only a fallback for
* stock upstream servers that lack `/pages/tree`.
*/
async listPages(spaceId?: string, limit: number = 50, tree: boolean = false) {
await this.ensureAuthenticated();
if (tree) {
if (!spaceId) {
throw new Error(
"listPages: tree mode requires a spaceId (a page tree is scoped to one space). Pass spaceId, or omit tree to get the recent-pages list.",
);
}
const { pages } = await this.enumerateSpacePages(spaceId);
return buildPageTree(pages);
}
const clampedLimit = Math.max(1, Math.min(100, limit));
const payload: Record<string, any> = { limit: clampedLimit, page: 1 };
if (spaceId) payload.spaceId = spaceId;
const response = await this.client.post("/pages/recent", payload);
const data = response.data;
const items = data.data?.items || data.items || [];
return items.map((page: any) => filterPage(page));
}
/**
* Fetch a space's page hierarchy (or one subtree) as a nested tree in a SINGLE
* request the #443 `getTree` tool. Shares its whole code path with
* `listPages(tree:true)`: `enumerateSpacePages` issues one `POST /pages/tree`
* (with the cursor-BFS only as a fallback for stock upstream servers that lack
* the endpoint), then `buildPageTree` nests the flat, permission-filtered,
* position-ordered list. No second tree fetch, no per-node BFS.
*
* - `rootPageId` restrict to that page's subtree; the server seeds the CTE
* with the page itself, so the result is exactly ONE root (the page and its
* descendants). Omit it for the whole space.
* - `maxDepth` trim the response to that many levels (roots = depth 1) to
* save tokens; the server still returns everything in one request, the cut
* is applied in `buildPageTree` AFTER the full tree is built. A node whose
* children were cut carries `hasChildren: true` (source of truth = the flat
* item's server `hasChildren`) so the caller can descend with a follow-up
* `getTree(spaceId, rootPageId=that node)` call.
*
* Output nodes are `{pageId, title, children?, hasChildren?}` only the UUID
* `pageId` is exposed (never `slugId`/`icon`/`position`). Requires `spaceId`
* (a page tree is scoped to one space).
*/
async getTree(spaceId: string, rootPageId?: string, maxDepth?: number) {
await this.ensureAuthenticated();
if (!spaceId) {
throw new Error(
"getTree: spaceId is required (a page tree is scoped to one space).",
);
}
const { pages } = await this.enumerateSpacePages(spaceId, rootPageId);
return buildPageTree(pages, { shape: "getTree", maxDepth });
}
/**
* "Where am I / what's around" for a single page the #443 `getPageContext`
* tool. Metadata only (no page content), using exactly TWO server requests:
*
* 1. `POST /pages/breadcrumbs` a recursive CTE that walks UP from the page.
* The server returns the chain root->page order (it `.reverse()`s the
* child-first walk before responding), INCLUDING the page itself as the
* LAST element. So the last element is the page and everything before it
* is the ancestor chain root->parent. This carries the page's own title
* and spaceId, so no extra page-info fetch is needed for a UUID input.
* 2. `listSidebarPages(spaceId, pageId)` the page's DIRECT children,
* cursor-paginated (a page with >20 children returns ALL of them, no
* dupes) and in sidebar `position` order, each carrying `hasChildren`.
*
* The input may be a slugId (agents copy them from URLs); it is run through
* `resolvePageId` first, exactly like the other page tools. A UUID input adds
* no request there (short-circuit), keeping the total at two; a slugId input
* adds one unavoidable resolve round-trip.
*
* INVARIANT: only the UUID `pageId` is exposed anywhere server `id` is
* mapped to `pageId` and `slugId` is never leaked. A nonexistent/inaccessible
* pageId makes the server 404/403, which propagates as a clear tool error
* (never a hollow empty object).
*/
async getPageContext(pageId: string) {
await this.ensureAuthenticated();
// Resolve a possibly-slugId input to the canonical UUID (no round-trip for a
// UUID). Errors here (bad/inaccessible id) propagate as a clear tool error.
const pageUuid = await this.resolvePageId(pageId);
// Request 1: the ancestor chain, root->page, page included as the LAST item.
const response = await this.client.post("/pages/breadcrumbs", {
pageId: pageUuid,
});
const chain: any[] = (response.data?.data ?? response.data) ?? [];
if (!Array.isArray(chain) || chain.length === 0) {
// The endpoint always includes the page itself, so an empty chain means
// the page is gone/inaccessible — surface a clear error, not {}.
throw new Error(`getPageContext: page "${pageId}" not found or inaccessible`);
}
// Split: the last element is the page, the rest (root->parent) are the
// breadcrumbs. A root page has no ancestors -> breadcrumbs is [].
const self = chain[chain.length - 1];
const ancestors = chain.slice(0, -1);
const page = {
pageId: self.id,
title: self.title,
spaceId: self.spaceId,
};
const breadcrumbs = ancestors.map((n: any) => ({
pageId: n.id,
title: n.title,
}));
// Request 2: direct children in sidebar order, each with hasChildren.
const childItems = await this.listSidebarPages(self.spaceId, pageUuid);
const children = childItems.map((c: any) => ({
pageId: c.id,
title: c.title,
hasChildren: Boolean(c.hasChildren),
}));
return { page, breadcrumbs, children };
}
/**
* List sidebar pages for a space. With no pageId the request returns the
* space ROOT pages; with a pageId it returns the direct CHILDREN of that
* page. pageId is therefore optional and is only included in the POST body
* when provided (an empty/undefined pageId would otherwise change the
* semantics on the server).
*/
async listSidebarPages(spaceId: string, pageId?: string) {
await this.ensureAuthenticated();
// Paginate via the server-issued cursor. The server switched from OFFSET
// (`page`) to CURSOR (`cursor`/`nextCursor`) pagination, and the global
// ValidationPipe(whitelist:true) SILENTLY STRIPS the obsolete `page` field
// — so the old offset loop got the SAME first page every time (with
// hasNextPage stuck true) and dropped every child beyond the first page.
const MAX_PAGES = 50;
let cursor: string | undefined;
let allItems: any[] = [];
let truncated = false;
for (let i = 0; i < MAX_PAGES; i++) {
// limit: 100 is the server-side Max; cuts request count 5x vs the default 20.
const payload: Record<string, any> = { spaceId, limit: 100 };
// Only send pageId when scoping to a page's children; omit it for roots.
if (pageId) payload.pageId = pageId;
if (cursor) payload.cursor = cursor;
const data = (await this.client.post("/pages/sidebar-pages", payload)).data
?.data;
allItems = allItems.concat(data?.items ?? []);
// Advance strictly via the server-issued cursor; a missing/repeated cursor
// means the protocol drifted again — stop instead of looping on page one.
const next = data?.meta?.hasNextPage ? data?.meta?.nextCursor : null;
if (!next || next === cursor) break;
cursor = next;
// Reaching the ceiling with more pages still available means the child
// list is truncated (mirrors paginateAll).
if (i === MAX_PAGES - 1) truncated = true;
}
// Warn on real truncation (ceiling hit while the server still had pages) so
// the caller is not silently handed an incomplete child list.
if (truncated) {
console.warn(
`listSidebarPages: children of "${pageId ?? spaceId}" truncated at the ${MAX_PAGES}-page cap; more pages exist on the server`,
);
}
return allItems;
}
/**
* Enumerate EVERY page in a space (or in a subtree, when rootPageId is given).
*
* Primary path (fork server): a SINGLE `POST /pages/tree` returns the whole
* space (or a subtree) as a flat, permission-filtered list in one request, in
* the exact node shape buildPageTree consumes. This replaces the old
* per-node BFS, which issued N sidebar requests and after the server moved
* to cursor pagination silently lost every child past the first sidebar
* page (the obsolete `page` param was stripped by ValidationPipe).
*
* The subtree variant (rootPageId given) INCLUDES the root node itself
* (getPageAndDescendants seeds with id = rootPageId), unlike the old BFS
* which started from the root's children.
*
* Fallback path (stdio mode may target STOCK upstream Docmost, which lacks
* `/pages/tree`): on a 404/405 it falls back to the cursor-based BFS below,
* walking direct children via the fixed cursor listSidebarPages. Safeguards:
* a `visited` Set of page ids prevents re-processing a node (cycles /
* duplicate references), and a hard node cap bounds pathological trees so the
* walk always terminates.
*
* Returns `{ pages, truncated }`. `truncated` is true ONLY when the fallback
* BFS stopped at its MAX_NODES cap the primary /pages/tree path is uncapped
* and always returns the complete set, so it never reports truncation.
*/
protected async enumerateSpacePages(
spaceId: string,
rootPageId?: string,
): Promise<{ pages: any[]; truncated: boolean }> {
await this.ensureAuthenticated();
// Single request replaces the whole BFS: /pages/tree returns the full
// permission-filtered flat page set of a space (or a subtree) at once. This
// path is uncapped, so it is never truncated.
const payload = rootPageId ? { pageId: rootPageId } : { spaceId };
try {
const response = await this.client.post("/pages/tree", payload);
const pages = (response.data?.data ?? response.data)?.items ?? [];
return { pages, truncated: false };
} catch (e: any) {
// Only fall back when the endpoint is absent (stock upstream Docmost);
// any other error is a genuine failure and must propagate.
if (
!axios.isAxiosError(e) ||
(e.response?.status !== 404 && e.response?.status !== 405)
) {
throw e;
}
}
// Fallback: cursor-based breadth-first walk via listSidebarPages.
const MAX_NODES = 10000;
const result: any[] = [];
const visited = new Set<string>();
// Seed with the root node itself when scoping to a subtree, so its own
// comments aren't dropped: the primary /pages/tree seeds
// getPageAndDescendants with id = rootPageId (root included), but
// listSidebarPages(spaceId, rootPageId) returns only the root's CHILDREN.
// The `visited` set below prevents a double-add if the root also appears
// among the children. getPageRaw returns a page whose id/title/spaceId are
// exactly what buildPageTree and checkNewComments consume.
if (rootPageId) {
try {
const root = await this.getPageRaw(rootPageId);
if (root?.id) {
result.push(root);
visited.add(root.id);
}
} catch {
// Non-fatal: if the root can't be read, fall through to children-only.
}
}
// Seed the queue with the starting level (subtree children or roots).
const queue: any[] = await this.listSidebarPages(spaceId, rootPageId);
while (queue.length > 0 && result.length < MAX_NODES) {
const node = queue.shift();
if (!node || typeof node !== "object" || !node.id) continue;
// Skip already-seen ids to guard against cycles / duplicate references.
if (visited.has(node.id)) continue;
visited.add(node.id);
result.push(node);
if (node.hasChildren) {
try {
const children = await this.listSidebarPages(spaceId, node.id);
for (const child of children) queue.push(child);
} catch (e: any) {
// A failure fetching one node's children must not abort the whole
// walk: skip this branch and keep enumerating the rest.
}
}
}
// Truncated only when the cap was hit with the queue still non-empty (real
// truncation, not a natural end at exactly MAX_NODES).
return {
pages: result,
truncated: result.length >= MAX_NODES && queue.length > 0,
};
}
/** Raw page info including the ProseMirror JSON content and slugId. */
async getPage(pageId: string) {
await this.ensureAuthenticated();
const resultData = await this.getPageRaw(pageId);
// Agent read: hide resolved-comment anchors so the agent sees only active
// discussions. Active anchors are kept. (The lossless exportPageMarkdown
// round-trip deliberately does NOT pass this flag — resolved anchors there
// must be preserved.)
let content = resultData.content
? convertProseMirrorToMarkdown(resultData.content, {
dropResolvedCommentAnchors: true,
})
: "";
// Always fetch subpages to provide context to the agent
let subpages: any[] = [];
try {
// `pageId` may be a slugId, but the sidebar-pages endpoint requires the
// UUID; `resultData.id` holds the resolved UUID returned by getPageRaw.
subpages = await this.listSidebarPages(resultData.spaceId, resultData.id);
} catch (e: any) {
console.warn("Failed to fetch subpages:", e);
}
// Resolve subpages if the placeholder exists
if (content && content.includes("{{SUBPAGES}}")) {
if (subpages && subpages.length > 0) {
const list = subpages
.map((p: any) => `- [${p.title}](page:${p.id})`)
.join("\n");
content = content.replace("{{SUBPAGES}}", `### Subpages\n${list}`);
} else {
content = content.replace("{{SUBPAGES}}", "");
}
}
return {
data: filterPage(resultData, content, subpages),
success: true,
};
}
/** Page info + raw ProseMirror JSON content (lossless representation). */
async getPageJson(pageId: string) {
const data = await this.getPageRaw(pageId);
return {
id: data.id,
slugId: data.slugId,
title: data.title,
parentPageId: data.parentPageId,
spaceId: data.spaceId,
updatedAt: data.updatedAt,
content: data.content || { type: "doc", content: [] },
};
}
/**
* Fetch an INTERNAL Docmost file (authed loopback) for sandbox mirroring.
* `src` is normalized to `/api/files/<id>/<file>`; `this.client.baseURL`
* already ends in `/api`, so we strip the leading `/api` and request the
* relative path with the client's Authorization header. Returns the raw bytes
* and the response Content-Type (mime), defaulting to octet-stream.
*
* The fetch is size-bounded (hard 64 MiB ceiling) purely to protect memory;
* the authoritative per-blob cap is enforced by the sandbox `put`. The path is
* resolved via resolveInternalFilePath, which REJECTS (throws) any traversal
* or percent-encoded src that would let an attacker-controlled `attrs.src`
* escape `/api/files/` and reach another internal endpoint (SSRF). That throw
* happens before this.client.get, so a malicious src is counted as a failed
* mirror it never reaches the network.
*/
/**
* Compact outline of a page's top-level blocks (no full document body).
* Cheap way to locate sections/tables and grab block ids before drilling in
* with getNode / patchNode / insertNode.
*/
async getOutline(pageId: string) {
await this.ensureAuthenticated();
const data = await this.getPageRaw(pageId);
return {
pageId,
slugId: data.slugId,
title: data.title,
outline: buildOutline(data.content ?? { type: "doc", content: [] }),
};
}
/**
* Fetch a single block for editing by reference: a block id (headings/
* paragraphs/callouts/images), or `#<index>` to select a top-level block by its
* outline index (the only way to reach tables/rows/cells, which carry no id).
*
* `format` (#413):
* - `"markdown"` (DEFAULT): serialize the block via the canonical converter
* (`{type:"doc",content:[node]}` -> `convertProseMirrorToMarkdown`) a read
* "for editing": pair it with `patchNode({markdown})` to rewrite the block.
* Comment anchors (`<span data-comment-id>`, INCLUDING resolved ones) are
* NOT stripped here (unlike getPage): losing them on write-back would
* orphan the thread. Returns `{ ..., format:"markdown", markdown }`.
* - `"json"`: return the raw ProseMirror subtree as-is (lossless; the previous
* default). Returns `{ ..., format:"json", node }`.
*
* AUTO fallback: a type that cannot be a document top-level child
* (tableRow/tableCell/tableHeader, addressed by `#<index>`) is NOT expressible
* as a standalone markdown document, so a `"markdown"` request for such a node
* transparently falls back to JSON with an explicit `format:"json"` field. The
* check derives from the schema's `doc` contentMatch, so it tracks the schema.
*/
async getNode(
pageId: string,
nodeId: string,
format: "markdown" | "json" = "markdown",
) {
await this.ensureAuthenticated();
const data = await this.getPageRaw(pageId);
const hit = getNodeByRef(
data.content ?? { type: "doc", content: [] },
nodeId,
);
if (!hit) {
throw new Error(
`getNode: no node found for "${nodeId}" on page ${pageId} (use a block id from getOutline, or "#<index>" for a top-level block such as a table)`,
);
}
// JSON requested (or a non-top-level type that markdown cannot represent as a
// standalone document): return the subtree verbatim.
if (format === "json" || !canBeDocChild(hit.type)) {
return {
pageId,
ref: nodeId,
path: hit.path,
type: hit.type,
format: "json" as const,
node: hit.node,
};
}
// Markdown: wrap the node as a one-block doc and run the canonical converter.
// Comment anchors are DELIBERATELY preserved (converter default) so a
// getNode(markdown) -> edit -> patchNode(markdown) round trip does not orphan
// a comment thread; this differs from getPage, which strips them.
const markdown = convertProseMirrorToMarkdown({
type: "doc",
content: [hit.node],
});
return {
pageId,
ref: nodeId,
path: hit.path,
type: hit.type,
format: "markdown" as const,
markdown,
};
}
/**
* Find every occurrence of `query` on a page IN MEMORY, over the plain text of
* each text container (reusing the same `getPageRaw` fetch as the other read
* tools) no server search endpoint, no whole-document round-trip through the
* model. Returns `{ total, truncated, matches }`; each match carries a ref for
* getNode/patchNode (the `#<index>` form resolves with getNode but NOT
* patchNode see SearchMatch.nodeId), plus the top-level block index and a
* short context window used to build a unique text `selection` for
* createComment (createComment has no nodeId param). The pure engine
* (`searchInDoc`) owns the traversal, glue, the RE2 ReDoS-safe regex engine
* and the empty-query / invalid-or-unsupported-regex errors.
*/
async searchInPage(pageId: string, query: string, opts: SearchOptions = {}) {
await this.ensureAuthenticated();
const data = await this.getPageRaw(pageId);
const result = searchInDoc(
data.content ?? { type: "doc", content: [] },
query,
opts,
);
return { pageId, query, ...result };
}
/**
* Read a table as a matrix. `tableRef` is `#<index>` (from getOutline) or a
* block id of any node inside the table. Returns the cell texts plus a
* parallel cellIds matrix (each cell's first paragraph id, or null) so a
* caller can patchNode a cell for rich-formatted edits. Throws when no table
* resolves for the reference.
*/
async getTable(pageId: string, tableRef: string) {
await this.ensureAuthenticated();
const data = await this.getPageRaw(pageId);
const t = readTable(data.content ?? { type: "doc", content: [] }, tableRef);
if (!t) {
throw new Error(
`tableGet: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
);
}
return {
pageId,
table: tableRef,
rows: t.rows,
cols: t.cols,
path: t.path,
cells: t.cells,
cellIds: t.cellIds,
};
}
/**
* Insert a row of plain-text cells into a table on the LIVE collab document.
* `tableRef` is `#<index>` or a block id inside the target table. `cells` is
* padded to the table's column count (more cells than columns throws); `index`
* is a 0-based insert position (omit/out-of-range to append). Throws when no
* table resolves for the reference.
*/
async search(
query: string,
spaceId?: string,
limit?: number,
opts: { parentPageId?: string; titleOnly?: boolean } = {},
) {
await this.ensureAuthenticated();
// Opt into the #443 agent-lookup mode: `substring: true` turns on the hybrid
// substring + FTS branch that returns path + snippet + score. A stock
// upstream server strips these unknown DTO fields (whitelist:true) and
// silently degrades to plain FTS — see the tool-registration comment.
const payload: Record<string, any> = {
query,
spaceId,
substring: true,
};
if (opts.parentPageId) payload.parentPageId = opts.parentPageId;
if (opts.titleOnly) payload.titleOnly = true;
// Clamp an optional caller-supplied limit into the lookup range (1..50)
// before forwarding; omit it when not provided so the server default applies.
if (limit !== undefined) {
payload.limit = Math.max(1, Math.min(50, limit));
}
const response = await this.client.post("/search", payload);
// Normalize both response shapes: bare array and paginated { items: [...] }
const data = response.data?.data;
const items = Array.isArray(data) ? data : data?.items || [];
const filteredItems = items.map((item: any) => filterSearchResult(item));
return {
items: filteredItems,
success: response.data?.success || false,
};
}
}
return ReadMixin;
}
+225
View File
@@ -0,0 +1,225 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import {
collectInternalFileNodes,
normalizeFileUrl,
resolveInternalFilePath,
} from "../lib/internal-file-urls.js";
// Public method surface of StashMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements IStashMixin` fails to compile on drift.
export interface IStashMixin {
stashPage(pageId: string): Promise<{ uri: string; sha256: string; size: number; images: { mirrored: number; failed: number }; }>;
}
export function StashMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & IStashMixin> & TBase {
abstract class StashMixin extends Base implements IStashMixin {
/**
* Fetch an INTERNAL Docmost file (authed loopback) for sandbox mirroring.
* `src` is normalized to `/api/files/<id>/<file>`; `this.client.baseURL`
* already ends in `/api`, so we strip the leading `/api` and request the
* relative path with the client's Authorization header. Returns the raw bytes
* and the response Content-Type (mime), defaulting to octet-stream.
*
* The fetch is size-bounded (hard 64 MiB ceiling) purely to protect memory;
* the authoritative per-blob cap is enforced by the sandbox `put`. The path is
* resolved via resolveInternalFilePath, which REJECTS (throws) any traversal
* or percent-encoded src that would let an attacker-controlled `attrs.src`
* escape `/api/files/` and reach another internal endpoint (SSRF). That throw
* happens before this.client.get, so a malicious src is counted as a failed
* mirror it never reaches the network.
*/
protected async fetchInternalFile(
src: string,
): Promise<{ buffer: Buffer; mime: string }> {
const HARD_CEILING = 64 * 1024 * 1024; // 64 MiB memory guard
const relPath = resolveInternalFilePath(src);
const response = await this.client.get(relPath, {
responseType: "arraybuffer",
timeout: 30000,
maxContentLength: HARD_CEILING,
maxBodyLength: HARD_CEILING,
});
const buffer = Buffer.from(response.data);
if (buffer.length === 0) {
throw new Error(`Empty file response from "${src}"`);
}
const rawCt = response.headers?.["content-type"];
const mime =
typeof rawCt === "string" && rawCt.length > 0
? rawCt.split(";")[0].trim().toLowerCase()
: "application/octet-stream";
return { buffer, mime };
}
/**
* Stash a page's full content into the in-RAM blob sandbox and return ONLY a
* short anonymous URL the body never enters the model context (this is the
* whole point: ~30KB+ ProseMirror docs blow the model context if passed as a
* tool argument). Every INTERNAL file/image src (the type-agnostic criterion,
* so drawio/excalidraw/video/file nodes are covered too) is mirrored into the
* sandbox and its `src` rewritten to the sandbox URL, so an external consumer
* can fetch the images anonymously. External http(s) srcs are left untouched.
*
* Blobs live in RAM with a short TTL and are cleared on restart consume the
* URLs within the TTL and one uptime. A failed image fetch never aborts the
* doc: the original src is kept and the failure counted.
*
* Returns { uri, sha256, size, images:{mirrored, failed} }. `uri` and `sha256`
* are for the document blob; `sha256` is also the blob's ETag (integrity).
*/
async stashPage(pageId: string): Promise<{
uri: string;
sha256: string;
size: number;
images: { mirrored: number; failed: number };
}> {
if (!this.sandboxPut) {
throw new Error(
"stashPage is unavailable: the blob sandbox is not configured on this server",
);
}
await this.ensureAuthenticated();
// Stash the SAME shape getPageJson returns (id/title/.../content), with a
// deep clone so the rewrite never mutates anything shared.
const pageJson = await this.getPageJson(pageId);
const cloned: any = structuredClone(pageJson);
// Group internal-file nodes by normalized src so each unique resource is
// fetched + stored ONCE (dedup), and every node sharing that src points at
// the one sandbox blob. Capture each node's ORIGINAL raw src per-node:
// dedup groups nodes whose normalized src is equal even when their raw srcs
// differ (e.g. `/api/files/...` vs the bare `/files/...`), so on a revert we
// must restore each node's own original value, not the group key.
const bySrc = new Map<string, Array<{ node: any; origSrc: string }>>();
for (const node of collectInternalFileNodes(cloned.content)) {
const origSrc = String(node.attrs.src);
const src = normalizeFileUrl(origSrc);
const entry = { node, origSrc };
const group = bySrc.get(src);
if (group) group.push(entry);
else bySrc.set(src, [entry]);
}
let mirrored = 0;
let failed = 0;
// Record every successful mirror so it can be (a) reverted if its blob gets
// FIFO-evicted by a LATER put in this same stash, and (b) freed if the final
// doc put throws.
const mirrors: Array<{
uri: string;
entries: Array<{ node: any; origSrc: string }>;
}> = [];
const MAX_CONCURRENCY = 5;
const groups = [...bySrc.entries()];
for (let i = 0; i < groups.length; i += MAX_CONCURRENCY) {
const batch = groups.slice(i, i + MAX_CONCURRENCY);
await Promise.all(
batch.map(async ([src, entries]) => {
try {
const { buffer, mime } = await this.fetchInternalFile(src);
// put may throw if the blob exceeds the per-blob/total caps.
const stored = this.sandboxPut!(buffer, mime);
for (const entry of entries) entry.node.attrs.src = stored.uri;
mirrors.push({ uri: stored.uri, entries });
mirrored++;
} catch (err) {
// One bad/oversized image (or a rejected traversal src) must not
// abort the document. Logged unconditionally (never the blob body),
// matching the package's ungated console.warn convention.
failed++;
console.warn(
`stashPage: failed to mirror "${src}": ${
err instanceof Error ? err.message : String(err)
}`,
);
}
}),
);
}
// Revert one mirror's nodes to their original internal srcs and re-count it
// as failed (its blob was FIFO-evicted before the doc could reference it
// safely).
const revertMirror = (mirror: {
uri: string;
entries: Array<{ node: any; origSrc: string }>;
}) => {
for (const entry of mirror.entries) entry.node.attrs.src = entry.origSrc;
mirrored--;
failed++;
console.warn(
`stashPage: mirrored blob ${mirror.uri} was evicted before the doc ` +
`could safely reference it; reverted its src and counted it as failed`,
);
};
// Pre-put reconciliation: an image put earlier in THIS stash can FIFO-evict
// an even-earlier image of the same stash. Drop those from the live set
// first so the first serialized doc is already mostly correct.
let liveMirrors = mirrors;
if (this.sandboxHas) {
liveMirrors = [];
for (const mirror of mirrors) {
if (this.sandboxHas(mirror.uri)) liveMirrors.push(mirror);
else revertMirror(mirror);
}
}
// Put the document, then reconcile against eviction caused by the doc put
// ITSELF (the doc is newest, FIFO drops oldest = this stash's images). Each
// iteration reverts >=1 mirror, so the loop terminates (worst case: all
// images reverted and the doc references no sandbox image URLs).
let stored: { uri: string; sha256: string; size: number };
for (;;) {
const docBuf = Buffer.from(JSON.stringify(cloned), "utf8");
let docStored: { uri: string; sha256: string; size: number };
try {
docStored = this.sandboxPut(docBuf, "application/json");
} catch (err) {
// The doc put failed (e.g. doc exceeds the cap). Free this op's image
// blobs instead of leaking them in RAM for the whole TTL, then
// re-throw.
if (this.sandboxEvict) {
for (const mirror of liveMirrors) this.sandboxEvict(mirror.uri);
}
throw err;
}
if (!this.sandboxHas) {
stored = docStored;
break;
}
const evictedNow = liveMirrors.filter((m) => !this.sandboxHas!(m.uri));
if (evictedNow.length === 0) {
stored = docStored;
break;
}
// The doc we just stored references now-dead blobs. Revert those nodes,
// drop the stale doc blob, and loop to re-serialize + re-put the
// corrected doc.
for (const mirror of evictedNow) revertMirror(mirror);
liveMirrors = liveMirrors.filter((m) => this.sandboxHas!(m.uri));
if (this.sandboxEvict) this.sandboxEvict(docStored.uri);
}
return {
uri: stored.uri,
sha256: stored.sha256,
size: stored.size,
images: { mirrored, failed },
};
}
/**
* Compact outline of a page's top-level blocks (no full document body).
* Cheap way to locate sections/tables and grab block ids before drilling in
* with getNode / patchNode / insertNode.
*/
}
return StashMixin;
}
+291
View File
@@ -0,0 +1,291 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import {
replaceNodeById,
replaceNodeByIdWithMany,
reassignCollidingBlockIds,
deleteNodeById,
assertUnambiguousMatch,
insertNodeRelative,
insertNodesRelative,
blockPlainText,
buildOutline,
getNodeByRef,
readTable,
insertTableRow,
deleteTableRow,
updateTableCell,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import { withPageLock, isUuid } from "../lib/page-lock.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
// Public method surface of TablesMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements ITablesMixin` fails to compile on drift.
export interface ITablesMixin {
insertFootnote(pageId: string, anchorText: string, text: string): any;
tableInsertRow(pageId: string, tableRef: string, cells: string[], index?: number): any;
tableDeleteRow(pageId: string, tableRef: string, index: number): any;
tableUpdateCell(pageId: string, tableRef: string, row: number, col: number, text: string): any;
}
export function TablesMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & ITablesMixin> & TBase {
abstract class TablesMixin extends Base implements ITablesMixin {
/**
* AUTHOR-INLINE footnote insertion. The agent supplies only WHERE
* (`anchorText`, a snippet of body text to attach the marker after) and WHAT
* (`text`, the footnote content as markdown). Numbering and the bottom
* `footnotesList` are derived deterministically server-side
* (`insertInlineFootnote` -> `canonicalizeFootnotes`): the agent never sees,
* assigns, or edits a footnote number or the list, so it CANNOT desync.
*
* Content DEDUP: when an existing definition has the same content, its id is
* reused (one number, one definition, several references). The write is atomic
* via `mutatePageContent` (single-writer, page-locked); if the anchor text is
* not found the transform aborts with a clear error and no write happens.
*/
async insertFootnote(pageId: string, anchorText: string, text: string) {
await this.ensureAuthenticated();
if (!anchorText || !anchorText.trim()) {
throw new Error("insertFootnote: anchorText is required");
}
if (text == null || `${text}`.trim() === "") {
throw new Error("insertFootnote: text is required");
}
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
let result: { footnoteId: string; reused: boolean } | null = null;
const mutation = await this.mutatePage(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc: any) => {
const r = insertInlineFootnote(liveDoc, { anchorText, text });
if (!r.inserted) {
// Abort the page-locked write by throwing: mutatePageContent does not
// persist when the transform throws, so a missing anchor leaves the
// page untouched (no partial write).
throw new Error(
`insertFootnote: anchor text not found: ${JSON.stringify(
anchorText.slice(0, 80),
)}`,
);
}
result = { footnoteId: r.footnoteId, reused: r.reused };
return r.doc;
},
);
// The not-found path throws inside the transform (aborting mutatePage), so by
// here `result` is always set.
const r = result!;
return {
success: true,
modified: true,
pageId,
footnoteId: r.footnoteId,
reused: r.reused,
message: r.reused
? "Footnote inserted (reused an existing same-content definition)."
: "Footnote inserted.",
verify: mutation.verify,
};
}
/**
* Page-locked write seam over collaboration.mutatePageContent. Production just
* delegates; it exists as an overridable method so the insertFootnote wrapper
* (transform abort-on-not-found + response shaping) can be unit-tested without
* standing up a live Hocuspocus collab socket.
*
* SELF-RESOLVES the pageId to the canonical UUID (issue #449, "resolve-then-
* lock"): every write must lock and key its CollabSession by the UUID, never a
* raw slugId (#260). resolvePageId is cached/idempotent, so a caller that
* already resolved pays no extra round-trip; centralizing it here means a
* caller that reaches this seam with a raw slugId still locks correctly instead
* of silently splitting the mutex key. withPageLock also asserts the key is a
* UUID as a hard backstop.
*/
/**
* Insert a row of plain-text cells into a table on the LIVE collab document.
* `tableRef` is `#<index>` or a block id inside the target table. `cells` is
* padded to the table's column count (more cells than columns throws); `index`
* is a 0-based insert position (omit/out-of-range to append). Throws when no
* table resolves for the reference.
*/
async tableInsertRow(
pageId: string,
tableRef: string,
cells: string[],
index?: number,
) {
await this.ensureAuthenticated();
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
// Track insertion in an outer var, reset per-transform, so a collab retry
// recomputes it cleanly (mirrors insertNode's pattern).
let inserted = false;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
inserted = false;
const { doc: nd, inserted: ins } = insertTableRow(
liveDoc,
tableRef,
cells,
index,
);
inserted = ins;
if (!inserted) return null; // table not found -> skip the write entirely
return nd;
},
);
if (!inserted) {
throw new Error(
`tableInsertRow: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
);
}
return {
success: true,
table: tableRef,
inserted: true,
verify: mutation.verify,
};
}
/**
* Delete the row at 0-based `index` from a table on the LIVE collab document.
* `tableRef` is `#<index>` or a block id inside the target table. The helper's
* out-of-range and last-row errors propagate; a missing table throws here.
*/
async tableDeleteRow(pageId: string, tableRef: string, index: number) {
await this.ensureAuthenticated();
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
let deleted = false;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
deleted = false;
const { doc: nd, deleted: del } = deleteTableRow(
liveDoc,
tableRef,
index,
);
deleted = del;
if (!deleted) return null; // table not found -> skip the write entirely
return nd;
},
);
if (!deleted) {
throw new Error(
`tableDeleteRow: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
);
}
return {
success: true,
table: tableRef,
deleted: true,
verify: mutation.verify,
};
}
/**
* Set the plain-text content of cell `[row, col]` (0-based) in a table on the
* LIVE collab document, replacing the cell's content with a single text
* paragraph (the cell's first-paragraph id is preserved). `tableRef` is
* `#<index>` or a block id inside the target table. The helper's out-of-range
* error propagates; a missing table throws here.
*/
async tableUpdateCell(
pageId: string,
tableRef: string,
row: number,
col: number,
text: string,
) {
await this.ensureAuthenticated();
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
let updated = false;
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
(liveDoc) => {
updated = false;
const { doc: nd, updated: upd } = updateTableCell(
liveDoc,
tableRef,
row,
col,
text,
);
updated = upd;
if (!updated) return null; // table not found -> skip the write entirely
return nd;
},
);
if (!updated) {
throw new Error(
`tableUpdateCell: no table found for "${tableRef}" on page ${pageId} (use "#<index>" from getOutline, or a block id inside the table)`,
);
}
return {
success: true,
table: tableRef,
row,
col,
verify: mutation.verify,
};
}
/**
* Create a new page with title and content.
* Uses the /pages/import workaround (the only endpoint accepting content),
* then moves the page and restores the exact title: the import endpoint
* derives the title from the FILENAME and replaces spaces with
* underscores, so we explicitly re-set it via /pages/update afterwards.
*/
}
return TablesMixin;
}
+232
View File
@@ -0,0 +1,232 @@
// Auto-split from client.ts (issue #450). Mixin over the shared client context.
// Bodies are VERBATIM from the original DocmostClient; only the enclosing class
// changed to a mixin factory. See client/context.ts for the shared base.
import type { GConstructor, DocmostClientContext } from "./context.js";
import {
updatePageContentRealtime,
replacePageContent,
markdownToProseMirror,
markdownToProseMirrorCanonical,
mutatePageContent,
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import { diffDocs, summarizeChange } from "../lib/diff.js";
import {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
mergeFootnoteDefinitions,
} from "../lib/transforms.js";
import { normalizeAndMergeFootnotes } from "../lib/footnote-normalize-merge.js";
import vm from "node:vm";
// Public method surface of TransformsMixin (issue #450) — a NAMED type so the factory
// return type is expressible in the emitted .d.ts (the anonymous mixin class
// carries the base's protected shared state, which would otherwise trip TS4094).
// Derived from the class below; `implements ITransformsMixin` fails to compile on drift.
export interface ITransformsMixin {
transformPage(pageId: string, transformJs: string, opts?: { dryRun?: boolean; deleteComments?: boolean }): any;
}
export function TransformsMixin<TBase extends GConstructor<DocmostClientContext>>(Base: TBase): GConstructor<DocmostClientContext & ITransformsMixin> & TBase {
abstract class TransformsMixin extends Base implements ITransformsMixin {
/**
* Edit a page by running an arbitrary user-supplied JS transform against the
* live document, with a diff preview + page-history safety net.
*
* The transform string is evaluated as `(doc, ctx) => doc` inside a node:vm
* sandbox: it gets ONLY `{ doc, ctx, structuredClone, console }` as globals,
* a 5s timeout, and NO access to require/process/fs/network. It must return a
* `{ type: "doc" }` node, which is validated structurally before any write.
*
* `ctx` exposes:
* - comments: the page's comments (fetched before the live read);
* - log: an array the transform can push diagnostics to (via console.log);
* - consume(id): mark a comment id as consumed (for deleteComments);
* - helpers: the transforms.ts primitives + commentsToFootnotes.
*
* Footnote convention used by the helpers: footnote markers are plain "[N]"
* text in the body, and the notes are an orderedList under a heading whose
* text is "Примечания переводчика".
*
* dryRun (default true): read the page's current content, run the transform,
* and return `{ pushed:false, diff, log }` WITHOUT opening the collab socket.
* Otherwise the transform runs atomically inside mutatePageContent, optionally
* deletes consumed comments, and returns the new historyId + diff + log.
*/
async transformPage(
pageId: string,
transformJs: string,
opts: { dryRun?: boolean; deleteComments?: boolean } = {},
) {
const dryRun = opts.dryRun ?? true;
const deleteComments = opts.deleteComments ?? false;
await this.ensureAuthenticated();
// Full feed (incl. resolved): a page transform (e.g. comments -> footnotes)
// must operate on every comment, so it opts into the unfiltered feed.
const comments = (await this.listComments(pageId, true)).items;
// ctx handed to the sandbox. consume() records ids; helpers are the pure
// transform primitives. log is captured from console.log inside the sandbox.
const ctx = {
comments,
log: [] as string[],
consumed: new Set<string>(),
consume(id: string) {
this.consumed.add(id);
},
helpers: {
blockText,
walk,
getList,
insertMarkerAfter,
setCalloutRange,
noteItem,
mdToInlineNodes,
commentsToFootnotes,
canonicalizeFootnotes,
insertInlineFootnote,
},
};
// Captured oldDoc / newDoc for the diff (set inside runTransform).
let oldDoc: any;
let newDoc: any;
// SYNCHRONOUS transform runner — safe to call inside mutatePageContent's
// onSynced (no await between the live read and the write).
const runTransform = (liveDoc: any): any => {
oldDoc = structuredClone(liveDoc);
const sandbox: Record<string, any> = {
doc: structuredClone(liveDoc),
ctx,
structuredClone,
console: {
log: (...a: any[]) => ctx.log.push(a.map((x) => String(x)).join(" ")),
},
};
// Wrap the provided string in parentheses so both an expression-arrow
// (`(doc, ctx) => {...}`) and a parenthesized function work. Run it in a
// fresh context with no require/process/module so the transform cannot
// touch fs/network/process. 5s wall-clock timeout.
let fn: any;
try {
fn = vm.runInNewContext("(" + transformJs + ")", sandbox, {
timeout: 5000,
});
} catch (e: any) {
throw new Error(`transform did not compile: ${e?.message ?? e}`);
}
if (typeof fn !== "function") {
throw new Error(
"transform must evaluate to a function (doc, ctx) => doc",
);
}
const raw = vm.runInNewContext(
"f(d, c)",
{ f: fn, d: sandbox.doc, c: ctx },
{ timeout: 5000 },
);
if (
!raw ||
typeof raw !== "object" ||
raw.type !== "doc" ||
!Array.isArray(raw.content)
) {
throw new Error(
'transform must return a ProseMirror doc node ({ type:"doc", content:[...] })',
);
}
// Validate the RAW transform output FIRST (structure — including the
// MAX_DEPTH guard — and URLs), mirroring updatePageJson. The canonicalizer
// recurses without a depth limiter, so validating after it would turn a
// too-deep doc into an opaque "Maximum call stack size exceeded" instead of
// the intended "nesting exceeds the maximum depth" error.
this.validateDocStructure(raw);
this.validateDocUrls(raw);
// Auto-canonicalize footnotes after the transform (idempotent): no write
// path can leave footnotes out of order / orphaned / in a raw `[^id]`
// block. In a dryRun preview this may surface footnote edits the script
// author did not write (the canonicalizer tidied them) — that is expected.
// #419: normalize + merge glyph-forked definitions before canonicalizing.
const result = canonicalizeFootnotes(normalizeAndMergeFootnotes(raw));
newDoc = result;
return result;
};
if (dryRun) {
// Preview only: run against the current REST snapshot, never open the
// socket. oldDoc/newDoc are captured by runTransform.
const raw = await this.getPageRaw(pageId);
const current = raw.content || { type: "doc", content: [] };
runTransform(current);
// Run an independent Yjs-encodability check (same sanitize + schema as the
// apply path), so the preview fails with the same descriptive error when
// the doc is not encodable instead of returning a misleadingly-green diff.
assertYjsEncodable(newDoc);
return {
pushed: false,
diff: diffDocs(oldDoc, newDoc),
log: ctx.log,
};
}
// Apply atomically against the live doc.
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260).
const pageUuid = await this.resolvePageId(pageId);
const mutation = await mutatePageContent(
pageUuid,
collabToken,
this.apiUrl,
runTransform,
);
// Optionally delete consumed comments (best-effort; a delete failure must
// not undo the successful write).
const deletedComments: string[] = [];
if (deleteComments) {
for (const id of ctx.consumed) {
try {
await this.deleteComment(id);
deletedComments.push(id);
} catch (e) {
if (process.env.DEBUG) {
console.error(`transform: failed to delete comment ${id}:`, e);
}
}
}
}
// Fetch the newest historyId (Docmost snapshots on the write above).
let historyId: string | null = null;
try {
const hist = await this.listPageHistory(pageId);
historyId = hist.items?.[0]?.id ?? null;
} catch (e) {
if (process.env.DEBUG) {
console.error("transform: failed to fetch history id:", e);
}
}
return {
pushed: true,
historyId,
diff: diffDocs(oldDoc, newDoc),
deletedComments,
log: ctx.log,
verify: mutation.verify,
};
}
}
return TransformsMixin;
}
+4 -7
View File
@@ -57,17 +57,14 @@ export const DEFAULT_COMMENT_SIGNAL_DEBOUNCE_MS = 20_000;
/**
* Tools whose OWN result must NOT carry the signal it would be tautological
* (the agent is already looking at comments) and noisy. Listed in BOTH the
* standalone MCP snake_case names AND the in-app camelCase keys so a single set
* covers both surfaces (the signal text itself uses the camelCase `listComments`
* per roadmap #412). `getComment` (single fetch) is intentionally NOT excluded.
* (the agent is already looking at comments) and noisy. Since issue #412 both
* the standalone MCP surface and the in-app agent use the same camelCase tool
* names, so a single set of camelCase names covers both surfaces. `getComment`
* (single fetch) is intentionally NOT excluded.
*/
export const COMMENT_SIGNAL_EXCLUDED_TOOLS: ReadonlySet<string> = new Set([
"list_comments",
"listComments",
"check_new_comments",
"checkNewComments",
"create_comment",
"createComment",
]);
+164 -563
View File
@@ -5,7 +5,10 @@ import { fileURLToPath } from "url";
import { dirname, join } from "path";
import { DocmostClient, DocmostMcpConfig } from "./client.js";
import { parseNodeArg } from "@docmost/prosemirror-markdown";
import { searchShapes } from "./lib/drawio-shapes.js";
import { getGuideSection } from "./lib/drawio-guide.js";
import { SHARED_TOOL_SPECS, SharedToolSpec } from "./tool-specs.js";
import { SERVER_INSTRUCTIONS } from "./server-instructions.js";
import {
createCommentSignalTracker,
CommentSignalTracker,
@@ -54,6 +57,13 @@ export type {
CommentSignalProbeResult,
CommentSignalTrackerOptions,
} from "./comment-signal.js";
// Re-export the pure, no-network draw.io helpers (#424) so the in-app AI-SDK
// service can wire drawioShapes / drawioGuide off the loaded module. These are
// NOT client methods (no page/backend hit) — the in-app handler calls them
// directly, mirroring how the standalone MCP server wires them here.
export { searchShapes } from "./lib/drawio-shapes.js";
export type { SearchShapesOptions } from "./lib/drawio-shapes.js";
export { getGuideSection } from "./lib/drawio-guide.js";
// Read version from package.json
const __filename = fileURLToPath(import.meta.url);
@@ -74,19 +84,17 @@ const VERSION = packageJson.version;
// Editing guide surfaced to MCP clients in the initialize result so they can
// pick the right tool by intent and avoid resending whole documents.
//
// MAINTENANCE RULE: when you ADD, RENAME, or REMOVE a tool (either an inline
// server.registerTool(...) here or a spec in tool-specs.ts), you MUST update
// this guide so the new tool is routed by intent. This is enforced by
// test/unit/server-instructions.test.mjs, which fails when a registered tool
// name is not mentioned below (see its EXCEPTIONS list for the rare opt-outs).
// Exported for that test.
export const SERVER_INSTRUCTIONS =
"Docmost editing guide — choose the tool by intent.\n" +
"READ: find a page -> search (workspace-wide full-text); list -> list_pages / list_spaces. Locate blocks and their ids CHEAPLY -> get_outline (compact top-level map; start here, not get_page_json). One block's subtree -> get_node (by attrs.id, or \"#<index>\" for tables, which carry no id). Find every occurrence of a string/regex ON a page (and where each is) -> search_in_page, NOT block-by-block get_node — it returns each hit's node ref + block index + context for a targeted comment. Whole page -> get_page (Markdown, lossy; inline <span data-comment-id> tags are comment anchors — markup, not text) or get_page_json (lossless ProseMirror with block ids). Hand a huge page (with images) to an external consumer without pulling it through the model context -> stash_page (returns a short-lived anonymous URL).\n" +
"EDIT: fix wording/typos/numbers -> edit_page_text (find/replace inside blocks, no node id needed). Change ONE block (paragraph/heading/callout/etc.) structurally -> patch_node (by attrs.id from get_outline). Add a block -> insert_node (before/after a block by attrs.id or by anchor text, or append). Remove a block -> delete_node (by attrs.id). Tables -> table_get / table_update_cell / table_insert_row / table_delete_row (address by \"#<index>\" from get_outline; table nodes have no attrs.id). Images -> insert_image (add from a web URL) / replace_image (swap an existing image). Draw.io diagrams -> drawio_create (create from mxGraph XML and insert), drawio_get (read a diagram as mxGraph XML + a hash), drawio_update (replace a diagram; pass the hash from drawio_get as baseHash for optimistic locking). Footnotes -> insert_footnote. Bulk/structural rewrite -> update_page_json (full ProseMirror replace; prefer the granular tools above to avoid resending the whole ~100KB+ document). Complex/scripted rewrite (multiple coordinated edits, renumbering) -> docmost_transform: write a JS `(doc, ctx) => doc` transform, preview the diff with dryRun (default), then apply with dryRun:false; ctx.helpers includes commentsToFootnotes for turning inline comments into numbered footnotes.\n" +
"PAGES: new -> create_page (Markdown). Rename (title only) -> rename_page. Move -> move_page. Delete -> delete_page (SOFT delete — the page goes to trash and is restorable; nothing is permanent). Copy/replace a page's whole content from another page (server-side, no document through the model) -> copy_page_content. Sharing -> share_page / unshare_page / list_shares; share_page makes the page PUBLICLY accessible — do it only when explicitly asked.\n" +
"COMMENTS: create_comment is always inline and requires an EXACT selection — contiguous text from a single block, <=250 chars (fails rather than leaving an unanchored comment); reply to a thread via parentCommentId. Propose a concrete text fix for one-click human approval -> create_comment with suggestedText (the exact plain-text replacement for the selection; the selection must then be UNIQUE in the page — extend it with context if needed); prefer this over editing directly when the change is subjective or needs the author's sign-off. Manage -> list_comments, update_comment, resolve_comment (resolve/reopen, reversible — prefer over delete to close), delete_comment, check_new_comments.\n" +
"HISTORY: review what changed -> diff_page_versions (a historyId vs current, or two versions). List saved versions -> list_page_history. Undo a bad edit -> restore_page_version (writes a past version back as current; itself revertible). Lossless markdown round-trip (download, edit, re-upload, incl. comment anchors) -> export_page_markdown / import_page_markdown.";
// The guide is now SPLIT (issue #448): the hand-written routing prose lives in
// server-instructions.ts and the tool INVENTORY is GENERATED from the registry
// (SHARED_TOOL_SPECS + INLINE_MCP_INVENTORY), so it can no longer drift out of
// sync with the registered tools. Re-exported here (its old home) so existing
// importers are unaffected; the composition lives in server-instructions.ts.
// The drawioShapes / drawioGuide tools (#424) stay in SHARED_TOOL_SPECS (so the
// generated <tool_inventory> picks them up from their catalogLine automatically)
// but are flagged `inlineBothHosts` and registered inline below (their pure
// helpers can't cross into tool-specs.ts); only the hand-written routing prose in
// server-instructions.ts is updated to mention them.
export { SERVER_INSTRUCTIONS };
// Helper to format JSON responses
const jsonContent = (data: any) => ({
@@ -202,10 +210,10 @@ export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
{ instructions: SERVER_INSTRUCTIONS },
);
// Single choke point for MCP tool timing. Both `registerShared` (below) and
// the inline `server.registerTool(...)` calls funnel through this one method,
// so monkeypatching it HERE — before any tool is registered and before
// `registerShared` captures a reference to it — times every tool with no
// Single choke point for MCP tool timing. Both `registerSharedFromSpec` (below)
// and the inline `server.registerTool(...)` calls funnel through this one
// method, so monkeypatching it HERE — before any tool is registered and before
// the registry loop captures a reference to it — times every tool with no
// per-tool boilerplate. The wrapped handler records wall-clock duration and,
// in a `finally`, feeds the host's dependency-neutral sink
// `config.onMetric("mcp_tool_duration_seconds", seconds, { tool })`. The tool
@@ -261,104 +269,112 @@ export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
return originalRegisterTool(...args.slice(0, -1), signalledHandler);
};
// Register a tool from the shared, zod-agnostic spec registry. The spec owns
// the canonical name + model-facing description + (optional) schema builder;
// only the execute body is supplied per call. buildShape is invoked with THIS
// package's zod (v3); the in-app layer passes its own zod (v4).
//
// The spec's schema builder returns a plain ZodRawShape (Record<string,
// unknown> in the shared module since it must stay zod-agnostic), so the
// McpServer.registerTool overloads cannot infer the execute arg's shape from
// it. We type `execute` loosely and cast the call through `any`; runtime
// behaviour is unchanged — each execute body destructures the same fields the
// builder declares.
const registerShared = (
spec: SharedToolSpec,
execute: (args: any) => Promise<{ content: { type: "text"; text: string }[] }>,
) =>
(server.registerTool as any)(
// Register EVERY shared tool from the zod-agnostic registry in one loop (#445).
// The spec owns the canonical name + description + (optional) schema builder AND
// the canonical execute mapping; the host only supplies the RESULT ENVELOPE. For
// each spec:
// - skip `inAppOnly` specs (they belong to the in-app host only);
// - if the spec has an `mcpExecute` override (a deliberate per-layer
// difference — a guardrail, an omitted param, or a non-JSON envelope like a
// resource_link/bare success line), the override OWNS the full MCP content
// result and is used VERBATIM;
// - otherwise the canonical `execute` returns RAW data and this host wraps it
// in the standard JSON text envelope (jsonContent), exactly as the old inline
// bodies did.
// buildShape is invoked with THIS package's zod (v3); the in-app layer passes its
// own zod (v4). The registry's execute returns `unknown` (it is zod-agnostic), so
// the wrapping is typed loosely and cast — runtime behaviour is unchanged.
const registerSharedFromSpec = (spec: SharedToolSpec) => {
if (spec.inAppOnly) return;
// `inlineBothHosts` specs (drawioShapes / drawioGuide) carry no execute —
// their pure helper cannot cross into the zod-agnostic tool-specs.ts, so they
// are registered INLINE below (searchShapes / getGuideSection). Skip them here
// so the loop never dereferences a missing `execute`.
if (spec.inlineBothHosts) return;
const handler = async (args: any) => {
if (spec.mcpExecute) {
// The override owns the full MCP result envelope (not re-wrapped).
return (await spec.mcpExecute(docmostClient, args)) as {
content: { type: "text"; text: string }[];
};
}
// Canonical execute returns raw data; wrap it as JSON text content.
const raw = await spec.execute!(docmostClient, args);
return jsonContent(raw);
};
return (server.registerTool as any)(
spec.mcpName,
spec.buildShape
? { description: spec.description, inputSchema: spec.buildShape(z) }
: { description: spec.description },
execute,
handler,
);
};
// Tool: get_workspace
registerShared(SHARED_TOOL_SPECS.getWorkspace, async () => {
const workspace = await docmostClient.getWorkspace();
return jsonContent(workspace);
});
for (const spec of Object.values(SHARED_TOOL_SPECS)) {
registerSharedFromSpec(spec as SharedToolSpec);
}
// Tool: list_spaces
registerShared(SHARED_TOOL_SPECS.listSpaces, async () => {
const spaces = await docmostClient.getSpaces();
return jsonContent(spaces);
});
// --- INLINE drawio helper tools (IN the shared registry, but inlineBothHosts) ---
// drawioShapes / drawioGuide (#424) live in SHARED_TOOL_SPECS (so the shared
// contract pins their name/description/schema across both hosts) but carry the
// `inlineBothHosts` flag and NO execute: their pure backing helpers
// (searchShapes / getGuideSection) cannot be value-imported into the
// zod-agnostic tool-specs.ts without breaking the in-app server's commonjs
// type-check (searchShapes' catalog loader uses import.meta). So both hosts wire
// them directly. Here on the MCP host they reuse the spec's name/description/
// schema and wrap the raw helper result as JSON text content — byte-identical to
// what the registry loop would have produced. The in-app host mirrors this in
// ai-chat-tools.service.ts.
{
// Cast registerTool like the loop's registerSharedFromSpec does: the spec's
// buildShape returns the loose zod-agnostic ZodRawShape (Record<string,
// unknown>) and the handler args are the SDK-validated, type-erased input.
const registerInline = server.registerTool as any;
const shapesSpec = SHARED_TOOL_SPECS.drawioShapes as SharedToolSpec;
registerInline(
shapesSpec.mcpName,
{
description: shapesSpec.description,
inputSchema: shapesSpec.buildShape!(z),
},
async ({ query, category, limit }: any) => {
const results = searchShapes(query, { category, limit });
return jsonContent({ query, count: results.length, results });
},
);
const guideSpec = SHARED_TOOL_SPECS.drawioGuide as SharedToolSpec;
registerInline(
guideSpec.mcpName,
{
description: guideSpec.description,
inputSchema: guideSpec.buildShape!(z),
},
async ({ section }: any) => jsonContent(getGuideSection(section)),
);
}
// Tool: list_pages
// INTENTIONAL per-transport divergence (not in the shared registry): this
// transport exposes a `tree:true` mode that returns the full nested hierarchy;
// the in-app copy keeps the same tree option but is worded for the in-app agent.
// Kept per-layer so each side can tune its own guidance.
// Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294). This
// transport keeps applying its own defaults (limit=50, tree=false) in execute.
registerShared(SHARED_TOOL_SPECS.listPages, async ({ spaceId, limit, tree }) => {
const result = await docmostClient.listPages(spaceId, limit ?? 50, tree ?? false);
return jsonContent(result);
});
// --- INLINE tools kept per-transport (NOT in the shared registry) ---
// Each stays inline for a documented reason: a snake_case/camelCase naming
// clash the registry convention forbids (tableGet), an intentional
// per-transport behaviour/schema divergence (search, docmostTransform), or a
// tool that exists ONLY on this standalone MCP surface (updateComment,
// deleteComment — the in-app agent deliberately exposes no hard comment
// edit/delete tool).
// Tool: get_page
// Schema + description now live in the shared registry (#294).
registerShared(SHARED_TOOL_SPECS.getPage, async ({ pageId }) => {
const page = await docmostClient.getPage(pageId);
return jsonContent(page);
});
// Tool: get_page_json
registerShared(SHARED_TOOL_SPECS.getPageJson, async ({ pageId }) => {
const page = await docmostClient.getPageJson(pageId);
return jsonContent(page);
});
// Tool: get_outline
registerShared(SHARED_TOOL_SPECS.getOutline, async ({ pageId }) => {
const result = await docmostClient.getOutline(pageId);
return jsonContent(result);
});
// Tool: get_node
registerShared(SHARED_TOOL_SPECS.getNode, async ({ pageId, nodeId }) => {
const result = await docmostClient.getNode(pageId, nodeId);
return jsonContent(result);
});
// Tool: search_in_page
registerShared(
SHARED_TOOL_SPECS.searchInPage,
async ({ pageId, query, regex, caseSensitive, limit }) => {
const result = await docmostClient.searchInPage(pageId, query, {
regex,
caseSensitive,
limit,
});
return jsonContent(result);
},
);
// Tool: table_get
// NOT in the shared registry: the MCP tool name `table_get` is noun-first while
// Tool: tableGet
// NOT in the shared registry: the MCP tool name `tableGet` is noun-first while
// the in-app key is `getTable` (verb-first), breaking the snake_case(inAppKey)
// convention the shared registry enforces (shared-tool-specs.contract.spec.ts).
// Renaming the public MCP tool would break external clients, so it stays inline.
server.registerTool(
"table_get",
"tableGet",
{
description:
"Read a table as a matrix. Returns {rows, cols, cells (text[][]), " +
"cellIds (paragraph id per cell, or null)}. `table` = `#<index>` from " +
"get_outline, or any block id inside the table. Use cellIds with " +
"patch_node for rich-formatted cell edits. `cols` is the FIRST row's " +
"getOutline, or any block id inside the table. Use cellIds with " +
"patchNode for rich-formatted cell edits. `cols` is the FIRST row's " +
"width; ragged tables may vary per row, so use the per-row length of " +
"`cells` for each row.",
inputSchema: {
@@ -372,386 +388,9 @@ server.registerTool(
},
);
// Tool: table_insert_row
// Schema + description now live in the shared registry (#294); the `table`
// parameter name is the canonical one (the in-app layer was unified to it).
registerShared(
SHARED_TOOL_SPECS.tableInsertRow,
async ({ pageId, table, cells, index }) => {
const result = await docmostClient.tableInsertRow(
pageId,
table,
cells,
index,
);
return jsonContent(result);
},
);
// Tool: table_delete_row
// Schema + description now live in the shared registry (#294).
registerShared(
SHARED_TOOL_SPECS.tableDeleteRow,
async ({ pageId, table, index }) => {
const result = await docmostClient.tableDeleteRow(pageId, table, index);
return jsonContent(result);
},
);
// Tool: table_update_cell
// Schema + description now live in the shared registry (#294).
registerShared(
SHARED_TOOL_SPECS.tableUpdateCell,
async ({ pageId, table, row, col, text }) => {
const result = await docmostClient.tableUpdateCell(
pageId,
table,
row,
col,
text,
);
return jsonContent(result);
},
);
// Tool: create_page
// Schema + description now live in the shared registry (#294).
registerShared(
SHARED_TOOL_SPECS.createPage,
async ({ title, content, spaceId, parentPageId }) => {
const result = await docmostClient.createPage(
title,
content,
spaceId,
parentPageId,
);
return jsonContent(result);
},
);
// Tool: update_page_json
// Schema + description now live in the shared registry (#294). The execute body
// keeps this transport's content normalization (parse a JSON-string content,
// pass undefined/null through for a title-only/no-op update).
registerShared(
SHARED_TOOL_SPECS.updatePageJson,
async ({ pageId, content, title }) => {
// Only parse/validate the document when it was actually supplied; when it
// is omitted, pass it straight through so the client performs a title-only
// (or no-op) update.
let doc;
if (content === undefined || content === null) {
doc = undefined;
} else {
// String -> JSON.parse (throwing on invalid); object passes through.
doc = parseNodeArg(content, "content was a string but not valid JSON");
}
const result = await docmostClient.updatePageJson(pageId, doc, title);
return jsonContent(result);
},
);
// Tool: export_page_markdown
// Schema + description now live in the shared registry (#294).
registerShared(SHARED_TOOL_SPECS.exportPageMarkdown, async ({ pageId }) => {
const md = await docmostClient.exportPageMarkdown(pageId);
return { content: [{ type: "text" as const, text: md }] };
});
// Tool: import_page_markdown
registerShared(
SHARED_TOOL_SPECS.importPageMarkdown,
async ({ pageId, markdown }) => {
const res = await docmostClient.importPageMarkdown(pageId, markdown);
return jsonContent(res);
},
);
// Tool: copy_page_content
registerShared(
SHARED_TOOL_SPECS.copyPageContent,
async ({ sourcePageId, targetPageId }) => {
const result = await docmostClient.copyPageContent(
sourcePageId,
targetPageId,
);
return jsonContent(result);
},
);
// Tool: rename_page
// Schema + description now live in the shared registry (#294).
registerShared(SHARED_TOOL_SPECS.renamePage, async ({ pageId, title }) => {
const result = await docmostClient.renamePage(pageId, title);
return jsonContent(result);
});
// Tool: edit_page_text
registerShared(SHARED_TOOL_SPECS.editPageText, async ({ pageId, edits }) => {
const result = await docmostClient.editPageText(pageId, edits);
return jsonContent(result);
});
// Tool: stash_page — returns a resource_link (NOT embedded text) so the doc
// body never enters the model context. Registered directly (not via
// registerShared) because that helper only emits text content. Also returns
// `structuredContent` carrying the full documented `{uri, sha256, size, images}`
// shape alongside the resource_link, so MCP clients receive the blob's sha256
// (its ETag, for integrity) and mirror counts, not just the link.
// Tool: updateComment
server.registerTool(
SHARED_TOOL_SPECS.stashPage.mcpName,
{
description: SHARED_TOOL_SPECS.stashPage.description,
inputSchema: SHARED_TOOL_SPECS.stashPage.buildShape!(z),
},
async ({ pageId }: { pageId: string }) => {
const result = await docmostClient.stashPage(pageId);
return {
content: [
{
type: "resource_link" as const,
uri: result.uri,
name: "page.json",
mimeType: "application/json",
size: result.size,
},
],
// Mirror the full documented result shape ({ uri, size, sha256, images })
// as structuredContent so MCP clients get the blob's sha256 (its ETag, for
// integrity) and the mirror counts, not just the resource_link.
structuredContent: {
uri: result.uri,
sha256: result.sha256,
size: result.size,
images: result.images,
},
};
},
);
// Tool: patch_node — schema + description from the shared registry (identical
// across both transports). The execute body keeps its own parseNodeArg
// normalization (the model sometimes serializes `node` as a JSON string).
registerShared(
SHARED_TOOL_SPECS.patchNode,
async ({ pageId, nodeId, node }) => {
const parsedNode = parseNodeArg(node);
const result = await docmostClient.patchNode(pageId, nodeId, parsedNode);
return jsonContent(result);
},
);
// Tool: insert_node — schema + description from the shared registry. As with
// patch_node, the execute body retains parseNodeArg on the incoming node.
registerShared(
SHARED_TOOL_SPECS.insertNode,
async ({ pageId, node, position, anchorNodeId, anchorText }) => {
const parsedNode = parseNodeArg(node);
const result = await docmostClient.insertNode(pageId, parsedNode, {
position,
anchorNodeId,
anchorText,
});
return jsonContent(result);
},
);
// Tool: delete_node
registerShared(SHARED_TOOL_SPECS.deleteNode, async ({ pageId, nodeId }) => {
const result = await docmostClient.deleteNode(pageId, nodeId);
return jsonContent(result);
});
// Tool: insert_image
// Schema + description now live in the shared registry (#410) so BOTH this MCP
// server and the in-app AI-chat agent expose it. The execute body is unchanged.
registerShared(
SHARED_TOOL_SPECS.insertImage,
async ({ pageId, imageUrl, align, alt, replaceText, afterText }) => {
const result = await docmostClient.insertImage(pageId, imageUrl, {
align,
alt,
replaceText,
afterText,
});
return jsonContent(result);
},
);
// Tool: replace_image
// Schema + description now live in the shared registry (#410).
registerShared(
SHARED_TOOL_SPECS.replaceImage,
async ({ pageId, attachmentId, imageUrl, align, alt }) => {
const result = await docmostClient.replaceImage(
pageId,
attachmentId,
imageUrl,
{
align,
alt,
},
);
return jsonContent(result);
},
);
// Tool: drawio_get — read a draw.io diagram as mxGraph XML (or the raw SVG).
registerShared(
SHARED_TOOL_SPECS.drawioGet,
async ({ pageId, node, format }) => {
const result = await docmostClient.drawioGet(pageId, node, format ?? "xml");
return jsonContent(result);
},
);
// Tool: drawio_create — lint mxGraph XML, build the .drawio.svg, insert a node.
registerShared(
SHARED_TOOL_SPECS.drawioCreate,
async ({ pageId, xml, position, anchorNodeId, anchorText, title }) => {
const result = await docmostClient.drawioCreate(
pageId,
{ position, anchorNodeId, anchorText },
xml,
title,
);
return jsonContent(result);
},
);
// Tool: drawio_update — optimistic-locked full replacement of a diagram.
registerShared(
SHARED_TOOL_SPECS.drawioUpdate,
async ({ pageId, node, xml, baseHash }) => {
const result = await docmostClient.drawioUpdate(pageId, node, xml, baseHash);
return jsonContent(result);
},
);
// Tool: share_page
// Schema + description now live in the shared registry (#294). The execute body
// keeps this transport's own `searchIndexing ?? true` default.
registerShared(
SHARED_TOOL_SPECS.sharePage,
async ({ pageId, searchIndexing }) => {
const result = await docmostClient.sharePage(pageId, searchIndexing ?? true);
return jsonContent(result);
},
);
// Tool: unshare_page
registerShared(SHARED_TOOL_SPECS.unsharePage, async ({ pageId }) => {
const result = await docmostClient.unsharePage(pageId);
return jsonContent(result);
});
// Tool: list_shares
registerShared(SHARED_TOOL_SPECS.listShares, async () => {
const result = await docmostClient.listShares();
return jsonContent(result);
});
// Tool: move_page
// Schema + description now live in the shared registry (#294). The execute body
// keeps this transport's cycle guard, its 'null'/'' -> null string coercion, and
// its positive-confirmation check on the move response.
registerShared(
SHARED_TOOL_SPECS.movePage,
async ({ pageId, parentPageId, position }) => {
const finalParentId =
parentPageId === "" || parentPageId === "null" ? null : parentPageId;
// Cheap cycle guard: a page cannot be moved directly under itself.
// (Deeper descendant-cycle detection is intentionally out of scope.)
if (finalParentId !== null && finalParentId === pageId) {
throw new Error("cannot move a page under itself");
}
const result = await docmostClient.movePage(
pageId,
finalParentId || null,
position,
);
// Require POSITIVE confirmation: the live /pages/move success shape is
// exactly { success: true, status: 200 }. An empty body, a 204, or any odd
// shape lacking success === true must NOT be reported as a successful move,
// so we surface the raw API result instead of declaring success.
if (!(result && typeof result === "object" && result.success === true)) {
throw new Error(
`Failed to move page ${pageId}: ${JSON.stringify(result)}`,
);
}
return jsonContent({
message: `Successfully moved page ${pageId} to parent ${finalParentId || "root"}`,
result,
});
},
);
// Tool: delete_page
// Schema + description now live in the shared registry (#294). The shared schema
// exposes ONLY pageId, so no permanent/force-delete flag can reach the client.
registerShared(SHARED_TOOL_SPECS.deletePage, async ({ pageId }) => {
await docmostClient.deletePage(pageId);
return {
content: [
{ type: "text" as const, text: `Successfully deleted page ${pageId}` },
],
};
});
// --- Comment tools (ported from upstream PR #3 by Max Nikitin) ---
// Tool: list_comments
registerShared(
SHARED_TOOL_SPECS.listComments,
async ({ pageId, includeResolved }) => {
const comments = await docmostClient.listComments(pageId, includeResolved);
return jsonContent(comments);
},
);
// Tool: create_comment
// Schema + description now live in the shared registry (#294). The execute body
// keeps this transport's own guards (require a selection for a top-level
// comment; reject suggestedText on a reply / without a selection).
registerShared(
SHARED_TOOL_SPECS.createComment,
async ({ pageId, content, selection, parentCommentId, suggestedText }) => {
if (!parentCommentId && (!selection || !selection.trim())) {
throw new Error(
"create_comment: a 'selection' (exact text to anchor on) is required for a top-level comment; omit it only when replying via parentCommentId.",
);
}
if (suggestedText !== undefined) {
if (parentCommentId) {
throw new Error(
"create_comment: 'suggestedText' cannot be attached to a reply; it applies only to a top-level inline comment.",
);
}
if (!selection || !selection.trim()) {
throw new Error(
"create_comment: 'suggestedText' requires a 'selection' to anchor and rewrite.",
);
}
}
const result = await docmostClient.createComment(
pageId,
content,
"inline",
selection,
parentCommentId,
suggestedText,
);
return jsonContent(result);
},
);
// Tool: update_comment
server.registerTool(
"update_comment",
"updateComment",
{
description:
"Update an existing comment's content. Only the comment creator can " +
@@ -770,9 +409,9 @@ server.registerTool(
},
);
// Tool: delete_comment
// Tool: deleteComment
server.registerTool(
"delete_comment",
"deleteComment",
{
description:
"Delete a comment. Only the comment creator or space admin can delete it.",
@@ -793,77 +432,77 @@ server.registerTool(
},
);
// Tool: resolve_comment
// Schema + description now live in the shared registry (#294).
registerShared(
SHARED_TOOL_SPECS.resolveComment,
async ({ commentId, resolved }) => {
const result = await docmostClient.resolveComment(commentId, resolved);
return jsonContent(result);
},
);
// Tool: check_new_comments
// Schema + description now live in the shared registry (#294). The execute body
// keeps this transport's own guard rejecting an unparseable `since` timestamp.
registerShared(
SHARED_TOOL_SPECS.checkNewComments,
async ({ spaceId, since, parentPageId }) => {
// Reject an unparseable timestamp up front: otherwise the comparison
// against NaN silently treats every comment as "not new" and the tool
// returns zero results without signalling the bad input.
if (Number.isNaN(Date.parse(since))) {
throw new Error(
`Invalid 'since' timestamp: ${JSON.stringify(since)} — expected an ISO 8601 date (e.g. '2026-03-10T00:00:00Z')`,
);
}
const result = await docmostClient.checkNewComments(
spaceId,
since,
parentPageId,
);
return jsonContent(result);
},
);
// Tool: search
// INTENTIONAL per-transport divergence (not shared): the in-app `searchPages`
// runs a semantic + keyword hybrid (RRF) with in-process access control and a
// different schema (limit 1-20); this transport is a plain REST full-text search
// (limit up to 100). Different behaviour AND schema, so kept per-layer.
// different schema; this transport is the #443 agent-lookup search — a hybrid
// substring + full-text search that also returns each hit's location (`path`)
// and a windowed `snippet`, so one call answers "where is it and what's in it".
// The in-app hybrid-RRF search is deliberately NOT touched. Different behaviour
// AND schema, so kept per-layer.
//
// STANDALONE-vs-STOCK-UPSTREAM: the client sends the opt-in `substring`/
// `parentPageId`/`titleOnly` DTO fields. A stock upstream server validates the
// DTO with `whitelist: true` and silently strips these unknown fields, so the
// request degrades gracefully to plain FTS (no path/snippet, current shape).
//
// EE/TYPESENSE DEGRADATION (#443): on an instance whose SEARCH_DRIVER is
// `typesense`, the server routes this request to the Typesense backend, which
// does NOT implement agent-lookup — the substring/path/snippet/tiering is
// ignored and the response degrades to plain Typesense FTS. The rich lookup
// shape is only produced by the native Postgres search driver.
server.registerTool(
"search",
{
description:
"Full-text search for pages and content across the whole workspace. " +
"Results are bounded by `limit` (1-100; when omitted the server applies " +
"its own default).",
"Find pages by a fragment of a technical string (hostnames, IPs, IDs " +
"like `srv.local`, `10.0.12`, `WB-MGE-30D86B`) — one call returns each " +
"hit's location (`path`: ancestor titles root→parent) and a `snippet` " +
"around the first match, so you rarely need a follow-up get_page. " +
"Matches substrings literally (dots/dashes/digits are not tokenized) as " +
"well as full-text. Returns `{ pageId, title, path, snippet, score }` " +
"sorted by `score` (a per-response relevance float).",
inputSchema: {
query: z.string().min(1).describe("Search query"),
spaceId: z
.string()
.optional()
.describe("Restrict the search to a single space"),
parentPageId: z
.string()
.optional()
.describe(
"Restrict to a page and all its descendants (the page itself included)",
),
titleOnly: z
.boolean()
.optional()
.describe("Match page titles only; skip page text"),
limit: z
.number()
.int()
.min(1)
.max(100)
.max(50)
.optional()
.describe("Max results to return (max 100)"),
.describe("Max results to return (1-50, default 10)"),
},
},
async ({ query, limit }) => {
// The tool exposes no spaceId filter, so pass undefined for the client's
// optional spaceId parameter and forward limit into its correct slot.
const result = await docmostClient.search(query, undefined, limit);
async ({ query, spaceId, parentPageId, titleOnly, limit }) => {
const result = await docmostClient.search(query, spaceId, limit, {
parentPageId,
titleOnly,
});
return jsonContent(result);
},
);
// Tool: docmost_transform
// Tool: docmostTransform
// INTENTIONAL per-transport divergence (not shared): the in-app `transformPage`
// deliberately omits the `deleteComments` schema field (comment-deletion
// guardrail) and carries a much shorter description; this transport exposes the
// full helper catalogue. Different schema, so kept per-layer.
server.registerTool(
"docmost_transform",
"docmostTransform",
{
description:
"Edit a page by running an arbitrary JS transform `(doc, ctx) => doc` " +
@@ -935,43 +574,5 @@ server.registerTool(
},
);
// Tool: insert_footnote
// Schema + description now live in the shared registry (#410) so the in-app
// AI-chat agent exposes it too. The execute body is unchanged.
registerShared(
SHARED_TOOL_SPECS.insertFootnote,
async ({ pageId, anchorText, text }) => {
const result = await docmostClient.insertFootnote(pageId, anchorText, text);
return jsonContent(result);
},
);
// Tool: diff_page_versions
registerShared(
SHARED_TOOL_SPECS.diffPageVersions,
async ({ pageId, from, to }) => {
const result = await docmostClient.diffPageVersions(pageId, from, to);
return jsonContent(result);
},
);
// Tool: list_page_history
registerShared(
SHARED_TOOL_SPECS.listPageHistory,
async ({ pageId, cursor }) => {
const result = await docmostClient.listPageHistory(pageId, cursor);
return jsonContent(result);
},
);
// Tool: restore_page_version
registerShared(
SHARED_TOOL_SPECS.restorePageVersion,
async ({ historyId }) => {
const result = await docmostClient.restorePageVersion(historyId);
return jsonContent(result);
},
);
return server;
}
+29 -4
View File
@@ -202,6 +202,21 @@ export class CollabSession {
this.ydoc = new Y.Doc();
}
/**
* Shared diagnostic suffix (issue #437) appended to the connect-timeout,
* persist-timeout and connection-closed error texts: names the offending
* pageId and tells the agent this class of failure is transient (retry once)
* vs. a persistent collab-server outage, so it can self-correct instead of
* blind-looping. The Yjs-encode error is deliberately NOT touched it
* already names the offending attribute.
*/
private hint(): string {
return (
`(pageId ${this.pageId}; transient — retry once; persistent failures ` +
`mean the collab server is unreachable/overloaded)`
);
}
/**
* A cached session may be reused only when it is fully ready, still synced,
* has not lost its connection, and has not exceeded its max age (invariant 5
@@ -232,7 +247,9 @@ export class CollabSession {
// The 25s connect timeout: the collab connection never became ready.
this.opts?.onConnectTimeout?.();
this.teardown(
new Error("Connection timeout to collaboration server"),
new Error(
`Connection timeout to collaboration server ${this.hint()}`,
),
false,
);
}, CONNECT_TIMEOUT_MS);
@@ -259,7 +276,7 @@ export class CollabSession {
if (process.env.DEBUG) console.error("WS Disconnect");
this.teardown(
new Error(
"Collaboration connection closed before the update was persisted/synced",
`Collaboration connection closed before the update was persisted/synced ${this.hint()}`,
),
true,
);
@@ -268,7 +285,7 @@ export class CollabSession {
if (process.env.DEBUG) console.error("WS Close");
this.teardown(
new Error(
"Collaboration connection closed before the update was persisted/synced",
`Collaboration connection closed before the update was persisted/synced ${this.hint()}`,
),
true,
);
@@ -403,7 +420,7 @@ export class CollabSession {
persistTimer = setTimeout(() => {
localFinish(
new Error(
"Timeout waiting for collaboration server to persist the update",
`Timeout waiting for collaboration server to persist the update ${this.hint()}`,
),
);
}, PERSIST_TIMEOUT_MS);
@@ -423,9 +440,16 @@ export class CollabSession {
// must stay synchronous (no await). While the JS event loop is not
// yielded, no incoming remote update can interleave, so any already-synced
// concurrent edits are preserved in liveDoc.
//
// INVARIANT 1 is machine-checked: the BEGIN/END markers below delimit the
// no-await window, and test/unit/no-await-critical-window.test.mjs scans
// this source and FAILS if any `await` (or `for await`/`yield`) appears
// between them. Do NOT add an await inside this block — an accidental
// async boundary here silently reopens the clobber-live-edits race (#152).
let newDoc: any;
let beforeDoc: any;
try {
// === MUTATE-CRITICAL-WINDOW: BEGIN (no await between here and END #449) ===
let liveDoc = TiptapTransformer.fromYdoc(this.ydoc, "default");
if (
!liveDoc ||
@@ -463,6 +487,7 @@ export class CollabSession {
// ids of unchanged nodes, so an open editor's cursor is not yanked to the
// end of the document on every agent write.
applyDocToFragment(this.ydoc, newDoc);
// === MUTATE-CRITICAL-WINDOW: END (#449) ===
} catch (e) {
// Includes errors thrown by transform (e.g. "afterText not found",
// "text not found"): propagate them verbatim to the caller.
+23 -5
View File
@@ -13,7 +13,11 @@ import { JSDOM } from "jsdom";
import { markdownToProseMirror } from "@docmost/prosemirror-markdown";
import { docmostExtensions, docmostSchema } from "./docmost-schema.js";
import { withPageLock } from "./page-lock.js";
import { sanitizeForYjs, findUnstorableAttr } from "@docmost/prosemirror-markdown";
import {
sanitizeForYjs,
findUnstorableAttr,
findInvalidNode,
} from "@docmost/prosemirror-markdown";
import { canonicalizeFootnotes } from "./footnote-canonicalize.js";
import { normalizeAndMergeFootnotes } from "./footnote-normalize-merge.js";
import { VerifyReport } from "./diff.js";
@@ -28,11 +32,25 @@ export { markdownToProseMirror };
* place. `label` names the stage that failed (diagnostic). `sanitizeForYjs`
* already stripped `undefined` attrs, so a remaining failure is pinpointed via
* `findUnstorableAttr`.
*
* Diagnostics precedence (#409): the dominant crash here is
* `Unknown node type: undefined` a nested node with an absent/unknown `type`
* (a SHAPE problem, e.g. `{"text":"foo"}` missing `"type":"text"`). That points
* at the node, not an attribute, so `findInvalidNode` is consulted FIRST and,
* on a hit, yields a path-anchored node-shape message. Only when the document
* shape is sound do we fall back to `findUnstorableAttr` (undefined/function/
* symbol/bigint attr values); the generic "attribute likely holds a value Yjs
* cannot store" sentence is the last resort.
*/
function unstorableYjsError(safe: any, label: string, e: unknown): Error {
const base = `Failed to encode document to Yjs (${label}): ${e instanceof Error ? e.message : String(e)}.`;
const badNode = findInvalidNode(safe);
if (badNode) {
return new Error(`${base} Invalid node: ${badNode.summary}`);
}
const bad = findUnstorableAttr(safe);
return new Error(
`Failed to encode document to Yjs (${label}): ${e instanceof Error ? e.message : String(e)}.${bad ? ` Offending attribute: ${bad}.` : " A node/mark attribute likely holds a value Yjs cannot store (e.g. undefined)."}`,
`${base}${bad ? ` Offending attribute: ${bad}.` : " A node/mark attribute likely holds a value Yjs cannot store (e.g. undefined)."}`,
);
}
@@ -69,8 +87,8 @@ global.WebSocket = WebSocket;
* bodies merged. So the import output is ALREADY in canonical footnote
* topology.
* - `canonicalizeFootnotes` runs AFTER as the mcp write-path invariant shared
* with every other full-document persist path (`update_page_json`,
* `docmost_transform`, `insert_footnote`, ). Because the package output is
* with every other full-document persist path (`updatePageJson`,
* `docmostTransform`, `insertFootnote`, ). Because the package output is
* already canonical, this layer is a no-op here (idempotent) it exists so
* the page-write contract is enforced uniformly regardless of how the PM doc
* was produced, not because the import needs fixing.
@@ -264,7 +282,7 @@ export async function mutatePageContent(
* it was produced from markdown (ids regenerate) or edited in place
* (existing block ids preserved).
*
* This is an intentional full replace (used by update_page / update_page_json),
* This is an intentional full replace (used by update_page / updatePageJson),
* but now runs under the per-page lock and waits for server persistence via
* mutatePageContent.
*/
+1 -1
View File
@@ -20,7 +20,7 @@
*
* MARKDOWN-STRIP FALLBACK: when the agent copies a selection that still carries
* inline markdown (`**bold**`, `` `code` ``, `[t](u)`), the raw locator will not
* match the document's plain text. Exactly like edit_page_text's json-edit
* match the document's plain text. Exactly like editPageText's json-edit
* fallback, we first try the verbatim selection and, ONLY if it anchors nowhere
* in the whole document, retry with `stripInlineMarkdown` applied. `canAnchorInDoc`,
* `getAnchoredText` and `applyAnchorInDoc` share this decision via
+163 -53
View File
@@ -14,7 +14,19 @@
* signature.
*
* If recreateTransform / the changeset throws on a pathological document pair,
* we fall back to a coarse block-level text diff so the tool never hard-fails.
* OR the pair is too large to diff cheaply (see the size guard below), we fall
* back to a coarse block-level text diff so the tool never hard-fails and never
* pins the event loop.
*
* SIZE GUARD (issue #464 prod CPU-DoS). recreateTransform computes its diff via
* rfc6902.createPatch, whose array diff is O(n·m) Levenshtein per array pair and
* whose per-run word diff is O(w²); on a large/heavily-changed doc this runs for
* seconds-to-hours and starves the whole process (BullMQ, Redis lock renewals,
* embeddings). It never THROWS it just never finishes so the try/catch below
* cannot save us. Because diffDocs runs on EVERY in-app/MCP content edit's verify
* report, we PRE-FLIGHT the doc size and route anything above a cheap cap straight
* to the coarse fallback (the same shape the catch produces). Same cap+fallback
* pattern as the ELK-layout DoS fix (#440 / c917dcc3).
*/
import { Node } from "@tiptap/pm/model";
@@ -72,6 +84,56 @@ function countNodes(doc: any, pred: (node: any) => boolean): number {
return n;
}
// --- Issue #464: pre-flight size guard for the precise diff ------------------
// Defaults are BENCHMARK-derived on the recreateTransform(complexSteps:false,
// wordDiffs:true, simplifyDiff:true) pipeline, chosen so the WORST case (a fully
// re-written doc — the adversarial shape that drove the incident) keeps the
// synchronous block under ~200ms REGARDLESS of input:
// - 150 total nodes: worst-case pair ~176ms; the O(node²) array diff crosses
// 200ms at ~170 nodes and then explodes super-linearly (400 nodes ~1.3s,
// 800 ~5.5s), so cap just below the crossover.
// - 12 KiB serialized JSON: an independent axis, because the per-run word diff
// is O(words²) — a FEW nodes with very long text runs is dangerous even at a
// low node count (17 nodes / ~11 KiB ~176ms, / ~14 KiB ~290ms). A node-light
// but byte-heavy doc is still refused.
// Either metric over its cap routes to the coarse fallback. Both are env-tunable
// for operators who accept more CPU in exchange for exact diffs on larger docs.
const DEFAULT_MAX_NODES = 150;
const DEFAULT_MAX_BYTES = 12 * 1024;
/**
* Read a positive-integer env override, falling back to `dflt`. Garbage / unset /
* non-finite / non-positive all fall back (so the guard can never be accidentally
* disabled by a malformed value). Read fresh on every call so a test / operator
* can flip the knob without a restart.
*/
function readPositiveIntEnv(name: string, dflt: number): number {
const raw = parseInt(process.env[name] ?? "", 10);
return Number.isFinite(raw) && raw > 0 ? raw : dflt;
}
/**
* True when the pair is too large for the precise (recreateTransform) diff and
* must degrade to the coarse fallback. Takes the MAX of the two docs on each
* metric so an ASYMMETRIC pair (a small new doc vs a huge old doc, or vice
* versa) which still explodes rfc6902 is caught. Cheap: one node walk +
* one JSON.stringify per doc, both O(size).
*/
function exceedsDiffSizeGuard(oldDoc: any, newDoc: any): boolean {
const maxNodes = readPositiveIntEnv("MCP_DIFF_MAX_NODES", DEFAULT_MAX_NODES);
const maxBytes = readPositiveIntEnv("MCP_DIFF_MAX_BYTES", DEFAULT_MAX_BYTES);
const nodes = Math.max(
countNodes(oldDoc, () => true),
countNodes(newDoc, () => true),
);
if (nodes > maxNodes) return true;
const bytes = Math.max(
JSON.stringify(oldDoc)?.length ?? 0,
JSON.stringify(newDoc)?.length ?? 0,
);
return bytes > maxBytes;
}
/**
* Count UNIQUE links in a JSON doc by their `href`. A single link can be split
* across several adjacent text runs (e.g. a "link+bold" run followed by a "link"
@@ -226,6 +288,81 @@ function coarseDiff(oldDoc: any, newDoc: any): DiffChange[] {
return changes;
}
/** Accumulated textual changes plus their derived char/block tallies. */
interface DiffTally {
changes: DiffChange[];
inserted: number;
deleted: number;
changedBlocks: Set<string>;
}
/**
* Produce the coarse-fallback tally for a pair. This is the SINGLE source of the
* `fellBack:true` result shape, shared by BOTH degrade paths in diffDocs (the
* pre-flight size guard and the recreateTransform catch) so they behave and
* report identically.
*/
function coarseDiffTally(oldDoc: any, newDoc: any): DiffTally {
const changes = coarseDiff(oldDoc, newDoc);
let inserted = 0;
let deleted = 0;
const changedBlocks = new Set<string>();
for (const c of changes) {
if (c.op === "insert") inserted += c.text.length;
else deleted += c.text.length;
if (c.block) changedBlocks.add(c.op[0] + ":" + c.block);
}
return { changes, inserted, deleted, changedBlocks };
}
/**
* Compute the PRECISE tally via the recreateTransform pipeline. Callers MUST
* gate this behind the size guard (it can block the event loop for a large pair)
* and wrap it in try/catch (a pathological pair can throw); on either the guard
* or a throw, use `coarseDiffTally` instead. Kept as a sibling of
* `coarseDiffTally` so both produce the same `DiffTally` shape.
*/
function preciseDiffTally(oldDocJson: any, newDocJson: any): DiffTally {
const oldNode = Node.fromJSON(docmostSchema, oldDocJson);
const newNode = Node.fromJSON(docmostSchema, newDocJson);
const tr = recreateTransform(oldNode, newNode, {
complexSteps: false,
wordDiffs: true,
simplifyDiff: true,
});
const changeSet = ChangeSet.create(oldNode).addSteps(tr.doc, tr.mapping.maps, []);
const simplified = simplifyChanges(changeSet.changes, newNode);
const changes: DiffChange[] = [];
let inserted = 0;
let deleted = 0;
const changedBlocks = new Set<string>();
for (const change of simplified) {
// Deleted text lives in the OLD doc coordinate range [fromA, toA).
if (change.toA > change.fromA) {
const text = oldNode.textBetween(change.fromA, change.toA, "\n", " ");
if (text.length > 0) {
deleted += text.length;
const block = blockContextAt(oldNode, change.fromA);
changes.push({ op: "delete", block, text });
if (block) changedBlocks.add("d:" + block);
}
}
// Inserted text lives in the NEW doc coordinate range [fromB, toB).
if (change.toB > change.fromB) {
const text = newNode.textBetween(change.fromB, change.toB, "\n", " ");
if (text.length > 0) {
inserted += text.length;
const block = blockContextAt(newNode, change.fromB);
changes.push({ op: "insert", block, text });
if (block) changedBlocks.add("i:" + block);
}
}
}
return { changes, inserted, deleted, changedBlocks };
}
/** Build the human-readable unified-ish markdown summary. */
function renderMarkdown(
result: Omit<DiffResult, "markdown">,
@@ -276,66 +413,39 @@ export function diffDocs(
newDocJson: any,
notesHeading: string = "Примечания переводчика",
): DiffResult {
// computeIntegrity is cheap (linear node walks) and its counts are needed in
// BOTH the precise and coarse paths, so it always runs first.
const integrity = computeIntegrity(oldDocJson, newDocJson, notesHeading);
let changes: DiffChange[] = [];
let inserted = 0;
let deleted = 0;
let fellBack = false;
const changedBlocks = new Set<string>();
let tally: DiffTally;
try {
const oldNode = Node.fromJSON(docmostSchema, oldDocJson);
const newNode = Node.fromJSON(docmostSchema, newDocJson);
const tr = recreateTransform(oldNode, newNode, {
complexSteps: false,
wordDiffs: true,
simplifyDiff: true,
});
const changeSet = ChangeSet.create(oldNode).addSteps(
tr.doc,
tr.mapping.maps,
[],
);
const simplified = simplifyChanges(changeSet.changes, newNode);
for (const change of simplified) {
// Deleted text lives in the OLD doc coordinate range [fromA, toA).
if (change.toA > change.fromA) {
const text = oldNode.textBetween(change.fromA, change.toA, "\n", " ");
if (text.length > 0) {
deleted += text.length;
const block = blockContextAt(oldNode, change.fromA);
changes.push({ op: "delete", block, text });
if (block) changedBlocks.add("d:" + block);
}
}
// Inserted text lives in the NEW doc coordinate range [fromB, toB).
if (change.toB > change.fromB) {
const text = newNode.textBetween(change.fromB, change.toB, "\n", " ");
if (text.length > 0) {
inserted += text.length;
const block = blockContextAt(newNode, change.fromB);
changes.push({ op: "insert", block, text });
if (block) changedBlocks.add("i:" + block);
}
}
}
} catch {
// Pathological pair: degrade to a coarse block-level diff so we never throw.
// Pre-flight size guard (#464): a too-large pair would make recreateTransform
// block the event loop for seconds-to-hours WITHOUT throwing, so route it to
// the coarse fallback BEFORE calling recreateTransform at all. Both this path
// and the catch below go through coarseDiffTally for an identical `fellBack`
// result shape.
if (exceedsDiffSizeGuard(oldDocJson, newDocJson)) {
fellBack = true;
changes = coarseDiff(oldDocJson, newDocJson);
for (const c of changes) {
if (c.op === "insert") inserted += c.text.length;
else deleted += c.text.length;
if (c.block) changedBlocks.add(c.op[0] + ":" + c.block);
tally = coarseDiffTally(oldDocJson, newDocJson);
} else {
try {
tally = preciseDiffTally(oldDocJson, newDocJson);
} catch {
// Pathological pair: degrade to a coarse block-level diff so we never throw.
fellBack = true;
tally = coarseDiffTally(oldDocJson, newDocJson);
}
}
const partial: Omit<DiffResult, "markdown"> = {
summary: { inserted, deleted, blocksChanged: changedBlocks.size },
summary: {
inserted: tally.inserted,
deleted: tally.deleted,
blocksChanged: tally.changedBlocks.size,
},
integrity,
changes,
changes: tally.changes,
};
return { ...partial, markdown: renderMarkdown(partial, fellBack) };
}
@@ -380,7 +490,7 @@ export interface VerifyReport {
/**
* ONLY structural integrity types whose count changed, as [before, after]
* (images/links/tables/callouts). Surfaces structural mutations that touch
* neither text nor marks (e.g. insert_image, deleting a table) which diffDocs
* neither text nor marks (e.g. insertImage, deleting a table) which diffDocs
* being TEXT-only would otherwise report as "no content change".
*/
structure?: Record<string, [number, number]>;
@@ -400,7 +510,7 @@ export interface VerifyReport {
*
* The structural integrity delta (from diffDocs's `integrity` tuples) is what
* makes `changed` true for an image/table/callout/link count change that diffs
* to zero text closing a verify blind spot for insert_image, delete_node on a
* to zero text closing a verify blind spot for insertImage, deleteNode on a
* table, etc.
*/
export function summarizeChange(before: any, after: any): VerifyReport {
+168
View File
@@ -0,0 +1,168 @@
// ID-based cell operations for `drawioEditCells` (issue #425, stage 3).
//
// Instead of resending the whole XML (whose diff is fragile — draw.io reorders
// attributes, and a {search,replace} text match breaks on it), the model sends
// targeted operations keyed by cell id:
//
// { op: "add", xml: "<mxCell .../>" } // append a new cell
// { op: "update", cellId: "n3", xml: "<mxCell .../>" } // replace that cell
// { op: "delete", cellId: "n5" } // + CASCADE
//
// `delete` CASCADES: it removes the cell, every descendant cell whose parent
// chain leads to it (container children), AND every edge whose source or target
// is any deleted cell. Ids are STABLE across edits so diffs stay meaningful.
//
// Operations apply to the parsed DOM of the current model; the caller re-lints
// and rebuilds the .drawio.svg through the existing #423 pipeline afterwards.
import { JSDOM } from "jsdom";
let _window: any = null;
function xmlWindow(): any {
if (!_window) _window = new JSDOM("").window;
return _window;
}
export type CellOp =
| { op: "add"; xml: string }
| { op: "update"; cellId: string; xml: string }
| { op: "delete"; cellId: string };
export class CellOpsError extends Error {
constructor(message: string) {
super(`drawioEditCells: ${message}`);
this.name = "CellOpsError";
}
}
// The mxGraph root sentinels. id="0" is the graph root; id="1" is the default
// layer that parents every real cell. A delete targeting either would cascade
// through the whole diagram body (every cell chains up to "1"), so such an op is
// rejected outright.
const SENTINEL_IDS = new Set(["0", "1"]);
/** Parse a single `<mxCell …>…</mxCell>` fragment into an element, or throw. */
function parseCellFragment(xml: string): any {
const parser = new (xmlWindow().DOMParser)();
// Wrap so a self-closed or child-bearing single cell parses as one root.
const doc = parser.parseFromString(`<root>${xml}</root>`, "application/xml");
if (doc.getElementsByTagName("parsererror").length > 0) {
throw new CellOpsError(`operation xml is not well-formed: ${xml.slice(0, 120)}`);
}
const cells = doc.getElementsByTagName("mxCell");
if (cells.length !== 1) {
throw new CellOpsError(
`each add/update op must carry exactly one <mxCell> (got ${cells.length})`,
);
}
return cells[0];
}
/** All ids reachable as descendants of `rootId` via the parent relation. */
function collectDescendants(
rootId: string,
parentOf: Map<string, string | undefined>,
): Set<string> {
const doomed = new Set<string>([rootId]);
let grew = true;
while (grew) {
grew = false;
for (const [id, parent] of parentOf) {
if (!doomed.has(id) && parent != null && doomed.has(parent)) {
doomed.add(id);
grew = true;
}
}
}
return doomed;
}
/**
* Apply the operation list to a model XML string and return the new model XML.
* Uses the DOM so attribute order / formatting is preserved for untouched cells.
* Throws CellOpsError on an unknown target id or a malformed op fragment (so the
* model gets a precise error and nothing is half-applied).
*/
export function applyCellOps(modelXml: string, ops: CellOp[]): string {
if (!Array.isArray(ops) || ops.length === 0) {
throw new CellOpsError("operations must be a non-empty array");
}
const parser = new (xmlWindow().DOMParser)();
const doc = parser.parseFromString(modelXml, "application/xml");
if (doc.getElementsByTagName("parsererror").length > 0) {
throw new CellOpsError("the current diagram XML is not well-formed");
}
const root = doc.getElementsByTagName("root")[0];
if (!root) throw new CellOpsError("the current diagram has no <root> element");
const cellEls = () => Array.from(root.getElementsByTagName("mxCell")) as any[];
const byId = () => {
const m = new Map<string, any>();
for (const el of cellEls()) m.set(el.getAttribute("id") ?? "", el);
return m;
};
for (const op of ops) {
if (op.op === "add") {
const frag = parseCellFragment(op.xml);
const id = frag.getAttribute("id");
if (!id) throw new CellOpsError("an add op's <mxCell> is missing an id");
if (byId().has(id))
throw new CellOpsError(`add op id "${id}" already exists (use update)`);
root.appendChild(doc.importNode(frag, true));
} else if (op.op === "update") {
const map = byId();
const target = map.get(op.cellId);
if (!target)
throw new CellOpsError(`update target cell "${op.cellId}" does not exist`);
const frag = parseCellFragment(op.xml);
const newId = frag.getAttribute("id");
if (newId && newId !== op.cellId)
throw new CellOpsError(
`update op cellId "${op.cellId}" != the <mxCell> id "${newId}" (ids are stable)`,
);
// Replace the element in place so surrounding cells are untouched.
const imported = doc.importNode(frag, true);
target.parentNode.replaceChild(imported, target);
} else if (op.op === "delete") {
// Reject a sentinel-targeted delete BEFORE collecting descendants: "0"/"1"
// parent the entire diagram, so a cascade from either would wipe the whole
// model body (doomed.delete("0"/"1") only spared the sentinel itself, not
// its children).
if (SENTINEL_IDS.has(op.cellId))
throw new CellOpsError(
`cannot delete sentinel cell "${op.cellId}" (the graph root/default layer)`,
);
const map = byId();
if (!map.has(op.cellId))
throw new CellOpsError(`delete target cell "${op.cellId}" does not exist`);
// Build the parent relation over the CURRENT cells for the cascade.
const parentOf = new Map<string, string | undefined>();
for (const el of cellEls()) {
parentOf.set(el.getAttribute("id") ?? "", el.getAttribute("parent") ?? undefined);
}
const doomed = collectDescendants(op.cellId, parentOf);
// Cascade to edges whose source/target is any doomed cell.
for (const el of cellEls()) {
if (el.getAttribute("edge") !== "1") continue;
const src = el.getAttribute("source");
const tgt = el.getAttribute("target");
if ((src && doomed.has(src)) || (tgt && doomed.has(tgt))) {
doomed.add(el.getAttribute("id") ?? "");
}
}
// Never delete the sentinels even if referenced by a malformed op.
doomed.delete("0");
doomed.delete("1");
for (const el of cellEls()) {
const id = el.getAttribute("id") ?? "";
if (doomed.has(id)) el.parentNode.removeChild(el);
}
} else {
throw new CellOpsError(`unknown op "${(op as any).op}"`);
}
}
const ser = new (xmlWindow().XMLSerializer)();
return ser.serializeToString(doc.documentElement);
}
+916
View File
@@ -0,0 +1,916 @@
// Semantic graph -> draw.io pipeline for `drawioFromGraph` (issue #425, stage 3).
//
// The model describes a diagram SEMANTICALLY — nodes with a `kind` and an
// optional `icon`, groups (containers), edges with a `kind` — and NEVER sees a
// coordinate or a style string. This module owns the whole server-side pipeline:
//
// 1. validateGraph — a hand-written validator (no zod dependency, so this
// lib stays importable by client.ts without coupling to
// a zod major) that rejects malformed graphs early.
// 2. resolveNodeStyle — `icon` -> exact style via the shape catalog (#424);
// an UNKNOWN icon degrades to a generic shape by `kind`
// WITH the label (never an empty square). `kind` -> the
// preset palette slot.
// 3. graphToElk — graph -> ELK-JSON, honouring the layout hints
// (`layer`/`sameLayerAs` -> layer constraints, `pinned`
// -> a fixed node) and compound group nodes.
// 4. assembleModel — graph + ELK coordinates -> a full mxGraphModel XML
// that satisfies the #423 linter BY CONSTRUCTION
// (sentinels, transparent containers, relative child
// coords, cross-container edges parent="1", >=150px
// gaps from ELK spacing, escaped labels).
//
// The `layout` mode: "full" re-lays everything; "incremental" fixes existing
// coordinates (ELK interactive mode) and places only new nodes; "none" keeps the
// caller-provided/prior coordinates untouched.
import ELK from "elkjs/lib/elk.bundled.js";
import { JSDOM } from "jsdom";
import {
searchShapes,
awsServiceStyle,
type ShapeResult,
} from "./drawio-shapes.js";
import {
getPreset,
genericNodeStyle,
iconNodeStyle,
edgeStyle,
groupStyle,
type PresetData,
} from "./drawio-presets.js";
import { MIN_SHAPE_GAP } from "./drawio-xml.js";
// --- graph schema (plain TS + a hand validator) ----------------------------
export interface GraphNode {
id: string;
label: string;
kind?: string;
/** Icon reference, e.g. "aws:lambda" | "azure:cosmos" | "lambda". */
icon?: string;
/** Group (container) id this node belongs to. */
group?: string;
/** Layer hint (ELK layerChoiceConstraint): 0-based column/row index. */
layer?: number;
/** Put this node in the same layer as another node id. */
sameLayerAs?: string;
/** Fix this node at exact coordinates (an ELK fixed node). */
pinned?: { x: number; y: number };
}
export interface GraphGroup {
id: string;
label: string;
kind?: string;
/** Parent group id — lets a group nest inside another group (e.g. subnet in VPC). */
group?: string;
}
export interface GraphEdge {
from: string;
to: string;
label?: string;
kind?: string;
}
export interface Graph {
nodes: GraphNode[];
groups?: GraphGroup[];
edges?: GraphEdge[];
direction?: "LR" | "RL" | "TB" | "BT";
preset?: string;
}
export type LayoutMode = "none" | "full" | "incremental";
/** A structured validation error (mirrors the drawio linter's shape loosely). */
export class GraphValidationError extends Error {
issues: string[];
constructor(issues: string[]) {
super(`drawioFromGraph: invalid graph — ${issues.join("; ")}`);
this.name = "GraphValidationError";
this.issues = issues;
}
}
const MAX_GRAPH_NODES = 500; // parity with drawio-layout's ELK_MAX_NODES.
// Edge/group caps mirror drawio-layout's ELK_MAX_EDGES. Without an edge cap a
// tiny node set with a huge edge list (e.g. 500 nodes / 200000 edges) passes
// node validation, then graphToElk/runElk exhausts the heap SYNCHRONOUSLY inside
// elk.bundled.js — before the 5s ELK timeout can fire and OUTSIDE it entirely
// for the mapper/assembler — crashing the worker on LLM-authored input. Reject
// the over-limit shape here, before any layout or assembly runs.
export const MAX_GRAPH_EDGES = 1000; // parity with drawio-layout's ELK_MAX_EDGES.
export const MAX_GRAPH_GROUPS = 500; // groups are compound ELK nodes; bound them too.
/**
* Validate the graph structure BEFORE any layout/assembly so the model gets a
* precise, actionable error instead of a corrupt diagram. Throws
* GraphValidationError listing every problem.
*/
export function validateGraph(graph: Graph): void {
const issues: string[] = [];
if (!graph || typeof graph !== "object") {
throw new GraphValidationError(["graph must be an object"]);
}
if (!Array.isArray(graph.nodes) || graph.nodes.length === 0) {
throw new GraphValidationError(["graph.nodes must be a non-empty array"]);
}
// Size caps FIRST (fail fast, before touching per-element loops) so an
// over-limit graph can never reach the layout engine and OOM the worker.
if (graph.nodes.length > MAX_GRAPH_NODES) {
throw new GraphValidationError([
`graph has ${graph.nodes.length} nodes (max ${MAX_GRAPH_NODES})`,
]);
}
if (Array.isArray(graph.edges) && graph.edges.length > MAX_GRAPH_EDGES) {
throw new GraphValidationError([
`graph has ${graph.edges.length} edges (max ${MAX_GRAPH_EDGES})`,
]);
}
if (Array.isArray(graph.groups) && graph.groups.length > MAX_GRAPH_GROUPS) {
throw new GraphValidationError([
`graph has ${graph.groups.length} groups (max ${MAX_GRAPH_GROUPS})`,
]);
}
const nodeIds = new Set<string>();
const groupIds = new Set<string>();
for (const g of graph.groups ?? []) {
if (!g.id) issues.push("a group is missing its id");
else if (groupIds.has(g.id)) issues.push(`duplicate group id "${g.id}"`);
groupIds.add(g.id);
}
for (const n of graph.nodes) {
if (!n.id) issues.push("a node is missing its id");
else if (nodeIds.has(n.id)) issues.push(`duplicate node id "${n.id}"`);
else if (groupIds.has(n.id))
issues.push(`node id "${n.id}" collides with a group id`);
nodeIds.add(n.id);
if (typeof n.label !== "string" || n.label === "")
issues.push(`node "${n.id}" is missing a label`);
if (n.group != null && !groupIds.has(n.group))
issues.push(`node "${n.id}" references unknown group "${n.group}"`);
if (n.pinned != null) {
if (
typeof n.pinned.x !== "number" ||
typeof n.pinned.y !== "number" ||
!Number.isFinite(n.pinned.x) ||
!Number.isFinite(n.pinned.y)
)
issues.push(`node "${n.id}" has an invalid pinned {x,y}`);
}
if (n.layer != null && (!Number.isInteger(n.layer) || n.layer < 0))
issues.push(`node "${n.id}" has an invalid layer (must be a >=0 integer)`);
}
// sameLayerAs must reference an existing node (checked after all ids known).
for (const n of graph.nodes) {
if (n.sameLayerAs != null && !nodeIds.has(n.sameLayerAs))
issues.push(
`node "${n.id}" sameLayerAs references unknown node "${n.sameLayerAs}"`,
);
}
for (const e of graph.edges ?? []) {
if (!e.from || !e.to) {
issues.push("an edge is missing from/to");
continue;
}
if (!nodeIds.has(e.from) && !groupIds.has(e.from))
issues.push(`edge from "${e.from}" resolves to no node/group`);
if (!nodeIds.has(e.to) && !groupIds.has(e.to))
issues.push(`edge to "${e.to}" resolves to no node/group`);
}
if (issues.length > 0) throw new GraphValidationError(issues);
}
// --- icon resolution -------------------------------------------------------
/**
* Resolve a node's `icon` reference to a concrete style-string + size via the
* shape catalog. Accepts "aws:lambda", "azure:cosmos", or a bare "lambda". An
* AWS `resIcon` name is built directly (exact service-icon template). Anything
* else goes through searchShapes. Returns null when nothing resolves the
* caller then falls back to a generic shape by kind (never an empty box).
*/
export function resolveIcon(icon: string): ShapeResult | null {
const raw = icon.trim();
if (raw === "") return null;
let provider = "";
let name = raw;
const colon = raw.indexOf(":");
if (colon !== -1) {
provider = raw.slice(0, colon).trim().toLowerCase();
name = raw.slice(colon + 1).trim();
}
if (provider === "aws") {
// Prefer an exact resIcon match from the catalog (carries the right size and
// any rebrand/blocklist note); if the underscore/space name doesn't hit,
// build the canonical service-icon style directly so it is never an empty box.
const results = searchShapes(name.replace(/_/g, " "), { limit: 5 });
const aws4 = results.find((r) => r.style.includes("mxgraph.aws4"));
if (aws4) return aws4;
return {
style: awsServiceStyle(name.replace(/\s+/g, "_")),
w: 78,
h: 78,
title: name,
type: "vertex",
};
}
// Non-AWS or bare name: fuzzy search the catalog. Take the top vertex hit,
// but REJECT a weak match (the fuzzy scorer can prefix-match an unrelated
// stencil, e.g. "not..." -> "Notebook"); require the hit's title to actually
// share a meaningful token with the query, otherwise degrade to generic-by-kind.
const q = provider ? `${provider} ${name}` : name;
const results = searchShapes(q, { limit: 8 });
const hit = results.find((r) => r.type !== "edge") ?? results[0];
if (!hit) return null;
if (!isRelevantMatch(name, hit.title)) return null;
return hit;
}
/**
* Whether a resolved stencil is a genuine match for the requested icon name (as
* opposed to a loose prefix hit on an unrelated shape). True if any 3+ char
* token of the query appears in the stencil title, or vice-versa.
*/
function isRelevantMatch(name: string, title: string): boolean {
const norm = (s: string) => s.toLowerCase().replace(/[^a-z0-9]+/g, " ").trim();
const qTokens = norm(name).split(/\s+/).filter((t) => t.length >= 3);
if (qTokens.length === 0) return true; // very short names: trust the scorer
const t = norm(title);
const tTokens = new Set(t.split(/\s+/));
for (const qt of qTokens) {
if (tTokens.has(qt)) return true;
if (t.includes(qt)) return true;
for (const tt of tTokens) if (tt.length >= 3 && qt.includes(tt)) return true;
}
return false;
}
/**
* Decide the final style + size for a node. When `icon` resolves, use the icon
* style (overlaid with a dark-preset font fix); otherwise a GENERIC shape by
* `kind` carrying the label. `resolved` reports whether an icon was found (used
* by the acceptance test that asserts no empty squares).
*/
export function resolveNodeStyle(
preset: PresetData,
node: GraphNode,
): { style: string; w: number; h: number; iconResolved: boolean } {
if (node.icon) {
const shape = resolveIcon(node.icon);
if (shape) {
return {
style: iconNodeStyle(preset, shape.style),
w: shape.w,
h: shape.h,
iconResolved: true,
};
}
}
// Generic shape by kind, sized to the label so a long label never overflows.
const w = Math.max(120, estimateLabelWidth(node.label) + 32);
return { style: genericNodeStyle(preset, node.kind), w, h: 60, iconResolved: false };
}
/** Rough rendered width of the longest label line at 12px (~0.6em/glyph). */
function estimateLabelWidth(label: string): number {
const lines = label.split(/\r?\n|&#xa;|<br\s*\/?>/i);
let longest = 0;
for (const l of lines) longest = Math.max(longest, l.trim().length);
return Math.ceil(longest * 12 * 0.6);
}
// --- graph -> ELK-JSON -----------------------------------------------------
interface ElkNode {
id: string;
width?: number;
height?: number;
x?: number;
y?: number;
children?: ElkNode[];
layoutOptions?: Record<string, string>;
}
interface ElkEdge {
id: string;
sources: string[];
targets: string[];
}
interface ElkGraph extends ElkNode {
edges?: ElkEdge[];
}
const ELK_DIRECTION: Record<string, string> = {
LR: "RIGHT",
RL: "LEFT",
TB: "DOWN",
BT: "UP",
};
/** Sizes resolved per node id (from resolveNodeStyle), fed to the ELK mapper. */
export interface NodeSize {
w: number;
h: number;
}
/**
* Build the ELK graph from the semantic graph + resolved node sizes. Compound
* group nodes nest their members (a group may itself nest in another group).
* `only` restricts the graph to a subset of node ids (used by the incremental
* path to lay out ONLY the new nodes). Layout HINTS (`layer`/`sameLayerAs`/
* `pinned`) are NOT encoded as ELK constraints here ELK's constraint knobs are
* unreliable across versions they are enforced deterministically AFTER layout
* by applyHints, which is exact and testable.
*/
export function graphToElk(
graph: Graph,
sizes: Map<string, NodeSize>,
opts: { only?: Set<string> } = {},
): ElkGraph {
const direction = ELK_DIRECTION[graph.direction ?? "LR"] ?? "RIGHT";
const only = opts.only;
const include = (id: string) => !only || only.has(id);
const makeNode = (n: GraphNode): ElkNode => {
const size = sizes.get(n.id) ?? { w: 140, h: 60 };
return { id: n.id, width: size.w, height: size.h };
};
// Group children nest under their group node; ungrouped nodes are roots.
const groupNode = new Map<string, ElkNode>();
const usedGroups = new Set<string>();
for (const g of graph.groups ?? []) {
const size = sizes.get(g.id) ?? { w: 200, h: 150 };
groupNode.set(g.id, {
id: g.id,
width: size.w,
height: size.h,
children: [],
layoutOptions: {
"elk.algorithm": "layered",
"elk.direction": direction,
"elk.padding": "[top=40,left=30,bottom=30,right=30]",
"elk.layered.spacing.nodeNodeBetweenLayers": "170",
"elk.spacing.nodeNode": "170",
},
});
}
const roots: ElkNode[] = [];
for (const n of graph.nodes) {
if (!include(n.id)) continue;
const en = makeNode(n);
if (n.group && groupNode.has(n.group)) {
groupNode.get(n.group)!.children!.push(en);
usedGroups.add(n.group);
} else {
roots.push(en);
}
}
// Nest group nodes into their parent group (a subnet inside a VPC); groups
// with no parent group become roots. Only groups that hold an included node.
const groupIdSet = new Set((graph.groups ?? []).map((g) => g.id));
for (const g of graph.groups ?? []) {
if (only && !usedGroups.has(g.id)) continue;
const en = groupNode.get(g.id)!;
if (g.group && groupIdSet.has(g.group) && g.group !== g.id && (!only || usedGroups.has(g.group))) {
groupNode.get(g.group)!.children!.push(en);
} else {
roots.push(en);
}
}
// Edges: endpoints may be nodes or groups; INCLUDE_CHILDREN spans the nesting.
const validIds = new Set<string>([
...graph.nodes.filter((n) => include(n.id)).map((n) => n.id),
...(graph.groups ?? []).map((g) => g.id),
]);
const edges: ElkEdge[] = [];
(graph.edges ?? []).forEach((e, i) => {
if (!validIds.has(e.from) || !validIds.has(e.to)) return;
edges.push({ id: `e${i}`, sources: [e.from], targets: [e.to] });
});
const rootOptions: Record<string, string> = {
"elk.algorithm": "layered",
"elk.direction": direction,
"elk.hierarchyHandling": "INCLUDE_CHILDREN",
"elk.layered.spacing.nodeNodeBetweenLayers": "170",
"elk.spacing.nodeNode": "170",
"elk.spacing.edgeNode": "40",
"elk.spacing.edgeEdge": "30",
"elk.padding": "[top=20,left=20,bottom=20,right=20]",
};
return { id: "root", layoutOptions: rootOptions, children: roots, edges };
}
/**
* Enforce the layout hints DETERMINISTICALLY on ELK's output (mutates `geo`):
* - `sameLayerAs`: snap the dependent node's LAYER-AXIS coordinate to its
* anchor's, so the pair lands in the same layer (x for LR/RL, y for TB/BT).
* `layer` groups nodes with the same index onto the same anchor coordinate.
* - `pinned`: override the node's coordinate with the exact pinned {x,y}.
* Applied only to top-level (ungrouped) nodes, whose ELK coords are absolute.
*/
export function applyHints(
graph: Graph,
geo: Map<string, { x: number; y: number; w: number; h: number }>,
): void {
const dir = graph.direction ?? "LR";
const layerAxis: "x" | "y" = dir === "TB" || dir === "BT" ? "y" : "x";
// The perpendicular (cross-layer) axis: members snapped onto one layer must be
// spread along THIS axis so they don't stack onto the same point.
const crossAxis: "x" | "y" = layerAxis === "x" ? "y" : "x";
const crossSize: "w" | "h" = crossAxis === "x" ? "w" : "h";
const grouped = new Set(
graph.nodes.filter((n) => n.group).map((n) => n.id),
);
// sameLayerAs / layer: co-assign the layer-axis coordinate.
// Build the effective layer key per node, then pick a representative coord.
const layerKeyOf = new Map<string, string>();
const explicitLayer = new Map<string, number>();
for (const n of graph.nodes) if (n.layer != null) explicitLayer.set(n.id, n.layer);
const byId = new Map(graph.nodes.map((n) => [n.id, n]));
const resolveKey = (n: GraphNode): string | null => {
if (explicitLayer.has(n.id)) return `L${explicitLayer.get(n.id)}`;
const seen = new Set<string>([n.id]);
let cur: GraphNode | undefined = n;
while (cur && cur.sameLayerAs != null && !seen.has(cur.sameLayerAs)) {
seen.add(cur.sameLayerAs);
const t = byId.get(cur.sameLayerAs);
if (!t) break;
if (explicitLayer.has(t.id)) return `L${explicitLayer.get(t.id)}`;
cur = t;
}
// A sameLayerAs chain with no explicit layer: key on the chain's root id.
if (n.sameLayerAs != null) {
let root = n.id;
const s2 = new Set<string>([n.id]);
let c: GraphNode | undefined = n;
while (c && c.sameLayerAs != null && !s2.has(c.sameLayerAs)) {
s2.add(c.sameLayerAs);
root = c.sameLayerAs;
c = byId.get(c.sameLayerAs);
}
return `C${root}`;
}
return null;
};
for (const n of graph.nodes) {
if (grouped.has(n.id)) continue; // group children are relative — skip
const key = resolveKey(n);
if (key) layerKeyOf.set(n.id, key);
}
// Group members of each layer key so we can snap AND spread them together.
const membersOf = new Map<string, string[]>();
for (const n of graph.nodes) {
const key = layerKeyOf.get(n.id);
if (key == null) continue;
if (!geo.has(n.id)) continue;
(membersOf.get(key) ?? membersOf.set(key, []).get(key)!).push(n.id);
}
// For each layer key: snap every member to the FIRST member's layer-axis coord,
// then SPREAD them along the perpendicular (cross-layer) axis with a >=
// MIN_SHAPE_GAP gap. Without the spread, a sameLayerAs chain whose nodes ELK
// happened to give the same cross-axis coordinate would collapse onto one point
// -> shape-overlap + edge-through-shape quality warnings (breaking the
// "0 warnings by construction" guarantee for these AUTO-positioned hints). We
// start from the members' minimum cross-axis coord and stack them with a gap
// of MIN_SHAPE_GAP beyond each shape's cross-axis size.
for (const [key, members] of membersOf) {
if (members.length === 0) continue;
// Snap layer-axis coord to the first member.
const repCoord = geo.get(members[0])![layerAxis];
// Preserve the members' existing relative order along the cross axis so the
// spread stays visually stable, then re-lay them contiguously.
const sorted = [...members].sort(
(a, b) => geo.get(a)![crossAxis] - geo.get(b)![crossAxis],
);
let cursor = geo.get(sorted[0])![crossAxis];
for (const id of sorted) {
const g = geo.get(id)!;
g[layerAxis] = repCoord;
g[crossAxis] = cursor;
cursor += g[crossSize] + MIN_SHAPE_GAP;
}
void key;
}
// pinned: exact override (wins over any layer snap). Explicit user coordinates
// are user intent, but CLAMP to non-negative so an out-of-bounds pin (e.g.
// x:-500) never renders off-canvas. Two user-pinned nodes at the same point is
// user error the server can't silently relocate — the assembler docstring
// documents that explicit pins are user-directed and MAY warn (see #423/#425
// acceptance: the "0 quality-warnings by construction" guarantee is for
// AUTO-LAYOUT, not for coordinates the user pinned by hand).
for (const n of graph.nodes) {
if (!n.pinned) continue;
const px = Math.max(0, n.pinned.x);
const py = Math.max(0, n.pinned.y);
const g = geo.get(n.id);
if (g) {
g.x = px;
g.y = py;
} else {
const sz = { w: 140, h: 60 };
geo.set(n.id, { x: px, y: py, w: sz.w, h: sz.h });
}
}
}
/**
* Incremental variant of applyHints: apply `pinned` only to NEW nodes (those
* absent from `existing`); an existing node's coordinates are NEVER changed
* (acceptance #3). sameLayerAs/layer snapping is intentionally skipped in the
* incremental path moving a new node's layer axis could still be desired, but
* it must never move an existing cell, so we keep the incremental contract
* simple: existing cells are frozen, new pinned nodes honour their pin.
*/
export function applyHintsForNew(
graph: Graph,
geo: Map<string, { x: number; y: number; w: number; h: number }>,
existing: Map<string, { x: number; y: number }>,
): void {
for (const n of graph.nodes) {
if (existing.has(n.id)) continue; // never move an existing cell
if (!n.pinned) continue;
const px = Math.max(0, n.pinned.x); // clamp out-of-bounds pins non-negative
const py = Math.max(0, n.pinned.y);
const g = geo.get(n.id);
if (g) {
g.x = px;
g.y = py;
} else {
geo.set(n.id, { x: px, y: py, w: 140, h: 60 });
}
}
}
// --- layout runner ---------------------------------------------------------
const ELK_TIMEOUT_MS = 5000;
/**
* Run ELK over the mapped graph and return computed geometry per id (coords are
* parent-relative, matching mxGraph's convention for container children). On any
* ELK failure/timeout the returned map is empty and the caller falls back to a
* deterministic grid placement (so the write never fails on a layout hiccup).
*/
export async function runElk(
elk: ElkGraph,
): Promise<Map<string, { x: number; y: number; w: number; h: number }>> {
const geo = new Map<string, { x: number; y: number; w: number; h: number }>();
let timer: ReturnType<typeof setTimeout> | undefined;
try {
const Ctor: any = (ELK as any).default ?? ELK;
const inst = new Ctor();
const timeout = new Promise<never>((_, reject) => {
timer = setTimeout(() => reject(new Error("ELK timed out")), ELK_TIMEOUT_MS);
});
const laid = (await Promise.race([inst.layout(elk as any), timeout])) as ElkGraph;
const walk = (n: ElkNode) => {
if (n.id !== "root") {
geo.set(n.id, {
x: Math.round(n.x ?? 0),
y: Math.round(n.y ?? 0),
w: Math.round(n.width ?? 140),
h: Math.round(n.height ?? 60),
});
}
for (const c of n.children ?? []) walk(c);
};
walk(laid);
} catch {
return new Map(); // best-effort: empty -> caller uses fallback grid.
} finally {
if (timer) clearTimeout(timer);
}
return geo;
}
// --- XML assembler ---------------------------------------------------------
/** Order groups so a parent group always precedes its nested children. */
function topoSortGroups(groups: GraphGroup[], groupIds: Set<string>): GraphGroup[] {
const byId = new Map(groups.map((g) => [g.id, g]));
const out: GraphGroup[] = [];
const done = new Set<string>();
const visit = (g: GraphGroup, stack: Set<string>) => {
if (done.has(g.id)) return;
if (stack.has(g.id)) return; // cycle guard
stack.add(g.id);
if (g.group && groupIds.has(g.group) && g.group !== g.id) {
const parent = byId.get(g.group);
if (parent) visit(parent, stack);
}
stack.delete(g.id);
if (!done.has(g.id)) {
done.add(g.id);
out.push(g);
}
};
for (const g of groups) visit(g, new Set());
return out;
}
function xmlEscapeAttr(s: string): string {
return s
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;")
.replace(/\r\n|\r|\n/g, "&#xa;"); // literal newline -> the linter-approved entity
}
export interface AssembleResult {
modelXml: string;
iconsResolved: number;
iconsMissing: string[];
}
/**
* Assemble the final mxGraphModel XML from the graph + resolved styles/coords.
* Guarantees BY CONSTRUCTION that the #423 linter passes:
* - id=0 and id=1(parent=0) sentinels;
* - each node/group is vertex="1" (containers get container=1 via the style);
* - each edge is edge="1" with a child <mxGeometry relative="1" as="geometry"/>;
* - group children set parent=<groupId> and RELATIVE coords; an edge between
* two different parents is parent="1";
* - labels are XML-escaped and any newline is &#xa;.
* `geo` may be empty (ELK failed) then a deterministic grid is used so the
* output is still valid and non-overlapping (>=170px stride).
*
* QUALITY-WARNING GUARANTEE: the "0 quality-warnings by construction" promise
* holds for AUTO-LAYOUT ELK spacing plus applyHints' cross-axis spread for the
* server-positioned `layer`/`sameLayerAs` hints keep shapes >=MIN_SHAPE_GAP
* apart. It does NOT extend to explicit `pinned` coordinates: those are
* user-directed, so two nodes the user pins to the same/overlapping point are
* user error the server honours verbatim (only clamped non-negative) and MAY
* therefore produce a quality warning.
*/
export function assembleModel(
graph: Graph,
opts: {
preset: PresetData;
styles: Map<string, { style: string; w: number; h: number; iconResolved: boolean }>;
geo: Map<string, { x: number; y: number; w: number; h: number }>;
},
): AssembleResult {
const { preset, styles, geo } = opts;
const groupIds = new Set((graph.groups ?? []).map((g) => g.id));
const nodeById = new Map(graph.nodes.map((n) => [n.id, n]));
// Fallback grid when ELK produced nothing: lay ungrouped nodes on a grid with
// a 190px stride (>150 gap). Grouped nodes/groups are placed inside their group.
const fallback = geo.size === 0;
const gridPos = (i: number) => ({ x: 40 + (i % 5) * 200, y: 40 + Math.floor(i / 5) * 140 });
const cells: string[] = ['<mxCell id="0"/>', '<mxCell id="1" parent="0"/>'];
// Groups first (they are parents of their members). A nested group sets
// parent=<parentGroupId>; emit parents before children so parent-exists holds.
let gi = 0;
const groupGeo = new Map<string, { x: number; y: number; w: number; h: number }>();
const orderedGroups = topoSortGroups(graph.groups ?? [], groupIds);
for (const g of orderedGroups) {
const gg = geo.get(g.id) ?? { ...gridPos(gi++), w: 320, h: 220 };
groupGeo.set(g.id, gg);
const style = groupStyle(preset);
const gParent = g.group && groupIds.has(g.group) && g.group !== g.id ? g.group : "1";
cells.push(
`<mxCell id="${xmlEscapeAttr(g.id)}" value="${xmlEscapeAttr(g.label)}" style="${style}" vertex="1" parent="${xmlEscapeAttr(gParent)}">` +
`<mxGeometry x="${gg.x}" y="${gg.y}" width="${gg.w}" height="${gg.h}" as="geometry"/></mxCell>`,
);
}
// Nodes. A grouped node's coords are RELATIVE to its group (ELK already
// returns child coords relative to the parent; for the fallback grid we place
// children on a small in-group grid).
let ungrouped = (graph.groups?.length ?? 0);
const inGroupIndex = new Map<string, number>();
let iconsResolved = 0;
const iconsMissing: string[] = [];
for (const n of graph.nodes) {
const st = styles.get(n.id)!;
if (n.icon) {
if (st.iconResolved) iconsResolved++;
else iconsMissing.push(n.id);
}
let x: number;
let y: number;
const g = geo.get(n.id);
if (g && !fallback) {
x = g.x;
y = g.y;
} else if (n.group && groupIds.has(n.group)) {
const k = inGroupIndex.get(n.group) ?? 0;
inGroupIndex.set(n.group, k + 1);
x = 30 + (k % 3) * 180;
y = 40 + Math.floor(k / 3) * 120;
} else {
const p = gridPos(ungrouped++);
x = p.x;
y = p.y;
}
const parent = n.group && groupIds.has(n.group) ? n.group : "1";
cells.push(
`<mxCell id="${xmlEscapeAttr(n.id)}" value="${xmlEscapeAttr(n.label)}" style="${st.style}" vertex="1" parent="${xmlEscapeAttr(parent)}">` +
`<mxGeometry x="${x}" y="${y}" width="${st.w}" height="${st.h}" as="geometry"/></mxCell>`,
);
}
// Edges. parent="1" whenever the two endpoints have different container
// parents (or either is a group); otherwise the shared group id.
(graph.edges ?? []).forEach((e, i) => {
const style = edgeStyle(preset, e.kind);
const fromNode = nodeById.get(e.from);
const toNode = nodeById.get(e.to);
const fromParent = fromNode?.group && groupIds.has(fromNode.group) ? fromNode.group : "1";
const toParent = toNode?.group && groupIds.has(toNode.group) ? toNode.group : "1";
const parent = fromParent === toParent ? fromParent : "1";
const label = e.label ? ` value="${xmlEscapeAttr(e.label)}"` : "";
cells.push(
`<mxCell id="ge${i}"${label} style="${style}" edge="1" parent="${xmlEscapeAttr(parent)}" ` +
`source="${xmlEscapeAttr(e.from)}" target="${xmlEscapeAttr(e.to)}">` +
`<mxGeometry relative="1" as="geometry"/></mxCell>`,
);
});
const modelAttrs =
'dx="0" dy="0" grid="1" gridSize="10" page="1" pageWidth="850" pageHeight="1100" adaptiveColors="auto"';
const modelXml = `<mxGraphModel ${modelAttrs}><root>${cells.join("")}</root></mxGraphModel>`;
return { modelXml, iconsResolved, iconsMissing };
}
// --- incremental merge -----------------------------------------------------
let _mergeWindow: any = null;
function mergeWindow(): any {
if (!_mergeWindow) _mergeWindow = new JSDOM("").window;
return _mergeWindow;
}
/**
* Merge the freshly-assembled graph XML with the EXISTING diagram model so an
* incremental "add a node" call never drops a hand-placed cell. `assembleModel`
* emits ONLY the passed graph's cells; on its own it would replace the whole
* model, wiping any existing cell the caller didn't re-list. This splices every
* existing cell that the graph does NOT re-list (preserved verbatim: coords,
* style, edges) into the assembled root:
* - id in the graph -> the graph's (re-laid) cell wins (already assembled;
* coords are frozen for existing ids via the incremental geo path);
* - id NOT in the graph -> the existing cell is preserved verbatim;
* - a graph node absent from the existing model -> added (offset clear).
* The sentinels ("0"/"1") come from the assembled model and are never doubled.
*/
function mergeExistingCells(
assembledXml: string,
existingModelXml: string,
graph: Graph,
): string {
const win = mergeWindow();
const parser = new win.DOMParser();
const existingDoc = parser.parseFromString(existingModelXml, "application/xml");
if (existingDoc.getElementsByTagName("parsererror").length > 0) {
// Existing model unreadable: fall back to the assembled model alone (still a
// valid diagram — better than throwing on a corrupt prior file).
return assembledXml;
}
const assembledDoc = parser.parseFromString(assembledXml, "application/xml");
const root = assembledDoc.getElementsByTagName("root")[0];
if (!root) return assembledXml;
// Ids the assembled model already emitted (graph nodes/groups/edges + sentinels).
const assembledIds = new Set<string>();
for (const el of Array.from(root.getElementsByTagName("mxCell")) as any[]) {
const id = el.getAttribute("id");
if (id) assembledIds.add(id);
}
// The graph's own ids: any existing cell with one of these is superseded by the
// assembled version and must NOT be re-imported.
const graphIds = new Set<string>([
...graph.nodes.map((n) => n.id),
...(graph.groups ?? []).map((g) => g.id),
]);
const existingCells = Array.from(
existingDoc.getElementsByTagName("mxCell"),
) as any[];
for (const el of existingCells) {
const id = el.getAttribute("id") ?? "";
if (id === "0" || id === "1") continue; // sentinels come from the assembled model
if (graphIds.has(id)) continue; // graph re-lists it -> assembled version wins
if (assembledIds.has(id)) continue; // id collision guard -> keep assembled
root.appendChild(assembledDoc.importNode(el, true));
assembledIds.add(id);
}
const ser = new win.XMLSerializer();
return ser.serializeToString(assembledDoc.documentElement);
}
// --- top-level: graph -> mxGraphModel XML ----------------------------------
export interface BuildFromGraphResult {
modelXml: string;
iconsResolved: number;
iconsMissing: string[];
layout: LayoutMode;
}
/**
* The full server-side pipeline: validate -> resolve styles/icons -> map to ELK
* -> run ELK (or fall back) -> assemble linter-clean XML. `existingCoords` is
* supplied for `layout:"incremental"` (the coordinates of the diagram's current
* cells, so they are preserved and only new nodes are placed). `existingModelXml`
* is the current diagram's full model XML in incremental mode every existing
* cell the graph does NOT re-list is MERGED back in verbatim so a hand-placed
* cell is never dropped (WARNING #4). Pure no network.
*/
export async function buildFromGraph(
graph: Graph,
layout: LayoutMode = "full",
existingCoords?: Map<string, { x: number; y: number }>,
existingModelXml?: string,
): Promise<BuildFromGraphResult> {
validateGraph(graph);
const preset = getPreset(graph.preset);
// Resolve every node's style + size (icon or generic-by-kind).
const styles = new Map<
string,
{ style: string; w: number; h: number; iconResolved: boolean }
>();
const sizes = new Map<string, NodeSize>();
for (const n of graph.nodes) {
const s = resolveNodeStyle(preset, n);
styles.set(n.id, s);
sizes.set(n.id, { w: s.w, h: s.h });
}
// Group sizes: seed a min box; ELK computes the real size when it lays out.
for (const g of graph.groups ?? []) sizes.set(g.id, { w: 240, h: 180 });
let geo = new Map<string, { x: number; y: number; w: number; h: number }>();
if (layout === "incremental" && existingCoords && existingCoords.size > 0) {
// INCREMENTAL: keep every existing cell's coords VERBATIM (acceptance #3 —
// never move a hand-arranged cell) and lay out ONLY the new nodes, then
// offset that block clear of the existing bbox so nothing overlaps.
for (const [id, c] of existingCoords) {
const sz = sizes.get(id) ?? { w: 140, h: 60 };
geo.set(id, { x: c.x, y: c.y, w: sz.w, h: sz.h });
}
const newIds = new Set(
graph.nodes.filter((n) => !existingCoords.has(n.id)).map((n) => n.id),
);
if (newIds.size > 0) {
const elk = graphToElk(graph, sizes, { only: newIds });
const laid = await runElk(elk);
// Place the new block below the existing content (a clear >=170px gap).
let maxY = 0;
for (const c of existingCoords.values()) maxY = Math.max(maxY, c.y);
const offsetY = maxY + 200;
for (const [id, g] of laid) {
if (newIds.has(id)) geo.set(id, { ...g, y: g.y + offsetY });
else if (!geo.has(id)) geo.set(id, g); // a new group container
}
}
// Hints still apply to NEW pinned nodes only (existing ones stay put).
applyHintsForNew(graph, geo, existingCoords);
} else if (layout !== "none") {
const elk = graphToElk(graph, sizes);
geo = await runElk(elk);
applyHints(graph, geo);
} else if (existingCoords) {
// layout:"none" with prior coords -> keep them verbatim.
for (const [id, c] of existingCoords) {
const sz = sizes.get(id) ?? { w: 140, h: 60 };
geo.set(id, { x: c.x, y: c.y, w: sz.w, h: sz.h });
}
}
const assembled = assembleModel(graph, { preset, styles, geo });
// In incremental mode, splice back every existing cell the graph didn't
// re-list so an "add one node" call preserves the user's manual layout.
let modelXml = assembled.modelXml;
if (
layout === "incremental" &&
existingModelXml &&
existingCoords &&
existingCoords.size > 0
) {
modelXml = mergeExistingCells(modelXml, existingModelXml, graph);
}
return {
modelXml,
iconsResolved: assembled.iconsResolved,
iconsMissing: assembled.iconsMissing,
layout,
};
}
+228
View File
@@ -0,0 +1,228 @@
// Progressive-disclosure authoring reference for the `drawioGuide` tool
// (issue #424, stage 2). The FULL draw.io authoring guide would bloat every
// context window, so it is split into small sections the model reads on demand:
// skeleton | layout | containers | icons-aws | icons-azure
// Content is written directly from the issue #424 appendix (the layout
// heuristics, container rules, AWS icon patterns + gotchas + blocklist, and
// Azure image-style paths). ACCEPTANCE: each section stays <= ~4 KB so pulling
// one is cheap.
export type GuideSection =
| "skeleton"
| "layout"
| "containers"
| "icons-aws"
| "icons-azure";
export const GUIDE_SECTIONS: GuideSection[] = [
"skeleton",
"layout",
"containers",
"icons-aws",
"icons-azure",
];
const SKELETON = `# drawioGuide: skeleton
Canonical mxGraph skeleton. id="0" and id="1" are MANDATORY sentinels; every
real cell has parent="1" (or a container id). Set adaptiveColors="auto" on the
model so Docmost's dark theme adapts strokeColor/fillColor/fontColor="default".
\`\`\`xml
<mxGraphModel dx="800" dy="600" grid="1" gridSize="10" adaptiveColors="auto"
page="1" pageWidth="850" pageHeight="1100">
<root>
<mxCell id="0"/>
<mxCell id="1" parent="0"/>
<mxCell id="2" value="Start" style="rounded=1;whiteSpace=wrap;html=1;"
vertex="1" parent="1">
<mxGeometry x="40" y="40" width="140" height="60" as="geometry"/>
</mxCell>
<mxCell id="3" value="Store" style="shape=cylinder3;whiteSpace=wrap;html=1;"
vertex="1" parent="1">
<mxGeometry x="40" y="200" width="80" height="80" as="geometry"/>
</mxCell>
<mxCell id="e1" edge="1" parent="1" source="2" target="3"
style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
</root>
</mxGraphModel>
\`\`\`
Three accepted inputs to drawioCreate/drawioUpdate: a bare <mxGraphModel>, a
full <mxfile> (decoded to its first page), or a raw list of <mxCell> (the server
wraps it and adds the id=0/id=1 sentinels).
Hard rules: a cell is vertex="1" XOR edge="1" (a container/group is neither);
every edge has a child <mxGeometry relative="1" as="geometry"/>; ids are unique;
no XML comments; put html=1 in styles and XML-escape value (& -> &amp;,
< -> &lt;); a newline in a label is &#xa;, never a literal \\n. Don't guess
shape=mxgraph.* names call drawioShapes first (a wrong name renders empty).`;
const LAYOUT = `# drawioGuide: layout
Turn "make it look good" into checkable numbers. Or pass layout:"elk" to
drawioCreate/drawioUpdate and the server computes coordinates for you (ELK
layered layout, honouring nested containers) you declare structure, it places
pixels.
Spacing (when placing by hand):
- Horizontal gap between shapes 200-220px; vertical between rows/lanes 250px;
auxiliary services (monitoring, DLQ) sit below the main flow with 280px+ gap.
- Coordinates are multiples of 10 (grid). Base sizes: rectangle 140x60, diamond
140x80, circle 60x60; cloud icons 78x78 primary / 65x65 secondary; font 12px.
- Main flow left-to-right, one primary axis; <=3-4 lanes/zones; one icon/service.
Edges:
- <=1 bend per edge (ideally 0); an edge must not cross another shape's bbox;
two edges must not lie on top of each other.
- Give explicit exitX/exitY/entryX/entryY for every non-straight link or the
orthogonal router drives lines through shapes. Vertical link:
exitX=0.5;exitY=1 -> entryX=0.5;entryY=0. For 2+ links on one node, spread the
attach points 0.25 / 0.5 / 0.75.
- Base edge style:
edgeStyle=orthogonalEdgeStyle;rounded=1;orthogonalLoop=1;jettySize=auto;html=1;strokeWidth=2;exitX=1;exitY=0.5;entryX=0;entryY=0.5;
- Edge labels: 1-2 words max, labelBackgroundColor=#F5F5F5;fontSize=11;. Don't
label an obvious flow (Lambda->DynamoDB needs no "Write"); prefer numbering
stages (1,2,3) over many labels.
- Line semantics: solid = main/sync; dashed=1 = async; red dashed
strokeColor=#DD344C = error path.
Alignment: centre a child under its parent by math, not by eye:
child.x = parent.center_x - child.width/2.
The linter returns quality WARNINGS (bbox overlap, edge through a shape,
edge-on-edge, gap <150px, label wider than its shape, negative/off-page coords).
They do not block the write fix them and retry, max 2 iterations.`;
const CONTAINERS = `# drawioGuide: containers
Groups/zones are TRANSPARENT containers. A coloured group fill is an instant
"AI-generated" tell never fill a group.
- Every group: container=1;dropTarget=1;fillColor=none;. It is a cell with
vertex unset AND edge unset.
- Children set parent="<groupId>" and their coordinates are RELATIVE to the
group's top-left, not absolute.
- An edge between cells in DIFFERENT containers must be parent="1" (the layer),
otherwise it is clipped to one container and disappears.
- Keep the group title off the group icon:
spacingLeft=40;spacingTop=-4;.
- Leave >=30px padding between children and the group frame.
- Draw edges on the BACK layer (place their <mxCell> BEFORE the shapes in XML)
and keep >=20px between an arrow and a label.
Swimlanes: style=swimlane;horizontal=0;startSize=110;. Lanes are parent="1";
their members are children of the lane.
Example (transparent zone with two children and an internal edge):
\`\`\`xml
<mxCell id="z1" value="VPC" style="rounded=0;container=1;dropTarget=1;fillColor=none;verticalAlign=top;spacingLeft=40;spacingTop=-4;html=1;"
vertex="1" parent="1">
<mxGeometry x="40" y="40" width="320" height="200" as="geometry"/>
</mxCell>
<mxCell id="a" value="App" style="rounded=1;html=1;" vertex="1" parent="z1">
<mxGeometry x="30" y="40" width="120" height="60" as="geometry"/>
</mxCell>
<mxCell id="b" value="DB" style="shape=cylinder3;html=1;" vertex="1" parent="z1">
<mxGeometry x="30" y="120" width="80" height="60" as="geometry"/>
</mxCell>
<mxCell id="ab" edge="1" parent="z1" source="a" target="b">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
\`\`\``;
const ICONS_AWS = `# drawioGuide: icons-aws
Two mutually-exclusive AWS icon patterns mixing them is the #1 cause of empty
boxes. Always call drawioShapes for the exact resIcon name; do not guess.
| Level | style | strokeColor |
|---|---|---|
| Service | shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.<NAME> | #ffffff (required) |
| Resource | shape=mxgraph.aws4.<NAME> | none (required) |
Full service-level template (fillColor is REQUIRED the glyph is invisible in
PNG export without it):
\`\`\`
sketch=0;outlineConnect=0;fontColor=#232F3E;fillColor=<category>;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.<NAME>
\`\`\`
Category fillColor: Compute #ED7100, Networking #8C4FFF, Database #C925D1,
Storage #3F8624, Security #DD344C, Integration #E7157B, AI/ML #01A88D.
Rebrandings (stencil name lags the product name):
- Amazon OpenSearch -> resIcon elasticsearch_service (renamed 2021)
- Amazon EventBridge -> resIcon eventbridge (was CloudWatch Events)
- VPC Peering -> resIcon peering (NOT vpc_peering -> empty box)
- Amazon MSK -> resIcon managed_streaming_for_kafka (NOT msk)
- IAM Identity Center -> resIcon single_sign_on (NOT iam_identity_center)
Blocklist -> replacement: dynamodb_table -> dynamodb; general_saml_token ->
traditional_server; kinesis_data_streams is unreliable. An unknown service ->
generic resIcon=mxgraph.aws4.general_AWScloud WITH a label; an unnamed coloured
rectangle is forbidden.
Group stencils (transparent containers): AWS Cloud group_aws_cloud_alt, VPC
group_vpc2, Subnet group_security_group, Account group_account; subnets use
shape=mxgraph.aws4.group;grIcon=mxgraph.aws4.group_public_subnet;.`;
const ICONS_AZURE = `# drawioGuide: icons-azure
shape=mxgraph.azure2.* does NOT render in every host. Use the portable
image-style instead:
\`\`\`
image;aspect=fixed;html=1;image=img/lib/azure2/<category>/<Icon>.svg;
\`\`\`
Known working paths:
- networking/Front_Doors.svg
- app_services/API_Management_Services.svg
- databases/Azure_Cosmos_DB.svg
- identity/Managed_Identities.svg
- management_governance/Monitor.svg
- devops/Application_Insights.svg
For maximum robustness (e.g. PNG export on a host without the bundled lib), use
an absolute URL fallback for the image:
\`\`\`
https://raw.githubusercontent.com/jgraph/drawio/dev/src/main/webapp/img/lib/azure2/<category>/<Icon>.svg
\`\`\`
Call drawioShapes with the service name (e.g. "cosmos", "api management",
"front door") to get the exact image-style string and default 68x68 size.`;
const CONTENT: Record<GuideSection, string> = {
skeleton: SKELETON,
layout: LAYOUT,
containers: CONTAINERS,
"icons-aws": ICONS_AWS,
"icons-azure": ICONS_AZURE,
};
/**
* Return one guide section, or (when `section` is omitted/unknown) an index
* listing the available sections plus a one-line summary each. Each section is
* kept under ~4 KB so pulling it does not bloat the model's context.
*/
export function getGuideSection(section?: string): {
section: string;
content: string;
sections: GuideSection[];
} {
const key = (section ?? "").trim().toLowerCase() as GuideSection;
if (section && GUIDE_SECTIONS.includes(key)) {
return { section: key, content: CONTENT[key], sections: GUIDE_SECTIONS };
}
const index =
"# drawioGuide\n\nProgressive-disclosure draw.io authoring reference. " +
"Call drawioGuide(section) with one of:\n" +
"- skeleton — canonical mxGraph XML, sentinels, the three accepted inputs, hard rules\n" +
"- layout — spacing heuristics, edge routing, the layout:\"elk\" option, quality warnings\n" +
"- containers — transparent groups, relative child coords, cross-container edges, swimlanes\n" +
"- icons-aws — the service/resource icon patterns, category colors, rebrandings, blocklist\n" +
"- icons-azure — the portable image-style paths\n\n" +
"Also call drawioShapes(query) for verified stencil style-strings.";
return { section: "index", content: index, sections: GUIDE_SECTIONS };
}
+240
View File
@@ -0,0 +1,240 @@
// ELK auto-layout for draw.io models (issue #424, stage 2). The model declares
// the LOGICAL structure (which nodes exist, which containers nest which
// children, which edges connect what) with rough or arbitrary coordinates; this
// module runs an Eclipse Layout Kernel "layered" pass (via elkjs — a pure-JS
// port, no native/browser deps) that HONOURS nested containers as compound
// nodes, then rewrites every vertex's <mxGeometry> with the computed pixels.
//
// Principle: "the model declares logical structure, the server computes pixels."
// Coordinates ELK returns for a node are relative to its parent, which is
// exactly mxGraph's convention for a child of a container, so they map across
// directly. Container sizes are computed by ELK; leaf sizes are preserved.
import ELK from "elkjs/lib/elk.bundled.js";
import { JSDOM } from "jsdom";
import { normalizeInput, parseCells, type DrawioCell } from "./drawio-xml.js";
// Default sizes when a vertex declares no geometry (appendix base sizes).
const DEFAULT_W = 140;
const DEFAULT_H = 60;
// DoS bounds for the in-process ELK layout. The mxGraph XML is LLM-supplied
// (layout:"elk" in drawioCreate/drawioUpdate) and elkjs runs synchronously on
// the MCP server's event loop, so an unbounded graph would block it for
// seconds-to-minutes. A ~1MB XML (well under the stage-1 16MB cap) can carry
// thousands of nodes. We cap the graph size and race the layout against a
// wall-clock timeout; on either bound we fall back to the ORIGINAL model, the
// same best-effort contract the catch already honours.
// - 500 nodes lays out in well under a second; beyond that ELK cost climbs
// steeply, so refuse and leave the (already-valid) model untouched.
// - Edges dominate the layered-crossing cost, so allow a bit more headroom
// (1000) than nodes but still bound them.
// - 5s is generous for any graph within the caps yet short enough that a
// pathological input can never wedge the server.
const ELK_MAX_NODES = 500;
const ELK_MAX_EDGES = 1000;
const ELK_TIMEOUT_MS = 5000;
// Spacing is set >=150px on purpose so an ELK layout never trips the linter's
// "gap between adjacent shapes < 150px" quality warning (acceptance #3).
const LAYOUT_OPTIONS: Record<string, string> = {
"elk.algorithm": "layered",
"elk.direction": "RIGHT",
// Route edges across container boundaries in a single hierarchical pass.
"elk.hierarchyHandling": "INCLUDE_CHILDREN",
"elk.layered.spacing.nodeNodeBetweenLayers": "170",
"elk.spacing.nodeNode": "170",
"elk.spacing.edgeNode": "40",
"elk.spacing.edgeEdge": "30",
"elk.padding": "[top=20,left=20,bottom=20,right=20]",
};
// Per-container options: pad children >=30px off the frame (appendix rule) and
// carry the same generous spacing so nested nodes never trip the "gap <150px"
// warning either.
const CONTAINER_OPTIONS: Record<string, string> = {
"elk.algorithm": "layered",
"elk.direction": "RIGHT",
"elk.padding": "[top=40,left=30,bottom=30,right=30]",
"elk.layered.spacing.nodeNodeBetweenLayers": "170",
"elk.spacing.nodeNode": "170",
};
interface ElkNode {
id: string;
width?: number;
height?: number;
x?: number;
y?: number;
children?: ElkNode[];
layoutOptions?: Record<string, string>;
}
interface ElkEdge {
id: string;
sources: string[];
targets: string[];
}
interface ElkGraph extends ElkNode {
edges?: ElkEdge[];
}
/**
* Apply an ELK layered layout to a drawio input and return a full mxGraphModel
* string with rewritten geometry. Accepts the same three input forms as
* drawioCreate (a bare model, an <mxfile>, or a <mxCell> list). Async because
* elkjs' layout() is promise-based. On any layout failure the ORIGINAL
* (normalized) model is returned unchanged layout is best-effort polish, never
* a reason to fail the write.
*/
export async function applyElkLayout(inputXml: string): Promise<string> {
const modelXml = normalizeInput(inputXml);
let cells: DrawioCell[];
try {
cells = parseCells(modelXml);
} catch {
return modelXml; // unparseable -> let the linter report it downstream
}
const byId = new Map(cells.map((c) => [c.id, c]));
const vertices = cells.filter(
(c) => c.vertex && c.id !== "0" && c.id !== "1",
);
if (vertices.length === 0) return modelXml;
// A vertex is a CONTAINER iff some other vertex names it as parent.
const childrenOf = new Map<string, DrawioCell[]>();
for (const v of vertices) {
const p = v.parent && byId.get(v.parent)?.vertex ? v.parent : "__root__";
if (!childrenOf.has(p)) childrenOf.set(p, []);
childrenOf.get(p)!.push(v);
}
const isContainer = (id: string) => childrenOf.has(id);
const buildNode = (v: DrawioCell): ElkNode => {
const kids = childrenOf.get(v.id);
const node: ElkNode = { id: v.id };
if (kids && kids.length > 0) {
node.children = kids.map(buildNode);
node.layoutOptions = { ...CONTAINER_OPTIONS };
} else {
node.width = v.geometry.width ?? DEFAULT_W;
node.height = v.geometry.height ?? DEFAULT_H;
}
return node;
};
const roots = (childrenOf.get("__root__") ?? []).map(buildNode);
// All edges at the root; INCLUDE_CHILDREN lets them span the hierarchy. Only
// edges whose endpoints are laid-out vertices are handed to ELK.
const vertexIds = new Set(vertices.map((v) => v.id));
const edges: ElkEdge[] = [];
for (const c of cells) {
if (!c.edge || !c.source || !c.target) continue;
if (!vertexIds.has(c.source) || !vertexIds.has(c.target)) continue;
edges.push({ id: c.id || `e${edges.length}`, sources: [c.source], targets: [c.target] });
}
// DoS guard: refuse to lay out an oversized LLM-supplied graph. elkjs runs
// in-process on the event loop, so bound the work before we ever call it and
// return the original model unchanged (best-effort, same as the catch below).
if (vertices.length > ELK_MAX_NODES || edges.length > ELK_MAX_EDGES) {
return modelXml;
}
const graph: ElkGraph = {
id: "root",
layoutOptions: LAYOUT_OPTIONS,
children: roots,
edges,
};
let laid: ElkGraph;
let timer: ReturnType<typeof setTimeout> | undefined;
try {
// elkjs ships a CJS default export whose interop shape varies across
// module systems; resolve the real constructor at runtime, then cast (the
// runtime call is verified — see the layout unit test).
const Ctor: any = (ELK as any).default ?? ELK;
const elk = new Ctor();
// Race the layout against a wall-clock timeout so a graph that is under the
// node/edge caps but still pathologically slow can never wedge the server.
const timeout = new Promise<never>((_, reject) => {
timer = setTimeout(
() => reject(new Error("ELK layout timed out")),
ELK_TIMEOUT_MS,
);
});
laid = (await Promise.race([elk.layout(graph as any), timeout])) as ElkGraph;
} catch {
return modelXml; // best-effort: keep the model as-is on timeout or ELK failure
} finally {
if (timer) clearTimeout(timer);
}
// Collect computed geometry per node id (coords are parent-relative already).
const geo = new Map<string, { x: number; y: number; w: number; h: number }>();
const walk = (n: ElkNode) => {
if (n.id !== "root") {
geo.set(n.id, {
x: Math.round(n.x ?? 0),
y: Math.round(n.y ?? 0),
w: Math.round(n.width ?? DEFAULT_W),
h: Math.round(n.height ?? DEFAULT_H),
});
}
for (const c of n.children ?? []) walk(c);
};
walk(laid);
return rewriteGeometry(modelXml, geo, isContainer);
}
/**
* Rewrite each vertex cell's <mxGeometry> x/y (and width/height for containers,
* whose size ELK computed) using the DOM, then serialize back. Leaf sizes are
* left untouched. Edges and non-geometry attributes are preserved verbatim.
*/
function rewriteGeometry(
modelXml: string,
geo: Map<string, { x: number; y: number; w: number; h: number }>,
isContainer: (id: string) => boolean,
): string {
const dom = new JSDOM("");
const parser = new dom.window.DOMParser();
const doc = parser.parseFromString(modelXml, "application/xml");
if (doc.getElementsByTagName("parsererror").length > 0) return modelXml;
const cellEls = doc.getElementsByTagName("mxCell");
for (let i = 0; i < cellEls.length; i++) {
const el = cellEls[i];
const id = el.getAttribute("id") || "";
const g = geo.get(id);
if (!g) continue;
let geoEl: any = null;
for (let j = 0; j < el.childNodes.length; j++) {
const ch = el.childNodes[j];
if (ch.nodeType === 1 && (ch as any).tagName === "mxGeometry") {
geoEl = ch;
break;
}
}
if (!geoEl) {
geoEl = doc.createElement("mxGeometry");
geoEl.setAttribute("as", "geometry");
el.appendChild(geoEl);
}
geoEl.setAttribute("x", String(g.x));
geoEl.setAttribute("y", String(g.y));
// Containers take ELK's computed size; leaves keep their authored size.
if (isContainer(id) || !geoEl.hasAttribute("width")) {
geoEl.setAttribute("width", String(g.w));
}
if (isContainer(id) || !geoEl.hasAttribute("height")) {
geoEl.setAttribute("height", String(g.h));
}
}
const ser = new dom.window.XMLSerializer();
return ser.serializeToString(doc.documentElement);
}
+347
View File
@@ -0,0 +1,347 @@
// Pure Mermaid `flowchart` -> graph-JSON parser for `drawioFromMermaid` (issue
// #425, stage 3, OPTIONAL). The escape clause in the issue: convert WITHOUT
// Electron/draw.io-CLI, so a pure text parser only. It handles the common wiki
// flowchart subset — node shapes, labelled/dashed edges, subgraphs (-> groups),
// and the direction header — and emits a Graph the drawioFromGraph pipeline
// renders as an EDITABLE draw.io diagram. Anything beyond flowchart (sequence /
// class / state) throws a clear error so the model falls back to drawioFromGraph.
//
// DELIBERATELY NARROW: this is not a full Mermaid grammar (Mermaid's own parser
// is a 100KB+ browser dependency). It covers `flowchart`/`graph` with the node
// shapes and edge arrows that show up in practice; unusual syntax is skipped
// rather than mis-parsed, and a diagram that yields no nodes throws.
import type { Graph, GraphNode, GraphEdge, GraphGroup } from "./drawio-graph.js";
export class MermaidParseError extends Error {
constructor(message: string) {
super(`drawioFromMermaid: ${message}`);
this.name = "MermaidParseError";
}
}
// Input-size bounds applied BEFORE parsing. Without them a pathological mermaid
// string (e.g. 300000 connection lines, or 20000 nested `subgraph`s) builds a
// huge intermediate node/edge/group structure that OOM-crashes the worker — the
// downstream validateGraph caps in drawio-graph can't help because the parser
// exhausts the heap constructing the intermediate FIRST. These caps reject the
// over-limit input fast, before a single line is parsed.
const MAX_MERMAID_CHARS = 200_000; // ~200 KB of source is far beyond any real diagram.
const MAX_MERMAID_LINES = 20_000;
const MAX_MERMAID_GROUPS = 500; // parity with drawio-graph's MAX_GRAPH_GROUPS.
// Per connection line, the number of chained nodes we will expand (`A-->B-->C`).
const MAX_CHAIN_NODES = 500;
const DIRECTIONS: Record<string, Graph["direction"]> = {
LR: "LR",
RL: "RL",
TB: "TB",
TD: "TB",
BT: "BT",
};
/**
* Node-shape delimiters -> a semantic `kind`. Mermaid encodes shape in the
* bracket style; we map the common ones to the palette kinds so the diagram is
* colored meaningfully (a decision/diamond -> queue, a database cylinder -> db,
* a rounded/stadium -> service, a subroutine/hexagon -> gateway, default rect ->
* service). The label text lives between the delimiters.
*/
interface ShapeDef {
open: string;
close: string;
kind: string;
}
// Order matters: longer/multi-char delimiters first so "([" beats "(".
const SHAPES: ShapeDef[] = [
{ open: "([", close: "])", kind: "service" }, // stadium
{ open: "[[", close: "]]", kind: "gateway" }, // subroutine
{ open: "[(", close: ")]", kind: "db" }, // cylinder-ish / database
{ open: "((", close: "))", kind: "external" }, // circle
{ open: "{{", close: "}}", kind: "gateway" }, // hexagon
{ open: "[", close: "]", kind: "service" }, // rectangle
{ open: "(", close: ")", kind: "service" }, // rounded
{ open: "{", close: "}", kind: "queue" }, // rhombus / decision
{ open: ">", close: "]", kind: "external" }, // asymmetric flag
];
/** Strip Mermaid label quoting/escapes and normalise whitespace. */
function cleanLabel(raw: string): string {
let s = raw.trim();
if (
(s.startsWith('"') && s.endsWith('"')) ||
(s.startsWith("'") && s.endsWith("'"))
) {
s = s.slice(1, -1);
}
return s.replace(/<br\s*\/?>/gi, " ").replace(/\s+/g, " ").trim();
}
/** A single edge-arrow spec: its regex and the resulting edge `kind`. */
interface ArrowDef {
re: RegExp;
kind: string;
}
// Dotted arrows (`-.->`) -> async; thick (`==>`) stay sync; normal `-->`/`---`.
// Each captures an optional `|label|` OR inline label between the two arrow
// halves. Applied to the segment between two node tokens.
const ARROWS: ArrowDef[] = [
{ re: /-\.->|-\.-/, kind: "async" },
{ re: /==>|===/, kind: "sync" },
{ re: /-->|---/, kind: "sync" },
];
interface ParsedRef {
id: string;
node?: GraphNode;
}
/**
* Parse a single node token like `A`, `A[Label]`, `db[(Orders)]`, `d{Choose}`.
* Returns the id and, when the token declares a shape/label, a GraphNode.
*/
function parseNodeToken(token: string): ParsedRef | null {
const t = token.trim();
if (t === "") return null;
for (const shape of SHAPES) {
const oi = t.indexOf(shape.open);
if (oi <= 0) continue;
if (!t.endsWith(shape.close)) continue;
const id = t.slice(0, oi).trim();
const label = cleanLabel(t.slice(oi + shape.open.length, t.length - shape.close.length));
if (!id) return null;
return { id, node: { id, label: label || id, kind: shape.kind } };
}
// Bare id (no shape declared here — may be defined elsewhere).
if (/^[A-Za-z0-9_.-]+$/.test(t)) return { id: t };
return null;
}
/**
* Split a connection line into [leftToken, arrowSegment, rightToken]. Returns
* null if the line has no arrow. The arrow segment may embed a label as
* `-->|text|` or `-- text -->`.
*/
function splitConnection(
line: string,
): { left: string; right: string; kind: string; label?: string } | null {
for (const arrow of ARROWS) {
// Find the arrow occurrence. Support a mid-arrow label: `A -- text --> B`.
const m = arrow.re.exec(line);
if (!m) continue;
const idx = m.index;
let left = line.slice(0, idx).trim();
let rest = line.slice(idx + m[0].length).trim();
let label: string | undefined;
// Pipe label: `-->|HTTPS| B`.
const pipe = /^\|([^|]*)\|\s*(.*)$/.exec(rest);
if (pipe) {
label = cleanLabel(pipe[1]);
rest = pipe[2].trim();
}
// Mid-arrow label on the left side: `A -- text` before the arrow half.
const midLeft = /^(.*?)\s*--\s*(.+)$/.exec(left);
if (!label && midLeft && /-\.|--|==/.test(line.slice(0, idx))) {
// Only treat as a label when there's clearly text after `--`.
if (!/[\[\](){}]/.test(midLeft[2])) {
left = midLeft[1].trim();
label = cleanLabel(midLeft[2]);
}
}
if (!left || !rest) return null;
return { left, right: rest, kind: arrow.kind, label };
}
return null;
}
/**
* Parse Mermaid flowchart text into a Graph. Handles the header
* (`flowchart LR` / `graph TD`), `subgraph <id>[title] … end` blocks (-> groups),
* node declarations, and connection lines. Throws MermaidParseError for a
* non-flowchart diagram or when nothing parses.
*/
export function mermaidToGraph(mermaid: string): Graph {
if (typeof mermaid !== "string" || mermaid.trim() === "") {
throw new MermaidParseError("empty mermaid input");
}
// Size guards FIRST — bound the raw input before building any intermediate.
if (mermaid.length > MAX_MERMAID_CHARS) {
throw new MermaidParseError(
`input is ${mermaid.length} chars (max ${MAX_MERMAID_CHARS}); split the diagram or use drawioFromGraph`,
);
}
const rawLines = mermaid.split(/\r?\n/);
if (rawLines.length > MAX_MERMAID_LINES) {
throw new MermaidParseError(
`input has ${rawLines.length} lines (max ${MAX_MERMAID_LINES}); split the diagram or use drawioFromGraph`,
);
}
const nodes = new Map<string, GraphNode>();
const groups: GraphGroup[] = [];
const edges: GraphEdge[] = [];
let direction: Graph["direction"] = "LR";
let sawHeader = false;
// Stack of active subgraph ids (nesting); the top is the current group.
const groupStack: string[] = [];
let anonGroup = 0;
const ensureNode = (ref: ParsedRef) => {
const existing = nodes.get(ref.id);
if (ref.node) {
if (existing) {
// Fill in a label/kind if this token declared a shape and the prior didn't.
if (existing.label === existing.id && ref.node.label !== ref.node.id)
existing.label = ref.node.label;
if (!existing.kind) existing.kind = ref.node.kind;
} else {
nodes.set(ref.id, { ...ref.node });
}
} else if (!existing) {
nodes.set(ref.id, { id: ref.id, label: ref.id, kind: "service" });
}
// Assign to the current subgraph if inside one and not yet grouped.
const cur = groupStack[groupStack.length - 1];
const n = nodes.get(ref.id)!;
if (cur && n.group == null) n.group = cur;
};
for (const raw of rawLines) {
let line = raw.trim();
if (line === "" || line.startsWith("%%")) continue; // blank / comment
// Header.
const header = /^(flowchart|graph)\s+([A-Za-z]{2})\b/.exec(line);
if (header) {
sawHeader = true;
const dir = DIRECTIONS[header[2].toUpperCase()];
if (dir) direction = dir;
continue;
}
if (/^(sequenceDiagram|classDiagram|stateDiagram|erDiagram|gantt|pie|journey)\b/.test(line)) {
throw new MermaidParseError(
`only 'flowchart'/'graph' is supported (got '${line.split(/\s+/)[0]}'); use drawioFromGraph instead`,
);
}
// Subgraph open: `subgraph id [Title]` or `subgraph Title`.
const sg = /^subgraph\s+(.+)$/.exec(line);
if (sg) {
const spec = sg[1].trim();
let id: string;
let label: string;
const bracket = /^([A-Za-z0-9_.-]+)\s*\[(.+)\]$/.exec(spec);
if (bracket) {
id = bracket[1];
label = cleanLabel(bracket[2]);
} else if (/^[A-Za-z0-9_.-]+$/.test(spec)) {
id = spec;
label = spec;
} else {
id = `sg${anonGroup++}`;
label = cleanLabel(spec);
}
if (!groups.some((g) => g.id === id)) {
if (groups.length >= MAX_MERMAID_GROUPS) {
throw new MermaidParseError(
`too many subgraphs (max ${MAX_MERMAID_GROUPS}); use drawioFromGraph for a diagram this large`,
);
}
groups.push({ id, label, kind: "group" });
}
groupStack.push(id);
continue;
}
if (/^end\b/.test(line)) {
groupStack.pop();
continue;
}
// `direction LR` inside a subgraph — apply to the top-level direction.
const innerDir = /^direction\s+([A-Za-z]{2})\b/.exec(line);
if (innerDir) {
const dir = DIRECTIONS[innerDir[1].toUpperCase()];
if (dir) direction = dir;
continue;
}
// Style/class/click directives: ignore (no visual mapping in our palette).
if (/^(style|classDef|class|click|linkStyle)\b/.test(line)) continue;
// Strip a trailing semicolon.
if (line.endsWith(";")) line = line.slice(0, -1).trim();
// Connection line (possibly chained: A --> B --> C).
const conn = splitConnection(line);
if (conn) {
// Handle a simple chain by re-splitting the right side.
let leftTok = conn.left;
let seg: typeof conn | null = conn;
let guard = 0;
while (seg) {
if (guard++ >= MAX_CHAIN_NODES) {
// Don't silently drop the tail of an over-long chain — surface it so
// the model knows the diagram was too large rather than getting a
// quietly-truncated result.
throw new MermaidParseError(
`a single connection chain exceeds ${MAX_CHAIN_NODES} nodes; split it or use drawioFromGraph`,
);
}
const leftRef = parseNodeToken(leftTok);
// The right side may itself contain another arrow (a chain).
const nextSeg = splitConnection(seg.right);
const rightTokenStr = nextSeg ? seg.right.slice(0, splitIndex(seg.right)) : seg.right;
const rightRef = parseNodeToken(nextSeg ? nextSeg.left : seg.right);
if (leftRef && rightRef) {
ensureNode(leftRef);
ensureNode(rightRef);
edges.push({
from: leftRef.id,
to: rightRef.id,
label: seg.label,
kind: seg.kind,
});
}
if (!nextSeg) break;
leftTok = nextSeg.left;
seg = nextSeg;
void rightTokenStr;
}
continue;
}
// Standalone node declaration `A[Label]` OR a bare member ref `C` inside a
// subgraph (which claims that node for the current group).
const nodeRef = parseNodeToken(line);
if (nodeRef && (nodeRef.node || groupStack.length > 0)) {
ensureNode(nodeRef);
continue;
}
// Unknown line: skip silently (robustness over strictness).
}
if (!sawHeader && nodes.size === 0) {
throw new MermaidParseError(
"input does not look like a mermaid flowchart (no 'flowchart'/'graph' header and no nodes)",
);
}
if (nodes.size === 0) {
throw new MermaidParseError("no nodes parsed from the flowchart");
}
const graph: Graph = {
nodes: Array.from(nodes.values()),
direction,
};
if (groups.length > 0) graph.groups = groups;
if (edges.length > 0) graph.edges = edges;
return graph;
}
/** Index of the first arrow in a segment (for chain splitting). */
function splitIndex(s: string): number {
let best = -1;
for (const arrow of ARROWS) {
const m = arrow.re.exec(s);
if (m && (best === -1 || m.index < best)) best = m.index;
}
return best === -1 ? s.length : best;
}
+151
View File
@@ -0,0 +1,151 @@
// Semantic color/line presets for the graph tools (issue #425, stage 3). The
// PALETTE is DATA (packages/mcp/data/drawio-presets.json), not code: a node
// `kind` maps to a { fillColor, strokeColor, fontColor } slot and an edge `kind`
// maps to line-style props, per named preset (`default` / `dark` /
// `colorblind-safe`). This module only loads that data and turns a slot into a
// draw.io style fragment. The INVARIANT of the graph tools is that the model
// never sees a style string — it names a `kind`, the server picks the slot.
//
// Loading mirrors drawio-shapes.ts: the JSON is read once via `import.meta.url`
// relative to the built module. That is why this module (and drawio-graph.ts
// which imports it) is reached ONLY through client.ts's ESM build and never
// value-imported into the zod-agnostic tool-specs.ts (which the in-app server
// type-checks under module:commonjs, where `import.meta` is a TS1343 error).
import { readFileSync } from "node:fs";
/** A node color slot: the three draw.io color values for a `kind`. */
export interface NodeSlot {
fillColor: string;
strokeColor: string;
fontColor: string;
}
/** An edge line style: the extra style props appended for an edge `kind`. */
export interface EdgeStyle {
props: string;
}
export interface PresetData {
canvasDark: boolean;
okabeIto?: string[];
nodes: Record<string, NodeSlot>;
edges: Record<string, EdgeStyle>;
edgeDefault: { strokeColor: string; fontColor: string };
group: { strokeColor: string; fontColor: string };
}
interface PresetsFile {
presets: Record<string, PresetData>;
}
/** The three shipped preset names. */
export const PRESET_NAMES = ["default", "dark", "colorblind-safe"] as const;
export type PresetName = (typeof PRESET_NAMES)[number];
/** Every node `kind` the base palette defines (also the generic-shape kinds). */
export const NODE_KINDS = [
"service",
"db",
"queue",
"gateway",
"error",
"external",
"security",
] as const;
export type NodeKind = (typeof NODE_KINDS)[number];
/** Edge `kind`s the palette styles; anything else falls back to `sync`. */
export const EDGE_KINDS = ["sync", "async", "error"] as const;
export type EdgeKind = (typeof EDGE_KINDS)[number];
let _presets: Record<string, PresetData> | null = null;
function presetsPath(): URL {
// build/lib/drawio-presets.js -> ../../data/drawio-presets.json
return new URL("../../data/drawio-presets.json", import.meta.url);
}
/** Load + parse the bundled preset table once, then cache it. */
export function loadPresets(): Record<string, PresetData> {
if (_presets) return _presets;
const json = readFileSync(presetsPath(), "utf-8");
const parsed = JSON.parse(json) as PresetsFile;
_presets = parsed.presets;
return _presets;
}
/** Resolve a preset by name, defaulting to `default` for an unknown name. */
export function getPreset(name?: string): PresetData {
const presets = loadPresets();
if (name && presets[name]) return presets[name];
return presets["default"];
}
/** The slot for a node `kind` in a preset, falling back to `service`. */
export function nodeSlot(preset: PresetData, kind?: string): NodeSlot {
if (kind && preset.nodes[kind]) return preset.nodes[kind];
return preset.nodes["service"];
}
/**
* Build the draw.io style string for a GENERIC (no-icon) node of a given kind.
* A rounded rectangle carrying the slot's fill/stroke/font. `whiteSpace=wrap`
* and `html=1` let a long label wrap inside the shape (the assembler also sizes
* the shape to the label, so the linter's label-overflow warning never fires).
*/
export function genericNodeStyle(preset: PresetData, kind?: string): string {
const s = nodeSlot(preset, kind);
return (
`rounded=1;whiteSpace=wrap;html=1;` +
`fillColor=${s.fillColor};strokeColor=${s.strokeColor};fontColor=${s.fontColor};`
);
}
/**
* Overlay the preset's node slot colors onto a resolved ICON style-string
* (from the shape catalog). An AWS/Azure icon carries its OWN mandatory
* fill/stroke (the category color / white outline) that MUST NOT be recolored,
* so for an icon we only ensure a readable fontColor when the preset is dark;
* otherwise the icon style is returned verbatim. Keeping the icon's own colors
* is deliberate: recoloring an AWS service icon breaks its category semantics.
*/
export function iconNodeStyle(preset: PresetData, iconStyle: string): string {
if (!preset.canvasDark) return iconStyle;
// On a dark canvas an icon's fontColor is usually a dark ink that vanishes;
// append a light fontColor (icons put their label BELOW the glyph, so this
// only affects the caption, never the glyph fill).
if (/fontColor=/.test(iconStyle)) {
return iconStyle.replace(/fontColor=[^;]*/, "fontColor=#e0e0e0");
}
return iconStyle + (iconStyle.endsWith(";") ? "" : ";") + "fontColor=#e0e0e0;";
}
/**
* Build the draw.io style for an edge of a given `kind`. Base is an orthogonal
* connector (edgeStyle=orthogonalEdgeStyle) with rounded corners and an open
* arrowhead, plus the preset's default stroke/font, then the kind's extra props
* (dashed / colored) overlaid. An unknown kind falls back to `sync` (solid).
*/
export function edgeStyle(preset: PresetData, kind?: string): string {
const k = kind && preset.edges[kind] ? kind : "sync";
const base =
`edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=open;` +
`strokeColor=${preset.edgeDefault.strokeColor};fontColor=${preset.edgeDefault.fontColor};`;
return base + preset.edges[k].props;
}
/**
* Group (container) style: ALWAYS transparent (`fillColor=none;container=1;`)
* per the spec, carrying the preset's group stroke/font. `dropTarget=1` marks it
* a drop target in the editor; `verticalAlign=top;align=left;spacingLeft=8;` puts
* the group label in the top-left like draw.io's own boundary containers.
*/
export function groupStyle(preset: PresetData): string {
return (
`rounded=0;whiteSpace=wrap;html=1;` +
`fillColor=none;container=1;dropTarget=1;collapsible=0;` +
`strokeColor=${preset.group.strokeColor};fontColor=${preset.group.fontColor};` +
`verticalAlign=top;align=left;spacingLeft=8;spacingTop=4;`
);
}
+420
View File
@@ -0,0 +1,420 @@
// Verified draw.io shape catalog for the `drawioShapes` tool (issue #424,
// stage 2). This is the fix for AI-generated diagrams' #1 defect: guessed
// `shape=mxgraph.*` names that render as EMPTY BOXES because the stencil does
// not exist. Instead of guessing, the model queries this catalog and gets back
// an exact, verified style-string + the stencil's default width/height.
//
// DATA SOURCE — the bundled index is the REAL jgraph/drawio-mcp shape index
// (`shape-search/search-index.json`, Apache-2.0, ~10 446 shapes), fetched
// verbatim and gzip-compressed to `packages/mcp/data/drawio-shape-index.json.gz`
// (~4.7 MB -> ~430 KB). Each record is `{ style, w, h, title, tags, type }`.
//
// REGENERATING THE INDEX (keeps the catalog from going stale as draw.io ships
// new stencils): jgraph publishes `shape-search/generate-index.js`, which
// rebuilds `search-index.json` from a draw.io release's `app.min.js`. To update:
// 1. clone https://github.com/jgraph/drawio-mcp (Apache-2.0)
// 2. run `node shape-search/generate-index.js` per its README
// 3. `gzip -9 -c search-index.json > packages/mcp/data/drawio-shape-index.json.gz`
// The record shape and this module's search stay unchanged.
//
// CURATED OVERLAY — on top of the raw index this module carries a small,
// hand-maintained overlay drawn from the issue #424 appendix (the aws-
// architecture-diagram-skill knowledge): AWS service rebrandings whose stencil
// name lags the product name, a BLOCKLIST of known-broken stencils mapped to
// working replacements, the category fillColor palette, the AWS group/subnet
// stencils, and the Azure image-style paths. The overlay is applied BEFORE the
// raw search so a query for a rebranded/blocked name returns the correct answer
// with an explanatory note instead of the empty-box stencil.
import { readFileSync } from "node:fs";
import { gunzipSync } from "node:zlib";
/** A single catalog record as returned to the model. */
export interface ShapeResult {
/** The exact draw.io style-string to put on the cell. */
style: string;
/** Default width in px for this stencil. */
w: number;
/** Default height in px for this stencil. */
h: number;
/** Human-readable stencil name. */
title: string;
/** "vertex" | "edge" (from the index). */
type: string;
/** AWS category (Compute/Database/…) when derivable, else undefined. */
category?: string;
/**
* Present when the overlay rewrote/annotated the answer: a rebrand, a
* blocklist replacement, or a usage hint. The model should surface it.
*/
note?: string;
}
/** Raw record shape in the bundled index. */
interface IndexRecord {
style: string;
w: number;
h: number;
title: string;
tags: string;
type: string;
}
// --- AWS category fillColor palette (appendix) -----------------------------
// Service-level icons MUST carry a fillColor (invisible in PNG export
// otherwise); the color is the AWS category color.
export const AWS_CATEGORY_FILL: Record<string, string> = {
Compute: "#ED7100",
Networking: "#8C4FFF",
Database: "#C925D1",
Storage: "#3F8624",
Security: "#DD344C",
Integration: "#E7157B",
"AI/ML": "#01A88D",
};
/** Reverse lookup: fillColor hex -> category name (for annotating results). */
const FILL_TO_CATEGORY: Record<string, string> = Object.fromEntries(
Object.entries(AWS_CATEGORY_FILL).map(([k, v]) => [v.toLowerCase(), k]),
);
/**
* Build the canonical service-level AWS icon style for a resIcon name. Mirrors
* the appendix's full template: strokeColor=#ffffff is MANDATORY and fillColor
* is the category color (defaults to AWS ink #232F3E when the category is
* unknown, so the glyph is never invisible).
*/
export function awsServiceStyle(resIcon: string, category?: string): string {
const fill = (category && AWS_CATEGORY_FILL[category]) || "#232F3E";
return (
"sketch=0;outlineConnect=0;fontColor=#232F3E;gradientColor=none;" +
`fillColor=${fill};strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;` +
"verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;" +
`shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.${resIcon}`
);
}
// --- AWS rebrandings (appendix "gotcha" table) -----------------------------
// The stencil name lags the AWS product name; a naive query for the product
// name would miss (or return an empty box). Each alias maps to the REAL resIcon.
interface Rebrand {
aliases: string[];
resIcon: string;
category?: string;
note: string;
}
export const AWS_REBRANDS: Rebrand[] = [
{
aliases: ["opensearch", "open search", "amazon opensearch"],
resIcon: "elasticsearch_service",
category: "Database",
note: "Amazon OpenSearch's stencil is still named `elasticsearch_service` (renamed in 2021).",
},
{
aliases: ["eventbridge", "event bridge", "cloudwatch events"],
resIcon: "eventbridge",
category: "Integration",
note: "Amazon EventBridge uses resIcon `eventbridge` (formerly CloudWatch Events).",
},
{
aliases: ["vpc peering", "peering"],
resIcon: "peering",
category: "Networking",
note: "VPC Peering is resIcon `peering`, NOT `vpc_peering` (which renders empty).",
},
{
aliases: ["msk", "kafka", "managed streaming", "amazon msk"],
resIcon: "managed_streaming_for_kafka",
category: "Integration",
note: "Amazon MSK is resIcon `managed_streaming_for_kafka`, NOT `msk`.",
},
{
aliases: ["iam identity center", "identity center", "sso", "single sign on"],
resIcon: "single_sign_on",
category: "Security",
note: "IAM Identity Center is resIcon `single_sign_on`, NOT `iam_identity_center`.",
},
];
// --- BLOCKLIST of broken stencils (appendix) -------------------------------
// A query that names one of these gets the working replacement + a note; the
// broken stencil is never returned.
interface Blocked {
bad: string;
good: string;
goodStyle?: (idx: IndexRecord[]) => ShapeResult | null;
note: string;
}
export const AWS_BLOCKLIST: Blocked[] = [
{
bad: "dynamodb_table",
good: "dynamodb",
note: "`dynamodb_table` renders as an empty box; use resIcon `dynamodb`.",
},
{
bad: "general_saml_token",
good: "traditional_server",
note: "`general_saml_token` is broken; use resIcon `traditional_server`.",
},
{
bad: "kinesis_data_streams",
good: "kinesis_data_streams",
note: "`kinesis_data_streams` is unreliable across draw.io versions; verify it renders, or fall back to resIcon `kinesis`.",
},
];
// --- AWS group / container stencils (appendix) -----------------------------
// Groups are transparent containers; these are the verified stencil names.
export const AWS_GROUP_STENCILS: ShapeResult[] = [
{
title: "AWS Cloud (group)",
style:
"points=[[0,0],[0.25,0],[0.5,0],[0.75,0],[1,0],[1,0.25],[1,0.5],[1,0.75],[1,1],[0.75,1],[0.5,1],[0.25,1],[0,1],[0,0.75],[0,0.5],[0,0.25]];" +
"outlineConnect=0;gradientColor=none;html=1;whiteSpace=wrap;fontSize=12;fontStyle=0;container=1;" +
"pointerEvents=0;collapsible=0;recursiveResize=0;shape=mxgraph.aws4.group;grIcon=mxgraph.aws4.group_aws_cloud_alt;" +
"strokeColor=#232F3E;fillColor=none;verticalAlign=top;align=left;spacingLeft=30;fontColor=#232F3E;dashed=0;",
w: 400,
h: 300,
type: "vertex",
note: "AWS Cloud boundary — transparent container (grIcon=group_aws_cloud_alt).",
},
{
title: "VPC (group)",
style:
"points=[[0,0],[0.25,0],[0.5,0],[0.75,0],[1,0],[1,0.25],[1,0.5],[1,0.75],[1,1],[0.75,1],[0.5,1],[0.25,1],[0,1],[0,0.75],[0,0.5],[0,0.25]];" +
"outlineConnect=0;gradientColor=none;html=1;whiteSpace=wrap;fontSize=12;fontStyle=0;container=1;" +
"pointerEvents=0;collapsible=0;recursiveResize=0;shape=mxgraph.aws4.group;grIcon=mxgraph.aws4.group_vpc2;" +
"strokeColor=#8C4FFF;fillColor=none;verticalAlign=top;align=left;spacingLeft=30;fontColor=#8C4FFF;dashed=0;",
w: 350,
h: 250,
type: "vertex",
note: "VPC boundary — transparent container (grIcon=group_vpc2).",
},
{
title: "Public Subnet (group)",
style:
"sketch=0;outlineConnect=0;gradientColor=none;html=1;whiteSpace=wrap;fontSize=12;fontStyle=0;container=1;" +
"pointerEvents=0;collapsible=0;recursiveResize=0;shape=mxgraph.aws4.group;grIcon=mxgraph.aws4.group_public_subnet;" +
"grStroke=0;strokeColor=none;fillColor=#E9F3E6;verticalAlign=top;align=left;spacingLeft=30;fontColor=#248814;dashed=0;",
w: 300,
h: 200,
type: "vertex",
note: "Public subnet — transparent container (grIcon=group_public_subnet).",
},
{
title: "Private Subnet (group)",
style:
"sketch=0;outlineConnect=0;gradientColor=none;html=1;whiteSpace=wrap;fontSize=12;fontStyle=0;container=1;" +
"pointerEvents=0;collapsible=0;recursiveResize=0;shape=mxgraph.aws4.group;grIcon=mxgraph.aws4.group_private_subnet;" +
"grStroke=0;strokeColor=none;fillColor=#E6F2F8;verticalAlign=top;align=left;spacingLeft=30;fontColor=#147EBA;dashed=0;",
w: 300,
h: 200,
type: "vertex",
note: "Private subnet — transparent container (grIcon=group_private_subnet).",
},
];
// --- Azure image-style stencils (appendix) ---------------------------------
// `shape=mxgraph.azure2.*` does not render in every host; the image-style path
// is the portable form. These are the verified known-working paths.
interface AzureIcon {
aliases: string[];
path: string;
title: string;
}
const AZURE_ICONS: AzureIcon[] = [
{ aliases: ["front door", "front doors"], path: "networking/Front_Doors.svg", title: "Azure Front Door" },
{ aliases: ["api management", "apim"], path: "app_services/API_Management_Services.svg", title: "Azure API Management" },
{ aliases: ["cosmos", "cosmos db"], path: "databases/Azure_Cosmos_DB.svg", title: "Azure Cosmos DB" },
{ aliases: ["managed identity", "managed identities"], path: "identity/Managed_Identities.svg", title: "Azure Managed Identity" },
{ aliases: ["azure monitor", "monitor"], path: "management_governance/Monitor.svg", title: "Azure Monitor" },
{ aliases: ["application insights", "app insights"], path: "devops/Application_Insights.svg", title: "Azure Application Insights" },
];
/** Build the portable Azure image-style for a lib path (appendix template). */
export function azureImageStyle(path: string): string {
return `sketch=0;points=[[0,0,0],[0.25,0,0],[0.5,0,0],[0.75,0,0],[1,0,0],[0,1,0],[0.25,1,0],[0.5,1,0],[0.75,1,0],[1,1,0],[0,0.25,0],[0,0.5,0],[0,0.75,0],[1,0.25,0],[1,0.5,0],[1,0.75,0]];shadow=0;dashed=0;html=1;strokeColor=none;fillColor=#5E9BD9;labelPosition=center;verticalLabelPosition=bottom;verticalAlign=top;align=center;outlineConnect=0;image;aspect=fixed;image=img/lib/azure2/${path};`;
}
// --- index loading (lazy, cached) ------------------------------------------
let _index: IndexRecord[] | null = null;
/** Path to the bundled gzipped index, resolved relative to the built module. */
function indexPath(): URL {
// build/lib/drawio-shapes.js -> ../../data/… -> packages/mcp/data/…
return new URL("../../data/drawio-shape-index.json.gz", import.meta.url);
}
/** Load + decompress + parse the bundled index once, then cache it. */
export function loadShapeIndex(): IndexRecord[] {
if (_index) return _index;
const gz = readFileSync(indexPath());
const json = gunzipSync(gz).toString("utf-8");
const arr = JSON.parse(json) as IndexRecord[];
_index = arr;
return arr;
}
/** Derive an AWS category from a service-level icon's fillColor, if present. */
function categoryOf(style: string): string | undefined {
const m = /fillColor=(#[0-9a-fA-F]{6})/.exec(style);
if (!m) return undefined;
return FILL_TO_CATEGORY[m[1].toLowerCase()];
}
function toResult(r: IndexRecord): ShapeResult {
return {
style: r.style,
w: r.w,
h: r.h,
title: r.title,
type: r.type,
category: categoryOf(r.style),
};
}
/** Find the best index record whose style carries `resIcon=<name>`. */
function findByResIcon(idx: IndexRecord[], name: string): IndexRecord | null {
const needle = `resIcon=mxgraph.aws4.${name}`;
// Prefer the service-level resourceIcon form; fall back to any style match.
let fallback: IndexRecord | null = null;
for (const r of idx) {
if (r.style.includes(needle) && r.style.includes("resourceIcon")) return r;
if (!fallback && r.style.includes(needle)) fallback = r;
}
return fallback;
}
/**
* Score a record against a lowercased query. Higher is better; 0 = no match.
* Exact title match ranks highest, then title substring, tag word, then a loose
* style/tag substring. This is a cheap substring+token scorer, not a real fuzzy
* matcher, which is plenty for the "give me the lambda icon" use case.
*/
function score(r: IndexRecord, q: string): number {
const title = r.title.toLowerCase();
const tags = r.tags.toLowerCase();
const style = r.style.toLowerCase();
let s = title === q ? 100 : 0;
if (title !== q && title.includes(q)) s += 40 - Math.min(20, title.length - q.length);
const words = q.split(/\s+/).filter(Boolean);
for (const w of words) {
if (title.includes(w)) s += 12;
if (new RegExp(`(^|\\W)${escapeRe(w)}(\\W|$)`).test(tags)) s += 8;
else if (tags.includes(w)) s += 4;
if (style.includes(w)) s += 2;
}
// Prefer the current AWS icon generation (aws4) over the deprecated aws3
// stencils, which are the older visual style and often not what's wanted.
if (s > 0) {
if (style.includes("mxgraph.aws4")) s += 6;
else if (style.includes("mxgraph.aws3")) s -= 12;
}
return s;
}
function escapeRe(s: string): string {
return s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
}
export interface SearchShapesOptions {
category?: string;
limit?: number;
}
/**
* Search the catalog. Applies the curated overlay first (blocklist replacement,
* AWS rebrand, AWS group stencils, Azure image-style), then substring/tag/fuzzy
* search over the bundled ~10 446-shape index. Returns up to `limit` results
* (default 12) with exact style-strings and default sizes.
*/
export function searchShapes(
query: string,
opts: SearchShapesOptions = {},
): ShapeResult[] {
const limit = Math.max(1, Math.min(50, opts.limit ?? 12));
const q = query.trim().toLowerCase();
if (q === "") return [];
const idx = loadShapeIndex();
const out: ShapeResult[] = [];
const seen = new Set<string>();
const push = (r: ShapeResult) => {
if (seen.has(r.style)) return;
seen.add(r.style);
out.push(r);
};
// 1. BLOCKLIST: a query naming a broken stencil returns the replacement.
for (const b of AWS_BLOCKLIST) {
if (q.includes(b.bad) || b.bad.includes(q.replace(/\s+/g, "_"))) {
const rec = findByResIcon(idx, b.good);
if (rec) push({ ...toResult(rec), note: b.note });
}
}
// 2. AWS rebrandings: surface the correct resIcon with the rename note.
for (const rb of AWS_REBRANDS) {
if (rb.aliases.some((a) => q === a || q.includes(a) || a.includes(q))) {
const rec = findByResIcon(idx, rb.resIcon);
if (rec) {
push({ ...toResult(rec), category: rec ? categoryOf(rec.style) ?? rb.category : rb.category, note: rb.note });
} else {
push({
style: awsServiceStyle(rb.resIcon, rb.category),
w: 78,
h: 78,
title: rb.resIcon,
type: "vertex",
category: rb.category,
note: rb.note,
});
}
}
}
// 3. Azure image-style icons.
for (const az of AZURE_ICONS) {
if (az.aliases.some((a) => q.includes(a) || a.includes(q))) {
push({
style: azureImageStyle(az.path),
w: 68,
h: 68,
title: az.title,
type: "vertex",
category: "Azure",
note: "Azure: portable image-style (shape=mxgraph.azure2.* does not render in every host).",
});
}
}
// 4. AWS group/container stencils.
if (/\b(group|container|boundary|vpc|subnet|cloud|account)\b/.test(q)) {
for (const g of AWS_GROUP_STENCILS) {
if (g.title.toLowerCase().includes(q) || q.split(/\s+/).some((w) => g.title.toLowerCase().includes(w))) {
push(g);
}
}
}
// 5. General index search (substring + tags + loose fuzzy).
const catFilter = opts.category?.toLowerCase();
const scored: { r: IndexRecord; s: number }[] = [];
for (const r of idx) {
const s = score(r, q);
if (s <= 0) continue;
if (catFilter) {
const cat = categoryOf(r.style)?.toLowerCase();
const inStyle = r.style.toLowerCase().includes(catFilter);
if (cat !== catFilter && !inStyle) continue;
}
scored.push({ r, s });
}
scored.sort((a, b) => b.s - a.s || a.r.title.length - b.r.title.length);
for (const { r } of scored) {
if (out.length >= limit) break;
push(toResult(r));
}
return out.slice(0, limit);
}
+307 -2
View File
@@ -461,6 +461,308 @@ export function absolutePos(
return { x, y };
}
// --- quality warnings (geometry, non-blocking) -----------------------------
//
// These are computed purely from geometry — NO rendering — and are returned as
// WARNINGS (never errors): they do not block the write, they nudge the model to
// self-correct ("fix the warnings and retry, max 2 iterations"). They replace
// the vision-self-check a render backend would have done.
interface Rect {
x: number;
y: number;
w: number;
h: number;
}
/** Absolute rect of a vertex (following the container chain), or null. */
function rectOf(cell: DrawioCell, byId: Map<string, DrawioCell>): Rect | null {
if (!cell.vertex || !cell.geometry.hasGeometry) return null;
const g = cell.geometry;
if (g.width == null || g.height == null) return null;
const { x, y } = absolutePos(cell, byId);
return { x, y, w: g.width, h: g.height };
}
/** True if `ancestorId` is somewhere up `cell`'s parent chain. */
function isAncestor(
ancestorId: string,
cell: DrawioCell,
byId: Map<string, DrawioCell>,
): boolean {
const seen = new Set<string>([cell.id]);
let p = cell.parent;
while (p && !seen.has(p)) {
if (p === ancestorId) return true;
seen.add(p);
p = byId.get(p)?.parent;
}
return false;
}
/** Strict interior overlap of two rects (touching edges do NOT count). */
function rectsOverlap(a: Rect, b: Rect): boolean {
return a.x < b.x + b.w && b.x < a.x + a.w && a.y < b.y + b.h && b.y < a.y + a.h;
}
function center(r: Rect): { x: number; y: number } {
return { x: r.x + r.w / 2, y: r.y + r.h / 2 };
}
/**
* Liang-Barsky: does segment p->q pass through the INTERIOR of rect r? Used to
* detect an edge crossing a shape that is not one of its endpoints.
*/
function segCrossesRect(
a: { x: number; y: number },
b: { x: number; y: number },
r: Rect,
): boolean {
const dx = b.x - a.x;
const dy = b.y - a.y;
// Canonical Liang-Barsky: for each of the 4 slabs, p*t <= q.
const p = [-dx, dx, -dy, dy];
const q = [a.x - r.x, r.x + r.w - a.x, a.y - r.y, r.y + r.h - a.y];
let t0 = 0;
let t1 = 1;
for (let i = 0; i < 4; i++) {
if (p[i] === 0) {
if (q[i] < 0) return false; // parallel to this slab AND outside it
continue;
}
const t = q[i] / p[i];
if (p[i] < 0) {
if (t > t1) return false;
if (t > t0) t0 = t;
} else {
if (t < t0) return false;
if (t < t1) t1 = t;
}
}
return t1 > t0; // strictly non-degenerate overlap with the rect interior
}
function cross(
ox: number,
oy: number,
ax: number,
ay: number,
bx: number,
by: number,
): number {
return (ax - ox) * (by - oy) - (ay - oy) * (bx - ox);
}
/** Collinear + overlapping test for two straight segments (edge-on-edge). */
function segmentsOverlap(
a1: { x: number; y: number },
a2: { x: number; y: number },
b1: { x: number; y: number },
b2: { x: number; y: number },
): boolean {
const EPS = 1;
// b1 and b2 must be (near-)collinear with segment a.
if (
Math.abs(cross(a1.x, a1.y, a2.x, a2.y, b1.x, b1.y)) > EPS * dist(a1, a2) ||
Math.abs(cross(a1.x, a1.y, a2.x, a2.y, b2.x, b2.y)) > EPS * dist(a1, a2)
) {
return false;
}
// Project all four points onto the dominant axis and test 1-D overlap length.
const horizontal = Math.abs(a2.x - a1.x) >= Math.abs(a2.y - a1.y);
const pa = horizontal ? [a1.x, a2.x] : [a1.y, a2.y];
const pb = horizontal ? [b1.x, b2.x] : [b1.y, b2.y];
const loA = Math.min(pa[0], pa[1]);
const hiA = Math.max(pa[0], pa[1]);
const loB = Math.min(pb[0], pb[1]);
const hiB = Math.max(pb[0], pb[1]);
const overlap = Math.min(hiA, hiB) - Math.max(loA, loB);
return overlap > 5; // >5px of shared collinear run
}
function dist(
a: { x: number; y: number },
b: { x: number; y: number },
): number {
return Math.hypot(a.x - b.x, a.y - b.y) || 1;
}
/** Approximate rendered text width (px) of a cell value at a font size. */
function estimateLabelWidth(value: string, fontSize: number): number {
// Decode explicit line breaks, strip tags/entities, take the longest line.
const lines = value
.replace(/&#xa;/gi, "\n")
.replace(/<br\s*\/?>/gi, "\n")
.replace(/<[^>]+>/g, "")
.replace(/&[a-z]+;/gi, "x")
.split("\n");
let longest = 0;
for (const l of lines) longest = Math.max(longest, l.trim().length);
// ~0.6em per glyph is a decent average for proportional fonts.
return longest * fontSize * 0.6;
}
/** Page size declared on the model root, defaulting to Letter (850x1100). */
function parsePageSize(modelXml: string): { w: number; h: number } {
const w = /pageWidth="(\d+)"/.exec(modelXml);
const h = /pageHeight="(\d+)"/.exec(modelXml);
return {
w: w ? Number(w[1]) : 850,
h: h ? Number(h[1]) : 1100,
};
}
/** Minimum required gap between adjacent shapes (appendix heuristic). */
export const MIN_SHAPE_GAP = 150;
/**
* Compute the geometry-derived quality warnings for a parsed model. Each is a
* `[rule] message` string. Pure no rendering, no I/O.
*/
export function computeQualityWarnings(
cells: DrawioCell[],
modelXml?: string,
): string[] {
const warnings: string[] = [];
const byId = new Map(cells.map((c) => [c.id, c]));
const verts = cells.filter((c) => c.vertex && c.id !== "0" && c.id !== "1");
const isContainer = (id: string) =>
verts.some((v) => v.parent === id);
const rects = new Map<string, Rect>();
for (const v of verts) {
const r = rectOf(v, byId);
if (r) rects.set(v.id, r);
}
// 1. Shape bbox overlap (excluding a container overlapping its own child).
for (let i = 0; i < verts.length; i++) {
for (let j = i + 1; j < verts.length; j++) {
const a = verts[i];
const b = verts[j];
const ra = rects.get(a.id);
const rb = rects.get(b.id);
if (!ra || !rb) continue;
if (isAncestor(a.id, b, byId) || isAncestor(b.id, a, byId)) continue;
if (rectsOverlap(ra, rb)) {
warnings.push(
`[shape-overlap] shapes "${a.id}" and "${b.id}" overlap; separate them (>=${MIN_SHAPE_GAP}px apart) or use layout:"elk"`,
);
}
}
}
// 2. Edge passing through a non-endpoint LEAF shape's bbox.
const edges = cells.filter((c) => c.edge);
for (const e of edges) {
if (!e.source || !e.target) continue;
const rs = rects.get(e.source);
const rt = rects.get(e.target);
if (!rs || !rt) continue;
const p = center(rs);
const q = center(rt);
for (const v of verts) {
if (v.id === e.source || v.id === e.target) continue;
if (isContainer(v.id)) continue; // an edge legitimately crosses container frames
const rv = rects.get(v.id);
if (!rv) continue;
// shrink to avoid flagging a graze at a shared layer boundary
const shrunk: Rect = { x: rv.x + 6, y: rv.y + 6, w: rv.w - 12, h: rv.h - 12 };
if (shrunk.w <= 0 || shrunk.h <= 0) continue;
if (segCrossesRect(p, q, shrunk)) {
warnings.push(
`[edge-through-shape] edge "${e.id}" passes through shape "${v.id}" (not its source/target); add exitX/exitY/entryX/entryY or a waypoint`,
);
break;
}
}
}
// 3. Edge-on-edge overlap (parallel duplicates or collinear shared runs).
const edgeSegs: { id: string; a: any; b: any; key: string }[] = [];
for (const e of edges) {
if (!e.source || !e.target) continue;
const rs = rects.get(e.source);
const rt = rects.get(e.target);
if (!rs || !rt) continue;
const key = [e.source, e.target].sort().join("::");
edgeSegs.push({ id: e.id, a: center(rs), b: center(rt), key });
}
for (let i = 0; i < edgeSegs.length; i++) {
for (let j = i + 1; j < edgeSegs.length; j++) {
const ea = edgeSegs[i];
const eb = edgeSegs[j];
const dup = ea.key === eb.key;
if (dup || segmentsOverlap(ea.a, ea.b, eb.a, eb.b)) {
warnings.push(
`[edge-overlap] edges "${ea.id}" and "${eb.id}" lie on top of each other; offset one (distinct exit/entry points) or reroute`,
);
}
}
}
// 4. Adjacent SIBLING leaf shapes closer than MIN_SHAPE_GAP.
for (let i = 0; i < verts.length; i++) {
for (let j = i + 1; j < verts.length; j++) {
const a = verts[i];
const b = verts[j];
if ((a.parent ?? "") !== (b.parent ?? "")) continue;
if (isContainer(a.id) || isContainer(b.id)) continue;
const ra = rects.get(a.id);
const rb = rects.get(b.id);
if (!ra || !rb || rectsOverlap(ra, rb)) continue;
const yOverlap = ra.y < rb.y + rb.h && rb.y < ra.y + ra.h;
const xOverlap = ra.x < rb.x + rb.w && rb.x < ra.x + ra.w;
let gap = Infinity;
if (yOverlap) {
gap = Math.min(
gap,
ra.x >= rb.x ? ra.x - (rb.x + rb.w) : rb.x - (ra.x + ra.w),
);
}
if (xOverlap) {
gap = Math.min(
gap,
ra.y >= rb.y ? ra.y - (rb.y + rb.h) : rb.y - (ra.y + ra.h),
);
}
if (gap > 0 && gap < MIN_SHAPE_GAP) {
warnings.push(
`[gap-too-small] shapes "${a.id}" and "${b.id}" are ${Math.round(gap)}px apart (<${MIN_SHAPE_GAP}px); increase spacing`,
);
}
}
}
// 5. Label visibly wider than its shape (skip labels drawn OUTSIDE the shape).
for (const v of verts) {
if (!v.value || isContainer(v.id)) continue;
if (v.styleMap.verticalLabelPosition || v.styleMap.labelPosition) continue;
const r = rects.get(v.id);
if (!r) continue;
const fontSize = Number(v.styleMap.fontSize) || 12;
const est = estimateLabelWidth(v.value, fontSize);
if (est > r.w * 1.15) {
warnings.push(
`[label-overflow] label of "${v.id}" (~${Math.round(est)}px) is wider than its shape (${r.w}px); widen it, shorten the text, or wrap with &#xa;`,
);
}
}
// 6. Negative / off-page (top-left) coordinates.
const page = parsePageSize(modelXml ?? "");
for (const v of verts) {
const r = rects.get(v.id);
if (!r) continue;
if (r.x < 0 || r.y < 0) {
warnings.push(
`[out-of-bounds] shape "${v.id}" has negative coordinates (${Math.round(r.x)},${Math.round(r.y)}); move it into the positive quadrant (page ${page.w}x${page.h})`,
);
}
}
return warnings;
}
// --- linter ----------------------------------------------------------------
/**
@@ -755,17 +1057,20 @@ export function prepareModel(inputXml: string): PreparedModel {
const modelXml = normalizeXml(rawModel);
const bbox = computeBBox(cells);
const cellCount = cells.filter((c) => c.id !== "0" && c.id !== "1").length;
// Geometry quality warnings (non-blocking) are appended to any structural
// warnings from the linter. The model surfaces these and can self-correct.
const quality = computeQualityWarnings(cells, modelXml);
return {
modelXml,
cells,
bbox,
cellCount,
warnings,
warnings: [...warnings, ...quality],
hash: mxHash(modelXml),
};
}
/** Cell count of a decoded model (user cells only) — used by drawio_get meta. */
/** Cell count of a decoded model (user cells only) — used by drawioGet meta. */
export function countUserCells(modelXml: string): number {
return parseCells(modelXml).filter((c) => c.id !== "0" && c.id !== "1").length;
}
+24 -8
View File
@@ -83,16 +83,32 @@ export function filterComment(comment: any, markdownContent?: string) {
};
}
// Map one server search hit to the MCP output contract (#443):
// { pageId, title, path, snippet, score }
//
// INVARIANT: the only page identifier exposed is `pageId` (the server `id`
// UUID). The server also carries `slugId` — it is NEVER surfaced.
//
// GRACEFUL DEGRADATION: against a stock upstream server the opt-in lookup DTO
// fields are stripped, so the response is the legacy FTS shape (no path/snippet/
// score, a `highlight` + `rank` instead). We synthesize the contract from
// whatever is present: `snippet` falls back to the FTS `highlight`, `score` to
// the FTS `rank`, and `path` to [] (upstream has no path). This keeps the tool
// usable even when the server has not been upgraded.
export function filterSearchResult(result: any) {
return {
id: result.id,
pageId: result.id,
title: result.title,
parentPageId: result.parentPageId,
createdAt: result.createdAt,
updatedAt: result.updatedAt,
rank: result.rank,
highlight: result.highlight,
spaceId: result.space?.id,
spaceName: result.space?.name,
path: Array.isArray(result.path) ? result.path : [],
snippet:
typeof result.snippet === "string"
? result.snippet
: (result.highlight ?? ""),
score:
typeof result.score === "number"
? result.score
: typeof result.rank === "number"
? result.rank
: 0,
};
}
@@ -16,8 +16,8 @@
* `insertInlineFootnote` live in `@docmost/prosemirror-markdown` (next to the
* importer's `assembleFootnotes`, #414), so this file stays a pure mirror.
*
* Why it exists: every NON-editor write path (markdown import, update_page_json,
* docmost_transform, insert_footnote) builds ProseMirror JSON directly, so the
* Why it exists: every NON-editor write path (markdown import, updatePageJson,
* docmostTransform, insertFootnote) builds ProseMirror JSON directly, so the
* editor's footnote plugins never run and the canonical topology (sequential
* numbering by first reference, one trailing list, no orphans, no raw `[^id]`)
* was never enforced. Running this at the end of every write path closes that
@@ -28,8 +28,8 @@
* `canonicalizeFootnotes(doc)` before writing the current callers are
* `markdownToProseMirrorCanonical` (page markdown import/update; the plain
* `markdownToProseMirror` used for COMMENT bodies must NOT, or it would drop a
* reference-less definition), `update_page_json`, `docmost_transform`,
* `insert_footnote`, and `copy_page_content`. Append/prepend FRAGMENT writes MUST
* reference-less definition), `updatePageJson`, `docmostTransform`,
* `insertFootnote`, and `copyPageContent`. Append/prepend FRAGMENT writes MUST
* NOT canonicalize. This is deliberately per-call-site (the replace-vs-fragment
* and comment-vs-page nuances make a single naive wrapper unsafe).
*/
@@ -6,7 +6,7 @@
* typographic quotes («»/) vs ASCII "…", em/en-dash vs `-`, non-breaking
* space vs normal space, differing space counts are not recognized as equal
* and "fork": two definitions appear where the author meant one. The existing
* de-dup paths miss this: `footnoteContentKey` (footnote-authoring.ts) only
* de-dup paths miss this: `footnoteContentKey` (@docmost/prosemirror-markdown) only
* collapses ASCII whitespace (quotes/dashes/NBSP untouched), and
* `canonicalizeFootnotes` keys purely by `attrs.id` (the two forks have
* different ids), so neither glues the forks together.
@@ -195,7 +195,7 @@ function stableAttrs(attrs: any): string {
/**
* ATTRS-AWARE merge key for a footnote definition. Deliberately DIVERGES from
* the shared `footnoteContentKey` (footnote-authoring.ts): that key's mark
* the shared `footnoteContentKey` (@docmost/prosemirror-markdown): that key's mark
* signature is TYPE-ONLY (`m.type`), so two definitions with identical visible
* text but marks differing only in ATTRIBUTES most importantly a `link` with a
* different `href` (footnotes are usually citations/links), also `code` /
+8 -8
View File
@@ -283,7 +283,7 @@ export function applyTextEdits(
for (const edit of edits) {
if (!edit.find) throw new Error("edit.find must be a non-empty string");
// HARD-REFUSE formatting changes. edit_page_text edits PLAIN TEXT only and
// HARD-REFUSE formatting changes. editPageText edits PLAIN TEXT only and
// writes the replacement verbatim, so it cannot add/remove marks. We refuse
// only a pure formatting TOGGLE: find and replace differ ONLY by balanced
// markdown markers (e.g. find:"~~$69~~" / replace:"$69", or find:"M5Stack" /
@@ -304,22 +304,22 @@ export function applyTextEdits(
failed.push({
find: edit.find,
reason:
"edit_page_text edits plain text only and cannot add or remove formatting marks (bold/italic/strike/code/link); it writes the replacement as LITERAL text. This edit looks like a formatting change (markdown markers in find/replace). To change marks, read the block with get_page_json and use patch_node (or update_page_json) to set the node's marks array.",
"editPageText edits plain text only and cannot add or remove formatting marks (bold/italic/strike/code/link); it writes the replacement as LITERAL text. This edit looks like a formatting change (markdown markers in find/replace). To change marks, read the block with getPageJson and use patchNode (or updatePageJson) to set the node's marks array.",
});
continue;
}
// HARD-REFUSE inline footnote tokens (#410). `^[...]` in a `replace` is
// markdown that only becomes a real footnote when a whole markdown body is
// written (create_page / update_page_content / import_page_markdown). Written
// through edit_page_text it stays a LITERAL string in the text — the exact
// written (createPage / update_page_content / importPageMarkdown). Written
// through editPageText it stays a LITERAL string in the text — the exact
// failure mode #410 fixes — so refuse it here (defense-in-depth) and point the
// caller at insert_footnote, mirroring the formatting-marker refusal above.
// caller at insertFootnote, mirroring the formatting-marker refusal above.
if (/\^\[[\s\S]*?\]/.test(edit.replace)) {
failed.push({
find: edit.find,
reason:
"edit_page_text writes the replacement as LITERAL text, so a `^[...]` footnote token does not parse into a real footnote (it would appear verbatim in the page). To add a footnote to existing text, use insert_footnote (anchorText = where, text = the note).",
"editPageText writes the replacement as LITERAL text, so a `^[...]` footnote token does not parse into a real footnote (it would appear verbatim in the page). To add a footnote to existing text, use insertFootnote (anchorText = where, text = the note).",
});
continue;
}
@@ -381,12 +381,12 @@ export function applyTextEdits(
let reason: string;
if (existsAcrossAtom) {
reason =
"match crosses a non-text inline node (image/break/mention); use update_page_json for structural changes.";
"match crosses a non-text inline node (image/break/mention); use updatePageJson for structural changes.";
} else {
// Append a bounded "closest text" hint: find the FIRST block that
// contains the longest whitespace-delimited token (>= 3 chars) of the
// (stripped, then raw) locator, and quote that block's plain text. Shared
// with create_comment via closestBlockHint so both give the same hint.
// with createComment via closestBlockHint so both give the same hint.
reason = "text not found in the document." + closestBlockHint(blockPlain, edit.find);
}
failed.push({ find: edit.find, reason });
+243
View File
@@ -0,0 +1,243 @@
/**
* Single-BLOCK markdown fragment support for `patch_node` / `insert_node`
* (#413). These tools accept EITHER a raw ProseMirror `node` (fine attr/mark
* work) OR a `markdown` string (the recommended default): a small markdown
* fragment is run through the canonical importer, yielding the SAME topology a
* full-page markdown import would so a block written via markdown is
* canonically identical to the same content imported whole (no "second canon").
*
* The importer produces a full `{type:"doc", content:[...blocks..., footnotesList?]}`.
* A fragment write needs the BLOCKS separately from the footnote DEFINITIONS so
* the caller can splice the blocks into the live document and merge the
* definitions into the page's TAIL footnote list via the existing footnote
* machinery (`insertInlineFootnote`'s `appendDefinition` + `canonicalizeFootnotes`).
*
* Footnote id-collision safety: the importer assigns sequential ids (`fn-1`,
* `fn-2`, ) starting from 1 for EVERY fragment, so a fragment's `fn-1` would
* collide with an existing page footnote also numbered `fn-1` and
* `canonicalizeFootnotes` matches references to definitions BY id, so the
* fragment's reference would silently re-hang onto the page's unrelated
* definition. To make the merge safe regardless of the page's current numbering,
* every fragment footnote id is REMAPPED to a fresh uuid (via the importer's own
* `generateFootnoteId`) across BOTH the references (inside the blocks) and the
* definitions before either is handed back. Content-identical notes still merge
* downstream via `normalizeAndMergeFootnotes` (content-key), and the whole doc is
* renumbered by `canonicalizeFootnotes`, so the caller-visible numbering stays
* canonical.
*/
import { markdownToProseMirror } from "./collaboration.js";
import { generateFootnoteId } from "@docmost/prosemirror-markdown";
import { docmostSchema } from "./docmost-schema.js";
/** True if `value` is a non-null, non-array object. */
function isObject(value: any): value is Record<string, any> {
return value != null && typeof value === "object" && !Array.isArray(value);
}
/**
* Deep-walk `node` collecting every footnote id it uses (on `footnoteReference`
* and `footnoteDefinition` nodes) and build a stable OLD->NEW remap, minting a
* fresh uuid per distinct old id. The map is shared across a fragment's blocks
* and definitions so a reference and its definition receive the SAME new id.
*/
function buildFootnoteIdRemap(nodes: any[]): Map<string, string> {
const remap = new Map<string, string>();
const visit = (node: any): void => {
if (!isObject(node)) return;
if (
(node.type === "footnoteReference" ||
node.type === "footnoteDefinition") &&
isObject(node.attrs) &&
typeof node.attrs.id === "string" &&
node.attrs.id !== ""
) {
if (!remap.has(node.attrs.id)) {
remap.set(node.attrs.id, generateFootnoteId());
}
}
if (Array.isArray(node.content)) {
for (const child of node.content) visit(child);
}
};
for (const n of nodes) visit(n);
return remap;
}
/** Rewrite every footnote id in `node` IN PLACE using `remap` (deep). */
function applyFootnoteIdRemap(node: any, remap: Map<string, string>): void {
if (!isObject(node)) return;
if (
(node.type === "footnoteReference" || node.type === "footnoteDefinition") &&
isObject(node.attrs) &&
typeof node.attrs.id === "string"
) {
const next = remap.get(node.attrs.id);
if (next) node.attrs.id = next;
}
if (Array.isArray(node.content)) {
for (const child of node.content) applyFootnoteIdRemap(child, remap);
}
}
/**
* Generate a short random block id for an imported block that arrives without one
* (the markdown importer emits `attrs.id: null`). Mirrors the mcp `freshId`
* convention (base36 random, unique within one document). The patch path then
* OVERWRITES the first block's id with the target id; every other block keeps the
* fresh id minted here so a 1 -> N section rewrite yields addressable,
* comment-anchorable blocks rather than a run of null-id paragraphs.
*/
function freshBlockId(): string {
return (
Math.random().toString(36).slice(2, 12) +
Math.random().toString(36).slice(2, 6)
);
}
/**
* Assign a fresh id to every top-level block whose `attrs.id` is null/missing,
* IN PLACE. Only the block's own id is touched (not descendants those keep the
* importer's structure). Ensures each imported block is independently addressable.
*/
function assignFreshBlockIds(blocks: any[]): void {
for (const b of blocks) {
if (!isObject(b)) continue;
if (!isObject(b.attrs)) b.attrs = {};
if (b.attrs.id == null || b.attrs.id === "") {
b.attrs.id = freshBlockId();
}
}
}
/** The parsed shape of a markdown fragment: its blocks + footnote definitions. */
export interface MarkdownFragment {
/** Top-level blocks, in order, with the trailing `footnotesList` removed. */
blocks: any[];
/**
* The `footnoteDefinition` nodes lifted from the imported `footnotesList`, with
* ids already remapped to match the references left inside `blocks`. Empty when
* the fragment used no footnotes.
*/
definitions: any[];
}
/**
* Import a markdown fragment and return its blocks separately from its footnote
* definitions, with all footnote ids remapped to fresh uuids (see the file
* header). The importer's `^[body]` inline-footnote handling is used verbatim
* `^[...]` in the fragment is a first-class footnote, NOT rejected so the
* markdown path matches the full-page import exactly.
*
* Throws when the fragment imports to zero blocks (an empty / whitespace-only
* markdown string is not a valid block write).
*/
export async function importMarkdownFragment(
markdown: string,
): Promise<MarkdownFragment> {
const doc = await markdownToProseMirror(markdown);
const content: any[] = Array.isArray(doc?.content) ? doc.content : [];
const blocks: any[] = [];
const definitions: any[] = [];
for (const node of content) {
if (isObject(node) && node.type === "footnotesList") {
// Lift the definitions out of the list; the list wrapper itself is
// reconstructed on the page by the canonicalizer after the merge.
if (Array.isArray(node.content)) {
for (const def of node.content) {
if (isObject(def) && def.type === "footnoteDefinition") {
definitions.push(def);
}
}
}
continue;
}
blocks.push(node);
}
if (blocks.length === 0) {
throw new Error(
"markdown fragment produced no blocks — provide non-empty markdown, or use `node` for a raw ProseMirror node",
);
}
// Remap footnote ids across BOTH blocks and definitions so a fragment `fn-1`
// cannot collide with a page footnote of the same number.
const remap = buildFootnoteIdRemap([...blocks, ...definitions]);
if (remap.size > 0) {
for (const b of blocks) applyFootnoteIdRemap(b, remap);
for (const d of definitions) applyFootnoteIdRemap(d, remap);
}
// Every top-level block needs a stable id (the importer leaves them null). The
// patch path OVERWRITES the first block's id with the target id afterwards.
assignFreshBlockIds(blocks);
return { blocks, definitions };
}
/**
* True when `type` is a valid TOP-LEVEL child of the document node per the
* canonical schema's content model i.e. `get_node` can serialize it to
* markdown by wrapping it in `{type:"doc",content:[node]}`. Derived from the
* schema's `doc` contentMatch (NOT a hand-written type list) so it tracks the
* schema automatically: `tableRow`/`tableCell`/`tableHeader` (addressed only via
* `#<index>`) are NOT doc children and yield false, so `get_node` auto-falls back
* to JSON for them.
*/
export function canBeDocChild(type: string | undefined): boolean {
if (typeof type !== "string") return false;
const nodeType = docmostSchema.nodes[type];
if (!nodeType) return false;
return docmostSchema.nodes.doc.contentMatch.matchType(nodeType) != null;
}
/**
* Table-cell attributes that CANNOT survive a markdown round-trip: the converter
* emits colspan/rowspan (and align) as HTML `<table>` cell attrs, but silently
* drops `colwidth`, `backgroundColor`, and `backgroundColorName`. A markdown
* `patch_node` on a block that carries any of these (a merged / colored /
* fixed-width cell) would therefore lose them so it is REJECTED, pointing the
* caller at the table tools or the raw-`node` JSON path. `align` is intentionally
* absent: it round-trips as GFM alignment.
*/
function cellCarriesUnrepresentableAttrs(node: any): boolean {
if (!isObject(node)) return false;
if (node.type !== "tableCell" && node.type !== "tableHeader") return false;
const a = isObject(node.attrs) ? node.attrs : {};
if ((a.colspan ?? 1) > 1) return true;
if ((a.rowspan ?? 1) > 1) return true;
if (a.colwidth != null) return true;
if (a.backgroundColor != null) return true;
if (a.backgroundColorName != null) return true;
return false;
}
/**
* Scan a target block (the node being replaced) for any table cell carrying an
* attribute markdown cannot represent (colspan/rowspan/colwidth/background). When
* one is found, return a human-readable list of the offending attr NAMES so the
* caller can build an actionable rejection message; return null when the block is
* safe to rewrite from markdown. Deep a colored cell nested inside a table
* inside a callout is still caught.
*/
export function findUnrepresentableTableAttrs(node: any): string | null {
const found = new Set<string>();
const visit = (n: any): void => {
if (!isObject(n)) return;
if (cellCarriesUnrepresentableAttrs(n)) {
const a = isObject(n.attrs) ? n.attrs : {};
if ((a.colspan ?? 1) > 1) found.add("colspan");
if ((a.rowspan ?? 1) > 1) found.add("rowspan");
if (a.colwidth != null) found.add("colwidth");
if (a.backgroundColor != null) found.add("backgroundColor");
if (a.backgroundColorName != null) found.add("backgroundColorName");
}
if (Array.isArray(n.content)) {
for (const child of n.content) visit(child);
}
};
visit(node);
return found.size > 0 ? Array.from(found).sort().join(", ") : null;
}
+33
View File
@@ -10,6 +10,20 @@
const chains = new Map<string, Promise<unknown>>();
// Canonical UUID shape (versions 1–8, matching the `uuid` package's `validate`
// that the server's isValidUUID uses). This is the SINGLE source of truth for
// "is this a canonical page UUID?" in the MCP: client.ts's resolvePageId
// imports isUuid from here to decide whether a pageId already IS a UUID (and so
// needs no /pages/info round-trip). page.repo.ts treats any non-UUID pageId as
// a slugId; a 10-char nanoid slugId never contains dashes, so it can never be
// misread as a UUID here.
export const UUID_RE =
/^[0-9a-f]{8}-[0-9a-f]{4}-[1-8][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
export function isUuid(value: string): boolean {
return typeof value === "string" && UUID_RE.test(value);
}
// The returned promise carries the real result/rejection of `fn` and MUST be
// awaited/handled by the caller; only the internal chaining tail swallows
// errors (purely to gate ordering).
@@ -17,6 +31,25 @@ export function withPageLock<T>(
pageId: string,
fn: () => Promise<T>,
): Promise<T> {
// STRUCTURAL INVARIANT (issue #449, "resolve-then-lock"): the mutex key MUST
// be the canonical page UUID, never a raw slugId. The whole write path relies
// on the lock key AND the CollabSession cache key being the resolved UUID
// (#260) — if a future write method forgot to call resolvePageId and locked
// under a slugId, two writes to the same page would take DIFFERENT mutex keys
// and silently lose serialization (clobbering live human edits). This was an
// invariant enforced only by comments/convention; assert it in CODE so the
// violation fails fast and loud at the lock instead of corrupting data in
// prod. The centralizing helper (mutatePageContent/replacePageContent) already
// guards a raw-input caller, but this backstop catches ANY path.
if (!isUuid(pageId)) {
throw new Error(
`withPageLock: key must be a canonical page UUID, got '${pageId}'. ` +
`The write path must resolvePageId(pageId) BEFORE locking so the ` +
`mutex/CollabSession cache key is the UUID (invariant "resolve-then-` +
`lock", #260/#449). A slugId or other non-UUID key would silently lose ` +
`per-page serialization.`,
);
}
// Wait for the previous op on this page; swallow its error so a failure does
// not poison the queue for the next caller.
const prev = (chains.get(pageId) ?? Promise.resolve()).catch(() => {});
+12 -12
View File
@@ -3,7 +3,7 @@
*
* `searchInDoc(doc, query, opts)` finds every occurrence of a literal substring
* (default) or a regular expression across the page's TEXT CONTAINERS and
* reports WHERE each match is the container's ref (for get_node/patch_node;
* reports WHERE each match is the container's ref (for getNode/patchNode;
* see the SearchMatch.nodeId note for the `#<index>` caveat), the top-level
* block index, and a short context window around the hit. It never touches the
* network, the DB, or the schema mirror; like `comment-anchor.ts` it is
@@ -69,24 +69,24 @@ export interface SearchOptions {
/** One located occurrence. */
export interface SearchMatch {
/**
* The container's ref, for addressing the block with get_node/patch_node: its
* The container's ref, for addressing the block with getNode/patchNode: its
* `attrs.id` when it has one, otherwise `#<topLevelIndex>` of the nearest
* top-level block. Table-cell/list-item paragraphs that carry no id fall back
* to the `#<index>` form.
*
* CAVEAT: the `#<index>` form is accepted by get_node (getNodeByRef resolves
* it by top-level index) but NOT by patch_node (replaceNodeById resolves only
* CAVEAT: the `#<index>` form is accepted by getNode (getNodeByRef resolves
* it by top-level index) but NOT by patchNode (replaceNodeById resolves only
* by `attrs.id`), so id-less table/cell content can be READ by this ref but
* not PATCHED by it.
*
* To anchor a comment, do NOT pass this ref to create_comment it has no
* To anchor a comment, do NOT pass this ref to createComment it has no
* nodeId parameter. A top-level comment needs an exact-text `selection` that
* occurs once on the page (it fails if the text isn't found), so build a
* UNIQUE `selection` from before+match+after and pass THAT as create_comment's
* UNIQUE `selection` from before+match+after and pass THAT as createComment's
* `selection`.
*/
nodeId: string;
/** The top-level block index (as in get_outline). */
/** The top-level block index (as in getOutline). */
blockIndex: number;
/** The container node's type (paragraph/heading/...). */
type: string | undefined;
@@ -188,12 +188,12 @@ export function searchInDoc(
// --- edge-case guards (fail loudly so the agent can correct the call) ---
if (typeof query !== "string" || query.trim().length === 0) {
throw new Error(
"search_in_page: query is empty — pass the text (or regex) to look for.",
"searchInPage: query is empty — pass the text (or regex) to look for.",
);
}
if (query.length > MAX_PATTERN_LENGTH) {
throw new Error(
`search_in_page: query is too long (${query.length} chars; max ${MAX_PATTERN_LENGTH}). Shorten the search text/pattern.`,
`searchInPage: query is too long (${query.length} chars; max ${MAX_PATTERN_LENGTH}). Shorten the search text/pattern.`,
);
}
@@ -212,7 +212,7 @@ export function searchInDoc(
re = new RE2(query, caseSensitive ? "g" : "gi");
} catch (e) {
throw new Error(
`search_in_page: invalid or unsupported regular expression: ${
`searchInPage: invalid or unsupported regular expression: ${
e instanceof Error ? e.message : String(e)
} RE2 does not support lookaround ((?=)/(?<=)) or backreferences (\\1); rewrite the pattern without them.`,
);
@@ -237,9 +237,9 @@ export function searchInDoc(
// in a very long container.
const text = blockPlainText(node);
// The container's own id addresses it verbatim in get_node/patch_node; a
// The container's own id addresses it verbatim in getNode/patchNode; a
// container with no id (e.g. a table-cell paragraph) falls back to the
// top-level block's #<index> (readable via get_node, but not patchable —
// top-level block's #<index> (readable via getNode, but not patchable —
// see the SearchMatch.nodeId note).
const id =
isObject(node.attrs) && typeof node.attrs.id === "string" && node.attrs.id.length > 0
+1 -1
View File
@@ -117,7 +117,7 @@ export function stripInlineMarkdown(s: string): string {
/**
* Build a bounded "closest text" hint for an anchor/find MISS, shared by
* edit_page_text (json-edit) and create_comment (client) so both surface the
* editPageText (json-edit) and createComment (client) so both surface the
* same self-correction affordance.
*
* Take the longest whitespace-delimited token (>= 3 chars) of the locator
+56 -1
View File
@@ -740,7 +740,7 @@ export function insertInlineFootnote(
// subtree, so a reference is never glued inside an existing definition (which
// the canonicalizer would then drop as an orphan, losing that definition's
// prose); and forbidBlockTypes refuses codeBlocks (an inline atom there is a
// schema-invalid doc; insert_footnote skips validateDocStructure).
// schema-invalid doc; insertFootnote skips validateDocStructure).
// When the only anchor match is in such a place, the insert is refused and the
// write aborts cleanly (inserted:false) instead of destroying content.
const boundaryIdx = Array.isArray(doc?.content)
@@ -774,6 +774,61 @@ export function insertInlineFootnote(
return { doc: working, inserted: true, footnoteId, reused };
}
/**
* Merge an ARRAY of footnote definitions (e.g. the definitions lifted from an
* imported markdown FRAGMENT) into `doc`\'s footnote list, then re-derive the
* canonical footnote topology the SAME two-step machinery `insertInlineFootnote`
* uses (`appendDefinition` -> `normalizeAndMergeFootnotes` -> `canonicalizeFootnotes`).
*
* The fragment\'s `footnoteReference` nodes are assumed to ALREADY be spliced into
* `doc` (inside the just-inserted blocks) with ids matching these definitions, so
* after appending the definitions the canonicalizer orders/numbers everything by
* first-reference order, merges content-identical notes, and drops any orphan.
* Same documented caveat as every other write path: full canonicalization drops a
* definition no reference points at.
*
* NOT merely a no-op when `definitions` is empty: it still canonicalizes when
* the (post-splice) `doc` carries footnote artifacts (a `footnotesList` or any
* `footnoteReference`), so a splice that removed the LAST referrer of a page
* footnote drops the now-orphaned definition matching a full page re-import
* (which always canonicalizes) and preserving the "canonically identical to the
* same content imported whole" invariant. A truly footnote-free doc (no artifacts
* and no definitions) is returned untouched the fast path, no clone. When the
* work runs it goes through the pure passes (which clone), so the caller\'s `doc`
* is not mutated.
*/
export function mergeFootnoteDefinitions(doc: any, definitions: any[]): any {
const defs = Array.isArray(definitions) ? definitions : [];
// True fast path ONLY when there is nothing to merge AND nothing to canonicalize
// away; otherwise fall through so an orphan left by a splice is still dropped.
if (defs.length === 0 && !hasFootnoteArtifacts(doc)) return doc;
// Clone before appending: `appendDefinition` mutates in place, and the caller
// must not see a half-merged doc if a later pass throws.
let working = clone(doc);
for (const def of defs) {
appendDefinition(working, def);
}
// #419: normalize + merge glyph-forked definitions before canonicalizing.
working = normalizeAndMergeFootnotes(working);
working = canonicalizeFootnotes(working);
return working;
}
/**
* True if `doc`'s tree contains any `footnotesList` node OR any
* `footnoteReference` node. Used to decide whether an empty-`definitions` merge
* must still canonicalize (to drop an orphan a splice left behind).
*/
function hasFootnoteArtifacts(doc: any): boolean {
let found = false;
walk(doc, (n) => {
if (isObject(n) && (n.type === "footnotesList" || n.type === "footnoteReference")) {
found = true;
}
});
return found;
}
/**
* Append a definition node so the canonicalizer can order/place it: into the
* first existing footnotesList, or a new trailing list when none exists.
+85 -8
View File
@@ -1,11 +1,30 @@
/**
* Options for `buildPageTree`. Fully OPTIONAL so the existing call form
* `buildPageTree(nodes)` keeps its historic behaviour (lean `{id, slugId,
* title, children?}` output, no depth cut) unchanged.
*
* - `shape: "getTree"` emit the #443 `getTree` output node shape
* `{pageId, title, children?, hasChildren?}` instead of the lean
* `{id, slugId, title, children?}` shape. `slugId`/`icon`/`position` are
* never exposed (INVARIANT: only the UUID `pageId` leaves the MCP layer).
* - `maxDepth` trim the built tree to this many levels (root nodes are
* depth 1). Only meaningful together with `shape: "getTree"` (the lean shape
* has no `hasChildren` to signal a cut). See the depth logic below.
*/
export interface BuildPageTreeOptions {
shape?: "lean" | "getTree";
maxDepth?: number;
}
/**
* Pure tree-builder: turn a flat array of sidebar-style page nodes (as produced
* by `enumerateSpacePages`) into a nested tree.
*
* Input: a flat array of nodes. Each node is expected to carry at least
* { id, slugId, title, position, parentPageId } (extra fields are ignored).
* { id, slugId, title, position, parentPageId } (extra fields are ignored),
* plus a server `hasChildren` boolean used by the `getTree` shape below.
*
* Output: an array of ROOT nodes, each shaped as
* Output (default / `shape: "lean"`): an array of ROOT nodes, each shaped as
* { id, slugId, title, children? }
* where `children` is the array of child nodes (same shape, recursively). The
* `children` key is OMITTED entirely when a node has no children consistent
@@ -13,6 +32,14 @@
* lean (nesting alone conveys the structure; parentPageId/position/hasChildren
* are intentionally dropped from the output).
*
* Output (`shape: "getTree"`, the #443 tool shape): each node is
* { pageId, title, children?, hasChildren? }
* the server `id` is exposed as `pageId` (never `slugId`/`icon`/`position`).
* `children` is omitted for leaves and for nodes trimmed by `maxDepth`.
* `hasChildren: true` is set ONLY on a node whose children exist on the server
* (per the flat item's `hasChildren`) but were CUT by `maxDepth`; on leaves and
* on fully-expanded interior nodes the field is omitted (see `maxDepth` below).
*
* Linking rule: a node is attached as a child of `parentPageId` only when that
* parent id is actually present in the input. Otherwise including a null /
* undefined `parentPageId`, or a parent that was capped out of the bounded walk
@@ -26,18 +53,42 @@
* fractional-index ASCII keys (e.g. "a0", "a1"). Nodes with a missing/undefined
* `position` sort last.
*
* maxDepth (getTree shape only): the tree is built in FULL first, then trimmed
* on the way out. Root nodes are depth 1. `maxDepth: N` keeps nodes at depth
* <= N and drops the `children` of any node AT depth N. A node whose children
* were dropped this way gets `hasChildren: true` when it actually had children
* in the flat input (source of truth = the server `hasChildren` flag), so the
* caller knows it can descend further with a follow-up `rootPageId` call. An
* absent/undefined `maxDepth` means no cut (whole tree). `maxDepth <= 0` is
* treated as "no cut" (defensive; the tool schema clamps to >= 1).
*
* Pure: no I/O, no network, deterministic.
*/
export function buildPageTree(nodes: any[]): any[] {
type OutputNode = {
export function buildPageTree(
nodes: any[],
options: BuildPageTreeOptions = {},
): any[] {
const getTreeShape = options.shape === "getTree";
// A finite, positive cut only; anything else means "no cut".
const maxDepth =
typeof options.maxDepth === "number" &&
Number.isFinite(options.maxDepth) &&
options.maxDepth > 0
? Math.floor(options.maxDepth)
: undefined;
type InternalNode = {
id: string;
// Retained internally for shaping; never all emitted at once.
slugId: any;
title: any;
children?: OutputNode[];
hasServerChildren: boolean;
children?: InternalNode[];
};
// Map id -> output node. Build the lean output shape up front.
const byId = new Map<string, OutputNode>();
// Map id -> internal node. Build up front; the output shape is projected at
// the very end so the maxDepth cut can consult `hasServerChildren`.
const byId = new Map<string, InternalNode>();
// Preserve the original position string for sorting (kept off the output).
const positionById = new Map<string, string | undefined>();
@@ -49,6 +100,7 @@ export function buildPageTree(nodes: any[]): any[] {
id: node.id,
slugId: node.slugId,
title: node.title,
hasServerChildren: node.hasChildren === true,
});
positionById.set(node.id, node.position);
}
@@ -90,5 +142,30 @@ export function buildPageTree(nodes: any[]): any[] {
}
roots.sort(byPosition);
return roots.map((id) => byId.get(id)!);
const rootNodes = roots.map((id) => byId.get(id)!);
// Project the internal nodes into the requested OUTPUT shape, applying the
// maxDepth cut for the getTree shape. `depth` is 1-based (roots = depth 1).
const project = (node: InternalNode, depth: number): any => {
if (getTreeShape) {
const out: any = { pageId: node.id, title: node.title };
const atCut = maxDepth !== undefined && depth >= maxDepth;
if (!atCut && node.children && node.children.length > 0) {
out.children = node.children.map((c) => project(c, depth + 1));
} else if (atCut && node.hasServerChildren) {
// Children exist on the server but were trimmed by maxDepth: signal it
// so the caller can descend with a follow-up rootPageId call.
out.hasChildren = true;
}
return out;
}
// Lean (historic) shape: cycle-safe, no depth cut, no hasChildren.
const out: any = { id: node.id, slugId: node.slugId, title: node.title };
if (node.children && node.children.length > 0) {
out.children = node.children.map((c) => project(c, depth + 1));
}
return out;
};
return rootNodes.map((n) => project(n, 1));
}
+246
View File
@@ -0,0 +1,246 @@
// SERVER_INSTRUCTIONS — the editing guide surfaced to MCP clients in the
// initialize result so they can pick the right tool by intent and avoid
// resending whole documents.
//
// This guide is split into TWO parts that are composed at the bottom:
//
// 1. ROUTING_PROSE — the hand-written "when to use what" intent hints (READ /
// EDIT / PAGES / COMMENTS / HISTORY). This is legitimately manual: it
// encodes editorial judgement (which tool for which situation, the cheap-
// first ordering, the guardrail nudges) that cannot be derived from the
// registry. It is NOT the drift-guard for the tool set.
//
// 2. A GENERATED <tool_inventory> — every tool the server registers, listed
// by name + one-line purpose, grouped by family, built from the SAME
// registry the server registers tools from (SHARED_TOOL_SPECS' mcpName +
// catalogLine) PLUS the handful of inline MCP-only tools (their inventory
// lines live in INLINE_MCP_INVENTORY below). Because this list is BUILT
// from the registry, it can never drift out of sync with the registered
// tools — adding/renaming/removing a spec changes it automatically, with no
// prose edit and no scraper test. An unmapped tool still appears (under
// "OTHER"), so a new tool can never silently vanish from the guide.
//
// This replaces the old hand-maintained monolithic guide + its regex scraper
// test (test/unit/server-instructions.test.mjs), which only checked that every
// registered name appeared SOMEWHERE in the prose and drifted whenever a name
// was reworded.
//
// OUT OF SCOPE (issue #448): the README / README.ru tool catalogs are still
// hand-maintained prose and are NOT generated from this registry. Regenerating
// them from SHARED_TOOL_SPECS is tracked separately as an optional docs script
// under issue #412 — until then a tool rename still needs a manual README edit.
import { SHARED_TOOL_SPECS, SharedToolSpec } from "./tool-specs.js";
/**
* The hand-written routing prose the intent hints that tell a client which
* tool to reach for in which situation. Kept manual on purpose (it encodes
* editorial judgement, not a mechanical name list). The generated inventory
* below is spliced in after it.
*/
export const ROUTING_PROSE =
"Docmost editing guide — choose the tool by intent. The <tool_inventory> at the end lists every tool with a one-line purpose; the notes below are the routing hints for WHEN to reach for each.\n" +
"READ: find a page by a fragment of a technical string (hostname/IP/ID like srv.local, 10.0.12, WB-MGE-30D86B) -> search — hybrid substring + full-text, returns each hit's location (path: root->parent titles) and a snippet around the match, so you rarely need a follow-up getPage; scope with spaceId or parentPageId (a subtree), titleOnly to match titles only. A space's page HIERARCHY (or one subtree) -> getTree (one request, complete, `{pageId,title,children?}`; rootPageId for a subtree, maxDepth to trim depth — a trimmed node gets hasChildren:true); prefer it over listPages tree:true (deprecated). Have a pageId, need WHERE-AM-I / what's around it (its breadcrumbs + direct children, metadata only) -> getPageContext (one call; parent = last breadcrumb, [] for a root page). list -> listPages / listSpaces. Locate blocks and their ids CHEAPLY -> getOutline (compact top-level map; start here, not getPageJson). One block, for editing -> getNode (by attrs.id, or \"#<index>\" for tables, which carry no id) — returns MARKDOWN by default (comment anchors kept for safe write-back); pass format:\"json\" for the raw ProseMirror subtree. Find every occurrence of a string/regex ON a page (and where each is) -> searchInPage, NOT block-by-block getNode — it returns each hit's node ref + block index + context for a targeted comment. Whole page -> getPage (Markdown, canonical for text; drops only block ids, resolved-comment anchors, and a fixed no-md-representation attr set: table spans/colwidth/bg, indent, callout.icon, orderedList.type, link internal/target/rel/class; inline <span data-comment-id> tags are comment anchors — markup, not text) or getPageJson (full ProseMirror with block ids, for those dropped attrs). Hand a huge page (with images) to an external consumer without pulling it through the model context -> stashPage (returns a short-lived anonymous URL).\n" +
"EDIT: fix wording/typos/numbers -> editPageText (find/replace inside blocks, no node id needed). Edit a block -> getNode(markdown) -> edit the markdown -> patchNode(markdown) (by attrs.id from getOutline; the markdown fragment may be several blocks — a 1->N section rewrite in one call, the first block keeps the id). Reach for patchNode's `node`-JSON only for fine attr/mark work; a table cell with spans/colors/fixed width -> the table tools (patchNode markdown refuses it). Add a block -> insertNode (markdown, before/after a block by attrs.id or by anchor text, or append; `node` for raw JSON or bare table structure). Remove a block -> deleteNode (by attrs.id). Tables -> tableGet / tableUpdateCell / tableInsertRow / tableDeleteRow (address by \"#<index>\" from getOutline; table nodes have no attrs.id). Images -> insertImage (add from a web URL) / replaceImage (swap an existing image). Draw.io diagrams -> PREFER the high-level semantic tools that hide coordinates/styles: drawioFromGraph (architecture/cloud/network diagrams — describe nodes/groups/edges by kind+icon, the server picks layout, colors and verified icons; hints layer/sameLayerAs/pinned and layout:full|incremental|none) and drawioFromMermaid (standard flowcharts — write Mermaid, get an editable diagram). For targeted tweaks of an existing diagram use drawioEditCells (id-based add/update/delete with cascade delete + baseHash lock). Raw mxGraph XML via drawioCreate/drawioUpdate is the escape-hatch for exotic/wireframe diagrams; drawioGet reads a diagram as mxGraph XML + a hash (pass it as baseHash to drawioUpdate/drawioEditCells for optimistic locking). Before authoring raw XML, drawioShapes (look up verified stencil style-strings so a shape name never renders as an empty box) and drawioGuide (on-demand authoring reference: skeleton/layout/containers/icons-aws/icons-azure), and pass layout:\"elk\" to drawioCreate/drawioUpdate to auto-place nodes. Footnotes -> insertFootnote. Bulk/structural rewrite -> updatePageJson (full ProseMirror replace) or updatePageMarkdown (full plain-Markdown body replace, re-imported — block ids regenerate); prefer the granular tools above to avoid resending the whole ~100KB+ document. Complex/scripted rewrite (multiple coordinated edits, renumbering) -> docmostTransform: write a JS `(doc, ctx) => doc` transform, preview the diff with dryRun (default), then apply with dryRun:false; ctx.helpers includes commentsToFootnotes for turning inline comments into numbered footnotes.\n" +
"PAGES: new -> createPage (Markdown). Rename (title only) -> renamePage. Move -> movePage. Delete -> deletePage (SOFT delete — the page goes to trash and is restorable; nothing is permanent). Copy/replace a page's whole content from another page (server-side, no document through the model) -> copyPageContent. Sharing -> sharePage / unsharePage / listShares; sharePage makes the page PUBLICLY accessible — do it only when explicitly asked.\n" +
"COMMENTS: createComment is always inline and requires an EXACT selection — contiguous text from a single block, <=250 chars (fails rather than leaving an unanchored comment); reply to a thread via parentCommentId. Propose a concrete text fix for one-click human approval -> createComment with suggestedText (the exact plain-text replacement for the selection; the selection must then be UNIQUE in the page — extend it with context if needed); prefer this over editing directly when the change is subjective or needs the author's sign-off. Manage -> listComments, updateComment, resolveComment (resolve/reopen, reversible — prefer over delete to close), deleteComment, checkNewComments.\n" +
"HISTORY: review what changed -> diffPageVersions (a historyId vs current, or two versions). List saved versions -> listPageHistory. Undo a bad edit -> restorePageVersion (writes a past version back as current; itself revertible). Export a page to self-contained Docmost Markdown (with comment anchors) -> exportPageMarkdown.";
/**
* A single generated inventory line: the tool's registered NAME + a one-line
* purpose. For a registry tool the purpose is its `catalogLine` (falling back
* to the first sentence of its description); for an inline MCP-only tool it is
* the hand-written line in INLINE_MCP_INVENTORY.
*/
export interface ToolInventoryLine {
name: string;
purpose: string;
}
/**
* The families the inventory is grouped under, in display order. A tool is
* placed by looking its mcpName up in TOOL_FAMILY; anything not listed there
* falls into "OTHER" so it is never dropped from the guide.
*/
const FAMILY_ORDER = [
"READ",
"EDIT",
"PAGES",
"COMMENTS",
"HISTORY",
"OTHER",
] as const;
type Family = (typeof FAMILY_ORDER)[number];
/**
* mcpName -> family for the generated inventory grouping. Purely cosmetic (it
* orders the inventory to mirror the routing prose); an unmapped tool still
* appears under OTHER, so forgetting to add an entry here can never drop a tool
* from the guide it only lands it in the catch-all group.
*/
const TOOL_FAMILY: Record<string, Family> = {
// READ
search: "READ",
listPages: "READ",
getTree: "READ",
getPageContext: "READ",
listSpaces: "READ",
getOutline: "READ",
getNode: "READ",
searchInPage: "READ",
getPage: "READ",
getPageJson: "READ",
getWorkspace: "READ",
stashPage: "READ",
// EDIT
editPageText: "EDIT",
patchNode: "EDIT",
insertNode: "EDIT",
deleteNode: "EDIT",
updatePageJson: "EDIT",
updatePageMarkdown: "EDIT",
tableGet: "EDIT",
tableUpdateCell: "EDIT",
tableInsertRow: "EDIT",
tableDeleteRow: "EDIT",
insertImage: "EDIT",
replaceImage: "EDIT",
insertFootnote: "EDIT",
drawioGet: "EDIT",
drawioCreate: "EDIT",
drawioUpdate: "EDIT",
drawioEditCells: "EDIT",
drawioFromGraph: "EDIT",
drawioFromMermaid: "EDIT",
drawioShapes: "EDIT",
drawioGuide: "EDIT",
docmostTransform: "EDIT",
// PAGES
createPage: "PAGES",
renamePage: "PAGES",
movePage: "PAGES",
deletePage: "PAGES",
copyPageContent: "PAGES",
sharePage: "PAGES",
unsharePage: "PAGES",
listShares: "PAGES",
// COMMENTS
createComment: "COMMENTS",
listComments: "COMMENTS",
updateComment: "COMMENTS",
resolveComment: "COMMENTS",
deleteComment: "COMMENTS",
checkNewComments: "COMMENTS",
// HISTORY
diffPageVersions: "HISTORY",
listPageHistory: "HISTORY",
restorePageVersion: "HISTORY",
exportPageMarkdown: "HISTORY",
// importPageMarkdown is now inAppOnly (#411) — it is not registered on the
// external MCP host, so it no longer appears in the generated inventory.
};
/**
* Inventory lines for the INLINE MCP-only tools the ones registered directly
* in index.ts (not via SHARED_TOOL_SPECS) because they diverge per transport or
* exist only on this standalone surface. They carry no `catalogLine`, so their
* one-line purpose is hand-written here. This is the ONLY hand-maintained tool
* list left, and it is tiny; a new inline tool without an entry here is caught
* by the completeness guard in `tool-inventory.test.mjs`.
*/
export const INLINE_MCP_INVENTORY: ToolInventoryLine[] = [
{
name: "tableGet",
purpose:
"read a table as a matrix of cell texts + per-cell paragraph ids.",
},
{
name: "search",
purpose:
"find pages by a fragment of a technical string (hybrid substring + full-text); returns each hit's path and a snippet.",
},
{
name: "docmostTransform",
purpose:
"edit a page by running a sandboxed JS `(doc, ctx) => doc` transform, with a dryRun diff preview.",
},
{
name: "updateComment",
purpose: "update an existing comment's content (creator only).",
},
{
name: "deleteComment",
purpose: "delete a comment (creator or space admin only).",
},
];
/**
* Derive the one-line purpose for a registry spec: prefer its hand-written
* `catalogLine` (already a "name — purpose" line we take the purpose after
* the em dash), else fall back to the first sentence of its description.
*/
function purposeForSpec(spec: SharedToolSpec): string {
const line = spec.catalogLine?.trim();
if (line) {
const dash = line.indexOf(" — ");
if (dash >= 0) return line.slice(dash + 3).trim();
return line;
}
const desc = (spec.description ?? "").replace(/\s+/g, " ").trim();
const firstSentence = desc.split(/(?<=[.!?])\s/)[0];
return firstSentence || desc || "(no description)";
}
/**
* Build the flat list of every registered tool's inventory line: one per shared
* registry spec (skipping `inAppOnly` specs, which are not registered on this
* MCP host) PLUS every inline MCP-only tool. Pure and deterministic the
* registry drives it, so it can never drift from what index.ts registers.
*/
export function buildToolInventoryLines(
specs: Record<string, SharedToolSpec> = SHARED_TOOL_SPECS,
inline: ToolInventoryLine[] = INLINE_MCP_INVENTORY,
): ToolInventoryLine[] {
const lines: ToolInventoryLine[] = [];
for (const spec of Object.values(specs)) {
if (spec.inAppOnly) continue; // not registered on the MCP host
lines.push({ name: spec.mcpName, purpose: purposeForSpec(spec) });
}
for (const l of inline) lines.push({ ...l });
return lines;
}
/**
* Render the generated `<tool_inventory>` block: every tool name + purpose,
* grouped by family (families in FAMILY_ORDER; tools within a family sorted by
* name for stable output; unmapped tools fall into OTHER). Pure.
*/
export function buildToolInventory(
specs: Record<string, SharedToolSpec> = SHARED_TOOL_SPECS,
inline: ToolInventoryLine[] = INLINE_MCP_INVENTORY,
): string {
const byFamily = new Map<Family, ToolInventoryLine[]>();
for (const family of FAMILY_ORDER) byFamily.set(family, []);
for (const line of buildToolInventoryLines(specs, inline)) {
const family = TOOL_FAMILY[line.name] ?? "OTHER";
byFamily.get(family)!.push(line);
}
const sections: string[] = [];
for (const family of FAMILY_ORDER) {
const items = byFamily.get(family)!;
if (items.length === 0) continue;
items.sort((a, b) => a.name.localeCompare(b.name));
for (const item of items) {
sections.push(` ${family} ${item.name}${item.purpose}`);
}
}
return ["<tool_inventory>", ...sections, "</tool_inventory>"].join("\n");
}
/**
* The composed editing guide: the hand-written routing prose followed by the
* generated, drift-proof tool inventory. Exported (and used by index.ts /
* createDocmostMcpServer) as the MCP server's `instructions`.
*/
export const SERVER_INSTRUCTIONS =
ROUTING_PROSE + "\n" + buildToolInventory();
File diff suppressed because it is too large Load Diff
+76 -76
View File
@@ -84,20 +84,20 @@ async function main() {
let pageId = null;
try {
// 1. create_page: title with spaces must survive (was: underscores bug)
// 1. createPage: title with spaces must survive (was: underscores bug)
const created = await client.createPage("Тест апгрейда MCP сервера", MD, spaceId);
pageId = created.data.id;
check("create_page: title keeps spaces", created.data.title === "Тест апгрейда MCP сервера", created.data.title);
check("create_page: slugId exposed", typeof created.data.slugId === "string" && created.data.slugId.length > 0, created.data.slugId);
check("createPage: title keeps spaces", created.data.title === "Тест апгрейда MCP сервера", created.data.title);
check("createPage: slugId exposed", typeof created.data.slugId === "string" && created.data.slugId.length > 0, created.data.slugId);
// 2. get_page_json: raw ProseMirror with callout + table
// 2. getPageJson: raw ProseMirror with callout + table
const pj = await client.getPageJson(pageId);
const types = pj.content.content.map((n) => n.type);
check("get_page_json: callout node present", types.includes("callout"), types.join(","));
check("get_page_json: table node present", types.includes("table"));
check("get_page_json: slugId present", !!pj.slugId);
check("getPageJson: callout node present", types.includes("callout"), types.join(","));
check("getPageJson: table node present", types.includes("table"));
check("getPageJson: slugId present", !!pj.slugId);
// 3. edit_page_text: surgical replace, ids preserved
// 3. editPageText: surgical replace, ids preserved
const idsBefore = JSON.stringify(
pj.content.content.filter((n) => n.attrs?.id).map((n) => n.attrs.id),
);
@@ -105,26 +105,26 @@ async function main() {
{ find: "БУКВОЕД", replace: "КНИГОЛЮБ" },
{ find: "[1]", replace: "[42]" },
]);
check("edit_page_text: both edits applied", editRes.applied.every((e) => e.replacements === 1));
check("editPageText: both edits applied", editRes.applied.every((e) => e.replacements === 1));
await new Promise((r) => setTimeout(r, 16000)); // wait for server persistence
const pj2 = await client.getPageJson(pageId);
const text2 = JSON.stringify(pj2.content);
check("edit_page_text: replacement visible", text2.includes("КНИГОЛЮБ") && text2.includes("[42]"));
check("edit_page_text: old text gone", !text2.includes("БУКВОЕД"));
check("editPageText: replacement visible", text2.includes("КНИГОЛЮБ") && text2.includes("[42]"));
check("editPageText: old text gone", !text2.includes("БУКВОЕД"));
const idsAfter = JSON.stringify(
pj2.content.content.filter((n) => n.attrs?.id).map((n) => n.attrs.id),
);
check("edit_page_text: block ids preserved", idsBefore === idsAfter);
check("edit_page_text: callout survived", JSON.stringify(pj2.content).includes('"callout"'));
check("edit_page_text: table survived", pj2.content.content.some((n) => n.type === "table"));
check("editPageText: block ids preserved", idsBefore === idsAfter);
check("editPageText: callout survived", JSON.stringify(pj2.content).includes('"callout"'));
check("editPageText: table survived", pj2.content.content.some((n) => n.type === "table"));
// 4. error reporting: ambiguous and missing finds
let err1 = "";
try { await client.editPageText(pageId, [{ find: "Колонка", replace: "X" }]); } catch (e) { err1 = e.message; }
check("edit_page_text: ambiguous match rejected", err1.includes("matches"), err1);
check("editPageText: ambiguous match rejected", err1.includes("matches"), err1);
let err2 = "";
try { await client.editPageText(pageId, [{ find: "НЕСУЩЕСТВУЮЩЕЕ", replace: "X" }]); } catch (e) { err2 = e.message; }
check("edit_page_text: missing text reported", err2.includes("not found"), err2);
check("editPageText: missing text reported", err2.includes("not found"), err2);
// 5. update_page (markdown): table + callout must survive the re-import
await client.updatePage(pageId, MD + "\nДобавленный абзац.\n");
@@ -137,21 +137,21 @@ async function main() {
const cellText = JSON.stringify(tableNode);
check("update_page md: table cells intact", cellText.includes("четыре") && cellText.includes("Колонка А"));
// 6. update_page_json: lossless write round-trip
// 6. updatePageJson: lossless write round-trip
pj3.content.content.push({
type: "paragraph",
attrs: { id: "testidjsonpush", indent: 0, textAlign: null },
content: [{ type: "text", text: "Абзац, добавленный через update_page_json." }],
content: [{ type: "text", text: "Абзац, добавленный через updatePageJson." }],
});
await client.updatePageJson(pageId, pj3.content);
await new Promise((r) => setTimeout(r, 16000));
const pj4 = await client.getPageJson(pageId);
const lastNode = pj4.content.content[pj4.content.content.length - 1];
check("update_page_json: paragraph appended", JSON.stringify(pj4.content).includes("добавленный через update_page_json"));
check("update_page_json: custom node id preserved", lastNode.attrs?.id === "testidjsonpush", lastNode.attrs?.id);
check("updatePageJson: paragraph appended", JSON.stringify(pj4.content).includes("добавленный через updatePageJson"));
check("updatePageJson: custom node id preserved", lastNode.attrs?.id === "testidjsonpush", lastNode.attrs?.id);
// 6b. images: upload / insert / replace (clean src, fresh attachment on replace).
// insert_image / replace_image take an http(s) URL that the SERVER fetches;
// insertImage / replaceImage take an http(s) URL that the SERVER fetches;
// local file paths are intentionally unsupported. The Docmost server runs on
// the same host as this test, so serve the PNG bytes over a throwaway
// localhost HTTP server it can reach.
@@ -186,13 +186,13 @@ async function main() {
validateStatus: () => true,
});
// insert_image: append the first PNG, src must be clean (no ?v=) and fetchable.
// insertImage: append the first PNG, src must be clean (no ?v=) and fetchable.
const ins = await client.insertImage(pageId, urlA);
check("insert_image: src has no ?v= cache-buster", !ins.src.includes("?v="), ins.src);
check("insertImage: src has no ?v= cache-buster", !ins.src.includes("?v="), ins.src);
const fileA = await fetchFile(ins.src);
check("insert_image: file fetch returns 200", fileA.status === 200, `status=${fileA.status}`);
check("insertImage: file fetch returns 200", fileA.status === 200, `status=${fileA.status}`);
check(
"insert_image: content-type is image/*",
"insertImage: content-type is image/*",
String(fileA.headers["content-type"] || "").startsWith("image/"),
String(fileA.headers["content-type"]),
);
@@ -209,25 +209,25 @@ async function main() {
};
const imgNode = findImage(pjImg.content.content);
const oldAttachmentId = imgNode?.attrs?.attachmentId;
check("insert_image: image node present after persist", !!oldAttachmentId, oldAttachmentId);
check("insertImage: image node present after persist", !!oldAttachmentId, oldAttachmentId);
// replace_image: must create a NEW attachment with a clean, fetchable URL.
// replaceImage: must create a NEW attachment with a clean, fetchable URL.
// The 200 fetch is the assertion that catches the in-place-overwrite HTTP 500 regression.
const rep = await client.replaceImage(pageId, oldAttachmentId, urlB);
check("replace_image: new attachment id differs from old", rep.newAttachmentId !== oldAttachmentId, `${oldAttachmentId} -> ${rep.newAttachmentId}`);
check("replace_image: src has no ?v= cache-buster", !rep.src.includes("?v="), rep.src);
check("replaceImage: new attachment id differs from old", rep.newAttachmentId !== oldAttachmentId, `${oldAttachmentId} -> ${rep.newAttachmentId}`);
check("replaceImage: src has no ?v= cache-buster", !rep.src.includes("?v="), rep.src);
const fileB = await fetchFile(rep.src);
check("replace_image: new file fetch returns 200", fileB.status === 200, `status=${fileB.status}`);
check("replaceImage: new file fetch returns 200", fileB.status === 200, `status=${fileB.status}`);
check(
"replace_image: new content-type is image/*",
"replaceImage: new content-type is image/*",
String(fileB.headers["content-type"] || "").startsWith("image/"),
String(fileB.headers["content-type"]),
);
await new Promise((r) => setTimeout(r, 16000));
const pjImg2 = await client.getPageJson(pageId);
check("replace_image: page has new attachment id", !!findImage(pjImg2.content.content, rep.newAttachmentId), rep.newAttachmentId);
check("replace_image: old attachment id repointed away", !findImage(pjImg2.content.content, oldAttachmentId), oldAttachmentId);
check("replaceImage: page has new attachment id", !!findImage(pjImg2.content.content, rep.newAttachmentId), rep.newAttachmentId);
check("replaceImage: old attachment id repointed away", !findImage(pjImg2.content.content, oldAttachmentId), oldAttachmentId);
} finally {
imgServer.close();
}
@@ -275,10 +275,10 @@ async function main() {
await client.editPageText(fid, [{ find: "PRICEMARK", replace: "$& costs $100" }]);
await new Promise((r) => setTimeout(r, 16000));
const ftext = JSON.stringify((await client.getPageJson(fid)).content);
check("feature: edit_page_text inserts $-pattern literally (no $& expansion)", ftext.includes("$& costs $100") && !ftext.includes("PRICEMARK costs"));
check("feature: editPageText inserts $-pattern literally (no $& expansion)", ftext.includes("$& costs $100") && !ftext.includes("PRICEMARK costs"));
let badThrew = false;
try { await client.replaceImage(fid, "00000000-0000-0000-0000-000000000000", featPng); } catch (e) { badThrew = /no image with attachmentId/.test(e.message); }
check("feature: replace_image with unknown id throws (no orphan upload)", badThrew);
check("feature: replaceImage with unknown id throws (no orphan upload)", badThrew);
} finally {
try { await client.deletePage(fid); } catch {}
try { unlinkSync(featPng); } catch {}
@@ -286,7 +286,7 @@ async function main() {
}
// 6d. node ops: patch / insert / delete a block by id on a throwaway page.
// Three paragraphs are written with KNOWN ids via update_page_json so the
// Three paragraphs are written with KNOWN ids via updatePageJson so the
// ids can be targeted directly; each op is verified via getPageJson after
// the standard 16s persistence wait.
{
@@ -348,7 +348,7 @@ async function main() {
}
}
// 6e. rename_page: title-only update must leave the content untouched.
// 6e. renamePage: title-only update must leave the content untouched.
{
const rp = await client.createPage("E2E rename before " + Date.now(), "Rename body marker RENAMEBODY.", spaceId);
const rid = rp.data.id;
@@ -357,19 +357,19 @@ async function main() {
const beforeContent = JSON.stringify(beforeJson);
const newTitle = "E2E rename AFTER " + Date.now();
const rr = await client.renamePage(rid, newTitle);
check("rename_page: returns success+title", rr.success === true && rr.title === newTitle, JSON.stringify(rr));
check("renamePage: returns success+title", rr.success === true && rr.title === newTitle, JSON.stringify(rr));
await new Promise((r) => setTimeout(r, 16000));
const afterJson = await client.getPageJson(rid);
check("rename_page: title changed", afterJson.title === newTitle, afterJson.title);
check("rename_page: content unchanged", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("RENAMEBODY"));
check("renamePage: title changed", afterJson.title === newTitle, afterJson.title);
check("renamePage: content unchanged", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("RENAMEBODY"));
const afterMd = (await client.getPage(rid)).data;
check("rename_page: get_page reflects new title", afterMd.title === newTitle, afterMd.title);
check("renamePage: getPage reflects new title", afterMd.title === newTitle, afterMd.title);
} finally {
try { await client.deletePage(rid); } catch {}
}
}
// 6f. update_page_json title-only: omitting content updates the title and
// 6f. updatePageJson title-only: omitting content updates the title and
// leaves the body intact; supplying neither content nor title throws.
{
const up = await client.createPage("E2E upj-title before " + Date.now(), "Title-only body marker UPJTITLEBODY.", spaceId);
@@ -378,20 +378,20 @@ async function main() {
const beforeContent = JSON.stringify((await client.getPageJson(uid)).content);
const newTitle = "E2E upj-title AFTER " + Date.now();
const ur = await client.updatePageJson(uid, undefined, newTitle);
check("update_page_json title-only: succeeds", ur.success === true, JSON.stringify(ur));
check("updatePageJson title-only: succeeds", ur.success === true, JSON.stringify(ur));
await new Promise((r) => setTimeout(r, 16000));
const afterJson = await client.getPageJson(uid);
check("update_page_json title-only: title updated", afterJson.title === newTitle, afterJson.title);
check("update_page_json title-only: content intact", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("UPJTITLEBODY"));
check("updatePageJson title-only: title updated", afterJson.title === newTitle, afterJson.title);
check("updatePageJson title-only: content intact", JSON.stringify(afterJson.content) === beforeContent && beforeContent.includes("UPJTITLEBODY"));
let upjErr = "";
try { await client.updatePageJson(uid); } catch (e) { upjErr = e.message; }
check("update_page_json: neither content nor title throws", upjErr.includes("nothing to update"), upjErr);
check("updatePageJson: neither content nor title throws", upjErr.includes("nothing to update"), upjErr);
} finally {
try { await client.deletePage(uid); } catch {}
}
}
// 6g. copy_page_content: B's body becomes a copy of A's body, server-side,
// 6g. copyPageContent: B's body becomes a copy of A's body, server-side,
// while B's title/slugId stay put. Both pages are throwaways.
{
let aid = null;
@@ -409,24 +409,24 @@ async function main() {
const aNodeCount = aJson.content.content.length;
const cr = await client.copyPageContent(aid, bid);
check("copy_page_content: returns success + node count", cr.success === true && cr.copiedNodes === aNodeCount, JSON.stringify(cr));
check("copyPageContent: returns success + node count", cr.success === true && cr.copiedNodes === aNodeCount, JSON.stringify(cr));
await new Promise((r) => setTimeout(r, 16000));
const bAfter = await client.getPageJson(bid);
const bText = JSON.stringify(bAfter.content);
check("copy_page_content: B now has A's marker", bText.includes("COPYSOURCE"));
check("copy_page_content: B's old marker gone", !bText.includes("COPYTARGET"));
check("copy_page_content: B node count equals A's", bAfter.content.content.length === aNodeCount, `${bAfter.content.content.length} vs ${aNodeCount}`);
check("copy_page_content: B title unchanged", bAfter.title === bTitleBefore, bAfter.title);
check("copy_page_content: B slugId unchanged", bAfter.slugId === bSlugBefore, bAfter.slugId);
check("copyPageContent: B now has A's marker", bText.includes("COPYSOURCE"));
check("copyPageContent: B's old marker gone", !bText.includes("COPYTARGET"));
check("copyPageContent: B node count equals A's", bAfter.content.content.length === aNodeCount, `${bAfter.content.content.length} vs ${aNodeCount}`);
check("copyPageContent: B title unchanged", bAfter.title === bTitleBefore, bAfter.title);
check("copyPageContent: B slugId unchanged", bAfter.slugId === bSlugBefore, bAfter.slugId);
// Source must be left untouched by the copy.
const aAfter = JSON.stringify((await client.getPageJson(aid)).content);
check("copy_page_content: source page unchanged", aAfter === JSON.stringify(aJson.content) && aAfter.includes("COPYSOURCE"));
check("copyPageContent: source page unchanged", aAfter === JSON.stringify(aJson.content) && aAfter.includes("COPYSOURCE"));
let copyErr = "";
try { await client.copyPageContent(aid, aid); } catch (e) { copyErr = e.message; }
check("copy_page_content: self-copy rejected", copyErr.includes("same page"), copyErr);
check("copyPageContent: self-copy rejected", copyErr.includes("same page"), copyErr);
} finally {
try { if (bid) await client.deletePage(bid); } catch {}
try { if (aid) await client.deletePage(aid); } catch {}
@@ -435,22 +435,22 @@ async function main() {
// 7. shares: create (idempotent), public access, list, unshare
const share = await client.sharePage(pageId);
check("share_page: returns public URL", share.publicUrl?.startsWith(`${APP}/share/`), share.publicUrl);
check("sharePage: returns public URL", share.publicUrl?.startsWith(`${APP}/share/`), share.publicUrl);
const share2 = await client.sharePage(pageId);
check("share_page: idempotent", share2.key === share.key);
check("sharePage: idempotent", share2.key === share.key);
const anon = await axios.post(`${API}/shares/page-info`, { pageId: pj4.slugId, shareId: share.key }, { validateStatus: () => true });
check("share_page: anonymous access works", anon.status === 200);
check("sharePage: anonymous access works", anon.status === 200);
const shares = await client.listShares();
check("list_shares: contains our page", shares.some((s) => s.pageId === pageId && s.publicUrl === share.publicUrl));
check("listShares: contains our page", shares.some((s) => s.pageId === pageId && s.publicUrl === share.publicUrl));
const un = await client.unsharePage(pageId);
check("unshare_page: success", un.success === true);
check("unsharePage: success", un.success === true);
const anon2 = await axios.post(`${API}/shares/page-info`, { pageId: pj4.slugId, shareId: share.key }, { validateStatus: () => true });
check("unshare_page: public access revoked", anon2.status !== 200, `status=${anon2.status}`);
check("unsharePage: public access revoked", anon2.status !== 200, `status=${anon2.status}`);
// 8. get_page markdown round-trip sanity (table separator present)
// 8. getPage markdown round-trip sanity (table separator present)
const md = await client.getPage(pageId);
check("get_page md: table separator emitted", md.data.content.includes("| --- |"), "");
check("get_page md: callout exported as Obsidian '> [!info]'", md.data.content.includes("> [!info]"));
check("getPage md: table separator emitted", md.data.content.includes("| --- |"), "");
check("getPage md: callout exported as Obsidian '> [!info]'", md.data.content.includes("> [!info]"));
// 9. comments: create / list / reply / update / check_new / delete
const beforeComments = new Date(Date.now() - 1000).toISOString();
@@ -458,34 +458,34 @@ async function main() {
// that exists in the persisted page to anchor on. "Добавленный абзац." is a
// plain paragraph re-imported in section 5 and still present here.
const c1 = await client.createComment(pageId, "Первый **комментарий** с [ссылкой](https://example.com).", "inline", "Добавленный абзац.");
check("create_comment: created", !!c1.data.id, c1.data.id);
check("create_comment: markdown round-trip", c1.data.content.includes("**комментарий**"), c1.data.content);
check("createComment: created", !!c1.data.id, c1.data.id);
check("createComment: markdown round-trip", c1.data.content.includes("**комментарий**"), c1.data.content);
const reply = await client.createComment(pageId, "Ответ на комментарий.", "page", undefined, c1.data.id);
check("create_comment: reply has parent", reply.data.parentCommentId === c1.data.id);
check("createComment: reply has parent", reply.data.parentCommentId === c1.data.id);
const list = (await client.listComments(pageId)).items;
check("list_comments: both visible", list.length === 2, `count=${list.length}`);
check("listComments: both visible", list.length === 2, `count=${list.length}`);
await client.updateComment(c1.data.id, "Обновлённый текст комментария.");
const got = await client.getComment(c1.data.id);
check("update_comment + get_comment: content updated", got.data.content.includes("Обновлённый"), got.data.content);
check("updateComment + get_comment: content updated", got.data.content.includes("Обновлённый"), got.data.content);
const news = await client.checkNewComments(spaceId, beforeComments, pageId);
check("check_new_comments: finds new comments in subtree", news.totalNewComments >= 2, `total=${news.totalNewComments}`);
// resolve_comment: close the top-level thread, verify resolvedAt surfaces, then reopen
check("checkNewComments: finds new comments in subtree", news.totalNewComments >= 2, `total=${news.totalNewComments}`);
// resolveComment: close the top-level thread, verify resolvedAt surfaces, then reopen
const resolvedRes = await client.resolveComment(c1.data.id, true);
check("resolve_comment: marks resolved", resolvedRes.success === true && resolvedRes.resolved === true);
check("resolveComment: marks resolved", resolvedRes.success === true && resolvedRes.resolved === true);
// c1 is now resolved; the default feed hides resolved threads, so pass
// includeResolved:true to still see it and assert its resolvedAt (#328).
const listResolved = (await client.listComments(pageId, true)).items;
const c1Resolved = listResolved.find((c) => c.id === c1.data.id);
check("resolve_comment: resolvedAt set in list", !!c1Resolved?.resolvedAt, `resolvedAt=${c1Resolved?.resolvedAt}`);
check("resolveComment: resolvedAt set in list", !!c1Resolved?.resolvedAt, `resolvedAt=${c1Resolved?.resolvedAt}`);
const reopenedRes = await client.resolveComment(c1.data.id, false);
check("resolve_comment: reopen succeeds", reopenedRes.resolved === false);
check("resolveComment: reopen succeeds", reopenedRes.resolved === false);
const listReopened = (await client.listComments(pageId)).items;
const c1Reopened = listReopened.find((c) => c.id === c1.data.id);
check("resolve_comment: resolvedAt cleared on reopen", !c1Reopened?.resolvedAt, `resolvedAt=${c1Reopened?.resolvedAt}`);
check("resolveComment: resolvedAt cleared on reopen", !c1Reopened?.resolvedAt, `resolvedAt=${c1Reopened?.resolvedAt}`);
await client.deleteComment(reply.data.id);
await client.deleteComment(c1.data.id);
const listAfter = (await client.listComments(pageId)).items;
check("delete_comment: comments removed", listAfter.length === 0, `count=${listAfter.length}`);
check("deleteComment: comments removed", listAfter.length === 0, `count=${listAfter.length}`);
} finally {
if (pageId) {
await client.deletePage(pageId);
@@ -1,4 +1,4 @@
// Mock collab regression for the AMBIGUOUS-id refusal in patch_node / delete_node
// Mock collab regression for the AMBIGUOUS-id refusal in patchNode / deleteNode
// (#159, PR #185 review pt 1). When a page has TWO blocks sharing one attrs.id
// (Docmost duplicates block ids on copy/paste), the transform's
// `if (replaced !== 1) return null` / `if (deleted !== 1) return null` guard must
@@ -126,18 +126,20 @@ after(async () => {
);
});
test("patch_node REFUSES an ambiguous (duplicate) id without writing to collab", async () => {
test("patchNode REFUSES an ambiguous (duplicate) id without writing to collab", async () => {
const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects(
() =>
client.patchNode("11111111-1111-4111-8111-111111111111", DUP_ID, {
type: "paragraph",
content: [{ type: "text", text: "replacement" }],
node: {
type: "paragraph",
content: [{ type: "text", text: "replacement" }],
},
}),
/ambiguous/i,
"patch_node must reject a duplicate-id target with an 'ambiguous' error",
"patchNode must reject a duplicate-id target with an 'ambiguous' error",
);
assert.equal(
@@ -147,14 +149,14 @@ test("patch_node REFUSES an ambiguous (duplicate) id without writing to collab",
);
});
test("delete_node REFUSES an ambiguous (duplicate) id without writing to collab", async () => {
test("deleteNode REFUSES an ambiguous (duplicate) id without writing to collab", async () => {
const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects(
() => client.deleteNode("22222222-2222-4222-8222-222222222222", DUP_ID),
/ambiguous/i,
"delete_node must reject a duplicate-id target with an 'ambiguous' error",
"deleteNode must reject a duplicate-id target with an 'ambiguous' error",
);
assert.equal(
@@ -203,14 +203,16 @@ test("a reply creates without selection or anchoring and is stored as type 'page
"reply body",
"inline",
undefined,
"parent-123",
// #437: a parentCommentId must be a full canonical UUID.
"019f499a-9f8c-7d68-b7be-ce100d7c6c56",
);
assert.equal(result.success, true, "a reply must resolve successfully");
assert.ok(createPayload, "/comments/create must have been called");
assert.equal(
createPayload.parentCommentId,
"parent-123",
// #437: a parentCommentId must be a full canonical UUID.
"019f499a-9f8c-7d68-b7be-ce100d7c6c56",
"the reply payload must carry the parentCommentId",
);
assert.equal(
@@ -321,7 +323,9 @@ test("suggestedText on a reply is rejected", async () => {
"body",
"inline",
undefined,
"parent-1",
// #437: use a valid full UUID so the reply+suggestion rejection fires
// (not the id-shape guard).
"019f499a-9f8c-7d68-b7be-ce100d7c6c56",
"replacement",
),
/reply/i,
@@ -0,0 +1,272 @@
// Contract tests for the stage-3 drawio client methods (issue #425):
// drawioEditCells / drawioFromGraph / drawioFromMermaid. Same seam-override
// pattern as drawio-tools.test.mjs: a DocmostClient subclass stubs the I/O seams
// so the tool logic runs without a live Docmost / collab socket.
import { test } from "node:test";
import assert from "node:assert/strict";
import { DocmostClient } from "../../build/client.js";
import {
buildDrawioSvg,
normalizeXml,
mxHash,
decodeDrawioSvg,
parseCells,
} from "../../build/lib/drawio-xml.js";
const DRAWIO_SCHEMA_ATTRS = new Set([
"src", "title", "alt", "width", "height", "size", "aspectRatio", "align", "attachmentId",
]);
function applyDrawioSchemaDrop(node) {
if (!node || typeof node !== "object") return;
if (node.type === "drawio" && node.attrs && typeof node.attrs === "object") {
for (const key of Object.keys(node.attrs))
if (!DRAWIO_SCHEMA_ATTRS.has(key)) delete node.attrs[key];
}
if (Array.isArray(node.content)) for (const c of node.content) applyDrawioSchemaDrop(c);
}
function svgFor(model, bbox = { width: 400, height: 300 }) {
return buildDrawioSvg(normalizeXml(model), "<g/>", bbox);
}
function makeClient({ pageDoc, attachmentSvg } = {}) {
const calls = { uploads: [], mutations: [] };
class TestClient extends DocmostClient {
async ensureAuthenticated() {}
async getCollabTokenWithReauth() {
return "collab-token";
}
async resolvePageId(pageId) {
return `uuid-${pageId}`;
}
async getPageRaw(pageId) {
return {
id: pageId, slugId: "s", title: "P", spaceId: "sp",
content: pageDoc ?? { type: "doc", content: [] },
};
}
async uploadAttachmentBuffer(pageId, buffer, fileName) {
const id = `att-${calls.uploads.length + 1}`;
calls.uploads.push({ pageId, fileName, svg: buffer.toString("utf-8") });
return { id, fileName, fileSize: buffer.length };
}
async fetchAttachmentText() {
return attachmentSvg;
}
mutatePage(pageId, token, apiUrl, transform) {
const clone = structuredClone(pageDoc ?? { type: "doc", content: [] });
const doc = transform(clone);
if (doc) applyDrawioSchemaDrop(doc);
calls.mutations.push({ pageId, doc });
return Promise.resolve({ doc, verify: { changed: doc != null } });
}
}
const client = new TestClient("http://127.0.0.1:1/api", "e@x.com", "pw");
return { client, calls };
}
function findDrawio(node, acc = []) {
if (!node || typeof node !== "object") return acc;
if (node.type === "drawio") acc.push(node);
if (Array.isArray(node.content)) for (const c of node.content) findDrawio(c, acc);
return acc;
}
// A stored diagram: a group with two children and an edge.
const STORED =
"<mxGraphModel><root><mxCell id=\"0\"/><mxCell id=\"1\" parent=\"0\"/>" +
'<mxCell id="grp" value="G" style="container=1;fillColor=none;" vertex="1" parent="1">' +
'<mxGeometry x="0" y="0" width="300" height="200" as="geometry"/></mxCell>' +
'<mxCell id="a" value="A" style="rounded=1;" vertex="1" parent="grp">' +
'<mxGeometry x="10" y="10" width="80" height="40" as="geometry"/></mxCell>' +
'<mxCell id="b" value="B" style="rounded=1;" vertex="1" parent="grp">' +
'<mxGeometry x="10" y="90" width="80" height="40" as="geometry"/></mxCell>' +
'<mxCell id="e" style="" edge="1" parent="grp" source="a" target="b">' +
'<mxGeometry relative="1" as="geometry"/></mxCell>' +
"</root></mxGraphModel>";
function drawioPageDoc() {
return {
type: "doc",
content: [
{
type: "drawio",
attrs: {
id: "d1", src: "/api/files/att-1/diagram.drawio.svg",
attachmentId: "att-1", width: 400, height: 300,
},
},
],
};
}
// --- drawioEditCells --------------------------------------------------------
test("drawioEditCells: applies ops and repoints the node (current baseHash)", async () => {
const { client, calls } = makeClient({
pageDoc: drawioPageDoc(),
attachmentSvg: svgFor(STORED),
});
const baseHash = mxHash(normalizeXml(STORED));
const res = await client.drawioEditCells(
"page1",
"d1",
[
{
op: "update",
cellId: "a",
xml:
'<mxCell id="a" value="Renamed" style="rounded=1;" vertex="1" parent="grp">' +
'<mxGeometry x="10" y="10" width="80" height="40" as="geometry"/></mxCell>',
},
],
baseHash,
);
assert.equal(res.success, true);
assert.equal(calls.uploads.length, 1);
const written = decodeDrawioSvg(calls.uploads[0].svg);
const cells = parseCells(written);
assert.equal(cells.find((c) => c.id === "a").value, "Renamed");
assert.equal(cells.find((c) => c.id === "b").value, "B"); // untouched
const n = findDrawio(calls.mutations[0].doc)[0];
// The stub numbers uploads from 1; this edit is the first upload -> att-1.
assert.equal(n.attrs.attachmentId, "att-1");
});
test("drawioEditCells: delete of the container cascades to children + edge", async () => {
const { client, calls } = makeClient({
pageDoc: drawioPageDoc(),
attachmentSvg: svgFor(STORED),
});
const baseHash = mxHash(normalizeXml(STORED));
await client.drawioEditCells("page1", "d1", [{ op: "delete", cellId: "grp" }], baseHash);
const written = decodeDrawioSvg(calls.uploads[0].svg);
const ids = parseCells(written).filter((c) => c.id !== "0" && c.id !== "1").map((c) => c.id);
assert.deepEqual(ids, [], "grp + a + b + edge all cascaded away");
});
test("drawioEditCells: stale baseHash -> conflict, no upload", async () => {
const { client, calls } = makeClient({
pageDoc: drawioPageDoc(),
attachmentSvg: svgFor(STORED),
});
await assert.rejects(
() => client.drawioEditCells("page1", "d1", [{ op: "delete", cellId: "a" }], "stale"),
/conflict/,
);
assert.equal(calls.uploads.length, 0);
});
test("drawioEditCells: baseHash is mandatory", async () => {
const { client } = makeClient({ pageDoc: drawioPageDoc(), attachmentSvg: svgFor(STORED) });
await assert.rejects(
() => client.drawioEditCells("page1", "d1", [{ op: "delete", cellId: "a" }], ""),
/baseHash is mandatory/,
);
});
// --- drawioFromGraph --------------------------------------------------------
test("drawioFromGraph: builds a diagram from a graph and inserts a node", async () => {
const pageDoc = { type: "doc", content: [{ type: "paragraph", attrs: { id: "p1" }, content: [] }] };
const { client, calls } = makeClient({ pageDoc });
const res = await client.drawioFromGraph(
"page1",
{ position: "append" },
{
nodes: [
{ id: "api", label: "API", kind: "gateway", icon: "aws:api_gateway", group: "vpc" },
{ id: "fn", label: "Handler", kind: "service", icon: "aws:lambda", group: "vpc" },
{ id: "db", label: "Orders", kind: "db", icon: "aws:dynamodb" },
],
groups: [{ id: "vpc", label: "VPC" }],
edges: [{ from: "api", to: "fn", kind: "sync" }, { from: "fn", to: "db", kind: "async" }],
},
"LR",
"default",
);
assert.equal(res.success, true);
assert.equal(res.nodeId, "#1");
assert.equal(res.iconsMissing.length, 0, `unresolved: ${res.iconsMissing}`);
assert.equal(res.iconsResolved, 3);
// The uploaded model decodes back and carries the group + nodes.
const written = decodeDrawioSvg(calls.uploads[0].svg);
const cells = parseCells(written);
assert.ok(cells.some((c) => c.id === "vpc"));
assert.ok(cells.some((c) => c.id === "api"));
// Group is transparent.
const vpc = cells.find((c) => c.id === "vpc");
assert.equal(vpc.styleMap.fillColor, "none");
assert.equal(vpc.styleMap.container, "1");
});
test("drawioFromGraph: an invalid graph throws before any upload", async () => {
const { client, calls } = makeClient({ pageDoc: { type: "doc", content: [] } });
await assert.rejects(
() => client.drawioFromGraph("page1", { position: "append" }, { nodes: [] }),
/non-empty/,
);
assert.equal(calls.uploads.length, 0);
});
test("drawioFromGraph incremental into an existing node keeps prior coords", async () => {
// The stored diagram has a,b at known coords; add a new node c incrementally.
const { client, calls } = makeClient({
pageDoc: drawioPageDoc(),
attachmentSvg: svgFor(STORED),
});
const res = await client.drawioFromGraph(
"page1",
{ position: "append" },
{
nodes: [
{ id: "a", label: "A" },
{ id: "b", label: "B" },
{ id: "c", label: "C new" },
],
edges: [{ from: "b", to: "c" }],
},
undefined,
undefined,
"incremental",
"d1", // target the existing diagram
);
assert.equal(res.success, true);
const written = decodeDrawioSvg(calls.uploads[0].svg);
const cells = parseCells(written);
const a = cells.find((c) => c.id === "a");
const b = cells.find((c) => c.id === "b");
// Existing coords preserved (the stored a/b absolute coords from STORED).
assert.equal(a.geometry.x, 10);
assert.equal(a.geometry.y, 10);
assert.equal(b.geometry.x, 10);
assert.equal(b.geometry.y, 90);
assert.ok(cells.some((c) => c.id === "c"), "new node c added");
});
// --- drawioFromMermaid ------------------------------------------------------
test("drawioFromMermaid: converts a flowchart and inserts a diagram", async () => {
const pageDoc = { type: "doc", content: [{ type: "paragraph", attrs: { id: "p1" }, content: [] }] };
const { client, calls } = makeClient({ pageDoc });
const res = await client.drawioFromMermaid(
"page1",
{ position: "append" },
"flowchart LR\n A[Start] --> B{Choose}\n B -->|yes| C[Done]\n B -->|no| D[Stop]",
);
assert.equal(res.success, true);
const written = decodeDrawioSvg(calls.uploads[0].svg);
const cells = parseCells(written);
for (const id of ["A", "B", "C", "D"]) {
assert.ok(cells.some((c) => c.id === id), `node ${id} present`);
}
});
test("drawioFromMermaid: a non-flowchart is rejected, no upload", async () => {
const { client, calls } = makeClient({ pageDoc: { type: "doc", content: [] } });
await assert.rejects(
() => client.drawioFromMermaid("page1", { position: "append" }, "sequenceDiagram\n A->>B: x"),
/only 'flowchart'\/'graph' is supported/,
);
assert.equal(calls.uploads.length, 0);
});
+21 -21
View File
@@ -1,4 +1,4 @@
// Contract tests for the drawio_get / drawio_create / drawio_update client
// Contract tests for the drawioGet / drawioCreate / drawioUpdate client
// methods (issue #423). Follows the repo's seam-override pattern (see
// full-doc-write-canonicalize.test.mjs): a DocmostClient subclass stubs the I/O
// seams (auth, collab token, page read, attachment upload/fetch, the mutatePage
@@ -114,9 +114,9 @@ function findDrawio(node, acc = []) {
return acc;
}
// --- drawio_create ---------------------------------------------------------
// --- drawioCreate ---------------------------------------------------------
test("drawio_create: lints, builds the .drawio.svg, uploads and inserts a node", async () => {
test("drawioCreate: lints, builds the .drawio.svg, uploads and inserts a node", async () => {
const pageDoc = {
type: "doc",
content: [{ type: "paragraph", attrs: { id: "p1" }, content: [] }],
@@ -148,7 +148,7 @@ test("drawio_create: lints, builds the .drawio.svg, uploads and inserts a node",
assert.equal(n.attrs.title, "My diagram");
});
test("drawio_create: a lint violation throws before any upload", async () => {
test("drawioCreate: a lint violation throws before any upload", async () => {
const { client, calls } = makeClient({ pageDoc: { type: "doc", content: [] } });
// Edge with no child geometry -> edge-geometry rule.
const bad =
@@ -162,7 +162,7 @@ test("drawio_create: a lint violation throws before any upload", async () => {
assert.equal(calls.uploads.length, 0, "no attachment uploaded on lint failure");
});
test("drawio_create: before/after requires exactly one anchor", async () => {
test("drawioCreate: before/after requires exactly one anchor", async () => {
const { client } = makeClient({ pageDoc: { type: "doc", content: [] } });
await assert.rejects(
() => client.drawioCreate("page1", { position: "before" }, MODEL),
@@ -170,9 +170,9 @@ test("drawio_create: before/after requires exactly one anchor", async () => {
);
});
// --- drawio_get ------------------------------------------------------------
// --- drawioGet ------------------------------------------------------------
test("drawio_get: decodes the model and returns meta with a hash", async () => {
test("drawioGet: decodes the model and returns meta with a hash", async () => {
const pageDoc = {
type: "doc",
content: [
@@ -198,7 +198,7 @@ test("drawio_get: decodes the model and returns meta with a hash", async () => {
assert.equal(res.meta.hash, mxHash(normalizeXml(MODEL)));
});
test("drawio_get: format=svg returns the raw .drawio.svg", async () => {
test("drawioGet: format=svg returns the raw .drawio.svg", async () => {
const svg = svgFor(MODEL);
const pageDoc = {
type: "doc",
@@ -211,7 +211,7 @@ test("drawio_get: format=svg returns the raw .drawio.svg", async () => {
assert.equal(res.content, svg);
});
test("drawio_get: reads a HUMAN-saved compressed diagram losslessly (pako)", async () => {
test("drawioGet: reads a HUMAN-saved compressed diagram losslessly (pako)", async () => {
const pageDoc = {
type: "doc",
content: [
@@ -223,7 +223,7 @@ test("drawio_get: reads a HUMAN-saved compressed diagram losslessly (pako)", asy
assert.equal(res.content, normalizeXml(MODEL));
});
// --- drawio_update ---------------------------------------------------------
// --- drawioUpdate ---------------------------------------------------------
const UPDATED_MODEL =
'<mxGraphModel><root>' +
@@ -250,7 +250,7 @@ function updatePageDoc() {
};
}
test("drawio_update: stale baseHash -> conflict, no upload", async () => {
test("drawioUpdate: stale baseHash -> conflict, no upload", async () => {
const { client, calls } = makeClient({
pageDoc: updatePageDoc(),
attachmentSvg: svgFor(MODEL),
@@ -262,7 +262,7 @@ test("drawio_update: stale baseHash -> conflict, no upload", async () => {
assert.equal(calls.uploads.length, 0, "no upload on conflict");
});
test("drawio_update: current baseHash -> uploads new attachment and repoints node dims", async () => {
test("drawioUpdate: current baseHash -> uploads new attachment and repoints node dims", async () => {
const currentHash = mxHash(normalizeXml(MODEL));
const { client, calls } = makeClient({
pageDoc: updatePageDoc(),
@@ -285,7 +285,7 @@ test("drawio_update: current baseHash -> uploads new attachment and repoints nod
assert.equal(n.attrs.id, undefined);
});
test("drawio_update: baseHash is mandatory", async () => {
test("drawioUpdate: baseHash is mandatory", async () => {
const { client } = makeClient({ pageDoc: updatePageDoc(), attachmentSvg: svgFor(MODEL) });
await assert.rejects(
() => client.drawioUpdate("page1", "d1", UPDATED_MODEL, ""),
@@ -295,7 +295,7 @@ test("drawio_update: baseHash is mandatory", async () => {
// --- Fix 1: the create handle must resolve on the SAVED doc (no id) ---------
test("drawio_create -> get/update: returned #<index> handle resolves on the saved doc (id dropped)", async () => {
test("drawioCreate -> get/update: returned #<index> handle resolves on the saved doc (id dropped)", async () => {
// Create appends a drawio node after the existing paragraph.
const createDoc = {
type: "doc",
@@ -316,13 +316,13 @@ test("drawio_create -> get/update: returned #<index> handle resolves on the save
const savedDoc = create.calls.mutations[0].doc;
assert.equal(findDrawio(savedDoc)[0].attrs.id, undefined);
// drawio_get with the returned handle resolves the just-created node.
// drawioGet with the returned handle resolves the just-created node.
const getClient = makeClient({ pageDoc: savedDoc, attachmentSvg: svgFor(MODEL) });
const got = await getClient.client.drawioGet("page1", res.nodeId, "xml");
assert.equal(got.nodeId, res.nodeId);
assert.equal(got.content, normalizeXml(MODEL));
// drawio_update with the same handle + the hash from get repoints that node.
// drawioUpdate with the same handle + the hash from get repoints that node.
const upClient = makeClient({ pageDoc: savedDoc, attachmentSvg: svgFor(MODEL) });
const upd = await upClient.client.drawioUpdate(
"page1",
@@ -342,7 +342,7 @@ test("drawio_create -> get/update: returned #<index> handle resolves on the save
// --- error paths: the LLM must get a clean error, not a crash --------------
test("drawio_get: a bad node ref -> clean 'no node found' error", async () => {
test("drawioGet: a bad node ref -> clean 'no node found' error", async () => {
// Page has one paragraph; the requested ref resolves to nothing.
const pageDoc = {
type: "doc",
@@ -355,7 +355,7 @@ test("drawio_get: a bad node ref -> clean 'no node found' error", async () => {
);
});
test("drawio_get: a drawio node with no src -> clean 'has no src to read' error", async () => {
test("drawioGet: a drawio node with no src -> clean 'has no src to read' error", async () => {
const pageDoc = {
type: "doc",
content: [
@@ -370,7 +370,7 @@ test("drawio_get: a drawio node with no src -> clean 'has no src to read' error"
);
});
test("drawio_update: the resolved node is NOT a drawio node -> clean error, no upload", async () => {
test("drawioUpdate: the resolved node is NOT a drawio node -> clean error, no upload", async () => {
// "#0" resolves to a paragraph. The update must refuse cleanly rather than
// crash or repoint the wrong node.
const pageDoc = {
@@ -386,7 +386,7 @@ test("drawio_update: the resolved node is NOT a drawio node -> clean error, no u
assert.equal(calls.mutations.length, 0, "no write when the node is not a diagram");
});
test("drawio_create: anchor not found -> clean error that reports the orphan attachment", async () => {
test("drawioCreate: anchor not found -> clean error that reports the orphan attachment", async () => {
// The upload happens before the mutate transform; when the anchor cannot be
// found the write is skipped and the (now unreferenced) attachment is named
// in the error, exactly as the code documents.
@@ -416,7 +416,7 @@ test("drawio_create: anchor not found -> clean error that reports the orphan att
// --- Fix 2: update targets ONLY the resolved node --------------------------
test("drawio_update: repoints ONLY the addressed node, not siblings sharing an attachmentId", async () => {
test("drawioUpdate: repoints ONLY the addressed node, not siblings sharing an attachmentId", async () => {
// A copied diagram: two drawio nodes share one attachmentId. Updating via the
// "#0" handle must touch node #0 only, never the sibling copy.
const shared = {
@@ -2,7 +2,7 @@
// (issue #228):
// - insertFootnote (#11): the required-argument guards reject BEFORE any write,
// and never touch the collab/mutate path.
// - transformPage / docmost_transform (#13): the auto-canonicalize step
// - transformPage / docmostTransform (#13): the auto-canonicalize step
// (`result = canonicalizeFootnotes(raw)`) runs after every transform, so a
// transform that introduces an orphan footnote definition is silently tidied
// away — observable as an EMPTY diff in a dryRun preview.
@@ -10,7 +10,7 @@
// These stand a local http.createServer in for Docmost and only exercise plain
// HTTP routes (login / comments / pages.info), deliberately avoiding the live
// Hocuspocus collab WebSocket: the insertFootnote guards short-circuit before it,
// and docmost_transform's dryRun preview never opens it. The collab mutate path
// and docmostTransform's dryRun preview never opens it. The collab mutate path
// itself — abort-via-throw on a missing anchor with NO persisted write, and the
// reused-vs-new response shaping — is covered in
// test/mock/insert-footnote-wrapper.test.mjs (which overrides the mutatePage
@@ -101,7 +101,7 @@ test("insertFootnote rejects an empty text before any write", async () => {
});
// ---------------------------------------------------------------------------
// #13 docmost_transform auto-canonicalization: a transform that adds an orphan
// #13 docmostTransform auto-canonicalization: a transform that adds an orphan
// footnote definition produces NO net change (the canonicalizer drops it), so a
// dryRun preview reports an empty diff. Without the auto-canonicalize step the
// orphan would survive and the diff would be non-empty.
@@ -1,9 +1,9 @@
// Footnote-canonicalization binding tests for the MCP FULL-document write tools
// (issue #228, review #4): update_page_json and copy_page_content must persist a
// (issue #228, review #4): updatePageJson and copyPageContent must persist a
// footnote-canonical doc. These override the `replacePage` seam (symmetric to the
// `mutatePage` seam used by the insert-footnote-wrapper test) to capture the
// persisted doc WITHOUT a live Hocuspocus collab socket. Symmetric to the
// server-side focus specs for createPage / updatePageContent('replace').
// server-side focus specs for createPage / updatePage (markdown 'replace').
import { test } from "node:test";
import assert from "node:assert/strict";
import { DocmostClient } from "../../build/client.js";
@@ -44,7 +44,7 @@ function makeClient(sourceDoc) {
return { client, calls };
}
test("update_page_json canonicalizes the persisted full doc (out-of-order -> reference order)", async () => {
test("updatePageJson canonicalizes the persisted full doc (out-of-order -> reference order)", async () => {
const { client, calls } = makeClient();
const outOfOrder = {
type: "doc",
@@ -60,7 +60,7 @@ test("update_page_json canonicalizes the persisted full doc (out-of-order -> ref
assert.equal(findAll(calls.replaced[0].doc, "footnotesList").length, 1);
});
test("copy_page_content canonicalizes the persisted copy (orphan definition dropped)", async () => {
test("copyPageContent canonicalizes the persisted copy (orphan definition dropped)", async () => {
const sourceDoc = {
type: "doc",
content: [
@@ -0,0 +1,157 @@
// #413: getNode's markdown-default format, its JSON opt-in, the non-top-level
// AUTO fallback to JSON, and comment-anchor preservation (incl. resolved) on the
// markdown read. getNode only reads (getPageRaw), so a lightweight subclass that
// stubs auth + the page fetch is enough — no collab socket needed.
import { test } from "node:test";
import assert from "node:assert/strict";
import { DocmostClient } from "../../build/client.js";
function makeClient(doc) {
class TestClient extends DocmostClient {
async ensureAuthenticated() {}
async getPageRaw(pageId) {
return { id: pageId, slugId: "s", title: "P", spaceId: "sp", content: doc };
}
}
return new TestClient("http://127.0.0.1:1/api", "e@x.com", "pw");
}
const P = "p1";
test("getNode defaults to markdown for a paragraph", async () => {
const doc = {
type: "doc",
content: [
{
type: "paragraph",
attrs: { id: "b1" },
content: [{ type: "text", text: "hello world" }],
},
],
};
const res = await makeClient(doc).getNode(P, "b1");
assert.equal(res.format, "markdown");
assert.equal(typeof res.markdown, "string");
assert.match(res.markdown, /hello world/);
assert.equal(res.node, undefined, "markdown result carries no raw node");
});
test("getNode format:'json' returns the raw subtree verbatim", async () => {
const target = {
type: "paragraph",
attrs: { id: "b1" },
content: [{ type: "text", text: "hello" }],
};
const doc = { type: "doc", content: [target] };
const res = await makeClient(doc).getNode(P, "b1", "json");
assert.equal(res.format, "json");
assert.deepEqual(res.node, target);
assert.equal(res.markdown, undefined);
});
test("getNode AUTO-falls back to JSON for a non-top-level type (tableRow via #index)", async () => {
const doc = {
type: "doc",
content: [
{
type: "table",
content: [
{
type: "tableRow",
content: [
{
type: "tableCell",
attrs: { colspan: 1, rowspan: 1 },
content: [
{
type: "paragraph",
attrs: { id: "cp" },
content: [{ type: "text", text: "x" }],
},
],
},
],
},
],
},
],
};
// "#0.0"-style refs are not supported; the whole table is "#0", a row is only
// reachable by drilling — but a tableRow IS a non-doc-child type. Address the
// table itself as "#0": a table CAN be a doc child, so markdown is fine there.
// To hit the fallback, address the row by walking: getNode resolves "#0" to the
// table (doc child -> markdown). Instead we verify the schema gate directly by
// asking for the table (markdown) and a row is exercised via the unit test on
// canBeDocChild; here confirm a table renders as markdown.
const tableRes = await makeClient(doc).getNode(P, "#0");
assert.equal(tableRes.format, "markdown", "a table is a doc child -> markdown");
// Now build a doc whose top-level block IS a tableRow (schematically invalid but
// exercises the getNode fallback branch): getNode("#0") resolves it and, because
// tableRow cannot be a doc child, must fall back to JSON.
const rowDoc = {
type: "doc",
content: [
{
type: "tableRow",
content: [
{
type: "tableCell",
attrs: { colspan: 1, rowspan: 1 },
content: [{ type: "paragraph", content: [{ type: "text", text: "y" }] }],
},
],
},
],
};
const rowRes = await makeClient(rowDoc).getNode(P, "#0");
assert.equal(rowRes.format, "json", "a tableRow cannot be a doc child -> JSON fallback");
assert.equal(rowRes.type, "tableRow");
assert.ok(rowRes.node, "the JSON fallback returns the raw subtree");
});
test("getNode(markdown) PRESERVES comment anchors — active and resolved", async () => {
// A paragraph with two comment marks: one active, one resolved. get_page strips
// resolved anchors; getNode must NOT (a read for editing/write-back).
const doc = {
type: "doc",
content: [
{
type: "paragraph",
attrs: { id: "b1" },
content: [
{ type: "text", text: "start " },
{
type: "text",
text: "active",
marks: [{ type: "comment", attrs: { commentId: "cid-active" } }],
},
{ type: "text", text: " mid " },
{
type: "text",
text: "resolved",
marks: [
{
type: "comment",
attrs: { commentId: "cid-resolved", resolved: true },
},
],
},
{ type: "text", text: " end" },
],
},
],
};
const res = await makeClient(doc).getNode(P, "b1");
assert.equal(res.format, "markdown");
assert.match(
res.markdown,
/data-comment-id="cid-active"/,
"the active comment anchor is preserved",
);
assert.match(
res.markdown,
/data-comment-id="cid-resolved"/,
"the RESOLVED comment anchor is ALSO preserved (unlike get_page)",
);
});
@@ -0,0 +1,375 @@
// Mock-HTTP tests for DocmostClient.getPageContext — the #443 "where am I /
// what's around" read tool. A local http.createServer stands in for Docmost
// (same harness style as pagination-cursor.test.mjs) so everything is
// deterministic and offline.
//
// Contract pinned here:
// - Two requests: POST /pages/breadcrumbs (ancestor chain root->page, page
// INCLUDED as the LAST element) + listSidebarPages (direct children).
// - Split: last chain element -> `page`; the rest (root->parent) ->
// `breadcrumbs`. A ROOT page (chain length 1) -> breadcrumbs: [].
// - children: {pageId, title, hasChildren} in sidebar order.
// - INVARIANT: only the UUID `pageId` is exposed, never `slugId`.
// - A slugId input is resolved via /pages/info first (adds one request); a
// UUID input short-circuits (stays at two requests).
// - A bad/inaccessible pageId throws a CLEAR error, not an empty object.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { DocmostClient } from "../../build/client.js";
function readBody(req) {
return new Promise((resolve) => {
let raw = "";
req.on("data", (chunk) => {
raw += chunk;
});
req.on("end", () => resolve(raw));
});
}
function startServer(handler) {
return new Promise((resolve) => {
const server = http.createServer(handler);
server.listen(0, "127.0.0.1", () => {
const { port } = server.address();
resolve({ server, baseURL: `http://127.0.0.1:${port}/api` });
});
});
}
function closeServer(server) {
return new Promise((resolve) => server.close(resolve));
}
function sendJson(res, status, obj, extraHeaders = {}) {
res.writeHead(status, { "Content-Type": "application/json", ...extraHeaders });
res.end(JSON.stringify(obj));
}
const openServers = [];
async function spawn(handler) {
const { server, baseURL } = await startServer(handler);
openServers.push(server);
return { server, baseURL };
}
after(async () => {
await Promise.all(openServers.map((s) => closeServer(s)));
});
function handleLogin(req, res) {
if (req.url === "/api/auth/login") {
sendJson(res, 200, { success: true }, {
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
});
return true;
}
return false;
}
// Two real UUIDs so resolvePageId short-circuits (no /pages/info round-trip).
const ROOT_UUID = "00000000-0000-4000-8000-000000000001";
const MID_UUID = "00000000-0000-4000-8000-000000000002";
const PAGE_UUID = "00000000-0000-4000-8000-000000000003";
const CHILD_A = "00000000-0000-4000-8000-00000000000a";
const CHILD_B = "00000000-0000-4000-8000-00000000000b";
// Build a breadcrumbs response as the server sends it: root->page order, page
// LAST, wrapped in the {data,success} envelope. slugId/icon/position are present
// on the wire (they must NOT leak into the tool output).
function breadcrumbsEnvelope(chain) {
return { success: true, data: chain };
}
// -----------------------------------------------------------------------------
// 1) 3rd-level page: page = last chain element; breadcrumbs = the two ancestors
// root->parent; children mapped {pageId,title,hasChildren} in order; no leak;
// exactly two requests for a UUID input.
// -----------------------------------------------------------------------------
test("getPageContext: 3rd-level page splits chain, maps children, no slugId leak, 2 requests", async () => {
let breadcrumbReqs = 0;
let sidebarReqs = 0;
let infoReqs = 0;
let breadcrumbBody = null;
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/info") {
infoReqs++;
sendJson(res, 404, {});
return;
}
if (req.url === "/api/pages/breadcrumbs") {
breadcrumbReqs++;
breadcrumbBody = JSON.parse(raw || "{}");
// root -> parent -> page (page LAST). slugId/icon/position on the wire.
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", position: "a", icon: null, parentPageId: null, hasChildren: true },
{ id: MID_UUID, slugId: "midSlug", title: "Datacenter A", spaceId: "sp1", position: "a", icon: null, parentPageId: ROOT_UUID, hasChildren: true },
{ id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1", position: "b", icon: null, parentPageId: MID_UUID, hasChildren: true },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
sidebarReqs++;
const body = JSON.parse(raw || "{}");
assert.equal(body.pageId, PAGE_UUID, "children scoped to the page UUID");
assert.equal(body.spaceId, "sp1", "children scoped to the page's space");
sendJson(res, 200, {
success: true,
data: {
items: [
{ id: CHILD_A, slugId: "aSlug", title: "Servers", parentPageId: PAGE_UUID, hasChildren: true, position: "a" },
{ id: CHILD_B, slugId: "bSlug", title: "Network", parentPageId: PAGE_UUID, hasChildren: false, position: "b" },
],
meta: { hasNextPage: false, nextCursor: null },
},
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext(PAGE_UUID);
assert.equal(infoReqs, 0, "UUID input short-circuits resolvePageId (no /pages/info)");
assert.equal(breadcrumbReqs, 1, "exactly one breadcrumbs request");
assert.equal(sidebarReqs, 1, "exactly one sidebar request");
assert.deepEqual(breadcrumbBody, { pageId: PAGE_UUID }, "breadcrumbs posts the UUID");
// page = the LAST chain element.
assert.deepEqual(result.page, {
pageId: PAGE_UUID,
title: "Rack 12",
spaceId: "sp1",
});
// breadcrumbs = root->parent (the chain minus the page itself).
assert.deepEqual(result.breadcrumbs, [
{ pageId: ROOT_UUID, title: "Infrastructure" },
{ pageId: MID_UUID, title: "Datacenter A" },
]);
// children mapped in order, hasChildren coerced to boolean.
assert.deepEqual(result.children, [
{ pageId: CHILD_A, title: "Servers", hasChildren: true },
{ pageId: CHILD_B, title: "Network", hasChildren: false },
]);
// No slugId anywhere in the output.
const dump = JSON.stringify(result);
assert.ok(!dump.includes("Slug"), "no slugId leaks into the output");
assert.ok(!/\bslugId\b/.test(dump), "no slugId key in the output");
});
// -----------------------------------------------------------------------------
// 2) ROOT page: chain has ONE element (the page itself) -> breadcrumbs: [].
// -----------------------------------------------------------------------------
test("getPageContext: a root page has breadcrumbs: []", async () => {
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
// A root page: the CTE returns only the page itself.
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", parentPageId: null, hasChildren: false },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
sendJson(res, 200, {
success: true,
data: { items: [], meta: { hasNextPage: false, nextCursor: null } },
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext(ROOT_UUID);
assert.deepEqual(result.page, {
pageId: ROOT_UUID,
title: "Infrastructure",
spaceId: "sp1",
});
assert.deepEqual(result.breadcrumbs, [], "root page: no ancestors");
assert.deepEqual(result.children, [], "no children");
});
// -----------------------------------------------------------------------------
// 3) A slugId input is resolved via /pages/info first (one extra request), then
// breadcrumbs/sidebar use the resolved UUID.
// -----------------------------------------------------------------------------
test("getPageContext: a slugId input is resolved via /pages/info", async () => {
let infoReqs = 0;
let infoBody = null;
let breadcrumbBody = null;
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/info") {
infoReqs++;
infoBody = JSON.parse(raw || "{}");
// getPageRaw: slugId -> canonical UUID.
sendJson(res, 200, {
success: true,
data: { id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1" },
});
return;
}
if (req.url === "/api/pages/breadcrumbs") {
breadcrumbBody = JSON.parse(raw || "{}");
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", parentPageId: null },
{ id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1", parentPageId: ROOT_UUID },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
sendJson(res, 200, {
success: true,
data: { items: [], meta: { hasNextPage: false, nextCursor: null } },
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext("pageSlug");
assert.equal(infoReqs, 1, "slugId resolved via one /pages/info");
assert.deepEqual(infoBody, { pageId: "pageSlug" }, "resolve posts the raw slugId");
assert.deepEqual(
breadcrumbBody,
{ pageId: PAGE_UUID },
"breadcrumbs posts the RESOLVED uuid, not the slugId",
);
assert.equal(result.page.pageId, PAGE_UUID, "page.pageId is the UUID");
assert.deepEqual(result.breadcrumbs, [
{ pageId: ROOT_UUID, title: "Infrastructure" },
]);
});
// -----------------------------------------------------------------------------
// 4) >20 children: cursor pagination returns ALL of them, no dupes (regression
// on the #442 bug class — getPageContext must not re-introduce a cap).
// -----------------------------------------------------------------------------
test("getPageContext: a page with >20 children returns ALL of them (no cap, no dupes)", async () => {
// 45 children spread over three cursor pages.
const all = Array.from({ length: 45 }, (_, i) => ({
id: `child-${i}`,
slugId: `slug-${i}`,
title: `Child ${i}`,
parentPageId: PAGE_UUID,
hasChildren: i % 2 === 0,
}));
const PAGES = {
"": { items: all.slice(0, 20), nextCursor: "c1" },
c1: { items: all.slice(20, 40), nextCursor: "c2" },
c2: { items: all.slice(40), nextCursor: null },
};
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: PAGE_UUID, slugId: "pageSlug", title: "Big Parent", spaceId: "sp1", parentPageId: null },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
const body = JSON.parse(raw || "{}");
const page = PAGES[body.cursor ?? ""] ?? { items: [], nextCursor: null };
sendJson(res, 200, {
success: true,
data: {
items: page.items,
meta: { hasNextPage: page.nextCursor != null, nextCursor: page.nextCursor },
},
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext(PAGE_UUID);
assert.equal(result.children.length, 45, "all 45 children returned");
const ids = result.children.map((c) => c.pageId);
assert.equal(new Set(ids).size, 45, "no duplicate children");
assert.deepEqual(ids, all.map((c) => c.id), "children in server order across cursor pages");
assert.equal(result.children[0].hasChildren, true, "hasChildren preserved (child 0)");
assert.equal(result.children[1].hasChildren, false, "hasChildren preserved (child 1)");
});
// -----------------------------------------------------------------------------
// 5) A nonexistent / inaccessible pageId -> a CLEAR error, NOT an empty object.
// -----------------------------------------------------------------------------
test("getPageContext: a bad/inaccessible pageId throws a clear error (not {})", async () => {
const { baseURL } = await spawn(async (req, res) => {
await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
// Server rejects an unknown/forbidden page.
sendJson(res, 404, { message: "Page not found" });
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects(
() => client.getPageContext(PAGE_UUID),
(err) => {
assert.ok(err instanceof Error, "throws an Error");
return true;
},
"a 404 from breadcrumbs propagates as a thrown error, not a hollow {}",
);
});
// -----------------------------------------------------------------------------
// 6) An empty breadcrumbs chain (should never happen — the endpoint always
// includes the page itself) is treated as not-found, not a hollow {page:...}.
// -----------------------------------------------------------------------------
test("getPageContext: an empty breadcrumbs chain throws (defensive)", async () => {
const { baseURL } = await spawn(async (req, res) => {
await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
sendJson(res, 200, breadcrumbsEnvelope([]));
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects(
() => client.getPageContext(PAGE_UUID),
/not found or inaccessible/,
"an empty chain is a clear error, not {}",
);
});
@@ -0,0 +1,246 @@
// Mock regression for the FAIL-FAST invalid-node validation (#409).
//
// A structural editor (patchNode / insertNode / updatePageJson) given a doc
// whose NESTED child has an absent/unknown `type` (the exact shape the Yjs
// encoder rejects with `Unknown node type: undefined`) must throw a RICH,
// path-anchored error BEFORE it ever opens a collab session or takes a page
// lock. We prove the fail-fast by standing up a collab stack whose HTTP handler
// records EVERY request: a correct fail-fast never even fetches the collab
// token (which `getCollabTokenWithReauth`, called AFTER the validation, would
// request), and never drives a document change on the Hocuspocus doc.
//
// The happy path (a well-formed doc) is exercised too: it must reach the collab
// write and succeed, so the gate is not over-eager.
//
// findInvalidNode's per-shape summaries are unit-tested in the package
// (test/find-invalid-node.test.ts); this exercises the END-TO-END wiring through
// the real client methods.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { WebSocketServer } from "ws";
import { Hocuspocus } from "@hocuspocus/server";
import { DocmostClient } from "../../build/client.js";
import { buildYDoc } from "../../build/lib/collaboration.js";
// A minimal valid seed doc with a real block id, so the happy-path patchNode
// finds its target.
const SEED_ID = "seed-para-id";
function seedDoc() {
return {
type: "doc",
content: [
{
type: "paragraph",
attrs: { id: SEED_ID },
content: [{ type: "text", text: "seed" }],
},
],
};
}
// Stand up an HTTP server that authenticates + hands out a collab token AND
// upgrades /collab to a Hocuspocus instance seeded with the doc. `state` records
// whether the collab token was ever fetched (proving the write path was entered)
// and whether the Hocuspocus doc ever changed.
async function spawnCollabStack() {
const state = { changed: false, collabTokenFetched: false };
const hocuspocus = new Hocuspocus({
quiet: true,
async onLoadDocument() {
return buildYDoc(seedDoc());
},
async onChange() {
state.changed = true;
},
});
const wss = new WebSocketServer({ noServer: true });
const server = http.createServer((req, res) => {
let raw = "";
req.on("data", (c) => (raw += c));
req.on("end", () => {
if (req.url === "/api/auth/login") {
res.writeHead(200, {
"Content-Type": "application/json",
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
});
res.end(JSON.stringify({ success: true }));
return;
}
if (req.url === "/api/auth/collab-token") {
state.collabTokenFetched = true;
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ data: { token: "collab-jwt" } }));
return;
}
res.writeHead(404, { "Content-Type": "application/json" });
res.end(JSON.stringify({ message: "not found" }));
});
});
server.on("upgrade", (request, socket, head) => {
if (!request.url || !request.url.startsWith("/collab")) {
socket.destroy();
return;
}
wss.handleUpgrade(request, socket, head, (ws) => {
hocuspocus.handleConnection(ws, request);
});
});
const baseURL = await new Promise((resolve) => {
server.listen(0, "127.0.0.1", () => {
const { port } = server.address();
resolve(`http://127.0.0.1:${port}/api`);
});
});
openStacks.push({ server, hocuspocus });
return { state, baseURL };
}
const openStacks = [];
after(async () => {
await Promise.all(
openStacks.map(
({ server, hocuspocus }) =>
new Promise((resolve) => {
server.close(() => {
Promise.resolve(hocuspocus.destroy?.()).finally(resolve);
});
}),
),
);
});
const PAGE = "11111111-1111-4111-8111-111111111111";
// A node whose NESTED text leaf is missing "type":"text" (dominant #409 shape).
const nestedTypelessNode = () => ({
type: "paragraph",
content: [{ text: "oops", marks: [] }],
});
// A node with a NESTED unknown type NAME (typo).
const nestedUnknownTypeNode = () => ({
type: "paragraph",
content: [{ type: "paragraf", content: [{ type: "text", text: "x" }] }],
});
test("patchNode fails fast on a nested typeless node — no collab connection", async () => {
const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects(
() => client.patchNode(PAGE, SEED_ID, { node: nestedTypelessNode() }),
(err) => {
assert.match(err.message, /patchNode: invalid node/);
assert.match(err.message, /missing "type"/);
assert.match(err.message, /content\[0\]/); // path-anchored
return true;
},
);
assert.equal(
state.collabTokenFetched,
false,
"must NOT fetch a collab token — validation runs before getCollabTokenWithReauth",
);
assert.equal(state.changed, false, "the collab doc must never be written");
});
test("insertNode fails fast on a nested UNKNOWN type — no collab connection", async () => {
const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects(
() =>
client.insertNode(
PAGE,
{ node: nestedUnknownTypeNode() },
{
position: "append",
},
),
(err) => {
assert.match(err.message, /insertNode: invalid node/);
assert.match(err.message, /unknown node type "paragraf"/);
return true;
},
);
assert.equal(state.collabTokenFetched, false);
assert.equal(state.changed, false);
});
test("updatePageJson fails fast on a nested typeless node — no collab connection", async () => {
const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const badDoc = {
type: "doc",
content: [{ type: "paragraph", content: [{ text: "oops" }] }],
};
await assert.rejects(
() => client.updatePageJson(PAGE, badDoc),
(err) => {
// updatePageJson runs validateDocStructure first (string-type check),
// which already rejects a typeless node — so the message may come from
// either guard, but the write must not happen.
assert.match(err.message, /type/i);
return true;
},
);
assert.equal(state.collabTokenFetched, false);
assert.equal(state.changed, false);
});
test("updatePageJson fails fast on a nested UNKNOWN type name — rich #409 message", async () => {
const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw");
// validateDocStructure passes (type is a string); assertValidNodeShape must
// catch the unknown schema name and produce the rich path-anchored message.
const badDoc = {
type: "doc",
content: [{ type: "paragraf", content: [{ type: "text", text: "x" }] }],
};
await assert.rejects(
() => client.updatePageJson(PAGE, badDoc),
(err) => {
assert.match(err.message, /updatePageJson: invalid node/);
assert.match(err.message, /unknown node type "paragraf"/);
return true;
},
);
assert.equal(state.collabTokenFetched, false);
assert.equal(state.changed, false);
});
test("patchNode with a well-formed node proceeds to the collab write", async () => {
const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.patchNode(PAGE, SEED_ID, {
node: {
type: "paragraph",
content: [{ type: "text", text: "replacement" }],
},
});
assert.equal(result.success, true);
assert.equal(result.replaced, 1);
assert.equal(
state.collabTokenFetched,
true,
"a valid node must reach the collab write path",
);
assert.equal(state.changed, true, "the collab doc must be written");
});
@@ -0,0 +1,538 @@
// Mock collab tests for the #413 MARKDOWN path of patchNode / insertNode and the
// markdown-default getNode. These stand up a real Hocuspocus collab server seeded
// with a chosen document (mirroring ambiguous-node-id.test.mjs), let the client
// run its real transform against a live Y.Doc, and read the persisted result back
// to assert on the written document.
//
// Coverage (issue #413):
// - CANON CONVERGENCE: a block written via patchNode(markdown) is canonically
// equal to the SAME content run through a full markdown import (no "second
// canon" appears on the block-level path).
// - id-THREAD on a 1->N splice: the first block inherits the target id, the rest
// get fresh ids, and every NEIGHBOUR block is byte-identical before/after.
// - XOR validation (both / neither markdown+node -> error).
// - span/color-attr GUARD on the target block (a merged/colored cell refuses a
// markdown patch, nothing written).
// - `^[...]` footnote in the fragment -> a definition in the tail list + renumber.
// - insertNode(markdown) inserts N blocks in order at the anchor.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { WebSocketServer } from "ws";
import { Hocuspocus } from "@hocuspocus/server";
import { DocmostClient } from "../../build/client.js";
import { buildYDoc } from "../../build/lib/collaboration.js";
import {
docsCanonicallyEqual,
markdownToProseMirror,
} from "@docmost/prosemirror-markdown";
const PAGE = "11111111-1111-4111-8111-111111111111";
// Deep JSON clone for byte-identity assertions.
const jclone = (v) => JSON.parse(JSON.stringify(v));
function findAll(node, type, acc = []) {
if (!node || typeof node !== "object") return acc;
if (node.type === type) acc.push(node);
if (Array.isArray(node.content))
for (const c of node.content) findAll(c, type, acc);
return acc;
}
// Stand up an HTTP+Hocuspocus stack seeded with `seedDoc`. `state.lastDoc` holds
// the most recently persisted document JSON (decoded from the live Y.Doc on every
// change) so a test can inspect exactly what was written.
async function spawnCollabStack(seedDoc) {
const state = { changed: false, lastDoc: null };
const hocuspocus = new Hocuspocus({
quiet: true,
async onLoadDocument() {
return buildYDoc(seedDoc);
},
async onChange(data) {
state.changed = true;
try {
const frag = data.document.getXmlFragment("default");
// Decode the live fragment back to JSON via the same helper the client
// reads with — but simpler: use the yjs->json path exposed by the doc.
state.lastDoc = fragmentToJson(frag);
} catch {
/* ignore decode errors in teardown races */
}
},
});
const wss = new WebSocketServer({ noServer: true });
const server = http.createServer((req, res) => {
let raw = "";
req.on("data", (c) => (raw += c));
req.on("end", () => {
if (req.url === "/api/auth/login") {
res.writeHead(200, {
"Content-Type": "application/json",
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
});
res.end(JSON.stringify({ success: true }));
return;
}
if (req.url === "/api/auth/collab-token") {
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ data: { token: "collab-jwt" } }));
return;
}
res.writeHead(404, { "Content-Type": "application/json" });
res.end(JSON.stringify({ message: "not found" }));
});
});
server.on("upgrade", (request, socket, head) => {
if (!request.url || !request.url.startsWith("/collab")) {
socket.destroy();
return;
}
wss.handleUpgrade(request, socket, head, (ws) => {
hocuspocus.handleConnection(ws, request);
});
});
const baseURL = await new Promise((resolve) => {
server.listen(0, "127.0.0.1", () => {
const { port } = server.address();
resolve(`http://127.0.0.1:${port}/api`);
});
});
openStacks.push({ server, hocuspocus });
return { state, baseURL };
}
// Minimal XmlFragment -> ProseMirror JSON decode, mirroring the shape Docmost
// stores. Reads element name as node type, attributes as attrs, and recurses into
// children; text nodes carry their string.
function fragmentToJson(frag) {
const decodeNode = (el) => {
if (el.constructor.name === "YXmlText") {
// A yjs text node: collect the string with its formatting deltas.
const delta = el.toDelta();
return delta.map((d) => {
const node = { type: "text", text: d.insert };
if (d.attributes && Object.keys(d.attributes).length) {
node.marks = Object.entries(d.attributes).map(([type, attrs]) =>
attrs && typeof attrs === "object" && Object.keys(attrs).length
? { type, attrs }
: { type },
);
}
return node;
});
}
const node = { type: el.nodeName };
const attrs = el.getAttributes();
if (attrs && Object.keys(attrs).length) node.attrs = attrs;
const children = [];
for (const child of el.toArray()) {
const decoded = decodeNode(child);
if (Array.isArray(decoded)) children.push(...decoded);
else children.push(decoded);
}
if (children.length) node.content = children;
return node;
};
const content = [];
for (const child of frag.toArray()) content.push(decodeNode(child));
return { type: "doc", content };
}
const openStacks = [];
after(async () => {
await Promise.all(
openStacks.map(
({ server, hocuspocus }) =>
new Promise((resolve) => {
server.close(() => {
Promise.resolve(hocuspocus.destroy?.()).finally(resolve);
});
}),
),
);
});
// A seed doc with two neighbour paragraphs around a target paragraph.
function seed3() {
return {
type: "doc",
content: [
{
type: "paragraph",
attrs: { id: "before-id" },
content: [{ type: "text", text: "before" }],
},
{
type: "paragraph",
attrs: { id: "target-id" },
content: [{ type: "text", text: "old target" }],
},
{
type: "paragraph",
attrs: { id: "after-id" },
content: [{ type: "text", text: "after" }],
},
],
};
}
test("patchNode(markdown): XOR — both markdown and node is rejected, nothing written", async () => {
const { state, baseURL } = await spawnCollabStack(seed3());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await assert.rejects(
() =>
client.patchNode(PAGE, "target-id", {
markdown: "hello",
node: { type: "paragraph" },
}),
/exactly one of/i,
);
assert.equal(state.changed, false, "no write on an XOR violation");
});
test("patchNode(markdown): XOR — neither markdown nor node is rejected", async () => {
const { baseURL } = await spawnCollabStack(seed3());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await assert.rejects(
() => client.patchNode(PAGE, "target-id", {}),
/exactly one of/i,
);
});
test("patchNode(markdown): single block keeps the id; neighbours byte-identical", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
const res = await client.patchNode(PAGE, "target-id", {
markdown: "the **new** target",
});
assert.equal(res.success, true);
assert.equal(res.replaced, 1);
assert.equal(res.blocks, 1);
const doc = state.lastDoc;
const paras = doc.content;
// The rewritten block still carries the target id.
const target = paras.find((p) => p.attrs?.id === "target-id");
assert.ok(target, "rewritten block inherits target-id");
assert.equal(target.content.some((n) => n.text === "new"), true);
// Neighbours are byte-identical to the seed.
const beforeNode = paras.find((p) => p.attrs?.id === "before-id");
const afterNode = paras.find((p) => p.attrs?.id === "after-id");
assert.deepEqual(beforeNode, before.content[0]);
assert.deepEqual(afterNode, before.content[2]);
});
test("patchNode(markdown): 1->N splice threads the id onto the first block; neighbours byte-identical", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
// Two paragraphs of markdown -> a 2-block fragment replacing one block.
const res = await client.patchNode(PAGE, "target-id", {
markdown: "first para\n\nsecond para",
});
assert.equal(res.blocks, 2);
const doc = state.lastDoc;
const idx = doc.content.findIndex((p) => p.attrs?.id === "target-id");
assert.ok(idx >= 0, "first spliced block inherits target-id");
const first = doc.content[idx];
const second = doc.content[idx + 1];
assert.equal(first.content.some((n) => n.text === "first para"), true);
assert.equal(second.content.some((n) => n.text === "second para"), true);
// The second block has a DIFFERENT (fresh) id.
assert.notEqual(second.attrs?.id, "target-id");
assert.ok(second.attrs?.id, "the extra block gets a fresh id");
// Neighbours untouched, byte-identical.
assert.deepEqual(
doc.content.find((p) => p.attrs?.id === "before-id"),
before.content[0],
);
assert.deepEqual(
doc.content.find((p) => p.attrs?.id === "after-id"),
before.content[2],
);
});
test("patchNode(markdown): CANON CONVERGENCE — block equals the same content full-imported", async () => {
const { state, baseURL } = await spawnCollabStack(seed3());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
const md = "a paragraph with **bold**, _italic_ and `code`";
await client.patchNode(PAGE, "target-id", { markdown: md });
// The block as persisted.
const target = state.lastDoc.content.find((p) => p.attrs?.id === "target-id");
// The same markdown run through the full-page importer.
const full = await markdownToProseMirror(md);
const fullBlock = full.content[0];
assert.ok(
docsCanonicallyEqual(
{ type: "doc", content: [target] },
{ type: "doc", content: [fullBlock] },
),
"a patchNode(markdown) block must be canonically equal to a full import — no second canon",
);
});
test("patchNode(markdown): a paragraph inside a merged (colspan) cell rewrites fine — the cell's span is preserved", async () => {
// A cell paragraph carries an id and IS id-targetable; rewriting ITS content
// from markdown replaces only the paragraph, so the cell's colspan is NOT lost
// (the span lives on the cell, which patchNode leaves in place). This is the
// correct behavior: no false guard, no loss. The guard's REJECTION logic (when
// the replaced block itself carries/contains an unrepresentable span) is proven
// by the findUnrepresentableTableAttrs unit test — that case is not reachable
// through the id-targeting API because tables/cells carry no addressable id.
const doc = {
type: "doc",
content: [
{
type: "table",
content: [
{
type: "tableRow",
content: [
{
type: "tableCell",
attrs: { colspan: 2, rowspan: 1 },
content: [
{
type: "paragraph",
attrs: { id: "cell-para" },
content: [{ type: "text", text: "merged" }],
},
],
},
],
},
],
},
],
};
const { state, baseURL } = await spawnCollabStack(doc);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
const res = await client.patchNode(PAGE, "cell-para", { markdown: "rewritten" });
assert.equal(res.success, true);
// The cell's colspan survives (the span is on the cell, not the paragraph).
const cell = findAll(state.lastDoc, "tableCell")[0];
assert.equal(cell.attrs.colspan, 2, "the cell's colspan is preserved");
const para = findAll(cell, "paragraph")[0];
assert.equal(
(para.content || []).some((n) => n.text === "rewritten"),
true,
);
});
test("patchNode(markdown): a `^[...]` footnote in the fragment lands in the tail list", async () => {
const { state, baseURL } = await spawnCollabStack(seed3());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await client.patchNode(PAGE, "target-id", {
markdown: "a claim^[the supporting note]",
});
const doc = state.lastDoc;
const lists = findAll(doc, "footnotesList");
assert.equal(lists.length, 1, "exactly one tail footnotesList");
const defs = findAll(doc, "footnoteDefinition");
assert.equal(defs.length, 1, "one definition for the fragment footnote");
const refs = findAll(doc, "footnoteReference");
assert.equal(refs.length, 1, "one reference in the body");
// Reference and definition share an id (renumbered canonically).
assert.equal(refs[0].attrs.id, defs[0].attrs.id);
});
test("insertNode(markdown): inserts N blocks in order after the anchor", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
const res = await client.insertNode(
PAGE,
{ markdown: "new one\n\nnew two" },
{ position: "after", anchorNodeId: "before-id" },
);
assert.equal(res.success, true);
assert.equal(res.blocks, 2);
const texts = state.lastDoc.content.map((p) => (p.content || []).map((n) => n.text).join(""));
// Order: before, new one, new two, target, after.
assert.deepEqual(texts, ["before", "new one", "new two", "old target", "after"]);
});
test("insertNode(markdown): XOR — both markdown and node is rejected", async () => {
const { baseURL } = await spawnCollabStack(seed3());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await assert.rejects(
() =>
client.insertNode(
PAGE,
{ markdown: "x", node: { type: "paragraph" } },
{ position: "append" },
),
/exactly one of/i,
);
});
// A seed page whose ONLY footnote reference lives in the target paragraph p1,
// with a matching definition in a trailing footnotesList. Rewriting p1 with a
// footnote-free fragment removes the last referrer -> the definition is orphaned.
function seedOrphanFootnote() {
return {
type: "doc",
content: [
{
type: "paragraph",
attrs: { id: "p1" },
content: [
{ type: "text", text: "a claim" },
{ type: "footnoteReference", attrs: { id: "fn-1", referenceNumber: 1 } },
],
},
{
type: "footnotesList",
content: [
{
type: "footnoteDefinition",
attrs: { id: "fn-1" },
content: [
{
type: "paragraph",
attrs: { id: "def-para" },
content: [{ type: "text", text: "the supporting note" }],
},
],
},
],
},
],
};
}
test("patchNode(markdown): removing the LAST footnote referrer drops the now-orphan definition (canonical convergence)", async () => {
const { state, baseURL } = await spawnCollabStack(seedOrphanFootnote());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
// The fragment has NO footnotes -> definitions=[]; the splice removes the only
// footnoteReference, leaving the tail definition orphaned. The canonicalization
// pass (which mergeFootnoteDefinitions must still run) has to drop it.
await client.patchNode(PAGE, "p1", { markdown: "just text" });
const doc = state.lastDoc;
assert.equal(
findAll(doc, "footnoteDefinition").length,
0,
"the orphaned definition is dropped",
);
assert.equal(
findAll(doc, "footnotesList").length,
0,
"the emptied footnotesList is removed",
);
assert.equal(findAll(doc, "footnoteReference").length, 0, "no references remain");
// Convergence: the persisted result equals the SAME content imported whole.
const full = await markdownToProseMirror("just text");
const target = doc.content.find((p) => p.attrs?.id === "p1");
assert.ok(
docsCanonicallyEqual(
{ type: "doc", content: [target] },
{ type: "doc", content: [full.content[0]] },
),
"the post-splice doc is canonically identical to a full re-import",
);
});
test("patchNode(markdown): a pure-text patch on a footnote-FREE page leaves footnote topology untouched (fast path)", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await client.patchNode(PAGE, "target-id", { markdown: "plain replacement" });
const doc = state.lastDoc;
assert.equal(findAll(doc, "footnotesList").length, 0, "no footnotesList appears");
assert.equal(findAll(doc, "footnoteDefinition").length, 0, "no definition appears");
assert.equal(findAll(doc, "footnoteReference").length, 0, "no reference appears");
// Neighbours byte-identical (the fast path does not clone/reshape the tree).
assert.deepEqual(
doc.content.find((p) => p.attrs?.id === "before-id"),
before.content[0],
);
assert.deepEqual(
doc.content.find((p) => p.attrs?.id === "after-id"),
before.content[2],
);
});
test("insertNode(markdown): a footnote-free insert on a page carrying a footnote still canonicalizes (definitions empty)", async () => {
// The page has an existing footnote (ref + tail def). Inserting a footnote-free
// fragment keeps the reference alive, so the definition stays — but the write
// path must still run canonicalization (definitions=[]), producing exactly one
// tail list with the reference/definition ids in sync.
const { state, baseURL } = await spawnCollabStack(seedOrphanFootnote());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
const res = await client.insertNode(
PAGE,
{ markdown: "unrelated one\n\nunrelated two" },
{ position: "after", anchorNodeId: "p1" },
);
assert.equal(res.success, true);
const doc = state.lastDoc;
assert.equal(findAll(doc, "footnoteReference").length, 1, "the existing reference survives");
assert.equal(findAll(doc, "footnotesList").length, 1, "exactly one tail list");
const defs = findAll(doc, "footnoteDefinition");
assert.equal(defs.length, 1, "the definition is kept (still referenced)");
assert.equal(findAll(doc, "footnoteReference")[0].attrs.id, defs[0].attrs.id);
});
// Collect every TOP-LEVEL block id in a doc (the invariant the splice dedup
// guarantees is page-wide top-level uniqueness).
function topLevelIds(doc) {
return doc.content
.map((b) => b?.attrs?.id)
.filter((id) => id != null);
}
test("patchNode(markdown): a 1->N splice yields page-wide UNIQUE top-level block ids", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await client.patchNode(PAGE, "target-id", {
markdown: "one\n\ntwo\n\nthree",
});
const ids = topLevelIds(state.lastDoc);
assert.equal(new Set(ids).size, ids.length, "all top-level block ids are unique");
// The target id is still present (threaded onto the first block).
assert.ok(ids.includes("target-id"), "the first block still inherits target-id");
});
test("insertNode(markdown): inserting multiple blocks yields page-wide UNIQUE top-level block ids", async () => {
const before = seed3();
const { state, baseURL } = await spawnCollabStack(before);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await client.insertNode(
PAGE,
{ markdown: "alpha\n\nbeta\n\ngamma" },
{ position: "after", anchorNodeId: "before-id" },
);
const ids = topLevelIds(state.lastDoc);
assert.equal(new Set(ids).size, ids.length, "all top-level block ids are unique");
});
@@ -155,7 +155,7 @@ test("listSidebarPages terminates (no dups) when the server ignores the cursor",
// -----------------------------------------------------------------------------
// 3a) enumerateSpacePages happy path: a SINGLE /pages/tree request.
// -----------------------------------------------------------------------------
test("enumerateSpacePages (via list_pages tree) uses one /pages/tree request", async () => {
test("enumerateSpacePages (via listPages tree) uses one /pages/tree request", async () => {
let treeRequests = 0;
let sidebarRequests = 0;
let treeBody = null;
@@ -184,7 +184,7 @@ test("enumerateSpacePages (via list_pages tree) uses one /pages/tree request", a
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
// list_pages tree:true -> enumerateSpacePages(spaceId) -> buildPageTree.
// listPages tree:true -> enumerateSpacePages(spaceId) -> buildPageTree.
const tree = await client.listPages("space-1", 50, true);
assert.equal(treeRequests, 1, "exactly one /pages/tree request for the space");
@@ -375,7 +375,7 @@ test("listComments terminates (no dups) when the server ignores the cursor", asy
});
// -----------------------------------------------------------------------------
// 4) check_new_comments subtree: the root is included in scope WITHOUT a
// 4) checkNewComments subtree: the root is included in scope WITHOUT a
// separate getPageRaw (/pages/info) request for the parent.
// -----------------------------------------------------------------------------
test("checkNewComments subtree includes the root without a separate getPageRaw", async () => {
@@ -244,7 +244,7 @@ test("tableInsertRow with a slugId opens the collab doc by the resolved UUID (#2
);
});
test("the generic mutate (insert_footnote) with a slugId opens by the resolved UUID (#260)", async () => {
test("the generic mutate (insertFootnote) with a slugId opens by the resolved UUID (#260)", async () => {
const { state, baseURL } = await spawnCollabStack();
const client = new DocmostClient(baseURL, "user@example.com", "pw");
@@ -254,7 +254,7 @@ test("the generic mutate (insert_footnote) with a slugId opens by the resolved U
assert.deepEqual(
state.docNames,
[`page.${UUID}`],
"insert_footnote (via the mutatePage seam) must open the collab doc by UUID",
"insertFootnote (via the mutatePage seam) must open the collab doc by UUID",
);
});
@@ -372,7 +372,11 @@ test("replaceImage opens by the resolved UUID AND keys its page lock by that UUI
// single flush. This proves the flush actually executes queued callbacks, so
// probeRan === false above means "blocked", not "the flush never ran anyone".
let freeRan = false;
const freeDone = withPageLock(`page.free-${UUID}`, async () => {
// A DIFFERENT canonical UUID (unrelated to the page under test). withPageLock
// now asserts its key is a canonical UUID (#449), so the "free" probe key must
// also be a valid — but distinct — UUID, not a synthetic label.
const FREE_UUID = "99999999-9999-4999-8999-999999999999";
const freeDone = withPageLock(FREE_UUID, async () => {
freeRan = true;
});
await new Promise((r) => setImmediate(r));
@@ -1,4 +1,4 @@
// Server round-trip test for the stash_page MCP tool result shape. The in-app
// Server round-trip test for the stashPage MCP tool result shape. The in-app
// path returns the full documented `{ uri, size, sha256, images }` object, but
// the MCP transport must deliver the SAME shape: a resource_link (primary
// payload) PLUS a `structuredContent` mirror carrying sha256 + image counts.
@@ -107,7 +107,7 @@ async function buildBaseURL() {
});
}
test("stash_page MCP tool returns a resource_link AND a structuredContent mirror", async () => {
test("stashPage MCP tool returns a resource_link AND a structuredContent mirror", async () => {
const baseURL = await buildBaseURL();
const sandbox = makeSandbox();
const server = createDocmostMcpServer({
@@ -124,7 +124,7 @@ test("stash_page MCP tool returns a resource_link AND a structuredContent mirror
try {
const res = await client.callTool({
name: "stash_page",
name: "stashPage",
arguments: { pageId: "page-1" },
});

Some files were not shown because too many files have changed in this diff Show More