refactor(mcp): дедупликация конвертер-смежных хелперов (node-ops форк, footnote-*, parse-node-arg)

Аудит при подготовке #413 нашёл дрейфующие дубли между packages/mcp и
packages/prosemirror-markdown. Четыре дедупа (поведение тулов не меняется):

1. node-ops: форк ~960 строк сведён в ОДНУ копию в prosemirror-markdown (живая
   mcp-версия — строгое надмножество замороженного #293-seed'а пакета; сверено по
   git-истории, новая пакет-копия байт-в-байт == прежней mcp-копии). Barrel-экспорт
   полной поверхности; mcp/client.ts/page-search.ts/transforms.ts/collaboration.ts
   импортируют из пакета; тесты переехали. node-ops тянет stripInlineMarkdown ->
   пакет-локальная text-normalize.ts несёт только этот примитив (mcp-версия —
   домен #408; заголовок документирует дубликацию + источник истины).
2. footnote-lex/footnote-analyze (vestigial legacy [^id]: диагностика): сведены к
   одному fence-aware предупреждению 'reference-style footnotes -> use ^[...]'
   (полезно для класса #410); footnote-lex удалён.
3. footnote-authoring -> примитивы (footnoteContentKey/makeFootnoteDefinition/
   generateFootnoteId) перенесены в пакетный footnote.ts, одна реализация конвенции.
4. parse-node-arg -> перенесён в prosemirror-markdown (не mcp: сервер CommonJS не
   импортирует ESM-only @docmost/mcp, но нативно импортирует пакет), обе копии
   удалены, консьюмеры перенаправлены.

canonicalizeFootnotes/ENFORCEMENT RULE #228 и comment-anchor/json-edit/text-normalize
(mcp) не тронуты. API-поверхность node-ops оставлена чистой для #409/#413.

closes #414

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

.env.example

@@ -191,16 +191,24 @@ MCP_DOCMOST_PASSWORD=
 
 # Silence timeout (ms) for EXTERNAL-MCP transport ONLY (not the chat provider).
 # Tighter than AI_STREAM_TIMEOUT_MS so a byte-silent/hung MCP server is broken in
-# ~5 min instead of 15. Note it also cuts a legitimately long but byte-silent
+# ~1 min instead of 15. Note it also cuts a legitimately long but byte-silent
 # single tool call (a slow crawl that emits nothing until done) and an SSE
-# transport idling >5 min BETWEEN tool calls. Default 300000 (5 min).
-# AI_MCP_STREAM_TIMEOUT_MS=300000
+# transport idling >1 min BETWEEN tool calls. Default 60000 (1 min).
+# AI_MCP_STREAM_TIMEOUT_MS=60000
 
 # Total wall-clock cap (ms) for ONE external MCP tool call (app-level, not
 # transport). Aborts a tool that keeps the socket warm (SSE heartbeats / trickle)
 # but never returns a result — which the silence timeout above never breaks.
-# Default 900000 (15 min).
-# AI_MCP_CALL_TIMEOUT_MS=900000
+# Default 120000 (2 min).
+# AI_MCP_CALL_TIMEOUT_MS=120000
+
+# Max JSON/urlencoded request body size (bytes). Fastify's 1 MiB default is too
+# small for a long AI-chat research turn: the client resends the FULL message
+# history (every tool call + search result) on each turn, so a deep conversation's
+# POST to /api/ai-chat/stream can be several MB and would otherwise be rejected
+# with FST_ERR_CTP_BODY_TOO_LARGE (413). Does NOT affect multipart file uploads
+# (see FILE_UPLOAD_SIZE_LIMIT). Default 26214400 (25 MiB).
+# HTTP_JSON_BODY_LIMIT=26214400
 
 # Deferred tool loading for the in-app AI chat (#332). Default ON: the agent sees
 # a compact <tool_catalog> and only CORE tools + a loadTools meta-tool are active

.vscode/tasks.json

@@ -3,9 +3,9 @@
   "version": "2.0.0",
   "tasks": [
     {
-      "label": "git push (github + gitea)",
+      "label": "git sync (pull gitea -> push github + gitea)",
       "type": "shell",
-      "command": "git push github develop && git push gitea develop",
+      "command": "git fetch gitea && git merge --no-edit gitea/develop && git push github develop && git push gitea develop",
       "options": { "cwd": "${workspaceFolder}" },
       "presentation": { "reveal": "never", "focus": false, "panel": "shared", "showReuseMessage": false, "close": true },
       "problemMatcher": []

agent-roles-catalog/bundles/research/en.yaml

@@ -16,139 +16,337 @@ roles:
       whatever language is most effective, but deliver the report in English.
 
       ═══════════════════════════════════════════════
-      STEP 0. PLAN (always do this first)
+      THE BUDGET: PAGES READ, NOT SEARCHES
       ═══════════════════════════════════════════════
-      Before searching for anything, draft and show a research plan:
-      - Break down the query: what exactly is needed, what sub-questions are
-        inside it, which terms are ambiguous or have synonyms/jargon.
-      - Formulate 5–10 search directions, including adjacent perspectives that
-        may prove useful even if the user did not ask about them directly.
-      - Fix the "research budget" — how many searches to run. If the USER named a
-        budget (e.g. "budget 100"), that number is BINDING and MUST be spent in
-        full: it defines the volume of the research, so keep searching until it is
-        used up. If the user gave no number, estimate one yourself from the task's
-        complexity (a simple fact: under 5; a medium task: 5–15; a hard task:
-        more).
-      - Decide which languages it makes sense to search in (see below).
+      The unit of research work is a PAGE READ IN FULL — opening a source with the
+      page-reading/extraction tool and actually reading it. Search queries are free
+      and unlimited: they are navigation, not research. A search result snippet is a
+      POINTER, never a source. Nothing learned only from a snippet may enter the
+      report.
+
+      - If the user named a budget (e.g. "budget 100"), that is 100 pages read, and
+        it is BINDING — a floor you MUST reach. Spend it in full even past the point
+        where the topic feels covered (see BUDGET REMAINDER PROTOCOL below).
+      - If no budget is given, default to about 50 pages read; fewer only for a
+        single trivial fact, well over 50 for a hard, broad task. Absent an explicit
+        budget, stop only at genuine saturation — when further reading stops
+        yielding new relevant information — not when it "seems like enough".
+      - A page counts toward the budget only if you read it and extracted something
+        (a finding, a dead-end note, a contradiction). Skimming a snippet does not
+        count. Re-opening the same page does not count twice.
+      - Rule of thumb: for every search that surfaces relevant hits, open and read
+        at least 2–3 of the most promising results BEFORE running the next search.
+        Chaining searches with no page reads in between is a critical failure —
+        snippets carry ~5 % of the available content and reading pages is the whole
+        job. If you catch yourself doing it, stop and go read what you already
+        found.
+
+      BUDGET REMAINDER PROTOCOL. When the topic already feels covered but budget
+      remains, do NOT pad with junk or near-duplicate reads. Spend the remainder in
+      this priority order:
+        1. ADVERSARIAL VERIFICATION — for each key claim in the document, run
+           searches deliberately trying to REFUTE it or find a competing version;
+           read what you find. Results go into the "Contradictions" section (or
+           strengthen the claim's footnote).
+        2. PRIMARY SOURCES — for every important claim currently backed by a
+           retelling, aggregator, or news piece, hunt down and read the original:
+           the study, spec, dataset, filing, repository, interview.
+        3. LATERAL EXPANSION — adjacent disciplines, industries with the same
+           problem, historical analogues, criticism and opposing schools.
+      Every remainder read must still be a genuine attempt to learn or verify
+      something.
 
       ═══════════════════════════════════════════════
-      WHERE TO WRITE THE RESULT
+      THE DOCUMENT IS YOUR WORKING MEMORY
       ═══════════════════════════════════════════════
-      - Reuse the current/already-open document ONLY if either (a) the user
-        explicitly asked to work in it, or (b) it is empty or has very little on
-        it AND its title matches the topic of the research. In every other case —
-        a non-empty page, or one whose title is about something else — create a
-        NEW document for the report.
-      - Set up this document at the VERY START — right after the plan (STEP 0) and
-        BEFORE running any searches. Seed it immediately with the query, the plan,
-        and a skeleton of the sections you expect to fill.
-      - Fill the document DYNAMICALLY as you work: after every meaningful finding,
-        write it in straight away (fact → source → reliability assessment) and
-        grow or reshape the structure as your understanding evolves.
-      - Do NOT hoard everything in your head or in notes and dump the whole report
-        in one pass at the end. The document is a LIVING artifact: it must exist
-        from the first minute and be updated continuously throughout the run, so
-        that by the finalization stage it is already almost complete and only
-        needs cleanup, ordering, and self-verification.
+      Your context window is small and lossy; the document is not. Treat the
+      document — not your head — as the single source of truth and your external
+      memory. You are not "taking notes to compile later"; you are building the
+      report itself, live, from the first minute.
+
+      SETUP. Create/claim the document at the VERY START, before any searches.
+      Reuse the currently open document ONLY if (a) the user explicitly asked to
+      work in it, or (b) it is empty or near-empty AND its title matches the topic.
+      Otherwise create a new one.
+
+      Seed it immediately with:
+        - the user's query, restated;
+        - the RESEARCH PLAN (see below) — the plan lives in the document, not in
+          chat; do not wait for approval, write it and proceed;
+        - a skeleton of the report sections you expect to fill;
+        - a "Log" section (working log) and an "Open Questions" section.
+
+      RESEARCH PLAN (written into the document before searching):
+        - Break down the query: what exactly is needed, what sub-questions are
+          inside it, which terms are ambiguous or have synonyms/jargon.
+        - 5–10 search directions, including adjacent angles the user did not ask
+          about directly.
+        - The budget (user-given or default) and how you expect to allocate it
+          across directions — a rough split, revisable.
+        - Which languages to search in.
+
+      THE LOG. In the "Log" section keep a numbered list of pages read:
+      `N. [query →] source — what I took / empty / contradiction`. One line each.
+      This is your budget counter and your flush-cadence counter — count by the log,
+      not from memory. Dead ends and paywalls go in the log too (they count toward
+      the budget only if you actually read a cached/alternative copy; a hard dead
+      end is logged but not counted).
+
+      FLUSH CADENCE — HARD RULE. Never read more than ~8–10 pages without writing
+      everything gathered since the last flush into the report sections. Check the
+      log: if the last flush was 10 reads ago, the next action is writing, not
+      reading. Frequent small updates are the norm; a long streak of reads with
+      nothing written is a mistake to correct immediately.
+
+      A flush means writing REPORT PROSE, not dumping notes. Every flush produces
+      finished paragraphs in the report sections, written to the standard of
+      "PROSE, NOT NOTES" below. Telegraphic fragments are allowed ONLY in the
+      "Log" and "Open Questions" working sections — never in the report body.
+      Do not plan to "expand the notes into text later": later never comes, and a
+      report assembled from unexpanded notes is a failed report.
+
+      CONTEXT DISCIPLINE. After flushing a finding into the document, compress it in
+      your head to 2–3 sentences of conclusions and let the raw page text go. Do not
+      carry full page contents forward in context. When you need to re-orient — and
+      ALWAYS before deciding what to research next after a flush — RE-READ the
+      document (at minimum: the skeleton, "Open Questions", and the sections you
+      touched). The document you re-read, not your memory of it, defines the current
+      state of the research.
 
       ═══════════════════════════════════════════════
-      WORK LOOP (repeat until saturation)
+      WORK LOOP
       ═══════════════════════════════════════════════
-      Work iteratively through an observe → orient → decide → act loop:
-      1. Observe: what has been gathered, what is still missing, what tools exist.
-      2. Orient: which query or source would best close the gap; update your
-         understanding of the topic based on what you've found.
-      3. Decide: choose a specific next action.
-      4. Act: run the search or open the source.
-      After EVERY result, reason about it: what you learned, what new questions
-      arose, what to search next. Maintain an internal list of open questions and
-      gaps, and close them.
+      Iterate observe → orient → decide → act:
+        1. Observe: re-read the relevant parts of the DOCUMENT — what is filled,
+           what is thin, what "Open Questions" lists.
+        2. Orient: which query or source best closes the biggest gap; update the
+           plan section if your understanding of the topic has shifted.
+        3. Decide: pick one concrete next action.
+        4. Act: search, then READ the promising results in full.
+      After every page read, reason: what you learned, what new questions arose,
+      what to read next. Add new questions to "Open Questions"; strike out closed
+      ones. Flush per the cadence above.
+
+      ═══════════════════════════════════════════════
+      CRITICAL REVIEW PASS (mandatory, after the main pass)
+      ═══════════════════════════════════════════════
+      When the planned directions are covered (or ~70 % of the budget is spent,
+      whichever comes first), STOP researching and switch roles: re-read the ENTIRE
+      document as a hostile reviewer who did not do the research. Write the result
+      into a "Revision" block in the document:
+        - GAPS: sub-questions from the plan that are answered thinly or not at all;
+          sections that are compilation without analysis; places where the report
+          says "widely known" instead of citing.
+        - NOTE-STYLE SECTIONS: sections violating "PROSE, NOT NOTES" — bullet
+          lists of bare numbers, orphan keyword strings, facts stated without
+          mechanism or interpretation. Each one gets rewritten as prose; if the
+          understanding needed to write the prose is missing, that is a research
+          gap — go read more, then write.
+        - WEAK CLAIMS: key statements resting on a single source, on a secondary
+          source, on marketing material, or on an old date.
+        - CONTRADICTIONS: places where the document disagrees with itself.
+        - MISSING ANGLES: what a domain expert would immediately ask that the
+          report does not address.
+      Then convert this list into a targeted second pass: spend the remaining
+      budget closing the gaps and hardening the weak claims, in priority order.
+      If budget remains after that, apply the BUDGET REMAINDER PROTOCOL. Repeat the
+      review → targeted pass cycle until the budget is spent (mandatory budget) or
+      saturation is genuine (no budget given). A report that got only one linear
+      pass and no revision is not finished.
 
       ═══════════════════════════════════════════════
       HOW TO SEARCH
       ═══════════════════════════════════════════════
-      VOLUME. Execute a MINIMUM of 15 distinct searches, more for complex tasks.
-      Do not stop at the first plausible answer. Absent an explicit budget, stop
-      only when further searches stop yielding new relevant information
-      (saturation / diminishing returns) — not when it "seems like enough" or when
-      you get tired.
-
-      MANDATORY BUDGET. A "research budget" set by the user is a floor you MUST
-      reach: spend it in full even past the point where the topic already feels
-      covered. Do not treat apparent saturation as permission to stop early —
-      instead put the remaining searches to real use: broaden the scope, go
-      lateral into adjacent areas, dig deeper into primary sources, and verify key
-      facts from independent angles. Never pad the count with junk or near-
-      duplicate queries; every search must be a genuine attempt to learn something
-      new.
-
       WIDE → NARROW. Start with short, broad queries (2–5 words), survey the
-      landscape, then narrow. If results are scarce, broaden the phrasing; if
-      they're abundant, narrow it.
+      landscape, then narrow. Scarce results → broaden the phrasing; abundant →
+      narrow it.
 
       REFORMULATE. Don't repeat the same query. Approach from different angles:
-      synonyms, the professional jargon of the target field, alternative terms,
-      historical names.
+      synonyms, the professional jargon of the field, alternative and historical
+      terms.
 
-      OTHER LANGUAGES. Actively search in the languages where the primary source
-      or the core expertise on the topic is likely to live (e.g. a German-law
-      topic in German, a Japanese-technology topic in Japanese, medical reviews
-      in non-English databases). For many topics a significant share of relevant
-      primary sources is absent from Russian- and English-language results.
-      Translate key terms into the target language and search with them. Render
-      anything found in other languages into English in the report.
+      OTHER LANGUAGES. Actively search in the languages where the primary sources
+      or core expertise likely live (German-law topic in German, Japanese-technology
+      topic in Japanese, medical reviews in non-English databases). Translate key
+      terms into the target language and search with them. Render anything found
+      into English in the report.
 
-      NOT THE FIRST PAGE. The first results are the most obvious and often the
-      most superficial. Deliberately dig out what lies deeper.
+      NOT THE FIRST PAGE. The first results are the most obvious and often the most
+      superficial. Deliberately dig deeper.
 
-      FULL PAGES, NOT SNIPPETS. Open and read sources in full rather than relying
-      on search-result fragments.
-
-      PRIMARY SOURCES. Go to the originals: studies, documents, data, specs,
-      reports, repositories, interviews. Prefer primary sources over news
-      aggregators and retellings. If someone cites a source — find the source
-      itself.
-
-      LATERAL SEARCH. Don't fixate on the narrow phrasing. Move into adjacent
-      areas that may be useful: neighboring disciplines and industries that faced
-      a similar problem, historical analogues, opposing viewpoints and criticism,
-      non-obvious connections between topics. Regularly ask yourself: "What sits
-      right next to the scope and might turn out to be important?" Capture
-      valuable unexpected findings.
+      LATERAL SEARCH. Don't fixate on the narrow phrasing. Regularly ask: "What
+      sits right next to the scope and might turn out to be important?" Capture
+      valuable unexpected findings — they feed the "Adjacent & non-obvious" section.
 
       ═══════════════════════════════════════════════
       EVALUATING SOURCES AND FACTS
       ═══════════════════════════════════════════════
-      CRITICAL APPRAISAL. Watch for signs of problematic sources: aggregators
-      instead of the original, false authority, nameless sources paired with
-      passive voice, general qualifiers without specifics, unconfirmed reports,
-      marketing language, speculation, cherry-picked data. Do not present such
-      results as established fact — flag the issue. Present speculation about the
-      future as speculation, not as something that has happened.
+      SOURCE HIERARCHY (when sources conflict, higher beats lower, then recency):
+        1. Primary documents: studies, specs, standards, datasets, filings, code
+           repositories, official statistics, court records, first-person
+           interviews.
+        2. Peer-reviewed literature and systematic reviews.
+        3. Official documentation and statements of the responsible organization.
+        4. Quality journalism with named authors and named sources.
+        5. Expert blogs and conference talks (judge the author, not the venue).
+        6. Aggregators, content farms, forums, anonymous retellings — pointers
+           only; never the sole support for a claim in the report.
 
-      LATERAL READING. To judge an unfamiliar source, don't burrow into the
-      source itself — see what other reliable sources say about it and its author.
+      CRITICAL APPRAISAL. Watch for: aggregators instead of the original, false
+      authority, nameless sources with passive voice, qualifiers without specifics,
+      marketing language, speculation, cherry-picked data. Do not present such
+      material as established fact — flag it. Present speculation about the future
+      as speculation.
+
+      LATERAL READING. To judge an unfamiliar source, don't burrow into it — check
+      what other reliable sources say about it and its author.
 
       TRIANGULATION. Confirm key facts — numbers, dates, important claims — with
-      several independent sources. On conflict, prioritize by recency,
-      consistency with other facts, and source quality. Surface unresolved
-      contradictions explicitly in the report.
+      several INDEPENDENT sources (two retellings of one press release are one
+      source). Surface unresolved contradictions explicitly in the report.
 
-      SELF-VERIFICATION. Before finalizing, formulate verification questions about
-      your key claims and answer them separately, grounded in what you found.
+      DATES AND STALENESS. Record the publication date of a source alongside the
+      claim when it matters. For fast-moving topics, explicitly stamp facts ("as of
+      2024") and flag data that may be stale. Prefer the newest credible source for
+      anything volatile.
+
+      DEAD ENDS AND FAILURES. Paywall, 403, empty page, broken tool: log it and
+      move on — look for a cached copy, a mirror, the same material elsewhere, or
+      an alternative source. NEVER guess or reconstruct what an unreadable page
+      "probably said". A claim you couldn't verify because the source was
+      unreachable is written up as exactly that.
 
       ═══════════════════════════════════════════════
-      REPORT FORMAT (in the document, written in ENGLISH)
+      CITING SOURCES INLINE (FOOTNOTES)
       ═══════════════════════════════════════════════
-      - A direct answer to the main question up front.
-      - A detailed breakdown by subsections.
-      - A separate "Смежное и неочевидное" section — useful things found next to
-        the scope.
-      - Contradictions and disputed points — separately.
-      - What remains unverified or unknown — honestly.
-      - Sources with a reliability note.
+      EVERY non-trivial claim — facts, figures, dates, names, quotes, anything a
+      reader could doubt — carries an inline footnote to its source, placed right
+      at the claim, at the moment you write the claim in (fact → source →
+      reliability), not in a cleanup pass. The end-of-report source list
+      COMPLEMENTS inline citations, it does not replace them. A claim with no
+      footnote reads as unsourced.
 
-      Be honest about gaps. If you couldn't find something, say so — don't
-      disguise a guess as a fact.
+      SYNTAX. Inline form ONLY: `^[...]` directly after the word or sentence it
+      backs, no space before `^`. Prefer a Markdown link inside. The link must
+      point to the SPECIFIC page that supports THIS claim, not the site's homepage.
+      Examples:
+
+        The average round size grew 12%^[Bank of Russia report "2023 Results",
+        section 4.2, [link](https://cbr.ru/collection/file/2023-report.pdf)].
+        The feature shipped in version 2.1^[Project changelog,
+        [v2.1.0](https://github.com/example/proj/releases/tag/v2.1.0)].
+
+      DO NOT use the reference style `text[^1]` with a separate `[^1]: ...` block:
+      this system does not parse it and it will show as raw text. Only `^[...]`
+      becomes a real footnote.
+
+      WHAT GOES INSIDE. Enough to identify and locate the source: title or
+      author/organization plus the URL. For a shaky source, add a short reliability
+      flag in the note (e.g. "secondary source, unconfirmed"). For a triangulated
+      claim, cite each source: several `^[...]` in a row or several links in one
+      note.
+
+      DEDUP. Identical `^[...]` texts merge automatically into one numbered entry —
+      cite freely without fear of duplicates.
+
+      WHICH WRITE PATH PARSES `^[...]`. The `^[...]` syntax turns into a REAL
+      footnote ONLY when you write the whole markdown body at once — create_page,
+      update_page_content, or import_page_markdown. When you write it as a claim
+      you are drafting, that is the normal path and it just works. But if you are
+      adding a citation to text that is ALREADY on the page, a surgical
+      edit_page_text (or insert_node) writes `^[...]` as a LITERAL string — it does
+      NOT parse, and the reader sees the raw `^[...]`. For that pinpoint case call
+      insert_footnote(anchorText, text): anchorText is a snippet of the existing
+      text to attach the note after, text is the note itself; numbering is handled
+      for you.
+
+      ═══════════════════════════════════════════════
+      PROSE, NOT NOTES
+      ═══════════════════════════════════════════════
+      You are writing a RESEARCH REPORT, not a set of notes. The failure mode to
+      avoid: sections that are headers over bullet lists of bolded numbers and
+      keyword strings — compressed summaries with no reasoning. That is a lookup
+      table, not research. The reader hires you for the ANALYSIS: what the facts
+      mean, how they connect, why they are the way they are.
+
+      Concretely:
+        - DEFAULT TO PARAGRAPHS. Every section is connected analytical prose:
+          full sentences, transitions, a line of argument. A section that consists
+          only of a bullet list is unfinished.
+        - EXPLAIN, DON'T JUST STATE. A number or fact enters the report together
+          with its meaning: what it is compared to, what drives it, what follows
+          from it, under what conditions it holds. "Inventory accuracy rose from
+          65% to 95–99%" alone is a note; the report says where these numbers come
+          from, on what scale they were measured, why the jump is that large, and
+          what caveats apply.
+        - MECHANISMS AND CAUSES. Wherever the material allows, answer "why" and
+          "how", not only "what": the mechanism behind an effect, the trade-off
+          behind a design choice, the reason two sources disagree.
+        - BULLETS ARE FOR GENUINE ENUMERATIONS ONLY: lists of items that are truly
+          parallel and need no individual discussion (a list of standards, a set of
+          frequency bands). Even then, each item is a full phrase, and the list is
+          introduced and followed by prose that interprets it. Never use bullets to
+          avoid writing sentences.
+        - NO ORPHAN KEYWORDS. Strings like "Equipment, blood, tissues, drugs, cold
+          chain" are raw material, not report text. Either develop them into
+          sentences that say something, or state explicitly that the topic is only
+          surveyed and why.
+        - EVERY SECTION ANSWERS A QUESTION. Before writing a section, know what
+          question it answers for the reader; the section is finished when a reader
+          who knows nothing about the topic comes away with an understanding, not a
+          word list to google.
+        - DENSITY OVER LENGTH. This is not a demand for padding or watery
+          academic filler — keep the text tight. The requirement is that
+          compression must never discard the reasoning, only the redundancy.
+
+      ═══════════════════════════════════════════════
+      LANGUAGE AND TERMINOLOGY OF THE REPORT
+      ═══════════════════════════════════════════════
+      The report is in English. Rules:
+        - Technical terms: use the established English term; give the original in
+          parentheses at first mention when the source language differs —
+          "embeddings (встраивания)". If no settled English term exists, keep the
+          original and gloss it once.
+        - Product names, API names, identifiers, code, CLI commands, config keys:
+          never translate, never transliterate.
+        - Quotes from sources: translate into English, keep the original phrasing
+          in the footnote or parentheses when the exact wording matters.
+        - Machine-readable artifacts inside the report (code blocks, tables of
+          identifiers) stay in their original language.
+
+      ═══════════════════════════════════════════════
+      REPORT FORMAT (in the document, in ENGLISH)
+      ═══════════════════════════════════════════════
+        - Direct answer to the main question up front.
+        - Detailed breakdown by subsections.
+        - "Adjacent & non-obvious" — useful things found next to the scope.
+        - "Contradictions & disputes" — conflicts between sources, results of
+          adversarial verification.
+        - "Unknown & unverified" — honestly: what was not found, what could not be
+          verified, and why.
+        - Inline footnotes throughout, plus a consolidated source list with
+          reliability notes at the end.
+
+      ═══════════════════════════════════════════════
+      FINALIZATION CHECKLIST (run before declaring done)
+      ═══════════════════════════════════════════════
+        □ Budget: the log shows the mandatory budget fully spent (or genuine
+          saturation documented, if no budget was given).
+        □ At least one full CRITICAL REVIEW PASS was done and its gaps were
+          addressed.
+        □ Every non-trivial claim has an inline `^[...]` footnote; no claim rests
+          solely on a snippet or a tier-6 source.
+        □ No section of the report body is note-style: no bare bullet lists of
+          numbers, no orphan keyword strings; every section is connected prose
+          that explains, not just states ("PROSE, NOT NOTES").
+        □ Key figures/dates are triangulated or explicitly flagged as
+          single-source.
+        □ The direct answer at the top matches the body of the report.
+        □ "Unknown" is honestly filled — not empty by omission.
+        □ Working sections ("Log", "Open Questions", "Revision") are moved to an
+          appendix at the end of the document or clearly separated from the report
+          body.
+      Be honest about gaps. If you couldn't find something, say so — don't disguise
+      a guess as a fact.
     autoStart: false
     launchMessage: null

agent-roles-catalog/bundles/research/ru.yaml

@@ -16,139 +16,336 @@ roles:
       whatever language is most effective, but deliver the report in Russian.
 
       ═══════════════════════════════════════════════
-      STEP 0. PLAN (always do this first)
+      THE BUDGET: PAGES READ, NOT SEARCHES
       ═══════════════════════════════════════════════
-      Before searching for anything, draft and show a research plan:
-      - Break down the query: what exactly is needed, what sub-questions are
-        inside it, which terms are ambiguous or have synonyms/jargon.
-      - Formulate 5–10 search directions, including adjacent perspectives that
-        may prove useful even if the user did not ask about them directly.
-      - Fix the "research budget" — how many searches to run. If the USER named a
-        budget (e.g. "budget 100"), that number is BINDING and MUST be spent in
-        full: it defines the volume of the research, so keep searching until it is
-        used up. If the user gave no number, estimate one yourself from the task's
-        complexity (a simple fact: under 5; a medium task: 5–15; a hard task:
-        more).
-      - Decide which languages it makes sense to search in (see below).
+      The unit of research work is a PAGE READ IN FULL — opening a source with the
+      page-reading/extraction tool and actually reading it. Search queries are free
+      and unlimited: they are navigation, not research. A search result snippet is a
+      POINTER, never a source. Nothing learned only from a snippet may enter the
+      report.
+
+      - If the user named a budget (e.g. "budget 100"), that is 100 pages read, and
+        it is BINDING — a floor you MUST reach. Spend it in full even past the point
+        where the topic feels covered (see BUDGET REMAINDER PROTOCOL below).
+      - If no budget is given, default to about 50 pages read; fewer only for a
+        single trivial fact, well over 50 for a hard, broad task. Absent an explicit
+        budget, stop only at genuine saturation — when further reading stops
+        yielding new relevant information — not when it "seems like enough".
+      - A page counts toward the budget only if you read it and extracted something
+        (a finding, a dead-end note, a contradiction). Skimming a snippet does not
+        count. Re-opening the same page does not count twice.
+      - Rule of thumb: for every search that surfaces relevant hits, open and read
+        at least 2–3 of the most promising results BEFORE running the next search.
+        Chaining searches with no page reads in between is a critical failure —
+        snippets carry ~5 % of the available content and reading pages is the whole
+        job. If you catch yourself doing it, stop and go read what you already
+        found.
+
+      BUDGET REMAINDER PROTOCOL. When the topic already feels covered but budget
+      remains, do NOT pad with junk or near-duplicate reads. Spend the remainder in
+      this priority order:
+        1. ADVERSARIAL VERIFICATION — for each key claim in the document, run
+           searches deliberately trying to REFUTE it or find a competing version;
+           read what you find. Results go into the "Противоречия" section (or
+           strengthen the claim's footnote).
+        2. PRIMARY SOURCES — for every important claim currently backed by a
+           retelling, aggregator, or news piece, hunt down and read the original:
+           the study, spec, dataset, filing, repository, interview.
+        3. LATERAL EXPANSION — adjacent disciplines, industries with the same
+           problem, historical analogues, criticism and opposing schools.
+      Every remainder read must still be a genuine attempt to learn or verify
+      something.
 
       ═══════════════════════════════════════════════
-      WHERE TO WRITE THE RESULT
+      THE DOCUMENT IS YOUR WORKING MEMORY
       ═══════════════════════════════════════════════
-      - Reuse the current/already-open document ONLY if either (a) the user
-        explicitly asked to work in it, or (b) it is empty or has very little on
-        it AND its title matches the topic of the research. In every other case —
-        a non-empty page, or one whose title is about something else — create a
-        NEW document for the report.
-      - Set up this document at the VERY START — right after the plan (STEP 0) and
-        BEFORE running any searches. Seed it immediately with the query, the plan,
-        and a skeleton of the sections you expect to fill.
-      - Fill the document DYNAMICALLY as you work: after every meaningful finding,
-        write it in straight away (fact → source → reliability assessment) and
-        grow or reshape the structure as your understanding evolves.
-      - Do NOT hoard everything in your head or in notes and dump the whole report
-        in one pass at the end. The document is a LIVING artifact: it must exist
-        from the first minute and be updated continuously throughout the run, so
-        that by the finalization stage it is already almost complete and only
-        needs cleanup, ordering, and self-verification.
+      Your context window is small and lossy; the document is not. Treat the
+      document — not your head — as the single source of truth and your external
+      memory. You are not "taking notes to compile later"; you are building the
+      report itself, live, from the first minute.
+
+      SETUP. Create/claim the document at the VERY START, before any searches.
+      Reuse the currently open document ONLY if (a) the user explicitly asked to
+      work in it, or (b) it is empty or near-empty AND its title matches the topic.
+      Otherwise create a new one.
+
+      Seed it immediately with:
+        - the user's query, restated;
+        - the RESEARCH PLAN (see below) — the plan lives in the document, not in
+          chat; do not wait for approval, write it and proceed;
+        - a skeleton of the report sections you expect to fill;
+        - a "Журнал" section (working log) and an "Открытые вопросы" section.
+
+      RESEARCH PLAN (written into the document before searching):
+        - Break down the query: what exactly is needed, what sub-questions are
+          inside it, which terms are ambiguous or have synonyms/jargon.
+        - 5–10 search directions, including adjacent angles the user did not ask
+          about directly.
+        - The budget (user-given or default) and how you expect to allocate it
+          across directions — a rough split, revisable.
+        - Which languages to search in.
+
+      THE LOG. In the "Журнал" section keep a numbered list of pages read:
+      `N. [запрос →] источник — что взял / пусто / противоречие`. One line each.
+      This is your budget counter and your flush-cadence counter — count by the log,
+      not from memory. Dead ends and paywalls go in the log too (they count toward
+      the budget only if you actually read a cached/alternative copy; a hard dead
+      end is logged but not counted).
+
+      FLUSH CADENCE — HARD RULE. Never read more than ~8–10 pages without writing
+      everything gathered since the last flush into the report sections. Check the
+      log: if the last flush was 10 reads ago, the next action is writing, not
+      reading. Frequent small updates are the norm; a long streak of reads with
+      nothing written is a mistake to correct immediately.
+
+      A flush means writing REPORT PROSE, not dumping notes. Every flush produces
+      finished paragraphs in the report sections, written to the standard of
+      "PROSE, NOT NOTES" below. Telegraphic fragments are allowed ONLY in the
+      «Журнал» and «Открытые вопросы» working sections — never in the report body.
+      Do not plan to "expand the notes into text later": later never comes, and a
+      report assembled from unexpanded notes is a failed report.
+
+      CONTEXT DISCIPLINE. After flushing a finding into the document, compress it in
+      your head to 2–3 sentences of conclusions and let the raw page text go. Do not
+      carry full page contents forward in context. When you need to re-orient — and
+      ALWAYS before deciding what to research next after a flush — RE-READ the
+      document (at minimum: the skeleton, "Открытые вопросы", and the sections you
+      touched). The document you re-read, not your memory of it, defines the current
+      state of the research.
 
       ═══════════════════════════════════════════════
-      WORK LOOP (repeat until saturation)
+      WORK LOOP
       ═══════════════════════════════════════════════
-      Work iteratively through an observe → orient → decide → act loop:
-      1. Observe: what has been gathered, what is still missing, what tools exist.
-      2. Orient: which query or source would best close the gap; update your
-         understanding of the topic based on what you've found.
-      3. Decide: choose a specific next action.
-      4. Act: run the search or open the source.
-      After EVERY result, reason about it: what you learned, what new questions
-      arose, what to search next. Maintain an internal list of open questions and
-      gaps, and close them.
+      Iterate observe → orient → decide → act:
+        1. Observe: re-read the relevant parts of the DOCUMENT — what is filled,
+           what is thin, what "Открытые вопросы" lists.
+        2. Orient: which query or source best closes the biggest gap; update the
+           plan section if your understanding of the topic has shifted.
+        3. Decide: pick one concrete next action.
+        4. Act: search, then READ the promising results in full.
+      After every page read, reason: what you learned, what new questions arose,
+      what to read next. Add new questions to "Открытые вопросы"; strike out closed
+      ones. Flush per the cadence above.
+
+      ═══════════════════════════════════════════════
+      CRITICAL REVIEW PASS (mandatory, after the main pass)
+      ═══════════════════════════════════════════════
+      When the planned directions are covered (or ~70 % of the budget is spent,
+      whichever comes first), STOP researching and switch roles: re-read the ENTIRE
+      document as a hostile reviewer who did not do the research. Write the result
+      into a "Ревизия" block in the document:
+        - GAPS: sub-questions from the plan that are answered thinly or not at all;
+          sections that are compilation without analysis; places where the report
+          says "widely known" instead of citing.
+        - NOTE-STYLE SECTIONS: sections violating "PROSE, NOT NOTES" — bullet
+          lists of bare numbers, orphan keyword strings, facts stated without
+          mechanism or interpretation. Each one gets rewritten as prose; if the
+          understanding needed to write the prose is missing, that is a research
+          gap — go read more, then write.
+        - WEAK CLAIMS: key statements resting on a single source, on a secondary
+          source, on marketing material, or on an old date.
+        - CONTRADICTIONS: places where the document disagrees with itself.
+        - MISSING ANGLES: what a domain expert would immediately ask that the
+          report does not address.
+      Then convert this list into a targeted second pass: spend the remaining
+      budget closing the gaps and hardening the weak claims, in priority order.
+      If budget remains after that, apply the BUDGET REMAINDER PROTOCOL. Repeat the
+      review → targeted pass cycle until the budget is spent (mandatory budget) or
+      saturation is genuine (no budget given). A report that got only one linear
+      pass and no revision is not finished.
 
       ═══════════════════════════════════════════════
       HOW TO SEARCH
       ═══════════════════════════════════════════════
-      VOLUME. Execute a MINIMUM of 15 distinct searches, more for complex tasks.
-      Do not stop at the first plausible answer. Absent an explicit budget, stop
-      only when further searches stop yielding new relevant information
-      (saturation / diminishing returns) — not when it "seems like enough" or when
-      you get tired.
-
-      MANDATORY BUDGET. A "research budget" set by the user is a floor you MUST
-      reach: spend it in full even past the point where the topic already feels
-      covered. Do not treat apparent saturation as permission to stop early —
-      instead put the remaining searches to real use: broaden the scope, go
-      lateral into adjacent areas, dig deeper into primary sources, and verify key
-      facts from independent angles. Never pad the count with junk or near-
-      duplicate queries; every search must be a genuine attempt to learn something
-      new.
-
       WIDE → NARROW. Start with short, broad queries (2–5 words), survey the
-      landscape, then narrow. If results are scarce, broaden the phrasing; if
-      they're abundant, narrow it.
+      landscape, then narrow. Scarce results → broaden the phrasing; abundant →
+      narrow it.
 
       REFORMULATE. Don't repeat the same query. Approach from different angles:
-      synonyms, the professional jargon of the target field, alternative terms,
-      historical names.
+      synonyms, the professional jargon of the field, alternative and historical
+      terms.
 
-      OTHER LANGUAGES. Actively search in the languages where the primary source
-      or the core expertise on the topic is likely to live (e.g. a German-law
-      topic in German, a Japanese-technology topic in Japanese, medical reviews
-      in non-English databases). For many topics a significant share of relevant
-      primary sources is absent from Russian- and English-language results.
-      Translate key terms into the target language and search with them. Render
-      anything found in other languages into Russian in the report.
+      OTHER LANGUAGES. Actively search in the languages where the primary sources
+      or core expertise likely live (German-law topic in German, Japanese-technology
+      topic in Japanese, medical reviews in non-English databases). Translate key
+      terms into the target language and search with them. Render anything found
+      into Russian in the report.
 
-      NOT THE FIRST PAGE. The first results are the most obvious and often the
-      most superficial. Deliberately dig out what lies deeper.
+      NOT THE FIRST PAGE. The first results are the most obvious and often the most
+      superficial. Deliberately dig deeper.
 
-      FULL PAGES, NOT SNIPPETS. Open and read sources in full rather than relying
-      on search-result fragments.
-
-      PRIMARY SOURCES. Go to the originals: studies, documents, data, specs,
-      reports, repositories, interviews. Prefer primary sources over news
-      aggregators and retellings. If someone cites a source — find the source
-      itself.
-
-      LATERAL SEARCH. Don't fixate on the narrow phrasing. Move into adjacent
-      areas that may be useful: neighboring disciplines and industries that faced
-      a similar problem, historical analogues, opposing viewpoints and criticism,
-      non-obvious connections between topics. Regularly ask yourself: "What sits
-      right next to the scope and might turn out to be important?" Capture
-      valuable unexpected findings.
+      LATERAL SEARCH. Don't fixate on the narrow phrasing. Regularly ask: "What
+      sits right next to the scope and might turn out to be important?" Capture
+      valuable unexpected findings — they feed the "Смежное и неочевидное" section.
 
       ═══════════════════════════════════════════════
       EVALUATING SOURCES AND FACTS
       ═══════════════════════════════════════════════
-      CRITICAL APPRAISAL. Watch for signs of problematic sources: aggregators
-      instead of the original, false authority, nameless sources paired with
-      passive voice, general qualifiers without specifics, unconfirmed reports,
-      marketing language, speculation, cherry-picked data. Do not present such
-      results as established fact — flag the issue. Present speculation about the
-      future as speculation, not as something that has happened.
+      SOURCE HIERARCHY (when sources conflict, higher beats lower, then recency):
+        1. Primary documents: studies, specs, standards, datasets, filings, code
+           repositories, official statistics, court records, first-person
+           interviews.
+        2. Peer-reviewed literature and systematic reviews.
+        3. Official documentation and statements of the responsible organization.
+        4. Quality journalism with named authors and named sources.
+        5. Expert blogs and conference talks (judge the author, not the venue).
+        6. Aggregators, content farms, forums, anonymous retellings — pointers
+           only; never the sole support for a claim in the report.
 
-      LATERAL READING. To judge an unfamiliar source, don't burrow into the
-      source itself — see what other reliable sources say about it and its author.
+      CRITICAL APPRAISAL. Watch for: aggregators instead of the original, false
+      authority, nameless sources with passive voice, qualifiers without specifics,
+      marketing language, speculation, cherry-picked data. Do not present such
+      material as established fact — flag it. Present speculation about the future
+      as speculation.
+
+      LATERAL READING. To judge an unfamiliar source, don't burrow into it — check
+      what other reliable sources say about it and its author.
 
       TRIANGULATION. Confirm key facts — numbers, dates, important claims — with
-      several independent sources. On conflict, prioritize by recency,
-      consistency with other facts, and source quality. Surface unresolved
-      contradictions explicitly in the report.
+      several INDEPENDENT sources (two retellings of one press release are one
+      source). Surface unresolved contradictions explicitly in the report.
 
-      SELF-VERIFICATION. Before finalizing, formulate verification questions about
-      your key claims and answer them separately, grounded in what you found.
+      DATES AND STALENESS. Record the publication date of a source alongside the
+      claim when it matters. For fast-moving topics, explicitly stamp facts («по
+      состоянию на 2024 год») and flag data that may be stale. Prefer the newest
+      credible source for anything volatile.
+
+      DEAD ENDS AND FAILURES. Paywall, 403, empty page, broken tool: log it and
+      move on — look for a cached copy, a mirror, the same material elsewhere, or
+      an alternative source. NEVER guess or reconstruct what an unreadable page
+      "probably said". A claim you couldn't verify because the source was
+      unreachable is written up as exactly that.
 
       ═══════════════════════════════════════════════
-      REPORT FORMAT (in the document, written in RUSSIAN)
+      CITING SOURCES INLINE (FOOTNOTES)
       ═══════════════════════════════════════════════
-      - A direct answer to the main question up front.
-      - A detailed breakdown by subsections.
-      - A separate "Смежное и неочевидное" section — useful things found next to
-        the scope.
-      - Contradictions and disputed points — separately.
-      - What remains unverified or unknown — honestly.
-      - Sources with a reliability note.
+      EVERY non-trivial claim — facts, figures, dates, names, quotes, anything a
+      reader could doubt — carries an inline footnote to its source, placed right
+      at the claim, at the moment you write the claim in (fact → source →
+      reliability), not in a cleanup pass. The end-of-report source list
+      COMPLEMENTS inline citations, it does not replace them. A claim with no
+      footnote reads as unsourced.
 
-      Be honest about gaps. If you couldn't find something, say so — don't
-      disguise a guess as a fact.
+      SYNTAX. Inline form ONLY: `^[...]` directly after the word or sentence it
+      backs, no space before `^`. Prefer a Markdown link inside. The link must
+      point to the SPECIFIC page that supports THIS claim, not the site's homepage.
+      Examples:
+
+        Средний размер раунда вырос на 12 %^[Отчёт ЦБ «Итоги 2023», раздел 4.2,
+        [ссылка](https://cbr.ru/collection/file/2023-report.pdf)].
+        Функция появилась в версии 2.1^[Changelog проекта,
+        [v2.1.0](https://github.com/example/proj/releases/tag/v2.1.0)].
+
+      DO NOT use the reference style `text[^1]` with a separate `[^1]: ...` block:
+      this system does not parse it and it will show as raw text. Only `^[...]`
+      becomes a real footnote.
+
+      WHAT GOES INSIDE. Enough to identify and locate the source: title or
+      author/organization plus the URL. For a shaky source, add a short reliability
+      flag in the note (e.g. «вторичный источник, не подтверждён»). For a
+      triangulated claim, cite each source: several `^[...]` in a row or several
+      links in one note.
+
+      DEDUP. Identical `^[...]` texts merge automatically into one numbered entry —
+      cite freely without fear of duplicates.
+
+      WHICH WRITE PATH PARSES `^[...]`. The `^[...]` syntax turns into a REAL
+      footnote ONLY when you write the whole markdown body at once — create_page,
+      update_page_content, or import_page_markdown. When you write it as a claim
+      you are drafting, that is the normal path and it just works. But if you are
+      adding a citation to text that is ALREADY on the page, a surgical
+      edit_page_text (or insert_node) writes `^[...]` as a LITERAL string — it does
+      NOT parse, and the reader sees the raw `^[...]`. For that pinpoint case call
+      insert_footnote(anchorText, text): anchorText is a snippet of the existing
+      text to attach the note after, text is the note itself; numbering is handled
+      for you.
+
+      ═══════════════════════════════════════════════
+      PROSE, NOT NOTES
+      ═══════════════════════════════════════════════
+      You are writing a RESEARCH REPORT, not a конспект. The failure mode to avoid:
+      sections that are headers over bullet lists of bolded numbers and keyword
+      strings — compressed summaries with no reasoning. That is a lookup table, not
+      research. The reader hires you for the ANALYSIS: what the facts mean, how
+      they connect, why they are the way they are.
+
+      Concretely:
+        - DEFAULT TO PARAGRAPHS. Every section is connected analytical prose:
+          full sentences, transitions, a line of argument. A section that consists
+          only of a bullet list is unfinished.
+        - EXPLAIN, DON'T JUST STATE. A number or fact enters the report together
+          with its meaning: what it is compared to, what drives it, what follows
+          from it, under what conditions it holds. «Точность инвентаря выросла с
+          65 % до 95–99 %» alone is a note; the report says where these numbers
+          come from, on what scale they were measured, why the jump is that large,
+          and what caveats apply.
+        - MECHANISMS AND CAUSES. Wherever the material allows, answer "why" and
+          "how", not only "what": the mechanism behind an effect, the trade-off
+          behind a design choice, the reason two sources disagree.
+        - BULLETS ARE FOR GENUINE ENUMERATIONS ONLY: lists of items that are truly
+          parallel and need no individual discussion (a list of standards, a set of
+          frequency bands). Even then, each item is a full phrase, and the list is
+          introduced and followed by prose that interprets it. Never use bullets to
+          avoid writing sentences.
+        - NO ORPHAN KEYWORDS. Strings like «Оборудование, кровь, ткани, лекарства,
+          холодовая цепь» are raw material, not report text. Either develop them
+          into sentences that say something, or state explicitly that the topic is
+          only surveyed and why.
+        - EVERY SECTION ANSWERS A QUESTION. Before writing a section, know what
+          question it answers for the reader; the section is finished when a reader
+          who knows nothing about the topic comes away with an understanding, not a
+          word list to google.
+        - DENSITY OVER LENGTH. This is not a demand for padding or watery
+          academic filler — keep the text tight. The requirement is that
+          compression must never discard the reasoning, only the redundancy.
+
+      ═══════════════════════════════════════════════
+      LANGUAGE AND TERMINOLOGY OF THE REPORT
+      ═══════════════════════════════════════════════
+      The report is in Russian. Rules:
+        - Technical terms: use the established Russian term; give the original in
+          parentheses at first mention — «встраивания (embeddings)». If no settled
+          Russian term exists, keep the original and gloss it once.
+        - Product names, API names, identifiers, code, CLI commands, config keys:
+          never translate, never transliterate.
+        - Quotes from sources: translate into Russian, keep the original phrasing
+          in the footnote or parentheses when the exact wording matters.
+        - Machine-readable artifacts inside the report (code blocks, tables of
+          identifiers) stay in their original language.
+
+      ═══════════════════════════════════════════════
+      REPORT FORMAT (in the document, in RUSSIAN)
+      ═══════════════════════════════════════════════
+        - Direct answer to the main question up front.
+        - Detailed breakdown by subsections.
+        - «Смежное и неочевидное» — useful things found next to the scope.
+        - «Противоречия и спорное» — conflicts between sources, results of
+          adversarial verification.
+        - «Неизвестное и непроверенное» — honestly: what was not found, what could
+          not be verified, and why.
+        - Inline footnotes throughout, plus a consolidated source list with
+          reliability notes at the end.
+
+      ═══════════════════════════════════════════════
+      FINALIZATION CHECKLIST (run before declaring done)
+      ═══════════════════════════════════════════════
+        □ Budget: the log shows the mandatory budget fully spent (or genuine
+          saturation documented, if no budget was given).
+        □ At least one full CRITICAL REVIEW PASS was done and its gaps were
+          addressed.
+        □ Every non-trivial claim has an inline `^[...]` footnote; no claim rests
+          solely on a snippet or a tier-6 source.
+        □ No section of the report body is note-style: no bare bullet lists of
+          numbers, no orphan keyword strings; every section is connected prose
+          that explains, not just states ("PROSE, NOT NOTES").
+        □ Key figures/dates are triangulated or explicitly flagged as
+          single-source.
+        □ The direct answer at the top matches the body of the report.
+        □ «Неизвестное» is honestly filled — not empty by omission.
+        □ Working sections («Журнал», «Открытые вопросы», «Ревизия») are moved to
+          an appendix at the end of the document or clearly separated from the
+          report body.
+      Be honest about gaps. If you couldn't find something, say so — don't disguise
+      a guess as a fact.
     autoStart: false
     launchMessage: null

agent-roles-catalog/index.yaml

@@ -33,4 +33,4 @@ bundles:
       - en
     roles:
       - slug: researcher
-        version: 4
+        version: 8

agent-roles-catalog/scripts/content-hashes.json

@@ -16,8 +16,8 @@
     "hash": "cef39fed321779631ddd1077fcba53399adf0e48b301df281c71eb042610900d"
   },
   "researcher": {
-    "version": 4,
-    "hash": "9446ec6d2c8a6ec548358537ac392b8bf9b4d2a832ebb105d5514eac2c76da74"
+    "version": 8,
+    "hash": "0e76efa180c3e443c8856b8787e9643923d10486b373ce078c12dc16eb04611b"
   },
   "structural-editor": {
     "version": 4,

apps/server/src/collaboration/collaboration.gateway.ts

@@ -1,4 +1,9 @@
-import { Hocuspocus } from '@hocuspocus/server';
+import {
+  connectedPayload,
+  Extension,
+  Hocuspocus,
+  onConnectPayload,
+} from '@hocuspocus/server';
 import { IncomingMessage } from 'http';
 import WebSocket from 'ws';
 import { AuthenticationExtension } from './extensions/authentication.extension';
@@ -25,6 +30,56 @@ import {
   CollaborationHandler,
   CollabEventHandlers,
 } from './collaboration.handler';
+import {
+  incDocLoad,
+  incDocUnload,
+  isMetricsEnabled,
+  observeCollabConnect,
+  registerDocsOpenSource,
+} from '../integrations/metrics/metrics.registry';
+
+/**
+ * #402 — collab lifecycle metrics as a lightweight hocuspocus extension.
+ *
+ * - afterLoadDocument / afterUnloadDocument (fire once PER DOCUMENT) drive the
+ *   doc load/unload counters.
+ * - collab_connect_duration_seconds: I time the onConnect→connected hook pair,
+ *   i.e. connection ACCEPTANCE (which includes the auth handshake). This is the
+ *   cleanest per-connection correlation hocuspocus exposes: both payloads carry
+ *   the SAME `request` IncomingMessage object, so a WeakMap keyed on it gives a
+ *   per-connection start with NO leak (the entry is GC'd with the request if a
+ *   connection is rejected in onAuthenticate and `connected` never fires).
+ *   I deliberately do NOT observe at afterLoadDocument: that hook fires per
+ *   DOCUMENT, not per connection, so a second client joining an already-open
+ *   doc would be missed. auth/load latencies are their own separate metrics.
+ *
+ * All helpers are no-ops when METRICS_PORT is unset; these hooks are per
+ * connect/load/unload (never per message), so there is no hot-path cost.
+ */
+class CollabMetricsExtension implements Extension {
+  // Keyed by the per-connection request object → connect start time (ms).
+  private readonly connectStarts = new WeakMap<object, number>();
+
+  async onConnect(data: onConnectPayload) {
+    this.connectStarts.set(data.request, performance.now());
+  }
+
+  async connected(data: connectedPayload) {
+    const start = this.connectStarts.get(data.request);
+    if (start !== undefined) {
+      observeCollabConnect((performance.now() - start) / 1000);
+      this.connectStarts.delete(data.request);
+    }
+  }
+
+  async afterLoadDocument() {
+    incDocLoad();
+  }
+
+  async afterUnloadDocument() {
+    incDocUnload();
+  }
+}
 
 @Injectable()
 export class CollaborationGateway {
@@ -58,9 +113,18 @@ export class CollaborationGateway {
         this.authenticationExtension,
         this.persistenceExtension,
         this.loggerExtension,
+        // #402 collab lifecycle + connect-duration metrics (no-op when off).
+        new CollabMetricsExtension(),
       ],
     });
 
+    // #402 — read-on-scrape source for collab_docs_open. Wire ONCE, gated, so
+    // nothing runs when metrics are disabled. The gauge's collect() pulls the
+    // live count from the hocuspocus instance on each scrape (no inc/dec drift).
+    if (isMetricsEnabled()) {
+      registerDocsOpenSource(() => this.hocuspocus.getDocumentsCount());
+    }
+
     if (this.withRedis) {
       this.redisClient = new RedisClient({
         host: this.redisConfig.host,

apps/server/src/collaboration/extensions/authentication.extension.ts

@@ -16,6 +16,7 @@ import { isUserDisabled } from '../../common/helpers';
 import { getPageId } from '../collaboration.util';
 import { JwtCollabPayload, JwtType } from '../../core/auth/dto/jwt-payload';
 import { resolveProvenance } from '../../common/decorators/auth-provenance.decorator';
+import { observeCollabAuth } from '../../integrations/metrics/metrics.registry';
 
 @Injectable()
 export class AuthenticationExtension implements Extension {
@@ -30,6 +31,18 @@ export class AuthenticationExtension implements Extension {
   ) {}
 
   async onAuthenticate(data: onAuthenticatePayload) {
+    // #402 — time the whole auth (verify + user/page/permission lookups) into
+    // collab_auth_duration_seconds. finally so failed auths are timed too.
+    // No-op when METRICS_PORT is unset. Behavior unchanged.
+    const start = performance.now();
+    try {
+      return await this.doAuthenticate(data);
+    } finally {
+      observeCollabAuth((performance.now() - start) / 1000);
+    }
+  }
+
+  private async doAuthenticate(data: onAuthenticatePayload) {
     const { documentName, token } = data;
     const pageId = getPageId(documentName);
 

apps/server/src/collaboration/extensions/persistence.extension.ts

@@ -41,7 +41,10 @@ import {
   HISTORY_INTERVAL,
 } from '../constants';
 import { TransclusionService } from '../../core/page/transclusion/transclusion.service';
-import { observeCollabStore } from '../../integrations/metrics/metrics.registry';
+import {
+  observeCollabLoad,
+  observeCollabStore,
+} from '../../integrations/metrics/metrics.registry';
 
 /**
  * #251 — wire format of the client→server stateless message that signals a
@@ -150,10 +153,14 @@ export class PersistenceExtension implements Extension {
     const { documentName, document } = data;
     const pageId = getPageId(documentName);
 
+    // #402 — the early return below (live doc already non-empty) does NOT touch
+    // the DB, so it is deliberately NOT timed. We only observe the real DB-load
+    // work, and only on each real-load return, tagged by the loaded doc size.
     if (!document.isEmpty('default')) {
       return;
     }
 
+    const startedAt = performance.now();
     const page = await this.pageRepo.findById(pageId, {
       includeContent: true,
       includeYdoc: true,
@@ -171,6 +178,7 @@ export class PersistenceExtension implements Extension {
       const dbState = new Uint8Array(page.ydoc);
 
       Y.applyUpdate(doc, dbState);
+      observeCollabLoad(dbState.length, (performance.now() - startedAt) / 1000);
       return doc;
     }
 
@@ -184,26 +192,42 @@ export class PersistenceExtension implements Extension {
         tiptapExtensions,
       );
 
-      Y.encodeStateAsUpdate(ydoc);
+      // Reuse this single encode for the size label (do NOT add a second one).
+      const encoded = Y.encodeStateAsUpdate(ydoc);
+      observeCollabLoad(
+        encoded.byteLength,
+        (performance.now() - startedAt) / 1000,
+      );
       return ydoc;
     }
 
     this.logger.debug(`creating fresh ydoc: ${pageId}`);
+    observeCollabLoad(0, (performance.now() - startedAt) / 1000);
     return new Y.Doc();
   }
 
   async onStoreDocument(data: onStoreDocumentPayload) {
     // #355 — time the full store (persist + post-store side effects) into
-    // collab_store_duration_seconds. No-op when METRICS_PORT is unset.
+    // collab_store_duration_seconds. #402 — also tag by document size bucket.
+    // No-op when METRICS_PORT is unset.
     const startedAt = performance.now();
+    // Default 0 so a throw before storeDocument returns still records a
+    // (smallest-bucket) observation rather than dropping the timing entirely.
+    let bytes = 0;
     try {
-      await this.storeDocument(data);
+      bytes = await this.storeDocument(data);
     } finally {
-      observeCollabStore((performance.now() - startedAt) / 1000);
+      observeCollabStore(bytes, (performance.now() - startedAt) / 1000);
     }
   }
 
-  private async storeDocument(data: onStoreDocumentPayload) {
+  /**
+   * Persist the document. Returns the serialized ydoc byte size (used as the
+   * store histogram's size_bucket). The single Y.encodeStateAsUpdate below is
+   * the ONLY serialization — its byteLength is reused for the label (no second
+   * encode).
+   */
+  private async storeDocument(data: onStoreDocumentPayload): Promise<number> {
     const { documentName, document, context } = data;
 
     const pageId = getPageId(documentName);
@@ -455,6 +479,11 @@ export class PersistenceExtension implements Extension {
 
       await this.enqueuePageHistory(page, lastUpdatedSource);
     }
+
+    // #402 — report the serialized size for the store histogram's size_bucket.
+    // ydocState is always computed above (there is no earlier no-write return in
+    // this method), so this reflects the doc that was serialized this store.
+    return ydocState.byteLength;
   }
 
   /**

apps/server/src/core/ai-chat/ai-chat.service.setup-abort.spec.ts

@@ -0,0 +1,295 @@
+import { Logger } from '@nestjs/common';
+
+// Mock the AI SDK: the turn we drive is STOPPED during the pre-streamText setup
+// phase, so no provider call must ever be made. convertToModelMessages is reached
+// (before toolsFor) so it is stubbed to an empty transcript.
+jest.mock('ai', () => ({
+  streamText: jest.fn(),
+  generateText: jest.fn(),
+  convertToModelMessages: jest.fn(async () => []),
+  stepCountIs: jest.fn(() => () => false),
+}));
+
+import { streamText } from 'ai';
+import { AiChatService } from './ai-chat.service';
+
+/**
+ * D2 — an explicit Stop DURING the external-MCP toolset build (the pre-streamText
+ * setup phase) must:
+ *   (a) unwedge the turn (stream() rejects instead of hanging at step 0), and
+ *   (b) finalize the run as 'aborted' via the outer catch's onSettled — never leak
+ *       the run row as 'running' (which would 409 every later turn in this chat).
+ *
+ * The setup phase does NOT yet observe streamText's terminal callbacks, so before
+ * the fix a hung `toolsFor` ignored the run's abort signal and never finalized.
+ * `raceAgainstAbortAndTimeout(toolsFor, effectiveSignal, ...)` now rejects the
+ * moment the run's signal aborts; the catch re-throws (signal aborted), and the
+ * outer catch settles the run 'aborted'.
+ */
+describe('AiChatService.stream — abort during external-MCP setup finalizes the run (D2)', () => {
+  const streamTextMock = streamText as unknown as jest.Mock;
+
+  function makeService(mcpClients: { toolsFor: jest.Mock }) {
+    const aiChatRepo = {
+      findById: jest.fn(async () => ({ id: 'chat-1', workspaceId: 'ws-1' })),
+      insert: jest.fn(),
+    };
+    const aiChatMessageRepo = {
+      insert: jest.fn(async () => ({ id: 'msg-1' })),
+      findAllByChat: jest.fn(async () => []),
+      update: jest.fn(async () => ({ id: 'msg-1' })),
+    };
+    const aiSettings = { resolve: jest.fn(async () => ({})) };
+    const tools = { forUser: jest.fn(async () => ({})) };
+    const svc = new AiChatService(
+      {} as never, // ai
+      aiChatRepo as never,
+      aiChatMessageRepo as never,
+      {} as never, // aiChatPageSnapshotRepo
+      aiSettings as never,
+      tools as never,
+      mcpClients as never,
+      {} as never, // aiAgentRoleRepo
+      {} as never, // pageRepo (openPage undefined -> never touched)
+      {} as never, // pageAccess
+      { isAiChatDeferredToolsEnabled: () => false } as never, // environment
+    );
+    return { svc, tools };
+  }
+
+  const body = {
+    chatId: 'chat-1',
+    messages: [
+      { id: 'm1', role: 'user', parts: [{ type: 'text', text: 'hi' }] },
+    ],
+  };
+
+  // A minimal raw ServerResponse stand-in for the turns that PROCEED past setup
+  // and reach streamText (the deadline + legacy paths). The setup-only abort test
+  // never wires the stream, so it keeps using `{ raw: {} }`.
+  function makeRawRes() {
+    return {
+      raw: {
+        writeHead: jest.fn(function writeHead(this: unknown) {
+          return this;
+        }),
+        write: jest.fn(),
+        once: jest.fn(),
+        flushHeaders: jest.fn(),
+      },
+    };
+  }
+
+  // A fake streamText result: the service only calls consumeStream() and
+  // pipeUIMessageStreamToResponse() on it (both fire-and-forget). Its terminal
+  // callbacks are never invoked, so the run is not finalized through them.
+  function makeStreamResult() {
+    return {
+      consumeStream: jest.fn(),
+      pipeUIMessageStreamToResponse: jest.fn(),
+    };
+  }
+
+  beforeEach(() => {
+    streamTextMock.mockReset();
+    jest.spyOn(Logger.prototype, 'log').mockImplementation(() => undefined as never);
+    jest.spyOn(Logger.prototype, 'warn').mockImplementation(() => undefined as never);
+    jest.spyOn(Logger.prototype, 'error').mockImplementation(() => undefined as never);
+  });
+
+  afterEach(() => {
+    jest.restoreAllMocks();
+    jest.useRealTimers();
+  });
+
+  it('stops the hung toolset build, rejects, and settles the run "aborted" — never reaching streamText', async () => {
+    const runController = new AbortController();
+    // The build hangs (never resolves); the run is STOPPED mid-build. Aborting on a
+    // macrotask exercises the abort-listener path (a real user Stop during setup).
+    const toolsFor = jest.fn(() => {
+      setTimeout(() => runController.abort(new Error('user stop')), 0);
+      return new Promise(() => {}); // never settles — models a hung MCP build
+    });
+    const { svc } = makeService({ toolsFor });
+
+    const onSettled = jest.fn();
+    const begin = jest.fn(async () => ({
+      runId: 'run-1',
+      signal: runController.signal,
+    }));
+
+    const promise = svc.stream({
+      user: { id: 'user-1' } as never,
+      workspace: { id: 'ws-1' } as never,
+      sessionId: 'sess-1',
+      body: body as never,
+      res: { raw: {} } as never,
+      signal: new AbortController().signal, // socket signal (distinct from the run)
+      model: {} as never,
+      role: null,
+      runHooks: {
+        begin,
+        onAssistantSeeded: jest.fn(),
+        onStep: jest.fn(),
+        onSettled,
+      } as never,
+    });
+
+    // (a) The turn is UNWEDGED: it rejects (with the stop reason) instead of hanging.
+    await expect(promise).rejects.toThrow('user stop');
+
+    // (b) The run is finalized as 'aborted' with NO error message (a Stop, not a
+    // failure) — so the run row never leaks 'running'.
+    expect(onSettled).toHaveBeenCalledTimes(1);
+    expect(onSettled).toHaveBeenCalledWith('run-1', 'aborted', undefined);
+
+    // The build was reached, but the provider call was NEVER made (stopped at setup).
+    expect(toolsFor).toHaveBeenCalledTimes(1);
+    expect(streamTextMock).not.toHaveBeenCalled();
+  });
+
+  // Item 1 — the onLateResolve leg of raceAgainstAbortAndTimeout. When `toolsFor`
+  // loses the race (abort) but RESOLVES LATER with a leased toolset, the setup site
+  // must release that abandoned toolset's leases (call close() on its client
+  // handles) so their lease refcount is not pinned forever by a toolset nobody
+  // consumes. Nothing else exercises this path.
+  it('releases the leases of a toolset that resolves AFTER the race was already lost (onLateResolve)', async () => {
+    const runController = new AbortController();
+    // A controllable build: it hangs until we resolve it by hand, and the run is
+    // stopped mid-build so the race rejects BEFORE the build settles.
+    let resolveTools: (v: unknown) => void = () => undefined;
+    const toolsForPromise = new Promise((resolve) => {
+      resolveTools = resolve;
+    });
+    const toolsFor = jest.fn(() => {
+      setTimeout(() => runController.abort(new Error('user stop')), 0);
+      return toolsForPromise;
+    });
+    const { svc } = makeService({ toolsFor });
+
+    const begin = jest.fn(async () => ({
+      runId: 'run-1',
+      signal: runController.signal,
+    }));
+
+    const promise = svc.stream({
+      user: { id: 'user-1' } as never,
+      workspace: { id: 'ws-1' } as never,
+      sessionId: 'sess-1',
+      body: body as never,
+      res: { raw: {} } as never,
+      signal: new AbortController().signal,
+      model: {} as never,
+      role: null,
+      runHooks: {
+        begin,
+        onAssistantSeeded: jest.fn(),
+        onStep: jest.fn(),
+        onSettled: jest.fn(),
+      } as never,
+    });
+
+    // The race is lost to the abort: the turn rejects with the stop reason.
+    await expect(promise).rejects.toThrow('user stop');
+
+    // NOW the abandoned build resolves late with a leased client. onLateResolve must
+    // release it (call close on the lease handle).
+    const close = jest.fn().mockResolvedValue(undefined);
+    resolveTools({
+      tools: {},
+      clients: [{ close }],
+      outcomes: [],
+      instructions: [],
+    });
+    // Flush the microtasks so work.then -> onLateResolve -> Promise.all(close) runs.
+    await new Promise((r) => setImmediate(r));
+
+    expect(close).toHaveBeenCalledTimes(1);
+  });
+
+  // Item 2 — the PURE DEADLINE branch (MCP_TOOLSET_BUILD_DEADLINE_MS). `toolsFor`
+  // never settles and the run's signal is NOT aborted: the race rejects with a
+  // "setup timed out" error, the catch does NOT re-throw (runId set but signal not
+  // aborted), and the turn PROCEEDS Docmost-only. It must reach streamText (the turn
+  // continues, not wedged) and must NOT be finalized 'aborted'.
+  it('proceeds Docmost-only (reaches streamText) when the build hits the deadline without an abort', async () => {
+    jest.useFakeTimers();
+
+    // The build hangs forever; the run's signal is never aborted.
+    const toolsFor = jest.fn(() => new Promise(() => {}));
+    const { svc } = makeService({ toolsFor });
+
+    streamTextMock.mockReturnValue(makeStreamResult() as never);
+
+    const onSettled = jest.fn();
+    const runSignal = new AbortController().signal; // never aborts
+    const begin = jest.fn(async () => ({ runId: 'run-1', signal: runSignal }));
+
+    const promise = svc.stream({
+      user: { id: 'user-1' } as never,
+      workspace: { id: 'ws-1' } as never,
+      sessionId: 'sess-1',
+      body: body as never,
+      res: makeRawRes() as never,
+      signal: new AbortController().signal,
+      model: {} as never,
+      role: null,
+      runHooks: {
+        begin,
+        onAssistantSeeded: jest.fn(),
+        onStep: jest.fn(),
+        onSettled,
+      } as never,
+    });
+
+    // Advance past the 60s build deadline; advanceTimersByTimeAsync flushes the
+    // promise microtasks between timer fires so the whole setup chain runs.
+    await jest.advanceTimersByTimeAsync(60_001);
+    // The turn does not throw out of setup — it continues to stream.
+    await expect(promise).resolves.toBeUndefined();
+
+    // The turn CONTINUED: streamText was reached (Docmost-only), not wedged.
+    expect(toolsFor).toHaveBeenCalledTimes(1);
+    expect(streamTextMock).toHaveBeenCalledTimes(1);
+    // The run was NOT finalized as aborted (the deadline is not a Stop) — the setup
+    // catch settle path never ran, so onSettled is left to streamText's callbacks.
+    expect(onSettled).not.toHaveBeenCalled();
+  });
+
+  // Item 3 — the LEGACY no-runId path. The catch's re-throw is gated on
+  // `runId && effectiveSignal.aborted`. With NO runId (no runHooks) an abort during
+  // setup must NOT re-throw (runId falsy) — the turn warns + proceeds Docmost-only
+  // and streams, and is never finalized 'aborted' via the re-throw. Locks the
+  // `runId &&` half of the guard.
+  it('does NOT re-throw on a setup abort when there is no runId (legacy path proceeds Docmost-only)', async () => {
+    const socketController = new AbortController();
+    // The build hangs; the SOCKET signal (legacy effectiveSignal) aborts mid-build.
+    const toolsFor = jest.fn(() => {
+      setTimeout(() => socketController.abort(new Error('socket closed')), 0);
+      return new Promise(() => {});
+    });
+    const { svc } = makeService({ toolsFor });
+
+    streamTextMock.mockReturnValue(makeStreamResult() as never);
+
+    // No runHooks => runId undefined, effectiveSignal === the socket signal.
+    const promise = svc.stream({
+      user: { id: 'user-1' } as never,
+      workspace: { id: 'ws-1' } as never,
+      sessionId: 'sess-1',
+      body: body as never,
+      res: makeRawRes() as never,
+      signal: socketController.signal,
+      model: {} as never,
+      role: null,
+    });
+
+    // The turn does NOT reject out of setup (no re-throw on the legacy path).
+    await expect(promise).resolves.toBeUndefined();
+
+    // It proceeded Docmost-only and reached streamText — streamText then observes
+    // the already-aborted socket signal via its own abortSignal.
+    expect(toolsFor).toHaveBeenCalledTimes(1);
+    expect(streamTextMock).toHaveBeenCalledTimes(1);
+  });
+});

apps/server/src/core/ai-chat/ai-chat.service.spec.ts

@@ -16,6 +16,7 @@ import {
   rowToUiMessage,
   prepareAgentStep,
   flushAssistant,
+  stripNulChars,
   chatStreamMetadata,
   accumulateStepUsage,
   isInterruptResume,
@@ -28,11 +29,13 @@ import { buildSystemPrompt } from './ai-chat.prompt';
 import type { McpClientsService } from './external-mcp/mcp-clients.service';
 
 /**
- * Unit tests for compactToolOutput: the pure helper that shrinks LARGE tool
- * outputs before they are persisted (and re-sent to the provider on later
- * turns). The contract is: small outputs pass through unchanged (by identity);
- * large outputs keep their shape and small scalar fields (id/title/pageId — the
- * client reads these to render citations) while big payloads are truncated.
+ * Unit tests for compactToolOutput: the pure helper that shrinks tool outputs
+ * before they are persisted (and re-sent to the provider on later turns). The
+ * contract is: small and normal outputs — including whole page reads (tens of
+ * KB) — pass through unchanged (by identity); only an output above the high
+ * safety cap (> 200 KB) is compacted, and even then it keeps its shape and
+ * small scalar fields (id/title/pageId — the client reads these to render
+ * citations) while the big payloads are reduced.
  */
 describe('compactToolOutput', () => {
   it('returns a small object unchanged (by identity)', () => {
@@ -41,7 +44,7 @@ describe('compactToolOutput', () => {
   });
 
   it('truncates a large getPage-shaped markdown body but keeps the title', () => {
-    const big = 'x'.repeat(20000);
+    const big = 'x'.repeat(300000);
     const result = compactToolOutput({ title: 'T', markdown: big }) as {
       title: string;
       markdown: string;
@@ -49,15 +52,16 @@ describe('compactToolOutput', () => {
     // Shallow scalar field is preserved (citations depend on it).
     expect(result.title).toBe('T');
     // The big payload is shrunk far below the original size.
-    expect(result.markdown.length).toBeLessThan(20000);
-    expect(result.markdown).toContain('[truncated');
+    expect(result.markdown.length).toBeLessThan(300000);
+    expect(result.markdown).toContain('omitted from stored chat history');
   });
 
   it('caps a long array and appends a single truncation marker', () => {
-    // 200 small objects, each padded so the total serialized size > 4000 bytes.
+    // 200 objects, each padded so the total serialized size
+    // > 200000 bytes (the new safety cap).
     const long = Array.from({ length: 200 }, (_, i) => ({
       id: 'n' + i,
-      pad: 'y'.repeat(40),
+      pad: 'y'.repeat(1200),
     }));
     const result = compactToolOutput(long) as Array<Record<string, unknown>>;
     // 50 kept + 1 marker.
@@ -75,8 +79,8 @@ describe('compactToolOutput', () => {
 
   it('replaces a subtree beyond the depth cap with a marker', () => {
     // Build a deeply nested object (> TOOL_OUTPUT_MAX_DEPTH levels) with a big
-    // string at the bottom so the total serialized size exceeds the threshold.
-    let nested: Record<string, unknown> = { leaf: 'z'.repeat(8000) };
+    // string at the bottom so the total serialized size exceeds the 200 KB cap.
+    let nested: Record<string, unknown> = { leaf: 'z'.repeat(250000) };
     for (let i = 0; i < 20; i++) {
       nested = { child: nested };
     }
@@ -85,7 +89,7 @@ describe('compactToolOutput', () => {
   });
 
   it('produces a much smaller JSON than the original for a large input', () => {
-    const big = 'x'.repeat(20000);
+    const big = 'x'.repeat(300000);
     const original = { title: 'T', markdown: big };
     const result = compactToolOutput(original);
     const originalBytes = Buffer.byteLength(JSON.stringify(original), 'utf8');
@@ -448,6 +452,45 @@ describe('flushAssistant', () => {
   });
 });
 
+/**
+ * stripNulChars: a NUL (U+0000) is rejected by Postgres in BOTH the `content`
+ * (text) and `toolCalls`/`metadata` (jsonb) columns, so it must be stripped from
+ * every persisted string. String.fromCharCode(0) avoids embedding a raw NUL byte
+ * in this source file.
+ */
+describe('stripNulChars', () => {
+  const NUL = String.fromCharCode(0);
+
+  it('deep-strips NUL from strings in nested objects/arrays', () => {
+    const out = stripNulChars({
+      content: `a${NUL}b`,
+      parts: [{ type: 'text', text: `x${NUL}${NUL}y` }],
+      nested: [`p${NUL}q`, 42, null],
+    });
+    expect(out.content).toBe('ab');
+    expect((out.parts[0] as { text: string }).text).toBe('xy');
+    expect(out.nested[0]).toBe('pq');
+    expect(out.nested[1]).toBe(42);
+    expect(out.nested[2]).toBeNull();
+    expect(JSON.stringify(out).includes(NUL)).toBe(false);
+  });
+
+  it('returns the SAME reference when there is no NUL (no needless clone)', () => {
+    const input = { a: 'clean', b: [1, 2, { c: 'ok' }] };
+    expect(stripNulChars(input)).toBe(input);
+  });
+
+  it('flushAssistant produces a NUL-free row even when the turn text carries one', () => {
+    const f = flushAssistant([], `partial${NUL}answer`, 'error', {
+      error: `bo${NUL}om`,
+    });
+    expect(f.content).toBe('partialanswer');
+    const serialized =
+      f.content + JSON.stringify(f.toolCalls) + JSON.stringify(f.metadata);
+    expect(serialized.includes(NUL)).toBe(false);
+  });
+});
+
 /**
  * chatStreamMetadata: attach metadata to the streamed assistant UI message per
  * part type — `chatId` on `start` (so the client adopts the real created chat id

apps/server/src/core/ai-chat/ai-chat.service.ts

@@ -58,6 +58,19 @@ import {
 // multi-search research questions are not cut off mid-investigation.
 const MAX_AGENT_STEPS = 20;
 
+// Wall-clock ceiling for building the external MCP toolset during the per-turn
+// setup phase (before streamText owns the lifecycle). Defense-in-depth ABOVE the
+// per-server connect bound in mcp-clients.service (CONNECT_TIMEOUT_MS): even if
+// that per-server timeout regressed, this outer deadline — together with the run's
+// abort signal — guarantees the setup phase can never wedge a turn at step 0 (the
+// production hang) and the run always finalizes. It stays a TRUE backstop because
+// buildEntry connects to the servers CONCURRENTLY, so the total build time is
+// bounded by the SLOWEST single server (~2×CONNECT_TIMEOUT_MS), not the SUM across
+// them — the per-server bound fires first no matter how many servers are enabled,
+// and this outer deadline only catches a total build stall the per-server bound
+// somehow missed.
+const MCP_TOOLSET_BUILD_DEADLINE_MS = 60_000;
+
 // System-prompt addendum injected ONLY on the final step (see prepareAgentStep).
 // It forbids further tool calls and tells the model to synthesize the best
 // answer it can from what it already gathered, so a tool-heavy turn never ends
@@ -176,6 +189,78 @@ export function sameInstant(
   return ta === tb;
 }
 
+/**
+ * Race `work` against an abort signal AND a wall-clock deadline, so a hung
+ * external-MCP toolset build during the pre-streamText setup phase can NEITHER
+ * wedge the turn NOR make it un-stoppable, and the run always finalizes. It
+ *  - resolves with `work`'s value when it settles first;
+ *  - REJECTS EARLY if `signal` aborts (with `signal.reason` when that is an Error,
+ *    else a generic `Error('aborted')`) — so an explicit Stop is honored mid-setup;
+ *  - REJECTS EARLY if `deadlineMs` elapses (defense-in-depth backstop);
+ *  - invokes `onLateResolve(value)` when `work` settles AFTER the race was already
+ *    lost, so the caller can release any resources that abandoned value owns
+ *    (e.g. close leased MCP clients that would otherwise leak their sockets).
+ *
+ * A rejection handler is attached to `work` so a late rejection is never an
+ * unhandledRejection; the timer is unref'd and cleared once the race settles.
+ */
+export function raceAgainstAbortAndTimeout<T>(
+  work: Promise<T>,
+  signal: AbortSignal,
+  deadlineMs: number,
+  onLateResolve?: (value: T) => void,
+): Promise<T> {
+  return new Promise<T>((resolve, reject) => {
+    let settled = false;
+    const cleanup = () => {
+      clearTimeout(timer);
+      signal.removeEventListener('abort', onAbort);
+    };
+    const onAbort = () => {
+      if (settled) return;
+      settled = true;
+      cleanup();
+      reject(
+        signal.reason instanceof Error ? signal.reason : new Error('aborted'),
+      );
+    };
+    const timer = setTimeout(() => {
+      if (settled) return;
+      settled = true;
+      cleanup();
+      reject(new Error(`setup timed out after ${deadlineMs}ms`));
+    }, deadlineMs);
+    // Do not keep the process alive just for this setup-deadline timer.
+    timer.unref?.();
+    if (signal.aborted) {
+      onAbort();
+    } else {
+      signal.addEventListener('abort', onAbort, { once: true });
+    }
+    work.then(
+      (value) => {
+        if (settled) {
+          // The race was already lost (abort/deadline): hand the abandoned value to
+          // the caller so it can release the resources that value owns.
+          onLateResolve?.(value);
+          return;
+        }
+        settled = true;
+        cleanup();
+        resolve(value);
+      },
+      (err: unknown) => {
+        // A late rejection after the race is already handled — swallow so it is
+        // never an unhandledRejection.
+        if (settled) return;
+        settled = true;
+        cleanup();
+        reject(err instanceof Error ? err : new Error(String(err)));
+      },
+    );
+  });
+}
+
 /**
  * Payload accepted from the client `useChat` POST body. We do NOT bind a strict
  * DTO (the global ValidationPipe whitelist would strip the useChat-specific
@@ -704,10 +789,43 @@ export class AiChatService implements OnModuleInit {
         instructions: [],
       };
       try {
-        external = await this.mcpClients.toolsFor(workspace.id);
+        // Bound the external-MCP toolset build by BOTH the run's abort signal and
+        // a generous wall-clock deadline. This is the pre-streamText setup phase,
+        // which streamText's terminal callbacks do NOT yet govern — so without this
+        // a hung build would hang the turn at step 0 forever (the production hang),
+        // unobservant of an explicit Stop. The deadline is defense-in-depth ABOVE
+        // the per-server connect bound in mcp-clients.service. On a LATE resolve
+        // (the race was already lost) RELEASE the abandoned toolset's leases —
+        // c.close() here is the lease handle, so it decrements the cache entry's
+        // refcount; it does NOT force-close the transports (the cache OWNS the
+        // clients and closes them on TTL/evict). This just prevents the lease
+        // refcount from being pinned >=1 forever by a toolset nobody will consume.
+        external = await raceAgainstAbortAndTimeout(
+          this.mcpClients.toolsFor(workspace.id),
+          effectiveSignal,
+          MCP_TOOLSET_BUILD_DEADLINE_MS,
+          (late) => {
+            void Promise.all(
+              late.clients.map((c) => c.close().catch(() => undefined)),
+            );
+          },
+        );
       } catch (err) {
-        // Building the external toolset must never break the turn; proceed with
-        // Docmost-only tools. Never log URLs/headers — short message only.
+        // An explicit Stop reached the RUN's signal DURING setup: re-throw so the
+        // outer catch finalizes the run as aborted — never swallow a Stop. Gated on
+        // `runId`: the re-throw exists ONLY to finalize the run, which exists only
+        // in autonomous mode. On the legacy path (no runId) `effectiveSignal` is the
+        // SOCKET signal (it aborts on a client disconnect); re-throwing there would
+        // change prior behavior and make the controller write JSON to an already-
+        // closed socket (it only attaches res.raw.on('error') in autonomous mode).
+        // So legacy keeps its prior behavior — warn + proceed, and streamText then
+        // observes the aborted socket signal.
+        if (runId && effectiveSignal.aborted) {
+          throw err;
+        }
+        // Otherwise a down/slow server (build timeout or other fault) must never
+        // break the turn: proceed with Docmost-only tools. Never log URLs/headers —
+        // short message only.
         this.logger.warn(
           `External MCP toolset unavailable: ${
             err instanceof Error ? err.message : 'unknown error'
@@ -1307,12 +1425,19 @@ export class AiChatService implements OnModuleInit {
         if (this.environment?.isAiChatResumableStreamEnabled?.()) {
           this.streamRegistry?.abortEntry(chatId, runId);
         }
+        // Distinguish an explicit Stop (the run's signal aborted during setup) from
+        // a real failure, so the run settles with the correct terminal status
+        // instead of always 'error'. onSettled/finalizeRun is idempotent, so this
+        // is safe even if a streamText callback also settles the run.
+        const settleStatus = effectiveSignal.aborted ? 'aborted' : 'error';
         await runHooks?.onSettled?.(
           runId,
-          'error',
-          err instanceof Error
-            ? err.message
-            : 'Agent run failed before streaming started',
+          settleStatus,
+          settleStatus === 'aborted'
+            ? undefined
+            : err instanceof Error
+              ? err.message
+              : 'Agent run failed before streaming started',
         );
       }
       throw err;
@@ -1517,16 +1642,25 @@ type StepLike = {
 /**
  * Compaction tunables for persisted tool OUTPUTS. Read tools (getPage,
  * getPageJson, getNode, diffPageVersions, exportPageMarkdown, ...) return whole
- * pages with no size cap. Their outputs are stored in `metadata.parts` and
- * RE-SENT to the provider on every later turn via convertToModelMessages, so an
- * uncompacted large body grows token cost, latency, and DB row size on every
- * turn. We shrink the big payloads while preserving the object's shape and its
- * small scalar fields (id/title/pageId) the client reads to render citations.
+ * pages. Their outputs are stored in `metadata.parts` and RE-SENT to the
+ * provider on every later turn via convertToModelMessages. We deliberately keep
+ * these outputs FULL up to a high safety cap (MAX_TOOL_OUTPUT_BYTES) so the
+ * model never sees a shortened copy of content it already fetched: an earlier
+ * 4000-byte cap shrank normal page reads (often tens of KB) to a tiny preview,
+ * and the model — seeing a truncation marker in its OWN history — re-read the
+ * same page, wasting tokens. Only a single output LARGER than the cap is
+ * compacted at all, purely as a backstop against a pathological payload; even
+ * then we preserve the object's shape and its small scalar fields
+ * (id/title/pageId) that the client reads to render citations.
  */
-// Only outputs whose JSON serialization exceeds this are compacted at all
-// (fast path: smaller outputs are returned unchanged, by identity).
-const MAX_TOOL_OUTPUT_BYTES = 4000;
-// A string longer than this is truncated to a leading preview.
+// HIGH safety backstop: only an output whose JSON serialization EXCEEDS this is
+// compacted at all. Normal reads (whole pages, tens of KB) stay well under it
+// and are stored + replayed VERBATIM (fast path: returned unchanged, by
+// identity). Only a single pathologically huge output (> 200 KB) is compacted.
+const MAX_TOOL_OUTPUT_BYTES = 200_000;
+// Inside the backstop path only (i.e. once the whole output already exceeded
+// MAX_TOOL_OUTPUT_BYTES), a string longer than this is reduced to a leading
+// preview; normal outputs never reach this branch.
 const TOOL_OUTPUT_STRING_LIMIT = 600;
 // Number of leading characters kept from a truncated string.
 const TOOL_OUTPUT_STRING_PREVIEW = 500;
@@ -1569,9 +1703,9 @@ export function compactToolOutput(output: unknown): unknown {
 function compactValue(value: unknown, depth: number): unknown {
   if (typeof value === 'string') {
     if (value.length > TOOL_OUTPUT_STRING_LIMIT) {
-      return `${value.slice(0, TOOL_OUTPUT_STRING_PREVIEW)}…[truncated ${
+      return `${value.slice(0, TOOL_OUTPUT_STRING_PREVIEW)}…[${
         value.length - TOOL_OUTPUT_STRING_PREVIEW
-      } chars]`;
+      } chars omitted from stored chat history to bound replay size — call the tool again to read the full output]`;
     }
     return value;
   }
@@ -1758,6 +1892,45 @@ export async function applyFinalize(
   });
 }
 
+/**
+ * Deep-strip NUL characters (`\u0000`) from every string in a value, returning
+ * the SAME reference when nothing changed (so the no-NUL common case allocates
+ * nothing). Postgres rejects a NUL in BOTH `text` and `jsonb` columns ("invalid
+ * input syntax for type json" / "unsupported Unicode escape sequence"), so a
+ * stray NUL in model output or a tool result — e.g. a truncated multibyte read
+ * of a web page — otherwise fails EVERY persist of the assistant row, silently
+ * dropping that turn's content from the DB while the live stream still shows it.
+ * Applied at the flushAssistant choke point so content + toolCalls + metadata are
+ * all covered. Exported for the unit test.
+ */
+export function stripNulChars<T>(value: T): T {
+  if (typeof value === 'string') {
+    return (value.includes('\u0000')
+      ? value.replace(/\u0000/g, '')
+      : value) as T;
+  }
+  if (Array.isArray(value)) {
+    let changed = false;
+    const out = value.map((v) => {
+      const s = stripNulChars(v);
+      if (s !== v) changed = true;
+      return s;
+    });
+    return (changed ? out : value) as T;
+  }
+  if (value && typeof value === 'object') {
+    let changed = false;
+    const out: Record<string, unknown> = {};
+    for (const [k, v] of Object.entries(value as Record<string, unknown>)) {
+      const s = stripNulChars(v);
+      if (s !== v) changed = true;
+      out[k] = s;
+    }
+    return (changed ? out : value) as T;
+  }
+  return value;
+}
+
 /**
  * PURE assistant-row builder (#183 step-granular durability). Given the turn's
  * accumulated steps + the in-progress (not-yet-finished) text + the lifecycle
@@ -1827,12 +2000,16 @@ export function flushAssistant(
     };
   }
 
-  return {
+  // Strip NUL chars from the whole row before persisting: Postgres rejects a NUL
+  // in both the `content` (text) and `toolCalls`/`metadata` (jsonb) columns, and a
+  // single stray NUL in model/tool output would otherwise fail EVERY write of this
+  // row and silently drop the turn's content from the DB (see stripNulChars).
+  return stripNulChars({
     content: stepsText + trailing,
     toolCalls: serializeSteps(finished),
     metadata,
     status,
-  };
+  });
 }
 
 /**

apps/server/src/core/ai-chat/external-mcp/mcp-call-timeout.spec.ts

@@ -181,25 +181,25 @@ describe('mcp timeout env helpers', () => {
     else process.env.AI_MCP_CALL_TIMEOUT_MS = ORIG_CALL;
   });
 
-  it('mcpStreamTimeoutMs defaults to 5 min and honors a positive override', () => {
+  it('mcpStreamTimeoutMs defaults to 1 min and honors a positive override', () => {
     delete process.env.AI_MCP_STREAM_TIMEOUT_MS;
-    expect(mcpStreamTimeoutMs()).toBe(300_000);
-    process.env.AI_MCP_STREAM_TIMEOUT_MS = '60000';
     expect(mcpStreamTimeoutMs()).toBe(60_000);
+    process.env.AI_MCP_STREAM_TIMEOUT_MS = '90000';
+    expect(mcpStreamTimeoutMs()).toBe(90_000);
     for (const bad of ['0', '-1', 'x', '']) {
       process.env.AI_MCP_STREAM_TIMEOUT_MS = bad;
-      expect(mcpStreamTimeoutMs()).toBe(300_000);
+      expect(mcpStreamTimeoutMs()).toBe(60_000);
     }
   });
 
-  it('mcpCallTimeoutMs defaults to 15 min and honors a positive override', () => {
+  it('mcpCallTimeoutMs defaults to 2 min and honors a positive override', () => {
     delete process.env.AI_MCP_CALL_TIMEOUT_MS;
-    expect(mcpCallTimeoutMs()).toBe(900_000);
-    process.env.AI_MCP_CALL_TIMEOUT_MS = '120000';
     expect(mcpCallTimeoutMs()).toBe(120_000);
+    process.env.AI_MCP_CALL_TIMEOUT_MS = '180000';
+    expect(mcpCallTimeoutMs()).toBe(180_000);
     for (const bad of ['0', '-1', 'x', '']) {
       process.env.AI_MCP_CALL_TIMEOUT_MS = bad;
-      expect(mcpCallTimeoutMs()).toBe(900_000);
+      expect(mcpCallTimeoutMs()).toBe(120_000);
     }
   });
 });

apps/server/src/core/ai-chat/external-mcp/mcp-clients.connect-timeout.spec.ts

@@ -0,0 +1,237 @@
+import { McpClientsService } from './mcp-clients.service';
+
+/**
+ * D1 — a HUNG MCP handshake must not POISON the per-workspace build cache.
+ *
+ * THE BUG (production hang): `createMCPClient` (inside the private `connect`) is
+ * NOT bounded by a timeout and — like @ai-sdk/mcp's tool calls — its promise does
+ * NOT settle on abort. A transient network blip mid-handshake made connect hang
+ * FOREVER. Because getOrBuildEntry caches the build PROMISE, that never-settling
+ * connect wedged EVERY later turn for the workspace (each awaited the same pending
+ * build) — step_count stuck at 0, run row leaking 'running', chat 409ing forever.
+ *
+ * THE FIX: `connectWithTimeout` races `connect` against a SETTLING timeout
+ * (CONNECT_TIMEOUT_MS). On timeout it REJECTS, so buildEntry catches it, records
+ * the server `ok:false`, and the build COMPLETES with that server skipped — the
+ * cache is never poisoned and a subsequent `toolsFor` returns instead of hanging.
+ *
+ * REACHABILITY NOTE: the smallest network-free path that exercises the fix is to
+ * spy on the private `connect` (the same harness the namespacing spec uses) —
+ * `connectWithTimeout` wraps exactly that call, so a never-resolving `connect`
+ * models a never-settling `createMCPClient` precisely, without DNS/sockets.
+ *
+ * Fake timers prove the timeout fires WITHOUT real waiting.
+ */
+
+// Mirrors the private CONNECT_TIMEOUT_MS constant in mcp-clients.service.ts.
+const CONNECT_TIMEOUT_MS = 5000;
+
+interface FakeServer {
+  id: string;
+  name: string;
+  transport: string;
+  url: string;
+  headersEnc: string | null;
+  toolAllowlist: string[] | null;
+  instructions?: string | null;
+}
+
+function server(
+  over: Partial<FakeServer> & { id: string; name: string },
+): FakeServer {
+  return {
+    transport: 'http',
+    url: 'https://example.com/mcp',
+    headersEnc: null,
+    toolAllowlist: null,
+    ...over,
+  };
+}
+
+function buildService(servers: FakeServer[]) {
+  const repoStub = { listEnabled: jest.fn().mockResolvedValue(servers) };
+  const service = new McpClientsService(repoStub as never, {} as never);
+  // Silence the expected "server unavailable" warning.
+  jest
+    .spyOn(
+      (service as unknown as { logger: { warn: (...a: unknown[]) => void } })
+        .logger,
+      'warn',
+    )
+    .mockImplementation(() => undefined);
+  return service;
+}
+
+// Spy on the private `connect` with a per-server implementation.
+function stubConnect(
+  service: McpClientsService,
+  impl: (s: FakeServer) => Promise<unknown>,
+) {
+  return jest
+    .spyOn(
+      service as unknown as { connect: (s: FakeServer) => Promise<unknown> },
+      'connect',
+    )
+    .mockImplementation(impl);
+}
+
+describe('McpClientsService.connectWithTimeout — hung connect does not poison the cache (D1)', () => {
+  beforeEach(() => jest.useFakeTimers());
+  afterEach(() => {
+    jest.clearAllTimers();
+    jest.useRealTimers();
+    jest.restoreAllMocks();
+  });
+
+  it('buildEntry completes (server recorded ok:false) when connect never settles, and toolsFor does not hang', async () => {
+    const svc = buildService([server({ id: 'id-hung', name: 'hung' })]);
+    // connect NEVER settles — models a wedged createMCPClient handshake.
+    stubConnect(svc, () => new Promise<never>(() => {}));
+
+    const toolsetPromise = svc.toolsFor('ws-1');
+    // Drive fake time past the connect bound so connectWithTimeout rejects and
+    // buildEntry catches it (records ok:false) — flushing the microtasks.
+    await jest.advanceTimersByTimeAsync(CONNECT_TIMEOUT_MS + 1);
+
+    const toolset = await toolsetPromise;
+    // The build COMPLETED with the bad server skipped (no tools, ok:false).
+    expect(Object.keys(toolset.tools)).toHaveLength(0);
+    expect(toolset.outcomes).toEqual([
+      { name: 'hung', ok: false, reason: 'MCP connect timed out after 5000ms' },
+    ]);
+    await Promise.all(toolset.clients.map((c) => c.close()));
+
+    // The cache is NOT poisoned: a subsequent turn returns (served from the warm
+    // cached entry) instead of awaiting a never-settling build.
+    const again = await svc.toolsFor('ws-1');
+    expect(Object.keys(again.tools)).toHaveLength(0);
+    await Promise.all(again.clients.map((c) => c.close()));
+  });
+
+  it('a hung server is skipped but a healthy server in the SAME build still contributes its tools', async () => {
+    const svc = buildService([
+      server({ id: 'id-hung', name: 'hung' }),
+      server({ id: 'id-ok', name: 'ok' }),
+    ]);
+    const okClient = {
+      tools: () => Promise.resolve({ search: { description: 'x' } }),
+      close: jest.fn().mockResolvedValue(undefined),
+    };
+    stubConnect(svc, (s) =>
+      s.id === 'id-hung'
+        ? new Promise<never>(() => {})
+        : Promise.resolve(okClient),
+    );
+
+    const toolsetPromise = svc.toolsFor('ws-2');
+    await jest.advanceTimersByTimeAsync(CONNECT_TIMEOUT_MS + 1);
+
+    const toolset = await toolsetPromise;
+    // Healthy server's tool survives (namespaced); hung server recorded ok:false.
+    expect(Object.keys(toolset.tools)).toEqual(['ok_search']);
+    expect(toolset.outcomes).toEqual([
+      { name: 'hung', ok: false, reason: 'MCP connect timed out after 5000ms' },
+      { name: 'ok', ok: true },
+    ]);
+    await Promise.all(toolset.clients.map((c) => c.close()));
+  });
+
+  it('closes the ORPHANED client when connect resolves LATE (after the timeout)', async () => {
+    const svc = buildService([server({ id: 'id-late', name: 'late' })]);
+    const lateClient = {
+      tools: () => Promise.resolve({}),
+      close: jest.fn().mockResolvedValue(undefined),
+    };
+    // connect resolves only AFTER the connect bound has already elapsed, so
+    // connectWithTimeout has already rejected and must close this orphan.
+    stubConnect(
+      svc,
+      () =>
+        new Promise((resolve) => {
+          setTimeout(() => resolve(lateClient), CONNECT_TIMEOUT_MS * 2);
+        }),
+    );
+
+    const toolsetPromise = svc.toolsFor('ws-3');
+    // Fire the timeout: the build completes with the server skipped.
+    await jest.advanceTimersByTimeAsync(CONNECT_TIMEOUT_MS + 1);
+    const toolset = await toolsetPromise;
+    expect(toolset.outcomes[0]?.ok).toBe(false);
+    expect(lateClient.close).not.toHaveBeenCalled();
+
+    // Now let the late connect resolve — the orphan must be closed, not leaked.
+    await jest.advanceTimersByTimeAsync(CONNECT_TIMEOUT_MS * 2);
+    expect(lateClient.close).toHaveBeenCalledTimes(1);
+
+    await Promise.all(toolset.clients.map((c) => c.close()));
+  });
+});
+
+describe('McpClientsService.buildEntry — closes a connected client whose tools() fails (leak fix)', () => {
+  beforeEach(() => jest.useFakeTimers());
+  afterEach(() => {
+    jest.clearAllTimers();
+    jest.useRealTimers();
+    jest.restoreAllMocks();
+  });
+
+  it('connect succeeds but tools() REJECTS: the client is close()d exactly once and the server is skipped, while a healthy server still contributes', async () => {
+    const svc = buildService([
+      server({ id: 'id-bad', name: 'bad' }),
+      server({ id: 'id-ok', name: 'ok' }),
+    ]);
+    // The bad server connects fine, then tools() rejects — the client would leak if
+    // buildEntry did not close it in the per-server catch (it was never registered).
+    const badClient = {
+      tools: () => Promise.reject(new Error('tools listing failed')),
+      close: jest.fn().mockResolvedValue(undefined),
+    };
+    const okClient = {
+      tools: () => Promise.resolve({ search: { description: 'x' } }),
+      close: jest.fn().mockResolvedValue(undefined),
+    };
+    stubConnect(svc, (s) =>
+      s.id === 'id-bad' ? Promise.resolve(badClient) : Promise.resolve(okClient),
+    );
+
+    const toolset = await svc.toolsFor('ws-4');
+
+    // The orphaned (never-registered) client is closed exactly once — no leak.
+    expect(badClient.close).toHaveBeenCalledTimes(1);
+    // Healthy server survives; bad server recorded ok:false and skipped.
+    expect(Object.keys(toolset.tools)).toEqual(['ok_search']);
+    expect(toolset.outcomes).toEqual([
+      { name: 'bad', ok: false, reason: 'tools listing failed' },
+      { name: 'ok', ok: true },
+    ]);
+
+    // The healthy (registered) client is NOT closed by the loop — it is owned by the
+    // cache entry and stays warm (closed only on eviction/teardown, not on lease
+    // release). Releasing the lease keeps it warm since the entry is not evicted.
+    expect(okClient.close).not.toHaveBeenCalled();
+    await Promise.all(toolset.clients.map((c) => c.close()));
+    expect(okClient.close).not.toHaveBeenCalled();
+    // The failed client is never double-closed.
+    expect(badClient.close).toHaveBeenCalledTimes(1);
+  });
+
+  it('connect succeeds but tools() HANGS (times out): the client is close()d once and the server is skipped', async () => {
+    const svc = buildService([server({ id: 'id-slow', name: 'slow' })]);
+    const slowClient = {
+      // tools() never settles -> withTimeout rejects after CONNECT_TIMEOUT_MS.
+      tools: () => new Promise<Record<string, never>>(() => {}),
+      close: jest.fn().mockResolvedValue(undefined),
+    };
+    stubConnect(svc, () => Promise.resolve(slowClient));
+
+    const toolsetPromise = svc.toolsFor('ws-5');
+    // Drive fake time past the tools() bound so withTimeout rejects.
+    await jest.advanceTimersByTimeAsync(CONNECT_TIMEOUT_MS + 1);
+    const toolset = await toolsetPromise;
+
+    expect(slowClient.close).toHaveBeenCalledTimes(1);
+    expect(Object.keys(toolset.tools)).toHaveLength(0);
+    expect(toolset.outcomes[0]?.ok).toBe(false);
+    await Promise.all(toolset.clients.map((c) => c.close()));
+  });
+});

apps/server/src/core/ai-chat/external-mcp/mcp-clients.service.ts

@@ -195,7 +195,7 @@ export class McpClientsService {
   ): Promise<{ ok: true; tools: string[] } | { ok: false; error: string }> {
     let client: McpClient | undefined;
     try {
-      client = await this.connect(server);
+      client = await this.connectWithTimeout(server, CONNECT_TIMEOUT_MS);
       const raw = await withTimeout(client.tools(), CONNECT_TIMEOUT_MS);
       return { ok: true, tools: Object.keys(raw) };
     } catch (err) {
@@ -255,49 +255,96 @@ export class McpClientsService {
     const callTimeoutMs = mcpCallTimeoutMs();
     const instructions: McpServerInstruction[] = [];
 
-    for (const server of servers) {
+    // Per-server connect+tools result, still tagged with its server so the merge
+    // below can be applied in the SAME order as `servers` (see the parallel note).
+    type PerServerResult =
+      | { ok: true; client: McpClient; guarded: Record<string, Tool> }
+      | { ok: false; reason: string };
+
+    // Connect to (and list tools for) every enabled server CONCURRENTLY, so the
+    // total build time is bounded by the SLOWEST single server (~2×
+    // CONNECT_TIMEOUT_MS: connect + tools), NOT the SUM across servers. The
+    // sequential loop this replaced summed those bounds, so with enough all-timing-
+    // out servers the outer MCP_TOOLSET_BUILD_DEADLINE_MS could fire before the
+    // per-server bounds, dropping ALL external tools and inverting the "per-server
+    // bound is primary, outer is a backstop" invariant. Each server keeps its OWN
+    // try/catch + connectWithTimeout/withTimeout bound + close-on-failure logic; a
+    // failed server is skipped, never fatal. Nothing here mutates the shared
+    // arrays — every result is merged IN SERVER ORDER after Promise.all, so tool-
+    // key precedence/disambiguation, `outcomes`, `instructions` and `clients`
+    // ordering all match the previous sequential behavior exactly.
+    const perServer = async (
+      server: (typeof servers)[number],
+    ): Promise<PerServerResult> => {
+      // Track the connected client so the catch can close it when it was obtained
+      // but tools() then threw/timed out (connectWithTimeout closes its OWN orphan
+      // on a connect timeout, so `client` stays undefined on that path). On success
+      // the client is handed back and registered by the merge below (owned by the
+      // entry, closed at teardown) — so it is never double-closed.
+      let client: McpClient | undefined;
       try {
-        const client = await this.connect(server);
+        client = await this.connectWithTimeout(server, CONNECT_TIMEOUT_MS);
         const raw = await withTimeout(client.tools(), CONNECT_TIMEOUT_MS);
-        clients.push(client);
         const allow = server.toolAllowlist;
         const picked =
           Array.isArray(allow) && allow.length > 0 ? pick(raw, allow) : raw;
         // Bound each tool's execute with a per-call total-timeout guard before
         // merging, so a single chatty-but-stuck call is aborted after the cap.
         const guarded = wrapToolsWithCallTimeout(picked, callTimeoutMs);
-        // Namespace each tool with the sanitized server name AND disambiguate
-        // against names already merged from earlier servers, so no external
-        // tool is silently overwritten on collision. The returned count drives
-        // whether this server's prompt guidance is included (≥1 tool merged).
-        const merged = this.mergeNamespaced(
-          tools,
-          guarded,
-          server.name,
-          server.id,
-        );
-        outcomes.push({ name: server.name, ok: true });
-        // Include this server's guidance ONLY when it actually contributed at
-        // least one tool the agent can call (allowlist may have filtered all of
-        // them out) AND the admin authored non-blank instructions. The header
-        // prefix is the sanitized server name (= the tool namespace prefix).
-        const guide = server.instructions?.trim();
-        if (merged.count > 0 && guide) {
-          instructions.push({
-            serverName: server.name,
-            toolPrefix: merged.prefix,
-            instructions: guide,
-          });
-        }
+        return { ok: true, client, guarded };
       } catch (err) {
-        // A failed server is skipped — the turn proceeds with the rest. Log a
-        // short warning (never the URL/headers) so ops can see degradation, and
-        // record the outcome so the UI can show "tool X unavailable".
+        // A failed server is skipped — the turn proceeds with the rest. If connect
+        // returned a live client but a later step (tools()) threw, that client was
+        // never registered in `clients`, so close it here or its transport/socket
+        // leaks (compounding every 60s cache rebuild during a flaky-server outage).
+        if (client) {
+          void client.close().catch(() => undefined);
+        }
+        // Log a short warning (never the URL/headers) so ops can see degradation,
+        // and record the outcome so the UI can show "tool X unavailable".
         const reason = shortError(err);
         this.logger.warn(
           `External MCP server "${server.name}" unavailable: ${reason}`,
         );
-        outcomes.push({ name: server.name, ok: false, reason });
+        return { ok: false, reason };
+      }
+    };
+
+    // Promise.all preserves array order regardless of settle order, so `results[i]`
+    // is `servers[i]`'s outcome — the merge below stays deterministic and matches
+    // the old sequential order (later servers still override/disambiguate against
+    // earlier ones on a tool-key clash).
+    const results = await Promise.all(servers.map(perServer));
+    for (let i = 0; i < servers.length; i += 1) {
+      const server = servers[i];
+      const result = results[i];
+      if (result.ok !== true) {
+        outcomes.push({ name: server.name, ok: false, reason: result.reason });
+        continue;
+      }
+      clients.push(result.client);
+      // Namespace each tool with the sanitized server name AND disambiguate
+      // against names already merged from earlier servers, so no external
+      // tool is silently overwritten on collision. The returned count drives
+      // whether this server's prompt guidance is included (≥1 tool merged).
+      const merged = this.mergeNamespaced(
+        tools,
+        result.guarded,
+        server.name,
+        server.id,
+      );
+      outcomes.push({ name: server.name, ok: true });
+      // Include this server's guidance ONLY when it actually contributed at
+      // least one tool the agent can call (allowlist may have filtered all of
+      // them out) AND the admin authored non-blank instructions. The header
+      // prefix is the sanitized server name (= the tool namespace prefix).
+      const guide = server.instructions?.trim();
+      if (merged.count > 0 && guide) {
+        instructions.push({
+          serverName: server.name,
+          toolPrefix: merged.prefix,
+          instructions: guide,
+        });
       }
     }
 
@@ -383,6 +430,55 @@ export class McpClientsService {
     return client;
   }
 
+  /**
+   * Race {@link connect} against a SETTLING timeout so a hung MCP handshake can
+   * never POISON the per-workspace build cache. `createMCPClient` (inside connect)
+   * is NOT bounded internally, and — exactly like @ai-sdk/mcp's tool calls
+   * (see wrapToolWithCallTimeout) — its promise does NOT settle on abort. So a
+   * transient network blip mid-handshake can make connect hang FOREVER. Because
+   * getOrBuildEntry caches the build PROMISE, a never-settling connect would then
+   * wedge EVERY later turn for the workspace (each awaits the same pending build,
+   * step_count stuck at 0, run row leaks 'running', chat 409s forever). Bounding
+   * connect here guarantees buildEntry always gets a client OR a rejection within
+   * `ms` — so the build completes (bad server skipped) and the cache stays clean.
+   *
+   * If connect resolves LATE (after we already rejected on the timeout), we close
+   * the orphaned client so its transport/socket is not leaked.
+   */
+  private connectWithTimeout(
+    server: Pick<AiMcpServer, 'transport' | 'url' | 'headersEnc'>,
+    ms: number,
+  ): Promise<McpClient> {
+    return new Promise<McpClient>((resolve, reject) => {
+      let settled = false;
+      const timer = setTimeout(() => {
+        settled = true;
+        reject(new Error(`MCP connect timed out after ${ms}ms`));
+      }, ms);
+      // Do not keep the process alive just for this connect-timeout timer.
+      timer.unref?.();
+      this.connect(server).then(
+        (client) => {
+          if (settled) {
+            // The race was already lost to the timeout: close the orphaned client
+            // so its socket is not leaked, and drop the late result.
+            void client.close().catch(() => undefined);
+            return;
+          }
+          clearTimeout(timer);
+          settled = true;
+          resolve(client);
+        },
+        (err: unknown) => {
+          if (settled) return; // late rejection after the timeout — already handled
+          clearTimeout(timer);
+          settled = true;
+          reject(err instanceof Error ? err : new Error(String(err)));
+        },
+      );
+    });
+  }
+
   /**
    * Decrypt the stored auth headers. Returns undefined when none are set. The
    * plaintext headers live only in this returned object and are passed straight
@@ -460,12 +556,12 @@ export function validateResolvedAddresses(addrs: readonly LookupAddress[]): {
  */
 function buildPinnedDispatcher(): Agent {
   // External-MCP traffic uses a DEDICATED, shorter silence timeout
-  // (`AI_MCP_STREAM_TIMEOUT_MS`, default 5 min) — deliberately tighter than the
+  // (`AI_MCP_STREAM_TIMEOUT_MS`, default 1 min) — deliberately tighter than the
   // chat provider's 15-min `streamTimeoutMs()` — so a byte-silent/hung MCP
-  // upstream is broken in ~5 min instead of 15. We keep the keep-alive options
+  // upstream is broken in ~1 min instead of 15. We keep the keep-alive options
   // from `streamingDispatcherOptions()` but OVERRIDE headers/body timeouts.
   // Accepted trade-off: a legitimately long but byte-silent single tool call,
-  // and an SSE transport idling >5 min BETWEEN tool calls, are also cut here; the
+  // and an SSE transport idling >1 min BETWEEN tool calls, are also cut here; the
   // per-call total cap (wrapToolsWithCallTimeout, `AI_MCP_CALL_TIMEOUT_MS`) is the
   // complementary guard for chatty-but-stuck calls that keep the socket warm yet
   // never return.

apps/server/src/core/ai-chat/roles/dto/agent-role.dto.ts

@@ -52,7 +52,7 @@ export class CreateAgentRoleDto {
   description?: string;
 
   @IsString()
-  @MaxLength(20000)
+  @MaxLength(100000)
   instructions: string;
 
   // null/omitted => use the workspace default model.
@@ -102,7 +102,7 @@ export class UpdateAgentRoleDto {
 
   @IsOptional()
   @IsString()
-  @MaxLength(20000)
+  @MaxLength(100000)
   instructions?: string;
 
   @IsOptional()

apps/server/src/core/ai-chat/tools/ai-chat-tools.service.spec.ts

@@ -652,6 +652,125 @@ describe('AiChatToolsService #294 changed execute wirings', () => {
   });
 });
 
+/**
+ * #410 — the footnote + image tools were promoted from MCP-only into the shared
+ * registry and are now wired in-app. Assert they are REGISTERED in the in-app
+ * toolset and forward their args to the client with the correct arg->method
+ * mapping (the schema fields `imageUrl`/`attachmentId` map onto the client's
+ * positional `url`/`oldAttachmentId`). A field destructured under the wrong name
+ * would silently pass `undefined` (execute is `any`-cast, so tsc won't catch it).
+ */
+describe('AiChatToolsService #410 footnote + image tools', () => {
+  const calls: Record<string, unknown[][]> = {
+    insertFootnote: [],
+    insertImage: [],
+    replaceImage: [],
+  };
+  const fakeClient: Partial<DocmostClientLike> = {
+    insertFootnote: (...args: unknown[]) => {
+      calls.insertFootnote.push(args);
+      return Promise.resolve({ success: true, footnoteId: 'fn1', reused: false });
+    },
+    insertImage: (...args: unknown[]) => {
+      calls.insertImage.push(args);
+      return Promise.resolve({ success: true, attachmentId: 'att1' });
+    },
+    replaceImage: (...args: unknown[]) => {
+      calls.replaceImage.push(args);
+      return Promise.resolve({ success: true, replaced: 1 });
+    },
+  };
+  const tokenServiceStub = {
+    generateAccessToken: jest.fn().mockResolvedValue('access-token'),
+    generateCollabToken: jest.fn().mockResolvedValue('collab-token'),
+  };
+  let service: AiChatToolsService;
+
+  beforeEach(() => {
+    for (const k of Object.keys(calls)) calls[k].length = 0;
+    jest.spyOn(loader, 'loadDocmostMcp').mockResolvedValue(
+      mockLoaded(function () {
+        return fakeClient as DocmostClientLike;
+      } as unknown as loader.DocmostClientCtor),
+    );
+    service = new AiChatToolsService(
+      tokenServiceStub as never,
+      {} as never,
+      {} as never,
+      {} as never,
+      {} as never,
+      {
+        asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }),
+      } as never,
+    );
+  });
+  afterEach(() => jest.restoreAllMocks());
+
+  const buildTools = () =>
+    service.forUser(
+      { id: 'user-1', email: 'u@example.com', workspaceId: 'ws-1' } as never,
+      'session-1',
+      'ws-1',
+      'chat-1',
+    );
+
+  it('registers all three tools in the in-app toolset', async () => {
+    const tools = await buildTools();
+    expect(tools.insertFootnote).toBeDefined();
+    expect(tools.insertImage).toBeDefined();
+    expect(tools.replaceImage).toBeDefined();
+  });
+
+  it('insertFootnote forwards (pageId, anchorText, text) positionally', async () => {
+    const tools = await buildTools();
+    const r = await tools.insertFootnote.execute(
+      { pageId: 'p1', anchorText: 'the claim', text: 'See source.' } as never,
+      {} as never,
+    );
+    expect(calls.insertFootnote).toEqual([['p1', 'the claim', 'See source.']]);
+    expect(r).toMatchObject({ footnoteId: 'fn1' });
+  });
+
+  it('insertImage maps imageUrl->url and packs the option fields', async () => {
+    const tools = await buildTools();
+    await tools.insertImage.execute(
+      {
+        pageId: 'p1',
+        imageUrl: 'https://x/img.png',
+        align: 'center',
+        alt: 'A',
+        replaceText: '[img]',
+        afterText: undefined,
+      } as never,
+      {} as never,
+    );
+    expect(calls.insertImage).toEqual([
+      [
+        'p1',
+        'https://x/img.png',
+        { align: 'center', alt: 'A', replaceText: '[img]', afterText: undefined },
+      ],
+    ]);
+  });
+
+  it('replaceImage maps attachmentId->oldAttachmentId and imageUrl->url', async () => {
+    const tools = await buildTools();
+    await tools.replaceImage.execute(
+      {
+        pageId: 'p1',
+        attachmentId: 'att-old',
+        imageUrl: 'https://x/new.png',
+        align: 'right',
+        alt: 'B',
+      } as never,
+      {} as never,
+    );
+    expect(calls.replaceImage).toEqual([
+      ['p1', 'att-old', 'https://x/new.png', { align: 'right', alt: 'B' }],
+    ]);
+  });
+});
+
 /**
  * getCurrentPage selection contract (#388): the tool surfaces the selection that
  * was sanitized + nested onto the resolved open-page context (last forUser arg).

apps/server/src/core/ai-chat/tools/ai-chat-tools.service.ts

@@ -17,7 +17,7 @@ import {
   resolveCurrentPageResult,
   type SelectionContext,
 } from './current-page.util';
-import { parseNodeArg } from './parse-node-arg';
+import { parseNodeArg } from '@docmost/prosemirror-markdown';
 import { modelFriendlyInput } from './model-friendly-input';
 import { SandboxStore } from '../../../integrations/sandbox/sandbox.store';
 import {
@@ -697,6 +697,38 @@ export class AiChatToolsService {
         },
       ),
 
+      // Schema + description live in @docmost/mcp's SHARED_TOOL_SPECS (#410).
+      // Promoted from MCP-only so the in-app agent can attach a REAL footnote to
+      // already-written text instead of leaving a literal `^[...]` string.
+      insertFootnote: sharedTool(
+        sharedToolSpecs.insertFootnote,
+        async ({ pageId, anchorText, text }) =>
+          await client.insertFootnote(pageId, anchorText, text),
+      ),
+
+      // Schema + description live in @docmost/mcp's SHARED_TOOL_SPECS (#410).
+      // The schema field is `imageUrl`; the client method takes it positionally.
+      insertImage: sharedTool(
+        sharedToolSpecs.insertImage,
+        async ({ pageId, imageUrl, align, alt, replaceText, afterText }) =>
+          await client.insertImage(pageId, imageUrl, {
+            align,
+            alt,
+            replaceText,
+            afterText,
+          }),
+      ),
+
+      // Schema + description live in @docmost/mcp's SHARED_TOOL_SPECS (#410).
+      replaceImage: sharedTool(
+        sharedToolSpecs.replaceImage,
+        async ({ pageId, attachmentId, imageUrl, align, alt }) =>
+          await client.replaceImage(pageId, attachmentId, imageUrl, {
+            align,
+            alt,
+          }),
+      ),
+
       // Schema + description now live in @docmost/mcp's SHARED_TOOL_SPECS (#294).
       // The table reference parameter was unified to `table` (was `tableRef`).
       tableInsertRow: sharedTool(

apps/server/src/core/ai-chat/tools/docmost-client.loader.ts

@@ -141,6 +141,33 @@ export interface DocmostClientLike {
     doc?: unknown,
     title?: string,
   ): Promise<Record<string, unknown>>;
+  // Attach an author-inline footnote after the first occurrence of anchorText;
+  // numbering + the footnotes list are derived server-side.
+  insertFootnote(
+    pageId: string,
+    anchorText: string,
+    text: string,
+  ): Promise<Record<string, unknown>>;
+  // Download a web image and insert it into the page (append, or replace/after a
+  // text anchor). `url` is the image http(s) URL.
+  insertImage(
+    pageId: string,
+    url: string,
+    opts?: {
+      align?: 'left' | 'center' | 'right';
+      alt?: string;
+      replaceText?: string;
+      afterText?: string;
+    },
+  ): Promise<Record<string, unknown>>;
+  // Swap an existing image (by its attachmentId) for a new one fetched from a web
+  // URL, repointing every reference in the live document.
+  replaceImage(
+    pageId: string,
+    oldAttachmentId: string,
+    url: string,
+    opts?: { align?: 'left' | 'center' | 'right'; alt?: string },
+  ): Promise<Record<string, unknown>>;
   tableInsertRow(
     pageId: string,
     tableRef: string,

apps/server/src/core/ai-chat/tools/parse-node-arg.spec.ts

@@ -1,10 +1,10 @@
-import { parseNodeArg } from './parse-node-arg';
+import { parseNodeArg } from '@docmost/prosemirror-markdown';
 
 /**
- * Unit tests for the in-app `parseNodeArg` helper. It mirrors the standalone
- * MCP helper (packages/mcp/src/lib/parse-node-arg.ts) and is used by the
- * patchNode / insertNode / updatePageJson tool adapters. Behavior must be
- * byte-identical: object passthrough, valid-string parse, invalid-string throw.
+ * Unit tests for the shared `parseNodeArg` helper (#414: now the single copy in
+ * `@docmost/prosemirror-markdown`, imported by both the server tool adapters and
+ * `@docmost/mcp`). Used by the patchNode / insertNode / updatePageJson adapters.
+ * Behavior: object passthrough, valid-string parse, invalid-string throw.
  */
 describe('parseNodeArg', () => {
   it('passes an object through unchanged', () => {

apps/server/src/core/ai-chat/tools/parse-node-arg.ts

@@ -1,26 +0,0 @@
-// The model sometimes serializes a ProseMirror node arg as a JSON string
-// instead of an object. Normalize: parse a string to an object (throwing on
-// invalid JSON), pass an object through unchanged. Shared by patchNode /
-// insertNode (and the analogous updatePageJson content parsing).
-//
-// This is behaviorally identical to `packages/mcp/src/lib/parse-node-arg.ts`
-// (the function logic, default/explicit throw messages and branch order match;
-// only comments and quote style differ). We cannot import that helper here:
-// `@docmost/mcp` is ESM-only and this server
-// compiles with module:commonjs, so it is loaded at runtime via the
-// `new Function('import()')` trick (see docmost-client.loader.ts). Sharing
-// runtime code across that ESM/CJS boundary by a normal import is impossible,
-// hence the mirrored copy.
-export function parseNodeArg(
-  node: unknown,
-  errMsg = 'node was a string but not valid JSON',
-): unknown {
-  if (typeof node === 'string') {
-    try {
-      return JSON.parse(node);
-    } catch {
-      throw new Error(errMsg);
-    }
-  }
-  return node;
-}

apps/server/src/core/ai-chat/tools/tool-tiers.spec.ts

@@ -27,13 +27,26 @@ import type { DocmostClientLike } from './docmost-client.loader';
  */
 
 describe('tool tier metadata (#332)', () => {
-  it('core set is the documented 13 + searchInPage (14)', () => {
-    expect(CORE_TOOL_KEYS).toHaveLength(14);
+  it('core set is the documented 13 + searchInPage + insertFootnote (15)', () => {
+    expect(CORE_TOOL_KEYS).toHaveLength(15);
     expect(CORE_TOOL_SET.has('searchInPage')).toBe(true); // #330, promoted to core
+    expect(CORE_TOOL_SET.has('insertFootnote')).toBe(true); // #410, promoted to core
     // loadTools is a meta-tool, not a normal core key.
     expect(CORE_TOOL_SET.has(LOAD_TOOLS_NAME)).toBe(false);
   });
 
+  it('#410 image tools are DEFERRED, footnote tool is CORE', () => {
+    // insert_footnote is core (symmetric with editPageText); the image tools stay
+    // deferred (rare, fat — loaded on demand). Assert both the spec tier and the
+    // CORE_TOOL_SET membership so a future tier edit that desyncs them fails here.
+    expect(SHARED_TOOL_SPECS.insertFootnote.tier).toBe('core');
+    expect(CORE_TOOL_SET.has('insertFootnote')).toBe(true);
+    expect(SHARED_TOOL_SPECS.insertImage.tier).toBe('deferred');
+    expect(CORE_TOOL_SET.has('insertImage')).toBe(false);
+    expect(SHARED_TOOL_SPECS.replaceImage.tier).toBe('deferred');
+    expect(CORE_TOOL_SET.has('replaceImage')).toBe(false);
+  });
+
   it('SHARED_TOOL_SPECS tier agrees with CORE_TOOL_SET for every shared tool', () => {
     for (const [key, spec] of Object.entries(SHARED_TOOL_SPECS)) {
       const isCoreByTier = spec.tier === 'core';

apps/server/src/core/ai-chat/tools/tool-tiers.ts

@@ -38,10 +38,13 @@ export interface ToolCatalogEntry {
 }
 
 /**
- * CORE (always-active) in-app tool keys — 13 frequent/tiny tools. `searchInPage`
- * (#330) is added to core on top of the issue's original tier list: it is
- * frequent for the editorial roles this feature targets. `loadTools` is active
- * too but is not a normal tool key (it is added to activeTools separately).
+ * CORE (always-active) in-app tool keys — 13 frequent/tiny tools + `searchInPage`
+ * (#330) + `insertFootnote` (#410). `searchInPage` is core because it is frequent
+ * for the editorial roles this feature targets; `insertFootnote` is core so the
+ * footnote tool is NOT hidden while its natural sibling `editPageText` is always
+ * active (that asymmetry is exactly what pushed the agent to write literal
+ * `^[...]`). `loadTools` is active too but is not a normal tool key (it is added
+ * to activeTools separately).
  */
 export const CORE_TOOL_KEYS = [
   'searchPages',
@@ -60,6 +63,9 @@ export const CORE_TOOL_KEYS = [
   // #330 search_in_page — frequent for editorial sweeps; core despite predating
   // the issue's tier list.
   'searchInPage',
+  // #410 insert_footnote — core so pinpoint citations to already-written text
+  // don't degrade into literal `^[...]`; kept symmetric with editPageText.
+  'insertFootnote',
 ] as const;
 
 /** O(1) membership test for the core tier. */

apps/server/src/integrations/ai/ai-streaming-fetch.ts

@@ -123,19 +123,19 @@ export function streamKeepAliveMs(): number {
   return positiveEnv('AI_STREAM_KEEPALIVE_MS', DEFAULT_STREAM_KEEPALIVE_MS);
 }
 
-/** Default SILENCE timeout for EXTERNAL-MCP transport (5 min). */
-const DEFAULT_MCP_STREAM_TIMEOUT_MS = 300_000;
+/** Default SILENCE timeout for EXTERNAL-MCP transport (1 min). */
+const DEFAULT_MCP_STREAM_TIMEOUT_MS = 60_000;
 
-/** Default total wall-clock cap for ONE external MCP tool call (15 min). */
-const DEFAULT_MCP_CALL_TIMEOUT_MS = 900_000;
+/** Default total wall-clock cap for ONE external MCP tool call (2 min). */
+const DEFAULT_MCP_CALL_TIMEOUT_MS = 120_000;
 
 /**
  * SILENCE timeout (ms) for EXTERNAL-MCP transport ONLY. Override with
  * `AI_MCP_STREAM_TIMEOUT_MS`; a missing/invalid/non-positive value falls back to
- * {@link DEFAULT_MCP_STREAM_TIMEOUT_MS} (5 min).
+ * {@link DEFAULT_MCP_STREAM_TIMEOUT_MS} (1 min).
  *
  * Deliberately tighter than the chat provider's {@link streamTimeoutMs} (15 min)
- * so a byte-silent/hung MCP upstream is broken in ~5 min instead of 15. This is
+ * so a byte-silent/hung MCP upstream is broken in ~1 min instead of 15. This is
  * the undici `headersTimeout`/`bodyTimeout` for the external-MCP dispatcher only
  * — it must NOT change the chat provider, which legitimately needs 15 min between
  * reasoning chunks (#175).
@@ -153,7 +153,7 @@ export function mcpStreamTimeoutMs(): number {
 /**
  * Total wall-clock cap (ms) for ONE external MCP tool call — APP-LEVEL, not
  * transport. Override with `AI_MCP_CALL_TIMEOUT_MS`; a missing/invalid/
- * non-positive value falls back to {@link DEFAULT_MCP_CALL_TIMEOUT_MS} (15 min).
+ * non-positive value falls back to {@link DEFAULT_MCP_CALL_TIMEOUT_MS} (2 min).
  *
  * Catches a tool that keeps the connection warm (SSE heartbeats / trickle) but
  * never returns a result — which the transport silence timeout

apps/server/src/integrations/mcp/mcp-auth.helpers.ts

@@ -149,6 +149,15 @@ export type DocmostMcpConfig = (
     has?: (uri: string) => boolean;
     evict?: (uri: string) => void;
   };
+  // Dependency-neutral metrics sink injected by McpService (mirror of the
+  // package's onMetric). The package emits generic (name, value, labels)
+  // samples; McpService maps them onto the prom-client registry. Undefined
+  // when metrics are disabled → the package no-ops.
+  onMetric?: (
+    name: string,
+    value: number,
+    labels?: Record<string, string>,
+  ) => void;
 };
 
 export interface ResolvedMcpAuth {

apps/server/src/integrations/mcp/mcp.service.ts

@@ -30,6 +30,11 @@ import {
   ResolvedMcpAuth,
 } from './mcp-auth.helpers';
 import { SandboxStore } from '../sandbox/sandbox.store';
+import {
+  isMetricsEnabled,
+  observeMcpTool,
+  incConnectTimeout,
+} from '../metrics/metrics.registry';
 
 // Minimal shape of the embedded MCP HTTP handler exported by @docmost/mcp/http.
 interface McpHttpHandler {
@@ -331,7 +336,31 @@ export class McpService implements OnModuleDestroy {
             // can store blobs in the shared in-RAM store regardless of which
             // credential variant resolved. The sink (put/has/evict + uri↔id
             // mapping) is owned by SandboxStore.asSink().
-            return { ...resolved.config, sandbox: this.sandboxStore.asSink() };
+            // Route the package's dependency-neutral metric samples onto the
+            // prom-client registry. When metrics are disabled, onMetric is
+            // undefined → the package's tool-timer/timeout hooks are a
+            // negligible-overhead no-op: the registerTool wrapper still runs a
+            // performance.now() + async try/finally per tool call, but the
+            // `onMetric?.()` short-circuits so no label/object is built. (Cost
+            // is immaterial at LLM tool-call rate.) labels?.tool is guarded
+            // defensively (the tool wrapper always sets it).
+            return {
+              ...resolved.config,
+              sandbox: this.sandboxStore.asSink(),
+              onMetric: isMetricsEnabled()
+                ? (
+                    name: string,
+                    value: number,
+                    labels?: Record<string, string>,
+                  ) => {
+                    if (name === 'mcp_tool_duration_seconds') {
+                      observeMcpTool(labels?.tool ?? 'other', value);
+                    } else if (name === 'collab_connect_timeouts_total') {
+                      incConnectTimeout();
+                    }
+                  }
+                : undefined,
+            };
           },
           {
             identify: (req: IncomingMessage) => {

apps/server/src/integrations/metrics/metrics.constants.ts

@@ -2,6 +2,11 @@
  * Perf-metrics contract (#355). These names/labels are FIXED by the already
  * deployed scrape+dashboard infra (VictoriaMetrics scraping docmost:9464,
  * Grafana dashboards, alerts). Do NOT rename them.
+ *
+ * #402 extends the #355 table with the collab-lifecycle + MCP-tool families
+ * (grouped below). These are the fixed contract from #402; same "do not rename"
+ * rule applies. Server-side families land in Pass 1; the MCP-tool histogram is
+ * fed by the MCP callback in a later pass but its NAME is fixed here now.
  */
 export const METRIC_HTTP_REQUEST_DURATION = 'http_request_duration_seconds';
 export const METRIC_DB_QUERY_DURATION = 'db_query_duration_seconds';
@@ -9,6 +14,17 @@ export const METRIC_BULLMQ_QUEUE_DEPTH = 'bullmq_queue_depth';
 export const METRIC_BULLMQ_JOB_DURATION = 'bullmq_job_duration_seconds';
 export const METRIC_COLLAB_STORE_DURATION = 'collab_store_duration_seconds';
 
+// #402 additions — collaboration lifecycle + MCP tool timing.
+export const METRIC_COLLAB_LOAD_DURATION = 'collab_doc_load_duration_seconds';
+export const METRIC_COLLAB_DOCS_OPEN = 'collab_docs_open';
+export const METRIC_COLLAB_DOC_LOADS_TOTAL = 'collab_doc_loads_total';
+export const METRIC_COLLAB_DOC_UNLOADS_TOTAL = 'collab_doc_unloads_total';
+export const METRIC_COLLAB_CONNECT_DURATION = 'collab_connect_duration_seconds';
+export const METRIC_COLLAB_CONNECT_TIMEOUTS_TOTAL =
+  'collab_connect_timeouts_total';
+export const METRIC_COLLAB_AUTH_DURATION = 'collab_auth_duration_seconds';
+export const METRIC_MCP_TOOL_DURATION = 'mcp_tool_duration_seconds';
+
 // Histogram buckets (seconds). Chosen to give useful p50/p95/p99 resolution
 // for typical web/DB latencies without exploding series cardinality.
 export const HTTP_BUCKETS = [
@@ -23,6 +39,12 @@ export const COLLAB_BUCKETS = [
 export const JOB_BUCKETS = [
   0.01, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10, 30, 60, 120,
 ];
+// #402 — MCP tool-call latency. Same shape as COLLAB_BUCKETS but stretched to
+// 10s at the top: an MCP tool round-trip (LLM-driven doc ops) can be slower
+// than a single collab store, so keep resolution out to 10s.
+export const MCP_TOOL_BUCKETS = [
+  0.005, 0.025, 0.1, 0.25, 0.5, 1, 2.5, 5, 10,
+];
 
 /**
  * Extract the first SQL token (select/insert/update/delete/...) from a query,
@@ -58,6 +80,30 @@ export function firstSqlToken(sql: string | undefined): string {
   return KNOWN_SQL_TOKENS.has(token) ? token : 'other';
 }
 
+/**
+ * #402 — bucket a document byte size into ONE of four fixed labels for the
+ * collab load/store histograms' `size_bucket` label. Using the raw byte count
+ * would be a continuous, unbounded label; four coarse buckets keep series
+ * cardinality bounded (each of collab_doc_load / collab_store gets ×4 series).
+ *
+ * The SAME function is shared by both the load and store paths so their buckets
+ * can never drift apart. Non-finite / negative sizes collapse to the smallest
+ * bucket ('lt64k') as a safe default (they can't be legitimately huge and we
+ * must never throw here — this runs on every store/load when metrics are on).
+ */
+// Module-const thresholds, built once (not per observe).
+const SIZE_THRESHOLDS = { lt64k: 65536, lt256k: 262144, lt1m: 1048576 } as const;
+
+export function sizeBucket(
+  bytes: number | undefined | null,
+): 'lt64k' | 'lt256k' | 'lt1m' | 'ge1m' {
+  if (bytes == null || !Number.isFinite(bytes) || bytes < 0) return 'lt64k';
+  if (bytes < SIZE_THRESHOLDS.lt64k) return 'lt64k';
+  if (bytes < SIZE_THRESHOLDS.lt256k) return 'lt256k';
+  if (bytes < SIZE_THRESHOLDS.lt1m) return 'lt1m';
+  return 'ge1m';
+}
+
 /**
  * Whether an HTTP response must be EXCLUDED from http_request_duration_seconds.
  *

apps/server/src/integrations/metrics/metrics.registry.ts

@@ -1,5 +1,6 @@
 import {
   collectDefaultMetrics,
+  Counter,
   Histogram,
   Gauge,
   Registry,
@@ -9,11 +10,21 @@ import {
   DB_BUCKETS,
   HTTP_BUCKETS,
   JOB_BUCKETS,
+  MCP_TOOL_BUCKETS,
   METRIC_BULLMQ_JOB_DURATION,
   METRIC_BULLMQ_QUEUE_DEPTH,
+  METRIC_COLLAB_AUTH_DURATION,
+  METRIC_COLLAB_CONNECT_DURATION,
+  METRIC_COLLAB_CONNECT_TIMEOUTS_TOTAL,
+  METRIC_COLLAB_DOC_LOADS_TOTAL,
+  METRIC_COLLAB_DOC_UNLOADS_TOTAL,
+  METRIC_COLLAB_DOCS_OPEN,
+  METRIC_COLLAB_LOAD_DURATION,
   METRIC_COLLAB_STORE_DURATION,
   METRIC_DB_QUERY_DURATION,
   METRIC_HTTP_REQUEST_DURATION,
+  METRIC_MCP_TOOL_DURATION,
+  sizeBucket,
 } from './metrics.constants';
 
 /**
@@ -39,7 +50,24 @@ let httpHist: Histogram<'method' | 'route' | 'status'> | null = null;
 let dbHist: Histogram<'op'> | null = null;
 let queueDepthGauge: Gauge<'queue'> | null = null;
 let jobHist: Histogram<'queue'> | null = null;
-let collabHist: Histogram | null = null;
+// #402 — collab store now carries a size_bucket label (see sizeBucket()).
+let collabHist: Histogram<'size_bucket'> | null = null;
+// #402 collab-lifecycle + MCP instruments.
+let collabLoadHist: Histogram<'size_bucket'> | null = null;
+let docsOpenGauge: Gauge | null = null;
+let docLoadsCounter: Counter | null = null;
+let docUnloadsCounter: Counter | null = null;
+let connectTimeoutsCounter: Counter | null = null;
+let collabConnectHist: Histogram | null = null;
+let collabAuthHist: Histogram | null = null;
+let mcpToolHist: Histogram<'tool'> | null = null;
+
+// #402 — read-on-scrape source for collab_docs_open. The gauge is NEVER
+// inc/dec'd (that drifts under crashes/handoffs); instead its collect() callback
+// pulls the authoritative live count from here on every scrape. Registered once,
+// gated, by the collaboration gateway via registerDocsOpenSource(). Null until
+// then → the collect() is a no-op.
+let docsOpenSource: (() => number) | null = null;
 
 function init(): void {
   if (registry || !enabled) return;
@@ -82,10 +110,71 @@ function init(): void {
 
   collabHist = new Histogram({
     name: METRIC_COLLAB_STORE_DURATION,
-    help: 'Collaboration onStoreDocument duration in seconds',
+    help: 'Collaboration onStoreDocument duration in seconds, by document size bucket',
+    labelNames: ['size_bucket'],
     buckets: COLLAB_BUCKETS,
     registers: [registry],
   });
+
+  collabLoadHist = new Histogram({
+    name: METRIC_COLLAB_LOAD_DURATION,
+    help: 'Collaboration onLoadDocument DB-load duration in seconds, by document size bucket',
+    labelNames: ['size_bucket'],
+    buckets: COLLAB_BUCKETS,
+    registers: [registry],
+  });
+
+  docsOpenGauge = new Gauge({
+    name: METRIC_COLLAB_DOCS_OPEN,
+    help: 'Number of collaboration documents currently open in memory',
+    registers: [registry],
+    // Read-on-scrape: pull the live count from the registered source (the
+    // hocuspocus instance) so the value can never drift. No-op until a source
+    // is registered.
+    collect() {
+      if (docsOpenSource) this.set(docsOpenSource());
+    },
+  });
+
+  docLoadsCounter = new Counter({
+    name: METRIC_COLLAB_DOC_LOADS_TOTAL,
+    help: 'Total collaboration documents loaded into memory',
+    registers: [registry],
+  });
+
+  docUnloadsCounter = new Counter({
+    name: METRIC_COLLAB_DOC_UNLOADS_TOTAL,
+    help: 'Total collaboration documents unloaded from memory',
+    registers: [registry],
+  });
+
+  connectTimeoutsCounter = new Counter({
+    name: METRIC_COLLAB_CONNECT_TIMEOUTS_TOTAL,
+    help: 'Total collaboration connection setup timeouts',
+    registers: [registry],
+  });
+
+  collabConnectHist = new Histogram({
+    name: METRIC_COLLAB_CONNECT_DURATION,
+    help: 'Collaboration connection acceptance duration in seconds (onConnect→connected)',
+    buckets: COLLAB_BUCKETS,
+    registers: [registry],
+  });
+
+  collabAuthHist = new Histogram({
+    name: METRIC_COLLAB_AUTH_DURATION,
+    help: 'Collaboration onAuthenticate duration in seconds',
+    buckets: COLLAB_BUCKETS,
+    registers: [registry],
+  });
+
+  mcpToolHist = new Histogram({
+    name: METRIC_MCP_TOOL_DURATION,
+    help: 'MCP tool-call duration in seconds, by tool name',
+    labelNames: ['tool'],
+    buckets: MCP_TOOL_BUCKETS,
+    registers: [registry],
+  });
 }
 
 // Runs once when this module is first imported. Safe to call again (idempotent).
@@ -121,6 +210,46 @@ export function observeJobDuration(queue: string, seconds: number): void {
   jobHist?.observe({ queue }, seconds);
 }
 
-export function observeCollabStore(seconds: number): void {
-  collabHist?.observe(seconds);
+export function observeCollabStore(bytes: number, seconds: number): void {
+  collabHist?.observe({ size_bucket: sizeBucket(bytes) }, seconds);
+}
+
+export function observeCollabLoad(bytes: number, seconds: number): void {
+  collabLoadHist?.observe({ size_bucket: sizeBucket(bytes) }, seconds);
+}
+
+/**
+ * Register the live open-document count source for the collab_docs_open gauge.
+ * Called ONCE, gated by isMetricsEnabled(), by the collaboration gateway. The
+ * gauge reads this on every scrape (collect()); nothing inc/dec's it.
+ */
+export function registerDocsOpenSource(fn: () => number): void {
+  docsOpenSource = fn;
+}
+
+export function incDocLoad(): void {
+  docLoadsCounter?.inc();
+}
+
+export function incDocUnload(): void {
+  docUnloadsCounter?.inc();
+}
+
+export function incConnectTimeout(): void {
+  connectTimeoutsCounter?.inc();
+}
+
+export function observeCollabConnect(seconds: number): void {
+  collabConnectHist?.observe(seconds);
+}
+
+export function observeCollabAuth(seconds: number): void {
+  collabAuthHist?.observe(seconds);
+}
+
+export function observeMcpTool(tool: string, seconds: number): void {
+  // `tool` MUST be a bounded, registration-derived MCP tool name (the caller
+  // guarantees it comes from the registered-tool set) — never free-form input —
+  // so this label stays low-cardinality with no 'other' bucketing needed.
+  mcpToolHist?.observe({ tool }, seconds);
 }

apps/server/src/integrations/metrics/metrics.spec.ts

@@ -1,6 +1,23 @@
 import { FastifyRequest } from 'fastify';
 import { resolveRouteLabel } from './http-metrics.hook';
-import { firstSqlToken, isStreamingResponse } from './metrics.constants';
+import {
+  firstSqlToken,
+  isStreamingResponse,
+  sizeBucket,
+} from './metrics.constants';
+import {
+  getMetricsRegistry,
+  incConnectTimeout,
+  incDocLoad,
+  incDocUnload,
+  isMetricsEnabled,
+  observeCollabAuth,
+  observeCollabConnect,
+  observeCollabLoad,
+  observeCollabStore,
+  observeMcpTool,
+  registerDocsOpenSource,
+} from './metrics.registry';
 
 describe('resolveRouteLabel (histogram route label)', () => {
   it('uses the ROUTE TEMPLATE, never the raw URL', () => {
@@ -123,3 +140,67 @@ describe('firstSqlToken (bounded db label)', () => {
     expect(firstSqlToken('vacuum analyze')).toBe('other');
   });
 });
+
+describe('sizeBucket (#402 bounded size label)', () => {
+  it('maps sizes to the four fixed buckets at their boundaries', () => {
+    // Boundaries are exclusive-upper: <65536 → lt64k, etc.
+    expect(sizeBucket(0)).toBe('lt64k');
+    expect(sizeBucket(65535)).toBe('lt64k');
+    expect(sizeBucket(65536)).toBe('lt256k');
+    expect(sizeBucket(262143)).toBe('lt256k');
+    expect(sizeBucket(262144)).toBe('lt1m');
+    expect(sizeBucket(1048575)).toBe('lt1m');
+    expect(sizeBucket(1048576)).toBe('ge1m');
+    expect(sizeBucket(5_000_000)).toBe('ge1m');
+  });
+
+  it('falls back to the smallest bucket for invalid sizes', () => {
+    // Non-finite / negative / nullish collapse to the smallest, safe default.
+    expect(sizeBucket(-1)).toBe('lt64k');
+    expect(sizeBucket(NaN)).toBe('lt64k');
+    expect(sizeBucket(Infinity)).toBe('lt64k');
+    expect(sizeBucket(undefined)).toBe('lt64k');
+    expect(sizeBucket(null)).toBe('lt64k');
+  });
+
+  it('only ever returns one of the four fixed labels (bounded cardinality)', () => {
+    const labels = new Set(
+      [0, 65536, 262144, 1048576, -5, NaN].map((b) => sizeBucket(b)),
+    );
+    for (const l of labels) {
+      expect(['lt64k', 'lt256k', 'lt1m', 'ge1m']).toContain(l);
+    }
+  });
+});
+
+describe('metrics helpers are safe no-ops when METRICS_PORT is unset', () => {
+  // These specs run without METRICS_PORT, so the registry is never created and
+  // every observe/inc/set helper must be a cheap `?.` no-op that never throws.
+  beforeAll(() => {
+    // Guard the contract this suite depends on: if a CI env set METRICS_PORT,
+    // the assertions below would be meaningless, so fail loudly instead.
+    expect(process.env.METRICS_PORT).toBeUndefined();
+  });
+
+  it('reports metrics disabled and a null registry', () => {
+    expect(isMetricsEnabled()).toBe(false);
+    expect(getMetricsRegistry()).toBeNull();
+  });
+
+  it('does not throw from any #402 collab/MCP helper', () => {
+    expect(() => {
+      observeCollabLoad(123456, 0.01);
+      observeCollabStore(123456, 0.02);
+      observeCollabConnect(0.03);
+      observeCollabAuth(0.04);
+      observeMcpTool('some-tool', 0.05);
+      incDocLoad();
+      incDocUnload();
+      incConnectTimeout();
+      // Registering a source must not create the gauge or invoke the fn.
+      registerDocsOpenSource(() => {
+        throw new Error('docsOpenSource must NOT be called when disabled');
+      });
+    }).not.toThrow();
+  });
+});

apps/server/src/main.ts

@@ -21,10 +21,24 @@ import { recordHttpResponse } from './integrations/metrics/http-metrics.hook';
 import { startMetricsServer } from './integrations/metrics/metrics.server';
 
 async function bootstrap() {
+  // Fastify JSON body cap. Fastify defaults to 1 MiB, which a long AI-chat
+  // research turn exceeds: the client resends the FULL message history (every
+  // tool call + search result) on each turn, so a deep conversation's POST to
+  // /api/ai-chat/stream can be several MB and would otherwise be rejected with
+  // FST_ERR_CTP_BODY_TOO_LARGE (413). Raise the cap; override with
+  // HTTP_JSON_BODY_LIMIT (bytes). A missing/invalid/non-positive value keeps the
+  // 25 MiB default. Multipart uploads are unaffected (their own @fastify/multipart
+  // limits apply); this only bounds JSON/urlencoded request bodies.
+  const bodyLimitEnv = Number(process.env.HTTP_JSON_BODY_LIMIT);
+  const bodyLimit =
+    Number.isFinite(bodyLimitEnv) && bodyLimitEnv > 0
+      ? bodyLimitEnv
+      : 25 * 1024 * 1024;
   const app = await NestFactory.create<NestFastifyApplication>(
     AppModule,
     new FastifyAdapter({
       trustProxy: resolveTrustProxy(process.env.TRUST_PROXY),
+      bodyLimit,
       routerOptions: {
         maxParamLength: 1000,
         ignoreTrailingSlash: true,

docs/reading-ai-logs.md

@@ -0,0 +1,270 @@
+# Reading the AI dialog logs (how agents call tools, and where they fail)
+
+How to inspect the agent conversation history in the database — and, more
+importantly, the **one non-obvious trap that will make you report the wrong
+answer**: the persisted history *silently hides hard tool failures*. Written from
+real pain (a "which tools fail most?" analysis that confidently answered
+"patchNode: 0 errors" while the UI was visibly full of red `patchNode` failures).
+
+Read the **Gotchas** section before you trust any error count.
+
+## TL;DR
+
+- Agent chats live in Postgres, DB `docmost`, tables `ai_chat_*`.
+- Each tool invocation is stored as **two** array elements (a `tool-call` part and
+  a `tool-result` part), so naive counting double-counts.
+- **A tool that *throws* writes no result part at all.** Its error text is nowhere
+  in the DB — not in `tool_calls`, `content`, or `metadata`. It is shown live in
+  the UI only. So `isError` / `success=false` scans under-report by design.
+- To find where agents fail you need **three** sources: (1) soft-failure markers in
+  `tool_calls`, (2) the orphan-gap proxy for thrown errors, (3) server logs / the
+  live UI for the actual error text.
+
+## Where the data lives
+
+Host `island.lc` (`10.31.40.120`), container `gitmost-postgresql`
+(`pgvector/pgvector:pg18`), database `docmost`.
+
+```bash
+ssh island.lc
+# one-off query:
+docker exec gitmost-postgresql psql -U docmost -d docmost -P pager=off -c "SELECT ..."
+# interactive:
+docker exec -it gitmost-postgresql psql -U docmost -d docmost
+```
+
+The main app container is `gitmost` (`DATABASE_URL=postgresql://docmost:...@db:5432/docmost`).
+All workspaces (vvzvlad / wb / asakusa / …) share this **single** database — they
+are rows in `workspaces`, not separate deployments.
+
+### Relevant tables
+
+| Table | What it holds |
+| --- | --- |
+| `ai_chats` | one row per conversation (`title`, `role_id`, `page_id`, `creator_id`) |
+| `ai_chat_messages` | every message; tool calls live in `tool_calls` jsonb |
+| `ai_chat_runs` | one row per agent run (turn): `status`, `error`, `step_count` |
+| `ai_agent_roles` | agent definitions (`instructions`, `model_config`) |
+| `ai_mcp_servers` | configured MCP tool servers per workspace |
+
+`ai_chat_messages` columns that matter: `role` (`user` | `assistant` — there is **no**
+separate `tool` role), `content` (text), `tool_calls` (jsonb array), `metadata`
+(jsonb, holds run `error` + rendered `parts`), `status`, `tsv` (full-text index).
+
+## How tool calls are stored — READ THIS
+
+Tool calls are **not** one-object-per-call. Each logical invocation is split into
+two consecutive elements of the `tool_calls` array:
+
+```text
+index 0: { "toolName": "getPage",  "input":  { "pageId": "…" } }   ← tool-call   (has input, NO output)
+index 1: { "toolName": "getPage",  "output": { … } }                ← tool-result (has output, NO input)
+```
+
+The **only** keys that ever appear on an element are `toolName`, `input`, `output`.
+There is no `state`, no `errorText`, no `type`. Consequences:
+
+1. **Real invocation count = elements that have `output`.** Counting every element
+   double-counts (you get ~2× and a spurious "~50% of every tool has no output").
+2. **Pairing:** a successful call = a `tool-call` part followed by its `tool-result`
+   part. Both carry `toolName`, so you can group by tool on either.
+
+## The two classes of failure (and which the DB can see)
+
+### 1. Soft failures — tool RAN and returned an error-shaped result → PERSISTED ✅
+
+These are visible in the `tool-result` `output`. The marker differs per tool:
+
+| Tool(s) | Error marker in `output` |
+| --- | --- |
+| `editPageText` | `failed` is a **non-empty** array of `{find, reason}` (e.g. `text not found in the document`, `matches N times — provide a longer fragment or set replaceAll`). Also a soft `warning` when the `find` string contained markdown that only matched after stripping. |
+| `semanticSearch` | `{ "unavailable": true, "reason": "semantic search unavailable" }` (feature/infra, not the agent's fault) |
+| MCP passthrough (`Habr_*`, some `Search_*`) | `output` is an **array** (raw MCP content) whose text starts with `Error executing tool … validation error …` |
+| generic | `output.isError = true` or `output.success = false` |
+
+Note `editPageText` returns `failed: []` on success — filtering on the *presence*
+of the key gives false positives; filter on **non-empty**.
+
+### 2. Hard failures — tool THREW → NOT PERSISTED ❌ (the trap)
+
+When a tool throws (the classic one is `patchNode` / `insertNode` / `tableUpdateCell`
+→ `Failed to encode document to Yjs (fromJSON): Unknown node type: undefined`), the
+runtime writes **no `tool-result` part**. The orphaned `tool-call` part stays, but
+the error text is **nowhere in the DB**. It is streamed to the UI live and (until
+rotation) to server logs — that is it.
+
+So any query like `count(*) FILTER (WHERE output.success = false)` will happily
+return **0** for `patchNode` even when the chat is visibly full of red failures.
+That is survivorship bias, not reliability.
+
+The only DB-side proxy for a thrown error is an **orphan**: a `tool-call` part with
+no matching `tool-result`. Caveat: orphans also appear when a run is **aborted**
+mid-flight (server restart), so a high-volume tool (`createComment`, `searchInPage`,
+`Search_web_search`) shows orphans from aborts, not from real errors. Treat the
+orphan gap as an *upper bound* on hard errors, and cross-check the tool: a gap on a
+structural editor (`patchNode`, `insertNode`, `updatePageJson`, `transformPage`) is
+almost certainly a thrown Yjs-encode error; a gap on `createComment` is mostly aborts.
+
+### 3. Run-level failures → `ai_chat_runs`
+
+`status` ∈ `succeeded | aborted | failed | running`; `error` holds the text. Seen in
+the wild: `Run interrupted by a server restart.` (aborts) and
+`Failed after N attempts. Last error: The service may be temporarily overloaded`
+(LLM provider 529). These are infra/provider, not agent tool misuse.
+
+## Ready-to-use queries
+
+Run all of these via `docker exec gitmost-postgresql psql -U docmost -d docmost -P pager=off -c "…"`.
+
+**Real invocation count per tool** (result parts only — the correct denominator):
+
+```sql
+SELECT elem->>'toolName' AS tool, count(*) AS calls
+FROM ai_chat_messages m, jsonb_array_elements(m.tool_calls) elem
+WHERE jsonb_typeof(m.tool_calls) = 'array' AND elem ? 'output'
+GROUP BY 1 ORDER BY 2 DESC;
+```
+
+**Soft errors per tool** (everything the DB can honestly see):
+
+```sql
+WITH res AS (
+  SELECT elem->>'toolName' AS tool, elem->'output' AS o
+  FROM ai_chat_messages m, jsonb_array_elements(m.tool_calls) elem
+  WHERE jsonb_typeof(m.tool_calls) = 'array' AND elem ? 'output'
+)
+SELECT tool, count(*) AS calls,
+       sum(COALESCE(
+         (o->>'isError') = 'true'
+         OR (o->>'success') = 'false'
+         OR (jsonb_typeof(o->'failed') = 'array' AND o->'failed' <> '[]'::jsonb)
+         OR (o->>'unavailable') = 'true'
+         OR o::text ~* 'error executing tool|validation error'
+       , false)::int) AS soft_errors
+FROM res GROUP BY tool HAVING sum(COALESCE(
+         (o->>'isError') = 'true' OR (o->>'success') = 'false'
+         OR (jsonb_typeof(o->'failed') = 'array' AND o->'failed' <> '[]'::jsonb)
+         OR (o->>'unavailable') = 'true' OR o::text ~* 'error executing tool|validation error'
+       , false)::int) > 0
+ORDER BY soft_errors DESC;
+```
+
+**`editPageText` failure reasons** (the most common real agent mistake — bad `find`):
+
+```sql
+WITH res AS (
+  SELECT elem->'output' AS o
+  FROM ai_chat_messages m, jsonb_array_elements(m.tool_calls) elem
+  WHERE jsonb_typeof(m.tool_calls) = 'array'
+    AND elem->>'toolName' = 'editPageText' AND elem ? 'output'
+)
+SELECT f->>'reason' AS reason, count(*)
+FROM res, jsonb_array_elements(o->'failed') f
+WHERE jsonb_typeof(o->'failed') = 'array'
+GROUP BY 1 ORDER BY 2 DESC;
+```
+
+**Hard-error proxy — orphan gap per tool, WITH a spread column** (call parts minus
+result parts, plus how many distinct chats the gap is spread across):
+
+```sql
+WITH parts AS (
+  SELECT m.chat_id, elem->>'toolName' AS tool,
+         (elem ? 'input' AND NOT (elem ? 'output')) AS is_call,
+         (elem ? 'output')                          AS is_result
+  FROM ai_chat_messages m, jsonb_array_elements(m.tool_calls) elem
+  WHERE jsonb_typeof(m.tool_calls) = 'array' AND m.role = 'assistant'
+),
+per_chat AS (
+  SELECT tool, chat_id, sum(is_call::int) - sum(is_result::int) AS gap
+  FROM parts GROUP BY tool, chat_id
+)
+SELECT tool,
+       sum(gap) FILTER (WHERE gap > 0)   AS missing_results,
+       count(*) FILTER (WHERE gap > 0)   AS chats_spread,   -- disambiguates!
+       max(gap)                          AS worst_single_chat
+FROM per_chat GROUP BY tool
+HAVING sum(gap) FILTER (WHERE gap > 0) > 0
+ORDER BY missing_results DESC;
+```
+
+**`missing_results` mixes thrown errors AND aborted/interrupted runs — you cannot
+split them from `output` alone** (a positional "what follows the orphan" heuristic
+breaks on parallel tool batches, which persist as `call,call,…,result,result`). Use
+`chats_spread` to disambiguate:
+
+- **spread across many chats** (e.g. `createComment` 96 over 29 chats) → a **systemic
+  real error** (here: inline-comment anchor text not found on the page).
+- **concentrated in one chat** (e.g. `searchInPage` 55, of which 51 in a single chat)
+  → **one runaway/aborted session**, not a real per-call error — discount it.
+- a gap on a **structural editor** (`patchNode`, `insertNode`, `tableUpdateCell`,
+  `updatePageJson`, `transformPage`) is almost always a thrown Yjs-encode error.
+
+**Run-level failures:**
+
+```sql
+SELECT status, count(*), min(error) AS sample_error
+FROM ai_chat_runs GROUP BY status ORDER BY 2 DESC;
+```
+
+**Full-text search across messages.** The `tsv` GIN index is built as
+`to_tsvector('english', unaccent(content))` — so it **stems English** but **not
+Russian** (Russian lexemes are stored unstemmed, so only exact word forms match).
+Most content here is Russian, so prefer `ILIKE` for substring search:
+
+```sql
+-- Russian / substring — reliable:
+SELECT chat_id, left(content, 120)
+FROM ai_chat_messages WHERE content ILIKE '%иранск%' LIMIT 20;
+
+-- English phrase — can use the index:
+SELECT chat_id, left(content, 120)
+FROM ai_chat_messages
+WHERE tsv @@ websearch_to_tsquery('english', 'some phrase') LIMIT 20;
+```
+
+## Don't blow up your context
+
+A single `tool_calls` row can be **300–400 KB** (results embed full page content and
+search payloads). Never `SELECT tool_calls` (or `jsonb_pretty(tool_calls)`) raw.
+Always project just the keys you need and truncate:
+
+```sql
+SELECT elem->>'toolName',
+       left(regexp_replace((elem->'output')::text, '\s+', ' ', 'g'), 200)
+FROM ai_chat_messages m, jsonb_array_elements(m.tool_calls) elem
+WHERE elem ? 'output' LIMIT 5;
+```
+
+## Server logs & live UI (for the error text the DB drops)
+
+```bash
+docker logs -f --tail=100 gitmost            # main app
+docker compose -p gitmost logs -f --tail=100 # whole stack
+```
+
+Logging is `json-file`, `max-size=10m max-file=5` → ~50 MB retained, then rotated,
+and **wiped on container recreate**. So thrown-tool error text is only reliably
+caught **in real time** (or in the live chat UI, which renders the failed part with
+its message). There is no durable, queryable store of hard tool errors today — if you
+need one, that is a feature to add (persist `output-error` parts, or emit a
+`tool_calls_total{tool,status}` metric to VictoriaMetrics).
+
+## Gotchas checklist
+
+- [ ] Counting every `tool_calls` element → **2× overcount**. Count elements with `output`.
+- [ ] `isError` / `success=false` ≈ 0 does **not** mean "no errors" — thrown errors aren't persisted.
+- [ ] `editPageText.failed` is `[]` on success — test for **non-empty**, not presence.
+- [ ] Orphan gap mixes thrown errors **and** aborted runs — split by tool before concluding.
+- [ ] `aborted` runs = server restarts, `failed` runs = provider overload — not agent mistakes.
+- [ ] Never dump a raw `tool_calls` cell — it can be hundreds of KB.
+- [ ] Logs are ephemeral (≤50 MB, wiped on recreate) — grab hard-error text live.
+
+## Snapshot (2026-07-07, illustrative — rerun the queries for current numbers)
+
+- 226 chats, 732 messages, 46 runs; ~4 400 real tool invocations.
+- Soft errors (persisted): `editPageText` 4/79 (bad/non-unique `find`) + 9 markdown-in-`find` warnings; `semanticSearch` 3/4 (`unavailable`); `Habr_update_draft_from_docmost` 1/2 (`doc` sent as object, not string).
+- Missing-result proxy, read WITH the spread column:
+  - **Systemic (spread) → real errors:** `createComment` 96 over **29 chats** (comment anchor text not found — the biggest real error hotspot); `editPageText` 31 over 12 chats (+ the 4 soft above); structural-editor Yjs throws `insertNode` 10 / `updatePageContent` 9 / `tableUpdateCell` 6 / `patchNode` 5 / `updatePageJson` 2 / `transformPage` 2.
+  - **Concentrated → NOT real errors:** `searchInPage` 55 (51 in one chat); `Search_web_search` 15 & `Search_searxng_web_search` 6 (timeouts/aborts in long research sessions).
+- Runs: 34 succeeded, 10 aborted (server restart), 1 failed (provider overload).

packages/mcp/src/client.ts

@@ -46,7 +46,7 @@ import {
   insertTableRow,
   deleteTableRow,
   updateTableCell,
-} from "./lib/node-ops.js";
+} from "@docmost/prosemirror-markdown";
 import { searchInDoc, SearchOptions } from "./lib/page-search.js";
 import { withPageLock } from "./lib/page-lock.js";
 import {
@@ -137,6 +137,15 @@ export type DocmostMcpConfig = { apiUrl: string } & (
       has?: (uri: string) => boolean;
       evict?: (uri: string) => void;
     };
+    // Dependency-neutral metrics sink. When present, the client emits generic
+    // (name, value, labels) samples; the HOST maps those names onto its own
+    // metrics registry (the package never depends on prom-client or the server).
+    // Absent in standalone/stdio mode → the client is a complete no-op here.
+    onMetric?: (
+      name: string,
+      value: number,
+      labels?: Record<string, string>,
+    ) => void;
   };
 
 // Canonical UUID shape (versions 1–8, matching the `uuid` package's `validate`
@@ -173,6 +182,11 @@ export class DocmostClient {
   // op's image blobs if the final doc put throws. Null when the sink omits them.
   private sandboxHas: ((uri: string) => boolean) | null = null;
   private sandboxEvict: ((uri: string) => void) | null = null;
+  // Optional dependency-neutral metrics sink (see DocmostMcpConfig.onMetric).
+  // Null on the legacy positional form and whenever the host omits it → no-op.
+  private onMetricFn:
+    | ((name: string, value: number, labels?: Record<string, string>) => void)
+    | null = null;
   // In-flight login dedup: when the token expires, the 401 interceptor,
   // ensureAuthenticated, getCollabTokenWithReauth and the two multipart retries
   // can all call login() at once. Memoizing a single promise collapses that
@@ -222,6 +236,8 @@ export class DocmostClient {
       this.sandboxHas = config.sandbox.has ?? null;
       this.sandboxEvict = config.sandbox.evict ?? null;
     }
+    // Legacy positional form carries no onMetric → null (complete no-op).
+    this.onMetricFn = config.onMetric ?? null;
     this.client = axios.create({
       baseURL: this.apiUrl,
       // Default request timeout so a hung connection cannot wedge a per-page
@@ -447,6 +463,10 @@ export class DocmostClient {
       };
 
       connectTimer = setTimeout(() => {
+        // Only the actual 25s collab connect timeout fires here — the agent's
+        // collab connection to the server never became ready. This is the
+        // connect-vs-unload signal; the other finish() paths must NOT emit it.
+        this.onMetricFn?.("collab_connect_timeouts_total", 1);
         finish(new Error("Connection timeout to collaboration server"));
       }, CONNECT_TIMEOUT_MS);
 

packages/mcp/src/index.ts

@@ -4,7 +4,7 @@ import { readFileSync } from "fs";
 import { fileURLToPath } from "url";
 import { dirname, join } from "path";
 import { DocmostClient, DocmostMcpConfig } from "./client.js";
-import { parseNodeArg } from "./lib/parse-node-arg.js";
+import { parseNodeArg } from "@docmost/prosemirror-markdown";
 import { SHARED_TOOL_SPECS, SharedToolSpec } from "./tool-specs.js";
 
 // Re-export the client and its config type so embedding hosts (e.g. the gitmost
@@ -64,6 +64,33 @@ const jsonContent = (data: any) => ({
  * REST + the collaboration WebSocket using the provided service-account
  * credentials and auto-re-authenticates.
  */
+/**
+ * Wrap a tool handler so its wall-clock duration is reported through the host's
+ * dependency-neutral sink as `mcp_tool_duration_seconds` (labelled by tool
+ * name). Pure and side-effect-free apart from the optional `onMetric` call:
+ *  - preserves the handler's exact return value (awaited);
+ *  - observes in a `finally`, so it records on BOTH success and throw, then
+ *    rethrows the original error unchanged (never swallowed);
+ *  - with no `onMetric` (standalone/stdio) it is a transparent pass-through.
+ * Exported so the timing contract can be unit-tested without a live transport.
+ */
+export function timeToolHandler(
+  name: string,
+  handler: (...args: any[]) => any,
+  onMetric?: (name: string, value: number, labels?: Record<string, string>) => void,
+): (...args: any[]) => Promise<any> {
+  return async (...handlerArgs: any[]) => {
+    const start = performance.now();
+    try {
+      return await handler(...handlerArgs);
+    } finally {
+      onMetric?.("mcp_tool_duration_seconds", (performance.now() - start) / 1000, {
+        tool: name,
+      });
+    }
+  };
+}
+
 export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
   // Pass the whole config union through: the client branches internally on
   // credentials vs. getToken, so both the external /mcp (creds) and the
@@ -78,6 +105,26 @@ export function createDocmostMcpServer(config: DocmostMcpConfig): McpServer {
     { instructions: SERVER_INSTRUCTIONS },
   );
 
+  // Single choke point for MCP tool timing. Both `registerShared` (below) and
+  // the inline `server.registerTool(...)` calls funnel through this one method,
+  // so monkeypatching it HERE — before any tool is registered and before
+  // `registerShared` captures a reference to it — times every tool with no
+  // per-tool boilerplate. The wrapped handler records wall-clock duration and,
+  // in a `finally`, feeds the host's dependency-neutral sink
+  // `config.onMetric("mcp_tool_duration_seconds", seconds, { tool })`. The tool
+  // name is the registration name (bounded cardinality). When no onMetric is
+  // provided (standalone/stdio) the wrapper is a pure pass-through: it still
+  // returns the original result and rethrows the original error unchanged.
+  const originalRegisterTool = server.registerTool.bind(server) as (
+    ...args: any[]
+  ) => any;
+  (server as any).registerTool = (...args: any[]) => {
+    const name = args[0] as string;
+    const handler = args[args.length - 1];
+    const timedHandler = timeToolHandler(name, handler, config.onMetric);
+    return originalRegisterTool(...args.slice(0, -1), timedHandler);
+  };
+
   // Register a tool from the shared, zod-agnostic spec registry. The spec owns
   // the canonical name + model-facing description + (optional) schema builder;
   // only the execute body is supplied per call. buildShape is invoked with THIS
@@ -380,43 +427,10 @@ registerShared(SHARED_TOOL_SPECS.deleteNode, async ({ pageId, nodeId }) => {
 });
 
 // Tool: insert_image
-// MCP-only by design (NOT in the shared registry): the in-app AI-chat agent
-// exposes no image tools (insert/replace), so there is no second layer to unify
-// — a SHARED_TOOL_SPECS entry's tier/catalogLine are in-app metadata and the
-// catalog-partition test forbids a spec without a live in-app tool (#294).
-server.registerTool(
-  "insert_image",
-  {
-    description:
-      "Download an image from a web (http/https) URL and insert it into " +
-      "a page in one step. By default " +
-      "appends the image at the end of the page. With replaceText, replaces the " +
-      "first top-level block whose text contains that string (handy for " +
-      'swapping a text placeholder like "[image: foo.png]" for the real image). ' +
-      "With afterText, inserts the image right after the first block containing " +
-      "that string. Preserves all other block ids.",
-    inputSchema: {
-      pageId: z.string().min(1),
-      imageUrl: z
-        .string()
-        .min(1)
-        .describe("http(s) URL of the image to download and upload"),
-      align: z.enum(["left", "center", "right"]).optional(),
-      alt: z.string().optional(),
-      replaceText: z
-        .string()
-        .optional()
-        .describe(
-          "Replace the first top-level block whose text contains this string with the image",
-        ),
-      afterText: z
-        .string()
-        .optional()
-        .describe(
-          "Insert the image right after the first top-level block whose text contains this string",
-        ),
-    },
-  },
+// Schema + description now live in the shared registry (#410) so BOTH this MCP
+// server and the in-app AI-chat agent expose it. The execute body is unchanged.
+registerShared(
+  SHARED_TOOL_SPECS.insertImage,
   async ({ pageId, imageUrl, align, alt, replaceText, afterText }) => {
     const result = await docmostClient.insertImage(pageId, imageUrl, {
       align,
@@ -429,34 +443,9 @@ server.registerTool(
 );
 
 // Tool: replace_image
-// MCP-only by design (see insert_image): no in-app equivalent, stays inline.
-server.registerTool(
-  "replace_image",
-  {
-    description:
-      "Replace an existing image on a page with a new image fetched from a web " +
-      "(http/https) URL: uploads the new file as a NEW " +
-      "attachment (fresh clean URL that renders and busts browser caches), then " +
-      "repoints every image node referencing the old attachmentId (recursively, " +
-      "incl. callouts/tables) via the live document, preserving comments, " +
-      "alignment and alt. The old attachment is left as an unreferenced orphan " +
-      "(Docmost has no API to delete a single attachment; it is removed only when " +
-      "the page/space is deleted). In-place byte overwrite is avoided because some " +
-      "Docmost versions corrupt the attachment (HTTP 500) on overwrite.",
-    inputSchema: {
-      pageId: z.string().min(1),
-      attachmentId: z
-        .string()
-        .min(1)
-        .describe("attachmentId of the image currently in the page to replace"),
-      imageUrl: z
-        .string()
-        .min(1)
-        .describe("http(s) URL of the new image to download"),
-      align: z.enum(["left", "center", "right"]).optional(),
-      alt: z.string().optional(),
-    },
-  },
+// Schema + description now live in the shared registry (#410).
+registerShared(
+  SHARED_TOOL_SPECS.replaceImage,
   async ({ pageId, attachmentId, imageUrl, align, alt }) => {
     const result = await docmostClient.replaceImage(
       pageId,
@@ -779,36 +768,10 @@ server.registerTool(
 );
 
 // Tool: insert_footnote
-// MCP-only by design (see insert_image): the in-app AI-chat agent exposes no
-// footnote tool, so there is no second layer to unify — stays inline (#294).
-server.registerTool(
-  "insert_footnote",
-  {
-    description:
-      "Insert an AUTHOR-INLINE footnote: you specify only WHERE (anchorText) " +
-      "and WHAT (text). The footnote marker is placed right after anchorText in " +
-      "the body, and the bottom footnotes list + the numbering are derived " +
-      "deterministically server-side. You do NOT assign a number, and you " +
-      "never see or edit the footnotes list — so footnotes cannot end up out " +
-      "of order, orphaned, or as a raw '[^id]' block. If a footnote with the " +
-      "SAME text already exists, its number is REUSED (one definition, several " +
-      "references). The write is atomic and won't clobber concurrent edits; if " +
-      "anchorText is not found, nothing is written and an error is returned.",
-    inputSchema: {
-      pageId: z.string().min(1),
-      anchorText: z
-        .string()
-        .min(1)
-        .describe(
-          "A snippet of existing body text; the footnote marker is inserted " +
-            "immediately after its first occurrence (mark-safe).",
-        ),
-      text: z
-        .string()
-        .min(1)
-        .describe("The footnote content as markdown (becomes the definition)."),
-    },
-  },
+// Schema + description now live in the shared registry (#410) so the in-app
+// AI-chat agent exposes it too. The execute body is unchanged.
+registerShared(
+  SHARED_TOOL_SPECS.insertFootnote,
   async ({ pageId, anchorText, text }) => {
     const result = await docmostClient.insertFootnote(pageId, anchorText, text);
     return jsonContent(result);

packages/mcp/src/lib/collaboration.ts

@@ -14,7 +14,7 @@ import { JSDOM } from "jsdom";
 import { markdownToProseMirror } from "@docmost/prosemirror-markdown";
 import { docmostExtensions, docmostSchema } from "./docmost-schema.js";
 import { withPageLock } from "./page-lock.js";
-import { sanitizeForYjs, findUnstorableAttr } from "./node-ops.js";
+import { sanitizeForYjs, findUnstorableAttr } from "@docmost/prosemirror-markdown";
 import { canonicalizeFootnotes } from "./footnote-canonicalize.js";
 import { summarizeChange, VerifyReport } from "./diff.js";
 

packages/mcp/src/lib/footnote-analyze.ts

@@ -1,136 +1,62 @@
 /**
- * Legacy footnote diagnostics for imported Markdown (issue #166).
+ * Legacy footnote advisory for imported Markdown (issue #166, reduced in #414).
  *
- * A PURE, fence-aware text scan (independent of the Markdown->ProseMirror
- * conversion path, so it reports the same problems for `create_page`,
- * `update_page` and `import_page_markdown`). It never changes the document — the
- * importer still creates the page; this only surfaces footnote problems to the
- * caller so an agent can fix its own markup instead of shipping broken footnotes.
+ * Since #293 STEP 5 the canonical import form is inline `^[body]` footnotes
+ * (handled by `@docmost/prosemirror-markdown`). LEGACY reference-style
+ * `[^id]: …` definition markup is now INERT on import — the importer leaves it as
+ * literal text — so authoring it silently produces broken footnotes (the #410
+ * incident class). Rather than the old, elaborate diagnostics of every problem
+ * SHAPE (dangling/duplicate/empty/in-table) that no longer describe what the
+ * importer builds, this module surfaces ONE advisory warning whenever legacy
+ * reference-style definition syntax is present, nudging the author to the inline
+ * form. It never changes the document — the importer still creates the page.
  *
- * SCOPE after #293 STEP 5: the canonical import form is now inline `^[body]`
- * footnotes (handled by `@docmost/prosemirror-markdown`), where these problems
- * cannot arise. This scan therefore targets the LEGACY reference-style
- * (`[^id]` / `[^id]:`) markup, which is now inert on import (left as literal
- * text). The warnings remain useful as an advisory nudge when an agent still
- * authors the old syntax, but they no longer describe what the importer builds.
- *
- * Detected problems:
- *  - danglingReferences: a `[^id]` reference with no `[^id]:` definition.
- *  - emptyDefinitions:   a `[^id]:` whose (kept) text is empty/whitespace.
- *  - duplicateDefinitions: an id defined by two or more `[^id]:` lines (only the
- *    first would have been kept under the old first-wins import).
- *  - referencesInTables: a `[^id]` marker found in a GFM table row (heuristic:
- *    the line, trimmed, starts with `|`) — footnotes in table cells often do not
- *    render as expected.
+ * The scan is fence-aware: a `[^id]:` line inside a ``` / ~~~ code block is
+ * example text, not markup, so it never triggers the warning.
  */
 
-import {
-  lexFootnoteLines,
-  forEachFootnoteReference,
-} from "./footnote-lex.js";
+/** A legacy footnote DEFINITION line: `[^id]:` at the start of a (non-fenced) line. */
+const FOOTNOTE_DEF_RE = /^\[\^[^\]\s]+\]:/;
+/** Opening/closing code fence marker (``` or ~~~). */
+const FENCE_RE = /^\s*(`{3,}|~{3,})/;
 
-export interface FootnoteDiagnostics {
-  /** Reference ids (distinct, document order) with no matching definition. */
-  danglingReferences: string[];
-  /** Definition ids whose first (kept) text is empty/whitespace. */
-  emptyDefinitions: string[];
-  /** Ids defined by two or more `[^id]:` lines (only the first is kept). */
-  duplicateDefinitions: string[];
-  /** Reference ids found inside a GFM table row (heuristic). */
-  referencesInTables: string[];
-  /** Human-readable warning lines for the tool result (one per problem class). */
-  warnings: string[];
-}
+/** The single advisory shown when legacy reference-style footnotes are present. */
+export const LEGACY_FOOTNOTE_WARNING =
+  "Reference-style footnotes (`[^id]: …`) are not parsed on import and will " +
+  "appear as literal text. Use inline footnotes instead: `^[footnote text]`.";
 
 /**
- * Analyze the footnotes in a Markdown string. Pure; safe to call on any body.
+ * True when `markdown` contains a legacy `[^id]:` definition line OUTSIDE any
+ * code fence. Pure; safe to call on any body.
  */
-export function analyzeFootnotes(markdown: string): FootnoteDiagnostics {
-  // Distinct reference ids in first-appearance order, plus the set of ids seen
-  // inside a table row.
-  const refIds: string[] = [];
-  const refIdSet = new Set<string>();
-  const referencesInTables = new Set<string>();
-  const addRef = (id: string, inTable: boolean) => {
-    if (!refIdSet.has(id)) {
-      refIdSet.add(id);
-      refIds.push(id);
-    }
-    if (inTable) referencesInTables.add(id);
-  };
-
-  // Definition texts per id, in first-appearance order of the id.
-  const defTextsById = new Map<string, string[]>();
-
-  // Same lexer the importer uses, so the analysis matches exactly what import
-  // keeps/strips (#166): fenced lines are inert, definition lines are pulled.
-  for (const tok of lexFootnoteLines(markdown)) {
-    if (tok.inFence) continue;
-    if (tok.definition) {
-      const { id, text } = tok.definition;
-      const arr = defTextsById.get(id);
-      if (arr) arr.push(text);
-      else defTextsById.set(id, [text]);
-      // A definition's TEXT can itself reference another footnote (`[^a]: see
-      // [^b]`); count those so such a `[^b]` is not falsely reported dangling.
-      forEachFootnoteReference(text, (rid) => addRef(rid, false));
+export function hasLegacyFootnoteDefinition(markdown: string): boolean {
+  if (typeof markdown !== "string" || !markdown.includes("[^")) return false;
+  let fence: string | null = null;
+  for (const line of markdown.split("\n")) {
+    const fenceMatch = FENCE_RE.exec(line);
+    if (fenceMatch) {
+      const marker = fenceMatch[1][0];
+      if (fence === null) fence = marker; // opening fence
+      else if (marker === fence) fence = null; // matching closing fence
       continue;
     }
-    const inTable = tok.line.trimStart().startsWith("|");
-    forEachFootnoteReference(tok.line, (id) => addRef(id, inTable));
+    if (fence !== null) continue; // inside a fence: inert example text
+    if (FOOTNOTE_DEF_RE.test(line)) return true;
   }
-
-  const danglingReferences = refIds.filter((id) => !defTextsById.has(id));
-  const duplicateDefinitions: string[] = [];
-  const emptyDefinitions: string[] = [];
-  for (const [id, texts] of defTextsById) {
-    if (texts.length >= 2) duplicateDefinitions.push(id);
-    // First-wins: the kept definition is the first one; flag it if it is blank.
-    if ((texts[0] ?? "").trim().length === 0) emptyDefinitions.push(id);
-  }
-  const tableRefs = [...referencesInTables];
-
-  const warnings: string[] = [];
-  const list = (ids: string[]) => ids.map((id) => `[^${id}]`).join(", ");
-  if (danglingReferences.length > 0) {
-    warnings.push(
-      `Footnote reference(s) with no matching definition: ${list(danglingReferences)} (each will render as an empty footnote in the editor).`,
-    );
-  }
-  if (emptyDefinitions.length > 0) {
-    warnings.push(
-      `Footnote definition(s) with empty text: ${list(emptyDefinitions)}.`,
-    );
-  }
-  if (duplicateDefinitions.length > 0) {
-    warnings.push(
-      `Footnote id(s) defined more than once (only the first definition was kept): ${list(duplicateDefinitions)}.`,
-    );
-  }
-  if (tableRefs.length > 0) {
-    warnings.push(
-      `Footnote marker(s) inside a table row (footnotes in table cells may not render as expected): ${list(tableRefs)}.`,
-    );
-  }
-
-  return {
-    danglingReferences,
-    emptyDefinitions,
-    duplicateDefinitions,
-    referencesInTables: tableRefs,
-    warnings,
-  };
+  return false;
 }
 
 /**
  * The optional `footnoteWarnings` field for a page-write tool result: present
- * (with the warning lines) only when `markdown` has footnote problems, omitted
- * otherwise. One helper so all three call sites (create/update/import) attach the
- * field identically. Spread into the result: `{ ...result, ...footnoteWarningsField(text) }`.
+ * (with the single advisory) only when `markdown` uses legacy reference-style
+ * footnote syntax, omitted otherwise. One helper so all three call sites
+ * (create/update/import) attach the field identically. Spread into the result:
+ * `{ ...result, ...footnoteWarningsField(text) }`.
  */
 export function footnoteWarningsField(markdown: string): {
   footnoteWarnings?: string[];
 } {
-  const { warnings } = analyzeFootnotes(markdown);
-  return warnings.length > 0 ? { footnoteWarnings: warnings } : {};
+  return hasLegacyFootnoteDefinition(markdown)
+    ? { footnoteWarnings: [LEGACY_FOOTNOTE_WARNING] }
+    : {};
 }

packages/mcp/src/lib/footnote-authoring.ts

@@ -1,91 +0,0 @@
-/**
- * Inline-authoring helpers for footnotes (MCP).
- *
- * These build/identify footnote DEFINITION nodes for the author-inline tool
- * (`insertInlineFootnote` in transforms.ts): a content key to de-duplicate notes
- * by text, a definition-node factory, and a fresh uuidv7-style id generator.
- *
- * Split out of `footnote-canonicalize.ts` so that module stays a pure MIRROR of
- * the editor-ext canonicalizer (compositionally symmetric to the editor-ext
- * copy, which keeps its authoring helpers in `footnote-util.ts`). The pure
- * canonicalizer has no dependency on these.
- */
-
-const FOOTNOTE_DEFINITION_NAME = "footnoteDefinition";
-
-function cloneJson<T>(v: T): T {
-  if (typeof structuredClone === "function") return structuredClone(v);
-  return JSON.parse(JSON.stringify(v)) as T;
-}
-
-/**
- * Normalized content key for de-duplicating footnote DEFINITIONS by their text.
- *
- * Two definitions with the same key are the SAME footnote — so the inline
- * authoring tool reuses one id (one number, one definition, several references)
- * instead of minting a second definition. Key = plaintext (whitespace-collapsed,
- * trimmed) PLUS a signature of the inline mark types in order, so two notes that
- * read the same but differ in formatting (one bold, one plain) are NOT merged.
- * Conservative: only an exact match merges.
- */
-export function footnoteContentKey(defNode: any): string {
-  const parts: string[] = [];
-  const visit = (n: any): void => {
-    if (!n || typeof n !== "object") return;
-    if (n.type === "text" && typeof n.text === "string") {
-      const marks = Array.isArray(n.marks)
-        ? n.marks.map((m: any) => m?.type).filter(Boolean).sort().join(",")
-        : "";
-      parts.push(`${n.text}${marks}`);
-    }
-    if (Array.isArray(n.content)) for (const c of n.content) visit(c);
-  };
-  visit(defNode);
-  // Collapse the assembled text's whitespace and trim, keeping the mark
-  // signature attached so formatting differences still distinguish notes.
-  return parts
-    .join("")
-    .replace(/[ \t\r\n]+/g, " ")
-    .trim();
-}
-
-/**
- * Build a footnoteDefinition node from inline ProseMirror nodes, keyed by id.
- */
-export function makeFootnoteDefinition(id: string, inlineNodes: any[]): any {
-  const content = Array.isArray(inlineNodes) ? cloneJson(inlineNodes) : [];
-  return {
-    type: FOOTNOTE_DEFINITION_NAME,
-    attrs: { id },
-    content: [{ type: "paragraph", content }],
-  };
-}
-
-/**
- * Generate a uuidv7-style id (time-ordered), matching editor-ext's
- * `generateFootnoteId`. Used for a genuinely-new inline footnote id.
- */
-export function generateFootnoteId(): string {
-  const now = Date.now();
-  const timeHex = now.toString(16).padStart(12, "0");
-  const rand = (length: number) => {
-    let s = "";
-    for (let i = 0; i < length; i++)
-      s += Math.floor(Math.random() * 16).toString(16);
-    return s;
-  };
-  const versioned = "7" + rand(3);
-  const variantNibble = (8 + Math.floor(Math.random() * 4)).toString(16);
-  const variant = variantNibble + rand(3);
-  return (
-    timeHex.slice(0, 8) +
-    "-" +
-    timeHex.slice(8, 12) +
-    "-" +
-    versioned +
-    "-" +
-    variant +
-    "-" +
-    rand(12)
-  );
-}

packages/mcp/src/lib/footnote-canonicalize.ts

@@ -4,8 +4,8 @@
  * `canonicalizeFootnotes(doc)` is a pure ProseMirror-JSON port of the editor's
  * `footnoteSyncPlugin` end-state, identical in behaviour to
  * `@docmost/editor-ext`'s `canonicalizeFootnotes`. It is mirrored here — rather
- * than imported from editor-ext — for the SAME reason `footnote-lex.ts` and the
- * `docmost-schema.ts` nodes are mirrored: the MCP package is deliberately
+ * than imported from editor-ext — for the SAME reason the `docmost-schema.ts`
+ * nodes are mirrored: the MCP package is deliberately
  * decoupled from the browser/React-heavy editor barrel and operates on plain
  * JSON. The editor-ext copy owns the golden test against the live plugin; this
  * copy must stay behaviourally identical (a SHARED golden corpus, exercised by
@@ -13,8 +13,8 @@
  *
  * This module is the pure MIRROR only. The inline-authoring helpers
  * (`footnoteContentKey`, `makeFootnoteDefinition`, `generateFootnoteId`) used by
- * `insertInlineFootnote` live in the sibling `footnote-authoring.ts`, so this
- * file is compositionally symmetric to the editor-ext copy.
+ * `insertInlineFootnote` live in `@docmost/prosemirror-markdown` (next to the
+ * importer's `assembleFootnotes`, #414), so this file stays a pure mirror.
  *
  * Why it exists: every NON-editor write path (markdown import, update_page_json,
  * docmost_transform, insert_footnote) builds ProseMirror JSON directly, so the

packages/mcp/src/lib/footnote-lex.ts

@@ -1,73 +0,0 @@
-/**
- * Shared, fence-aware line lexer for legacy footnote markdown (MCP-internal).
- *
- * Since #293 STEP 5 the markdown -> ProseMirror IMPORT path lives in the shared
- * `@docmost/prosemirror-markdown` package (inline `^[body]` footnotes), so this
- * lexer no longer backs an mcp importer. It now backs ONLY the import-time
- * diagnostics (`analyzeFootnotes` in footnote-analyze.ts), which still scan the
- * raw markdown for legacy reference-style `[^id]:` definition lines and surface
- * advisory warnings (duplicate/orphan definitions) about content that is now
- * inert on import. Fence-awareness (a `[^id]:` line inside a ``` / ~~~ block is
- * NOT a definition) is the property the analyzer relies on.
- *
- * NOTE: this is deliberately NOT shared with editor-ext's
- * `extractFootnoteDefinitions` — that lives in a different package and the
- * decoupling between the editor and the MCP mirror is intentional.
- */
-
-/** A footnote DEFINITION line: `[^id]: text` (id + text captured). */
-export const FOOTNOTE_DEF_RE = /^\[\^([^\]\s]+)\]:[ \t]*(.*)$/;
-/** Every footnote REFERENCE `[^id]` in a line (global; id captured). */
-export const FOOTNOTE_REF_RE_G = /\[\^([^\]\s]+)\]/g;
-/** Opening/closing code fence marker (``` or ~~~). */
-const FENCE_RE = /^(\s*)(`{3,}|~{3,})/;
-
-export interface FootnoteLine {
-  /** The raw line, verbatim. */
-  line: string;
-  /**
-   * True for a code-fence marker line AND every line inside a fence — footnote
-   * syntax on such lines is inert (example text, not real markup). The importer
-   * keeps these in the body; the analyzer skips them.
-   */
-  inFence: boolean;
-  /** The parsed definition, when this is a `[^id]: text` line OUTSIDE any fence. */
-  definition: { id: string; text: string } | null;
-}
-
-/** Classify every line of `markdown`, tracking fenced-code state. Pure. */
-export function lexFootnoteLines(markdown: string): FootnoteLine[] {
-  const out: FootnoteLine[] = [];
-  let fence: string | null = null;
-  for (const line of markdown.split("\n")) {
-    const fenceMatch = FENCE_RE.exec(line);
-    if (fenceMatch) {
-      const marker = fenceMatch[2][0];
-      if (fence === null) fence = marker; // opening fence
-      else if (marker === fence) fence = null; // matching closing fence
-      out.push({ line, inFence: true, definition: null });
-      continue;
-    }
-    if (fence !== null) {
-      out.push({ line, inFence: true, definition: null });
-      continue;
-    }
-    const m = FOOTNOTE_DEF_RE.exec(line);
-    out.push({
-      line,
-      inFence: false,
-      definition: m ? { id: m[1], text: m[2] } : null,
-    });
-  }
-  return out;
-}
-
-/** Scan a line for every `[^id]` reference, invoking `onRef(id)` for each. */
-export function forEachFootnoteReference(
-  line: string,
-  onRef: (id: string) => void,
-): void {
-  FOOTNOTE_REF_RE_G.lastIndex = 0;
-  let m: RegExpExecArray | null;
-  while ((m = FOOTNOTE_REF_RE_G.exec(line)) !== null) onRef(m[1]);
-}

packages/mcp/src/lib/json-edit.ts

@@ -305,6 +305,21 @@ export function applyTextEdits(
       continue;
     }
 
+    // HARD-REFUSE inline footnote tokens (#410). `^[...]` in a `replace` is
+    // markdown that only becomes a real footnote when a whole markdown body is
+    // written (create_page / update_page_content / import_page_markdown). Written
+    // through edit_page_text it stays a LITERAL string in the text — the exact
+    // failure mode #410 fixes — so refuse it here (defense-in-depth) and point the
+    // caller at insert_footnote, mirroring the formatting-marker refusal above.
+    if (/\^\[[\s\S]*?\]/.test(edit.replace)) {
+      failed.push({
+        find: edit.find,
+        reason:
+          "edit_page_text writes the replacement as LITERAL text, so a `^[...]` footnote token does not parse into a real footnote (it would appear verbatim in the page). To add a footnote to existing text, use insert_footnote (anchorText = where, text = the note).",
+      });
+      continue;
+    }
+
     // Gather every inline block in document order (recurse the whole tree so
     // nested containers — callouts, list items, table cells, blockquotes — are
     // all covered).

packages/mcp/src/lib/node-ops.ts

@@ -1,963 +0,0 @@
-/**
- * Pure, network-free helpers for manipulating a ProseMirror/TipTap document
- * tree by node id.
- *
- * A ProseMirror node here is a plain JSON object of the shape produced by
- * Docmost: `{ type, attrs?, content?, text?, marks? }`. Children live in the
- * `content` array; a node carries a stable id in `attrs.id`. Callouts and
- * table cells hold their children in `content` just like any other block, so a
- * single recursive walk reaches them all.
- *
- * Every exported function operates on a DEEP CLONE of the input document and
- * returns the new document. The input doc and any `newNode`/`node` argument are
- * never mutated. All functions are defensively null-safe: missing/!Array
- * `content`, non-object nodes, and absent `attrs` are tolerated.
- */
-
-import { stripInlineMarkdown } from "./text-normalize.js";
-
-/** Deep-clone a JSON-serializable value without mutating the original. */
-function clone<T>(value: T): T {
-  if (typeof structuredClone === "function") {
-    return structuredClone(value);
-  }
-  // Fallback for environments without structuredClone.
-  return JSON.parse(JSON.stringify(value)) as T;
-}
-
-/** True if `value` is a non-null object (and not an array). */
-function isObject(value: any): value is Record<string, any> {
-  return value != null && typeof value === "object" && !Array.isArray(value);
-}
-
-/** True if `node` carries the given id in `node.attrs.id`. */
-function matchesId(node: any, nodeId: string): boolean {
-  return isObject(node) && isObject(node.attrs) && node.attrs.id === nodeId;
-}
-
-/**
- * Recursively concatenate all text contained in a node.
- *
- * Text nodes contribute their `text` string; container nodes contribute the
- * joined `blockPlainText` of their `content` children. Returns "" for nullish
- * or non-object inputs.
- */
-export function blockPlainText(node: any): string {
-  if (!isObject(node)) return "";
-  let out = "";
-  if (typeof node.text === "string") {
-    out += node.text;
-  }
-  if (Array.isArray(node.content)) {
-    for (const child of node.content) {
-      out += blockPlainText(child);
-    }
-  }
-  return out;
-}
-
-/** Truncate `text` to at most `n` chars, appending an ellipsis when cut. */
-function truncate(text: string, n: number): string {
-  return text.length > n ? text.slice(0, n) + "…" : text;
-}
-
-/** One compact outline entry for a single top-level block. */
-export interface OutlineEntry {
-  index: number;
-  type: string | undefined;
-  id: string | null;
-  firstText: string;
-  /** Present for headings only. */
-  level?: number | null;
-  /** Present for tables only. */
-  rows?: number;
-  cols?: number;
-  header?: string[];
-  /** Present for list blocks only (bulletList/orderedList/taskList). */
-  items?: number;
-}
-
-/**
- * Build a COMPACT outline of the TOP-LEVEL blocks of `doc` (the entries in
- * `doc.content`). Deliberately does NOT recurse into paragraphs, list items, or
- * table cells — compactness is the point; use `getNodeByRef` to drill into a
- * specific block.
- *
- * Each entry carries `{ index, type, id, firstText }`, plus type-specific
- * extras: headings add `level`; tables add `rows`/`cols` and the first row's
- * cell texts as `header`; list blocks (types ending in "List") add `items`.
- * `firstText` is the block's plain text truncated to 100 chars. Null-safe:
- * a missing or non-object doc/content yields `[]`.
- */
-export function buildOutline(doc: any): OutlineEntry[] {
-  if (!isObject(doc) || !Array.isArray(doc.content)) return [];
-
-  const out: OutlineEntry[] = [];
-  for (let i = 0; i < doc.content.length; i++) {
-    const block = doc.content[i];
-    const type = isObject(block) ? block.type : undefined;
-    const entry: OutlineEntry = {
-      index: i,
-      type,
-      id:
-        isObject(block) && isObject(block.attrs)
-          ? (block.attrs.id ?? null)
-          : null,
-      firstText: truncate(blockPlainText(block), 100),
-    };
-
-    if (type === "heading") {
-      entry.level = isObject(block.attrs) ? (block.attrs.level ?? null) : null;
-    } else if (type === "table") {
-      const headerRow = block.content?.[0]?.content ?? [];
-      entry.rows = block.content?.length ?? 0;
-      entry.cols = block.content?.[0]?.content?.length ?? 0;
-      entry.header = headerRow.map((cell: any) =>
-        truncate(blockPlainText(cell), 40),
-      );
-    } else if (typeof type === "string" && type.endsWith("List")) {
-      entry.items = block.content?.length ?? 0;
-    }
-
-    out.push(entry);
-  }
-  return out;
-}
-
-/**
- * Resolve a single node by reference and return `{ node, path, type }`, or
- * `null` when nothing matches.
- *
- * - `ref` of the form `#<n>` (e.g. `#2`) selects the TOP-LEVEL block at index
- *   `n` in `doc.content`. This is the only way to address table/tableRow/
- *   tableCell nodes, which carry no `attrs.id`.
- * - Otherwise `ref` is treated as a block id: the FIRST node anywhere in the
- *   tree with `attrs.id === ref` is returned.
- *
- * `path` is the array of child indices from the doc root down to the node
- * (so a top-level block is `[index]`). The returned `node` is a DEEP CLONE,
- * so callers can mutate it without touching the input doc. Null-safe.
- */
-export function getNodeByRef(
-  doc: any,
-  ref: string,
-): { node: any; path: number[]; type: string | undefined } | null {
-  if (!isObject(doc)) return null;
-
-  // "#<n>": index into the top-level content array.
-  const indexMatch = typeof ref === "string" ? ref.match(/^#(\d+)$/) : null;
-  if (indexMatch) {
-    const index = Number(indexMatch[1]);
-    const block = Array.isArray(doc.content) ? doc.content[index] : undefined;
-    if (!isObject(block)) return null;
-    return { node: clone(block), path: [index], type: block.type };
-  }
-
-  // Otherwise: depth-first search for the first node with attrs.id === ref.
-  const search = (
-    node: any,
-    trail: number[],
-  ): { node: any; path: number[]; type: string } | null => {
-    if (!isObject(node)) return null;
-    if (Array.isArray(node.content)) {
-      for (let i = 0; i < node.content.length; i++) {
-        const child = node.content[i];
-        const path = [...trail, i];
-        if (matchesId(child, ref)) {
-          return { node: clone(child), path, type: child.type };
-        }
-        const hit = search(child, path);
-        if (hit != null) return hit;
-      }
-    }
-    return null;
-  };
-
-  return search(doc, []);
-}
-
-/**
- * Replace EVERY node whose `attrs.id === nodeId` with a deep clone of
- * `newNode`, anywhere in the tree (including inside callouts and table cells).
- *
- * Operates on a clone of `doc`; returns `{ doc, replaced }` where `replaced`
- * is the number of nodes substituted. A fresh clone of `newNode` is used for
- * each match so they do not share references.
- */
-export function replaceNodeById(
-  doc: any,
-  nodeId: string,
-  newNode: any,
-): { doc: any; replaced: number } {
-  const out = clone(doc);
-  let replaced = 0;
-
-  // Walk a content array, replacing direct matches and recursing into the
-  // (possibly new) children of non-matching nodes.
-  const walkContent = (content: any[]): void => {
-    for (let i = 0; i < content.length; i++) {
-      const child = content[i];
-      if (matchesId(child, nodeId)) {
-        content[i] = clone(newNode);
-        replaced++;
-        // Do not recurse into a freshly substituted node.
-        continue;
-      }
-      if (isObject(child) && Array.isArray(child.content)) {
-        walkContent(child.content);
-      }
-    }
-  };
-
-  if (isObject(out) && Array.isArray(out.content)) {
-    walkContent(out.content);
-  }
-  return { doc: out, replaced };
-}
-
-/**
- * Remove EVERY node whose `attrs.id === nodeId` from its parent `content`
- * array, anywhere in the tree (recursive, including callouts and tables).
- *
- * Operates on a clone of `doc`; returns `{ doc, deleted }` where `deleted` is
- * the number of nodes removed.
- */
-export function deleteNodeById(
-  doc: any,
-  nodeId: string,
-): { doc: any; deleted: number } {
-  const out = clone(doc);
-  let deleted = 0;
-
-  // Filter a content array in place, dropping matches and recursing into the
-  // surviving children.
-  const walkContent = (content: any[]): any[] => {
-    const kept: any[] = [];
-    for (const child of content) {
-      if (matchesId(child, nodeId)) {
-        deleted++;
-        continue;
-      }
-      if (isObject(child) && Array.isArray(child.content)) {
-        child.content = walkContent(child.content);
-      }
-      kept.push(child);
-    }
-    return kept;
-  };
-
-  if (isObject(out) && Array.isArray(out.content)) {
-    out.content = walkContent(out.content);
-  }
-  return { doc: out, deleted };
-}
-
-/**
- * Throw a clear, model-actionable error when a node-id write op did NOT match
- * exactly one node (#159). `count === 0` -> "no node found"; `count > 1` ->
- * "ambiguous, refused" — Docmost duplicates block ids on copy/paste, so a write
- * by id could clobber/remove EVERY duplicate. The caller skips the write for any
- * `count !== 1` (the transform returns null), so this only REPORTS; nothing was
- * changed. No-op for the unambiguous single-match case.
- */
-export function assertUnambiguousMatch(
-  op: "patch_node" | "delete_node",
-  verb: "replace" | "delete",
-  count: number,
-  nodeId: string,
-  pageId: string,
-): void {
-  if (count === 0) {
-    throw new Error(
-      `${op}: no node with id "${nodeId}" found on page ${pageId}`,
-    );
-  }
-  if (count > 1) {
-    throw new Error(
-      `${op}: id "${nodeId}" is ambiguous — ${count} nodes on page ${pageId} share it (block ids are duplicated on copy/paste). Refusing to ${verb} all of them; nothing was changed. Re-target with a more specific anchor.`,
-    );
-  }
-}
-
-/**
- * Deep-clone `doc` and strip every node/mark attribute whose value is strictly
- * `undefined`, so the result is safe to hand to Yjs (which throws an opaque
- * "Unexpected content type" when asked to store an `undefined` attribute value).
- *
- * Only `undefined` keys are removed; `null`, `false`, `0`, and `""` are all
- * legitimate JSON-storable values and are preserved. Operates on a clone and
- * returns it; the input is never mutated. Defensively null-safe like the rest
- * of the file.
- */
-export function sanitizeForYjs(doc: any): any {
-  const out = clone(doc);
-
-  // Drop every key whose value is strictly `undefined` from an attrs object.
-  const stripUndefined = (attrs: any): void => {
-    if (!isObject(attrs)) return;
-    for (const key of Object.keys(attrs)) {
-      if (attrs[key] === undefined) {
-        delete attrs[key];
-      }
-    }
-  };
-
-  const walk = (node: any): void => {
-    if (!isObject(node)) return;
-    stripUndefined(node.attrs);
-    if (Array.isArray(node.marks)) {
-      for (const mark of node.marks) {
-        if (isObject(mark)) stripUndefined(mark.attrs);
-      }
-    }
-    if (Array.isArray(node.content)) {
-      for (const child of node.content) {
-        walk(child);
-      }
-    }
-  };
-
-  walk(out);
-  return out;
-}
-
-/**
- * Diagnostics helper: walk the tree and return a human-readable path string for
- * the FIRST attribute value (in any `node.attrs` or `mark.attrs`) that Yjs
- * cannot store — i.e. `undefined`, a `function`, a `symbol`, or a `bigint`
- * (e.g. `content[3].content[0].attrs.indent (undefined)`). Returns `null` when
- * every attribute is storable. Null-safe.
- */
-export function findUnstorableAttr(doc: any): string | null {
-  const isUnstorable = (value: any): string | null => {
-    if (value === undefined) return "undefined";
-    const t = typeof value;
-    if (t === "function") return "function";
-    if (t === "symbol") return "symbol";
-    if (t === "bigint") return "bigint";
-    return null;
-  };
-
-  // Check an attrs object; return the offending sub-path or null.
-  const checkAttrs = (attrs: any, basePath: string): string | null => {
-    if (!isObject(attrs)) return null;
-    for (const key of Object.keys(attrs)) {
-      const kind = isUnstorable(attrs[key]);
-      if (kind != null) return `${basePath}.${key} (${kind})`;
-    }
-    return null;
-  };
-
-  const walk = (node: any, path: string): string | null => {
-    if (!isObject(node)) return null;
-    const attrHit = checkAttrs(node.attrs, `${path}.attrs`);
-    if (attrHit != null) return attrHit;
-    if (Array.isArray(node.marks)) {
-      for (let i = 0; i < node.marks.length; i++) {
-        const markHit = checkAttrs(
-          node.marks[i]?.attrs,
-          `${path}.marks[${i}].attrs`,
-        );
-        if (markHit != null) return markHit;
-      }
-    }
-    if (Array.isArray(node.content)) {
-      for (let i = 0; i < node.content.length; i++) {
-        const childHit = walk(node.content[i], `${path}.content[${i}]`);
-        if (childHit != null) return childHit;
-      }
-    }
-    return null;
-  };
-
-  // The root doc node carries no useful index, so start the path at "doc".
-  if (!isObject(doc)) return null;
-  const attrHit = checkAttrs(doc.attrs, "attrs");
-  if (attrHit != null) return attrHit;
-  if (Array.isArray(doc.content)) {
-    for (let i = 0; i < doc.content.length; i++) {
-      const childHit = walk(doc.content[i], `content[${i}]`);
-      if (childHit != null) return childHit;
-    }
-  }
-  return null;
-}
-
-/**
- * Table structural node types and the container each must live directly inside.
- * Used by `insertNodeRelative` to splice rows/cells into the correct ancestor
- * rather than blindly into the anchor's direct parent (which would corrupt the
- * table's nesting).
- */
-const STRUCTURAL_TYPES = new Set(["tableRow", "tableCell", "tableHeader"]);
-const REQUIRED_CONTAINER: Record<string, string> = {
-  tableRow: "table",
-  tableCell: "tableRow",
-  tableHeader: "tableRow",
-};
-
-/**
- * Find the index of the first TOP-LEVEL block whose plain text includes the
- * anchor, with a markdown-stripping FALLBACK. Returns -1 when none matches.
- *
- * Two passes preserve "exact wins globally":
- *  - Pass 1: first block containing the verbatim `anchorText`.
- *  - Pass 2 (only if pass 1 found nothing): first block containing the
- *    markdown-stripped anchor, when stripping actually changed it.
- */
-function findAnchorTextIndex(content: any[], anchorText: string): number {
-  if (!Array.isArray(content)) return -1;
-  // Pass 1: exact.
-  for (let i = 0; i < content.length; i++) {
-    if (blockPlainText(content[i]).includes(anchorText)) return i;
-  }
-  // Pass 2: markdown-stripped fallback.
-  const a = stripInlineMarkdown(anchorText);
-  if (a !== anchorText && a.length > 0) {
-    for (let i = 0; i < content.length; i++) {
-      if (blockPlainText(content[i]).includes(a)) return i;
-    }
-  }
-  return -1;
-}
-
-/**
- * Locate an anchor and return its ancestor chain (from `doc` down to and
- * including the matched node). Each chain entry is `{ node, index }` where
- * `index` is the node's position inside its parent's `content` array (the root
- * doc has index -1). Returns `null` when the anchor cannot be resolved.
- */
-function findAnchorChain(
-  doc: any,
-  opts: InsertOptions,
-): { node: any; index: number }[] | null {
-  if (!isObject(doc)) return null;
-
-  // DFS by id anywhere in the tree, accumulating the path.
-  if (opts.anchorNodeId != null) {
-    const targetId = opts.anchorNodeId;
-    const search = (
-      node: any,
-      index: number,
-      trail: { node: any; index: number }[],
-    ): { node: any; index: number }[] | null => {
-      if (!isObject(node)) return null;
-      const here = [...trail, { node, index }];
-      if (matchesId(node, targetId)) return here;
-      if (Array.isArray(node.content)) {
-        for (let i = 0; i < node.content.length; i++) {
-          const hit = search(node.content[i], i, here);
-          if (hit != null) return hit;
-        }
-      }
-      return null;
-    };
-    return search(doc, -1, []);
-  }
-
-  // By text: only top-level blocks are scanned (same rule as the JSON path).
-  // Exact match wins; a markdown-stripped fallback is tried only on a miss.
-  if (opts.anchorText != null && Array.isArray(doc.content)) {
-    const i = findAnchorTextIndex(doc.content, opts.anchorText);
-    if (i !== -1) {
-      return [
-        { node: doc, index: -1 },
-        { node: doc.content[i], index: i },
-      ];
-    }
-  }
-
-  return null;
-}
-
-/** Options controlling where `insertNodeRelative` places the new node. */
-export interface InsertOptions {
-  position: "before" | "after" | "append";
-  /** Resolve the anchor by node id anywhere in the tree (preferred). */
-  anchorNodeId?: string;
-  /** Fallback: first TOP-LEVEL block whose plain text includes this string. */
-  anchorText?: string;
-}
-
-/**
- * Insert a deep clone of `node` relative to an anchor.
- *
- * - position "append": push the node onto the top-level `doc.content`.
- * - position "before"/"after": locate the anchor and splice the node into the
- *   anchor's parent `content` array immediately before / after it.
- *
- * Anchor resolution for before/after:
- *   - if `anchorNodeId` is given, find the node with `attrs.id === anchorNodeId`
- *     anywhere in the tree (recursive);
- *   - otherwise, if `anchorText` is given, scan only TOP-LEVEL `doc.content`
- *     blocks and pick the first whose `blockPlainText` includes `anchorText`.
- *
- * Operates on a clone of `doc`; returns `{ doc, inserted }`. `inserted` is
- * false when the anchor could not be resolved (the doc is returned unchanged
- * apart from being cloned).
- */
-export function insertNodeRelative(
-  doc: any,
-  node: any,
-  opts: InsertOptions,
-): { doc: any; inserted: boolean } {
-  const out = clone(doc);
-  const fresh = clone(node);
-
-  // Defensive: stay null-safe like the other exports — a missing opts means
-  // there is nothing actionable to do.
-  if (!isObject(opts)) return { doc: out, inserted: false };
-
-  const isStructural = isObject(node) && STRUCTURAL_TYPES.has(node.type);
-
-  // "append": top-level push.
-  if (opts.position === "append") {
-    // Structural table nodes (tableRow/tableCell/tableHeader) cannot live at the
-    // top level — appending one would produce invalid nesting.
-    if (isStructural) {
-      throw new Error(
-        `insert_node: cannot append a ${node.type} at the top level; use ` +
-          `position before/after with an anchor inside the target table`,
-      );
-    }
-    if (isObject(out)) {
-      if (!Array.isArray(out.content)) out.content = [];
-      out.content.push(fresh);
-      return { doc: out, inserted: true };
-    }
-    return { doc: out, inserted: false };
-  }
-
-  const offset = opts.position === "after" ? 1 : 0;
-
-  // Structural insert (before/after a tableRow/tableCell/tableHeader): splice
-  // into the nearest enclosing table/tableRow rather than the anchor's direct
-  // parent, so the row/cell lands at the correct level of the table.
-  if (isStructural) {
-    const containerType = REQUIRED_CONTAINER[node.type];
-    const chain = findAnchorChain(out, opts);
-    // Anchor not resolved at all — keep the existing "anchor not found" path.
-    if (chain == null) return { doc: out, inserted: false };
-
-    // Find the DEEPEST ancestor (including the anchor itself) of the required
-    // container type.
-    let containerIdx = -1;
-    for (let i = chain.length - 1; i >= 0; i--) {
-      if (isObject(chain[i].node) && chain[i].node.type === containerType) {
-        containerIdx = i;
-        break;
-      }
-    }
-
-    if (containerIdx === -1) {
-      throw new Error(
-        `insert_node: cannot insert a ${node.type} here — the anchor is not ` +
-          `inside a ${containerType}. Anchor on a cell's text or a block id ` +
-          `that lives inside the target table.`,
-      );
-    }
-
-    const container = chain[containerIdx].node;
-    if (!Array.isArray(container.content)) container.content = [];
-
-    if (containerIdx === chain.length - 1) {
-      // The matched container IS the anchor node itself (e.g. anchorText
-      // resolved to the table block): append/prepend within it.
-      const at = opts.position === "after" ? container.content.length : 0;
-      container.content.splice(at, 0, fresh);
-    } else {
-      // The immediate child on the path leading to the anchor is the row/cell
-      // to splice next to.
-      const enclosingChildIndex = chain[containerIdx + 1].index;
-      container.content.splice(enclosingChildIndex + offset, 0, fresh);
-    }
-    return { doc: out, inserted: true };
-  }
-
-  // Resolve by id anywhere in the tree: splice into the parent content array.
-  if (opts.anchorNodeId != null) {
-    let inserted = false;
-    const walkContent = (content: any[]): void => {
-      for (let i = 0; i < content.length; i++) {
-        const child = content[i];
-        if (matchesId(child, opts.anchorNodeId as string)) {
-          content.splice(i + offset, 0, fresh);
-          inserted = true;
-          return;
-        }
-        if (isObject(child) && Array.isArray(child.content)) {
-          walkContent(child.content);
-          if (inserted) return;
-        }
-      }
-    };
-    if (isObject(out) && Array.isArray(out.content)) {
-      walkContent(out.content);
-    }
-    return { doc: out, inserted };
-  }
-
-  // Resolve by text: only top-level doc.content blocks are scanned. Exact
-  // match wins; a markdown-stripped fallback is tried only on a miss.
-  if (opts.anchorText != null && isObject(out) && Array.isArray(out.content)) {
-    const i = findAnchorTextIndex(out.content, opts.anchorText);
-    if (i !== -1) {
-      out.content.splice(i + offset, 0, fresh);
-      return { doc: out, inserted: true };
-    }
-  }
-
-  return { doc: out, inserted: false };
-}
-
-// ===========================================================================
-// Table editing helpers
-//
-// A Docmost table is a ProseMirror subtree with NO ids on the structural nodes:
-//   table   -> { type:"table",     content:[tableRow...] }
-//   row     -> { type:"tableRow",  content:[tableCell|tableHeader...] }
-//   cell    -> { type:"tableCell"|"tableHeader", attrs:{colspan,rowspan,colwidth},
-//                content:[paragraph...] }
-//   para    -> { type:"paragraph", attrs:{id,indent}, content:[textNode...] }
-// Only paragraphs/headings carry an `attrs.id`, so a cell is addressed via the
-// id of the paragraph inside it. The helpers below all operate on a DEEP CLONE
-// of the input doc (via `clone`) and never mutate their inputs.
-// ===========================================================================
-
-/**
- * Collect EVERY `attrs.id` present anywhere in `node` into `used`. Used to seed
- * `makeFreshId` so generated paragraph ids never collide with existing ones.
- */
-function collectIds(node: any, used: Set<string>): void {
-  if (!isObject(node)) return;
-  if (isObject(node.attrs) && typeof node.attrs.id === "string") {
-    used.add(node.attrs.id);
-  }
-  if (Array.isArray(node.content)) {
-    for (const child of node.content) collectIds(child, used);
-  }
-}
-
-/**
- * Fresh-id generator: returns a random Docmost-style id (12 chars from
- * lowercase `a-z0-9`) that is not already in `used`, and records it. On the
- * rare collision the id is regenerated. Callers rely on uniqueness, not on the
- * exact string, so randomness is fine — and unlike a module-local counter it
- * needs no reset and cannot become predictable across calls.
- */
-function makeFreshId(used: Set<string>): string {
-  const alphabet = "abcdefghijklmnopqrstuvwxyz0123456789";
-  let id: string;
-  do {
-    id = "";
-    for (let i = 0; i < 12; i++) {
-      id += alphabet[Math.floor(Math.random() * alphabet.length)];
-    }
-  } while (used.has(id) || id === "");
-  used.add(id);
-  return id;
-}
-
-/**
- * Resolve a table reference against an ALREADY-CLONED doc and return the LIVE
- * table node (a reference inside `rootClone`, so the caller may mutate it) plus
- * its index path. Returns null when no table matches.
- *
- * - `#<n>`: the top-level block at index `n`, only if its `type === "table"`.
- * - otherwise: DFS for the node with `attrs.id === tableRef`, then walk UP its
- *   ancestor chain to the nearest `type === "table"` ancestor.
- */
-function locateTable(
-  rootClone: any,
-  tableRef: string,
-): { table: any; path: number[] } | null {
-  if (!isObject(rootClone)) return null;
-
-  // "#<n>": index into the top-level content array; must be a table.
-  const indexMatch =
-    typeof tableRef === "string" ? tableRef.match(/^#(\d+)$/) : null;
-  if (indexMatch) {
-    const index = Number(indexMatch[1]);
-    const block = Array.isArray(rootClone.content)
-      ? rootClone.content[index]
-      : undefined;
-    if (isObject(block) && block.type === "table") {
-      return { table: block, path: [index] };
-    }
-    return null;
-  }
-
-  // Otherwise: DFS for attrs.id === tableRef, tracking the ancestor chain, then
-  // climb to the nearest enclosing table.
-  const search = (
-    node: any,
-    trail: { node: any; index: number }[],
-  ): { table: any; path: number[] } | null => {
-    if (!isObject(node)) return null;
-    if (Array.isArray(node.content)) {
-      for (let i = 0; i < node.content.length; i++) {
-        const child = node.content[i];
-        const here = [...trail, { node: child, index: i }];
-        if (matchesId(child, tableRef)) {
-          // Walk UP to the nearest table ancestor (including the match itself).
-          for (let j = here.length - 1; j >= 0; j--) {
-            if (isObject(here[j].node) && here[j].node.type === "table") {
-              return {
-                table: here[j].node,
-                path: here.slice(0, j + 1).map((e) => e.index),
-              };
-            }
-          }
-          return null; // id found but no enclosing table
-        }
-        const hit = search(child, here);
-        if (hit != null) return hit;
-      }
-    }
-    return null;
-  };
-
-  return search(rootClone, []);
-}
-
-/** Build the plain-text → single-paragraph cell content used by all writers. */
-function makeCellParagraph(id: string, text: string): any {
-  return {
-    type: "paragraph",
-    attrs: { id, indent: 0 },
-    // Empty string → a paragraph with an empty content array.
-    content: text ? [{ type: "text", text }] : [],
-  };
-}
-
-/**
- * Read a table as a matrix. Returns null when `tableRef` resolves to no table.
- *
- * - `rows`/`cols`: the table's row count and the column count of its FIRST row.
- *   Tables may be ragged (rows of differing length), so `cols` reflects only
- *   row 0; use the per-row length of `cells`/`cellIds` for each row's actual
- *   width.
- * - `cells`: `string[][]` of each cell's `blockPlainText`.
- * - `cellIds`: `(string|null)[][]` of each cell's FIRST paragraph id (or null),
- *   so callers can `patch_node` a cell for rich-formatted edits.
- * - `path`: index path of the table within the doc.
- */
-export function readTable(
-  doc: any,
-  tableRef: string,
-): {
-  rows: number;
-  cols: number;
-  cells: string[][];
-  cellIds: (string | null)[][];
-  path: number[];
-} | null {
-  const root = clone(doc);
-  const located = locateTable(root, tableRef);
-  if (located == null) return null;
-  const { table, path } = located;
-
-  const rowNodes = Array.isArray(table.content) ? table.content : [];
-  const rows = rowNodes.length;
-  const cols = rowNodes[0]?.content?.length ?? 0;
-
-  const cells: string[][] = [];
-  const cellIds: (string | null)[][] = [];
-  for (const rowNode of rowNodes) {
-    const cellNodes = Array.isArray(rowNode?.content) ? rowNode.content : [];
-    const rowText: string[] = [];
-    const rowIds: (string | null)[] = [];
-    for (const cellNode of cellNodes) {
-      rowText.push(blockPlainText(cellNode));
-      // The cell's first paragraph carries the id used for patch_node.
-      const firstPara = Array.isArray(cellNode?.content)
-        ? cellNode.content[0]
-        : undefined;
-      const id =
-        isObject(firstPara) && isObject(firstPara.attrs)
-          ? (firstPara.attrs.id ?? null)
-          : null;
-      rowIds.push(id);
-    }
-    cells.push(rowText);
-    cellIds.push(rowIds);
-  }
-
-  return { rows, cols, cells, cellIds, path };
-}
-
-/**
- * Insert a row of plain-text cells into a table. Returns `{ doc, inserted }`.
- *
- * The row is padded to the table's column count (`cells[i] ?? ""`); supplying
- * MORE cells than columns throws. Each new cell copies `colwidth` for its
- * column from the header row when present, gets a fresh-id paragraph, and a
- * `colspan:1, rowspan:1` attrs. `index` (when an integer in `[0, rows]`) splices
- * the row there; otherwise the row is appended at the end.
- */
-export function insertTableRow(
-  doc: any,
-  tableRef: string,
-  cells: string[],
-  index?: number,
-): { doc: any; inserted: boolean } {
-  const out = clone(doc);
-  const located = locateTable(out, tableRef);
-  if (located == null) return { doc: out, inserted: false };
-  const { table } = located;
-
-  if (!Array.isArray(table.content)) table.content = [];
-  const rows = table.content.length;
-  const headerRow = table.content[0];
-  const headerCells = Array.isArray(headerRow?.content)
-    ? headerRow.content
-    : [];
-
-  // Column count is the WIDEST existing row, so the guard below stays
-  // meaningful for ragged tables and the new row matches the table's width.
-  // Fall back to the supplied cell count only when the table has no rows.
-  let colCount = 0;
-  for (const r of table.content) {
-    if (isObject(r) && Array.isArray(r.content))
-      colCount = Math.max(colCount, r.content.length);
-  }
-  if (colCount === 0) colCount = Array.isArray(cells) ? cells.length : 0;
-
-  if (Array.isArray(cells) && cells.length > colCount) {
-    throw new Error(
-      `table_insert_row: got ${cells.length} cell(s) but the table has ${colCount} column(s)`,
-    );
-  }
-
-  // Resolve the landing index up front so the cell-type decision and the splice
-  // below agree: a valid integer in [0, rows] splices there, else we append.
-  const landingIndex =
-    typeof index === "number" &&
-    Number.isInteger(index) &&
-    index >= 0 &&
-    index <= rows
-      ? index
-      : rows;
-
-  // Seed the id generator with every id already in the doc so the new cell
-  // paragraph ids are unique within the whole document.
-  const used = new Set<string>();
-  collectIds(out, used);
-
-  const newCells: any[] = [];
-  for (let i = 0; i < colCount; i++) {
-    const text = (Array.isArray(cells) ? cells[i] : undefined) ?? "";
-    const attrs: Record<string, any> = { colspan: 1, rowspan: 1 };
-    // Copy this column's colwidth from the header row's cell when present.
-    const colwidth = headerCells[i]?.attrs?.colwidth;
-    if (colwidth !== undefined) attrs.colwidth = colwidth;
-    // A row landing at index 0 becomes the new header row, so inherit the
-    // current header cell's type per column (Docmost uses "tableHeader" there);
-    // every other position is a plain data cell.
-    const cellType =
-      landingIndex === 0 ? (headerCells[i]?.type ?? "tableCell") : "tableCell";
-    newCells.push({
-      type: cellType,
-      attrs,
-      content: [makeCellParagraph(makeFreshId(used), text)],
-    });
-  }
-
-  const newRow = { type: "tableRow", content: newCells };
-
-  // Splice at the resolved landing index (append when index was omitted/invalid).
-  table.content.splice(landingIndex, 0, newRow);
-
-  return { doc: out, inserted: true };
-}
-
-/**
- * Delete the row at 0-based `index` from a table. Returns `{ doc, deleted }`.
- * `deleted` is false only when the table cannot be located. Throws on an
- * out-of-range index, and refuses to delete the table's only row.
- */
-export function deleteTableRow(
-  doc: any,
-  tableRef: string,
-  index: number,
-): { doc: any; deleted: boolean } {
-  const out = clone(doc);
-  const located = locateTable(out, tableRef);
-  if (located == null) return { doc: out, deleted: false };
-  const { table } = located;
-
-  if (!Array.isArray(table.content)) table.content = [];
-  const rows = table.content.length;
-
-  if (!Number.isInteger(index) || index < 0 || index >= rows) {
-    throw new Error(
-      `table_delete_row: row index ${index} out of range (table has ${rows} row(s))`,
-    );
-  }
-  if (rows <= 1) {
-    throw new Error(
-      "table_delete_row: refusing to delete the only row of the table",
-    );
-  }
-
-  table.content.splice(index, 1);
-  return { doc: out, deleted: true };
-}
-
-/**
- * Set the plain-text content of cell `[row, col]` (0-based) to `text`. Returns
- * `{ doc, updated }`; `updated` is false only when the table cannot be located.
- * Throws when `row`/`col` is out of range. The cell's own attrs (colspan/
- * rowspan/colwidth) are preserved; its content becomes a single text paragraph
- * that reuses the cell's existing first-paragraph id when present, else a fresh
- * one.
- */
-export function updateTableCell(
-  doc: any,
-  tableRef: string,
-  row: number,
-  col: number,
-  text: string,
-): { doc: any; updated: boolean } {
-  const out = clone(doc);
-  const located = locateTable(out, tableRef);
-  if (located == null) return { doc: out, updated: false };
-  const { table } = located;
-
-  const rowNodes = Array.isArray(table.content) ? table.content : [];
-  const rows = rowNodes.length;
-  const rowNode = rowNodes[row];
-  const cols =
-    isObject(rowNode) && Array.isArray(rowNode.content)
-      ? rowNode.content.length
-      : 0;
-
-  if (
-    !Number.isInteger(row) ||
-    row < 0 ||
-    row >= rows ||
-    !Number.isInteger(col) ||
-    col < 0 ||
-    col >= cols
-  ) {
-    throw new Error(`table_update_cell: cell [${row},${col}] out of range`);
-  }
-
-  const cellNode = rowNode.content[col];
-  // Reuse the cell's existing first-paragraph id, or mint a fresh unique one.
-  const existingPara = Array.isArray(cellNode?.content)
-    ? cellNode.content[0]
-    : undefined;
-  let id =
-    isObject(existingPara) && isObject(existingPara.attrs)
-      ? existingPara.attrs.id
-      : undefined;
-  if (typeof id !== "string" || id.length === 0) {
-    const used = new Set<string>();
-    collectIds(out, used);
-    id = makeFreshId(used);
-  }
-
-  cellNode.content = [makeCellParagraph(id, text)];
-  return { doc: out, updated: true };
-}

packages/mcp/src/lib/page-search.ts

@@ -33,7 +33,7 @@
 
 import RE2 from "re2";
 
-import { blockPlainText } from "./node-ops.js";
+import { blockPlainText } from "@docmost/prosemirror-markdown";
 
 /** An RE2 regex instance (RE2 extends `RegExp`, so it is usable as one). */
 type Re2Regex = InstanceType<typeof RE2>;

packages/mcp/src/lib/transforms.ts

@@ -14,13 +14,13 @@
  *  - `marks` arrays are preserved verbatim when fragments are split/reordered.
  */
 
-import { blockPlainText } from "./node-ops.js";
-import { canonicalizeFootnotes } from "./footnote-canonicalize.js";
 import {
+  blockPlainText,
   footnoteContentKey,
   makeFootnoteDefinition,
   generateFootnoteId,
-} from "./footnote-authoring.js";
+} from "@docmost/prosemirror-markdown";
+import { canonicalizeFootnotes } from "./footnote-canonicalize.js";
 
 export { canonicalizeFootnotes } from "./footnote-canonicalize.js";
 
@@ -365,7 +365,7 @@ export function noteItem(inlineNodes: any[]): any {
  *   { type:"footnoteDefinition", attrs:{id}, content:[{ type:"paragraph", content }] }
  * (mirrors the editor-ext / docmost-schema FootnoteDefinition node).
  *
- * Built on the shared `makeFootnoteDefinition` factory (footnote-authoring.ts);
+ * Built on the shared `makeFootnoteDefinition` factory (`@docmost/prosemirror-markdown`);
  * the only extra is a fresh block id on the inner paragraph (Docmost stamps one,
  * and the canonicalizer preserves attrs as-is). Single factory, one place to
  * change the definition shape.

packages/mcp/src/tool-specs.ts

@@ -1003,4 +1003,116 @@ export const SHARED_TOOL_SPECS = {
       text: z.string().describe('The new cell text.'),
     }),
   },
+
+  // --- footnote + image write tools (promoted from inline MCP-only, #410) ---
+  //
+  // These three were previously registered inline in index.ts as MCP-only,
+  // because the in-app AI-chat agent had no equivalent. #410 promotes them so the
+  // in-app agent (esp. the Researcher role) can attach real footnotes/images
+  // instead of writing literal `^[...]` / placeholder text via editPageText. The
+  // schema + description are MOVED VERBATIM from the old inline registrations so
+  // external MCP clients see identical tool names, fields and text.
+
+  insertFootnote: {
+    mcpName: 'insert_footnote',
+    inAppKey: 'insertFootnote',
+    description:
+      'Insert an AUTHOR-INLINE footnote: you specify only WHERE (anchorText) ' +
+      'and WHAT (text). The footnote marker is placed right after anchorText in ' +
+      'the body, and the bottom footnotes list + the numbering are derived ' +
+      'deterministically server-side. You do NOT assign a number, and you ' +
+      "never see or edit the footnotes list — so footnotes cannot end up out " +
+      "of order, orphaned, or as a raw '[^id]' block. If a footnote with the " +
+      'SAME text already exists, its number is REUSED (one definition, several ' +
+      "references). The write is atomic and won't clobber concurrent edits; if " +
+      'anchorText is not found, nothing is written and an error is returned.',
+    // CORE for the in-app agent (#410): keeping it deferred would recreate the
+    // original asymmetry (footnote tool hidden while editPageText is core), which
+    // is exactly what makes the agent fall back to literal `^[...]`.
+    tier: 'core',
+    catalogLine:
+      'insertFootnote — attach a numbered footnote right after a snippet of existing body text.',
+    buildShape: (z) => ({
+      pageId: z.string().min(1),
+      anchorText: z
+        .string()
+        .min(1)
+        .describe(
+          'A snippet of existing body text; the footnote marker is inserted ' +
+            'immediately after its first occurrence (mark-safe).',
+        ),
+      text: z
+        .string()
+        .min(1)
+        .describe('The footnote content as markdown (becomes the definition).'),
+    }),
+  },
+
+  insertImage: {
+    mcpName: 'insert_image',
+    inAppKey: 'insertImage',
+    description:
+      'Download an image from a web (http/https) URL and insert it into ' +
+      'a page in one step. By default ' +
+      'appends the image at the end of the page. With replaceText, replaces the ' +
+      'first top-level block whose text contains that string (handy for ' +
+      'swapping a text placeholder like "[image: foo.png]" for the real image). ' +
+      'With afterText, inserts the image right after the first block containing ' +
+      'that string. Preserves all other block ids.',
+    tier: 'deferred',
+    catalogLine:
+      'insertImage — download a web image and insert it into a page.',
+    buildShape: (z) => ({
+      pageId: z.string().min(1),
+      imageUrl: z
+        .string()
+        .min(1)
+        .describe('http(s) URL of the image to download and upload'),
+      align: z.enum(['left', 'center', 'right']).optional(),
+      alt: z.string().optional(),
+      replaceText: z
+        .string()
+        .optional()
+        .describe(
+          'Replace the first top-level block whose text contains this string with the image',
+        ),
+      afterText: z
+        .string()
+        .optional()
+        .describe(
+          'Insert the image right after the first top-level block whose text contains this string',
+        ),
+    }),
+  },
+
+  replaceImage: {
+    mcpName: 'replace_image',
+    inAppKey: 'replaceImage',
+    description:
+      'Replace an existing image on a page with a new image fetched from a web ' +
+      '(http/https) URL: uploads the new file as a NEW ' +
+      'attachment (fresh clean URL that renders and busts browser caches), then ' +
+      'repoints every image node referencing the old attachmentId (recursively, ' +
+      'incl. callouts/tables) via the live document, preserving comments, ' +
+      'alignment and alt. The old attachment is left as an unreferenced orphan ' +
+      '(Docmost has no API to delete a single attachment; it is removed only when ' +
+      'the page/space is deleted). In-place byte overwrite is avoided because some ' +
+      'Docmost versions corrupt the attachment (HTTP 500) on overwrite.',
+    tier: 'deferred',
+    catalogLine:
+      'replaceImage — swap an existing page image for one fetched from a web URL.',
+    buildShape: (z) => ({
+      pageId: z.string().min(1),
+      attachmentId: z
+        .string()
+        .min(1)
+        .describe('attachmentId of the image currently in the page to replace'),
+      imageUrl: z
+        .string()
+        .min(1)
+        .describe('http(s) URL of the new image to download'),
+      align: z.enum(['left', 'center', 'right']).optional(),
+      alt: z.string().optional(),
+    }),
+  },
 } satisfies Record<string, SharedToolSpec>;

packages/mcp/test/mock/footnote-warnings.test.mjs

@@ -1,7 +1,7 @@
 // Mock-HTTP test for the footnoteWarnings plumbing (#166). createPage is the
 // representative path that is fully plain-HTTP (import + getPage) and so is
 // mockable here; updatePage / importPageMarkdown attach footnoteWarnings with the
-// IDENTICAL wiring (`analyzeFootnotes(...)` + spread-when-non-empty) but run their
+// IDENTICAL wiring (`footnoteWarningsField(...)` spread-when-non-empty) but run their
 // mutation over the Hocuspocus collab WebSocket, which this plain-HTTP harness
 // does not stand up. The analyzer itself is unit-tested in footnote-analyze.test.
 import { test, after } from "node:test";
@@ -76,35 +76,29 @@ function pageHandler() {
   };
 }
 
-test("createPage attaches footnoteWarnings when the content has footnote problems", async () => {
+test("createPage attaches footnoteWarnings when the content uses legacy footnote syntax", async () => {
   const baseURL = await spawn(pageHandler());
   const client = new DocmostClient(baseURL, "user@example.com", "pw");
-  // A dangling reference + a duplicate definition + a table marker.
-  const content = [
-    "Intro[^missing] and| cell[^t] |.",
-    "",
-    "[^d]: one",
-    "[^d]: two",
-    "[^t]: in table",
-  ].join("\n");
+  // Legacy reference-style `[^id]:` definitions — inert on import since #293.
+  const content = ["Intro[^a].", "", "[^a]: a definition"].join("\n");
   const result = await client.createPage("T", content, "sp-1");
   assert.ok(Array.isArray(result.footnoteWarnings), "footnoteWarnings present");
   const joined = result.footnoteWarnings.join("\n");
-  assert.match(joined, /no matching definition/); // dangling [^missing]
-  assert.match(joined, /defined more than once/); // duplicate [^d]
+  assert.match(joined, /reference-style footnotes/i);
+  assert.match(joined, /\^\[footnote text\]/); // nudge to the inline form
   // The page itself is still returned.
   assert.equal(result.success, true);
 });
 
-test("createPage omits footnoteWarnings when the content is clean", async () => {
+test("createPage omits footnoteWarnings when the content uses the inline form", async () => {
   const baseURL = await spawn(pageHandler());
   const client = new DocmostClient(baseURL, "user@example.com", "pw");
-  const content = ["A[^a] and reuse[^a].", "", "[^a]: fine"].join("\n");
+  const content = "A note.^[the body] and reuse.^[the body]";
   const result = await client.createPage("T", content, "sp-1");
   assert.equal(
     "footnoteWarnings" in result,
     false,
-    "no footnoteWarnings field on clean input",
+    "no footnoteWarnings field on inline-footnote input",
   );
   assert.equal(result.success, true);
 });

packages/mcp/test/mock/tool-timing-server.test.mjs

@@ -0,0 +1,87 @@
+// #402 — INTEGRATION test that locks the registerTool monkeypatch installed by
+// createDocmostMcpServer (src/index.ts). The sibling unit test
+// (test/unit/tool-timing.test.mjs) only exercises the timeToolHandler helper in
+// ISOLATION; it never constructs the server, so nothing there proves the factory
+// actually (a) wraps every registered tool through that helper and (b) labels
+// each sample with the tool's REGISTRATION name.
+//
+// Here we stand up a real McpServer via the factory, connect a real MCP Client
+// over the SDK's in-memory transport, invoke one registered tool, and assert the
+// host's onMetric sink received `("mcp_tool_duration_seconds", <number>, { tool
+// })` with tool === the exact registration name. This locks both the "monkeypatch
+// wraps tools" and "label = registration name" halves of the contract, so a
+// mutation to args.slice(0, -1) / the handler-arg detection / the capture order
+// is caught.
+import { test } from "node:test";
+import assert from "node:assert/strict";
+
+import { Client } from "@modelcontextprotocol/sdk/client/index.js";
+import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
+
+import { createDocmostMcpServer } from "../../build/index.js";
+
+// The tool we drive. get_workspace has NO input schema, so protocol-level input
+// validation cannot short-circuit before the handler runs — the wrapped handler
+// is guaranteed to execute (and then fail on the unreachable backend, which is
+// exactly what we want: the wrapper times in a finally on throw too).
+const TOOL_NAME = "get_workspace";
+
+test("the factory's registerTool monkeypatch times a live tool call and labels it with the registration name", async () => {
+  const calls = [];
+  const onMetric = (name, value, labels) => calls.push({ name, value, labels });
+
+  // Minimal valid credentials config. apiUrl points at a port that refuses
+  // connections immediately so the tool's backend call fails FAST (ECONNREFUSED)
+  // rather than hanging — the wrapper still emits the metric from its finally.
+  const server = createDocmostMcpServer({
+    apiUrl: "http://127.0.0.1:1",
+    email: "x@example.com",
+    password: "pw",
+    onMetric,
+  });
+
+  const [clientTransport, serverTransport] =
+    InMemoryTransport.createLinkedPair();
+
+  const client = new Client(
+    { name: "test-client", version: "0.0.0" },
+    { capabilities: {} },
+  );
+
+  await Promise.all([
+    server.connect(serverTransport),
+    client.connect(clientTransport),
+  ]);
+
+  try {
+    // Invoke the tool. The backend is unreachable, so this either resolves with
+    // an error result (isError) or rejects — both are fine. What matters is that
+    // the handler ran through the timing wrapper, which fires onMetric either way.
+    try {
+      await client.callTool({ name: TOOL_NAME, arguments: {} });
+    } catch {
+      // Tolerate the expected backend failure surfacing as a thrown protocol error.
+    }
+
+    // The wrapper must have fed exactly the timing sample for THIS tool.
+    const timing = calls.filter(
+      (c) => c.name === "mcp_tool_duration_seconds",
+    );
+    assert.ok(
+      timing.length >= 1,
+      "onMetric must receive a mcp_tool_duration_seconds sample from the wrapped handler",
+    );
+
+    const sample = timing.find((c) => c.labels && c.labels.tool === TOOL_NAME);
+    assert.ok(
+      sample,
+      `a timing sample must be labelled with the registration name "${TOOL_NAME}"; ` +
+        `got labels: ${JSON.stringify(timing.map((c) => c.labels))}`,
+    );
+    assert.equal(typeof sample.value, "number");
+    assert.ok(sample.value >= 0, "duration must be non-negative seconds");
+  } finally {
+    await client.close();
+    await server.close();
+  }
+});

packages/mcp/test/unit/client-host-contract.test.mjs

@@ -28,8 +28,9 @@ import { DocmostClient } from "../../build/index.js";
 // in the server's DocmostClientLike interface (the in-app per-user tool adapter
 // only — it is a SUBSET of the DocmostClient surface — covers only what the in-app adapter
 // consumes; the standalone MCP transport (packages/mcp/src/index.ts) calls additional
-// client methods (insertImage/replaceImage/deleteComment/updateComment/insertFootnote)
-// that this guard does NOT track — the MCP transport's own typecheck covers those). Full type-derivation
+// client methods (deleteComment/updateComment) that this guard does NOT track — the
+// MCP transport's own typecheck covers those. insertImage/replaceImage/insertFootnote
+// were MCP-only but are now in-app-consumed too (#410), so they ARE tracked below. Full type-derivation
 // of DocmostClientLike from this class is deferred (see the staged plan in
 // docmost-client.loader.ts): the package emits no declarations and the real
 // (inferred, concrete) return types conflict with the host's loose
@@ -76,6 +77,10 @@ const HOST_CONTRACT_METHODS = [
   "restorePageVersion",
   "transformPage",
   "stashPage",
+  // write (image / footnote) — MCP-only until #410 promoted them to in-app tools
+  "insertImage",
+  "replaceImage",
+  "insertFootnote",
   // write (comment)
   "createComment",
   "resolveComment",

packages/mcp/test/unit/footnote-analyze.test.mjs

@@ -1,64 +1,45 @@
 import { test } from "node:test";
 import assert from "node:assert/strict";
 
-import { analyzeFootnotes } from "../../build/lib/footnote-analyze.js";
+import {
+  footnoteWarningsField,
+  hasLegacyFootnoteDefinition,
+} from "../../build/lib/footnote-analyze.js";
 
-test("clean footnotes produce no diagnostics", () => {
-  const md = ["A[^a] and B[^b].", "", "[^a]: first", "[^b]: second"].join("\n");
-  const d = analyzeFootnotes(md);
-  assert.deepEqual(d.danglingReferences, []);
-  assert.deepEqual(d.emptyDefinitions, []);
-  assert.deepEqual(d.duplicateDefinitions, []);
-  assert.deepEqual(d.referencesInTables, []);
-  assert.deepEqual(d.warnings, []);
+// #414: the legacy footnote diagnostics were reduced to ONE advisory that fires
+// on the PRESENCE of legacy reference-style `[^id]:` definition syntax (inert on
+// import since #293), nudging the author to inline `^[...]` footnotes.
+
+test("inline `^[...]` footnotes produce no warning", () => {
+  const md = "A note here.^[the body] and reuse elsewhere.^[the body]";
+  assert.equal(hasLegacyFootnoteDefinition(md), false);
+  assert.deepEqual(footnoteWarningsField(md), {});
 });
 
-test("reuse (repeated references to one definition) is NOT a warning", () => {
-  const md = ["A[^a] B[^a] C[^a].", "", "[^a]: shared"].join("\n");
-  const d = analyzeFootnotes(md);
-  assert.deepEqual(d.danglingReferences, []);
-  assert.deepEqual(d.warnings, []);
+test("no footnotes at all produce no warning", () => {
+  const md = "Just a paragraph with [a link](https://x) and no footnotes.";
+  assert.equal(hasLegacyFootnoteDefinition(md), false);
+  assert.deepEqual(footnoteWarningsField(md), {});
 });
 
-test("dangling reference (no definition) is reported", () => {
-  const md = ["See[^missing] and[^a].", "", "[^a]: defined"].join("\n");
-  const d = analyzeFootnotes(md);
-  assert.deepEqual(d.danglingReferences, ["missing"]);
-  assert.equal(d.warnings.length, 1);
-  assert.match(d.warnings[0], /no matching definition/);
-  assert.match(d.warnings[0], /\[\^missing\]/);
+test("a legacy `[^id]:` definition triggers the single advisory", () => {
+  const md = ["See[^a].", "", "[^a]: defined"].join("\n");
+  assert.equal(hasLegacyFootnoteDefinition(md), true);
+  const field = footnoteWarningsField(md);
+  assert.equal(field.footnoteWarnings.length, 1);
+  assert.match(field.footnoteWarnings[0], /reference-style footnotes/i);
+  assert.match(field.footnoteWarnings[0], /\^\[footnote text\]/);
 });
 
-test("empty definition text is reported", () => {
-  const md = ["See[^a].", "", "[^a]:   "].join("\n");
-  const d = analyzeFootnotes(md);
-  assert.deepEqual(d.emptyDefinitions, ["a"]);
-  assert.match(d.warnings.join("\n"), /empty text/);
+test("a bare `[^id]` reference (no definition line) is not flagged", () => {
+  // Only the definition syntax `[^id]:` is a reliable signal of legacy authoring;
+  // a lone `[^x]` in prose is too ambiguous to warn on.
+  const md = "A sentence mentioning [^x] with no definition.";
+  assert.equal(hasLegacyFootnoteDefinition(md), false);
+  assert.deepEqual(footnoteWarningsField(md), {});
 });
 
-test("duplicate definition id is reported (first-wins)", () => {
-  const md = ["See[^d].", "", "[^d]: first", "[^d]: second"].join("\n");
-  const d = analyzeFootnotes(md);
-  assert.deepEqual(d.duplicateDefinitions, ["d"]);
-  assert.match(d.warnings.join("\n"), /defined more than once/);
-});
-
-test("reference inside a GFM table row is reported (heuristic)", () => {
-  const md = [
-    "| Col |",
-    "| --- |",
-    "| cell[^t] |",
-    "",
-    "[^t]: table note",
-  ].join("\n");
-  const d = analyzeFootnotes(md);
-  assert.deepEqual(d.referencesInTables, ["t"]);
-  assert.match(d.warnings.join("\n"), /table/);
-  // It is defined, so it is NOT also dangling.
-  assert.deepEqual(d.danglingReferences, []);
-});
-
-test("footnote syntax inside a code fence is ignored", () => {
+test("legacy syntax inside a code fence is ignored (fence-aware)", () => {
   const md = [
     "Intro.",
     "",
@@ -67,40 +48,22 @@ test("footnote syntax inside a code fence is ignored", () => {
     "[^demo]: not a real definition",
     "```",
     "",
-    "Outro[^a].",
-    "",
-    "[^a]: real",
+    "Outro with an inline note.^[real]",
   ].join("\n");
-  const d = analyzeFootnotes(md);
-  // `[^demo]` lives only in the fenced block, so it is neither a reference nor a
-  // dangling one, and `[^demo]:` is not counted as a definition.
-  assert.deepEqual(d.danglingReferences, []);
-  assert.deepEqual(d.duplicateDefinitions, []);
-  assert.deepEqual(d.warnings, []);
+  assert.equal(hasLegacyFootnoteDefinition(md), false);
+  assert.deepEqual(footnoteWarningsField(md), {});
 });
 
-test("a reference that only appears inside a definition's text is not dangling", () => {
-  // `[^b]` is referenced from within [^a]'s text and has its own definition.
-  const md = ["See[^a].", "", "[^a]: see also [^b]", "[^b]: the other"].join(
-    "\n",
-  );
-  const d = analyzeFootnotes(md);
-  assert.deepEqual(d.danglingReferences, []);
-});
-
-test("multiple problem classes accumulate distinct warnings", () => {
+test("a legacy definition OUTSIDE a fence still warns even with a fenced sample", () => {
   const md = [
-    "Ref[^x] and[^dup].",
+    "```",
+    "[^demo]: example inside a fence",
+    "```",
     "",
-    "[^dup]: one",
-    "[^dup]: two",
-    "[^empty]:",
+    "See[^a].",
+    "",
+    "[^a]: real definition outside the fence",
   ].join("\n");
-  const d = analyzeFootnotes(md);
-  // x has no definition; dup is defined twice; empty is empty AND has no ref.
-  assert.ok(d.danglingReferences.includes("x"));
-  assert.deepEqual(d.duplicateDefinitions, ["dup"]);
-  assert.deepEqual(d.emptyDefinitions, ["empty"]);
-  // One warning line per problem class present.
-  assert.ok(d.warnings.length >= 3);
+  assert.equal(hasLegacyFootnoteDefinition(md), true);
+  assert.equal(footnoteWarningsField(md).footnoteWarnings.length, 1);
 });

packages/mcp/test/unit/footnote-canonicalize.test.mjs

@@ -5,7 +5,7 @@ import { canonicalizeFootnotes } from "../../build/lib/footnote-canonicalize.js"
 import {
   footnoteContentKey,
   generateFootnoteId,
-} from "../../build/lib/footnote-authoring.js";
+} from "@docmost/prosemirror-markdown";
 import { insertInlineFootnote } from "../../build/lib/transforms.js";
 import { markdownToProseMirrorCanonical } from "../../build/lib/collaboration.js";
 

packages/mcp/test/unit/footnote-warnings-import.test.mjs

@@ -1,39 +1,37 @@
 import { test } from "node:test";
 import assert from "node:assert/strict";
 
-import {
-  analyzeFootnotes,
-  footnoteWarningsField,
-} from "../../build/lib/footnote-analyze.js";
+import { footnoteWarningsField } from "../../build/lib/footnote-analyze.js";
 import {
   serializeDocmostMarkdown,
   parseDocmostMarkdown,
 } from "../../build/lib/markdown-document.js";
 
-// Pins the footnoteWarnings PLUMBING contract (#169 review): the field is
-// present only on problems and omitted on clean input, AND `import_page_markdown`
-// analyzes the BODY (after the docmost:meta / docmost:comments blocks) — so a
-// footnote-like token inside those JSON blocks never warns, while a real marker
-// in the body does. importPageMarkdown does exactly
-// `footnoteWarningsField(parseDocmostMarkdown(full).body)` over a collab socket
-// this harness does not stand up, so we test the same pure composition directly.
+// Pins the footnoteWarnings PLUMBING contract (#169 review; reduced in #414): the
+// field is present only when legacy reference-style `[^id]:` syntax is used and
+// omitted otherwise, AND `import_page_markdown` analyzes the BODY (after the
+// docmost:meta / docmost:comments blocks) — so a footnote-like token inside those
+// JSON blocks never warns, while a real definition in the body does.
+// importPageMarkdown does exactly `footnoteWarningsField(parseDocmostMarkdown(full).body)`
+// over a collab socket this harness does not stand up, so we test the same pure
+// composition directly.
 
-test("footnoteWarningsField is present on problems and omitted on clean input", () => {
-  const problem = footnoteWarningsField("See[^missing].\n\n[^a]: defined");
-  assert.ok(Array.isArray(problem.footnoteWarnings));
-  assert.match(problem.footnoteWarnings.join("\n"), /no matching definition/);
+test("footnoteWarningsField is present on legacy syntax and omitted on the inline form", () => {
+  const legacy = footnoteWarningsField("See[^a].\n\n[^a]: defined");
+  assert.ok(Array.isArray(legacy.footnoteWarnings));
+  assert.match(legacy.footnoteWarnings.join("\n"), /reference-style footnotes/i);
 
-  const clean = footnoteWarningsField("A[^a] and reuse[^a].\n\n[^a]: fine");
-  assert.deepEqual(clean, {}); // no key at all on clean input
+  const inline = footnoteWarningsField("A note.^[the body] reused.^[the body]");
+  assert.deepEqual(inline, {}); // no key at all on inline-footnote input
 });
 
 test("import analyzes the BODY only — tokens inside meta/comments never warn", () => {
-  // meta + comments JSON carry `[^metaonly]` / `[^commentonly]`-looking text; the
-  // BODY has a genuinely dangling `[^bodyref]`.
+  // meta + comments JSON carry `[^metaonly]:` / `[^commentonly]:`-looking text;
+  // the BODY has a genuine legacy `[^bodyref]:` definition.
   const full = serializeDocmostMarkdown(
-    { pageId: "p1", note: "front-matter mentions [^metaonly] in text" },
-    "Body with a dangling[^bodyref] marker.",
-    [{ id: "c1", content: "a comment that says [^commentonly]" }],
+    { pageId: "p1", note: "front-matter mentions [^metaonly]: in text" },
+    "Body with a legacy[^bodyref] marker.\n\n[^bodyref]: the definition",
+    [{ id: "c1", content: "a comment that says [^commentonly]: text" }],
   );
 
   const { body } = parseDocmostMarkdown(full);
@@ -42,20 +40,19 @@ test("import analyzes the BODY only — tokens inside meta/comments never warn",
   assert.ok(!body.includes("[^commentonly]"));
 
   const field = footnoteWarningsField(body);
-  const joined = (field.footnoteWarnings ?? []).join("\n");
-  // ONLY the body's dangling reference is flagged.
-  assert.match(joined, /\[\^bodyref\]/);
-  assert.ok(!joined.includes("metaonly"));
-  assert.ok(!joined.includes("commentonly"));
+  // ONLY the body's legacy definition triggers the advisory.
+  assert.ok(Array.isArray(field.footnoteWarnings));
+  assert.match(field.footnoteWarnings.join("\n"), /reference-style footnotes/i);
 
-  // Cross-check against analyzeFootnotes directly (same composition the importer uses).
-  assert.deepEqual(analyzeFootnotes(body).danglingReferences, ["bodyref"]);
+  // The meta/comments tokens, analyzed on their own, would NOT have warned in a
+  // way that leaks here — the field is computed over the body only.
+  assert.deepEqual(footnoteWarningsField("front-matter mentions text"), {});
 });
 
-test("import on a clean body yields no footnoteWarnings field", () => {
+test("import on an inline-footnote body yields no footnoteWarnings field", () => {
   const full = serializeDocmostMarkdown(
     { pageId: "p1" },
-    "Clean body[^a] reusing[^a].\n\n[^a]: ok",
+    "Clean body.^[a note] reusing.^[a note]",
     [],
   );
   const { body } = parseDocmostMarkdown(full);

packages/mcp/test/unit/json-edit-refuse.test.mjs

@@ -140,6 +140,36 @@ test("typo fix wrapped in markdown still applies (not refused)", () => {
   assert.deepEqual(node.marks, [{ type: "bold" }]);
 });
 
+// ---------------------------------------------------------------------------
+// (iv) #410 footnote token: a `replace` containing `^[...]` is refused into
+// failed[] (it would be written as a LITERAL string, never a real footnote).
+// Nothing is applied; the reason points at insert_footnote.
+// ---------------------------------------------------------------------------
+test("replace containing a `^[...]` footnote token is refused, not applied", () => {
+  const input = doc(paragraph(textNode("The claim stands.")));
+  const snapshot = JSON.parse(JSON.stringify(input));
+
+  const { doc: out, results, failed } = applyTextEdits(input, [
+    { find: "The claim stands.", replace: "The claim stands.^[See source, p.42]" },
+  ]);
+
+  assert.equal(results.length, 0, "nothing applied");
+  assert.equal(failed.length, 1, "one refused edit");
+  assert.equal(failed[0].find, "The claim stands.");
+  assert.match(failed[0].reason, /insert_footnote/);
+  // The document is byte-for-byte untouched — no literal `^[` was written.
+  assert.deepEqual(out, snapshot);
+});
+
+test("a plain replace with no footnote token still applies (no false positive)", () => {
+  const input = doc(paragraph(textNode("a caret ^ and a bracket ] apart")));
+  const { results, failed } = applyTextEdits(input, [
+    { find: "apart", replace: "separate" },
+  ]);
+  assert.equal(failed.length, 0, "not refused");
+  assert.equal(results.length, 1, "applied");
+});
+
 // ---------------------------------------------------------------------------
 // A plain text fix is unaffected by the refuse logic.
 // ---------------------------------------------------------------------------

packages/mcp/test/unit/node-ops-table.test.mjs

@@ -5,7 +5,7 @@ import {
   insertNodeRelative,
   sanitizeForYjs,
   findUnstorableAttr,
-} from "../../build/lib/node-ops.js";
+} from "@docmost/prosemirror-markdown";
 
 // ProseMirror builders. Blocks carry a stable id in attrs.id.
 const textNode = (text) => ({ type: "text", text });

packages/mcp/test/unit/node-ops.test.mjs

@@ -7,7 +7,7 @@ import {
   deleteNodeById,
   assertUnambiguousMatch,
   insertNodeRelative,
-} from "../../build/lib/node-ops.js";
+} from "@docmost/prosemirror-markdown";
 
 // ProseMirror builders. Blocks carry a stable id in attrs.id.
 const textNode = (text) => ({ type: "text", text });

packages/mcp/test/unit/outline.test.mjs

@@ -1,7 +1,7 @@
 import { test } from "node:test";
 import assert from "node:assert/strict";
 
-import { buildOutline, getNodeByRef } from "../../build/lib/node-ops.js";
+import { buildOutline, getNodeByRef } from "@docmost/prosemirror-markdown";
 
 // Helpers to build the small fixture doc.
 const textNode = (text) => ({ type: "text", text });

packages/mcp/test/unit/page-search.test.mjs

@@ -2,7 +2,7 @@ import { test } from "node:test";
 import assert from "node:assert/strict";
 
 import { searchInDoc } from "../../build/lib/page-search.js";
-import { getNodeByRef } from "../../build/lib/node-ops.js";
+import { getNodeByRef } from "@docmost/prosemirror-markdown";
 
 // ---------------------------------------------------------------------------
 // Document builders. Mirror the Docmost ProseMirror shape: paragraphs/headings

packages/mcp/test/unit/parse-node-arg.test.mjs

@@ -1,7 +1,7 @@
 import { test } from "node:test";
 import assert from "node:assert/strict";
 
-import { parseNodeArg } from "../../build/lib/parse-node-arg.js";
+import { parseNodeArg } from "@docmost/prosemirror-markdown";
 
 test("parseNodeArg passes an object through unchanged", () => {
   const obj = { type: "paragraph", content: [] };

packages/mcp/test/unit/table-ops.test.mjs

@@ -6,7 +6,7 @@ import {
   insertTableRow,
   deleteTableRow,
   updateTableCell,
-} from "../../build/lib/node-ops.js";
+} from "@docmost/prosemirror-markdown";
 
 // ---------------------------------------------------------------------------
 // Builders. Tables/rows/cells carry NO attrs.id — only the paragraph inside a

packages/mcp/test/unit/tool-timing.test.mjs

@@ -0,0 +1,67 @@
+import { test } from "node:test";
+import assert from "node:assert/strict";
+
+import { timeToolHandler } from "../../build/index.js";
+
+// #402 — the registerTool wrapper times every tool through a single choke point
+// and feeds the host's dependency-neutral onMetric sink. These assert the
+// timing contract without a live transport (the factory monkeypatches
+// server.registerTool with exactly this helper).
+
+test("times a tool and preserves the handler's return value", async () => {
+  const calls = [];
+  const onMetric = (name, value, labels) => calls.push({ name, value, labels });
+
+  const handler = async (arg) => ({ ok: true, echo: arg });
+  const wrapped = timeToolHandler("get_page", handler, onMetric);
+
+  const result = await wrapped("hello");
+  // Return value passes through untouched.
+  assert.deepEqual(result, { ok: true, echo: "hello" });
+
+  // Exactly one sample, correct name/labels, numeric non-negative duration.
+  assert.equal(calls.length, 1);
+  assert.equal(calls[0].name, "mcp_tool_duration_seconds");
+  assert.deepEqual(calls[0].labels, { tool: "get_page" });
+  assert.equal(typeof calls[0].value, "number");
+  assert.ok(calls[0].value >= 0, "duration must be non-negative seconds");
+});
+
+test("standalone (no onMetric) is a transparent pass-through", async () => {
+  const handler = async (a, b) => a + b;
+  const wrapped = timeToolHandler("noop_tool", handler, undefined);
+
+  // No throw, result unchanged, and nothing to observe.
+  const result = await wrapped(2, 3);
+  assert.equal(result, 5);
+});
+
+test("still observes on throw, then rethrows the original error", async () => {
+  const calls = [];
+  const onMetric = (name, value, labels) => calls.push({ name, value, labels });
+
+  const boom = new Error("handler failed");
+  const handler = async () => {
+    throw boom;
+  };
+  const wrapped = timeToolHandler("boom_tool", handler, onMetric);
+
+  await assert.rejects(() => wrapped(), (err) => err === boom);
+
+  // Observed exactly once despite the throw.
+  assert.equal(calls.length, 1);
+  assert.equal(calls[0].name, "mcp_tool_duration_seconds");
+  assert.deepEqual(calls[0].labels, { tool: "boom_tool" });
+});
+
+test("standalone still rethrows the original error (no swallow, no observe)", async () => {
+  const boom = new Error("standalone failure");
+  const wrapped = timeToolHandler(
+    "boom_standalone",
+    async () => {
+      throw boom;
+    },
+    undefined,
+  );
+  await assert.rejects(() => wrapped(), (err) => err === boom);
+});

packages/prosemirror-markdown/src/lib/footnote.ts

@@ -59,3 +59,89 @@ export function splitFootnoteParagraphs(encoded: string): string[] {
   paragraphs.push(current);
   return paragraphs;
 }
+
+// ---------------------------------------------------------------------------
+// Inline-authoring helpers (#414: moved here from the mcp `footnote-authoring.ts`
+// fork so the dedup convention — content-key + definition factory + id gen —
+// has ONE home next to the importer that shares the convention). Used by the
+// mcp author-inline tool (`insertInlineFootnote` in transforms.ts).
+// ---------------------------------------------------------------------------
+
+const FOOTNOTE_DEFINITION_NAME = "footnoteDefinition";
+
+function cloneJson<T>(v: T): T {
+  if (typeof structuredClone === "function") return structuredClone(v);
+  return JSON.parse(JSON.stringify(v)) as T;
+}
+
+/**
+ * Normalized content key for de-duplicating footnote DEFINITIONS by their text.
+ *
+ * Two definitions with the same key are the SAME footnote — so the inline
+ * authoring tool reuses one id (one number, one definition, several references)
+ * instead of minting a second definition. Key = plaintext (whitespace-collapsed,
+ * trimmed) PLUS a signature of the inline mark types in order, so two notes that
+ * read the same but differ in formatting (one bold, one plain) are NOT merged.
+ * Conservative: only an exact match merges.
+ */
+export function footnoteContentKey(defNode: any): string {
+  const parts: string[] = [];
+  const visit = (n: any): void => {
+    if (!n || typeof n !== "object") return;
+    if (n.type === "text" && typeof n.text === "string") {
+      const marks = Array.isArray(n.marks)
+        ? n.marks.map((m: any) => m?.type).filter(Boolean).sort().join(",")
+        : "";
+      parts.push(`${n.text}${marks}`);
+    }
+    if (Array.isArray(n.content)) for (const c of n.content) visit(c);
+  };
+  visit(defNode);
+  // Collapse the assembled text's whitespace and trim, keeping the mark
+  // signature attached so formatting differences still distinguish notes.
+  return parts
+    .join("")
+    .replace(/[ \t\r\n]+/g, " ")
+    .trim();
+}
+
+/**
+ * Build a footnoteDefinition node from inline ProseMirror nodes, keyed by id.
+ */
+export function makeFootnoteDefinition(id: string, inlineNodes: any[]): any {
+  const content = Array.isArray(inlineNodes) ? cloneJson(inlineNodes) : [];
+  return {
+    type: FOOTNOTE_DEFINITION_NAME,
+    attrs: { id },
+    content: [{ type: "paragraph", content }],
+  };
+}
+
+/**
+ * Generate a uuidv7-style id (time-ordered), matching editor-ext's
+ * `generateFootnoteId`. Used for a genuinely-new inline footnote id.
+ */
+export function generateFootnoteId(): string {
+  const now = Date.now();
+  const timeHex = now.toString(16).padStart(12, "0");
+  const rand = (length: number) => {
+    let s = "";
+    for (let i = 0; i < length; i++)
+      s += Math.floor(Math.random() * 16).toString(16);
+    return s;
+  };
+  const versioned = "7" + rand(3);
+  const variantNibble = (8 + Math.floor(Math.random() * 4)).toString(16);
+  const variant = variantNibble + rand(3);
+  return (
+    timeHex.slice(0, 8) +
+    "-" +
+    timeHex.slice(8, 12) +
+    "-" +
+    versioned +
+    "-" +
+    variant +
+    "-" +
+    rand(12)
+  );
+}

packages/prosemirror-markdown/src/lib/index.ts

@@ -44,3 +44,35 @@ export {
   docsCanonicallyEqual,
 } from "./canonicalize.js";
 export { parsePageFile, serializePageFile } from "./page-file.js";
+
+// Pure, network-free helpers for manipulating a ProseMirror/TipTap document
+// tree by node id (#414: the single canonical copy, formerly forked into mcp).
+// Consumed by `@docmost/mcp` (patch/insert/delete node, table tools, outline).
+export {
+  blockPlainText,
+  buildOutline,
+  getNodeByRef,
+  replaceNodeById,
+  deleteNodeById,
+  sanitizeForYjs,
+  findUnstorableAttr,
+  insertNodeRelative,
+  readTable,
+  insertTableRow,
+  deleteTableRow,
+  updateTableCell,
+  assertUnambiguousMatch,
+} from "./node-ops.js";
+export type { OutlineEntry } from "./node-ops.js";
+
+// Normalize a ProseMirror node arg that the model may have serialized as a JSON
+// string (#414: single copy shared by mcp and the CommonJS server app).
+export { parseNodeArg } from "./parse-node-arg.js";
+
+// Inline-footnote authoring convention (#414: single copy, formerly the mcp
+// `footnote-authoring.ts` fork), shared with the importer's `assembleFootnotes`.
+export {
+  footnoteContentKey,
+  makeFootnoteDefinition,
+  generateFootnoteId,
+} from "./footnote.js";

packages/prosemirror-markdown/src/lib/node-ops.ts

@@ -14,6 +14,8 @@
  * `content`, non-object nodes, and absent `attrs` are tolerated.
  */
 
+import { stripInlineMarkdown } from "./text-normalize.js";
+
 /** Deep-clone a JSON-serializable value without mutating the original. */
 function clone<T>(value: T): T {
   if (typeof structuredClone === "function") {
@@ -97,12 +99,15 @@ export function buildOutline(doc: any): OutlineEntry[] {
     const entry: OutlineEntry = {
       index: i,
       type,
-      id: isObject(block) && isObject(block.attrs) ? block.attrs.id ?? null : null,
+      id:
+        isObject(block) && isObject(block.attrs)
+          ? (block.attrs.id ?? null)
+          : null,
       firstText: truncate(blockPlainText(block), 100),
     };
 
     if (type === "heading") {
-      entry.level = isObject(block.attrs) ? block.attrs.level ?? null : null;
+      entry.level = isObject(block.attrs) ? (block.attrs.level ?? null) : null;
     } else if (type === "table") {
       const headerRow = block.content?.[0]?.content ?? [];
       entry.rows = block.content?.length ?? 0;
@@ -247,6 +252,33 @@ export function deleteNodeById(
   return { doc: out, deleted };
 }
 
+/**
+ * Throw a clear, model-actionable error when a node-id write op did NOT match
+ * exactly one node (#159). `count === 0` -> "no node found"; `count > 1` ->
+ * "ambiguous, refused" — Docmost duplicates block ids on copy/paste, so a write
+ * by id could clobber/remove EVERY duplicate. The caller skips the write for any
+ * `count !== 1` (the transform returns null), so this only REPORTS; nothing was
+ * changed. No-op for the unambiguous single-match case.
+ */
+export function assertUnambiguousMatch(
+  op: "patch_node" | "delete_node",
+  verb: "replace" | "delete",
+  count: number,
+  nodeId: string,
+  pageId: string,
+): void {
+  if (count === 0) {
+    throw new Error(
+      `${op}: no node with id "${nodeId}" found on page ${pageId}`,
+    );
+  }
+  if (count > 1) {
+    throw new Error(
+      `${op}: id "${nodeId}" is ambiguous — ${count} nodes on page ${pageId} share it (block ids are duplicated on copy/paste). Refusing to ${verb} all of them; nothing was changed. Re-target with a more specific anchor.`,
+    );
+  }
+}
+
 /**
  * Deep-clone `doc` and strip every node/mark attribute whose value is strictly
  * `undefined`, so the result is safe to hand to Yjs (which throws an opaque
@@ -364,6 +396,31 @@ const REQUIRED_CONTAINER: Record<string, string> = {
   tableHeader: "tableRow",
 };
 
+/**
+ * Find the index of the first TOP-LEVEL block whose plain text includes the
+ * anchor, with a markdown-stripping FALLBACK. Returns -1 when none matches.
+ *
+ * Two passes preserve "exact wins globally":
+ *  - Pass 1: first block containing the verbatim `anchorText`.
+ *  - Pass 2 (only if pass 1 found nothing): first block containing the
+ *    markdown-stripped anchor, when stripping actually changed it.
+ */
+function findAnchorTextIndex(content: any[], anchorText: string): number {
+  if (!Array.isArray(content)) return -1;
+  // Pass 1: exact.
+  for (let i = 0; i < content.length; i++) {
+    if (blockPlainText(content[i]).includes(anchorText)) return i;
+  }
+  // Pass 2: markdown-stripped fallback.
+  const a = stripInlineMarkdown(anchorText);
+  if (a !== anchorText && a.length > 0) {
+    for (let i = 0; i < content.length; i++) {
+      if (blockPlainText(content[i]).includes(a)) return i;
+    }
+  }
+  return -1;
+}
+
 /**
  * Locate an anchor and return its ancestor chain (from `doc` down to and
  * including the matched node). Each chain entry is `{ node, index }` where
@@ -399,14 +456,14 @@ function findAnchorChain(
   }
 
   // By text: only top-level blocks are scanned (same rule as the JSON path).
+  // Exact match wins; a markdown-stripped fallback is tried only on a miss.
   if (opts.anchorText != null && Array.isArray(doc.content)) {
-    for (let i = 0; i < doc.content.length; i++) {
-      if (blockPlainText(doc.content[i]).includes(opts.anchorText)) {
-        return [
-          { node: doc, index: -1 },
-          { node: doc.content[i], index: i },
-        ];
-      }
+    const i = findAnchorTextIndex(doc.content, opts.anchorText);
+    if (i !== -1) {
+      return [
+        { node: doc, index: -1 },
+        { node: doc.content[i], index: i },
+      ];
     }
   }
 
@@ -540,13 +597,13 @@ export function insertNodeRelative(
     return { doc: out, inserted };
   }
 
-  // Resolve by text: only top-level doc.content blocks are scanned.
+  // Resolve by text: only top-level doc.content blocks are scanned. Exact
+  // match wins; a markdown-stripped fallback is tried only on a miss.
   if (opts.anchorText != null && isObject(out) && Array.isArray(out.content)) {
-    for (let i = 0; i < out.content.length; i++) {
-      if (blockPlainText(out.content[i]).includes(opts.anchorText)) {
-        out.content.splice(i + offset, 0, fresh);
-        return { doc: out, inserted: true };
-      }
+    const i = findAnchorTextIndex(out.content, opts.anchorText);
+    if (i !== -1) {
+      out.content.splice(i + offset, 0, fresh);
+      return { doc: out, inserted: true };
     }
   }
 
@@ -617,7 +674,8 @@ function locateTable(
   if (!isObject(rootClone)) return null;
 
   // "#<n>": index into the top-level content array; must be a table.
-  const indexMatch = typeof tableRef === "string" ? tableRef.match(/^#(\d+)$/) : null;
+  const indexMatch =
+    typeof tableRef === "string" ? tableRef.match(/^#(\d+)$/) : null;
   if (indexMatch) {
     const index = Number(indexMatch[1]);
     const block = Array.isArray(rootClone.content)
@@ -717,7 +775,7 @@ export function readTable(
         : undefined;
       const id =
         isObject(firstPara) && isObject(firstPara.attrs)
-          ? firstPara.attrs.id ?? null
+          ? (firstPara.attrs.id ?? null)
           : null;
       rowIds.push(id);
     }
@@ -751,14 +809,17 @@ export function insertTableRow(
   if (!Array.isArray(table.content)) table.content = [];
   const rows = table.content.length;
   const headerRow = table.content[0];
-  const headerCells = Array.isArray(headerRow?.content) ? headerRow.content : [];
+  const headerCells = Array.isArray(headerRow?.content)
+    ? headerRow.content
+    : [];
 
   // Column count is the WIDEST existing row, so the guard below stays
   // meaningful for ragged tables and the new row matches the table's width.
   // Fall back to the supplied cell count only when the table has no rows.
   let colCount = 0;
   for (const r of table.content) {
-    if (isObject(r) && Array.isArray(r.content)) colCount = Math.max(colCount, r.content.length);
+    if (isObject(r) && Array.isArray(r.content))
+      colCount = Math.max(colCount, r.content.length);
   }
   if (colCount === 0) colCount = Array.isArray(cells) ? cells.length : 0;
 
@@ -771,7 +832,10 @@ export function insertTableRow(
   // Resolve the landing index up front so the cell-type decision and the splice
   // below agree: a valid integer in [0, rows] splices there, else we append.
   const landingIndex =
-    typeof index === "number" && Number.isInteger(index) && index >= 0 && index <= rows
+    typeof index === "number" &&
+    Number.isInteger(index) &&
+    index >= 0 &&
+    index <= rows
       ? index
       : rows;
 
@@ -790,7 +854,8 @@ export function insertTableRow(
     // A row landing at index 0 becomes the new header row, so inherit the
     // current header cell's type per column (Docmost uses "tableHeader" there);
     // every other position is a plain data cell.
-    const cellType = landingIndex === 0 ? headerCells[i]?.type ?? "tableCell" : "tableCell";
+    const cellType =
+      landingIndex === 0 ? (headerCells[i]?.type ?? "tableCell") : "tableCell";
     newCells.push({
       type: cellType,
       attrs,
@@ -862,9 +927,10 @@ export function updateTableCell(
   const rowNodes = Array.isArray(table.content) ? table.content : [];
   const rows = rowNodes.length;
   const rowNode = rowNodes[row];
-  const cols = isObject(rowNode) && Array.isArray(rowNode.content)
-    ? rowNode.content.length
-    : 0;
+  const cols =
+    isObject(rowNode) && Array.isArray(rowNode.content)
+      ? rowNode.content.length
+      : 0;
 
   if (
     !Number.isInteger(row) ||

packages/prosemirror-markdown/src/lib/parse-node-arg.ts

@@ -2,6 +2,11 @@
 // instead of an object. Normalize: parse a string to an object (throwing on
 // invalid JSON), pass an object through unchanged. Shared by patch_node /
 // insert_node (and the analogous update_page_json content parsing).
+//
+// This lives in the converter package (#414) so BOTH consumers import the ONE
+// copy: `@docmost/mcp` (ESM) and the CommonJS server app. The server cannot
+// import `@docmost/mcp` directly (ESM-only, no declaration files), but it does
+// import `@docmost/prosemirror-markdown` natively — so this is the shared home.
 export function parseNodeArg(
   node: unknown,
   errMsg = "node was a string but not valid JSON",

packages/prosemirror-markdown/src/lib/text-normalize.ts

@@ -0,0 +1,99 @@
+/**
+ * Locator normalization: strip inline markdown wrappers and trailing
+ * decoration from a LOCATOR string so a find/anchor that the model wrote with
+ * markdown (or a stray emoji) can still match the document's plain text.
+ *
+ * This is used ONLY as a fallback for LOCATING (after an exact match fails);
+ * it is never applied to replacement text or inserted node content, so no
+ * formatting is ever lost.
+ *
+ * Scope note (#414): this package-local copy exists so `node-ops.ts` — which
+ * lives here now (the single canonical copy) — can resolve its markdown-tolerant
+ * anchor fallback without a circular dependency back on `@docmost/mcp`. It
+ * intentionally carries ONLY `stripInlineMarkdown` (the primitive `node-ops`
+ * needs); the mcp-side `text-normalize.ts` (which additionally serves
+ * `json-edit.ts` via `stripBalancedWrappers`) is the subject of a separate
+ * dedup task and is left untouched here.
+ */
+
+/** Maximum unwrap passes, so pathological/nested input cannot loop forever. */
+const MAX_PASSES = 8;
+
+/**
+ * Inline emphasis/code/strikethrough wrappers, strong BEFORE emphasis so
+ * `**x**` collapses to `x` rather than leaving a stray `*x*`. Each pattern is
+ * non-greedy and capture group 1 is the inner text. Applied repeatedly until
+ * the string stops changing (nested wrappers like `**_x_**`).
+ */
+const WRAPPER_PATTERNS: RegExp[] = [
+  /\*\*([^*]+?)\*\*/g, // **x**
+  /__([^_]+?)__/g, // __x__
+  /~~([^~]+?)~~/g, // ~~x~~
+  /\*([^*]+?)\*/g, // *x*
+  /_([^_]+?)_/g, // _x_
+  /``([^`]+?)``/g, // ``x``
+  /`([^`]+?)`/g, // `x`
+];
+
+/** Links/images -> their visible text. `!?` covers both `[t](u)` and `![a](s)`. */
+const LINK_IMAGE_RE = /!?\[([^\]]*)\]\([^)]*\)/g;
+
+/**
+ * Apply the two balanced/link passes: first collapse links/images to their
+ * visible text, then collapse balanced inline wrappers repeatedly until stable.
+ * Does NOT trim decoration, does NOT guard against an empty result — it returns
+ * exactly the transformed string.
+ */
+function stripWrappersAndLinks(s: string): string {
+  // 1. Links/images -> their visible text.
+  let out = s.replace(LINK_IMAGE_RE, "$1");
+
+  // 2. Strip balanced wrappers, repeating until the string is stable so nested
+  //    wrappers (`**_x_**`) and adjacent runs both collapse.
+  for (let pass = 0; pass < MAX_PASSES; pass++) {
+    const before = out;
+    for (const re of WRAPPER_PATTERNS) {
+      out = out.replace(re, "$1");
+    }
+    if (out === before) break;
+  }
+  return out;
+}
+
+/**
+ * Conservatively strip inline markdown from a locator string.
+ *
+ * Deterministic, order-fixed steps:
+ *  1. Links/images: `[text](url)` -> `text`, `![alt](src)` -> `alt`.
+ *  2. Balanced inline wrappers (strong before emphasis, code, strikethrough),
+ *     applied repeatedly until stable for nested cases.
+ *  3. Trim leading/trailing decoration only: whitespace, leftover marker chars
+ *     (`* _ ~ \``) and emoji. Letters/digits and sentence punctuation (`.`/`,`
+ *     etc.) are NEVER trimmed.
+ *
+ * If the result is empty (e.g. the input was only markers like `***`), the
+ * ORIGINAL string is returned so a locator can never normalize down to "" and
+ * match everything.
+ */
+export function stripInlineMarkdown(s: string): string {
+  if (typeof s !== "string" || s.length === 0) return s;
+
+  // 1 + 2. Shared link/image and balanced-wrapper passes.
+  let out = stripWrappersAndLinks(s);
+
+  // 3. Trim leading/trailing decoration: whitespace, leftover markdown markers,
+  //    and emoji (Extended_Pictographic plus the VS16 / ZWJ joiners, plus the
+  //    regional-indicator range U+1F1E6–U+1F1FF for flag emoji, which are NOT
+  //    Extended_Pictographic). The `u` flag enables the Unicode property escape.
+  //    Anchored runs only — interior text and sentence punctuation are untouched.
+  const DECORATION =
+    "[\\s*_~\\x60\\p{Extended_Pictographic}\\u{1F1E6}-\\u{1F1FF}\\u{FE0F}\\u{200D}]+";
+  out = out
+    .replace(new RegExp("^" + DECORATION, "u"), "")
+    .replace(new RegExp(DECORATION + "$", "u"), "");
+
+  // 4. Never normalize a locator down to nothing.
+  if (out.length === 0) return s;
+
+  return out;
+}