Compare commits

...

16 Commits

Author SHA1 Message Date
agent_coder fdc37de3e8 fix(ai-chat): ревью-раунд #500 — красный серверный сьют + регрессия Redis-health
B1: переименование строки ошибки (#394) не обновило 4 предсуществующих
share-ассерта → полный серверный сьют был красный. Обновлены ассерты в
public-share-chat.spec.ts и public-share-chat-tools.service.spec.ts под
новую классифицированную строку.

B2: /health первая проба после старта врала DOWN при живом Redis
(lazyConnect + enableOfflineQueue:false + maxRetriesPerRequest:1 → первый
ping до открытия сокета). Добавлен ensureConnected() с bounded-таймаутом
перед первым ping; покрывает и путь пересоздания после onModuleDestroy.
Тест UP-с-первой-пробы против реального ioredis (mutation-verified).

Устранён open-handle leak в redis.health.spec (drain ioredis
force-destroy-таймера в teardown; без forceExit) и окно ложного DOWN при
конкурентных пробах (мемоизированный connectingPromise).

B3: комментарий про orphan-чат при провале beginRun (insert до begin).
B4: описание listPages упоминает поле truncated в tree-режиме.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 07:19:14 +03:00
agent_coder 4a750c1e7f fix(#486): ревью — CHANGELOG + AGENTS.md + два теста
Ре-ревью PR #500 (changes-requested, 4 мелких, все doc/test):

F1 [CHANGELOG] CHANGELOG.md [Unreleased]:
- Breaking Changes: metrics-листенер 0.0.0.0→127.0.0.1 — кросс-контейнерный
  скрейп (docmost:9464) молча умрёт без METRICS_BIND=0.0.0.0 + METRICS_TOKEN
  (миграция в .env.example).
- Security: утечка errorText тулов/провайдера анониму (closes #394);
  /metrics под Bearer (METRICS_TOKEN).
- Fixed: ioredis-утечка в /health; ELK вешал event loop; beginRun-призрак →
  честный 503 A_RUN_BEGIN_FAILED; ai drain-hang.

F2 [AGENTS.md] строка про ai-патч: теперь он несёт ДВА фикса (#184 O(n²)
partialOutput И #486 drain-hang), оба тривайра названы.

F3 [test] metrics.server.spec: добавлен кейс токена ТОЙ ЖЕ длины
(Bearer topsecreX) → 401 — пиннит constant-time сравнение (прежние кейсы
коротили на length-guard, до timingSafeEqual не доходили).

F4 [test] output-degeneration.spec: behavior-тест, гоняющий РЕАЛЬНЫЕ
onChunk/onStepFinish из stream() — длинный чистый шаг → граница → свежий
дегенеративный бёрст → ассерт abortSignal.aborted (было хардкодом
resetWatermark=0, ревёрт правки не краснил).

Мутационные пруфы (non-vacuous): F3 — форс compare→true роняет same-length
кейс (401→200); F4 — ревёрт lastDegenerationCheckLen=0 роняет behavior-тест
(aborted true→false). Оба восстановлены, специи зелёные (34/34).
2026-07-11 07:19:14 +03:00
agent_coder ea7c4d7cd2 fix(#486): ревью — run-race контракт под новую политику + timing-safe metrics-токен
Ревью полной ветки #486 нашло два пункта в моих коммитах (3 и 4):

- BLOCKER (коммит 4): ai-chat.service.run-race.spec.ts (#184 F14) пинил СТАРУЮ
  политику «plain begin() failure → swallow + стрим UNTRACKED» (resolves
  toBeUndefined). Коммит 4 её развернул → тест падал (1 failed/6). Кейс
  ИНВЕРТИРОВАН под новую политику: plain begin-failure теперь REJECTS с 503
  A_RUN_BEGIN_FAILED, до первого байта, user-строка не вставлена, streamText не
  вызван — путь остаётся явно запинён, а не удалён.

- NIT (коммит 3): metrics-токен сравнивался наивным !== (единственный слой auth
  эндпоинта) → тайминг-утечка токена. Заменено на crypto.timingSafeEqual с
  length-guard (разная длина → reject), семантика 401/200 без изменений.

Внесено отдельным fixup-коммитом (rebase -i недоступен в окружении; ветка не
запушена). Тесты: run-race 7/7 + metrics.server 7/7 зелёные.
2026-07-11 07:19:14 +03:00
agent_coder 255024fbe2 chore(mcp): McpService.onModuleDestroy → destroyAllSessions + удаление мёртвого кода (#486)
onModuleDestroy чистил только sweep-таймер; loopback-collab-сессии держали доки
открытыми на collab-сервере до идла — рестарт мог гонять с доком, запиненным
умирающим воркером. Теперь дергает destroyAllSessions() через переопределяемый
шов (для юнит-теста без ESM-пакета), best-effort. Плюс удалён мёртвый код:
неиспользуемый импорт parseNodeArg в mcp/index.ts и мёртвые enum-члены
SEARCH_REMOVE_* в queue.constants (подтверждено grep'ом — ни диспетчера, ни
процессора).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 07:19:14 +03:00
agent_coder b934fb2292 fix(mcp): пробросить truncated в tree-mode listPages (#486)
listPages(tree:true) деструктурировал только pages из enumerateSpacePages и
возвращал голое дерево, теряя truncated — на неполном дереве (stdio-fallback BFS
упёрся в node-cap) вызывающий не знал, что страницы потеряны. Возвращаем
{ tree, truncated } (по образцу check_new_comments); основной /pages/tree путь
беспредельный, там false.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 07:19:14 +03:00
agent_coder 14da295185 fix(auth): провенанс для API-key пути (#486)
validateApiKey возвращал результат до резолва провенанса — REST-записи по
is_agent API-ключу не получали маркер 'agent'. Перенести «выше» нельзя: payload
API-ключа не несёт подписанный actor-клейм, а user (с isAgent) неизвестен до
валидации ключа. Резолвим провенанс от возвращённого user: isAgent -> 'agent',
иначе 'user'; aiChatId у API-ключа всегда null (нет ai_chats-строки). Загрузка
EE ApiKeyService вынесена в переопределяемый шов resolveApiKeyService для юнит-
теста без EE-бандла.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 07:19:14 +03:00
agent_coder de8f9c804c fix(client): flushNext в onFinish гейтится mountedRef (#486)
Финальный onFinish->flushNext() не проверял live-mount флаг. Чистый onFinish
может прийти ПОСЛЕ анмаунта треда (New-chat / переключение чата мид-стрим —
асинхронные attach/resume оседают поздно): flush дергал очередь и re-POST'ил
сообщение из брошенного треда — «призрачные» отправки/чаты-призраки. Остальные
обращения к очереди уже гейтятся mountedRef; закрываем последнюю дыру.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 07:19:14 +03:00
agent_coder 8ebdfe156f fix(ai-chat): сброс lastDegenerationCheckLen в onStepFinish (#486)
onStepFinish обнулял inProgressText, но НЕ lastDegenerationCheckLen — а это
смещение В аккумулятор. После первого длинного шага протухшая (большая) отметка
делала throttle-условие отрицательным, и детектор токен-лупов молчал весь
следующий шаг, пока текст не перерастал старую отметку. Обнуляем отметку в
onStepFinish. Throttle-предикат вынесен в output-degeneration
(shouldCheckDegeneration + DEGENERATION_CHECK_STEP), чтобы юнит гонял ту же
логику, что и стрим.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 07:19:14 +03:00
agent_coder 31176d5f93 fix(mcp): сброс collab-токена на WS-auth-failure с ретраем (#486)
Кэш collab-токена (#435) инвалидировался только на HTTP-401/403 (REST-
интерцептор и login()); отклонённый Hocuspocus-handshake оставлял протухший
токен в кэше — каждая последующая мутация переотправляла тот же битый токен до
истечения TTL (минуты) без self-heal. collab-session помечает ошибку
onAuthenticationFailed маркером; клиентские write-швы (mutatePage/replacePage/
mutateLiveContentUnlocked) обёрнуты в writeWithCollabAuthRetry: на помеченной
ошибке кэш сбрасывается и запись ретраится ровно один раз со свежим токеном —
симметрично HTTP-пути.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 07:19:14 +03:00
agent_coder 5f0c49d47c fix(mcp): REGISTRY_STAMP хэширует всё src/**, а не только tool-specs.ts (#486)
Стамп детектил build/src-skew только по tool-specs.ts — правка client.ts,
client/*-модуля, comment-signal или drawio-* без пересборки проходила молча,
и build/ отдавал старый код. Теперь codegen и рантайм-лоадер хэшируют весь
src/**/*.ts (кроме *.generated.ts — иначе цикл через собственный выход),
симметрично: одинаковый обход, POSIX-сортировка, нормализация и sha256.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 07:19:14 +03:00
agent_coder b2e5b51420 fix(ai): drain-hang в writeToServerResponse — расширить patches/ai@6.0.134.patch (#486)
Серверная ai@6.0.134 в writeToServerResponse при backpressure (write()===false)
ждала ТОЛЬКО once('drain'). Если клиент отвалился мид-запись, сокет не дренится,
await не резолвится: read-цикл паркуется НАВСЕГДА, finally{response.end()}
недостижим, reader и буферы висят до рестарта. В autonomous ран продолжает лить
вывод после дисконнекта → КАЖДЫЙ дисконнект мид-ран оставляет висящий пайп.
Плюс read() — fire-and-forget с throw → unhandledRejection.

Патч расширен (index.js и index.mjs): Promise.race drain/close/error с гигиеной
once-слушателей (все три снимаются на первом settle — не копятся по одному на
stall); при close/error — reader.cancel() и выход (безопасно для detached-ранов:
независимый дренаж делает consumeStream); rejection read() поглощается с логом.
pnpm-lock patch_hash перегенерён (patch-commit). Выравнивание версии ai —
отдельный коммит (#495), не здесь.

Тест: трипвайр-спека в apps/server (только там резолвится патченная копия) по
образцу ai-sdk-partial-output.patch.spec — дисконнект мид-запись без drain
завершает ответ (end() вызван, reader не висит), read()-throw не даёт
unhandledRejection, оба dist-билда несут маркер PATCH(docmost #486).
2026-07-11 07:19:14 +03:00
agent_coder 26d9a70a1c fix(share): не утекать errorText тулов и провайдера анониму в публичном шэре (closes #394)
SECURITY. В публичном share-чате сырой текст ошибки тула или провайдера утекал
анонимному читателю. Три слоя, все обязательны:

(1) Рендер-гейт: prop showErrors в ToolCallCard (протянут через MessageList/
MessageItem), share-виджет передаёт false — сырой errorText не рисуется. Но
рендер-гейт маскирует только DOM, не байты.

(2) Санитизация на уровне share-тулсета (авторитетно): forShare оборачивает
execute каждого тула catch'ем. Своя ShareToolError (безопасные строки: «page not
available in this share») пробрасывается, ЛЮБАЯ другая ошибка → generic «tool
could not complete», полный текст только в серверный лог. Одно место закрывает
байты (атомарный tool-output-error фрейм v6), рендер и контекст модели; self-
correction сохранена.

(3) Анонимный onError пайпа: ShareToolError → её безопасное сообщение; иначе
describeProviderError (statusCode + тело: внутренний baseUrl/модель) только в лог,
читателю — фиксированная классифицированная строка (rate-limited/unavailable/
provider error).

Тест: интеграционный с РЕАЛЬНЫМ падением тула и провайдера — assert по СЫРЫМ SSE-
БАЙТАМ (не по DOM): секрет/baseUrl/стек отсутствуют, видна безопасная строка,
полный текст провайдера ушёл в серверный лог.
2026-07-11 07:19:14 +03:00
agent_coder e1ebd79f30 fix(ai-chat): провал beginRun фейлит ход честным 503 (A_RUN_BEGIN_FAILED) (#486)
Раньше провал beginRun (кроме unique-violation), напр. blip пула БД, логировался
и ход ПРОДОЛЖАЛСЯ без run-строки. В autonomous такой ран никто не абортит: /stop
его не видит, дисконнект не абортит, one-run-гейт пропускает ВТОРОЙ ран — невидимый
неостанавливаемый ран до рестарта.

Теперь провал beginRun (кроме RunAlreadyActiveError → прежний 409) бросает
ServiceUnavailableException с кодом A_RUN_BEGIN_FAILED ДО первого байта и до
вставки user-строки (post-hijack catch контроллера отдаёт честный 503 на raw-
сокет). Без ветвления по режимам — #487 наследует ту же политику. В тело кладём
statusCode: 503 (object-arg исключение его не добавляет), чтобы клиент видел
статус.

Клиентский классификатор: ветка A_RUN_BEGIN_FAILED добавлена СТРОГО ДО generic-
503-матча — иначе показал бы «provider is not configured» вместо «временно,
повторите».

Тесты: unit fail-fast (stream() бросает 503 A_RUN_BEGIN_FAILED, ни байта в сокет,
user-строка не вставлена; RunAlreadyActiveError по-прежнему 409); unit клиентского
классификатора из ПОЛНОГО реального тела ответа с гвардом порядка.
2026-07-11 07:19:14 +03:00
agent_coder fc83ea28ab fix(metrics): bind на 127.0.0.1 по умолчанию + METRICS_BIND/METRICS_TOKEN (#486)
/metrics слушал на 0.0.0.0 без какой-либо аутентификации — auth-less
эндпоинт на всех интерфейсах. Теперь дефолтный bind — loopback 127.0.0.1;
env METRICS_BIND переопределяет интерфейс (0.0.0.0 для скрейпера в
отдельном контейнере, docmost:9464); опциональный METRICS_TOKEN включает
Bearer-аутентификацию (запросы без точного токена получают 401). Доки
скрейпа в .env.example обновлены.

Тест: unit на дефолтный bind + env-переопределение + резолв токена;
интеграционный по РЕАЛЬНОМУ сокету — listener забинден на 127.0.0.1,
без токена /metrics отдаётся, с токеном без/с неверным Bearer → 401, с
верным → 200.
2026-07-11 07:19:14 +03:00
agent_coder ee15278c8f fix(health): устранить утечку ioredis-клиента в /health-пробе (#486)
pingCheck строил new Redis(...) на КАЖДЫЙ вызов и делал disconnect() только
на success-пути. Пока Redis лежит, каждый тик health-пробы добавлял свежий,
вечно реконнектящийся клиент — неограниченный рост хэндлов/клиентов на всё
время недоступности Redis.

Теперь один долгоживущий probe-клиент, переиспользуемый между тиками:
lazyConnect (конструктор не бросает и не коннектится жадно),
maxRetriesPerRequest: 1 и enableOfflineQueue: false (проба фейлится быстро,
команды не буферизуются), плюс listener на 'error' (иначе unhandled error
роняет процесс). onModuleDestroy закрывает клиент при shutdown.

Тест: интеграционный — N проб при лежащем Redis (реальный refused-порт, не
мок поведения) создают РОВНО ОДИН клиент (на баге было бы N); onModuleDestroy
освобождает клиент, следующая проба лениво строит новый.
2026-07-11 07:19:14 +03:00
agent_coder 09ab92eccf fix(mcp): ELK-лейаут в worker_thread — таймаут через terminate() (#486)
elkjs.layout() возвращает Promise, но саму раскладку крутит СИНХРОННО и
блокирует поток целиком. На in-app хосте это был главный event loop:
патологический граф у капа 500 узлов вешал ВСЕ HTTP/SSE/loopback. Прежняя
защита (Promise.race с setTimeout(5s)) была иллюзией — таймер физически не
мог сработать, пока тот же поток заблокирован внутри elkjs (комментарий в
коде это сам признавал).

Теперь elk.layout() исполняется в worker_thread, а таймаут форсится
worker.terminate() — единственный способ прервать синхронный JS. Главный
поток остаётся свободным; на таймауте/ошибке — best-effort откат к
исходной модели, как и раньше. Лживый комментарий «can never wedge the
server» убран.

Тесты: unit на terminate-по-таймауту (крошечный ceiling → hard-kill →
исходная модель нетронута) и бенчмарк-гард на worst-case графе у капа
(500 узлов/~1000 рёбер раскладывается, а главный event loop продолжает
тикать во время раскладки).
2026-07-11 07:19:14 +03:00
48 changed files with 2938 additions and 304 deletions
+14
View File
@@ -335,6 +335,20 @@ MCP_DOCMOST_PASSWORD=
# VictoriaMetrics/Prometheus reaching it as <host>:<port>/metrics. # VictoriaMetrics/Prometheus reaching it as <host>:<port>/metrics.
# METRICS_PORT=9464 # METRICS_PORT=9464
# #
# METRICS_BIND — interface the /metrics listener binds to. DEFAULT 127.0.0.1
# (loopback only), so the unauthenticated endpoint is NOT exposed on all
# interfaces. If the scraper runs in a SEPARATE container and reaches this as
# docmost:9464, set METRICS_BIND=0.0.0.0 — but then also set METRICS_TOKEN
# and/or keep the port on a private network, since /metrics is otherwise open.
# METRICS_BIND=127.0.0.1
#
# METRICS_TOKEN — optional Bearer token guarding /metrics. When set, every
# scrape MUST send `Authorization: Bearer <token>` (others get 401). Configure
# the scraper with the same bearer token (e.g. VictoriaMetrics/vmagent
# `bearer_token`, Prometheus `authorization.credentials`). Leave unset only
# when the endpoint is bound to loopback or an otherwise-trusted network.
# METRICS_TOKEN=
#
# 2) CLIENT_TELEMETRY_ENABLED — the public client perf-telemetry sink. # 2) CLIENT_TELEMETRY_ENABLED — the public client perf-telemetry sink.
# OFF by default. When true, the unauthenticated POST /api/telemetry/vitals # OFF by default. When true, the unauthenticated POST /api/telemetry/vitals
# endpoint is registered and browsers collect + send web-vitals / editor # endpoint is registered and browsers collect + send web-vitals / editor
+1 -1
View File
@@ -470,7 +470,7 @@ Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirro
- **Errors must never be swallowed or shown as generic messages.** Every caught error MUST (1) be logged in full to the console/logger — error name, message, stack, `cause`, and (for HTTP/provider failures) the status code and response body — and (2) be surfaced to the user with a *specific, human-readable explanation of what actually went wrong*, never a bare generic string like "Something went wrong" / "Could not start recording" / "Transcription failed". Include the real reason (the underlying error/provider message) in the user-facing text. On the server, wrap third-party/provider failures with `describeProviderError` (or equivalent) and rethrow as a meaningful HTTP status + message — never let them collapse into an opaque 500. On the client, `console.error(<context>, err)` the raw error AND show the extracted reason (e.g. `err.response?.data?.message`, or the error `name: message`) in the notification. - **Errors must never be swallowed or shown as generic messages.** Every caught error MUST (1) be logged in full to the console/logger — error name, message, stack, `cause`, and (for HTTP/provider failures) the status code and response body — and (2) be surfaced to the user with a *specific, human-readable explanation of what actually went wrong*, never a bare generic string like "Something went wrong" / "Could not start recording" / "Transcription failed". Include the real reason (the underlying error/provider message) in the user-facing text. On the server, wrap third-party/provider failures with `describeProviderError` (or equivalent) and rethrow as a meaningful HTTP status + message — never let them collapse into an opaque 500. On the client, `console.error(<context>, err)` the raw error AND show the extracted reason (e.g. `err.response?.data?.message`, or the error `name: message`) in the notification.
- The version string shown in the UI comes from `APP_VERSION` (CI/Docker) or `git describe --tags --always` (local), resolved in `vite.config.ts` — not from `package.json`. - The version string shown in the UI comes from `APP_VERSION` (CI/Docker) or `git describe --tags --always` (local), resolved in `vite.config.ts` — not from `package.json`.
- Server TS config is permissive (`noImplicitAny: false`, `strictNullChecks: false`, `no-explicit-any` lint disabled). Follow the existing relaxed style rather than tightening types broadly. - Server TS config is permissive (`noImplicitAny: false`, `strictNullChecks: false`, `no-explicit-any` lint disabled). Follow the existing relaxed style rather than tightening types broadly.
- Dependency versions are heavily pinned via `pnpm.overrides` and `pnpm.patchedDependencies` (`scimmy`, `yjs`, `ai`) in the root `package.json`. Don't bump pinned/patched deps casually; the patches and overrides exist for compatibility/security reasons. The `ai@6.0.134` patch disables the SDK's O(n²) cumulative `partialOutput` accumulation when no output strategy is requested (server heap OOM on long agent runs, #184; tripwire test: `apps/server/src/integrations/ai/ai-sdk-partial-output.patch.spec.ts`) — it MUST be re-created via `pnpm patch` when bumping `ai`. - Dependency versions are heavily pinned via `pnpm.overrides` and `pnpm.patchedDependencies` (`scimmy`, `yjs`, `ai`) in the root `package.json`. Don't bump pinned/patched deps casually; the patches and overrides exist for compatibility/security reasons. The `ai@6.0.134` patch carries TWO independent server fixes, each with its own tripwire test: (1) it disables the SDK's O(n²) cumulative `partialOutput` accumulation when no output strategy is requested (server heap OOM on long agent runs, #184; tripwire: `apps/server/src/integrations/ai/ai-sdk-partial-output.patch.spec.ts`); (2) it fixes `writeToServerResponse`'s drain-hang — the loop awaited only `"drain"` under backpressure, so a mid-write client disconnect parked the pipe forever and leaked the reader/buffers until restart; it now races `"drain"` against `"close"`/`"error"`, cancels the reader on disconnect, and swallows the fire-and-forget read rejection (#486; tripwire: `apps/server/src/integrations/ai/ai-sdk-drain-hang.patch.spec.ts`). Both tripwires assert BOTH installed dist builds carry their patch marker. The patch MUST be re-created via `pnpm patch` when bumping `ai`.
- **The MCP tool inventory in `SERVER_INSTRUCTIONS` is GENERATED from the registry** (`packages/mcp/src/server-instructions.ts`: `buildToolInventory()` over `SHARED_TOOL_SPECS`) and spliced into the hand-written routing prose (`ROUTING_PROSE`). So adding/renaming/removing a **shared** spec in `packages/mcp/src/tool-specs.ts` auto-updates the `<tool_inventory>` — no manual `SERVER_INSTRUCTIONS` edit needed. Only an **inline** MCP-only tool (those registered via `server.registerTool(...)` in `index.ts`, not through the registry) needs a one-line entry in `INLINE_MCP_INVENTORY`. Enforced by `packages/mcp/test/unit/tool-inventory.test.mjs`, which fails when a registered tool is missing from the generated inventory (there is no `EXCEPTIONS` opt-out anymore — every tool must appear). Update `ROUTING_PROSE` when a tool's *intent guidance* (when-to-use) changes. `packages/mcp/build/` is gitignored and rebuilt in CI/Docker via `pnpm build` (same convention as `git-sync`/`prosemirror-markdown`) — never commit it; rebuild locally after editing to run the tests. - **The MCP tool inventory in `SERVER_INSTRUCTIONS` is GENERATED from the registry** (`packages/mcp/src/server-instructions.ts`: `buildToolInventory()` over `SHARED_TOOL_SPECS`) and spliced into the hand-written routing prose (`ROUTING_PROSE`). So adding/renaming/removing a **shared** spec in `packages/mcp/src/tool-specs.ts` auto-updates the `<tool_inventory>` — no manual `SERVER_INSTRUCTIONS` edit needed. Only an **inline** MCP-only tool (those registered via `server.registerTool(...)` in `index.ts`, not through the registry) needs a one-line entry in `INLINE_MCP_INVENTORY`. Enforced by `packages/mcp/test/unit/tool-inventory.test.mjs`, which fails when a registered tool is missing from the generated inventory (there is no `EXCEPTIONS` opt-out anymore — every tool must appear). Update `ROUTING_PROSE` when a tool's *intent guidance* (when-to-use) changes. `packages/mcp/build/` is gitignored and rebuilt in CI/Docker via `pnpm build` (same convention as `git-sync`/`prosemirror-markdown`) — never commit it; rebuild locally after editing to run the tests.
## CI / release ## CI / release
+63
View File
@@ -115,6 +115,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
the old ProseMirror-JSON output. Released together with the `#411`/`#412` the old ProseMirror-JSON output. Released together with the `#411`/`#412`
breaking window so external configs break exactly once. (#413) breaking window so external configs break exactly once. (#413)
- **The Prometheus `/metrics` listener now binds to `127.0.0.1` (loopback) by
default instead of `0.0.0.0` (all interfaces).** This closes an unauthenticated
endpoint that was previously reachable on every interface. **DEPLOY MIGRATION —
cross-container scraping breaks silently otherwise:** if your scraper runs in a
SEPARATE container and reaches the app as `docmost:9464` (the exact topology the
old `0.0.0.0` hardcode served), you MUST now set `METRICS_BIND=0.0.0.0` — and,
because that re-exposes the endpoint, also set `METRICS_TOKEN=<secret>` and
configure the scraper with a matching Bearer token. Without `METRICS_BIND`, the
scraper can no longer connect and metrics go dark with no error. See the
`METRICS_BIND` / `METRICS_TOKEN` block in `.env.example` for the migration.
Same-host (loopback) scrapers need no change. (#486)
### Added ### Added
- **Place several images side by side in a row.** A new "Inline (side by - **Place several images side by side in a row.** A new "Inline (side by
@@ -310,6 +322,39 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`tee()` branch of the stream result — a ~20-step, ~28k-chunk agent run `tee()` branch of the stream result — a ~20-step, ~28k-chunk agent run
retained ~1.7 GB and OOM'd the 2 GB JS heap. Streaming granularity is retained ~1.7 GB and OOM'd the 2 GB JS heap. Streaming granularity is
unchanged; the patch must be re-created if `ai` is ever bumped. (#184) unchanged; the patch must be re-created if `ai` is ever bumped. (#184)
- **The server no longer leaks a hung stream pipe on every mid-run client
disconnect.** The same `ai@6.0.134` pnpm patch now also fixes the SDK's
`writeToServerResponse`, which awaited only a `"drain"` event under
backpressure: when a client disconnected mid-write the socket never drained, so
the write loop parked forever, `response.end()` was unreachable, and the stream
reader plus buffered chunks were pinned until process restart (every mid-run
disconnect in autonomous mode leaked one). The patch races `"drain"` against
`"close"`/`"error"`, cancels the reader and ends the response on disconnect, and
swallows the fire-and-forget read rejection instead of crashing on an
unhandledRejection. (#486)
- **A failed autonomous agent-run start no longer becomes an unstoppable ghost
run.** When `beginRun` failed for a transient reason (e.g. a DB-pool blip),
the turn previously continued with NO run row — invisible to `/stop`, not
aborted on disconnect, and able to slip a second run past the one-run-per-chat
gate, leaving an unstoppable run until restart. The turn now fails fast with an
honest `503 A_RUN_BEGIN_FAILED` before the first byte (no orphan state), and the
client shows a "temporary — please try again" message instead of a misleading
"provider not configured". (#486)
- **A pathological draw.io graph can no longer wedge the whole server.** The ELK
auto-layout (`layout:"elk"`) ran elkjs synchronously on the main event loop, so
a graph at the node/edge cap blocked ALL HTTP/SSE/loopback traffic while it
churned — and the old `setTimeout` "timeout" could never fire because the same
thread was blocked. Layout now runs in a worker thread with the timeout enforced
by `worker.terminate()`; the main loop stays responsive. (#486)
- **The `/health` Redis probe no longer leaks a client on every tick while Redis
is down.** It built a new `ioredis` client per probe and disconnected it only on
success, so during an outage each health tick added another forever-reconnecting
client (an unbounded handle leak). A single long-lived probe client is now
reused and closed on shutdown. (#486)
- **Internal links in exported Markdown no longer lose their visible text.** A - **Internal links in exported Markdown no longer lose their visible text.** A
link whose target page name had no file extension (e.g. a bare title) was link whose target page name had no file extension (e.g. a bare title) was
collapsed to empty text during export, producing an unclickable, label-less collapsed to empty text during export, producing an unclickable, label-less
@@ -386,6 +431,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
share); any other value now returns the generic "not found" instead of share); any other value now returns the generic "not found" instead of
serving the page. (#218) serving the page. (#218)
- **Tool and provider error text no longer leaks to anonymous readers in the
public-share AI chat.** A failing tool's raw error (which could carry an
internal page title or a stack fragment) and a provider error (which bundles the
provider `statusCode` and response body — potentially the internal baseUrl or
model name) were streamed verbatim to the anonymous reader over SSE. Errors are
now sanitized at the source: the share toolset collapses any unclassified tool
error to a safe generic string (safe, classified tool messages still pass
through for the model's self-correction), and the anonymous stream `onError`
maps provider failures to a fixed set of neutral strings — the full detail goes
only to the server log. A UI render gate is layered on top. (closes #394)
- **The Prometheus `/metrics` endpoint can now require Bearer authentication and
is loopback-bound by default.** Previously it listened on all interfaces with no
auth. Setting `METRICS_TOKEN` requires every scrape to present
`Authorization: Bearer <token>` (compared in constant time), and the listener
defaults to `127.0.0.1` (see the Breaking Changes entry for the cross-container
migration). (#486)
## [0.94.0] - 2026-06-26 ## [0.94.0] - 2026-06-26
This release makes AI chat durable and fast: assistant turns are persisted to This release makes AI chat durable and fast: assistant turns are persisted to
@@ -203,6 +203,52 @@ describe("ChatThread — send now (#198)", () => {
}); });
}); });
// #486: the final onFinish -> flushNext() must be gated on the live-mount flag.
// A clean onFinish can land AFTER the thread unmounts (New-chat / chat-switch
// mid-stream — the async attach/resume settles late); flushing then dequeues and
// re-POSTs a queued message from an abandoned thread (a "ghost" send).
describe("ChatThread — onFinish flush gated on mount (#486)", () => {
beforeEach(resetState);
afterEach(cleanup);
it("a clean onFinish WHILE MOUNTED flushes the queued message (control)", () => {
renderThread();
fireEvent.click(screen.getByTestId("queue-btn")); // enqueue "queued text"
expect(h.state.sendMessage).not.toHaveBeenCalled();
act(() => {
h.state.onFinish?.({
message: { id: "a", role: "assistant", parts: [] },
isAbort: false,
isDisconnect: false,
isError: false,
});
});
// Mounted: the queue flushes normally.
expect(h.state.sendMessage).toHaveBeenCalledWith({ text: "queued text" });
});
it("a clean onFinish AFTER unmount does NOT flush (no ghost send)", () => {
const { unmount } = renderThread();
fireEvent.click(screen.getByTestId("queue-btn")); // enqueue "queued text"
h.state.sendMessage.mockClear();
// Chat switched away mid-stream: the streamer unmounts...
unmount();
// ...and a late, clean onFinish lands on the abandoned thread.
act(() => {
h.state.onFinish?.({
message: { id: "a", role: "assistant", parts: [] },
isAbort: false,
isDisconnect: false,
isError: false,
});
});
// Gated on mountedRef: NOTHING is sent from the dead thread.
expect(h.state.sendMessage).not.toHaveBeenCalled();
});
});
// #396: in autonomous mode a live sendNow must additionally request the // #396: in autonomous mode a live sendNow must additionally request the
// AUTHORITATIVE server stop of the detached run (a local abort is only a client // AUTHORITATIVE server stop of the detached run (a local abort is only a client
// disconnect the server ignores) and arm a bounded 409 retry so the re-POST // disconnect the server ignores) and arm a bounded 409 retry so the re-POST
@@ -659,7 +659,13 @@ export default function ChatThread({
return; return;
} }
if (isAbort || isDisconnect || isError) return; if (isAbort || isDisconnect || isError) return;
flushNext(); // Gate the final flush on the live-mount flag (#486): a clean onFinish can
// land AFTER this thread unmounted (a New-chat / chat-switch mid-stream —
// the async attach/resume settles late). Flushing then dequeues and POSTs a
// queued message from an abandoned thread — a "ghost" send / ghost chat.
// Every other queue side effect already guards on mountedRef; this last one
// was the gap.
if (mountedRef.current) flushNext();
}, },
// `onError` runs in addition to `onFinish` (which ai@6 also calls on error). // `onError` runs in addition to `onFinish` (which ai@6 also calls on error).
// Log the raw failure here for devtools; the UI shows a friendly classified // Log the raw failure here for devtools; the UI shows a friendly classified
@@ -47,6 +47,13 @@ interface MessageItemProps {
* agent's raw query/argument text. * agent's raw query/argument text.
*/ */
showInput?: boolean; showInput?: boolean;
/**
* Forwarded to ToolCallCard: whether a failed tool card renders its raw
* errorText. Defaults to true (internal chat). The public share passes false so
* internal detail in a tool error is never painted (belt to the server-side
* byte sanitization).
*/
showErrors?: boolean;
/** /**
* Neutralize internal/relative markdown links in the rendered answer (drop * Neutralize internal/relative markdown links in the rendered answer (drop
* their href so they become inert text). Defaults to false (internal chat, * their href so they become inert text). Defaults to false (internal chat,
@@ -125,6 +132,7 @@ function MessageItem({
message, message,
showCitations = true, showCitations = true,
showInput = true, showInput = true,
showErrors = true,
neutralizeInternalLinks = false, neutralizeInternalLinks = false,
assistantName, assistantName,
turnStreaming = false, turnStreaming = false,
@@ -219,6 +227,7 @@ function MessageItem({
part={part as unknown as ToolUiPart} part={part as unknown as ToolUiPart}
showCitations={showCitations} showCitations={showCitations}
showInput={showInput} showInput={showInput}
showErrors={showErrors}
/> />
); );
} }
@@ -284,6 +293,7 @@ export function arePropsEqual(
prev.signature === next.signature && prev.signature === next.signature &&
prev.showCitations === next.showCitations && prev.showCitations === next.showCitations &&
prev.showInput === next.showInput && prev.showInput === next.showInput &&
prev.showErrors === next.showErrors &&
prev.neutralizeInternalLinks === next.neutralizeInternalLinks && prev.neutralizeInternalLinks === next.neutralizeInternalLinks &&
prev.assistantName === next.assistantName && prev.assistantName === next.assistantName &&
// The turn-end flip re-renders every row once (cheap, terminal event) — // The turn-end flip re-renders every row once (cheap, terminal event) —
@@ -32,6 +32,12 @@ interface MessageListProps {
* doesn't see the agent's raw query/argument text. * doesn't see the agent's raw query/argument text.
*/ */
showInput?: boolean; showInput?: boolean;
/**
* Forwarded to MessageItem -> ToolCallCard: whether a failed tool card renders
* its raw errorText. Defaults to true (internal chat). The public share passes
* false so internal detail in a tool error is never painted.
*/
showErrors?: boolean;
/** /**
* Forwarded to MessageItem: neutralize internal/relative markdown links in * Forwarded to MessageItem: neutralize internal/relative markdown links in
* the rendered answers (drop their href so they render as inert text). * the rendered answers (drop their href so they render as inert text).
@@ -127,6 +133,7 @@ export default function MessageList({
emptyState, emptyState,
showCitations = true, showCitations = true,
showInput = true, showInput = true,
showErrors = true,
neutralizeInternalLinks = false, neutralizeInternalLinks = false,
assistantName, assistantName,
}: MessageListProps) { }: MessageListProps) {
@@ -217,6 +224,7 @@ export default function MessageList({
signature={messageSignature(message)} signature={messageSignature(message)}
showCitations={showCitations} showCitations={showCitations}
showInput={showInput} showInput={showInput}
showErrors={showErrors}
neutralizeInternalLinks={neutralizeInternalLinks} neutralizeInternalLinks={neutralizeInternalLinks}
assistantName={assistantName} assistantName={assistantName}
// Turn-level liveness, gated to the TAIL row: only the tail message // Turn-level liveness, gated to the TAIL row: only the tail message
@@ -30,6 +30,16 @@ interface ToolCallCardProps {
* the extra summary line, leaving the card (the action log) intact. * the extra summary line, leaving the card (the action log) intact.
*/ */
showInput?: boolean; showInput?: boolean;
/**
* Whether to render the tool's raw errorText on a failed call. Defaults to true
* (the internal chat, where the operator may debug). The public share passes
* false: a tool error string can carry internal detail (an internal page title,
* a stack fragment, a provider message). This is the RENDER gate only the
* authoritative fix also sanitizes the bytes server-side (see
* PublicShareChatToolsService.forShare), so a share reader never receives raw
* error text over the wire, not just never sees it painted (#394).
*/
showErrors?: boolean;
} }
/** /**
@@ -41,6 +51,7 @@ export default function ToolCallCard({
part, part,
showCitations = true, showCitations = true,
showInput = true, showInput = true,
showErrors = true,
}: ToolCallCardProps) { }: ToolCallCardProps) {
const { t } = useTranslation(); const { t } = useTranslation();
const toolName = getToolName(part); const toolName = getToolName(part);
@@ -74,7 +85,7 @@ export default function ToolCallCard({
</Text> </Text>
)} )}
{state === "error" && part.errorText && ( {state === "error" && showErrors && part.errorText && (
<Text size="xs" c="red" mt={2}> <Text size="xs" c="red" mt={2}>
{part.errorText} {part.errorText}
</Text> </Text>
@@ -23,6 +23,25 @@ describe("describeChatError", () => {
}); });
}); });
it("classifies an A_RUN_BEGIN_FAILED 503 as a temporary run-start failure, NOT provider-not-configured (#486)", () => {
// The FULL real body the server writes for a beginRun failure: a
// ServiceUnavailableException(object) whose response is serialized verbatim
// onto the raw socket, self-describing statusCode 503 + the run-start code.
const body =
'{"message":"Could not start the agent run. This is usually temporary — please try again.","code":"A_RUN_BEGIN_FAILED","statusCode":503}';
expect(describeChatError(body, t)).toEqual({
title: "Could not start the run",
detail:
"The agent run could not be started. This is usually temporary — please try again.",
});
// ORDER GUARD: even though the body ALSO carries statusCode 503 (which the
// generic branch matches), the A_RUN_BEGIN_FAILED branch runs first, so it is
// never mislabeled "AI provider not configured".
expect(describeChatError(body, t).title).not.toBe(
"AI provider not configured",
);
});
it("classifies a dropped connection (ECONNRESET) as a lost-connection error", () => { it("classifies a dropped connection (ECONNRESET) as a lost-connection error", () => {
expect( expect(
describeChatError("Cannot connect to API: read ECONNRESET", t).title, describeChatError("Cannot connect to API: read ECONNRESET", t).title,
@@ -24,6 +24,21 @@ export function describeChatError(
): ChatErrorView { ): ChatErrorView {
const msg = message ?? ""; const msg = message ?? "";
// Our own "could not start the run" gate (A_RUN_BEGIN_FAILED, #486): a 503
// whose body carries this code is a TEMPORARY server-side failure while
// starting the run (e.g. a DB-pool blip), NOT an unconfigured provider. It MUST
// be matched STRICTLY BEFORE the generic 503 branch below, which would
// otherwise mislabel it "The AI provider is not configured" and tell the user
// to call an admin instead of just retrying.
if (/"code"\s*:\s*"A_RUN_BEGIN_FAILED"/.test(msg)) {
return {
title: t("Could not start the run"),
detail: t(
"The agent run could not be started. This is usually temporary — please try again.",
),
};
}
if (/"statusCode"\s*:\s*403\b/.test(msg)) { if (/"statusCode"\s*:\s*403\b/.test(msg)) {
return { return {
title: t("AI chat is disabled"), title: t("AI chat is disabled"),
@@ -168,6 +168,10 @@ export default function ShareAiWidget({
// Anonymous reader: suppress the tool-argument summary line so the // Anonymous reader: suppress the tool-argument summary line so the
// agent's raw query/argument text isn't shown on the public share. // agent's raw query/argument text isn't shown on the public share.
showInput={false} showInput={false}
// Anonymous reader: never paint a tool's raw errorText (it can carry
// internal detail). This is the render gate; the bytes are also
// sanitized server-side in PublicShareChatToolsService.forShare (#394).
showErrors={false}
// Anonymous reader: neutralize internal/relative links in the // Anonymous reader: neutralize internal/relative links in the
// assistant's markdown so internal UUIDs/auth-gated routes don't // assistant's markdown so internal UUIDs/auth-gated routes don't
// leak as clickable links (external http(s) links are kept). // leak as clickable links (external http(s) links are kept).
@@ -0,0 +1,109 @@
import {
ConflictException,
Logger,
ServiceUnavailableException,
} from '@nestjs/common';
import { AiChatService } from './ai-chat.service';
import { RunAlreadyActiveError } from './ai-chat-run.service';
/**
* Fail-fast guard for beginRun failures (#486, commit 4).
*
* When runHooks.begin() rejects for a reason OTHER than RunAlreadyActiveError
* (e.g. a DB-pool blip), the turn must NOT continue untracked. The old code
* logged and streamed anyway, leaving a run with NO run-row: in autonomous mode
* nobody could abort it (/stop can't see it, disconnect doesn't abort it, and the
* one-run gate would admit a SECOND run) an unstoppable invisible run until
* restart. The fix throws A_RUN_BEGIN_FAILED (503) BEFORE the first byte and
* before the user row is persisted.
*
* We drive `stream()` directly on a prototype instance wired with only the
* collaborators it touches before the throw, so the assertion is on the REAL
* control flow, not a mock of it.
*/
describe('AiChatService beginRun failure (#486)', () => {
function makeService(insertSpy: jest.Mock): AiChatService {
// Bypass the (heavy) DI constructor: exercise the real stream() method on a
// bare prototype instance with just the fields reached before the throw.
// `any` because the private `logger` field makes a typed intersection collapse.
const svc = Object.create(AiChatService.prototype);
svc.aiChatRepo = {
// Existing chat -> no insert path; chatId is kept as-is.
findById: jest.fn().mockResolvedValue({ id: 'chat1' }),
};
svc.aiChatMessageRepo = { insert: insertSpy };
svc.logger = new Logger('test');
return svc as AiChatService;
}
const baseArgs = () => {
const write = jest.fn();
const res = {
raw: { write, writableEnded: false, headersSent: false },
};
return {
user: { id: 'u1' } as never,
workspace: { id: 'w1' } as never,
sessionId: 's1',
// openPage undefined -> resolveOpenPageContext returns null without any DB
// call; chatId present -> the existing-chat path.
body: { chatId: 'chat1', messages: [] } as never,
res: res as never,
signal: new AbortController().signal,
model: {} as never,
role: null,
write,
};
};
it('throws A_RUN_BEGIN_FAILED (503) before the first byte and before persisting the user turn', async () => {
const insertSpy = jest.fn();
const svc = makeService(insertSpy);
const { write, ...args } = baseArgs();
const runHooks = {
begin: jest.fn().mockRejectedValue(new Error('DB pool exhausted')),
} as never;
let caught: unknown;
try {
await svc.stream({ ...args, runHooks });
} catch (e) {
caught = e;
}
expect(caught).toBeInstanceOf(ServiceUnavailableException);
const http = caught as ServiceUnavailableException;
expect(http.getStatus()).toBe(503);
expect(http.getResponse()).toMatchObject({ code: 'A_RUN_BEGIN_FAILED' });
// Fail-fast: nothing was written to the socket and NO user message row was
// persisted, so the turn left no orphan state to clean up.
expect(write).not.toHaveBeenCalled();
expect(insertSpy).not.toHaveBeenCalled();
});
it('still maps a lost-the-race RunAlreadyActiveError to a 409, not A_RUN_BEGIN_FAILED', async () => {
const insertSpy = jest.fn();
const svc = makeService(insertSpy);
const { write, ...args } = baseArgs();
const runHooks = {
begin: jest.fn().mockRejectedValue(new RunAlreadyActiveError('chat1')),
} as never;
let caught: unknown;
try {
await svc.stream({ ...args, runHooks });
} catch (e) {
caught = e;
}
expect(caught).toBeInstanceOf(ConflictException);
expect((caught as ConflictException).getResponse()).toMatchObject({
code: 'A_RUN_ALREADY_ACTIVE',
});
expect(write).not.toHaveBeenCalled();
expect(insertSpy).not.toHaveBeenCalled();
});
});
@@ -1,4 +1,8 @@
import { ConflictException, Logger } from '@nestjs/common'; import {
ConflictException,
Logger,
ServiceUnavailableException,
} from '@nestjs/common';
// Mock the AI SDK so we can PROVE no provider call is made for the turn we are // Mock the AI SDK so we can PROVE no provider call is made for the turn we are
// about to reject. The race rejection happens at runHooks.begin(), long before // about to reject. The race rejection happens at runHooks.begin(), long before
@@ -360,22 +364,22 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
}); });
/** /**
* F14 the begin-failure RESILIENCE branch (the `else` of the run-race guard). * F14 the begin-failure branch (the `else` of the run-race guard).
* *
* stream() wraps runHooks.begin in try/catch with TWO branches: * stream() wraps runHooks.begin in try/catch with TWO branches:
* - RunAlreadyActiveError -> 409 ConflictException (pinned above). * - RunAlreadyActiveError -> 409 ConflictException (pinned above).
* - ANY OTHER begin failure -> SWALLOW + continue UNTRACKED on the socket signal * - ANY OTHER begin failure -> throw ServiceUnavailableException(A_RUN_BEGIN_FAILED)
* (legacy fallback): it logs "...streaming without run tracking", leaves * BEFORE the first byte (#486, commit 4).
* `effectiveSignal = signal` (runId undefined) and serves the turn anyway.
* *
* The contract: a transient beginRun failure (e.g. a non-unique DB error inserting * POLICY CHANGE (#486): the OLD contract here was "SWALLOW + stream the turn
* the run row) must STILL serve the user's turn it must NOT re-throw and must NOT * UNTRACKED on the socket signal". That was reversed: an untracked run is
* be misclassified as a 409. A regression that re-threw here would break EVERY turn * invisible to /stop, is not aborted on disconnect, and slips past the one-run
* on a begin failure with nothing to catch it. This branch is otherwise undriven by * gate an unstoppable ghost run in autonomous mode. Now a plain begin failure
* any spec, so it is pinned here SEPARATELY from the 409 path: a plain begin error * FAILS the turn fast with a 503 A_RUN_BEGIN_FAILED, before any user row is
* proceeds to streamText with the SOCKET signal and still persists the user turn. * persisted and before streamText runs. This case is INVERTED (not deleted) so
* the "plain begin failure" path stays explicitly pinned under the new policy.
*/ */
describe('AiChatService.stream — begin-failure resilience / legacy fallback (#184 F14)', () => { describe('AiChatService.stream — begin-failure fails the turn (#184 F14 / #486)', () => {
const streamTextMock = streamText as unknown as jest.Mock; const streamTextMock = streamText as unknown as jest.Mock;
function makeStreamResult() { function makeStreamResult() {
@@ -455,7 +459,7 @@ describe('AiChatService.stream — begin-failure resilience / legacy fallback (#
afterEach(() => jest.restoreAllMocks()); afterEach(() => jest.restoreAllMocks());
it('a PLAIN begin() failure (NOT RunAlreadyActiveError) does NOT 409 — it swallows, logs, and streams the turn UNTRACKED on the socket signal', async () => { it('a PLAIN begin() failure (NOT RunAlreadyActiveError) FAILS the turn with a 503 A_RUN_BEGIN_FAILED before the first byte — NO untracked stream (#486)', async () => {
const errorSpy = jest const errorSpy = jest
.spyOn(Logger.prototype, 'error') .spyOn(Logger.prototype, 'error')
.mockImplementation(() => undefined as never); .mockImplementation(() => undefined as never);
@@ -487,28 +491,26 @@ describe('AiChatService.stream — begin-failure resilience / legacy fallback (#
} as never, } as never,
}); });
// The turn proceeds: NO throw at all (in particular NOT a 409). // NEW POLICY: the turn is REJECTED with a 503 A_RUN_BEGIN_FAILED (not a 409,
await expect(promise).resolves.toBeUndefined(); // and NOT swallowed into an untracked stream).
await expect(promise).rejects.toBeInstanceOf(ServiceUnavailableException);
const err = (await promise.catch(
(e) => e,
)) as ServiceUnavailableException;
expect(err.getStatus()).toBe(503);
expect(err.getResponse()).toMatchObject({ code: 'A_RUN_BEGIN_FAILED' });
expect(begin).toHaveBeenCalledTimes(1); expect(begin).toHaveBeenCalledTimes(1);
// The resilience branch logged the legacy-fallback warning. // It logged the fail-the-turn line.
expect(errorSpy).toHaveBeenCalledWith( expect(errorSpy).toHaveBeenCalledWith(
expect.stringContaining('streaming without run tracking'), expect.stringContaining('failing the turn'),
expect.anything(), expect.anything(),
); );
// The turn really streamed: the user message was persisted and streamText ran. // Fail-fast: the turn NEVER streamed — no user row persisted, no streamText
expect(aiChatMessageRepo.insert).toHaveBeenCalled(); // call, so no orphan/untracked run was left behind.
expect(streamTextMock).toHaveBeenCalledTimes(1); expect(aiChatMessageRepo.insert).not.toHaveBeenCalled();
expect(streamTextMock).not.toHaveBeenCalled();
// The decisive wiring: with no run handle, the fallback uses the SOCKET signal
// (effectiveSignal = signal, runId undefined) — not a run-bound signal. #444:
// the signal is unioned with the degeneration controller via AbortSignal.any,
// so assert the socket abort still reaches the turn rather than identity.
const passed = streamTextMock.mock.calls[0][0].abortSignal as AbortSignal;
expect(passed.aborted).toBe(false);
socketController.abort();
expect(passed.aborted).toBe(true);
}); });
}); });
@@ -4,6 +4,7 @@ import {
Injectable, Injectable,
Logger, Logger,
OnModuleInit, OnModuleInit,
ServiceUnavailableException,
} from '@nestjs/common'; } from '@nestjs/common';
import { FastifyReply } from 'fastify'; import { FastifyReply } from 'fastify';
import { import {
@@ -55,6 +56,7 @@ import {
import { import {
isDegenerateOutput, isDegenerateOutput,
truncateDegeneratedTail, truncateDegeneratedTail,
shouldCheckDegeneration,
} from './output-degeneration'; } from './output-degeneration';
// Max agent steps per turn. One step = one model generation; a step that calls // Max agent steps per turn. One step = one model generation; a step that calls
@@ -756,6 +758,13 @@ export class AiChatService implements OnModuleInit {
// or violate the page_id FK on insert (this runs after res.hijack(), so a // or violate the page_id FK on insert (this runs after res.hijack(), so a
// DB error would break the stream). // DB error would break the stream).
const originPageId: string | null = openPageContext?.id ?? null; const originPageId: string | null = openPageContext?.id ?? null;
// ORPHAN-ON-BEGIN-FAILURE tradeoff (#486, B3): the chat row is inserted
// HERE, before runHooks.begin below. If begin fails (e.g. a 503 / run-slot
// rejection) the turn aborts before the client is told this new chatId, so
// an empty chat is left behind and a retry mints ANOTHER one. We accept this
// over reordering: begin needs a chatId to bind the run to, and inserting
// the chat first keeps the id stable + the FK/history-join invariants above
// intact. Orphan empty chats are cheap and swept by normal chat cleanup.
const chat = await this.aiChatRepo.insert({ const chat = await this.aiChatRepo.insert({
creatorId: user.id, creatorId: user.id,
workspaceId: workspace.id, workspaceId: workspace.id,
@@ -797,12 +806,32 @@ export class AiChatService implements OnModuleInit {
code: 'A_RUN_ALREADY_ACTIVE', code: 'A_RUN_ALREADY_ACTIVE',
}); });
} }
// Any OTHER run-start failure must not break the turn — fall back to the // Any OTHER run-start failure (e.g. a DB-pool blip) must FAIL THE TURN,
// socket signal (legacy behavior) and stream anyway. // not silently stream without a run-row. The old fallback let the turn
// continue untracked: in autonomous mode nobody could then abort it —
// /stop can't see a run that doesn't exist, a client disconnect doesn't
// abort it, and the one-run-per-chat gate would let a SECOND run in. That
// is an unstoppable, invisible run until process restart. Reject NOW,
// BEFORE the first byte (nothing is written yet, no user row inserted, no
// MCP lease taken), so the controller's post-hijack catch turns this
// HttpException into an honest 503 on the raw socket. Same policy for BOTH
// modes — #487 inherits it (no mode-branching here).
this.logger.error( this.logger.error(
`Failed to begin agent run (chat ${chatId}); streaming without run tracking`, `Failed to begin agent run (chat ${chatId}); failing the turn`,
err as Error, err as Error,
); );
throw new ServiceUnavailableException({
message:
'Could not start the agent run. This is usually temporary — please try again.',
code: 'A_RUN_BEGIN_FAILED',
// Self-describe the status in the body: the controller's post-hijack
// catch writes getResponse() verbatim onto the raw socket, and an
// object-arg HttpException does NOT inject statusCode. Without it the
// client's 503 classifier (which reads the body JSON) could not see the
// status. With it present, the client's A_RUN_BEGIN_FAILED branch (which
// runs strictly before the generic-503 branch) shows "temporary, retry".
statusCode: 503,
});
} }
} }
@@ -1080,7 +1109,6 @@ export class AiChatService implements OnModuleInit {
const degenerationController = new AbortController(); const degenerationController = new AbortController();
let degenerationDetected = false; let degenerationDetected = false;
let lastDegenerationCheckLen = 0; let lastDegenerationCheckLen = 0;
const DEGENERATION_CHECK_STEP = 2000;
// Step-granular durability (#183): create the assistant row UPFRONT in the // Step-granular durability (#183): create the assistant row UPFRONT in the
// 'streaming' state (before any token), then UPDATE it as each step finishes // 'streaming' state (before any token), then UPDATE it as each step finishes
@@ -1254,8 +1282,10 @@ export class AiChatService implements OnModuleInit {
// trigger, abort the run ONCE with a distinguishable reason. // trigger, abort the run ONCE with a distinguishable reason.
if ( if (
!degenerationDetected && !degenerationDetected &&
inProgressText.length - lastDegenerationCheckLen >= shouldCheckDegeneration(
DEGENERATION_CHECK_STEP inProgressText.length,
lastDegenerationCheckLen,
)
) { ) {
lastDegenerationCheckLen = inProgressText.length; lastDegenerationCheckLen = inProgressText.length;
if (isDegenerateOutput(inProgressText)) { if (isDegenerateOutput(inProgressText)) {
@@ -1275,6 +1305,13 @@ export class AiChatService implements OnModuleInit {
// the in-progress accumulator for the next step. // the in-progress accumulator for the next step.
capturedSteps.push(step as StepLike); capturedSteps.push(step as StepLike);
inProgressText = ''; inProgressText = '';
// Reset the degeneration-check watermark too (#486): it tracks a byte
// offset INTO inProgressText, so once that resets to '' a stale (large)
// mark makes `inProgressText.length - lastDegenerationCheckLen` go
// negative and the throttled detector stays silent until a later step's
// text re-grows past the old offset — a whole degenerate step could slip
// through undetected. Zeroing it re-arms the check from the next byte.
lastDegenerationCheckLen = 0;
// Step-granular durability (#183): persist this finished step (its text + // Step-granular durability (#183): persist this finished step (its text +
// tool calls + tool RESULTS) the moment it ends, so a process death after // tool calls + tool RESULTS) the moment it ends, so a process death after
// this point still recovers the step. Not awaited here (never block the // this point still recovers the step. Not awaited here (never block the
@@ -1,11 +1,24 @@
import { Logger } from '@nestjs/common';
import { streamText } from 'ai';
import { import {
hasRepeatedLineRun, hasRepeatedLineRun,
hasPeriodicTail, hasPeriodicTail,
isDegenerateOutput, isDegenerateOutput,
truncateDegeneratedTail, truncateDegeneratedTail,
shouldCheckDegeneration,
DEGENERATION_CHECK_STEP,
REPEATED_LINES_THRESHOLD, REPEATED_LINES_THRESHOLD,
MIN_PERIOD_REPEATS, MIN_PERIOD_REPEATS,
} from './output-degeneration'; } from './output-degeneration';
import { AiChatService } from './ai-chat.service';
// Mock ONLY streamText so we can capture the onChunk/onStepFinish callbacks the
// service registers and drive them by hand; every other `ai` export the service
// uses (convertToModelMessages, stepCountIs, …) stays real.
jest.mock('ai', () => {
const actual = jest.requireActual('ai');
return { ...actual, streamText: jest.fn() };
});
/** /**
* Unit tests for the token-degeneration detector (#444) the sole anti-babble * Unit tests for the token-degeneration detector (#444) the sole anti-babble
@@ -180,3 +193,188 @@ describe('truncateDegeneratedTail', () => {
expect(truncateDegeneratedTail(text)).toBe(text); expect(truncateDegeneratedTail(text)).toBe(text);
}); });
}); });
/**
* Throttle + step-boundary reset (#486). The stream keeps a watermark
* (`lastDegenerationCheckLen`) that is an OFFSET into the accumulated step text.
* On a step boundary the accumulator resets to '', so the watermark MUST reset to
* 0 too otherwise the throttle goes silent for the whole next step. These tests
* pin the pure decision AND the reset property that ai-chat.service.onStepFinish
* now enforces.
*/
describe('shouldCheckDegeneration (throttle) + step-boundary reset (#486)', () => {
it('fires once the text grows a full DEGENERATION_CHECK_STEP past the mark', () => {
expect(shouldCheckDegeneration(DEGENERATION_CHECK_STEP, 0)).toBe(true);
expect(shouldCheckDegeneration(DEGENERATION_CHECK_STEP - 1, 0)).toBe(false);
expect(shouldCheckDegeneration(5000, 3000)).toBe(true); // grew 2000 since mark
expect(shouldCheckDegeneration(4000, 3000)).toBe(false); // grew only 1000
});
it('BUG (no reset): a stale large watermark silences the next step', () => {
// End of a long step: the watermark sits at 5000. The step ends and the
// accumulator resets to '' — but if the watermark is NOT reset, a fresh short
// degenerate burst (length 2000) never triggers a check: 2000 - 5000 < STEP.
const staleWatermark = 5000;
const nextStepLen = DEGENERATION_CHECK_STEP; // a fresh 2KB burst
expect(shouldCheckDegeneration(nextStepLen, staleWatermark)).toBe(false);
});
it('FIX (reset to 0): the same short degenerate burst IS checked and detected', () => {
// onStepFinish now zeroes the watermark, so the fresh burst re-arms the check.
const resetWatermark = 0;
const degenerateBurst = 'loadTools.\n'.repeat(300); // real degeneration
expect(degenerateBurst.length).toBeGreaterThanOrEqual(DEGENERATION_CHECK_STEP);
// The throttle now fires...
expect(
shouldCheckDegeneration(degenerateBurst.length, resetWatermark),
).toBe(true);
// ...and the detector catches the loop that would otherwise stream unchecked.
expect(isDegenerateOutput(degenerateBurst)).toBe(true);
});
});
/**
* BEHAVIOR guard for the ACTUAL fix (#486, ai-chat.service.onStepFinish resets
* lastDegenerationCheckLen to 0). The pure tests above use a hard-coded
* resetWatermark, so a REVERT of the real `lastDegenerationCheckLen = 0` line
* would not redden any of them. This drives the REAL onChunk/onStepFinish
* closures from stream() end to end and asserts the run is aborted when a fresh
* degenerate burst arrives in the step AFTER a long clean step which only
* happens if the watermark was actually zeroed on the step boundary.
*/
describe('AiChatService: onStepFinish re-arms the degeneration watermark (#486)', () => {
const streamTextMock = streamText as unknown as jest.Mock;
function makeRes() {
return {
raw: {
writeHead: jest.fn(),
write: jest.fn(),
once: jest.fn(),
on: jest.fn(),
flushHeaders: jest.fn(),
writableEnded: false,
destroyed: false,
},
};
}
function makeService() {
const aiChatRepo = {
findById: jest.fn(async () => ({ id: 'chat-1', workspaceId: 'ws-1' })),
insert: jest.fn(),
};
const aiChatMessageRepo = {
insert: jest.fn(async () => ({ id: 'msg-1' })),
findAllByChat: jest.fn(async () => []),
update: jest.fn(async () => ({ id: 'msg-1' })),
};
const aiSettings = { resolve: jest.fn(async () => ({})) };
const tools = { forUser: jest.fn(async () => ({})) };
const mcpClients = {
toolsFor: jest.fn(async () => ({
tools: {},
clients: [],
outcomes: [],
instructions: [],
})),
};
return new AiChatService(
{} as never, // ai
aiChatRepo as never,
aiChatMessageRepo as never,
{} as never, // aiChatPageSnapshotRepo
aiSettings as never,
tools as never,
mcpClients as never,
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo
{} as never, // pageAccess
{
isAiChatDeferredToolsEnabled: () => false,
// Lockdown OFF -> the degeneration guard is the active anti-babble path.
isAiChatFinalStepLockdownEnabled: () => false,
} as never, // environment
);
}
beforeEach(() => {
streamTextMock.mockReset();
jest.spyOn(Logger.prototype, 'log').mockImplementation(() => undefined as never);
jest.spyOn(Logger.prototype, 'warn').mockImplementation(() => undefined as never);
});
afterEach(() => jest.restoreAllMocks());
it('aborts on a fresh degenerate burst in the NEXT step (reverting the reset line reddens this)', async () => {
let captured:
| {
onChunk?: (e: { chunk: { type: string; text: string } }) => void;
onStepFinish?: (step: unknown) => void;
abortSignal?: AbortSignal;
}
| undefined;
streamTextMock.mockImplementation((opts: never) => {
captured = opts;
return {
consumeStream: jest.fn(),
pipeUIMessageStreamToResponse: jest.fn(),
};
});
const svc = makeService();
await svc.stream({
user: { id: 'user-1' } as never,
workspace: { id: 'ws-1' } as never,
sessionId: 'sess-1',
body: {
chatId: 'chat-1',
messages: [
{ id: 'm1', role: 'user', parts: [{ type: 'text', text: 'hi' }] },
],
} as never,
res: makeRes() as never,
signal: new AbortController().signal,
model: {} as never,
role: null,
// No runHooks -> legacy path (socket signal), degeneration guard active.
});
expect(streamTextMock).toHaveBeenCalledTimes(1);
const onChunk = captured!.onChunk!;
const onStepFinish = captured!.onStepFinish!;
const abortSignal = captured!.abortSignal!;
expect(abortSignal.aborted).toBe(false);
// STEP 1: a LONG, non-degenerate first step. Distinct lines never trip the
// detector, but they advance the throttle watermark far past the burst size
// that follows (to ~5x the step). This is the stale watermark that, WITHOUT
// the reset, would silence step 2.
let counter = 0;
let accumulated = 0;
while (accumulated < DEGENERATION_CHECK_STEP * 5) {
const line = `unique clean line number ${counter++} with distinct words\n`;
accumulated += line.length;
onChunk({ chunk: { type: 'text-delta', text: line } });
}
expect(abortSignal.aborted).toBe(false); // clean step must not abort
// STEP BOUNDARY: the real onStepFinish resets inProgressText AND (the fix)
// zeroes lastDegenerationCheckLen.
onStepFinish({ text: 'a clean first step', toolCalls: [], toolResults: [] });
// STEP 2: a FRESH, short degenerate burst (~3.3KB). Its length is far below
// the step-1 stale watermark (~10KB), so WITHOUT the reset the throttle stays
// silent and this streams unchecked. WITH the reset (watermark 0) it re-arms,
// the detector fires, and the run aborts.
const burst = 'loadTools.\n'.repeat(300);
expect(burst.length).toBeGreaterThanOrEqual(DEGENERATION_CHECK_STEP);
expect(burst.length).toBeLessThan(DEGENERATION_CHECK_STEP * 5);
onChunk({ chunk: { type: 'text-delta', text: burst } });
// The decisive assertion: the composed abortSignal (unioned with the
// degeneration controller) is now aborted. Reverting `lastDegenerationCheckLen
// = 0` in onStepFinish makes this stay false.
expect(abortSignal.aborted).toBe(true);
});
});
@@ -131,6 +131,32 @@ export function isDegenerateOutput(text: string): boolean {
return hasRepeatedLineRun(text) || hasPeriodicTail(text); return hasRepeatedLineRun(text) || hasPeriodicTail(text);
} }
/**
* How many bytes the in-progress text must grow before the (amortized) tail
* heuristics are re-run. Shared with ai-chat.service so the throttle the stream
* applies is the SAME one the unit test drives.
*/
export const DEGENERATION_CHECK_STEP = 2000;
/**
* Throttle decision for the degeneration guard (#444/#486). Returns true when
* the accumulated text has grown at least DEGENERATION_CHECK_STEP bytes past the
* last-checked offset, so the pure rules only fire every ~2KB. Pure; the caller
* updates its watermark to `textLen` when this returns true.
*
* The watermark is an offset INTO the accumulator, so when the accumulator is
* reset to '' on a step boundary the caller MUST reset the watermark to 0 too
* (#486). Otherwise `textLen - lastCheckLen` goes negative after the reset and
* this returns false until a later step re-grows past the stale offset a whole
* degenerate step could stream unchecked.
*/
export function shouldCheckDegeneration(
textLen: number,
lastCheckLen: number,
): boolean {
return textLen - lastCheckLen >= DEGENERATION_CHECK_STEP;
}
/** /**
* Truncate a degenerated tail before persist so hundreds of KB of garbage never * Truncate a degenerated tail before persist so hundreds of KB of garbage never
* reach the DB / replay (#444). Keeps everything up to and including the FIRST * reach the DB / replay (#444). Keeps everything up to and including the FIRST
@@ -0,0 +1,241 @@
// Break the editor-ext import chain (share.service -> collaboration.util ->
// @docmost/editor-ext -> @tiptap/core) that is unresolvable in this jest env and
// pre-existingly breaks these specs. jsonToMarkdown is never reached in these
// tests (the tools fail before rendering markdown).
jest.mock('../../collaboration/collaboration.util', () => ({
jsonToMarkdown: () => '',
}));
import { Logger } from '@nestjs/common';
import { MockLanguageModelV3, simulateReadableStream } from 'ai/test';
import { PublicShareChatService } from './public-share-chat.service';
import { PublicShareChatToolsService } from './tools/public-share-chat-tools.service';
/**
* SECURITY integration guard for #394 (commit 5): a tool's or the provider's raw
* error text must NOT leak to an anonymous public-share reader.
*
* The render gate (ToolCallCard showErrors=false) hides the text in the DOM but
* NOT on the wire, so this test asserts on the RAW SSE BYTES the server writes
* exactly the channel the render gate masks. We drive the real
* PublicShareChatService.stream() with a real share toolset (its underlying
* services mocked to fail) and a mock model, then inspect every byte piped to the
* fake socket.
*/
// A minimal ServerResponse stand-in that records every written chunk.
class FakeSocket {
chunks: string[] = [];
statusCode = 200;
writableEnded = false;
destroyed = false;
headersSent = false;
writeHead(): this {
this.headersSent = true;
return this;
}
setHeader(): void {}
removeHeader(): void {}
getHeader(): undefined {
return undefined;
}
flushHeaders(): void {}
write(chunk: unknown): boolean {
this.chunks.push(
typeof chunk === 'string' ? chunk : Buffer.from(chunk as never).toString('utf8'),
);
return true;
}
end(chunk?: unknown): void {
if (chunk) this.write(chunk);
this.writableEnded = true;
}
on(): this {
return this;
}
once(): this {
return this;
}
get body(): string {
return this.chunks.join('');
}
}
/** Mock model that issues one getSharePage tool call, then finishes with text. */
function toolCallingModel(): MockLanguageModelV3 {
let call = 0;
return new MockLanguageModelV3({
doStream: async () => {
call++;
if (call === 1) {
return {
stream: simulateReadableStream({
chunks: [
{ type: 'stream-start' as const, warnings: [] },
{ type: 'tool-input-start' as const, id: 't1', toolName: 'getSharePage' },
{ type: 'tool-input-end' as const, id: 't1' },
{
type: 'tool-call' as const,
toolCallId: 't1',
toolName: 'getSharePage',
input: '{"pageId":"secret-page"}',
},
{
type: 'finish' as const,
finishReason: { unified: 'tool-calls' as const, raw: 'tool_calls' },
usage: {
inputTokens: { total: 1, noCache: undefined, cacheRead: undefined, cacheWrite: undefined },
outputTokens: { total: 1, text: 1, reasoning: undefined },
},
},
],
}),
};
}
return {
stream: simulateReadableStream({
chunks: [
{ type: 'stream-start' as const, warnings: [] },
{ type: 'text-start' as const, id: '1' },
{ type: 'text-delta' as const, id: '1', delta: 'Sorry.' },
{ type: 'text-end' as const, id: '1' },
{
type: 'finish' as const,
finishReason: { unified: 'stop' as const, raw: 'stop' },
usage: {
inputTokens: { total: 1, noCache: undefined, cacheRead: undefined, cacheWrite: undefined },
outputTokens: { total: 1, text: 1, reasoning: undefined },
},
},
],
}),
};
},
});
}
/** Mock model whose stream emits a provider error carrying an internal secret. */
function providerErrorModel(secret: string): MockLanguageModelV3 {
return new MockLanguageModelV3({
doStream: async () => ({
stream: simulateReadableStream({
chunks: [
{ type: 'stream-start' as const, warnings: [] },
{
type: 'error' as const,
error: {
statusCode: 503,
message: 'Service Unavailable',
responseBody: `upstream ${secret} model=internal-gpt`,
},
},
],
}),
}),
});
}
function makeService(toolsService: PublicShareChatToolsService): {
svc: PublicShareChatService;
logSpy: jest.SpyInstance;
} {
const svc = Object.create(PublicShareChatService.prototype);
const logger = new Logger('test');
const logSpy = jest.spyOn(logger, 'error').mockImplementation(() => undefined);
jest.spyOn(logger, 'warn').mockImplementation(() => undefined);
svc.tools = toolsService;
svc.logger = logger;
svc.tokenBudget = { record: jest.fn().mockResolvedValue(undefined) };
return { svc, logSpy };
}
async function runStream(
svc: PublicShareChatService,
model: MockLanguageModelV3,
): Promise<FakeSocket> {
const socket = new FakeSocket();
await svc.stream({
workspaceId: 'ws1',
shareId: 'share1',
share: { id: 'share1', pageId: 'p1', sharedPage: { id: 'p1', title: 'Docs' } },
openedPage: null,
messages: [
{ id: 'm1', role: 'user', parts: [{ type: 'text', text: 'read the page' }] } as never,
],
res: { raw: socket } as never,
signal: new AbortController().signal,
model: model as never,
role: null,
});
// Let the piped stream drain fully.
await new Promise((r) => setTimeout(r, 300));
return socket;
}
describe('public share chat error leak (#394)', () => {
afterEach(() => jest.restoreAllMocks());
it('does NOT leak a tool\'s raw internal error to the SSE bytes (generic classified string instead)', async () => {
const SECRET = 'INTERNAL_baseUrl_http://provider.internal:8080/v1';
const shareService = {
// The canonical boundary throws a RAW internal error (with a secret).
resolveReadableSharePage: jest
.fn()
.mockRejectedValue(new Error(`db failed at ${SECRET} stack@line42`)),
};
const tools = new PublicShareChatToolsService(
shareService as never,
{} as never,
{} as never,
);
const { svc } = makeService(tools);
const socket = await runStream(svc, toolCallingModel());
// The tool-output-error frame is present on the wire...
expect(socket.body).toContain('tool-output-error');
// ...but it carries ONLY the generic classified string — never the secret,
// the raw driver message, or a stack fragment.
expect(socket.body).toContain('The tool could not complete the request.');
expect(socket.body).not.toContain(SECRET);
expect(socket.body).not.toContain('stack@line42');
expect(socket.body).not.toContain('db failed');
});
it('passes a SAFE ShareToolError message (page not available) through to the bytes', async () => {
const shareService = {
// Not found in this share -> the tool throws the classified SAFE message.
resolveReadableSharePage: jest.fn().mockResolvedValue(null),
};
const tools = new PublicShareChatToolsService(
shareService as never,
{} as never,
{} as never,
);
const { svc } = makeService(tools);
const socket = await runStream(svc, toolCallingModel());
expect(socket.body).toContain('tool-output-error');
expect(socket.body).toContain('not available in this share');
});
it('does NOT leak a provider error (statusCode + response body) to the SSE bytes', async () => {
const SECRET = 'http://provider.internal:8080';
const tools = new PublicShareChatToolsService(
{} as never,
{} as never,
{} as never,
);
const { svc, logSpy } = makeService(tools);
const socket = await runStream(svc, providerErrorModel(SECRET));
// The anon sees a fixed classified string, not the provider body/baseUrl/model.
expect(socket.body).toContain('temporarily unavailable');
expect(socket.body).not.toContain(SECRET);
expect(socket.body).not.toContain('internal-gpt');
// The FULL provider detail is logged server-side only.
const logged = logSpy.mock.calls.map((c) => String(c[0])).join('\n');
expect(logged).toContain(SECRET);
});
});
@@ -12,7 +12,10 @@ import { AiAgentRoleRepo } from '@docmost/db/repos/ai-agent-roles/ai-agent-roles
import { AiAgentRole } from '@docmost/db/types/entity.types'; import { AiAgentRole } from '@docmost/db/types/entity.types';
import { AiService } from '../../integrations/ai/ai.service'; import { AiService } from '../../integrations/ai/ai.service';
import { AiSettingsService } from '../../integrations/ai/ai-settings.service'; import { AiSettingsService } from '../../integrations/ai/ai-settings.service';
import { PublicShareChatToolsService } from './tools/public-share-chat-tools.service'; import {
PublicShareChatToolsService,
ShareToolError,
} from './tools/public-share-chat-tools.service';
import { buildShareSystemPrompt } from './public-share-chat.prompt'; import { buildShareSystemPrompt } from './public-share-chat.prompt';
import { roleModelOverride } from './roles/role-model-config'; import { roleModelOverride } from './roles/role-model-config';
import { import {
@@ -102,6 +105,30 @@ export function filterShareTranscript(messages: UIMessage[]): UIMessage[] {
); );
} }
/**
* Fixed, classified strings an ANONYMOUS share reader may see when the assistant
* stream fails (#394). These reveal NOTHING about the internal provider, its
* baseUrl, the model name, or the raw response body unlike describeProviderError
* (which is for the server log / the authenticated operator only). We classify by
* HTTP status where available so the reader still gets a useful hint (retry vs.
* give up) without any internal detail.
*/
export function classifyAnonStreamError(error: unknown): string {
const status =
typeof error === 'object' && error !== null
? (error as { statusCode?: number }).statusCode
: undefined;
if (status === 429) {
return 'The assistant is receiving too many requests right now. Please try again shortly.';
}
if (typeof status === 'number' && status >= 500) {
return 'The assistant is temporarily unavailable. Please try again.';
}
// Any other failure (including a bare connection error with no status): a
// single neutral line. No provider identity, no config, no response body.
return 'The assistant could not complete your request. Please try again.';
}
/** /**
* Anonymous, read-only AI assistant for a single PUBLIC share tree. * Anonymous, read-only AI assistant for a single PUBLIC share tree.
* *
@@ -318,11 +345,28 @@ export class PublicShareChatService {
result.pipeUIMessageStreamToResponse(res.raw, { result.pipeUIMessageStreamToResponse(res.raw, {
headers: { 'X-Accel-Buffering': 'no' }, headers: { 'X-Accel-Buffering': 'no' },
onError: (error: unknown) => { onError: (error: unknown) => {
// Reuse the shared formatter so provider error formatting stays // SECURITY (#394): the string this returns is written verbatim into the
// unified between the log line and the streamed error message — a // SSE error frame delivered to an ANONYMOUS reader (for a tool failure
// share reader sees 402/429/503 causes consistently with the // it becomes the atomic `tool-output-error` frame's errorText; for a
// authenticated path. // stream/provider failure, the terminal error frame).
return describeProviderError(error, 'AI stream error'); //
// A ShareToolError is already a classified, safe tool message (see
// PublicShareChatToolsService.wrapToolErrors) — pass it through so the
// reader still gets the useful "page not available in this share" hint.
if (error instanceof ShareToolError) {
return error.message;
}
// Anything else is a provider/stream error. describeProviderError
// bundles the provider statusCode AND response body, which can carry the
// internal baseUrl or model name — NEVER expose that to the public. Log
// the full detail server-side only and return a fixed classified string.
this.logger.error(
`Public share chat pipe error: ${describeProviderError(
error,
'AI stream error',
)}`,
);
return classifyAnonStreamError(error);
}, },
}); });
@@ -808,7 +808,7 @@ describe('PublicShareChatToolsService share scoping', () => {
}; };
await expect(getSharePage.execute({ pageId: 'p-outside' })).rejects.toThrow( await expect(getSharePage.execute({ pageId: 'p-outside' })).rejects.toThrow(
/not part of this published share/i, /not available in this share/i,
); );
// The tool delegated the resolve to the canonical boundary with the // The tool delegated the resolve to the canonical boundary with the
// forShare-scoped shareId, and returned NO content for a non-resolving page. // forShare-scoped shareId, and returned NO content for a non-resolving page.
@@ -841,7 +841,7 @@ describe('PublicShareChatToolsService share scoping', () => {
await expect( await expect(
getSharePage.execute({ pageId: 'p-restricted' }), getSharePage.execute({ pageId: 'p-restricted' }),
).rejects.toThrow(/not part of this published share/i); ).rejects.toThrow(/not available in this share/i);
// No content was ever sanitized/returned for the blocked page. // No content was ever sanitized/returned for the blocked page.
expect(shareService.updatePublicAttachments).not.toHaveBeenCalled(); expect(shareService.updatePublicAttachments).not.toHaveBeenCalled();
}); });
@@ -1003,7 +1003,7 @@ describe('public-share assistant boundary locks (red-team regression guards)', (
}; };
await expect( await expect(
getSharePage.execute({ pageId: 'p-elsewhere' }), getSharePage.execute({ pageId: 'p-elsewhere' }),
).rejects.toThrow(/not part of this published share/i); ).rejects.toThrow(/not available in this share/i);
// The forged share id is the scope the boundary re-derivation rejects against. // The forged share id is the scope the boundary re-derivation rejects against.
expect(shareService.resolveReadableSharePage).toHaveBeenCalledWith( expect(shareService.resolveReadableSharePage).toHaveBeenCalledWith(
'FORGED-SHARE', 'FORGED-SHARE',
@@ -1,7 +1,15 @@
import { createHash } from 'node:crypto'; import { createHash } from 'node:crypto';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs'; import {
mkdtempSync,
mkdirSync,
writeFileSync,
rmSync,
readdirSync,
statSync,
readFileSync,
} from 'node:fs';
import { tmpdir } from 'node:os'; import { tmpdir } from 'node:os';
import { join } from 'node:path'; import { dirname, join, relative, sep } from 'node:path';
import { computeSrcRegistryStamp } from './docmost-client.loader'; import { computeSrcRegistryStamp } from './docmost-client.loader';
@@ -30,10 +38,14 @@ function assertStaleGuard(
} }
} }
// Build a throwaway `<pkg>/build/index.js` + optional `<pkg>/src/tool-specs.ts` // Build a throwaway `<pkg>/build/index.js` + optional `<pkg>/src/` tree so
// layout so `computeSrcRegistryStamp(<pkg>/build/index.js)` resolves src the same // `computeSrcRegistryStamp(<pkg>/build/index.js)` resolves src the same way the
// way the loader does (dirname(dirname(entry))/src/tool-specs.ts). // loader does (dirname(dirname(entry))/src). Since #486 the stamp hashes the WHOLE
function makeFakePackage(toolSpecsSource: string | null): { // src tree, so a fixture is a { relPath: content } map. A bare string is sugar for
// a single `tool-specs.ts`; `null` means "no src tree" (the prod no-op path).
function makeFakePackage(
src: string | Record<string, string> | null,
): {
entry: string; entry: string;
cleanup: () => void; cleanup: () => void;
} { } {
@@ -42,10 +54,15 @@ function makeFakePackage(toolSpecsSource: string | null): {
mkdirSync(buildDir, { recursive: true }); mkdirSync(buildDir, { recursive: true });
const entry = join(buildDir, 'index.js'); const entry = join(buildDir, 'index.js');
writeFileSync(entry, '// fake @docmost/mcp build entry\n', 'utf8'); writeFileSync(entry, '// fake @docmost/mcp build entry\n', 'utf8');
if (toolSpecsSource !== null) { if (src !== null) {
const files =
typeof src === 'string' ? { 'tool-specs.ts': src } : src;
const srcDir = join(root, 'src'); const srcDir = join(root, 'src');
mkdirSync(srcDir, { recursive: true }); for (const [rel, content] of Object.entries(files)) {
writeFileSync(join(srcDir, 'tool-specs.ts'), toolSpecsSource, 'utf8'); const full = join(srcDir, rel);
mkdirSync(dirname(full), { recursive: true });
writeFileSync(full, content, 'utf8');
}
} }
return { entry, cleanup: () => rmSync(root, { recursive: true, force: true }) }; return { entry, cleanup: () => rmSync(root, { recursive: true, force: true }) };
} }
@@ -93,34 +110,109 @@ describe('computeSrcRegistryStamp (#447 stale-build guard)', () => {
} }
}); });
// CROSS-IMPL EQUALITY (covers reviewer suggestion 2). The SAME fixed input and // #486 CORE (negative): an edit to a NON-tool-specs src file (client.ts) with a
// rebuild NOT run must move the src stamp away from the built REGISTRY_STAMP, so
// the loader's stale-check refuses. Under the old tool-specs.ts-only hash this
// edit was invisible and a stale build/ served the old client.ts silently.
it('a client.ts edit (no rebuild) moves the src stamp -> loader refuses (#486)', () => {
// "Built" state: the package as it was compiled.
const built = makeFakePackage({
'tool-specs.ts': 'export const SPECS = 1;\n',
'client.ts': "export const impl = 'v1';\n",
});
// "Dev edited src, forgot to rebuild": client.ts changed, tool-specs.ts not.
const edited = makeFakePackage({
'tool-specs.ts': 'export const SPECS = 1;\n',
'client.ts': "export const impl = 'v2';\n",
});
try {
const builtStamp = computeSrcRegistryStamp(built.entry);
const editedStamp = computeSrcRegistryStamp(edited.entry);
expect(builtStamp).not.toBeNull();
expect(editedStamp).not.toBe(builtStamp);
// build/ still carries builtStamp; src now hashes to editedStamp -> refuse.
expect(() => assertStaleGuard(editedStamp, builtStamp as string)).toThrow(
STALE_BUILD_MESSAGE,
);
} finally {
built.cleanup();
edited.cleanup();
}
});
// *.generated.ts is excluded (the codegen's own output — a fixed-point cycle
// otherwise): its presence/content must not move the stamp.
it('excludes *.generated.ts from the stamp', () => {
const without = makeFakePackage({ 'tool-specs.ts': 'x\n' });
const withGen = makeFakePackage({
'tool-specs.ts': 'x\n',
'registry-stamp.generated.ts': 'export const REGISTRY_STAMP = "abc";\n',
});
try {
expect(computeSrcRegistryStamp(withGen.entry)).toBe(
computeSrcRegistryStamp(without.entry),
);
} finally {
without.cleanup();
withGen.cleanup();
}
});
// CROSS-IMPL EQUALITY (covers reviewer suggestion 2). The SAME fixed tree and
// EXPECTED hash are asserted in the mcp-side node test // EXPECTED hash are asserted in the mcp-side node test
// (packages/mcp/test/unit/registry-stamp.test.mjs) against the codegen's // (packages/mcp/test/unit/registry-stamp.test.mjs) against the codegen's
// `computeRegistryStamp`. Asserting the SAME pair here against the loader's // `computeRegistryStamp`. Asserting the SAME pair here against the loader's
// `computeSrcRegistryStamp` proves both implementations normalize+hash // `computeSrcRegistryStamp` proves both implementations enumerate+normalize+hash
// identically; a divergence in EITHER side reddens one of the two tests. // identically; a divergence in EITHER side reddens one of the two tests.
it('matches the documented cross-impl hash for a fixed input', () => { const CROSS_IMPL_TREE = {
const FIXED_INPUT = 'line1\r\nline2\n'; 'tool-specs.ts': 'line1\r\nline2\n',
const EXPECTED = 'client/read.ts': 'export const R = 1;\n',
'683376e290829b482c2655745caffa7a1dccfa10afaa62dac2b42dd6c68d0f83'; 'registry-stamp.generated.ts': 'export const REGISTRY_STAMP="ignored";\n',
const { entry, cleanup } = makeFakePackage(FIXED_INPUT); };
const CROSS_IMPL_EXPECTED =
'131c1b9e4e2f5a7d6cef91ca8df619822b442f52bc45ebd09474a4c1d6728616';
it('matches the documented cross-impl hash for a fixed tree', () => {
const { entry, cleanup } = makeFakePackage(CROSS_IMPL_TREE);
try { try {
expect(computeSrcRegistryStamp(entry)).toBe(EXPECTED); expect(computeSrcRegistryStamp(entry)).toBe(CROSS_IMPL_EXPECTED);
} finally { } finally {
cleanup(); cleanup();
} }
}); });
it('the documented EXPECTED is the normalize+sha256 of the fixed input', () => { it('the documented EXPECTED is the enumerate+normalize+sha256 of the tree', () => {
// Proves EXPECTED is not a magic constant but the documented computation. // Proves EXPECTED is not a magic constant but the documented computation — a
const FIXED_INPUT = 'line1\r\nline2\n'; // local re-implementation of the loader's tree walk.
const normalized = FIXED_INPUT.replace(/\r\n/g, '\n').replace(/\n$/, ''); const { entry, cleanup } = makeFakePackage(CROSS_IMPL_TREE);
const expected = createHash('sha256')
.update(normalized, 'utf8')
.digest('hex');
const { entry, cleanup } = makeFakePackage(FIXED_INPUT);
try { try {
expect(computeSrcRegistryStamp(entry)).toBe(expected); const srcDir = join(dirname(dirname(entry)), 'src');
const collect = (dir: string): string[] => {
const out: string[] = [];
for (const e of readdirSync(dir)) {
const f = join(dir, e);
if (statSync(f).isDirectory()) out.push(...collect(f));
else if (e.endsWith('.ts') && !e.endsWith('.generated.ts'))
out.push(f);
}
return out;
};
const files = collect(srcDir)
.map((abs) => ({ rel: relative(srcDir, abs).split(sep).join('/'), abs }))
.sort((a, b) => (a.rel < b.rel ? -1 : a.rel > b.rel ? 1 : 0));
const h = createHash('sha256');
for (const { rel, abs } of files) {
const n = readFileSync(abs, 'utf8')
.replace(/\r\n/g, '\n')
.replace(/\n$/, '');
h.update(rel, 'utf8');
h.update('\0', 'utf8');
h.update(n, 'utf8');
h.update('\0', 'utf8');
}
const localHash = h.digest('hex');
expect(computeSrcRegistryStamp(entry)).toBe(localHash);
expect(localHash).toBe(CROSS_IMPL_EXPECTED);
} finally { } finally {
cleanup(); cleanup();
} }
@@ -1,6 +1,6 @@
import { createHash } from 'node:crypto'; import { createHash } from 'node:crypto';
import { existsSync, readFileSync } from 'node:fs'; import { existsSync, readdirSync, readFileSync, statSync } from 'node:fs';
import { dirname, join } from 'node:path'; import { dirname, join, relative, sep } from 'node:path';
import { pathToFileURL } from 'node:url'; import { pathToFileURL } from 'node:url';
import type { DocmostClient, SharedToolSpec } from '@docmost/mcp'; import type { DocmostClient, SharedToolSpec } from '@docmost/mcp';
@@ -191,33 +191,52 @@ interface DocmostMcpModule {
* present. Returns the stamp string, or `null` when the source is absent (a prod * present. Returns the stamp string, or `null` when the source is absent (a prod
* image ships only build/, no src/). MUST stay byte-for-byte identical to * image ships only build/, no src/). MUST stay byte-for-byte identical to
* packages/mcp/scripts/gen-registry-stamp.mjs's `computeRegistryStamp` so the * packages/mcp/scripts/gen-registry-stamp.mjs's `computeRegistryStamp` so the
* build-time and src-time hashes agree: same input file (src/tool-specs.ts), same * build-time and src-time hashes agree: same file set (every src/**\/*.ts except
* normalization (CRLF -> LF, strip a single trailing newline), same sha256. * *.generated.ts), same POSIX-relative sort, same per-file normalization (CRLF ->
* LF, strip a single trailing newline) with the same path+content framing, same
* sha256. Hashing the WHOLE src tree (not just tool-specs.ts) is #486: an edit to
* client.ts / a client/* module / comment-signal / drawio-* without a rebuild
* must also be caught, otherwise build/ silently serves the old code.
* *
* DEV vs PROD detection is by FILE EXISTENCE, not NODE_ENV: we resolve the * DEV vs PROD detection is by FILE EXISTENCE, not NODE_ENV: we resolve the
* package's own directory from `require.resolve('@docmost/mcp')` (which points at * package's own directory from `require.resolve('@docmost/mcp')` (which points at
* build/index.js) and look for ../src/tool-specs.ts next to it. In a dev/test * build/index.js) and look for ../src next to it. In a dev/test worktree that
* worktree that file exists; in a prod image (build/ only, src/ stripped) it does * directory exists; in a prod image (build/ only, src/ stripped) it does not, so
* not, so this returns null and the caller skips the check. Any error (ENOENT, a * this returns null and the caller skips the check. Any error (ENOENT, a bad
* bad resolve) is swallowed to null the stale-check must NEVER break startup. * resolve) is swallowed to null the stale-check must NEVER break startup.
* *
* Exported for unit testing (docmost-client.loader.spec.ts): the export keyword * Exported for unit testing (docmost-client.loader.spec.ts): the export keyword
* is behaviourally a no-op the module-internal caller `loadDocmostMcp` is * is behaviourally a no-op the module-internal caller `loadDocmostMcp` is
* unaffected. The test drives the null (no-src) path and asserts this * unaffected. The test drives the null (no-src) path and asserts this
* normalize+sha256 stays identical to the codegen's `computeRegistryStamp`. * enumerate+normalize+sha256 stays identical to the codegen's
* `computeRegistryStamp`.
*/ */
export function computeSrcRegistryStamp(packageEntry: string): string | null { export function computeSrcRegistryStamp(packageEntry: string): string | null {
try { try {
// packageEntry is <pkg>/build/index.js; the source lives at <pkg>/src/. // packageEntry is <pkg>/build/index.js; the source lives at <pkg>/src/.
const toolSpecsPath = join( const srcDir = join(dirname(dirname(packageEntry)), 'src');
dirname(dirname(packageEntry)), if (!existsSync(srcDir)) return null; // prod: no src tree -> skip.
'src', // Enumerate every src/**\/*.ts except the codegen's own *.generated.ts
'tool-specs.ts', // output (including it would be a fixed-point cycle). Sort by POSIX-relative
); // path so ordering is platform-independent, then fold each file's relative
if (!existsSync(toolSpecsPath)) return null; // prod: no src tree -> skip. // path + normalized content into one hash — identical to the codegen.
const source = readFileSync(toolSpecsPath, 'utf8'); const files = collectStampFiles(srcDir)
const normalized = source.replace(/\r\n/g, '\n').replace(/\n$/, ''); .map((abs) => ({
return createHash('sha256').update(normalized, 'utf8').digest('hex'); rel: relative(srcDir, abs).split(sep).join('/'),
abs,
}))
.sort((a, b) => (a.rel < b.rel ? -1 : a.rel > b.rel ? 1 : 0));
const hash = createHash('sha256');
for (const { rel, abs } of files) {
const normalized = readFileSync(abs, 'utf8')
.replace(/\r\n/g, '\n')
.replace(/\n$/, '');
hash.update(rel, 'utf8');
hash.update('\0', 'utf8');
hash.update(normalized, 'utf8');
hash.update('\0', 'utf8');
}
return hash.digest('hex');
} catch { } catch {
// Never let a resolution/read hiccup break server startup — treat as "no // Never let a resolution/read hiccup break server startup — treat as "no
// src available" and skip the check (identical to the prod no-op path). // src available" and skip the check (identical to the prod no-op path).
@@ -225,6 +244,24 @@ export function computeSrcRegistryStamp(packageEntry: string): string | null {
} }
} }
/**
* Recursively enumerate every `*.ts` under `dir`, EXCLUDING `*.generated.ts`.
* Mirror of the codegen's `collectStampFiles` (packages/mcp/scripts/
* gen-registry-stamp.mjs) keep the two walk/filter rules identical.
*/
function collectStampFiles(dir: string): string[] {
const out: string[] = [];
for (const entry of readdirSync(dir)) {
const full = join(dir, entry);
if (statSync(full).isDirectory()) {
out.push(...collectStampFiles(full));
} else if (entry.endsWith('.ts') && !entry.endsWith('.generated.ts')) {
out.push(full);
}
}
return out;
}
// TS with module:commonjs downlevels a literal `import()` to `require()`, which // TS with module:commonjs downlevels a literal `import()` to `require()`, which
// cannot load the ESM-only `@docmost/mcp` package. Indirect through Function so // cannot load the ESM-only `@docmost/mcp` package. Indirect through Function so
// the real dynamic `import()` survives compilation and can load ESM from // the real dynamic `import()` survives compilation and can load ESM from
@@ -224,7 +224,7 @@ describe('PublicShareChatToolsService.forShare', () => {
(tools.getSharePage as unknown as ToolExec).execute({ (tools.getSharePage as unknown as ToolExec).execute({
pageId: 'page-1', pageId: 'page-1',
}), }),
).rejects.toThrow('That page is not part of this published share.'); ).rejects.toThrow('The requested page is not available in this share.');
// No content is ever fetched/returned for a non-resolving page. // No content is ever fetched/returned for a non-resolving page.
expect(shareService.updatePublicAttachments).not.toHaveBeenCalled(); expect(shareService.updatePublicAttachments).not.toHaveBeenCalled();
@@ -7,6 +7,22 @@ import { PageRepo } from '@docmost/db/repos/page/page.repo';
import { jsonToMarkdown } from '../../../collaboration/collaboration.util'; import { jsonToMarkdown } from '../../../collaboration/collaboration.util';
import { modelFriendlyInput } from './model-friendly-input'; import { modelFriendlyInput } from './model-friendly-input';
/**
* A tool error whose message is DELIBERATELY safe to expose to an anonymous
* share reader (and to the model, for self-correction). Every OTHER thrown error
* is treated as internal and replaced with a generic string by `wrapToolErrors`,
* so a raw exception message an internal page title, a DB/stack fragment, a
* driver detail never rides the public UI stream (#394).
*/
export class ShareToolError extends Error {}
// The only two classified strings an anonymous reader may ever see from a tool
// failure. The specific one keeps the model's self-correction useful ("try a
// different page"); the generic one reveals nothing about the internal fault.
const SHARE_TOOL_ERROR_NOT_AVAILABLE =
'The requested page is not available in this share.';
const SHARE_TOOL_ERROR_GENERIC = 'The tool could not complete the request.';
/** /**
* Isolated, READ-ONLY toolset for the ANONYMOUS public-share assistant. * Isolated, READ-ONLY toolset for the ANONYMOUS public-share assistant.
* *
@@ -44,7 +60,7 @@ export class PublicShareChatToolsService {
* are NO write tools, NO comments/history, NO cross-space or external tools. * are NO write tools, NO comments/history, NO cross-space or external tools.
*/ */
forShare(shareId: string, workspaceId: string): Record<string, Tool> { forShare(shareId: string, workspaceId: string): Record<string, Tool> {
return { return this.wrapToolErrors({
searchSharePages: tool({ searchSharePages: tool({
description: description:
'Search the pages of THIS published documentation share for a ' + 'Search the pages of THIS published documentation share for a ' +
@@ -96,7 +112,7 @@ export class PublicShareChatToolsService {
execute: async ({ pageId }) => { execute: async ({ pageId }) => {
const id = (pageId ?? '').trim(); const id = (pageId ?? '').trim();
if (!id) { if (!id) {
throw new Error('A pageId is required.'); throw new ShareToolError('A pageId is required.');
} }
// Resolve via the SINGLE canonical share-access boundary: confirms the // Resolve via the SINGLE canonical share-access boundary: confirms the
// page resolves to THIS share (recursive CTE up the tree, honouring // page resolves to THIS share (recursive CTE up the tree, honouring
@@ -112,7 +128,7 @@ export class PublicShareChatToolsService {
workspaceId, workspaceId,
); );
if (!resolved) { if (!resolved) {
throw new Error('That page is not part of this published share.'); throw new ShareToolError(SHARE_TOOL_ERROR_NOT_AVAILABLE);
} }
const { page } = resolved; const { page } = resolved;
@@ -193,6 +209,57 @@ export class PublicShareChatToolsService {
} }
}, },
}), }),
}; });
}
/**
* Wrap every tool's `execute` so a THROWN error is sanitized in ONE place
* closing the byte leak, the render, and the model context at once (#394).
*
* The AI SDK surfaces a tool-execution throw as an atomic `tool-output-error`
* frame on the v6 UI stream whose `errorText` is the thrown message; on the
* public share that frame goes straight to an anonymous reader. Unwrapped, a
* raw exception (an internal page title, a DB/stack fragment, a driver detail)
* would ride that frame verbatim. Here we catch it, LOG the full detail
* server-side only, and re-throw a CLASSIFIED, safe error: the tool's own
* intentional ShareToolError messages pass through (they keep the model's
* self-correction useful), everything else collapses to a generic string.
*/
private wrapToolErrors(
tools: Record<string, Tool>,
): Record<string, Tool> {
const wrapped: Record<string, Tool> = {};
for (const [name, t] of Object.entries(tools)) {
const original = t.execute;
if (typeof original !== 'function') {
wrapped[name] = t;
continue;
}
wrapped[name] = {
...t,
execute: async (args: unknown, options: unknown) => {
try {
return await (
original as (a: unknown, o: unknown) => Promise<unknown>
)(args, options);
} catch (err) {
const safe =
err instanceof ShareToolError
? err.message
: SHARE_TOOL_ERROR_GENERIC;
// Full detail to the server log ONLY — never to the anon.
this.logger.warn(
`Public share tool "${name}" failed: ${
err instanceof Error ? err.message : String(err)
}`,
);
// This safe string is ALL that rides the tool-output-error frame,
// becomes model context, and could be rendered — one choke point.
throw new ShareToolError(safe);
}
},
} as Tool;
}
return wrapped;
} }
} }
@@ -120,3 +120,102 @@ describe('JwtStrategy — provenance derivation', () => {
expect(req.raw.actor).toBeUndefined(); expect(req.raw.actor).toBeUndefined();
}); });
}); });
/**
* Provenance derivation on the API-KEY path (jwt.strategy.validateApiKey, #486).
*
* The access-token path stamped provenance; the API-key path returned early
* WITHOUT it, so an is_agent API key's REST writes recorded no 'agent' marker.
* The API-key payload carries no signed claim, so provenance is resolved from the
* SERVER-SIDE user returned by ApiKeyService.validateApiKey: isAgent -> 'agent',
* otherwise 'user'; aiChatId is always null (an API key has no ai_chats row).
*
* The enterprise ApiKeyService is not bundled in the OSS build, so the strategy
* loads it through an overridable `resolveApiKeyService` seam that we stub here.
*/
describe('JwtStrategy — API-key provenance derivation (#486)', () => {
function makeApiKeyStrategy(validateApiKeyImpl: (p: any) => Promise<any>) {
const userRepo: any = { findById: jest.fn() };
const workspaceRepo: any = { findById: jest.fn() };
const userSessionRepo: any = { findActiveById: jest.fn() };
const sessionActivityService: any = { trackActivity: jest.fn() };
const environmentService: any = { getAppSecret: () => 'test-secret' };
const moduleRef: any = {};
const strategy = new JwtStrategy(
userRepo,
workspaceRepo,
userSessionRepo,
sessionActivityService,
environmentService,
moduleRef,
);
// Stub the EE ApiKeyService seam (the real module is not in the OSS build).
const validateApiKey = jest.fn(validateApiKeyImpl);
jest
.spyOn(strategy as any, 'resolveApiKeyService')
.mockReturnValue({ validateApiKey });
return { strategy, validateApiKey };
}
const makeReq = () => ({ raw: {} as Record<string, any> });
const apiKeyPayload = () => ({
sub: 'svc-1',
workspaceId: 'ws-1',
apiKeyId: 'key-1',
type: JwtType.API_KEY,
});
it("stamps actor='agent' for an is_agent API key (from the validated user)", async () => {
const validated = {
user: { id: 'svc-1', isAgent: true },
workspace: { id: 'ws-1' },
};
const { strategy, validateApiKey } = makeApiKeyStrategy(
async () => validated,
);
const req = makeReq();
const result = await strategy.validate(req, apiKeyPayload() as any);
expect(validateApiKey).toHaveBeenCalledTimes(1);
expect(req.raw.actor).toBe('agent');
// API keys carry no internal ai_chats row -> null.
expect(req.raw.aiChatId).toBeNull();
// The validated auth object is returned unchanged (req.user shape preserved).
expect(result).toBe(validated);
});
it("stamps actor='user' for an ordinary (non-agent) API key", async () => {
const { strategy } = makeApiKeyStrategy(async () => ({
user: { id: 'u-1', isAgent: false },
workspace: { id: 'ws-1' },
}));
const req = makeReq();
await strategy.validate(req, apiKeyPayload() as any);
expect(req.raw.actor).toBe('user');
expect(req.raw.aiChatId).toBeNull();
});
it('throws Unauthorized (and stamps nothing) when the EE module is missing', async () => {
const userRepo: any = { findById: jest.fn() };
const strategy = new JwtStrategy(
userRepo,
{ findById: jest.fn() } as any,
{ findActiveById: jest.fn() } as any,
{ trackActivity: jest.fn() } as any,
{ getAppSecret: () => 'test-secret' } as any,
{} as any,
);
// EE not bundled: the seam returns null.
jest.spyOn(strategy as any, 'resolveApiKeyService').mockReturnValue(null);
const req = makeReq();
await expect(
strategy.validate(req, apiKeyPayload() as any),
).rejects.toThrow(UnauthorizedException);
expect(req.raw.actor).toBeUndefined();
});
});
@@ -102,28 +102,49 @@ export class JwtStrategy extends PassportStrategy(Strategy, 'jwt') {
} }
private async validateApiKey(req: any, payload: JwtApiKeyPayload) { private async validateApiKey(req: any, payload: JwtApiKeyPayload) {
let ApiKeyModule: any; const apiKeyService = this.resolveApiKeyService();
let isApiKeyModuleReady = false; if (!apiKeyService) {
throw new UnauthorizedException('Enterprise API Key module missing');
}
const result = await apiKeyService.validateApiKey(payload);
// Stamp the agent-edit provenance for the API-KEY path too (#486). Unlike the
// access-token path above, it CANNOT be resolved before this point: the
// API-key payload carries no signed actor/aiChatId claim, and the user (with
// its isAgent flag) is unknown until the key is validated. Claim semantics for
// API keys: an is_agent API key (an agent service account) stamps 'agent' on
// every REST write; an ordinary API key resolves to 'user'. An API key has no
// internal ai_chats row, so aiChatId is always null. Derived from the
// SERVER-SIDE user (never a client field), so an 'agent' badge is unspoofable
// — mirroring the access-token path. Passing `null` for the claim means the
// actor is decided solely by user.isAgent.
const provenance = resolveProvenance((result as any)?.user, null);
req.raw.actor = provenance.actor;
req.raw.aiChatId = provenance.aiChatId;
return result;
}
/**
* Resolve the enterprise ApiKeyService, or `null` when the EE module is not
* bundled in this build (community build). Extracted as an overridable seam so
* the API-key provenance stamping can be unit-tested without the EE package
* present (docmost is OSS + a separate EE bundle; `require` of the EE path
* throws here). Any load/resolve error is treated as "module missing".
*/
protected resolveApiKeyService(): {
validateApiKey: (payload: JwtApiKeyPayload) => Promise<unknown>;
} | null {
try { try {
// eslint-disable-next-line @typescript-eslint/no-require-imports // eslint-disable-next-line @typescript-eslint/no-require-imports
ApiKeyModule = require('./../../../ee/api-key/api-key.service'); const ApiKeyModule = require('./../../../ee/api-key/api-key.service');
isApiKeyModuleReady = true; return this.moduleRef.get(ApiKeyModule.ApiKeyService, { strict: false });
} catch (err) { } catch (err) {
this.logger.debug( this.logger.debug(
'API Key module requested but enterprise module not bundled in this build', 'API Key module requested but enterprise module not bundled in this build',
); );
isApiKeyModuleReady = false; return null;
} }
if (isApiKeyModuleReady) {
const ApiKeyService = this.moduleRef.get(ApiKeyModule.ApiKeyService, {
strict: false,
});
return ApiKeyService.validateApiKey(payload);
}
throw new UnauthorizedException('Enterprise API Key module missing');
} }
} }
@@ -0,0 +1,133 @@
import { readFileSync } from 'fs';
import { EventEmitter } from 'node:events';
import { streamText } from 'ai';
import { MockLanguageModelV3, simulateReadableStream } from 'ai/test';
/**
* Regression tests for the writeToServerResponse drain-hang fix in
* patches/ai@6.0.134.patch (#486, commit 6).
*
* Unpatched ai@6.0.134's writeToServerResponse awaits ONLY `once("drain")` when
* response.write() returns false (backpressure). If the client disconnects
* mid-write the socket never drains, so that await never resolves: the read loop
* parks FOREVER, its `finally { response.end() }` is unreachable, and the stream
* reader + buffered chunks are pinned until process restart. In autonomous mode
* the run keeps producing output after the disconnect, so EVERY mid-run
* disconnect leaks a hung pipe. The patch races drain against close/error, and on
* a terminal socket event cancels the reader and breaks so `finally` always runs.
*
* This drives the REAL patched writeToServerResponse through the public
* pipeUIMessageStreamToResponse API with a response that never drains and closes
* mid-write exactly the leak scenario.
*/
/** A ServerResponse-like emitter whose first write() stalls (returns false) and
* then "closes" like a disconnecting client never firing 'drain'. */
class DisconnectingResponse extends EventEmitter {
ended = false;
writeCount = 0;
statusCode = 200;
writableEnded = false;
destroyed = false;
writeHead(): this {
return this;
}
setHeader(): void {}
flushHeaders(): void {}
write(): boolean {
this.writeCount++;
if (this.writeCount === 1) {
// Simulate the client vanishing mid-write: backpressure (false) and then a
// 'close' on the next tick, and CRUCIALLY never a 'drain'. Unpatched, the
// loop would await drain forever here.
setImmediate(() => this.emit('close'));
return false;
}
return true;
}
end(): void {
this.ended = true;
this.writableEnded = true;
this.emit('finish');
}
}
function makeModel() {
return new MockLanguageModelV3({
doStream: async () => ({
stream: simulateReadableStream({
chunks: [
{ type: 'stream-start' as const, warnings: [] },
{ type: 'text-start' as const, id: '1' },
{ type: 'text-delta' as const, id: '1', delta: 'hello ' },
{ type: 'text-delta' as const, id: '1', delta: 'world' },
{ type: 'text-end' as const, id: '1' },
{
type: 'finish' as const,
finishReason: { unified: 'stop' as const, raw: 'stop' },
usage: {
inputTokens: { total: 1, noCache: undefined, cacheRead: undefined, cacheWrite: undefined },
outputTokens: { total: 1, text: 1, reasoning: undefined },
},
},
],
}),
}),
});
}
describe('ai@6.0.134 pnpm patch: writeToServerResponse drain-hang (#486)', () => {
it('ends the response (does NOT hang) when the socket closes mid-write without draining', async () => {
const result = streamText({ model: makeModel(), prompt: 'hi' });
const res = new DisconnectingResponse();
// Drain the SDK stream independently, like the production detached path.
void result.consumeStream({ onError: () => undefined });
result.pipeUIMessageStreamToResponse(res as never);
// TRIPWIRE: the patched loop exits on 'close' and runs finally -> end().
// Unpatched, it awaits 'drain' forever and this never becomes true.
await new Promise<void>((resolve, reject) => {
const started = Date.now();
const poll = setInterval(() => {
if (res.ended) {
clearInterval(poll);
resolve();
} else if (Date.now() - started > 3000) {
clearInterval(poll);
reject(new Error('writeToServerResponse hung: response never ended'));
}
}, 20);
});
expect(res.ended).toBe(true);
});
it('does not emit an unhandledRejection when the fire-and-forget read() throws', async () => {
// The patch swallows read()'s rejection (fire-and-forget) with a log instead
// of letting it surface as a process-killing unhandledRejection.
const rejections: unknown[] = [];
const onUnhandled = (e: unknown) => rejections.push(e);
process.on('unhandledRejection', onUnhandled);
// Silence the patch's diagnostic console.error for the throwing read().
const errSpy = jest.spyOn(console, 'error').mockImplementation(() => undefined);
try {
const result = streamText({ model: makeModel(), prompt: 'hi' });
const res = new DisconnectingResponse();
void result.consumeStream({ onError: () => undefined });
result.pipeUIMessageStreamToResponse(res as never);
await new Promise((r) => setTimeout(r, 300));
} finally {
process.off('unhandledRejection', onUnhandled);
errSpy.mockRestore();
}
expect(rejections).toEqual([]);
});
it('both installed dist builds (CJS and ESM) carry the #486 patch marker', () => {
const cjsPath = require.resolve('ai');
const mjsPath = cjsPath.replace(/index\.js$/, 'index.mjs');
expect(cjsPath).toMatch(/index\.js$/);
expect(readFileSync(cjsPath, 'utf8')).toContain('PATCH(docmost #486)');
expect(readFileSync(mjsPath, 'utf8')).toContain('PATCH(docmost #486)');
});
});
@@ -0,0 +1,211 @@
import type { HealthIndicatorService } from '@nestjs/terminus';
import type { EnvironmentService } from '../environment/environment.service';
/**
* Integration guard for the /health Redis-probe handle leak (#486, commit 2).
*
* The bug: `pingCheck` built `new Redis(...)` per call and only disconnected on
* the SUCCESS path, so when Redis is DOWN every probe tick added ANOTHER
* forever-reconnecting client an unbounded handle/client leak for the duration
* of the outage. The fix reuses ONE long-lived probe client.
*
* This is an OBSERVABLE-property test, not an assertion on a mocked return value:
* we point the indicator at a REAL, refused TCP endpoint (a dead port) so ioredis
* genuinely fails to connect, run many probes, and assert the number of live
* Redis CLIENTS created stays at exactly ONE. `ioredis` is delegated to its real
* implementation (requireActual) only the constructor is wrapped to COUNT the
* real clients it creates, which is precisely the leaking resource.
*/
import type { Redis } from 'ioredis';
const mockLiveClients: Redis[] = [];
/**
* Fully tear a REAL ioredis client down so NO timer survives jest's 1s exit
* window (this suite must exit cleanly WITHOUT forceExit; see #382).
*
* `connector.disconnect()` arms a ~12s "force-destroy the stream" `setTimeout`
* that is cleared ONLY by the stream's 'close' event but only when the
* connector still holds a stream. Two problem cases:
* - a LIVE/connecting socket: disconnect arms the timer and 'close' may lag
* past jest's window, so we destroy the socket to make 'close' fire NOW;
* - a client BETWEEN reconnect attempts to a dead port: the held socket is
* ALREADY destroyed (its 'close' fired long ago), so disconnect would arm a
* timer whose clearing 'close' can never come again. We drop that dead stream
* reference BEFORE disconnect so the doomed timer is never armed.
* `disconnect()` itself also clears ioredis' own reconnect backoff timer.
*/
type DrainableStream = { destroyed?: boolean; destroy?: () => void } | null;
type DrainableClient = {
removeAllListeners: (event: string) => void;
disconnect: () => void;
stream?: DrainableStream;
connector?: { stream?: DrainableStream };
};
async function drainClient(client: Redis): Promise<void> {
if (!client || client.status === 'end') return;
const c = client as unknown as DrainableClient;
c.removeAllListeners('error');
// Drop an already-dead held socket so disconnect() can't arm a timer whose
// clearing 'close' will never fire again.
if (c.connector?.stream && c.connector.stream.destroyed) {
c.connector.stream = null;
}
if (c.stream && c.stream.destroyed) {
c.stream = null;
}
await new Promise<void>((resolve) => {
let done = false;
const finish = () => {
if (done) return;
done = true;
resolve();
};
client.once('end', finish);
// reconnect=false (the default): stop the retry loop and close the socket.
client.disconnect();
// Force any still-live socket closed NOW so the connector's stream-destroy
// timer clears inside jest's window instead of lagging behind a real 'close'.
if (c.stream && !c.stream.destroyed) {
c.stream.destroy?.();
}
// Fallback for a client with no live stream to emit 'end' (unref'd so it
// can never itself hold the loop open).
const fallback = setTimeout(finish, 500);
(fallback as { unref?: () => void }).unref?.();
});
}
async function drainAll(): Promise<void> {
await Promise.all(mockLiveClients.map((c) => drainClient(c)));
}
jest.mock('ioredis', () => {
const actual = jest.requireActual('ioredis');
const RealRedis = actual.Redis ?? actual.default ?? actual;
class CountingRedis extends RealRedis {
constructor(...args: unknown[]) {
super(...(args as []));
mockLiveClients.push(this as never);
}
}
return { ...actual, Redis: CountingRedis, default: CountingRedis };
});
// Import AFTER the mock is registered so the class picks up the counting client.
import { RedisHealthIndicator } from './redis.health';
describe('RedisHealthIndicator handle leak (#486)', () => {
const indicatorService = {
check: (key: string) => ({
up: () => ({ [key]: { status: 'up' } }),
down: (message: string) => ({ [key]: { status: 'down', message } }),
}),
} as unknown as HealthIndicatorService;
// A port with (almost certainly) nothing listening -> connection refused fast.
const environmentService = {
getRedisUrl: () => 'redis://127.0.0.1:6399/0',
} as unknown as EnvironmentService;
let indicator: RedisHealthIndicator;
beforeEach(() => {
mockLiveClients.length = 0;
indicator = new RedisHealthIndicator(indicatorService, environmentService);
});
afterEach(async () => {
// Drain (destroy socket + AWAIT 'end') every client the test created FIRST,
// so each is fully 'end' before onModuleDestroy's disconnect runs — that way
// no ioredis reconnect / stream-destroy timer outlives jest's exit window.
await drainAll();
indicator.onModuleDestroy();
});
it('creates exactly ONE Redis client across many probes while Redis is DOWN', async () => {
const N = 8;
for (let i = 0; i < N; i++) {
const result = await indicator.pingCheck('redis');
// Down endpoint -> every probe reports "down" (not an unhandled crash).
expect(result.redis.status).toBe('down');
}
// THE OBSERVABLE LEAK: on the buggy code this is N (a fresh, never-cleaned
// reconnecting client per probe). The fix reuses one shared client.
expect(mockLiveClients).toHaveLength(1);
});
it('onModuleDestroy releases the probe client (a later probe builds a fresh one)', async () => {
await indicator.pingCheck('redis');
expect(mockLiveClients).toHaveLength(1);
indicator.onModuleDestroy();
// A second destroy is a safe no-op (probeClient was nulled).
indicator.onModuleDestroy();
// After shutdown the indicator lazily builds a NEW client on the next probe,
// proving the old one was truly released rather than reused.
await indicator.pingCheck('redis');
expect(mockLiveClients).toHaveLength(2);
});
});
/**
* Happy-path regression guard (#486, B2): the FIRST probe against a LIVE Redis
* must report UP.
*
* With `lazyConnect: true` + `enableOfflineQueue: false`, a freshly-built client
* is in the `wait` state and the socket opens lazily. If the very first `ping()`
* is issued before an explicit `connect()`, ioredis rejects it instantly with
* "Stream isn't writeable and enableOfflineQueue options is false" a FALSE
* DOWN even though Redis is alive. The fix opens the socket before the first
* ping. This exercises a REAL ioredis client against a REAL TCP redis server
* (not a mock), so a regression genuinely reddens it.
*/
describe('RedisHealthIndicator live Redis first-probe (#486, B2)', () => {
const indicatorService = {
check: (key: string) => ({
up: () => ({ [key]: { status: 'up' } }),
down: (message: string) => ({ [key]: { status: 'down', message } }),
}),
} as unknown as HealthIndicatorService;
// A REAL running redis (see the neighboring harness / CI env).
const environmentService = {
getRedisUrl: () => 'redis://127.0.0.1:6379/0',
} as unknown as EnvironmentService;
let indicator: RedisHealthIndicator;
beforeEach(() => {
mockLiveClients.length = 0;
indicator = new RedisHealthIndicator(indicatorService, environmentService);
});
afterEach(async () => {
// Await full socket close of every live client (see drainClient) BEFORE
// onModuleDestroy: a real, connected ioredis client MUST be drained to 'end'
// or its stream-destroy timer keeps the jest worker alive past the 1s window.
await drainAll();
indicator.onModuleDestroy();
});
it('reports UP on the FIRST probe against a live Redis', async () => {
// The VERY FIRST probe — no warm-up ping — must be UP.
const result = await indicator.pingCheck('redis');
expect(result.redis.status).toBe('up');
});
it('stays UP on a probe AFTER onModuleDestroy re-creates the client', async () => {
await indicator.pingCheck('redis');
indicator.onModuleDestroy();
// The re-created client is again in `wait`; the first ping on it must still
// open the socket (the false-DOWN also recurs on the post-destroy path).
const result = await indicator.pingCheck('redis');
expect(result.redis.status).toBe('up');
});
});
@@ -2,33 +2,173 @@ import {
HealthIndicatorResult, HealthIndicatorResult,
HealthIndicatorService, HealthIndicatorService,
} from '@nestjs/terminus'; } from '@nestjs/terminus';
import { Injectable, Logger } from '@nestjs/common'; import { Injectable, Logger, OnModuleDestroy } from '@nestjs/common';
import { EnvironmentService } from '../environment/environment.service'; import { EnvironmentService } from '../environment/environment.service';
import { Redis } from 'ioredis'; import { Redis } from 'ioredis';
@Injectable() @Injectable()
export class RedisHealthIndicator { export class RedisHealthIndicator implements OnModuleDestroy {
private readonly logger = new Logger(RedisHealthIndicator.name); private readonly logger = new Logger(RedisHealthIndicator.name);
/**
* ONE long-lived probe connection, reused across every /health tick. The old
* code built `new Redis(...)` per call and only `disconnect()`d on the SUCCESS
* path, so while Redis was DOWN every probe added a fresh, forever-reconnecting
* client a handle leak that grew without bound for as long as the outage (and
* the health checker keeps polling) lasted. A single shared client keeps at most
* ONE background reconnect loop regardless of how many probes run.
*/
private probeClient: Redis | null = null;
/**
* How long the first-ping `connect()` may take before a probe gives up and
* reports DOWN. A `connect()` against a truly-down Redis never settles on its
* own (ioredis retries the socket indefinitely per its retryStrategy), so the
* probe MUST bound it or the /health handler would hang. Kept short so a real
* outage is reported fast; localhost/live Redis connects well within it.
*/
private static readonly CONNECT_TIMEOUT_MS = 2000;
/**
* The single in-flight first-`connect()`, memoized so CONCURRENT probes share
* it. k8s liveness+readiness hit /health in parallel on startup: without this,
* probe A drives `connect()` (the client leaves the `wait` state) and probe B,
* seeing a not-`wait`/not-`ready` client, would skip connect and fire `ping()`
* at a still-opening socket an instant FALSE DOWN. With the memo, B awaits
* the SAME connect. Cleared once it settles so a later disconnect / re-create
* starts a fresh connect.
*/
private connectingPromise: Promise<void> | null = null;
constructor( constructor(
private readonly healthIndicatorService: HealthIndicatorService, private readonly healthIndicatorService: HealthIndicatorService,
private environmentService: EnvironmentService, private environmentService: EnvironmentService,
) {} ) {}
private getProbeClient(): Redis {
if (!this.probeClient) {
this.probeClient = new Redis(this.environmentService.getRedisUrl(), {
// Constructing must never throw or eagerly connect; the first ping opens
// the socket. This lets us build the client once and reuse it.
lazyConnect: true,
// A health probe must fail FAST, not queue behind a stuck reconnect: one
// retry per request, and no offline queue so a ping while disconnected
// rejects immediately instead of buffering commands that pile up in RAM.
maxRetriesPerRequest: 1,
enableOfflineQueue: false,
});
// ioredis emits 'error' on every failed (re)connect; with no listener that
// surfaces as an unhandled 'error' event and can crash the process. Swallow
// it here — pingCheck already reports health — and log at debug so a Redis
// outage does not flood the logs.
this.probeClient.on('error', (err) => {
this.logger.debug(
`Redis probe connection error: ${
err instanceof Error ? err.message : String(err)
}`,
);
});
}
return this.probeClient;
}
/**
* Open the probe socket BEFORE the first ping. `lazyConnect: true` leaves a
* freshly-built (or post-destroy re-built) client in the `wait` state: the
* socket is NOT open yet, so with `enableOfflineQueue: false` the very first
* `ping()` rejects instantly with "Stream isn't writeable and
* enableOfflineQueue options is false" even when Redis is perfectly alive a
* false DOWN on the happy path. We drive `connect()` ONLY from `wait`; once
* the client is connected, ioredis owns its own (re)connect loop and a ping
* issued while it reconnects still fast-fails to a correct DOWN (offline queue
* stays off). A failed/timed-out connect rejects reported DOWN, which is the
* right signal for a truly-down Redis.
*/
private ensureConnected(client: Redis): Promise<void> {
// Already open — steady state, nothing to do.
if (client.status === 'ready') return Promise.resolve();
// A first-connect is already in flight (possibly started by a CONCURRENT
// probe): await the SAME one instead of racing a second connect() (ioredis
// throws "already connecting") or firing ping() at a not-yet-open socket.
if (this.connectingPromise) return this.connectingPromise;
// Only DRIVE connect() from the initial `wait` state (fresh / post-destroy
// re-created client). In any other non-ready state ioredis already owns its
// (re)connect loop; a ping there fast-fails to a correct DOWN, so we must not
// start a competing connect.
if (client.status !== 'wait') return Promise.resolve();
const promise = this.connectWithTimeout(client).finally(() => {
// Clear only if still ours, so a later disconnect / re-create can connect
// again. Whether it resolved or rejected, the memo has served its window.
if (this.connectingPromise === promise) {
this.connectingPromise = null;
}
});
this.connectingPromise = promise;
return promise;
}
private connectWithTimeout(client: Redis): Promise<void> {
return new Promise<void>((resolve, reject) => {
let settled = false;
const timer = setTimeout(() => {
if (settled) return;
settled = true;
reject(new Error('Redis probe connect timed out'));
}, RedisHealthIndicator.CONNECT_TIMEOUT_MS);
// Never let THIS timer alone keep the event loop (or a jest worker) alive;
// it is cleared on settle anyway, this is belt-and-braces.
timer.unref?.();
// `.catch` is always attached, so a connect() that rejects AFTER we have
// already timed out is handled here (guarded by `settled`) and never
// surfaces as an unhandled rejection.
client
.connect()
.then(() => {
if (settled) return;
settled = true;
clearTimeout(timer);
resolve();
})
.catch((err) => {
if (settled) return;
settled = true;
clearTimeout(timer);
reject(err);
});
});
}
async pingCheck(key: string): Promise<HealthIndicatorResult> { async pingCheck(key: string): Promise<HealthIndicatorResult> {
const indicator = this.healthIndicatorService.check(key); const indicator = this.healthIndicatorService.check(key);
try { try {
const redis = new Redis(this.environmentService.getRedisUrl(), { const redis = this.getProbeClient();
maxRetriesPerRequest: 15, // Open the socket before the first ping (see ensureConnected); without
}); // this the first probe after (re)creation falsely reports DOWN on a live
// Redis because lazyConnect defers the connect past the first ping.
await this.ensureConnected(redis);
await redis.ping(); await redis.ping();
redis.disconnect();
return indicator.up(); return indicator.up();
} catch (e) { } catch (e) {
this.logger.error(e); this.logger.error(e);
return indicator.down(`${key} is not available`); return indicator.down(`${key} is not available`);
} }
} }
onModuleDestroy(): void {
if (this.probeClient) {
// disconnect() (not quit()) tears the socket + reconnect loop down
// immediately without waiting on a round-trip to a possibly-down server.
// Do NOT removeAllListeners() with no event name — that would also strip
// ioredis' OWN internal listeners and break its teardown; our 'error'
// listener is harmless and dies with the dropped client reference.
this.probeClient.disconnect();
this.probeClient = null;
}
// Drop any in-flight first-connect memo so the NEXT client (lazily rebuilt on
// the next probe) starts a fresh connect rather than awaiting a promise tied
// to the client we just tore down.
this.connectingPromise = null;
}
} }
@@ -16,6 +16,7 @@ import {
} from './mcp-auth.helpers'; } from './mcp-auth.helpers';
import { JwtType } from '../../core/auth/dto/jwt-payload'; import { JwtType } from '../../core/auth/dto/jwt-payload';
import { CREDENTIALS_MISMATCH_MESSAGE } from '../../core/auth/auth.constants'; import { CREDENTIALS_MISMATCH_MESSAGE } from '../../core/auth/auth.constants';
import { McpService } from './mcp.service';
// The /mcp per-user auth decision logic is tested through the framework-free // The /mcp per-user auth decision logic is tested through the framework-free
// `resolveMcpSessionConfig` helper that McpService delegates to. McpService // `resolveMcpSessionConfig` helper that McpService delegates to. McpService
@@ -1179,3 +1180,46 @@ describe('mapAuthResultToResponse (handle status/body mapping, refactor R2)', ()
}); });
}); });
}); });
// #486: onModuleDestroy must ALSO tear down the live loopback CollabSessions, not
// just clear the sweep timer — otherwise the embedded MCP's collab sockets keep
// docs pinned open on the collab server past process exit. The teardown goes
// through an overridable seam (destroyAllMcpSessions) so it can be spied without
// loading the ESM-only @docmost/mcp package.
describe('McpService.onModuleDestroy — CollabSession teardown (#486)', () => {
function makeService(): McpService {
// The constructor only stores its deps and starts the (unref'd) sweep timer,
// so bare stubs suffice. onModuleDestroy clears that timer, so no leak.
return new McpService(
{} as any,
{} as any,
{} as any,
{} as any,
{} as any,
{} as any,
{} as any,
);
}
it('destroys all sessions AND clears the sweep timer on shutdown', async () => {
const svc = makeService();
const destroy = jest.fn().mockResolvedValue(undefined);
(svc as any).destroyAllMcpSessions = destroy;
const clearSpy = jest.spyOn(global, 'clearInterval');
await svc.onModuleDestroy();
expect(destroy).toHaveBeenCalledTimes(1);
expect(clearSpy).toHaveBeenCalledWith((svc as any).sweepTimer);
clearSpy.mockRestore();
});
it('swallows a teardown failure so shutdown never throws', async () => {
const svc = makeService();
(svc as any).destroyAllMcpSessions = jest
.fn()
.mockRejectedValue(new Error('collab teardown boom'));
await expect(svc.onModuleDestroy()).resolves.toBeUndefined();
});
});
@@ -119,9 +119,41 @@ export class McpService implements OnModuleDestroy {
this.sweepTimer.unref?.(); this.sweepTimer.unref?.();
} }
onModuleDestroy(): void { async onModuleDestroy(): Promise<void> {
clearInterval(this.sweepTimer); clearInterval(this.sweepTimer);
// Tear down any live loopback CollabSession providers at shutdown (#486). The
// embedded MCP (and the in-app AI agent) open Hocuspocus collab sockets against
// THIS process; without an explicit teardown those sessions keep their docs
// "open" on the collab server and hold providers/buffers until they idle out,
// so a restart can race a doc still pinned by the dying worker. Best-effort:
// any failure is logged, never allowed to break shutdown.
try {
await this.destroyAllMcpSessions();
} catch (err) {
this.logger.error(
'MCP CollabSession teardown on shutdown failed',
err as Error,
);
} }
}
/**
* Resolve @docmost/mcp's `destroyAllSessions` and invoke it (#486). The live
* CollabSession registry is a module-level singleton in the ESM package, shared
* by every entry (`.`/`./http`), so this tears down ALL sessions regardless of
* which surface opened them. The module is already loaded whenever MCP was used;
* if it was never loaded (or is absent) the import + no-op is harmless.
*
* Held as an overridable field so a unit test can spy the teardown without
* loading the ESM-only package or standing up the DI graph.
*/
private destroyAllMcpSessions: () => Promise<void> = async () => {
const entry = require.resolve('@docmost/mcp');
const mod = (await esmImport(pathToFileURL(entry).href)) as {
destroyAllSessions?: () => void;
};
mod.destroyAllSessions?.();
};
// Service account the embedded MCP uses to talk back to this Docmost // Service account the embedded MCP uses to talk back to this Docmost
// instance over loopback REST + the collaboration WebSocket. Now OPTIONAL: // instance over loopback REST + the collaboration WebSocket. Now OPTIONAL:
@@ -0,0 +1,148 @@
import { get as httpGet } from 'node:http';
import { AddressInfo } from 'node:net';
import { createServer } from 'node:http';
// Drive the metrics HTTP server without the load-time METRICS_PORT gate: mock the
// registry so isMetricsEnabled()/getMetricsRegistry() are always satisfied. What
// we assert is observed over a REAL socket (bind address, status codes), not on
// the mock.
jest.mock('./metrics.registry', () => ({
isMetricsEnabled: () => true,
getMetricsRegistry: () => ({
metrics: async () => '# HELP up test\nup 1\n',
contentType: 'text/plain; version=0.0.4',
}),
}));
import {
startMetricsServer,
closeMetricsServer,
resolveMetricsBind,
resolveMetricsToken,
} from './metrics.server';
/** Find a free TCP port (the metrics server requires METRICS_PORT > 0). */
function freePort(): Promise<number> {
return new Promise((resolve, reject) => {
const s = createServer();
s.once('error', reject);
s.listen(0, '127.0.0.1', () => {
const p = (s.address() as AddressInfo).port;
s.close(() => resolve(p));
});
});
}
/** Minimal GET against 127.0.0.1:port with optional Authorization header. */
function req(
port: number,
headers: Record<string, string> = {},
): Promise<{ status: number; body: string }> {
return new Promise((resolve, reject) => {
const r = httpGet(
{ host: '127.0.0.1', port, path: '/metrics', headers },
(res) => {
let body = '';
res.on('data', (c) => (body += c));
res.on('end', () =>
resolve({ status: res.statusCode ?? 0, body }),
);
},
);
r.on('error', reject);
});
}
describe('metrics server bind + auth (#486)', () => {
const saved = {
bind: process.env.METRICS_BIND,
token: process.env.METRICS_TOKEN,
port: process.env.METRICS_PORT,
};
afterEach(async () => {
await closeMetricsServer();
process.env.METRICS_BIND = saved.bind;
process.env.METRICS_TOKEN = saved.token;
process.env.METRICS_PORT = saved.port;
delete process.env.METRICS_BIND;
delete process.env.METRICS_TOKEN;
});
describe('resolveMetricsBind', () => {
it('defaults to loopback 127.0.0.1', () => {
delete process.env.METRICS_BIND;
expect(resolveMetricsBind()).toBe('127.0.0.1');
});
it('honours the METRICS_BIND override', () => {
process.env.METRICS_BIND = '0.0.0.0';
expect(resolveMetricsBind()).toBe('0.0.0.0');
});
it('treats a blank override as unset (loopback)', () => {
process.env.METRICS_BIND = ' ';
expect(resolveMetricsBind()).toBe('127.0.0.1');
});
});
describe('resolveMetricsToken', () => {
it('is null when unset', () => {
delete process.env.METRICS_TOKEN;
expect(resolveMetricsToken()).toBeNull();
});
it('returns the trimmed token when set', () => {
process.env.METRICS_TOKEN = ' s3cret ';
expect(resolveMetricsToken()).toBe('s3cret');
});
});
it('binds to loopback by default and serves /metrics without auth when no token', async () => {
delete process.env.METRICS_BIND;
delete process.env.METRICS_TOKEN;
const port = await freePort();
process.env.METRICS_PORT = String(port);
const server = startMetricsServer();
expect(server).not.toBeNull();
await new Promise<void>((resolve) => {
if (server!.listening) resolve();
else server!.once('listening', () => resolve());
});
// OBSERVABLE: the listener bound to loopback, not 0.0.0.0.
expect((server!.address() as AddressInfo).address).toBe('127.0.0.1');
const res = await req(port);
expect(res.status).toBe(200);
expect(res.body).toContain('up 1');
});
it('rejects unauthenticated scrapes with 401 and accepts the exact Bearer token', async () => {
delete process.env.METRICS_BIND;
process.env.METRICS_TOKEN = 'topsecret';
const port = await freePort();
process.env.METRICS_PORT = String(port);
const server = startMetricsServer();
expect(server).not.toBeNull();
// No auth -> 401.
const noAuth = await req(port);
expect(noAuth.status).toBe(401);
// Wrong token, DIFFERENT length -> 401 (short-circuits on the length guard).
const wrong = await req(port, { authorization: 'Bearer nope' });
expect(wrong.status).toBe(401);
// Wrong token, SAME length -> 401. This drives the timingSafeEqual compare
// itself (the length guard passes: 'Bearer topsecreX' has the same length as
// 'Bearer topsecret'). Pins the constant-time compare: a regression that made
// it return true would let this equal-length wrong token through — the
// different-length case above would NOT catch that.
const sameLen = await req(port, { authorization: 'Bearer topsecreX' });
expect(sameLen.status).toBe(401);
// Correct token -> 200 with the metrics body.
const ok = await req(port, { authorization: 'Bearer topsecret' });
expect(ok.status).toBe(200);
expect(ok.body).toContain('up 1');
});
});
@@ -1,7 +1,27 @@
import { createServer, Server } from 'node:http'; import { createServer, Server } from 'node:http';
import { timingSafeEqual } from 'node:crypto';
import { Logger } from '@nestjs/common'; import { Logger } from '@nestjs/common';
import { getMetricsRegistry, isMetricsEnabled } from './metrics.registry'; import { getMetricsRegistry, isMetricsEnabled } from './metrics.registry';
/**
* Constant-time compare of the presented Authorization header against the
* expected `Bearer <token>`. This is the ONLY auth layer for the metrics
* endpoint, so a naive `!==` would leak the token byte-by-byte via timing.
* timingSafeEqual requires equal-length buffers, so a length mismatch short-
* circuits to "not equal" (its own length is not itself a useful oracle: the
* expected string length is fixed by config, not secret-derived).
*/
function bearerMatches(
presented: string | undefined,
expected: string,
): boolean {
if (typeof presented !== 'string') return false;
const a = Buffer.from(presented);
const b = Buffer.from(expected);
if (a.length !== b.length) return false;
return timingSafeEqual(a, b);
}
/** /**
* Start the Prometheus scrape endpoint on a SEPARATE port, taken from * Start the Prometheus scrape endpoint on a SEPARATE port, taken from
* `METRICS_PORT`. There is NO default port: when `METRICS_PORT` is unset the * `METRICS_PORT`. There is NO default port: when `METRICS_PORT` is unset the
@@ -16,6 +36,30 @@ import { getMetricsRegistry, isMetricsEnabled } from './metrics.registry';
*/ */
let metricsServer: Server | null = null; let metricsServer: Server | null = null;
/**
* Interface the metrics endpoint binds to. Defaults to LOOPBACK (127.0.0.1) so
* the unauthenticated `/metrics` surface is NOT exposed on all interfaces by
* default the old `0.0.0.0` bind put an auth-less endpoint on every interface.
* Deployments where the scraper runs in a SEPARATE container (and reaches this as
* `docmost:9464`) set `METRICS_BIND=0.0.0.0`, ideally together with METRICS_TOKEN
* and/or a private network so the port is not world-readable.
*/
export function resolveMetricsBind(): string {
const raw = (process.env.METRICS_BIND ?? '').trim();
return raw.length > 0 ? raw : '127.0.0.1';
}
/**
* Optional Bearer token guarding `/metrics`. When `METRICS_TOKEN` is set, every
* scrape must present `Authorization: Bearer <token>`; unset (default) leaves the
* endpoint open (safe when bound to loopback / a trusted network). Returns the
* trimmed token or null when unset/blank.
*/
export function resolveMetricsToken(): string | null {
const raw = (process.env.METRICS_TOKEN ?? '').trim();
return raw.length > 0 ? raw : null;
}
export function startMetricsServer(): Server | null { export function startMetricsServer(): Server | null {
if (!isMetricsEnabled()) return null; if (!isMetricsEnabled()) return null;
@@ -31,8 +75,22 @@ export function startMetricsServer(): Server | null {
return null; return null;
} }
const bind = resolveMetricsBind();
const token = resolveMetricsToken();
const server = createServer(async (req, res) => { const server = createServer(async (req, res) => {
if (req.method === 'GET' && req.url === '/metrics') { if (req.method === 'GET' && req.url === '/metrics') {
// Optional Bearer auth: reject scrapes without the exact token when one is
// configured. This is the auth layer the old all-interfaces bind lacked.
if (token) {
const auth = req.headers['authorization'];
if (!bearerMatches(auth, `Bearer ${token}`)) {
res.statusCode = 401;
res.setHeader('WWW-Authenticate', 'Bearer');
res.end();
return;
}
}
try { try {
const body = await register.metrics(); const body = await register.metrics();
res.setHeader('Content-Type', register.contentType); res.setHeader('Content-Type', register.contentType);
@@ -48,10 +106,14 @@ export function startMetricsServer(): Server | null {
res.end(); res.end();
}); });
// Bind on all interfaces: the scraper (VictoriaMetrics) reaches this from // Bind to loopback by default so the auth-less endpoint is not exposed on all
// another container as docmost:9464. The port is not published to the host. // interfaces. Set METRICS_BIND=0.0.0.0 (ideally with METRICS_TOKEN) when the
server.listen(port, '0.0.0.0', () => { // scraper runs in a separate container and reaches this as docmost:9464.
logger.log(`Metrics endpoint listening on :${port}/metrics`); server.listen(port, bind, () => {
logger.log(
`Metrics endpoint listening on ${bind}:${port}/metrics` +
(token ? ' (Bearer auth required)' : ''),
);
}); });
server.on('error', (err) => { server.on('error', (err) => {
@@ -31,9 +31,6 @@ export enum QueueJob {
IMPORT_TASK = 'import-task', IMPORT_TASK = 'import-task',
EXPORT_TASK = 'export-task', EXPORT_TASK = 'export-task',
SEARCH_REMOVE_PAGE = 'search-remove-page',
SEARCH_REMOVE_ASSET = 'search-remove-attachment',
SEARCH_REMOVE_FACE = 'search-remove-comment',
TYPESENSE_FLUSH = 'typesense-flush', TYPESENSE_FLUSH = 'typesense-flush',
PAGE_CREATED = 'page-created', PAGE_CREATED = 'page-created',
+77 -37
View File
@@ -1,60 +1,100 @@
// Codegen: emit src/registry-stamp.generated.ts with a REGISTRY_STAMP hash of // Codegen: emit src/registry-stamp.generated.ts with a REGISTRY_STAMP hash of
// the tool-specs REGISTRY CONTENT, so a build/ vs src/ skew (issue #447) is // the ENTIRE src/ tree, so a build/ vs src/ skew (issue #447) is detectable at
// detectable at runtime. // runtime for ANY source file — not just tool-specs.ts.
// //
// WHY hash the raw source text (not extracted structured data): // WHY hash the whole src tree (not just tool-specs.ts): the runtime tools are
// SHARED_TOOL_SPECS carries `buildShape` functions (the input SCHEMAS) which are // assembled from far more than the spec registry — client.ts, the client/*
// NOT serializable. The input schema is exactly one of the things that MUST stay // domain modules, comment-signal.ts and the drawio-* helpers all ship in build/
// in sync between build/ and src/, so we cannot drop it from the hash. Rather // and are loaded by the in-app server. Hashing ONLY tool-specs.ts meant an edit
// than probe zod with a fragile shim to reconstruct the schema shape, we hash the // to any of those (e.g. a behavioural fix in client.ts) left the stamp unchanged,
// STABLE, deterministic source TEXT of tool-specs.ts. That text fully captures // so a stale build/ served the OLD code silently (issue #486). Hashing every
// every field that must stay in sync — mcpName, inAppKey, description, tier, // src/**/*.ts closes that gap: any source edit changes the stamp.
// catalogLine AND the buildShape bodies (input schemas) — with zero probing
// fragility. Any edit to a spec (a renamed tool, a reworded description, a
// changed schema field) changes the text and therefore the stamp.
// //
// DETERMINISM: the hash is computed over the file bytes with line endings // WHY hash the raw source text (not extracted structured data): the tool input
// normalized to LF and a single trailing newline stripped, so a CRLF checkout or // SCHEMAS live as `buildShape` functions which are NOT serializable, so we cannot
// an editor's trailing-newline habit cannot make build/ and src/ disagree. No // reduce them to structured data without a fragile zod shim. Hashing the STABLE,
// Date.now / randomness. The loader's dev-only stale-check (docmost-client.loader.ts) // deterministic source TEXT captures every field that must stay in sync with zero
// re-runs THIS SAME normalization + sha256 over src/tool-specs.ts and compares to // probing fragility. Any edit to any source file changes the text → the stamp.
// the built REGISTRY_STAMP; the two must compute identically. //
// DETERMINISM: files are enumerated recursively, filtered to *.ts EXCLUDING
// *.generated.ts (the codegen's OWN output — including it would create a
// fixed-point cycle), and sorted by their POSIX-normalized path relative to src/
// so the order is platform-independent. Each file contributes its relative path
// AND its content with line endings normalized to LF and a single trailing
// newline stripped, so a CRLF checkout or an editor's trailing-newline habit
// cannot make build/ and src/ disagree. No Date.now / randomness. The loader's
// dev-only stale-check (docmost-client.loader.ts) re-runs THIS SAME enumeration +
// normalization + sha256 and compares to the built REGISTRY_STAMP; the two must
// compute identically.
// //
// This script runs from the `build` and `pretest` npm scripts BEFORE tsc, so // This script runs from the `build` and `pretest` npm scripts BEFORE tsc, so
// build/ always carries a stamp derived from the tool-specs.ts that was compiled. // build/ always carries a stamp derived from the src/ tree that was compiled.
import { createHash } from 'node:crypto'; import { createHash } from 'node:crypto';
import { readFileSync, writeFileSync } from 'node:fs'; import { readdirSync, readFileSync, statSync, writeFileSync } from 'node:fs';
import { fileURLToPath } from 'node:url'; import { fileURLToPath } from 'node:url';
import { dirname, join } from 'node:path'; import { dirname, join, relative, sep } from 'node:path';
const __dirname = dirname(fileURLToPath(import.meta.url)); const __dirname = dirname(fileURLToPath(import.meta.url));
const SRC_DIR = join(__dirname, '..', 'src'); const SRC_DIR = join(__dirname, '..', 'src');
const TOOL_SPECS_PATH = join(SRC_DIR, 'tool-specs.ts');
const OUT_PATH = join(SRC_DIR, 'registry-stamp.generated.ts'); const OUT_PATH = join(SRC_DIR, 'registry-stamp.generated.ts');
/** /**
* Deterministic stamp of the tool-specs registry content. Kept as a plain * Recursively enumerate every `*.ts` file under `dir`, EXCLUDING the codegen's
* function (exported) so the algorithm has a single home; the loader duplicates * own `*.generated.ts` output (a self-referential cycle otherwise). Returns
* only the tiny normalize+sha256 steps because it lives in the CJS server build * absolute paths, unsorted (the caller sorts by relative path for determinism).
* and cannot import this ESM script. If you change the normalization here, mirror * Kept as a plain exported function so the algorithm has a single home; the
* it in apps/server/src/core/ai-chat/tools/docmost-client.loader.ts. * loader duplicates it because it lives in the CJS server build and cannot import
* this ESM script. If you change the walk/filter here, mirror it in
* apps/server/src/core/ai-chat/tools/docmost-client.loader.ts.
*/ */
export function computeRegistryStamp(toolSpecsSource) { export function collectStampFiles(dir) {
const normalized = toolSpecsSource.replace(/\r\n/g, '\n').replace(/\n$/, ''); const out = [];
return createHash('sha256').update(normalized, 'utf8').digest('hex'); for (const entry of readdirSync(dir)) {
const full = join(dir, entry);
if (statSync(full).isDirectory()) {
out.push(...collectStampFiles(full));
} else if (entry.endsWith('.ts') && !entry.endsWith('.generated.ts')) {
out.push(full);
}
}
return out;
}
/**
* Deterministic stamp of the whole src/ tree. Enumerate + sort by POSIX-relative
* path, then fold each file's relative path AND normalized content into one
* sha256. MUST stay byte-for-byte identical to the loader's recompute.
*/
export function computeRegistryStamp(srcDir) {
const files = collectStampFiles(srcDir)
.map((abs) => ({
rel: relative(srcDir, abs).split(sep).join('/'),
abs,
}))
.sort((a, b) => (a.rel < b.rel ? -1 : a.rel > b.rel ? 1 : 0));
const hash = createHash('sha256');
for (const { rel, abs } of files) {
const normalized = readFileSync(abs, 'utf8')
.replace(/\r\n/g, '\n')
.replace(/\n$/, '');
hash.update(rel, 'utf8');
hash.update('\0', 'utf8');
hash.update(normalized, 'utf8');
hash.update('\0', 'utf8');
}
return hash.digest('hex');
} }
function main() { function main() {
const source = readFileSync(TOOL_SPECS_PATH, 'utf8'); const stamp = computeRegistryStamp(SRC_DIR);
const stamp = computeRegistryStamp(source);
const out = const out =
'// AUTO-GENERATED by scripts/gen-registry-stamp.mjs — DO NOT EDIT BY HAND.\n' + '// AUTO-GENERATED by scripts/gen-registry-stamp.mjs — DO NOT EDIT BY HAND.\n' +
'// A deterministic hash of src/tool-specs.ts content (tool names, descriptions,\n' + '// A deterministic hash of the whole src/ tree (every src/**/*.ts except\n' +
'// tiers, catalog lines and input schemas). Regenerated on every build/pretest\n' + '// *.generated.ts). Regenerated on every build/pretest so build/ always\n' +
'// so build/ always matches the compiled src. The in-app loader recomputes this\n' + '// matches the compiled src. The in-app loader recomputes this from src and\n' +
'// from src and refuses to run on a mismatch (issue #447). This file is\n' + '// refuses to run on a mismatch (issue #447/#486). This file is gitignored\n' +
'// gitignored and produced by the build — see .gitignore.\n' + '// and produced by the build — see .gitignore.\n' +
`export const REGISTRY_STAMP = ${JSON.stringify(stamp)};\n`; `export const REGISTRY_STAMP = ${JSON.stringify(stamp)};\n`;
writeFileSync(OUT_PATH, out, 'utf8'); writeFileSync(OUT_PATH, out, 'utf8');
// eslint-disable-next-line no-console // eslint-disable-next-line no-console
+51 -4
View File
@@ -19,7 +19,10 @@ import {
assertYjsEncodable, assertYjsEncodable,
MutationResult, MutationResult,
} from "../lib/collaboration.js"; } from "../lib/collaboration.js";
import { acquireCollabSession } from "../lib/collab-session.js"; import {
acquireCollabSession,
isCollabAuthFailedError,
} from "../lib/collab-session.js";
import { withPageLock, isUuid } from "../lib/page-lock.js"; import { withPageLock, isUuid } from "../lib/page-lock.js";
import { getCollabToken, performLogin } from "../lib/auth-utils.js"; import { getCollabToken, performLogin } from "../lib/auth-utils.js";
import { formatDocmostAxiosError } from "./errors.js"; import { formatDocmostAxiosError } from "./errors.js";
@@ -492,6 +495,37 @@ export abstract class DocmostClientContext {
); );
} }
/**
* Run a collab write and, on a Hocuspocus HANDSHAKE auth failure, self-heal
* once (#486). Symmetric to the HTTP-401 path in getCollabTokenWithReauth: the
* REST interceptor and login() already drop the cached collab token on a 401/
* 403, but a rejected WEBSOCKET handshake left the stale token in the cache, so
* every subsequent mutation kept re-presenting the same bad token for up to the
* collab-token TTL (minutes) with no self-heal. Here, when the write rejects
* with the tagged collab-auth error, we invalidate the cached token and retry
* the write EXACTLY once with a force-refreshed token. Not a loop: a second
* failure (or any non-auth error) propagates unchanged.
*
* `write` receives the token to use, so the retry can hand it a genuinely fresh
* one rather than re-running with the same stale string.
*/
protected async writeWithCollabAuthRetry<T>(
collabToken: string,
write: (token: string) => Promise<T>,
): Promise<T> {
try {
return await write(collabToken);
} catch (e) {
if (!isCollabAuthFailedError(e)) throw e;
// The WS handshake rejected our token: drop it from the cache so it can't
// be reused for the rest of the TTL, mint a fresh one (forceRefresh bypasses
// the cache and re-invokes the provider/login), and retry the write once.
this.collabTokenCache = null;
const fresh = await this.getCollabTokenWithReauth(true);
return await write(fresh);
}
}
/** /**
* Connect to the collaboration websocket, read the live doc, apply * Connect to the collaboration websocket, read the live doc, apply
* `transform`, write the result, and wait for the server to persist it * `transform`, write the result, and wait for the server to persist it
@@ -526,7 +560,11 @@ export abstract class DocmostClientContext {
// unsyncedChanges/connectionLost ack logic live in CollabSession.mutate, // unsyncedChanges/connectionLost ack logic live in CollabSession.mutate,
// preserved verbatim from the old inline machine (incl. the #152 structural // preserved verbatim from the old inline machine (incl. the #152 structural
// diff that keeps a live editor's cursor anchored). // diff that keeps a live editor's cursor anchored).
const session = await acquireCollabSession(pageId, collabToken, this.apiUrl, { // Wrap in the collab-auth self-heal (#486): a rejected WS handshake drops the
// cached collab token and retries once with a fresh one (the retry passes the
// refreshed token down to acquireCollabSession via `token`).
return this.writeWithCollabAuthRetry(collabToken, async (token) => {
const session = await acquireCollabSession(pageId, token, this.apiUrl, {
// Only the actual 25s collab connect timeout emits this — the connect-vs- // Only the actual 25s collab connect timeout emits this — the connect-vs-
// unload signal; the other failure paths must NOT emit it. // unload signal; the other failure paths must NOT emit it.
onConnectTimeout: () => onConnectTimeout: () =>
@@ -539,6 +577,7 @@ export abstract class DocmostClientContext {
session.destroy("mutate failed"); session.destroy("mutate failed");
throw e; throw e;
} }
});
} }
/** /**
@@ -667,7 +706,11 @@ export abstract class DocmostClientContext {
transform: (doc: any) => any, transform: (doc: any) => any,
): Promise<{ doc?: any; verify?: any }> { ): Promise<{ doc?: any; verify?: any }> {
const pageUuid = await this.resolvePageId(pageId); const pageUuid = await this.resolvePageId(pageId);
return mutatePageContent(pageUuid, collabToken, apiUrl, transform); // #486: on a rejected collab-WS handshake, invalidate + refresh the token and
// retry the write once (symmetric to the HTTP-401 reauth path).
return this.writeWithCollabAuthRetry(collabToken, (token) =>
mutatePageContent(pageUuid, token, apiUrl, transform),
);
} }
/** /**
@@ -687,7 +730,11 @@ export abstract class DocmostClientContext {
apiUrl: string, apiUrl: string,
): Promise<{ doc?: any; verify?: any }> { ): Promise<{ doc?: any; verify?: any }> {
const pageUuid = await this.resolvePageId(pageId); const pageUuid = await this.resolvePageId(pageId);
return replacePageContent(pageUuid, doc, collabToken, apiUrl); // #486: on a rejected collab-WS handshake, invalidate + refresh the token and
// retry the write once (symmetric to the HTTP-401 reauth path).
return this.writeWithCollabAuthRetry(collabToken, (token) =>
replacePageContent(pageUuid, doc, token, apiUrl),
);
} }
/** /**
+7 -2
View File
@@ -127,8 +127,13 @@ export function ReadMixin<TBase extends GConstructor<DocmostClientContext>>(Base
"listPages: tree mode requires a spaceId (a page tree is scoped to one space). Pass spaceId, or omit tree to get the recent-pages list.", "listPages: tree mode requires a spaceId (a page tree is scoped to one space). Pass spaceId, or omit tree to get the recent-pages list.",
); );
} }
const { pages } = await this.enumerateSpacePages(spaceId); // #486: propagate `truncated` (same pattern as check_new_comments). The old
return buildPageTree(pages); // code dropped it, so a caller handed an INCOMPLETE tree (the stdio-fallback
// BFS hit its node cap) had no way to know pages were missing. Return the
// tree alongside the flag; the primary /pages/tree path is uncapped so this
// is false there.
const { pages, truncated } = await this.enumerateSpacePages(spaceId);
return { tree: buildPageTree(pages), truncated };
} }
const clampedLimit = Math.max(1, Math.min(100, limit)); const clampedLimit = Math.max(1, Math.min(100, limit));
-1
View File
@@ -4,7 +4,6 @@ import { readFileSync } from "fs";
import { fileURLToPath } from "url"; import { fileURLToPath } from "url";
import { dirname, join } from "path"; import { dirname, join } from "path";
import { DocmostClient, DocmostMcpConfig } from "./client.js"; import { DocmostClient, DocmostMcpConfig } from "./client.js";
import { parseNodeArg } from "@docmost/prosemirror-markdown";
import { searchShapes } from "./lib/drawio-shapes.js"; import { searchShapes } from "./lib/drawio-shapes.js";
import { getGuideSection } from "./lib/drawio-guide.js"; import { getGuideSection } from "./lib/drawio-guide.js";
import { SHARED_TOOL_SPECS, SharedToolSpec } from "./tool-specs.js"; import { SHARED_TOOL_SPECS, SharedToolSpec } from "./tool-specs.js";
+26 -4
View File
@@ -37,6 +37,25 @@ const CONNECT_TIMEOUT_MS = 25000;
/** Time we wait for the server to acknowledge our write before giving up. */ /** Time we wait for the server to acknowledge our write before giving up. */
const PERSIST_TIMEOUT_MS = 20000; const PERSIST_TIMEOUT_MS = 20000;
/**
* Marker property set on the Error thrown when the Hocuspocus handshake REJECTS
* our collab token (onAuthenticationFailed). The client wraps content writes so
* that on this specific failure it invalidates its cached collab token and
* retries once with a fresh one symmetric to the HTTP-401 reauth path (#486).
* A plain message-match would be brittle; a tagged property is unambiguous and
* survives teardown (which rejects pending ops with this SAME error object).
*/
const COLLAB_AUTH_FAILED_MARKER = "collabAuthFailed";
/** True when `e` is the tagged collab-WS auth-failure error (see marker above). */
export function isCollabAuthFailedError(e: unknown): boolean {
return !!(
e &&
typeof e === "object" &&
(e as Record<string, unknown>)[COLLAB_AUTH_FAILED_MARKER] === true
);
}
/** /**
* Tunables, read fresh from the environment on every acquire so tests (and a * Tunables, read fresh from the environment on every acquire so tests (and a
* live rollback) can change them without reloading the module. Mirrors how * live rollback) can change them without reloading the module. Mirrors how
@@ -302,10 +321,13 @@ export class CollabSession {
this.openResolve?.(); this.openResolve?.();
}, },
onAuthenticationFailed: () => { onAuthenticationFailed: () => {
this.teardown( // Tag the error so the client can tell a REJECTED collab token apart
new Error("Authentication failed for collaboration connection"), // from a generic disconnect and invalidate + refresh it (#486).
true, const err = new Error(
); "Authentication failed for collaboration connection",
) as Error & { [COLLAB_AUTH_FAILED_MARKER]?: boolean };
err[COLLAB_AUTH_FAILED_MARKER] = true;
this.teardown(err, true);
}, },
}); });
}); });
+74 -31
View File
@@ -10,7 +10,7 @@
// exactly mxGraph's convention for a child of a container, so they map across // exactly mxGraph's convention for a child of a container, so they map across
// directly. Container sizes are computed by ELK; leaf sizes are preserved. // directly. Container sizes are computed by ELK; leaf sizes are preserved.
import ELK from "elkjs/lib/elk.bundled.js"; import { Worker } from "node:worker_threads";
import { JSDOM } from "jsdom"; import { JSDOM } from "jsdom";
import { normalizeInput, parseCells, type DrawioCell } from "./drawio-xml.js"; import { normalizeInput, parseCells, type DrawioCell } from "./drawio-xml.js";
@@ -18,22 +18,33 @@ import { normalizeInput, parseCells, type DrawioCell } from "./drawio-xml.js";
const DEFAULT_W = 140; const DEFAULT_W = 140;
const DEFAULT_H = 60; const DEFAULT_H = 60;
// DoS bounds for the in-process ELK layout. The mxGraph XML is LLM-supplied // DoS bounds for the ELK layout. The mxGraph XML is LLM-supplied (layout:"elk"
// (layout:"elk" in drawioCreate/drawioUpdate) and elkjs runs synchronously on // in drawioCreate/drawioUpdate). elkjs' layout() returns a Promise but runs the
// the MCP server's event loop, so an unbounded graph would block it for // crossing-minimisation SYNCHRONOUSLY — it blocks whatever thread it runs on for
// seconds-to-minutes. A ~1MB XML (well under the stage-1 16MB cap) can carry // the whole pass. A ~1MB XML (well under the stage-1 16MB cap) can carry
// thousands of nodes. We cap the graph size and race the layout against a // thousands of nodes. We (a) cap the graph size before ever calling ELK and
// wall-clock timeout; on either bound we fall back to the ORIGINAL model, the // (b) run the layout in a WORKER THREAD so the main event loop stays free, with
// same best-effort contract the catch already honours. // the wall-clock timeout enforced by terminating that worker. On either bound we
// fall back to the ORIGINAL model, the same best-effort contract the catch honours.
// - 500 nodes lays out in well under a second; beyond that ELK cost climbs // - 500 nodes lays out in well under a second; beyond that ELK cost climbs
// steeply, so refuse and leave the (already-valid) model untouched. // steeply, so refuse and leave the (already-valid) model untouched.
// - Edges dominate the layered-crossing cost, so allow a bit more headroom // - Edges dominate the layered-crossing cost, so allow a bit more headroom
// (1000) than nodes but still bound them. // (1000) than nodes but still bound them.
// - 5s is generous for any graph within the caps yet short enough that a // - The timeout is a HARD kill of the worker thread — the only way to interrupt
// pathological input can never wedge the server. // synchronous JS. The in-process setTimeout race we used before was an
// illusion: the timer could never fire while the SAME thread was blocked
// inside elkjs, so it "protected" nothing. Now the timer runs on the main
// thread while ELK runs on the worker, so it can actually fire and terminate.
const ELK_MAX_NODES = 500; const ELK_MAX_NODES = 500;
const ELK_MAX_EDGES = 1000; const ELK_MAX_EDGES = 1000;
const ELK_TIMEOUT_MS = 5000; // Wall-clock ceiling for a single layout pass. Overridable for tests (a tiny
// value forces the terminate-on-timeout path deterministically); a non-positive
// or unparseable override falls back to the default.
const ELK_TIMEOUT_DEFAULT_MS = 5000;
function resolveElkTimeoutMs(): number {
const raw = Number(process.env.DRAWIO_ELK_TIMEOUT_MS);
return Number.isFinite(raw) && raw > 0 ? Math.floor(raw) : ELK_TIMEOUT_DEFAULT_MS;
}
// Spacing is set >=150px on purpose so an ELK layout never trips the linter's // Spacing is set >=150px on purpose so an ELK layout never trips the linter's
// "gap between adjacent shapes < 150px" quality warning (acceptance #3). // "gap between adjacent shapes < 150px" quality warning (acceptance #3).
@@ -78,13 +89,57 @@ interface ElkGraph extends ElkNode {
edges?: ElkEdge[]; edges?: ElkEdge[];
} }
/**
* Run one ELK layered layout on a worker thread and resolve with the laid-out
* graph. The timeout is enforced by `worker.terminate()` a HARD kill, which is
* the only way to interrupt elkjs' synchronous crossing-minimisation once it has
* started. Rejects on timeout, worker error, or an early exit; the caller treats
* any rejection as "keep the original model" (best-effort layout).
*/
function layoutInWorker(graph: ElkGraph, timeoutMs: number): Promise<ElkGraph> {
return new Promise((resolve, reject) => {
const worker = new Worker(
new URL("./drawio-layout.worker.js", import.meta.url),
{ workerData: { graph } },
);
let settled = false;
const finish = (fn: () => void) => {
if (settled) return;
settled = true;
clearTimeout(timer);
// Always tear the worker down: on the happy path so it does not linger,
// on timeout so the blocked synchronous ELK run is actually interrupted.
void worker.terminate();
fn();
};
const timer = setTimeout(
() => finish(() => reject(new Error("ELK layout timed out"))),
timeoutMs,
);
worker.once("message", (msg: { ok?: boolean; laid?: ElkGraph; error?: string }) => {
finish(() =>
msg?.ok
? resolve(msg.laid as ElkGraph)
: reject(new Error(msg?.error ?? "ELK layout failed")),
);
});
worker.once("error", (err) => finish(() => reject(err)));
worker.once("exit", (code) => {
// A clean exit after we already settled is normal (terminate()); only an
// unexpected early exit while still pending is a failure.
if (settled) return;
finish(() => reject(new Error(`ELK worker exited early (code ${code})`)));
});
});
}
/** /**
* Apply an ELK layered layout to a drawio input and return a full mxGraphModel * Apply an ELK layered layout to a drawio input and return a full mxGraphModel
* string with rewritten geometry. Accepts the same three input forms as * string with rewritten geometry. Accepts the same three input forms as
* drawioCreate (a bare model, an <mxfile>, or a <mxCell> list). Async because * drawioCreate (a bare model, an <mxfile>, or a <mxCell> list). Async because
* elkjs' layout() is promise-based. On any layout failure the ORIGINAL * the layout runs on a worker thread. On any layout failure (including a
* (normalized) model is returned unchanged layout is best-effort polish, never * terminate-on-timeout) the ORIGINAL (normalized) model is returned unchanged
* a reason to fail the write. * layout is best-effort polish, never a reason to fail the write.
*/ */
export async function applyElkLayout(inputXml: string): Promise<string> { export async function applyElkLayout(inputXml: string): Promise<string> {
const modelXml = normalizeInput(inputXml); const modelXml = normalizeInput(inputXml);
@@ -150,26 +205,14 @@ export async function applyElkLayout(inputXml: string): Promise<string> {
}; };
let laid: ElkGraph; let laid: ElkGraph;
let timer: ReturnType<typeof setTimeout> | undefined;
try { try {
// elkjs ships a CJS default export whose interop shape varies across // Run the (synchronous-under-the-hood) ELK pass on a worker thread so the
// module systems; resolve the real constructor at runtime, then cast (the // main event loop is never blocked, and enforce the wall-clock ceiling by
// runtime call is verified — see the layout unit test). // terminating that worker on timeout. A graph under the node/edge caps but
const Ctor: any = (ELK as any).default ?? ELK; // still pathologically slow is hard-killed instead of wedging anything.
const elk = new Ctor(); laid = await layoutInWorker(graph, resolveElkTimeoutMs());
// Race the layout against a wall-clock timeout so a graph that is under the
// node/edge caps but still pathologically slow can never wedge the server.
const timeout = new Promise<never>((_, reject) => {
timer = setTimeout(
() => reject(new Error("ELK layout timed out")),
ELK_TIMEOUT_MS,
);
});
laid = (await Promise.race([elk.layout(graph as any), timeout])) as ElkGraph;
} catch { } catch {
return modelXml; // best-effort: keep the model as-is on timeout or ELK failure return modelXml; // best-effort: keep the model as-is on timeout or ELK failure
} finally {
if (timer) clearTimeout(timer);
} }
// Collect computed geometry per node id (coords are parent-relative already). // Collect computed geometry per node id (coords are parent-relative already).
@@ -0,0 +1,36 @@
// Worker-thread entry for the ELK layered layout (issue #486, commit 1).
//
// elkjs' layout() returns a Promise but runs the actual crossing-minimisation
// SYNCHRONOUSLY — it blocks whatever thread it runs on for the whole pass. On
// the in-app MCP host that thread used to be the main NestJS event loop, so a
// pathological graph at the node/edge cap could wedge ALL HTTP/SSE/loopback
// traffic while it churned. Running it HERE, on a dedicated worker thread, keeps
// the main loop free; the parent enforces the wall-clock timeout by calling
// `worker.terminate()` — the only way to interrupt synchronous JS — since the
// in-process `setTimeout` race the parent used before could never fire while the
// same thread was blocked inside elkjs.
import { parentPort, workerData } from "node:worker_threads";
import ELK from "elkjs/lib/elk.bundled.js";
interface WorkerInput {
graph: unknown;
}
const { graph } = (workerData ?? {}) as WorkerInput;
// elkjs ships a CJS default export whose interop shape varies across module
// systems; resolve the real constructor at runtime (same as the parent did).
const Ctor: any = (ELK as any).default ?? ELK;
const elk = new Ctor();
elk
.layout(graph as any)
.then((laid: unknown) => {
parentPort?.postMessage({ ok: true, laid });
})
.catch((err: unknown) => {
parentPort?.postMessage({
ok: false,
error: err instanceof Error ? err.message : String(err),
});
});
+4 -3
View File
@@ -923,9 +923,10 @@ export const SHARED_TOOL_SPECS = {
'List the most recent pages (ordered by updatedAt, descending), ' + 'List the most recent pages (ordered by updatedAt, descending), ' +
'optionally scoped to a single space. Returns a bounded list (default ' + 'optionally scoped to a single space. Returns a bounded list (default ' +
'50, max 100) — use search for lookups in large spaces. tree:true (with ' + '50, max 100) — use search for lookups in large spaces. tree:true (with ' +
"spaceId) returns the space's full page hierarchy as a nested tree, but " + "spaceId) returns { tree, truncated } — the space's full page hierarchy " +
'is DEPRECATED — use getTree instead (leaner nodes, plus rootPageId / ' + 'as a nested tree, plus a `truncated` flag that is true when the tree was ' +
'maxDepth).', 'capped and is INCOMPLETE — but is DEPRECATED, use getTree instead ' +
'(leaner nodes, plus rootPageId / maxDepth).',
tier: 'core', tier: 'core',
catalogLine: catalogLine:
"listPages — list recent pages (tree:true is deprecated; use getTree for the hierarchy).", "listPages — list recent pages (tree:true is deprecated; use getTree for the hierarchy).",
@@ -0,0 +1,137 @@
// Unit tests for the collab-token reset on a Hocuspocus WS auth failure (#486).
//
// Before this fix the cached collab token (#435) was dropped ONLY on an HTTP
// 401/403 (the REST interceptor + login()); a rejected collab-WEBSOCKET handshake
// left the stale token in the cache, so every subsequent mutation re-presented
// the SAME bad token for up to the collab-token TTL (minutes) with no self-heal.
//
// The fix wraps collab writes in `writeWithCollabAuthRetry`: when the write
// rejects with the tagged collab-auth error (collab-session.ts's
// onAuthenticationFailed), it invalidates the cached token and retries the write
// ONCE with a force-refreshed token — symmetric to the HTTP-401 path.
//
// writeWithCollabAuthRetry / getCollabTokenWithReauth are protected in TS but
// plain methods on the compiled build, so the tests call them directly (same
// convention as collab-token-cache.test.mjs).
import { test, afterEach } from "node:test";
import assert from "node:assert/strict";
import { DocmostClient } from "../../build/client.js";
const ENV_KEY = "MCP_COLLAB_TOKEN_TTL_MS";
afterEach(() => {
delete process.env[ENV_KEY];
});
// A counting provider that returns a distinct token each call so a cached
// (reused) token is visibly the SAME string while a fresh mint is different.
function countingProvider() {
let n = 0;
const fn = async () => {
n++;
return `provider-token-${n}`;
};
return {
fn,
get calls() {
return n;
},
};
}
// The tagged error collab-session.ts throws on a rejected WS handshake.
function collabAuthError() {
const err = new Error("Authentication failed for collaboration connection");
err.collabAuthFailed = true;
return err;
}
test("a WS auth failure clears the cached token and retries the write with a FRESH one (#486)", async () => {
process.env[ENV_KEY] = "300000"; // 5 min: the cache is warm across the burst.
const p = countingProvider();
const client = new DocmostClient({
apiUrl: "http://127.0.0.1:1/api",
getToken: async () => "access",
getCollabToken: p.fn,
});
// Warm the cache the way a real write would (mints provider-token-1).
const initial = await client.getCollabTokenWithReauth();
assert.equal(initial, "provider-token-1");
assert.equal(p.calls, 1);
const tokensSeen = [];
const write = async (token) => {
tokensSeen.push(token);
// The FIRST attempt (with the stale cached token) fails the WS handshake;
// the retry (with a fresh token) succeeds.
if (tokensSeen.length === 1) throw collabAuthError();
return `written-with:${token}`;
};
const result = await client.writeWithCollabAuthRetry(initial, write);
assert.equal(tokensSeen.length, 2, "write attempted exactly twice (one retry)");
assert.equal(tokensSeen[0], "provider-token-1", "first attempt used the stale token");
assert.equal(
tokensSeen[1],
"provider-token-2",
"retry used a FRESH force-refreshed token, not the stale cached one",
);
assert.equal(result, "written-with:provider-token-2", "the retry's result wins");
assert.equal(p.calls, 2, "exactly one extra mint for the retry — no loop");
// The cache now holds the fresh token, so a subsequent op reuses it (proving
// the stale token was evicted and the fresh one cached, not re-minted).
const next = await client.getCollabTokenWithReauth();
assert.equal(next, "provider-token-2", "the fresh token replaced the stale cache");
assert.equal(p.calls, 2, "served from cache — provider not re-invoked");
});
test("a successful write is NOT retried and mints nothing extra", async () => {
process.env[ENV_KEY] = "300000";
const p = countingProvider();
const client = new DocmostClient({
apiUrl: "http://127.0.0.1:1/api",
getToken: async () => "access",
getCollabToken: p.fn,
});
const initial = await client.getCollabTokenWithReauth(); // provider-token-1
let attempts = 0;
const result = await client.writeWithCollabAuthRetry(initial, async (token) => {
attempts++;
return `ok:${token}`;
});
assert.equal(attempts, 1, "no retry on success");
assert.equal(result, "ok:provider-token-1");
assert.equal(p.calls, 1, "no extra mint");
});
test("a NON-auth write error propagates unchanged (no reset, no retry)", async () => {
process.env[ENV_KEY] = "300000";
const p = countingProvider();
const client = new DocmostClient({
apiUrl: "http://127.0.0.1:1/api",
getToken: async () => "access",
getCollabToken: p.fn,
});
const initial = await client.getCollabTokenWithReauth(); // provider-token-1
let attempts = 0;
await assert.rejects(
client.writeWithCollabAuthRetry(initial, async () => {
attempts++;
throw new Error("collab connection closed before persist"); // NOT tagged.
}),
/closed before persist/,
);
assert.equal(attempts, 1, "a non-auth error is not retried");
assert.equal(p.calls, 1, "the cache is untouched -> no fresh mint");
// Cache still holds the original token (was never invalidated).
const still = await client.getCollabTokenWithReauth();
assert.equal(still, "provider-token-1");
assert.equal(p.calls, 1);
});
@@ -0,0 +1,55 @@
// Unit test: listPages tree mode must propagate the `truncated` flag (#486).
//
// enumerateSpacePages returns { pages, truncated } — truncated is true ONLY when
// the stdio-fallback BFS hit its node cap (the primary /pages/tree path is
// uncapped). The old tree-mode listPages destructured only `pages` and returned a
// bare tree, dropping `truncated`, so a caller handed an INCOMPLETE tree had no
// way to know pages were missing. The fix returns { tree, truncated } (same
// pattern check_new_comments uses).
//
// Reaching the real cap (MAX_NODES = 10000) in a mock is impractical, so we stub
// enumerateSpacePages directly to assert the flag is threaded through verbatim.
import { test } from "node:test";
import assert from "node:assert/strict";
import { DocmostClient } from "../../build/client.js";
function stubClient() {
const client = new DocmostClient({
apiUrl: "http://127.0.0.1:1/api",
getToken: async () => "access",
});
// No network: the tree path only calls ensureAuthenticated + enumerateSpacePages.
client.ensureAuthenticated = async () => {};
return client;
}
const onePage = [{ id: "r1", title: "Root", parentPageId: null }];
test("tree mode carries truncated:true when the enumeration truncated (#486)", async () => {
const client = stubClient();
client.enumerateSpacePages = async () => ({ pages: onePage, truncated: true });
const res = await client.listPages("space-1", 50, true);
assert.equal(res.truncated, true, "the truncated flag is threaded through");
assert.ok(Array.isArray(res.tree), "the built tree rides alongside the flag");
assert.equal(res.tree[0].id, "r1");
});
test("tree mode carries truncated:false for a complete enumeration", async () => {
const client = stubClient();
client.enumerateSpacePages = async () => ({ pages: onePage, truncated: false });
const res = await client.listPages("space-1", 50, true);
assert.equal(res.truncated, false);
assert.equal(res.tree[0].id, "r1");
});
test("tree mode still requires a spaceId", async () => {
const client = stubClient();
await assert.rejects(
client.listPages(undefined, 50, true),
/tree mode requires a spaceId/,
);
});
@@ -184,12 +184,14 @@ test("enumerateSpacePages (via listPages tree) uses one /pages/tree request", as
}); });
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
// listPages tree:true -> enumerateSpacePages(spaceId) -> buildPageTree. // listPages tree:true -> enumerateSpacePages(spaceId) -> { tree, truncated }.
const tree = await client.listPages("space-1", 50, true); const { tree, truncated } = await client.listPages("space-1", 50, true);
assert.equal(treeRequests, 1, "exactly one /pages/tree request for the space"); assert.equal(treeRequests, 1, "exactly one /pages/tree request for the space");
assert.equal(sidebarRequests, 0, "no per-node sidebar BFS requests"); assert.equal(sidebarRequests, 0, "no per-node sidebar BFS requests");
assert.deepEqual(treeBody, { spaceId: "space-1" }, "space scope posts spaceId only"); assert.deepEqual(treeBody, { spaceId: "space-1" }, "space scope posts spaceId only");
// The uncapped /pages/tree path is never truncated (#486).
assert.equal(truncated, false, "primary /pages/tree path is not truncated");
// buildPageTree nests c1 under r1; two roots at the top level. // buildPageTree nests c1 under r1; two roots at the top level.
assert.equal(tree.length, 2, "two root nodes"); assert.equal(tree.length, 2, "two root nodes");
const r1 = tree.find((n) => n.id === "r1"); const r1 = tree.find((n) => n.id === "r1");
@@ -249,7 +251,7 @@ test("enumerateSpacePages falls back to the cursor BFS on /pages/tree 404", asyn
}); });
const client = new DocmostClient(baseURL, "user@example.com", "pw"); const client = new DocmostClient(baseURL, "user@example.com", "pw");
const tree = await client.listPages("space-1", 50, true); const { tree, truncated } = await client.listPages("space-1", 50, true);
assert.ok(treeRequests >= 1, "the tree endpoint was attempted first"); assert.ok(treeRequests >= 1, "the tree endpoint was attempted first");
assert.deepEqual( assert.deepEqual(
@@ -257,6 +259,8 @@ test("enumerateSpacePages falls back to the cursor BFS on /pages/tree 404", asyn
["<root>", "r1"], ["<root>", "r1"],
"fell back to the sidebar BFS: roots then the root's children", "fell back to the sidebar BFS: roots then the root's children",
); );
// Small fallback walk well under the node cap -> not truncated (#486).
assert.equal(truncated, false, "fallback BFS below the cap is not truncated");
assert.equal(tree.length, 1, "one root in the built tree"); assert.equal(tree.length, 1, "one root in the built tree");
assert.equal(tree[0].children[0].id, "c1", "leaf nested via the BFS"); assert.equal(tree[0].children[0].id, "c1", "leaf nested via the BFS");
}); });
@@ -101,6 +101,88 @@ test("DoS guard: a graph over the node cap is returned unchanged, quickly", asyn
assert.ok(dt < 2000, `cap path should be fast, took ${dt}ms`); assert.ok(dt < 2000, `cap path should be fast, took ${dt}ms`);
}); });
/** Build a layered DAG near the caps: `n` vertices, up to ~2 edges each into the
* next layer of `layerSize`. Used as a real worst-case graph for the benchmark. */
function layeredGraph(n, layerSize) {
let cells = "";
for (let i = 2; i < 2 + n; i++) {
cells +=
`<mxCell id="${i}" value="N${i}" style="rounded=1;html=1;" vertex="1" parent="1">` +
`<mxGeometry x="10" y="10" width="120" height="60" as="geometry"/></mxCell>`;
}
let ei = 0;
for (let i = 2; i < 2 + n; i++) {
for (const off of [layerSize, layerSize + 1]) {
const t = i + off;
if (t < 2 + n) cells += `<mxCell id="e${ei++}" edge="1" parent="1" source="${i}" target="${t}"><mxGeometry relative="1" as="geometry"/></mxCell>`;
}
}
return (
'<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/>' +
cells +
"</root></mxGraphModel>"
);
}
test("terminate-on-timeout: a layout that exceeds the wall-clock ceiling is hard-killed and the original model is returned (#486)", async () => {
// A 1ms ceiling fires before the worker can even finish loading elkjs, so the
// parent must terminate() the worker and fall back to the ORIGINAL model. On
// the OLD in-process race this timer could never fire while the SAME thread was
// blocked inside elkjs — the fallback path was unreachable; here it works.
const prev = process.env.DRAWIO_ELK_TIMEOUT_MS;
process.env.DRAWIO_ELK_TIMEOUT_MS = "1";
try {
const model = layeredGraph(400, 20);
const t0 = Date.now();
const laid = await applyElkLayout(model);
const dt = Date.now() - t0;
// Original geometry is preserved verbatim: every vertex is still stacked at
// (10,10), proving NO ELK coordinates were applied (the pass was killed).
const verts = parseCells(laid).filter((c) => c.vertex);
assert.equal(verts.length, 400, "all vertices survived the fallback");
for (const v of verts) {
assert.equal(v.geometry.x, 10, "x untouched -> layout was terminated");
assert.equal(v.geometry.y, 10, "y untouched -> layout was terminated");
}
// The kill is prompt: terminate() returns the call well under the natural
// layout time for a 400-node graph.
assert.ok(dt < 2000, `terminate path should be prompt, took ${dt}ms`);
} finally {
if (prev === undefined) delete process.env.DRAWIO_ELK_TIMEOUT_MS;
else process.env.DRAWIO_ELK_TIMEOUT_MS = prev;
}
});
test("benchmark guard: a worst-case graph AT the cap lays out without wedging the main event loop (#486)", async () => {
// ~500 nodes / ~1000 edges — a real worst case at the node/edge caps. The
// layout runs on a WORKER thread, so the MAIN event loop must stay responsive
// throughout: a timer scheduled on the main thread keeps firing while ELK
// churns. On the OLD synchronous-on-main-thread code this counter would be
// pinned at 0 for the whole layout (event loop wedged) — exactly the prod fire.
const model = layeredGraph(500, 20);
let mainLoopTicks = 0;
const iv = setInterval(() => {
mainLoopTicks++;
}, 2);
const t0 = Date.now();
const laid = await applyElkLayout(model);
const dt = Date.now() - t0;
clearInterval(iv);
assert.ok(
mainLoopTicks > 0,
"main event loop must stay responsive while ELK runs on the worker",
);
// Benchmark guard: the worst-case graph actually LAYS OUT within the default
// ceiling (it did not fall back). At least one vertex moved off the stack.
const verts = parseCells(laid).filter((c) => c.vertex);
assert.equal(verts.length, 500, "all vertices survived");
const moved = verts.some((v) => v.geometry.x !== 10 || v.geometry.y !== 10);
assert.ok(moved, "layout was applied (did not time out / fall back)");
// Sanity ceiling well under the 5s wall-clock timeout.
assert.ok(dt < 5000, `worst-case layout should be under the ceiling, took ${dt}ms`);
});
test("layout is best-effort: an empty/degenerate model is returned intact", async () => { test("layout is best-effort: an empty/degenerate model is returned intact", async () => {
const model = const model =
'<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/></root></mxGraphModel>'; '<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/></root></mxGraphModel>';
+194 -75
View File
@@ -1,101 +1,220 @@
import { test } from "node:test"; import { test } from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { createHash } from "node:crypto"; import {
import { readFileSync } from "node:fs"; mkdtempSync,
mkdirSync,
writeFileSync,
rmSync,
readdirSync,
statSync,
readFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { fileURLToPath } from "node:url"; import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path"; import { dirname, join, relative, sep } from "node:path";
import { createHash } from "node:crypto";
import { computeRegistryStamp } from "../../scripts/gen-registry-stamp.mjs"; import { computeRegistryStamp } from "../../scripts/gen-registry-stamp.mjs";
import { REGISTRY_STAMP } from "../../build/index.js"; import { REGISTRY_STAMP } from "../../build/index.js";
// Guard tests for the build/src-skew stamp (issue #447). The codegen script // Guard tests for the build/src-skew stamp (issues #447/#486). The codegen script
// exports `computeRegistryStamp(sourceText)` — a sha256 over normalized source // exports `computeRegistryStamp(srcDir)` — a sha256 over the WHOLE src/ tree
// text (CRLF->LF, single trailing newline stripped). The in-app loader // (every src/**/*.ts EXCEPT *.generated.ts), each file folded in as its
// (apps/server/.../docmost-client.loader.ts) DUPLICATES that normalize+sha256 to // POSIX-relative path + its normalized content (CRLF->LF, single trailing newline
// recompute the stamp from src and refuse a stale build. These tests pin the // stripped). Hashing the whole tree (not just tool-specs.ts) is #486: an edit to
// algorithm's behaviour AND assert the built stamp matches the current src, so a // client.ts / a client/* module without a rebuild must ALSO redden. The in-app
// stale generated file OR a normalize divergence reddens. // loader (apps/server/.../docmost-client.loader.ts) DUPLICATES this enumerate+
// normalize+sha256 to refuse a stale build. These tests pin the algorithm and
// assert the built stamp matches the current src.
const __dirname = dirname(fileURLToPath(import.meta.url)); const __dirname = dirname(fileURLToPath(import.meta.url));
const TOOL_SPECS_PATH = join(__dirname, "..", "..", "src", "tool-specs.ts"); const SRC_DIR = join(__dirname, "..", "..", "src");
test("computeRegistryStamp is deterministic: same input -> same hash", () => { // Build a throwaway src/ tree from a { relPath: content } map and return its dir.
const input = "export const X = 1;\nexport const Y = 2;\n"; function makeSrcTree(files) {
assert.equal(computeRegistryStamp(input), computeRegistryStamp(input)); const root = mkdtempSync(join(tmpdir(), "mcp-stamp-tree-"));
const src = join(root, "src");
for (const [rel, content] of Object.entries(files)) {
const full = join(src, rel);
mkdirSync(dirname(full), { recursive: true });
writeFileSync(full, content, "utf8");
}
return { src, cleanup: () => rmSync(root, { recursive: true, force: true }) };
}
test("computeRegistryStamp is deterministic: same tree -> same hash", () => {
const a = makeSrcTree({ "tool-specs.ts": "export const X = 1;\n" });
const b = makeSrcTree({ "tool-specs.ts": "export const X = 1;\n" });
try {
assert.equal(computeRegistryStamp(a.src), computeRegistryStamp(b.src));
} finally {
a.cleanup();
b.cleanup();
}
}); });
test("computeRegistryStamp returns a 64-char lowercase hex sha256", () => { test("computeRegistryStamp returns a 64-char lowercase hex sha256", () => {
const stamp = computeRegistryStamp("anything"); const t = makeSrcTree({ "tool-specs.ts": "anything\n" });
assert.match(stamp, /^[0-9a-f]{64}$/); try {
assert.match(computeRegistryStamp(t.src), /^[0-9a-f]{64}$/);
} finally {
t.cleanup();
}
}); });
test("normalizes CRLF vs LF: the same content hashes equal", () => { // #486 CORE: an edit to a NON-tool-specs source file (client.ts) must change the
const lf = "line1\nline2\nline3"; // stamp. Under the old single-file (tool-specs.ts only) hash this edit was
const crlf = "line1\r\nline2\r\nline3"; // invisible and a stale build/ served the old client.ts silently.
assert.equal(computeRegistryStamp(crlf), computeRegistryStamp(lf)); test("editing client.ts (not tool-specs.ts) changes the stamp (#486)", () => {
const before = makeSrcTree({
"tool-specs.ts": "export const SPECS = 1;\n",
"client.ts": "export const impl = 'v1';\n",
}); });
const after = makeSrcTree({
test("normalizes a trailing newline: with/without a final \\n hashes equal", () => { "tool-specs.ts": "export const SPECS = 1;\n",
const noTrailing = "alpha\nbeta"; "client.ts": "export const impl = 'v2';\n",
const trailing = "alpha\nbeta\n";
assert.equal(computeRegistryStamp(trailing), computeRegistryStamp(noTrailing));
}); });
try {
test("a CRLF checkout WITH a trailing CRLF still hashes equal to bare LF", () => {
// A worst-case Windows checkout: CRLF line endings + a trailing CRLF. Both the
// \r\n->\n replace and the trailing-newline strip must apply for parity.
const bare = "alpha\nbeta";
const crlfTrailing = "alpha\r\nbeta\r\n";
assert.equal(
computeRegistryStamp(crlfTrailing),
computeRegistryStamp(bare),
);
});
test("a real content change hashes differently", () => {
const before = "export const description = 'search a page';\n";
const after = "export const description = 'search a PAGE';\n";
assert.notEqual(computeRegistryStamp(before), computeRegistryStamp(after));
});
// Only a SINGLE trailing newline is stripped — a second blank line is content and
// must change the hash. This pins the exact `/\n$/` semantics the loader mirrors.
test("only ONE trailing newline is stripped (two differ from one)", () => {
assert.notEqual( assert.notEqual(
computeRegistryStamp("x\n"), computeRegistryStamp(before.src),
computeRegistryStamp("x\n\n"), computeRegistryStamp(after.src),
"a client.ts edit with an unchanged tool-specs.ts must move the stamp",
); );
} finally {
before.cleanup();
after.cleanup();
}
}); });
// Cross-impl equality against a fixed, documented input. The SAME literal input test("editing a nested client/* module changes the stamp", () => {
// and expected hash are asserted in the server-side jest test const before = makeSrcTree({
// (docmost-client.loader.spec.ts). If either side's normalize+sha256 ever "tool-specs.ts": "x\n",
// diverges, one of the two tests reddens. Input exercises BOTH normalize steps. "client/read.ts": "export const READ = 1;\n",
test("fixed-input hash matches the documented cross-impl value", () => { });
const FIXED_INPUT = "line1\r\nline2\n"; const after = makeSrcTree({
const EXPECTED = "tool-specs.ts": "x\n",
"683376e290829b482c2655745caffa7a1dccfa10afaa62dac2b42dd6c68d0f83"; "client/read.ts": "export const READ = 2;\n",
assert.equal(computeRegistryStamp(FIXED_INPUT), EXPECTED); });
try {
assert.notEqual(
computeRegistryStamp(before.src),
computeRegistryStamp(after.src),
);
} finally {
before.cleanup();
after.cleanup();
}
}); });
// DESYNC GUARD (covers reviewer suggestion 2). Recompute the stamp from the // *.generated.ts is EXCLUDED (else the codegen's own output is a fixed-point
// actual src/tool-specs.ts and assert it equals the REGISTRY_STAMP baked into the // cycle): adding/removing/changing it must not move the stamp.
// freshly-built build/index.js. This reddens if the generated file is stale OR if test("*.generated.ts is excluded from the stamp", () => {
// the codegen normalize ever diverges from what produced the built stamp. const without = makeSrcTree({ "tool-specs.ts": "x\n" });
test("built REGISTRY_STAMP equals the stamp recomputed from src/tool-specs.ts", () => { const withGen = makeSrcTree({
const source = readFileSync(TOOL_SPECS_PATH, "utf8"); "tool-specs.ts": "x\n",
assert.equal(computeRegistryStamp(source), REGISTRY_STAMP); "registry-stamp.generated.ts": 'export const REGISTRY_STAMP = "abc";\n',
});
try {
assert.equal(
computeRegistryStamp(without.src),
computeRegistryStamp(withGen.src),
"a *.generated.ts file must not affect the stamp",
);
} finally {
without.cleanup();
withGen.cleanup();
}
}); });
// Sanity: the fixed-input helper computes the SAME way the codegen does, proving test("a CRLF checkout WITH trailing CRLF hashes equal to bare LF", () => {
// the EXPECTED constant above is not an arbitrary magic value but the documented const bare = makeSrcTree({ "tool-specs.ts": "alpha\nbeta" });
// normalize+sha256 of FIXED_INPUT. Belt-and-braces so a bad EXPECTED can't hide a const crlfTrailing = makeSrcTree({ "tool-specs.ts": "alpha\r\nbeta\r\n" });
// real regression. try {
test("the documented EXPECTED constant is the normalize+sha256 of FIXED_INPUT", () => { assert.equal(
const FIXED_INPUT = "line1\r\nline2\n"; computeRegistryStamp(crlfTrailing.src),
const normalized = FIXED_INPUT.replace(/\r\n/g, "\n").replace(/\n$/, ""); computeRegistryStamp(bare.src),
const expected = createHash("sha256") );
.update(normalized, "utf8") } finally {
.digest("hex"); bare.cleanup();
assert.equal(computeRegistryStamp(FIXED_INPUT), expected); crlfTrailing.cleanup();
}
});
// Only a SINGLE trailing newline is stripped — a second blank line is content.
test("only ONE trailing newline is stripped (two differ from one)", () => {
const one = makeSrcTree({ "tool-specs.ts": "x\n" });
const two = makeSrcTree({ "tool-specs.ts": "x\n\n" });
try {
assert.notEqual(
computeRegistryStamp(one.src),
computeRegistryStamp(two.src),
);
} finally {
one.cleanup();
two.cleanup();
}
});
// Cross-impl equality against a fixed, documented tree. The SAME literal tree and
// expected hash are asserted in the server-side jest test
// (docmost-client.loader.spec.ts). If either side's enumerate+normalize+sha256
// ever diverges, one of the two tests reddens. The tree exercises: a nested file,
// BOTH normalize steps (tool-specs.ts uses CRLF + trailing \n) and the
// *.generated.ts exclusion.
const CROSS_IMPL_TREE = {
"tool-specs.ts": "line1\r\nline2\n",
"client/read.ts": "export const R = 1;\n",
"registry-stamp.generated.ts": 'export const REGISTRY_STAMP="ignored";\n',
};
const CROSS_IMPL_EXPECTED =
"131c1b9e4e2f5a7d6cef91ca8df619822b442f52bc45ebd09474a4c1d6728616";
test("fixed-tree hash matches the documented cross-impl value", () => {
const t = makeSrcTree(CROSS_IMPL_TREE);
try {
assert.equal(computeRegistryStamp(t.src), CROSS_IMPL_EXPECTED);
} finally {
t.cleanup();
}
});
// Sanity: the EXPECTED constant is not a magic value but the documented
// enumerate+normalize+sha256 of CROSS_IMPL_TREE (a local re-implementation).
test("the documented EXPECTED is the enumerate+normalize+sha256 of the tree", () => {
const t = makeSrcTree(CROSS_IMPL_TREE);
try {
const collect = (dir) => {
const out = [];
for (const e of readdirSync(dir)) {
const f = join(dir, e);
if (statSync(f).isDirectory()) out.push(...collect(f));
else if (e.endsWith(".ts") && !e.endsWith(".generated.ts")) out.push(f);
}
return out;
};
const files = collect(t.src)
.map((abs) => ({ rel: relative(t.src, abs).split(sep).join("/"), abs }))
.sort((a, b) => (a.rel < b.rel ? -1 : a.rel > b.rel ? 1 : 0));
const h = createHash("sha256");
for (const { rel, abs } of files) {
const n = readFileSync(abs, "utf8")
.replace(/\r\n/g, "\n")
.replace(/\n$/, "");
h.update(rel, "utf8");
h.update("\0", "utf8");
h.update(n, "utf8");
h.update("\0", "utf8");
}
assert.equal(h.digest("hex"), CROSS_IMPL_EXPECTED);
} finally {
t.cleanup();
}
});
// DESYNC GUARD. Recompute the stamp from the REAL src/ tree and assert it equals
// the REGISTRY_STAMP baked into the freshly-built build/index.js. This reddens if
// the generated file is stale OR if the codegen ever diverges from what produced
// the built stamp.
test("built REGISTRY_STAMP equals the stamp recomputed from src/", () => {
assert.equal(computeRegistryStamp(SRC_DIR), REGISTRY_STAMP);
}); });
+114 -6
View File
@@ -1,8 +1,62 @@
diff --git a/dist/index.js b/dist/index.js diff --git a/dist/index.js b/dist/index.js
index ae447a12f7823ec0a00837ee9f0eb809a610d5f8..a3402b2c2d021ef432cfa76e35d370073d525135 100644 index ae447a12f7823ec0a00837ee9f0eb809a610d5f8..210b5a9009e5cf1537cb2007c524007c1a01253c 100644
--- a/dist/index.js --- a/dist/index.js
+++ b/dist/index.js +++ b/dist/index.js
@@ -6578,9 +6578,19 @@ function createOutputTransformStream(output) { @@ -5036,9 +5036,40 @@ function writeToServerResponse({
break;
const canContinue = response.write(value);
if (!canContinue) {
- await new Promise((resolve3) => {
- response.once("drain", resolve3);
+ // PATCH(docmost #486): race "drain" against "close"/"error". The
+ // original awaited ONLY "drain", so a client that disconnected mid-write
+ // (the socket never drains) parked this loop FOREVER: the finally never
+ // ran, response.end() was unreachable, and the reader + buffered chunks
+ // were held until process restart. On close/error we cancel the reader
+ // and break so the finally always runs (safe for detached runs:
+ // consumeStream drains the SDK stream independently). Listener hygiene:
+ // all three once-listeners are removed on the first settle, so they
+ // cannot pile up one-per-stall.
+ const closed = await new Promise((resolve3) => {
+ function finish(isClosed) {
+ response.removeListener("drain", onDrain);
+ response.removeListener("close", onClose);
+ response.removeListener("error", onError);
+ resolve3(isClosed);
+ }
+ function onDrain() {
+ finish(false);
+ }
+ function onClose() {
+ finish(true);
+ }
+ function onError() {
+ finish(true);
+ }
+ response.once("drain", onDrain);
+ response.once("close", onClose);
+ response.once("error", onError);
});
+ if (closed) {
+ await reader.cancel().catch(() => {
+ });
+ break;
+ }
}
}
} catch (error) {
@@ -5047,7 +5078,9 @@ function writeToServerResponse({
response.end();
}
};
- read();
+ read().catch((error) => {
+ console.error("ai writeToServerResponse read() failed:", error);
+ });
}
// src/text-stream/pipe-text-stream-to-response.ts
@@ -6578,9 +6611,19 @@ function createOutputTransformStream(output) {
controller.enqueue({ part: chunk, partialOutput: void 0 }); controller.enqueue({ part: chunk, partialOutput: void 0 });
return; return;
} }
@@ -23,7 +77,7 @@ index ae447a12f7823ec0a00837ee9f0eb809a610d5f8..a3402b2c2d021ef432cfa76e35d37007
const result = await output.parsePartialOutput({ text: text2 }); const result = await output.parsePartialOutput({ text: text2 });
if (result !== void 0) { if (result !== void 0) {
const currentJson = JSON.stringify(result.partial); const currentJson = JSON.stringify(result.partial);
@@ -6959,7 +6969,7 @@ var DefaultStreamTextResult = class { @@ -6959,7 +7002,7 @@ var DefaultStreamTextResult = class {
}) })
); );
} }
@@ -33,10 +87,64 @@ index ae447a12f7823ec0a00837ee9f0eb809a610d5f8..a3402b2c2d021ef432cfa76e35d37007
maxRetries: maxRetriesArg, maxRetries: maxRetriesArg,
abortSignal abortSignal
diff --git a/dist/index.mjs b/dist/index.mjs diff --git a/dist/index.mjs b/dist/index.mjs
index 663875332e3f9a9bd167c25583c515876f42951b..b840b0502c9894df983e0154805abb80e70e6331 100644 index 663875332e3f9a9bd167c25583c515876f42951b..a51514a390d22811a407edd8b703e21793586cc8 100644
--- a/dist/index.mjs --- a/dist/index.mjs
+++ b/dist/index.mjs +++ b/dist/index.mjs
@@ -6501,9 +6501,19 @@ function createOutputTransformStream(output) { @@ -4957,9 +4957,40 @@ function writeToServerResponse({
break;
const canContinue = response.write(value);
if (!canContinue) {
- await new Promise((resolve3) => {
- response.once("drain", resolve3);
+ // PATCH(docmost #486): race "drain" against "close"/"error". The
+ // original awaited ONLY "drain", so a client that disconnected mid-write
+ // (the socket never drains) parked this loop FOREVER: the finally never
+ // ran, response.end() was unreachable, and the reader + buffered chunks
+ // were held until process restart. On close/error we cancel the reader
+ // and break so the finally always runs (safe for detached runs:
+ // consumeStream drains the SDK stream independently). Listener hygiene:
+ // all three once-listeners are removed on the first settle, so they
+ // cannot pile up one-per-stall.
+ const closed = await new Promise((resolve3) => {
+ function finish(isClosed) {
+ response.removeListener("drain", onDrain);
+ response.removeListener("close", onClose);
+ response.removeListener("error", onError);
+ resolve3(isClosed);
+ }
+ function onDrain() {
+ finish(false);
+ }
+ function onClose() {
+ finish(true);
+ }
+ function onError() {
+ finish(true);
+ }
+ response.once("drain", onDrain);
+ response.once("close", onClose);
+ response.once("error", onError);
});
+ if (closed) {
+ await reader.cancel().catch(() => {
+ });
+ break;
+ }
}
}
} catch (error) {
@@ -4968,7 +4999,9 @@ function writeToServerResponse({
response.end();
}
};
- read();
+ read().catch((error) => {
+ console.error("ai writeToServerResponse read() failed:", error);
+ });
}
// src/text-stream/pipe-text-stream-to-response.ts
@@ -6501,9 +6534,19 @@ function createOutputTransformStream(output) {
controller.enqueue({ part: chunk, partialOutput: void 0 }); controller.enqueue({ part: chunk, partialOutput: void 0 });
return; return;
} }
@@ -57,7 +165,7 @@ index 663875332e3f9a9bd167c25583c515876f42951b..b840b0502c9894df983e0154805abb80
const result = await output.parsePartialOutput({ text: text2 }); const result = await output.parsePartialOutput({ text: text2 });
if (result !== void 0) { if (result !== void 0) {
const currentJson = JSON.stringify(result.partial); const currentJson = JSON.stringify(result.partial);
@@ -6882,7 +6892,7 @@ var DefaultStreamTextResult = class { @@ -6882,7 +6925,7 @@ var DefaultStreamTextResult = class {
}) })
); );
} }
+6 -6
View File
@@ -48,7 +48,7 @@ patchedDependencies:
hash: d8dc66e5ec3b9d23a876b979f493b6aa901fd2d965be54729495da5136296a42 hash: d8dc66e5ec3b9d23a876b979f493b6aa901fd2d965be54729495da5136296a42
path: patches/@hocuspocus__server@3.4.4.patch path: patches/@hocuspocus__server@3.4.4.patch
ai@6.0.134: ai@6.0.134:
hash: f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9 hash: e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b
path: patches/ai@6.0.134.patch path: patches/ai@6.0.134.patch
scimmy@1.3.5: scimmy@1.3.5:
hash: 775d80f86830b2c5dd1a250c9802c10f8fc3da3c7898373de5aa0c23993d1673 hash: 775d80f86830b2c5dd1a250c9802c10f8fc3da3c7898373de5aa0c23993d1673
@@ -644,10 +644,10 @@ importers:
version: 8.3.0(socket.io-adapter@2.5.4) version: 8.3.0(socket.io-adapter@2.5.4)
ai: ai:
specifier: ^6.0.134 specifier: ^6.0.134
version: 6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6) version: 6.0.134(patch_hash=e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b)(zod@4.3.6)
ai-sdk-ollama: ai-sdk-ollama:
specifier: ^3.8.1 specifier: ^3.8.1
version: 3.8.1(ai@6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6))(zod@4.3.6) version: 3.8.1(ai@6.0.134(patch_hash=e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b)(zod@4.3.6))(zod@4.3.6)
bcrypt: bcrypt:
specifier: ^6.0.0 specifier: ^6.0.0
version: 6.0.0 version: 6.0.0
@@ -16455,17 +16455,17 @@ snapshots:
agent-base@7.1.4: {} agent-base@7.1.4: {}
ai-sdk-ollama@3.8.1(ai@6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6))(zod@4.3.6): ai-sdk-ollama@3.8.1(ai@6.0.134(patch_hash=e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b)(zod@4.3.6))(zod@4.3.6):
dependencies: dependencies:
'@ai-sdk/provider': 3.0.8 '@ai-sdk/provider': 3.0.8
'@ai-sdk/provider-utils': 4.0.21(zod@4.3.6) '@ai-sdk/provider-utils': 4.0.21(zod@4.3.6)
ai: 6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6) ai: 6.0.134(patch_hash=e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b)(zod@4.3.6)
jsonrepair: 3.13.3 jsonrepair: 3.13.3
ollama: 0.6.3 ollama: 0.6.3
transitivePeerDependencies: transitivePeerDependencies:
- zod - zod
ai@6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6): ai@6.0.134(patch_hash=e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b)(zod@4.3.6):
dependencies: dependencies:
'@ai-sdk/gateway': 3.0.77(zod@4.3.6) '@ai-sdk/gateway': 3.0.77(zod@4.3.6)
'@ai-sdk/provider': 3.0.8 '@ai-sdk/provider': 3.0.8