Compare commits

...

73 Commits

Author SHA1 Message Date
vvzvlad 220142bef6 Merge pull request 'fix(page-tree/dnd): drop-раскрытие + guard от потери детей (#523)' (#532) from fix/523-tree-dnd into fix/525-insert-unloaded
Reviewed-on: #532
2026-07-12 05:08:42 +03:00
agent_coder 0af7eb30c3 docs(page-tree): correct isUnloadedBranch comment — no DnD-guard consumer (#525 review)
The comment falsely listed a 'DnD move guard' consumer; no DnD path routes
through the predicate (local DnD/create-page use the raw index-based insert).
List the real consumers (handleToggle + realtime insertByPosition/placeByPosition)
and note the local raw-insert path as a #525 follow-up. Comment-only.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 04:13:49 +03:00
vvzvlad 9a94c8df18 Merge pull request 'perf(ai-chat): дисциплина данных — дедуп tool-outputs + токен-бюджет реплея (#490)' (#510) from feat/490-data-discipline into develop
Reviewed-on: #510
2026-07-12 04:11:08 +03:00
vvzvlad 8abde99611 Merge pull request 'ci: гейты наблюдаемых свойств перед publish — image-smoke, migration-order на push, allowlist fail-closed, property-тесты (#476)' (#477) from ci/476-publish-gates into develop
Reviewed-on: #477
2026-07-12 04:07:24 +03:00
vvzvlad d608311cae Merge pull request 'fix(ui/toasts): глобальная видимость (тон+рамка+тень) + перенос в top-center без перекрытия (#517)' (#528) from fix/517-toasts into develop
Reviewed-on: #528
2026-07-12 04:06:41 +03:00
agent_coder 6dad309b51 fix(ai-chat): ревью #510 — Опция A бюджета + 4 DO ревьюера
Эскалация (владелец) — Опция A: 100k только фолбэк для НЕсконфигурированных
инсталляций; при заданном chatContextWindow бюджет = floor(0.7×window) БЕЗ капа
(бюджетер — защита от брика об контекст-окно, не эконом-лимитер). Спек репиннут
resolveReplayBudget(1_000_000)→700_000.

DO1: агрессивный next-turn recovery ×0.5 вынесен в чистую resolveEffectiveReplayThreshold
+ тест линковки replayOverflow→0.5×бюджет (mutation-verified).
DO2: checkNewComments partial-failure — per-page reject скипается (→null), скан
резолвится, порядок выживших сохранён; тест #7 (mutation-verified).
DO3: ai-chat.write-volume.spec.ts → .int-spec.ts (WAL-гард не бежал НИ в одном
CI-lane) + маппер @docmost/token-estimate в jest-integration.json; реальный WAL
на pg:5432 зелёный (трейс v1 140MB→v2 0.04MB).
DO4: CHANGELOG [Unreleased] по #490.
Follow-up: issue #520 (эскалация агрессивной доли при незаданном окне + малом
реальном контексте).

Ребейзнут на develop (волна 1 смержена): только 6 коммитов #490 над develop.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 04:06:24 +03:00
agent_coder f34542c881 perf(mcp): checkNewComments — параллелизм с капом (#490)
checkNewComments делал O(N) последовательных REST-вызовов listComments по страницам
working set — большой space линеен по round-trip'ам. Теперь per-page фетчи идут с
ограниченным параллелизмом (cap 6, середина полосы 5–8): независимые чтения не ждут
друг друга, но и не заваливают сервер/сокеты.

mapWithConcurrency — крошечный пул без зависимости от p-limit: N воркеров тянут
следующий индекс с общего курсора. Порядок результатов сохраняется (по входному
порядку страниц), поэтому вывод детерминирован независимо от того, какой фетч
завершился первым. Серверный batch-эндпоинт «comments updated since T по space» —
опционально, отдельным заходом.

Тест (mock-HTTP): 13 страниц, задержанный /api/comments — maxInFlight > 1 и <= 6
(последовательная реализация дала бы 1), порядок результатов = порядок обхода.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 04:05:48 +03:00
agent_coder b038d96708 perf(ai-chat): snapshotOpenPage fast-path (#490)
snapshotOpenPage делал полный экспорт Markdown + upsert каждый ход. Fast-path: если
снапшот уже существует на ТЕКУЩЕЙ версии страницы (тот же instant updated_at), его
контент уже актуален — пропускаем экспорт+upsert целиком. Ход, не тронувший
открытую страницу (частый случай), больше не делает работы по снапшоту.

Зеркалит read-side fast-path в detectPageChange (sameInstant): оба доверяют, что
правка страницы двигает updated_at. Когда агент/человек ПРАВИЛ страницу этим ходом,
updated_at продвинулся → не совпадает → экспортируем как раньше (правки агента
запекаются в снапшот, инвариант #274 сохранён).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 04:05:48 +03:00
agent_coder f5bbfdb2d4 perf(ai-chat): deferred-активация тулов в metadata чата (#490)
Активированный set сбрасывался каждый ход → модель заново гоняла loadTools, чтобы
переактивировать те же тулы (лишний round-trip на каждом ходу). Теперь набор
персистится в metadata чата и сидируется на следующем ходу.

- Миграция: jsonb-колонка metadata на ai_chats (default '{}'); db.d.ts дополнен
  вручную (AiChats.metadata: Generated<Json>).
- seedActivatedTools(metadata, validDeferredNames): читает сохранённый набор,
  ПЕРЕСЕКАЯ с актуальными validDeferredNames — смена allowlist/ролей не воскресит
  несуществующий тул (иначе prepareAgentStep получил бы фантомное активное имя).
  Сид только при deferredEnabled.
- Персист на завершении хода (once-guard, во всех терминальных ветках рядом со
  snapshotTurnEnd): детерминированно отсортированный набор, merge в существующий
  bag (другие ключи сохраняются), запись пропускается если ничего нового не
  активировано (обычный ход не даёт лишней записи).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 04:05:48 +03:00
agent_coder 875f6ba9c5 feat(ai-chat): токен-бюджет реплея истории + реактивная ветка (#490)
Вся персистентная история реплеится провайдеру КАЖДЫЙ ход, поэтому длинный чат
рано или поздно упирается в контекстное окно и получает провайдерский 400 на
каждом ходу — навсегда (чат «кирпичится»). Бюджетер ограничивает РЕПЛЕЙ (никогда
не мутирует персист — в БД остаётся полная запись), детерминированно и byte-stable
(обрезанный префикс идентичен от хода к ходу → дружелюбно к prompt-cache).

Единый оценщик chars/2.5 (кириллица; chars/4 занижает вдвое) вынесен в shared-пакет
packages/token-estimate; клиентский count-stream-tokens.ts переведён на него ТЕМ ЖЕ
коммитом (два расходящихся оценщика = «бейдж 60%, а бюджет уже режет»).

history-budget.ts (чистый, покрыт тестами):
- resolveReplayBudget(raw): min(100k, 0.7×window) при заданном окне; флэт 100k при
  незаданном (именно эти инсталляции ловят терминальный overflow — warn-лог); 0 =
  явный off-switch. Читается СЫРОЙ chatContextWindow, т.к. parsePositiveInt схлопывает
  0 и unset в undefined (новое поле ResolvedAiConfig.chatContextWindowRaw).
- trimHistoryForReplay: первичный сигнал — провайдерский факт metadata.contextTokens
  прошлого хода; chars-оценка — дельта/раскройка/фолбэк. Порядок: обрезка tool-outputs
  старых ходов (head+tail+маркер) → механическое схлопывание старейших ходов
  (конкатенация, НЕ LLM) → текущий + последние N ходов всегда полные. Пейринг
  tool-call/result сохраняется (схлопывание убирает ОБЕ части).
- isContextOverflowError: классификация провайдерского 400 (статус + паттерны).

Реактивная ветка: превентивная оценка не даёт инварианта (первый переполняющий ход
не имеет usage). onError классифицирует context-overflow → пишет различимую причину
и штампует metadata.replayOverflow; следующий ход бюджетер режет агрессивно
(0.5×), что и раскирпичивает чат. Наблюдаемость: metadata.replayTrimmedToTokens.

ПРИМЕЧАНИЕ по реактивной ветке (форк, требует решения ревьюера): истинный in-turn
re-pipe (перезапуск streamText в тот же ответ) архитектурно несовместим с текущим
пайпом — pipeUIMessageStreamToResponse пишет writeHead СИНХРОННО (подтверждено в
ai@6.0.207), а suite ожидает await stream() c моком, не дёргающим колбэки, — так что
отложенный пайп/ожидание сигнала повесит тесты. Поэтому реализована реактивная
рекавери «классификация → штамп → агрессивный ре-трим на следующем ходу», что даёт
тот же инвариант (чат не кирпичится) без рискованного рефактора стрима.

Тесты (наблюдаемые свойства): объём записи через дельту pg_current_wal_lsn() на живой
gitmost-test-pg вокруг 50-шагового прогона (несжимаемые payload'ы) — trace-колонка
v1=140МБ → v2=0.04МБ (в 3206× меньше), полная строка 289МБ → 140МБ (−51%); dual-shape
не нужен здесь; «окно не задано → бюджет применяется»; реактивная классификация на
реальном 400-шейпе; parity клиент/сервер оценщика.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 04:05:48 +03:00
agent_coder 7741821ee3 perf(ai-chat): кэш compactToolOutput по identity шага (#490)
compactToolOutput делает JSON.stringify каждого output на КАЖДОМ flush. Т.к.
onStepFinish на шаге N перестраивает всю assistant-строку по всем N накопленным
шагам, а каждый output — 50–200 KB, это O(N²) stringify за ход.

Мемоизация по identity шага: finished-шаг в capturedSteps неизменен и держит
стабильную ссылку между flush'ами, поэтому его parts (и дорогой stringify output)
строятся ровно раз за ход. buildStepParts вынесен в чистую функцию; assistantParts
принимает опциональный StepPartsCache (WeakMap<step, parts>), flushAssistant
пробрасывает его, stream() заводит один WeakMap на ход и передаёт во все flush'и.
Промах кэша (или его отсутствие в тестах/легаси-вызовах) просто пересобирает —
байтового расхождения нет.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 04:05:48 +03:00
agent_coder 48a074e0d2 perf(ai-chat): формат трейса tool_calls v2 — outputs только в parts (#490)
Каждый tool-output хранился ДВАЖДЫ: в metadata.parts (assistantParts) И в
tool_calls (serializeSteps). При 50-шаговом ране с outputs по 50–200 KB это
127–510 МБ записи в Postgres за ход (+WAL/TOAST/dead tuples), т.к. onStepFinish
переписывает всю строку. Копия в parts — та, что реально реплеится модели и
рендерится UI/markdown-экспортом, так что копия в трейсе была чистым дублем.

Новый формат элементов tool_calls (v2), парно на каждый вызов:
  {toolName, input}                      — вызов
  {toolName, ok: true}                   — успех (БЕЗ output)
  {toolName, error, kind: 'thrown'}      — брошенный tool-error
  {toolName, error, kind: 'interrupted'} — прерван mid-step (abort/restart)

kind обязателен: синтетический «Tool call did not complete.» при прерывании иначе
неотличим от реального hard-fail и загрязняет error-rate. Различие структурное
(errorsById-хит против синтетической ветки), НЕ per-tool классификатор — soft-
маркеры в трейс не выносятся (остаются в metadata.parts).

metadata.toolTraceVersion: 2 — маркер эры; старые строки НЕ мигрируются
(перезапись гигантских jsonb — тот самый WAL-чарн). serializeSteps пейрит
результаты/ошибки по toolCallId (как assistantParts); общая константа
TOOL_CALL_INCOMPLETE_TEXT держит текст реплея и трейса в синхроне.

docs/reading-ai-logs.md переписан dual-shape: ветвление по toolTraceVersion,
soft-анализ v2 через metadata.parts, правило «не сравнивать агрегаты через границу
эр». UI action-log и markdown-экспорт читают только parts — не затронуты.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 04:05:48 +03:00
vvzvlad 17e3b3d882 Merge pull request 'fix(converter): block-escape + attribute contract + dedup mirrors (#493)' (#514) from feat/493-converter into develop
Reviewed-on: #514
2026-07-12 04:04:24 +03:00
vvzvlad 801add63e8 Merge pull request 'fix(ai-chat+security+cache): хвосты стабилизации — транспорт, share-alias, temp-notes, guards (#495)' (#516) from feat/495-tails into develop
Reviewed-on: #516
2026-07-12 04:03:55 +03:00
vvzvlad 7ad072ba2c Merge pull request 'feat(mcp): гарды зеркал реестра + write-целостность (#494)' (#513) from feat/494-mcp-registry-guards into develop
Reviewed-on: #513
2026-07-12 04:03:44 +03:00
agent_coder b041944a23 Merge remote-tracking branch 'gitea/develop' into feat/493-converter
# Conflicts:
#	CHANGELOG.md
2026-07-12 03:28:31 +03:00
agent_coder 401d62119c Merge remote-tracking branch 'gitea/develop' into feat/493-converter
# Conflicts:
#	CHANGELOG.md
2026-07-12 03:23:14 +03:00
vvzvlad 4f8563b5b5 Merge pull request 'feat(comments): audit trail + целостность apply suggestions (#496)' (#512) from feat/496-suggestions-audit into develop
Reviewed-on: #512
2026-07-12 03:21:37 +03:00
agent_coder 27d51303ba fix(page-tree/dnd): drag авто-раскрытие — задержка 2с, не раскрывать при drop внутрь + guard от потери детей и сигнал «уехало сюда»
Пять точечных клиентских правок (дизайн из адверсариального разбора #523):

1. Задержка hover-раскрытия при drag: `AUTO_EXPAND_MS` 500 → 2000 мс, чтобы
   проведение курсора через дерево не раскрывало ряды — только осознанное
   удержание.
2. Не раскрывать цель при drop внутрь (make-child): удалена строка
   `onToggle(target, true)` в `onDrop` (drop оставляет узел свёрнутым; раскрытие
   только по hover-hold). Восстановление раскрытости самого source не тронуто.
3. Guard от потери детей в `handleMove` (обязателен вместе с п.2): раньше эта же
   `onToggle` была ЕДИНСТВЕННЫМ триггером корректирующего lazy-load. `treeModel.move`
   материализует `target.children = [source]`; для НЕзагруженной цели (предикат
   gate `treeModel.isUnloadedBranch`) это скрыло бы остальных серверных детей папки
   (класс #159 #1). Теперь для незагруженной make-child-цели оптимистичное дерево
   строится БЕЗ материализации source: source удаляется из старого родителя, цели
   ставится `hasChildren:true`; gate остаётся взведён и раскрытие догружает полный
   набор. Для загруженной цели поведение прежнее (append сохраняет детей).
4. Инвалидация крошек: `updateCacheOnMovePage` инвалидирует
   `["breadcrumbs", pageId]` — guard делает `remove(source)`, и для текущей
   открытой страницы `findBreadcrumbPath` промахивается → крошки показывали бы
   старого родителя до refocus.
5. Сигнал «уехало сюда»: транзиентный teal-пульс на строке при make-child-drop в
   свёрнутую цель (узел НЕ раскрывается), отдельный `data-landed-child` +
   `DROP_LANDED_HIGHLIGHT_MS`, таймер чистится при анмаунте; уважает
   prefers-reduced-motion.

Тесты: `handleMove` — незагруженная make-child-цель не материализует `[source]`
(мутация guard'а краснит), загруженная цель append'ит и сохраняет детей,
genuinely-empty лист материализует; инвалидация `["breadcrumbs", pageId]`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:15:02 +03:00
agent_coder 51260793c0 fix(ui/toasts): глобальная видимость тостов + перенос в top-center (#517)
Тост-уведомления Mantine сливались с фоном: у бесцветных тостов фон
карточки == var(--mantine-color-body) (белый, как страница) при слабой
тени, поэтому на белых страницах у карточки не было видимого края. Плюс
тосты всплывали снизу по центру и перекрывали контент.

Чиним глобально, без правок в 213 местах вызова:

- notification-overrides.css: каждому тосту даём тонированный по типу фон,
  рамку с контрастом WCAG >= 3:1 и усиленную тень (shadow-xl). Селектор
  [data-mantine-color-scheme=...] .mantine-Notification-root имеет
  специфичность (0,2,0) и стабильно бьёт правила Mantine (0,1,0) (у Mantine
  атрибут схемы обёрнут в :where()) — независимо от порядка стилей. Тон/рамка
  идут от --notification-color (определён на том же элементе), поэтому следуют
  типу тоста и покрывают loading/импортный тост (полосы-акцента нет — несут
  тон+рамка+тень+цветной спиннер). Обе темы; текст-с-заголовком поднят до
  gray-7 ради AA-контраста на тонированном фоне.

- main.tsx: position bottom-center -> top-center. Вертикальное смещение
  контейнера ниже верхней хромы делаем НЕ инлайн-стилем, а CSS-правилом со
  скоупом по позиции: Mantine рендерит все шесть позиционных контейнеров
  одновременно, и корневой style-проп ушёл бы во все шесть — нижним (bottom:16)
  добавился бы top:96 → position:fixed + оба края + height:auto растянули бы их
  на весь вьюпорт; у корня нет pointer-events:none/фона → прозрачные оверлеи
  z-10000 перехватывали бы клики по всей странице.

- notification-overrides.css: .mantine-Notifications-root[data-position^='top']
  { top:96px } (шапка 45 + опц. тулбар 45 + зазор). Скоуп ^='top' смещает
  только верхние контейнеры; нижние остаются height:0 и кликов не перехватывают.
  Специфичность (0,2,0) бьёт mantine top:16px (0,1,0), тост z-10000 стоит ниже
  шапки/тулбара (z-99) и их не перекрывает.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:14:38 +03:00
agent_coder c765483947 fix(page-tree/realtime): insertByPosition теряет непоказанных детей — предикат «незагружено» приведён к gate
`insertByPosition` считал узел незагруженным только при `children === undefined`,
но канонический незагруженный вид в этом коде — `children: []` с `hasChildren: true`
(его ставит `pageToTreeNode`, к нему сбрасывает свёрнутые ветки `pruneCollapsedChildren`).
Для реального незагруженного узла guard `=== undefined` не срабатывал → в пустой
список вставлялся `[node]` → lazy-load gate видел `length !== 0` и не догружал
остальных серверных детей папки (скрыты до полной перезагрузки) — тот же класс
потери данных #159 #1.

- Новый общий предикат `treeModel.isUnloadedBranch(node)`:
  `hasChildren === true && (children == null || children.length === 0)` — единый
  источник истины для «отложить ли фетч/материализацию».
- `insertByPosition` использует его вместо `=== undefined`; для действительно
  пустой папки (`hasChildren: false`) вставка первого ребёнка сохраняется.
- gate в `handleToggle` (`space-tree.tsx`) переведён на тот же хелпер, чтобы
  предикаты больше не разъезжались (R3).

Тесты: юнит на `isUnloadedBranch`; `insertByPosition` не материализует
`[node]` в `children:[]`+`hasChildren:true` (и оставляет ветку незагруженной),
но вставляет в genuinely-empty. Мутация: старый `=== undefined` предикат
краснит эти тесты (в т.ч. на уровне `applyMoveTreeNode`).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:06:42 +03:00
agent_coder 45ff922dd4 refactor(mcp): убрать мёртвый runtime-слой page-id, оставить бренд PageId
Ревью #516: isPageId/asPageId/isSlugId/asSlugId/SLUG_ID_RE + типы SlugId/PageRef
имели НОЛЬ вызовов в монорепе — только барные ре-экспорты в client.ts и кейсы
теста. Несущий смысл — только compile-time бренд PageId (минтится as PageId в
resolvePageId, единственном узле канонизации; asPageId туда намеренно не звался).
Докстринг заявлял «asPageId() guards the untrusted PUBLIC boundary» — но никто не
звал: спекулятивный вес.

Удалил мёртвые runtime-символы + типы SlugId/PageRef + ре-экспорты + их тест-файл.
Оставил тип PageId + касты. Поправил врущий коммент в resolvePageId (бренд —
чистый compile-time маркер, гарантия — что этот узел единственный производитель).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder e3ca2dc1d5 test(db): executeTx — покрыть rollback-ветку after-commit-хука
Ревью #516: fakeDb.execute всегда пушил commit — моделировался только commit-путь,
несущий негатив (тело кидает → tx реджектит → дренаж после awaited-tx не бежит →
хук НЕ фаerr) не был заперт. Исходник корректен, это была дыра в тесте.

Добавлен тест: тело бросает → executeTx реджектит, commit не случился, хук не
побежал. Mutation-verify: перенос дренажа в finally → тест краснеет.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder f919ced8c9 fix(db): concurrent-pre-build индексов — переопределять f_unaccent ПЕРЕД циклом
Ревью #516: фича была мёртвой-при-рождении ровно на целевом кейсе. Все три
CONCURRENT_INDEXES используют LOWER(f_unaccent(col)), но index-инлайнящаяся
1-арг форма f_unaccent (SELECT public.unaccent($1)) создаётся ВНУТРИ миграции
20260705, а ensureConcurrentIndexes зовётся ПЕРЕД мигратором. На существующем
тенанте живой f_unaccent — ещё старая 2-арг форма из 20250729, которая НЕ
инлайнится: CREATE INDEX CONCURRENTLY падает («function unaccent(unknown, text)
does not exist … during inlining»), best-effort глотает, и мигратор строит
индексы НЕ-concurrently под тем самым SHARE-локом (эмпирически подтверждено
ревьюером на живом pg).

Фикс: перед циклом идемпотентно переопределяем f_unaccent в 1-арг форму
(output-identical, в lockstep с 20260705), в том же best-effort try/swallow.

Плюс честный докстринг: убрал ложное «worst case = previous behaviour».
Прерванный CONCURRENTLY оставляет INVALID-индекс, который name-based IF NOT
EXISTS не чинит — новый режим отказа (старый in-tx build такого не оставлял).

Тест: переопределялка вызывается ПЕРВОЙ (order), затем индексы CONCURRENTLY вне
транзакции; провал переопределялки не рвёт цикл. Mutation-verify: убрать
pre-loop redef → order-тест краснеет.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder 4109b2ef7f fix(security): share-alias reassign 409 — гейт раскрытия по view-праву (доведение #495 к.4)
Коммит 4 закрыл утечку currentPageId на анонимном /availability, но
эквивалентная (и более широкая) дыра оставалась на POST /aliases/set (reassign):
при занятом алиасе и confirmReassign=false 409 отдавал currentPageId И
currentPageTitle ЦЕЛЕВОЙ страницы, а контроллер гейтил только validateCanEdit на
ИСХОДНОЙ странице. Любой участник с одной редактируемой+расшаренной страницей мог
перебирать имена алиасов и мапить их на (id, title) чужих страниц без права
просмотра — тот же класс перечисления, плюс ещё и заголовок.

Фикс: setAlias теперь гейтит раскрытие. currentPageId НЕ отдаётся никогда
(клиент им не пользуется, это перечислимая идентичность). currentPageTitle —
только если validateCanView(целевая, user) проходит; иначе голый «занят» (клиент
и так показывает generic confirm-модалку без заголовка — UX не ломается).
Гейт живёт в сервисе, где строится раскрытие (PageAccessService — @Global, без
цикла); контроллер прокидывает user. Поправлен неточный коммент checkAvailability.

Тесты: viewer → 409 с title, БЕЗ id; не-viewer → 409 без title и без id.
Mutation-verify: вернул утечку (id + безусловный title) → оба теста краснеют.
Контроллер-спек и int-spec обновлены под новую сигнатуру; tsc чист.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder 6b27ff0652 feat(ai,mcp): идентичность прогона реиндекса + брендированные PageId/SlugId (#495 item 14)
Аспект A — идентичность прогона реиндекса эмбеддингов.
У статуса/поллинга реиндекса не было идентичности КОНКРЕТНОГО прогона, из-за
чего класс багов «это тот же прогон или новый?» чинили дважды. Теперь каждый
прогон получает свой runId (crypto.randomUUID в start()), он хранится в Redis-
хэше рядом с total/done/startedAt и возвращается в ReindexProgress. Эндпоинт
статуса (getMasked -> MaskedAiSettings) отдаёт runId и reindexStartedAt. Клиент
кеит поллинг на (runId, startedAt): смена runId = НОВЫЙ прогон (сбрасываем
залатанное per-run состояние поллинга), тот же runId = тот же прогон. Всё
best-effort/косметика как и остальной код прогресса: пустой/отсутствующий runId
деградирует мягко и никогда не ломает реиндекс.

Аспект B — брендированные PageId/SlugId в MCP-клиенте + валидация формата в
серверных DTO (семейство инцидентов #435: двойная идентичность страницы —
внутренний id и slugId — гонялась как голая строка и молча путалась).
- packages/mcp/src/lib/page-id.ts: номинальные типы PageId/SlugId + валидирующие
  конструкторы asPageId/asSlugId и гварды isPageId/isSlugId (формат UUID и
  10-символьного slugId). PageId протянут через единственный узел канонизации и
  записи: resolvePageId() теперь возвращает PageId, а ключ per-page лока
  (withPageLock) и точки записи в collab (mutatePageContent/replacePageContent/
  updatePageContentRealtime) требуют бренд — сырой slugId/непроверенный id больше
  не проходит проверку типов (инвариант #260 «resolve-then-lock» теперь на уровне
  компилятора). Публичный вход методов остаётся строкой (это legitimно UUID ИЛИ
  slugId), брендируется каноническое значение.
- apps/server core/page/dto: PageIdDto.pageId получает валидацию формата
  (@Matches, UUID или 10-символьный slugId), так что кривая/подменённая
  идентичность отклоняется на границе, а не падает в repo голой строкой.

Тесты (все зелёные, с mutation-verify каждого):
- server: getMasked отдаёт runId/reindexStartedAt; стор пишет/читает runId и
  мягко деградирует его до ''; DTO отклоняет кривой pageId/slugId и принимает
  валидный.
- client: чистый хелпер reindexRunKey/isNewReindexRun (кеинг поллинга на runId).
- mcp: конструкторы/гварды PageId/SlugId (валидация формата).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder e133b86982 fix(ci): дедуп ночного фаззера по хэшу контрпримера, а не по префиксу заголовка
Ночной property-фаззер при находке контрпримера заводил/обновлял issue,
дедуплицируя по ПРЕФИКСУ ЗАГОЛОВКА. Из-за этого при уже открытом issue по
багу A другой баг B с тем же префиксом заголовка считался дубликатом и
молча терялся до закрытия первого issue — реальные вторые баги глотались.

Теперь дедуп идёт по стабильному короткому хэшу самого контрпримера:
- из вывода fast-check извлекается блок «Counterexample:» (минимальный
  падающий вход) до строки «Shrunk N time(s)»/«Got error»; сид, path и
  счётчик усадки в хэш НЕ входят, поэтому один и тот же баг с разными
  сидами даёт один хэш;
- sha256, первые 12 hex-символов, кладутся в заголовок и в
  машиночитаемый маркер тела `<!-- counterexample-hash: ... -->`;
- поиск открытого issue матчит этот хэш в заголовке или маркер в теле.

Итог: два РАЗНЫХ контрпримера дают два РАЗНЫХ issue, а повторная находка
ТОГО ЖЕ контрпримера по-прежнему схлопывается в существующий.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder 579c82617b i18n(ru): перевести недостающие строки в ru-RU
Добавлены переводы для 151 ключа, присутствовавшего в en-US, но
отсутствовавшего в ru-RU (настройки ИИ, диктовка, MCP, HTML-вставки,
роли агента и др.), включая ранее непереведённые «Streaming dictation»
и «Save and test». Технические токены и бренды (MCP, URL, Docmost AI,
плейсхолдеры версий) намеренно оставлены как есть. Добавлены русские
формы множественного числа (_few/_many) для «result found». Файл
переупорядочен в соответствии с порядком ключей en-US.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder 173f35e473 fix(editor): защитить чтение приватных стеков y-undo в тулбаре
canUndo/canRedo в use-toolbar-state читали приватные внутренности y-undo
(undoManager.undoStack.length / redoStack.length). Апгрейд yjs / y-undo,
переименовавший или перестроивший эти поля, тихо сломал бы состояние кнопок
undo/redo (или упал бы на .length у undefined) без единой ошибки.

Оставляем дешёвое чтение длины стеков (сознательно не используем дорогой
editor.can().undo()/.redo(), который делает dry-run на каждый keystroke,
см. комментарий в файле), но теперь feature-detect: доверяем стекам только
если это реально массивы, иначе откатываемся на безопасный дефолт
(prosemirror-history undoDepth/redoDepth -> 0). Логика вынесена в чистую
функцию yHistoryAvailability.

Добавлен pin-test, фиксирующий текущую форму библиотеки: реальный
Y.UndoManager по-прежнему отдаёт undoStack/redoStack массивами. Апгрейд,
меняющий контракт, упадёт громко в тесте, а не тихо в UI.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder 696b96ac18 fix(client): авто-перезагрузка чанков в окне 5 минут вместо one-shot флага
При ChunkLoadError граница перезагружала страницу один раз, гейтируя
булевым флагом в sessionStorage, который никогда не сбрасывался. Из-за
этого ВТОРОЙ деплой за время жизни вкладки не давал авто-восстановления:
пользователь застревал на битом чанке без перезагрузки.

Заменяю one-shot флаг на счётчик по временному окну: не более одной
авто-перезагрузки за 5 минут. В sessionStorage храню метку времени
последней перезагрузки; на ChunkLoadError перезагружаемся только если
прошлая была раньше окна (или её не было), иначе проваливаемся в ручной
UI без перезагрузки. Это восстанавливает работу через несколько деплоев,
но не даёт бесконечного цикла при навсегда битом lazy-чанке (сброс флага
после успешного маунта отвергнут: оболочка монтируется, чанк 404 —
и цикл).

Решение об окне вынесено в чистый хелпер shouldAutoReload(now,
lastReloadAt, windowMs) и покрыто юнит-тестами: никогда-не-грузили →
можно; 6 минут назад → можно; 1 минуту назад → нельзя.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder a96ca8e26b perf(db): GIN trigram-индексы строятся CONCURRENTLY вне транзакции
2 GIN trigram-индекса (pages.title, pages.text_content) + users.name строились
plain CREATE INDEX внутри Kysely-транзакции миграции: SHARE-lock блокирует записи
на pages/users на минуты при автодеплое.

Kysely гоняет миграцию в транзакции, а CONCURRENTLY внутри транзакции нельзя.
Поэтому ensureConcurrentIndexes (concurrent-indexes.ts) пре-строит эти индексы
через CREATE INDEX CONCURRENTLY (raw, вне транзакции) ДО миграатора — на
существующей БД миграционный CREATE INDEX IF NOT EXISTS становится no-op и лок не
берётся. Best-effort: на свежей БД (нет pages/f_unaccent) пре-build молча
пропускается, а миграция строит индекс на пустой таблице. Худший случай = прежнее
поведение, лучший — без лока.

CONCURRENT_INDEXES — канонические определения; drift-guard тест сверяет их с
выражениями в миграциях. Тест раннера: CONCURRENTLY+IF NOT EXISTS, вне
транзакции, best-effort (падение одного не рвёт остальные).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder f84386f24a fix(temp-notes): свип на старте + FOR UPDATE SKIP LOCKED в транзакции
Две дырки свипера временных заметок:
- Не было свипа на старте: заметки, протухшие во время простоя, ждали до часа
  до первого тика @Interval. Добавил onApplicationBootstrap (best-effort, не
  блокирует boot).
- Гонка без блокировки: re-check и removePage не были атомарны, «Сделать
  постоянной» могла проскочить между ними. Теперь re-check идёт в транзакции с
  FOR UPDATE SKIP LOCKED: сериализуется с toggleTemporary (тот же row-lock) и
  пропускает строки, захваченные другим воркером/инстансом (без двойной
  обработки). Удаление идёт ВНУТРИ этой транзакции.

removePage получил опциональный trx: чтобы удалять под блокировкой без deadlock
на вложенной независимой транзакции. Броадкаст PAGE_SOFT_DELETED при переданном
trx отложен на commit через registerAfterCommit (откат больше не рассылает
фантомное удаление); без trx поведение прежнее.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder c0c44fddb9 fix(security): client-vitals — whitelist route/attr на анонимном эндпоинте
POST /api/telemetry/vitals анонимный, а route/attr не сверялись со словарём:
любой мог писать свободный текст в client_metrics (высокая кардинальность,
инъекция текста/PII/разметки).
- route: экспортировал полный словарь шаблонов из клиентского route-template.ts
  (KNOWN_ROUTE_TEMPLATES — канонический источник), сервер валидирует по зеркалу
  ALLOWED_ROUTE_TEMPLATES: нет в словаре → drop (null), событие остаётся.
- attr: это web-vitals attribution target (CSS-селектор), а не enum — ограничил
  консервативным charset CSS-селектора; всё вне набора → drop.
Клиентский self-consistency тест: templateRoute выдаёт ТОЛЬКО значения из
словаря (иначе легитимные метрики отбрасывались бы). Серверные тесты: raw-путь
и инъекция в route отброшены, PII/разметка в attr отброшены.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder 91e58e3c9f fix(cache): bustWorkspaceCache — после коммита, не внутри транзакции
bustWorkspaceCache звался сразу после write, но ВНУТРИ переданной транзакции
(до коммита). Окно: параллельный читатель промахивается мимо инвалидированного
ключа, читает ещё НЕ закоммиченную (старую) строку и репопулирует кэш старым
значением; после коммита кэш держит устаревшее до TTL (15 c).

Добавил post-commit-хук в executeTx: registerAfterCommit(trx, fn) регистрирует
side-effect, который дренится ТОЛЬКО после коммита транзакции — причём внешним
executeTx, владеющим trx (проброшенный existingTrx срабатывает на настоящей
границе коммита, а не во вложенном вызове). WeakMap по trx — без утечки.
bustWorkspaceCache теперь: без trx — del сразу (уже автокоммит); с trx —
регистрирует del на post-commit. Ошибка хука не валит уже закоммиченный write.

Тест: порядок body→commit→hook, дренаж на внешней границе, глушение падения хука.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder 8d062728e5 fix(page): move-position — валидация charset вместо длины
MovePageDto.position — это fractional-indexing ключ (generateJitteredKeyBetween,
тот же генератор, что в page.service). @MinLength(5)/@MaxLength(12) не совпадали
с реальным диапазоном генератора: плотные between-вставки в глубоком дереве
растят ключ далеко за 12 символов (замерено >40), и валидный ключ, который
сервер сам сгенерил, отклонялся 400 (Gitea #139, п.6).

Теперь валидируем по charset — base-62 алфавит [0-9A-Za-z] — плюс щедрый
@MaxLength(256) как чистый DoS-guard, сильно выше любого реального ключа.
test.failing (bug-lock) распинен в обычный it; добавлен кейс на отклонение
символов вне алфавита (control/separator/инъекция/пустая строка).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:02:13 +03:00
agent_coder f1cffc2d0f fix(security): share-alias availability не отдаёт currentPageId
Проба доступности алиаса возвращала currentPageId — id страницы, на которую
алиас уже указывает — ЛЮБОМУ аутентифицированному участнику воркспейса без
проверки прав на просмотр этой страницы. Перебором имён алиасов можно было
смапить их на id страниц, к которым доступа нет.

Теперь checkAvailability отдаёт только {alias, valid, available}. Бита
taken/free достаточно для пробы; заголовок целевой страницы всплывает лишь
ПОСЛЕ реальной попытки setAlias (путь 409 ALIAS_REASSIGN_REQUIRED), который
проверяет права. Клиент currentPageId нигде не использовал — убран из типа,
стейта и теста. Серверный спек утверждает отсутствие currentPageId в ответе.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:53:44 +03:00
agent_coder bb9a6fd765 chore(ai-chat): guard-пороги в env + единый маркер дегенерации
Три хвоста детектора петель (#444):
- Пороги детектора дегенерации теперь конфигурируются из env (по образцу
  AI_CHAT_FINAL_STEP_LOCKDOWN): AI_CHAT_DEGENERATION_REPEATED_LINES /
  _PERIOD_MAX_LEN / _PERIOD_MIN_REPEATS / _CHECK_STEP. Резолвер читает сырую
  строку (пусто = unset → компилируемый дефолт), требует ≥1, иначе безопасный
  откат к дефолту (0/отриц. сломал бы детектор). Оператор перенастраивает
  анти-babble-guard без редеплоя.
- Единый маркер: при дегенерации live-стрим показывает нейтральное «Response
  stopped.» (клиент не отличает от ручного Stop), а персист-баннер после refetch
  падал в дженерик. Классифицировал OUTPUT_DEGENERATION_ERROR на клиенте под тот
  же заголовок «Response stopped.» + деталь про петлю — live и refetch больше не
  расходятся.
- STEP_LIMIT_NO_ANSWER_MARKER: вместо хардкодной русской строки в content —
  локаль-нейтральная английская (базовая локаль = i18n-ключ этого репо, читаема
  моделью на реплее); не-русские юзеры больше не видят русский текст.

Тесты: env-пороги гоняются против реальных форм повторов с mutation-проверкой
(поднятый checkStep глушит burst; сниженный repeatedLines триггерит короткий
ран); клиентская классификация — новый кейс в error-message.test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:53:44 +03:00
agent_coder 1c083fbd3d chore(deps): вендорные ai-патчи — upstream-трекинг и план выравнивания версии
Оформил как план, а не сюрприз деплоя (строго после drain-патча из #486):
- upstream-репортинг двух ai-фиксов (O(n²) partialOutput heap-OOM; drain-hang
  в writeToServerResponse) и hocuspocus connect-vs-unload (#401) — анализ уже
  в PATCH()-заголовках самих патчей; ссылки держим в AGENTS.md, а НЕ в .patch
  (байты патча идут в patch_hash lockfile e8c599b3 — правка десинхронит пин и
  ломает pnpm install).
- рассинхрон версии ai в монорепе (клиент 6.0.207 vs сервер 6.0.134-patched):
  пока безвреден (серверные фиксы — мёртвый код в браузере), но это дрейф.
  Выравнивание — install-gated шаг с явным «портировать все три ai-патча на
  целевую версию»: оффсеты строк сдвинутся, текущий патч НЕ приложится как есть,
  pnpm install падёт на неприложенном патче — это и есть страховка.

Патч-файлы и lockfile не тронуты; оба tripwire-спека находят маркеры.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:51:13 +03:00
agent_coder 52763998d3 fix(ai): gemini/ollama через aiStreamingFetch + явный maxRetries
Провайдер-фабрики gemini и ollama (chat-путь) шли на глобальном undici-fetch:
без keep-alive recycle, без ретраев на pre-response reset, с дефолтным
(безграничным по паузе) таймаутом. Классы инцидентов #140/#175/#310 для них
воспроизводимы так же, как для openai. Прокинул this.aiProviderFetch (одна
строка на провайдера) — тот же слоёный instrumented streaming fetch, что уже
стоит на openai.

Плюс явно закрепил maxRetries=2 в обоих streamText-вызовах (authenticated и
public-share): совпадает с дефолтом SDK, но фиксирует потолок против дрейфа
дефолта. Арифметика коннектов на ход: (1 + maxRetries=2) × (1 +
AI_STREAM_PRE_RESPONSE_RETRIES) — два слоя ретраев композируются.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:50:35 +03:00
agent_coder bc433d12e6 fix(mcp): ревью #513 — гонка LRU-эвикции + доводки контракта/доков
WARNING [stability]: эвикция могла убить ЧУЖУЮ ещё-connecting сессию как
idle-жертву — isBusy()=false во время open(), сессия вставлена в мапу до
await open(); параллельный acquire на ДРУГУЮ страницу при насыщении выселял
её → спурьёзный reject не-начатой записи / старвейшн новых записей. Фикс:
idle-victim скан теперь  — connecting
падает в last-resort oldestBusy, честный idle по-прежнему предпочитается,
кап держится, коалесинг того же ключа (per-page lock) не задет. Тест на
интерливинг (connecting не выселяется под насыщением), mutation-verified.

Доводки (проза, логику не меняют): drawioCreate description (nodeId:null на
nested-вставке); forward-коммент у writeWithCollabAuthRetry (ретрай только
auth; при расширении — проверять isCollabIndeterminateError, #435); хедер
comment-anchor (все 4 точки делегируют в resolveAnchorSelection); CHANGELOG.

Ребейзнут на develop (волна 1 смержена): только 4 коммита #494.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 00:03:59 +03:00
agent_coder 141ebb4864 fix(mcp): LRU-эвикция не реджектит чужой in-flight mutate как failure (#494)
Коммит 4. При достижении cap реестра live-сессий эвиктился LRU-кандидат через
destroy(), что реджектило его in-flight mutate терминальной ошибкой. Но update
такого mutate мог УЖЕ дойти до сервера и персиститься → эвикция превращала
удачную запись в ложный proval → retry-склонный агент повторял запись →
ДУБЛИКАТ (тот же механизм, что в инциденте #435).

Правка:
- цикл эвикции теперь ПРЕДПОЧИТАЕТ idle-жертву: идёт по LRU-порядку, пропускает
  busy-сессии (isBusy() — есть in-flight mutate) и эвиктит старейшую idle;
- если ВСЕ сессии busy (эвикция неизбежна для приёма новой записи) — эвиктит
  LRU busy через evictForCap(), который реджектит in-flight op ПОМЕЧЕННОЙ
  ошибкой INDETERMINATE «write may have applied — verify before retry», а не
  плоским failure. Маркер collabIndeterminate + guard isCollabIndeterminateError
  (симметрично isCollabAuthFailedError), чтобы «проверь перед ретраем» можно
  было отличить от чистого провала.

Тесты (мутационные): busy LRU-сессия сохраняется, эвиктится younger idle, и её
запись доезжает на ack; when-all-busy — эвикция реджектит запись именно
INDETERMINATE-ошибкой с маркером и текстом verify-before-retry.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:46:37 +03:00
agent_coder 045a0afaad fix(mcp): drawioCreate — успех с warning на вложенной вставке, а не throw (#494)
Коммит 3. При вставке диаграммы во вложенный контейнер (анкор внутри
callout/ячейки таблицы) узел УЖЕ записан и закоммичен мутацией, но тул кидал
ошибку «no addressable #<index> handle». Retry-склонный агент воспринимал это
как провал записи и повторял drawioCreate → ДУБЛИКАТ диаграммы (тот же класс
double-apply, что в инциденте #435).

Правка: ветка insertedIndex<0 возвращает success:true с nodeId:null и
warning'ом «written NESTED, saved — do NOT re-create; re-read via
getOutline/getPageJson (attachmentId …)» вместо throw. Запись подтверждается,
агент знает, что хендла нет и как перечитать — и не ретраит приземлившуюся
запись. nodeId стал string|null в drawioCreate и наследующих drawioFromGraph/
drawioFromMermaid (там тот же путь) + в интерфейсе IDrawioMixin.

Мок-тест: вложенная вставка не кидает, отдаёт success/nodeId:null/warning и
пишет диаграмму РОВНО один раз вложенной (краснеет, если вернуть throw).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:46:37 +03:00
agent_coder be433d40f0 refactor(mcp): гарды остальных зеркал реестра — проза, ярлыки, зонды, счётчик (#494)
Коммит 2. Каждое ручное зеркало получает настоящий гард/деривацию/parity-тест
вместо комментария «mirror this»:

- ROUTING_PROSE → ОБРАТНЫЙ гард (server-instructions.ts): прямой уже покрыт
  генерируемым <tool_inventory> (каждый зарегистрированный тул в списке); теперь
  `unregisteredProseToolMentions` краснеет, если проза ссылается на
  несуществующий/переименованный тул (camelCase-токены прозы ⊆ реестр, минус
  явный список не-тул-терминов PROSE_NON_TOOL_TERMS). Раньше мёртвая ссылка в
  прозе не краснела. Мутационный тест: `getPageContentz` ловится.

- LABELS экспорта чата (chat-markdown.util.ts) → parity-тест: каждый ключ-ярлык
  обязан быть реальным in-app тулом (иначе переименованный тул молча
  сваливается на generic «Ran tool <name>»), и оба языка (en/ru) размечают
  ОДИН набор тулов.

- зонд comment-signal ×2 (оба хоста) → общий `createListCommentsProbe` в
  packages/mcp: index.ts и ai-chat-tools.service.ts (через loader) строят
  tracker.probe из ОДНОЙ фабрики — тела больше не могут разойтись (например,
  один считает resolved-комментарии, другой нет). Проброшен через loader-границу
  как опциональный (отсутствует на устаревшем билде → сигнал выключен).

- countAnchorMatches (comment-anchor.ts) → делегирует решение
  exact-wins/strip-fallback единственному резолверу resolveAnchorSelection
  вместо параллельной копии; поведение идентично (rawCanAnchor ⟺ rawCount>0),
  parity-тест по корпусу краснеет при расхождении count↔resolve.

- normalize+sha256 ×2 (gen-registry-stamp.mjs + docmost-client.loader.ts):
  зеркало УЖЕ закрыто cross-impl parity-тестом (CROSS_IMPL_TREE/EXPECTED
  проверяется с обеих сторон) — критерий issue «либо parity-тест» уже выполнен;
  извлечение общего модуля через границу пакета/билд-шага регрессионно-опасно
  для load-bearing integrity-проверки (#486), поэтому оставлено как есть.

Тесты: mcp node --test unit+mock зелёные (844); затронутые server-specs
(chat-markdown, comment-signal-inapp, loader, service, tiers, contract, cap)
зелёные (351).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:46:37 +03:00
agent_coder e56a05926d fix(mcp): registration-time assert — каждый не-inline спек регистрируем (#494)
Коммит 1. Спек без execute падал по-разному на двух хостах: MCP-хост
(index.ts) в цикле регистрации делает `mcpExecute` иначе `spec.execute!` —
без execute это TypeError в момент ВЫЗОВА тула (то есть в проде, только когда
модель выберет именно этот тул); in-app-хост (ai-chat-tools.service.ts) делает
`inAppExecute ?? execute`, затем `if (!run) continue` — то есть МОЛЧА роняет
тул, он просто исчезает у агента без единой ошибки.

Комментарий «mirror this» гардом не считается: закрываем зеркало настоящим
структурным assert'ом. `assertEverySpecIsRegisterable()` гоняется при загрузке
модуля tool-specs на ОБОИХ хостах (оба его импортируют) и кидает исключение,
если не-inline спек, который хост регистрирует, не несёт исполнителя для этого
хоста — латентный рантайм-TypeError / тихий дроп превращается в громкий отказ
на старте. `inlineBothHosts` освобождён (оба хоста регистрируют его inline,
execute у него намеренно нет); `inAppOnly`/`mcpOnly` проверяются только для
своего хоста.

Тест по реестру с мутационной проверкой: синтетические плохие реестры
(без execute; inAppOnly без inAppExecute) обязаны кидать, а mcpExecute-only /
inAppExecute-only / inlineBothHosts — проходить. Часть (б) — assert
объявления write-класса — уже приземлилась в #489.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:46:37 +03:00
agent_coder eae7640f30 test(comments): не-вакуумные тесты audit-swallow + dominant-run; CHANGELOG (#512 ревью)
DO1: тест audit-swallow был вакуумен (log() — fire-and-forget void persist(),
.not.toThrow() зелен независимо от try/catch). Теперь: после failInsert()
шпионим Logger.warn + слушаем unhandledRejection, флашим микро/макротаски,
ассертим warn=1 и 0 floating-rejection. Mutation: убрать try/catch у persist
→ красный.
DO2: dominant-run тест не отличал longest от first (самый длинный ран был и
первым). Перестроено: короткий plain ран впереди, длинный bold — следом →
ассерт bold:true держится только при genuine longest; +тест tie→first.
Mutation: reduce→segments[0] → красный.
DO3: CHANGELOG [Unreleased] — 3 записи по #496.
DO4: убран no-op override insertAttrs.comment (=dominant.markAttrs, а он и есть
attributes['comment'] — сегменты фильтруются по нему; {...attributes,comment:
attributes.comment}={...attributes}); коммент про 'defensive re-assert' исправлен.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:44:27 +03:00
agent_coder 0099ba272d fix(comments): suggestion с вечным 409 (#496)
expectedText брался из debounced REST-снапшота (getAnchoredText по
pages/info), а метка ставилась в live-доке — при расхождении (док ушёл
вперёд за окно дебаунса) apply строго сравнивал текст под меткой с
устаревшим stored selection и давал 409 на каждый вызов.

MCP-клиент теперь в transform-фазе (та же версия live-дока, где ставится
метка) перечитывает фактическую подстроку под меткой и, если она
отличается от сохранённого selection, синкает её через новый эндпоинт
POST /comments/resync-suggestion-anchor. Best-effort: сбой синка не
откатывает уже заякоренный комментарий, а лишь выдаёт мягкое
предупреждение. Совпадающий снапшот не делает лишнего round-trip.

Сервер: resyncSuggestionAnchor правит только stored selection
незаселённой suggestion своего автора (guards: top-level, есть
suggestedText, не applied/resolved, отличается от suggestedText),
идемпотентно, без ws-бродкаста.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:43:42 +03:00
agent_coder 826bb491ca fix(comments): orphan-anchor reconcile + docstring (#496)
deleteEphemeralSuggestion docstring обещал инвариант «метка снимается
FIRST and FATALLY» синхронно — после #399 это уже не так: fatal только
ENQUEUE снятия метки, сама операция идёт в воркере с ретраями. Docstring
переписан под фактическое поведение.

Reconcile: воркер COMMENT_MARK_UPDATE на resolve/unresolve, обнаружив что
строки комментария больше нет (hard-delete гонкой с ephemeral apply/
dismiss), теперь СНИМАЕТ осиротевшую метку вместо тихого return. Это
самозаживляет тихую дивергенцию и закрывает fire-and-forget resolve/
unresolve enqueue из resolveComment. Операция идемпотентна.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:43:42 +03:00
agent_coder 7be49c1280 fix(comments): дедуп двойного WS-broadcast при apply треда с ответами (#496)
finalizeAppliedSuggestion на ветке «есть ответы» вызывал resolveComment
(бродкаст commentResolved с обогащённой строкой), а затем сам слал ещё и
commentUpdated — клиент получал два события на один apply. Теперь
commentUpdated шлётся только когда resolveComment НЕ вызывался (редкий
повторный вход по уже разрешённому треду), иначе один бродкаст.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:43:42 +03:00
agent_coder c7073b62d1 fix(comments): apply не стирает форматирование (#496)
replaceYjsMarkedText вставлял замену только с меткой `{comment}`, молча
теряя bold/italic/code/link исходного run'а. Теперь захватываем полный
набор атрибутов доминирующего (самого длинного) сегмента заменяемого
диапазона и применяем его к вставке: однородное форматирование
сохраняется точно, для смешанного run'а берётся преобладающий стиль
вместо полной потери. Метка комментария при этом гарантированно
сохраняется (переутверждается явно).

Клиентское предупреждение в превью диффа не добавлялось: у клиента нет
марок из документа (только plain selection/suggestedText), а фикс на
сервере уже сохраняет форматирование, так что баннер был бы неточным.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:43:42 +03:00
agent_coder 11b2c55485 feat(audit): DB-backed audit trail вместо Noop (#496)
Событие comment.suggestion_applied/dismissed раньше улетало в
NoopAuditService и молча терялось; при childless-ветке apply/dismiss
комментарий hard-delete'ится, поэтому восстановить, кто и что решил,
было невозможно.

- DatabaseAuditService пишет в уже существующую таблицу `audit`
  (миграция 20260228T223532); actor/workspace/ip берутся из CLS
  AuditContext, вне HTTP — из явного контекста (logWithContext).
  Аудит — побочная запись: сбой записи не ломает исходный запрос,
  EXCLUDED_AUDIT_EVENTS отбрасываются.
- Биндинг AUDIT_SERVICE переключён с Noop на DatabaseAuditService.
- payload apply/dismiss дополнен suggestedText/selection/commentAuthor/
  decidedBy — на childless-ветке это единственная уцелевшая запись.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:43:42 +03:00
agent_coder 3a344626db fix(converter): block-escape закрывает setext-подчёркивание -- и одиночный = (#514 ревью)
CRITICAL из ревью: escapeLeadingBlockTrigger не покрывал setext-underline
из ровно двух дефисов (--) и одиночного = → строка-продолжение после
hardBreak, равная -- или =, репарсилась как setext-heading, а текст
предыдущей строки терялся. Тот же класс потери данных, что PR и чинит.

Добавлена setext-рука после тематической: целая строка ^-+[ \t]*$ или
^=+[ \t]*$ экранирует ведущий символ. Заякорено на всю строку → mid-content
-/= не задевается; после тематической руки → ---/---- не двойно-экранируются;
идемпотентно против уже-экранированного \=\= (инлайн-escape отрабатывает
раньше). Пины --/----/=/==== + генеративный hardBreakThenSetextArb, оба
mutation-verified. CHANGELOG.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 23:40:26 +03:00
vvzvlad bfb6a52eea Merge pull request 'epic #497 волна 1 — интеграция четырёх approved-итераций (#486/#487/#488/#489) → develop' (#511) from integ/497-wave1 into develop
Reviewed-on: #511
2026-07-11 19:29:27 +03:00
agent_coder 0503e8b4b1 fix(ai-chat): W1 — клиент читает 409-поле activeRunId (было runId=undefined)
Ревью волны 1 (agent_vscode): сервер эмитит id текущего рана в activeRunId
на обоих 409-бранчах (SUPERSEDE_TARGET_MISMATCH, A_RUN_ALREADY_ACTIVE), а
клиентский read409 читал runId → SUPERSEDE_MISMATCH{currentRunId} всегда
undefined: быстрый supersede-хинт мёртв в проде, а клиентские тесты
ложно-зелёные (мокали поле runId, которого сервер не шлёт).

- read409 читает activeRunId (undefined-safe); оба мока — на реальную форму
  + ассерт на усвоенный currentRunId (mutation: revert→runId краснит тесты).
- S4 (groundwork): activeRunId из A_RUN_ALREADY_ACTIVE поглощается в runFact
  (to(), без бампа эпохи/ownership — инварианты #488 целы). Полная обвязка
  supersede чужой вкладки из фазы error требует расширения gate в sendNow —
  отдельным follow-up; сейчас это безопасная заготовка, не рабочая фича.
- spec.md: 2 строки контракт-таблицы приведены к activeRunId.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 19:23:10 +03:00
agent_coder 31f51eaa47 fix(mcp): agent-write НЕ срезает ведущий ---…--- (front-matter strip — только импорт)
Ревью #493 (MEDIUM): вшив normalizeForeignMarkdown первым шагом в
markdownToProseMirrorCanonical, коммит 4 распространил срез YAML front-matter
(YAML_FRONT_MATTER_RE) на КАЖДЫЙ agent-write путь. convertProseMirrorToMarkdown
эмитит `---` для horizontalRule, поэтому страница, начинающаяся с
horizontalRule и содержащая второй `---`, при полном agent-write теряла всё до
второго `---` — молчаливая потеря ранее сохранённого контента.

Правка: разделил нормализацию. normalizeForeignMarkdown (серверный file-import
boundary) по-прежнему срезает front-matter. Новый normalizeAgentMarkdown
(agent-write, markdownToProseMirrorCanonical) делает ТОЛЬКО CRLF-нормализацию +
rewrite GFM reference-сносок (тот самый drift, ради которого коммит 4) и НЕ
трогает ведущий `---…---`. На каноническом сериализованном контенте rewrite —
no-op (он не эмитит `[^id]:`-строк).

Тесты: agent-write horizontalRule-led дока со вторым `---` сохраняет весь
контент (round-trip); file-import с реальным YAML front-matter его по-прежнему
срезает; agent-write всё ещё канонизирует GFM reference-сноски.
Mutation-verify: strip обратно на agent-write → тесты потери контента краснеют.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 18:54:38 +03:00
agent_coder b66929714f fix(converter): block-escape КАЖДОЙ строки параграфа (не только первой)
Ревью #493 (HIGH): escapeLeadingBlockTrigger применялся к всему результату
renderInlineChildren один раз, а регэкспы якорены на ^ без флага m → защищалась
только ПЕРВАЯ строка. hardBreak сериализуется как `  \n`, поэтому триггер на
строке-продолжении не экранировался и на git-sync round-trip re-парсился в
другой блок; для setext/thematic `---` текст строки терялся ЦЕЛИКОМ
([text "a", hardBreak, text "---"] → heading, "---" пропадал).

Правка: параграф теперь бьётся на `\n`-строки и escapeLeadingBlockTrigger
применяется к КАЖДОЙ. Фаззер расширен: hardBreakThenTriggerArb вставляет
триггер ПОСЛЕ hardBreak в inlineContentArb, так что P1/P2/P3 структурно
покрывают подслучай (раньше триггер стоял только первым run'ом). Добавлен
детерминированный пин на continuation-line триггеры, включая `a  \n---`
(текст «---» сохраняется, не setext).

Mutation-verify: со старым single-line escape новый пин + P1/P2 краснеют.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 18:54:38 +03:00
agent_coder a09935aa29 fix(mcp): resolved-якоря переживают полный markdown-write
Read-путь прячет resolved comment-анкоры (#337), поэтому markdown, который
агент шлёт в updatePageMarkdown, их уже не содержит — наивный full-write
стирал ВСЕ resolved comment-марки (потеря данных). Активные комментарии
переживают round-trip сами (read отдаёт их <span data-comment-id>), а
resolved — нет.

Правка: в write-пути (updatePageContentRealtime) пере-прививаем resolved-
марки из ЖИВОГО дока на совпадающие текстовые диапазоны свежеимпортированного
тела, механикой анкоринга comment-anchor. spliceCommentMark обобщён на
произвольную марку; добавлены applyCommentMarkInDoc (сохраняет resolved:true
+ attrs), collectResolvedCommentSpans и regraftResolvedComments (чистая, не
мутирует входы). Спан, чей текст агент изменил/удалил, просто не
переанкорится и отбрасывается (он и так resolved). first-occurrence-семантика
как у остального анкоринга.

Проверено: 6 новых наблюдаемых тестов + весь MCP unit-suite (691) зелёные.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 18:19:10 +03:00
agent_coder 047433595e refactor(mcp): дедуп stripInlineMarkdown — единый источник в каноническом пакете
Локаторная нормализация markdown (stripInlineMarkdown + примитив
stripWrappersAndLinks с WRAPPER_PATTERNS/LINK_IMAGE_RE) была ФОРКНУТА один-в-
один в packages/mcp/src/lib/text-normalize.ts и в каноническом
@docmost/prosemirror-markdown (где ей пользуется node-ops). MCP теперь
импортирует оба примитива из пакета (mcp и так от него зависит — цикла нет) и
держит на них лишь свои тонкие надстройки stripBalancedWrappers/
closestBlockHint. ~60 строк дубля удалено, дрейф закрыт.

Проверено: пакет node-ops (103) + весь MCP unit-suite (685) зелёные.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 18:14:39 +03:00
agent_coder 9e95412695 refactor(converter): normalizeForeignMarkdown -> в пакет, единый import-boundary
Нормализация чужого markdown (GFM reference-сноски [^id] -> инлайн ^[body],
срез ведущего YAML front-matter) жила только в apps/server, поэтому MCP-путь
записи страницы (updatePageMarkdown -> markdownToProseMirrorCanonical) тот же
ввод обрабатывал ИНАЧЕ, чем серверный импорт: front-matter и [^id] утекали
как литеральный текст / битая ссылка.

Перенёс normalizeForeignMarkdown в @docmost/prosemirror-markdown и вызвал его
первым шагом в MCP markdownToProseMirrorCanonical — теперь агентский
updatePageMarkdown нормализуется точно как серверный импорт. Серверные
импортёры (import.service, file-import-task.service, page.service) берут
функцию из пакета. Тест-корпус перенесён в пакет (foreign-markdown.test.ts).

Проверено: пакет (17 тестов corpus) + весь MCP unit-suite (685) зелёные,
включая reference-footnote/fence-кейсы на canonical-пути.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 18:05:47 +03:00
agent_coder 2fa86e2a33 fix(converter): warnings вместо тихой потери незнакомых нод/марок
Незнакомый тип ноды (default-ветка switch) молча схлопывался в свои дети, а
незнакомая марка молча выбрасывалась — тихая потеря данных. Теперь
сериализатор РЕПОРТИТ потерю:
- по умолчанию поведение байт-в-байт прежнее (graceful degrade), но при
  переданном options.warnings в сток кладётся по одному сообщению на
  незамапленный тип (дедуп по типу) — потеря наблюдаема;
- options.strict бросает ConverterLossError на ПЕРВОМ незнакомом типе
  (warning = ошибка).

git-sync (lossless-путь) включает strict в stabilizePageBody: тип без
серизализующей ветки падает громко на записи, а не пишет lossy .md. Валидный
контент не затронут — у всех текущих типов схемы есть ветка.

Покрытие: converter-loss-warnings.test.ts (нода/марка × strict/non-strict,
дедуп, чистый контент) и strict-пин в git-sync stabilize.test.ts — всё через
реальный конвертер.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 17:56:11 +03:00
agent_coder e3eece78c3 test(converter): атрибутный contract-тест схем editor-ext <-> mirror
Name-level контракт ловил пропажу целой ноды/марки, но не дрейф АТРИБУТОВ
внутри вендоренной ноды — класс, из-за которого молча потерялся
subpages.recursive. Добавлен атрибутный контракт: для каждой ноды/марки
@docmost/editor-ext сравниваются её СОБСТВЕННЫЕ объявленные атрибуты (имена
+ дефолты, читаются из config.addAttributes) с spec.attrs собранной схемы
зеркала.

Направление editor-ext -> mirror: зеркало намеренно надмножество (глобальные
id/textAlign/indent, нормализация части дефолтов в null), поэтому обратное
сравнение — ложный дрейф. Значимый провал — атрибут, который зеркало РОНЯЕТ
(имя) или чей дефолт молча меняет. Два blessed вида расхождений вынесены в
обоснованные allowlist'ы (highlight.colorName — нет md-формы;
image.src/link.internal/pdf.width/height — null-нормализация непереносимых
атрибутов), оба со stale-guard, чтобы список не сгнил.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 17:49:03 +03:00
agent_coder e1b8ef5b8b fix(converter): block-escape начала параграфа — закрытие класса потерь данных
Строка параграфа, начинающаяся с блочного триггера (`#`/`-`/`*`/`+`/`>`,
упорядоченного `N.`/`N)`, фенса ```/~~~, таблицы `|` или тематического
разрыва `---`/`***`/`___`), на round-trip doc->markdown->doc молча
превращалась в heading/list/quote/code block/table/horizontalRule. Худший
случай — тематический разрыв: horizontalRule не несёт текста, и строка
теряла его целиком.

Сериализатор параграфа теперь backslash-экранирует ведущий блочный триггер
(escapeLeadingBlockTrigger): экранируется только ПЕРВЫЙ значащий символ,
токенизатор CommonMark декодирует `\` обратно в литерал И снимает блочную
интерпретацию, так что строка round-trip'ится байт-в-байт как параграф.
Emphasis `**x**`, inline-code и обычная проза триггерами не являются и не
трогаются (нет мусорных backslash).

Класс раньше не чинили, а ОБХОДИЛИ; обход убран у обоих потребителей:
- клиентский мост (gitmost-recording.ts) больше не подставляет ZWSP-хак;
- генеративный корпус (text-arbitraries.ts) снял самоцензуру — добавлен
  blockTriggerLeadRunArb, параграф теперь МОЖЕТ открываться триггером, и
  P1/P2/P3 сами доказывают закрытие класса.

Пины на каждый триггер — детерминированные round-trip через реальный
конвертер (gitmost-transcript-neutralization.test.ts). Обновлён
документировавший старую потерю gap-тест (spec 13).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 17:43:24 +03:00
agent_coder b97cad0ebe Merge remote-tracking branch 'gitea/feat/488-client-fsm' into integ/497-wave1 2026-07-11 17:29:35 +03:00
agent_coder 080dd82023 fix(client): ре-ревью #509 — 5 пунктов Do-листа (стабильность/регрессии) (#488)
1 [stability] Позитивные attach-исходы гвардятся по ИСХОДНОЙ фазе. Одного epoch-
фильтра мало: POLL_TERMINAL использует to() (epoch не инкрементит) и не шлёт
abortAttach, поэтому медленный GET, вернувший live 2xx уже ПОСЛЕ того как армленный
poll увёл машину в idle, воскрешал осевший ран в фантомный streaming. RECONNECT_
ATTACHED теперь bail'ит если фаза != reconnecting; ATTACH_LIVE/ATTACH_NONE — если
!= attaching. Тест на гонку (POLL_TERMINAL до RECONNECT_ATTACHED → idle) + mutation.

2 [regressions] ownership сбрасывается в "local" на ВСЕХ терминальных переходах
(FINISH_CLEAN/ABORT/ERROR, POLL_TERMINAL, RUN_FACT{null}→idle, honor-in-stopping,
disconnect→idle). Иначе observer-attach + очередь + clean-финиш → idle, но
ownership навсегда observer → «Send now» скрыт при свободном композере. Безопасно
для I2: рантайм захватывает wasObserver из machineRef ДО dispatch. Тест + mutation.

3 [coverage] Тест happy-path CAS-supersede: SUPERSEDE_READY-dispatch (200-исход
transport.fetch) раньше НЕ исполнялся ни в одном тесте — риск залипания в
superseding на весь стрим B. Тест вводит в superseding, гонит A.onFinish→B, затем
POST 200 → SUPERSEDE_READY → streaming, проверяет повторный supersede (не залип).
Сиблинги: 409 SUPERSEDE_TARGET_MISMATCH → getRun/verify; plain-409
A_RUN_ALREADY_ACTIVE → классифицированный баннер. Mutation (no-op READY → красный).

4 [regressions] Inactivity-бэкстоп для poll, армленного в stopping. STOP_REQUESTED
армит poll и входит в stopping, но idle-cap покрывал только polling/reconnecting →
observer-стоп без SDK-стрима и без серверного терминала поллил БД вечно. Добавлен
stopping в фаза-чек эффекта + переход POLL_IDLE_CAP: stopping→idle+disarm (НЕ
stalled — Stop уже нажат). FSM + компонент-тест (idle-cap→disarm) + mutation.

5 [docs] Счёт §2 «Net»: 8→FSM (#1-6,#11,#13) + 3 deleted + reconnectTimerRef
(effect-owned) + mountedRef (retained) = 13; attachAbortRef вне набора #1-13.

Не трогал DROP-блок (мёртвый FINISH_* в superseding, ErrorKind.kind, неиспользуемые
enum-варианты, epochRef-зеркало). Всё прошлое цело (disconnect-first, epoch, honor-
in-stopping, render-gate, supersede). Полный ai-chat 35 файлов / 388 / 0; tsc 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 11:07:02 +03:00
agent_coder 5adcd2f08b fix(mcp): форма сохраняет deny-all allowlist вместо тихого расширения до allow-all (#477 ревью)
F1: сервер с tool_allowlist=[] (deny-all) грузился в форму пустым TagsInput;
submit слал null (allow-all) → админ, открывший deny-all сервер сменить
имя/URL, молча отдавал агенту ВСЕ тулы (тот же silent-widen класс #476, что
PR закрывает на read-стороне). Вынесен pure-хелпер resolveToolAllowlist:
пустое поле + сервер был [] → [] (deny-all сохранён); пустое + был null →
null (реально неограниченный остаётся). +тест (5 кейсов, различие []vs null).

F2: CHANGELOG/Security — флип семантики allowlist ([]=deny-all, было
NULL/allow-all; corrupt→fail-closed deny-all; форма не расширяет deny-all).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 10:24:03 +03:00
agent_vscode 1e7bd1f9d2 ci(#476): гейты наблюдаемых свойств перед publish — image-smoke, migration-order на push, allowlist fail-closed, property-тесты
Retrospective of 22.06-10.07 merges showed one recurring miss class: local
logic verified, integration property never checked (#361, #353, #452, #172,
#435). This lands four gates so each of those classes fails BEFORE the
:develop image is pushed:

1. Image boot-smoke in the publish job (develop.yml + scripts/ci/image-smoke.sh):
   the exact image watchtower pulls is booted against postgres/redis services
   before the push — /api/health (startup migrator, #361-boot/#353), auth/setup,
   client dist served, hashed assets immutable + brotli (#452).
2. migration-order gate now also runs on push (test.yml): direct pushes used to
   bypass the PR-only gate; base = event.before, zero-SHA skips, force-push
   fails closed.
3. External-MCP tool allowlist fails closed (#172 class): corrupt stored value
   now reads as [] (deny-all) with an error log instead of null (allow-all);
   [] round-trips as jsonb [] via jsonbBind({preserveEmpty}) and means deny-all
   in the toolset filter. The settings form sends null for an empty tag field
   so existing "unrestricted" servers are not silently narrowed.
4. Property tests for the silent-degradation classes: converter fixpoint
   through the live server path (mcp e2e), and CollabSession cache-key
   stability under per-call fresh tokens (#435/#439 lesson) incl. a negative
   control with the token cache disabled.

Closes #476

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 10:24:03 +03:00
agent_coder 803bbd40b4 fix(client): фаза FSM решает баннер — reconnect не маскируется остаточным error (#488)
Браузерный QA: маршрутизация disconnect-first работает (лесенка реально стартует),
но пользователь всё равно видел терминальный «Lost connection… reload», а не
«reconnecting… (N/5)».

Трасса (подтверждена по коду): рендер `{errorView ? <ChatErrorAlert/> : phase ===
"reconnecting" ? ...}` даёт errorView ПРИОРИТЕТ над recovery-фазой. errorView =
`error && describeChatError(...)`, где `error` — из useChat. Реальный ai@6 при
isError выставляет useChat `error`, а дроп ВСЕГДА isError+isDisconnect → после
починки маршрутизации FSM уходит в `reconnecting`, но `error` остаётся выставленным
→ errorView перекрывает reconnect-баннер. Тот же класс вакуум-мока, что и с
isDisconnect, но на поле `error` (мок хардкодил error:null → в тестах маски не было).

Фикс рендера: фаза FSM — источник истины. Терминальный errorView показывается
ТОЛЬКО когда FSM реально терминален: `showError = errorView && phase === "error"`.
В recovery-фазах (reconnecting/polling/stalled/superseding/stopping) выигрывает
recovery-баннер (или контент стрима). Классифицированные 409 (#487) целы: supersede/
gate-409 ставят FSM в error(kind) → errorView показывается там, где должен.

Мок useChat сделан реалистичным СИСТЕМНО: добавлено поле h.state.error, мок его
возвращает; при любом isError-финише (дроп или провайдерская ошибка) тест ставит
error в реальную форму, зеркаля связку SDK. MUTATION-VERIFY: откат рендер-гейта
(errorView-first) → 8 reconnect/stalled-тестов краснеют (баннер маскируется);
с гейтом — зелёные. Плюс отдельный тест «reconnect-баннер виден, не замаскирован».

Всё прошлое цело: disconnect-first, epoch-штамп, honor-in-stopping, supersede-
исходы, stalled/no-poll. Полный ai-chat 35 файлов / 378 / 0; tsc ai-chat 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 09:55:39 +03:00
agent_coder c7a38e274e fix(client): реальный обрыв SSE → reconnect-лесенка, не терминальный error (#488)
Браузерный QA поймал баг, который юнит-тесты пропускали из-за вакуумной SDK-формы.

Премиса (подтверждена по ai@6.0.207 AbstractChat.makeRequest, catch ~13763): при
сетевом дропе SDK ставит isError=true БЕЗУСЛОВНО, а isError=true → isDisconnect=true
ставится РЯДОМ (только для fetch/network TypeError). Т.е. реальный обрыв ВСЕГДА
даёт { isError:true, isDisconnect:true }; форма { isDisconnect:true, isError:false }
SDK'ом НЕ эмитится.

Баг: в onFinish проверка `if (isError) return` стояла РАНЬШЕ ветки isDisconnect →
реальный дроп уходил в терминальный error-баннер, а FINISH_DISCONNECT (единственный
вход в reconnect-лесенку) не диспатчился НИКОГДА. Сценарии 1/2 (commit 2 «обрыв до
первого кадра» и commit 3 «два обрыва») в браузере не работали.

Фикс: маршрутизация по disconnect ПЕРВЫМ: isDisconnect → FINISH_DISCONNECT
(reconnect); НЕ-disconnect error (isError && !isDisconnect, напр. провайдерский 500)
→ FINISH_ERROR (терминал); затем isAbort; затем clean. Порядок веток supersede-
блока тоже disconnect-first (для консистентности; там всё равно всё дропается I1).

Инварианты сохранены: epoch-штамп turnEpochRef на всех FINISH_*; honor-in-stopping
в редьюсере честит ЛЮБОЙ финиш в фазе stopping → на пути Stop дроп-финиш уходит в
idle, а не в ложный reconnect (новый тест). F1 supersede-drop чужого поколения цел.

Тесты сделаны НЕ вакуумными: дроп подаётся реальной формой { isError:true,
isDisconnect:true }, терминальная ошибка — { isError:true, isDisconnect:false }.
MUTATION-VERIFY: багованный isError-first порядок → 6 reconnect-тестов краснеют
(нет баннера «reconnecting»); с фиксом зелены. Полный ai-chat 35 файлов / 377 / 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 09:37:07 +03:00
agent_coder 1685b32333 fix(client): ре-ревью F1 — правка задетых Stop и supersede-таймингов (#488)
MEDIUM — эпоха-штамп из F1 сломал выход из `stopping` для локального Stop.
STOP_REQUESTED бампит epoch (E1→E2), а onFinish аборчиваемого стрима стампится
ПРЕД-стоп-эпохой E1 (start-эпоха стрима), поэтому фильтр I1 дропал FINISH_ABORT и
машина зависала в `stopping` навсегда (idle-cap покрывает только polling/
reconnecting). Фикс в редьюсере: ЛЮБОЙ finish (`FINISH_*`/`STREAM_INCOMPLETE`) в
фазе `stopping` честится и выводит `stopping→idle` МИНУЯ epoch-фильтр — у обычного
Stop нет преемника, финиш аборта и есть ожидаемое завершение (I4). Для `superseding`
фильтр сохранён (это и есть F1-drop). Тест переписан на ПРЕД-стоп-эпоху E1 (реальная
проводка); MUTATION-VERIFY: снятие honor-in-stopping → зависание → красный.

LOW-1 — supersede терял B, если A уже settled, а statusRef ещё «streaming».
Ливнесс в sendNow теперь берётся из ФАЗЫ FSM (machineRef, обновляется onFinish'ем
СИНХРОННО), а не из отрендеренного statusRef: settled-A (фаза idle) → B шлётся
немедленно, без аборта мёртвого стрима и залипания в `superseding`. statusRef
удалён (больше не нужен). Тест на под-кадровое окно.

LOW-2 — двойной «Send now» в окне аборта A перезаписывал pendingSupersedeTextRef.
Второй клик при уже летящем supersede (pendingText взведён / фаза `superseding`) —
NO-OP: сообщение остаётся в очереди, ничего не теряется/не перетирается. Тест.

Тесты: FSM 36 + chat-thread 39 (+error 26 +adopt 16) = 117 зелёных; tsc ai-chat 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 08:02:04 +03:00
agent_coder 4b78e336a2 fix(client): внутреннее ревью миграции FSM — F1..F4 (#488)
F1 [correctness] — supersede: перекрытие стримов рушило свежий run.
ai@6 AbstractChat.makeRequest в finally читает и обнуляет общий activeResponse,
поэтому параллельные стрим-A и стрим-B корёжат друг друга, а незастемпленный
onFinish мёртвого A уводил ЖИВОЙ новый run в ложный reconnect / сбрасывал runFact.
Фикс: (1) FINISH_*/STREAM_INCOMPLETE штампуются per-stream generation (turnEpochRef,
взводится в момент старта стрима) → фильтр I1 отбрасывает финиш чужого поколения;
(2) sendNow-supersede АБОРТИТ A и стартует B только из onFinish A (микротаск,
после того как finally A обнулил activeResponse) — гарантия отсутствия перекрытия.
Тест на поздний isDisconnect A после SUPERSEDE_REQUESTED: машина НЕ уходит в
reconnect, B отправлен. MUTATION-VERIFY: снятие epoch-штампа у FINISH_DISCONNECT →
тест краснеет («Connection lost — reconnecting»).

F2 [correctness] — гонка mount getRun→ATTACH_START с локальным send. Редьюсер
ATTACH_START теперь игнорирует любую не-idle фазу, поэтому поздний резолв getRun не
перехватывает начавшийся локальный турн в observer-attach. Тест на гонку.

F3 [ghost feature] — RUN_SUPERSEDED объявлен+покрыт тестом, но НИКОГДА не
диспатчился. Удалён (событие+обработчик+тест+postRun-reason observer-follow) как
сознательный scope-cut: наблюдатель убитого supersede-рана и так следует за новым
через деградированный поллинг (свежие строки истории, независимо от runId).

F4 [hygiene] — мёртвые события. STREAM_START подключён (первый ассистент-фрейм
локального турна: sending→streaming, спека↔код совпали). RECONNECT_BEGIN и
POLL_ACTIVITY удалены (не диспатчились). Множество событий редьюсера = множеству
диспатчащихся; редьюсер тотален.

Тесты: FSM 35 + chat-thread 37 (+error 26 +adopt 16) = 114 зелёных; tsc ai-chat 0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 08:02:04 +03:00
agent_coder 77e20ecb41 fix(client): миграция chat-thread на FSM — reconnect×N, stalled, supersede (#488)
Полная миграция chat-thread.tsx на автомат run-fsm: 13 ref-флагов жизненного
цикла resume/reconnect/poll/ownership УБРАНЫ (карта ref'ов в run-fsm.spec.md,
колонка pending пуста). Коммиты 3/4/5 приезжают одной атомарной миграцией —
три фикса делят одну модель состояний (раздельные коммиты давали бы несобираемые
промежуточные состояния, что противоречит смыслу единого автомата).

Коммит 3 — повторные циклы reconnect: attached→reconnecting разрешён многократно.
Различие «live-follow (лестница reconnect) vs mount-resume» вынесено в ctx-поле
liveFollow (НЕ новый ref — это и есть смысл FSM): live-follow-обрыв перезаходит в
лестницу (сброс счётчика после успешного re-attach), mount-resume-обрыв уходит в
poll. Тест «два обрыва подряд → два цикла».

Коммит 4 — (a) polling→stalled по idle-капу (баннер+Retry вместо тихого
«вечно-полуготового»); кап переехал в тред (idleCapTimerRef, effect-owned, не
флаг), окно теперь тупо поллит по armed-флагу. (b) resume армится ТОЛЬКО при
серверном подтверждении активного рана: streaming-tail (статус) или POST /run для
user-tail — чат без активного рана больше не порождает ~240 req/10мин. Тесты:
stalled-баннер; user-tail с/без активного рана.

Коммит 5 (supersede) — удалены SUPERSEDE_RETRY_DELAYS_MS/isRunAlreadyActive/
supersedeRetryRef (клиентская лестница ретраев). «Прервать и отправить» идёт через
FSM superseding → POST /stream {supersede:{runId}} (runId из start-метаданных,
extractRunId). Транспорт разбирает CAS-исход: ok→SUPERSEDE_READY (новый стрим),
409 MISMATCH→verify через /run, TIMEOUT/INVALID→классифицированная ошибка без
авто-ретрая; голый 409 A_RUN_ALREADY_ACTIVE→RUN_ALREADY_ACTIVE. pendingSupersedeRef
(send-плумбинг data) — единственная замена трёх удалённых one-shot'ов.

Инвариант epoch (I1) гейтит каждый command-outcome (attach/reconnect/supersede/
postRun): устаревшее поколение колбэка отбрасывается редьюсером; DISPOSE на unmount
инкрементит epoch. mountedRef оставлен как React-liveness (ортогонален lifecycle).

Тесты: FSM 37 переходов; chat-thread 35 (переписан на FSM-переходы); все зелёные.
E2E (реальный SSE/reconnect/supersede через редиплой) — на staging QA.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 08:02:04 +03:00
agent_coder d533daa45f fix(client): обрыв SSE до первого кадра ассистента → ран подхватывается (#488)
Коммит 2. Раньше вход в reconnect требовал `message?.role === "assistant"`, и
обрыв в setup-фазе (до первого кадра ассистента, включая сборку MCP-тулсета с
дедлайном до 60 c) не давал НИ реконнекта, НИ поллинга — а detached-ран
продолжал писать в страницы (тихая дыра целостности).

Правка: вход в reconnect по РАН-ФАКТУ (активный detached-ран), а не по наличию
assistant-сообщения. В autonomous-режиме ран активен весь ход, поэтому здесь
сигнал run-факта — сам autonomousRunsEnabled; более богатый серверный run-факт
(POST /run / runId из start-метаданных) смоделирован и покрыт тестами в FSM
(run-fsm.ts FINISH_DISCONNECT по ctx.runFact) и приезжает с полной миграцией
компонента. При отсутствии assistant-строки reconnect идёт БЕЗ strip/anchor
(простой live-attach — на экране нечего перестраивать).

Тест: «обрыв до первого кадра → баннер reconnect + resumeStream + attach без
anchor», плюс FSM-переход (уже зелёный в коммите 1).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 08:02:04 +03:00
agent_coder ba58493909 fix(client): классификация 409-кодов + run-факт плумбинг (#488)
Потребляет реальный контракт #487 на клиенте (без выдумывания кодов):
- error-message.ts: ветки для 409 A_RUN_ALREADY_ACTIVE / SUPERSEDE_TARGET_MISMATCH
  / SUPERSEDE_TIMEOUT / SUPERSEDE_INVALID — человеческие тексты СТРОГО ДО generic-
  веток по статусу (иначе юзер видит сырой JSON {"code":"A_RUN_ALREADY_ACTIVE"});
- extractRunId(message): чтение runId из start-метаданных (зеркало
  extractServerChatId) — live-обновление run-факта для FSM;
- getRun(chatId): POST /ai-chat/run — first-class run-факт с сервера (init на
  маунте + verify после supersede-mismatch).

Плумбинг под FSM-обвязку коммитов 2–5. Тесты: классификатор (все 4 кода + order-
guard), extractRunId.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 08:02:04 +03:00
agent_coder 947796adef refactor(client): FSM skeleton + spec для run-lifecycle (#488)
Заменяет зоопарк из ~26 useRef-флагов в chat-thread.tsx на один чистый
редьюсер с перечислимыми переходами (event × state → next state — впервые
юнит-тестируемо напрямую).

Коммит 1 из 5. Содержит СПЕКУ (пишется первой, входит в PR) и каркас:
- run-fsm.spec.md: таблица «событие × состояние», карта всех ref'ов
  → {состояние | контекст | данные}, протокол run-факта, список инвариантов;
- run-fsm.ts: чистый reduce(machine, event) → machine с epoch-инвариантом (I1),
  состояниями idle|sending|streaming|attaching|reconnecting|polling|stalled|
  stopping|superseding|error, ownership как ПОЛЕ контекста (I2), run-фактом
  как first-class (I3), выходом из stopping по данным (I4), dispose-протоколом
  (I5) и слоем command-эффектов (attach/postStream/postRun/stop/supersede);
- run-fsm.test.ts: 31 тест переходов, включая поведение коммитов 2–5 как
  переходы автомата (reconnect по run-факту; повторные циклы reconnect;
  polling→stalled; supersede CAS-исходы; фильтрация позднего колбэка по epoch).

8 зафиксированных решений реализованы; epoch-инвариант неотключаем.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 08:02:04 +03:00
167 changed files with 11579 additions and 2645 deletions
+63
View File
@@ -62,6 +62,38 @@ jobs:
needs: [test, e2e-server, e2e-mcp, build]
runs-on: ubuntu-latest
timeout-minutes: 30
# Image boot-smoke (issue #476): every other job tests code from the working
# tree, but the :develop IMAGE that watchtower pulls was never actually
# started anywhere (incident classes #353/#452/#361-boot: startup-migrator
# crash-loop, runtime module missing from the image, wrong static-asset
# headers). The services below back a smoke boot of the exact image right
# before it is pushed; a smoke failure blocks the push.
services:
postgres:
# via mirror.gcr.io (Docker Hub pull-through cache; avoids Hub anonymous
# pull rate-limit that randomly fails on shared GitHub runner IPs).
image: mirror.gcr.io/pgvector/pgvector:pg18
env:
POSTGRES_DB: docmost
POSTGRES_USER: docmost
POSTGRES_PASSWORD: docmost
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U docmost"
--health-interval 5s
--health-timeout 5s
--health-retries 20
redis:
# via mirror.gcr.io (see postgres note above).
image: mirror.gcr.io/library/redis:7
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 5s
--health-retries 20
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -82,6 +114,37 @@ jobs:
id: version
run: echo "value=$(git describe --tags --always)" >> "$GITHUB_OUTPUT"
# Load the image into the local docker daemon so it can be booted (the
# push step below exports straight to the registry and leaves nothing
# runnable locally). CONVENTION: build-args here must stay TEXTUALLY
# IDENTICAL to the push step's build-args — same cache scope + same args
# means the layers are reused and the image we smoke IS the image we push.
- name: Build image for smoke (load, no push)
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64
build-args: |
APP_VERSION=${{ steps.version.outputs.value }}
AI_AGENT_ROLES_CATALOG_URL=https://raw.githubusercontent.com/vvzvlad/gitmost/develop/agent-roles-catalog
load: true
push: false
tags: gitmost:smoke
cache-from: type=gha,scope=develop-amd64
# Boot-smoke the exact image against the job services (see the comment on
# `services:` above): health (startup migrator), auth/setup, client dist
# served, immutable + brotli asset headers. Fails the job (and therefore
# the push) on any miss.
- name: Smoke the built image
run: bash scripts/ci/image-smoke.sh gitmost:smoke
# The smoke script leaves the container running on failure precisely so
# the boot error (migration mismatch, stack trace) is diagnosable here.
- name: Dump smoke container log on failure
if: failure()
run: docker logs gitmost-smoke 2>&1 | tail -200 || true
- name: Build and push develop image
uses: docker/build-push-action@v6
with:
+41 -10
View File
@@ -124,9 +124,17 @@ jobs:
exit "$FAILED"
# A GENUINE counterexample: fast-check printed a shrunk minimal case and its
# reproducing seed into property-output.txt. File a dedup-guarded issue whose
# title prefix is UNIQUE to counterexamples, so an infra failure (handled by
# the next step under a different title) can never poison this dedup.
# reproducing seed into property-output.txt. File a dedup-guarded issue.
#
# Dedup is keyed on a HASH of the SHRUNK COUNTEREXAMPLE (the minimal failing
# input), NOT on the issue title prefix. Keying on the prefix would let a
# single open issue swallow every OTHER counterexample (a different bug B whose
# title shares the prefix would be treated as a duplicate and stay silent until
# the first issue is closed). Hashing the shrunk example instead means two
# DIFFERENT counterexamples get two DIFFERENT issues, while a re-find of the
# SAME counterexample still dedupes onto the existing one. The infra-failure
# step (below) still keys on its own distinct title, so it can never poison
# this dedup either.
- name: File counterexample issue
# always() is REQUIRED: the fuzz step exits nonzero on a failing shard,
# so a bare `if:` (implicitly success() && ...) would skip this step
@@ -146,25 +154,48 @@ jobs:
echo "No fast-check counterexample signature — infra failure, handled by the next step."
exit 0
fi
TITLE="${TITLE_PREFIX} (seed=${FAIL_SEED})"
# Extract the SHRUNK counterexample block: the "Counterexample:" line(s)
# up to (but excluding) the "Shrunk N time(s)" / "Got error" line. This is
# the minimal failing INPUT and is STABLE across the different seeds/paths
# that reach the same bug — unlike the seed, path, or shrink count (which
# precede/follow this block and vary run-to-run) and unlike the whole
# output (which embeds those varying parts). Hashing THIS is what makes the
# dedup identity the bug itself rather than an incidental run detail.
CE_TEXT=$(awk '/Counterexample:/{c=1} /Shrunk [0-9]+ time|Got error/{c=0} c{print}' property-output.txt)
if [ -z "$CE_TEXT" ]; then
# No parseable shrunk block (unexpected — the signature check above
# already confirmed fast-check output). Fall back to the reproducing
# seed so we still emit a stable identity instead of silently deduping.
CE_TEXT="seed:${FAIL_SEED}"
fi
# Stable short id: first 12 hex chars of sha256 over the counterexample.
CE_HASH=$(printf '%s' "$CE_TEXT" | sha256sum | cut -c1-12)
# Machine-readable marker embedded in the issue body; the open-issue search
# below matches on it (and on the hash in the title) so identity travels
# with the issue regardless of any human title edits.
CE_MARKER="<!-- counterexample-hash: ${CE_HASH} -->"
export CE_HASH CE_MARKER
TITLE="${TITLE_PREFIX} [${CE_HASH}] (seed=${FAIL_SEED})"
# Best-effort dedup: skip if an open issue with the counterexample title
# prefix already exists. A failure of this check must NOT block creation.
# Dedup on the counterexample hash: skip only if an OPEN issue already
# carries this exact hash (in its title or its body marker). A different
# counterexample has a different hash and is NOT deduped. A failure of this
# check must NOT block creation.
EXISTING=""
if EXISTING=$(curl -sS \
-H "Authorization: token ${GITHUB_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues?state=open&limit=100"); then
if printf '%s' "$EXISTING" \
| node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{let a;try{a=JSON.parse(s)}catch{process.exit(1)}if(!Array.isArray(a))process.exit(1);const p=process.env.TITLE_PREFIX;process.exit(a.some(i=>typeof i.title==="string"&&i.title.startsWith(p))?0:1)})'; then
echo "An open '${TITLE_PREFIX}' issue already exists — skipping creation."
| node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{let a;try{a=JSON.parse(s)}catch{process.exit(1)}if(!Array.isArray(a))process.exit(1);const h=process.env.CE_HASH,m=process.env.CE_MARKER;process.exit(a.some(i=>(typeof i.title==="string"&&i.title.includes(h))||(typeof i.body==="string"&&i.body.includes(m)))?0:1)})'; then
echo "An open issue for counterexample ${CE_HASH} already exists — skipping creation."
exit 0
fi
fi
# Build the JSON body with the test output SAFELY escaped (never hand-
# interpolate the counterexample into JSON).
BODY_TEXT=$(printf 'A nightly property fuzz SHARD failed with a fast-check counterexample.\n\n- failing shard seed: `%s`\n- NUM_RUNS (per shard): `%s`\n- run: %s\n\nReproduce locally:\n\n```\nPROPERTY_SEED=%s PROPERTY_NUM_RUNS=%s pnpm --filter @docmost/prosemirror-markdown exec vitest run test/generative/\n```\n\nfast-check shrinks the failure to a minimal counterexample. Commit it as a permanent fixture under `packages/prosemirror-markdown/test/fixtures/counterexamples/` + a case in `counterexamples.test.ts`, then FIX the converter (do not weaken a property). See `packages/prosemirror-markdown/README.md`.\n\nTail of the test output (contains the shrunk counterexample):\n\n```\n%s\n```\n' \
"$FAIL_SEED" "$NUM_RUNS" "$RUN_URL" "$FAIL_SEED" "$NUM_RUNS" "$(tail -n 120 property-output.txt)")
BODY_TEXT=$(printf 'A nightly property fuzz SHARD failed with a fast-check counterexample.\n\n- counterexample hash: `%s`\n- failing shard seed: `%s`\n- NUM_RUNS (per shard): `%s`\n- run: %s\n\nReproduce locally:\n\n```\nPROPERTY_SEED=%s PROPERTY_NUM_RUNS=%s pnpm --filter @docmost/prosemirror-markdown exec vitest run test/generative/\n```\n\nfast-check shrinks the failure to a minimal counterexample. Commit it as a permanent fixture under `packages/prosemirror-markdown/test/fixtures/counterexamples/` + a case in `counterexamples.test.ts`, then FIX the converter (do not weaken a property). See `packages/prosemirror-markdown/README.md`.\n\nTail of the test output (contains the shrunk counterexample):\n\n```\n%s\n```\n\n%s\n' \
"$CE_HASH" "$FAIL_SEED" "$NUM_RUNS" "$RUN_URL" "$FAIL_SEED" "$NUM_RUNS" "$(tail -n 120 property-output.txt)" "$CE_MARKER")
jq -n --arg title "$TITLE" --arg body "$BODY_TEXT" \
'{title: $title, body: $body}' > payload.json
+41 -13
View File
@@ -25,37 +25,65 @@ jobs:
# filename sorts BEFORE migrations already applied on the target branch (and
# thus in prod). The Kysely startup migrator rejects that as "corrupted
# migrations" and crash-loops the app on boot (incident #361). This gate fails
# the PR so the migration is renamed to a current timestamp before merge. Only
# runs for pull_request events (needs a base branch to diff against).
# the PR so the migration is renamed to a current timestamp before merge.
# Runs for pull_request (diff against the base branch) AND for push (#476
# retrospective: a DIRECT push to develop used to bypass this PR-only gate
# entirely — now the push is diffed against its `before` SHA; workflow_call
# from develop.yml inherits the caller's push event). workflow_dispatch has
# nothing to diff against and still skips the job.
migration-order:
if: github.event_name == 'pull_request'
if: github.event_name == 'pull_request' || github.event_name == 'push'
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout (full history for the base-branch diff)
- name: Checkout (full history for the base diff)
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Added migrations must sort after the newest on the base branch
- name: Added migrations must sort after the newest on the base
env:
TARGET_BRANCH: ${{ github.base_ref }}
BEFORE_SHA: ${{ github.event.before }}
run: |
set -euo pipefail
MIG_DIR="apps/server/src/database/migrations"
# checkout above already did fetch-depth:0 (full history). Fetch the base
# WITHOUT --depth (a shallow graft would truncate the base history and
# break the merge-base when the base has moved ahead of the PR merge —
# exactly the long-branch-vs-moving-base case this gate guards, #361).
git fetch --no-tags origin "$TARGET_BRANCH"
newest_on_target=$(git ls-tree -r --name-only "origin/${TARGET_BRANCH}" "$MIG_DIR" | sort | tail -1)
if [ "${GITHUB_EVENT_NAME}" = "pull_request" ]; then
# checkout above already did fetch-depth:0 (full history). Fetch the base
# WITHOUT --depth (a shallow graft would truncate the base history and
# break the merge-base when the base has moved ahead of the PR merge —
# exactly the long-branch-vs-moving-base case this gate guards, #361).
git fetch --no-tags origin "$TARGET_BRANCH"
BASE="origin/${TARGET_BRANCH}"
else
# push event: compare against the pre-push tip of the branch.
if [ "$BEFORE_SHA" = "0000000000000000000000000000000000000000" ]; then
echo "::notice::branch creation push — nothing to compare"
exit 0
fi
if ! git cat-file -e "${BEFORE_SHA}^{commit}" 2>/dev/null; then
# The before-SHA is not in the clone (a force-push rewrote history).
# One recovery attempt — refresh every remote head (cheap: the
# checkout is already fetch-depth:0); a fetch failure aborts via
# `set -e`, which is fail-closed too.
git fetch --no-tags origin '+refs/heads/*:refs/remotes/origin/*'
fi
if ! git cat-file -e "${BEFORE_SHA}^{commit}" 2>/dev/null; then
# FAIL-CLOSED: without the before-SHA there is no base to prove the
# ordering against, and a gate whose job is to BLOCK must not guess.
echo "::error::force-push detected — verify migration order manually, then re-run via workflow_dispatch"
exit 1
fi
BASE="$BEFORE_SHA"
fi
newest_on_target=$(git ls-tree -r --name-only "$BASE" "$MIG_DIR" | sort | tail -1)
# NO `|| true`: a diff failure (e.g. an unresolved merge-base) must fail
# the job CLOSED — a gate whose job is to BLOCK must never pass on error.
# `set -e` above already aborts on a non-zero diff exit.
added=$(git diff --diff-filter=A --name-only "origin/${TARGET_BRANCH}...HEAD" -- "$MIG_DIR")
added=$(git diff --diff-filter=A --name-only "${BASE}...HEAD" -- "$MIG_DIR")
bad=0
for f in $added; do
if [[ "$f" < "$newest_on_target" || "$f" == "$newest_on_target" ]]; then
echo "::error::Migration $f sorts at or before the newest on ${TARGET_BRANCH} ($newest_on_target) — rename it with a CURRENT timestamp before merge (do not change its contents). See incident #361."
echo "::error::Migration $f sorts at or before the newest on the base ($newest_on_target) — rename it with a CURRENT timestamp before merge (do not change its contents). See incident #361."
bad=1
fi
done
+4
View File
@@ -29,6 +29,10 @@ packages/mcp/build/
# is a build artifact like build/ — never committed, always fresh.
packages/mcp/src/registry-stamp.generated.ts
# token-estimate compiled output (#490; built in CI/Docker via `pnpm build` /
# the server `pretest`, never committed, so src/ and prod can never diverge).
packages/token-estimate/dist/
# Logs
logs
*.log
+2
View File
@@ -471,6 +471,8 @@ Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirro
- The version string shown in the UI comes from `APP_VERSION` (CI/Docker) or `git describe --tags --always` (local), resolved in `vite.config.ts` — not from `package.json`.
- Server TS config is permissive (`noImplicitAny: false`, `strictNullChecks: false`, `no-explicit-any` lint disabled). Follow the existing relaxed style rather than tightening types broadly.
- Dependency versions are heavily pinned via `pnpm.overrides` and `pnpm.patchedDependencies` (`scimmy`, `yjs`, `ai`) in the root `package.json`. Don't bump pinned/patched deps casually; the patches and overrides exist for compatibility/security reasons. The `ai@6.0.134` patch carries TWO independent server fixes, each with its own tripwire test: (1) it disables the SDK's O(n²) cumulative `partialOutput` accumulation when no output strategy is requested (server heap OOM on long agent runs, #184; tripwire: `apps/server/src/integrations/ai/ai-sdk-partial-output.patch.spec.ts`); (2) it fixes `writeToServerResponse`'s drain-hang — the loop awaited only `"drain"` under backpressure, so a mid-write client disconnect parked the pipe forever and leaked the reader/buffers until restart; it now races `"drain"` against `"close"`/`"error"`, cancels the reader on disconnect, and swallows the fire-and-forget read rejection (#486; tripwire: `apps/server/src/integrations/ai/ai-sdk-drain-hang.patch.spec.ts`). Both tripwires assert BOTH installed dist builds carry their patch marker. The patch MUST be re-created via `pnpm patch` when bumping `ai`.
- **Upstream tracking (report the analysis upstream, don't just carry it):** both `ai` fixes and the hocuspocus one are candidates for upstreaming so we can eventually drop the local patch — the analysis is already written up in each patch's `PATCH(...)` header comments. File (a) an upstream **issue** on `vercel/ai` for the O(n²) cumulative `partialOutput` accumulation (heap OOM), (b) an upstream **issue** on `vercel/ai` for the `writeToServerResponse` drain-hang, and (c) an upstream **PR** on `@hocuspocus/server` for the connect-vs-unload race (local marker `PATCH(gitmost #401)` in `patches/@hocuspocus__server@3.4.4.patch`). Do NOT edit the patch files to add links — the patch bytes feed `patch_hash` in `pnpm-lock.yaml` (`ai@6.0.134``e8c599b3…`), so any content change there desyncs the lockfile pin and breaks `pnpm install`; keep upstream references here instead.
- **`ai` version is split across the monorepo and MUST be aligned deliberately, NOT casually:** the server pins `ai@6.0.134` (patched, exact — the `patchedDependencies` key forces that version), while the client declares `ai@6.0.207` (unpatched — the server-side `writeToServerResponse`/`partialOutput` fixes are dead code in the browser, so the mismatch is currently benign but is real drift). Alignment is a **planned, install-gated step**, never a bare `package.json` edit: (1) choose the target version; (2) re-create ALL THREE patch hunks (partialOutput publish-each, the `DefaultStreamTextResult` lazy-`output` wiring, and the drain-hang race) against the target dist via `pnpm patch` — the line offsets shift between versions, so the current patch WILL fail to apply as-is; (3) run a full `pnpm install` so the lockfile + new `patch_hash` regenerate together; (4) confirm both tripwire specs still find their markers. `pnpm install` FAILS HARD on an unapplied patch — that failure is the guardrail, so treat the port as a deliberate plan rather than discovering it as a deploy-time surprise.
- **The MCP tool inventory in `SERVER_INSTRUCTIONS` is GENERATED from the registry** (`packages/mcp/src/server-instructions.ts`: `buildToolInventory()` over `SHARED_TOOL_SPECS`) and spliced into the hand-written routing prose (`ROUTING_PROSE`). So adding/renaming/removing a **shared** spec in `packages/mcp/src/tool-specs.ts` auto-updates the `<tool_inventory>` — no manual `SERVER_INSTRUCTIONS` edit needed. Only an **inline** MCP-only tool (those registered via `server.registerTool(...)` in `index.ts`, not through the registry) needs a one-line entry in `INLINE_MCP_INVENTORY`. Enforced by `packages/mcp/test/unit/tool-inventory.test.mjs`, which fails when a registered tool is missing from the generated inventory (there is no `EXCEPTIONS` opt-out anymore — every tool must appear). Update `ROUTING_PROSE` when a tool's *intent guidance* (when-to-use) changes. `packages/mcp/build/` is gitignored and rebuilt in CI/Docker via `pnpm build` (same convention as `git-sync`/`prosemirror-markdown`) — never commit it; rebuild locally after editing to run the tests.
## CI / release
+85 -1
View File
@@ -129,6 +129,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- **A drifted comment suggestion can be re-synced instead of failing forever
with a 409.** A suggestion whose stored anchor no longer matched the live
document used to reject every apply attempt with an unrecoverable conflict; a
new resync path re-reads the live anchor so the suggestion applies against the
current text, and orphaned anchors (whose marked run was deleted) are
reconciled rather than left blocking. (#496)
- **Place several images side by side in a row.** A new "Inline (side by
side)" alignment mode in the image bubble menu renders consecutive inline
images as a row that wraps onto the next line on narrow screens. The row is
@@ -304,6 +311,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
longer controls whether a turn is a run — it now governs **only** the
browser-disconnect semantics (ON = detached/survives a disconnect; OFF = a
disconnect stops the run). (#487)
- **Vendor `ai` patch: upstream-tracking + version-alignment plan documented.**
The two local `ai@6.0.134` fixes (O(n²) `partialOutput` heap-OOM; the
`writeToServerResponse` drain-hang) and the hocuspocus connect-vs-unload race
now have explicit upstream-reporting and `ai`-version-alignment steps recorded
in `AGENTS.md` (client `ai@6.0.207` vs server `ai@6.0.134`-patched drift). The
patch bytes are unchanged — they feed the lockfile `patch_hash`, so the
alignment is called out as an install-gated plan rather than a bare version
bump. No runtime change.
- **Client markdown paste/copy and AI-chat rendering now go through the canonical
converter.** Pasting markdown into the editor, "Copy as markdown", the AI title
generator, and the AI-chat markdown renderer all now use
@@ -336,6 +351,40 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Fixed
- **MCP write tools no longer report a false failure that provokes a duplicate
write.** `drawioCreate` used to throw when the diagram landed as a NESTED block
(anchored inside a callout or table cell) because there is no `#<index>` handle
for it — but the diagram was already written, so a retry-prone agent re-created
it and produced a duplicate. It now returns success with `nodeId: null` plus a
warning that explains the write landed and how to re-read it (via
`getOutline` / `getPageJson` by `attachmentId`). Separately, when the live
collaboration-session cache hits its LRU entry cap, evicting a session whose
write is still in flight no longer rejects that write as a hard failure — it is
reported as INDETERMINATE ("the update may already have persisted; verify
before retry") so the agent re-reads instead of blind-retrying, and a
still-connecting session is no longer picked as an idle eviction victim by a
parallel acquire. (#494)
- **A long AI chat no longer bricks on the model's context window, and each turn
stops re-persisting the whole tool-output history.** Tool outputs are now
stored ONCE, in `metadata.parts`; the `tool_calls` trace keeps only per-step
outcome flags (a v2 trace shape), ending the O(N²) write amplification that
re-wrote every prior output on every step (measured on a live Postgres via the
`pg_current_wal_lsn()` delta: the trace column shrank ~3200×, the full
assistant row ~51%). The persisted record is unchanged in content — the full
history still lives in `metadata.parts`. At REPLAY time only, the history sent
to the provider is now bounded by a deterministic, prompt-cache-friendly token
budget: `floor(0.7 × chatContextWindow)` when a window is configured (no cap —
anti-brick protection, not a cost limiter), a flat 100k fallback for installs
with no window set (exactly the ones that hit terminal overflow), or off when
the window is explicitly `0`. Trimming truncates old tool outputs first, then
mechanically collapses the oldest turns, always keeping the recent turns full
and the tool-call/result pairing balanced. A provider context-overflow 400 is
now classified and used as a reactive signal: the row is stamped so the NEXT
turn re-trims aggressively (0.5×), which un-bricks a chat that just 400'd. The
client token badge and the server budgeter now share one estimator (new
`@docmost/token-estimate` package) so they can never diverge. Deferred-tool
activation is also cached in the chat metadata to avoid re-resolving it each
turn. (#490)
- **A chat with one malformed message part no longer 500s on every turn, and a
failed send no longer duplicates the user's message.** Incoming client parts
are now whitelisted to `text` (a forged tool-result part can no longer reach
@@ -352,7 +401,29 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
body-timeout so a legitimate >1-min idle between the model's tool calls no
longer breaks a long-lived SSE socket (new `AI_MCP_SSE_BODY_TIMEOUT_MS`, default
10 min; see `.env.example`). (#489)
- **Decisions on comment suggestions now leave a durable audit record.**
Applying or dismissing a comment suggestion hard-deletes the (childless)
subject comment, so the only surviving trace of who decided what is the audit
event — but the audit trail was wired to a Noop service that silently
swallowed every event. The trail is now DB-backed, so
`comment.suggestion_applied` / `comment.suggestion_dismissed` (and the other
comment-decision events) persist to the `audit` table and can be reviewed
after the comment is gone. A persistence failure is still swallowed with a
warning so it never breaks the originating request. (#496)
- **Applying a comment suggestion no longer strips the replaced run's inline
formatting.** The suggested text was re-inserted carrying only the comment
anchor mark, silently dropping bold/italic/code/link on the affected run; the
prevailing formatting of the replaced run is now carried onto the applied
text. (#496)
- **Markdown round-trips no longer silently drop a line that opens with a block
trigger.** When a document is exported to Markdown and re-imported (git-sync
stabilize, agent writes), a paragraph or continuation line (after a hard break)
that begins with a block marker — an ATX heading `#`, a blockquote/callout `>`,
a list marker (`-`/`*`/`+`/`N.`/`N)`), a code fence, a table `|`, a thematic
break (`---`), or a setext underline (`--`, `----`, or a lone `=`) — is now
backslash-escaped so it round-trips as text instead of being re-parsed into a
heading/list/quote/rule and losing its content. Front-matter stripping is
scoped to the import path only. (#493)
- **The server no longer runs out of heap during long autonomous agent runs.** A
new pnpm patch on `ai@6.0.134` stops the SDK from building a cumulative
snapshot of the ENTIRE turn text on every streamed text-delta when no output
@@ -469,6 +540,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
through that exact share (its own share or an ancestor `includeSubPages`
share); any other value now returns the generic "not found" instead of
serving the page. (#218)
- **MCP tool-allowlist semantics flipped: an empty `[]` now means deny-all
(previously it was coerced to "no restrictions").** For an external MCP server,
a stored `tool_allowlist` of `[]` now denies **every** tool of that server
(zero tools reach the agent) instead of being treated as an empty/unset filter
that allowed all of them. A corrupt or non-array stored value now **fails
closed** to deny-all rather than silently allowing everything. The admin form
no longer silently widens an existing deny-all server: leaving its tag field
empty preserves `[]` (deny-all) on save instead of NULL-ing the column to
allow-all, so a routine rename/toggle can no longer grant the agent every tool.
"No restrictions" is still expressible — a genuinely unrestricted server stores
NULL, and clearing the field on such a server keeps it NULL. Operationally
significant: audit any server that was created or left with a literal `[]`, as
it now exposes no tools until an explicit allowlist (or NULL) is set. (#476)
- **Tool and provider error text no longer leaks to anonymous readers in the
public-share AI chat.** A failing tool's raw error (which could carry an
+1
View File
@@ -22,6 +22,7 @@
"@casl/react": "5.0.1",
"@docmost/editor-ext": "workspace:*",
"@docmost/prosemirror-markdown": "workspace:*",
"@docmost/token-estimate": "workspace:*",
"@excalidraw/excalidraw": "0.18.0-3a5ef40",
"@mantine/core": "8.3.18",
"@mantine/dates": "8.3.18",
+230 -78
View File
@@ -256,6 +256,9 @@
"Invite link": "Ссылка для приглашения",
"Copy": "Копировать",
"Copy to space": "Копировать в пространство",
"Copy chat": "Копировать чат",
"Dock to sidebar": "Закрепить в боковой панели",
"Undock": "Открепить",
"Copied": "Скопировано",
"Failed to export chat": "Не удалось экспортировать чат",
"Duplicate": "Дублировать",
@@ -285,6 +288,9 @@
"Alt text": "Альтернативный текст",
"Describe this for accessibility.": "Опишите это для специальных возможностей.",
"Add a description": "Добавить описание",
"Caption": "Подпись",
"Add a caption": "Добавить подпись",
"Shown below the image.": "Отображается под изображением.",
"Justify": "По ширине",
"Merge cells": "Объединить ячейки",
"Split cell": "Разделить ячейку",
@@ -388,22 +394,6 @@
"Quote": "Цитата",
"Image": "Изображение",
"Audio": "Аудио",
"Transcribe": "Транскрибировать",
"Transcribing…": "Транскрибация…",
"No speech detected": "Речь не распознана",
"Transcription failed": "Не удалось распознать речь",
"Voice dictation is not configured": "Голосовой ввод не настроен",
"Start dictation": "Начать диктовку",
"Stop recording": "Остановить запись",
"Microphone access denied": "Доступ к микрофону запрещён",
"No microphone found": "Микрофон не найден",
"Microphone is unavailable or already in use": "Микрофон недоступен или уже используется",
"Could not start recording": "Не удалось начать запись",
"Audio recording is not available in this browser/context": "Запись аудио недоступна в этом браузере/контексте",
"Dictation": "Диктовка",
"Dictation becomes available once the page finishes connecting": "Диктовка станет доступна после подключения к документу",
"No connection to the collaboration server — dictation unavailable": "Нет связи с сервером совместного редактирования — диктовка недоступна",
"This page is read-only": "Страница открыта только для чтения",
"Embed PDF": "Встроить PDF",
"Upload and embed a PDF file.": "Загрузите и встроите PDF-файл.",
"Embed as PDF": "Встроить как PDF",
@@ -419,9 +409,6 @@
"Footnote {{number}}": "Сноска {{number}}",
"Go to footnote": "Перейти к сноске",
"Back to reference": "Вернуться к ссылке",
"Back to references": "Вернуться к ссылкам",
"Back to reference {{label}}": "Вернуться к ссылке {{label}}",
"Empty footnote": "Пустая сноска",
"Math inline": "Строчная формула",
"Insert inline math equation.": "Вставить математическое выражение в строку.",
"Math block": "Блок формулы",
@@ -447,6 +434,9 @@
"{{count}} command available_other": "Доступно {{count}} команд",
"{{count}} result available_one": "Доступен 1 результат",
"{{count}} result available_other": "Доступно {{count}} результатов",
"{{count}} result found_one": "Найден {{count}} результат",
"{{count}} result found_few": "Найдено {{count}} результата",
"{{count}} result found_other": "Найдено {{count}} результатов",
"Equal columns": "Равные столбцы",
"Left sidebar": "Левая боковая панель",
"Right sidebar": "Правая боковая панель",
@@ -456,6 +446,7 @@
"Names do not match": "Названия не совпадают",
"Today, {{time}}": "Сегодня, {{time}}",
"Yesterday, {{time}}": "Вчера, {{time}}",
"now": "сейчас",
"Space created successfully": "Пространство успешно создано",
"Space updated successfully": "Пространство успешно обновлено",
"Space deleted successfully": "Пространство успешно удалено",
@@ -559,6 +550,7 @@
"Add 2FA method": "Добавить метод 2FA",
"Backup codes": "Резервные коды",
"Disable": "Отключить",
"disabled": "отключено",
"Invalid verification code": "Недействительный код подтверждения",
"New backup codes have been generated": "Новые резервные коды сгенерированы",
"Failed to regenerate backup codes": "Не удалось заново сгенерировать резервные коды",
@@ -702,62 +694,6 @@
"AI search": "Поиск ИИ",
"AI Answer": "Ответ ИИ",
"Ask AI": "Спросить ИИ",
"AI agent": "AI-агент",
"Take a look at the current document": "Посмотри текущий документ",
"Start automatically": "Запускать автоматически",
"When on, picking this role sends a launch message and starts the chat. When off, the role is selected and you type the first message yourself.": "Когда включено, выбор этой роли отправляет стартовое сообщение и начинает чат. Когда выключено, роль выбирается, а первое сообщение вы вводите сами.",
"Launch message": "Стартовое сообщение",
"Sent automatically when this role is picked. Leave empty to use the default text. Ignored when “Start automatically” is off.": "Отправляется автоматически при выборе этой роли. Оставьте пустым, чтобы использовать текст по умолчанию. Игнорируется, когда «Запускать автоматически» выключено.",
"AI agent is typing…": "AI-агент печатает…",
"{{name}} is typing…": "{{name}} печатает…",
"Thinking…": "Думаю…",
"Thinking… · {{count}} tokens": "Думаю… · {{count}} токенов",
"Thinking… · {{count}} tokens_one": "Думаю… · {{count}} токен",
"Thinking… · {{count}} tokens_few": "Думаю… · {{count}} токена",
"Thinking… · {{count}} tokens_many": "Думаю… · {{count}} токенов",
"Thinking · {{count}} tokens": "Размышления · {{count}} токенов",
"Thinking · {{count}} tokens_one": "Размышления · {{count}} токен",
"Thinking · {{count}} tokens_few": "Размышления · {{count}} токена",
"Thinking · {{count}} tokens_many": "Размышления · {{count}} токенов",
"Agent role": "Роль агента",
"AI chat": "AI-чат",
"AI chat is disabled for this workspace.": "AI-чат отключён для этого рабочего пространства.",
"Ask a question about this documentation.": "Задайте вопрос об этой документации.",
"Ask a question…": "Задайте вопрос…",
"Ask the AI agent anything about your workspace.": "Спросите AI-агента о чём угодно по вашему рабочему пространству.",
"Ask the AI agent…": "Спросите AI-агента…",
"Copy chat": "Копировать чат",
"Dock to sidebar": "Закрепить в боковой панели",
"Undock": "Открепить",
"Created successfully": "Успешно создано",
"Context size / model limit": "Размер контекста / лимит модели",
"Context window (tokens)": "Окно контекста (токены)",
"Shown as used / total in the chat header. Leave empty to hide the limit.": "Показывается в шапке чата как использовано / всего. Пусто — лимит скрыт.",
"Delete this chat?": "Удалить этот чат?",
"Deleted successfully": "Успешно удалено",
"AI agent «{{role}}» on behalf of {{person}}": "AI-агент «{{role}}» от имени {{person}}",
"AI agent {{name}}": "AI-агент {{name}}",
"Failed to delete chat": "Не удалось удалить чат",
"Failed to rename chat": "Не удалось переименовать чат",
"Failed": "Ошибка",
"OK · {{n}}": "OK · {{n}}",
"Test": "Тест",
"No tools available": "Инструменты недоступны",
"Available tools": "Доступные инструменты",
"Minimize": "Свернуть",
"No chats yet.": "Чатов пока нет.",
"Send": "Отправить",
"Send when the agent finishes": "Отправить, когда агент закончит",
"Queue message": "Поставить в очередь",
"Remove queued message": "Убрать из очереди",
"Send now": "Отправить сейчас",
"Interrupt and send now": "Прервать и отправить сейчас",
"Something went wrong": "Что-то пошло не так",
"Stop": "Стоп",
"The AI agent could not respond. Please try again.": "AI-агент не смог ответить. Попробуйте ещё раз.",
"The AI provider is not configured. Ask an administrator to set it up.": "AI-провайдер не настроен. Попросите администратора настроить его.",
"Universal assistant": "Универсальный ассистент",
"You": "Вы",
"AI is thinking...": "ИИ обрабатывает запрос...",
"Thinking": "Думаю",
"Ask a question...": "Задайте вопрос...",
@@ -784,8 +720,40 @@
"Manage API keys for all users in the workspace. View the <anchor>API documentation</anchor> for usage details.": "Управляйте API-ключами для всех пользователей в рабочем пространстве. Смотрите <anchor>документацию по API</anchor> для получения информации об использовании.",
"View the <anchor>API documentation</anchor> for usage details.": "Смотрите <anchor>документацию по API</anchor> для получения информации об использовании.",
"View the <anchor>MCP documentation</anchor>.": "Смотрите <anchor>документацию по MCP</anchor>.",
"Instructions": нструкции",
"AI / Models": И / Модели",
"AI / External tools (MCP)": "ИИ / Внешние инструменты (MCP)",
"Add server": "Добавить сервер",
"Edit server": "Изменить сервер",
"Delete server": "Удалить сервер",
"Are you sure you want to delete this MCP server?": "Вы уверены, что хотите удалить этот MCP-сервер?",
"No external servers configured": "Внешние серверы не настроены",
"Server name": "Имя сервера",
"Transport": "Транспорт",
"URL": "URL",
"Authorization header": "Заголовок авторизации",
"Tool allowlist": "Список разрешённых инструментов",
"Optional. Leave empty to allow all tools the server exposes.": "Необязательно. Оставьте пустым, чтобы разрешить все инструменты, которые предоставляет сервер.",
"Optional guidance for the agent on how and when to use this server's tools. Injected into the system prompt. The server's tools are namespaced as \"<server name>_*\".": "Необязательное указание агенту, как и когда использовать инструменты этого сервера. Добавляется в системный промпт. Инструменты сервера именуются с префиксом «<имя сервера>_*».",
"Test": "Тест",
"Available tools": "Доступные инструменты",
"No tools available": "Инструменты недоступны",
"Failed": "Ошибка",
"OK · {{n}}": "OK · {{n}}",
"Created successfully": "Успешно создано",
"Deleted successfully": "Успешно удалено",
"Clear": "Очистить",
"Provider": "Провайдер",
"•••• set": "•••• задан",
"Clear key": "Очистить ключ",
"Base URL": "Базовый URL",
"Chat model": "Модель чата",
"Embedding model": "Модель эмбеддингов",
"System message": "Системное сообщение",
"A built-in safety framework is always appended.": "Встроенный набор правил безопасности всегда добавляется автоматически.",
"Test connection": "Проверить соединение",
"Connection successful": "Соединение установлено",
"Connection failed": "Не удалось установить соединение",
"Only workspace admins can manage AI provider settings.": "Управлять настройками провайдера ИИ могут только администраторы рабочего пространства.",
"Sources": "Источники",
"AI Answers not available for attachments": "Ответы ИИ недоступны для вложений",
"No answer available": "Ответ недоступен",
@@ -1013,6 +981,7 @@
"Try again": "Попробовать снова",
"Untitled chat": "Чат без названия",
"No document": "Без документа",
"You": "Вы",
"What can I help you with?": "Чем я могу вам помочь?",
"Are you sure you want to revoke this {{credential}}": "Вы уверены, что хотите отозвать этот {{credential}}",
"Automatically provision users and groups from your identity provider via SCIM.": "Автоматически предоставляйте доступ пользователям и группам из вашего провайдера удостоверений через SCIM.",
@@ -1041,6 +1010,9 @@
"Page menu": "Меню страницы",
"Expand": "Развернуть",
"Collapse": "Свернуть",
"Expand all": "Развернуть все",
"Collapse all": "Свернуть все",
"Couldn't expand the tree: {{reason}}": "Не удалось развернуть дерево: {{reason}}",
"Comment menu": "Меню комментария",
"Group menu": "Меню группы",
"Show hidden breadcrumbs": "Показать скрытые хлебные крошки",
@@ -1077,7 +1049,7 @@
"Search pages and spaces...": "Поиск страниц и пространств...",
"No results found": "Результаты не найдены",
"You don't have permission to create pages here": "У вас нет прав на создание страниц здесь",
"Chat menu": "Меню чата",
"Chat menu for {{title}}": "Меню чата для {{title}}",
"API key menu": "Меню API-ключа",
"Jump to comment selection": "Перейти к выбору комментария",
"Slash commands": "Команды со слешем",
@@ -1131,6 +1103,9 @@
"Undo": "Отменить",
"Redo": "Повторить",
"Backlinks": "Обратные ссылки",
"Back to references": "Вернуться к ссылкам",
"Back to reference {{label}}": "Вернуться к ссылке {{label}}",
"Empty footnote": "Пустая сноска",
"Last updated by": "Последний изменивший",
"Last updated": "Последнее обновление",
"Stats": "Статистика",
@@ -1164,6 +1139,7 @@
"Page title": "Заголовок страницы",
"Page content": "Содержимое страницы",
"Member actions": "Действия с участником",
"Member actions for {{name}}": "Действия с участником {{name}}",
"Toggle password visibility": "Переключить видимость пароля",
"Send comment": "Отправить комментарий",
"Token actions": "Действия с токеном",
@@ -1183,11 +1159,187 @@
"Removed from favorites": "Удалено из избранного",
"Added {{name}} to favorites": "{{name}} добавлено в избранное",
"Removed {{name}} from favorites": "{{name}} удалено из избранного",
"Label added": "Метка добавлена",
"Label removed": "Метка удалена",
"Image updated": "Изображение обновлено",
"Unsupported image type": "Неподдерживаемый тип изображения",
"Member deactivated": "Участник деактивирован",
"Member activated": "Участник активирован",
"Name is required": "Укажите имя",
"Name must be 40 characters or fewer": "Имя должно содержать не более 40 символов",
"Group name must be at least 2 characters": "Название группы должно содержать не менее 2 символов",
"Group name must be 100 characters or fewer": "Название группы должно содержать не более 100 символов",
"Description must be 500 characters or fewer": "Описание должно содержать не более 500 символов",
"Invalid invitation link": "Недействительная ссылка-приглашение",
"Page menu for {{name}}": "Меню страницы для {{name}}",
"Create subpage of {{name}}": "Создать подстраницу для {{name}}",
"AI chat": "AI-чат",
"Ask a question about this documentation.": "Задайте вопрос об этой документации.",
"Ask a question…": "Задайте вопрос…",
"Thinking…": "Думаю…",
"Thinking… · {{count}} tokens": "Думаю… · {{count}} токенов",
"Thinking… · {{count}} tokens_one": "Думаю… · {{count}} токен",
"Thinking… · {{count}} tokens_few": "Думаю… · {{count}} токена",
"Thinking… · {{count}} tokens_many": "Думаю… · {{count}} токенов",
"Thinking… · {{count}} tokens_other": "Думаю… · {{count}} токенов",
"Thinking · {{count}} tokens": "Размышления · {{count}} токенов",
"Thinking · {{count}} tokens_one": "Размышления · {{count}} токен",
"Thinking · {{count}} tokens_few": "Размышления · {{count}} токена",
"Thinking · {{count}} tokens_many": "Размышления · {{count}} токенов",
"Thinking · {{count}} tokens_other": "Размышления · {{count}} токенов",
"The assistant is unavailable right now. Please try again.": "Ассистент сейчас недоступен. Попробуйте ещё раз.",
"Public share assistant": "Ассистент публичного доступа",
"Let anonymous visitors of public shares ask an AI assistant scoped to that share's pages. You pay for the tokens.": "Позвольте анонимным посетителям публичных ссылок обращаться к ИИ-ассистенту в рамках страниц этой публикации. Токены оплачиваете вы.",
"Public assistant model": "Модель публичного ассистента",
"Defaults to the chat model": "По умолчанию используется модель чата",
"Optional cheaper model id for the public assistant. Empty uses the chat model above.": "Необязательный более дешёвый идентификатор модели для публичного ассистента. Если пусто, используется модель чата выше.",
"Assistant identity": "Личность ассистента",
"Pick an agent role whose persona the public assistant adopts. The safety rules always still apply.": "Выберите роль агента, чью личность примет публичный ассистент. Правила безопасности всегда остаются в силе.",
"Built-in assistant persona": "Встроенная личность ассистента",
"Minimize": "Свернуть",
"Context size / model limit": "Размер контекста / лимит модели",
"Context window (tokens)": "Окно контекста (токены)",
"Shown as used / total in the chat header. Leave empty to hide the limit.": "Показывается в шапке чата как использовано / всего. Пусто — лимит скрыт.",
"AI agent": "AI-агент",
"Take a look at the current document": "Посмотри текущий документ",
"AI agent is typing…": "AI-агент печатает…",
"{{name}} is typing…": "{{name}} печатает…",
"Send": "Отправить",
"Send when the agent finishes": "Отправить, когда агент закончит",
"Queue message": "Поставить в очередь",
"Remove queued message": "Убрать из очереди",
"Send now": "Отправить сейчас",
"Interrupt and send now": "Прервать и отправить сейчас",
"Stop": "Стоп",
"Response stopped.": "Ответ остановлен.",
"Connection lost — the answer was interrupted.": "Соединение потеряно — ответ был прерван.",
"Response stopped (manually or the connection dropped).": "Ответ остановлен (вручную или из-за разрыва соединения).",
"Chat menu": "Меню чата",
"No chats yet.": "Чатов пока нет.",
"Delete this chat?": "Удалить этот чат?",
"Ask the AI agent…": "Спросите AI-агента…",
"Ask the AI agent anything about your workspace.": "Спросите AI-агента о чём угодно по вашему рабочему пространству.",
"Failed to rename chat": "Не удалось переименовать чат",
"Failed to delete chat": "Не удалось удалить чат",
"Something went wrong": "Что-то пошло не так",
"AI chat is disabled for this workspace.": "AI-чат отключён для этого рабочего пространства.",
"The AI provider is not configured. Ask an administrator to set it up.": "AI-провайдер не настроен. Попросите администратора настроить его.",
"The AI agent could not respond. Please try again.": "AI-агент не смог ответить. Попробуйте ещё раз.",
"Searched pages": "Поиск по страницам",
"Read page": "Прочитана страница",
"Created page": "Создана страница",
"Updated page": "Обновлена страница",
"Renamed page": "Переименована страница",
"Moved page": "Перемещена страница",
"Deleted page (to trash)": "Удалена страница (в корзину)",
"Commented": "Добавлен комментарий",
"Resolved comment": "Комментарий решён",
"Ran tool {{name}}": "Выполнен инструмент {{name}}",
"AI agent «{{role}}» on behalf of {{person}}": "AI-агент «{{role}}» от имени {{person}}",
"AI agent {{name}}": "AI-агент {{name}}",
"Endpoints": "Эндпоинты",
"where we fetch models": "откуда мы получаем модели",
"All endpoints are OpenAI-compatible. Point the Base URL at OpenAI, OpenRouter, a local Ollama, or any self-hosted server.": "Все эндпоинты совместимы с OpenAI. Укажите в базовом URL адрес OpenAI, OpenRouter, локального Ollama или любого self-hosted сервера.",
"Chat / LLM": "Чат / LLM",
"root": "корневой",
"Semantic search": "Семантический поиск",
"Voice / STT": "Голос / STT",
"Voice dictation": "Голосовой ввод",
"Streaming dictation": "Потоковый голосовой ввод",
"Transcribe as you speak, cutting on pauses": "Транскрибирование по мере речи, с разбивкой на паузах",
"Voice dictation is not available yet.": "Голосовой ввод пока недоступен.",
"Test endpoint": "Проверить эндпоинт",
"Save and test": "Сохранить и проверить",
"Save endpoints": "Сохранить эндпоинты",
"Configured and enabled": "Настроено и включено",
"Configured but disabled": "Настроено, но отключено",
"Enabled but not configured": "Включено, но не настроено",
"Not configured": "Не настроено",
"External tools": "Внешние инструменты",
"Gitmost as MCP client": "Gitmost как MCP-клиент",
"Servers the agent calls out to.": "Серверы, к которым обращается агент.",
"MCP server": "MCP-сервер",
"expose the workspace": "открыть доступ к рабочему пространству",
"Enable MCP server": "Включить MCP-сервер",
"Exposes the workspace as an MCP server at /mcp — this provides a capability, it doesn't consume a model.": "Открывает рабочее пространство как MCP-сервер по адресу /mcp — это предоставляет возможность, а не потребляет модель.",
"Resolves to {{url}}": "Разрешается в {{url}}",
"Model": "Модель",
"Done": "Готово",
"shared prompt · safety framework appended automatically": "общий промпт · правила безопасности добавляются автоматически",
"/v1/chat/completions · root endpoint — Embeddings and Voice inherit its URL and key": "/v1/chat/completions · корневой эндпоинт — Эмбеддинги и Голос наследуют его URL и ключ",
"/v1/embeddings · embeds pages so semantic search can find them": "/v1/embeddings · создаёт эмбеддинги страниц, чтобы их находил семантический поиск",
"/v1/audio/transcriptions · works with local whisper (speaches / faster-whisper-server)": "/v1/audio/transcriptions · работает с локальным whisper (speaches / faster-whisper-server)",
"Vector search · requires pgvector": "Векторный поиск · требуется pgvector",
"Embedding API key": "API-ключ для эмбеддингов",
"Embeddings": "Эмбеддинги",
"Leave empty to use the chat API key": "Оставьте пустым, чтобы использовать API-ключ чата",
"Leave empty to use the chat base URL": "Оставьте пустым, чтобы использовать базовый URL чата",
"Reindex now": "Переиндексировать сейчас",
"Start dictation": "Начать диктовку",
"Stop recording": "Остановить запись",
"Transcribing…": "Транскрибация…",
"Microphone access denied": "Доступ к микрофону запрещён",
"No microphone found": "Микрофон не найден",
"Could not start recording": "Не удалось начать запись",
"Transcription failed": "Не удалось распознать речь",
"Transcribe": "Транскрибировать",
"No speech detected": "Речь не распознана",
"Voice dictation is not configured": "Голосовой ввод не настроен",
"Microphone is unavailable or already in use": "Микрофон недоступен или уже используется",
"Audio recording is not available in this browser/context": "Запись аудио недоступна в этом браузере/контексте",
"Dictation": "Диктовка",
"Dictation becomes available once the page finishes connecting": "Диктовка станет доступна после подключения к документу",
"No connection to the collaboration server — dictation unavailable": "Нет связи с сервером совместного редактирования — диктовка недоступна",
"This page is read-only": "Страница открыта только для чтения",
"Request format": "Формат запроса",
"How transcription requests are sent to the endpoint": "Как запросы на транскрибирование отправляются на эндпоинт",
"OpenAI-compatible (multipart/form-data)": "Совместимо с OpenAI (multipart/form-data)",
"OpenRouter (JSON, base64 audio)": "OpenRouter (JSON, аудио в base64)",
"Dictation language": "Язык диктовки",
"Auto-detect": "Автоопределение",
"Spoken language hint sent to the transcription model. Auto-detect lets the model decide.": "Подсказка языка речи для модели транскрипции. «Автоопределение» оставляет выбор за моделью.",
"Agent role": "Роль агента",
"Universal assistant": "Универсальный ассистент",
"Add role": "Добавить роль",
"Edit role": "Изменить роль",
"Role name": "Название роли",
"e.g. Proofreader": "напр. Корректор",
"Optional. Shown as the chat badge.": "Необязательно. Отображается как значок чата.",
"Optional. A short note about what this role does.": "Необязательно. Краткое описание того, что делает эта роль.",
"Instructions": "Инструкции",
"The built-in safety framework is always added automatically.": "Встроенный набор правил безопасности всегда добавляется автоматически.",
"Model provider override": "Переопределение провайдера модели",
"Optional. Defaults to the workspace provider.": "Необязательно. По умолчанию используется провайдер рабочего пространства.",
"Model override": "Переопределение модели",
"Optional. Defaults to the workspace model.": "Необязательно. По умолчанию используется модель рабочего пространства.",
"e.g. gpt-4o-mini": "напр. gpt-4o-mini",
"If you choose a different provider, it must already be configured in AI settings.": "Если вы выбираете другого провайдера, он уже должен быть настроен в настройках ИИ.",
"Start automatically": "Запускать автоматически",
"When on, picking this role sends a launch message and starts the chat. When off, the role is selected and you type the first message yourself.": "Когда включено, выбор этой роли отправляет стартовое сообщение и начинает чат. Когда выключено, роль выбирается, а первое сообщение вы вводите сами.",
"Launch message": "Стартовое сообщение",
"Sent automatically when this role is picked. Leave empty to use the default text. Ignored when “Start automatically” is off.": "Отправляется автоматически при выборе этой роли. Оставьте пустым, чтобы использовать текст по умолчанию. Игнорируется, когда «Запускать автоматически» выключено.",
"Agent roles": "Роли агента",
"Reusable presets that shape the agent's behavior (and optionally its model). Picked when starting a new chat.": "Многоразовые пресеты, определяющие поведение агента (и, при желании, его модель). Выбираются при запуске нового чата.",
"No roles configured": "Роли не настроены",
"Delete role": "Удалить роль",
"Are you sure you want to delete this role?": "Вы уверены, что хотите удалить эту роль?",
"HTML embed": "HTML-вставка",
"Edit HTML embed": "Изменить HTML-вставку",
"HTML embed is disabled in this workspace": "HTML-вставки отключены в этом рабочем пространстве",
"Click to add HTML / CSS / JS": "Нажмите, чтобы добавить HTML / CSS / JS",
"This HTML/CSS/JS runs in a sandboxed frame and cannot access the viewer's session, cookies, or API.": "Этот HTML/CSS/JS выполняется в изолированном фрейме и не имеет доступа к сессии, cookie или API просматривающего.",
"<script>...</script>": "<script>...</script>",
"Height (px, blank = auto)": "Высота (px, пусто = авто)",
"advanced": "дополнительно",
"Enable HTML embed": "Включить HTML-вставки",
"Allow members to insert raw HTML/CSS/JavaScript blocks. The block renders in a sandboxed frame and cannot access the viewer's session, cookies, or API. Off by default.": "Разрешить участникам вставлять блоки с необработанным HTML/CSS/JavaScript. Блок отображается в изолированном фрейме и не имеет доступа к сессии, cookie или API просматривающего. По умолчанию выключено.",
"When enabled, any member can insert an HTML embed block. The toggle just enables or disables the block type workspace-wide.": "Когда включено, любой участник может вставить блок HTML-вставки. Переключатель просто включает или отключает этот тип блока во всём рабочем пространстве.",
"Embeds run inside a sandboxed iframe with a separate origin, so they cannot read or modify the page they are embedded in.": "Вставки выполняются в изолированном iframe с отдельным источником, поэтому они не могут читать или изменять страницу, в которую встроены.",
"Turning this off hides existing embeds (they render as a disabled placeholder) and stops serving them on public share pages.": "Отключение этой опции скрывает существующие вставки (они отображаются как отключённая заглушка) и прекращает их показ на публичных страницах.",
"Analytics / tracker": "Аналитика / трекер",
"Injected verbatim into the <head> of PUBLIC SHARE pages only (same-origin). For analytics snippets (Google Analytics, Yandex.Metrika, etc.). Admin only.": "Вставляется дословно в <head> только ПУБЛИЧНЫХ страниц (тот же источник). Для сниппетов аналитики (Google Analytics, Яндекс.Метрика и т. п.). Только для администраторов.",
"Go to login page": "Перейти на страницу входа",
"Move to space": "Переместить в пространство",
"Float left (wrap text)": "Обтекание слева",
"Float right (wrap text)": "Обтекание справа",
"Inline (side by side)": "В ряд",
@@ -1199,6 +1351,7 @@
"Showing {{count}} subpages_one": "Показано {{count}} подстраница",
"Showing {{count}} subpages_few": "Показано {{count}} подстраницы",
"Showing {{count}} subpages_many": "Показано {{count}} подстраниц",
"Showing {{count}} subpages_other": "Показано {{count}} подстраниц",
"Protocol": "Протокол",
"How chat requests are sent and how reasoning is surfaced": "Как отправляются запросы чата и как показывается reasoning",
"OpenAI-compatible (surfaces reasoning)": "OpenAI-совместимый (показывает reasoning)",
@@ -1268,7 +1421,6 @@
"Retry": "Повторить",
"The catalog is empty": "Каталог пуст",
"No role bundles are published for this language yet. Try switching the content language.": "Для этого языка ещё не опубликовано ни одного набора ролей. Попробуйте сменить язык контента.",
"No roles configured": "Роли не настроены",
"Already up to date": "Уже актуальна",
"Updated to the latest version": "Обновлено до последней версии",
"This role is no longer in the catalog": "Эта роль больше не представлена в каталоге",
@@ -1,5 +1,5 @@
import { describe, it, expect } from "vitest";
import { isChunkLoadError } from "./chunk-load-error-boundary";
import { isChunkLoadError, shouldAutoReload } from "./chunk-load-error-boundary";
// The detector decides whether a caught render error is a stale-deploy chunk-404
// (→ auto-reload to fetch the new manifest) vs a genuine app error (→ generic
@@ -35,3 +35,31 @@ describe("isChunkLoadError", () => {
expect(isChunkLoadError(err)).toBe(false);
});
});
// The window gate replaces the old one-shot flag: it must permit recovery across
// several deploys in one tab (each > window apart) while still stopping an infinite
// reload loop when a lazy chunk is permanently broken (a second failure < window).
describe("shouldAutoReload", () => {
const WINDOW = 5 * 60 * 1000;
const NOW = 1_000_000_000_000;
it("allows a reload when we have never auto-reloaded", () => {
expect(shouldAutoReload(NOW, null, WINDOW)).toBe(true);
});
it("allows a reload when the last one was 6 minutes ago (outside the window)", () => {
expect(shouldAutoReload(NOW, NOW - 6 * 60 * 1000, WINDOW)).toBe(true);
});
it("blocks a reload when the last one was 1 minute ago (inside the window)", () => {
expect(shouldAutoReload(NOW, NOW - 1 * 60 * 1000, WINDOW)).toBe(false);
});
it("blocks a reload exactly at the window boundary (not strictly older)", () => {
expect(shouldAutoReload(NOW, NOW - WINDOW, WINDOW)).toBe(false);
});
it("allows a reload when the stored timestamp is unparseable (NaN)", () => {
expect(shouldAutoReload(NOW, NaN, WINDOW)).toBe(true);
});
});
@@ -2,7 +2,25 @@ import { ReactNode } from "react";
import { ErrorBoundary } from "react-error-boundary";
import { Button, Center, Stack, Text } from "@mantine/core";
const RELOAD_FLAG = "chunk-reload-attempted";
// sessionStorage key holding the epoch-ms timestamp of the last automatic reload.
const RELOAD_AT_KEY = "chunk-reload-at";
// Allow at most one automatic reload per this window. A stale-deploy 404 is cured
// by a single reload, so anything inside the window is treated as a reload loop
// (permanently-broken chunk) and falls through to the manual UI. A window (rather
// than a one-shot flag) lets a SECOND deploy in the same tab's lifetime recover too.
const RELOAD_WINDOW_MS = 5 * 60 * 1000;
// Pure window decision, unit-tested in isolation: auto-reload only if we have never
// auto-reloaded (lastReloadAt null/NaN) or the last one was strictly older than the
// window. Anything inside the window is suppressed to break an infinite reload loop.
export function shouldAutoReload(
now: number,
lastReloadAt: number | null,
windowMs: number,
): boolean {
if (lastReloadAt === null || Number.isNaN(lastReloadAt)) return true;
return now - lastReloadAt > windowMs;
}
// Heuristic detection of a failed dynamic import. Since the code-splitting work,
// every route (plus Aside / AiChatWindow) is React.lazy: when a new deploy
@@ -24,12 +42,16 @@ export function isChunkLoadError(error: unknown): boolean {
function handleError(error: unknown) {
if (!isChunkLoadError(error)) return;
// A stale-chunk 404 is cured by a full reload that re-fetches index.html and
// the new chunk manifest. Auto-reload once, guarding against a reload loop
// (e.g. a genuinely missing chunk) with a one-shot sessionStorage flag. If the
// flag is already set we fall through to the manual recovery UI below.
// the new chunk manifest. Auto-reload at most once per RELOAD_WINDOW_MS: this
// recovers across multiple deploys in a single tab's lifetime, yet a
// permanently-broken lazy chunk (which would loop) is stopped after the first
// reload and falls through to the manual recovery UI below.
try {
if (sessionStorage.getItem(RELOAD_FLAG)) return;
sessionStorage.setItem(RELOAD_FLAG, "1");
const raw = sessionStorage.getItem(RELOAD_AT_KEY);
const lastReloadAt = raw === null ? null : Number.parseInt(raw, 10);
const now = Date.now();
if (!shouldAutoReload(now, lastReloadAt, RELOAD_WINDOW_MS)) return;
sessionStorage.setItem(RELOAD_AT_KEY, String(now));
} catch {
// sessionStorage unavailable (private mode / disabled): skip the automatic
// reload rather than risk an unguarded loop; the fallback UI still recovers.
@@ -86,19 +86,11 @@ const MIN_HEIGHT = 400;
// Margin kept between the window and the viewport edges while dragging.
const EDGE_MARGIN = 8;
// #184 phase 1.5 / #430: backstop for the degraded-poll fallback. The poll is
// armed when a resume attempt could not attach to the live run and disarmed by the
// thread on settle / local stream; this cap is the ONLY backstop against an endless
// tick (a stuck 'streaming' row before the boot-sweep, or a user-tail 204 with no
// run).
//
// #430: measured from RUN ACTIVITY, not from arm-time. A real autonomous run takes
// 11-25 min — longer than a fixed 10-min-from-start cap, which used to cut the poll
// off mid-run. Instead we cap on INACTIVITY: keep polling as long as the run is
// still making progress (its persisted rows keep changing), and only give up after
// this long with NO new activity. A genuinely stuck run produces no row changes, so
// the idle cap still bounds it; a long-but-progressing run polls to completion.
const DEGRADED_POLL_IDLE_MAX_MS = 10 * 60_000;
// #184 phase 1.5 / #430 / #488: the degraded-poll fallback. The window owns only
// a DUMB 2.5s timer, gated by an armed flag; the THREAD's run-lifecycle FSM owns
// arm/disarm AND the inactivity cap that turns a stuck run into a `stalled` banner
// (#488 commit 4a — the cap moved into the thread so polling->stalled is a single
// FSM transition; the window no longer silently stops polling at the cap).
/** Compact token formatter: 1.2M / 3.4k / 950. */
function formatTokens(n: number): string {
@@ -259,17 +251,13 @@ export default function AiChatWindow() {
[roles],
);
// #184 phase 1.5: degraded-poll fallback (replaces the F4/F5/F7 latches). When
// ChatThread could not attach to a still-running run it arms this via
// onResumeFallback(true); the thread disarms it on settle / local stream. The
// window only OWNS the timer (armedAtRef stamps when it was armed for the cap).
// #184 phase 1.5 / #488: degraded-poll fallback. ChatThread's FSM arms this via
// onResumeFallback(true) when it enters a poll-bearing recovery (attach 204 /
// starved finish / stop) and disarms it on settle / local stream / stalled. The
// window owns ONLY the dumb 2.5s timer; the THREAD owns arm/disarm AND the
// inactivity cap (a stuck run -> the thread's `stalled` banner disarms this).
const [degradedPoll, setDegradedPoll] = useState(false);
// #430: timestamp of the LAST run activity while the poll is armed — stamped on
// arm and re-stamped whenever the polled rows change (see the effect below). The
// idle cap is measured from this, so a long-but-progressing run keeps polling.
const lastActivityAtRef = useRef(0);
const onResumeFallback = useCallback((active: boolean): void => {
if (active) lastActivityAtRef.current = Date.now();
setDegradedPoll(active);
}, []);
// Reset the degraded poll whenever the open chat changes: it is scoped to the
@@ -281,33 +269,17 @@ export default function AiChatWindow() {
const { data: messageRows, isLoading: messagesLoading } =
useAiChatMessagesQuery(
activeChatId ?? undefined,
// DELIBERATELY DUMB (invariant 8 / task 2.4): poll every 2.5s while armed
// and while the run is still active (#430: under the INACTIVITY cap, not a
// fixed-from-start cap); otherwise off. NO error checks (TanStack v5 resets
// fetchFailureCount each fetch, so consecutive errors are not expressible —
// and the poll must survive a server restart) and NO tail checks (the
// settled/local-stream semantics live in ChatThread, which disarms via
// onResumeFallback(false)). The idle cap is the only backstop.
() =>
degradedPoll === true &&
Date.now() - lastActivityAtRef.current < DEGRADED_POLL_IDLE_MAX_MS
? 2500
: false,
// DELIBERATELY DUMB: poll every 2.5s WHILE ARMED, otherwise off. NO error
// checks (TanStack resets fetchFailureCount each fetch; the poll must survive
// a server restart), NO tail checks, NO cap here — the settled/stalled/idle-cap
// semantics all live in ChatThread's FSM, which disarms via onResumeFallback.
() => (degradedPoll === true ? 2500 : false),
// #344: gate on windowOpen too — no message history is fetched (and no
// degraded poll runs) while the window is closed; it loads when the window
// opens with an active chat.
windowOpen,
);
// #430: re-stamp the activity clock whenever the polled rows change while the
// poll is armed. TanStack keeps the same `messageRows` reference across refetches
// that return deep-equal data (structural sharing), so a new reference means the
// run genuinely progressed — which extends the inactivity cap above. A stuck run
// yields no reference change, so the cap eventually fires and stops the poll.
useEffect(() => {
if (degradedPoll) lastActivityAtRef.current = Date.now();
}, [degradedPoll, messageRows]);
// #184 reconnect-and-live-follow. Whether detached agent runs are enabled for
// this workspace. When the feature is off no runs are ever created, so the
// resume attempt would only ever 204; gating ChatThread's resume on it avoids a
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -57,6 +57,25 @@ export async function stopRun(
return req.data;
}
/**
* #488: the run-fact "is a run active on this chat?" first-class from the
* server (POST /ai-chat/run). Called on mount to seed the client FSM's run-fact
* and to VERIFY after a supersede mismatch (an observer following a superseded
* run asks for the latest run and follows it). Returns the latest run row (with
* its `id` and `status`) and its projected assistant message, or `run: null` when
* the chat has never had a run. Owner-gated server-side.
*/
export async function getRun(chatId: string): Promise<{
run: { id: string; status: string } | null;
message: IAiChatMessageRow | null;
}> {
const req = await api.post<{
run: { id: string; status: string } | null;
message: IAiChatMessageRow | null;
}>("/ai-chat/run", { chatId });
return req.data;
}
/**
* Resolve the chat bound to a document (the current user's most-recent chat
* created on that page), or null when there is none. Drives auto-open-on-page.
@@ -0,0 +1,183 @@
# AI-chat run-lifecycle FSM — design spec (#488)
This is the written design that `run-fsm.ts` implements. It ships in the PR (issue
#488 commit 1: "the spec is written FIRST and enters the PR"). It has four parts:
(1) the event × state transition table, (2) the map of every `chat-thread.tsx` ref
to {FSM state | FSM context | stays data}, (3) the run-fact protocol, (4) the
invariants.
The reducer is a **pure function** `reduce(machine, event) → machine`. The returned
machine carries the **command effects** for that transition; a thin runtime in
`chat-thread.tsx` dispatches events and executes effects. Because it is pure, the
whole machine is enumerable and unit-tested directly (event × state → next state is
the observable property) — see `run-fsm.test.ts`.
---
## 1. Event × state transition table
Phases: `idle | sending | streaming | attaching | reconnecting(attempt,failed) |
polling(reason) | stalled | stopping | superseding | error(kind)`.
Context (orthogonal): `epoch`, `ownership: local|observer`, `runFact: {runId}|null`,
`liveFollow` (are we following a live run we locally streamed — the reconnect
ladder — vs a one-shot mount-attach resume? both are `observer`, but a live-follow
drop RE-ENTERS the ladder (#488 commit 3) while a mount-resume drop polls).
Legend: **†** = command-transition (bumps `epoch`, I1). Effects in `[…]`.
| Event (source) | From phase(s) | → To phase | Effects / ctx |
|---|---|---|---|
| `SEND_LOCAL` (user send) | idle, error, polling, stalled, reconnecting | sending **†** | `[cancelReconnect, disarmPoll]`, ownership=local |
| `STREAM_START{runId}` (SDK `start` metadata) | sending, attaching, reconnecting, superseding | streaming | `[cancelReconnect, disarmPoll]`, runFact←runId |
| `FINISH_CLEAN` (onFinish clean) | streaming, … | idle | `[disarmPoll, cancelReconnect]`, runFact←null |
| `FINISH_ABORT` (onFinish isAbort) | streaming, stopping | idle | `[disarmPoll, cancelReconnect]`, runFact←null (I4 exits stopping by this DATA) |
| `FINISH_DISCONNECT` (observer, NOT liveFollow) | streaming(observer) | polling(disconnect-visible) | `[armPoll]` (a mount-resume drop polls) |
| `FINISH_DISCONNECT{hasVisibleContent}` (local drop OR liveFollow) | streaming | reconnecting(1) **†** *iff runFact\|liveFollow* | `[scheduleReconnect(1)]` (+`armPoll` if visible), ownership=observer, liveFollow=true (commit 3: repeatable) |
| `FINISH_DISCONNECT` (no runFact, not liveFollow) | streaming | idle | runFact←null (plain terminal "connection lost") |
| `STREAM_INCOMPLETE{reason}` (observer starved/torn clean finish) | streaming(observer) | polling(reason) | `[armPoll(reason)]` |
| `FINISH_ERROR{kind}` (onFinish isError) | any | error(kind) | `[disarmPoll, cancelReconnect]`, runFact←null |
| `STREAM_START{runId}` (first assistant frame of a local turn) | sending | streaming | runFact←runId, `[cancelReconnect, disarmPoll]` |
| `ATTACH_START{runId}` (mount resume) | **idle only** (F2) | attaching **†** | `[resumeStream]`, ownership=observer, runFact←runId; ignored from any non-idle phase |
| `ATTACH_LIVE` (attach GET 2xx) | attaching | streaming | — |
| `ATTACH_NONE` (attach GET 204/err/throw) | attaching | polling(attach-none) | `[armPoll(attach-none)]` |
| `RECONNECT_ATTEMPT{n}` (backoff timer) | reconnecting | reconnecting(n) **†** | `[resumeStream]` |
| `RECONNECT_ATTACHED` (reconnect GET 2xx) | reconnecting | streaming | `[cancelReconnect, disarmPoll]`**counter reset** (commit 3) |
| `RECONNECT_NONE` (reconnect GET 204/err), attempt<MAX | reconnecting | reconnecting(n+1) **†** | `[armPoll(attach-none), scheduleReconnect(n+1)]` |
| `RECONNECT_NONE`, attempt=MAX | reconnecting | reconnecting(MAX, failed) | `[armPoll(reconnect-exhausted)]` |
| `RETRY` (manual, failed banner) | reconnecting(failed) | reconnecting(1) **†** | `[resumeStream]` |
| `RETRY` (manual, stalled banner) | stalled | polling(attach-none) **†** | `[armPoll]` |
| `POLL_TERMINAL` (settled tail merged) | polling, reconnecting, stopping | idle | `[disarmPoll, cancelReconnect]`, runFact←null (I4) |
| `POLL_IDLE_CAP` (inactivity cap) | polling, reconnecting | stalled | `[disarmPoll, cancelReconnect]` (commit 4a — no more silent) |
| `RUN_FACT{null}` (POST /run → null/terminal, 204) | reconnecting/attaching/polling/stopping | idle | `[cancelReconnect, disarmPoll]`, runFact←null (I3 fresh-negative gate) |
| `RUN_FACT{runId}` | any | (same) | runFact←runId (pessimism toward an attempt) |
| `STOP_REQUESTED` (user Stop) | streaming, reconnecting, polling | stopping **†** | `[stopRun, abortAttach, cancelReconnect, armPoll]` (poll drives the terminal — I4 exit by data) |
| `SUPERSEDE_REQUESTED{targetRunId}` (interrupt+send) | streaming, reconnecting, polling, error | superseding **†** | `[supersede(target), cancelReconnect, disarmPoll]` |
| `SUPERSEDE_READY{runId}` (CAS ok) | superseding | streaming | ownership=local, runFact←runId |
| `SUPERSEDE_MISMATCH{currentRunId}` (409 SUPERSEDE_TARGET_MISMATCH) | superseding | error(supersede-mismatch) | `[postRun(verify)]`, runFact←currentRunId |
| `SUPERSEDE_TIMEOUT` (409 SUPERSEDE_TIMEOUT) | superseding | error(supersede-timeout) | — (composer keeps text; no auto-retry) |
| `SUPERSEDE_INVALID` (409 SUPERSEDE_INVALID) | superseding | error(supersede-invalid) | — |
| `RUN_ALREADY_ACTIVE{activeRunId}` (409 A_RUN_ALREADY_ACTIVE, plain POST) | sending | error(run-already-active) | runFact←activeRunId (composer offers supersede; NO auto-retry) |
| `DISPOSE` (unmount) | any | idle **†** | `[abortAttach, cancelReconnect, disarmPoll]` (I1/I5 — epoch++ kills late callbacks) |
**`stopping` honors any finish (re-review MEDIUM):** BEFORE the epoch filter, a
stream finish (`FINISH_*`/`STREAM_INCOMPLETE`) arriving in phase `stopping` exits
`stopping -> idle` regardless of generation. A plain Stop has no successor stream,
so the aborted stream's finish IS the expected end (I4 exit by data) — and it
carries the PRE-stop generation (STOP_REQUESTED bumped the epoch), so the filter
would otherwise strand the machine in `stopping` (no idle-cap covers it). The filter
stays in force for `superseding` (that is the F1 supersede drop).
**Epoch filter (I1):** the reducer then drops any event carrying an `epoch` that
does not equal the current `ctx.epoch`. Outcome events (`STREAM_START`, `ATTACH_*`,
`RECONNECT_*`, `SUPERSEDE_*`, **`FINISH_*`/`STREAM_INCOMPLETE`**, `RUN_FACT`) are
stamped with the generation the corresponding STREAM started under (the runtime
holds a per-owned-stream `turnEpoch`); trigger events (user actions, fresh
disconnects) carry no epoch. **F1:** this is what makes a SUPERSEDED stream's late
`onFinish` (a dead stream A closing after the CAS started stream B) get dropped, so
A cannot drive the live new run into a false reconnect or reset its run-fact. The
supersede path additionally ABORTS A and starts B only from A's onFinish (a
microtask), because ai@6 `AbstractChat.makeRequest` corrupts overlapping streams
(A's `finally` reads then nulls the shared `activeResponse`).
**Removed events (scope-cut, internal review):** `RUN_SUPERSEDED` (a ghost feature —
never dispatched; the observer-superseded case is handled by the degraded poll,
which follows the latest rows regardless of runId), `RECONNECT_BEGIN` (reconnect is
entered by `FINISH_DISCONNECT`), and `POLL_ACTIVITY` (the window's activity clock was
removed when the idle-cap moved into the thread). The reducer and this table now
share exactly the dispatched event set.
### 409-code → event map (the real #487 contract consumed here)
| Server response | Event dispatched | error kind → banner |
|---|---|---|
| 409 `A_RUN_ALREADY_ACTIVE` (+ body.activeRunId) | `RUN_ALREADY_ACTIVE{activeRunId}` | run-already-active → "already answering / interrupt & send" |
| 409 `SUPERSEDE_TARGET_MISMATCH` (+ body.activeRunId) | `SUPERSEDE_MISMATCH{currentRunId}` | supersede-mismatch → verify via /run |
| 409 `SUPERSEDE_TIMEOUT` | `SUPERSEDE_TIMEOUT` | supersede-timeout → "couldn't interrupt in time, resend" |
| 409 `SUPERSEDE_INVALID` | `SUPERSEDE_INVALID` | supersede-invalid → "couldn't interrupt this run" |
| 503 `A_RUN_BEGIN_FAILED` | `FINISH_ERROR{begin-failed}` | begin-failed → "could not start, temporary" |
---
## 2. Ref-map — every `chat-thread.tsx` ref → its new home (MIGRATION RESOLVED)
The migration is COMPLETE: the 13 run-lifecycle FLAGS below are GONE from
`chat-thread.tsx` (collapsed into FSM phase/ctx/effects, or deleted). What remains
are identity/data mirrors, effect-owned controllers/timers, and ONE React-liveness
bit — none of which is a run-lifecycle flag, so the post-merge "no new flags" rule
holds. **Pending column: empty.**
| # | Old ref | Resolved to | Where now |
|---|---|---|---|
| 1 | `reconcileTailRef` | **FSM phase** | reconcile-merge gated on `phase ∈ {polling, reconnecting, stopping}` |
| 2 | `noStreamHandledRef` | **FSM epoch (I1)** | the attach outcome's epoch guard drops the stale/second outcome |
| 3 | `onNoActiveStreamRef` | **FSM event** | transport → `handleAttachOutcome` dispatches `ATTACH_NONE`/`RECONNECT_NONE` |
| 4 | `onReconnectAttachedRef` | **FSM event** | transport dispatches `ATTACH_LIVE` / `RECONNECT_ATTACHED` |
| 5 | `resumedTurnRef` + `resumedTurn` state | **FSM ctx `ownership`** | `ownership==='observer'` ⇒ never flush; hides "Send now" |
| 6 | `reconnectStateRef` + `reconnectState` state | **FSM phase** | `reconnecting(attempt,failed)` renders the banner |
| 7 | `reconnectTimerRef` | **effect-owned timer** | owned by `scheduleReconnect`/`cancelReconnect` effects (not a flag) |
| 8 | `flushOnAbortRef` | **DELETED** | the stop→flush dance is replaced by the CAS supersede (commit 5) |
| 9 | `interruptNextSendRef` | **DELETED** | the server injects the interrupt note from the supersede itself |
| 10 | `supersedeRetryRef` | **DELETED** (commit 5) | the client 409 retry ladder is gone; CAS supersede replaces it |
| 11 | `stopPendingRef` | **FSM phase `stopping`** | the deferred stop fires from the chat-id adoption effect while `stopping` |
| 12 | `mountedRef` | **retained (React liveness)** | orthogonal to run-lifecycle; gates imperative onFinish side-effects post-unmount. Epoch (I1) handles stale COMMAND-outcomes; DISPOSE bumps it |
| 13 | `attemptResumeRef` | **FSM `ATTACH_START` + run-fact** | mount arms attach ONLY on a confirmed active run (commit 4b: streaming-tail status, or POST /run for a user tail) |
| 14 | `stripRef` | **data** (attachStrategy) | strip+replay detail; the `resumeStream` effect reads it |
| 15 | `strippedRowRef` | **data** (attachStrategy) | the anchor row |
| 16 | `attachAbortRef` | **effect-owned controller** | aborted by the `abortAttach` effect in cleanup (I5) |
| 17–25 | `chatIdRef`, `openPageRef`, `getEditorSelectionRef`, `roleIdRef`, `stableIdRef`, `queuedRef`, `sendMessageRef`, `statusRef`, `lastForwardedChatIdRef` | **data** (identity/send mirrors) | unchanged — not lifecycle flags |
| NEW | `pendingSupersedeRef` | **data** (send-plumbing) | the runId injected into the next `POST /stream {supersede}`; the single replacement for the 3 DELETED one-shots (#8/#9/#10) — net −2 refs |
| NEW | `idleCapTimerRef` | **effect-owned timer** | the stalled inactivity cap → `POLL_IDLE_CAP` (commit 4a); not a flag |
Net: the 13 lifecycle flags (#1#13) are eliminated: **8** → FSM phase/ctx/epoch/event
(#1#6, #11, #13), **3** deleted (#8/#9/#10), **`reconnectTimerRef` (#7)** becomes an
effect-owned controller, and **`mountedRef` (#12)** is retained as React liveness
(8 + 3 + 1 + 1 = 13). (`attachAbortRef` (#16) is outside the #1#13 set — it was
already an effect-owned controller.) Two effect-owned timers + one send-plumbing data
ref are added — none is a boolean lifecycle latch.
---
## 3. Run-fact protocol (`runFact: {runId} | null`) — I3
"A run is active" is first-class from the SERVER, not inferred from an assistant
message. Sources, in the order they update `ctx.runFact`:
1. **Init (mount):** `POST /ai-chat/run { chatId }``{ run, message }`. A `run`
with a non-terminal `status` seeds `runFact = { runId: run.id }`; a null/terminal
run seeds `null`. This is what arms the resume attempt (`ATTACH_START`) — the
attempt is armed ONLY on a positive fact (commit 4b: a user-tail with no active
run no longer arms a pointless poll on every open).
2. **Live update:** the `start` stream metadata carries `runId``STREAM_START{runId}`.
3. **Attach outcomes:** `ATTACH_LIVE` (2xx) confirms active; a 204 on a non-stripped
path is an authoritative NEGATIVE fact → the runtime dispatches `RUN_FACT{null}`,
which cancels recovery (I3 fresh-negative gate).
4. **Poll (future resume-stack iteration #491):** the delta will carry the run field;
until then the poll drives to a terminal ROW, dispatched as `POLL_TERMINAL`.
Pessimism rule: a stale-but-positive fact PERMITS entering recovery (attach); the
204 then cuts it. A fresh negative fact gates recovery OUT immediately.
---
## 4. Invariants
- **I1 — Epoch (generation counter).** Every command-emitting transition bumps
`ctx.epoch`; every async outcome event carries its issuing epoch; the reducer
drops stale-epoch outcomes. Replaces the one-shot-ref zoo (`noStreamHandledRef`,
the flush/interrupt/supersede one-shots, the `mountedRef` late-callback gate).
- **I2 — Ownership is context, not state.** `local | observer` is orthogonal to the
transport phase. The queue flushes ONLY under local ownership; an observer
following a detached run never flushes (was `resumedTurnRef`).
- **I3 — Run-fact is first-class from the server.** Reconnect is entered by the
run-fact, not by an assistant message (commit 2). A fresh negative fact cancels
recovery.
- **I4 — Exit `stopping` by DATA.** A terminal row / negative run-fact / terminal
finish exits `stopping`, never the stopRun HTTP response (which returns after the
abort but before finalization — keying off it would unlock the composer on a 409).
- **I5 — Dispose protocol.** Command controllers (attach GET, POST /stream, POST
/run) are effect-owned and aborted in cleanup (`abortAttach` on `DISPOSE`), not
render-phase refs. A client abort of an already-sent POST does not cancel the
server action, so disarming on unmount is safe.
- **attachStrategy** (strip+replay today) is behind the `resumeStream` effect; the
resume-stack iteration (#491) swaps it to tail-only WITHOUT touching the FSM.
- **Queue** stays a data structure; flush/interrupt decisions are transitions.
@@ -0,0 +1,482 @@
import { describe, it, expect } from "vitest";
import {
reduce,
initialMachine,
reconnectDelayMs,
RECONNECT_MAX_ATTEMPTS,
type Machine,
type Effect,
type Event,
} from "./run-fsm";
// Drive a sequence of events through the reducer, returning the final machine.
function run(m: Machine, ...events: Event[]): Machine {
return events.reduce(reduce, m);
}
function withRunFact(runId = "run-1"): Machine {
return {
...initialMachine(),
ctx: { epoch: 0, ownership: "local", runFact: { runId }, liveFollow: false },
};
}
function effectTypes(m: Machine): string[] {
return m.effects.map((e) => e.type);
}
function hasEffect(m: Machine, type: Effect["type"]): boolean {
return m.effects.some((e) => e.type === type);
}
describe("run-fsm — epoch invariant (I1)", () => {
it("drops an outcome carrying a stale epoch", () => {
// A command bumps the epoch; an outcome stamped with the OLD epoch is dropped.
const m0 = reduce(initialMachine(), { type: "ATTACH_START", runId: "r" }); // epoch 0->1, attaching
expect(m0.ctx.epoch).toBe(1);
expect(m0.phase.name).toBe("attaching");
// A late ATTACH_LIVE from a SUPERSEDED attempt (epoch 0) must NOT drive us.
const stale = reduce(m0, { type: "ATTACH_LIVE", epoch: 0 });
expect(stale.phase.name).toBe("attaching");
expect(stale.effects).toEqual([]);
});
it("applies an outcome carrying the current epoch", () => {
const m0 = reduce(initialMachine(), { type: "ATTACH_START", runId: "r" });
const live = reduce(m0, { type: "ATTACH_LIVE", epoch: m0.ctx.epoch });
expect(live.phase.name).toBe("streaming");
});
it("an outcome with no epoch is never dropped (trigger events)", () => {
const m0 = reduce(initialMachine(), { type: "ATTACH_START", runId: "r" });
const disposed = reduce(m0, { type: "DISPOSE" });
expect(disposed.phase.name).toBe("idle");
expect(hasEffect(disposed, "abortAttach")).toBe(true);
});
it("every command-transition increments the epoch exactly once", () => {
let m = initialMachine();
const before = m.ctx.epoch;
m = reduce(m, { type: "SEND_LOCAL" });
expect(m.ctx.epoch).toBe(before + 1);
m = reduce(m, { type: "STOP_REQUESTED" });
expect(m.ctx.epoch).toBe(before + 2);
});
});
describe("run-fsm — local turn", () => {
it("SEND_LOCAL → sending, local ownership, cancels recovery", () => {
const m = reduce(withRunFact(), { type: "SEND_LOCAL" });
expect(m.phase.name).toBe("sending");
expect(m.ctx.ownership).toBe("local");
expect(effectTypes(m)).toEqual(
expect.arrayContaining(["cancelReconnect", "disarmPoll"]),
);
});
it("STREAM_START adopts the runId into the run-fact and goes streaming", () => {
const m = run(initialMachine(), { type: "SEND_LOCAL" });
const s = reduce(m, { type: "STREAM_START", runId: "run-9", epoch: m.ctx.epoch });
expect(s.phase.name).toBe("streaming");
expect(s.ctx.runFact).toEqual({ runId: "run-9" });
});
it("FINISH_CLEAN → idle, run-fact cleared, poll/reconnect disarmed", () => {
const streaming = run(initialMachine(), { type: "SEND_LOCAL" }, { type: "STREAM_START", runId: "r" });
const done = reduce(streaming, { type: "FINISH_CLEAN" });
expect(done.phase.name).toBe("idle");
expect(done.ctx.runFact).toBeNull();
});
});
// #488 commit 2 — SSE break BEFORE the first assistant frame must still recover.
describe("run-fsm — commit 2: reconnect by run-fact, not by assistant message", () => {
it("FINISH_DISCONNECT with an active run-fact → reconnecting (even with no visible content)", () => {
// Setup-phase break: no assistant frame yet, but a run-fact exists.
const streaming = withRunFact("run-2");
const m = reduce(streaming, {
type: "FINISH_DISCONNECT",
hasVisibleContent: false,
epoch: streaming.ctx.epoch,
});
expect(m.phase.name).toBe("reconnecting");
if (m.phase.name === "reconnecting") expect(m.phase.attempt).toBe(1);
expect(m.ctx.ownership).toBe("observer");
expect(hasEffect(m, "scheduleReconnect")).toBe(true);
// No visible content -> no poll arm yet (the reconnect ladder rebuilds it).
expect(hasEffect(m, "armPoll")).toBe(false);
});
it("FINISH_DISCONNECT WITH visible content also arms the poll", () => {
const m = reduce(withRunFact("run-2"), {
type: "FINISH_DISCONNECT",
hasVisibleContent: true,
epoch: 0,
});
expect(m.phase.name).toBe("reconnecting");
expect(hasEffect(m, "armPoll")).toBe(true);
});
it("FINISH_DISCONNECT with NO run-fact → idle (plain connection-lost)", () => {
const m = reduce(initialMachine(), {
type: "FINISH_DISCONNECT",
hasVisibleContent: true,
epoch: 0,
});
expect(m.phase.name).toBe("idle");
});
});
// #488 commit 3 — a SECOND break after a successful re-attach starts a NEW ladder.
describe("run-fsm — commit 3: repeated reconnect cycles", () => {
it("two breaks in a row produce two reconnect cycles (counter resets on attach)", () => {
let m = withRunFact("run-3");
// First break -> reconnecting(1).
m = reduce(m, { type: "FINISH_DISCONNECT", hasVisibleContent: false, epoch: m.ctx.epoch });
expect(m.phase.name).toBe("reconnecting");
// Attempt fires, re-attaches live.
m = reduce(m, { type: "RECONNECT_ATTEMPT", attempt: 1, epoch: m.ctx.epoch });
m = reduce(m, { type: "RECONNECT_ATTACHED", epoch: m.ctx.epoch });
expect(m.phase.name).toBe("streaming");
// SECOND break: the counter was reset, so a fresh ladder starts at attempt 1
// (the old one-shot !wasResumed gate would have sent this to silent poll).
m = reduce(m, { type: "FINISH_DISCONNECT", hasVisibleContent: false, epoch: m.ctx.epoch });
expect(m.phase.name).toBe("reconnecting");
if (m.phase.name === "reconnecting") expect(m.phase.attempt).toBe(1);
expect(hasEffect(m, "scheduleReconnect")).toBe(true);
});
it("a MOUNT-attach observer drop falls to POLL, not the reconnect ladder", () => {
// Distinguishes commit 3 from a one-shot resume: an observer that never
// live-followed (liveFollow false) polls on a drop.
let m = reduce(initialMachine(), { type: "ATTACH_START", runId: "r" });
m = reduce(m, { type: "ATTACH_LIVE", epoch: m.ctx.epoch });
expect(m.ctx.ownership).toBe("observer");
expect(m.ctx.liveFollow).toBe(false);
m = reduce(m, { type: "FINISH_DISCONNECT", hasVisibleContent: true, epoch: m.ctx.epoch });
expect(m.phase.name).toBe("polling");
expect(hasEffect(m, "armPoll")).toBe(true);
});
it("STREAM_INCOMPLETE (observer starved/torn finish) → polling", () => {
let m = reduce(initialMachine(), { type: "ATTACH_START", runId: "r" });
m = reduce(m, { type: "ATTACH_LIVE", epoch: m.ctx.epoch });
m = reduce(m, { type: "STREAM_INCOMPLETE", reason: "starved", epoch: m.ctx.epoch });
expect(m.phase).toEqual({ name: "polling", reason: "starved" });
expect(hasEffect(m, "armPoll")).toBe(true);
});
it("liveFollow is set on the first local drop and kept across a re-attach", () => {
let m = withRunFact("run-3");
m = reduce(m, { type: "FINISH_DISCONNECT", hasVisibleContent: false, epoch: m.ctx.epoch });
expect(m.ctx.liveFollow).toBe(true);
m = reduce(m, { type: "RECONNECT_ATTEMPT", attempt: 1, epoch: m.ctx.epoch });
m = reduce(m, { type: "RECONNECT_ATTACHED", epoch: m.ctx.epoch });
expect(m.ctx.liveFollow).toBe(true); // kept — so a second drop reconnects
// A clean finish clears it.
m = reduce(m, { type: "FINISH_CLEAN", epoch: m.ctx.epoch });
expect(m.ctx.liveFollow).toBe(false);
});
it("RECONNECT_NONE backs off through the ladder, then fails at the cap", () => {
let m = withRunFact("run-3");
m = reduce(m, { type: "FINISH_DISCONNECT", hasVisibleContent: false, epoch: m.ctx.epoch });
for (let n = 1; n < RECONNECT_MAX_ATTEMPTS; n++) {
m = reduce(m, { type: "RECONNECT_ATTEMPT", attempt: n, epoch: m.ctx.epoch });
m = reduce(m, { type: "RECONNECT_NONE", epoch: m.ctx.epoch });
expect(m.phase.name).toBe("reconnecting");
if (m.phase.name === "reconnecting") {
expect(m.phase.attempt).toBe(n + 1);
expect(m.phase.failed).toBe(false);
}
// The belt-and-suspenders poll is armed each failed attempt.
expect(hasEffect(m, "armPoll")).toBe(true);
}
// Final attempt fails -> failed banner (Retry), poll armed.
m = reduce(m, { type: "RECONNECT_ATTEMPT", attempt: RECONNECT_MAX_ATTEMPTS, epoch: m.ctx.epoch });
m = reduce(m, { type: "RECONNECT_NONE", epoch: m.ctx.epoch });
expect(m.phase.name).toBe("reconnecting");
if (m.phase.name === "reconnecting") expect(m.phase.failed).toBe(true);
// RETRY restarts at attempt 1.
m = reduce(m, { type: "RETRY" });
expect(m.phase.name).toBe("reconnecting");
if (m.phase.name === "reconnecting") {
expect(m.phase.attempt).toBe(1);
expect(m.phase.failed).toBe(false);
}
expect(hasEffect(m, "resumeStream")).toBe(true);
});
it("reconnectDelayMs is the exponential backoff 1s,2s,4s,8s,16s", () => {
expect([1, 2, 3, 4, 5].map(reconnectDelayMs)).toEqual([1000, 2000, 4000, 8000, 16000]);
});
});
// #488 commit 4 — polling stalled-state + user-tail gating.
describe("run-fsm — commit 4: stalled + run-fact gating", () => {
it("POLL_IDLE_CAP: polling → stalled with a banner (poll disarmed), not silent", () => {
let m = reduce(withRunFact(), { type: "ATTACH_START", runId: "r" });
m = reduce(m, { type: "ATTACH_NONE", epoch: m.ctx.epoch });
expect(m.phase.name).toBe("polling");
m = reduce(m, { type: "POLL_IDLE_CAP" });
expect(m.phase.name).toBe("stalled");
expect(hasEffect(m, "disarmPoll")).toBe(true);
});
it("RETRY from stalled re-arms the poll", () => {
let m = reduce(withRunFact(), { type: "ATTACH_START", runId: "r" });
m = reduce(m, { type: "ATTACH_NONE", epoch: m.ctx.epoch });
m = reduce(m, { type: "POLL_IDLE_CAP" });
m = reduce(m, { type: "RETRY" });
expect(m.phase.name).toBe("polling");
expect(hasEffect(m, "armPoll")).toBe(true);
});
it("a fresh NEGATIVE run-fact while attaching cancels recovery (user-tail, no active run)", () => {
// The mount POST /run returns no active run: attaching → idle, no poll armed.
let m = reduce(withRunFact(), { type: "ATTACH_START", runId: "r" });
m = reduce(m, { type: "RUN_FACT", runFact: null, epoch: m.ctx.epoch });
expect(m.phase.name).toBe("idle");
expect(m.ctx.runFact).toBeNull();
expect(hasEffect(m, "disarmPoll")).toBe(true);
});
it("a negative run-fact while polling stops the poll", () => {
let m = reduce(withRunFact(), { type: "ATTACH_START", runId: "r" });
m = reduce(m, { type: "ATTACH_NONE", epoch: m.ctx.epoch });
m = reduce(m, { type: "RUN_FACT", runFact: null, epoch: m.ctx.epoch });
expect(m.phase.name).toBe("idle");
});
it("POLL_TERMINAL settles polling → idle (I4 data-driven exit)", () => {
let m = reduce(withRunFact(), { type: "ATTACH_START", runId: "r" });
m = reduce(m, { type: "ATTACH_NONE", epoch: m.ctx.epoch });
m = reduce(m, { type: "POLL_TERMINAL" });
expect(m.phase.name).toBe("idle");
expect(m.ctx.runFact).toBeNull();
});
});
// #488 commit 5 — error classification + supersede CAS transitions.
describe("run-fsm — commit 5: supersede CAS + error classification", () => {
it("SUPERSEDE_REQUESTED → superseding, fires the CAS effect, bumps epoch", () => {
const streaming = withRunFact("run-old");
const m = reduce(streaming, { type: "SUPERSEDE_REQUESTED", targetRunId: "run-old" });
expect(m.phase.name).toBe("superseding");
expect(m.ctx.epoch).toBe(streaming.ctx.epoch + 1);
const sup = m.effects.find((e) => e.type === "supersede");
expect(sup).toEqual({ type: "supersede", targetRunId: "run-old" });
});
it("SUPERSEDE_READY → streaming as the new local owner", () => {
let m = reduce(withRunFact("run-old"), { type: "SUPERSEDE_REQUESTED", targetRunId: "run-old" });
m = reduce(m, { type: "SUPERSEDE_READY", runId: "run-new", epoch: m.ctx.epoch });
expect(m.phase.name).toBe("streaming");
expect(m.ctx.ownership).toBe("local");
expect(m.ctx.runFact).toEqual({ runId: "run-new" });
});
it("SUPERSEDE_MISMATCH → error(supersede-mismatch) + verify via /run (no blind banner)", () => {
let m = reduce(withRunFact("run-old"), { type: "SUPERSEDE_REQUESTED", targetRunId: "run-old" });
m = reduce(m, { type: "SUPERSEDE_MISMATCH", currentRunId: "run-x", epoch: m.ctx.epoch });
expect(m.phase).toEqual({ name: "error", kind: "supersede-mismatch" });
expect(hasEffect(m, "postRun")).toBe(true);
expect(m.ctx.runFact).toEqual({ runId: "run-x" });
});
it("SUPERSEDE_TIMEOUT → error(supersede-timeout), no auto-retry effect", () => {
let m = reduce(withRunFact("run-old"), { type: "SUPERSEDE_REQUESTED", targetRunId: "run-old" });
m = reduce(m, { type: "SUPERSEDE_TIMEOUT", epoch: m.ctx.epoch });
expect(m.phase).toEqual({ name: "error", kind: "supersede-timeout" });
expect(m.effects).toEqual([]);
});
it("SUPERSEDE_INVALID → error(supersede-invalid)", () => {
let m = reduce(withRunFact("run-old"), { type: "SUPERSEDE_REQUESTED", targetRunId: "run-old" });
m = reduce(m, { type: "SUPERSEDE_INVALID", epoch: m.ctx.epoch });
expect(m.phase).toEqual({ name: "error", kind: "supersede-invalid" });
});
it("a stale SUPERSEDE outcome from a superseded epoch is dropped", () => {
let m = reduce(withRunFact("run-old"), { type: "SUPERSEDE_REQUESTED", targetRunId: "run-old" });
const supersedingEpoch = m.ctx.epoch;
// The user retriggers, bumping the epoch again.
m = reduce(m, { type: "SUPERSEDE_REQUESTED", targetRunId: "run-old" });
// The first CAS's late TIMEOUT (old epoch) must NOT knock us out of superseding.
const late = reduce(m, { type: "SUPERSEDE_TIMEOUT", epoch: supersedingEpoch });
expect(late.phase.name).toBe("superseding");
});
it("RUN_ALREADY_ACTIVE (plain POST gate) → error(run-already-active), no retry effect", () => {
const m = reduce(run(initialMachine(), { type: "SEND_LOCAL" }), { type: "RUN_ALREADY_ACTIVE" });
expect(m.phase).toEqual({ name: "error", kind: "run-already-active" });
expect(m.effects).toEqual([]);
});
it("#497/S4: RUN_ALREADY_ACTIVE{activeRunId} ADOPTS the server's active run as the run-fact", () => {
// The server sends `activeRunId` so a later supersede can TARGET that run
// instead of a blind promote+abort. Absorb it into runFact.
const m = reduce(run(initialMachine(), { type: "SEND_LOCAL" }), {
type: "RUN_ALREADY_ACTIVE",
activeRunId: "run-foreign",
});
expect(m.phase).toEqual({ name: "error", kind: "run-already-active" });
expect(m.ctx.runFact).toEqual({ runId: "run-foreign" });
expect(m.effects).toEqual([]);
});
it("#497/S4: RUN_ALREADY_ACTIVE without an activeRunId keeps the prior run-fact", () => {
const seeded = reduce(run(initialMachine(), { type: "SEND_LOCAL" }), {
type: "RUN_FACT",
runFact: { runId: "run-prior" },
});
const m = reduce(seeded, { type: "RUN_ALREADY_ACTIVE" });
expect(m.ctx.runFact).toEqual({ runId: "run-prior" });
});
});
// #488 F2 — a late mount `getRun → ATTACH_START` must not hijack a local turn.
describe("run-fsm — F2: ATTACH_START only from idle", () => {
it("ATTACH_START from a local `sending` turn is ignored (no observer hijack)", () => {
const sending = reduce(initialMachine(), { type: "SEND_LOCAL" }); // idle -> sending, local
const m = reduce(sending, { type: "ATTACH_START", runId: "r" });
expect(m.phase.name).toBe("sending");
expect(m.ctx.ownership).toBe("local"); // NOT flipped to observer
expect(m.effects).toEqual([]); // no resumeStream
});
it("ATTACH_START from idle attaches as normal", () => {
const m = reduce(initialMachine(), { type: "ATTACH_START", runId: "r" });
expect(m.phase.name).toBe("attaching");
expect(m.ctx.ownership).toBe("observer");
expect(hasEffect(m, "resumeStream")).toBe(true);
});
});
describe("run-fsm — stop (I4: exit by data)", () => {
it("STOP_REQUESTED → stopping, fires stopRun + abortAttach, no data-independent exit", () => {
const m = reduce(withRunFact(), { type: "STOP_REQUESTED" });
expect(m.phase.name).toBe("stopping");
expect(effectTypes(m)).toEqual(expect.arrayContaining(["stopRun", "abortAttach"]));
});
it("stopping exits on the aborted stream's finish carrying the PRE-STOP epoch", () => {
// MEDIUM (#488 re-review): STOP_REQUESTED is a command that BUMPS the epoch, but
// the runtime stamps the aborted stream's onFinish with the stream's START (pre-
// stop) generation — exactly what the component sends. `stopping` must HONOR
// that finish regardless of generation (no idle-cap covers `stopping`).
// MUTATION-VERIFY: remove the honor-in-`stopping` branch and this hangs in
// `stopping` (the epoch filter drops the pre-stop finish) -> red.
const preStopEpoch = withRunFact().ctx.epoch; // E1 (the stream's start epoch)
let m = reduce(withRunFact(), { type: "STOP_REQUESTED" }); // E1 -> E2, stopping
expect(m.ctx.epoch).toBe(preStopEpoch + 1);
m = reduce(m, { type: "FINISH_ABORT", epoch: preStopEpoch }); // NOT the current epoch
expect(m.phase.name).toBe("idle");
expect(m.ctx.runFact).toBeNull();
});
it("stopping exits on a clean finish carrying the pre-stop epoch too", () => {
const preStopEpoch = withRunFact().ctx.epoch;
let m = reduce(withRunFact(), { type: "STOP_REQUESTED" });
m = reduce(m, { type: "FINISH_CLEAN", epoch: preStopEpoch });
expect(m.phase.name).toBe("idle");
});
it("stopping exits on a negative run-fact (data)", () => {
let m = reduce(withRunFact(), { type: "STOP_REQUESTED" });
m = reduce(m, { type: "RUN_FACT", runFact: null, epoch: m.ctx.epoch });
expect(m.phase.name).toBe("idle");
});
// Review #4: `stopping` arms the poll but had no inactivity backstop.
it("review-4: POLL_IDLE_CAP in `stopping` exits to idle (bounded), NOT stalled", () => {
let m = reduce(withRunFact(), { type: "STOP_REQUESTED" });
expect(m.phase.name).toBe("stopping");
expect(hasEffect(m, "armPoll")).toBe(true);
// MUTATION-VERIFY: drop the `stopping` branch in POLL_IDLE_CAP and this hangs
// in `stopping` (poll forever) -> red.
m = reduce(m, { type: "POLL_IDLE_CAP" });
expect(m.phase.name).toBe("idle");
expect(hasEffect(m, "disarmPoll")).toBe(true);
expect(m.ctx.ownership).toBe("local");
});
});
// Review #1: positive attach outcomes must be guarded by the SOURCE phase — the
// epoch filter alone is insufficient because POLL_TERMINAL uses to() (no epoch
// bump) and does not abort the in-flight GET.
describe("run-fsm — review-1: attach outcomes guarded by source phase", () => {
it("a late RECONNECT_ATTACHED after POLL_TERMINAL stays idle (no phantom streaming)", () => {
let m = withRunFact("run-1");
m = reduce(m, { type: "FINISH_DISCONNECT", hasVisibleContent: true, epoch: m.ctx.epoch });
m = reduce(m, { type: "RECONNECT_ATTEMPT", attempt: 1, epoch: m.ctx.epoch }); // attach GET
const epoch = m.ctx.epoch;
// The armed degraded poll reaches the terminal row FIRST (epoch unchanged).
m = reduce(m, { type: "POLL_TERMINAL" });
expect(m.phase.name).toBe("idle");
expect(m.ctx.epoch).toBe(epoch); // POLL_TERMINAL did NOT bump the epoch
// The slow GET returns live 2xx under the SAME epoch — must NOT resurrect.
m = reduce(m, { type: "RECONNECT_ATTACHED", epoch });
expect(m.phase.name).toBe("idle");
});
it("a late ATTACH_LIVE / ATTACH_NONE after leaving `attaching` is ignored", () => {
let m = reduce(initialMachine(), { type: "ATTACH_START", runId: "r" });
const epoch = m.ctx.epoch;
m = reduce(m, { type: "ATTACH_NONE", epoch }); // attaching -> polling
m = reduce(m, { type: "POLL_TERMINAL" }); // -> idle (epoch unchanged)
expect(m.phase.name).toBe("idle");
m = reduce(m, { type: "ATTACH_LIVE", epoch }); // late 2xx, same epoch
expect(m.phase.name).toBe("idle");
// And a late ATTACH_NONE (not `attaching`) is a no-op too.
m = reduce(m, { type: "ATTACH_NONE", epoch });
expect(m.phase.name).toBe("idle");
});
});
// Review #2: every terminal transition resets ownership to local.
describe("run-fsm — review-2: terminal transitions reset ownership to local", () => {
const observer = (): Machine => {
let m = reduce(initialMachine(), { type: "ATTACH_START", runId: "r" });
m = reduce(m, { type: "ATTACH_LIVE", epoch: m.ctx.epoch });
expect(m.ctx.ownership).toBe("observer");
return m;
};
it("FINISH_CLEAN resets ownership", () => {
const m = reduce(observer(), { type: "FINISH_CLEAN", epoch: observer().ctx.epoch });
expect(m.ctx.ownership).toBe("local");
});
it("FINISH_ERROR / POLL_TERMINAL / RUN_FACT(null) reset ownership", () => {
let o = observer();
expect(reduce(o, { type: "FINISH_ERROR", kind: "stream", epoch: o.ctx.epoch }).ctx.ownership).toBe("local");
// POLL_TERMINAL from an observer polling phase
let p = reduce(observer(), { type: "STREAM_INCOMPLETE", reason: "starved", epoch: observer().ctx.epoch });
expect(reduce(p, { type: "POLL_TERMINAL" }).ctx.ownership).toBe("local");
// RUN_FACT(null) from an observer attaching phase
let a = reduce(initialMachine(), { type: "ATTACH_START", runId: "r" });
expect(reduce(a, { type: "RUN_FACT", runFact: null, epoch: a.ctx.epoch }).ctx.ownership).toBe("local");
});
});
describe("run-fsm — ownership (I2) is context, orthogonal to phase", () => {
it("attach/reconnect set observer; send/supersede-ready set local", () => {
let m = reduce(initialMachine(), { type: "ATTACH_START", runId: "r" });
expect(m.ctx.ownership).toBe("observer");
m = reduce(m, { type: "ATTACH_LIVE", epoch: m.ctx.epoch });
expect(m.phase.name).toBe("streaming");
expect(m.ctx.ownership).toBe("observer"); // still observing a detached run
// A local send flips ownership back to local.
m = reduce(m, { type: "SEND_LOCAL" });
expect(m.ctx.ownership).toBe("local");
});
});
describe("run-fsm — dispose (I5)", () => {
it("DISPOSE from any phase aborts controllers and bumps epoch", () => {
let m = reduce(withRunFact(), { type: "ATTACH_START", runId: "r" });
const before = m.ctx.epoch;
m = reduce(m, { type: "DISPOSE" });
expect(m.phase.name).toBe("idle");
expect(m.ctx.epoch).toBe(before + 1);
expect(effectTypes(m)).toEqual(
expect.arrayContaining(["abortAttach", "cancelReconnect", "disarmPoll"]),
);
});
});
@@ -0,0 +1,600 @@
/**
* Run-lifecycle finite state machine for a single AI-chat thread (#488).
*
* ============================================================================
* WHY THIS EXISTS
* ----------------------------------------------------------------------------
* The resume/reconnect/poll/stop/supersede lifecycle used to be spread across
* ~26 `useRef` one-shot flags in `chat-thread.tsx`, each disarmed "on every
* path". Ownerless flag combinations produced silent UI freezes, and every fix
* added another ref (the #381 -> #432 -> #456 spiral). This module replaces that
* ref-zoo with ONE pure reducer whose transitions are enumerable and unit-
* testable in isolation (event x state -> next state is the observable property).
*
* The reducer is PURE: it owns no timers, no fetches, no React state. It maps
* `(machine, event) -> machine`, where the returned machine carries the list of
* COMMAND EFFECTS to run for that transition. A thin runtime in `chat-thread.tsx`
* dispatches events (from SDK callbacks / HTTP outcomes) and executes the
* effects (attach GET, POST /stream, POST /run, POST /stop, backoff timers,
* poll arm/disarm). The runtime lives in a THREAD, not the window, so a late SDK
* callback dies with the owner (kills the "event from a dead view" class, #161).
*
* ============================================================================
* INVARIANTS (see run-fsm.spec.md for the full spec + tables)
* ----------------------------------------------------------------------------
* I1 EPOCH (generation counter). Commands (`resumeStream`, `postRun`, `stop`,
* `supersede`, `scheduleReconnect`) are async; their outcomes arrive on the
* SAME SDK/HTTP callbacks. Every command-emitting transition increments
* `ctx.epoch`; every OUTCOME event carries the epoch it was issued under;
* the reducer DROPS an outcome whose epoch != the current epoch. This is
* what the one-shot-ref zoo used to approximate by hand.
* I2 OWNERSHIP is a CONTEXT FIELD (`'local' | 'observer'`), not a state
* orthogonal to the transport phase. The queue is flushed ONLY by a local
* owner (an observer following a detached run never flushes).
* I3 RUN-FACT ("a run is active") is first-class from the server: `runFact`
* holds the server-confirmed active run id (POST /run on mount, the `start`
* metadata runId, attach outcomes). Reconnect is entered by the RUN-FACT,
* not by the presence of an assistant message (#488 commit 2). A fresh
* negative fact (null) cancels reconnect immediately.
* I4 Exit `stopping` by DATA (a terminal row / negative run-fact), NEVER by the
* stopRun HTTP response (which returns after abort, before finalization).
* I5 Command controllers are effect-owned (abort in cleanup), NOT render-phase
* refs expressed here as the `abortAttach` effect on disposing transitions.
* ============================================================================
*/
// ---------------------------------------------------------------------------
// Phases (the transport lifecycle). Ownership / runFact are CONTEXT, not here.
// ---------------------------------------------------------------------------
/** Why the degraded poll is the active recovery. */
export type PollReason =
| "attach-none" // mount attach returned 204 / error — nothing live to attach
| "starved" // a resumed finish carried no visible content
| "disconnect-visible" // a live disconnect WITH on-screen content — poll to terminal
| "reconnect-exhausted"; // the live re-attach ladder gave up
/** The classified error kind (drives the banner text + composer behavior). */
export type ErrorKind =
| "stream" // a generic provider/network stream error (useChat error)
| "run-already-active" // 409 A_RUN_ALREADY_ACTIVE (a plain POST hit the gate)
| "supersede-mismatch" // 409 SUPERSEDE_TARGET_MISMATCH (CAS target moved)
| "supersede-timeout" // 409 SUPERSEDE_TIMEOUT (old run did not settle in W)
| "supersede-invalid" // 409 SUPERSEDE_INVALID (bad supersede target)
| "begin-failed"; // 503 A_RUN_BEGIN_FAILED (could not start the run)
export type Phase =
| { name: "idle" }
| { name: "sending" } // local POST in flight, before the first frame
| { name: "streaming" } // receiving frames
| { name: "attaching" } // mount-time attach GET in flight
| { name: "reconnecting"; attempt: number; failed: boolean }
| { name: "polling"; reason: PollReason }
| { name: "stalled" } // poll hit the inactivity cap — banner + Retry
| { name: "stopping" }
| { name: "superseding" }
| { name: "error"; kind: ErrorKind };
export type Ownership = "local" | "observer";
/** The server-confirmed active run, or null when no run is active. */
export type RunFact = { runId: string } | null;
export interface Ctx {
/** I1: generation counter — every command-transition increments it. */
epoch: number;
/** I2: does THIS client own the turn's writes (local streamer) or observe? */
ownership: Ownership;
/** I3: the server-confirmed active run. */
runFact: RunFact;
/**
* Are we FOLLOWING a live run we were locally streaming (the reconnect ladder),
* as opposed to a one-shot mount-attach resume? Both are `ownership: 'observer'`,
* but they recover DIFFERENTLY on a drop: a live-follow drop RE-ENTERS the
* reconnect ladder (#488 commit 3 the second break after a successful re-attach
* must reconnect again, not fall to silent poll), while a mount-resume drop falls
* to the degraded poll. This is the ctx bit that separates the two WITHOUT a new
* component ref (it is why commit 3 needs the FSM, not a surgical patch).
*/
liveFollow: boolean;
}
export interface Machine {
phase: Phase;
ctx: Ctx;
/** Command effects to run for the transition that produced THIS machine.
* The runtime executes them and does not read them again. */
effects: Effect[];
}
// ---------------------------------------------------------------------------
// Command effects (the reducer's only side-channel — executed by the runtime).
// ---------------------------------------------------------------------------
export type Effect =
/** POST /run to (re)establish or verify the run-fact. `reason` is diagnostic. */
| { type: "postRun"; reason: "mount" | "verify" }
/** Trigger the SDK `resumeStream()` (attach GET via prepareReconnectToStream). */
| { type: "resumeStream" }
/** Schedule a reconnect attempt after a backoff, then dispatch RECONNECT_ATTEMPT. */
| { type: "scheduleReconnect"; attempt: number; delayMs: number }
/** Cancel any pending reconnect backoff timer. */
| { type: "cancelReconnect" }
/** Arm the degraded poll (the window's dumb timer follows the run in the DB). */
| { type: "armPoll"; reason: PollReason }
/** Disarm the degraded poll. */
| { type: "disarmPoll" }
/** POST /stop the chat's active run (authoritative detached-run stop). */
| { type: "stopRun" }
/** POST /stream { supersede: { runId } } — the CAS "interrupt and send now". */
| { type: "supersede"; targetRunId: string }
/** Abort the in-flight attach/reconnect GET controller (dispose / observer stop). */
| { type: "abortAttach" };
// ---------------------------------------------------------------------------
// Events. An OUTCOME event MAY carry `epoch`; if it does and it does not equal
// the current epoch, the reducer drops it (I1). Trigger events (user actions,
// fresh disconnects) carry no epoch and are never dropped.
// ---------------------------------------------------------------------------
export type Event =
// -- local turn --
| { type: "SEND_LOCAL" }
| { type: "STREAM_START"; runId?: string; epoch?: number }
/** An OBSERVER's attached stream ended WITHOUT reaching terminal (a starved
* clean replay, or a torn resume) fall to the degraded poll to drive the row
* to its real terminal state. (A live-follow drop uses FINISH_DISCONNECT.) */
| { type: "STREAM_INCOMPLETE"; reason: PollReason; epoch?: number }
| { type: "FINISH_CLEAN"; epoch?: number }
| { type: "FINISH_ABORT"; epoch?: number }
| { type: "FINISH_DISCONNECT"; hasVisibleContent: boolean; epoch?: number }
| { type: "FINISH_ERROR"; kind: ErrorKind; epoch?: number }
// -- mount attach (resume) --
| { type: "ATTACH_START"; runId?: string }
| { type: "ATTACH_LIVE"; epoch?: number }
| { type: "ATTACH_NONE"; epoch?: number }
// -- reconnect after a live disconnect (entered by FINISH_DISCONNECT, #488 c2) --
| { type: "RECONNECT_ATTEMPT"; attempt: number; epoch?: number }
| { type: "RECONNECT_ATTACHED"; epoch?: number }
| { type: "RECONNECT_NONE"; epoch?: number }
| { type: "RETRY" }
// -- degraded poll --
| { type: "POLL_TERMINAL" }
| { type: "POLL_IDLE_CAP" }
// -- run-fact (server-confirmed active run) --
| { type: "RUN_FACT"; runFact: RunFact; epoch?: number }
// -- stop --
| { type: "STOP_REQUESTED" }
// -- supersede (CAS) --
| { type: "SUPERSEDE_REQUESTED"; targetRunId: string }
| { type: "SUPERSEDE_READY"; runId?: string; epoch?: number }
| { type: "SUPERSEDE_MISMATCH"; currentRunId?: string; epoch?: number }
| { type: "SUPERSEDE_TIMEOUT"; epoch?: number }
| { type: "SUPERSEDE_INVALID"; epoch?: number }
| { type: "RUN_ALREADY_ACTIVE"; activeRunId?: string }
// -- lifecycle --
| { type: "DISPOSE" };
export const RECONNECT_MAX_ATTEMPTS = 5;
export const RECONNECT_BASE_DELAY_MS = 1000;
/** Backoff before attempt N (1-based): 1s, 2s, 4s, 8s, 16s. */
export function reconnectDelayMs(attempt: number): number {
return RECONNECT_BASE_DELAY_MS * 2 ** (attempt - 1);
}
// ---------------------------------------------------------------------------
// Constructors / helpers.
// ---------------------------------------------------------------------------
export function initialMachine(overrides?: Partial<Ctx>): Machine {
return {
phase: { name: "idle" },
ctx: { epoch: 0, ownership: "local", runFact: null, liveFollow: false, ...overrides },
effects: [],
};
}
/** Build a machine result: a phase, optional ctx patch, and effects. Empty
* effects by default. Never mutates the input. */
function to(
m: Machine,
phase: Phase,
opts?: { ctx?: Partial<Ctx>; effects?: Effect[] },
): Machine {
return {
phase,
ctx: { ...m.ctx, ...(opts?.ctx ?? {}) },
effects: opts?.effects ?? [],
};
}
/** No transition: keep the phase, clear effects (so a re-run does not re-fire). */
function stay(m: Machine): Machine {
return { phase: m.phase, ctx: m.ctx, effects: [] };
}
/** A command-transition: same as `to` but bumps the epoch (I1). Any outcome
* event issued under the old epoch is dropped once this lands. */
function command(
m: Machine,
phase: Phase,
effects: Effect[],
ctx?: Partial<Ctx>,
): Machine {
return {
phase,
ctx: { ...m.ctx, ...(ctx ?? {}), epoch: m.ctx.epoch + 1 },
effects,
};
}
// ---------------------------------------------------------------------------
// The pure reducer.
// ---------------------------------------------------------------------------
/** The terminal stream-finish events (one turn's stream ended). */
function isFinishEvent(event: Event): boolean {
return (
event.type === "FINISH_ABORT" ||
event.type === "FINISH_CLEAN" ||
event.type === "FINISH_DISCONNECT" ||
event.type === "FINISH_ERROR" ||
event.type === "STREAM_INCOMPLETE"
);
}
export function reduce(m: Machine, event: Event): Machine {
// MEDIUM (#488 re-review): honor ANY stream finish in `stopping` regardless of
// generation. A plain user Stop has NO successor stream — the aborted stream's
// finish IS the expected end of the stop, so exit `stopping -> idle` by that DATA
// (I4). The epoch filter below must NOT drop it: STOP_REQUESTED bumped the epoch,
// but the finish carries the PRE-stop generation (the runtime stamps it with the
// stream's start epoch), so I1 would otherwise strand the machine in `stopping`
// forever (no idle-cap covers `stopping`). The epoch filter stays in force for
// `superseding` (a successor B owns) — that is the F1 supersede drop.
if (m.phase.name === "stopping" && isFinishEvent(event)) {
return to(m, { name: "idle" }, {
// Reset ownership to local on this terminal transition (review #2): otherwise
// an observer-stop leaves ownership 'observer' and hides "Send now" forever.
ctx: { runFact: null, liveFollow: false, ownership: "local" },
effects: [{ type: "disarmPoll" }, { type: "cancelReconnect" }],
});
}
// I1: drop a stale outcome (an event issued under a superseded epoch).
if ("epoch" in event && event.epoch !== undefined && event.epoch !== m.ctx.epoch) {
return stay(m);
}
switch (event.type) {
// ---- local turn ----------------------------------------------------
case "SEND_LOCAL":
// A local send owns the view: leave any recovery, become the local
// streamer, disarm poll/reconnect. epoch++ so a late recovery outcome
// from the previous phase is dropped.
return command(
m,
{ name: "sending" },
[{ type: "cancelReconnect" }, { type: "disarmPoll" }],
{ ownership: "local", liveFollow: false },
);
case "STREAM_INCOMPLETE":
// An OBSERVER's attached stream ended incomplete (starved / torn) — follow
// the run to terminal via the degraded poll.
return to(m, { name: "polling", reason: event.reason }, {
effects: [{ type: "armPoll", reason: event.reason }],
});
case "STREAM_START": {
// First frame arrived. Adopt the run-fact runId if present. sending ->
// streaming; a reconnect/attach that just went live also lands here.
const runFact = event.runId ? { runId: event.runId } : m.ctx.runFact;
return to(m, { name: "streaming" }, {
ctx: { runFact },
effects: [{ type: "cancelReconnect" }, { type: "disarmPoll" }],
});
}
case "FINISH_CLEAN":
// A clean terminal outcome. The run is done — clear the run-fact and go
// idle. (The queue flush is a component concern gated by ownership; the
// FSM only models the phase.) Review #2: reset ownership to local so a
// just-finished observer-attach turn re-exposes "Send now" for the queue.
return to(m, { name: "idle" }, {
ctx: { runFact: null, liveFollow: false, ownership: "local" },
effects: [{ type: "disarmPoll" }, { type: "cancelReconnect" }],
});
case "FINISH_ABORT":
// A user Stop / intentional abort finished. If we were stopping, the
// terminal data has now arrived (I4) — go idle. The run-fact is cleared.
return to(m, { name: "idle" }, {
ctx: { runFact: null, liveFollow: false, ownership: "local" },
effects: [{ type: "disarmPoll" }, { type: "cancelReconnect" }],
});
case "FINISH_DISCONNECT":
// A LIVE SSE drop. Recovery depends on WHO we are (I2 + liveFollow):
// - a mount-attach OBSERVER (a one-shot resume, NOT live-follow) that drops
// -> the degraded poll drives the row to terminal from the DB.
if (m.ctx.ownership === "observer" && !m.ctx.liveFollow) {
return to(m, { name: "polling", reason: "disconnect-visible" }, {
effects: [{ type: "armPoll", reason: "disconnect-visible" }],
});
}
// - a LOCAL live turn (first drop) OR a live-follow re-attach (a SUBSEQUENT
// drop) -> (re-)enter the reconnect ladder. #488 commit 3: allowed
// REPEATEDLY — `liveFollow` is kept across a successful re-attach, so the
// second break reconnects again instead of falling to silent poll.
// #488 commit 2: gated on the RUN-FACT (or an existing live-follow), NOT on
// the presence of an assistant message — a setup-phase break still recovers.
// - visible content already on screen -> keep it, ALSO poll to terminal
// (a full replay could clobber the fuller live tail);
// - no visible content -> the reconnect ladder rebuilds it.
if (m.ctx.runFact || m.ctx.liveFollow) {
const effects: Effect[] = [
{ type: "scheduleReconnect", attempt: 1, delayMs: reconnectDelayMs(1) },
];
if (event.hasVisibleContent) effects.push({ type: "armPoll", reason: "disconnect-visible" });
return command(m, { name: "reconnecting", attempt: 1, failed: false }, effects, {
ownership: "observer",
liveFollow: true,
});
}
// No run to recover: a plain disconnect. Surface the terminal notice.
return to(m, { name: "idle" }, {
ctx: { runFact: null, liveFollow: false, ownership: "local" },
});
case "FINISH_ERROR":
return to(m, { name: "error", kind: event.kind }, {
ctx: { runFact: null, liveFollow: false, ownership: "local" },
effects: [{ type: "disarmPoll" }, { type: "cancelReconnect" }],
});
// ---- mount attach (resume) ----------------------------------------
case "ATTACH_START":
// A reopened tab attaches to a still-running run: observer ownership.
// #488 F2: ONLY from idle. The mount `getRun` round-trip resolves async, and
// a local send may have started meanwhile (phase `sending`, ownership local);
// a late ATTACH_START must NOT hijack that local turn into an observer-attach
// (queue would stop flushing, "Send now" would hide). Guarding in the reducer
// covers every dispatch source.
if (m.phase.name !== "idle") return stay(m);
return command(m, { name: "attaching" }, [{ type: "resumeStream" }], {
ownership: "observer",
runFact: event.runId ? { runId: event.runId } : m.ctx.runFact,
});
case "ATTACH_LIVE":
// The attach GET returned a live 2xx stream — follow it as an observer.
// Review #1: guard by SOURCE phase. The epoch filter alone is not enough — a
// POLL_TERMINAL uses to() (no epoch bump) and does not abort the in-flight
// GET, so a slow 2xx landing after the machine already left `attaching` (e.g.
// the armed poll saw the terminal row -> idle) would resurrect a settled run
// into a phantom `streaming`. Only enter streaming FROM `attaching`.
if (m.phase.name !== "attaching") return stay(m);
return to(m, { name: "streaming" });
case "ATTACH_NONE":
// 204 / non-2xx / throw: nothing live to attach. Arm the degraded poll to
// follow the run to terminal from the DB. This is a soft-negative run-fact
// (204 on a non-stripped path is authoritative-negative; the runtime may
// pass a RUN_FACT null separately). Keep the run-fact as-is here.
// Review #1: guard by source phase for consistency (a late outcome after the
// machine already left `attaching` must not re-arm a poll).
if (m.phase.name !== "attaching") return stay(m);
return to(m, { name: "polling", reason: "attach-none" }, {
effects: [{ type: "armPoll", reason: "attach-none" }],
});
// ---- reconnect after a live disconnect ----------------------------
case "RECONNECT_ATTEMPT":
// A scheduled backoff fired — fire the attach GET. epoch++ so the previous
// attempt's late outcome cannot drive this one.
if (m.phase.name !== "reconnecting") return stay(m);
return command(
m,
{ name: "reconnecting", attempt: event.attempt, failed: false },
[{ type: "resumeStream" }],
);
case "RECONNECT_ATTACHED":
// #488 commit 3: a live re-attach succeeded. Reset to streaming — the
// attempt counter is dropped, so a LATER disconnect can start a fresh
// ladder from attempt 1 (the old one-shot `!wasResumed` gate forbade a
// second cycle, sending the second break to silent poll).
// Review #1: guard by SOURCE phase. The armed degraded poll can reach the
// terminal row (POLL_TERMINAL -> idle, via to(), NO epoch bump, GET not
// aborted) BEFORE a slow reconnect GET returns 2xx; without this guard that
// late RECONNECT_ATTACHED (same epoch) would resurrect a settled run into a
// phantom `streaming`. Only re-enter streaming FROM `reconnecting`.
if (m.phase.name !== "reconnecting") return stay(m);
return to(m, { name: "streaming" }, {
effects: [{ type: "cancelReconnect" }, { type: "disarmPoll" }],
});
case "RECONNECT_NONE": {
// 204 / error during a reconnect attempt. Arm the degraded poll as the
// belt-and-suspenders fallback, then either back off to the next attempt
// or, at the cap, surface the manual Retry ("failed").
if (m.phase.name !== "reconnecting") return stay(m);
const attempt = m.phase.attempt;
if (attempt < RECONNECT_MAX_ATTEMPTS) {
return command(
m,
{ name: "reconnecting", attempt: attempt + 1, failed: false },
[
{ type: "armPoll", reason: "attach-none" },
{ type: "scheduleReconnect", attempt: attempt + 1, delayMs: reconnectDelayMs(attempt + 1) },
],
);
}
return to(m, { name: "reconnecting", attempt, failed: true }, {
effects: [{ type: "armPoll", reason: "reconnect-exhausted" }],
});
}
case "RETRY":
// Manual Retry from the "failed" reconnect banner OR the stalled banner.
if (m.phase.name === "reconnecting" && m.phase.failed) {
return command(
m,
{ name: "reconnecting", attempt: 1, failed: false },
[{ type: "resumeStream" }],
);
}
if (m.phase.name === "stalled") {
// Re-arm the poll to try to catch the run up again.
return command(m, { name: "polling", reason: "attach-none" }, [
{ type: "armPoll", reason: "attach-none" },
]);
}
return stay(m);
// ---- degraded poll -------------------------------------------------
case "POLL_TERMINAL":
// The run reached a terminal row via the poll (or the reconcile merge). Go
// idle and disarm everything (I4: this is a DATA-driven exit, incl. exit
// from `stopping`). Review #2: reset ownership to local.
return to(m, { name: "idle" }, {
ctx: { runFact: null, liveFollow: false, ownership: "local" },
effects: [{ type: "disarmPoll" }, { type: "cancelReconnect" }],
});
case "POLL_IDLE_CAP":
// Review #4: `stopping` also arms the poll (STOP_REQUESTED) but has NO other
// backstop — an observer-stop with no SDK stream to fire onFinish, whose
// server stop never drives the run terminal, would poll the DB forever. Give
// it a bounded exit: cap -> idle + disarm (NOT `stalled`; Stop was already
// pressed, so there is nothing for the user to retry).
if (m.phase.name === "stopping") {
return to(m, { name: "idle" }, {
ctx: { runFact: null, liveFollow: false, ownership: "local" },
effects: [{ type: "disarmPoll" }, { type: "cancelReconnect" }],
});
}
// #488 commit 4a: the poll hit the inactivity cap. Instead of going SILENT
// (the old "forever half-done answer"), surface a stalled banner + Retry.
if (m.phase.name !== "polling" && m.phase.name !== "reconnecting") return stay(m);
return to(m, { name: "stalled" }, {
effects: [{ type: "disarmPoll" }, { type: "cancelReconnect" }],
});
// ---- run-fact ------------------------------------------------------
case "RUN_FACT": {
const runFact = event.runFact;
// A fresh NEGATIVE fact (no active run) cancels recovery immediately (I3):
// there is nothing to reconnect to / poll for.
if (!runFact) {
if (
m.phase.name === "reconnecting" ||
m.phase.name === "attaching" ||
m.phase.name === "polling" ||
m.phase.name === "stopping"
) {
return to(m, { name: "idle" }, {
// Review #2: reset ownership to local on this terminal transition.
ctx: { runFact: null, liveFollow: false, ownership: "local" },
effects: [{ type: "cancelReconnect" }, { type: "disarmPoll" }],
});
}
return to(m, m.phase, { ctx: { runFact: null } });
}
// A positive fact just updates the context (pessimism toward an attempt: a
// stale-but-positive fact permits entering recovery; a 204 will cut it).
return to(m, m.phase, { ctx: { runFact } });
}
// ---- stop ----------------------------------------------------------
case "STOP_REQUESTED":
// Authoritative stop of a detached run. Enter `stopping` and fire stopRun +
// abort the local/attach reader. ALSO arm the poll so the terminal row is
// observed — the exit is by DATA (I4: a terminal row / negative run-fact),
// never by the stopRun HTTP response (which returns after abort, before
// finalization). For a local turn the aborted stream's onFinish (ANY finish)
// is HONORED in `stopping` at the top of reduce() — regardless of generation
// — and exits to idle; the armed poll is the fallback for an observer stop
// with no local onFinish.
return command(
m,
{ name: "stopping" },
[
{ type: "stopRun" },
{ type: "abortAttach" },
{ type: "cancelReconnect" },
{ type: "armPoll", reason: "attach-none" },
],
);
// ---- supersede (CAS) ----------------------------------------------
case "SUPERSEDE_REQUESTED":
// "Interrupt and send now": CAS POST /stream { supersede }. epoch++ so a
// late outcome of the interrupted run is dropped.
return command(
m,
{ name: "superseding" },
[{ type: "supersede", targetRunId: event.targetRunId }, { type: "cancelReconnect" }, { type: "disarmPoll" }],
);
case "SUPERSEDE_READY": {
// CAS succeeded (old run stopped/settled, slot taken, new run begun). We
// are now the local streamer of the NEW run. Adopt its runId if provided.
const runFact = event.runId ? { runId: event.runId } : m.ctx.runFact;
return to(m, { name: "streaming" }, {
ctx: { ownership: "local", runFact, liveFollow: false },
});
}
case "SUPERSEDE_MISMATCH":
// The active run moved between the click and the CAS. Per the spec: verify
// via /run rather than blindly banner — the mismatch may be our own already-
// superseded run. Surface a classified error AND fire a run-fact verify.
return to(m, { name: "error", kind: "supersede-mismatch" }, {
ctx: { runFact: event.currentRunId ? { runId: event.currentRunId } : m.ctx.runFact },
effects: [{ type: "postRun", reason: "verify" }],
});
case "SUPERSEDE_TIMEOUT":
// The old run did not settle within W. Nothing persisted; the composer keeps
// its text. Classified error, NO auto-retry (the old client retry ladder is
// removed in #488 commit 5).
return to(m, { name: "error", kind: "supersede-timeout" });
case "SUPERSEDE_INVALID":
return to(m, { name: "error", kind: "supersede-invalid" });
case "RUN_ALREADY_ACTIVE":
// A plain POST hit the one-active-run gate. NO auto-retry — the composer
// offers "interrupt and send" (supersede) instead. #497/S4: adopt the
// server's activeRunId as the run-fact so that supersede can TARGET the
// (possibly foreign-tab) active run via the CAS, rather than a blind
// promote+abort that just 409s again. A stale/absent id keeps the prior fact.
return to(m, { name: "error", kind: "run-already-active" }, {
ctx: { runFact: event.activeRunId ? { runId: event.activeRunId } : m.ctx.runFact },
});
// ---- lifecycle -----------------------------------------------------
case "DISPOSE":
// Unmount: abort in-flight controllers, drop timers, and bump the epoch so
// NO late callback can drive this (now dead) machine (I5).
return command(
m,
{ name: "idle" },
[
{ type: "abortAttach" },
{ type: "cancelReconnect" },
{ type: "disarmPoll" },
],
{ liveFollow: false },
);
default: {
// Exhaustiveness guard.
const _never: never = event;
void _never;
return stay(m);
}
}
}
@@ -3,6 +3,7 @@ import {
resolveAdoptedChatId,
newlyAddedChatIds,
extractServerChatId,
extractRunId,
} from "./adopt-chat-id";
describe("resolveAdoptedChatId", () => {
@@ -70,3 +71,17 @@ describe("extractServerChatId", () => {
expect(extractServerChatId(undefined)).toBeUndefined();
});
});
describe("extractRunId", () => {
it("reads a string runId from the start metadata", () => {
expect(extractRunId({ metadata: { runId: "run-1" } })).toBe("run-1");
});
it("returns undefined when runId is absent", () => {
expect(extractRunId({ metadata: { chatId: "c" } })).toBeUndefined();
expect(extractRunId({})).toBeUndefined();
expect(extractRunId(undefined)).toBeUndefined();
});
it("returns undefined for a non-string runId", () => {
expect(extractRunId({ metadata: { runId: 7 } })).toBeUndefined();
});
});
@@ -56,6 +56,20 @@ export function extractServerChatId(
return typeof m?.chatId === "string" ? m.chatId : undefined;
}
/**
* #488: read the authoritative RUN id off a streaming assistant message. The
* server attaches it as `message.metadata.runId` on the `start` part when a run
* wraps the turn (see server `chatStreamMetadata`, #184/#487). This is the live
* run-fact update the client FSM adopts (mirrors `extractServerChatId`). Returns
* it only when it is a string; undefined otherwise.
*/
export function extractRunId(
message: { metadata?: unknown } | undefined,
): string | undefined {
const m = message?.metadata as { runId?: string } | undefined;
return typeof m?.runId === "string" ? m.runId : undefined;
}
/**
* The deduped set of ids present in `afterIds` but not in `beforeIds`. A
* paginated/flatMapped list can repeat the same id, so dedupe: one genuinely-new
@@ -6,10 +6,13 @@ describe("estimateTokens", () => {
expect(estimateTokens("")).toBe(0);
});
it("ceils chars/4 so any non-empty text is at least 1 token", () => {
// #490: migrated onto the shared @docmost/token-estimate module (chars/2.5, up
// from the old client-only chars/4) so the client counter and the server replay
// budgeter can never diverge.
it("ceils chars/2.5 so any non-empty text is at least 1 token", () => {
expect(estimateTokens("a")).toBe(1);
expect(estimateTokens("abcd")).toBe(1);
expect(estimateTokens("abcde")).toBe(2);
expect(estimateTokens("12345678")).toBe(2);
expect(estimateTokens("ab")).toBe(1);
expect(estimateTokens("abcde")).toBe(2); // 5 / 2.5 = 2
expect(estimateTokens("x".repeat(10))).toBe(4); // 10 / 2.5 = 4
});
});
@@ -2,18 +2,10 @@
* Rough client-side token estimation for AI-chat UI affordances.
*
* No provider streams exact per-token usage mid-stream, so any in-flight figure
* is a CLIENT ESTIMATE (chars/4 heuristic). Pure + unit-testable: it never runs
* a real BPE tokenizer (that would be O(n²) on the hot path, bloat the bundle,
* and be wrong for Gemini/Ollama anyway). Used by the in-body reasoning counter
* ("Thinking · N tokens").
* is a CLIENT ESTIMATE. This re-exports the SHARED estimator from
* `@docmost/token-estimate` (chars/2.5) so the in-body counter and the server's
* replay budgeter use the SAME heuristic two divergent estimators would mean
* "the badge shows 60%" while "the budgeter already trimmed" (#490). Used by the
* in-body reasoning counter ("Thinking · N tokens").
*/
/**
* Rough token estimate for a piece of text using the standard chars/4 heuristic.
* Returns 0 for empty/whitespace-free-of-content input, and ceils so any
* non-empty text counts as at least one token.
*/
export function estimateTokens(text: string): number {
if (!text) return 0;
return Math.ceil(text.length / 4);
}
export { estimateTokens } from "@docmost/token-estimate";
@@ -42,6 +42,70 @@ describe("describeChatError", () => {
);
});
// #488 commit 5: the #487 concurrency-gate / supersede 409s. FULL real bodies:
// a ConflictException(object) whose response is serialized verbatim, carrying a
// `code` and statusCode 409. Each must classify to a human text, not raw JSON.
it("classifies A_RUN_ALREADY_ACTIVE (409) as already-answering, not raw JSON", () => {
const body =
'{"message":"A run is already active for this chat","code":"A_RUN_ALREADY_ACTIVE","statusCode":409}';
expect(describeChatError(body, t).title).toBe(
"The agent is already answering",
);
// Never leaks the raw code as the detail.
expect(describeChatError(body, t).detail).not.toContain("A_RUN_ALREADY_ACTIVE");
});
it("classifies SUPERSEDE_TARGET_MISMATCH (409) as run-changed", () => {
// Real server body shape: the current run id is `activeRunId` (NOT `runId`) —
// see ai-chat.controller.ts. describeChatError classifies off `code` only.
const body =
'{"message":"active run does not match the supersede target","code":"SUPERSEDE_TARGET_MISMATCH","activeRunId":"run-x","statusCode":409}';
expect(describeChatError(body, t).title).toBe(
"Couldn't interrupt — the run changed",
);
});
it("classifies SUPERSEDE_TIMEOUT (409) as couldn't-interrupt-in-time", () => {
const body =
'{"message":"the run did not settle within the supersede window","code":"SUPERSEDE_TIMEOUT","statusCode":409}';
expect(describeChatError(body, t).title).toBe("Couldn't interrupt in time");
});
it("classifies SUPERSEDE_INVALID (409) as couldn't-interrupt-that-run", () => {
const body =
'{"message":"supervise requires chatId","code":"SUPERSEDE_INVALID","statusCode":409}';
expect(describeChatError(body, t).title).toBe(
"Couldn't interrupt that run",
);
});
it("ORDER GUARD: A_RUN_ALREADY_ACTIVE wins over any generic status branch", () => {
// Even though the body could superficially look 4xx-ish, the code branch runs
// first, so it is never mislabeled by a generic status heading.
const body =
'{"message":"conflict","code":"A_RUN_ALREADY_ACTIVE","statusCode":409}';
const view = describeChatError(body, t);
expect(view.title).not.toBe("Something went wrong");
expect(view.title).not.toBe("AI provider not configured");
});
it("classifies a token-degeneration abort under the SAME 'Response stopped.' marker the live view shows (#495)", () => {
// The exact reason the server persists in metadata.error on a degeneration
// abort (ai-chat.service OUTPUT_DEGENERATION_ERROR). Live, this event shows
// the neutral "Response stopped." notice; the persisted banner MUST match it
// so live and refetch never disagree.
const view = describeChatError(
"Output degeneration detected (repeated token loop)",
t,
);
expect(view.title).toBe("Response stopped.");
expect(view.detail).toBe(
"The answer was stopped automatically because the model fell into a repeated output loop.",
);
// Regression guard: it must NOT fall through to the generic heading.
expect(view.title).not.toBe("Something went wrong");
});
it("classifies a dropped connection (ECONNRESET) as a lost-connection error", () => {
expect(
describeChatError("Cannot connect to API: read ECONNRESET", t).title,
@@ -39,6 +39,60 @@ export function describeChatError(
};
}
// #488 commit 5: the #487 concurrency-gate / supersede 409s. These arrive as a
// ConflictException(object) body carrying a `code` (and statusCode 409). They
// MUST be classified by `code` STRICTLY BEFORE any generic status branch, or the
// user sees the raw JSON `{"code":"A_RUN_ALREADY_ACTIVE",…}`. The code strings
// are the real #487 server contract (ai-chat.controller.ts) — do not invent.
if (/"code"\s*:\s*"A_RUN_ALREADY_ACTIVE"/.test(msg)) {
return {
title: t("The agent is already answering"),
detail: t(
"This chat already has a run in progress. Wait for it to finish, or interrupt it and send now.",
),
};
}
if (/"code"\s*:\s*"SUPERSEDE_TARGET_MISMATCH"/.test(msg)) {
return {
title: t("Couldn't interrupt — the run changed"),
detail: t(
"The run you tried to interrupt is no longer the active one. Check the latest answer and try again.",
),
};
}
if (/"code"\s*:\s*"SUPERSEDE_TIMEOUT"/.test(msg)) {
return {
title: t("Couldn't interrupt in time"),
detail: t(
"The previous run didn't stop in time. Nothing was sent — try sending again.",
),
};
}
if (/"code"\s*:\s*"SUPERSEDE_INVALID"/.test(msg)) {
return {
title: t("Couldn't interrupt that run"),
detail: t(
"The run to interrupt doesn't belong to this chat. Reload and try again.",
),
};
}
// Our own token-degeneration abort (#444): the server aborts a runaway
// repetition loop and persists this exact reason in metadata.error. LIVE, the
// same abort surfaces as the neutral "Response stopped." notice (the client
// cannot tell it from a manual Stop mid-stream), so the persisted banner must
// read the SAME "Response stopped." marker — otherwise the live view and a
// later refetch show two different texts for one event. The detail explains the
// loop-guard cause without contradicting the shared heading.
if (/output degeneration detected|repeated token loop/i.test(msg)) {
return {
title: t("Response stopped."),
detail: t(
"The answer was stopped automatically because the model fell into a repeated output loop.",
),
};
}
if (/"statusCode"\s*:\s*403\b/.test(msg)) {
return {
title: t("AI chat is disabled"),
@@ -0,0 +1,65 @@
import { describe, it, expect } from "vitest";
import * as Y from "yjs";
import { yHistoryAvailability } from "./use-toolbar-state.ts";
// Undo/redo availability is derived from the Yjs UndoManager's PRIVATE
// `undoStack` / `redoStack` fields (see use-toolbar-state.ts for why we read the
// stack lengths directly instead of the expensive `editor.can().undo()` dry-run).
// These tests lock in the behavior AND pin the library shape so a yjs / y-undo
// upgrade that renames/restructures those internals fails loudly here rather than
// silently enabling/disabling the toolbar buttons in production.
describe("yHistoryAvailability", () => {
it("reports availability from the stack lengths", () => {
expect(yHistoryAvailability({ undoStack: [], redoStack: [] })).toEqual({
canUndo: false,
canRedo: false,
});
expect(
yHistoryAvailability({ undoStack: [{}], redoStack: [] }),
).toEqual({ canUndo: true, canRedo: false });
expect(
yHistoryAvailability({ undoStack: [{}], redoStack: [{}, {}] }),
).toEqual({ canUndo: true, canRedo: true });
});
it("returns null when the private stack shape is unrecognized (upgrade guard)", () => {
// Simulates a yjs / y-undo upgrade that renames or restructures the private
// fields: the caller then falls back to the safe prosemirror-history default
// instead of throwing on `.length` of undefined or reading garbage.
expect(yHistoryAvailability(undefined)).toBeNull();
expect(yHistoryAvailability(null)).toBeNull();
expect(yHistoryAvailability({})).toBeNull();
expect(yHistoryAvailability({ undoStack: 5, redoStack: 5 })).toBeNull();
// Only one stack present (partial rename) is still not trusted.
expect(yHistoryAvailability({ undoStack: [] })).toBeNull();
});
it("pin-test: a real yjs UndoManager still exposes undoStack/redoStack arrays", () => {
const doc = new Y.Doc();
const text = doc.getText("prosemirror");
const undoManager = new Y.UndoManager(text);
// Fresh manager: both stacks empty -> nothing to undo/redo.
expect(yHistoryAvailability(undoManager)).toEqual({
canUndo: false,
canRedo: false,
});
// A tracked edit must push onto the private undoStack. If a future yjs
// renames these fields, yHistoryAvailability(undoManager) returns null and
// the expectation below fails loudly.
text.insert(0, "hello");
undoManager.stopCapturing();
expect(yHistoryAvailability(undoManager)).toEqual({
canUndo: true,
canRedo: false,
});
// Undoing moves the item to the redoStack -> redo becomes available.
undoManager.undo();
expect(yHistoryAvailability(undoManager)).toEqual({
canUndo: false,
canRedo: true,
});
});
});
@@ -35,6 +35,30 @@ export interface ToolbarState {
// When neither history backend is installed (the pre-sync static editor —
// mainExtensions only, undoRedo disabled), both fall through to 0 -> false,
// matching the previous `safeCan` behavior.
// Reads the Yjs UndoManager's undo/redo availability from its stack lengths.
//
// `undoStack` / `redoStack` are PRIVATE y-undo / yjs internals, so we touch them
// defensively: a yjs or y-undo upgrade that renames or restructures these fields
// must not silently mis-drive the toolbar buttons (nor throw on `.length` of
// `undefined`). We only trust them when they are actually arrays; otherwise this
// returns null and the caller falls back to a safe default. The pin-test in
// use-toolbar-state.test.ts asserts the current library shape, so an upgrade that
// breaks this contract fails loudly there instead of failing silently in the UI.
export function yHistoryAvailability(
undoManager: unknown,
): { canUndo: boolean; canRedo: boolean } | null {
if (!undoManager || typeof undoManager !== "object") return null;
const { undoStack, redoStack } = undoManager as {
undoStack?: unknown;
redoStack?: unknown;
};
if (!Array.isArray(undoStack) || !Array.isArray(redoStack)) return null;
return {
canUndo: undoStack.length > 0,
canRedo: redoStack.length > 0,
};
}
function historyAvailability(editor: Editor): {
canUndo: boolean;
canRedo: boolean;
@@ -43,16 +67,14 @@ function historyAvailability(editor: Editor): {
// Collaboration history (Yjs) takes precedence when present.
const yState = yUndoPluginKey.getState(state) as
| { undoManager?: { undoStack: unknown[]; redoStack: unknown[] } }
| { undoManager?: unknown }
| undefined;
if (yState?.undoManager) {
return {
canUndo: yState.undoManager.undoStack.length > 0,
canRedo: yState.undoManager.redoStack.length > 0,
};
}
const yAvail = yHistoryAvailability(yState?.undoManager);
if (yAvail) return yAvail;
// Plain prosemirror-history (returns 0 when the history plugin is absent).
// This is also the safe default when a Yjs UndoManager is present but its
// private stack shape is no longer recognized (yHistoryAvailability -> null).
return {
canUndo: undoDepth(state) > 0,
canRedo: redoDepth(state) > 0,
@@ -9,7 +9,7 @@ import { Italic } from "@tiptap/extension-italic";
import { Link } from "@tiptap/extension-link";
import { gitmostInsertTranscriptIntoEditor } from "./gitmost-recording.ts";
const ZWSP = "​"; // U+200B, the helper's block-trigger neutralizer
const ZWSP = "​"; // U+200B — asserted ABSENT (the block-escape lives in the serializer now)
/**
* #377 the web-side bridge must append the native host's transcript below the
@@ -18,8 +18,9 @@ const ZWSP = "​"; // U+200B, the helper's block-trigger neutralizer
* regression would be caught), asserting the resulting document rather than
* mocking the editor: transcript present -> "Transcript" heading + one paragraph
* per non-empty line; content is inserted as LITERAL TEXT (no HTML/markdown
* parsing); col-0 markdown block triggers are neutralized so git-sync keeps them
* paragraphs; absent/empty/non-string -> no-op.
* parsing); col-0 markdown block triggers are stored verbatim (the git-sync
* serializer block-escapes them, so no client-side ZWSP is needed);
* absent/empty/non-string -> no-op.
*/
describe("gitmostInsertTranscriptIntoEditor", () => {
const makeEditor = () =>
@@ -91,19 +92,22 @@ describe("gitmostInsertTranscriptIntoEditor", () => {
editor.destroy();
});
it("neutralizes col-0 markdown block triggers with a leading ZWSP (git-sync safety)", () => {
it("inserts col-0 markdown block triggers as verbatim paragraph text (no ZWSP workaround)", () => {
const editor = makeEditor();
// Trigger lines (some with a leaked indent) + a normal prefixed line.
// Trigger lines (some with a leaked indent) + a normal prefixed line. The
// git-sync serializer now block-escapes a leading trigger itself, so the
// bridge inserts each line's TEXT byte-exact (only the leaked indent is
// trimmed) — no invisible ZWSP is prepended anymore.
const inserted = gitmostInsertTranscriptIntoEditor(
editor,
[
"- dash",
" > quote", // leading indent must be trimmed then neutralized
" > quote", // leading indent is trimmed, text otherwise verbatim
"# hash",
"1. one",
"> [!info] note",
"```js",
"---", // solid thematic break -> horizontalRule (text-losing) if unneutralized
"---",
"***",
"___",
"You: normal line",
@@ -116,20 +120,23 @@ describe("gitmostInsertTranscriptIntoEditor", () => {
.map((n: any) => n.content?.[0]?.text)
.filter((t: any) => typeof t === "string") as string[];
// Every block-trigger line is prefixed with the invisible ZWSP (indent
// trimmed first); the normal `You:` line is left byte-exact.
// Each trigger line is stored as its own byte-exact text (indent trimmed);
// the git-sync round-trip keeps it a paragraph via the serializer's
// block-escape, so no ZWSP is needed here.
expect(texts).toEqual([
ZWSP + "- dash",
ZWSP + "> quote",
ZWSP + "# hash",
ZWSP + "1. one",
ZWSP + "> [!info] note",
ZWSP + "```js",
ZWSP + "---",
ZWSP + "***",
ZWSP + "___",
"- dash",
"> quote",
"# hash",
"1. one",
"> [!info] note",
"```js",
"---",
"***",
"___",
"You: normal line",
]);
// Guard: no invisible ZWSP leaked into any inserted line.
for (const t of texts) expect(t).not.toContain(ZWSP);
editor.destroy();
});
@@ -240,45 +240,22 @@ export async function gitmostUploadFileToEditor(
}
}
// Zero-width space (U+200B). Prepended to a transcript line that begins with a
// markdown BLOCK trigger: it is invisible in the rendered doc but shifts the
// trigger off column 0, so the git-sync doc->markdown->doc round-trip keeps the
// line a plain paragraph (see GITMOST_MD_BLOCK_TRIGGER_RE).
const GITMOST_ZWSP = "​";
// A markdown BLOCK-level construct that, sitting at column 0 of a paragraph
// line, the git-sync markdown serializer (packages/prosemirror-markdown
// markdown-converter.ts, `case "paragraph"`) would re-parse into a NON-paragraph
// block on the doc->markdown->doc cycle. That serializer emits paragraph text
// verbatim with NO block-escape (the pre-existing root cause), so a leading
// `#`/`-`/`*`/`+`/`>`, an ordered-list `N.`/`N)`, a code fence ```/~~~, a table
// `|`, or a `> [!info]` callout opener would silently become a heading / list /
// quote / code block / table / callout. The final alternative matches a WHOLE-
// LINE thematic break — solid `---`/`***`/`___` or spaced `- - -`/`_ _ _` (3+ of
// the same `-`/`*`/`_`) — which round-trips into a `horizontalRule`; because
// that node carries NO text, an un-neutralized separator line would LOSE its
// text entirely (worse than the list/quote case). This matches a TRIMMED line's
// start; the transcript's own `You:` / `Speaker N:` prefix begins with a letter
// and never matches, so prefixed lines are left byte-exact.
const GITMOST_MD_BLOCK_TRIGGER_RE =
/^(?:#{1,6}(?:\s|$)|[-*+](?:\s|$)|>|\d+[.)](?:\s|$)|```|~~~|\||([-*_])(?:\s*\1){2,}\s*$)/;
// Append a transcript block BELOW the recording's audio node in a live editor:
// a "Transcript" heading followed by one paragraph per non-empty transcript
// line. The transcript is plain text, `\n`-separated, each line already
// formatted as `You: ...` / `Speaker N: ...` by the native host — line text is
// inserted as a TEXT node (never HTML/markdown), so there is no injection or
// mark-parsing surface. Each kept line is trimmed (drops an indent that would
// both leak into the display and, at col 0, form a markdown block trigger) and,
// if it still begins with a col-0 markdown block trigger, gets an invisible
// zero-width space prepended so the git-sync round-trip cannot turn it into a
// list/quote/heading/callout/code/table (defensive boundary against the
// serializer's missing block-escape). This is best-effort and meant to run
// AFTER the audio has already been inserted; the caller must guard against a
// throw so a transcript failure never fails the (already successful) recording.
// Returns true when a block was inserted, false when there was nothing to
// insert (transcript undefined/empty/not-a-string). A non-string value is a
// no-op, not an error.
// leak into the display). A line that begins with a col-0 markdown block
// trigger (`#`/`-`/`>`/`1.`/fence/`---`/…) needs no client-side workaround: the
// git-sync serializer (packages/prosemirror-markdown, `case "paragraph"`) now
// block-escapes such a leading trigger, so the doc->markdown->doc round-trip
// keeps the line a paragraph on its own — the former invisible-ZWSP defense is
// gone. This is best-effort and meant to run AFTER the audio has already been
// inserted; the caller must guard against a throw so a transcript failure never
// fails the (already successful) recording. Returns true when a block was
// inserted, false when there was nothing to insert (transcript
// undefined/empty/not-a-string). A non-string value is a no-op, not an error.
export function gitmostInsertTranscriptIntoEditor(
editor: Editor,
transcript: unknown,
@@ -288,13 +265,7 @@ export function gitmostInsertTranscriptIntoEditor(
.split("\n")
// Trim each line and drop blank (whitespace-only) ones.
.map((line) => line.trim())
.filter((line) => line.length > 0)
// Neutralize a col-0 markdown block trigger with an invisible ZWSP so the
// git-sync round-trip keeps the line a paragraph. Host lines (`You:` /
// `Speaker N:`) never match and stay byte-exact.
.map((line) =>
GITMOST_MD_BLOCK_TRIGGER_RE.test(line) ? GITMOST_ZWSP + line : line,
);
.filter((line) => line.length > 0);
if (lines.length === 0) return false;
const content = [
@@ -665,6 +665,13 @@ export function updateCacheOnMovePage(
pageData: Partial<IPage>,
) {
invalidatePageTree();
// Invalidate the moved page's breadcrumbs (#523). The tree-side child-loss
// guard removes the moved node from the local tree when its new parent is an
// unloaded branch, so `findBreadcrumbPath` misses it and the breadcrumb bar
// falls back to the server `["breadcrumbs", pageId]` query — which this move
// must invalidate, otherwise the crumbs keep showing the OLD parent until a
// refocus/navigation.
queryClient.invalidateQueries({ queryKey: ["breadcrumbs", pageId] });
// Remove page from old parent's cache
const oldQueryKey =
oldParentId === null
@@ -0,0 +1,40 @@
import { describe, it, expect, beforeEach, vi } from "vitest";
import type { IPage } from "@/features/page/types/page.types";
// A fresh QueryClient stands in for the app singleton (importing the real
// @/main.tsx would run ReactDOM.createRoot, which has no DOM root in jsdom).
vi.mock("@/main.tsx", async () => {
const { QueryClient } = await import("@tanstack/react-query");
return { queryClient: new QueryClient() };
});
import { queryClient } from "@/main.tsx";
import { updateCacheOnMovePage } from "./page-query";
// #523: the tree-side child-loss guard removes the moved node from the local
// tree when its new parent is an unloaded branch, so `findBreadcrumbPath` misses
// it and the breadcrumb bar falls back to the server `["breadcrumbs", pageId]`
// query. That query MUST be invalidated by a move, or the crumbs keep showing
// the OLD parent until a refocus/navigation.
describe("updateCacheOnMovePage — breadcrumbs invalidation (#523)", () => {
beforeEach(() => {
queryClient.clear();
vi.restoreAllMocks();
});
it("invalidates the moved page's ['breadcrumbs', pageId] query", () => {
const spy = vi.spyOn(queryClient, "invalidateQueries");
updateCacheOnMovePage("s1", "moved-page", "old-parent", "new-parent", {
id: "moved-page",
} as Partial<IPage>);
const invalidatedBreadcrumbs = spy.mock.calls.some(
([arg]) =>
Array.isArray((arg as { queryKey?: unknown[] })?.queryKey) &&
(arg as { queryKey: unknown[] }).queryKey[0] === "breadcrumbs" &&
(arg as { queryKey: unknown[] }).queryKey[1] === "moved-page",
);
expect(invalidatedBreadcrumbs).toBe(true);
});
});
@@ -55,7 +55,14 @@ type Props<T extends object> = {
};
const DRAG_TYPE = 'doc-tree-item';
const AUTO_EXPAND_MS = 500;
// Hover-hold before a collapsed row auto-expands during a drag. 2s (not ~0.5s)
// so merely dragging the cursor THROUGH the tree never expands rows — only a
// deliberate hold does (#523).
const AUTO_EXPAND_MS = 2000;
// How long the "a page just moved in here" cue stays on a collapsed target after
// a make-child drop. Long enough to notice at a glance during frequent
// collapsed-drops, short enough not to linger. Code-only UX constant (#523).
const DROP_LANDED_HIGHLIGHT_MS = 1800;
function DocTreeRowInner<T extends object>(props: Props<T>) {
const {
@@ -93,7 +100,11 @@ function DocTreeRowInner<T extends object>(props: Props<T>) {
const rowRef = useRef<HTMLElement>(null);
const [isDragging, setIsDragging] = useState(false);
const [instruction, setInstruction] = useState<Instruction | null>(null);
// Transient "just received a child" cue: a make-child drop no longer expands
// the (collapsed) target, so flash the row instead so the move isn't invisible.
const [landedChild, setLandedChild] = useState(false);
const autoExpandTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const landedChildTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const cancelAutoExpand = useCallback(() => {
if (autoExpandTimerRef.current) {
@@ -249,11 +260,24 @@ function DocTreeRowInner<T extends object>(props: Props<T>) {
? getDragLabel(sourceNode)
: 'item';
liveRegion.announce(`Moved ${sourceLabel} under ${parentName}.`);
// After a make-child drop, expand this row so the user sees the
// just-dropped child — especially important when the row had no
// children before (chevron just appeared) so the drop would
// otherwise be invisible.
if (op.kind === 'make-child') onToggle(node.id, true);
// Do NOT auto-expand the target on drop: a drop must leave the node
// collapsed. Intentional expansion is handled solely by the
// hover-hold timer (AUTO_EXPAND_MS). Feedback that the drop landed is
// given by the post-move flash + landed-cue highlight + live-region
// announce above. When the make-child target is collapsed, flash a
// distinct "child moved in here" cue on the row (it stays collapsed).
if (op.kind === 'make-child' && !isOpen) {
if (landedChildTimerRef.current) {
clearTimeout(landedChildTimerRef.current);
}
setLandedChild(true);
landedChildTimerRef.current = setTimeout(() => {
setLandedChild(false);
landedChildTimerRef.current = null;
}, DROP_LANDED_HIGHLIGHT_MS);
}
// Restore the openness of the MOVED page itself (source) — untouched
// by the above; the target is never expanded here.
if (source.data.isOpenOnDragStart) onToggle(sourceId, true);
},
}),
@@ -281,6 +305,17 @@ function DocTreeRowInner<T extends object>(props: Props<T>) {
useEffect(() => () => cancelAutoExpand(), [cancelAutoExpand]);
// Clear the landed-child cue timer on unmount (mirrors autoExpandTimerRef).
useEffect(
() => () => {
if (landedChildTimerRef.current) {
clearTimeout(landedChildTimerRef.current);
landedChildTimerRef.current = null;
}
},
[],
);
const effectiveInst =
instruction?.type === 'instruction-blocked'
? instruction.desired
@@ -317,6 +352,7 @@ function DocTreeRowInner<T extends object>(props: Props<T>) {
className={styles.node}
data-dragging={isDragging || undefined}
data-selected={isSelected || undefined}
data-landed-child={landedChild || undefined}
data-receiving-drop={
receivingDrop === 'make-child'
? blocked
@@ -281,10 +281,12 @@ const SpaceTree = forwardRef<SpaceTreeApi, SpaceTreeProps>(function SpaceTree(
setOpenTreeNodes((prev) => ({ ...prev, [id]: isOpen }));
if (isOpen) {
const node = treeModel.find(data, id) as SpaceTreeNode | null;
if (
node?.hasChildren &&
(!node.children || node.children.length === 0)
) {
// Same "unloaded branch" predicate the realtime insert paths use
// (`isUnloadedBranch`) so the lazy-load gate and the realtime inserts
// (`insertByPosition` / `placeByPosition`) can never disagree about what
// counts as unloaded (#525). Note: local raw `insert` (DnD/create-page)
// does not yet route through it — see #525 follow-up.
if (treeModel.isUnloadedBranch(node)) {
const fetched = await fetchAllAncestorChildren({
pageId: id,
spaceId: node.spaceId,
@@ -0,0 +1,149 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { renderHook, act } from "@testing-library/react";
import { Provider, createStore } from "jotai";
import type { ReactNode } from "react";
import { treeDataAtom } from "@/features/page/tree/atoms/tree-data-atom.ts";
import { treeModel } from "@/features/page/tree/model/tree-model";
import type { SpaceTreeNode } from "@/features/page/tree/types";
// --- Boundary mocks: only the network/query + router/i18n surfaces the hook
// touches. The tree math (treeModel, dropOpToMovePayload) runs for real so the
// child-loss guard is exercised end-to-end.
const moveMutate = vi.fn().mockResolvedValue({});
const updateCacheOnMovePageMock = vi.fn();
vi.mock("@/features/page/queries/page-query.ts", () => ({
useCreatePageMutation: () => ({ mutateAsync: vi.fn() }),
useUpdatePageMutation: () => ({ mutateAsync: vi.fn() }),
useRemovePageMutation: () => ({ mutateAsync: vi.fn() }),
useMovePageMutation: () => ({ mutateAsync: moveMutate }),
updateCacheOnMovePage: (...args: unknown[]) =>
updateCacheOnMovePageMock(...args),
}));
vi.mock("react-router-dom", () => ({
useNavigate: () => vi.fn(),
useParams: () => ({ spaceSlug: "space", pageSlug: undefined }),
}));
vi.mock("react-i18next", () => ({
useTranslation: () => ({ t: (s: string) => s }),
}));
vi.mock("@mantine/notifications", () => ({
notifications: { show: vi.fn() },
}));
// Import AFTER mocks so the hook binds to them.
import { useTreeMutation } from "./use-tree-mutation";
function node(
id: string,
over: Partial<SpaceTreeNode> = {},
): SpaceTreeNode {
return {
id,
slugId: `slug-${id}`,
name: id.toUpperCase(),
position: "a0",
spaceId: "space-1",
parentPageId: null as unknown as string,
hasChildren: false,
children: [],
...over,
};
}
function setup(before: SpaceTreeNode[]) {
const store = createStore();
store.set(treeDataAtom, before);
const wrapper = ({ children }: { children: ReactNode }) => (
<Provider store={store}>{children}</Provider>
);
const { result } = renderHook(() => useTreeMutation("space-1"), { wrapper });
return { store, result };
}
describe("useTreeMutation.handleMove — child-loss guard (#523)", () => {
beforeEach(() => {
vi.clearAllMocks();
moveMutate.mockResolvedValue({});
});
it("make-child into an UNLOADED folder does NOT materialize [source] — keeps it lazy-loadable", async () => {
// F has children on the server but none are loaded here (canonical unloaded
// form: hasChildren + children:[]). X sits at root.
const before = [
node("F", { position: "a0", hasChildren: true, children: [] }),
node("X", { position: "a5" }),
];
const { store, result } = setup(before);
await act(async () => {
await result.current.handleMove("X", {
kind: "make-child",
targetId: "F",
});
});
const tree = store.get(treeDataAtom);
const f = treeModel.find(tree, "F");
// The guard leaves F unloaded (children stay []), so a later expand fetches
// the FULL server set (incl. X) instead of showing a misleading partial [X].
// MUTATION: dropping the guard (using the `move` result) would put children
// === [X] here and redden this.
expect(f?.children).toEqual([]);
expect(f?.hasChildren).toBe(true);
// X is removed from its old (root) slot; it reappears on expand/load of F.
expect(treeModel.find(tree, "X")).toBeNull();
// The server move is still persisted.
expect(moveMutate).toHaveBeenCalledTimes(1);
});
it("make-child into a LOADED folder appends source and KEEPS the existing children", async () => {
// F is loaded with one child c1; moving X in must preserve c1 (no loss) and
// append X — this path does NOT hit the guard.
const before = [
node("F", {
position: "a0",
hasChildren: true,
children: [node("c1", { position: "a1", parentPageId: "F" })],
}),
node("X", { position: "a5" }),
];
const { store, result } = setup(before);
await act(async () => {
await result.current.handleMove("X", {
kind: "make-child",
targetId: "F",
});
});
const tree = store.get(treeDataAtom);
const f = treeModel.find(tree, "F");
expect(f?.children?.map((n) => n.id)).toEqual(["c1", "X"]);
expect(f?.hasChildren).toBe(true);
// X now lives under F.
expect(treeModel.find(tree, "X")?.parentPageId).toBe("F");
});
it("make-child into a genuinely-empty leaf materializes the child (nothing to lose)", async () => {
// Leaf L has no server children (hasChildren:false). Dropping X in should
// show X immediately — the guard must NOT fire here.
const before = [
node("L", { position: "a0", hasChildren: false, children: [] }),
node("X", { position: "a5" }),
];
const { store, result } = setup(before);
await act(async () => {
await result.current.handleMove("X", {
kind: "make-child",
targetId: "L",
});
});
const tree = store.get(treeDataAtom);
const l = treeModel.find(tree, "L");
expect(l?.children?.map((n) => n.id)).toEqual(["X"]);
expect(l?.hasChildren).toBe(true);
});
});
@@ -61,11 +61,48 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
if (!source) return;
const oldParentId = source.parentPageId ?? null;
// optimistic apply with the new position from the payload
let optimistic = treeModel.update(after, sourceId, {
position: payload.position,
parentPageId: payload.parentPageId,
} as Partial<SpaceTreeNode>);
// Child-loss guard (#523, twin of #525's realtime `insertByPosition` fix).
// We no longer auto-expand the make-child target on drop, so the old
// `onToggle(target, true)` — which was ALSO the only trigger of the
// corrective lazy-load — is gone. `treeModel.move` materialized
// `target.children = [source]` (only the moved node); if the target is an
// UNLOADED branch (server has children but none are loaded here), keeping
// that partial `[source]` list would defeat the lazy-load gate and hide the
// target's OTHER server children (the #159 #1 data-loss class). So for an
// unloaded make-child target, build the optimistic tree WITHOUT
// materializing source under it: just remove source from its old parent and
// flag the target `hasChildren`. The gate stays armed and a later manual
// expand fetches the FULL set (incl. the moved page, which the awaited
// server move persists). Predicate is the gate's (`isUnloadedBranch`), NOT
// `insertByPosition`'s old `=== undefined` (canonical unloaded is `[]`).
const target =
op.kind === "make-child"
? (treeModel.find(before, op.targetId) as SpaceTreeNode | null)
: null;
const unloadedMakeChild =
op.kind === "make-child" && treeModel.isUnloadedBranch(target);
let optimistic: SpaceTreeNode[];
if (unloadedMakeChild) {
// Do NOT materialize [source] into the unloaded target.
optimistic = treeModel.remove(before, sourceId);
optimistic = treeModel.update(optimistic, op.targetId, {
hasChildren: true,
} as Partial<SpaceTreeNode>);
} else {
// optimistic apply with the new position from the payload
optimistic = treeModel.update(after, sourceId, {
position: payload.position,
parentPageId: payload.parentPageId,
} as Partial<SpaceTreeNode>);
// For make-child onto a previously-childless (loaded) target: flip
// hasChildren on so the new parent shows its chevron.
if (op.kind === "make-child") {
optimistic = treeModel.update(optimistic, op.targetId, {
hasChildren: true,
} as Partial<SpaceTreeNode>);
}
}
// If the old parent has no children left, mark hasChildren: false so the
// chevron disappears. Without this, the empty parent keeps rendering an
@@ -79,14 +116,6 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
}
}
// For make-child onto a previously-childless target: flip hasChildren on
// so the new parent shows its chevron.
if (op.kind === "make-child") {
optimistic = treeModel.update(optimistic, op.targetId, {
hasChildren: true,
} as Partial<SpaceTreeNode>);
}
setData(optimistic);
try {
@@ -74,6 +74,48 @@ describe("treeModel.isDescendant", () => {
});
});
// #525: the single "is this branch unloaded?" predicate shared by the lazy-load
// gate and the insert paths. Unloaded == server says hasChildren but none are
// present locally (canonical form `children: []`, also `undefined`). A parent
// without hasChildren is genuinely empty, not unloaded.
describe("treeModel.isUnloadedBranch", () => {
type PH = TreeNode<{ name: string; hasChildren?: boolean }>;
it("true for hasChildren + empty array (canonical unloaded form)", () => {
const n: PH = { id: "p", name: "P", hasChildren: true, children: [] };
expect(treeModel.isUnloadedBranch(n)).toBe(true);
});
it("true for hasChildren + undefined children", () => {
const n: PH = { id: "p", name: "P", hasChildren: true };
expect(treeModel.isUnloadedBranch(n)).toBe(true);
});
it("false for hasChildren + already-loaded children", () => {
const n: PH = {
id: "p",
name: "P",
hasChildren: true,
children: [{ id: "c", name: "C" }],
};
expect(treeModel.isUnloadedBranch(n)).toBe(false);
});
it("false for a genuinely-empty parent (no hasChildren)", () => {
expect(
treeModel.isUnloadedBranch({
id: "p",
name: "P",
hasChildren: false,
children: [],
} as PH),
).toBe(false);
expect(
treeModel.isUnloadedBranch({ id: "p", name: "P" } as PH),
).toBe(false);
});
it("false for null/undefined", () => {
expect(treeModel.isUnloadedBranch(null)).toBe(false);
expect(treeModel.isUnloadedBranch(undefined)).toBe(false);
});
});
describe("treeModel.visible", () => {
it("returns only root nodes when no openIds", () => {
const v = treeModel.visible(fixture, new Set());
@@ -197,43 +239,64 @@ describe("treeModel.insertByPosition", () => {
]);
});
// #159 #1: inserting/moving a node under a parent whose children are NOT
// loaded (`children === undefined`, e.g. a collapsed page) must NOT materialize
// a partial `[node]` list — that would defeat the lazy-load gate and hide the
// parent's other real children. The node is left to be lazy-loaded; only
// `hasChildren` is flagged so the chevron appears.
it("does NOT materialize a child under an UNLOADED parent (children undefined)", () => {
type PH = TreeNode<{
name: string;
position?: string;
hasChildren?: boolean;
}>;
type PH = TreeNode<{
name: string;
position?: string;
hasChildren?: boolean;
}>;
// #159 #1 / #525: inserting/moving a node under an UNLOADED parent must NOT
// materialize a partial `[node]` list — that would defeat the lazy-load gate and
// hide the parent's other real children. The canonical unloaded form here is
// `children: []` + `hasChildren: true` (from `pageToTreeNode` /
// `pruneCollapsedChildren`), which the pre-#525 `=== undefined` guard MISSED.
// The node is left to be lazy-loaded; the chevron stays enabled.
it("does NOT materialize a child under an UNLOADED parent (children: [], hasChildren: true)", () => {
const tree: PH[] = [
{ id: "p", name: "P", position: "a0", hasChildren: false }, // children: undefined
{ id: "p", name: "P", position: "a0", hasChildren: true, children: [] },
];
const node: PH = { id: "x", name: "X", position: "a1" };
const t = treeModel.insertByPosition(tree, "p", node);
const parent = treeModel.find(t, "p");
// The node was NOT inserted (children stay unloaded -> lazy-load fetches the
// full set, including this node, on expand).
expect(parent?.children).toBeUndefined();
// full set, including this node, on expand). MUTATION: the pre-#525 predicate
// `children === undefined` does not fire for `[]`, so it would insert `[x]`
// here and reredden this expectation.
expect(parent?.children).toEqual([]);
expect(treeModel.find(t, "x")).toBeNull();
// ...but the chevron is enabled so the user can expand to load it.
// ...and the chevron stays enabled so the user can expand to load it.
expect((parent as PH).hasChildren).toBe(true);
});
it("DOES insert under a LOADED-but-empty parent (children: [])", () => {
type PH = TreeNode<{
name: string;
position?: string;
hasChildren?: boolean;
}>;
it("does NOT materialize a child under an UNLOADED parent (children undefined, hasChildren: true)", () => {
const tree: PH[] = [
{ id: "p", name: "P", position: "a0", hasChildren: true }, // children: undefined
];
const node: PH = { id: "x", name: "X", position: "a1" };
const t = treeModel.insertByPosition(tree, "p", node);
const parent = treeModel.find(t, "p");
expect(parent?.children).toBeUndefined();
expect(treeModel.find(t, "x")).toBeNull();
expect((parent as PH).hasChildren).toBe(true);
});
it("DOES insert under a genuinely-empty parent (children: [], hasChildren: false)", () => {
const tree: PH[] = [
{ id: "p", name: "P", position: "a0", hasChildren: false, children: [] },
];
const node: PH = { id: "x", name: "X", position: "a1" };
const t = treeModel.insertByPosition(tree, "p", node);
// A loaded (empty) child list is complete, so the node IS inserted.
// No server children (`hasChildren: false`), so materializing the first child
// is correct — nothing is hidden.
expect(treeModel.find(t, "p")?.children?.map((n) => n.id)).toEqual(["x"]);
});
it("DOES insert under a genuinely-empty parent (children undefined, hasChildren: false)", () => {
const tree: PH[] = [
{ id: "p", name: "P", position: "a0", hasChildren: false }, // children: undefined
];
const node: PH = { id: "x", name: "X", position: "a1" };
const t = treeModel.insertByPosition(tree, "p", node);
expect(treeModel.find(t, "p")?.children?.map((n) => n.id)).toEqual(["x"]);
});
@@ -43,6 +43,26 @@ export const treeModel = {
};
},
// A branch is "unloaded" when the server says it HAS children (`hasChildren`)
// but none are present locally. The canonical unloaded form in this codebase
// is `children: []` (produced by `pageToTreeNode` and by `pruneCollapsedChildren`
// resetting collapsed branches), NOT `children: undefined` — so a predicate that
// only checks `=== undefined` misses the real case and materializes a misleading
// partial list (#525). This is the SINGLE source of truth for "should a
// fetch/materialize be deferred?", shared by the lazy-load gate (`handleToggle`)
// and the realtime insert paths (`insertByPosition` / `placeByPosition`), so they
// can never drift apart again. (The local raw `insert` primitive and its DnD/
// create-page callers do NOT yet route through this predicate — see #525
// follow-up.) A parent WITHOUT `hasChildren` is genuinely empty
// (no server children) — inserting its first child is correct, not deferred.
isUnloadedBranch<T extends object>(
node: TreeNode<T> | null | undefined,
): boolean {
if (!node) return false;
const hasChildren = (node as { hasChildren?: boolean }).hasChildren === true;
return hasChildren && (node.children == null || node.children.length === 0);
},
isDescendant<T extends object>(
tree: TreeNode<T>[],
ancestorId: string,
@@ -127,14 +147,15 @@ export const treeModel = {
}
const parent = treeModel.find(tree, parentId);
// The parent is in the tree but its children have NOT been lazy-loaded yet
// (`children === undefined`, distinct from a loaded-but-empty `[]`). Inserting
// (`hasChildren` set + children absent/empty — see `isUnloadedBranch`; the
// canonical unloaded form is `children: []`, NOT just `undefined`). Inserting
// here would MATERIALIZE a misleading partial child list (`[node]`) that
// defeats the lazy-load gate — which fetches only when children are
// absent/empty — so the parent's OTHER real children would never load and the
// moved/added node would be the only one shown (a silent data loss, #159 #1).
// Instead, leave the children unloaded and just flag `hasChildren` so the
// chevron appears; expanding fetches the FULL set (including this node).
if (parent && parent.children === undefined) {
if (parent && treeModel.isUnloadedBranch(parent)) {
return treeModel.update(
tree,
parentId,
@@ -150,6 +150,45 @@
);
}
/* "A page just moved in here" cue (#523). A make-child drop no longer expands
the collapsed target, so the moved page is momentarily invisible; pulse the
target row in a distinct teal (NOT the blue make-child highlight, NOT the
neutral post-move flash) so the landing is noticeable while the node stays
collapsed. Two short pulses fit inside DROP_LANDED_HIGHLIGHT_MS (1.8s), after
which the row clears the attribute and the animation stops. */
@keyframes landedChildPulse {
0% {
background-color: light-dark(
var(--mantine-color-teal-2),
rgba(45, 212, 191, 0.30)
);
outline-color: light-dark(
var(--mantine-color-teal-6),
var(--mantine-color-teal-5)
);
}
100% {
background-color: transparent;
outline-color: transparent;
}
}
.node[data-landed-child="true"] {
outline: 2px solid transparent;
outline-offset: -1px;
animation: landedChildPulse 0.9s ease-out 2;
}
@media (prefers-reduced-motion: reduce) {
.node[data-landed-child="true"] {
animation: none;
background-color: light-dark(
var(--mantine-color-teal-1),
rgba(45, 212, 191, 0.18)
);
}
}
.dropLine {
position: absolute;
left: var(--drop-line-indent, 0);
@@ -13,8 +13,7 @@ let currentAlias: IShareAlias | null = null;
let availabilityResult: {
valid: boolean;
available: boolean;
currentPageId: string | null;
} = { valid: true, available: true, currentPageId: null };
} = { valid: true, available: true };
vi.mock("@/features/share/queries/share-query.ts", () => ({
useShareAliasForPageQuery: () => ({ data: currentAlias }),
@@ -56,7 +55,7 @@ describe("ShareAliasSection — taken-name handling is never a dead end", () =>
beforeEach(() => {
setMutateAsync.mockReset();
currentAlias = null;
availabilityResult = { valid: true, available: true, currentPageId: null };
availabilityResult = { valid: true, available: true };
});
it("shows a 'will move it here' HINT (not a terminal error) when the name belongs to another page, and keeps Save enabled", async () => {
@@ -65,7 +64,6 @@ describe("ShareAliasSection — taken-name handling is never a dead end", () =>
availabilityResult = {
valid: true,
available: false,
currentPageId: "page-X",
};
renderSection("page-Y");
@@ -97,7 +95,6 @@ describe("ShareAliasSection — taken-name handling is never a dead end", () =>
availabilityResult = {
valid: true,
available: false,
currentPageId: "page-X",
};
// The server rejects the un-confirmed save asking the client to confirm.
setMutateAsync.mockRejectedValueOnce({
@@ -106,7 +103,6 @@ describe("ShareAliasSection — taken-name handling is never a dead end", () =>
status: 409,
data: {
code: "ALIAS_REASSIGN_REQUIRED",
currentPageId: "page-X",
currentPageTitle: "Alias Test Page X",
},
},
@@ -48,7 +48,6 @@ export default function ShareAliasSection({
const [availability, setAvailability] = useState<{
valid: boolean;
available: boolean;
currentPageId: string | null;
} | null>(null);
const [reassign, setReassign] = useState<{
alias: string;
@@ -76,7 +75,6 @@ export default function ShareAliasSection({
setAvailability({
valid: res.valid,
available: res.available,
currentPageId: res.currentPageId,
});
} catch {
setAvailability(null);
@@ -108,7 +108,6 @@ export interface IShareAliasAvailability {
alias: string;
valid: boolean;
available: boolean;
currentPageId: string | null;
}
export interface ISharedPageTree {
@@ -82,17 +82,19 @@ describe("applyMoveTreeNode", () => {
]);
});
it("does NOT create a partial child list when the destination is loaded-but-collapsed (children unloaded) — keeps it lazy-loadable (#159)", () => {
// `dstCollapsed` is in the tree but its children were never lazy-loaded
// (children === undefined). The OLD behavior inserted `src` as the ONLY
// child ([src]), which defeated the lazy-load gate and HID the parent's
// other real children. Now the move leaves children unloaded (so expanding
// fetches the FULL set, including src) and just flags hasChildren.
it("does NOT create a partial child list when the destination is loaded-but-collapsed (children unloaded) — keeps it lazy-loadable (#159 #525)", () => {
// `dstCollapsed` is in the tree but its children were never lazy-loaded. The
// CANONICAL unloaded form here is `hasChildren: true` + `children: []` (from
// `pageToTreeNode` / `pruneCollapsedChildren`), NOT `children: undefined`.
// The pre-#525 predicate (`children === undefined`) missed this form and
// inserted `src` as the ONLY child ([src]), defeating the lazy-load gate and
// HIDING the parent's other real children. Now the move leaves children
// unloaded (so expanding fetches the FULL set, including src).
const tree: SpaceTreeNode[] = [
node("dstCollapsed", {
position: "a0",
hasChildren: false,
children: undefined as unknown as SpaceTreeNode[],
hasChildren: true,
children: [],
}),
node("src", { position: "a9" }),
];
@@ -105,9 +107,10 @@ describe("applyMoveTreeNode", () => {
pageData: {},
});
const dst = treeModel.find(next, "dstCollapsed");
// Children stay unloaded -> the lazy-load gate fetches the FULL set (incl.
// src) on expand, rather than showing a misleading partial [src] list.
expect(dst?.children).toBeUndefined();
// Children stay unloaded ([] not materialized to [src]) -> the lazy-load gate
// fetches the FULL set (incl. src) on expand. MUTATION: the pre-#525
// `=== undefined` predicate would insert [src] here and redden this.
expect(dst?.children).toEqual([]);
expect(dst?.hasChildren).toBe(true);
// src moved away from its old root slot (it lives under dstCollapsed
// server-side and reappears when the parent is expanded/loaded).
@@ -28,6 +28,7 @@ import {
IAiMcpServerCreate,
IAiMcpServerUpdate,
} from "@/features/workspace/services/ai-mcp-server-service.ts";
import { resolveToolAllowlist } from "./ai-mcp-server-form.utils.ts";
const formSchema = z.object({
name: z.string().min(1),
@@ -121,13 +122,20 @@ export default function AiMcpServerForm({
async function handleSubmit(values: FormValues) {
const headers = resolveHeaders();
// An empty tag field means "no restriction" (sent as null) — since #476 the
// server persists a literal `[]` as deny-all (zero tools). But a server that
// was ALREADY deny-all loads into an empty field too; sending null there
// would silently widen it to allow-all on a routine edit, so preserve `[]`.
// See resolveToolAllowlist for the full rationale.
const toolAllowlist = resolveToolAllowlist(values.toolAllowlist, server);
if (isEdit && server) {
const payload: IAiMcpServerUpdate = {
id: server.id,
name: values.name,
transport: values.transport,
url: values.url,
toolAllowlist: values.toolAllowlist,
toolAllowlist,
// Always sent: a blank value clears the stored guidance (server -> null).
instructions: values.instructions,
enabled: values.enabled,
@@ -140,7 +148,7 @@ export default function AiMcpServerForm({
name: values.name,
transport: values.transport,
url: values.url,
toolAllowlist: values.toolAllowlist,
toolAllowlist,
// Blank => server stores null (no guidance).
instructions: values.instructions,
enabled: values.enabled,
@@ -0,0 +1,29 @@
import { describe, it, expect } from "vitest";
import { resolveToolAllowlist } from "./ai-mcp-server-form.utils.ts";
describe("resolveToolAllowlist", () => {
it("sends the typed tools when the field is non-empty", () => {
expect(resolveToolAllowlist(["a", "b"], { toolAllowlist: null })).toEqual([
"a",
"b",
]);
});
it("creates as null (unrestricted) when empty and there is no server", () => {
expect(resolveToolAllowlist([], undefined)).toBeNull();
});
it("sends null for an empty field on a previously-unrestricted server", () => {
expect(resolveToolAllowlist([], { toolAllowlist: null })).toBeNull();
});
it("preserves deny-all: an empty field on a `[]` server stays `[]`, not null", () => {
// The core #476/#477 guard: editing a deny-all server (rename/toggle) with
// an empty tag field must NOT silently widen it to allow-all.
expect(resolveToolAllowlist([], { toolAllowlist: [] })).toEqual([]);
});
it("still sends explicit tools even if the server was deny-all", () => {
expect(resolveToolAllowlist(["x"], { toolAllowlist: [] })).toEqual(["x"]);
});
});
@@ -0,0 +1,22 @@
import { IAiMcpServer } from "@/features/workspace/services/ai-mcp-server-service.ts";
// Resolve the tool allowlist value to persist from the form field.
//
// An empty tag field normally means "no restriction" and is sent as null so
// the server drops the column (all tools allowed). But a server that was
// ALREADY deny-all (a stored literal `[]`, meaning zero tools — creatable via
// the API) loads into the form as an empty field too. Coercing that empty
// field to null on submit would SILENTLY widen a deny-all server to allow-all
// on any routine edit (rename, toggle) — the exact silent-widen class #476
// closed on the read side. So when the edited server was deny-all, preserve
// `[]` (deny-all); only a genuinely-unrestricted server (stored null/absent)
// stays null.
export function resolveToolAllowlist(
fieldValue: string[],
server?: Pick<IAiMcpServer, "toolAllowlist">,
): string[] | null {
if (fieldValue.length > 0) return fieldValue;
const wasDenyAll =
Array.isArray(server?.toolAllowlist) && server.toolAllowlist.length === 0;
return wasDenyAll ? [] : null;
}
@@ -6,6 +6,8 @@ import {
nextReindexPollInterval,
isReindexComplete,
isReindexButtonLoading,
reindexRunKey,
isNewReindexRun,
} from './ai-provider-settings';
describe('resolveCardStatus', () => {
@@ -221,6 +223,128 @@ describe('isReindexComplete', () => {
});
});
describe('reindexRunKey', () => {
it('is null when the status carries no run identity', () => {
expect(reindexRunKey(undefined)).toBeNull();
expect(
reindexRunKey({ reindexing: false, indexedPages: 5, totalPages: 5 }),
).toBeNull();
});
it('is null for a legacy/degraded record with an empty runId', () => {
// The server sends runId='' for a record written before the field existed;
// the client must treat that as "no identity" (fall back to prior behaviour).
expect(
reindexRunKey({
reindexing: true,
indexedPages: 0,
totalPages: 10,
runId: '',
reindexStartedAt: 1000,
}),
).toBeNull();
});
it('folds runId and startedAt into one stable key', () => {
expect(
reindexRunKey({
reindexing: true,
indexedPages: 0,
totalPages: 10,
runId: 'run-a',
reindexStartedAt: 1000,
}),
).toBe('run-a:1000');
});
it('changes when the runId changes for the same startedAt', () => {
const a = reindexRunKey({
reindexing: true,
indexedPages: 0,
totalPages: 10,
runId: 'run-a',
reindexStartedAt: 1000,
});
const b = reindexRunKey({
reindexing: true,
indexedPages: 0,
totalPages: 10,
runId: 'run-b',
reindexStartedAt: 1000,
});
expect(a).not.toBe(b);
});
it('changes when the same runId restarts at a new startedAt', () => {
const a = reindexRunKey({
reindexing: true,
indexedPages: 0,
totalPages: 10,
runId: 'run-a',
reindexStartedAt: 1000,
});
const b = reindexRunKey({
reindexing: true,
indexedPages: 0,
totalPages: 10,
runId: 'run-a',
reindexStartedAt: 2000,
});
expect(a).not.toBe(b);
});
});
describe('isNewReindexRun (poll keying on runId)', () => {
// Derive the status shape from the helper itself so the test needs no export
// of the component-internal ReindexStatus type.
type ReindexStatusLike = NonNullable<Parameters<typeof reindexRunKey>[0]>;
const run = (runId: string, startedAt: number): ReindexStatusLike => ({
reindexing: true,
indexedPages: 0,
totalPages: 10,
runId,
reindexStartedAt: startedAt,
});
it('first identity after none latched is a NEW run', () => {
expect(isNewReindexRun(null, run('run-a', 1000))).toBe(true);
});
it('the SAME identity is not a new run (same run being watched)', () => {
const key = reindexRunKey(run('run-a', 1000));
expect(isNewReindexRun(key, run('run-a', 1000))).toBe(false);
});
it('a DIFFERENT runId is a new run (reset per-run poll state)', () => {
const key = reindexRunKey(run('run-a', 1000));
expect(isNewReindexRun(key, run('run-b', 1000))).toBe(true);
});
it('an identity-less poll (no runId / cleared record) is never a new run', () => {
const key = reindexRunKey(run('run-a', 1000));
expect(
isNewReindexRun(key, {
reindexing: false,
indexedPages: 10,
totalPages: 10,
}),
).toBe(false);
});
it('a legacy empty-runId poll does not spuriously reset a latched run', () => {
const key = reindexRunKey(run('run-a', 1000));
expect(
isNewReindexRun(key, {
reindexing: true,
indexedPages: 3,
totalPages: 10,
runId: '',
reindexStartedAt: 1000,
}),
).toBe(false);
});
});
describe('isReindexButtonLoading', () => {
it('loads while the POST mutation is pending', () => {
expect(
@@ -173,9 +173,43 @@ export function resolveKeyField(
// Subset of the status payload that drives the reindex poll decisions.
type ReindexStatus = Pick<
IAiSettings,
"reindexing" | "indexedPages" | "totalPages"
"reindexing" | "indexedPages" | "totalPages" | "runId" | "reindexStartedAt"
>;
/**
* A stable per-RUN key for the reindex poll: `runId:startedAt`, or `null` when
* the status carries no run identity (no active run, or a legacy/degraded
* server record with an empty runId). Two polls of the SAME run share a key; a
* new run mints a fresh runId and so a different key.
*
* This is the single place the client turns the server's run identity into the
* value it keys on it removes the "is this the same run I've been watching or
* a brand-new one?" ambiguity that made a class of reindex-status bugs (a stale
* pre-reindex snapshot vs a fresh run) get fixed twice (#262). `startedAt` is
* folded in so a run that somehow reuses a runId but restarted is still new.
*/
export function reindexRunKey(status: ReindexStatus | undefined): string | null {
const runId = status?.runId;
if (!runId) return null;
return `${runId}:${status?.reindexStartedAt ?? ""}`;
}
/**
* Decide whether the latest poll represents a NEW reindex run relative to the
* run key the client last latched (`prevKey`, `null` if none yet). True only
* when the status carries an identity AND it differs from the latched one the
* signal to reset any per-run poll state (the "seen active" latch / progress the
* UI held). The same identity (or no identity) is NOT a new run, so an unchanged
* or identity-less poll never resets mid-run.
*/
export function isNewReindexRun(
prevKey: string | null,
status: ReindexStatus | undefined,
): boolean {
const key = reindexRunKey(status);
return key !== null && key !== prevKey;
}
/**
* Decide the TanStack Query `refetchInterval` while a reindex may be running.
* Returns the poll interval (ms) to keep polling, or `false` to stop.
@@ -320,6 +354,13 @@ export default function AiProviderSettings() {
// counter at 0 until a manual reload. A ref (not state) because it must not
// trigger a render and is only ever read where `reindexing` is already false.
const reindexSeenActiveRef = useRef(false);
// The run identity (runId:startedAt) the current poll window is keyed on. When
// a poll reports a DIFFERENT runId the server has started a NEW run, so we
// re-latch to it and reset `reindexSeenActiveRef` — a fresh run must never
// inherit the previous run's "seen active"/completion state (which would stop
// polling immediately or read the old run's counters as this run's). null =
// no run keyed yet (steady state, or a legacy record without a runId).
const reindexRunKeyRef = useRef<string | null>(null);
// Only admins may read the (masked) AI settings; the server enforces this too.
const { data: settings, isLoading } = useAiSettingsQuery(isAdmin, (query) =>
@@ -336,6 +377,14 @@ export default function AiProviderSettings() {
// unmount because the deadline state goes away with the component.
useEffect(() => {
if (reindexDeadline === null) return;
// Key the poll on the run identity: if this poll carries a runId different
// from the one we latched, the server started a NEW run, so adopt it and
// drop the per-run "seen active" latch (a fresh run must not inherit the
// previous run's completion state). Same runId => same run, leave it alone.
if (isNewReindexRun(reindexRunKeyRef.current, settings)) {
reindexRunKeyRef.current = reindexRunKey(settings);
reindexSeenActiveRef.current = false;
}
// Latch "we have seen the active run" the moment a poll reports it, so the
// completion check below (and the refetchInterval's) only fires once the run
// has genuinely started — never on the stale pre-reindex snapshot.
@@ -1220,6 +1269,10 @@ export default function AiProviderSettings() {
// immediately.
onSuccess: () => {
reindexSeenActiveRef.current = false;
// Forget the previous run's identity so the first poll of
// this window (carrying the new run's runId) is recognized
// as a new run and keyed afresh.
reindexRunKeyRef.current = null;
setReindexDeadline(Date.now() + REINDEX_POLL_CAP_MS);
},
})
@@ -27,7 +27,9 @@ export interface IAiMcpServerCreate {
// Auth headers map (e.g. { Authorization: 'Bearer ...' }). Encrypted on save;
// never returned.
headers?: Record<string, string>;
toolAllowlist?: string[];
// Omit/null => no restriction; `[]` is persisted verbatim and means
// deny-all (zero tools) since #476.
toolAllowlist?: string[] | null;
// Admin-authored prompt guidance (#180). Blank => stored as null.
instructions?: string;
enabled?: boolean;
@@ -43,7 +45,9 @@ export interface IAiMcpServerUpdate {
transport?: McpTransport;
url?: string;
headers?: Record<string, string>;
toolAllowlist?: string[];
// Absent => unchanged; null => no restriction; `[]` is persisted verbatim
// and means deny-all (zero tools) since #476.
toolAllowlist?: string[] | null;
// Admin-authored prompt guidance (#180). Absent => unchanged; blank => cleared.
instructions?: string;
enabled?: boolean;
@@ -51,6 +51,14 @@ export interface IAiSettings {
// True while a full workspace reindex is actively running; the counts above
// then reflect the live run progress (done climbs 0 -> total).
reindexing?: boolean;
// Identity of the ACTIVE reindex run (present only while `reindexing`). The
// poll keys on `runId`: a changed value means a NEW run (reset the per-run
// poll state the UI latched), the same value is the run already being watched.
// Absent/empty ('') => no identity available; the client keeps prior behaviour.
runId?: string;
// Epoch-ms the active run started; paired with `runId` so a restart with a
// recycled id is still detected as a new run.
reindexStartedAt?: number;
}
// Update payload. Key semantics (same for `apiKey` and `embeddingApiKey`):
@@ -1,5 +1,5 @@
import { describe, it, expect } from "vitest";
import { templateRoute } from "./route-template";
import { templateRoute, KNOWN_ROUTE_TEMPLATES } from "./route-template";
describe("templateRoute", () => {
it("templates a space page path (never leaks slugs)", () => {
@@ -32,4 +32,30 @@ describe("templateRoute", () => {
expect(templateRoute("/weird/unknown/thing")).toBe("other");
expect(templateRoute("/s/team/p/slug/extra/segments")).toBe("other");
});
// The server's /api/telemetry/vitals mirror (ALLOWED_ROUTE_TEMPLATES) drops any
// route outside KNOWN_ROUTE_TEMPLATES, so templateRoute must NEVER emit a label
// that is not in that dictionary — otherwise legit client metrics get dropped.
it("only ever emits labels contained in KNOWN_ROUTE_TEMPLATES (#495)", () => {
const samples = [
"/",
"/home",
"/settings/members",
"/settings/groups/g-1",
"/s/team",
"/s/team/trash",
"/s/team/p/slug",
"/p/slug",
"/share/abc",
"/share/abc/p/slug",
"/share/p/slug",
"/labels/urgent",
"/invites/inv-1",
"/weird/unknown/thing", // -> "other"
"/deep/unmatched/x/y/z", // -> "other"
];
for (const path of samples) {
expect(KNOWN_ROUTE_TEMPLATES.has(templateRoute(path))).toBe(true);
}
});
});
@@ -44,6 +44,22 @@ const STATIC_ROUTES = new Set<string>([
'/settings/sharing',
]);
/**
* The COMPLETE, finite vocabulary `templateRoute` can ever emit: the two
* synthetic labels (`/` and `other`), the static routes, and the dynamic
* templates. Exported so the public `/api/telemetry/vitals` endpoint can reject
* any `route` outside this dictionary server-side (the endpoint is anonymous, so
* an un-checked `route` is a free-text write surface). The server keeps a mirror
* (`ALLOWED_ROUTE_TEMPLATES` in client-metrics.constants.ts) this is the
* canonical source; keep them in lockstep.
*/
export const KNOWN_ROUTE_TEMPLATES: ReadonlySet<string> = new Set<string>([
'/',
'other',
...STATIC_ROUTES,
...ROUTE_PATTERNS.map((p) => p.template),
]);
export function templateRoute(pathname: string): string {
// Normalise a trailing slash (except root).
const path =
+10 -1
View File
@@ -3,6 +3,7 @@ import "@mantine/spotlight/styles.css";
import "@mantine/notifications/styles.css";
import '@mantine/dates/styles.css';
import "@/styles/a11y-overrides.css";
import "@/styles/notification-overrides.css";
import ReactDOM from "react-dom/client";
import App from "./App.tsx";
@@ -47,7 +48,15 @@ function renderApp() {
<MantineProvider theme={theme} cssVariablesResolver={mantineCssResolver}>
<ModalsProvider>
<QueryClientProvider client={queryClient}>
<Notifications position="bottom-center" limit={3} zIndex={10000} />
{/* top-center: toasts sit in the top of the viewport, in the line
of sight, and no longer cover centered content (e.g. "Load
more"). The below-chrome vertical offset is applied via a
position-scoped CSS rule in notification-overrides.css (NOT an
inline `style`): Mantine renders all six position containers at
once and an inline root style would land on every one, giving the
bottom-* containers both top+bottom full-viewport transparent
overlays that swallow clicks. */}
<Notifications position="top-center" limit={3} zIndex={10000} />
<HelmetProvider>
{/* Root boundary above every lazy route's Suspense: a stale-chunk
404 after a deploy is caught and recovered here instead of
@@ -0,0 +1,64 @@
/*
* Toast (Mantine Notification) visibility overrides.
* Mantine renders colorless toasts on --mantine-color-body (== the page
* background: white in light mode) with a faint shadow, so on white pages the
* card has no visible edge. These rules give every toast a type-tinted
* background, a WCAG-checked border and a stronger shadow so it separates from
* the page. The [data-mantine-color-scheme] + static-class selector (0,2,0)
* beats Mantine's own (0,1,0) rules regardless of stylesheet order (Mantine's
* bg/border rules wrap the scheme attribute in :where(), so they stay (0,1,0)).
* --notification-color is defined on the same element (defaults to primary,
* set per `color` prop), so tint/border follow the toast type. This also covers
* the loading/import toast (no accent bar, since the spinner takes the icon
* slot): its visibility comes from tone + border + shadow + the colored spinner.
*/
/*
* Push the top-anchored toast containers below the top chrome (fixed 45px
* header + optional 45px format toolbar + ~6px gap) so a toast (z-index 10000)
* neither covers nor intercepts clicks on the header/toolbar (both z-index 99).
*
* Scoped to [data-position^='top'] on purpose. Mantine renders ALL SIX position
* containers simultaneously (`position` only routes toasts into one via the
* store); the root `style` prop would be applied to every one of them by
* getStyles("root"). A blanket `top` would land on the bottom-* containers too
* (which carry `bottom:16px`) position:fixed + both edges + height:auto makes
* them stretch the full viewport height, and the container root has neither
* pointer-events:none nor a background, so those transparent z-10000 overlays
* would swallow clicks across the whole page. Restricting to top-* leaves the
* bottom containers at height:0.
*
* Specificity: `.mantine-Notifications-root[data-position^='top']` is (0,2,0)
* (class + attribute) and beats Mantine's own top rule
* `.m_b37d9ac7:where([data-position='top-center']){top:16px}` which is (0,1,0)
* (the :where() contributes 0), regardless of stylesheet order.
*/
.mantine-Notifications-root[data-position^='top'] {
top: 96px;
}
[data-mantine-color-scheme='light'] .mantine-Notification-root {
/* ~10% type color over white: clearly off-white, text contrast preserved */
background-color: color-mix(in srgb, var(--notification-color) 10%, var(--mantine-color-white));
/* Border must clear WCAG 3:1 non-text contrast on white. The repo rejects
gray-4 for this (a11y-overrides.css); gray-6 base (~3.32:1) darkened by the
type color stays >= 3:1. */
border: 1px solid color-mix(in srgb, var(--notification-color) 45%, var(--mantine-color-gray-6));
box-shadow: var(--mantine-shadow-xl);
}
[data-mantine-color-scheme='dark'] .mantine-Notification-root {
/* Dark page (dark-7/8) vs toast (dark-6) already separate a little; border +
shadow carry the type cue here (a 7% dark tint was near-invisible). */
background-color: color-mix(in srgb, var(--notification-color) 14%, var(--mantine-color-dark-6));
border: 1px solid color-mix(in srgb, var(--notification-color) 45%, var(--mantine-color-dark-3));
box-shadow: var(--mantine-shadow-xl);
}
/* Mantine's message-with-title color is gray-6 (#868e96, already only ~3.32:1
on white below AA 4.5:1); the new tint pushes it lower. Bump to gray-7 to
keep multi-line colored toasts readable, consistent with the repo's existing
WCAG tuning (theme.ts already bumps this same gray-6 up elsewhere). */
[data-mantine-color-scheme='light'] .mantine-Notification-description[data-with-title] {
color: var(--mantine-color-gray-7);
}
+3 -1
View File
@@ -23,7 +23,7 @@
"migration:reset": "tsx src/database/migrate.ts down-to NO_MIGRATIONS",
"migration:codegen": "kysely-codegen --dialect=postgres --camel-case --env-file=../../.env --out-file=./src/database/types/db.d.ts",
"lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
"pretest": "pnpm --filter @docmost/editor-ext build && pnpm --filter @docmost/prosemirror-markdown build",
"pretest": "pnpm --filter @docmost/editor-ext build && pnpm --filter @docmost/prosemirror-markdown build && pnpm --filter @docmost/token-estimate build",
"test": "jest",
"test:int": "jest --config test/jest-integration.json",
"test:watch": "jest --watch",
@@ -44,6 +44,7 @@
"@docmost/mcp": "workspace:*",
"@docmost/pdf-inspector": "1.9.6",
"@docmost/prosemirror-markdown": "workspace:*",
"@docmost/token-estimate": "workspace:*",
"@fastify/compress": "^9.0.0",
"@fastify/cookie": "^11.0.2",
"@fastify/multipart": "^10.0.0",
@@ -206,6 +207,7 @@
"^@docmost/db/(.*)$": "<rootDir>/database/$1",
"^@docmost/transactional/(.*)$": "<rootDir>/integrations/transactional/$1",
"^@docmost/ee/(.*)$": "<rootDir>/ee/$1",
"^@docmost/token-estimate$": "<rootDir>/../../../packages/token-estimate/src/index.ts",
"^src/(.*)$": "<rootDir>/$1",
"^@tiptap/react$": "<rootDir>/../test/stubs/tiptap-react.js"
}
+2 -2
View File
@@ -25,7 +25,7 @@ import { CacheModule } from '@nestjs/cache-manager';
import KeyvRedis from '@keyv/redis';
import { LoggerModule } from './common/logger/logger.module';
import { ClsModule } from 'nestjs-cls';
import { NoopAuditModule } from './integrations/audit/audit.module';
import { AuditModule } from './integrations/audit/audit.module';
import { ThrottleModule } from './integrations/throttle/throttle.module';
import { McpModule } from './integrations/mcp/mcp.module';
import { SandboxModule } from './integrations/sandbox/sandbox.module';
@@ -55,7 +55,7 @@ try {
middleware: { mount: true },
}),
LoggerModule,
NoopAuditModule,
AuditModule,
CoreModule,
DatabaseModule,
EnvironmentModule,
@@ -529,4 +529,107 @@ describe('replaceYjsMarkedText', () => {
expect(result).toEqual({ applied: false, currentText: 'abcdef' });
expect(text.toDelta()).toEqual(before);
});
// #496: apply must NOT silently strip the replaced run's inline formatting.
// Build a paragraph and format the marked range with extra marks, then assert
// the replacement carries them.
function buildFormatted(
runs: Array<{ text: string; attrs?: Record<string, any> }>,
): { fragment: Y.XmlFragment; text: Y.XmlText } {
const ydoc = new Y.Doc();
const fragment = ydoc.getXmlFragment('default');
const para = new Y.XmlElement('paragraph');
fragment.insert(0, [para]);
const text = new Y.XmlText();
para.insert(0, [text]);
text.insert(0, runs.map((r) => r.text).join(''));
let offset = 0;
for (const run of runs) {
if (run.attrs) text.format(offset, run.text.length, run.attrs);
offset += run.text.length;
}
return { fragment, text };
}
it('preserves the original run formatting (bold + link) on the replacement', () => {
const { fragment, text } = buildFormatted([
{ text: 'see ' },
{
text: 'old',
attrs: {
comment: { commentId: 'c1', resolved: false },
bold: true,
link: { href: 'https://x.test' },
},
},
{ text: ' end' },
]);
const result = replaceYjsMarkedText(fragment, 'c1', 'old', 'new');
expect(result).toEqual({ applied: true, currentText: 'new' });
// The comment anchor AND the bold/link marks survive the delete+insert.
expect(text.toDelta()).toEqual([
{ insert: 'see ' },
{
insert: 'new',
attributes: {
comment: { commentId: 'c1', resolved: false },
bold: true,
link: { href: 'https://x.test' },
},
},
{ insert: ' end' },
]);
});
it('mixed formatting under the mark: replacement takes the DOMINANT (longest) run, NOT the leading one', () => {
// Leading run is SHORT + plain ("x", 1 char); the following run is LONGER +
// bold ("bolded", 6 chars), same commentId. The longest run is deliberately
// NOT first: a "first-wins" pick would carry plain (no bold), so asserting
// bold on the result only holds if the code genuinely selects the LONGEST run.
const { fragment, text } = buildFormatted([
{ text: 'x', attrs: { comment: { commentId: 'c1', resolved: false } } },
{
text: 'bolded',
attrs: { comment: { commentId: 'c1', resolved: false }, bold: true },
},
]);
const result = replaceYjsMarkedText(fragment, 'c1', 'xbolded', 'Z');
expect(result).toEqual({ applied: true, currentText: 'Z' });
expect(text.toDelta()).toEqual([
{
insert: 'Z',
attributes: { comment: { commentId: 'c1', resolved: false }, bold: true },
},
]);
});
it('mixed formatting under the mark: on a length tie the FIRST run wins', () => {
// Two equal-length runs (2 chars each) with different formatting, same
// commentId. The reduce keeps the accumulator on a tie, so the FIRST run
// (italic) prevails over the later bold one.
const { fragment, text } = buildFormatted([
{
text: 'AA',
attrs: { comment: { commentId: 'c1', resolved: false }, italic: true },
},
{
text: 'BB',
attrs: { comment: { commentId: 'c1', resolved: false }, bold: true },
},
]);
const result = replaceYjsMarkedText(fragment, 'c1', 'AABB', 'Z');
expect(result).toEqual({ applied: true, currentText: 'Z' });
expect(text.toDelta()).toEqual([
{
insert: 'Z',
attributes: { comment: { commentId: 'c1', resolved: false }, italic: true },
},
]);
});
});
+20 -5
View File
@@ -145,6 +145,10 @@ type MarkedSegment = {
length: number;
text: string;
markAttrs: Record<string, any>;
// The FULL attribute set of this delta run — the `comment` mark plus any
// inline formatting (bold/italic/code/link/…). Captured so apply can carry the
// original run's formatting onto the replacement instead of dropping it.
attributes: Record<string, any>;
};
/**
@@ -202,6 +206,7 @@ export function replaceYjsMarkedText(
length,
text: insert,
markAttrs: markAttr,
attributes,
});
}
offset += length;
@@ -251,15 +256,25 @@ export function replaceYjsMarkedText(
return { applied: false, currentText: joinedText };
}
// 3. All guards passed: delete the marked run and re-insert newText with the
// same comment attributes at the same offset. Atomic within the caller's
// transaction.
// 3. All guards passed: delete the marked run and re-insert newText at the
// same offset. Atomic within the caller's transaction.
const start = segments[0].offset;
const len = segments.reduce((sum, s) => sum + s.length, 0);
const markAttrs = segments[0].markAttrs;
// Carry the ORIGINAL run's formatting onto the replacement (#496): inserting
// with only the `comment` mark silently dropped bold/italic/code/link of the
// replaced text. Yjs applies one flat attribute set to the whole insert, so
// when the marked run mixes formatting we pick the DOMINANT segment (the one
// covering the most characters) and apply its attributes — a v1 that preserves
// the common single-format case exactly and, for a mixed run, keeps the
// prevailing style rather than losing all of it. `attributes` already carries
// the `comment` mark (every collected segment is filtered on it above), so the
// anchor is preserved by copying the run's attribute set verbatim.
const dominant = segments.reduce((a, b) => (b.length > a.length ? b : a));
const insertAttrs = { ...dominant.attributes };
node.delete(start, len);
node.insert(start, newText, { comment: markAttrs });
node.insert(start, newText, insertAttrs);
return { applied: true, currentText: newText };
}
@@ -181,7 +181,7 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
{} as never, // pageAccess
{ isAiChatDeferredToolsEnabled: () => false, isAiChatFinalStepLockdownEnabled: () => false } as never, // environment
);
return { svc };
return { svc, aiChatMessageRepo };
}
const body = {
@@ -287,7 +287,7 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
// Drive stream() to the point streamText is called, capturing the options object
// (which carries onStepFinish/onFinish/onError/onAbort) and the run hooks.
async function captureStreamCallbacks() {
const { svc } = makeService();
const { svc, aiChatMessageRepo } = makeService();
let capturedOpts: any;
streamTextMock.mockImplementation((opts: any) => {
capturedOpts = opts;
@@ -314,7 +314,7 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
runHooks: runHooks as never,
});
expect(capturedOpts).toBeDefined();
return { capturedOpts, runHooks };
return { capturedOpts, runHooks, aiChatMessageRepo };
}
it('F9: onStepFinish bumps the run step count, onFinish settles the run "completed" (the dominant autonomous-run path)', async () => {
@@ -369,6 +369,51 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
expect.stringContaining('provider exploded'),
);
});
// #490 reactive branch: a provider CONTEXT-OVERFLOW 400 in onError is classified,
// records a distinguishable cause, and stamps metadata.replayOverflow so the NEXT
// turn's budgeter trims aggressively (the recovery that un-bricks the chat).
it('#490: a context-overflow 400 stamps replayOverflow on the finalized row', async () => {
jest
.spyOn(Logger.prototype, 'error')
.mockImplementation(() => undefined as never);
jest
.spyOn(Logger.prototype, 'warn')
.mockImplementation(() => undefined as never);
const { capturedOpts, aiChatMessageRepo } = await captureStreamCallbacks();
const overflow = Object.assign(new Error('too large'), {
statusCode: 400,
message:
"This model's maximum context length is 128000 tokens. However, your messages resulted in 214000 tokens. Please reduce the length.",
});
await capturedOpts.onError({ error: overflow });
// The seed row exists (finalizeOwner is the owner-write path).
expect(aiChatMessageRepo.finalizeOwner).toHaveBeenCalled();
const calls = aiChatMessageRepo.finalizeOwner.mock.calls as any[][];
const patch = calls[calls.length - 1][2] as {
status: string;
metadata: Record<string, unknown>;
};
expect(patch.status).toBe('error');
expect(patch.metadata.replayOverflow).toBe(true);
expect(patch.metadata.error).toContain('контекстное окно');
});
it('#490: a non-overflow error does NOT stamp replayOverflow', async () => {
jest
.spyOn(Logger.prototype, 'error')
.mockImplementation(() => undefined as never);
const { capturedOpts, aiChatMessageRepo } = await captureStreamCallbacks();
await capturedOpts.onError({ error: new Error('network reset') });
const calls = aiChatMessageRepo.finalizeOwner.mock.calls as any[][];
const patch = calls[calls.length - 1][2] as {
status: string;
metadata: Record<string, unknown>;
};
expect('replayOverflow' in patch.metadata).toBe(false);
});
});
/**
@@ -13,6 +13,7 @@ import {
compactToolOutput,
assistantParts,
serializeSteps,
type StepPartsCache,
rowToUiMessage,
prepareAgentStep,
stepBudgetWarning,
@@ -28,10 +29,14 @@ import {
FINAL_STEP_NUDGE,
STEP_LIMIT_NO_ANSWER_MARKER,
OUTPUT_DEGENERATION_ERROR,
lastAssistantContextTokens,
lastAssistantReplayOverflow,
seedActivatedTools,
} from './ai-chat.service';
import type { AiChatMessage, Workspace } from '@docmost/db/types/entity.types';
import { buildSystemPrompt } from './ai-chat.prompt';
import type { McpClientsService } from './external-mcp/mcp-clients.service';
import { resolveEffectiveReplayThreshold } from './history-budget';
/**
* Unit tests for compactToolOutput: the pure helper that shrinks tool outputs
@@ -114,6 +119,54 @@ describe('compactToolOutput', () => {
describe('assistantParts', () => {
type AnyPart = Record<string, unknown>;
// #490 memoization: assistantParts builds each step's parts once and caches
// them by the step OBJECT's identity, so a mid-stream flush does not
// re-stringify every prior step's (large) output. Observable property: with a
// shared cache, the second call over the SAME step object returns the cached
// (identical) part array even if the step's underlying output was swapped —
// proving the work was memoized, not redone.
it('memoizes a step by identity (shared cache => one build per step)', () => {
const cache: StepPartsCache = new WeakMap();
const step = {
text: 'x',
toolCalls: [{ toolCallId: 'c1', toolName: 'getPage', input: {} }],
toolResults: [{ toolCallId: 'c1', toolName: 'getPage', output: { v: 1 } }],
};
const first = assistantParts([step], '', cache) as AnyPart[];
expect((first.find((p) => p.type === 'tool-getPage')!.output as any).v).toBe(
1,
);
// Swap the output for a NEW value; a re-build would pick it up, a cache hit
// keeps the first result.
step.toolResults[0] = {
toolCallId: 'c1',
toolName: 'getPage',
output: { v: 2 },
};
const second = assistantParts([step], '', cache) as AnyPart[];
expect((second.find((p) => p.type === 'tool-getPage')!.output as any).v).toBe(
1,
);
// Same cached part objects are reused.
expect(second.find((p) => p.type === 'tool-getPage')).toBe(
first.find((p) => p.type === 'tool-getPage'),
);
});
it('without a cache, each call rebuilds (no stale memo)', () => {
const step = {
text: 'x',
toolCalls: [{ toolCallId: 'c1', toolName: 'getPage', input: {} }],
toolResults: [{ toolCallId: 'c1', toolName: 'getPage', output: { v: 1 } }],
};
const first = assistantParts([step], '') as AnyPart[];
step.toolResults[0].output = { v: 2 };
const second = assistantParts([step], '') as AnyPart[];
expect((second.find((p) => p.type === 'tool-getPage')!.output as any).v).toBe(
2,
);
});
it('emits output-available for a tool-call WITH a paired result', () => {
const steps = [
{
@@ -231,61 +284,320 @@ describe('assistantParts', () => {
});
});
describe('serializeSteps', () => {
// #490 trace format v2: per call the trace stores { input } for the call and an
// OUTCOME element — { ok: true } on success, { error, kind: 'thrown' } on a
// thrown tool-error, { error, kind: 'interrupted' } on a mid-step abort. The tool
// OUTPUT is no longer duplicated here (it lives once in metadata.parts).
describe('serializeSteps (trace v2)', () => {
it('returns null when there are no calls or results', () => {
expect(serializeSteps([])).toBeNull();
});
it('flattens calls and results into a compact trace', () => {
it('pairs a successful call with an { ok: true } outcome and NO output', () => {
const trace = serializeSteps([
{
toolCalls: [{ toolName: 'getPage', input: { id: 'p1' } }],
toolResults: [{ toolName: 'getPage', output: { title: 'T' } }],
toolCalls: [{ toolCallId: 'c1', toolName: 'getPage', input: { id: 'p1' } }],
toolResults: [{ toolCallId: 'c1', toolName: 'getPage' }],
},
]) as Array<Record<string, unknown>>;
expect(trace).toHaveLength(2);
expect(trace[0]).toEqual({ toolName: 'getPage', input: { id: 'p1' } });
expect(trace[1]).toEqual({ toolName: 'getPage', output: { title: 'T' } });
expect(trace[1]).toEqual({ toolName: 'getPage', ok: true });
// The output is NOT stored in the trace any more (dedup: it lives in parts).
expect(trace.some((e) => 'output' in e)).toBe(false);
});
it('records a THROWN tool failure (tool-error part) with its error message', () => {
it('records a THROWN failure with { error, kind: "thrown" }', () => {
const trace = serializeSteps([
{
toolCalls: [{ toolName: 'editPageText', input: { id: 'p1' } }],
toolCalls: [
{ toolCallId: 'c1', toolName: 'editPageText', input: { id: 'p1' } },
],
toolResults: [],
content: [
{
type: 'tool-error',
toolCallId: 'c1',
toolName: 'editPageText',
error: new Error('page is locked'),
},
],
},
]) as Array<Record<string, unknown>>;
// The call element is followed by a paired error element (mirroring how a
// successful result is appended), so the failure survives in the trace.
expect(trace).toHaveLength(2);
expect(trace[0]).toEqual({ toolName: 'editPageText', input: { id: 'p1' } });
expect(trace[1]).toEqual({
toolName: 'editPageText',
error: 'page is locked',
kind: 'thrown',
});
});
it('truncates a very long tool-error message to the tool-output limit', () => {
it('marks an interrupted call (no result, no throw) with kind "interrupted"', () => {
const trace = serializeSteps([
{
toolCalls: [
{ toolCallId: 'c1', toolName: 'createComment', input: { x: 1 } },
],
toolResults: [],
content: [],
},
]) as Array<Record<string, unknown>>;
expect(trace).toHaveLength(2);
expect(trace[1]).toEqual({
toolName: 'createComment',
error: 'Tool call did not complete.',
kind: 'interrupted',
});
// Structurally distinct from a thrown hard-fail so it never inflates an
// error-rate scan.
expect((trace[1] as { kind: string }).kind).not.toBe('thrown');
});
it('truncates a very long thrown-error message to the tool-output limit', () => {
const long = 'x'.repeat(5000);
const trace = serializeSteps([
{
toolCalls: [{ toolName: 'editPageText', input: {} }],
toolCalls: [{ toolCallId: 'c1', toolName: 'editPageText', input: {} }],
toolResults: [],
content: [{ type: 'tool-error', toolName: 'editPageText', error: long }],
content: [
{
type: 'tool-error',
toolCallId: 'c1',
toolName: 'editPageText',
error: long,
},
],
},
]) as Array<Record<string, unknown>>;
const errorText = trace[1].error as string;
// Truncated (not the full 5000 chars) and carries the omission marker.
expect(errorText.length).toBeLessThan(long.length);
expect(errorText).toContain('chars omitted');
});
it('pairs parallel calls in one step with their outcomes by id', () => {
const trace = serializeSteps([
{
toolCalls: [
{ toolCallId: 'a', toolName: 'getPage', input: {} },
{ toolCallId: 'b', toolName: 'searchPages', input: {} },
],
toolResults: [{ toolCallId: 'b', toolName: 'searchPages' }],
content: [
{ type: 'tool-error', toolCallId: 'a', toolName: 'getPage', error: 'nope' },
],
},
]) as Array<Record<string, unknown>>;
// call a, outcome a (thrown), call b, outcome b (ok)
expect(trace).toHaveLength(4);
expect(trace[1]).toEqual({ toolName: 'getPage', error: 'nope', kind: 'thrown' });
expect(trace[3]).toEqual({ toolName: 'searchPages', ok: true });
});
});
// #490: every assistant row flushAssistant writes carries the v2 era marker so a
// dual-shape diagnostic query can branch on the trace shape without inspecting it.
describe('toolTraceVersion era marker (#490)', () => {
it('stamps metadata.toolTraceVersion = 2 on every flushed row', () => {
const seed = flushAssistant([], '', 'streaming');
expect(seed.metadata.toolTraceVersion).toBe(2);
const done = flushAssistant(
[
{
text: 'ok',
toolCalls: [{ toolCallId: 'c1', toolName: 'getPage', input: {} }],
toolResults: [{ toolCallId: 'c1', toolName: 'getPage' }],
},
],
'',
'completed',
{ finishReason: 'stop' },
);
expect(done.metadata.toolTraceVersion).toBe(2);
});
});
// #490 replay-budget signal helpers over persisted history.
describe('lastAssistantContextTokens', () => {
const row = (
role: string,
metadata: Record<string, unknown> | null,
): AiChatMessage => ({ role, metadata }) as unknown as AiChatMessage;
it('reads the most recent assistant turn contextTokens (provider fact)', () => {
const hist = [
row('user', null),
row('assistant', { contextTokens: 12000 }),
row('user', null),
row('assistant', { contextTokens: 41000 }),
];
expect(lastAssistantContextTokens(hist)).toBe(41000);
});
it('returns undefined when the last assistant turn recorded no usage', () => {
const hist = [row('assistant', { error: 'boom' }), row('user', null)];
expect(lastAssistantContextTokens(hist)).toBeUndefined();
expect(lastAssistantContextTokens([])).toBeUndefined();
});
});
// #490 snapshotOpenPage fast-path: skip the full Markdown export + upsert when a
// snapshot already exists at the page's CURRENT version (same updated_at instant).
describe('snapshotOpenPage fast-path (#490)', () => {
function makeSvc(existingSnapshot: unknown, pageUpdatedAt: Date) {
const exportPageMarkdown = jest.fn(async () => '# md');
const upsert = jest.fn(async () => undefined);
const findByChatPage = jest.fn(async () => existingSnapshot);
const pageRepo = {
findById: jest.fn(async () => ({
id: 'p1',
workspaceId: 'ws1',
updatedAt: pageUpdatedAt,
})),
};
const svc = new AiChatService(
{} as never, // ai
{} as never, // aiChatRepo
{} as never, // aiChatMessageRepo
{ findByChatPage, upsert } as never, // aiChatPageSnapshotRepo
{} as never, // aiSettings
{ exportPageMarkdown } as never, // tools
{} as never, // mcpClients
{} as never, // aiAgentRoleRepo
pageRepo as never, // pageRepo
{} as never, // pageAccess
{} as never, // environment
);
return { svc, exportPageMarkdown, upsert, findByChatPage };
}
const args = () =>
[
'chat1',
'p1',
{ id: 'ws1' } as never,
{ id: 'u1' } as never,
'sess',
] as const;
it('skips export + upsert when the snapshot is already at this page version', async () => {
const t = new Date('2026-07-07T10:00:00Z');
const { svc, exportPageMarkdown, upsert } = makeSvc(
{ pageUpdatedAt: t, contentMd: '# md' },
t,
);
await (svc as unknown as { snapshotOpenPage: (...a: unknown[]) => Promise<void> })
.snapshotOpenPage(...args());
expect(exportPageMarkdown).not.toHaveBeenCalled();
expect(upsert).not.toHaveBeenCalled();
});
it('exports + upserts when the page advanced since the snapshot', async () => {
const { svc, exportPageMarkdown, upsert } = makeSvc(
{ pageUpdatedAt: new Date('2026-07-07T10:00:00Z'), contentMd: 'old' },
new Date('2026-07-07T11:00:00Z'),
);
await (svc as unknown as { snapshotOpenPage: (...a: unknown[]) => Promise<void> })
.snapshotOpenPage(...args());
expect(exportPageMarkdown).toHaveBeenCalledTimes(1);
expect(upsert).toHaveBeenCalledTimes(1);
});
it('seeds (exports + upserts) on the first turn (no snapshot yet)', async () => {
const { svc, exportPageMarkdown, upsert } = makeSvc(
undefined,
new Date('2026-07-07T10:00:00Z'),
);
await (svc as unknown as { snapshotOpenPage: (...a: unknown[]) => Promise<void> })
.snapshotOpenPage(...args());
expect(exportPageMarkdown).toHaveBeenCalledTimes(1);
expect(upsert).toHaveBeenCalledTimes(1);
});
});
// #490 deferred-tool activation persisted across turns.
describe('seedActivatedTools', () => {
const valid = new Set(['Search_web', 'getPageJson', 'diffPageVersions']);
it('seeds from persisted metadata, intersected with current valid names', () => {
expect(
seedActivatedTools(
{ activatedTools: ['Search_web', 'getPageJson'] },
valid,
),
).toEqual(['Search_web', 'getPageJson']);
});
it('drops a stored tool that is no longer valid (allowlist/role changed)', () => {
// 'Habr_publish' was activated before but is not in the current allowlist.
expect(
seedActivatedTools({ activatedTools: ['Search_web', 'Habr_publish'] }, valid),
).toEqual(['Search_web']);
});
it('is empty/robust for missing, non-array, or unknown-shaped metadata', () => {
expect(seedActivatedTools(undefined, valid)).toEqual([]);
expect(seedActivatedTools({}, valid)).toEqual([]);
expect(seedActivatedTools({ activatedTools: 'nope' }, valid)).toEqual([]);
expect(
seedActivatedTools({ activatedTools: [1, 'getPageJson', null] }, valid),
).toEqual(['getPageJson']);
});
it('de-duplicates stored names', () => {
expect(
seedActivatedTools(
{ activatedTools: ['getPageJson', 'getPageJson'] },
valid,
),
).toEqual(['getPageJson']);
});
});
describe('lastAssistantReplayOverflow', () => {
const row = (
role: string,
metadata: Record<string, unknown> | null,
): AiChatMessage => ({ role, metadata }) as unknown as AiChatMessage;
it('is true only when the LAST assistant turn overflowed', () => {
expect(
lastAssistantReplayOverflow([
row('assistant', { replayOverflow: true }),
row('user', null),
]),
).toBe(true);
// A recovered (later, non-overflow) assistant turn clears it.
expect(
lastAssistantReplayOverflow([
row('assistant', { replayOverflow: true }),
row('user', null),
row('assistant', { contextTokens: 5 }),
]),
).toBe(false);
expect(lastAssistantReplayOverflow([])).toBe(false);
});
// #490 reactive recovery: a prior turn stamped `replayOverflow` must make the
// NEXT turn's effective budget the AGGRESSIVE 0.5x cut — that harder trim is
// what un-bricks a chat that just 400'd on the context window. This exercises
// the exact wiring the service uses: read the stamp, then scale the threshold.
it('#490: a prior replayOverflow drives the next turn to the 0.5x aggressive budget', () => {
const history = [
row('assistant', { replayOverflow: true }),
row('user', null),
];
const priorOverflowed = lastAssistantReplayOverflow(history);
expect(priorOverflowed).toBe(true);
// Base budget 100k -> aggressive recovery halves it to 50k this turn.
expect(resolveEffectiveReplayThreshold(100_000, priorOverflowed)).toBe(50_000);
// Odd base floors, not rounds.
expect(resolveEffectiveReplayThreshold(99_999, true)).toBe(49_999);
// No prior overflow -> the base budget is used verbatim (no aggressive cut).
expect(resolveEffectiveReplayThreshold(100_000, false)).toBe(100_000);
// An explicit off-switch (null) is never overridden, even on recovery.
expect(resolveEffectiveReplayThreshold(null, true)).toBeNull();
});
});
describe('rowToUiMessage', () => {
@@ -618,6 +930,23 @@ describe('flushAssistant', () => {
expect(flushed.metadata.error).toBe('boom');
});
// #490 observability: the replay budgeter's decision is stamped on the turn.
it('records replayTrimmedToTokens + replayOverflow when provided', () => {
const f = flushAssistant([], '', 'error', {
error: 'ctx',
replayTrimmedToTokens: 42_000,
replayOverflow: true,
});
expect(f.metadata.replayTrimmedToTokens).toBe(42_000);
expect(f.metadata.replayOverflow).toBe(true);
});
it('omits the replay metadata when not provided', () => {
const f = flushAssistant([], '', 'completed', { finishReason: 'stop' });
expect('replayTrimmedToTokens' in f.metadata).toBe(false);
expect('replayOverflow' in f.metadata).toBe(false);
});
// #274 observability: the page-change diff the agent saw this turn is persisted
// to metadata.pageChanged when a non-empty diff was injected, and omitted when
// the diff is empty/whitespace or the arg is not supplied.
+423 -88
View File
@@ -55,6 +55,12 @@ import {
type SelectionContext,
} from './tools/current-page.util';
import { roleModelOverride } from './roles/role-model-config';
import {
resolveReplayBudget,
resolveEffectiveReplayThreshold,
isContextOverflowError,
trimHistoryForReplay,
} from './history-budget';
import {
startSseHeartbeat,
stripStreamingHopByHopHeaders,
@@ -117,9 +123,14 @@ const FINAL_STEP_NUDGE =
// NO text at all (#444, mitigates the "empty turn" the lockdown used to prevent
// when the toggle is OFF). Makes the exhausted-without-answer state explicit to
// the user and, on replay, to the model on the next turn.
// The persisted content is the app's base locale (en-US) — which is ALSO the
// i18n key the client localizes through `t()` — instead of a hardcoded Russian
// string (it used to render Russian for every locale, and fed Russian back to
// the model on replay). Keep it a plain, model-readable English sentence so the
// next turn's replay reads cleanly; the client resolves the locale.
const STEP_LIMIT_NO_ANSWER_MARKER =
'(Достигнут лимит шагов — итоговый ответ не сформулирован; работа могла ' +
'остаться незавершённой. Напишите «продолжай», чтобы агент продолжил.)';
'(Step limit reached — no final answer was produced; the work may be ' +
'unfinished. Reply "continue" to let the agent carry on.)';
// Reason recorded in ai_chat_runs.error / the assistant row when the token-
// degeneration detector (#444) aborts a run. Distinct from a user Stop (no error)
@@ -127,6 +138,15 @@ const STEP_LIMIT_NO_ANSWER_MARKER =
const OUTPUT_DEGENERATION_ERROR =
'Output degeneration detected (repeated token loop)';
// Prefix recorded on the assistant row when the provider rejected the turn for
// CONTEXT OVERFLOW (#490): the replayed history exceeded the model's window. The
// row is ALSO stamped `metadata.replayOverflow` so the NEXT turn's budgeter trims
// aggressively (the reactive recovery — the overflowing turn had no usage signal
// to trigger preventive trimming, so the classified 400 is what un-bricks it).
export const CONTEXT_OVERFLOW_ERROR_PREFIX =
'Диалог превысил контекстное окно модели; история будет агрессивно ' +
'сокращена на следующем ходу.';
/**
* Compute the step-budget warning text (#444), or '' when this step is outside
* the warning band. The warning fires on steps
@@ -882,6 +902,21 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
const freshPage = await this.pageRepo.findById(pageId);
// Page deleted during the turn (or somehow foreign) => don't write.
if (!freshPage || freshPage.workspaceId !== workspace.id) return;
// Fast-path (#490): if a snapshot already exists at THIS page version
// (same updated_at instant), its content is already current — skip the full
// Markdown export + upsert entirely. A turn that did NOT touch the open page
// (the common case) thus does no snapshot work. This mirrors the read-side
// fast path in detectPageChange (sameInstant): both trust that a page edit
// bumps updated_at. When the agent (or a human) DID edit the page this turn,
// updated_at advanced, so this does not match and we re-export as before.
const existing = await this.aiChatPageSnapshotRepo.findByChatPage(
chatId,
pageId,
workspace.id,
);
if (existing && sameInstant(existing.pageUpdatedAt, freshPage.updatedAt)) {
return;
}
const currentMd = await this.tools.exportPageMarkdown(
user,
sessionId,
@@ -921,10 +956,17 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
// supplied or the supplied one does not belong to this workspace.
let isNewChat = false;
let chatId = body.chatId;
// Persisted chat-level metadata bag (#490): read once here so the deferred-tool
// activation set can be seeded from the previous turn. Undefined for a new chat.
let chatMetadata: Record<string, unknown> | undefined;
if (chatId) {
const existing = await this.aiChatRepo.findById(chatId, workspace.id);
if (!existing) {
chatId = undefined;
} else {
chatMetadata = (existing.metadata ?? undefined) as
| Record<string, unknown>
| undefined;
}
}
// The open page the client sent is attacker-controllable — BOTH its id and
@@ -1086,7 +1128,7 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
// per-row conversion and degraded to plain text with a "[tool context
// omitted]" marker rather than 500-ing the whole turn (silent loss of tool
// context is not acceptable — the model must see the truncation).
const messages = await convertHistoryResilient(uiMessages, (index, err) =>
let messages = await convertHistoryResilient(uiMessages, (index, err) =>
this.logger.warn(
`Degraded unconvertible history row ${index} on chat ${chatId} to text: ${
err instanceof Error ? err.message : 'unknown error'
@@ -1135,6 +1177,56 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
// Here we only need the admin-configured system prompt.
const resolved = await this.aiSettings.resolve(workspace.id);
// History-replay token budget (#490). The full conversation is replayed to
// the provider every turn, so a long chat eventually 400s on the context
// window — forever. Bound the REPLAYED history (never the persisted rows).
// PRIMARY signal is the provider's own fact: the last turn's contextTokens.
const replayBudget = resolveReplayBudget(resolved?.chatContextWindowRaw);
if (replayBudget.usedDefault) {
// The default fires precisely for installs with NO configured window —
// the ones that hit terminal overflow. Warn so it is observable.
this.logger.warn(
`AI chat (chat ${chatId}): no chatContextWindow configured; ` +
`applying the default replay budget (${replayBudget.thresholdTokens} tokens).`,
);
}
// Last turn's provider-reported context size (authoritative when present).
const priorContextTokens = lastAssistantContextTokens(oldHistory);
// Reactive recovery (#490): if the LAST turn was rejected for context
// overflow (stamped by onError), trim AGGRESSIVELY this turn — the
// overflowing turn produced no usage signal, so a normal-threshold trim may
// not shrink enough to fit. This is what un-bricks a chat that just 400'd.
const priorOverflowed = lastAssistantReplayOverflow(oldHistory);
const effectiveThreshold = resolveEffectiveReplayThreshold(
replayBudget.thresholdTokens,
priorOverflowed,
);
if (priorOverflowed) {
this.logger.warn(
`AI chat (chat ${chatId}): previous turn hit context overflow; ` +
`applying aggressive replay budget (${effectiveThreshold} tokens).`,
);
}
const preTrim = trimHistoryForReplay(
messages,
effectiveThreshold,
// A prior OVERFLOW means the provider count is stale/absent — force the
// char-estimate path by ignoring priorContextTokens on recovery.
priorOverflowed ? undefined : priorContextTokens,
);
messages = preTrim.messages;
// Observability (#490): record the budgeter's decision on the turn so the UI
// can surface "replay truncated at N tokens". Threaded into flushAssistant.
let replayTrimmedToTokens: number | undefined = preTrim.trimmed
? preTrim.estimatedTokens
: undefined;
if (preTrim.trimmed) {
this.logger.log(
`AI chat (chat ${chatId}): replay history trimmed to ~${preTrim.estimatedTokens} ` +
`tokens (budget ${replayBudget.thresholdTokens}).`,
);
}
// Build the external MCP toolset FIRST so the system prompt can carry each
// connected server's admin-authored guidance (#180). Merge in admin-
// configured external MCP tools (web search, etc.; §6.8). A down/slow
@@ -1326,10 +1418,19 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
// tools + ALL external MCP tools), computed from the ACTUAL toolset so an
// external tool is loadable by its namespaced name. loadTools rejects any
// name outside this set.
const activatedTools = new Set<string>();
const validDeferredNames = new Set<string>(
Object.keys(baseTools).filter((k) => !CORE_TOOL_SET.has(k)),
);
// #490: seed the activation set from the chat's PERSISTED set so the model
// does not re-run loadTools every turn to re-activate the same tools. Only
// when deferred loading is enabled, and ALWAYS intersected with the CURRENT
// valid deferred names — an allowlist/role change must never resurrect a tool
// that no longer exists (prepareAgentStep would get a phantom active name).
const activatedTools = new Set<string>(
deferredEnabled
? seedActivatedTools(chatMetadata, validDeferredNames)
: [],
);
// Add the loadTools meta-tool ONLY when the feature is enabled; when off the
// toolset and behavior are exactly as before.
const tools = deferredEnabled
@@ -1339,6 +1440,39 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
}
: baseTools;
// #490: persist the (deterministically ordered) activation set back onto the
// chat metadata at turn end, so the NEXT turn seeds from it. Once-guarded and
// skipped when nothing new was activated (the set equals its seed) so an
// ordinary turn adds no extra write. Preserves other metadata keys.
let activatedToolsPersisted = false;
const persistActivatedTools = async (): Promise<void> => {
if (!deferredEnabled || activatedToolsPersisted || !chatId) return;
activatedToolsPersisted = true;
const current = [...activatedTools].sort();
const seeded = seedActivatedTools(chatMetadata, validDeferredNames).sort();
if (current.length === 0 || current.join('') === seeded.join('')) {
return; // nothing new activated -> no write
}
try {
await this.aiChatRepo.update(
chatId,
{
metadata: {
...(chatMetadata ?? {}),
activatedTools: current,
},
} as never,
workspace.id,
);
} catch (err) {
this.logger.warn(
`Failed to persist activated tools (chat ${chatId}): ${
err instanceof Error ? err.message : 'unknown error'
}`,
);
}
};
// Accumulate the turn's streamed output so a provider error / disconnect can
// persist the PARTIAL answer the user already saw — the SDK's onError/onAbort
// callbacks don't hand us the in-progress text. `capturedSteps` holds finished
@@ -1347,6 +1481,11 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
const capturedSteps: StepLike[] = [];
let inProgressText = '';
// Per-turn step->parts memo (#490): shared across every flushAssistant call
// this turn so each finished step's (large) output is JSON-stringified ONCE,
// not re-stringified on every subsequent onStepFinish flush (was O(N²)).
const partsCache: StepPartsCache = new WeakMap();
// Token-degeneration guard (#444). When the final-step lockdown is OFF, a
// runaway repetition loop (the 255KB "loadTools." incident) is aborted via
// this internal controller, unioned with the run/socket signal below. The
@@ -1416,7 +1555,10 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
await this.aiChatMessageRepo.update(
assistantId,
workspace.id,
flushAssistant(capturedSteps, '', 'streaming', { pageChanged }),
flushAssistant(capturedSteps, '', 'streaming', {
pageChanged,
partsCache,
}),
{ onlyIfStreaming: true },
);
} catch (err) {
@@ -1514,6 +1656,13 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
system,
messages,
tools,
// Pin the AI SDK per-request retry budget explicitly instead of relying
// on its default (which is also 2). Connection arithmetic per turn:
// (1 + maxRetries=2) × (1 + AI_STREAM_PRE_RESPONSE_RETRIES) network
// connects worst-case — the two retry layers compose, so making the SDK
// side explicit keeps that ceiling visible and pinned against SDK-default
// drift.
maxRetries: 2,
// No maxOutputTokens cap on the agent: tool-call arguments (e.g. a full
// page body for the write tools) are emitted as OUTPUT tokens, so a fixed
// cap would truncate complex tool calls mid-argument. Let the model use its
@@ -1649,6 +1798,8 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
// closure scope here). Omitted/0 = no limit.
maxContextTokens: resolved?.chatContextWindow,
pageChanged,
partsCache,
replayTrimmedToTokens,
}),
);
// #184/#487: the RUN is finalized ALWAYS (never gated on the message).
@@ -1673,6 +1824,8 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
// own edits are baked in — and this also SEEDS the snapshot on the first
// turn. Runs once across every terminal path (see snapshotTurnEnd).
await snapshotTurnEnd();
// #490: persist the deferred-tool activation set for the next turn.
await persistActivatedTools();
// Generate the chat title for a freshly created chat AFTER the stream's
// provider call has completed — NOT concurrently with it. The z.ai coding
@@ -1696,7 +1849,16 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
// object, so the actual provider cause is clearly logged. Reuse the
// shared formatter so provider error formatting stays unified.
const e = error as { stack?: string };
const errorText = describeProviderError(error, String(error));
// #490 reactive branch: classify a CONTEXT-OVERFLOW rejection (the
// replayed history exceeded the model window). The overflowing turn had
// no prior usage to trigger preventive trimming, so we record a clear,
// distinguishable cause AND stamp the row so the NEXT turn's budgeter
// trims aggressively — the reactive recovery that un-bricks the chat.
const overflow = isContextOverflowError(error);
const providerError = describeProviderError(error, String(error));
const errorText = overflow
? `${CONTEXT_OVERFLOW_ERROR_PREFIX} (${providerError})`
: providerError;
this.logger.error(`AI chat stream error: ${errorText}`, e?.stack);
// DIAGNOSTIC (Safari stream-drop investigation) — temporary: timing of
// an error-terminated stream.
@@ -1714,6 +1876,9 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
flushAssistant(capturedSteps, inProgressText, 'error', {
error: errorText,
pageChanged,
partsCache,
replayTrimmedToTokens,
replayOverflow: overflow || undefined,
}),
);
// #184: settle the RUN as failed, carrying the provider/transport cause.
@@ -1723,6 +1888,8 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
// committed before the error must be baked into the snapshot, or the
// next turn would mis-report it as a user edit.
await snapshotTurnEnd();
// #490: persist the deferred-tool activation set for the next turn.
await persistActivatedTools();
},
onAbort: async ({ steps }) => {
// #444: distinguish a degeneration abort (our internal controller) from
@@ -1737,6 +1904,7 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
flushAssistant(capturedSteps, truncated, 'error', {
error: OUTPUT_DEGENERATION_ERROR,
pageChanged,
partsCache,
}),
);
if (runId)
@@ -1747,6 +1915,8 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
);
await closeExternalClients();
await snapshotTurnEnd();
// #490: persist the deferred-tool activation set for the next turn.
await persistActivatedTools();
return;
}
const partialChars =
@@ -1771,6 +1941,7 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
await finalizeAssistant(
flushAssistant(capturedSteps, inProgressText, 'aborted', {
pageChanged,
partsCache,
}),
);
// #184: settle the RUN as aborted (an explicit user stop reached the
@@ -1781,6 +1952,8 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
// committed before the client disconnect / stop() must be baked into the
// snapshot, or the next turn would mis-report it as a user edit.
await snapshotTurnEnd();
// #490: persist the deferred-tool activation set for the next turn.
await persistActivatedTools();
},
});
@@ -2091,6 +2264,70 @@ export function chatStreamMetadata(
return undefined;
}
/**
* The provider-reported context size of the most recent assistant turn, read from
* its persisted `metadata.contextTokens` (#490 replay budgeter's PRIMARY signal
* the provider's own fact, not an estimate). Returns undefined for a chat with no
* assistant turn yet, or one whose last turn recorded no usage (e.g. it errored),
* in which case the budgeter falls back to the char-estimate.
*/
export function lastAssistantContextTokens(
history: ReadonlyArray<AiChatMessage>,
): number | undefined {
for (let i = history.length - 1; i >= 0; i--) {
const row = history[i];
if (row.role !== 'assistant') continue;
const meta = (row.metadata ?? {}) as { contextTokens?: unknown };
const n = meta.contextTokens;
return typeof n === 'number' && Number.isFinite(n) && n > 0 ? n : undefined;
}
return undefined;
}
/**
* Seed the per-turn deferred-tool activation set from a chat's persisted metadata
* (#490), INTERSECTED with the current valid deferred names. Persisting the set
* across turns saves the model re-running loadTools every turn to re-activate the
* same tools; intersecting on load means a changed allowlist / role can never
* resurrect a tool that no longer exists (which would hand prepareAgentStep a
* phantom active name). Tolerant of any stored shape a non-array is ignored.
*/
export function seedActivatedTools(
metadata: Record<string, unknown> | undefined,
validDeferredNames: ReadonlySet<string>,
): string[] {
const stored = metadata?.activatedTools;
if (!Array.isArray(stored)) return [];
const seen = new Set<string>();
const out: string[] = [];
for (const name of stored) {
if (typeof name === 'string' && validDeferredNames.has(name) && !seen.has(name)) {
seen.add(name);
out.push(name);
}
}
return out;
}
/**
* Whether the most recent assistant turn was rejected for CONTEXT OVERFLOW
* (#490): its row carries `metadata.replayOverflow` (stamped by the stream's
* onError). The next turn's budgeter reads this to trim aggressively the
* reactive recovery. Only the LAST assistant turn matters (an older overflow was
* already recovered), so we stop at the first assistant row scanning backwards.
*/
export function lastAssistantReplayOverflow(
history: ReadonlyArray<AiChatMessage>,
): boolean {
for (let i = history.length - 1; i >= 0; i--) {
const row = history[i];
if (row.role !== 'assistant') continue;
const meta = (row.metadata ?? {}) as { replayOverflow?: unknown };
return meta.replayOverflow === true;
}
return false;
}
/** The last message with role 'user' from a useChat payload, if any. */
function lastUserMessage(
messages: UIMessage[] | undefined,
@@ -2152,6 +2389,15 @@ export function sanitizeUserParts(
/** Marker for a history row whose tool parts could not be replayed (#489). */
export const TOOL_CONTEXT_OMITTED_MARKER = '[tool context omitted]';
/**
* Synthetic error text for a tool call that neither returned a result nor threw
* a `tool-error` i.e. it was interrupted mid-step (an abort / server restart).
* Shared by `assistantParts` (the replayed `output-error` part) and
* `serializeSteps` (the `{ kind: 'interrupted' }` trace element) so the replay
* text and the trace stay in lockstep (#490).
*/
export const TOOL_CALL_INCOMPLETE_TEXT = 'Tool call did not complete.';
/**
* Convert persisted UI history to model messages, tolerating a single poisoned
* row (#489). `convertToModelMessages` over the WHOLE array throws if ANY row is
@@ -2359,71 +2605,97 @@ function normalizeToolError(error: unknown): string {
*/
// Exported only so the unit tests can import these pure helpers; exporting
// them does not change runtime behavior.
/**
* Per-turn memo for {@link assistantParts}: a step's rebuilt parts keyed by the
* step OBJECT's identity (#490). A finished step in `capturedSteps` keeps a stable
* reference across every mid-stream flush, and `compactToolOutput` inside it does a
* `JSON.stringify` of the whole (often 50200 KB) output so without a memo each
* `onStepFinish` re-stringifies EVERY prior step's output (O(N²) stringify over a
* turn). Keyed by step identity => one stringify per step per turn. WeakMap so a
* turn's steps are GC'd with the turn.
*/
export type StepPartsCache = WeakMap<object, Array<Record<string, unknown>>>;
/** Build the parts for ONE step (text + a part per tool call). Pure. */
function buildStepParts(step: StepLike): Array<Record<string, unknown>> {
const parts: Array<Record<string, unknown>> = [];
if (step.text) {
parts.push({ type: 'text', text: step.text });
}
// Index this step's results by tool call id to pair calls with outputs.
const resultsById = new Map<string, unknown>();
for (const r of step.toolResults ?? []) {
if (r.toolCallId) resultsById.set(r.toolCallId, r.output);
}
// Index this step's THROWN tool failures (ai@6 `tool-error` content parts)
// by tool call id, so a call that failed replays with its real error text.
const errorsById = new Map<string, unknown>();
for (const part of step.content ?? []) {
if (part.type === 'tool-error' && part.toolCallId) {
errorsById.set(part.toolCallId, part.error);
}
}
for (const call of step.toolCalls ?? []) {
if (!call.toolName || !call.toolCallId) continue;
const hasResult = resultsById.has(call.toolCallId);
if (hasResult) {
// output-available: the tool returned; the next turn replays its result.
parts.push({
type: `tool-${call.toolName}`,
toolCallId: call.toolCallId,
state: 'output-available',
input: call.input,
output: compactToolOutput(resultsById.get(call.toolCallId)),
});
} else if (errorsById.has(call.toolCallId)) {
// The tool THREW: replay the REAL error so the model on the next turn
// knows WHY the call failed (and does not blindly repeat it). An
// output-error round-trips through convertToModelMessages as a balanced
// tool-call + tool-result, keeping the rebuilt history valid.
parts.push({
type: `tool-${call.toolName}`,
toolCallId: call.toolCallId,
state: 'output-error',
input: call.input,
errorText: normalizeToolError(errorsById.get(call.toolCallId)),
});
} else {
// No paired result AND no tool-error (e.g. aborted mid-step). Persisting
// a bare tool-call (input-available) would replay as an unpaired call and
// throw MissingToolResultsError on the next turn (convertToModelMessages
// emits no tool-result for it). Emit a SYNTHETIC paired result instead:
// an output-error round-trips through convertToModelMessages as a
// balanced tool-call + tool-result, keeping the rebuilt history valid.
parts.push({
type: `tool-${call.toolName}`,
toolCallId: call.toolCallId,
state: 'output-error',
input: call.input,
errorText: TOOL_CALL_INCOMPLETE_TEXT,
});
}
}
return parts;
}
export function assistantParts(
steps: ReadonlyArray<StepLike> | undefined,
fallbackText: string,
cache?: StepPartsCache,
): UIMessage['parts'] {
const parts: Array<Record<string, unknown>> = [];
let sawText = false;
for (const step of steps ?? []) {
if (step.text) {
parts.push({ type: 'text', text: step.text });
sawText = true;
}
// Index this step's results by tool call id to pair calls with outputs.
const resultsById = new Map<string, unknown>();
for (const r of step.toolResults ?? []) {
if (r.toolCallId) resultsById.set(r.toolCallId, r.output);
}
// Index this step's THROWN tool failures (ai@6 `tool-error` content parts)
// by tool call id, so a call that failed replays with its real error text.
const errorsById = new Map<string, unknown>();
for (const part of step.content ?? []) {
if (part.type === 'tool-error' && part.toolCallId) {
errorsById.set(part.toolCallId, part.error);
}
}
for (const call of step.toolCalls ?? []) {
if (!call.toolName || !call.toolCallId) continue;
const hasResult = resultsById.has(call.toolCallId);
if (hasResult) {
// output-available: the tool returned; the next turn replays its result.
parts.push({
type: `tool-${call.toolName}`,
toolCallId: call.toolCallId,
state: 'output-available',
input: call.input,
output: compactToolOutput(resultsById.get(call.toolCallId)),
});
} else if (errorsById.has(call.toolCallId)) {
// The tool THREW: replay the REAL error so the model on the next turn
// knows WHY the call failed (and does not blindly repeat it). An
// output-error round-trips through convertToModelMessages as a balanced
// tool-call + tool-result, keeping the rebuilt history valid.
parts.push({
type: `tool-${call.toolName}`,
toolCallId: call.toolCallId,
state: 'output-error',
input: call.input,
errorText: normalizeToolError(errorsById.get(call.toolCallId)),
});
} else {
// No paired result AND no tool-error (e.g. aborted mid-step). Persisting
// a bare tool-call (input-available) would replay as an unpaired call and
// throw MissingToolResultsError on the next turn (convertToModelMessages
// emits no tool-result for it). Emit a SYNTHETIC paired result instead:
// an output-error round-trips through convertToModelMessages as a
// balanced tool-call + tool-result, keeping the rebuilt history valid.
parts.push({
type: `tool-${call.toolName}`,
toolCallId: call.toolCallId,
state: 'output-error',
input: call.input,
errorText: 'Tool call did not complete.',
});
}
// Memoize per step object (#490): a finished step is immutable and keeps its
// reference across flushes, so its parts (and the costly output stringify) are
// built exactly once per turn. A cache miss (or no cache) just rebuilds.
let stepParts = cache?.get(step as object);
if (!stepParts) {
stepParts = buildStepParts(step);
cache?.set(step as object, stepParts);
}
parts.push(...stepParts);
}
const sawText = parts.some((p) => p.type === 'text');
if (!sawText && fallbackText) {
// No per-step text (e.g. a single final block): append the final text after
// any tool parts so the natural call -> result -> answer order is preserved.
@@ -2586,6 +2858,16 @@ export function flushAssistant(
maxContextTokens?: number;
error?: string;
pageChanged?: { title: string; diff: string } | null;
// Per-turn step->parts memo (#490): pass the SAME cache on every flush of a
// turn so each finished step's output is stringified once, not once per flush.
partsCache?: StepPartsCache;
// #490 observability: when the replay budgeter trimmed this turn's history,
// the (estimated) token size it trimmed to — the UI can show "replay truncated
// at N tokens". Omitted when nothing was trimmed.
replayTrimmedToTokens?: number;
// #490 reactive branch: set when the provider rejected this turn for context
// overflow. Stamped into metadata so the NEXT turn's budgeter trims aggressively.
replayOverflow?: boolean;
},
): AssistantFlush {
const finished = capturedSteps ?? [];
@@ -2595,13 +2877,20 @@ export function flushAssistant(
// in-progress step's text (the partial answer cut off by an error/abort, or
// simply not yet flushed mid-stream) as the last text part so the persisted
// parts match what streamed to the client.
const parts = assistantParts(finished, '') as unknown as Array<
Record<string, unknown>
>;
const parts = assistantParts(
finished,
'',
extra?.partsCache,
) as unknown as Array<Record<string, unknown>>;
if (trailing) parts.push({ type: 'text', text: trailing });
const metadata: Record<string, unknown> = {
parts: parts as unknown as UIMessage['parts'],
// Era marker for the `tool_calls` trace shape (#490): v2 stores outcome flags
// ({ ok } / { error, kind }) and NO tool output (the output lives once in
// `parts`). Old rows have no marker and the legacy { output } shape; a
// dual-shape query branches on this. Old rows are deliberately NOT migrated.
toolTraceVersion: 2,
};
// finishReason: prefer an explicit one; else derive a sensible value from the
// terminal status (so onError/onAbort records keep their historical reason).
@@ -2617,6 +2906,9 @@ export function flushAssistant(
if (extra?.contextTokens) metadata.contextTokens = extra.contextTokens;
if (extra?.maxContextTokens)
metadata.maxContextTokens = extra.maxContextTokens;
if (extra?.replayTrimmedToTokens)
metadata.replayTrimmedToTokens = extra.replayTrimmedToTokens;
if (extra?.replayOverflow) metadata.replayOverflow = true;
if (extra?.error) metadata.error = extra.error;
// Persist the page-change diff the agent saw this turn (#274 observability),
// so history / the Markdown export can show what the user changed. Only when
@@ -2642,42 +2934,85 @@ export function flushAssistant(
/**
* Reduce SDK step objects to a compact, JSON-serializable trace for the
* `tool_calls` column. Stores only what the UI action-log and history need
* never raw provider payloads or keys.
* `tool_calls` column trace format **v2** (#490).
*
* v2 stores, per call, ONLY the metadata a queryable trace needs never the
* tool OUTPUT. Before #490 each output was persisted TWICE: once here (compacted)
* and once in `metadata.parts` (via `assistantParts`), so a 50-step run with
* 50200 KB outputs wrote hundreds of MB per turn (each `onStepFinish` rewrote
* the whole row). The parts copy is the one the model replays and the UI/Markdown
* export render, so the trace copy of the output was pure duplication. v2 keeps
* the output ONLY in parts and reduces the trace to outcome flags.
*
* Element shapes (paired per call, in order):
* - `{ toolName, input }` the call
* - `{ toolName, ok: true }` it returned a result (success)
* - `{ toolName, error, kind: 'thrown' }` it threw a `tool-error`
* - `{ toolName, error, kind: 'interrupted' }` no result and no throw (an
* abort / server restart mid-step). `kind` is MANDATORY: without it a
* synthetic "Tool call did not complete." is indistinguishable from a real
* hard-fail and pollutes any error-rate scan. The distinction is STRUCTURAL
* (an `errorsById` hit vs the synthetic fallback branch), NOT a per-tool
* classifier soft failures stay OUT of the trace (they live in
* `metadata.parts` outputs; a per-tool mirror would persist its own bugs).
*
* Rows carry `metadata.toolTraceVersion: 2` (set by {@link flushAssistant}) so a
* dual-shape query can branch on the era. Old rows are NOT migrated (rewriting
* giant jsonb is the very WAL churn this removes); see docs/reading-ai-logs.md.
*/
export function serializeSteps(
steps: ReadonlyArray<{
toolCalls?: ReadonlyArray<{ toolName?: string; input?: unknown }>;
toolResults?: ReadonlyArray<{ toolName?: string; output?: unknown }>;
toolCalls?: ReadonlyArray<{
toolCallId?: string;
toolName?: string;
input?: unknown;
}>;
toolResults?: ReadonlyArray<{ toolCallId?: string; toolName?: string }>;
content?: ReadonlyArray<{
type?: string;
toolCallId?: string;
toolName?: string;
error?: unknown;
}>;
}>,
): unknown {
const calls: Array<{
toolName?: string;
input?: unknown;
output?: unknown;
error?: string;
}> = [];
const calls: Array<
| { toolName?: string; input?: unknown }
| { toolName?: string; ok: true }
| { toolName?: string; error: string; kind: 'thrown' | 'interrupted' }
> = [];
for (const step of steps ?? []) {
// Index this step's results + thrown errors by tool call id, so each call is
// paired with its outcome (mirrors assistantParts' pairing exactly).
const resultIds = new Set<string>();
for (const r of step.toolResults ?? []) {
if (r.toolCallId) resultIds.add(r.toolCallId);
}
const errorsById = new Map<string, unknown>();
for (const part of step.content ?? []) {
if (part.type === 'tool-error' && part.toolCallId) {
errorsById.set(part.toolCallId, part.error);
}
}
for (const call of step.toolCalls ?? []) {
calls.push({ toolName: call.toolName, input: call.input });
}
for (const r of step.toolResults ?? []) {
calls.push({ toolName: r.toolName, output: compactToolOutput(r.output) });
}
// ai@6 surfaces a THROWN tool failure as a `tool-error` content part, NOT as
// a `toolResults` entry. Record it as its own paired element (mirroring how a
// successful result is appended) so the failure and its reason survive in the
// trace instead of leaving an orphaned call with no result.
for (const part of step.content ?? []) {
if (part.type === 'tool-error') {
if (call.toolCallId && resultIds.has(call.toolCallId)) {
// Success: the output itself lives in metadata.parts, not here.
calls.push({ toolName: call.toolName, ok: true });
} else if (call.toolCallId && errorsById.has(call.toolCallId)) {
// Hard fail: the tool threw. Persist the real (bounded) reason.
calls.push({
toolName: part.toolName,
error: normalizeToolError(part.error),
toolName: call.toolName,
error: normalizeToolError(errorsById.get(call.toolCallId)),
kind: 'thrown',
});
} else {
// Neither a result nor a throw: interrupted mid-step (abort/restart).
// Marked structurally so it never inflates a thrown-error count.
calls.push({
toolName: call.toolName,
error: TOOL_CALL_INCOMPLETE_TEXT,
kind: 'interrupted',
});
}
}
@@ -0,0 +1,209 @@
import { randomBytes } from 'crypto';
import { Client } from 'pg';
import { flushAssistant, serializeSteps } from './ai-chat.service';
/**
* #490 write-volume regression an OBSERVABLE-PROPERTY test on a LIVE Postgres,
* not "bytes through a mock repo" (a mock measures exactly the thing that does not
* hurt). It drives a realistic 50-step run where each step returns a ~100 KB tool
* output and, at every `onStepFinish`, UPDATEs the assistant row the way the
* service does then reads the REAL write volume via the `pg_current_wal_lsn()`
* delta around the run.
*
* The property proven: v2 stores each tool OUTPUT only in `metadata.parts`, no
* longer ALSO in the `tool_calls` trace. So:
* 1. the trace (`tool_calls`) column's write volume is now O(Σ steps) tiny,
* linear outcome flags vs the pre-#490 O(N²) that re-persisted every prior
* output on every step; and
* 2. the FULL-row write volume drops sharply (the duplicated output copy is gone).
*
* Connects to the local gitmost test Postgres (docker `gitmost-test-pg` on :5432);
* SKIPS cleanly when that DB is not reachable so it never breaks a DB-less CI.
*/
const CONN =
process.env.WAL_TEST_DATABASE_URL ??
'postgresql://docmost:docmost_dev_pw@localhost:5432/docmost';
// A step whose tool output is ~100 KB (a page read), in the SDK StepLike shape.
// The body is INCOMPRESSIBLE random text — a `'x'.repeat()` filler would TOAST-
// compress to nothing and hide the real write volume (a page body does not).
function makeStep(i: number, outputBytes = 100_000) {
const body = randomBytes(Math.ceil(outputBytes * 0.75)).toString('base64');
return {
text: `step ${i} reasoning`,
toolCalls: [{ toolCallId: `c${i}`, toolName: 'getPage', input: { id: `p${i}` } }],
toolResults: [
{
toolCallId: `c${i}`,
toolName: 'getPage',
output: { id: `p${i}`, title: `Page ${i}`, body },
},
],
};
}
// The pre-#490 (v1) trace: outputs stored a SECOND time in `tool_calls`
// (the duplication #490 removed). Mirrors the OLD serializeSteps shape.
function v1Trace(steps: ReturnType<typeof makeStep>[]): unknown {
const calls: unknown[] = [];
for (const s of steps) {
for (const c of s.toolCalls) calls.push({ toolName: c.toolName, input: c.input });
for (const r of s.toolResults)
calls.push({ toolName: r.toolName, output: r.output });
}
return calls;
}
async function walDelta(
client: Client,
fn: () => Promise<void>,
): Promise<number> {
const before = (await client.query('SELECT pg_current_wal_lsn() AS l')).rows[0]
.l as string;
await fn();
// NOTE: do NOT pg_switch_wal() here — a segment switch pads the LSN to the next
// 16 MB boundary and would swamp the actual write delta. The raw LSN advances by
// the bytes of WAL emitted, which is exactly what we want to measure.
const after = (await client.query('SELECT pg_current_wal_lsn() AS l')).rows[0]
.l as string;
return Number(
(await client.query('SELECT pg_wal_lsn_diff($1,$2) AS d', [after, before]))
.rows[0].d,
);
}
describe('#490 write-volume on a live Postgres (pg_current_wal_lsn delta)', () => {
let client: Client | undefined;
let available = false;
beforeAll(async () => {
try {
client = new Client(CONN);
await client.connect();
await client.query('SELECT pg_current_wal_lsn()');
available = true;
} catch {
available = false;
client = undefined;
}
});
afterAll(async () => {
await client?.end().catch(() => undefined);
});
const STEPS = 50;
it('v2 trace write volume is O(Σ steps) — a tiny fraction of the v1 duplicate', async () => {
if (!available || !client) {
console.warn('SKIP: gitmost-test-pg not reachable; skipping WAL test.');
return;
}
const c = client;
// Isolated table so we measure only the tool_calls (trace) column's writes.
await c.query('DROP TABLE IF EXISTS _wal_trace');
await c.query('CREATE TABLE _wal_trace(id int primary key, tool_calls jsonb)');
await c.query("INSERT INTO _wal_trace VALUES (1, '[]'::jsonb)");
const steps: ReturnType<typeof makeStep>[] = [];
// v1: each step re-persists ALL prior outputs into the trace (the O(N²) churn).
const v1 = await walDelta(c, async () => {
const acc: ReturnType<typeof makeStep>[] = [];
for (let i = 0; i < STEPS; i++) {
acc.push(makeStep(i));
await c.query('UPDATE _wal_trace SET tool_calls=$1 WHERE id=1', [
JSON.stringify(v1Trace(acc)),
]);
}
steps.push(...acc);
});
await c.query("UPDATE _wal_trace SET tool_calls='[]'::jsonb WHERE id=1");
// v2: the REAL serializeSteps — outcome flags only, NO outputs.
const v2 = await walDelta(c, async () => {
const acc: ReturnType<typeof makeStep>[] = [];
for (let i = 0; i < STEPS; i++) {
acc.push(makeStep(i));
await c.query('UPDATE _wal_trace SET tool_calls=$1 WHERE id=1', [
JSON.stringify(serializeSteps(acc)),
]);
}
});
await c.query('DROP TABLE IF EXISTS _wal_trace');
// eslint-disable-next-line no-console
console.log(
`[#490 WAL] trace column over ${STEPS} steps: v1=${(v1 / 1e6).toFixed(1)}MB ` +
`v2=${(v2 / 1e6).toFixed(2)}MB (${(v1 / v2).toFixed(0)}x smaller)`,
);
// The trace no longer carries outputs: v2 is a tiny fraction of v1's WAL.
expect(v2).toBeLessThan(v1 * 0.1);
// And v2's trace WAL is small in absolute terms — O(Σ steps) of flags, not
// O(N² × output). 50 steps of ~40-byte flags is well under a few MB of WAL.
expect(v2).toBeLessThan(5_000_000);
// v1's duplicate alone is huge (≈ the 100 KB output re-written N² times).
expect(v1).toBeGreaterThan(50_000_000);
}, 120_000);
it('the full assistant row write drops sharply once the duplicate is gone', async () => {
if (!available || !client) return;
const c = client;
await c.query('DROP TABLE IF EXISTS _wal_full');
await c.query(
'CREATE TABLE _wal_full(id int primary key, content text, tool_calls jsonb, metadata jsonb, status text)',
);
await c.query("INSERT INTO _wal_full VALUES (1, '', '[]'::jsonb, '{}'::jsonb, 'streaming')");
const writeRow = async (patch: {
content: string;
toolCalls: unknown;
metadata: unknown;
status: string;
}) =>
c.query(
'UPDATE _wal_full SET content=$1, tool_calls=$2, metadata=$3, status=$4 WHERE id=1',
[
patch.content,
JSON.stringify(patch.toolCalls ?? null),
JSON.stringify(patch.metadata),
patch.status,
],
);
// v2 (real flushAssistant): outputs live once, in metadata.parts.
const v2 = await walDelta(c, async () => {
const acc: ReturnType<typeof makeStep>[] = [];
for (let i = 0; i < STEPS; i++) {
acc.push(makeStep(i));
await writeRow(flushAssistant(acc as never, '', 'streaming'));
}
});
await c.query("UPDATE _wal_full SET content='', tool_calls='[]'::jsonb, metadata='{}'::jsonb WHERE id=1");
// v1: same row PLUS the duplicated outputs in the trace column.
const v1 = await walDelta(c, async () => {
const acc: ReturnType<typeof makeStep>[] = [];
for (let i = 0; i < STEPS; i++) {
acc.push(makeStep(i));
const f = flushAssistant(acc as never, '', 'streaming');
await writeRow({ ...f, toolCalls: v1Trace(acc) });
}
});
await c.query('DROP TABLE IF EXISTS _wal_full');
// eslint-disable-next-line no-console
console.log(
`[#490 WAL] full row over ${STEPS} steps: v1=${(v1 / 1e6).toFixed(1)}MB ` +
`v2=${(v2 / 1e6).toFixed(1)}MB (saved ${((1 - v2 / v1) * 100).toFixed(0)}%)`,
);
// Removing the duplicated trace copy is a large, real write-volume reduction.
expect(v2).toBeLessThan(v1 * 0.75);
}, 120_000);
});
@@ -1,5 +1,10 @@
import { buildChatMarkdown, normalizeLang } from './chat-markdown.util';
import {
buildChatMarkdown,
normalizeLang,
labelledToolNames,
} from './chat-markdown.util';
import type { AiChatMessage } from '@docmost/db/types/entity.types';
import { SHARED_TOOL_SPECS } from '../../../../../packages/mcp/src/tool-specs';
/**
* normalizeLang: the client sends `i18n.language` a FULL locale tag like
@@ -455,3 +460,43 @@ describe('buildChatMarkdown (server) — structure', () => {
expect(md).toContain('````');
});
});
/**
* #494 REVERSE drift-guard for the export's friendly tool labels. A label keyed
* by a tool name that no longer exists silently degrades to the generic
* "Ran tool <name>" line; nothing reddened before. This asserts every labelled
* name is a real in-app tool and that both languages label the same set.
*/
describe('tool-label parity (#494)', () => {
// In-app tool names come from the shared registry (inAppKey, excluding
// mcpOnly specs) PLUS the inline in-app-only tools that carry a friendly label.
// The only labelled inline tool is the hybrid semantic search.
const INLINE_INAPP_LABELLED = new Set(['searchPages']);
function validInAppToolNames(): Set<string> {
const names = new Set<string>(INLINE_INAPP_LABELLED);
for (const spec of Object.values(SHARED_TOOL_SPECS)) {
if ((spec as { mcpOnly?: boolean }).mcpOnly) continue;
names.add((spec as { inAppKey: string }).inAppKey);
}
return names;
}
it('en and ru label the SAME set of tools', () => {
expect(labelledToolNames('en').sort()).toEqual(
labelledToolNames('ru').sort(),
);
});
it('every labelled tool name is a real in-app tool', () => {
const valid = validInAppToolNames();
const dead = labelledToolNames('en').filter((n) => !valid.has(n));
expect(dead).toEqual([]);
});
it('the guard REDDENS for an unknown label key (mutation check)', () => {
const valid = validInAppToolNames();
// A hypothetical renamed-away label must be caught.
expect(valid.has('getPageRenamedAway')).toBe(false);
});
});
@@ -154,6 +154,17 @@ function toolLabel(name: string, lang: ExportLang): string {
return LABELS[lang].tools[name] ?? LABELS[lang].ranTool(name);
}
/**
* The tool names that carry a hand-written friendly export label, per language.
* Exported for the drift-guard (#494): a label keyed by a tool name that no
* longer exists is a DEAD entry (the tool was renamed and now silently falls back
* to the generic `ranTool(name)` line). The guard asserts every key here is a
* real in-app tool AND that the two languages label the SAME set of tools.
*/
export function labelledToolNames(lang: ExportLang): string[] {
return Object.keys(LABELS[lang].tools);
}
/**
* Stringify an arbitrary tool input/output value for a fenced block. Strings
* pass through as-is; everything else is pretty-printed JSON, falling back to
@@ -37,10 +37,13 @@ export class CreateMcpServerDto {
@IsObject()
headers?: Record<string, string>;
// Omit/null => no restriction; `[]` is persisted verbatim and means deny-all
// (zero tools) since #476. @IsOptional() skips validation for null as well,
// so an explicit null is accepted.
@IsOptional()
@IsArray()
@IsString({ each: true })
toolAllowlist?: string[];
toolAllowlist?: string[] | null;
// Admin-authored guidance ("how/when to use this server's tools") injected
// into the agent system prompt next to the tool descriptions (#180). Trusted,
@@ -38,10 +38,13 @@ export class UpdateMcpServerDto {
@IsObject()
headers?: Record<string, string>;
// Absent => unchanged; null => no restriction; `[]` is persisted verbatim
// and means deny-all (zero tools) since #476. @IsOptional() skips validation
// for null as well, so an explicit null is accepted.
@IsOptional()
@IsArray()
@IsString({ each: true })
toolAllowlist?: string[];
toolAllowlist?: string[] | null;
// Admin-authored prompt guidance (#180). Absent => unchanged; blank => cleared
// (stored as null by the repo). Capped to bound prompt/token size.
@@ -0,0 +1,169 @@
import { type Tool } from 'ai';
import { McpClientsService } from './mcp-clients.service';
/**
* Tool-allowlist filtering semantics on the merged external toolset (#476).
*
* COVERAGE CHOICE (documented per issue #476): the full corrupt-row chain
* (DB value -> repo normalizeRow -> toolsFor filter) is covered on TWO levels
* instead of one live-stub-MCP-server integration test:
* (a) apps/server/test/integration/ai-mcp-server-repo.int-spec.ts pins the
* repo read/write semantics against a real Postgres `[]` round-trips
* as jsonb `[]`, a present-but-corrupt value fails CLOSED to `[]` with
* an error log;
* (b) THIS spec pins what the toolset builder does with the repo's output
* null = unrestricted, `['alpha']` = only alpha, `[]` (including the
* corrupt-row fallback) = ZERO tools.
* Together they prove the end-to-end property "corrupt/empty allowlist can
* never widen to all tools" without a live stub HTTP MCP server.
*
* The drive path mirrors mcp-namespacing.spec.ts: stub the repo's listEnabled,
* spy the private `connect` to return a fake client, inspect the merged keys.
*/
function fakeTool(): Tool {
return { description: 'x', inputSchema: undefined } as unknown as Tool;
}
interface FakeServer {
id: string;
name: string;
transport: string;
url: string;
headersEnc: string | null;
toolAllowlist: string[] | null;
}
function server(
over: Partial<FakeServer> & { id: string; name: string },
): FakeServer {
return {
transport: 'http',
url: 'https://example.com/mcp',
headersEnc: null,
toolAllowlist: null,
...over,
};
}
/**
* Build a service whose repo returns `servers` and whose fake clients expose
* `rawTools` from tools(). Returns the merged tool keys produced by toolsFor.
*/
async function mergedKeysFor(
servers: FakeServer[],
rawTools: Record<string, Tool>,
): Promise<string[]> {
const repoStub = {
listEnabled: jest.fn().mockResolvedValue(servers),
};
const service = new McpClientsService(repoStub as never, {} as never);
jest
.spyOn(
service as unknown as { connect: (s: FakeServer) => unknown },
'connect',
)
.mockImplementation(() =>
Promise.resolve({
tools: () => Promise.resolve(rawTools),
close: () => Promise.resolve(),
}),
);
const toolset = await service.toolsFor('ws-1');
// Release the lease so the service does not hold the fake clients open.
await Promise.all(toolset.clients.map((c) => c.close()));
return Object.keys(toolset.tools);
}
describe('external MCP tool-allowlist filtering (via toolsFor, #476)', () => {
afterEach(() => jest.restoreAllMocks());
const RAW = () => ({
alpha: fakeTool(),
beta: fakeTool(),
gamma: fakeTool(),
});
it("['alpha'] lets ONLY alpha through", async () => {
const keys = await mergedKeysFor(
[server({ id: 'id-1', name: 'srv', toolAllowlist: ['alpha'] })],
RAW(),
);
expect(keys).toEqual(['srv_alpha']);
});
it('null (no restriction) lets every tool through', async () => {
const keys = await mergedKeysFor(
[server({ id: 'id-1', name: 'srv', toolAllowlist: null })],
RAW(),
);
expect(keys.sort()).toEqual(['srv_alpha', 'srv_beta', 'srv_gamma']);
});
it('[] (deny-all) yields ZERO tools — an empty array is authoritative, not falsy (#476)', async () => {
// This is the regression the #476 change guards: `[]` used to fall through
// the old `allow.length > 0` check and expose ALL tools. It must expose NONE.
const keys = await mergedKeysFor(
[server({ id: 'id-1', name: 'srv', toolAllowlist: [] })],
RAW(),
);
expect(keys).toEqual([]);
});
it('the corrupt-row fallback ([] from the repo) also yields ZERO tools (#476)', async () => {
// The repo turns a present-but-corrupt tool_allowlist into `[]` (fail-closed,
// see normalizeRow in ai-mcp-server.repo.ts + the int-spec); this pins that
// the toolset builder honours that fallback as deny-all rather than allow-all.
const corruptFallback: string[] = [];
const keys = await mergedKeysFor(
[server({ id: 'id-1', name: 'srv', toolAllowlist: corruptFallback })],
RAW(),
);
expect(keys).toEqual([]);
});
it('allowlisted names not exposed by the server are ignored (no phantom tools)', async () => {
const keys = await mergedKeysFor(
[
server({
id: 'id-1',
name: 'srv',
toolAllowlist: ['alpha', 'does-not-exist'],
}),
],
RAW(),
);
expect(keys).toEqual(['srv_alpha']);
});
it('a deny-all server contributes no prompt instructions (0 tools merged)', async () => {
const repoStub = {
listEnabled: jest.fn().mockResolvedValue([
{
...server({ id: 'id-1', name: 'srv', toolAllowlist: [] }),
instructions: 'use the tools wisely',
},
]),
};
const service = new McpClientsService(repoStub as never, {} as never);
jest
.spyOn(
service as unknown as { connect: (s: FakeServer) => unknown },
'connect',
)
.mockImplementation(() =>
Promise.resolve({
tools: () => Promise.resolve(RAW()),
close: () => Promise.resolve(),
}),
);
const toolset = await service.toolsFor('ws-1');
await Promise.all(toolset.clients.map((c) => c.close()));
expect(Object.keys(toolset.tools)).toEqual([]);
// mergeNamespaced reported 0 contributed tools, so no guidance is attached.
expect(toolset.instructions).toEqual([]);
});
});
@@ -444,9 +444,13 @@ export class McpClientsService {
try {
client = await this.connectWithTimeout(server, CONNECT_TIMEOUT_MS);
const raw = await withTimeout(client.tools(), CONNECT_TIMEOUT_MS);
// Allowlist semantics (#476): null/absent = no restriction (all tools);
// ANY array — including `[]` — is authoritative, so an EMPTY allowlist
// yields ZERO tools (deny-all). Do NOT add a `.length > 0` escape here:
// that read `[]` as falsy and silently widened deny-all to allow-all
// (the repo also fails corrupt rows closed to `[]` for the same reason).
const allow = server.toolAllowlist;
const picked =
Array.isArray(allow) && allow.length > 0 ? pick(raw, allow) : raw;
const picked = Array.isArray(allow) ? pick(raw, allow) : raw;
// Bound each tool's execute with a per-call total-timeout guard before
// merging, so a single chatty-but-stuck call is aborted after the cap.
const guarded = wrapToolsWithCallTimeout(picked, callTimeoutMs);
@@ -100,7 +100,8 @@ export class McpServersService {
transport: dto.transport,
url: dto.url,
headersEnc,
// undefined => unchanged; [] / value handled by repo (empty => null).
// undefined => unchanged; null => no restriction; `[]` is persisted
// verbatim and means deny-all (#476).
toolAllowlist: dto.toolAllowlist,
// undefined => unchanged; blank => cleared (null) by the repo.
instructions: dto.instructions,
@@ -0,0 +1,266 @@
import type { ModelMessage } from 'ai';
import {
resolveReplayBudget,
isContextOverflowError,
estimateMessagesTokens,
trimHistoryForReplay,
REPLAY_BUDGET_DEFAULT_TOKENS,
REPLAY_TRUNCATION_MARKER,
REPLAY_TURN_COLLAPSED_MARKER,
} from './history-budget';
describe('resolveReplayBudget', () => {
it('uses floor(0.7 x window) for a configured window (no cap)', () => {
// 0.7 x 60k = 42k
expect(resolveReplayBudget(60_000)).toEqual({
thresholdTokens: 42_000,
usedDefault: false,
});
// 0.7 x 1M = 700k — NOT capped (anti-brick vs the window, not a cost limiter).
expect(resolveReplayBudget(1_000_000)).toEqual({
thresholdTokens: 700_000,
usedDefault: false,
});
});
it('accepts the raw ::text stored form', () => {
expect(resolveReplayBudget('60000').thresholdTokens).toBe(42_000);
});
// The crux (#490): a chat with NO context window configured must STILL be
// budgeted — those are exactly the installs that hit terminal overflow.
it('applies the flat default when the window is unset/empty', () => {
expect(resolveReplayBudget(undefined)).toEqual({
thresholdTokens: REPLAY_BUDGET_DEFAULT_TOKENS,
usedDefault: true,
});
expect(resolveReplayBudget('')).toEqual({
thresholdTokens: REPLAY_BUDGET_DEFAULT_TOKENS,
usedDefault: true,
});
expect(resolveReplayBudget(' ')).toEqual({
thresholdTokens: REPLAY_BUDGET_DEFAULT_TOKENS,
usedDefault: true,
});
});
it('treats an explicit 0 as the off-switch (distinct from unset)', () => {
expect(resolveReplayBudget(0)).toEqual({
thresholdTokens: null,
usedDefault: false,
});
expect(resolveReplayBudget('0')).toEqual({
thresholdTokens: null,
usedDefault: false,
});
});
it('falls back to the default on a negative/garbage value', () => {
expect(resolveReplayBudget(-5).usedDefault).toBe(true);
expect(resolveReplayBudget('abc').usedDefault).toBe(true);
});
});
describe('isContextOverflowError', () => {
it('classifies a real provider 400 context-overflow shape', () => {
// OpenAI-compatible shape.
expect(
isContextOverflowError({
statusCode: 400,
message:
"This model's maximum context length is 128000 tokens. However, your messages resulted in 214000 tokens. Please reduce the length of the messages.",
}),
).toBe(true);
// Anthropic-style wording.
expect(
isContextOverflowError({
status: 400,
message: 'prompt is too long: 250000 tokens > 200000 maximum',
}),
).toBe(true);
// Nested body + string status.
expect(
isContextOverflowError({
response: { status: '400' },
message: 'input is too long for the requested model',
}),
).toBe(true);
// Error instance with the cause carrying the body.
const e = new Error('Bad request');
(e as any).statusCode = 400;
(e as any).cause = new Error('maximum context window exceeded');
expect(isContextOverflowError(e)).toBe(true);
});
it('does NOT classify unrelated 400s or auth/rate-limit errors', () => {
expect(
isContextOverflowError({ statusCode: 400, message: 'invalid tool schema' }),
).toBe(false);
expect(
isContextOverflowError({
statusCode: 429,
message: 'context length exceeded but rate limited',
}),
).toBe(false);
expect(isContextOverflowError({ statusCode: 500, message: 'server error' })).toBe(
false,
);
expect(isContextOverflowError(undefined)).toBe(false);
expect(isContextOverflowError('some random string')).toBe(false);
});
});
// Helpers to build ModelMessage fixtures in the ai@6 shape.
const userMsg = (text: string): ModelMessage =>
({ role: 'user', content: [{ type: 'text', text }] }) as ModelMessage;
const assistantMsg = (
text: string,
toolCallId?: string,
toolName?: string,
): ModelMessage =>
({
role: 'assistant',
content: [
{ type: 'text', text },
...(toolCallId
? [{ type: 'tool-call', toolCallId, toolName, input: {} }]
: []),
],
}) as ModelMessage;
const toolMsg = (
toolCallId: string,
toolName: string,
value: unknown,
): ModelMessage =>
({
role: 'tool',
content: [
{ type: 'tool-result', toolCallId, toolName, output: { type: 'json', value } },
],
}) as ModelMessage;
describe('trimHistoryForReplay', () => {
it('null budget disables trimming (returns the same reference)', () => {
const msgs = [userMsg('hi'), assistantMsg('yo')];
const r = trimHistoryForReplay(msgs, null);
expect(r.trimmed).toBe(false);
expect(r.messages).toBe(msgs);
});
it('leaves history under budget untouched (same reference)', () => {
const msgs = [userMsg('hi'), assistantMsg('a short answer')];
const r = trimHistoryForReplay(msgs, 100_000);
expect(r.trimmed).toBe(false);
expect(r.messages).toBe(msgs);
});
it('truncates OLD tool outputs but keeps recent turns full', () => {
const big = 'X'.repeat(40_000); // ~16k tokens on its own
const msgs: ModelMessage[] = [];
// 6 OLD turns (indices 0..5), each with a huge tool output.
for (let i = 0; i < 6; i++) {
msgs.push(userMsg(`old q${i}`));
msgs.push(assistantMsg('looking', `c${i}`, 'getPage'));
msgs.push(toolMsg(`c${i}`, 'getPage', { body: big }));
msgs.push(assistantMsg(`old a${i}`));
}
// 3 small recent turns, then the CURRENT turn with its own huge tool output.
// With REPLAY_KEEP_RECENT_TURNS=4 the last 4 user-turns stay full, so only
// these small recent turns + the current big one are kept full; the 6 old
// turns above fall in the trim region.
for (let i = 0; i < 3; i++) {
msgs.push(userMsg(`recent q${i}`));
msgs.push(assistantMsg(`recent a${i}`));
}
msgs.push(userMsg('current q'));
msgs.push(assistantMsg('looking', 'cR', 'getPage'));
msgs.push(toolMsg('cR', 'getPage', { body: big }));
msgs.push(assistantMsg('current a'));
// Budget large enough that phase-1 tool truncation alone brings it under.
const r = trimHistoryForReplay(msgs, 30_000);
expect(r.trimmed).toBe(true);
const flat = JSON.stringify(r.messages);
// The CURRENT turn's tool output survives in full.
expect(flat).toContain(big);
// Old outputs were truncated with the marker.
expect(flat).toContain(REPLAY_TRUNCATION_MARKER);
// Phase 1 sufficed: the oldest turns were NOT collapsed.
expect(flat).not.toContain(REPLAY_TURN_COLLAPSED_MARKER);
expect(estimateMessagesTokens(r.messages)).toBeLessThan(
estimateMessagesTokens(msgs),
);
});
it('collapses the oldest turns when tool truncation is not enough', () => {
// Many turns with LARGE assistant TEXT (not tool output) so phase 1 can't help.
const bigText = 'слово '.repeat(8_000); // large Cyrillic text per turn
const msgs: ModelMessage[] = [];
for (let i = 0; i < 12; i++) {
msgs.push(userMsg(`q${i}`));
msgs.push(assistantMsg(bigText));
}
const r = trimHistoryForReplay(msgs, 30_000);
expect(r.trimmed).toBe(true);
// Oldest turns collapsed; result fits (best-effort) and is much smaller.
expect(estimateMessagesTokens(r.messages)).toBeLessThan(
estimateMessagesTokens(msgs),
);
// The LAST turn's text is preserved in full (recent turns stay full).
expect(JSON.stringify(r.messages[r.messages.length - 1])).toContain(bigText);
});
it('is deterministic / byte-stable for identical inputs', () => {
const big = 'Y'.repeat(30_000);
const build = (): ModelMessage[] => {
const m: ModelMessage[] = [];
for (let i = 0; i < 10; i++) {
m.push(userMsg(`q${i}`));
m.push(assistantMsg('t', `c${i}`, 'getPage'));
m.push(toolMsg(`c${i}`, 'getPage', { body: big }));
}
return m;
};
const a = trimHistoryForReplay(build(), 15_000);
const b = trimHistoryForReplay(build(), 15_000);
expect(JSON.stringify(a.messages)).toBe(JSON.stringify(b.messages));
});
it('never leaves an unpaired tool-call after collapsing (balanced history)', () => {
const big = 'Z'.repeat(40_000);
const msgs: ModelMessage[] = [];
for (let i = 0; i < 10; i++) {
msgs.push(userMsg(`q${i}`));
msgs.push(assistantMsg('t', `c${i}`, 'getPage'));
msgs.push(toolMsg(`c${i}`, 'getPage', { body: big }));
}
const r = trimHistoryForReplay(msgs, 8_000);
// Count tool-call vs tool-result parts in the trimmed output.
let calls = 0;
let results = 0;
for (const m of r.messages) {
if (!Array.isArray(m.content)) continue;
for (const p of m.content as Array<{ type?: string }>) {
if (p.type === 'tool-call') calls++;
if (p.type === 'tool-result' || p.type === 'tool-error') results++;
}
}
// Every surviving tool-call has a surviving result (collapsing drops BOTH).
expect(calls).toBe(results);
// Collapsed turns carry the marker.
expect(JSON.stringify(r.messages)).toContain(REPLAY_TURN_COLLAPSED_MARKER);
});
it('respects the provider fact: under-budget contextTokens skips trimming', () => {
const big = 'W'.repeat(60_000);
const msgs = [
userMsg('q'),
assistantMsg('t', 'c1', 'getPage'),
toolMsg('c1', 'getPage', { body: big }),
];
// char-estimate is high, but the provider says we are well under budget.
const r = trimHistoryForReplay(msgs, 100_000, 5_000);
expect(r.trimmed).toBe(false);
expect(r.messages).toBe(msgs);
});
});
@@ -0,0 +1,375 @@
/**
* History-replay token budget (#490).
*
* The whole persisted conversation is replayed to the provider on EVERY turn, so
* a long chat eventually exceeds the model's context window and the provider 400s
* on every turn terminally (the chat "bricks"). This module bounds the replayed
* history at REPLAY TIME only: it never mutates what is persisted (the DB stays
* the full record), and its output is a deterministic, byte-stable function of its
* input so the trimmed prefix is identical turn to turn (provider prompt-cache
* friendliness real money on long chats).
*
* The PRIMARY signal is the provider's own fact: `metadata.contextTokens` from the
* last turn. The chars-based {@link estimateTokens} (shared with the client) is
* used only for the DELTA of not-yet-sent messages, to decide WHAT to trim, and as
* the fallback for chats with no usage yet.
*/
import type { ModelMessage } from 'ai';
import { estimateTokens } from '@docmost/token-estimate';
/** Flat default budget when no context window is configured (tokens). */
export const REPLAY_BUDGET_DEFAULT_TOKENS = 100_000;
/** Fraction of a configured context window used as the budget. */
export const REPLAY_BUDGET_WINDOW_FRACTION = 0.7;
/**
* Fraction of the normal budget used for the REACTIVE re-trim after a provider
* context-overflow 400 the preventive estimate under-counted, so cut harder.
*/
export const REPLAY_AGGRESSIVE_FRACTION = 0.5;
/**
* Turns (a user message + its assistant/tool replies) kept FULL at the tail,
* including the current one never trimmed. Older turns are compacted first.
*/
export const REPLAY_KEEP_RECENT_TURNS = 4;
/** Leading chars kept from a truncated old tool output. */
export const REPLAY_TOOL_OUTPUT_HEAD = 800;
/** Trailing chars kept from a truncated old tool output. */
export const REPLAY_TOOL_OUTPUT_TAIL = 300;
/** Marker inserted where an old tool output was truncated for replay. */
export const REPLAY_TRUNCATION_MARKER =
'[…truncated for replay; call the tool again to read the full output]';
/** Marker for a whole old turn collapsed to its text. */
export const REPLAY_TURN_COLLAPSED_MARKER =
'[earlier tool activity omitted for replay]';
export interface ReplayBudget {
/** Token threshold above which replay history is trimmed; `null` = OFF. */
thresholdTokens: number | null;
/** True when the flat default was used (no context window configured). */
usedDefault: boolean;
}
/**
* Resolve the replay budget from the RAW stored `chatContextWindow` (text/number).
* - a positive value -> `floor(fraction × window)` (NO cap the budgeter is
* anti-brick protection against the window itself, not a cost/economy limiter,
* exactly as the codebase already treats maxOutputTokens; the reactive branch
* still guarantees anti-brick regardless of how high this budget is)
* - explicit `0` -> OFF (admin opt-out; `null` threshold)
* - unset/empty/invalid-> the flat default (still protects the installations
* that hit terminal overflow are exactly the ones that never set a window)
*
* Note the raw value is needed because the parsed `chatContextWindow` collapses
* both `0` and unset to `undefined`, which would erase the explicit off-switch.
*/
export function resolveReplayBudget(rawContextWindow: unknown): ReplayBudget {
let n: number | undefined;
if (typeof rawContextWindow === 'number') {
n = rawContextWindow;
} else if (typeof rawContextWindow === 'string') {
const t = rawContextWindow.trim();
n = t === '' ? undefined : Number(t);
}
// Unset / empty / non-numeric / negative -> flat default (the protective case).
if (n === undefined || !Number.isFinite(n) || n < 0) {
return { thresholdTokens: REPLAY_BUDGET_DEFAULT_TOKENS, usedDefault: true };
}
// Explicit 0 -> off-switch.
if (n === 0) {
return { thresholdTokens: null, usedDefault: false };
}
return {
thresholdTokens: Math.floor(REPLAY_BUDGET_WINDOW_FRACTION * n),
usedDefault: false,
};
}
/**
* The effective replay threshold for THIS turn, given the base budget and whether
* the PREVIOUS turn hit a context-overflow 400 (the reactive-recovery signal,
* `metadata.replayOverflow`). On recovery the base budget is scaled down by
* {@link REPLAY_AGGRESSIVE_FRACTION}: the overflowing turn produced no usage
* signal, so the preventive estimate under-counted and a normal-threshold trim may
* not shrink enough to fit this harder cut is what un-bricks the chat.
*
* A `null` base budget (trimming OFF) is passed through unchanged: an explicit
* off-switch is never overridden by the recovery path.
*/
export function resolveEffectiveReplayThreshold(
thresholdTokens: number | null,
priorOverflowed: boolean,
): number | null {
if (!priorOverflowed || thresholdTokens == null) return thresholdTokens;
return Math.floor(thresholdTokens * REPLAY_AGGRESSIVE_FRACTION);
}
/**
* True when a provider error is a CONTEXT-OVERFLOW rejection (the prompt exceeds
* the model's window). Providers surface this as an HTTP 400 with a recognizable
* message; match both the status and the message patterns robustly across
* OpenAI-compatible / Anthropic / Gemini wordings, since the exact shape varies.
*/
export function isContextOverflowError(error: unknown): boolean {
const status = extractStatus(error);
const msg = extractMessage(error).toLowerCase();
// Message patterns seen across providers for "prompt too long".
const overflowPattern =
/context (?:length|window)|maximum context|too many tokens|too large for|reduce the length|prompt is too long|input (?:is )?too long|exceeds? the (?:maximum )?(?:context|token)|maximum.*tokens|string too long/;
if (!overflowPattern.test(msg)) return false;
// A 400/413 with an overflow-shaped message is an overflow. Some providers
// omit/rewrite the status, so accept the message match when the status is
// unknown, but reject it for auth/rate-limit statuses that never mean overflow.
if (status === 400 || status === 413) return true;
if (status === 401 || status === 403 || status === 429) return false;
return true;
}
function extractStatus(error: unknown): number | undefined {
if (!error || typeof error !== 'object') return undefined;
const e = error as Record<string, unknown>;
for (const k of ['statusCode', 'status']) {
const v = e[k];
if (typeof v === 'number') return v;
if (typeof v === 'string' && /^\d+$/.test(v)) return Number(v);
}
// Nested (e.g. { response: { status } } / { cause: { statusCode } }).
for (const k of ['response', 'cause', 'data']) {
const nested = e[k];
if (nested && typeof nested === 'object') {
const s = extractStatus(nested);
if (s !== undefined) return s;
}
}
return undefined;
}
function extractMessage(error: unknown): string {
if (error == null) return '';
if (typeof error === 'string') return error;
if (error instanceof Error) {
// Include nested causes (provider libs wrap the real body in `cause`).
const cause = (error as { cause?: unknown }).cause;
return `${error.message} ${cause ? extractMessage(cause) : ''}`;
}
if (typeof error === 'object') {
const e = error as Record<string, unknown>;
const parts: string[] = [];
for (const k of ['message', 'error', 'body', 'responseBody', 'data']) {
const v = e[k];
if (typeof v === 'string') parts.push(v);
else if (v && typeof v === 'object') parts.push(extractMessage(v));
}
return parts.join(' ');
}
return String(error);
}
/** Rough token size of a ModelMessage array via the shared chars estimator. */
export function estimateMessagesTokens(
messages: ReadonlyArray<ModelMessage>,
): number {
let total = 0;
for (const m of messages) {
total += estimateTokens(serializeContent(m.content));
}
return total;
}
function serializeContent(content: unknown): string {
if (typeof content === 'string') return content;
try {
return JSON.stringify(content) ?? '';
} catch {
return '';
}
}
/** Deep JSON string of an arbitrary value, bounded so estimation never throws. */
function stringifyValue(value: unknown): string {
if (typeof value === 'string') return value;
try {
return JSON.stringify(value) ?? String(value);
} catch {
return String(value);
}
}
export interface TrimResult {
messages: ModelMessage[];
/** Whether any trimming was applied. */
trimmed: boolean;
/** Estimated tokens of the returned messages (chars-based). */
estimatedTokens: number;
}
/**
* Bound the replayed history to `budgetTokens`, deterministically. Returns the
* SAME array reference (no copy) when nothing needs trimming, so the common case
* is free and byte-identical. Trimming order (spec #490):
* 1. truncate OLD turns' tool outputs (head+tail + marker) the bulk of the size
* 2. mechanically collapse the OLDEST turns to their text (concatenation, no LLM)
* 3. the current + last {@link REPLAY_KEEP_RECENT_TURNS} turns stay FULL
*
* `budgetTokens === null` disables trimming. `priorContextTokens` (the provider's
* fact from last turn) short-circuits the decision: when it is known and already
* under budget we skip trimming even if the char-estimate is higher (the provider
* count is authoritative). The char-estimate drives WHAT to cut.
*/
export function trimHistoryForReplay(
messages: ModelMessage[],
budgetTokens: number | null,
priorContextTokens?: number,
): TrimResult {
if (budgetTokens == null) {
return { messages, trimmed: false, estimatedTokens: 0 };
}
const estimated = estimateMessagesTokens(messages);
// Decision signal: prefer the provider's fact (last turn's contextTokens) plus
// the estimated delta of the messages appended since; fall back to the pure
// char-estimate for a chat with no usage yet.
const projected =
priorContextTokens != null
? Math.max(priorContextTokens, estimated)
: estimated;
if (projected <= budgetTokens) {
return { messages, trimmed: false, estimatedTokens: estimated };
}
// The tail we always keep full: from the Nth-from-last user message onward.
const boundary = recentBoundaryIndex(messages, REPLAY_KEEP_RECENT_TURNS);
const tail = messages.slice(boundary);
let head = messages.slice(0, boundary).map(cloneMessage);
// Phase 1: truncate old tool outputs.
for (const m of head) {
if (m.role === 'tool') truncateToolMessage(m);
}
let out = [...head, ...tail];
let est = estimateMessagesTokens(out);
if (est <= budgetTokens) {
return { messages: out, trimmed: true, estimatedTokens: est };
}
// Phase 2: collapse the oldest turns (in `head`) to their text, one at a time,
// from the oldest, until we fit or the whole head is collapsed.
const turns = splitTurns(head);
const collapsed: ModelMessage[] = [];
let i = 0;
for (; i < turns.length; i++) {
if (est <= budgetTokens) break;
collapsed.push(...collapseTurn(turns[i]));
// Re-estimate the whole prospective output.
const remaining = turns.slice(i + 1).flat();
out = [...collapsed, ...remaining, ...tail];
est = estimateMessagesTokens(out);
}
// Include any turns we didn't need to collapse.
const remaining = turns.slice(i).flat();
out = [...collapsed, ...remaining, ...tail];
est = estimateMessagesTokens(out);
return { messages: out, trimmed: true, estimatedTokens: est };
}
/** Index of the first message of the Nth-from-last user turn (0 if fewer). */
function recentBoundaryIndex(
messages: ReadonlyArray<ModelMessage>,
keepTurns: number,
): number {
const userIdx: number[] = [];
for (let i = 0; i < messages.length; i++) {
if (messages[i].role === 'user') userIdx.push(i);
}
if (userIdx.length <= keepTurns) return 0;
return userIdx[userIdx.length - keepTurns];
}
/** Split a message list into turns; each turn starts at a `user` message. */
function splitTurns(messages: ModelMessage[]): ModelMessage[][] {
const turns: ModelMessage[][] = [];
for (const m of messages) {
if (m.role === 'user' || turns.length === 0) turns.push([m]);
else turns[turns.length - 1].push(m);
}
return turns;
}
/**
* Collapse a whole turn to its plain text (mechanical concatenation, not an LLM
* summary). Keeps the user message; replaces the assistant/tool messages with a
* single assistant text message = the assistant's concatenated text + a marker
* when tool activity was dropped. Dropping BOTH the tool-call and tool-result
* parts together keeps the rebuilt history balanced (no unpaired calls).
*/
function collapseTurn(turn: ModelMessage[]): ModelMessage[] {
const out: ModelMessage[] = [];
let assistantText = '';
let hadTools = false;
for (const m of turn) {
if (m.role === 'user') {
out.push(m);
} else if (m.role === 'assistant') {
const { text, tools } = extractAssistantText(m.content);
assistantText += text;
hadTools = hadTools || tools;
} else if (m.role === 'tool') {
hadTools = true;
} else {
out.push(m);
}
}
const note =
(assistantText ? assistantText.trimEnd() : '') +
(hadTools
? `${assistantText ? '\n\n' : ''}${REPLAY_TURN_COLLAPSED_MARKER}`
: '');
if (note) out.push({ role: 'assistant', content: note } as ModelMessage);
return out;
}
function extractAssistantText(content: unknown): {
text: string;
tools: boolean;
} {
if (typeof content === 'string') return { text: content, tools: false };
if (!Array.isArray(content)) return { text: '', tools: false };
let text = '';
let tools = false;
for (const part of content) {
const type = (part as { type?: string })?.type;
if (type === 'text') text += (part as { text?: string }).text ?? '';
else if (type === 'tool-call') tools = true;
}
return { text, tools };
}
/** Truncate every tool-result output in a `tool` message to head+tail+marker. */
function truncateToolMessage(message: ModelMessage): void {
const content = message.content;
if (!Array.isArray(content)) return;
for (const part of content) {
const p = part as { type?: string; output?: { type?: string; value?: unknown } };
if (p.type !== 'tool-result' && p.type !== 'tool-error') continue;
if (!p.output) continue;
const raw = stringifyValue(p.output.value);
const budget = REPLAY_TOOL_OUTPUT_HEAD + REPLAY_TOOL_OUTPUT_TAIL;
if (raw.length <= budget + REPLAY_TRUNCATION_MARKER.length) continue;
const truncated =
raw.slice(0, REPLAY_TOOL_OUTPUT_HEAD) +
`\n${REPLAY_TRUNCATION_MARKER}\n` +
raw.slice(raw.length - REPLAY_TOOL_OUTPUT_TAIL);
// Represent the shrunk output as a text output (a valid tool-result output).
p.output = { type: 'text', value: truncated };
}
}
/** Shallow-ish clone so trimming never mutates the caller's (persisted-derived)
* message objects only the OLD region is cloned before it is edited. */
function cloneMessage(m: ModelMessage): ModelMessage {
if (typeof m.content === 'string') return { ...m };
return {
...m,
content: (m.content as unknown[]).map((p) =>
p && typeof p === 'object' ? { ...(p as object) } : p,
),
} as ModelMessage;
}
@@ -9,9 +9,73 @@ import {
DEGENERATION_CHECK_STEP,
REPEATED_LINES_THRESHOLD,
MIN_PERIOD_REPEATS,
degenerationThresholds,
} from './output-degeneration';
import { AiChatService } from './ai-chat.service';
// Part A (#495 iter10): the detector thresholds are env-tunable. These drive the
// resolver against real repeat-count shapes and mutation-verify that the env
// override actually changes the trigger point (not a vacuous read).
describe('degeneration thresholds are env-configurable', () => {
const VARS = [
'AI_CHAT_DEGENERATION_REPEATED_LINES',
'AI_CHAT_DEGENERATION_PERIOD_MAX_LEN',
'AI_CHAT_DEGENERATION_PERIOD_MIN_REPEATS',
'AI_CHAT_DEGENERATION_CHECK_STEP',
];
const saved: Record<string, string | undefined> = {};
beforeEach(() => {
for (const v of VARS) saved[v] = process.env[v];
});
afterEach(() => {
for (const v of VARS) {
if (saved[v] === undefined) delete process.env[v];
else process.env[v] = saved[v];
}
});
it('defaults to the compiled constants when unset', () => {
for (const v of VARS) delete process.env[v];
expect(degenerationThresholds()).toEqual({
repeatedLines: REPEATED_LINES_THRESHOLD,
maxPeriodLen: 150,
minPeriodRepeats: MIN_PERIOD_REPEATS,
checkStep: DEGENERATION_CHECK_STEP,
});
});
it('falls back to the default on blank / invalid / non-positive values', () => {
for (const bad of ['', ' ', 'abc', '0', '-3', '1.5']) {
process.env.AI_CHAT_DEGENERATION_REPEATED_LINES = bad;
// '1.5' floors to 1 (still ≥1, valid); every other bad value → default.
const expected = bad === '1.5' ? 1 : REPEATED_LINES_THRESHOLD;
expect(degenerationThresholds().repeatedLines).toBe(expected);
}
});
it('a RAISED check-step suppresses a burst the default would have flagged', () => {
// A ~3.3KB periodic burst is periodic-degenerate, but shouldCheckDegeneration
// is the throttle gate. Default checkStep=2000 arms on it; raising the step
// above the burst size means the throttle never re-fires for it.
const burstLen = 'loadTools.\n'.repeat(300).length; // ~3300
delete process.env.AI_CHAT_DEGENERATION_CHECK_STEP;
expect(shouldCheckDegeneration(burstLen, 0)).toBe(true); // default 2000
process.env.AI_CHAT_DEGENERATION_CHECK_STEP = String(burstLen + 1);
expect(shouldCheckDegeneration(burstLen, 0)).toBe(false); // raised gate
});
it('a LOWERED repeated-lines threshold trips on a shorter identical-line run', () => {
// 8 identical lines: below the default 25 (rule 1) and below the periodic
// rule's 20 repeats — so isDegenerateOutput is false by default.
const shortRun = 'x\n'.repeat(8);
delete process.env.AI_CHAT_DEGENERATION_REPEATED_LINES;
expect(isDegenerateOutput(shortRun)).toBe(false);
// Lower rule 1 to 5 → the 8-line run now trips.
process.env.AI_CHAT_DEGENERATION_REPEATED_LINES = '5';
expect(isDegenerateOutput(shortRun)).toBe(true);
});
});
// Mock ONLY streamText so we can capture the onChunk/onStepFinish callbacks the
// service registers and drive them by hand; every other `ai` export the service
// uses (convertToModelMessages, stepCountIs, …) stays real.
@@ -23,6 +23,54 @@ export const MAX_PERIOD_LEN = 150;
/** Rule 2: minimum number of consecutive block repeats to trigger. */
export const MIN_PERIOD_REPEATS = 20;
/**
* Read a positive-integer threshold from an env var, falling back to `fallback`
* on unset/blank/invalid/non-positive. Mirrors the `AI_STREAM_PRE_RESPONSE_RETRIES`
* resolver in `ai-streaming-fetch.ts`: read the RAW string first so a blank value
* is treated as "unset" ( fallback) rather than coercing to 0. Thresholds must
* stay 1 a 0/negative would make the detector fire on any text (or never), so
* a bad value degrades to the safe compiled default instead. Env-tunable so an
* operator can retune the anti-babble guard (#444) without a redeploy, following
* the `AI_CHAT_FINAL_STEP_LOCKDOWN` toggle convention.
*/
function envThreshold(name: string, fallback: number): number {
const rawStr = process.env[name];
if (rawStr === undefined || rawStr.trim() === '') return fallback;
const raw = Number(rawStr);
return Number.isFinite(raw) && raw >= 1 ? Math.floor(raw) : fallback;
}
/**
* Resolve the degeneration-detector thresholds from the environment, each
* defaulting to the compiled constant above. Read fresh per call (not cached at
* import) so a test or a runtime env change takes effect deterministically.
*/
export function degenerationThresholds(): {
repeatedLines: number;
maxPeriodLen: number;
minPeriodRepeats: number;
checkStep: number;
} {
return {
repeatedLines: envThreshold(
'AI_CHAT_DEGENERATION_REPEATED_LINES',
REPEATED_LINES_THRESHOLD,
),
maxPeriodLen: envThreshold(
'AI_CHAT_DEGENERATION_PERIOD_MAX_LEN',
MAX_PERIOD_LEN,
),
minPeriodRepeats: envThreshold(
'AI_CHAT_DEGENERATION_PERIOD_MIN_REPEATS',
MIN_PERIOD_REPEATS,
),
checkStep: envThreshold(
'AI_CHAT_DEGENERATION_CHECK_STEP',
DEGENERATION_CHECK_STEP,
),
};
}
/**
* Rule 1 `REPEATED_LINES_THRESHOLD` consecutive IDENTICAL non-empty lines at
* the tail. Catches the classic newline-delimited loop ("loadTools.\n" ×N).
@@ -128,7 +176,11 @@ export function hasPeriodicTail(
* Pure the caller owns the abort side effect.
*/
export function isDegenerateOutput(text: string): boolean {
return hasRepeatedLineRun(text) || hasPeriodicTail(text);
const cfg = degenerationThresholds();
return (
hasRepeatedLineRun(text, cfg.repeatedLines) ||
hasPeriodicTail(text, cfg.maxPeriodLen, cfg.minPeriodRepeats)
);
}
/**
@@ -154,7 +206,7 @@ export function shouldCheckDegeneration(
textLen: number,
lastCheckLen: number,
): boolean {
return textLen - lastCheckLen >= DEGENERATION_CHECK_STEP;
return textLen - lastCheckLen >= degenerationThresholds().checkStep;
}
/**
@@ -307,6 +307,10 @@ export class PublicShareChatService {
system,
messages: modelMessages,
tools,
// Pin the AI SDK per-request retry budget explicitly (matches the SDK
// default of 2). Connection arithmetic: (1 + maxRetries) × (1 +
// AI_STREAM_PRE_RESPONSE_RETRIES) worst-case connects per turn.
maxRetries: 2,
// Bound the agent loop for anonymous callers.
stopWhen: stepCountIs(5),
// Cap per-request output so one anonymous call cannot run up the provider
@@ -426,6 +426,7 @@ export class AiChatToolsService {
const {
sharedToolSpecs,
createCommentSignalTracker,
createListCommentsProbe,
searchShapes,
getGuideSection,
} = await loadDocmostMcp();
@@ -718,7 +719,11 @@ export class AiChatToolsService {
if (spec.mcpOnly) continue;
if (spec.inlineBothHosts) continue;
const run = spec.inAppExecute ?? spec.execute;
if (!run) continue; // defensive: a shared spec always carries one of them.
// Guaranteed present by assertEverySpecIsRegisterable() (#494), which runs
// at tool-specs module load and throws if a non-inline spec the in-app host
// registers carries neither inAppExecute nor execute — so this can no longer
// silently drop a mis-declared tool. Kept as a type-narrowing guard.
if (!run) continue;
tools[spec.inAppKey] = sharedTool(
spec,
(async (args) =>
@@ -764,35 +769,21 @@ export class AiChatToolsService {
// wrapper below) so the race governs the whole call. The client carries the
// per-call composite signal via setToolAbortSignal.
const capMs = inAppToolCallCapMs();
if (!createCommentSignalTracker) {
// The signal needs BOTH the tracker factory AND the shared count-source probe
// factory (#494). Either being absent (a stale @docmost/mcp build or a mocked
// loader) => signal disabled, tool results byte-identical.
if (!createCommentSignalTracker || !createListCommentsProbe) {
return wrapInAppToolsWithCap(tools, client, capMs);
}
// Shared probe (#494): the SAME factory the standalone MCP host uses, so the
// in-app probe body is no longer a hand-mirror that could drift (counting the
// full feed newer than the watermark, labelling a hit with the light page
// title). `client` supplies the loopback listComments/getPageRaw reads.
const tracker = createCommentSignalTracker({
probe: async (pageId: string, sinceMs: number) => {
const { items } = await client.listComments(pageId, true);
const count = (items as Array<{ createdAt?: string }>).filter((c) => {
const created = c?.createdAt ? new Date(c.createdAt).getTime() : NaN;
return Number.isFinite(created) && created > sinceMs;
}).length;
let title: string | undefined;
if (count > 0) {
// Title labels the signal; untrusted, defanged by the shared builder.
// Fetched only on a hit so the no-signal path never pays for it. Uses
// the LIGHT raw page info (title only) — mirroring the standalone MCP
// probe's getPageRaw — instead of the heavy getPage (which also renders
// Markdown + subpages) just to read one field.
try {
const res = (await client.getPageRaw(pageId)) as {
title?: string;
} | null;
title = res?.title ?? undefined;
} catch {
// Title is optional — omit it when the page can't be fetched.
}
}
return { count, title };
},
probe: createListCommentsProbe(
client as unknown as Parameters<typeof createListCommentsProbe>[0],
),
});
return wrapInAppToolsWithCap(
@@ -21,7 +21,10 @@ import { SHARED_TOOL_SPECS } from '../../../../../../packages/mcp/src/tool-specs
// The REAL shared tracker factory, imported from source (same cross-boundary
// approach the tool-specs spec uses) so the in-app wiring is exercised against
// exactly the watermark/debounce/injection-safe logic the package ships.
import { createCommentSignalTracker } from '../../../../../../packages/mcp/src/comment-signal';
import {
createCommentSignalTracker,
createListCommentsProbe,
} from '../../../../../../packages/mcp/src/comment-signal';
// The REAL client-side citation extractor: proves that the passive signal does
// NOT strip a tool's citations (the #417 in-app regression this spec guards).
import { toolCitations } from '../../../../../../apps/client/src/features/ai-chat/utils/tool-parts';
@@ -284,9 +287,13 @@ describe('AiChatToolsService forUser + comment signal (real tracker)', () => {
return fakeClient as DocmostClientLike;
} as unknown as loader.DocmostClientCtor,
sharedToolSpecs: SHARED_TOOL_SPECS as unknown as Record<string, loader.SharedToolSpec>,
// Wire the REAL factory so the in-app path is exercised end to end.
// Wire the REAL factories so the in-app path is exercised end to end
// including the shared count-source probe (#494) the service now builds the
// tracker's `probe` from.
createCommentSignalTracker:
createCommentSignalTracker as unknown as loader.CommentSignalTrackerFactory,
createListCommentsProbe:
createListCommentsProbe as unknown as loader.CreateListCommentsProbeFn,
// Pure no-network draw.io helpers (#424) — required on the loader return;
// this comment-signal test doesn't exercise them, so no-op stubs suffice.
searchShapes: (() => []) as unknown as loader.SearchShapesFn,
@@ -150,6 +150,27 @@ export type CommentSignalTrackerFactory = (options: {
debounceMs?: number;
}) => CommentSignalTrackerLike;
/**
* Local mirror of `@docmost/mcp`'s `createListCommentsProbe` (#494): the SHARED
* count-source probe both hosts use, so the in-app probe body is no longer a
* hand-copy of the standalone MCP one. Given a client with the light comment feed
* + raw-page-title reads, it returns the tracker's `probe` (count comments newer
* than the watermark, label a hit with the page title). Loosely typed at this
* cross-package boundary, like the rest of this loader.
*/
export type CreateListCommentsProbeFn = (client: {
listComments(
pageId: string,
includeResolved: boolean,
): Promise<{ items: Array<{ createdAt?: string | null }> }>;
getPageRaw(
pageId: string,
): Promise<{ title?: string | null } | null | undefined>;
}) => (
pageId: string,
sinceMs: number,
) => Promise<CommentSignalProbeResultLike>;
// Pure, no-network draw.io helpers (#424). These are plain functions on the
// module (NOT DocmostClient methods) — the in-app AI-SDK service calls them
// directly to wire drawioShapes / drawioGuide, mirroring the MCP server.
@@ -170,6 +191,10 @@ interface DocmostMcpModule {
// loader in unit tests. The in-app layer treats an absent factory as "signal
// disabled" — a pure no-op that leaves tool results byte-identical.
createCommentSignalTracker?: CommentSignalTrackerFactory;
// Optional (#494): the shared count-source probe factory. Absent on a pre-#494
// build or a mocked loader; the in-app layer only builds a probe when the
// signal factory above is also present.
createListCommentsProbe?: CreateListCommentsProbeFn;
// Optional (#447): a deterministic hash of the tool-specs registry content,
// generated into build/ by the package's build. Absent on a pre-#447 build (or
// the mocked loader in unit tests) — the stale-check below is a NO-OP when it
@@ -284,6 +309,7 @@ export async function loadDocmostMcp(): Promise<{
DocmostClient: DocmostClientCtor;
sharedToolSpecs: Record<string, SharedToolSpec>;
createCommentSignalTracker?: CommentSignalTrackerFactory;
createListCommentsProbe?: CreateListCommentsProbeFn;
searchShapes: SearchShapesFn;
getGuideSection: GetGuideSectionFn;
}> {
@@ -331,6 +357,9 @@ export async function loadDocmostMcp(): Promise<{
// Optional: forwarded when present so the in-app layer can build the passive
// comment signal (#417); undefined on a stale build => signal disabled.
createCommentSignalTracker: mod.createCommentSignalTracker,
// Optional (#494): the shared count-source probe factory; undefined on a
// stale build => the in-app layer falls back to no signal.
createListCommentsProbe: mod.createListCommentsProbe,
// Pure no-network draw.io helpers (#424); not client methods.
searchShapes: mod.searchShapes,
getGuideSection: mod.getGuideSection,
@@ -16,6 +16,7 @@ import { UpdateCommentDto } from './dto/update-comment.dto';
import { ResolveCommentDto } from './dto/resolve-comment.dto';
import { ApplySuggestionDto } from './dto/apply-suggestion.dto';
import { DismissSuggestionDto } from './dto/dismiss-suggestion.dto';
import { ResyncSuggestionAnchorDto } from './dto/resync-suggestion-anchor.dto';
import { PageIdDto, CommentIdDto } from './dto/comments.input';
import { AuthUser } from '../../common/decorators/auth-user.decorator';
import { AuthWorkspace } from '../../common/decorators/auth-workspace.decorator';
@@ -235,6 +236,39 @@ export class CommentController {
return this.commentService.applySuggestion(comment, user, provenance);
}
@HttpCode(HttpStatus.OK)
@Post('resync-suggestion-anchor')
async resyncSuggestionAnchor(
@Body() dto: ResyncSuggestionAnchorDto,
@AuthUser() user: User,
@AuthWorkspace() workspace: Workspace,
) {
const comment = await this.commentRepo.findById(dto.commentId, {
includeCreator: true,
includeResolvedBy: true,
});
if (!comment) {
throw new NotFoundException('Comment not found');
}
const page = await this.pageRepo.findById(comment.pageId);
if (!page || page.deletedAt) {
throw new NotFoundException('Page not found');
}
// Authorize BEFORE revealing structural detail (mirrors apply/dismiss).
// Re-anchoring does NOT change the page text — it only corrects the stored
// selection metadata — so the page-level gate is comment access. The service
// further restricts it to the suggestion's own author.
await this.pageAccessService.validateCanComment(page, user, workspace.id);
return this.commentService.resyncSuggestionAnchor(
comment,
dto.selection,
user,
);
}
@HttpCode(HttpStatus.OK)
@Post('dismiss-suggestion')
async dismissSuggestion(
@@ -146,11 +146,19 @@ describe('CommentService — applySuggestion', () => {
'page-1',
expect.objectContaining({ operation: 'commentDeleted', commentId: 'c-1' }),
);
// #496: hard-deleted row → the audit payload is the only surviving record.
expect(auditService.log).toHaveBeenCalledWith(
expect.objectContaining({
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
resourceType: AuditResource.COMMENT,
resourceId: 'c-1',
metadata: expect.objectContaining({
pageId: 'page-1',
suggestedText: 'new text',
selection: 'old text',
commentAuthor: 'user-1',
decidedBy: 'user-1',
}),
}),
);
expect(result.outcome).toBe('deleted');
@@ -189,17 +197,25 @@ describe('CommentService — applySuggestion', () => {
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
expect(resolvePatch.resolvedById).toBe('user-1');
// NOT deleted; broadcast an update, not a deletion.
// NOT deleted.
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalledWith(
'deleteCommentMark',
expect.anything(),
expect.anything(),
);
// #496 dedup: resolveComment broadcasts `commentResolved` with the enriched
// row; finalize must NOT ALSO emit a redundant `commentUpdated`. So the
// thread receives exactly ONE resolve broadcast and no update broadcast.
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentUpdated', comment: UPDATED }),
expect.objectContaining({ operation: 'commentResolved', comment: UPDATED }),
);
expect(wsService.emitCommentEvent).not.toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentUpdated' }),
);
expect(auditService.log).toHaveBeenCalledWith(
@@ -211,6 +227,36 @@ describe('CommentService — applySuggestion', () => {
expect(result.outcome).toBe('resolved');
});
it('re-entry: already applied+resolved WITH replies → emits commentUpdated (dedup does not over-suppress)', async () => {
// suggestionAppliedAt set → idempotent finalize; resolvedAt set → resolveComment
// is skipped, so there is NO commentResolved broadcast. The applied-stamp state
// must still reach clients via a single commentUpdated.
const { service, wsService } = makeService(
{ applied: false, currentText: 'new text' },
true,
);
await service.applySuggestion(
suggestionComment({
suggestionAppliedAt: new Date(),
resolvedAt: new Date(),
}),
user(),
);
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentUpdated', comment: UPDATED }),
);
// Nothing resolved this time (already resolved) → no resolve broadcast.
expect(wsService.emitCommentEvent).not.toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentResolved' }),
);
});
// --- error / rejection branches -----------------------------------------
it('applied=false and currentText differs → ConflictException with currentText in payload', async () => {
@@ -107,11 +107,21 @@ describe('CommentService — dismissSuggestion', () => {
'page-1',
expect.objectContaining({ operation: 'commentDeleted', commentId: 'c-1' }),
);
// #496: the row is hard-deleted, so the audit payload must carry the
// decision's substance (what was suggested, the anchored text, who authored
// it, who decided) — it is the only surviving record.
expect(auditService.log).toHaveBeenCalledWith(
expect.objectContaining({
event: AuditEvent.COMMENT_SUGGESTION_DISMISSED,
resourceType: AuditResource.COMMENT,
resourceId: 'c-1',
metadata: expect.objectContaining({
pageId: 'page-1',
suggestedText: 'new text',
selection: 'old text',
commentAuthor: 'user-1',
decidedBy: 'user-1',
}),
}),
);
expect(result.outcome).toBe('deleted');
@@ -0,0 +1,126 @@
import { BadRequestException, ForbiddenException } from '@nestjs/common';
import { CommentService } from './comment.service';
/**
* Coverage for CommentService.resyncSuggestionAnchor (#496): re-anchoring a
* suggestion's stored selection (== apply-time expectedText) to the live-doc
* substring. The service is built directly with jest-mocked deps (the
* @InjectQueue tokens can't be resolved by Test.createTestingModule see the
* sibling specs).
*/
describe('CommentService — resyncSuggestionAnchor', () => {
const UPDATED = { id: 'c-1', selection: 'new anchor', __updated: true } as any;
function makeService() {
const commentRepo: any = {
updateComment: jest.fn(async () => undefined),
findById: jest.fn(async () => UPDATED),
};
const service = new CommentService(
commentRepo,
{} as any,
{ emitCommentEvent: jest.fn() } as any,
{} as any,
{ add: jest.fn() } as any,
{ add: jest.fn() } as any,
{ log: jest.fn() } as any,
);
return { service, commentRepo };
}
const suggestion = (over?: Partial<any>): any => ({
id: 'c-1',
creatorId: 'user-1',
parentCommentId: null,
selection: 'old anchor',
suggestedText: 'new text',
suggestionAppliedAt: null,
resolvedAt: null,
...over,
});
const user = (over?: Partial<any>): any => ({ id: 'user-1', ...over });
it('persists the new selection and returns the enriched comment', async () => {
const { service, commentRepo } = makeService();
const out = await service.resyncSuggestionAnchor(
suggestion(),
'new anchor',
user(),
);
expect(commentRepo.updateComment).toHaveBeenCalledWith(
{ selection: 'new anchor' },
'c-1',
);
expect(out).toBe(UPDATED);
});
it('is idempotent: no write when the anchor already matches', async () => {
const { service, commentRepo } = makeService();
const out = await service.resyncSuggestionAnchor(
suggestion({ selection: 'same' }),
'same',
user(),
);
expect(commentRepo.updateComment).not.toHaveBeenCalled();
expect(out).toEqual(suggestion({ selection: 'same' }));
});
it('rejects a non-author (only the suggestion owner may re-anchor)', async () => {
const { service, commentRepo } = makeService();
await expect(
service.resyncSuggestionAnchor(suggestion(), 'new anchor', user({ id: 'other' })),
).rejects.toBeInstanceOf(ForbiddenException);
expect(commentRepo.updateComment).not.toHaveBeenCalled();
});
it('rejects a reply / a comment with no suggestion', async () => {
const { service } = makeService();
await expect(
service.resyncSuggestionAnchor(
suggestion({ parentCommentId: 'p-1' }),
'x',
user(),
),
).rejects.toBeInstanceOf(BadRequestException);
await expect(
service.resyncSuggestionAnchor(
suggestion({ suggestedText: null }),
'x',
user(),
),
).rejects.toBeInstanceOf(BadRequestException);
});
it('rejects re-anchoring an already applied or resolved suggestion', async () => {
const { service } = makeService();
await expect(
service.resyncSuggestionAnchor(
suggestion({ suggestionAppliedAt: new Date() }),
'x',
user(),
),
).rejects.toBeInstanceOf(BadRequestException);
await expect(
service.resyncSuggestionAnchor(
suggestion({ resolvedAt: new Date() }),
'x',
user(),
),
).rejects.toBeInstanceOf(BadRequestException);
});
it('rejects a no-op selection equal to the suggested text', async () => {
const { service } = makeService();
await expect(
service.resyncSuggestionAnchor(
suggestion({ suggestedText: 'new text' }),
'new text',
user(),
),
).rejects.toBeInstanceOf(BadRequestException);
});
});
+122 -17
View File
@@ -370,6 +370,76 @@ export class CommentService {
return updatedComment;
}
/**
* Re-sync a suggestion's stored `selection` (== apply-time expectedText) to the
* RAW substring the inline mark actually covers in the LIVE document (#496).
*
* The MCP client creates the comment from a DEBOUNCED REST snapshot, then
* anchors the mark in the live collab doc. When the two disagree (the doc moved
* on in the debounce window) the stored selection no longer equals the marked
* text, so EVERY apply 409s ("the commented text changed"). After anchoring the
* client re-reads the exact marked substring and calls this to store it, making
* apply's strict equality hold.
*
* Only meaningful for an un-settled top-level suggestion authored by the
* caller: applying/resolving freezes the anchor, and a reply-carrying thread is
* preserved rather than mutated. The new text must still differ from the
* suggestion (else "apply" would be a no-op), preserving create()'s invariant.
*/
async resyncSuggestionAnchor(
comment: Comment,
selection: string,
user: User,
): Promise<Comment> {
if (comment.creatorId !== user.id) {
throw new ForbiddenException(
'You can only re-anchor your own suggestion',
);
}
if (comment.parentCommentId) {
throw new BadRequestException(
'Only a top-level comment can carry a suggested edit',
);
}
if (!comment.suggestedText) {
throw new BadRequestException('This comment has no suggested edit');
}
// A settled suggestion's anchor is frozen: re-anchoring an applied/resolved
// thread is meaningless and could resurrect a stale expectedText.
if (comment.suggestionAppliedAt || comment.resolvedAt) {
throw new BadRequestException(
'Cannot re-anchor a suggestion that was already applied or resolved',
);
}
const trimmed = selection.trim();
if (trimmed.length === 0) {
throw new BadRequestException('The re-anchored selection cannot be empty');
}
// Same no-op guard as create(): the suggestion must differ from the text it
// replaces, or apply becomes indistinguishable from already-applied.
if (trimmed === comment.suggestedText.trim()) {
throw new BadRequestException(
'A suggested edit must differ from the selected text',
);
}
// Idempotent: nothing to persist when the anchor already matches.
if (comment.selection === selection) {
return comment;
}
await this.commentRepo.updateComment({ selection }, comment.id);
const updatedComment = await this.commentRepo.findById(comment.id, {
includeCreator: true,
includeResolvedBy: true,
});
// Re-anchoring only corrects stored metadata; it does not change the page
// text or the comment body, so no ws broadcast / notification is warranted.
return updatedComment;
}
/**
* Apply the suggested edit carried by a top-level inline comment: atomically
* replace the text under the comment mark in the collaborative document with
@@ -524,7 +594,7 @@ export class CommentService {
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: { pageId: comment.pageId },
metadata: this.suggestionAuditMetadata(comment, user),
});
return { ...updatedComment, outcome: 'resolved' };
}
@@ -538,7 +608,7 @@ export class CommentService {
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: { pageId: comment.pageId },
metadata: this.suggestionAuditMetadata(comment, user),
});
return settled;
}
@@ -577,8 +647,10 @@ export class CommentService {
// Auto-resolve the thread. resolveComment handles the resolve mark, its ws
// broadcast and the resolve notification. Stay defensive on re-entry.
let didResolveBroadcast = false;
if (!comment.resolvedAt) {
await this.resolveComment(comment, true, user, provenance);
didResolveBroadcast = true;
}
const updatedComment = await this.commentRepo.findById(comment.id, {
@@ -586,18 +658,27 @@ export class CommentService {
includeResolvedBy: true,
});
this.wsService.emitCommentEvent(comment.spaceId, comment.pageId, {
operation: 'commentUpdated',
pageId: comment.pageId,
comment: updatedComment,
});
// #496 dedup: resolveComment already broadcast `commentResolved` carrying
// the fully-enriched row (the applied stamps were persisted above, before
// that call, so its re-read reflects them). Emitting `commentUpdated` here
// too made the client receive TWO events for one apply. Broadcast the
// update ONLY when we did NOT resolve — i.e. the rare re-entry on an
// already-resolved thread, where the applied-stamp change still needs a
// broadcast and resolveComment did not run.
if (!didResolveBroadcast) {
this.wsService.emitCommentEvent(comment.spaceId, comment.pageId, {
operation: 'commentUpdated',
pageId: comment.pageId,
comment: updatedComment,
});
}
this.auditService.log({
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: { pageId: comment.pageId },
metadata: this.suggestionAuditMetadata(comment, user),
});
return { ...updatedComment, outcome: 'resolved' };
@@ -616,7 +697,7 @@ export class CommentService {
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: { pageId: comment.pageId },
metadata: this.suggestionAuditMetadata(comment, user),
});
return settled;
@@ -627,14 +708,17 @@ export class CommentService {
* inline `comment` anchor mark, then ATOMICALLY hard-delete the row only if it
* is still childless. Shared by the apply/dismiss no-replies branches (#329).
*
* ORDER MATTERS: the anchor mark is removed FIRST and FATALLY (mirrors
* applySuggestion, which mutates the doc before writing the DB). The row
* delete is irreversible, so if the mark removal fails including the
* COLLAB_DISABLE_REDIS "no live instance" hard-error we must NOT delete the
* row and report success, or the document is left with a permanent orphan
* anchor pointing at a comment that no longer exists (the exact data-integrity
* bug #329 targets). Let the exception propagate ( 5xx); the operation is
* then repeatable with row + mark still consistent.
* ORDER MATTERS (updated #399 #496): what runs FIRST and FATALLY here is the
* mark-removal ENQUEUE (a fast, durable Redis add), NOT the mark op itself.
* deleteCommentMark awaits only the enqueue, so a failed add throws BEFORE the
* irreversible row delete the row + mark stay consistent and the operation is
* repeatable. The actual anchor strip then runs off the HTTP path in the worker
* (idempotent, 3 retries). Only an EXHAUSTED-retries job could leave the doc
* with an orphan anchor pointing at a hard-deleted comment (the data-integrity
* bug #329 targets); that residual divergence is now self-healed by the
* resolve/unresolve mark worker, which strips an orphan mark whenever its
* comment row is gone (#496), and it is meanwhile VISIBLE via BullMQ failed-job
* metrics rather than a silently-swallowed warn.
*
* RACE (#338 F4): the caller read `hasChildren` BEFORE the (slow) mark
* removal, so a reply can land in that window. `comments.parent_comment_id` is
@@ -732,6 +816,27 @@ export class CommentService {
return this.generalQueue.add(QueueJob.COMMENT_MARK_UPDATE, jobData);
}
/**
* Build the audit metadata for a suggestion apply/dismiss decision (#496).
* The subject comment is HARD-DELETED on the childless path, so the audit row
* is the only surviving record capture the decision's substance (what was
* suggested, the anchored text it replaced, who authored it, who decided)
* before the row can vanish. `decidedBy` is the acting user; `commentAuthor`
* is the suggestion's creator.
*/
private suggestionAuditMetadata(
comment: Comment,
user: User,
): Record<string, any> {
return {
pageId: comment.pageId,
suggestedText: comment.suggestedText ?? null,
selection: comment.selection ?? null,
commentAuthor: comment.creatorId ?? null,
decidedBy: user.id,
};
}
private async queueCommentNotification(
content: any,
oldMentionIds: string[],
@@ -0,0 +1,20 @@
import { IsString, IsUUID, MaxLength, MinLength } from 'class-validator';
/**
* #496: after the MCP client anchors a suggestion in the LIVE collab doc, it
* re-reads the exact substring under the new mark and syncs it here as the
* comment's stored `selection` (== apply-time expectedText). Fixes the perpetual
* 409 where expectedText came from a debounced REST snapshot while the mark sat
* in the live doc.
*/
export class ResyncSuggestionAnchorDto {
@IsUUID()
commentId: string;
// The raw substring the mark now covers in the live document. Bounded like the
// create-time selection (2000) so a legitimate anchored span is never cut.
@IsString()
@MinLength(1)
@MaxLength(2000)
selection: string;
}
@@ -17,9 +17,10 @@ import { MovePageDto } from './move-page.dto';
// a valid ordering key the server itself generated would be refused on move.
//
// The tests below assert the CORRECT contract: any key the generator can produce
// must satisfy the DTO. The genuinely-failing case is marked `test.failing` so the
// suite stays green while locking the bug; it flips red (alerting us) once the DTO
// bounds are widened to cover the generator's real range.
// must satisfy the DTO. FIXED (#495 item 9): the DTO now validates `position` by
// CHARSET ([0-9A-Za-z], the generator's base-62 alphabet) instead of the wrong
// @MaxLength(12) length bound, so dense between-inserts are accepted; the former
// `test.failing` bug-lock is now a passing assertion.
function constraintErrors(position: unknown) {
const dto = plainToInstance(MovePageDto, {
@@ -47,24 +48,33 @@ describe('MovePageDto.position vs generateJitteredKeyBetween parity', () => {
expect(hasError(errors, 'position')).toBe(false);
});
// BUG LOCK: dense between-inserts produce keys longer than 12 chars, which
// MaxLength(12) rejects even though they are valid ordering keys. This SHOULD
// pass; it currently fails. Flips green when the DTO bound is fixed.
test.failing(
'accepts dense between-inserted keys (currently rejected by MaxLength(12))',
async () => {
let lo = generateJitteredKeyBetween(null, null);
let hi = generateJitteredKeyBetween(lo, null);
// Repeatedly insert just above `lo`, shrinking the gap so the key grows.
let longest = lo;
for (let i = 0; i < 40; i++) {
const mid = generateJitteredKeyBetween(lo, hi);
if (mid.length > longest.length) longest = mid;
hi = mid;
}
expect(longest.length).toBeGreaterThan(12); // sanity: we produced a long key
const errors = await constraintErrors(longest);
expect(hasError(errors, 'position')).toBe(false);
},
);
// FIXED: dense between-inserts produce keys longer than 12 chars, which the old
// MaxLength(12) rejected even though they are valid ordering keys. Now accepted.
it('accepts dense between-inserted keys longer than 12 chars', async () => {
let lo = generateJitteredKeyBetween(null, null);
let hi = generateJitteredKeyBetween(lo, null);
// Repeatedly insert just above `lo`, shrinking the gap so the key grows. The
// generator is JITTERED (random), so use enough iterations that the longest
// key reliably clears the old 12-char bound: measured min-over-50-trials is
// ~11 at 40 iterations (flaky) but ~36 at 200 (robust margin).
let longest = lo;
for (let i = 0; i < 200; i++) {
const mid = generateJitteredKeyBetween(lo, hi);
if (mid.length > longest.length) longest = mid;
hi = mid;
}
expect(longest.length).toBeGreaterThan(12); // sanity: we produced a long key
const errors = await constraintErrors(longest);
expect(hasError(errors, 'position')).toBe(false);
});
// The charset guard replaces the length bound: reject anything outside the
// generator's [0-9A-Za-z] alphabet (control chars, separators, injection) and
// the empty string, while still accepting every real key.
it('rejects a position with characters outside the fractional-index alphabet', async () => {
for (const bad of ['a0/b', 'a b', 'a\n0', 'a.b', '', "a';--"]) {
const errors = await constraintErrors(bad);
expect(hasError(errors, 'position')).toBe(true);
}
});
});
+13 -3
View File
@@ -1,8 +1,8 @@
import {
IsString,
IsOptional,
MinLength,
MaxLength,
Matches,
IsNotEmpty,
} from 'class-validator';
@@ -10,9 +10,19 @@ export class MovePageDto {
@IsString()
pageId: string;
// `position` is a fractional-indexing key from `generateJitteredKeyBetween`
// (the SAME generator page.service uses). Validate by CHARSET, not length: the
// generator's default base-62 alphabet is [0-9A-Za-z], and DENSE between-inserts
// legitimately grow a key well past a dozen chars (measured >40), so the old
// @MinLength(5)/@MaxLength(12) bounds wrongly 400'd valid ordering keys the
// server itself produced (Gitea #139 item 6). The charset regex rejects control
// chars / separators / injection, and a generous MaxLength stays only as a
// DoS guard — far above any realistic key, so it never rejects a real move.
@IsString()
@MinLength(5)
@MaxLength(12)
@Matches(/^[0-9A-Za-z]+$/, {
message: 'position must be a fractional-index key ([0-9A-Za-z])',
})
@MaxLength(256)
position: string;
@IsOptional()
@@ -0,0 +1,49 @@
import 'reflect-metadata';
import { plainToInstance } from 'class-transformer';
import { validate } from 'class-validator';
import { PageIdDto } from './page.dto';
// #435: PageIdDto.pageId carries a page's DOUBLE identity (internal UUID OR
// public 10-char slugId), both as bare strings. The DTO must accept exactly
// those two FORMATS and reject a malformed / swapped identity at the boundary.
async function pageIdErrors(pageId: unknown) {
const dto = plainToInstance(PageIdDto, { pageId });
const errors = await validate(dto as object);
return errors.some((e) => e.property === 'pageId');
}
const UUID = '019f499a-9f8c-7d68-b7be-ce100d7c6c56';
const SLUG = 'aB3xQ7kR2p';
describe('PageIdDto pageId format validation', () => {
it('accepts a canonical page UUID', async () => {
expect(await pageIdErrors(UUID)).toBe(false);
});
it('accepts a 10-char slugId', async () => {
expect(await pageIdErrors(SLUG)).toBe(false);
});
it('rejects a truncated / wrong-length slug', async () => {
expect(await pageIdErrors('aB3xQ7kR2')).toBe(true); // 9 chars
expect(await pageIdErrors('aB3xQ7kR2pX')).toBe(true); // 11 chars
});
it('rejects a slug with an illegal character', async () => {
expect(await pageIdErrors('aB3xQ7kR2!')).toBe(true);
});
it('rejects a full URL / path-shaped identity (not the bare id)', async () => {
expect(await pageIdErrors(`my-page-title-${SLUG}`)).toBe(true);
expect(await pageIdErrors(`https://x/p/${SLUG}`)).toBe(true);
});
it('rejects a malformed UUID', async () => {
expect(await pageIdErrors('019f499a-9f8c-7d68-b7be')).toBe(true);
expect(await pageIdErrors('not-a-uuid-at-all-really')).toBe(true);
});
it('rejects an empty string', async () => {
expect(await pageIdErrors('')).toBe(true);
});
});
@@ -0,0 +1,39 @@
import { applyDecorators } from '@nestjs/common';
import { Matches, ValidationOptions } from 'class-validator';
/**
* A page identity at the API boundary is EITHER the internal page UUID or the
* public 10-char slugId (page.repo.findById matches a non-UUID input as a
* slugId). Both arrive as bare strings, which is exactly how the two got swapped
* silently (incident family #435). This regex pins the two accepted FORMATS so a
* malformed / cross-wired identity (a truncated slug, a full URL, an email, an
* id from another entity kind that isn't even shaped like either) is rejected at
* the boundary instead of falling through to a confusing 404.
*
* - UUID: canonical 8-4-4-4-12 hex (version-agnostic page ids are UUIDv7,
* so only the shape/length is enforced, matching the MCP's UUID_RE
* and the server's isValidUUID acceptance).
* - slugId: exactly 10 chars over [0-9A-Za-z] (nanoid `generateSlugId`).
*
* The two are disjoint (a UUID is 36 chars WITH dashes, a slugId 10 chars
* WITHOUT), so a value can only satisfy one branch.
*/
export const PAGE_ID_OR_SLUG_ID_REGEX =
/^(?:[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}|[0-9A-Za-z]{10})$/i;
/**
* Validate that a DTO string field is a well-formed page identity (page UUID OR
* 10-char slugId). Composed decorator so the same format rule is applied
* consistently wherever a DTO accepts a `pageId` that may be either form.
*/
export function IsPageIdOrSlugId(
validationOptions?: ValidationOptions,
): PropertyDecorator {
return applyDecorators(
Matches(PAGE_ID_OR_SLUG_ID_REGEX, {
message:
'pageId must be a page UUID or a 10-character slugId',
...validationOptions,
}),
);
}
@@ -9,10 +9,16 @@ import {
import { Transform } from 'class-transformer';
import { ContentFormat } from './create-page.dto';
import { IsPageIdOrSlugId } from './page-identity.validator';
export class PageIdDto {
@IsString()
@IsNotEmpty()
// Format-validate the double identity (#435): accept only a page UUID or a
// 10-char slugId so a malformed / swapped identity is rejected at the boundary
// rather than passed to the repo as a bare string. Base for PageInfoDto,
// DeletePageDto, BacklinksListDto, AddLabelsDto/RemoveLabelDto, etc.
@IsPageIdOrSlugId()
pageId: string;
}
@@ -53,8 +53,10 @@ import {
extractPageSlugId,
} from '../../../integrations/export/utils';
import { canonicalizeFootnotes } from '@docmost/editor-ext';
import { markdownToProseMirror } from '@docmost/prosemirror-markdown';
import { normalizeForeignMarkdown } from '../../../integrations/import/utils/foreign-markdown';
import {
markdownToProseMirror,
normalizeForeignMarkdown,
} from '@docmost/prosemirror-markdown';
import { WatcherService } from '../../watcher/watcher.service';
import { sql } from 'kysely';
import { TransclusionService } from '../transclusion/transclusion.service';
@@ -1,19 +1,38 @@
import { TemporaryNoteCleanupService } from '../temporary-note-cleanup.service';
/**
* Chainable Kysely stub that records every `.where(...)` call so the test can
* assert the sweep only selects armed, expired, not-yet-trashed notes. The
* terminal `.execute()` resolves the configured expired rows (the batch SELECT);
* `.executeTakeFirst()` resolves the per-row deadline re-read done just before
* each `removePage`. By default the re-read reports the note as still armed and
* still expired (epoch deadline < now), so the sweep proceeds to delete it;
* tests override `reReadFirst` to simulate a concurrent "Make permanent".
* Chainable Kysely stub for the temporary-note sweep.
*
* `this.db` serves the non-locking candidate SELECT (selectFrom/select/where/
* limit/execute -> the configured expired rows) AND `.transaction().execute(cb)`,
* which runs `cb` with a separate `trx` builder. The `trx` builder serves the
* per-row LOCKED re-check (selectFrom/select/where/forUpdate/skipLocked/
* executeTakeFirst). `lockedRows` drives what that locked re-check returns per
* candidate an id/creator/workspace row means "still expired, delete it";
* `undefined` means the predicate no longer matched (made permanent / re-armed /
* already trashed) or the row was SKIP-LOCKED by another worker, so it is skipped.
*/
function makeDbStub(expiredRows: any[]) {
function makeDbStub(expiredRows: any[], lockedRows?: any[]) {
const whereCalls: any[][] = [];
const reReadFirst = jest
.fn()
.mockResolvedValue({ temporaryExpiresAt: new Date(0), deletedAt: null });
const locked = [
...(lockedRows ??
expiredRows.map((r) => ({
id: r.id,
creatorId: r.creatorId,
workspaceId: r.workspaceId,
}))),
];
const lockedTakeFirst = jest.fn(() => Promise.resolve(locked.shift()));
const forUpdate = jest.fn(() => trxBuilder);
const skipLocked = jest.fn(() => trxBuilder);
const trxBuilder: any = {
selectFrom: jest.fn(() => trxBuilder),
select: jest.fn(() => trxBuilder),
where: jest.fn(() => trxBuilder),
forUpdate,
skipLocked,
executeTakeFirst: lockedTakeFirst,
};
const builder: any = {
selectFrom: jest.fn(() => builder),
select: jest.fn(() => builder),
@@ -23,9 +42,11 @@ function makeDbStub(expiredRows: any[]) {
}),
limit: jest.fn(() => builder),
execute: jest.fn().mockResolvedValue(expiredRows),
executeTakeFirst: reReadFirst,
transaction: jest.fn(() => ({
execute: (cb: (trx: any) => Promise<any>) => Promise.resolve(cb(trxBuilder)),
})),
};
return { builder, whereCalls, reReadFirst };
return { builder, whereCalls, lockedTakeFirst, forUpdate, skipLocked };
}
describe('TemporaryNoteCleanupService.sweepExpiredTemporaryNotes', () => {
@@ -52,20 +73,36 @@ describe('TemporaryNoteCleanupService.sweepExpiredTemporaryNotes', () => {
expect(builder.limit.mock.calls[0][0]).toBeGreaterThan(0);
});
it('soft-deletes each expired note via removePage, attributed to its creator', async () => {
it('soft-deletes each expired note via removePage under a row lock, attributed to its creator', async () => {
const expired = [
{ id: 'p1', creatorId: 'u1', workspaceId: 'w1' },
{ id: 'p2', creatorId: 'u2', workspaceId: 'w1' },
];
const { builder } = makeDbStub(expired);
const { builder, forUpdate, skipLocked } = makeDbStub(expired);
const pageRepo = { removePage: jest.fn().mockResolvedValue(undefined) } as any;
const service = new TemporaryNoteCleanupService(builder, pageRepo);
await service.sweepExpiredTemporaryNotes();
expect(pageRepo.removePage).toHaveBeenCalledTimes(2);
expect(pageRepo.removePage).toHaveBeenNthCalledWith(1, 'p1', 'u1', 'w1');
expect(pageRepo.removePage).toHaveBeenNthCalledWith(2, 'p2', 'u2', 'w1');
// The 4th arg is the locking transaction — the delete runs inside it.
expect(pageRepo.removePage).toHaveBeenNthCalledWith(
1,
'p1',
'u1',
'w1',
expect.anything(),
);
expect(pageRepo.removePage).toHaveBeenNthCalledWith(
2,
'p2',
'u2',
'w1',
expect.anything(),
);
// The re-check acquired a FOR UPDATE SKIP LOCKED lock (once per candidate).
expect(forUpdate).toHaveBeenCalledTimes(2);
expect(skipLocked).toHaveBeenCalledTimes(2);
});
it('continues past a failing note (one bad removePage does not abort the sweep)', async () => {
@@ -86,60 +123,29 @@ describe('TemporaryNoteCleanupService.sweepExpiredTemporaryNotes', () => {
service.sweepExpiredTemporaryNotes(),
).resolves.toBeUndefined();
expect(pageRepo.removePage).toHaveBeenCalledTimes(2);
expect(pageRepo.removePage).toHaveBeenNthCalledWith(2, 'good', 'u2', 'w1');
expect(pageRepo.removePage).toHaveBeenNthCalledWith(
2,
'good',
'u2',
'w1',
expect.anything(),
);
});
it('does NOT trash a note made permanent in the race window', async () => {
// The batch SELECT saw the note as expired, but before its turn in the loop
// the user clicked "Make permanent" (temporary_expires_at -> null). The
// deadline re-read must catch this and skip the delete so the keep wins.
it('does NOT trash a note made permanent / re-armed / already trashed (locked re-check returns nothing)', async () => {
// The batch SELECT saw the note as expired, but by the time the LOCKED
// re-check runs the row no longer matches the still-armed+expired+not-trashed
// predicate (make-permanent, re-arm to a future deadline, or already trashed),
// OR another worker holds the row (SKIP LOCKED). In every case the locked
// SELECT returns nothing and the delete is skipped so the keep/other worker wins.
const expired = [{ id: 'p1', creatorId: 'u1', workspaceId: 'w1' }];
const { builder, reReadFirst } = makeDbStub(expired);
reReadFirst.mockResolvedValueOnce({
temporaryExpiresAt: null,
deletedAt: null,
});
const { builder, lockedTakeFirst } = makeDbStub(expired, [undefined]);
const pageRepo = { removePage: jest.fn() } as any;
const service = new TemporaryNoteCleanupService(builder, pageRepo);
await service.sweepExpiredTemporaryNotes();
expect(reReadFirst).toHaveBeenCalledTimes(1);
expect(pageRepo.removePage).not.toHaveBeenCalled();
});
it('skips a note already trashed since the batch SELECT', async () => {
const expired = [{ id: 'p1', creatorId: 'u1', workspaceId: 'w1' }];
const { builder, reReadFirst } = makeDbStub(expired);
reReadFirst.mockResolvedValueOnce({
temporaryExpiresAt: new Date(0),
deletedAt: new Date(),
});
const pageRepo = { removePage: jest.fn() } as any;
const service = new TemporaryNoteCleanupService(builder, pageRepo);
await service.sweepExpiredTemporaryNotes();
expect(pageRepo.removePage).not.toHaveBeenCalled();
});
it('does NOT trash a note re-armed to a future deadline in the race window', async () => {
// The batch SELECT saw the note as expired, but before its turn in the loop
// the user disarmed it and re-armed it to a fresh, still-future deadline
// (temporary_expires_at -> now + 1h). The deadline re-read must catch that
// the note is no longer expired and skip the delete so the keep wins.
const expired = [{ id: 'p1', creatorId: 'u1', workspaceId: 'w1' }];
const { builder, reReadFirst } = makeDbStub(expired);
reReadFirst.mockResolvedValueOnce({
temporaryExpiresAt: new Date(Date.now() + 60 * 60 * 1000),
deletedAt: null,
});
const pageRepo = { removePage: jest.fn() } as any;
const service = new TemporaryNoteCleanupService(builder, pageRepo);
await service.sweepExpiredTemporaryNotes();
expect(reReadFirst).toHaveBeenCalledTimes(1);
expect(lockedTakeFirst).toHaveBeenCalledTimes(1);
expect(pageRepo.removePage).not.toHaveBeenCalled();
});
@@ -151,4 +157,31 @@ describe('TemporaryNoteCleanupService.sweepExpiredTemporaryNotes', () => {
await service.sweepExpiredTemporaryNotes();
expect(pageRepo.removePage).not.toHaveBeenCalled();
});
it('sweeps once on application bootstrap (catches notes expired during downtime)', async () => {
const expired = [{ id: 'p1', creatorId: 'u1', workspaceId: 'w1' }];
const { builder } = makeDbStub(expired);
const pageRepo = { removePage: jest.fn().mockResolvedValue(undefined) } as any;
const service = new TemporaryNoteCleanupService(builder, pageRepo);
await service.onApplicationBootstrap();
expect(pageRepo.removePage).toHaveBeenCalledTimes(1);
expect(pageRepo.removePage).toHaveBeenCalledWith(
'p1',
'u1',
'w1',
expect.anything(),
);
});
it('a startup-sweep failure never blocks application boot', async () => {
const { builder } = makeDbStub([]);
// Make the candidate SELECT throw to simulate a boot-time DB hiccup.
builder.execute.mockRejectedValueOnce(new Error('db not ready'));
const pageRepo = { removePage: jest.fn() } as any;
const service = new TemporaryNoteCleanupService(builder, pageRepo);
await expect(service.onApplicationBootstrap()).resolves.toBeUndefined();
});
});
@@ -1,8 +1,13 @@
import { Injectable, Logger } from '@nestjs/common';
import {
Injectable,
Logger,
OnApplicationBootstrap,
} from '@nestjs/common';
import { Interval } from '@nestjs/schedule';
import { InjectKysely } from 'nestjs-kysely';
import { KyselyDB } from '@docmost/db/types/kysely.types';
import { PageRepo } from '@docmost/db/repos/page/page.repo';
import { executeTx } from '@docmost/db/utils';
/**
* Background sweeper for temporary notes ("structure or die"). A note whose
@@ -11,7 +16,7 @@ import { PageRepo } from '@docmost/db/repos/page/page.repo';
* TrashCleanupService; `@nestjs/schedule` is already enabled globally.
*/
@Injectable()
export class TemporaryNoteCleanupService {
export class TemporaryNoteCleanupService implements OnApplicationBootstrap {
private readonly logger = new Logger(TemporaryNoteCleanupService.name);
// Cap a single sweep so a large backlog (e.g. many notes created during
@@ -24,6 +29,20 @@ export class TemporaryNoteCleanupService {
private readonly pageRepo: PageRepo,
) {}
// Sweep once at startup so notes that expired during downtime are trashed
// right away instead of waiting up to an hour for the first @Interval tick.
// Best-effort: never let a startup-sweep failure block application boot.
async onApplicationBootstrap() {
try {
await this.sweepExpiredTemporaryNotes();
} catch (error) {
this.logger.error(
'Temporary-note startup sweep failed',
error instanceof Error ? error.stack : undefined,
);
}
}
// Hourly granularity: lifetimes are configured in hours, so a sub-hour
// overshoot past the deadline is acceptable.
@Interval('temporary-note-cleanup', 60 * 60 * 1000)
@@ -31,9 +50,11 @@ export class TemporaryNoteCleanupService {
try {
const now = new Date();
// Candidate ids (non-locking). The authoritative re-check happens per row
// under a row lock below, so this cheap pass just bounds the batch.
const expired = await this.db
.selectFrom('pages')
.select(['id', 'creatorId', 'workspaceId'])
.select(['id'])
.where('temporaryExpiresAt', 'is not', null)
.where('temporaryExpiresAt', '<', now)
.where('deletedAt', 'is', null) // not already in trash
@@ -41,50 +62,53 @@ export class TemporaryNoteCleanupService {
.execute();
let trashed = 0;
for (const page of expired) {
for (const candidate of expired) {
try {
// Re-check the deadline at deletion time. The SELECT above is not
// transactional, so a user may click "Make permanent"
// (toggleTemporary sets temporary_expires_at = null) in the window
// between the SELECT and this per-row removePage. removePage deletes
// by id with only a `deletedAt IS NULL` filter and never re-reads the
// deadline, so without this guard a concurrently-kept note would
// still be trashed. Re-read the row and skip it unless it is still
// armed AND still expired, so a concurrent make-permanent wins.
const current = await this.db
.selectFrom('pages')
.select(['temporaryExpiresAt', 'deletedAt'])
.where('id', '=', page.id)
.executeTakeFirst();
const didTrash = await executeTx(this.db, async (trx) => {
// Re-check the row UNDER A LOCK inside the transaction. `FOR UPDATE
// SKIP LOCKED`:
// - serialises against a concurrent "Make permanent"
// (toggleTemporary UPDATE takes the same row lock): if it commits
// first, the deadline predicate below no longer matches and we
// skip; if we lock first, it waits until this delete commits.
// - SKIP LOCKED lets a second worker/instance skip a row another
// sweeper already claimed instead of blocking on it (no double
// processing, no thundering herd).
// The predicate re-asserts still-armed AND still-expired AND
// not-already-trashed, so a make-permanent / prior sweep drops the row.
const locked = await trx
.selectFrom('pages')
.select(['id', 'creatorId', 'workspaceId'])
.where('id', '=', candidate.id)
.where('temporaryExpiresAt', 'is not', null)
.where('temporaryExpiresAt', '<', now)
.where('deletedAt', 'is', null)
.forUpdate()
.skipLocked()
.executeTakeFirst();
if (
!current ||
current.deletedAt !== null ||
current.temporaryExpiresAt === null ||
new Date(current.temporaryExpiresAt) >= now
) {
// Made permanent, already trashed, or no longer expired since the
// SELECT — leave it alone.
continue;
}
if (!locked) return false;
// Reuse the exact soft-delete path: recursive over children, removes
// shares in a transaction, and emits PAGE_SOFT_DELETED (tree
// invalidation + watcher notifications). Attribute the automatic
// deletion to the note's creator (no schema change). Both the SELECT
// above and removePage filter `deletedAt IS NULL`, so a double sweep
// is idempotent.
await this.pageRepo.removePage(
page.id,
// creatorId is set on every created page; a temporary note always
// has one. Cast to satisfy the non-null deletedById parameter.
page.creatorId as string,
page.workspaceId,
);
trashed++;
// Reuse the exact soft-delete path (recursive children + share
// removal + PAGE_SOFT_DELETED broadcast), running IN this locked
// transaction so the delete is atomic with the re-check and cannot
// deadlock on a nested independent transaction. The broadcast is
// deferred by removePage to this transaction's commit. Attribute the
// automatic deletion to the note's creator (no schema change).
await this.pageRepo.removePage(
locked.id,
// creatorId is set on every created page; a temporary note always
// has one. Cast to satisfy the non-null deletedById parameter.
locked.creatorId as string,
locked.workspaceId,
trx,
);
return true;
});
if (didTrash) trashed++;
} catch (error) {
this.logger.error(
`Failed to trash expired temporary note ${page.id}`,
`Failed to trash expired temporary note ${candidate.id}`,
error instanceof Error ? error.stack : undefined,
);
}
@@ -133,6 +133,9 @@ describe('ShareAliasController authz gates', () => {
creatorId: 'u-1',
alias: 'promo',
confirmReassign: true,
// The requesting user is forwarded so setAlias can gate the reassign
// 409 title disclosure on target-page view permission (#495).
user,
});
expect(result).toEqual({ id: 'alias-1' });
});
@@ -79,6 +79,9 @@ export class ShareAliasController {
creatorId: user.id,
alias: dto.alias,
confirmReassign: dto.confirmReassign,
// Gates whether the reassign 409 may reveal the current target's title
// (view-permission check on that page) — see setAlias (#495).
user,
});
}
@@ -1,4 +1,8 @@
import { BadRequestException, ConflictException } from '@nestjs/common';
import {
BadRequestException,
ConflictException,
ForbiddenException,
} from '@nestjs/common';
import { NoResultError } from 'kysely';
import { ShareAliasService } from './share-alias.service';
@@ -7,6 +11,8 @@ import { ShareAliasService } from './share-alias.service';
* 409 reassign guard, uniqueness-race handling, availability probe, and the
* request-time readable-target resolution (which re-runs the share boundary).
*/
const USER = { id: 'u-1' } as any;
describe('ShareAliasService', () => {
// Sentinel handed to repo calls so tests can assert they ran inside the tx.
const trx = { __trx: true };
@@ -27,6 +33,10 @@ describe('ShareAliasService', () => {
resolveReadableSharePage: jest.fn(),
isSharingAllowed: jest.fn(),
};
// Default: the requester CAN view the target page (validateCanView resolves),
// so the reassign 409 may disclose its title. Tests override to reject to
// assert the no-leak path.
const pageAccessService = { validateCanView: jest.fn().mockResolvedValue(undefined) };
// Fake kysely db: only .transaction().execute(cb) is used by setAlias.
const db = {
transaction: jest.fn(() => ({
@@ -37,9 +47,10 @@ describe('ShareAliasService', () => {
shareAliasRepo as any,
pageRepo as any,
shareService as any,
pageAccessService as any,
db as any,
);
return { service, shareAliasRepo, pageRepo, shareService, db };
return { service, shareAliasRepo, pageRepo, shareService, pageAccessService, db };
}
describe('setAlias', () => {
@@ -50,6 +61,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'A', // too short + uppercase
}),
).rejects.toBeInstanceOf(BadRequestException);
@@ -66,6 +78,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: ' My Page ',
});
@@ -114,6 +127,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'ted',
});
@@ -144,6 +158,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'foo',
});
@@ -179,6 +194,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'new',
});
@@ -190,30 +206,77 @@ describe('ShareAliasService', () => {
);
});
it('throws 409 with current target when name is taken and not confirmed', async () => {
const { service, shareAliasRepo, pageRepo } = makeService();
it('throws 409 with the target TITLE (never its id) when the requester CAN view it', async () => {
const { service, shareAliasRepo, pageRepo, pageAccessService } =
makeService();
shareAliasRepo.findByAliasAndWorkspace.mockResolvedValue({
id: 'a-1',
alias: 'foo',
pageId: 'p-other',
});
pageRepo.findById.mockResolvedValue({ id: 'p-other', title: 'Other' });
pageAccessService.validateCanView.mockResolvedValue(undefined); // can view
try {
await service.setAlias({
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'foo',
});
fail('expected ConflictException');
} catch (err) {
expect(err).toBeInstanceOf(ConflictException);
expect((err as ConflictException).getResponse()).toMatchObject({
const body = (err as ConflictException).getResponse();
expect(body).toMatchObject({
code: 'ALIAS_REASSIGN_REQUIRED',
currentPageId: 'p-other',
currentPageTitle: 'Other',
});
// SECURITY (#495): the page id is NEVER disclosed, even to a viewer.
expect(body).not.toHaveProperty('currentPageId');
expect(pageAccessService.validateCanView).toHaveBeenCalledWith(
expect.objectContaining({ id: 'p-other' }),
USER,
);
}
expect(shareAliasRepo.updatePageId).not.toHaveBeenCalled();
});
it('throws 409 WITHOUT the title or id when the requester CANNOT view the target (#495)', async () => {
const { service, shareAliasRepo, pageRepo, pageAccessService } =
makeService();
shareAliasRepo.findByAliasAndWorkspace.mockResolvedValue({
id: 'a-1',
alias: 'foo',
pageId: 'p-secret',
});
pageRepo.findById.mockResolvedValue({ id: 'p-secret', title: 'Secret' });
// No view permission on the target page -> validateCanView throws.
pageAccessService.validateCanView.mockRejectedValue(
new ForbiddenException(),
);
try {
await service.setAlias({
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'foo',
});
fail('expected ConflictException');
} catch (err) {
expect(err).toBeInstanceOf(ConflictException);
const body = (err as ConflictException).getResponse() as Record<
string,
unknown
>;
expect(body).toMatchObject({ code: 'ALIAS_REASSIGN_REQUIRED' });
// The enumeration hole: neither the id nor the title of a page the
// requester cannot see may leak.
expect(body).not.toHaveProperty('currentPageId');
expect(body.currentPageTitle ?? null).toBeNull();
}
expect(shareAliasRepo.updatePageId).not.toHaveBeenCalled();
});
@@ -231,6 +294,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'foo',
confirmReassign: true,
});
@@ -269,6 +333,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'foo',
});
fail('expected ConflictException');
@@ -294,6 +359,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'foo',
});
fail('expected ConflictException');
@@ -317,6 +383,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'foo',
});
fail('expected ConflictException');
@@ -346,6 +413,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'foo',
});
fail('expected ConflictException');
@@ -375,6 +443,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'foo',
confirmReassign: true,
});
@@ -406,6 +475,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'ted',
});
fail('expected ConflictException');
@@ -428,6 +498,7 @@ describe('ShareAliasService', () => {
workspaceId: 'ws-1',
pageId: 'p-1',
creatorId: 'u-1',
user: USER,
alias: 'foo',
}),
).rejects.toBeInstanceOf(BadRequestException);
@@ -450,18 +521,22 @@ describe('ShareAliasService', () => {
alias: 'free-name',
valid: true,
available: true,
currentPageId: null,
});
// SECURITY (#495): the availability probe must NOT leak any page id.
expect(res).not.toHaveProperty('currentPageId');
});
it('reports taken with the current target page', async () => {
it('reports taken WITHOUT leaking the current target page id (#495)', async () => {
const { service, shareAliasRepo } = makeService();
shareAliasRepo.findByAliasAndWorkspace.mockResolvedValue({
id: 'a-1',
pageId: 'p-9',
});
const res = await service.checkAvailability('taken', 'ws-1');
expect(res).toMatchObject({ available: false, currentPageId: 'p-9' });
expect(res).toMatchObject({ available: false });
// The row exists (available:false) but its pageId is never returned — an
// authenticated member cannot map an alias name to a page id it can't view.
expect(res).not.toHaveProperty('currentPageId');
});
});
@@ -7,7 +7,8 @@ import {
import { ShareAliasRepo } from '@docmost/db/repos/share-alias/share-alias.repo';
import { PageRepo } from '@docmost/db/repos/page/page.repo';
import { ShareService } from './share.service';
import { Page, ShareAlias } from '@docmost/db/types/entity.types';
import { PageAccessService } from '../page/page-access/page-access.service';
import { Page, ShareAlias, User } from '@docmost/db/types/entity.types';
import { isValidShareAlias, normalizeShareAlias } from './share-alias.util';
import { InjectKysely } from 'nestjs-kysely';
import { KyselyDB } from '@docmost/db/types/kysely.types';
@@ -43,6 +44,7 @@ export class ShareAliasService {
private readonly shareAliasRepo: ShareAliasRepo,
private readonly pageRepo: PageRepo,
private readonly shareService: ShareService,
private readonly pageAccessService: PageAccessService,
@InjectKysely() private readonly db: KyselyDB,
) {}
@@ -55,9 +57,13 @@ export class ShareAliasService {
* `/l/<old>` link survives
* - name already points at pageId -> no-op (idempotent)
* - name points at ANOTHER page -> the "swap". Without confirmReassign
* we throw 409 carrying the current target so the client can confirm;
* with it we UPDATE the single row's page_id (every /l/<alias> link
* follows the 302 to the new page instantly no stale cache).
* we throw 409 so the client can confirm. SECURITY (#495): the 409 reveals
* the current target's title ONLY when `user` may VIEW that page, and never
* its id otherwise any member with one editable+shared page could iterate
* alias names with confirmReassign=false and map them to (id, title) of
* pages they cannot see. With confirmReassign we UPDATE the single row's
* page_id (every /l/<alias> link follows the 302 to the new page instantly
* no stale cache).
*
* To keep the invariant self-healing we DELETE every other alias row still
* pointing at this page (a legacy duplicate, or the target page's own former
@@ -77,8 +83,12 @@ export class ShareAliasService {
creatorId: string;
alias: string;
confirmReassign?: boolean;
// The requesting user — used ONLY to gate whether the reassign 409 may reveal
// the current target page's title (view-permission check). Not an authz gate
// for the write itself (the controller already validated edit on `pageId`).
user: User;
}): Promise<ShareAlias> {
const { workspaceId, pageId, creatorId, confirmReassign } = opts;
const { workspaceId, pageId, creatorId, confirmReassign, user } = opts;
const alias = normalizeShareAlias(opts.alias);
if (!isValidShareAlias(alias)) {
throw new BadRequestException(
@@ -97,14 +107,30 @@ export class ShareAliasService {
// The name is occupied by a DIFFERENT (or dangling) target page.
if (byName && byName.pageId !== pageId) {
if (!confirmReassign) {
// SECURITY (#495): only disclose the current target's TITLE, and only
// when the requester may VIEW that page. Never disclose its id (the
// client's confirm-reassign UX doesn't use it, and it is an enumerable
// identity). A member with one editable+shared page must NOT be able to
// iterate alias names and map them to (id, title) of pages they cannot
// see. When view is denied (or the alias is dangling) the 409 is the
// bare "occupied" fact — the client still shows a generic confirm modal.
const currentPage = byName.pageId
? await this.pageRepo.findById(byName.pageId)
: null;
let currentPageTitle: string | null = null;
if (currentPage) {
try {
await this.pageAccessService.validateCanView(currentPage, user);
currentPageTitle = currentPage.title ?? null;
} catch {
// No view permission on the target -> do not reveal its title.
currentPageTitle = null;
}
}
throw new ConflictException({
message: 'Alias already in use',
code: 'ALIAS_REASSIGN_REQUIRED',
currentPageId: byName.pageId,
currentPageTitle: currentPage?.title ?? null,
currentPageTitle,
});
}
// Confirmed swap. ORDER MATTERS: the partial unique index on
@@ -223,21 +249,27 @@ export class ShareAliasService {
alias: string;
valid: boolean;
available: boolean;
currentPageId: string | null;
}> {
const alias = normalizeShareAlias(rawAlias);
if (!isValidShareAlias(alias)) {
return { alias, valid: false, available: false, currentPageId: null };
return { alias, valid: false, available: false };
}
const existing = await this.shareAliasRepo.findByAliasAndWorkspace(
alias,
workspaceId,
);
// SECURITY (#495): return ONLY the boolean availability. The previous shape
// leaked `currentPageId` — the id of whatever page the alias already targets —
// to ANY authenticated workspace member, with no view-permission check on that
// page. An attacker could enumerate alias names and map them to page ids they
// have no access to. The taken/free bit is all the "is this address free"
// probe needs. The reassign flow (setAlias 409) may surface the target's
// TITLE, but only behind a `validateCanView` on that page (see setAlias); it
// never returns the page id.
return {
alias,
valid: true,
available: !existing,
currentPageId: existing?.pageId ?? null,
};
}
@@ -24,6 +24,54 @@ export const ALLOWED_RATINGS = new Set<string>([
'poor',
]);
// The ONLY route labels accepted. The endpoint is anonymous, so an un-checked
// `route` is a free-text write surface (arbitrary high-cardinality strings /
// injected text into the metrics table). The client only ever sends a label from
// a finite template dictionary (`templateRoute`), so we drop anything not in it.
//
// PARITY: this MUST mirror `KNOWN_ROUTE_TEMPLATES` in the client's
// `apps/client/src/lib/telemetry/route-template.ts` (the canonical source). A
// drift means legit client routes get dropped — keep the two in lockstep; the
// client self-consistency test asserts `templateRoute` only emits these values.
export const ALLOWED_ROUTE_TEMPLATES = new Set<string>([
'/',
'other',
// Static routes.
'/home',
'/spaces',
'/favorites',
'/login',
'/forgot-password',
'/password-reset',
'/setup/register',
'/settings/account/profile',
'/settings/account/preferences',
'/settings/workspace',
'/settings/ai',
'/settings/members',
'/settings/groups',
'/settings/spaces',
'/settings/sharing',
// Dynamic templates (slugs/ids are already collapsed to `:param`).
'/share/:shareId/p/:slug',
'/share/p/:slug',
'/share/:shareId',
'/p/:slug',
'/s/:space/p/:slug',
'/s/:space/trash',
'/s/:space',
'/labels/:label',
'/invites/:invitationId',
'/settings/groups/:groupId',
]);
// `attr` is a web-vitals attribution TARGET: a CSS-selector-ish string (an
// element path like `html>body>div#app>button.cta`), never free prose. Constrain
// it to a conservative CSS-selector charset so the anonymous endpoint cannot be
// used to write arbitrary text / PII / markup into the metrics table. A value
// containing anything outside this set is DROPPED (-> null); the event is kept.
export const ATTR_ALLOWED_CHARSET = /^[A-Za-z0-9#.\-_> :()[\]="'*+~,]+$/;
// Max events accepted per batch; the rest are ignored.
export const MAX_EVENTS_PER_BATCH = 50;
@@ -77,14 +125,20 @@ export function sanitizeVitalEvent(
? e.rating
: null;
// route: accept ONLY a known template label (dictionary check), else drop to
// null. The length cap stays as a cheap pre-guard before the Set lookup.
let route: string | null = null;
if (typeof e.route === 'string' && e.route.length > 0) {
route = e.route.slice(0, MAX_ROUTE_LENGTH);
const candidate = e.route.slice(0, MAX_ROUTE_LENGTH);
route = ALLOWED_ROUTE_TEMPLATES.has(candidate) ? candidate : null;
}
// attr: truncate, then accept ONLY if it is a CSS-selector-shaped string
// (charset whitelist); anything with characters outside the set is dropped.
let attr: string | null = null;
if (typeof e.attr === 'string' && e.attr.length > 0) {
attr = e.attr.slice(0, MAX_ATTR_LENGTH);
const candidate = e.attr.slice(0, MAX_ATTR_LENGTH);
attr = ATTR_ALLOWED_CHARSET.test(candidate) ? candidate : null;
}
let docSize: number | null = null;
@@ -90,6 +90,44 @@ describe('VitalsService.buildRows', () => {
expect(rows[0].attr).toHaveLength(MAX_ATTR_LENGTH);
});
it('keeps a known route template but DROPS an unknown/free-text route (#495)', () => {
const rows = svc.buildRows(
{
events: [
{ name: 'INP', value: 1, route: '/s/:space/p/:slug' }, // known
{ name: 'INP', value: 2, route: '/s/acme-corp/p/secret-slug' }, // raw path (slugs) — not a template
{ name: 'INP', value: 3, route: 'DROP TABLE client_metrics;--' }, // injected free text
{ name: 'INP', value: 4, route: '/home' }, // known static
],
},
WS,
);
expect(rows.map((r) => r.route)).toEqual([
'/s/:space/p/:slug',
null, // raw path dropped
null, // free text dropped
'/home',
]);
});
it('DROPS an attr that is not a CSS-selector-shaped string (#495)', () => {
const rows = svc.buildRows(
{
events: [
{ name: 'INP', value: 1, attr: 'div#app>button.cta' }, // valid selector
{ name: 'INP', value: 2, attr: 'user@example.com wrote a note' }, // free text / PII
{ name: 'INP', value: 3, attr: '<script>alert(1)</script>' }, // markup
],
},
WS,
);
expect(rows.map((r) => r.attr)).toEqual([
'div#app>button.cta',
null,
null,
]);
});
it('caps the batch at 50 events', () => {
const events = Array.from({ length: 200 }, () => ({ name: 'CLS', value: 1 }));
const rows = svc.buildRows({ events }, WS);
@@ -0,0 +1,114 @@
import * as path from 'path';
import { readFileSync } from 'fs';
// Mock ONLY kysely's `sql.raw(...).execute()` so we can observe what
// ensureConcurrentIndexes runs and how it handles failures, without a DB.
const execMock = jest.fn((_db: unknown) => Promise.resolve(undefined));
const rawMock = jest.fn((_stmt: string) => ({ execute: execMock }));
jest.mock('kysely', () => {
const actual = jest.requireActual('kysely');
return { ...actual, sql: { ...actual.sql, raw: rawMock } };
});
import { CONCURRENT_INDEXES, ensureConcurrentIndexes } from './concurrent-indexes';
describe('ensureConcurrentIndexes', () => {
const fakeDb = { __topLevelKysely: true } as never;
beforeEach(() => {
execMock.mockReset().mockResolvedValue(undefined);
rawMock.mockClear();
});
it('redefines the inlinable f_unaccent BEFORE building any index, then builds each CONCURRENTLY outside a transaction', async () => {
const onLog = jest.fn();
await ensureConcurrentIndexes(fakeDb, onLog);
// One f_unaccent redefine + one statement per index.
expect(rawMock).toHaveBeenCalledTimes(CONCURRENT_INDEXES.length + 1);
const statements = rawMock.mock.calls.map((c) => c[0] as string);
// ORDER: the f_unaccent redefine MUST be first — otherwise an existing
// tenant's old 2-arg f_unaccent makes every CONCURRENTLY build fail.
expect(statements[0]).toContain('CREATE OR REPLACE FUNCTION f_unaccent');
expect(statements[0]).toContain('SELECT public.unaccent($1)');
expect(statements[0]).not.toContain('CREATE INDEX');
// The rest are the CONCURRENTLY index builds.
for (const stmt of statements.slice(1)) {
expect(stmt).toContain('CREATE INDEX CONCURRENTLY');
expect(stmt).toContain('IF NOT EXISTS');
}
// Executed against the top-level db (a transaction would forbid CONCURRENTLY).
for (const call of execMock.mock.calls) {
expect(call[0]).toBe(fakeDb);
}
expect(onLog).toHaveBeenCalledTimes(CONCURRENT_INDEXES.length + 1);
});
it('still builds the indexes when the f_unaccent redefine fails (fresh DB, best-effort)', async () => {
// Redefine (first execute) throws; the index loop must still run.
execMock
.mockRejectedValueOnce(new Error('extension "unaccent" does not exist'))
.mockResolvedValue(undefined);
const onLog = jest.fn();
await expect(
ensureConcurrentIndexes(fakeDb, onLog),
).resolves.toBeUndefined();
// 1 redefine attempt + one per index.
expect(execMock).toHaveBeenCalledTimes(CONCURRENT_INDEXES.length + 1);
const errored = onLog.mock.calls.filter((c) => c[1] !== undefined);
expect(errored).toHaveLength(1);
expect(String(errored[0][1])).toContain('unaccent');
});
it('is best-effort: a failing index does not abort the rest and is reported', async () => {
// Redefine ok; fail the FIRST index; the remaining ones must still be tried.
execMock
.mockResolvedValueOnce(undefined) // f_unaccent redefine
.mockRejectedValueOnce(new Error('relation "pages" does not exist'))
.mockResolvedValue(undefined);
const onLog = jest.fn();
await expect(
ensureConcurrentIndexes(fakeDb, onLog),
).resolves.toBeUndefined();
expect(execMock).toHaveBeenCalledTimes(CONCURRENT_INDEXES.length + 1);
const errored = onLog.mock.calls.filter((c) => c[1] !== undefined);
expect(errored).toHaveLength(1);
expect(String(errored[0][1])).toContain('does not exist');
});
});
// DRIFT GUARD: each CONCURRENT_INDEXES entry pre-builds an index that a plain
// migration ALSO creates with `IF NOT EXISTS`. If the two expressions diverge,
// Postgres would treat them as different indexes and the pre-build would NOT
// make the migration a no-op. Assert the migration files still contain each
// index's functional expression.
describe('CONCURRENT_INDEXES parity with the migrations', () => {
const migrationsDir = path.join(__dirname, 'migrations');
const files = [
'20260705T120000-perf-indexes.ts',
'20260706T120000-search-lookup-trgm.ts',
].map((f) => readFileSync(path.join(migrationsDir, f), 'utf8'));
const allMigrationSrc = files.join('\n');
it.each(CONCURRENT_INDEXES.map((i) => [i.name, i]))(
'migration source still creates %s with the same expression',
(_name, idx) => {
// Extract the `USING gin ((...expr...) gin_trgm_ops)` tail from the
// canonical create and assert the migration source contains it verbatim.
const m = (idx as { create: string }).create.match(
/ON \w+ (USING gin .+)$/,
);
expect(m).not.toBeNull();
const expr = (m as RegExpMatchArray)[1];
expect(allMigrationSrc).toContain(expr);
// And the migration must build it by the SAME index name.
expect(allMigrationSrc).toContain(
`CREATE INDEX IF NOT EXISTS ${(idx as { name: string }).name}`,
);
},
);
});
@@ -0,0 +1,142 @@
import { Kysely, sql } from 'kysely';
/**
* Indexes that MUST be built with `CREATE INDEX CONCURRENTLY` so an auto-deploy
* migration never takes a `SHARE` lock that blocks writes on a hot table
* (`pages`) for the potentially minutes-long GIN trigram build (#495 item 12).
*
* Kysely runs each migration INSIDE a transaction (Postgres has transactional
* DDL), and `CREATE INDEX CONCURRENTLY` cannot run inside a transaction block, so
* these cannot live in an ordinary migration. Instead {@link ensureConcurrentIndexes}
* builds them out-of-band (no transaction) BEFORE the migrator runs; the matching
* migrations keep a plain `CREATE INDEX IF NOT EXISTS` as a backstop, which then
* no-ops because the index already exists. So:
* - existing prod DB, incremental deploy: the pre-build FIRST redefines
* `f_unaccent` to its index-inlinable 1-arg form (see below), THEN builds each
* index CONCURRENTLY (no write lock); the migration's IF NOT EXISTS no-ops
* the write-blocking build is gone;
* - fresh DB (or a DB whose `pages` / `unaccent` extension does not exist yet):
* the pre-build fails and is swallowed (best-effort), and the migration builds
* the index normally on an empty/small table where the lock is irrelevant.
*
* NOT a strict "worst case = previous behaviour":
* - The f_unaccent form matters. The trigram expressions use `LOWER(f_unaccent(col))`.
* The INDEX-INLINABLE 1-arg `f_unaccent(text)` (`SELECT public.unaccent($1)`)
* is created INSIDE migration 20260705, which runs AFTER this pre-build. On an
* existing tenant the live `f_unaccent` is still the OLD 2-arg form from
* 20250729, which Postgres CANNOT inline into a CONCURRENTLY index expression
* (`function unaccent(unknown, text) does not exist ... during inlining`) the
* build fails outright. So this pre-build redefines `f_unaccent` to the 1-arg
* form FIRST (idempotent, output-identical, in lockstep with 20260705). Without
* that redefine the whole feature is dead-on-arrival for the very case it
* targets.
* - An INTERRUPTED `CREATE INDEX CONCURRENTLY` (killed pod, cancelled query) with
* a working `f_unaccent` leaves an INVALID index behind. A subsequent name-based
* `IF NOT EXISTS` in BOTH this pre-build and the migration backstop sees the
* name and skips, so the invalid index is NEVER repaired automatically and the
* query keeps seq-scanning until an operator `DROP INDEX`es it. The old
* in-transaction build could not leave an invalid index (a failed tx rolled the
* whole index back), so this is a genuinely new failure mode, not "= previous".
* (A future hardening could `DROP` an `indisvalid = false` index before
* rebuilding; not done here.)
*
* The `create` text is the CANONICAL definition it MUST match the migration's
* `IF NOT EXISTS` create expression exactly (same functional expression + opclass)
* or Postgres would treat them as two different indexes.
*/
export const CONCURRENT_INDEXES: ReadonlyArray<{
name: string;
create: string;
}> = [
{
// #348 perf-indexes — pages.title trigram (coalesce-free functional expr).
name: 'idx_pages_title_trgm',
create:
'CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_pages_title_trgm ' +
'ON pages USING gin ((LOWER(f_unaccent(title))) gin_trgm_ops)',
},
{
// #348 perf-indexes — users.name trigram (member search-suggest).
name: 'idx_users_name_trgm',
create:
'CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_users_name_trgm ' +
'ON users USING gin ((LOWER(f_unaccent(name))) gin_trgm_ops)',
},
{
// #443 search-lookup-trgm — pages.text_content trigram (the slow, large one).
name: 'idx_pages_text_content_trgm',
create:
'CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_pages_text_content_trgm ' +
'ON pages USING gin ((LOWER(f_unaccent(text_content))) gin_trgm_ops)',
},
];
/**
* Idempotent, output-identical redefinition of `f_unaccent` to the INDEX-INLINABLE
* 1-arg form. MUST stay byte-for-byte in lockstep with migration
* `20260705T120000-perf-indexes.ts` (same signature + body): the trigram indexes
* above only build CONCURRENTLY once `f_unaccent(text)` inlines, and on an existing
* tenant it is still the old 2-arg form until that migration runs which is AFTER
* this pre-build.
*/
const F_UNACCENT_REDEF = `
CREATE OR REPLACE FUNCTION f_unaccent(text)
RETURNS text
LANGUAGE sql
IMMUTABLE PARALLEL SAFE STRICT
AS $func$
SELECT public.unaccent($1);
$func$
`;
/**
* Best-effort, non-transactional pre-build of {@link CONCURRENT_INDEXES}. Run
* BEFORE the migrator so the blocking `CREATE INDEX` in the corresponding
* migration becomes an `IF NOT EXISTS` no-op.
*
* `db` MUST be the top-level Kysely instance (NOT a transaction): each statement
* then executes on its own connection with no surrounding `BEGIN`, which is
* required for `CONCURRENTLY`. Every statement is independent and swallowed on
* error: the migration backstop still builds the index, so a failure here is
* never fatal. `onLog` reports progress/failures for the caller to route to its
* logger.
*
* ORDER MATTERS: redefine `f_unaccent` to the inlinable form BEFORE the index
* loop otherwise an existing tenant's old 2-arg `f_unaccent` makes every
* CONCURRENTLY build fail and the feature is a no-op (see the module docstring).
*/
export async function ensureConcurrentIndexes(
db: Kysely<any>,
onLog?: (message: string, error?: unknown) => void,
): Promise<void> {
// Make f_unaccent inlinable FIRST. On a fresh DB (no `unaccent` extension yet)
// this throws harmlessly and is swallowed — the migration builds everything.
try {
await sql.raw(F_UNACCENT_REDEF).execute(db);
onLog?.('f_unaccent redefined to the index-inlinable 1-arg form');
} catch (error) {
onLog?.(
'f_unaccent redefine skipped (fresh DB / no unaccent extension) — ' +
'the migration will define it and build the indexes',
error,
);
}
for (const idx of CONCURRENT_INDEXES) {
try {
await sql.raw(idx.create).execute(db);
onLog?.(`Concurrent index ensured: ${idx.name}`);
} catch (error) {
// Non-fatal by design — the migration's IF NOT EXISTS create is the
// backstop. Benign on a fresh DB (the table/extension does not exist yet).
// NOT benign, but still swallowed, on an existing tenant whose f_unaccent
// could not be redefined above (then the migration builds the index NON-
// concurrently under the SHARE lock this feature meant to avoid).
onLog?.(
`Concurrent index pre-build skipped for ${idx.name} ` +
`(will fall back to the in-migration build)`,
error,
);
}
}
}
@@ -0,0 +1,113 @@
import { executeTx, registerAfterCommit } from './utils';
import { KyselyDB, KyselyTransaction } from './types/kysely.types';
// Post-commit hook contract (#495 item 13): a side effect registered via
// registerAfterCommit must run ONLY AFTER the owning transaction commits, and a
// hook registered against a passed-through existingTrx must fire at the OUTER
// commit boundary — never inside the inner call. We fake the Kysely transaction
// runner so the ordering is observable without a real DB.
/**
* A minimal db whose `.transaction().execute(cb)` records the commit ORDER: it
* runs `cb(trx)`, pushes 'commit' onto `log` (simulating the real commit that
* happens after the callback resolves), then returns the callback's result.
*/
function fakeDb(log: string[]): { db: KyselyDB; trx: KyselyTransaction } {
const trx = { __fakeTrx: true } as unknown as KyselyTransaction;
const db = {
transaction: () => ({
execute: async (cb: (t: KyselyTransaction) => Promise<unknown>) => {
const result = await cb(trx);
log.push('commit');
return result;
},
}),
} as unknown as KyselyDB;
return { db, trx };
}
describe('executeTx post-commit hooks', () => {
it('runs an afterCommit hook only AFTER the transaction commits', async () => {
const log: string[] = [];
const { db } = fakeDb(log);
await executeTx(db, async (trx) => {
log.push('body');
registerAfterCommit(trx, () => {
log.push('hook');
});
// The hook must NOT have run yet — the tx is still open.
expect(log).toEqual(['body']);
});
// Order proves post-commit: body → commit → hook (never body → hook → commit).
expect(log).toEqual(['body', 'commit', 'hook']);
});
it('drains hooks registered against a passed-through existingTrx at the OUTER commit', async () => {
const log: string[] = [];
const { db, trx: outerTrx } = fakeDb(log);
await executeTx(db, async (outer) => {
// Nested executeTx reuses the outer trx: it must NOT commit or drain now.
await executeTx(
db,
async (inner) => {
registerAfterCommit(inner, () => {
log.push('inner-hook');
});
},
outer,
);
// Still inside the outer tx — the inner hook has not fired.
expect(log).toEqual([]);
});
// The single (outer) commit drains the hook registered on the shared trx.
expect(log).toEqual(['commit', 'inner-hook']);
// Sanity: the trx the hooks were registered against is the outer one.
expect(outerTrx).toBeDefined();
});
it('a hook failure does not reject the already-committed executeTx', async () => {
const log: string[] = [];
const { db } = fakeDb(log);
await expect(
executeTx(db, async (trx) => {
registerAfterCommit(trx, () => {
throw new Error('cache del blew up');
});
registerAfterCommit(trx, () => {
log.push('second-hook-still-runs');
});
return 'ok';
}),
).resolves.toBe('ok');
// The throwing hook is swallowed; a later hook still runs.
expect(log).toEqual(['commit', 'second-hook-still-runs']);
});
it('does NOT run afterCommit hooks when the transaction body throws (rollback)', async () => {
// The body rejects -> the fake transaction never pushes 'commit' and
// db.transaction().execute() rejects, mirroring a real rolled-back tx. The
// drain runs only AFTER the awaited (committed) transaction, so a rollback
// must leave every registered hook UN-run — otherwise a cache-bust / event
// would fire for a write that never landed.
const log: string[] = [];
const { db } = fakeDb(log);
const hook = jest.fn();
await expect(
executeTx(db, async (trx) => {
registerAfterCommit(trx, hook);
throw new Error('write failed -> rollback');
}),
).rejects.toThrow('write failed -> rollback');
// No commit happened, and the post-commit hook never ran.
expect(log).toEqual([]); // no 'commit'
expect(hook).not.toHaveBeenCalled();
});
});
@@ -35,4 +35,25 @@ describe('jsonbBind', () => {
expect(out).not.toBeNull();
expect(out).toBeDefined();
});
// preserveEmpty (#476): opts a column OUT of the empty-to-null collapse so an
// empty container is persisted verbatim (e.g. `[]` = deny-all for
// tool_allowlist). null stays null regardless of the flag.
describe('preserveEmpty', () => {
it('returns a (non-null) bind for an empty array', () => {
const out = jsonbBind([], { preserveEmpty: true });
expect(out).not.toBeNull();
expect(out).toBeDefined();
});
it('returns a (non-null) bind for an empty object', () => {
const out = jsonbBind({}, { preserveEmpty: true });
expect(out).not.toBeNull();
expect(out).toBeDefined();
});
it('still returns null for null (null means null, flag or not)', () => {
expect(jsonbBind(null, { preserveEmpty: true })).toBeNull();
});
});
});
@@ -33,16 +33,18 @@ import { type Kysely, sql } from 'kysely';
* - comments: `findPageComments` does WHERE page_id ORDER BY id ASC, but only
* `(page_id)` exists extra sort.
*
* DEPLOY-TIME LOCK WARNING: these are plain (non-CONCURRENT) CREATE INDEX
* statements CONCURRENTLY is impossible because Kysely runs each migration in a
* transaction. They take a SHARE lock that BLOCKS writes (INSERT/UPDATE/DELETE) on
* pages/users/groups/comments/page_history for the duration of the build. The two
* GIN trigram builds on pages.title / users.name are the slow ones and can take
* minutes on a large tenant a write-outage window during the deploy migration.
* For large installations, run this migration in a maintenance window, or build
* the trigram indexes out-of-band with CREATE INDEX CONCURRENTLY before deploying
* (then this migration's `IF NOT EXISTS` is a no-op). Small/typical tenants are
* unaffected.
* DEPLOY-TIME LOCK: these are plain (non-CONCURRENT) CREATE INDEX statements
* CONCURRENTLY is impossible HERE because Kysely runs each migration in a
* transaction. The two GIN trigram builds on pages.title / users.name are the
* slow ones and would take a SHARE lock that BLOCKS writes on pages/users for
* minutes on a large tenant. To avoid that, `ensureConcurrentIndexes`
* (database/concurrent-indexes.ts) now pre-builds BOTH trigram indexes with
* CREATE INDEX CONCURRENTLY (no transaction) BEFORE the migrator runs, so on an
* existing DB the two `IF NOT EXISTS` trigram creates below no-op and no write
* lock is taken. On a fresh DB the pre-build is skipped and they build on an
* empty table. Keep the two trigram creates in lockstep with their CANONICAL
* definitions in CONCURRENT_INDEXES. (The plain b-tree indexes further down are
* fast metadata-only builds; they are not pre-built.)
*/
export async function up(db: Kysely<any>): Promise<void> {
// Index-compatible, output-identical redefinition of f_unaccent (see header).
@@ -26,13 +26,16 @@ import { type Kysely, sql } from 'kysely';
* to update than b-trees); on the small instances this fork targets that cost
* is acceptable and the read win on agent lookups is the priority.
*
* DEPLOY-TIME LOCK WARNING: plain (non-CONCURRENT) CREATE INDEX Kysely runs
* each migration in a transaction, so CONCURRENTLY is impossible. The build takes
* a SHARE lock that BLOCKS writes on `pages` for its duration. The text_content
* GIN build is the slow one and can take minutes on a large tenant. For large
* installations, run this in a maintenance window or build the index out-of-band
* with CREATE INDEX CONCURRENTLY before deploying (then `IF NOT EXISTS` no-ops
* here). Small/typical tenants are unaffected.
* DEPLOY-TIME LOCK: this is a plain (non-CONCURRENT) CREATE INDEX Kysely runs
* each migration in a transaction, so CONCURRENTLY is impossible HERE, and the
* build would take a SHARE lock that BLOCKS writes on `pages` for its duration
* (the text_content GIN build can take minutes on a large tenant). To avoid that,
* `ensureConcurrentIndexes` (database/concurrent-indexes.ts) now pre-builds this
* index with CREATE INDEX CONCURRENTLY (no transaction) BEFORE the migrator runs,
* so on an existing DB the `IF NOT EXISTS` below no-ops and no write lock is taken.
* On a fresh DB the pre-build is skipped and this builds it on an empty table
* where the lock is irrelevant. Keep this create in lockstep with the CANONICAL
* definition in CONCURRENT_INDEXES (same expression + opclass).
*/
export async function up(db: Kysely<any>): Promise<void> {
// The title predicate is served by #348's idx_pages_title_trgm — see header.

Some files were not shown because too many files have changed in this diff Show More