Compare commits

..

32 Commits

Author SHA1 Message Date
agent_vscode 16b476a205 fix(ai-chat): bound MCP connect + guard turn setup so a hung handshake can't wedge every run
A transient network blip during an external-MCP handshake left createMCPClient
pending forever (@ai-sdk/mcp does not settle on abort). getOrBuildEntry caches the
per-workspace build PROMISE, so the never-settling connect poisoned the cache and
every later turn hung at step_count=0 before streamText — the run never finalized,
the row stayed 'running', and the chat was permanently blocked with
A_RUN_ALREADY_ACTIVE (an explicit Stop could not interrupt the un-signalled setup).

- mcp-clients: wrap connect() in a settling timeout (connectWithTimeout) that
  closes a late-arriving client, so a hung handshake rejects instead of poisoning
  the build cache; the bad server is skipped and the build completes.
- mcp-clients: close a connected-but-unregistered client when tools() fails,
  fixing a pre-existing transport leak in buildEntry.
- ai-chat.service: bound the toolsFor build with the run's abort signal AND a
  deadline (raceAgainstAbortAndTimeout); re-throw an explicit Stop only when a run
  exists (runId) so the run finalizes as 'aborted', and keep the legacy
  socket-bound path unchanged; settle the run 'aborted' vs 'error' accordingly.
- tests: cache-not-poisoned + orphan-client-close-once + Stop-during-setup
  finalizes the run once.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 23:27:51 +03:00
agent_vscode 834684c37a Merge branch 'develop' of https://gitea.vvzvlad.xyz/vvzvlad/gitmost into develop 2026-07-06 21:43:41 +03:00
vvzvlad 080d1b6051 Merge pull request 'feat(ai-chat): показывать текст запроса/аргументы в карточке вызова инструмента (#392)' (#393) from feat/392-tool-input-summary into develop
Reviewed-on: #393
2026-07-06 21:43:23 +03:00
agent_coder 7f88b0b441 test(ai-chat): pin toolInputSummary field priority + clamp boundary (#392)
Review round: add the two missing test-locks the reviewer asked for —
(1) priority order of PRIMARY_INPUT_FIELDS is now pinned (`{query,title}`
-> "Q", so a reordering breaks the test); (2) the clamp boundary is pinned
exactly (140 chars -> unchanged, no ellipsis; 141 -> 140 + "…"), catching
an off-by-one / `>` vs `>=` regression. Test-only, no production change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:40:03 +03:00
agent_vscode 8f7664eb04 feat(agent-roles): restrict when the researcher reuses the current page
The researcher now reuses the current/already-open document only when the
user explicitly asked for it, OR the page is empty/near-empty AND its title
matches the research topic; otherwise it creates a new document.

- Rewrite the reuse rule in the WHERE TO WRITE THE RESULT section
- Reword "Create this document" -> "Set up this document" so it fits both
  the create-new and reuse-current-empty cases
- Apply identically to bundles/research/ru.yaml and en.yaml
- Bump researcher version 3 -> 4 in index.yaml; refresh content-hashes.json

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:29:03 +03:00
agent_coder cb445f0966 feat(ai-chat): show tool-call arguments in the action-log card (#392)
For tools without a friendly label (esp. external MCP tools like
Search_web_search) the card showed a generic "Ran tool <name>" with no
sense of what was searched. Add a compact one-line, dimmed summary of the
call's arguments under the label, pulled from part.input.

New pure toolInputSummary(part): picks the first present "primary" field
(query/q/searchQuery/url/urls/title/name/text/prompt), collapses
whitespace, clamps to ~140 chars; arrays render "first (+N)". It returns
undefined during input-streaming (the input grows while state is fixed and
messageSignature doesn't track input, so a live summary would freeze) —
the state flip to input-available re-renders the row with the final value,
so message-signature.ts is left untouched. The value renders ONLY through
Mantine <Text> (React-escaped) — no markdown/HTML, no XSS.

A showInput prop (default true) is threaded MessageList -> MessageItem
(+memo) -> ToolCallCard; the public share widget passes showInput={false}
so an anonymous reader never sees the agent's raw query text (mirrors
showCitations). No JSON fallback when no primary field is present.

closes #392

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 21:28:24 +03:00
vvzvlad cafe29b153 Merge pull request 'test(converter): хвост #351 — nightly property cron + README + запиненные баги + media-фазз' (#391) from feat/351-remainder into develop
Reviewed-on: #391
2026-07-06 20:46:59 +03:00
agent_coder 35f2c06f42 test(converter): #391 review round — orderedList start guard + nightly hardening (#351)
DO-1 (regression): the orderedList start hardening `Number(x)||1` let a
negative/fractional start through into the marker ("-3.", "2.5.") — marked
can't tokenize it, so the whole list re-imported as a paragraph (structure
corruption); start=0 churned. Both the markdown and raw-HTML export paths
now use `Number.isInteger(raw) && raw > 1 ? raw : 1`, so any degenerate
start collapses to the default "1." markers / bare <ol> (list always valid).
New ordered-list-start-normalization test pins {0,-3,2.5} → valid start=1
list, byte-stable; the round-trip fuzz keeps to integers >=2 (num 2,3,5,42).

DO-2 (nightly was non-functional): NUM_RUNS=5000 OOM'd the worker and the
crash was misreported as a "counterexample" issue whose prefix-dedup then
locked out all future issues. Reworked to shard 8 fresh vitest processes
(600 runs each, distinct seeds, --max-old-space-size) so deep fuzzing
never OOMs; a failing shard's output is preserved, and issue creation
discriminates a real fast-check counterexample from an infra/OOM failure
(distinct titles + scoped dedup). The two issue steps use `always() &&`
so they actually run on the failure path.

DO-3: envInt extracted to test/generative/env-int.ts + unit-tested.
DO-4: nightly dispatch inputs go through env: (no ${{ }} in run:).
DO-5: attr-arbitraries.ts docblock synced (column.width/orderedList.start
are fixed+fuzzed, not pinned it.fails).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 20:32:29 +03:00
agent_coder ce9f1a8980 test(converter): nightly property cron + env knobs + counterexample README (#351)
Items 1 & 2 of the #351 remainder.

Item 1 — the flat/nested generative property tests now read SEED and
NUM_RUNS from PROPERTY_SEED / PROPERTY_NUM_RUNS (via an envInt helper that
honors an explicit 0 and falls back to the current defaults —
20250705/300 flat, /100 nested — on unset/empty/non-numeric). New
.github/workflows/nightly-property.yml runs the generative suite daily
(and on workflow_dispatch) with a random seed and NUM_RUNS≈5000; on
failure it files a Gitea issue containing fast-check's shrunk
counterexample (jq-escaped, dedup'd by title). No build step — the suite
imports the converter from src/, so a tsc error can't masquerade as a
property failure.

Item 2 — new packages/prosemirror-markdown/README.md documents the
counterexample process (surface -> shrink -> permanent fixture in
test/fixtures/counterexamples/ + counterexamples.test.ts -> fix the
converter, never weaken a property; maintainer-approved ACCEPTED/allowlist
entries carry a reason) plus the two golden layers, the coverage
allowlist, and how to run with the env knobs. Linked from AGENTS.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 19:51:53 +03:00
agent_coder 15fe998d3d test(converter): value-fuzz the deferred media attributes (#351)
Item 4 of the #351 remainder — implement value-fuzz for the media
dimension/family attributes previously parked in the flat property
suite's ATTR_VALUE_FUZZ_ALLOWLIST as "round-trip candidates deferred to
a later PR": width/height/align/aspectRatio/size/caption/title/alt across
image, video, youtube, pdf, drawio, excalidraw, embed.

Each gets a non-default-value arbitrary in attr-arbitraries.ts and is
removed from the allowlist; the P1 (semantic) + P2 (byte-stability)
property gate now exercises them. All round-trip green — the comment-JSON
serialization (stable key order, String()-stringified) carries every one
faithfully, so no new pins or accepted-limitations were needed.

embed.width/height are fuzzed as numeric STRINGS (not numbers): a
non-default embed dimension round-trips through the comment JSON as a
string (the numeric 800/600 default is still omitted/re-materialized),
so authoring a number diverged under P1.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 19:51:53 +03:00
agent_coder 54f0ba681e fix(converter): preserve orderedList start; fuzz column.width numerically (#351)
Item 3 of the #351 remainder — resolve the two pinned converter
counterexamples, fixtures-first.

orderedList.start (genuine P1 loss): the converter always emitted "1."
and dropped attrs.start. Now the markdown path emits `${start + index}.`
(Number-coerced, guards a stray non-numeric start) and the raw-HTML path
emits `<ol start="N">` when start>1; a default (start=1) list is
byte-unchanged. tiptap StarterKit reads both forms back. Un-pinned as a
passing regression test; orderedList.start is now value-fuzzed.

column.width: the "50% churn" counterexample rested on a false premise —
the canonical editor (editor-ext column.ts) stores width as a unitless
flex-grow NUMBER (parseFloat, `flex:${width}`), never a "%" string, and
docmost-schema.ts is a vendored mirror that MUST match it. The original
parseFloat was correct parity and already byte-stable for numeric widths.
Reverted the mirror to parseFloat, deleted the fabricated counterexample
fixture, and now value-fuzz column.width as a number (real type). No src
behaviour change for column.width — parity with editor-ext preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 19:51:53 +03:00
vvzvlad 27f7791a0e Merge pull request 'feat(ai-chat): getCurrentPage отдаёт текущее выделение пользователя (#388)' (#390) from feat/388-getcurrentpage-selection into develop
Reviewed-on: #390
2026-07-06 19:08:26 +03:00
agent_coder 15859e3b9f feat(ai-chat): getCurrentPage returns the user's editor selection (#388)
Snapshot the editor selection at send time (same live-ref pattern as
openPageRef in prepareSendMessagesRequest), carry it nested inside
openPage so it dies with the page on a fail-closed resolve, and surface
it to the model only through the existing core getCurrentPage tool.

The selection TEXT is returned exclusively in the tool result (untrusted
collaborative-page content, treated as data by SAFETY_FRAMEWORK); the
system prompt gets only a fixed one-line flag, never the text/before/
after. sanitizeSelection caps text (4000), before/after (200), blockIds
(<=64 chars, <=20). Selection is a hint, not ground truth — the tool
description tells the agent to localize the fragment before editing.

closes #388

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 18:50:55 +03:00
agent_vscode ebf132d5ac feat(agent-roles): make the researcher's search budget mandatory to spend
A user-provided "research budget" now defines the required search volume:
it is binding and must be spent in full, overriding the default "stop at
saturation" rule.

- STEP 0: split the budget into user-set (binding, must be used up) vs
  self-estimated (when the user gave no number)
- VOLUME: limit "stop at saturation" to the no-budget case; add a
  MANDATORY BUDGET block requiring the full budget be spent on genuine
  broadening/lateral/primary-source/verification searches, not padding
- Apply identically to bundles/research/ru.yaml and en.yaml
- Bump researcher version 2 -> 3 in index.yaml; refresh content-hashes.json

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 18:29:57 +03:00
agent_vscode 05d3456f5c feat(agent-roles): researcher builds the report doc live from the start
Rework the "WHERE TO WRITE THE RESULT" section of the researcher role so
the report document is created at the very beginning of the run (right
after the plan, before any searches) and filled dynamically after each
finding, instead of being dumped in one pass at the end.

- Rewrite the section identically in bundles/research/ru.yaml and en.yaml
- Bump researcher version 1 -> 2 in index.yaml
- Refresh scripts/content-hashes.json via check.mjs --update-hashes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 18:09:40 +03:00
agent_vscode d910180772 docs(readme): reverse-proxy requirements for SSE streaming paths
The AI chat SSE endpoints (POST /api/ai-chat/stream, GET
/api/ai-chat/runs/<chatId>/stream, POST /api/shares/ai/stream) must
bypass response buffering AND compression at every proxy in front of
the app — a compressing proxy silently buffers SSE frames until the
response closes (pending request, tokens arriving in one burst,
reloaded tabs degrading to polling). Document the affected paths, the
DevTools tell (Content-Encoding on text/event-stream), and concrete
nginx/Traefik configuration in both READMEs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 18:02:37 +03:00
agent_vscode 89a4bf28ce test(auth): de-flake verify-user-credentials.live.spec — hoist bcrypt hash + 30s timeout
Under a fully parallel `pnpm -r test` run the suite computed five separate
bcrypt cost-12 hashes (one per test, ~300ms idle, multi-second with all cores
saturated) and tripped jest's default 5s per-test timeout ("DISABLED user"
test, suite 31s under load vs 6s isolated).

- compute the hash ONCE in a top-level beforeAll (covers both describe
  blocks) and share the read-only string across the five former call sites
- jest.setTimeout(30_000) at module scope so the per-test bcrypt compares
  inside verifyUserCredentials get headroom under load too

No change to test names, assertions, order, or the CREDENTIALS_MISMATCH
contract semantics. Suite: 8/8 green, 4.8s isolated (was ~6s).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 17:24:24 +03:00
agent_vscode da94b1589c test(mcp): align nested-list goldens with the #351 loose-container converter fix
The #351 converter fix (79a461f7) made multi-block list items emit a blank-line
separator between block children (loose list) to stop silent content merging on
re-parse. The prosemirror-markdown goldens were updated in that PR, but the two
mcp unit tests asserting the old tight output were missed — packages/mcp
re-exports the shared converter, so they broke the develop CI run (test job,
`pnpm -r test`, 2/480 failures).

- expect "- Parent\n\n  - A..." instead of "- Parent\n  - A..."
- expect "1. Parent\n\n   - Child" instead of "1. Parent\n   - Child"

Full mcp suite: 480/480 green; editor-ext / prosemirror-markdown / client /
git-sync suites green as well.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 17:13:34 +03:00
vvzvlad 40227bbf51 Merge pull request 'feat(ai-chat): resumable SSE — клиент (#381 PR 2, переоткрыт на develop после стек-мёржа)' (#389) from feat/381-resumable-sse-pr2 into develop
Reviewed-on: #389
2026-07-06 16:29:04 +03:00
agent_vscode a26803a1bc docs(env): document AI_CHAT_RESUMABLE_STREAM staged-rollout flag (#381)
The resumable-SSE run-stream registry (PR #386/#387) ships behind the
server-side AI_CHAT_RESUMABLE_STREAM env flag, OFF by default: with the
flag off attach always answers 204 and reopened tabs of an active run
fall back to degraded history polling. Document the flag, its default,
its relation to the per-workspace autonomousRuns setting, and the
single-instance constraint in .env.example next to the autonomous-runs
section.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-06 16:27:06 +03:00
agent_coder 18eee98b7f fix(ai-chat): #381 PR2 review round 1 — F7 restart-survival + unmount abort + anchor-orphan
Do 1 [F7 regression]: транзиентный сбой attach больше не роняет строку и не
теряет ран. В transport fetch-wrapper `204 || !response.ok` оба зовут
onNoActiveStream (восстановить stripped-строку + invalidate + арм poll), и catch
зовёт его перед rethrow — раньше !ok/throw только сбрасывали флаг, и на 5xx/502/
network-blip in-progress ассистент-турн исчезал, durable-ран не отслеживался.
onNoActiveStream — суперсет (его часть-г всё ещё чистит флаг), идемпотентен.
Расширяет литеральный block-3 спеки (там был только сброс флага) — по ревью и
в согласии с интенцией окна «poll must survive a server restart».

Do 2 [stability]: attach-GET абортится при unmount + mount-гейтинг сайд-эффектов.
mountedRef: mount-эффект ре-армит true и в cleanup ставит false + abort
attachAbortRef; onNoActiveStream рано выходит на !mounted, onFinish-recovery
гейтится `wasResumed && mountedRef.current`. Снимает до-10-мин спурьёзный поллинг
+ чужую invalidateQueries + утёкший fetch на новооткрытом чате (и StrictMode
double-resume).

Do 3 [coherence]: anchor-mismatch не оставляет вечную dots-строку. Reconcile
после мержа хвоста мержит fresh-history версию stripped-строки, если её id !=
id хвоста — settl'ит осиротевшую streaming-A над раном B.

Тесты: F7 500 → restore+арм; F7 network-throw → restore+арм; unmount при pending
attach → abort + поздние колбэки не летят. vitest src/features/ai-chat 304
зелёных, grep-guard пуст.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 16:26:27 +03:00
agent_coder 067fc46170 feat(ai-chat): единый resumable SSE-транспорт — клиент + удаление поллинга/латчей (#381 PR 2)
PR 2 из 2 (#381, фаза 1.5 #184). Все вкладки теперь на ОДНОМ транспорте:
любая подключается к рану через GET-attach (реплей кадров + живой хвост,
реестр из PR 1). Наблюдатель — обычный стример; Stop, точки, инвалидации
работают штатно. Двухпутёвый поллинг снапшотов и вся latch-механика F4/F5/F7
удалены.

- utils/resume-helpers.ts (новый): isStreamingTail / isSettledAssistantTail /
  seedRows / mergeById (дословный перенос mergeObservedMessage из run-polling).
- components/chat-thread.tsx: resume-машинерия (гейтинг по не-settled хвосту,
  strip streaming-хвоста + attach ?expect=live&anchor=<row id>, транспорт
  prepareReconnectToStreamRequest+fetch, 204-обработчик из 4 частей,
  reconcile+degraded-merge, recovery с АСИММЕТРИЕЙ arm-vs-restore — при
  isDisconnect с видимым контентом только arm, без restore-клоббера живого
  стрима (инв. 9), строгий порядок onFinish с ранним return до обеих веток
  отправки (инв. 7), «Send now» скрыт на resumed-ходе, Stop абортит attach).
- components/ai-chat-window.tsx: degraded-poll фолбэк вместо латчей — тупой
  таймер (2500ms, 10-мин кап, без проверок ошибок/хвоста; переживает рестарт
  сервера), гасится тредом через onResumeFallback(false).
- Удалено: run-polling.ts(+test), useAiChatRunQuery/AI_CHAT_RUN_RQ_KEY,
  getAiChatRun (stopRun оставлен), IAiChatRun/IAiChatRunResponse, латчи
  stoppingRun/localStreaming/observedRow/onStreamingChange, F7-эффект,
  observer-merge. Серверный POST /ai-chat/run не тронут.

Проверка: tsc (мои файлы чисты), vitest src/features/ai-chat 34 файла/301 тест
зелёные, grep-guard по удалённым символам пуст. Отдельное внутреннее ревью на
инварианты 7/8/9 + 204-null-safety — чисто.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 16:26:27 +03:00
vvzvlad ab1da408e5 Merge pull request 'feat(ai-chat): resumable SSE run-stream registry — сервер, спящий (#381 PR 1)' (#386) from feat/381-resumable-sse-pr1 into develop
Reviewed-on: #386
2026-07-06 16:24:38 +03:00
vvzvlad da7bb95d4f Merge pull request 'test(int): убрать избыточный forceExit — bounded teardown уже расхангивает test:int (#382)' (#383) from fix/382-int-open-handles into develop
Reviewed-on: #383
2026-07-06 16:08:22 +03:00
vvzvlad 6ab2e989b9 Merge pull request 'test(converter): вложенный variant-A генератор + P4 + фикс всех round-trip багов (#351)' (#385) from test/351-nested-generator into develop
Reviewed-on: #385
2026-07-06 16:07:48 +03:00
vvzvlad 169e34d766 Merge pull request 'docs(agents): build shared packages before a consumer's tsc/tests in isolation' (#384) from docs/agents-workspace-build-order into develop
Reviewed-on: #384
2026-07-06 16:06:34 +03:00
agent_coder 10d5220f5e fix(ai-chat): #381 PR1 review round 1 — open()-gate test + paused-pending byte-cap
Do 1 [test-coverage]: контроллерный тест на флаг-гейт begin-hook open() — при
OFF beginRun зовётся (durable-ран независим), а streamRegistry.open НЕ зовётся
(закрывает регресс: пустая entry → non-null paused attach → зависший SSE вместо
204); при ON — open зовётся с (chatId, runId).

Do 2 [stability]: байт-кап очереди pending paused-подписчика. pendingBytes +
overflowed на Subscriber; в paused-ветке ingestFrame при превышении
SUBSCRIBER_MAX_BUFFERED_BYTES (8MB) подписчик помечается overflowed, pending
чистится, он выбрасывается из entry.subscribers (как overflowed-entry). start()
на overflowed → onEnd (чистый 204-эквивалент, без частичного реплея). Контракт
«start() в том же тике, что attach()» задокументирован в коде — кап это
структурный бэкстоп для phase-2 Redis-await шва. Юнит-тест: paused A + live B,
9×1MB > cap → A выброшен (0 доставок), B получает все 9 живьём, поздний start(A)
→ один onEnd без реплея.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:42:36 +03:00
agent_coder 52ee3c1f3e feat(ai-chat): resumable SSE run-stream registry — server, dormant (#381 PR 1)
PR 1 of 2 (#381, фаза 1.5 #184): серверный реестр SSE-стримов агентских ранов,
чтобы любая вкладка могла подключиться к живому рану с реплеем кадров + живым
хвостом. «Спящий» — весь провод за флагом AI_CHAT_RESUMABLE_STREAM (off по
умолчанию); клиент (PR 2) ещё не написан.

- ai-chat-stream-registry.service.ts: in-memory реестр (open/bind/abortEntry/
  attach). attach — снапшот+подписка в ОДНОМ синхронном блоке (инвариант 4: нет
  await между `subscribers.add` и `frames.slice()`), paused-подписчик, overflow,
  retention с identity-guard (инвариант 2), open поверх live entry даёт ровно
  один onEnd (инвариант 3), anchor против кросс-ранового реплея (инвариант 6).
- ai-chat.controller.ts: begin-хук open(chatId, runId) + GET-attach эндпоинт
  (403 чужой чат; 204 нет-entry/finished/anchor-мисматч; cleanup до первой
  записи + recheck req.raw.destroyed; cap→destroy).
- ai-chat.service.ts: tee SSE-кадров в реестр (consumeSseStream + generateMessageId,
  гейт на runId && flag) + abortEntry из внешнего catch.
- environment.service.ts: флаг isAiChatResumableStreamEnabled().

Флаг OFF ⇒ байт-в-байт legacy И #184-фаза-1 (нет start.messageId, нет tee).
Инжектируемые провайдеры НЕ @Optional() → поломка вайринга роняет старт, а не
тихо выключает фичу.

Тесты: registry unit (16), controller.attach (9), service pipe-options (4, вкл.
flag-off-with-runId негатив), int-spec ai-chat-attach (6, реальный MockLanguageModelV3).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 07:22:20 +03:00
agent_coder ccee32cb0b test(git-sync): assert stable drawio markers, not the omittable center default (#351 review)
stabilize.test.ts used `data-align="center"` as proof the convergence pass
materialized the drawio node. Since this PR correctly stops emitting the
schema-default center align in the media builders, that marker is no longer a
reliable convergence proof. Assert on the stable canonical markers instead —
`data-type="drawio"` + `data-src="/d.drawio"` — which are always materialized
regardless of the align default. The fixpoint assertion (file2 === file1) is
unchanged; round-trip stays byte-stable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 06:44:58 +03:00
agent_coder 79a461f79d test(converter): вложенный variant-A генератор + P4-фазз, и фикс всех найденных round-trip багов (#351)
Финальная ступень #351: генератор ЦЕЛЫХ вложенных документов (random walk
по ContentMatch схемы, min-depth fixpoint для терминирования, глубина/размер
ограничены) + инвариант P4 (фазз парсера: для ЛЮБОЙ строки markdownToProseMirror
не бросает и результат валиден по схеме). Инварианты P1 (семантический
round-trip), P2 (байтовый fixpoint со 2-го прохода), P3 (тотальность) — строгие.

Генератор сразу нашёл классы реальных багов конвертера; все починены (инварианты
НЕ ослаблялись — чинился конвертер):

- loose (много-блочные) контейнеры (listItem/taskItem/callout/detailsContent)
  склеивали блоки при реимпорте — ТИХАЯ ПОТЕРЯ ДАННЫХ; теперь blank-line
  разделитель между блок-детьми (по образцу blockquote).
- соседние sibling-списки одного marker-family (task+bullet и т.п.) сливались
  в один список с ПОТЕРЕЙ чекбокса — теперь между ними эмитится инертный
  `<!-- -->` разделитель (byte-stable round-trip).
- paragraph textAlign терялся во вложенных li/td/th.
- вложенный codeBlock терял хвостовой перевод строки.
- pageBreak/pageEmbed/subpages/transclusion дропались во вложении
  (blockquote/callout/details/li).
- медиа в columns: number→string ширины/высоты и лишний data-align (churn).
- callout `> [!type]`, вложенный в список/цитату, парсился неверно
  (prefix-aware regex).

Golden-обновления (6) — прямые следствия loose-container фикса, каждое
round-trip'ится. 2100+ сгенерированных документов (3 seed) — 0 падений P1/P2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 06:29:58 +03:00
claude_code 24946ad820 docs(agents): build shared packages before a consumer's tsc/tests in isolation
Document the TS2307 trap: the gitignored build/ of @docmost/prosemirror-markdown,
@docmost/git-sync and @docmost/mcp is not honoured by a single-package
pnpm --filter <pkg> test/tsc or a bare pnpm -r test (Nx dependsOn ^build is only
applied by nx run-many), so a consumer's typecheck fails with
Cannot find module '@docmost/...' until those packages are built first.
Mirrors the order .github/workflows/test.yml already uses.
2026-07-06 05:50:34 +03:00
agent_coder 84334a1f34 test(int): drop redundant forceExit — bounded teardown already un-hangs test:int (#382)
The int suite could not self-exit after the ESM fix (8e125799) unmasked 4
specs, and was patched with two things: forceExit:true AND a bounded
destroyTestDb (sql.end({ timeout: 5 })). The bounded teardown is the real
fix — postgres.js .end() without a timeout blocks indefinitely on a stuck
pooled connection (the CI-observed "Jest did not exit"); the { timeout: 5 }
grace drains then force-closes sockets so teardown always completes.

forceExit was redundant belt-and-suspenders that also HID whether the
process truly exits on its own. Removing it: every handle-creating spec is
verified to close its handle — ai-chat-stream closes its http.createServer
in a finally, public-share-workspace-limiter closes its ioredis via
redis.quit() in afterAll, and the shared DB pools close via the bounded
destroyTestDb. --detectOpenHandles is clean and the suite self-exits.

Kept: the bounded destroyTestDb (defense for a genuinely stuck connection).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-06 04:07:44 +03:00
94 changed files with 6972 additions and 1730 deletions
+12
View File
@@ -222,6 +222,18 @@ MCP_DOCMOST_PASSWORD=
# CLOUD=true) — run a single instance instead. The server logs a startup WARNING # CLOUD=true) — run a single instance instead. The server logs a startup WARNING
# when it detects a multi-instance deployment (CLOUD=true) so the constraint is # when it detects a multi-instance deployment (CLOUD=true) so the constraint is
# visible, and a startup sweep settles any run left dangling by a restart. # visible, and a startup sweep settles any run left dangling by a restart.
#
# Resumable run streams (#184 phase 1.5, #381). With the flag ON, an active
# durable run tees its SSE frames into an in-memory registry, and a
# reloaded/second tab attaches via GET /ai-chat/runs/:chatId/stream to follow the
# run LIVE (replay of the buffered frames + the live tail). With the flag OFF
# (default) the registry is never populated and attach always answers 204, so a
# reopened tab of an active run silently falls back to degraded 2.5s history
# polling — every wire path stays byte-for-byte identical to a build without the
# feature. Staged-rollout switch: only meaningful when autonomousRuns (above) is
# enabled for a workspace, and the same single-instance constraint applies (the
# registry is process-local).
# AI_CHAT_RESUMABLE_STREAM=false
# --- Anonymous public-share AI assistant --- # --- Anonymous public-share AI assistant ---
# Opt-in per workspace (AI settings -> "public share assistant"; off by default). # Opt-in per workspace (AI settings -> "public share assistant"; off by default).
+224
View File
@@ -0,0 +1,224 @@
name: Nightly property fuzz
# The daily heavy property run for the ProseMirror<->Markdown converter
# (packages/prosemirror-markdown). The PR/CI test run keeps NUM_RUNS modest to
# stay under budget; this cron cranks up total coverage with random seeds to hunt
# for deeper round-trip counterexamples than a fixed-seed PR run can reach.
#
# WHY SHARDING: a single mega-run (~10000 fast-check runs) OOMs the vitest worker
# (empirically ~1625 runs -> "JS heap out of memory", ~2GB) because heap
# accumulates across the whole property run in one process. Instead this job runs
# SHARDS fresh vitest processes, each a MODERATE per-shard count with a DISTINCT
# derived seed, so total coverage ~= SHARDS x PER_SHARD_NUM_RUNS across processes
# that never accumulate heap. On the first failing shard we stop and keep that
# shard's output for triage.
#
# Counterexample -> fixture workflow: when a shard fails, fast-check prints the
# SHRUNK minimal counterexample plus the reproducing seed. This job files a Gitea
# issue containing that seed + counterexample ONLY when the output actually holds
# a fast-check counterexample; an infra failure (OOM/tsc/install, no
# counterexample) is filed under a DISTINCT title so it can never poison the
# counterexample dedup. A human then commits the shrunk doc as a PERMANENT fixture
# under packages/prosemirror-markdown/test/fixtures/counterexamples/ with a case in
# counterexamples.test.ts, and FIXES the converter (never weakens a property to
# hide the bug). See packages/prosemirror-markdown/README.md.
on:
schedule:
# 03:00 UTC daily.
- cron: '0 3 * * *'
workflow_dispatch:
inputs:
num_runs:
description: 'fast-check runs PER SHARD (8 shards run in sequence)'
required: false
default: '600'
seed:
description: 'base fast-check seed (empty = random); shard i uses base+i'
required: false
default: ''
permissions:
contents: read
issues: write
jobs:
property-fuzz:
runs-on: ubuntu-latest
timeout-minutes: 60
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up pnpm
uses: pnpm/action-setup@v4
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: 22
cache: pnpm
- name: Install dependencies
run: pnpm install --frozen-lockfile
# No build step: the generative suite imports the converter from src/
# directly (e.g. `from '../../src/lib/markdown-converter.js'`), so it runs
# against source without the package's build/. Skipping the build also
# keeps a tsc build error from masquerading as a property-test failure and
# filing a bogus counterexample issue.
- name: Resolve base seed and per-shard run count
id: params
# Dispatch inputs are read via env (NOT interpolated into the shell body)
# to avoid script injection through a crafted input value.
env:
SEED_INPUT: ${{ inputs.seed }}
NUM_RUNS_INPUT: ${{ inputs.num_runs }}
run: |
set -euo pipefail
SEED="${SEED_INPUT:-}"
# Empty seed (cron, or a dispatch that left it blank) -> random. Combine
# two RANDOMs so the seed spans more than RANDOM's 0..32767 range.
[ -z "$SEED" ] && SEED=$(( (RANDOM << 15) | RANDOM ))
NUM_RUNS="${NUM_RUNS_INPUT:-}"
[ -z "$NUM_RUNS" ] && NUM_RUNS=600
echo "seed=$SEED" >> "$GITHUB_OUTPUT"
echo "num_runs=$NUM_RUNS" >> "$GITHUB_OUTPUT"
echo "Sharded property fuzz: BASE_SEED=$SEED PER_SHARD_NUM_RUNS=$NUM_RUNS SHARDS=8"
- name: Run generative property suite (sharded)
id: fuzz
env:
BASE_SEED: ${{ steps.params.outputs.seed }}
PER_SHARD_NUM_RUNS: ${{ steps.params.outputs.num_runs }}
SHARDS: '8'
run: |
set -uo pipefail
# Give each fresh process headroom, but rely on SHARDING (not a big heap)
# to avoid OOM: a moderate per-shard count in a process that starts clean.
export NODE_OPTIONS=--max-old-space-size=4096
: > property-output.txt
FAILED=0
FAIL_SEED=""
i=0
while [ "$i" -lt "$SHARDS" ]; do
SHARD_SEED=$(( BASE_SEED + i ))
echo "=== shard $((i + 1))/$SHARDS: PROPERTY_SEED=$SHARD_SEED PROPERTY_NUM_RUNS=$PER_SHARD_NUM_RUNS ==="
# tee OVERWRITES property-output.txt each shard; since we break on the
# first failure, the file ends up holding exactly the failing shard's
# output (which carries the shrunk counterexample + reproducing seed).
if PROPERTY_SEED="$SHARD_SEED" PROPERTY_NUM_RUNS="$PER_SHARD_NUM_RUNS" \
pnpm --filter @docmost/prosemirror-markdown exec \
vitest run test/generative/ 2>&1 | tee property-output.txt; then
echo "shard $((i + 1)) passed"
else
echo "shard $((i + 1)) FAILED (seed=$SHARD_SEED) — stopping; keeping its output"
FAILED=1
FAIL_SEED="$SHARD_SEED"
break
fi
i=$(( i + 1 ))
done
echo "failed=$FAILED" >> "$GITHUB_OUTPUT"
echo "fail_seed=$FAIL_SEED" >> "$GITHUB_OUTPUT"
exit "$FAILED"
# A GENUINE counterexample: fast-check printed a shrunk minimal case and its
# reproducing seed into property-output.txt. File a dedup-guarded issue whose
# title prefix is UNIQUE to counterexamples, so an infra failure (handled by
# the next step under a different title) can never poison this dedup.
- name: File counterexample issue
# always() is REQUIRED: the fuzz step exits nonzero on a failing shard,
# so a bare `if:` (implicitly success() && ...) would skip this step
# exactly when it must run. always() lets it run on the failure path.
if: always() && steps.fuzz.outputs.failed == '1'
env:
FAIL_SEED: ${{ steps.fuzz.outputs.fail_seed }}
NUM_RUNS: ${{ steps.params.outputs.num_runs }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TITLE_PREFIX: 'Nightly property counterexample'
run: |
set -uo pipefail
# Discriminate counterexample vs infra failure by the fast-check
# signature. No signature -> leave it to the infra-failure step.
if ! grep -Eq 'Property failed after|Counterexample' property-output.txt; then
echo "No fast-check counterexample signature — infra failure, handled by the next step."
exit 0
fi
TITLE="${TITLE_PREFIX} (seed=${FAIL_SEED})"
# Best-effort dedup: skip if an open issue with the counterexample title
# prefix already exists. A failure of this check must NOT block creation.
EXISTING=""
if EXISTING=$(curl -sS \
-H "Authorization: token ${GITHUB_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues?state=open&limit=100"); then
if printf '%s' "$EXISTING" \
| node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{let a;try{a=JSON.parse(s)}catch{process.exit(1)}if(!Array.isArray(a))process.exit(1);const p=process.env.TITLE_PREFIX;process.exit(a.some(i=>typeof i.title==="string"&&i.title.startsWith(p))?0:1)})'; then
echo "An open '${TITLE_PREFIX}' issue already exists — skipping creation."
exit 0
fi
fi
# Build the JSON body with the test output SAFELY escaped (never hand-
# interpolate the counterexample into JSON).
BODY_TEXT=$(printf 'A nightly property fuzz SHARD failed with a fast-check counterexample.\n\n- failing shard seed: `%s`\n- NUM_RUNS (per shard): `%s`\n- run: %s\n\nReproduce locally:\n\n```\nPROPERTY_SEED=%s PROPERTY_NUM_RUNS=%s pnpm --filter @docmost/prosemirror-markdown exec vitest run test/generative/\n```\n\nfast-check shrinks the failure to a minimal counterexample. Commit it as a permanent fixture under `packages/prosemirror-markdown/test/fixtures/counterexamples/` + a case in `counterexamples.test.ts`, then FIX the converter (do not weaken a property). See `packages/prosemirror-markdown/README.md`.\n\nTail of the test output (contains the shrunk counterexample):\n\n```\n%s\n```\n' \
"$FAIL_SEED" "$NUM_RUNS" "$RUN_URL" "$FAIL_SEED" "$NUM_RUNS" "$(tail -n 120 property-output.txt)")
jq -n --arg title "$TITLE" --arg body "$BODY_TEXT" \
'{title: $title, body: $body}' > payload.json
curl -sS -X POST \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues" \
-H "Authorization: token ${GITHUB_TOKEN}" \
-H 'Content-Type: application/json' \
-d @payload.json
# An INFRA failure (OOM, tsc, install) has NO counterexample signature. File
# it under a DISTINCT title so it is visible but keeps the counterexample
# dedup (above) uncontaminated — a real counterexample can still file even
# while an infra issue is open.
- name: File infra failure issue
# always() is REQUIRED: the fuzz step exits nonzero on a failing shard,
# so a bare `if:` (implicitly success() && ...) would skip this step
# exactly when it must run. always() lets it run on the failure path.
if: always() && steps.fuzz.outputs.failed == '1'
env:
FAIL_SEED: ${{ steps.fuzz.outputs.fail_seed }}
NUM_RUNS: ${{ steps.params.outputs.num_runs }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
TITLE_PREFIX: 'Nightly property run infra failure'
run: |
set -uo pipefail
# Only file when there is NO counterexample signature (else the
# counterexample step owns it).
if grep -Eq 'Property failed after|Counterexample' property-output.txt; then
echo "Counterexample present — owned by the counterexample step."
exit 0
fi
TITLE="${TITLE_PREFIX} (seed=${FAIL_SEED})"
EXISTING=""
if EXISTING=$(curl -sS \
-H "Authorization: token ${GITHUB_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues?state=open&limit=100"); then
if printf '%s' "$EXISTING" \
| node -e 'let s="";process.stdin.on("data",d=>s+=d).on("end",()=>{let a;try{a=JSON.parse(s)}catch{process.exit(1)}if(!Array.isArray(a))process.exit(1);const p=process.env.TITLE_PREFIX;process.exit(a.some(i=>typeof i.title==="string"&&i.title.startsWith(p))?0:1)})'; then
echo "An open '${TITLE_PREFIX}' issue already exists — skipping creation."
exit 0
fi
fi
BODY_TEXT=$(printf 'A nightly property fuzz SHARD failed WITHOUT a fast-check counterexample (infra failure: OOM / build / install). This is NOT a converter round-trip bug.\n\n- failing shard seed: `%s`\n- NUM_RUNS (per shard): `%s`\n- run: %s\n\nInvestigate the run log (memory, dependency install, or a tsc/import error). The nightly counterexample dedup is intentionally separate from this issue.\n\nTail of the test output:\n\n```\n%s\n```\n' \
"$FAIL_SEED" "$NUM_RUNS" "$RUN_URL" "$(tail -n 120 property-output.txt)")
jq -n --arg title "$TITLE" --arg body "$BODY_TEXT" \
'{title: $title, body: $body}' > payload.json
curl -sS -X POST \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues" \
-H "Authorization: token ${GITHUB_TOKEN}" \
-H 'Content-Type: application/json' \
-d @payload.json
+19 -1
View File
@@ -230,6 +230,24 @@ pnpm build # nx run-many -t build (all packages)
pnpm collab:dev # run the collaboration server process standalone (see "Two server processes") pnpm collab:dev # run the collaboration server process standalone (see "Two server processes")
``` ```
> **Build the shared packages before running a consumer's `tsc`/tests in
> isolation.** The `build/` dirs of `@docmost/prosemirror-markdown`,
> `@docmost/git-sync`, and `@docmost/mcp` are **gitignored** (not committed), and
> a single-package `pnpm --filter <pkg> test` / `tsc` or a bare `pnpm -r test`
> does **NOT** honour the Nx `dependsOn: ["^build"]` ordering. So a consumer — the
> server's `tsc`, `git-sync`'s vitest typecheck, `mcp`'s `pretest: tsc` — fails
> with `error TS2307: Cannot find module '@docmost/…'` until those packages are
> built first:
> ```bash
> pnpm --filter @docmost/prosemirror-markdown build
> pnpm --filter @docmost/editor-ext build
> pnpm --filter @docmost/git-sync build && pnpm --filter @docmost/mcp build
> ```
> `pnpm build` (nx run-many) does this for you; CI does it explicitly in
> `.github/workflows/test.yml` (prosemirror-markdown → git-sync/mcp → server, in
> that order). Reach for it whenever you run a consumer package's checks on their
> own rather than through the full `pnpm build`.
**Lint** (per package — there is no root lint script): **Lint** (per package — there is no root lint script):
```bash ```bash
pnpm --filter server lint # eslint --fix on server .ts pnpm --filter server lint # eslint --fix on server .ts
@@ -293,7 +311,7 @@ The API server is a Fastify app with a global `/api` prefix (`main.ts` excludes
### Client structure ### Client structure
Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirrors the server domains: `page`, `space`, `comment`, `ai-chat`, `editor`, …). Conventions: Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirrors the server domains: `page`, `space`, `comment`, `ai-chat`, `editor`, …). Conventions:
- **TanStack Query** for server state (one `queries/` file per feature), **Jotai** atoms for local/shared UI state, **Mantine 8** + CSS modules (`*.module.css`) + `postcss-preset-mantine` for UI. - **TanStack Query** for server state (one `queries/` file per feature), **Jotai** atoms for local/shared UI state, **Mantine 8** + CSS modules (`*.module.css`) + `postcss-preset-mantine` for UI.
- The editor is Tiptap; shared node/mark extensions live in `packages/editor-ext` and are imported by **both the client and the server** (collaboration, schema, `canonicalizeFootnotes`) — editor schema changes often need to be made in `editor-ext`, not just the client. Server-side markdown import/export no longer lives in `editor-ext`: it goes through the canonical converter (#345, see below). The ProseMirror↔Markdown converter and its Docmost schema mirror now live in a SINGLE package, `@docmost/prosemirror-markdown` (#293), consumed by `mcp`, `git-sync`, and `apps/server` (#345) — do NOT reintroduce a per-package copy. `editor-ext` is the upstream source of the Tiptap schema; the package's `docmost-schema.ts` mirrors it and a serializer-contract test (`packages/prosemirror-markdown/test/serializer-contract.test.ts`) guards the boundary (every schema node must have a converter case), so a drift surfaces as a failing test rather than silent divergence. - The editor is Tiptap; shared node/mark extensions live in `packages/editor-ext` and are imported by **both the client and the server** (collaboration, schema, `canonicalizeFootnotes`) — editor schema changes often need to be made in `editor-ext`, not just the client. Server-side markdown import/export no longer lives in `editor-ext`: it goes through the canonical converter (#345, see below). The ProseMirror↔Markdown converter and its Docmost schema mirror now live in a SINGLE package, `@docmost/prosemirror-markdown` (#293), consumed by `mcp`, `git-sync`, and `apps/server` (#345) — do NOT reintroduce a per-package copy. `editor-ext` is the upstream source of the Tiptap schema; the package's `docmost-schema.ts` mirrors it and a serializer-contract test (`packages/prosemirror-markdown/test/serializer-contract.test.ts`) guards the boundary (every schema node must have a converter case), so a drift surfaces as a failing test rather than silent divergence. For the converter's property-testing and counterexample→fixture process (P1–P4 invariants, the `PROPERTY_SEED`/`PROPERTY_NUM_RUNS` knobs, and the nightly fuzz workflow), see `packages/prosemirror-markdown/README.md`.
- API access goes through `apps/client/src/lib/api-client.ts` (axios). The `@` alias maps to `apps/client/src`. - API access goes through `apps/client/src/lib/api-client.ts` (axios). The `@` alias maps to `apps/client/src`.
- Runtime config is injected at build time by `vite.config.ts` via `define` (`APP_URL`, `COLLAB_URL`, `APP_VERSION`, …) — these come from the root `.env`, not from `import.meta.env`. - Runtime config is injected at build time by `vite.config.ts` via `define` (`APP_URL`, `COLLAB_URL`, `APP_VERSION`, …) — these come from the root `.env`, not from `import.meta.env`.
+26
View File
@@ -125,6 +125,32 @@ Gitmost follows the upstream Docmost setup. See the Docmost
[documentation](https://docmost.com/docs) for self-hosting and development instructions; replace the [documentation](https://docmost.com/docs) for self-hosting and development instructions; replace the
`docmost/docmost` image with `ghcr.io/vvzvlad/gitmost` where applicable. `docmost/docmost` image with `ghcr.io/vvzvlad/gitmost` where applicable.
### Reverse proxy: SSE streaming paths
The AI agent streams its answers over Server-Sent Events. These endpoints produce a
long-lived `text/event-stream` response and **must bypass response buffering AND response
compression** at every proxy in front of the app:
- `POST /api/ai-chat/stream` — the live agent turn stream
- `GET /api/ai-chat/runs/<chatId>/stream` — attach/resume of a detached agent run
(`AI_CHAT_RESUMABLE_STREAM`)
- `POST /api/shares/ai/stream` — the anonymous public-share assistant
A buffering or compressing proxy does not break these with an error — it silently ruins them:
the request hangs in `pending`, tokens stop streaming and arrive in one burst when the turn
ends, or a reloaded tab falls back to coarse polling. The tell in DevTools is a
`Content-Encoding: gzip/zstd` response header on a `text/event-stream` response.
The server already sends `X-Accel-Buffering: no` (honored by nginx unless ignored), but
compression middleware is applied by proxy configuration, not headers:
- **nginx** — `proxy_buffering off; proxy_cache off; gzip off;` for these locations, e.g.
`location ~ ^/api/(ai-chat/(stream$|runs/.+/stream$)|shares/ai/) { ... }`
- **Traefik** — route these paths through a dedicated router **without** the `compress`
middleware (a `compress` middleware buffers SSE frames until the response closes), e.g.
``PathPrefix(`/api/ai-chat/stream`) || PathPrefix(`/api/ai-chat/runs/`)``. Belt-and-braces:
`traefik.http.middlewares.<name>.compress.excludedcontenttypes: text/event-stream`.
## Migration from Docmost ## Migration from Docmost
Gitmost's database schema is a **strict superset** of Docmost's. Every Gitmost-specific migration Gitmost's database schema is a **strict superset** of Docmost's. Every Gitmost-specific migration
+26
View File
@@ -126,6 +126,32 @@ Gitmost повторяет процесс установки upstream-Docmost.
смотрите в [документации](https://docmost.com/docs) Docmost; где это применимо, заменяйте образ смотрите в [документации](https://docmost.com/docs) Docmost; где это применимо, заменяйте образ
`docmost/docmost` на `ghcr.io/vvzvlad/gitmost`. `docmost/docmost` на `ghcr.io/vvzvlad/gitmost`.
### Reverse proxy: SSE-стриминговые пути
AI-агент стримит ответы через Server-Sent Events. Эти эндпоинты отдают долгоживущий
`text/event-stream`-ответ и **обязаны обходить буферизацию И сжатие ответов** на каждом
прокси перед приложением:
- `POST /api/ai-chat/stream` — живой стрим хода агента
- `GET /api/ai-chat/runs/<chatId>/stream` — подключение/резюм detached-рана
(`AI_CHAT_RESUMABLE_STREAM`)
- `POST /api/shares/ai/stream` — анонимный ассистент публичных шар
Буферизующий или сжимающий прокси не ломает эти пути с ошибкой — он тихо их портит:
запрос висит в `pending`, токены не стримятся и вываливаются одним куском в конце хода,
а перезагруженная вкладка падает в грубый поллинг. Диагностический признак в DevTools —
заголовок `Content-Encoding: gzip/zstd` на ответе с `text/event-stream`.
Сервер уже шлёт `X-Accel-Buffering: no` (nginx учитывает его по умолчанию), но
compression-мидлвари управляются конфигом прокси, а не заголовками:
- **nginx** — `proxy_buffering off; proxy_cache off; gzip off;` для этих location,
например `location ~ ^/api/(ai-chat/(stream$|runs/.+/stream$)|shares/ai/) { ... }`
- **Traefik** — вести эти пути через отдельный роутер **без** `compress`-мидлвари
(compress буферизует SSE-кадры до закрытия ответа), например
``PathPrefix(`/api/ai-chat/stream`) || PathPrefix(`/api/ai-chat/runs/`)``. Для надёжности:
`traefik.http.middlewares.<name>.compress.excludedcontenttypes: text/event-stream`.
## Миграция с Docmost ## Миграция с Docmost
Схема БД Gitmost — это **строгий superset** схемы Docmost. Все Gitmost-специфичные миграции только Схема БД Gitmost — это **строгий superset** схемы Docmost. Все Gitmost-специфичные миграции только
+35 -10
View File
@@ -23,18 +23,33 @@ roles:
inside it, which terms are ambiguous or have synonyms/jargon. inside it, which terms are ambiguous or have synonyms/jargon.
- Formulate 5–10 search directions, including adjacent perspectives that - Formulate 5–10 search directions, including adjacent perspectives that
may prove useful even if the user did not ask about them directly. may prove useful even if the user did not ask about them directly.
- Set a "research budget" — roughly how many searches the task's complexity - Fix the "research budget" — how many searches to run. If the USER named a
warrants (a simple fact: under 5; a medium task: 5–15; a hard task: more). budget (e.g. "budget 100"), that number is BINDING and MUST be spent in
full: it defines the volume of the research, so keep searching until it is
used up. If the user gave no number, estimate one yourself from the task's
complexity (a simple fact: under 5; a medium task: 5–15; a hard task:
more).
- Decide which languages it makes sense to search in (see below). - Decide which languages it makes sense to search in (see below).
═══════════════════════════════════════════════ ═══════════════════════════════════════════════
WHERE TO WRITE THE RESULT WHERE TO WRITE THE RESULT
═══════════════════════════════════════════════ ═══════════════════════════════════════════════
- If the user explicitly asks to work in the current/already-open document, - Reuse the current/already-open document ONLY if either (a) the user
work in it. explicitly asked to work in it, or (b) it is empty or has very little on
- If this is not specified, create a NEW document for the report. it AND its title matches the topic of the research. In every other case —
- Keep a working draft in the document or in notes: fact → source a non-empty page, or one whose title is about something else — create a
reliability assessment. Update the structure as you go. NEW document for the report.
- Set up this document at the VERY START — right after the plan (STEP 0) and
BEFORE running any searches. Seed it immediately with the query, the plan,
and a skeleton of the sections you expect to fill.
- Fill the document DYNAMICALLY as you work: after every meaningful finding,
write it in straight away (fact → source → reliability assessment) and
grow or reshape the structure as your understanding evolves.
- Do NOT hoard everything in your head or in notes and dump the whole report
in one pass at the end. The document is a LIVING artifact: it must exist
from the first minute and be updated continuously throughout the run, so
that by the finalization stage it is already almost complete and only
needs cleanup, ordering, and self-verification.
═══════════════════════════════════════════════ ═══════════════════════════════════════════════
WORK LOOP (repeat until saturation) WORK LOOP (repeat until saturation)
@@ -53,9 +68,19 @@ roles:
HOW TO SEARCH HOW TO SEARCH
═══════════════════════════════════════════════ ═══════════════════════════════════════════════
VOLUME. Execute a MINIMUM of 15 distinct searches, more for complex tasks. VOLUME. Execute a MINIMUM of 15 distinct searches, more for complex tasks.
Do not stop at the first plausible answer. Stop only when further searches Do not stop at the first plausible answer. Absent an explicit budget, stop
stop yielding new relevant information (saturation / diminishing returns) — only when further searches stop yielding new relevant information
not when it "seems like enough" or when you get tired. (saturation / diminishing returns) — not when it "seems like enough" or when
you get tired.
MANDATORY BUDGET. A "research budget" set by the user is a floor you MUST
reach: spend it in full even past the point where the topic already feels
covered. Do not treat apparent saturation as permission to stop early —
instead put the remaining searches to real use: broaden the scope, go
lateral into adjacent areas, dig deeper into primary sources, and verify key
facts from independent angles. Never pad the count with junk or near-
duplicate queries; every search must be a genuine attempt to learn something
new.
WIDE → NARROW. Start with short, broad queries (2–5 words), survey the WIDE → NARROW. Start with short, broad queries (2–5 words), survey the
landscape, then narrow. If results are scarce, broaden the phrasing; if landscape, then narrow. If results are scarce, broaden the phrasing; if
+35 -10
View File
@@ -23,18 +23,33 @@ roles:
inside it, which terms are ambiguous or have synonyms/jargon. inside it, which terms are ambiguous or have synonyms/jargon.
- Formulate 5–10 search directions, including adjacent perspectives that - Formulate 5–10 search directions, including adjacent perspectives that
may prove useful even if the user did not ask about them directly. may prove useful even if the user did not ask about them directly.
- Set a "research budget" — roughly how many searches the task's complexity - Fix the "research budget" — how many searches to run. If the USER named a
warrants (a simple fact: under 5; a medium task: 5–15; a hard task: more). budget (e.g. "budget 100"), that number is BINDING and MUST be spent in
full: it defines the volume of the research, so keep searching until it is
used up. If the user gave no number, estimate one yourself from the task's
complexity (a simple fact: under 5; a medium task: 5–15; a hard task:
more).
- Decide which languages it makes sense to search in (see below). - Decide which languages it makes sense to search in (see below).
═══════════════════════════════════════════════ ═══════════════════════════════════════════════
WHERE TO WRITE THE RESULT WHERE TO WRITE THE RESULT
═══════════════════════════════════════════════ ═══════════════════════════════════════════════
- If the user explicitly asks to work in the current/already-open document, - Reuse the current/already-open document ONLY if either (a) the user
work in it. explicitly asked to work in it, or (b) it is empty or has very little on
- If this is not specified, create a NEW document for the report. it AND its title matches the topic of the research. In every other case —
- Keep a working draft in the document or in notes: fact → source a non-empty page, or one whose title is about something else — create a
reliability assessment. Update the structure as you go. NEW document for the report.
- Set up this document at the VERY START — right after the plan (STEP 0) and
BEFORE running any searches. Seed it immediately with the query, the plan,
and a skeleton of the sections you expect to fill.
- Fill the document DYNAMICALLY as you work: after every meaningful finding,
write it in straight away (fact → source → reliability assessment) and
grow or reshape the structure as your understanding evolves.
- Do NOT hoard everything in your head or in notes and dump the whole report
in one pass at the end. The document is a LIVING artifact: it must exist
from the first minute and be updated continuously throughout the run, so
that by the finalization stage it is already almost complete and only
needs cleanup, ordering, and self-verification.
═══════════════════════════════════════════════ ═══════════════════════════════════════════════
WORK LOOP (repeat until saturation) WORK LOOP (repeat until saturation)
@@ -53,9 +68,19 @@ roles:
HOW TO SEARCH HOW TO SEARCH
═══════════════════════════════════════════════ ═══════════════════════════════════════════════
VOLUME. Execute a MINIMUM of 15 distinct searches, more for complex tasks. VOLUME. Execute a MINIMUM of 15 distinct searches, more for complex tasks.
Do not stop at the first plausible answer. Stop only when further searches Do not stop at the first plausible answer. Absent an explicit budget, stop
stop yielding new relevant information (saturation / diminishing returns) — only when further searches stop yielding new relevant information
not when it "seems like enough" or when you get tired. (saturation / diminishing returns) — not when it "seems like enough" or when
you get tired.
MANDATORY BUDGET. A "research budget" set by the user is a floor you MUST
reach: spend it in full even past the point where the topic already feels
covered. Do not treat apparent saturation as permission to stop early —
instead put the remaining searches to real use: broaden the scope, go
lateral into adjacent areas, dig deeper into primary sources, and verify key
facts from independent angles. Never pad the count with junk or near-
duplicate queries; every search must be a genuine attempt to learn something
new.
WIDE → NARROW. Start with short, broad queries (2–5 words), survey the WIDE → NARROW. Start with short, broad queries (2–5 words), survey the
landscape, then narrow. If results are scarce, broaden the phrasing; if landscape, then narrow. If results are scarce, broaden the phrasing; if
+1 -1
View File
@@ -33,4 +33,4 @@ bundles:
- en - en
roles: roles:
- slug: researcher - slug: researcher
version: 1 version: 4
@@ -16,8 +16,8 @@
"hash": "cef39fed321779631ddd1077fcba53399adf0e48b301df281c71eb042610900d" "hash": "cef39fed321779631ddd1077fcba53399adf0e48b301df281c71eb042610900d"
}, },
"researcher": { "researcher": {
"version": 1, "version": 4,
"hash": "853658fda43ddbe0a4d08f2c6e50b5116d29a2e9ccd7f46e173e65920d8f6ace" "hash": "9446ec6d2c8a6ec548358537ac392b8bf9b4d2a832ebb105d5514eac2c76da74"
}, },
"structural-editor": { "structural-editor": {
"version": 4, "version": 4,
-1
View File
@@ -13,7 +13,6 @@
}, },
"dependencies": { "dependencies": {
"@ai-sdk/react": "^3.0.208", "@ai-sdk/react": "^3.0.208",
"@braintree/sanitize-url": "7.1.2",
"@atlaskit/pragmatic-drag-and-drop": "1.8.1", "@atlaskit/pragmatic-drag-and-drop": "1.8.1",
"@atlaskit/pragmatic-drag-and-drop-auto-scroll": "2.1.5", "@atlaskit/pragmatic-drag-and-drop-auto-scroll": "2.1.5",
"@atlaskit/pragmatic-drag-and-drop-flourish": "2.0.15", "@atlaskit/pragmatic-drag-and-drop-flourish": "2.0.15",
+24 -58
View File
@@ -1,72 +1,38 @@
import { lazy, Suspense } from "react";
import { Navigate, Route, Routes } from "react-router-dom"; import { Navigate, Route, Routes } from "react-router-dom";
import { Center, Loader } from "@mantine/core";
import { Error404 } from "@/components/ui/error-404.tsx";
import Layout from "@/components/layouts/global/layout.tsx";
import { useTrackOrigin } from "@/hooks/use-track-origin";
// ShareLayout is route-split: its ShareShell chrome pulls in the table of
// contents (and thus TipTap), so keeping it out of the eager graph removes the
// editor engine from startup for authenticated users too.
const ShareLayout = lazy(
() => import("@/features/share/components/share-layout.tsx"),
);
// Auth / entry pages stay eager: they are the first paint for an unauthenticated
// visitor (e.g. /login) and are already small, so code-splitting them would only
// add a cold-chunk round trip to the most common cold-start path.
import SetupWorkspace from "@/pages/auth/setup-workspace.tsx"; import SetupWorkspace from "@/pages/auth/setup-workspace.tsx";
import LoginPage from "@/pages/auth/login"; import LoginPage from "@/pages/auth/login";
import Home from "@/pages/dashboard/home";
import Page from "@/pages/page/page";
import AccountSettings from "@/pages/settings/account/account-settings";
import WorkspaceMembers from "@/pages/settings/workspace/workspace-members";
import WorkspaceSettings from "@/pages/settings/workspace/workspace-settings";
import AiSettings from "@/pages/settings/workspace/ai-settings";
import Groups from "@/pages/settings/group/groups";
import GroupInfo from "./pages/settings/group/group-info";
import Spaces from "@/pages/settings/space/spaces.tsx";
import { Error404 } from "@/components/ui/error-404.tsx";
import AccountPreferences from "@/pages/settings/account/account-preferences.tsx";
import SpaceHome from "@/pages/space/space-home.tsx";
import PageRedirect from "@/pages/page/page-redirect.tsx";
import Layout from "@/components/layouts/global/layout.tsx";
import InviteSignup from "@/pages/auth/invite-signup.tsx"; import InviteSignup from "@/pages/auth/invite-signup.tsx";
import ForgotPassword from "@/pages/auth/forgot-password.tsx"; import ForgotPassword from "@/pages/auth/forgot-password.tsx";
import PasswordReset from "./pages/auth/password-reset"; import PasswordReset from "./pages/auth/password-reset";
import PageRedirect from "@/pages/page/page-redirect.tsx"; import SharedPage from "@/pages/share/shared-page.tsx";
import Shares from "@/pages/settings/shares/shares.tsx";
import ShareLayout from "@/features/share/components/share-layout.tsx";
import ShareRedirect from "@/pages/share/share-redirect.tsx"; import ShareRedirect from "@/pages/share/share-redirect.tsx";
import { useTrackOrigin } from "@/hooks/use-track-origin";
// Heavy / leaf pages are route-split with React.lazy so their code (most import SpacesPage from "@/pages/spaces/spaces.tsx";
// importantly the whole TipTap editor + KaTeX + lowlight grammars + drawio that import SpaceTrash from "@/pages/space/space-trash.tsx";
// the page editor and the readonly share editor pull in) is fetched only when import FavoritesPage from "@/pages/favorites/favorites-page";
// the matching route is actually visited. The <Suspense> boundaries live inside import LabelPage from "@/pages/label/label-page";
// each Layout (around its <Outlet/>), so the app shell stays mounted while a
// route chunk loads.
const Home = lazy(() => import("@/pages/dashboard/home"));
const Page = lazy(() => import("@/pages/page/page"));
const SpaceHome = lazy(() => import("@/pages/space/space-home.tsx"));
const SpaceTrash = lazy(() => import("@/pages/space/space-trash.tsx"));
const SpacesPage = lazy(() => import("@/pages/spaces/spaces.tsx"));
const FavoritesPage = lazy(() => import("@/pages/favorites/favorites-page"));
const LabelPage = lazy(() => import("@/pages/label/label-page"));
const SharedPage = lazy(() => import("@/pages/share/shared-page.tsx"));
const AccountSettings = lazy(
() => import("@/pages/settings/account/account-settings"),
);
const AccountPreferences = lazy(
() => import("@/pages/settings/account/account-preferences.tsx"),
);
const WorkspaceSettings = lazy(
() => import("@/pages/settings/workspace/workspace-settings"),
);
const AiSettings = lazy(() => import("@/pages/settings/workspace/ai-settings"));
const WorkspaceMembers = lazy(
() => import("@/pages/settings/workspace/workspace-members"),
);
const Groups = lazy(() => import("@/pages/settings/group/groups"));
const GroupInfo = lazy(() => import("./pages/settings/group/group-info"));
const Spaces = lazy(() => import("@/pages/settings/space/spaces.tsx"));
const Shares = lazy(() => import("@/pages/settings/shares/shares.tsx"));
export default function App() { export default function App() {
useTrackOrigin(); useTrackOrigin();
return ( return (
<Suspense <>
fallback={
<Center h="100vh">
<Loader size="sm" />
</Center>
}
>
<Routes> <Routes>
<Route index element={<Navigate to="/home" />} /> <Route index element={<Navigate to="/home" />} />
<Route path={"/login"} element={<LoginPage />} /> <Route path={"/login"} element={<LoginPage />} />
@@ -117,6 +83,6 @@ export default function App() {
<Route path="*" element={<Error404 />} /> <Route path="*" element={<Error404 />} />
</Routes> </Routes>
</Suspense> </>
); );
} }
@@ -1,37 +0,0 @@
import { describe, it, expect } from "vitest";
import { isChunkLoadError } from "./chunk-load-error-boundary";
// The detector decides whether a caught render error is a stale-deploy chunk-404
// (→ auto-reload to fetch the new manifest) vs a genuine app error (→ generic
// recovery UI, no reload). A false negative on a real chunk failure re-blanks the
// app; a false positive would auto-reload on an ordinary error. Pin both sides.
describe("isChunkLoadError", () => {
it("detects the ChunkLoadError name", () => {
expect(isChunkLoadError({ name: "ChunkLoadError", message: "x" })).toBe(true);
});
it.each([
"Failed to fetch dynamically imported module: https://x/assets/index-abc.js",
"error loading dynamically imported module",
"Importing a module script failed.",
])("detects the dynamic-import failure message %#", (message) => {
expect(isChunkLoadError({ name: "TypeError", message })).toBe(true);
});
it("is case-insensitive on the message", () => {
expect(
isChunkLoadError({ message: "FAILED TO FETCH DYNAMICALLY IMPORTED MODULE" }),
).toBe(true);
});
it.each([
null,
undefined,
{},
{ name: "TypeError", message: "Cannot read properties of undefined" },
{ message: "Network request failed" },
new Error("some ordinary render error"),
])("returns false for a non-chunk error %#", (err) => {
expect(isChunkLoadError(err)).toBe(false);
});
});
@@ -1,71 +0,0 @@
import { ReactNode } from "react";
import { ErrorBoundary } from "react-error-boundary";
import { Button, Center, Stack, Text } from "@mantine/core";
const RELOAD_FLAG = "chunk-reload-attempted";
// Heuristic detection of a failed dynamic import. Since the code-splitting work,
// every route (plus Aside / AiChatWindow) is React.lazy: when a new deploy
// replaces the hashed chunks, a tab left open on the old index.html requests a
// chunk URL that now 404s, and React.lazy rejects. Browsers / Vite surface these
// with a ChunkLoadError name or one of these messages.
export function isChunkLoadError(error: unknown): boolean {
if (!error) return false;
const name = (error as { name?: string }).name ?? "";
const message = (error as { message?: string }).message ?? "";
return (
name === "ChunkLoadError" ||
/Failed to fetch dynamically imported module/i.test(message) ||
/error loading dynamically imported module/i.test(message) ||
/Importing a module script failed/i.test(message)
);
}
function handleError(error: unknown) {
if (!isChunkLoadError(error)) return;
// A stale-chunk 404 is cured by a full reload that re-fetches index.html and
// the new chunk manifest. Auto-reload once, guarding against a reload loop
// (e.g. a genuinely missing chunk) with a one-shot sessionStorage flag. If the
// flag is already set we fall through to the manual recovery UI below.
try {
if (sessionStorage.getItem(RELOAD_FLAG)) return;
sessionStorage.setItem(RELOAD_FLAG, "1");
} catch {
// sessionStorage unavailable (private mode / disabled): skip the automatic
// reload rather than risk an unguarded loop; the fallback UI still recovers.
return;
}
window.location.reload();
}
// Root-level boundary that sits ABOVE every route-level Suspense boundary so a
// lazy route/component chunk failure is caught here instead of unmounting the
// whole tree into a blank white screen. Per-feature ErrorBoundaries (page.tsx,
// transclusion, page-embed) remain in place underneath for their local errors.
export function ChunkLoadErrorBoundary({ children }: { children: ReactNode }) {
return (
<ErrorBoundary
onError={handleError}
fallbackRender={({ error }) => {
const chunk = isChunkLoadError(error);
return (
<Center h="100vh" p="md">
<Stack align="center" gap="sm" maw={420}>
<Text fw={600}>
{chunk ? "A new version is available" : "Something went wrong"}
</Text>
<Text size="sm" c="dimmed" ta="center">
{chunk
? "Please reload the page to load the latest version."
: "An unexpected error occurred. Reloading the page may help."}
</Text>
<Button onClick={() => window.location.reload()}>Reload</Button>
</Stack>
</Center>
);
}}
>
{children}
</ErrorBoundary>
);
}
@@ -1,10 +1,9 @@
import { AppShell, Container } from "@mantine/core"; import { AppShell, Container } from "@mantine/core";
import React, { Suspense, useEffect, useRef, useState } from "react"; import React, { useEffect, useRef, useState } from "react";
import { useLocation } from "react-router-dom"; import { useLocation } from "react-router-dom";
import { useTranslation } from "react-i18next"; import { useTranslation } from "react-i18next";
import SettingsSidebar from "@/components/settings/settings-sidebar.tsx"; import SettingsSidebar from "@/components/settings/settings-sidebar.tsx";
import { useAtom, useAtomValue } from "jotai"; import { useAtom } from "jotai";
import { aiChatWindowOpenAtom } from "@/features/ai-chat/atoms/ai-chat-atom.ts";
import { import {
APP_NAVBAR_ID, APP_NAVBAR_ID,
NAVBAR_COLLAPSE_BREAKPOINT, NAVBAR_COLLAPSE_BREAKPOINT,
@@ -15,6 +14,8 @@ import {
} from "@/components/layouts/global/hooks/atoms/sidebar-atom.ts"; } from "@/components/layouts/global/hooks/atoms/sidebar-atom.ts";
import { SpaceSidebar } from "@/features/space/components/sidebar/space-sidebar.tsx"; import { SpaceSidebar } from "@/features/space/components/sidebar/space-sidebar.tsx";
import { AppHeader } from "@/components/layouts/global/app-header.tsx"; import { AppHeader } from "@/components/layouts/global/app-header.tsx";
import Aside from "@/components/layouts/global/aside.tsx";
import AiChatWindow from "@/features/ai-chat/components/ai-chat-window.tsx";
import GitmostGlobalBridge from "@/features/editor/gitmost/gitmost-global-bridge.tsx"; import GitmostGlobalBridge from "@/features/editor/gitmost/gitmost-global-bridge.tsx";
import classes from "./app-shell.module.css"; import classes from "./app-shell.module.css";
import { useToggleSidebar } from "@/components/layouts/global/hooks/hooks/use-toggle-sidebar.ts"; import { useToggleSidebar } from "@/components/layouts/global/hooks/hooks/use-toggle-sidebar.ts";
@@ -22,21 +23,6 @@ import GlobalSidebar from "@/components/layouts/global/global-sidebar.tsx";
import { ASIDE_PANEL_ID } from "@/hooks/use-toggle-aside.tsx"; import { ASIDE_PANEL_ID } from "@/hooks/use-toggle-aside.tsx";
import { MAIN_CONTENT_ID, SkipToMain } from "@/components/ui/skip-to-main.tsx"; import { MAIN_CONTENT_ID, SkipToMain } from "@/components/ui/skip-to-main.tsx";
// Lazily load the AI chat window so the AI SDK runtime it pulls in is fetched
// only after the user first opens the chat, instead of for every authenticated
// user on load. The window itself renders null while closed, so there is no
// behavior difference — it simply is not mounted until first opened.
const AiChatWindow = React.lazy(
() => import("@/features/ai-chat/components/ai-chat-window.tsx"),
);
// The right aside hosts the comment panel and table of contents, both of which
// pull in TipTap. It only ever renders on page routes, so lazy-loading it keeps
// the whole editor engine out of the eager global-shell startup graph.
const Aside = React.lazy(
() => import("@/components/layouts/global/aside.tsx"),
);
export default function GlobalAppShell({ export default function GlobalAppShell({
children, children,
}: { }: {
@@ -51,15 +37,6 @@ export default function GlobalAppShell({
const [isResizing, setIsResizing] = useState(false); const [isResizing, setIsResizing] = useState(false);
const sidebarRef = useRef(null); const sidebarRef = useRef(null);
// Latch: once the AI chat window has been opened, keep it mounted so an
// in-flight stream is never torn down. Before the first open the AI chat chunk
// is never fetched.
const aiChatOpen = useAtomValue(aiChatWindowOpenAtom);
const [aiChatEverOpened, setAiChatEverOpened] = useState(false);
useEffect(() => {
if (aiChatOpen) setAiChatEverOpened(true);
}, [aiChatOpen]);
const startResizing = React.useCallback((mouseDownEvent) => { const startResizing = React.useCallback((mouseDownEvent) => {
mouseDownEvent.preventDefault(); mouseDownEvent.preventDefault();
setIsResizing(true); setIsResizing(true);
@@ -183,21 +160,13 @@ export default function GlobalAppShell({
: undefined : undefined
} }
> >
<Suspense fallback={null}> <Aside />
<Aside />
</Suspense>
</AppShell.Aside> </AppShell.Aside>
)} )}
</AppShell> </AppShell>
{/* Floating AI chat window. Mounted once globally on first open; it is {/* Floating AI chat window. Mounted once globally; it is position: fixed
position: fixed and self-hides when closed, so its place in the tree is and self-hides when closed, so its place in the tree is not critical. */}
not critical. Kept mounted after the first open so a live stream is not <AiChatWindow />
aborted. */}
{aiChatEverOpened && (
<Suspense fallback={null}>
<AiChatWindow />
</Suspense>
)}
{/* Global gitmost native bridge: registers listSpaces / listPages / {/* Global gitmost native bridge: registers listSpaces / listPages /
createPageWithRecording on window.gitmost so the native host can createPageWithRecording on window.gitmost so the native host can
create a page with a recording even when no page editor is open. */} create a page with a recording even when no page editor is open. */}
@@ -1,7 +1,5 @@
import { Suspense, useEffect } from "react";
import { UserProvider } from "@/features/user/user-provider.tsx"; import { UserProvider } from "@/features/user/user-provider.tsx";
import { Outlet, useParams } from "react-router-dom"; import { Outlet, useParams } from "react-router-dom";
import { Center, Loader } from "@mantine/core";
import GlobalAppShell from "@/components/layouts/global/global-app-shell.tsx"; import GlobalAppShell from "@/components/layouts/global/global-app-shell.tsx";
import { SearchSpotlight } from "@/features/search/components/search-spotlight.tsx"; import { SearchSpotlight } from "@/features/search/components/search-spotlight.tsx";
import { useGetSpaceBySlugQuery } from "@/features/space/queries/space-query.ts"; import { useGetSpaceBySlugQuery } from "@/features/space/queries/space-query.ts";
@@ -10,39 +8,10 @@ export default function Layout() {
const { spaceSlug } = useParams(); const { spaceSlug } = useParams();
const { data: space } = useGetSpaceBySlugQuery(spaceSlug); const { data: space } = useGetSpaceBySlugQuery(spaceSlug);
// Warm the (now route-split) editor chunk during idle time on authenticated
// routes, so the first navigation to a page renders from cache instead of a
// cold chunk fetch. Best-effort: gated on requestIdleCallback and never blocks
// startup — the dynamic import mirrors the App.tsx route lazy loader so both
// resolve to the same chunk.
useEffect(() => {
const ric =
typeof window !== "undefined" && (window as any).requestIdleCallback;
const warm = () => {
// Best-effort prefetch: a failed warm-up (offline, stale 404) is harmless
// and must not surface as an unhandledrejection.
void import("@/pages/page/page").catch(() => {});
};
if (ric) {
const id = ric(warm);
return () => (window as any).cancelIdleCallback?.(id);
}
const timer = setTimeout(warm, 2000);
return () => clearTimeout(timer);
}, []);
return ( return (
<UserProvider> <UserProvider>
<GlobalAppShell> <GlobalAppShell>
<Suspense <Outlet />
fallback={
<Center h="60vh">
<Loader size="sm" />
</Center>
}
>
<Outlet />
</Suspense>
</GlobalAppShell> </GlobalAppShell>
<SearchSpotlight spaceId={space?.id} /> <SearchSpotlight spaceId={space?.id} />
</UserProvider> </UserProvider>
@@ -37,21 +37,22 @@ import {
mobileSidebarAtom, mobileSidebarAtom,
} from "@/components/layouts/global/hooks/atoms/sidebar-atom.ts"; } from "@/components/layouts/global/hooks/atoms/sidebar-atom.ts";
import { usePageQuery } from "@/features/page/queries/page-query.ts"; import { usePageQuery } from "@/features/page/queries/page-query.ts";
import {
pageEditorAtom,
readOnlyEditorAtom,
} from "@/features/editor/atoms/editor-atoms.ts";
import {
getEditorSelectionContext,
type EditorSelectionContext,
} from "@/features/editor/utils/get-editor-selection.ts";
import { extractPageSlugId } from "@/lib"; import { extractPageSlugId } from "@/lib";
import { import {
AI_CHATS_RQ_KEY, AI_CHATS_RQ_KEY,
AI_CHAT_MESSAGES_RQ_KEY, AI_CHAT_MESSAGES_RQ_KEY,
AI_CHAT_RUN_RQ_KEY,
useAiChatMessagesQuery, useAiChatMessagesQuery,
useAiChatRunQuery,
useAiChatsQuery, useAiChatsQuery,
useAiRolesQuery, useAiRolesQuery,
} from "@/features/ai-chat/queries/ai-chat-query.ts"; } from "@/features/ai-chat/queries/ai-chat-query.ts";
import {
shouldClearLatchOnQueryError,
shouldClearStoppingLatch,
shouldObserveRun,
} from "@/features/ai-chat/utils/run-polling.ts";
import { workspaceAtom } from "@/features/user/atoms/current-user-atom"; import { workspaceAtom } from "@/features/user/atoms/current-user-atom";
import ConversationList from "@/features/ai-chat/components/conversation-list.tsx"; import ConversationList from "@/features/ai-chat/components/conversation-list.tsx";
import ChatThread from "@/features/ai-chat/components/chat-thread.tsx"; import ChatThread from "@/features/ai-chat/components/chat-thread.tsx";
@@ -85,6 +86,12 @@ const MIN_HEIGHT = 400;
// Margin kept between the window and the viewport edges while dragging. // Margin kept between the window and the viewport edges while dragging.
const EDGE_MARGIN = 8; const EDGE_MARGIN = 8;
// #184 phase 1.5: hard cap on the degraded-poll fallback. The poll is armed when
// a resume attempt could not attach to the live run and disarmed by the thread on
// settle / local stream; this cap is the ONLY backstop against an endless tick
// (a stuck 'streaming' row before the boot-sweep, or a user-tail 204 with no run).
const DEGRADED_POLL_MAX_MS = 10 * 60_000;
/** Compact token formatter: 1.2M / 3.4k / 950. */ /** Compact token formatter: 1.2M / 3.4k / 950. */
function formatTokens(n: number): string { function formatTokens(n: number): string {
if (n >= 1_000_000) return `${(n / 1_000_000).toFixed(1)}M`; if (n >= 1_000_000) return `${(n / 1_000_000).toFixed(1)}M`;
@@ -242,150 +249,62 @@ export default function AiChatWindow() {
[roles], [roles],
); );
// #184 phase 1.5: degraded-poll fallback (replaces the F4/F5/F7 latches). When
// ChatThread could not attach to a still-running run it arms this via
// onResumeFallback(true); the thread disarms it on settle / local stream. The
// window only OWNS the timer (armedAtRef stamps when it was armed for the cap).
const [degradedPoll, setDegradedPoll] = useState(false);
const armedAtRef = useRef(0);
const onResumeFallback = useCallback((active: boolean): void => {
if (active) armedAtRef.current = Date.now();
setDegradedPoll(active);
}, []);
// Reset the degraded poll whenever the open chat changes: it is scoped to the
// resume attempt of the previously-open chat (invariant 8).
useEffect(() => {
setDegradedPoll(false);
}, [activeChatId]);
const { data: messageRows, isLoading: messagesLoading } = const { data: messageRows, isLoading: messagesLoading } =
useAiChatMessagesQuery(activeChatId ?? undefined); useAiChatMessagesQuery(
activeChatId ?? undefined,
// DELIBERATELY DUMB (invariant 8 / task 2.4): poll every 2.5s while armed
// and under the 10-min cap; otherwise off. NO error checks (TanStack v5
// resets fetchFailureCount each fetch, so consecutive errors are not
// expressible — and the poll must survive a server restart) and NO tail
// checks (the settled/local-stream semantics live in ChatThread, which
// disarms via onResumeFallback(false)). The time cap is the only backstop.
() =>
degradedPoll === true &&
Date.now() - armedAtRef.current < DEGRADED_POLL_MAX_MS
? 2500
: false,
);
// #184 reconnect-and-live-follow. Whether detached agent runs are enabled for // #184 reconnect-and-live-follow. Whether detached agent runs are enabled for
// this workspace. The reconnect endpoint itself is NOT flag-gated server-side // this workspace. When the feature is off no runs are ever created, so the
// (it is only owner-gated and returns `{ run: null }` when the chat has no // resume attempt would only ever 204; gating ChatThread's resume on it avoids a
// run); but when the feature is off no runs are ever created, so polling it // pointless attach round-trip.
// would always come back empty — we gate it off here to avoid pointless polls.
const workspace = useAtomValue(workspaceAtom); const workspace = useAtomValue(workspaceAtom);
const autonomousRunsEnabled = const autonomousRunsEnabled =
workspace?.settings?.ai?.autonomousRuns === true; workspace?.settings?.ai?.autonomousRuns === true;
// Whether THIS tab is the one actively streaming the open chat's run locally
// (it started the run here and holds the SSE). Reported up from ChatThread. We
// are the STREAMER while true and a passive OBSERVER while false — the basis of
// the observer-vs-streamer detection. Reset to false by the fresh ChatThread's
// mount effect on every chat switch.
const [localStreaming, setLocalStreaming] = useState(false);
const onStreamingChange = useCallback((streaming: boolean) => {
setLocalStreaming(streaming);
}, []);
// #184 Stop wiring. While a detached run is being stopped we SUPPRESS the
// observer merge so the stopping run's still-persisting output does not
// re-stream back into view between the moment the user pressed Stop and the run
// actually settling as 'aborted' server-side. Polling itself keeps running (so
// the terminal transition is still detected) — only the visual merge is gated.
// Cleared when the run is observed terminal (below) or the chat is switched.
const [stoppingRun, setStoppingRun] = useState(false);
// Reset the stopping latch whenever the open chat changes: it is scoped to the
// run of the previously-open chat.
useEffect(() => {
setStoppingRun(false);
}, [activeChatId]);
// Authoritative stop of the open chat's detached run (the Stop button in // Authoritative stop of the open chat's detached run (the Stop button in
// autonomous mode). Latch "stopping" first (suppresses the re-stream flash), // autonomous mode). Request the server stop — the ONLY thing that ends a
// then request the server stop — the ONLY thing that ends a detached run; a mere // detached run; a mere local SSE abort is a client disconnect the server
// local SSE abort is a client disconnect the server ignores. On failure we // ignores. On failure surface the error.
// release the latch so the observer resumes (better to show the live run than to
// freeze the view) and surface the error.
const handleServerStop = useCallback( const handleServerStop = useCallback(
(chatId: string): void => { (chatId: string): void => {
setStoppingRun(true);
// #234 F4: drop the PREVIOUS turn's run from the cache so `run` becomes null
// until the CURRENT turn's run is fetched fresh. Without this, once the local
// stream aborts (localStreaming -> false) the run query re-enables and
// react-query SYNCHRONOUSLY returns the still-cached prior terminal run; the
// terminal effect would then clear the stopping latch against that STALE run
// before the current turn's (still-running, detached, growing) run is ever
// observed — re-opening the observer merge and flashing the growing output
// over the frozen row. With the cache cleared the terminal effect's
// `if (!run) return` holds the latch until the current run itself is observed
// terminal (see shouldClearStoppingLatch).
queryClient.removeQueries({ queryKey: AI_CHAT_RUN_RQ_KEY(chatId) });
void stopRun(chatId).catch(() => { void stopRun(chatId).catch(() => {
setStoppingRun(false);
notifications.show({ notifications.show({
message: t("Failed to stop the run"), message: t("Failed to stop the run"),
color: "red", color: "red",
}); });
}); });
}, },
[t, queryClient], [t],
); );
// Poll the latest run of the open chat ONLY when we are a passive observer:
// feature on, a chat is open, and we are NOT the local streamer (the streamer
// already has the live SSE — polling/merging too would double-render). The
// query's own status-keyed refetchInterval stops once the run is terminal.
const { data: runData, isError: runQueryFailed } = useAiChatRunQuery(
activeChatId ?? undefined,
autonomousRunsEnabled && !localStreaming,
);
const run = runData?.run ?? null;
// Safety net (#234 F4 review): after handleServerStop clears the run cache,
// `run` is null until the current turn's run is fetched fresh, and the terminal
// effect below holds the latch via `if (!run) return`. If that refetch instead
// ERRORS PERMANENTLY (the GET-run keeps failing) while we are no longer the
// streamer, the run stays null, its status-keyed refetchInterval is off, and
// nothing would ever observe a terminal run — freezing the view with the
// observer merge suppressed. Release the latch on that error so the live view
// resumes rather than stays stuck (the local stopRun may already have succeeded
// independently).
//
// #234 F7: this must NOT fire on a TRANSIENT error while `run` is still an
// ACTIVE held run. In TanStack Query v5 (retry:false) the query's `data` is
// RETAINED on error, so `runQueryFailed` can be true while `run` is still
// pending/running — releasing then would re-open the observer merge and flash
// the growing detached run over the frozen row (the very flash F4 prevents). The
// decision is the pure, unit-tested `shouldClearLatchOnQueryError`, which gates
// on the run NOT being active: it cures only the genuine permanent-null-freeze
// (`run === null`) and never releases against an active run.
useEffect(() => {
if (
shouldClearLatchOnQueryError({
stoppingRun,
isLocalStreaming: localStreaming,
runQueryFailed,
run,
})
)
setStoppingRun(false);
}, [stoppingRun, localStreaming, runQueryFailed, run]);
// The run's incrementally-persisted assistant message to merge into the thread,
// but only while we are an observer (never when we are the streamer — guards
// against a stale poll fighting the live stream). Includes a terminal run so the
// final persisted output is shown on reopen.
const observedRow =
shouldObserveRun(run, localStreaming) && !stoppingRun
? (runData?.message ?? null)
: null;
// When the observed run reaches a terminal status, do a final messages refetch
// so the persisted final state (token/context badge, export source) is shown,
// then the query's refetchInterval has already stopped polling. Deduped per run
// id so it fires exactly once per run, not on every subsequent poll-less render.
const finalizedRunIdRef = useRef<string | null>(null);
useEffect(() => {
if (!run || !activeChatId) return;
if (run.status === "pending" || run.status === "running") {
// Active again (a new run) — re-arm so its terminal transition fires once.
finalizedRunIdRef.current = null;
return;
}
// Terminal: a stop we requested has landed (or the run finished on its own),
// so release the stopping latch — the observer merge can now show the final
// persisted (aborted/finished) output without any live re-stream. The decision
// is the pure, unit-tested `shouldClearStoppingLatch` (run-polling.ts): release
// ONLY when we requested a stop, this tab is no longer the streamer, AND the
// CURRENT run is terminal. The #234 F4 cache removal in handleServerStop makes
// `run` null (this branch's `if (!run) return` above holds) until the current
// turn's run is fetched fresh, so the latch can never clear against a stale
// cached run.
if (shouldClearStoppingLatch({ stoppingRun, run, isLocalStreaming: localStreaming }))
setStoppingRun(false);
if (finalizedRunIdRef.current === run.id) return;
finalizedRunIdRef.current = run.id;
queryClient.invalidateQueries({
queryKey: AI_CHAT_MESSAGES_RQ_KEY(activeChatId),
});
}, [run, activeChatId, queryClient, stoppingRun, localStreaming]);
// The page the user is currently viewing. AiChatWindow lives in a pathless // The page the user is currently viewing. AiChatWindow lives in a pathless
// parent layout route, so useParams() can't see :pageSlug. Match the full // parent layout route, so useParams() can't see :pageSlug. Match the full
// pathname against the authenticated page route instead so "the current page" // pathname against the authenticated page route instead so "the current page"
@@ -403,6 +322,27 @@ export default function AiChatWindow() {
? { id: openPageData.id, title: openPageData.title } ? { id: openPageData.id, title: openPageData.title }
: null; : null;
// Live editor handles for the selection snapshot (#388). Both are published by
// the page editor; the read-only editor is used in read mode. Reading the
// selection off `editor.state` stays valid after the editor blurs (ProseMirror
// keeps state.selection), mirroring the comment button (comment-dialog.tsx).
const pageEditor = useAtomValue(pageEditorAtom);
const readOnlyEditor = useAtomValue(readOnlyEditorAtom);
// Snapshot the user's current editor selection at send time. Edit-mode editor
// wins; the read-only editor is the fallback (read mode). Null when neither
// holds a non-empty selection. Passed to <ChatThread>, which reads it live
// from a ref inside prepareSendMessagesRequest — so each turn ships a fresh
// snapshot and multi-turn works without recreating the transport.
const getEditorSelection = useCallback((): EditorSelectionContext | null => {
for (const editor of [pageEditor, readOnlyEditor]) {
if (!editor || editor.isDestroyed) continue;
const sel = getEditorSelectionContext(editor.state);
if (sel) return sel;
}
return null;
}, [pageEditor, readOnlyEditor]);
// The AI-chat thread-identity lifecycle (mount key, both new-chat id adoption // The AI-chat thread-identity lifecycle (mount key, both new-chat id adoption
// paths, the history-loaded latch, the render-phase reconciler) lives in this // paths, the history-loaded latch, the render-phase reconciler) lives in this
// hook. See adopt-chat-id.ts for the canonical #137 two-tab race explanation. // hook. See adopt-chat-id.ts for the canonical #137 two-tab race explanation.
@@ -1025,6 +965,9 @@ export default function AiChatWindow() {
chatId={activeChatId} chatId={activeChatId}
initialRows={activeChatId ? messageRows : []} initialRows={activeChatId ? messageRows : []}
openPage={openPage} openPage={openPage}
// #388: live snapshotter for the user's editor selection, read at
// send time and nested inside openPage on the wire.
getEditorSelection={getEditorSelection}
// Honoured only for a new chat; null = universal assistant. // Honoured only for a new chat; null = universal assistant.
roleId={activeChatId === null ? selectedRoleId : null} roleId={activeChatId === null ? selectedRoleId : null}
// Role cards are the new-chat empty-state; offered only when this // Role cards are the new-chat empty-state; offered only when this
@@ -1034,16 +977,13 @@ export default function AiChatWindow() {
assistantName={currentRole?.name} assistantName={currentRole?.name}
onTurnFinished={onTurnFinished} onTurnFinished={onTurnFinished}
onServerChatId={onServerChatId} onServerChatId={onServerChatId}
// #184: live-follow a still-running run when we reopened the chat as // #184 phase 1.5: arm/disarm the degraded-poll fallback when a
// a passive observer; null when there is nothing to observe or this // resume attempt could not attach to the live run; the thread
// tab is the streamer. onStreamingChange lets the window stop polling // disarms it on settle / local stream.
// while we are the streamer. onResumeFallback={onResumeFallback}
observedRow={observedRow}
onStreamingChange={onStreamingChange}
// #184: in autonomous mode the Stop button must hit the authoritative // #184: in autonomous mode the Stop button must hit the authoritative
// server stop (a local SSE abort is a client disconnect the server // server stop (a local SSE abort is a client disconnect the server
// ignores). onServerStop also arms the "stopping" latch above so the // ignores).
// stopped run's output does not re-stream via the observer merge.
autonomousRunsEnabled={autonomousRunsEnabled} autonomousRunsEnabled={autonomousRunsEnabled}
onServerStop={handleServerStop} onServerStop={handleServerStop}
/> />
@@ -1,6 +1,13 @@
import { describe, it, expect, beforeEach, vi } from "vitest"; import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
import { render, screen, fireEvent, act, cleanup } from "@testing-library/react"; import {
render,
screen,
fireEvent,
act,
cleanup,
} from "@testing-library/react";
import { MantineProvider } from "@mantine/core"; import { MantineProvider } from "@mantine/core";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
// Shared, hoisted mock state so the @ai-sdk/react and "ai" module mocks (hoisted // Shared, hoisted mock state so the @ai-sdk/react and "ai" module mocks (hoisted
// above the imports) can expose the captured useChat callbacks / transport and // above the imports) can expose the captured useChat callbacks / transport and
@@ -12,50 +19,61 @@ const h = vi.hoisted(() => ({
sendMessage: vi.fn(), sendMessage: vi.fn(),
stop: vi.fn(), stop: vi.fn(),
setMessages: vi.fn(), setMessages: vi.fn(),
resumeStream: vi.fn(),
// The messages array useChat was seeded with (to assert strip/seed behavior).
seededMessages: null as null | unknown[],
transport: null as null | { transport: null as null | {
prepareSendMessagesRequest: (arg: { prepareSendMessagesRequest?: (arg: {
messages: unknown[]; messages: unknown[];
body: Record<string, unknown>; body: Record<string, unknown>;
}) => { body: Record<string, unknown> }; }) => { body: Record<string, unknown> };
prepareReconnectToStreamRequest?: () => { api?: string };
fetch?: (input: unknown, init?: { method?: string }) => Promise<unknown>;
}, },
}, },
})); }));
// Mock useChat: capture onFinish, return the spies and the controllable status. // Mock useChat: capture onFinish + seeded messages, return the spies and the
// controllable status.
vi.mock("@ai-sdk/react", () => ({ vi.mock("@ai-sdk/react", () => ({
useChat: (opts: { onFinish?: (arg: Record<string, unknown>) => void }) => { useChat: (opts: {
messages?: unknown[];
onFinish?: (arg: Record<string, unknown>) => void;
}) => {
h.state.onFinish = opts.onFinish ?? null; h.state.onFinish = opts.onFinish ?? null;
h.state.seededMessages = opts.messages ?? null;
return { return {
messages: [], messages: [],
sendMessage: h.state.sendMessage, sendMessage: h.state.sendMessage,
status: h.state.status, status: h.state.status,
stop: h.state.stop, stop: h.state.stop,
error: null, error: null,
// #184: ChatThread reads setMessages to merge a polled observer run.
setMessages: h.state.setMessages, setMessages: h.state.setMessages,
resumeStream: h.state.resumeStream,
}; };
}, },
})); }));
// Mock "ai": deterministic ids + a transport that records its options so the test // Mock "ai": deterministic ids + a transport that records its options so the test
// can invoke prepareSendMessagesRequest and assert the `interrupted` flag. // can invoke prepareSendMessagesRequest / prepareReconnectToStreamRequest / fetch.
vi.mock("ai", () => { vi.mock("ai", () => {
let counter = 0; let counter = 0;
return { return {
generateId: () => `gid-${counter++}`, generateId: () => `gid-${counter++}`,
DefaultChatTransport: class { DefaultChatTransport: class {
constructor(opts: { constructor(opts: Record<string, unknown>) {
prepareSendMessagesRequest: (arg: { h.state.transport = opts as never;
messages: unknown[];
body: Record<string, unknown>;
}) => { body: Record<string, unknown> };
}) {
h.state.transport = opts;
} }
}, },
}; };
}); });
// Keep the ai-chat-query import light: ChatThread only needs the messages RQ key,
// so stub the module to avoid pulling axios / i18n transitively.
vi.mock("@/features/ai-chat/queries/ai-chat-query.ts", () => ({
AI_CHAT_MESSAGES_RQ_KEY: (chatId: string) => ["ai-chat-messages", chatId],
}));
// Stub the heavy children: MessageList (markdown/render) and ChatInput (the // Stub the heavy children: MessageList (markdown/render) and ChatInput (the
// composer). The ChatInput stub exposes a button that queues a message, the only // composer). The ChatInput stub exposes a button that queues a message, the only
// interaction this test needs to populate the queue while "streaming". // interaction this test needs to populate the queue while "streaming".
@@ -63,49 +81,90 @@ vi.mock("@/features/ai-chat/components/message-list.tsx", () => ({
default: () => <div data-testid="message-list" />, default: () => <div data-testid="message-list" />,
})); }));
vi.mock("@/features/ai-chat/components/chat-input.tsx", () => ({ vi.mock("@/features/ai-chat/components/chat-input.tsx", () => ({
default: ({ onQueue }: { onQueue: (text: string) => void }) => ( default: ({
<button data-testid="queue-btn" onClick={() => onQueue("queued text")}> onQueue,
queue onStop,
</button> }: {
onQueue: (text: string) => void;
onStop: () => void;
}) => (
<>
<button data-testid="queue-btn" onClick={() => onQueue("queued text")}>
queue
</button>
<button aria-label="Stop" onClick={() => onStop()}>
stop
</button>
</>
), ),
})); }));
import ChatThread from "./chat-thread"; import ChatThread from "./chat-thread";
import type { IAiChatMessageRow } from "@/features/ai-chat/types/ai-chat.types.ts";
function renderThread() { function row(
id: string,
role: string,
status?: string,
text = "",
): IAiChatMessageRow {
return { id, role, content: text, status, createdAt: "2026-01-01T00:00:00Z" };
}
function renderThread(props?: {
chatId?: string | null;
initialRows?: IAiChatMessageRow[];
autonomousRunsEnabled?: boolean;
}) {
const onTurnFinished = vi.fn(); const onTurnFinished = vi.fn();
render( const onResumeFallback = vi.fn();
<MantineProvider> const onServerStop = vi.fn();
<ChatThread chatId="c1" initialRows={[]} onTurnFinished={onTurnFinished} /> const queryClient = new QueryClient({
</MantineProvider>, defaultOptions: { queries: { retry: false } },
});
const invalidateSpy = vi.spyOn(queryClient, "invalidateQueries");
const { unmount } = render(
<QueryClientProvider client={queryClient}>
<MantineProvider>
<ChatThread
chatId={props?.chatId === undefined ? "c1" : props.chatId}
initialRows={props?.initialRows ?? []}
autonomousRunsEnabled={props?.autonomousRunsEnabled}
onTurnFinished={onTurnFinished}
onResumeFallback={onResumeFallback}
onServerStop={onServerStop}
/>
</MantineProvider>
</QueryClientProvider>,
); );
return { onTurnFinished }; return { onTurnFinished, onResumeFallback, onServerStop, invalidateSpy, unmount };
}
function resetState() {
h.state.status = "streaming";
h.state.onFinish = null;
h.state.seededMessages = null;
h.state.transport = null;
h.state.sendMessage.mockClear();
h.state.stop.mockClear();
h.state.setMessages.mockClear();
h.state.resumeStream.mockClear();
} }
describe("ChatThread — send now (#198)", () => { describe("ChatThread — send now (#198)", () => {
beforeEach(() => { beforeEach(resetState);
h.state.status = "streaming";
h.state.onFinish = null;
h.state.sendMessage.mockClear();
h.state.stop.mockClear();
h.state.transport = null;
});
it("aborts the current turn and resends the queued message on the abort", () => { it("aborts the current turn and resends the queued message on the abort", () => {
renderThread(); renderThread();
// Queue a message while the turn is streaming.
fireEvent.click(screen.getByTestId("queue-btn")); fireEvent.click(screen.getByTestId("queue-btn"));
const sendNowBtn = screen.getByLabelText("Send now"); const sendNowBtn = screen.getByLabelText("Send now");
expect(sendNowBtn).toBeTruthy(); expect(sendNowBtn).toBeTruthy();
// "Send now" interrupts the current turn (stop), but does NOT send yet —
// the resend happens once the abort lands in onFinish.
fireEvent.click(sendNowBtn); fireEvent.click(sendNowBtn);
expect(h.state.stop).toHaveBeenCalledTimes(1); expect(h.state.stop).toHaveBeenCalledTimes(1);
expect(h.state.sendMessage).not.toHaveBeenCalled(); expect(h.state.sendMessage).not.toHaveBeenCalled();
// The abort we triggered reaches onFinish: the promoted head is flushed.
act(() => { act(() => {
h.state.onFinish?.({ h.state.onFinish?.({
message: { id: "a", role: "assistant", parts: [] }, message: { id: "a", role: "assistant", parts: [] },
@@ -122,10 +181,8 @@ describe("ChatThread — send now (#198)", () => {
fireEvent.click(screen.getByTestId("queue-btn")); fireEvent.click(screen.getByTestId("queue-btn"));
fireEvent.click(screen.getByLabelText("Send now")); fireEvent.click(screen.getByLabelText("Send now"));
const prep = h.state.transport!.prepareSendMessagesRequest; const prep = h.state.transport!.prepareSendMessagesRequest!;
// The send right after "send now" carries interrupted: true...
expect(prep({ messages: [], body: {} }).body.interrupted).toBe(true); expect(prep({ messages: [], body: {} }).body.interrupted).toBe(true);
// ...and only that one (the flag is read-and-cleared).
expect(prep({ messages: [], body: {} }).body.interrupted).toBe(false); expect(prep({ messages: [], body: {} }).body.interrupted).toBe(false);
}); });
@@ -136,42 +193,92 @@ describe("ChatThread — send now (#198)", () => {
fireEvent.click(screen.getByTestId("queue-btn")); fireEvent.click(screen.getByTestId("queue-btn"));
fireEvent.click(screen.getByLabelText("Send now")); fireEvent.click(screen.getByLabelText("Send now"));
// No turn to interrupt: sent straight away, no abort, not flagged.
expect(h.state.stop).not.toHaveBeenCalled(); expect(h.state.stop).not.toHaveBeenCalled();
expect(h.state.sendMessage).toHaveBeenCalledWith({ text: "queued text" }); expect(h.state.sendMessage).toHaveBeenCalledWith({ text: "queued text" });
const prep = h.state.transport!.prepareSendMessagesRequest; const prep = h.state.transport!.prepareSendMessagesRequest!;
expect(prep({ messages: [], body: {} }).body.interrupted).toBe(false); expect(prep({ messages: [], body: {} }).body.interrupted).toBe(false);
}); });
}); });
// The turn-end decision lives in the `onFinish` handler: given the terminal // #388: the editor selection is snapshotted at send time and nested inside
// outcome of a turn (`isAbort` / `isDisconnect` / `isError`, or none = clean), // openPage on the wire. The getter is read live from a ref, so each send ships a
// it decides whether to CONTINUE (flush the next queued message) or END (leave // fresh snapshot.
// the queue intact for the user), and which stop notice — if any — to show. describe("ChatThread — editor selection wiring (#388)", () => {
// `sendNow` is exercised above; these tests pin down the plain outcomes. beforeEach(resetState);
describe("ChatThread — turn-end decision (onFinish)", () => { afterEach(cleanup);
beforeEach(() => {
h.state.status = "streaming"; function renderWithSelection(props: {
h.state.onFinish = null; openPage?: { id: string; title: string } | null;
h.state.sendMessage.mockClear(); getEditorSelection?: () => unknown;
h.state.stop.mockClear(); }) {
h.state.transport = null; const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false } },
});
render(
<QueryClientProvider client={queryClient}>
<MantineProvider>
<ChatThread
chatId="c1"
initialRows={[]}
openPage={props.openPage as never}
getEditorSelection={props.getEditorSelection as never}
onTurnFinished={vi.fn()}
onResumeFallback={vi.fn()}
onServerStop={vi.fn()}
/>
</MantineProvider>
</QueryClientProvider>,
);
}
it("nests the snapshot from the getter into openPage.selection at send time", () => {
const selection = { text: "fix this", blockIds: ["b1"], before: "a " };
renderWithSelection({
openPage: { id: "p1", title: "Doc" },
getEditorSelection: () => selection,
});
const prep = h.state.transport!.prepareSendMessagesRequest!;
const openPage = prep({ messages: [], body: {} }).body.openPage as Record<
string,
unknown
>;
expect(openPage).toEqual({ id: "p1", title: "Doc", selection });
}); });
// Drive a fresh onFinish with the given terminal flags after queueing a it("sends selection: null when the getter returns null", () => {
// message, and report both what the parent was told and whether the queue was renderWithSelection({
// flushed (a resend to the sendMessage spy). openPage: { id: "p1", title: "Doc" },
getEditorSelection: () => null,
});
const prep = h.state.transport!.prepareSendMessagesRequest!;
const openPage = prep({ messages: [], body: {} }).body.openPage as Record<
string,
unknown
>;
expect(openPage).toEqual({ id: "p1", title: "Doc", selection: null });
});
it("does not send selection at all on a non-page route (openPage null)", () => {
const getter = vi.fn(() => ({ text: "sel" }));
renderWithSelection({ openPage: null, getEditorSelection: getter });
const prep = h.state.transport!.prepareSendMessagesRequest!;
expect(prep({ messages: [], body: {} }).body.openPage).toBeNull();
// The getter must not even be consulted when there is no page.
expect(getter).not.toHaveBeenCalled();
});
});
describe("ChatThread — turn-end decision (onFinish)", () => {
beforeEach(resetState);
function finishWith(flags: { function finishWith(flags: {
isAbort?: boolean; isAbort?: boolean;
isDisconnect?: boolean; isDisconnect?: boolean;
isError?: boolean; isError?: boolean;
}) { }) {
// Tear down any prior render so the loop-driven "every outcome" case does
// not leave duplicate queue buttons in the DOM.
cleanup(); cleanup();
h.state.sendMessage.mockClear(); h.state.sendMessage.mockClear();
const { onTurnFinished } = renderThread(); const { onTurnFinished } = renderThread();
// Populate the queue while the turn is streaming.
fireEvent.click(screen.getByTestId("queue-btn")); fireEvent.click(screen.getByTestId("queue-btn"));
act(() => { act(() => {
h.state.onFinish?.({ h.state.onFinish?.({
@@ -187,16 +294,12 @@ describe("ChatThread — turn-end decision (onFinish)", () => {
it("CONTINUES — flushes the next queued message on a clean finish", () => { it("CONTINUES — flushes the next queued message on a clean finish", () => {
finishWith({}); finishWith({});
// Clean finish (no terminal flag): the queued message is auto-sent.
expect(h.state.sendMessage).toHaveBeenCalledWith({ text: "queued text" }); expect(h.state.sendMessage).toHaveBeenCalledWith({ text: "queued text" });
// A clean finish shows no stop notice.
expect(screen.queryByText("Response stopped.")).toBeNull(); expect(screen.queryByText("Response stopped.")).toBeNull();
}); });
it("ENDS — keeps the queue intact on a user abort and shows the stopped notice", () => { it("ENDS — keeps the queue intact on a user abort and shows the stopped notice", () => {
finishWith({ isAbort: true }); finishWith({ isAbort: true });
// A plain Stop (not the sendNow interrupt path) must NOT auto-resend: the
// queue is preserved for the user to decide.
expect(h.state.sendMessage).not.toHaveBeenCalled(); expect(h.state.sendMessage).not.toHaveBeenCalled();
expect(screen.getByText("Response stopped.")).toBeTruthy(); expect(screen.getByText("Response stopped.")).toBeTruthy();
}); });
@@ -211,15 +314,11 @@ describe("ChatThread — turn-end decision (onFinish)", () => {
it("ENDS — keeps the queue intact on a stream error (no auto-retry, no stopped notice)", () => { it("ENDS — keeps the queue intact on a stream error (no auto-retry, no stopped notice)", () => {
finishWith({ isError: true }); finishWith({ isError: true });
// Blindly retrying after a failure would be wrong; the queue is left alone.
expect(h.state.sendMessage).not.toHaveBeenCalled(); expect(h.state.sendMessage).not.toHaveBeenCalled();
// isError clears the neutral notice (the error banner covers this case).
expect(screen.queryByText("Response stopped.")).toBeNull(); expect(screen.queryByText("Response stopped.")).toBeNull();
}); });
it("notifies the parent on EVERY terminal outcome", () => { it("notifies the parent on EVERY terminal outcome", () => {
// The chat-list refresh / new-chat id adoption must run on success and on
// every failure path alike.
for (const flags of [ for (const flags of [
{}, {},
{ isAbort: true }, { isAbort: true },
@@ -232,55 +331,411 @@ describe("ChatThread — turn-end decision (onFinish)", () => {
}); });
}); });
// #184 passive-observer merge: when reconnecting to a still-running run, the // #184 phase 1.5: the resumable-SSE client. A reopened tab resumes the live run
// parent feeds the polled run message via `observedRow`; ChatThread merges it via // via the SDK's reconnect transport (attach: replay + tail) instead of polling.
// setMessages — but ONLY when this tab is NOT itself streaming (the streamer's describe("ChatThread — resume (attach) machinery (#184)", () => {
// SSE owns the view, so a stale observedRow must never overwrite it). const streamingTail = () => [
describe("ChatThread — observer run merge (#184)", () => { row("u1", "user", undefined, "hi"),
beforeEach(() => { row("a1", "assistant", "streaming", "partial"),
h.state.onFinish = null; ];
h.state.setMessages.mockReset(); const settledTail = () => [
row("u1", "user", undefined, "hi"),
row("a1", "assistant", "succeeded", "done"),
];
const userTail = () => [row("u1", "user", undefined, "hi")];
const visibleMsg = {
id: "a1",
role: "assistant",
parts: [{ type: "text", text: "streamed answer" }],
};
const emptyMsg = { id: "a1", role: "assistant", parts: [] };
beforeEach(resetState);
// NOTE: do NOT vi.unstubAllGlobals() here — vitest.setup.ts installs
// matchMedia/localStorage via vi.stubGlobal and unstubbing wipes them for the
// rest of the file. Fetch is re-stubbed per test that needs it.
afterEach(cleanup);
it("resumes on mount only when the flag is on, chatId is set, and the tail is not a settled assistant", () => {
// streaming tail -> resume
renderThread({ autonomousRunsEnabled: true, initialRows: streamingTail() });
expect(h.state.resumeStream).toHaveBeenCalledTimes(1);
// user tail -> resume (the assistant row may not be seeded yet)
cleanup();
h.state.resumeStream.mockClear();
renderThread({ autonomousRunsEnabled: true, initialRows: userTail() });
expect(h.state.resumeStream).toHaveBeenCalledTimes(1);
// settled assistant tail -> NO resume
cleanup();
h.state.resumeStream.mockClear();
renderThread({ autonomousRunsEnabled: true, initialRows: settledTail() });
expect(h.state.resumeStream).not.toHaveBeenCalled();
// flag off -> NO resume
cleanup();
h.state.resumeStream.mockClear();
renderThread({ autonomousRunsEnabled: false, initialRows: streamingTail() });
expect(h.state.resumeStream).not.toHaveBeenCalled();
// no chatId -> NO resume
cleanup();
h.state.resumeStream.mockClear();
renderThread({
autonomousRunsEnabled: true,
chatId: null,
initialRows: streamingTail(),
});
expect(h.state.resumeStream).not.toHaveBeenCalled();
}); });
const observedRow = { it("strips the streaming tail from the seed, but keeps a user tail whole", () => {
id: "a-run", renderThread({ autonomousRunsEnabled: true, initialRows: streamingTail() });
role: "assistant", // 2 rows in, streaming tail stripped -> 1 seeded message.
content: "step 1\nstep 2", expect(h.state.seededMessages).toHaveLength(1);
metadata: {
parts: [{ type: "text", text: "step 1\nstep 2" }],
},
createdAt: "2026-01-01T00:00:00Z",
} as const;
function renderObserver(status: string) { cleanup();
h.state.status = status; renderThread({ autonomousRunsEnabled: true, initialRows: userTail() });
render( // user tail is not stripped.
expect(h.state.seededMessages).toHaveLength(1);
});
it("builds the attach URL with expect=live&anchor only when the streaming tail was stripped", () => {
renderThread({ autonomousRunsEnabled: true, initialRows: streamingTail() });
expect(h.state.transport!.prepareReconnectToStreamRequest!().api).toBe(
"/api/ai-chat/runs/c1/stream?expect=live&anchor=a1",
);
cleanup();
renderThread({ autonomousRunsEnabled: true, initialRows: userTail() });
expect(h.state.transport!.prepareReconnectToStreamRequest!().api).toBe(
"/api/ai-chat/runs/c1/stream",
);
});
async function fetch204() {
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue({ status: 204, ok: false }),
);
await act(async () => {
await h.state.transport!.fetch!("http://x", { method: "GET" });
});
}
it("204 on a user tail: no crash, no restore, reconcile+invalidate, onResumeFallback(true)", async () => {
const { onResumeFallback, invalidateSpy } = renderThread({
autonomousRunsEnabled: true,
initialRows: userTail(),
});
await fetch204();
// No stripped row -> no restore merge.
expect(h.state.setMessages).not.toHaveBeenCalled();
expect(invalidateSpy).toHaveBeenCalledWith({
queryKey: ["ai-chat-messages", "c1"],
});
expect(onResumeFallback).toHaveBeenCalledWith(true);
});
it("204 on a streaming tail: restore + invalidate + onResumeFallback(true)", async () => {
const { onResumeFallback, invalidateSpy } = renderThread({
autonomousRunsEnabled: true,
initialRows: streamingTail(),
});
await fetch204();
// Stripped row is restored to the store.
expect(h.state.setMessages).toHaveBeenCalledTimes(1);
expect(invalidateSpy).toHaveBeenCalledWith({
queryKey: ["ai-chat-messages", "c1"],
});
expect(onResumeFallback).toHaveBeenCalledWith(true);
});
it("F7 restart-survival: a 500 attach failure restores the stripped row AND arms the poll (not lost)", async () => {
const { onResumeFallback, invalidateSpy } = renderThread({
autonomousRunsEnabled: true,
initialRows: streamingTail(),
});
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue({ status: 500, ok: false }),
);
await act(async () => {
await h.state.transport!.fetch!("http://x", { method: "GET" });
});
expect(h.state.setMessages).toHaveBeenCalledTimes(1); // stripped row restored
expect(invalidateSpy).toHaveBeenCalledWith({
queryKey: ["ai-chat-messages", "c1"],
});
expect(onResumeFallback).toHaveBeenCalledWith(true); // degraded poll armed
});
it("F7 restart-survival: a network throw restores the stripped row AND arms the poll", async () => {
const { onResumeFallback, invalidateSpy } = renderThread({
autonomousRunsEnabled: true,
initialRows: streamingTail(),
});
vi.stubGlobal(
"fetch",
vi.fn().mockRejectedValue(new Error("network down")),
);
await act(async () => {
await h.state
.transport!.fetch!("http://x", { method: "GET" })
.catch(() => undefined); // the wrapper rethrows; swallow here
});
expect(h.state.setMessages).toHaveBeenCalledTimes(1);
expect(invalidateSpy).toHaveBeenCalledWith({
queryKey: ["ai-chat-messages", "c1"],
});
expect(onResumeFallback).toHaveBeenCalledWith(true);
});
it("unmount during a pending attach aborts the controller and gates late callbacks", async () => {
const { onResumeFallback, invalidateSpy, unmount } = renderThread({
autonomousRunsEnabled: true,
initialRows: streamingTail(),
});
let abortSeen = false;
let resolveFetch!: (v: unknown) => void;
vi.stubGlobal(
"fetch",
vi.fn().mockImplementation((_input: unknown, init: RequestInit) => {
init.signal?.addEventListener("abort", () => {
abortSeen = true;
});
return new Promise((res) => {
resolveFetch = res;
});
}),
);
// Kick a reconnect GET (stays pending).
let pending!: Promise<unknown>;
act(() => {
pending = h.state.transport!.fetch!("http://x", { method: "GET" });
});
// Unmount: the cleanup aborts the in-flight attach.
unmount();
expect(abortSeen).toBe(true);
// A late 204 landing after unmount must NOT arm a poll / invalidate the (now
// different) chat.
onResumeFallback.mockClear();
invalidateSpy.mockClear();
await act(async () => {
resolveFetch({ status: 204, ok: false });
await pending;
});
expect(onResumeFallback).not.toHaveBeenCalledWith(true);
expect(invalidateSpy).not.toHaveBeenCalled();
});
it("a resume fetch error clears resumedTurn so the next local turn flushes the queue", async () => {
renderThread({ autonomousRunsEnabled: true, initialRows: streamingTail() });
h.state.status = "ready";
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue({ status: 500, ok: false }),
);
await act(async () => {
await h.state.transport!.fetch!("http://x", { method: "GET" });
});
// Queue then clean-finish: suppression was cleared, so the queue flushes.
fireEvent.click(screen.getByTestId("queue-btn"));
act(() => {
h.state.onFinish?.({
message: visibleMsg,
isAbort: false,
isDisconnect: false,
isError: false,
});
});
expect(h.state.sendMessage).toHaveBeenCalledWith({ text: "queued text" });
});
it("a resumed turn's onFinish does NOT flush the queue", () => {
renderThread({ autonomousRunsEnabled: true, initialRows: streamingTail() });
fireEvent.click(screen.getByTestId("queue-btn"));
act(() => {
h.state.onFinish?.({
message: visibleMsg,
isAbort: false,
isDisconnect: false,
isError: false,
});
});
expect(h.state.sendMessage).not.toHaveBeenCalled();
});
it("a healthy resumed finish (visible content) arms nothing and keeps the store", () => {
h.state.status = "ready";
const { onResumeFallback } = renderThread({
autonomousRunsEnabled: true,
initialRows: streamingTail(),
});
h.state.setMessages.mockClear();
onResumeFallback.mockClear();
act(() => {
h.state.onFinish?.({
message: visibleMsg,
isAbort: false,
isDisconnect: false,
isError: false,
});
});
// No restore (would clobber the fuller streamed message), no poll arm.
expect(h.state.setMessages).not.toHaveBeenCalled();
expect(onResumeFallback).not.toHaveBeenCalledWith(true);
});
it("isDisconnect WITH visible content arms the poll but does NOT restore", () => {
h.state.status = "ready";
const { onResumeFallback, invalidateSpy } = renderThread({
autonomousRunsEnabled: true,
initialRows: streamingTail(),
});
h.state.setMessages.mockClear();
onResumeFallback.mockClear();
invalidateSpy.mockClear();
act(() => {
h.state.onFinish?.({
message: visibleMsg,
isAbort: false,
isDisconnect: true,
isError: false,
});
});
expect(onResumeFallback).toHaveBeenCalledWith(true);
expect(invalidateSpy).toHaveBeenCalledWith({
queryKey: ["ai-chat-messages", "c1"],
});
// Restore forbidden: the on-screen partial must not roll back.
expect(h.state.setMessages).not.toHaveBeenCalled();
});
it("an empty resumed message (starved replay) restores the stripped row AND arms the poll", () => {
h.state.status = "ready";
const { onResumeFallback } = renderThread({
autonomousRunsEnabled: true,
initialRows: streamingTail(),
});
h.state.setMessages.mockClear();
onResumeFallback.mockClear();
act(() => {
h.state.onFinish?.({
message: emptyMsg,
isAbort: false,
isDisconnect: false,
isError: false,
});
});
expect(h.state.setMessages).toHaveBeenCalledTimes(1); // restore
expect(onResumeFallback).toHaveBeenCalledWith(true); // arm
});
it("degraded-merge: merges the tail per initialRows update, and settles disarm the poll", async () => {
h.state.status = "ready";
const { rerender, onResumeFallback } = renderResumable(streamingTail());
// Arm reconcile via a 204.
vi.stubGlobal(
"fetch",
vi.fn().mockResolvedValue({ status: 204, ok: false }),
);
await act(async () => {
await h.state.transport!.fetch!("http://x", { method: "GET" });
});
h.state.setMessages.mockClear();
onResumeFallback.mockClear();
// A streaming-tail update: merge, poll stays armed.
rerender([
row("u1", "user", undefined, "hi"),
row("a1", "assistant", "streaming", "step 1\nstep 2"),
]);
expect(h.state.setMessages).toHaveBeenCalledTimes(1);
expect(onResumeFallback).not.toHaveBeenCalledWith(false);
// A settled-tail update: merge + disarm.
h.state.setMessages.mockClear();
rerender([
row("u1", "user", undefined, "hi"),
row("a1", "assistant", "succeeded", "final"),
]);
expect(h.state.setMessages).toHaveBeenCalledTimes(1);
expect(onResumeFallback).toHaveBeenCalledWith(false);
});
it("a local stream disarms both the merge and the poll", () => {
h.state.status = "streaming";
const { rerender, onResumeFallback } = renderResumable(streamingTail());
onResumeFallback.mockClear();
// A re-render while streaming: the reconciliation effect disarms.
rerender(streamingTail());
expect(onResumeFallback).toHaveBeenCalledWith(false);
});
it("Send now is hidden on a resumed turn but visible on a local stream", () => {
// Resumed turn: hidden.
renderThread({ autonomousRunsEnabled: true, initialRows: streamingTail() });
fireEvent.click(screen.getByTestId("queue-btn"));
expect(screen.queryByLabelText("Send now")).toBeNull();
// Local streaming turn (no resume): visible.
cleanup();
resetState();
renderThread({ initialRows: [] });
fireEvent.click(screen.getByTestId("queue-btn"));
expect(screen.getByLabelText("Send now")).toBeTruthy();
});
it("handleStop aborts the attach controller and calls onServerStop", async () => {
const { onServerStop } = renderThread({
autonomousRunsEnabled: true,
initialRows: streamingTail(),
});
// Establish an attach controller via a (pending) reconnect GET.
let abortSeen = false;
vi.stubGlobal(
"fetch",
vi.fn().mockImplementation((_input: unknown, init: RequestInit) => {
init.signal?.addEventListener("abort", () => {
abortSeen = true;
});
return new Promise(() => undefined); // never resolves
}),
);
act(() => {
void h.state.transport!.fetch!("http://x", { method: "GET" });
});
fireEvent.click(screen.getByLabelText("Stop"));
expect(abortSeen).toBe(true);
expect(onServerStop).toHaveBeenCalledWith("c1");
});
});
// Helper: render a resumable thread and expose a rerender that only swaps
// initialRows (the degraded-merge effect depends on it).
function renderResumable(initialRows: IAiChatMessageRow[]) {
const onResumeFallback = vi.fn();
const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false } },
});
const Wrapper = ({ rows }: { rows: IAiChatMessageRow[] }) => (
<QueryClientProvider client={queryClient}>
<MantineProvider> <MantineProvider>
<ChatThread <ChatThread
chatId="c1" chatId="c1"
initialRows={[]} initialRows={rows}
autonomousRunsEnabled
onTurnFinished={vi.fn()} onTurnFinished={vi.fn()}
observedRow={observedRow as never} onResumeFallback={onResumeFallback}
/> />
</MantineProvider>, </MantineProvider>
); </QueryClientProvider>
} );
const view = render(<Wrapper rows={initialRows} />);
it("merges the polled run message when this tab is a passive observer", () => { const rerender = (rows: IAiChatMessageRow[]) =>
renderObserver("ready"); act(() => view.rerender(<Wrapper rows={rows} />));
expect(h.state.setMessages).toHaveBeenCalledTimes(1); return { rerender, onResumeFallback };
// The updater replaces/append the observed assistant row by id. }
const updater = h.state.setMessages.mock.calls[0][0] as (
prev: { id: string; parts: { text: string }[] }[],
) => { id: string; parts: { text: string }[] }[];
const merged = updater([{ id: "u1", parts: [{ text: "hi" }] }]);
expect(merged).toHaveLength(2);
expect(merged[1].id).toBe("a-run");
expect(merged[1].parts[0].text).toBe("step 1\nstep 2");
});
it("does NOT merge while THIS tab is the streamer (no double-render)", () => {
renderObserver("streaming");
expect(h.state.setMessages).not.toHaveBeenCalled();
});
});
@@ -1,4 +1,5 @@
import { useCallback, useEffect, useMemo, useRef, useState } from "react"; import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { useQueryClient } from "@tanstack/react-query";
import { generateId } from "ai"; import { generateId } from "ai";
import { ActionIcon, Box, Group, Stack, Text, Tooltip } from "@mantine/core"; import { ActionIcon, Box, Group, Stack, Text, Tooltip } from "@mantine/core";
import { import {
@@ -24,7 +25,15 @@ import {
} from "@/features/ai-chat/utils/role-launch.ts"; } from "@/features/ai-chat/utils/role-launch.ts";
import { describeChatError } from "@/features/ai-chat/utils/error-message.ts"; import { describeChatError } from "@/features/ai-chat/utils/error-message.ts";
import { extractServerChatId } from "@/features/ai-chat/utils/adopt-chat-id.ts"; import { extractServerChatId } from "@/features/ai-chat/utils/adopt-chat-id.ts";
import { mergeObservedMessage } from "@/features/ai-chat/utils/run-polling.ts"; import { assistantMessageHasVisibleContent } from "@/features/ai-chat/utils/message-content.ts";
import {
isStreamingTail,
isSettledAssistantTail,
seedRows,
mergeById,
} from "@/features/ai-chat/utils/resume-helpers.ts";
import { AI_CHAT_MESSAGES_RQ_KEY } from "@/features/ai-chat/queries/ai-chat-query.ts";
import type { EditorSelectionContext } from "@/features/editor/utils/get-editor-selection.ts";
import { import {
dequeue, dequeue,
enqueueMessage, enqueueMessage,
@@ -61,6 +70,10 @@ interface ChatThreadProps {
/** The page currently open in the workspace, or null on a non-page route. /** The page currently open in the workspace, or null on a non-page route.
* Sent with each turn so the agent knows what "this page" refers to. */ * Sent with each turn so the agent knows what "this page" refers to. */
openPage?: OpenPageContext | null; openPage?: OpenPageContext | null;
/** #388: snapshot the user's current editor selection at SEND time. Invoked
* inside prepareSendMessagesRequest and nested into openPage on the wire, so a
* fresh snapshot ships each turn. Null/absent => nothing selected. */
getEditorSelection?: () => EditorSelectionContext | null;
/** The agent role selected for a NEW chat (null = universal assistant). Sent /** The agent role selected for a NEW chat (null = universal assistant). Sent
* in the request body so the server persists it on chat creation; ignored by * in the request body so the server persists it on chat creation; ignored by
* the server for existing chats (the role is read from the chat row). */ * the server for existing chats (the role is read from the chat row). */
@@ -87,19 +100,13 @@ interface ChatThreadProps {
* Copy/export button available mid-stream). Distinct from onTurnFinished, * Copy/export button available mid-stream). Distinct from onTurnFinished,
* which fires only at the terminal outcome. */ * which fires only at the terminal outcome. */
onServerChatId?: (serverChatId?: string) => void; onServerChatId?: (serverChatId?: string) => void;
/** #184 reconnect-and-live-follow. When THIS tab reopened a chat whose agent /** #184 phase 1.5: arm/disarm the parent's degraded-poll fallback for THIS
* run is still going (it is a PASSIVE OBSERVER — it did not start the run here), * chat's window. Called `true` when a resume attempt could not attach to the
* the parent polls the reconnect endpoint and feeds the run's incrementally- * live run (attach 204 / starved-or-torn resumed finish), so the window starts
* persisted assistant message here; we merge it into the live list so new * a dumb timed poll of the message history to follow the detached run to settle;
* steps/tool-calls appear as they are persisted. Null when there is nothing to * called `false` the moment a local stream starts or the terminal settled row is
* observe (no run, feature off, or this tab IS the streamer). The merge is * merged (invariant 8). The window owns the timer + its 10-min cap. */
* ADDITIONALLY guarded by our own `isStreaming`, so a stale value can never onResumeFallback?: (active: boolean) => void;
* fight the local stream when we are the streamer. */
observedRow?: IAiChatMessageRow | null;
/** Report this tab's live streaming status up to the parent, so it can stop
* polling the run while WE are the active streamer (the SSE owns the view) and
* resume once we go idle. Called from an effect on every transition. */
onStreamingChange?: (streaming: boolean) => void;
/** #184: whether detached/autonomous agent runs are enabled for this workspace. /** #184: whether detached/autonomous agent runs are enabled for this workspace.
* When true the Stop button must additionally hit the AUTHORITATIVE server stop * When true the Stop button must additionally hit the AUTHORITATIVE server stop
* (via onServerStop) — aborting only the local SSE is just a client disconnect, * (via onServerStop) — aborting only the local SSE is just a client disconnect,
@@ -149,21 +156,65 @@ export default function ChatThread({
threadKey, threadKey,
initialRows, initialRows,
openPage, openPage,
getEditorSelection,
roleId, roleId,
roles, roles,
onRolePicked, onRolePicked,
assistantName, assistantName,
onTurnFinished, onTurnFinished,
onServerChatId, onServerChatId,
observedRow, onResumeFallback,
onStreamingChange,
autonomousRunsEnabled, autonomousRunsEnabled,
onServerStop, onServerStop,
}: ChatThreadProps) { }: ChatThreadProps) {
const { t } = useTranslation(); const { t } = useTranslation();
const queryClient = useQueryClient();
// resume machinery refs (#184 phase 1.5)
const attachAbortRef = useRef<AbortController | null>(null);
const reconcileTailRef = useRef(false);
const noStreamHandledRef = useRef(false);
const onNoActiveStreamRef = useRef<(() => void) | null>(null);
// Live mount flag. The attach GET and the resumed `onFinish` are async and can
// land AFTER this thread unmounts (the parent remounts per chat via `key`); with
// chatIdRef then pointing at the NEW chat, an ungated late callback would arm a
// spurious poll + foreign invalidation on the newly-opened chat. Every parent-
// facing resume side-effect is gated on this.
const mountedRef = useRef(true);
const [resumedTurn, setResumedTurn] = useState(false);
const resumedTurnRef = useRef(false);
// Identity-stable pair setter (bare useState setter + ref write): it is closed
// over by the transport useMemo([]), so it MUST NOT capture state.
const setResumedTurnPair = useCallback((v: boolean) => {
resumedTurnRef.current = v;
setResumedTurn(v);
}, []);
// Mount-time resume gating (in refs — computed once for this mount; the parent
// remounts per chat via `key`).
//
// Attempt resume for any non-settled tail: a streaming tail (strip + expect
// live replay) or a user tail (the run may exist but its assistant row is not
// seeded yet — attach to the pre-opened registry entry and wait for frames).
// A settled assistant tail must NEVER resume: replaying a finished run into a
// store that already contains its message duplicates parts (SDK text-start
// always pushes a new part).
const stripRef = useRef(chatId !== null && isStreamingTail(initialRows ?? []));
const attemptResumeRef = useRef(
autonomousRunsEnabled === true &&
chatId !== null &&
!isSettledAssistantTail(initialRows ?? []),
);
const strippedRowRef = useRef<IAiChatMessageRow | null>(
stripRef.current ? (initialRows ?? [])[initialRows!.length - 1] : null,
);
const initialMessages = useMemo<UIMessage[]>( const initialMessages = useMemo<UIMessage[]>(
() => (initialRows ?? []).map(rowToUiMessage), () =>
seedRows(
initialRows ?? [],
attemptResumeRef.current && stripRef.current,
).map(rowToUiMessage),
[initialRows], [initialRows],
); );
@@ -181,6 +232,14 @@ export default function ChatThread({
const openPageRef = useRef<OpenPageContext | null>(openPage ?? null); const openPageRef = useRef<OpenPageContext | null>(openPage ?? null);
openPageRef.current = openPage ?? null; openPageRef.current = openPage ?? null;
// Keep the selection snapshotter in a ref, same rationale as openPageRef: the
// transport useMemo([]) closes it over, so prop-identity churn must not matter.
// Called at send time inside prepareSendMessagesRequest (#388).
const getEditorSelectionRef = useRef<
(() => EditorSelectionContext | null) | undefined
>(getEditorSelection);
getEditorSelectionRef.current = getEditorSelection;
// Keep the selected role id in a ref, same rationale as openPageRef. Only the // Keep the selected role id in a ref, same rationale as openPageRef. Only the
// FIRST request of a brand-new chat uses it (the server persists it then and // FIRST request of a brand-new chat uses it (the server persists it then and
// ignores it for existing chats), but sending it on every send is harmless. // ignores it for existing chats), but sending it on every send is harmless.
@@ -261,9 +320,12 @@ export default function ChatThread({
const { head, rest } = dequeue(queuedRef.current); const { head, rest } = dequeue(queuedRef.current);
if (!head) return false; if (!head) return false;
setQueue(rest); setQueue(rest);
// Local send: clear any resume-suppression flag so this genuine local turn's
// onFinish flushes normally (invariant 8).
setResumedTurnPair(false);
sendMessageRef.current?.({ text: head.text }); sendMessageRef.current?.({ text: head.text });
return true; return true;
}, [setQueue]); }, [setQueue, setResumedTurnPair]);
const enqueue = useCallback( const enqueue = useCallback(
(text: string) => { (text: string) => {
@@ -283,6 +345,47 @@ export default function ChatThread({
new DefaultChatTransport<UIMessage>({ new DefaultChatTransport<UIMessage>({
api: "/api/ai-chat/stream", api: "/api/ai-chat/stream",
credentials: "include", credentials: "include",
prepareReconnectToStreamRequest: () => ({
// SDK default URL uses the useChat STORE id — always build from the real chat id.
// ?expect=live&anchor=<row id> ONLY when we stripped a streaming tail: expect=live
// is the only case where a finished-retained replay is safe (the row is stripped,
// replay rebuilds it), and the anchor pins the replay to OUR run — a mismatching
// (newer) run must 204 into the restore+poll path instead of replaying a foreign
// transcript into this store.
api: `/api/ai-chat/runs/${chatIdRef.current}/stream${
stripRef.current
? `?expect=live&anchor=${strippedRowRef.current!.id}`
: ""
}`,
}),
fetch: async (input: RequestInfo | URL, init: RequestInit = {}) => {
if ((init.method ?? "GET") !== "GET") return fetch(input, init); // send path untouched
// Reconnect GET: the SDK passes no AbortSignal, so wire our own controller
// for observer Stop / unmount abort.
const controller = new AbortController();
attachAbortRef.current = controller;
try {
const response = await fetch(input, {
...init,
signal: controller.signal,
});
// No onFinish will come for a 204 (silent no-op) OR any non-2xx
// (5xx/502 — a server restart mid-attach). Both run the same
// no-active-stream recovery: restore the stripped row, invalidate, and
// arm the degraded poll (idempotent via noStreamHandledRef; its part-d
// also clears the resumedTurn flag). This is the restart-survival path
// the removed F7 latch used to guard — a transient attach failure must
// NOT drop the in-progress row or stop tracking the durable run.
if (response.status === 204 || !response.ok)
onNoActiveStreamRef.current?.();
return response;
} catch (err) {
// Network throw: same no-onFinish recovery, then rethrow so the SDK
// still surfaces the error to its own machinery.
onNoActiveStreamRef.current?.();
throw err;
}
},
// Inject the chat id and the currently-open page alongside the useChat // Inject the chat id and the currently-open page alongside the useChat
// messages so the server can resolve an existing chat (or create one // messages so the server can resolve an existing chat (or create one
// when null) and tell the agent which page "this page" refers to. Both // when null) and tell the agent which page "this page" refers to. Both
@@ -299,7 +402,16 @@ export default function ChatThread({
body: { body: {
...body, ...body,
chatId: chatIdRef.current, chatId: chatIdRef.current,
openPage: openPageRef.current, // Attach the live editor selection to the open-page context at send
// time — "this"/"here" in the user's message means THIS selection.
// Nested inside openPage so it dies with the page when the server
// rejects the page id (#388). Null when nothing is selected.
openPage: openPageRef.current
? {
...openPageRef.current,
selection: getEditorSelectionRef.current?.() ?? null,
}
: null,
// Honoured by the server only when creating a new chat; null => // Honoured by the server only when creating a new chat; null =>
// universal assistant. // universal assistant.
roleId: roleIdRef.current, roleId: roleIdRef.current,
@@ -312,7 +424,15 @@ export default function ChatThread({
[], [],
); );
const { messages, sendMessage, status, stop, error, setMessages } = useChat({ const {
messages,
sendMessage,
status,
stop,
error,
setMessages,
resumeStream,
} = useChat({
// Stable per-mount key. Existing chats use their real id; new chats use a // Stable per-mount key. Existing chats use their real id; new chats use a
// generated client id (never `undefined`) so the store is NOT re-created on // generated client id (never `undefined`) so the store is NOT re-created on
// every render mid-stream (see `chatStoreId` above). // every render mid-stream (see `chatStoreId` above).
@@ -330,6 +450,38 @@ export default function ChatThread({
// would be wrong, so on Stop/disconnect/error the queue is left intact for // would be wrong, so on Stop/disconnect/error the queue is left intact for
// the user to decide. // the user to decide.
onFinish: ({ message, isAbort, isDisconnect, isError }) => { onFinish: ({ message, isAbort, isDisconnect, isError }) => {
// (1) Capture whether THIS finish belongs to a resumed (attach) turn and
// immediately clear the flag so it can never suppress a LATER local turn.
const wasResumed = resumedTurnRef.current;
setResumedTurnPair(false);
// (2) Recovery after a starved/torn resumed finish (invariant 9). The arm
// and the stripped-row restore are gated DIFFERENTLY. Skip entirely once
// unmounted (an abort-triggered onFinish landing after a chat switch must
// not arm a poll / invalidate on the new chat).
if (wasResumed && mountedRef.current) {
const hasVisibleContent = assistantMessageHasVisibleContent(message);
// ARM the reconcile + degraded poll when the resumed message carries no
// visible content (starved replay) OR the connection dropped mid-run — in
// both cases the poll must drive the row to its real terminal state.
if (isDisconnect || !hasVisibleContent) {
reconcileTailRef.current = true;
queryClient.invalidateQueries({
queryKey: AI_CHAT_MESSAGES_RQ_KEY(chatIdRef.current),
});
onResumeFallback?.(true);
}
// RESTORE the stripped streaming row ONLY when the resumed message has no
// visible content. On isDisconnect WITH visible content restore is
// FORBIDDEN: the live stream may have advanced far past the mount-time
// snapshot, so restoring would clobber on-screen content (invariant 9) —
// the arm above suffices, the poll reaches the true terminal.
if (!hasVisibleContent && strippedRowRef.current) {
setMessages((prev) =>
mergeById(prev, rowToUiMessage(strippedRowRef.current!)),
);
}
}
// (3) Standard branches.
// Forward the authoritative server chatId (streamed on the assistant // Forward the authoritative server chatId (streamed on the assistant
// message metadata) so the parent adopts the REAL created chat id for a new // message metadata) so the parent adopts the REAL created chat id for a new
// chat — see adopt-chat-id.ts for the full #137 design. `threadKey` lets the // chat — see adopt-chat-id.ts for the full #137 design. `threadKey` lets the
@@ -342,6 +494,10 @@ export default function ChatThread({
else if (isAbort) setStopNotice("manual"); else if (isAbort) setStopNotice("manual");
else if (isDisconnect) setStopNotice("disconnect"); else if (isDisconnect) setStopNotice("disconnect");
else setStopNotice(null); else setStopNotice(null);
// A resumed turn NEVER flushes the queue (invariant 7): skip BOTH the
// flush-on-abort branch and the plain flush. The local streamer is the only
// tab that owns the queue.
if (wasResumed) return;
// "Send now": WE triggered this abort to interrupt the current turn and // "Send now": WE triggered this abort to interrupt the current turn and
// immediately send the promoted head. Flush it even though the turn was // immediately send the promoted head. Flush it even though the turn was
// aborted (the normal abort path below keeps the queue intact). The // aborted (the normal abort path below keeps the queue intact). The
@@ -423,26 +579,98 @@ export default function ChatThread({
const isStreaming = status === "submitted" || status === "streaming"; const isStreaming = status === "submitted" || status === "streaming";
// #184: report our live streaming status up so the parent stops polling the run // 204-handler (`onNoActiveStream`): the attach returned 204 — nothing live to
// while WE are the streamer (the SSE owns the view) and resumes once we go idle. // resume (overflow / begin-failure / after retention / anchor-mismatch). One-
// Effect (not render) so it never updates parent state during our own render; // shot via noStreamHandledRef (we do NOT null onNoActiveStreamRef). Exactly four
// fires on mount with `false`, which also re-syncs the parent after a chat // parts. Kept in a ref (read by the transport's fetch closure) and refreshed
// switch remounts this thread (a fresh mount is idle until the user sends). // each render below.
useEffect(() => { const onNoActiveStream = useCallback(() => {
onStreamingChange?.(isStreaming); // A late attach outcome after unmount must not arm a poll / invalidate on the
}, [isStreaming, onStreamingChange]); // now-different chat this thread's refs were reused for.
if (!mountedRef.current) return;
if (noStreamHandledRef.current) return;
noStreamHandledRef.current = true;
// (a) Restore the stripped streaming row to the store — ONLY when we actually
// stripped one (a user-tail 204 does NOT reach here with a stripped row, so do
// not dereference null).
if (strippedRowRef.current) {
setMessages((prev) =>
mergeById(prev, rowToUiMessage(strippedRowRef.current!)),
);
}
// (b) Reconcile the tail from the message history + invalidate it so the
// degraded poll starts from a fresh fetch.
reconcileTailRef.current = true;
queryClient.invalidateQueries({
queryKey: AI_CHAT_MESSAGES_RQ_KEY(chatIdRef.current),
});
// (c) Arm the degraded poll (a dumb timer with a 10-min cap in the window);
// the thread disarms it via onResumeFallback(false) on settle / local stream.
onResumeFallback?.(true);
// (d) 204 means onFinish will NOT fire — clear the suppression flag so it
// cannot swallow the NEXT local turn's queue flush.
setResumedTurnPair(false);
}, [setMessages, queryClient, onResumeFallback, setResumedTurnPair]);
onNoActiveStreamRef.current = onNoActiveStream;
// #184 passive-observer merge: when the parent feeds a polled run message (we // Mount effect: kick off the resume attempt for a non-settled tail. Marking the
// reopened a chat whose run is still going and did NOT start it here), merge it // turn as resumed BEFORE resumeStream so onFinish (invariant 7/8) sees it.
// into the live list so new steps/tool-calls appear as they are persisted. Hard-
// gated by `!isStreaming`: if THIS tab is actually the streamer, the local SSE
// owns the view and a stale observedRow must never overwrite it. `observedRow`
// is a stable per-poll object, so this runs once per poll, not per render.
useEffect(() => { useEffect(() => {
if (isStreaming || !observedRow) return; // Re-arm on (re)mount — StrictMode dev-mounts twice, and the cleanup below
const observed = rowToUiMessage(observedRow); // flips this false between the two.
setMessages((prev) => mergeObservedMessage(prev, observed)); mountedRef.current = true;
}, [observedRow, isStreaming, setMessages]); if (attemptResumeRef.current) {
setResumedTurnPair(true);
void resumeStream();
}
// Unmount: mark unmounted (gates late attach/onFinish side-effects) and abort
// the in-flight attach GET so its callbacks don't fire against the next chat.
return () => {
mountedRef.current = false;
attachAbortRef.current?.abort();
};
// Mount-only by design; the parent remounts per chat via `key`.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, []);
// Reconciliation + degraded-merge (invariant 8). Deps are EXACTLY
// [initialRows, isStreaming, setMessages].
useEffect(() => {
// A local stream owns the view: disarm BOTH the merge and the window poll.
if (isStreaming) {
reconcileTailRef.current = false;
onResumeFallback?.(false);
return;
}
if (!reconcileTailRef.current) return;
const rows = initialRows ?? [];
const tail = rows[rows.length - 1];
if (!tail || tail.role !== "assistant") return;
// Merge the polled assistant tail on EVERY initialRows update — while the
// degraded poll is active this IS the live per-step progress.
setMessages((prev) => mergeById(prev, rowToUiMessage(tail)));
// Anchor-mismatch coherence: when we restored a stripped streaming row A but a
// DIFFERENT run's row B is now the tail (A finished, B replaced the registry
// entry, so the attach 204'd), A would otherwise linger forever as an orphan
// jumping-dots row over the real run. Settle it from fresh history (where A is
// now persisted) so no phantom row survives. No-op in the common case where A
// IS the tail (id match).
const stripped = strippedRowRef.current;
if (stripped && stripped.id !== tail.id) {
const historical = rows.find((r) => r.id === stripped.id);
if (historical)
setMessages((prev) => mergeById(prev, rowToUiMessage(historical)));
}
// Settled: the terminal merge is done — disarm the flag AND the window poll
// explicitly (the window only has a time cap, it will not disarm itself).
if (tail.status !== "streaming") {
reconcileTailRef.current = false;
onResumeFallback?.(false);
}
// onResumeFallback intentionally omitted (parent-stable callback); deps are
// fixed by the resume design.
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [initialRows, isStreaming, setMessages]);
// "Send now" on a queued message: interrupt the current turn and immediately // "Send now" on a queued message: interrupt the current turn and immediately
// send THIS message, keeping the agent's partial output. Other queued messages // send THIS message, keeping the agent's partial output. Other queued messages
@@ -469,10 +697,12 @@ export default function ChatThread({
const msg = queuedRef.current.find((m) => m.id === id); const msg = queuedRef.current.find((m) => m.id === id);
if (!msg) return; if (!msg) return;
setQueue(removeQueuedById(queuedRef.current, id)); setQueue(removeQueuedById(queuedRef.current, id));
// Local send: clear any resume-suppression flag (invariant 8).
setResumedTurnPair(false);
sendMessageRef.current?.({ text: msg.text }); sendMessageRef.current?.({ text: msg.text });
} }
}, },
[setQueue, stop], [setQueue, stop, setResumedTurnPair],
); );
// Stop the current turn. ALWAYS abort the local SSE (`stop()`) so the composer // Stop the current turn. ALWAYS abort the local SSE (`stop()`) so the composer
@@ -485,6 +715,9 @@ export default function ChatThread({
// is not known yet — a brand-new chat in the first moment of its first turn — // is not known yet — a brand-new chat in the first moment of its first turn —
// only the local abort happens (there is no server-side run handle to stop yet). // only the local abort happens (there is no server-side run handle to stop yet).
const handleStop = useCallback(() => { const handleStop = useCallback(() => {
// Abort the resume/attach GET first: the SDK does not pass it a signal, so an
// observer's Stop would otherwise leave the attach fetch running.
attachAbortRef.current?.abort();
stop(); stop();
if (!autonomousRunsEnabled) return; if (!autonomousRunsEnabled) return;
if (chatIdRef.current) { if (chatIdRef.current) {
@@ -617,17 +850,23 @@ export default function ChatThread({
<Text size="xs" lineClamp={2} className={classes.queuedText}> <Text size="xs" lineClamp={2} className={classes.queuedText}>
{m.text} {m.text}
</Text> </Text>
<Tooltip label={t("Interrupt and send now")} withArrow> {/* "Send now" (interrupt) is hidden on a RESUMED turn: a local
<ActionIcon stop() does not abort the resumed attach fetch, so the click
size="xs" would be swallowed while flushOnAbortRef would fire minutes
variant="subtle" later on the natural finish. Only the remove affordance stays. */}
color="blue" {!resumedTurn && (
onClick={() => sendNow(m.id)} <Tooltip label={t("Interrupt and send now")} withArrow>
aria-label={t("Send now")} <ActionIcon
> size="xs"
<IconPlayerPlayFilled size={12} /> variant="subtle"
</ActionIcon> color="blue"
</Tooltip> onClick={() => sendNow(m.id)}
aria-label={t("Send now")}
>
<IconPlayerPlayFilled size={12} />
</ActionIcon>
</Tooltip>
)}
<ActionIcon <ActionIcon
size="xs" size="xs"
variant="subtle" variant="subtle"
@@ -642,7 +881,11 @@ export default function ChatThread({
</Stack> </Stack>
)} )}
<ChatInput <ChatInput
onSend={(text) => sendMessage({ text })} onSend={(text) => {
// Local send: clear any resume-suppression flag (invariant 8).
setResumedTurnPair(false);
sendMessage({ text });
}}
onQueue={enqueue} onQueue={enqueue}
onStop={handleStop} onStop={handleStop}
isStreaming={isStreaming} isStreaming={isStreaming}
@@ -40,6 +40,13 @@ interface MessageItemProps {
* Defaults to true (internal chat). The public share passes false. * Defaults to true (internal chat). The public share passes false.
*/ */
showCitations?: boolean; showCitations?: boolean;
/**
* Forwarded to ToolCallCard: whether tool cards render the one-line summary of
* a call's arguments (e.g. the search query). Defaults to true (internal
* chat). The public share passes false so an anonymous reader doesn't see the
* agent's raw query/argument text.
*/
showInput?: boolean;
/** /**
* Neutralize internal/relative markdown links in the rendered answer (drop * Neutralize internal/relative markdown links in the rendered answer (drop
* their href so they become inert text). Defaults to false (internal chat, * their href so they become inert text). Defaults to false (internal chat,
@@ -117,6 +124,7 @@ const MarkdownPart = memo(function MarkdownPart({
function MessageItem({ function MessageItem({
message, message,
showCitations = true, showCitations = true,
showInput = true,
neutralizeInternalLinks = false, neutralizeInternalLinks = false,
assistantName, assistantName,
turnStreaming = false, turnStreaming = false,
@@ -210,6 +218,7 @@ function MessageItem({
key={index} key={index}
part={part as unknown as ToolUiPart} part={part as unknown as ToolUiPart}
showCitations={showCitations} showCitations={showCitations}
showInput={showInput}
/> />
); );
} }
@@ -274,6 +283,7 @@ export function arePropsEqual(
return ( return (
prev.signature === next.signature && prev.signature === next.signature &&
prev.showCitations === next.showCitations && prev.showCitations === next.showCitations &&
prev.showInput === next.showInput &&
prev.neutralizeInternalLinks === next.neutralizeInternalLinks && prev.neutralizeInternalLinks === next.neutralizeInternalLinks &&
prev.assistantName === next.assistantName && prev.assistantName === next.assistantName &&
// The turn-end flip re-renders every row once (cheap, terminal event) — // The turn-end flip re-renders every row once (cheap, terminal event) —
@@ -25,6 +25,13 @@ interface MessageListProps {
* false because an anonymous reader cannot open the linked internal pages. * false because an anonymous reader cannot open the linked internal pages.
*/ */
showCitations?: boolean; showCitations?: boolean;
/**
* Forwarded to MessageItem -> ToolCallCard: whether tool cards render the
* one-line summary of a call's arguments (e.g. the search query). Defaults to
* true (internal chat). The public share passes false so an anonymous reader
* doesn't see the agent's raw query/argument text.
*/
showInput?: boolean;
/** /**
* Forwarded to MessageItem: neutralize internal/relative markdown links in * Forwarded to MessageItem: neutralize internal/relative markdown links in
* the rendered answers (drop their href so they render as inert text). * the rendered answers (drop their href so they render as inert text).
@@ -119,6 +126,7 @@ export default function MessageList({
isStreaming, isStreaming,
emptyState, emptyState,
showCitations = true, showCitations = true,
showInput = true,
neutralizeInternalLinks = false, neutralizeInternalLinks = false,
assistantName, assistantName,
}: MessageListProps) { }: MessageListProps) {
@@ -208,6 +216,7 @@ export default function MessageList({
message={message} message={message}
signature={messageSignature(message)} signature={messageSignature(message)}
showCitations={showCitations} showCitations={showCitations}
showInput={showInput}
neutralizeInternalLinks={neutralizeInternalLinks} neutralizeInternalLinks={neutralizeInternalLinks}
assistantName={assistantName} assistantName={assistantName}
// Turn-level liveness, gated to the TAIL row: only the tail message // Turn-level liveness, gated to the TAIL row: only the tail message
@@ -5,6 +5,7 @@ import { useTranslation } from "react-i18next";
import { import {
getToolName, getToolName,
toolCitations, toolCitations,
toolInputSummary,
toolLabelKey, toolLabelKey,
toolRunState, toolRunState,
ToolUiPart, ToolUiPart,
@@ -21,6 +22,14 @@ interface ToolCallCardProps {
* (the action log itself) while dropping the unusable links. * (the action log itself) while dropping the unusable links.
*/ */
showCitations?: boolean; showCitations?: boolean;
/**
* Whether to render the one-line summary of the call's arguments (e.g. the
* search query) under the label. Defaults to true (the internal chat). The
* public share passes false: an anonymous reader should not see the agent's
* raw query/argument text. Conservative and reversible — it only suppresses
* the extra summary line, leaving the card (the action log) intact.
*/
showInput?: boolean;
} }
/** /**
@@ -31,12 +40,14 @@ interface ToolCallCardProps {
export default function ToolCallCard({ export default function ToolCallCard({
part, part,
showCitations = true, showCitations = true,
showInput = true,
}: ToolCallCardProps) { }: ToolCallCardProps) {
const { t } = useTranslation(); const { t } = useTranslation();
const toolName = getToolName(part); const toolName = getToolName(part);
const state = toolRunState(part.state); const state = toolRunState(part.state);
const { key, values } = toolLabelKey(toolName); const { key, values } = toolLabelKey(toolName);
const citations = showCitations ? toolCitations(part) : []; const citations = showCitations ? toolCitations(part) : [];
const inputSummary = showInput ? toolInputSummary(part) : undefined;
return ( return (
<div className={classes.toolCard}> <div className={classes.toolCard}>
@@ -57,6 +68,12 @@ export default function ToolCallCard({
</Text> </Text>
</Group> </Group>
{inputSummary && (
<Text size="xs" c="dimmed" mt={2} lineClamp={2}>
{inputSummary}
</Text>
)}
{state === "error" && part.errorText && ( {state === "error" && part.errorText && (
<Text size="xs" c="red" mt={2}> <Text size="xs" c="red" mt={2}>
{part.errorText} {part.errorText}
@@ -0,0 +1,90 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import React from "react";
import { renderHook, waitFor } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
// react-i18next / notifications are pulled in transitively by ai-chat-query.ts
// (the mutation hooks use them); stub so the module imports cleanly.
vi.mock("react-i18next", () => ({
useTranslation: () => ({ t: (key: string) => key }),
}));
vi.mock("@mantine/notifications", () => ({
notifications: { show: vi.fn() },
}));
// Mock the service module; only getAiChatMessages is exercised, but the other
// named exports must exist so ai-chat-query.ts imports resolve.
vi.mock("@/features/ai-chat/services/ai-chat-service.ts", () => ({
getAiChatMessages: vi.fn(),
getAiChats: vi.fn(),
getAiRoleCatalog: vi.fn(),
getAiRoleCatalogBundle: vi.fn(),
getAiRoles: vi.fn(),
importAiRolesFromCatalog: vi.fn(),
createAiRole: vi.fn(),
deleteAiChat: vi.fn(),
deleteAiRole: vi.fn(),
renameAiChat: vi.fn(),
updateAiRole: vi.fn(),
updateAiRoleFromCatalog: vi.fn(),
}));
import { getAiChatMessages } from "@/features/ai-chat/services/ai-chat-service.ts";
import { useAiChatMessagesQuery } from "@/features/ai-chat/queries/ai-chat-query.ts";
const emptyPage = { items: [], meta: { hasNextPage: false, nextCursor: null } };
function createWrapper() {
const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false } },
});
return function Wrapper({ children }: { children: React.ReactNode }) {
return (
<QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
);
};
}
// The degraded-poll fallback (#184 phase 1.5) is threaded into this query as a
// `refetchInterval`; AiChatWindow supplies the deliberately-dumb callback. These
// pin the plumbing the window depends on: the interval polls the message history,
// and — critically — fetch ERRORS do NOT stop the tick (TanStack v5 resets the
// failure count each fetch, so the poll must survive a server restart).
describe("useAiChatMessagesQuery — degraded refetchInterval", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("re-polls at the interval while the callback returns a duration", async () => {
vi.mocked(getAiChatMessages).mockResolvedValue(emptyPage as never);
renderHook(() => useAiChatMessagesQuery("c1", () => 30), {
wrapper: createWrapper(),
});
await waitFor(() =>
expect(vi.mocked(getAiChatMessages).mock.calls.length).toBeGreaterThan(2),
);
});
it("does NOT re-poll when the callback returns false", async () => {
vi.mocked(getAiChatMessages).mockResolvedValue(emptyPage as never);
renderHook(() => useAiChatMessagesQuery("c1", () => false), {
wrapper: createWrapper(),
});
await waitFor(() =>
expect(vi.mocked(getAiChatMessages)).toHaveBeenCalledTimes(1),
);
// Give any errant interval a chance to fire, then assert it did not.
await new Promise((r) => setTimeout(r, 60));
expect(vi.mocked(getAiChatMessages)).toHaveBeenCalledTimes(1);
});
it("keeps ticking through fetch errors (errors do not gate the poll)", async () => {
vi.mocked(getAiChatMessages).mockRejectedValue(new Error("server down"));
renderHook(() => useAiChatMessagesQuery("c1", () => 30), {
wrapper: createWrapper(),
});
await waitFor(() =>
expect(vi.mocked(getAiChatMessages).mock.calls.length).toBeGreaterThan(2),
);
});
});
@@ -13,7 +13,6 @@ import {
deleteAiChat, deleteAiChat,
deleteAiRole, deleteAiRole,
getAiChatMessages, getAiChatMessages,
getAiChatRun,
getAiChats, getAiChats,
getAiRoleCatalog, getAiRoleCatalog,
getAiRoleCatalogBundle, getAiRoleCatalogBundle,
@@ -26,7 +25,6 @@ import {
import { import {
IAiChat, IAiChat,
IAiChatMessageRow, IAiChatMessageRow,
IAiChatRunResponse,
IAiRole, IAiRole,
IAiRoleCatalog, IAiRoleCatalog,
IAiRoleCatalogBundle, IAiRoleCatalogBundle,
@@ -37,7 +35,6 @@ import {
IAiRoleUpdateFromCatalogResult, IAiRoleUpdateFromCatalogResult,
} from "@/features/ai-chat/types/ai-chat.types.ts"; } from "@/features/ai-chat/types/ai-chat.types.ts";
import { IPagination } from "@/lib/types.ts"; import { IPagination } from "@/lib/types.ts";
import { runPollInterval } from "@/features/ai-chat/utils/run-polling.ts";
export const AI_CHATS_RQ_KEY = ["ai-chats"]; export const AI_CHATS_RQ_KEY = ["ai-chats"];
export const AI_ROLES_RQ_KEY = ["ai-roles"]; export const AI_ROLES_RQ_KEY = ["ai-roles"];
@@ -55,7 +52,6 @@ export const AI_CHAT_MESSAGES_RQ_KEY = (chatId: string) => [
"ai-chat-messages", "ai-chat-messages",
chatId, chatId,
]; ];
export const AI_CHAT_RUN_RQ_KEY = (chatId: string) => ["ai-chat-run", chatId];
/** Paginated list of the current user's chats (auto-loads further pages). */ /** Paginated list of the current user's chats (auto-loads further pages). */
export function useAiChatsQuery() { export function useAiChatsQuery() {
@@ -89,7 +85,15 @@ export function useAiChatsQuery() {
* Load all persisted messages of a chat (oldest first), flattening the * Load all persisted messages of a chat (oldest first), flattening the
* paginated server response. Used to seed `useChat` initial messages. * paginated server response. Used to seed `useChat` initial messages.
*/ */
export function useAiChatMessagesQuery(chatId: string | undefined) { export function useAiChatMessagesQuery(
chatId: string | undefined,
// #184 phase 1.5: the degraded-poll fallback. When a tab could not attach to a
// still-running run (the attach returned 204 / the resumed stream ended with no
// terminal row), the window arms a dumb timed poll of the message history to
// follow the detached run to settle. The callback form lives in AiChatWindow;
// threaded here verbatim so this query owns the polling. Undefined => no poll.
refetchInterval?: number | false | (() => number | false),
) {
const query = useInfiniteQuery({ const query = useInfiniteQuery({
queryKey: AI_CHAT_MESSAGES_RQ_KEY(chatId ?? ""), queryKey: AI_CHAT_MESSAGES_RQ_KEY(chatId ?? ""),
queryFn: ({ pageParam }) => queryFn: ({ pageParam }) =>
@@ -100,6 +104,7 @@ export function useAiChatMessagesQuery(chatId: string | undefined) {
? (lastPage.meta.nextCursor ?? undefined) ? (lastPage.meta.nextCursor ?? undefined)
: undefined, : undefined,
enabled: !!chatId, enabled: !!chatId,
refetchInterval,
}); });
// useInfiniteQuery only fetches the first page on its own. The hook's contract // useInfiniteQuery only fetches the first page on its own. The hook's contract
@@ -139,34 +144,6 @@ export function useAiChatMessagesQuery(chatId: string | undefined) {
}; };
} }
/**
* Reconnect to a chat's latest agent run and LIVE-FOLLOW it (#184). While the run
* is active the query re-polls every {@link runPollInterval} ms (driven off the
* fetched `run.status`, the same status-keyed refetchInterval pattern as the
* embeddings reindex polling); once the run reaches a terminal status — or there
* is no run — the interval returns `false` and polling stops on its own. Polling
* is thus naturally bounded by the run terminating; no separate timeout cap.
*
* `enabled` gates the whole thing: callers pass `false` when the autonomous-runs
* feature is off (the endpoint is NOT flag-gated server-side, but with the feature
* off the chat has no runs, so polling would only ever return `{ run: null }`) OR
* when THIS tab is the one actively streaming the run (the live SSE owns the view,
* so we must not also poll/merge). The global `retry: false` means a failed fetch
* leaves `data` undefined, so refetchInterval(undefined run) returns false — a
* failed fetch can never spin a tight loop.
*/
export function useAiChatRunQuery(
chatId: string | undefined,
enabled: boolean,
) {
return useQuery<IAiChatRunResponse, Error>({
queryKey: AI_CHAT_RUN_RQ_KEY(chatId ?? ""),
queryFn: () => getAiChatRun(chatId as string),
enabled: !!chatId && enabled,
refetchInterval: (query) => runPollInterval(query.state.data?.run),
});
}
export function useRenameAiChatMutation() { export function useRenameAiChatMutation() {
const queryClient = useQueryClient(); const queryClient = useQueryClient();
const { t } = useTranslation(); const { t } = useTranslation();
@@ -1,92 +0,0 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import React from "react";
import { renderHook, waitFor } from "@testing-library/react";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import type { IAiChatRunResponse } from "@/features/ai-chat/types/ai-chat.types.ts";
// react-i18next is pulled in transitively by ai-chat-query.ts (the mutation hooks
// use it); stub it so the module imports cleanly in this hook test.
vi.mock("react-i18next", () => ({
useTranslation: () => ({ t: (key: string) => key }),
}));
vi.mock("@mantine/notifications", () => ({
notifications: { show: vi.fn() },
}));
// Mock the whole service module; only getAiChatRun is exercised here, but the
// other named exports must exist so ai-chat-query.ts imports resolve.
vi.mock("@/features/ai-chat/services/ai-chat-service.ts", () => ({
getAiChatRun: vi.fn(),
getAiChatMessages: vi.fn(),
getAiChats: vi.fn(),
getAiRoleCatalog: vi.fn(),
getAiRoleCatalogBundle: vi.fn(),
getAiRoles: vi.fn(),
importAiRolesFromCatalog: vi.fn(),
createAiRole: vi.fn(),
deleteAiChat: vi.fn(),
deleteAiRole: vi.fn(),
renameAiChat: vi.fn(),
updateAiRole: vi.fn(),
updateAiRoleFromCatalog: vi.fn(),
}));
import { getAiChatRun } from "@/features/ai-chat/services/ai-chat-service.ts";
import { useAiChatRunQuery } from "@/features/ai-chat/queries/ai-chat-query.ts";
function createWrapper() {
const queryClient = new QueryClient({
defaultOptions: { queries: { retry: false } },
});
return function Wrapper({ children }: { children: React.ReactNode }) {
return (
<QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
);
};
}
const runningResponse: IAiChatRunResponse = {
run: { id: "run-1", chatId: "c1", status: "running" },
message: {
id: "a1",
role: "assistant",
content: "working...",
createdAt: "2026-01-01T00:00:00Z",
},
};
describe("useAiChatRunQuery — enable gating", () => {
beforeEach(() => {
vi.clearAllMocks();
});
it("fetches the run when enabled (passive observer, feature on)", async () => {
vi.mocked(getAiChatRun).mockResolvedValue(runningResponse);
const { result } = renderHook(() => useAiChatRunQuery("c1", true), {
wrapper: createWrapper(),
});
await waitFor(() => expect(result.current.isSuccess).toBe(true));
expect(getAiChatRun).toHaveBeenCalledWith("c1");
expect(result.current.data?.run?.status).toBe("running");
});
it("does NOT fetch when disabled (this tab is the streamer / feature off)", async () => {
vi.mocked(getAiChatRun).mockResolvedValue(runningResponse);
renderHook(() => useAiChatRunQuery("c1", false), {
wrapper: createWrapper(),
});
// Give any errant fetch a chance to fire, then assert none did.
await new Promise((r) => setTimeout(r, 20));
expect(getAiChatRun).not.toHaveBeenCalled();
});
it("does NOT fetch when there is no chat id", async () => {
vi.mocked(getAiChatRun).mockResolvedValue(runningResponse);
renderHook(() => useAiChatRunQuery(undefined, true), {
wrapper: createWrapper(),
});
await new Promise((r) => setTimeout(r, 20));
expect(getAiChatRun).not.toHaveBeenCalled();
});
});
@@ -5,7 +5,6 @@ import {
IAiChatListParams, IAiChatListParams,
IAiChatMessageRow, IAiChatMessageRow,
IAiChatMessagesParams, IAiChatMessagesParams,
IAiChatRunResponse,
IAiRole, IAiRole,
IAiRoleCatalog, IAiRoleCatalog,
IAiRoleCatalogBundle, IAiRoleCatalogBundle,
@@ -43,23 +42,6 @@ export async function getAiChatMessages(
return req.data; return req.data;
} }
/**
* Reconnect to the latest agent run of a chat (#184). Returns the run's
* persisted lifecycle state and the assistant message it materializes (the
* partial output while the run is in-flight, the final output once it finished).
* The DB is the source of truth, so this works for an in-flight run (the browser
* dropped, the run kept going) and a finished one alike; `{ run: null }` when the
* chat has never had a run. Owner-gated server-side (the requesting user must own
* the chat); it is NOT flag-gated — when the feature is off the chat simply has no
* runs, so the endpoint returns `{ run: null }`.
*/
export async function getAiChatRun(
chatId: string,
): Promise<IAiChatRunResponse> {
const req = await api.post<IAiChatRunResponse>("/ai-chat/run", { chatId });
return req.data;
}
/** /**
* Explicitly STOP the active agent run of a chat (#184). This is the ONLY thing * Explicitly STOP the active agent run of a chat (#184). This is the ONLY thing
* that ends a DETACHED run — a mere browser disconnect (aborting the local SSE) * that ends a DETACHED run — a mere browser disconnect (aborting the local SSE)
@@ -210,41 +210,14 @@ export interface IAiChatMessageRow {
// renders a "stopped" marker on interrupted turns. // renders a "stopped" marker on interrupted turns.
finishReason?: string; finishReason?: string;
} | null; } | null;
// Persisted lifecycle status of the row's turn, carried on the wire by
// `baseFields`. 'streaming' marks a still-in-progress assistant row (used by
// the resume machinery to decide whether a tail is a live stream to attach to
// or a settled row that must not be replayed).
status?: string;
createdAt: string; createdAt: string;
} }
/**
* A persisted agent-run row (#184), mirroring the `ai_chat_runs` fields the
* client reads from `POST /ai-chat/run`. Only `status` is load-bearing for the
* reconnect-and-live-update UX (it drives the poll cadence); the rest are carried
* for display/diagnostics. The DB is the source of truth, so this resolves for an
* in-flight run (the browser dropped, the run kept going) and a finished one.
*/
export interface IAiChatRun {
id: string;
chatId: string;
// 'pending' | 'running' | 'succeeded' | 'failed' | 'aborted'. The first two are
// ACTIVE (keep polling); the rest are TERMINAL (stop polling).
status: "pending" | "running" | "succeeded" | "failed" | "aborted" | string;
error?: string | null;
stepCount?: number;
assistantMessageId?: string | null;
startedAt?: string | null;
finishedAt?: string | null;
createdAt?: string;
updatedAt?: string;
}
/**
* Response of `POST /ai-chat/run` (#184): the latest run of a chat and the
* assistant message it materializes (the partial/final output, projected from the
* persisted rows). Both are `null` when the chat has never had a run.
*/
export interface IAiChatRunResponse {
run: IAiChatRun | null;
message: IAiChatMessageRow | null;
}
export interface IAiChatListParams extends QueryParams {} export interface IAiChatListParams extends QueryParams {}
export interface IAiChatMessagesParams { export interface IAiChatMessagesParams {
@@ -0,0 +1,112 @@
import { describe, it, expect } from "vitest";
import type { UIMessage } from "@ai-sdk/react";
import type { IAiChatMessageRow } from "@/features/ai-chat/types/ai-chat.types.ts";
import {
isStreamingTail,
isSettledAssistantTail,
seedRows,
mergeById,
} from "./resume-helpers.ts";
function row(
id: string,
role: string,
status?: string,
): IAiChatMessageRow {
return { id, role, content: "", status, createdAt: "2026-01-01T00:00:00Z" };
}
function makeMsg(id: string, text: string): UIMessage {
return {
id,
role: "assistant",
parts: [{ type: "text", text }],
} as UIMessage;
}
describe("isStreamingTail", () => {
it("is true when the last row is a streaming assistant row", () => {
expect(
isStreamingTail([row("u1", "user"), row("a1", "assistant", "streaming")]),
).toBe(true);
});
it("is false for a settled assistant tail", () => {
expect(isStreamingTail([row("a1", "assistant", "succeeded")])).toBe(false);
expect(isStreamingTail([row("a1", "assistant")])).toBe(false);
});
it("is false when the tail is a user row or the list is empty", () => {
expect(isStreamingTail([row("u1", "user")])).toBe(false);
expect(isStreamingTail([])).toBe(false);
});
});
describe("isSettledAssistantTail", () => {
it("is true for an assistant tail whose status is not streaming", () => {
expect(isSettledAssistantTail([row("a1", "assistant", "succeeded")])).toBe(
true,
);
expect(isSettledAssistantTail([row("a1", "assistant")])).toBe(true);
expect(isSettledAssistantTail([row("a1", "assistant", "aborted")])).toBe(
true,
);
});
it("is false for a streaming assistant tail", () => {
expect(isSettledAssistantTail([row("a1", "assistant", "streaming")])).toBe(
false,
);
});
it("is false when the tail is a user row or the list is empty", () => {
expect(isSettledAssistantTail([row("u1", "user")])).toBe(false);
expect(isSettledAssistantTail([])).toBe(false);
});
});
describe("seedRows", () => {
const rows = [row("u1", "user"), row("a1", "assistant", "streaming")];
it("returns the rows unchanged when not stripping", () => {
expect(seedRows(rows, false)).toBe(rows);
});
it("drops the last row when stripping", () => {
const seeded = seedRows(rows, true);
expect(seeded).toHaveLength(1);
expect(seeded[0].id).toBe("u1");
});
it("returns an empty list when stripping a single-row list", () => {
expect(seedRows([row("a1", "assistant", "streaming")], true)).toHaveLength(
0,
);
});
});
describe("mergeById", () => {
it("replaces the message with the same id in place (per-step growth)", () => {
const prev = [makeMsg("u1", "hi"), makeMsg("a1", "step 1")];
const incoming = makeMsg("a1", "step 1\nstep 2");
const next = mergeById(prev, incoming);
expect(next).toHaveLength(2);
expect(next[1]).toBe(incoming);
expect(next[0]).toBe(prev[0]); // untouched
expect(next).not.toBe(prev); // new array (never mutates input)
});
it("appends when the incoming message is not yet present", () => {
const prev = [makeMsg("u1", "hi")];
const incoming = makeMsg("a1", "first token");
const next = mergeById(prev, incoming);
expect(next).toHaveLength(2);
expect(next[1]).toBe(incoming);
});
it("returns the original list unchanged when there is nothing to merge", () => {
const prev = [makeMsg("u1", "hi")];
expect(mergeById(prev, null)).toBe(prev);
expect(mergeById(prev, undefined)).toBe(prev);
});
});
@@ -0,0 +1,62 @@
import type { UIMessage } from "@ai-sdk/react";
import type { IAiChatMessageRow } from "@/features/ai-chat/types/ai-chat.types.ts";
/**
* Pure decisions for the resumable-SSE resume machinery (#184 phase 1.5). A tab
* that reopens a chat whose agent run is still going attaches to the server's
* run-stream registry (replay + live tail) instead of polling snapshots; these
* small predicates decide WHICH tail is safe to resume and how to seed the store,
* extracted so they can be unit-tested in isolation.
*/
/**
* A STREAMING tail: the last persisted row is an assistant row still marked
* `status === 'streaming'`. Such a tail is stripped from the seed and rebuilt by
* the replay (`expect=live`), since the SDK's `text-start` always pushes a new
* part and replaying over a seeded in-progress row would duplicate its text.
*/
export function isStreamingTail(rows: IAiChatMessageRow[]): boolean {
const tail = rows[rows.length - 1];
return !!tail && tail.role === "assistant" && tail.status === "streaming";
}
/**
* A SETTLED assistant tail: the last row is an assistant row whose status is
* anything OTHER than 'streaming'. A settled assistant tail must NEVER resume —
* replaying a finished run into a store that already holds its message duplicates
* parts (`text-start` always pushes a new part).
*/
export function isSettledAssistantTail(rows: IAiChatMessageRow[]): boolean {
const tail = rows[rows.length - 1];
return !!tail && tail.role === "assistant" && tail.status !== "streaming";
}
/**
* Seed rows for `useChat`: return the rows unchanged, or without the last row when
* `strip` is set (the streaming tail is stripped so the live replay rebuilds it
* without duplicating parts).
*/
export function seedRows(
rows: IAiChatMessageRow[],
strip: boolean,
): IAiChatMessageRow[] {
return strip ? rows.slice(0, -1) : rows;
}
/**
* Merge an assistant message into the rendered list by id: replace the message
* with the same id in place (the in-progress assistant row is already seeded from
* history, so per-step growth replaces it), or append it when absent. Returns a
* new array; the input is never mutated.
*/
export function mergeById(
messages: UIMessage[],
incoming: UIMessage | null | undefined,
): UIMessage[] {
if (!incoming) return messages;
const idx = messages.findIndex((m) => m.id === incoming.id);
if (idx === -1) return [...messages, incoming];
const next = messages.slice();
next[idx] = incoming;
return next;
}
@@ -1,303 +0,0 @@
import { describe, it, expect } from "vitest";
import type { UIMessage } from "@ai-sdk/react";
import type { IAiChatRun } from "@/features/ai-chat/types/ai-chat.types.ts";
import {
RUN_POLL_INTERVAL_MS,
isRunActive,
runPollInterval,
shouldObserveRun,
shouldClearStoppingLatch,
shouldClearLatchOnQueryError,
mergeObservedMessage,
} from "./run-polling.ts";
function makeRun(status: string): IAiChatRun {
return { id: "run-1", chatId: "c1", status };
}
function makeMsg(id: string, text: string): UIMessage {
return {
id,
role: "assistant",
parts: [{ type: "text", text }],
} as UIMessage;
}
describe("isRunActive", () => {
it("treats pending and running as active", () => {
expect(isRunActive(makeRun("pending"))).toBe(true);
expect(isRunActive(makeRun("running"))).toBe(true);
});
it("treats terminal / unknown / nullish as not active", () => {
expect(isRunActive(makeRun("succeeded"))).toBe(false);
expect(isRunActive(makeRun("failed"))).toBe(false);
expect(isRunActive(makeRun("aborted"))).toBe(false);
expect(isRunActive(makeRun("weird-future-status"))).toBe(false);
expect(isRunActive(null)).toBe(false);
expect(isRunActive(undefined)).toBe(false);
});
});
describe("runPollInterval (the refetchInterval helper)", () => {
it("returns 2000ms while the run is pending/running", () => {
expect(runPollInterval(makeRun("pending"))).toBe(RUN_POLL_INTERVAL_MS);
expect(runPollInterval(makeRun("running"))).toBe(RUN_POLL_INTERVAL_MS);
expect(RUN_POLL_INTERVAL_MS).toBe(2000);
});
it("returns false (stop polling) once the run is terminal", () => {
expect(runPollInterval(makeRun("succeeded"))).toBe(false);
expect(runPollInterval(makeRun("failed"))).toBe(false);
expect(runPollInterval(makeRun("aborted"))).toBe(false);
});
it("returns false (no polling) when there is no run", () => {
expect(runPollInterval(null)).toBe(false);
expect(runPollInterval(undefined)).toBe(false);
});
});
describe("shouldObserveRun (observer-vs-streamer decision)", () => {
it("observes an active run when this tab is NOT the local streamer", () => {
expect(shouldObserveRun(makeRun("running"), false)).toBe(true);
expect(shouldObserveRun(makeRun("pending"), false)).toBe(true);
});
it("observes a terminal run too (so the final output shows on reopen)", () => {
expect(shouldObserveRun(makeRun("succeeded"), false)).toBe(true);
});
it("does NOT observe when this tab IS the streamer (no double-render)", () => {
expect(shouldObserveRun(makeRun("running"), true)).toBe(false);
expect(shouldObserveRun(makeRun("succeeded"), true)).toBe(false);
});
it("does NOT observe when there is no run", () => {
expect(shouldObserveRun(null, false)).toBe(false);
expect(shouldObserveRun(undefined, false)).toBe(false);
});
});
describe("shouldClearStoppingLatch (#234 latch-release decision)", () => {
// The one case the latch SHOULD clear: we requested a stop, we are the passive
// observer (not streaming), and the CURRENT run is terminal.
it("clears only when stopping, observing, and the run is terminal", () => {
expect(
shouldClearStoppingLatch({
stoppingRun: true,
run: makeRun("aborted"),
isLocalStreaming: false,
}),
).toBe(true);
expect(
shouldClearStoppingLatch({
stoppingRun: true,
run: makeRun("succeeded"),
isLocalStreaming: false,
}),
).toBe(true);
expect(
shouldClearStoppingLatch({
stoppingRun: true,
run: makeRun("failed"),
isLocalStreaming: false,
}),
).toBe(true);
});
// Round-3 regression: clearing while THIS tab is still the local streamer would
// re-open the flash for the current turn the moment we switch to observer role.
// A predicate lacking the streaming gate would (wrongly) return true here.
it("does NOT clear while this tab is the local streamer", () => {
expect(
shouldClearStoppingLatch({
stoppingRun: true,
run: makeRun("aborted"),
isLocalStreaming: true,
}),
).toBe(false);
expect(
shouldClearStoppingLatch({
stoppingRun: true,
run: makeRun("succeeded"),
isLocalStreaming: true,
}),
).toBe(false);
});
// The detached run keeps growing after a local abort — while it is still
// active the latch MUST hold so the observer merge stays suppressed.
it("does NOT clear while the run is still active", () => {
expect(
shouldClearStoppingLatch({
stoppingRun: true,
run: makeRun("running"),
isLocalStreaming: false,
}),
).toBe(false);
expect(
shouldClearStoppingLatch({
stoppingRun: true,
run: makeRun("pending"),
isLocalStreaming: false,
}),
).toBe(false);
});
// #234 F4: on Stop the stale PREVIOUS-turn run is removed from the cache, so the
// observed `run` is null until the current turn's run is fetched fresh. A null
// run HOLDS the latch — it can never clear against the just-removed stale run,
// only against the current turn's own terminal run once observed.
it("does NOT clear against a removed/absent run (F4 stale-run guard)", () => {
expect(
shouldClearStoppingLatch({
stoppingRun: true,
run: null,
isLocalStreaming: false,
}),
).toBe(false);
expect(
shouldClearStoppingLatch({
stoppingRun: true,
run: undefined,
isLocalStreaming: false,
}),
).toBe(false);
});
it("does NOT clear when no stop was requested", () => {
expect(
shouldClearStoppingLatch({
stoppingRun: false,
run: makeRun("aborted"),
isLocalStreaming: false,
}),
).toBe(false);
});
});
describe("shouldClearLatchOnQueryError (#234 F7 error-safety-net decision)", () => {
// This guards the REAL anti-flash decision the component's run-query-error
// safety-net effect uses (ai-chat-window.tsx wires the effect to THIS helper,
// not a copy — so the test is non-vacuous vs the live code).
// (b) The F7 hole: a TRANSIENT run-query error while `run` is STILL ACTIVE must
// NOT clear the latch. TanStack Query v5 retains `data` on error, so
// runQueryFailed can be true while the held run is still pending/running.
// Against the PRE-F7 condition (without `!isRunActive(run)`) this would return
// true — so this assertion fails on the buggy code (non-vacuous).
it("does NOT clear on a transient error while the run is still ACTIVE (F7)", () => {
expect(
shouldClearLatchOnQueryError({
stoppingRun: true,
isLocalStreaming: false,
runQueryFailed: true,
run: makeRun("running"),
}),
).toBe(false);
expect(
shouldClearLatchOnQueryError({
stoppingRun: true,
isLocalStreaming: false,
runQueryFailed: true,
run: makeRun("pending"),
}),
).toBe(false);
});
// (a) The genuine permanent-null-freeze: run cache cleared by removeQueries +
// the refetch keeps ERRORING, so `run === null`. This is the ONLY case the
// safety-net exists to cure — it MUST clear so the frozen view resumes.
it("clears on a permanent error when the run is null (permanent-null-freeze)", () => {
expect(
shouldClearLatchOnQueryError({
stoppingRun: true,
isLocalStreaming: false,
runQueryFailed: true,
run: null,
}),
).toBe(true);
expect(
shouldClearLatchOnQueryError({
stoppingRun: true,
isLocalStreaming: false,
runQueryFailed: true,
run: undefined,
}),
).toBe(true);
});
// A TERMINAL run also satisfies `!isRunActive`; clearing then is harmless — the
// terminal effect (shouldClearStoppingLatch) already clears for a terminal run,
// so this only ever agrees with it. Asserted so the (c) reasoning is pinned.
it("clears on an error when the run is terminal (harmless, agrees with terminal effect)", () => {
expect(
shouldClearLatchOnQueryError({
stoppingRun: true,
isLocalStreaming: false,
runQueryFailed: true,
run: makeRun("aborted"),
}),
).toBe(true);
});
it("does NOT clear without an actual query error", () => {
expect(
shouldClearLatchOnQueryError({
stoppingRun: true,
isLocalStreaming: false,
runQueryFailed: false,
run: null,
}),
).toBe(false);
});
it("does NOT clear while this tab is the local streamer", () => {
expect(
shouldClearLatchOnQueryError({
stoppingRun: true,
isLocalStreaming: true,
runQueryFailed: true,
run: null,
}),
).toBe(false);
});
it("does NOT clear when no stop was requested", () => {
expect(
shouldClearLatchOnQueryError({
stoppingRun: false,
isLocalStreaming: false,
runQueryFailed: true,
run: null,
}),
).toBe(false);
});
});
describe("mergeObservedMessage", () => {
it("replaces the message with the same id in place (per-step growth)", () => {
const prev = [makeMsg("u1", "hi"), makeMsg("a1", "step 1")];
const observed = makeMsg("a1", "step 1\nstep 2");
const next = mergeObservedMessage(prev, observed);
expect(next).toHaveLength(2);
expect(next[1]).toBe(observed);
expect(next[0]).toBe(prev[0]); // untouched
expect(next).not.toBe(prev); // new array (never mutates input)
});
it("appends when the observed message is not yet present", () => {
const prev = [makeMsg("u1", "hi")];
const observed = makeMsg("a1", "first token");
const next = mergeObservedMessage(prev, observed);
expect(next).toHaveLength(2);
expect(next[1]).toBe(observed);
});
it("returns the original list unchanged when there is nothing to merge", () => {
const prev = [makeMsg("u1", "hi")];
expect(mergeObservedMessage(prev, null)).toBe(prev);
expect(mergeObservedMessage(prev, undefined)).toBe(prev);
});
});
@@ -1,151 +0,0 @@
import type { UIMessage } from "@ai-sdk/react";
import type { IAiChatRun } from "@/features/ai-chat/types/ai-chat.types.ts";
/**
* Reconnect-and-live-follow helpers (#184). When a chat is reopened while its
* agent run is STILL going, this tab is a PASSIVE OBSERVER: it did not start the
* run here (no local SSE stream), so it catches up by POLLING the reconnect
* endpoint (`POST /ai-chat/run`) and merging the run's incrementally-persisted
* assistant message into the rendered thread. These are the small pure decisions
* that machinery hangs off, extracted so they can be unit-tested in isolation
* (mirrors how reindex polling / editor-sync-state are tested).
*/
/** How often to re-poll the reconnect endpoint while a run is ACTIVE. */
export const RUN_POLL_INTERVAL_MS = 2000;
// 'pending' and 'running' are the two ACTIVE statuses; 'succeeded' | 'failed' |
// 'aborted' are TERMINAL (and any unknown future status is treated as terminal,
// so a stale/odd value never polls forever).
const ACTIVE_STATUSES = new Set(["pending", "running"]);
/** Whether a run is still going (worth polling / merging live updates from). */
export function isRunActive(run: IAiChatRun | null | undefined): boolean {
return !!run && ACTIVE_STATUSES.has(run.status);
}
/**
* The TanStack Query `refetchInterval` value for the run query: poll every
* {@link RUN_POLL_INTERVAL_MS} while the run is active, and `false` (stop) once
* it is terminal or there is no run. Polling is thus naturally bounded by the run
* reaching a terminal status — no separate timeout cap is needed.
*/
export function runPollInterval(
run: IAiChatRun | null | undefined,
): number | false {
return isRunActive(run) ? RUN_POLL_INTERVAL_MS : false;
}
/**
* Observer-vs-streamer decision. We render the polled run message (catch up +
* keep advancing) ONLY when this tab is a passive observer: there IS a run AND
* this tab is NOT the one locally streaming it (we reconnected, we didn't start
* it here). When this tab is the streamer, the live SSE stream owns the view, so
* we neither poll nor merge — avoiding a double-render fight. Terminal runs still
* merge (so the final persisted output is shown on reopen); the poll itself is
* stopped separately by {@link runPollInterval}.
*/
export function shouldObserveRun(
run: IAiChatRun | null | undefined,
localStreaming: boolean,
): boolean {
return !!run && !localStreaming;
}
/**
* Should the "stopping" latch — which suppresses the observer re-stream flash
* after the user pressed Stop — be RELEASED now? All three must hold:
* - `stoppingRun`: we actually requested a stop (otherwise nothing to release);
* - `!isLocalStreaming`: this tab is NOT the local streamer. While we are the
* streamer the run query is disabled, so the observed `run` is not the run we
* are following — releasing the latch then would re-open the flash for the
* current turn the instant we switch to observer role;
* - the observed `run` EXISTS and has reached a TERMINAL status.
*
* The null / still-active `run` case is the #234 F4 invariant. On Stop the stale
* PREVIOUS-turn run is removed from the query cache (`removeQueries`), so `run`
* is null until the CURRENT turn's run is re-fetched fresh; a null or active run
* therefore HOLDS the latch, so it can only ever clear against the current turn's
* OWN terminal run — never a stale cached one. (The cache removal itself is
* integration-level in AiChatWindow; this predicate encodes the decision given
* whatever run is currently observed, and a stale terminal run is
* indistinguishable from a current terminal run at the predicate level — hence
* the cache removal is what guarantees only the current run is ever passed here.)
*/
export function shouldClearStoppingLatch(args: {
stoppingRun: boolean;
run: IAiChatRun | null | undefined;
isLocalStreaming: boolean;
}): boolean {
const { stoppingRun, run, isLocalStreaming } = args;
if (!stoppingRun || isLocalStreaming) return false;
return !!run && !isRunActive(run);
}
/**
* Should the "stopping" latch be RELEASED by the run-query ERROR safety-net?
* (#234 F7 — a NEW path of the same re-stream flash the F4 latch exists to
* prevent.) After Stop, `handleServerStop` clears the run cache; the terminal
* effect then holds the latch via `if (!run) return` until the CURRENT turn's run
* is fetched fresh. If that refetch instead ERRORS permanently, `run` stays null,
* its status-keyed refetchInterval is off, and nothing would ever observe a
* terminal run — freezing the view with the observer merge suppressed. This
* safety-net cures ONLY that genuine permanent-null-freeze.
*
* All four must hold:
* - `stoppingRun`: we actually requested a stop (otherwise nothing to release);
* - `!isLocalStreaming`: this tab is NOT the local streamer (same reason as
* {@link shouldClearStoppingLatch});
* - `runQueryFailed`: the run query is in its error state (TanStack Query v5 with
* retry:false — isError);
* - `!isRunActive(run)`: the observed `run` is NOT an active (pending/running)
* held run. This is the F7 gate. In TanStack Query v5 the query's `data` is
* RETAINED on error, so `runQueryFailed` can be true while `run` is STILL an
* ACTIVE run (a single transient GET-run failure in the window between Stop and
* settle). Without this gate a transient error would release the latch early —
* re-opening the observer merge and flashing the growing detached run over the
* frozen row (exactly the F4 flash). Gating on the run NOT being active means we
* only ever cure the permanent-null-freeze (`run === null`, so
* `isRunActive(null)` is false), never release against an active run.
*
* (A terminal `run` also satisfies `!isRunActive(run)`; clearing then is harmless
* — the terminal effect's {@link shouldClearStoppingLatch} already clears the
* latch for a terminal run, so this only ever agrees with it, never conflicts.)
*
* INVARIANT (do not break): clearing the latch on the `run === null` branch is safe
* ONLY because the run query's `refetchInterval` (see {@link runPollInterval}) stops
* polling when the data is empty — so after we clear on null+error there is no
* subsequent auto-poll that could return a still-active detached run and re-open the
* merge. If `refetchInterval` is ever changed to keep polling on `run === null`/on
* error, this null-branch clear would re-open the F7 flash through the null path.
* Do not change the run query's refetchInterval without re-checking this path.
*/
export function shouldClearLatchOnQueryError(args: {
stoppingRun: boolean;
isLocalStreaming: boolean;
runQueryFailed: boolean;
run: IAiChatRun | null | undefined;
}): boolean {
const { stoppingRun, isLocalStreaming, runQueryFailed, run } = args;
return (
stoppingRun && !isLocalStreaming && runQueryFailed && !isRunActive(run)
);
}
/**
* Merge an observed assistant message into the rendered list: replace the message
* with the same id in place (the in-progress assistant row is already seeded from
* history, so per-step growth replaces it), or append it when absent. Returns a
* new array; the input is never mutated.
*/
export function mergeObservedMessage(
messages: UIMessage[],
observed: UIMessage | null | undefined,
): UIMessage[] {
if (!observed) return messages;
const idx = messages.findIndex((m) => m.id === observed.id);
if (idx === -1) return [...messages, observed];
const next = messages.slice();
next[idx] = observed;
return next;
}
@@ -1,6 +1,7 @@
import { describe, it, expect } from "vitest"; import { describe, it, expect } from "vitest";
import { import {
toolCitations, toolCitations,
toolInputSummary,
toolRunState, toolRunState,
type ToolUiPart, type ToolUiPart,
} from "./tool-parts"; } from "./tool-parts";
@@ -77,6 +78,138 @@ describe("toolCitations", () => {
}); });
}); });
describe("toolInputSummary", () => {
it("returns the primary `query` string", () => {
const part: ToolUiPart = {
type: "tool-Search_web_search",
state: "input-available",
input: { query: "hello world" },
};
expect(toolInputSummary(part)).toBe("hello world");
});
it("summarizes a primary array field with a (+N) suffix", () => {
// `urls` is an external MCP read_pages-style list; the first element plus a
// count of the rest.
const part: ToolUiPart = {
type: "tool-read_pages",
state: "input-available",
input: { urls: ["a", "b", "c"] },
};
expect(toolInputSummary(part)).toBe("a (+2)");
});
it("omits the (+N) suffix for a single-element array", () => {
const part: ToolUiPart = {
type: "tool-read_pages",
state: "input-available",
input: { urls: ["only"] },
};
expect(toolInputSummary(part)).toBe("only");
});
it("falls back to `title` for a page op with no query", () => {
const part: ToolUiPart = {
type: "tool-createPage",
state: "input-available",
input: { pageId: "x", title: "My Page" },
};
expect(toolInputSummary(part)).toBe("My Page");
});
it("prefers the earlier primary field when several are present", () => {
const part: ToolUiPart = {
type: "tool-x",
state: "input-available",
// `query` outranks `title` in PRIMARY_INPUT_FIELDS — the ordered list is
// the contract, so a reordering must break this test.
input: { query: "Q", title: "T" },
};
expect(toolInputSummary(part)).toBe("Q");
});
it("does not clamp a value exactly at the 140-char limit", () => {
const exact = "a".repeat(140);
const part: ToolUiPart = {
type: "tool-Search_web_search",
state: "input-available",
input: { query: exact },
};
const out = toolInputSummary(part)!;
expect(out).toBe(exact);
expect(out.endsWith("…")).toBe(false);
expect(out.length).toBe(140);
});
it("clamps one char over the limit (141 -> 140 + ellipsis)", () => {
const part: ToolUiPart = {
type: "tool-Search_web_search",
state: "input-available",
input: { query: "a".repeat(141) },
};
const out = toolInputSummary(part)!;
expect(out.endsWith("…")).toBe(true);
expect(out.length).toBe(141);
expect(out).toBe("a".repeat(140) + "…");
});
it("clamps a long value to ~140 chars with an ellipsis", () => {
const long = "a".repeat(300);
const part: ToolUiPart = {
type: "tool-Search_web_search",
state: "input-available",
input: { query: long },
};
const out = toolInputSummary(part)!;
expect(out.endsWith("…")).toBe(true);
expect(out.length).toBeLessThanOrEqual(141);
});
it("collapses newlines and repeated spaces to single spaces", () => {
const part: ToolUiPart = {
type: "tool-Search_web_search",
state: "input-available",
input: { query: " foo\n\n bar baz " },
};
expect(toolInputSummary(part)).toBe("foo bar baz");
});
it("returns undefined with no input", () => {
expect(
toolInputSummary({ type: "tool-x", state: "input-available" }),
).toBeUndefined();
});
it("returns undefined for an empty object input", () => {
expect(
toolInputSummary({
type: "tool-x",
state: "input-available",
input: {},
}),
).toBeUndefined();
});
it("returns undefined for a non-object input", () => {
expect(
toolInputSummary({
type: "tool-x",
state: "input-available",
input: "just a string",
}),
).toBeUndefined();
});
it("returns undefined while the input is still streaming (even with a full input)", () => {
const part: ToolUiPart = {
type: "tool-Search_web_search",
state: "input-streaming",
input: { query: "hello world" },
};
expect(toolInputSummary(part)).toBeUndefined();
});
});
describe("toolRunState", () => { describe("toolRunState", () => {
it('maps "output-error" to error', () => { it('maps "output-error" to error', () => {
expect(toolRunState("output-error")).toBe("error"); expect(toolRunState("output-error")).toBe("error");
@@ -97,6 +97,69 @@ function asString(value: unknown): string | undefined {
return typeof value === "string" && value.length > 0 ? value : undefined; return typeof value === "string" && value.length > 0 ? value : undefined;
} }
/** Collapse runs of whitespace/newlines to a single space and trim. */
function collapse(s: string): string {
return s.replace(/\s+/g, " ").trim();
}
/** Truncate to ~140 chars, appending an ellipsis when it overflows. */
function clamp(s: string): string {
const MAX = 140;
return s.length > MAX ? s.slice(0, MAX).trimEnd() + "…" : s;
}
/**
* Priority "primary" argument fields, in order. The first present one supplies
* the summary. `urls` is included (external MCP `read_pages`-style tools take a
* list of URLs) and is handled as an array; `url` covers the single-URL form.
*/
const PRIMARY_INPUT_FIELDS = [
"query",
"q",
"searchQuery",
"url",
"urls",
"title",
"name",
"text",
"prompt",
] as const;
/**
* A short, PLAIN-TEXT one-line summary of a tool call's arguments (e.g. the
* search query), or undefined when no recognizable primary field is present.
* Rendered under the tool label so tools without a friendly name (external MCP
* tools like `Search_web_search`) still show WHAT was requested, not just a
* generic "Ran tool {{name}}". The returned string is plain text and MUST be
* rendered React-escaped (Mantine `<Text>`), never as markdown/HTML.
*
* Streaming gate: while `state === "input-streaming"` the `input` object grows
* chunk by chunk but `messageSignature` deliberately does NOT track `input`, so
* a live summary computed here would freeze at its first captured value and go
* stale. We therefore return undefined until the state flips to
* `input-available` (input finalized) — that state change IS tracked by the
* signature, so the row re-renders and shows the complete summary. Do NOT add
* `input` to `message-signature.ts` to work around this.
*/
export function toolInputSummary(part: ToolUiPart): string | undefined {
if (part.state === "input-streaming") return undefined;
if (!part.input || typeof part.input !== "object") return undefined;
const input = part.input as Record<string, unknown>;
for (const field of PRIMARY_INPUT_FIELDS) {
const value = input[field];
if (typeof value === "string" && value.length > 0) {
return clamp(collapse(value));
}
if (Array.isArray(value) && value.length > 0) {
const first = collapse(String(value[0]));
if (first.length === 0) continue;
return clamp(first + (value.length > 1 ? ` (+${value.length - 1})` : ""));
}
}
return undefined;
}
/** /**
* Resolve the page citation(s) a tool part references, from its input/output. * Resolve the page citation(s) a tool part references, from its input/output.
* Only output-available parts (the tool returned) yield citations. Search * Only output-available parts (the tool returned) yield citations. Search
@@ -1,8 +1,5 @@
import { atom } from "jotai"; import { atom } from "jotai";
// Type-only: these atoms only hold an Editor reference for typing. A value import { Editor } from "@tiptap/core";
// import would drag the whole @tiptap/core engine into the eager graph of every
// shell component that reads one of these atoms.
import type { Editor } from "@tiptap/core";
import { PageEditMode } from "@/features/user/types/user.types.ts"; import { PageEditMode } from "@/features/user/types/user.types.ts";
import type { DictationUnavailableReason } from "@/features/dictation/dictation-status"; import type { DictationUnavailableReason } from "@/features/dictation/dictation-status";
@@ -1,16 +0,0 @@
import { lazy, Suspense } from "react";
import { EditorMenuProps } from "@/features/editor/components/table/types/types.ts";
// Lazily load the drawio bubble menu so it is split out of the editor chunk and
// fetched only when an editable editor is mounted (mirrors excalidraw-menu-lazy).
const DrawioMenu = lazy(
() => import("@/features/editor/components/drawio/drawio-menu.tsx"),
);
export default function DrawioMenuLazy(props: EditorMenuProps) {
return (
<Suspense fallback={null}>
<DrawioMenu {...props} />
</Suspense>
);
}
@@ -1,17 +0,0 @@
import { lazy, Suspense } from "react";
import { NodeViewProps } from "@tiptap/react";
// Lazily load the drawio node view so the heavy react-drawio embed runtime is
// split into its own chunk and fetched only when a drawio diagram is actually
// rendered (mirrors excalidraw-view-lazy).
const DrawioView = lazy(
() => import("@/features/editor/components/drawio/drawio-view.tsx"),
);
export default function DrawioViewLazy(props: NodeViewProps) {
return (
<Suspense fallback={null}>
<DrawioView {...props} />
</Suspense>
);
}
@@ -1,19 +0,0 @@
import { lazy, Suspense } from "react";
import { NodeViewProps } from "@tiptap/react";
// Lazily load the KaTeX-backed block math view so the katex chunk is fetched
// only when a document actually contains a math node (mirrors the mermaid/
// excalidraw lazy pattern). The local Suspense keeps a slow katex chunk from
// crashing or blocking the whole editor: while it loads we render the raw
// LaTeX source as a node-sized placeholder.
const MathBlockView = lazy(
() => import("@/features/editor/components/math/math-block.tsx"),
);
export default function MathBlockViewLazy(props: NodeViewProps) {
return (
<Suspense fallback={<div data-katex="true">{props.node.attrs.text}</div>}>
<MathBlockView {...props} />
</Suspense>
);
}
@@ -1,19 +0,0 @@
import { lazy, Suspense } from "react";
import { NodeViewProps } from "@tiptap/react";
// Lazily load the KaTeX-backed inline math view so the katex chunk is fetched
// only when a document actually contains a math node (mirrors the mermaid/
// excalidraw lazy pattern). The local Suspense keeps a slow katex chunk from
// crashing or blocking the whole editor: while it loads we render the raw
// LaTeX source as a node-sized placeholder.
const MathInlineView = lazy(
() => import("@/features/editor/components/math/math-inline.tsx"),
);
export default function MathInlineViewLazy(props: NodeViewProps) {
return (
<Suspense fallback={<span data-katex="true">{props.node.attrs.text}</span>}>
<MathInlineView {...props} />
</Suspense>
);
}
@@ -81,8 +81,8 @@ import {
createResizeHandle, createResizeHandle,
buildResizeClasses, buildResizeClasses,
} from "@/features/editor/components/common/node-resize-handles.ts"; } from "@/features/editor/components/common/node-resize-handles.ts";
import MathInlineView from "@/features/editor/components/math/math-inline-lazy.tsx"; import MathInlineView from "@/features/editor/components/math/math-inline.tsx";
import MathBlockView from "@/features/editor/components/math/math-block-lazy.tsx"; import MathBlockView from "@/features/editor/components/math/math-block.tsx";
import ImageView from "@/features/editor/components/image/image-view.tsx"; import ImageView from "@/features/editor/components/image/image-view.tsx";
import CalloutView from "@/features/editor/components/callout/callout-view.tsx"; import CalloutView from "@/features/editor/components/callout/callout-view.tsx";
import StatusView from "@/features/editor/components/status/status-view.tsx"; import StatusView from "@/features/editor/components/status/status-view.tsx";
@@ -90,7 +90,7 @@ import VideoView from "@/features/editor/components/video/video-view.tsx";
import AudioView from "@/features/editor/components/audio/audio-view.tsx"; import AudioView from "@/features/editor/components/audio/audio-view.tsx";
import AttachmentView from "@/features/editor/components/attachment/attachment-view.tsx"; import AttachmentView from "@/features/editor/components/attachment/attachment-view.tsx";
import CodeBlockView from "@/features/editor/components/code-block/code-block-view.tsx"; import CodeBlockView from "@/features/editor/components/code-block/code-block-view.tsx";
import DrawioView from "../components/drawio/drawio-view-lazy.tsx"; import DrawioView from "../components/drawio/drawio-view";
import ExcalidrawView from "@/features/editor/components/excalidraw/excalidraw-view-lazy.tsx"; import ExcalidrawView from "@/features/editor/components/excalidraw/excalidraw-view-lazy.tsx";
import EmbedView from "@/features/editor/components/embed/embed-view.tsx"; import EmbedView from "@/features/editor/components/embed/embed-view.tsx";
import HtmlEmbedView from "@/features/editor/components/html-embed/html-embed-view.tsx"; import HtmlEmbedView from "@/features/editor/components/html-embed/html-embed-view.tsx";
@@ -1,17 +1,8 @@
import { useEffect, useRef } from "react"; import { useEffect, useRef } from "react";
import { useNavigate } from "react-router-dom"; import { useNavigate } from "react-router-dom";
import { getDefaultStore } from "jotai"; import { getDefaultStore } from "jotai";
import { WebSocketStatus } from "@hocuspocus/provider";
// Literal value of WebSocketStatus.Connected from @hocuspocus/provider. Inlined import { Editor } from "@tiptap/core";
// so this always-mounted global bridge does not statically import
// @hocuspocus/provider — that import pulls Yjs (and, through a shared chunk, the
// whole TipTap engine) into the eager startup graph. yjsConnectionStatusAtom
// already stores these raw status strings.
const YJS_STATUS_CONNECTED = "connected";
// Type-only: importing Editor as a type keeps @tiptap/core (the whole editor
// engine) out of the eager global-shell graph — the bridge only uses it for
// annotations/casts, never as a runtime value.
import type { Editor } from "@tiptap/core";
import { import {
pageEditorAtom, pageEditorAtom,
yjsConnectionStatusAtom, yjsConnectionStatusAtom,
@@ -25,19 +16,16 @@ import {
getSidebarPages, getSidebarPages,
} from "@/features/page/services/page-service.ts"; } from "@/features/page/services/page-service.ts";
import { buildPageUrl } from "@/features/page/page.utils.ts"; import { buildPageUrl } from "@/features/page/page.utils.ts";
// Types are erased at build time, so importing them does not pull the module's import {
// runtime (which drags in @tiptap + the editor-ext barrel). The actual recording
// helpers are dynamically imported at call time inside createPageWithRecording,
// keeping the editor engine out of the eager global-shell startup graph — the
// bridge is mounted for every authenticated user but recording is a rare,
// native-host-driven action.
import type {
GitmostBridge, GitmostBridge,
GitmostCreatePagePayload, GitmostCreatePagePayload,
GitmostCreatePageResult, GitmostCreatePageResult,
GitmostListPagesPayload, GitmostListPagesPayload,
GitmostListPagesResult, GitmostListPagesResult,
GitmostListSpacesResult, GitmostListSpacesResult,
gitmostDecodePayloadToFile,
gitmostInsertTranscriptIntoEditor,
gitmostUploadFileToEditor,
} from "@/features/editor/gitmost/gitmost-recording.ts"; } from "@/features/editor/gitmost/gitmost-recording.ts";
// How long to wait for a freshly-navigated page's editor to mount, become // How long to wait for a freshly-navigated page's editor to mount, become
@@ -70,7 +58,7 @@ function gitmostWaitForEditor(
!editor.isDestroyed && !editor.isDestroyed &&
editor.isEditable && editor.isEditable &&
editorPageId === pageId && editorPageId === pageId &&
yjsStatus === YJS_STATUS_CONNECTED; yjsStatus === WebSocketStatus.Connected;
if (ready) { if (ready) {
resolve(editor); resolve(editor);
return; return;
@@ -184,15 +172,6 @@ export default function GitmostGlobalBridge() {
}; };
} }
// Load the recording helpers on demand (see the import note above). This
// is the only place they are needed, so the @tiptap/editor-ext code they
// pull in stays out of the eager startup graph.
const {
gitmostDecodePayloadToFile,
gitmostUploadFileToEditor,
gitmostInsertTranscriptIntoEditor,
} = await import("@/features/editor/gitmost/gitmost-recording.ts");
// Validate/decode the recording BEFORE creating the page so a bad // Validate/decode the recording BEFORE creating the page so a bad
// payload never leaves an empty junk page behind. Per the createPage // payload never leaves an empty junk page behind. Per the createPage
// error contract, any decode failure collapses to "insert-failed" (the // error contract, any decode failure collapses to "insert-failed" (the
@@ -59,7 +59,7 @@ import {
handlePaste, handlePaste,
} from "@/features/editor/components/common/editor-paste-handler.tsx"; } from "@/features/editor/components/common/editor-paste-handler.tsx";
import ExcalidrawMenu from "./components/excalidraw/excalidraw-menu-lazy"; import ExcalidrawMenu from "./components/excalidraw/excalidraw-menu-lazy";
import DrawioMenu from "./components/drawio/drawio-menu-lazy"; import DrawioMenu from "./components/drawio/drawio-menu";
import { useCollabToken } from "@/features/auth/queries/auth-query.tsx"; import { useCollabToken } from "@/features/auth/queries/auth-query.tsx";
import SearchAndReplaceDialog from "@/features/editor/components/search-and-replace/search-and-replace-dialog.tsx"; import SearchAndReplaceDialog from "@/features/editor/components/search-and-replace/search-and-replace-dialog.tsx";
import { useDebouncedCallback, useDocumentVisibility } from "@mantine/hooks"; import { useDebouncedCallback, useDocumentVisibility } from "@mantine/hooks";
@@ -0,0 +1,113 @@
import { describe, it, expect } from "vitest";
import { Editor } from "@tiptap/core";
import { Document } from "@tiptap/extension-document";
import { Paragraph } from "@tiptap/extension-paragraph";
import { Text } from "@tiptap/extension-text";
import { EditorState, TextSelection } from "@tiptap/pm/state";
import type { Node as PMNode } from "@tiptap/pm/model";
import { UniqueID } from "@docmost/editor-ext";
import { getEditorSelectionContext } from "./get-editor-selection";
/**
* Unit tests for getEditorSelectionContext (#388). Built on a headless
* ProseMirror schema (Document + Paragraph + Text + the block-id UniqueID
* extension), mirroring the editor-ext test style. We assemble docs with
* explicit block ids so the covered-blockIds assertions are deterministic.
*/
// A schema that carries the `id` block attribute (UniqueID) on paragraphs, just
// like the real editor.
const { schema } = new Editor({
extensions: [
Document,
Paragraph,
Text,
UniqueID.configure({ types: ["paragraph"] }),
],
content: "",
});
function docOf(blocks: { id: string; text: string }[]): PMNode {
return schema.node(
"doc",
null,
blocks.map((b) =>
schema.node("paragraph", { id: b.id }, b.text ? schema.text(b.text) : []),
),
);
}
function stateWith(doc: PMNode, from: number, to: number): EditorState {
const base = EditorState.create({ schema, doc });
return base.apply(base.tr.setSelection(TextSelection.create(doc, from, to)));
}
// Select every text position of the doc (pos 1 .. content.size - 1).
function selectAll(doc: PMNode): EditorState {
return stateWith(doc, 1, doc.content.size - 1);
}
describe("getEditorSelectionContext", () => {
it("returns null for an empty (collapsed) selection", () => {
const doc = docOf([{ id: "b1", text: "Hello world" }]);
const state = stateWith(doc, 3, 3); // caret, from === to
expect(getEditorSelectionContext(state)).toBeNull();
});
it("returns null for the default caret-at-start of a fresh editor", () => {
const editor = new Editor({
extensions: [
Document,
Paragraph,
Text,
UniqueID.configure({ types: ["paragraph"] }),
],
content: "<p>fresh</p>",
});
expect(getEditorSelectionContext(editor.state)).toBeNull();
editor.destroy();
});
it("reads a single-paragraph selection with no block-separator artifacts", () => {
const doc = docOf([{ id: "b1", text: "Hello world" }]);
const sel = getEditorSelectionContext(selectAll(doc))!;
expect(sel.text).toBe("Hello world");
expect(sel.blockIds).toEqual(["b1"]);
expect(sel.truncated).toBeUndefined();
});
it("joins multiple blocks with a newline and collects all covered blockIds", () => {
const doc = docOf([
{ id: "b1", text: "First" },
{ id: "b2", text: "Second" },
]);
const sel = getEditorSelectionContext(selectAll(doc))!;
expect(sel.text).toBe("First\nSecond");
expect(sel.blockIds).toEqual(["b1", "b2"]);
});
it("caps the text at 2000 chars and flags truncated", () => {
const doc = docOf([{ id: "b1", text: "x".repeat(2500) }]);
const sel = getEditorSelectionContext(selectAll(doc))!;
expect(sel.text).toHaveLength(2000);
expect(sel.truncated).toBe(true);
});
it("computes before/after context and clamps it to the doc bounds", () => {
// One paragraph "0123456789abcdefghij"; select the middle "56789".
const doc = docOf([{ id: "b1", text: "0123456789abcdefghij" }]);
// text char i lives at pos (1 + i); select chars index 5..9 -> pos 6..11.
const sel = getEditorSelectionContext(stateWith(doc, 6, 11))!;
expect(sel.text).toBe("56789");
expect(sel.before).toBe("01234");
expect(sel.after).toBe("abcdefghij");
});
it("omits before/after at the document boundaries (never reads past 0/size)", () => {
const doc = docOf([{ id: "b1", text: "Edge" }]);
const sel = getEditorSelectionContext(selectAll(doc))!;
// Selection spans the whole single block: nothing before or after it.
expect(sel.before).toBeUndefined();
expect(sel.after).toBeUndefined();
});
});
@@ -0,0 +1,71 @@
import type { EditorState } from "@tiptap/pm/state";
export interface EditorSelectionContext {
text: string;
truncated?: boolean;
blockIds?: string[];
before?: string;
after?: string;
}
// Client-side caps. The server re-caps every field independently (defence in
// depth — the payload is attacker-controllable), so these only keep the wire
// small for the common case.
const TEXT_CAP = 2000;
const CONTEXT_CHARS = 160;
const MAX_BLOCK_IDS = 20;
// Pure: takes an EditorState so it is unit-testable with a headless editor.
// Snapshots the user's current selection into the wire shape carried inside
// openPage — plain text + the ids of the blocks it covers + a little surrounding
// context. Returns null when nothing meaningful is selected.
//
// Deliberately does NOT emit the ProseMirror positions (from/to): they rot the
// instant the document changes and the server tools address content by block id
// + text (getNode / editPageText find-replace), never by position.
export function getEditorSelectionContext(
state: EditorState,
): EditorSelectionContext | null {
const { selection, doc } = state;
// An empty selection (incl. the default caret-at-start of a fresh editor) is
// never a "this"/"here" — bail before reading any text.
if (selection.empty) return null;
const { from, to } = selection;
let text = doc.textBetween(from, to, "\n");
let truncated = false;
if (text.length > TEXT_CAP) {
text = text.slice(0, TEXT_CAP);
truncated = true;
}
// A selection spanning only non-text nodes (e.g. an image) trims to empty ->
// treat as no selection.
if (text.trim().length === 0) return null;
// Ids of every block the selection covers, deduped and capped. These bridge
// the plain-text selection to the server tools (getNode / editPageText).
const blockIds: string[] = [];
doc.nodesBetween(from, to, (node) => {
const id = node.isBlock ? node.attrs?.id : undefined;
if (typeof id === "string" && id.length > 0 && !blockIds.includes(id)) {
blockIds.push(id);
}
});
// ~160 chars of plain text on each side, clamped to the document bounds, so
// editPageText can disambiguate a duplicate of the selected text.
const before = doc.textBetween(Math.max(0, from - CONTEXT_CHARS), from, "\n");
const after = doc.textBetween(
to,
Math.min(doc.content.size, to + CONTEXT_CHARS),
"\n",
);
const result: EditorSelectionContext = { text };
if (truncated) result.truncated = true;
if (blockIds.length > 0) result.blockIds = blockIds.slice(0, MAX_BLOCK_IDS);
if (before.length > 0) result.before = before;
if (after.length > 0) result.after = after;
return result;
}
@@ -165,6 +165,9 @@ export default function ShareAiWidget({
isStreaming={isStreaming} isStreaming={isStreaming}
assistantName={assistantName} assistantName={assistantName}
showCitations={false} showCitations={false}
// Anonymous reader: suppress the tool-argument summary line so the
// agent's raw query/argument text isn't shown on the public share.
showInput={false}
// Anonymous reader: neutralize internal/relative links in the // Anonymous reader: neutralize internal/relative links in the
// assistant's markdown so internal UUIDs/auth-gated routes don't // assistant's markdown so internal UUIDs/auth-gated routes don't
// leak as clickable links (external http(s) links are kept). // leak as clickable links (external http(s) links are kept).
@@ -1,20 +1,10 @@
import { Suspense } from "react";
import { Outlet } from "react-router-dom"; import { Outlet } from "react-router-dom";
import { Center, Loader } from "@mantine/core";
import ShareShell from "@/features/share/components/share-shell.tsx"; import ShareShell from "@/features/share/components/share-shell.tsx";
export default function ShareLayout() { export default function ShareLayout() {
return ( return (
<ShareShell> <ShareShell>
<Suspense <Outlet />
fallback={
<Center h="60vh">
<Loader size="sm" />
</Center>
}
>
<Outlet />
</Suspense>
</ShareShell> </ShareShell>
); );
} }
+1 -1
View File
@@ -1,7 +1,7 @@
// Source: https://github.com/mantinedev/mantine/blob/master/packages/@mantine/hooks/src/use-clipboard/use-clipboard.ts // Source: https://github.com/mantinedev/mantine/blob/master/packages/@mantine/hooks/src/use-clipboard/use-clipboard.ts
// polyfilled to support execCommand fallback // polyfilled to support execCommand fallback
import { useState } from "react"; import { useState } from "react";
import { execCommandCopy } from "@/lib/copy-to-clipboard.ts"; import { execCommandCopy } from "@docmost/editor-ext";
export type UseClipboardOptions = { export type UseClipboardOptions = {
timeout?: number; timeout?: number;
+1 -1
View File
@@ -1,7 +1,7 @@
import bytes from "bytes"; import bytes from "bytes";
import { castToBoolean } from "@/lib/utils.tsx"; import { castToBoolean } from "@/lib/utils.tsx";
import { AvatarIconType } from "@/features/attachments/types/attachment.types.ts"; import { AvatarIconType } from "@/features/attachments/types/attachment.types.ts";
import { sanitizeUrl } from "@/lib/sanitize-url.ts"; import { sanitizeUrl } from "@docmost/editor-ext";
declare global { declare global {
interface Window { interface Window {
-16
View File
@@ -1,16 +0,0 @@
// Client-local execCommand copy fallback (previously imported from
// @docmost/editor-ext). It lives here so the ubiquitous useClipboard / CopyButton
// path does not pull in the editor-ext barrel — and with it the whole TipTap
// engine — through the eager startup graph. Behavior is identical to the
// editor-ext helper it replaces.
export function execCommandCopy(text: string): void {
const textarea = document.createElement("textarea");
textarea.value = text;
textarea.style.position = "fixed";
textarea.style.left = "-9999px";
textarea.style.top = "-9999px";
document.body.appendChild(textarea);
textarea.select();
document.execCommand("copy");
document.body.removeChild(textarea);
}
-31
View File
@@ -1,31 +0,0 @@
import { describe, it, expect } from "vitest";
import { sanitizeUrl } from "./sanitize-url";
// `sanitizeUrl` is a byte-identical client-local copy of editor-ext's wrapper
// around @braintree/sanitize-url: it maps the sanitizer's "about:blank" XSS
// sentinel to "". These assertions mirror editor-ext's own security-contract
// test so the extracted copy keeps the same guarantees.
describe("sanitizeUrl", () => {
it("blocks dangerous schemes (returns empty string)", () => {
expect(sanitizeUrl("javascript:alert(1)")).toBe("");
expect(sanitizeUrl("data:text/html,<script>alert(1)</script>")).toBe("");
expect(sanitizeUrl("vbscript:msgbox(1)")).toBe("");
// Case / whitespace obfuscation must not slip past the sanitizer.
expect(sanitizeUrl(" JaVaScRiPt:alert(1)")).toBe("");
});
it("returns empty string for empty / undefined input", () => {
expect(sanitizeUrl(undefined)).toBe("");
expect(sanitizeUrl("")).toBe("");
});
it("allows safe https, relative file and mailto URLs", () => {
expect(sanitizeUrl("https://example.com/page")).toMatch(
/^https:\/\/example\.com\/page/,
);
expect(sanitizeUrl("/api/files/abc-123")).toBe("/api/files/abc-123");
expect(sanitizeUrl("mailto:user@example.com")).toBe(
"mailto:user@example.com",
);
});
});
-15
View File
@@ -1,15 +0,0 @@
import { sanitizeUrl as braintreeSanitizeUrl } from "@braintree/sanitize-url";
// Client-local copy of editor-ext's sanitizeUrl wrapper. Importing it from the
// editor-ext barrel dragged the whole TipTap engine into the eager startup graph
// via the app-wide config module (getFileUrl). This keeps the exact same
// behavior (braintree sanitize + normalize "about:blank" -> "") without that
// dependency.
export function sanitizeUrl(url: string | undefined): string {
if (!url) return "";
const sanitized = braintreeSanitizeUrl(url);
// Return an empty string instead of "about:blank".
return sanitized === "about:blank" ? "" : sanitized;
}
+27 -60
View File
@@ -13,14 +13,15 @@ import { ModalsProvider } from "@mantine/modals";
import { Notifications } from "@mantine/notifications"; import { Notifications } from "@mantine/notifications";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query"; import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { HelmetProvider } from "react-helmet-async"; import { HelmetProvider } from "react-helmet-async";
import { ChunkLoadErrorBoundary } from "@/components/chunk-load-error-boundary.tsx";
import "./i18n"; import "./i18n";
import { PostHogProvider } from "posthog-js/react";
import { import {
getPostHogHost, getPostHogHost,
getPostHogKey, getPostHogKey,
isCloud, isCloud,
isPostHogEnabled, isPostHogEnabled,
} from "@/lib/config.ts"; } from "@/lib/config.ts";
import posthog from "posthog-js";
import { initVitals } from "@/lib/telemetry/vitals"; import { initVitals } from "@/lib/telemetry/vitals";
export const queryClient = new QueryClient({ export const queryClient = new QueryClient({
@@ -34,6 +35,15 @@ export const queryClient = new QueryClient({
}, },
}); });
if (isCloud() && isPostHogEnabled) {
posthog.init(getPostHogKey(), {
api_host: getPostHogHost(),
defaults: "2025-05-24",
disable_session_recording: true,
capture_pageleave: false,
});
}
// #355 — client perf-telemetry. Decides sampling ONCE (25%/session) before // #355 — client perf-telemetry. Decides sampling ONCE (25%/session) before
// subscribing to any observer; non-sampled sessions send nothing. // subscribing to any observer; non-sampled sessions send nothing.
initVitals(); initVitals();
@@ -41,62 +51,19 @@ initVitals();
const container = document.getElementById("root") as HTMLElement; const container = document.getElementById("root") as HTMLElement;
const root = (container as any).__reactRoot ??= ReactDOM.createRoot(container); const root = (container as any).__reactRoot ??= ReactDOM.createRoot(container);
function renderApp() { root.render(
root.render( <BrowserRouter>
<BrowserRouter> <MantineProvider theme={theme} cssVariablesResolver={mantineCssResolver}>
<MantineProvider theme={theme} cssVariablesResolver={mantineCssResolver}> <ModalsProvider>
<ModalsProvider> <QueryClientProvider client={queryClient}>
<QueryClientProvider client={queryClient}> <Notifications position="bottom-center" limit={3} zIndex={10000} />
<Notifications position="bottom-center" limit={3} zIndex={10000} /> <HelmetProvider>
<HelmetProvider> <PostHogProvider client={posthog}>
{/* Root boundary above every lazy route's Suspense: a stale-chunk <App />
404 after a deploy is caught and recovered here instead of </PostHogProvider>
blanking the whole app. */} </HelmetProvider>
<ChunkLoadErrorBoundary> </QueryClientProvider>
<App /> </ModalsProvider>
</ChunkLoadErrorBoundary> </MantineProvider>
</HelmetProvider> </BrowserRouter>,
</QueryClientProvider> );
</ModalsProvider>
</MantineProvider>
</BrowserRouter>,
);
}
async function initAnalytics() {
// posthog-js is only pulled in for cloud deployments with analytics enabled, so
// self-hosted builds never download it. The gate is kept identical to the
// previous eager code so cloud analytics behavior is unchanged; the import is
// simply deferred behind it.
//
// Crucially this runs AFTER the immediate first render below, so first paint is
// never gated on the analytics chunk. Any failure (network, stale 404, or an
// ad-blocker blocking a chunk named "posthog") is swallowed so the user keeps a
// working app without analytics instead of a permanently blank page.
//
// NOTE: we init the posthog SINGLETON only and do NOT wrap the tree in
// <PostHogProvider>. The app has zero consumers of the PostHog React context
// (no usePostHog / useFeatureFlag* / PostHogFeature), and PostHogProvider given
// an already-initialized `client` is a no-op — all capture goes through the
// singleton. Re-rendering to attach the provider would only REMOUNT the whole
// App (running every mount effect twice and dropping local state / focus /
// in-progress input on cloud cold-load) for no functional gain.
if (!(isCloud() && isPostHogEnabled)) return;
try {
const { default: posthog } = await import("posthog-js");
posthog.init(getPostHogKey(), {
api_host: getPostHogHost(),
defaults: "2025-05-24",
disable_session_recording: true,
capture_pageleave: false,
});
} catch {
// Analytics failed to load — degrade gracefully; the app already rendered.
}
}
// Paint immediately for everyone (self-hosted stays exactly as instant as before,
// cloud no longer blocks on the analytics import). The posthog singleton is
// initialized after, without re-rendering the tree.
renderApp();
void initAnalytics();
-14
View File
@@ -63,20 +63,6 @@ export default defineConfig(({ mode }) => {
name: "vendor-mantine", name: "vendor-mantine",
test: /[\\/]node_modules[\\/]@mantine[\\/]/, test: /[\\/]node_modules[\\/]@mantine[\\/]/,
}, },
// NOTE: TipTap/ProseMirror/Yjs are intentionally NOT force-grouped
// into a single vendor chunk. Doing so backfires: rolldown co-locates
// a small module shared with the (eager) react-i18next runtime into
// that group chunk, which then drags the whole ~590KB editor engine
// into the eager modulepreload graph. Left to the default splitting,
// the editor engine stays in lazily-loaded chunks pulled only by the
// route-split editor/share pages. KaTeX is safe to group (nothing
// eager references it).
// KaTeX in its own stable chunk; loaded on demand by the lazy math
// node views (never in the startup path).
{
name: "vendor-katex",
test: /[\\/]node_modules[\\/]katex[\\/]/,
},
], ],
}, },
}, },
@@ -0,0 +1,329 @@
import { Injectable, Logger, OnModuleDestroy } from '@nestjs/common';
/**
* In-memory run-stream registry (#184 phase 1.5). A durable agent run tees its
* SSE frames here (via `pipeUIMessageStreamToResponse({ consumeSseStream })`)
* so a LATE tab one that reloaded, or opened after the starter dropped can
* attach through `GET /ai-chat/runs/:chatId/stream`, replay the frames buffered
* so far, and then follow the live tail as a normal streamer.
*
* This is deliberately single-process and best-effort: it holds nothing the DB
* does not (the run + assistant row are the source of truth), so a process
* restart simply drops in-flight entries and the client falls back to its
* restore + degraded-poll path. The async `attach` return type is the seam for a
* future phase-2 cross-process backend (Redis) the interface does not change.
*/
/** How long a finished entry is retained for late attach (replay + immediate end). */
export const RUN_STREAM_RETAIN_FINISHED_MS = 30_000;
/** Per-run replay buffer cap. Past this the buffer is dropped (attach -> 204). */
export const RUN_STREAM_MAX_BUFFER_BYTES = 4 * 1024 * 1024;
// 2x the replay cap: a just-written 4MB replay burst alone can never trip the
// per-subscriber cap (see controller); only a genuinely stalled socket can.
export const SUBSCRIBER_MAX_BUFFERED_BYTES = 2 * RUN_STREAM_MAX_BUFFER_BYTES;
export interface RunStreamCallbacks {
onFrame: (frame: string) => void;
onEnd: () => void;
}
export interface RunStreamAttachment {
replay: string[];
finished: boolean;
start(): void; // drain pending frames (order preserved) and go live
unsubscribe(): void; // safe to call at any point, idempotent
}
interface Subscriber extends RunStreamCallbacks {
started: boolean;
pending: string[];
// Byte size of `pending`, capped at SUBSCRIBER_MAX_BUFFERED_BYTES. `start()` is
// called in the SAME tick as `attach()` today (see attach), so `pending` never
// holds more than one microtask of frames — but the async `attach` signature is
// a phase-2 seam: an await between attach and start would let a stalled paused
// subscriber buffer the WHOLE run here. The cap is the structural backstop.
pendingBytes: number;
overflowed: boolean;
pendingEnd: boolean;
}
interface Entry {
runId: string;
// The persisted assistant row id of this run (set at bind; undefined if the
// seed failed). Used by the attach anchor check (invariant 6).
assistantMessageId?: string;
frames: string[];
bytes: number;
overflowed: boolean;
finished: boolean;
subscribers: Set<Subscriber>;
retainTimer?: NodeJS.Timeout;
}
@Injectable()
export class AiChatStreamRegistryService implements OnModuleDestroy {
private readonly logger = new Logger(AiChatStreamRegistryService.name);
private readonly entries = new Map<string, Entry>(); // key: chatId
/**
* Register a fresh entry at the START of a run (before any frame), so a tab
* that attaches in the begin->seed window finds an entry to wait on. If an
* entry already exists for this chat (a previous, possibly still-live run whose
* tee loop is draining), it is terminated MIRRORING the done-path (invariant 3)
* so its subscribers are released and its retention timer is cleared; a late
* `done` from that old tee then fires against the closed-over old reference and,
* thanks to identity checks, never touches this new entry.
*/
open(chatId: string, runId: string): void {
const existing = this.entries.get(chatId);
if (existing) {
if (existing.retainTimer) {
clearTimeout(existing.retainTimer);
existing.retainTimer = undefined;
}
// Started subscribers get exactly one onEnd() and are removed; paused ones
// are marked pendingEnd (their start() will end them). finished=true guards
// any later done from the old tee loop from double-notifying.
this.terminateSubscribers(existing);
}
this.entries.set(chatId, {
runId,
frames: [],
bytes: 0,
overflowed: false,
finished: false,
subscribers: new Set<Subscriber>(),
});
}
/**
* Tee a run's SSE frame stream into its entry (called from consumeSseStream).
* No-op with a warning when there is no entry or the entry belongs to a
* different run (invariant 1). The reader loop is fire-and-forget: the tee
* branch outlives the client socket by design.
*/
bind(
chatId: string,
runId: string,
assistantMessageId: string | undefined,
stream: ReadableStream<string>,
): void {
const entry = this.entries.get(chatId);
if (!entry || entry.runId !== runId) {
// Invariant 1: only the matching run may mutate the entry.
this.logger.warn(
`bind: no matching run-stream entry for chat=${chatId} run=${runId}`,
);
return;
}
entry.assistantMessageId = assistantMessageId;
const reader = stream.getReader();
const pump = async (): Promise<void> => {
try {
for (;;) {
const { done, value } = await reader.read();
if (done) break;
this.ingestFrame(entry, value);
}
this.finalizeEntry(chatId, entry);
} catch {
// A read error is a terminal event too — release subscribers.
this.finalizeEntry(chatId, entry);
}
};
void pump();
}
/**
* Terminate a run's entry from the OUTER catch of the stream method (a failure
* before/while wiring the pipe, so `done` will never arrive). Identity-checked
* on runId (invariant 1); the shared terminal path is idempotent.
*/
abortEntry(chatId: string, runId: string): void {
const entry = this.entries.get(chatId);
if (!entry || entry.runId !== runId) return;
this.finalizeEntry(chatId, entry);
}
/**
* Attach to a run's stream. Async only for the phase-2 Redis seam the body
* runs synchronously so the replay snapshot and the subscriber registration
* happen in ONE tick with no await between them (invariant 4): a frame ingested
* concurrently cannot slip into the gap and be lost or duplicated.
*
* Returns null (-> the caller answers 204) when:
* - there is no entry, or it overflowed (replay is gone);
* - expect=live with an anchor that does not match this run's assistant id
* (invariant 6: a stripped tab must never replay a FOREIGN run's transcript);
* - the run finished and the caller did not expect a live tail.
* A finished run with expect=live yields a replay-only attachment (no
* subscriber registered). Otherwise a paused subscriber is registered and the
* caller replays `replay`, then calls start() to drain and go live.
*/
async attach(
chatId: string,
expectLive: boolean,
anchor: string | undefined,
cb: RunStreamCallbacks,
): Promise<RunStreamAttachment | null> {
const entry = this.entries.get(chatId);
if (!entry || entry.overflowed) return null;
// Invariant 6: cross-run replay is forbidden. Before bind, assistantMessageId
// is undefined and mismatches any anchor -> 204 -> client restore+poll path.
if (expectLive && anchor && entry.assistantMessageId !== anchor) return null;
if (entry.finished && !expectLive) return null;
if (entry.finished && expectLive) {
// Replay-only: the run is done, no subscriber is registered.
return {
replay: entry.frames.slice(),
finished: true,
start: () => undefined,
unsubscribe: () => undefined,
};
}
const sub: Subscriber = {
onFrame: cb.onFrame,
onEnd: cb.onEnd,
started: false,
pending: [],
pendingBytes: 0,
overflowed: false,
pendingEnd: false,
};
entry.subscribers.add(sub);
// Snapshot in the SAME synchronous block as the registration (invariant 4).
const replay = entry.frames.slice();
// CONTRACT: the caller MUST call start() in the SAME tick as this attach()
// returns — no await between them. While a subscriber is paused, every frame
// is buffered in sub.pending; a delayed start() lets a whole run accumulate
// there. The pendingBytes cap (see ingestFrame) is the structural backstop if
// that contract is ever broken (e.g. the phase-2 Redis await seam).
return {
replay,
finished: false,
start: () => {
if (sub.overflowed) {
// The pending buffer overflowed while paused: end the stream instead of
// replaying a partial (a 204-equivalent post-attach degrade).
try {
sub.onEnd();
} catch {
// The socket is gone; nothing to end.
}
entry.subscribers.delete(sub);
return;
}
// Deliver frames buffered while paused, in order, then go live.
for (const frame of sub.pending) {
try {
sub.onFrame(frame);
} catch {
entry.subscribers.delete(sub);
return;
}
}
sub.pending = [];
sub.started = true;
if (sub.pendingEnd) {
try {
sub.onEnd();
} catch {
// The socket is gone; nothing to end.
}
entry.subscribers.delete(sub);
}
},
unsubscribe: () => {
entry.subscribers.delete(sub);
},
};
}
onModuleDestroy(): void {
for (const entry of this.entries.values()) {
if (entry.retainTimer) clearTimeout(entry.retainTimer);
}
this.entries.clear();
}
/** Buffer + fan-out a single frame. See invariant/overflow semantics inline. */
private ingestFrame(entry: Entry, frame: string): void {
entry.bytes += Buffer.byteLength(frame);
if (!entry.overflowed) {
entry.frames.push(frame);
if (entry.bytes > RUN_STREAM_MAX_BUFFER_BYTES) {
// The crossing frame was already counted AND (below) fanned out; only the
// replay buffer is dropped. After overflow no more frames are buffered,
// but live fan-out continues.
entry.overflowed = true;
entry.frames = [];
this.logger.warn(
`run-stream buffer overflow for run=${entry.runId}; ` +
`late attach will 204 until the run ends`,
);
}
}
for (const sub of entry.subscribers) {
if (sub.started) {
try {
sub.onFrame(frame);
} catch {
entry.subscribers.delete(sub);
}
} else {
sub.pending.push(frame);
sub.pendingBytes += Buffer.byteLength(frame);
if (sub.pendingBytes > SUBSCRIBER_MAX_BUFFERED_BYTES) {
// The paused subscriber's buffer overflowed — only possible if start()
// was delayed past the same-tick contract (the phase-2 await seam).
// Drop it rather than buffer the whole run; on start() it degrades to an
// immediate end (a 204-equivalent) instead of replaying a partial.
sub.overflowed = true;
sub.pending = [];
entry.subscribers.delete(sub);
}
}
}
}
/**
* Shared terminal path for done / read-error / external-abort. Idempotent: a
* second call (already finished) is a no-op, so an open()-replaced or
* abort-then-done entry is never double-armed or double-ended.
*/
private finalizeEntry(chatId: string, entry: Entry): void {
if (entry.finished) return;
this.terminateSubscribers(entry);
const timer = setTimeout(() => {
// Invariant 2: only delete OUR entry (a replacement may already own the key).
if (this.entries.get(chatId) === entry) this.entries.delete(chatId);
}, RUN_STREAM_RETAIN_FINISHED_MS);
timer.unref?.();
entry.retainTimer = timer;
}
/**
* Mark the entry finished and release its subscribers, mirroring the done-path:
* started subscribers get exactly one onEnd() and are removed; paused ones are
* flagged pendingEnd so their start() ends them. Deleting the current element
* during Set iteration is safe.
*/
private terminateSubscribers(entry: Entry): void {
entry.finished = true;
for (const sub of entry.subscribers) {
if (sub.started) {
try {
sub.onEnd();
} catch {
// The socket is gone; nothing to end.
}
entry.subscribers.delete(sub);
} else {
sub.pendingEnd = true;
}
}
}
}
@@ -0,0 +1,388 @@
import {
AiChatStreamRegistryService,
RUN_STREAM_MAX_BUFFER_BYTES,
RUN_STREAM_RETAIN_FINISHED_MS,
RunStreamCallbacks,
} from './ai-chat-stream-registry.service';
/**
* Unit tests for the in-memory run-stream registry (#184 phase 1.5). The registry
* is the whole of the resumable-transport contract: replay ordering, paused ->
* live hand-off, overflow, retention, the anchor check (invariant 6), and the
* mirror-the-done-path replace semantics (invariant 3). Every enumerated case in
* the issue's task 1.5 has a test here.
*/
// A ReadableStream whose frames the test pushes explicitly, plus close/error.
function makePushStream(): {
stream: ReadableStream<string>;
push: (f: string) => void;
close: () => void;
error: (e?: unknown) => void;
} {
let controller!: ReadableStreamDefaultController<string>;
const stream = new ReadableStream<string>({
start(c) {
controller = c;
},
});
return {
stream,
push: (f) => controller.enqueue(f),
close: () => controller.close(),
error: (e) => controller.error(e ?? new Error('read error')),
};
}
// Let the fire-and-forget pump drain queued frames (reader.read() resolves on a
// macrotask boundary for an already-enqueued value).
const flush = (): Promise<void> => new Promise((r) => setTimeout(r, 0));
function collector(): {
cb: RunStreamCallbacks;
frames: string[];
ended: () => number;
} {
const frames: string[] = [];
let ends = 0;
return {
frames,
ended: () => ends,
cb: {
onFrame: (f) => frames.push(f),
onEnd: () => {
ends += 1;
},
},
};
}
describe('AiChatStreamRegistryService', () => {
const CHAT = 'chat-1';
let registry: AiChatStreamRegistryService;
beforeEach(() => {
registry = new AiChatStreamRegistryService();
jest.spyOn((registry as any).logger, 'warn').mockImplementation(() => {});
});
afterEach(() => {
registry.onModuleDestroy();
});
it('replays frames in arrival order (live attach)', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
src.push('a');
src.push('b');
src.push('c');
await flush();
const c = collector();
const att = await registry.attach(CHAT, false, undefined, c.cb);
expect(att).not.toBeNull();
expect(att!.replay).toEqual(['a', 'b', 'c']);
expect(att!.finished).toBe(false);
});
it('late attach gets the full prefix as replay plus the live tail', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
src.push('a');
src.push('b');
await flush();
const c = collector();
const att = (await registry.attach(CHAT, false, undefined, c.cb))!;
expect(att.replay).toEqual(['a', 'b']);
att.start();
// Live tail arrives after start().
src.push('c');
src.push('d');
await flush();
expect(c.frames).toEqual(['c', 'd']);
});
it('a paused subscriber receives frames buffered during pause in order, then live (no loss/reorder)', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
src.push('a');
await flush();
const c = collector();
// Attach (paused). Frames that arrive BEFORE start() must queue, not drop.
const att = (await registry.attach(CHAT, false, undefined, c.cb))!;
expect(att.replay).toEqual(['a']);
src.push('b'); // arrives while paused -> pending
src.push('c');
await flush();
expect(c.frames).toEqual([]); // nothing delivered yet (paused)
att.start(); // drains pending in order
expect(c.frames).toEqual(['b', 'c']);
src.push('d'); // now live
await flush();
expect(c.frames).toEqual(['b', 'c', 'd']);
});
it('a run that finishes while a subscriber is paused ends it on start()', async () => {
registry.open(CHAT, 'run-1');
const c = collector();
const att = (await registry.attach(CHAT, false, undefined, c.cb))!;
// Terminate the run while the subscriber is still paused.
registry.abortEntry(CHAT, 'run-1');
expect(c.ended()).toBe(0); // paused: not ended yet
att.start();
expect(c.ended()).toBe(1); // start() drains + ends
});
it('finished + expect=live returns a replay WITHOUT registering a subscriber', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
src.push('a');
src.push('b');
src.close();
await flush();
const c = collector();
const att = (await registry.attach(CHAT, true, undefined, c.cb))!;
expect(att.finished).toBe(true);
expect(att.replay).toEqual(['a', 'b']);
// No subscriber registered: start()/unsubscribe are no-ops and the entry has
// zero subscribers.
const entry = (registry as any).entries.get(CHAT);
expect(entry.subscribers.size).toBe(0);
att.start();
expect(c.frames).toEqual([]);
});
it('finished WITHOUT expect=live returns null', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
src.push('a');
src.close();
await flush();
const c = collector();
expect(await registry.attach(CHAT, false, undefined, c.cb)).toBeNull();
});
it('anchor mismatch with expect=live returns null (and null before bind sets assistantMessageId)', async () => {
registry.open(CHAT, 'run-1');
const c = collector();
// Before bind: assistantMessageId is undefined -> mismatches any anchor.
expect(
await registry.attach(CHAT, true, 'assist-1', c.cb),
).toBeNull();
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
src.push('a');
await flush();
// Wrong anchor -> null (cross-run replay forbidden, invariant 6).
expect(await registry.attach(CHAT, true, 'other-id', c.cb)).toBeNull();
});
it('matching anchor with expect=live attaches', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
src.push('a');
await flush();
const c = collector();
const att = await registry.attach(CHAT, true, 'assist-1', c.cb);
expect(att).not.toBeNull();
expect(att!.replay).toEqual(['a']);
});
it('overflow: attach returns null, but the LIVE subscriber keeps receiving (incl. the crossing frame)', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
// A live (started) subscriber attached before the flood.
const c = collector();
const att = (await registry.attach(CHAT, false, undefined, c.cb))!;
att.start();
const oneMb = 'x'.repeat(1024 * 1024);
// 5 x 1MB = 5MB > 4MB cap; the 5th frame is the one that crosses.
for (let i = 0; i < 5; i++) src.push(oneMb + i);
await flush();
const entry = (registry as any).entries.get(CHAT);
expect(entry.overflowed).toBe(true);
expect(entry.bytes).toBeGreaterThan(RUN_STREAM_MAX_BUFFER_BYTES);
// The live subscriber received ALL 5 frames, including the crossing one.
expect(c.frames).toHaveLength(5);
expect(c.frames[4]).toBe(oneMb + 4);
// A NEW attach after overflow gets null (replay buffer is gone).
const c2 = collector();
expect(await registry.attach(CHAT, false, undefined, c2.cb)).toBeNull();
});
it('a paused subscriber whose pending buffer overflows is dropped and ends on start(); other subscribers keep receiving', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
// A: paused (start() deliberately delayed to simulate the phase-2 await seam).
const a = collector();
const attA = (await registry.attach(CHAT, false, undefined, a.cb))!;
// B: live (started) — its delivery must be unaffected by A's overflow.
const b = collector();
const attB = (await registry.attach(CHAT, false, undefined, b.cb))!;
attB.start();
const oneMb = 'x'.repeat(1024 * 1024);
// 9 x 1MB = 9MB > 8MB per-subscriber cap; A's pending overflows, B streams live.
for (let i = 0; i < 9; i++) src.push(oneMb + i);
await flush();
const entry = (registry as any).entries.get(CHAT);
// A was dropped from the subscriber set on overflow; B (started) remains.
expect(entry.subscribers.size).toBe(1);
expect(a.frames).toEqual([]); // paused + overflowed: nothing was delivered
// B received every frame live (delivery unaffected by A's overflow).
expect(b.frames).toHaveLength(9);
// A's start() (arriving late) degrades to an immediate end, not a partial replay.
attA.start();
expect(a.frames).toEqual([]);
expect(a.ended()).toBe(1);
});
it('open() over a LIVE entry ends started subscribers exactly once and a late done does not touch the new entry (invariant 3)', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
src.push('a');
await flush();
const c = collector();
const att = (await registry.attach(CHAT, false, undefined, c.cb))!;
att.start(); // started subscriber on run-1
// run-2 starts on the same chat while run-1's tee is still reading.
registry.open(CHAT, 'run-2');
expect(c.ended()).toBe(1); // exactly one onEnd from the replace
const newEntry = (registry as any).entries.get(CHAT);
expect(newEntry.runId).toBe('run-2');
expect(newEntry.finished).toBe(false);
// The old tee now completes: its late done must NOT double-end nor delete the
// new entry.
src.push('b');
src.close();
await flush();
expect(c.ended()).toBe(1); // still exactly one
const still = (registry as any).entries.get(CHAT);
expect(still).toBe(newEntry);
expect(still.runId).toBe('run-2');
});
it('bind with a foreign runId is a no-op (invariant 1)', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'WRONG-run', 'assist-x', src.stream);
src.push('a');
await flush();
const entry = (registry as any).entries.get(CHAT);
// Frames were NOT ingested (bind bailed), assistantMessageId untouched.
expect(entry.frames).toEqual([]);
expect(entry.assistantMessageId).toBeUndefined();
});
it('abortEntry with a foreign runId is a no-op (invariant 1)', async () => {
registry.open(CHAT, 'run-1');
registry.abortEntry(CHAT, 'WRONG-run');
const entry = (registry as any).entries.get(CHAT);
expect(entry.finished).toBe(false);
});
it('a throwing onFrame ejects only that subscriber; the ingest loop stays alive', async () => {
registry.open(CHAT, 'run-1');
const src = makePushStream();
registry.bind(CHAT, 'run-1', 'assist-1', src.stream);
const bad = collector();
const badAtt = (await registry.attach(CHAT, false, undefined, {
onFrame: () => {
throw new Error('boom');
},
onEnd: bad.cb.onEnd,
}))!;
badAtt.start();
const good = collector();
const goodAtt = (await registry.attach(CHAT, false, undefined, good.cb))!;
goodAtt.start();
src.push('a'); // bad throws on this frame -> ejected
src.push('b'); // good still receives both
await flush();
const entry = (registry as any).entries.get(CHAT);
expect(entry.subscribers.size).toBe(1); // bad ejected, good remains
expect(good.frames).toEqual(['a', 'b']);
});
});
/**
* Retention + replace timer behavior. Fake timers, and entries are finalized via
* the synchronous abortEntry() path so no stream pump / microtask juggling is
* needed.
*/
describe('AiChatStreamRegistryService retention timers', () => {
const CHAT = 'chat-r';
let registry: AiChatStreamRegistryService;
beforeEach(() => {
jest.useFakeTimers();
registry = new AiChatStreamRegistryService();
jest.spyOn((registry as any).logger, 'warn').mockImplementation(() => {});
});
afterEach(() => {
registry.onModuleDestroy();
jest.useRealTimers();
});
it('a finished entry is removed after the retention window', () => {
registry.open(CHAT, 'run-1');
registry.abortEntry(CHAT, 'run-1'); // finalize -> retention armed
expect((registry as any).entries.get(CHAT)).toBeDefined();
jest.advanceTimersByTime(RUN_STREAM_RETAIN_FINISHED_MS + 1);
expect((registry as any).entries.get(CHAT)).toBeUndefined();
});
it('retention deletes ONLY its own entry (invariant 2)', () => {
registry.open(CHAT, 'run-1');
registry.abortEntry(CHAT, 'run-1'); // arm retention for entry A
// Simulate the race where the key was replaced without clearing A's timer.
const sentinel = { marker: true };
(registry as any).entries.set(CHAT, sentinel);
jest.advanceTimersByTime(RUN_STREAM_RETAIN_FINISHED_MS + 1);
// A's timer saw entries.get(CHAT) !== A, so it did NOT delete the successor.
expect((registry as any).entries.get(CHAT)).toBe(sentinel);
});
it('open() over a retained entry clears its timer and the successor survives', () => {
registry.open(CHAT, 'run-1');
registry.abortEntry(CHAT, 'run-1'); // retained, timer armed
const clearSpy = jest.spyOn(global, 'clearTimeout');
registry.open(CHAT, 'run-2'); // must clear run-1's retain timer
expect(clearSpy).toHaveBeenCalled();
jest.advanceTimersByTime(RUN_STREAM_RETAIN_FINISHED_MS + 1);
const entry = (registry as any).entries.get(CHAT);
expect(entry).toBeDefined();
expect(entry.runId).toBe('run-2');
});
});
@@ -0,0 +1,423 @@
import { ForbiddenException } from '@nestjs/common';
import { AiChatController } from './ai-chat.controller';
import type {
RunStreamAttachment,
RunStreamCallbacks,
} from './ai-chat-stream-registry.service';
import { SUBSCRIBER_MAX_BUFFERED_BYTES } from './ai-chat-stream-registry.service';
import type { User, Workspace } from '@docmost/db/types/entity.types';
/**
* Wiring spec for the #184 phase 1.5 attach endpoint
* (`GET /ai-chat/runs/:chatId/stream`). Owner-gated via assertOwnedChat; the
* registry is mocked so this exercises ONLY the controller's replay/live/204/
* cleanup wiring against a fake raw socket. Constructor order is (aiChatService,
* aiChatRunService, aiChatRepo, aiChatMessageRepo, aiTranscription, pageRepo,
* streamRegistry, environment).
*/
describe('AiChatController attach endpoint (#184 phase 1.5)', () => {
const user = { id: 'u1' } as User;
const workspace = { id: 'ws1' } as Workspace;
function makeRawRes() {
const raw: any = {
writableEnded: false,
writableLength: 0,
destroyed: false,
written: [] as string[],
head: null as any,
write: jest.fn((f: string) => {
raw.written.push(f);
return true;
}),
writeHead: jest.fn((code: number, headers: any) => {
raw.head = { code, headers };
return raw;
}),
flushHeaders: jest.fn(),
end: jest.fn(() => {
raw.writableEnded = true;
}),
destroy: jest.fn(() => {
raw.destroyed = true;
}),
on: jest.fn(),
once: jest.fn(),
};
const res: any = {
raw,
status: jest.fn(() => res),
send: jest.fn(),
hijack: jest.fn(),
};
return { res, raw };
}
function makeReq(destroyed = false) {
const handlers: Record<string, () => void> = {};
const raw: any = {
destroyed,
once: jest.fn((ev: string, fn: () => void) => {
handlers[ev] = fn;
}),
};
return { req: { raw } as any, raw, fireClose: () => handlers['close']?.() };
}
function makeAttachment(
over: Partial<RunStreamAttachment> = {},
): RunStreamAttachment {
return {
replay: [],
finished: false,
start: jest.fn(),
unsubscribe: jest.fn(),
...over,
};
}
function makeController(opts: {
chat?: unknown;
attachment?: RunStreamAttachment | null;
}) {
const aiChatRepo = { findById: jest.fn().mockResolvedValue(opts.chat) };
let capturedCb: RunStreamCallbacks | undefined;
const streamRegistry = {
attach: jest.fn(
(
_chatId: string,
_live: boolean,
_anchor: string | undefined,
cb: RunStreamCallbacks,
) => {
capturedCb = cb;
return Promise.resolve(
opts.attachment === undefined ? makeAttachment() : opts.attachment,
);
},
),
};
const environment = { isAiChatResumableStreamEnabled: () => true };
const controller = new AiChatController(
{} as never, // aiChatService
{} as never, // aiChatRunService
aiChatRepo as never,
{} as never, // aiChatMessageRepo
{} as never, // aiTranscription
{} as never, // pageRepo
streamRegistry as never,
environment as never,
);
return {
controller,
aiChatRepo,
streamRegistry,
getCb: () => capturedCb!,
};
}
const owned = { id: 'c1', creatorId: 'u1' };
it('owner-gates: a foreign chat throws ForbiddenException and never attaches', async () => {
const { controller, streamRegistry } = makeController({
chat: { id: 'c1', creatorId: 'someone-else' },
});
const { res } = makeRawRes();
const { req } = makeReq();
await expect(
controller.attachRunStream(
'c1',
undefined,
undefined,
req,
res,
user,
workspace,
),
).rejects.toBeInstanceOf(ForbiddenException);
expect(streamRegistry.attach).not.toHaveBeenCalled();
});
it('answers 204 when the registry has nothing to resume (no entry / finished / anchor-mismatch)', async () => {
const { controller } = makeController({ chat: owned, attachment: null });
const { res } = makeRawRes();
const { req } = makeReq();
await controller.attachRunStream(
'c1',
undefined,
undefined,
req,
res,
user,
workspace,
);
expect(res.status).toHaveBeenCalledWith(204);
expect(res.send).toHaveBeenCalled();
expect(res.hijack).not.toHaveBeenCalled();
});
it('threads expect=live and anchor through to the registry', async () => {
const { controller, streamRegistry } = makeController({
chat: owned,
attachment: null,
});
const { res } = makeRawRes();
const { req } = makeReq();
await controller.attachRunStream(
'c1',
'live',
'anchor-1',
req,
res,
user,
workspace,
);
expect(streamRegistry.attach).toHaveBeenCalledWith(
'c1',
true,
'anchor-1',
expect.anything(),
);
});
it('passes expect=false when the query is absent', async () => {
const { controller, streamRegistry } = makeController({
chat: owned,
attachment: null,
});
const { res } = makeRawRes();
const { req } = makeReq();
await controller.attachRunStream(
'c1',
undefined,
undefined,
req,
res,
user,
workspace,
);
expect(streamRegistry.attach).toHaveBeenCalledWith(
'c1',
false,
undefined,
expect.anything(),
);
});
it('hijacks, writes headers + replay, registers a close cleanup, then goes live', async () => {
const start = jest.fn();
const attachment = makeAttachment({
replay: ['f1', 'f2'],
finished: false,
start,
});
const { controller } = makeController({ chat: owned, attachment });
const { res, raw } = makeRawRes();
const { req } = makeReq();
await controller.attachRunStream(
'c1',
undefined,
undefined,
req,
res,
user,
workspace,
);
expect(res.hijack).toHaveBeenCalled();
expect(raw.writeHead).toHaveBeenCalledWith(
200,
expect.objectContaining({ 'content-type': 'text/event-stream' }),
);
expect(raw.written).toEqual(['f1', 'f2']); // replay
expect(start).toHaveBeenCalled(); // go live after replay
expect(req.raw.once).toHaveBeenCalledWith('close', expect.any(Function));
});
it('finished replay ends the response immediately without going live', async () => {
const start = jest.fn();
const attachment = makeAttachment({
replay: ['f1'],
finished: true,
start,
});
const { controller } = makeController({ chat: owned, attachment });
const { res, raw } = makeRawRes();
const { req } = makeReq();
await controller.attachRunStream(
'c1',
'live',
'a1',
req,
res,
user,
workspace,
);
expect(raw.written).toEqual(['f1']);
expect(raw.end).toHaveBeenCalled();
expect(start).not.toHaveBeenCalled(); // finished -> returns before start()
});
it('a close during the awaits (req.raw.destroyed) unsubscribes and writes nothing', async () => {
const attachment = makeAttachment({ replay: ['f1'] });
const { controller } = makeController({ chat: owned, attachment });
const { res, raw } = makeRawRes();
const { req } = makeReq(true); // destroyed already at registration time
await controller.attachRunStream(
'c1',
undefined,
undefined,
req,
res,
user,
workspace,
);
expect(attachment.unsubscribe).toHaveBeenCalled();
expect(raw.writeHead).not.toHaveBeenCalled();
expect(raw.written).toEqual([]);
});
it('the registered close handler unsubscribes the attachment', async () => {
const attachment = makeAttachment({ replay: [] });
const { controller } = makeController({ chat: owned, attachment });
const { res } = makeRawRes();
const { req, fireClose } = makeReq();
await controller.attachRunStream(
'c1',
undefined,
undefined,
req,
res,
user,
workspace,
);
expect(attachment.unsubscribe).not.toHaveBeenCalled();
fireClose(); // socket closed
expect(attachment.unsubscribe).toHaveBeenCalled();
});
it('onFrame destroys the socket when the buffered length exceeds the cap', async () => {
const attachment = makeAttachment({ replay: [] });
const { controller, getCb } = makeController({ chat: owned, attachment });
const { res, raw } = makeRawRes();
const { req } = makeReq();
await controller.attachRunStream(
'c1',
undefined,
undefined,
req,
res,
user,
workspace,
);
const cb = getCb();
// Normal live frame writes.
raw.writableLength = 0;
cb.onFrame('live-1');
expect(raw.written).toContain('live-1');
// A stalled socket over the cap is destroyed instead of buffering.
raw.writableLength = SUBSCRIBER_MAX_BUFFERED_BYTES + 1;
raw.write.mockClear();
cb.onFrame('too-much');
expect(raw.destroy).toHaveBeenCalled();
expect(raw.write).not.toHaveBeenCalled();
});
});
/**
* The begin-hook `open()` flag gate (#184 phase 1.5). `open()` lives ONLY in the
* stream() begin-hook, gated on the resumable flag. If it regressed, a flag-off
* turn would create an EMPTY registry entry (never bound, never finished) and a
* later attach would find a non-null paused attachment -> a hung SSE that never
* gets a frame and never ends, instead of a clean 204. These drive stream() only
* far enough to capture the runHooks it hands to the service, then invoke the
* begin-hook and assert whether the registry was opened.
*/
describe('AiChatController begin-hook open() flag gate (#184 phase 1.5)', () => {
const user = { id: 'u1' } as User;
const workspace = {
id: 'ws1',
settings: { ai: { chat: true, autonomousRuns: true } },
} as unknown as Workspace;
function makeController(opts: { resumable: boolean }) {
let capturedArgs: any;
const aiChatService = {
resolveRoleForRequest: jest.fn(async () => null),
getChatModel: jest.fn(async () => ({})),
stream: jest.fn(async (args: any) => {
capturedArgs = args;
}),
};
const aiChatRunService = {
getActiveForChat: jest.fn(async () => undefined),
beginRun: jest.fn(async () => ({
runId: 'run-1',
signal: new AbortController().signal,
})),
};
const streamRegistry = { open: jest.fn(), attach: jest.fn() };
const environment = {
isAiChatResumableStreamEnabled: () => opts.resumable,
};
const controller = new AiChatController(
aiChatService as never,
aiChatRunService as never,
{} as never, // aiChatRepo
{} as never, // aiChatMessageRepo
{} as never, // aiTranscription
{} as never, // pageRepo
streamRegistry as never,
environment as never,
);
return {
controller,
streamRegistry,
aiChatRunService,
getRunHooks: () => capturedArgs?.runHooks,
};
}
function makeReqRes() {
const req: any = {
raw: { sessionId: 'sess-1', once: jest.fn() },
body: {
messages: [
{ id: 'm1', role: 'user', parts: [{ type: 'text', text: 'hi' }] },
],
},
};
const res: any = {
raw: { once: jest.fn(), on: jest.fn(), headersSent: false, writableEnded: false },
hijack: jest.fn(),
};
return { req, res };
}
it('flag OFF: the begin-hook does NOT open a registry entry (no hung empty entry)', async () => {
const { controller, streamRegistry, aiChatRunService, getRunHooks } =
makeController({ resumable: false });
const { req, res } = makeReqRes();
await controller.stream(req, res, user, workspace);
const runHooks = getRunHooks();
expect(runHooks).toBeDefined();
const handle = await runHooks.begin('chat-1');
// The run still begins (the durable-run feature is independent of resume)...
expect(aiChatRunService.beginRun).toHaveBeenCalled();
expect(handle).toEqual({ runId: 'run-1', signal: expect.anything() });
// ...but with the flag off the registry entry is NEVER opened.
expect(streamRegistry.open).not.toHaveBeenCalled();
});
it('flag ON: the begin-hook opens the registry entry with (chatId, runId)', async () => {
const { controller, streamRegistry, getRunHooks } = makeController({
resumable: true,
});
const { req, res } = makeReqRes();
await controller.stream(req, res, user, workspace);
const runHooks = getRunHooks();
await runHooks.begin('chat-1');
expect(streamRegistry.open).toHaveBeenCalledWith('chat-1', 'run-1');
});
});
@@ -4,11 +4,15 @@ import {
ConflictException, ConflictException,
Controller, Controller,
ForbiddenException, ForbiddenException,
Get,
HttpCode, HttpCode,
HttpException, HttpException,
HttpStatus, HttpStatus,
Logger, Logger,
Param,
ParseUUIDPipe,
Post, Post,
Query,
Req, Req,
Res, Res,
ServiceUnavailableException, ServiceUnavailableException,
@@ -54,6 +58,12 @@ import {
} from './dto/ai-chat.dto'; } from './dto/ai-chat.dto';
import { describeProviderError } from '../../integrations/ai/ai-error.util'; import { describeProviderError } from '../../integrations/ai/ai-error.util';
import { buildChatMarkdown } from './chat-markdown.util'; import { buildChatMarkdown } from './chat-markdown.util';
import {
AiChatStreamRegistryService,
SUBSCRIBER_MAX_BUFFERED_BYTES,
} from './ai-chat-stream-registry.service';
import { startSseHeartbeat } from './sse-resilience';
import { EnvironmentService } from '../../integrations/environment/environment.service';
/** /**
* Per-user AI chat API (§6.1). Routes are POST to match this codebase's * Per-user AI chat API (§6.1). Routes are POST to match this codebase's
@@ -72,6 +82,11 @@ export class AiChatController {
private readonly aiChatMessageRepo: AiChatMessageRepo, private readonly aiChatMessageRepo: AiChatMessageRepo,
private readonly aiTranscription: AiTranscriptionService, private readonly aiTranscription: AiTranscriptionService,
private readonly pageRepo: PageRepo, private readonly pageRepo: PageRepo,
// #184 phase 1.5. OPTIONAL so existing positional constructions (controller
// specs) compile unchanged; Nest always injects the real providers in
// production. Only touched on the resumable-stream (flag-on) path.
private readonly streamRegistry?: AiChatStreamRegistryService,
private readonly environment?: EnvironmentService,
) {} ) {}
/** List the requesting user's chats in this workspace (paginated). */ /** List the requesting user's chats in this workspace (paginated). */
@@ -233,6 +248,102 @@ export class AiChatController {
return { stopped }; return { stopped };
} }
/**
* Attach to a chat's live run stream (#184 phase 1.5). A late/reloaded tab
* replays the frames buffered so far and then follows the live tail as a normal
* streamer. Owner-gated via assertOwnedChat (same gate as getRun). When there is
* nothing to resume no entry, a finished run without expect=live, an
* overflowed buffer, or an anchor that pins a DIFFERENT run the endpoint
* answers 204, the ONLY "nothing to resume" signal the AI SDK's reconnect
* accepts (it maps 204 to a silent no-op). With AI_CHAT_RESUMABLE_STREAM off the
* registry is never populated, so attach always 204s.
*
* `expect=live` opts into replaying a finished-but-retained run (safe only when
* the client stripped the streaming tail); `anchor` is the client's assistant
* row id, which must match this run's (invariant 6) or a foreign run's
* transcript would be replayed into the store.
*/
@SkipTransform()
@UseGuards(JwtAuthGuard, UserThrottlerGuard)
@Throttle({ [AI_CHAT_THROTTLER]: { limit: 60, ttl: 60000 } })
@Get('runs/:chatId/stream')
async attachRunStream(
@Param('chatId', new ParseUUIDPipe()) chatId: string,
@Query('expect') expect: string | undefined,
@Query('anchor') anchor: string | undefined,
@Req() req: FastifyRequest,
@Res() res: FastifyReply,
@AuthUser() user: User,
@AuthWorkspace() workspace: Workspace,
): Promise<void> {
await this.assertOwnedChat(chatId, user, workspace); // same gate as getRun
let stopHeartbeat: () => void = () => undefined;
const attachment = await this.streamRegistry?.attach(
chatId,
expect === 'live',
anchor,
{
onFrame: (frame) => {
// Backpressure guard: 2x the replay cap, so the initial replay burst
// alone can never trip it; only a genuinely stalled socket can.
try {
if (res.raw.writableLength > SUBSCRIBER_MAX_BUFFERED_BYTES) {
res.raw.destroy(); // 'close' fires -> unsubscribe below
return;
}
if (!res.raw.writableEnded) res.raw.write(frame);
} catch {
res.raw.destroy();
}
},
onEnd: () => {
stopHeartbeat();
if (!res.raw.writableEnded) res.raw.end();
},
},
);
if (!attachment) {
res.status(204).send(); // the ONLY "nothing to resume" signal the SDK accepts
return;
}
res.hijack();
// Cleanup BEFORE any write (invariant 5): a torn-down socket must not orphan
// a paused subscriber whose pending queue would buffer the whole run.
req.raw.once('close', () => {
attachment.unsubscribe();
stopHeartbeat();
});
// A close emitted DURING the awaits above was missed by the listener — check.
// (Healthy pending GETs have req.raw.destroyed === false, so no false
// positives; returning without end() is fine — the socket is gone.)
if (req.raw.destroyed) {
attachment.unsubscribe();
return;
}
res.raw.on('error', () => undefined);
try {
res.raw.writeHead(200, {
'content-type': 'text/event-stream',
'cache-control': 'no-cache',
'x-vercel-ai-ui-message-stream': 'v1',
'x-accel-buffering': 'no',
// deliberately NO Connection/Keep-Alive (hop-by-hop; Safari/HTTP2)
});
res.raw.flushHeaders?.();
for (const frame of attachment.replay) res.raw.write(frame);
if (attachment.finished) {
res.raw.end();
return;
}
stopHeartbeat = startSseHeartbeat(res.raw, 15_000);
attachment.start(); // drain pending accumulated during replay, go live
} catch {
attachment.unsubscribe();
stopHeartbeat();
res.raw.destroy();
}
}
/** Rename a chat. */ /** Rename a chat. */
@HttpCode(HttpStatus.OK) @HttpCode(HttpStatus.OK)
@Post('rename') @Post('rename')
@@ -344,13 +455,25 @@ export class AiChatController {
// its progress, and settle its terminal status — see AiChatRunService. // its progress, and settle its terminal status — see AiChatRunService.
const runHooks: AiChatRunHooks | undefined = autonomousRuns const runHooks: AiChatRunHooks | undefined = autonomousRuns
? { ? {
begin: (chatId) => begin: async (chatId) => {
this.aiChatRunService.beginRun({ const handle = await this.aiChatRunService.beginRun({
chatId, chatId,
workspaceId: workspace.id, workspaceId: workspace.id,
userId: user.id, userId: user.id,
trigger: 'user', trigger: 'user',
}), });
// #184 phase 1.5: register the run-stream entry at BEGIN (before any
// frame) so a tab that attaches in the begin->seed window finds an
// entry to wait on. Gated on AI_CHAT_RESUMABLE_STREAM: with the flag
// off nothing is registered and attach always 204s.
if (
handle?.runId &&
this.environment?.isAiChatResumableStreamEnabled?.()
) {
this.streamRegistry?.open(chatId, handle.runId);
}
return handle;
},
onAssistantSeeded: (runId, messageId) => onAssistantSeeded: (runId, messageId) =>
this.aiChatRunService.linkAssistantMessage( this.aiChatRunService.linkAssistantMessage(
runId, runId,
@@ -4,6 +4,7 @@ import { TokenModule } from '../auth/token.module';
import { AiChatController } from './ai-chat.controller'; import { AiChatController } from './ai-chat.controller';
import { AiChatService } from './ai-chat.service'; import { AiChatService } from './ai-chat.service';
import { AiChatRunService } from './ai-chat-run.service'; import { AiChatRunService } from './ai-chat-run.service';
import { AiChatStreamRegistryService } from './ai-chat-stream-registry.service';
import { AiTranscriptionService } from './ai-transcription.service'; import { AiTranscriptionService } from './ai-transcription.service';
import { AiChatToolsService } from './tools/ai-chat-tools.service'; import { AiChatToolsService } from './tools/ai-chat-tools.service';
import { EmbeddingModule } from './embedding/embedding.module'; import { EmbeddingModule } from './embedding/embedding.module';
@@ -44,6 +45,7 @@ import { PublicShareChatToolsService } from './tools/public-share-chat-tools.ser
providers: [ providers: [
AiChatService, AiChatService,
AiChatRunService, AiChatRunService,
AiChatStreamRegistryService,
AiTranscriptionService, AiTranscriptionService,
AiChatToolsService, AiChatToolsService,
PublicShareChatService, PublicShareChatService,
@@ -153,6 +153,41 @@ describe('buildSystemPrompt current-page context', () => {
expect(prompt).not.toContain('pageId:'); expect(prompt).not.toContain('pageId:');
}); });
// #388: editor-selection flag. Only a FIXED one-liner is added — the selection
// TEXT (untrusted page content) must never reach the prompt.
const SELECTION_FLAG = 'currently has text SELECTED on this page';
it('adds the selection flag when a selection is present with a page', () => {
const prompt = buildSystemPrompt({
workspace,
openedPage: {
id: 'pg-123',
title: 'Doc',
selection: { text: 'SECRET-SELECTED-TEXT', blockIds: ['b1'] },
},
});
expect(prompt).toContain(SELECTION_FLAG);
// The selection TEXT itself is NEVER in the prompt.
expect(prompt).not.toContain('SECRET-SELECTED-TEXT');
expect(prompt).not.toContain('b1');
});
it('omits the selection flag when there is no selection', () => {
const prompt = buildSystemPrompt({
workspace,
openedPage: { id: 'pg-123', title: 'Doc' },
});
expect(prompt).not.toContain(SELECTION_FLAG);
});
it('omits the selection flag when selection is null', () => {
const prompt = buildSystemPrompt({
workspace,
openedPage: { id: 'pg-123', title: 'Doc', selection: null },
});
expect(prompt).not.toContain(SELECTION_FLAG);
});
it('escapes a malicious opened-page title so it cannot inject tags (F1)', () => { it('escapes a malicious opened-page title so it cannot inject tags (F1)', () => {
const prompt = buildSystemPrompt({ const prompt = buildSystemPrompt({
workspace, workspace,
+14 -1
View File
@@ -156,8 +156,13 @@ export interface BuildSystemPromptInput {
* has an id, a CONTEXT line is added so the agent can resolve "this page" / * has an id, a CONTEXT line is added so the agent can resolve "this page" /
* "the current page" to that pageId. The page is NOT fetched here the agent * "the current page" to that pageId. The page is NOT fetched here the agent
* uses its CASL-enforced read/write page tools with the id when needed. * uses its CASL-enforced read/write page tools with the id when needed.
*
* `selection` (#388) is present only when the user has a non-empty editor
* selection; the prompt adds ONLY a fixed one-line flag from it the
* selection TEXT is untrusted page content and stays out of the prompt (it is
* surfaced solely via the getCurrentPage tool result).
*/ */
openedPage?: { id?: string; title?: string } | null; openedPage?: { id?: string; title?: string; selection?: object | null } | null;
/** /**
* Admin-authored, per-EXTERNAL-MCP-server guidance ("how/when to use this * Admin-authored, per-EXTERNAL-MCP-server guidance ("how/when to use this
* server's tools"), built by `McpClientsService.toolsFor` for servers that * server's tools"), built by `McpClientsService.toolsFor` for servers that
@@ -309,6 +314,14 @@ export function buildSystemPrompt({
? escapeAttr(openedPage.title) ? escapeAttr(openedPage.title)
: 'Untitled'; : 'Untitled';
context += `\nThe user is currently viewing the page "${title}" (pageId: ${pageId.trim()}). When they refer to "this page", "the current page", or similar, operate on that pageId — use the read/write page tools with it.`; context += `\nThe user is currently viewing the page "${title}" (pageId: ${pageId.trim()}). When they refer to "this page", "the current page", or similar, operate on that pageId — use the read/write page tools with it.`;
// Editor-selection flag (#388). A FIXED one-liner only — the selection TEXT
// is untrusted collaborative-page content and must never enter the prompt; it
// is surfaced solely through the getCurrentPage tool result (SAFETY_FRAMEWORK
// treats a tool result as data). Nested under the page block so it is added
// only alongside a resolved page (a selection cannot outlive its page).
if (openedPage?.selection) {
context += `\nThe user currently has text SELECTED on this page — call getCurrentPage to see the selection. When they say "this", "here", "the selected text" or similar, they mean that selection.`;
}
} }
// Interrupt-resume marker (#198). Added to the context section (inside the // Interrupt-resume marker (#198). Added to the context section (inside the
@@ -0,0 +1,121 @@
import { Logger } from '@nestjs/common';
// Mock the AI SDK: the turn we drive is STOPPED during the pre-streamText setup
// phase, so no provider call must ever be made. convertToModelMessages is reached
// (before toolsFor) so it is stubbed to an empty transcript.
jest.mock('ai', () => ({
streamText: jest.fn(),
generateText: jest.fn(),
convertToModelMessages: jest.fn(async () => []),
stepCountIs: jest.fn(() => () => false),
}));
import { streamText } from 'ai';
import { AiChatService } from './ai-chat.service';
/**
* D2 an explicit Stop DURING the external-MCP toolset build (the pre-streamText
* setup phase) must:
* (a) unwedge the turn (stream() rejects instead of hanging at step 0), and
* (b) finalize the run as 'aborted' via the outer catch's onSettled never leak
* the run row as 'running' (which would 409 every later turn in this chat).
*
* The setup phase does NOT yet observe streamText's terminal callbacks, so before
* the fix a hung `toolsFor` ignored the run's abort signal and never finalized.
* `raceAgainstAbortAndTimeout(toolsFor, effectiveSignal, ...)` now rejects the
* moment the run's signal aborts; the catch re-throws (signal aborted), and the
* outer catch settles the run 'aborted'.
*/
describe('AiChatService.stream — abort during external-MCP setup finalizes the run (D2)', () => {
const streamTextMock = streamText as unknown as jest.Mock;
function makeService(mcpClients: { toolsFor: jest.Mock }) {
const aiChatRepo = {
findById: jest.fn(async () => ({ id: 'chat-1', workspaceId: 'ws-1' })),
insert: jest.fn(),
};
const aiChatMessageRepo = {
insert: jest.fn(async () => ({ id: 'msg-1' })),
findAllByChat: jest.fn(async () => []),
update: jest.fn(async () => ({ id: 'msg-1' })),
};
const aiSettings = { resolve: jest.fn(async () => ({})) };
const tools = { forUser: jest.fn(async () => ({})) };
const svc = new AiChatService(
{} as never, // ai
aiChatRepo as never,
aiChatMessageRepo as never,
{} as never, // aiChatPageSnapshotRepo
aiSettings as never,
tools as never,
mcpClients as never,
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo (openPage undefined -> never touched)
{} as never, // pageAccess
{ isAiChatDeferredToolsEnabled: () => false } as never, // environment
);
return { svc, tools };
}
const body = {
chatId: 'chat-1',
messages: [
{ id: 'm1', role: 'user', parts: [{ type: 'text', text: 'hi' }] },
],
};
beforeEach(() => {
streamTextMock.mockReset();
jest.spyOn(Logger.prototype, 'log').mockImplementation(() => undefined as never);
jest.spyOn(Logger.prototype, 'warn').mockImplementation(() => undefined as never);
jest.spyOn(Logger.prototype, 'error').mockImplementation(() => undefined as never);
});
afterEach(() => jest.restoreAllMocks());
it('stops the hung toolset build, rejects, and settles the run "aborted" — never reaching streamText', async () => {
const runController = new AbortController();
// The build hangs (never resolves); the run is STOPPED mid-build. Aborting on a
// macrotask exercises the abort-listener path (a real user Stop during setup).
const toolsFor = jest.fn(() => {
setTimeout(() => runController.abort(new Error('user stop')), 0);
return new Promise(() => {}); // never settles — models a hung MCP build
});
const { svc } = makeService({ toolsFor });
const onSettled = jest.fn();
const begin = jest.fn(async () => ({
runId: 'run-1',
signal: runController.signal,
}));
const promise = svc.stream({
user: { id: 'user-1' } as never,
workspace: { id: 'ws-1' } as never,
sessionId: 'sess-1',
body: body as never,
res: { raw: {} } as never,
signal: new AbortController().signal, // socket signal (distinct from the run)
model: {} as never,
role: null,
runHooks: {
begin,
onAssistantSeeded: jest.fn(),
onStep: jest.fn(),
onSettled,
} as never,
});
// (a) The turn is UNWEDGED: it rejects (with the stop reason) instead of hanging.
await expect(promise).rejects.toThrow('user stop');
// (b) The run is finalized as 'aborted' with NO error message (a Stop, not a
// failure) — so the run row never leaks 'running'.
expect(onSettled).toHaveBeenCalledTimes(1);
expect(onSettled).toHaveBeenCalledWith('run-1', 'aborted', undefined);
// The build was reached, but the provider call was NEVER made (stopped at setup).
expect(toolsFor).toHaveBeenCalledTimes(1);
expect(streamTextMock).not.toHaveBeenCalled();
});
});
@@ -1,4 +1,13 @@
import { ForbiddenException } from '@nestjs/common'; import { ForbiddenException, Logger } from '@nestjs/common';
// Mock ONLY streamText so a driven stream() call can capture the pipe-options
// object (consumeSseStream / generateMessageId). Everything else in the AI SDK
// stays REAL (requireActual), so the pure-helper suites in this file are
// unaffected — none of them call stream()/streamText.
jest.mock('ai', () => ({
...jest.requireActual('ai'),
streamText: jest.fn(),
}));
import { streamText } from 'ai';
import { import {
AiChatService, AiChatService,
compactToolOutput, compactToolOutput,
@@ -689,6 +698,7 @@ describe('AiChatService.resolveOpenPageContext (#159 current-page validation)',
id: string; id: string;
title: string; title: string;
updatedAt: Date; updatedAt: Date;
selection: unknown;
} | null>; } | null>;
it('returns null when no page is open (no id)', async () => { it('returns null when no page is open (no id)', async () => {
@@ -734,8 +744,14 @@ describe('AiChatService.resolveOpenPageContext (#159 current-page validation)',
}); });
// The client claims it is on "Page A" but the id points at page B. // The client claims it is on "Page A" but the id points at page B.
const result = await call(svc, { id: 'p-1', title: 'Page A' }); const result = await call(svc, { id: 'p-1', title: 'Page A' });
// updatedAt (#274 page-change fast path) is carried through from the DB row. // updatedAt (#274 page-change fast path) is carried through from the DB row;
expect(result).toEqual({ id: 'p-1', title: 'Real Title B', updatedAt }); // selection is null when the client sent none (#388).
expect(result).toEqual({
id: 'p-1',
title: 'Real Title B',
updatedAt,
selection: null,
});
}); });
it('coerces a null DB title to an empty string', async () => { it('coerces a null DB title to an empty string', async () => {
@@ -748,8 +764,55 @@ describe('AiChatService.resolveOpenPageContext (#159 current-page validation)',
id: 'p-1', id: 'p-1',
title: '', title: '',
updatedAt, updatedAt,
selection: null,
}); });
}); });
// #388: the selection rides ONLY on a successful page resolve, and is
// sanitized on the way through.
it('attaches the SANITIZED selection on a successful resolve', async () => {
const updatedAt = new Date('2026-07-02T10:00:00Z');
const svc = makeService({
page: { id: 'p-1', workspaceId: 'ws-1', title: 'Doc', updatedAt },
canView: true,
});
const result = await call(svc, {
id: 'p-1',
selection: {
text: 'fix this',
blockIds: ['b1', 123, 'y'.repeat(65)], // garbage stripped by sanitize
before: 'please ',
},
});
expect(result).toEqual({
id: 'p-1',
title: 'Doc',
updatedAt,
selection: { text: 'fix this', blockIds: ['b1'], before: 'please ' },
});
});
it('drops a blank/garbage selection to null on a successful resolve', async () => {
const updatedAt = new Date('2026-07-02T10:00:00Z');
const svc = makeService({
page: { id: 'p-1', workspaceId: 'ws-1', title: 'Doc', updatedAt },
canView: true,
});
expect(
await call(svc, { id: 'p-1', selection: { text: ' ' } }),
).toEqual({ id: 'p-1', title: 'Doc', updatedAt, selection: null });
});
it('selection does NOT survive a foreign/inaccessible page (dies with the page)', async () => {
// Forbidden page => the WHOLE context is null, so the selection is gone too.
const svc = makeService({
page: { id: 'p-1', workspaceId: 'ws-1', title: 'Restricted' },
canView: false,
});
expect(
await call(svc, { id: 'p-1', selection: { text: 'secret sel' } }),
).toBeNull();
});
}); });
/** /**
@@ -1059,3 +1122,181 @@ describe('isInterruptResume', () => {
expect(isInterruptResume(withPrev(null), true)).toBe(false); expect(isInterruptResume(withPrev(null), true)).toBe(false);
}); });
}); });
/**
* #184 phase 1.5 the run-wrapped pipe options (unit). Drives stream() to the
* pipe call with streamText mocked, capturing the options object, and asserts:
* - flag OFF while a runId IS present -> the LEGACY option shape (no
* consumeSseStream, no generateMessageId), and the registry is never touched.
* This is the exact dormancy guarantee this PR rests on.
* - flag ON + runId -> consumeSseStream tees into the registry and
* generateMessageId returns the seeded assistant DB row id.
* - flag ON but no runHooks (runId undefined) -> legacy (the runId gate).
* - flag ON + runId -> the outer catch releases the entry via abortEntry.
*/
describe('AiChatService.stream — resumable pipe options (#184 phase 1.5)', () => {
const streamTextMock = streamText as unknown as jest.Mock;
let pipeMock: jest.Mock;
beforeEach(() => {
streamTextMock.mockReset();
pipeMock = jest.fn();
streamTextMock.mockReturnValue({
consumeStream: jest.fn(),
pipeUIMessageStreamToResponse: pipeMock,
});
// Silence the service's diagnostic logging for a clean test run.
jest.spyOn(Logger.prototype, 'log').mockImplementation(() => undefined as never);
jest
.spyOn(Logger.prototype, 'error')
.mockImplementation(() => undefined as never);
jest
.spyOn(Logger.prototype, 'warn')
.mockImplementation(() => undefined as never);
});
afterEach(() => jest.restoreAllMocks());
// A raw-response stub sufficient for the post-streamText wiring.
function makeRes() {
return {
raw: {
writeHead: jest.fn(),
write: jest.fn(),
once: jest.fn(),
on: jest.fn(),
flushHeaders: jest.fn(),
writableEnded: false,
destroyed: false,
},
};
}
// Wire only the deps reached on the way to the pipe call, plus a spy registry.
function makeService(opts: { resumable: boolean }) {
const aiChatRepo = {
findById: jest.fn(async () => ({ id: 'chat-1', workspaceId: 'ws-1' })),
insert: jest.fn(),
};
const aiChatMessageRepo = {
// Both the user insert and the assistant seed return the same row id.
insert: jest.fn(async () => ({ id: 'msg-1' })),
findAllByChat: jest.fn(async () => []),
update: jest.fn(async () => ({ id: 'msg-1' })),
};
const aiSettings = { resolve: jest.fn(async () => ({})) };
const tools = { forUser: jest.fn(async () => ({})) };
const mcpClients = {
toolsFor: jest.fn(async () => ({
tools: {},
clients: [],
outcomes: [],
instructions: [],
})),
};
const streamRegistry = {
open: jest.fn(),
bind: jest.fn(),
abortEntry: jest.fn(),
};
const svc = new AiChatService(
{} as never, // ai (model is injected)
aiChatRepo as never,
aiChatMessageRepo as never,
{} as never, // aiChatPageSnapshotRepo
aiSettings as never,
tools as never,
mcpClients as never,
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo (openPage undefined -> never touched)
{} as never, // pageAccess
{
isAiChatDeferredToolsEnabled: () => false,
isAiChatResumableStreamEnabled: () => opts.resumable,
} as never,
streamRegistry as never,
);
return { svc, streamRegistry };
}
const body = {
chatId: 'chat-1',
messages: [
{ id: 'm1', role: 'user', parts: [{ type: 'text', text: 'hi' }] },
],
};
const makeRunHooks = () => ({
begin: jest.fn(async () => ({
runId: 'run-1',
signal: new AbortController().signal,
})),
onAssistantSeeded: jest.fn(),
onStep: jest.fn(),
onSettled: jest.fn(),
});
async function drive(svc: AiChatService, hooks: unknown): Promise<void> {
await svc.stream({
user: { id: 'u1' } as never,
workspace: { id: 'ws-1' } as never,
sessionId: 's1',
body: body as never,
res: makeRes() as never,
signal: new AbortController().signal,
model: {} as never,
role: null,
runHooks: hooks as never,
});
}
it('flag OFF + runId present: LEGACY option shape (no consumeSseStream / generateMessageId); registry untouched', async () => {
const { svc, streamRegistry } = makeService({ resumable: false });
await drive(svc, makeRunHooks());
expect(pipeMock).toHaveBeenCalledTimes(1);
const options = pipeMock.mock.calls[0][1];
// The dormancy guarantee: a live run with the flag off tees NOTHING and does
// not stamp a message id — byte-for-byte the pre-1.5 wire.
expect(options.consumeSseStream).toBeUndefined();
expect(options.generateMessageId).toBeUndefined();
expect(streamRegistry.bind).not.toHaveBeenCalled();
expect(streamRegistry.abortEntry).not.toHaveBeenCalled();
});
it('flag ON + runId: consumeSseStream tees into the registry; generateMessageId returns the seeded row id', async () => {
const { svc, streamRegistry } = makeService({ resumable: true });
await drive(svc, makeRunHooks());
const options = pipeMock.mock.calls[0][1];
expect(typeof options.consumeSseStream).toBe('function');
expect(typeof options.generateMessageId).toBe('function');
// generateMessageId stamps the seeded assistant DB row id.
expect(options.generateMessageId()).toBe('msg-1');
// consumeSseStream binds the tee: (chatId, runId, assistantId, stream).
const fakeStream = {} as ReadableStream<string>;
options.consumeSseStream({ stream: fakeStream });
expect(streamRegistry.bind).toHaveBeenCalledWith(
'chat-1',
'run-1',
'msg-1',
fakeStream,
);
});
it('flag ON but NO runHooks (runId undefined): pipe options stay legacy (the runId gate)', async () => {
const { svc, streamRegistry } = makeService({ resumable: true });
await drive(svc, undefined);
const options = pipeMock.mock.calls[0][1];
expect(options.consumeSseStream).toBeUndefined();
expect(options.generateMessageId).toBeUndefined();
expect(streamRegistry.bind).not.toHaveBeenCalled();
});
it('flag ON + runId: the outer catch calls abortEntry when the stream throws', async () => {
const { svc, streamRegistry } = makeService({ resumable: true });
streamTextMock.mockImplementation(() => {
throw new Error('boom');
});
await expect(drive(svc, makeRunHooks())).rejects.toThrow('boom');
expect(streamRegistry.abortEntry).toHaveBeenCalledWith('chat-1', 'run-1');
});
});
+197 -11
View File
@@ -32,6 +32,7 @@ import {
import { AiChatToolsService } from './tools/ai-chat-tools.service'; import { AiChatToolsService } from './tools/ai-chat-tools.service';
import { McpClientsService } from './external-mcp/mcp-clients.service'; import { McpClientsService } from './external-mcp/mcp-clients.service';
import { EnvironmentService } from '../../integrations/environment/environment.service'; import { EnvironmentService } from '../../integrations/environment/environment.service';
import { AiChatStreamRegistryService } from './ai-chat-stream-registry.service';
import { buildSystemPrompt } from './ai-chat.prompt'; import { buildSystemPrompt } from './ai-chat.prompt';
import { import {
CORE_TOOL_KEYS, CORE_TOOL_KEYS,
@@ -42,6 +43,10 @@ import {
} from './tools/tool-tiers'; } from './tools/tool-tiers';
import { RunAlreadyActiveError } from './ai-chat-run.service'; import { RunAlreadyActiveError } from './ai-chat-run.service';
import { computePageChange } from './page-change/page-change.util'; import { computePageChange } from './page-change/page-change.util';
import {
sanitizeSelection,
type SelectionContext,
} from './tools/current-page.util';
import { roleModelOverride } from './roles/role-model-config'; import { roleModelOverride } from './roles/role-model-config';
import { import {
startSseHeartbeat, startSseHeartbeat,
@@ -53,6 +58,15 @@ import {
// multi-search research questions are not cut off mid-investigation. // multi-search research questions are not cut off mid-investigation.
const MAX_AGENT_STEPS = 20; const MAX_AGENT_STEPS = 20;
// Wall-clock ceiling for building the external MCP toolset during the per-turn
// setup phase (before streamText owns the lifecycle). Defense-in-depth ABOVE the
// per-server connect bound in mcp-clients.service (CONNECT_TIMEOUT_MS): even if
// that per-server timeout regressed, this outer deadline — together with the run's
// abort signal — guarantees the setup phase can never wedge a turn at step 0 (the
// production hang) and the run always finalizes. Generous: the per-server bound
// should fire first in practice; this only backstops a total build stall.
const MCP_TOOLSET_BUILD_DEADLINE_MS = 60_000;
// System-prompt addendum injected ONLY on the final step (see prepareAgentStep). // System-prompt addendum injected ONLY on the final step (see prepareAgentStep).
// It forbids further tool calls and tells the model to synthesize the best // It forbids further tool calls and tells the model to synthesize the best
// answer it can from what it already gathered, so a tool-heavy turn never ends // answer it can from what it already gathered, so a tool-heavy turn never ends
@@ -171,6 +185,78 @@ export function sameInstant(
return ta === tb; return ta === tb;
} }
/**
* Race `work` against an abort signal AND a wall-clock deadline, so a hung
* external-MCP toolset build during the pre-streamText setup phase can NEITHER
* wedge the turn NOR make it un-stoppable, and the run always finalizes. It
* - resolves with `work`'s value when it settles first;
* - REJECTS EARLY if `signal` aborts (with `signal.reason` when that is an Error,
* else a generic `Error('aborted')`) so an explicit Stop is honored mid-setup;
* - REJECTS EARLY if `deadlineMs` elapses (defense-in-depth backstop);
* - invokes `onLateResolve(value)` when `work` settles AFTER the race was already
* lost, so the caller can release any resources that abandoned value owns
* (e.g. close leased MCP clients that would otherwise leak their sockets).
*
* A rejection handler is attached to `work` so a late rejection is never an
* unhandledRejection; the timer is unref'd and cleared once the race settles.
*/
export function raceAgainstAbortAndTimeout<T>(
work: Promise<T>,
signal: AbortSignal,
deadlineMs: number,
onLateResolve?: (value: T) => void,
): Promise<T> {
return new Promise<T>((resolve, reject) => {
let settled = false;
const cleanup = () => {
clearTimeout(timer);
signal.removeEventListener('abort', onAbort);
};
const onAbort = () => {
if (settled) return;
settled = true;
cleanup();
reject(
signal.reason instanceof Error ? signal.reason : new Error('aborted'),
);
};
const timer = setTimeout(() => {
if (settled) return;
settled = true;
cleanup();
reject(new Error(`setup timed out after ${deadlineMs}ms`));
}, deadlineMs);
// Do not keep the process alive just for this setup-deadline timer.
timer.unref?.();
if (signal.aborted) {
onAbort();
} else {
signal.addEventListener('abort', onAbort, { once: true });
}
work.then(
(value) => {
if (settled) {
// The race was already lost (abort/deadline): hand the abandoned value to
// the caller so it can release the resources that value owns.
onLateResolve?.(value);
return;
}
settled = true;
cleanup();
resolve(value);
},
(err: unknown) => {
// A late rejection after the race is already handled — swallow so it is
// never an unhandledRejection.
if (settled) return;
settled = true;
cleanup();
reject(err instanceof Error ? err : new Error(String(err)));
},
);
});
}
/** /**
* Payload accepted from the client `useChat` POST body. We do NOT bind a strict * Payload accepted from the client `useChat` POST body. We do NOT bind a strict
* DTO (the global ValidationPipe whitelist would strip the useChat-specific * DTO (the global ValidationPipe whitelist would strip the useChat-specific
@@ -188,7 +274,12 @@ export interface AiChatStreamBody {
// page" refers to; the page itself is never fetched server-side here. The id // page" refers to; the page itself is never fetched server-side here. The id
// is attacker-controllable but harmless: the agent reads/writes via its // is attacker-controllable but harmless: the agent reads/writes via its
// CASL-enforced page tools, which 403 on a page the user cannot access. // CASL-enforced page tools, which 403 on a page the user cannot access.
openPage?: { id?: string; title?: string } | null; //
// `selection` is the user's editor selection snapshotted client-side at send
// time (#388). It is CLIENT-controlled and UNTRUSTED — a loose `unknown` here
// (the body is parsed off req.body without a DTO) that is type-checked and
// capped by `sanitizeSelection` before it is ever surfaced to the model.
openPage?: { id?: string; title?: string; selection?: unknown } | null;
// Set by the client "send now" action (#198): this turn immediately follows a // Set by the client "send now" action (#198): this turn immediately follows a
// user interruption of the previous turn. A hint only — the server re-confirms // user interruption of the previous turn. A hint only — the server re-confirms
// it against persisted history (`isInterruptResume`) before injecting the // it against persisted history (`isInterruptResume`) before injecting the
@@ -276,6 +367,10 @@ export class AiChatService implements OnModuleInit {
// Reads the AI_CHAT_DEFERRED_TOOLS toggle (#332). Injected last so existing // Reads the AI_CHAT_DEFERRED_TOOLS toggle (#332). Injected last so existing
// positional constructor callers (tests) only append one stub. // positional constructor callers (tests) only append one stub.
private readonly environment: EnvironmentService, private readonly environment: EnvironmentService,
// #184 phase 1.5 run-stream registry. OPTIONAL so existing positional
// constructions (int-specs) compile unchanged; Nest always injects the real
// provider in production. Only ever touched on the run-wrapped + flag-on path.
private readonly streamRegistry?: AiChatStreamRegistryService,
) {} ) {}
/** /**
@@ -360,10 +455,18 @@ export class AiChatService implements OnModuleInit {
* page, or any non-Forbidden access-check fault, returns null. * page, or any non-Forbidden access-check fault, returns null.
*/ */
private async resolveOpenPageContext( private async resolveOpenPageContext(
openPage: { id?: string; title?: string } | null | undefined, openPage:
| { id?: string; title?: string; selection?: unknown }
| null
| undefined,
workspace: Workspace, workspace: Workspace,
user: User, user: User,
): Promise<{ id: string; title: string; updatedAt: Date } | null> { ): Promise<{
id: string;
title: string;
updatedAt: Date;
selection: SelectionContext | null;
} | null> {
const candidatePageId = openPage?.id; const candidatePageId = openPage?.id;
if (!candidatePageId) return null; if (!candidatePageId) return null;
const page = await this.pageRepo.findById(candidatePageId); const page = await this.pageRepo.findById(candidatePageId);
@@ -385,7 +488,18 @@ export class AiChatService implements OnModuleInit {
// updatedAt is the page's last-modified instant, used by the #274 per-turn // updatedAt is the page's last-modified instant, used by the #274 per-turn
// page-change detection as a cheap fast path (unchanged instant => skip the // page-change detection as a cheap fast path (unchanged instant => skip the
// render + diff). The system-prompt / tool consumers ignore the extra field. // render + diff). The system-prompt / tool consumers ignore the extra field.
return { id: page.id, title: page.title ?? '', updatedAt: page.updatedAt }; //
// The sanitized editor selection (#388) is attached ONLY here, on a
// successful page resolve: the fail-closed branches above return null for the
// WHOLE context, so a selection can never outlive a foreign/missing/deleted
// page (decision 3). Downstream consumers that don't care (detectPageChange,
// snapshotOpenPage) ignore the extra field, same as updatedAt.
return {
id: page.id,
title: page.title ?? '',
updatedAt: page.updatedAt,
selection: sanitizeSelection(openPage?.selection),
};
} }
/** /**
@@ -671,10 +785,40 @@ export class AiChatService implements OnModuleInit {
instructions: [], instructions: [],
}; };
try { try {
external = await this.mcpClients.toolsFor(workspace.id); // Bound the external-MCP toolset build by BOTH the run's abort signal and
// a generous wall-clock deadline. This is the pre-streamText setup phase,
// which streamText's terminal callbacks do NOT yet govern — so without this
// a hung build would hang the turn at step 0 forever (the production hang),
// unobservant of an explicit Stop. The deadline is defense-in-depth ABOVE
// the per-server connect bound in mcp-clients.service. On a LATE resolve
// (the race was already lost) close the abandoned toolset's leased clients
// so their transports are not leaked.
external = await raceAgainstAbortAndTimeout(
this.mcpClients.toolsFor(workspace.id),
effectiveSignal,
MCP_TOOLSET_BUILD_DEADLINE_MS,
(late) => {
void Promise.all(
late.clients.map((c) => c.close().catch(() => undefined)),
);
},
);
} catch (err) { } catch (err) {
// Building the external toolset must never break the turn; proceed with // An explicit Stop reached the RUN's signal DURING setup: re-throw so the
// Docmost-only tools. Never log URLs/headers — short message only. // outer catch finalizes the run as aborted — never swallow a Stop. Gated on
// `runId`: the re-throw exists ONLY to finalize the run, which exists only
// in autonomous mode. On the legacy path (no runId) `effectiveSignal` is the
// SOCKET signal (it aborts on a client disconnect); re-throwing there would
// change prior behavior and make the controller write JSON to an already-
// closed socket (it only attaches res.raw.on('error') in autonomous mode).
// So legacy keeps its prior behavior — warn + proceed, and streamText then
// observes the aborted socket signal.
if (runId && effectiveSignal.aborted) {
throw err;
}
// Otherwise a down/slow server (build timeout or other fault) must never
// break the turn: proceed with Docmost-only tools. Never log URLs/headers —
// short message only.
this.logger.warn( this.logger.warn(
`External MCP toolset unavailable: ${ `External MCP toolset unavailable: ${
err instanceof Error ? err.message : 'unknown error' err instanceof Error ? err.message : 'unknown error'
@@ -1174,6 +1318,35 @@ export class AiChatService implements OnModuleInit {
// as the cumulative authoritative usage so the client never jumps DOWN. // as the cumulative authoritative usage so the client never jumps DOWN.
let cumulativeStepUsage: ChatStreamUsage | undefined; let cumulativeStepUsage: ChatStreamUsage | undefined;
result.pipeUIMessageStreamToResponse(res.raw, { result.pipeUIMessageStreamToResponse(res.raw, {
// #184 phase 1.5: run-wrapped mode only — the legacy path (flag off) stays
// byte-for-byte identical, including the absence of start.messageId. Both
// fields are gated on `runId` (present only for a durable run) AND the
// AI_CHAT_RESUMABLE_STREAM flag; the seed `assistantId` is unconditional,
// so gating on `assistantId` alone would change the legacy wire.
...(runId && this.environment?.isAiChatResumableStreamEnabled?.()
? {
// Tee the SSE frames into the run-stream registry so late tabs can
// attach (replay + live tail).
consumeSseStream: ({
stream,
}: {
stream: ReadableStream<string>;
}) =>
this.streamRegistry?.bind(
chatId,
runId!,
assistantId,
stream,
),
// Stamp the persisted assistant row's DB id onto the streamed
// message so every tab renders the SAME id as the DB row (id-based
// reconciliation). Seeding is best-effort: when it failed, let the
// client generate the id.
...(assistantId
? { generateMessageId: () => assistantId }
: {}),
}
: {}),
headers: { 'X-Accel-Buffering': 'no' }, headers: { 'X-Accel-Buffering': 'no' },
// Surface the authoritative chatId on the streamed assistant UI message so // Surface the authoritative chatId on the streamed assistant UI message so
// the client adopts the REAL id of the row we created, instead of guessing // the client adopts the REAL id of the row we created, instead of guessing
@@ -1239,12 +1412,25 @@ export class AiChatService implements OnModuleInit {
// finalizeRun (onSettled) is idempotent — a settle here and a settle from a // finalizeRun (onSettled) is idempotent — a settle here and a settle from a
// streamText callback collapse to a single terminal write. // streamText callback collapse to a single terminal write.
if (runId) { if (runId) {
// #184 phase 1.5: a failure here means the tee `done` will never arrive,
// so release the registry entry's subscribers explicitly — otherwise an
// attached tab hangs forever. Same flag gate as the tee wiring above.
if (this.environment?.isAiChatResumableStreamEnabled?.()) {
this.streamRegistry?.abortEntry(chatId, runId);
}
// Distinguish an explicit Stop (the run's signal aborted during setup) from
// a real failure, so the run settles with the correct terminal status
// instead of always 'error'. onSettled/finalizeRun is idempotent, so this
// is safe even if a streamText callback also settles the run.
const settleStatus = effectiveSignal.aborted ? 'aborted' : 'error';
await runHooks?.onSettled?.( await runHooks?.onSettled?.(
runId, runId,
'error', settleStatus,
err instanceof Error settleStatus === 'aborted'
? err.message ? undefined
: 'Agent run failed before streaming started', : err instanceof Error
? err.message
: 'Agent run failed before streaming started',
); );
} }
throw err; throw err;
@@ -0,0 +1,237 @@
import { McpClientsService } from './mcp-clients.service';
/**
* D1 a HUNG MCP handshake must not POISON the per-workspace build cache.
*
* THE BUG (production hang): `createMCPClient` (inside the private `connect`) is
* NOT bounded by a timeout and like @ai-sdk/mcp's tool calls its promise does
* NOT settle on abort. A transient network blip mid-handshake made connect hang
* FOREVER. Because getOrBuildEntry caches the build PROMISE, that never-settling
* connect wedged EVERY later turn for the workspace (each awaited the same pending
* build) step_count stuck at 0, run row leaking 'running', chat 409ing forever.
*
* THE FIX: `connectWithTimeout` races `connect` against a SETTLING timeout
* (CONNECT_TIMEOUT_MS). On timeout it REJECTS, so buildEntry catches it, records
* the server `ok:false`, and the build COMPLETES with that server skipped the
* cache is never poisoned and a subsequent `toolsFor` returns instead of hanging.
*
* REACHABILITY NOTE: the smallest network-free path that exercises the fix is to
* spy on the private `connect` (the same harness the namespacing spec uses)
* `connectWithTimeout` wraps exactly that call, so a never-resolving `connect`
* models a never-settling `createMCPClient` precisely, without DNS/sockets.
*
* Fake timers prove the timeout fires WITHOUT real waiting.
*/
// Mirrors the private CONNECT_TIMEOUT_MS constant in mcp-clients.service.ts.
const CONNECT_TIMEOUT_MS = 5000;
interface FakeServer {
id: string;
name: string;
transport: string;
url: string;
headersEnc: string | null;
toolAllowlist: string[] | null;
instructions?: string | null;
}
function server(
over: Partial<FakeServer> & { id: string; name: string },
): FakeServer {
return {
transport: 'http',
url: 'https://example.com/mcp',
headersEnc: null,
toolAllowlist: null,
...over,
};
}
function buildService(servers: FakeServer[]) {
const repoStub = { listEnabled: jest.fn().mockResolvedValue(servers) };
const service = new McpClientsService(repoStub as never, {} as never);
// Silence the expected "server unavailable" warning.
jest
.spyOn(
(service as unknown as { logger: { warn: (...a: unknown[]) => void } })
.logger,
'warn',
)
.mockImplementation(() => undefined);
return service;
}
// Spy on the private `connect` with a per-server implementation.
function stubConnect(
service: McpClientsService,
impl: (s: FakeServer) => Promise<unknown>,
) {
return jest
.spyOn(
service as unknown as { connect: (s: FakeServer) => Promise<unknown> },
'connect',
)
.mockImplementation(impl);
}
describe('McpClientsService.connectWithTimeout — hung connect does not poison the cache (D1)', () => {
beforeEach(() => jest.useFakeTimers());
afterEach(() => {
jest.clearAllTimers();
jest.useRealTimers();
jest.restoreAllMocks();
});
it('buildEntry completes (server recorded ok:false) when connect never settles, and toolsFor does not hang', async () => {
const svc = buildService([server({ id: 'id-hung', name: 'hung' })]);
// connect NEVER settles — models a wedged createMCPClient handshake.
stubConnect(svc, () => new Promise<never>(() => {}));
const toolsetPromise = svc.toolsFor('ws-1');
// Drive fake time past the connect bound so connectWithTimeout rejects and
// buildEntry catches it (records ok:false) — flushing the microtasks.
await jest.advanceTimersByTimeAsync(CONNECT_TIMEOUT_MS + 1);
const toolset = await toolsetPromise;
// The build COMPLETED with the bad server skipped (no tools, ok:false).
expect(Object.keys(toolset.tools)).toHaveLength(0);
expect(toolset.outcomes).toEqual([
{ name: 'hung', ok: false, reason: 'MCP connect timed out after 5000ms' },
]);
await Promise.all(toolset.clients.map((c) => c.close()));
// The cache is NOT poisoned: a subsequent turn returns (served from the warm
// cached entry) instead of awaiting a never-settling build.
const again = await svc.toolsFor('ws-1');
expect(Object.keys(again.tools)).toHaveLength(0);
await Promise.all(again.clients.map((c) => c.close()));
});
it('a hung server is skipped but a healthy server in the SAME build still contributes its tools', async () => {
const svc = buildService([
server({ id: 'id-hung', name: 'hung' }),
server({ id: 'id-ok', name: 'ok' }),
]);
const okClient = {
tools: () => Promise.resolve({ search: { description: 'x' } }),
close: jest.fn().mockResolvedValue(undefined),
};
stubConnect(svc, (s) =>
s.id === 'id-hung'
? new Promise<never>(() => {})
: Promise.resolve(okClient),
);
const toolsetPromise = svc.toolsFor('ws-2');
await jest.advanceTimersByTimeAsync(CONNECT_TIMEOUT_MS + 1);
const toolset = await toolsetPromise;
// Healthy server's tool survives (namespaced); hung server recorded ok:false.
expect(Object.keys(toolset.tools)).toEqual(['ok_search']);
expect(toolset.outcomes).toEqual([
{ name: 'hung', ok: false, reason: 'MCP connect timed out after 5000ms' },
{ name: 'ok', ok: true },
]);
await Promise.all(toolset.clients.map((c) => c.close()));
});
it('closes the ORPHANED client when connect resolves LATE (after the timeout)', async () => {
const svc = buildService([server({ id: 'id-late', name: 'late' })]);
const lateClient = {
tools: () => Promise.resolve({}),
close: jest.fn().mockResolvedValue(undefined),
};
// connect resolves only AFTER the connect bound has already elapsed, so
// connectWithTimeout has already rejected and must close this orphan.
stubConnect(
svc,
() =>
new Promise((resolve) => {
setTimeout(() => resolve(lateClient), CONNECT_TIMEOUT_MS * 2);
}),
);
const toolsetPromise = svc.toolsFor('ws-3');
// Fire the timeout: the build completes with the server skipped.
await jest.advanceTimersByTimeAsync(CONNECT_TIMEOUT_MS + 1);
const toolset = await toolsetPromise;
expect(toolset.outcomes[0]?.ok).toBe(false);
expect(lateClient.close).not.toHaveBeenCalled();
// Now let the late connect resolve — the orphan must be closed, not leaked.
await jest.advanceTimersByTimeAsync(CONNECT_TIMEOUT_MS * 2);
expect(lateClient.close).toHaveBeenCalledTimes(1);
await Promise.all(toolset.clients.map((c) => c.close()));
});
});
describe('McpClientsService.buildEntry — closes a connected client whose tools() fails (leak fix)', () => {
beforeEach(() => jest.useFakeTimers());
afterEach(() => {
jest.clearAllTimers();
jest.useRealTimers();
jest.restoreAllMocks();
});
it('connect succeeds but tools() REJECTS: the client is close()d exactly once and the server is skipped, while a healthy server still contributes', async () => {
const svc = buildService([
server({ id: 'id-bad', name: 'bad' }),
server({ id: 'id-ok', name: 'ok' }),
]);
// The bad server connects fine, then tools() rejects — the client would leak if
// buildEntry did not close it in the per-server catch (it was never registered).
const badClient = {
tools: () => Promise.reject(new Error('tools listing failed')),
close: jest.fn().mockResolvedValue(undefined),
};
const okClient = {
tools: () => Promise.resolve({ search: { description: 'x' } }),
close: jest.fn().mockResolvedValue(undefined),
};
stubConnect(svc, (s) =>
s.id === 'id-bad' ? Promise.resolve(badClient) : Promise.resolve(okClient),
);
const toolset = await svc.toolsFor('ws-4');
// The orphaned (never-registered) client is closed exactly once — no leak.
expect(badClient.close).toHaveBeenCalledTimes(1);
// Healthy server survives; bad server recorded ok:false and skipped.
expect(Object.keys(toolset.tools)).toEqual(['ok_search']);
expect(toolset.outcomes).toEqual([
{ name: 'bad', ok: false, reason: 'tools listing failed' },
{ name: 'ok', ok: true },
]);
// The healthy (registered) client is NOT closed by the loop — it is owned by the
// cache entry and stays warm (closed only on eviction/teardown, not on lease
// release). Releasing the lease keeps it warm since the entry is not evicted.
expect(okClient.close).not.toHaveBeenCalled();
await Promise.all(toolset.clients.map((c) => c.close()));
expect(okClient.close).not.toHaveBeenCalled();
// The failed client is never double-closed.
expect(badClient.close).toHaveBeenCalledTimes(1);
});
it('connect succeeds but tools() HANGS (times out): the client is close()d once and the server is skipped', async () => {
const svc = buildService([server({ id: 'id-slow', name: 'slow' })]);
const slowClient = {
// tools() never settles -> withTimeout rejects after CONNECT_TIMEOUT_MS.
tools: () => new Promise<Record<string, never>>(() => {}),
close: jest.fn().mockResolvedValue(undefined),
};
stubConnect(svc, () => Promise.resolve(slowClient));
const toolsetPromise = svc.toolsFor('ws-5');
// Drive fake time past the tools() bound so withTimeout rejects.
await jest.advanceTimersByTimeAsync(CONNECT_TIMEOUT_MS + 1);
const toolset = await toolsetPromise;
expect(slowClient.close).toHaveBeenCalledTimes(1);
expect(Object.keys(toolset.tools)).toHaveLength(0);
expect(toolset.outcomes[0]?.ok).toBe(false);
await Promise.all(toolset.clients.map((c) => c.close()));
});
});
@@ -195,7 +195,7 @@ export class McpClientsService {
): Promise<{ ok: true; tools: string[] } | { ok: false; error: string }> { ): Promise<{ ok: true; tools: string[] } | { ok: false; error: string }> {
let client: McpClient | undefined; let client: McpClient | undefined;
try { try {
client = await this.connect(server); client = await this.connectWithTimeout(server, CONNECT_TIMEOUT_MS);
const raw = await withTimeout(client.tools(), CONNECT_TIMEOUT_MS); const raw = await withTimeout(client.tools(), CONNECT_TIMEOUT_MS);
return { ok: true, tools: Object.keys(raw) }; return { ok: true, tools: Object.keys(raw) };
} catch (err) { } catch (err) {
@@ -256,10 +256,18 @@ export class McpClientsService {
const instructions: McpServerInstruction[] = []; const instructions: McpServerInstruction[] = [];
for (const server of servers) { for (const server of servers) {
// Track the connected client OUTSIDE the try so the catch can close it when
// it was obtained but not yet registered in `clients` (e.g. tools() threw or
// timed out after a successful connect). `registered` flips only once the
// client is owned by `clients` (closed at entry teardown), so the catch never
// double-closes a registered client.
let client: McpClient | undefined;
let registered = false;
try { try {
const client = await this.connect(server); client = await this.connectWithTimeout(server, CONNECT_TIMEOUT_MS);
const raw = await withTimeout(client.tools(), CONNECT_TIMEOUT_MS); const raw = await withTimeout(client.tools(), CONNECT_TIMEOUT_MS);
clients.push(client); clients.push(client);
registered = true;
const allow = server.toolAllowlist; const allow = server.toolAllowlist;
const picked = const picked =
Array.isArray(allow) && allow.length > 0 ? pick(raw, allow) : raw; Array.isArray(allow) && allow.length > 0 ? pick(raw, allow) : raw;
@@ -290,9 +298,15 @@ export class McpClientsService {
}); });
} }
} catch (err) { } catch (err) {
// A failed server is skipped — the turn proceeds with the rest. Log a // A failed server is skipped — the turn proceeds with the rest. If connect
// short warning (never the URL/headers) so ops can see degradation, and // returned a live client but a later step (tools()) threw, that client was
// record the outcome so the UI can show "tool X unavailable". // never registered in `clients`, so close it here or its transport/socket
// leaks (compounding every 60s cache rebuild during a flaky-server outage).
if (client && !registered) {
void client.close().catch(() => undefined);
}
// Log a short warning (never the URL/headers) so ops can see degradation,
// and record the outcome so the UI can show "tool X unavailable".
const reason = shortError(err); const reason = shortError(err);
this.logger.warn( this.logger.warn(
`External MCP server "${server.name}" unavailable: ${reason}`, `External MCP server "${server.name}" unavailable: ${reason}`,
@@ -383,6 +397,55 @@ export class McpClientsService {
return client; return client;
} }
/**
* Race {@link connect} against a SETTLING timeout so a hung MCP handshake can
* never POISON the per-workspace build cache. `createMCPClient` (inside connect)
* is NOT bounded internally, and exactly like @ai-sdk/mcp's tool calls
* (see wrapToolWithCallTimeout) its promise does NOT settle on abort. So a
* transient network blip mid-handshake can make connect hang FOREVER. Because
* getOrBuildEntry caches the build PROMISE, a never-settling connect would then
* wedge EVERY later turn for the workspace (each awaits the same pending build,
* step_count stuck at 0, run row leaks 'running', chat 409s forever). Bounding
* connect here guarantees buildEntry always gets a client OR a rejection within
* `ms` so the build completes (bad server skipped) and the cache stays clean.
*
* If connect resolves LATE (after we already rejected on the timeout), we close
* the orphaned client so its transport/socket is not leaked.
*/
private connectWithTimeout(
server: Pick<AiMcpServer, 'transport' | 'url' | 'headersEnc'>,
ms: number,
): Promise<McpClient> {
return new Promise<McpClient>((resolve, reject) => {
let settled = false;
const timer = setTimeout(() => {
settled = true;
reject(new Error(`MCP connect timed out after ${ms}ms`));
}, ms);
// Do not keep the process alive just for this connect-timeout timer.
timer.unref?.();
this.connect(server).then(
(client) => {
if (settled) {
// The race was already lost to the timeout: close the orphaned client
// so its socket is not leaked, and drop the late result.
void client.close().catch(() => undefined);
return;
}
clearTimeout(timer);
settled = true;
resolve(client);
},
(err: unknown) => {
if (settled) return; // late rejection after the timeout — already handled
clearTimeout(timer);
settled = true;
reject(err instanceof Error ? err : new Error(String(err)));
},
);
});
}
/** /**
* Decrypt the stored auth headers. Returns undefined when none are set. The * Decrypt the stored auth headers. Returns undefined when none are set. The
* plaintext headers live only in this returned object and are passed straight * plaintext headers live only in this returned object and are passed straight
@@ -651,3 +651,69 @@ describe('AiChatToolsService #294 changed execute wirings', () => {
expect(calls.tableUpdateCell).toEqual([['p1', '#0', 1, 2, 'x']]); expect(calls.tableUpdateCell).toEqual([['p1', '#0', 1, 2, 'x']]);
}); });
}); });
/**
* getCurrentPage selection contract (#388): the tool surfaces the selection that
* was sanitized + nested onto the resolved open-page context (last forUser arg).
* No page => selection is null. The tool never fetches or verifies anything it
* just projects the resolved context.
*/
describe('AiChatToolsService getCurrentPage selection (#388)', () => {
const tokenServiceStub = {
generateAccessToken: jest.fn().mockResolvedValue('access-token'),
generateCollabToken: jest.fn().mockResolvedValue('collab-token'),
};
let service: AiChatToolsService;
beforeEach(() => {
jest.spyOn(loader, 'loadDocmostMcp').mockResolvedValue(
mockLoaded(function () {
return {} as DocmostClientLike;
} as unknown as loader.DocmostClientCtor),
);
service = new AiChatToolsService(
tokenServiceStub as never,
{} as never,
{} as never,
{} as never,
{} as never,
{
asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }),
} as never,
);
});
afterEach(() => jest.restoreAllMocks());
const buildTools = (openedPage: unknown) =>
service.forUser(
{ id: 'user-1', email: 'u@example.com', workspaceId: 'ws-1' } as never,
'session-1',
'ws-1',
'chat-1',
openedPage as never,
);
it('returns the nested selection from the resolved context', async () => {
const selection = { text: 'fix this', blockIds: ['b1'], before: 'a ' };
const tools = await buildTools({ id: 'p1', title: 'Doc', selection });
expect(await tools.getCurrentPage.execute({} as never, {} as never)).toEqual(
{ page: { id: 'p1', title: 'Doc' }, selection },
);
});
it('returns selection: null when the context has no selection', async () => {
const tools = await buildTools({ id: 'p1', title: 'Doc' });
expect(await tools.getCurrentPage.execute({} as never, {} as never)).toEqual(
{ page: { id: 'p1', title: 'Doc' }, selection: null },
);
});
it('returns { page: null, selection: null } when no page is open', async () => {
const tools = await buildTools(null);
expect(await tools.getCurrentPage.execute({} as never, {} as never)).toEqual(
{ page: null, selection: null },
);
});
});
@@ -13,7 +13,10 @@ import {
type DocmostClientLike, type DocmostClientLike,
type SharedToolSpec, type SharedToolSpec,
} from './docmost-client.loader'; } from './docmost-client.loader';
import { resolveCurrentPageResult } from './current-page.util'; import {
resolveCurrentPageResult,
type SelectionContext,
} from './current-page.util';
import { parseNodeArg } from './parse-node-arg'; import { parseNodeArg } from './parse-node-arg';
import { modelFriendlyInput } from './model-friendly-input'; import { modelFriendlyInput } from './model-friendly-input';
import { SandboxStore } from '../../../integrations/sandbox/sandbox.store'; import { SandboxStore } from '../../../integrations/sandbox/sandbox.store';
@@ -153,8 +156,13 @@ export class AiChatToolsService {
// The page the user currently has open (from the request context), exposed // The page the user currently has open (from the request context), exposed
// to the model via getCurrentPage. Optional and last so existing callers // to the model via getCurrentPage. Optional and last so existing callers
// keep compiling. Kept proxy-robust: the model can CALL for the current // keep compiling. Kept proxy-robust: the model can CALL for the current
// page instead of relying on it surviving in the system prompt text. // page instead of relying on it surviving in the system prompt text. The
openedPage?: { id?: string; title?: string } | null, // `selection` (#388) is already sanitized + nested by resolveOpenPageContext.
openedPage?: {
id?: string;
title?: string;
selection?: SelectionContext | null;
} | null,
): Promise<Record<string, Tool>> { ): Promise<Record<string, Tool>> {
// Build the per-user loopback client (carrying the access + collab // Build the per-user loopback client (carrying the access + collab
// provenance tokens) and load the shared tool-spec registry. Client // provenance tokens) and load the shared tool-spec registry. Client
@@ -309,9 +317,15 @@ export class AiChatToolsService {
getCurrentPage: tool({ getCurrentPage: tool({
description: description:
'Return the page the user is currently viewing — i.e. what "this page", ' + 'Return the page the user is currently viewing — i.e. what "this page", ' +
'"the current page", or "here" refers to. Returns the page id and title, ' + '"the current page", or "here" refers to — plus the text the user ' +
'or null if the user is not currently on a page. Call this first whenever ' + 'currently has SELECTED on that page (what "this", "here", "the selected ' +
'the user refers to the current page without giving an explicit id.', 'fragment" refers to), or selection: null when nothing is selected. The ' +
'selection is a client-side snapshot taken when the user sent the message ' +
'and includes the ids of the blocks it covers plus surrounding context; ' +
'it is NOT verified server-side — locate it in the page (searchInPage / ' +
'getNode) before editing. Returns page: null if the user is not currently ' +
'on a page. Call this first whenever the user refers to the current page ' +
'or a selected fragment without giving an explicit id.',
inputSchema: modelFriendlyInput({}), inputSchema: modelFriendlyInput({}),
execute: async () => resolveCurrentPageResult(openedPage), execute: async () => resolveCurrentPageResult(openedPage),
}), }),
@@ -1,43 +1,180 @@
import { resolveCurrentPageResult } from './current-page.util'; import {
resolveCurrentPageResult,
sanitizeSelection,
} from './current-page.util';
/** /**
* Unit tests for resolveCurrentPageResult (pure function). Mirrors the * Unit tests for resolveCurrentPageResult (pure function). Mirrors the
* getCurrentPage tool's contract: { page: null } when no page is open (no id), * getCurrentPage tool's contract: { page: null, selection: null } when no page
* otherwise { page: { id, title } } with title defaulting to ''. * is open (no id), otherwise { page: { id, title }, selection } with title
* defaulting to '' and the selection passed through from the resolved context.
*/ */
describe('resolveCurrentPageResult', () => { describe('resolveCurrentPageResult', () => {
it('returns { page: null } when openedPage is undefined', () => { it('returns { page: null, selection: null } when openedPage is undefined', () => {
expect(resolveCurrentPageResult(undefined)).toEqual({ page: null }); expect(resolveCurrentPageResult(undefined)).toEqual({
page: null,
selection: null,
});
}); });
it('returns { page: null } when openedPage is null', () => { it('returns { page: null, selection: null } when openedPage is null', () => {
expect(resolveCurrentPageResult(null)).toEqual({ page: null }); expect(resolveCurrentPageResult(null)).toEqual({
page: null,
selection: null,
});
}); });
it('returns { page: null } when openedPage has no id', () => { it('returns { page: null, selection: null } when openedPage has no id', () => {
expect(resolveCurrentPageResult({})).toEqual({ page: null }); expect(resolveCurrentPageResult({})).toEqual({
expect(resolveCurrentPageResult({ title: 'x' })).toEqual({ page: null }); page: null,
selection: null,
});
expect(resolveCurrentPageResult({ title: 'x' })).toEqual({
page: null,
selection: null,
});
}); });
it('returns { page: null } when id is an empty string', () => { it('returns { page: null, selection: null } when id is an empty string', () => {
expect(resolveCurrentPageResult({ id: '' })).toEqual({ page: null }); expect(resolveCurrentPageResult({ id: '' })).toEqual({
page: null,
selection: null,
});
}); });
it('returns the page id and title when both are present', () => { it('drops the selection when there is no page (selection dies with the page)', () => {
// Even if a selection somehow rode along without a page id, a null page
// always yields a null selection.
expect(
resolveCurrentPageResult({ selection: { text: 'orphan' } }),
).toEqual({ page: null, selection: null });
});
it('returns the page id and title with a null selection by default', () => {
expect(resolveCurrentPageResult({ id: 'p1', title: 'Hello' })).toEqual({ expect(resolveCurrentPageResult({ id: 'p1', title: 'Hello' })).toEqual({
page: { id: 'p1', title: 'Hello' }, page: { id: 'p1', title: 'Hello' },
selection: null,
});
});
it('passes the nested selection through verbatim', () => {
const selection = {
text: 'fix this',
blockIds: ['b1'],
before: 'please ',
after: ' now',
};
expect(
resolveCurrentPageResult({ id: 'p1', title: 'Hello', selection }),
).toEqual({
page: { id: 'p1', title: 'Hello' },
selection,
}); });
}); });
it('defaults title to "" when it is missing', () => { it('defaults title to "" when it is missing', () => {
expect(resolveCurrentPageResult({ id: 'p1' })).toEqual({ expect(resolveCurrentPageResult({ id: 'p1' })).toEqual({
page: { id: 'p1', title: '' }, page: { id: 'p1', title: '' },
selection: null,
}); });
}); });
it('keeps an explicit empty-string title as ""', () => { it('keeps an explicit empty-string title as ""', () => {
expect(resolveCurrentPageResult({ id: 'p1', title: '' })).toEqual({ expect(resolveCurrentPageResult({ id: 'p1', title: '' })).toEqual({
page: { id: 'p1', title: '' }, page: { id: 'p1', title: '' },
selection: null,
}); });
}); });
}); });
/**
* Unit tests for sanitizeSelection (#388). The selection is an attacker-
* controllable client snapshot: every field is type-checked and capped, and
* anything that is not a real selection collapses to null. It is NEVER verified
* against the page content (decision 5 a hint, not ground truth).
*/
describe('sanitizeSelection', () => {
it('accepts a well-formed payload unchanged', () => {
const raw = {
text: 'the selected fragment',
truncated: true,
blockIds: ['b1', 'b2'],
before: 'context before ',
after: ' context after',
};
expect(sanitizeSelection(raw)).toEqual(raw);
});
it('returns null for non-objects', () => {
expect(sanitizeSelection(null)).toBeNull();
expect(sanitizeSelection(undefined)).toBeNull();
expect(sanitizeSelection('text')).toBeNull();
expect(sanitizeSelection(42)).toBeNull();
expect(sanitizeSelection([])).toBeNull();
});
it('returns null when text is missing, non-string or blank-after-trim', () => {
expect(sanitizeSelection({})).toBeNull();
expect(sanitizeSelection({ text: 123 })).toBeNull();
expect(sanitizeSelection({ text: '' })).toBeNull();
expect(sanitizeSelection({ text: ' \n ' })).toBeNull();
});
it('keeps only text when the other fields are garbage', () => {
expect(
sanitizeSelection({
text: 'hello',
truncated: 'yes',
blockIds: 'nope',
before: 5,
after: {},
}),
).toEqual({ text: 'hello' });
});
it('caps text at 4000 and forces truncated', () => {
const raw = { text: 'a'.repeat(5000) };
const out = sanitizeSelection(raw)!;
expect(out.text).toHaveLength(4000);
expect(out.truncated).toBe(true);
});
it('does not set truncated for text under the cap', () => {
expect(sanitizeSelection({ text: 'short' })).toEqual({ text: 'short' });
});
it('slices blockIds to 20 and drops non-string / oversized ids', () => {
const ids = Array.from({ length: 30 }, (_, i) => `b${i}`);
const out = sanitizeSelection({
text: 'x',
blockIds: [...ids, 123, '', 'y'.repeat(65)],
})!;
// The 30 valid ids cap to the first 20; the number, empty string and the
// 65-char id are dropped before the slice.
expect(out.blockIds).toHaveLength(20);
expect(out.blockIds).toEqual(ids.slice(0, 20));
});
it('keeps a 64-char id but drops a 65-char one (boundary)', () => {
const ok = 'z'.repeat(64);
const tooLong = 'z'.repeat(65);
expect(
sanitizeSelection({ text: 'x', blockIds: [ok, tooLong] })!.blockIds,
).toEqual([ok]);
});
it('omits blockIds entirely when none survive', () => {
const out = sanitizeSelection({ text: 'x', blockIds: [123, ''] })!;
expect(out.blockIds).toBeUndefined();
});
it('caps before/after at 200 chars and drops empty ones', () => {
const out = sanitizeSelection({
text: 'x',
before: 'b'.repeat(300),
after: '',
})!;
expect(out.before).toHaveLength(200);
expect(out.after).toBeUndefined();
});
});
@@ -1,21 +1,91 @@
export interface SelectionContext {
text: string;
truncated?: boolean;
blockIds?: string[];
before?: string;
after?: string;
}
// Server-side caps for the client-reported selection. Intentionally >= the
// client caps: the client pre-trims for a small wire, but this layer re-checks
// everything because the payload is attacker-controllable.
const TEXT_CAP = 4000;
const CONTEXT_CAP = 200;
const MAX_BLOCK_IDS = 20;
const BLOCK_ID_CAP = 64;
// Sanitize the client-reported selection: type-check every field, cap sizes
// (text 4000, before/after 200, blockIds 20 x 64 chars), drop garbage to null.
// The selection is a CLIENT-side snapshot — never verified against the page
// content (#159 lesson: treat as a hint, not ground truth). The agent is told
// (getCurrentPage's description) to localize it before editing.
export function sanitizeSelection(raw: unknown): SelectionContext | null {
if (!raw || typeof raw !== 'object') return null;
const r = raw as Record<string, unknown>;
// text is the only required field; anything else is a non-selection.
if (typeof r.text !== 'string' || r.text.trim().length === 0) return null;
let text = r.text;
let truncated = r.truncated === true;
if (text.length > TEXT_CAP) {
text = text.slice(0, TEXT_CAP);
truncated = true;
}
const result: SelectionContext = { text };
if (truncated) result.truncated = true;
if (Array.isArray(r.blockIds)) {
// Keep only well-formed, in-range ids (an oversize id is DROPPED, not
// truncated — a mangled id is worse than a missing one), then cap the count.
const ids = r.blockIds
.filter(
(x): x is string =>
typeof x === 'string' && x.length > 0 && x.length <= BLOCK_ID_CAP,
)
.slice(0, MAX_BLOCK_IDS);
if (ids.length > 0) result.blockIds = ids;
}
if (typeof r.before === 'string' && r.before.length > 0) {
result.before = r.before.slice(0, CONTEXT_CAP);
}
if (typeof r.after === 'string' && r.after.length > 0) {
result.after = r.after.slice(0, CONTEXT_CAP);
}
return result;
}
export interface CurrentPageInput { export interface CurrentPageInput {
id?: string; id?: string;
title?: string; title?: string;
// The already-sanitized selection nested onto the resolved open-page context
// by resolveOpenPageContext (never the raw client value). Passed through to
// the tool result verbatim; null when nothing is selected.
selection?: SelectionContext | null;
} }
export interface CurrentPageResult { export interface CurrentPageResult {
page: { id: string; title: string } | null; page: { id: string; title: string } | null;
selection: SelectionContext | null; // null when nothing is selected or no page
} }
// Resolve the "current page" tool result from the client-supplied open-page // Resolve the "current page" tool result from the client-supplied open-page
// context. Returns { page: null } when no page is open (no id), otherwise the // context. Returns { page: null, selection: null } when no page is open (no id),
// page id + title (title defaults to '' when absent). Mirrors the getCurrentPage // otherwise the page id + title (title defaults to '' when absent) plus the
// tool's contract so it can be unit-tested without the ESM Docmost client. // selection already sanitized+nested by resolveOpenPageContext. A null page
// always yields a null selection (the selection dies with the page). Mirrors the
// getCurrentPage tool's contract so it can be unit-tested without the ESM
// Docmost client.
export function resolveCurrentPageResult( export function resolveCurrentPageResult(
openedPage?: CurrentPageInput | null, openedPage?: CurrentPageInput | null,
): CurrentPageResult { ): CurrentPageResult {
if (!openedPage?.id) { if (!openedPage?.id) {
return { page: null }; return { page: null, selection: null };
} }
return { page: { id: openedPage.id, title: openedPage.title ?? '' } }; return {
page: { id: openedPage.id, title: openedPage.title ?? '' },
selection: openedPage.selection ?? null,
};
} }
@@ -98,7 +98,8 @@ export const INLINE_TOOL_TIERS: Record<
}, },
getCurrentPage: { getCurrentPage: {
tier: 'core', tier: 'core',
catalogLine: 'getCurrentPage — the page the user is currently viewing.', catalogLine:
'getCurrentPage — the page the user is currently viewing and their current text selection on it.',
}, },
// NOTE: getPage and listPages moved to @docmost/mcp's SHARED_TOOL_SPECS // NOTE: getPage and listPages moved to @docmost/mcp's SHARED_TOOL_SPECS
// (#294); they carry their own tier ('core') + catalogLine there. // (#294); they carry their own tier ('core') + catalogLine there.
@@ -23,8 +23,21 @@ import { hashPassword } from '../../../common/helpers';
* unthrottled password-guessing oracle. * unthrottled password-guessing oracle.
*/ */
// bcrypt cost-12 hashing/compare takes ~300ms idle but multiple seconds when
// parallel jest workers saturate all CPU cores; the 5s default flakes.
jest.setTimeout(30_000);
const WORKSPACE_ID = 'ws-1'; const WORKSPACE_ID = 'ws-1';
let passwordHash: string;
// Hoist the expensive work: compute ONE bcrypt cost-12 hash shared by all
// tests instead of five. The hash is a read-only string and each test builds
// its own user object around it, so sharing is safe.
beforeAll(async () => {
passwordHash = await hashPassword('correct-horse');
}, 30_000);
// Build an AuthService with the dependencies verifyUserCredentials/login touch // Build an AuthService with the dependencies verifyUserCredentials/login touch
// stubbed, and a userRepo whose findByEmail is overridable per test. Only the // stubbed, and a userRepo whose findByEmail is overridable per test. Only the
// collaborators actually reached on these paths need real behaviour; the rest // collaborators actually reached on these paths need real behaviour; the rest
@@ -95,7 +108,6 @@ describe('AuthService.verifyUserCredentials (live credentials-mismatch contract)
it('DISABLED user -> throws exactly CREDENTIALS_MISMATCH_MESSAGE (no password oracle)', async () => { it('DISABLED user -> throws exactly CREDENTIALS_MISMATCH_MESSAGE (no password oracle)', async () => {
// A deactivated user must be indistinguishable from a wrong password: same // A deactivated user must be indistinguishable from a wrong password: same
// message, before any password comparison. // message, before any password comparison.
const passwordHash = await hashPassword('correct-horse');
const disabledUser = { const disabledUser = {
id: 'u-1', id: 'u-1',
email: 'disabled@example.com', email: 'disabled@example.com',
@@ -117,7 +129,6 @@ describe('AuthService.verifyUserCredentials (live credentials-mismatch contract)
}); });
it('WRONG password -> throws exactly CREDENTIALS_MISMATCH_MESSAGE', async () => { it('WRONG password -> throws exactly CREDENTIALS_MISMATCH_MESSAGE', async () => {
const passwordHash = await hashPassword('correct-horse');
const user = { const user = {
id: 'u-1', id: 'u-1',
email: 'user@example.com', email: 'user@example.com',
@@ -139,7 +150,6 @@ describe('AuthService.verifyUserCredentials (live credentials-mismatch contract)
}); });
it('CORRECT credentials -> resolves the matched user (no side effects here)', async () => { it('CORRECT credentials -> resolves the matched user (no side effects here)', async () => {
const passwordHash = await hashPassword('correct-horse');
const user = { const user = {
id: 'u-1', id: 'u-1',
email: 'user@example.com', email: 'user@example.com',
@@ -179,7 +189,6 @@ describe('AuthService.login (live credentials-mismatch contract via verifyUserCr
}); });
it('WRONG password -> login throws exactly CREDENTIALS_MISMATCH_MESSAGE', async () => { it('WRONG password -> login throws exactly CREDENTIALS_MISMATCH_MESSAGE', async () => {
const passwordHash = await hashPassword('correct-horse');
const user = { const user = {
id: 'u-1', id: 'u-1',
email: 'user@example.com', email: 'user@example.com',
@@ -201,7 +210,6 @@ describe('AuthService.login (live credentials-mismatch contract via verifyUserCr
}); });
it('CORRECT credentials -> login mints the session (the side-effecting path)', async () => { it('CORRECT credentials -> login mints the session (the side-effecting path)', async () => {
const passwordHash = await hashPassword('correct-horse');
const user = { const user = {
id: 'u-1', id: 'u-1',
email: 'user@example.com', email: 'user@example.com',
@@ -292,6 +292,23 @@ export class EnvironmentService {
return enabled === 'true'; return enabled === 'true';
} }
/**
* Resumable SSE transport for durable agent runs (#184 phase 1.5). When
* enabled, a run tees its SSE frames into the in-memory run-stream registry so
* a late/reloaded tab can attach (replay + live tail) via
* `GET /ai-chat/runs/:chatId/stream`. Defaults to DISABLED: PR 1 ships the
* server code dormant with the flag off, `open`/`bind`/`generateMessageId`
* are never called and attach always answers 204, so the legacy and #184
* phase-1 wire paths stay byte-for-byte identical. Set
* AI_CHAT_RESUMABLE_STREAM=true to activate it (paired with the PR 2 client).
*/
isAiChatResumableStreamEnabled(): boolean {
const enabled = this.configService
.get<string>('AI_CHAT_RESUMABLE_STREAM', 'false')
.toLowerCase();
return enabled === 'true';
}
getPostHogHost(): string { getPostHogHost(): string {
return this.configService.get<string>('POSTHOG_HOST'); return this.configService.get<string>('POSTHOG_HOST');
} }
@@ -0,0 +1,564 @@
import * as http from 'node:http';
import { Kysely } from 'kysely';
import {
MockLanguageModelV3,
convertArrayToReadableStream,
} from 'ai/test';
import { AiChatRepo } from '@docmost/db/repos/ai-chat/ai-chat.repo';
import { AiChatMessageRepo } from '@docmost/db/repos/ai-chat/ai-chat-message.repo';
import { AiChatRunRepo } from '@docmost/db/repos/ai-chat/ai-chat-run.repo';
import { AiChatService } from 'src/core/ai-chat/ai-chat.service';
import { AiChatRunService } from 'src/core/ai-chat/ai-chat-run.service';
import {
AiChatStreamRegistryService,
RunStreamCallbacks,
} from 'src/core/ai-chat/ai-chat-stream-registry.service';
import {
getTestDb,
destroyTestDb,
createWorkspace,
createUser,
createChat,
} from './db';
/**
* #184 phase 1.5 the resumable transport end to end against REAL Postgres,
* the REAL `streamText` (seeded via MockLanguageModelV3) and a REAL Node
* ServerResponse, driving the REAL `AiChatService.stream` run-wrapped path with a
* REAL `AiChatStreamRegistryService`. The run-hooks mirror the controller: they
* begin a durable run and `open()` the registry entry at begin, and the service
* tees the SSE frames into it via `consumeSseStream` while stamping the DB row id
* via `generateMessageId` (both gated on runId + the resumable flag).
*
* Proven here: a finished run's replay is the full frame sequence incl `[DONE]`
* with `start.messageId` == the seeded DB row id; the anchor check (invariant 6);
* an attach opened BEFORE the first frame follows the live stream from frame 0; an
* explicit stop surfaces `{"type":"abort"}` + `[DONE]` + end to the subscriber;
* and the legacy (non-run) path tees nothing.
*/
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
async function waitFor(
cond: () => Promise<boolean> | boolean,
{ timeoutMs = 15_000, stepMs = 25 } = {},
): Promise<void> {
const start = Date.now();
while (Date.now() - start < timeoutMs) {
if (await cond()) return;
await sleep(stepMs);
}
throw new Error('waitFor: condition not met within timeout');
}
// A real Node ServerResponse wired to a live socket (as in the stream int-spec).
function makeRealResponse(): Promise<{
res: http.ServerResponse;
cleanup: () => Promise<void>;
}> {
return new Promise((resolve) => {
const server = http.createServer((_req, res) => {
resolve({
res,
cleanup: () =>
new Promise<void>((done) => {
try {
if (!res.writableEnded) res.end();
} catch {
/* socket already gone */
}
server.close(() => done());
}),
});
});
server.listen(0, () => {
const port = (server.address() as any).port;
const creq = http.request({ port, method: 'GET' }, (cres) => {
cres.resume();
});
creq.on('error', () => undefined);
creq.end();
});
});
}
// A full, successful single-step turn.
function successStream() {
return convertArrayToReadableStream([
{ type: 'stream-start', warnings: [] },
{ type: 'text-start', id: 't1' },
{ type: 'text-delta', id: 't1', delta: 'Hello' },
{ type: 'text-delta', id: 't1', delta: ' there' },
{ type: 'text-end', id: 't1' },
{
type: 'finish',
finishReason: 'stop',
usage: { inputTokens: 10, outputTokens: 5, totalTokens: 15 },
},
] as any);
}
// A stream the test feeds chunk by chunk (to attach mid-flight / drive a stop).
function makeControlledStream() {
let controller!: ReadableStreamDefaultController<any>;
const stream = new ReadableStream<any>({
start(c) {
controller = c;
},
});
return {
stream,
emit: (chunk: any) => controller.enqueue(chunk),
close: () => controller.close(),
};
}
// Collect replay + live frames from an attachment.
function liveSink(): {
cb: RunStreamCallbacks;
frames: string[];
ended: () => boolean;
} {
const frames: string[] = [];
let ended = false;
return {
frames,
ended: () => ended,
cb: {
onFrame: (f) => frames.push(f),
onEnd: () => {
ended = true;
},
},
};
}
// The SSE `start` frame carries the message id; pull it out of a `data: {...}`.
function parseStartMessageId(frames: string[]): string | undefined {
for (const f of frames) {
const m = /^data: (\{.*\})\s*$/m.exec(f.trim());
if (!m) continue;
try {
const json = JSON.parse(m[1]);
if (json.type === 'start') return json.messageId;
} catch {
/* not this frame */
}
}
return undefined;
}
describe('AiChatService run-stream attach [integration]', () => {
let db: Kysely<any>;
let aiChatRepo: AiChatRepo;
let msgRepo: AiChatMessageRepo;
let runRepo: AiChatRunRepo;
let workspaceId: string;
let userId: string;
const mcpClients = {
toolsFor: async () => ({
tools: {},
clients: [],
outcomes: [],
instructions: [],
}),
};
// Build the service with the run-stream registry wired and the resumable flag
// ON (the property under test). Deferred tools OFF (irrelevant here).
function buildService(registry: AiChatStreamRegistryService): AiChatService {
return new AiChatService(
{ getChatModel: async () => null } as any,
aiChatRepo,
msgRepo,
{} as any,
{ resolve: async () => null } as any,
{ forUser: async () => ({}) } as any,
mcpClients as any,
{} as any,
{} as any,
{} as any,
{
isAiChatDeferredToolsEnabled: () => false,
isAiChatResumableStreamEnabled: () => true,
} as any,
registry,
);
}
// Run-hooks mirroring the controller: begin the durable run AND open() the
// registry entry at begin. Captures the runId so a test can stop it.
function makeRunHooks(
runService: AiChatRunService,
registry: AiChatStreamRegistryService,
box: { runId?: string },
) {
return {
begin: async (chatId: string) => {
const handle = await runService.beginRun({
chatId,
workspaceId,
userId,
trigger: 'user',
});
box.runId = handle.runId;
registry.open(chatId, handle.runId);
return handle;
},
onAssistantSeeded: (runId: string, messageId: string) =>
runService.linkAssistantMessage(runId, workspaceId, messageId),
onStep: (runId: string, n: number) =>
void runService.recordStep(runId, workspaceId, n),
onSettled: (runId: string, status: any, error?: string) =>
runService.finalizeRun(runId, workspaceId, status, error),
};
}
function userUiMessage(text: string) {
return {
id: `u-${Math.random()}`,
role: 'user',
parts: [{ type: 'text', text }],
};
}
async function startRun(opts: {
registry: AiChatStreamRegistryService;
runService?: AiChatRunService;
model: MockLanguageModelV3;
chatId: string;
body: any;
box?: { runId?: string };
}): Promise<{ res: http.ServerResponse; cleanup: () => Promise<void> }> {
const service = buildService(opts.registry);
const { res, cleanup } = await makeRealResponse();
const runHooks = opts.runService
? makeRunHooks(opts.runService, opts.registry, opts.box ?? {})
: undefined;
await service.stream({
user: { id: userId, workspaceId } as any,
workspace: { id: workspaceId, name: 'WS' } as any,
sessionId: 'sess-1',
body: opts.body,
res: { raw: res } as any,
signal: new AbortController().signal,
model: opts.model as any,
role: null,
runHooks,
} as any);
return { res, cleanup };
}
async function assistantRowId(chatId: string): Promise<string> {
const rows = await msgRepo.findAllByChat(chatId, workspaceId);
const row = rows.find((r: any) => r.role === 'assistant');
return row!.id as string;
}
beforeAll(async () => {
db = getTestDb();
aiChatRepo = new AiChatRepo(db as any);
msgRepo = new AiChatMessageRepo(db as any);
runRepo = new AiChatRunRepo(db as any);
workspaceId = (await createWorkspace(db)).id;
userId = (await createUser(db, workspaceId)).id;
});
afterAll(async () => {
await destroyTestDb();
});
it('run-wrapped: replay is the full frame sequence incl [DONE], start.messageId == the seeded DB row id', async () => {
const chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
const registry = new AiChatStreamRegistryService();
const runService = new AiChatRunService(runRepo, {
isCloud: () => false,
} as never);
const model = new MockLanguageModelV3({
doStream: async () => ({ stream: successStream() }),
} as any);
const box: { runId?: string } = {};
const { cleanup } = await startRun({
registry,
runService,
model,
chatId,
body: { chatId, messages: [userUiMessage('Hi')] },
box,
});
try {
// Wait for the assistant row to settle (terminal callbacks run async).
await waitFor(async () => {
const rows = await msgRepo.findAllByChat(chatId, workspaceId);
return rows.some(
(r: any) =>
r.role === 'assistant' &&
['completed', 'error', 'aborted'].includes(r.status),
);
});
const rowId = await assistantRowId(chatId);
// Finished-run replay with expect=live + the correct anchor.
const sink = liveSink();
const att = await registry.attach(chatId, true, rowId, sink.cb);
expect(att).not.toBeNull();
expect(att!.finished).toBe(true);
// The tee captured frames (consumeSseStream was wired).
expect(att!.replay.length).toBeGreaterThan(0);
// generateMessageId stamped the DB row id onto the streamed start frame.
expect(parseStartMessageId(att!.replay)).toBe(rowId);
// The full sequence includes the streamed text and the terminal marker.
const joined = att!.replay.join('');
expect(joined).toContain('Hello');
expect(att!.replay.some((f) => f.includes('[DONE]'))).toBe(true);
} finally {
registry.onModuleDestroy();
await cleanup();
}
});
it('anchor mismatch with expect=live returns null (invariant 6)', async () => {
const chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
const registry = new AiChatStreamRegistryService();
const runService = new AiChatRunService(runRepo, {
isCloud: () => false,
} as never);
const model = new MockLanguageModelV3({
doStream: async () => ({ stream: successStream() }),
} as any);
const { cleanup } = await startRun({
registry,
runService,
model,
chatId,
body: { chatId, messages: [userUiMessage('Hi')] },
});
try {
await waitFor(async () => {
const rows = await msgRepo.findAllByChat(chatId, workspaceId);
return rows.some(
(r: any) => r.role === 'assistant' && r.status === 'completed',
);
});
const sink = liveSink();
// A foreign anchor must NOT replay this run's transcript.
expect(
await registry.attach(chatId, true, 'a-different-run-row', sink.cb),
).toBeNull();
} finally {
registry.onModuleDestroy();
await cleanup();
}
});
it('an attach opened BEFORE the first frame follows the live stream from frame 0', async () => {
const chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
const registry = new AiChatStreamRegistryService();
const runService = new AiChatRunService(runRepo, {
isCloud: () => false,
} as never);
const controlled = makeControlledStream();
const model = new MockLanguageModelV3({
doStream: async () => ({ stream: controlled.stream }),
} as any);
const { cleanup } = await startRun({
registry,
runService,
model,
chatId,
body: { chatId, messages: [userUiMessage('Slow please')] },
});
try {
// Attach while the entry exists (opened at begin) but before any frame.
const sink = liveSink();
const att = (await registry.attach(chatId, false, undefined, sink.cb))!;
expect(att.replay).toEqual([]); // nothing streamed yet -> replay from 0
att.start(); // go live (drains nothing, then follows)
// Now emit the whole turn.
controlled.emit({ type: 'stream-start', warnings: [] });
controlled.emit({ type: 'text-start', id: 't1' });
controlled.emit({ type: 'text-delta', id: 't1', delta: 'Zero' });
controlled.emit({ type: 'text-end', id: 't1' });
controlled.emit({
type: 'finish',
finishReason: 'stop',
usage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 },
});
controlled.close();
await waitFor(() => sink.frames.some((f) => f.includes('[DONE]')));
// The subscriber saw the stream from the very first frame (`start`) through
// the terminal marker, with the streamed text present.
expect(sink.frames.some((f) => f.includes('"type":"start"'))).toBe(true);
expect(sink.frames.join('')).toContain('Zero');
expect(sink.frames[sink.frames.length - 1]).toContain('[DONE]');
expect(sink.ended()).toBe(true);
} finally {
registry.onModuleDestroy();
await cleanup();
}
});
it('requestStop surfaces {"type":"abort"} + [DONE] + end to the attached subscriber', async () => {
const chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
const registry = new AiChatStreamRegistryService();
const runService = new AiChatRunService(runRepo, {
isCloud: () => false,
} as never);
// An abort-AWARE model: it streams some partial output, then errors the model
// stream with an AbortError when the run signal aborts — exactly as a real
// provider network stream is torn down on abort (a plain in-memory stream
// would just stall, so streamText would never observe the stop).
const model = new MockLanguageModelV3({
doStream: async ({ abortSignal }: any) => {
const stream = new ReadableStream<any>({
start(controller) {
controller.enqueue({ type: 'stream-start', warnings: [] });
controller.enqueue({ type: 'text-start', id: 't1' });
controller.enqueue({ type: 'text-delta', id: 't1', delta: 'partial' });
abortSignal?.addEventListener('abort', () => {
try {
controller.error(
new DOMException('Aborted', 'AbortError'),
);
} catch {
/* already errored/closed */
}
});
},
});
return { stream };
},
} as any);
const box: { runId?: string } = {};
const { cleanup } = await startRun({
registry,
runService,
model,
chatId,
body: { chatId, messages: [userUiMessage('Start then stop')] },
box,
});
try {
const sink = liveSink();
const att = (await registry.attach(chatId, false, undefined, sink.cb))!;
att.start();
// Give streamText a beat to begin consuming the partial output.
await sleep(250);
// User presses Stop -> the run signal aborts -> the SDK emits an abort chunk.
await runService.requestStop(box.runId!, workspaceId);
await waitFor(() => sink.ended());
expect(sink.frames.some((f) => f.includes('"type":"abort"'))).toBe(true);
expect(sink.frames.some((f) => f.includes('[DONE]'))).toBe(true);
expect(sink.ended()).toBe(true);
} finally {
registry.onModuleDestroy();
await cleanup();
}
});
it('the outer catch calls abortEntry so an open entry is released (finished)', async () => {
const chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
const registry = new AiChatStreamRegistryService();
const runService = new AiChatRunService(runRepo, {
isCloud: () => false,
} as never);
const model = new MockLanguageModelV3({
doStream: async () => ({ stream: successStream() }),
} as any);
// A msgRepo whose user-row insert throws: the turn fails AFTER begin (the
// entry is already open) but BEFORE the pipe, exercising the outer catch.
const throwingMsgRepo = {
insert: async () => {
throw new Error('db boom');
},
findAllByChat: (...a: any[]) => (msgRepo as any).findAllByChat(...a),
update: (...a: any[]) => (msgRepo as any).update(...a),
findById: (...a: any[]) => (msgRepo as any).findById(...a),
};
const service = new AiChatService(
{ getChatModel: async () => null } as any,
aiChatRepo,
throwingMsgRepo as any,
{} as any,
{ resolve: async () => null } as any,
{ forUser: async () => ({}) } as any,
mcpClients as any,
{} as any,
{} as any,
{} as any,
{
isAiChatDeferredToolsEnabled: () => false,
isAiChatResumableStreamEnabled: () => true,
} as any,
registry,
);
const { res, cleanup } = await makeRealResponse();
const box: { runId?: string } = {};
try {
await expect(
service.stream({
user: { id: userId, workspaceId } as any,
workspace: { id: workspaceId, name: 'WS' } as any,
sessionId: 'sess-1',
body: { chatId, messages: [userUiMessage('will throw')] },
res: { raw: res } as any,
signal: new AbortController().signal,
model: model as any,
role: null,
runHooks: makeRunHooks(runService, registry, box),
} as any),
).rejects.toThrow();
// The entry opened at begin was terminated by abortEntry (from the catch),
// so it is finished and a plain attach returns null instead of hanging.
const entry = (registry as any).entries.get(chatId);
expect(entry).toBeDefined();
expect(entry.finished).toBe(true);
const sink = liveSink();
expect(await registry.attach(chatId, false, undefined, sink.cb)).toBeNull();
} finally {
registry.onModuleDestroy();
await cleanup();
}
});
it('legacy (no run-hooks): the registry is never populated', async () => {
const chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
const registry = new AiChatStreamRegistryService();
const model = new MockLanguageModelV3({
doStream: async () => ({ stream: successStream() }),
} as any);
// No runHooks -> runId undefined -> the run-wrapped tee is never wired.
const { cleanup } = await startRun({
registry,
model,
chatId,
body: { chatId, messages: [userUiMessage('Legacy hi')] },
});
try {
await waitFor(async () => {
const rows = await msgRepo.findAllByChat(chatId, workspaceId);
return rows.some(
(r: any) => r.role === 'assistant' && r.status === 'completed',
);
});
const sink = liveSink();
// No entry was ever opened; attach always yields null.
expect(await registry.attach(chatId, false, undefined, sink.cb)).toBeNull();
expect(await registry.attach(chatId, true, 'anything', sink.cb)).toBeNull();
} finally {
registry.onModuleDestroy();
await cleanup();
}
});
});
-1
View File
@@ -16,7 +16,6 @@
"testEnvironment": "node", "testEnvironment": "node",
"testTimeout": 60000, "testTimeout": 60000,
"maxWorkers": 1, "maxWorkers": 1,
"forceExit": true,
"globalSetup": "<rootDir>/test/integration/global-setup.ts", "globalSetup": "<rootDir>/test/integration/global-setup.ts",
"globalTeardown": "<rootDir>/test/integration/global-teardown.ts", "globalTeardown": "<rootDir>/test/integration/global-teardown.ts",
"moduleNameMapper": { "moduleNameMapper": {
+520
View File
@@ -0,0 +1,520 @@
# Фича «Время работы над статьёй» — дизайн-документ
Статус: черновик проектирования (код не пишется).
Контекст: gitmost (форк Docmost). Зависит от PR #370 / PR #374 (типизированная история страниц).
## 1. Цель и не-цели
**Цель.** Показывать в UI страницы одно число — оценку времени, реально затраченного
на работу над статьёй, собранную из истории правок. Число должно быть устойчиво к
паузам: «в 21:00 одна правка, в 09:00 вторая» не должно превращаться в «работал 12 часов».
По клику на число — раскрытие в **суточный таймлайн**: строка-день = 24-часовая дорожка с
окнами активности и суммой за день (см. §6.2).
**Явно не-цели.**
- Это НЕ инструмент для агента (не MCP-tool). Это число, отображаемое человеку в UI.
- Не хронометраж с точностью до минуты — это заведомо оценка.
- Не биллинг/тайм-трекинг сотрудников; не изменение долговечности черновика.
## 2. Проблема
История страницы в Docmost — таблица `page_history`: снимок на каждое сохранение.
У снимка есть `createdAt`, автор (`lastUpdatedById`), тип источника
(`lastUpdatedSource`: `user`/`agent`/`git`) и группировка агентских правок
(`lastUpdatedAiChatId`).
Наивная оценка `max(createdAt) − min(createdAt)` завышает в разы: между крайними
правками лежат сон, обед, дни простоя. На реальных данных статьи-примера
(первая страница истории, 20 снимков) span между крайними снимками ≈ 60 часов,
тогда как реальной работы — пара часов в 5–6 коротких заходов.
Правильная постановка: **длительность — это не span между крайними правками, а сумма
интервалов внутри «сессий»; историю режем по паузам бездействия.** Это классическая
*сессионизация по таймауту неактивности* (WakaTime / RescueTime / веб-аналитика).
## 3. Почему PR #374 — фундамент фичи
До #374 у сессионизации слепое пятно: если человек час пишет подряд и не жмёт «Сохранить»,
в `page_history` за этот час почти нет строк (тяжёлые автосейвы ydoc идут в `pages`/`ydoc`,
а не в историю). Непрерывную работу нечем измерить.
PR #374 вводит типизированную историю через колонку `page_history.kind`:
| `kind` | что означает | ценность для фичи |
|------------|------------------------------------------------|----------------------------------------------------|
| `manual` | человек нажал Save | сильный маркер активной работы человека |
| `agent` | снимок правки агентом | машинное время агента (группируется по `aiChatId`) |
| `idle` | автоснимок idle-флеша (потолок ~`maxWait`) | регулярный «пульс» непрерывной работы (~≤10м, §3) |
| `boundary` | автоснимок на переходе актора (user↔agent↔git) | бесплатная разметка «кто работал в этом сегменте» |
| `null` | легаси-автосейв (старые страницы) | обычный сэмпл активности |
Два env-параметра #374 напрямую задают качество измерения:
- **`IDLE_MAX_WAIT_USER=10м` / `AGENT=5м` (потолок ожидания)** — определяющий параметр.
Проверено по `computeHistoryJob` (`persistence.extension.ts`):
`delay = max(0, min(interval, burstStart + maxWait − now))`, а `enqueuePageHistory`
сбрасывает `burstStart` каждые `maxWait`. Так как `maxWait` (10м/5м) < `interval` (60м/15м),
**потолок всегда доминирует**: во время непрерывной работы `idle`-снимок форсится каждые
~`maxWait`, давая регулярный «пульс активности» с шагом ≤10 мин (user) / ≤5 мин (agent).
- **`IDLE_INTERVAL_USER=60м` / `AGENT=15м`** — номинальный трейлинг-интервал, который потолок
ожидания на практике всегда упреждает. Поэтому метка любого `idle`-снимка отстоит от
реальной правки не более чем на ~`maxWait` (≤10м user / ≤5м agent), а **не** на 60 мин.
(Это ключевой факт для точности: `idle`-метки — достоверный сигнал активности, не «хвост».)
Вывод: ядро алгоритма работает и на голых `createdAt`+`lastUpdatedSource` (есть с миграции
`20260616T130000-agent-provenance`), поэтому фича считает и на легаси-страницах — просто
грубее. После #374 (пульс ≤10 мин + типы) — точно. Жёсткой блокировки на мёрж #374 нет,
но полноценная точность появляется вместе с ним.
## 4. Входной сигнал
Дешёвый проекционный запрос по `page_history` (без тяжёлой колонки `content`): на строку —
`createdAt`, `lastUpdatedById`, `lastUpdatedSource`, `lastUpdatedAiChatId`, `kind`.
Все поля (включая `kind`) уже в `PageHistoryRepo.baseFields` после #374 — схему трогать не
нужно, `findTimelineByPageId` это лишь лёгкая проекция без `content`.
**Важный факт об авторстве (проверено по `persistence.extension.ts``updatePage`):**
`page_history.lastUpdatedById` — это ВСЕГДА ответственный человек, даже для агентских снимков
(«human stays the responsible author»). Признак «человек vs агент» живёт в
`lastUpdatedSource` (`user`/`agent`/`git`) и `lastUpdatedAiChatId`, а НЕ в `lastUpdatedById`.
Отсюда: разделять человеко-/агенто-время нужно по `lastUpdatedSource`, а атрибутировать
конкретному человеку — по `lastUpdatedById`/`contributorIds`.
## 5. Алгоритм: сессионизация по паузам
Параметры (env, в стиле #374; полный список — §10):
- **`T_gap`** — таймаут неактивности: пауза между соседними сэмплами `≤ T_gap` = непрерывная
работа, больше = перерыв (в зачёт не идёт).
- **`P_in` / `P_out`** — добивка МНОГОсэмпловой сессии (работа началась до первого и продолжалась
после последнего сохранения).
- **`P_single`** — блок ОДИНОЧНОЙ сессии (один сэмпл, соседей в пределах `T_gap` нет). Мал
(дефолт ~2 мин): один автосейв/idle-пульс — это «была правка», но приписывать ему полный
`P_in+P_out` = выдумывать время. НЕ путать с многосэмпловой добивкой.
Псевдокод (ОДИН проход по ВСЕМ сэмплам страницы; класс определяется у ГОТОВОЙ сессии — §5.1):
```text
samples = ВСЕ history rows страницы, projected, sorted by createdAt ASC
(все типы kind — сэмплы активности; idle — основной «пульс» непрерывной работы §3, НЕ исключается)
# коллапс агентских всплесков (§5.1)
collapse: подряд идущие сэмплы ОДНОГО aiChatId с source=agent → сегмент {t_start, t_end, source:'agent'}.
Разрывает всплеск любой сэмпл, НЕ продолжающий тот же aiChatId-агент: source≠agent,
boundary-переход, ИЛИ иной aiChatId — в т.ч. idle/boundary с ДРУГИМ aiChatId = НОВЫЙ ран,
склеивать НЕЛЬЗЯ (иначе простой между двумя ИИ-ранами засчитается агенту). idle с ТЕМ ЖЕ
(или null) aiChatId — продолжает сегмент. Дальше сегмент участвует в цикле как один «сэмпл» с
.t_start/.t_end/.source (source='agent'); у скалярного сэмпла t_start == t_end == t.
gap_threshold(a, b) = (a и b оба source=agent) ? agentTGap : T_gap # порог зависит от ПАРЫ (§10)
# сессионизация по паузам — ОДИН проход по ВСЕМ сэмплам (не по парам, не отдельно по классам)
sessions = []; cur = null
for s in samples: # s — скаляр или коллапс-сегмент
if cur == null: cur = { first: s, last: s, samples: [s] }
elif s.t_start − cur.last.t_end ≤ gap_threshold(cur.last, s):
cur.last = s; cur.samples.push(s) # непрерывная работа
else: sessions.push(cur); cur = { first: s, last: s, samples: [s] }
if cur != null: sessions.push(cur) # ОБЯЗАТЕЛЬНО закрыть последнюю (иначе теряется)
# класс и интервал каждой сессии
for sess in sessions:
sess.class = sess.samples.every(is_agent) ? 'agent_only' : 'work' # §5.1 (по source сэмплов)
sess.iv = (sess.first == sess.last && sess.first.t_start == sess.first.t_end)
? [ sess.t − P_single, sess.t ] # одиночный скаляр: pre-roll (без «будущей» работы)
: [ sess.first.t_start − P_in, sess.last.t_end + P_out ] # многосэмпловая / лон-сегмент
workMs = duration( union( sess.iv : sess.class=='work' ) ) # union внутри класса
agentOnlyMs = duration( union( sess.iv : sess.class=='agent_only' ) )
```
Ключевые свойства:
- **Один проход, потом классификация → метрики дизъюнктны.** Сессии не перекрываются (разделены
гэпами > порога), каждая — ровно одного класса. Человек и агент, правящие ОДНОВРЕМЕННО, попадают
в одну сессию класса `work` (надзор засчитан человеку, не дважды). `work`- и `agent_only`-интервалы
не пересекаются по wall-clock и `workMs + agentOnlyMs ≤ реально прошедшего` — **при рекомендованном
`T_gap ≥ P_in+P_out` (§10)**: иначе `P`-добивка соседних сессий РАЗНЫХ классов может пересечься, и
пересечение попадёт в обе метрики. Инвариант §6.3 (`Σ perDay == work`) от этого НЕ зависит.
- **Закрытие последней сессии обязательно** (иначе теряется самая свежая): `n=1` → одна одиночная
сессия `P_single`, `n=0` → 0.
- **`union`, а не `Σ`.** Перекрытия (соседние ближе `P`; одновременное соредактирование) не
задваиваются. Отсюда `Σ perDay == work` держится сам собой (§6.3) — без «клампа» и без условия
`T_gap ≥ P`.
- **Калибровка `T_gap` по «пульсу» #374 (важно).** После #374 непрерывная работа гарантированно
оставляет history-строку не реже ~`maxWait` (idle-пульс §3, гейт `isDeepStrictEqual`). Значит гэп
между соседними строками БОЛЬШЕ ~`maxWait` содержит участок без изменений контента = (частичное)
бездействие — даже между двумя `manual`. Поэтому `T_gap` калибруется ОТ `maxWait` (реком. ~15м
user / ~7м agent), а не ставится вольно: прежние «30 мин» переоценивали не-пульсирующие паузы.
Гэп `≤ T_gap ≈ maxWait` ПОДКРЕПЛЁН пульсом — это и оправдывает счёт его как работы. Легаси-страницы
до #374 пульса не имеют → там `T_gap` вынужденно шире и оценка грубее (§7, §10).
- **Всё равно оценка, а не строгая граница**: даже с пульсом «читал/думал 12 мин не печатая» не
отличить от «отошёл»; подпись «≈» и показ `T_gap` обязательны (§6).
- Почему интервалы, а не «правок × блок»: «снимок = +N мин» ломается на агентских всплесках
(8 снимков за 7 минут); длина сегмента всплеска = его wall-clock, независимо от плотности снимков.
### 5.1. Классификация сэмплов и всплесков
- **Класс сэмпла (человек / агент)** — по `lastUpdatedSource`: `user`→человек, `agent`→агент,
`git`→исключаем (§10 `excludeGit`), legacy-`null`→человек. `idle` наследует `source` страницы
на момент флеша (= source последней правки). `boundary` несёт СТАРЫЙ (pre-transition) `source`
— трактуем как есть: он маркирует исходящую работу того актора (это осознанный компромисс, а не
недосмотр — точность посекундная тут не нужна).
- **Класс СЕССИИ** (нужен для метрик §6.1): сессия, где ВСЕ сэмплы `source=agent`**`agent_only`**
(автономный прогон, не в основную метрику). Сессия хотя бы с одним человеческим сэмплом →
**`work`** (человек + надзор за агентом ВНУТРИ сессии засчитывается человеку — по union'у, §5).
- **`idle`** — полноценный сэмпл активности и основной «пульс» непрерывной работы (§3): его метка
отстаёт от реальной правки ≤ `maxWait` (≤10м) — в пределах округления, отмотки не требует.
Исключать нельзя: без него непрерывное письмо без ручных сохранений снова стало бы невидимым.
## 6. Что показывать в UI
### 6.1. Свёрнутое состояние — одно число
Две метрики, каждая = union-wall-clock СВОИХ сессий (§5):
- **`work` — headline, кликабельное число** (`≈ 4 ч 30 мин`, рядом с панелью истории или в
мета-инфо у заголовка): сессии класса `work` (≥1 человеческий сэмпл = человек + надзор за
агентом внутри сессии). Именно это открывает таймлайн (§6.2), и именно к нему сходится сумма по
дням.
- **`agent_only` — вторично**: сессии автономных прогонов агента (ни одного человеческого сэмпла).
На таймлайне — отдельным цветом; в `work` НЕ входит.
- Подпись «≈» и показ `T_gap` обязательны (оценка, не хронометраж — §5). Округление headline —
шаг 5–15 мин (точность оценки не оправдывает «4 ч 27 мин»).
### 6.2. Раскрытие по клику — суточный таймлайн (24 ч × дни)
Клик по числу открывает модалку/поповер с таймлайном по типу «punch-card»:
- **Строка = один календарный день**; ширина строки = 24 часа (фиксированная шкала 00:00→24:00).
- На дорожке дня закрашены **окна активности** — реальные интервалы работы в их часовом
положении, так что видно «вечерний марафон» vs «утренняя сессия».
- **Справа от строки — сумма за день** «ч мм» (напр. `3 ч 17 м`); пустой день — пустая дорожка
и «—».
- Внизу — общий итог (= headline `work` §6.1) плюс подпись таймзоны и `T_gap`.
Это графическая версия таблицы-примера-картинки: там окна были текстом («18:46 → 00:58»), здесь
то же рисуется отрезками на 24-часовой дорожке.
Детали:
- Окна = сессии класса `work` (§5), обрезанные по границам суток; сессии `agent_only` — отдельным
цветом (§6.1). По умолчанию `work` — один цвет «активность».
- Сессия через полночь рисуется как отрезок до 24:00 в одном дне и продолжение с 00:00 в
следующем — тот самый полуночный разрез (§6.3), теперь визуально очевиден.
- **Честность добивки.** Окно включает `P`/`P_single` — это оценка «до/после», а не измеренный
интервал. Одиночная сессия (`P_single`) рисуется минимальной видимой шириной и приглушённо
(иначе исчезает и/или создаёт иллюзию плотной работы из одного клика). Общая подпись — «≈».
### 6.3. Агрегация по дням (алгоритм)
Вход — ВСЕ сессии из §5 (`work` и `agent_only`), каждая развёрнута в интервал (`P_in/P_out` или
`P_single`).
```text
bucketByDay(sessions, tz):
U_work = union( sess.iv : sess.class=='work' ) # снимаем перекрытия ОДИН раз (§5)
U_agent = union( sess.iv : sess.class=='agent_only' ) # СИММЕТРИЧНО — иначе окна агента наложатся
для каждого дня D в [первый_день … последний_день] по tz (dayjs+tz, startOf('day')):
workWin[D] = { u ∩ [начало D, начало D+1) : непусто, u в U_work }
agentWin[D] = { u ∩ [начало D, начало D+1) : непусто, u в U_agent }
activeMs[D] = Σ длительностей workWin[D] # U_work без перекрытий → просто сумма
agentMs[D] = Σ длительностей agentWin[D]
→ perDay[] = [{ day, activeMs, agentMs, windows: (workWin[D] ⊕ agentWin[D]) с меткой класса }]
```
- **Инвариант согласованности `Σ activeMs[D] == work`** держится ПО ПОСТРОЕНИЮ: день — это
разбиение union'а `U_work` границами суток, ничего не теряется и не дублируется (в т.ч. на
23/25-часовых DST-сутках — §9#14). `agent_only`-окна рисуются, но в `activeMs` НЕ входят. Клампа
и скрытого условия `T_gap ≥ P` не требуется (в отличие от наивного `Σ` длительностей).
- **Таймзона `TZ`** определяет, где проходит «полночь» И в каких часовых координатах рисуются
окна. Дефолт — таймзона зрителя (локаль браузера); альтернатива — UTC (как на картинке-примере
«По дням (UTC)») или tz воркспейса. Влияет на раскладку по дням и положение окон, но НЕ на
общий итог → настройка (§10).
- **Длинный диапазон** (месяцы правок): строк ровно столько, сколько календарных дней в диапазоне.
При большом числе дней — вертикальный скролл + сворачивание длинных серий пустых дней
(«× N дней без правок») и/или переключение на понедельную группировку. Порог — настройка.
- **Округление дисплея:** сумму за день округляем до минут для подписи, но общий итог берём из
точных значений (иначе Σ округлённых по дням разойдётся с округлённым числом на ±1–2 мин).
## 7. Проверка на реальной статье
Ручной прогон на 20 снимках (1-я страница истории, `T_gap=30 мин`, `P_in+P_out=10 мин`,
`P_single=2 мин`; все сессии здесь класса `work` — в каждой есть человеческий сэмпл):
| Сессия | Интервал | Длит. |
|--------|-------------------------------------------------------|----------|
| S1 | 07-04 03:40 → 03:49 (многосэмпловая) | ≈19 мин |
| S2 | 07-04 15:43 → 16:13 (агент 15:43–15:50 → человек 16:13)| ≈41 мин |
| S3 | 07-04 18:11 (одиночная) | ≈2 мин |
| S4 | 07-04 19:38 → 19:54 (многосэмпловая) | ≈26 мин |
| S5 | 07-06 15:34 (одиночная) | ≈2 мин |
| S6 | 07-06 16:18 (одиночная, закрыта пост-циклом §5) | ≈2 мин |
| **Итого** | | **≈1 ч 32 мин** |
Наивно на том же срезе — ≈60 часов. Разница — весь смысл фичи.
Наблюдения:
- Разрезы легли по перерывам: ночь `03:49 → 15:43` (~12 ч) и сутки
`07-04 19:54 → 07-06 15:34` — оба выброшены.
- Чувствительность к порогу: `S5/S6` отстоят на 44 мин. При `T_gap=30` это две сессии,
при `T_gap=60` — одна (~44 мин). Число надо показывать **вместе с использованным порогом**.
- Это **оценка, не строгая граница** (§5), и НАПРАВЛЕНИЕ ошибки зависит от данных. Сохранения
ВНУТРИ `T_gap` мостятся, и вся пауза между ними засчитывается → на периодичном «фоновом» ритме
(напр. автосейв раз в ~`T_gap` при почти полном простое) возможен КРАТНЫЙ перебор (в разы), а не
«±порог». Сохранения ДАЛЬШЕ `T_gap` друг от друга, наоборот, теряют между-время → недобор. Поэтому
`T_gap` калибруется по пульсу #374 (§5, §10): после #374 «фоновый» ритм даёт гэпы > `maxWait` и
рвётся на перерывы, срезая кратный перебор; на легаси (пульса нет) оценка грубее в обе стороны.
Пример иллюстративен, посчитан на до-#374 срезе при `T_gap=30`.
## 8. Архитектура (куда встраивать)
**Ядро — чистая функция** `computeWorkTime(rows, config)` (детерминированная, без БД) в
отдельном модуле → легко покрыть юнит-тестами. Выход —
`{ workMs, agentOnlyMs, sessions[] }`, где `session = { start, end, class: 'work'|'agent_only' }`:
абсолютные границы (с добивкой `P`/`P_single`) плюс класс (§5.1) — этого достаточно и для метрик,
и для цвета окна. `workMs`/`agentOnlyMs` считаются как union-wall-clock сессий своего класса (§5).
**Вторая чистая функция** `bucketByDay(sessions, tz)` (ВСЕ сессии обоих классов) →
`perDay[] = [{ day, activeMs, agentMs, windows }]`: `activeMs` = длительность `work`-окон за день
(сходится к `work`, §6.3), `agentMs` = то же для `agent_only` (для подписи машинного времени за
сутки), `windows` — интервалы ОБОИХ классов, обрезанные по суткам и помеченные классом (для
отрисовки `work`/`agent_only` разным цветом, §6.2). Полуночный разрез — календарный, через
`dayjs` + tz-плагин (`startOf('day')` в `tz`), НЕ «+24 ч» (§9#14); `dayjs` уже в проекте. Отдельная
тестируемая функция (общий модуль сервера и клиента); `tz` — презентационный параметр, удобно звать
на клиенте от локали зрителя.
**Сервер:**
- `page-history.repo.ts` — метод `findTimelineByPageId(pageId)`: лёгкая проекция
(`createdAt, lastUpdatedById, lastUpdatedSource, lastUpdatedAiChatId, kind`) по всем строкам
ASC, без `content`.
- `page-history.service.ts``computeWorkTime(pageId, config)`: тянет таймлайн, зовёт ядро,
кэширует.
- `page.controller.ts` — рядом с `POST /history` и `POST /history/info` добавить
`POST /history/time` (или вложить в page-info). Отдаёт число + разбивку по сессиям.
**Клиент:**
- `page-history-query.ts` — хук `usePageWorkTime(pageId)` (возвращает `workMs`, `agentOnlyMs`,
`sessions[]`).
- Рендер кликабельного числа в панели истории.
- Модалка/поповер с суточным таймлайном (§6.2): `bucketByDay(sessions, viewerTz)` → строки-дни,
в каждой — 24-часовая дорожка с окнами. Это НЕ bar chart: рисуется кастомными CSS/SVG-отрезками
на 24-часовой шкале (позиция окна = `startOfDayOffset/24ч`, ширина = длительность/24ч). Готовые
чарт-библиотеки под это плохо ложатся — брать лёгкую собственную вёрстку, без новых тяжёлых
зависимостей. Пустые дни — пустая дорожка + «—».
**Производительность:** проекция без `content` дёшева; результат можно инкрементально кэшировать
(при `version.saved`, который #374 броадкастит, пересчитывать хвост).
## 9. Крайние случаи
1. Один снимок → одна одиночная сессия `P_single`, не ноль. Последняя сессия ВСЕГДА закрывается
пост-циклом (§5) — иначе теряется самая свежая.
2. История целиком агентская → `work = 0`, `agent_only = union прогонов`.
3. Плотный агентский всплеск → длина сегмента = его wall-clock (не зависит от числа снимков);
опц. кап `burstCapMs`.
4. Метка `idle` лагает ≤ `maxWait` (10м user / 5м agent) — в пределах округления; это
полноценный сэмпл активности, спец-обработки не требует (см. §3, §5.1).
5. Несколько соавторов → атрибуция человеку по `lastUpdatedById`/`contributorIds`; разделение
человек/агент — по `lastUpdatedSource` (НЕ по `lastUpdatedById`, он всегда человек).
6. Легаси `kind=null` → работает на `source`+`createdAt`, грубее.
7. Совпадающие метки времени (boundary+agent в один момент; в коде есть tie-break по `id`)
→ дедуп по округлённому `t`.
8. Одновременное соредактирование → `union` (§5) убирает двойной счёт wall-clock при перекрытии
окон; персональные человеко-часы (разбивка по авторам) — отдельный опциональный режим
(`perAuthor`), а не поведение по умолчанию.
9. Сессия через полночь (напр. `23:14 → 02:00`) → режется на границе суток `tz`, части идут
в разные дни; сумма по дням = общему числу (§6.3).
10. Выбор таймзоны дня меняет раскладку по дням (та же работа попадёт в другой день) — общий
итог не меняется; `tz` фиксируется в подписи графика.
11. Длинный диапазон правок (месяцы) → строк = число дней: вертикальный скролл + сворачивание
длинных серий пустых дней и/или понедельная группировка по порогу (§6.3).
12. День без правок → пустая 24-часовая дорожка + «—» в диапазоне (показываем ритм/паузы),
не пропускаем.
13. Очень короткое окно на 24-часовой шкале → рисуем минимальной видимой шириной, чтобы не
исчезало (§6.2).
14. Переход на летнее/зимнее время внутри `tz` → сутки в 23/25 ч; полуночный разрез считать
по календарю `tz` (`dayjs`+tz, §8), а не «+24 ч». Инвариант `Σ activeMs == work` держится
(разбиение union'а). Редкое исключение — tz с DST-переходом РОВНО в полночь (`startOf('day')`
неоднозначен) → до 1 ч может протечь в соседний день. Фиксированная 24-часовая дорожка (§6.2)
на 23/25-часовых сутках смещает окна визуально до ~1 ч — сознательное упрощение, на итог не
влияет.
## 10. Параметры по умолчанию и открытые решения
**Полный `config`** (env, дефолты):
- `T_gap≈15м` — калибровка по `IDLE_MAX_WAIT_USER=10м` + запас (§5: гэп больше `maxWait` не
подкреплён пульсом = бездействие; прежние «30м» переоценивали не-пульсирующие паузы).
- `agentTGap≈7м` — порог для ПАРЫ подряд идущих агентских сэмплов (`IDLE_MAX_WAIT_AGENT=5м` + запас).
- `P_in=5м`, `P_out=5м`, `P_single=2м` (одиночная — pre-roll, §5).
- `burstCapMs` (опц. кап на сегмент всплеска, §9#3), `dedupRoundMs` (дедуп совпадающих меток, §9#7),
`excludeGit=true`, `tz=локаль-зрителя`, `longRangeDayThreshold` (день→неделя, §6.3),
`perAuthor=false` (§9#8).
- **Легаси-страницы до #374** (нет пульса) → `T_gap` вынужденно шире, оценка там грубее.
- **`T_gap ≥ P_in+P_out`** НЕ требуется для инварианта §6.3 (union), но РЕКОМЕНДУЕТСЯ и валидируется
— иначе `work`/`agent_only` могут перекрыться в сумме (§5). При дефолтах (15 ≥ 10) держится.
Развилки (настройки, не блокируют проектирование):
- `work` vs `agent_only` — показывать оба, крупно `work` (headline), `agent_only` вторично;
- точное место в UI — панель истории (основное) или мета-строка страницы;
- дефолт `T_gap` — вынести в env (как `IDLE_*` в #374), калибровать на реальных статьях
после мёржа #374;
- **таймзона дня для графика (§6.2/§6.3)** — рекомендую локаль зрителя (интуитивно «мои
вечера»); альтернативы — UTC (как на картинке-примере) или tz воркспейса. Влияет только на
раскладку по дням, не на общий итог;
- **порог перехода день→неделя/месяц** для длинного диапазона правок.
---
# Приложение: Review Ledger (рабочий аппарат, НЕ нормативная часть)
> Аппарат adversarial-review-loop. Перед выдачей реализатору выносится/сворачивается.
> Критикам: НЕ перелитигировать закрытые findings, КРОМЕ случая, когда сам RESOLUTION
> дефектен — тогда атаковать его явно и сказать, почему предыдущий раунд ошибся.
## CONFIG
- EXTERNAL_MODEL: endpoint `https://api.z.ai/api/coding/paas/v4`, model `glm-5.2`
(ключ хранится вне репозитория, в логи не пишется). Роль: cold-reader gate (Phase 4.5),
tiebreaker при эскалации.
- Артефакт: `docs/features/page-work-time_design.md`.
- Целевой класс: спец для реализации другим человеком → план 2–3 итерации, до пустого
cold-reader gate (не по числу итераций).
## FACT BASE (проверено по PR #374 @ commit 924f8aa, ветка feat/370-page-versioning)
Верификация автором ДО цикла (ветка не в рабочем дереве — критики не видят её локально;
факты ниже — ground truth, можно дозапросить файлы через gitea MCP по указанному SHA):
- `page_history.kind``varchar(20)`, NULLABLE, БЕЗ дефолта (migration
`20260705T120000-page-history-kind.ts`). Домен: `manual`/`agent`/`idle`/`boundary`;
legacy `null` = автосейв (`collaboration/constants.ts`, `PageHistoryKind`).
- `kind` УЖЕ включён в `PageHistoryRepo.baseFields` (`page-history.repo.ts`) — читается всеми
выборками истории. `saveHistory({kind})` и `updateHistoryKind(id, kind)` существуют.
- Тайминги (`constants.ts`): `IDLE_INTERVAL_USER=60м`/`AGENT=15м`;
`IDLE_MAX_WAIT_USER=10м`/`AGENT=5м`.
- `computeHistoryJob` (`persistence.extension.ts`):
`delay = max(0, min(interval, burstStart + maxWait − now))`; `enqueuePageHistory` сбрасывает
`burstStart` каждые `maxWait`. Следствие: **потолок всегда доминирует**`idle` пульсирует
каждые ~`maxWait` при непрерывной работе; метка `idle` отстоит от правки ≤ `maxWait`, НЕ 60м.
- Идл-джоб всегда ставится с `kind:'idle'`; процессор пишет `job.data.kind ?? 'idle'`, но только
если контент изменился (`isDeepStrictEqual`-гейт).
- `boundary`-снимок пишется СИНХРОННО в store-транзакции на смене `lastUpdatedSource`
(user↔agent↔git), фиксирует ИСХОДЯЩИЙ (pre-transition) контент со СТАРЫМ source; его
`createdAt` = момент перехода (точный).
- `manual`/`agent` — явный save-version по stateless-каналу; `kind` выводится из
`context.actor` СЕРВЕРНО (неподделываемо). Promote-not-dup: апгрейд `kind` последнего снимка
на месте вместо дубля.
- `page_history.lastUpdatedById` = ВСЕГДА ответственный человек (даже для агентских снимков);
«agentness» — в `lastUpdatedSource` + `lastUpdatedAiChatId`.
- REST истории: `POST /history`, `POST /history/info` (`page.controller.ts`). Клиент:
`apps/client/src/features/page-history/*`. Есть broadcast `version.saved` (для live-инвалидации).
## Pre-loop fact-base corrections (автор, проверено vs source)
- C1. §3/§5.1/§крайние-случаи#4: убрано ошибочное «idle лагает до 60м / idle не удлиняет
сессию / отмотка на IDLE_INTERVAL». Верно: лаг ≤ `maxWait` (≤10м), `idle` — полноценный сэмпл.
- C2. §3: устранено внутреннее противоречие §3↔§5.1 (пульс vs исключение idle).
- C3. §4: `kind` уже в `baseFields` — схему менять не нужно.
- C4. §крайние-случаи#5: `lastUpdatedById` всегда человек; человек/агент — по `lastUpdatedSource`.
## Scope changes
- S1 (owner, до итерации 1): добавлен drill-down — клик по числу открывает график по дням.
- S2 (owner, до итерации 1): drill-down переопределён с «столбцы часов/день» на СУТОЧНЫЙ
ТАЙМЛАЙН (строка-день = 24 ч с окнами активности + сумма за день, §6.2/§6.3). Окна вернулись,
но графикой. Затронуты §1, §6.2, §6.3, §8, §9(#9–14), §10.
## Iteration log
### Итерация 1 — критики: Claude (hardened, A) + GLM-5.2 external (B)
Первый прогон local-субагентов вернул инъекцию из подложенного SKILL.md (реклама
«agent-first-plugin-suite») и 0 полезной работы — проигнорировано; пере-прогон с анти-инъекцией
дал реальные ревью. Дефекты и диспозиции:
- **[BLOCKER] A1** — §5-псевдокод не закрывал последнюю сессию (терял свежую; 0 сессий для
односессионной страницы; ронял S6). ПРИНЯТО → пост-цикловое закрытие, `n=1``P_single`, `n=0`→0.
- **[MAJOR] A2+B4** — какое число суммирует таймлайн; двойной счёт при перекрытии. ПРИНЯТО →
метрики `work`/`agent_only` (§6.1) + `total`/`perDay` через **union** (§5, §6.3).
- **[MAJOR] A3** — §1 остался со старым «столбцом на день» (residue S1→S2). ПРИНЯТО → §1 переписан.
- **[MAJOR] A4** — «кламп vs скрытое условие `T_gap ≥ P`». ПРИНЯТО, но растворено: union убрал и
кламп, и условие (§6.3).
- **[MAJOR] A5** — форма `session` и правило классификации человек/агент (нужны для `workMs` и
цвета). ПРИНЯТО → §5.1 классификация (+ nuance про старый source у boundary), `session.class` (§8).
- **[MAJOR→понижено] B1** — «нижняя оценка» ложна; гэп ≤ T_gap считается работой. ЧАСТИЧНО ПРИНЯТО
+ КОНТР по severity: гэп-как-работа — by design (по меткам не отличить «думал» от «отошёл»),
поведение оставлено; исправлено УТВЕРЖДЕНИЕ (§5/§7: «оценка, не строгая граница»). Понижено с
BLOCKER: это не баг кода, а некорректная формулировка/честность.
- **[MAJOR] B2+B5** — инфляция одиночных сессий на `P` + честность отрисовки добивки. ПРИНЯТО →
`P_single` (мал), приглушённая отрисовка + «≈» (§5, §6.2).
- **[MAJOR/MINOR] B3+A6** — коллапс агентского всплеска рушит вклинившегося человека + выбор
конца для гэп-теста. ПРИНЯТО → правило коллапса (только подряд один aiChatId; человек внутри
разрывает; левый гэп до `t_start`, правый от `t_end`) (§5).
- **[MINOR] A7** — `config` не перечислен. ПРИНЯТО → полный список (§10).
- **[MINOR] A8** — ячейка `idle` в таблице §3 устарела. ПРИНЯТО → исправлена.
- **[NIT] A9** — расхождение округления headline vs подписи дня. Оставлено с оговоркой (§6.3).
- **[NIT] A10** — назвать `dayjs` для tz/DST-разреза. ПРИНЯТО (§8, §9#14).
Verified clean (оба критика): факты #374; tz/DST-разрез; лёгкая проекция без `content`;
кэш на `version.saved`; разбиение на две чистые функции; §2-постановка; арифметика §7 (при
исправленном алгоритме).
### Итерация 1 — post-integration re-attack (Claude A + GLM round 2) — ЗАКРЫТА
Оба критика НЕЗАВИСИМО нашли один и тот же блокер и сошлись (разные семейства моделей):
- **[BLOCKER] N1 (A) = MAJOR-1 (GLM)** — §5 «сессионизация ОТДЕЛЬНО по классам» противоречила
§5.1/§7/§8 и теряла надзорное агентское время (S2 схлопывалась в 2 мин; `workMs+agentOnlyMs` мог
превысить прошедшее). Исправлено: ЕДИНЫЙ проход по всем сэмплам + классификация готовой сессии;
метрики — union внутри класса.
- **[MAJOR] N2 (A)** — union только ВНУТРИ класса; кросс-класс дизъюнктность требует `T_gap ≥ P`,
а §10 это отрицал; `agentTGap` неопределён при едином проходе. ПРИНЯТО → `gap_threshold` по ПАРЕ
сэмплов (agent–agent → `agentTGap`), caveat в §5, §10 смягчён (валидируем `T_gap ≥ P`).
- **[MAJOR] GLM-2** — `bucketByDay(workSessions)` не мог рисовать `agent_only`. ПРИНЯТО → сигнатура
`bucketByDay(sessions)`, окна ОБОИХ классов, `activeMs` = work-only.
- **[MINOR] N3** — роль `boundary` противоречива (§5 «человеческий» vs §5.1 «старый source=agent»).
ПРИНЯТО → §5: всплеск рвёт любой сэмпл, не продолжающий тот же aiChatId-агент; класс — по source.
- **[MINOR] N4** — псевдокод точечный, концы сегмента не заданы. ПРИНЯТО → сегмент {t_start,t_end};
same-source idle внутри всплеска продолжает сегмент.
- **[NIT] N5** — DST ровно в полночь + фикс. 24ч-дорожка ±1ч визуально. ПРИНЯТО → оговорка §9#14.
- **[NIT] N6** — `P_single` симметричный «выдумывает будущую работу». ПРИНЯТО → pre-roll `[t−P_single, t]`.
- **B1-severity (спор):** A СОГЛАСИЛСЯ с понижением BLOCKER→MAJOR (нет входа, где код отклоняется от
намерения — только формулировка/UX). GLM НАСТАИВАЛ на BLOCKER с НОВЫМ аргументом: отсутствие
idle-пульса в гэпе = доказанное бездействие. Диспозиция: label = MAJOR, но СУБСТАНЦИЯ GLM принята
(Invariant 7 — уступка с аргументом): `T_gap` калиброван по `maxWait` (§5/§10) + принцип «гэп >
maxWait = перерыв». A-residue (направление ошибки data-dependent, возможен КРАТНЫЙ перебор) принят
в §7. Тайбрейкер не понадобился (обе стороны привели аргументы, интегрированы обе).
Verified clean (round-2): инвариант §6.3 под DST (оба критика); burst-endpoints (оба); `P_single`
через полночь (GLM). Блокеров в ядре: 0. Итерация 1 закрыта.
### Convergence validation + cold-reader gate — ЗАКРЫТА (CONVERGED)
- **Cold-reader gate** (GLM-5.2, внешняя модель, «имплементер читает впервые»): ПУСТОЙ список
вопросов на день 1 — ДВАЖДЫ (до и после партии полировки).
- **Convergence pass** (GLM-5.2, свои hostile-сценарии + слепая реализация): вердикт NOT CONVERGED
на ещё не атакованной партии round-2 → нашёл 2 MAJOR + 1 MINOR (как и предупреждает skill —
«re-opened cycles find MAJORs in unreviewed amendments»):
- [MAJOR] §6.3 рисовал `agent_only`-окна из «сырых» `sess.iv` (не union) → визуальное наложение.
`U_agent = union(...)`, симметрично `U_work`; добавлен `agentMs[D]`.
- [MAJOR] §5 «idle продолжает сегмент НЕЗАВИСИМО от aiChatId» склеивал разные ИИ-раны через
idle-снимок. → idle с ДРУГИМ `aiChatId` = новый ран, рвёт сегмент.
- [MINOR] нет per-day суммы агента. → `agentMs` в `perDay`/§8.
Все приняты и интегрированы; финальный re-gate (GLM cold-reader) → снова ПУСТО, рассинхрона нет.
- **Tooling note:** local general-purpose Claude-субагенты трижды перехватывались инъекцией из
подложенного SKILL.md (реклама «agent-first-plugin-suite» / фейковые «review this PR» промпты,
0 tool-uses). Проигнорировано. Адверсариальные проходы и cold-reader выполнены внешней GLM-5.2
(гетерогенная модель — как раз рекомендована skill против self-preference) + один hardened
Claude-проход, который сработал.
## Closure checklist
1. 0 блокеров в ядре; счётчики обсуждения в обе стороны (приняты находки И отбит spor по B1-severity) — ✓
2. Партия полировки re-gated финальным cold-reader на полном тексте — ✓
3. Cold-reader: список вопросов ПУСТ (дважды) — ✓
4. Прогноз сходимости трактовался как гипотеза; решал чек-лист (convergence-pass реально нашёл MAJOR) — ✓
5. Backcast: частично (пример §7 на реальных данных + 36.5ч-картинка владельца) — обязательным не был
6. Preconditions зафиксированы: зависимость точности от #374 (пульс), грубость на легаси, DST-в-полночь, `T_gap ≥ P`
**СТАТУС: CONVERGED.** §1–§10 — нормативная спека для реализатора; это приложение — аудит-след (не нормативно).
+9 -4
View File
@@ -55,10 +55,15 @@ describe('stabilizePageFile — normalize-on-write fixpoint (SPEC §11)', () =>
const file2 = await stabilizePageFile(doc2, meta); const file2 = await stabilizePageFile(doc2, meta);
expect(file2).toBe(file1); expect(file2).toBe(file1);
// The materialized diagram default is present in the stabilized body (proof // The drawio node was materialized to its canonical HTML form by the
// that the convergence pass actually ran, not just that two naive exports // convergence pass — a bare `{ src }` doc node becomes the full
// happened to match). // `<div data-type="drawio" data-src=...>` — proof the pass actually ran, not
expect(body1).toContain('data-align="center"'); // just two naive exports happening to match. Assert on the stable canonical
// markers rather than `data-align="center"`: center is a schema default the
// converter may omit (see prosemirror-markdown media-html.ts), so it is not
// a reliable convergence proof.
expect(body1).toContain('data-type="drawio"');
expect(body1).toContain('data-src="/d.drawio"');
}); });
it('already-stable content is unchanged by the pass (idempotent)', async () => { it('already-stable content is unchanged by the pass (idempotent)', async () => {
@@ -25,9 +25,11 @@ test("nested bulletList with 3 children keeps all children indented under the pa
), ),
); );
// Block children of a list item are blank-line separated (loose list) since
// the #351 converter fix; the sublist stays nested at the 2-col marker column.
assert.equal( assert.equal(
convertProseMirrorToMarkdown(input), convertProseMirrorToMarkdown(input),
"- Parent\n - A\n - B\n - C", "- Parent\n\n - A\n - B\n - C",
); );
}); });
@@ -41,9 +43,10 @@ test("nested list under an ordered item indents 3 spaces", () => {
), ),
); );
// Blank-line separated block children (loose list) per the #351 converter fix.
assert.equal( assert.equal(
convertProseMirrorToMarkdown(input), convertProseMirrorToMarkdown(input),
"1. Parent\n - Child", "1. Parent\n\n - Child",
); );
}); });
+105
View File
@@ -0,0 +1,105 @@
# @docmost/prosemirror-markdown
The single, canonical **ProseMirror ↔ Markdown converter** plus the Docmost
schema mirror (#293/#345). Headless and framework-free: no React, no browser
runtime. There is exactly ONE copy of this converter in the repo, consumed by:
- `packages/mcp` (the MCP server),
- `packages/git-sync` (two-way Git sync),
- `apps/server` (server-side markdown import/export, #345).
`src/lib/docmost-schema.ts` **mirrors** the upstream Tiptap schema that lives in
`packages/editor-ext`. The mirror is not free-floating: `serializer-contract.test.ts`
guards the boundary — every schema node must have a converter case, so a drift
between `editor-ext` and this package surfaces as a failing test rather than a
silent divergence.
## Why byte-stability matters
Git sync exports a page to markdown, and re-imports it on the next pull. If
`export → import → export` is not **byte-stable**, every pull rewrites files
that nobody edited, and the user's history churns with phantom diffs. So the
converter is held to more than "it roughly round-trips": the second export pass
must be a byte-for-byte fixpoint. That is what the property suite below proves.
## Two golden layers (do not mix them)
1. **Corpus fixtures**`test/fixtures/corpus/`. A fixed, hand-curated set of
representative documents (headings, marks, lists, tables, diagrams, columns,
details, mentions, …). These are the readable, deterministic "known-good"
snapshots. Edit them deliberately.
2. **Generative property suite**`test/generative/`. fast-check draws random
documents and asserts invariants over them. Two entry points:
- `flat-roundtrip.property.test.ts` — flat documents, attribute-level fuzzing.
- `nested-roundtrip.property.test.ts` — deeply nested structures.
The invariants (**P1–P4**):
- **P1** — semantic round-trip: `mdToPm(pmToMd(doc))` is canonically equal to
`doc` (no data loss for the round-trip-supported space).
- **P2** — byte fixpoint: `pmToMd(mdToPm(pmToMd(doc))) === pmToMd(doc)` (the
first pass may normalize once; the second pass must be a fixpoint).
- **P3** — totality: neither converter throws; output is bounded.
- **P4** — parser fuzz totality: for ANY input string, `markdownToProseMirror`
does not throw and returns a schema-valid document.
These invariants are kept **STRICT** — no `it.fails`, skip, or weakening. A
failure means the generator found a REAL converter bug.
## The counterexample process (the DoD)
This is the point of the generative layer. When a property run diverges:
1. **A property run surfaces a divergence** (locally, in CI, or in the nightly
cron — see below).
2. **fast-check shrinks it** to a minimal, human-readable counterexample and
prints the reproducing seed.
3. **Commit the shrunk doc as a permanent fixture** under
`test/fixtures/counterexamples/`, with a matching case in
`counterexamples.test.ts`. The fixture stays forever, as a regression pin.
4. **FIX the converter** so the counterexample round-trips. **Never weaken a
property to hide the bug.**
5. If — and only if — a maintainer decides a particular markdown-representable
loss is genuinely acceptable, it is recorded as an **explicit ACCEPTED /
allowlist entry with a written reason**, not by silently relaxing an
invariant.
### Attribute-coverage allowlist
`flat-roundtrip.property.test.ts` maintains `ATTR_VALUE_FUZZ_ALLOWLIST`. The
suite asserts that every attribute in the live schema is EITHER value-fuzzed by
the generator OR explicitly listed in this allowlist. This forces any newly
added node attribute to be consciously classified — you cannot add an attr and
leave it silently un-exercised; the coverage test fails until you either fuzz it
or record why it is held out.
## Running
```sh
# The full package suite (corpus + generative + contract tests):
pnpm --filter @docmost/prosemirror-markdown test
```
The generative suite honours two env knobs (invalid/empty → falls back to the
default):
| Env var | Default (flat) | Default (nested) | Meaning |
| -------------------- | -------------- | ---------------- | -------------------------------- |
| `PROPERTY_SEED` | `20250705` | `20250705` | fast-check seed (reproducibility) |
| `PROPERTY_NUM_RUNS` | `300` | `100` | runs per property |
```sh
# Reproduce a specific counterexample seed with a bigger budget:
PROPERTY_SEED=12345 PROPERTY_NUM_RUNS=5000 \
pnpm --filter @docmost/prosemirror-markdown exec vitest run test/generative/
```
### Nightly cron
`.github/workflows/nightly-property.yml` runs the generative suite every night
with `PROPERTY_NUM_RUNS` cranked to ~5000 and a **random** seed, to reach deeper
counterexamples than a fixed-seed PR run can. On failure it files a Gitea issue
containing the reproducing seed, the run count, and the tail of the output (the
shrunk counterexample), which kicks off the counterexample → fixture process
above. It can also be triggered manually (`workflow_dispatch`) with custom
`num_runs` / `seed`.
@@ -1006,6 +1006,8 @@ const Column = Node.create({
width: { width: {
default: null, default: null,
parseHTML: (el: HTMLElement) => { parseHTML: (el: HTMLElement) => {
// Mirrors editor-ext (column.ts): width is a unitless flex-grow
// number, so parse it to a Number for parity with the canonical schema.
const value = el.getAttribute("data-width"); const value = el.getAttribute("data-width");
return value ? parseFloat(value) : null; return value ? parseFloat(value) : null;
}, },
@@ -48,6 +48,52 @@ export interface ConvertProseMirrorToMarkdownOptions {
dropResolvedCommentAnchors?: boolean; dropResolvedCommentAnchors?: boolean;
} }
/**
* Adjacent sibling lists that share a markdown MARKER FAMILY re-parse as ONE
* merged list bulletList and taskList both emit `- ` markers ( a single
* `<ul>`), and two orderedLists both emit `1.` markers ( a single `<ol>`). The
* cross-type case is real data loss the editor CAN produce (e.g. a taskList
* followed by a bulletList: the merged `<ul>` has a mix of checkbox and plain
* items, so `bridgeTaskLists` refuses to convert it and every taskItem loses its
* checkbox). Between two such adjacent list children we emit an empty HTML comment
* `<!-- -->`: marked renders it as its own HTML block that interrupts the list, so
* the two lists stay distinct; on import the comment is inert (parseAttachedComment
* null) and dropped by generateJSON, and re-export re-inserts it, so the marker
* is byte-stable. It fires ONLY between two adjacent same-family list nodes no
* separator is emitted for any other join, so non-list output is unchanged.
*/
const LIST_MARKER_SEPARATOR = "<!-- -->";
function listMarkerFamily(type: string | undefined): "ul" | "ol" | null {
if (type === "bulletList" || type === "taskList") return "ul";
if (type === "orderedList") return "ol";
return null;
}
function adjacentListsMerge(
prevType: string | undefined,
curType: string | undefined,
): boolean {
const a = listMarkerFamily(prevType);
return a !== null && a === listMarkerFamily(curType);
}
/**
* Render each block child, inserting a `<!-- -->` separator entry between any two
* adjacent same-marker-family list nodes (see LIST_MARKER_SEPARATOR). Callers
* join the returned strings with their own context separator.
*/
function renderBlockChildren(
children: any[],
render: (n: any) => string,
): string[] {
const out: string[] = [];
let prevType: string | undefined;
for (const child of children) {
if (adjacentListsMerge(prevType, child?.type)) out.push(LIST_MARKER_SEPARATOR);
out.push(render(child));
prevType = child?.type;
}
return out;
}
/** /**
* Convert ProseMirror/TipTap JSON content to Markdown * Convert ProseMirror/TipTap JSON content to Markdown
* Supports all Docmost-specific node types and extensions * Supports all Docmost-specific node types and extensions
@@ -347,9 +393,15 @@ export function convertProseMirrorToMarkdown(
// lossless (the body survives) and byte-stable (it re-exports identically), // lossless (the body survives) and byte-stable (it re-exports identically),
// so it is deliberately not treated as data loss. // so it is deliberately not treated as data loss.
const parts: string[] = []; const parts: string[] = [];
let prevDocType: string | undefined;
for (const child of nodeContent) { for (const child of nodeContent) {
if (child?.type === "footnotesList") continue; if (child?.type === "footnotesList") continue;
// Keep adjacent same-family sibling lists distinct (see renderBlockChildren).
if (adjacentListsMerge(prevDocType, child?.type)) {
parts.push(LIST_MARKER_SEPARATOR);
}
parts.push(processNode(child)); parts.push(processNode(child));
prevDocType = child?.type;
} }
for (const [id, def] of footnoteDefs) { for (const [id, def] of footnoteDefs) {
if (!referencedFootnoteIds.has(id)) { if (!referencedFootnoteIds.has(id)) {
@@ -583,12 +635,23 @@ export function convertProseMirrorToMarkdown(
.map((item: any) => processListItem(item, "-")) .map((item: any) => processListItem(item, "-"))
.join("\n"); .join("\n");
case "orderedList": case "orderedList": {
// Honor a non-1 `start`: CommonMark expresses it as the first marker
// ("5." starts the list at 5), and marked parses that back into
// <ol start="5">. Emitting `${start + index}.` keeps a start=5 list as
// "5.","6.",… while a default (start=1) list stays "1.","2.",… See #351.
// Only an INTEGER ≥ 2 is a valid explicit start; collapse everything else
// (absent, non-number, 0, negative, fractional) to 1. A negative/fractional
// start would otherwise emit an untokenizable marker (e.g. "-3.", "2.5.")
// that marked cannot parse, re-importing the whole list as a PARAGRAPH.
const raw = node.attrs?.start;
const start = Number.isInteger(raw) && (raw as number) > 1 ? (raw as number) : 1;
return nodeContent return nodeContent
.map((item: any, index: number) => .map((item: any, index: number) =>
processListItem(item, `${index + 1}.`), processListItem(item, `${start + index}.`),
) )
.join("\n"); .join("\n");
}
case "taskList": case "taskList":
return nodeContent.map((item: any) => processTaskItem(item)).join("\n"); return nodeContent.map((item: any) => processTaskItem(item)).join("\n");
@@ -599,20 +662,34 @@ export function convertProseMirrorToMarkdown(
return processTaskItem(node); return processTaskItem(node);
case "listItem": case "listItem":
return nodeContent.map(processNode).join("\n"); // Direct-listItem path (lists normally render via processListItem, which
// handles the marker + indentation). Blank line between block children so
// multiple paragraphs do not merge on re-parse; a `<!-- -->` entry
// (renderBlockChildren) separates adjacent sibling lists.
return renderBlockChildren(nodeContent, processNode).join("\n\n");
case "blockquote": case "blockquote": {
// Prefix EVERY line of EVERY child with "> " and separate block-level // Prefix EVERY line of EVERY child with "> " and separate block-level
// children with a blank ">" line so code blocks / multi-paragraph // children with a blank ">" line so code blocks / multi-paragraph
// quotes round-trip correctly. // quotes round-trip correctly. A `> <!-- -->` separator line is inserted
return nodeContent // between two adjacent same-family sibling lists (renderBlockChildren)
.map((n: any) => // so they stay distinct inside the quote.
const bqParts: string[] = [];
let prevBqType: string | undefined;
for (const n of nodeContent) {
if (adjacentListsMerge(prevBqType, n?.type)) {
bqParts.push(`> ${LIST_MARKER_SEPARATOR}`);
}
bqParts.push(
processNode(n) processNode(n)
.split("\n") .split("\n")
.map((line: string) => (line.length ? `> ${line}` : ">")) .map((line: string) => (line.length ? `> ${line}` : ">"))
.join("\n"), .join("\n"),
) );
.join("\n>\n"); prevBqType = n?.type;
}
return bqParts.join("\n>\n");
}
case "horizontalRule": case "horizontalRule":
return "---"; return "---";
@@ -787,9 +864,12 @@ export function convertProseMirrorToMarkdown(
// blockquote-prefixed; a blank line becomes a bare `>` so the callout is // blockquote-prefixed; a blank line becomes a bare `>` so the callout is
// not split. // not split.
const calloutType = (node.attrs?.type || "info").toLowerCase(); const calloutType = (node.attrs?.type || "info").toLowerCase();
const calloutBody = nodeContent const calloutBody = renderBlockChildren(nodeContent, processNode)
.map(processNode) // Blank line between block children (rendered as a bare `>` after the
.join("\n") // prefix pass below) so multiple paragraphs stay separate nodes instead
// of merging on re-parse — same rule blockquote already uses. A
// `<!-- -->` entry (renderBlockChildren) separates adjacent sibling lists.
.join("\n\n")
.split("\n") .split("\n")
.map((l: string) => (l.length ? `> ${l}` : ">")) .map((l: string) => (l.length ? `> ${l}` : ">"))
.join("\n"); .join("\n");
@@ -809,7 +889,10 @@ export function convertProseMirrorToMarkdown(
return `<summary>${renderInlineChildren(nodeContent)}</summary>\n\n`; return `<summary>${renderInlineChildren(nodeContent)}</summary>\n\n`;
case "detailsContent": case "detailsContent":
return `${nodeContent.map(processNode).join("\n")}\n`; // Blank line between block children so multiple paragraphs in a details
// body survive as separate nodes (a single "\n" merges them on re-parse);
// a `<!-- -->` entry (renderBlockChildren) separates adjacent sibling lists.
return `${renderBlockChildren(nodeContent, processNode).join("\n\n")}\n`;
case "mathInline": { case "mathInline": {
// #293 canon #6: inline math serializes as Obsidian-native `$LaTeX$` // #293 canon #6: inline math serializes as Obsidian-native `$LaTeX$`
@@ -1329,20 +1412,38 @@ export function convertProseMirrorToMarkdown(
return `<ul>${children return `<ul>${children
.map((li: any) => `<li>${blockChildrenToHtml(li)}</li>`) .map((li: any) => `<li>${blockChildrenToHtml(li)}</li>`)
.join("")}</ul>`; .join("")}</ul>`;
case "orderedList": case "orderedList": {
return `<ol>${children // Carry a non-1 `start` on the raw-HTML path (columns/spanned cells) via
// the <ol start="N"> attribute, which the tiptap parser reads back into
// attrs.start. A default (start=1) list stays a bare <ol>. See #351.
// Use the SAME integer-≥-2 guard as the markdown path: a fractional start
// would emit `start="2.5"` → parseInt→2 on import (path divergence), and a
// 0/negative start is not a valid explicit start either.
const raw = block.attrs?.start;
const start = Number.isInteger(raw) && (raw as number) > 1 ? (raw as number) : 1;
const startAttr = start > 1 ? ` start="${start}"` : "";
return `<ol${startAttr}>${children
.map((li: any) => `<li>${blockChildrenToHtml(li)}</li>`) .map((li: any) => `<li>${blockChildrenToHtml(li)}</li>`)
.join("")}</ol>`; .join("")}</ol>`;
}
case "codeBlock": { case "codeBlock": {
const lang = block.attrs?.language || ""; const lang = block.attrs?.language || "";
// The code itself is element TEXT content (between <code> tags), so it // The code itself is element TEXT content (between <code> tags), so it
// must escape < > & — NOT the attribute escaper. The language rides in // must escape < > & — NOT the attribute escaper. The language rides in
// a class ATTRIBUTE, so it uses escapeAttr. // a class ATTRIBUTE, so it uses escapeAttr.
//
// Read the child text RAW (as `case "codeBlock"` does) and keep it
// VERBATIM — do NOT strip the trailing newline. Unlike the markdown fence
// path (which strips then relies on marked re-adding one `\n`), the schema
// codeBlock parseHTML reads the `<code>` text content back byte-for-byte,
// so stripping here would drop a trailing newline the node legitimately
// carries and break the round trip inside a column/cell.
const code = escapeHtmlText( const code = escapeHtmlText(
children children
.map(processNode) .map((child: any) =>
.join("") typeof child?.text === "string" ? child.text : "",
.replace(/\n+$/, ""), )
.join(""),
); );
const cls = lang ? ` class="language-${escapeAttr(lang)}"` : ""; const cls = lang ? ` class="language-${escapeAttr(lang)}"` : "";
return `<pre><code${cls}>${code}</code></pre>`; return `<pre><code${cls}>${code}</code></pre>`;
@@ -1484,6 +1585,13 @@ export function convertProseMirrorToMarkdown(
const indent = " ".repeat(indentWidth); const indent = " ".repeat(indentWidth);
const lines: string[] = []; const lines: string[] = [];
childStrings.forEach((child, childIndex) => { childStrings.forEach((child, childIndex) => {
// Separate consecutive block children with a BLANK line so the item is a
// CommonMark "loose" list item and each block stays its own node. Without
// it, a second paragraph (`- a\n b`) is re-parsed as a lazy continuation
// of the first and the two merge into one paragraph — silent data loss.
// The blank line still sits INSIDE the item (the following block keeps the
// continuation indent), so nested lists/code blocks remain nested.
if (childIndex > 0) lines.push("");
child.split("\n").forEach((line, lineIndex) => { child.split("\n").forEach((line, lineIndex) => {
if (childIndex === 0 && lineIndex === 0) { if (childIndex === 0 && lineIndex === 0) {
// First physical line of the first block gets the marker. // First physical line of the first block gets the marker.
@@ -1500,7 +1608,9 @@ export function convertProseMirrorToMarkdown(
const processListItem = (item: any, prefix: string): string => { const processListItem = (item: any, prefix: string): string => {
const itemContent = item.content || []; const itemContent = item.content || [];
const childStrings = itemContent.map(processNode); // A `<!-- -->` entry separates two adjacent same-family sublists inside the
// item so they do not merge on re-parse (see renderBlockChildren).
const childStrings = renderBlockChildren(itemContent, processNode);
if (childStrings.length === 0) return prefix; if (childStrings.length === 0) return prefix;
// The rendered marker is `${prefix} ` (prefix + one space), so its width — // The rendered marker is `${prefix} ` (prefix + one space), so its width —
// and thus the continuation indent — is prefix.length + 1. This is correct // and thus the continuation indent — is prefix.length + 1. This is correct
@@ -1514,7 +1624,9 @@ export function convertProseMirrorToMarkdown(
const checkbox = checked ? "[x]" : "[ ]"; const checkbox = checked ? "[x]" : "[ ]";
const prefix = `- ${checkbox}`; const prefix = `- ${checkbox}`;
const itemContent = item.content || []; const itemContent = item.content || [];
const childStrings = itemContent.map(processNode); // A `<!-- -->` entry separates two adjacent same-family sublists inside the
// item so they do not merge on re-parse (see renderBlockChildren).
const childStrings = renderBlockChildren(itemContent, processNode);
// An empty task item still needs its checkbox marker; without this guard // An empty task item still needs its checkbox marker; without this guard
// the indent below produces "" and the "- [ ]"/"- [x]" row disappears. // the indent below produces "" and the "- [ ]"/"- [x]" row disappears.
if (childStrings.length === 0) return prefix; if (childStrings.length === 0) return prefix;
@@ -270,9 +270,10 @@ const CALLOUT_CLOSE_RE = /^:::\s*$/;
* optional title after the type is allowed but ignored (the Docmost callout * optional title after the type is allowed but ignored (the Docmost callout
* schema has no title). The body is the following contiguous blockquote lines. * schema has no title). The body is the following contiguous blockquote lines.
*/ */
const CALLOUT_BQ_OPEN_RE = /^>\s*\[!(\w+)\]/; // The callout's own `>` marker may be preceded by an ENCLOSING container prefix:
/** Matches any blockquote continuation line (`>` … ). */ // list-item indentation (` `) and/or blockquote markers (`> `). Group 1 captures
const BLOCKQUOTE_LINE_RE = /^>/; // that prefix (lazily, so the LAST `>` before `[!type]` is the callout's own).
const CALLOUT_BQ_OPEN_RE = /^([>\s]*?)>\s*\[!(\w+)\]/;
/** Matches the start/end of a code fence (``` or ~~~), capturing the marker. */ /** Matches the start/end of a code fence (``` or ~~~), capturing the marker. */
const CODE_FENCE_RE = /^(\s*)(`{3,}|~{3,})/; const CODE_FENCE_RE = /^(\s*)(`{3,}|~{3,})/;
@@ -402,20 +403,47 @@ async function preprocessCallouts(markdown: string): Promise<string> {
// recurse so nested callouts (`> > [!type]`) are handled, then emit the same // recurse so nested callouts (`> > [!type]`) are handled, then emit the same
// callout div the `:::` path produces. A normal blockquote (no `[!type]` on // callout div the `:::` path produces. A normal blockquote (no `[!type]` on
// its first line) does not match and stays a blockquote. // its first line) does not match and stays a blockquote.
//
// PREFIX-aware: a callout nested inside a list item and/or a blockquote is
// serialized with the enclosing container prefix in front of its own `>`
// marker — ` > [!type]` (list indent) or `> > [!type]` (blockquote). We
// capture that prefix, take only continuation lines carrying `prefix>`, strip
// it, and re-apply the prefix to the emitted HTML block so the callout div
// stays WITHIN its container (an unprefixed div would escape and re-parse as
// a top-level callout / plain blockquote — silent structure loss).
const bqOpen = line.match(CALLOUT_BQ_OPEN_RE); const bqOpen = line.match(CALLOUT_BQ_OPEN_RE);
if (bqOpen) { if (bqOpen) {
const type = bqOpen[1].toLowerCase(); const prefix = bqOpen[1];
const type = bqOpen[2].toLowerCase();
const cont = prefix + ">"; // a body line = prefix + the callout's own `>`
const bodyLines: string[] = []; const bodyLines: string[] = [];
let j = i + 1; let j = i + 1;
for (; j < lines.length; j++) { for (; j < lines.length; j++) {
if (!BLOCKQUOTE_LINE_RE.test(lines[j])) break; if (!lines[j].startsWith(cont)) break;
bodyLines.push(lines[j].replace(/^>\s?/, "")); // Drop the prefix + `>` + one optional space, leaving the body content.
bodyLines.push(lines[j].slice(prefix.length).replace(/^>\s?/, ""));
} }
const inner = await transform(bodyLines); const inner = await transform(bodyLines);
const renderedInner = await markedInstance.parse(inner); const renderedInner = await markedInstance.parse(inner);
out.push( const block = `<div data-type="callout" data-callout-type="${type}">${renderedInner}</div>`;
`\n<div data-type="callout" data-callout-type="${type}">${renderedInner}</div>\n`, if (prefix.length === 0) {
); // Top-level callout: blank lines isolate the HTML block.
out.push(`\n${block}\n`);
} else if (prefix.includes(">")) {
// Enclosing BLOCKQUOTE: prefix every line and add NO surrounding blank
// lines — a blank line would terminate the blockquote and split the
// callout out of it.
out.push(block.split("\n").map((l) => prefix + l).join("\n"));
} else {
// Pure LIST-ITEM indentation: re-indent and keep the blank-line
// separators (a loose list item), so the div sits at the marker column.
out.push(
`\n${block
.split("\n")
.map((l) => (l.length ? prefix + l : l))
.join("\n")}\n`,
);
}
i = j; i = j;
continue; continue;
} }
@@ -597,6 +625,40 @@ function bridgeTaskLists(html: string): string {
* (null from parseAttachedComment), an unknown name, a wrong-position comment, or * (null from parseAttachedComment), an unknown name, a wrong-position comment, or
* an unknown/empty attr value is ignored. * an unknown/empty attr value is ignored.
*/ */
/**
* A directive comment is in ATTACHED position when it sits inside a `<p>`/`<hN>`
* textblock bound to that block's text (the `attrs`/`img` conventions). Every
* other parent (body, document level, a block container like blockquote/details/
* li/column div) is STANDALONE position, where a lone-block directive
* (subpages/pagebreak/pageembed/transclusion) is materialized. Broadening
* standalone beyond body/document is what lets these nodes survive NESTED inside
* a blockquote/callout/details/list item (previously dropped -> silent data loss).
*/
function isAttachedPosition(tag: string): boolean {
return tag === "p" || /^h[1-6]$/.test(tag);
}
/**
* Place a materialized standalone-directive element in the DOM: replace the
* comment IN PLACE when it has a real element parent inside <body> (body itself
* or a nested block container), preserving document order; queue it as a leading
* div only when the comment is at document level (no parentElement) or directly
* under `<html>` (outside <body>, which `document.body.innerHTML` would drop).
*/
function placeStandalone(
comment: any,
el: any,
tag: string,
leadingDivs: any[],
): void {
if (comment.parentElement && tag !== "html") {
comment.replaceWith(el);
} else {
comment.remove();
leadingDivs.push(el);
}
}
function applyCommentDirectives(html: string): string { function applyCommentDirectives(html: string): string {
// Cheap early-out: no comments at all -> nothing to intercept. // Cheap early-out: no comments at all -> nothing to intercept.
if (!html.includes("<!--")) return html; if (!html.includes("<!--")) return html;
@@ -664,13 +726,13 @@ function applyCommentDirectives(html: string): string {
if (parsed.name === "subpages" || parsed.name === "pagebreak") { if (parsed.name === "subpages" || parsed.name === "pagebreak") {
// #293 canon #5 STANDALONE machinery. A lone comment line is rendered by // #293 canon #5 STANDALONE machinery. A lone comment line is rendered by
// marked as an HTML block; the parser places it either directly under // marked as its own HTML block; the parser places it under <body>, at
// <body> (when other content surrounds it) or at document level (when it // document level (leading), or — when the directive is NESTED — inside a
// leads the output). Both are STANDALONE position. A `subpages`/`pagebreak` // block CONTAINER (`<blockquote>` for blockquote/callout, `<details>`,
// comment sitting inside a `<p>`/`<hN>` (or any other element) is attached // `<li>`, a column `<div>`, …). All of those are STANDALONE position. Only a
// position -> INERT. // comment ATTACHED inside a `<p>`/`<hN>` (bound to that block's text) is
const standalone = tag === "" || tag === "body" || tag === "html"; // attached position -> INERT.
if (!standalone) continue; // wrong position -> inert if (isAttachedPosition(tag)) continue; // wrong position -> inert
const div = document.createElement("div"); const div = document.createElement("div");
if (parsed.name === "pagebreak") { if (parsed.name === "pagebreak") {
div.setAttribute("data-type", "pageBreak"); div.setAttribute("data-type", "pageBreak");
@@ -680,26 +742,18 @@ function applyCommentDirectives(html: string): string {
div.setAttribute("data-recursive", "true"); div.setAttribute("data-recursive", "true");
} }
} }
if (tag === "body") { placeStandalone(comment, div, tag, leadingDivs);
// In-body: replace in place so surrounding content keeps its order.
comment.replaceWith(div);
} else {
// Document-level (leading): drop the stray comment and queue the div to
// be prepended into body below.
comment.remove();
leadingDivs.push(div);
}
continue; continue;
} }
if (parsed.name === "pageembed" || parsed.name === "transclusion") { if (parsed.name === "pageembed" || parsed.name === "transclusion") {
// #293 canon #8 STANDALONE media. Like subpages/pagebreak: a lone comment // #293 canon #8 STANDALONE media. Like subpages/pagebreak: a lone comment
// line placed under <body> or at document level (leading). An attached- // line placed under <body>, at document level (leading), or NESTED inside a
// position comment (inside a <p>/<hN> with a sibling) is INERT. We rebuild // block container (blockquote/callout/details/li/column). An ATTACHED-
// the schema div the raw-HTML path emits (media-html.ts) from the decoded // position comment (inside a `<p>`/`<hN>`) is INERT. We rebuild the schema
// attrs so serialize/parse stay in sync. // div the raw-HTML path emits (media-html.ts) from the decoded attrs so
const standalone = tag === "" || tag === "body" || tag === "html"; // serialize/parse stay in sync.
if (!standalone) continue; // wrong position -> inert if (isAttachedPosition(tag)) continue; // wrong position -> inert
const el = buildElement( const el = buildElement(
parsed.name === "pageembed" parsed.name === "pageembed"
? pageEmbedToHtml({ sourcePageId: parsed.attrs.sourcePageId }) ? pageEmbedToHtml({ sourcePageId: parsed.attrs.sourcePageId })
@@ -709,12 +763,7 @@ function applyCommentDirectives(html: string): string {
}), }),
); );
if (!el) continue; // defensive: builder always yields an element if (!el) continue; // defensive: builder always yields an element
if (tag === "body") { placeStandalone(comment, el, tag, leadingDivs);
comment.replaceWith(el);
} else {
comment.remove();
leadingDivs.push(el);
}
continue; continue;
} }
@@ -806,18 +855,36 @@ function applyCommentDirectives(html: string): string {
if (!parent) continue; // attrs comment must have an element parent if (!parent) continue; // attrs comment must have an element parent
if (parsed.name !== "attrs") continue; // unknown name -> inert if (parsed.name !== "attrs") continue; // unknown name -> inert
// #293 canon #9 ATTACHED attrs: honored only in attached position.
const isBlock = tag === "p" || /^h[1-6]$/.test(tag);
if (!isBlock) continue; // misplaced comment -> inert
const align = parsed.attrs.textAlign; const align = parsed.attrs.textAlign;
if (typeof align === "string" && align) { // #293 canon #9 ATTACHED attrs: honored only in attached position.
// Re-express as an inline style; the schema's textAlign parseHTML reads if (tag === "p" || /^h[1-6]$/.test(tag)) {
// `el.style.textAlign` back onto the paragraph/heading node. // A real <p>/<hN> host (loose list item, top-level block, …): re-express as
parent.style.textAlign = align; // an inline style; the schema's textAlign parseHTML reads `el.style.textAlign`
// back onto the paragraph/heading node.
if (typeof align === "string" && align) parent.style.textAlign = align;
comment.remove();
} else if (tag === "li" || tag === "td" || tag === "th") {
// TIGHT list item / GFM table cell: marked emits the paragraph's inline
// content DIRECTLY inside the <li>/<td>/<th> with NO <p> wrapper, so there
// is no element to carry the style — generateJSON materializes the
// paragraph later. Wrap the host's LEADING inline content (everything up to
// the comment; any trailing block child such as a nested list stays put) in
// a <p> carrying the alignment, so the materialized paragraph re-reads it.
if (typeof align === "string" && align) {
const p = document.createElement("p");
p.style.textAlign = align;
while (parent.firstChild && parent.firstChild !== comment) {
p.appendChild(parent.firstChild);
}
parent.insertBefore(p, comment);
}
comment.remove();
} else {
// Misplaced `attrs` comment (not a textblock/li/cell host): inert. Consume
// it anyway so no attached marker ever survives into the parsed body
// (matches the pre-existing "consume regardless" behaviour).
comment.remove();
} }
// Consume the marker regardless (unknown keys are simply ignored) so no
// attached comment ever survives into the parsed body.
comment.remove();
} }
// Prepend any document-level (leading) standalone divs into body, preserving // Prepend any document-level (leading) standalone divs into body, preserving
// their document order relative to each other and ahead of existing content. // their document order relative to each other and ahead of existing content.
@@ -46,7 +46,11 @@ export function videoToHtml(attrs: Record<string, any>): string {
if (attrs.width != null) parts.push(`width="${escapeAttr(attrs.width)}"`); if (attrs.width != null) parts.push(`width="${escapeAttr(attrs.width)}"`);
if (attrs.height != null) parts.push(`height="${escapeAttr(attrs.height)}"`); if (attrs.height != null) parts.push(`height="${escapeAttr(attrs.height)}"`);
if (attrs.size != null) parts.push(`data-size="${escapeAttr(attrs.size)}"`); if (attrs.size != null) parts.push(`data-size="${escapeAttr(attrs.size)}"`);
if (attrs.align) parts.push(`data-align="${escapeAttr(attrs.align)}"`); // align default is "center" (schema): OMIT it so a bare/center node stays
// clean and parse's re-materialized "center" default is not a P2 churn — only
// a genuinely non-default left/right emits data-align (mirrors imageToHtml).
if (attrs.align && attrs.align !== "center")
parts.push(`data-align="${escapeAttr(attrs.align)}"`);
if (attrs.aspectRatio != null) if (attrs.aspectRatio != null)
parts.push(`data-aspect-ratio="${escapeAttr(attrs.aspectRatio)}"`); parts.push(`data-aspect-ratio="${escapeAttr(attrs.aspectRatio)}"`);
return `<div><video ${parts.join(" ")}></video></div>`; return `<div><video ${parts.join(" ")}></video></div>`;
@@ -62,7 +66,9 @@ export function youtubeToHtml(attrs: Record<string, any>): string {
parts.push(`data-width="${escapeAttr(attrs.width)}"`); parts.push(`data-width="${escapeAttr(attrs.width)}"`);
if (attrs.height != null) if (attrs.height != null)
parts.push(`data-height="${escapeAttr(attrs.height)}"`); parts.push(`data-height="${escapeAttr(attrs.height)}"`);
if (attrs.align) parts.push(`data-align="${escapeAttr(attrs.align)}"`); // "center" is the schema default -> omit (see videoToHtml rationale).
if (attrs.align && attrs.align !== "center")
parts.push(`data-align="${escapeAttr(attrs.align)}"`);
return `<div ${parts.join(" ")}></div>`; return `<div ${parts.join(" ")}></div>`;
} }
@@ -97,7 +103,9 @@ export function diagramToHtml(
if (attrs.size != null) parts.push(`data-size="${escapeAttr(attrs.size)}"`); if (attrs.size != null) parts.push(`data-size="${escapeAttr(attrs.size)}"`);
if (attrs.aspectRatio != null) if (attrs.aspectRatio != null)
parts.push(`data-aspect-ratio="${escapeAttr(attrs.aspectRatio)}"`); parts.push(`data-aspect-ratio="${escapeAttr(attrs.aspectRatio)}"`);
if (attrs.align) parts.push(`data-align="${escapeAttr(attrs.align)}"`); // "center" is the schema default -> omit (see videoToHtml rationale).
if (attrs.align && attrs.align !== "center")
parts.push(`data-align="${escapeAttr(attrs.align)}"`);
if (attrs.attachmentId) if (attrs.attachmentId)
parts.push(`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`); parts.push(`data-attachment-id="${escapeAttr(attrs.attachmentId)}"`);
return `<div ${parts.join(" ")}></div>`; return `<div ${parts.join(" ")}></div>`;
@@ -110,10 +118,18 @@ export function embedToHtml(attrs: Record<string, any>): string {
`data-src="${escapeAttr(attrs.src ?? "")}"`, `data-src="${escapeAttr(attrs.src ?? "")}"`,
`data-provider="${escapeAttr(attrs.provider ?? "")}"`, `data-provider="${escapeAttr(attrs.provider ?? "")}"`,
]; ];
if (attrs.align) parts.push(`data-align="${escapeAttr(attrs.align)}"`); // "center" is the schema default -> omit (see videoToHtml rationale).
if (attrs.width != null) if (attrs.align && attrs.align !== "center")
parts.push(`data-align="${escapeAttr(attrs.align)}"`);
// embed width/height default to the NUMBERS 800/600 (schema). Getting a data-
// attribute back always yields a STRING, so emitting the default here would
// round-trip 800 -> "800" (a number->string P1 divergence canonicalize does
// NOT normalize). OMIT the defaults so parse re-materializes the numeric
// default instead — mirrors the top-level embed path (markdown-converter.ts),
// which also emits width/height only when they differ from 800/600.
if (attrs.width != null && attrs.width !== 800)
parts.push(`data-width="${escapeAttr(attrs.width)}"`); parts.push(`data-width="${escapeAttr(attrs.width)}"`);
if (attrs.height != null) if (attrs.height != null && attrs.height !== 600)
parts.push(`data-height="${escapeAttr(attrs.height)}"`); parts.push(`data-height="${escapeAttr(attrs.height)}"`);
return `<div ${parts.join(" ")}></div>`; return `<div ${parts.join(" ")}></div>`;
} }
@@ -1,16 +0,0 @@
{
"_bug": "BUG #351: a `column` whose `width` is a percentage string (e.g. \"50%\") is NOT byte-stable across export->import->export (violates P2). The `column` schema's parseHTML does `parseFloat(getAttribute('data-width'))`, which silently drops the '%' unit and returns the NUMBER 50. So the first export emits data-width=\"50%\" but the re-import stores width=50, and the second export emits data-width=\"50\": md2 !== md1, a permanent GS-EDIT-REVERT churn (every git-sync pull rewrites the column width). The editor authors column widths as percentages, so this is a real data/round-trip defect. Fix belongs in src/lib/docmost-schema.ts column.width parseHTML (preserve the unit / keep the string), which is OUT OF SCOPE for this test-only PR and must be a separate, maintainer-approved change. This flat generator therefore keeps `column.width` frozen (never generates a non-default width).",
"doc": {
"type": "doc",
"content": [
{
"type": "columns",
"attrs": { "layout": "two_equal", "widthMode": "normal" },
"content": [
{ "type": "column", "attrs": { "width": "50%" }, "content": [{ "type": "paragraph", "content": [{ "type": "text", "text": "L" }] }] },
{ "type": "column", "attrs": { "width": "50%" }, "content": [{ "type": "paragraph", "content": [{ "type": "text", "text": "R" }] }] }
]
}
]
}
}
@@ -37,14 +37,19 @@
* tagged `// ACCEPTED:` inline. Freezing them is correct there is * tagged `// ACCEPTED:` inline. Freezing them is correct there is
* nothing to preserve in the target format. * nothing to preserve in the target format.
* *
* (b) PINNED BUG the attribute IS representable in markdown but the * (b) FIXED & VALUE-FUZZED attributes that were once PINNED converter
* converter drops it anyway (a real defect). These are NOT silently * bugs (representable in markdown but dropped) and are now FIXED in
* frozen: each is captured as a LOUD `it.fails` counterexample in * src/, so they are value-fuzzed here at legal non-default values like
* test/fixtures/counterexamples/ + counterexamples.test.ts, and the * any healthy attr. `orderedList.start` (the non-1 start once rendered
* freeze here only keeps the P1/P2 union green until a MAINTAINER rules * as `1.`; the converter now emits the start marker / `<ol start="N">`)
* on accept-vs-fix (the epic guardrail reserves that call). These: * and `column.width` (a unitless flex-grow number that round-trips via
* `column.width` (parseFloat drops `%`), `orderedList.start` (non-1 * parseFloat) are both fuzzed in OVERRIDES below. The former held-out
* start renders as `1.`). Tagged `// PINNED-BUG:` inline. * `it.fails` cases are gone; `ordered-list-start.json` +
* counterexamples.test.ts now stand as PASSING regression pins (per the
* epic guardrail, the minimal doc stays forever to guard re-regression).
* The #351 media-family sizing attrs (image/video/youtube/pdf/drawio/
* excalidraw/embed width/height/size/aspectRatio) are likewise fuzzed
* now that they ride round-trip-safely in the per-node `<!--…-->` JSON.
* *
* (c) DEFERRED-BUG representable AND round-trips, frozen only because the * (c) DEFERRED-BUG representable AND round-trips, frozen only because the
* flat generator can't yet build a valid instance. Table * flat generator can't yet build a valid instance. Table
@@ -64,8 +69,10 @@
* number re-parses as a string), EXCEPT `embed.width/height` which the * number re-parses as a string), EXCEPT `embed.width/height` which the
* embed schema keeps numeric handled per-attr. * embed schema keeps numeric handled per-attr.
* *
* Both PINNED-BUG attrs (`column.width` P2 churn, `orderedList.start` P1 loss) * The two former PINNED-BUG attrs (`column.width` P2 churn, `orderedList.start`
* are captured as committed `it.fails` counterexamples NOT hidden here. * P1 loss) are now FIXED and value-fuzzed; `ordered-list-start.json` in
* counterexamples.test.ts is a permanent PASSING regression pin, not an
* `it.fails` hold-out.
*/ */
import fc from 'fast-check'; import fc from 'fast-check';
import { getSchema } from '@tiptap/core'; import { getSchema } from '@tiptap/core';
@@ -120,6 +127,11 @@ interface AttrPolicy {
const num = (...xs: number[]) => fc.constantFrom(...xs); const num = (...xs: number[]) => fc.constantFrom(...xs);
const str = (...xs: string[]) => fc.constantFrom(...xs); const str = (...xs: string[]) => fc.constantFrom(...xs);
const widthStr = str('120', '320', '640'); const widthStr = str('120', '320', '640');
// Media `aspectRatio`/`size` are stringified numerics too (converter emits
// String(value)); the schema parseHTML reads them back as strings. Fuzz as
// plausible numeric strings so they round-trip byte-stably like widthStr.
const aspectRatioStr = str('1.5', '0.75', '1');
const sizeStr = str('320', '640');
// The documented override table, keyed `type.attr`. Every entry is grounded in // The documented override table, keyed `type.attr`. Every entry is grounded in
// the empirical converter probe (see flat-roundtrip.property.test.ts header). // the empirical converter probe (see flat-roundtrip.property.test.ts header).
@@ -134,10 +146,10 @@ const OVERRIDES: Record<string, AttrPolicy> = {
'heading.textAlign': { arb: str('center', 'right', 'justify') }, 'heading.textAlign': { arb: str('center', 'right', 'justify') },
'heading.indent': { frozen: true }, // ACCEPTED: no md representation 'heading.indent': { frozen: true }, // ACCEPTED: no md representation
// ── lists ──────────────────────────────────────────────────────────────── // ── lists ────────────────────────────────────────────────────────────────
// PINNED-BUG: markdown CAN express a non-1 start ("5."), but the converter // FIXED (#351): the converter now emits the start marker ("5." / <ol start="5">)
// renders "1." and drops it -> P1 loss. See counterexamples.test.ts // and it round-trips, so the start number is value-fuzzed. See
// (ordered-list-start.json). Frozen only until the maintainer rules accept-vs-fix. // counterexamples.test.ts (ordered-list-start.json) for the regression pin.
'orderedList.start': { always: true, frozen: true }, 'orderedList.start': { always: true, arb: num(2, 3, 5, 42) },
'orderedList.type': { frozen: true }, // ACCEPTED: a/A/i markers not expressible in GFM 'orderedList.type': { frozen: true }, // ACCEPTED: a/A/i markers not expressible in GFM
'taskItem.checked': { always: true, arb: fc.constant(true) }, // boolean, default false 'taskItem.checked': { always: true, arb: fc.constant(true) }, // boolean, default false
// ── codeBlock ──────────────────────────────────────────────────────────── // ── codeBlock ────────────────────────────────────────────────────────────
@@ -149,16 +161,57 @@ const OVERRIDES: Record<string, AttrPolicy> = {
'image.title': { arb: letterPhraseArb }, 'image.title': { arb: letterPhraseArb },
'image.width': { arb: widthStr, degen: '' }, 'image.width': { arb: widthStr, degen: '' },
'image.height': { arb: widthStr, degen: '' }, 'image.height': { arb: widthStr, degen: '' },
// #351 (this PR): media sizing/family attrs ride in the `<!--img {…}-->`
// comment JSON via String(value); parseHTML reads them back as strings, so a
// numeric string round-trips byte-stably (mirrors image.width).
'image.size': { arb: sizeStr, degen: '' },
'image.aspectRatio': { arb: aspectRatioStr },
// caption is text carried verbatim in the comment JSON (mirrors image.alt).
'image.caption': { arb: letterPhraseArb, degen: '' },
'video.src': { noDefault: true, arb: urlArb, degen: '' }, 'video.src': { noDefault: true, arb: urlArb, degen: '' },
'video.alt': { arb: letterPhraseArb }, 'video.alt': { arb: letterPhraseArb },
'video.width': { arb: widthStr }, 'video.width': { arb: widthStr },
'video.height': { arb: widthStr }, 'video.height': { arb: widthStr },
// #351 (this PR): video sizing/align ride in the `<!--video {…}-->` comment.
'video.size': { arb: sizeStr },
'video.aspectRatio': { arb: aspectRatioStr },
// align default "center" is dropped on export; fuzz non-center only.
'video.align': { arb: str('left', 'right') },
'audio.src': { noDefault: true, arb: urlArb, degen: '' }, 'audio.src': { noDefault: true, arb: urlArb, degen: '' },
'youtube.src': { noDefault: true, arb: urlArb }, 'youtube.src': { noDefault: true, arb: urlArb },
// #351 (this PR): youtube width/height/align ride in the `<!--youtube {…}-->`
// comment JSON (String()-coerced dimensions, non-center align only).
'youtube.width': { arb: widthStr },
'youtube.height': { arb: widthStr },
'youtube.align': { arb: str('left', 'right') },
'pdf.src': { noDefault: true, arb: urlArb }, 'pdf.src': { noDefault: true, arb: urlArb },
'pdf.name': { arb: phraseArb }, 'pdf.name': { arb: phraseArb },
// #351 (this PR): pdf size/width/height ride in the `<!--pdf {…}-->` comment
// (String()-coerced dimensions, read back as strings).
'pdf.size': { arb: sizeStr },
'pdf.width': { arb: widthStr },
'pdf.height': { arb: widthStr },
'drawio.src': { noDefault: true, arb: urlArb }, 'drawio.src': { noDefault: true, arb: urlArb },
// #351 (this PR): drawio family attrs ride in the `<!--drawio {…}-->` comment.
// Dimensions/size/aspectRatio are String()-coerced numeric strings; title/alt
// are text carried verbatim; align fuzzed non-center only.
'drawio.width': { arb: widthStr },
'drawio.height': { arb: widthStr },
'drawio.size': { arb: sizeStr },
'drawio.aspectRatio': { arb: aspectRatioStr },
'drawio.align': { arb: str('left', 'right') },
'drawio.title': { arb: letterPhraseArb },
'drawio.alt': { arb: letterPhraseArb },
'excalidraw.src': { noDefault: true, arb: urlArb }, 'excalidraw.src': { noDefault: true, arb: urlArb },
// #351 (this PR): excalidraw family attrs ride in the `<!--excalidraw {…}-->`
// comment (same shape as the drawio family above).
'excalidraw.width': { arb: widthStr },
'excalidraw.height': { arb: widthStr },
'excalidraw.size': { arb: sizeStr },
'excalidraw.aspectRatio': { arb: aspectRatioStr },
'excalidraw.align': { arb: str('left', 'right') },
'excalidraw.title': { arb: letterPhraseArb },
'excalidraw.alt': { arb: letterPhraseArb },
'attachment.url': { noDefault: true, arb: urlArb }, 'attachment.url': { noDefault: true, arb: urlArb },
'attachment.name': { arb: phraseArb }, 'attachment.name': { arb: phraseArb },
// ── callout / status ───────────────────────────────────────────────────── // ── callout / status ─────────────────────────────────────────────────────
@@ -194,14 +247,26 @@ const OVERRIDES: Record<string, AttrPolicy> = {
// widthMode round-trips via the `data-width-mode` attribute (verified P1+P2), // widthMode round-trips via the `data-width-mode` attribute (verified P1+P2),
// so it is fuzzed, not frozen. // so it is fuzzed, not frozen.
'columns.widthMode': { always: true, arb: str('custom') }, 'columns.widthMode': { always: true, arb: str('custom') },
// PINNED-BUG: parseFloat import drops the `%` unit -> P2 churn. See // column.width is a unitless flex-grow NUMBER (matches editor-ext column.ts);
// counterexamples.test.ts (columns-column-width-percent.json). // parseHTML does parseFloat, so String(50) === "50" both ways and a numeric
'column.width': { frozen: true }, // width round-trips byte-stably. Value-fuzzed as a number.
'column.width': { arb: num(25, 50, 75) },
// ── embed (schema keeps width/height NUMERIC, not string-coerced) ───────── // ── embed (schema keeps width/height NUMERIC, not string-coerced) ─────────
'embed.src': { noDefault: true, arb: urlArb, degen: '' }, 'embed.src': { noDefault: true, arb: urlArb, degen: '' },
'embed.provider': { noDefault: true, arb: str('iframe', 'youtube', 'vimeo') }, 'embed.provider': { noDefault: true, arb: str('iframe', 'youtube', 'vimeo') },
'embed.width': { always: true, frozen: true }, // #351 (this PR): the embed schema defaults width/height to the NUMBERS 800/600
'embed.height': { always: true, frozen: true }, // and the converter only emits them when they differ. But the value round-trips
// as a STRING: export stringifies into the comment JSON (String(width)) and the
// import path (embedToHtml -> data-width -> embed parseHTML) reads it back as a
// string, so an authored NUMBER 400 would diverge under P1 (400 vs "400"). Fuzz
// as numeric STRINGS avoiding "800"/"600" so they round-trip byte-stably. The
// 800/600 numeric default state still round-trips (omitted on export, re-
// materialized as the numeric default). `always` stays because these are
// materialized on import but absent from canonicalize's KNOWN_DEFAULTS.
'embed.width': { always: true, arb: str('400', '1000', '1200') },
'embed.height': { always: true, arb: str('300', '500', '900') },
// align default "center" is dropped on export; fuzz non-center only.
'embed.align': { arb: str('left', 'right') },
// ── subpages / math / htmlEmbed ────────────────────────────────────────── // ── subpages / math / htmlEmbed ──────────────────────────────────────────
'subpages.recursive': { always: true, arb: fc.constant(true) }, // boolean, default false 'subpages.recursive': { always: true, arb: fc.constant(true) }, // boolean, default false
'mathBlock.text': { noDefault: true, arb: str('x^2', 'a < b', '\\frac{1}{2}'), degen: '' }, 'mathBlock.text': { noDefault: true, arb: str('x^2', 'a < b', '\\frac{1}{2}'), degen: '' },
@@ -7,18 +7,17 @@ import { markdownToProseMirror } from '../../src/lib/markdown-to-prosemirror.js'
import { docsCanonicallyEqual } from '../../src/lib/canonicalize.js'; import { docsCanonicallyEqual } from '../../src/lib/canonicalize.js';
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// #351 committed counterexamples — REAL round-trip bugs surfaced by the flat // #351 committed counterexample — a REAL round-trip bug surfaced by the flat
// generative probing (attribute level). Each is pinned here as an `it.fails` // generative probing (attribute level). The bug below is now FIXED in src/
// (vitest passes ONLY WHILE the assertion still fails), so that the day the // (maintainer-approved), so this case is a permanent PASSING regression pin:
// underlying src/ bug is fixed, the `it.fails` starts PASSING and vitest turns // the exact document that once lost data now round-trips cleanly, and the
// this test RED — forcing us to delete the counterexample and (per the epic // fixture stays in test/fixtures/counterexamples/ forever (per the epic
// guardrail) tighten the generator. A bare `it.fails` would ship silent // guardrail) to guard against a re-regression. The case carries a loud
// corruption, so every case below carries a loud `// BUG #351:` explanation. // `// BUG #351 (FIXED):` note recording the defect and its fix site.
// //
// These bugs are NOT worked around by weakening any property: the offending // With the bug fixed, the offending attribute is now VALUE-FUZZED by the
// attribute is kept OUT of the P1/P2 generators (documented in // generator (see attr-arbitraries.ts: orderedList.start), not held out — this
// attr-arbitraries.ts), and the exact failing document lives here as the // committed document is the minimal, human-readable pin.
// regression pin. FIXING the bug is a separate, maintainer-approved src/ change.
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
const here = path.dirname(fileURLToPath(import.meta.url)); const here = path.dirname(fileURLToPath(import.meta.url));
@@ -28,47 +27,20 @@ function loadDoc(file: string): any {
return JSON.parse(readFileSync(path.join(fixtureDir, file), 'utf8')).doc; return JSON.parse(readFileSync(path.join(fixtureDir, file), 'utf8')).doc;
} }
describe('#351 counterexamples (known round-trip bugs, pinned as it.fails)', () => { describe('#351 counterexamples (round-trip bugs, now FIXED — permanent regression pins)', () => {
// BUG #351: a `column` with a PERCENTAGE width ("50%") is not byte-stable. // BUG #351 (FIXED): an `orderedList` with a non-1 `start` lost its start
// The column schema parses `data-width` with parseFloat, dropping the '%': // number. CommonMark CAN express this ("5." starts the list at 5), but the
// md1 = '...data-width="50%"...' (first export) // converter always emitted "1." and ignored `attrs.start`:
// re-import stores width = 50 (number)
// md2 = '...data-width="50"...' (second export) => md2 !== md1
// A permanent GS-EDIT-REVERT churn on every git-sync pull. The editor stores
// column widths as percentages, so this is a genuine defect. The fix is in
// src/lib/docmost-schema.ts (column.width parseHTML must preserve the unit)
// and is out of scope for this test-only PR.
it.fails('column percentage width is byte-stable (P2)', async () => {
const doc = loadDoc('columns-column-width-percent.json');
const md1 = convertProseMirrorToMarkdown(doc);
const doc2 = await markdownToProseMirror(md1);
const md2 = convertProseMirrorToMarkdown(doc2);
// This assertion currently FAILS (md2 drops the '%'), which is exactly what
// `it.fails` expects. When the schema is fixed, it will PASS and flip this
// test red — our cue to remove the pin.
expect(md2).toBe(md1);
});
// BUG #351: an `orderedList` with a non-1 `start` loses its start number.
// CommonMark CAN express this ("5." starts the list at 5), but the converter
// always emits "1." and ignores `attrs.start` (markdown-converter.ts renders
// `${index + 1}.`; the <ol> HTML path also omits `start`):
// doc.start = 5 -> md1 = "1. alpha" (start dropped on export) // doc.start = 5 -> md1 = "1. alpha" (start dropped on export)
// re-import stores start = 1 => docsCanonicallyEqual(rt, doc) === false // re-import stored start = 1 => docsCanonicallyEqual(rt, doc) === false
// This is a P1 (semantic round-trip) loss of the SAME class as column.width: // A P1 (semantic round-trip) loss of the SAME class as column.width. FIXED in
// representable in markdown, silently dropped by the converter. It is pinned // this PR in src/lib/markdown-converter.ts (the orderedList markdown path emits
// here as the LOUD counterexample rather than being masked as an "accepted // `${start + index}.`; the raw-HTML path emits `<ol start="N">`). tiptap's
// normalization" in the generator — per the epic guardrail, deciding // StarterKit / marked parse the start back, so it round-trips. Permanent P1 pin.
// accept-vs-fix for a markdown-representable loss is a MAINTAINER call, so this it('ordered list start number is preserved (P1)', async () => {
// stays a visible known-bug until the maintainer rules on it. The fix would be
// in src/lib/markdown-converter.ts (emit the start number on the first item)
// and is out of scope for this test-only PR.
it.fails('ordered list start number is preserved (P1)', async () => {
const doc = loadDoc('ordered-list-start.json'); const doc = loadDoc('ordered-list-start.json');
const md1 = convertProseMirrorToMarkdown(doc); const md1 = convertProseMirrorToMarkdown(doc);
const doc2 = await markdownToProseMirror(md1); const doc2 = await markdownToProseMirror(md1);
// Currently FAILS: doc2.start === 1 while doc.start === 5. When the converter
// preserves `start`, this PASSES and flips the test red — remove the pin then.
expect(docsCanonicallyEqual(doc2, doc)).toBe(true); expect(docsCanonicallyEqual(doc2, doc)).toBe(true);
}); });
}); });
@@ -0,0 +1,392 @@
/**
* Nested whole-document generator (#351, PR 2) "variant A": a random walk over
* the schema's ContentMatch automaton.
*
* Where the FLAT generator (node-generators.ts) emits `{doc:[ <one target> ]}`,
* this module produces arbitrarily DEEP, valid ProseMirror documents. Validity is
* NOT hand-asserted: it comes straight from the schema. To fill a container's
* block content we start at `nodeType.contentMatch` and walk the automaton
* enumerate the legal next node types (`match.edge(i).type`), let fast-check pick
* one (or STOP once `match.validEnd`), generate that child RECURSIVELY, then
* advance the automaton via `match.matchType(childType)`. A bad walk therefore
* cannot emit a structurally-invalid doc (the `generator validity` test guards
* this with `schema.nodeFromJSON(json).check()`).
*
* What the walk drives, and what it delegates
* The walk owns BLOCK STRUCTURE (which blocks nest inside which containers, in
* what order and how deep). Two things it deliberately delegates, to avoid
* REGRESSING the byte-stable space the flat suite already proved empirically:
*
* - INLINE content of textblocks (paragraph/heading/codeBlock/detailsSummary)
* is filled from text-arbitraries.ts the exact hostile-but-byte-stable
* inline corpus the flat suite established. Walking the inline ContentMatch
* instead would re-derive (and re-fail) the text-space limitations the flat
* suite already pins, which is not this generator's job.
* - Node ATTRS come from nodeAttrsArb(type, 'p1') the round-trip-safe
* attribute space. Attribute-degenerate fuzzing (P2/P3 over 'fuzz') is the
* flat suite's concern; the nested suite isolates STRUCTURAL round-trip, so
* it stays in 'p1' and does not re-litigate frozen/pinned attributes.
*
* Coordination the automaton cannot express
* A ContentMatch guarantees a child SEQUENCE is legal, but not cross-sibling
* invariants. Two nodes need coordination the walk injects by hand:
* - `table`: GFM needs a RECTANGULAR grid with column-consistent alignment.
* The automaton happily allows ragged rows / per-cell align, which are not a
* converter bug just a malformed table. So a table is generated ATOMICALLY
* (same shape the flat suite proved) rather than walked.
* - `columns`: the `layout` attr must agree with the column COUNT. We pick the
* layout, derive the count, then walk each column's block body normally so
* columns still gain real nested content, only the count is coordinated.
*
* Excluded from the nested walk
* Footnote nodes (footnoteReference / footnotesList / footnoteDefinition) need a
* DOCUMENT-GLOBAL id match between a reference and its definition. That
* coordination is owned by the flat suite's `footnotes` generator; placing them
* independently here would fabricate id mismatches that look like converter bugs
* but are generator defects. They are filtered out of the walk (documented in
* EXCLUDED). The completeness contract lives in the FLAT suite and is unaffected.
*
* Termination / budgets
* Two bounds keep every doc finite and the suite fast:
* - MAX_DEPTH a hard cap on block-nesting depth. A precomputed `minDepth`
* fixpoint (the minimum extra nesting a subtree of each type needs to be
* valid) lets the walk pick a CONTAINER child only when there is depth
* headroom to complete it so the walk can never paint itself into a corner
* where a required child cannot fit (no invalid docs, guaranteed termination).
* - NODE_BUDGET a soft cap on total nodes; as it runs low the walk biases
* toward STOP (when validEnd) or toward cheap terminating children.
*/
import fc from 'fast-check';
import { getSchema } from '@tiptap/core';
import { docmostExtensions } from '../../src/lib/docmost-schema.js';
import { nodeAttrsArb } from './attr-arbitraries.js';
import {
inlineContentArb,
headingInlineContentArb,
plainInlineContentArb,
phraseArb,
} from './text-arbitraries.js';
/** The exact ProseMirror schema the converter targets (built per the issue). */
export const schema = getSchema(docmostExtensions as never);
/** Hard cap on block-nesting depth (doc = depth 0). Kept in the issue's 4–5 band. */
export const MAX_DEPTH = 4;
/**
* Soft cap on total node count per generated document. Kept moderate: every P1/P2
* run parses the emitted markdown through jsdom (heavy), so 100+ node docs across
* hundreds of runs exhaust the worker heap. 60 still yields deeply-nested docs
* (depth 4) while keeping the suite within memory.
*/
export const NODE_BUDGET = 60;
/**
* Nodes kept OUT of the nested walk: footnote nodes need a doc-global id match a
* local walk cannot coordinate (owned by the flat suite's `footnotes` generator).
*/
const EXCLUDED = new Set<string>([
'footnoteReference',
'footnotesList',
'footnoteDefinition',
]);
/**
* Structural-only children that carry `group: "block"` in the schema and so leak
* into EVERY block container's ContentMatch, even though the editor only ever
* places them inside their one true parent. Choosing them freely (e.g. a bare
* `column` at the document root) fabricates documents no editor produces and that
* the converter is not designed to round-trip a GENERATOR artifact, not a
* converter bug. They are admitted ONLY when the container being filled is their
* dedicated parent. (`column` is in fact always built inside columnsArb, so this
* just double-guards it.)
*/
const DEDICATED_PARENT: Record<string, string> = {
column: 'columns',
detailsSummary: 'details',
detailsContent: 'details',
};
/** Is child type `t` legal as a freely-chosen child of container `parentType`? */
function childAllowedUnder(t: string, parentType: string): boolean {
const dedicated = DEDICATED_PARENT[t];
return dedicated === undefined || dedicated === parentType;
}
/** Textblock (inlineContent) types — filled from the proven inline corpus. */
function isTextblock(typeName: string): boolean {
return !!schema.nodes[typeName]?.isTextblock;
}
/** Leaf/atom types — no content, only generated attrs. */
function isLeaf(typeName: string): boolean {
return !!schema.nodes[typeName]?.isLeaf;
}
// ---------------------------------------------------------------------------
// minDepth fixpoint: the minimum EXTRA block-nesting depth a valid subtree
// rooted at each node type requires. Leaves and textblocks need 0 (a textblock
// is satisfied by inline content, no block recursion). A container needs
// 1 + the cheapest way to satisfy its ContentMatch. Computed as a min–max path
// to `validEnd` over the automaton, iterated to a fixpoint over node types.
// ---------------------------------------------------------------------------
/** Cheapest (min over reachable validEnd of max child minDepth) to complete a match. */
function minCompletion(
match: any,
md: Record<string, number>,
seen: Set<any>,
parentType: string,
): number {
let best = match.validEnd ? 0 : Infinity;
if (seen.has(match)) return best; // a cycle never completes more cheaply
seen.add(match);
for (let i = 0; i < match.edgeCount; i++) {
const edge = match.edge(i);
const t = edge.type.name;
if (EXCLUDED.has(t) || t === 'text') continue;
if (!childAllowedUnder(t, parentType)) continue;
const childCost = md[t];
if (childCost === undefined || childCost === Infinity) continue;
const rest = minCompletion(edge.next, md, seen, parentType);
if (rest === Infinity) continue;
best = Math.min(best, Math.max(childCost, rest));
}
seen.delete(match);
return best;
}
function computeMinDepth(): Record<string, number> {
const md: Record<string, number> = {};
for (const name of Object.keys(schema.nodes)) {
md[name] = isLeaf(name) || isTextblock(name) ? 0 : Infinity;
}
let changed = true;
while (changed) {
changed = false;
for (const name of Object.keys(schema.nodes)) {
if (md[name] === 0) continue; // leaves/textblocks fixed at 0
const nt: any = schema.nodes[name];
const completion = minCompletion(nt.contentMatch, md, new Set(), name);
const next = completion === Infinity ? Infinity : 1 + completion;
if (next < md[name]) {
md[name] = next;
changed = true;
}
}
}
return md;
}
const MIN_DEPTH = computeMinDepth();
// ---------------------------------------------------------------------------
// Leaf / textblock builders (attrs from 'p1', inline from the proven corpus).
// ---------------------------------------------------------------------------
function attachAttrs(typeName: string, base: Record<string, unknown> = {}) {
return nodeAttrsArb(typeName, 'p1', base).map((attrs) => {
const node: any = { type: typeName };
if (Object.keys(attrs).length) node.attrs = attrs;
return node;
});
}
/** A leaf/atom block: attrs only, no content. */
function leafArb(typeName: string): fc.Arbitrary<any> {
return attachAttrs(typeName);
}
/** A textblock, inline content taken from the byte-stable flat corpus. */
function textblockArb(typeName: string): fc.Arbitrary<any> {
if (typeName === 'codeBlock') {
return fc
.tuple(
nodeAttrsArb('codeBlock', 'p1'),
// Fenced code re-imports with a TRAILING NEWLINE (flat suite finding);
// author it so the doc is already at the round-trip fixpoint.
fc.array(phraseArb, { minLength: 1, maxLength: 3 }).map((l) => l.join('\n') + '\n'),
)
.map(([attrs, code]) => ({
type: 'codeBlock',
...(Object.keys(attrs).length ? { attrs } : {}),
content: [{ type: 'text', text: code }],
}));
}
const inline =
typeName === 'heading'
? headingInlineContentArb
: typeName === 'detailsSummary'
? plainInlineContentArb
: inlineContentArb;
return fc
.tuple(nodeAttrsArb(typeName, 'p1'), inline)
.map(([attrs, content]) => ({
type: typeName,
...(Object.keys(attrs).length ? { attrs } : {}),
content,
}));
}
// ---------------------------------------------------------------------------
// Coordinated builders: table (atomic, rectangular, column-consistent align)
// and columns (layout coupled to count, bodies walked).
// ---------------------------------------------------------------------------
/** A rectangular GFM-safe table (mirrors the flat suite's proven shape). */
function tableArb(): fc.Arbitrary<any> {
return fc.integer({ min: 1, max: 3 }).chain((cols) => {
// One alignment per COLUMN, identical on header + every body cell, so the
// second export cannot re-align and churn.
const alignsArb = fc.array(fc.constantFrom(undefined, 'left', 'center', 'right'), {
minLength: cols,
maxLength: cols,
});
const cell = (header: boolean, align?: string) =>
phraseArb.map((t) => ({
type: header ? 'tableHeader' : 'tableCell',
attrs: { colspan: 1, rowspan: 1, ...(align ? { align } : {}) },
content: [{ type: 'paragraph', content: [{ type: 'text', text: t }] }],
}));
return alignsArb.chain((aligns) => {
const headerRow = fc
.tuple(...aligns.map((a) => cell(true, a)))
.map((cells) => ({ type: 'tableRow', content: cells }));
const bodyRow = fc
.tuple(...aligns.map((a) => cell(false, a)))
.map((cells) => ({ type: 'tableRow', content: cells }));
return fc
.tuple(headerRow, fc.array(bodyRow, { minLength: 1, maxLength: 2 }))
.map(([h, body]) => ({ type: 'table', content: [h, ...body] }));
});
});
}
/** A columns block: layout ↔ count coupled, each column body walked as blocks. */
function columnsArb(depth: number, budget: number): fc.Arbitrary<any> {
return fc
.constantFrom('two_equal', 'three_equal', 'left_sidebar', 'right_sidebar')
.chain((layout) => {
const count = layout === 'three_equal' ? 3 : 2;
const columnType: any = schema.nodes.column;
// Split the remaining budget across the fixed number of columns.
const per = Math.max(2, Math.floor((budget - 1) / count));
return nodeAttrsArb('columns', 'p1', { layout, widthMode: 'normal' }).chain((attrs) =>
fc
.tuple(
...Array.from({ length: count }, () =>
fillMatch(columnType.contentMatch, depth + 1, per, 'column').map(({ children }) => ({
type: 'column',
content: children,
})),
),
)
.map((cols) => ({ type: 'columns', attrs, content: cols })),
);
});
}
// ---------------------------------------------------------------------------
// The ContentMatch walk.
// ---------------------------------------------------------------------------
/** Count every node in a subtree (block + inline), for budget accounting. */
function countNodes(node: any): number {
let n = 1;
for (const c of node.content ?? []) n += countNodes(c);
return n;
}
/** Build a single child node of a given type at `depth`, within `budget`. */
function blockNode(typeName: string, depth: number, budget: number): fc.Arbitrary<any> {
if (typeName === 'table') return tableArb();
if (typeName === 'columns') return columnsArb(depth, budget);
if (isTextblock(typeName)) return textblockArb(typeName);
if (isLeaf(typeName)) return leafArb(typeName);
// Generic container: attrs from 'p1', block content from the automaton walk.
const nt: any = schema.nodes[typeName];
return nodeAttrsArb(typeName, 'p1').chain((attrs) =>
fillMatch(nt.contentMatch, depth, budget - 1, typeName).map(({ children }) => {
const node: any = { type: typeName };
if (Object.keys(attrs).length) node.attrs = attrs;
if (children.length) node.content = children;
return node;
}),
);
}
/**
* Fill a container's block content by walking its ContentMatch automaton from
* `match`. Returns the children array plus the budget left after them.
*/
function fillMatch(
match: any,
depth: number,
budget: number,
parentType: string,
): fc.Arbitrary<{ children: any[]; budget: number }> {
const canStop = match.validEnd;
// A child lives at depth+1; only pick it if its subtree can complete within
// MAX_DEPTH. This headroom rule is what makes the walk deadlock-free.
const headroom = MAX_DEPTH - (depth + 1);
const edges: { t: string; next: any }[] = [];
if (headroom >= 0) {
for (let i = 0; i < match.edgeCount; i++) {
const edge = match.edge(i);
const t = edge.type.name;
if (EXCLUDED.has(t) || t === 'text') continue;
if (!childAllowedUnder(t, parentType)) continue;
if ((MIN_DEPTH[t] ?? Infinity) > headroom) continue;
edges.push({ t, next: edge.next });
}
}
// Decide the next action: STOP (if allowed) or extend with one more child.
// Bias toward stopping when the budget is spent; force a child only when the
// match is not yet at a valid end.
const pool: { weight: number; arbitrary: fc.Arbitrary<{ t: string; next: any } | null> }[] = [];
const canGo = edges.length > 0 && (budget > 0 || !canStop);
if (canStop) {
// Stop is weighted higher when the budget is low so docs stay bounded.
pool.push({ weight: budget > 0 ? 2 : 5, arbitrary: fc.constant(null) });
}
if (canGo && !(canStop && budget <= 0)) {
pool.push({ weight: 3, arbitrary: fc.constantFrom(...edges) });
}
// Forced continuation: not a valid end yet and (budget exhausted) — must place
// a mandatory child regardless of budget.
if (pool.length === 0) {
if (edges.length > 0) {
pool.push({ weight: 1, arbitrary: fc.constantFrom(...edges) });
} else {
// No legal child and not required to place one: stop with what we have.
return fc.constant({ children: [], budget });
}
}
return fc.oneof(...pool).chain((choice) => {
if (choice === null) return fc.constant({ children: [], budget });
return blockNode(choice.t, depth + 1, budget).chain((node) => {
const cost = countNodes(node);
return fillMatch(choice.next, depth, budget - cost, parentType).map(
({ children, budget: left }) => ({
children: [node, ...children],
budget: left,
}),
);
});
});
}
/**
* The nested-document arbitrary: a valid, arbitrarily-deep ProseMirror doc built
* by walking the schema from the document root. Attrs stay in the round-trip-safe
* 'p1' space; inline content reuses the byte-stable flat corpus.
*/
export const docArb: fc.Arbitrary<any> = fillMatch(
schema.nodes.doc.contentMatch,
0,
NODE_BUDGET,
'doc',
).map(({ children }) => ({ type: 'doc', content: children }));
/** The precomputed minDepth table, exported for inspection/debugging. */
export { MIN_DEPTH };
@@ -0,0 +1,29 @@
import { describe, expect, it } from 'vitest';
import { envInt } from './env-int.js';
// Unit coverage for the shared PROPERTY_SEED / PROPERTY_NUM_RUNS parser. The key
// contract is that an explicit "0" is honored (a valid fast-check seed) while
// empty/absent/non-numeric fall back to the default.
describe('envInt', () => {
it('honors an explicit "0" (does not fall back)', () => {
expect(envInt('0', 42)).toBe(0);
});
it('falls back on empty string', () => {
expect(envInt('', 42)).toBe(42);
});
it('falls back on undefined', () => {
expect(envInt(undefined, 42)).toBe(42);
});
it('falls back on a non-numeric string', () => {
expect(envInt('abc', 42)).toBe(42);
});
it('parses a plain integer string', () => {
expect(envInt('300', 42)).toBe(300);
});
it('parses a negative integer', () => {
expect(envInt('-5', 42)).toBe(-5);
});
it('parses a large integer', () => {
expect(envInt('1073741824', 42)).toBe(1073741824);
});
});
@@ -0,0 +1,10 @@
/**
* Parse an integer-ish environment variable with a default fallback.
*
* Used to read PROPERTY_SEED / PROPERTY_NUM_RUNS in the generative property
* suites. An unset/empty/non-numeric value falls back to `dflt`, but an explicit
* `"0"` is honored (a valid fast-check seed) `Number(x) || dflt` would wrongly
* swallow 0. See flat-roundtrip.property.test.ts / nested-roundtrip.property.test.ts.
*/
export const envInt = (v: string | undefined, dflt: number): number =>
v !== undefined && v !== '' && Number.isFinite(Number(v)) ? Number(v) : dflt;
@@ -18,6 +18,7 @@ import {
coveredTypes, coveredTypes,
KNOWN_UNCOVERED, KNOWN_UNCOVERED,
} from './node-generators.js'; } from './node-generators.js';
import { envInt } from './env-int.js';
// ── Attribute-value coverage allowlist ────────────────────────────────────── // ── Attribute-value coverage allowlist ──────────────────────────────────────
// The node/mark completeness contract guarantees every TYPE is generated, but // The node/mark completeness contract guarantees every TYPE is generated, but
@@ -28,34 +29,32 @@ import {
// a NEW attribute (or a newly-frozen one) that lands in this bucket flips the // a NEW attribute (or a newly-frozen one) that lands in this bucket flips the
// snapshot test red and forces a reviewer to classify it. Each belongs to one of: // snapshot test red and forces a reviewer to classify it. Each belongs to one of:
// - internal/opaque ids & placeholders (attachmentId, slugId, placeholder, // - internal/opaque ids & placeholders (attachmentId, slugId, placeholder,
// creatorId, anchorId) — no meaningful non-default to assert; // creatorId, anchorId, mime) — no meaningful non-default to assert. These stay
// - dimensions/among the media family with no standalone md form here // frozen: their value is an opaque token carried verbatim, not a round-trip
// (aspectRatio, size, caption, drawio/excalidraw/pdf/video/youtube w/h/align) // shape worth fuzzing;
// — round-trip candidates deferred to a later PR, not silently dropped;
// - ACCEPTED limitations with no md representation (indent, callout.icon, // - ACCEPTED limitations with no md representation (indent, callout.icon,
// orderedList.type, table spans/bg/colwidth); // orderedList.type, table spans/bg/colwidth).
// - PINNED bugs (column.width, orderedList.start) tracked in // The media dimension/family attrs (image/video/youtube/drawio/excalidraw/pdf/
// counterexamples.test.ts. // embed width/height/align/size/aspectRatio/caption/title/alt) that were
// previously "deferred to a later PR" are IMPLEMENTED (value-fuzzed) in THIS PR
// via the OVERRIDES table in attr-arbitraries.ts — they ride in the discriminator
// comment JSON and round-trip byte-stably, so they are no longer allowlisted.
const ATTR_VALUE_FUZZ_ALLOWLIST = new Set<string>([ const ATTR_VALUE_FUZZ_ALLOWLIST = new Set<string>([
'attachment.attachmentId', 'attachment.mime', 'attachment.placeholder', 'attachment.size', 'attachment.attachmentId', 'attachment.mime', 'attachment.placeholder', 'attachment.size',
'audio.attachmentId', 'audio.placeholder', 'audio.size', 'audio.attachmentId', 'audio.placeholder', 'audio.size',
'callout.icon', 'column.width', 'callout.icon',
'drawio.align', 'drawio.alt', 'drawio.aspectRatio', 'drawio.attachmentId', 'drawio.attachmentId',
'drawio.height', 'drawio.size', 'drawio.title', 'drawio.width', 'excalidraw.attachmentId',
'embed.align', 'embed.height', 'embed.width',
'excalidraw.align', 'excalidraw.alt', 'excalidraw.aspectRatio', 'excalidraw.attachmentId',
'excalidraw.height', 'excalidraw.size', 'excalidraw.title', 'excalidraw.width',
'heading.indent', 'heading.indent',
'image.aspectRatio', 'image.attachmentId', 'image.caption', 'image.placeholder', 'image.size', 'image.attachmentId', 'image.placeholder',
'mention.anchorId', 'mention.creatorId', 'mention.slugId', 'mention.anchorId', 'mention.creatorId', 'mention.slugId',
'orderedList.start', 'orderedList.type', 'paragraph.indent', 'orderedList.type', 'paragraph.indent',
'pdf.attachmentId', 'pdf.height', 'pdf.placeholder', 'pdf.size', 'pdf.width', 'pdf.attachmentId', 'pdf.placeholder',
'tableCell.backgroundColor', 'tableCell.backgroundColorName', 'tableCell.colspan', 'tableCell.backgroundColor', 'tableCell.backgroundColorName', 'tableCell.colspan',
'tableCell.colwidth', 'tableCell.rowspan', 'tableCell.colwidth', 'tableCell.rowspan',
'tableHeader.backgroundColor', 'tableHeader.backgroundColorName', 'tableHeader.colspan', 'tableHeader.backgroundColor', 'tableHeader.backgroundColorName', 'tableHeader.colspan',
'tableHeader.colwidth', 'tableHeader.rowspan', 'tableHeader.colwidth', 'tableHeader.rowspan',
'video.align', 'video.aspectRatio', 'video.attachmentId', 'video.placeholder', 'video.size', 'video.attachmentId', 'video.placeholder',
'youtube.align', 'youtube.height', 'youtube.width',
]); ]);
// ── MARK attribute-value coverage ─────────────────────────────────────────── // ── MARK attribute-value coverage ───────────────────────────────────────────
@@ -113,13 +112,20 @@ vi.setConfig({ testTimeout: 30000 });
// Fixed seed so every failure is reproducible; fast-check also prints the // Fixed seed so every failure is reproducible; fast-check also prints the
// shrunk counterexample. numRuns starts modest to keep CI under budget — the // shrunk counterexample. numRuns starts modest to keep CI under budget — the
// issue's CI target is ~300-500 per property; the nightly / PR 3 will crank // issue's CI target is ~300-500 per property. Both are overridable via the
// this up further. Each property runs over the UNION (fc.oneof) of all flat // PROPERTY_SEED / PROPERTY_NUM_RUNS env vars (an invalid/empty value → NaN →
// node generators, so the runs are shared across node types (one test per // falls back to the default below): the nightly cron
// property keeps the jsdom import cost and memory bounded — a per-generator × // (.github/workflows/nightly-property.yml) cranks NUM_RUNS to ~5000 with a
// per-property matrix is ~200 heavy tests that OOMs the worker). // random seed to hunt for deeper counterexamples. Each property runs over the
const SEED = 20250705; // UNION (fc.oneof) of all flat node generators, so the runs are shared across
const NUM_RUNS = 300; // node types (one test per property keeps the jsdom import cost and memory
// bounded — a per-generator × per-property matrix is ~200 heavy tests that
// OOMs the worker).
// An unset/empty/non-numeric value falls back to the default; an explicit 0 is
// honored (a valid fast-check seed) — `Number(x) || default` would swallow it.
// The parser is shared with the nested suite (env-int.ts) and unit-tested there.
const SEED = envInt(process.env.PROPERTY_SEED, 20250705);
const NUM_RUNS = envInt(process.env.PROPERTY_NUM_RUNS, 300);
const P1_GENERATORS = buildGenerators('p1'); const P1_GENERATORS = buildGenerators('p1');
const FUZZ_GENERATORS = buildGenerators('fuzz'); const FUZZ_GENERATORS = buildGenerators('fuzz');
@@ -0,0 +1,193 @@
import { describe, expect, it, vi } from 'vitest';
import fc from 'fast-check';
// Real converters. Importing markdownToProseMirror (transitively, via index)
// mutates the global DOM via jsdom at module load — expected, required for
// @tiptap/html's generateJSON under Node (same as the flat sibling suite).
import {
convertProseMirrorToMarkdown,
markdownToProseMirror,
docsCanonicallyEqual,
canonicalizeContent,
} from '../../src/lib/index.js';
import { firstDivergence } from '../roundtrip-helpers.js';
import { schema, docArb } from './doc-generator.js';
import { envInt } from './env-int.js';
// Each run does a real convert + jsdom parse; give ample headroom so the suite
// is deterministic under parallel worker load (matching the flat sibling suite).
vi.setConfig({ testTimeout: 60000 });
// ---------------------------------------------------------------------------
// #351 PR 2 — GENERATIVE round-trip over NESTED (whole-document) docs produced
// by the ContentMatch random walk (doc-generator.ts). The invariants mirror the
// flat suite, plus a parser-fuzz totality property (P4):
//
// P1 — semantic round-trip: docsCanonicallyEqual(mdToPm(pmToMd(d)), d)
// P2 — byte fixpoint (2nd pass): pmToMd(mdToPm(pmToMd(d))) === pmToMd(d)
// (the FIRST pass may normalize once; the SECOND pass must be a fixpoint)
// P3 — totality: neither converter throws; bounded.
// P4 — parser fuzz totality: for ANY string, markdownToProseMirror does NOT
// throw and returns a SCHEMA-VALID document.
//
// GUARDRAIL: a P1/P2/P3/P4 failure means the generator FOUND A REAL CONVERTER
// BUG. These invariants are kept STRICT — no it.fails / skip / weakening. A
// failure prints the shrunk minimal counterexample for triage.
// ---------------------------------------------------------------------------
// Both are overridable via the PROPERTY_SEED / PROPERTY_NUM_RUNS env vars (an
// invalid/empty value → NaN → falls back to the default below); the nightly
// cron (.github/workflows/nightly-property.yml) cranks NUM_RUNS up with a
// random seed to hunt for deeper counterexamples.
// An unset/empty/non-numeric value falls back to the default; an explicit 0 is
// honored (a valid fast-check seed) — `Number(x) || default` would swallow it.
// The parser is shared with the flat suite (env-int.ts) and unit-tested there.
const SEED = envInt(process.env.PROPERTY_SEED, 20250705);
// The nested walk builds far heavier docs than the flat suite (each P1/P2 run
// parses the emitted markdown through jsdom), so keep the run count moderate to
// hold runtime and worker memory in budget while still exercising deep
// structures. P4 (cheap string parsing) runs at a higher count below.
const NUM_RUNS = envInt(process.env.PROPERTY_NUM_RUNS, 100);
const pmToMd = (doc: unknown): string => convertProseMirrorToMarkdown(doc);
const mdToPm = (md: string): Promise<any> => markdownToProseMirror(md);
async function roundTrip(doc: unknown): Promise<{ md1: string; md2: string; doc2: any }> {
const md1 = pmToMd(doc);
const doc2 = await mdToPm(md1);
const md2 = pmToMd(doc2);
return { md1, md2, doc2 };
}
describe('#351 nested generative round-trip — generator validity', () => {
it('every generated nested doc passes schema.nodeFromJSON(...).check()', () => {
// A nested generator that emits an invalid ProseMirror document is a
// GENERATOR bug — the ContentMatch walk must only produce schema-valid docs.
fc.assert(
fc.property(docArb, (doc) => {
schema.nodeFromJSON(doc).check(); // throws on an invalid doc
return true;
}),
{ numRuns: NUM_RUNS * 2, seed: SEED },
);
});
});
// ── STATUS: P1/P2/P3/P4 all GREEN. The nested generator originally surfaced a
// batch of real converter bugs; all were fixed in the serializer/parser (see the
// #351 hand-off). For the record, the classes it found and that are now fixed:
// • Loose (multi-block) list items / task items / callouts / details bodies were
// joined with a single "\n", so every block after the first merged into the
// first paragraph on re-parse (silent content loss) — now blank-line separated.
// • Paragraph `textAlign` was dropped inside a TIGHT list item (no <p> host).
// • A nested codeBlock lost its trailing newline on the raw-HTML path.
// • Media (embed/video/youtube/drawio/excalidraw) inside `columns` churned a
// default `data-align` and coerced embed's numeric width/height to strings.
// • pageBreak / pageEmbed / subpages / transclusion were dropped when nested in
// blockquote / callout / details / list item (standalone-comment position).
// • Callouts nested in a list item or a blockquote (` > [!type]` / `> > [!type]`)
// were re-parsed as plain blockquotes (prefix-unaware callout preprocessor).
// • Two adjacent sibling lists sharing a marker family (bulletList/taskList →
// `<ul>`; orderedList → `<ol>`) merged into one list on re-parse — and for the
// cross-type case (taskList beside bulletList) the merged `<ul>` LOST every
// taskItem checkbox. The serializer now emits a `<!-- -->` separator between
// such adjacent lists (markdown-converter.ts renderBlockChildren), so they stay
// distinct and round-trip; the generator therefore emits them freely again.
describe('#351 nested generative round-trip — properties', () => {
it('P1 — semantic round-trip: docsCanonicallyEqual(mdToPm(pmToMd(d)), d)', async () => {
await fc.assert(
fc.asyncProperty(docArb, async (doc) => {
const { doc2 } = await roundTrip(doc);
if (!docsCanonicallyEqual(doc2, doc)) {
const div = firstDivergence(
JSON.parse(JSON.stringify(canonicalizeContent(doc2))),
JSON.parse(JSON.stringify(canonicalizeContent(doc))),
);
throw new Error(
`P1 divergence @ ${div?.path}: got=${JSON.stringify(div?.a)} want=${JSON.stringify(div?.b)}`,
);
}
}),
{ numRuns: NUM_RUNS, seed: SEED },
);
});
it('P2 — byte fixpoint: pmToMd(mdToPm(pmToMd(d))) === pmToMd(d)', async () => {
await fc.assert(
fc.asyncProperty(docArb, async (doc) => {
const { md1, md2 } = await roundTrip(doc);
expect(md2).toBe(md1);
}),
{ numRuns: NUM_RUNS, seed: SEED },
);
});
it('P3 — totality: neither converter throws', async () => {
await fc.assert(
fc.asyncProperty(docArb, async (doc) => {
await roundTrip(doc);
}),
{ numRuns: NUM_RUNS, seed: SEED },
);
});
});
// ---------------------------------------------------------------------------
// P4 — parser fuzz. Independent of the doc generator: for ANY input string the
// PARSER (markdownToProseMirror) must be TOTAL — never throw — and must always
// return a schema-valid document. The corpus mixes raw unicode strings with
// strings assembled from markdown-significant fragments (headings, list bullets,
// fences, pipes, thematic breaks, HTML-ish snippets) to probe the block/inline
// parsers on hostile but plausible input.
// ---------------------------------------------------------------------------
const mdFragmentArb: fc.Arbitrary<string> = fc.constantFrom(
'# ', '## ', '### ###', '- ', '* ', '+ ', '1. ', '> ', '>> ',
'```', '```js', '~~~', '---', '***', '___', '| a | b |', '|---|---|',
'[link](http://x)', '![img](http://x)', '**', '__', '~~', '`code`',
'<div>', '</div>', '<b>', '<!-- c -->', '<table>', '<br>', '&amp;',
'\t', '\n', ' ', '\\', '^[fn]', '[^1]:', '- [ ] ', '- [x] ',
'$$', '$x$', ':::', '{.class}', '\u0000', '\uFEFF', '😀', 'مرحبا',
);
// Full-unicode strings (fast-check v4 replaced fullUnicodeString with the
// `unit: 'binary'` string option, which draws over the whole code-point range).
const fullUnicodeStringArb = (max?: number) =>
fc.string({ unit: 'binary', ...(max !== undefined ? { maxLength: max } : {}) });
const assembledMarkdownArb: fc.Arbitrary<string> = fc
.array(fc.oneof(mdFragmentArb, fc.string(), fullUnicodeStringArb(8)), {
minLength: 1,
maxLength: 12,
})
.map((parts) => parts.join(''));
const parserInputArb: fc.Arbitrary<string> = fc.oneof(
{ weight: 2, arbitrary: fc.string() },
{ weight: 2, arbitrary: fullUnicodeStringArb() },
{ weight: 3, arbitrary: assembledMarkdownArb },
{ weight: 1, arbitrary: fc.array(mdFragmentArb, { minLength: 1, maxLength: 8 }).map((p) => p.join('\n')) },
);
describe('#351 parser fuzz — totality on arbitrary input (P4)', () => {
it('P4 — markdownToProseMirror never throws and always returns a schema-valid doc', async () => {
await fc.assert(
fc.asyncProperty(parserInputArb, async (s) => {
let result: any;
try {
result = await mdToPm(s);
} catch (e: any) {
throw new Error(`P4 parser THREW on input ${JSON.stringify(s)}: ${e?.message ?? e}`);
}
try {
schema.nodeFromJSON(result).check();
} catch (e: any) {
throw new Error(
`P4 parser produced an INVALID doc for input ${JSON.stringify(s)}: ${e?.message ?? e}\n` +
`doc=${JSON.stringify(result).slice(0, 600)}`,
);
}
}),
{ numRuns: NUM_RUNS * 2, seed: SEED },
);
});
});
@@ -217,8 +217,9 @@ const colChildOf = (doc2: any) =>
doc2?.content?.[0]?.content?.[0]?.content?.[0]; doc2?.content?.[0]?.content?.[0]?.content?.[0];
describe('converter gap coverage — emission branches (specs 1–11)', () => { describe('converter gap coverage — emission branches (specs 1–11)', () => {
// 1. orderedList renders index+1 and DROPS the start attribute. // 1. orderedList honors the start attribute (FIXED #351): markers count up
it('orderedList start:5 restarts numbering at 1 (start attr ignored)', () => { // from `start` ("5.","6.",…), which CommonMark round-trips.
it('orderedList start:5 numbers from 5 (start attr honored)', () => {
const out = convertProseMirrorToMarkdown( const out = convertProseMirrorToMarkdown(
doc({ doc({
type: 'orderedList', type: 'orderedList',
@@ -229,7 +230,7 @@ describe('converter gap coverage — emission branches (specs 1–11)', () => {
], ],
}), }),
); );
expect(out).toBe('1. a\n2. b'); expect(out).toBe('5. a\n6. b');
}); });
// 2. An empty paragraph contributes an empty segment between two "\n\n" joins. // 2. An empty paragraph contributes an empty segment between two "\n\n" joins.
@@ -374,7 +375,9 @@ describe('converter gap coverage — emission branches (specs 1–11)', () => {
], ],
}), }),
); );
expect(out).toBe('- [ ] top\n - child'); // Block children of a task item are blank-line separated (loose list) per the
// #351 fix; the sublist stays at the fixed 2-column continuation indent.
expect(out).toBe('- [ ] top\n\n - child');
}); });
// 10. A bulletList inside a blockquote: each list line independently prefixed. // 10. A bulletList inside a blockquote: each list line independently prefixed.
@@ -365,7 +365,7 @@ describe('media / attachment / container full-attribute golden coverage', () =>
); );
}); });
it('orderedList inside a column renders via blockToHtml as <ol> (start attr DROPPED) with bold->strong, code->code', () => { it('orderedList inside a column renders via blockToHtml as <ol start="N"> (start attr PRESERVED) with bold->strong, code->code', () => {
const out = c({ const out = c({
type: 'columns', type: 'columns',
attrs: { layout: 'two' }, attrs: { layout: 'two' },
@@ -391,13 +391,13 @@ describe('media / attachment / container full-attribute golden coverage', () =>
}, },
], ],
}); });
// blockToHtml orderedList path emits a plain <ol> with no start attribute, // blockToHtml orderedList path emits <ol start="3"> (FIXED #351), and
// and inlineToHtml maps bold->strong, code->code. // inlineToHtml maps bold->strong, code->code.
expect(out).toContain( expect(out).toContain(
'<ol><li><p><strong>a</strong></p></li><li><p><code>b</code></p></li></ol>', '<ol start="3"><li><p><strong>a</strong></p></li><li><p><code>b</code></p></li></ol>',
); );
// The start:3 attr is NOT preserved in the HTML/column container path. // The start:3 attr IS preserved in the HTML/column container path.
expect(out).not.toContain('start='); expect(out).toContain('start="3"');
}); });
it('hardBreak inside a column renders as <br> via inlineToHtml (not the markdown two-space form)', () => { it('hardBreak inside a column renders as <br> via inlineToHtml (not the markdown two-space form)', () => {
@@ -178,12 +178,12 @@ describe('blockToHtml: heading / codeBlock(lang & no-lang) / bulletList inside c
// A colspan>1 cell forces the WHOLE table to the raw-<table> HTML fallback // A colspan>1 cell forces the WHOLE table to the raw-<table> HTML fallback
// (markdown-converter.ts ~287-331). renderHtmlCell emits colspan + align attrs // (markdown-converter.ts ~287-331). renderHtmlCell emits colspan + align attrs
// (312-316) and renders each block child via blockToHtml. An orderedList child // (312-316) and renders each block child via blockToHtml. An orderedList child
// hits the blockToHtml orderedList branch (726-729), which emits // hits the blockToHtml orderedList branch, which emits
// <ol><li><p>..</p></li>..</ol> — the schema's `start` attr is NOT emitted by // <ol start="N"><li><p>..</p></li>..</ol> — the schema's non-1 `start` attr IS
// this HTML <ol> branch. // emitted by this HTML <ol> branch (FIXED #351).
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
describe('spanned table: renderHtmlCell colspan/align + orderedList block child', () => { describe('spanned table: renderHtmlCell colspan/align + orderedList block child', () => {
it('renders the colspan/align cell with an <ol> (start attr is dropped)', () => { it('renders the colspan/align cell with an <ol start="N"> (start attr preserved)', () => {
const out = convertProseMirrorToMarkdown( const out = convertProseMirrorToMarkdown(
doc({ doc({
type: 'table', type: 'table',
@@ -213,11 +213,11 @@ describe('spanned table: renderHtmlCell colspan/align + orderedList block child'
expect(out).toBe( expect(out).toBe(
'<table><tbody><tr>' + '<table><tbody><tr>' +
'<td colspan="2" align="center">' + '<td colspan="2" align="center">' +
'<ol><li><p>one</p></li><li><p>two</p></li></ol>' + '<ol start="3"><li><p>one</p></li><li><p>two</p></li></ol>' +
'</td>' + '</td>' +
'</tr></tbody></table>', '</tr></tbody></table>',
); );
// The HTML <ol> branch does not propagate the ProseMirror `start` attribute. // The HTML <ol> branch propagates the ProseMirror `start` attribute.
expect(out).not.toContain('start'); expect(out).toContain('start="3"');
}); });
}); });
@@ -198,7 +198,11 @@ describe('convertProseMirrorToMarkdown', () => {
}), }),
); );
// First line carries the marker; the nested list is indented 2 columns. // First line carries the marker; the nested list is indented 2 columns.
expect(out).toBe('- parent\n - child'); // Block children of a list item are separated by a BLANK line (loose list):
// this is the #351 fix — a single "\n" let a following block merge into the
// first paragraph on re-parse (silent content loss). The blank line stays
// inside the item, so the sublist remains nested at the 2-col marker column.
expect(out).toBe('- parent\n\n - child');
}); });
it('nested ordered list indents by the wider 3-col marker width', () => { it('nested ordered list indents by the wider 3-col marker width', () => {
@@ -219,8 +223,9 @@ describe('convertProseMirrorToMarkdown', () => {
], ],
}), }),
); );
// "1. " is 3 columns wide, so the continuation indent is 3 spaces. // "1. " is 3 columns wide, so the continuation indent is 3 spaces. Block
expect(out).toBe('1. parent\n 1. child'); // children are blank-line separated (loose list) per the #351 fix.
expect(out).toBe('1. parent\n\n 1. child');
}); });
}); });
@@ -539,11 +544,12 @@ describe('convertProseMirrorToMarkdown', () => {
); );
// The 10th marker is the 4-column "10. "; the nested sublist line must be // The 10th marker is the 4-column "10. "; the nested sublist line must be
// indented exactly 4 spaces (prefix.length 3 + 1), NOT 3. // indented exactly 4 spaces (prefix.length 3 + 1), NOT 3. Block children are
expect(out).toContain('10. j\n 1. x'); // blank-line separated (loose list) per the #351 fix.
expect(out).toContain('10. j\n\n 1. x');
// Guard against the off-by-one (3-space) regression that would re-parse // Guard against the off-by-one (3-space) regression that would re-parse
// the sublist as loose/sibling content on import. // the sublist as loose/sibling content on import.
expect(out).not.toContain('10. j\n 1. x'); expect(out).not.toContain('10. j\n\n 1. x');
// And the single-digit items keep the narrower 3-column marker (no body // And the single-digit items keep the narrower 3-column marker (no body
// continuation here, but the marker itself must stay "1. ".."9. "). // continuation here, but the marker itself must stay "1. ".."9. ").
expect(out.startsWith('1. a\n2. b\n')).toBe(true); expect(out.startsWith('1. a\n2. b\n')).toBe(true);
@@ -580,17 +586,18 @@ describe('convertProseMirrorToMarkdown', () => {
content: [para(text('line1')), para(text('line2'))], content: [para(text('line1')), para(text('line2'))],
}), }),
); );
// NOTE(review): the spec predicted ':::warning\nline1\n\nline2\n:::' (a // The converter emits an Obsidian-native callout: a `> [!type]` opener plus
// The converter joins the callout's rendered children with a single '\n' // one `>`-prefixed body line per content line. Block children are separated
// and emits an Obsidian-native callout: a `> [!type]` opener plus one // by a blank `>` line (#351 fix): a single '\n' let the two paragraphs merge
// `>`-prefixed body line per content line. We pin the lowercasing // into one on re-parse. We pin the lowercasing (WARNING -> warning) and the
// (WARNING -> warning) and the multi-child join. // blank-line-separated multi-child join.
expect(out).toBe('> [!warning]\n> line1\n> line2'); expect(out).toBe('> [!warning]\n> line1\n>\n> line2');
// The type is lowercased (an uppercase `[!WARNING]` would not re-import). // The type is lowercased (an uppercase `[!WARNING]` would not re-import).
expect(out.startsWith('> [!warning]\n')).toBe(true); expect(out.startsWith('> [!warning]\n')).toBe(true);
expect(out).not.toContain('[!WARNING]'); expect(out).not.toContain('[!WARNING]');
// Both paragraph children are present, each blockquote-prefixed. // Both paragraph children are present, each blockquote-prefixed, blank-`>`
expect(out).toContain('> line1\n> line2'); // separated so they stay distinct paragraphs on re-parse.
expect(out).toContain('> line1\n>\n> line2');
}); });
// Spec 4 — blockquote per-line prefixer over a multi-line nested callout. // Spec 4 — blockquote per-line prefixer over a multi-line nested callout.
@@ -607,13 +614,11 @@ describe('convertProseMirrorToMarkdown', () => {
], ],
}), }),
); );
// NOTE(review): the spec predicted '> :::info\n> a\n>\n> b\n> :::', // The nested callout renders as an Obsidian callout '> [!info]\n> a\n>\n> b'
// assuming the nested callout body contains a blank line between 'a' and // (blank-`>` separated children per the #351 fix). The outer blockquote
// The nested callout renders as an Obsidian callout '> [!info]\n> a\n> b' // prefixer then prefixes each of those lines with '> ' again, yielding a
// (single-'\n' join, no blank line). The outer blockquote prefixer then // doubly-nested blockquote — the per-line-prefix loop over a multi-line child.
// prefixes each of those lines with '> ' again, yielding a doubly-nested expect(out).toBe('> > [!info]\n> > a\n> >\n> > b');
// blockquote — the realistic per-line-prefix loop over a multi-line child.
expect(out).toBe('> > [!info]\n> > a\n> > b');
// Every produced line carries the '> ' prefix (no line escapes to col 0). // Every produced line carries the '> ' prefix (no line escapes to col 0).
for (const line of out.split('\n')) { for (const line of out.split('\n')) {
expect(line.startsWith('>')).toBe(true); expect(line.startsWith('>')).toBe(true);
@@ -0,0 +1,75 @@
import { describe, expect, it } from 'vitest';
import { convertProseMirrorToMarkdown } from '../src/lib/markdown-converter.js';
import { markdownToProseMirror } from '../src/lib/markdown-to-prosemirror.js';
// ---------------------------------------------------------------------------
// #351 regression — DEGENERATE orderedList `start` normalization (DO-1).
//
// Only an INTEGER >= 2 is a valid explicit start. A degenerate start (0,
// negative, fractional, non-number) MUST collapse to the default 1 on export:
// emitting the raw value would produce an untokenizable marker (e.g. "-3.",
// "2.5.") that `marked` cannot parse, causing the WHOLE orderedList (and its
// listItems) to re-import as a PARAGRAPH — structural corruption. This test
// pins that each degenerate start:
// (1) exports to "1."/"2." markers (default numbering),
// (2) re-imports as a VALID `orderedList` (not a paragraph) with `start`
// defaulting, structure preserved,
// (3) is byte-stable (md1 === md2).
//
// Mutation check: revert the converter guard to `Number(raw) || 1` and this
// test goes RED (a fractional/negative start leaks into the marker).
// ---------------------------------------------------------------------------
const makeDoc = (start: unknown) => ({
type: 'doc',
content: [
{
type: 'orderedList',
attrs: { type: null, start },
content: [
{
type: 'listItem',
content: [
{ type: 'paragraph', content: [{ type: 'text', text: 'alpha' }] },
],
},
{
type: 'listItem',
content: [
{ type: 'paragraph', content: [{ type: 'text', text: 'beta' }] },
],
},
],
},
],
});
describe('#351 orderedList degenerate start normalization', () => {
for (const start of [0, -3, 2.5]) {
it(`start=${start} exports as default "1."/"2." markers`, () => {
const md = convertProseMirrorToMarkdown(makeDoc(start));
expect(md).toContain('1. alpha');
expect(md).toContain('2. beta');
// The degenerate value must NOT leak into any marker.
expect(md).not.toMatch(/-?\d*\.\d+\.\s/); // no fractional marker like "2.5."
expect(md).not.toContain('-3.');
});
it(`start=${start} re-imports as a valid orderedList (not a paragraph)`, async () => {
const md = convertProseMirrorToMarkdown(makeDoc(start));
const doc = (await markdownToProseMirror(md)) as any;
const top = doc.content?.[0];
expect(top?.type).toBe('orderedList');
// start defaults (never the degenerate value); tiptap materializes 1.
expect(top?.attrs?.start ?? 1).toBe(1);
expect(top?.content?.length).toBe(2);
});
it(`start=${start} is byte-stable across a re-export`, async () => {
const md1 = convertProseMirrorToMarkdown(makeDoc(start));
const doc = await markdownToProseMirror(md1);
const md2 = convertProseMirrorToMarkdown(doc);
expect(md2).toBe(md1);
});
}
});
-3
View File
@@ -272,9 +272,6 @@ importers:
'@atlaskit/pragmatic-drag-and-drop-live-region': '@atlaskit/pragmatic-drag-and-drop-live-region':
specifier: 1.3.4 specifier: 1.3.4
version: 1.3.4 version: 1.3.4
'@braintree/sanitize-url':
specifier: 7.1.2
version: 7.1.2
'@casl/react': '@casl/react':
specifier: 5.0.1 specifier: 5.0.1
version: 5.0.1(@casl/ability@6.8.0)(react@18.3.1) version: 5.0.1(@casl/ability@6.8.0)(react@18.3.1)