Compare commits

..

4 Commits

Author SHA1 Message Date
agent_coder 65da62c382 fix(#370): ребейз на develop + правки ревью agent_vscode (раунд 3)
Ребейз ветки на текущий gitea/develop (устраняет mergeable:false).
Конфликты разрешены вручную:
- editor-atoms.ts: сохранён type-only импорт Editor (сплит-код с develop),
  добавлен import type HocuspocusProvider из #370.
- collaboration/constants.ts: оставлен EMBED_DEBOUNCE_MS (embed-дебаунс develop),
  убраны более не используемые HISTORY_* (их единственный потребитель — старая
  эвристика computeHistoryJob — удалён в #370), добавлены idle-константы и
  PageHistoryKind.
- persistence.extension.ts: 3-way смёржены метрики/#348/#402 develop поверх
  idle-конвейера #370; computeHistoryJob остаётся idle-версией без остатков.

Миграция переименована 20260705T120000 -> 20260707T120000 (класс #361):
таймстамп был занят perf-indexes на develop; новый строго позже свежайшей
на develop (20260706T120000-search-lookup-trgm). Содержимое не менялось,
внешних ссылок на имя файла нет (Kysely находит по директории).

Документирование (findings 1-2):
- remove-vs-active гонка в idiome remove()->add() enqueuePageHistory: окно,
  где отложенная job уходит в active между remove (проглатывается на active) и
  add (BullMQ отбрасывает add с существующим jobId); ограничено и
  самовосстанавливается (следующий store перевзводит), кроме худшего случая —
  гонящий store был ПОСЛЕДНИМ в сессии: хвостовые правки без trailing-снапшота
  до следующей правки. Явно указано, почему нельзя «унифицировать» с соседним
  embed-дебаунсом (стабильный jobId, без remove).
- допущение single-process у Map idleBurstStart: в памяти процесса, рестарт
  collab теряет метки начала всплеска -> непрерывный всплеск через рестарт может
  ждать до 2x cap. Ограничено и безопасно.

Тест (finding 3): интеграционный тест idle-конвейера против реального BullMQ
(короткие интервалы через jest.mock констант): непрерывный всплеск в неск. cap
-> периодические idle-снапшоты не реже cap и не по одному на store;
прерывистый всплеск -> ровно один trailing-снапшот. Жёсткий teardown
(force-close + settle), чтобы фоновый BullMQ не влиял на соседние suite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 12:18:41 +03:00
agent_coder 8bd883c998 fix(#370): thread trx into addPageWatchers (F7 self-deadlock) + restore contributors on commit-failure (F8) + assert the lock (F9) (review round 2)
The round-1 F3 fix (wrapping the processor's find+save in a locked tx) itself
introduced two regressions:

F7 [CRITICAL] addPageWatchers ran WITHOUT trx inside the tx holding FOR UPDATE on
pages[pageId]. The watcher insert's FK check takes FOR KEY SHARE on the same row,
but on a DIFFERENT pool connection — a true self-deadlock (our tx connection sits
idle-in-transaction awaiting the JS await, the insert connection blocks on the
lock). Now passes trx (addPageWatchers already accepts it and routes it through
insertMany), so the FK lock is taken on the connection that already holds FOR
UPDATE — no self-conflict.

F8 [WARNING] popContributors is a destructive Redis SPOP; the inner catch only
restores on a throw INSIDE the callback. A COMMIT failure throws OUTSIDE it,
rolling the snapshot back while the pop is gone → a retry writes an unattributed
version. Now tracks the popped set and restores it in an outer catch (idempotent
SADD), leaving BullMQ to retry with attribution intact.

F9 [WARNING] The spec asserted saveHistory args with a loosened objectContaining
that stopped verifying trx, and never pinned withLock/trx on findById or the trx
on addPageWatchers — which is exactly why F7 slipped. Restored the exact
saveHistory(trx) assertion and added findById({withLock,trx}) + addPageWatchers
trx assertions (the latter would have caught F7), plus a commit-failure test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 11:46:45 +03:00
agent_coder f7a91eb706 fix(#370): ES2021-safe spec, source-specific idle ceiling, processor lock, tested index mapping (review round 1)
F1 [BLOCKER] persistence-store.spec used Array.prototype.at(-1) (ES2022) but the
server targets ES2021, so server tsc failed (TS2550) and ts-jest could not
compile the suite — 22 core manual-save/idle/boundary tests silently did not run
in CI. Replaced with [length - 1] index access.

F2 [WARNING] The idle burst-reset used a hardcoded IDLE_MAX_WAIT_USER for both
tiers, but computeHistoryJob's ceiling is source-specific. On a continuously
agent-edited page the burst marker stayed stale for 5..10m, forcing delay=0 on
every store and writing one idle row per store — the exact per-store bloat the
debounce prevents. The reset now uses the same source-specific max-wait.

F3 [WARNING] The processor did an unlocked findPageLastHistory -> saveHistory,
which TOCTOU-races a concurrent manual-save (that runs under a page-row lock),
producing two page_history rows with identical content (one idle, one manual) and
defeating promote-not-dup. The snapshot decision is now wrapped in executeTx with
the same page-row lock, so the second writer observes the first's committed row
and the isDeepStrictEqual gate collapses the duplicate.

F4 [WARNING] The risky client filtered-index -> full-list mapping had no tests.
Extracted it to a pure resolvePrevSnapshotId(fullItems, id) helper (diff/restore
baseline against the true previous snapshot in the FULL list, never the previous
visible version) and unit-tested it; removed the now-vestigial index threading.

F5/F6 [low] Renamed the misleading ceiling test + fixed its comment; added a
CHANGELOG entry for the user-facing versioning feature.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 11:46:45 +03:00
agent_coder 9f2d03ce6b feat(#370): page-history intentionality tiers — kind column + intentional/idle/boundary triggers (PR-1 core)
PR-1 'core' of #370: introduces page_history.kind ('manual'|'agent'|'idle'|
'boundary'; legacy null = autosave) and rebuilds the snapshot triggers around a
three-tier intentionality model. Draft durability (pages/ydoc hocuspocus
autosave) is unchanged; only the frequency and labelling of history points change.

- Migration 20260705T120000: page_history.kind nullable varchar(20), no default.
- Manual Save: one stateless 'save-version' path for human AND agent; kind is
  derived SERVER-SIDE from the signed context.actor (never the payload), readOnly
  connections rejected, the fresh ydoc runs through the existing store path (no
  REST race), then broadcasts version.saved.
- Idle-flush: trailing debounce (one BullMQ job per page, remove-then-readd) with
  IDLE_INTERVAL_USER=60m / AGENT=15m AND a max-wait ceiling
  (IDLE_MAX_WAIT_USER=10m / AGENT=5m) so a continuous editing session can't starve
  the autosnapshot (review round-1 WARNING).
- Boundary: generalized from the user→agent special-case to ANY lastUpdatedSource
  transition (user↔agent↔git), same isDeepStrictEqual gate — covers git-sync free.
- Removed the agent delay=0 fast path and the old HISTORY_FAST_* constants; the
  agent joins the common idle pipeline.
- Promote-not-dup: a manual save on unchanged content promotes the latest
  autosave's kind in place (or no-ops if already manual) instead of duplicating a
  heavy content row.
- Client: mod+S hotkey + menu button (hidden when readOnly), history-panel kind
  badges, dimmed autosaves, a 'versions only' filter (indices map to the full
  list so diff/restore still target the true previous snapshot), live refresh on
  version.saved.

Internal review: APPROVE-with-suggestions; the round-1 WARNING (idle starvation)
is fixed here via the max-wait ceiling, and the generalized-boundary + ceiling
behaviours are pinned with new tests (115 collab/repo specs green, server tsc 0).

Deferred to later PRs: shares.published_mode (PR-2), the save_page_version MCP
tool + role prompts (PR-3), actor='git' wiring into #359 (PR-4).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 11:46:45 +03:00
71 changed files with 1551 additions and 3117 deletions
-14
View File
@@ -335,20 +335,6 @@ MCP_DOCMOST_PASSWORD=
# VictoriaMetrics/Prometheus reaching it as <host>:<port>/metrics.
# METRICS_PORT=9464
#
# METRICS_BIND — interface the /metrics listener binds to. DEFAULT 127.0.0.1
# (loopback only), so the unauthenticated endpoint is NOT exposed on all
# interfaces. If the scraper runs in a SEPARATE container and reaches this as
# docmost:9464, set METRICS_BIND=0.0.0.0 — but then also set METRICS_TOKEN
# and/or keep the port on a private network, since /metrics is otherwise open.
# METRICS_BIND=127.0.0.1
#
# METRICS_TOKEN — optional Bearer token guarding /metrics. When set, every
# scrape MUST send `Authorization: Bearer <token>` (others get 401). Configure
# the scraper with the same bearer token (e.g. VictoriaMetrics/vmagent
# `bearer_token`, Prometheus `authorization.credentials`). Leave unset only
# when the endpoint is bound to loopback or an otherwise-trusted network.
# METRICS_TOKEN=
#
# 2) CLIENT_TELEMETRY_ENABLED — the public client perf-telemetry sink.
# OFF by default. When true, the unauthenticated POST /api/telemetry/vitals
# endpoint is registered and browsers collect + send web-vitals / editor
+1 -1
View File
@@ -470,7 +470,7 @@ Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirro
- **Errors must never be swallowed or shown as generic messages.** Every caught error MUST (1) be logged in full to the console/logger — error name, message, stack, `cause`, and (for HTTP/provider failures) the status code and response body — and (2) be surfaced to the user with a *specific, human-readable explanation of what actually went wrong*, never a bare generic string like "Something went wrong" / "Could not start recording" / "Transcription failed". Include the real reason (the underlying error/provider message) in the user-facing text. On the server, wrap third-party/provider failures with `describeProviderError` (or equivalent) and rethrow as a meaningful HTTP status + message — never let them collapse into an opaque 500. On the client, `console.error(<context>, err)` the raw error AND show the extracted reason (e.g. `err.response?.data?.message`, or the error `name: message`) in the notification.
- The version string shown in the UI comes from `APP_VERSION` (CI/Docker) or `git describe --tags --always` (local), resolved in `vite.config.ts` — not from `package.json`.
- Server TS config is permissive (`noImplicitAny: false`, `strictNullChecks: false`, `no-explicit-any` lint disabled). Follow the existing relaxed style rather than tightening types broadly.
- Dependency versions are heavily pinned via `pnpm.overrides` and `pnpm.patchedDependencies` (`scimmy`, `yjs`, `ai`) in the root `package.json`. Don't bump pinned/patched deps casually; the patches and overrides exist for compatibility/security reasons. The `ai@6.0.134` patch carries TWO independent server fixes, each with its own tripwire test: (1) it disables the SDK's O(n²) cumulative `partialOutput` accumulation when no output strategy is requested (server heap OOM on long agent runs, #184; tripwire: `apps/server/src/integrations/ai/ai-sdk-partial-output.patch.spec.ts`); (2) it fixes `writeToServerResponse`'s drain-hang — the loop awaited only `"drain"` under backpressure, so a mid-write client disconnect parked the pipe forever and leaked the reader/buffers until restart; it now races `"drain"` against `"close"`/`"error"`, cancels the reader on disconnect, and swallows the fire-and-forget read rejection (#486; tripwire: `apps/server/src/integrations/ai/ai-sdk-drain-hang.patch.spec.ts`). Both tripwires assert BOTH installed dist builds carry their patch marker. The patch MUST be re-created via `pnpm patch` when bumping `ai`.
- Dependency versions are heavily pinned via `pnpm.overrides` and `pnpm.patchedDependencies` (`scimmy`, `yjs`, `ai`) in the root `package.json`. Don't bump pinned/patched deps casually; the patches and overrides exist for compatibility/security reasons. The `ai@6.0.134` patch disables the SDK's O(n²) cumulative `partialOutput` accumulation when no output strategy is requested (server heap OOM on long agent runs, #184; tripwire test: `apps/server/src/integrations/ai/ai-sdk-partial-output.patch.spec.ts`) — it MUST be re-created via `pnpm patch` when bumping `ai`.
- **The MCP tool inventory in `SERVER_INSTRUCTIONS` is GENERATED from the registry** (`packages/mcp/src/server-instructions.ts`: `buildToolInventory()` over `SHARED_TOOL_SPECS`) and spliced into the hand-written routing prose (`ROUTING_PROSE`). So adding/renaming/removing a **shared** spec in `packages/mcp/src/tool-specs.ts` auto-updates the `<tool_inventory>` — no manual `SERVER_INSTRUCTIONS` edit needed. Only an **inline** MCP-only tool (those registered via `server.registerTool(...)` in `index.ts`, not through the registry) needs a one-line entry in `INLINE_MCP_INVENTORY`. Enforced by `packages/mcp/test/unit/tool-inventory.test.mjs`, which fails when a registered tool is missing from the generated inventory (there is no `EXCEPTIONS` opt-out anymore — every tool must appear). Update `ROUTING_PROSE` when a tool's *intent guidance* (when-to-use) changes. `packages/mcp/build/` is gitignored and rebuilt in CI/Docker via `pnpm build` (same convention as `git-sync`/`prosemirror-markdown`) — never commit it; rebuild locally after editing to run the tests.
## CI / release
+8 -63
View File
@@ -115,20 +115,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
the old ProseMirror-JSON output. Released together with the `#411`/`#412`
breaking window so external configs break exactly once. (#413)
- **The Prometheus `/metrics` listener now binds to `127.0.0.1` (loopback) by
default instead of `0.0.0.0` (all interfaces).** This closes an unauthenticated
endpoint that was previously reachable on every interface. **DEPLOY MIGRATION —
cross-container scraping breaks silently otherwise:** if your scraper runs in a
SEPARATE container and reaches the app as `docmost:9464` (the exact topology the
old `0.0.0.0` hardcode served), you MUST now set `METRICS_BIND=0.0.0.0` — and,
because that re-exposes the endpoint, also set `METRICS_TOKEN=<secret>` and
configure the scraper with a matching Bearer token. Without `METRICS_BIND`, the
scraper can no longer connect and metrics go dark with no error. See the
`METRICS_BIND` / `METRICS_TOKEN` block in `.env.example` for the migration.
Same-host (loopback) scrapers need no change. (#486)
### Added
- **Save intentional page versions.** Press `Cmd/Ctrl+S` (or use the page menu)
to save a named version of a page. The history panel now distinguishes
intentional versions (a "Saved" / "Agent version" badge) from automatic
snapshots, dims autosaves, and offers an "Only versions" filter. Automatic
snapshots switched from a fixed interval to a trailing idle-flush with a
max-wait ceiling, and a boundary snapshot is pinned whenever the editing source
changes (e.g. a person's edits followed by the AI agent). (#370)
- **Place several images side by side in a row.** A new "Inline (side by
side)" alignment mode in the image bubble menu renders consecutive inline
images as a row that wraps onto the next line on narrow screens. The row is
@@ -322,39 +318,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`tee()` branch of the stream result — a ~20-step, ~28k-chunk agent run
retained ~1.7 GB and OOM'd the 2 GB JS heap. Streaming granularity is
unchanged; the patch must be re-created if `ai` is ever bumped. (#184)
- **The server no longer leaks a hung stream pipe on every mid-run client
disconnect.** The same `ai@6.0.134` pnpm patch now also fixes the SDK's
`writeToServerResponse`, which awaited only a `"drain"` event under
backpressure: when a client disconnected mid-write the socket never drained, so
the write loop parked forever, `response.end()` was unreachable, and the stream
reader plus buffered chunks were pinned until process restart (every mid-run
disconnect in autonomous mode leaked one). The patch races `"drain"` against
`"close"`/`"error"`, cancels the reader and ends the response on disconnect, and
swallows the fire-and-forget read rejection instead of crashing on an
unhandledRejection. (#486)
- **A failed autonomous agent-run start no longer becomes an unstoppable ghost
run.** When `beginRun` failed for a transient reason (e.g. a DB-pool blip),
the turn previously continued with NO run row — invisible to `/stop`, not
aborted on disconnect, and able to slip a second run past the one-run-per-chat
gate, leaving an unstoppable run until restart. The turn now fails fast with an
honest `503 A_RUN_BEGIN_FAILED` before the first byte (no orphan state), and the
client shows a "temporary — please try again" message instead of a misleading
"provider not configured". (#486)
- **A pathological draw.io graph can no longer wedge the whole server.** The ELK
auto-layout (`layout:"elk"`) ran elkjs synchronously on the main event loop, so
a graph at the node/edge cap blocked ALL HTTP/SSE/loopback traffic while it
churned — and the old `setTimeout` "timeout" could never fire because the same
thread was blocked. Layout now runs in a worker thread with the timeout enforced
by `worker.terminate()`; the main loop stays responsive. (#486)
- **The `/health` Redis probe no longer leaks a client on every tick while Redis
is down.** It built a new `ioredis` client per probe and disconnected it only on
success, so during an outage each health tick added another forever-reconnecting
client (an unbounded handle leak). A single long-lived probe client is now
reused and closed on shutdown. (#486)
- **Internal links in exported Markdown no longer lose their visible text.** A
link whose target page name had no file extension (e.g. a bare title) was
collapsed to empty text during export, producing an unclickable, label-less
@@ -431,24 +394,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
share); any other value now returns the generic "not found" instead of
serving the page. (#218)
- **Tool and provider error text no longer leaks to anonymous readers in the
public-share AI chat.** A failing tool's raw error (which could carry an
internal page title or a stack fragment) and a provider error (which bundles the
provider `statusCode` and response body — potentially the internal baseUrl or
model name) were streamed verbatim to the anonymous reader over SSE. Errors are
now sanitized at the source: the share toolset collapses any unclassified tool
error to a safe generic string (safe, classified tool messages still pass
through for the model's self-correction), and the anonymous stream `onError`
maps provider failures to a fixed set of neutral strings — the full detail goes
only to the server log. A UI render gate is layered on top. (closes #394)
- **The Prometheus `/metrics` endpoint can now require Bearer authentication and
is loopback-bound by default.** Previously it listened on all interfaces with no
auth. Setting `METRICS_TOKEN` requires every scrape to present
`Authorization: Bearer <token>` (compared in constant time), and the listener
defaults to `127.0.0.1` (see the Breaking Changes entry for the cross-container
migration). (#486)
## [0.94.0] - 2026-06-26
This release makes AI chat durable and fast: assistant turns are persisted to
@@ -1418,5 +1418,14 @@
"The commented text changed since this suggestion was made; it was not applied.": "The commented text changed since this suggestion was made; it was not applied.",
"Dismiss": "Dismiss",
"Suggestion dismissed": "Suggestion dismissed",
"Failed to dismiss suggestion": "Failed to dismiss suggestion"
"Failed to dismiss suggestion": "Failed to dismiss suggestion",
"Save version": "Save version",
"Ctrl+S": "Ctrl+S",
"Version saved": "Version saved",
"Already saved as the latest version": "Already saved as the latest version",
"Agent version": "Agent version",
"Boundary": "Boundary",
"Autosave": "Autosave",
"Only versions": "Only versions",
"No saved versions yet.": "No saved versions yet."
}
@@ -1281,5 +1281,14 @@
"The commented text changed since this suggestion was made; it was not applied.": "Прокомментированный текст изменился после создания предложения; оно не было применено.",
"Dismiss": "Не применять",
"Suggestion dismissed": "Предложение отклонено",
"Failed to dismiss suggestion": "Не удалось отклонить предложение"
"Failed to dismiss suggestion": "Не удалось отклонить предложение",
"Save version": "Сохранить версию",
"Ctrl+S": "Ctrl+S",
"Version saved": "Версия сохранена",
"Already saved as the latest version": "Уже сохранено как последняя версия",
"Agent version": "Версия агента",
"Boundary": "Граница",
"Autosave": "Автосейв",
"Only versions": "Только версии",
"No saved versions yet.": "Пока нет сохранённых версий."
}
@@ -203,52 +203,6 @@ describe("ChatThread — send now (#198)", () => {
});
});
// #486: the final onFinish -> flushNext() must be gated on the live-mount flag.
// A clean onFinish can land AFTER the thread unmounts (New-chat / chat-switch
// mid-stream — the async attach/resume settles late); flushing then dequeues and
// re-POSTs a queued message from an abandoned thread (a "ghost" send).
describe("ChatThread — onFinish flush gated on mount (#486)", () => {
beforeEach(resetState);
afterEach(cleanup);
it("a clean onFinish WHILE MOUNTED flushes the queued message (control)", () => {
renderThread();
fireEvent.click(screen.getByTestId("queue-btn")); // enqueue "queued text"
expect(h.state.sendMessage).not.toHaveBeenCalled();
act(() => {
h.state.onFinish?.({
message: { id: "a", role: "assistant", parts: [] },
isAbort: false,
isDisconnect: false,
isError: false,
});
});
// Mounted: the queue flushes normally.
expect(h.state.sendMessage).toHaveBeenCalledWith({ text: "queued text" });
});
it("a clean onFinish AFTER unmount does NOT flush (no ghost send)", () => {
const { unmount } = renderThread();
fireEvent.click(screen.getByTestId("queue-btn")); // enqueue "queued text"
h.state.sendMessage.mockClear();
// Chat switched away mid-stream: the streamer unmounts...
unmount();
// ...and a late, clean onFinish lands on the abandoned thread.
act(() => {
h.state.onFinish?.({
message: { id: "a", role: "assistant", parts: [] },
isAbort: false,
isDisconnect: false,
isError: false,
});
});
// Gated on mountedRef: NOTHING is sent from the dead thread.
expect(h.state.sendMessage).not.toHaveBeenCalled();
});
});
// #396: in autonomous mode a live sendNow must additionally request the
// AUTHORITATIVE server stop of the detached run (a local abort is only a client
// disconnect the server ignores) and arm a bounded 409 retry so the re-POST
@@ -659,13 +659,7 @@ export default function ChatThread({
return;
}
if (isAbort || isDisconnect || isError) return;
// Gate the final flush on the live-mount flag (#486): a clean onFinish can
// land AFTER this thread unmounted (a New-chat / chat-switch mid-stream —
// the async attach/resume settles late). Flushing then dequeues and POSTs a
// queued message from an abandoned thread — a "ghost" send / ghost chat.
// Every other queue side effect already guards on mountedRef; this last one
// was the gap.
if (mountedRef.current) flushNext();
flushNext();
},
// `onError` runs in addition to `onFinish` (which ai@6 also calls on error).
// Log the raw failure here for devtools; the UI shows a friendly classified
@@ -47,13 +47,6 @@ interface MessageItemProps {
* agent's raw query/argument text.
*/
showInput?: boolean;
/**
* Forwarded to ToolCallCard: whether a failed tool card renders its raw
* errorText. Defaults to true (internal chat). The public share passes false so
* internal detail in a tool error is never painted (belt to the server-side
* byte sanitization).
*/
showErrors?: boolean;
/**
* Neutralize internal/relative markdown links in the rendered answer (drop
* their href so they become inert text). Defaults to false (internal chat,
@@ -132,7 +125,6 @@ function MessageItem({
message,
showCitations = true,
showInput = true,
showErrors = true,
neutralizeInternalLinks = false,
assistantName,
turnStreaming = false,
@@ -227,7 +219,6 @@ function MessageItem({
part={part as unknown as ToolUiPart}
showCitations={showCitations}
showInput={showInput}
showErrors={showErrors}
/>
);
}
@@ -293,7 +284,6 @@ export function arePropsEqual(
prev.signature === next.signature &&
prev.showCitations === next.showCitations &&
prev.showInput === next.showInput &&
prev.showErrors === next.showErrors &&
prev.neutralizeInternalLinks === next.neutralizeInternalLinks &&
prev.assistantName === next.assistantName &&
// The turn-end flip re-renders every row once (cheap, terminal event) —
@@ -32,12 +32,6 @@ interface MessageListProps {
* doesn't see the agent's raw query/argument text.
*/
showInput?: boolean;
/**
* Forwarded to MessageItem -> ToolCallCard: whether a failed tool card renders
* its raw errorText. Defaults to true (internal chat). The public share passes
* false so internal detail in a tool error is never painted.
*/
showErrors?: boolean;
/**
* Forwarded to MessageItem: neutralize internal/relative markdown links in
* the rendered answers (drop their href so they render as inert text).
@@ -133,7 +127,6 @@ export default function MessageList({
emptyState,
showCitations = true,
showInput = true,
showErrors = true,
neutralizeInternalLinks = false,
assistantName,
}: MessageListProps) {
@@ -224,7 +217,6 @@ export default function MessageList({
signature={messageSignature(message)}
showCitations={showCitations}
showInput={showInput}
showErrors={showErrors}
neutralizeInternalLinks={neutralizeInternalLinks}
assistantName={assistantName}
// Turn-level liveness, gated to the TAIL row: only the tail message
@@ -30,16 +30,6 @@ interface ToolCallCardProps {
* the extra summary line, leaving the card (the action log) intact.
*/
showInput?: boolean;
/**
* Whether to render the tool's raw errorText on a failed call. Defaults to true
* (the internal chat, where the operator may debug). The public share passes
* false: a tool error string can carry internal detail (an internal page title,
* a stack fragment, a provider message). This is the RENDER gate only — the
* authoritative fix also sanitizes the bytes server-side (see
* PublicShareChatToolsService.forShare), so a share reader never receives raw
* error text over the wire, not just never sees it painted (#394).
*/
showErrors?: boolean;
}
/**
@@ -51,7 +41,6 @@ export default function ToolCallCard({
part,
showCitations = true,
showInput = true,
showErrors = true,
}: ToolCallCardProps) {
const { t } = useTranslation();
const toolName = getToolName(part);
@@ -85,7 +74,7 @@ export default function ToolCallCard({
</Text>
)}
{state === "error" && showErrors && part.errorText && (
{state === "error" && part.errorText && (
<Text size="xs" c="red" mt={2}>
{part.errorText}
</Text>
@@ -23,25 +23,6 @@ describe("describeChatError", () => {
});
});
it("classifies an A_RUN_BEGIN_FAILED 503 as a temporary run-start failure, NOT provider-not-configured (#486)", () => {
// The FULL real body the server writes for a beginRun failure: a
// ServiceUnavailableException(object) whose response is serialized verbatim
// onto the raw socket, self-describing statusCode 503 + the run-start code.
const body =
'{"message":"Could not start the agent run. This is usually temporary — please try again.","code":"A_RUN_BEGIN_FAILED","statusCode":503}';
expect(describeChatError(body, t)).toEqual({
title: "Could not start the run",
detail:
"The agent run could not be started. This is usually temporary — please try again.",
});
// ORDER GUARD: even though the body ALSO carries statusCode 503 (which the
// generic branch matches), the A_RUN_BEGIN_FAILED branch runs first, so it is
// never mislabeled "AI provider not configured".
expect(describeChatError(body, t).title).not.toBe(
"AI provider not configured",
);
});
it("classifies a dropped connection (ECONNRESET) as a lost-connection error", () => {
expect(
describeChatError("Cannot connect to API: read ECONNRESET", t).title,
@@ -24,21 +24,6 @@ export function describeChatError(
): ChatErrorView {
const msg = message ?? "";
// Our own "could not start the run" gate (A_RUN_BEGIN_FAILED, #486): a 503
// whose body carries this code is a TEMPORARY server-side failure while
// starting the run (e.g. a DB-pool blip), NOT an unconfigured provider. It MUST
// be matched STRICTLY BEFORE the generic 503 branch below, which would
// otherwise mislabel it "The AI provider is not configured" and tell the user
// to call an admin instead of just retrying.
if (/"code"\s*:\s*"A_RUN_BEGIN_FAILED"/.test(msg)) {
return {
title: t("Could not start the run"),
detail: t(
"The agent run could not be started. This is usually temporary — please try again.",
),
};
}
if (/"statusCode"\s*:\s*403\b/.test(msg)) {
return {
title: t("AI chat is disabled"),
@@ -3,11 +3,20 @@ import { atom } from "jotai";
// import would drag the whole @tiptap/core engine into the eager graph of every
// shell component that reads one of these atoms.
import type { Editor } from "@tiptap/core";
import type { HocuspocusProvider } from "@hocuspocus/provider";
import { PageEditMode } from "@/features/user/types/user.types.ts";
import type { DictationUnavailableReason } from "@/features/dictation/dictation-status";
export const pageEditorAtom = atom<Editor | null>(null);
// #370 — the active page's collab provider, published by the page editor so the
// header menu can emit the "save-version" stateless signal (Cmd+S / button).
// Null when the page is read-only / collab isn't connected. A typed initial
// value (rather than an explicit generic) keeps jotai's overload resolution on
// the writable PrimitiveAtom branch.
const initialCollabProvider: HocuspocusProvider | null = null;
export const collabProviderAtom = atom(initialCollabProvider);
export const titleEditorAtom = atom<Editor | null>(null);
export const readOnlyEditorAtom = atom<Editor | null>(null);
@@ -31,11 +31,18 @@ import { useAtom, useAtomValue, useSetAtom } from "jotai";
import useCollaborationUrl from "@/features/editor/hooks/use-collaboration-url";
import { currentUserAtom } from "@/features/user/atoms/current-user-atom";
import {
collabProviderAtom,
currentPageEditModeAtom,
dictationAvailabilityAtom,
pageEditorAtom,
yjsConnectionStatusAtom,
} from "@/features/editor/atoms/editor-atoms";
import { notifications } from "@mantine/notifications";
import {
VERSION_SAVED_MESSAGE_TYPE,
type VersionSavedMessage,
saveVersionPending,
} from "@/features/page-history/version-messages";
import { asideStateAtom } from "@/components/layouts/global/hooks/atoms/sidebar-atom";
import {
activeCommentIdAtom,
@@ -124,6 +131,7 @@ export default function PageEditor({
const [currentUser] = useAtom(currentUserAtom);
const [, setEditor] = useAtom(pageEditorAtom);
const setCollabProvider = useSetAtom(collabProviderAtom);
const [, setAsideState] = useAtom(asideStateAtom);
const [, setActiveCommentId] = useAtom(activeCommentIdAtom);
const [showCommentPopup, setShowCommentPopup] = useAtom(showCommentPopupAtom);
@@ -181,6 +189,24 @@ export default function PageEditor({
const onStatelessHandler = ({ payload }: onStatelessParameters) => {
try {
const message = JSON.parse(payload);
// #370 — a version was saved somewhere; live-refresh the history panel
// on every client. Only the client that pressed Save (tracked by the
// module-level flag) shows the confirmation toast.
if (message?.type === VERSION_SAVED_MESSAGE_TYPE) {
const versionMsg = message as VersionSavedMessage;
queryClient.invalidateQueries({
queryKey: ["page-history-list"],
});
if (saveVersionPending.current) {
saveVersionPending.current = false;
notifications.show({
message: versionMsg.alreadySaved
? t("Already saved as the latest version")
: t("Version saved"),
});
}
return;
}
if (message?.type !== "page.updated" || !message.updatedAt) return;
const pageData = queryClient.getQueryData<IPage>(["pages", slugId]);
if (pageData) {
@@ -238,12 +264,16 @@ export default function PageEditor({
local.on("synced", onLocalSyncedHandler);
providersRef.current = { socket, local, remote };
// #370 — publish the provider so the header menu can emit save-version.
setCollabProvider(remote);
setProvidersReady(true);
} else {
setCollabProvider(providersRef.current.remote);
setProvidersReady(true);
}
// Only destroy on final unmount
return () => {
setCollabProvider(null);
providersRef.current?.socket.destroy();
providersRef.current?.remote.destroy();
providersRef.current?.local.destroy();
@@ -1,4 +1,11 @@
import { Text, Group, UnstyledButton, Avatar, Tooltip } from "@mantine/core";
import {
Text,
Group,
UnstyledButton,
Avatar,
Tooltip,
Badge,
} from "@mantine/core";
import { CustomAvatar } from "@/components/ui/custom-avatar.tsx";
import { AgentAvatarStack } from "@/components/ui/agent-avatar-stack.tsx";
import { formattedDate } from "@/lib/time";
@@ -7,36 +14,59 @@ import clsx from "clsx";
import { IPageHistory } from "@/features/page-history/types/page.types";
import { memo, useCallback } from "react";
import { useSetAtom } from "jotai";
import { useTranslation } from "react-i18next";
import { historyAtoms } from "@/features/page-history/atoms/history-atoms.ts";
const MAX_VISIBLE_AVATARS = 5;
/**
* #370 — map a snapshot's intentionality tier to its badge. `version: true`
* marks the intentional points (manual / agent); autosaves (boundary / idle /
* legacy null) are non-versions and get dimmed in the list.
*/
type HistoryKindMeta = { labelKey: string; color: string; version: boolean };
export function historyKindMeta(kind?: string | null): HistoryKindMeta {
switch (kind) {
case "manual":
return { labelKey: "Saved", color: "blue", version: true };
case "agent":
return { labelKey: "Agent version", color: "violet", version: true };
case "boundary":
return { labelKey: "Boundary", color: "gray", version: false };
default: // "idle" | null | undefined (legacy autosave)
return { labelKey: "Autosave", color: "gray", version: false };
}
}
interface HistoryItemProps {
historyItem: IPageHistory;
index: number;
onSelect: (id: string, index: number) => void;
onHover?: (id: string, index: number) => void;
// The previous snapshot for diff/restore is resolved by id from the FULL list
// in the parent (resolvePrevSnapshotId), so the item only needs to report its
// own id — never a list index (which would be the filtered-view index).
onSelect: (id: string) => void;
onHover?: (id: string) => void;
onHoverEnd?: () => void;
isActive: boolean;
}
const HistoryItem = memo(function HistoryItem({
historyItem,
index,
onSelect,
onHover,
onHoverEnd,
isActive,
}: HistoryItemProps) {
const setHistoryModalOpen = useSetAtom(historyAtoms);
const { t } = useTranslation();
const kindMeta = historyKindMeta(historyItem.kind);
const handleClick = useCallback(() => {
onSelect(historyItem.id, index);
}, [onSelect, historyItem.id, index]);
onSelect(historyItem.id);
}, [onSelect, historyItem.id]);
const handleMouseEnter = useCallback(() => {
onHover?.(historyItem.id, index);
}, [onHover, historyItem.id, index]);
onHover?.(historyItem.id);
}, [onHover, historyItem.id]);
const contributors = historyItem.contributors;
const hasContributors = contributors && contributors.length > 0;
@@ -49,8 +79,20 @@ const HistoryItem = memo(function HistoryItem({
onMouseEnter={handleMouseEnter}
onMouseLeave={onHoverEnd}
className={clsx(classes.history, { [classes.active]: isActive })}
// #370 — dim autosnapshots so intentional versions stand out.
style={{ opacity: kindMeta.version ? 1 : 0.55 }}
>
<Text size="sm">{formattedDate(new Date(historyItem.createdAt))}</Text>
<Group gap={6} wrap="nowrap" justify="space-between">
<Text size="sm">{formattedDate(new Date(historyItem.createdAt))}</Text>
<Badge
size="xs"
radius="sm"
variant={kindMeta.version ? "filled" : "light"}
color={kindMeta.color}
>
{t(kindMeta.labelKey)}
</Badge>
</Group>
<Group gap={6} wrap="nowrap" mt={4}>
{hasContributors ? (
@@ -9,7 +9,7 @@ import {
historyAtoms,
} from "@/features/page-history/atoms/history-atoms";
import { useAtom, useSetAtom } from "jotai";
import { useCallback, useEffect, useMemo, useRef } from "react";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import {
Button,
ScrollArea,
@@ -17,9 +17,12 @@ import {
Divider,
Loader,
Center,
Switch,
Text,
} from "@mantine/core";
import { useTranslation } from "react-i18next";
import { useHistoryRestore } from "@/features/page-history/hooks";
import { resolvePrevSnapshotId } from "@/features/page-history/utils/resolve-prev-snapshot";
const PREFETCH_DELAY_MS = 150;
@@ -47,6 +50,23 @@ function HistoryList({ pageId }: Props) {
[pageHistoryData],
);
// #370 — "only versions" filter: hide autosnapshots (idle/boundary/legacy
// null), keep only intentional points (manual/agent). Filtering is over the
// already-loaded pages; the diff/restore still targets the true previous
// snapshot, so items carry their index within the FULL list.
const [onlyVersions, setOnlyVersions] = useState(false);
const isVersion = useCallback(
(kind?: string | null) => kind === "manual" || kind === "agent",
[],
);
const visibleItems = useMemo(
() =>
onlyVersions
? historyItems.filter((item) => isVersion(item.kind))
: historyItems,
[historyItems, onlyVersions, isVersion],
);
const loadMoreRef = useRef<HTMLDivElement>(null);
const prefetchTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
@@ -60,11 +80,13 @@ function HistoryList({ pageId }: Props) {
}, []);
const handleHover = useCallback(
(historyId: string, index: number) => {
(historyId: string) => {
clearPrefetchTimeout();
prefetchTimeoutRef.current = setTimeout(() => {
prefetchPageHistory(historyId);
const prevId = historyItems[index + 1]?.id;
// The true previous snapshot in the FULL list (not the previous visible
// one under the "only versions" filter).
const prevId = resolvePrevSnapshotId(historyItems, historyId);
if (prevId) {
prefetchPageHistory(prevId);
}
@@ -78,9 +100,11 @@ function HistoryList({ pageId }: Props) {
}, [clearPrefetchTimeout]);
const handleSelect = useCallback(
(id: string, index: number) => {
(id: string) => {
setActiveHistoryId(id);
setActiveHistoryPrevId(historyItems[index + 1]?.id ?? "");
// Baseline = true previous snapshot in the FULL list, so the "only
// versions" filter never diffs/restores against the wrong item.
setActiveHistoryPrevId(resolvePrevSnapshotId(historyItems, id));
},
[historyItems, setActiveHistoryId, setActiveHistoryPrevId],
);
@@ -128,12 +152,27 @@ function HistoryList({ pageId }: Props) {
return (
<div>
<Group px="xs" py={6} justify="flex-end">
<Switch
size="xs"
checked={onlyVersions}
onChange={(e) => setOnlyVersions(e.currentTarget.checked)}
label={t("Only versions")}
/>
</Group>
<ScrollArea h={620} w="100%" type="scroll" scrollbarSize={5}>
{historyItems.map((historyItem, index) => (
{onlyVersions && visibleItems.length === 0 && (
<Center py="md">
<Text size="sm" c="dimmed">
{t("No saved versions yet.")}
</Text>
</Center>
)}
{visibleItems.map((historyItem) => (
<HistoryItem
key={historyItem.id}
historyItem={historyItem}
index={index}
onSelect={handleSelect}
onHover={handleHover}
onHoverEnd={clearPrefetchTimeout}
@@ -24,6 +24,10 @@ export interface IPageHistory {
updatedAt: string;
lastUpdatedBy: IPageHistoryUser;
contributors?: IPageHistoryUser[];
// #370 — intentionality tier: 'manual'/'agent' are versions (intentional
// points), 'idle'/'boundary' are autosnapshots; null/undefined = legacy
// autosave. Derived server-side, drives the history badge + "versions" filter.
kind?: "manual" | "agent" | "idle" | "boundary" | null;
// Provenance markers copied off the page row when the snapshot was saved.
// `'agent'` marks a version written by the AI agent; `lastUpdatedAiChatId`
// (when present) deep-links to the chat that produced the edit.
@@ -0,0 +1,42 @@
import { describe, it, expect } from "vitest";
import { resolvePrevSnapshotId } from "./resolve-prev-snapshot";
// #370 F4 — the risky client path: with the "only versions" filter active, diff
// and restore must still baseline against the TRUE previous snapshot in the FULL
// list, never the previous VISIBLE version (which would skip the autosnapshots
// between two versions). These pin that the resolution is by FULL-list order.
describe("resolvePrevSnapshotId", () => {
// Newest-first, as the history list stores it: a version, then two autosaves,
// then an older version.
const full = [
{ id: "v2", kind: "manual" },
{ id: "a2", kind: "idle" },
{ id: "a1", kind: "boundary" },
{ id: "v1", kind: "manual" },
{ id: "a0", kind: null },
];
it("returns the immediate FULL-list successor, not the previous visible version", () => {
// Selecting v2 while filtered to versions-only must baseline against a2 (the
// real chronological predecessor), NOT v1 (the previous visible version).
expect(resolvePrevSnapshotId(full, "v2")).toBe("a2");
});
it("resolves an autosnapshot's predecessor by full-list order", () => {
expect(resolvePrevSnapshotId(full, "a1")).toBe("v1");
});
it("returns '' for the oldest item (no predecessor)", () => {
expect(resolvePrevSnapshotId(full, "a0")).toBe("");
});
it("returns '' for an id not in the list", () => {
expect(resolvePrevSnapshotId(full, "missing")).toBe("");
});
it("does not depend on a filtered subset — same result whatever is visible", () => {
// The helper only ever sees the full list; a filtered view cannot change the
// baseline it computes.
expect(resolvePrevSnapshotId(full, "v1")).toBe("a0");
});
});
@@ -0,0 +1,22 @@
/**
* #370 — resolve the TRUE previous snapshot for a history item.
*
* The history panel can be filtered to "only versions" (manual/agent), but diff
* and restore must always compare against the immediately-preceding snapshot in
* the FULL, unfiltered list — NOT the previous VISIBLE item. Comparing against
* the previous visible version would silently skip the autosnapshots between two
* versions and diff/restore the wrong baseline.
*
* Given the full (newest-first) list and an item id, this returns the id of the
* item right after it in the full list (its chronological predecessor), or "" if
* it is the oldest / not found. Pure and list-order-preserving so it can be unit
* tested without mounting the component.
*/
export function resolvePrevSnapshotId(
fullItems: ReadonlyArray<{ id: string }>,
id: string,
): string {
const index = fullItems.findIndex((item) => item.id === id);
if (index === -1) return "";
return fullItems[index + 1]?.id ?? "";
}
@@ -0,0 +1,28 @@
/**
* #370 — page-version stateless wire formats. Kept in one place so the client
* emitter (Save hotkey / button) and the client listener (page-editor) agree
* with the server (PersistenceExtension) on the message shapes.
*/
/** Client → server: "save a version now". The server derives the tier
* (manual/agent) from the signed connection actor, never from this payload. */
export const SAVE_VERSION_MESSAGE_TYPE = "save-version";
/** Server → all clients: a version was saved (or promoted / already existed). */
export const VERSION_SAVED_MESSAGE_TYPE = "version.saved";
export interface VersionSavedMessage {
type: typeof VERSION_SAVED_MESSAGE_TYPE;
historyId: string;
kind: "manual" | "agent";
/** True when the latest snapshot was already a manual version (a no-op save). */
alreadySaved: boolean;
}
/**
* Cross-component coordination flag so only the client that pressed Save shows
* the confirmation toast, while every other client silently refreshes its
* history panel on the broadcast. A module-level ref avoids stale-closure
* pitfalls in the editor's long-lived stateless handler.
*/
export const saveVersionPending = { current: false };
@@ -3,6 +3,7 @@ import {
IconArrowRight,
IconArrowsHorizontal,
IconClockHour4,
IconDeviceFloppy,
IconDots,
IconEye,
IconEyeOff,
@@ -17,7 +18,7 @@ import {
IconTrash,
IconWifiOff,
} from "@tabler/icons-react";
import React, { useEffect, useRef, useState } from "react";
import React, { useCallback, useEffect, useRef, useState } from "react";
import { useAsideTriggerProps } from "@/hooks/use-toggle-aside.tsx";
import { useAtom, useAtomValue } from "jotai";
import { historyAtoms } from "@/features/page-history/atoms/history-atoms.ts";
@@ -39,9 +40,14 @@ import { Trans, useTranslation } from "react-i18next";
import ExportModal from "@/components/common/export-modal";
import { convertProseMirrorToMarkdown } from "@docmost/prosemirror-markdown/browser";
import {
collabProviderAtom,
pageEditorAtom,
yjsConnectionStatusAtom,
} from "@/features/editor/atoms/editor-atoms.ts";
import {
SAVE_VERSION_MESSAGE_TYPE,
saveVersionPending,
} from "@/features/page-history/version-messages.ts";
import { formattedDate } from "@/lib/time.ts";
import { PageEditModeToggle } from "@/features/user/components/page-state-pref.tsx";
import MovePageModal from "@/features/page/components/move-page-modal.tsx";
@@ -72,9 +78,34 @@ export default function PageHeaderMenu({ readOnly }: PageHeaderMenuProps) {
});
const isDeleted = !!page?.deletedAt;
const [workspace] = useAtom(workspaceAtom);
const collabProvider = useAtomValue(collabProviderAtom);
// Community public-sharing entry point (replaces the removed EE PageShareModal)
const workspaceSharingDisabled = workspace?.settings?.sharing?.disabled === true;
// #370 — explicit "save a version" (Cmd+S / Save button). One path for the
// human; the server derives the tier from the signed actor. Readers can't save
// (the button is hidden and the collab connection is read-only server-side).
const handleSaveVersion = useCallback(() => {
if (readOnly || !collabProvider) return;
// Flag this client as the initiator so only it shows the confirmation toast;
// a safety timeout clears it if no broadcast comes back (e.g. offline).
saveVersionPending.current = true;
window.setTimeout(() => {
saveVersionPending.current = false;
}, 5000);
collabProvider.sendStateless(
JSON.stringify({ type: SAVE_VERSION_MESSAGE_TYPE }),
);
}, [readOnly, collabProvider]);
// mod+S must also block the browser's "Save page" dialog. `triggerOnContent-
// Editable` + empty ignore-list so it fires while typing in the editor/title.
useHotkeys(
[["mod+S", handleSaveVersion, { preventDefault: true }]],
[],
true,
);
useHotkeys(
[
[
@@ -133,15 +164,16 @@ export default function PageHeaderMenu({ readOnly }: PageHeaderMenuProps) {
</ActionIcon>
</Tooltip>
<PageActionMenu readOnly={readOnly} />
<PageActionMenu readOnly={readOnly} onSaveVersion={handleSaveVersion} />
</>
);
}
interface PageActionMenuProps {
readOnly?: boolean;
onSaveVersion?: () => void;
}
function PageActionMenu({ readOnly }: PageActionMenuProps) {
function PageActionMenu({ readOnly, onSaveVersion }: PageActionMenuProps) {
const { t } = useTranslation();
const [, setHistoryModalOpen] = useAtom(historyAtoms);
const clipboard = useClipboard({ timeout: 500 });
@@ -303,6 +335,20 @@ function PageActionMenu({ readOnly }: PageActionMenuProps) {
</Group>
</Menu.Item>
{!readOnly && (
<Menu.Item
leftSection={<IconDeviceFloppy size={16} />}
onClick={onSaveVersion}
rightSection={
<Text size="xs" c="dimmed">
{t("Ctrl+S")}
</Text>
}
>
{t("Save version")}
</Menu.Item>
)}
<Menu.Item
leftSection={<IconHistory size={16} />}
onClick={openHistoryModal}
@@ -168,10 +168,6 @@ export default function ShareAiWidget({
// Anonymous reader: suppress the tool-argument summary line so the
// agent's raw query/argument text isn't shown on the public share.
showInput={false}
// Anonymous reader: never paint a tool's raw errorText (it can carry
// internal detail). This is the render gate; the bytes are also
// sanitized server-side in PublicShareChatToolsService.forShare (#394).
showErrors={false}
// Anonymous reader: neutralize internal/relative links in the
// assistant's markdown so internal UUIDs/auth-gated routes don't
// leak as clickable links (external http(s) links are kept).
+30 -4
View File
@@ -1,10 +1,36 @@
export const HISTORY_INTERVAL = 5 * 60 * 1000;
export const HISTORY_FAST_INTERVAL = 60 * 1000;
export const HISTORY_FAST_THRESHOLD = 5 * 60 * 1000;
// #348 — debounce window for the per-page RAG re-embed job. Repeated saves
// within this window collapse to a single delayed job (coalesced by a stable
// jobId), so active editing does not pile up expensive re-embeds (external API
// + page_embeddings rewrite, concurrency 1). The worker reads the CURRENT page
// state at run time, so the last content within the window wins.
export const EMBED_DEBOUNCE_MS = 30 * 1000;
/**
* #370 — page-history intentionality tiers. Domain of `page_history.kind`.
* - 'manual' / 'agent' → Tier 1 versions (intentional points)
* - 'idle' / 'boundary' → Tier 0 autosnapshots (safety net)
* A legacy `null` kind is treated as an autosave.
*/
export type PageHistoryKind = 'manual' | 'agent' | 'idle' | 'boundary';
/**
* #370 — trailing idle-flush windows. A page's pending idle snapshot is
* re-armed on every store and fires this long after edits go quiet, so a burst
* of edits collapses into a single autosnapshot instead of one-per-store. Human
* sessions are noisier and less risky, so they flush less often than the agent.
*/
export const IDLE_INTERVAL_USER = 60 * 60 * 1000; // 60m
export const IDLE_INTERVAL_AGENT = 15 * 60 * 1000; // 15m
/**
* #370 — max-wait ceiling for the idle flush. Pure trailing debounce starves the
* safety net: hocuspocus stores at least every ~45s, so a CONTINUOUS editing
* session would re-arm the trailing timer forever and never take an idle
* snapshot until edits finally go quiet (up to IDLE_INTERVAL_USER = 60m). This
* ceiling bounds the actual wait from the FIRST edit of a burst, so an idle
* snapshot fires at least this often during a long unbroken session — restoring
* a recovery point cadence closer to the old heuristic without one-per-store
* noise. Mirrors hocuspocus's own maxDebounce idea.
*/
export const IDLE_MAX_WAIT_USER = 10 * 60 * 1000; // 10m
export const IDLE_MAX_WAIT_AGENT = 5 * 60 * 1000; // 5m
@@ -1,84 +1,93 @@
import { computeHistoryJob, resolveSource } from './persistence.extension';
import {
computeHistoryJob,
resolveSource,
} from './persistence.extension';
import {
HISTORY_FAST_INTERVAL,
HISTORY_FAST_THRESHOLD,
HISTORY_INTERVAL,
IDLE_INTERVAL_AGENT,
IDLE_INTERVAL_USER,
IDLE_MAX_WAIT_AGENT,
IDLE_MAX_WAIT_USER,
} from '../constants';
// A fixed clock + fixed createdAt make pageAge deterministic.
const NOW = 1_700_000_000_000;
const PAGE_ID = '550e8400-e29b-41d4-a716-446655440000';
// Build a minimal page whose age (NOW - createdAt) is exactly `ageMs`.
const pageAged = (ageMs: number) => ({
id: PAGE_ID,
createdAt: new Date(NOW - ageMs),
});
const page = { id: PAGE_ID };
describe('computeHistoryJob', () => {
it('agent edit → delay MUST be 0 and job id is source-keyed', () => {
// INVARIANT (§15 H2 / persistence.extension): the agent delay MUST stay 0.
// The worker re-reads the page row at run time, so any non-zero delay risks
// snapshotting content a later human edit has already overwritten. This is
// the load-bearing assertion of this spec — do not relax it.
const { jobId, delay } = computeHistoryJob(pageAged(0), 'agent', NOW);
expect(delay).toBe(0);
expect(jobId).toBe(`${PAGE_ID}-agent`);
});
it('agent edit on an OLD page is still delay 0 (age never applies to agents)', () => {
// Even when the page is far older than the fast threshold, the agent path
// must short-circuit to 0 — age-based debounce is a human-only concern.
const { jobId, delay } = computeHistoryJob(
pageAged(HISTORY_FAST_THRESHOLD + 60_000),
'agent',
NOW,
);
expect(delay).toBe(0);
expect(jobId).toBe(`${PAGE_ID}-agent`);
});
it('human edit on a YOUNG page (age < threshold) → fast interval, bare job id', () => {
const { jobId, delay } = computeHistoryJob(
pageAged(HISTORY_FAST_THRESHOLD - 1),
'user',
NOW,
);
expect(delay).toBe(HISTORY_FAST_INTERVAL);
describe('computeHistoryJob (#370 — shared trailing idle pipeline)', () => {
it('human edit → user idle window, bare page.id job', () => {
// Humans and the agent now share ONE idle job per page (jobId = page.id).
// The agent's old delay=0 fast path is GONE — intentional agent points now
// arrive via the explicit save-version signal, not a zero-delay snapshot.
const { jobId, delay } = computeHistoryJob(page, 'user');
expect(delay).toBe(IDLE_INTERVAL_USER);
expect(jobId).toBe(PAGE_ID);
});
it('human edit on an OLD page (age > threshold) → standard interval', () => {
const { jobId, delay } = computeHistoryJob(
pageAged(HISTORY_FAST_THRESHOLD + 1),
'user',
NOW,
);
expect(delay).toBe(HISTORY_INTERVAL);
it('agent edit → agent idle window (shorter), still the bare page.id job', () => {
const { jobId, delay } = computeHistoryJob(page, 'agent');
expect(delay).toBe(IDLE_INTERVAL_AGENT);
// No `-agent` suffix anymore: the agent joins the common idle pipeline.
expect(jobId).toBe(PAGE_ID);
});
it('boundary: pageAge EXACTLY === threshold takes the slow branch (the `<` is strict)', () => {
// Off-by-one guard: the condition is `pageAge < HISTORY_FAST_THRESHOLD`, so
// an age of exactly the threshold is NOT "fast" — it must use HISTORY_INTERVAL.
const { delay } = computeHistoryJob(
pageAged(HISTORY_FAST_THRESHOLD),
'user',
NOW,
);
expect(delay).toBe(HISTORY_INTERVAL);
it('agent flushes sooner than a human', () => {
expect(IDLE_INTERVAL_AGENT).toBeLessThan(IDLE_INTERVAL_USER);
});
it('treats any non-"agent" source string as human', () => {
// resolveSource only ever yields 'agent' | 'user', but guard the contract:
// the agent branch keys strictly on === 'agent'.
const { jobId, delay } = computeHistoryJob(pageAged(0), 'user', NOW);
expect(delay).toBe(HISTORY_FAST_INTERVAL);
it('treats any non-"agent" source string as human (keys strictly on === agent)', () => {
const { jobId, delay } = computeHistoryJob(page, 'user');
expect(delay).toBe(IDLE_INTERVAL_USER);
expect(jobId).toBe(PAGE_ID);
});
// #370 review round-1 WARNING: the max-wait ceiling prevents autosnapshot
// starvation during a continuous editing session (the trailing timer would
// otherwise re-arm forever and never fire).
describe('max-wait ceiling', () => {
const T0 = 1_000_000; // arbitrary fixed epoch for deterministic tests
it('once a burst is armed, delay clamps to the remaining max-wait budget', () => {
// 1 minute into the burst the USER interval (60m) far exceeds the remaining
// max-wait budget (10m - 1m = 9m), so the delay is clamped DOWN to that
// remaining budget — the full interval is NOT used once a ceiling applies.
const { delay } = computeHistoryJob(page, 'user', T0, T0 + 60_000);
expect(delay).toBe(IDLE_MAX_WAIT_USER - 60_000);
});
it('never waits longer than the max-wait budget from the burst start', () => {
// A store arriving right at the ceiling → delay 0 (fire promptly).
const { delay } = computeHistoryJob(
page,
'user',
T0,
T0 + IDLE_MAX_WAIT_USER,
);
expect(delay).toBe(0);
});
it('past the ceiling never returns a negative delay', () => {
const { delay } = computeHistoryJob(
page,
'user',
T0,
T0 + IDLE_MAX_WAIT_USER + 5 * 60_000,
);
expect(delay).toBe(0);
});
it('the agent ceiling is shorter than the user ceiling', () => {
expect(IDLE_MAX_WAIT_AGENT).toBeLessThan(IDLE_MAX_WAIT_USER);
const { delay } = computeHistoryJob(
page,
'agent',
T0,
T0 + IDLE_MAX_WAIT_AGENT,
);
expect(delay).toBe(0);
});
it('without a burstStart there is no ceiling (backward-compatible)', () => {
expect(computeHistoryJob(page, 'user').delay).toBe(IDLE_INTERVAL_USER);
expect(computeHistoryJob(page, 'agent').delay).toBe(IDLE_INTERVAL_AGENT);
});
});
});
describe('resolveSource (truth table)', () => {
@@ -40,11 +40,12 @@ describe('PersistenceExtension.onStoreDocument — Approach-A boundary snapshot'
let pageHistoryRepo: {
saveHistory: jest.Mock;
findPageLastHistory: jest.Mock;
updateHistoryKind: jest.Mock;
};
let aiQueue: { add: jest.Mock };
let historyQueue: { add: jest.Mock };
let historyQueue: { add: jest.Mock; remove: jest.Mock };
let notificationQueue: { add: jest.Mock };
let collabHistory: { addContributors: jest.Mock };
let collabHistory: { addContributors: jest.Mock; popContributors: jest.Mock };
let transclusionService: {
syncPageTransclusions: jest.Mock;
syncPageReferences: jest.Mock;
@@ -93,13 +94,22 @@ describe('PersistenceExtension.onStoreDocument — Approach-A boundary snapshot'
pageHistoryRepo = {
saveHistory: jest.fn().mockImplementation(async () => {
callOrder.push('saveHistory');
return { id: 'history-1' };
}),
findPageLastHistory: jest.fn().mockResolvedValue(null),
updateHistoryKind: jest.fn().mockResolvedValue(undefined),
};
aiQueue = { add: jest.fn().mockResolvedValue(undefined) };
historyQueue = { add: jest.fn().mockResolvedValue(undefined) };
historyQueue = {
add: jest.fn().mockResolvedValue(undefined),
// #370 — enqueuePageHistory now removes any pending idle job before re-adding.
remove: jest.fn().mockResolvedValue(undefined),
};
notificationQueue = { add: jest.fn().mockResolvedValue(undefined) };
collabHistory = { addContributors: jest.fn().mockResolvedValue(undefined) };
collabHistory = {
addContributors: jest.fn().mockResolvedValue(undefined),
popContributors: jest.fn().mockResolvedValue([]),
};
transclusionService = {
syncPageTransclusions: jest.fn().mockResolvedValue(undefined),
syncPageReferences: jest.fn().mockResolvedValue(undefined),
@@ -165,6 +175,50 @@ describe('PersistenceExtension.onStoreDocument — Approach-A boundary snapshot'
expect(pageRepo.updatePage.mock.calls[0][0].lastUpdatedSource).toBe('user');
});
// #370 review round-1 SUGGESTION: the boundary was GENERALIZED from a
// user→agent special-case to ANY lastUpdatedSource transition. These pin the
// generalized behaviour it was rebuilt for.
describe('generalized boundary — any source transition', () => {
// Same persisted page but with an explicit prior source.
const pageWithPriorSource = (prior: string | null) => ({
...persistedHumanPage('NEW CONTENT'),
lastUpdatedSource: prior,
});
it('agent→user transition fires the boundary (pins the prior agent revision)', async () => {
const document = ydocFor(doc('NEW CONTENT'));
pageRepo.findById.mockResolvedValue(pageWithPriorSource('agent'));
pageHistoryRepo.findPageLastHistory.mockResolvedValue(null);
await ext.onStoreDocument(buildData(document, 'user') as any);
expect(pageHistoryRepo.saveHistory).toHaveBeenCalledTimes(1);
expect(callOrder).toEqual(['saveHistory', 'updatePage']);
expect(pageRepo.updatePage.mock.calls[0][0].lastUpdatedSource).toBe('user');
});
it('git→user transition fires the boundary (git-sync overwrite is a source change)', async () => {
const document = ydocFor(doc('NEW CONTENT'));
pageRepo.findById.mockResolvedValue(pageWithPriorSource('git'));
pageHistoryRepo.findPageLastHistory.mockResolvedValue(null);
await ext.onStoreDocument(buildData(document, 'user') as any);
expect(pageHistoryRepo.saveHistory).toHaveBeenCalledTimes(1);
expect(callOrder).toEqual(['saveHistory', 'updatePage']);
});
it('a null prior source (first-ever edit) does NOT fire the boundary', async () => {
const document = ydocFor(doc('NEW CONTENT'));
pageRepo.findById.mockResolvedValue(pageWithPriorSource(null));
await ext.onStoreDocument(buildData(document, 'agent') as any);
expect(pageHistoryRepo.saveHistory).not.toHaveBeenCalled();
expect(pageRepo.updatePage).toHaveBeenCalledTimes(1);
});
});
it('idempotency: unchanged content → no updatePage, no history, no queues', async () => {
// The Y.Doc content equals the persisted content deeply → early skip.
// A Y.Doc round-trip normalizes attrs (e.g. paragraph indent), so derive
@@ -479,4 +533,125 @@ describe('PersistenceExtension.onStoreDocument — Approach-A boundary snapshot'
// Contributors keyed by the UUID so they match the PAGE_HISTORY job (page.id).
expect(collabHistory.addContributors.mock.calls[0][0]).toBe(PAGE_ID);
});
// #370 — explicit save-version (Cmd+S / agent save tool) over the stateless
// seam. The tier is derived from the SIGNED connection actor, the store path
// is reused, and promote-not-dup avoids duplicating heavy content rows.
describe('save-version (#370)', () => {
const emitSave = (document: any, actor: 'user' | 'agent') =>
ext.onStateless({
connection: {
readOnly: false,
context: { user: { id: USER_ID, name: 'Alice' }, actor },
} as any,
documentName: `page.${PAGE_ID}`,
document: document as any,
payload: JSON.stringify({ type: 'save-version' }),
} as any);
// findById returns a page whose content already equals the live doc, so the
// store path is a no-op and we isolate the versioning decision.
const pageMatchingDoc = (document: any) => ({
...persistedHumanPage('IGNORED'),
content: TiptapTransformer.fromYdoc(document, 'default'),
});
it('human save with no prior snapshot → writes a manual version + broadcasts', async () => {
const document = ydocFor(doc('VERSION ME'));
pageRepo.findById.mockResolvedValue(pageMatchingDoc(document));
pageHistoryRepo.findPageLastHistory.mockResolvedValue(null);
await emitSave(document, 'user');
expect(pageHistoryRepo.saveHistory).toHaveBeenCalledTimes(1);
expect(pageHistoryRepo.saveHistory.mock.calls[0][1]).toEqual(
expect.objectContaining({ kind: 'manual' }),
);
// The pending idle autosnapshot is cancelled by the explicit version.
expect(historyQueue.remove).toHaveBeenCalledWith(PAGE_ID);
const msg = JSON.parse(
(document as any).broadcastStateless.mock.calls[(document as any).broadcastStateless.mock.calls.length - 1][0],
);
expect(msg).toMatchObject({
type: 'version.saved',
kind: 'manual',
alreadySaved: false,
});
});
it('agent save derives kind=agent from the signed actor', async () => {
const document = ydocFor(doc('AGENT VERSION'));
pageRepo.findById.mockResolvedValue(pageMatchingDoc(document));
pageHistoryRepo.findPageLastHistory.mockResolvedValue(null);
await emitSave(document, 'agent');
expect(pageHistoryRepo.saveHistory.mock.calls[pageHistoryRepo.saveHistory.mock.calls.length - 1][1]).toEqual(
expect.objectContaining({ kind: 'agent' }),
);
});
it('promote-not-dup: latest snapshot is an autosave with identical content → upgrades in place', async () => {
const document = ydocFor(doc('SAME'));
const page = pageMatchingDoc(document);
pageRepo.findById.mockResolvedValue(page);
pageHistoryRepo.findPageLastHistory.mockResolvedValue({
id: 'auto-1',
content: page.content,
kind: 'idle',
});
await emitSave(document, 'user');
// No heavy new content row — the existing autosave is promoted to manual.
expect(pageHistoryRepo.updateHistoryKind).toHaveBeenCalledWith(
'auto-1',
'manual',
expect.anything(),
);
expect(pageHistoryRepo.saveHistory).not.toHaveBeenCalled();
const msg = JSON.parse(
(document as any).broadcastStateless.mock.calls[(document as any).broadcastStateless.mock.calls.length - 1][0],
);
expect(msg).toMatchObject({ historyId: 'auto-1', alreadySaved: false });
});
it('no-op when the latest snapshot is already a manual version of this content', async () => {
const document = ydocFor(doc('ALREADY SAVED'));
const page = pageMatchingDoc(document);
pageRepo.findById.mockResolvedValue(page);
pageHistoryRepo.findPageLastHistory.mockResolvedValue({
id: 'ver-1',
content: page.content,
kind: 'manual',
});
await emitSave(document, 'user');
expect(pageHistoryRepo.updateHistoryKind).not.toHaveBeenCalled();
expect(pageHistoryRepo.saveHistory).not.toHaveBeenCalled();
const msg = JSON.parse(
(document as any).broadcastStateless.mock.calls[(document as any).broadcastStateless.mock.calls.length - 1][0],
);
expect(msg).toMatchObject({ alreadySaved: true, kind: 'manual' });
});
it('a read-only connection cannot save a version', async () => {
const document = ydocFor(doc('READER'));
pageRepo.findById.mockResolvedValue(pageMatchingDoc(document));
await ext.onStateless({
connection: {
readOnly: true,
context: { user: { id: USER_ID }, actor: 'user' },
} as any,
documentName: `page.${PAGE_ID}`,
document: document as any,
payload: JSON.stringify({ type: 'save-version' }),
} as any);
expect(pageHistoryRepo.saveHistory).not.toHaveBeenCalled();
expect(pageHistoryRepo.updateHistoryKind).not.toHaveBeenCalled();
});
});
});
@@ -37,9 +37,11 @@ import { Page } from '@docmost/db/types/entity.types';
import { CollabHistoryService } from '../services/collab-history.service';
import {
EMBED_DEBOUNCE_MS,
HISTORY_FAST_INTERVAL,
HISTORY_FAST_THRESHOLD,
HISTORY_INTERVAL,
IDLE_INTERVAL_AGENT,
IDLE_INTERVAL_USER,
IDLE_MAX_WAIT_AGENT,
IDLE_MAX_WAIT_USER,
PageHistoryKind,
} from '../constants';
import { TransclusionService } from '../../core/page/transclusion/transclusion.service';
import {
@@ -56,6 +58,16 @@ import { hasTransclusionFamilyNodes } from '../../core/page/transclusion/utils/t
*/
export const INTENTIONAL_CLEAR_MESSAGE_TYPE = 'intentional-clear';
/**
* #370 — wire format of the client→server "save a version" signal. Sent by the
* human (Cmd+S / Save button) and by the agent's explicit save tool over the
* SAME stateless channel. The intentionality tier ('manual' vs 'agent') is
* derived SERVER-SIDE from the signed connection actor, never from this
* payload, so a version's type is unforgeable. The document is taken from the
* connection (not the payload), so the signal cannot be aimed at another page.
*/
export const SAVE_VERSION_MESSAGE_TYPE = 'save-version';
/**
* #251 — how long an intentional-clear signal stays "pending" before it is
* ignored. The signal is set on the clearing keystroke but consumed by the
@@ -92,35 +104,39 @@ export function resolveSource(
}
/**
* Compute the BullMQ job id + delay for a page-history snapshot job. Pure so
* the data-loss-sensitive timing arithmetic is unit-testable; `now` is injected
* (caller passes `Date.now()`) for determinism.
* #370 — compute the BullMQ job id + delay for a page's trailing idle-flush
* autosnapshot. Pure so the timing is unit-testable.
*
* - Agent edits: delay 0 and a source-keyed job id `${page.id}-agent`. The
* delay MUST stay 0 — the worker re-reads the page row at run time, so any
* delay risks reading content a later human edit has already overwritten
* (mis-tagged snapshot). 0 minimizes that window. The `-agent` suffix keeps
* the job from coalescing with the bare-page.id human job.
* - Human edits: age-based debounce so rapid human edits coalesce into one
* snapshot; job id is the bare `page.id`.
*
* BullMQ forbids ':' in custom job ids (Redis key separator), so '-' is used;
* page.id is a UUID, so `${page.id}-agent` cannot collide with a human job.
* Both humans and the agent now share ONE idle pipeline (the agent's old
* `delay=0` fast path is gone — intentional agent points arrive via the
* explicit save-version signal instead). The job id is the bare `page.id`, so a
* page has at most one pending idle job; the caller removes-and-re-adds it on
* every store to keep it debounced to the trailing edge of an edit burst. The
* window differs by source only: the agent flushes sooner than a human.
*/
export function computeHistoryJob(
page: Pick<Page, 'id' | 'createdAt'>,
page: Pick<Page, 'id'>,
source: string,
now: number,
// Epoch ms of the FIRST edit in the current burst (when the pending idle job
// was first armed). Used to enforce the max-wait ceiling so a continuous
// editing session cannot re-arm the trailing timer forever. `now` is injectable
// for tests; both default to a live clock / no ceiling when omitted.
burstStart?: number,
now: number = Date.now(),
): { jobId: string; delay: number } {
const isAgent = source === 'agent';
const pageAge = now - new Date(page.createdAt).getTime();
const delay = isAgent
? 0
: pageAge < HISTORY_FAST_THRESHOLD
? HISTORY_FAST_INTERVAL
: HISTORY_INTERVAL;
const jobId = isAgent ? `${page.id}-agent` : page.id;
return { jobId, delay };
const interval = isAgent ? IDLE_INTERVAL_AGENT : IDLE_INTERVAL_USER;
const maxWait = isAgent ? IDLE_MAX_WAIT_AGENT : IDLE_MAX_WAIT_USER;
let delay = interval;
if (burstStart !== undefined) {
// Time already elapsed since the burst's first edit; the snapshot must fire
// no later than `maxWait` after that, so shrink the trailing delay to the
// remaining budget (never negative, so BullMQ fires it promptly).
const remaining = burstStart + maxWait - now;
delay = Math.max(0, Math.min(interval, remaining));
}
return { jobId: page.id, delay };
}
@Injectable()
@@ -132,6 +148,23 @@ export class PersistenceExtension implements Extension {
// coalescing window" per document and OR it across all edits in the window,
// so the snapshot is marked 'agent' regardless of who wrote last.
private agentTouched: Map<string, boolean> = new Map();
// #370 — epoch ms of the FIRST edit in the current idle-flush burst, per page.
// Set when the pending idle job is first armed (empty entry), read to enforce
// the max-wait ceiling in computeHistoryJob, and cleared when the idle job is
// consumed/cancelled so the next burst starts a fresh window.
//
// Single-process assumption (like `contributors` / `agentTouched` above): this
// lives only in THIS collab process's memory. A restart, or a page's ownership
// moving to another node, loses the burst-start marker. Consequence: a burst
// that spans the restart looks like a fresh burst to the surviving process, so
// its max-wait ceiling is re-anchored to the first post-restart edit — a single
// continuous session straddling a restart can therefore wait up to ~2× the cap
// for its idle snapshot (once for the lost pre-restart window, once for the new
// one). Bounded and benign (it only DELAYS a safety-net autosnapshot; manual
// saves are unaffected and the next quiet period always flushes), but the
// assumption and its consequence are recorded here so no one mistakes the
// in-memory marker for a durable, cross-process guarantee.
private idleBurstStart: Map<string, number> = new Map();
// #251 — per-document "intentional clear pending" flags. Keyed by
// documentName, value = expiry timestamp (ms). Set by onStateless when the
// client reports a deliberate clear; consumed once by the next
@@ -363,20 +396,19 @@ export class PersistenceExtension implements Extension {
//this.logger.debug('Contributors error:' + err?.['message']);
}
// Approach A — boundary snapshot before the agent's first edit.
// When this store is the agent's and the page's currently persisted
// state was authored by a human, pin that human state as its own
// history version BEFORE the agent overwrites it. `page` still holds
// the OLD content/provenance here, so saveHistory(page) captures the
// pre-agent state tagged 'user'. The agent's new content is
// snapshotted later by the debounced PAGE_HISTORY job ('agent'). Skip
// if the prior state is already agent-authored (boundary already
// pinned on the user->agent transition), if the page is effectively
// empty, or if the latest existing snapshot already equals this human
// state (avoid duplicates).
// #370 — boundary snapshot on ANY source transition. When the store
// flips the page's provenance (user↔agent↔git), pin the OUTGOING
// state as its own history version BEFORE the incoming source
// overwrites it. `page` still holds the OLD content/provenance here,
// so saveHistory(page) captures the pre-transition state tagged with
// its own source, kind='boundary'. The incoming content is snapshotted
// later by the debounced idle job. Skip if the page is effectively
// empty or if the latest existing snapshot already equals this state
// (the shared isDeepStrictEqual gate — avoids duplicates). Generalizing
// beyond the old user→agent special-case also covers git-sync for free.
if (
lastUpdatedSource === 'agent' &&
page.lastUpdatedSource !== 'agent'
page.lastUpdatedSource &&
page.lastUpdatedSource !== lastUpdatedSource
) {
// pageHistory.pageId is uuid-typed; use page.id (never the doc-name
// slugId) so a `page.<slugId>` doc cannot throw 22P02 here (#260).
@@ -384,15 +416,13 @@ export class PersistenceExtension implements Extension {
page.id,
{ includeContent: true, trx },
);
const humanBaselineMissing =
const baselineMissing =
!lastHistory ||
!isDeepStrictEqual(lastHistory.content, page.content);
if (
!isEmptyParagraphDoc(page.content as any) &&
humanBaselineMissing
) {
if (!isEmptyParagraphDoc(page.content as any) && baselineMissing) {
await this.pageHistoryRepo.saveHistory(page, {
contributorIds: page.contributorIds ?? undefined,
kind: 'boundary',
trx,
});
}
@@ -554,6 +584,14 @@ export class PersistenceExtension implements Extension {
return; // unrelated / malformed stateless message
}
// #370 — explicit "save a version" (human Cmd+S / agent save tool). Edit
// rights are already enforced by the readOnly reject above (a reader can't
// create a version), exactly as intentional-clear requires.
if (message?.type === SAVE_VERSION_MESSAGE_TYPE) {
await this.handleSaveVersion(data);
return;
}
if (message?.type !== INTENTIONAL_CLEAR_MESSAGE_TYPE) return;
this.intentionalClear.set(
@@ -562,6 +600,117 @@ export class PersistenceExtension implements Extension {
);
}
/**
* #370 — persist an intentional version from the live in-memory ydoc.
*
* One stateless path serves BOTH the human and the agent; the tier is derived
* SERVER-SIDE from the signed connection actor ('agent' → 'agent', anything
* else → 'manual'), so the version type cannot be spoofed by the client. We
* take the fresh ydoc from the collab process memory and run it through the
* EXISTING store path first (so pages.content/ydoc reflect the exact content
* being versioned — a REST endpoint would race the up-to-10s-stale page row),
* then snapshot it into page_history with the intentional kind.
*
* Promote-not-dup: if the latest history row already holds this exact content
* and it is an autosave (idle/boundary/legacy-null), upgrade its kind in place
* instead of duplicating a heavy content row; if it is already 'manual', it is
* a no-op (the client shows an "already saved" toast). Otherwise a fresh
* version row is written, popping the aggregated contributors from Redis.
*/
private async handleSaveVersion(data: onStatelessPayload): Promise<void> {
const { connection, document, documentName } = data;
const context = connection?.context;
const pageId = getPageId(documentName);
// Unforgeable: 'agent' only for a signed agent connection, else 'manual'.
const kind: PageHistoryKind =
context?.actor === 'agent' ? 'agent' : 'manual';
// Flush the live ydoc through the normal store path so the page row + ydoc
// hold exactly what we are about to version (also fires the idle enqueue we
// supersede below, plus any source-transition boundary). onStoreDocument
// only needs document/documentName/context.
await this.onStoreDocument({
document,
documentName,
context,
} as onStoreDocumentPayload);
let result:
| { historyId: string; kind: PageHistoryKind; alreadySaved: boolean }
| undefined;
await executeTx(this.db, async (trx) => {
const page = await this.pageRepo.findById(pageId, {
withLock: true,
includeContent: true,
trx,
});
if (!page) return;
// Never version an effectively-empty page (mirrors the processor's
// first-history guard); there is nothing intentional to pin.
if (isEmptyParagraphDoc(page.content as any)) return;
const lastHistory = await this.pageHistoryRepo.findPageLastHistory(
page.id,
{ includeContent: true, trx },
);
if (
lastHistory &&
isDeepStrictEqual(lastHistory.content, page.content)
) {
// Content is already snapshotted. Promote-not-dup.
if (lastHistory.kind === 'manual') {
result = {
historyId: lastHistory.id,
kind: 'manual',
alreadySaved: true,
};
return;
}
await this.pageHistoryRepo.updateHistoryKind(
lastHistory.id,
kind,
trx,
);
result = { historyId: lastHistory.id, kind, alreadySaved: false };
return;
}
// Fresh version row. Pop the contributors aggregated since the last
// snapshot (SPOP); restore them if the write fails so they aren't lost.
const contributorIds = await this.collabHistory.popContributors(page.id);
try {
const saved = await this.pageHistoryRepo.saveHistory(page, {
contributorIds,
kind,
trx,
});
result = { historyId: saved.id, kind, alreadySaved: false };
} catch (err) {
await this.collabHistory.addContributors(page.id, contributorIds);
throw err;
}
});
// Housekeeping: this explicit version supersedes the page's pending idle
// autosnapshot, so cancel it (delayed job → remove() just deletes it) and
// end the current idle burst so the next edit starts a fresh max-wait window.
await this.historyQueue.remove(pageId).catch(() => undefined);
this.idleBurstStart.delete(pageId);
if (result) {
document.broadcastStateless(
JSON.stringify({
type: 'version.saved',
historyId: result.historyId,
kind: result.kind,
alreadySaved: result.alreadySaved,
}),
);
}
}
async onChange(data: onChangePayload) {
const documentName = data.documentName;
const userId = data.context?.user?.id;
@@ -619,17 +768,75 @@ export class PersistenceExtension implements Extension {
page: Page,
lastUpdatedSource: string,
): Promise<void> {
// Job id + delay arithmetic lives in the pure `computeHistoryJob` (see its
// doc comment for the agent-delay-0 / age-based-debounce invariants).
// #370 — trailing idle debounce with a max-wait ceiling. One pending idle
// job per page (jobId = page.id); on every store we remove the pending
// delayed job and re-add it, so the snapshot lands `delay` after edits go
// quiet rather than once per store (precedent: workspace.service.ts).
// remove() on a delayed job simply deletes it (0 if absent, no throw); if the
// job is already ACTIVE and the remove is a no-op, the add still de-dups and
// the processor's isDeepStrictEqual gate collapses the duplicate content.
//
// The FIRST arm of a burst records `burstStart`; computeHistoryJob shrinks
// the delay to the remaining max-wait budget from that point, so a continuous
// session cannot re-arm the trailing timer forever and starve the snapshot.
// A burst marker older than THIS TIER's max-wait means the previous idle job
// has already fired — start a fresh window instead of firing immediately on
// the next edit. Must use the SAME source-specific max-wait computeHistoryJob
// uses (agent 5m / user 10m): a hardcoded USER ceiling would leave an agent
// burst's marker stale for 5..10m, forcing delay=0 on every store in that
// window and writing one idle row per store — exactly the per-store bloat the
// debounce exists to prevent, on the continuous-agent path.
const maxWait =
lastUpdatedSource === 'agent' ? IDLE_MAX_WAIT_AGENT : IDLE_MAX_WAIT_USER;
const now = Date.now();
let burstStart = this.idleBurstStart.get(page.id);
if (burstStart === undefined || now - burstStart >= maxWait) {
burstStart = now;
this.idleBurstStart.set(page.id, burstStart);
}
const { jobId, delay } = computeHistoryJob(
page,
lastUpdatedSource,
Date.now(),
burstStart,
now,
);
// remove-then-add trailing-debounce idiom, and its ONE race. We delete the
// pending delayed job and re-add it under the same jobId so the timer resets
// to the trailing edge of the burst. The race is the small window between
// these two awaits: if the delayed job's `delay` elapses in that gap it goes
// ACTIVE, and then:
// - remove() on an active/locked job is a no-op (BullMQ won't yank a job a
// worker holds), and our `.catch(() => undefined)` swallows that too; and
// - add() with a jobId that already exists (the now-active job's id) is
// DROPPED by BullMQ — a duplicate add is a no-op.
// So this store fails to re-arm the trailing job: the just-fired snapshot
// captured content up to the moment it went active, and THIS edit is left
// without a pending trailing job. It is bounded and self-healing — the NEXT
// store re-arms a fresh delayed job (the id is free again once the active job
// completes / removeOnComplete frees it), and the processor's
// isDeepStrictEqual gate collapses any content-identical duplicate. The only
// uncovered case is when the racing store was the LAST in the session: the
// tail edits made after the job went active get NO trailing snapshot until
// the next edit re-arms one. That is an acceptable safety-net gap (a manual
// Save, a source-transition boundary, or simply the next edit all still cover
// it), which is why the reviewer accepts documenting it here rather than
// adding a post-add "did the add actually arm a job?" re-check.
//
// NOTE — do NOT "unify" this with the neighbouring embed-debounce idiom
// (aiQueue.add of PAGE_CONTENT_UPDATED above): that one uses a STABLE jobId
// and NO remove(), relying purely on BullMQ coalescing a repeated add under
// the same id, because a re-embed only needs to eventually run once on the
// latest content and re-anchoring its delay on every keystroke is undesirable.
// THIS idiom deliberately removes-then-adds precisely to PUSH the delay back
// to the trailing edge on every store (a true debounce), which coalescing
// alone cannot do. Collapsing them would silently change the history cadence.
await this.historyQueue.remove(jobId).catch(() => undefined);
await this.historyQueue.add(
QueueJob.PAGE_HISTORY,
{ pageId: page.id } as IPageHistoryJob,
{ pageId: page.id, kind: 'idle' } as IPageHistoryJob,
{ jobId, delay },
);
}
@@ -66,6 +66,15 @@ describe('HistoryProcessor.process', () => {
notificationQueue = { add: jest.fn().mockResolvedValue(undefined) };
generalQueue = { add: jest.fn().mockResolvedValue(undefined) };
// #370 F3 — the processor now serializes its find+save under a page-row lock
// via executeTx. A db whose transaction().execute(fn) runs fn with a trx stub
// drives the real executeTx() helper without a database.
const db = {
transaction: () => ({
execute: (fn: (trx: any) => Promise<any>) => fn({ __trx: true }),
}),
};
// WorkerHost's constructor reads `this.worker`; passing repos positionally
// matches the constructor and avoids the Nest DI container.
proc = new HistoryProcessor(
@@ -73,6 +82,7 @@ describe('HistoryProcessor.process', () => {
pageRepo as any,
collabHistory as any,
watcherService as any,
db as any,
notificationQueue as any,
generalQueue as any,
);
@@ -126,15 +136,26 @@ describe('HistoryProcessor.process', () => {
await proc.process(buildJob());
expect(collabHistory.popContributors).toHaveBeenCalledWith(PAGE_ID);
// #370 F3/F9 — the snapshot decision runs under a page-row lock. Pin the lock
// structurally so a refactor that drops withLock/trx (silently reintroducing
// the TOCTOU double-insert) turns this red. The tx stub is { __trx: true }.
expect(pageRepo.findById).toHaveBeenCalledWith(
PAGE_ID,
expect.objectContaining({ withLock: true, trx: { __trx: true } }),
);
// #370 F7 — addPageWatchers MUST receive the trx, or its FK-check runs on a
// separate connection and self-deadlocks against our FOR UPDATE. Asserting
// the trx arg here is exactly what would have caught that regression.
expect(watcherService.addPageWatchers).toHaveBeenCalledWith(
['u1', 'u2'],
PAGE_ID,
SPACE_ID,
WORKSPACE_ID,
{ __trx: true },
);
expect(pageHistoryRepo.saveHistory).toHaveBeenCalledWith(
expect.objectContaining({ id: PAGE_ID }),
{ contributorIds: ['u1', 'u2'] },
{ contributorIds: ['u1', 'u2'], kind: 'idle', trx: { __trx: true } },
);
expect(generalQueue.add).toHaveBeenCalledWith(
QueueJob.PAGE_BACKLINKS,
@@ -186,6 +207,48 @@ describe('HistoryProcessor.process', () => {
]);
});
it('COMMIT failure (throw outside the tx callback) → contributors RESTORED', async () => {
// #370 F8 — a commit-time failure throws OUTSIDE the callback, so the inner
// try/catch does not run; the outer catch must restore the popped set (else a
// BullMQ retry writes an unattributed version). Use a db whose execute() runs
// the callback THEN throws, simulating a commit abort.
pageHistoryRepo.findPageLastHistory.mockResolvedValue({
content: { type: 'doc', content: [] },
});
const commitFail = {
transaction: () => ({
execute: async (fn: (trx: any) => Promise<any>) => {
await fn({ __trx: true }); // callback succeeds (saveHistory ok)
throw new Error('commit aborted'); // ...but the COMMIT fails
},
}),
};
const procCommitFail = new HistoryProcessor(
pageHistoryRepo as any,
pageRepo as any,
collabHistory as any,
watcherService as any,
commitFail as any,
notificationQueue as any,
generalQueue as any,
);
jest
.spyOn(procCommitFail['logger'], 'error')
.mockImplementation(() => undefined);
await expect(procCommitFail.process(buildJob())).rejects.toThrow(
'commit aborted',
);
// The inner catch did NOT run (save succeeded), so only the outer catch can
// restore — assert it did.
expect(collabHistory.addContributors).toHaveBeenCalledWith(PAGE_ID, [
'u1',
'u2',
]);
// And the post-snapshot queue work must NOT have run (we rethrew).
expect(generalQueue.add).not.toHaveBeenCalled();
});
it('backlinks + notification queue failures are swallowed (history still committed)', async () => {
pageHistoryRepo.findPageLastHistory.mockResolvedValue({
content: { type: 'doc', content: [] },
@@ -19,6 +19,9 @@ import { isDeepStrictEqual } from 'node:util';
import { CollabHistoryService } from '../services/collab-history.service';
import { WatcherService } from '../../core/watcher/watcher.service';
import { isEmptyParagraphDoc } from '../collaboration.util';
import { InjectKysely } from 'nestjs-kysely';
import { KyselyDB } from '@docmost/db/types/kysely.types';
import { executeTx } from '@docmost/db/utils';
@Processor(QueueName.HISTORY_QUEUE)
export class HistoryProcessor extends WorkerHost implements OnModuleDestroy {
@@ -29,6 +32,7 @@ export class HistoryProcessor extends WorkerHost implements OnModuleDestroy {
private readonly pageRepo: PageRepo,
private readonly collabHistory: CollabHistoryService,
private readonly watcherService: WatcherService,
@InjectKysely() private readonly db: KyselyDB,
@InjectQueue(QueueName.NOTIFICATION_QUEUE) private notificationQueue: Queue,
@InjectQueue(QueueName.GENERAL_QUEUE) private generalQueue: Queue,
) {
@@ -41,6 +45,9 @@ export class HistoryProcessor extends WorkerHost implements OnModuleDestroy {
try {
const { pageId } = job.data;
// Read the page WITHOUT a lock first, only to bail early on the two cheap
// no-write cases (page gone / empty first snapshot) without opening a
// transaction. The authoritative check-then-write happens locked below.
const page = await this.pageRepo.findById(pageId, {
includeContent: true,
});
@@ -51,40 +58,109 @@ export class HistoryProcessor extends WorkerHost implements OnModuleDestroy {
return;
}
const lastHistory = await this.pageHistoryRepo.findPageLastHistory(
pageId,
{ includeContent: true },
);
// #370 F3 — the snapshot decision (findPageLastHistory → saveHistory) must
// be serialized against manual-save/boundary writers, which run under a
// page-row lock in onStoreDocument. Without it, this processor and a
// concurrent manual-save each read the same lastHistory (MVCC), both see
// content != lastHistory, and both insert — producing two page_history rows
// with IDENTICAL content (one 'idle', one 'manual'), defeating
// promote-not-dup and the version-vs-autosave split. Taking the same
// page-row lock makes the second writer observe the first's committed row so
// the isDeepStrictEqual gate collapses the duplicate. Only the read+write
// is transacted; the post-snapshot queue work stays outside.
let contributorIds: string[] = [];
let snapshotWritten = false;
let lastHistoryContent: unknown;
// #370 F8 — the contributor set popped from Redis (destructive SPOP) must be
// restored if the snapshot does not durably land. The inner try/catch only
// covers a throw INSIDE the callback; a COMMIT failure (connection drop,
// serialization/deadlock abort on commit — the transient class the epic
// already retries) throws OUTSIDE it, rolling the snapshot back while the
// pop is already gone. We track the popped set here and restore it in the
// outer catch so a BullMQ retry re-attributes the version. addContributors
// is an idempotent Redis SADD, so a double-restore is harmless.
let poppedForRestore: string[] = [];
if (!lastHistory && isEmptyParagraphDoc(page.content as any)) {
this.logger.debug(
`Skipping first history for page ${pageId}: empty content`,
);
await this.collabHistory.clearContributors(pageId);
try {
await executeTx(this.db, async (trx) => {
const lockedPage = await this.pageRepo.findById(pageId, {
includeContent: true,
withLock: true,
trx,
});
if (!lockedPage) return;
const lastHistory = await this.pageHistoryRepo.findPageLastHistory(
pageId,
{ includeContent: true, trx },
);
lastHistoryContent = lastHistory?.content;
if (!lastHistory && isEmptyParagraphDoc(lockedPage.content as any)) {
this.logger.debug(
`Skipping first history for page ${pageId}: empty content`,
);
return;
}
if (
lastHistory &&
isDeepStrictEqual(lastHistory.content, lockedPage.content)
) {
return; // already snapshotted at this content — nothing to write
}
contributorIds = await this.collabHistory.popContributors(pageId);
poppedForRestore = contributorIds;
try {
// Pass `trx` so the watcher insert's FK check (FOR KEY SHARE on
// pages[pageId]) runs on the SAME connection that already holds the
// FOR UPDATE lock from findById — otherwise it takes the FK lock on a
// separate pool connection and self-deadlocks against our own tx.
await this.watcherService.addPageWatchers(
contributorIds,
pageId,
lockedPage.spaceId,
lockedPage.workspaceId,
trx,
);
// #370 — every job on this queue is a trailing idle-flush autosnapshot.
await this.pageHistoryRepo.saveHistory(lockedPage, {
contributorIds,
kind: job.data.kind ?? 'idle',
trx,
});
snapshotWritten = true;
this.logger.debug(`History created for page: ${pageId}`);
} catch (err) {
await this.collabHistory.addContributors(pageId, contributorIds);
poppedForRestore = [];
throw err;
}
});
} catch (err) {
// A throw here means the tx did NOT commit (callback threw, or the commit
// itself failed and rolled back). If we popped contributors and the inner
// catch did not already restore them, restore now so the retry keeps
// attribution. snapshotWritten is irrelevant: it is set before commit, so
// it can be true even when the commit rolled the snapshot back.
if (poppedForRestore.length) {
await this.collabHistory.addContributors(pageId, poppedForRestore);
}
throw err;
}
// No snapshot written (page vanished / empty-first / unchanged content) →
// clear the contributor set for the skip cases and stop.
if (!snapshotWritten) {
if (!lastHistoryContent && isEmptyParagraphDoc(page.content as any)) {
await this.collabHistory.clearContributors(pageId);
}
return;
}
if (
!lastHistory ||
!isDeepStrictEqual(lastHistory.content, page.content)
) {
const contributorIds = await this.collabHistory.popContributors(pageId);
try {
await this.watcherService.addPageWatchers(
contributorIds,
pageId,
page.spaceId,
page.workspaceId,
);
await this.pageHistoryRepo.saveHistory(page, { contributorIds });
this.logger.debug(`History created for page: ${pageId}`);
} catch (err) {
await this.collabHistory.addContributors(pageId, contributorIds);
throw err;
}
{
const mentions = extractMentions(page.content);
const pageMentions = extractPageMentions(mentions);
const internalLinkSlugIds = extractInternalLinkSlugIds(page.content);
@@ -102,7 +178,7 @@ export class HistoryProcessor extends WorkerHost implements OnModuleDestroy {
);
});
if (contributorIds.length > 0 && lastHistory?.content) {
if (contributorIds.length > 0 && lastHistoryContent) {
await this.notificationQueue
.add(QueueJob.PAGE_UPDATED, {
pageId,
@@ -1,109 +0,0 @@
import {
ConflictException,
Logger,
ServiceUnavailableException,
} from '@nestjs/common';
import { AiChatService } from './ai-chat.service';
import { RunAlreadyActiveError } from './ai-chat-run.service';
/**
* Fail-fast guard for beginRun failures (#486, commit 4).
*
* When runHooks.begin() rejects for a reason OTHER than RunAlreadyActiveError
* (e.g. a DB-pool blip), the turn must NOT continue untracked. The old code
* logged and streamed anyway, leaving a run with NO run-row: in autonomous mode
* nobody could abort it (/stop can't see it, disconnect doesn't abort it, and the
* one-run gate would admit a SECOND run) — an unstoppable invisible run until
* restart. The fix throws A_RUN_BEGIN_FAILED (503) BEFORE the first byte and
* before the user row is persisted.
*
* We drive `stream()` directly on a prototype instance wired with only the
* collaborators it touches before the throw, so the assertion is on the REAL
* control flow, not a mock of it.
*/
describe('AiChatService beginRun failure (#486)', () => {
function makeService(insertSpy: jest.Mock): AiChatService {
// Bypass the (heavy) DI constructor: exercise the real stream() method on a
// bare prototype instance with just the fields reached before the throw.
// `any` because the private `logger` field makes a typed intersection collapse.
const svc = Object.create(AiChatService.prototype);
svc.aiChatRepo = {
// Existing chat -> no insert path; chatId is kept as-is.
findById: jest.fn().mockResolvedValue({ id: 'chat1' }),
};
svc.aiChatMessageRepo = { insert: insertSpy };
svc.logger = new Logger('test');
return svc as AiChatService;
}
const baseArgs = () => {
const write = jest.fn();
const res = {
raw: { write, writableEnded: false, headersSent: false },
};
return {
user: { id: 'u1' } as never,
workspace: { id: 'w1' } as never,
sessionId: 's1',
// openPage undefined -> resolveOpenPageContext returns null without any DB
// call; chatId present -> the existing-chat path.
body: { chatId: 'chat1', messages: [] } as never,
res: res as never,
signal: new AbortController().signal,
model: {} as never,
role: null,
write,
};
};
it('throws A_RUN_BEGIN_FAILED (503) before the first byte and before persisting the user turn', async () => {
const insertSpy = jest.fn();
const svc = makeService(insertSpy);
const { write, ...args } = baseArgs();
const runHooks = {
begin: jest.fn().mockRejectedValue(new Error('DB pool exhausted')),
} as never;
let caught: unknown;
try {
await svc.stream({ ...args, runHooks });
} catch (e) {
caught = e;
}
expect(caught).toBeInstanceOf(ServiceUnavailableException);
const http = caught as ServiceUnavailableException;
expect(http.getStatus()).toBe(503);
expect(http.getResponse()).toMatchObject({ code: 'A_RUN_BEGIN_FAILED' });
// Fail-fast: nothing was written to the socket and NO user message row was
// persisted, so the turn left no orphan state to clean up.
expect(write).not.toHaveBeenCalled();
expect(insertSpy).not.toHaveBeenCalled();
});
it('still maps a lost-the-race RunAlreadyActiveError to a 409, not A_RUN_BEGIN_FAILED', async () => {
const insertSpy = jest.fn();
const svc = makeService(insertSpy);
const { write, ...args } = baseArgs();
const runHooks = {
begin: jest.fn().mockRejectedValue(new RunAlreadyActiveError('chat1')),
} as never;
let caught: unknown;
try {
await svc.stream({ ...args, runHooks });
} catch (e) {
caught = e;
}
expect(caught).toBeInstanceOf(ConflictException);
expect((caught as ConflictException).getResponse()).toMatchObject({
code: 'A_RUN_ALREADY_ACTIVE',
});
expect(write).not.toHaveBeenCalled();
expect(insertSpy).not.toHaveBeenCalled();
});
});
@@ -1,8 +1,4 @@
import {
ConflictException,
Logger,
ServiceUnavailableException,
} from '@nestjs/common';
import { ConflictException, Logger } from '@nestjs/common';
// Mock the AI SDK so we can PROVE no provider call is made for the turn we are
// about to reject. The race rejection happens at runHooks.begin(), long before
@@ -364,22 +360,22 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
});
/**
* F14 — the begin-failure branch (the `else` of the run-race guard).
* F14 — the begin-failure RESILIENCE branch (the `else` of the run-race guard).
*
* stream() wraps runHooks.begin in try/catch with TWO branches:
* - RunAlreadyActiveError -> 409 ConflictException (pinned above).
* - ANY OTHER begin failure -> throw ServiceUnavailableException(A_RUN_BEGIN_FAILED)
* BEFORE the first byte (#486, commit 4).
* - ANY OTHER begin failure -> SWALLOW + continue UNTRACKED on the socket signal
* (legacy fallback): it logs "...streaming without run tracking", leaves
* `effectiveSignal = signal` (runId undefined) and serves the turn anyway.
*
* POLICY CHANGE (#486): the OLD contract here was "SWALLOW + stream the turn
* UNTRACKED on the socket signal". That was reversed: an untracked run is
* invisible to /stop, is not aborted on disconnect, and slips past the one-run
* gate — an unstoppable ghost run in autonomous mode. Now a plain begin failure
* FAILS the turn fast with a 503 A_RUN_BEGIN_FAILED, before any user row is
* persisted and before streamText runs. This case is INVERTED (not deleted) so
* the "plain begin failure" path stays explicitly pinned under the new policy.
* The contract: a transient beginRun failure (e.g. a non-unique DB error inserting
* the run row) must STILL serve the user's turn — it must NOT re-throw and must NOT
* be misclassified as a 409. A regression that re-threw here would break EVERY turn
* on a begin failure with nothing to catch it. This branch is otherwise undriven by
* any spec, so it is pinned here SEPARATELY from the 409 path: a plain begin error
* proceeds to streamText with the SOCKET signal and still persists the user turn.
*/
describe('AiChatService.stream — begin-failure fails the turn (#184 F14 / #486)', () => {
describe('AiChatService.stream — begin-failure resilience / legacy fallback (#184 F14)', () => {
const streamTextMock = streamText as unknown as jest.Mock;
function makeStreamResult() {
@@ -459,7 +455,7 @@ describe('AiChatService.stream — begin-failure fails the turn (#184 F14 / #486
afterEach(() => jest.restoreAllMocks());
it('a PLAIN begin() failure (NOT RunAlreadyActiveError) FAILS the turn with a 503 A_RUN_BEGIN_FAILED before the first byte — NO untracked stream (#486)', async () => {
it('a PLAIN begin() failure (NOT RunAlreadyActiveError) does NOT 409 — it swallows, logs, and streams the turn UNTRACKED on the socket signal', async () => {
const errorSpy = jest
.spyOn(Logger.prototype, 'error')
.mockImplementation(() => undefined as never);
@@ -491,26 +487,28 @@ describe('AiChatService.stream — begin-failure fails the turn (#184 F14 / #486
} as never,
});
// NEW POLICY: the turn is REJECTED with a 503 A_RUN_BEGIN_FAILED (not a 409,
// and NOT swallowed into an untracked stream).
await expect(promise).rejects.toBeInstanceOf(ServiceUnavailableException);
const err = (await promise.catch(
(e) => e,
)) as ServiceUnavailableException;
expect(err.getStatus()).toBe(503);
expect(err.getResponse()).toMatchObject({ code: 'A_RUN_BEGIN_FAILED' });
// The turn proceeds: NO throw at all (in particular NOT a 409).
await expect(promise).resolves.toBeUndefined();
expect(begin).toHaveBeenCalledTimes(1);
// It logged the fail-the-turn line.
// The resilience branch logged the legacy-fallback warning.
expect(errorSpy).toHaveBeenCalledWith(
expect.stringContaining('failing the turn'),
expect.stringContaining('streaming without run tracking'),
expect.anything(),
);
// Fail-fast: the turn NEVER streamed — no user row persisted, no streamText
// call, so no orphan/untracked run was left behind.
expect(aiChatMessageRepo.insert).not.toHaveBeenCalled();
expect(streamTextMock).not.toHaveBeenCalled();
// The turn really streamed: the user message was persisted and streamText ran.
expect(aiChatMessageRepo.insert).toHaveBeenCalled();
expect(streamTextMock).toHaveBeenCalledTimes(1);
// The decisive wiring: with no run handle, the fallback uses the SOCKET signal
// (effectiveSignal = signal, runId undefined) — not a run-bound signal. #444:
// the signal is unioned with the degeneration controller via AbortSignal.any,
// so assert the socket abort still reaches the turn rather than identity.
const passed = streamTextMock.mock.calls[0][0].abortSignal as AbortSignal;
expect(passed.aborted).toBe(false);
socketController.abort();
expect(passed.aborted).toBe(true);
});
});
@@ -4,7 +4,6 @@ import {
Injectable,
Logger,
OnModuleInit,
ServiceUnavailableException,
} from '@nestjs/common';
import { FastifyReply } from 'fastify';
import {
@@ -56,7 +55,6 @@ import {
import {
isDegenerateOutput,
truncateDegeneratedTail,
shouldCheckDegeneration,
} from './output-degeneration';
// Max agent steps per turn. One step = one model generation; a step that calls
@@ -758,13 +756,6 @@ export class AiChatService implements OnModuleInit {
// or violate the page_id FK on insert (this runs after res.hijack(), so a
// DB error would break the stream).
const originPageId: string | null = openPageContext?.id ?? null;
// ORPHAN-ON-BEGIN-FAILURE tradeoff (#486, B3): the chat row is inserted
// HERE, before runHooks.begin below. If begin fails (e.g. a 503 / run-slot
// rejection) the turn aborts before the client is told this new chatId, so
// an empty chat is left behind and a retry mints ANOTHER one. We accept this
// over reordering: begin needs a chatId to bind the run to, and inserting
// the chat first keeps the id stable + the FK/history-join invariants above
// intact. Orphan empty chats are cheap and swept by normal chat cleanup.
const chat = await this.aiChatRepo.insert({
creatorId: user.id,
workspaceId: workspace.id,
@@ -806,32 +797,12 @@ export class AiChatService implements OnModuleInit {
code: 'A_RUN_ALREADY_ACTIVE',
});
}
// Any OTHER run-start failure (e.g. a DB-pool blip) must FAIL THE TURN,
// not silently stream without a run-row. The old fallback let the turn
// continue untracked: in autonomous mode nobody could then abort it —
// /stop can't see a run that doesn't exist, a client disconnect doesn't
// abort it, and the one-run-per-chat gate would let a SECOND run in. That
// is an unstoppable, invisible run until process restart. Reject NOW,
// BEFORE the first byte (nothing is written yet, no user row inserted, no
// MCP lease taken), so the controller's post-hijack catch turns this
// HttpException into an honest 503 on the raw socket. Same policy for BOTH
// modes — #487 inherits it (no mode-branching here).
// Any OTHER run-start failure must not break the turn — fall back to the
// socket signal (legacy behavior) and stream anyway.
this.logger.error(
`Failed to begin agent run (chat ${chatId}); failing the turn`,
`Failed to begin agent run (chat ${chatId}); streaming without run tracking`,
err as Error,
);
throw new ServiceUnavailableException({
message:
'Could not start the agent run. This is usually temporary — please try again.',
code: 'A_RUN_BEGIN_FAILED',
// Self-describe the status in the body: the controller's post-hijack
// catch writes getResponse() verbatim onto the raw socket, and an
// object-arg HttpException does NOT inject statusCode. Without it the
// client's 503 classifier (which reads the body JSON) could not see the
// status. With it present, the client's A_RUN_BEGIN_FAILED branch (which
// runs strictly before the generic-503 branch) shows "temporary, retry".
statusCode: 503,
});
}
}
@@ -1109,6 +1080,7 @@ export class AiChatService implements OnModuleInit {
const degenerationController = new AbortController();
let degenerationDetected = false;
let lastDegenerationCheckLen = 0;
const DEGENERATION_CHECK_STEP = 2000;
// Step-granular durability (#183): create the assistant row UPFRONT in the
// 'streaming' state (before any token), then UPDATE it as each step finishes
@@ -1282,10 +1254,8 @@ export class AiChatService implements OnModuleInit {
// trigger, abort the run ONCE with a distinguishable reason.
if (
!degenerationDetected &&
shouldCheckDegeneration(
inProgressText.length,
lastDegenerationCheckLen,
)
inProgressText.length - lastDegenerationCheckLen >=
DEGENERATION_CHECK_STEP
) {
lastDegenerationCheckLen = inProgressText.length;
if (isDegenerateOutput(inProgressText)) {
@@ -1305,13 +1275,6 @@ export class AiChatService implements OnModuleInit {
// the in-progress accumulator for the next step.
capturedSteps.push(step as StepLike);
inProgressText = '';
// Reset the degeneration-check watermark too (#486): it tracks a byte
// offset INTO inProgressText, so once that resets to '' a stale (large)
// mark makes `inProgressText.length - lastDegenerationCheckLen` go
// negative and the throttled detector stays silent until a later step's
// text re-grows past the old offset — a whole degenerate step could slip
// through undetected. Zeroing it re-arms the check from the next byte.
lastDegenerationCheckLen = 0;
// Step-granular durability (#183): persist this finished step (its text +
// tool calls + tool RESULTS) the moment it ends, so a process death after
// this point still recovers the step. Not awaited here (never block the
@@ -1,24 +1,11 @@
import { Logger } from '@nestjs/common';
import { streamText } from 'ai';
import {
hasRepeatedLineRun,
hasPeriodicTail,
isDegenerateOutput,
truncateDegeneratedTail,
shouldCheckDegeneration,
DEGENERATION_CHECK_STEP,
REPEATED_LINES_THRESHOLD,
MIN_PERIOD_REPEATS,
} from './output-degeneration';
import { AiChatService } from './ai-chat.service';
// Mock ONLY streamText so we can capture the onChunk/onStepFinish callbacks the
// service registers and drive them by hand; every other `ai` export the service
// uses (convertToModelMessages, stepCountIs, …) stays real.
jest.mock('ai', () => {
const actual = jest.requireActual('ai');
return { ...actual, streamText: jest.fn() };
});
/**
* Unit tests for the token-degeneration detector (#444) — the sole anti-babble
@@ -193,188 +180,3 @@ describe('truncateDegeneratedTail', () => {
expect(truncateDegeneratedTail(text)).toBe(text);
});
});
/**
* Throttle + step-boundary reset (#486). The stream keeps a watermark
* (`lastDegenerationCheckLen`) that is an OFFSET into the accumulated step text.
* On a step boundary the accumulator resets to '', so the watermark MUST reset to
* 0 too — otherwise the throttle goes silent for the whole next step. These tests
* pin the pure decision AND the reset property that ai-chat.service.onStepFinish
* now enforces.
*/
describe('shouldCheckDegeneration (throttle) + step-boundary reset (#486)', () => {
it('fires once the text grows a full DEGENERATION_CHECK_STEP past the mark', () => {
expect(shouldCheckDegeneration(DEGENERATION_CHECK_STEP, 0)).toBe(true);
expect(shouldCheckDegeneration(DEGENERATION_CHECK_STEP - 1, 0)).toBe(false);
expect(shouldCheckDegeneration(5000, 3000)).toBe(true); // grew 2000 since mark
expect(shouldCheckDegeneration(4000, 3000)).toBe(false); // grew only 1000
});
it('BUG (no reset): a stale large watermark silences the next step', () => {
// End of a long step: the watermark sits at 5000. The step ends and the
// accumulator resets to '' — but if the watermark is NOT reset, a fresh short
// degenerate burst (length 2000) never triggers a check: 2000 - 5000 < STEP.
const staleWatermark = 5000;
const nextStepLen = DEGENERATION_CHECK_STEP; // a fresh 2KB burst
expect(shouldCheckDegeneration(nextStepLen, staleWatermark)).toBe(false);
});
it('FIX (reset to 0): the same short degenerate burst IS checked and detected', () => {
// onStepFinish now zeroes the watermark, so the fresh burst re-arms the check.
const resetWatermark = 0;
const degenerateBurst = 'loadTools.\n'.repeat(300); // real degeneration
expect(degenerateBurst.length).toBeGreaterThanOrEqual(DEGENERATION_CHECK_STEP);
// The throttle now fires...
expect(
shouldCheckDegeneration(degenerateBurst.length, resetWatermark),
).toBe(true);
// ...and the detector catches the loop that would otherwise stream unchecked.
expect(isDegenerateOutput(degenerateBurst)).toBe(true);
});
});
/**
* BEHAVIOR guard for the ACTUAL fix (#486, ai-chat.service.onStepFinish resets
* lastDegenerationCheckLen to 0). The pure tests above use a hard-coded
* resetWatermark, so a REVERT of the real `lastDegenerationCheckLen = 0` line
* would not redden any of them. This drives the REAL onChunk/onStepFinish
* closures from stream() end to end and asserts the run is aborted when a fresh
* degenerate burst arrives in the step AFTER a long clean step — which only
* happens if the watermark was actually zeroed on the step boundary.
*/
describe('AiChatService: onStepFinish re-arms the degeneration watermark (#486)', () => {
const streamTextMock = streamText as unknown as jest.Mock;
function makeRes() {
return {
raw: {
writeHead: jest.fn(),
write: jest.fn(),
once: jest.fn(),
on: jest.fn(),
flushHeaders: jest.fn(),
writableEnded: false,
destroyed: false,
},
};
}
function makeService() {
const aiChatRepo = {
findById: jest.fn(async () => ({ id: 'chat-1', workspaceId: 'ws-1' })),
insert: jest.fn(),
};
const aiChatMessageRepo = {
insert: jest.fn(async () => ({ id: 'msg-1' })),
findAllByChat: jest.fn(async () => []),
update: jest.fn(async () => ({ id: 'msg-1' })),
};
const aiSettings = { resolve: jest.fn(async () => ({})) };
const tools = { forUser: jest.fn(async () => ({})) };
const mcpClients = {
toolsFor: jest.fn(async () => ({
tools: {},
clients: [],
outcomes: [],
instructions: [],
})),
};
return new AiChatService(
{} as never, // ai
aiChatRepo as never,
aiChatMessageRepo as never,
{} as never, // aiChatPageSnapshotRepo
aiSettings as never,
tools as never,
mcpClients as never,
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo
{} as never, // pageAccess
{
isAiChatDeferredToolsEnabled: () => false,
// Lockdown OFF -> the degeneration guard is the active anti-babble path.
isAiChatFinalStepLockdownEnabled: () => false,
} as never, // environment
);
}
beforeEach(() => {
streamTextMock.mockReset();
jest.spyOn(Logger.prototype, 'log').mockImplementation(() => undefined as never);
jest.spyOn(Logger.prototype, 'warn').mockImplementation(() => undefined as never);
});
afterEach(() => jest.restoreAllMocks());
it('aborts on a fresh degenerate burst in the NEXT step (reverting the reset line reddens this)', async () => {
let captured:
| {
onChunk?: (e: { chunk: { type: string; text: string } }) => void;
onStepFinish?: (step: unknown) => void;
abortSignal?: AbortSignal;
}
| undefined;
streamTextMock.mockImplementation((opts: never) => {
captured = opts;
return {
consumeStream: jest.fn(),
pipeUIMessageStreamToResponse: jest.fn(),
};
});
const svc = makeService();
await svc.stream({
user: { id: 'user-1' } as never,
workspace: { id: 'ws-1' } as never,
sessionId: 'sess-1',
body: {
chatId: 'chat-1',
messages: [
{ id: 'm1', role: 'user', parts: [{ type: 'text', text: 'hi' }] },
],
} as never,
res: makeRes() as never,
signal: new AbortController().signal,
model: {} as never,
role: null,
// No runHooks -> legacy path (socket signal), degeneration guard active.
});
expect(streamTextMock).toHaveBeenCalledTimes(1);
const onChunk = captured!.onChunk!;
const onStepFinish = captured!.onStepFinish!;
const abortSignal = captured!.abortSignal!;
expect(abortSignal.aborted).toBe(false);
// STEP 1: a LONG, non-degenerate first step. Distinct lines never trip the
// detector, but they advance the throttle watermark far past the burst size
// that follows (to ~5x the step). This is the stale watermark that, WITHOUT
// the reset, would silence step 2.
let counter = 0;
let accumulated = 0;
while (accumulated < DEGENERATION_CHECK_STEP * 5) {
const line = `unique clean line number ${counter++} with distinct words\n`;
accumulated += line.length;
onChunk({ chunk: { type: 'text-delta', text: line } });
}
expect(abortSignal.aborted).toBe(false); // clean step must not abort
// STEP BOUNDARY: the real onStepFinish resets inProgressText AND (the fix)
// zeroes lastDegenerationCheckLen.
onStepFinish({ text: 'a clean first step', toolCalls: [], toolResults: [] });
// STEP 2: a FRESH, short degenerate burst (~3.3KB). Its length is far below
// the step-1 stale watermark (~10KB), so WITHOUT the reset the throttle stays
// silent and this streams unchecked. WITH the reset (watermark 0) it re-arms,
// the detector fires, and the run aborts.
const burst = 'loadTools.\n'.repeat(300);
expect(burst.length).toBeGreaterThanOrEqual(DEGENERATION_CHECK_STEP);
expect(burst.length).toBeLessThan(DEGENERATION_CHECK_STEP * 5);
onChunk({ chunk: { type: 'text-delta', text: burst } });
// The decisive assertion: the composed abortSignal (unioned with the
// degeneration controller) is now aborted. Reverting `lastDegenerationCheckLen
// = 0` in onStepFinish makes this stay false.
expect(abortSignal.aborted).toBe(true);
});
});
@@ -131,32 +131,6 @@ export function isDegenerateOutput(text: string): boolean {
return hasRepeatedLineRun(text) || hasPeriodicTail(text);
}
/**
* How many bytes the in-progress text must grow before the (amortized) tail
* heuristics are re-run. Shared with ai-chat.service so the throttle the stream
* applies is the SAME one the unit test drives.
*/
export const DEGENERATION_CHECK_STEP = 2000;
/**
* Throttle decision for the degeneration guard (#444/#486). Returns true when
* the accumulated text has grown at least DEGENERATION_CHECK_STEP bytes past the
* last-checked offset, so the pure rules only fire every ~2KB. Pure; the caller
* updates its watermark to `textLen` when this returns true.
*
* The watermark is an offset INTO the accumulator, so when the accumulator is
* reset to '' on a step boundary the caller MUST reset the watermark to 0 too
* (#486). Otherwise `textLen - lastCheckLen` goes negative after the reset and
* this returns false until a later step re-grows past the stale offset — a whole
* degenerate step could stream unchecked.
*/
export function shouldCheckDegeneration(
textLen: number,
lastCheckLen: number,
): boolean {
return textLen - lastCheckLen >= DEGENERATION_CHECK_STEP;
}
/**
* Truncate a degenerated tail before persist so hundreds of KB of garbage never
* reach the DB / replay (#444). Keeps everything up to and including the FIRST
@@ -1,241 +0,0 @@
// Break the editor-ext import chain (share.service -> collaboration.util ->
// @docmost/editor-ext -> @tiptap/core) that is unresolvable in this jest env and
// pre-existingly breaks these specs. jsonToMarkdown is never reached in these
// tests (the tools fail before rendering markdown).
jest.mock('../../collaboration/collaboration.util', () => ({
jsonToMarkdown: () => '',
}));
import { Logger } from '@nestjs/common';
import { MockLanguageModelV3, simulateReadableStream } from 'ai/test';
import { PublicShareChatService } from './public-share-chat.service';
import { PublicShareChatToolsService } from './tools/public-share-chat-tools.service';
/**
* SECURITY integration guard for #394 (commit 5): a tool's or the provider's raw
* error text must NOT leak to an anonymous public-share reader.
*
* The render gate (ToolCallCard showErrors=false) hides the text in the DOM but
* NOT on the wire, so this test asserts on the RAW SSE BYTES the server writes —
* exactly the channel the render gate masks. We drive the real
* PublicShareChatService.stream() with a real share toolset (its underlying
* services mocked to fail) and a mock model, then inspect every byte piped to the
* fake socket.
*/
// A minimal ServerResponse stand-in that records every written chunk.
class FakeSocket {
chunks: string[] = [];
statusCode = 200;
writableEnded = false;
destroyed = false;
headersSent = false;
writeHead(): this {
this.headersSent = true;
return this;
}
setHeader(): void {}
removeHeader(): void {}
getHeader(): undefined {
return undefined;
}
flushHeaders(): void {}
write(chunk: unknown): boolean {
this.chunks.push(
typeof chunk === 'string' ? chunk : Buffer.from(chunk as never).toString('utf8'),
);
return true;
}
end(chunk?: unknown): void {
if (chunk) this.write(chunk);
this.writableEnded = true;
}
on(): this {
return this;
}
once(): this {
return this;
}
get body(): string {
return this.chunks.join('');
}
}
/** Mock model that issues one getSharePage tool call, then finishes with text. */
function toolCallingModel(): MockLanguageModelV3 {
let call = 0;
return new MockLanguageModelV3({
doStream: async () => {
call++;
if (call === 1) {
return {
stream: simulateReadableStream({
chunks: [
{ type: 'stream-start' as const, warnings: [] },
{ type: 'tool-input-start' as const, id: 't1', toolName: 'getSharePage' },
{ type: 'tool-input-end' as const, id: 't1' },
{
type: 'tool-call' as const,
toolCallId: 't1',
toolName: 'getSharePage',
input: '{"pageId":"secret-page"}',
},
{
type: 'finish' as const,
finishReason: { unified: 'tool-calls' as const, raw: 'tool_calls' },
usage: {
inputTokens: { total: 1, noCache: undefined, cacheRead: undefined, cacheWrite: undefined },
outputTokens: { total: 1, text: 1, reasoning: undefined },
},
},
],
}),
};
}
return {
stream: simulateReadableStream({
chunks: [
{ type: 'stream-start' as const, warnings: [] },
{ type: 'text-start' as const, id: '1' },
{ type: 'text-delta' as const, id: '1', delta: 'Sorry.' },
{ type: 'text-end' as const, id: '1' },
{
type: 'finish' as const,
finishReason: { unified: 'stop' as const, raw: 'stop' },
usage: {
inputTokens: { total: 1, noCache: undefined, cacheRead: undefined, cacheWrite: undefined },
outputTokens: { total: 1, text: 1, reasoning: undefined },
},
},
],
}),
};
},
});
}
/** Mock model whose stream emits a provider error carrying an internal secret. */
function providerErrorModel(secret: string): MockLanguageModelV3 {
return new MockLanguageModelV3({
doStream: async () => ({
stream: simulateReadableStream({
chunks: [
{ type: 'stream-start' as const, warnings: [] },
{
type: 'error' as const,
error: {
statusCode: 503,
message: 'Service Unavailable',
responseBody: `upstream ${secret} model=internal-gpt`,
},
},
],
}),
}),
});
}
function makeService(toolsService: PublicShareChatToolsService): {
svc: PublicShareChatService;
logSpy: jest.SpyInstance;
} {
const svc = Object.create(PublicShareChatService.prototype);
const logger = new Logger('test');
const logSpy = jest.spyOn(logger, 'error').mockImplementation(() => undefined);
jest.spyOn(logger, 'warn').mockImplementation(() => undefined);
svc.tools = toolsService;
svc.logger = logger;
svc.tokenBudget = { record: jest.fn().mockResolvedValue(undefined) };
return { svc, logSpy };
}
async function runStream(
svc: PublicShareChatService,
model: MockLanguageModelV3,
): Promise<FakeSocket> {
const socket = new FakeSocket();
await svc.stream({
workspaceId: 'ws1',
shareId: 'share1',
share: { id: 'share1', pageId: 'p1', sharedPage: { id: 'p1', title: 'Docs' } },
openedPage: null,
messages: [
{ id: 'm1', role: 'user', parts: [{ type: 'text', text: 'read the page' }] } as never,
],
res: { raw: socket } as never,
signal: new AbortController().signal,
model: model as never,
role: null,
});
// Let the piped stream drain fully.
await new Promise((r) => setTimeout(r, 300));
return socket;
}
describe('public share chat error leak (#394)', () => {
afterEach(() => jest.restoreAllMocks());
it('does NOT leak a tool\'s raw internal error to the SSE bytes (generic classified string instead)', async () => {
const SECRET = 'INTERNAL_baseUrl_http://provider.internal:8080/v1';
const shareService = {
// The canonical boundary throws a RAW internal error (with a secret).
resolveReadableSharePage: jest
.fn()
.mockRejectedValue(new Error(`db failed at ${SECRET} stack@line42`)),
};
const tools = new PublicShareChatToolsService(
shareService as never,
{} as never,
{} as never,
);
const { svc } = makeService(tools);
const socket = await runStream(svc, toolCallingModel());
// The tool-output-error frame is present on the wire...
expect(socket.body).toContain('tool-output-error');
// ...but it carries ONLY the generic classified string — never the secret,
// the raw driver message, or a stack fragment.
expect(socket.body).toContain('The tool could not complete the request.');
expect(socket.body).not.toContain(SECRET);
expect(socket.body).not.toContain('stack@line42');
expect(socket.body).not.toContain('db failed');
});
it('passes a SAFE ShareToolError message (page not available) through to the bytes', async () => {
const shareService = {
// Not found in this share -> the tool throws the classified SAFE message.
resolveReadableSharePage: jest.fn().mockResolvedValue(null),
};
const tools = new PublicShareChatToolsService(
shareService as never,
{} as never,
{} as never,
);
const { svc } = makeService(tools);
const socket = await runStream(svc, toolCallingModel());
expect(socket.body).toContain('tool-output-error');
expect(socket.body).toContain('not available in this share');
});
it('does NOT leak a provider error (statusCode + response body) to the SSE bytes', async () => {
const SECRET = 'http://provider.internal:8080';
const tools = new PublicShareChatToolsService(
{} as never,
{} as never,
{} as never,
);
const { svc, logSpy } = makeService(tools);
const socket = await runStream(svc, providerErrorModel(SECRET));
// The anon sees a fixed classified string, not the provider body/baseUrl/model.
expect(socket.body).toContain('temporarily unavailable');
expect(socket.body).not.toContain(SECRET);
expect(socket.body).not.toContain('internal-gpt');
// The FULL provider detail is logged server-side only.
const logged = logSpy.mock.calls.map((c) => String(c[0])).join('\n');
expect(logged).toContain(SECRET);
});
});
@@ -12,10 +12,7 @@ import { AiAgentRoleRepo } from '@docmost/db/repos/ai-agent-roles/ai-agent-roles
import { AiAgentRole } from '@docmost/db/types/entity.types';
import { AiService } from '../../integrations/ai/ai.service';
import { AiSettingsService } from '../../integrations/ai/ai-settings.service';
import {
PublicShareChatToolsService,
ShareToolError,
} from './tools/public-share-chat-tools.service';
import { PublicShareChatToolsService } from './tools/public-share-chat-tools.service';
import { buildShareSystemPrompt } from './public-share-chat.prompt';
import { roleModelOverride } from './roles/role-model-config';
import {
@@ -105,30 +102,6 @@ export function filterShareTranscript(messages: UIMessage[]): UIMessage[] {
);
}
/**
* Fixed, classified strings an ANONYMOUS share reader may see when the assistant
* stream fails (#394). These reveal NOTHING about the internal provider, its
* baseUrl, the model name, or the raw response body — unlike describeProviderError
* (which is for the server log / the authenticated operator only). We classify by
* HTTP status where available so the reader still gets a useful hint (retry vs.
* give up) without any internal detail.
*/
export function classifyAnonStreamError(error: unknown): string {
const status =
typeof error === 'object' && error !== null
? (error as { statusCode?: number }).statusCode
: undefined;
if (status === 429) {
return 'The assistant is receiving too many requests right now. Please try again shortly.';
}
if (typeof status === 'number' && status >= 500) {
return 'The assistant is temporarily unavailable. Please try again.';
}
// Any other failure (including a bare connection error with no status): a
// single neutral line. No provider identity, no config, no response body.
return 'The assistant could not complete your request. Please try again.';
}
/**
* Anonymous, read-only AI assistant for a single PUBLIC share tree.
*
@@ -345,28 +318,11 @@ export class PublicShareChatService {
result.pipeUIMessageStreamToResponse(res.raw, {
headers: { 'X-Accel-Buffering': 'no' },
onError: (error: unknown) => {
// SECURITY (#394): the string this returns is written verbatim into the
// SSE error frame delivered to an ANONYMOUS reader (for a tool failure
// it becomes the atomic `tool-output-error` frame's errorText; for a
// stream/provider failure, the terminal error frame).
//
// A ShareToolError is already a classified, safe tool message (see
// PublicShareChatToolsService.wrapToolErrors) — pass it through so the
// reader still gets the useful "page not available in this share" hint.
if (error instanceof ShareToolError) {
return error.message;
}
// Anything else is a provider/stream error. describeProviderError
// bundles the provider statusCode AND response body, which can carry the
// internal baseUrl or model name — NEVER expose that to the public. Log
// the full detail server-side only and return a fixed classified string.
this.logger.error(
`Public share chat pipe error: ${describeProviderError(
error,
'AI stream error',
)}`,
);
return classifyAnonStreamError(error);
// Reuse the shared formatter so provider error formatting stays
// unified between the log line and the streamed error message — a
// share reader sees 402/429/503 causes consistently with the
// authenticated path.
return describeProviderError(error, 'AI stream error');
},
});
@@ -808,7 +808,7 @@ describe('PublicShareChatToolsService share scoping', () => {
};
await expect(getSharePage.execute({ pageId: 'p-outside' })).rejects.toThrow(
/not available in this share/i,
/not part of this published share/i,
);
// The tool delegated the resolve to the canonical boundary with the
// forShare-scoped shareId, and returned NO content for a non-resolving page.
@@ -841,7 +841,7 @@ describe('PublicShareChatToolsService share scoping', () => {
await expect(
getSharePage.execute({ pageId: 'p-restricted' }),
).rejects.toThrow(/not available in this share/i);
).rejects.toThrow(/not part of this published share/i);
// No content was ever sanitized/returned for the blocked page.
expect(shareService.updatePublicAttachments).not.toHaveBeenCalled();
});
@@ -1003,7 +1003,7 @@ describe('public-share assistant boundary locks (red-team regression guards)', (
};
await expect(
getSharePage.execute({ pageId: 'p-elsewhere' }),
).rejects.toThrow(/not available in this share/i);
).rejects.toThrow(/not part of this published share/i);
// The forged share id is the scope the boundary re-derivation rejects against.
expect(shareService.resolveReadableSharePage).toHaveBeenCalledWith(
'FORGED-SHARE',
@@ -1,15 +1,7 @@
import { createHash } from 'node:crypto';
import {
mkdtempSync,
mkdirSync,
writeFileSync,
rmSync,
readdirSync,
statSync,
readFileSync,
} from 'node:fs';
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { dirname, join, relative, sep } from 'node:path';
import { join } from 'node:path';
import { computeSrcRegistryStamp } from './docmost-client.loader';
@@ -38,14 +30,10 @@ function assertStaleGuard(
}
}
// Build a throwaway `<pkg>/build/index.js` + optional `<pkg>/src/` tree so
// `computeSrcRegistryStamp(<pkg>/build/index.js)` resolves src the same way the
// loader does (dirname(dirname(entry))/src). Since #486 the stamp hashes the WHOLE
// src tree, so a fixture is a { relPath: content } map. A bare string is sugar for
// a single `tool-specs.ts`; `null` means "no src tree" (the prod no-op path).
function makeFakePackage(
src: string | Record<string, string> | null,
): {
// Build a throwaway `<pkg>/build/index.js` + optional `<pkg>/src/tool-specs.ts`
// layout so `computeSrcRegistryStamp(<pkg>/build/index.js)` resolves src the same
// way the loader does (dirname(dirname(entry))/src/tool-specs.ts).
function makeFakePackage(toolSpecsSource: string | null): {
entry: string;
cleanup: () => void;
} {
@@ -54,15 +42,10 @@ function makeFakePackage(
mkdirSync(buildDir, { recursive: true });
const entry = join(buildDir, 'index.js');
writeFileSync(entry, '// fake @docmost/mcp build entry\n', 'utf8');
if (src !== null) {
const files =
typeof src === 'string' ? { 'tool-specs.ts': src } : src;
if (toolSpecsSource !== null) {
const srcDir = join(root, 'src');
for (const [rel, content] of Object.entries(files)) {
const full = join(srcDir, rel);
mkdirSync(dirname(full), { recursive: true });
writeFileSync(full, content, 'utf8');
}
mkdirSync(srcDir, { recursive: true });
writeFileSync(join(srcDir, 'tool-specs.ts'), toolSpecsSource, 'utf8');
}
return { entry, cleanup: () => rmSync(root, { recursive: true, force: true }) };
}
@@ -110,109 +93,34 @@ describe('computeSrcRegistryStamp (#447 stale-build guard)', () => {
}
});
// #486 CORE (negative): an edit to a NON-tool-specs src file (client.ts) with a
// rebuild NOT run must move the src stamp away from the built REGISTRY_STAMP, so
// the loader's stale-check refuses. Under the old tool-specs.ts-only hash this
// edit was invisible and a stale build/ served the old client.ts silently.
it('a client.ts edit (no rebuild) moves the src stamp -> loader refuses (#486)', () => {
// "Built" state: the package as it was compiled.
const built = makeFakePackage({
'tool-specs.ts': 'export const SPECS = 1;\n',
'client.ts': "export const impl = 'v1';\n",
});
// "Dev edited src, forgot to rebuild": client.ts changed, tool-specs.ts not.
const edited = makeFakePackage({
'tool-specs.ts': 'export const SPECS = 1;\n',
'client.ts': "export const impl = 'v2';\n",
});
try {
const builtStamp = computeSrcRegistryStamp(built.entry);
const editedStamp = computeSrcRegistryStamp(edited.entry);
expect(builtStamp).not.toBeNull();
expect(editedStamp).not.toBe(builtStamp);
// build/ still carries builtStamp; src now hashes to editedStamp -> refuse.
expect(() => assertStaleGuard(editedStamp, builtStamp as string)).toThrow(
STALE_BUILD_MESSAGE,
);
} finally {
built.cleanup();
edited.cleanup();
}
});
// *.generated.ts is excluded (the codegen's own output — a fixed-point cycle
// otherwise): its presence/content must not move the stamp.
it('excludes *.generated.ts from the stamp', () => {
const without = makeFakePackage({ 'tool-specs.ts': 'x\n' });
const withGen = makeFakePackage({
'tool-specs.ts': 'x\n',
'registry-stamp.generated.ts': 'export const REGISTRY_STAMP = "abc";\n',
});
try {
expect(computeSrcRegistryStamp(withGen.entry)).toBe(
computeSrcRegistryStamp(without.entry),
);
} finally {
without.cleanup();
withGen.cleanup();
}
});
// CROSS-IMPL EQUALITY (covers reviewer suggestion 2). The SAME fixed tree and
// CROSS-IMPL EQUALITY (covers reviewer suggestion 2). The SAME fixed input and
// EXPECTED hash are asserted in the mcp-side node test
// (packages/mcp/test/unit/registry-stamp.test.mjs) against the codegen's
// `computeRegistryStamp`. Asserting the SAME pair here against the loader's
// `computeSrcRegistryStamp` proves both implementations enumerate+normalize+hash
// `computeSrcRegistryStamp` proves both implementations normalize+hash
// identically; a divergence in EITHER side reddens one of the two tests.
const CROSS_IMPL_TREE = {
'tool-specs.ts': 'line1\r\nline2\n',
'client/read.ts': 'export const R = 1;\n',
'registry-stamp.generated.ts': 'export const REGISTRY_STAMP="ignored";\n',
};
const CROSS_IMPL_EXPECTED =
'131c1b9e4e2f5a7d6cef91ca8df619822b442f52bc45ebd09474a4c1d6728616';
it('matches the documented cross-impl hash for a fixed tree', () => {
const { entry, cleanup } = makeFakePackage(CROSS_IMPL_TREE);
it('matches the documented cross-impl hash for a fixed input', () => {
const FIXED_INPUT = 'line1\r\nline2\n';
const EXPECTED =
'683376e290829b482c2655745caffa7a1dccfa10afaa62dac2b42dd6c68d0f83';
const { entry, cleanup } = makeFakePackage(FIXED_INPUT);
try {
expect(computeSrcRegistryStamp(entry)).toBe(CROSS_IMPL_EXPECTED);
expect(computeSrcRegistryStamp(entry)).toBe(EXPECTED);
} finally {
cleanup();
}
});
it('the documented EXPECTED is the enumerate+normalize+sha256 of the tree', () => {
// Proves EXPECTED is not a magic constant but the documented computation — a
// local re-implementation of the loader's tree walk.
const { entry, cleanup } = makeFakePackage(CROSS_IMPL_TREE);
it('the documented EXPECTED is the normalize+sha256 of the fixed input', () => {
// Proves EXPECTED is not a magic constant but the documented computation.
const FIXED_INPUT = 'line1\r\nline2\n';
const normalized = FIXED_INPUT.replace(/\r\n/g, '\n').replace(/\n$/, '');
const expected = createHash('sha256')
.update(normalized, 'utf8')
.digest('hex');
const { entry, cleanup } = makeFakePackage(FIXED_INPUT);
try {
const srcDir = join(dirname(dirname(entry)), 'src');
const collect = (dir: string): string[] => {
const out: string[] = [];
for (const e of readdirSync(dir)) {
const f = join(dir, e);
if (statSync(f).isDirectory()) out.push(...collect(f));
else if (e.endsWith('.ts') && !e.endsWith('.generated.ts'))
out.push(f);
}
return out;
};
const files = collect(srcDir)
.map((abs) => ({ rel: relative(srcDir, abs).split(sep).join('/'), abs }))
.sort((a, b) => (a.rel < b.rel ? -1 : a.rel > b.rel ? 1 : 0));
const h = createHash('sha256');
for (const { rel, abs } of files) {
const n = readFileSync(abs, 'utf8')
.replace(/\r\n/g, '\n')
.replace(/\n$/, '');
h.update(rel, 'utf8');
h.update('\0', 'utf8');
h.update(n, 'utf8');
h.update('\0', 'utf8');
}
const localHash = h.digest('hex');
expect(computeSrcRegistryStamp(entry)).toBe(localHash);
expect(localHash).toBe(CROSS_IMPL_EXPECTED);
expect(computeSrcRegistryStamp(entry)).toBe(expected);
} finally {
cleanup();
}
@@ -1,6 +1,6 @@
import { createHash } from 'node:crypto';
import { existsSync, readdirSync, readFileSync, statSync } from 'node:fs';
import { dirname, join, relative, sep } from 'node:path';
import { existsSync, readFileSync } from 'node:fs';
import { dirname, join } from 'node:path';
import { pathToFileURL } from 'node:url';
import type { DocmostClient, SharedToolSpec } from '@docmost/mcp';
@@ -191,52 +191,33 @@ interface DocmostMcpModule {
* present. Returns the stamp string, or `null` when the source is absent (a prod
* image ships only build/, no src/). MUST stay byte-for-byte identical to
* packages/mcp/scripts/gen-registry-stamp.mjs's `computeRegistryStamp` so the
* build-time and src-time hashes agree: same file set (every src/**\/*.ts except
* *.generated.ts), same POSIX-relative sort, same per-file normalization (CRLF ->
* LF, strip a single trailing newline) with the same path+content framing, same
* sha256. Hashing the WHOLE src tree (not just tool-specs.ts) is #486: an edit to
* client.ts / a client/* module / comment-signal / drawio-* without a rebuild
* must also be caught, otherwise build/ silently serves the old code.
* build-time and src-time hashes agree: same input file (src/tool-specs.ts), same
* normalization (CRLF -> LF, strip a single trailing newline), same sha256.
*
* DEV vs PROD detection is by FILE EXISTENCE, not NODE_ENV: we resolve the
* package's own directory from `require.resolve('@docmost/mcp')` (which points at
* build/index.js) and look for ../src next to it. In a dev/test worktree that
* directory exists; in a prod image (build/ only, src/ stripped) it does not, so
* this returns null and the caller skips the check. Any error (ENOENT, a bad
* resolve) is swallowed to null the stale-check must NEVER break startup.
* build/index.js) and look for ../src/tool-specs.ts next to it. In a dev/test
* worktree that file exists; in a prod image (build/ only, src/ stripped) it does
* not, so this returns null and the caller skips the check. Any error (ENOENT, a
* bad resolve) is swallowed to null the stale-check must NEVER break startup.
*
* Exported for unit testing (docmost-client.loader.spec.ts): the export keyword
* is behaviourally a no-op the module-internal caller `loadDocmostMcp` is
* unaffected. The test drives the null (no-src) path and asserts this
* enumerate+normalize+sha256 stays identical to the codegen's
* `computeRegistryStamp`.
* normalize+sha256 stays identical to the codegen's `computeRegistryStamp`.
*/
export function computeSrcRegistryStamp(packageEntry: string): string | null {
try {
// packageEntry is <pkg>/build/index.js; the source lives at <pkg>/src/.
const srcDir = join(dirname(dirname(packageEntry)), 'src');
if (!existsSync(srcDir)) return null; // prod: no src tree -> skip.
// Enumerate every src/**\/*.ts except the codegen's own *.generated.ts
// output (including it would be a fixed-point cycle). Sort by POSIX-relative
// path so ordering is platform-independent, then fold each file's relative
// path + normalized content into one hash — identical to the codegen.
const files = collectStampFiles(srcDir)
.map((abs) => ({
rel: relative(srcDir, abs).split(sep).join('/'),
abs,
}))
.sort((a, b) => (a.rel < b.rel ? -1 : a.rel > b.rel ? 1 : 0));
const hash = createHash('sha256');
for (const { rel, abs } of files) {
const normalized = readFileSync(abs, 'utf8')
.replace(/\r\n/g, '\n')
.replace(/\n$/, '');
hash.update(rel, 'utf8');
hash.update('\0', 'utf8');
hash.update(normalized, 'utf8');
hash.update('\0', 'utf8');
}
return hash.digest('hex');
const toolSpecsPath = join(
dirname(dirname(packageEntry)),
'src',
'tool-specs.ts',
);
if (!existsSync(toolSpecsPath)) return null; // prod: no src tree -> skip.
const source = readFileSync(toolSpecsPath, 'utf8');
const normalized = source.replace(/\r\n/g, '\n').replace(/\n$/, '');
return createHash('sha256').update(normalized, 'utf8').digest('hex');
} catch {
// Never let a resolution/read hiccup break server startup — treat as "no
// src available" and skip the check (identical to the prod no-op path).
@@ -244,24 +225,6 @@ export function computeSrcRegistryStamp(packageEntry: string): string | null {
}
}
/**
* Recursively enumerate every `*.ts` under `dir`, EXCLUDING `*.generated.ts`.
* Mirror of the codegen's `collectStampFiles` (packages/mcp/scripts/
* gen-registry-stamp.mjs) keep the two walk/filter rules identical.
*/
function collectStampFiles(dir: string): string[] {
const out: string[] = [];
for (const entry of readdirSync(dir)) {
const full = join(dir, entry);
if (statSync(full).isDirectory()) {
out.push(...collectStampFiles(full));
} else if (entry.endsWith('.ts') && !entry.endsWith('.generated.ts')) {
out.push(full);
}
}
return out;
}
// TS with module:commonjs downlevels a literal `import()` to `require()`, which
// cannot load the ESM-only `@docmost/mcp` package. Indirect through Function so
// the real dynamic `import()` survives compilation and can load ESM from
@@ -224,7 +224,7 @@ describe('PublicShareChatToolsService.forShare', () => {
(tools.getSharePage as unknown as ToolExec).execute({
pageId: 'page-1',
}),
).rejects.toThrow('The requested page is not available in this share.');
).rejects.toThrow('That page is not part of this published share.');
// No content is ever fetched/returned for a non-resolving page.
expect(shareService.updatePublicAttachments).not.toHaveBeenCalled();
@@ -7,22 +7,6 @@ import { PageRepo } from '@docmost/db/repos/page/page.repo';
import { jsonToMarkdown } from '../../../collaboration/collaboration.util';
import { modelFriendlyInput } from './model-friendly-input';
/**
* A tool error whose message is DELIBERATELY safe to expose to an anonymous
* share reader (and to the model, for self-correction). Every OTHER thrown error
* is treated as internal and replaced with a generic string by `wrapToolErrors`,
* so a raw exception message an internal page title, a DB/stack fragment, a
* driver detail never rides the public UI stream (#394).
*/
export class ShareToolError extends Error {}
// The only two classified strings an anonymous reader may ever see from a tool
// failure. The specific one keeps the model's self-correction useful ("try a
// different page"); the generic one reveals nothing about the internal fault.
const SHARE_TOOL_ERROR_NOT_AVAILABLE =
'The requested page is not available in this share.';
const SHARE_TOOL_ERROR_GENERIC = 'The tool could not complete the request.';
/**
* Isolated, READ-ONLY toolset for the ANONYMOUS public-share assistant.
*
@@ -60,7 +44,7 @@ export class PublicShareChatToolsService {
* are NO write tools, NO comments/history, NO cross-space or external tools.
*/
forShare(shareId: string, workspaceId: string): Record<string, Tool> {
return this.wrapToolErrors({
return {
searchSharePages: tool({
description:
'Search the pages of THIS published documentation share for a ' +
@@ -112,7 +96,7 @@ export class PublicShareChatToolsService {
execute: async ({ pageId }) => {
const id = (pageId ?? '').trim();
if (!id) {
throw new ShareToolError('A pageId is required.');
throw new Error('A pageId is required.');
}
// Resolve via the SINGLE canonical share-access boundary: confirms the
// page resolves to THIS share (recursive CTE up the tree, honouring
@@ -128,7 +112,7 @@ export class PublicShareChatToolsService {
workspaceId,
);
if (!resolved) {
throw new ShareToolError(SHARE_TOOL_ERROR_NOT_AVAILABLE);
throw new Error('That page is not part of this published share.');
}
const { page } = resolved;
@@ -209,57 +193,6 @@ export class PublicShareChatToolsService {
}
},
}),
});
}
/**
* Wrap every tool's `execute` so a THROWN error is sanitized in ONE place
* closing the byte leak, the render, and the model context at once (#394).
*
* The AI SDK surfaces a tool-execution throw as an atomic `tool-output-error`
* frame on the v6 UI stream whose `errorText` is the thrown message; on the
* public share that frame goes straight to an anonymous reader. Unwrapped, a
* raw exception (an internal page title, a DB/stack fragment, a driver detail)
* would ride that frame verbatim. Here we catch it, LOG the full detail
* server-side only, and re-throw a CLASSIFIED, safe error: the tool's own
* intentional ShareToolError messages pass through (they keep the model's
* self-correction useful), everything else collapses to a generic string.
*/
private wrapToolErrors(
tools: Record<string, Tool>,
): Record<string, Tool> {
const wrapped: Record<string, Tool> = {};
for (const [name, t] of Object.entries(tools)) {
const original = t.execute;
if (typeof original !== 'function') {
wrapped[name] = t;
continue;
}
wrapped[name] = {
...t,
execute: async (args: unknown, options: unknown) => {
try {
return await (
original as (a: unknown, o: unknown) => Promise<unknown>
)(args, options);
} catch (err) {
const safe =
err instanceof ShareToolError
? err.message
: SHARE_TOOL_ERROR_GENERIC;
// Full detail to the server log ONLY — never to the anon.
this.logger.warn(
`Public share tool "${name}" failed: ${
err instanceof Error ? err.message : String(err)
}`,
);
// This safe string is ALL that rides the tool-output-error frame,
// becomes model context, and could be rendered — one choke point.
throw new ShareToolError(safe);
}
},
} as Tool;
}
return wrapped;
};
}
}
@@ -120,102 +120,3 @@ describe('JwtStrategy — provenance derivation', () => {
expect(req.raw.actor).toBeUndefined();
});
});
/**
* Provenance derivation on the API-KEY path (jwt.strategy.validateApiKey, #486).
*
* The access-token path stamped provenance; the API-key path returned early
* WITHOUT it, so an is_agent API key's REST writes recorded no 'agent' marker.
* The API-key payload carries no signed claim, so provenance is resolved from the
* SERVER-SIDE user returned by ApiKeyService.validateApiKey: isAgent -> 'agent',
* otherwise 'user'; aiChatId is always null (an API key has no ai_chats row).
*
* The enterprise ApiKeyService is not bundled in the OSS build, so the strategy
* loads it through an overridable `resolveApiKeyService` seam that we stub here.
*/
describe('JwtStrategy — API-key provenance derivation (#486)', () => {
function makeApiKeyStrategy(validateApiKeyImpl: (p: any) => Promise<any>) {
const userRepo: any = { findById: jest.fn() };
const workspaceRepo: any = { findById: jest.fn() };
const userSessionRepo: any = { findActiveById: jest.fn() };
const sessionActivityService: any = { trackActivity: jest.fn() };
const environmentService: any = { getAppSecret: () => 'test-secret' };
const moduleRef: any = {};
const strategy = new JwtStrategy(
userRepo,
workspaceRepo,
userSessionRepo,
sessionActivityService,
environmentService,
moduleRef,
);
// Stub the EE ApiKeyService seam (the real module is not in the OSS build).
const validateApiKey = jest.fn(validateApiKeyImpl);
jest
.spyOn(strategy as any, 'resolveApiKeyService')
.mockReturnValue({ validateApiKey });
return { strategy, validateApiKey };
}
const makeReq = () => ({ raw: {} as Record<string, any> });
const apiKeyPayload = () => ({
sub: 'svc-1',
workspaceId: 'ws-1',
apiKeyId: 'key-1',
type: JwtType.API_KEY,
});
it("stamps actor='agent' for an is_agent API key (from the validated user)", async () => {
const validated = {
user: { id: 'svc-1', isAgent: true },
workspace: { id: 'ws-1' },
};
const { strategy, validateApiKey } = makeApiKeyStrategy(
async () => validated,
);
const req = makeReq();
const result = await strategy.validate(req, apiKeyPayload() as any);
expect(validateApiKey).toHaveBeenCalledTimes(1);
expect(req.raw.actor).toBe('agent');
// API keys carry no internal ai_chats row -> null.
expect(req.raw.aiChatId).toBeNull();
// The validated auth object is returned unchanged (req.user shape preserved).
expect(result).toBe(validated);
});
it("stamps actor='user' for an ordinary (non-agent) API key", async () => {
const { strategy } = makeApiKeyStrategy(async () => ({
user: { id: 'u-1', isAgent: false },
workspace: { id: 'ws-1' },
}));
const req = makeReq();
await strategy.validate(req, apiKeyPayload() as any);
expect(req.raw.actor).toBe('user');
expect(req.raw.aiChatId).toBeNull();
});
it('throws Unauthorized (and stamps nothing) when the EE module is missing', async () => {
const userRepo: any = { findById: jest.fn() };
const strategy = new JwtStrategy(
userRepo,
{ findById: jest.fn() } as any,
{ findActiveById: jest.fn() } as any,
{ trackActivity: jest.fn() } as any,
{ getAppSecret: () => 'test-secret' } as any,
{} as any,
);
// EE not bundled: the seam returns null.
jest.spyOn(strategy as any, 'resolveApiKeyService').mockReturnValue(null);
const req = makeReq();
await expect(
strategy.validate(req, apiKeyPayload() as any),
).rejects.toThrow(UnauthorizedException);
expect(req.raw.actor).toBeUndefined();
});
});
@@ -102,49 +102,28 @@ export class JwtStrategy extends PassportStrategy(Strategy, 'jwt') {
}
private async validateApiKey(req: any, payload: JwtApiKeyPayload) {
const apiKeyService = this.resolveApiKeyService();
if (!apiKeyService) {
throw new UnauthorizedException('Enterprise API Key module missing');
}
let ApiKeyModule: any;
let isApiKeyModuleReady = false;
const result = await apiKeyService.validateApiKey(payload);
// Stamp the agent-edit provenance for the API-KEY path too (#486). Unlike the
// access-token path above, it CANNOT be resolved before this point: the
// API-key payload carries no signed actor/aiChatId claim, and the user (with
// its isAgent flag) is unknown until the key is validated. Claim semantics for
// API keys: an is_agent API key (an agent service account) stamps 'agent' on
// every REST write; an ordinary API key resolves to 'user'. An API key has no
// internal ai_chats row, so aiChatId is always null. Derived from the
// SERVER-SIDE user (never a client field), so an 'agent' badge is unspoofable
// — mirroring the access-token path. Passing `null` for the claim means the
// actor is decided solely by user.isAgent.
const provenance = resolveProvenance((result as any)?.user, null);
req.raw.actor = provenance.actor;
req.raw.aiChatId = provenance.aiChatId;
return result;
}
/**
* Resolve the enterprise ApiKeyService, or `null` when the EE module is not
* bundled in this build (community build). Extracted as an overridable seam so
* the API-key provenance stamping can be unit-tested without the EE package
* present (docmost is OSS + a separate EE bundle; `require` of the EE path
* throws here). Any load/resolve error is treated as "module missing".
*/
protected resolveApiKeyService(): {
validateApiKey: (payload: JwtApiKeyPayload) => Promise<unknown>;
} | null {
try {
// eslint-disable-next-line @typescript-eslint/no-require-imports
const ApiKeyModule = require('./../../../ee/api-key/api-key.service');
return this.moduleRef.get(ApiKeyModule.ApiKeyService, { strict: false });
ApiKeyModule = require('./../../../ee/api-key/api-key.service');
isApiKeyModuleReady = true;
} catch (err) {
this.logger.debug(
'API Key module requested but enterprise module not bundled in this build',
);
return null;
isApiKeyModuleReady = false;
}
if (isApiKeyModuleReady) {
const ApiKeyService = this.moduleRef.get(ApiKeyModule.ApiKeyService, {
strict: false,
});
return ApiKeyService.validateApiKey(payload);
}
throw new UnauthorizedException('Enterprise API Key module missing');
}
}
@@ -0,0 +1,27 @@
import { type Kysely } from 'kysely';
/**
* #370 page-versioning intentionality tier on a history snapshot.
*
* Adds `page_history.kind`, the three-tier "how intentional was this snapshot"
* marker that lets versions (intentional points) be told apart from autosaves:
* - 'manual' a human explicitly saved a version (Cmd+S / Save button)
* - 'agent' the AI agent explicitly saved a version
* - 'idle' trailing idle-flush autosnapshot (safety net)
* - 'boundary' autosnapshot pinned on a source transition (useragentgit)
*
* Nullable with NO default (mirrors last_updated_source in the agent-provenance
* migration): legacy rows predate the marker and read back as `null`, which the
* client renders as a plain autosave. Stored as a short varchar to stay
* forward-compatible without an enum migration.
*/
export async function up(db: Kysely<any>): Promise<void> {
await db.schema
.alterTable('page_history')
.addColumn('kind', 'varchar(20)', (col) => col)
.execute();
}
export async function down(db: Kysely<any>): Promise<void> {
await db.schema.alterTable('page_history').dropColumn('kind').execute();
}
@@ -13,6 +13,7 @@ import { jsonArrayFrom, jsonObjectFrom } from 'kysely/helpers/postgres';
import { ExpressionBuilder, sql } from 'kysely';
import { DB } from '@docmost/db/types/db';
import { resolveAgentProvenance } from '../agent-provenance';
import { PageHistoryKind } from '../../../collaboration/constants';
/**
* Role-resolution subquery for a page-history row's bound AI chat (#300). Joins
@@ -46,6 +47,9 @@ export class PageHistoryRepo {
'lastUpdatedById',
'lastUpdatedSource',
'lastUpdatedAiChatId',
// #370 — intentionality tier ('manual' | 'agent' | 'idle' | 'boundary');
// null on legacy rows (= autosave). Selected so callers can read/promote it.
'kind',
'contributorIds',
'spaceId',
'workspaceId',
@@ -85,9 +89,15 @@ export class PageHistoryRepo {
async saveHistory(
page: Page,
opts?: { contributorIds?: string[]; trx?: KyselyTransaction },
): Promise<void> {
await this.insertPageHistory(
opts?: {
contributorIds?: string[];
// #370 — intentionality tier for this snapshot. Omitted → null (legacy
// autosave semantics). Callers derive it server-side, never from a client.
kind?: PageHistoryKind;
trx?: KyselyTransaction;
},
): Promise<PageHistory> {
return await this.insertPageHistory(
{
pageId: page.id,
slugId: page.slugId,
@@ -99,6 +109,7 @@ export class PageHistoryRepo {
// Copy the provenance marker off the page row, as for lastUpdatedById.
lastUpdatedSource: page.lastUpdatedSource,
lastUpdatedAiChatId: page.lastUpdatedAiChatId,
kind: opts?.kind ?? null,
contributorIds: opts?.contributorIds,
spaceId: page.spaceId,
workspaceId: page.workspaceId,
@@ -107,6 +118,25 @@ export class PageHistoryRepo {
);
}
/**
* #370 promote an existing snapshot's intentionality tier in place. Used by
* the manual-save "promote-not-dup" path: when the latest history row already
* holds the exact content being versioned, we upgrade its `kind` instead of
* duplicating a heavy content row.
*/
async updateHistoryKind(
pageHistoryId: string,
kind: PageHistoryKind,
trx?: KyselyTransaction,
): Promise<void> {
const db = dbOrTx(this.db, trx);
await db
.updateTable('pageHistory')
.set({ kind })
.where('id', '=', pageHistoryId)
.execute();
}
async findPageHistoryByPageId(pageId: string, pagination: PaginationOptions) {
const query = this.db
.selectFrom('pageHistory')
+1
View File
@@ -280,6 +280,7 @@ export interface PageHistory {
createdAt: Generated<Timestamp>;
icon: string | null;
id: Generated<string>;
kind: string | null;
lastUpdatedAiChatId: string | null;
lastUpdatedById: string | null;
lastUpdatedSource: string | null;
@@ -1,133 +0,0 @@
import { readFileSync } from 'fs';
import { EventEmitter } from 'node:events';
import { streamText } from 'ai';
import { MockLanguageModelV3, simulateReadableStream } from 'ai/test';
/**
* Regression tests for the writeToServerResponse drain-hang fix in
* patches/ai@6.0.134.patch (#486, commit 6).
*
* Unpatched ai@6.0.134's writeToServerResponse awaits ONLY `once("drain")` when
* response.write() returns false (backpressure). If the client disconnects
* mid-write the socket never drains, so that await never resolves: the read loop
* parks FOREVER, its `finally { response.end() }` is unreachable, and the stream
* reader + buffered chunks are pinned until process restart. In autonomous mode
* the run keeps producing output after the disconnect, so EVERY mid-run
* disconnect leaks a hung pipe. The patch races drain against close/error, and on
* a terminal socket event cancels the reader and breaks so `finally` always runs.
*
* This drives the REAL patched writeToServerResponse through the public
* pipeUIMessageStreamToResponse API with a response that never drains and closes
* mid-write exactly the leak scenario.
*/
/** A ServerResponse-like emitter whose first write() stalls (returns false) and
* then "closes" like a disconnecting client never firing 'drain'. */
class DisconnectingResponse extends EventEmitter {
ended = false;
writeCount = 0;
statusCode = 200;
writableEnded = false;
destroyed = false;
writeHead(): this {
return this;
}
setHeader(): void {}
flushHeaders(): void {}
write(): boolean {
this.writeCount++;
if (this.writeCount === 1) {
// Simulate the client vanishing mid-write: backpressure (false) and then a
// 'close' on the next tick, and CRUCIALLY never a 'drain'. Unpatched, the
// loop would await drain forever here.
setImmediate(() => this.emit('close'));
return false;
}
return true;
}
end(): void {
this.ended = true;
this.writableEnded = true;
this.emit('finish');
}
}
function makeModel() {
return new MockLanguageModelV3({
doStream: async () => ({
stream: simulateReadableStream({
chunks: [
{ type: 'stream-start' as const, warnings: [] },
{ type: 'text-start' as const, id: '1' },
{ type: 'text-delta' as const, id: '1', delta: 'hello ' },
{ type: 'text-delta' as const, id: '1', delta: 'world' },
{ type: 'text-end' as const, id: '1' },
{
type: 'finish' as const,
finishReason: { unified: 'stop' as const, raw: 'stop' },
usage: {
inputTokens: { total: 1, noCache: undefined, cacheRead: undefined, cacheWrite: undefined },
outputTokens: { total: 1, text: 1, reasoning: undefined },
},
},
],
}),
}),
});
}
describe('ai@6.0.134 pnpm patch: writeToServerResponse drain-hang (#486)', () => {
it('ends the response (does NOT hang) when the socket closes mid-write without draining', async () => {
const result = streamText({ model: makeModel(), prompt: 'hi' });
const res = new DisconnectingResponse();
// Drain the SDK stream independently, like the production detached path.
void result.consumeStream({ onError: () => undefined });
result.pipeUIMessageStreamToResponse(res as never);
// TRIPWIRE: the patched loop exits on 'close' and runs finally -> end().
// Unpatched, it awaits 'drain' forever and this never becomes true.
await new Promise<void>((resolve, reject) => {
const started = Date.now();
const poll = setInterval(() => {
if (res.ended) {
clearInterval(poll);
resolve();
} else if (Date.now() - started > 3000) {
clearInterval(poll);
reject(new Error('writeToServerResponse hung: response never ended'));
}
}, 20);
});
expect(res.ended).toBe(true);
});
it('does not emit an unhandledRejection when the fire-and-forget read() throws', async () => {
// The patch swallows read()'s rejection (fire-and-forget) with a log instead
// of letting it surface as a process-killing unhandledRejection.
const rejections: unknown[] = [];
const onUnhandled = (e: unknown) => rejections.push(e);
process.on('unhandledRejection', onUnhandled);
// Silence the patch's diagnostic console.error for the throwing read().
const errSpy = jest.spyOn(console, 'error').mockImplementation(() => undefined);
try {
const result = streamText({ model: makeModel(), prompt: 'hi' });
const res = new DisconnectingResponse();
void result.consumeStream({ onError: () => undefined });
result.pipeUIMessageStreamToResponse(res as never);
await new Promise((r) => setTimeout(r, 300));
} finally {
process.off('unhandledRejection', onUnhandled);
errSpy.mockRestore();
}
expect(rejections).toEqual([]);
});
it('both installed dist builds (CJS and ESM) carry the #486 patch marker', () => {
const cjsPath = require.resolve('ai');
const mjsPath = cjsPath.replace(/index\.js$/, 'index.mjs');
expect(cjsPath).toMatch(/index\.js$/);
expect(readFileSync(cjsPath, 'utf8')).toContain('PATCH(docmost #486)');
expect(readFileSync(mjsPath, 'utf8')).toContain('PATCH(docmost #486)');
});
});
@@ -1,211 +0,0 @@
import type { HealthIndicatorService } from '@nestjs/terminus';
import type { EnvironmentService } from '../environment/environment.service';
/**
* Integration guard for the /health Redis-probe handle leak (#486, commit 2).
*
* The bug: `pingCheck` built `new Redis(...)` per call and only disconnected on
* the SUCCESS path, so when Redis is DOWN every probe tick added ANOTHER
* forever-reconnecting client an unbounded handle/client leak for the duration
* of the outage. The fix reuses ONE long-lived probe client.
*
* This is an OBSERVABLE-property test, not an assertion on a mocked return value:
* we point the indicator at a REAL, refused TCP endpoint (a dead port) so ioredis
* genuinely fails to connect, run many probes, and assert the number of live
* Redis CLIENTS created stays at exactly ONE. `ioredis` is delegated to its real
* implementation (requireActual) only the constructor is wrapped to COUNT the
* real clients it creates, which is precisely the leaking resource.
*/
import type { Redis } from 'ioredis';
const mockLiveClients: Redis[] = [];
/**
* Fully tear a REAL ioredis client down so NO timer survives jest's 1s exit
* window (this suite must exit cleanly WITHOUT forceExit; see #382).
*
* `connector.disconnect()` arms a ~12s "force-destroy the stream" `setTimeout`
* that is cleared ONLY by the stream's 'close' event but only when the
* connector still holds a stream. Two problem cases:
* - a LIVE/connecting socket: disconnect arms the timer and 'close' may lag
* past jest's window, so we destroy the socket to make 'close' fire NOW;
* - a client BETWEEN reconnect attempts to a dead port: the held socket is
* ALREADY destroyed (its 'close' fired long ago), so disconnect would arm a
* timer whose clearing 'close' can never come again. We drop that dead stream
* reference BEFORE disconnect so the doomed timer is never armed.
* `disconnect()` itself also clears ioredis' own reconnect backoff timer.
*/
type DrainableStream = { destroyed?: boolean; destroy?: () => void } | null;
type DrainableClient = {
removeAllListeners: (event: string) => void;
disconnect: () => void;
stream?: DrainableStream;
connector?: { stream?: DrainableStream };
};
async function drainClient(client: Redis): Promise<void> {
if (!client || client.status === 'end') return;
const c = client as unknown as DrainableClient;
c.removeAllListeners('error');
// Drop an already-dead held socket so disconnect() can't arm a timer whose
// clearing 'close' will never fire again.
if (c.connector?.stream && c.connector.stream.destroyed) {
c.connector.stream = null;
}
if (c.stream && c.stream.destroyed) {
c.stream = null;
}
await new Promise<void>((resolve) => {
let done = false;
const finish = () => {
if (done) return;
done = true;
resolve();
};
client.once('end', finish);
// reconnect=false (the default): stop the retry loop and close the socket.
client.disconnect();
// Force any still-live socket closed NOW so the connector's stream-destroy
// timer clears inside jest's window instead of lagging behind a real 'close'.
if (c.stream && !c.stream.destroyed) {
c.stream.destroy?.();
}
// Fallback for a client with no live stream to emit 'end' (unref'd so it
// can never itself hold the loop open).
const fallback = setTimeout(finish, 500);
(fallback as { unref?: () => void }).unref?.();
});
}
async function drainAll(): Promise<void> {
await Promise.all(mockLiveClients.map((c) => drainClient(c)));
}
jest.mock('ioredis', () => {
const actual = jest.requireActual('ioredis');
const RealRedis = actual.Redis ?? actual.default ?? actual;
class CountingRedis extends RealRedis {
constructor(...args: unknown[]) {
super(...(args as []));
mockLiveClients.push(this as never);
}
}
return { ...actual, Redis: CountingRedis, default: CountingRedis };
});
// Import AFTER the mock is registered so the class picks up the counting client.
import { RedisHealthIndicator } from './redis.health';
describe('RedisHealthIndicator handle leak (#486)', () => {
const indicatorService = {
check: (key: string) => ({
up: () => ({ [key]: { status: 'up' } }),
down: (message: string) => ({ [key]: { status: 'down', message } }),
}),
} as unknown as HealthIndicatorService;
// A port with (almost certainly) nothing listening -> connection refused fast.
const environmentService = {
getRedisUrl: () => 'redis://127.0.0.1:6399/0',
} as unknown as EnvironmentService;
let indicator: RedisHealthIndicator;
beforeEach(() => {
mockLiveClients.length = 0;
indicator = new RedisHealthIndicator(indicatorService, environmentService);
});
afterEach(async () => {
// Drain (destroy socket + AWAIT 'end') every client the test created FIRST,
// so each is fully 'end' before onModuleDestroy's disconnect runs — that way
// no ioredis reconnect / stream-destroy timer outlives jest's exit window.
await drainAll();
indicator.onModuleDestroy();
});
it('creates exactly ONE Redis client across many probes while Redis is DOWN', async () => {
const N = 8;
for (let i = 0; i < N; i++) {
const result = await indicator.pingCheck('redis');
// Down endpoint -> every probe reports "down" (not an unhandled crash).
expect(result.redis.status).toBe('down');
}
// THE OBSERVABLE LEAK: on the buggy code this is N (a fresh, never-cleaned
// reconnecting client per probe). The fix reuses one shared client.
expect(mockLiveClients).toHaveLength(1);
});
it('onModuleDestroy releases the probe client (a later probe builds a fresh one)', async () => {
await indicator.pingCheck('redis');
expect(mockLiveClients).toHaveLength(1);
indicator.onModuleDestroy();
// A second destroy is a safe no-op (probeClient was nulled).
indicator.onModuleDestroy();
// After shutdown the indicator lazily builds a NEW client on the next probe,
// proving the old one was truly released rather than reused.
await indicator.pingCheck('redis');
expect(mockLiveClients).toHaveLength(2);
});
});
/**
* Happy-path regression guard (#486, B2): the FIRST probe against a LIVE Redis
* must report UP.
*
* With `lazyConnect: true` + `enableOfflineQueue: false`, a freshly-built client
* is in the `wait` state and the socket opens lazily. If the very first `ping()`
* is issued before an explicit `connect()`, ioredis rejects it instantly with
* "Stream isn't writeable and enableOfflineQueue options is false" a FALSE
* DOWN even though Redis is alive. The fix opens the socket before the first
* ping. This exercises a REAL ioredis client against a REAL TCP redis server
* (not a mock), so a regression genuinely reddens it.
*/
describe('RedisHealthIndicator live Redis first-probe (#486, B2)', () => {
const indicatorService = {
check: (key: string) => ({
up: () => ({ [key]: { status: 'up' } }),
down: (message: string) => ({ [key]: { status: 'down', message } }),
}),
} as unknown as HealthIndicatorService;
// A REAL running redis (see the neighboring harness / CI env).
const environmentService = {
getRedisUrl: () => 'redis://127.0.0.1:6379/0',
} as unknown as EnvironmentService;
let indicator: RedisHealthIndicator;
beforeEach(() => {
mockLiveClients.length = 0;
indicator = new RedisHealthIndicator(indicatorService, environmentService);
});
afterEach(async () => {
// Await full socket close of every live client (see drainClient) BEFORE
// onModuleDestroy: a real, connected ioredis client MUST be drained to 'end'
// or its stream-destroy timer keeps the jest worker alive past the 1s window.
await drainAll();
indicator.onModuleDestroy();
});
it('reports UP on the FIRST probe against a live Redis', async () => {
// The VERY FIRST probe — no warm-up ping — must be UP.
const result = await indicator.pingCheck('redis');
expect(result.redis.status).toBe('up');
});
it('stays UP on a probe AFTER onModuleDestroy re-creates the client', async () => {
await indicator.pingCheck('redis');
indicator.onModuleDestroy();
// The re-created client is again in `wait`; the first ping on it must still
// open the socket (the false-DOWN also recurs on the post-destroy path).
const result = await indicator.pingCheck('redis');
expect(result.redis.status).toBe('up');
});
});
@@ -2,173 +2,33 @@ import {
HealthIndicatorResult,
HealthIndicatorService,
} from '@nestjs/terminus';
import { Injectable, Logger, OnModuleDestroy } from '@nestjs/common';
import { Injectable, Logger } from '@nestjs/common';
import { EnvironmentService } from '../environment/environment.service';
import { Redis } from 'ioredis';
@Injectable()
export class RedisHealthIndicator implements OnModuleDestroy {
export class RedisHealthIndicator {
private readonly logger = new Logger(RedisHealthIndicator.name);
/**
* ONE long-lived probe connection, reused across every /health tick. The old
* code built `new Redis(...)` per call and only `disconnect()`d on the SUCCESS
* path, so while Redis was DOWN every probe added a fresh, forever-reconnecting
* client a handle leak that grew without bound for as long as the outage (and
* the health checker keeps polling) lasted. A single shared client keeps at most
* ONE background reconnect loop regardless of how many probes run.
*/
private probeClient: Redis | null = null;
/**
* How long the first-ping `connect()` may take before a probe gives up and
* reports DOWN. A `connect()` against a truly-down Redis never settles on its
* own (ioredis retries the socket indefinitely per its retryStrategy), so the
* probe MUST bound it or the /health handler would hang. Kept short so a real
* outage is reported fast; localhost/live Redis connects well within it.
*/
private static readonly CONNECT_TIMEOUT_MS = 2000;
/**
* The single in-flight first-`connect()`, memoized so CONCURRENT probes share
* it. k8s liveness+readiness hit /health in parallel on startup: without this,
* probe A drives `connect()` (the client leaves the `wait` state) and probe B,
* seeing a not-`wait`/not-`ready` client, would skip connect and fire `ping()`
* at a still-opening socket an instant FALSE DOWN. With the memo, B awaits
* the SAME connect. Cleared once it settles so a later disconnect / re-create
* starts a fresh connect.
*/
private connectingPromise: Promise<void> | null = null;
constructor(
private readonly healthIndicatorService: HealthIndicatorService,
private environmentService: EnvironmentService,
) {}
private getProbeClient(): Redis {
if (!this.probeClient) {
this.probeClient = new Redis(this.environmentService.getRedisUrl(), {
// Constructing must never throw or eagerly connect; the first ping opens
// the socket. This lets us build the client once and reuse it.
lazyConnect: true,
// A health probe must fail FAST, not queue behind a stuck reconnect: one
// retry per request, and no offline queue so a ping while disconnected
// rejects immediately instead of buffering commands that pile up in RAM.
maxRetriesPerRequest: 1,
enableOfflineQueue: false,
});
// ioredis emits 'error' on every failed (re)connect; with no listener that
// surfaces as an unhandled 'error' event and can crash the process. Swallow
// it here — pingCheck already reports health — and log at debug so a Redis
// outage does not flood the logs.
this.probeClient.on('error', (err) => {
this.logger.debug(
`Redis probe connection error: ${
err instanceof Error ? err.message : String(err)
}`,
);
});
}
return this.probeClient;
}
/**
* Open the probe socket BEFORE the first ping. `lazyConnect: true` leaves a
* freshly-built (or post-destroy re-built) client in the `wait` state: the
* socket is NOT open yet, so with `enableOfflineQueue: false` the very first
* `ping()` rejects instantly with "Stream isn't writeable and
* enableOfflineQueue options is false" even when Redis is perfectly alive a
* false DOWN on the happy path. We drive `connect()` ONLY from `wait`; once
* the client is connected, ioredis owns its own (re)connect loop and a ping
* issued while it reconnects still fast-fails to a correct DOWN (offline queue
* stays off). A failed/timed-out connect rejects reported DOWN, which is the
* right signal for a truly-down Redis.
*/
private ensureConnected(client: Redis): Promise<void> {
// Already open — steady state, nothing to do.
if (client.status === 'ready') return Promise.resolve();
// A first-connect is already in flight (possibly started by a CONCURRENT
// probe): await the SAME one instead of racing a second connect() (ioredis
// throws "already connecting") or firing ping() at a not-yet-open socket.
if (this.connectingPromise) return this.connectingPromise;
// Only DRIVE connect() from the initial `wait` state (fresh / post-destroy
// re-created client). In any other non-ready state ioredis already owns its
// (re)connect loop; a ping there fast-fails to a correct DOWN, so we must not
// start a competing connect.
if (client.status !== 'wait') return Promise.resolve();
const promise = this.connectWithTimeout(client).finally(() => {
// Clear only if still ours, so a later disconnect / re-create can connect
// again. Whether it resolved or rejected, the memo has served its window.
if (this.connectingPromise === promise) {
this.connectingPromise = null;
}
});
this.connectingPromise = promise;
return promise;
}
private connectWithTimeout(client: Redis): Promise<void> {
return new Promise<void>((resolve, reject) => {
let settled = false;
const timer = setTimeout(() => {
if (settled) return;
settled = true;
reject(new Error('Redis probe connect timed out'));
}, RedisHealthIndicator.CONNECT_TIMEOUT_MS);
// Never let THIS timer alone keep the event loop (or a jest worker) alive;
// it is cleared on settle anyway, this is belt-and-braces.
timer.unref?.();
// `.catch` is always attached, so a connect() that rejects AFTER we have
// already timed out is handled here (guarded by `settled`) and never
// surfaces as an unhandled rejection.
client
.connect()
.then(() => {
if (settled) return;
settled = true;
clearTimeout(timer);
resolve();
})
.catch((err) => {
if (settled) return;
settled = true;
clearTimeout(timer);
reject(err);
});
});
}
async pingCheck(key: string): Promise<HealthIndicatorResult> {
const indicator = this.healthIndicatorService.check(key);
try {
const redis = this.getProbeClient();
// Open the socket before the first ping (see ensureConnected); without
// this the first probe after (re)creation falsely reports DOWN on a live
// Redis because lazyConnect defers the connect past the first ping.
await this.ensureConnected(redis);
const redis = new Redis(this.environmentService.getRedisUrl(), {
maxRetriesPerRequest: 15,
});
await redis.ping();
redis.disconnect();
return indicator.up();
} catch (e) {
this.logger.error(e);
return indicator.down(`${key} is not available`);
}
}
onModuleDestroy(): void {
if (this.probeClient) {
// disconnect() (not quit()) tears the socket + reconnect loop down
// immediately without waiting on a round-trip to a possibly-down server.
// Do NOT removeAllListeners() with no event name — that would also strip
// ioredis' OWN internal listeners and break its teardown; our 'error'
// listener is harmless and dies with the dropped client reference.
this.probeClient.disconnect();
this.probeClient = null;
}
// Drop any in-flight first-connect memo so the NEXT client (lazily rebuilt on
// the next probe) starts a fresh connect rather than awaiting a promise tied
// to the client we just tore down.
this.connectingPromise = null;
}
}
@@ -16,7 +16,6 @@ import {
} from './mcp-auth.helpers';
import { JwtType } from '../../core/auth/dto/jwt-payload';
import { CREDENTIALS_MISMATCH_MESSAGE } from '../../core/auth/auth.constants';
import { McpService } from './mcp.service';
// The /mcp per-user auth decision logic is tested through the framework-free
// `resolveMcpSessionConfig` helper that McpService delegates to. McpService
@@ -1180,46 +1179,3 @@ describe('mapAuthResultToResponse (handle status/body mapping, refactor R2)', ()
});
});
});
// #486: onModuleDestroy must ALSO tear down the live loopback CollabSessions, not
// just clear the sweep timer — otherwise the embedded MCP's collab sockets keep
// docs pinned open on the collab server past process exit. The teardown goes
// through an overridable seam (destroyAllMcpSessions) so it can be spied without
// loading the ESM-only @docmost/mcp package.
describe('McpService.onModuleDestroy — CollabSession teardown (#486)', () => {
function makeService(): McpService {
// The constructor only stores its deps and starts the (unref'd) sweep timer,
// so bare stubs suffice. onModuleDestroy clears that timer, so no leak.
return new McpService(
{} as any,
{} as any,
{} as any,
{} as any,
{} as any,
{} as any,
{} as any,
);
}
it('destroys all sessions AND clears the sweep timer on shutdown', async () => {
const svc = makeService();
const destroy = jest.fn().mockResolvedValue(undefined);
(svc as any).destroyAllMcpSessions = destroy;
const clearSpy = jest.spyOn(global, 'clearInterval');
await svc.onModuleDestroy();
expect(destroy).toHaveBeenCalledTimes(1);
expect(clearSpy).toHaveBeenCalledWith((svc as any).sweepTimer);
clearSpy.mockRestore();
});
it('swallows a teardown failure so shutdown never throws', async () => {
const svc = makeService();
(svc as any).destroyAllMcpSessions = jest
.fn()
.mockRejectedValue(new Error('collab teardown boom'));
await expect(svc.onModuleDestroy()).resolves.toBeUndefined();
});
});
@@ -119,42 +119,10 @@ export class McpService implements OnModuleDestroy {
this.sweepTimer.unref?.();
}
async onModuleDestroy(): Promise<void> {
onModuleDestroy(): void {
clearInterval(this.sweepTimer);
// Tear down any live loopback CollabSession providers at shutdown (#486). The
// embedded MCP (and the in-app AI agent) open Hocuspocus collab sockets against
// THIS process; without an explicit teardown those sessions keep their docs
// "open" on the collab server and hold providers/buffers until they idle out,
// so a restart can race a doc still pinned by the dying worker. Best-effort:
// any failure is logged, never allowed to break shutdown.
try {
await this.destroyAllMcpSessions();
} catch (err) {
this.logger.error(
'MCP CollabSession teardown on shutdown failed',
err as Error,
);
}
}
/**
* Resolve @docmost/mcp's `destroyAllSessions` and invoke it (#486). The live
* CollabSession registry is a module-level singleton in the ESM package, shared
* by every entry (`.`/`./http`), so this tears down ALL sessions regardless of
* which surface opened them. The module is already loaded whenever MCP was used;
* if it was never loaded (or is absent) the import + no-op is harmless.
*
* Held as an overridable field so a unit test can spy the teardown without
* loading the ESM-only package or standing up the DI graph.
*/
private destroyAllMcpSessions: () => Promise<void> = async () => {
const entry = require.resolve('@docmost/mcp');
const mod = (await esmImport(pathToFileURL(entry).href)) as {
destroyAllSessions?: () => void;
};
mod.destroyAllSessions?.();
};
// Service account the embedded MCP uses to talk back to this Docmost
// instance over loopback REST + the collaboration WebSocket. Now OPTIONAL:
// it is only a fallback when no per-user Basic/Bearer credentials are sent.
@@ -1,148 +0,0 @@
import { get as httpGet } from 'node:http';
import { AddressInfo } from 'node:net';
import { createServer } from 'node:http';
// Drive the metrics HTTP server without the load-time METRICS_PORT gate: mock the
// registry so isMetricsEnabled()/getMetricsRegistry() are always satisfied. What
// we assert is observed over a REAL socket (bind address, status codes), not on
// the mock.
jest.mock('./metrics.registry', () => ({
isMetricsEnabled: () => true,
getMetricsRegistry: () => ({
metrics: async () => '# HELP up test\nup 1\n',
contentType: 'text/plain; version=0.0.4',
}),
}));
import {
startMetricsServer,
closeMetricsServer,
resolveMetricsBind,
resolveMetricsToken,
} from './metrics.server';
/** Find a free TCP port (the metrics server requires METRICS_PORT > 0). */
function freePort(): Promise<number> {
return new Promise((resolve, reject) => {
const s = createServer();
s.once('error', reject);
s.listen(0, '127.0.0.1', () => {
const p = (s.address() as AddressInfo).port;
s.close(() => resolve(p));
});
});
}
/** Minimal GET against 127.0.0.1:port with optional Authorization header. */
function req(
port: number,
headers: Record<string, string> = {},
): Promise<{ status: number; body: string }> {
return new Promise((resolve, reject) => {
const r = httpGet(
{ host: '127.0.0.1', port, path: '/metrics', headers },
(res) => {
let body = '';
res.on('data', (c) => (body += c));
res.on('end', () =>
resolve({ status: res.statusCode ?? 0, body }),
);
},
);
r.on('error', reject);
});
}
describe('metrics server bind + auth (#486)', () => {
const saved = {
bind: process.env.METRICS_BIND,
token: process.env.METRICS_TOKEN,
port: process.env.METRICS_PORT,
};
afterEach(async () => {
await closeMetricsServer();
process.env.METRICS_BIND = saved.bind;
process.env.METRICS_TOKEN = saved.token;
process.env.METRICS_PORT = saved.port;
delete process.env.METRICS_BIND;
delete process.env.METRICS_TOKEN;
});
describe('resolveMetricsBind', () => {
it('defaults to loopback 127.0.0.1', () => {
delete process.env.METRICS_BIND;
expect(resolveMetricsBind()).toBe('127.0.0.1');
});
it('honours the METRICS_BIND override', () => {
process.env.METRICS_BIND = '0.0.0.0';
expect(resolveMetricsBind()).toBe('0.0.0.0');
});
it('treats a blank override as unset (loopback)', () => {
process.env.METRICS_BIND = ' ';
expect(resolveMetricsBind()).toBe('127.0.0.1');
});
});
describe('resolveMetricsToken', () => {
it('is null when unset', () => {
delete process.env.METRICS_TOKEN;
expect(resolveMetricsToken()).toBeNull();
});
it('returns the trimmed token when set', () => {
process.env.METRICS_TOKEN = ' s3cret ';
expect(resolveMetricsToken()).toBe('s3cret');
});
});
it('binds to loopback by default and serves /metrics without auth when no token', async () => {
delete process.env.METRICS_BIND;
delete process.env.METRICS_TOKEN;
const port = await freePort();
process.env.METRICS_PORT = String(port);
const server = startMetricsServer();
expect(server).not.toBeNull();
await new Promise<void>((resolve) => {
if (server!.listening) resolve();
else server!.once('listening', () => resolve());
});
// OBSERVABLE: the listener bound to loopback, not 0.0.0.0.
expect((server!.address() as AddressInfo).address).toBe('127.0.0.1');
const res = await req(port);
expect(res.status).toBe(200);
expect(res.body).toContain('up 1');
});
it('rejects unauthenticated scrapes with 401 and accepts the exact Bearer token', async () => {
delete process.env.METRICS_BIND;
process.env.METRICS_TOKEN = 'topsecret';
const port = await freePort();
process.env.METRICS_PORT = String(port);
const server = startMetricsServer();
expect(server).not.toBeNull();
// No auth -> 401.
const noAuth = await req(port);
expect(noAuth.status).toBe(401);
// Wrong token, DIFFERENT length -> 401 (short-circuits on the length guard).
const wrong = await req(port, { authorization: 'Bearer nope' });
expect(wrong.status).toBe(401);
// Wrong token, SAME length -> 401. This drives the timingSafeEqual compare
// itself (the length guard passes: 'Bearer topsecreX' has the same length as
// 'Bearer topsecret'). Pins the constant-time compare: a regression that made
// it return true would let this equal-length wrong token through — the
// different-length case above would NOT catch that.
const sameLen = await req(port, { authorization: 'Bearer topsecreX' });
expect(sameLen.status).toBe(401);
// Correct token -> 200 with the metrics body.
const ok = await req(port, { authorization: 'Bearer topsecret' });
expect(ok.status).toBe(200);
expect(ok.body).toContain('up 1');
});
});
@@ -1,27 +1,7 @@
import { createServer, Server } from 'node:http';
import { timingSafeEqual } from 'node:crypto';
import { Logger } from '@nestjs/common';
import { getMetricsRegistry, isMetricsEnabled } from './metrics.registry';
/**
* Constant-time compare of the presented Authorization header against the
* expected `Bearer <token>`. This is the ONLY auth layer for the metrics
* endpoint, so a naive `!==` would leak the token byte-by-byte via timing.
* timingSafeEqual requires equal-length buffers, so a length mismatch short-
* circuits to "not equal" (its own length is not itself a useful oracle: the
* expected string length is fixed by config, not secret-derived).
*/
function bearerMatches(
presented: string | undefined,
expected: string,
): boolean {
if (typeof presented !== 'string') return false;
const a = Buffer.from(presented);
const b = Buffer.from(expected);
if (a.length !== b.length) return false;
return timingSafeEqual(a, b);
}
/**
* Start the Prometheus scrape endpoint on a SEPARATE port, taken from
* `METRICS_PORT`. There is NO default port: when `METRICS_PORT` is unset the
@@ -36,30 +16,6 @@ function bearerMatches(
*/
let metricsServer: Server | null = null;
/**
* Interface the metrics endpoint binds to. Defaults to LOOPBACK (127.0.0.1) so
* the unauthenticated `/metrics` surface is NOT exposed on all interfaces by
* default the old `0.0.0.0` bind put an auth-less endpoint on every interface.
* Deployments where the scraper runs in a SEPARATE container (and reaches this as
* `docmost:9464`) set `METRICS_BIND=0.0.0.0`, ideally together with METRICS_TOKEN
* and/or a private network so the port is not world-readable.
*/
export function resolveMetricsBind(): string {
const raw = (process.env.METRICS_BIND ?? '').trim();
return raw.length > 0 ? raw : '127.0.0.1';
}
/**
* Optional Bearer token guarding `/metrics`. When `METRICS_TOKEN` is set, every
* scrape must present `Authorization: Bearer <token>`; unset (default) leaves the
* endpoint open (safe when bound to loopback / a trusted network). Returns the
* trimmed token or null when unset/blank.
*/
export function resolveMetricsToken(): string | null {
const raw = (process.env.METRICS_TOKEN ?? '').trim();
return raw.length > 0 ? raw : null;
}
export function startMetricsServer(): Server | null {
if (!isMetricsEnabled()) return null;
@@ -75,22 +31,8 @@ export function startMetricsServer(): Server | null {
return null;
}
const bind = resolveMetricsBind();
const token = resolveMetricsToken();
const server = createServer(async (req, res) => {
if (req.method === 'GET' && req.url === '/metrics') {
// Optional Bearer auth: reject scrapes without the exact token when one is
// configured. This is the auth layer the old all-interfaces bind lacked.
if (token) {
const auth = req.headers['authorization'];
if (!bearerMatches(auth, `Bearer ${token}`)) {
res.statusCode = 401;
res.setHeader('WWW-Authenticate', 'Bearer');
res.end();
return;
}
}
try {
const body = await register.metrics();
res.setHeader('Content-Type', register.contentType);
@@ -106,14 +48,10 @@ export function startMetricsServer(): Server | null {
res.end();
});
// Bind to loopback by default so the auth-less endpoint is not exposed on all
// interfaces. Set METRICS_BIND=0.0.0.0 (ideally with METRICS_TOKEN) when the
// scraper runs in a separate container and reaches this as docmost:9464.
server.listen(port, bind, () => {
logger.log(
`Metrics endpoint listening on ${bind}:${port}/metrics` +
(token ? ' (Bearer auth required)' : ''),
);
// Bind on all interfaces: the scraper (VictoriaMetrics) reaches this from
// another container as docmost:9464. The port is not published to the host.
server.listen(port, '0.0.0.0', () => {
logger.log(`Metrics endpoint listening on :${port}/metrics`);
});
server.on('error', (err) => {
@@ -31,6 +31,9 @@ export enum QueueJob {
IMPORT_TASK = 'import-task',
EXPORT_TASK = 'export-task',
SEARCH_REMOVE_PAGE = 'search-remove-page',
SEARCH_REMOVE_ASSET = 'search-remove-attachment',
SEARCH_REMOVE_FACE = 'search-remove-comment',
TYPESENSE_FLUSH = 'typesense-flush',
PAGE_CREATED = 'page-created',
@@ -20,6 +20,10 @@ export interface IStripeSeatsSyncJob {
export interface IPageHistoryJob {
pageId: string;
// #370 — intentionality tier the worker stamps on the snapshot. All jobs on
// this queue are trailing idle-flush autosnapshots, so this is 'idle' (absent
// → treated as 'idle' by the processor).
kind?: 'idle';
}
/**
@@ -0,0 +1,162 @@
import { randomUUID } from 'node:crypto';
import { Queue, Worker } from 'bullmq';
import { PersistenceExtension } from '../../src/collaboration/extensions/persistence.extension';
/**
* #370 integration property of the idle-snapshot pipeline against REAL BullMQ.
*
* This is deliberately NOT a unit test of computeHistoryJob (that lives in
* compute-history-job.spec.ts). The point here is the OBSERVABLE end-to-end
* behaviour of the production `enqueuePageHistory` remove-then-add debounce
* driving a real Redis-backed delayed queue + worker (the #431#439 class: a
* locally-correct function whose queue/timer property was never exercised):
*
* - a CONTINUOUS burst of stores lasting several caps yields periodic idle
* snapshots at least one per max-wait cap, NOT one-per-store; and
* - an INTERMITTENT burst (a few stores, then quiet) yields exactly ONE
* trailing snapshot.
*
* We shrink the idle windows to milliseconds (jest.mock of collaboration
* constants) so real BullMQ delayed jobs actually promote within the test
* fake timers cannot advance Redis's own delayed-set clock, so the intervals
* must be real but tiny. The production method under test is called verbatim.
*/
// NOTE: jest.mock is hoisted above the module's const initializers, so its
// factory cannot close over MAX_WAIT_MS/INTERVAL_MS — the literals are inlined
// here and MUST stay in sync with the consts below (a single source of truth is
// impossible across the hoist boundary).
jest.mock('../../src/collaboration/constants', () => {
const actual = jest.requireActual('../../src/collaboration/constants');
return {
...actual,
IDLE_MAX_WAIT_USER: 300,
IDLE_MAX_WAIT_AGENT: 300,
IDLE_INTERVAL_USER: 1000,
IDLE_INTERVAL_AGENT: 1000,
};
});
// Mirrors the mocked IDLE_MAX_WAIT_* above (IDLE_INTERVAL_* is 1000 > this, so
// the max-wait ceiling is what actually governs the trailing delay).
const MAX_WAIT_MS = 300;
const REDIS_CONNECTION = {
host: process.env.TEST_REDIS_HOST ?? '127.0.0.1',
port: Number(process.env.TEST_REDIS_PORT ?? 6379),
};
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
describe('#370 idle-snapshot pipeline (real BullMQ)', () => {
let queue: Queue;
let worker: Worker;
let extension: PersistenceExtension;
// Every processed snapshot, tagged by pageId so the two scenarios stay isolated.
const processed: Array<{ pageId: string; kind: string; at: number }> = [];
const queueName = `history-idle-int-${randomUUID()}`;
beforeAll(async () => {
queue = new Queue(queueName, {
connection: REDIS_CONNECTION,
// Mirror the production default (BullModule.forRoot removeOnComplete): the
// enqueue idiom relies on the jobId being freed once a job completes so the
// next burst can re-arm the same id.
defaultJobOptions: { removeOnComplete: true, removeOnFail: true },
});
await queue.waitUntilReady();
worker = new Worker(
queueName,
async (job) => {
processed.push({
pageId: job.data?.pageId,
kind: job.data?.kind,
at: Date.now(),
});
},
{ connection: REDIS_CONNECTION },
);
await worker.waitUntilReady();
// Construct the real extension; only historyQueue (5th ctor arg) and the
// internal idleBurstStart map are exercised by enqueuePageHistory, so the
// other collaborators can be null — the constructor only assigns fields.
extension = new PersistenceExtension(
null as any, // pageRepo
null as any, // pageHistoryRepo
null as any, // db
null as any, // aiQueue
queue as any, // historyQueue
null as any, // notificationQueue
null as any, // collabHistory
null as any, // transclusionService
);
});
afterAll(async () => {
// Force-close and fully drain so no BullMQ background activity (delayed-set
// polling, blocking BRPOPLPUSH) bleeds into later suites in this single
// shared jest worker (maxWorkers: 1).
await worker?.close(true).catch(() => undefined);
await queue?.obliterate({ force: true }).catch(() => undefined);
await queue?.close();
// Let the redis sockets settle before the next suite starts.
await sleep(150);
});
const arm = (pageId: string) =>
(extension as any).enqueuePageHistory({ id: pageId }, 'user');
it('continuous burst over several caps → periodic idle snapshots (≥1 per cap, not one-per-store)', async () => {
const pageId = randomUUID();
const runMs = 6 * MAX_WAIT_MS; // ~6 caps of unbroken editing
// Store cadence that does NOT evenly divide the cap: real hocuspocus stores
// are not aligned to cap boundaries, so a boundary job promotes in the gap
// before the next store's remove(). A cap-aligned cadence would instead land
// a store exactly on every boundary and lose the snapshot to the documented
// remove-vs-active race — an artefact of the test clock, not the pipeline.
const stepMs = 70;
const stores = Math.floor(runMs / stepMs);
const start = Date.now();
let count = 0;
while (Date.now() - start < runMs) {
await arm(pageId);
count++;
await sleep(stepMs);
}
// Let the final armed job flush.
await sleep(2 * MAX_WAIT_MS);
const snaps = processed.filter((p) => p.pageId === pageId);
// Every autosnapshot is an idle-kind row.
expect(snaps.every((s) => s.kind === 'idle')).toBe(true);
// Periodic: at least one per cap over a multi-cap burst (lower-bounded loosely
// to stay robust; the property is "fires at least every cap", not a single
// trailing snapshot).
expect(snaps.length).toBeGreaterThanOrEqual(3);
// But NOT one-per-store: ~`stores` stores were issued; the debounce must
// collapse them to a small multiple of the cap count, nowhere near per-store.
expect(snaps.length).toBeLessThanOrEqual(Math.ceil(stores / 2));
});
it('intermittent burst (a few stores, then quiet) → exactly ONE trailing snapshot', async () => {
const pageId = randomUUID();
// A short burst well within a single cap window, then silence.
await arm(pageId);
await sleep(40);
await arm(pageId);
await sleep(40);
await arm(pageId);
// Wait comfortably past the cap so the single pending trailing job fires.
await sleep(4 * MAX_WAIT_MS);
const snaps = processed.filter((p) => p.pageId === pageId);
expect(snaps).toHaveLength(1);
expect(snaps[0].kind).toBe('idle');
});
});
+1 -1
View File
@@ -387,7 +387,7 @@ bucketByDay(sessions, tz):
факты ниже — ground truth, можно дозапросить файлы через gitea MCP по указанному SHA):
- `page_history.kind``varchar(20)`, NULLABLE, БЕЗ дефолта (migration
`20260705T120000-page-history-kind.ts`). Домен: `manual`/`agent`/`idle`/`boundary`;
`20260707T120000-page-history-kind.ts`). Домен: `manual`/`agent`/`idle`/`boundary`;
legacy `null` = автосейв (`collaboration/constants.ts`, `PageHistoryKind`).
- `kind` УЖЕ включён в `PageHistoryRepo.baseFields` (`page-history.repo.ts`) — читается всеми
выборками истории. `saveHistory({kind})` и `updateHistoryKind(id, kind)` существуют.
+37 -77
View File
@@ -1,100 +1,60 @@
// Codegen: emit src/registry-stamp.generated.ts with a REGISTRY_STAMP hash of
// the ENTIRE src/ tree, so a build/ vs src/ skew (issue #447) is detectable at
// runtime for ANY source file — not just tool-specs.ts.
// the tool-specs REGISTRY CONTENT, so a build/ vs src/ skew (issue #447) is
// detectable at runtime.
//
// WHY hash the whole src tree (not just tool-specs.ts): the runtime tools are
// assembled from far more than the spec registry — client.ts, the client/*
// domain modules, comment-signal.ts and the drawio-* helpers all ship in build/
// and are loaded by the in-app server. Hashing ONLY tool-specs.ts meant an edit
// to any of those (e.g. a behavioural fix in client.ts) left the stamp unchanged,
// so a stale build/ served the OLD code silently (issue #486). Hashing every
// src/**/*.ts closes that gap: any source edit changes the stamp.
// WHY hash the raw source text (not extracted structured data):
// SHARED_TOOL_SPECS carries `buildShape` functions (the input SCHEMAS) which are
// NOT serializable. The input schema is exactly one of the things that MUST stay
// in sync between build/ and src/, so we cannot drop it from the hash. Rather
// than probe zod with a fragile shim to reconstruct the schema shape, we hash the
// STABLE, deterministic source TEXT of tool-specs.ts. That text fully captures
// every field that must stay in sync — mcpName, inAppKey, description, tier,
// catalogLine AND the buildShape bodies (input schemas) — with zero probing
// fragility. Any edit to a spec (a renamed tool, a reworded description, a
// changed schema field) changes the text and therefore the stamp.
//
// WHY hash the raw source text (not extracted structured data): the tool input
// SCHEMAS live as `buildShape` functions which are NOT serializable, so we cannot
// reduce them to structured data without a fragile zod shim. Hashing the STABLE,
// deterministic source TEXT captures every field that must stay in sync with zero
// probing fragility. Any edit to any source file changes the text → the stamp.
//
// DETERMINISM: files are enumerated recursively, filtered to *.ts EXCLUDING
// *.generated.ts (the codegen's OWN output — including it would create a
// fixed-point cycle), and sorted by their POSIX-normalized path relative to src/
// so the order is platform-independent. Each file contributes its relative path
// AND its content with line endings normalized to LF and a single trailing
// newline stripped, so a CRLF checkout or an editor's trailing-newline habit
// cannot make build/ and src/ disagree. No Date.now / randomness. The loader's
// dev-only stale-check (docmost-client.loader.ts) re-runs THIS SAME enumeration +
// normalization + sha256 and compares to the built REGISTRY_STAMP; the two must
// compute identically.
// DETERMINISM: the hash is computed over the file bytes with line endings
// normalized to LF and a single trailing newline stripped, so a CRLF checkout or
// an editor's trailing-newline habit cannot make build/ and src/ disagree. No
// Date.now / randomness. The loader's dev-only stale-check (docmost-client.loader.ts)
// re-runs THIS SAME normalization + sha256 over src/tool-specs.ts and compares to
// the built REGISTRY_STAMP; the two must compute identically.
//
// This script runs from the `build` and `pretest` npm scripts BEFORE tsc, so
// build/ always carries a stamp derived from the src/ tree that was compiled.
// build/ always carries a stamp derived from the tool-specs.ts that was compiled.
import { createHash } from 'node:crypto';
import { readdirSync, readFileSync, statSync, writeFileSync } from 'node:fs';
import { readFileSync, writeFileSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import { dirname, join, relative, sep } from 'node:path';
import { dirname, join } from 'node:path';
const __dirname = dirname(fileURLToPath(import.meta.url));
const SRC_DIR = join(__dirname, '..', 'src');
const TOOL_SPECS_PATH = join(SRC_DIR, 'tool-specs.ts');
const OUT_PATH = join(SRC_DIR, 'registry-stamp.generated.ts');
/**
* Recursively enumerate every `*.ts` file under `dir`, EXCLUDING the codegen's
* own `*.generated.ts` output (a self-referential cycle otherwise). Returns
* absolute paths, unsorted (the caller sorts by relative path for determinism).
* Kept as a plain exported function so the algorithm has a single home; the
* loader duplicates it because it lives in the CJS server build and cannot import
* this ESM script. If you change the walk/filter here, mirror it in
* apps/server/src/core/ai-chat/tools/docmost-client.loader.ts.
* Deterministic stamp of the tool-specs registry content. Kept as a plain
* function (exported) so the algorithm has a single home; the loader duplicates
* only the tiny normalize+sha256 steps because it lives in the CJS server build
* and cannot import this ESM script. If you change the normalization here, mirror
* it in apps/server/src/core/ai-chat/tools/docmost-client.loader.ts.
*/
export function collectStampFiles(dir) {
const out = [];
for (const entry of readdirSync(dir)) {
const full = join(dir, entry);
if (statSync(full).isDirectory()) {
out.push(...collectStampFiles(full));
} else if (entry.endsWith('.ts') && !entry.endsWith('.generated.ts')) {
out.push(full);
}
}
return out;
}
/**
* Deterministic stamp of the whole src/ tree. Enumerate + sort by POSIX-relative
* path, then fold each file's relative path AND normalized content into one
* sha256. MUST stay byte-for-byte identical to the loader's recompute.
*/
export function computeRegistryStamp(srcDir) {
const files = collectStampFiles(srcDir)
.map((abs) => ({
rel: relative(srcDir, abs).split(sep).join('/'),
abs,
}))
.sort((a, b) => (a.rel < b.rel ? -1 : a.rel > b.rel ? 1 : 0));
const hash = createHash('sha256');
for (const { rel, abs } of files) {
const normalized = readFileSync(abs, 'utf8')
.replace(/\r\n/g, '\n')
.replace(/\n$/, '');
hash.update(rel, 'utf8');
hash.update('\0', 'utf8');
hash.update(normalized, 'utf8');
hash.update('\0', 'utf8');
}
return hash.digest('hex');
export function computeRegistryStamp(toolSpecsSource) {
const normalized = toolSpecsSource.replace(/\r\n/g, '\n').replace(/\n$/, '');
return createHash('sha256').update(normalized, 'utf8').digest('hex');
}
function main() {
const stamp = computeRegistryStamp(SRC_DIR);
const source = readFileSync(TOOL_SPECS_PATH, 'utf8');
const stamp = computeRegistryStamp(source);
const out =
'// AUTO-GENERATED by scripts/gen-registry-stamp.mjs — DO NOT EDIT BY HAND.\n' +
'// A deterministic hash of the whole src/ tree (every src/**/*.ts except\n' +
'// *.generated.ts). Regenerated on every build/pretest so build/ always\n' +
'// matches the compiled src. The in-app loader recomputes this from src and\n' +
'// refuses to run on a mismatch (issue #447/#486). This file is gitignored\n' +
'// and produced by the build — see .gitignore.\n' +
'// A deterministic hash of src/tool-specs.ts content (tool names, descriptions,\n' +
'// tiers, catalog lines and input schemas). Regenerated on every build/pretest\n' +
'// so build/ always matches the compiled src. The in-app loader recomputes this\n' +
'// from src and refuses to run on a mismatch (issue #447). This file is\n' +
'// gitignored and produced by the build — see .gitignore.\n' +
`export const REGISTRY_STAMP = ${JSON.stringify(stamp)};\n`;
writeFileSync(OUT_PATH, out, 'utf8');
// eslint-disable-next-line no-console
+15 -62
View File
@@ -19,10 +19,7 @@ import {
assertYjsEncodable,
MutationResult,
} from "../lib/collaboration.js";
import {
acquireCollabSession,
isCollabAuthFailedError,
} from "../lib/collab-session.js";
import { acquireCollabSession } from "../lib/collab-session.js";
import { withPageLock, isUuid } from "../lib/page-lock.js";
import { getCollabToken, performLogin } from "../lib/auth-utils.js";
import { formatDocmostAxiosError } from "./errors.js";
@@ -495,37 +492,6 @@ export abstract class DocmostClientContext {
);
}
/**
* Run a collab write and, on a Hocuspocus HANDSHAKE auth failure, self-heal
* once (#486). Symmetric to the HTTP-401 path in getCollabTokenWithReauth: the
* REST interceptor and login() already drop the cached collab token on a 401/
* 403, but a rejected WEBSOCKET handshake left the stale token in the cache, so
* every subsequent mutation kept re-presenting the same bad token for up to the
* collab-token TTL (minutes) with no self-heal. Here, when the write rejects
* with the tagged collab-auth error, we invalidate the cached token and retry
* the write EXACTLY once with a force-refreshed token. Not a loop: a second
* failure (or any non-auth error) propagates unchanged.
*
* `write` receives the token to use, so the retry can hand it a genuinely fresh
* one rather than re-running with the same stale string.
*/
protected async writeWithCollabAuthRetry<T>(
collabToken: string,
write: (token: string) => Promise<T>,
): Promise<T> {
try {
return await write(collabToken);
} catch (e) {
if (!isCollabAuthFailedError(e)) throw e;
// The WS handshake rejected our token: drop it from the cache so it can't
// be reused for the rest of the TTL, mint a fresh one (forceRefresh bypasses
// the cache and re-invokes the provider/login), and retry the write once.
this.collabTokenCache = null;
const fresh = await this.getCollabTokenWithReauth(true);
return await write(fresh);
}
}
/**
* Connect to the collaboration websocket, read the live doc, apply
* `transform`, write the result, and wait for the server to persist it
@@ -560,24 +526,19 @@ export abstract class DocmostClientContext {
// unsyncedChanges/connectionLost ack logic live in CollabSession.mutate,
// preserved verbatim from the old inline machine (incl. the #152 structural
// diff that keeps a live editor's cursor anchored).
// Wrap in the collab-auth self-heal (#486): a rejected WS handshake drops the
// cached collab token and retries once with a fresh one (the retry passes the
// refreshed token down to acquireCollabSession via `token`).
return this.writeWithCollabAuthRetry(collabToken, async (token) => {
const session = await acquireCollabSession(pageId, token, this.apiUrl, {
// Only the actual 25s collab connect timeout emits this — the connect-vs-
// unload signal; the other failure paths must NOT emit it.
onConnectTimeout: () =>
this.onMetricFn?.("collab_connect_timeouts_total", 1),
});
try {
return await session.mutate(transform);
} catch (e) {
// Drop the session on any failure so the next call reconnects fresh.
session.destroy("mutate failed");
throw e;
}
const session = await acquireCollabSession(pageId, collabToken, this.apiUrl, {
// Only the actual 25s collab connect timeout emits this — the connect-vs-
// unload signal; the other failure paths must NOT emit it.
onConnectTimeout: () =>
this.onMetricFn?.("collab_connect_timeouts_total", 1),
});
try {
return await session.mutate(transform);
} catch (e) {
// Drop the session on any failure so the next call reconnects fresh.
session.destroy("mutate failed");
throw e;
}
}
/**
@@ -706,11 +667,7 @@ export abstract class DocmostClientContext {
transform: (doc: any) => any,
): Promise<{ doc?: any; verify?: any }> {
const pageUuid = await this.resolvePageId(pageId);
// #486: on a rejected collab-WS handshake, invalidate + refresh the token and
// retry the write once (symmetric to the HTTP-401 reauth path).
return this.writeWithCollabAuthRetry(collabToken, (token) =>
mutatePageContent(pageUuid, token, apiUrl, transform),
);
return mutatePageContent(pageUuid, collabToken, apiUrl, transform);
}
/**
@@ -730,11 +687,7 @@ export abstract class DocmostClientContext {
apiUrl: string,
): Promise<{ doc?: any; verify?: any }> {
const pageUuid = await this.resolvePageId(pageId);
// #486: on a rejected collab-WS handshake, invalidate + refresh the token and
// retry the write once (symmetric to the HTTP-401 reauth path).
return this.writeWithCollabAuthRetry(collabToken, (token) =>
replacePageContent(pageUuid, doc, token, apiUrl),
);
return replacePageContent(pageUuid, doc, collabToken, apiUrl);
}
/**
+2 -7
View File
@@ -127,13 +127,8 @@ export function ReadMixin<TBase extends GConstructor<DocmostClientContext>>(Base
"listPages: tree mode requires a spaceId (a page tree is scoped to one space). Pass spaceId, or omit tree to get the recent-pages list.",
);
}
// #486: propagate `truncated` (same pattern as check_new_comments). The old
// code dropped it, so a caller handed an INCOMPLETE tree (the stdio-fallback
// BFS hit its node cap) had no way to know pages were missing. Return the
// tree alongside the flag; the primary /pages/tree path is uncapped so this
// is false there.
const { pages, truncated } = await this.enumerateSpacePages(spaceId);
return { tree: buildPageTree(pages), truncated };
const { pages } = await this.enumerateSpacePages(spaceId);
return buildPageTree(pages);
}
const clampedLimit = Math.max(1, Math.min(100, limit));
+1
View File
@@ -4,6 +4,7 @@ import { readFileSync } from "fs";
import { fileURLToPath } from "url";
import { dirname, join } from "path";
import { DocmostClient, DocmostMcpConfig } from "./client.js";
import { parseNodeArg } from "@docmost/prosemirror-markdown";
import { searchShapes } from "./lib/drawio-shapes.js";
import { getGuideSection } from "./lib/drawio-guide.js";
import { SHARED_TOOL_SPECS, SharedToolSpec } from "./tool-specs.js";
+4 -26
View File
@@ -37,25 +37,6 @@ const CONNECT_TIMEOUT_MS = 25000;
/** Time we wait for the server to acknowledge our write before giving up. */
const PERSIST_TIMEOUT_MS = 20000;
/**
* Marker property set on the Error thrown when the Hocuspocus handshake REJECTS
* our collab token (onAuthenticationFailed). The client wraps content writes so
* that on this specific failure it invalidates its cached collab token and
* retries once with a fresh one symmetric to the HTTP-401 reauth path (#486).
* A plain message-match would be brittle; a tagged property is unambiguous and
* survives teardown (which rejects pending ops with this SAME error object).
*/
const COLLAB_AUTH_FAILED_MARKER = "collabAuthFailed";
/** True when `e` is the tagged collab-WS auth-failure error (see marker above). */
export function isCollabAuthFailedError(e: unknown): boolean {
return !!(
e &&
typeof e === "object" &&
(e as Record<string, unknown>)[COLLAB_AUTH_FAILED_MARKER] === true
);
}
/**
* Tunables, read fresh from the environment on every acquire so tests (and a
* live rollback) can change them without reloading the module. Mirrors how
@@ -321,13 +302,10 @@ export class CollabSession {
this.openResolve?.();
},
onAuthenticationFailed: () => {
// Tag the error so the client can tell a REJECTED collab token apart
// from a generic disconnect and invalidate + refresh it (#486).
const err = new Error(
"Authentication failed for collaboration connection",
) as Error & { [COLLAB_AUTH_FAILED_MARKER]?: boolean };
err[COLLAB_AUTH_FAILED_MARKER] = true;
this.teardown(err, true);
this.teardown(
new Error("Authentication failed for collaboration connection"),
true,
);
},
});
});
+31 -74
View File
@@ -10,7 +10,7 @@
// exactly mxGraph's convention for a child of a container, so they map across
// directly. Container sizes are computed by ELK; leaf sizes are preserved.
import { Worker } from "node:worker_threads";
import ELK from "elkjs/lib/elk.bundled.js";
import { JSDOM } from "jsdom";
import { normalizeInput, parseCells, type DrawioCell } from "./drawio-xml.js";
@@ -18,33 +18,22 @@ import { normalizeInput, parseCells, type DrawioCell } from "./drawio-xml.js";
const DEFAULT_W = 140;
const DEFAULT_H = 60;
// DoS bounds for the ELK layout. The mxGraph XML is LLM-supplied (layout:"elk"
// in drawioCreate/drawioUpdate). elkjs' layout() returns a Promise but runs the
// crossing-minimisation SYNCHRONOUSLY — it blocks whatever thread it runs on for
// the whole pass. A ~1MB XML (well under the stage-1 16MB cap) can carry
// thousands of nodes. We (a) cap the graph size before ever calling ELK and
// (b) run the layout in a WORKER THREAD so the main event loop stays free, with
// the wall-clock timeout enforced by terminating that worker. On either bound we
// fall back to the ORIGINAL model, the same best-effort contract the catch honours.
// DoS bounds for the in-process ELK layout. The mxGraph XML is LLM-supplied
// (layout:"elk" in drawioCreate/drawioUpdate) and elkjs runs synchronously on
// the MCP server's event loop, so an unbounded graph would block it for
// seconds-to-minutes. A ~1MB XML (well under the stage-1 16MB cap) can carry
// thousands of nodes. We cap the graph size and race the layout against a
// wall-clock timeout; on either bound we fall back to the ORIGINAL model, the
// same best-effort contract the catch already honours.
// - 500 nodes lays out in well under a second; beyond that ELK cost climbs
// steeply, so refuse and leave the (already-valid) model untouched.
// - Edges dominate the layered-crossing cost, so allow a bit more headroom
// (1000) than nodes but still bound them.
// - The timeout is a HARD kill of the worker thread — the only way to interrupt
// synchronous JS. The in-process setTimeout race we used before was an
// illusion: the timer could never fire while the SAME thread was blocked
// inside elkjs, so it "protected" nothing. Now the timer runs on the main
// thread while ELK runs on the worker, so it can actually fire and terminate.
// - 5s is generous for any graph within the caps yet short enough that a
// pathological input can never wedge the server.
const ELK_MAX_NODES = 500;
const ELK_MAX_EDGES = 1000;
// Wall-clock ceiling for a single layout pass. Overridable for tests (a tiny
// value forces the terminate-on-timeout path deterministically); a non-positive
// or unparseable override falls back to the default.
const ELK_TIMEOUT_DEFAULT_MS = 5000;
function resolveElkTimeoutMs(): number {
const raw = Number(process.env.DRAWIO_ELK_TIMEOUT_MS);
return Number.isFinite(raw) && raw > 0 ? Math.floor(raw) : ELK_TIMEOUT_DEFAULT_MS;
}
const ELK_TIMEOUT_MS = 5000;
// Spacing is set >=150px on purpose so an ELK layout never trips the linter's
// "gap between adjacent shapes < 150px" quality warning (acceptance #3).
@@ -89,57 +78,13 @@ interface ElkGraph extends ElkNode {
edges?: ElkEdge[];
}
/**
* Run one ELK layered layout on a worker thread and resolve with the laid-out
* graph. The timeout is enforced by `worker.terminate()` a HARD kill, which is
* the only way to interrupt elkjs' synchronous crossing-minimisation once it has
* started. Rejects on timeout, worker error, or an early exit; the caller treats
* any rejection as "keep the original model" (best-effort layout).
*/
function layoutInWorker(graph: ElkGraph, timeoutMs: number): Promise<ElkGraph> {
return new Promise((resolve, reject) => {
const worker = new Worker(
new URL("./drawio-layout.worker.js", import.meta.url),
{ workerData: { graph } },
);
let settled = false;
const finish = (fn: () => void) => {
if (settled) return;
settled = true;
clearTimeout(timer);
// Always tear the worker down: on the happy path so it does not linger,
// on timeout so the blocked synchronous ELK run is actually interrupted.
void worker.terminate();
fn();
};
const timer = setTimeout(
() => finish(() => reject(new Error("ELK layout timed out"))),
timeoutMs,
);
worker.once("message", (msg: { ok?: boolean; laid?: ElkGraph; error?: string }) => {
finish(() =>
msg?.ok
? resolve(msg.laid as ElkGraph)
: reject(new Error(msg?.error ?? "ELK layout failed")),
);
});
worker.once("error", (err) => finish(() => reject(err)));
worker.once("exit", (code) => {
// A clean exit after we already settled is normal (terminate()); only an
// unexpected early exit while still pending is a failure.
if (settled) return;
finish(() => reject(new Error(`ELK worker exited early (code ${code})`)));
});
});
}
/**
* Apply an ELK layered layout to a drawio input and return a full mxGraphModel
* string with rewritten geometry. Accepts the same three input forms as
* drawioCreate (a bare model, an <mxfile>, or a <mxCell> list). Async because
* the layout runs on a worker thread. On any layout failure (including a
* terminate-on-timeout) the ORIGINAL (normalized) model is returned unchanged
* layout is best-effort polish, never a reason to fail the write.
* elkjs' layout() is promise-based. On any layout failure the ORIGINAL
* (normalized) model is returned unchanged layout is best-effort polish, never
* a reason to fail the write.
*/
export async function applyElkLayout(inputXml: string): Promise<string> {
const modelXml = normalizeInput(inputXml);
@@ -205,14 +150,26 @@ export async function applyElkLayout(inputXml: string): Promise<string> {
};
let laid: ElkGraph;
let timer: ReturnType<typeof setTimeout> | undefined;
try {
// Run the (synchronous-under-the-hood) ELK pass on a worker thread so the
// main event loop is never blocked, and enforce the wall-clock ceiling by
// terminating that worker on timeout. A graph under the node/edge caps but
// still pathologically slow is hard-killed instead of wedging anything.
laid = await layoutInWorker(graph, resolveElkTimeoutMs());
// elkjs ships a CJS default export whose interop shape varies across
// module systems; resolve the real constructor at runtime, then cast (the
// runtime call is verified — see the layout unit test).
const Ctor: any = (ELK as any).default ?? ELK;
const elk = new Ctor();
// Race the layout against a wall-clock timeout so a graph that is under the
// node/edge caps but still pathologically slow can never wedge the server.
const timeout = new Promise<never>((_, reject) => {
timer = setTimeout(
() => reject(new Error("ELK layout timed out")),
ELK_TIMEOUT_MS,
);
});
laid = (await Promise.race([elk.layout(graph as any), timeout])) as ElkGraph;
} catch {
return modelXml; // best-effort: keep the model as-is on timeout or ELK failure
} finally {
if (timer) clearTimeout(timer);
}
// Collect computed geometry per node id (coords are parent-relative already).
@@ -1,36 +0,0 @@
// Worker-thread entry for the ELK layered layout (issue #486, commit 1).
//
// elkjs' layout() returns a Promise but runs the actual crossing-minimisation
// SYNCHRONOUSLY — it blocks whatever thread it runs on for the whole pass. On
// the in-app MCP host that thread used to be the main NestJS event loop, so a
// pathological graph at the node/edge cap could wedge ALL HTTP/SSE/loopback
// traffic while it churned. Running it HERE, on a dedicated worker thread, keeps
// the main loop free; the parent enforces the wall-clock timeout by calling
// `worker.terminate()` — the only way to interrupt synchronous JS — since the
// in-process `setTimeout` race the parent used before could never fire while the
// same thread was blocked inside elkjs.
import { parentPort, workerData } from "node:worker_threads";
import ELK from "elkjs/lib/elk.bundled.js";
interface WorkerInput {
graph: unknown;
}
const { graph } = (workerData ?? {}) as WorkerInput;
// elkjs ships a CJS default export whose interop shape varies across module
// systems; resolve the real constructor at runtime (same as the parent did).
const Ctor: any = (ELK as any).default ?? ELK;
const elk = new Ctor();
elk
.layout(graph as any)
.then((laid: unknown) => {
parentPort?.postMessage({ ok: true, laid });
})
.catch((err: unknown) => {
parentPort?.postMessage({
ok: false,
error: err instanceof Error ? err.message : String(err),
});
});
+3 -4
View File
@@ -923,10 +923,9 @@ export const SHARED_TOOL_SPECS = {
'List the most recent pages (ordered by updatedAt, descending), ' +
'optionally scoped to a single space. Returns a bounded list (default ' +
'50, max 100) — use search for lookups in large spaces. tree:true (with ' +
"spaceId) returns { tree, truncated } — the space's full page hierarchy " +
'as a nested tree, plus a `truncated` flag that is true when the tree was ' +
'capped and is INCOMPLETE — but is DEPRECATED, use getTree instead ' +
'(leaner nodes, plus rootPageId / maxDepth).',
"spaceId) returns the space's full page hierarchy as a nested tree, but " +
'is DEPRECATED — use getTree instead (leaner nodes, plus rootPageId / ' +
'maxDepth).',
tier: 'core',
catalogLine:
"listPages — list recent pages (tree:true is deprecated; use getTree for the hierarchy).",
@@ -1,137 +0,0 @@
// Unit tests for the collab-token reset on a Hocuspocus WS auth failure (#486).
//
// Before this fix the cached collab token (#435) was dropped ONLY on an HTTP
// 401/403 (the REST interceptor + login()); a rejected collab-WEBSOCKET handshake
// left the stale token in the cache, so every subsequent mutation re-presented
// the SAME bad token for up to the collab-token TTL (minutes) with no self-heal.
//
// The fix wraps collab writes in `writeWithCollabAuthRetry`: when the write
// rejects with the tagged collab-auth error (collab-session.ts's
// onAuthenticationFailed), it invalidates the cached token and retries the write
// ONCE with a force-refreshed token — symmetric to the HTTP-401 path.
//
// writeWithCollabAuthRetry / getCollabTokenWithReauth are protected in TS but
// plain methods on the compiled build, so the tests call them directly (same
// convention as collab-token-cache.test.mjs).
import { test, afterEach } from "node:test";
import assert from "node:assert/strict";
import { DocmostClient } from "../../build/client.js";
const ENV_KEY = "MCP_COLLAB_TOKEN_TTL_MS";
afterEach(() => {
delete process.env[ENV_KEY];
});
// A counting provider that returns a distinct token each call so a cached
// (reused) token is visibly the SAME string while a fresh mint is different.
function countingProvider() {
let n = 0;
const fn = async () => {
n++;
return `provider-token-${n}`;
};
return {
fn,
get calls() {
return n;
},
};
}
// The tagged error collab-session.ts throws on a rejected WS handshake.
function collabAuthError() {
const err = new Error("Authentication failed for collaboration connection");
err.collabAuthFailed = true;
return err;
}
test("a WS auth failure clears the cached token and retries the write with a FRESH one (#486)", async () => {
process.env[ENV_KEY] = "300000"; // 5 min: the cache is warm across the burst.
const p = countingProvider();
const client = new DocmostClient({
apiUrl: "http://127.0.0.1:1/api",
getToken: async () => "access",
getCollabToken: p.fn,
});
// Warm the cache the way a real write would (mints provider-token-1).
const initial = await client.getCollabTokenWithReauth();
assert.equal(initial, "provider-token-1");
assert.equal(p.calls, 1);
const tokensSeen = [];
const write = async (token) => {
tokensSeen.push(token);
// The FIRST attempt (with the stale cached token) fails the WS handshake;
// the retry (with a fresh token) succeeds.
if (tokensSeen.length === 1) throw collabAuthError();
return `written-with:${token}`;
};
const result = await client.writeWithCollabAuthRetry(initial, write);
assert.equal(tokensSeen.length, 2, "write attempted exactly twice (one retry)");
assert.equal(tokensSeen[0], "provider-token-1", "first attempt used the stale token");
assert.equal(
tokensSeen[1],
"provider-token-2",
"retry used a FRESH force-refreshed token, not the stale cached one",
);
assert.equal(result, "written-with:provider-token-2", "the retry's result wins");
assert.equal(p.calls, 2, "exactly one extra mint for the retry — no loop");
// The cache now holds the fresh token, so a subsequent op reuses it (proving
// the stale token was evicted and the fresh one cached, not re-minted).
const next = await client.getCollabTokenWithReauth();
assert.equal(next, "provider-token-2", "the fresh token replaced the stale cache");
assert.equal(p.calls, 2, "served from cache — provider not re-invoked");
});
test("a successful write is NOT retried and mints nothing extra", async () => {
process.env[ENV_KEY] = "300000";
const p = countingProvider();
const client = new DocmostClient({
apiUrl: "http://127.0.0.1:1/api",
getToken: async () => "access",
getCollabToken: p.fn,
});
const initial = await client.getCollabTokenWithReauth(); // provider-token-1
let attempts = 0;
const result = await client.writeWithCollabAuthRetry(initial, async (token) => {
attempts++;
return `ok:${token}`;
});
assert.equal(attempts, 1, "no retry on success");
assert.equal(result, "ok:provider-token-1");
assert.equal(p.calls, 1, "no extra mint");
});
test("a NON-auth write error propagates unchanged (no reset, no retry)", async () => {
process.env[ENV_KEY] = "300000";
const p = countingProvider();
const client = new DocmostClient({
apiUrl: "http://127.0.0.1:1/api",
getToken: async () => "access",
getCollabToken: p.fn,
});
const initial = await client.getCollabTokenWithReauth(); // provider-token-1
let attempts = 0;
await assert.rejects(
client.writeWithCollabAuthRetry(initial, async () => {
attempts++;
throw new Error("collab connection closed before persist"); // NOT tagged.
}),
/closed before persist/,
);
assert.equal(attempts, 1, "a non-auth error is not retried");
assert.equal(p.calls, 1, "the cache is untouched -> no fresh mint");
// Cache still holds the original token (was never invalidated).
const still = await client.getCollabTokenWithReauth();
assert.equal(still, "provider-token-1");
assert.equal(p.calls, 1);
});
@@ -1,55 +0,0 @@
// Unit test: listPages tree mode must propagate the `truncated` flag (#486).
//
// enumerateSpacePages returns { pages, truncated } — truncated is true ONLY when
// the stdio-fallback BFS hit its node cap (the primary /pages/tree path is
// uncapped). The old tree-mode listPages destructured only `pages` and returned a
// bare tree, dropping `truncated`, so a caller handed an INCOMPLETE tree had no
// way to know pages were missing. The fix returns { tree, truncated } (same
// pattern check_new_comments uses).
//
// Reaching the real cap (MAX_NODES = 10000) in a mock is impractical, so we stub
// enumerateSpacePages directly to assert the flag is threaded through verbatim.
import { test } from "node:test";
import assert from "node:assert/strict";
import { DocmostClient } from "../../build/client.js";
function stubClient() {
const client = new DocmostClient({
apiUrl: "http://127.0.0.1:1/api",
getToken: async () => "access",
});
// No network: the tree path only calls ensureAuthenticated + enumerateSpacePages.
client.ensureAuthenticated = async () => {};
return client;
}
const onePage = [{ id: "r1", title: "Root", parentPageId: null }];
test("tree mode carries truncated:true when the enumeration truncated (#486)", async () => {
const client = stubClient();
client.enumerateSpacePages = async () => ({ pages: onePage, truncated: true });
const res = await client.listPages("space-1", 50, true);
assert.equal(res.truncated, true, "the truncated flag is threaded through");
assert.ok(Array.isArray(res.tree), "the built tree rides alongside the flag");
assert.equal(res.tree[0].id, "r1");
});
test("tree mode carries truncated:false for a complete enumeration", async () => {
const client = stubClient();
client.enumerateSpacePages = async () => ({ pages: onePage, truncated: false });
const res = await client.listPages("space-1", 50, true);
assert.equal(res.truncated, false);
assert.equal(res.tree[0].id, "r1");
});
test("tree mode still requires a spaceId", async () => {
const client = stubClient();
await assert.rejects(
client.listPages(undefined, 50, true),
/tree mode requires a spaceId/,
);
});
@@ -184,14 +184,12 @@ test("enumerateSpacePages (via listPages tree) uses one /pages/tree request", as
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
// listPages tree:true -> enumerateSpacePages(spaceId) -> { tree, truncated }.
const { tree, truncated } = await client.listPages("space-1", 50, true);
// listPages tree:true -> enumerateSpacePages(spaceId) -> buildPageTree.
const tree = await client.listPages("space-1", 50, true);
assert.equal(treeRequests, 1, "exactly one /pages/tree request for the space");
assert.equal(sidebarRequests, 0, "no per-node sidebar BFS requests");
assert.deepEqual(treeBody, { spaceId: "space-1" }, "space scope posts spaceId only");
// The uncapped /pages/tree path is never truncated (#486).
assert.equal(truncated, false, "primary /pages/tree path is not truncated");
// buildPageTree nests c1 under r1; two roots at the top level.
assert.equal(tree.length, 2, "two root nodes");
const r1 = tree.find((n) => n.id === "r1");
@@ -251,7 +249,7 @@ test("enumerateSpacePages falls back to the cursor BFS on /pages/tree 404", asyn
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const { tree, truncated } = await client.listPages("space-1", 50, true);
const tree = await client.listPages("space-1", 50, true);
assert.ok(treeRequests >= 1, "the tree endpoint was attempted first");
assert.deepEqual(
@@ -259,8 +257,6 @@ test("enumerateSpacePages falls back to the cursor BFS on /pages/tree 404", asyn
["<root>", "r1"],
"fell back to the sidebar BFS: roots then the root's children",
);
// Small fallback walk well under the node cap -> not truncated (#486).
assert.equal(truncated, false, "fallback BFS below the cap is not truncated");
assert.equal(tree.length, 1, "one root in the built tree");
assert.equal(tree[0].children[0].id, "c1", "leaf nested via the BFS");
});
@@ -101,88 +101,6 @@ test("DoS guard: a graph over the node cap is returned unchanged, quickly", asyn
assert.ok(dt < 2000, `cap path should be fast, took ${dt}ms`);
});
/** Build a layered DAG near the caps: `n` vertices, up to ~2 edges each into the
* next layer of `layerSize`. Used as a real worst-case graph for the benchmark. */
function layeredGraph(n, layerSize) {
let cells = "";
for (let i = 2; i < 2 + n; i++) {
cells +=
`<mxCell id="${i}" value="N${i}" style="rounded=1;html=1;" vertex="1" parent="1">` +
`<mxGeometry x="10" y="10" width="120" height="60" as="geometry"/></mxCell>`;
}
let ei = 0;
for (let i = 2; i < 2 + n; i++) {
for (const off of [layerSize, layerSize + 1]) {
const t = i + off;
if (t < 2 + n) cells += `<mxCell id="e${ei++}" edge="1" parent="1" source="${i}" target="${t}"><mxGeometry relative="1" as="geometry"/></mxCell>`;
}
}
return (
'<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/>' +
cells +
"</root></mxGraphModel>"
);
}
test("terminate-on-timeout: a layout that exceeds the wall-clock ceiling is hard-killed and the original model is returned (#486)", async () => {
// A 1ms ceiling fires before the worker can even finish loading elkjs, so the
// parent must terminate() the worker and fall back to the ORIGINAL model. On
// the OLD in-process race this timer could never fire while the SAME thread was
// blocked inside elkjs — the fallback path was unreachable; here it works.
const prev = process.env.DRAWIO_ELK_TIMEOUT_MS;
process.env.DRAWIO_ELK_TIMEOUT_MS = "1";
try {
const model = layeredGraph(400, 20);
const t0 = Date.now();
const laid = await applyElkLayout(model);
const dt = Date.now() - t0;
// Original geometry is preserved verbatim: every vertex is still stacked at
// (10,10), proving NO ELK coordinates were applied (the pass was killed).
const verts = parseCells(laid).filter((c) => c.vertex);
assert.equal(verts.length, 400, "all vertices survived the fallback");
for (const v of verts) {
assert.equal(v.geometry.x, 10, "x untouched -> layout was terminated");
assert.equal(v.geometry.y, 10, "y untouched -> layout was terminated");
}
// The kill is prompt: terminate() returns the call well under the natural
// layout time for a 400-node graph.
assert.ok(dt < 2000, `terminate path should be prompt, took ${dt}ms`);
} finally {
if (prev === undefined) delete process.env.DRAWIO_ELK_TIMEOUT_MS;
else process.env.DRAWIO_ELK_TIMEOUT_MS = prev;
}
});
test("benchmark guard: a worst-case graph AT the cap lays out without wedging the main event loop (#486)", async () => {
// ~500 nodes / ~1000 edges — a real worst case at the node/edge caps. The
// layout runs on a WORKER thread, so the MAIN event loop must stay responsive
// throughout: a timer scheduled on the main thread keeps firing while ELK
// churns. On the OLD synchronous-on-main-thread code this counter would be
// pinned at 0 for the whole layout (event loop wedged) — exactly the prod fire.
const model = layeredGraph(500, 20);
let mainLoopTicks = 0;
const iv = setInterval(() => {
mainLoopTicks++;
}, 2);
const t0 = Date.now();
const laid = await applyElkLayout(model);
const dt = Date.now() - t0;
clearInterval(iv);
assert.ok(
mainLoopTicks > 0,
"main event loop must stay responsive while ELK runs on the worker",
);
// Benchmark guard: the worst-case graph actually LAYS OUT within the default
// ceiling (it did not fall back). At least one vertex moved off the stack.
const verts = parseCells(laid).filter((c) => c.vertex);
assert.equal(verts.length, 500, "all vertices survived");
const moved = verts.some((v) => v.geometry.x !== 10 || v.geometry.y !== 10);
assert.ok(moved, "layout was applied (did not time out / fall back)");
// Sanity ceiling well under the 5s wall-clock timeout.
assert.ok(dt < 5000, `worst-case layout should be under the ceiling, took ${dt}ms`);
});
test("layout is best-effort: an empty/degenerate model is returned intact", async () => {
const model =
'<mxGraphModel><root><mxCell id="0"/><mxCell id="1" parent="0"/></root></mxGraphModel>';
+70 -189
View File
@@ -1,220 +1,101 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import {
mkdtempSync,
mkdirSync,
writeFileSync,
rmSync,
readdirSync,
statSync,
readFileSync,
} from "node:fs";
import { tmpdir } from "node:os";
import { fileURLToPath } from "node:url";
import { dirname, join, relative, sep } from "node:path";
import { createHash } from "node:crypto";
import { readFileSync } from "node:fs";
import { fileURLToPath } from "node:url";
import { dirname, join } from "node:path";
import { computeRegistryStamp } from "../../scripts/gen-registry-stamp.mjs";
import { REGISTRY_STAMP } from "../../build/index.js";
// Guard tests for the build/src-skew stamp (issues #447/#486). The codegen script
// exports `computeRegistryStamp(srcDir)` — a sha256 over the WHOLE src/ tree
// (every src/**/*.ts EXCEPT *.generated.ts), each file folded in as its
// POSIX-relative path + its normalized content (CRLF->LF, single trailing newline
// stripped). Hashing the whole tree (not just tool-specs.ts) is #486: an edit to
// client.ts / a client/* module without a rebuild must ALSO redden. The in-app
// loader (apps/server/.../docmost-client.loader.ts) DUPLICATES this enumerate+
// normalize+sha256 to refuse a stale build. These tests pin the algorithm and
// assert the built stamp matches the current src.
// Guard tests for the build/src-skew stamp (issue #447). The codegen script
// exports `computeRegistryStamp(sourceText)` — a sha256 over normalized source
// text (CRLF->LF, single trailing newline stripped). The in-app loader
// (apps/server/.../docmost-client.loader.ts) DUPLICATES that normalize+sha256 to
// recompute the stamp from src and refuse a stale build. These tests pin the
// algorithm's behaviour AND assert the built stamp matches the current src, so a
// stale generated file OR a normalize divergence reddens.
const __dirname = dirname(fileURLToPath(import.meta.url));
const SRC_DIR = join(__dirname, "..", "..", "src");
const TOOL_SPECS_PATH = join(__dirname, "..", "..", "src", "tool-specs.ts");
// Build a throwaway src/ tree from a { relPath: content } map and return its dir.
function makeSrcTree(files) {
const root = mkdtempSync(join(tmpdir(), "mcp-stamp-tree-"));
const src = join(root, "src");
for (const [rel, content] of Object.entries(files)) {
const full = join(src, rel);
mkdirSync(dirname(full), { recursive: true });
writeFileSync(full, content, "utf8");
}
return { src, cleanup: () => rmSync(root, { recursive: true, force: true }) };
}
test("computeRegistryStamp is deterministic: same tree -> same hash", () => {
const a = makeSrcTree({ "tool-specs.ts": "export const X = 1;\n" });
const b = makeSrcTree({ "tool-specs.ts": "export const X = 1;\n" });
try {
assert.equal(computeRegistryStamp(a.src), computeRegistryStamp(b.src));
} finally {
a.cleanup();
b.cleanup();
}
test("computeRegistryStamp is deterministic: same input -> same hash", () => {
const input = "export const X = 1;\nexport const Y = 2;\n";
assert.equal(computeRegistryStamp(input), computeRegistryStamp(input));
});
test("computeRegistryStamp returns a 64-char lowercase hex sha256", () => {
const t = makeSrcTree({ "tool-specs.ts": "anything\n" });
try {
assert.match(computeRegistryStamp(t.src), /^[0-9a-f]{64}$/);
} finally {
t.cleanup();
}
const stamp = computeRegistryStamp("anything");
assert.match(stamp, /^[0-9a-f]{64}$/);
});
// #486 CORE: an edit to a NON-tool-specs source file (client.ts) must change the
// stamp. Under the old single-file (tool-specs.ts only) hash this edit was
// invisible and a stale build/ served the old client.ts silently.
test("editing client.ts (not tool-specs.ts) changes the stamp (#486)", () => {
const before = makeSrcTree({
"tool-specs.ts": "export const SPECS = 1;\n",
"client.ts": "export const impl = 'v1';\n",
});
const after = makeSrcTree({
"tool-specs.ts": "export const SPECS = 1;\n",
"client.ts": "export const impl = 'v2';\n",
});
try {
assert.notEqual(
computeRegistryStamp(before.src),
computeRegistryStamp(after.src),
"a client.ts edit with an unchanged tool-specs.ts must move the stamp",
);
} finally {
before.cleanup();
after.cleanup();
}
test("normalizes CRLF vs LF: the same content hashes equal", () => {
const lf = "line1\nline2\nline3";
const crlf = "line1\r\nline2\r\nline3";
assert.equal(computeRegistryStamp(crlf), computeRegistryStamp(lf));
});
test("editing a nested client/* module changes the stamp", () => {
const before = makeSrcTree({
"tool-specs.ts": "x\n",
"client/read.ts": "export const READ = 1;\n",
});
const after = makeSrcTree({
"tool-specs.ts": "x\n",
"client/read.ts": "export const READ = 2;\n",
});
try {
assert.notEqual(
computeRegistryStamp(before.src),
computeRegistryStamp(after.src),
);
} finally {
before.cleanup();
after.cleanup();
}
test("normalizes a trailing newline: with/without a final \\n hashes equal", () => {
const noTrailing = "alpha\nbeta";
const trailing = "alpha\nbeta\n";
assert.equal(computeRegistryStamp(trailing), computeRegistryStamp(noTrailing));
});
// *.generated.ts is EXCLUDED (else the codegen's own output is a fixed-point
// cycle): adding/removing/changing it must not move the stamp.
test("*.generated.ts is excluded from the stamp", () => {
const without = makeSrcTree({ "tool-specs.ts": "x\n" });
const withGen = makeSrcTree({
"tool-specs.ts": "x\n",
"registry-stamp.generated.ts": 'export const REGISTRY_STAMP = "abc";\n',
});
try {
assert.equal(
computeRegistryStamp(without.src),
computeRegistryStamp(withGen.src),
"a *.generated.ts file must not affect the stamp",
);
} finally {
without.cleanup();
withGen.cleanup();
}
test("a CRLF checkout WITH a trailing CRLF still hashes equal to bare LF", () => {
// A worst-case Windows checkout: CRLF line endings + a trailing CRLF. Both the
// \r\n->\n replace and the trailing-newline strip must apply for parity.
const bare = "alpha\nbeta";
const crlfTrailing = "alpha\r\nbeta\r\n";
assert.equal(
computeRegistryStamp(crlfTrailing),
computeRegistryStamp(bare),
);
});
test("a CRLF checkout WITH trailing CRLF hashes equal to bare LF", () => {
const bare = makeSrcTree({ "tool-specs.ts": "alpha\nbeta" });
const crlfTrailing = makeSrcTree({ "tool-specs.ts": "alpha\r\nbeta\r\n" });
try {
assert.equal(
computeRegistryStamp(crlfTrailing.src),
computeRegistryStamp(bare.src),
);
} finally {
bare.cleanup();
crlfTrailing.cleanup();
}
test("a real content change hashes differently", () => {
const before = "export const description = 'search a page';\n";
const after = "export const description = 'search a PAGE';\n";
assert.notEqual(computeRegistryStamp(before), computeRegistryStamp(after));
});
// Only a SINGLE trailing newline is stripped — a second blank line is content.
// Only a SINGLE trailing newline is stripped — a second blank line is content and
// must change the hash. This pins the exact `/\n$/` semantics the loader mirrors.
test("only ONE trailing newline is stripped (two differ from one)", () => {
const one = makeSrcTree({ "tool-specs.ts": "x\n" });
const two = makeSrcTree({ "tool-specs.ts": "x\n\n" });
try {
assert.notEqual(
computeRegistryStamp(one.src),
computeRegistryStamp(two.src),
);
} finally {
one.cleanup();
two.cleanup();
}
assert.notEqual(
computeRegistryStamp("x\n"),
computeRegistryStamp("x\n\n"),
);
});
// Cross-impl equality against a fixed, documented tree. The SAME literal tree and
// expected hash are asserted in the server-side jest test
// (docmost-client.loader.spec.ts). If either side's enumerate+normalize+sha256
// ever diverges, one of the two tests reddens. The tree exercises: a nested file,
// BOTH normalize steps (tool-specs.ts uses CRLF + trailing \n) and the
// *.generated.ts exclusion.
const CROSS_IMPL_TREE = {
"tool-specs.ts": "line1\r\nline2\n",
"client/read.ts": "export const R = 1;\n",
"registry-stamp.generated.ts": 'export const REGISTRY_STAMP="ignored";\n',
};
const CROSS_IMPL_EXPECTED =
"131c1b9e4e2f5a7d6cef91ca8df619822b442f52bc45ebd09474a4c1d6728616";
test("fixed-tree hash matches the documented cross-impl value", () => {
const t = makeSrcTree(CROSS_IMPL_TREE);
try {
assert.equal(computeRegistryStamp(t.src), CROSS_IMPL_EXPECTED);
} finally {
t.cleanup();
}
// Cross-impl equality against a fixed, documented input. The SAME literal input
// and expected hash are asserted in the server-side jest test
// (docmost-client.loader.spec.ts). If either side's normalize+sha256 ever
// diverges, one of the two tests reddens. Input exercises BOTH normalize steps.
test("fixed-input hash matches the documented cross-impl value", () => {
const FIXED_INPUT = "line1\r\nline2\n";
const EXPECTED =
"683376e290829b482c2655745caffa7a1dccfa10afaa62dac2b42dd6c68d0f83";
assert.equal(computeRegistryStamp(FIXED_INPUT), EXPECTED);
});
// Sanity: the EXPECTED constant is not a magic value but the documented
// enumerate+normalize+sha256 of CROSS_IMPL_TREE (a local re-implementation).
test("the documented EXPECTED is the enumerate+normalize+sha256 of the tree", () => {
const t = makeSrcTree(CROSS_IMPL_TREE);
try {
const collect = (dir) => {
const out = [];
for (const e of readdirSync(dir)) {
const f = join(dir, e);
if (statSync(f).isDirectory()) out.push(...collect(f));
else if (e.endsWith(".ts") && !e.endsWith(".generated.ts")) out.push(f);
}
return out;
};
const files = collect(t.src)
.map((abs) => ({ rel: relative(t.src, abs).split(sep).join("/"), abs }))
.sort((a, b) => (a.rel < b.rel ? -1 : a.rel > b.rel ? 1 : 0));
const h = createHash("sha256");
for (const { rel, abs } of files) {
const n = readFileSync(abs, "utf8")
.replace(/\r\n/g, "\n")
.replace(/\n$/, "");
h.update(rel, "utf8");
h.update("\0", "utf8");
h.update(n, "utf8");
h.update("\0", "utf8");
}
assert.equal(h.digest("hex"), CROSS_IMPL_EXPECTED);
} finally {
t.cleanup();
}
// DESYNC GUARD (covers reviewer suggestion 2). Recompute the stamp from the
// actual src/tool-specs.ts and assert it equals the REGISTRY_STAMP baked into the
// freshly-built build/index.js. This reddens if the generated file is stale OR if
// the codegen normalize ever diverges from what produced the built stamp.
test("built REGISTRY_STAMP equals the stamp recomputed from src/tool-specs.ts", () => {
const source = readFileSync(TOOL_SPECS_PATH, "utf8");
assert.equal(computeRegistryStamp(source), REGISTRY_STAMP);
});
// DESYNC GUARD. Recompute the stamp from the REAL src/ tree and assert it equals
// the REGISTRY_STAMP baked into the freshly-built build/index.js. This reddens if
// the generated file is stale OR if the codegen ever diverges from what produced
// the built stamp.
test("built REGISTRY_STAMP equals the stamp recomputed from src/", () => {
assert.equal(computeRegistryStamp(SRC_DIR), REGISTRY_STAMP);
// Sanity: the fixed-input helper computes the SAME way the codegen does, proving
// the EXPECTED constant above is not an arbitrary magic value but the documented
// normalize+sha256 of FIXED_INPUT. Belt-and-braces so a bad EXPECTED can't hide a
// real regression.
test("the documented EXPECTED constant is the normalize+sha256 of FIXED_INPUT", () => {
const FIXED_INPUT = "line1\r\nline2\n";
const normalized = FIXED_INPUT.replace(/\r\n/g, "\n").replace(/\n$/, "");
const expected = createHash("sha256")
.update(normalized, "utf8")
.digest("hex");
assert.equal(computeRegistryStamp(FIXED_INPUT), expected);
});
+6 -114
View File
@@ -1,62 +1,8 @@
diff --git a/dist/index.js b/dist/index.js
index ae447a12f7823ec0a00837ee9f0eb809a610d5f8..210b5a9009e5cf1537cb2007c524007c1a01253c 100644
index ae447a12f7823ec0a00837ee9f0eb809a610d5f8..a3402b2c2d021ef432cfa76e35d370073d525135 100644
--- a/dist/index.js
+++ b/dist/index.js
@@ -5036,9 +5036,40 @@ function writeToServerResponse({
break;
const canContinue = response.write(value);
if (!canContinue) {
- await new Promise((resolve3) => {
- response.once("drain", resolve3);
+ // PATCH(docmost #486): race "drain" against "close"/"error". The
+ // original awaited ONLY "drain", so a client that disconnected mid-write
+ // (the socket never drains) parked this loop FOREVER: the finally never
+ // ran, response.end() was unreachable, and the reader + buffered chunks
+ // were held until process restart. On close/error we cancel the reader
+ // and break so the finally always runs (safe for detached runs:
+ // consumeStream drains the SDK stream independently). Listener hygiene:
+ // all three once-listeners are removed on the first settle, so they
+ // cannot pile up one-per-stall.
+ const closed = await new Promise((resolve3) => {
+ function finish(isClosed) {
+ response.removeListener("drain", onDrain);
+ response.removeListener("close", onClose);
+ response.removeListener("error", onError);
+ resolve3(isClosed);
+ }
+ function onDrain() {
+ finish(false);
+ }
+ function onClose() {
+ finish(true);
+ }
+ function onError() {
+ finish(true);
+ }
+ response.once("drain", onDrain);
+ response.once("close", onClose);
+ response.once("error", onError);
});
+ if (closed) {
+ await reader.cancel().catch(() => {
+ });
+ break;
+ }
}
}
} catch (error) {
@@ -5047,7 +5078,9 @@ function writeToServerResponse({
response.end();
}
};
- read();
+ read().catch((error) => {
+ console.error("ai writeToServerResponse read() failed:", error);
+ });
}
// src/text-stream/pipe-text-stream-to-response.ts
@@ -6578,9 +6611,19 @@ function createOutputTransformStream(output) {
@@ -6578,9 +6578,19 @@ function createOutputTransformStream(output) {
controller.enqueue({ part: chunk, partialOutput: void 0 });
return;
}
@@ -77,7 +23,7 @@ index ae447a12f7823ec0a00837ee9f0eb809a610d5f8..210b5a9009e5cf1537cb2007c524007c
const result = await output.parsePartialOutput({ text: text2 });
if (result !== void 0) {
const currentJson = JSON.stringify(result.partial);
@@ -6959,7 +7002,7 @@ var DefaultStreamTextResult = class {
@@ -6959,7 +6969,7 @@ var DefaultStreamTextResult = class {
})
);
}
@@ -87,64 +33,10 @@ index ae447a12f7823ec0a00837ee9f0eb809a610d5f8..210b5a9009e5cf1537cb2007c524007c
maxRetries: maxRetriesArg,
abortSignal
diff --git a/dist/index.mjs b/dist/index.mjs
index 663875332e3f9a9bd167c25583c515876f42951b..a51514a390d22811a407edd8b703e21793586cc8 100644
index 663875332e3f9a9bd167c25583c515876f42951b..b840b0502c9894df983e0154805abb80e70e6331 100644
--- a/dist/index.mjs
+++ b/dist/index.mjs
@@ -4957,9 +4957,40 @@ function writeToServerResponse({
break;
const canContinue = response.write(value);
if (!canContinue) {
- await new Promise((resolve3) => {
- response.once("drain", resolve3);
+ // PATCH(docmost #486): race "drain" against "close"/"error". The
+ // original awaited ONLY "drain", so a client that disconnected mid-write
+ // (the socket never drains) parked this loop FOREVER: the finally never
+ // ran, response.end() was unreachable, and the reader + buffered chunks
+ // were held until process restart. On close/error we cancel the reader
+ // and break so the finally always runs (safe for detached runs:
+ // consumeStream drains the SDK stream independently). Listener hygiene:
+ // all three once-listeners are removed on the first settle, so they
+ // cannot pile up one-per-stall.
+ const closed = await new Promise((resolve3) => {
+ function finish(isClosed) {
+ response.removeListener("drain", onDrain);
+ response.removeListener("close", onClose);
+ response.removeListener("error", onError);
+ resolve3(isClosed);
+ }
+ function onDrain() {
+ finish(false);
+ }
+ function onClose() {
+ finish(true);
+ }
+ function onError() {
+ finish(true);
+ }
+ response.once("drain", onDrain);
+ response.once("close", onClose);
+ response.once("error", onError);
});
+ if (closed) {
+ await reader.cancel().catch(() => {
+ });
+ break;
+ }
}
}
} catch (error) {
@@ -4968,7 +4999,9 @@ function writeToServerResponse({
response.end();
}
};
- read();
+ read().catch((error) => {
+ console.error("ai writeToServerResponse read() failed:", error);
+ });
}
// src/text-stream/pipe-text-stream-to-response.ts
@@ -6501,9 +6534,19 @@ function createOutputTransformStream(output) {
@@ -6501,9 +6501,19 @@ function createOutputTransformStream(output) {
controller.enqueue({ part: chunk, partialOutput: void 0 });
return;
}
@@ -165,7 +57,7 @@ index 663875332e3f9a9bd167c25583c515876f42951b..a51514a390d22811a407edd8b703e217
const result = await output.parsePartialOutput({ text: text2 });
if (result !== void 0) {
const currentJson = JSON.stringify(result.partial);
@@ -6882,7 +6925,7 @@ var DefaultStreamTextResult = class {
@@ -6882,7 +6892,7 @@ var DefaultStreamTextResult = class {
})
);
}
+6 -6
View File
@@ -48,7 +48,7 @@ patchedDependencies:
hash: d8dc66e5ec3b9d23a876b979f493b6aa901fd2d965be54729495da5136296a42
path: patches/@hocuspocus__server@3.4.4.patch
ai@6.0.134:
hash: e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b
hash: f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9
path: patches/ai@6.0.134.patch
scimmy@1.3.5:
hash: 775d80f86830b2c5dd1a250c9802c10f8fc3da3c7898373de5aa0c23993d1673
@@ -644,10 +644,10 @@ importers:
version: 8.3.0(socket.io-adapter@2.5.4)
ai:
specifier: ^6.0.134
version: 6.0.134(patch_hash=e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b)(zod@4.3.6)
version: 6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6)
ai-sdk-ollama:
specifier: ^3.8.1
version: 3.8.1(ai@6.0.134(patch_hash=e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b)(zod@4.3.6))(zod@4.3.6)
version: 3.8.1(ai@6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6))(zod@4.3.6)
bcrypt:
specifier: ^6.0.0
version: 6.0.0
@@ -16455,17 +16455,17 @@ snapshots:
agent-base@7.1.4: {}
ai-sdk-ollama@3.8.1(ai@6.0.134(patch_hash=e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b)(zod@4.3.6))(zod@4.3.6):
ai-sdk-ollama@3.8.1(ai@6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6))(zod@4.3.6):
dependencies:
'@ai-sdk/provider': 3.0.8
'@ai-sdk/provider-utils': 4.0.21(zod@4.3.6)
ai: 6.0.134(patch_hash=e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b)(zod@4.3.6)
ai: 6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6)
jsonrepair: 3.13.3
ollama: 0.6.3
transitivePeerDependencies:
- zod
ai@6.0.134(patch_hash=e8c599b3963eb01b9ed1481683b7b795ce94137aa1a0c951917c20c8b870299b)(zod@4.3.6):
ai@6.0.134(patch_hash=f60bfc3357e01e1f3978c6c40fdd65aeb33fefaad7179cde8676465b6c5ff4d9)(zod@4.3.6):
dependencies:
'@ai-sdk/gateway': 3.0.77(zod@4.3.6)
'@ai-sdk/provider': 3.0.8