Compare commits
61 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| fbac8e6976 | |||
| ec8dd7d110 | |||
| 4423b19850 | |||
| 688cb54f26 | |||
| ec5416068b | |||
| 2c4fc565b6 | |||
| a990ebd604 | |||
| 94ca907476 | |||
| fff772cbe2 | |||
| 16c2a4b623 | |||
| 58f13a7efd | |||
| 2bb48f841f | |||
| daa96eb132 | |||
| 4c5230d677 | |||
| 50ca27c5d4 | |||
| de25c258f9 | |||
| afc50ead38 | |||
| ec932e89a3 | |||
| 490d7965a1 | |||
| 32c9361969 | |||
| c30f910d66 | |||
| 7007f6bcf9 | |||
| 4bf06c68cf | |||
| af44736fb9 | |||
| 2a4d1acfd7 | |||
| 11eb87d58a | |||
| f2c8aa70f3 | |||
| 661ea8ba07 | |||
| 059edccb64 | |||
| 693a9a350b | |||
| 79b2da686b | |||
| ed3a8f8174 | |||
| c18b4c132c | |||
| 3163f50c98 | |||
| 3e81f64415 | |||
| d36021a111 | |||
| 1f2999e5ad | |||
| c674db2b2f | |||
| 44cdb5a4c3 | |||
| 03af65dd06 | |||
| dead5fa8a8 | |||
| 5b17503df7 | |||
| 46bb55dbd1 | |||
| 8b34a428f4 | |||
| e236782260 | |||
| 12ed3a0332 | |||
| b3cc6a7ff8 | |||
| c42cb42413 | |||
| 41030b2c78 | |||
| 515a1aaf1d | |||
| 74387bb047 | |||
| 6ec2981743 | |||
| dd1fe90515 | |||
| 64566e9327 | |||
| 10a4326fbf | |||
| 68409a8ae9 | |||
| fc624f5a4b | |||
| be70bd2e8e | |||
| 69e04349a0 | |||
| 5bd5995ef0 | |||
| 575125a5dc |
@@ -463,6 +463,7 @@ Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirro
|
||||
- The editor is Tiptap; shared node/mark extensions live in `packages/editor-ext` and are imported by **both the client and the server** (collaboration, schema, `canonicalizeFootnotes`) — editor schema changes often need to be made in `editor-ext`, not just the client. Server-side markdown import/export no longer lives in `editor-ext`: it goes through the canonical converter (#345, see below). The ProseMirror↔Markdown converter and its Docmost schema mirror now live in a SINGLE package, `@docmost/prosemirror-markdown` (#293), consumed by `mcp`, `git-sync`, `apps/server` (#345), and `apps/client` (#347) — do NOT reintroduce a per-package copy. The client uses the package's `browser` entry (`@docmost/prosemirror-markdown/browser`): markdown paste (`markdown-clipboard.ts`), copy-as-markdown, and AI-chat rendering now all go through the canonical converter, so the hand-written `marked`/`turndown` markdown layer that used to live in `editor-ext` was deleted (#347). The browser entry runs the HTML→DOM stage on the native `DOMParser`, so jsdom stays out of the client bundle. `editor-ext` is the upstream source of the Tiptap schema; the package's `docmost-schema.ts` mirrors it and a serializer-contract test (`packages/prosemirror-markdown/test/serializer-contract.test.ts`) guards the boundary (every schema node must have a converter case), so a drift surfaces as a failing test rather than silent divergence. For the converter's property-testing and counterexample→fixture process (P1–P4 invariants, the `PROPERTY_SEED`/`PROPERTY_NUM_RUNS` knobs, and the nightly fuzz workflow), see `packages/prosemirror-markdown/README.md`.
|
||||
- API access goes through `apps/client/src/lib/api-client.ts` (axios). The `@` alias maps to `apps/client/src`.
|
||||
- Runtime config is injected at build time by `vite.config.ts` via `define` (`APP_URL`, `COLLAB_URL`, `APP_VERSION`, …) — these come from the root `.env`, not from `import.meta.env`.
|
||||
- The build also emits `client/dist/version.json` (`{"version": …}`) from a small `vite.config.ts` plugin using the **same** `appVersion` that feeds `define.APP_VERSION`, so the file and the baked-in bundle version are identical by construction. The server reads it at startup (`ws.gateway.ts` via `readClientBuildVersion`/`resolveClientDistPath`) and announces it to each socket on connect (`app-version` event) so a tab left open across a redeploy can guard-reload before hitting a stale chunk (version-coherence). No runtime env / Dockerfile change — the file already ships in `client/dist`; missing/empty file ⇒ feature inert.
|
||||
|
||||
## Conventions
|
||||
|
||||
|
||||
@@ -142,6 +142,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
snapshots switched from a fixed interval to a trailing idle-flush with a
|
||||
max-wait ceiling, and a boundary snapshot is pinned whenever the editing source
|
||||
changes (e.g. a person's edits followed by the AI agent). (#370)
|
||||
- **Open tabs pick up a new deploy on their own.** After the server is
|
||||
redeployed while a tab is left open for hours, the tab now learns the new
|
||||
build version over the existing WebSocket (announced per-connect, so a natural
|
||||
reconnect delivers it) and shows a "A new version is available" banner with an
|
||||
Update button. To avoid dropping a half-written comment or form, the tab is
|
||||
not reloaded when you merely switch away from it; instead it auto-reloads at
|
||||
the next safe point — the next in-app navigation (or immediately if you click
|
||||
Update) — before it can hit a stale lazy-loaded chunk. At most one automatic
|
||||
reload happens per 5-minute window, shared with the existing chunk-load
|
||||
recovery, so a permanent version skew degrades to the banner rather than a
|
||||
reload loop while a second deploy in the same tab still recovers. When the
|
||||
build carries no version info the feature stays inert. (#481)
|
||||
|
||||
- **Place several images side by side in a row.** A new "Inline (side by
|
||||
side)" alignment mode in the image bubble menu renders consecutive inline
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"A new version is available": "A new version is available",
|
||||
"Account": "Account",
|
||||
"Active": "Active",
|
||||
"Add": "Add",
|
||||
@@ -239,6 +240,8 @@
|
||||
"Comment re-opened successfully": "Comment re-opened successfully",
|
||||
"Comment unresolved successfully": "Comment unresolved successfully",
|
||||
"Failed to resolve comment": "Failed to resolve comment",
|
||||
"Failed to re-open comment": "Failed to re-open comment",
|
||||
"Comment no longer exists": "Comment no longer exists",
|
||||
"Resolve comment": "Resolve comment",
|
||||
"Unresolve comment": "Unresolve comment",
|
||||
"Resolve Comment Thread": "Resolve Comment Thread",
|
||||
@@ -1427,5 +1430,20 @@
|
||||
"Boundary": "Boundary",
|
||||
"Autosave": "Autosave",
|
||||
"Only versions": "Only versions",
|
||||
"No saved versions yet.": "No saved versions yet."
|
||||
"No saved versions yet.": "No saved versions yet.",
|
||||
"Time worked on this article": "Time worked on this article",
|
||||
"Show time worked on this page": "Show time worked on this page",
|
||||
"Estimated time worked (inactivity gap {{gap}} min)": "Estimated time worked (inactivity gap {{gap}} min)",
|
||||
"Estimate · timezone {{tz}} · inactivity gap {{gap}} min": "Estimate · timezone {{tz}} · inactivity gap {{gap}} min",
|
||||
"No editing activity recorded yet.": "No editing activity recorded yet.",
|
||||
"× {{count}} days without edits": "× {{count}} days without edits",
|
||||
"agent: {{value}}": "agent: {{value}}",
|
||||
"Work": "Work",
|
||||
"Agent": "Agent",
|
||||
"≈ {{hours}}h {{minutes}}m": "≈ {{hours}}h {{minutes}}m",
|
||||
"≈ {{hours}}h": "≈ {{hours}}h",
|
||||
"≈ {{minutes}}m": "≈ {{minutes}}m",
|
||||
"{{hours}}h {{minutes}}m": "{{hours}}h {{minutes}}m",
|
||||
"{{hours}}h": "{{hours}}h",
|
||||
"{{minutes}}m": "{{minutes}}m"
|
||||
}
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"A new version is available": "Доступна новая версия",
|
||||
"Account": "Аккаунт",
|
||||
"Active": "Активный",
|
||||
"Add": "Добавить",
|
||||
@@ -239,6 +240,8 @@
|
||||
"Comment re-opened successfully": "Комментарий успешно открыт повторно",
|
||||
"Comment unresolved successfully": "Комментарий успешно переведён в нерешённые",
|
||||
"Failed to resolve comment": "Не удалось разрешить комментарий",
|
||||
"Failed to re-open comment": "Не удалось переоткрыть комментарий",
|
||||
"Comment no longer exists": "Комментарий больше не существует",
|
||||
"Resolve comment": "Решить комментарий",
|
||||
"Unresolve comment": "Снять статус решённого с комментария",
|
||||
"Resolve Comment Thread": "Решить ветку комментариев",
|
||||
@@ -1442,5 +1445,20 @@
|
||||
"Boundary": "Граница",
|
||||
"Autosave": "Автосейв",
|
||||
"Only versions": "Только версии",
|
||||
"No saved versions yet.": "Пока нет сохранённых версий."
|
||||
"No saved versions yet.": "Пока нет сохранённых версий.",
|
||||
"Time worked on this article": "Время работы над статьёй",
|
||||
"Show time worked on this page": "Показать время работы над страницей",
|
||||
"Estimated time worked (inactivity gap {{gap}} min)": "Оценка времени работы (порог паузы {{gap}} мин)",
|
||||
"Estimate · timezone {{tz}} · inactivity gap {{gap}} min": "Оценка · таймзона {{tz}} · порог паузы {{gap}} мин",
|
||||
"No editing activity recorded yet.": "Правок пока нет.",
|
||||
"× {{count}} days without edits": "× {{count}} дн. без правок",
|
||||
"agent: {{value}}": "агент: {{value}}",
|
||||
"Work": "Работа",
|
||||
"Agent": "Агент",
|
||||
"≈ {{hours}}h {{minutes}}m": "≈ {{hours}} ч {{minutes}} мин",
|
||||
"≈ {{hours}}h": "≈ {{hours}} ч",
|
||||
"≈ {{minutes}}m": "≈ {{minutes}} мин",
|
||||
"{{hours}}h {{minutes}}m": "{{hours}} ч {{minutes}} м",
|
||||
"{{hours}}h": "{{hours}} ч",
|
||||
"{{minutes}}m": "{{minutes}} м"
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { isChunkLoadError, shouldAutoReload } from "./chunk-load-error-boundary";
|
||||
import { isChunkLoadError } from "./chunk-load-error-boundary";
|
||||
|
||||
// The detector decides whether a caught render error is a stale-deploy chunk-404
|
||||
// (→ auto-reload to fetch the new manifest) vs a genuine app error (→ generic
|
||||
@@ -35,31 +35,3 @@ describe("isChunkLoadError", () => {
|
||||
expect(isChunkLoadError(err)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
// The window gate replaces the old one-shot flag: it must permit recovery across
|
||||
// several deploys in one tab (each > window apart) while still stopping an infinite
|
||||
// reload loop when a lazy chunk is permanently broken (a second failure < window).
|
||||
describe("shouldAutoReload", () => {
|
||||
const WINDOW = 5 * 60 * 1000;
|
||||
const NOW = 1_000_000_000_000;
|
||||
|
||||
it("allows a reload when we have never auto-reloaded", () => {
|
||||
expect(shouldAutoReload(NOW, null, WINDOW)).toBe(true);
|
||||
});
|
||||
|
||||
it("allows a reload when the last one was 6 minutes ago (outside the window)", () => {
|
||||
expect(shouldAutoReload(NOW, NOW - 6 * 60 * 1000, WINDOW)).toBe(true);
|
||||
});
|
||||
|
||||
it("blocks a reload when the last one was 1 minute ago (inside the window)", () => {
|
||||
expect(shouldAutoReload(NOW, NOW - 1 * 60 * 1000, WINDOW)).toBe(false);
|
||||
});
|
||||
|
||||
it("blocks a reload exactly at the window boundary (not strictly older)", () => {
|
||||
expect(shouldAutoReload(NOW, NOW - WINDOW, WINDOW)).toBe(false);
|
||||
});
|
||||
|
||||
it("allows a reload when the stored timestamp is unparseable (NaN)", () => {
|
||||
expect(shouldAutoReload(NOW, NaN, WINDOW)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,26 +1,11 @@
|
||||
import { ReactNode } from "react";
|
||||
import { ErrorBoundary } from "react-error-boundary";
|
||||
import { Button, Center, Stack, Text } from "@mantine/core";
|
||||
|
||||
// sessionStorage key holding the epoch-ms timestamp of the last automatic reload.
|
||||
const RELOAD_AT_KEY = "chunk-reload-at";
|
||||
// Allow at most one automatic reload per this window. A stale-deploy 404 is cured
|
||||
// by a single reload, so anything inside the window is treated as a reload loop
|
||||
// (permanently-broken chunk) and falls through to the manual UI. A window (rather
|
||||
// than a one-shot flag) lets a SECOND deploy in the same tab's lifetime recover too.
|
||||
const RELOAD_WINDOW_MS = 5 * 60 * 1000;
|
||||
|
||||
// Pure window decision, unit-tested in isolation: auto-reload only if we have never
|
||||
// auto-reloaded (lastReloadAt null/NaN) or the last one was strictly older than the
|
||||
// window. Anything inside the window is suppressed to break an infinite reload loop.
|
||||
export function shouldAutoReload(
|
||||
now: number,
|
||||
lastReloadAt: number | null,
|
||||
windowMs: number,
|
||||
): boolean {
|
||||
if (lastReloadAt === null || Number.isNaN(lastReloadAt)) return true;
|
||||
return now - lastReloadAt > windowMs;
|
||||
}
|
||||
import {
|
||||
hasAutoReloaded,
|
||||
markAutoReloaded,
|
||||
recordReloadBreadcrumb,
|
||||
} from "@/lib/reload-guard";
|
||||
|
||||
// Heuristic detection of a failed dynamic import. Since the code-splitting work,
|
||||
// every route (plus Aside / AiChatWindow) is React.lazy: when a new deploy
|
||||
@@ -39,24 +24,26 @@ export function isChunkLoadError(error: unknown): boolean {
|
||||
);
|
||||
}
|
||||
|
||||
function handleError(error: unknown) {
|
||||
// Exported for tests: the reactive chunk-load reload decision, so the shared
|
||||
// window budget (invariant: ≤1 auto-reload per window across this path AND the
|
||||
// proactive version-coherence path) can be exercised against the real guard.
|
||||
export function handleError(error: unknown) {
|
||||
if (!isChunkLoadError(error)) return;
|
||||
// A stale-chunk 404 is cured by a full reload that re-fetches index.html and
|
||||
// the new chunk manifest. Auto-reload at most once per RELOAD_WINDOW_MS: this
|
||||
// recovers across multiple deploys in a single tab's lifetime, yet a
|
||||
// permanently-broken lazy chunk (which would loop) is stopped after the first
|
||||
// reload and falls through to the manual recovery UI below.
|
||||
try {
|
||||
const raw = sessionStorage.getItem(RELOAD_AT_KEY);
|
||||
const lastReloadAt = raw === null ? null : Number.parseInt(raw, 10);
|
||||
const now = Date.now();
|
||||
if (!shouldAutoReload(now, lastReloadAt, RELOAD_WINDOW_MS)) return;
|
||||
sessionStorage.setItem(RELOAD_AT_KEY, String(now));
|
||||
} catch {
|
||||
// sessionStorage unavailable (private mode / disabled): skip the automatic
|
||||
// reload rather than risk an unguarded loop; the fallback UI still recovers.
|
||||
return;
|
||||
}
|
||||
// the new chunk manifest. Auto-reload at most once per window via the SHARED
|
||||
// window-based reload guard (see @/lib/reload-guard — the same budget the
|
||||
// proactive version-coherence path consumes, so a mismatch that arrives on
|
||||
// both paths reloads at most once per window across BOTH). This recovers
|
||||
// across multiple deploys in a single tab's lifetime, yet a permanently-broken
|
||||
// lazy chunk (which would loop) is stopped after the first reload and falls
|
||||
// through to the manual recovery UI below. If the shared budget is already
|
||||
// spent this window, or the stamp write fails (storage unavailable), we return
|
||||
// without reloading rather than risk a loop.
|
||||
if (hasAutoReloaded()) return;
|
||||
if (!markAutoReloaded()) return;
|
||||
// Trace before the reload clears the console (same diagnostic breadcrumb the
|
||||
// proactive version-coherence path writes, tagged with this path).
|
||||
recordReloadBreadcrumb({ path: "chunk-boundary" });
|
||||
window.location.reload();
|
||||
}
|
||||
|
||||
|
||||
@@ -1254,4 +1254,88 @@ describe("ChatThread — live reconnect + stalled", () => {
|
||||
});
|
||||
expect(onResumeFallback).toHaveBeenCalledWith(false); // POLL_IDLE_CAP -> idle -> disarm
|
||||
});
|
||||
|
||||
it("#541: getRun HANGS on a live disconnect — the timeout race still enters reconnect (no silent freeze in `streaming`)", async () => {
|
||||
// MUTATION-VERIFY: revert the race to a bare `getRun(cid).then/.catch` and this
|
||||
// reddens — a HUNG (never-settling, NOT rejected) getRun leaves the FSM stuck in
|
||||
// `streaming` with no reconnect banner and no poll (the axios client sets no
|
||||
// request timeout, and the stalled-idle cap only arms AFTER reconnecting/polling).
|
||||
renderLive();
|
||||
h.state.getRun.mockReset();
|
||||
h.state.getRun.mockReturnValue(new Promise(() => {})); // getRun HANGS forever
|
||||
await disconnect(); // live partial = liveMsg (id "a2")
|
||||
expect(h.state.getRun).toHaveBeenCalledWith("c1");
|
||||
// BEFORE the bound fires the FSM is still in the live turn — the very freeze bug.
|
||||
expect(screen.queryByText(/reconnecting/i)).toBeNull();
|
||||
// The recovery-start bound fires -> the SAME fallback as the reject path.
|
||||
act(() => {
|
||||
vi.advanceTimersByTime(4_000);
|
||||
});
|
||||
expect(screen.getByText(/reconnecting/i)).toBeTruthy();
|
||||
// The live partial (a2) was DROPPED from the store (replay-from-start, no stale
|
||||
// tail-apply base). Mirrors the getRun-REJECT test's inspection.
|
||||
const removedLivePartial = (
|
||||
h.state.setMessages as unknown as {
|
||||
mock: { calls: [unknown][] };
|
||||
}
|
||||
).mock.calls.some(([updater]) => {
|
||||
if (typeof updater !== "function") return false;
|
||||
const out = (updater as (p: { id: string }[]) => { id: string }[])([
|
||||
{ id: "a2" },
|
||||
{ id: "u1" },
|
||||
]);
|
||||
return !out.some((m) => m.id === "a2");
|
||||
});
|
||||
expect(removedLivePartial).toBe(true);
|
||||
// ...and the anchor was NULLED -> replay-from-start (no ?anchor=&n= over the partial).
|
||||
advanceToAttempt(1);
|
||||
expect(h.state.resumeStream).toHaveBeenCalledTimes(1);
|
||||
expect(h.state.transport!.prepareReconnectToStreamRequest!().api).toBe(
|
||||
"/api/ai-chat/runs/c1/stream",
|
||||
);
|
||||
});
|
||||
|
||||
it("#541: a getRun resolve AFTER the timeout already fired is IGNORED (no double reconnect, no stale re-seed)", async () => {
|
||||
// The timeout wins first and enters the ladder via replay-from-start. When the
|
||||
// hung getRun FINALLY answers, its late `.then` must be a full no-op: it must not
|
||||
// re-seed the store from the (now stale) persisted row, must not re-set the
|
||||
// anchor, and must not re-enter reconnect. The local `settled` flag makes the
|
||||
// resolve/reject/timeout branches mutually exclusive.
|
||||
// MUTATION-VERIFY: drop the `settled` guard (let the late `.then` run) and the
|
||||
// late re-seed re-sets the anchor (URL regains ?anchor=a2&n=3) + calls setMessages.
|
||||
renderLive();
|
||||
let resolveGetRun!: (v: unknown) => void;
|
||||
h.state.getRun.mockReset();
|
||||
h.state.getRun.mockReturnValue(
|
||||
new Promise((r) => {
|
||||
resolveGetRun = r;
|
||||
}),
|
||||
);
|
||||
await disconnect();
|
||||
act(() => {
|
||||
vi.advanceTimersByTime(4_000); // bound fires -> replay-from-start, reconnecting
|
||||
});
|
||||
expect(screen.getByText(/reconnecting/i)).toBeTruthy();
|
||||
advanceToAttempt(1);
|
||||
expect(h.state.resumeStream).toHaveBeenCalledTimes(1);
|
||||
// Anchor is null -> replay-from-start URL (the pre-condition the late resolve must
|
||||
// not undo).
|
||||
expect(h.state.transport!.prepareReconnectToStreamRequest!().api).toBe(
|
||||
"/api/ai-chat/runs/c1/stream",
|
||||
);
|
||||
const setMessagesCallsBefore = h.state.setMessages.mock.calls.length;
|
||||
// NOW the hung getRun finally resolves with a persisted anchor (id a2, steps 3).
|
||||
await act(async () => {
|
||||
resolveGetRun(persistedAnchor());
|
||||
await Promise.resolve();
|
||||
});
|
||||
// The late resolve did NOT re-seed the store...
|
||||
expect(h.state.setMessages.mock.calls.length).toBe(setMessagesCallsBefore);
|
||||
// ...did NOT re-set the anchor (URL stays replay-from-start, no ?anchor=&n=)...
|
||||
expect(h.state.transport!.prepareReconnectToStreamRequest!().api).toBe(
|
||||
"/api/ai-chat/runs/c1/stream",
|
||||
);
|
||||
// ...and did NOT trigger a fresh reconnect attach.
|
||||
expect(h.state.resumeStream).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -81,6 +81,17 @@ const STREAM_THROTTLE_MS = 50;
|
||||
// THREAD now (the FSM owns polling->stalled); the window just polls while armed.
|
||||
const DEGRADED_POLL_IDLE_MAX_MS = 10 * 60_000;
|
||||
|
||||
// #541: the bound on the persist re-seed (getRun) wait when ENTERING the reconnect
|
||||
// ladder after a live SSE drop. The axios client (lib/api-client.ts) sets NO request
|
||||
// timeout, and the stalled-idle cap only arms AFTER the FSM enters reconnecting/
|
||||
// polling — which never happens if getRun HANGS (connection established, no response).
|
||||
// Without this bound a live drop + a hung getRun sticks the FSM in `streaming` with
|
||||
// no banner and no poll until the browser socket timeout (minutes) — the silent-freeze
|
||||
// class #497 eliminates. This is a deliberate RECOVERY-START bound (drop the live
|
||||
// partial + enter the ladder so the poll/idle-cap arms), intentionally much shorter
|
||||
// than any network socket timeout — not a network read timeout.
|
||||
const RECONNECT_RESEED_TIMEOUT_MS = 4_000;
|
||||
|
||||
/** The #487 active (non-terminal) run statuses — mirrors the server's
|
||||
* ACTIVE_RUN_STATUSES. A run-fact is "active" only for these. */
|
||||
function isActiveRunStatus(status: string | null | undefined): boolean {
|
||||
@@ -286,6 +297,9 @@ export default function ChatThread({
|
||||
// stalled inactivity cap. Cleared by the cancelReconnect effect / the cap effect.
|
||||
const reconnectTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
|
||||
const idleCapTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
|
||||
// #541: the persist re-seed (getRun) timeout race on the disconnect->reconnect
|
||||
// entry. Held in a ref so unmount (DISPOSE cleanup) clears it — no dangling timer.
|
||||
const reseedTimerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
|
||||
|
||||
// #491 tail-only: seed EVERY persisted row unchanged (no strip). The streaming
|
||||
// tail holds steps 0..N-1; the run-stream registry's tail (steps >= N) is APPENDED
|
||||
@@ -748,37 +762,68 @@ export default function ChatThread({
|
||||
anchorRef.current = null;
|
||||
};
|
||||
if (cid) {
|
||||
void getRun(cid)
|
||||
.then((res) => {
|
||||
if (!mountedRef.current) return;
|
||||
const persisted = res.message;
|
||||
if (persisted && persisted.role === "assistant") {
|
||||
anchorRef.current = {
|
||||
id: persisted.id,
|
||||
stepsPersisted: stepsPersistedOf(persisted),
|
||||
};
|
||||
// Replace the live partial with the persisted row IN PLACE by id —
|
||||
// the re-seed from persist. The attach's tail (steps >= N) then
|
||||
// appends to a store holding EXACTLY steps 0..N-1: no duplication.
|
||||
setMessages((prev) => mergeById(prev, rowToUiMessage(persisted)));
|
||||
} else {
|
||||
// No persisted assistant row (pre-first-frame break): drop the live
|
||||
// partial + replay from start (no anchor/n) so nothing is duplicated.
|
||||
dropLivePartialAndReplayFromStart();
|
||||
}
|
||||
enterReconnect(res.run?.id ?? runId);
|
||||
})
|
||||
.catch(() => {
|
||||
if (!mountedRef.current) return;
|
||||
// Persist read FAILED: we cannot re-seed from fresh persist, and a
|
||||
// stale mount-time anchor over the live partial would tail-apply
|
||||
// already-present steps -> duplication (a flaky-network blip:
|
||||
// SSE + getRun both fail, network recovers in ~1s, the registry still
|
||||
// covers from the mount frontier). Restore the removed-filter guarantee
|
||||
// instead: drop the live partial + replay from start / 204 -> poll.
|
||||
// #541: bound the persist re-seed wait with a timeout race. getRun goes
|
||||
// through the axios client, which has NO request timeout; a HUNG getRun
|
||||
// (connection open, no response) — distinct from a REJECT, which the
|
||||
// `.catch` already handles — would otherwise never let us enter the ladder,
|
||||
// leaving the FSM stuck in `streaming` with no banner and no poll until the
|
||||
// browser socket timeout. `settled` makes the three branches (resolve /
|
||||
// reject / timeout) mutually exclusive: whichever fires FIRST wins and the
|
||||
// others become no-ops. So a LATE getRun resolve AFTER the timeout is fully
|
||||
// ignored — it cannot (i) re-enter reconnect a second time, (ii) overwrite
|
||||
// the timeout's replay-from-start with a stale re-seed, or (iii) re-arm any
|
||||
// timer. On timeout we take the SAME fallback as the reject path (drop the
|
||||
// live partial + enter the ladder, so the poll and the stalled-idle cap arm).
|
||||
let settled = false;
|
||||
const finishReseed = (apply: () => void): void => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
if (reseedTimerRef.current) {
|
||||
clearTimeout(reseedTimerRef.current);
|
||||
reseedTimerRef.current = null;
|
||||
}
|
||||
if (!mountedRef.current) return;
|
||||
apply();
|
||||
};
|
||||
reseedTimerRef.current = setTimeout(() => {
|
||||
finishReseed(() => {
|
||||
dropLivePartialAndReplayFromStart();
|
||||
enterReconnect(runId);
|
||||
});
|
||||
}, RECONNECT_RESEED_TIMEOUT_MS);
|
||||
void getRun(cid)
|
||||
.then((res) => {
|
||||
finishReseed(() => {
|
||||
const persisted = res.message;
|
||||
if (persisted && persisted.role === "assistant") {
|
||||
anchorRef.current = {
|
||||
id: persisted.id,
|
||||
stepsPersisted: stepsPersistedOf(persisted),
|
||||
};
|
||||
// Replace the live partial with the persisted row IN PLACE by id —
|
||||
// the re-seed from persist. The attach's tail (steps >= N) then
|
||||
// appends to a store holding EXACTLY steps 0..N-1: no duplication.
|
||||
setMessages((prev) => mergeById(prev, rowToUiMessage(persisted)));
|
||||
} else {
|
||||
// No persisted assistant row (pre-first-frame break): drop the live
|
||||
// partial + replay from start (no anchor/n) so nothing is duplicated.
|
||||
dropLivePartialAndReplayFromStart();
|
||||
}
|
||||
enterReconnect(res.run?.id ?? runId);
|
||||
});
|
||||
})
|
||||
.catch(() => {
|
||||
finishReseed(() => {
|
||||
// Persist read FAILED: we cannot re-seed from fresh persist, and a
|
||||
// stale mount-time anchor over the live partial would tail-apply
|
||||
// already-present steps -> duplication (a flaky-network blip:
|
||||
// SSE + getRun both fail, network recovers in ~1s, the registry still
|
||||
// covers from the mount frontier). Restore the removed-filter guarantee
|
||||
// instead: drop the live partial + replay from start / 204 -> poll.
|
||||
dropLivePartialAndReplayFromStart();
|
||||
enterReconnect(runId);
|
||||
});
|
||||
});
|
||||
} else {
|
||||
dropLivePartialAndReplayFromStart();
|
||||
enterReconnect(runId);
|
||||
@@ -903,6 +948,12 @@ export default function ChatThread({
|
||||
}
|
||||
return () => {
|
||||
mountedRef.current = false;
|
||||
// #541: clear the in-flight persist re-seed timeout (not an FSM effect timer,
|
||||
// so DISPOSE does not touch it) — no dangling setTimeout after unmount.
|
||||
if (reseedTimerRef.current) {
|
||||
clearTimeout(reseedTimerRef.current);
|
||||
reseedTimerRef.current = null;
|
||||
}
|
||||
dispatch({ type: "DISPOSE" }); // aborts attach + timers, bumps epoch (I5)
|
||||
};
|
||||
// Mount-only by design; the parent remounts per chat via `key`.
|
||||
|
||||
@@ -27,11 +27,15 @@ vi.mock("@/features/space/queries/space-query.ts", () => ({
|
||||
import {
|
||||
buildChildrenByParent,
|
||||
CommentEditorWithActions,
|
||||
sortResolvedByResolvedAt,
|
||||
} from "./comment-list-with-tabs";
|
||||
|
||||
const c = (id: string, parentCommentId: string | null = null): IComment =>
|
||||
({ id, parentCommentId }) as IComment;
|
||||
|
||||
const resolvedAtComment = (id: string, resolvedAt: unknown): IComment =>
|
||||
({ id, resolvedAt }) as unknown as IComment;
|
||||
|
||||
describe("buildChildrenByParent (childrenByParent grouping)", () => {
|
||||
it("returns an empty map for undefined or empty input", () => {
|
||||
expect(buildChildrenByParent(undefined).size).toBe(0);
|
||||
@@ -71,6 +75,48 @@ describe("buildChildrenByParent (childrenByParent grouping)", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("sortResolvedByResolvedAt (Resolved tab order, #542)", () => {
|
||||
it("orders by resolvedAt DESC — newest resolve first — with ISO-STRING values", () => {
|
||||
// At runtime resolvedAt is an ISO string (axios JSON / WS subscription), so
|
||||
// the sort must coerce with new Date(...) before .getTime().
|
||||
const older = resolvedAtComment("older", "2026-07-10T10:00:00.000Z");
|
||||
const newest = resolvedAtComment("newest", "2026-07-12T10:00:00.000Z");
|
||||
const middle = resolvedAtComment("middle", "2026-07-11T10:00:00.000Z");
|
||||
|
||||
const out = sortResolvedByResolvedAt([older, newest, middle]);
|
||||
expect(out.map((x) => x.id)).toEqual(["newest", "middle", "older"]);
|
||||
});
|
||||
|
||||
it("also handles Date instances (optimistic onMutate window)", () => {
|
||||
const older = resolvedAtComment("older", new Date("2026-01-01T00:00:00Z"));
|
||||
const newer = resolvedAtComment("newer", new Date("2026-06-01T00:00:00Z"));
|
||||
expect(sortResolvedByResolvedAt([older, newer]).map((x) => x.id)).toEqual([
|
||||
"newer",
|
||||
"older",
|
||||
]);
|
||||
});
|
||||
|
||||
it("does not mutate the input array", () => {
|
||||
const a = resolvedAtComment("a", "2026-01-01T00:00:00.000Z");
|
||||
const b = resolvedAtComment("b", "2026-02-01T00:00:00.000Z");
|
||||
const input = [a, b];
|
||||
sortResolvedByResolvedAt(input);
|
||||
expect(input.map((x) => x.id)).toEqual(["a", "b"]);
|
||||
});
|
||||
|
||||
it("keeps stable order for equal resolvedAt timestamps", () => {
|
||||
const ts = "2026-03-03T03:03:03.000Z";
|
||||
const x = resolvedAtComment("x", ts);
|
||||
const y = resolvedAtComment("y", ts);
|
||||
const z = resolvedAtComment("z", ts);
|
||||
expect(sortResolvedByResolvedAt([x, y, z]).map((c) => c.id)).toEqual([
|
||||
"x",
|
||||
"y",
|
||||
"z",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
function renderReplyEditor() {
|
||||
return render(
|
||||
<MantineProvider>
|
||||
|
||||
@@ -53,6 +53,22 @@ export function buildChildrenByParent(
|
||||
return m;
|
||||
}
|
||||
|
||||
// Sort the Resolved tab by resolve time, newest first, on a COPY (never mutate
|
||||
// the react-query cache array). `resolvedAt` is typed `Date` but at runtime it
|
||||
// is an ISO STRING (from the axios-JSON onSuccess and the WS subscription) — a
|
||||
// real Date only during the optimistic onMutate window — so it MUST be coerced
|
||||
// with `new Date(...)` before `.getTime()`, or a raw `.getTime()` on the string
|
||||
// throws / yields NaN. ES2019's stable sort preserves order for equal
|
||||
// timestamps. Callers pass a list already filtered to a truthy `resolvedAt`, so
|
||||
// the non-null assertion is safe.
|
||||
// Exported for unit testing.
|
||||
export function sortResolvedByResolvedAt(resolved: IComment[]): IComment[] {
|
||||
return [...resolved].sort(
|
||||
(a, b) =>
|
||||
new Date(b.resolvedAt!).getTime() - new Date(a.resolvedAt!).getTime(),
|
||||
);
|
||||
}
|
||||
|
||||
function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
|
||||
const { t } = useTranslation();
|
||||
const { pageSlug } = useParams();
|
||||
@@ -91,7 +107,10 @@ function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
|
||||
(comment: IComment) => comment.resolvedAt,
|
||||
);
|
||||
|
||||
return { activeComments: active, resolvedComments: resolved };
|
||||
return {
|
||||
activeComments: active,
|
||||
resolvedComments: sortResolvedByResolvedAt(resolved),
|
||||
};
|
||||
}, [comments]);
|
||||
|
||||
// Index replies by their parent once, instead of an O(n^2) filter per thread.
|
||||
|
||||
@@ -0,0 +1,353 @@
|
||||
import { describe, it, expect, vi, beforeEach } from "vitest";
|
||||
import React from "react";
|
||||
import { renderHook, waitFor } from "@testing-library/react";
|
||||
import {
|
||||
QueryClient,
|
||||
QueryClientProvider,
|
||||
InfiniteData,
|
||||
} from "@tanstack/react-query";
|
||||
|
||||
/**
|
||||
* Coverage for the resolve/reopen mutation (#542): the Undo-in-toast reopen and
|
||||
* its double-click guard, the terminal 404 branch (drop from cache + clear the
|
||||
* inline mark, no rollback), and the directional error copy.
|
||||
*/
|
||||
|
||||
// A fake TipTap editor injected via the mocked pageEditorAtom, so we can assert
|
||||
// the mutation clears the inline comment mark (unsetComment / setCommentResolved).
|
||||
const editorMock = vi.hoisted(() => ({
|
||||
current: {
|
||||
isDestroyed: false,
|
||||
commands: { unsetComment: vi.fn(), setCommentResolved: vi.fn() },
|
||||
} as {
|
||||
isDestroyed: boolean;
|
||||
commands: {
|
||||
unsetComment: (id: string) => void;
|
||||
setCommentResolved: (id: string, v: boolean) => void;
|
||||
};
|
||||
} | null,
|
||||
}));
|
||||
|
||||
vi.mock("@mantine/notifications", () => ({
|
||||
notifications: { show: vi.fn(), hide: vi.fn() },
|
||||
}));
|
||||
|
||||
vi.mock("jotai", () => ({
|
||||
atom: (v: unknown) => v,
|
||||
useAtomValue: () => editorMock.current,
|
||||
}));
|
||||
|
||||
vi.mock("@/features/comment/services/comment-service", () => ({
|
||||
applySuggestion: vi.fn(),
|
||||
dismissSuggestion: vi.fn(),
|
||||
createComment: vi.fn(),
|
||||
updateComment: vi.fn(),
|
||||
deleteComment: vi.fn(),
|
||||
resolveComment: vi.fn(),
|
||||
getPageComments: vi.fn(),
|
||||
}));
|
||||
|
||||
import { notifications } from "@mantine/notifications";
|
||||
import { resolveComment } from "@/features/comment/services/comment-service";
|
||||
import {
|
||||
useResolveCommentMutation,
|
||||
RESOLVE_UNDO_AUTOCLOSE_MS,
|
||||
RQ_KEY,
|
||||
} from "@/features/comment/queries/comment-query";
|
||||
import { IComment } from "@/features/comment/types/comment.types";
|
||||
|
||||
const PAGE_ID = "page-1";
|
||||
|
||||
function seededClient(comment: IComment) {
|
||||
const queryClient = new QueryClient({
|
||||
defaultOptions: { mutations: { retry: false } },
|
||||
});
|
||||
const seed: InfiniteData<any> = {
|
||||
pageParams: [undefined],
|
||||
pages: [
|
||||
{ items: [comment], meta: { hasNextPage: false, nextCursor: null } },
|
||||
],
|
||||
};
|
||||
queryClient.setQueryData(RQ_KEY(PAGE_ID), seed);
|
||||
const wrapper = ({ children }: { children: React.ReactNode }) => (
|
||||
<QueryClientProvider client={queryClient}>{children}</QueryClientProvider>
|
||||
);
|
||||
return { queryClient, wrapper };
|
||||
}
|
||||
|
||||
function items(queryClient: QueryClient): IComment[] {
|
||||
const cache = queryClient.getQueryData(RQ_KEY(PAGE_ID)) as
|
||||
| InfiniteData<any>
|
||||
| undefined;
|
||||
return cache?.pages.flatMap((p) => p.items) ?? [];
|
||||
}
|
||||
|
||||
const comment = (over?: Partial<IComment>): IComment =>
|
||||
({
|
||||
id: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
content: "{}",
|
||||
creatorId: "u-1",
|
||||
workspaceId: "ws-1",
|
||||
createdAt: new Date(),
|
||||
resolvedAt: null,
|
||||
...over,
|
||||
}) as IComment;
|
||||
|
||||
// Pull the inline Undo button's onClick out of the success toast's message tree.
|
||||
function undoOnClickFromToast(): () => void {
|
||||
const call = vi
|
||||
.mocked(notifications.show)
|
||||
.mock.calls.map((c) => c[0])
|
||||
.find((arg: any) => arg?.autoClose === RESOLVE_UNDO_AUTOCLOSE_MS);
|
||||
expect(call).toBeTruthy();
|
||||
const message: any = (call as any).message;
|
||||
// message = Group( Text, Button ); grab the Button element's onClick.
|
||||
const children = message.props.children as any[];
|
||||
const button = children[1];
|
||||
return button.props.onClick;
|
||||
}
|
||||
|
||||
describe("useResolveCommentMutation — Undo toast (#542)", () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
editorMock.current = {
|
||||
isDestroyed: false,
|
||||
commands: { unsetComment: vi.fn(), setCommentResolved: vi.fn() },
|
||||
};
|
||||
});
|
||||
|
||||
it("resolve shows an Undo toast with autoClose=10000ms; reopen shows NO Undo", async () => {
|
||||
vi.mocked(resolveComment).mockImplementation(async (data) =>
|
||||
comment({
|
||||
resolvedAt: data.resolved ? (new Date() as any) : null,
|
||||
}),
|
||||
);
|
||||
const { wrapper } = seededClient(comment());
|
||||
const { result } = renderHook(() => useResolveCommentMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
|
||||
await result.current.mutateAsync({
|
||||
commentId: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
resolved: true,
|
||||
});
|
||||
await waitFor(() => expect(result.current.isSuccess).toBe(true));
|
||||
|
||||
const resolveToast = vi
|
||||
.mocked(notifications.show)
|
||||
.mock.calls.map((c) => c[0])
|
||||
.find((a: any) => a?.autoClose === RESOLVE_UNDO_AUTOCLOSE_MS);
|
||||
expect(resolveToast).toBeTruthy();
|
||||
expect((resolveToast as any).id).toBe("resolve-undo-c-1");
|
||||
expect((resolveToast as any).autoClose).toBe(10000);
|
||||
|
||||
// Now a reopen → plain toast, no autoClose/Undo, no id.
|
||||
vi.clearAllMocks();
|
||||
await result.current.mutateAsync({
|
||||
commentId: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
resolved: false,
|
||||
});
|
||||
await waitFor(() => expect(result.current.isSuccess).toBe(true));
|
||||
const calls = vi.mocked(notifications.show).mock.calls.map((c) => c[0]);
|
||||
expect(
|
||||
calls.some((a: any) => a?.autoClose === RESOLVE_UNDO_AUTOCLOSE_MS),
|
||||
).toBe(false);
|
||||
expect(calls).toContainEqual({ message: "Comment re-opened successfully" });
|
||||
});
|
||||
|
||||
it("double/fast Undo click fires reopen EXACTLY once (guard)", async () => {
|
||||
vi.mocked(resolveComment).mockImplementation(async (data) =>
|
||||
comment({ resolvedAt: data.resolved ? (new Date() as any) : null }),
|
||||
);
|
||||
const { wrapper } = seededClient(comment());
|
||||
const { result } = renderHook(() => useResolveCommentMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
|
||||
await result.current.mutateAsync({
|
||||
commentId: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
resolved: true,
|
||||
});
|
||||
await waitFor(() => expect(result.current.isSuccess).toBe(true));
|
||||
|
||||
const onClick = undoOnClickFromToast();
|
||||
// Fire twice synchronously (notifications.hide is not synchronous).
|
||||
onClick();
|
||||
onClick();
|
||||
|
||||
await waitFor(() => {
|
||||
const reopenCalls = vi
|
||||
.mocked(resolveComment)
|
||||
.mock.calls.filter(([d]) => d.resolved === false);
|
||||
expect(reopenCalls).toHaveLength(1);
|
||||
});
|
||||
// The mark was cleared once via setCommentResolved(id, false).
|
||||
expect(editorMock.current!.commands.setCommentResolved).toHaveBeenCalledWith(
|
||||
"c-1",
|
||||
false,
|
||||
);
|
||||
// The toast was hidden.
|
||||
expect(notifications.hide).toHaveBeenCalledWith("resolve-undo-c-1");
|
||||
});
|
||||
|
||||
it("404 → drops the comment from cache, clears the inline mark, no rollback, no Undo", async () => {
|
||||
vi.mocked(resolveComment).mockRejectedValue({ response: { status: 404 } });
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
const { result } = renderHook(() => useResolveCommentMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
|
||||
await result.current
|
||||
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID, resolved: true })
|
||||
.catch(() => undefined);
|
||||
await waitFor(() => expect(result.current.isError).toBe(true));
|
||||
|
||||
// Removed from cache (NOT rolled back to a phantom).
|
||||
expect(items(queryClient)).toHaveLength(0);
|
||||
// Inline mark cleared via unsetComment (mandatory — no panel row left to do it).
|
||||
expect(editorMock.current!.commands.unsetComment).toHaveBeenCalledWith(
|
||||
"c-1",
|
||||
);
|
||||
// Neutral message, red, and crucially NOT the success copy and NO Undo toast.
|
||||
expect(notifications.show).toHaveBeenCalledWith({
|
||||
message: "Comment no longer exists",
|
||||
color: "red",
|
||||
});
|
||||
const calls = vi.mocked(notifications.show).mock.calls.map((c) => c[0]);
|
||||
expect(
|
||||
calls.some((a: any) => a?.autoClose === RESOLVE_UNDO_AUTOCLOSE_MS),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("404 does not crash when the editor is gone (read-only / panel closed)", async () => {
|
||||
editorMock.current = null;
|
||||
vi.mocked(resolveComment).mockRejectedValue({ response: { status: 404 } });
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
const { result } = renderHook(() => useResolveCommentMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
|
||||
await result.current
|
||||
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID, resolved: true })
|
||||
.catch(() => undefined);
|
||||
await waitFor(() => expect(result.current.isError).toBe(true));
|
||||
expect(items(queryClient)).toHaveLength(0);
|
||||
expect(notifications.show).toHaveBeenCalledWith({
|
||||
message: "Comment no longer exists",
|
||||
color: "red",
|
||||
});
|
||||
});
|
||||
|
||||
it("non-404 error on REOPEN shows 'Failed to re-open comment' and rolls back", async () => {
|
||||
vi.mocked(resolveComment).mockRejectedValue({ response: { status: 500 } });
|
||||
// Seed a RESOLVED comment (the reopen target).
|
||||
const resolved = comment({ resolvedAt: new Date() as any });
|
||||
const { queryClient, wrapper } = seededClient(resolved);
|
||||
const { result } = renderHook(() => useResolveCommentMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
|
||||
await result.current
|
||||
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID, resolved: false })
|
||||
.catch(() => undefined);
|
||||
await waitFor(() => expect(result.current.isError).toBe(true));
|
||||
|
||||
expect(notifications.show).toHaveBeenCalledWith({
|
||||
message: "Failed to re-open comment",
|
||||
color: "red",
|
||||
});
|
||||
expect(notifications.show).not.toHaveBeenCalledWith(
|
||||
expect.objectContaining({ message: "Failed to resolve comment" }),
|
||||
);
|
||||
// Rolled back: the comment is still present and still resolved.
|
||||
expect(items(queryClient)).toHaveLength(1);
|
||||
expect(items(queryClient)[0].resolvedAt).toBeTruthy();
|
||||
});
|
||||
|
||||
it("reopen via Undo FAILS (non-404) → inline mark is NOT left cleared (doc↔panel stay consistent)", async () => {
|
||||
// First resolve succeeds → produces the Undo toast (no mark change on resolve).
|
||||
vi.mocked(resolveComment).mockResolvedValueOnce(
|
||||
comment({ resolvedAt: new Date() as any }),
|
||||
);
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
const { result } = renderHook(() => useResolveCommentMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
|
||||
await result.current.mutateAsync({
|
||||
commentId: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
resolved: true,
|
||||
});
|
||||
await waitFor(() => expect(result.current.isSuccess).toBe(true));
|
||||
|
||||
// Now the reopen fired by Undo fails with a 500.
|
||||
vi.mocked(resolveComment).mockRejectedValue({ response: { status: 500 } });
|
||||
const onClick = undoOnClickFromToast();
|
||||
onClick();
|
||||
|
||||
await waitFor(() => expect(result.current.isError).toBe(true));
|
||||
|
||||
// Core F1 guarantee: the mark-clear now lives in the reopen onSuccess, so a
|
||||
// FAILED reopen must never flip the inline mark to unresolved — otherwise the
|
||||
// doc would show an active highlight the panel still treats as resolved and
|
||||
// the collab mark would diverge with nothing committed on the server.
|
||||
expect(
|
||||
editorMock.current!.commands.setCommentResolved,
|
||||
).not.toHaveBeenCalledWith("c-1", false);
|
||||
// Cache rolled back: the comment stays resolved and present.
|
||||
expect(items(queryClient)).toHaveLength(1);
|
||||
expect(items(queryClient)[0].resolvedAt).toBeTruthy();
|
||||
});
|
||||
|
||||
it("reopen success with a null editorRef degrades gracefully (no throw, no-op)", async () => {
|
||||
// Read-only view / panel closed: pageEditorAtom is null on the success path.
|
||||
editorMock.current = null;
|
||||
vi.mocked(resolveComment).mockResolvedValue(comment({ resolvedAt: null }));
|
||||
const resolved = comment({ resolvedAt: new Date() as any });
|
||||
const { queryClient, wrapper } = seededClient(resolved);
|
||||
const { result } = renderHook(() => useResolveCommentMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
|
||||
await result.current.mutateAsync({
|
||||
commentId: "c-1",
|
||||
pageId: PAGE_ID,
|
||||
resolved: false,
|
||||
});
|
||||
await waitFor(() => expect(result.current.isSuccess).toBe(true));
|
||||
|
||||
// No crash from the reopen mark-clear; the plain reopen toast is still shown.
|
||||
expect(notifications.show).toHaveBeenCalledWith({
|
||||
message: "Comment re-opened successfully",
|
||||
});
|
||||
// Cache updated to reopened (resolvedAt cleared by the server payload).
|
||||
expect(items(queryClient)).toHaveLength(1);
|
||||
expect(items(queryClient)[0].resolvedAt).toBeFalsy();
|
||||
});
|
||||
|
||||
it("non-404 error on RESOLVE shows 'Failed to resolve comment' and rolls back", async () => {
|
||||
vi.mocked(resolveComment).mockRejectedValue({ response: { status: 500 } });
|
||||
const { queryClient, wrapper } = seededClient(comment());
|
||||
const { result } = renderHook(() => useResolveCommentMutation(), {
|
||||
wrapper,
|
||||
});
|
||||
|
||||
await result.current
|
||||
.mutateAsync({ commentId: "c-1", pageId: PAGE_ID, resolved: true })
|
||||
.catch(() => undefined);
|
||||
await waitFor(() => expect(result.current.isError).toBe(true));
|
||||
|
||||
expect(notifications.show).toHaveBeenCalledWith({
|
||||
message: "Failed to resolve comment",
|
||||
color: "red",
|
||||
});
|
||||
// Rolled back to open (previousCache), still present.
|
||||
expect(items(queryClient)).toHaveLength(1);
|
||||
expect(items(queryClient)[0].resolvedAt).toBeFalsy();
|
||||
});
|
||||
});
|
||||
@@ -20,12 +20,19 @@ import {
|
||||
ISuggestionOutcome,
|
||||
} from "@/features/comment/types/comment.types";
|
||||
import { notifications } from "@mantine/notifications";
|
||||
import { Button, Group, Text } from "@mantine/core";
|
||||
import { IPagination } from "@/lib/types.ts";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import { useEffect, useMemo } from "react";
|
||||
import React, { useEffect, useMemo, useRef } from "react";
|
||||
import { useAtomValue } from "jotai";
|
||||
import { pageEditorAtom } from "@/features/editor/atoms/editor-atoms";
|
||||
|
||||
export const RQ_KEY = (pageId: string) => ["comments", pageId];
|
||||
|
||||
// How long the resolve success toast (with its inline Undo) stays up before it
|
||||
// auto-closes. Policy constant — no env override.
|
||||
export const RESOLVE_UNDO_AUTOCLOSE_MS = 10000;
|
||||
|
||||
export function useCommentsQuery(params: ICommentParams) {
|
||||
const query = useInfiniteQuery({
|
||||
queryKey: RQ_KEY(params.pageId),
|
||||
@@ -376,7 +383,25 @@ export function useResolveCommentMutation() {
|
||||
const queryClient = useQueryClient();
|
||||
const { t } = useTranslation();
|
||||
|
||||
return useMutation({
|
||||
// Keep the live editor in a ref: the toast's Undo (and the 404 branch) must
|
||||
// clear the inline comment mark AFTER the originating CommentListItem has
|
||||
// unmounted (resolving pulls the comment out of the Open list, so its item is
|
||||
// already gone by the time the 10s toast is clicked). In read-only view
|
||||
// pageEditorAtom is null and the mark converges via the server's
|
||||
// COMMENT_MARK_UPDATE job instead.
|
||||
const editor = useAtomValue(pageEditorAtom);
|
||||
const editorRef = useRef(editor);
|
||||
editorRef.current = editor;
|
||||
|
||||
// Self-reference the mutation so the toast's Undo can re-invoke it (reopen)
|
||||
// long after the triggering component unmounted. Declared BEFORE useMutation
|
||||
// and assigned AFTER; the onClick reads mutationRef.current at CALL time, not
|
||||
// definition time, so there is no initialization cycle.
|
||||
const mutationRef = useRef<{
|
||||
mutate: (vars: IResolveComment) => void;
|
||||
} | null>(null);
|
||||
|
||||
const mutation = useMutation({
|
||||
mutationFn: (data: IResolveComment) => resolveComment(data),
|
||||
onMutate: async (variables) => {
|
||||
await queryClient.cancelQueries({ queryKey: RQ_KEY(variables.pageId) });
|
||||
@@ -401,7 +426,39 @@ export function useResolveCommentMutation() {
|
||||
|
||||
return { previousCache };
|
||||
},
|
||||
onError: (_err, variables, context) => {
|
||||
onError: (err: any, variables, context) => {
|
||||
// Terminal 404: the comment was really deleted (missing comment or deleted
|
||||
// page — access denial is 403, resolve is idempotent so no 400). Do NOT
|
||||
// roll back (that would resurrect a phantom row in Resolved); instead drop
|
||||
// it from the cache and clear its now-orphaned inline mark. Mirrors
|
||||
// handleDeleteComment and the dismiss-mutation 404 branch.
|
||||
if (err?.response?.status === 404) {
|
||||
const cache = queryClient.getQueryData(RQ_KEY(variables.pageId)) as
|
||||
| InfiniteData<IPagination<IComment>>
|
||||
| undefined;
|
||||
if (cache) {
|
||||
queryClient.setQueryData(
|
||||
RQ_KEY(variables.pageId),
|
||||
removeCommentFromCache(cache, variables.commentId),
|
||||
);
|
||||
}
|
||||
const ed = editorRef.current;
|
||||
if (ed && !ed.isDestroyed) {
|
||||
try {
|
||||
ed.commands.unsetComment(variables.commentId);
|
||||
} catch {
|
||||
/* editor gone / mark already removed */
|
||||
}
|
||||
}
|
||||
notifications.show({
|
||||
message: t("Comment no longer exists"),
|
||||
color: "red",
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
// Generic failure: roll back the optimistic update and show a DIRECTIONAL
|
||||
// error (resolve vs. reopen), not always "resolve".
|
||||
if (context?.previousCache) {
|
||||
queryClient.setQueryData(
|
||||
RQ_KEY(variables.pageId),
|
||||
@@ -409,7 +466,9 @@ export function useResolveCommentMutation() {
|
||||
);
|
||||
}
|
||||
notifications.show({
|
||||
message: t("Failed to resolve comment"),
|
||||
message: variables.resolved
|
||||
? t("Failed to resolve comment")
|
||||
: t("Failed to re-open comment"),
|
||||
color: "red",
|
||||
});
|
||||
},
|
||||
@@ -430,11 +489,72 @@ export function useResolveCommentMutation() {
|
||||
);
|
||||
}
|
||||
|
||||
// Reopen keeps the plain toast without an Undo.
|
||||
if (!variables.resolved) {
|
||||
// Clear the inline mark ONLY after the server confirms the reopen, so a
|
||||
// failed reopen never leaves an active highlight the panel still treats
|
||||
// as resolved. Mirrors the 404 branch's editor-liveness guard/try-catch.
|
||||
// The button-triggered reopen already set the mark, so this is an
|
||||
// idempotent no-op there.
|
||||
const ed = editorRef.current;
|
||||
if (ed && !ed.isDestroyed) {
|
||||
try {
|
||||
ed.commands.setCommentResolved(variables.commentId, false);
|
||||
} catch {
|
||||
/* editor gone — server COMMENT_MARK_UPDATE converges it */
|
||||
}
|
||||
}
|
||||
notifications.show({ message: t("Comment re-opened successfully") });
|
||||
return;
|
||||
}
|
||||
|
||||
// Resolve: attach an inline Undo (reopen) to the success toast. Built with
|
||||
// React.createElement because this is a .ts module (no JSX).
|
||||
const { commentId, pageId } = variables;
|
||||
const notificationId = `resolve-undo-${commentId}`;
|
||||
// Double-click guard: notifications.hide is NOT synchronous, so the button
|
||||
// stays clickable for a frame or two — without this a fast double-click
|
||||
// would fire reopen twice.
|
||||
let done = false;
|
||||
notifications.show({
|
||||
message: variables.resolved
|
||||
? t("Comment resolved successfully")
|
||||
: t("Comment re-opened successfully"),
|
||||
id: notificationId,
|
||||
autoClose: RESOLVE_UNDO_AUTOCLOSE_MS,
|
||||
message: React.createElement(
|
||||
Group,
|
||||
{ justify: "space-between", wrap: "nowrap", gap: "md" },
|
||||
React.createElement(
|
||||
Text,
|
||||
{ size: "sm" },
|
||||
t("Comment resolved successfully"),
|
||||
),
|
||||
React.createElement(
|
||||
Button,
|
||||
{
|
||||
variant: "subtle",
|
||||
size: "compact-sm",
|
||||
onClick: () => {
|
||||
if (done) return;
|
||||
done = true;
|
||||
// Reopen via the SAME mutation (read at click time — the
|
||||
// originating item is already unmounted).
|
||||
mutationRef.current?.mutate({
|
||||
commentId,
|
||||
pageId,
|
||||
resolved: false,
|
||||
});
|
||||
// The inline mark is cleared in the reopen mutation's onSuccess
|
||||
// (bound to server confirmation), NOT here — clearing it eagerly
|
||||
// would desync the doc from the panel if reopen then fails.
|
||||
notifications.hide(notificationId);
|
||||
},
|
||||
},
|
||||
t("Undo"),
|
||||
),
|
||||
),
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
mutationRef.current = mutation;
|
||||
return mutation;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import {
|
||||
formatHeadline,
|
||||
formatDayTotal,
|
||||
formatGapMinutes,
|
||||
} from "./format-work-time";
|
||||
|
||||
const MIN = 60 * 1000;
|
||||
// Fake translator: renders the key with {{tokens}} substituted, so the tests
|
||||
// assert the rounding + branch selection without depending on the i18n catalogue.
|
||||
const t = (key: string, opts?: Record<string, unknown>) =>
|
||||
key.replace(/\{\{(\w+)\}\}/g, (_, k) => String(opts?.[k] ?? ""));
|
||||
|
||||
describe("formatHeadline", () => {
|
||||
it("prefixes ≈ and rounds to a 5-minute step", () => {
|
||||
expect(formatHeadline(4 * 60 * MIN + 27 * MIN, t)).toBe("≈ 4h 25m");
|
||||
expect(formatHeadline(90 * MIN, t)).toBe("≈ 1h 30m");
|
||||
});
|
||||
|
||||
it("shows hours only / minutes only cleanly", () => {
|
||||
expect(formatHeadline(120 * MIN, t)).toBe("≈ 2h");
|
||||
expect(formatHeadline(35 * MIN, t)).toBe("≈ 35m");
|
||||
});
|
||||
|
||||
it("floors a tiny non-zero estimate to 5m, never 0", () => {
|
||||
expect(formatHeadline(2 * MIN, t)).toBe("≈ 5m");
|
||||
});
|
||||
|
||||
it("empty string for zero (widget hidden)", () => {
|
||||
expect(formatHeadline(0, t)).toBe("");
|
||||
});
|
||||
});
|
||||
|
||||
describe("formatDayTotal", () => {
|
||||
it('renders "h m" and shows — for empty days', () => {
|
||||
expect(formatDayTotal(3 * 60 * MIN + 17 * MIN, t)).toBe("3h 17m");
|
||||
expect(formatDayTotal(0, t)).toBe("—");
|
||||
});
|
||||
});
|
||||
|
||||
describe("formatGapMinutes", () => {
|
||||
it("converts the tGap ms threshold to whole minutes", () => {
|
||||
expect(formatGapMinutes(15 * MIN)).toBe(15);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,45 @@
|
||||
// #395 — display formatting for the work-time estimate. Pure functions that take
|
||||
// a translator so ru-RU / en-US wording lives in the i18n catalogue and the
|
||||
// rounding logic stays unit-testable.
|
||||
|
||||
type Translate = (key: string, opts?: Record<string, unknown>) => string;
|
||||
|
||||
const MIN = 60 * 1000;
|
||||
|
||||
function hm(totalMinutes: number): { hours: number; minutes: number } {
|
||||
return {
|
||||
hours: Math.floor(totalMinutes / 60),
|
||||
minutes: totalMinutes % 60,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Headline number (§6.1): an ESTIMATE, so rounded to a coarse 5-minute step and
|
||||
* prefixed with "≈". A non-zero-but-tiny estimate floors to 5m rather than
|
||||
* rounding down to "0" (which would read as "no work"). Zero → empty string
|
||||
* (the caller hides the widget).
|
||||
*/
|
||||
export function formatHeadline(workMs: number, t: Translate): string {
|
||||
if (workMs <= 0) return "";
|
||||
let minutes = Math.round(workMs / MIN / 5) * 5;
|
||||
if (minutes === 0) minutes = 5;
|
||||
const { hours, minutes: m } = hm(minutes);
|
||||
if (hours > 0 && m > 0) return t("≈ {{hours}}h {{minutes}}m", { hours, minutes: m });
|
||||
if (hours > 0) return t("≈ {{hours}}h", { hours });
|
||||
return t("≈ {{minutes}}m", { minutes: m });
|
||||
}
|
||||
|
||||
/** Per-day sum (§6.2), rounded to the minute. Zero → "—". */
|
||||
export function formatDayTotal(activeMs: number, t: Translate): string {
|
||||
if (activeMs <= 0) return "—";
|
||||
const minutes = Math.max(1, Math.round(activeMs / MIN));
|
||||
const { hours, minutes: m } = hm(minutes);
|
||||
if (hours > 0 && m > 0) return t("{{hours}}h {{minutes}}m", { hours, minutes: m });
|
||||
if (hours > 0) return t("{{hours}}h", { hours });
|
||||
return t("{{minutes}}m", { minutes: m });
|
||||
}
|
||||
|
||||
/** The inactivity threshold, for the "estimate · gap = N min" caption. */
|
||||
export function formatGapMinutes(tGapMs: number): number {
|
||||
return Math.round(tGapMs / MIN);
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import { useQuery, UseQueryResult } from "@tanstack/react-query";
|
||||
import { IPageWorkTime } from "./work-time.types";
|
||||
import { getPageWorkTime, viewerTimezone } from "./work-time-service";
|
||||
|
||||
const WORK_TIME_STALE_TIME = 5 * 60 * 1000;
|
||||
|
||||
/**
|
||||
* #395 — the "time worked on this article" estimate + per-day punch-card
|
||||
* buckets. The buckets are computed server-side in the viewer's timezone (so a
|
||||
* midnight-crossing session lands on the right calendar day for the reader).
|
||||
* `enabled` is opt-in so the (cheap but non-trivial) projection query only fires
|
||||
* when the number is actually shown.
|
||||
*/
|
||||
export function usePageWorkTime(
|
||||
pageId: string,
|
||||
enabled = true,
|
||||
): UseQueryResult<IPageWorkTime, Error> {
|
||||
const tz = viewerTimezone();
|
||||
return useQuery({
|
||||
queryKey: ["page-work-time", pageId, tz],
|
||||
queryFn: () => getPageWorkTime(pageId, tz),
|
||||
enabled: enabled && !!pageId,
|
||||
staleTime: WORK_TIME_STALE_TIME,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
import { Group, Stack, Text } from "@mantine/core";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import { useMemo } from "react";
|
||||
import { IPageWorkTime, IPerDay, IDayWindow } from "./work-time.types";
|
||||
import {
|
||||
formatDayTotal,
|
||||
formatGapMinutes,
|
||||
formatHeadline,
|
||||
} from "./format-work-time";
|
||||
import classes from "./work-time.module.css";
|
||||
|
||||
const DAY_MS = 24 * 60 * 60 * 1000;
|
||||
// Collapse a run of this many (or more) consecutive edit-free days into a single
|
||||
// "× N days" separator (§6.2 long-range) — the row is still always one day.
|
||||
const EMPTY_RUN_COLLAPSE = 8;
|
||||
|
||||
type Row =
|
||||
| { type: "day"; day: IPerDay }
|
||||
| { type: "gap"; count: number };
|
||||
|
||||
function collapseEmptyRuns(perDay: IPerDay[]): Row[] {
|
||||
const rows: Row[] = [];
|
||||
let emptyRun: IPerDay[] = [];
|
||||
const flush = () => {
|
||||
if (emptyRun.length >= EMPTY_RUN_COLLAPSE) {
|
||||
rows.push({ type: "gap", count: emptyRun.length });
|
||||
} else {
|
||||
for (const d of emptyRun) rows.push({ type: "day", day: d });
|
||||
}
|
||||
emptyRun = [];
|
||||
};
|
||||
for (const d of perDay) {
|
||||
if (d.activeMs === 0 && d.agentMs === 0) {
|
||||
emptyRun.push(d);
|
||||
} else {
|
||||
flush();
|
||||
rows.push({ type: "day", day: d });
|
||||
}
|
||||
}
|
||||
flush();
|
||||
return rows;
|
||||
}
|
||||
|
||||
function dayHeading(day: number): string {
|
||||
return new Date(day).toLocaleDateString(undefined, {
|
||||
weekday: "short",
|
||||
day: "numeric",
|
||||
month: "short",
|
||||
});
|
||||
}
|
||||
|
||||
function DayTrack({
|
||||
day,
|
||||
pSingle,
|
||||
}: {
|
||||
day: IPerDay;
|
||||
pSingle: number;
|
||||
}) {
|
||||
const { t } = useTranslation();
|
||||
const ticks = [6, 12, 18];
|
||||
return (
|
||||
<div className={classes.row}>
|
||||
<span className={classes.dayLabel}>{dayHeading(day.day)}</span>
|
||||
<div className={classes.track}>
|
||||
{ticks.map((h) => (
|
||||
<div
|
||||
key={h}
|
||||
className={classes.hourTick}
|
||||
style={{ left: `${(h / 24) * 100}%` }}
|
||||
/>
|
||||
))}
|
||||
{day.windows.map((w: IDayWindow, i) => {
|
||||
const leftPct = ((w.start - day.day) / DAY_MS) * 100;
|
||||
const widthPct = ((w.end - w.start) / DAY_MS) * 100;
|
||||
const isSingle = w.end - w.start <= pSingle;
|
||||
const cls = [
|
||||
classes.window,
|
||||
w.class === "work" ? classes.windowWork : classes.windowAgent,
|
||||
isSingle ? classes.windowSingle : "",
|
||||
].join(" ");
|
||||
return (
|
||||
<div
|
||||
key={i}
|
||||
className={cls}
|
||||
style={{
|
||||
left: `${Math.max(0, Math.min(100, leftPct))}%`,
|
||||
width: `${Math.max(0, Math.min(100, widthPct))}%`,
|
||||
}}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
<span className={classes.daySum}>
|
||||
{formatDayTotal(day.activeMs, t)}
|
||||
</span>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
interface Props {
|
||||
data: IPageWorkTime;
|
||||
}
|
||||
|
||||
export default function WorkTimePunchCard({ data }: Props) {
|
||||
const { t } = useTranslation();
|
||||
const rows = useMemo(() => collapseEmptyRuns(data.perDay), [data.perDay]);
|
||||
const gapMin = formatGapMinutes(data.config.tGap);
|
||||
|
||||
if (data.workMs <= 0 && data.agentOnlyMs <= 0) {
|
||||
return (
|
||||
<Text size="sm" c="dimmed" py="md">
|
||||
{t("No editing activity recorded yet.")}
|
||||
</Text>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<Stack gap="xs">
|
||||
<Group gap="lg">
|
||||
<Text size="sm" fw={500}>
|
||||
{formatHeadline(data.workMs, t)}
|
||||
</Text>
|
||||
{data.agentOnlyMs > 0 && (
|
||||
<Text size="xs" c="dimmed">
|
||||
{t("agent: {{value}}", { value: formatHeadline(data.agentOnlyMs, t) })}
|
||||
</Text>
|
||||
)}
|
||||
</Group>
|
||||
|
||||
<Group gap="md">
|
||||
<Text size="xs" c="dimmed">
|
||||
<span
|
||||
className={`${classes.legendSwatch} ${classes.windowWork}`}
|
||||
style={{ marginRight: 4 }}
|
||||
/>
|
||||
{t("Work")}
|
||||
</Text>
|
||||
<Text size="xs" c="dimmed">
|
||||
<span
|
||||
className={`${classes.legendSwatch} ${classes.windowAgent}`}
|
||||
style={{ marginRight: 4 }}
|
||||
/>
|
||||
{t("Agent")}
|
||||
</Text>
|
||||
</Group>
|
||||
|
||||
<div>
|
||||
{rows.map((row, i) =>
|
||||
row.type === "day" ? (
|
||||
<DayTrack
|
||||
key={row.day.dayISO}
|
||||
day={row.day}
|
||||
pSingle={data.config.pSingle}
|
||||
/>
|
||||
) : (
|
||||
<div key={`gap-${i}`} className={classes.gapRow}>
|
||||
{t("× {{count}} days without edits", { count: row.count })}
|
||||
</div>
|
||||
),
|
||||
)}
|
||||
</div>
|
||||
|
||||
<Text size="xs" c="dimmed" mt="xs">
|
||||
{t("Estimate · timezone {{tz}} · inactivity gap {{gap}} min", {
|
||||
tz: data.tz,
|
||||
gap: gapMin,
|
||||
})}
|
||||
</Text>
|
||||
</Stack>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
import api from "@/lib/api-client";
|
||||
import { IPageWorkTime } from "./work-time.types";
|
||||
|
||||
/** The viewer's IANA timezone (browser locale) — the punch-card lays days out
|
||||
* in "my evenings", per §6.3/§10. Falls back to UTC if the runtime hides it. */
|
||||
export function viewerTimezone(): string {
|
||||
try {
|
||||
return Intl.DateTimeFormat().resolvedOptions().timeZone || "UTC";
|
||||
} catch {
|
||||
return "UTC";
|
||||
}
|
||||
}
|
||||
|
||||
export async function getPageWorkTime(
|
||||
pageId: string,
|
||||
tz: string,
|
||||
): Promise<IPageWorkTime> {
|
||||
const req = await api.post<IPageWorkTime>("/pages/history/time", {
|
||||
pageId,
|
||||
tz,
|
||||
});
|
||||
return req.data;
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
import { Modal, Text, Tooltip, UnstyledButton } from "@mantine/core";
|
||||
import { useDisclosure } from "@mantine/hooks";
|
||||
import { IconClockHour4 } from "@tabler/icons-react";
|
||||
import { useTranslation } from "react-i18next";
|
||||
import { usePageWorkTime } from "./use-page-work-time";
|
||||
import { formatGapMinutes, formatHeadline } from "./format-work-time";
|
||||
import WorkTimePunchCard from "./work-time-punch-card";
|
||||
|
||||
interface Props {
|
||||
pageId: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* #395 — the clickable "time worked on this article" headline (§6.1). Renders
|
||||
* the `work` estimate with a "≈" sign and the inactivity threshold in a tooltip
|
||||
* (it is an estimate, not a stopwatch). Clicking opens the daily punch-card
|
||||
* (§6.2). Renders nothing until there is a non-zero human OR agent estimate, so a
|
||||
* brand-new / never-edited page shows no widget. For an agent-only-edited page
|
||||
* (workMs===0, agentOnlyMs>0) the headline shows the agent estimate (labelled
|
||||
* `agent:`, matching the punch-card) so the punch-card stays reachable (#395:
|
||||
* "how much a HUMAN and separately the AGENT").
|
||||
*/
|
||||
export default function WorkTimeStat({ pageId }: Props) {
|
||||
const { t } = useTranslation();
|
||||
const [opened, { open, close }] = useDisclosure(false);
|
||||
const { data } = usePageWorkTime(pageId);
|
||||
|
||||
if (!data || (data.workMs <= 0 && data.agentOnlyMs <= 0)) return null;
|
||||
|
||||
const agentOnly = data.workMs <= 0;
|
||||
const label = agentOnly
|
||||
? t("agent: {{value}}", { value: formatHeadline(data.agentOnlyMs, t) })
|
||||
: formatHeadline(data.workMs, t);
|
||||
const gapMin = formatGapMinutes(data.config.tGap);
|
||||
|
||||
return (
|
||||
<>
|
||||
<Tooltip
|
||||
label={t("Estimated time worked (inactivity gap {{gap}} min)", {
|
||||
gap: gapMin,
|
||||
})}
|
||||
position="bottom"
|
||||
>
|
||||
<UnstyledButton
|
||||
onClick={open}
|
||||
aria-label={t("Show time worked on this page")}
|
||||
>
|
||||
<Text
|
||||
size="xs"
|
||||
c="dimmed"
|
||||
style={{ display: "inline-flex", alignItems: "center", gap: 4 }}
|
||||
>
|
||||
<IconClockHour4 size={14} />
|
||||
{label}
|
||||
</Text>
|
||||
</UnstyledButton>
|
||||
</Tooltip>
|
||||
|
||||
<Modal
|
||||
opened={opened}
|
||||
onClose={close}
|
||||
title={t("Time worked on this article")}
|
||||
size="lg"
|
||||
>
|
||||
<WorkTimePunchCard data={data} />
|
||||
</Modal>
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
/* #395 — 24h × days punch-card. Custom CSS segments on a fixed 24-hour track
|
||||
(position = offset-in-day / 24h, width = duration / 24h), no chart library. */
|
||||
|
||||
.row {
|
||||
display: grid;
|
||||
grid-template-columns: 96px 1fr 64px;
|
||||
align-items: center;
|
||||
gap: 12px;
|
||||
padding: 3px 0;
|
||||
}
|
||||
|
||||
.dayLabel {
|
||||
font-size: var(--mantine-font-size-xs);
|
||||
color: var(--mantine-color-dimmed);
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.track {
|
||||
position: relative;
|
||||
height: 16px;
|
||||
border-radius: 4px;
|
||||
background-color: light-dark(
|
||||
var(--mantine-color-gray-1),
|
||||
var(--mantine-color-dark-6)
|
||||
);
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
/* Faint hour grid so the eye can read "morning vs evening". */
|
||||
.hourTick {
|
||||
position: absolute;
|
||||
top: 0;
|
||||
bottom: 0;
|
||||
width: 1px;
|
||||
background-color: light-dark(
|
||||
var(--mantine-color-gray-3),
|
||||
var(--mantine-color-dark-4)
|
||||
);
|
||||
}
|
||||
|
||||
.window {
|
||||
position: absolute;
|
||||
top: 2px;
|
||||
bottom: 2px;
|
||||
border-radius: 3px;
|
||||
min-width: 3px;
|
||||
}
|
||||
|
||||
.windowWork {
|
||||
background-color: var(--mantine-color-blue-5);
|
||||
}
|
||||
|
||||
.windowAgent {
|
||||
background-color: var(--mantine-color-grape-5);
|
||||
}
|
||||
|
||||
/* A lone single-sample (P_single) window: minimal + dimmed, so it neither
|
||||
vanishes nor fakes dense work (§6.2). */
|
||||
.windowSingle {
|
||||
opacity: 0.5;
|
||||
}
|
||||
|
||||
.daySum {
|
||||
font-size: var(--mantine-font-size-xs);
|
||||
text-align: right;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.gapRow {
|
||||
padding: 6px 0 6px 108px;
|
||||
font-size: var(--mantine-font-size-xs);
|
||||
color: var(--mantine-color-dimmed);
|
||||
font-style: italic;
|
||||
}
|
||||
|
||||
.legendSwatch {
|
||||
display: inline-block;
|
||||
width: 12px;
|
||||
height: 12px;
|
||||
border-radius: 3px;
|
||||
vertical-align: middle;
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
// #395 — client-side mirror of the server work-time payload
|
||||
// (apps/server/src/core/page/work-time). Shapes returned by POST /pages/history/time.
|
||||
|
||||
export type WorkSessionClass = "work" | "agent_only";
|
||||
|
||||
export interface IDayWindow {
|
||||
start: number;
|
||||
end: number;
|
||||
class: WorkSessionClass;
|
||||
}
|
||||
|
||||
export interface IPerDay {
|
||||
day: number;
|
||||
dayISO: string;
|
||||
activeMs: number;
|
||||
agentMs: number;
|
||||
windows: IDayWindow[];
|
||||
}
|
||||
|
||||
export interface IWorkTimeConfig {
|
||||
tGap: number;
|
||||
agentTGap: number;
|
||||
pIn: number;
|
||||
pOut: number;
|
||||
pSingle: number;
|
||||
excludeGit: boolean;
|
||||
burstCapMs?: number;
|
||||
dedupRoundMs: number;
|
||||
}
|
||||
|
||||
export interface IPageWorkTime {
|
||||
workMs: number;
|
||||
agentOnlyMs: number;
|
||||
perDay: IPerDay[];
|
||||
config: IWorkTimeConfig;
|
||||
tz: string;
|
||||
}
|
||||
@@ -51,6 +51,7 @@ import {
|
||||
import { formattedDate } from "@/lib/time.ts";
|
||||
import { PageEditModeToggle } from "@/features/user/components/page-state-pref.tsx";
|
||||
import MovePageModal from "@/features/page/components/move-page-modal.tsx";
|
||||
import WorkTimeStat from "@/features/page-history/work-time/work-time-stat.tsx";
|
||||
import { useTimeAgo } from "@/hooks/use-time-ago.tsx";
|
||||
import {
|
||||
useFavoriteIds,
|
||||
@@ -265,6 +266,8 @@ function PageActionMenu({ readOnly, onSaveVersion }: PageActionMenuProps) {
|
||||
|
||||
return (
|
||||
<>
|
||||
{page?.id && <WorkTimeStat pageId={page.id} />}
|
||||
|
||||
<Menu
|
||||
shadow="xl"
|
||||
position="bottom-end"
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { Spotlight } from "@mantine/spotlight";
|
||||
import { IconSearch } from "@tabler/icons-react";
|
||||
import { Group, VisuallyHidden } from "@mantine/core";
|
||||
import { Group, Text, VisuallyHidden } from "@mantine/core";
|
||||
import { useState, useMemo } from "react";
|
||||
import { useDebouncedValue } from "@mantine/hooks";
|
||||
import { useTranslation } from "react-i18next";
|
||||
@@ -84,6 +84,11 @@ export function SearchSpotlight({ spaceId }: SearchSpotlightProps) {
|
||||
onFiltersChange={handleFiltersChange}
|
||||
spaceId={spaceId}
|
||||
/>
|
||||
{/* #529: operator hint — matches ANY word by default; "…" for an exact
|
||||
phrase, +term to require, -term to exclude. */}
|
||||
<Text size="xs" c="dimmed" mt={4}>
|
||||
{t('Tip: "exact phrase", +required, -excluded')}
|
||||
</Text>
|
||||
</div>
|
||||
|
||||
<VisuallyHidden role="status" aria-live="polite">
|
||||
|
||||
@@ -5,6 +5,9 @@ import { IPage } from "@/features/page/types/page.types.ts";
|
||||
|
||||
export interface IPageSearch {
|
||||
id: string;
|
||||
// #529 A7 superset: `pageId` aliases `id`; `rank`/`highlight` are null for
|
||||
// substring-only hits (the UI already falls back to the title/snippet).
|
||||
pageId?: string;
|
||||
title: string;
|
||||
icon: string;
|
||||
parentPageId: string;
|
||||
@@ -12,9 +15,36 @@ export interface IPageSearch {
|
||||
creatorId: string;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
rank: string;
|
||||
highlight: string;
|
||||
rank: string | number | null;
|
||||
highlight: string | null;
|
||||
space: Partial<ISpace>;
|
||||
// New #529 fields (present from the native Postgres search driver).
|
||||
snippet?: string;
|
||||
score?: number;
|
||||
path?: string[];
|
||||
matchedFields?: string[];
|
||||
matchedTerms?: string[];
|
||||
}
|
||||
|
||||
// #529 A5 pagination envelope returned by POST /search (native driver). The web
|
||||
// list helpers read `items`; these travel alongside for pagination + diagnostics.
|
||||
export interface IPageSearchResponse {
|
||||
items: IPageSearch[];
|
||||
total: number;
|
||||
hasMore: boolean;
|
||||
truncatedAtCap: boolean;
|
||||
offset: number;
|
||||
query?: {
|
||||
raw: string;
|
||||
parsed: {
|
||||
positive: string[];
|
||||
required: string[];
|
||||
excluded: string[];
|
||||
reason?: string;
|
||||
};
|
||||
mode: "or" | "and";
|
||||
match: string;
|
||||
};
|
||||
}
|
||||
|
||||
export interface SearchSuggestionParams {
|
||||
@@ -37,6 +67,10 @@ export interface IPageSearchParams {
|
||||
query: string;
|
||||
spaceId?: string;
|
||||
shareId?: string;
|
||||
// #529 A9: match mode (auto default) + pagination.
|
||||
match?: "auto" | "word" | "prefix" | "substring";
|
||||
limit?: number;
|
||||
offset?: number;
|
||||
}
|
||||
|
||||
export interface IAttachmentSearch {
|
||||
|
||||
@@ -29,6 +29,7 @@ import ShareAliasSection from "@/features/share/components/share-alias-section.t
|
||||
import { useAtom } from "jotai";
|
||||
import { workspaceAtom } from "@/features/user/atoms/current-user-atom.ts";
|
||||
import { useSpaceQuery } from "@/features/space/queries/space-query.ts";
|
||||
import { usePageHistoryListQuery } from "@/features/page-history/queries/page-history-query.ts";
|
||||
|
||||
interface ShareModalProps {
|
||||
readOnly: boolean;
|
||||
@@ -54,6 +55,33 @@ export default function ShareModal({ readOnly }: ShareModalProps) {
|
||||
// if level is greater than zero, then it is a descendant page from a shared page
|
||||
const isDescendantShared = share && share.level > 0;
|
||||
|
||||
// #370 Stage B — "publish only the saved version". Mirrors the server XOR:
|
||||
// approved and includeSubPages are mutually exclusive.
|
||||
const isApproved = share?.publishedMode === "approved";
|
||||
const includeSubPages = share?.includeSubPages ?? false;
|
||||
|
||||
// Resolve the latest MANUAL version (the one an approved share publishes) so
|
||||
// the modal can caption it and warn when the live draft has drifted ahead.
|
||||
// Only fetched for a directly-shared page whose modal is actually open.
|
||||
const { data: historyData } = usePageHistoryListQuery(
|
||||
pageIsShared ? pageId : "",
|
||||
);
|
||||
const latestManual = useMemo(() => {
|
||||
const items = historyData?.pages?.flatMap((p) => p.items) ?? [];
|
||||
// The list is newest-first; the first manual row is the published version.
|
||||
return items.find((h) => h.kind === "manual") ?? null;
|
||||
}, [historyData]);
|
||||
|
||||
// The live draft has edits newer than the published saved version when the
|
||||
// page was updated after that manual snapshot was taken.
|
||||
const hasUnpublishedEdits = useMemo(() => {
|
||||
if (!isApproved || !latestManual || !page?.updatedAt) return false;
|
||||
return (
|
||||
new Date(page.updatedAt).getTime() >
|
||||
new Date(latestManual.createdAt).getTime()
|
||||
);
|
||||
}, [isApproved, latestManual, page?.updatedAt]);
|
||||
|
||||
const publicLink = `${getAppUrl()}/share/${share?.key}/p/${pageSlug}`;
|
||||
|
||||
const [isPagePublic, setIsPagePublic] = useState<boolean>(false);
|
||||
@@ -115,6 +143,23 @@ export default function ShareModal({ readOnly }: ShareModalProps) {
|
||||
}
|
||||
};
|
||||
|
||||
// #370 Stage B — toggle between publishing the live draft ('live') and the
|
||||
// last saved version ('approved'). The server rejects approved + sub-pages, so
|
||||
// the UI keeps the two toggles mutually exclusive (see the disabled props).
|
||||
const handlePublishedModeChange = async (
|
||||
event: React.ChangeEvent<HTMLInputElement>,
|
||||
) => {
|
||||
const value = event.currentTarget.checked;
|
||||
try {
|
||||
await updateShareMutation.mutateAsync({
|
||||
shareId: share.id,
|
||||
publishedMode: value ? "approved" : "live",
|
||||
});
|
||||
} catch {
|
||||
// query invalidation will revert the UI
|
||||
}
|
||||
};
|
||||
|
||||
const shareLink = useMemo(
|
||||
() => (
|
||||
<Group my="sm" gap={4} wrap="nowrap">
|
||||
@@ -238,11 +283,42 @@ export default function ShareModal({ readOnly }: ShareModalProps) {
|
||||
|
||||
<Switch
|
||||
onChange={handleSubPagesChange}
|
||||
checked={share.includeSubPages}
|
||||
checked={includeSubPages}
|
||||
size="xs"
|
||||
disabled={readOnly}
|
||||
// XOR with approved mode: a version-frozen share cannot also
|
||||
// publish a whole sub-tree.
|
||||
disabled={readOnly || isApproved}
|
||||
/>
|
||||
</Group>
|
||||
<Group justify="space-between" wrap="nowrap" gap="xl" mt="sm">
|
||||
<div>
|
||||
<Text size="sm">
|
||||
{t("Publish only the saved version")}
|
||||
</Text>
|
||||
<Text size="xs" c="dimmed">
|
||||
{isApproved
|
||||
? t(
|
||||
"Visitors see the last saved version, not live edits",
|
||||
)
|
||||
: t("Visitors see live edits as you make them")}
|
||||
</Text>
|
||||
</div>
|
||||
<Switch
|
||||
onChange={handlePublishedModeChange}
|
||||
checked={isApproved}
|
||||
size="xs"
|
||||
// XOR with sub-pages: hidden intent is enforced by disabling
|
||||
// this toggle whenever sub-pages are shared.
|
||||
disabled={readOnly || includeSubPages}
|
||||
/>
|
||||
</Group>
|
||||
{isApproved && hasUnpublishedEdits && (
|
||||
<Text size="xs" c="orange" mt={4}>
|
||||
{t(
|
||||
"You have unsaved changes newer than the published version",
|
||||
)}
|
||||
</Text>
|
||||
)}
|
||||
<Group justify="space-between" wrap="nowrap" gap="xl" mt="sm">
|
||||
<div>
|
||||
<Text size="sm">{t("Search engine indexing")}</Text>
|
||||
|
||||
@@ -6,6 +6,9 @@ export interface IShare {
|
||||
pageId: string;
|
||||
includeSubPages: boolean;
|
||||
searchIndexing: boolean;
|
||||
// #370 Stage B — 'live' serves the current draft; 'approved' serves the last
|
||||
// manually-saved version. Mutually exclusive with includeSubPages.
|
||||
publishedMode: "live" | "approved";
|
||||
creatorId: string;
|
||||
spaceId: string;
|
||||
workspaceId: string;
|
||||
@@ -75,6 +78,7 @@ export interface ICreateShare {
|
||||
pageId?: string;
|
||||
includeSubPages?: boolean;
|
||||
searchIndexing?: boolean;
|
||||
publishedMode?: "live" | "approved";
|
||||
}
|
||||
|
||||
export type IUpdateShare = ICreateShare & { shareId: string; pageId?: string };
|
||||
|
||||
@@ -0,0 +1,195 @@
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
|
||||
import { render, act, cleanup } from "@testing-library/react";
|
||||
import { MemoryRouter, useNavigate } from "react-router-dom";
|
||||
|
||||
// Mocks for the dirty shell's side-effecting collaborators.
|
||||
vi.mock("@mantine/notifications", () => ({
|
||||
notifications: { show: vi.fn() },
|
||||
}));
|
||||
vi.mock("@/i18n.ts", () => ({ default: { t: (k: string) => k } }));
|
||||
vi.mock("@/lib/reload-guard", () => ({
|
||||
hasAutoReloaded: vi.fn(() => false),
|
||||
markAutoReloaded: vi.fn(() => true),
|
||||
recordReloadBreadcrumb: vi.fn(),
|
||||
takeReloadBreadcrumb: vi.fn(() => null),
|
||||
}));
|
||||
|
||||
import { notifications } from "@mantine/notifications";
|
||||
import { hasAutoReloaded, markAutoReloaded } from "@/lib/reload-guard";
|
||||
import {
|
||||
triggerGuardedReload,
|
||||
useVersionReloadOnNavigation,
|
||||
__resetGuardedReloadForTests,
|
||||
} from "./guarded-reload";
|
||||
|
||||
const show = notifications.show as unknown as ReturnType<typeof vi.fn>;
|
||||
const mockHasAutoReloaded = hasAutoReloaded as unknown as ReturnType<
|
||||
typeof vi.fn
|
||||
>;
|
||||
const mockMarkAutoReloaded = markAutoReloaded as unknown as ReturnType<
|
||||
typeof vi.fn
|
||||
>;
|
||||
|
||||
let reload: ReturnType<typeof vi.fn>;
|
||||
let visibility: DocumentVisibilityState;
|
||||
|
||||
// Test harness mounted inside a router: it installs the navigation hook and
|
||||
// exposes `navigate` so a test can drive an in-app router navigation.
|
||||
let doNavigate: (to: string) => void;
|
||||
function Harness() {
|
||||
useVersionReloadOnNavigation();
|
||||
const navigate = useNavigate();
|
||||
doNavigate = navigate;
|
||||
return null;
|
||||
}
|
||||
|
||||
function mountHarness() {
|
||||
render(
|
||||
<MemoryRouter initialEntries={["/start"]}>
|
||||
<Harness />
|
||||
</MemoryRouter>,
|
||||
);
|
||||
}
|
||||
|
||||
function navigateTo(path: string) {
|
||||
act(() => {
|
||||
doNavigate(path);
|
||||
});
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
__resetGuardedReloadForTests();
|
||||
vi.clearAllMocks();
|
||||
mockHasAutoReloaded.mockReturnValue(false);
|
||||
mockMarkAutoReloaded.mockReturnValue(true);
|
||||
|
||||
vi.stubGlobal("APP_VERSION", "test-A");
|
||||
|
||||
reload = vi.fn();
|
||||
Object.defineProperty(window, "location", {
|
||||
configurable: true,
|
||||
value: { reload },
|
||||
});
|
||||
|
||||
visibility = "visible";
|
||||
Object.defineProperty(document, "visibilityState", {
|
||||
configurable: true,
|
||||
get: () => visibility,
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
describe("triggerGuardedReload (variant C)", () => {
|
||||
it("noop when versions match: no banner, no reload", () => {
|
||||
triggerGuardedReload("test-A");
|
||||
expect(show).not.toHaveBeenCalled();
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("noop when the server version is empty (fail-safe)", () => {
|
||||
triggerGuardedReload("");
|
||||
triggerGuardedReload(undefined);
|
||||
expect(show).not.toHaveBeenCalled();
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("real mismatch shows the banner but does NOT reload immediately", () => {
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
expect(show).toHaveBeenCalledTimes(1);
|
||||
expect(show.mock.calls[0][0]).toMatchObject({
|
||||
id: "app-version-reload",
|
||||
autoClose: false,
|
||||
withCloseButton: true,
|
||||
});
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("reloads EXACTLY ONCE on the first in-app navigation after a mismatch", () => {
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
|
||||
navigateTo("/next");
|
||||
expect(reload).toHaveBeenCalledTimes(1);
|
||||
|
||||
// A second navigation must NOT reload again (one-shot was consumed).
|
||||
navigateTo("/again");
|
||||
expect(reload).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("does NOT reload on a 2nd mismatch after the first was armed/consumed (one-shot)", () => {
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
navigateTo("/next");
|
||||
expect(reload).toHaveBeenCalledTimes(1);
|
||||
|
||||
// Another app-version mismatch arrives (reconnect): must not re-arm.
|
||||
triggerGuardedReload("test-C");
|
||||
navigateTo("/again");
|
||||
expect(reload).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("does NOT reload merely from the tab going to the background", () => {
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
|
||||
visibility = "hidden";
|
||||
act(() => {
|
||||
document.dispatchEvent(new Event("visibilitychange"));
|
||||
});
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("a hidden-at-receipt tab is also NOT reloaded immediately (variant C uniform); reloads on next navigation", () => {
|
||||
visibility = "hidden";
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
expect(show).toHaveBeenCalledTimes(1);
|
||||
|
||||
navigateTo("/next");
|
||||
expect(reload).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("the banner's Update button reloads immediately", () => {
|
||||
triggerGuardedReload("test-B");
|
||||
const message = show.mock.calls[0][0].message as {
|
||||
props: { onClick: () => void };
|
||||
};
|
||||
message.props.onClick();
|
||||
expect(reload).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("banner-only (auto-reload already spent): banner, never auto-reload on navigation", () => {
|
||||
mockHasAutoReloaded.mockReturnValue(true);
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
expect(show).toHaveBeenCalledTimes(1);
|
||||
|
||||
navigateTo("/next");
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does NOT reload when the flag write fails; falls back to the banner", () => {
|
||||
mockMarkAutoReloaded.mockReturnValue(false);
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
navigateTo("/next");
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
// performAutoReload falls back to showing the banner (initial + fallback).
|
||||
expect(show).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("is idempotent within a tab-load: repeated emits do not stack banners", () => {
|
||||
triggerGuardedReload("test-B");
|
||||
triggerGuardedReload("test-B");
|
||||
triggerGuardedReload("test-C");
|
||||
expect(show).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,188 @@
|
||||
import { useEffect, useRef } from "react";
|
||||
import { useLocation } from "react-router-dom";
|
||||
import { Button } from "@mantine/core";
|
||||
import { notifications } from "@mantine/notifications";
|
||||
import i18n from "@/i18n.ts";
|
||||
import {
|
||||
hasAutoReloaded,
|
||||
markAutoReloaded,
|
||||
recordReloadBreadcrumb,
|
||||
takeReloadBreadcrumb,
|
||||
} from "@/lib/reload-guard";
|
||||
import { decideVersionAction } from "@/features/user/version-coherence";
|
||||
|
||||
// Dirty shell around the pure `decideVersionAction`: it reads globals
|
||||
// (APP_VERSION), touches sessionStorage via the shared reload-guard, drives the
|
||||
// Mantine notification, and arms the router-navigation reload hook. Kept
|
||||
// separate from the pure module so the decision stays unit-testable without a
|
||||
// DOM.
|
||||
|
||||
// One fixed id so repeated app-version signals (e.g. every reconnect) update a
|
||||
// single banner instead of stacking a new one each time.
|
||||
const BANNER_ID = "app-version-reload";
|
||||
|
||||
// Module-level idempotency for the current tab-load: once a mismatch has been
|
||||
// handled we don't re-arm the navigation reload or re-show the banner on
|
||||
// subsequent app-version emits.
|
||||
let handled = false;
|
||||
|
||||
// Variant C: on a real mismatch we do NOT reload the tab when it merely goes to
|
||||
// the background (that would silently drop a half-written comment/form). Instead
|
||||
// we arm a one-shot reload for the NEXT in-app router navigation — a point where
|
||||
// the user is already leaving the current page, so an in-app navigation would
|
||||
// discard that unsaved component-state anyway and the reload adds no extra loss.
|
||||
let pendingNavReload = false;
|
||||
|
||||
// Remembered from the last detected mismatch for the pre-reload breadcrumb and
|
||||
// the (already-visible) banner.
|
||||
let lastServerVersion = "";
|
||||
let lastClientVersion = "";
|
||||
|
||||
// Read the build version baked into THIS bundle. The `typeof` guard avoids a
|
||||
// ReferenceError where the `APP_VERSION` global is absent (e.g. under vitest,
|
||||
// where Vite's `define` did not run) — an unknown client version makes the
|
||||
// pure decision no-op (fail-safe).
|
||||
function readClientVersion(): string {
|
||||
return (typeof APP_VERSION !== "undefined" ? APP_VERSION : "").trim();
|
||||
}
|
||||
|
||||
// Perform the actual reload — but only after the shared one-shot flag is
|
||||
// persisted. If the write fails (storage unavailable) we must NOT reload
|
||||
// (mirrors the reactive chunk-load boundary's `catch → return`), and fall back
|
||||
// to the manual banner so the user can still recover.
|
||||
function performAutoReload(): void {
|
||||
if (!markAutoReloaded()) {
|
||||
showReloadBanner();
|
||||
return;
|
||||
}
|
||||
// Trace right before the reload (which clears the console): a persistent
|
||||
// breadcrumb + a log line so the auto-reload is observable in a field report.
|
||||
recordReloadBreadcrumb({
|
||||
path: "proactive",
|
||||
serverVersion: lastServerVersion,
|
||||
clientVersion: lastClientVersion,
|
||||
});
|
||||
console.warn(
|
||||
`[version-coherence] auto-reloading: client=${lastClientVersion} -> server=${lastServerVersion}`,
|
||||
);
|
||||
window.location.reload();
|
||||
}
|
||||
|
||||
function showReloadBanner(): void {
|
||||
notifications.show({
|
||||
id: BANNER_ID,
|
||||
title: i18n.t("A new version is available"),
|
||||
message: (
|
||||
<Button size="xs" mt="xs" onClick={() => performAutoReload()}>
|
||||
{i18n.t("Update")}
|
||||
</Button>
|
||||
),
|
||||
autoClose: false,
|
||||
withCloseButton: true,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Handle a server `app-version` announcement: compare it to this bundle's
|
||||
* version and, on a real mismatch, show the banner and arm a guarded reload for
|
||||
* the next in-app navigation (variant C).
|
||||
*
|
||||
* - real mismatch (window budget available) → banner + arm navigation reload.
|
||||
* The banner's "Update" button reloads immediately (same shared window guard).
|
||||
* The tab is NOT reloaded on visibility change.
|
||||
* - auto-reload already used this window / storage error → banner only (no arm),
|
||||
* so there is at most one automatic reload per RELOAD_WINDOW_MS window (loop
|
||||
* safety).
|
||||
* - in sync / unknown version → noop (fail-safe).
|
||||
*/
|
||||
export function triggerGuardedReload(
|
||||
rawServerVersion: string | undefined | null,
|
||||
): void {
|
||||
const serverVersion = (rawServerVersion ?? "").trim();
|
||||
const clientVersion = readClientVersion();
|
||||
|
||||
// A storage read error surfaces as autoReloadUsed=true → fail toward NOT
|
||||
// reloading (banner only).
|
||||
const autoReloadUsed = hasAutoReloaded();
|
||||
|
||||
const action = decideVersionAction({
|
||||
serverVersion,
|
||||
clientVersion,
|
||||
autoReloadUsed,
|
||||
});
|
||||
if (action === "noop") return;
|
||||
|
||||
// Idempotent per tab-load: don't re-arm or re-stack the banner across repeated
|
||||
// emits (reconnects) once we've already acted.
|
||||
if (handled) return;
|
||||
handled = true;
|
||||
|
||||
lastServerVersion = serverVersion;
|
||||
lastClientVersion = clientVersion;
|
||||
|
||||
if (action === "banner") {
|
||||
// Entered banner-only (permanent skew, node oscillation, or the window's
|
||||
// auto-reload budget already spent). Log for diagnosability; show the banner.
|
||||
console.warn(
|
||||
`[version-coherence] server=${serverVersion} client=${clientVersion}: ` +
|
||||
"auto-reload budget already spent this window — showing manual banner",
|
||||
);
|
||||
showReloadBanner();
|
||||
return;
|
||||
}
|
||||
|
||||
// action === "reload" (variant C): show the banner and defer the auto-reload
|
||||
// to the next in-app navigation instead of reloading now / on visibility.
|
||||
showReloadBanner();
|
||||
pendingNavReload = true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Consume the armed one-shot navigation reload, if any. Called by
|
||||
* `useVersionReloadOnNavigation` on each in-app router navigation.
|
||||
*/
|
||||
export function consumeNavigationReload(): void {
|
||||
if (!pendingNavReload) return;
|
||||
pendingNavReload = false;
|
||||
performAutoReload();
|
||||
}
|
||||
|
||||
/**
|
||||
* Hook (mounted inside the Router) that fires the armed one-shot reload on the
|
||||
* NEXT in-app router navigation after a version mismatch. Skips the initial
|
||||
* render so it only reacts to real navigations, not the first location.
|
||||
*/
|
||||
export function useVersionReloadOnNavigation(): void {
|
||||
const location = useLocation();
|
||||
const firstRender = useRef(true);
|
||||
useEffect(() => {
|
||||
if (firstRender.current) {
|
||||
firstRender.current = false;
|
||||
return;
|
||||
}
|
||||
consumeNavigationReload();
|
||||
}, [location.key]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Surface (log once) the breadcrumb left by an auto-reload in the previous page
|
||||
* load — the reload cleared the console, so this makes a "tab reloaded itself"
|
||||
* report diagnosable. Call once on app startup.
|
||||
*/
|
||||
export function surfacePreviousReloadBreadcrumb(): void {
|
||||
const crumb = takeReloadBreadcrumb();
|
||||
if (!crumb) return;
|
||||
console.info(
|
||||
`[version-coherence] previous auto-reload: path=${crumb.path} ` +
|
||||
`client=${crumb.clientVersion ?? ""} -> server=${crumb.serverVersion ?? ""} ` +
|
||||
`at=${new Date(crumb.at).toISOString()}`,
|
||||
);
|
||||
}
|
||||
|
||||
// Test-only: reset module-level latches between cases.
|
||||
export function __resetGuardedReloadForTests(): void {
|
||||
handled = false;
|
||||
pendingNavReload = false;
|
||||
lastServerVersion = "";
|
||||
lastClientVersion = "";
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
|
||||
import { render, act, cleanup } from "@testing-library/react";
|
||||
import { MemoryRouter, useNavigate } from "react-router-dom";
|
||||
|
||||
// Integration test for the SHARED, window-based auto-reload budget (invariant a):
|
||||
// the reactive chunk-load boundary and the proactive version-coherence path both
|
||||
// route through the REAL @/lib/reload-guard, so at most one automatic reload
|
||||
// happens per RELOAD_WINDOW_MS across BOTH paths combined. Only the two paths'
|
||||
// side-effecting collaborators are mocked — the reload guard is intentionally
|
||||
// REAL so this exercises the actual shared sessionStorage budget.
|
||||
vi.mock("@mantine/notifications", () => ({
|
||||
notifications: { show: vi.fn() },
|
||||
}));
|
||||
vi.mock("@/i18n.ts", () => ({ default: { t: (k: string) => k } }));
|
||||
|
||||
import { handleError } from "@/components/chunk-load-error-boundary";
|
||||
import {
|
||||
triggerGuardedReload,
|
||||
useVersionReloadOnNavigation,
|
||||
__resetGuardedReloadForTests,
|
||||
} from "./guarded-reload";
|
||||
import { RELOAD_WINDOW_MS } from "@/lib/reload-guard";
|
||||
|
||||
const CHUNK_ERROR = { name: "ChunkLoadError", message: "boom" };
|
||||
const T0 = 1_000_000_000_000;
|
||||
|
||||
let reload: ReturnType<typeof vi.fn>;
|
||||
let nowMock: ReturnType<typeof vi.spyOn>;
|
||||
|
||||
// Harness mounted inside a router: installs the navigation hook and exposes
|
||||
// `navigate` so a test can drive an in-app router navigation (the point where the
|
||||
// proactive path fires its armed reload).
|
||||
let doNavigate: (to: string) => void;
|
||||
function Harness() {
|
||||
useVersionReloadOnNavigation();
|
||||
doNavigate = useNavigate();
|
||||
return null;
|
||||
}
|
||||
function mountHarness() {
|
||||
render(
|
||||
<MemoryRouter initialEntries={["/start"]}>
|
||||
<Harness />
|
||||
</MemoryRouter>,
|
||||
);
|
||||
}
|
||||
function navigateTo(path: string) {
|
||||
act(() => {
|
||||
doNavigate(path);
|
||||
});
|
||||
}
|
||||
function setNow(t: number) {
|
||||
nowMock.mockReturnValue(t);
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
sessionStorage.clear();
|
||||
__resetGuardedReloadForTests();
|
||||
vi.clearAllMocks();
|
||||
nowMock = vi.spyOn(Date, "now").mockReturnValue(T0);
|
||||
vi.stubGlobal("APP_VERSION", "test-A");
|
||||
reload = vi.fn();
|
||||
Object.defineProperty(window, "location", {
|
||||
configurable: true,
|
||||
value: { reload },
|
||||
});
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
vi.unstubAllGlobals();
|
||||
vi.restoreAllMocks();
|
||||
sessionStorage.clear();
|
||||
});
|
||||
|
||||
describe("shared window-based reload budget (invariant a)", () => {
|
||||
it("a chunk-load reload spends the budget: a version-coherence mismatch within the window shows the banner but does NOT reload", () => {
|
||||
// Path 1 (reactive): a stale-chunk 404 auto-reloads once and stamps the window.
|
||||
handleError(CHUNK_ERROR);
|
||||
expect(reload).toHaveBeenCalledTimes(1);
|
||||
reload.mockClear();
|
||||
|
||||
// Path 2 (proactive), 1 min later — still inside the window. The shared budget
|
||||
// is spent, so the version mismatch degrades to the banner and never reloads.
|
||||
setNow(T0 + 60_000);
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
navigateTo("/next");
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("a version-coherence reload spends the SAME budget: a chunk-load error within the window does NOT reload", () => {
|
||||
// Path 2 (proactive) first: real mismatch → arm → fire on navigation.
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
navigateTo("/next");
|
||||
expect(reload).toHaveBeenCalledTimes(1);
|
||||
reload.mockClear();
|
||||
|
||||
// Path 1 (reactive), 2 min later — inside the window. Budget already spent by
|
||||
// the proactive path, so the stale-chunk error must NOT trigger a second reload.
|
||||
setNow(T0 + 2 * 60_000);
|
||||
handleError(CHUNK_ERROR);
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("recovers after the window: a reload strictly older than the window is allowed again (second deploy)", () => {
|
||||
// First auto-reload (reactive) stamps the window.
|
||||
handleError(CHUNK_ERROR);
|
||||
expect(reload).toHaveBeenCalledTimes(1);
|
||||
reload.mockClear();
|
||||
|
||||
// A second deploy arrives after the window has fully elapsed → the proactive
|
||||
// path is allowed to reload again (window, not a permanent one-shot).
|
||||
__resetGuardedReloadForTests();
|
||||
setNow(T0 + RELOAD_WINDOW_MS + 1);
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
navigateTo("/next");
|
||||
expect(reload).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("reactive path fails closed when storage READS but cannot WRITE (quota / Safari private): no unguarded reload", () => {
|
||||
// getItem→null makes hasAutoReloaded() report the budget as available, so
|
||||
// handleError passes the first guard and reaches `if (!markAutoReloaded())
|
||||
// return;`. setItem throws → the stamp cannot stick, so markAutoReloaded()
|
||||
// returns false and that guard MUST bail — otherwise the reactive path would
|
||||
// reload on every stale-chunk error with no persisted budget (an unguarded
|
||||
// loop). This is the asymmetric gap: the proactive path's equivalent is
|
||||
// covered by guarded-reload.test.tsx "does NOT reload when the flag write
|
||||
// fails".
|
||||
vi.stubGlobal("sessionStorage", {
|
||||
getItem: () => null,
|
||||
setItem: () => {
|
||||
throw new Error("quota exceeded");
|
||||
},
|
||||
removeItem: () => {},
|
||||
clear: () => {},
|
||||
});
|
||||
try {
|
||||
handleError(CHUNK_ERROR);
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
|
||||
it("sessionStorage unavailable: neither path performs an unguarded reload", () => {
|
||||
// The real guard fails toward NOT reloading when storage throws.
|
||||
vi.stubGlobal("sessionStorage", {
|
||||
getItem: () => {
|
||||
throw new Error("storage disabled");
|
||||
},
|
||||
setItem: () => {
|
||||
throw new Error("storage disabled");
|
||||
},
|
||||
removeItem: () => {},
|
||||
clear: () => {},
|
||||
});
|
||||
try {
|
||||
handleError(CHUNK_ERROR);
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
|
||||
mountHarness();
|
||||
triggerGuardedReload("test-B");
|
||||
navigateTo("/next");
|
||||
expect(reload).not.toHaveBeenCalled();
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -13,6 +13,12 @@ import { useCollabToken } from "@/features/auth/queries/auth-query.tsx";
|
||||
import { Error404 } from "@/components/ui/error-404.tsx";
|
||||
import { queryClient } from "@/main.tsx";
|
||||
import { makeConnectHandler } from "@/features/user/connect-resync.ts";
|
||||
import {
|
||||
triggerGuardedReload,
|
||||
useVersionReloadOnNavigation,
|
||||
surfacePreviousReloadBreadcrumb,
|
||||
} from "@/features/user/guarded-reload.tsx";
|
||||
import type { AppVersionSocketPayload } from "@/features/user/version-coherence.ts";
|
||||
|
||||
export function UserProvider({ children }: React.PropsWithChildren) {
|
||||
const [, setCurrentUser] = useAtom(currentUserAtom);
|
||||
@@ -22,6 +28,16 @@ export function UserProvider({ children }: React.PropsWithChildren) {
|
||||
// fetch collab token on load
|
||||
const { data: collab } = useCollabToken();
|
||||
|
||||
// version-coherence: fire the armed one-shot reload on the next in-app
|
||||
// navigation (variant C — a safe point, not on tab backgrounding).
|
||||
useVersionReloadOnNavigation();
|
||||
|
||||
// Surface any breadcrumb left by an auto-reload in the previous page load
|
||||
// (the reload cleared the console) so a field report stays diagnosable.
|
||||
useEffect(() => {
|
||||
surfacePreviousReloadBreadcrumb();
|
||||
}, []);
|
||||
|
||||
useEffect(() => {
|
||||
if (isLoading || isError) {
|
||||
return;
|
||||
@@ -47,6 +63,16 @@ export function UserProvider({ children }: React.PropsWithChildren) {
|
||||
handleConnect();
|
||||
});
|
||||
|
||||
// Register the version-coherence listener SYNCHRONOUSLY, before the socket
|
||||
// connects: the server emits `app-version` immediately in handleConnection,
|
||||
// so a listener attached after connect would miss it on a fast localhost
|
||||
// connect. On a version mismatch the client shows a banner and defers the
|
||||
// auto-reload to the next in-app navigation (variant C — avoids reloading a
|
||||
// backgrounded tab that may hold unsaved input) before it hits a stale chunk.
|
||||
newSocket.on("app-version", (payload?: AppVersionSocketPayload) => {
|
||||
triggerGuardedReload(payload?.version);
|
||||
});
|
||||
|
||||
return () => {
|
||||
console.log("ws disconnected");
|
||||
newSocket.disconnect();
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import { decideVersionAction } from "./version-coherence";
|
||||
|
||||
describe("decideVersionAction", () => {
|
||||
it("noop when the server version is empty (fail-safe)", () => {
|
||||
expect(
|
||||
decideVersionAction({
|
||||
serverVersion: "",
|
||||
clientVersion: "v1",
|
||||
autoReloadUsed: false,
|
||||
}),
|
||||
).toBe("noop");
|
||||
});
|
||||
|
||||
it("noop when the client version is empty (fail-safe)", () => {
|
||||
expect(
|
||||
decideVersionAction({
|
||||
serverVersion: "v1",
|
||||
clientVersion: "",
|
||||
autoReloadUsed: false,
|
||||
}),
|
||||
).toBe("noop");
|
||||
});
|
||||
|
||||
it("noop when versions are equal (in sync)", () => {
|
||||
expect(
|
||||
decideVersionAction({
|
||||
serverVersion: "v1",
|
||||
clientVersion: "v1",
|
||||
autoReloadUsed: false,
|
||||
}),
|
||||
).toBe("noop");
|
||||
});
|
||||
|
||||
it("reload on a real mismatch the first time this session", () => {
|
||||
expect(
|
||||
decideVersionAction({
|
||||
serverVersion: "test-B",
|
||||
clientVersion: "test-A",
|
||||
autoReloadUsed: false,
|
||||
}),
|
||||
).toBe("reload");
|
||||
});
|
||||
|
||||
it("banner on a mismatch once the session auto-reload is spent", () => {
|
||||
expect(
|
||||
decideVersionAction({
|
||||
serverVersion: "test-B",
|
||||
clientVersion: "test-A",
|
||||
autoReloadUsed: true,
|
||||
}),
|
||||
).toBe("banner");
|
||||
});
|
||||
|
||||
it("equal versions stay noop even if auto-reload was already used", () => {
|
||||
expect(
|
||||
decideVersionAction({
|
||||
serverVersion: "v1",
|
||||
clientVersion: "v1",
|
||||
autoReloadUsed: true,
|
||||
}),
|
||||
).toBe("noop");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,32 @@
|
||||
// Payload of the per-connect `app-version` socket.io event announced by the
|
||||
// server (ws.gateway.ts) after a successful auth. A dedicated event — NOT a
|
||||
// member of the room-scoped `WebSocketEvent` union (which is discriminated by
|
||||
// `operation`), so it never touches use-query-subscription.
|
||||
export type AppVersionSocketPayload = { version: string };
|
||||
|
||||
/**
|
||||
* Pure decision for the version-coherence guard.
|
||||
*
|
||||
* All inputs are injected (no globals, no side effects) so it is unit-testable
|
||||
* without a DOM or the build-time `APP_VERSION` global (undefined under vitest).
|
||||
*
|
||||
* - `autoReloadUsed` = an automatic reload has already happened within the
|
||||
* current ~5-min window, so we must not auto-reload again (loop safety,
|
||||
* shared window budget with the reactive chunk-load boundary).
|
||||
*
|
||||
* Returns:
|
||||
* - "noop" — do nothing (unknown version on either side, or already in sync).
|
||||
* - "banner" — show the manual "update available" banner only (no auto-reload).
|
||||
* - "reload" — real first-time mismatch: eligible for a guarded auto-reload.
|
||||
*/
|
||||
export function decideVersionAction(args: {
|
||||
serverVersion: string;
|
||||
clientVersion: string;
|
||||
autoReloadUsed: boolean;
|
||||
}): "reload" | "banner" | "noop" {
|
||||
const { serverVersion, clientVersion, autoReloadUsed } = args;
|
||||
if (!serverVersion || !clientVersion) return "noop"; // fail-safe: unknown version → never act
|
||||
if (serverVersion === clientVersion) return "noop"; // in sync
|
||||
if (autoReloadUsed) return "banner"; // one auto-reload per RELOAD_WINDOW_MS window already spent
|
||||
return "reload"; // real mismatch, window budget available
|
||||
}
|
||||
@@ -0,0 +1,145 @@
|
||||
import { describe, it, expect, beforeEach, afterEach, vi } from "vitest";
|
||||
import {
|
||||
hasAutoReloaded,
|
||||
markAutoReloaded,
|
||||
shouldAutoReload,
|
||||
recordReloadBreadcrumb,
|
||||
takeReloadBreadcrumb,
|
||||
RELOAD_WINDOW_MS,
|
||||
} from "./reload-guard";
|
||||
|
||||
// The shared budget is a single sessionStorage timestamp keyed here; both the
|
||||
// reactive chunk-load boundary and the proactive version-coherence path read and
|
||||
// stamp it, so at most one auto-reload happens per RELOAD_WINDOW_MS across BOTH.
|
||||
const RELOAD_AT_KEY = "chunk-reload-at";
|
||||
const NOW = 1_000_000_000_000;
|
||||
|
||||
describe("reload-guard", () => {
|
||||
beforeEach(() => {
|
||||
sessionStorage.clear();
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
sessionStorage.clear();
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it("hasAutoReloaded is false before any reload, true within the window after mark", () => {
|
||||
expect(hasAutoReloaded(NOW)).toBe(false);
|
||||
expect(markAutoReloaded(NOW)).toBe(true);
|
||||
// Same key both paths share; stores the reload timestamp, not a flag.
|
||||
expect(sessionStorage.getItem(RELOAD_AT_KEY)).toBe(String(NOW));
|
||||
// Inside the window → budget spent → true (fall through to manual UI).
|
||||
expect(hasAutoReloaded(NOW)).toBe(true);
|
||||
expect(hasAutoReloaded(NOW + RELOAD_WINDOW_MS)).toBe(true);
|
||||
});
|
||||
|
||||
it("hasAutoReloaded is false again once the window has elapsed (a later deploy recovers)", () => {
|
||||
markAutoReloaded(NOW);
|
||||
// Strictly older than the window → a new deploy's mismatch may reload again.
|
||||
expect(hasAutoReloaded(NOW + RELOAD_WINDOW_MS + 1)).toBe(false);
|
||||
});
|
||||
|
||||
it("hasAutoReloaded returns true when reading storage throws (fail toward not reloading)", () => {
|
||||
vi.stubGlobal("sessionStorage", {
|
||||
getItem: () => {
|
||||
throw new Error("storage disabled");
|
||||
},
|
||||
setItem: () => {
|
||||
throw new Error("storage disabled");
|
||||
},
|
||||
});
|
||||
try {
|
||||
expect(hasAutoReloaded()).toBe(true);
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
|
||||
it("hasAutoReloaded treats an unparseable stored timestamp as never-reloaded", () => {
|
||||
sessionStorage.setItem(RELOAD_AT_KEY, "not-a-number");
|
||||
expect(hasAutoReloaded(NOW)).toBe(false);
|
||||
});
|
||||
|
||||
it("markAutoReloaded returns false when writing storage throws", () => {
|
||||
vi.stubGlobal("sessionStorage", {
|
||||
getItem: () => null,
|
||||
setItem: () => {
|
||||
throw new Error("storage disabled");
|
||||
},
|
||||
});
|
||||
try {
|
||||
expect(markAutoReloaded()).toBe(false);
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
|
||||
it("records and then takes a breadcrumb once (cleared on read)", () => {
|
||||
recordReloadBreadcrumb({
|
||||
path: "proactive",
|
||||
serverVersion: "test-B",
|
||||
clientVersion: "test-A",
|
||||
});
|
||||
const crumb = takeReloadBreadcrumb();
|
||||
expect(crumb).toMatchObject({
|
||||
path: "proactive",
|
||||
serverVersion: "test-B",
|
||||
clientVersion: "test-A",
|
||||
});
|
||||
expect(typeof crumb?.at).toBe("number");
|
||||
// Cleared on read → a second take returns null.
|
||||
expect(takeReloadBreadcrumb()).toBeNull();
|
||||
});
|
||||
|
||||
it("takeReloadBreadcrumb returns null when nothing was recorded", () => {
|
||||
expect(takeReloadBreadcrumb()).toBeNull();
|
||||
});
|
||||
|
||||
it("recordReloadBreadcrumb swallows a storage-write error (diagnostics only)", () => {
|
||||
vi.stubGlobal("sessionStorage", {
|
||||
getItem: () => null,
|
||||
setItem: () => {
|
||||
throw new Error("storage disabled");
|
||||
},
|
||||
removeItem: () => {},
|
||||
});
|
||||
try {
|
||||
expect(() =>
|
||||
recordReloadBreadcrumb({ path: "chunk-boundary" }),
|
||||
).not.toThrow();
|
||||
} finally {
|
||||
vi.unstubAllGlobals();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// The pure window gate replaces the old one-shot flag: it must permit recovery
|
||||
// across several deploys in one tab (each > window apart) while still stopping an
|
||||
// infinite reload loop when a lazy chunk is permanently broken (a second failure
|
||||
// < window). Moved here from the chunk-load boundary now that it is the shared
|
||||
// guard both paths route through.
|
||||
describe("shouldAutoReload", () => {
|
||||
const WINDOW = RELOAD_WINDOW_MS;
|
||||
|
||||
it("allows a reload when we have never auto-reloaded", () => {
|
||||
expect(shouldAutoReload(NOW, null, WINDOW)).toBe(true);
|
||||
});
|
||||
|
||||
it("allows a reload when the last one was 6 minutes ago (outside the window)", () => {
|
||||
expect(shouldAutoReload(NOW, NOW - 6 * 60 * 1000, WINDOW)).toBe(true);
|
||||
});
|
||||
|
||||
it("blocks a reload when the last one was 1 minute ago (inside the window)", () => {
|
||||
expect(shouldAutoReload(NOW, NOW - 1 * 60 * 1000, WINDOW)).toBe(false);
|
||||
});
|
||||
|
||||
it("blocks a reload exactly at the window boundary (not strictly older)", () => {
|
||||
expect(shouldAutoReload(NOW, NOW - WINDOW, WINDOW)).toBe(false);
|
||||
});
|
||||
|
||||
it("allows a reload when the stored timestamp is unparseable (NaN)", () => {
|
||||
expect(shouldAutoReload(NOW, NaN, WINDOW)).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,121 @@
|
||||
// Shared, window-based auto-reload budget.
|
||||
//
|
||||
// Both auto-reload paths — the reactive chunk-load-error-boundary (recovers
|
||||
// AFTER a stale lazy chunk 404s) and the proactive version-coherence feature
|
||||
// (reloads BEFORE the tab hits a stale chunk) — go through these functions so
|
||||
// they share ONE window-scoped reload budget: at most a single automatic
|
||||
// reload per RELOAD_WINDOW_MS across BOTH paths. A window (rather than a
|
||||
// permanent one-shot flag) lets a SECOND deploy in the same tab's lifetime
|
||||
// recover too, while a permanent skew, node oscillation, or a genuinely-missing
|
||||
// chunk still degrades to a manual banner/UI after the first reload instead of
|
||||
// looping. When sessionStorage is unavailable every mismatch degrades to the
|
||||
// manual UI — no unguarded reload.
|
||||
|
||||
// sessionStorage key holding the epoch-ms timestamp of the last automatic reload
|
||||
// (shared by both paths).
|
||||
const RELOAD_AT_KEY = "chunk-reload-at";
|
||||
|
||||
// Allow at most one automatic reload per this window. A stale-deploy 404 is cured
|
||||
// by a single reload, so anything inside the window is treated as a reload loop
|
||||
// (permanently-broken chunk / permanent skew) and falls through to the manual UI.
|
||||
export const RELOAD_WINDOW_MS = 5 * 60 * 1000;
|
||||
|
||||
/**
|
||||
* Pure window decision, unit-tested in isolation: auto-reload only if we have
|
||||
* never auto-reloaded (lastReloadAt null/NaN) or the last one was strictly older
|
||||
* than the window. Anything inside the window is suppressed to break an infinite
|
||||
* reload loop.
|
||||
*/
|
||||
export function shouldAutoReload(
|
||||
now: number,
|
||||
lastReloadAt: number | null,
|
||||
windowMs: number,
|
||||
): boolean {
|
||||
if (lastReloadAt === null || Number.isNaN(lastReloadAt)) return true;
|
||||
return now - lastReloadAt > windowMs;
|
||||
}
|
||||
|
||||
/**
|
||||
* Has an automatic reload already happened within the current window (so the
|
||||
* shared budget is spent right now)? Both paths check this before reloading; a
|
||||
* `true` return means fall through to the manual banner/UI instead of reloading.
|
||||
*
|
||||
* A storage read error (private mode / disabled) is reported as `true` so the
|
||||
* caller fails toward NOT reloading — an unguarded loop is worse than a stale
|
||||
* tab the user can reload manually. Note a window (not a permanent flag): once
|
||||
* the window elapses a later deploy's mismatch is allowed to reload again.
|
||||
*/
|
||||
export function hasAutoReloaded(now: number = Date.now()): boolean {
|
||||
try {
|
||||
const raw = sessionStorage.getItem(RELOAD_AT_KEY);
|
||||
const lastReloadAt = raw === null ? null : Number.parseInt(raw, 10);
|
||||
return !shouldAutoReload(now, lastReloadAt, RELOAD_WINDOW_MS);
|
||||
} catch {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Stamp the shared window as consumed now — record that an automatic reload is
|
||||
* being performed within the current RELOAD_WINDOW_MS window.
|
||||
*
|
||||
* Returns whether the write succeeded. A `false` return (storage unavailable)
|
||||
* means the caller MUST NOT reload — otherwise the stamp would never stick and
|
||||
* the reload could loop.
|
||||
*/
|
||||
export function markAutoReloaded(now: number = Date.now()): boolean {
|
||||
try {
|
||||
sessionStorage.setItem(RELOAD_AT_KEY, String(now));
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
// Diagnostic breadcrumb for an automatic reload. Written right before
|
||||
// window.location.reload() (which clears the console) and read back on the next
|
||||
// page load, so a "the tab reloaded itself / it's looping" field report is
|
||||
// diagnosable: which path fired (proactive version-coherence vs the reactive
|
||||
// chunk-load boundary) and which version pair triggered it. sessionStorage
|
||||
// survives a same-tab reload, unlike the console.
|
||||
const RELOAD_BREADCRUMB_KEY = "reload-breadcrumb";
|
||||
|
||||
export type ReloadBreadcrumb = {
|
||||
path: "proactive" | "chunk-boundary";
|
||||
serverVersion?: string;
|
||||
clientVersion?: string;
|
||||
at: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* Persist a best-effort breadcrumb just before an automatic reload. Failures
|
||||
* (storage unavailable) are swallowed — this is diagnostics only and must never
|
||||
* block or alter the reload decision.
|
||||
*/
|
||||
export function recordReloadBreadcrumb(
|
||||
entry: Omit<ReloadBreadcrumb, "at">,
|
||||
): void {
|
||||
try {
|
||||
sessionStorage.setItem(
|
||||
RELOAD_BREADCRUMB_KEY,
|
||||
JSON.stringify({ ...entry, at: Date.now() }),
|
||||
);
|
||||
} catch {
|
||||
// best-effort diagnostics only
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read and clear the breadcrumb left by an auto-reload in the previous page
|
||||
* load. Cleared on read so it surfaces exactly once per reload.
|
||||
*/
|
||||
export function takeReloadBreadcrumb(): ReloadBreadcrumb | null {
|
||||
try {
|
||||
const raw = sessionStorage.getItem(RELOAD_BREADCRUMB_KEY);
|
||||
if (!raw) return null;
|
||||
sessionStorage.removeItem(RELOAD_BREADCRUMB_KEY);
|
||||
return JSON.parse(raw) as ReloadBreadcrumb;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -1,7 +1,8 @@
|
||||
import { defineConfig, loadEnv } from "vite";
|
||||
import { defineConfig, loadEnv, type Plugin } from "vite";
|
||||
import react from "@vitejs/plugin-react";
|
||||
import { compression } from "vite-plugin-compression2";
|
||||
import * as path from "path";
|
||||
import * as fs from "node:fs";
|
||||
import { execSync } from "node:child_process";
|
||||
|
||||
const envPath = path.resolve(process.cwd(), "..", "..");
|
||||
@@ -24,7 +25,32 @@ function resolveAppVersion(cwd: string): string {
|
||||
}
|
||||
}
|
||||
|
||||
// Emit <outDir>/version.json = { "version": appVersion } so the server can read
|
||||
// the exact same build id the bundle was compiled with. The value is the SAME
|
||||
// `appVersion` fed into `define.APP_VERSION`, so version.json and the baked-in
|
||||
// global are identical by construction — the single source of truth (no
|
||||
// runtime-env second copy that could drift and cause a false version mismatch).
|
||||
function versionJsonPlugin(version: string): Plugin {
|
||||
let outDir = "dist";
|
||||
return {
|
||||
name: "emit-version-json",
|
||||
apply: "build",
|
||||
configResolved(config) {
|
||||
outDir = config.build.outDir;
|
||||
},
|
||||
writeBundle() {
|
||||
const root = path.resolve(process.cwd(), outDir);
|
||||
fs.mkdirSync(root, { recursive: true });
|
||||
fs.writeFileSync(
|
||||
path.join(root, "version.json"),
|
||||
JSON.stringify({ version }),
|
||||
);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export default defineConfig(({ mode }) => {
|
||||
const appVersion = resolveAppVersion(envPath);
|
||||
const {
|
||||
APP_URL,
|
||||
FILE_UPLOAD_SIZE_LIMIT,
|
||||
@@ -52,10 +78,11 @@ export default defineConfig(({ mode }) => {
|
||||
POSTHOG_HOST,
|
||||
POSTHOG_KEY,
|
||||
},
|
||||
APP_VERSION: JSON.stringify(resolveAppVersion(envPath)),
|
||||
APP_VERSION: JSON.stringify(appVersion),
|
||||
},
|
||||
plugins: [
|
||||
react(),
|
||||
versionJsonPlugin(appVersion),
|
||||
// Emit .br and .gz next to every built asset so the server can serve the
|
||||
// precompressed copy (see @fastify/static preCompressed in static.module.ts).
|
||||
compression({
|
||||
|
||||
@@ -23,7 +23,7 @@
|
||||
"migration:reset": "tsx src/database/migrate.ts down-to NO_MIGRATIONS",
|
||||
"migration:codegen": "kysely-codegen --dialect=postgres --camel-case --env-file=../../.env --out-file=./src/database/types/db.d.ts",
|
||||
"lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
|
||||
"pretest": "pnpm --filter @docmost/editor-ext build && pnpm --filter @docmost/prosemirror-markdown build && pnpm --filter @docmost/token-estimate build",
|
||||
"pretest": "pnpm --filter @docmost/editor-ext build && pnpm --filter @docmost/prosemirror-markdown build && pnpm --filter @docmost/token-estimate build && pnpm --filter @docmost/mcp build",
|
||||
"test": "jest",
|
||||
"test:int": "jest --config test/jest-integration.json",
|
||||
"test:watch": "jest --watch",
|
||||
|
||||
@@ -141,7 +141,57 @@ export function htmlToJson(html: string) {
|
||||
}
|
||||
}
|
||||
|
||||
export function jsonToText(tiptapJson: JSONContent) {
|
||||
/**
|
||||
* Deterministic text-serializer overrides for the `format:"text"` page read
|
||||
* (#502). Non-text nodes render to a STABLE placeholder instead of their
|
||||
* (structure-dependent) inner text, so a machine diff of two text reads is
|
||||
* driven only by the page's actual prose — output stability across package
|
||||
* versions IS the contract (pinned by a snapshot test). Returning a string from
|
||||
* a `textSerializer` also stops `generateText` descending into the node, so a
|
||||
* table renders as ONE token rather than its flattened cell text.
|
||||
*
|
||||
* Only nodes with no meaningful flat-text form are overridden; every other node
|
||||
* (paragraph/heading/list/code/blockquote/callout/…) keeps its natural text so
|
||||
* a config written as markdown reads back byte-identical.
|
||||
*/
|
||||
const TEXT_READ_SERIALIZERS: Record<string, (props: { node: any }) => string> =
|
||||
{
|
||||
// Image atom: no inner text -> a fixed placeholder.
|
||||
image: () => '[image]',
|
||||
// Table: `[table RxC]` where R = row count, C = the first row's cell count
|
||||
// (a table's columns are uniform per the schema). Computed from the PM node,
|
||||
// so it is independent of cell contents.
|
||||
table: ({ node }) => {
|
||||
const rows = node?.childCount ?? 0;
|
||||
const cols = rows > 0 ? (node.child(0)?.childCount ?? 0) : 0;
|
||||
return `[table ${rows}x${cols}]`;
|
||||
},
|
||||
};
|
||||
|
||||
/**
|
||||
* Serialize a ProseMirror/TipTap document to plain text.
|
||||
*
|
||||
* Default (no options): the long-standing search-index behavior — bare
|
||||
* concatenated node text with `generateText`'s default `\n\n` block separator.
|
||||
* This feeds the page `textContent` tsvector and MUST NOT change.
|
||||
*
|
||||
* `deterministic:true` (#502 `format:"text"` page read): a flat, machine-diffable
|
||||
* rendering — one line per block (`\n` block separator; `hardBreak` already
|
||||
* serializes to `\n`), inline marks/anchors/autoformat dropped, and non-text
|
||||
* nodes replaced by the stable placeholders above (`[image]`, `[table RxC]`).
|
||||
*/
|
||||
export function jsonToText(
|
||||
// `any` (like jsonToHtml/jsonToMarkdown) so a loosely-typed DB `page.content`
|
||||
// (JsonValue) can be passed straight through, as the controller does.
|
||||
tiptapJson: any,
|
||||
options?: { deterministic?: boolean },
|
||||
) {
|
||||
if (options?.deterministic) {
|
||||
return generateText(tiptapJson, tiptapExtensions, {
|
||||
blockSeparator: '\n',
|
||||
textSerializers: TEXT_READ_SERIALIZERS,
|
||||
});
|
||||
}
|
||||
return generateText(tiptapJson, tiptapExtensions);
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
import { jsonToText } from './collaboration.util';
|
||||
|
||||
// #502 READ contract: `jsonToText(json, { deterministic: true })` — the flat,
|
||||
// machine-diffable text rendering behind getPage `format:"text"`. Block-per-line
|
||||
// (`\n` separator), inline marks/anchors dropped, hardBreak -> `\n`, and non-text
|
||||
// nodes replaced by STABLE placeholders (`[image]`, `[table RxC]`). Output
|
||||
// stability across package versions IS the contract, so it is pinned by a
|
||||
// snapshot below. The DEFAULT (no options) path is the search-index serializer
|
||||
// and MUST be unchanged — asserted separately.
|
||||
|
||||
const doc = (...content: any[]) => ({ type: 'doc', content });
|
||||
const para = (...content: any[]) => ({ type: 'paragraph', content });
|
||||
const text = (t: string, marks?: any[]) =>
|
||||
marks ? { type: 'text', text: t, marks } : { type: 'text', text: t };
|
||||
|
||||
describe('jsonToText — default (search index) behavior is unchanged', () => {
|
||||
it('uses the `\\n\\n` block separator and drops non-text nodes to empty', () => {
|
||||
const d = doc(para(text('alpha')), para(text('beta')));
|
||||
expect(jsonToText(d)).toBe('alpha\n\nbeta');
|
||||
});
|
||||
|
||||
it('an image contributes no text in the default (tsvector) mode', () => {
|
||||
const d = doc(para(text('cap')), { type: 'image', attrs: { src: 's' } });
|
||||
// No `[image]` placeholder leaks into the search index.
|
||||
expect(jsonToText(d)).not.toContain('[image]');
|
||||
});
|
||||
});
|
||||
|
||||
describe('jsonToText — deterministic:true (getPage format:"text")', () => {
|
||||
it('renders one line per block with `\\n` separators, marks dropped', () => {
|
||||
const d = doc(
|
||||
{ type: 'heading', attrs: { level: 2 }, content: [text('Title')] },
|
||||
para(
|
||||
text('hello ', [{ type: 'bold' }]),
|
||||
text('world', [{ type: 'italic' }]),
|
||||
),
|
||||
);
|
||||
expect(jsonToText(d, { deterministic: true })).toBe('Title\nhello world');
|
||||
});
|
||||
|
||||
it('a hardBreak renders as a newline', () => {
|
||||
const d = doc(para(text('a'), { type: 'hardBreak' }, text('b')));
|
||||
expect(jsonToText(d, { deterministic: true })).toBe('a\nb');
|
||||
});
|
||||
|
||||
it('an image node -> the stable `[image]` placeholder', () => {
|
||||
const d = doc(para(text('before')), { type: 'image', attrs: { src: 's' } });
|
||||
expect(jsonToText(d, { deterministic: true })).toBe('before\n[image]');
|
||||
});
|
||||
|
||||
it('a table -> `[table RxC]` (rows x columns) and its cell text is NOT flattened in', () => {
|
||||
const cell = (t: string) => ({
|
||||
type: 'tableCell',
|
||||
content: [para(text(t))],
|
||||
});
|
||||
const row = (...cells: any[]) => ({ type: 'tableRow', content: cells });
|
||||
const table = {
|
||||
type: 'table',
|
||||
content: [
|
||||
row(cell('CELLONE'), cell('CELLTWO'), cell('CELLTHREE')),
|
||||
row(cell('CELLFOUR'), cell('CELLFIVE'), cell('CELLSIX')),
|
||||
],
|
||||
};
|
||||
const d = doc(para(text('grid:')), table);
|
||||
const out = jsonToText(d, { deterministic: true });
|
||||
expect(out).toBe('grid:\n[table 2x3]');
|
||||
expect(out).not.toContain('CELL'); // cell text is not flattened into the read
|
||||
});
|
||||
|
||||
it('SNAPSHOT: a mixed document renders to a stable, deterministic string', () => {
|
||||
const cell = (t: string) => ({
|
||||
type: 'tableCell',
|
||||
content: [para(text(t))],
|
||||
});
|
||||
const d = doc(
|
||||
{ type: 'heading', attrs: { level: 1 }, content: [text('Config')] },
|
||||
para(text('key = ', [{ type: 'bold' }]), text('value')),
|
||||
{
|
||||
type: 'bulletList',
|
||||
content: [
|
||||
{ type: 'listItem', content: [para(text('one'))] },
|
||||
{ type: 'listItem', content: [para(text('two'))] },
|
||||
],
|
||||
},
|
||||
{ type: 'image', attrs: { src: 's' } },
|
||||
{
|
||||
type: 'table',
|
||||
content: [{ type: 'tableRow', content: [cell('x'), cell('y')] }],
|
||||
},
|
||||
);
|
||||
expect(jsonToText(d, { deterministic: true })).toMatchInlineSnapshot(`
|
||||
"Config
|
||||
key = value
|
||||
|
||||
|
||||
one
|
||||
|
||||
two
|
||||
[image]
|
||||
[table 1x2]"
|
||||
`);
|
||||
});
|
||||
|
||||
it('SCENARIO: a config written as a code block reads back byte-identical (diff empty)', () => {
|
||||
const config =
|
||||
'export TICKET_LIFETIME=$2592000\nservers:\n - www.internal.host';
|
||||
const d = doc({
|
||||
type: 'codeBlock',
|
||||
attrs: { language: 'yaml' },
|
||||
content: [text(config)],
|
||||
});
|
||||
// The code block is one block; its text (dollars, bare domain, newlines) is
|
||||
// preserved verbatim, so a read-as-text of a stored config diffs empty.
|
||||
expect(jsonToText(d, { deterministic: true })).toBe(config);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,52 @@
|
||||
import { join } from 'path';
|
||||
import * as fs from 'node:fs';
|
||||
import * as os from 'node:os';
|
||||
import { readClientBuildVersion } from './client-version';
|
||||
|
||||
describe('readClientBuildVersion', () => {
|
||||
let dir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
dir = fs.mkdtempSync(join(os.tmpdir(), 'client-version-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
fs.rmSync(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
const writeVersionJson = (content: string) =>
|
||||
fs.writeFileSync(join(dir, 'version.json'), content);
|
||||
|
||||
it('returns the version from a valid version.json', () => {
|
||||
writeVersionJson(JSON.stringify({ version: 'test-A' }));
|
||||
expect(readClientBuildVersion(dir)).toBe('test-A');
|
||||
});
|
||||
|
||||
it('trims surrounding whitespace in the version', () => {
|
||||
writeVersionJson(JSON.stringify({ version: ' v1.2.3 ' }));
|
||||
expect(readClientBuildVersion(dir)).toBe('v1.2.3');
|
||||
});
|
||||
|
||||
it('returns "" when version.json is missing', () => {
|
||||
expect(readClientBuildVersion(dir)).toBe('');
|
||||
});
|
||||
|
||||
it('returns "" on malformed JSON', () => {
|
||||
writeVersionJson('{ not json');
|
||||
expect(readClientBuildVersion(dir)).toBe('');
|
||||
});
|
||||
|
||||
it('returns "" when the version field is absent', () => {
|
||||
writeVersionJson(JSON.stringify({ notVersion: 'x' }));
|
||||
expect(readClientBuildVersion(dir)).toBe('');
|
||||
});
|
||||
|
||||
it('returns "" when the version field is not a string', () => {
|
||||
writeVersionJson(JSON.stringify({ version: 123 }));
|
||||
expect(readClientBuildVersion(dir)).toBe('');
|
||||
});
|
||||
|
||||
it('returns "" when the path does not exist at all', () => {
|
||||
expect(readClientBuildVersion(join(dir, 'nope'))).toBe('');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,36 @@
|
||||
import { join } from 'path';
|
||||
import * as fs from 'node:fs';
|
||||
|
||||
/**
|
||||
* Resolve the absolute path to the built client bundle directory
|
||||
* (`apps/client/dist`) shipped into the runtime image.
|
||||
*
|
||||
* The `../` depth is anchored on THIS module's compiled location
|
||||
* (`dist/common/helpers`). `integrations/static` sits at the same depth under
|
||||
* the compiled root, so both callers (StaticModule and readClientBuildVersion)
|
||||
* MUST share this single helper rather than duplicating the depth — a copy in a
|
||||
* module at a different depth would silently resolve to the wrong directory.
|
||||
*/
|
||||
export function resolveClientDistPath(): string {
|
||||
return join(__dirname, '..', '..', '..', '..', 'client/dist');
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the build version the client bundle was compiled with, from
|
||||
* `<clientDistPath>/version.json` (written by the Vite build — the single
|
||||
* source of truth shared by the baked-in `APP_VERSION` global and this file).
|
||||
*
|
||||
* Fail-safe: any error (missing file, unreadable, bad JSON, non-string
|
||||
* version) yields `''`. The caller treats an empty version as "unknown" and
|
||||
* the whole version-coherence feature stays silently inert — existing deploys
|
||||
* without the file keep working unchanged.
|
||||
*/
|
||||
export function readClientBuildVersion(clientDistPath: string): string {
|
||||
try {
|
||||
const raw = fs.readFileSync(join(clientDistPath, 'version.json'), 'utf8');
|
||||
const version = (JSON.parse(raw) as { version?: unknown }).version;
|
||||
return typeof version === 'string' ? version.trim() : '';
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
@@ -3,3 +3,4 @@ export * from './nanoid.utils';
|
||||
export * from './file.helper';
|
||||
export * from './constants';
|
||||
export * from './security-headers';
|
||||
export * from './client-version';
|
||||
|
||||
@@ -370,10 +370,12 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
|
||||
);
|
||||
});
|
||||
|
||||
// #490 reactive branch: a provider CONTEXT-OVERFLOW 400 in onError is classified,
|
||||
// records a distinguishable cause, and stamps metadata.replayOverflow so the NEXT
|
||||
// turn's budgeter trims aggressively (the recovery that un-bricks the chat).
|
||||
it('#490: a context-overflow 400 stamps replayOverflow on the finalized row', async () => {
|
||||
// #490/#520 reactive branch: a provider CONTEXT-OVERFLOW 400 in onError is
|
||||
// classified, records a distinguishable cause, and stamps the consecutive-overflow
|
||||
// COUNTER (metadata.replayOverflowCount) so the NEXT turn's budgeter trims with
|
||||
// escalating aggression (the recovery that un-bricks the chat). This is a fresh
|
||||
// chat (empty history -> prior streak 0), so the first overflow stamps count 1.
|
||||
it('#490/#520: a context-overflow 400 stamps replayOverflowCount=1 on the finalized row', async () => {
|
||||
jest
|
||||
.spyOn(Logger.prototype, 'error')
|
||||
.mockImplementation(() => undefined as never);
|
||||
@@ -397,11 +399,14 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
|
||||
metadata: Record<string, unknown>;
|
||||
};
|
||||
expect(patch.status).toBe('error');
|
||||
expect(patch.metadata.replayOverflow).toBe(true);
|
||||
// First overflow on a fresh chat -> k = prior(0) + 1 = 1.
|
||||
expect(patch.metadata.replayOverflowCount).toBe(1);
|
||||
// The legacy boolean is no longer written (the counter supersedes it).
|
||||
expect('replayOverflow' in patch.metadata).toBe(false);
|
||||
expect(patch.metadata.error).toContain('контекстное окно');
|
||||
});
|
||||
|
||||
it('#490: a non-overflow error does NOT stamp replayOverflow', async () => {
|
||||
it('#490/#520: a non-overflow error does NOT stamp the overflow counter', async () => {
|
||||
jest
|
||||
.spyOn(Logger.prototype, 'error')
|
||||
.mockImplementation(() => undefined as never);
|
||||
@@ -412,6 +417,7 @@ describe('AiChatService.stream — abortSignal wiring (#184 F3)', () => {
|
||||
status: string;
|
||||
metadata: Record<string, unknown>;
|
||||
};
|
||||
expect('replayOverflowCount' in patch.metadata).toBe(false);
|
||||
expect('replayOverflow' in patch.metadata).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -30,13 +30,16 @@ import {
|
||||
STEP_LIMIT_NO_ANSWER_MARKER,
|
||||
OUTPUT_DEGENERATION_ERROR,
|
||||
lastAssistantContextTokens,
|
||||
lastAssistantReplayOverflow,
|
||||
lastAssistantReplayOverflowCount,
|
||||
seedActivatedTools,
|
||||
} from './ai-chat.service';
|
||||
import type { AiChatMessage, Workspace } from '@docmost/db/types/entity.types';
|
||||
import { buildSystemPrompt } from './ai-chat.prompt';
|
||||
import type { McpClientsService } from './external-mcp/mcp-clients.service';
|
||||
import { resolveEffectiveReplayThreshold } from './history-budget';
|
||||
import {
|
||||
resolveEffectiveReplayThreshold,
|
||||
REPLAY_MIN_FLOOR_TOKENS,
|
||||
} from './history-budget';
|
||||
|
||||
/**
|
||||
* Unit tests for compactToolOutput: the pure helper that shrinks tool outputs
|
||||
@@ -554,49 +557,129 @@ describe('seedActivatedTools', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('lastAssistantReplayOverflow', () => {
|
||||
describe('lastAssistantReplayOverflowCount', () => {
|
||||
const row = (
|
||||
role: string,
|
||||
metadata: Record<string, unknown> | null,
|
||||
): AiChatMessage => ({ role, metadata }) as unknown as AiChatMessage;
|
||||
|
||||
it('is true only when the LAST assistant turn overflowed', () => {
|
||||
it('reads the consecutive-overflow count from the LAST assistant turn', () => {
|
||||
expect(
|
||||
lastAssistantReplayOverflow([
|
||||
row('assistant', { replayOverflow: true }),
|
||||
lastAssistantReplayOverflowCount([
|
||||
row('assistant', { replayOverflowCount: 3 }),
|
||||
row('user', null),
|
||||
]),
|
||||
).toBe(true);
|
||||
// A recovered (later, non-overflow) assistant turn clears it.
|
||||
).toBe(3);
|
||||
// A recovered (later, non-overflow) assistant turn resets it to 0 — the read
|
||||
// stops at the most recent assistant row, which carries no count.
|
||||
expect(
|
||||
lastAssistantReplayOverflow([
|
||||
row('assistant', { replayOverflow: true }),
|
||||
lastAssistantReplayOverflowCount([
|
||||
row('assistant', { replayOverflowCount: 3 }),
|
||||
row('user', null),
|
||||
row('assistant', { contextTokens: 5 }),
|
||||
]),
|
||||
).toBe(false);
|
||||
expect(lastAssistantReplayOverflow([])).toBe(false);
|
||||
).toBe(0);
|
||||
expect(lastAssistantReplayOverflowCount([])).toBe(0);
|
||||
});
|
||||
|
||||
// #490 reactive recovery: a prior turn stamped `replayOverflow` must make the
|
||||
// NEXT turn's effective budget the AGGRESSIVE 0.5x cut — that harder trim is
|
||||
// what un-bricks a chat that just 400'd on the context window. This exercises
|
||||
// the exact wiring the service uses: read the stamp, then scale the threshold.
|
||||
it('#490: a prior replayOverflow drives the next turn to the 0.5x aggressive budget', () => {
|
||||
// BACK-COMPAT (#520): an in-flight row written by the pre-#520 boolean stamp
|
||||
// (`replayOverflow: true`, no count) reads as k=1 — the old single 0.5× behavior —
|
||||
// so a chat mid-recovery across the deploy does not regress.
|
||||
it('#520 back-compat: a legacy boolean replayOverflow reads as k=1', () => {
|
||||
expect(
|
||||
lastAssistantReplayOverflowCount([
|
||||
row('assistant', { replayOverflow: true }),
|
||||
row('user', null),
|
||||
]),
|
||||
).toBe(1);
|
||||
// A legacy row with the flag absent/false is k=0.
|
||||
expect(
|
||||
lastAssistantReplayOverflowCount([row('assistant', { contextTokens: 5 })]),
|
||||
).toBe(0);
|
||||
});
|
||||
|
||||
// A corrupt/negative persisted count never yields a negative k.
|
||||
it('clamps a corrupt negative count to 0', () => {
|
||||
expect(
|
||||
lastAssistantReplayOverflowCount([
|
||||
row('assistant', { replayOverflowCount: -4 }),
|
||||
]),
|
||||
).toBe(0);
|
||||
});
|
||||
|
||||
// #490/#520 reactive recovery: the prior consecutive-overflow count `k` drives
|
||||
// the next turn's effective budget to an ESCALATING cut (0.5**k) — each further
|
||||
// consecutive 400 tightens it, which is what un-bricks a chat that keeps
|
||||
// overflowing. This exercises the exact wiring the service uses: read the count,
|
||||
// then scale the threshold.
|
||||
it('#490/#520: the prior count drives the next turn to the escalating aggressive budget', () => {
|
||||
const history = [
|
||||
row('assistant', { replayOverflow: true }),
|
||||
row('assistant', { replayOverflowCount: 1 }),
|
||||
row('user', null),
|
||||
];
|
||||
const priorOverflowed = lastAssistantReplayOverflow(history);
|
||||
expect(priorOverflowed).toBe(true);
|
||||
// Base budget 100k -> aggressive recovery halves it to 50k this turn.
|
||||
expect(resolveEffectiveReplayThreshold(100_000, priorOverflowed)).toBe(50_000);
|
||||
const k = lastAssistantReplayOverflowCount(history);
|
||||
expect(k).toBe(1);
|
||||
// Base budget 100k -> first-overflow recovery halves it to 50k this turn.
|
||||
expect(resolveEffectiveReplayThreshold(100_000, k)).toBe(50_000);
|
||||
// A second consecutive overflow (k=2) quarters it.
|
||||
expect(resolveEffectiveReplayThreshold(100_000, 2)).toBe(25_000);
|
||||
// Odd base floors, not rounds.
|
||||
expect(resolveEffectiveReplayThreshold(99_999, true)).toBe(49_999);
|
||||
// No prior overflow -> the base budget is used verbatim (no aggressive cut).
|
||||
expect(resolveEffectiveReplayThreshold(100_000, false)).toBe(100_000);
|
||||
expect(resolveEffectiveReplayThreshold(99_999, 1)).toBe(49_999);
|
||||
// No prior overflow (k=0) -> the base budget is used verbatim (no cut).
|
||||
expect(resolveEffectiveReplayThreshold(100_000, 0)).toBe(100_000);
|
||||
// An explicit off-switch (null) is never overridden, even on recovery.
|
||||
expect(resolveEffectiveReplayThreshold(null, true)).toBeNull();
|
||||
expect(resolveEffectiveReplayThreshold(null, 3)).toBeNull();
|
||||
});
|
||||
|
||||
// #520 escalation table + convergence: the cut deepens each consecutive overflow
|
||||
// and is CLAMPED at the floor so it converges (un-bricks even against a small
|
||||
// real window), instead of the old fixed single 0.5× that stuck at 50k forever.
|
||||
it('#520: escalates and converges to the floor, un-bricking a small real window', () => {
|
||||
const base = 100_000;
|
||||
expect(resolveEffectiveReplayThreshold(base, 0)).toBe(base);
|
||||
expect(resolveEffectiveReplayThreshold(base, 1)).toBe(50_000);
|
||||
expect(resolveEffectiveReplayThreshold(base, 2)).toBe(25_000);
|
||||
expect(resolveEffectiveReplayThreshold(base, 3)).toBe(12_500);
|
||||
|
||||
// Residual-brick regression (#520): with the flat-default base (100k) and a real
|
||||
// model window of ~40k, the OLD fixed 0.5× stuck at 50k forever (> 40k -> 400s
|
||||
// again, never recovers). The escalating cut drops BELOW 40k after enough
|
||||
// consecutive overflows -> the history finally fits -> the chat un-bricks.
|
||||
const realWindow = 40_000;
|
||||
// k=1 (50k) still exceeds the window — the old behavior's terminal state.
|
||||
expect(resolveEffectiveReplayThreshold(base, 1)).toBeGreaterThan(realWindow);
|
||||
// But escalation converges under the window within a couple more turns.
|
||||
const converged = [2, 3, 4, 5].some(
|
||||
(k) => (resolveEffectiveReplayThreshold(base, k) as number) < realWindow,
|
||||
);
|
||||
expect(converged).toBe(true);
|
||||
|
||||
// Convergence is bounded BELOW by the floor: a large k never trims below it.
|
||||
for (const k of [4, 8, 20, 100]) {
|
||||
expect(resolveEffectiveReplayThreshold(base, k)).toBe(REPLAY_MIN_FLOOR_TOKENS);
|
||||
expect(
|
||||
resolveEffectiveReplayThreshold(base, k) as number,
|
||||
).toBeGreaterThanOrEqual(REPLAY_MIN_FLOOR_TOKENS);
|
||||
}
|
||||
});
|
||||
|
||||
// The floor never RAISES a legitimately small configured budget above itself —
|
||||
// that would re-overflow the very window it was configured for. Under #520 Option B
|
||||
// recovery MAY cut it BELOW itself (down to floor(0.5×base) = the old 0.5× cut).
|
||||
it('#520: never inflates a small configured budget above itself (may cut below)', () => {
|
||||
const small = 5_000; // below the floor
|
||||
const floor = Math.min(REPLAY_MIN_FLOOR_TOKENS, Math.floor(small * 0.5)); // 2500
|
||||
expect(resolveEffectiveReplayThreshold(small, 0)).toBe(small);
|
||||
// Even under escalation the effective threshold never exceeds the base, and never
|
||||
// drops below the absolute floor.
|
||||
for (const k of [1, 2, 3, 10]) {
|
||||
const t = resolveEffectiveReplayThreshold(small, k) as number;
|
||||
expect(t).toBeLessThanOrEqual(small);
|
||||
expect(t).toBeGreaterThanOrEqual(floor);
|
||||
}
|
||||
// Option B: on overflow it DOES cut below the configured budget (old floor==budget
|
||||
// behavior would keep this at `small`).
|
||||
expect(resolveEffectiveReplayThreshold(small, 1)).toBe(floor);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -930,21 +1013,50 @@ describe('flushAssistant', () => {
|
||||
expect(flushed.metadata.error).toBe('boom');
|
||||
});
|
||||
|
||||
// #490 observability: the replay budgeter's decision is stamped on the turn.
|
||||
it('records replayTrimmedToTokens + replayOverflow when provided', () => {
|
||||
// #490/#520 observability: the replay budgeter's decision is stamped on the turn,
|
||||
// now including the consecutive-overflow COUNTER (#520) the next turn escalates on.
|
||||
it('records replayTrimmedToTokens + replayOverflowCount when provided', () => {
|
||||
const f = flushAssistant([], '', 'error', {
|
||||
error: 'ctx',
|
||||
replayTrimmedToTokens: 42_000,
|
||||
replayOverflow: true,
|
||||
replayOverflowCount: 2,
|
||||
});
|
||||
expect(f.metadata.replayTrimmedToTokens).toBe(42_000);
|
||||
expect(f.metadata.replayOverflow).toBe(true);
|
||||
expect(f.metadata.replayOverflowCount).toBe(2);
|
||||
});
|
||||
|
||||
it('omits the replay metadata when not provided', () => {
|
||||
const f = flushAssistant([], '', 'completed', { finishReason: 'stop' });
|
||||
expect('replayTrimmedToTokens' in f.metadata).toBe(false);
|
||||
expect('replayOverflow' in f.metadata).toBe(false);
|
||||
expect('replayOverflowCount' in f.metadata).toBe(false);
|
||||
expect('replayBelowConfiguredBudget' in f.metadata).toBe(false);
|
||||
});
|
||||
|
||||
// #520 Option B observability: the turn is marked when recovery replayed BELOW the
|
||||
// admin-configured budget, so the UI/telemetry can surface "config too large".
|
||||
it('stamps replayBelowConfiguredBudget when recovery cut below the configured budget', () => {
|
||||
const f = flushAssistant([], '', 'error', {
|
||||
error: 'ctx',
|
||||
replayOverflowCount: 2,
|
||||
replayBelowConfiguredBudget: true,
|
||||
});
|
||||
// MUTATION SENTINEL: dropping the metadata stamp in flushAssistant reddens this.
|
||||
expect(f.metadata.replayBelowConfiguredBudget).toBe(true);
|
||||
});
|
||||
|
||||
it('omits replayBelowConfiguredBudget on a normal in-budget replay', () => {
|
||||
// false/omitted must NOT leave the flag on the turn (only the below-budget case).
|
||||
const off = flushAssistant([], '', 'completed', {
|
||||
replayBelowConfiguredBudget: false,
|
||||
});
|
||||
expect('replayBelowConfiguredBudget' in off.metadata).toBe(false);
|
||||
});
|
||||
|
||||
// A clean finalize (no overflow -> count 0/omitted) leaves NO counter, which the
|
||||
// next turn reads as k=0 — the reset that ends a recovery streak.
|
||||
it('omits replayOverflowCount for a zero/absent count (reset semantics)', () => {
|
||||
const zero = flushAssistant([], '', 'completed', { replayOverflowCount: 0 });
|
||||
expect('replayOverflowCount' in zero.metadata).toBe(false);
|
||||
});
|
||||
|
||||
// #274 observability: the page-change diff the agent saw this turn is persisted
|
||||
|
||||
@@ -141,9 +141,10 @@ const OUTPUT_DEGENERATION_ERROR =
|
||||
|
||||
// Prefix recorded on the assistant row when the provider rejected the turn for
|
||||
// CONTEXT OVERFLOW (#490): the replayed history exceeded the model's window. The
|
||||
// row is ALSO stamped `metadata.replayOverflow` so the NEXT turn's budgeter trims
|
||||
// aggressively (the reactive recovery — the overflowing turn had no usage signal
|
||||
// to trigger preventive trimming, so the classified 400 is what un-bricks it).
|
||||
// row is ALSO stamped `metadata.replayOverflowCount` (the consecutive-overflow
|
||||
// counter, #520) so the NEXT turn's budgeter trims with escalating aggression (the
|
||||
// reactive recovery — the overflowing turn had no usage signal to trigger
|
||||
// preventive trimming, so the classified 400 is what un-bricks it).
|
||||
export const CONTEXT_OVERFLOW_ERROR_PREFIX =
|
||||
'Диалог превысил контекстное окно модели; история будет агрессивно ' +
|
||||
'сокращена на следующем ходу.';
|
||||
@@ -1228,27 +1229,52 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
|
||||
}
|
||||
// Last turn's provider-reported context size (authoritative when present).
|
||||
const priorContextTokens = lastAssistantContextTokens(oldHistory);
|
||||
// Reactive recovery (#490): if the LAST turn was rejected for context
|
||||
// overflow (stamped by onError), trim AGGRESSIVELY this turn — the
|
||||
// overflowing turn produced no usage signal, so a normal-threshold trim may
|
||||
// not shrink enough to fit. This is what un-bricks a chat that just 400'd.
|
||||
const priorOverflowed = lastAssistantReplayOverflow(oldHistory);
|
||||
// Reactive recovery (#490/#520): `k` = how many CONSECUTIVE preceding turns
|
||||
// were rejected for context overflow (stamped by onError). Each consecutive
|
||||
// overflow trims MORE aggressively (resolveEffectiveReplayThreshold scales the
|
||||
// budget by 0.5**k, clamped at the floor) so recovery ESCALATES until the
|
||||
// history fits — the overflowing turn produced no usage signal, so a single
|
||||
// fixed cut may not shrink enough when the real model window is small. This is
|
||||
// what un-bricks a chat that keeps 400'ing on the context window.
|
||||
const priorOverflowCount = lastAssistantReplayOverflowCount(oldHistory);
|
||||
const effectiveThreshold = resolveEffectiveReplayThreshold(
|
||||
replayBudget.thresholdTokens,
|
||||
priorOverflowed,
|
||||
priorOverflowCount,
|
||||
);
|
||||
if (priorOverflowed) {
|
||||
this.logger.warn(
|
||||
`AI chat (chat ${chatId}): previous turn hit context overflow; ` +
|
||||
`applying aggressive replay budget (${effectiveThreshold} tokens).`,
|
||||
);
|
||||
// #520 Option B: recovery MAY cut the replay budget BELOW the admin-configured
|
||||
// window when the provider keeps proving non-fit (the anti-brick invariant of
|
||||
// the reactive branch beats a config that a 400 has disproved). Surface it so
|
||||
// the admin sees their window is factually too large, and mark the turn so the
|
||||
// UI/telemetry can show it. Only true when actually below (not on in-budget
|
||||
// escalation edge cases); k>0 implies below for any budget >= 2 tokens.
|
||||
const replayBelowConfiguredBudget =
|
||||
replayBudget.thresholdTokens != null &&
|
||||
effectiveThreshold != null &&
|
||||
effectiveThreshold < replayBudget.thresholdTokens;
|
||||
if (priorOverflowCount > 0) {
|
||||
if (replayBelowConfiguredBudget) {
|
||||
this.logger.warn(
|
||||
`AI chat (chat ${chatId}): configured replay budget ` +
|
||||
`(${replayBudget.thresholdTokens} tokens) does not fit the model — ` +
|
||||
`replaying BELOW it at ${effectiveThreshold} tokens after ` +
|
||||
`${priorOverflowCount} consecutive context overflow(s) ` +
|
||||
`(escalation level ${priorOverflowCount}). Lower chatContextWindow ` +
|
||||
`to match the model's real context window.`,
|
||||
);
|
||||
} else {
|
||||
this.logger.warn(
|
||||
`AI chat (chat ${chatId}): ${priorOverflowCount} consecutive context ` +
|
||||
`overflow(s); applying escalated aggressive replay budget ` +
|
||||
`(${effectiveThreshold} tokens).`,
|
||||
);
|
||||
}
|
||||
}
|
||||
const preTrim = trimHistoryForReplay(
|
||||
messages,
|
||||
effectiveThreshold,
|
||||
// A prior OVERFLOW means the provider count is stale/absent — force the
|
||||
// char-estimate path by ignoring priorContextTokens on recovery.
|
||||
priorOverflowed ? undefined : priorContextTokens,
|
||||
priorOverflowCount > 0 ? undefined : priorContextTokens,
|
||||
);
|
||||
messages = preTrim.messages;
|
||||
// Observability (#490): record the budgeter's decision on the turn so the UI
|
||||
@@ -1903,6 +1929,10 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
|
||||
pageChanged,
|
||||
partsCache,
|
||||
replayTrimmedToTokens,
|
||||
// #520 Option B: mark the turn when recovery replayed BELOW the
|
||||
// configured budget (only when it actually did).
|
||||
replayBelowConfiguredBudget:
|
||||
replayBelowConfiguredBudget || undefined,
|
||||
}),
|
||||
);
|
||||
// #184/#487: the RUN is finalized ALWAYS (never gated on the message).
|
||||
@@ -1981,7 +2011,16 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
|
||||
pageChanged,
|
||||
partsCache,
|
||||
replayTrimmedToTokens,
|
||||
replayOverflow: overflow || undefined,
|
||||
// #520: escalate the consecutive-overflow counter so the NEXT turn
|
||||
// trims MORE aggressively (0.5**k). k grows by 1 each consecutive
|
||||
// overflow; a clean finalize omits the field, resetting it to 0.
|
||||
replayOverflowCount: overflow
|
||||
? priorOverflowCount + 1
|
||||
: undefined,
|
||||
// #520 Option B: mark the turn when THIS replay was already below the
|
||||
// configured budget (only when it actually was).
|
||||
replayBelowConfiguredBudget:
|
||||
replayBelowConfiguredBudget || undefined,
|
||||
}),
|
||||
);
|
||||
// #184: settle the RUN as failed, carrying the provider/transport cause.
|
||||
@@ -2413,22 +2452,39 @@ export function seedActivatedTools(
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether the most recent assistant turn was rejected for CONTEXT OVERFLOW
|
||||
* (#490): its row carries `metadata.replayOverflow` (stamped by the stream's
|
||||
* onError). The next turn's budgeter reads this to trim aggressively — the
|
||||
* reactive recovery. Only the LAST assistant turn matters (an older overflow was
|
||||
* already recovered), so we stop at the first assistant row scanning backwards.
|
||||
* How many CONSECUTIVE recent turns were rejected for CONTEXT OVERFLOW (#490/#520):
|
||||
* `k`, read from the most recent assistant row's `metadata.replayOverflowCount`
|
||||
* (stamped by the stream's onError, incremented each consecutive overflow and reset
|
||||
* to 0 on any clean finalize). The next turn's budgeter feeds this to
|
||||
* {@link resolveEffectiveReplayThreshold} to trim with ESCALATING aggression — the
|
||||
* reactive recovery. Only the LAST assistant turn matters (its count already carries
|
||||
* the consecutive streak; an older overflow followed by a clean turn was recovered),
|
||||
* so we stop at the first assistant row scanning backwards.
|
||||
*
|
||||
* BACK-COMPAT: a row written by the pre-#520 boolean stamp (`replayOverflow: true`,
|
||||
* no count) is read as k=1 — the old single 0.5× behavior — so in-flight chats do
|
||||
* not regress across the deploy.
|
||||
*/
|
||||
export function lastAssistantReplayOverflow(
|
||||
export function lastAssistantReplayOverflowCount(
|
||||
history: ReadonlyArray<AiChatMessage>,
|
||||
): boolean {
|
||||
): number {
|
||||
for (let i = history.length - 1; i >= 0; i--) {
|
||||
const row = history[i];
|
||||
if (row.role !== 'assistant') continue;
|
||||
const meta = (row.metadata ?? {}) as { replayOverflow?: unknown };
|
||||
return meta.replayOverflow === true;
|
||||
const meta = (row.metadata ?? {}) as {
|
||||
replayOverflowCount?: unknown;
|
||||
replayOverflow?: unknown;
|
||||
};
|
||||
if (typeof meta.replayOverflowCount === 'number') {
|
||||
// Guard against a corrupt/negative persisted value.
|
||||
return meta.replayOverflowCount > 0
|
||||
? Math.floor(meta.replayOverflowCount)
|
||||
: 0;
|
||||
}
|
||||
// Back-compat: legacy boolean stamp -> one overflow (0.5× cut).
|
||||
return meta.replayOverflow === true ? 1 : 0;
|
||||
}
|
||||
return false;
|
||||
return 0;
|
||||
}
|
||||
|
||||
/** The last message with role 'user' from a useChat payload, if any. */
|
||||
@@ -3084,9 +3140,16 @@ export function flushAssistant(
|
||||
// the (estimated) token size it trimmed to — the UI can show "replay truncated
|
||||
// at N tokens". Omitted when nothing was trimmed.
|
||||
replayTrimmedToTokens?: number;
|
||||
// #490 reactive branch: set when the provider rejected this turn for context
|
||||
// overflow. Stamped into metadata so the NEXT turn's budgeter trims aggressively.
|
||||
replayOverflow?: boolean;
|
||||
// #490/#520 reactive branch: the consecutive context-overflow count for THIS
|
||||
// turn (prior streak + 1) when the provider rejected it for context overflow.
|
||||
// Stamped into metadata so the NEXT turn's budgeter trims with escalating
|
||||
// aggression (0.5**k). Omitted (undefined) on a clean turn, which resets k to 0.
|
||||
replayOverflowCount?: number;
|
||||
// #520 Option B observability: true when recovery replayed this turn's history
|
||||
// BELOW the admin-configured budget (the configured window did not fit and the
|
||||
// anti-brick invariant cut past it). Omitted on a normal in-budget replay so the
|
||||
// flag marks ONLY the "config is factually too large" case.
|
||||
replayBelowConfiguredBudget?: boolean;
|
||||
},
|
||||
): AssistantFlush {
|
||||
const finished = capturedSteps ?? [];
|
||||
@@ -3139,7 +3202,10 @@ export function flushAssistant(
|
||||
metadata.maxContextTokens = extra.maxContextTokens;
|
||||
if (extra?.replayTrimmedToTokens)
|
||||
metadata.replayTrimmedToTokens = extra.replayTrimmedToTokens;
|
||||
if (extra?.replayOverflow) metadata.replayOverflow = true;
|
||||
if (extra?.replayOverflowCount && extra.replayOverflowCount > 0)
|
||||
metadata.replayOverflowCount = extra.replayOverflowCount;
|
||||
if (extra?.replayBelowConfiguredBudget)
|
||||
metadata.replayBelowConfiguredBudget = true;
|
||||
if (extra?.error) metadata.error = extra.error;
|
||||
// Persist the page-change diff the agent saw this turn (#274 observability),
|
||||
// so history / the Markdown export can show what the user changed. Only when
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
import { randomBytes } from 'crypto';
|
||||
import { Client } from 'pg';
|
||||
import { flushAssistant, serializeSteps } from './ai-chat.service';
|
||||
import {
|
||||
flushAssistant,
|
||||
serializeSteps,
|
||||
lastAssistantReplayOverflowCount,
|
||||
} from './ai-chat.service';
|
||||
import type { AiChatMessage } from '@docmost/db/types/entity.types';
|
||||
|
||||
/**
|
||||
* #490 write-volume regression — an OBSERVABLE-PROPERTY test on a LIVE Postgres,
|
||||
@@ -207,3 +212,111 @@ describe('#490 write-volume on a live Postgres (pg_current_wal_lsn delta)', () =
|
||||
expect(v2).toBeLessThan(v1 * 0.75);
|
||||
}, 120_000);
|
||||
});
|
||||
|
||||
/**
|
||||
* #520 reactive-recovery COUNTER lifecycle on a LIVE Postgres — proves the
|
||||
* consecutive-overflow count survives a real jsonb metadata round-trip (the persist
|
||||
* path), not just an in-memory object. flushAssistant BUILDS the row metadata, we
|
||||
* WRITE it to a jsonb column, READ it back, then reconstruct the assistant row and
|
||||
* run lastAssistantReplayOverflowCount over it — exactly the read the next turn does.
|
||||
*
|
||||
* The lifecycle proven end-to-end through pg:
|
||||
* - consecutive overflows INCREMENT k (1 -> 2 -> 3);
|
||||
* - a CLEAN finalize omits the field, which the reader treats as a RESET to 0;
|
||||
* - a legacy boolean row (`replayOverflow: true`) reads back as k=1 (back-compat).
|
||||
*/
|
||||
describe('#520 overflow-counter lifecycle on a live Postgres (jsonb round-trip)', () => {
|
||||
let client: Client | undefined;
|
||||
let available = false;
|
||||
|
||||
beforeAll(async () => {
|
||||
try {
|
||||
client = new Client(CONN);
|
||||
await client.connect();
|
||||
await client.query('SELECT 1');
|
||||
available = true;
|
||||
} catch {
|
||||
available = false;
|
||||
client = undefined;
|
||||
}
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await client?.end().catch(() => undefined);
|
||||
});
|
||||
|
||||
// Round-trip an arbitrary metadata object through a real jsonb column and read it
|
||||
// back as the reconstructed assistant row the next turn would load.
|
||||
async function roundTrip(
|
||||
c: Client,
|
||||
metadata: unknown,
|
||||
): Promise<AiChatMessage> {
|
||||
await c.query('UPDATE _wal_counter SET metadata=$1 WHERE id=1', [
|
||||
JSON.stringify(metadata),
|
||||
]);
|
||||
const back = (await c.query('SELECT metadata FROM _wal_counter WHERE id=1'))
|
||||
.rows[0].metadata as Record<string, unknown>;
|
||||
return { role: 'assistant', metadata: back } as unknown as AiChatMessage;
|
||||
}
|
||||
|
||||
it('increments across consecutive overflows, resets on a clean turn, and honors the legacy boolean', async () => {
|
||||
if (!available || !client) {
|
||||
console.warn('SKIP: gitmost-test-pg not reachable; skipping counter test.');
|
||||
return;
|
||||
}
|
||||
const c = client;
|
||||
await c.query('DROP TABLE IF EXISTS _wal_counter');
|
||||
await c.query('CREATE TABLE _wal_counter(id int primary key, metadata jsonb)');
|
||||
await c.query("INSERT INTO _wal_counter VALUES (1, '{}'::jsonb)");
|
||||
|
||||
// Turn 1 overflow: prior streak 0 -> stamp k=1 (as the service does: prior+1).
|
||||
let prior = lastAssistantReplayOverflowCount([]); // fresh chat
|
||||
expect(prior).toBe(0);
|
||||
let row = await roundTrip(
|
||||
c,
|
||||
flushAssistant([], '', 'error', {
|
||||
error: 'ctx',
|
||||
replayOverflowCount: prior + 1,
|
||||
}).metadata,
|
||||
);
|
||||
prior = lastAssistantReplayOverflowCount([row]);
|
||||
expect(prior).toBe(1);
|
||||
|
||||
// Turn 2 overflow: prior 1 -> stamp k=2.
|
||||
row = await roundTrip(
|
||||
c,
|
||||
flushAssistant([], '', 'error', {
|
||||
error: 'ctx',
|
||||
replayOverflowCount: prior + 1,
|
||||
}).metadata,
|
||||
);
|
||||
prior = lastAssistantReplayOverflowCount([row]);
|
||||
expect(prior).toBe(2);
|
||||
|
||||
// Turn 3 overflow: prior 2 -> stamp k=3.
|
||||
row = await roundTrip(
|
||||
c,
|
||||
flushAssistant([], '', 'error', {
|
||||
error: 'ctx',
|
||||
replayOverflowCount: prior + 1,
|
||||
}).metadata,
|
||||
);
|
||||
prior = lastAssistantReplayOverflowCount([row]);
|
||||
expect(prior).toBe(3);
|
||||
|
||||
// Turn 4 CLEAN finalize: no overflow -> the field is omitted -> reset to 0.
|
||||
row = await roundTrip(
|
||||
c,
|
||||
flushAssistant([], 'all good', 'completed', { finishReason: 'stop' })
|
||||
.metadata,
|
||||
);
|
||||
expect('replayOverflowCount' in (row.metadata as object)).toBe(false);
|
||||
expect(lastAssistantReplayOverflowCount([row])).toBe(0);
|
||||
|
||||
// Back-compat: a row persisted by the pre-#520 boolean stamp reads back as k=1.
|
||||
row = await roundTrip(c, { replayOverflow: true });
|
||||
expect(lastAssistantReplayOverflowCount([row])).toBe(1);
|
||||
|
||||
await c.query('DROP TABLE IF EXISTS _wal_counter');
|
||||
}, 60_000);
|
||||
});
|
||||
|
||||
@@ -1,14 +1,146 @@
|
||||
import type { ModelMessage } from 'ai';
|
||||
import {
|
||||
resolveReplayBudget,
|
||||
resolveEffectiveReplayThreshold,
|
||||
isContextOverflowError,
|
||||
estimateMessagesTokens,
|
||||
trimHistoryForReplay,
|
||||
REPLAY_BUDGET_DEFAULT_TOKENS,
|
||||
REPLAY_BUDGET_WINDOW_FRACTION,
|
||||
REPLAY_MIN_FLOOR_TOKENS,
|
||||
REPLAY_TRUNCATION_MARKER,
|
||||
REPLAY_TURN_COLLAPSED_MARKER,
|
||||
} from './history-budget';
|
||||
|
||||
describe('resolveEffectiveReplayThreshold (#520 iterative escalation)', () => {
|
||||
// The escalation table: each consecutive overflow (k) deepens the cut by 0.5×.
|
||||
it('scales the base by 0.5**k, flooring (not rounding) fractional tokens', () => {
|
||||
const base = 100_000;
|
||||
expect(resolveEffectiveReplayThreshold(base, 0)).toBe(base); // k=0: unchanged
|
||||
expect(resolveEffectiveReplayThreshold(base, 1)).toBe(50_000); // 0.5×
|
||||
expect(resolveEffectiveReplayThreshold(base, 2)).toBe(25_000); // 0.25×
|
||||
expect(resolveEffectiveReplayThreshold(base, 3)).toBe(12_500); // 0.125×
|
||||
// Floors, not rounds.
|
||||
expect(resolveEffectiveReplayThreshold(99_999, 1)).toBe(49_999);
|
||||
});
|
||||
|
||||
it('passes a null base (trimming OFF) through unchanged for any k', () => {
|
||||
for (const k of [0, 1, 2, 5, 100]) {
|
||||
expect(resolveEffectiveReplayThreshold(null, k)).toBeNull();
|
||||
}
|
||||
});
|
||||
|
||||
// The crux of #520: convergence. A large k is clamped at REPLAY_MIN_FLOOR_TOKENS,
|
||||
// so the escalation CONVERGES to a small-but-usable budget instead of trimming to
|
||||
// zero — and, unlike the old fixed 0.5× that stuck at 50k, it drops far enough to
|
||||
// fit a small real model window.
|
||||
it('clamps a large k at the floor (converges, never below)', () => {
|
||||
const base = 100_000;
|
||||
for (const k of [4, 6, 10, 50, 200]) {
|
||||
const t = resolveEffectiveReplayThreshold(base, k) as number;
|
||||
expect(t).toBe(REPLAY_MIN_FLOOR_TOKENS);
|
||||
expect(t).toBeGreaterThanOrEqual(REPLAY_MIN_FLOOR_TOKENS);
|
||||
}
|
||||
});
|
||||
|
||||
// Residual-brick regression (#520): flat-default base 100k, real window ~40k. The
|
||||
// OLD fixed single 0.5× stuck at 50k > 40k forever (re-overflows every turn — the
|
||||
// brick). The iterative cut drops BELOW 40k after a couple more consecutive
|
||||
// overflows, so the history finally fits and the chat un-bricks.
|
||||
it('un-bricks: escalation drops below a small real window the fixed 0.5× never could', () => {
|
||||
const base = 100_000;
|
||||
const realWindow = 40_000;
|
||||
// The old terminal state: 0.5× = 50k, still above the window.
|
||||
expect(resolveEffectiveReplayThreshold(base, 1)).toBeGreaterThan(realWindow);
|
||||
// Escalation converges under the window.
|
||||
const converged = [2, 3, 4, 5].some(
|
||||
(k) => (resolveEffectiveReplayThreshold(base, k) as number) < realWindow,
|
||||
);
|
||||
expect(converged).toBe(true);
|
||||
// MUTATION SENTINEL: reverting `** k` to `** 1` (fixed 0.5×) makes every k yield
|
||||
// 50k, so `converged` above would be FALSE and this test reddens. Removing the
|
||||
// floor reddens the clamp test instead.
|
||||
});
|
||||
|
||||
// "Don't INFLATE above itself" is still an invariant: the floor never RAISES a
|
||||
// legitimately small configured budget above itself (that would re-overflow the
|
||||
// very window it was set for). Under #520 Option B the floor is min(FLOOR,
|
||||
// floor(0.5×base)), so for a base BELOW the floor recovery MAY cut it further —
|
||||
// down to floor(0.5×base), i.e. AT LEAST the old single 0.5× cut — but never above.
|
||||
it('never inflates a small configured budget above itself (may cut below it, #520 Option B)', () => {
|
||||
const small = 5_000; // below REPLAY_MIN_FLOOR_TOKENS
|
||||
const floor = Math.min(REPLAY_MIN_FLOOR_TOKENS, Math.floor(small * 0.5)); // 2500
|
||||
expect(resolveEffectiveReplayThreshold(small, 0)).toBe(small); // k=0 unchanged
|
||||
for (const k of [1, 2, 3, 10]) {
|
||||
const t = resolveEffectiveReplayThreshold(small, k) as number;
|
||||
// Invariant: never RAISED above the configured budget.
|
||||
expect(t).toBeLessThanOrEqual(small);
|
||||
// Option B: never trimmed below the absolute floor (converges).
|
||||
expect(t).toBeGreaterThanOrEqual(floor);
|
||||
}
|
||||
// The old single 0.5× cut is reached (and pinned) — recovery is never WORSE than
|
||||
// before, and DOES cut below the configured budget on overflow (Option B). Under
|
||||
// the old floor==budget behavior this would be `small` (5000), not `floor`.
|
||||
expect(resolveEffectiveReplayThreshold(small, 1)).toBe(floor);
|
||||
});
|
||||
|
||||
// #520 Option B: with a configured window (NOT the flat default), repeated overflow
|
||||
// escalates the replay budget BELOW the configured budget, down to the absolute
|
||||
// floor — the chat must not brick even when the configured window is too large for
|
||||
// the model. A CLEAN turn (k back to 0) restores the full configured budget.
|
||||
it('escalates below the configured budget down to the floor, and resets on a clean turn', () => {
|
||||
// Configured context window 8000 -> budget = floor(0.7 × 8000) = 5600 (the code's
|
||||
// window->budget ratio; compute it, do not hardcode).
|
||||
const window = 8_000;
|
||||
const budget = resolveReplayBudget(window).thresholdTokens as number;
|
||||
expect(budget).toBe(Math.floor(REPLAY_BUDGET_WINDOW_FRACTION * window)); // 5600
|
||||
|
||||
// The absolute floor for this budget = min(FLOOR, floor(0.5 × budget)) = 2800
|
||||
// (a "small window": 0.5×budget < REPLAY_MIN_FLOOR_TOKENS), which is BELOW the
|
||||
// 5600 configured budget.
|
||||
const floor = Math.min(
|
||||
REPLAY_MIN_FLOOR_TOKENS,
|
||||
Math.floor(budget * 0.5),
|
||||
); // 2800
|
||||
expect(floor).toBe(2_800);
|
||||
expect(floor).toBeLessThan(budget); // below the configured budget
|
||||
|
||||
// k=0 -> full configured budget (no overflow yet).
|
||||
expect(resolveEffectiveReplayThreshold(budget, 0)).toBe(budget);
|
||||
// k=1 -> cut BELOW the configured budget to the 0.5× floor (2800). This is the
|
||||
// MUTATION SENTINEL: with the OLD floor==configured-budget behavior this would be
|
||||
// max(2800, 5600) = 5600 (never below budget), so the assertion reddens.
|
||||
expect(resolveEffectiveReplayThreshold(budget, 1)).toBe(floor);
|
||||
expect(resolveEffectiveReplayThreshold(budget, 1)).toBeLessThan(budget);
|
||||
// k>=2 stays at the floor (converges, never below it).
|
||||
for (const k of [2, 3, 5, 20]) {
|
||||
expect(resolveEffectiveReplayThreshold(budget, k)).toBe(floor);
|
||||
}
|
||||
|
||||
// A CLEAN turn resets k to 0 -> the full configured budget is restored (recovery
|
||||
// is not sticky; it only bites while the provider keeps proving non-fit).
|
||||
expect(resolveEffectiveReplayThreshold(budget, 0)).toBe(budget);
|
||||
});
|
||||
|
||||
// #520 Option B, LARGE window: a big configured budget escalates in DISTINCT steps
|
||||
// below itself, converging to the fixed absolute floor REPLAY_MIN_FLOOR_TOKENS.
|
||||
it('a large configured budget escalates below itself down to REPLAY_MIN_FLOOR_TOKENS', () => {
|
||||
const budget = resolveReplayBudget(200_000).thresholdTokens as number; // 140000
|
||||
// The floor for a large window is the fixed REPLAY_MIN_FLOOR_TOKENS (8k), well
|
||||
// BELOW the configured budget.
|
||||
expect(
|
||||
Math.min(REPLAY_MIN_FLOOR_TOKENS, Math.floor(budget * 0.5)),
|
||||
).toBe(REPLAY_MIN_FLOOR_TOKENS);
|
||||
const seq = [0, 1, 2, 3, 4, 5, 6].map(
|
||||
(k) => resolveEffectiveReplayThreshold(budget, k) as number,
|
||||
);
|
||||
expect(seq).toEqual([140_000, 70_000, 35_000, 17_500, 8_750, 8_000, 8_000]);
|
||||
// Every escalated step (k>=1) is below the configured budget; convergence floor.
|
||||
for (const t of seq.slice(1)) expect(t).toBeLessThan(budget);
|
||||
expect(seq[seq.length - 1]).toBe(REPLAY_MIN_FLOOR_TOKENS);
|
||||
});
|
||||
});
|
||||
|
||||
describe('resolveReplayBudget', () => {
|
||||
it('uses floor(0.7 x window) for a configured window (no cap)', () => {
|
||||
// 0.7 x 60k = 42k
|
||||
|
||||
@@ -22,10 +22,35 @@ export const REPLAY_BUDGET_DEFAULT_TOKENS = 100_000;
|
||||
/** Fraction of a configured context window used as the budget. */
|
||||
export const REPLAY_BUDGET_WINDOW_FRACTION = 0.7;
|
||||
/**
|
||||
* Fraction of the normal budget used for the REACTIVE re-trim after a provider
|
||||
* context-overflow 400 — the preventive estimate under-counted, so cut harder.
|
||||
* Per-step fraction of the normal budget applied on the REACTIVE re-trim after a
|
||||
* provider context-overflow 400 — the preventive estimate under-counted, so cut
|
||||
* harder. This is now applied ITERATIVELY: with `k` consecutive overflow turns the
|
||||
* budget is scaled by `fraction ** k` (k=1 -> 0.5×, k=2 -> 0.25×, …), so recovery
|
||||
* ESCALATES turn over turn until the replayed history finally fits, instead of the
|
||||
* old single fixed 0.5× cut that could never un-brick a chat whose real model
|
||||
* window is smaller than 0.5 × the (unconfigured, flat-default) base budget (#520).
|
||||
*/
|
||||
export const REPLAY_AGGRESSIVE_FRACTION = 0.5;
|
||||
/**
|
||||
* Absolute lower bound (tokens) on the escalating reactive budget: the iterative
|
||||
* cut is clamped here so it CONVERGES (a fixed floor, not an ever-shrinking value
|
||||
* that would eventually trim everything). Rationale: below ~8k tokens a chat cannot
|
||||
* carry meaningful recent context, and even a small real model window comfortably
|
||||
* fits this much — keep-recent-turns still applies on top, so a handful of recent
|
||||
* turns survive.
|
||||
*
|
||||
* #520 Option B: for a LARGE configured window this floor sits BELOW the configured
|
||||
* replay budget, and recovery is allowed to escalate PAST the configured budget down
|
||||
* to it — "the chat must not brick on overflow" is an INVARIANT of the reactive
|
||||
* branch, and a configured window is a claim about model capacity, not a promise
|
||||
* about replay size; once the provider proves non-fit with a 400, reality beats the
|
||||
* config. This does NOT change the NORMAL path's upper bound (#510 Option A still
|
||||
* respects the configured budget while it fits) — it is survival once non-fit is
|
||||
* proven. For a SMALL configured window the effective floor is instead the old
|
||||
* single 0.5× cut (never worse than before, and never RAISED above the budget); see
|
||||
* {@link resolveEffectiveReplayThreshold}.
|
||||
*/
|
||||
export const REPLAY_MIN_FLOOR_TOKENS = 8_000;
|
||||
/**
|
||||
* Turns (a user message + its assistant/tool replies) kept FULL at the tail,
|
||||
* including the current one — never trimmed. Older turns are compacted first.
|
||||
@@ -85,22 +110,48 @@ export function resolveReplayBudget(rawContextWindow: unknown): ReplayBudget {
|
||||
}
|
||||
|
||||
/**
|
||||
* The effective replay threshold for THIS turn, given the base budget and whether
|
||||
* the PREVIOUS turn hit a context-overflow 400 (the reactive-recovery signal,
|
||||
* `metadata.replayOverflow`). On recovery the base budget is scaled down by
|
||||
* {@link REPLAY_AGGRESSIVE_FRACTION}: the overflowing turn produced no usage
|
||||
* signal, so the preventive estimate under-counted and a normal-threshold trim may
|
||||
* not shrink enough to fit — this harder cut is what un-bricks the chat.
|
||||
* The effective replay threshold for THIS turn, given the base budget and `k` — the
|
||||
* number of CONSECUTIVE preceding turns that hit a context-overflow 400 (the
|
||||
* reactive-recovery signal `metadata.replayOverflowCount`, read from the last
|
||||
* assistant row). On recovery the base budget is scaled down ITERATIVELY by
|
||||
* {@link REPLAY_AGGRESSIVE_FRACTION} ** k and clamped at {@link REPLAY_MIN_FLOOR_TOKENS}:
|
||||
* - k=0 -> base unchanged (no overflow: nothing to recover from).
|
||||
* - k=1 -> floor(0.5 × base); k=2 -> floor(0.25 × base); … each further consecutive
|
||||
* overflow tightens the cut, so recovery ESCALATES until the history fits.
|
||||
* - the escalation is clamped at the floor so it CONVERGES — this is what un-bricks
|
||||
* a chat whose real model window is smaller than a single 0.5× cut of the base
|
||||
* (e.g. an unconfigured window: flat-default base 100k, real window <50k) (#520).
|
||||
*
|
||||
* The overflowing turn produced no usage signal, so the preventive estimate
|
||||
* under-counted and a normal-threshold (or single fixed 0.5×) trim may not shrink
|
||||
* enough to fit; the escalating cut is what recovers such a chat.
|
||||
*
|
||||
* A `null` base budget (trimming OFF) is passed through unchanged: an explicit
|
||||
* off-switch is never overridden by the recovery path.
|
||||
*
|
||||
* The absolute floor is `min(REPLAY_MIN_FLOOR_TOKENS, floor(0.5 × base))` (#520
|
||||
* Option B):
|
||||
* - LARGE window (floor(0.5 × base) >= REPLAY_MIN_FLOOR_TOKENS) -> the fixed
|
||||
* REPLAY_MIN_FLOOR_TOKENS, which is BELOW the configured budget: escalation is
|
||||
* ALLOWED to cut past the configured budget down to this absolute minimum, so a
|
||||
* chat whose real model window is far smaller than the configured window still
|
||||
* un-bricks (the invariant beats a config reality disproved with a 400).
|
||||
* - SMALL window (floor(0.5 × base) < REPLAY_MIN_FLOOR_TOKENS) -> floor(0.5 ×
|
||||
* base), i.e. AT LEAST the old single 0.5× cut: recovery is never WORSE than the
|
||||
* pre-#520 behavior, and the floor is still never RAISED above the configured
|
||||
* budget itself (that would re-overflow the very window it was set for).
|
||||
*/
|
||||
export function resolveEffectiveReplayThreshold(
|
||||
thresholdTokens: number | null,
|
||||
priorOverflowed: boolean,
|
||||
k: number,
|
||||
): number | null {
|
||||
if (!priorOverflowed || thresholdTokens == null) return thresholdTokens;
|
||||
return Math.floor(thresholdTokens * REPLAY_AGGRESSIVE_FRACTION);
|
||||
if (thresholdTokens == null || k <= 0) return thresholdTokens;
|
||||
const scaled = Math.floor(thresholdTokens * REPLAY_AGGRESSIVE_FRACTION ** k);
|
||||
const floor = Math.min(
|
||||
REPLAY_MIN_FLOOR_TOKENS,
|
||||
Math.floor(thresholdTokens * REPLAY_AGGRESSIVE_FRACTION),
|
||||
);
|
||||
return Math.max(scaled, floor);
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -210,6 +210,55 @@ describe('PublicShareChatToolsService.forShare', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('getSharePage under approved publication (owner decision: assistant sees the frozen saved version)', () => {
|
||||
it('reads the FROZEN saved content/title returned by the boundary, not a live re-fetch', async () => {
|
||||
// #370 Stage B owner decision: ALL public reads — including the share
|
||||
// assistant's page-read tool — see the published saved version, so a
|
||||
// visitor and the assistant see the same bytes. The content swap lives in
|
||||
// resolveReadableSharePage (the single canonical boundary), so the tool
|
||||
// simply consumes the frozen { page } it returns: the assistant must never
|
||||
// re-read the live draft behind the boundary's back.
|
||||
const frozenContent = {
|
||||
type: 'doc',
|
||||
content: [
|
||||
{
|
||||
type: 'paragraph',
|
||||
content: [{ type: 'text', text: 'FROZEN saved version body' }],
|
||||
},
|
||||
],
|
||||
};
|
||||
// The boundary already swapped content+title to the manual version while
|
||||
// preserving the live id (its unit test pins that); the tool sees this.
|
||||
const frozenPage = {
|
||||
id: 'page-1',
|
||||
title: 'Saved Version Title',
|
||||
deletedAt: null,
|
||||
content: frozenContent,
|
||||
};
|
||||
|
||||
const { svc, shareService } = makeService({
|
||||
resolveReadableSharePage: jest
|
||||
.fn()
|
||||
.mockResolvedValue({ share: { id: 'SHARE-A' }, page: frozenPage }),
|
||||
});
|
||||
// updatePublicAttachments sanitizes the FROZEN page it was handed.
|
||||
shareService.updatePublicAttachments.mockResolvedValue(frozenContent);
|
||||
|
||||
const tools = svc.forShare('SHARE-A', 'ws-1');
|
||||
const out = (await (tools.getSharePage as unknown as ToolExec).execute({
|
||||
pageId: 'page-1',
|
||||
})) as { title: string; markdown: string };
|
||||
|
||||
// Sanitizer is handed the frozen page (never a separate live fetch).
|
||||
expect(shareService.updatePublicAttachments).toHaveBeenCalledWith(
|
||||
frozenPage,
|
||||
);
|
||||
// Title + body come from the saved (frozen) version.
|
||||
expect(out.title).toBe('Saved Version Title');
|
||||
expect(out.markdown).toContain('FROZEN saved version body');
|
||||
});
|
||||
});
|
||||
|
||||
describe('getSharePage non-resolving page (deleted / restricted / out-of-share)', () => {
|
||||
it('resolveReadableSharePage returns null (e.g. soft-deleted page) => generic error, NO content sanitized/returned', async () => {
|
||||
// The canonical boundary 404s a soft-deleted / restricted / out-of-tree
|
||||
|
||||
@@ -10,6 +10,12 @@ import { Transform } from 'class-transformer';
|
||||
|
||||
export type ContentFormat = 'json' | 'markdown' | 'html';
|
||||
|
||||
// READ-only rendering formats for `getPage` (#502). A superset of the writable
|
||||
// `ContentFormat` with `text` — a flat, deterministic, machine-diffable text
|
||||
// rendering — added. Kept SEPARATE from `ContentFormat` so the write path
|
||||
// (createPage/updatePage `parseProsemirrorContent`) can never be handed `text`.
|
||||
export type PageReadFormat = ContentFormat | 'text';
|
||||
|
||||
export class CreatePageDto {
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
|
||||
@@ -5,10 +5,11 @@ import {
|
||||
IsOptional,
|
||||
IsString,
|
||||
IsUUID,
|
||||
MaxLength,
|
||||
} from 'class-validator';
|
||||
import { Transform } from 'class-transformer';
|
||||
|
||||
import { ContentFormat } from './create-page.dto';
|
||||
import { PageReadFormat } from './create-page.dto';
|
||||
import { IsPageIdOrSlugId } from './page-identity.validator';
|
||||
|
||||
export class PageIdDto {
|
||||
@@ -43,8 +44,19 @@ export class PageInfoDto extends PageIdDto {
|
||||
|
||||
@IsOptional()
|
||||
@Transform(({ value }) => value?.toLowerCase())
|
||||
@IsIn(['json', 'markdown', 'html'])
|
||||
format?: ContentFormat;
|
||||
@IsIn(['json', 'markdown', 'html', 'text'])
|
||||
format?: PageReadFormat;
|
||||
}
|
||||
|
||||
export class PageWorkTimeDto extends PageIdDto {
|
||||
// Viewer IANA timezone for the per-day punch-card buckets (§6.3). Optional —
|
||||
// falls back to UTC server-side. Length-capped so a bogus value cannot bloat
|
||||
// the request; the value is only ever handed to Intl.DateTimeFormat, which
|
||||
// throws on an unknown zone (caught by the controller → 400).
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
@MaxLength(64)
|
||||
tz?: string;
|
||||
}
|
||||
|
||||
export class DeletePageDto extends PageIdDto {
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
import { NotFoundException } from '@nestjs/common';
|
||||
import { PageController } from './page.controller';
|
||||
import { jsonToText, jsonToMarkdown } from '../../collaboration/collaboration.util';
|
||||
|
||||
// #502 READ: getPage `format:"text"` routes through the deterministic jsonToText
|
||||
// path (placeholders for non-text nodes, block-per-line), while `format:"json"`
|
||||
// (or none) returns the raw content and `format:"markdown"` still converts. This
|
||||
// pins the CONTROLLER wiring with lightweight mocks (no DB needed — the jsonToText
|
||||
// output contract itself is pinned in json-to-text-deterministic.spec.ts).
|
||||
|
||||
const CONTENT = {
|
||||
type: 'doc',
|
||||
content: [
|
||||
{ type: 'heading', attrs: { level: 1 }, content: [{ type: 'text', text: 'Config' }] },
|
||||
{
|
||||
type: 'paragraph',
|
||||
content: [
|
||||
{ type: 'text', text: 'key=', marks: [{ type: 'bold' }] },
|
||||
{ type: 'text', text: 'value' },
|
||||
],
|
||||
},
|
||||
{ type: 'image', attrs: { src: 's' } },
|
||||
],
|
||||
};
|
||||
|
||||
function makeController(page: any): PageController {
|
||||
const pageRepo = { findById: jest.fn().mockResolvedValue(page) } as any;
|
||||
const pageAccessService = {
|
||||
validateCanViewWithPermissions: jest
|
||||
.fn()
|
||||
.mockResolvedValue({ canEdit: true, hasRestriction: false }),
|
||||
} as any;
|
||||
// Only pageRepo + pageAccessService are exercised by getPage; the rest are
|
||||
// never touched on this path, so undefined placeholders are fine.
|
||||
return new PageController(
|
||||
undefined as any, // pageService
|
||||
pageRepo,
|
||||
undefined as any, // pageHistoryService
|
||||
undefined as any, // spaceAbility
|
||||
pageAccessService,
|
||||
undefined as any, // backlinkService
|
||||
undefined as any, // labelService
|
||||
undefined as any, // auditService
|
||||
);
|
||||
}
|
||||
|
||||
const user = { id: 'u1' } as any;
|
||||
|
||||
describe('PageController.getPage — format:"text" (#502)', () => {
|
||||
it('returns deterministic flat text with placeholders for non-text nodes', async () => {
|
||||
const controller = makeController({ id: 'p1', content: CONTENT });
|
||||
const res: any = await controller.getPage({ pageId: 'p1', format: 'text' } as any, user);
|
||||
expect(res.content).toBe(jsonToText(CONTENT, { deterministic: true }));
|
||||
expect(res.content).toBe('Config\nkey=value\n[image]');
|
||||
expect(res.permissions).toEqual({ canEdit: true, hasRestriction: false });
|
||||
});
|
||||
|
||||
it('markdown format still converts to markdown (unchanged)', async () => {
|
||||
const controller = makeController({ id: 'p1', content: CONTENT });
|
||||
const res: any = await controller.getPage({ pageId: 'p1', format: 'markdown' } as any, user);
|
||||
expect(res.content).toBe(jsonToMarkdown(CONTENT));
|
||||
});
|
||||
|
||||
it('json / no format returns the raw ProseMirror content object', async () => {
|
||||
const controller = makeController({ id: 'p1', content: CONTENT });
|
||||
const res: any = await controller.getPage({ pageId: 'p1' } as any, user);
|
||||
expect(res.content).toEqual(CONTENT); // untouched object, not a string
|
||||
});
|
||||
|
||||
it('missing page -> NotFoundException', async () => {
|
||||
const controller = makeController(null);
|
||||
await expect(
|
||||
controller.getPage({ pageId: 'nope', format: 'text' } as any, user),
|
||||
).rejects.toBeInstanceOf(NotFoundException);
|
||||
});
|
||||
});
|
||||
@@ -1,3 +1,8 @@
|
||||
import {
|
||||
BadRequestException,
|
||||
ForbiddenException,
|
||||
NotFoundException,
|
||||
} from '@nestjs/common';
|
||||
import { PageController } from './page.controller';
|
||||
|
||||
// Direct instantiation with stub deps. The Test.createTestingModule form failed
|
||||
@@ -22,4 +27,88 @@ describe('PageController', () => {
|
||||
it('should be defined', () => {
|
||||
expect(controller).toBeDefined();
|
||||
});
|
||||
|
||||
// #395 — the work-time endpoint must be gated exactly like /history.
|
||||
describe('getPageWorkTime', () => {
|
||||
const user = { id: 'u1' } as any;
|
||||
|
||||
function build(overrides: {
|
||||
page?: any;
|
||||
validate?: jest.Mock;
|
||||
compute?: jest.Mock;
|
||||
}) {
|
||||
const pageRepo = { findById: jest.fn().mockResolvedValue(overrides.page) };
|
||||
const pageAccessService = {
|
||||
validateCanView: overrides.validate ?? jest.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
const pageHistoryService = {
|
||||
computeWorkTime:
|
||||
overrides.compute ?? jest.fn().mockResolvedValue({ workMs: 0 }),
|
||||
};
|
||||
const c = new PageController(
|
||||
{} as any,
|
||||
pageRepo as any,
|
||||
pageHistoryService as any,
|
||||
{} as any,
|
||||
pageAccessService as any,
|
||||
{} as any,
|
||||
{} as any,
|
||||
{} as any,
|
||||
);
|
||||
return { c, pageRepo, pageAccessService, pageHistoryService };
|
||||
}
|
||||
|
||||
it('404s when the page does not exist', async () => {
|
||||
const { c } = build({ page: null });
|
||||
await expect(
|
||||
c.getPageWorkTime({ pageId: 'p1' } as any, user),
|
||||
).rejects.toBeInstanceOf(NotFoundException);
|
||||
});
|
||||
|
||||
it('enforces validateCanView before computing, then delegates with tz', async () => {
|
||||
const validate = jest.fn().mockResolvedValue(undefined);
|
||||
const compute = jest.fn().mockResolvedValue({ workMs: 42 });
|
||||
const { c } = build({ page: { id: 'pg' }, validate, compute });
|
||||
const out = await c.getPageWorkTime(
|
||||
{ pageId: 'pg', tz: 'Europe/Moscow' } as any,
|
||||
user,
|
||||
);
|
||||
expect(validate).toHaveBeenCalledWith({ id: 'pg' }, user);
|
||||
expect(compute).toHaveBeenCalledWith('pg', 'Europe/Moscow');
|
||||
expect(out).toEqual({ workMs: 42 });
|
||||
});
|
||||
|
||||
it('propagates a denied view gate and does NOT reach compute (security)', async () => {
|
||||
// If validateCanView is moved AFTER computeWorkTime, the timeline of a page
|
||||
// the caller may not see would be read/estimated before the gate — this
|
||||
// locks the order: a rejecting gate must short-circuit before any compute.
|
||||
const validate = jest.fn().mockRejectedValue(new ForbiddenException());
|
||||
const compute = jest.fn().mockResolvedValue({ workMs: 1 });
|
||||
const { c, pageHistoryService } = build({
|
||||
page: { id: 'pg' },
|
||||
validate,
|
||||
compute,
|
||||
});
|
||||
await expect(
|
||||
c.getPageWorkTime({ pageId: 'pg' } as any, user),
|
||||
).rejects.toBeInstanceOf(ForbiddenException);
|
||||
expect(pageHistoryService.computeWorkTime).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('maps an unknown-timezone RangeError to a 400', async () => {
|
||||
const compute = jest.fn().mockRejectedValue(new RangeError('bad tz'));
|
||||
const { c } = build({ page: { id: 'pg' }, compute });
|
||||
await expect(
|
||||
c.getPageWorkTime({ pageId: 'pg', tz: 'X/Y' } as any, user),
|
||||
).rejects.toBeInstanceOf(BadRequestException);
|
||||
});
|
||||
|
||||
it('does not swallow a non-RangeError from the service', async () => {
|
||||
const compute = jest.fn().mockRejectedValue(new Error('db down'));
|
||||
const { c } = build({ page: { id: 'pg' }, compute });
|
||||
await expect(
|
||||
c.getPageWorkTime({ pageId: 'pg' } as any, user),
|
||||
).rejects.toThrow('db down');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -21,6 +21,7 @@ import {
|
||||
PageHistoryIdDto,
|
||||
PageIdDto,
|
||||
PageInfoDto,
|
||||
PageWorkTimeDto,
|
||||
} from './dto/page.dto';
|
||||
import { PageHistoryService } from './services/page-history.service';
|
||||
import { AuthUser } from '../../common/decorators/auth-user.decorator';
|
||||
@@ -49,6 +50,7 @@ import { AddLabelsDto, RemoveLabelDto } from '../label/dto/label.dto';
|
||||
import {
|
||||
jsonToHtml,
|
||||
jsonToMarkdown,
|
||||
jsonToText,
|
||||
} from '../../collaboration/collaboration.util';
|
||||
import { AuditEvent, AuditResource } from '../../common/events/audit-events';
|
||||
import {
|
||||
@@ -93,10 +95,16 @@ export class PageController {
|
||||
const permissions = { canEdit, hasRestriction };
|
||||
|
||||
if (dto.format && dto.format !== 'json' && page.content) {
|
||||
const contentOutput =
|
||||
dto.format === 'markdown'
|
||||
? jsonToMarkdown(page.content)
|
||||
: jsonToHtml(page.content);
|
||||
let contentOutput: string;
|
||||
if (dto.format === 'markdown') {
|
||||
contentOutput = jsonToMarkdown(page.content);
|
||||
} else if (dto.format === 'text') {
|
||||
// #502: flat, deterministic, machine-diffable text (block-per-line,
|
||||
// inline marks/anchors dropped, non-text nodes -> stable placeholders).
|
||||
contentOutput = jsonToText(page.content, { deterministic: true });
|
||||
} else {
|
||||
contentOutput = jsonToHtml(page.content);
|
||||
}
|
||||
return {
|
||||
...page,
|
||||
content: contentOutput,
|
||||
@@ -524,6 +532,32 @@ export class PageController {
|
||||
return this.pageHistoryService.findHistoryByPageId(page.id, pagination);
|
||||
}
|
||||
|
||||
@HttpCode(HttpStatus.OK)
|
||||
@Post('/history/time')
|
||||
async getPageWorkTime(
|
||||
@Body() dto: PageWorkTimeDto,
|
||||
@AuthUser() user: User,
|
||||
) {
|
||||
const page = await this.pageRepo.findById(dto.pageId);
|
||||
if (!page) {
|
||||
throw new NotFoundException('Page not found');
|
||||
}
|
||||
|
||||
// Same view gate as /history and /history/info.
|
||||
await this.pageAccessService.validateCanView(page, user);
|
||||
|
||||
try {
|
||||
return await this.pageHistoryService.computeWorkTime(page.id, dto.tz);
|
||||
} catch (e) {
|
||||
// Intl.DateTimeFormat throws RangeError on an unknown IANA zone; surface
|
||||
// it as a 400 rather than a 500.
|
||||
if (e instanceof RangeError) {
|
||||
throw new BadRequestException('Invalid timezone');
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
|
||||
@HttpCode(HttpStatus.OK)
|
||||
@Post('/history/info')
|
||||
async getPageHistoryInfo(
|
||||
@@ -818,6 +852,11 @@ export class PageController {
|
||||
throw new NotFoundException('Page not found');
|
||||
}
|
||||
|
||||
// Target-only validateCanView is intentional: getPageBreadCrumbs returns
|
||||
// the full ancestor chain WITHOUT per-ancestor permission filtering. Safe
|
||||
// because page restrictions inherit down the tree, so any ancestor the
|
||||
// caller could not view would already hide the target here — see the
|
||||
// getPageBreadCrumbs docstring / #471.
|
||||
await this.pageAccessService.validateCanView(page, user);
|
||||
|
||||
return this.pageService.getPageBreadCrumbs(page.id);
|
||||
|
||||
@@ -3,6 +3,23 @@ import { PageHistoryRepo } from '@docmost/db/repos/page/page-history.repo';
|
||||
import { PageHistory } from '@docmost/db/types/entity.types';
|
||||
import { PaginationOptions } from '@docmost/db/pagination/pagination-options';
|
||||
import { CursorPaginationResult } from '@docmost/db/pagination/cursor-pagination';
|
||||
import {
|
||||
computeWorkTime,
|
||||
bucketByDay,
|
||||
DEFAULT_WORK_TIME_CONFIG,
|
||||
WorkTimeConfig,
|
||||
PerDay,
|
||||
} from '../work-time';
|
||||
|
||||
export interface PageWorkTime {
|
||||
workMs: number;
|
||||
agentOnlyMs: number;
|
||||
perDay: PerDay[];
|
||||
/** the config actually used, so the UI can show "≈" + the T_gap threshold. */
|
||||
config: WorkTimeConfig;
|
||||
/** the tz the per-day buckets were computed in (echoed back for the label). */
|
||||
tz: string;
|
||||
}
|
||||
|
||||
@Injectable()
|
||||
export class PageHistoryService {
|
||||
@@ -23,4 +40,33 @@ export class PageHistoryService {
|
||||
paginationOptions,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* #395 — estimate time worked on a page (§5) and bucket it into the viewer's
|
||||
* calendar days for the punch-card (§6.3). Reads only the cheap history
|
||||
* projection (no `content`); the estimate itself is a pure, deterministic
|
||||
* function so it is unit-tested exhaustively without a DB.
|
||||
*
|
||||
* `tz` is the viewer's IANA zone (browser locale) — it moves which day a
|
||||
* session lands in and where its windows sit, but never the total (§10).
|
||||
*/
|
||||
async computeWorkTime(
|
||||
pageId: string,
|
||||
tz = 'UTC',
|
||||
config?: Partial<WorkTimeConfig>,
|
||||
): Promise<PageWorkTime> {
|
||||
const rows = await this.pageHistoryRepo.findTimelineByPageId(pageId);
|
||||
const result = computeWorkTime(rows, config);
|
||||
const usedConfig: WorkTimeConfig = { ...DEFAULT_WORK_TIME_CONFIG, ...config };
|
||||
// `bucketByDay` consumes the pure core's un-bucketed sessions here; the
|
||||
// full session list is NOT shipped on the response (no client reads it).
|
||||
const perDay = bucketByDay(result.sessions, tz);
|
||||
return {
|
||||
workMs: result.workMs,
|
||||
agentOnlyMs: result.agentOnlyMs,
|
||||
perDay,
|
||||
config: usedConfig,
|
||||
tz,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1071,6 +1071,29 @@ export class PageService {
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk the ancestor chain of `childPageId` up to the space root, filtered
|
||||
* ONLY by `deletedAt` (+ MAX_PAGE_TREE_DEPTH) — WITHOUT per-ancestor
|
||||
* permission filtering. Callers that expose this to a user (the
|
||||
* `/breadcrumbs` endpoint) validate `validateCanView` on the TARGET page
|
||||
* only, then return the whole chain of ancestor titles (#471).
|
||||
*
|
||||
* This is safe — NOT a title leak — because page restrictions inherit DOWN
|
||||
* the tree: to view a page the caller must hold permission on EVERY
|
||||
* restricted ancestor (`validateCanView` -> `canUserAccessPage` checks the
|
||||
* full ancestor chain — see page-access.service.ts / page-permission.repo.ts
|
||||
* `canUserEditPage`). A restricted ancestor the caller may not see would
|
||||
* therefore already hide the TARGET page itself, so every ancestor reachable
|
||||
* here is one the caller is already entitled to view (content stays gated
|
||||
* regardless — getPage/getNode re-check permissions).
|
||||
*
|
||||
* Note the guarantee is the narrow "may view a descendant => may view its
|
||||
* ancestors", NOT "space membership sees every page" — restricted subtrees do
|
||||
* hide pages from members. Per-ancestor permission filtering here was
|
||||
* considered and declined as redundant given the inheritance invariant above
|
||||
* (#471). The same chain feeds the web-UI breadcrumb bar under identical CASL
|
||||
* scope.
|
||||
*/
|
||||
async getPageBreadCrumbs(childPageId: string, trx?: KyselyTransaction) {
|
||||
const ancestors = await dbOrTx(this.db, trx)
|
||||
.withRecursive('page_ancestors', (db) =>
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
import { bucketByDay, zonedDayStart } from './bucket-by-day';
|
||||
import { computeWorkTime } from './compute-work-time';
|
||||
import { WorkSession, TimelineSample } from './work-time.types';
|
||||
|
||||
const MIN = 60 * 1000;
|
||||
const HOUR = 60 * MIN;
|
||||
|
||||
function work(start: number, end: number): WorkSession {
|
||||
return { start, end, class: 'work' };
|
||||
}
|
||||
function agent(start: number, end: number): WorkSession {
|
||||
return { start, end, class: 'agent_only' };
|
||||
}
|
||||
|
||||
function sumActive(perDay: ReturnType<typeof bucketByDay>): number {
|
||||
return perDay.reduce((a, d) => a + d.activeMs, 0);
|
||||
}
|
||||
|
||||
describe('bucketByDay', () => {
|
||||
it('Σ activeMs == workMs — the §6.3 consistency invariant', () => {
|
||||
const rows: TimelineSample[] = [
|
||||
{ createdAt: '2026-07-04T03:40:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
|
||||
{ createdAt: '2026-07-04T03:49:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
|
||||
{ createdAt: '2026-07-04T18:11:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
|
||||
{ createdAt: '2026-07-06T15:34:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
|
||||
];
|
||||
const r = computeWorkTime(rows);
|
||||
const perDay = bucketByDay(r.sessions, 'UTC');
|
||||
expect(sumActive(perDay)).toBe(r.workMs);
|
||||
});
|
||||
|
||||
it('empty input → no days', () => {
|
||||
expect(bucketByDay([], 'UTC')).toEqual([]);
|
||||
});
|
||||
|
||||
it('midnight-crossing session splits across two days, sum preserved (§9#9)', () => {
|
||||
const start = Date.UTC(2026, 0, 10, 23, 14);
|
||||
const end = Date.UTC(2026, 0, 11, 0, 40);
|
||||
const perDay = bucketByDay([work(start, end)], 'UTC');
|
||||
expect(perDay).toHaveLength(2);
|
||||
expect(perDay[0].dayISO).toBe('2026-01-10');
|
||||
expect(perDay[1].dayISO).toBe('2026-01-11');
|
||||
expect(perDay[0].activeMs).toBe(46 * MIN); // 23:14 → 24:00
|
||||
expect(perDay[1].activeMs).toBe(40 * MIN); // 00:00 → 00:40
|
||||
expect(sumActive(perDay)).toBe(end - start);
|
||||
});
|
||||
|
||||
it('empty days between active days are emitted, not skipped (§9#12)', () => {
|
||||
const d1 = work(Date.UTC(2026, 0, 10, 10, 0), Date.UTC(2026, 0, 10, 11, 0));
|
||||
const d3 = work(Date.UTC(2026, 0, 12, 10, 0), Date.UTC(2026, 0, 12, 11, 0));
|
||||
const perDay = bucketByDay([d1, d3], 'UTC');
|
||||
expect(perDay.map((d) => d.dayISO)).toEqual([
|
||||
'2026-01-10',
|
||||
'2026-01-11',
|
||||
'2026-01-12',
|
||||
]);
|
||||
expect(perDay[1].activeMs).toBe(0);
|
||||
expect(perDay[1].windows).toEqual([]);
|
||||
});
|
||||
|
||||
it('agent_only windows are drawn but excluded from activeMs', () => {
|
||||
const w = work(Date.UTC(2026, 0, 10, 9, 0), Date.UTC(2026, 0, 10, 10, 0));
|
||||
const a = agent(Date.UTC(2026, 0, 10, 14, 0), Date.UTC(2026, 0, 10, 14, 30));
|
||||
const perDay = bucketByDay([w, a], 'UTC');
|
||||
expect(perDay).toHaveLength(1);
|
||||
expect(perDay[0].activeMs).toBe(1 * HOUR);
|
||||
expect(perDay[0].agentMs).toBe(30 * MIN);
|
||||
expect(perDay[0].windows.map((x) => x.class)).toEqual(['work', 'agent_only']);
|
||||
});
|
||||
|
||||
it('work and agent_only are unioned SEPARATELY (agent does not swallow work)', () => {
|
||||
// Overlapping work + agent windows on the same day.
|
||||
const w = work(Date.UTC(2026, 0, 10, 9, 0), Date.UTC(2026, 0, 10, 11, 0));
|
||||
const a = agent(Date.UTC(2026, 0, 10, 10, 0), Date.UTC(2026, 0, 10, 12, 0));
|
||||
const perDay = bucketByDay([w, a], 'UTC');
|
||||
expect(perDay[0].activeMs).toBe(2 * HOUR);
|
||||
expect(perDay[0].agentMs).toBe(2 * HOUR);
|
||||
});
|
||||
|
||||
it('overlapping same-class sessions are UNIONed, not summed (no double-count)', () => {
|
||||
// Two work sessions that overlap 10:00–10:30 on one day.
|
||||
const a = work(Date.UTC(2026, 0, 10, 9, 0), Date.UTC(2026, 0, 10, 10, 30));
|
||||
const b = work(Date.UTC(2026, 0, 10, 10, 0), Date.UTC(2026, 0, 10, 11, 0));
|
||||
const perDay = bucketByDay([a, b], 'UTC');
|
||||
expect(perDay).toHaveLength(1);
|
||||
// Union 09:00–11:00 = 2h, NOT 90m + 60m = 150m.
|
||||
expect(perDay[0].activeMs).toBe(2 * HOUR);
|
||||
// The drawn windows are also merged to one, so the punch-card cannot render
|
||||
// an overlapping double bar.
|
||||
expect(perDay[0].windows).toHaveLength(1);
|
||||
expect(perDay[0].windows[0].start).toBe(a.start);
|
||||
expect(perDay[0].windows[0].end).toBe(b.end);
|
||||
});
|
||||
|
||||
it('DST fall-back: a full 25-hour day still balances (§9#14)', () => {
|
||||
// America/New_York ends DST 2026-11-01 (25h day).
|
||||
const tz = 'America/New_York';
|
||||
const dayStart = zonedDayStart(Date.UTC(2026, 10, 1, 12, 0), tz);
|
||||
const nextStart = zonedDayStart(dayStart + 26 * HOUR, tz);
|
||||
expect(nextStart - dayStart).toBe(25 * HOUR);
|
||||
const perDay = bucketByDay([work(dayStart, nextStart)], tz);
|
||||
expect(perDay).toHaveLength(1);
|
||||
expect(perDay[0].dayISO).toBe('2026-11-01');
|
||||
expect(perDay[0].activeMs).toBe(25 * HOUR);
|
||||
expect(sumActive(perDay)).toBe(nextStart - dayStart);
|
||||
});
|
||||
|
||||
it('DST spring-forward: a full 23-hour day still balances (§9#14)', () => {
|
||||
// America/New_York starts DST 2026-03-08 (23h day).
|
||||
const tz = 'America/New_York';
|
||||
const dayStart = zonedDayStart(Date.UTC(2026, 2, 8, 12, 0), tz);
|
||||
const nextStart = zonedDayStart(dayStart + 26 * HOUR, tz);
|
||||
expect(nextStart - dayStart).toBe(23 * HOUR);
|
||||
const perDay = bucketByDay([work(dayStart, nextStart)], tz);
|
||||
expect(perDay).toHaveLength(1);
|
||||
expect(perDay[0].activeMs).toBe(23 * HOUR);
|
||||
expect(sumActive(perDay)).toBe(nextStart - dayStart);
|
||||
});
|
||||
|
||||
it('tz changes the day a session lands in but not the total', () => {
|
||||
const start = Date.UTC(2026, 0, 10, 2, 0); // 02:00 UTC
|
||||
const end = Date.UTC(2026, 0, 10, 3, 0);
|
||||
const utc = bucketByDay([work(start, end)], 'UTC');
|
||||
const ny = bucketByDay([work(start, end)], 'America/New_York'); // 21:00 prev day
|
||||
expect(utc[0].dayISO).toBe('2026-01-10');
|
||||
expect(ny[0].dayISO).toBe('2026-01-09');
|
||||
expect(sumActive(utc)).toBe(sumActive(ny));
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,180 @@
|
||||
import { WorkSession, PerDay, DayWindow } from './work-time.types';
|
||||
|
||||
/**
|
||||
* Merge intervals into a disjoint, sorted union. Overlapping OR touching
|
||||
* intervals are joined. Empty input → [].
|
||||
*/
|
||||
function union(intervals: Array<[number, number]>): Array<[number, number]> {
|
||||
if (intervals.length === 0) return [];
|
||||
const sorted = [...intervals].sort((a, b) => a[0] - b[0]);
|
||||
const out: Array<[number, number]> = [];
|
||||
let [curStart, curEnd] = sorted[0];
|
||||
for (let i = 1; i < sorted.length; i++) {
|
||||
const [s, e] = sorted[i];
|
||||
if (s <= curEnd) {
|
||||
if (e > curEnd) curEnd = e;
|
||||
} else {
|
||||
out.push([curStart, curEnd]);
|
||||
curStart = s;
|
||||
curEnd = e;
|
||||
}
|
||||
}
|
||||
out.push([curStart, curEnd]);
|
||||
return out;
|
||||
}
|
||||
|
||||
// Cache one Intl formatter per tz — constructing them is comparatively costly.
|
||||
const fmtCache = new Map<string, Intl.DateTimeFormat>();
|
||||
|
||||
function partsFmt(tz: string): Intl.DateTimeFormat {
|
||||
let fmt = fmtCache.get(tz);
|
||||
if (!fmt) {
|
||||
fmt = new Intl.DateTimeFormat('en-US', {
|
||||
timeZone: tz,
|
||||
year: 'numeric',
|
||||
month: '2-digit',
|
||||
day: '2-digit',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
second: '2-digit',
|
||||
hour12: false,
|
||||
});
|
||||
fmtCache.set(tz, fmt);
|
||||
}
|
||||
return fmt;
|
||||
}
|
||||
|
||||
interface WallParts {
|
||||
year: number;
|
||||
month: number;
|
||||
day: number;
|
||||
hour: number;
|
||||
minute: number;
|
||||
second: number;
|
||||
}
|
||||
|
||||
/** Wall-clock parts of an instant in `tz` (DST-correct, via Intl). */
|
||||
function wallParts(ms: number, tz: string): WallParts {
|
||||
const parts = partsFmt(tz).formatToParts(new Date(ms));
|
||||
const get = (type: string) =>
|
||||
Number(parts.find((p) => p.type === type)?.value ?? '0');
|
||||
let hour = get('hour');
|
||||
// Intl emits "24" for midnight under some engines/locales; normalize to 0.
|
||||
if (hour === 24) hour = 0;
|
||||
return {
|
||||
year: get('year'),
|
||||
month: get('month'),
|
||||
day: get('day'),
|
||||
hour,
|
||||
minute: get('minute'),
|
||||
second: get('second'),
|
||||
};
|
||||
}
|
||||
|
||||
/** tz offset (wall − real) at an instant, in ms. */
|
||||
function offset(ms: number, tz: string): number {
|
||||
const p = wallParts(ms, tz);
|
||||
const asUTC = Date.UTC(p.year, p.month - 1, p.day, p.hour, p.minute, p.second);
|
||||
return asUTC - ms;
|
||||
}
|
||||
|
||||
/**
|
||||
* Epoch-ms of the local-midnight day start of `ms` in `tz`. DST-correct: takes
|
||||
* the calendar day of the instant, its wall-midnight, then converts back with
|
||||
* the offset that actually applies AT that midnight (refined once). The rare
|
||||
* tz-with-a-DST-transition-exactly-at-midnight case is a documented edge (§9#14).
|
||||
*/
|
||||
export function zonedDayStart(ms: number, tz: string): number {
|
||||
const p = wallParts(ms, tz);
|
||||
const wallMidnightAsUTC = Date.UTC(p.year, p.month - 1, p.day, 0, 0, 0);
|
||||
let start = wallMidnightAsUTC - offset(ms, tz);
|
||||
// Refine with the offset at the computed midnight (DST may differ from `ms`).
|
||||
start = wallMidnightAsUTC - offset(start, tz);
|
||||
return start;
|
||||
}
|
||||
|
||||
/** The next local midnight after `dayStart` (handles 23/25h DST days). */
|
||||
function nextDayStart(dayStart: number, tz: string): number {
|
||||
// +26h always lands inside the NEXT calendar day (day length ∈ [23h,25h]),
|
||||
// never two days ahead; startOf('day') of it is the next midnight.
|
||||
return zonedDayStart(dayStart + 26 * 60 * 60 * 1000, tz);
|
||||
}
|
||||
|
||||
function isoDay(dayStart: number, tz: string): string {
|
||||
const p = wallParts(dayStart, tz);
|
||||
const pad = (n: number) => String(n).padStart(2, '0');
|
||||
return `${p.year}-${pad(p.month)}-${pad(p.day)}`;
|
||||
}
|
||||
|
||||
/** Clip a union to [lo, hi) and emit windows of `class`. */
|
||||
function clip(
|
||||
merged: Array<[number, number]>,
|
||||
lo: number,
|
||||
hi: number,
|
||||
cls: DayWindow['class'],
|
||||
): DayWindow[] {
|
||||
const out: DayWindow[] = [];
|
||||
for (const [s, e] of merged) {
|
||||
const start = Math.max(s, lo);
|
||||
const end = Math.min(e, hi);
|
||||
if (end > start) out.push({ start, end, class: cls });
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* #395 §6.3 — bucket sessions into calendar days of `tz` for the punch-card.
|
||||
* Pure and deterministic. `work` and `agent_only` are unioned SEPARATELY (else
|
||||
* agent windows would swallow work windows on overlap), then each union is split
|
||||
* at tz midnight boundaries (`startOf('day')` in tz, NOT "+24h" — DST-safe §9#14)
|
||||
* and clipped to each day.
|
||||
*
|
||||
* By construction Σ perDay.activeMs == workMs: the days are a partition of the
|
||||
* `work` union — no loss, no dup, even on 23/25h DST days. `agent_only` windows
|
||||
* are drawn but NOT in activeMs. Empty days between the first and last active day
|
||||
* are emitted (empty track + "—") so the rhythm/pauses stay visible.
|
||||
*/
|
||||
export function bucketByDay(sessions: WorkSession[], tz: string): PerDay[] {
|
||||
const uWork = union(
|
||||
sessions.filter((s) => s.class === 'work').map((s) => [s.start, s.end]),
|
||||
);
|
||||
const uAgent = union(
|
||||
sessions
|
||||
.filter((s) => s.class === 'agent_only')
|
||||
.map((s) => [s.start, s.end]),
|
||||
);
|
||||
|
||||
if (uWork.length === 0 && uAgent.length === 0) return [];
|
||||
|
||||
const minStart = Math.min(
|
||||
uWork.length ? uWork[0][0] : Infinity,
|
||||
uAgent.length ? uAgent[0][0] : Infinity,
|
||||
);
|
||||
const maxEnd = Math.max(
|
||||
uWork.length ? uWork[uWork.length - 1][1] : -Infinity,
|
||||
uAgent.length ? uAgent[uAgent.length - 1][1] : -Infinity,
|
||||
);
|
||||
|
||||
const perDay: PerDay[] = [];
|
||||
let dayStart = zonedDayStart(minStart, tz);
|
||||
// Guard against a pathological non-advancing boundary.
|
||||
let guard = 0;
|
||||
while (dayStart < maxEnd && guard < 100000) {
|
||||
guard++;
|
||||
const dayEnd = nextDayStart(dayStart, tz);
|
||||
const workWin = clip(uWork, dayStart, dayEnd, 'work');
|
||||
const agentWin = clip(uAgent, dayStart, dayEnd, 'agent_only');
|
||||
const activeMs = workWin.reduce((a, w) => a + (w.end - w.start), 0);
|
||||
const agentMs = agentWin.reduce((a, w) => a + (w.end - w.start), 0);
|
||||
const windows = [...workWin, ...agentWin].sort((a, b) => a.start - b.start);
|
||||
perDay.push({
|
||||
day: dayStart,
|
||||
dayISO: isoDay(dayStart, tz),
|
||||
activeMs,
|
||||
agentMs,
|
||||
windows,
|
||||
});
|
||||
dayStart = dayEnd;
|
||||
}
|
||||
return perDay;
|
||||
}
|
||||
@@ -0,0 +1,358 @@
|
||||
import { computeWorkTime } from './compute-work-time';
|
||||
import { bucketByDay } from './bucket-by-day';
|
||||
import { TimelineSample, WorkSession } from './work-time.types';
|
||||
|
||||
const MIN = 60 * 1000;
|
||||
|
||||
/** Union wall-clock of a set of intervals (touching intervals merge). */
|
||||
function unionMs(intervals: Array<[number, number]>): number {
|
||||
if (intervals.length === 0) return 0;
|
||||
const sorted = [...intervals].sort((a, b) => a[0] - b[0]);
|
||||
let total = 0;
|
||||
let [cs, ce] = sorted[0];
|
||||
for (let i = 1; i < sorted.length; i++) {
|
||||
const [s, e] = sorted[i];
|
||||
if (s <= ce) {
|
||||
if (e > ce) ce = e;
|
||||
} else {
|
||||
total += ce - cs;
|
||||
cs = s;
|
||||
ce = e;
|
||||
}
|
||||
}
|
||||
return total + (ce - cs);
|
||||
}
|
||||
|
||||
const ivsOf = (sessions: WorkSession[], cls?: string): Array<[number, number]> =>
|
||||
sessions
|
||||
.filter((x) => cls == null || x.class === cls)
|
||||
.map((x) => [x.start, x.end] as [number, number]);
|
||||
|
||||
function s(
|
||||
iso: string,
|
||||
opts: {
|
||||
source?: string | null;
|
||||
chat?: string | null;
|
||||
kind?: string | null;
|
||||
by?: string | null;
|
||||
} = {},
|
||||
): TimelineSample {
|
||||
return {
|
||||
createdAt: `${iso}Z`,
|
||||
lastUpdatedById: opts.by ?? 'human-1',
|
||||
lastUpdatedSource: opts.source === undefined ? 'user' : opts.source,
|
||||
lastUpdatedAiChatId: opts.chat ?? null,
|
||||
kind: opts.kind ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
// §7 config: T_gap=30m, P_in+P_out=10m, P_single=2m.
|
||||
const S7 = { tGap: 30 * MIN, agentTGap: 30 * MIN, pIn: 5 * MIN, pOut: 5 * MIN, pSingle: 2 * MIN };
|
||||
|
||||
describe('computeWorkTime', () => {
|
||||
it('§7 fixture — sessionizes 20-ish samples to ≈1h32m, not the ≈60h naive span', () => {
|
||||
const rows: TimelineSample[] = [
|
||||
// S1: multi-sample morning session
|
||||
s('2026-07-04T03:40:00'),
|
||||
s('2026-07-04T03:45:00'),
|
||||
s('2026-07-04T03:49:00'),
|
||||
// S2: agent burst (one run) then human supervising → class work
|
||||
s('2026-07-04T15:43:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
s('2026-07-04T15:47:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
s('2026-07-04T15:50:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
s('2026-07-04T16:13:00'),
|
||||
// S3: single
|
||||
s('2026-07-04T18:11:00'),
|
||||
// S4: multi-sample evening session
|
||||
s('2026-07-04T19:38:00'),
|
||||
s('2026-07-04T19:44:00'),
|
||||
s('2026-07-04T19:54:00'),
|
||||
// S5 / S6: two singles two days later, 44m apart → two sessions at T_gap=30
|
||||
s('2026-07-06T15:34:00'),
|
||||
s('2026-07-06T16:18:00'),
|
||||
];
|
||||
|
||||
const r = computeWorkTime(rows, S7);
|
||||
|
||||
// 19 + 40 + 2 + 26 + 2 + 2 = 91 minutes.
|
||||
expect(r.workMs).toBe(91 * MIN);
|
||||
expect(r.agentOnlyMs).toBe(0);
|
||||
expect(r.sessions).toHaveLength(6);
|
||||
expect(r.sessions.every((x) => x.class === 'work')).toBe(true);
|
||||
|
||||
const naiveSpan =
|
||||
new Date('2026-07-06T16:18:00Z').getTime() -
|
||||
new Date('2026-07-04T03:40:00Z').getTime();
|
||||
expect(naiveSpan).toBeGreaterThan(60 * 60 * MIN); // ≈60h
|
||||
expect(r.workMs).toBeLessThan(naiveSpan / 30); // dramatically smaller
|
||||
});
|
||||
|
||||
it('n=0 → zero, no sessions', () => {
|
||||
const r = computeWorkTime([]);
|
||||
expect(r).toEqual({ workMs: 0, agentOnlyMs: 0, sessions: [] });
|
||||
});
|
||||
|
||||
it('n=1 human → one P_single work session', () => {
|
||||
const r = computeWorkTime([s('2026-07-04T10:00:00')], S7);
|
||||
expect(r.sessions).toHaveLength(1);
|
||||
expect(r.sessions[0].class).toBe('work');
|
||||
expect(r.workMs).toBe(2 * MIN);
|
||||
expect(r.agentOnlyMs).toBe(0);
|
||||
// pre-roll only: [t − P_single, t]
|
||||
expect(r.sessions[0].end).toBe(new Date('2026-07-04T10:00:00Z').getTime());
|
||||
});
|
||||
|
||||
it('n=1 agent → one P_single agent_only session, work=0 (§9#2)', () => {
|
||||
const r = computeWorkTime(
|
||||
[s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' })],
|
||||
S7,
|
||||
);
|
||||
expect(r.sessions).toHaveLength(1);
|
||||
expect(r.sessions[0].class).toBe('agent_only');
|
||||
expect(r.workMs).toBe(0);
|
||||
expect(r.agentOnlyMs).toBe(2 * MIN);
|
||||
});
|
||||
|
||||
it('MUST close the last session — the newest session is not lost (§9#1)', () => {
|
||||
// Two singles a day apart: without the post-loop close, the 2nd is dropped.
|
||||
const rows = [s('2026-07-04T10:00:00'), s('2026-07-05T10:00:00')];
|
||||
const r = computeWorkTime(rows, S7);
|
||||
expect(r.sessions).toHaveLength(2);
|
||||
const lastStart = Math.max(...r.sessions.map((x) => x.start));
|
||||
expect(lastStart).toBe(
|
||||
new Date('2026-07-05T10:00:00Z').getTime() - 2 * MIN,
|
||||
);
|
||||
expect(r.workMs).toBe(4 * MIN);
|
||||
});
|
||||
|
||||
it('agent-burst collapse: density does not inflate — length = wall-clock', () => {
|
||||
const span = ['00', '01', '02', '03', '04', '05', '06'];
|
||||
const dense: TimelineSample[] = span.map((sec) =>
|
||||
s(`2026-07-04T10:00:${sec}`, { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
);
|
||||
const sparse: TimelineSample[] = [
|
||||
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
s('2026-07-04T10:00:06', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
];
|
||||
const rDense = computeWorkTime(dense, S7);
|
||||
const rSparse = computeWorkTime(sparse, S7);
|
||||
// Same 6-second wall-clock span → same estimate regardless of snapshot count.
|
||||
expect(rDense.agentOnlyMs).toBe(rSparse.agentOnlyMs);
|
||||
expect(rDense.sessions).toHaveLength(1);
|
||||
expect(rDense.sessions[0].class).toBe('agent_only');
|
||||
});
|
||||
|
||||
it('supervisory agent time inside a human session counts as work, not agent', () => {
|
||||
const rows = [
|
||||
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
s('2026-07-04T10:05:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
s('2026-07-04T10:12:00'), // human within T_gap
|
||||
];
|
||||
const r = computeWorkTime(rows, S7);
|
||||
expect(r.sessions).toHaveLength(1);
|
||||
expect(r.sessions[0].class).toBe('work');
|
||||
expect(r.agentOnlyMs).toBe(0);
|
||||
expect(r.workMs).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('a DIFFERENT aiChatId breaks the burst — two agent runs, idle gap excluded', () => {
|
||||
// Run c1 ends 10:05, run c2 starts 10:20 (15m > agentTGap 7m) → two sessions.
|
||||
const rows = [
|
||||
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
s('2026-07-04T10:05:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
s('2026-07-04T10:20:00', { source: 'agent', chat: 'c2', kind: 'agent' }),
|
||||
s('2026-07-04T10:25:00', { source: 'agent', chat: 'c2', kind: 'agent' }),
|
||||
];
|
||||
const r = computeWorkTime(rows); // default agentTGap = 7m
|
||||
expect(r.sessions).toHaveLength(2);
|
||||
expect(r.sessions.every((x) => x.class === 'agent_only')).toBe(true);
|
||||
// The 15m idle gap between the two runs is NOT counted.
|
||||
const run1 = 5 * MIN + 5 * MIN + 5 * MIN; // pIn + span + pOut
|
||||
expect(r.agentOnlyMs).toBe(2 * run1);
|
||||
});
|
||||
|
||||
it('idle pulse (same/null run) is a full activity sample that continues a burst', () => {
|
||||
const rows = [
|
||||
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
// idle flush 4m later, null run id → continues the burst, not a new one
|
||||
s('2026-07-04T10:04:00', { source: 'agent', chat: null, kind: 'idle' }),
|
||||
s('2026-07-04T10:08:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
];
|
||||
const r = computeWorkTime(rows);
|
||||
expect(r.sessions).toHaveLength(1);
|
||||
// burst span 10:00→10:08 (+pIn/pOut) = 8 + 10 = 18m
|
||||
expect(r.agentOnlyMs).toBe(18 * MIN);
|
||||
});
|
||||
|
||||
it('a USER-sourced idle breaks an agent burst → session is work, not agent_only', () => {
|
||||
// A human supervision idle inherits source=user (aiChatId:null) and must NOT
|
||||
// be swallowed into the agent burst. Δ=3m is within the default agentTGap so
|
||||
// the two samples stay one session — but its class flips to `work`.
|
||||
const rows = [
|
||||
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
|
||||
s('2026-07-04T10:03:00', { source: 'user', chat: null, kind: 'idle' }),
|
||||
];
|
||||
const r = computeWorkTime(rows);
|
||||
expect(r.sessions).toHaveLength(1);
|
||||
expect(r.sessions[0].class).toBe('work');
|
||||
expect(r.workMs).toBeGreaterThan(0);
|
||||
// The human idle is NOT captured as agent_only time.
|
||||
expect(r.agentOnlyMs).toBe(0);
|
||||
// Σ over `work` sessions == workMs and Σ over `agent_only` == agentOnlyMs.
|
||||
const sum = (cls: string) =>
|
||||
r.sessions
|
||||
.filter((x) => x.class === cls)
|
||||
.reduce((acc, x) => acc + (x.end - x.start), 0);
|
||||
expect(sum('work')).toBe(r.workMs);
|
||||
expect(sum('agent_only')).toBe(r.agentOnlyMs);
|
||||
});
|
||||
|
||||
it('idle pulse keeps a human writing session visible (not excluded)', () => {
|
||||
const rows = [
|
||||
s('2026-07-04T10:00:00'),
|
||||
s('2026-07-04T10:08:00', { kind: 'idle' }), // pulse within T_gap
|
||||
s('2026-07-04T10:15:00'),
|
||||
];
|
||||
const r = computeWorkTime(rows);
|
||||
expect(r.sessions).toHaveLength(1);
|
||||
expect(r.sessions[0].class).toBe('work');
|
||||
// span 10:00→10:15 + pIn/pOut = 15 + 10 = 25m
|
||||
expect(r.workMs).toBe(25 * MIN);
|
||||
});
|
||||
|
||||
it('git-source samples are excluded (§10 excludeGit)', () => {
|
||||
const rows = [
|
||||
s('2026-07-04T10:00:00', { source: 'git', kind: 'boundary' }),
|
||||
s('2026-07-04T10:01:00', { source: 'git', kind: 'boundary' }),
|
||||
];
|
||||
expect(computeWorkTime(rows).workMs).toBe(0);
|
||||
// ...but honoured off:
|
||||
expect(
|
||||
computeWorkTime(rows, { excludeGit: false }).workMs,
|
||||
).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('rejects an invalid config (tGap < pIn + pOut)', () => {
|
||||
expect(() =>
|
||||
computeWorkTime([s('2026-07-04T10:00:00')], {
|
||||
tGap: 5 * MIN,
|
||||
pIn: 5 * MIN,
|
||||
pOut: 5 * MIN,
|
||||
}),
|
||||
).toThrow(/tGap/);
|
||||
});
|
||||
|
||||
it('rejects an invalid config (2·agentTGap < pIn + pOut)', () => {
|
||||
// tGap (default 15m) still ≥ pIn+pOut, so only the 2·agentTGap guard trips.
|
||||
// Without it a short session of one class between two of the other could
|
||||
// produce a NON-adjacent cross-class overlap the adjacent-only clip misses.
|
||||
expect(() =>
|
||||
computeWorkTime([s('2026-07-04T10:00:00')], {
|
||||
agentTGap: 2 * MIN,
|
||||
pIn: 5 * MIN,
|
||||
pOut: 5 * MIN,
|
||||
}),
|
||||
).toThrow(/agentTGap/);
|
||||
});
|
||||
|
||||
// F1 — cross-class double-count. On the DEFAULT config agentTGap (7m) < pIn+pOut
|
||||
// (10m), so a `work` session ending in an agent segment and a nearby separate
|
||||
// `agent_only` run (gap in (7m,10m]) used to produce OVERLAPPING padded
|
||||
// intervals — the same wall-clock counted into BOTH workMs and agentOnlyMs. The
|
||||
// cross-class padding clip must make the two per-class unions disjoint.
|
||||
it('does NOT double-count wall-clock across work/agent_only (§F1)', () => {
|
||||
// user@0s ; agent(chatX)@60s (breaks into a work session with the human) ;
|
||||
// agent(chatY)@560s,590s (a separate agent_only run). Raw gap between the work
|
||||
// session (ends 60s) and the agent run (starts 560s) is 500s ∈ (agentTGap,
|
||||
// pIn+pOut] once padded — the classic overlap window.
|
||||
const rows: TimelineSample[] = [
|
||||
s('2026-07-04T00:00:00'), // user @ 0s
|
||||
s('2026-07-04T00:01:00', { source: 'agent', chat: 'cX', kind: 'agent' }), // @ 60s
|
||||
s('2026-07-04T00:09:20', { source: 'agent', chat: 'cY', kind: 'agent' }), // @ 560s
|
||||
s('2026-07-04T00:09:50', { source: 'agent', chat: 'cY', kind: 'agent' }), // @ 590s
|
||||
];
|
||||
const r = computeWorkTime(rows); // DEFAULT config
|
||||
|
||||
// Both classes present.
|
||||
expect(r.workMs).toBeGreaterThan(0);
|
||||
expect(r.agentOnlyMs).toBeGreaterThan(0);
|
||||
|
||||
// Per-class metrics are exactly their own union (union, not Σ).
|
||||
expect(r.workMs).toBe(unionMs(ivsOf(r.sessions, 'work')));
|
||||
expect(r.agentOnlyMs).toBe(unionMs(ivsOf(r.sessions, 'agent_only')));
|
||||
|
||||
// The F1 invariant: work-union and agent-union are cross-class-disjoint, so
|
||||
// the union of ALL padded intervals equals workMs + agentOnlyMs (no overlap).
|
||||
// With the clip disabled this fails (union < sum by the 100s overlap).
|
||||
expect(unionMs(ivsOf(r.sessions))).toBe(r.workMs + r.agentOnlyMs);
|
||||
});
|
||||
|
||||
// F1 property/fuzz — random timelines across several timezones must uphold the
|
||||
// work-time invariants. Backs the (corrected) PR claim of a real fuzz test.
|
||||
it('property: random timelines uphold union & cross-class-disjoint invariants', () => {
|
||||
// Deterministic LCG (numerical-recipes constants) so a failure is reproducible.
|
||||
let seed = 0x9e3779b9 >>> 0;
|
||||
const rand = () => {
|
||||
seed = (Math.imul(seed, 1664525) + 1013904223) >>> 0;
|
||||
return seed / 0x100000000;
|
||||
};
|
||||
const pick = <T>(arr: T[]): T => arr[Math.floor(rand() * arr.length)];
|
||||
|
||||
const tzs = [
|
||||
'UTC',
|
||||
'America/New_York',
|
||||
'Europe/Moscow',
|
||||
'Australia/Lord_Howe', // 30-min DST offset — a nasty bucket stress
|
||||
];
|
||||
const base = Date.UTC(2026, 5, 1, 0, 0, 0); // 2026-06-01Z
|
||||
const chats = ['c1', 'c2', 'c3'];
|
||||
|
||||
for (let iter = 0; iter < 250; iter++) {
|
||||
const tz = pick(tzs);
|
||||
const n = 2 + Math.floor(rand() * 18); // 2..19 rows
|
||||
const rows: TimelineSample[] = [];
|
||||
// Walk time forward by a random inter-sample gap. The gap distribution is
|
||||
// centred on the DANGEROUS band — a bit under to a bit over pIn+pOut (10m)
|
||||
// AND straddling agentTGap (7m) — so adjacent samples routinely split into
|
||||
// separate sessions whose ±P padding would overlap if a class boundary sits
|
||||
// there. Mixing user/agent classes at these gaps reliably manufactures the
|
||||
// work-ending-in-agent → agent_only cross-class boundary F1 is about, plus
|
||||
// dense within-class runs (occasional 0–2m gaps) that exercise the union.
|
||||
let t = base + Math.floor(rand() * 60 * MIN);
|
||||
for (let i = 0; i < n; i++) {
|
||||
const roll = rand();
|
||||
const gap =
|
||||
roll < 0.25
|
||||
? Math.floor(rand() * 2 * MIN) // dense burst (same-class union)
|
||||
: roll < 0.85
|
||||
? 5 * MIN + Math.floor(rand() * 8 * MIN) // 5–13m: the split band
|
||||
: 20 * MIN + Math.floor(rand() * 40 * MIN); // long idle → new day-ish
|
||||
t += gap;
|
||||
const iso = new Date(t).toISOString().slice(0, 19); // 'YYYY-MM-DDTHH:MM:SS'
|
||||
const isAgent = rand() < 0.5;
|
||||
rows.push(
|
||||
isAgent
|
||||
? s(iso, { source: 'agent', chat: pick(chats), kind: 'agent' })
|
||||
: s(iso, { source: 'user', kind: rand() < 0.3 ? 'idle' : 'manual' }),
|
||||
);
|
||||
}
|
||||
|
||||
const r = computeWorkTime(rows); // DEFAULT config
|
||||
|
||||
const workIvs = ivsOf(r.sessions, 'work');
|
||||
const agentIvs = ivsOf(r.sessions, 'agent_only');
|
||||
|
||||
// (1) each metric is exactly its per-class union (catches a union→Σ regress).
|
||||
expect(r.workMs).toBe(unionMs(workIvs));
|
||||
expect(r.agentOnlyMs).toBe(unionMs(agentIvs));
|
||||
|
||||
// (2) NO cross-class overlap: union(all) == workMs + agentOnlyMs (F1).
|
||||
expect(unionMs(ivsOf(r.sessions))).toBe(r.workMs + r.agentOnlyMs);
|
||||
|
||||
// (3) bucket invariant: Σ per-day activeMs == workMs (§6.3).
|
||||
const perDay = bucketByDay(r.sessions, tz);
|
||||
const sumActive = perDay.reduce((a, d) => a + d.activeMs, 0);
|
||||
expect(sumActive).toBe(r.workMs);
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,274 @@
|
||||
import {
|
||||
TimelineSample,
|
||||
WorkSession,
|
||||
WorkTimeResult,
|
||||
} from './work-time.types';
|
||||
import { WorkTimeConfig, resolveWorkTimeConfig } from './work-time.config';
|
||||
|
||||
/** A normalized activity sample (one history row), createdAt as epoch-ms. */
|
||||
interface NormSample {
|
||||
t: number;
|
||||
isAgent: boolean;
|
||||
aiChatId: string | null;
|
||||
kind: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* A collapsed segment: either a scalar sample (t_start == t_end) or an
|
||||
* agent-burst spanning several agent samples of one run (§5.1). It participates
|
||||
* in sessionization as a single "sample".
|
||||
*/
|
||||
interface Segment {
|
||||
tStart: number;
|
||||
tEnd: number;
|
||||
isAgent: boolean;
|
||||
}
|
||||
|
||||
function toMs(v: Date | string | number): number {
|
||||
if (v instanceof Date) return v.getTime();
|
||||
if (typeof v === 'number') return v;
|
||||
return new Date(v).getTime();
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalize raw rows → sorted, deduped activity samples. `git` is dropped when
|
||||
* configured; every other kind (incl. `idle` — the continuous-work pulse §3) is
|
||||
* a real activity sample. Sort is by createdAt ASC; samples whose timestamps
|
||||
* fall in the same `dedupRoundMs` bucket collapse to one (§9#7: a synchronous
|
||||
* boundary row + the immediate agent snapshot can share a createdAt). A merged
|
||||
* sample is human unless EVERY member is an agent, so supervision never gets
|
||||
* mis-attributed to the agent.
|
||||
*/
|
||||
function normalize(
|
||||
rows: TimelineSample[],
|
||||
config: WorkTimeConfig,
|
||||
): NormSample[] {
|
||||
const samples: NormSample[] = [];
|
||||
for (const row of rows) {
|
||||
const source = row.lastUpdatedSource;
|
||||
if (config.excludeGit && source === 'git') continue;
|
||||
samples.push({
|
||||
t: toMs(row.createdAt),
|
||||
isAgent: source === 'agent',
|
||||
aiChatId: row.lastUpdatedAiChatId ?? null,
|
||||
kind: row.kind ?? null,
|
||||
});
|
||||
}
|
||||
samples.sort((a, b) => a.t - b.t);
|
||||
|
||||
if (config.dedupRoundMs <= 0 || samples.length < 2) return samples;
|
||||
|
||||
const deduped: NormSample[] = [];
|
||||
for (const s of samples) {
|
||||
const prev = deduped[deduped.length - 1];
|
||||
if (prev && s.t - prev.t < config.dedupRoundMs) {
|
||||
// Merge into the previous sample. Human wins the class; keep the earliest
|
||||
// t; keep a non-null aiChatId if either has one (so a bare boundary row
|
||||
// does not erase the run id).
|
||||
prev.isAgent = prev.isAgent && s.isAgent;
|
||||
prev.aiChatId = prev.aiChatId ?? s.aiChatId;
|
||||
// Prefer the more specific kind (a real kind over a null/boundary) only
|
||||
// matters for burst continuation; keep prev.kind (earliest) as-is.
|
||||
continue;
|
||||
}
|
||||
deduped.push({ ...s });
|
||||
}
|
||||
return deduped;
|
||||
}
|
||||
|
||||
/**
|
||||
* Collapse consecutive same-run agent samples into one burst segment (§5.1) so a
|
||||
* dense burst (8 snapshots in 7 minutes) contributes its wall-clock, not a count
|
||||
* × block. A burst is broken by any sample NOT continuing the same aiChatId
|
||||
* agent run: a non-agent sample, a `boundary` (actor transition), or a DIFFERENT
|
||||
* aiChatId. Only an AGENT-sourced `idle` pulse with the SAME or a null aiChatId
|
||||
* continues the burst (its label lags the real edit ≤ maxWait, well within
|
||||
* rounding); a user-sourced `idle` (a human supervision pulse) breaks it.
|
||||
*/
|
||||
function collapse(samples: NormSample[], config: WorkTimeConfig): Segment[] {
|
||||
const segments: Segment[] = [];
|
||||
let burst: { chatId: string | null; tStart: number; tEnd: number } | null =
|
||||
null;
|
||||
|
||||
const flush = () => {
|
||||
if (!burst) return;
|
||||
let tEnd = burst.tEnd;
|
||||
if (config.burstCapMs != null && tEnd - burst.tStart > config.burstCapMs) {
|
||||
tEnd = burst.tStart + config.burstCapMs;
|
||||
}
|
||||
segments.push({ tStart: burst.tStart, tEnd, isAgent: true });
|
||||
burst = null;
|
||||
};
|
||||
|
||||
for (const s of samples) {
|
||||
// An agent-sourced idle pulse continues the current agent burst (same or
|
||||
// null run id). A user-sourced idle (human supervision) must NOT be swallowed
|
||||
// here — it falls through to the human branch so the session flips to `work`.
|
||||
if (
|
||||
burst &&
|
||||
s.kind === 'idle' &&
|
||||
s.isAgent &&
|
||||
(s.aiChatId === burst.chatId || s.aiChatId == null)
|
||||
) {
|
||||
burst.tEnd = s.t;
|
||||
continue;
|
||||
}
|
||||
if (s.isAgent && s.kind !== 'boundary') {
|
||||
if (burst && burst.chatId === s.aiChatId) {
|
||||
burst.tEnd = s.t;
|
||||
} else {
|
||||
flush();
|
||||
burst = { chatId: s.aiChatId, tStart: s.t, tEnd: s.t };
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// A human sample, a boundary, or an agent-boundary: breaks the burst and is
|
||||
// itself a zero-width segment (its class follows its own source).
|
||||
flush();
|
||||
segments.push({ tStart: s.t, tEnd: s.t, isAgent: s.isAgent });
|
||||
}
|
||||
flush();
|
||||
return segments;
|
||||
}
|
||||
|
||||
function gapThreshold(
|
||||
a: Segment,
|
||||
b: Segment,
|
||||
config: WorkTimeConfig,
|
||||
): number {
|
||||
return a.isAgent && b.isAgent ? config.agentTGap : config.tGap;
|
||||
}
|
||||
|
||||
/** Merge intervals; overlapping OR touching intervals are unioned. */
|
||||
function unionDuration(intervals: Array<[number, number]>): number {
|
||||
if (intervals.length === 0) return 0;
|
||||
const sorted = [...intervals].sort((a, b) => a[0] - b[0]);
|
||||
let total = 0;
|
||||
let [curStart, curEnd] = sorted[0];
|
||||
for (let i = 1; i < sorted.length; i++) {
|
||||
const [s, e] = sorted[i];
|
||||
if (s <= curEnd) {
|
||||
if (e > curEnd) curEnd = e;
|
||||
} else {
|
||||
total += curEnd - curStart;
|
||||
curStart = s;
|
||||
curEnd = e;
|
||||
}
|
||||
}
|
||||
total += curEnd - curStart;
|
||||
return total;
|
||||
}
|
||||
|
||||
/**
|
||||
* #395 core — estimate time worked on a page from its history timeline (§5).
|
||||
* Pure and deterministic: no DB, no clock, no I/O.
|
||||
*
|
||||
* Pipeline: normalize+dedup → collapse agent bursts → ONE sessionization pass
|
||||
* over all segments (threshold depends on the pair: both-agent → agentTGap, else
|
||||
* tGap; the last session is ALWAYS closed after the loop) → class per finished
|
||||
* session (all-agent → agent_only, else work) → pad each session (multi-sample
|
||||
* → [first−P_in, last+P_out]; lone scalar → [t−P_single, t]) → clip padding of
|
||||
* adjacent DIFFERENT-class sessions at the raw-gap midpoint (so work/agent_only
|
||||
* never overlap) → metrics are the union wall-clock within each class (union, not
|
||||
* Σ, so overlaps never double, and cross-class-disjoint by the clip above).
|
||||
*/
|
||||
export function computeWorkTime(
|
||||
rows: TimelineSample[],
|
||||
config?: Partial<WorkTimeConfig>,
|
||||
): WorkTimeResult {
|
||||
const cfg = resolveWorkTimeConfig(config);
|
||||
const samples = normalize(rows, cfg);
|
||||
const segments = collapse(samples, cfg);
|
||||
|
||||
// Sessionize — one pass over ALL segments.
|
||||
const rawSessions: Segment[][] = [];
|
||||
let cur: Segment[] | null = null;
|
||||
for (const seg of segments) {
|
||||
if (cur == null) {
|
||||
cur = [seg];
|
||||
} else {
|
||||
const last = cur[cur.length - 1];
|
||||
if (seg.tStart - last.tEnd <= gapThreshold(last, seg, cfg)) {
|
||||
cur.push(seg);
|
||||
} else {
|
||||
rawSessions.push(cur);
|
||||
cur = [seg];
|
||||
}
|
||||
}
|
||||
}
|
||||
if (cur != null) rawSessions.push(cur); // MUST close the last session (§5, §9#1)
|
||||
|
||||
// A finished session with BOTH its raw (unpadded) span and its padded bounds.
|
||||
// `rawSessions` are already in ascending time order, so `built` is too.
|
||||
interface BuiltSession {
|
||||
rawStart: number;
|
||||
rawEnd: number;
|
||||
padStart: number;
|
||||
padEnd: number;
|
||||
cls: WorkSession['class'];
|
||||
}
|
||||
const built: BuiltSession[] = [];
|
||||
|
||||
for (const segs of rawSessions) {
|
||||
const first = segs[0];
|
||||
const last = segs[segs.length - 1];
|
||||
const cls = segs.every((s) => s.isAgent) ? 'agent_only' : 'work';
|
||||
|
||||
let padStart: number;
|
||||
let padEnd: number;
|
||||
if (segs.length === 1 && first.tStart === first.tEnd) {
|
||||
// Lone single-instant session (one scalar, or a one-snapshot agent run):
|
||||
// pre-roll only, no invented "future" work (§5).
|
||||
padStart = first.tStart - cfg.pSingle;
|
||||
padEnd = first.tStart;
|
||||
} else {
|
||||
padStart = first.tStart - cfg.pIn;
|
||||
padEnd = last.tEnd + cfg.pOut;
|
||||
}
|
||||
|
||||
built.push({
|
||||
rawStart: first.tStart,
|
||||
rawEnd: last.tEnd,
|
||||
padStart,
|
||||
padEnd,
|
||||
cls,
|
||||
});
|
||||
}
|
||||
|
||||
// Clip cross-class padding so a `work` and an `agent_only` session that abut
|
||||
// never claim the same wall-clock. For each ADJACENT pair of DIFFERENT classes,
|
||||
// cap the earlier session's trailing pad and the later session's leading pad at
|
||||
// the MIDPOINT of the raw (unpadded) inactivity gap between them: the earlier
|
||||
// padded interval then ends ≤ midpoint and the later one starts ≥ midpoint, so
|
||||
// the two are disjoint (they touch at most at the midpoint). This makes the
|
||||
// per-class unions (workMs / agentOnlyMs) cross-class-disjoint BY CONSTRUCTION
|
||||
// — closing the double-count where a work session ending in an agent segment
|
||||
// and a nearby agent_only session (gap in (agentTGap, pIn+pOut]) overlapped and
|
||||
// were counted into both metrics (§5, §9). Within-class adjacency is left
|
||||
// untouched: `unionDuration` already dedups it, and clipping there could perturb
|
||||
// the per-class metric value.
|
||||
for (let i = 1; i < built.length; i++) {
|
||||
const a = built[i - 1];
|
||||
const b = built[i];
|
||||
if (a.cls === b.cls) continue;
|
||||
const midpoint = (a.rawEnd + b.rawStart) / 2;
|
||||
if (a.padEnd > midpoint) a.padEnd = midpoint;
|
||||
if (b.padStart < midpoint) b.padStart = midpoint;
|
||||
}
|
||||
|
||||
const sessions: WorkSession[] = [];
|
||||
const workIvs: Array<[number, number]> = [];
|
||||
const agentIvs: Array<[number, number]> = [];
|
||||
for (const s of built) {
|
||||
sessions.push({ start: s.padStart, end: s.padEnd, class: s.cls });
|
||||
(s.cls === 'work' ? workIvs : agentIvs).push([s.padStart, s.padEnd]);
|
||||
}
|
||||
|
||||
sessions.sort((a, b) => a.start - b.start);
|
||||
|
||||
return {
|
||||
workMs: unionDuration(workIvs),
|
||||
agentOnlyMs: unionDuration(agentIvs),
|
||||
sessions,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
export { computeWorkTime } from './compute-work-time';
|
||||
export { bucketByDay, zonedDayStart } from './bucket-by-day';
|
||||
export {
|
||||
DEFAULT_WORK_TIME_CONFIG,
|
||||
resolveWorkTimeConfig,
|
||||
} from './work-time.config';
|
||||
export type { WorkTimeConfig } from './work-time.config';
|
||||
export type {
|
||||
TimelineSample,
|
||||
WorkSession,
|
||||
WorkTimeResult,
|
||||
SessionClass,
|
||||
DayWindow,
|
||||
PerDay,
|
||||
} from './work-time.types';
|
||||
@@ -0,0 +1,102 @@
|
||||
import {
|
||||
IDLE_MAX_WAIT_USER,
|
||||
IDLE_MAX_WAIT_AGENT,
|
||||
} from '../../../collaboration/constants';
|
||||
|
||||
/**
|
||||
* #395 — tunables for the work-time estimate (§10). Defaults are calibrated off
|
||||
* #374's idle-pulse ceilings: after #374 a continuous editing session leaves a
|
||||
* history row at least every ~IDLE_MAX_WAIT (10m user / 5m agent), so a gap
|
||||
* WIDER than that ceiling contains un-pulsed idle time = (partial) inactivity.
|
||||
* `tGap` therefore sits a little above the user ceiling, `agentTGap` a little
|
||||
* above the agent ceiling — a gap within the threshold is pulse-backed and
|
||||
* counts as work.
|
||||
*/
|
||||
export interface WorkTimeConfig {
|
||||
/** user inactivity timeout: gap ≤ tGap between samples = continuous work. */
|
||||
tGap: number;
|
||||
/** timeout for a pair of consecutive agent samples (tighter than tGap). */
|
||||
agentTGap: number;
|
||||
/** pre-roll padding for a multi-sample session (work began before sample 1). */
|
||||
pIn: number;
|
||||
/** post-roll padding for a multi-sample session (work continued after last). */
|
||||
pOut: number;
|
||||
/** block for a lone single-sample session (pre-roll only, no invented future). */
|
||||
pSingle: number;
|
||||
/** drop `git`-source samples (they are not human/agent article work). */
|
||||
excludeGit: boolean;
|
||||
/** optional cap on one collapsed agent-burst segment's wall-clock (§9#3). */
|
||||
burstCapMs?: number;
|
||||
/** samples whose createdAt round to the same bucket dedup to one (§9#7). */
|
||||
dedupRoundMs: number;
|
||||
}
|
||||
|
||||
export const DEFAULT_WORK_TIME_CONFIG: WorkTimeConfig = {
|
||||
// ~15m: IDLE_MAX_WAIT_USER (10m) + headroom. Empirically backcast on a real
|
||||
// 307-snapshot article (≈24h at 15m matched the owner's estimate; 30/45m
|
||||
// over-counted). See #395 §10.
|
||||
tGap: 15 * 60 * 1000,
|
||||
// ~7m: IDLE_MAX_WAIT_AGENT (5m) + headroom.
|
||||
agentTGap: 7 * 60 * 1000,
|
||||
pIn: 5 * 60 * 1000,
|
||||
pOut: 5 * 60 * 1000,
|
||||
pSingle: 2 * 60 * 1000,
|
||||
excludeGit: true,
|
||||
burstCapMs: undefined,
|
||||
dedupRoundMs: 1000,
|
||||
};
|
||||
|
||||
// Compile-time cross-check that the defaults really are pulse-anchored — if a
|
||||
// future edit moves the #374 ceilings, this reminds us to re-calibrate.
|
||||
void IDLE_MAX_WAIT_USER;
|
||||
void IDLE_MAX_WAIT_AGENT;
|
||||
|
||||
/**
|
||||
* Fill a partial config with defaults and validate it. Cross-class metric
|
||||
* disjointness is guaranteed jointly by `computeWorkTime`'s adjacent-pair padding
|
||||
* clip (it caps the padding of adjacent DIFFERENT-class sessions at the raw-gap
|
||||
* midpoint) AND the two bounds enforced below (§5):
|
||||
* - `tGap ≥ pIn + pOut`: a session's own padding never exceeds its inactivity
|
||||
* window.
|
||||
* - `2·agentTGap ≥ pIn + pOut`: makes the adjacent-only clip provably COMPLETE.
|
||||
* A NON-adjacent (i, i+2) cross-class overlap could only arise from two
|
||||
* same-class sessions separated by a full intervening session of the other
|
||||
* class; that separation spans at least two inter-session gaps, each strictly
|
||||
* `> agentTGap`, so it is `> 2·agentTGap`. Requiring `2·agentTGap ≥ pIn + pOut`
|
||||
* means even the widest padded reach (pIn + pOut) cannot bridge it — so the
|
||||
* only cross-class overlaps possible are between ADJACENT sessions, which the
|
||||
* clip handles. `workMs`/`agentOnlyMs` are therefore disjoint by construction.
|
||||
*/
|
||||
export function resolveWorkTimeConfig(
|
||||
partial?: Partial<WorkTimeConfig>,
|
||||
): WorkTimeConfig {
|
||||
const config = { ...DEFAULT_WORK_TIME_CONFIG, ...(partial ?? {}) };
|
||||
|
||||
for (const key of [
|
||||
'tGap',
|
||||
'agentTGap',
|
||||
'pIn',
|
||||
'pOut',
|
||||
'pSingle',
|
||||
'dedupRoundMs',
|
||||
] as const) {
|
||||
const value = config[key];
|
||||
if (!Number.isFinite(value) || value < 0) {
|
||||
throw new Error(`work-time config: ${key} must be a non-negative number`);
|
||||
}
|
||||
}
|
||||
if (config.burstCapMs != null && config.burstCapMs <= 0) {
|
||||
throw new Error('work-time config: burstCapMs must be > 0 when set');
|
||||
}
|
||||
if (config.tGap < config.pIn + config.pOut) {
|
||||
throw new Error(
|
||||
"work-time config: tGap must be ≥ pIn + pOut (a session's padding may not exceed its inactivity window)",
|
||||
);
|
||||
}
|
||||
if (2 * config.agentTGap < config.pIn + config.pOut) {
|
||||
throw new Error(
|
||||
'work-time config: 2·agentTGap must be ≥ pIn + pOut (so non-adjacent cross-class padding cannot overlap)',
|
||||
);
|
||||
}
|
||||
return config;
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
/**
|
||||
* #395 — "time worked on an article" domain types.
|
||||
*
|
||||
* The estimate is built by sessionizing a page's `page_history` timeline on
|
||||
* inactivity gaps (WakaTime-style), NOT by taking the span between the first and
|
||||
* last edit (which over-counts sleep / lunch / idle days). See the design doc in
|
||||
* issue #395 §5–§6.3 for the normative algorithm.
|
||||
*/
|
||||
|
||||
/**
|
||||
* A single `page_history` row projected for the work-time computation — the
|
||||
* cheap columns only (no `content`). Produced by
|
||||
* `PageHistoryRepo.findTimelineByPageId`. `createdAt` is whatever the DB driver
|
||||
* hands back (Date); the pure core normalizes it to epoch-ms itself so it stays
|
||||
* deterministic and DB-free.
|
||||
*/
|
||||
export interface TimelineSample {
|
||||
createdAt: Date | string | number;
|
||||
lastUpdatedById: string | null;
|
||||
/** 'user' | 'agent' | 'git' | null (legacy autosave = human). */
|
||||
lastUpdatedSource: string | null;
|
||||
lastUpdatedAiChatId: string | null;
|
||||
/** #370 tier: 'manual' | 'agent' | 'idle' | 'boundary' | null (legacy). */
|
||||
kind: string | null;
|
||||
}
|
||||
|
||||
/** A finished session's class (§5.1). */
|
||||
export type SessionClass = 'work' | 'agent_only';
|
||||
|
||||
/**
|
||||
* A finished session: absolute wall-clock bounds already padded with P_in/P_out
|
||||
* (multi-sample) or P_single (single scalar), plus its class. This is enough for
|
||||
* both the metrics and the per-day punch-card colouring.
|
||||
*/
|
||||
export interface WorkSession {
|
||||
/** epoch-ms, inclusive lower bound (already P-padded). */
|
||||
start: number;
|
||||
/** epoch-ms, exclusive upper bound (already P-padded). */
|
||||
end: number;
|
||||
class: SessionClass;
|
||||
}
|
||||
|
||||
/** Output of {@link computeWorkTime}. */
|
||||
export interface WorkTimeResult {
|
||||
/** union wall-clock of `work` sessions, ms (the headline metric). */
|
||||
workMs: number;
|
||||
/** union wall-clock of `agent_only` sessions, ms (secondary). */
|
||||
agentOnlyMs: number;
|
||||
sessions: WorkSession[];
|
||||
}
|
||||
|
||||
/** One activity window inside a calendar day (already clipped to the day). */
|
||||
export interface DayWindow {
|
||||
/** epoch-ms. */
|
||||
start: number;
|
||||
/** epoch-ms. */
|
||||
end: number;
|
||||
class: SessionClass;
|
||||
}
|
||||
|
||||
/** One calendar day of the punch-card (§6.3). */
|
||||
export interface PerDay {
|
||||
/** epoch-ms of the local-midnight day start in the requested tz. */
|
||||
day: number;
|
||||
/** 'YYYY-MM-DD' in the requested tz — stable, tz-independent label. */
|
||||
dayISO: string;
|
||||
/** Σ of `work` windows this day, ms. Σ over days == workMs (invariant §6.3). */
|
||||
activeMs: number;
|
||||
/** Σ of `agent_only` windows this day, ms (drawn, NOT in activeMs). */
|
||||
agentMs: number;
|
||||
/** both classes, clipped to the day, sorted by start (for drawing). */
|
||||
windows: DayWindow[];
|
||||
}
|
||||
@@ -1,33 +1,56 @@
|
||||
import { Space } from '@docmost/db/types/entity.types';
|
||||
|
||||
export class SearchResponseDto {
|
||||
// #529 A7 — the single per-hit SUPERSET returned by the unified search engine.
|
||||
// The web-UI reads id/highlight/icon/space/title/…; the MCP agent maps id→pageId
|
||||
// and reads snippet/score/path. `rank`/`highlight` are null for substring-only
|
||||
// hits (the web already falls back). Nothing the legacy web response carried is
|
||||
// dropped.
|
||||
export class SearchResultDto {
|
||||
id: string;
|
||||
title: string;
|
||||
// Alias of `id` for the MCP layer (it addresses pages by pageId).
|
||||
pageId: string;
|
||||
slugId: string;
|
||||
icon: string;
|
||||
parentPageId: string;
|
||||
title: string;
|
||||
space?: Partial<Space>;
|
||||
creatorId: string;
|
||||
rank: number;
|
||||
highlight: string;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
space: Partial<Space>;
|
||||
// ts_rank_cd of the FTS branch; null for substring-only hits.
|
||||
rank: number | null;
|
||||
// ts_headline marked HTML; null for substring-only hits.
|
||||
highlight: string | null;
|
||||
// Plain windowed snippet around the match (empty for titleOnly).
|
||||
snippet: string;
|
||||
// Ancestor titles root → direct parent ([] for a root page).
|
||||
path: string[];
|
||||
// Per-response ordering proxy (falls back to rank).
|
||||
score: number;
|
||||
// Which fields matched: 'title' and/or 'text'.
|
||||
matchedFields: string[];
|
||||
// Which parsed positive/required terms this hit matched.
|
||||
matchedTerms: string[];
|
||||
}
|
||||
|
||||
// Response shape for the opt-in agent-lookup mode (#443, `substring: true`).
|
||||
// Additive to the FTS response: carries the location (`path`), a windowed
|
||||
// `snippet` around the first match and a per-response sort `score`. The MCP
|
||||
// layer maps `id → pageId`; `slugId` is never exposed.
|
||||
export class SearchLookupResponseDto {
|
||||
id: string;
|
||||
slugId: string;
|
||||
title: string;
|
||||
parentPageId: string | null;
|
||||
// Ancestor titles from the space root down to the direct parent; [] for a
|
||||
// root page.
|
||||
path: string[];
|
||||
// ~300–500 chars around the first match (or a leading text window / extended
|
||||
// ts_headline fallback).
|
||||
snippet: string;
|
||||
// 0..1 float, meaningful ONLY for sorting within one response.
|
||||
score: number;
|
||||
// The paginated envelope (A5). `total` is the EXACT permission-filtered count of
|
||||
// pages matching the positive lexical query (fail-closed). `hasMore` is true when
|
||||
// more results exist WITHIN the fusion window; `truncatedAtCap` signals the match
|
||||
// set exceeded CANDIDATE_CAP and the tail is unreachable by pagination.
|
||||
export class SearchResponseDto {
|
||||
items: SearchResultDto[];
|
||||
total: number;
|
||||
hasMore: boolean;
|
||||
truncatedAtCap: boolean;
|
||||
offset: number;
|
||||
query: {
|
||||
raw: string;
|
||||
parsed: {
|
||||
positive: string[];
|
||||
required: string[];
|
||||
excluded: string[];
|
||||
reason?: string;
|
||||
};
|
||||
mode: 'or' | 'and';
|
||||
match: string;
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,16 +1,30 @@
|
||||
import {
|
||||
IsBoolean,
|
||||
IsIn,
|
||||
IsNotEmpty,
|
||||
IsNumber,
|
||||
IsOptional,
|
||||
IsString,
|
||||
MaxLength,
|
||||
} from 'class-validator';
|
||||
|
||||
export class SearchDTO {
|
||||
// Defense-in-depth cap on the raw query length. The real stack-depth bound is
|
||||
// the parser's MAX_PARSED_TERMS term cap (see search-query-parser.ts); this
|
||||
// just rejects absurd payloads early. 10k chars still comfortably holds any
|
||||
// legitimate query.
|
||||
@IsNotEmpty()
|
||||
@IsString()
|
||||
@MaxLength(10000)
|
||||
query: string;
|
||||
|
||||
// #529 A3 — match mode. `auto` (default) routes identifier-like terms
|
||||
// (10.31.41, esp32, WB-MGE-30D86B) to the substring/trigram branch and words
|
||||
// to full-text; `word`/`prefix`/`substring` are explicit overrides.
|
||||
@IsOptional()
|
||||
@IsIn(['auto', 'word', 'prefix', 'substring'])
|
||||
match?: 'auto' | 'word' | 'prefix' | 'substring';
|
||||
|
||||
@IsOptional()
|
||||
@IsString()
|
||||
spaceId: string;
|
||||
@@ -33,15 +47,19 @@ export class SearchDTO {
|
||||
|
||||
// --- Opt-in agent-lookup mode (#443). ------------------------------------
|
||||
// These fields are ADDITIVE and default-off: a web client that sends none of
|
||||
// them gets byte-identical FTS behaviour and result shape. They are only read
|
||||
// by the substring/path/snippet code path in SearchService.searchPage.
|
||||
// them gets byte-identical FTS behaviour and result shape. In the unified #529
|
||||
// engine, `parentPageId` and `titleOnly` are read by SearchService.searchPage
|
||||
// (subtree scoping and title-only matching, respectively). `substring` is NOT
|
||||
// read by the native driver — it is accepted-but-ignored, kept only for
|
||||
// back-compat with the upstream lookup request shape.
|
||||
//
|
||||
// NOTE (standalone stdio vs stock upstream): stock upstream validates this DTO
|
||||
// with `whitelist: true`, so an older server silently strips these unknown
|
||||
// fields and the request degrades gracefully to the plain FTS behaviour.
|
||||
|
||||
// Enables the hybrid substring branch (title + text_content LIKE) merged with
|
||||
// the existing FTS branch, plus tiered ranking, path and windowed snippet.
|
||||
// Accepted-but-ignored by the #529 native driver (kept for upstream lookup
|
||||
// back-compat). The unified engine ALWAYS runs the hybrid FTS + substring/
|
||||
// trigram branches with tiered ranking, so this flag no longer toggles anything.
|
||||
@IsOptional()
|
||||
@IsBoolean()
|
||||
substring?: boolean;
|
||||
|
||||
@@ -0,0 +1,168 @@
|
||||
import {
|
||||
parseSearchQuery,
|
||||
hasPositiveRecall,
|
||||
MAX_PARSED_TERMS,
|
||||
} from './search-query-parser';
|
||||
|
||||
describe('parseSearchQuery — tokenization & operators (A2)', () => {
|
||||
it('splits on whitespace into positive terms (OR recall)', () => {
|
||||
const p = parseSearchQuery('стамбул роснефть');
|
||||
expect(p.positive.map((t) => t.text)).toEqual(['стамбул', 'роснефть']);
|
||||
expect(p.required).toEqual([]);
|
||||
expect(p.excluded).toEqual([]);
|
||||
expect(p.mode).toBe('or');
|
||||
});
|
||||
|
||||
it('treats +term as required and -term as excluded', () => {
|
||||
const p = parseSearchQuery('+кофейня -архив');
|
||||
expect(p.required.map((t) => t.text)).toEqual(['кофейня']);
|
||||
expect(p.excluded.map((t) => t.text)).toEqual(['архив']);
|
||||
expect(p.positive).toEqual([]);
|
||||
});
|
||||
|
||||
it('keeps a leading-operator-free hyphen/dot/colon token as ONE literal term', () => {
|
||||
// WB-MGE-30D86B, 10.0.12.5, a:b — internal -,.,: are literal, one term each.
|
||||
expect(parseSearchQuery('WB-MGE-30D86B').positive[0].text).toBe(
|
||||
'WB-MGE-30D86B',
|
||||
);
|
||||
expect(parseSearchQuery('10.0.12.5').positive[0].text).toBe('10.0.12.5');
|
||||
expect(parseSearchQuery('host:8080').positive[0].text).toBe('host:8080');
|
||||
});
|
||||
|
||||
it('only a LEADING +/- is an operator; -архив excludes архив', () => {
|
||||
const p = parseSearchQuery('-архив');
|
||||
expect(p.excluded.map((t) => t.text)).toEqual(['архив']);
|
||||
expect(p.positive).toEqual([]);
|
||||
});
|
||||
|
||||
it('drops a bare "-" / "+" and an all-operator remainder', () => {
|
||||
const p = parseSearchQuery('- + foo -- ++');
|
||||
expect(p.positive.map((t) => t.text)).toEqual(['foo']);
|
||||
expect(p.required).toEqual([]);
|
||||
expect(p.excluded).toEqual([]);
|
||||
});
|
||||
|
||||
it('parses a quoted phrase as one adjacency term', () => {
|
||||
const p = parseSearchQuery('"воздушный шар" кофе');
|
||||
expect(p.positive[0]).toMatchObject({ text: 'воздушный шар', branch: 'phrase' });
|
||||
expect(p.positive[1].text).toBe('кофе');
|
||||
});
|
||||
|
||||
it('applies +/- to a phrase', () => {
|
||||
const req = parseSearchQuery('+"воздушный шар"');
|
||||
expect(req.required[0]).toMatchObject({ text: 'воздушный шар', branch: 'phrase' });
|
||||
const exc = parseSearchQuery('-"воздушный шар"');
|
||||
expect(exc.excluded[0]).toMatchObject({ text: 'воздушный шар', branch: 'phrase' });
|
||||
});
|
||||
|
||||
it('drops an unbalanced quote token', () => {
|
||||
const p = parseSearchQuery('kafka "unclosed here');
|
||||
expect(p.positive.map((t) => t.text)).toEqual(['kafka']);
|
||||
});
|
||||
|
||||
it('strips tsquery metacharacters from a bare FTS term (no 500)', () => {
|
||||
const p = parseSearchQuery('foo|bar');
|
||||
// `|` is not an identifier signal → FTS branch; the metachar is stripped so
|
||||
// the term becomes the two words that survive.
|
||||
expect(p.positive[0].branch === 'fts' || p.positive[0].branch === 'ftsPrefix').toBe(true);
|
||||
expect(p.positive[0].text).toBe('foo bar');
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseSearchQuery — match=auto classification (A3)', () => {
|
||||
it('routes identifier-like terms to the substring branch', () => {
|
||||
expect(parseSearchQuery('10.31.41').positive[0].branch).toBe('substring');
|
||||
expect(parseSearchQuery('esp32').positive[0].branch).toBe('substring');
|
||||
expect(parseSearchQuery('WB-MGE-30D86B').positive[0].branch).toBe('substring');
|
||||
});
|
||||
|
||||
it('routes purely-alphabetic words to the FTS (prefix) branch', () => {
|
||||
expect(parseSearchQuery('печат').positive[0].branch).toBe('ftsPrefix');
|
||||
expect(parseSearchQuery('ресторан').positive[0].branch).toBe('ftsPrefix');
|
||||
});
|
||||
|
||||
it('explicit match overrides: word / prefix / substring', () => {
|
||||
expect(parseSearchQuery('печат', { match: 'word' }).positive[0].branch).toBe('fts');
|
||||
expect(parseSearchQuery('печат', { match: 'prefix' }).positive[0].branch).toBe(
|
||||
'ftsPrefix',
|
||||
);
|
||||
expect(parseSearchQuery('печат', { match: 'substring' }).positive[0].branch).toBe(
|
||||
'substring',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseSearchQuery — reasons & recall', () => {
|
||||
it('only-negation yields reason only-negation and no positive recall', () => {
|
||||
const p = parseSearchQuery('-архив');
|
||||
expect(p.reason).toBe('only-negation');
|
||||
expect(hasPositiveRecall(p)).toBe(false);
|
||||
});
|
||||
|
||||
it('empty / whitespace / garbage yields reason empty', () => {
|
||||
expect(parseSearchQuery('').reason).toBe('empty');
|
||||
expect(parseSearchQuery(' ').reason).toBe('empty');
|
||||
// A bare operator drops to nothing → empty (no exclusion survived).
|
||||
expect(parseSearchQuery('+ -').reason).toBe('empty');
|
||||
});
|
||||
|
||||
it('a required term alone IS positive recall (no reason)', () => {
|
||||
const p = parseSearchQuery('+кофейня');
|
||||
expect(p.reason).toBeUndefined();
|
||||
expect(hasPositiveRecall(p)).toBe(true);
|
||||
});
|
||||
|
||||
it('mode flag flows through', () => {
|
||||
expect(parseSearchQuery('a b', { mode: 'and' }).mode).toBe('and');
|
||||
expect(parseSearchQuery('a b').mode).toBe('or');
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseSearchQuery — term cap (stack-depth guard)', () => {
|
||||
it('caps the total parsed terms at MAX_PARSED_TERMS without throwing', () => {
|
||||
// A pasted text block: far more words than the cap. The parser must bound the
|
||||
// SQL tsquery nesting depth (else Postgres blows its stack → HTTP 500).
|
||||
const words = Array.from({ length: 5000 }, (_, i) => `w${i}`);
|
||||
let p!: ReturnType<typeof parseSearchQuery>;
|
||||
expect(() => {
|
||||
p = parseSearchQuery(words.join(' '));
|
||||
}).not.toThrow();
|
||||
const total = p.positive.length + p.required.length + p.excluded.length;
|
||||
expect(total).toBe(MAX_PARSED_TERMS);
|
||||
// Stable order: the FIRST cap terms are kept.
|
||||
expect(p.positive.slice(0, 3).map((t) => t.text)).toEqual(['w0', 'w1', 'w2']);
|
||||
expect(p.positive[MAX_PARSED_TERMS - 1].text).toBe(`w${MAX_PARSED_TERMS - 1}`);
|
||||
});
|
||||
|
||||
it('counts positive + required + excluded together toward the cap', () => {
|
||||
// Interleave operators so overflow can fall on any bucket. Total must still
|
||||
// never exceed the cap, and the first cap terms (in stable order) win.
|
||||
const tokens: string[] = [];
|
||||
for (let i = 0; i < 200; i++) {
|
||||
const op = i % 3 === 0 ? '+' : i % 3 === 1 ? '-' : '';
|
||||
tokens.push(`${op}t${i}`);
|
||||
}
|
||||
const p = parseSearchQuery(tokens.join(' '));
|
||||
const total = p.positive.length + p.required.length + p.excluded.length;
|
||||
expect(total).toBe(MAX_PARSED_TERMS);
|
||||
// t0 (+, required) and t1 (-, excluded) and t2 (bare, positive) are all within
|
||||
// the first cap tokens → each bucket got its leading terms.
|
||||
expect(p.required[0].text).toBe('t0');
|
||||
expect(p.excluded[0].text).toBe('t1');
|
||||
expect(p.positive[0].text).toBe('t2');
|
||||
});
|
||||
|
||||
it('leaves a normal (<= cap) query completely unchanged', () => {
|
||||
const p = parseSearchQuery('+кофейня -архив "воздушный шар" ресторан 10.31.41');
|
||||
expect(p.required.map((t) => t.text)).toEqual(['кофейня']);
|
||||
expect(p.excluded.map((t) => t.text)).toEqual(['архив']);
|
||||
expect(p.positive.map((t) => t.text)).toEqual([
|
||||
'воздушный шар',
|
||||
'ресторан',
|
||||
'10.31.41',
|
||||
]);
|
||||
// Phrase / substring branch handling survives under the cap.
|
||||
expect(p.positive[0].branch).toBe('phrase');
|
||||
expect(p.positive[2].branch).toBe('substring');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,242 @@
|
||||
// #529 Phase A — server-side query parser (the SINGLE source of query semantics).
|
||||
//
|
||||
// Clients (web-UI, MCP agent, public share) send a RAW query string plus flags
|
||||
// (`match`, `mode`); ALL query interpretation happens HERE, so every consumer
|
||||
// gets identical operator/phrase/morphology behaviour. The parser is PURE (no
|
||||
// SQL, no DB) so it is exhaustively unit-testable; SearchService turns the parsed
|
||||
// AST into a parameterized tsquery/predicate tree (never string-concatenated SQL).
|
||||
//
|
||||
// Grammar (A2):
|
||||
// - Whitespace splits tokens, but a double-quoted run is ONE token ("a b" is a
|
||||
// phrase). An unbalanced quote is dropped.
|
||||
// - A token is an OPERATOR token only when it STARTS with `+` or `-`, the
|
||||
// remainder is non-empty, and the remainder is not itself only operators.
|
||||
// `+`/`-` inside a token (`WB-MGE-30D86B`, `10.0.12.5`, `a:b`) is a LITERAL —
|
||||
// the token is a single term. A bare `-`/`+` is dropped.
|
||||
// - `"phrase"` → phrase term; `+"phrase"`/`-"phrase"` apply the operator to it.
|
||||
// - Positive terms (bare + phrase, no operator) form the OR recall set.
|
||||
// `+term` is a REQUIRED predicate, `-term` an EXCLUDED predicate (A2): both
|
||||
// are applied in SQL WHERE against the whole candidate set, not folded into
|
||||
// the positive tsquery.
|
||||
// - Only-negation (no positive term) short-circuits to an empty result with
|
||||
// reason `only-negation` (never runs a costly NOT-scan).
|
||||
|
||||
export type SearchMatchMode = 'auto' | 'word' | 'prefix' | 'substring';
|
||||
export type SearchBooleanMode = 'or' | 'and';
|
||||
|
||||
// Hard cap on the total number of parsed terms (positive + required + excluded).
|
||||
// SearchService folds each FTS term into a LEFT-NESTED SQL tsquery expression
|
||||
// `(((t1)||(t2))||(t3))…`, embedded several times per query — so nesting depth
|
||||
// grows with the term count. A pasted text block (thousands of words) would nest
|
||||
// deep enough to blow Postgres' `stack depth limit` → an ERROR → HTTP 500 for the
|
||||
// caller. Capping in the parser bounds that depth for EVERY consumer (web / MCP /
|
||||
// share), since they all route through here. 64 comfortably covers any real query
|
||||
// while keeping the SQL nesting shallow. Overflow terms (beyond the first 64, in
|
||||
// stable order) are dropped rather than throwing.
|
||||
export const MAX_PARSED_TERMS = 64;
|
||||
|
||||
// How a single term is matched against the index.
|
||||
// - 'fts' : full-text lexeme, exact (no trailing prefix).
|
||||
// - 'ftsPrefix' : full-text lexeme with a `:*` prefix match.
|
||||
// - 'phrase' : an adjacency phrase (phraseto_tsquery).
|
||||
// - 'substring' : a literal LOWER(f_unaccent(col)) LIKE '%needle%' branch
|
||||
// (identifiers the tokenizer mangles: IPs, hostnames, IDs).
|
||||
export type SearchTermBranch = 'fts' | 'ftsPrefix' | 'phrase' | 'substring';
|
||||
|
||||
export interface ParsedTerm {
|
||||
// The user-visible term text, operator stripped, quotes removed. This is what
|
||||
// `matchedTerms` echoes back per hit.
|
||||
text: string;
|
||||
branch: SearchTermBranch;
|
||||
}
|
||||
|
||||
export interface ParsedQuery {
|
||||
raw: string;
|
||||
// OR-recall set (bare + phrase terms with no operator).
|
||||
positive: ParsedTerm[];
|
||||
// AND predicates (`+term`) — the candidate MUST match each of these.
|
||||
required: ParsedTerm[];
|
||||
// NOT predicates (`-term`) — the candidate must match NONE of these.
|
||||
excluded: ParsedTerm[];
|
||||
mode: SearchBooleanMode;
|
||||
// Set only when the query yields no positive recall: 'empty' (nothing usable)
|
||||
// or 'only-negation' (there were exclusions but no positive term).
|
||||
reason?: 'empty' | 'only-negation';
|
||||
}
|
||||
|
||||
interface RawToken {
|
||||
op: '' | '+' | '-';
|
||||
kind: 'word' | 'phrase';
|
||||
text: string;
|
||||
}
|
||||
|
||||
// tsquery metacharacters that must never reach to_tsquery from a bare term — they
|
||||
// are what turned adversarial input into a 500 before (#139). Stripped for the FTS
|
||||
// branch; the substring branch keeps them (they are literal there).
|
||||
const TSQUERY_META = /[:&|!()*<>\\]+/g;
|
||||
|
||||
// A term is "identifier-like" when it carries a digit or one of . _ : / - AND is
|
||||
// not purely alphabetic (letters only). Such tokens (10.31.41, esp32,
|
||||
// WB-MGE-30D86B) are mangled by the FTS tokenizer, so `match: auto` routes them
|
||||
// to the substring branch. A purely-alphabetic word (печат, ресторан) stays FTS.
|
||||
const IDENTIFIER_SIGNAL = /[0-9._:/\\-]/;
|
||||
const PURELY_ALPHA = /^\p{L}+$/u;
|
||||
|
||||
function isIdentifierLike(text: string): boolean {
|
||||
return IDENTIFIER_SIGNAL.test(text) && !PURELY_ALPHA.test(text);
|
||||
}
|
||||
|
||||
// Clean a bare term for the FTS branch: NFC-normalize, drop tsquery metacharacters
|
||||
// and collapse whitespace. Returns '' when nothing usable remains.
|
||||
export function cleanFtsLexeme(raw: string): string {
|
||||
return (raw ?? '')
|
||||
.normalize('NFC')
|
||||
.replace(TSQUERY_META, ' ')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim();
|
||||
}
|
||||
|
||||
// Split a raw query into tokens, honouring double quotes and a single leading
|
||||
// +/- operator. Unbalanced quotes and bare operators are dropped.
|
||||
function tokenize(raw: string): RawToken[] {
|
||||
const tokens: RawToken[] = [];
|
||||
const s = raw ?? '';
|
||||
let i = 0;
|
||||
const n = s.length;
|
||||
|
||||
const isSpace = (c: string) => /\s/.test(c);
|
||||
|
||||
while (i < n) {
|
||||
// Skip leading whitespace.
|
||||
while (i < n && isSpace(s[i])) i++;
|
||||
if (i >= n) break;
|
||||
|
||||
let op: '' | '+' | '-' = '';
|
||||
// A single leading +/- is a tentative operator. Only ONE leading operator is
|
||||
// consumed; a second (`--x`) leaves `-x` as the remainder (a literal dash).
|
||||
if (s[i] === '+' || s[i] === '-') {
|
||||
op = s[i] as '+' | '-';
|
||||
i++;
|
||||
}
|
||||
|
||||
if (i < n && s[i] === '"') {
|
||||
// Quoted phrase: read until the closing quote.
|
||||
const close = s.indexOf('"', i + 1);
|
||||
if (close === -1) {
|
||||
// Unbalanced quote → drop this token and everything the open quote would
|
||||
// have consumed (the rest of the string).
|
||||
break;
|
||||
}
|
||||
const phrase = s.slice(i + 1, close);
|
||||
i = close + 1;
|
||||
if (phrase.trim().length > 0) {
|
||||
tokens.push({ op, kind: 'phrase', text: phrase.trim() });
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
// Bare word: read until the next whitespace.
|
||||
let j = i;
|
||||
while (j < n && !isSpace(s[j])) j++;
|
||||
const word = s.slice(i, j);
|
||||
i = j;
|
||||
|
||||
// A bare operator (`-`/`+` with no remainder) or an all-operator remainder is
|
||||
// dropped.
|
||||
if (word.length === 0) continue;
|
||||
if (op && /^[+-]+$/.test(word)) continue;
|
||||
|
||||
tokens.push({ op, kind: 'word', text: word });
|
||||
}
|
||||
|
||||
return tokens;
|
||||
}
|
||||
|
||||
// Resolve the match branch for a single term given the global match mode.
|
||||
function branchForTerm(
|
||||
text: string,
|
||||
kind: 'word' | 'phrase',
|
||||
mode: SearchMatchMode,
|
||||
): SearchTermBranch {
|
||||
if (kind === 'phrase') return 'phrase';
|
||||
switch (mode) {
|
||||
case 'word':
|
||||
return 'fts';
|
||||
case 'prefix':
|
||||
return 'ftsPrefix';
|
||||
case 'substring':
|
||||
return 'substring';
|
||||
case 'auto':
|
||||
default:
|
||||
// Identifiers the tokenizer mangles go to substring; words get a prefix
|
||||
// FTS match (so `печат` still finds `печатать`, but `печат` no longer drags
|
||||
// in `впечатления` because the russian stemmer anchors the stem).
|
||||
return isIdentifierLike(text) ? 'substring' : 'ftsPrefix';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a raw user query + flags into a structured, SQL-agnostic AST.
|
||||
* Pure and total: never throws, always returns a ParsedQuery.
|
||||
*/
|
||||
export function parseSearchQuery(
|
||||
raw: string,
|
||||
opts: { match?: SearchMatchMode; mode?: SearchBooleanMode } = {},
|
||||
): ParsedQuery {
|
||||
const match: SearchMatchMode = opts.match ?? 'auto';
|
||||
const mode: SearchBooleanMode = opts.mode ?? 'or';
|
||||
|
||||
const positive: ParsedTerm[] = [];
|
||||
const required: ParsedTerm[] = [];
|
||||
const excluded: ParsedTerm[] = [];
|
||||
|
||||
for (const tok of tokenize(raw)) {
|
||||
// For an FTS branch, the token must survive metacharacter cleaning; for the
|
||||
// substring/phrase branch the literal text is used. A term that cleans to
|
||||
// nothing AND is not usable as a substring is dropped.
|
||||
const branch = branchForTerm(tok.text, tok.kind, match);
|
||||
|
||||
let usableText: string;
|
||||
if (branch === 'fts' || branch === 'ftsPrefix') {
|
||||
usableText = cleanFtsLexeme(tok.text);
|
||||
} else {
|
||||
// phrase / substring keep the literal (trimmed) text.
|
||||
usableText = tok.text.trim();
|
||||
}
|
||||
if (!usableText) continue;
|
||||
|
||||
// Stack-depth guard: stop after MAX_PARSED_TERMS surviving terms (positive +
|
||||
// required + excluded, combined) so the SQL tsquery nesting stays shallow.
|
||||
// The first 64 terms are kept in stable order; the rest are dropped.
|
||||
if (positive.length + required.length + excluded.length >= MAX_PARSED_TERMS) {
|
||||
break;
|
||||
}
|
||||
|
||||
const term: ParsedTerm = { text: usableText, branch };
|
||||
|
||||
if (tok.op === '+') required.push(term);
|
||||
else if (tok.op === '-') excluded.push(term);
|
||||
else positive.push(term);
|
||||
}
|
||||
|
||||
const parsed: ParsedQuery = { raw: raw ?? '', positive, required, excluded, mode };
|
||||
|
||||
if (positive.length === 0) {
|
||||
// Required terms with no positive recall still form a valid positive set (the
|
||||
// required predicates ARE the recall). Only when there is neither a positive
|
||||
// nor a required term is the query empty / only-negation.
|
||||
if (required.length === 0) {
|
||||
parsed.reason = excluded.length > 0 ? 'only-negation' : 'empty';
|
||||
}
|
||||
}
|
||||
|
||||
return parsed;
|
||||
}
|
||||
|
||||
/**
|
||||
* Does this parsed query have any positive recall to run? False means we must
|
||||
* short-circuit to an empty result (with `reason`), never a costly NOT-only scan.
|
||||
*/
|
||||
export function hasPositiveRecall(parsed: ParsedQuery): boolean {
|
||||
return parsed.positive.length > 0 || parsed.required.length > 0;
|
||||
}
|
||||
@@ -1,19 +1,14 @@
|
||||
import {
|
||||
computeLookupScore,
|
||||
escapeLikePattern,
|
||||
SearchLookupTier,
|
||||
} from './search.service';
|
||||
import { escapeLikePattern } from './search.service';
|
||||
|
||||
/**
|
||||
* Pure-function coverage for the #443 agent-lookup helpers:
|
||||
* - escapeLikePattern: LIKE-metacharacter escaping so `%`/`_`/`\` are literals
|
||||
* (the acceptance-table requirement that a query of `%` or `_` does NOT match
|
||||
* everything);
|
||||
* - computeLookupScore: the tiered 0..1 ranking score, where a stronger tier
|
||||
* always outranks a weaker one regardless of the in-tier secondary signal.
|
||||
* Pure-function coverage for `escapeLikePattern` — LIKE-metacharacter escaping so
|
||||
* `%`/`_`/`\` are matched literally (the acceptance requirement that a query of
|
||||
* `%` or `_` does NOT match everything, #529 acceptance #10). The substring
|
||||
* branch's DB behaviour is covered by the integration spec.
|
||||
*
|
||||
* The DB-touching branch (substring UNION FTS, path CTE, snippet window) is
|
||||
* covered by the integration spec against the real schema.
|
||||
* NOTE (#529): the old tiered `computeLookupScore` was replaced by RRF rank
|
||||
* fusion in the unified engine, so its unit coverage moved to the integration
|
||||
* ordering tests; only the escaping helper remains a pure unit here.
|
||||
*/
|
||||
describe('escapeLikePattern', () => {
|
||||
it('escapes the LIKE metacharacters % _ and \\', () => {
|
||||
@@ -43,53 +38,3 @@ describe('escapeLikePattern', () => {
|
||||
expect(escapeLikePattern(null as any)).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('computeLookupScore', () => {
|
||||
it('keeps every score within (0, 1]', () => {
|
||||
for (const tier of [
|
||||
SearchLookupTier.TITLE_EXACT,
|
||||
SearchLookupTier.TITLE_SUBSTRING,
|
||||
SearchLookupTier.TEXT,
|
||||
]) {
|
||||
for (const secondary of [0, 0.001, 1, 100, 1e6]) {
|
||||
const s = computeLookupScore({ tier, secondary });
|
||||
expect(s).toBeGreaterThan(0);
|
||||
expect(s).toBeLessThanOrEqual(1);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
it('a stronger tier ALWAYS outranks a weaker tier, whatever the secondary', () => {
|
||||
// Weak tier with a huge secondary must still lose to a strong tier with a
|
||||
// tiny secondary — tiers dominate.
|
||||
const strongLowSecondary = computeLookupScore({
|
||||
tier: SearchLookupTier.TITLE_EXACT,
|
||||
secondary: 0,
|
||||
});
|
||||
const weakHighSecondary = computeLookupScore({
|
||||
tier: SearchLookupTier.TEXT,
|
||||
secondary: 1e9,
|
||||
});
|
||||
expect(strongLowSecondary).toBeGreaterThan(weakHighSecondary);
|
||||
});
|
||||
|
||||
it('within a tier a larger secondary sorts higher', () => {
|
||||
const lo = computeLookupScore({
|
||||
tier: SearchLookupTier.TEXT,
|
||||
secondary: 0.1,
|
||||
});
|
||||
const hi = computeLookupScore({
|
||||
tier: SearchLookupTier.TEXT,
|
||||
secondary: 5,
|
||||
});
|
||||
expect(hi).toBeGreaterThan(lo);
|
||||
});
|
||||
|
||||
it('treats a negative/absent secondary as 0', () => {
|
||||
const zero = computeLookupScore({ tier: SearchLookupTier.TEXT, secondary: 0 });
|
||||
expect(computeLookupScore({ tier: SearchLookupTier.TEXT })).toBe(zero);
|
||||
expect(
|
||||
computeLookupScore({ tier: SearchLookupTier.TEXT, secondary: -5 }),
|
||||
).toBe(zero);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,74 +1,45 @@
|
||||
import { SearchService } from './search.service';
|
||||
|
||||
/**
|
||||
* Coverage for SearchService.searchPage query-mode selection (search.service.ts
|
||||
* @25). searchPage chooses HOW the result set is scoped — by explicit space, by
|
||||
* the authenticated user's member spaces, or by a share — and must return an
|
||||
* empty set (without leaking data) for every disallowed combination.
|
||||
* Unit coverage for SearchService.searchPage SCOPE-SECURITY early returns — the
|
||||
* branches that must yield an empty result WITHOUT ever touching the DB, so they
|
||||
* can leak nothing. The happy-path scope SQL (explicit space / member spaces /
|
||||
* share id set) is covered against the real schema in the integration spec.
|
||||
*
|
||||
* The kysely query builder is mocked with the same chainable pattern as the
|
||||
* existing search.service.spec.ts: every builder method returns the same builder
|
||||
* and `.execute()` resolves the supplied rows. Each `.where(...)` call is
|
||||
* recorded so we can assert exactly which scope clause was applied — that is the
|
||||
* mutation-resistant signal that distinguishes one query mode from another.
|
||||
*
|
||||
* These specs catch cross-space / cross-workspace search leakage and
|
||||
* share-scope bypass (data exposure).
|
||||
* Every case here returns BEFORE the raw-SQL candidate query runs, so a bare `db`
|
||||
* stub (never called) is enough — a call to it would itself be a failure signal.
|
||||
*/
|
||||
describe('SearchService.searchPage — query-mode selection', () => {
|
||||
// Build a chainable selectFrom('pages') builder that records its calls. The
|
||||
// builder is returned from `db.selectFrom` and is the single object every
|
||||
// chained call mutates/returns, mirroring the existing spec's pattern.
|
||||
function makeBuilder(rows: Array<{ id: string; highlight?: string }>) {
|
||||
const builder: any = {};
|
||||
builder.select = jest.fn(() => builder);
|
||||
builder.where = jest.fn(() => builder);
|
||||
builder.$if = jest.fn(() => builder);
|
||||
builder.orderBy = jest.fn(() => builder);
|
||||
builder.limit = jest.fn(() => builder);
|
||||
builder.offset = jest.fn(() => builder);
|
||||
builder.execute = jest.fn(async () => rows);
|
||||
return builder;
|
||||
}
|
||||
|
||||
describe('SearchService.searchPage — scope-security early returns', () => {
|
||||
function makeService(opts?: {
|
||||
rows?: Array<{ id: string; highlight?: string }>;
|
||||
share?: any;
|
||||
isRestricted?: boolean;
|
||||
descendants?: Array<{ id: string }>;
|
||||
memberSpaceIds?: string[];
|
||||
}) {
|
||||
const builder = makeBuilder(opts?.rows ?? []);
|
||||
|
||||
const db: any = {
|
||||
selectFrom: jest.fn(() => builder),
|
||||
};
|
||||
|
||||
// `getUserSpaceIdsQuery` returns a sub-query object that searchPage passes
|
||||
// straight into `.where('spaceId', 'in', <subquery>)`. A sentinel is enough
|
||||
// to assert the user-scoped branch was taken.
|
||||
const userSpaceIdsQuery = { __userSpaceIdsQuery: true };
|
||||
// A db that THROWS if touched — these branches must not reach SQL.
|
||||
const db: any = new Proxy(
|
||||
{},
|
||||
{
|
||||
get() {
|
||||
throw new Error('db must not be touched on an empty-scope branch');
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
const pageRepo = {
|
||||
// `.select((eb) => this.pageRepo.withSpace(eb))` — value ignored by stub.
|
||||
withSpace: jest.fn(() => ({ __withSpace: true })),
|
||||
getPageAndDescendantsExcludingRestricted: jest
|
||||
.fn()
|
||||
.mockResolvedValue(opts?.descendants ?? []),
|
||||
getPageAndDescendantsExcludingRestricted: jest.fn(),
|
||||
getPageAndDescendants: jest.fn(),
|
||||
};
|
||||
const shareRepo = {
|
||||
findById: jest.fn().mockResolvedValue(opts?.share ?? null),
|
||||
};
|
||||
const spaceMemberRepo = {
|
||||
getUserSpaceIdsQuery: jest.fn(() => userSpaceIdsQuery),
|
||||
getUserSpaceIds: jest.fn().mockResolvedValue(opts?.memberSpaceIds ?? []),
|
||||
};
|
||||
const pagePermissionRepo = {
|
||||
hasRestrictedAncestor: jest
|
||||
.fn()
|
||||
.mockResolvedValue(opts?.isRestricted ?? false),
|
||||
// Let everything through page-level permission filtering by default.
|
||||
filterAccessiblePageIds: jest
|
||||
.fn()
|
||||
.mockImplementation(async ({ pageIds }: { pageIds: string[] }) => pageIds),
|
||||
filterAccessiblePageIds: jest.fn(),
|
||||
};
|
||||
|
||||
const service = new SearchService(
|
||||
@@ -78,145 +49,81 @@ describe('SearchService.searchPage — query-mode selection', () => {
|
||||
spaceMemberRepo as any,
|
||||
pagePermissionRepo as any,
|
||||
);
|
||||
|
||||
return {
|
||||
service,
|
||||
db,
|
||||
builder,
|
||||
pageRepo,
|
||||
shareRepo,
|
||||
spaceMemberRepo,
|
||||
pagePermissionRepo,
|
||||
userSpaceIdsQuery,
|
||||
};
|
||||
return { service, pageRepo, shareRepo, spaceMemberRepo, pagePermissionRepo };
|
||||
}
|
||||
|
||||
const whereCallFor = (builder: any, column: any) =>
|
||||
builder.where.mock.calls.find((c: any[]) => c[0] === column);
|
||||
|
||||
it('returns {items:[]} for a blank query WITHOUT touching the DB', async () => {
|
||||
const { service, db } = makeService();
|
||||
|
||||
it('returns total:0 for a blank query WITHOUT touching the DB or any repo', async () => {
|
||||
const { service, shareRepo, spaceMemberRepo } = makeService();
|
||||
const result = await service.searchPage(
|
||||
{ query: '' } as any,
|
||||
{ userId: 'user-1', workspaceId: 'ws-1' },
|
||||
);
|
||||
|
||||
expect(result).toEqual({ items: [] });
|
||||
// Blank query is rejected before any query builder is constructed.
|
||||
expect(db.selectFrom).not.toHaveBeenCalled();
|
||||
expect(result.items).toEqual([]);
|
||||
expect(result.total).toBe(0);
|
||||
expect(shareRepo.findById).not.toHaveBeenCalled();
|
||||
expect(spaceMemberRepo.getUserSpaceIds).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('scopes to the explicit spaceId branch', async () => {
|
||||
const { service, builder, db, spaceMemberRepo, shareRepo } = makeService({
|
||||
rows: [{ id: 'p-1' }],
|
||||
});
|
||||
|
||||
it('only-negation short-circuits with reason "only-negation", never scanning', async () => {
|
||||
const { service, spaceMemberRepo } = makeService();
|
||||
const result = await service.searchPage(
|
||||
{ query: 'plan', spaceId: 'space-42' } as any,
|
||||
{ query: '-архив' } as any,
|
||||
{ userId: 'user-1', workspaceId: 'ws-1' },
|
||||
);
|
||||
|
||||
expect(db.selectFrom).toHaveBeenCalledWith('pages');
|
||||
// The explicit-space branch adds exactly `.where('spaceId', '=', 'space-42')`.
|
||||
expect(whereCallFor(builder, 'spaceId')).toEqual([
|
||||
'spaceId',
|
||||
'=',
|
||||
'space-42',
|
||||
]);
|
||||
// It must NOT fall through to the user-member-spaces or share branch.
|
||||
expect(spaceMemberRepo.getUserSpaceIdsQuery).not.toHaveBeenCalled();
|
||||
expect(shareRepo.findById).not.toHaveBeenCalled();
|
||||
expect(result.items.map((i: any) => i.id)).toEqual(['p-1']);
|
||||
expect(result.total).toBe(0);
|
||||
expect(result.query.parsed.reason).toBe('only-negation');
|
||||
// Never resolves scope (returns before) — no expensive NOT-only scan.
|
||||
expect(spaceMemberRepo.getUserSpaceIds).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('scopes an authenticated user WITHOUT spaceId to their member spaces', async () => {
|
||||
const { service, builder, spaceMemberRepo, userSpaceIdsQuery, shareRepo } =
|
||||
makeService({ rows: [{ id: 'p-9' }] });
|
||||
|
||||
await service.searchPage(
|
||||
{ query: 'plan' } as any,
|
||||
{ userId: 'user-7', workspaceId: 'ws-1' },
|
||||
);
|
||||
|
||||
// The user-scoped branch resolves the member-spaces sub-query for that user
|
||||
// and restricts both spaceId (to that sub-query) and workspaceId.
|
||||
expect(spaceMemberRepo.getUserSpaceIdsQuery).toHaveBeenCalledWith('user-7');
|
||||
expect(whereCallFor(builder, 'spaceId')).toEqual([
|
||||
'spaceId',
|
||||
'in',
|
||||
userSpaceIdsQuery,
|
||||
]);
|
||||
expect(whereCallFor(builder, 'workspaceId')).toEqual([
|
||||
'workspaceId',
|
||||
'=',
|
||||
'ws-1',
|
||||
]);
|
||||
// Authenticated user path must not consult shares.
|
||||
expect(shareRepo.findById).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('returns {items:[]} when the share belongs to a DIFFERENT workspace', async () => {
|
||||
const { service, builder, shareRepo, pagePermissionRepo } = makeService({
|
||||
share: {
|
||||
id: 'share-1',
|
||||
pageId: 'page-1',
|
||||
workspaceId: 'OTHER-ws',
|
||||
includeSubPages: false,
|
||||
},
|
||||
it('returns empty when the share belongs to a DIFFERENT workspace (no leak)', async () => {
|
||||
const { service, shareRepo, pagePermissionRepo } = makeService({
|
||||
share: { id: 's1', pageId: 'p1', workspaceId: 'OTHER', includeSubPages: false },
|
||||
});
|
||||
|
||||
const result = await service.searchPage(
|
||||
{ query: 'plan', shareId: 'share-1' } as any,
|
||||
{ query: 'plan', shareId: 's1' } as any,
|
||||
{ workspaceId: 'ws-1' },
|
||||
);
|
||||
|
||||
expect(shareRepo.findById).toHaveBeenCalledWith('share-1');
|
||||
expect(result).toEqual({ items: [] });
|
||||
// Workspace mismatch short-circuits before any restricted-ancestor / id
|
||||
// scoping or DB execution: no leak across workspaces.
|
||||
expect(shareRepo.findById).toHaveBeenCalledWith('s1');
|
||||
expect(result.items).toEqual([]);
|
||||
// Workspace mismatch short-circuits before restricted-ancestor / enumeration.
|
||||
expect(pagePermissionRepo.hasRestrictedAncestor).not.toHaveBeenCalled();
|
||||
expect(builder.execute).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('returns {items:[]} when the shared page has a restricted ancestor', async () => {
|
||||
const { service, builder, pagePermissionRepo, pageRepo } = makeService({
|
||||
share: {
|
||||
id: 'share-1',
|
||||
pageId: 'page-1',
|
||||
workspaceId: 'ws-1',
|
||||
includeSubPages: true,
|
||||
},
|
||||
it('returns empty when the shared page has a restricted ancestor', async () => {
|
||||
const { service, pagePermissionRepo, pageRepo } = makeService({
|
||||
share: { id: 's1', pageId: 'p1', workspaceId: 'ws-1', includeSubPages: true },
|
||||
isRestricted: true,
|
||||
});
|
||||
|
||||
const result = await service.searchPage(
|
||||
{ query: 'plan', shareId: 'share-1' } as any,
|
||||
{ query: 'plan', shareId: 's1' } as any,
|
||||
{ workspaceId: 'ws-1' },
|
||||
);
|
||||
|
||||
expect(pagePermissionRepo.hasRestrictedAncestor).toHaveBeenCalledWith(
|
||||
'page-1',
|
||||
);
|
||||
expect(result).toEqual({ items: [] });
|
||||
// Restricted ancestor must block before page enumeration and DB execution.
|
||||
expect(pagePermissionRepo.hasRestrictedAncestor).toHaveBeenCalledWith('p1');
|
||||
expect(result.items).toEqual([]);
|
||||
expect(
|
||||
pageRepo.getPageAndDescendantsExcludingRestricted,
|
||||
).not.toHaveBeenCalled();
|
||||
expect(builder.execute).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('returns {items:[]} with no userId, no spaceId and no shareId', async () => {
|
||||
const { service, builder, shareRepo } = makeService();
|
||||
|
||||
it('returns empty with no userId, no spaceId and no shareId', async () => {
|
||||
const { service, shareRepo } = makeService();
|
||||
const result = await service.searchPage(
|
||||
{ query: 'plan' } as any,
|
||||
{ workspaceId: 'ws-1' },
|
||||
);
|
||||
|
||||
expect(result).toEqual({ items: [] });
|
||||
// The catch-all else returns empty without scoping/executing or hitting shares.
|
||||
expect(result.items).toEqual([]);
|
||||
expect(shareRepo.findById).not.toHaveBeenCalled();
|
||||
expect(builder.execute).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('an authenticated user with NO member spaces gets an empty result', async () => {
|
||||
const { service, spaceMemberRepo } = makeService({ memberSpaceIds: [] });
|
||||
const result = await service.searchPage(
|
||||
{ query: 'plan' } as any,
|
||||
{ userId: 'user-1', workspaceId: 'ws-1' },
|
||||
);
|
||||
expect(spaceMemberRepo.getUserSpaceIds).toHaveBeenCalledWith('user-1');
|
||||
expect(result.items).toEqual([]);
|
||||
expect(result.total).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { SearchService, buildTsQuery } from './search.service';
|
||||
import { SearchService } from './search.service';
|
||||
|
||||
describe('SearchService', () => {
|
||||
it('should be defined', () => {
|
||||
@@ -99,59 +99,3 @@ describe('SearchService.searchSuggestions — onlyTemplates filter', () => {
|
||||
expect(isTemplateWhereCall(pageBuilder)).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
// Unit tests for `buildTsQuery` (extracted from search.service.ts). It turns a raw
|
||||
// user query into a prefix tsquery string fed to `to_tsquery('english', ...)`.
|
||||
//
|
||||
// REAL BUG (Gitea #139, item 10): the previous inline `tsquery(query.trim() + '*')`
|
||||
// let to_tsquery operator characters through, so adversarial inputs could produce a
|
||||
// fragment that to_tsquery rejects -> 500. The extraction sanitizes the input
|
||||
// (strip everything but letters/numbers/whitespace) so these inputs degrade to a
|
||||
// safe, neutral query with NO throw, while normal queries keep working.
|
||||
describe('buildTsQuery', () => {
|
||||
it('builds a prefix query for a normal single word', () => {
|
||||
expect(buildTsQuery('hello')).toBe('hello:*');
|
||||
});
|
||||
|
||||
it('joins multiple words with AND and a trailing prefix match', () => {
|
||||
expect(buildTsQuery('foo bar')).toBe('foo&bar:*');
|
||||
});
|
||||
|
||||
it('preserves accented and non-Latin words', () => {
|
||||
expect(buildTsQuery('héllo café')).toBe('héllo&café:*');
|
||||
expect(buildTsQuery('日本語')).toBe('日本語:*');
|
||||
});
|
||||
|
||||
it('neutralizes to_tsquery operator inputs without throwing', () => {
|
||||
// Each of these previously risked an invalid to_tsquery -> 500. They must now
|
||||
// produce a safe (here empty) query and never throw.
|
||||
for (const input of ['&', '!', '*', '<->', '\\']) {
|
||||
expect(() => buildTsQuery(input)).not.toThrow();
|
||||
expect(buildTsQuery(input)).toBe('');
|
||||
}
|
||||
});
|
||||
|
||||
it('handles stopword-only input safely', () => {
|
||||
// pg-tsquery still tokenizes stopwords; to_tsquery reduces them to nothing.
|
||||
// The important contract is: no throw, and a deterministic string.
|
||||
expect(() => buildTsQuery('the a of')).not.toThrow();
|
||||
expect(buildTsQuery('the a of')).toBe('the&a&of:*');
|
||||
});
|
||||
|
||||
it('returns empty string for empty / whitespace-only / null-ish input', () => {
|
||||
expect(buildTsQuery('')).toBe('');
|
||||
expect(buildTsQuery(' ')).toBe('');
|
||||
expect(buildTsQuery(undefined as unknown as string)).toBe('');
|
||||
});
|
||||
|
||||
it('handles a very long input without throwing', () => {
|
||||
const long = 'a'.repeat(10000);
|
||||
expect(() => buildTsQuery(long)).not.toThrow();
|
||||
expect(buildTsQuery(long)).toBe(`${long}:*`);
|
||||
});
|
||||
|
||||
it('strips punctuation embedded in otherwise valid words', () => {
|
||||
expect(buildTsQuery('c++ code')).toBe('c&code:*');
|
||||
expect(buildTsQuery('a-b-c')).toBe('a&b&c:*');
|
||||
});
|
||||
});
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,5 +1,6 @@
|
||||
import {
|
||||
IsBoolean,
|
||||
IsIn,
|
||||
IsNotEmpty,
|
||||
IsOptional,
|
||||
IsString,
|
||||
@@ -18,6 +19,13 @@ export class CreateShareDto {
|
||||
@IsOptional()
|
||||
@IsBoolean()
|
||||
searchIndexing: boolean;
|
||||
|
||||
// #370 Stage B — publication mode. 'live' (default) serves the current draft;
|
||||
// 'approved' serves the last manually-saved version (page_history.kind
|
||||
// ='manual'). Mutually exclusive with includeSubPages (enforced server-side).
|
||||
@IsOptional()
|
||||
@IsIn(['live', 'approved'])
|
||||
publishedMode?: 'live' | 'approved';
|
||||
}
|
||||
|
||||
export class UpdateShareDto extends CreateShareDto {
|
||||
|
||||
@@ -39,6 +39,7 @@ function buildService() {
|
||||
tokenService as any,
|
||||
{} as any, // transclusionService (unused)
|
||||
workspaceRepo as any,
|
||||
{} as any, // pageHistoryRepo (unused on this path)
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -39,6 +39,7 @@ function buildService(over: {
|
||||
{} as any, // tokenService
|
||||
{} as any, // transclusionService
|
||||
{} as any, // workspaceRepo
|
||||
{} as any, // pageHistoryRepo
|
||||
);
|
||||
|
||||
jest
|
||||
|
||||
@@ -62,6 +62,7 @@ function buildService(opts: {
|
||||
tokenService as any,
|
||||
{} as any, // transclusionService (unused)
|
||||
workspaceRepo as any,
|
||||
{} as any, // pageHistoryRepo (unused on this path)
|
||||
);
|
||||
|
||||
// getSharedPage resolves the share via getShareForPage (a raw db query).
|
||||
@@ -181,6 +182,7 @@ describe('ShareService.lookupTransclusionForShare htmlEmbed kill-switch (real co
|
||||
tokenService as any,
|
||||
transclusionService as any,
|
||||
workspaceRepo as any,
|
||||
{} as any, // pageHistoryRepo (unused on this path)
|
||||
);
|
||||
|
||||
// isSharingAllowed and getShareForPage hit the raw db; stub them so the
|
||||
|
||||
@@ -23,6 +23,9 @@ function buildService(over: {
|
||||
resolvedShare?: unknown;
|
||||
page?: unknown;
|
||||
restricted?: boolean;
|
||||
// #370 Stage B — the latest manual page-history row returned for the approved
|
||||
// swap. `undefined` (the default) means the repo finds no manual version.
|
||||
manualHistory?: unknown;
|
||||
} = {}) {
|
||||
const pageRepo = {
|
||||
findById: jest.fn(async () =>
|
||||
@@ -34,6 +37,9 @@ function buildService(over: {
|
||||
const pagePermissionRepo = {
|
||||
hasRestrictedAncestor: jest.fn(async () => over.restricted ?? false),
|
||||
};
|
||||
const pageHistoryRepo = {
|
||||
findLatestByPageIdAndKind: jest.fn(async () => over.manualHistory ?? undefined),
|
||||
};
|
||||
|
||||
const service = new ShareService(
|
||||
{} as any, // shareRepo (unused on this path)
|
||||
@@ -43,6 +49,7 @@ function buildService(over: {
|
||||
{} as any, // tokenService (unused)
|
||||
{} as any, // transclusionService (unused)
|
||||
{} as any, // workspaceRepo (unused)
|
||||
pageHistoryRepo as any,
|
||||
);
|
||||
|
||||
jest
|
||||
@@ -53,7 +60,7 @@ function buildService(over: {
|
||||
: { id: SHARE, pageId: PAGE, spaceId: 'space-1' }) as any,
|
||||
);
|
||||
|
||||
return { service, pageRepo, pagePermissionRepo };
|
||||
return { service, pageRepo, pagePermissionRepo, pageHistoryRepo };
|
||||
}
|
||||
|
||||
describe('ShareService.resolveReadableSharePage (the share-access boundary)', () => {
|
||||
@@ -133,4 +140,90 @@ describe('ShareService.resolveReadableSharePage (the share-access boundary)', ()
|
||||
includeCreator: true,
|
||||
});
|
||||
});
|
||||
|
||||
// #370 Stage B — "approved" publication freezes the served content/title to
|
||||
// the latest manual page-history version, WITHOUT touching page.id or
|
||||
// page.workspaceId (downstream attachment tokens are minted off those, so
|
||||
// freezing them would mint tokens for the wrong owner).
|
||||
describe('approved publication mode (content freeze)', () => {
|
||||
const LIVE_PAGE = {
|
||||
id: PAGE,
|
||||
workspaceId: WS,
|
||||
deletedAt: null,
|
||||
title: 'Live draft title',
|
||||
content: { type: 'doc', live: true },
|
||||
};
|
||||
const MANUAL = {
|
||||
title: 'Saved version title',
|
||||
content: { type: 'doc', saved: true },
|
||||
};
|
||||
|
||||
it('swaps content + title to the latest manual version but PRESERVES id + workspaceId', async () => {
|
||||
const { service, pageHistoryRepo } = buildService({
|
||||
resolvedShare: {
|
||||
id: SHARE,
|
||||
pageId: PAGE,
|
||||
spaceId: 'space-1',
|
||||
publishedMode: 'approved',
|
||||
},
|
||||
page: { ...LIVE_PAGE },
|
||||
manualHistory: MANUAL,
|
||||
});
|
||||
|
||||
const out = await service.resolveReadableSharePage(SHARE, PAGE, WS);
|
||||
|
||||
expect(out).not.toBeNull();
|
||||
// Content + title come from the saved version.
|
||||
expect(out!.page.content).toEqual(MANUAL.content);
|
||||
expect(out!.page.title).toBe(MANUAL.title);
|
||||
// id + workspaceId stay LIVE (attachment-token ownership must not shift).
|
||||
expect(out!.page.id).toBe(LIVE_PAGE.id);
|
||||
expect(out!.page.workspaceId).toBe(LIVE_PAGE.workspaceId);
|
||||
// Resolved against the LIVE page id and the manual tier, with content.
|
||||
expect(pageHistoryRepo.findLatestByPageIdAndKind).toHaveBeenCalledWith(
|
||||
LIVE_PAGE.id,
|
||||
'manual',
|
||||
{ includeContent: true },
|
||||
);
|
||||
});
|
||||
|
||||
it('falls back to live content when no manual version exists yet', async () => {
|
||||
const { service, pageHistoryRepo } = buildService({
|
||||
resolvedShare: {
|
||||
id: SHARE,
|
||||
pageId: PAGE,
|
||||
spaceId: 'space-1',
|
||||
publishedMode: 'approved',
|
||||
},
|
||||
page: { ...LIVE_PAGE },
|
||||
// manualHistory omitted → repo returns undefined
|
||||
});
|
||||
|
||||
const out = await service.resolveReadableSharePage(SHARE, PAGE, WS);
|
||||
|
||||
expect(out).not.toBeNull();
|
||||
expect(out!.page.content).toEqual(LIVE_PAGE.content);
|
||||
expect(out!.page.title).toBe(LIVE_PAGE.title);
|
||||
expect(pageHistoryRepo.findLatestByPageIdAndKind).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does NOT consult page-history for a live share', async () => {
|
||||
const { service, pageHistoryRepo } = buildService({
|
||||
resolvedShare: {
|
||||
id: SHARE,
|
||||
pageId: PAGE,
|
||||
spaceId: 'space-1',
|
||||
publishedMode: 'live',
|
||||
},
|
||||
page: { ...LIVE_PAGE },
|
||||
manualHistory: MANUAL,
|
||||
});
|
||||
|
||||
const out = await service.resolveReadableSharePage(SHARE, PAGE, WS);
|
||||
|
||||
expect(out).not.toBeNull();
|
||||
expect(out!.page.content).toEqual(LIVE_PAGE.content);
|
||||
expect(pageHistoryRepo.findLatestByPageIdAndKind).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -33,6 +33,7 @@ function buildService() {
|
||||
tokenService as any,
|
||||
{} as any, // transclusionService (unused)
|
||||
workspaceRepo as any,
|
||||
{} as any, // pageHistoryRepo (unused on this path)
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -19,6 +19,7 @@ import {
|
||||
} from '../../common/helpers/prosemirror/utils';
|
||||
import { Node } from '@tiptap/pm/model';
|
||||
import { ShareRepo } from '@docmost/db/repos/share/share.repo';
|
||||
import { PageHistoryRepo } from '@docmost/db/repos/page/page-history.repo';
|
||||
import { PagePermissionRepo } from '@docmost/db/repos/page/page-permission.repo';
|
||||
import { updateAttachmentAttr } from './share.util';
|
||||
import { Page } from '@docmost/db/types/entity.types';
|
||||
@@ -44,6 +45,7 @@ export class ShareService {
|
||||
private readonly tokenService: TokenService,
|
||||
private readonly transclusionService: TransclusionService,
|
||||
private readonly workspaceRepo: WorkspaceRepo,
|
||||
private readonly pageHistoryRepo: PageHistoryRepo,
|
||||
) {}
|
||||
|
||||
/**
|
||||
@@ -92,21 +94,39 @@ export class ShareService {
|
||||
}) {
|
||||
const { authUserId, workspaceId, page, createShareDto } = opts;
|
||||
|
||||
const includeSubPages = createShareDto.includeSubPages ?? false;
|
||||
const publishedMode = createShareDto.publishedMode ?? 'live';
|
||||
|
||||
// #370 Stage B — 'approved' freezes a SINGLE page to its saved version; a
|
||||
// sub-tree cannot be version-frozen in the MVP, so the two are mutually
|
||||
// exclusive. Thrown before the try so the specific 400 reaches the client
|
||||
// instead of being flattened to the generic "Failed to share page".
|
||||
this.assertPublishedModeCompatible(publishedMode, includeSubPages);
|
||||
|
||||
try {
|
||||
const shares = await this.shareRepo.findByPageId(page.id);
|
||||
if (shares) {
|
||||
return shares;
|
||||
}
|
||||
|
||||
return await this.shareRepo.insertShare({
|
||||
const share = await this.shareRepo.insertShare({
|
||||
key: nanoIdGen().toLowerCase(),
|
||||
pageId: page.id,
|
||||
includeSubPages: createShareDto.includeSubPages ?? false,
|
||||
includeSubPages,
|
||||
searchIndexing: createShareDto.searchIndexing ?? false,
|
||||
publishedMode,
|
||||
creatorId: authUserId,
|
||||
spaceId: page.spaceId,
|
||||
workspaceId,
|
||||
});
|
||||
|
||||
// Guarantee an approved share always has a saved version to serve: mint
|
||||
// the first manual baseline from the page's current content on enable.
|
||||
if (publishedMode === 'approved') {
|
||||
await this.ensureApprovedBaseline(page.id);
|
||||
}
|
||||
|
||||
return share;
|
||||
} catch (err) {
|
||||
this.logger.error(err);
|
||||
throw new BadRequestException('Failed to share page');
|
||||
@@ -114,20 +134,84 @@ export class ShareService {
|
||||
}
|
||||
|
||||
async updateShare(shareId: string, updateShareDto: UpdateShareDto) {
|
||||
// Resolve the current share so the XOR gate can be evaluated against the
|
||||
// EFFECTIVE post-update state (either field may be absent from the DTO).
|
||||
const current = await this.shareRepo.findById(shareId);
|
||||
if (!current) {
|
||||
throw new NotFoundException('Share not found');
|
||||
}
|
||||
|
||||
const effectiveIncludeSubPages =
|
||||
updateShareDto.includeSubPages ?? current.includeSubPages ?? false;
|
||||
const effectivePublishedMode = (updateShareDto.publishedMode ??
|
||||
current.publishedMode ??
|
||||
'live') as 'live' | 'approved';
|
||||
|
||||
// #370 Stage B — enforce the approved/includeSubPages XOR on the merged
|
||||
// state. Thrown before the try so the specific 400 reaches the client.
|
||||
this.assertPublishedModeCompatible(
|
||||
effectivePublishedMode,
|
||||
effectiveIncludeSubPages,
|
||||
);
|
||||
|
||||
try {
|
||||
return this.shareRepo.updateShare(
|
||||
const updated = await this.shareRepo.updateShare(
|
||||
{
|
||||
includeSubPages: updateShareDto.includeSubPages,
|
||||
searchIndexing: updateShareDto.searchIndexing,
|
||||
publishedMode: updateShareDto.publishedMode,
|
||||
},
|
||||
shareId,
|
||||
);
|
||||
|
||||
// On (or while) enabling approved mode, ensure a manual baseline exists so
|
||||
// public readers always have a saved version to serve. Idempotent: a no-op
|
||||
// when the page already has a manual history row.
|
||||
if (effectivePublishedMode === 'approved') {
|
||||
await this.ensureApprovedBaseline(current.pageId);
|
||||
}
|
||||
|
||||
return updated;
|
||||
} catch (err) {
|
||||
this.logger.error(err);
|
||||
throw new BadRequestException('Failed to update share');
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* #370 Stage B — 'approved' publication is mutually exclusive with
|
||||
* includeSubPages: a whole sub-tree cannot be version-frozen in the MVP.
|
||||
*/
|
||||
private assertPublishedModeCompatible(
|
||||
publishedMode: 'live' | 'approved',
|
||||
includeSubPages: boolean,
|
||||
): void {
|
||||
if (publishedMode === 'approved' && includeSubPages) {
|
||||
throw new BadRequestException(
|
||||
'Approved publication cannot be combined with including sub-pages',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* #370 Stage B — guarantee an approved share has a saved version to serve.
|
||||
* When the page has NO manual history row yet, snapshot its CURRENT content as
|
||||
* the first manual version. Idempotent: returns early when a manual version
|
||||
* already exists, so it is safe to call on every approved update.
|
||||
*/
|
||||
private async ensureApprovedBaseline(pageId: string): Promise<void> {
|
||||
const existing = await this.pageHistoryRepo.findLatestByPageIdAndKind(
|
||||
pageId,
|
||||
'manual',
|
||||
);
|
||||
if (existing) return;
|
||||
|
||||
const page = await this.pageRepo.findById(pageId, { includeContent: true });
|
||||
if (!page) return;
|
||||
|
||||
await this.pageHistoryRepo.saveHistory(page, { kind: 'manual' });
|
||||
}
|
||||
|
||||
/**
|
||||
* THE share access boundary in ONE place.
|
||||
*
|
||||
@@ -173,7 +257,7 @@ export class ShareService {
|
||||
// passes no shareId (it resolved the share from the page itself).
|
||||
if (shareId != null && share.id !== shareId) return null;
|
||||
|
||||
const page = await this.pageRepo.findById(pageId, {
|
||||
let page = await this.pageRepo.findById(pageId, {
|
||||
includeContent: true,
|
||||
includeCreator: opts?.includeCreator ?? false,
|
||||
});
|
||||
@@ -185,6 +269,30 @@ export class ShareService {
|
||||
return null;
|
||||
}
|
||||
|
||||
// #370 Stage B — "approved" publication freezes what public readers see to
|
||||
// the LAST manually-saved version (page_history.kind='manual'), not the live
|
||||
// draft. This is the SINGLE canonical resolver every public read funnels
|
||||
// through (the shared-page view, the SEO controller, AND the share
|
||||
// assistant's page-read tool), so the swap here freezes ALL of them
|
||||
// consistently — a public visitor and the assistant see the same bytes.
|
||||
//
|
||||
// Swap ONLY content + title. page.id and page.workspaceId stay LIVE on
|
||||
// purpose: downstream updatePublicAttachments mints per-attachment tokens
|
||||
// off page.id/workspaceId, so freezing those would mint tokens for the
|
||||
// wrong owner. The SEO controller reads resolved.page.title, so the frozen
|
||||
// title flows through automatically. If no manual version exists yet (rare —
|
||||
// enabling an approved share auto-creates a baseline), fall back to live.
|
||||
if (share.publishedMode === 'approved') {
|
||||
const hist = await this.pageHistoryRepo.findLatestByPageIdAndKind(
|
||||
page.id,
|
||||
'manual',
|
||||
{ includeContent: true },
|
||||
);
|
||||
if (hist) {
|
||||
page = { ...page, content: hist.content, title: hist.title };
|
||||
}
|
||||
}
|
||||
|
||||
return { share, page };
|
||||
}
|
||||
|
||||
@@ -302,6 +410,7 @@ export class ShareService {
|
||||
'shares.key as shareKey',
|
||||
'shares.includeSubPages',
|
||||
'shares.searchIndexing',
|
||||
'shares.publishedMode',
|
||||
'shares.creatorId',
|
||||
'shares.spaceId',
|
||||
'shares.workspaceId',
|
||||
@@ -326,6 +435,7 @@ export class ShareService {
|
||||
's.key as shareKey',
|
||||
's.includeSubPages',
|
||||
's.searchIndexing',
|
||||
's.publishedMode',
|
||||
's.creatorId',
|
||||
's.spaceId',
|
||||
's.workspaceId',
|
||||
@@ -355,6 +465,7 @@ export class ShareService {
|
||||
key: share.shareKey,
|
||||
includeSubPages: share.includeSubPages,
|
||||
searchIndexing: share.searchIndexing,
|
||||
publishedMode: share.publishedMode,
|
||||
pageId: share.id,
|
||||
creatorId: share.creatorId,
|
||||
spaceId: share.spaceId,
|
||||
|
||||
@@ -0,0 +1,242 @@
|
||||
import { type Kysely, sql } from 'kysely';
|
||||
|
||||
/**
|
||||
* #529 Phase A1 — the `ru_en` text-search configuration + the config swap.
|
||||
*
|
||||
* WHY: the search stack was pinned to the `english` FTS config, which stems only
|
||||
* Latin words. On a Russian-language wiki that is a morphology black hole:
|
||||
* «ресторанов москвы» never matched a page titled «ресторан в москве». `ru_en`
|
||||
* layers the russian_stem over the Cyrillic token classes and english_stem over
|
||||
* the ascii ones, so BOTH languages get proper morphology from one config.
|
||||
*
|
||||
* CREATE TEXT SEARCH CONFIGURATION ru_en (COPY = simple);
|
||||
* ALTER ... asciiword/asciihword/hword_asciipart WITH english_stem;
|
||||
* ALTER ... word/hword/hword_part WITH russian_stem;
|
||||
*
|
||||
* `to_tsvector('ru_en', …)` with the LITERAL config name is IMMUTABLE, so it is
|
||||
* valid inside a trigger, a generated column and an index expression.
|
||||
*
|
||||
* THE INVARIANT (acceptance #13): the config of the STORED column and the config
|
||||
* of the QUERY must change together. This migration flips BOTH stored sides
|
||||
* (pages.tsv via its trigger + a reindex; page_embeddings.fts, the RAG lexical
|
||||
* leg, via its generated expression); the matching QUERY-side flips
|
||||
* (search.service.ts and page-embedding.repo.ts `hybridSearch`) ship in the SAME
|
||||
* commit. Trigram indexes are LOWER(f_unaccent(...)) and do NOT depend on the FTS
|
||||
* config, so they are untouched.
|
||||
*
|
||||
* REINDEX / LOCK MODEL (deploy-critical). Kysely runs EACH migration in its OWN
|
||||
* transaction (see sibling 20260706T120000 "Kysely runs each migration in a
|
||||
* transaction"; migrate.ts / migration.service.ts set no `disableTransactions`,
|
||||
* and PostgresJSDialect has transactional DDL). A single migration file therefore
|
||||
* cannot commit between batches, so the issue's "procedural batch job OUTSIDE the
|
||||
* transaction + dual-config read window + migration_complete gate" is not
|
||||
* expressible in-migration here. That machinery bridges a reindex spread over
|
||||
* MANY committed batches, during which some rows are still `english` while others
|
||||
* are already `ru_en`. Our pages.tsv reindex is a SINGLE `UPDATE pages SET tsv`,
|
||||
* ATOMIC within THIS migration's own transaction: at COMMIT every row is `ru_en`
|
||||
* at once, so no morphology-desync window exists and no dual-config read path is
|
||||
* required — the query config flips to `ru_en` in the very same release. This is
|
||||
* the deliberate, correct adaptation to this framework (see the PR notes).
|
||||
*
|
||||
* - pages.tsv: swapping the trigger is a cheap catalog change (no table lock),
|
||||
* and the reindex is a single `UPDATE pages SET tsv = <ru_en expr>` — a
|
||||
* ROW-level-lock (RowExclusiveLock) backfill, NOT an ACCESS EXCLUSIVE rewrite
|
||||
* (mirrors the existing space_id backfill in 20250725T052004). On a LARGE
|
||||
* tenant this still writes every row + its WAL and leaves dead tuples, so it
|
||||
* can take MINUTES and blocks the startup migrator for that time — but it
|
||||
* never blocks concurrent reads. It stays inline unconditionally.
|
||||
*
|
||||
* - page_embeddings.fts is a GENERATED STORED column; Postgres cannot change a
|
||||
* generated expression without DROP+ADD, which is a full-table ACCESS
|
||||
* EXCLUSIVE REWRITE of page_embeddings — it blocks ALL reads AND writes on
|
||||
* that table (including the RAG agent) for the rewrite's duration. That inline
|
||||
* rewrite is appropriate for small/typical tenants (this fork's target) and
|
||||
* is the DEFAULT.
|
||||
*
|
||||
* LARGE TENANTS have two documented escape hatches, either of which makes the
|
||||
* migration genuinely no-op the rewrite (it is NOT a blind DROP+ADD):
|
||||
* (a) Set `SEARCH_EMBEDDINGS_FTS_INLINE_REWRITE=false`. The migration then
|
||||
* SKIPS the embeddings rewrite entirely and logs a WARNING. The operator
|
||||
* MUST perform the ru_en fts swap out-of-band; until they do, the RAG
|
||||
* lexical leg stays on `english` while the query config is `ru_en` — a
|
||||
* documented, operator-owned desync window. (pages.tsv still swaps
|
||||
* inline — the gate is ONLY the embeddings rewrite.)
|
||||
* (b) Perform the swap out-of-band BEFORE deploy — add a plain column →
|
||||
* batched backfill → brief-lock swap → CREATE INDEX CONCURRENTLY — so
|
||||
* the `fts` column's generated expression already references the TARGET
|
||||
* config when the migration runs. The migration detects this (it reads
|
||||
* the column's actual generation expression from pg_catalog) and does a
|
||||
* TRUE no-op — no DROP, no ADD, no rewrite. This is real idempotency,
|
||||
* not the old (false) "IF-EXISTS guards no-op" claim: `DROP COLUMN IF
|
||||
* EXISTS` guards against ABSENCE, not presence, so it would have dropped
|
||||
* and recreated an existing `fts` regardless. The at-target check is the
|
||||
* only honest no-op path.
|
||||
*
|
||||
* Same documented trade-off family as the #443 trgm GIN migration (20260706T120000).
|
||||
*/
|
||||
|
||||
// pages.tsv trigger body for a given FTS config — mirrors the latest form
|
||||
// (20250729T213756): f_unaccent + a 1MB text cap on text_content, weights A/B.
|
||||
function pagesTriggerSql(config: 'ru_en' | 'english') {
|
||||
return sql`
|
||||
CREATE OR REPLACE FUNCTION pages_tsvector_trigger() RETURNS trigger AS $$
|
||||
begin
|
||||
new.tsv :=
|
||||
setweight(to_tsvector('${sql.raw(config)}', f_unaccent(coalesce(new.title, ''))), 'A') ||
|
||||
setweight(to_tsvector('${sql.raw(config)}', f_unaccent(substring(coalesce(new.text_content, ''), 1, 1000000))), 'B');
|
||||
return new;
|
||||
end;
|
||||
$$ LANGUAGE plpgsql;
|
||||
`;
|
||||
}
|
||||
|
||||
async function swapPagesConfig(db: Kysely<any>, config: 'ru_en' | 'english') {
|
||||
// 1. Point the trigger at the target config (new/edited rows use it going
|
||||
// forward). CREATE OR REPLACE FUNCTION takes only a brief catalog lock.
|
||||
await pagesTriggerSql(config).execute(db);
|
||||
|
||||
// 2. Reindex existing rows: recompute tsv directly with the target config. A
|
||||
// plain UPDATE — row locks, no ACCESS EXCLUSIVE. Equivalent to firing the
|
||||
// trigger but cheaper (no self-update round trip).
|
||||
await sql`
|
||||
UPDATE pages
|
||||
SET tsv =
|
||||
setweight(to_tsvector('${sql.raw(config)}', f_unaccent(coalesce(title, ''))), 'A') ||
|
||||
setweight(to_tsvector('${sql.raw(config)}', f_unaccent(substring(coalesce(text_content, ''), 1, 1000000))), 'B')
|
||||
`.execute(db);
|
||||
}
|
||||
|
||||
// The default in-migration ACCESS EXCLUSIVE rewrite of page_embeddings.fts is
|
||||
// ON unless the operator explicitly opts out with the env flag. Parsed strictly
|
||||
// (mirrors CLIENT_TELEMETRY_ENABLED / DEBUG_MODE in common/): only a literal
|
||||
// (case-insensitive) 'false' disables it; anything else — unset included —
|
||||
// keeps the default true.
|
||||
function inlineEmbeddingsRewriteEnabled(): boolean {
|
||||
return (
|
||||
(process.env.SEARCH_EMBEDDINGS_FTS_INLINE_REWRITE ?? 'true').toLowerCase() !==
|
||||
'false'
|
||||
);
|
||||
}
|
||||
|
||||
// Read page_embeddings.fts's ACTUAL generated-column expression from pg_catalog
|
||||
// (the generation expression is stored as a column default marked generated).
|
||||
// Returns '' when the column is absent.
|
||||
async function embeddingsFtsExpr(db: Kysely<any>): Promise<string> {
|
||||
const r = await sql<{ def: string }>`
|
||||
SELECT pg_get_expr(d.adbin, d.adrelid) AS def
|
||||
FROM pg_attrdef d
|
||||
JOIN pg_attribute a ON a.attrelid = d.adrelid AND a.attnum = d.adnum
|
||||
WHERE a.attname = 'fts'
|
||||
AND a.attrelid = 'page_embeddings'::regclass
|
||||
AND NOT a.attisdropped
|
||||
`.execute(db);
|
||||
return r.rows[0]?.def ?? '';
|
||||
}
|
||||
|
||||
async function swapEmbeddingsFtsConfig(
|
||||
db: Kysely<any>,
|
||||
config: 'ru_en' | 'english',
|
||||
) {
|
||||
// 1. TRUE no-op path (real out-of-band escape hatch): if the column's current
|
||||
// generation expression already references the TARGET config, there is
|
||||
// nothing to do. An operator who pre-swapped the column out-of-band lands
|
||||
// here and the migration does NOT rewrite the table. ('ru_en' and 'english'
|
||||
// are disjoint tokens, neither a substring of the other or of the rest of
|
||||
// the expression, so a plain contains-check is unambiguous.)
|
||||
const currentExpr = await embeddingsFtsExpr(db);
|
||||
if (currentExpr.includes(config)) return;
|
||||
|
||||
// 2. Env-gated opt-out: large tenants set SEARCH_EMBEDDINGS_FTS_INLINE_REWRITE
|
||||
// =false to skip the ACCESS EXCLUSIVE rewrite in-migration and own the swap
|
||||
// out-of-band. Warn loudly so the desync window is not silent.
|
||||
if (!inlineEmbeddingsRewriteEnabled()) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn(
|
||||
`[migration 20260707T130000] SEARCH_EMBEDDINGS_FTS_INLINE_REWRITE=false: ` +
|
||||
`SKIPPING the page_embeddings.fts rewrite to '${config}'. The operator MUST ` +
|
||||
`perform this fts swap out-of-band. Until then the RAG lexical leg stays on ` +
|
||||
`its current config while the query config is '${config}' (documented, ` +
|
||||
`operator-owned desync window).`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
// 3. Default inline path: the generated `fts` expression can only change via
|
||||
// DROP+ADD (a full-table ACCESS EXCLUSIVE rewrite; see the lock note in the
|
||||
// header). The GIN index depends on the column, so it is dropped with it and
|
||||
// recreated.
|
||||
await sql`DROP INDEX IF EXISTS idx_page_embeddings_fts`.execute(db);
|
||||
await sql`ALTER TABLE page_embeddings DROP COLUMN IF EXISTS fts`.execute(db);
|
||||
await sql`
|
||||
ALTER TABLE page_embeddings
|
||||
ADD COLUMN fts tsvector
|
||||
GENERATED ALWAYS AS (to_tsvector('${sql.raw(config)}', f_unaccent(content))) STORED
|
||||
`.execute(db);
|
||||
await sql`
|
||||
CREATE INDEX IF NOT EXISTS idx_page_embeddings_fts
|
||||
ON page_embeddings USING gin(fts)
|
||||
`.execute(db);
|
||||
}
|
||||
|
||||
async function ruEnConfigExists(db: Kysely<any>): Promise<boolean> {
|
||||
const r = await sql<{ n: number }>`
|
||||
SELECT count(*)::int AS n FROM pg_ts_config WHERE cfgname = 'ru_en'
|
||||
`.execute(db);
|
||||
return (r.rows[0]?.n ?? 0) > 0;
|
||||
}
|
||||
|
||||
async function ensureRuEnConfig(db: Kysely<any>): Promise<void> {
|
||||
// Idempotent by EXISTENCE, not by drop-recreate. The old `DROP ... IF EXISTS;
|
||||
// CREATE` was safe only on a first run: on a re-run the page_embeddings.fts
|
||||
// generated column already has a hard dependency on ru_en, so dropping the
|
||||
// config would fail. Create only when it is genuinely missing.
|
||||
if (await ruEnConfigExists(db)) return;
|
||||
await sql`CREATE TEXT SEARCH CONFIGURATION ru_en (COPY = simple)`.execute(db);
|
||||
// Latin token classes → english_stem.
|
||||
await sql`
|
||||
ALTER TEXT SEARCH CONFIGURATION ru_en
|
||||
ALTER MAPPING FOR asciiword, asciihword, hword_asciipart
|
||||
WITH english_stem
|
||||
`.execute(db);
|
||||
// Cyrillic / non-ascii token classes → russian_stem.
|
||||
await sql`
|
||||
ALTER TEXT SEARCH CONFIGURATION ru_en
|
||||
ALTER MAPPING FOR word, hword, hword_part
|
||||
WITH russian_stem
|
||||
`.execute(db);
|
||||
}
|
||||
|
||||
export async function up(db: Kysely<any>): Promise<void> {
|
||||
await ensureRuEnConfig(db);
|
||||
|
||||
// Flip both stored sides to ru_en (query-side flips in the same commit).
|
||||
// swapEmbeddingsFtsConfig no-ops when fts already references ru_en, so a
|
||||
// re-run of up() is idempotent and does NOT re-rewrite the embeddings table.
|
||||
await swapPagesConfig(db, 'ru_en');
|
||||
await swapEmbeddingsFtsConfig(db, 'ru_en');
|
||||
}
|
||||
|
||||
export async function down(db: Kysely<any>): Promise<void> {
|
||||
// Reverse ORDER matters: the trigger and the generated column reference the
|
||||
// `ru_en` config by name, so they must be moved back to `english` BEFORE the
|
||||
// config can be dropped (a generated column that still depends on `ru_en` would
|
||||
// block the DROP with a dependency error).
|
||||
await swapEmbeddingsFtsConfig(db, 'english');
|
||||
await swapPagesConfig(db, 'english');
|
||||
|
||||
// Drop the config ONLY if nothing still references it. When the embeddings
|
||||
// rewrite was gated off (SEARCH_EMBEDDINGS_FTS_INLINE_REWRITE=false), the fts
|
||||
// column can still reference ru_en — dropping the config would then fail with a
|
||||
// dependency error. Skip + warn so down() stays non-fatal; the operator drops
|
||||
// ru_en after completing the out-of-band english swap.
|
||||
if ((await embeddingsFtsExpr(db)).includes('ru_en')) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.warn(
|
||||
`[migration 20260707T130000] down(): page_embeddings.fts still references ` +
|
||||
`ru_en (inline rewrite was gated off) — leaving the ru_en text-search ` +
|
||||
`configuration in place. Drop it out-of-band once fts is back on 'english'.`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
await sql`DROP TEXT SEARCH CONFIGURATION IF EXISTS ru_en`.execute(db);
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
import { type Kysely, sql } from 'kysely';
|
||||
|
||||
/**
|
||||
* #370 Stage B — share "approved" (publish-only-the-saved-version) mode.
|
||||
*
|
||||
* Adds `shares.published_mode`, the per-share switch between:
|
||||
* - 'live' — public readers see the current draft (legacy behavior)
|
||||
* - 'approved' — public readers see the LAST manually-saved version
|
||||
* (page_history.kind='manual'), not the live draft
|
||||
*
|
||||
* NOT NULL with a 'live' default so every existing share keeps its current
|
||||
* (live) semantics with no data backfill. Standalone + single-concern +
|
||||
* additive per the #363 crash-loop rule — no coupled baseline write here;
|
||||
* the first manual baseline is minted by the service on enable, not by this
|
||||
* migration. Stored as a short varchar to stay forward-compatible without an
|
||||
* enum migration.
|
||||
*/
|
||||
export async function up(db: Kysely<any>): Promise<void> {
|
||||
await db.schema
|
||||
.alterTable('shares')
|
||||
.addColumn('published_mode', 'varchar(20)', (col) =>
|
||||
col.notNull().defaultTo(sql`'live'`),
|
||||
)
|
||||
.execute();
|
||||
}
|
||||
|
||||
export async function down(db: Kysely<any>): Promise<void> {
|
||||
await db.schema.alterTable('shares').dropColumn('published_mode').execute();
|
||||
}
|
||||
@@ -200,8 +200,11 @@ export class PageEmbeddingRepo {
|
||||
*
|
||||
* The `model_dimensions = $dim` filter applies ONLY on the semantic side
|
||||
* (cosine compares same-dimension vectors; pgvector errors otherwise). The
|
||||
* lexical side (`fts`) is dimension-independent. If `websearch_to_tsquery`
|
||||
* yields an EMPTY query (e.g. the text is all stopwords) the `@@` matches
|
||||
* lexical side (`fts`) is dimension-independent. Its query config is `ru_en`,
|
||||
* matched IN LOCKSTEP with the `page_embeddings.fts` generated column's config
|
||||
* (#529 acceptance #13): a mismatch silently breaks Cyrillic RAG retrieval. If
|
||||
* `websearch_to_tsquery` yields an EMPTY query (e.g. the text is all stopwords)
|
||||
* the `@@` matches
|
||||
* nothing and the lexical CTE is empty, so results degrade to pure-semantic —
|
||||
* which is correct behaviour, not an error.
|
||||
*
|
||||
@@ -249,7 +252,7 @@ export class PageEmbeddingRepo {
|
||||
row_number() OVER (ORDER BY ts_rank(pe.fts, q.query) DESC) AS rank_ix
|
||||
FROM page_embeddings pe
|
||||
JOIN pages p ON p.id = pe.page_id,
|
||||
websearch_to_tsquery('english', f_unaccent(${queryText})) AS q(query)
|
||||
websearch_to_tsquery('ru_en', f_unaccent(${queryText})) AS q(query)
|
||||
WHERE pe.workspace_id = ${workspaceId}
|
||||
AND pe.space_id IN (${spaceList})
|
||||
AND p.deleted_at IS NULL
|
||||
|
||||
@@ -157,6 +157,44 @@ export class PageHistoryRepo {
|
||||
return { ...result, items: result.items.map(attachPageHistoryAgent) };
|
||||
}
|
||||
|
||||
/**
|
||||
* #395 — cheap projection of a page's FULL history timeline for the work-time
|
||||
* estimate: only the columns the sessionizer needs, no heavy `content`, sorted
|
||||
* oldest→newest. The secondary `id` tie-break keeps rows sharing a `createdAt`
|
||||
* (e.g. a synchronous pre-agent boundary row + the immediate agent snapshot)
|
||||
* in a deterministic order.
|
||||
*/
|
||||
async findTimelineByPageId(
|
||||
pageId: string,
|
||||
trx?: KyselyTransaction,
|
||||
): Promise<
|
||||
Array<
|
||||
Pick<
|
||||
PageHistory,
|
||||
| 'createdAt'
|
||||
| 'lastUpdatedById'
|
||||
| 'lastUpdatedSource'
|
||||
| 'lastUpdatedAiChatId'
|
||||
| 'kind'
|
||||
>
|
||||
>
|
||||
> {
|
||||
const db = dbOrTx(this.db, trx);
|
||||
return db
|
||||
.selectFrom('pageHistory')
|
||||
.select([
|
||||
'createdAt',
|
||||
'lastUpdatedById',
|
||||
'lastUpdatedSource',
|
||||
'lastUpdatedAiChatId',
|
||||
'kind',
|
||||
])
|
||||
.where('pageId', '=', pageId)
|
||||
.orderBy('createdAt', 'asc')
|
||||
.orderBy('id', 'asc')
|
||||
.execute();
|
||||
}
|
||||
|
||||
async findPageLastHistory(
|
||||
pageId: string,
|
||||
opts?: {
|
||||
@@ -181,6 +219,36 @@ export class PageHistoryRepo {
|
||||
.executeTakeFirst();
|
||||
}
|
||||
|
||||
/**
|
||||
* #370 Stage B — resolve the latest snapshot of a page at a given
|
||||
* intentionality tier. Used by the "approved" share mode to serve the last
|
||||
* manually-saved version (`kind='manual'`) to public readers instead of the
|
||||
* live draft. Modeled on `findPageLastHistory` with an added `kind` filter and
|
||||
* the same deterministic `(createdAt desc, id desc)` tie-break, so two rows
|
||||
* sharing a createdAt still resolve to a single stable "latest".
|
||||
*/
|
||||
async findLatestByPageIdAndKind(
|
||||
pageId: string,
|
||||
kind: PageHistoryKind,
|
||||
opts?: {
|
||||
includeContent?: boolean;
|
||||
trx?: KyselyTransaction;
|
||||
},
|
||||
) {
|
||||
const db = dbOrTx(this.db, opts?.trx);
|
||||
|
||||
return await db
|
||||
.selectFrom('pageHistory')
|
||||
.select(this.baseFields)
|
||||
.$if(opts?.includeContent, (qb) => qb.select('content'))
|
||||
.where('pageId', '=', pageId)
|
||||
.where('kind', '=', kind)
|
||||
.limit(1)
|
||||
.orderBy('createdAt', 'desc')
|
||||
.orderBy('id', 'desc')
|
||||
.executeTakeFirst();
|
||||
}
|
||||
|
||||
withLastUpdatedBy(eb: ExpressionBuilder<DB, 'pageHistory'>) {
|
||||
return jsonObjectFrom(
|
||||
eb
|
||||
|
||||
@@ -28,6 +28,7 @@ export class ShareRepo {
|
||||
'pageId',
|
||||
'includeSubPages',
|
||||
'searchIndexing',
|
||||
'publishedMode',
|
||||
'creatorId',
|
||||
'spaceId',
|
||||
'workspaceId',
|
||||
|
||||
+1
@@ -340,6 +340,7 @@ export interface Shares {
|
||||
includeSubPages: Generated<boolean | null>;
|
||||
key: string;
|
||||
pageId: string | null;
|
||||
publishedMode: Generated<string>;
|
||||
searchIndexing: Generated<boolean | null>;
|
||||
spaceId: string;
|
||||
updatedAt: Generated<Timestamp>;
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
import { INTERNAL_LINK_REGEX } from './utils';
|
||||
import { isInternalPagePath } from '@docmost/prosemirror-markdown';
|
||||
|
||||
/**
|
||||
* Cross-package DRIFT GUARD for the internal-link subset invariant (#522).
|
||||
*
|
||||
* The client-side `isInternalPagePath`
|
||||
* (`packages/prosemirror-markdown/src/lib/internal-links.ts`) promotes a markdown
|
||||
* link to `internal: true` on import. Every link it marks internal MUST be one
|
||||
* the server would backlink and export-rewrite — i.e. the client matcher MUST be
|
||||
* a STRICT SUBSET of the server's canonical `INTERNAL_LINK_REGEX`
|
||||
* (`./utils.ts`). If the client ever accepts a path the server rejects, that link
|
||||
* is stored internal but silently dropped from the backlink graph and broken on
|
||||
* export — the exact bug #522 fixed.
|
||||
*
|
||||
* This spec is the load-bearing guard: it imports the LIVE server regex AND the
|
||||
* LIVE `isInternalPagePath` (no hand-copied regex on either side). A narrowing of
|
||||
* EITHER — most dangerously the server regex — reddens here. The in-package
|
||||
* accept/reject test documents the client's behaviour but cannot see the server
|
||||
* regex; this top-layer spec is what makes the subset relation mechanical
|
||||
* (AGENTS.md rule #7: a CI test that fails on drift of the source of truth).
|
||||
*/
|
||||
describe('internal-link subset parity (client isInternalPagePath ⊆ server INTERNAL_LINK_REGEX)', () => {
|
||||
// Every path the CLIENT accepts. Kept deliberately broad across the risky
|
||||
// dimensions — the slug charset (digits, hyphens, mixed case, leading/trailing
|
||||
// hyphen), the space charset, and the optional trailing slash — so a narrowing
|
||||
// of the server regex on any of them reddens the subset assertion below.
|
||||
const CLIENT_ACCEPTS = [
|
||||
'/s/eng/p/abc123',
|
||||
'/s/eng/p/abc123/', // trailing slash
|
||||
'/s/My-Space/p/A-b-C-9', // hyphens + mixed case in both segments
|
||||
'/s/x/p/z', // shortest
|
||||
'/s/eng/p/my-page-abc123', // slug with hyphens (extractPageSlugId shape)
|
||||
'/s/eng/p/-lead', // leading hyphen in slug
|
||||
'/s/eng/p/trail-', // trailing hyphen in slug
|
||||
'/s/eng/p/0123456789', // all-digit slug
|
||||
'/s/a.b/p/abc', // dot in the SPACE segment (space charset is [^/]+)
|
||||
'/s/space with space/p/abc', // space char in the SPACE segment
|
||||
];
|
||||
|
||||
it('every corpus path is actually client-accepted (guards the corpus itself)', () => {
|
||||
// If a path here stopped being client-accepted the subset test would pass
|
||||
// vacuously; assert acceptance up front so the corpus stays meaningful. The
|
||||
// filter-to-empty form names the offending paths on failure.
|
||||
const notAccepted = CLIENT_ACCEPTS.filter((h) => !isInternalPagePath(h));
|
||||
expect(notAccepted).toEqual([]);
|
||||
});
|
||||
|
||||
it('every client-accepted path also matches the LIVE server regex (subset)', () => {
|
||||
// The mechanical drift guard: narrow the server INTERNAL_LINK_REGEX and at
|
||||
// least one hyphen/charset/structure case appears here.
|
||||
const notInServer = CLIENT_ACCEPTS.filter((h) => !INTERNAL_LINK_REGEX.test(h));
|
||||
expect(notInServer).toEqual([]);
|
||||
});
|
||||
|
||||
it('the client rejects forbidden-slug-char paths the server also rejects', () => {
|
||||
// Documents the subset BOUNDARY: correct shape, forbidden slug char. The
|
||||
// client and the LIVE server regex must agree on rejection.
|
||||
const forbidden = [
|
||||
'/s/eng/p/abc.def',
|
||||
'/s/eng/p/abc_def',
|
||||
'/s/eng/p/abc%20',
|
||||
'/s/eng/p/abc~x',
|
||||
];
|
||||
const clientAccepts = forbidden.filter((h) => isInternalPagePath(h));
|
||||
const serverAccepts = forbidden.filter((h) => INTERNAL_LINK_REGEX.test(h));
|
||||
expect(clientAccepts).toEqual([]);
|
||||
expect(serverAccepts).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -85,6 +85,14 @@ export class ImportController {
|
||||
throw new BadRequestException('spaceId is required');
|
||||
}
|
||||
|
||||
// #502: optional multipart field. Only the MCP agent `createPage` path sends
|
||||
// `disableMarkdownExtensions=true` (its body is agent-authored plain prose /
|
||||
// config, so a `$…$` span must stay literal and a bare `www.host` must not
|
||||
// autolink). A HUMAN file upload omits the field, so it stays false and math
|
||||
// + autolink remain ON for human imports. Settable ONLY via this API param.
|
||||
const disableMarkdownExtensions =
|
||||
file.fields?.disableMarkdownExtensions?.value === 'true';
|
||||
|
||||
const ability = await this.spaceAbility.createForUser(user, spaceId);
|
||||
if (ability.cannot(SpaceCaslAction.Edit, SpaceCaslSubject.Page)) {
|
||||
throw new ForbiddenException();
|
||||
@@ -95,6 +103,7 @@ export class ImportController {
|
||||
user.id,
|
||||
spaceId,
|
||||
workspace.id,
|
||||
disableMarkdownExtensions,
|
||||
);
|
||||
|
||||
const ext = path.extname(file.filename).toLowerCase();
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
// Importing ImportService transitively loads import-formatter.ts, which imports
|
||||
// the ESM-only @sindresorhus/slugify (not in jest's transform allowlist). It is
|
||||
// irrelevant to this path, so mock it to keep the module graph loadable (mirrors
|
||||
// the sibling import.service specs).
|
||||
jest.mock('@sindresorhus/slugify', () => ({
|
||||
__esModule: true,
|
||||
default: (input: string) => String(input),
|
||||
}));
|
||||
|
||||
import { ImportService } from './import.service';
|
||||
|
||||
// #502 BLOCKER 1: the server markdown import path (`/pages/import`) now accepts an
|
||||
// optional `disableMarkdownExtensions` flag threaded into `processMarkdown`.
|
||||
// - MCP agent `createPage` sends it TRUE -> extensions OFF (a `$…$` config span
|
||||
// stays literal, a schemeless `www.host` is not autolinked).
|
||||
// - a HUMAN file upload omits it (default FALSE) -> extensions ON, so a real
|
||||
// `$x^2$` still becomes a formula (human imports unaffected).
|
||||
// `processMarkdown` only uses the imported converter (no injected deps on this
|
||||
// path), so the service is constructed with null deps for this focused unit test.
|
||||
|
||||
function makeService(): ImportService {
|
||||
return new ImportService(null as any, null as any, null as any, null as any);
|
||||
}
|
||||
|
||||
function findAll(node: any, type: string, acc: any[] = []): any[] {
|
||||
if (!node || typeof node !== 'object') return acc;
|
||||
if (node.type === type) acc.push(node);
|
||||
if (Array.isArray(node.content)) for (const c of node.content) findAll(c, type, acc);
|
||||
return acc;
|
||||
}
|
||||
function hasLink(node: any): boolean {
|
||||
return findAll(node, 'text').some((t: any) =>
|
||||
t.marks?.some((m: any) => m.type === 'link'),
|
||||
);
|
||||
}
|
||||
|
||||
describe('ImportService.processMarkdown — #502 disableMarkdownExtensions', () => {
|
||||
it('disableMarkdownExtensions=true (MCP createPage): `$…$` stays literal, no math', async () => {
|
||||
const svc = makeService();
|
||||
const doc = await svc.processMarkdown('export A=$FOO and B=$BAR done', true);
|
||||
expect(findAll(doc, 'mathInline')).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('disableMarkdownExtensions=true: a schemeless www is NOT autolinked', async () => {
|
||||
const svc = makeService();
|
||||
const doc = await svc.processMarkdown('see www.example.com here', true);
|
||||
expect(hasLink(doc)).toBe(false);
|
||||
});
|
||||
|
||||
it('disableMarkdownExtensions=true: an explicit https:// STILL links', async () => {
|
||||
const svc = makeService();
|
||||
const doc = await svc.processMarkdown('see https://example.com here', true);
|
||||
expect(hasLink(doc)).toBe(true);
|
||||
});
|
||||
|
||||
it('DEFAULT (human upload): a real `$x^2$` DOES become a math node', async () => {
|
||||
const svc = makeService();
|
||||
const doc = await svc.processMarkdown('$x^2$');
|
||||
expect(findAll(doc, 'mathInline')).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('DEFAULT (human upload): a schemeless www IS autolinked', async () => {
|
||||
const svc = makeService();
|
||||
const doc = await svc.processMarkdown('see www.example.com here');
|
||||
expect(hasLink(doc)).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -52,6 +52,12 @@ export class ImportService {
|
||||
userId: string,
|
||||
spaceId: string,
|
||||
workspaceId: string,
|
||||
// #502: when true, the markdown importer runs with the two layered
|
||||
// extensions OFF (a `$…$` span stays literal text; a schemeless `www.host` is
|
||||
// NOT autolinked). ONLY the MCP agent `createPage` path sets this; a HUMAN
|
||||
// file upload never passes it, so it defaults false and math/autolink stay ON
|
||||
// for human imports (their `$x^2$` still becomes a formula).
|
||||
disableMarkdownExtensions = false,
|
||||
) {
|
||||
const file = await filePromise;
|
||||
const fileBuffer = await file.toBuffer();
|
||||
@@ -66,7 +72,10 @@ export class ImportService {
|
||||
|
||||
try {
|
||||
if (fileExtension.endsWith('.md')) {
|
||||
prosemirrorState = await this.processMarkdown(fileContent);
|
||||
prosemirrorState = await this.processMarkdown(
|
||||
fileContent,
|
||||
disableMarkdownExtensions,
|
||||
);
|
||||
} else if (fileExtension.endsWith('.html')) {
|
||||
prosemirrorState = await this.processHTML(fileContent);
|
||||
}
|
||||
@@ -138,7 +147,12 @@ export class ImportService {
|
||||
return createdPage;
|
||||
}
|
||||
|
||||
async processMarkdown(markdownInput: string): Promise<any> {
|
||||
async processMarkdown(
|
||||
markdownInput: string,
|
||||
// #502: forwarded to the importer. DEFAULT false keeps math + fuzzy autolink
|
||||
// ON (human uploads unaffected); the MCP agent `createPage` path passes true.
|
||||
disableMarkdownExtensions = false,
|
||||
): Promise<any> {
|
||||
// Canonical markdown -> ProseMirror JSON directly via
|
||||
// `@docmost/prosemirror-markdown` (issue #345) — no HTML intermediate and no
|
||||
// second editor-ext markdown layer. Foreign markdown surfaces the strict
|
||||
@@ -147,7 +161,12 @@ export class ImportService {
|
||||
// The HTML-cleanup pass (`normalizeImportHtml`) is intentionally skipped here:
|
||||
// it targets foreign *HTML* (Notion/XWiki), which only ever arrives on the
|
||||
// `.html` path (`processHTML`), never as canonical markdown.
|
||||
return markdownToProseMirror(normalizeForeignMarkdown(markdownInput));
|
||||
return markdownToProseMirror(
|
||||
normalizeForeignMarkdown(markdownInput),
|
||||
disableMarkdownExtensions
|
||||
? { parseMath: false, fuzzyLinkify: false }
|
||||
: undefined,
|
||||
);
|
||||
}
|
||||
|
||||
async processHTML(htmlInput: string): Promise<any> {
|
||||
|
||||
@@ -4,6 +4,7 @@ import { join } from 'path';
|
||||
import * as fs from 'node:fs';
|
||||
import fastifyStatic from '@fastify/static';
|
||||
import { EnvironmentService } from '../environment/environment.service';
|
||||
import { resolveClientDistPath } from '../../common/helpers/client-version';
|
||||
|
||||
/**
|
||||
* Resolve the response headers for a statically served client asset.
|
||||
@@ -56,14 +57,7 @@ export class StaticModule implements OnModuleInit {
|
||||
const httpAdapter = this.httpAdapterHost.httpAdapter;
|
||||
const app = httpAdapter.getInstance();
|
||||
|
||||
const clientDistPath = join(
|
||||
__dirname,
|
||||
'..',
|
||||
'..',
|
||||
'..',
|
||||
'..',
|
||||
'client/dist',
|
||||
);
|
||||
const clientDistPath = resolveClientDistPath();
|
||||
|
||||
const indexFilePath = join(clientDistPath, 'index.html');
|
||||
|
||||
|
||||
@@ -9,10 +9,14 @@ import {
|
||||
import { Server, Socket } from 'socket.io';
|
||||
import { TokenService } from '../core/auth/services/token.service';
|
||||
import { JwtPayload, JwtType } from '../core/auth/dto/jwt-payload';
|
||||
import { OnModuleDestroy } from '@nestjs/common';
|
||||
import { Logger, OnModuleDestroy, OnModuleInit } from '@nestjs/common';
|
||||
import { SpaceMemberRepo } from '@docmost/db/repos/space/space-member.repo';
|
||||
import { WsService } from './ws.service';
|
||||
import { getSpaceRoomName, getUserRoomName } from './ws.utils';
|
||||
import {
|
||||
readClientBuildVersion,
|
||||
resolveClientDistPath,
|
||||
} from '../common/helpers/client-version';
|
||||
import * as cookie from 'cookie';
|
||||
|
||||
@WebSocketGateway({
|
||||
@@ -20,17 +24,40 @@ import * as cookie from 'cookie';
|
||||
transports: ['websocket'],
|
||||
})
|
||||
export class WsGateway
|
||||
implements OnGatewayConnection, OnGatewayInit, OnModuleDestroy
|
||||
implements
|
||||
OnGatewayConnection,
|
||||
OnGatewayInit,
|
||||
OnModuleInit,
|
||||
OnModuleDestroy
|
||||
{
|
||||
@WebSocketServer()
|
||||
server: Server;
|
||||
|
||||
private readonly logger = new Logger(WsGateway.name);
|
||||
|
||||
// The build version of the client bundle shipped in this image, read once at
|
||||
// startup from client/dist/version.json (single source of truth, same value
|
||||
// baked into the client's APP_VERSION). Empty string => version.json missing
|
||||
// or empty => the proactive version-coherence reload feature stays inert.
|
||||
private appVersion = '';
|
||||
|
||||
constructor(
|
||||
private tokenService: TokenService,
|
||||
private spaceMemberRepo: SpaceMemberRepo,
|
||||
private wsService: WsService,
|
||||
) {}
|
||||
|
||||
onModuleInit(): void {
|
||||
this.appVersion = readClientBuildVersion(resolveClientDistPath());
|
||||
if (this.appVersion) {
|
||||
this.logger.log(`app-version reload: ACTIVE (v=${this.appVersion})`);
|
||||
} else {
|
||||
this.logger.log(
|
||||
'app-version reload: DISABLED (version.json missing/empty)',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
afterInit(server: Server): void {
|
||||
this.wsService.setServer(server);
|
||||
}
|
||||
@@ -55,6 +82,14 @@ export class WsGateway
|
||||
const spaceRooms = userSpaceIds.map((id) => getSpaceRoomName(id));
|
||||
|
||||
client.join([userRoom, workspaceRoom, ...spaceRooms]);
|
||||
|
||||
// Announce this container's client build version to the freshly
|
||||
// authenticated socket. On a redeploy the client reconnects to the new
|
||||
// container and receives the new version here, letting it guard-reload
|
||||
// before it hits a stale lazy chunk. Per-connect only (no broadcast):
|
||||
// natural reconnect covers both single-container and cluster without a
|
||||
// thundering-herd fleet reload.
|
||||
client.emit('app-version', { version: this.appVersion });
|
||||
} catch (err) {
|
||||
client.emit('Unauthorized');
|
||||
client.disconnect();
|
||||
|
||||
@@ -0,0 +1,137 @@
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { Kysely } from 'kysely';
|
||||
import { PageHistoryRepo } from '../../src/database/repos/page/page-history.repo';
|
||||
import { PageHistoryService } from '../../src/core/page/services/page-history.service';
|
||||
import { computeWorkTime } from '../../src/core/page/work-time';
|
||||
import {
|
||||
getTestDb,
|
||||
destroyTestDb,
|
||||
createWorkspace,
|
||||
createSpace,
|
||||
createPage,
|
||||
createUser,
|
||||
createChat,
|
||||
} from './db';
|
||||
|
||||
/**
|
||||
* #395 — real-Postgres coverage for the work-time timeline projection and the
|
||||
* service that computes the estimate. The pure sessionizer is unit-tested
|
||||
* exhaustively (compute-work-time.spec.ts); this asserts the SQL projection
|
||||
* (right rows, ASC, no `content`) and that the service's numbers agree with the
|
||||
* pure core over the exact rows the DB returns.
|
||||
*/
|
||||
describe('PageHistory work-time [integration]', () => {
|
||||
let db: Kysely<any>;
|
||||
let repo: PageHistoryRepo;
|
||||
let service: PageHistoryService;
|
||||
let workspaceId: string;
|
||||
let spaceId: string;
|
||||
let pageId: string;
|
||||
let userId: string;
|
||||
let chatId: string;
|
||||
|
||||
const MIN = 60 * 1000;
|
||||
|
||||
beforeAll(async () => {
|
||||
db = getTestDb();
|
||||
repo = new PageHistoryRepo(db as any);
|
||||
service = new PageHistoryService(repo);
|
||||
workspaceId = (await createWorkspace(db)).id;
|
||||
spaceId = (await createSpace(db, workspaceId)).id;
|
||||
pageId = (await createPage(db, { workspaceId, spaceId })).id;
|
||||
userId = (await createUser(db, workspaceId)).id;
|
||||
chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await destroyTestDb();
|
||||
});
|
||||
|
||||
async function insertHistory(rows: Array<{
|
||||
createdAt: string;
|
||||
source: string | null;
|
||||
chat?: string | null;
|
||||
kind?: string | null;
|
||||
content?: unknown;
|
||||
}>) {
|
||||
for (const r of rows) {
|
||||
await db
|
||||
.insertInto('pageHistory')
|
||||
.values({
|
||||
id: randomUUID(),
|
||||
pageId,
|
||||
spaceId,
|
||||
workspaceId,
|
||||
title: 'x',
|
||||
content: r.content ?? { type: 'doc', content: [] },
|
||||
lastUpdatedById: userId,
|
||||
lastUpdatedSource: r.source,
|
||||
lastUpdatedAiChatId: r.chat ?? null,
|
||||
kind: r.kind ?? null,
|
||||
createdAt: new Date(r.createdAt),
|
||||
})
|
||||
.execute();
|
||||
}
|
||||
}
|
||||
|
||||
it('findTimelineByPageId projects the cheap columns, ASC, without content', async () => {
|
||||
await insertHistory([
|
||||
{ createdAt: '2026-07-04T19:54:00Z', source: 'user', kind: 'manual' },
|
||||
{ createdAt: '2026-07-04T03:40:00Z', source: 'user', kind: null },
|
||||
{ createdAt: '2026-07-04T15:43:00Z', source: 'agent', chat: chatId, kind: 'agent' },
|
||||
]);
|
||||
|
||||
const timeline = await repo.findTimelineByPageId(pageId);
|
||||
expect(timeline).toHaveLength(3);
|
||||
// Sorted oldest → newest.
|
||||
const times = timeline.map((r) => new Date(r.createdAt).getTime());
|
||||
expect(times).toEqual([...times].sort((a, b) => a - b));
|
||||
// Projection carries exactly the sessionizer's inputs, and NO content.
|
||||
for (const row of timeline) {
|
||||
expect(row).toHaveProperty('createdAt');
|
||||
expect(row).toHaveProperty('lastUpdatedById');
|
||||
expect(row).toHaveProperty('lastUpdatedSource');
|
||||
expect(row).toHaveProperty('lastUpdatedAiChatId');
|
||||
expect(row).toHaveProperty('kind');
|
||||
expect(row).not.toHaveProperty('content');
|
||||
}
|
||||
// Agent row keeps its provenance.
|
||||
const agent = timeline.find((r) => r.lastUpdatedSource === 'agent');
|
||||
expect(agent?.lastUpdatedAiChatId).toBe(chatId);
|
||||
});
|
||||
|
||||
it('service estimate matches the pure core and satisfies Σ perDay == workMs', async () => {
|
||||
const rows = await repo.findTimelineByPageId(pageId);
|
||||
const pure = computeWorkTime(rows);
|
||||
|
||||
const result = await service.computeWorkTime(pageId, 'UTC');
|
||||
expect(result.workMs).toBe(pure.workMs);
|
||||
expect(result.agentOnlyMs).toBe(pure.agentOnlyMs);
|
||||
expect(result.tz).toBe('UTC');
|
||||
expect(result.config.tGap).toBe(15 * MIN);
|
||||
|
||||
const sumActive = result.perDay.reduce((a, d) => a + d.activeMs, 0);
|
||||
expect(sumActive).toBe(result.workMs);
|
||||
|
||||
// The 3 seeded rows sessionize into ≤ their span; not the naive span.
|
||||
const naive =
|
||||
new Date('2026-07-04T19:54:00Z').getTime() -
|
||||
new Date('2026-07-04T03:40:00Z').getTime();
|
||||
expect(result.workMs).toBeGreaterThan(0);
|
||||
expect(result.workMs).toBeLessThan(naive);
|
||||
});
|
||||
|
||||
it('an unknown timezone surfaces as a RangeError (controller maps to 400)', async () => {
|
||||
await expect(
|
||||
service.computeWorkTime(pageId, 'Not/AZone'),
|
||||
).rejects.toBeInstanceOf(RangeError);
|
||||
});
|
||||
|
||||
it('a page with no history → zeros, no days', async () => {
|
||||
const emptyPage = (await createPage(db, { workspaceId, spaceId })).id;
|
||||
const result = await service.computeWorkTime(emptyPage, 'UTC');
|
||||
expect(result.workMs).toBe(0);
|
||||
expect(result.agentOnlyMs).toBe(0);
|
||||
expect(result.perDay).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,612 @@
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { Kysely, sql } from 'kysely';
|
||||
import { SearchService } from 'src/core/search/search.service';
|
||||
import { PageRepo } from '@docmost/db/repos/page/page.repo';
|
||||
import {
|
||||
getTestDb,
|
||||
destroyTestDb,
|
||||
createWorkspace,
|
||||
createSpace,
|
||||
} from './db';
|
||||
|
||||
/**
|
||||
* #529 Phase A — the lexical overhaul, on the REAL migrated schema (ru_en config).
|
||||
*
|
||||
* Covers every acceptance criterion of the issue: RU+EN morphology + OR default,
|
||||
* match=auto identifier routing, "phrase"/+/- operators, RRF ordering, exact
|
||||
* permission-filtered total (fail-closed) + pagination, only-negation / garbage
|
||||
* short-circuits, the A8 path fix, the response superset, and the RAG lockstep
|
||||
* config (acceptance #13).
|
||||
*
|
||||
* The tsv column is populated by the pages_tsvector_trigger (now ru_en), so the
|
||||
* FTS branch is exercised end to end.
|
||||
*/
|
||||
describe('SearchService #529 lexical overhaul [integration]', () => {
|
||||
let db: Kysely<any>;
|
||||
let workspaceId: string;
|
||||
let spaceId: string;
|
||||
|
||||
async function insertPage(args: {
|
||||
title: string;
|
||||
textContent?: string;
|
||||
parentPageId?: string | null;
|
||||
spaceId?: string;
|
||||
deletedAt?: Date | null;
|
||||
}): Promise<string> {
|
||||
const id = randomUUID();
|
||||
await db
|
||||
.insertInto('pages')
|
||||
.values({
|
||||
id,
|
||||
slugId: `slug-${id.slice(0, 12)}`,
|
||||
title: args.title,
|
||||
textContent: args.textContent ?? null,
|
||||
parentPageId: args.parentPageId ?? null,
|
||||
spaceId: args.spaceId ?? spaceId,
|
||||
workspaceId,
|
||||
deletedAt: args.deletedAt ?? null,
|
||||
})
|
||||
.execute();
|
||||
return id;
|
||||
}
|
||||
|
||||
// Service wired to the real DB + real PageRepo (recursive descendants) with
|
||||
// stubbed space-membership + permission repos so a test controls scope and the
|
||||
// permission filter explicitly. `accessibleIds` (when set) is the KEEP list.
|
||||
function buildService(opts?: {
|
||||
userSpaceIds?: string[];
|
||||
accessibleIds?: string[] | null;
|
||||
filterThrows?: boolean;
|
||||
}): SearchService {
|
||||
const pageRepo = new PageRepo(db as any, null as any, null as any);
|
||||
const spaceMemberRepo = {
|
||||
getUserSpaceIds: async () => opts?.userSpaceIds ?? [spaceId],
|
||||
};
|
||||
const pagePermissionRepo = {
|
||||
hasRestrictedAncestor: async () => false,
|
||||
filterAccessiblePageIds: async ({ pageIds }: { pageIds: string[] }) => {
|
||||
if (opts?.filterThrows) throw new Error('permission query failed');
|
||||
return opts?.accessibleIds
|
||||
? pageIds.filter((id) => opts.accessibleIds!.includes(id))
|
||||
: pageIds;
|
||||
},
|
||||
};
|
||||
return new SearchService(
|
||||
db as any,
|
||||
pageRepo as any,
|
||||
{} as any,
|
||||
spaceMemberRepo as any,
|
||||
pagePermissionRepo as any,
|
||||
);
|
||||
}
|
||||
|
||||
const search = (service: SearchService, params: any) =>
|
||||
service.searchPage(params, { userId: 'u-1', workspaceId }) as any;
|
||||
|
||||
beforeAll(async () => {
|
||||
db = getTestDb();
|
||||
workspaceId = (await createWorkspace(db)).id;
|
||||
spaceId = (await createSpace(db, workspaceId)).id;
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await destroyTestDb();
|
||||
});
|
||||
|
||||
// 1. RU morphology + OR: «ресторанов москвы» finds «ресторан в москве».
|
||||
it('#1 russian morphology + OR: finds «ресторан в москве» for «ресторанов москвы»', async () => {
|
||||
const page = await insertPage({
|
||||
title: 'ресторан в москве',
|
||||
textContent: 'Лучший ресторан столицы.',
|
||||
});
|
||||
const res = await search(buildService(), {
|
||||
query: 'ресторанов москвы',
|
||||
spaceId,
|
||||
});
|
||||
expect(res.items.map((i: any) => i.id)).toContain(page);
|
||||
expect(res.total).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
|
||||
// 2. OR non-empty: «Стамбул Роснефть».
|
||||
it('#2 OR yields a hit when only one term matches', async () => {
|
||||
const page = await insertPage({ title: 'Роснефть отчёт', textContent: 'x' });
|
||||
const res = await search(buildService(), {
|
||||
query: 'Стамбул Роснефть',
|
||||
spaceId,
|
||||
});
|
||||
expect(res.items.map((i: any) => i.id)).toContain(page);
|
||||
});
|
||||
|
||||
// 3. Multi-word OR: a page matching >=1 term is returned.
|
||||
it('#3 «3D принтер» returns pages that matched at least one term', async () => {
|
||||
const models = await insertPage({
|
||||
title: 'Модели для печати 3D',
|
||||
textContent: 'коллекция моделей',
|
||||
});
|
||||
const wish = await insertPage({
|
||||
title: 'Хотеть напечатать на принтере',
|
||||
textContent: 'очередь печати',
|
||||
});
|
||||
const res = await search(buildService(), { query: '3D принтер', spaceId });
|
||||
const ids = res.items.map((i: any) => i.id);
|
||||
expect(ids).toContain(models); // matched "3D"
|
||||
expect(ids).toContain(wish); // matched "принтер"
|
||||
// matchedTerms is populated per hit.
|
||||
const hit = res.items.find((i: any) => i.id === wish);
|
||||
expect(hit.matchedTerms).toContain('принтер');
|
||||
});
|
||||
|
||||
// 4. match=auto FTS stemming: «печат» must NOT drag in «впечатления».
|
||||
it('#4 «печат» (auto) matches «печать» but NOT «впечатления»', async () => {
|
||||
const good = await insertPage({
|
||||
title: 'Печать документов',
|
||||
textContent: 'настройка печати',
|
||||
});
|
||||
const bad = await insertPage({
|
||||
title: 'Впечатления от поездки',
|
||||
textContent: 'много впечатлений',
|
||||
});
|
||||
const res = await search(buildService(), { query: 'печат', spaceId });
|
||||
const ids = res.items.map((i: any) => i.id);
|
||||
expect(ids).toContain(good);
|
||||
expect(ids).not.toContain(bad);
|
||||
});
|
||||
|
||||
// 5. Identifier → substring branch.
|
||||
it('#5 `10.31.41` (auto→substring) finds the page with that IP', async () => {
|
||||
const page = await insertPage({
|
||||
title: 'Сетевой узел',
|
||||
textContent: 'Адрес устройства: 10.31.41.7 в сети.',
|
||||
});
|
||||
const res = await search(buildService(), { query: '10.31.41', spaceId });
|
||||
const hit = res.items.find((i: any) => i.id === page);
|
||||
expect(hit).toBeDefined();
|
||||
expect(hit.matchedFields).toContain('text');
|
||||
});
|
||||
|
||||
// 6. +required / -excluded.
|
||||
it('#6 `+кофейня -архив`: keeps «кофейня», drops pages with «архив»', async () => {
|
||||
const keep = await insertPage({ title: 'Кофейня в центре', textContent: 'уют' });
|
||||
const drop = await insertPage({
|
||||
title: 'Кофейня старый архив',
|
||||
textContent: 'архивные записи',
|
||||
});
|
||||
const res = await search(buildService(), { query: '+кофейня -архив', spaceId });
|
||||
const ids = res.items.map((i: any) => i.id);
|
||||
expect(ids).toContain(keep);
|
||||
expect(ids).not.toContain(drop);
|
||||
});
|
||||
|
||||
// 7. Phrase operator: only adjacent phrase hits survive.
|
||||
it('#7 `+"воздушный шар" кофе`: every hit contains the adjacent phrase', async () => {
|
||||
const adjacent = await insertPage({
|
||||
title: 'Воздушный шар и кофе',
|
||||
textContent: 'воздушный шар над городом, чашка кофе',
|
||||
});
|
||||
const nonAdjacent = await insertPage({
|
||||
title: 'Красный воздушный большой шар',
|
||||
textContent: 'воздушный красный шар и кофе рядом',
|
||||
});
|
||||
const res = await search(buildService(), {
|
||||
query: '+"воздушный шар" кофе',
|
||||
spaceId,
|
||||
});
|
||||
const ids = res.items.map((i: any) => i.id);
|
||||
expect(ids).toContain(adjacent);
|
||||
// The non-adjacent page (words separated) must NOT match the phrase.
|
||||
expect(ids).not.toContain(nonAdjacent);
|
||||
});
|
||||
|
||||
// 8. Pagination determinism + exact total.
|
||||
it('#8 >50 matches: total>50, hasMore, and offset paginates without dupes', async () => {
|
||||
const svc = buildService();
|
||||
const created: string[] = [];
|
||||
for (let i = 0; i < 60; i++) {
|
||||
created.push(
|
||||
await insertPage({
|
||||
title: `паджинация запись ${i}`,
|
||||
textContent: 'общий паджинационный маркер',
|
||||
}),
|
||||
);
|
||||
}
|
||||
const p1 = await search(svc, {
|
||||
query: 'паджинационный',
|
||||
spaceId,
|
||||
limit: 25,
|
||||
offset: 0,
|
||||
});
|
||||
expect(p1.total).toBeGreaterThanOrEqual(60);
|
||||
expect(p1.hasMore).toBe(true);
|
||||
const p2 = await search(svc, {
|
||||
query: 'паджинационный',
|
||||
spaceId,
|
||||
limit: 25,
|
||||
offset: 25,
|
||||
});
|
||||
const p3 = await search(svc, {
|
||||
query: 'паджинационный',
|
||||
spaceId,
|
||||
limit: 25,
|
||||
offset: 50,
|
||||
});
|
||||
const ids = [
|
||||
...p1.items.map((i: any) => i.id),
|
||||
...p2.items.map((i: any) => i.id),
|
||||
...p3.items.map((i: any) => i.id),
|
||||
];
|
||||
// No duplicates across the three pages.
|
||||
expect(new Set(ids).size).toBe(ids.length);
|
||||
// Deterministic: same query twice → identical order.
|
||||
const p1b = await search(svc, {
|
||||
query: 'паджинационный',
|
||||
spaceId,
|
||||
limit: 25,
|
||||
offset: 0,
|
||||
});
|
||||
expect(p1b.items.map((i: any) => i.id)).toEqual(p1.items.map((i: any) => i.id));
|
||||
});
|
||||
|
||||
// 9. Only-negation.
|
||||
it('#9 `-архив` only-negation: total 0, reason only-negation, no throw', async () => {
|
||||
const res = await search(buildService(), { query: '-архив', spaceId });
|
||||
expect(res.total).toBe(0);
|
||||
expect(res.items).toEqual([]);
|
||||
expect(res.query.parsed.reason).toBe('only-negation');
|
||||
});
|
||||
|
||||
// 10. Garbage input.
|
||||
it('#10 garbage `%` / `_` / empty: total 0, does not match everything', async () => {
|
||||
await insertPage({ title: 'какая-то страница', textContent: 'текст' });
|
||||
for (const q of ['%', '_', ' ', '%%__']) {
|
||||
const res = await search(buildService(), { query: q, spaceId });
|
||||
expect(res.total).toBe(0);
|
||||
expect(res.items).toEqual([]);
|
||||
}
|
||||
});
|
||||
|
||||
// 11. Permission-filtered total (fail-closed) + mutation guard.
|
||||
it('#11 a permission-hidden page is absent from items AND total', async () => {
|
||||
const visible = await insertPage({
|
||||
title: 'разрешённая пермишен-страница',
|
||||
textContent: 'пермишенмаркер',
|
||||
});
|
||||
const hidden = await insertPage({
|
||||
title: 'скрытая пермишен-страница',
|
||||
textContent: 'пермишенмаркер',
|
||||
});
|
||||
// Filter keeps only the visible page.
|
||||
const filtered = buildService({ accessibleIds: [visible] });
|
||||
const res = await search(filtered, { query: 'пермишенмаркер', spaceId });
|
||||
const ids = res.items.map((i: any) => i.id);
|
||||
expect(ids).toContain(visible);
|
||||
expect(ids).not.toContain(hidden);
|
||||
// total is the POST-permission count — the hidden page does not leak into it.
|
||||
expect(res.total).toBe(1);
|
||||
|
||||
// MUTATION: disable the guard (passthrough) → the hidden page reappears in
|
||||
// BOTH items and total. If this did NOT change, the guard is not load-bearing.
|
||||
const open = buildService({ accessibleIds: null });
|
||||
const res2 = await search(open, { query: 'пермишенмаркер', spaceId });
|
||||
expect(res2.items.map((i: any) => i.id)).toContain(hidden);
|
||||
expect(res2.total).toBe(2);
|
||||
});
|
||||
|
||||
it('#11b a permission-query error PROPAGATES (fail-closed, never empty)', async () => {
|
||||
await insertPage({ title: 'failclosed маркер', textContent: 'failclosedmarker' });
|
||||
const svc = buildService({ filterThrows: true });
|
||||
await expect(
|
||||
search(svc, { query: 'failclosedmarker', spaceId }),
|
||||
).rejects.toThrow(/permission query failed/);
|
||||
});
|
||||
|
||||
// A8 path fix.
|
||||
it('#11c path: a soft-deleted / cross-space ancestor title does not leak', async () => {
|
||||
const otherSpace = (await createSpace(db, workspaceId)).id;
|
||||
const root = await insertPage({ title: 'Живой корень' });
|
||||
const deletedMid = await insertPage({
|
||||
title: 'УдалённыйПредок',
|
||||
parentPageId: root,
|
||||
deletedAt: new Date(),
|
||||
});
|
||||
const leaf = await insertPage({
|
||||
title: 'a8leaf уникальный',
|
||||
parentPageId: deletedMid,
|
||||
textContent: 'a8leafmarker',
|
||||
});
|
||||
const res = await search(buildService(), { query: 'a8leafmarker', spaceId });
|
||||
const hit = res.items.find((i: any) => i.id === leaf);
|
||||
expect(hit).toBeDefined();
|
||||
// The walk stops at the deleted ancestor — no deleted title in the path.
|
||||
expect(hit.path).not.toContain('УдалённыйПредок');
|
||||
expect(hit.path).not.toContain('Живой корень');
|
||||
|
||||
// Cross-space parent must also not leak.
|
||||
const foreignParent = await insertPage({
|
||||
title: 'ЧужойСпейс',
|
||||
spaceId: otherSpace,
|
||||
});
|
||||
const crossLeaf = await insertPage({
|
||||
title: 'crossleaf узел',
|
||||
parentPageId: foreignParent,
|
||||
textContent: 'crossleafmarker',
|
||||
});
|
||||
const res2 = await search(buildService(), {
|
||||
query: 'crossleafmarker',
|
||||
spaceId,
|
||||
});
|
||||
const hit2 = res2.items.find((i: any) => i.id === crossLeaf);
|
||||
expect(hit2).toBeDefined();
|
||||
expect(hit2.path).not.toContain('ЧужойСпейс');
|
||||
});
|
||||
|
||||
// 12. Web-UI superset.
|
||||
it('#12 web path (no flags) returns the OR result with the icon/space/highlight superset', async () => {
|
||||
const page = await insertPage({
|
||||
title: 'веб суперсет страница',
|
||||
textContent: 'суперсетмаркер контент',
|
||||
});
|
||||
const res = await search(buildService(), { query: 'суперсетмаркер', spaceId });
|
||||
const hit = res.items.find((i: any) => i.id === page);
|
||||
expect(hit).toBeDefined();
|
||||
// Superset fields the web-UI relies on.
|
||||
expect('icon' in hit).toBe(true);
|
||||
expect('space' in hit).toBe(true);
|
||||
expect('highlight' in hit).toBe(true);
|
||||
expect('rank' in hit).toBe(true);
|
||||
// Plus the new fields.
|
||||
expect('path' in hit).toBe(true);
|
||||
expect('snippet' in hit).toBe(true);
|
||||
expect('score' in hit).toBe(true);
|
||||
// FTS hit carries a non-null rank + highlight.
|
||||
expect(hit.rank).not.toBeNull();
|
||||
});
|
||||
|
||||
// 13. RAG lockstep: the page_embeddings.fts generated column is ru_en.
|
||||
it('#13 page_embeddings.fts uses ru_en (cyrillic stemming) — RAG lockstep', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'rag страница',
|
||||
textContent: 'ресторанов много',
|
||||
});
|
||||
// Insert a chunk row; the generated fts column is computed by Postgres.
|
||||
await sql`
|
||||
INSERT INTO page_embeddings
|
||||
(id, page_id, workspace_id, space_id, attachment_id, chunk_index,
|
||||
chunk_start, chunk_length, content, model_name, model_dimensions, embedding)
|
||||
VALUES
|
||||
(${randomUUID()}, ${pageId}, ${workspaceId}, ${spaceId}, NULL, 0,
|
||||
0, 20, ${'ресторанов москвы много'}, 'test-model', 3, '[0.1,0.2,0.3]'::vector)
|
||||
`.execute(db);
|
||||
// A ru_en query stems «москва» → «москв», matching the stored «москвы».
|
||||
// Under the old `english` config the cyrillic word would not stem and this
|
||||
// inflected-form query would miss — so this asserts the ru_en lockstep.
|
||||
const row = await sql<{ m: boolean }>`
|
||||
SELECT fts @@ to_tsquery('ru_en', f_unaccent('москва')) AS m
|
||||
FROM page_embeddings WHERE page_id = ${pageId}
|
||||
`.execute(db);
|
||||
expect(row.rows[0].m).toBe(true);
|
||||
});
|
||||
|
||||
// W4 — substring-tier dominance under RRF: title-exact (tier 3) > title-
|
||||
// substring (tier 2) > text-only (tier 1). The engine encodes this via
|
||||
// sub_tier DESC → rn_sub → RRF; no test asserted the end-to-end ordering, so
|
||||
// this restores that guarantee. match:'substring' routes the term to the
|
||||
// substring branch (no FTS leg), so sub_tier alone drives the order.
|
||||
it('W4 tier dominance: title-exact > title-substring > text-only', async () => {
|
||||
const exact = await insertPage({ title: 'tierdomxyz' }); // tier 3
|
||||
const titleSub = await insertPage({
|
||||
title: 'prefix tierdomxyz suffix', // tier 2
|
||||
});
|
||||
const textOnly = await insertPage({
|
||||
title: 'w4 unrelated heading',
|
||||
textContent: 'body has tierdomxyz here', // tier 1
|
||||
});
|
||||
const res = await search(buildService(), {
|
||||
query: 'tierdomxyz',
|
||||
match: 'substring',
|
||||
spaceId,
|
||||
});
|
||||
const ids = res.items.map((i: any) => i.id);
|
||||
expect(ids).toContain(exact);
|
||||
expect(ids).toContain(titleSub);
|
||||
expect(ids).toContain(textOnly);
|
||||
// Strict tier order.
|
||||
expect(ids.indexOf(exact)).toBeLessThan(ids.indexOf(titleSub));
|
||||
expect(ids.indexOf(titleSub)).toBeLessThan(ids.indexOf(textOnly));
|
||||
});
|
||||
|
||||
// S3 — an exact-title hit must NOT be lost when the match set exceeds
|
||||
// CANDIDATE_CAP: it ranks first under RRF (tier 3) so it lands in the reachable
|
||||
// window even with a tiny cap. Restores a guarantee the old lookup suite gave.
|
||||
it('S3 exact-title survives the CANDIDATE_CAP window', async () => {
|
||||
const exact = await insertPage({ title: 'capmarkerxyz' }); // tier 3
|
||||
for (let i = 0; i < 6; i++) {
|
||||
await insertPage({
|
||||
title: `cap filler ${i}`,
|
||||
textContent: 'noise capmarkerxyz noise', // tier 1
|
||||
});
|
||||
}
|
||||
process.env.SEARCH_CANDIDATE_CAP = '2';
|
||||
try {
|
||||
const res = await search(buildService(), {
|
||||
query: 'capmarkerxyz',
|
||||
match: 'substring',
|
||||
spaceId,
|
||||
limit: 25,
|
||||
});
|
||||
// More matches than the cap → truncated, but the exact-title survives.
|
||||
expect(res.total).toBeGreaterThan(2);
|
||||
expect(res.truncatedAtCap).toBe(true);
|
||||
expect(res.items.length).toBe(2); // the reachable window
|
||||
expect(res.items.map((i: any) => i.id)).toContain(exact);
|
||||
} finally {
|
||||
delete process.env.SEARCH_CANDIDATE_CAP;
|
||||
}
|
||||
});
|
||||
|
||||
// S1 — a required-only query (`+term`, no bare positive) really matches, so its
|
||||
// hits must report matchedFields / rank / highlight, not [] / null. Before the
|
||||
// fix these detail exprs were built from parsed.positive only.
|
||||
it('S1 required-only query populates matchedFields + rank on a title match', async () => {
|
||||
const page = await insertPage({
|
||||
title: 'Кофейня s1маркер центр',
|
||||
textContent: 'обычный текст без ключевого слова',
|
||||
});
|
||||
const res = await search(buildService(), { query: '+кофейня', spaceId });
|
||||
const hit = res.items.find((i: any) => i.id === page);
|
||||
expect(hit).toBeDefined();
|
||||
// The title matches the required term → matchedFields includes 'title'.
|
||||
expect(hit.matchedFields).toContain('title');
|
||||
// FTS rank is populated (was null before the S1 fix).
|
||||
expect(hit.rank).not.toBeNull();
|
||||
// matchedTerms already echoed the required term; still true.
|
||||
expect(hit.matchedTerms).toContain('кофейня');
|
||||
});
|
||||
|
||||
// F1 — stack-depth guard: a pasted text block (thousands of FTS terms) used to
|
||||
// nest the combined tsquery so deep that Postgres raised `stack depth limit
|
||||
// exceeded` → HTTP 500 for the caller. The parser now caps terms at
|
||||
// MAX_PARSED_TERMS, so a huge query returns 200 and the leading (in-cap) term
|
||||
// still matches. Mutation: remove the cap → this reddens (500 / stack depth).
|
||||
it('F1 a huge multi-term query returns 200, not a stack-depth 500', async () => {
|
||||
const page = await insertPage({
|
||||
title: 'qfonemarker заголовок',
|
||||
textContent: 'тело страницы',
|
||||
});
|
||||
// Purely-alphabetic filler words → FTS branch (the branch that nests in
|
||||
// combineTsq). A digit-bearing token would route to substring and not nest.
|
||||
const alphaWord = (i: number): string => {
|
||||
let s = '';
|
||||
let n = i + 1;
|
||||
while (n > 0) {
|
||||
s = String.fromCharCode(97 + (n % 26)) + s;
|
||||
n = Math.floor(n / 26);
|
||||
}
|
||||
return 'q' + s;
|
||||
};
|
||||
const words = [
|
||||
'qfonemarker',
|
||||
...Array.from({ length: 5000 }, (_, i) => alphaWord(i)),
|
||||
];
|
||||
const res = await search(buildService(), {
|
||||
query: words.join(' '),
|
||||
spaceId,
|
||||
});
|
||||
// Bounded nesting → no 500; the leading in-cap term still recalls the page.
|
||||
expect(res.items.map((i: any) => i.id)).toContain(page);
|
||||
});
|
||||
|
||||
// F2 — titleOnly leak-guard (restores coverage the deleted lookup spec gave).
|
||||
// A term present ONLY in text_content must NOT match under titleOnly, and a
|
||||
// title hit must carry NO body snippet. Uses match:'substring' because titleOnly
|
||||
// gates the substring branch (the FTS `pages.tsv` already spans title+body).
|
||||
it('F2 titleOnly does not match text_content and yields no body snippet', async () => {
|
||||
// Marker lives only in the body, never in the title.
|
||||
const bodyOnly = await insertPage({
|
||||
title: 'нейтральный заголовок f2',
|
||||
textContent: 'секрет titleonlybody конец',
|
||||
});
|
||||
const res = await search(buildService(), {
|
||||
query: 'titleonlybody',
|
||||
match: 'substring',
|
||||
spaceId,
|
||||
titleOnly: true,
|
||||
});
|
||||
// titleOnly must NOT leak a body-substring match.
|
||||
expect(res.items.map((i: any) => i.id)).not.toContain(bodyOnly);
|
||||
|
||||
// Sanity: WITHOUT titleOnly the same term DOES find it via the body.
|
||||
const resOpen = await search(buildService(), {
|
||||
query: 'titleonlybody',
|
||||
match: 'substring',
|
||||
spaceId,
|
||||
});
|
||||
expect(resOpen.items.map((i: any) => i.id)).toContain(bodyOnly);
|
||||
|
||||
// A page that hits on its TITLE: the snippet must be empty (no body leaks in).
|
||||
const titleHit = await insertPage({
|
||||
title: 'titleonlytitle страница',
|
||||
textContent: 'какой-то текст тела здесь для сниппета',
|
||||
});
|
||||
const res2 = await search(buildService(), {
|
||||
query: 'titleonlytitle',
|
||||
match: 'substring',
|
||||
spaceId,
|
||||
titleOnly: true,
|
||||
});
|
||||
const hit = res2.items.find((i: any) => i.id === titleHit);
|
||||
expect(hit).toBeDefined();
|
||||
expect(hit.snippet).toBe('');
|
||||
});
|
||||
|
||||
// F3 — parentPageId subtree scoping (restores coverage the deleted lookup spec
|
||||
// gave). Two sibling subtrees share one term; scoping to ONE root returns only
|
||||
// that subtree's hits INCLUDING the root/parent page itself, and NONE of the
|
||||
// sibling subtree. Mutation: drop the ANY(descendantIds) filter → this reddens.
|
||||
it('F3 parentPageId scopes to one subtree incl. the parent, excludes siblings', async () => {
|
||||
const rootA = await insertPage({
|
||||
title: 'subtreeA корень',
|
||||
textContent: 'f3marker в корне A',
|
||||
});
|
||||
const childA = await insertPage({
|
||||
title: 'subtreeA потомок',
|
||||
textContent: 'f3marker в потомке A',
|
||||
parentPageId: rootA,
|
||||
});
|
||||
const rootB = await insertPage({
|
||||
title: 'subtreeB корень',
|
||||
textContent: 'f3marker в корне B',
|
||||
});
|
||||
const childB = await insertPage({
|
||||
title: 'subtreeB потомок',
|
||||
textContent: 'f3marker в потомке B',
|
||||
parentPageId: rootB,
|
||||
});
|
||||
const res = await search(buildService(), {
|
||||
query: 'f3marker',
|
||||
spaceId,
|
||||
parentPageId: rootA,
|
||||
});
|
||||
const ids = res.items.map((i: any) => i.id);
|
||||
expect(ids).toContain(rootA); // the parent/root page itself
|
||||
expect(ids).toContain(childA);
|
||||
expect(ids).not.toContain(rootB); // sibling subtree cut off
|
||||
expect(ids).not.toContain(childB);
|
||||
});
|
||||
|
||||
// F4 — positive path + snippet content asserts (the #11c test only asserts what
|
||||
// must NOT be in path). (a) a nested hit's path is root→parent ordered and a
|
||||
// root-level hit's path is []; (b) a text-body hit returns a NON-empty snippet.
|
||||
it('F4 path is root→parent ordered ([] at root) and body hits carry a snippet', async () => {
|
||||
const root = await insertPage({ title: 'F4Root' });
|
||||
const parent = await insertPage({ title: 'F4Parent', parentPageId: root });
|
||||
const leaf = await insertPage({
|
||||
title: 'f4leaf лист',
|
||||
parentPageId: parent,
|
||||
textContent: 'f4leafmarker тело с содержимым для сниппета',
|
||||
});
|
||||
const res = await search(buildService(), {
|
||||
query: 'f4leafmarker',
|
||||
spaceId,
|
||||
});
|
||||
const hit = res.items.find((i: any) => i.id === leaf);
|
||||
expect(hit).toBeDefined();
|
||||
// Positive ancestry: root → direct parent, hit's own title excluded.
|
||||
expect(hit.path).toEqual(['F4Root', 'F4Parent']);
|
||||
// Snippet content: a text-body hit returns a non-empty snippet with the term.
|
||||
expect(hit.snippet.length).toBeGreaterThan(0);
|
||||
expect(hit.snippet).toContain('f4leafmarker');
|
||||
|
||||
// A root-level hit (no parent) → empty path.
|
||||
const rootHit = await insertPage({
|
||||
title: 'f4rootlevel уникальный',
|
||||
textContent: 'f4rootmarker в теле',
|
||||
});
|
||||
const res2 = await search(buildService(), {
|
||||
query: 'f4rootmarker',
|
||||
spaceId,
|
||||
});
|
||||
const hit2 = res2.items.find((i: any) => i.id === rootHit);
|
||||
expect(hit2).toBeDefined();
|
||||
expect(hit2.path).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -1,462 +0,0 @@
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { Kysely } from 'kysely';
|
||||
import { SearchService } from 'src/core/search/search.service';
|
||||
import { PageRepo } from '@docmost/db/repos/page/page.repo';
|
||||
import {
|
||||
getTestDb,
|
||||
destroyTestDb,
|
||||
createWorkspace,
|
||||
createSpace,
|
||||
} from './db';
|
||||
|
||||
/**
|
||||
* #443 — agent-lookup search mode, acceptance on the REAL DB schema.
|
||||
*
|
||||
* Exercises SearchService.searchPage(..., { substring: true }) against a
|
||||
* migrated Postgres: substring matching of technical tokens the FTS tokenizer
|
||||
* mangles (backup-srv.local, 10.0.12.5, WB-MGE-30D86B, "Теги: Docker"), the
|
||||
* populated path + snippet, parentPageId subtree scoping, titleOnly, the empty
|
||||
* result, LIKE-metacharacter escaping (`%`/`_` must NOT match everything), the
|
||||
* permission post-filter applied BEFORE the limit, and the web-UI path staying
|
||||
* on the legacy FTS shape when `substring` is absent.
|
||||
*
|
||||
* The tsv column is populated by the pages_tsvector_trigger on insert, so the
|
||||
* FTS branch is exercised too.
|
||||
*/
|
||||
describe('SearchService agent-lookup mode [integration]', () => {
|
||||
let db: Kysely<any>;
|
||||
let service: SearchService;
|
||||
let workspaceId: string;
|
||||
let spaceId: string;
|
||||
|
||||
// Direct page insert (the shared createPage seeder omits text_content /
|
||||
// parent_page_id, both of which this mode depends on). Returns the id.
|
||||
async function insertPage(args: {
|
||||
title: string;
|
||||
textContent?: string;
|
||||
parentPageId?: string | null;
|
||||
spaceId?: string;
|
||||
}): Promise<string> {
|
||||
const id = randomUUID();
|
||||
await db
|
||||
.insertInto('pages')
|
||||
.values({
|
||||
id,
|
||||
slugId: `slug-${id.slice(0, 12)}`,
|
||||
title: args.title,
|
||||
textContent: args.textContent ?? null,
|
||||
parentPageId: args.parentPageId ?? null,
|
||||
spaceId: args.spaceId ?? spaceId,
|
||||
workspaceId,
|
||||
})
|
||||
.execute();
|
||||
return id;
|
||||
}
|
||||
|
||||
// Build a SearchService wired to the real DB + a real PageRepo (only its
|
||||
// recursive-descendants method is used by this mode, and it needs only `db`),
|
||||
// with lightweight stubs for the space-membership and permission repos so a
|
||||
// test can drive scope + the permission post-filter explicitly.
|
||||
function buildService(opts?: {
|
||||
userSpaceIds?: string[];
|
||||
// ids to KEEP after the permission post-filter; undefined = keep all.
|
||||
accessibleIds?: string[];
|
||||
}): SearchService {
|
||||
const pageRepo = new PageRepo(db as any, null as any, null as any);
|
||||
const spaceMemberRepo = {
|
||||
getUserSpaceIds: async () => opts?.userSpaceIds ?? [spaceId],
|
||||
};
|
||||
const pagePermissionRepo = {
|
||||
filterAccessiblePageIds: async ({ pageIds }: { pageIds: string[] }) =>
|
||||
opts?.accessibleIds
|
||||
? pageIds.filter((id) => opts.accessibleIds!.includes(id))
|
||||
: pageIds,
|
||||
};
|
||||
return new SearchService(
|
||||
db as any,
|
||||
pageRepo as any,
|
||||
{} as any, // shareRepo — unused by the lookup path
|
||||
spaceMemberRepo as any,
|
||||
pagePermissionRepo as any,
|
||||
);
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
db = getTestDb();
|
||||
workspaceId = (await createWorkspace(db)).id;
|
||||
spaceId = (await createSpace(db, workspaceId)).id;
|
||||
service = buildService();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await destroyTestDb();
|
||||
});
|
||||
|
||||
it('finds `backup-srv.local` by the fragment `srv.local`', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'backup-srv.local',
|
||||
textContent: 'A backup server node.',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'srv.local', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
expect(items.map((i: any) => i.id)).toContain(pageId);
|
||||
const hit = items.find((i: any) => i.id === pageId);
|
||||
expect(hit.title).toBe('backup-srv.local');
|
||||
// slugId must never be part of the server response shape.
|
||||
expect('slugId' in hit).toBe(true); // server carries it; MCP strips it
|
||||
});
|
||||
|
||||
it('finds a page whose TEXT contains `10.0.12.5` by the fragment `10.0.12` (empty-tsquery case)', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'Server inventory',
|
||||
textContent: 'The backup box lives at IP: 10.0.12.5. Debian 12, backups.',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: '10.0.12', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const hit = items.find((i: any) => i.id === pageId);
|
||||
expect(hit).toBeDefined();
|
||||
// The windowed snippet must include the matched text.
|
||||
expect(hit.snippet).toContain('10.0.12.5');
|
||||
});
|
||||
|
||||
it('finds `WB-MGE-30D86B` (alphanumeric token with dashes) by title', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'WB-MGE-30D86B',
|
||||
textContent: 'Device page.',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'WB-MGE-30D86B', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const hit = items.find((i: any) => i.id === pageId);
|
||||
expect(hit).toBeDefined();
|
||||
// Exact title match → top tier (TITLE_EXACT=3) → score in [0.75, 1].
|
||||
expect(hit.score).toBeGreaterThanOrEqual(0.75);
|
||||
// And it is the top-ranked hit of its own result set.
|
||||
expect(items[0].id).toBe(pageId);
|
||||
});
|
||||
|
||||
it('finds every page whose text literally contains `Теги: Docker`', async () => {
|
||||
const a = await insertPage({
|
||||
title: 'Container host A',
|
||||
textContent: 'Some notes.\nТеги: Docker, compose\nmore.',
|
||||
});
|
||||
const b = await insertPage({
|
||||
title: 'Container host B',
|
||||
textContent: 'Prelude.\nТеги: Docker\nepilogue.',
|
||||
});
|
||||
const noise = await insertPage({
|
||||
title: 'Unrelated',
|
||||
textContent: 'Теги: Kubernetes',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'Теги: Docker', spaceId, substring: true, limit: 50 } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
expect(ids).toContain(a);
|
||||
expect(ids).toContain(b);
|
||||
expect(ids).not.toContain(noise);
|
||||
});
|
||||
|
||||
it('populates a non-empty `path` for a nested hit and `[]` for a root hit', async () => {
|
||||
const root = await insertPage({ title: 'Infrastructure' });
|
||||
const mid = await insertPage({ title: 'Datacenter A', parentPageId: root });
|
||||
const leaf = await insertPage({
|
||||
title: 'unique-nested-host',
|
||||
parentPageId: mid,
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'unique-nested-host', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const hit = items.find((i: any) => i.id === leaf);
|
||||
expect(hit.path).toEqual(['Infrastructure', 'Datacenter A']);
|
||||
|
||||
const rootHits = (await service.searchPage(
|
||||
{ query: 'Infrastructure', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
const rootHit = rootHits.items.find((i: any) => i.id === root);
|
||||
expect(rootHit.path).toEqual([]);
|
||||
});
|
||||
|
||||
it('scopes to a subtree with parentPageId (cutting off sibling branches)', async () => {
|
||||
const branchA = await insertPage({ title: 'BranchA-root' });
|
||||
const inA = await insertPage({
|
||||
title: 'scoped-target-xyz',
|
||||
parentPageId: branchA,
|
||||
});
|
||||
const branchB = await insertPage({ title: 'BranchB-root' });
|
||||
const inB = await insertPage({
|
||||
title: 'scoped-target-xyz',
|
||||
parentPageId: branchB,
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{
|
||||
query: 'scoped-target-xyz',
|
||||
spaceId,
|
||||
substring: true,
|
||||
parentPageId: branchA,
|
||||
} as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
expect(ids).toContain(inA);
|
||||
expect(ids).not.toContain(inB);
|
||||
});
|
||||
|
||||
it('includes the parent page itself in the parentPageId subtree', async () => {
|
||||
const parent = await insertPage({ title: 'self-included-parent' });
|
||||
await insertPage({ title: 'child-of-self', parentPageId: parent });
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{
|
||||
query: 'self-included-parent',
|
||||
spaceId,
|
||||
substring: true,
|
||||
parentPageId: parent,
|
||||
} as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
expect(items.map((i: any) => i.id)).toContain(parent);
|
||||
});
|
||||
|
||||
it('titleOnly does NOT match on text_content', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'Plain title',
|
||||
textContent: 'body mentions the-secret-token here',
|
||||
});
|
||||
|
||||
const withText = (await service.searchPage(
|
||||
{ query: 'the-secret-token', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
expect(withText.items.map((i: any) => i.id)).toContain(pageId);
|
||||
|
||||
const titleOnly = (await service.searchPage(
|
||||
{
|
||||
query: 'the-secret-token',
|
||||
spaceId,
|
||||
substring: true,
|
||||
titleOnly: true,
|
||||
} as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
expect(titleOnly.items.map((i: any) => i.id)).not.toContain(pageId);
|
||||
});
|
||||
|
||||
// #443 Fix #1 regression: f_unaccent is NOT length-preserving, so an
|
||||
// expanding char (ß→ss, …→...) BEFORE the match shifted the strpos position
|
||||
// relative to the ORIGINAL text and the snippet slice ran past end → empty.
|
||||
// The position and the slice now share the LOWER(f_unaccent(...)) space, so
|
||||
// the window is aligned and always contains the matched (unaccented) token.
|
||||
it('returns a populated snippet when an unaccent-EXPANDING char precedes the match', async () => {
|
||||
// 300 × `ß` (each f_unaccent-expands to `ss`) before the needle. Under the
|
||||
// old code strpos returned a position ~593 in the expanded space but the
|
||||
// slice ran over the ORIGINAL (~360 char) text → empty snippet, match lost.
|
||||
const prefix = 'ß'.repeat(300);
|
||||
const pageId = await insertPage({
|
||||
title: 'Expanding-unaccent page',
|
||||
textContent: `${prefix} needle-token-xyz trailing.`,
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'needle-token-xyz', spaceId, substring: true } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const hit = items.find((i: any) => i.id === pageId);
|
||||
expect(hit).toBeDefined();
|
||||
// Snippet must be non-empty AND contain the matched token (unaccented form).
|
||||
expect(hit.snippet.length).toBeGreaterThan(0);
|
||||
expect(hit.snippet).toContain('needle-token-xyz');
|
||||
});
|
||||
|
||||
// #443 Fix #2 regression: >200 matching pages for a broad substring, with
|
||||
// exactly ONE exact-title hit. Without an ORDER BY on the 200-cap the exact
|
||||
// hit could be among the arbitrarily-dropped rows; the ORDER BY keeps the
|
||||
// strongest candidates so it must survive the cap and rank at the top.
|
||||
it('keeps an exact-title hit through the 200-cap on a >200-row match set', async () => {
|
||||
const isoSpace = (await createSpace(db, workspaceId)).id;
|
||||
const svc = buildService({ userSpaceIds: [isoSpace] });
|
||||
|
||||
// 250 low-tier TEXT hits: the shared substring `capword` appears only in the
|
||||
// body, never the title, so each is a TEXT-tier match (weakest tier).
|
||||
for (let i = 0; i < 250; i++) {
|
||||
await insertPage({
|
||||
title: `filler-page-${i}`,
|
||||
textContent: `body contains capword here #${i}`,
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
}
|
||||
// Exactly one EXACT-title hit for the same query token.
|
||||
const exact = await insertPage({
|
||||
title: 'capword',
|
||||
textContent: 'unrelated body text',
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
|
||||
const { items } = (await svc.searchPage(
|
||||
{ query: 'capword', spaceId: isoSpace, substring: true, limit: 10 } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
// The exact-title hit must survive the 200-cap and appear in the top `limit`.
|
||||
expect(ids).toContain(exact);
|
||||
// And, being TITLE_EXACT, it must be the single strongest hit.
|
||||
expect(items[0].id).toBe(exact);
|
||||
});
|
||||
|
||||
// #443 Fix #3: titleOnly matches only the title, so it must not leak the page
|
||||
// body as the snippet (the old "first 300 chars of text_content" fallback).
|
||||
it('titleOnly does NOT return a text-body snippet', async () => {
|
||||
const pageId = await insertPage({
|
||||
title: 'titleonly-snippet-page',
|
||||
textContent: 'SECRET-BODY-CONTENT-NOT-IN-TITLE that must not leak.',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{
|
||||
query: 'titleonly-snippet-page',
|
||||
spaceId,
|
||||
substring: true,
|
||||
titleOnly: true,
|
||||
} as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const hit = items.find((i: any) => i.id === pageId);
|
||||
expect(hit).toBeDefined();
|
||||
// The body text must not appear in the snippet; titleOnly → empty snippet.
|
||||
expect(hit.snippet).not.toContain('SECRET-BODY-CONTENT-NOT-IN-TITLE');
|
||||
expect(hit.snippet).toBe('');
|
||||
});
|
||||
|
||||
it('returns [] (not an error) for a query that matches nothing', async () => {
|
||||
const { items } = (await service.searchPage(
|
||||
{
|
||||
query: 'zzz-no-such-string-anywhere-42',
|
||||
spaceId,
|
||||
substring: true,
|
||||
} as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
expect(items).toEqual([]);
|
||||
});
|
||||
|
||||
it('a `%` query does NOT match everything (LIKE metacharacter escaped)', async () => {
|
||||
// Fresh space so we can assert on total counts without cross-test noise.
|
||||
const isoSpace = (await createSpace(db, workspaceId)).id;
|
||||
const svc = buildService({ userSpaceIds: [isoSpace] });
|
||||
await insertPage({ title: 'alpha', spaceId: isoSpace });
|
||||
await insertPage({ title: 'beta', spaceId: isoSpace });
|
||||
const literal = await insertPage({
|
||||
title: '100%-coverage',
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
|
||||
const { items } = (await svc.searchPage(
|
||||
{ query: '%', spaceId: isoSpace, substring: true, limit: 50 } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
// `%` is a literal → matches only the page that actually contains '%'.
|
||||
expect(ids).toContain(literal);
|
||||
expect(ids).not.toContain(
|
||||
items.find((i: any) => i.title === 'alpha')?.id,
|
||||
);
|
||||
expect(items.length).toBe(1);
|
||||
});
|
||||
|
||||
it('an `_` query does NOT match everything (LIKE metacharacter escaped)', async () => {
|
||||
const isoSpace = (await createSpace(db, workspaceId)).id;
|
||||
const svc = buildService({ userSpaceIds: [isoSpace] });
|
||||
await insertPage({ title: 'gamma', spaceId: isoSpace });
|
||||
const literal = await insertPage({
|
||||
title: 'snake_case_name',
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
|
||||
const { items } = (await svc.searchPage(
|
||||
{ query: '_', spaceId: isoSpace, substring: true, limit: 50 } as any,
|
||||
{ workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
expect(ids).toContain(literal);
|
||||
expect(items.length).toBe(1);
|
||||
});
|
||||
|
||||
it('applies the permission post-filter to the MERGED set BEFORE the limit', async () => {
|
||||
const isoSpace = (await createSpace(db, workspaceId)).id;
|
||||
const keep = await insertPage({
|
||||
title: 'perm-visible-target',
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
const hidden = await insertPage({
|
||||
title: 'perm-hidden-target',
|
||||
spaceId: isoSpace,
|
||||
});
|
||||
|
||||
// Authenticated (userId set) so the permission filter runs; only `keep` is
|
||||
// accessible. limit 1 must NOT be able to select `hidden`.
|
||||
const svc = buildService({
|
||||
userSpaceIds: [isoSpace],
|
||||
accessibleIds: [keep],
|
||||
});
|
||||
const { items } = (await svc.searchPage(
|
||||
{
|
||||
query: 'perm-',
|
||||
spaceId: isoSpace,
|
||||
substring: true,
|
||||
limit: 1,
|
||||
} as any,
|
||||
{ userId: 'user-1', workspaceId },
|
||||
)) as any;
|
||||
|
||||
const ids = items.map((i: any) => i.id);
|
||||
expect(ids).toContain(keep);
|
||||
expect(ids).not.toContain(hidden);
|
||||
});
|
||||
|
||||
it('web-UI path (no `substring` flag) keeps the legacy FTS response shape', async () => {
|
||||
await insertPage({
|
||||
title: 'legacy shape page',
|
||||
textContent: 'searchable legacyword content',
|
||||
});
|
||||
|
||||
const { items } = (await service.searchPage(
|
||||
{ query: 'legacyword', spaceId } as any,
|
||||
{ userId: 'user-1', workspaceId },
|
||||
)) as any;
|
||||
|
||||
// Legacy hits carry rank + highlight + space, and NO path/snippet/score.
|
||||
const hit = items[0];
|
||||
expect(hit).toBeDefined();
|
||||
expect('rank' in hit).toBe(true);
|
||||
expect('highlight' in hit).toBe(true);
|
||||
expect('path' in hit).toBe(false);
|
||||
expect('snippet' in hit).toBe(false);
|
||||
expect('score' in hit).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,111 @@
|
||||
import { Kysely, sql } from 'kysely';
|
||||
import { getTestDb, destroyTestDb } from './db';
|
||||
import * as migration from '../../src/database/migrations/20260707T130000-search-ru-en-config';
|
||||
|
||||
/**
|
||||
* #529 A1 — the ru_en config migration must be REVERSIBLE in the correct order:
|
||||
* down() moves pages.tsv + page_embeddings.fts back to `english` BEFORE dropping
|
||||
* the ru_en config (a generated column still depending on ru_en would block the
|
||||
* DROP). This roundtrips down()→up() on the already-migrated test DB and asserts
|
||||
* the config, the pages trigger and the fts generated expression each flip and
|
||||
* flip back — the deploy-critical property.
|
||||
*/
|
||||
describe('search ru_en config migration [integration]', () => {
|
||||
let db: Kysely<any>;
|
||||
|
||||
const configExists = async () => {
|
||||
const r = await sql<{ n: number }>`
|
||||
SELECT count(*)::int AS n FROM pg_ts_config WHERE cfgname = 'ru_en'
|
||||
`.execute(db);
|
||||
return r.rows[0].n > 0;
|
||||
};
|
||||
|
||||
const ftsDef = async () => {
|
||||
const r = await sql<{ def: string }>`
|
||||
SELECT pg_get_expr(adbin, adrelid) AS def
|
||||
FROM pg_attrdef d
|
||||
JOIN pg_attribute a ON a.attrelid = d.adrelid AND a.attnum = d.adnum
|
||||
WHERE a.attname = 'fts' AND a.attrelid = 'page_embeddings'::regclass
|
||||
`.execute(db);
|
||||
return r.rows[0]?.def ?? '';
|
||||
};
|
||||
|
||||
const triggerSrc = async () => {
|
||||
const r = await sql<{ src: string }>`
|
||||
SELECT prosrc AS src FROM pg_proc WHERE proname = 'pages_tsvector_trigger'
|
||||
`.execute(db);
|
||||
return r.rows[0]?.src ?? '';
|
||||
};
|
||||
|
||||
beforeAll(() => {
|
||||
db = getTestDb();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
// Restore the canonical ru_en state for any later suite regardless of where
|
||||
// a test left off. up() is now safe on an existing config (ensureRuEnConfig
|
||||
// no-ops) and re-asserts both stored sides to ru_en.
|
||||
delete process.env.SEARCH_EMBEDDINGS_FTS_INLINE_REWRITE;
|
||||
await migration.up(db);
|
||||
await destroyTestDb();
|
||||
});
|
||||
|
||||
it('starts at ru_en (config present, trigger + fts on ru_en)', async () => {
|
||||
expect(await configExists()).toBe(true);
|
||||
expect(await ftsDef()).toContain('ru_en');
|
||||
expect(await triggerSrc()).toContain('ru_en');
|
||||
});
|
||||
|
||||
it('down() reverts tsv + fts to english THEN drops the config', async () => {
|
||||
await migration.down(db);
|
||||
expect(await configExists()).toBe(false);
|
||||
expect(await ftsDef()).toContain('english');
|
||||
expect(await ftsDef()).not.toContain('ru_en');
|
||||
expect(await triggerSrc()).toContain('english');
|
||||
});
|
||||
|
||||
it('up() re-applies ru_en cleanly (idempotent config create)', async () => {
|
||||
await migration.up(db);
|
||||
expect(await configExists()).toBe(true);
|
||||
expect(await ftsDef()).toContain('ru_en');
|
||||
expect(await triggerSrc()).toContain('ru_en');
|
||||
});
|
||||
|
||||
// B1.1 — running up() a SECOND time on an already-migrated DB is a true no-op:
|
||||
// it must NOT throw (the old drop-recreate-config would fail on the fts hard
|
||||
// dependency) and must NOT re-rewrite the embeddings table — the fts column
|
||||
// already references ru_en, so the at-target check short-circuits.
|
||||
it('up() is idempotent: a 2nd run does not error and leaves ru_en intact', async () => {
|
||||
await expect(migration.up(db)).resolves.toBeUndefined();
|
||||
expect(await configExists()).toBe(true);
|
||||
expect(await ftsDef()).toContain('ru_en');
|
||||
expect(await triggerSrc()).toContain('ru_en');
|
||||
});
|
||||
|
||||
// B1.2 — env-gate opt-out: with SEARCH_EMBEDDINGS_FTS_INLINE_REWRITE=false the
|
||||
// embeddings rewrite is SKIPPED (fts stays where it was) but pages.tsv still
|
||||
// swaps inline, and the ru_en config is left in place (fts still depends on it).
|
||||
// Runs down() as the vehicle: english is NOT the current fts config, so the
|
||||
// skip path — not the at-target no-op — is exercised.
|
||||
it('env-gate=false: skips the fts rewrite but still swaps pages.tsv', async () => {
|
||||
// Precondition: fts + trigger on ru_en (from the prior test).
|
||||
expect(await ftsDef()).toContain('ru_en');
|
||||
process.env.SEARCH_EMBEDDINGS_FTS_INLINE_REWRITE = 'false';
|
||||
try {
|
||||
await migration.down(db);
|
||||
// fts rewrite skipped → still ru_en (the ACCESS EXCLUSIVE rewrite avoided).
|
||||
expect(await ftsDef()).toContain('ru_en');
|
||||
expect(await ftsDef()).not.toContain('english');
|
||||
// pages.tsv trigger still swapped inline to english (gate is fts-only).
|
||||
expect(await triggerSrc()).toContain('english');
|
||||
// Config left in place because fts still references it (guarded drop).
|
||||
expect(await configExists()).toBe(true);
|
||||
} finally {
|
||||
// Restore ru_en fully for the afterAll / later suites.
|
||||
delete process.env.SEARCH_EMBEDDINGS_FTS_INLINE_REWRITE;
|
||||
await migration.up(db);
|
||||
expect(await ftsDef()).toContain('ru_en');
|
||||
expect(await triggerSrc()).toContain('ru_en');
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,276 @@
|
||||
import { Kysely } from 'kysely';
|
||||
import { randomUUID } from 'node:crypto';
|
||||
import { BadRequestException } from '@nestjs/common';
|
||||
import { ShareService } from 'src/core/share/share.service';
|
||||
import { ShareRepo } from '@docmost/db/repos/share/share.repo';
|
||||
import { PageRepo } from '@docmost/db/repos/page/page.repo';
|
||||
import { PageHistoryRepo } from '@docmost/db/repos/page/page-history.repo';
|
||||
import * as publishedModeMigration from 'src/database/migrations/20260712T130000-shares-published-mode';
|
||||
import {
|
||||
getTestDb,
|
||||
destroyTestDb,
|
||||
createWorkspace,
|
||||
createSpace,
|
||||
createPage,
|
||||
} from './db';
|
||||
|
||||
/**
|
||||
* #370 Stage B — "approved" (publish-only-the-saved-version) share mode, against
|
||||
* real Postgres (so the migration column + the hand-written recursive-CTE
|
||||
* threading in getShareForPage are exercised for real, not mocked).
|
||||
*
|
||||
* Guards:
|
||||
* - getShareForPage carries `publishedMode` through ALL of the CTE points
|
||||
* (anchor select, recursive-union select, returned object) — a direct share
|
||||
* AND a descendant (union) share both read it back. If ANY of the 4 threading
|
||||
* points is dropped, one of these reddens.
|
||||
* - createShare / updateShare auto-mint the first manual baseline ON ENABLE.
|
||||
* - approved + includeSubPages is rejected (BadRequestException) on both paths.
|
||||
* - resolveReadableSharePage under approved swaps content+title to the manual
|
||||
* version while PRESERVING page.id + workspaceId.
|
||||
*/
|
||||
describe('share published_mode (approved) [integration]', () => {
|
||||
let db: Kysely<any>;
|
||||
let service: ShareService;
|
||||
let shareRepo: ShareRepo;
|
||||
let pageHistoryRepo: PageHistoryRepo;
|
||||
let wsId: string;
|
||||
let spaceId: string;
|
||||
|
||||
// shares.creator_id / page_history.last_updated_by_id are uuid columns; use
|
||||
// null (nullable) rather than a synthetic non-uuid id.
|
||||
const USER = null as any;
|
||||
|
||||
// resolveReadableSharePage runs the restricted-ancestor gate; keep it open.
|
||||
const pagePermissionRepo = {
|
||||
hasRestrictedAncestor: async () => false,
|
||||
};
|
||||
|
||||
beforeAll(async () => {
|
||||
db = getTestDb();
|
||||
shareRepo = new ShareRepo(db as any, {} as any);
|
||||
const pageRepo = new PageRepo(db as any, {} as any, {} as any);
|
||||
pageHistoryRepo = new PageHistoryRepo(db as any);
|
||||
service = new ShareService(
|
||||
shareRepo as any,
|
||||
pageRepo as any,
|
||||
pagePermissionRepo as any,
|
||||
db as any,
|
||||
{} as any, // tokenService — no attachments in these docs
|
||||
{} as any, // transclusionService
|
||||
{ findById: async () => ({ settings: {} }) } as any, // workspaceRepo
|
||||
pageHistoryRepo as any,
|
||||
);
|
||||
wsId = (await createWorkspace(db)).id;
|
||||
spaceId = (await createSpace(db, wsId)).id;
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await destroyTestDb();
|
||||
});
|
||||
|
||||
const docWith = (text: string) => ({
|
||||
type: 'doc',
|
||||
content: [
|
||||
{ type: 'paragraph', content: [{ type: 'text', text }] },
|
||||
],
|
||||
});
|
||||
|
||||
const newPage = async (
|
||||
text = 'live body',
|
||||
parentPageId?: string,
|
||||
): Promise<string> => {
|
||||
const { id } = await createPage(db, { workspaceId: wsId, spaceId });
|
||||
await db
|
||||
.updateTable('pages')
|
||||
.set({ content: docWith(text) as any, parentPageId: parentPageId ?? null })
|
||||
.where('id', '=', id)
|
||||
.execute();
|
||||
return id;
|
||||
};
|
||||
|
||||
const insertShare = async (args: {
|
||||
pageId: string;
|
||||
publishedMode?: string;
|
||||
includeSubPages?: boolean;
|
||||
}) => {
|
||||
const id = randomUUID();
|
||||
await db
|
||||
.insertInto('shares')
|
||||
.values({
|
||||
id,
|
||||
key: `k-${id.slice(0, 12)}`,
|
||||
pageId: args.pageId,
|
||||
includeSubPages: args.includeSubPages ?? false,
|
||||
publishedMode: args.publishedMode ?? 'live',
|
||||
creatorId: USER,
|
||||
spaceId,
|
||||
workspaceId: wsId,
|
||||
})
|
||||
.execute();
|
||||
return id;
|
||||
};
|
||||
|
||||
const manualRows = (pageId: string) =>
|
||||
db
|
||||
.selectFrom('pageHistory')
|
||||
.select(['id', 'kind', 'title'])
|
||||
.where('pageId', '=', pageId)
|
||||
.where('kind', '=', 'manual')
|
||||
.execute();
|
||||
|
||||
const publishedModeColumn = async () =>
|
||||
db
|
||||
.selectFrom('information_schema.columns' as any)
|
||||
.select(['columnName', 'dataType', 'isNullable', 'columnDefault'] as any)
|
||||
.where('tableName' as any, '=', 'shares')
|
||||
.where('columnName' as any, '=', 'published_mode')
|
||||
.executeTakeFirst();
|
||||
|
||||
// --- migration up/down round-trip ----------------------------------------
|
||||
|
||||
it('migration down() drops published_mode; up() re-adds it NOT NULL default live', async () => {
|
||||
// global-setup already migrated up, so the column is present.
|
||||
const before: any = await publishedModeColumn();
|
||||
expect(before).toBeDefined();
|
||||
expect(before.isNullable).toBe('NO');
|
||||
expect(String(before.columnDefault)).toContain("'live'");
|
||||
|
||||
await publishedModeMigration.down(db as any);
|
||||
expect(await publishedModeColumn()).toBeUndefined();
|
||||
|
||||
// Restore so the rest of the suite (and afterAll) sees the real schema.
|
||||
await publishedModeMigration.up(db as any);
|
||||
const after: any = await publishedModeColumn();
|
||||
expect(after).toBeDefined();
|
||||
expect(after.isNullable).toBe('NO');
|
||||
});
|
||||
|
||||
// --- getShareForPage 4-point threading -----------------------------------
|
||||
|
||||
it('getShareForPage carries publishedMode for a DIRECT share (anchor select + returned object)', async () => {
|
||||
const pageId = await newPage();
|
||||
await insertShare({ pageId, publishedMode: 'approved' });
|
||||
|
||||
const share = await service.getShareForPage(pageId, wsId);
|
||||
expect(share).toBeDefined();
|
||||
expect(share!.publishedMode).toBe('approved');
|
||||
});
|
||||
|
||||
it('getShareForPage carries publishedMode for a DESCENDANT share (recursive-union select)', async () => {
|
||||
// Parent shared with includeSubPages + approved (inserted directly, bypassing
|
||||
// the service XOR gate, purely to exercise the CTE union threading on a
|
||||
// child that inherits the ancestor share).
|
||||
const parentId = await newPage('parent body');
|
||||
await insertShare({
|
||||
pageId: parentId,
|
||||
publishedMode: 'approved',
|
||||
includeSubPages: true,
|
||||
});
|
||||
const childId = await newPage('child body', parentId);
|
||||
|
||||
const share = await service.getShareForPage(childId, wsId);
|
||||
expect(share).toBeDefined();
|
||||
// Resolved via the recursive union up to the ancestor share.
|
||||
expect((share!.level as number) > 0).toBe(true);
|
||||
expect(share!.publishedMode).toBe('approved');
|
||||
});
|
||||
|
||||
// --- baseline auto-create on enable --------------------------------------
|
||||
|
||||
it('createShare(approved) mints the first manual baseline from current content', async () => {
|
||||
const pageId = await newPage('baseline body');
|
||||
const page = { id: pageId, spaceId } as any;
|
||||
|
||||
expect(await manualRows(pageId)).toHaveLength(0);
|
||||
|
||||
await service.createShare({
|
||||
authUserId: USER,
|
||||
workspaceId: wsId,
|
||||
page,
|
||||
createShareDto: { pageId, includeSubPages: false, publishedMode: 'approved' } as any,
|
||||
});
|
||||
|
||||
const rows = await manualRows(pageId);
|
||||
expect(rows).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('updateShare enabling approved on a live share mints the baseline; a second call is idempotent', async () => {
|
||||
const pageId = await newPage('later-saved body');
|
||||
const shareId = await insertShare({ pageId, publishedMode: 'live' });
|
||||
|
||||
expect(await manualRows(pageId)).toHaveLength(0);
|
||||
|
||||
await service.updateShare(shareId, {
|
||||
shareId,
|
||||
publishedMode: 'approved',
|
||||
} as any);
|
||||
expect(await manualRows(pageId)).toHaveLength(1);
|
||||
|
||||
// Idempotent: staying approved does not duplicate the baseline.
|
||||
await service.updateShare(shareId, {
|
||||
shareId,
|
||||
searchIndexing: true,
|
||||
} as any);
|
||||
expect(await manualRows(pageId)).toHaveLength(1);
|
||||
});
|
||||
|
||||
// --- XOR gate ------------------------------------------------------------
|
||||
|
||||
it('createShare rejects approved + includeSubPages', async () => {
|
||||
const pageId = await newPage();
|
||||
await expect(
|
||||
service.createShare({
|
||||
authUserId: USER,
|
||||
workspaceId: wsId,
|
||||
page: { id: pageId, spaceId } as any,
|
||||
createShareDto: {
|
||||
pageId,
|
||||
includeSubPages: true,
|
||||
publishedMode: 'approved',
|
||||
} as any,
|
||||
}),
|
||||
).rejects.toBeInstanceOf(BadRequestException);
|
||||
});
|
||||
|
||||
it('updateShare rejects setting approved while includeSubPages is already on', async () => {
|
||||
const pageId = await newPage();
|
||||
const shareId = await insertShare({ pageId, includeSubPages: true });
|
||||
await expect(
|
||||
service.updateShare(shareId, { shareId, publishedMode: 'approved' } as any),
|
||||
).rejects.toBeInstanceOf(BadRequestException);
|
||||
});
|
||||
|
||||
// --- resolveReadableSharePage content freeze -----------------------------
|
||||
|
||||
it('resolveReadableSharePage under approved swaps content+title but PRESERVES id + workspaceId', async () => {
|
||||
const pageId = await newPage('LIVE draft body');
|
||||
const shareId = await insertShare({ pageId, publishedMode: 'approved' });
|
||||
|
||||
// Seed a manual version distinct from the live draft.
|
||||
await pageHistoryRepo.insertPageHistory({
|
||||
pageId,
|
||||
slugId: `hist-${pageId.slice(0, 8)}`,
|
||||
title: 'SAVED title',
|
||||
content: docWith('SAVED version body') as any,
|
||||
kind: 'manual',
|
||||
spaceId,
|
||||
workspaceId: wsId,
|
||||
} as any);
|
||||
|
||||
const resolved = await service.resolveReadableSharePage(
|
||||
shareId,
|
||||
pageId,
|
||||
wsId,
|
||||
);
|
||||
expect(resolved).not.toBeNull();
|
||||
// Content + title are the SAVED version.
|
||||
expect(resolved!.page.title).toBe('SAVED title');
|
||||
expect(JSON.stringify(resolved!.page.content)).toContain(
|
||||
'SAVED version body',
|
||||
);
|
||||
// id + workspaceId stay LIVE (attachment-token ownership must not shift).
|
||||
expect(resolved!.page.id).toBe(pageId);
|
||||
expect(resolved!.page.workspaceId).toBe(wsId);
|
||||
});
|
||||
});
|
||||
@@ -118,7 +118,10 @@ All 41 tools, grouped by what you'd reach for them.
|
||||
- **`getPage`** — A page's content as clean **Markdown** (canonical for text; drops only
|
||||
block ids, resolved-comment anchors, and a fixed no-Markdown-representation attr set —
|
||||
table spans/colwidth/background, indent, `callout.icon`, `orderedList.type`, and link
|
||||
`internal`/`target`/`rel`/`class`; use `getPageJson` when you need those).
|
||||
`internal`/`target`/`rel`/`class`; use `getPageJson` when you need those). Pass
|
||||
`format:"text"` for a flat, deterministic plain-text rendering (one line per block,
|
||||
marks dropped, stable `[image]`/`[table RxC]` placeholders) to machine-diff what you
|
||||
wrote against what was stored.
|
||||
- **`getPageJson`** — A page's **lossless ProseMirror/TipTap JSON**, including every
|
||||
block's `attrs.id` and the `slugId` used in URLs. This is what the per-block editing
|
||||
tools consume.
|
||||
|
||||
@@ -123,7 +123,10 @@ Docmost-MCP не сочетают:
|
||||
лишь id блоков, якоря разрешённых комментариев и фиксированный набор атрибутов без
|
||||
markdown-представления — спаны/colwidth/фон ячеек таблиц, отступы (indent),
|
||||
`callout.icon`, `orderedList.type` и `internal`/`target`/`rel`/`class` у ссылок;
|
||||
используйте `getPageJson`, когда они нужны).
|
||||
используйте `getPageJson`, когда они нужны). Передайте `format:"text"` для плоского
|
||||
детерминированного plain-text рендера (одна строка на блок, маркировка убрана,
|
||||
стабильные плейсхолдеры `[image]`/`[table RxC]`) — чтобы machine-diff'ом сверить то,
|
||||
что вы записали, с тем, что сохранилось.
|
||||
- **`getPageJson`** — **Lossless ProseMirror/TipTap JSON** страницы, включая `attrs.id`
|
||||
каждого блока и `slugId`, используемый в URL. Именно его потребляют инструменты
|
||||
поблочного редактирования.
|
||||
|
||||
@@ -31,7 +31,11 @@ import { TransformsMixin, type ITransformsMixin } from "./client/transforms.js";
|
||||
// existing importer (index.ts, http.ts, stdio.ts, the in-app host) keeps working
|
||||
// with ZERO changes.
|
||||
export type { DocmostMcpConfig, SandboxPut } from "./client/context.js";
|
||||
export { formatDocmostAxiosError, assertFullUuid } from "./client/errors.js";
|
||||
export {
|
||||
formatDocmostAxiosError,
|
||||
assertFullUuid,
|
||||
formatSpaceNotAccessible,
|
||||
} from "./client/errors.js";
|
||||
|
||||
// Branded canonical page-identity type (#435): the internal page UUID is a
|
||||
// distinct nominal type so an unresolved raw/slug string can't be swapped into
|
||||
|
||||
@@ -725,10 +725,15 @@ export function CommentsMixin<TBase extends GConstructor<DocmostClientContext>>(
|
||||
// The subtree scope (parentPageId given) already INCLUDES the root node
|
||||
// itself: /pages/tree seeds getPageAndDescendants with id = parentPageId, so
|
||||
// no separate getPageRaw fetch for the parent is needed.
|
||||
const { pages: pagesInScope, truncated } = await this.enumerateSpacePages(
|
||||
spaceId,
|
||||
parentPageId,
|
||||
);
|
||||
// #534: the enumerateSpacePages seed (`/pages/tree`) 404s for a bad or
|
||||
// inaccessible spaceId; wrap it so that 404 becomes an actionable "spaceId
|
||||
// not accessible" hint instead of the opaque "Space permissions not found".
|
||||
// Only the whole enumeration is wrapped (the only 404 source here) — see the
|
||||
// wrap-allowlist invariant on withSpaceAccessDiagnostics.
|
||||
const { pages: pagesInScope, truncated } =
|
||||
await this.withSpaceAccessDiagnostics(spaceId, "checkNewComments", () =>
|
||||
this.enumerateSpacePages(spaceId, parentPageId),
|
||||
);
|
||||
|
||||
// 2. Fetch comments for each page, keep ones created after since. Runs with
|
||||
// bounded concurrency (#490) instead of one-at-a-time — the per-page reads are
|
||||
|
||||
@@ -26,7 +26,10 @@ import {
|
||||
import { withPageLock, isUuid } from "../lib/page-lock.js";
|
||||
import type { PageId } from "../lib/page-id.js";
|
||||
import { getCollabToken, performLogin } from "../lib/auth-utils.js";
|
||||
import { formatDocmostAxiosError } from "./errors.js";
|
||||
import {
|
||||
formatDocmostAxiosError,
|
||||
formatSpaceNotAccessible,
|
||||
} from "./errors.js";
|
||||
import { GetPageConversionCache } from "./getpage-cache.js";
|
||||
|
||||
// A generic mixin base constructor (issue #450). Each domain mixin is a factory
|
||||
@@ -117,6 +120,36 @@ function readCollabTokenTtlMs(): number {
|
||||
return Number.isFinite(raw) ? Math.max(0, raw) : 5 * 60 * 1000;
|
||||
}
|
||||
|
||||
/**
|
||||
* Accessible-space index cache TTL in milliseconds (issue #534). Read fresh from
|
||||
* the environment on every access — mirroring readCollabTokenTtlMs above — so a
|
||||
* test or a live rollback can change it without reloading the module.
|
||||
*
|
||||
* The index (see getAccessibleSpaceIndex) is fetched ONLY on the enrich-on-404
|
||||
* slow path to turn an opaque "Space permissions not found" 404 into a factual
|
||||
* "spaceId X is not among your accessible spaces" hint; a short TTL keeps a burst
|
||||
* of failing tool calls from re-sweeping /spaces each time while never widening
|
||||
* the permission-staleness window meaningfully. Default 60s. An EXPLICIT 0 (or
|
||||
* negative) DISABLES the cache (exact fetch-per-enrichment). Unset/unparseable
|
||||
* (NaN) falls back to the 60s default with the cache ON.
|
||||
*/
|
||||
function readSpacesCacheTtlMs(): number {
|
||||
const raw = parseInt(process.env.MCP_SPACES_CACHE_TTL_MS ?? "", 10);
|
||||
return Number.isFinite(raw) ? Math.max(0, raw) : 60000;
|
||||
}
|
||||
|
||||
/**
|
||||
* The set of spaces the current token can see, plus a `complete` flag that is
|
||||
* false when the /spaces listing was truncated at the pagination ceiling. Used
|
||||
* by the enrich-on-404 diagnostics: an authoritative membership test is only
|
||||
* possible when `complete` is true (see withSpaceAccessDiagnostics).
|
||||
*/
|
||||
export type AccessibleSpaceIndex = {
|
||||
ids: Set<string>;
|
||||
spaces: { id: string; name: string }[];
|
||||
complete: boolean;
|
||||
};
|
||||
|
||||
export abstract class DocmostClientContext {
|
||||
protected client: AxiosInstance;
|
||||
protected token: string | null = null;
|
||||
@@ -164,6 +197,27 @@ export abstract class DocmostClientContext {
|
||||
// bypassed on a forced refresh (the 401/403 reauth path). null = no token yet.
|
||||
protected collabTokenCache: { token: string; mintedAt: number } | null = null;
|
||||
|
||||
// Accessible-space index cache + single-flight (issue #534). TWO separate
|
||||
// fields, mirroring loginPromise (in-flight dedup) vs collabTokenCache
|
||||
// (persistent value):
|
||||
// - spaceIndexCache: the last SUCCESSFULLY-FETCHED, COMPLETE index plus the
|
||||
// wall-clock time it was fetched. Written ONLY from a resolved /spaces
|
||||
// sweep whose result was complete (a truncated list is never cached, since
|
||||
// it cannot answer "is this spaceId missing?"). Per-instance (a
|
||||
// DocmostClient is built per user / per chat) so it can never leak across
|
||||
// identities; invalidated on every identity change exactly like
|
||||
// collabTokenCache (login() + the 401/403 reauth interceptor).
|
||||
// - spaceIndexInFlight: dedups concurrent enrich-on-404 fetches into ONE
|
||||
// /spaces sweep. CRITICAL INVARIANT: this promise is nulled in `.finally`
|
||||
// on BOTH resolve AND reject — a rejected/settled promise is NEVER
|
||||
// memoized, so a transient /spaces blip during one failed tool call cannot
|
||||
// poison the diagnostics for the rest of the session.
|
||||
protected spaceIndexCache: {
|
||||
index: AccessibleSpaceIndex;
|
||||
fetchedAt: number;
|
||||
} | null = null;
|
||||
protected spaceIndexInFlight: Promise<AccessibleSpaceIndex> | null = null;
|
||||
|
||||
// Content-addressed conversion cache for getPage (issue #479). Keyed on
|
||||
// (canonical pageId, updatedAt, optionsHash) -> the converted Markdown, so a
|
||||
// re-read of an UNCHANGED page skips the expensive convertProseMirrorToMarkdown
|
||||
@@ -281,6 +335,9 @@ export abstract class DocmostClientContext {
|
||||
// keep serving a collab token minted under the old one.
|
||||
this.token = null;
|
||||
this.collabTokenCache = null;
|
||||
// #534: a new identity/login must not keep serving a space index
|
||||
// computed under the old token (same reasoning as collabTokenCache).
|
||||
this.spaceIndexCache = null;
|
||||
delete this.client.defaults.headers.common["Authorization"];
|
||||
try {
|
||||
await this.login();
|
||||
@@ -413,6 +470,8 @@ export abstract class DocmostClientContext {
|
||||
// Identity (re)established: drop any collab token minted under a
|
||||
// previous identity so the #435 cache can never outlive it.
|
||||
this.collabTokenCache = null;
|
||||
// #534: likewise drop the accessible-space index of the old identity.
|
||||
this.spaceIndexCache = null;
|
||||
this.client.defaults.headers.common["Authorization"] =
|
||||
`Bearer ${token}`;
|
||||
})
|
||||
@@ -630,13 +689,34 @@ export abstract class DocmostClientContext {
|
||||
}
|
||||
|
||||
/**
|
||||
* Generic pagination handler for Docmost API endpoints
|
||||
* Generic pagination handler for Docmost API endpoints. Thin wrapper over
|
||||
* paginateAllWithMeta that discards the `truncated` flag — the historical
|
||||
* contract every caller (getSpaces, etc.) relies on. Callers that need to KNOW
|
||||
* whether the result set was complete (e.g. #534's getAccessibleSpaceIndex,
|
||||
* which must not assert "spaceId missing" against a truncated list) call
|
||||
* paginateAllWithMeta directly.
|
||||
*/
|
||||
async paginateAll<T = any>(
|
||||
endpoint: string,
|
||||
basePayload: Record<string, any> = {},
|
||||
limit: number = 100,
|
||||
): Promise<T[]> {
|
||||
return (await this.paginateAllWithMeta<T>(endpoint, basePayload, limit))
|
||||
.items;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generic pagination handler that ALSO surfaces whether the result was
|
||||
* truncated at the MAX_PAGES ceiling. `paginateAll` swallows this flag (it only
|
||||
* warns); callers that must distinguish "complete listing" from "gave up at the
|
||||
* cap" use this overload. `truncated` is true iff the loop stopped at the
|
||||
* ceiling while the server still reported more pages.
|
||||
*/
|
||||
async paginateAllWithMeta<T = any>(
|
||||
endpoint: string,
|
||||
basePayload: Record<string, any> = {},
|
||||
limit: number = 100,
|
||||
): Promise<{ items: T[]; truncated: boolean }> {
|
||||
await this.ensureAuthenticated();
|
||||
|
||||
const clampedLimit = Math.max(1, Math.min(100, limit));
|
||||
@@ -697,14 +777,154 @@ export abstract class DocmostClientContext {
|
||||
);
|
||||
}
|
||||
|
||||
return allItems;
|
||||
return { items: allItems, truncated };
|
||||
}
|
||||
|
||||
/**
|
||||
* The set of spaces the current token can access (issue #534), fetched from the
|
||||
* single source of truth — the `/spaces` listing — with a per-instance
|
||||
* short-TTL cache and single-flight dedup. Used ONLY by the enrich-on-404 slow
|
||||
* path (withSpaceAccessDiagnostics), so the happy path incurs ZERO extra
|
||||
* requests.
|
||||
*
|
||||
* `complete` is `!truncated`: it is false when the /spaces listing was cut at
|
||||
* the pagination ceiling. A truncated index can never authoritatively answer
|
||||
* "is this spaceId missing?", so only a complete result is cached AND only a
|
||||
* complete result is allowed to drive the "not accessible" rewrite.
|
||||
*
|
||||
* Cache/single-flight discipline (see the spaceIndexCache / spaceIndexInFlight
|
||||
* field docs):
|
||||
* - serve a fresh, complete cached index without any request;
|
||||
* - otherwise collapse concurrent callers onto ONE in-flight /spaces sweep;
|
||||
* - write the persistent cache ONLY from a resolved, complete fetch;
|
||||
* - null the in-flight promise on BOTH resolve and reject (never memoize a
|
||||
* rejected promise — a transient /spaces failure must be retried fresh).
|
||||
*/
|
||||
async getAccessibleSpaceIndex(): Promise<AccessibleSpaceIndex> {
|
||||
const ttl = readSpacesCacheTtlMs();
|
||||
|
||||
// Fast path: a still-fresh, complete cached index needs no request at all.
|
||||
if (
|
||||
ttl > 0 &&
|
||||
this.spaceIndexCache &&
|
||||
Date.now() - this.spaceIndexCache.fetchedAt < ttl
|
||||
) {
|
||||
return this.spaceIndexCache.index;
|
||||
}
|
||||
|
||||
// Single-flight: a concurrent enrichment joins the in-flight sweep instead of
|
||||
// issuing its own. (A settled/rejected promise is never left here — see the
|
||||
// `.finally` below — so this only ever joins a genuinely in-progress fetch.)
|
||||
if (this.spaceIndexInFlight) return this.spaceIndexInFlight;
|
||||
|
||||
const fetchPromise = (async (): Promise<AccessibleSpaceIndex> => {
|
||||
const { items, truncated } = await this.paginateAllWithMeta("/spaces", {});
|
||||
const spaces = items.map((s: any) => ({
|
||||
id: s?.id,
|
||||
name: s?.name,
|
||||
}));
|
||||
return {
|
||||
ids: new Set(spaces.map((s) => s.id)),
|
||||
spaces,
|
||||
complete: !truncated,
|
||||
};
|
||||
})();
|
||||
|
||||
this.spaceIndexInFlight = fetchPromise
|
||||
.then((index) => {
|
||||
// Cache ONLY a complete result, and only while the cache is enabled.
|
||||
if (ttl > 0 && index.complete) {
|
||||
this.spaceIndexCache = { index, fetchedAt: Date.now() };
|
||||
}
|
||||
return index;
|
||||
})
|
||||
.finally(() => {
|
||||
// CRITICAL (#534): clear the in-flight slot on BOTH resolve and reject.
|
||||
// Nulling on reject too means a transient /spaces error is retried by the
|
||||
// NEXT enrichment with a fresh fetch, never re-serving the rejection.
|
||||
this.spaceIndexInFlight = null;
|
||||
});
|
||||
|
||||
return this.spaceIndexInFlight;
|
||||
}
|
||||
|
||||
/**
|
||||
* Wrap a client method whose 404 means "the supplied spaceId is not accessible"
|
||||
* and, ONLY on that 404, replace the opaque server text ("Space permissions not
|
||||
* found") with a factual, actionable message naming the spaceId and the spaces
|
||||
* the token can actually see (issue #534). A HINT layered on top of the
|
||||
* backend, which stays authoritative — so it FAILS OPEN on ANY uncertainty:
|
||||
* every branch below that is not a confident "this spaceId is genuinely
|
||||
* missing" rethrows the ORIGINAL server error unchanged. The happy path returns
|
||||
* fn()'s value with zero extra requests.
|
||||
*
|
||||
* WRAP-ALLOWLIST INVARIANT (load-bearing — read before wrapping a new method):
|
||||
* among the currently wrapped tools a 404 comes ONLY from the spaceId
|
||||
* membership / space-permissions check — their pageId / rootPageId /
|
||||
* parentPageId branches resolve to 403 or 200, NEVER 404. If a future change
|
||||
* adds a `NotFoundException` to `/pages/tree`, `/pages/recent`,
|
||||
* `/pages/sidebar-pages` or `/search` (e.g. "page not found"), this enrichment
|
||||
* would MISATTRIBUTE that 404 to the spaceId. Re-audit the wrapped call before
|
||||
* relying on this, and only wrap paths where the sole 404 cause is the space.
|
||||
*/
|
||||
protected async withSpaceAccessDiagnostics<T>(
|
||||
spaceId: string,
|
||||
mcpName: string,
|
||||
fn: () => Promise<T>,
|
||||
): Promise<T> {
|
||||
try {
|
||||
return await fn();
|
||||
} catch (e) {
|
||||
// Abort/cap wins FIRST and is detected by the SIGNAL FLAG, not e.name: a
|
||||
// per-call cap may be an AbortSignal.timeout() (reason name "TimeoutError")
|
||||
// or a custom reason, so `e.name === 'AbortError'` is NOT reliable (#534
|
||||
// hole B). A stopped/capped turn must propagate its reason, never trigger a
|
||||
// /spaces sweep or a rewrite.
|
||||
if (this.toolAbortSignal?.aborted) throw e;
|
||||
|
||||
// Only a 404 is enrichable; any other status/shape is a different failure.
|
||||
if (!(axios.isAxiosError(e) && e.response?.status === 404)) throw e;
|
||||
|
||||
let idx: AccessibleSpaceIndex;
|
||||
try {
|
||||
idx = await this.getAccessibleSpaceIndex();
|
||||
} catch (fetchErr) {
|
||||
// The /spaces sweep itself failed. If we were aborted mid-sweep,
|
||||
// propagate the abort reason; otherwise FAIL OPEN with the ORIGINAL
|
||||
// server error rather than a misleading "not found".
|
||||
if (this.toolAbortSignal?.aborted) throw fetchErr;
|
||||
if (process.env.DEBUG) {
|
||||
console.error("space-diag: /spaces fetch failed:", fetchErr);
|
||||
}
|
||||
throw e;
|
||||
}
|
||||
|
||||
// Fail open when the listing is incomplete (can't assert "missing") or when
|
||||
// the spaceId IS present (the 404 is about something else, not the space).
|
||||
if (!idx.complete) throw e;
|
||||
if (idx.ids.has(spaceId)) throw e;
|
||||
|
||||
// Confident: the spaceId is well-formed but not among the accessible
|
||||
// spaces. Replace the opaque server text with the actionable fact.
|
||||
throw new Error(formatSpaceNotAccessible(mcpName, spaceId, idx.spaces));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/** Raw page info including the ProseMirror JSON content and slugId. */
|
||||
async getPageRaw(pageId: string) {
|
||||
/**
|
||||
* Raw page info including the ProseMirror JSON content and slugId.
|
||||
*
|
||||
* With `format:"text"` (#502) the server instead renders `content` as a flat,
|
||||
* deterministic text string (its `jsonToText` path — the SAME serializer that
|
||||
* feeds search), so the MCP text read reuses the server's ONE serializer
|
||||
* rather than shipping a second one. Every other caller omits `format` and
|
||||
* gets the JSON content unchanged.
|
||||
*/
|
||||
async getPageRaw(pageId: string, format?: "text") {
|
||||
await this.ensureAuthenticated();
|
||||
const response = await this.client.post("/pages/info", { pageId });
|
||||
const body: Record<string, unknown> = { pageId };
|
||||
if (format) body.format = format;
|
||||
const response = await this.client.post("/pages/info", body);
|
||||
return response.data?.data ?? response.data;
|
||||
}
|
||||
|
||||
|
||||
@@ -46,6 +46,60 @@ export function assertFullUuid(
|
||||
}
|
||||
}
|
||||
|
||||
// Max number of accessible spaces to enumerate inline in the "space not
|
||||
// accessible" message (issue #534) before collapsing the rest into a "(+N ещё)"
|
||||
// tail, so a workspace with many spaces cannot blow up the model context.
|
||||
const SPACE_LIST_CAP = 10;
|
||||
|
||||
/**
|
||||
* Compose the model-facing "spaceId is not accessible" message (issue #534).
|
||||
* This is FACT text about the supplied spaceId that REPLACES the opaque server
|
||||
* string ("Space permissions not found") on the enrich-on-404 path — it names
|
||||
* the exact bad id, lists the spaces the token can actually see (id + name, so
|
||||
* the agent can copy the right id verbatim), and points at `listSpaces`.
|
||||
*
|
||||
* Deliberately Russian: like the other agent-facing tool guidance in this repo,
|
||||
* this is the message the acting agent reads to self-correct.
|
||||
*/
|
||||
export function formatSpaceNotAccessible(
|
||||
mcpName: string,
|
||||
spaceId: string,
|
||||
spaces: { id: string; name: string }[],
|
||||
): string {
|
||||
// No accessible spaces at all — a distinct diagnosis (token has no space
|
||||
// access), not "you picked the wrong one from this list".
|
||||
if (!Array.isArray(spaces) || spaces.length === 0) {
|
||||
return `${mcpName}: spaceId "${spaceId}" недоступен, и доступных тебе спейсов нет — проверь доступ токена / вызови listSpaces.`;
|
||||
}
|
||||
|
||||
const shown = spaces.slice(0, SPACE_LIST_CAP);
|
||||
const remaining = spaces.length - shown.length;
|
||||
const tail =
|
||||
remaining > 0 ? ` (+${remaining} ещё, см. listSpaces)` : "";
|
||||
|
||||
// Cap the whole message at ERROR_MESSAGE_CAP (same budget as
|
||||
// formatDocmostAxiosError) so ~10 long space names cannot blow up the model
|
||||
// context. Truncate ONLY the interpolated space LIST — the fixed prefix (which
|
||||
// carries the bad spaceId) and the fixed suffix (the "…из listSpaces"
|
||||
// instruction) are always kept intact, so the actionable parts survive even
|
||||
// when the list is trimmed.
|
||||
const prefix = `${mcpName}: spaceId "${spaceId}" не найден среди доступных тебе спейсов. Доступные: `;
|
||||
const suffix = ` — скопируй нужный id дословно из listSpaces.`;
|
||||
let listed = shown.map((s) => `${s.id} (${s.name})`).join(", ") + tail;
|
||||
const budget = ERROR_MESSAGE_CAP - prefix.length - suffix.length;
|
||||
if (listed.length > budget) {
|
||||
listed = listed.slice(0, Math.max(0, budget - 1)) + "…";
|
||||
}
|
||||
const message = `${prefix}${listed}${suffix}`;
|
||||
// Unconditional final backstop (mirrors formatDocmostAxiosError): the list
|
||||
// cap above assumes a well-formed prefix, but a pathologically long
|
||||
// agent-supplied spaceId lives in the prefix and would otherwise blow past the
|
||||
// budget. Cap the WHOLE message so nothing bloats the model context.
|
||||
return message.length > ERROR_MESSAGE_CAP
|
||||
? message.slice(0, ERROR_MESSAGE_CAP - 1) + "…"
|
||||
: message;
|
||||
}
|
||||
|
||||
// Keep ONLY the pathname of a request (no host, no query string, no fragment)
|
||||
// so the message never leaks a host or query params. Resolves a relative
|
||||
// config.url against config.baseURL, then discards everything but the path.
|
||||
|
||||
@@ -93,6 +93,13 @@ export function PagesMixin<TBase extends GConstructor<DocmostClientContext>>(Bas
|
||||
const buildForm = () => {
|
||||
const form = new FormData();
|
||||
form.append("spaceId", spaceId);
|
||||
// #502: this is an AGENT-authored body (plain prose / config), so tell the
|
||||
// server import path to run the markdown importer with the two layered
|
||||
// extensions OFF — a `$…$` span stays literal text (real math via
|
||||
// `update_page_json`) and a schemeless `www.host`/email is not autolinked
|
||||
// (an explicit `https://…` still links). A human file upload never sends
|
||||
// this field, so human imports keep math + autolink ON.
|
||||
form.append("disableMarkdownExtensions", "true");
|
||||
form.append("file", fileContent, {
|
||||
filename: `${title || "import"}.md`,
|
||||
contentType: "text/markdown",
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user