Compare commits

..

15 Commits

Author SHA1 Message Date
agent_coder 9d1b033fe8 fix(#344 review F1-F4): test-mock coverage + getSpaces freshness + comment/test fixes
- F1 [blocking]: share-modal.test.tsx + comment-content-view.test.tsx mocked
  page-query without usePageMetaQuery → 3 tests threw (ShareModal uses it
  directly, comment-content-view via MentionContent). Added usePageMetaQuery to
  both mocks (the space-tree mocks were already fixed; these two were missed).
- F2: restored refetchOnMount:true on useGetSpacesQuery — ["spaces"] is
  invalidated only by same-tab mutations (no socket path), so a cross-actor
  change (an admin adding/removing THIS user from a space) left the list stale
  until a hard reload. The other refetchOnMount removals (favorites/watched —
  per-user, same-tab-only gap) stay removed.
- F3: corrected the trash-list + recent-changes KEEP comments — both keys ARE
  invalidated (trash-list by 3 mutations, recent-changes by page CRUD), but
  invalidateQueries only marks an UNMOUNTED query stale without refetching, so the
  mount refetch closes the gap. The old "never invalidated" wording was wrong and
  risked a maintainer deleting a live invalidation as dead code.
- F4: tests for the two load-bearing pure paths — invalidate-on-update-page (the
  undefined-guard: a title-only event keeps the icon; sibling/unrelated subtrees
  untouched) and breadcrumb-path-equal (equal chain → true; any id/slugId/name/
  icon change or length diff → false; both-null → true). Exported
  breadcrumbPathEqual for the test.

Gate: client tsc 0; the 4 affected/new test files 33 passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 01:05:32 +03:00
agent_coder fcbe840c74 perf(client): cut background re-renders + duplicate work (#344)
Outside the editor the UI did background work on every tree event, socket
reconnect, and navigation. Tree infra (virtualization/memo/O(N) utils) was
already good — the cost was in the subscriptions and duplicates around it.
Client-only; behavior 1:1.

- Setter-only atom subscriptions → useSetAtom: space-tree-row, use-tree-mutation,
  use-tree-socket no longer subscribe every visible row to the WHOLE treeDataAtom
  value (a tree event re-rendered all ~20-30 rows, bypassing the DocTreeRow memo).
  space-tree-node-menu / mention-list read the tree imperatively (store.get) in
  their handlers only. breadcrumb.tsx uses a selectAtom slice (ancestor chain +
  field equality) instead of the whole-tree subscription.
- Socket handler cleanup (BUG): use-tree-socket + use-query-subscription now
  socket.off() their named handlers on cleanup (were accumulating listeners on
  every reconnect → duplicated invalidations/tree-walks). Mirrors
  use-notification-socket.
- Field-update tree path: invalidateOnUpdatePage does a pointwise patch of the
  cached embed subtrees instead of a blanket invalidatePageTree() (refetch storm);
  structural events keep the blanket invalidate.
- usePageMetaQuery: a content-less select slice for the 13 peripheral subscribers
  that read only title/permissions/id, so they stop re-rendering every ~3s while
  typing / on every collab page.updated (page.tsx keeps the full query for content).
- page.tsx: skeleton + placeholderData keepPreviousData (no blank flash on nav).
- Removed refetchOnMount:true where socket/mutation invalidation already keeps the
  cache fresh (favorite/space/space-watcher/workspace). KEPT it on the 3 queries
  with NO other freshness path (trash-list, created-by, recent-changes) — the
  global default is refetchOnMount:false, so those overrides are load-bearing.
- Small: resize mousemove/up attached only while dragging; per-row emoji-picker
  keydown gated on `opened`; AiChatWindow queries enabled only when the window is
  open.

Gate: client tsc 0, client vitest page+websocket 200 passed (+editor suites),
build ok.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 00:34:51 +03:00
agent_vscode 5336f06d10 Merge pull request 'fix(e2e)+ci: канон callout '> [!info]' в e2e-mcp + параллельная сборка с гейтом на publish' (#356) from fix/e2e-callout-and-gate-build into develop 2026-07-04 22:42:11 +03:00
agent_vscode 4bd579f7f6 ci(develop): build image in parallel with tests, gate only the publish
Two-phase scheme instead of the sequential gate: the build job runs in
parallel with test/e2e jobs and only warms the buildx GHA cache
(push:false, cache-to mode=max); a new publish job (needs: test,
e2e-server, e2e-mcp, build) rebuilds from the warm cache (near-instant
on hit, full rebuild on eviction — same as the old sequential timing)
and pushes :develop. GHCR login moved to publish; build-args blocks are
kept textually identical between the two jobs so the cache hits.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:41:25 +03:00
agent_vscode 7bf1c91a95 ci(develop): gate the :develop image build on e2e suites
Reverse the previous policy where e2e jobs only turned the run red
without blocking the image publish: build.needs now lists test,
e2e-server and e2e-mcp, so a failing test of any kind stops the
:develop image from being built and pushed. Stale policy comments
updated accordingly.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:33:06 +03:00
agent_vscode 6c82c54470 test(mcp): expect Obsidian '> [!info]' callout export in e2e (#333 canon)
PR #333 deliberately changed the canonical markdown export of callout
nodes to the Obsidian-native format ('> [!type]' + blockquote body,
pinned by packages/prosemirror-markdown unit tests); the importer still
parses both ':::type' fences and '> [!type]'. The get_page e2e assertion
was missed in that switch and still expected ':::info', failing the
e2e-mcp job on develop since 4369bbc5.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:33:06 +03:00
agent_vscode 382e5196da Merge pull request 'fix(docker): toolchain python3/make/g++ для нативной сборки re2' (#353) from fix/docker-re2-toolchain into develop 2026-07-04 22:11:49 +03:00
agent_vscode 76e0c08cec fix(docker): install python3/make/g++ toolchain for re2 native build
The develop image build broke at `pnpm install --frozen-lockfile`: the new
native dependency re2@1.25.0 (packages/mcp, search_in_page #330) always
compiles from source under pnpm — its prebuilt-binary downloader
(install-artifact-from-github) cannot identify the GitHub repo because pnpm
does not populate npm_package_repository_*/npm_package_json env vars ("No
github repository was identified. Building locally ..."), and node:22-slim
ships no python3/make/g++ for the node-gyp fallback.

- builder stage: add a cache-friendly apt layer with python3 make g++
  before COPY; the stage is discarded so the toolchain may stay.
- installer stage: install the toolchain, run the prod install as the node
  user via `su node -c`, and purge the toolchain — all in one RUN layer so
  the final image stays slim and node_modules ownership needs no extra
  chown layer; USER node is restored right after.

Fixes the failed run 28715009124 (develop docker build); release.yml uses
the same Dockerfile and is covered too.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-04 22:09:40 +03:00
vvzvlad 8978d69f3e Merge pull request 'fix(converter): стабильность round-trip image/медиа — «» ≡ absent (класс defaults-instability)' (#350) from fix/media-roundtrip-stability into develop
Reviewed-on: #350
2026-07-04 21:30:12 +03:00
agent_coder c192f2a2e1 test(prosemirror-markdown): pin the third state — explicit "" converges once, then idempotent
Reviewer addition to the round-trip stability matrix: besides "attr absent" and
"attr has a real value", a string attr in the empty-string class has a third,
degenerate state — a LITERAL "" (a user types alt/title/name in the editor then
deletes it, and Tiptap persists `attr: ""`, distinct from never-set). The fix's
`getAttribute(...) || null` coercion normalizes such a stored "" to the default
on the FIRST round-trip (a one-time "" -> null diff) and is byte-stable from the
SECOND round-trip on.

Adds a convergence contract to the reusable matrix helper (emptyStringClass flag
+ runConvergenceCase): pass 1 must converge the attr to its schema default (NOT
asserted byte-stable vs the "" input — that is the intended one-time
normalization); pass 2 must deep-equal pass 1 (idempotent thereafter). Driven for
every empty-string-class attr across image + the media family (image/drawio
alt+title, video alt via aria-label, pdf/attachment name, attachment mime).
Documents the one-time normalization so a future sync/QA diff does not flag the
single "" -> null change as converter corruption.

Gate: package suite 33 files / 682 tests passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 21:17:17 +03:00
vvzvlad d78b985062 Merge pull request 'perf(comment): статический рендер + ленивые редакторы + мемоизация панели (#340)' (#349) from fix/340-comment-panel-perf into develop
Reviewed-on: #349
2026-07-04 20:55:11 +03:00
agent_coder 2ce672709a fix(prosemirror-markdown): stabilize image round-trip — "" ≡ absent on parse (empty-string class)
A stored image authored without `alt` gained a phantom `alt: ""` on every
round-trip (`markdownToProseMirror(convertProseMirrorToMarkdown(doc))`): `marked`
renders `![](src)` as `<img alt="">`, and the stock tiptap Image `alt` parseHTML
(`getAttribute("alt")`) materialized the empty string where the original had no
attribute. That false diff is a real GS-EDIT-REVERT churn source — an agent /
git-sync touch of a page with an image mutates the stored JSON (`absent -> ""`),
producing phantom diffs that can overwrite live edits.

Fix is PARSE-SIDE ("" ≡ absent), so the RAW round-trip is idempotent — not only
the canonical form (history / stored JSON diff on the raw shape; masking it only
in canonicalize would leave that noise). `image.alt`/`title` parseHTML now coerce
`getAttribute(...) || null`, plus defense-in-depth `|| null` across the at-risk
empty-string class (video aria-label, drawio/excalidraw title+alt, pdf name,
attachment name+mime) matching the existing `image.caption || null` precedent.

NOTE — image `align` is NOT changed: it round-trips correctly (center via the
schema default "center", left/right via the `<!--img {...}-->` comment). Its
`toBeUndefined()` in the git-sync gate is canonical-form normalization, not a loss.

Intentional divergence from editor-ext: editor-ext's literal `alt` parseHTML
returns "" verbatim, but this coercion CONVERGES on editor-ext's real STORED
shape (an image inserted without alt has no `alt` attribute -> re-parses absent,
never ""), so the round-trip is idempotent and matches real documents.

Adds a reusable, node-agnostic round-trip-stability matrix helper
(test/roundtrip-stability.helper.ts) — given a node + attr spec it enumerates
default/non-default combos and asserts byte-stability of BOTH the raw and the
canonical round-trip (the documented numeric width/height→string coercion encoded
as an explicit allowed normalization) — driven over image + the whole media
family (video/audio/pdf/attachment/embed/drawio/excalidraw). The only raw
empty-string instability it found was image.alt; the family was already stable.

Gate: package suite 33 files / 672 tests passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 20:51:34 +03:00
vvzvlad c252068672 Merge pull request 'feat(ai-chat): отложенная загрузка инструментов (deferred tools + loadTools) (#332)' (#341) from fix/332-deferred-tools into develop
Reviewed-on: #341
2026-07-04 20:47:45 +03:00
agent_coder 68caf8157a test(ai-chat): document AI_CHAT_DEFERRED_TOOLS + pin ON-path & catalog completeness (#341 review F1-F3)
- F1: document AI_CHAT_DEFERRED_TOOLS in .env.example (AI_* section) — default
  ON = deferred loading (compact catalog + loadTools), =false restores the old
  "all tools always active" behavior.
- F2: integration test of the ON path in ai-chat-stream.int-spec.ts — a deferred
  tool activated via loadTools is active on the SAME turn's next step but a fresh
  turn starts cold (CORE + loadTools only), proving the per-turn activatedTools
  Set does not leak across turns/chats. Drives the real streamText loop with a
  MockLanguageModelV3 and inspects recorded per-step activeTools-filtered tools.
- F3: replace the magic toHaveLength(28) in tool-tiers.spec.ts with a two-way
  partition against the LIVE in-app toolset (AiChatToolsService.forUser keys):
  every non-core tool must appear in buildInAppDeferredCatalog and every catalog
  entry must map to a real non-core tool — so a future tool forgotten in
  INLINE_TOOL_TIERS fails the suite instead of silently vanishing from the agent.

No production logic change (mechanism was already reviewed correct).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 20:34:42 +03:00
claude code agent 227 e431b33bb1 feat(ai-chat): deferred tool loading (tiers + loadTools meta-tool) (#332)
The in-app AI agent shipped all ~41 tool schemas on every model step. This
adds a two-tier catalog: core tools (frequent or one-line) stay always-active;
the rest are advertised as a compact catalog and their full schema is fetched
on demand via the loadTools meta-tool, wired through ai@6 prepareStep's
per-step activeTools.

- tools/tool-tiers.ts: CORE_TOOL_KEYS, INLINE_TOOL_TIERS, applyLoadTools,
  catalog builders (+ tool-tiers.spec.ts, 13 cases).
- ai-chat.service.ts prepareAgentStep: returns activeTools =
  [...CORE_TOOL_KEYS, loadTools, ...activatedTools]; per-turn activated Set.
- ai-chat.prompt.ts: buildToolCatalogBlock renders the deferred catalog.
- mcp/tool-specs.ts: tier + catalogLine metadata (external snake_case /mcp
  transport unchanged).
- EnvironmentService.isAiChatDeferredToolsEnabled(): AI_CHAT_DEFERRED_TOOLS,
  default ON per issue intent (kill-switch =false restores old behavior).

Gate: server ai-chat 631/631, tool-tiers 13/13, mcp 472/472, tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 19:57:11 +03:00
53 changed files with 2279 additions and 115 deletions
+7
View File
@@ -202,6 +202,13 @@ MCP_DOCMOST_PASSWORD=
# Default 900000 (15 min).
# AI_MCP_CALL_TIMEOUT_MS=900000
# Deferred tool loading for the in-app AI chat (#332). Default ON: the agent sees
# a compact <tool_catalog> and only CORE tools + a loadTools meta-tool are active
# each step; deferred tools (the fat/rare ones + all external MCP tools) load on
# demand. Set AI_CHAT_DEFERRED_TOOLS=false to restore the old "all tools always
# active" behavior.
# AI_CHAT_DEFERRED_TOOLS=true
# --- Anonymous public-share AI assistant ---
# Opt-in per workspace (AI settings -> "public share assistant"; off by default).
# When enabled, anonymous visitors of a published share can ask an AI about that
+42 -11
View File
@@ -18,12 +18,48 @@ env:
IMAGE: ghcr.io/vvzvlad/gitmost
jobs:
# Run the reusable test suite first so a failing test blocks the image build.
# Run the reusable test suite. Together with the e2e jobs below it gates the
# publish job (the image push), not the build itself — build runs in parallel.
test:
uses: ./.github/workflows/test.yml
# Runs in parallel with the test/e2e jobs and only warms the buildx cache
# (GHA cache, scope develop-amd64). No push happens here — the publish job
# below is the only one that pushes the image.
build:
needs: test
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Resolve version
id: version
run: echo "value=$(git describe --tags --always)" >> "$GITHUB_OUTPUT"
- name: Build develop image (warm cache, no push)
uses: docker/build-push-action@v6
with:
context: .
platforms: linux/amd64
build-args: |
APP_VERSION=${{ steps.version.outputs.value }}
AI_AGENT_ROLES_CATALOG_URL=https://raw.githubusercontent.com/vvzvlad/gitmost/develop/agent-roles-catalog
push: false
cache-from: type=gha,scope=develop-amd64
cache-to: type=gha,scope=develop-amd64,mode=max,ignore-error=true
# The gate: rebuilds from the cache the build job just wrote (near-instant on
# a cache hit; worst case — cache eviction — a full rebuild, which matches the
# old sequential timing) and pushes :develop only when unit tests AND both
# e2e suites AND the build are green.
publish:
needs: [test, e2e-server, e2e-mcp, build]
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
@@ -57,13 +93,10 @@ jobs:
push: true
tags: ${{ env.IMAGE }}:develop
cache-from: type=gha,scope=develop-amd64
cache-to: type=gha,scope=develop-amd64,mode=max,ignore-error=true
# e2e jobs run on every develop push but DO NOT gate the build/publish above:
# `build` stays `needs: test` only, so the :develop image still ships even if
# e2e fails. A failing e2e job turns the run red and triggers GitHub's email
# to the pusher — that red run + email is the intended notification, not a
# deploy block.
# e2e jobs gate the publish (image push), not the build: the :develop image
# is pushed only when unit tests AND both e2e suites pass (publish.needs
# lists them all).
e2e-server:
runs-on: ubuntu-latest
# Hard cap: the full-AppModule e2e leaks open handles and hung jest to the 6h max.
@@ -124,9 +157,7 @@ jobs:
- name: Run server e2e
run: pnpm --filter ./apps/server test:e2e
# Same rationale as e2e-server: this job is intentionally NOT in
# `build.needs`. Deploy of the :develop image must not be blocked by e2e;
# a red run plus GitHub's email to the pusher is the notification mechanism.
# Gates the publish too — see the comment above e2e-server.
e2e-mcp:
runs-on: ubuntu-latest
timeout-minutes: 20
+16 -2
View File
@@ -5,6 +5,13 @@ RUN npm install -g pnpm@10.4.0
FROM base AS builder
# re2 (packages/mcp) always compiles from source under pnpm (the prebuilt-binary
# download cannot identify the GitHub repo), so node-gyp needs python3/make/g++.
# This stage is discarded, so the toolchain can stay installed.
RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 make g++ \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY . .
@@ -57,9 +64,16 @@ COPY --from=builder /app/patches /app/patches
RUN chown -R node:node /app
USER node
# Toolchain is needed transiently to compile re2 during the prod install; install
# and purge it in one layer to keep the final image slim. The install itself runs
# as the node user via su to keep node_modules ownership without a costly chown layer.
RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 make g++ \
&& su node -c "pnpm install --frozen-lockfile --prod" \
&& apt-get purge -y --auto-remove python3 make g++ \
&& rm -rf /var/lib/apt/lists/*
RUN pnpm install --frozen-lockfile --prod
USER node
RUN mkdir -p /app/data/storage
@@ -67,14 +67,20 @@ export default function GlobalAppShell({
);
useEffect(() => {
//https://codesandbox.io/p/sandbox/kz9de
// Attach the global mousemove/mouseup only WHILE resizing (started on the
// handle's mousedown via startResizing → isResizing=true) and detach on
// mouseup (stopResizing → isResizing=false). Previously these listeners were
// attached for the whole app lifetime, so every mouse move over the app ran
// the resize handler.
// https://codesandbox.io/p/sandbox/kz9de
if (!isResizing) return;
window.addEventListener("mousemove", resize);
window.addEventListener("mouseup", stopResizing);
return () => {
window.removeEventListener("mousemove", resize);
window.removeEventListener("mouseup", stopResizing);
};
}, [resize, stopResizing]);
}, [isResizing, resize, stopResizing]);
const location = useLocation();
const isSettingsRoute = location.pathname.startsWith("/settings");
+17 -9
View File
@@ -5,7 +5,7 @@ import {
Button,
useMantineColorScheme,
} from "@mantine/core";
import { useClickOutside, useDisclosure, useWindowEvent } from "@mantine/hooks";
import { useClickOutside, useDisclosure } from "@mantine/hooks";
import { Suspense } from "react";
import { useTranslation } from "react-i18next";
@@ -57,14 +57,22 @@ function EmojiPicker({
[dropdown, target],
);
// We need this because the default Mantine popover closeOnEscape does not work
useWindowEvent("keydown", (event) => {
if (opened && event.key === "Escape") {
event.stopPropagation();
event.preventDefault();
handlers.close();
}
});
// We need this because the default Mantine popover closeOnEscape does not work.
// Attach the global keydown ONLY while the picker is open (every tree row
// renders an EmojiPicker, so an always-on window listener meant ~20-30 idle
// keydown handlers firing on each keystroke).
useEffect(() => {
if (!opened) return;
const handleKeydown = (event: KeyboardEvent) => {
if (event.key === "Escape") {
event.stopPropagation();
event.preventDefault();
handlers.close();
}
};
window.addEventListener("keydown", handleKeydown);
return () => window.removeEventListener("keydown", handleKeydown);
}, [opened, handlers]);
// emoji-mart's built-in autoFocus calls .focus() without preventScroll, which
// makes the browser scroll every scrollable ancestor of the search input to
@@ -36,7 +36,7 @@ import {
desktopSidebarAtom,
mobileSidebarAtom,
} from "@/components/layouts/global/hooks/atoms/sidebar-atom.ts";
import { usePageQuery } from "@/features/page/queries/page-query.ts";
import { usePageMetaQuery } from "@/features/page/queries/page-query.ts";
import { extractPageSlugId } from "@/lib";
import {
AI_CHATS_RQ_KEY,
@@ -219,7 +219,9 @@ export default function AiChatWindow() {
// left partly off-screen).
const [geom, setGeom] = useAtom(aiChatWindowGeomAtom);
const { data: chats } = useAiChatsQuery();
// Gated on windowOpen: the chat list is only needed once the window is open,
// so a closed window issues no chat-list request/refetch on navigation.
const { data: chats } = useAiChatsQuery(windowOpen);
// Roles for the new-chat picker (any member may list them). Only fetched while
// the window is open.
const { data: roles } = useAiRolesQuery(windowOpen);
@@ -231,8 +233,10 @@ export default function AiChatWindow() {
[roles],
);
// Gated on windowOpen too: no message history is fetched while the window is
// closed (it is loaded when the window opens with an active chat).
const { data: messageRows, isLoading: messagesLoading } =
useAiChatMessagesQuery(activeChatId ?? undefined);
useAiChatMessagesQuery(activeChatId ?? undefined, windowOpen);
// The page the user is currently viewing. AiChatWindow lives in a pathless
// parent layout route, so useParams() can't see :pageSlug. Match the full
@@ -244,7 +248,7 @@ export default function AiChatWindow() {
// reads/writes via its CASL-enforced page tools using the id.
const pageRouteMatch = useMatch("/s/:spaceSlug/p/:pageSlug");
const pageSlug = pageRouteMatch?.params?.pageSlug;
const { data: openPageData } = usePageQuery({
const { data: openPageData } = usePageMetaQuery({
pageId: extractPageSlugId(pageSlug),
});
const openPage = openPageData
@@ -52,8 +52,12 @@ export const AI_CHAT_MESSAGES_RQ_KEY = (chatId: string) => [
chatId,
];
/** Paginated list of the current user's chats (auto-loads further pages). */
export function useAiChatsQuery() {
/**
* Paginated list of the current user's chats (auto-loads further pages).
* `enabled` (default true) lets the AI chat window skip fetching while it is
* closed — the list is only needed once the window is open.
*/
export function useAiChatsQuery(enabled: boolean = true) {
const query = useInfiniteQuery({
queryKey: AI_CHATS_RQ_KEY,
queryFn: ({ pageParam }) =>
@@ -61,6 +65,7 @@ export function useAiChatsQuery() {
initialPageParam: undefined as string | undefined,
getNextPageParam: (lastPage) =>
lastPage.meta.hasNextPage ? (lastPage.meta.nextCursor ?? undefined) : undefined,
enabled,
});
const data = useMemo<IPagination<IAiChat> | undefined>(() => {
@@ -83,7 +88,10 @@ export function useAiChatsQuery() {
* Load all persisted messages of a chat (oldest first), flattening the
* paginated server response. Used to seed `useChat` initial messages.
*/
export function useAiChatMessagesQuery(chatId: string | undefined) {
export function useAiChatMessagesQuery(
chatId: string | undefined,
enabled: boolean = true,
) {
const query = useInfiniteQuery({
queryKey: AI_CHAT_MESSAGES_RQ_KEY(chatId ?? ""),
queryFn: ({ pageParam }) =>
@@ -91,7 +99,7 @@ export function useAiChatMessagesQuery(chatId: string | undefined) {
initialPageParam: undefined as string | undefined,
getNextPageParam: (lastPage) =>
lastPage.meta.hasNextPage ? (lastPage.meta.nextCursor ?? undefined) : undefined,
enabled: !!chatId,
enabled: !!chatId && enabled,
});
// useInfiniteQuery only fetches the first page on its own. The hook's contract
@@ -15,6 +15,7 @@ vi.mock("@/features/comment/components/comment-editor", () => ({
// case renders in isolation.
vi.mock("@/features/page/queries/page-query.ts", () => ({
usePageQuery: () => ({ data: undefined, isLoading: false, isError: false }),
usePageMetaQuery: () => ({ data: undefined, isLoading: false, isError: false }),
}));
vi.mock("@/features/share/queries/share-query.ts", () => ({
useSharePageQuery: () => ({ data: undefined }),
@@ -22,7 +22,7 @@ import CommentEditor from "@/features/comment/components/comment-editor";
import CommentActions from "@/features/comment/components/comment-actions";
import { useFocusWithin } from "@mantine/hooks";
import { IComment } from "@/features/comment/types/comment.types.ts";
import { usePageQuery } from "@/features/page/queries/page-query.ts";
import { usePageMetaQuery } from "@/features/page/queries/page-query.ts";
import { extractPageSlugId } from "@/lib";
import { useTranslation } from "react-i18next";
import { useGetSpaceBySlugQuery } from "@/features/space/queries/space-query.ts";
@@ -56,7 +56,7 @@ export function buildChildrenByParent(
function CommentListWithTabs({ onClose }: CommentListWithTabsProps) {
const { t } = useTranslation();
const { pageSlug } = useParams();
const { data: page } = usePageQuery({ pageId: extractPageSlugId(pageSlug) });
const { data: page } = usePageMetaQuery({ pageId: extractPageSlugId(pageSlug) });
const {
data: comments,
isLoading: isCommentsLoading,
@@ -24,7 +24,7 @@ import classes from "./link.module.css";
import { useTranslation } from "react-i18next";
import { INTERNAL_LINK_REGEX } from "@/lib/constants";
import { LinkEditorPanel } from "@/features/editor/components/link/link-editor-panel.tsx";
import { usePageQuery } from "@/features/page/queries/page-query.ts";
import { usePageMetaQuery } from "@/features/page/queries/page-query.ts";
import { useSharePageQuery } from "@/features/share/queries/share-query.ts";
import { buildSharedPageUrl } from "@/features/page/page.utils.ts";
import { extractPageSlugId } from "@/lib";
@@ -83,7 +83,7 @@ export default function LinkView(props: MarkViewProps) {
const isPopoverVisible = popoverState !== "closed";
const activeView = isPopoverVisible ? popoverState : lastOpenState.current;
const { data: linkedPage } = usePageQuery({
const { data: linkedPage } = usePageMetaQuery({
pageId: isPopoverVisible && slugId && !isShareRoute ? slugId : null,
});
@@ -25,7 +25,7 @@ import { IconFileDescription, IconPlus } from "@tabler/icons-react";
import { useSpaceQuery } from "@/features/space/queries/space-query.ts";
import { useParams } from "react-router-dom";
import { v7 as uuid7 } from "uuid";
import { useAtom } from "jotai";
import { useAtom, useSetAtom, useStore } from "jotai";
import { currentUserAtom } from "@/features/user/atoms/current-user-atom.ts";
import {
MentionListProps,
@@ -34,7 +34,7 @@ import {
import { IPage } from "@/features/page/types/page.types";
import {
useCreatePageMutation,
usePageQuery,
usePageMetaQuery,
} from "@/features/page/queries/page-query";
import { treeDataAtom } from "@/features/page/tree/atoms/tree-data-atom";
import { treeModel } from "@/features/page/tree/model/tree-model";
@@ -50,12 +50,16 @@ const MentionList = forwardRef<any, MentionListProps>((props, ref) => {
const [countAnnouncement, setCountAnnouncement] = useState("");
const [selectionAnnouncement, setSelectionAnnouncement] = useState("");
const { pageSlug, spaceSlug } = useParams();
const { data: page } = usePageQuery({ pageId: extractPageSlugId(pageSlug) });
const { data: page } = usePageMetaQuery({ pageId: extractPageSlugId(pageSlug) });
const { data: space } = useSpaceQuery(spaceSlug);
const [currentUser] = useAtom(currentUserAtom);
const [renderItems, setRenderItems] = useState<MentionSuggestionItem[]>([]);
const { t } = useTranslation();
const [data, setData] = useAtom(treeDataAtom);
// Setter-only: the tree value is read only imperatively inside createPage
// (via `store` below), never at render, so useSetAtom avoids re-rendering the
// mention popup on any tree event.
const setData = useSetAtom(treeDataAtom);
const store = useStore();
const createPageMutation = useCreatePageMutation();
const emit = useQueryEmit();
const isInCommentContext = props.isInCommentContext ?? false;
@@ -272,9 +276,11 @@ const MentionList = forwardRef<any, MentionListProps>((props, ref) => {
children: [],
};
const lastIndex = data.length;
// Read the live tree imperatively at call time.
const currentTree = store.get(treeDataAtom);
const lastIndex = currentTree.length;
setData(treeModel.insert(data, parentId, newNode, lastIndex));
setData(treeModel.insert(currentTree, parentId, newNode, lastIndex));
props.command({
id: uuid7(),
@@ -2,7 +2,7 @@ import { NodeViewProps, NodeViewWrapper } from "@tiptap/react";
import { ActionIcon, Anchor, Text } from "@mantine/core";
import { IconFileDescription } from "@tabler/icons-react";
import { Link, useLocation, useNavigate, useParams } from "react-router-dom";
import { usePageQuery } from "@/features/page/queries/page-query.ts";
import { usePageMetaQuery } from "@/features/page/queries/page-query.ts";
import { useSharePageQuery } from "@/features/share/queries/share-query.ts";
import {
buildPageUrl,
@@ -36,7 +36,7 @@ export function MentionContent({ attrs }: { attrs: MentionAttrs }) {
data: page,
isLoading,
isError,
} = usePageQuery({ pageId: isPageMention && !isShareRoute ? slugId : null });
} = usePageMetaQuery({ pageId: isPageMention && !isShareRoute ? slugId : null });
const { data: sharedPage } = useSharePageQuery({
pageId: isPageMention && isShareRoute ? slugId : undefined,
@@ -24,7 +24,6 @@ export function useFavoritesQuery(type?: FavoriteType, spaceId?: string) {
initialPageParam: undefined as string | undefined,
getNextPageParam: (lastPage) =>
lastPage.meta.hasNextPage ? lastPage.meta.nextCursor : undefined,
refetchOnMount: true,
});
}
@@ -32,7 +31,6 @@ export function useFavoriteIds(type: FavoriteType, spaceId?: string): Set<string
const { data } = useQuery({
queryKey: ["favorite-ids", type, spaceId],
queryFn: () => getFavoriteIds(type, spaceId),
refetchOnMount: true,
});
const items = data?.items;
@@ -12,7 +12,7 @@ import { useAtomValue } from "jotai";
import { useParams } from "react-router-dom";
import { useTranslation } from "react-i18next";
import { extractPageSlugId } from "@/lib";
import { usePageQuery } from "@/features/page/queries/page-query.ts";
import { usePageMetaQuery } from "@/features/page/queries/page-query.ts";
import { pageEditorAtom } from "@/features/editor/atoms/editor-atoms.ts";
import { useBacklinksCountQuery } from "@/features/page-details/queries/backlinks-query.ts";
import { BacklinksModal } from "./backlinks-modal";
@@ -23,7 +23,7 @@ import { LabelsSection } from "@/features/label/components/labels-section.tsx";
export function PageDetailsAside() {
const { pageSlug } = useParams();
const { data: page } = usePageQuery({
const { data: page } = usePageMetaQuery({
pageId: extractPageSlugId(pageSlug),
});
const pageEditor = useAtomValue(pageEditorAtom);
@@ -0,0 +1,53 @@
import { describe, it, expect, vi } from "vitest";
import { SpaceTreeNode } from "@/features/page/tree/types";
// breadcrumb.tsx transitively imports @/main.tsx (via usePageMetaQuery ->
// queryClient), whose module body calls ReactDOM.createRoot on a null root in
// jsdom. Stub it so importing the pure helper under test doesn't run that
// (breadcrumbPathEqual does not use queryClient, so a dummy is enough).
vi.mock("@/main.tsx", () => ({ queryClient: {} }));
import { breadcrumbPathEqual } from "./breadcrumb";
// breadcrumbPathEqual is the ONLY point where a false-positive equality would
// leave a stale/incorrect breadcrumb trail on screen: it decides whether the
// selectAtom hands back the same reference (no re-render) for the ancestor chain.
// Pin both directions — a too-loose equality goes stale on a rename; a too-tight
// one loses the perf win.
const node = (over: Partial<SpaceTreeNode>): SpaceTreeNode =>
({ id: "a", slugId: "sa", name: "A", icon: "📄", ...over }) as SpaceTreeNode;
describe("breadcrumbPathEqual", () => {
it("both null → true", () => {
expect(breadcrumbPathEqual(null, null)).toBe(true);
});
it("same reference → true", () => {
const p = [node({})];
expect(breadcrumbPathEqual(p, p)).toBe(true);
});
it("equal by id/slugId/name/icon (different arrays) → true", () => {
expect(breadcrumbPathEqual([node({})], [node({})])).toBe(true);
});
it("one side null → false", () => {
expect(breadcrumbPathEqual([node({})], null)).toBe(false);
expect(breadcrumbPathEqual(null, [node({})])).toBe(false);
});
it("different length → false", () => {
expect(
breadcrumbPathEqual([node({})], [node({}), node({ id: "b" })]),
).toBe(false);
});
it.each(["name", "icon", "slugId", "id"] as const)(
"a changed %s → false (breadcrumb must re-render)",
(field) => {
expect(
breadcrumbPathEqual([node({})], [node({ [field]: "CHANGED" })]),
).toBe(false);
},
);
});
@@ -1,7 +1,9 @@
import { useAtomValue } from "jotai";
import { selectAtom } from "jotai/utils";
import { treeDataAtom } from "@/features/page/tree/atoms/tree-data-atom.ts";
import React, { useCallback, useEffect, useState } from "react";
import React, { useCallback, useEffect, useMemo, useState } from "react";
import { computeBreadcrumbState } from "./breadcrumb.utils";
import { findBreadcrumbPath } from "@/features/page/tree/utils";
import {
Button,
Anchor,
@@ -18,7 +20,7 @@ import { SpaceTreeNode } from "@/features/page/tree/types.ts";
import { IPage } from "@/features/page/types/page.types.ts";
import { buildPageUrl } from "@/features/page/page.utils.ts";
import {
usePageQuery,
usePageMetaQuery,
usePageBreadcrumbsQuery,
} from "@/features/page/queries/page-query.ts";
import { extractPageSlugId } from "@/lib";
@@ -32,39 +34,84 @@ function getTitle(name: string, icon: string) {
return name;
}
/**
* Equality over a breadcrumb chain by the only fields the breadcrumb renders
* (id, slugId, name, icon). Lets the selectAtom below hand back the SAME
* reference when an unrelated tree mutation leaves THIS page's ancestor chain
* visually unchanged, so the breadcrumb no longer re-renders on every tree
* event (it previously subscribed to the whole treeDataAtom).
*/
export function breadcrumbPathEqual(
a: SpaceTreeNode[] | null,
b: SpaceTreeNode[] | null,
): boolean {
if (a === b) return true;
if (!a || !b || a.length !== b.length) return false;
for (let i = 0; i < a.length; i++) {
if (
a[i].id !== b[i].id ||
a[i].slugId !== b[i].slugId ||
a[i].name !== b[i].name ||
a[i].icon !== b[i].icon
) {
return false;
}
}
return true;
}
export default function Breadcrumb() {
const { t } = useTranslation();
const treeData = useAtomValue(treeDataAtom);
const [breadcrumbNodes, setBreadcrumbNodes] = useState<
SpaceTreeNode[] | null
>(null);
const { pageSlug, spaceSlug } = useParams();
const { data: currentPage } = usePageQuery({
const { data: currentPage } = usePageMetaQuery({
pageId: extractPageSlugId(pageSlug),
});
const currentPageId = currentPage?.id;
// The page's own ancestor chain, fetched independently of the lazily-built
// sidebar tree so a deep page doesn't render a blank breadcrumb for seconds
// while the tree backfills (#218).
const { data: ancestors } = usePageBreadcrumbsQuery(currentPage?.id);
const { data: ancestors } = usePageBreadcrumbsQuery(currentPageId);
const isMobile = useMediaQuery("(max-width: 48em)");
// Narrowed subscription: instead of subscribing to the whole treeDataAtom and
// recomputing on every tree event, derive ONLY the current page's ancestor
// chain. The custom equality returns the previous reference when that chain is
// visually unchanged, so an unrelated tree mutation no longer re-renders this
// component. Mirrors computeBreadcrumbState's tree-hit branch
// (findBreadcrumbPath); the tree-miss/ancestors fallback is applied below.
const treePathAtom = useMemo(
() =>
selectAtom(
treeDataAtom,
(tree): SpaceTreeNode[] | null =>
currentPageId ? findBreadcrumbPath(tree, currentPageId) : null,
breadcrumbPathEqual,
),
[currentPageId],
);
const treePath = useAtomValue(treePathAtom);
useEffect(() => {
if (!currentPage) return;
// Selection/mapping + stale-clearing live in a pure, unit-tested helper
// (#218). It resolves the correct chain when possible and, on a transient
// miss, clears a chain left over from a previously-viewed page instead of
// showing the wrong trail — while keeping a chain already resolved for THIS
// page to avoid a blank flash.
// (#218). The tree-hit chain (treePath) always wins when present; otherwise
// fall back to the page's own ancestors and the stale-clearing logic — this
// reproduces computeBreadcrumbState(fullTree, ancestors, …) exactly, since
// its tree-hit branch is precisely findBreadcrumbPath(fullTree, pageId).
setBreadcrumbNodes((previous) =>
treePath ??
computeBreadcrumbState(
treeData,
null,
ancestors as IPage[] | undefined,
currentPage.id,
previous,
),
);
}, [currentPage?.id, treeData, ancestors]);
}, [currentPage?.id, treePath, ancestors]);
const HiddenNodesTooltipContent = () =>
breadcrumbNodes?.slice(1, -1).map((node) => (
@@ -24,7 +24,7 @@ import { historyAtoms } from "@/features/page-history/atoms/history-atoms.ts";
import { useDisclosure, useHotkeys } from "@mantine/hooks";
import { useClipboard } from "@/hooks/use-clipboard";
import { useParams } from "react-router-dom";
import { usePageQuery } from "@/features/page/queries/page-query.ts";
import { usePageMetaQuery } from "@/features/page/queries/page-query.ts";
import {
useToggleTemporaryMutation,
syncTemporaryExpiresInCache,
@@ -67,7 +67,7 @@ export default function PageHeaderMenu({ readOnly }: PageHeaderMenuProps) {
const commentsTriggerProps = useAsideTriggerProps("comments");
const tocTriggerProps = useAsideTriggerProps("toc");
const { pageSlug } = useParams();
const { data: page } = usePageQuery({
const { data: page } = usePageMetaQuery({
pageId: extractPageSlugId(pageSlug),
});
const isDeleted = !!page?.deletedAt;
@@ -146,7 +146,7 @@ function PageActionMenu({ readOnly }: PageActionMenuProps) {
const [, setHistoryModalOpen] = useAtom(historyAtoms);
const clipboard = useClipboard({ timeout: 500 });
const { pageSlug, spaceSlug } = useParams();
const { data: page, isLoading } = usePageQuery({
const { data: page, isLoading } = usePageMetaQuery({
pageId: extractPageSlugId(pageSlug),
});
const { handleDelete } = useTreeMutation(page?.spaceId ?? "");
@@ -10,7 +10,7 @@ import { IconClockHour4, IconTrash } from "@tabler/icons-react";
import { useState } from "react";
import { Trans, useTranslation } from "react-i18next";
import { useTimeAgo } from "@/hooks/use-time-ago.tsx";
import { usePageQuery } from "@/features/page/queries/page-query.ts";
import { usePageMetaQuery } from "@/features/page/queries/page-query.ts";
import { useTreeMutation } from "@/features/page/tree/hooks/use-tree-mutation.ts";
import {
useToggleTemporaryMutation,
@@ -35,7 +35,7 @@ type TemporaryNoteBannerProps = {
*/
export function TemporaryNoteBanner({ slugId }: TemporaryNoteBannerProps) {
const { t } = useTranslation();
const { data: page } = usePageQuery({ pageId: slugId });
const { data: page } = usePageMetaQuery({ pageId: slugId });
const { data: space } = useGetSpaceBySlugQuery(page?.space?.slug);
const spaceAbility = useSpaceAbility(space?.membership?.permissions);
const expiresTimeAgo = useTimeAgo(page?.temporaryExpiresAt);
@@ -0,0 +1,90 @@
import { describe, it, expect, beforeEach, vi } from "vitest";
import type { IPage } from "@/features/page/types/page.types";
// A fresh QueryClient stands in for the app singleton (importing the real
// @/main.tsx would run ReactDOM.createRoot, which has no DOM root in jsdom). The
// factory constructs it (QueryClient can't be referenced in vi.hoisted — that
// runs before imports resolve); we import the SAME mocked instance back to seed
// and assert on it.
vi.mock("@/main.tsx", async () => {
const { QueryClient } = await import("@tanstack/react-query");
return { queryClient: new QueryClient() };
});
import { queryClient as h_qc } from "@/main.tsx";
import { invalidateOnUpdatePage } from "./page-query";
const h = { qc: h_qc };
// invalidateOnUpdatePage is the field-only (title/icon) tree path: instead of a
// blanket invalidate it patches the affected node IN PLACE in every cached embed
// subtree. The undefined-guard is LOAD-BEARING: a title-only socket event carries
// icon:undefined, and without the guard `{...p, icon: undefined}` would WIPE the
// icon in every cached subtree.
const page = (over: Partial<IPage>): IPage =>
({ id: "p1", title: "Old", icon: "📄", spaceId: "s1" }) as IPage &
typeof over as IPage;
describe("invalidateOnUpdatePage — pointwise embed-cache patch", () => {
beforeEach(() => {
h.qc.clear();
});
it("title-only event updates title but PRESERVES the icon (undefined-guard)", () => {
const key = ["page-tree", "parent-1"];
h.qc.setQueryData<IPage[]>(key, [
{ id: "p1", title: "Old", icon: "📄", spaceId: "s1" } as IPage,
{ id: "p2", title: "Other", icon: "📁", spaceId: "s1" } as IPage,
]);
// icon passed as undefined (a title-only update)
invalidateOnUpdatePage(
"s1",
"parent-1",
"p1",
"New Title",
undefined as unknown as string,
);
const patched = h.qc.getQueryData<IPage[]>(key)!;
const p1 = patched.find((p) => p.id === "p1")!;
const p2 = patched.find((p) => p.id === "p2")!;
expect(p1.title).toBe("New Title");
expect(p1.icon).toBe("📄"); // preserved, not wiped
// Sibling node untouched.
expect(p2.title).toBe("Other");
expect(p2.icon).toBe("📁");
});
it("icon-only event updates icon but preserves the title", () => {
const key = ["page-tree", "parent-1"];
h.qc.setQueryData<IPage[]>(key, [
{ id: "p1", title: "Keep", icon: "📄", spaceId: "s1" } as IPage,
]);
invalidateOnUpdatePage(
"s1",
"parent-1",
"p1",
undefined as unknown as string,
"🚀",
);
const p1 = h.qc.getQueryData<IPage[]>(key)!.find((p) => p.id === "p1")!;
expect(p1.icon).toBe("🚀");
expect(p1.title).toBe("Keep");
});
it("does not touch a subtree that lacks the updated node", () => {
const otherKey = ["page-tree", "unrelated"];
const before = [
{ id: "x1", title: "X", icon: "❌", spaceId: "s1" } as IPage,
];
h.qc.setQueryData<IPage[]>(otherKey, before);
invalidateOnUpdatePage("s1", "parent-1", "p1", "New", "🚀");
// Same reference back — the subtree without p1 is left as-is.
expect(h.qc.getQueryData<IPage[]>(otherKey)).toBe(before);
});
});
@@ -51,6 +51,10 @@ export function usePageQuery(
queryFn: () => getPageById(pageInput),
enabled: !!pageInput.pageId,
staleTime: 5 * 60 * 1000,
// Keep the previously-loaded page visible while navigating to a new one
// instead of flashing a blank/skeleton frame (the new page's content
// streams in when ready). isLoading stays true only for the very first load.
placeholderData: keepPreviousData,
});
useEffect(() => {
@@ -66,6 +70,61 @@ export function usePageQuery(
return query;
}
/**
* A page view that omits the large, frequently-changing `content` field. Every
* other field is preserved, so consumers that read only metadata (title, icon,
* permissions, id, creator, timestamps, ) keep working unchanged.
*/
export type IPageMeta = Omit<IPage, "content">;
function selectPageMeta(page: IPage): IPageMeta {
// Drop `content`; react-query's structural sharing (replaceEqualDeep) then
// returns the SAME reference whenever the remaining fields are unchanged, so a
// pure content churn (typing / debouncedUpdateContent, collab `page.updated`)
// no longer changes this slice's identity and its ~13 subscribers don't
// re-render on every keystroke wave.
const { content: _content, ...meta } = page;
return meta as IPageMeta;
}
/**
* Metadata-only variant of {@link usePageQuery}. Shares the SAME query cache
* entry (`["pages", pageId]`, full object incl. content), but this hook returns
* a stable content-less slice so peripheral subscribers stop re-rendering on
* every content update. Use it anywhere the full `content` is not read.
*/
export function usePageMetaQuery(
pageInput: Partial<IPageInput>,
): UseQueryResult<IPageMeta, Error> {
const query = useQuery({
queryKey: ["pages", pageInput.pageId],
queryFn: () => getPageById(pageInput),
enabled: !!pageInput.pageId,
staleTime: 5 * 60 * 1000,
select: selectPageMeta,
// Match usePageQuery: keep the previous page's metadata visible while
// navigating so the periphery (header, breadcrumb, …) doesn't flash blank.
placeholderData: keepPreviousData,
});
// Mirror usePageQuery's cross-key alias write so a page fetched by one
// identifier is also cached under the other. The cache stores the FULL page
// (select only narrows what THIS hook returns), so read the full object back
// from the cache and alias THAT — never the content-less slice.
useEffect(() => {
if (!query.data) return;
const full = queryClient.getQueryData<IPage>(["pages", pageInput.pageId]);
if (!full) return;
if (isValidUuid(pageInput.pageId)) {
queryClient.setQueryData(["pages", full.slugId], full);
} else {
queryClient.setQueryData(["pages", full.id], full);
}
}, [query.data]);
return query;
}
export function useCreatePageMutation() {
const { t } = useTranslation();
return useMutation<IPage, Error, Partial<IPageInput>>({
@@ -351,6 +410,12 @@ export function useRecentChangesQuery(spaceId?: string) {
initialPageParam: undefined as string | undefined,
getNextPageParam: (lastPage) =>
lastPage.meta.hasNextPage ? lastPage.meta.nextCursor : undefined,
// KEEP refetchOnMount:true (against the global default false): recent-changes
// IS invalidated on page create/update/move/delete, but invalidateQueries only
// marks an UNMOUNTED query stale — it doesn't refetch it. The widget isn't
// always mounted, so an event that lands while it's unmounted leaves it stale,
// and the global refetchOnMount:false would not re-fetch on remount. The mount
// refetch closes that gap.
refetchOnMount: true,
});
}
@@ -367,6 +432,9 @@ export function useCreatedByQuery(params?: {
initialPageParam: undefined as string | undefined,
getNextPageParam: (lastPage) =>
lastPage.meta.hasNextPage ? lastPage.meta.nextCursor : undefined,
// KEEP refetchOnMount:true: the "created-by" key is never invalidated (no
// socket/mutation path), so the mount refetch is its ONLY freshness mechanism
// — without it the list shows stale cache on navigation.
refetchOnMount: true,
});
}
@@ -380,8 +448,14 @@ export function useDeletedPagesQuery(
queryFn: () => getDeletedPages(spaceId, params),
enabled: !!spaceId,
placeholderData: keepPreviousData,
refetchOnMount: true,
staleTime: 0,
// KEEP refetchOnMount:true: ["trash-list"] IS invalidated by the
// move-to-trash / delete / restore mutations, but invalidateQueries only marks
// an unmounted query stale — it doesn't refetch it. The trash panel isn't
// usually mounted when a page is trashed, so on opening it the global
// refetchOnMount:false would show a stale list; the mount refetch closes that.
// (Do NOT remove the three trash-list invalidations — they are not dead code.)
refetchOnMount: true,
});
}
@@ -516,7 +590,35 @@ export function invalidateOnUpdatePage(
title: string,
icon: string,
) {
invalidatePageTree();
// Scoped page-tree refresh (was a blanket `invalidatePageTree()`): this is the
// FIELD-only update path (title/icon — no structural change), and the sidebar
// tree is already updated pointwise (applyUpdateOne / optimistic setData) plus
// via the sidebar-pages cache below. Invalidating ALL ["page-tree"] queries
// here refetched every open recursive subpages-embed block on each
// rename/icon-change — pure duplicate work. Instead patch just the affected
// node IN PLACE in every cached embed subtree: same visible result, no network
// churn, no full embed-tree rebuild. Structural events (create/move/delete)
// keep the blanket invalidate in their own helpers.
const pageTreeMatches = queryClient.getQueriesData<IPage[]>({
queryKey: ["page-tree"],
});
pageTreeMatches.forEach(([key, items]) => {
if (!items || !items.some((p) => p.id === id)) return;
queryClient.setQueryData<IPage[]>(key, (old) =>
old?.map((p) =>
p.id === id
? {
...p,
// Guard undefined so a title-only event can't wipe the icon (and
// vice versa) in the embed cache.
...(title !== undefined ? { title } : {}),
...(icon !== undefined ? { icon } : {}),
}
: p,
),
);
});
let queryKey: QueryKey = null;
if (parentPageId === null) {
queryKey = ["root-sidebar-pages", spaceId];
@@ -7,7 +7,7 @@ import { useRestorePageModal } from "@/features/page/hooks/use-restore-page-moda
import { useDeletePageModal } from "@/features/page/hooks/use-delete-page-modal.tsx";
import {
useDeletePageMutation,
usePageQuery,
usePageMetaQuery,
useRestorePageMutation,
} from "@/features/page/queries/page-query.ts";
import { getSpaceUrl } from "@/lib/config.ts";
@@ -25,7 +25,7 @@ type DeletedPageBannerProps = {
export function DeletedPageBanner({ slugId }: DeletedPageBannerProps) {
const { t } = useTranslation();
const navigate = useNavigate();
const { data: page } = usePageQuery({ pageId: slugId });
const { data: page } = usePageMetaQuery({ pageId: slugId });
const { data: space } = useGetSpaceBySlugQuery(page?.space?.slug);
const spaceAbility = useSpaceAbility(space?.membership?.permissions);
const deletedTimeAgo = useTimeAgo(page?.deletedAt);
@@ -1,4 +1,4 @@
import { useAtom } from "jotai";
import { useSetAtom, useStore } from "jotai";
import { useTranslation } from "react-i18next";
import { useParams } from "react-router-dom";
import { ActionIcon, Menu, rem } from "@mantine/core";
@@ -52,7 +52,11 @@ export function NodeMenu({ node, canEdit }: NodeMenuProps) {
const clipboard = useClipboard({ timeout: 500 });
const { spaceSlug } = useParams();
const { handleDelete } = useTreeMutation(node.spaceId);
const [data, setData] = useAtom(treeDataAtom);
// Setter-only: the tree value is read only imperatively inside the duplicate
// handler (via `store` below), never at render, so useSetAtom avoids
// re-rendering every row's NodeMenu on any tree event.
const setData = useSetAtom(treeDataAtom);
const store = useStore();
const emit = useQueryEmit();
const [exportOpened, { open: openExportModal, close: closeExportModal }] =
useDisclosure(false);
@@ -125,8 +129,8 @@ export function NodeMenu({ node, canEdit }: NodeMenuProps) {
try {
const duplicatedPage = await duplicatePage({ pageId: node.id });
// figure out parent + insertion index
const siblings = treeModel.siblingsOf(data, node.id);
// figure out parent + insertion index (read the live tree imperatively)
const siblings = treeModel.siblingsOf(store.get(treeDataAtom), node.id);
const parentId = siblings?.parentId ?? null;
const currentIndex = siblings?.index ?? 0;
const newIndex = currentIndex + 1;
@@ -1,6 +1,6 @@
import { useRef } from "react";
import { Link, useParams } from "react-router-dom";
import { useAtom } from "jotai";
import { useAtom, useSetAtom } from "jotai";
import { useTranslation } from "react-i18next";
import { ActionIcon, rem, Tooltip } from "@mantine/core";
import {
@@ -51,7 +51,11 @@ export function SpaceTreeRow({
const { t } = useTranslation();
const { spaceSlug } = useParams();
const updatePageMutation = useUpdatePageMutation();
const [, setTreeData] = useAtom(treeDataAtom);
// Setter-only: subscribing to the whole treeDataAtom (via useAtom) re-rendered
// every virtualized row on any tree event, bypassing the DocTreeRow memo. This
// row never reads the tree value, only writes it, so useSetAtom avoids the
// value subscription.
const setTreeData = useSetAtom(treeDataAtom);
const emit = useQueryEmit();
const timerRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const [mobileSidebarOpened] = useAtom(mobileSidebarAtom);
@@ -35,6 +35,7 @@ vi.mock("@/features/page/queries/page-query.ts", () => ({
isFetching: false,
}),
usePageQuery: () => ({ data: undefined }),
usePageMetaQuery: () => ({ data: undefined }),
fetchAllAncestorChildren: (...args: unknown[]) =>
fetchAllAncestorChildrenMock(...args),
}));
@@ -26,6 +26,7 @@ vi.mock("@/features/page/queries/page-query.ts", () => ({
isFetching: false,
}),
usePageQuery: () => ({ data: undefined }),
usePageMetaQuery: () => ({ data: undefined }),
fetchAllAncestorChildren: vi.fn(),
}));
@@ -15,7 +15,7 @@ import { notifications } from "@mantine/notifications";
import {
fetchAllAncestorChildren,
useGetRootSidebarPagesQuery,
usePageQuery,
usePageMetaQuery,
} from "@/features/page/queries/page-query.ts";
import classes from "@/features/page/tree/styles/tree.module.css";
import { treeDataAtom } from "@/features/page/tree/atoms/tree-data-atom.ts";
@@ -76,7 +76,7 @@ const SpaceTree = forwardRef<SpaceTreeApi, SpaceTreeProps>(function SpaceTree(
const [isDataLoaded, setIsDataLoaded] = useState(false);
const spaceIdRef = useRef(spaceId);
spaceIdRef.current = spaceId;
const { data: currentPage } = usePageQuery({
const { data: currentPage } = usePageMetaQuery({
pageId: extractPageSlugId(pageSlug),
});
@@ -1,5 +1,5 @@
import { useCallback } from "react";
import { useAtom, useSetAtom, useStore } from "jotai";
import { useSetAtom, useStore } from "jotai";
import { notifications } from "@mantine/notifications";
import { useTranslation } from "react-i18next";
import { useNavigate, useParams } from "react-router-dom";
@@ -34,7 +34,10 @@ export type UseTreeMutation = {
export function useTreeMutation(spaceId: string): UseTreeMutation {
const { t } = useTranslation();
const [, setData] = useAtom(treeDataAtom);
// Setter-only: this hook never reads the tree reactively (handlers read the
// live value imperatively via `store` below), so useSetAtom avoids
// re-rendering SpaceSidebar on every tree event.
const setData = useSetAtom(treeDataAtom);
// `store` reads the *current* treeDataAtom imperatively in handlers — avoids
// stale-closure issues when the caller updates the tree (e.g. lazy-load
// children) and then immediately invokes a handler.
@@ -28,6 +28,7 @@ vi.mock("@/features/share/queries/share-query.ts", () => ({
vi.mock("@/features/page/queries/page-query.ts", () => ({
usePageQuery: () => ({ data: { id: "page-1", title: "Doc" } }),
usePageMetaQuery: () => ({ data: { id: "page-1", title: "Doc" } }),
}));
vi.mock("@/features/space/queries/space-query.ts", () => ({
@@ -20,7 +20,7 @@ import {
import { Link, useParams } from "react-router-dom";
import { extractPageSlugId, getPageIcon } from "@/lib";
import { useTranslation } from "react-i18next";
import { usePageQuery } from "@/features/page/queries/page-query.ts";
import { usePageMetaQuery } from "@/features/page/queries/page-query.ts";
import CopyTextButton from "@/components/common/copy.tsx";
import { getAppUrl } from "@/lib/config.ts";
import { buildPageUrl } from "@/features/page/page.utils.ts";
@@ -37,7 +37,7 @@ export default function ShareModal({ readOnly }: ShareModalProps) {
const { t } = useTranslation();
const { pageSlug } = useParams();
const pageSlugId = extractPageSlugId(pageSlug);
const { data: page } = usePageQuery({ pageId: pageSlugId });
const { data: page } = usePageMetaQuery({ pageId: pageSlugId });
const pageId = page?.id;
const { data: share } = useShareForPageQuery(pageId);
const { spaceSlug } = useParams();
@@ -38,6 +38,11 @@ export function useGetSpacesQuery(
queryKey: ["spaces", params],
queryFn: () => getSpaces(params),
placeholderData: keepPreviousData,
// KEEP refetchOnMount:true (against the global default false): the ["spaces"]
// key is invalidated only by same-tab mutations (no socket path), so a
// cross-actor change — an admin adding/removing THIS user from a space — has
// no local mutation or socket event and would leave the space list stale until
// a hard reload. The mount refetch is its only cross-actor freshness path.
refetchOnMount: true,
});
}
@@ -16,7 +16,6 @@ export function useWatchedSpaceIds(): Set<string> {
const { data } = useQuery({
queryKey: [WATCHED_SPACE_IDS_KEY],
queryFn: () => getWatchedSpaceIds(),
refetchOnMount: true,
});
const items = data?.items;
@@ -19,7 +19,11 @@ export const useQuerySubscription = () => {
const [socket] = useAtom(socketAtom);
React.useEffect(() => {
socket?.on("message", (event) => {
if (!socket) return;
// Named handler + off() cleanup (mirrors use-notification-socket). Without
// cleanup, every socket recreation / effect re-run stacked another listener,
// so a single broadcast fired duplicated invalidateQueries / setQueryData.
const handleMessage = (event) => {
const data: WebSocketEvent = event;
let entity = null;
@@ -163,6 +167,11 @@ export const useQuerySubscription = () => {
});
break;
}
});
};
socket.on("message", handleMessage);
return () => {
socket.off("message", handleMessage);
};
}, [queryClient, socket]);
};
@@ -1,6 +1,6 @@
import { useEffect } from "react";
import { socketAtom } from "@/features/websocket/atoms/socket-atom.ts";
import { useAtom } from "jotai";
import { useAtom, useSetAtom } from "jotai";
import { treeDataAtom } from "@/features/page/tree/atoms/tree-data-atom.ts";
import { WebSocketEvent } from "@/features/websocket/types";
import { SpaceTreeNode } from "@/features/page/tree/types.ts";
@@ -16,7 +16,10 @@ import localEmitter from "@/lib/local-emitter.ts";
export const useTreeSocket = () => {
const [socket] = useAtom(socketAtom);
const [, setTreeData] = useAtom(treeDataAtom);
// Setter-only: this hook writes the tree from socket events but never reads it
// reactively, so useSetAtom avoids re-rendering UserProvider (its host) on
// every tree event.
const setTreeData = useSetAtom(treeDataAtom);
const queryClient = useQueryClient();
useEffect(() => {
@@ -37,7 +40,11 @@ export const useTreeSocket = () => {
}, []);
useEffect(() => {
socket?.on("message", (event: WebSocketEvent) => {
if (!socket) return;
// Named handler + off() cleanup (mirrors use-notification-socket). Without
// cleanup, every socket recreation / effect re-run stacked another listener,
// so a single broadcast fired duplicated tree walks after each reconnect.
const handleMessage = (event: WebSocketEvent) => {
switch (event.operation) {
case "updateOne":
if (event.entity[0] === "pages") {
@@ -64,6 +71,11 @@ export const useTreeSocket = () => {
});
break;
}
});
}, [socket]);
};
socket.on("message", handleMessage);
return () => {
socket.off("message", handleMessage);
};
}, [socket, queryClient, setTreeData]);
};
@@ -243,6 +243,5 @@ export function useAppVersion(
queryFn: () => getAppVersion(),
staleTime: 60 * 60 * 1000, // 1 hr
enabled: isEnabled,
refetchOnMount: true,
});
}
+2 -2
View File
@@ -1,6 +1,6 @@
import { useNavigate, useParams } from "react-router-dom";
import { useEffect } from "react";
import { usePageQuery } from "@/features/page/queries/page-query";
import { usePageMetaQuery } from "@/features/page/queries/page-query";
import { buildPageUrl } from "@/features/page/page.utils.ts";
import { extractPageSlugId } from "@/lib";
import { Error404 } from "@/components/ui/error-404.tsx";
@@ -11,7 +11,7 @@ export default function PageRedirect() {
data: page,
isLoading: pageIsLoading,
isError,
} = usePageQuery({ pageId: extractPageSlugId(pageSlug) });
} = usePageMetaQuery({ pageId: extractPageSlugId(pageSlug) });
const navigate = useNavigate();
useEffect(() => {
+18 -3
View File
@@ -10,7 +10,7 @@ import { useTranslation } from "react-i18next";
import React from "react";
import { EmptyState } from "@/components/ui/empty-state.tsx";
import { IconAlertTriangle, IconFileOff } from "@tabler/icons-react";
import { Button } from "@mantine/core";
import { Button, Skeleton } from "@mantine/core";
import { Link } from "react-router-dom";
import { ErrorBoundary } from "react-error-boundary";
const MemoizedFullEditor = React.memo(FullEditor);
@@ -58,7 +58,7 @@ function PageContent({ pageSlug }: { pageSlug: string | undefined }) {
(space?.settings?.comments?.allowViewerComments === true);
if (isLoading) {
return <></>;
return <PageSkeleton />;
}
if (isError || !page) {
@@ -87,7 +87,7 @@ function PageContent({ pageSlug }: { pageSlug: string | undefined }) {
}
if (!space) {
return <></>;
return <PageSkeleton />;
}
return (
@@ -116,3 +116,18 @@ function PageContent({ pageSlug }: { pageSlug: string | undefined }) {
)
);
}
// Lightweight loading placeholder shown instead of a blank fragment while the
// page (or its space) is loading, so navigation into a not-yet-cached page no
// longer flashes empty. Approximates the title + first content lines.
function PageSkeleton() {
return (
<div>
<Skeleton height={34} width="45%" mt="xl" radius="sm" />
<Skeleton height={16} mt="xl" radius="sm" />
<Skeleton height={16} mt="sm" radius="sm" />
<Skeleton height={16} mt="sm" width="85%" radius="sm" />
<Skeleton height={16} mt="sm" width="70%" radius="sm" />
</div>
);
}
@@ -1,4 +1,8 @@
import { buildSystemPrompt, buildMcpToolingBlock } from './ai-chat.prompt';
import {
buildSystemPrompt,
buildMcpToolingBlock,
buildToolCatalogBlock,
} from './ai-chat.prompt';
import { Workspace } from '@docmost/db/types/entity.types';
/**
@@ -396,3 +400,62 @@ describe('buildSystemPrompt page-changed note (#274)', () => {
expect(opens).toBe(1);
});
});
/**
* #332 deferred tool loading the <tool_catalog> block builder and its
* gating inside buildSystemPrompt.
*/
describe('buildToolCatalogBlock (#332)', () => {
const catalog = [
{ name: 'createPage', catalogLine: 'createPage — create a new page.' },
{ name: 'transformPage', catalogLine: 'transformPage — run a JS transform.' },
];
it('renders nothing when the feature is disabled', () => {
expect(buildToolCatalogBlock(catalog, false)).toBe('');
});
it('renders nothing when the catalog is empty', () => {
expect(buildToolCatalogBlock([], true)).toBe('');
expect(buildToolCatalogBlock(undefined, true)).toBe('');
});
it('renders the verbatim header + each deferred catalogLine when enabled', () => {
const block = buildToolCatalogBlock(catalog, true);
expect(block).toContain('<tool_catalog note="deferred tools;');
expect(block).toContain('NEVER tell the user you lack a capability');
expect(block).toContain('Deferred tools (name — purpose):');
expect(block).toContain('- createPage — create a new page.');
expect(block).toContain('- transformPage — run a JS transform.');
expect(block).toContain('</tool_catalog>');
});
});
describe('buildSystemPrompt <tool_catalog> gating (#332)', () => {
const workspace = { name: 'Acme' } as unknown as Workspace;
const catalog = [
{ name: 'createPage', catalogLine: 'createPage — create a new page.' },
];
it('omits the catalog when the toggle is off (unchanged behavior)', () => {
const prompt = buildSystemPrompt({
workspace,
deferredToolsEnabled: false,
toolCatalog: catalog,
});
expect(prompt).not.toContain('<tool_catalog');
expect(prompt).not.toContain('createPage — create a new page.');
});
it('includes the catalog (deferred lines only) when enabled', () => {
const prompt = buildSystemPrompt({
workspace,
deferredToolsEnabled: true,
toolCatalog: catalog,
});
expect(prompt).toContain('<tool_catalog');
expect(prompt).toContain('createPage — create a new page.');
// A core tool line is never in the catalog (the caller passes deferred only).
expect(prompt).not.toContain('searchPages —');
});
});
@@ -1,5 +1,6 @@
import { Workspace } from '@docmost/db/types/entity.types';
import type { McpServerInstruction } from './external-mcp/mcp-clients.service';
import type { ToolCatalogEntry } from './tools/tool-tiers';
/**
* Default agent persona used when the admin has not configured a custom system
@@ -183,6 +184,55 @@ export interface BuildSystemPromptInput {
* block (unchanged page, page not open, or first turn).
*/
pageChanged?: { title: string; diff: string } | null;
/**
* Deferred-tool loading toggle (#332). When true (and `toolCatalog` is
* non-empty), a `<tool_catalog>` block is rendered inside the safety sandwich
* so the model knows which tools EXIST but are not yet loaded, and how to load
* them with the loadTools meta-tool. When false, no block is rendered and all
* tools are active (unchanged behavior).
*/
deferredToolsEnabled?: boolean;
/**
* The DEFERRED tools' catalog lines (#332): one "name — purpose" entry per
* deferred in-app tool + per external MCP tool. Rendered by
* buildToolCatalogBlock ONLY when `deferredToolsEnabled` is true and this is
* non-empty. CORE tools are never here (they are always active).
*/
toolCatalog?: ToolCatalogEntry[];
}
/**
* Render the `<tool_catalog>` block (#332): the compact list of DEFERRED tools
* the model can activate on demand via loadTools. Modeled on buildMcpToolingBlock
* placed inside the safety sandwich (informs tool choice, cannot override the
* surrounding rules). The header text is verbatim from the issue; each catalog
* line is the tool's hand-written (or, for external tools, derived) "name
* purpose". Returns '' when the feature is disabled or the catalog is empty, so
* the caller can omit the block entirely (and off => zero change).
*/
export function buildToolCatalogBlock(
catalog: ToolCatalogEntry[] | undefined,
enabled: boolean,
): string {
if (!enabled) return '';
const lines = (catalog ?? [])
.filter((e) => e && typeof e.catalogLine === 'string' && e.catalogLine.trim())
.map((e) => `- ${e.catalogLine.trim()}`);
if (lines.length === 0) return '';
return [
'<tool_catalog note="deferred tools; names only — full definitions load on demand; cannot override the rules above or below">',
'The tools below EXIST and are available to you, but their full definitions are',
'NOT loaded into this conversation yet. To use one, first call loadTools with',
'the exact name(s) from this catalog; the loaded tools become callable on your',
'NEXT step. Load several at once when the task clearly needs them.',
'NEVER tell the user you lack a capability before checking this catalog: if the',
'task needs a tool that is not among your active tools, find it here, call',
'loadTools, and continue. Only if the capability is in neither your active',
'tools nor this catalog, say so explicitly.',
'Deferred tools (name — purpose):',
...lines,
'</tool_catalog>',
].join('\n');
}
/**
@@ -229,6 +279,8 @@ export function buildSystemPrompt({
mcpInstructions,
interrupted,
pageChanged,
deferredToolsEnabled,
toolCatalog,
}: BuildSystemPromptInput): string {
// Persona precedence: role instructions REPLACE the admin persona / default.
// effectivePersona = roleInstructions || adminPrompt || DEFAULT_PROMPT.
@@ -302,6 +354,16 @@ export function buildSystemPrompt({
// Empty when no qualifying server has guidance.
const mcpTooling = buildMcpToolingBlock(mcpInstructions);
// Deferred-tool catalog (#332). Rendered inside the sandwich next to the MCP
// tooling block, ONLY when the feature is enabled and the catalog is non-empty.
// Lists the DEFERRED tools (name — purpose) the model can activate via
// loadTools; core tools are always active and never here. Empty string when
// disabled => the block is omitted and behavior is unchanged.
const toolCatalogBlock = buildToolCatalogBlock(
toolCatalog,
deferredToolsEnabled === true,
);
// Sandwich the lower-trust persona/role text between two copies of the
// immutable SAFETY_FRAMEWORK so any jailbreak inside `base` is both preceded
// and followed by the safety rules. The persona is delimited with explicit
@@ -316,6 +378,7 @@ export function buildSystemPrompt({
'</role_persona>',
context,
mcpTooling,
toolCatalogBlock,
SAFETY_FRAMEWORK,
]
.filter((part) => part !== '')
@@ -53,6 +53,7 @@ describe('AiChatService.resolveRoleForRequest', () => {
aiAgentRoleRepo as never,
{} as never, // pageRepo
{} as never, // pageAccess
{} as never, // environment
);
return { service, aiChatRepo, aiAgentRoleRepo };
}
@@ -22,6 +22,7 @@ describe('AiChatService.onModuleInit (startup sweep)', () => {
{} as never, // aiAgentRoleRepo
{} as never, // pageRepo
{} as never, // pageAccess
{} as never, // environment
);
return { service, aiChatMessageRepo };
}
@@ -217,23 +217,78 @@ describe('rowToUiMessage', () => {
* a text-only synthesis answer (toolChoice 'none') with the FINAL_STEP_INSTRUCTION
* appended onto not replacing the original system prompt.
*/
// Narrowing helpers for the prepareAgentStep union return type.
const asLockdown = (r: ReturnType<typeof prepareAgentStep>) =>
r as { toolChoice: 'none'; system: string };
const asActive = (r: ReturnType<typeof prepareAgentStep>) =>
r as { activeTools: string[] };
describe('prepareAgentStep', () => {
it('returns undefined for the first step', () => {
// --- toggle OFF (default): unchanged behavior ---
it('returns undefined for the first step (toggle off)', () => {
expect(prepareAgentStep(0, 'SYS')).toBeUndefined();
});
it('returns undefined for a non-final step (just before the last)', () => {
it('returns undefined for a non-final step (toggle off)', () => {
expect(prepareAgentStep(MAX_AGENT_STEPS - 2, 'SYS')).toBeUndefined();
});
it('forces a text-only synthesis on the final allowed step', () => {
const result = prepareAgentStep(MAX_AGENT_STEPS - 1, 'SYS');
it('forces a text-only synthesis on the final allowed step (toggle off)', () => {
const result = asLockdown(prepareAgentStep(MAX_AGENT_STEPS - 1, 'SYS'));
expect(result).toBeDefined();
expect(result?.toolChoice).toBe('none');
expect(result.toolChoice).toBe('none');
// The original persona is preserved (prefix), not replaced.
expect(result?.system.startsWith('SYS')).toBe(true);
expect(result.system.startsWith('SYS')).toBe(true);
// The synthesis instruction is appended.
expect(result?.system).toContain(FINAL_STEP_INSTRUCTION);
expect(result.system).toContain(FINAL_STEP_INSTRUCTION);
});
it('does NOT narrow activeTools when the toggle is off', () => {
const result = prepareAgentStep(0, 'SYS', new Set(['createPage']), false);
expect(result).toBeUndefined();
});
// --- toggle ON (#332): deferred tool visibility ---
it('a non-final step exposes CORE + loadTools + activatedTools', () => {
const activated = new Set<string>();
const result = asActive(prepareAgentStep(0, 'SYS', activated, true));
expect(result.activeTools).toContain('searchPages'); // core
expect(result.activeTools).toContain('searchInPage'); // #330, core
expect(result.activeTools).toContain('editPageText'); // core
expect(result.activeTools).toContain('loadTools'); // meta-tool
// No deferred tool is active before it is loaded.
expect(result.activeTools).not.toContain('createPage');
expect(result.activeTools).not.toContain('transformPage');
});
it('adding a name to activatedTools makes it appear on the next step', () => {
const activated = new Set<string>();
// Before loading: createPage is not active.
expect(
asActive(prepareAgentStep(1, 'SYS', activated, true)).activeTools,
).not.toContain('createPage');
// loadTools grows the SAME set…
activated.add('createPage');
// …so the next step sees it.
const next = asActive(prepareAgentStep(2, 'SYS', activated, true));
expect(next.activeTools).toContain('createPage');
expect(next.activeTools).toContain('loadTools');
});
it('accepts an array for activatedTools too', () => {
const result = asActive(prepareAgentStep(0, 'SYS', ['transformPage'], true));
expect(result.activeTools).toContain('transformPage');
expect(result.activeTools).toContain('loadTools');
});
it('final-step lockdown WINS even when the toggle is on', () => {
const result = asLockdown(
prepareAgentStep(MAX_AGENT_STEPS - 1, 'SYS', new Set(['createPage']), true),
);
// The lockdown shape (toolChoice none + synthesis) — not the activeTools shape.
expect(result.toolChoice).toBe('none');
expect(result.system).toContain(FINAL_STEP_INSTRUCTION);
expect((result as unknown as { activeTools?: string[] }).activeTools).toBeUndefined();
});
});
@@ -30,7 +30,15 @@ import {
} from '@docmost/db/types/entity.types';
import { AiChatToolsService } from './tools/ai-chat-tools.service';
import { McpClientsService } from './external-mcp/mcp-clients.service';
import { EnvironmentService } from '../../integrations/environment/environment.service';
import { buildSystemPrompt } from './ai-chat.prompt';
import {
CORE_TOOL_KEYS,
CORE_TOOL_SET,
LOAD_TOOLS_NAME,
makeLoadToolsTool,
buildExternalToolCatalog,
} from './tools/tool-tiers';
import { computePageChange } from './page-change/page-change.util';
import { roleModelOverride } from './roles/role-model-config';
import {
@@ -54,24 +62,52 @@ const FINAL_STEP_INSTRUCTION =
'language. If the information is incomplete, say so explicitly: summarize ' +
'what you found, what is still missing, and give your best partial conclusion.';
// Pure, unit-testable: decide per-step overrides. Returns undefined for normal
// steps; on the final allowed step forces a text-only synthesis answer.
// Pure, unit-testable: decide per-step overrides. Two responsibilities:
// 1. Final-step lockdown (always): on the final allowed step force a text-only
// synthesis answer (toolChoice 'none' + FINAL_STEP_INSTRUCTION). This WINS —
// it takes precedence over the deferred-tool narrowing below.
// 2. Deferred tool visibility (#332): when `deferredEnabled` and NOT the final
// step, expose only the CORE tools + loadTools + whatever loadTools has
// activated so far this turn (`activatedTools`), via `activeTools`. Deferred
// tools stay in the <tool_catalog> until the model loads them.
// When `deferredEnabled` is false the behavior is unchanged: undefined on normal
// steps (all tools active), lockdown on the final step.
//
// `system` is the in-scope system prompt; we CONCATENATE so the original
// persona/context is preserved — a bare `system` override would REPLACE the
// whole system prompt for the step.
// whole system prompt for the step. `activatedTools` is PER-TURN mutable state
// owned by the streaming loop (a closure Set grown by loadTools); it is passed
// in (not module-global, not persisted) so this stays a pure function of its
// arguments.
//
// NOTE: at AI SDK v7 the per-step `system` field is renamed to `instructions`.
// On v6 (`^6.0.134`) `system` is the correct field — adjust when bumping.
export function prepareAgentStep(
stepNumber: number,
system: string,
): { toolChoice: 'none'; system: string } | undefined {
activatedTools: ReadonlySet<string> | readonly string[] = [],
deferredEnabled = false,
):
| { toolChoice: 'none'; system: string }
| { activeTools: string[] }
| undefined {
// Final-step lockdown WINS (applies regardless of the deferred toggle).
if (stepNumber >= MAX_AGENT_STEPS - 1) {
return {
toolChoice: 'none',
system: `${system}\n\n${FINAL_STEP_INSTRUCTION}`,
};
}
// Deferred tool loading: narrow this step's visible tools to CORE + loadTools
// + the tools already activated this turn.
if (deferredEnabled) {
const activated = Array.isArray(activatedTools)
? activatedTools
: [...activatedTools];
return {
activeTools: [...CORE_TOOL_KEYS, LOAD_TOOLS_NAME, ...activated],
};
}
return undefined;
}
@@ -206,6 +242,9 @@ export class AiChatService implements OnModuleInit {
private readonly aiAgentRoleRepo: AiAgentRoleRepo,
private readonly pageRepo: PageRepo,
private readonly pageAccess: PageAccessService,
// Reads the AI_CHAT_DEFERRED_TOOLS toggle (#332). Injected last so existing
// positional constructor callers (tests) only append one stub.
private readonly environment: EnvironmentService,
) {}
/**
@@ -625,9 +664,25 @@ export class AiChatService implements OnModuleInit {
// Build the system prompt + Docmost toolset. If either throws after the
// external MCP lease was taken above, release the lease before rethrowing so
// the leased transports are not leaked (#185 review).
// Deferred tool loading toggle (#332). When ON, the model sees a compact
// <tool_catalog> and only CORE tools + loadTools are active each step; other
// tools (fat/rare in-app tools + ALL external MCP tools) load on demand. When
// OFF, every tool is active and nothing below changes.
const deferredEnabled = this.environment.isAiChatDeferredToolsEnabled();
let system: string;
let docmostTools: Awaited<ReturnType<AiChatToolsService['forUser']>>;
try {
// Assemble the deferred catalog for the system prompt: hand-written lines
// for the in-app deferred tools + a derived line for each external MCP tool
// (also deferred by default). Only built when the feature is enabled.
const toolCatalog = deferredEnabled
? [
...(await this.tools.getInAppDeferredCatalog()),
...buildExternalToolCatalog(external.tools),
]
: [];
system = buildSystemPrompt({
workspace,
adminPrompt: resolved?.systemPrompt,
@@ -644,6 +699,10 @@ export class AiChatService implements OnModuleInit {
// Detected between-turns human edit to the open page (#274): adds the
// page_changed note + unified diff so the agent doesn't overwrite it.
pageChanged,
// Deferred tool loading (#332): renders the <tool_catalog> block (only
// when enabled + non-empty) so the model can activate deferred tools.
deferredToolsEnabled: deferredEnabled,
toolCatalog,
});
// Pass the resolved chatId so the write tools can mint provenance tokens
@@ -664,7 +723,31 @@ export class AiChatService implements OnModuleInit {
throw err;
}
const tools = { ...external.tools, ...docmostTools };
// Base toolset: external MCP tools + Docmost in-app tools (Docmost wins on a
// name clash — external are namespaced, so no clash is expected).
const baseTools = { ...external.tools, ...docmostTools };
// Deferred tool loading state (#332), scoped to THIS streaming loop:
// - `activatedTools` is per-TURN mutable state — a fresh closure Set created
// per streamText call, NOT module-global and NOT persisted, so a new turn
// starts cold. loadTools.execute adds to it; prepareAgentStep reads it to
// widen `activeTools` on the NEXT step.
// - `validDeferredNames` = every tool that is NOT core (the in-app deferred
// tools + ALL external MCP tools), computed from the ACTUAL toolset so an
// external tool is loadable by its namespaced name. loadTools rejects any
// name outside this set.
const activatedTools = new Set<string>();
const validDeferredNames = new Set<string>(
Object.keys(baseTools).filter((k) => !CORE_TOOL_SET.has(k)),
);
// Add the loadTools meta-tool ONLY when the feature is enabled; when off the
// toolset and behavior are exactly as before.
const tools = deferredEnabled
? {
...baseTools,
[LOAD_TOOLS_NAME]: makeLoadToolsTool(activatedTools, validDeferredNames),
}
: baseTools;
// Accumulate the turn's streamed output so a provider error / disconnect can
// persist the PARTIAL answer the user already saw — the SDK's onError/onAbort
@@ -799,7 +882,8 @@ export class AiChatService implements OnModuleInit {
// ends with no assistant text (an empty turn). prepareAgentStep forbids
// further tool calls and appends a synthesis instruction on that step,
// concatenated onto the original `system` so the persona is preserved.
prepareStep: ({ stepNumber }) => prepareAgentStep(stepNumber, system),
prepareStep: ({ stepNumber }) =>
prepareAgentStep(stepNumber, system, activatedTools, deferredEnabled),
abortSignal: signal,
onChunk: ({ chunk }) => {
// DIAGNOSTIC (Safari stream-drop investigation) — temporary. Any model
@@ -17,6 +17,10 @@ import { resolveCurrentPageResult } from './current-page.util';
import { parseNodeArg } from './parse-node-arg';
import { modelFriendlyInput } from './model-friendly-input';
import { SandboxStore } from '../../../integrations/sandbox/sandbox.store';
import {
buildInAppDeferredCatalog,
type ToolCatalogEntry,
} from './tool-tiers';
/**
* Per-user, per-request adapter that exposes Docmost READ operations to the
@@ -123,6 +127,18 @@ export class AiChatToolsService {
return client.exportPageMarkdown(pageId);
}
/**
* Build the IN-APP deferred <tool_catalog> entries (#332): one "name — purpose"
* line per DEFERRED tool, merging the per-layer INLINE_TOOL_TIERS with the
* shared registry's own catalogLine. Loads @docmost/mcp for the shared specs
* (memoized). Core tools are always active and are NOT listed here. External
* MCP tools are catalogued separately by the caller (they are runtime-scoped).
*/
async getInAppDeferredCatalog(): Promise<ToolCatalogEntry[]> {
const { sharedToolSpecs } = await loadDocmostMcp();
return buildInAppDeferredCatalog(sharedToolSpecs);
}
async forUser(
user: User,
sessionId: string,
@@ -241,6 +241,11 @@ export interface SharedToolSpec {
mcpName: string;
inAppKey: string;
description: string;
// Deferred-tool metadata (#332). Optional in this mirror so an older/stale
// @docmost/mcp build (pre-#332) still type-checks; the in-app catalog builder
// reads them defensively. The external /mcp server ignores both fields.
tier?: 'core' | 'deferred';
catalogLine?: string;
// Loose `z` on purpose: the registry is zod-agnostic so the server can pass
// its own zod (v4) and the MCP package its own (v3) into the same builder.
buildShape?: (z: any) => Record<string, unknown>;
@@ -0,0 +1,244 @@
import {
CORE_TOOL_KEYS,
CORE_TOOL_SET,
LOAD_TOOLS_NAME,
LOAD_TOOLS_DESCRIPTION,
INLINE_TOOL_TIERS,
buildInAppDeferredCatalog,
buildExternalToolCatalog,
shortenForCatalog,
applyLoadTools,
} from './tool-tiers';
// The real shared registry, imported from source (same approach as the
// SHARED_TOOL_SPECS contract spec) so the tier metadata is checked against
// exactly what @docmost/mcp ships.
import { SHARED_TOOL_SPECS } from '../../../../../../packages/mcp/src/tool-specs';
// For the live-toolset partition test (F3): the REAL adapter, so the catalog is
// checked against the tools AiChatToolsService.forUser() actually builds — not a
// static list that could drift from it.
import { AiChatToolsService } from './ai-chat-tools.service';
import * as loader from './docmost-client.loader';
import type { DocmostClientLike } from './docmost-client.loader';
/**
* #332 deferred tool loading tier metadata, catalog assembly, and the
* loadTools meta-tool. Pure units; no Nest graph, no @docmost/mcp build (the
* registry is imported from TS source).
*/
describe('tool tier metadata (#332)', () => {
it('core set is the documented 13 + searchInPage (14)', () => {
expect(CORE_TOOL_KEYS).toHaveLength(14);
expect(CORE_TOOL_SET.has('searchInPage')).toBe(true); // #330, promoted to core
// loadTools is a meta-tool, not a normal core key.
expect(CORE_TOOL_SET.has(LOAD_TOOLS_NAME)).toBe(false);
});
it('SHARED_TOOL_SPECS tier agrees with CORE_TOOL_SET for every shared tool', () => {
for (const [key, spec] of Object.entries(SHARED_TOOL_SPECS)) {
const isCoreByTier = spec.tier === 'core';
const isCoreByList = CORE_TOOL_SET.has(key);
expect(isCoreByTier).toBe(isCoreByList);
// Every spec carries a non-empty catalogLine (core tools too).
expect(typeof spec.catalogLine).toBe('string');
expect(spec.catalogLine.trim().length).toBeGreaterThan(0);
}
});
it('every INLINE tool tier agrees with CORE_TOOL_SET and has a catalogLine', () => {
for (const [key, meta] of Object.entries(INLINE_TOOL_TIERS)) {
expect(meta.tier === 'core').toBe(CORE_TOOL_SET.has(key));
expect(meta.catalogLine.trim().length).toBeGreaterThan(0);
}
});
});
describe('buildInAppDeferredCatalog (#332)', () => {
const catalog = buildInAppDeferredCatalog(SHARED_TOOL_SPECS as never);
const names = catalog.map((e) => e.name);
it('includes deferred tools from BOTH the inline map and the shared registry', () => {
expect(names).toContain('transformPage'); // inline deferred
expect(names).toContain('getPageJson'); // shared deferred
expect(names).toContain('patchNode'); // shared deferred
expect(names).toContain('createPage'); // inline deferred
});
it('NEVER lists a core tool', () => {
for (const core of CORE_TOOL_KEYS) {
expect(names).not.toContain(core);
}
// spot-check a couple that are core in each source.
expect(names).not.toContain('searchInPage'); // shared core
expect(names).not.toContain('searchPages'); // inline core
expect(names).not.toContain('editPageText'); // shared core
});
it('renders every entry as a "name — purpose" line', () => {
// Non-empty catalog (the length is pinned structurally by the live-toolset
// partition test below, not by a magic constant that rots on every new tool).
expect(catalog.length).toBeGreaterThan(0);
for (const entry of catalog) {
expect(entry.catalogLine).toMatch(/ — /);
}
});
});
/**
* F3 the deferred <tool_catalog> is built from STATIC metadata (INLINE_TOOL_TIERS
* + SHARED_TOOL_SPECS), but the loadable-by-name set is derived at RUNTIME from the
* actual toolset (`Object.keys(baseTools)` in ai-chat.service.ts). Those two must
* agree or a tool becomes loadable-but-invisible (agent thinks it doesn't exist) or
* catalogued-but-phantom. INLINE_TOOL_TIERS is a plain hand-maintained Record with
* no compile-time link to the tools AiChatToolsService.forUser() builds, so nothing
* else catches that drift. This test uses forUser()'s LIVE keys as the source of
* truth (mirroring ai-chat-tools.service.spec.ts's loader mock) and asserts a
* two-way partition against buildInAppDeferredCatalog replacing the old magic
* toHaveLength(28), so a tool added to forUser() without a catalog line (or a
* catalog line without a real tool) fails the suite instead of silently vanishing.
*/
describe('deferred catalog ↔ live forUser() toolset partition (#332, F3)', () => {
let toolKeys: string[];
const catalogNames = buildInAppDeferredCatalog(SHARED_TOOL_SPECS as never).map(
(e) => e.name,
);
beforeAll(async () => {
// Intercept the ESM loader so forUser() builds against the TS-source shared
// specs (no @docmost/mcp build) and never touches the network.
jest.spyOn(loader, 'loadDocmostMcp').mockResolvedValue({
DocmostClient: function () {
return {} as DocmostClientLike;
} as unknown as loader.DocmostClientCtor,
sharedToolSpecs: SHARED_TOOL_SPECS as Record<string, loader.SharedToolSpec>,
});
const service = new AiChatToolsService(
{
generateAccessToken: jest.fn().mockResolvedValue('access-token'),
generateCollabToken: jest.fn().mockResolvedValue('collab-token'),
} as never,
{} as never, // aiService — not exercised while merely BUILDING the tools
{} as never, // pageEmbeddingRepo
{} as never, // spaceMemberRepo
{} as never, // pagePermissionRepo
// sandboxStore: forUser() eagerly calls asSink() to wire the stash tool.
{
asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }),
} as never,
);
const tools = await service.forUser(
{ id: 'user-1', email: 'u@example.com', workspaceId: 'ws-1' } as never,
'session-1',
'ws-1',
'chat-1',
);
toolKeys = Object.keys(tools);
});
afterAll(() => {
jest.restoreAllMocks();
});
it('exposes a non-trivial toolset (sanity: the mock actually built tools)', () => {
expect(toolKeys.length).toBeGreaterThan(20);
});
it('every non-core live tool is present in the catalog (no capability silently hidden)', () => {
// forUser() does not itself add loadTools (ai-chat.service does), but guard
// anyway. Every remaining non-core key MUST have a catalog line.
const catalogSet = new Set(catalogNames);
const missing = toolKeys.filter(
(k) => !CORE_TOOL_SET.has(k) && k !== LOAD_TOOLS_NAME && !catalogSet.has(k),
);
expect(missing).toEqual([]);
});
it('every catalog entry corresponds to a real, non-core live tool (no phantom)', () => {
const liveSet = new Set(toolKeys);
const phantom = catalogNames.filter(
(n) => !liveSet.has(n) || CORE_TOOL_SET.has(n),
);
expect(phantom).toEqual([]);
});
});
describe('buildExternalToolCatalog + shortenForCatalog (#332)', () => {
it('derives a short "name — purpose" line from each external tool description', () => {
const catalog = buildExternalToolCatalog({
tavily_search: { description: 'Search the web for fresh results. More detail here.' },
tavily_extract: { description: '' },
});
expect(catalog).toEqual([
{ name: 'tavily_search', catalogLine: 'tavily_search — Search the web for fresh results.' },
{ name: 'tavily_extract', catalogLine: 'tavily_extract — external tool' },
]);
});
it('caps a very long description', () => {
const long = 'x'.repeat(500);
expect(shortenForCatalog(long).length).toBeLessThanOrEqual(140);
expect(shortenForCatalog(long).endsWith('…')).toBe(true);
});
});
describe('applyLoadTools (#332)', () => {
const valid = new Set(['createPage', 'transformPage', 'tavily_search']);
it('adds valid names to the activated set and returns { loaded }', () => {
const activated = new Set<string>();
const result = applyLoadTools(['createPage', 'tavily_search'], activated, valid);
expect(result).toEqual({ loaded: ['createPage', 'tavily_search'] });
expect(activated.has('createPage')).toBe(true);
expect(activated.has('tavily_search')).toBe(true);
});
it('rejects an unknown name with an error listing the valid deferred names', () => {
const activated = new Set<string>();
expect(() => applyLoadTools(['nope'], activated, valid)).toThrow(/unknown tool name/i);
try {
applyLoadTools(['nope'], activated, valid);
} catch (e) {
const msg = (e as Error).message;
// Lists every valid name (sorted).
expect(msg).toContain('createPage');
expect(msg).toContain('transformPage');
expect(msg).toContain('tavily_search');
}
// Nothing is activated on a rejected call.
expect(activated.size).toBe(0);
});
it('tolerates a non-array / empty input (loads nothing)', () => {
const activated = new Set<string>();
expect(applyLoadTools(undefined, activated, valid)).toEqual({ loaded: [] });
expect(applyLoadTools([], activated, valid)).toEqual({ loaded: [] });
expect(activated.size).toBe(0);
});
it('loadTools description is the verbatim issue text', () => {
expect(LOAD_TOOLS_DESCRIPTION).toContain('only ACTIVATES them');
expect(LOAD_TOOLS_DESCRIPTION).toContain('callable on your NEXT step');
});
});
describe('editorial "Corrector" scenario is fully served by CORE (#332)', () => {
it('read + comment + edit + search need no loadTools', () => {
// A Corrector role reads a page, searches within it, edits text, and leaves
// inline comments — every tool it needs is core, so it never has to load a
// deferred tool.
const needed = [
'getCurrentPage',
'getPage',
'searchPages',
'searchInPage',
'editPageText',
'createComment',
'listComments',
'getComment',
'resolveComment',
];
for (const t of needed) {
expect(CORE_TOOL_SET.has(t)).toBe(true);
}
});
});
@@ -0,0 +1,309 @@
import { tool, type Tool } from 'ai';
import { z } from 'zod';
import type { SharedToolSpec } from './docmost-client.loader';
/**
* Deferred tool loading for the in-app AI chat (#332).
*
* The agent otherwise sends ALL ~41 tool definitions on EVERY model call every
* step, bloating context. Instead we split the in-app tools into two tiers:
*
* - CORE (hot, always active): frequent OR tiny tools whose full schema is
* always visible, plus the `loadTools` meta-tool. Deferring a one-line tool is
* pure loss, so tiny tools stay core even if rare.
* - DEFERRED (loaded on demand): the fat/rare tools + ALL external MCP tools by
* default. The model sees only a compact <tool_catalog> (name purpose) and
* calls `loadTools(names)` to ACTIVATE a tool's full schema for the NEXT step
* (one extra round-trip on first use).
*
* This module is the single source of truth for the IN-APP tiering:
* - CORE_TOOL_KEYS / CORE_TOOL_SET the authoritative core list (used by
* prepareAgentStep to build per-step `activeTools`).
* - INLINE_TOOL_TIERS tier + catalogLine for the per-layer INLINE tools (the
* ones NOT in @docmost/mcp's SHARED_TOOL_SPECS, which carry their own).
* - buildInAppDeferredCatalog / buildExternalToolCatalog assemble the
* <tool_catalog> deferred lines.
* - applyLoadTools / makeLoadToolsTool the loadTools meta-tool.
*
* The tier/catalogLine fields on SHARED_TOOL_SPECS are IN-APP metadata only; the
* external /mcp server ignores them and exposes every tool normally.
*/
/** A single rendered <tool_catalog> line: the tool name + its "name — purpose". */
export interface ToolCatalogEntry {
/** Exact tool name the model must pass to loadTools. */
name: string;
/** Hand-written (in-app) or derived (external) "name — purpose" line. */
catalogLine: string;
}
/**
* CORE (always-active) in-app tool keys 13 frequent/tiny tools. `searchInPage`
* (#330) is added to core on top of the issue's original tier list: it is
* frequent for the editorial roles this feature targets. `loadTools` is active
* too but is not a normal tool key (it is added to activeTools separately).
*/
export const CORE_TOOL_KEYS = [
'searchPages',
'listPages',
'listSpaces',
'getWorkspace',
'getCurrentPage',
'getPage',
'getOutline',
'getNode',
'createComment',
'getComment',
'listComments',
'resolveComment',
'editPageText',
// #330 search_in_page — frequent for editorial sweeps; core despite predating
// the issue's tier list.
'searchInPage',
] as const;
/** O(1) membership test for the core tier. */
export const CORE_TOOL_SET: ReadonlySet<string> = new Set(CORE_TOOL_KEYS);
/** The meta-tool name (always active alongside the core tools when enabled). */
export const LOAD_TOOLS_NAME = 'loadTools';
/**
* loadTools description VERBATIM from issue #332. Tells the model that the
* catalog names EXIST, that loadTools only ACTIVATES them (callable next step),
* and to load several at once.
*/
export const LOAD_TOOLS_DESCRIPTION =
'loadTools — Load the full definitions of deferred tools from the <tool_catalog>\n' +
'block in your instructions. Pass the EXACT tool names from the catalog; this\n' +
'call only ACTIVATES them and returns { loaded: [...] } — the tools become\n' +
'callable on your NEXT step. Load several names in one call when the task clearly\n' +
'needs them. Unknown names are rejected with the list of valid ones.';
/**
* Tier + catalogLine for the INLINE ai-chat tools those defined per-layer in
* ai-chat-tools.service.ts and NOT present in @docmost/mcp's SHARED_TOOL_SPECS
* (which carries its own tier/catalogLine). Together with the shared registry
* this describes every in-app tool. catalogLine is present for core tools too
* (uniformity), but only DEFERRED tools are rendered into the catalog.
*/
export const INLINE_TOOL_TIERS: Record<
string,
{ tier: 'core' | 'deferred'; catalogLine: string }
> = {
// --- core inline ---
searchPages: {
tier: 'core',
catalogLine: 'searchPages — hybrid semantic + keyword search across the wiki.',
},
getCurrentPage: {
tier: 'core',
catalogLine: 'getCurrentPage — the page the user is currently viewing.',
},
getPage: {
tier: 'core',
catalogLine: 'getPage — fetch a page as Markdown by its id.',
},
listPages: {
tier: 'core',
catalogLine: "listPages — list recent pages, or a space's full page tree.",
},
listComments: {
tier: 'core',
catalogLine: 'listComments — list all comments on a page (including resolved).',
},
getComment: {
tier: 'core',
catalogLine: 'getComment — fetch a single comment by id.',
},
createComment: {
tier: 'core',
catalogLine:
'createComment — add an inline comment (optionally with a suggested edit).',
},
resolveComment: {
tier: 'core',
catalogLine: 'resolveComment — resolve or reopen a comment thread.',
},
// --- deferred inline ---
createPage: {
tier: 'deferred',
catalogLine: 'createPage — create a new page with a Markdown body in a space.',
},
updatePageContent: {
tier: 'deferred',
catalogLine:
"updatePageContent — replace a page's body (and optionally title) with new Markdown.",
},
renamePage: {
tier: 'deferred',
catalogLine: "renamePage — change a page's title only (body untouched).",
},
movePage: {
tier: 'deferred',
catalogLine: 'movePage — move a page under a new parent or to the space root.',
},
deletePage: {
tier: 'deferred',
catalogLine: 'deletePage — move a page to trash (soft delete, reversible).',
},
listSidebarPages: {
tier: 'deferred',
catalogLine:
"listSidebarPages — list a space's root pages or a page's direct children.",
},
getTable: {
tier: 'deferred',
catalogLine: 'getTable — read a table as a matrix of cell texts and cell ids.',
},
checkNewComments: {
tier: 'deferred',
catalogLine:
'checkNewComments — find comments in a space created after a timestamp.',
},
getPageHistory: {
tier: 'deferred',
catalogLine:
'getPageHistory — fetch one page-history version with its ProseMirror content.',
},
exportPageMarkdown: {
tier: 'deferred',
catalogLine:
'exportPageMarkdown — export a page to self-contained Markdown (body + comments).',
},
updatePageJson: {
tier: 'deferred',
catalogLine:
"updatePageJson — overwrite a page's body with a full ProseMirror document.",
},
tableInsertRow: {
tier: 'deferred',
catalogLine: 'tableInsertRow — insert a row of plain-text cells into a table.',
},
tableDeleteRow: {
tier: 'deferred',
catalogLine: 'tableDeleteRow — delete a table row at a 0-based index.',
},
tableUpdateCell: {
tier: 'deferred',
catalogLine: 'tableUpdateCell — set the text of a table cell at [row, col].',
},
sharePage: {
tier: 'deferred',
catalogLine: 'sharePage — make a page publicly accessible and return its URL.',
},
transformPage: {
tier: 'deferred',
catalogLine: "transformPage — run a sandboxed JS transform over a page's document.",
},
};
/**
* Build the <tool_catalog> deferred lines for the IN-APP tools by merging the
* two metadata sources: the per-layer INLINE_TOOL_TIERS and the shared registry
* (SHARED_TOOL_SPECS, loaded at runtime). Only DEFERRED tools are included; core
* tools are always active and never appear in the catalog. Pure the caller
* passes the loaded specs so this stays unit-testable.
*/
export function buildInAppDeferredCatalog(
sharedToolSpecs: Record<string, SharedToolSpec>,
): ToolCatalogEntry[] {
const entries: ToolCatalogEntry[] = [];
// Inline deferred tools (hand-written lines).
for (const [name, meta] of Object.entries(INLINE_TOOL_TIERS)) {
if (meta.tier === 'deferred') {
entries.push({ name, catalogLine: meta.catalogLine });
}
}
// Shared deferred tools (line comes from the registry's own catalogLine).
for (const [name, spec] of Object.entries(sharedToolSpecs)) {
if (spec.tier === 'deferred' && spec.catalogLine) {
entries.push({ name, catalogLine: spec.catalogLine });
}
}
return entries;
}
/**
* Cap an external tool's (untrusted) description into a short catalog purpose.
* External MCP tools have no hand-written catalogLine, so we derive one from the
* first sentence of the description, hard-capped. Whitespace is collapsed.
*/
export function shortenForCatalog(description: string, max = 140): string {
const flat = description.replace(/\s+/g, ' ').trim();
if (!flat) return 'external tool';
// Prefer the first sentence if it is reasonably short.
const firstSentence = flat.split(/(?<=[.!?])\s/)[0];
const base =
firstSentence.length > 0 && firstSentence.length <= max
? firstSentence
: flat;
return base.length > max ? `${base.slice(0, max - 1).trimEnd()}` : base;
}
/**
* Build catalog lines for the EXTERNAL MCP tools (all deferred by default,
* #332). Their names are the namespaced tool keys; the purpose is derived from
* each tool's own description (no hand-written line exists). Pure.
*/
export function buildExternalToolCatalog(
externalTools: Record<string, { description?: string } | undefined>,
): ToolCatalogEntry[] {
return Object.entries(externalTools).map(([name, t]) => ({
name,
catalogLine: `${name}${shortenForCatalog(t?.description ?? '')}`,
}));
}
/**
* Pure core of the loadTools meta-tool. Validates the requested names against
* the per-turn set of valid deferred names, ADDS the valid ones to the caller's
* mutable `activatedTools` set (so they become callable next step), and returns
* `{ loaded }`. An unknown name throws a clear error listing the valid deferred
* names surfaced to the model as a tool error so it can retry.
*/
export function applyLoadTools(
names: unknown,
activatedTools: Set<string>,
validDeferredNames: ReadonlySet<string>,
): { loaded: string[] } {
const requested = Array.isArray(names)
? names.filter((n): n is string => typeof n === 'string')
: [];
const unknown = requested.filter((n) => !validDeferredNames.has(n));
if (unknown.length > 0) {
const valid = [...validDeferredNames].sort().join(', ');
throw new Error(
`loadTools: unknown tool name(s): ${unknown.join(', ')}. ` +
`Valid deferred tools are: ${valid || '(none)'}.`,
);
}
for (const n of requested) activatedTools.add(n);
return { loaded: requested };
}
/**
* Build the loadTools AI-SDK tool bound to THIS turn's mutable state: the
* `activatedTools` set (grown by execute, read by prepareAgentStep next step)
* and the `validDeferredNames` set (every non-core tool in this turn's toolset,
* incl. external MCP). Created per streamText call never module-global.
*/
export function makeLoadToolsTool(
activatedTools: Set<string>,
validDeferredNames: ReadonlySet<string>,
): Tool {
return tool({
description: LOAD_TOOLS_DESCRIPTION,
inputSchema: z.object({
names: z
.array(z.string())
.describe(
'EXACT deferred tool names from the <tool_catalog> to activate for ' +
'your next step.',
),
}),
execute: async ({ names }) =>
applyLoadTools(names, activatedTools, validDeferredNames),
});
}
@@ -261,6 +261,21 @@ export class EnvironmentService {
return disable === 'true';
}
/**
* Deferred tool loading for the in-app AI chat (#332). When enabled, the agent
* sees a compact <tool_catalog> and only CORE tools + the loadTools meta-tool
* are active each step; deferred tools (the fat/rare ones + all external MCP
* tools) load on demand. Defaults to ENABLED the issue treats deferred
* loading as the new behavior; set AI_CHAT_DEFERRED_TOOLS=false to restore the
* old "all tools always active" behavior.
*/
isAiChatDeferredToolsEnabled(): boolean {
const enabled = this.configService
.get<string>('AI_CHAT_DEFERRED_TOOLS', 'true')
.toLowerCase();
return enabled === 'true';
}
getPostHogHost(): string {
return this.configService.get<string>('POSTHOG_HOST');
}
@@ -1,5 +1,7 @@
import * as http from 'node:http';
import { Kysely } from 'kysely';
import { tool } from 'ai';
import { z } from 'zod';
import { MockLanguageModelV3, convertArrayToReadableStream } from 'ai/test';
import { AiChatRepo } from '@docmost/db/repos/ai-chat/ai-chat.repo';
import { AiChatMessageRepo } from '@docmost/db/repos/ai-chat/ai-chat-message.repo';
@@ -146,6 +148,9 @@ describe('AiChatService.stream [integration]', () => {
{} as any, // aiAgentRoleRepo (role is pre-resolved + passed in)
{} as any, // pageRepo (only used when body.openPage is set)
{} as any, // pageAccess (idem)
// environment (#332): keep deferred tool loading OFF for this lifecycle
// harness so the toolset/behavior is exactly as before.
{ isAiChatDeferredToolsEnabled: () => false } as any,
);
}
@@ -315,4 +320,174 @@ describe('AiChatService.stream [integration]', () => {
true,
);
});
/**
* #332 deferred tool loading, the ON path. The riskiest property is that the
* per-turn `activatedTools` Set is created FRESH inside each stream() call, so a
* tool a previous turn activated via loadTools is NOT still active when the next
* turn starts the new turn begins "cold" (CORE + loadTools only). The unit
* tests only exercise pure prepareAgentStep with hand-fed Sets; this pins the
* real wiring end-to-end (loadTools.execute -> activatedTools -> prepareStep ->
* per-step activeTools) against the real streamText loop, and proves there is no
* cross-turn leak. We drive a MockLanguageModelV3 whose step 1 calls
* loadTools(['createPage']) and assert, via the model's recorded per-step
* CallOptions.tools (the AI SDK filters the provider tool list by activeTools),
* that the deferred tool becomes active on the SAME turn's next step but NOT on a
* fresh turn's first step.
*/
describe('deferred tool loading ON — per-turn activation, no leak (#332)', () => {
// A stub deferred (non-core) tool the agent can activate. Its execute is never
// called — the model only needs to SEE it become active — but it must be a
// valid AI-SDK tool so the SDK includes it in a step's tool list once active.
const createPageStub = tool({
description: 'create a new page',
inputSchema: z.object({ title: z.string() }),
execute: async () => ({ id: 'p-stub' }),
});
// A CORE tool in the toolset, so a cold step shows CORE tools ARE active while
// the deferred createPage is not. `searchPages` is in CORE_TOOL_SET.
const searchPagesStub = tool({
description: 'search the wiki',
inputSchema: z.object({ query: z.string() }),
execute: async () => [],
});
// Same lifecycle harness as buildService() above, but with deferred loading ON
// and a toolset that exposes exactly one deferred tool (createPage) so it is
// catalogued + loadable-by-name. Kept separate so the OFF scenarios are
// untouched.
function buildDeferredService(): AiChatService {
return new AiChatService(
{ getChatModel: async () => null } as any,
aiChatRepo,
msgRepo,
{} as any,
{ resolve: async () => null } as any,
{
forUser: async () => ({
searchPages: searchPagesStub,
createPage: createPageStub,
}),
getInAppDeferredCatalog: async () => [
{ name: 'createPage', catalogLine: 'createPage — create a new page.' },
],
} as any,
mcpClients as any,
{} as any,
{} as any,
{} as any,
// #332: deferred tool loading ON — the property under test.
{ isAiChatDeferredToolsEnabled: () => true } as any,
);
}
// Drive ONE stream() turn against `model` and wait for the assistant row to
// settle (mirrors runStream, but builds the deferred-ON service).
async function runDeferredTurn(
model: MockLanguageModelV3,
chatId: string,
body: any,
): Promise<void> {
closeCalls = 0;
const service = buildDeferredService();
const { res, cleanup } = await makeRealResponse();
try {
await service.stream({
user: { id: userId, workspaceId } as any,
workspace: { id: workspaceId, name: 'WS' } as any,
sessionId: 'sess-1',
body,
res: { raw: res } as any,
signal: new AbortController().signal,
model: model as any,
role: null,
} as any);
await waitFor(async () => {
const rows = await msgRepo.findAllByChat(chatId, workspaceId);
return rows.some(
(r) =>
r.role === 'assistant' &&
['completed', 'error', 'aborted'].includes(r.status as string),
);
});
await waitFor(() => closeCalls > 0, { timeoutMs: 5_000 });
} finally {
await cleanup();
}
}
// Tool names the provider actually received for a recorded step (activeTools
// filters this list, so it reflects what was active that step).
const toolNames = (call: any): string[] =>
((call?.tools ?? []) as any[]).map((t) => t?.name).filter(Boolean);
// A model that, on step 1, calls loadTools(['createPage']); on step 2, answers.
function loadThenAnswerModel(): MockLanguageModelV3 {
let step = 0;
return new MockLanguageModelV3({
doStream: async () => {
const n = step++;
if (n === 0) {
return {
stream: convertArrayToReadableStream([
{ type: 'stream-start', warnings: [] },
{
type: 'tool-call',
toolCallId: 'lt1',
toolName: 'loadTools',
input: JSON.stringify({ names: ['createPage'] }),
},
{
type: 'finish',
finishReason: 'tool-calls',
usage: { inputTokens: 5, outputTokens: 3, totalTokens: 8 },
},
] as any),
};
}
return { stream: successStream() };
},
} as any);
}
it('activates a deferred tool for the SAME turn, and a NEW turn starts cold (no leak)', async () => {
const chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
// --- Turn 1: loadTools(createPage) on step 1, then answer on step 2. ---
const model1 = loadThenAnswerModel();
await runDeferredTurn(model1, chatId, {
chatId,
messages: [userUiMessage('Make me a page')],
});
// The turn ran at least two steps (the load round-trip + the answer).
expect(model1.doStreamCalls.length).toBeGreaterThanOrEqual(2);
const step1Tools = toolNames(model1.doStreamCalls[0]);
const step2Tools = toolNames(model1.doStreamCalls[1]);
// Step 1 starts cold: CORE tools + the loadTools meta-tool are active, but
// the deferred createPage is NOT yet.
expect(step1Tools).toContain('loadTools');
expect(step1Tools).toContain('searchPages'); // a CORE tool, always active
expect(step1Tools).not.toContain('createPage');
// Step 2 of the SAME turn sees the just-activated deferred tool.
expect(step2Tools).toContain('createPage');
// --- Turn 2 on the SAME chat: must start cold again. ---
const model2 = new MockLanguageModelV3({
doStream: async () => ({ stream: successStream() }),
} as any);
await runDeferredTurn(model2, chatId, {
chatId,
messages: [userUiMessage('And another thing')],
});
const nextTurnFirstStep = toolNames(model2.doStreamCalls[0]);
expect(nextTurnFirstStep).toContain('loadTools');
// The activated set is per-turn: the prior turn's createPage did NOT leak,
// so the fresh turn's first step sees it deferred again.
expect(nextTurnFirstStep).not.toContain('createPage');
});
});
});
+65
View File
@@ -31,6 +31,22 @@ export interface SharedToolSpec {
inAppKey: string;
/** Single canonical model-facing description used by both layers. */
description: string;
/**
* Deferred-tool tier for the IN-APP agent (#332). 'core' tools are always
* active; 'deferred' tools are hidden behind the <tool_catalog> and loaded on
* demand via the loadTools meta-tool. This is an IN-APP concern only: the
* standalone /mcp server ignores this field and registers every tool normally
* (registerShared in index.ts reads mcpName/description/buildShape only).
*/
tier: 'core' | 'deferred';
/**
* Hand-written one-liner "name — purpose" shown in the in-app agent's
* <tool_catalog> for a DEFERRED tool (#332). Deliberately NOT derived from the
* description's first sentence a concise, accurate purpose line. Present on
* every spec (core tools too) for uniformity; only deferred ones are rendered.
* Inert for the external /mcp server.
*/
catalogLine: string;
/**
* Builds the tool's input schema as a plain object of zod fields (a
* ZodRawShape). Called with the consumer's own zod namespace. Omitted for
@@ -47,6 +63,8 @@ export const SHARED_TOOL_SPECS = {
mcpName: 'get_workspace',
inAppKey: 'getWorkspace',
description: 'Fetch metadata about the current workspace (name, settings).',
tier: 'core',
catalogLine: 'getWorkspace — fetch current workspace metadata (name, settings).',
},
listSpaces: {
@@ -55,6 +73,8 @@ export const SHARED_TOOL_SPECS = {
description:
'List the spaces the current user can access. Returns the array of ' +
'spaces (id, name, slug, ...).',
tier: 'core',
catalogLine: 'listSpaces — list the spaces the user can access (id, name, slug).',
},
listShares: {
@@ -62,6 +82,8 @@ export const SHARED_TOOL_SPECS = {
inAppKey: 'listShares',
description:
'List all public shares in the workspace with page titles and public URLs.',
tier: 'deferred',
catalogLine: 'listShares — list all public shares in the workspace with their URLs.',
},
// --- single-pageId read tools ---
@@ -74,6 +96,9 @@ export const SHARED_TOOL_SPECS = {
'includes block ids, callouts, tables, link/image attributes) plus the ' +
'slugId used in URLs. Use the block ids it returns to make precise ' +
'structural edits or surgical text edits without resending the page.',
tier: 'deferred',
catalogLine:
"getPageJson — get a page's raw ProseMirror JSON (lossless, with block ids).",
buildShape: (z) => ({
pageId: z.string().min(1),
}),
@@ -88,6 +113,9 @@ export const SHARED_TOOL_SPECS = {
'count) WITHOUT the full document body. Use it to locate sections/tables ' +
'and grab block ids cheaply before fetching, patching or inserting ' +
'individual blocks.',
tier: 'core',
catalogLine:
"getOutline — compact outline of a page's top-level blocks with their ids.",
buildShape: (z) => ({
pageId: z.string().min(1),
}),
@@ -104,6 +132,9 @@ export const SHARED_TOOL_SPECS = {
'outline or page-JSON view (works for headings/paragraphs/callouts/images), OR ' +
'`#<index>` to fetch a top-level block by its outline index — use the ' +
'`#<index>` form for tables/rows/cells, which carry no id.',
tier: 'core',
catalogLine:
"getNode — fetch one block's ProseMirror subtree by block id or #index.",
buildShape: (z) => ({
pageId: z.string().min(1),
nodeId: z.string().min(1),
@@ -137,6 +168,9 @@ export const SHARED_TOOL_SPECS = {
'caseSensitive:true to match case. Ideal for systematic ' +
'editorial sweeps (unquoted "ё", straight quotes, "т.е.", stray units). An ' +
'invalid regex or an empty query returns a clear error to fix.',
tier: 'core',
catalogLine:
'searchInPage — find every occurrence of a string/regex inside one page, with locations.',
buildShape: (z) => ({
pageId: z.string().min(1).describe('ID of the page to search'),
query: z
@@ -172,6 +206,8 @@ export const SHARED_TOOL_SPECS = {
description:
'Remove a single block by its attrs.id (from the page outline or ' +
'page-JSON view) WITHOUT resending the whole document.',
tier: 'deferred',
catalogLine: 'deleteNode — remove a single content block by its block id.',
buildShape: (z) => ({
pageId: z.string().min(1),
nodeId: z.string().min(1),
@@ -203,6 +239,9 @@ export const SHARED_TOOL_SPECS = {
'JSON object or a JSON string (both accepted). Cheaper and safer than ' +
'replacing the whole document for one-block structural edits. Reversible: ' +
'the previous version is kept in page history.',
tier: 'deferred',
catalogLine:
'patchNode — replace one block with a new ProseMirror node, keeping its id.',
buildShape: (z) => ({
pageId: z.string().min(1).describe('ID of the page containing the block'),
nodeId: z
@@ -245,6 +284,9 @@ export const SHARED_TOOL_SPECS = {
'[{"type":"text","text":"Title"}]}. Bold is a mark: ' +
'{"type":"text","text":"x","marks":[{"type":"bold"}]}. The node may be a ' +
'JSON object or a JSON string (both accepted). Reversible via page history.',
tier: 'deferred',
catalogLine:
'insertNode — insert a block before/after an anchor, or append at the end.',
buildShape: (z) => ({
pageId: z.string().min(1),
node: z
@@ -278,6 +320,8 @@ export const SHARED_TOOL_SPECS = {
mcpName: 'unshare_page',
inAppKey: 'unsharePage',
description: 'Remove the public share of a page (revokes the public URL).',
tier: 'deferred',
catalogLine: "unsharePage — revoke a page's public share (removes the public URL).",
buildShape: (z) => ({
pageId: z.string().min(1).describe('ID of the page to unshare'),
}),
@@ -295,6 +339,9 @@ export const SHARED_TOOL_SPECS = {
"`from`/`to` each accept a historyId, or null/'current' for the page's " +
'current content (defaults: from=current, to=current — pass a historyId ' +
'from the page-history list to compare against the live page).',
tier: 'deferred',
catalogLine:
'diffPageVersions — diff two page versions and return the change set + summary.',
buildShape: (z) => ({
pageId: z.string().min(1),
from: z
@@ -315,6 +362,9 @@ export const SHARED_TOOL_SPECS = {
"List a page's saved versions (Docmost auto-snapshots on every save), " +
'newest first, cursor-paginated. Returns { items, nextCursor }; each ' +
"item's id is the historyId to pass to the page diff or restore tools.",
tier: 'deferred',
catalogLine:
"listPageHistory — list a page's saved versions (newest first, paginated).",
buildShape: (z) => ({
pageId: z.string().min(1),
cursor: z
@@ -332,6 +382,9 @@ export const SHARED_TOOL_SPECS = {
'as the page\'s current content (Docmost has no restore endpoint, so ' +
'this creates a NEW history snapshot — the restore is itself revertible). ' +
'Get the historyId from the page-history list.',
tier: 'deferred',
catalogLine:
'restorePageVersion — restore a page to a saved history version (revertible).',
buildShape: (z) => ({
historyId: z.string().min(1),
}),
@@ -349,6 +402,9 @@ export const SHARED_TOOL_SPECS = {
'thread records are NOT created/updated/deleted on the server by this ' +
'tool — only the page body + inline comment marks are written; manage ' +
'comment threads via the comment tools/UI.',
tier: 'deferred',
catalogLine:
"importPageMarkdown — replace a page's content from exported Docmost Markdown.",
buildShape: (z) => ({
pageId: z.string().min(1),
markdown: z.string().min(1),
@@ -365,6 +421,9 @@ export const SHARED_TOOL_SPECS = {
'entirely server-side — the document is NOT sent through the model. The ' +
'target keeps its own title and slug; only its body is replaced. Ideal ' +
"for 'make page A's content equal to B' or 'replace A with B but keep A's URL'.",
tier: 'deferred',
catalogLine:
"copyPageContent — replace one page's body with a copy of another page's body.",
buildShape: (z) => ({
sourcePageId: z.string().min(1).describe('Page to copy content FROM'),
targetPageId: z
@@ -402,6 +461,9 @@ export const SHARED_TOOL_SPECS = {
'page JSON and use a structural node patch/update to set its marks. ' +
'Examples: edits:[{find:"teh",replace:"the"}]; edits:[{find:"Hello ' +
'world",replace:"Hello there"}] (crosses a bold boundary).',
tier: 'core',
catalogLine:
"editPageText — surgical find/replace of plain text in a page, preserving ids/marks.",
buildShape: (z) => ({
pageId: z.string().describe('ID of the page to edit'),
edits: z
@@ -440,6 +502,9 @@ export const SHARED_TOOL_SPECS = {
'server instance that created it: in a multi-replica deployment without ' +
'sticky sessions a blob stored on one instance is not retrievable via the ' +
'sandbox URL on another (it 404s like an expired one).',
tier: 'deferred',
catalogLine:
'stashPage — serialize a whole page to a short anonymous URL without loading its body.',
buildShape: (z) => ({
pageId: z.string().min(1),
}),
+1 -1
View File
@@ -450,7 +450,7 @@ async function main() {
// 8. get_page markdown round-trip sanity (table separator present)
const md = await client.getPage(pageId);
check("get_page md: table separator emitted", md.data.content.includes("| --- |"), "");
check("get_page md: callout exported as :::", md.data.content.includes(":::info"));
check("get_page md: callout exported as Obsidian '> [!info]'", md.data.content.includes("> [!info]"));
// 9. comments: create / list / reply / update / check_new / delete
const beforeComments = new Date(Date.now() - 1000).toISOString();
@@ -635,13 +635,17 @@ const Attachment = Node.create({
},
name: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-attachment-name"),
// Empty-string-vs-absent idempotency (GS-EDIT-REVERT class): "" -> default.
parseHTML: (el: HTMLElement) =>
el.getAttribute("data-attachment-name") || null,
renderHTML: (attrs: Record<string, any>) =>
attrs.name ? { "data-attachment-name": attrs.name } : {},
},
mime: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-attachment-mime"),
// Empty-string-vs-absent idempotency (GS-EDIT-REVERT class): "" -> default.
parseHTML: (el: HTMLElement) =>
el.getAttribute("data-attachment-mime") || null,
renderHTML: (attrs: Record<string, any>) =>
attrs.mime ? { "data-attachment-mime": attrs.mime } : {},
},
@@ -689,7 +693,10 @@ const Video = Node.create({
},
alt: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("aria-label"),
// Empty-string-vs-absent idempotency: coerce "" back to the default so a
// stray empty `aria-label` never materializes `alt: ""` on a video stored
// with no alt (same GS-EDIT-REVERT class as the image `alt` fix).
parseHTML: (el: HTMLElement) => el.getAttribute("aria-label") || null,
renderHTML: (attrs: Record<string, any>) =>
attrs.alt ? { "aria-label": attrs.alt } : {},
},
@@ -864,13 +871,15 @@ const diagramAttributes = () => ({
},
title: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-title"),
// Empty-string-vs-absent idempotency (GS-EDIT-REVERT class): "" -> default.
parseHTML: (el: HTMLElement) => el.getAttribute("data-title") || null,
renderHTML: (attrs: Record<string, any>) =>
attrs.title ? { "data-title": attrs.title } : {},
},
alt: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-alt"),
// Empty-string-vs-absent idempotency (GS-EDIT-REVERT class): "" -> default.
parseHTML: (el: HTMLElement) => el.getAttribute("data-alt") || null,
renderHTML: (attrs: Record<string, any>) =>
attrs.alt ? { "data-alt": attrs.alt } : {},
},
@@ -1106,7 +1115,8 @@ const Pdf = Node.create({
},
name: {
default: null,
parseHTML: (el: HTMLElement) => el.getAttribute("data-name"),
// Empty-string-vs-absent idempotency (GS-EDIT-REVERT class): "" -> default.
parseHTML: (el: HTMLElement) => el.getAttribute("data-name") || null,
renderHTML: (attrs: Record<string, any>) =>
attrs.name ? { "data-name": attrs.name } : {},
},
@@ -1491,6 +1501,29 @@ export const docmostExtensions = [
...parent.height,
parseHTML: (el: HTMLElement) => el.getAttribute("height"),
},
// Empty-string-vs-absent idempotency (GS-EDIT-REVERT class). `marked`
// renders `![](src)` as `<img alt="">`, so the stock Image `alt`
// parseHTML (`getAttribute("alt")`) materializes `alt: ""` on an image
// that was stored with NO alt (attr absent). That is a false diff against
// the editor-stored form (a no-alt image has alt ABSENT, not ""), so a
// git-sync / ai-chat touch of a page with a plain image produced phantom
// churn. Coerce an empty string back to the attr's default (null) so the
// import is idempotent. A real alt survives verbatim (`|| undefined` keeps
// the truthy value; the default fills the empty case). `title` is coerced
// the same way for the whole class, even though `marked` does not
// currently emit `title=""` — defence in depth against any path that does.
// NOTE: this DIVERGES from editor-ext's literal image `alt` parseHTML
// (`getAttribute("alt")`, which returns "" verbatim), but CONVERGES on
// editor-ext's real STORED shape: an editor image inserted without alt
// renders with no `alt` attribute and re-parses as absent, never "".
alt: {
...parent.alt,
parseHTML: (el: HTMLElement) => el.getAttribute("alt") || null,
},
title: {
...parent.title,
parseHTML: (el: HTMLElement) => el.getAttribute("title") || null,
},
};
},
}).configure({ inline: false }),
@@ -0,0 +1,443 @@
/**
* Reusable round-trip-STABILITY matrix helper (fixtures-first).
*
* A single stored node authored WITHOUT a given string attribute (attr
* absent / undefined) must not gain a phantom EMPTY-STRING value after a
* markdown round-trip the "empty-string-vs-absent" churn class. This helper,
* given a node spec, drives a matrix of attribute combinations through the REAL
* converter (`convertProseMirrorToMarkdown` -> `markdownToProseMirror`) and
* asserts byte-stability on two contours:
*
* 1. RAW round-trip: for the node under test, every attribute the round-trip
* materializes must equal what the INPUT authored an authored attr keeps
* its value, an ABSENT attr may only reappear at its SCHEMA DEFAULT. If an
* absent attr comes back as a NON-default value (e.g. `alt: ""` where the
* default is `null`), that is an instability and is reported precisely as
* `type.attr: absent -> "<got>"`. This is the contour git-sync / stored
* JSON diffs on, so masking it only in `canonicalize` would leave the noise.
*
* 2. CANONICAL round-trip: `canonicalizeContent(original)` must deep-equal
* `canonicalizeContent(roundtrip)` (a second, semantic contour).
*
* The ONLY normalization the helper treats as allowed (not an instability) is
* the DOCUMENTED numeric width/height/size/aspectRatio -> string coercion the
* converter performs on purpose (a stored numeric `640` re-parses via
* `getAttribute` as the string `"640"`). It is encoded here as an explicit
* per-spec `numericStringAttrs` set applied to BOTH contours, NOT a silent skip.
*
* The helper is node-type agnostic: image and the whole media family share the
* `align !== "center"` predicate + `<!--name {…}-->` comment machinery, so one
* matrix guards the shared class.
*/
import { getSchema } from "@tiptap/core";
import {
convertProseMirrorToMarkdown,
markdownToProseMirror,
canonicalizeContent,
docmostExtensions,
} from "../src/lib/index.js";
import { firstDivergence } from "./roundtrip-helpers.js";
/** One attribute's two probe values. */
export interface AttrMatrixEntry {
/** Attribute name on the node. */
attr: string;
/**
* The "default" pick. `undefined` means the attribute is OMITTED entirely
* (the absent case the one that can materialize an empty string on import).
* A concrete value is authored verbatim.
*/
default: unknown;
/** A representative NON-default value to exercise (must survive verbatim). */
nonDefault: unknown;
/**
* Marks the attr as a member of the EMPTY-STRING class the fix targets: a
* string attr whose schema default is `null`/absent and whose parseHTML
* coerces `"" -> default` (image/drawio `alt`+`title`, video `alt` via
* aria-label, pdf/attachment `name`, attachment `mime`). Set true to also
* drive the THIRD-STATE convergence case (see runConvergenceCase) for this
* attr. Attrs whose default is NOT null (e.g. embed `provider`, default "")
* or that are not `""`-coerced (control attrs) are left unset.
*/
emptyStringClass?: boolean;
}
/** A node type + the attribute matrix to sweep for it. */
export interface NodeStabilitySpec {
/** Node type (e.g. "image", "video"). */
type: string;
/** Attributes always present on the node (e.g. `{ src: "/i.png" }`). */
baseAttrs?: Record<string, unknown>;
/** Attributes to sweep at default and non-default. */
attrMatrix: AttrMatrixEntry[];
/**
* Attributes whose numeric -> string coercion on round-trip is DOCUMENTED and
* intentional; compared modulo `String(x)` on both sides. Defaults to the
* converter's known sizing set.
*/
numericStringAttrs?: string[];
}
/** A single unstable finding, legible enough to tie a gate-lock to. */
export interface Instability {
type: string;
attr: string;
/** What the input authored: the literal value, or the ABSENT sentinel. */
authored: unknown | typeof ABSENT;
/** What the round-trip produced. */
got: unknown;
/** What a stable round-trip should have produced (authored value or default). */
expected: unknown;
}
/** One matrix cell's result. */
export interface ComboResult {
label: string;
authored: Record<string, unknown>;
/** RAW-contour instabilities on the node under test. */
raw: Instability[];
/** CANONICAL-contour divergence (path + values) or null when equal. */
canonical: { path: string; a: unknown; b: unknown } | null;
/** True when the node type failed to round-trip at all (structural loss). */
missing: boolean;
md: string;
}
/** Whole-matrix report for one node spec. */
export interface MatrixReport {
type: string;
combos: ComboResult[];
}
/** Sentinel marking an attribute the input did NOT author. */
export const ABSENT = Symbol("ABSENT");
const DEFAULT_NUMERIC_STRING_ATTRS = [
"width",
"height",
"size",
"aspectRatio",
];
// The ProseMirror schema the converter targets — its attribute `default`s are
// the authoritative "what an absent attr should re-materialize as" oracle.
const schema = getSchema(docmostExtensions);
/** Read the schema default for every attribute of a node type. */
function schemaDefaults(type: string): Record<string, unknown> {
const specAttrs = (schema.nodes[type]?.spec?.attrs ?? {}) as Record<
string,
{ default: unknown }
>;
const out: Record<string, unknown> = {};
for (const [k, v] of Object.entries(specAttrs)) out[k] = v.default;
return out;
}
/** Find the first node of a given type anywhere in a PM doc tree. */
function findFirst(node: any, type: string): any {
if (node && node.type === type) return node;
for (const child of node?.content ?? []) {
const hit = findFirst(child, type);
if (hit) return hit;
}
return null;
}
/** Coerce a scalar for the documented numeric->string comparison. */
const numStr = (x: unknown): unknown => (x == null ? x : String(x));
/**
* Enumerate the cartesian product of the matrix: every attribute independently
* at its default (index 0) or non-default (index 1) pick. The all-default
* corner is included (the baseline). Small by construction (2^N over a handful
* of at-risk string attrs).
*/
function enumerateCombos(matrix: AttrMatrixEntry[]): number[][] {
let combos: number[][] = [[]];
for (let i = 0; i < matrix.length; i++) {
const next: number[][] = [];
for (const c of combos) {
next.push([...c, 0]);
next.push([...c, 1]);
}
combos = next;
}
return combos;
}
/** Build the authored attrs for one combo pick vector. */
function authoredAttrs(
spec: NodeStabilitySpec,
picks: number[],
): Record<string, unknown> {
const attrs: Record<string, unknown> = { ...(spec.baseAttrs ?? {}) };
spec.attrMatrix.forEach((entry, i) => {
if (picks[i] === 1) {
attrs[entry.attr] = entry.nonDefault;
} else if (entry.default !== undefined) {
attrs[entry.attr] = entry.default;
}
// default === undefined -> OMIT the attr entirely (the absent case).
});
return attrs;
}
/** Human-readable label for a combo (which attrs are at non-default). */
function comboLabel(spec: NodeStabilitySpec, picks: number[]): string {
const on = spec.attrMatrix
.filter((_, i) => picks[i] === 1)
.map((e) => e.attr);
return on.length === 0 ? "<all-default>" : on.join("+");
}
/**
* Run the full stability matrix for one node spec and return a structured
* report (does NOT throw the caller asserts, so a failure can print the whole
* report). Every combo runs the real export->import pipeline once.
*/
export async function runStabilityMatrix(
spec: NodeStabilitySpec,
): Promise<MatrixReport> {
const numericStringAttrs = new Set(
spec.numericStringAttrs ?? DEFAULT_NUMERIC_STRING_ATTRS,
);
const defaults = schemaDefaults(spec.type);
const combos: ComboResult[] = [];
for (const picks of enumerateCombos(spec.attrMatrix)) {
const authored = authoredAttrs(spec, picks);
const doc = { type: "doc", content: [{ type: spec.type, attrs: authored }] };
const md = convertProseMirrorToMarkdown(doc);
const rt = await markdownToProseMirror(md);
const node = findFirst(rt, spec.type);
const result: ComboResult = {
label: comboLabel(spec, picks),
authored,
raw: [],
canonical: null,
missing: node == null,
md,
};
if (node != null) {
// RAW contour: every materialized attr must equal the authored value, or
// (for an absent attr) the schema default — modulo the documented numeric
// string coercion.
const rtAttrs = (node.attrs ?? {}) as Record<string, unknown>;
for (const key of Object.keys(rtAttrs)) {
const authoredHas = Object.prototype.hasOwnProperty.call(authored, key);
const expected = authoredHas ? authored[key] : defaults[key];
let got = rtAttrs[key];
let exp = expected;
if (numericStringAttrs.has(key)) {
got = numStr(got);
exp = numStr(exp);
}
if (firstDivergence(got, exp) !== null) {
result.raw.push({
type: spec.type,
attr: key,
authored: authoredHas ? authored[key] : ABSENT,
got: rtAttrs[key],
expected,
});
}
}
// CANONICAL contour: canonical forms deep-equal, modulo the same numeric
// string coercion (applied to both trees so a documented coercion is not
// counted as a divergence).
const ca = normalizeNumeric(canonicalizeContent(doc), numericStringAttrs);
const cb = normalizeNumeric(canonicalizeContent(rt), numericStringAttrs);
result.canonical = firstDivergence(ca, cb);
}
combos.push(result);
}
return { type: spec.type, combos };
}
/**
* Deep-copy a canonical tree, coercing the documented numeric->string attrs to
* their string form so an intentional `640 -> "640"` coercion is not reported
* as a canonical divergence. Only touches the listed attribute keys.
*/
function normalizeNumeric(node: any, attrs: Set<string>): any {
if (Array.isArray(node)) return node.map((n) => normalizeNumeric(n, attrs));
if (node === null || typeof node !== "object") return node;
const out: Record<string, unknown> = {};
for (const key of Object.keys(node)) {
if (key === "attrs" && node.attrs && typeof node.attrs === "object") {
const a: Record<string, unknown> = {};
for (const [k, v] of Object.entries(node.attrs)) {
a[k] = attrs.has(k) ? numStr(v) : v;
}
out.attrs = a;
} else {
out[key] = normalizeNumeric(node[key], attrs);
}
}
return out;
}
/** Flatten a report to just its unstable combos (for a terse assertion). */
export function unstableCombos(report: MatrixReport): ComboResult[] {
return report.combos.filter(
(c) => c.missing || c.raw.length > 0 || c.canonical !== null,
);
}
// ---------------------------------------------------------------------------
// THIRD STATE: an EXPLICITLY-STORED empty string on a string attr.
//
// The matrix above sweeps TWO states per string attr: absent/default and a
// non-default value — and asserts FIRST-pass byte-stability for both. There is
// a third, degenerate state the matrix does NOT cover: the attr stored as a
// LITERAL `""`. This is DISTINCT from "the node never had the attr": a user
// types an alt in the editor, then deletes it, and Tiptap's
// `updateAttributes({ alt: "" })` persists a literal `alt: ""` in the stored
// JSON. There is no absent-vs-"" distinction in the DOM once serialized, so the
// fix's `getAttribute("alt") || null` coercion canonicalizes BOTH to the
// default (`null`).
//
// Consequence — and this is CORRECT, not a bug: a doc carrying an explicit `""`
// converges to the default on the FIRST round-trip (a ONE-TIME diff: `"" ->
// null`), then is byte-stable from the SECOND round-trip on (idempotent). So
// this state must be pinned with a DIFFERENT contract than the matrix's:
// - do NOT assert first-pass byte-stability (the first pass legitimately
// changes `""` -> default), and
// - DO assert the first pass converges to the default AND the second pass is
// idempotent (rt2 deep-equals rt1).
//
// A future sync/QA pass diffing stored pages will see this one-time `"" -> null`
// normalization exactly once per affected node; it is the converter canon, not
// corruption, and must not be flagged as data loss.
// ---------------------------------------------------------------------------
/** Result of the third-state ("explicit empty string") convergence probe. */
export interface ConvergenceResult {
type: string;
attr: string;
/** The schema default the attr must converge to on pass 1 (null / absent). */
expectedDefault: unknown;
/** rt1's materialized value for the attr — must equal `expectedDefault`. */
firstPassValue: unknown;
/** True when the node round-tripped AND rt1 converged the attr to default. */
convergedToDefault: boolean;
/** rt1-vs-rt2 divergence; MUST be null (idempotent from pass 2 on). */
secondPassDivergence: { path: string; a: unknown; b: unknown } | null;
/** True when the node type failed to round-trip at all (structural loss). */
missing: boolean;
}
/** Round-trip a full PM doc through the real converter once. */
async function roundtripDoc(doc: any): Promise<any> {
return markdownToProseMirror(convertProseMirrorToMarkdown(doc));
}
/**
* Third-state convergence probe for one string attr of the empty-string class.
*
* (a) builds a doc with the attr EXPLICITLY set to `""` (baseAttrs + `""`),
* (b) rt1 = roundtrip(doc); asserts rt1's attr equals the schema default the
* documented ONE-TIME `"" -> default` normalization (NOT byte-stable vs the
* `""` input, so first-pass stability is deliberately NOT asserted here),
* (c) rt2 = roundtrip(rt1); asserts rt2 deep-equals rt1 idempotent from the
* second round-trip on.
*
* Returns a structured result (does NOT throw) so the caller can assert and
* print. Reusable across the whole node family: drive it for every attr flagged
* `emptyStringClass` on every spec (see convergenceCasesFor / the test driver).
*/
export async function runConvergenceCase(
spec: NodeStabilitySpec,
attr: string,
): Promise<ConvergenceResult> {
const expectedDefault = schemaDefaults(spec.type)[attr];
// (a) The degenerate third state: attr persisted as a LITERAL "".
const authored = { ...(spec.baseAttrs ?? {}), [attr]: "" };
const doc = { type: "doc", content: [{ type: spec.type, attrs: authored }] };
// (b) First round-trip: "" must normalize to the default (a one-time diff).
const rt1 = await roundtripDoc(doc);
const node1 = findFirst(rt1, spec.type);
const firstPassValue = node1?.attrs?.[attr];
const convergedToDefault =
node1 != null && firstDivergence(firstPassValue, expectedDefault) === null;
// (c) Second round-trip: must be byte-stable (rt2 deep-equals rt1). We compare
// the WHOLE docs — both are converter OUTPUTS already in the same materialized
// form (numeric attrs are strings on both sides), so no numeric normalization
// is needed here, unlike the raw/canonical contours above.
const rt2 = node1 != null ? await roundtripDoc(rt1) : rt1;
const secondPassDivergence =
node1 != null ? firstDivergence(rt1, rt2) : null;
return {
type: spec.type,
attr,
expectedDefault,
firstPassValue,
convergedToDefault,
secondPassDivergence,
missing: node1 == null,
};
}
/** The attrs of a spec flagged as members of the empty-string class. */
export function convergenceCasesFor(spec: NodeStabilitySpec): string[] {
return spec.attrMatrix
.filter((e) => e.emptyStringClass)
.map((e) => e.attr);
}
/** True when a convergence result honours the "converges once, then stable" contract. */
export function convergenceOk(r: ConvergenceResult): boolean {
return !r.missing && r.convergedToDefault && r.secondPassDivergence === null;
}
/** Render a convergence result as a legible one-liner for a failed assertion. */
export function formatConvergence(r: ConvergenceResult): string {
if (r.missing) return `${r.type}.${r.attr}: DID-NOT-ROUND-TRIP`;
const parts: string[] = [];
if (!r.convergedToDefault) {
parts.push(
`pass1 did NOT converge: got ${JSON.stringify(r.firstPassValue)} (expected default ${JSON.stringify(r.expectedDefault)})`,
);
}
if (r.secondPassDivergence) {
parts.push(
`pass2 NOT idempotent @ ${r.secondPassDivergence.path}: ${JSON.stringify(r.secondPassDivergence.a)} vs ${JSON.stringify(r.secondPassDivergence.b)}`,
);
}
const status = parts.length === 0 ? "converges-once-then-stable" : parts.join("; ");
return `${r.type}.${r.attr}: ${status}`;
}
/** Render a report as a legible multi-line string for a failed assertion. */
export function formatReport(report: MatrixReport): string {
const lines: string[] = [`node "${report.type}":`];
for (const c of report.combos) {
const flags: string[] = [];
if (c.missing) flags.push("DID-NOT-ROUND-TRIP");
for (const i of c.raw) {
const authored =
i.authored === ABSENT ? "absent" : JSON.stringify(i.authored);
flags.push(
`RAW ${i.type}.${i.attr}: ${authored} -> ${JSON.stringify(i.got)} (expected ${JSON.stringify(i.expected)})`,
);
}
if (c.canonical) {
flags.push(
`CANON @ ${c.canonical.path}: ${JSON.stringify(c.canonical.a)} vs ${JSON.stringify(c.canonical.b)}`,
);
}
const status = flags.length === 0 ? "stable" : flags.join("; ");
lines.push(` [${c.label}] ${status}`);
}
return lines.join("\n");
}
@@ -0,0 +1,164 @@
import { describe, expect, it } from "vitest";
import {
runStabilityMatrix,
unstableCombos,
formatReport,
runConvergenceCase,
convergenceCasesFor,
convergenceOk,
formatConvergence,
type NodeStabilitySpec,
} from "./roundtrip-stability.helper.js";
// ---------------------------------------------------------------------------
// Round-trip STABILITY matrix for image + the media family.
//
// Guards the "empty-string-vs-absent" churn class (GS-EDIT-REVERT family): a
// stored node authored WITHOUT a string attr (alt/title/caption/aria-label/...)
// must not gain a phantom `attr: ""` after `markdownToProseMirror(convert…)`.
// Each spec sweeps the at-risk string attrs at DEFAULT (absent) and at a real
// NON-default value; the helper asserts both the RAW round-trip (attrs equal the
// input's, modulo the documented numeric width/height/size/aspectRatio -> string
// coercion) and the CANONICAL round-trip (canonical forms deep-equal).
//
// The image + media family share the `align !== "center"` predicate and the
// `<!--name {…}-->` comment machinery, so one matrix guards the shared class.
// align is NOT part of this class (it round-trips correctly) and is not swept.
// ---------------------------------------------------------------------------
const SPECS: NodeStabilitySpec[] = [
{
// Image carries the most at-risk string attrs. `alt` is the one marked
// materializes as `<img alt="">` on `![](src)` import (the real bug); title
// and caption are covered as the same class. attachmentId is a string attr
// that must stay absent when unset (control).
type: "image",
baseAttrs: { src: "/i.png" },
attrMatrix: [
{ attr: "alt", default: undefined, nonDefault: "a real alt text", emptyStringClass: true },
{ attr: "title", default: undefined, nonDefault: "a real title", emptyStringClass: true },
{ attr: "caption", default: undefined, nonDefault: "a real caption" },
{ attr: "attachmentId", default: undefined, nonDefault: "att-42" },
],
},
{
// Video's `alt` rides the `aria-label` attribute (media aria-label at risk).
type: "video",
baseAttrs: { src: "/v.mp4" },
attrMatrix: [
{ attr: "alt", default: undefined, nonDefault: "a clip", emptyStringClass: true },
{ attr: "attachmentId", default: undefined, nonDefault: "att-1" },
],
},
{
// Audio carries no alt/title; attachmentId is its only optional string attr.
type: "audio",
baseAttrs: { src: "/a.mp3" },
attrMatrix: [
{ attr: "attachmentId", default: undefined, nonDefault: "att-2" },
],
},
{
// pdf: link-form media. `name` (filename) is its at-risk string attr.
type: "pdf",
baseAttrs: { src: "/d.pdf" },
attrMatrix: [
{ attr: "name", default: undefined, nonDefault: "report.pdf", emptyStringClass: true },
{ attr: "attachmentId", default: undefined, nonDefault: "att-3" },
],
},
{
// attachment: link-form media (file card). `name` + `mime` string attrs.
type: "attachment",
baseAttrs: { url: "/f.zip" },
attrMatrix: [
{ attr: "name", default: undefined, nonDefault: "bundle.zip", emptyStringClass: true },
{ attr: "mime", default: undefined, nonDefault: "application/zip", emptyStringClass: true },
{ attr: "attachmentId", default: undefined, nonDefault: "att-4" },
],
},
{
// embed: link-form media. `provider` is its at-risk string attr (schema
// default ""). embed's numeric width/height defaults (800/600) are a SEPARATE,
// documented limitation OUTSIDE the empty-string class: they are not in
// canonicalize's KNOWN_DEFAULTS, so an ABSENT width/height re-imports as the
// 800/600 default and diverges canonically (see the note in canonicalize.ts).
// That is canonicalize-owned and out of scope here, so we author the
// dimensions at their defaults (as real editor embeds carry them) to keep this
// guard focused on the empty-string/provider class.
// provider's schema default is "" (NOT null), so a re-imported "" is the
// correct value, not a phantom — it is outside the null-default empty-string
// class. We author it at its "" default (the default pick) so the sweep still
// asserts a non-default provider ("youtube") round-trips, without tripping the
// canonicalize KNOWN_DEFAULTS gap for embed's non-null defaults.
type: "embed",
baseAttrs: { src: "https://example.com/x", width: 800, height: 600 },
attrMatrix: [
{ attr: "provider", default: "", nonDefault: "youtube" },
],
},
{
// drawio: image-form diagram. `title` + `alt` string attrs (data-title/-alt).
type: "drawio",
baseAttrs: { src: "blob:drawio" },
attrMatrix: [
{ attr: "title", default: undefined, nonDefault: "flow chart", emptyStringClass: true },
{ attr: "alt", default: undefined, nonDefault: "an alt", emptyStringClass: true },
{ attr: "attachmentId", default: undefined, nonDefault: "att-5" },
],
},
{
// excalidraw: image-form diagram, same shared diagramAttributes set.
type: "excalidraw",
baseAttrs: { src: "blob:excalidraw" },
attrMatrix: [
{ attr: "title", default: undefined, nonDefault: "sketch", emptyStringClass: true },
{ attr: "alt", default: undefined, nonDefault: "an alt", emptyStringClass: true },
{ attr: "attachmentId", default: undefined, nonDefault: "att-6" },
],
},
];
describe("round-trip stability matrix (image + media family)", () => {
for (const spec of SPECS) {
it(`${spec.type}: no attr materializes an empty-string / phantom value`, async () => {
const report = await runStabilityMatrix(spec);
const unstable = unstableCombos(report);
// On failure, print the WHOLE matrix so which (attr, value) combos are
// unstable is legible.
expect(unstable, `\n${formatReport(report)}\n`).toEqual([]);
});
}
});
// ---------------------------------------------------------------------------
// THIRD STATE: an attr EXPLICITLY stored as a literal "" (GS-EDIT-REVERT: a user
// typed alt/title/name/... then deleted it, so Tiptap persisted `attr: ""` — a
// value DISTINCT from "attr was never set"). Unlike the absent case above, this
// state is NOT first-pass byte-stable: the fix's `"" -> default` coercion is a
// deliberate ONE-TIME normalization on the FIRST sync round-trip, stable
// thereafter. We therefore assert a DIFFERENT contract — "converges to default
// on pass 1, then idempotent from pass 2 on" — for every empty-string-class attr
// across the whole node family (image/video/pdf/attachment/drawio/excalidraw).
//
// IMPORTANT for a future sync/QA pass: the pass-1 `"" -> null` diff is the
// converter canon, not corruption. It appears at most once per affected node and
// must NOT be flagged as "the converter is losing/corrupting page data".
// ---------------------------------------------------------------------------
describe("round-trip third state: explicit empty string converges once, then idempotent", () => {
for (const spec of SPECS) {
for (const attr of convergenceCasesFor(spec)) {
it(`${spec.type}.${attr}: "" normalizes to default on pass 1, byte-stable from pass 2`, async () => {
const r = await runConvergenceCase(spec, attr);
// Pass 1 must converge "" -> the schema default (the one-time diff) and
// pass 2 (roundtrip of pass-1 output) must be byte-stable. formatConvergence
// prints exactly which half failed.
expect(convergenceOk(r), `\n${formatConvergence(r)}\n`).toBe(true);
// Spell the contract out explicitly so the intent is legible in the test:
expect(r.convergedToDefault, `\n${formatConvergence(r)}\n`).toBe(true);
expect(r.firstPassValue).toEqual(r.expectedDefault);
expect(r.secondPassDivergence, `\n${formatConvergence(r)}\n`).toBeNull();
});
}
}
});