Compare commits

..

1 Commits

Author SHA1 Message Date
agent_coder 51260793c0 fix(ui/toasts): глобальная видимость тостов + перенос в top-center (#517)
Тост-уведомления Mantine сливались с фоном: у бесцветных тостов фон
карточки == var(--mantine-color-body) (белый, как страница) при слабой
тени, поэтому на белых страницах у карточки не было видимого края. Плюс
тосты всплывали снизу по центру и перекрывали контент.

Чиним глобально, без правок в 213 местах вызова:

- notification-overrides.css: каждому тосту даём тонированный по типу фон,
  рамку с контрастом WCAG >= 3:1 и усиленную тень (shadow-xl). Селектор
  [data-mantine-color-scheme=...] .mantine-Notification-root имеет
  специфичность (0,2,0) и стабильно бьёт правила Mantine (0,1,0) (у Mantine
  атрибут схемы обёрнут в :where()) — независимо от порядка стилей. Тон/рамка
  идут от --notification-color (определён на том же элементе), поэтому следуют
  типу тоста и покрывают loading/импортный тост (полосы-акцента нет — несут
  тон+рамка+тень+цветной спиннер). Обе темы; текст-с-заголовком поднят до
  gray-7 ради AA-контраста на тонированном фоне.

- main.tsx: position bottom-center -> top-center. Вертикальное смещение
  контейнера ниже верхней хромы делаем НЕ инлайн-стилем, а CSS-правилом со
  скоупом по позиции: Mantine рендерит все шесть позиционных контейнеров
  одновременно, и корневой style-проп ушёл бы во все шесть — нижним (bottom:16)
  добавился бы top:96 → position:fixed + оба края + height:auto растянули бы их
  на весь вьюпорт; у корня нет pointer-events:none/фона → прозрачные оверлеи
  z-10000 перехватывали бы клики по всей странице.

- notification-overrides.css: .mantine-Notifications-root[data-position^='top']
  { top:96px } (шапка 45 + опц. тулбар 45 + зазор). Скоуп ^='top' смещает
  только верхние контейнеры; нижние остаются height:0 и кликов не перехватывают.
  Специфичность (0,2,0) бьёт mantine top:16px (0,1,0), тост z-10000 стоит ниже
  шапки/тулбара (z-99) и их не перекрывает.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 01:14:38 +03:00
42 changed files with 289 additions and 2530 deletions
+1 -28
View File
@@ -129,20 +129,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- **A drifted comment suggestion can be re-synced instead of failing forever
with a 409.** A suggestion whose stored anchor no longer matched the live
document used to reject every apply attempt with an unrecoverable conflict; a
new resync path re-reads the live anchor so the suggestion applies against the
current text, and orphaned anchors (whose marked run was deleted) are
reconciled rather than left blocking. (#496)
- **Save intentional page versions.** Press `Cmd/Ctrl+S` (or use the page menu)
to save a named version of a page. The history panel now distinguishes
intentional versions (a "Saved" / "Agent version" badge) from automatic
snapshots, dims autosaves, and offers an "Only versions" filter. Automatic
snapshots switched from a fixed interval to a trailing idle-flush with a
max-wait ceiling, and a boundary snapshot is pinned whenever the editing source
changes (e.g. a person's edits followed by the AI agent). (#370)
- **Place several images side by side in a row.** A new "Inline (side by
side)" alignment mode in the image bubble menu renders consecutive inline
images as a row that wraps onto the next line on narrow screens. The row is
@@ -366,20 +352,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
body-timeout so a legitimate >1-min idle between the model's tool calls no
longer breaks a long-lived SSE socket (new `AI_MCP_SSE_BODY_TIMEOUT_MS`, default
10 min; see `.env.example`). (#489)
- **Decisions on comment suggestions now leave a durable audit record.**
Applying or dismissing a comment suggestion hard-deletes the (childless)
subject comment, so the only surviving trace of who decided what is the audit
event — but the audit trail was wired to a Noop service that silently
swallowed every event. The trail is now DB-backed, so
`comment.suggestion_applied` / `comment.suggestion_dismissed` (and the other
comment-decision events) persist to the `audit` table and can be reviewed
after the comment is gone. A persistence failure is still swallowed with a
warning so it never breaks the originating request. (#496)
- **Applying a comment suggestion no longer strips the replaced run's inline
formatting.** The suggested text was re-inserted carrying only the comment
anchor mark, silently dropping bold/italic/code/link on the affected run; the
prevailing formatting of the replaced run is now carried onto the applied
text. (#496)
- **The server no longer runs out of heap during long autonomous agent runs.** A
new pnpm patch on `ai@6.0.134` stops the SDK from building a cumulative
snapshot of the ENTIRE turn text on every streamed text-delta when no output
@@ -1418,14 +1418,5 @@
"The commented text changed since this suggestion was made; it was not applied.": "The commented text changed since this suggestion was made; it was not applied.",
"Dismiss": "Dismiss",
"Suggestion dismissed": "Suggestion dismissed",
"Failed to dismiss suggestion": "Failed to dismiss suggestion",
"Save version": "Save version",
"Ctrl+S": "Ctrl+S",
"Version saved": "Version saved",
"Already saved as the latest version": "Already saved as the latest version",
"Agent version": "Agent version",
"Boundary": "Boundary",
"Autosave": "Autosave",
"Only versions": "Only versions",
"No saved versions yet.": "No saved versions yet."
"Failed to dismiss suggestion": "Failed to dismiss suggestion"
}
@@ -1281,14 +1281,5 @@
"The commented text changed since this suggestion was made; it was not applied.": "Прокомментированный текст изменился после создания предложения; оно не было применено.",
"Dismiss": "Не применять",
"Suggestion dismissed": "Предложение отклонено",
"Failed to dismiss suggestion": "Не удалось отклонить предложение",
"Save version": "Сохранить версию",
"Ctrl+S": "Ctrl+S",
"Version saved": "Версия сохранена",
"Already saved as the latest version": "Уже сохранено как последняя версия",
"Agent version": "Версия агента",
"Boundary": "Граница",
"Autosave": "Автосейв",
"Only versions": "Только версии",
"No saved versions yet.": "Пока нет сохранённых версий."
"Failed to dismiss suggestion": "Не удалось отклонить предложение"
}
@@ -3,20 +3,11 @@ import { atom } from "jotai";
// import would drag the whole @tiptap/core engine into the eager graph of every
// shell component that reads one of these atoms.
import type { Editor } from "@tiptap/core";
import type { HocuspocusProvider } from "@hocuspocus/provider";
import { PageEditMode } from "@/features/user/types/user.types.ts";
import type { DictationUnavailableReason } from "@/features/dictation/dictation-status";
export const pageEditorAtom = atom<Editor | null>(null);
// #370 — the active page's collab provider, published by the page editor so the
// header menu can emit the "save-version" stateless signal (Cmd+S / button).
// Null when the page is read-only / collab isn't connected. A typed initial
// value (rather than an explicit generic) keeps jotai's overload resolution on
// the writable PrimitiveAtom branch.
const initialCollabProvider: HocuspocusProvider | null = null;
export const collabProviderAtom = atom(initialCollabProvider);
export const titleEditorAtom = atom<Editor | null>(null);
export const readOnlyEditorAtom = atom<Editor | null>(null);
@@ -31,18 +31,11 @@ import { useAtom, useAtomValue, useSetAtom } from "jotai";
import useCollaborationUrl from "@/features/editor/hooks/use-collaboration-url";
import { currentUserAtom } from "@/features/user/atoms/current-user-atom";
import {
collabProviderAtom,
currentPageEditModeAtom,
dictationAvailabilityAtom,
pageEditorAtom,
yjsConnectionStatusAtom,
} from "@/features/editor/atoms/editor-atoms";
import { notifications } from "@mantine/notifications";
import {
VERSION_SAVED_MESSAGE_TYPE,
type VersionSavedMessage,
saveVersionPending,
} from "@/features/page-history/version-messages";
import { asideStateAtom } from "@/components/layouts/global/hooks/atoms/sidebar-atom";
import {
activeCommentIdAtom,
@@ -131,7 +124,6 @@ export default function PageEditor({
const [currentUser] = useAtom(currentUserAtom);
const [, setEditor] = useAtom(pageEditorAtom);
const setCollabProvider = useSetAtom(collabProviderAtom);
const [, setAsideState] = useAtom(asideStateAtom);
const [, setActiveCommentId] = useAtom(activeCommentIdAtom);
const [showCommentPopup, setShowCommentPopup] = useAtom(showCommentPopupAtom);
@@ -189,24 +181,6 @@ export default function PageEditor({
const onStatelessHandler = ({ payload }: onStatelessParameters) => {
try {
const message = JSON.parse(payload);
// #370 — a version was saved somewhere; live-refresh the history panel
// on every client. Only the client that pressed Save (tracked by the
// module-level flag) shows the confirmation toast.
if (message?.type === VERSION_SAVED_MESSAGE_TYPE) {
const versionMsg = message as VersionSavedMessage;
queryClient.invalidateQueries({
queryKey: ["page-history-list"],
});
if (saveVersionPending.current) {
saveVersionPending.current = false;
notifications.show({
message: versionMsg.alreadySaved
? t("Already saved as the latest version")
: t("Version saved"),
});
}
return;
}
if (message?.type !== "page.updated" || !message.updatedAt) return;
const pageData = queryClient.getQueryData<IPage>(["pages", slugId]);
if (pageData) {
@@ -264,16 +238,12 @@ export default function PageEditor({
local.on("synced", onLocalSyncedHandler);
providersRef.current = { socket, local, remote };
// #370 — publish the provider so the header menu can emit save-version.
setCollabProvider(remote);
setProvidersReady(true);
} else {
setCollabProvider(providersRef.current.remote);
setProvidersReady(true);
}
// Only destroy on final unmount
return () => {
setCollabProvider(null);
providersRef.current?.socket.destroy();
providersRef.current?.remote.destroy();
providersRef.current?.local.destroy();
@@ -1,11 +1,4 @@
import {
Text,
Group,
UnstyledButton,
Avatar,
Tooltip,
Badge,
} from "@mantine/core";
import { Text, Group, UnstyledButton, Avatar, Tooltip } from "@mantine/core";
import { CustomAvatar } from "@/components/ui/custom-avatar.tsx";
import { AgentAvatarStack } from "@/components/ui/agent-avatar-stack.tsx";
import { formattedDate } from "@/lib/time";
@@ -14,59 +7,36 @@ import clsx from "clsx";
import { IPageHistory } from "@/features/page-history/types/page.types";
import { memo, useCallback } from "react";
import { useSetAtom } from "jotai";
import { useTranslation } from "react-i18next";
import { historyAtoms } from "@/features/page-history/atoms/history-atoms.ts";
const MAX_VISIBLE_AVATARS = 5;
/**
* #370 — map a snapshot's intentionality tier to its badge. `version: true`
* marks the intentional points (manual / agent); autosaves (boundary / idle /
* legacy null) are non-versions and get dimmed in the list.
*/
type HistoryKindMeta = { labelKey: string; color: string; version: boolean };
export function historyKindMeta(kind?: string | null): HistoryKindMeta {
switch (kind) {
case "manual":
return { labelKey: "Saved", color: "blue", version: true };
case "agent":
return { labelKey: "Agent version", color: "violet", version: true };
case "boundary":
return { labelKey: "Boundary", color: "gray", version: false };
default: // "idle" | null | undefined (legacy autosave)
return { labelKey: "Autosave", color: "gray", version: false };
}
}
interface HistoryItemProps {
historyItem: IPageHistory;
// The previous snapshot for diff/restore is resolved by id from the FULL list
// in the parent (resolvePrevSnapshotId), so the item only needs to report its
// own id — never a list index (which would be the filtered-view index).
onSelect: (id: string) => void;
onHover?: (id: string) => void;
index: number;
onSelect: (id: string, index: number) => void;
onHover?: (id: string, index: number) => void;
onHoverEnd?: () => void;
isActive: boolean;
}
const HistoryItem = memo(function HistoryItem({
historyItem,
index,
onSelect,
onHover,
onHoverEnd,
isActive,
}: HistoryItemProps) {
const setHistoryModalOpen = useSetAtom(historyAtoms);
const { t } = useTranslation();
const kindMeta = historyKindMeta(historyItem.kind);
const handleClick = useCallback(() => {
onSelect(historyItem.id);
}, [onSelect, historyItem.id]);
onSelect(historyItem.id, index);
}, [onSelect, historyItem.id, index]);
const handleMouseEnter = useCallback(() => {
onHover?.(historyItem.id);
}, [onHover, historyItem.id]);
onHover?.(historyItem.id, index);
}, [onHover, historyItem.id, index]);
const contributors = historyItem.contributors;
const hasContributors = contributors && contributors.length > 0;
@@ -79,20 +49,8 @@ const HistoryItem = memo(function HistoryItem({
onMouseEnter={handleMouseEnter}
onMouseLeave={onHoverEnd}
className={clsx(classes.history, { [classes.active]: isActive })}
// #370 — dim autosnapshots so intentional versions stand out.
style={{ opacity: kindMeta.version ? 1 : 0.55 }}
>
<Group gap={6} wrap="nowrap" justify="space-between">
<Text size="sm">{formattedDate(new Date(historyItem.createdAt))}</Text>
<Badge
size="xs"
radius="sm"
variant={kindMeta.version ? "filled" : "light"}
color={kindMeta.color}
>
{t(kindMeta.labelKey)}
</Badge>
</Group>
<Text size="sm">{formattedDate(new Date(historyItem.createdAt))}</Text>
<Group gap={6} wrap="nowrap" mt={4}>
{hasContributors ? (
@@ -2,16 +2,14 @@ import {
usePageHistoryListQuery,
prefetchPageHistory,
} from "@/features/page-history/queries/page-history-query";
import HistoryItem, {
historyKindMeta,
} from "@/features/page-history/components/history-item";
import HistoryItem from "@/features/page-history/components/history-item";
import {
activeHistoryIdAtom,
activeHistoryPrevIdAtom,
historyAtoms,
} from "@/features/page-history/atoms/history-atoms";
import { useAtom, useSetAtom } from "jotai";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { useCallback, useEffect, useMemo, useRef } from "react";
import {
Button,
ScrollArea,
@@ -19,12 +17,9 @@ import {
Divider,
Loader,
Center,
Switch,
Text,
} from "@mantine/core";
import { useTranslation } from "react-i18next";
import { useHistoryRestore } from "@/features/page-history/hooks";
import { resolvePrevSnapshotId } from "@/features/page-history/utils/resolve-prev-snapshot";
const PREFETCH_DELAY_MS = 150;
@@ -52,22 +47,6 @@ function HistoryList({ pageId }: Props) {
[pageHistoryData],
);
// #370 — "only versions" filter: hide autosnapshots (idle/boundary/legacy
// null), keep only intentional points (manual/agent). Filtering is over the
// already-loaded pages; the diff/restore still targets the true previous
// snapshot, so items carry their index within the FULL list.
const [onlyVersions, setOnlyVersions] = useState(false);
// Reuse historyKindMeta().version — the SAME predicate the badge (HistoryItem)
// uses to mark intentional points — so the "Only versions" filter and the badge
// can never drift apart when a future intentional kind is added.
const visibleItems = useMemo(
() =>
onlyVersions
? historyItems.filter((item) => historyKindMeta(item.kind).version)
: historyItems,
[historyItems, onlyVersions],
);
const loadMoreRef = useRef<HTMLDivElement>(null);
const prefetchTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
@@ -81,13 +60,11 @@ function HistoryList({ pageId }: Props) {
}, []);
const handleHover = useCallback(
(historyId: string) => {
(historyId: string, index: number) => {
clearPrefetchTimeout();
prefetchTimeoutRef.current = setTimeout(() => {
prefetchPageHistory(historyId);
// The true previous snapshot in the FULL list (not the previous visible
// one under the "only versions" filter).
const prevId = resolvePrevSnapshotId(historyItems, historyId);
const prevId = historyItems[index + 1]?.id;
if (prevId) {
prefetchPageHistory(prevId);
}
@@ -101,11 +78,9 @@ function HistoryList({ pageId }: Props) {
}, [clearPrefetchTimeout]);
const handleSelect = useCallback(
(id: string) => {
(id: string, index: number) => {
setActiveHistoryId(id);
// Baseline = true previous snapshot in the FULL list, so the "only
// versions" filter never diffs/restores against the wrong item.
setActiveHistoryPrevId(resolvePrevSnapshotId(historyItems, id));
setActiveHistoryPrevId(historyItems[index + 1]?.id ?? "");
},
[historyItems, setActiveHistoryId, setActiveHistoryPrevId],
);
@@ -153,27 +128,12 @@ function HistoryList({ pageId }: Props) {
return (
<div>
<Group px="xs" py={6} justify="flex-end">
<Switch
size="xs"
checked={onlyVersions}
onChange={(e) => setOnlyVersions(e.currentTarget.checked)}
label={t("Only versions")}
/>
</Group>
<ScrollArea h={620} w="100%" type="scroll" scrollbarSize={5}>
{onlyVersions && visibleItems.length === 0 && (
<Center py="md">
<Text size="sm" c="dimmed">
{t("No saved versions yet.")}
</Text>
</Center>
)}
{visibleItems.map((historyItem) => (
{historyItems.map((historyItem, index) => (
<HistoryItem
key={historyItem.id}
historyItem={historyItem}
index={index}
onSelect={handleSelect}
onHover={handleHover}
onHoverEnd={clearPrefetchTimeout}
@@ -24,10 +24,6 @@ export interface IPageHistory {
updatedAt: string;
lastUpdatedBy: IPageHistoryUser;
contributors?: IPageHistoryUser[];
// #370 — intentionality tier: 'manual'/'agent' are versions (intentional
// points), 'idle'/'boundary' are autosnapshots; null/undefined = legacy
// autosave. Derived server-side, drives the history badge + "versions" filter.
kind?: "manual" | "agent" | "idle" | "boundary" | null;
// Provenance markers copied off the page row when the snapshot was saved.
// `'agent'` marks a version written by the AI agent; `lastUpdatedAiChatId`
// (when present) deep-links to the chat that produced the edit.
@@ -1,42 +0,0 @@
import { describe, it, expect } from "vitest";
import { resolvePrevSnapshotId } from "./resolve-prev-snapshot";
// #370 F4 — the risky client path: with the "only versions" filter active, diff
// and restore must still baseline against the TRUE previous snapshot in the FULL
// list, never the previous VISIBLE version (which would skip the autosnapshots
// between two versions). These pin that the resolution is by FULL-list order.
describe("resolvePrevSnapshotId", () => {
// Newest-first, as the history list stores it: a version, then two autosaves,
// then an older version.
const full = [
{ id: "v2", kind: "manual" },
{ id: "a2", kind: "idle" },
{ id: "a1", kind: "boundary" },
{ id: "v1", kind: "manual" },
{ id: "a0", kind: null },
];
it("returns the immediate FULL-list successor, not the previous visible version", () => {
// Selecting v2 while filtered to versions-only must baseline against a2 (the
// real chronological predecessor), NOT v1 (the previous visible version).
expect(resolvePrevSnapshotId(full, "v2")).toBe("a2");
});
it("resolves an autosnapshot's predecessor by full-list order", () => {
expect(resolvePrevSnapshotId(full, "a1")).toBe("v1");
});
it("returns '' for the oldest item (no predecessor)", () => {
expect(resolvePrevSnapshotId(full, "a0")).toBe("");
});
it("returns '' for an id not in the list", () => {
expect(resolvePrevSnapshotId(full, "missing")).toBe("");
});
it("does not depend on a filtered subset — same result whatever is visible", () => {
// The helper only ever sees the full list; a filtered view cannot change the
// baseline it computes.
expect(resolvePrevSnapshotId(full, "v1")).toBe("a0");
});
});
@@ -1,22 +0,0 @@
/**
* #370 — resolve the TRUE previous snapshot for a history item.
*
* The history panel can be filtered to "only versions" (manual/agent), but diff
* and restore must always compare against the immediately-preceding snapshot in
* the FULL, unfiltered list — NOT the previous VISIBLE item. Comparing against
* the previous visible version would silently skip the autosnapshots between two
* versions and diff/restore the wrong baseline.
*
* Given the full (newest-first) list and an item id, this returns the id of the
* item right after it in the full list (its chronological predecessor), or "" if
* it is the oldest / not found. Pure and list-order-preserving so it can be unit
* tested without mounting the component.
*/
export function resolvePrevSnapshotId(
fullItems: ReadonlyArray<{ id: string }>,
id: string,
): string {
const index = fullItems.findIndex((item) => item.id === id);
if (index === -1) return "";
return fullItems[index + 1]?.id ?? "";
}
@@ -1,28 +0,0 @@
/**
* #370 — page-version stateless wire formats. Kept in one place so the client
* emitter (Save hotkey / button) and the client listener (page-editor) agree
* with the server (PersistenceExtension) on the message shapes.
*/
/** Client → server: "save a version now". The server derives the tier
* (manual/agent) from the signed connection actor, never from this payload. */
export const SAVE_VERSION_MESSAGE_TYPE = "save-version";
/** Server → all clients: a version was saved (or promoted / already existed). */
export const VERSION_SAVED_MESSAGE_TYPE = "version.saved";
export interface VersionSavedMessage {
type: typeof VERSION_SAVED_MESSAGE_TYPE;
historyId: string;
kind: "manual" | "agent";
/** True when the latest snapshot was already a manual version (a no-op save). */
alreadySaved: boolean;
}
/**
* Cross-component coordination flag so only the client that pressed Save shows
* the confirmation toast, while every other client silently refreshes its
* history panel on the broadcast. A module-level ref avoids stale-closure
* pitfalls in the editor's long-lived stateless handler.
*/
export const saveVersionPending = { current: false };
@@ -3,7 +3,6 @@ import {
IconArrowRight,
IconArrowsHorizontal,
IconClockHour4,
IconDeviceFloppy,
IconDots,
IconEye,
IconEyeOff,
@@ -18,7 +17,7 @@ import {
IconTrash,
IconWifiOff,
} from "@tabler/icons-react";
import React, { useCallback, useEffect, useRef, useState } from "react";
import React, { useEffect, useRef, useState } from "react";
import { useAsideTriggerProps } from "@/hooks/use-toggle-aside.tsx";
import { useAtom, useAtomValue } from "jotai";
import { historyAtoms } from "@/features/page-history/atoms/history-atoms.ts";
@@ -40,14 +39,9 @@ import { Trans, useTranslation } from "react-i18next";
import ExportModal from "@/components/common/export-modal";
import { convertProseMirrorToMarkdown } from "@docmost/prosemirror-markdown/browser";
import {
collabProviderAtom,
pageEditorAtom,
yjsConnectionStatusAtom,
} from "@/features/editor/atoms/editor-atoms.ts";
import {
SAVE_VERSION_MESSAGE_TYPE,
saveVersionPending,
} from "@/features/page-history/version-messages.ts";
import { formattedDate } from "@/lib/time.ts";
import { PageEditModeToggle } from "@/features/user/components/page-state-pref.tsx";
import MovePageModal from "@/features/page/components/move-page-modal.tsx";
@@ -78,34 +72,9 @@ export default function PageHeaderMenu({ readOnly }: PageHeaderMenuProps) {
});
const isDeleted = !!page?.deletedAt;
const [workspace] = useAtom(workspaceAtom);
const collabProvider = useAtomValue(collabProviderAtom);
// Community public-sharing entry point (replaces the removed EE PageShareModal)
const workspaceSharingDisabled = workspace?.settings?.sharing?.disabled === true;
// #370 — explicit "save a version" (Cmd+S / Save button). One path for the
// human; the server derives the tier from the signed actor. Readers can't save
// (the button is hidden and the collab connection is read-only server-side).
const handleSaveVersion = useCallback(() => {
if (readOnly || !collabProvider) return;
// Flag this client as the initiator so only it shows the confirmation toast;
// a safety timeout clears it if no broadcast comes back (e.g. offline).
saveVersionPending.current = true;
window.setTimeout(() => {
saveVersionPending.current = false;
}, 5000);
collabProvider.sendStateless(
JSON.stringify({ type: SAVE_VERSION_MESSAGE_TYPE }),
);
}, [readOnly, collabProvider]);
// mod+S must also block the browser's "Save page" dialog. `triggerOnContent-
// Editable` + empty ignore-list so it fires while typing in the editor/title.
useHotkeys(
[["mod+S", handleSaveVersion, { preventDefault: true }]],
[],
true,
);
useHotkeys(
[
[
@@ -164,16 +133,15 @@ export default function PageHeaderMenu({ readOnly }: PageHeaderMenuProps) {
</ActionIcon>
</Tooltip>
<PageActionMenu readOnly={readOnly} onSaveVersion={handleSaveVersion} />
<PageActionMenu readOnly={readOnly} />
</>
);
}
interface PageActionMenuProps {
readOnly?: boolean;
onSaveVersion?: () => void;
}
function PageActionMenu({ readOnly, onSaveVersion }: PageActionMenuProps) {
function PageActionMenu({ readOnly }: PageActionMenuProps) {
const { t } = useTranslation();
const [, setHistoryModalOpen] = useAtom(historyAtoms);
const clipboard = useClipboard({ timeout: 500 });
@@ -335,20 +303,6 @@ function PageActionMenu({ readOnly, onSaveVersion }: PageActionMenuProps) {
</Group>
</Menu.Item>
{!readOnly && (
<Menu.Item
leftSection={<IconDeviceFloppy size={16} />}
onClick={onSaveVersion}
rightSection={
<Text size="xs" c="dimmed">
{t("Ctrl+S")}
</Text>
}
>
{t("Save version")}
</Menu.Item>
)}
<Menu.Item
leftSection={<IconHistory size={16} />}
onClick={openHistoryModal}
+10 -1
View File
@@ -3,6 +3,7 @@ import "@mantine/spotlight/styles.css";
import "@mantine/notifications/styles.css";
import '@mantine/dates/styles.css';
import "@/styles/a11y-overrides.css";
import "@/styles/notification-overrides.css";
import ReactDOM from "react-dom/client";
import App from "./App.tsx";
@@ -47,7 +48,15 @@ function renderApp() {
<MantineProvider theme={theme} cssVariablesResolver={mantineCssResolver}>
<ModalsProvider>
<QueryClientProvider client={queryClient}>
<Notifications position="bottom-center" limit={3} zIndex={10000} />
{/* top-center: toasts sit in the top of the viewport, in the line
of sight, and no longer cover centered content (e.g. "Load
more"). The below-chrome vertical offset is applied via a
position-scoped CSS rule in notification-overrides.css (NOT an
inline `style`): Mantine renders all six position containers at
once and an inline root style would land on every one, giving the
bottom-* containers both top+bottom → full-viewport transparent
overlays that swallow clicks. */}
<Notifications position="top-center" limit={3} zIndex={10000} />
<HelmetProvider>
{/* Root boundary above every lazy route's Suspense: a stale-chunk
404 after a deploy is caught and recovered here instead of
@@ -0,0 +1,64 @@
/*
* Toast (Mantine Notification) visibility overrides.
* Mantine renders colorless toasts on --mantine-color-body (== the page
* background: white in light mode) with a faint shadow, so on white pages the
* card has no visible edge. These rules give every toast a type-tinted
* background, a WCAG-checked border and a stronger shadow so it separates from
* the page. The [data-mantine-color-scheme] + static-class selector (0,2,0)
* beats Mantine's own (0,1,0) rules regardless of stylesheet order (Mantine's
* bg/border rules wrap the scheme attribute in :where(), so they stay (0,1,0)).
* --notification-color is defined on the same element (defaults to primary,
* set per `color` prop), so tint/border follow the toast type. This also covers
* the loading/import toast (no accent bar, since the spinner takes the icon
* slot): its visibility comes from tone + border + shadow + the colored spinner.
*/
/*
* Push the top-anchored toast containers below the top chrome (fixed 45px
* header + optional 45px format toolbar + ~6px gap) so a toast (z-index 10000)
* neither covers nor intercepts clicks on the header/toolbar (both z-index 99).
*
* Scoped to [data-position^='top'] on purpose. Mantine renders ALL SIX position
* containers simultaneously (`position` only routes toasts into one via the
* store); the root `style` prop would be applied to every one of them by
* getStyles("root"). A blanket `top` would land on the bottom-* containers too
* (which carry `bottom:16px`) → position:fixed + both edges + height:auto makes
* them stretch the full viewport height, and the container root has neither
* pointer-events:none nor a background, so those transparent z-10000 overlays
* would swallow clicks across the whole page. Restricting to top-* leaves the
* bottom containers at height:0.
*
* Specificity: `.mantine-Notifications-root[data-position^='top']` is (0,2,0)
* (class + attribute) and beats Mantine's own top rule
* `.m_b37d9ac7:where([data-position='top-center']){top:16px}` which is (0,1,0)
* (the :where() contributes 0), regardless of stylesheet order.
*/
.mantine-Notifications-root[data-position^='top'] {
top: 96px;
}
[data-mantine-color-scheme='light'] .mantine-Notification-root {
/* ~10% type color over white: clearly off-white, text contrast preserved */
background-color: color-mix(in srgb, var(--notification-color) 10%, var(--mantine-color-white));
/* Border must clear WCAG 3:1 non-text contrast on white. The repo rejects
gray-4 for this (a11y-overrides.css); gray-6 base (~3.32:1) darkened by the
type color stays >= 3:1. */
border: 1px solid color-mix(in srgb, var(--notification-color) 45%, var(--mantine-color-gray-6));
box-shadow: var(--mantine-shadow-xl);
}
[data-mantine-color-scheme='dark'] .mantine-Notification-root {
/* Dark page (dark-7/8) vs toast (dark-6) already separate a little; border +
shadow carry the type cue here (a 7% dark tint was near-invisible). */
background-color: color-mix(in srgb, var(--notification-color) 14%, var(--mantine-color-dark-6));
border: 1px solid color-mix(in srgb, var(--notification-color) 45%, var(--mantine-color-dark-3));
box-shadow: var(--mantine-shadow-xl);
}
/* Mantine's message-with-title color is gray-6 (#868e96, already only ~3.32:1
on white — below AA 4.5:1); the new tint pushes it lower. Bump to gray-7 to
keep multi-line colored toasts readable, consistent with the repo's existing
WCAG tuning (theme.ts already bumps this same gray-6 up elsewhere). */
[data-mantine-color-scheme='light'] .mantine-Notification-description[data-with-title] {
color: var(--mantine-color-gray-7);
}
+2 -2
View File
@@ -25,7 +25,7 @@ import { CacheModule } from '@nestjs/cache-manager';
import KeyvRedis from '@keyv/redis';
import { LoggerModule } from './common/logger/logger.module';
import { ClsModule } from 'nestjs-cls';
import { AuditModule } from './integrations/audit/audit.module';
import { NoopAuditModule } from './integrations/audit/audit.module';
import { ThrottleModule } from './integrations/throttle/throttle.module';
import { McpModule } from './integrations/mcp/mcp.module';
import { SandboxModule } from './integrations/sandbox/sandbox.module';
@@ -55,7 +55,7 @@ try {
middleware: { mount: true },
}),
LoggerModule,
AuditModule,
NoopAuditModule,
CoreModule,
DatabaseModule,
EnvironmentModule,
+4 -30
View File
@@ -1,36 +1,10 @@
export const HISTORY_INTERVAL = 5 * 60 * 1000;
export const HISTORY_FAST_INTERVAL = 60 * 1000;
export const HISTORY_FAST_THRESHOLD = 5 * 60 * 1000;
// #348 — debounce window for the per-page RAG re-embed job. Repeated saves
// within this window collapse to a single delayed job (coalesced by a stable
// jobId), so active editing does not pile up expensive re-embeds (external API
// + page_embeddings rewrite, concurrency 1). The worker reads the CURRENT page
// state at run time, so the last content within the window wins.
export const EMBED_DEBOUNCE_MS = 30 * 1000;
/**
* #370 — page-history intentionality tiers. Domain of `page_history.kind`.
* - 'manual' / 'agent' → Tier 1 versions (intentional points)
* - 'idle' / 'boundary' → Tier 0 autosnapshots (safety net)
* A legacy `null` kind is treated as an autosave.
*/
export type PageHistoryKind = 'manual' | 'agent' | 'idle' | 'boundary';
/**
* #370 — trailing idle-flush windows. A page's pending idle snapshot is
* re-armed on every store and fires this long after edits go quiet, so a burst
* of edits collapses into a single autosnapshot instead of one-per-store. Human
* sessions are noisier and less risky, so they flush less often than the agent.
*/
export const IDLE_INTERVAL_USER = 60 * 60 * 1000; // 60m
export const IDLE_INTERVAL_AGENT = 15 * 60 * 1000; // 15m
/**
* #370 — max-wait ceiling for the idle flush. Pure trailing debounce starves the
* safety net: hocuspocus stores at least every ~45s, so a CONTINUOUS editing
* session would re-arm the trailing timer forever and never take an idle
* snapshot until edits finally go quiet (up to IDLE_INTERVAL_USER = 60m). This
* ceiling bounds the actual wait from the FIRST edit of a burst, so an idle
* snapshot fires at least this often during a long unbroken session — restoring
* a recovery point cadence closer to the old heuristic without one-per-store
* noise. Mirrors hocuspocus's own maxDebounce idea.
*/
export const IDLE_MAX_WAIT_USER = 10 * 60 * 1000; // 10m
export const IDLE_MAX_WAIT_AGENT = 5 * 60 * 1000; // 5m
@@ -1,93 +1,84 @@
import { computeHistoryJob, resolveSource } from './persistence.extension';
import {
IDLE_INTERVAL_AGENT,
IDLE_INTERVAL_USER,
IDLE_MAX_WAIT_AGENT,
IDLE_MAX_WAIT_USER,
computeHistoryJob,
resolveSource,
} from './persistence.extension';
import {
HISTORY_FAST_INTERVAL,
HISTORY_FAST_THRESHOLD,
HISTORY_INTERVAL,
} from '../constants';
// A fixed clock + fixed createdAt make pageAge deterministic.
const NOW = 1_700_000_000_000;
const PAGE_ID = '550e8400-e29b-41d4-a716-446655440000';
const page = { id: PAGE_ID };
// Build a minimal page whose age (NOW - createdAt) is exactly `ageMs`.
const pageAged = (ageMs: number) => ({
id: PAGE_ID,
createdAt: new Date(NOW - ageMs),
});
describe('computeHistoryJob (#370 — shared trailing idle pipeline)', () => {
it('human edit → user idle window, bare page.id job', () => {
// Humans and the agent now share ONE idle job per page (jobId = page.id).
// The agent's old delay=0 fast path is GONE — intentional agent points now
// arrive via the explicit save-version signal, not a zero-delay snapshot.
const { jobId, delay } = computeHistoryJob(page, 'user');
expect(delay).toBe(IDLE_INTERVAL_USER);
describe('computeHistoryJob', () => {
it('agent edit → delay MUST be 0 and job id is source-keyed', () => {
// INVARIANT (§15 H2 / persistence.extension): the agent delay MUST stay 0.
// The worker re-reads the page row at run time, so any non-zero delay risks
// snapshotting content a later human edit has already overwritten. This is
// the load-bearing assertion of this spec — do not relax it.
const { jobId, delay } = computeHistoryJob(pageAged(0), 'agent', NOW);
expect(delay).toBe(0);
expect(jobId).toBe(`${PAGE_ID}-agent`);
});
it('agent edit on an OLD page is still delay 0 (age never applies to agents)', () => {
// Even when the page is far older than the fast threshold, the agent path
// must short-circuit to 0 — age-based debounce is a human-only concern.
const { jobId, delay } = computeHistoryJob(
pageAged(HISTORY_FAST_THRESHOLD + 60_000),
'agent',
NOW,
);
expect(delay).toBe(0);
expect(jobId).toBe(`${PAGE_ID}-agent`);
});
it('human edit on a YOUNG page (age < threshold) → fast interval, bare job id', () => {
const { jobId, delay } = computeHistoryJob(
pageAged(HISTORY_FAST_THRESHOLD - 1),
'user',
NOW,
);
expect(delay).toBe(HISTORY_FAST_INTERVAL);
expect(jobId).toBe(PAGE_ID);
});
it('agent edit → agent idle window (shorter), still the bare page.id job', () => {
const { jobId, delay } = computeHistoryJob(page, 'agent');
expect(delay).toBe(IDLE_INTERVAL_AGENT);
// No `-agent` suffix anymore: the agent joins the common idle pipeline.
it('human edit on an OLD page (age > threshold) → standard interval', () => {
const { jobId, delay } = computeHistoryJob(
pageAged(HISTORY_FAST_THRESHOLD + 1),
'user',
NOW,
);
expect(delay).toBe(HISTORY_INTERVAL);
expect(jobId).toBe(PAGE_ID);
});
it('agent flushes sooner than a human', () => {
expect(IDLE_INTERVAL_AGENT).toBeLessThan(IDLE_INTERVAL_USER);
it('boundary: pageAge EXACTLY === threshold takes the slow branch (the `<` is strict)', () => {
// Off-by-one guard: the condition is `pageAge < HISTORY_FAST_THRESHOLD`, so
// an age of exactly the threshold is NOT "fast" — it must use HISTORY_INTERVAL.
const { delay } = computeHistoryJob(
pageAged(HISTORY_FAST_THRESHOLD),
'user',
NOW,
);
expect(delay).toBe(HISTORY_INTERVAL);
});
it('treats any non-"agent" source string as human (keys strictly on === agent)', () => {
const { jobId, delay } = computeHistoryJob(page, 'user');
expect(delay).toBe(IDLE_INTERVAL_USER);
it('treats any non-"agent" source string as human', () => {
// resolveSource only ever yields 'agent' | 'user', but guard the contract:
// the agent branch keys strictly on === 'agent'.
const { jobId, delay } = computeHistoryJob(pageAged(0), 'user', NOW);
expect(delay).toBe(HISTORY_FAST_INTERVAL);
expect(jobId).toBe(PAGE_ID);
});
// #370 review round-1 WARNING: the max-wait ceiling prevents autosnapshot
// starvation during a continuous editing session (the trailing timer would
// otherwise re-arm forever and never fire).
describe('max-wait ceiling', () => {
const T0 = 1_000_000; // arbitrary fixed epoch for deterministic tests
it('once a burst is armed, delay clamps to the remaining max-wait budget', () => {
// 1 minute into the burst the USER interval (60m) far exceeds the remaining
// max-wait budget (10m - 1m = 9m), so the delay is clamped DOWN to that
// remaining budget — the full interval is NOT used once a ceiling applies.
const { delay } = computeHistoryJob(page, 'user', T0, T0 + 60_000);
expect(delay).toBe(IDLE_MAX_WAIT_USER - 60_000);
});
it('never waits longer than the max-wait budget from the burst start', () => {
// A store arriving right at the ceiling → delay 0 (fire promptly).
const { delay } = computeHistoryJob(
page,
'user',
T0,
T0 + IDLE_MAX_WAIT_USER,
);
expect(delay).toBe(0);
});
it('past the ceiling never returns a negative delay', () => {
const { delay } = computeHistoryJob(
page,
'user',
T0,
T0 + IDLE_MAX_WAIT_USER + 5 * 60_000,
);
expect(delay).toBe(0);
});
it('the agent ceiling is shorter than the user ceiling', () => {
expect(IDLE_MAX_WAIT_AGENT).toBeLessThan(IDLE_MAX_WAIT_USER);
const { delay } = computeHistoryJob(
page,
'agent',
T0,
T0 + IDLE_MAX_WAIT_AGENT,
);
expect(delay).toBe(0);
});
it('without a burstStart there is no ceiling (backward-compatible)', () => {
expect(computeHistoryJob(page, 'user').delay).toBe(IDLE_INTERVAL_USER);
expect(computeHistoryJob(page, 'agent').delay).toBe(IDLE_INTERVAL_AGENT);
});
});
});
describe('resolveSource (truth table)', () => {
@@ -40,12 +40,11 @@ describe('PersistenceExtension.onStoreDocument — Approach-A boundary snapshot'
let pageHistoryRepo: {
saveHistory: jest.Mock;
findPageLastHistory: jest.Mock;
updateHistoryKind: jest.Mock;
};
let aiQueue: { add: jest.Mock };
let historyQueue: { add: jest.Mock; remove: jest.Mock };
let historyQueue: { add: jest.Mock };
let notificationQueue: { add: jest.Mock };
let collabHistory: { addContributors: jest.Mock; popContributors: jest.Mock };
let collabHistory: { addContributors: jest.Mock };
let transclusionService: {
syncPageTransclusions: jest.Mock;
syncPageReferences: jest.Mock;
@@ -94,22 +93,13 @@ describe('PersistenceExtension.onStoreDocument — Approach-A boundary snapshot'
pageHistoryRepo = {
saveHistory: jest.fn().mockImplementation(async () => {
callOrder.push('saveHistory');
return { id: 'history-1' };
}),
findPageLastHistory: jest.fn().mockResolvedValue(null),
updateHistoryKind: jest.fn().mockResolvedValue(undefined),
};
aiQueue = { add: jest.fn().mockResolvedValue(undefined) };
historyQueue = {
add: jest.fn().mockResolvedValue(undefined),
// #370 — enqueuePageHistory now removes any pending idle job before re-adding.
remove: jest.fn().mockResolvedValue(undefined),
};
historyQueue = { add: jest.fn().mockResolvedValue(undefined) };
notificationQueue = { add: jest.fn().mockResolvedValue(undefined) };
collabHistory = {
addContributors: jest.fn().mockResolvedValue(undefined),
popContributors: jest.fn().mockResolvedValue([]),
};
collabHistory = { addContributors: jest.fn().mockResolvedValue(undefined) };
transclusionService = {
syncPageTransclusions: jest.fn().mockResolvedValue(undefined),
syncPageReferences: jest.fn().mockResolvedValue(undefined),
@@ -175,50 +165,6 @@ describe('PersistenceExtension.onStoreDocument — Approach-A boundary snapshot'
expect(pageRepo.updatePage.mock.calls[0][0].lastUpdatedSource).toBe('user');
});
// #370 review round-1 SUGGESTION: the boundary was GENERALIZED from a
// user→agent special-case to ANY lastUpdatedSource transition. These pin the
// generalized behaviour it was rebuilt for.
describe('generalized boundary — any source transition', () => {
// Same persisted page but with an explicit prior source.
const pageWithPriorSource = (prior: string | null) => ({
...persistedHumanPage('NEW CONTENT'),
lastUpdatedSource: prior,
});
it('agent→user transition fires the boundary (pins the prior agent revision)', async () => {
const document = ydocFor(doc('NEW CONTENT'));
pageRepo.findById.mockResolvedValue(pageWithPriorSource('agent'));
pageHistoryRepo.findPageLastHistory.mockResolvedValue(null);
await ext.onStoreDocument(buildData(document, 'user') as any);
expect(pageHistoryRepo.saveHistory).toHaveBeenCalledTimes(1);
expect(callOrder).toEqual(['saveHistory', 'updatePage']);
expect(pageRepo.updatePage.mock.calls[0][0].lastUpdatedSource).toBe('user');
});
it('git→user transition fires the boundary (git-sync overwrite is a source change)', async () => {
const document = ydocFor(doc('NEW CONTENT'));
pageRepo.findById.mockResolvedValue(pageWithPriorSource('git'));
pageHistoryRepo.findPageLastHistory.mockResolvedValue(null);
await ext.onStoreDocument(buildData(document, 'user') as any);
expect(pageHistoryRepo.saveHistory).toHaveBeenCalledTimes(1);
expect(callOrder).toEqual(['saveHistory', 'updatePage']);
});
it('a null prior source (first-ever edit) does NOT fire the boundary', async () => {
const document = ydocFor(doc('NEW CONTENT'));
pageRepo.findById.mockResolvedValue(pageWithPriorSource(null));
await ext.onStoreDocument(buildData(document, 'agent') as any);
expect(pageHistoryRepo.saveHistory).not.toHaveBeenCalled();
expect(pageRepo.updatePage).toHaveBeenCalledTimes(1);
});
});
it('idempotency: unchanged content → no updatePage, no history, no queues', async () => {
// The Y.Doc content equals the persisted content deeply → early skip.
// A Y.Doc round-trip normalizes attrs (e.g. paragraph indent), so derive
@@ -533,231 +479,4 @@ describe('PersistenceExtension.onStoreDocument — Approach-A boundary snapshot'
// Contributors keyed by the UUID so they match the PAGE_HISTORY job (page.id).
expect(collabHistory.addContributors.mock.calls[0][0]).toBe(PAGE_ID);
});
// #370 — explicit save-version (Cmd+S / agent save tool) over the stateless
// seam. The tier is derived from the SIGNED connection actor, the store path
// is reused, and promote-not-dup avoids duplicating heavy content rows.
describe('save-version (#370)', () => {
const emitSave = (document: any, actor: 'user' | 'agent') =>
ext.onStateless({
connection: {
readOnly: false,
context: { user: { id: USER_ID, name: 'Alice' }, actor },
} as any,
documentName: `page.${PAGE_ID}`,
document: document as any,
payload: JSON.stringify({ type: 'save-version' }),
} as any);
// findById returns a page whose content already equals the live doc, so the
// store path is a no-op and we isolate the versioning decision.
const pageMatchingDoc = (document: any) => ({
...persistedHumanPage('IGNORED'),
content: TiptapTransformer.fromYdoc(document, 'default'),
});
it('human save with no prior snapshot → writes a manual version + broadcasts', async () => {
const document = ydocFor(doc('VERSION ME'));
pageRepo.findById.mockResolvedValue(pageMatchingDoc(document));
pageHistoryRepo.findPageLastHistory.mockResolvedValue(null);
await emitSave(document, 'user');
expect(pageHistoryRepo.saveHistory).toHaveBeenCalledTimes(1);
expect(pageHistoryRepo.saveHistory.mock.calls[0][1]).toEqual(
expect.objectContaining({ kind: 'manual' }),
);
// The pending idle autosnapshot is cancelled by the explicit version.
expect(historyQueue.remove).toHaveBeenCalledWith(PAGE_ID);
const msg = JSON.parse(
(document as any).broadcastStateless.mock.calls[(document as any).broadcastStateless.mock.calls.length - 1][0],
);
expect(msg).toMatchObject({
type: 'version.saved',
kind: 'manual',
alreadySaved: false,
});
});
it('agent save derives kind=agent from the signed actor', async () => {
const document = ydocFor(doc('AGENT VERSION'));
pageRepo.findById.mockResolvedValue(pageMatchingDoc(document));
pageHistoryRepo.findPageLastHistory.mockResolvedValue(null);
await emitSave(document, 'agent');
expect(pageHistoryRepo.saveHistory.mock.calls[pageHistoryRepo.saveHistory.mock.calls.length - 1][1]).toEqual(
expect.objectContaining({ kind: 'agent' }),
);
});
it('promote-not-dup: latest snapshot is an autosave with identical content → upgrades in place', async () => {
const document = ydocFor(doc('SAME'));
const page = pageMatchingDoc(document);
pageRepo.findById.mockResolvedValue(page);
pageHistoryRepo.findPageLastHistory.mockResolvedValue({
id: 'auto-1',
content: page.content,
kind: 'idle',
});
await emitSave(document, 'user');
// No heavy new content row — the existing autosave is promoted to manual.
expect(pageHistoryRepo.updateHistoryKind).toHaveBeenCalledWith(
'auto-1',
'manual',
expect.anything(),
);
expect(pageHistoryRepo.saveHistory).not.toHaveBeenCalled();
const msg = JSON.parse(
(document as any).broadcastStateless.mock.calls[(document as any).broadcastStateless.mock.calls.length - 1][0],
);
expect(msg).toMatchObject({ historyId: 'auto-1', alreadySaved: false });
});
it('no-op when the latest snapshot is already a manual version of this content', async () => {
const document = ydocFor(doc('ALREADY SAVED'));
const page = pageMatchingDoc(document);
pageRepo.findById.mockResolvedValue(page);
pageHistoryRepo.findPageLastHistory.mockResolvedValue({
id: 'ver-1',
content: page.content,
kind: 'manual',
});
await emitSave(document, 'user');
expect(pageHistoryRepo.updateHistoryKind).not.toHaveBeenCalled();
expect(pageHistoryRepo.saveHistory).not.toHaveBeenCalled();
const msg = JSON.parse(
(document as any).broadcastStateless.mock.calls[(document as any).broadcastStateless.mock.calls.length - 1][0],
);
expect(msg).toMatchObject({ alreadySaved: true, kind: 'manual' });
});
it('a read-only connection cannot save a version', async () => {
const document = ydocFor(doc('READER'));
pageRepo.findById.mockResolvedValue(pageMatchingDoc(document));
await ext.onStateless({
connection: {
readOnly: true,
context: { user: { id: USER_ID }, actor: 'user' },
} as any,
documentName: `page.${PAGE_ID}`,
document: document as any,
payload: JSON.stringify({ type: 'save-version' }),
} as any);
expect(pageHistoryRepo.saveHistory).not.toHaveBeenCalled();
expect(pageHistoryRepo.updateHistoryKind).not.toHaveBeenCalled();
});
// #370 F8-twin — a COMMIT abort (serialization/deadlock/conn-drop) rejects
// OUTSIDE the tx callback, AFTER the destructive popContributors (SPOP) and
// saveHistory ran but the INSERT rolled back. onStateless has no retry, so
// the outer catch MUST re-add (SADD) the popped set or attribution is lost
// irrecoverably. MUTATION: drop the outer catch → addContributors is never
// called → this reddens.
it('restores popped contributors when the commit aborts after the callback', async () => {
const document = ydocFor(doc('VERSION ME'));
pageRepo.findById.mockResolvedValue(pageMatchingDoc(document));
// No matching snapshot → fresh version branch → pops contributors.
pageHistoryRepo.findPageLastHistory.mockResolvedValue(null);
collabHistory.popContributors.mockResolvedValue(['u1', 'u2']);
// A db whose commit REJECTS after the callback body resolved: the SPOP and
// saveHistory already ran, then the tx aborts. onStoreDocument's flush uses
// the same db but its content matches (no-op branch) and its own retry loop
// swallows the throw, so only the versioning tx exercises the restore.
const commitFailingDb = {
transaction: () => ({
execute: async (fn: (trx: any) => Promise<any>) => {
await fn(trxStub);
throw new Error('commit aborted (serialization_failure)');
},
}),
};
const ext2 = new PersistenceExtension(
pageRepo as any,
pageHistoryRepo as any,
commitFailingDb as any,
aiQueue as any,
historyQueue as any,
notificationQueue as any,
collabHistory as any,
transclusionService as any,
);
jest.spyOn(ext2['logger'], 'debug').mockImplementation(() => undefined);
jest.spyOn(ext2['logger'], 'warn').mockImplementation(() => undefined);
jest.spyOn(ext2['logger'], 'error').mockImplementation(() => undefined);
await expect(
ext2.onStateless({
connection: {
readOnly: false,
context: { user: { id: USER_ID, name: 'Alice' }, actor: 'user' },
} as any,
documentName: `page.${PAGE_ID}`,
document: document as any,
payload: JSON.stringify({ type: 'save-version' }),
} as any),
).rejects.toThrow();
// Attribution preserved: the popped set is SADD-restored, keyed by the page
// UUID it was popped under.
expect(collabHistory.addContributors).toHaveBeenCalledWith(PAGE_ID, [
'u1',
'u2',
]);
});
// #370 #260 — for a `page.<slugId>` document the idle job is armed under the
// page UUID (computeHistoryJob's jobId = page.id), so the supersede-remove
// must target page.id, not the raw slugId doc-name id, or it silently misses.
it('cancels the superseded idle job by the page UUID for a slugId doc', async () => {
const SLUG = 'slug-1'; // persistedHumanPage.slugId
const document = ydocFor(doc('VERSION ME'));
pageRepo.findById.mockResolvedValue(pageMatchingDoc(document));
pageHistoryRepo.findPageLastHistory.mockResolvedValue(null);
await ext.onStateless({
connection: {
readOnly: false,
context: { user: { id: USER_ID, name: 'Alice' }, actor: 'user' },
} as any,
documentName: `page.${SLUG}`,
document: document as any,
payload: JSON.stringify({ type: 'save-version' }),
} as any);
// remove() keyed by the UUID (the real jobId), never the slugId.
expect(historyQueue.remove).toHaveBeenCalledWith(PAGE_ID);
expect(historyQueue.remove).not.toHaveBeenCalledWith(SLUG);
});
});
// #370 — the in-memory idle-burst marker must be dropped on doc unload (like
// its sibling per-document maps) or it grows unbounded for every page that was
// edited but never manually saved. MUTATION: drop the afterUnloadDocument
// delete → the entry survives → this reddens.
describe('idleBurstStart housekeeping', () => {
it('afterUnloadDocument clears the idle-burst marker armed by a store', async () => {
const document = ydocFor(doc('EDIT'));
pageRepo.findById.mockResolvedValue(persistedHumanPage('EDIT'));
await ext.onStoreDocument(buildData(document, 'user') as any);
const map = ext['idleBurstStart'] as Map<string, number>;
// Keyed by documentName (buildData uses `page.${PAGE_ID}`).
expect(map.has(`page.${PAGE_ID}`)).toBe(true);
await ext.afterUnloadDocument({
documentName: `page.${PAGE_ID}`,
} as any);
expect(map.has(`page.${PAGE_ID}`)).toBe(false);
});
});
});
@@ -37,11 +37,9 @@ import { Page } from '@docmost/db/types/entity.types';
import { CollabHistoryService } from '../services/collab-history.service';
import {
EMBED_DEBOUNCE_MS,
IDLE_INTERVAL_AGENT,
IDLE_INTERVAL_USER,
IDLE_MAX_WAIT_AGENT,
IDLE_MAX_WAIT_USER,
PageHistoryKind,
HISTORY_FAST_INTERVAL,
HISTORY_FAST_THRESHOLD,
HISTORY_INTERVAL,
} from '../constants';
import { TransclusionService } from '../../core/page/transclusion/transclusion.service';
import {
@@ -58,16 +56,6 @@ import { hasTransclusionFamilyNodes } from '../../core/page/transclusion/utils/t
*/
export const INTENTIONAL_CLEAR_MESSAGE_TYPE = 'intentional-clear';
/**
* #370 — wire format of the client→server "save a version" signal. Sent by the
* human (Cmd+S / Save button) and by the agent's explicit save tool over the
* SAME stateless channel. The intentionality tier ('manual' vs 'agent') is
* derived SERVER-SIDE from the signed connection actor, never from this
* payload, so a version's type is unforgeable. The document is taken from the
* connection (not the payload), so the signal cannot be aimed at another page.
*/
export const SAVE_VERSION_MESSAGE_TYPE = 'save-version';
/**
* #251 — how long an intentional-clear signal stays "pending" before it is
* ignored. The signal is set on the clearing keystroke but consumed by the
@@ -104,39 +92,35 @@ export function resolveSource(
}
/**
* #370 — compute the BullMQ job id + delay for a page's trailing idle-flush
* autosnapshot. Pure so the timing is unit-testable.
* Compute the BullMQ job id + delay for a page-history snapshot job. Pure so
* the data-loss-sensitive timing arithmetic is unit-testable; `now` is injected
* (caller passes `Date.now()`) for determinism.
*
* Both humans and the agent now share ONE idle pipeline (the agent's old
* `delay=0` fast path is gone — intentional agent points arrive via the
* explicit save-version signal instead). The job id is the bare `page.id`, so a
* page has at most one pending idle job; the caller removes-and-re-adds it on
* every store to keep it debounced to the trailing edge of an edit burst. The
* window differs by source only: the agent flushes sooner than a human.
* - Agent edits: delay 0 and a source-keyed job id `${page.id}-agent`. The
* delay MUST stay 0 — the worker re-reads the page row at run time, so any
* delay risks reading content a later human edit has already overwritten
* (mis-tagged snapshot). 0 minimizes that window. The `-agent` suffix keeps
* the job from coalescing with the bare-page.id human job.
* - Human edits: age-based debounce so rapid human edits coalesce into one
* snapshot; job id is the bare `page.id`.
*
* BullMQ forbids ':' in custom job ids (Redis key separator), so '-' is used;
* page.id is a UUID, so `${page.id}-agent` cannot collide with a human job.
*/
export function computeHistoryJob(
page: Pick<Page, 'id'>,
page: Pick<Page, 'id' | 'createdAt'>,
source: string,
// Epoch ms of the FIRST edit in the current burst (when the pending idle job
// was first armed). Used to enforce the max-wait ceiling so a continuous
// editing session cannot re-arm the trailing timer forever. `now` is injectable
// for tests; both default to a live clock / no ceiling when omitted.
burstStart?: number,
now: number = Date.now(),
now: number,
): { jobId: string; delay: number } {
const isAgent = source === 'agent';
const interval = isAgent ? IDLE_INTERVAL_AGENT : IDLE_INTERVAL_USER;
const maxWait = isAgent ? IDLE_MAX_WAIT_AGENT : IDLE_MAX_WAIT_USER;
let delay = interval;
if (burstStart !== undefined) {
// Time already elapsed since the burst's first edit; the snapshot must fire
// no later than `maxWait` after that, so shrink the trailing delay to the
// remaining budget (never negative, so BullMQ fires it promptly).
const remaining = burstStart + maxWait - now;
delay = Math.max(0, Math.min(interval, remaining));
}
return { jobId: page.id, delay };
const pageAge = now - new Date(page.createdAt).getTime();
const delay = isAgent
? 0
: pageAge < HISTORY_FAST_THRESHOLD
? HISTORY_FAST_INTERVAL
: HISTORY_INTERVAL;
const jobId = isAgent ? `${page.id}-agent` : page.id;
return { jobId, delay };
}
@Injectable()
@@ -148,28 +132,6 @@ export class PersistenceExtension implements Extension {
// coalescing window" per document and OR it across all edits in the window,
// so the snapshot is marked 'agent' regardless of who wrote last.
private agentTouched: Map<string, boolean> = new Map();
// #370 — epoch ms of the FIRST edit in the current idle-flush burst. Keyed by
// documentName (like its sibling per-document maps above), NOT by page.id, so
// it can be cleaned in afterUnloadDocument alongside `contributors` /
// `agentTouched` / `intentionalClear` when the doc unloads — otherwise any page
// that was edited but never manually saved (the common case) would keep its
// entry forever and the Map would grow unbounded in this long-lived process.
// Set when the pending idle job is first armed (empty entry), read to enforce
// the max-wait ceiling in computeHistoryJob, and cleared on doc unload or when
// a manual save cancels the idle job so the next burst starts a fresh window.
//
// Single-process assumption (like `contributors` / `agentTouched` above): this
// lives only in THIS collab process's memory. A restart, or a page's ownership
// moving to another node, loses the burst-start marker. Consequence: a burst
// that spans the restart looks like a fresh burst to the surviving process, so
// its max-wait ceiling is re-anchored to the first post-restart edit — a single
// continuous session straddling a restart can therefore wait up to ~2× the cap
// for its idle snapshot (once for the lost pre-restart window, once for the new
// one). Bounded and benign (it only DELAYS a safety-net autosnapshot; manual
// saves are unaffected and the next quiet period always flushes), but the
// assumption and its consequence are recorded here so no one mistakes the
// in-memory marker for a durable, cross-process guarantee.
private idleBurstStart: Map<string, number> = new Map();
// #251 — per-document "intentional clear pending" flags. Keyed by
// documentName, value = expiry timestamp (ms). Set by onStateless when the
// client reports a deliberate clear; consumed once by the next
@@ -401,19 +363,20 @@ export class PersistenceExtension implements Extension {
//this.logger.debug('Contributors error:' + err?.['message']);
}
// #370 — boundary snapshot on ANY source transition. When the store
// flips the page's provenance (user↔agent↔git), pin the OUTGOING
// state as its own history version BEFORE the incoming source
// overwrites it. `page` still holds the OLD content/provenance here,
// so saveHistory(page) captures the pre-transition state tagged with
// its own source, kind='boundary'. The incoming content is snapshotted
// later by the debounced idle job. Skip if the page is effectively
// empty or if the latest existing snapshot already equals this state
// (the shared isDeepStrictEqual gate — avoids duplicates). Generalizing
// beyond the old user→agent special-case also covers git-sync for free.
// Approach A — boundary snapshot before the agent's first edit.
// When this store is the agent's and the page's currently persisted
// state was authored by a human, pin that human state as its own
// history version BEFORE the agent overwrites it. `page` still holds
// the OLD content/provenance here, so saveHistory(page) captures the
// pre-agent state tagged 'user'. The agent's new content is
// snapshotted later by the debounced PAGE_HISTORY job ('agent'). Skip
// if the prior state is already agent-authored (boundary already
// pinned on the user->agent transition), if the page is effectively
// empty, or if the latest existing snapshot already equals this human
// state (avoid duplicates).
if (
page.lastUpdatedSource &&
page.lastUpdatedSource !== lastUpdatedSource
lastUpdatedSource === 'agent' &&
page.lastUpdatedSource !== 'agent'
) {
// pageHistory.pageId is uuid-typed; use page.id (never the doc-name
// slugId) so a `page.<slugId>` doc cannot throw 22P02 here (#260).
@@ -421,13 +384,15 @@ export class PersistenceExtension implements Extension {
page.id,
{ includeContent: true, trx },
);
const baselineMissing =
const humanBaselineMissing =
!lastHistory ||
!isDeepStrictEqual(lastHistory.content, page.content);
if (!isEmptyParagraphDoc(page.content as any) && baselineMissing) {
if (
!isEmptyParagraphDoc(page.content as any) &&
humanBaselineMissing
) {
await this.pageHistoryRepo.saveHistory(page, {
contributorIds: page.contributorIds ?? undefined,
kind: 'boundary',
trx,
});
}
@@ -557,7 +522,7 @@ export class PersistenceExtension implements Extension {
{ jobId: `embed-${page.id}`, delay: EMBED_DEBOUNCE_MS },
);
await this.enqueuePageHistory(page, documentName, lastUpdatedSource);
await this.enqueuePageHistory(page, lastUpdatedSource);
}
// #402 — report the serialized size for the store histogram's size_bucket.
@@ -589,14 +554,6 @@ export class PersistenceExtension implements Extension {
return; // unrelated / malformed stateless message
}
// #370 — explicit "save a version" (human Cmd+S / agent save tool). Edit
// rights are already enforced by the readOnly reject above (a reader can't
// create a version), exactly as intentional-clear requires.
if (message?.type === SAVE_VERSION_MESSAGE_TYPE) {
await this.handleSaveVersion(data);
return;
}
if (message?.type !== INTENTIONAL_CLEAR_MESSAGE_TYPE) return;
this.intentionalClear.set(
@@ -605,160 +562,6 @@ export class PersistenceExtension implements Extension {
);
}
/**
* #370 — persist an intentional version from the live in-memory ydoc.
*
* One stateless path serves BOTH the human and the agent; the tier is derived
* SERVER-SIDE from the signed connection actor ('agent' → 'agent', anything
* else → 'manual'), so the version type cannot be spoofed by the client. We
* take the fresh ydoc from the collab process memory and run it through the
* EXISTING store path first (so pages.content/ydoc reflect the exact content
* being versioned — a REST endpoint would race the up-to-10s-stale page row),
* then snapshot it into page_history with the intentional kind.
*
* Promote-not-dup: if the latest history row already holds this exact content
* and it is an autosave (idle/boundary/legacy-null), upgrade its kind in place
* instead of duplicating a heavy content row; if it is already 'manual', it is
* a no-op (the client shows an "already saved" toast). Otherwise a fresh
* version row is written, popping the aggregated contributors from Redis.
*/
private async handleSaveVersion(data: onStatelessPayload): Promise<void> {
const { connection, document, documentName } = data;
const context = connection?.context;
const pageId = getPageId(documentName);
// Unforgeable: 'agent' only for a signed agent connection, else 'manual'.
const kind: PageHistoryKind =
context?.actor === 'agent' ? 'agent' : 'manual';
// Flush the live ydoc through the normal store path so the page row + ydoc
// hold exactly what we are about to version (also fires the idle enqueue we
// supersede below, plus any source-transition boundary). onStoreDocument
// only needs document/documentName/context.
await this.onStoreDocument({
document,
documentName,
context,
} as onStoreDocumentPayload);
let result:
| { historyId: string; kind: PageHistoryKind; alreadySaved: boolean }
| undefined;
// #370 F8-twin — the contributor set popped from Redis (destructive SPOP)
// must be restored if the version row does not durably land. The inner
// try/catch below only covers a throw INSIDE the callback; but executeTx
// COMMITS after the callback, so a commit-abort (serialization/deadlock/
// connection drop — the transient class the epic retries in the processor)
// rejects OUTSIDE the callback, after saveHistory already ran and the SPOP
// already happened, while the INSERT rolls back. onStateless does NOT retry,
// so an unrestored pop is a one-shot irrecoverable attribution loss (the
// processor got exactly this fix: poppedForRestore + an outer catch). We
// track the popped set here (keyed by the page UUID it was popped by — never
// the doc-name id, which may be a slugId, #260) and restore it in the outer
// catch. addContributors is an idempotent Redis SADD, so a double-restore is
// harmless. versionedPageId is also reused below to remove the superseded
// idle job by its real jobId (page.id).
let poppedForRestore: string[] = [];
let versionedPageId: string | undefined;
try {
await executeTx(this.db, async (trx) => {
const page = await this.pageRepo.findById(pageId, {
withLock: true,
includeContent: true,
trx,
});
if (!page) return;
versionedPageId = page.id;
// Never version an effectively-empty page (mirrors the processor's
// first-history guard); there is nothing intentional to pin.
if (isEmptyParagraphDoc(page.content as any)) return;
const lastHistory = await this.pageHistoryRepo.findPageLastHistory(
page.id,
{ includeContent: true, trx },
);
if (
lastHistory &&
isDeepStrictEqual(lastHistory.content, page.content)
) {
// Content is already snapshotted. Promote-not-dup.
if (lastHistory.kind === 'manual') {
result = {
historyId: lastHistory.id,
kind: 'manual',
alreadySaved: true,
};
return;
}
await this.pageHistoryRepo.updateHistoryKind(
lastHistory.id,
kind,
trx,
);
result = { historyId: lastHistory.id, kind, alreadySaved: false };
return;
}
// Fresh version row. Pop the contributors aggregated since the last
// snapshot (SPOP); restore them if the write fails so they aren't lost.
const contributorIds = await this.collabHistory.popContributors(
page.id,
);
poppedForRestore = contributorIds;
try {
const saved = await this.pageHistoryRepo.saveHistory(page, {
contributorIds,
kind,
trx,
});
result = { historyId: saved.id, kind, alreadySaved: false };
} catch (err) {
await this.collabHistory.addContributors(page.id, contributorIds);
poppedForRestore = [];
throw err;
}
});
} catch (err) {
// A throw here means the tx did NOT commit (callback threw, or the commit
// itself failed and rolled back). If we popped contributors and the inner
// catch did not already restore them, restore now so attribution is not
// lost — onStateless has no retry to recover it. Restore by the page UUID
// the pop was keyed under (versionedPageId is always set before the pop).
if (poppedForRestore.length && versionedPageId) {
await this.collabHistory.addContributors(
versionedPageId,
poppedForRestore,
);
}
throw err;
}
// Housekeeping: this explicit version supersedes the page's pending idle
// autosnapshot, so cancel it and end the current idle burst so the next edit
// starts a fresh max-wait window. Remove the idle job by its REAL jobId
// (page.id UUID — computeHistoryJob arms it under page.id), not the raw
// doc-name id which may be a slugId for a `page.<slugId>` doc (#260), or the
// remove silently misses. The burst marker is keyed by documentName (like its
// sibling per-document maps), and is also cleaned in afterUnloadDocument.
if (versionedPageId) {
await this.historyQueue.remove(versionedPageId).catch(() => undefined);
}
this.idleBurstStart.delete(documentName);
if (result) {
document.broadcastStateless(
JSON.stringify({
type: 'version.saved',
historyId: result.historyId,
kind: result.kind,
alreadySaved: result.alreadySaved,
}),
);
}
}
async onChange(data: onChangePayload) {
const documentName = data.documentName;
const userId = data.context?.user?.id;
@@ -783,10 +586,6 @@ export class PersistenceExtension implements Extension {
this.contributors.delete(documentName);
this.agentTouched.delete(documentName);
this.intentionalClear.delete(documentName);
// #370 — drop the idle-burst marker with the other per-document maps so it
// cannot accumulate across the process lifetime for never-manually-saved
// pages. The pending idle job (if any) is a self-expiring BullMQ delayed job.
this.idleBurstStart.delete(documentName);
}
private consumeContributors(documentName: string): string[] {
@@ -818,80 +617,19 @@ export class PersistenceExtension implements Extension {
private async enqueuePageHistory(
page: Page,
documentName: string,
lastUpdatedSource: string,
): Promise<void> {
// #370 — trailing idle debounce with a max-wait ceiling. One pending idle
// job per page (jobId = page.id); on every store we remove the pending
// delayed job and re-add it, so the snapshot lands `delay` after edits go
// quiet rather than once per store (precedent: workspace.service.ts).
// remove() on a delayed job simply deletes it (0 if absent, no throw); if the
// job is already ACTIVE and the remove is a no-op, the add still de-dups and
// the processor's isDeepStrictEqual gate collapses the duplicate content.
//
// The FIRST arm of a burst records `burstStart`; computeHistoryJob shrinks
// the delay to the remaining max-wait budget from that point, so a continuous
// session cannot re-arm the trailing timer forever and starve the snapshot.
// A burst marker older than THIS TIER's max-wait means the previous idle job
// has already fired — start a fresh window instead of firing immediately on
// the next edit. Must use the SAME source-specific max-wait computeHistoryJob
// uses (agent 5m / user 10m): a hardcoded USER ceiling would leave an agent
// burst's marker stale for 5..10m, forcing delay=0 on every store in that
// window and writing one idle row per store — exactly the per-store bloat the
// debounce exists to prevent, on the continuous-agent path.
const maxWait =
lastUpdatedSource === 'agent' ? IDLE_MAX_WAIT_AGENT : IDLE_MAX_WAIT_USER;
const now = Date.now();
// Keyed by documentName (see the map declaration) so afterUnloadDocument can
// clean it; the queue jobId stays page.id (computeHistoryJob) as required.
let burstStart = this.idleBurstStart.get(documentName);
if (burstStart === undefined || now - burstStart >= maxWait) {
burstStart = now;
this.idleBurstStart.set(documentName, burstStart);
}
// Job id + delay arithmetic lives in the pure `computeHistoryJob` (see its
// doc comment for the agent-delay-0 / age-based-debounce invariants).
const { jobId, delay } = computeHistoryJob(
page,
lastUpdatedSource,
burstStart,
now,
Date.now(),
);
// remove-then-add trailing-debounce idiom, and its ONE race. We delete the
// pending delayed job and re-add it under the same jobId so the timer resets
// to the trailing edge of the burst. The race is the small window between
// these two awaits: if the delayed job's `delay` elapses in that gap it goes
// ACTIVE, and then:
// - remove() on an active/locked job is a no-op (BullMQ won't yank a job a
// worker holds), and our `.catch(() => undefined)` swallows that too; and
// - add() with a jobId that already exists (the now-active job's id) is
// DROPPED by BullMQ — a duplicate add is a no-op.
// So this store fails to re-arm the trailing job: the just-fired snapshot
// captured content up to the moment it went active, and THIS edit is left
// without a pending trailing job. It is bounded and self-healing — the NEXT
// store re-arms a fresh delayed job (the id is free again once the active job
// completes / removeOnComplete frees it), and the processor's
// isDeepStrictEqual gate collapses any content-identical duplicate. The only
// uncovered case is when the racing store was the LAST in the session: the
// tail edits made after the job went active get NO trailing snapshot until
// the next edit re-arms one. That is an acceptable safety-net gap (a manual
// Save, a source-transition boundary, or simply the next edit all still cover
// it), which is why the reviewer accepts documenting it here rather than
// adding a post-add "did the add actually arm a job?" re-check.
//
// NOTE — do NOT "unify" this with the neighbouring embed-debounce idiom
// (aiQueue.add of PAGE_CONTENT_UPDATED above): that one uses a STABLE jobId
// and NO remove(), relying purely on BullMQ coalescing a repeated add under
// the same id, because a re-embed only needs to eventually run once on the
// latest content and re-anchoring its delay on every keystroke is undesirable.
// THIS idiom deliberately removes-then-adds precisely to PUSH the delay back
// to the trailing edge on every store (a true debounce), which coalescing
// alone cannot do. Collapsing them would silently change the history cadence.
await this.historyQueue.remove(jobId).catch(() => undefined);
await this.historyQueue.add(
QueueJob.PAGE_HISTORY,
{ pageId: page.id, kind: 'idle' } as IPageHistoryJob,
{ pageId: page.id } as IPageHistoryJob,
{ jobId, delay },
);
}
@@ -66,15 +66,6 @@ describe('HistoryProcessor.process', () => {
notificationQueue = { add: jest.fn().mockResolvedValue(undefined) };
generalQueue = { add: jest.fn().mockResolvedValue(undefined) };
// #370 F3 — the processor now serializes its find+save under a page-row lock
// via executeTx. A db whose transaction().execute(fn) runs fn with a trx stub
// drives the real executeTx() helper without a database.
const db = {
transaction: () => ({
execute: (fn: (trx: any) => Promise<any>) => fn({ __trx: true }),
}),
};
// WorkerHost's constructor reads `this.worker`; passing repos positionally
// matches the constructor and avoids the Nest DI container.
proc = new HistoryProcessor(
@@ -82,7 +73,6 @@ describe('HistoryProcessor.process', () => {
pageRepo as any,
collabHistory as any,
watcherService as any,
db as any,
notificationQueue as any,
generalQueue as any,
);
@@ -136,26 +126,15 @@ describe('HistoryProcessor.process', () => {
await proc.process(buildJob());
expect(collabHistory.popContributors).toHaveBeenCalledWith(PAGE_ID);
// #370 F3/F9 — the snapshot decision runs under a page-row lock. Pin the lock
// structurally so a refactor that drops withLock/trx (silently reintroducing
// the TOCTOU double-insert) turns this red. The tx stub is { __trx: true }.
expect(pageRepo.findById).toHaveBeenCalledWith(
PAGE_ID,
expect.objectContaining({ withLock: true, trx: { __trx: true } }),
);
// #370 F7 — addPageWatchers MUST receive the trx, or its FK-check runs on a
// separate connection and self-deadlocks against our FOR UPDATE. Asserting
// the trx arg here is exactly what would have caught that regression.
expect(watcherService.addPageWatchers).toHaveBeenCalledWith(
['u1', 'u2'],
PAGE_ID,
SPACE_ID,
WORKSPACE_ID,
{ __trx: true },
);
expect(pageHistoryRepo.saveHistory).toHaveBeenCalledWith(
expect.objectContaining({ id: PAGE_ID }),
{ contributorIds: ['u1', 'u2'], kind: 'idle', trx: { __trx: true } },
{ contributorIds: ['u1', 'u2'] },
);
expect(generalQueue.add).toHaveBeenCalledWith(
QueueJob.PAGE_BACKLINKS,
@@ -207,48 +186,6 @@ describe('HistoryProcessor.process', () => {
]);
});
it('COMMIT failure (throw outside the tx callback) → contributors RESTORED', async () => {
// #370 F8 — a commit-time failure throws OUTSIDE the callback, so the inner
// try/catch does not run; the outer catch must restore the popped set (else a
// BullMQ retry writes an unattributed version). Use a db whose execute() runs
// the callback THEN throws, simulating a commit abort.
pageHistoryRepo.findPageLastHistory.mockResolvedValue({
content: { type: 'doc', content: [] },
});
const commitFail = {
transaction: () => ({
execute: async (fn: (trx: any) => Promise<any>) => {
await fn({ __trx: true }); // callback succeeds (saveHistory ok)
throw new Error('commit aborted'); // ...but the COMMIT fails
},
}),
};
const procCommitFail = new HistoryProcessor(
pageHistoryRepo as any,
pageRepo as any,
collabHistory as any,
watcherService as any,
commitFail as any,
notificationQueue as any,
generalQueue as any,
);
jest
.spyOn(procCommitFail['logger'], 'error')
.mockImplementation(() => undefined);
await expect(procCommitFail.process(buildJob())).rejects.toThrow(
'commit aborted',
);
// The inner catch did NOT run (save succeeded), so only the outer catch can
// restore — assert it did.
expect(collabHistory.addContributors).toHaveBeenCalledWith(PAGE_ID, [
'u1',
'u2',
]);
// And the post-snapshot queue work must NOT have run (we rethrew).
expect(generalQueue.add).not.toHaveBeenCalled();
});
it('backlinks + notification queue failures are swallowed (history still committed)', async () => {
pageHistoryRepo.findPageLastHistory.mockResolvedValue({
content: { type: 'doc', content: [] },
@@ -19,9 +19,6 @@ import { isDeepStrictEqual } from 'node:util';
import { CollabHistoryService } from '../services/collab-history.service';
import { WatcherService } from '../../core/watcher/watcher.service';
import { isEmptyParagraphDoc } from '../collaboration.util';
import { InjectKysely } from 'nestjs-kysely';
import { KyselyDB } from '@docmost/db/types/kysely.types';
import { executeTx } from '@docmost/db/utils';
@Processor(QueueName.HISTORY_QUEUE)
export class HistoryProcessor extends WorkerHost implements OnModuleDestroy {
@@ -32,7 +29,6 @@ export class HistoryProcessor extends WorkerHost implements OnModuleDestroy {
private readonly pageRepo: PageRepo,
private readonly collabHistory: CollabHistoryService,
private readonly watcherService: WatcherService,
@InjectKysely() private readonly db: KyselyDB,
@InjectQueue(QueueName.NOTIFICATION_QUEUE) private notificationQueue: Queue,
@InjectQueue(QueueName.GENERAL_QUEUE) private generalQueue: Queue,
) {
@@ -45,9 +41,6 @@ export class HistoryProcessor extends WorkerHost implements OnModuleDestroy {
try {
const { pageId } = job.data;
// Read the page WITHOUT a lock first, only to bail early on the two cheap
// no-write cases (page gone / empty first snapshot) without opening a
// transaction. The authoritative check-then-write happens locked below.
const page = await this.pageRepo.findById(pageId, {
includeContent: true,
});
@@ -58,109 +51,40 @@ export class HistoryProcessor extends WorkerHost implements OnModuleDestroy {
return;
}
// #370 F3 — the snapshot decision (findPageLastHistory → saveHistory) must
// be serialized against manual-save/boundary writers, which run under a
// page-row lock in onStoreDocument. Without it, this processor and a
// concurrent manual-save each read the same lastHistory (MVCC), both see
// content != lastHistory, and both insert — producing two page_history rows
// with IDENTICAL content (one 'idle', one 'manual'), defeating
// promote-not-dup and the version-vs-autosave split. Taking the same
// page-row lock makes the second writer observe the first's committed row so
// the isDeepStrictEqual gate collapses the duplicate. Only the read+write
// is transacted; the post-snapshot queue work stays outside.
let contributorIds: string[] = [];
let snapshotWritten = false;
let lastHistoryContent: unknown;
// #370 F8 — the contributor set popped from Redis (destructive SPOP) must be
// restored if the snapshot does not durably land. The inner try/catch only
// covers a throw INSIDE the callback; a COMMIT failure (connection drop,
// serialization/deadlock abort on commit — the transient class the epic
// already retries) throws OUTSIDE it, rolling the snapshot back while the
// pop is already gone. We track the popped set here and restore it in the
// outer catch so a BullMQ retry re-attributes the version. addContributors
// is an idempotent Redis SADD, so a double-restore is harmless.
let poppedForRestore: string[] = [];
const lastHistory = await this.pageHistoryRepo.findPageLastHistory(
pageId,
{ includeContent: true },
);
try {
await executeTx(this.db, async (trx) => {
const lockedPage = await this.pageRepo.findById(pageId, {
includeContent: true,
withLock: true,
trx,
});
if (!lockedPage) return;
const lastHistory = await this.pageHistoryRepo.findPageLastHistory(
pageId,
{ includeContent: true, trx },
);
lastHistoryContent = lastHistory?.content;
if (!lastHistory && isEmptyParagraphDoc(lockedPage.content as any)) {
this.logger.debug(
`Skipping first history for page ${pageId}: empty content`,
);
return;
}
if (
lastHistory &&
isDeepStrictEqual(lastHistory.content, lockedPage.content)
) {
return; // already snapshotted at this content — nothing to write
}
contributorIds = await this.collabHistory.popContributors(pageId);
poppedForRestore = contributorIds;
try {
// Pass `trx` so the watcher insert's FK check (FOR KEY SHARE on
// pages[pageId]) runs on the SAME connection that already holds the
// FOR UPDATE lock from findById — otherwise it takes the FK lock on a
// separate pool connection and self-deadlocks against our own tx.
await this.watcherService.addPageWatchers(
contributorIds,
pageId,
lockedPage.spaceId,
lockedPage.workspaceId,
trx,
);
// #370 — every job on this queue is a trailing idle-flush autosnapshot.
await this.pageHistoryRepo.saveHistory(lockedPage, {
contributorIds,
kind: job.data.kind ?? 'idle',
trx,
});
snapshotWritten = true;
this.logger.debug(`History created for page: ${pageId}`);
} catch (err) {
await this.collabHistory.addContributors(pageId, contributorIds);
poppedForRestore = [];
throw err;
}
});
} catch (err) {
// A throw here means the tx did NOT commit (callback threw, or the commit
// itself failed and rolled back). If we popped contributors and the inner
// catch did not already restore them, restore now so the retry keeps
// attribution. snapshotWritten is irrelevant: it is set before commit, so
// it can be true even when the commit rolled the snapshot back.
if (poppedForRestore.length) {
await this.collabHistory.addContributors(pageId, poppedForRestore);
}
throw err;
}
// No snapshot written (page vanished / empty-first / unchanged content) →
// clear the contributor set for the skip cases and stop.
if (!snapshotWritten) {
if (!lastHistoryContent && isEmptyParagraphDoc(page.content as any)) {
await this.collabHistory.clearContributors(pageId);
}
if (!lastHistory && isEmptyParagraphDoc(page.content as any)) {
this.logger.debug(
`Skipping first history for page ${pageId}: empty content`,
);
await this.collabHistory.clearContributors(pageId);
return;
}
{
if (
!lastHistory ||
!isDeepStrictEqual(lastHistory.content, page.content)
) {
const contributorIds = await this.collabHistory.popContributors(pageId);
try {
await this.watcherService.addPageWatchers(
contributorIds,
pageId,
page.spaceId,
page.workspaceId,
);
await this.pageHistoryRepo.saveHistory(page, { contributorIds });
this.logger.debug(`History created for page: ${pageId}`);
} catch (err) {
await this.collabHistory.addContributors(pageId, contributorIds);
throw err;
}
const mentions = extractMentions(page.content);
const pageMentions = extractPageMentions(mentions);
const internalLinkSlugIds = extractInternalLinkSlugIds(page.content);
@@ -178,7 +102,7 @@ export class HistoryProcessor extends WorkerHost implements OnModuleDestroy {
);
});
if (contributorIds.length > 0 && lastHistoryContent) {
if (contributorIds.length > 0 && lastHistory?.content) {
await this.notificationQueue
.add(QueueJob.PAGE_UPDATED, {
pageId,
@@ -529,107 +529,4 @@ describe('replaceYjsMarkedText', () => {
expect(result).toEqual({ applied: false, currentText: 'abcdef' });
expect(text.toDelta()).toEqual(before);
});
// #496: apply must NOT silently strip the replaced run's inline formatting.
// Build a paragraph and format the marked range with extra marks, then assert
// the replacement carries them.
function buildFormatted(
runs: Array<{ text: string; attrs?: Record<string, any> }>,
): { fragment: Y.XmlFragment; text: Y.XmlText } {
const ydoc = new Y.Doc();
const fragment = ydoc.getXmlFragment('default');
const para = new Y.XmlElement('paragraph');
fragment.insert(0, [para]);
const text = new Y.XmlText();
para.insert(0, [text]);
text.insert(0, runs.map((r) => r.text).join(''));
let offset = 0;
for (const run of runs) {
if (run.attrs) text.format(offset, run.text.length, run.attrs);
offset += run.text.length;
}
return { fragment, text };
}
it('preserves the original run formatting (bold + link) on the replacement', () => {
const { fragment, text } = buildFormatted([
{ text: 'see ' },
{
text: 'old',
attrs: {
comment: { commentId: 'c1', resolved: false },
bold: true,
link: { href: 'https://x.test' },
},
},
{ text: ' end' },
]);
const result = replaceYjsMarkedText(fragment, 'c1', 'old', 'new');
expect(result).toEqual({ applied: true, currentText: 'new' });
// The comment anchor AND the bold/link marks survive the delete+insert.
expect(text.toDelta()).toEqual([
{ insert: 'see ' },
{
insert: 'new',
attributes: {
comment: { commentId: 'c1', resolved: false },
bold: true,
link: { href: 'https://x.test' },
},
},
{ insert: ' end' },
]);
});
it('mixed formatting under the mark: replacement takes the DOMINANT (longest) run, NOT the leading one', () => {
// Leading run is SHORT + plain ("x", 1 char); the following run is LONGER +
// bold ("bolded", 6 chars), same commentId. The longest run is deliberately
// NOT first: a "first-wins" pick would carry plain (no bold), so asserting
// bold on the result only holds if the code genuinely selects the LONGEST run.
const { fragment, text } = buildFormatted([
{ text: 'x', attrs: { comment: { commentId: 'c1', resolved: false } } },
{
text: 'bolded',
attrs: { comment: { commentId: 'c1', resolved: false }, bold: true },
},
]);
const result = replaceYjsMarkedText(fragment, 'c1', 'xbolded', 'Z');
expect(result).toEqual({ applied: true, currentText: 'Z' });
expect(text.toDelta()).toEqual([
{
insert: 'Z',
attributes: { comment: { commentId: 'c1', resolved: false }, bold: true },
},
]);
});
it('mixed formatting under the mark: on a length tie the FIRST run wins', () => {
// Two equal-length runs (2 chars each) with different formatting, same
// commentId. The reduce keeps the accumulator on a tie, so the FIRST run
// (italic) prevails over the later bold one.
const { fragment, text } = buildFormatted([
{
text: 'AA',
attrs: { comment: { commentId: 'c1', resolved: false }, italic: true },
},
{
text: 'BB',
attrs: { comment: { commentId: 'c1', resolved: false }, bold: true },
},
]);
const result = replaceYjsMarkedText(fragment, 'c1', 'AABB', 'Z');
expect(result).toEqual({ applied: true, currentText: 'Z' });
expect(text.toDelta()).toEqual([
{
insert: 'Z',
attributes: { comment: { commentId: 'c1', resolved: false }, italic: true },
},
]);
});
});
+5 -20
View File
@@ -145,10 +145,6 @@ type MarkedSegment = {
length: number;
text: string;
markAttrs: Record<string, any>;
// The FULL attribute set of this delta run — the `comment` mark plus any
// inline formatting (bold/italic/code/link/…). Captured so apply can carry the
// original run's formatting onto the replacement instead of dropping it.
attributes: Record<string, any>;
};
/**
@@ -206,7 +202,6 @@ export function replaceYjsMarkedText(
length,
text: insert,
markAttrs: markAttr,
attributes,
});
}
offset += length;
@@ -256,25 +251,15 @@ export function replaceYjsMarkedText(
return { applied: false, currentText: joinedText };
}
// 3. All guards passed: delete the marked run and re-insert newText at the
// same offset. Atomic within the caller's transaction.
// 3. All guards passed: delete the marked run and re-insert newText with the
// same comment attributes at the same offset. Atomic within the caller's
// transaction.
const start = segments[0].offset;
const len = segments.reduce((sum, s) => sum + s.length, 0);
// Carry the ORIGINAL run's formatting onto the replacement (#496): inserting
// with only the `comment` mark silently dropped bold/italic/code/link of the
// replaced text. Yjs applies one flat attribute set to the whole insert, so
// when the marked run mixes formatting we pick the DOMINANT segment (the one
// covering the most characters) and apply its attributes — a v1 that preserves
// the common single-format case exactly and, for a mixed run, keeps the
// prevailing style rather than losing all of it. `attributes` already carries
// the `comment` mark (every collected segment is filtered on it above), so the
// anchor is preserved by copying the run's attribute set verbatim.
const dominant = segments.reduce((a, b) => (b.length > a.length ? b : a));
const insertAttrs = { ...dominant.attributes };
const markAttrs = segments[0].markAttrs;
node.delete(start, len);
node.insert(start, newText, insertAttrs);
node.insert(start, newText, { comment: markAttrs });
return { applied: true, currentText: newText };
}
@@ -16,7 +16,6 @@ import { UpdateCommentDto } from './dto/update-comment.dto';
import { ResolveCommentDto } from './dto/resolve-comment.dto';
import { ApplySuggestionDto } from './dto/apply-suggestion.dto';
import { DismissSuggestionDto } from './dto/dismiss-suggestion.dto';
import { ResyncSuggestionAnchorDto } from './dto/resync-suggestion-anchor.dto';
import { PageIdDto, CommentIdDto } from './dto/comments.input';
import { AuthUser } from '../../common/decorators/auth-user.decorator';
import { AuthWorkspace } from '../../common/decorators/auth-workspace.decorator';
@@ -236,39 +235,6 @@ export class CommentController {
return this.commentService.applySuggestion(comment, user, provenance);
}
@HttpCode(HttpStatus.OK)
@Post('resync-suggestion-anchor')
async resyncSuggestionAnchor(
@Body() dto: ResyncSuggestionAnchorDto,
@AuthUser() user: User,
@AuthWorkspace() workspace: Workspace,
) {
const comment = await this.commentRepo.findById(dto.commentId, {
includeCreator: true,
includeResolvedBy: true,
});
if (!comment) {
throw new NotFoundException('Comment not found');
}
const page = await this.pageRepo.findById(comment.pageId);
if (!page || page.deletedAt) {
throw new NotFoundException('Page not found');
}
// Authorize BEFORE revealing structural detail (mirrors apply/dismiss).
// Re-anchoring does NOT change the page text — it only corrects the stored
// selection metadata — so the page-level gate is comment access. The service
// further restricts it to the suggestion's own author.
await this.pageAccessService.validateCanComment(page, user, workspace.id);
return this.commentService.resyncSuggestionAnchor(
comment,
dto.selection,
user,
);
}
@HttpCode(HttpStatus.OK)
@Post('dismiss-suggestion')
async dismissSuggestion(
@@ -146,19 +146,11 @@ describe('CommentService — applySuggestion', () => {
'page-1',
expect.objectContaining({ operation: 'commentDeleted', commentId: 'c-1' }),
);
// #496: hard-deleted row → the audit payload is the only surviving record.
expect(auditService.log).toHaveBeenCalledWith(
expect.objectContaining({
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
resourceType: AuditResource.COMMENT,
resourceId: 'c-1',
metadata: expect.objectContaining({
pageId: 'page-1',
suggestedText: 'new text',
selection: 'old text',
commentAuthor: 'user-1',
decidedBy: 'user-1',
}),
}),
);
expect(result.outcome).toBe('deleted');
@@ -197,25 +189,17 @@ describe('CommentService — applySuggestion', () => {
expect(resolvePatch.resolvedAt).toBeInstanceOf(Date);
expect(resolvePatch.resolvedById).toBe('user-1');
// NOT deleted.
// NOT deleted; broadcast an update, not a deletion.
expect(commentRepo.deleteComment).not.toHaveBeenCalled();
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalledWith(
'deleteCommentMark',
expect.anything(),
expect.anything(),
);
// #496 dedup: resolveComment broadcasts `commentResolved` with the enriched
// row; finalize must NOT ALSO emit a redundant `commentUpdated`. So the
// thread receives exactly ONE resolve broadcast and no update broadcast.
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentResolved', comment: UPDATED }),
);
expect(wsService.emitCommentEvent).not.toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentUpdated' }),
expect.objectContaining({ operation: 'commentUpdated', comment: UPDATED }),
);
expect(auditService.log).toHaveBeenCalledWith(
@@ -227,36 +211,6 @@ describe('CommentService — applySuggestion', () => {
expect(result.outcome).toBe('resolved');
});
it('re-entry: already applied+resolved WITH replies → emits commentUpdated (dedup does not over-suppress)', async () => {
// suggestionAppliedAt set → idempotent finalize; resolvedAt set → resolveComment
// is skipped, so there is NO commentResolved broadcast. The applied-stamp state
// must still reach clients via a single commentUpdated.
const { service, wsService } = makeService(
{ applied: false, currentText: 'new text' },
true,
);
await service.applySuggestion(
suggestionComment({
suggestionAppliedAt: new Date(),
resolvedAt: new Date(),
}),
user(),
);
expect(wsService.emitCommentEvent).toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentUpdated', comment: UPDATED }),
);
// Nothing resolved this time (already resolved) → no resolve broadcast.
expect(wsService.emitCommentEvent).not.toHaveBeenCalledWith(
'space-1',
'page-1',
expect.objectContaining({ operation: 'commentResolved' }),
);
});
// --- error / rejection branches -----------------------------------------
it('applied=false and currentText differs → ConflictException with currentText in payload', async () => {
@@ -107,21 +107,11 @@ describe('CommentService — dismissSuggestion', () => {
'page-1',
expect.objectContaining({ operation: 'commentDeleted', commentId: 'c-1' }),
);
// #496: the row is hard-deleted, so the audit payload must carry the
// decision's substance (what was suggested, the anchored text, who authored
// it, who decided) — it is the only surviving record.
expect(auditService.log).toHaveBeenCalledWith(
expect.objectContaining({
event: AuditEvent.COMMENT_SUGGESTION_DISMISSED,
resourceType: AuditResource.COMMENT,
resourceId: 'c-1',
metadata: expect.objectContaining({
pageId: 'page-1',
suggestedText: 'new text',
selection: 'old text',
commentAuthor: 'user-1',
decidedBy: 'user-1',
}),
}),
);
expect(result.outcome).toBe('deleted');
@@ -1,126 +0,0 @@
import { BadRequestException, ForbiddenException } from '@nestjs/common';
import { CommentService } from './comment.service';
/**
* Coverage for CommentService.resyncSuggestionAnchor (#496): re-anchoring a
* suggestion's stored selection (== apply-time expectedText) to the live-doc
* substring. The service is built directly with jest-mocked deps (the
* @InjectQueue tokens can't be resolved by Test.createTestingModule — see the
* sibling specs).
*/
describe('CommentService — resyncSuggestionAnchor', () => {
const UPDATED = { id: 'c-1', selection: 'new anchor', __updated: true } as any;
function makeService() {
const commentRepo: any = {
updateComment: jest.fn(async () => undefined),
findById: jest.fn(async () => UPDATED),
};
const service = new CommentService(
commentRepo,
{} as any,
{ emitCommentEvent: jest.fn() } as any,
{} as any,
{ add: jest.fn() } as any,
{ add: jest.fn() } as any,
{ log: jest.fn() } as any,
);
return { service, commentRepo };
}
const suggestion = (over?: Partial<any>): any => ({
id: 'c-1',
creatorId: 'user-1',
parentCommentId: null,
selection: 'old anchor',
suggestedText: 'new text',
suggestionAppliedAt: null,
resolvedAt: null,
...over,
});
const user = (over?: Partial<any>): any => ({ id: 'user-1', ...over });
it('persists the new selection and returns the enriched comment', async () => {
const { service, commentRepo } = makeService();
const out = await service.resyncSuggestionAnchor(
suggestion(),
'new anchor',
user(),
);
expect(commentRepo.updateComment).toHaveBeenCalledWith(
{ selection: 'new anchor' },
'c-1',
);
expect(out).toBe(UPDATED);
});
it('is idempotent: no write when the anchor already matches', async () => {
const { service, commentRepo } = makeService();
const out = await service.resyncSuggestionAnchor(
suggestion({ selection: 'same' }),
'same',
user(),
);
expect(commentRepo.updateComment).not.toHaveBeenCalled();
expect(out).toEqual(suggestion({ selection: 'same' }));
});
it('rejects a non-author (only the suggestion owner may re-anchor)', async () => {
const { service, commentRepo } = makeService();
await expect(
service.resyncSuggestionAnchor(suggestion(), 'new anchor', user({ id: 'other' })),
).rejects.toBeInstanceOf(ForbiddenException);
expect(commentRepo.updateComment).not.toHaveBeenCalled();
});
it('rejects a reply / a comment with no suggestion', async () => {
const { service } = makeService();
await expect(
service.resyncSuggestionAnchor(
suggestion({ parentCommentId: 'p-1' }),
'x',
user(),
),
).rejects.toBeInstanceOf(BadRequestException);
await expect(
service.resyncSuggestionAnchor(
suggestion({ suggestedText: null }),
'x',
user(),
),
).rejects.toBeInstanceOf(BadRequestException);
});
it('rejects re-anchoring an already applied or resolved suggestion', async () => {
const { service } = makeService();
await expect(
service.resyncSuggestionAnchor(
suggestion({ suggestionAppliedAt: new Date() }),
'x',
user(),
),
).rejects.toBeInstanceOf(BadRequestException);
await expect(
service.resyncSuggestionAnchor(
suggestion({ resolvedAt: new Date() }),
'x',
user(),
),
).rejects.toBeInstanceOf(BadRequestException);
});
it('rejects a no-op selection equal to the suggested text', async () => {
const { service } = makeService();
await expect(
service.resyncSuggestionAnchor(
suggestion({ suggestedText: 'new text' }),
'new text',
user(),
),
).rejects.toBeInstanceOf(BadRequestException);
});
});
+17 -122
View File
@@ -370,76 +370,6 @@ export class CommentService {
return updatedComment;
}
/**
* Re-sync a suggestion's stored `selection` (== apply-time expectedText) to the
* RAW substring the inline mark actually covers in the LIVE document (#496).
*
* The MCP client creates the comment from a DEBOUNCED REST snapshot, then
* anchors the mark in the live collab doc. When the two disagree (the doc moved
* on in the debounce window) the stored selection no longer equals the marked
* text, so EVERY apply 409s ("the commented text changed"). After anchoring the
* client re-reads the exact marked substring and calls this to store it, making
* apply's strict equality hold.
*
* Only meaningful for an un-settled top-level suggestion authored by the
* caller: applying/resolving freezes the anchor, and a reply-carrying thread is
* preserved rather than mutated. The new text must still differ from the
* suggestion (else "apply" would be a no-op), preserving create()'s invariant.
*/
async resyncSuggestionAnchor(
comment: Comment,
selection: string,
user: User,
): Promise<Comment> {
if (comment.creatorId !== user.id) {
throw new ForbiddenException(
'You can only re-anchor your own suggestion',
);
}
if (comment.parentCommentId) {
throw new BadRequestException(
'Only a top-level comment can carry a suggested edit',
);
}
if (!comment.suggestedText) {
throw new BadRequestException('This comment has no suggested edit');
}
// A settled suggestion's anchor is frozen: re-anchoring an applied/resolved
// thread is meaningless and could resurrect a stale expectedText.
if (comment.suggestionAppliedAt || comment.resolvedAt) {
throw new BadRequestException(
'Cannot re-anchor a suggestion that was already applied or resolved',
);
}
const trimmed = selection.trim();
if (trimmed.length === 0) {
throw new BadRequestException('The re-anchored selection cannot be empty');
}
// Same no-op guard as create(): the suggestion must differ from the text it
// replaces, or apply becomes indistinguishable from already-applied.
if (trimmed === comment.suggestedText.trim()) {
throw new BadRequestException(
'A suggested edit must differ from the selected text',
);
}
// Idempotent: nothing to persist when the anchor already matches.
if (comment.selection === selection) {
return comment;
}
await this.commentRepo.updateComment({ selection }, comment.id);
const updatedComment = await this.commentRepo.findById(comment.id, {
includeCreator: true,
includeResolvedBy: true,
});
// Re-anchoring only corrects stored metadata; it does not change the page
// text or the comment body, so no ws broadcast / notification is warranted.
return updatedComment;
}
/**
* Apply the suggested edit carried by a top-level inline comment: atomically
* replace the text under the comment mark in the collaborative document with
@@ -594,7 +524,7 @@ export class CommentService {
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: this.suggestionAuditMetadata(comment, user),
metadata: { pageId: comment.pageId },
});
return { ...updatedComment, outcome: 'resolved' };
}
@@ -608,7 +538,7 @@ export class CommentService {
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: this.suggestionAuditMetadata(comment, user),
metadata: { pageId: comment.pageId },
});
return settled;
}
@@ -647,10 +577,8 @@ export class CommentService {
// Auto-resolve the thread. resolveComment handles the resolve mark, its ws
// broadcast and the resolve notification. Stay defensive on re-entry.
let didResolveBroadcast = false;
if (!comment.resolvedAt) {
await this.resolveComment(comment, true, user, provenance);
didResolveBroadcast = true;
}
const updatedComment = await this.commentRepo.findById(comment.id, {
@@ -658,27 +586,18 @@ export class CommentService {
includeResolvedBy: true,
});
// #496 dedup: resolveComment already broadcast `commentResolved` carrying
// the fully-enriched row (the applied stamps were persisted above, before
// that call, so its re-read reflects them). Emitting `commentUpdated` here
// too made the client receive TWO events for one apply. Broadcast the
// update ONLY when we did NOT resolve — i.e. the rare re-entry on an
// already-resolved thread, where the applied-stamp change still needs a
// broadcast and resolveComment did not run.
if (!didResolveBroadcast) {
this.wsService.emitCommentEvent(comment.spaceId, comment.pageId, {
operation: 'commentUpdated',
pageId: comment.pageId,
comment: updatedComment,
});
}
this.wsService.emitCommentEvent(comment.spaceId, comment.pageId, {
operation: 'commentUpdated',
pageId: comment.pageId,
comment: updatedComment,
});
this.auditService.log({
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: this.suggestionAuditMetadata(comment, user),
metadata: { pageId: comment.pageId },
});
return { ...updatedComment, outcome: 'resolved' };
@@ -697,7 +616,7 @@ export class CommentService {
resourceType: AuditResource.COMMENT,
resourceId: comment.id,
spaceId: comment.spaceId,
metadata: this.suggestionAuditMetadata(comment, user),
metadata: { pageId: comment.pageId },
});
return settled;
@@ -708,17 +627,14 @@ export class CommentService {
* inline `comment` anchor mark, then ATOMICALLY hard-delete the row only if it
* is still childless. Shared by the apply/dismiss no-replies branches (#329).
*
* ORDER MATTERS (updated #399 → #496): what runs FIRST and FATALLY here is the
* mark-removal ENQUEUE (a fast, durable Redis add), NOT the mark op itself.
* deleteCommentMark awaits only the enqueue, so a failed add throws BEFORE the
* irreversible row delete — the row + mark stay consistent and the operation is
* repeatable. The actual anchor strip then runs off the HTTP path in the worker
* (idempotent, 3 retries). Only an EXHAUSTED-retries job could leave the doc
* with an orphan anchor pointing at a hard-deleted comment (the data-integrity
* bug #329 targets); that residual divergence is now self-healed by the
* resolve/unresolve mark worker, which strips an orphan mark whenever its
* comment row is gone (#496), and it is meanwhile VISIBLE via BullMQ failed-job
* metrics rather than a silently-swallowed warn.
* ORDER MATTERS: the anchor mark is removed FIRST and FATALLY (mirrors
* applySuggestion, which mutates the doc before writing the DB). The row
* delete is irreversible, so if the mark removal fails — including the
* COLLAB_DISABLE_REDIS "no live instance" hard-error — we must NOT delete the
* row and report success, or the document is left with a permanent orphan
* anchor pointing at a comment that no longer exists (the exact data-integrity
* bug #329 targets). Let the exception propagate (→ 5xx); the operation is
* then repeatable with row + mark still consistent.
*
* RACE (#338 F4): the caller read `hasChildren` BEFORE the (slow) mark
* removal, so a reply can land in that window. `comments.parent_comment_id` is
@@ -816,27 +732,6 @@ export class CommentService {
return this.generalQueue.add(QueueJob.COMMENT_MARK_UPDATE, jobData);
}
/**
* Build the audit metadata for a suggestion apply/dismiss decision (#496).
* The subject comment is HARD-DELETED on the childless path, so the audit row
* is the only surviving record — capture the decision's substance (what was
* suggested, the anchored text it replaced, who authored it, who decided)
* before the row can vanish. `decidedBy` is the acting user; `commentAuthor`
* is the suggestion's creator.
*/
private suggestionAuditMetadata(
comment: Comment,
user: User,
): Record<string, any> {
return {
pageId: comment.pageId,
suggestedText: comment.suggestedText ?? null,
selection: comment.selection ?? null,
commentAuthor: comment.creatorId ?? null,
decidedBy: user.id,
};
}
private async queueCommentNotification(
content: any,
oldMentionIds: string[],
@@ -1,20 +0,0 @@
import { IsString, IsUUID, MaxLength, MinLength } from 'class-validator';
/**
* #496: after the MCP client anchors a suggestion in the LIVE collab doc, it
* re-reads the exact substring under the new mark and syncs it here as the
* comment's stored `selection` (== apply-time expectedText). Fixes the perpetual
* 409 where expectedText came from a debounced REST snapshot while the mark sat
* in the live doc.
*/
export class ResyncSuggestionAnchorDto {
@IsUUID()
commentId: string;
// The raw substring the mark now covers in the live document. Bounded like the
// create-time selection (2000) so a legitimate anchored span is never cut.
@IsString()
@MinLength(1)
@MaxLength(2000)
selection: string;
}
@@ -1,27 +0,0 @@
import { type Kysely } from 'kysely';
/**
* #370 — page-versioning intentionality tier on a history snapshot.
*
* Adds `page_history.kind`, the three-tier "how intentional was this snapshot"
* marker that lets versions (intentional points) be told apart from autosaves:
* - 'manual' — a human explicitly saved a version (Cmd+S / Save button)
* - 'agent' — the AI agent explicitly saved a version
* - 'idle' — trailing idle-flush autosnapshot (safety net)
* - 'boundary' — autosnapshot pinned on a source transition (user↔agent↔git)
*
* Nullable with NO default (mirrors last_updated_source in the agent-provenance
* migration): legacy rows predate the marker and read back as `null`, which the
* client renders as a plain autosave. Stored as a short varchar to stay
* forward-compatible without an enum migration.
*/
export async function up(db: Kysely<any>): Promise<void> {
await db.schema
.alterTable('page_history')
.addColumn('kind', 'varchar(20)', (col) => col)
.execute();
}
export async function down(db: Kysely<any>): Promise<void> {
await db.schema.alterTable('page_history').dropColumn('kind').execute();
}
@@ -13,7 +13,6 @@ import { jsonArrayFrom, jsonObjectFrom } from 'kysely/helpers/postgres';
import { ExpressionBuilder, sql } from 'kysely';
import { DB } from '@docmost/db/types/db';
import { resolveAgentProvenance } from '../agent-provenance';
import { PageHistoryKind } from '../../../collaboration/constants';
/**
* Role-resolution subquery for a page-history row's bound AI chat (#300). Joins
@@ -47,9 +46,6 @@ export class PageHistoryRepo {
'lastUpdatedById',
'lastUpdatedSource',
'lastUpdatedAiChatId',
// #370 — intentionality tier ('manual' | 'agent' | 'idle' | 'boundary');
// null on legacy rows (= autosave). Selected so callers can read/promote it.
'kind',
'contributorIds',
'spaceId',
'workspaceId',
@@ -89,15 +85,9 @@ export class PageHistoryRepo {
async saveHistory(
page: Page,
opts?: {
contributorIds?: string[];
// #370 — intentionality tier for this snapshot. Omitted → null (legacy
// autosave semantics). Callers derive it server-side, never from a client.
kind?: PageHistoryKind;
trx?: KyselyTransaction;
},
): Promise<PageHistory> {
return await this.insertPageHistory(
opts?: { contributorIds?: string[]; trx?: KyselyTransaction },
): Promise<void> {
await this.insertPageHistory(
{
pageId: page.id,
slugId: page.slugId,
@@ -109,7 +99,6 @@ export class PageHistoryRepo {
// Copy the provenance marker off the page row, as for lastUpdatedById.
lastUpdatedSource: page.lastUpdatedSource,
lastUpdatedAiChatId: page.lastUpdatedAiChatId,
kind: opts?.kind ?? null,
contributorIds: opts?.contributorIds,
spaceId: page.spaceId,
workspaceId: page.workspaceId,
@@ -118,25 +107,6 @@ export class PageHistoryRepo {
);
}
/**
* #370 — promote an existing snapshot's intentionality tier in place. Used by
* the manual-save "promote-not-dup" path: when the latest history row already
* holds the exact content being versioned, we upgrade its `kind` instead of
* duplicating a heavy content row.
*/
async updateHistoryKind(
pageHistoryId: string,
kind: PageHistoryKind,
trx?: KyselyTransaction,
): Promise<void> {
const db = dbOrTx(this.db, trx);
await db
.updateTable('pageHistory')
.set({ kind })
.where('id', '=', pageHistoryId)
.execute();
}
async findPageHistoryByPageId(pageId: string, pagination: PaginationOptions) {
const query = this.db
.selectFrom('pageHistory')
-1
View File
@@ -280,7 +280,6 @@ export interface PageHistory {
createdAt: Generated<Timestamp>;
icon: string | null;
id: Generated<string>;
kind: string | null;
lastUpdatedAiChatId: string | null;
lastUpdatedById: string | null;
lastUpdatedSource: string | null;
@@ -1,19 +1,14 @@
import { Global, Module } from '@nestjs/common';
import { AUDIT_SERVICE } from './audit.service';
import { DatabaseAuditService } from './database-audit.service';
import { AUDIT_SERVICE, NoopAuditService } from './audit.service';
// #496: bind the audit token to a real DB-backed trail (was NoopAuditService,
// which silently dropped every event). Kysely (@Global DatabaseModule) and
// ClsService (@Global ClsModule) are both globally available, so this module
// needs no extra imports.
@Global()
@Module({
providers: [
{
provide: AUDIT_SERVICE,
useClass: DatabaseAuditService,
useClass: NoopAuditService,
},
],
exports: [AUDIT_SERVICE],
})
export class AuditModule {}
export class NoopAuditModule {}
@@ -1,198 +0,0 @@
import { Logger } from '@nestjs/common';
import { DatabaseAuditService } from './database-audit.service';
import { AUDIT_CONTEXT_KEY } from '../../common/middlewares/audit-context.middleware';
import { AuditEvent, AuditResource } from '../../common/events/audit-events';
/**
* Observable-property coverage for the DB-backed audit trail (#496): every
* assertion pins what actually reaches the `audit` table (or that nothing does),
* driven through a chainable Kysely mock that captures the inserted rows.
*/
describe('DatabaseAuditService', () => {
function makeService(clsContext: any) {
const inserted: any[] = [];
const updated: any[] = [];
let failNextInsert = false;
const db: any = {
insertInto: jest.fn(() => ({
values: jest.fn((rows: any) => ({
execute: jest.fn(async () => {
if (failNextInsert) {
failNextInsert = false;
throw new Error('boom');
}
inserted.push(rows);
}),
})),
})),
updateTable: jest.fn(() => ({
set: jest.fn((patch: any) => ({
where: jest.fn(() => ({
execute: jest.fn(async () => {
updated.push(patch);
}),
})),
})),
})),
};
const store: Record<string, any> = { [AUDIT_CONTEXT_KEY]: clsContext };
const cls: any = {
get: jest.fn((key: string) => store[key]),
set: jest.fn((key: string, val: any) => {
store[key] = val;
}),
};
const service = new DatabaseAuditService(db, cls);
return {
service,
inserted,
updated,
cls,
store,
failInsert: () => {
failNextInsert = true;
},
};
}
const applyPayload = () => ({
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
resourceType: AuditResource.COMMENT,
resourceId: 'c-1',
spaceId: 'space-1',
metadata: { pageId: 'p-1', suggestedText: 'new', decidedBy: 'u-2' },
});
const ctx = (over?: any) => ({
workspaceId: 'ws-1',
actorId: 'u-2',
actorType: 'user',
ipAddress: '10.0.0.1',
userAgent: 'jest',
...over,
});
it('log() persists a row with the CLS context merged onto the payload', async () => {
const { service, inserted } = makeService(ctx());
service.log(applyPayload());
// log() is fire-and-forget; flush the microtask queue.
await Promise.resolve();
await Promise.resolve();
expect(inserted).toHaveLength(1);
expect(inserted[0]).toMatchObject({
workspaceId: 'ws-1',
actorId: 'u-2',
actorType: 'user',
event: AuditEvent.COMMENT_SUGGESTION_APPLIED,
resourceType: AuditResource.COMMENT,
resourceId: 'c-1',
spaceId: 'space-1',
ipAddress: '10.0.0.1',
metadata: { pageId: 'p-1', suggestedText: 'new', decidedBy: 'u-2' },
});
});
it('log() is a no-op when there is no workspace in scope', async () => {
const { service, inserted } = makeService(ctx({ workspaceId: null }));
service.log(applyPayload());
await Promise.resolve();
expect(inserted).toHaveLength(0);
});
it('log() drops EXCLUDED_AUDIT_EVENTS (e.g. comment.created)', async () => {
const { service, inserted } = makeService(ctx());
service.log({
event: AuditEvent.COMMENT_CREATED,
resourceType: AuditResource.COMMENT,
resourceId: 'c-1',
spaceId: 'space-1',
});
await Promise.resolve();
await Promise.resolve();
expect(inserted).toHaveLength(0);
});
it('a failed insert is swallowed with a warn and floats no rejection (audit is a side-record)', async () => {
// This pins the load-bearing swallow inside persist(). Because log() is
// fire-and-forget (`void this.persist(...)`), it always returns synchronously
// without throwing — so `not.toThrow()` alone would stay green even if the
// try/catch were removed. We instead observe the two effects the catch is
// responsible for: a warn IS emitted, and NO unhandled rejection floats.
// Removing persist()'s try/catch reddens both assertions (warn count 0 + a
// captured rejection).
const warnSpy = jest
.spyOn(Logger.prototype, 'warn')
.mockImplementation(() => undefined as any);
const rejections: unknown[] = [];
const onRejection = (err: unknown) => rejections.push(err);
process.on('unhandledRejection', onRejection);
try {
const { service, failInsert } = makeService(ctx());
failInsert();
expect(() => service.log(applyPayload())).not.toThrow();
// Flush microtasks so the rejected insert settles, then give any floated
// rejection a macrotask tick to be reported by the runtime.
await Promise.resolve();
await Promise.resolve();
await new Promise((r) => setImmediate(r));
expect(warnSpy).toHaveBeenCalledTimes(1);
expect(String(warnSpy.mock.calls[0][0])).toContain(
'Failed to persist audit event',
);
expect(rejections).toHaveLength(0);
} finally {
process.off('unhandledRejection', onRejection);
warnSpy.mockRestore();
}
});
it('logWithContext() persists with an explicit (non-request) context', async () => {
const { service, inserted } = makeService(undefined);
service.logWithContext(applyPayload(), ctx({ actorType: 'system' }) as any);
await Promise.resolve();
await Promise.resolve();
expect(inserted).toHaveLength(1);
expect(inserted[0].actorType).toBe('system');
expect(inserted[0].workspaceId).toBe('ws-1');
});
it('logBatchWithContext() inserts only non-excluded events', async () => {
const { service, inserted } = makeService(undefined);
service.logBatchWithContext(
[
applyPayload(),
{
event: AuditEvent.COMMENT_CREATED,
resourceType: AuditResource.COMMENT,
resourceId: 'c-2',
},
],
ctx() as any,
);
await Promise.resolve();
await Promise.resolve();
// Batch is a single insert call carrying only the applied event.
expect(inserted).toHaveLength(1);
expect(inserted[0]).toHaveLength(1);
expect(inserted[0][0].event).toBe(AuditEvent.COMMENT_SUGGESTION_APPLIED);
});
it('setActorId / setActorType mutate the ambient CLS context', () => {
const { service, store } = makeService(ctx({ actorId: null }));
service.setActorId('u-9');
service.setActorType('api_key');
expect(store[AUDIT_CONTEXT_KEY].actorId).toBe('u-9');
expect(store[AUDIT_CONTEXT_KEY].actorType).toBe('api_key');
});
it('updateRetention() writes the workspace retention window', async () => {
const { service, updated } = makeService(ctx());
await service.updateRetention('ws-1', 30);
expect(updated).toEqual([{ auditRetentionDays: 30 }]);
});
});
@@ -1,160 +0,0 @@
import { Injectable, Logger } from '@nestjs/common';
import { InjectKysely } from 'nestjs-kysely';
import { ClsService } from 'nestjs-cls';
import { KyselyDB } from '@docmost/db/types/kysely.types';
import {
AuditContext,
AUDIT_CONTEXT_KEY,
} from '../../common/middlewares/audit-context.middleware';
import {
AuditLogPayload,
ActorType,
EXCLUDED_AUDIT_EVENTS,
} from '../../common/events/audit-events';
import { AuditLogContext, IAuditService } from './audit.service';
/**
* Minimal DB-backed audit trail (#496). Replaces NoopAuditService so that
* decision-bearing events — notably comment.suggestion_applied /
* comment.suggestion_dismissed, whose subject comment is HARD-DELETED on the
* childless path — leave a durable record of who decided what. Without this the
* events were emitted (comment.service / *.controller) but swallowed, so an
* applied/dismissed suggestion was unrecoverable once the row was gone.
*
* Rows land in the pre-existing `audit` table (migration 20260228T223532). The
* per-request actor/workspace/ip come from the CLS AuditContext populated by
* AuditContextMiddleware + AuditActorInterceptor; callers that run OUTSIDE a
* request (queue workers, imports) pass an explicit context via
* logWithContext / logBatchWithContext.
*
* Audit is a side-record: a write failure MUST NOT break the originating
* request, so every persistence path swallows its error with a warn. Events in
* EXCLUDED_AUDIT_EVENTS (high-volume/low-signal) are dropped.
*/
@Injectable()
export class DatabaseAuditService implements IAuditService {
private readonly logger = new Logger(DatabaseAuditService.name);
constructor(
@InjectKysely() private readonly db: KyselyDB,
private readonly cls: ClsService,
) {}
/**
* Persist a single event using the ambient request-scoped AuditContext. A
* no-op when there is no workspace in scope (the table's workspace_id is NOT
* NULL) or the event is excluded. Fire-and-forget: the returned promise is not
* awaited by hot callers, and its rejection is swallowed here.
*/
log(payload: AuditLogPayload): void {
const context = this.cls?.get<AuditContext>(AUDIT_CONTEXT_KEY);
if (!context?.workspaceId) {
// No workspace in scope — nothing we can attribute the row to. This is
// expected for events emitted outside an HTTP request; those callers must
// use logWithContext instead.
return;
}
void this.persist(payload, {
workspaceId: context.workspaceId,
actorId: context.actorId ?? undefined,
actorType: context.actorType,
ipAddress: context.ipAddress ?? undefined,
userAgent: context.userAgent ?? undefined,
});
}
/** Persist a single event with an explicit (non-request) context. */
logWithContext(payload: AuditLogPayload, context: AuditLogContext): void {
if (!context?.workspaceId) return;
void this.persist(payload, context);
}
/** Persist a batch of events sharing one explicit context (imports). */
logBatchWithContext(
payloads: AuditLogPayload[],
context: AuditLogContext,
): void {
if (!context?.workspaceId || payloads.length === 0) return;
const rows = payloads
.filter((p) => !EXCLUDED_AUDIT_EVENTS.has(p.event))
.map((p) => this.toRow(p, context));
if (rows.length === 0) return;
this.db
.insertInto('audit')
.values(rows)
.execute()
.catch((err: any) =>
this.logger.warn(`Failed to persist ${rows.length} audit events: ${err?.message}`),
);
}
/** Update the ambient request actor (e.g. after login resolves the user). */
setActorId(actorId: string): void {
const context = this.cls?.get<AuditContext>(AUDIT_CONTEXT_KEY);
if (context) {
context.actorId = actorId;
this.cls.set(AUDIT_CONTEXT_KEY, context);
}
}
/** Update the ambient request actor type (user | system | api_key). */
setActorType(actorType: ActorType): void {
const context = this.cls?.get<AuditContext>(AUDIT_CONTEXT_KEY);
if (context) {
context.actorType = actorType;
this.cls.set(AUDIT_CONTEXT_KEY, context);
}
}
/** Persist a workspace's audit-log retention window (days). */
async updateRetention(
workspaceId: string,
retentionDays: number,
): Promise<void> {
try {
await this.db
.updateTable('workspaces')
.set({ auditRetentionDays: retentionDays })
.where('id', '=', workspaceId)
.execute();
} catch (err: any) {
this.logger.warn(
`Failed to update audit retention for workspace ${workspaceId}: ${err?.message}`,
);
}
}
private async persist(
payload: AuditLogPayload,
context: AuditLogContext,
): Promise<void> {
if (EXCLUDED_AUDIT_EVENTS.has(payload.event)) return;
try {
await this.db
.insertInto('audit')
.values(this.toRow(payload, context))
.execute();
} catch (err: any) {
// Audit is a side-record; never let a failed write surface to the caller.
this.logger.warn(
`Failed to persist audit event ${payload.event}: ${err?.message}`,
);
}
}
private toRow(payload: AuditLogPayload, context: AuditLogContext) {
return {
workspaceId: context.workspaceId,
actorId: context.actorId ?? null,
actorType: context.actorType ?? 'user',
event: payload.event,
resourceType: payload.resourceType,
resourceId: payload.resourceId ?? null,
spaceId: payload.spaceId ?? null,
// jsonb columns: node-postgres serializes plain objects to JSON.
changes: payload.changes ? (payload.changes as any) : null,
metadata: payload.metadata ? (payload.metadata as any) : null,
ipAddress: context.ipAddress ?? null,
};
}
}
@@ -20,10 +20,6 @@ export interface IStripeSeatsSyncJob {
export interface IPageHistoryJob {
pageId: string;
// #370 — intentionality tier the worker stamps on the snapshot. All jobs on
// this queue are trailing idle-flush autosnapshots, so this is 'idle' (absent
// → treated as 'idle' by the processor).
kind?: 'idle';
}
/**
@@ -139,19 +139,13 @@ describe('GeneralQueueProcessor — COMMENT_MARK_UPDATE (#399)', () => {
);
});
it('reconcile (#496): comment row vanished → strips the orphan anchor mark', async () => {
it('skips (no throw) when the comment row has vanished', async () => {
const { proc, collaborationGateway, commentRepo } = makeProc();
commentRepo.findById.mockResolvedValue(undefined);
await expect(
proc.process(job({ ...base, action: 'resolve', ts: 1000 })),
).resolves.toBeUndefined();
// A resolve/unresolve mark job whose comment row is gone leaves a silent
// orphan; the worker self-heals by stripping the anchor instead of returning.
expect(collaborationGateway.handleYjsEvent).toHaveBeenCalledWith(
'deleteCommentMark',
'page.page-1',
{ commentId: 'c-1', user: { id: 'user-1' } },
);
expect(collaborationGateway.handleYjsEvent).not.toHaveBeenCalled();
});
});
@@ -95,8 +95,7 @@ export class GeneralQueueProcessor
* #399: apply a comment's inline-mark mirror in the collab Y.Doc, off the HTTP
* critical path. Runs the SAME gateway path the synchronous comment.service
* code used (byte-identical mark op):
* - resolve / unresolve → resolveCommentMark (flip the `resolved` attribute),
* OR strip an orphan anchor when the comment row has vanished (#496);
* - resolve / unresolve → resolveCommentMark (flip the `resolved` attribute);
* - delete → deleteCommentMark (strip the ephemeral-suggestion anchor #329).
* The op is idempotent, so a BullMQ retry is safe. Throwing propagates to
* WorkerHost → the job is retried and, on exhaustion, surfaces in failed-job
@@ -134,18 +133,7 @@ export class GeneralQueueProcessor
// of this resolve), skip it rather than flip the mark to a stale state.
const comment = await this.commentRepo.findById(commentId);
if (!comment) {
// #496 reconcile: the comment row is GONE (e.g. an ephemeral apply/dismiss
// hard-deleted it while this resolve/unresolve mark job sat in the queue),
// but its inline anchor may still live in the doc — a silent orphan mark
// pointing at a comment that no longer exists. Self-heal by stripping it
// instead of just returning: this closes the divergence the fire-and-forget
// resolve/unresolve enqueue (comment.service resolveComment) could leave.
// Idempotent — deleteCommentMark on an already-absent mark is a no-op.
await this.getCollaborationGateway().handleYjsEvent(
'deleteCommentMark',
documentName,
{ commentId, user },
);
// The comment vanished (e.g. hard-deleted) → nothing left to mirror.
return;
}
const wantResolved = action === 'resolve';
@@ -1,162 +0,0 @@
import { randomUUID } from 'node:crypto';
import { Queue, Worker } from 'bullmq';
import { PersistenceExtension } from '../../src/collaboration/extensions/persistence.extension';
/**
* #370 — integration property of the idle-snapshot pipeline against REAL BullMQ.
*
* This is deliberately NOT a unit test of computeHistoryJob (that lives in
* compute-history-job.spec.ts). The point here is the OBSERVABLE end-to-end
* behaviour of the production `enqueuePageHistory` remove-then-add debounce
* driving a real Redis-backed delayed queue + worker (the #431→#439 class: a
* locally-correct function whose queue/timer property was never exercised):
*
* - a CONTINUOUS burst of stores lasting several caps yields periodic idle
* snapshots — at least one per max-wait cap, NOT one-per-store; and
* - an INTERMITTENT burst (a few stores, then quiet) yields exactly ONE
* trailing snapshot.
*
* We shrink the idle windows to milliseconds (jest.mock of collaboration
* constants) so real BullMQ delayed jobs actually promote within the test —
* fake timers cannot advance Redis's own delayed-set clock, so the intervals
* must be real but tiny. The production method under test is called verbatim.
*/
// NOTE: jest.mock is hoisted above the module's const initializers, so its
// factory cannot close over MAX_WAIT_MS/INTERVAL_MS — the literals are inlined
// here and MUST stay in sync with the consts below (a single source of truth is
// impossible across the hoist boundary).
jest.mock('../../src/collaboration/constants', () => {
const actual = jest.requireActual('../../src/collaboration/constants');
return {
...actual,
IDLE_MAX_WAIT_USER: 300,
IDLE_MAX_WAIT_AGENT: 300,
IDLE_INTERVAL_USER: 1000,
IDLE_INTERVAL_AGENT: 1000,
};
});
// Mirrors the mocked IDLE_MAX_WAIT_* above (IDLE_INTERVAL_* is 1000 > this, so
// the max-wait ceiling is what actually governs the trailing delay).
const MAX_WAIT_MS = 300;
const REDIS_CONNECTION = {
host: process.env.TEST_REDIS_HOST ?? '127.0.0.1',
port: Number(process.env.TEST_REDIS_PORT ?? 6379),
};
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
describe('#370 idle-snapshot pipeline (real BullMQ)', () => {
let queue: Queue;
let worker: Worker;
let extension: PersistenceExtension;
// Every processed snapshot, tagged by pageId so the two scenarios stay isolated.
const processed: Array<{ pageId: string; kind: string; at: number }> = [];
const queueName = `history-idle-int-${randomUUID()}`;
beforeAll(async () => {
queue = new Queue(queueName, {
connection: REDIS_CONNECTION,
// Mirror the production default (BullModule.forRoot removeOnComplete): the
// enqueue idiom relies on the jobId being freed once a job completes so the
// next burst can re-arm the same id.
defaultJobOptions: { removeOnComplete: true, removeOnFail: true },
});
await queue.waitUntilReady();
worker = new Worker(
queueName,
async (job) => {
processed.push({
pageId: job.data?.pageId,
kind: job.data?.kind,
at: Date.now(),
});
},
{ connection: REDIS_CONNECTION },
);
await worker.waitUntilReady();
// Construct the real extension; only historyQueue (5th ctor arg) and the
// internal idleBurstStart map are exercised by enqueuePageHistory, so the
// other collaborators can be null — the constructor only assigns fields.
extension = new PersistenceExtension(
null as any, // pageRepo
null as any, // pageHistoryRepo
null as any, // db
null as any, // aiQueue
queue as any, // historyQueue
null as any, // notificationQueue
null as any, // collabHistory
null as any, // transclusionService
);
});
afterAll(async () => {
// Force-close and fully drain so no BullMQ background activity (delayed-set
// polling, blocking BRPOPLPUSH) bleeds into later suites in this single
// shared jest worker (maxWorkers: 1).
await worker?.close(true).catch(() => undefined);
await queue?.obliterate({ force: true }).catch(() => undefined);
await queue?.close();
// Let the redis sockets settle before the next suite starts.
await sleep(150);
});
const arm = (pageId: string) =>
(extension as any).enqueuePageHistory({ id: pageId }, 'user');
it('continuous burst over several caps → periodic idle snapshots (≥1 per cap, not one-per-store)', async () => {
const pageId = randomUUID();
const runMs = 6 * MAX_WAIT_MS; // ~6 caps of unbroken editing
// Store cadence that does NOT evenly divide the cap: real hocuspocus stores
// are not aligned to cap boundaries, so a boundary job promotes in the gap
// before the next store's remove(). A cap-aligned cadence would instead land
// a store exactly on every boundary and lose the snapshot to the documented
// remove-vs-active race — an artefact of the test clock, not the pipeline.
const stepMs = 70;
const stores = Math.floor(runMs / stepMs);
const start = Date.now();
let count = 0;
while (Date.now() - start < runMs) {
await arm(pageId);
count++;
await sleep(stepMs);
}
// Let the final armed job flush.
await sleep(2 * MAX_WAIT_MS);
const snaps = processed.filter((p) => p.pageId === pageId);
// Every autosnapshot is an idle-kind row.
expect(snaps.every((s) => s.kind === 'idle')).toBe(true);
// Periodic: at least one per cap over a multi-cap burst (lower-bounded loosely
// to stay robust; the property is "fires at least every cap", not a single
// trailing snapshot).
expect(snaps.length).toBeGreaterThanOrEqual(3);
// But NOT one-per-store: ~`stores` stores were issued; the debounce must
// collapse them to a small multiple of the cap count, nowhere near per-store.
expect(snaps.length).toBeLessThanOrEqual(Math.ceil(stores / 2));
});
it('intermittent burst (a few stores, then quiet) → exactly ONE trailing snapshot', async () => {
const pageId = randomUUID();
// A short burst well within a single cap window, then silence.
await arm(pageId);
await sleep(40);
await arm(pageId);
await sleep(40);
await arm(pageId);
// Wait comfortably past the cap so the single pending trailing job fires.
await sleep(4 * MAX_WAIT_MS);
const snaps = processed.filter((p) => p.pageId === pageId);
expect(snaps).toHaveLength(1);
expect(snaps[0].kind).toBe('idle');
});
});
+1 -1
View File
@@ -387,7 +387,7 @@ bucketByDay(sessions, tz):
факты ниже — ground truth, можно дозапросить файлы через gitea MCP по указанному SHA):
- `page_history.kind``varchar(20)`, NULLABLE, БЕЗ дефолта (migration
`20260707T120000-page-history-kind.ts`). Домен: `manual`/`agent`/`idle`/`boundary`;
`20260705T120000-page-history-kind.ts`). Домен: `manual`/`agent`/`idle`/`boundary`;
legacy `null` = автосейв (`collaboration/constants.ts`, `PageHistoryKind`).
- `kind` УЖЕ включён в `PageHistoryRepo.baseFields` (`page-history.repo.ts`) — читается всеми
выборками истории. `saveHistory({kind})` и `updateHistoryKind(id, kind)` существуют.
-42
View File
@@ -450,12 +450,6 @@ export function CommentsMixin<TBase extends GConstructor<DocmostClientContext>>(
// can surface the closest-block / spans-multiple-blocks hint built from the
// LIVE document (the pre-check page is not in scope there).
let liveNotFoundError: Error | null = null;
// #496: the RAW substring the mark actually covers in the LIVE doc. The
// stored selection (payload.selection) came from a DEBOUNCED REST snapshot,
// which can differ from the live doc — and apply compares the marked live
// text to the stored selection strictly, so a stale snapshot 409s on EVERY
// apply. Captured here (same doc version the mark is set in) and synced below.
let liveAnchoredSelection: string | null = null;
try {
const collabToken = await this.getCollabTokenWithReauth();
// Open the collab doc by the canonical UUID, never the slugId (#260). The
@@ -495,12 +489,6 @@ export function CommentsMixin<TBase extends GConstructor<DocmostClientContext>>(
}
if (applyAnchorInDoc(doc, selection as string, newCommentId)) {
anchored = true;
// For a suggestion, re-read the exact substring now under the mark
// (the mark is an attribute, so it does not change the raw text) to
// sync as the stored expectedText after the mutation resolves.
if (hasSuggestion) {
liveAnchoredSelection = getAnchoredText(doc, selection as string);
}
return doc;
}
// Selection text not found in the LIVE document: abort the write. The
@@ -539,36 +527,6 @@ export function CommentsMixin<TBase extends GConstructor<DocmostClientContext>>(
);
}
// #496: sync the stored selection (== apply-time expectedText) to the RAW
// substring the mark actually covers in the LIVE doc when it diverged from
// the debounced REST snapshot we stored at create time. Without this, apply
// strictly compares the marked live text to a stale stored selection and
// 409s every time. Best-effort: the comment is already correctly anchored, so
// a resync failure must NOT roll it back — it only risks a later apply 409,
// which we surface as a soft warning.
if (
hasSuggestion &&
liveAnchoredSelection != null &&
liveAnchoredSelection !== payload.selection
) {
try {
await this.client.post("/comments/resync-suggestion-anchor", {
commentId: newCommentId,
selection: liveAnchoredSelection,
});
// Reflect the corrected anchor in the returned comment.
if (result.data) result.data.selection = liveAnchoredSelection;
} catch (e) {
if (process.env.DEBUG) {
console.error("Failed to resync suggestion anchor:", e);
}
result.warning =
"The suggestion was anchored, but its stored selection could not be " +
"synced to the live document; applying it may report a conflict if the " +
"text changed. Re-create the suggestion if Apply fails.";
}
}
// Soft warning (like editPageText): the selection only matched after
// stripping markdown, so the caller likely quoted a styled fragment.
if (anchorNormalized) {
@@ -553,189 +553,6 @@ test("suggestedText: the stored selection is the doc's RAW typographic substring
assert.equal(createPayload.suggestedText, "goodbye");
});
// -----------------------------------------------------------------------------
// 8b) #496: the DEBOUNCED REST snapshot (pages/info) DIFFERS from the LIVE collab
// doc (mutatePage) — the doc moved on in the debounce window. The stored
// selection is captured from the snapshot at create time, but the mark is set
// in the live doc, so apply would 409 forever. After anchoring, the client
// re-reads the RAW substring under the mark from the LIVE doc and POSTs it to
// /comments/resync-suggestion-anchor so the stored expectedText matches.
// -----------------------------------------------------------------------------
test("suggestion: re-syncs the stored selection to the LIVE doc substring when the REST snapshot lagged", async () => {
let createPayload = null;
let resyncPayload = null;
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (req.url === "/api/auth/login") {
sendJson(res, 200, { success: true }, {
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
});
return;
}
if (req.url === "/api/pages/info") {
// DEBOUNCED snapshot: ASCII quotes (stale — the live doc has since been
// typographically corrected).
sendJson(res, 200, {
data: {
id: "33333333-3333-3333-3333-333333333333",
content: {
type: "doc",
content: [
{
type: "paragraph",
content: [{ type: "text", text: 'he said "hello" loudly' }],
},
],
},
},
});
return;
}
if (req.url === "/api/comments/create") {
createPayload = JSON.parse(raw);
sendJson(res, 200, {
data: {
id: "cmt-resync-1",
content: createPayload.content,
selection: createPayload.selection,
suggestedText: createPayload.suggestedText,
type: createPayload.type,
},
});
return;
}
if (req.url === "/api/comments/resync-suggestion-anchor") {
resyncPayload = JSON.parse(raw);
sendJson(res, 200, { data: { id: "cmt-resync-1" } });
return;
}
sendJson(res, 404, { message: "not found" });
});
class TestClient extends DocmostClient {
async getCollabTokenWithReauth() {
return "collab-token";
}
async resolvePageId() {
return "33333333-3333-3333-3333-333333333333";
}
async mutatePage(pageId, collabToken, apiUrl, transform) {
// LIVE doc: SMART quotes (what the mark is actually set over).
const doc = {
type: "doc",
content: [
{
type: "paragraph",
content: [{ type: "text", text: "he said “hello” loudly" }],
},
],
};
const out = transform(doc);
return { doc: out, verify: { ok: true } };
}
}
const client = new TestClient(baseURL, "user@example.com", "pw");
const result = await client.createComment(
"33333333-3333-3333-3333-333333333333",
"please change",
"inline",
'"hello"', // ASCII quotes
undefined,
"goodbye",
);
assert.equal(result.success, true);
assert.equal(result.anchored, true);
// Create stored the STALE snapshot substring (ASCII quotes).
assert.equal(createPayload.selection, '"hello"');
// …then the client re-synced to the LIVE marked substring (smart quotes).
assert.ok(resyncPayload, "/comments/resync-suggestion-anchor must be called");
assert.equal(resyncPayload.commentId, "cmt-resync-1");
assert.equal(resyncPayload.selection, "“hello”");
// The returned comment reflects the corrected anchor.
assert.equal(result.data.selection, "“hello”");
});
// -----------------------------------------------------------------------------
// 8c) #496: when the REST snapshot and the LIVE doc AGREE, no resync round-trip
// is made (the common case must stay a single write).
// -----------------------------------------------------------------------------
test("suggestion: no resync call when the snapshot already matches the live doc", async () => {
let resyncCalls = 0;
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (req.url === "/api/auth/login") {
sendJson(res, 200, { success: true }, {
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
});
return;
}
if (req.url === "/api/pages/info") {
sendJson(res, 200, {
data: {
id: "44444444-4444-4444-4444-444444444444",
content: {
type: "doc",
content: [
{ type: "paragraph", content: [{ type: "text", text: "Hello brave world" }] },
],
},
},
});
return;
}
if (req.url === "/api/comments/create") {
const p = JSON.parse(raw);
sendJson(res, 200, {
data: { id: "cmt-nosync-1", content: p.content, selection: p.selection, suggestedText: p.suggestedText, type: p.type },
});
return;
}
if (req.url === "/api/comments/resync-suggestion-anchor") {
resyncCalls++;
sendJson(res, 200, { data: {} });
return;
}
sendJson(res, 404, { message: "not found" });
});
class TestClient extends DocmostClient {
async getCollabTokenWithReauth() {
return "collab-token";
}
async resolvePageId() {
return "44444444-4444-4444-4444-444444444444";
}
async mutatePage(pageId, collabToken, apiUrl, transform) {
const doc = {
type: "doc",
content: [
{ type: "paragraph", content: [{ type: "text", text: "Hello brave world" }] },
],
};
const out = transform(doc);
return { doc: out, verify: { ok: true } };
}
}
const client = new TestClient(baseURL, "user@example.com", "pw");
const result = await client.createComment(
"44444444-4444-4444-4444-444444444444",
"rename",
"inline",
"brave",
undefined,
"bold",
);
assert.equal(result.anchored, true);
assert.equal(resyncCalls, 0, "matching snapshot must NOT trigger a resync round-trip");
});
// -----------------------------------------------------------------------------
// 8) #408: a not-found selection error QUOTES the closest block text so the
// model can self-correct instead of blind-retrying.