Compare commits

..

3 Commits

Author SHA1 Message Date
agent_coder 79b2da686b fix(work-time): clip cross-class padding; gate & payload fixes (#395)
F1: pad-clip adjacent different-class sessions at the raw-gap midpoint so
workMs and agentOnlyMs can never double-count the same wall-clock (default
agentTGap 7m < pIn+pOut 10m made a work-session-ending-in-agent overlap a
nearby agent_only run). Reword the config guard/comment: cross-class
disjointness is now structural, tGap≥pIn+pOut is a kept sanity bound. Add a
cross-class no-double-count test and a real seeded property/fuzz test (250
random timelines × 4 tz) asserting per-class union, cross-class disjointness,
and Σ per-day activeMs == workMs.

F2: page.controller.spec — add a ForbiddenException view-gate reject test that
asserts the rejection propagates and computeWorkTime is NOT reached, locking
validateCanView before compute.

F3: WorkTimeStat renders for agent-only pages (workMs==0, agentOnlyMs>0) with
an `agent:` headline so the punch-card stays reachable.

F4: drop the dead un-bucketed `sessions` list from the PageWorkTime response
and the client work-time types (no client reads it; bucketByDay still consumes
the core sessions internally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 08:31:42 +03:00
agent_coder 3e81f64415 fix(work-time): user idle breaks agent burst; cap tz length (#395)
Only an agent-sourced idle pulse extends an agent burst; a user idle
(human supervision) now falls through to the human branch so the session
is classified `work`, not `agent_only`. Add @MaxLength(64) to tz to match
its comment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 06:53:59 +03:00
agent_coder c674db2b2f feat(page-history): «время работы над статьёй» — число в UI + суточный punch-card (#395)
Оценка времени работы над страницей как сессионизация истории page_history по
паузам бездействия (WakaTime-подобно), а не span между крайними правками, плюс
drill-down в суточный таймлайн 24ч×дни. Зависит от #374 (kind + idle-пульс).

Server:
- Чистая computeWorkTime(rows, config) (§5): нормализация+дедуп → коллапс
  агентских всплесков по aiChatId → ОДИН проход сессионизации (порог зависит от
  пары: оба agent → agentTGap, иначе tGap; последняя сессия обязательно
  закрывается) → класс готовой сессии (все agent → agent_only, иначе work) →
  добивка (много-сэмпл → [first−P_in, last+P_out], одиночный скаляр →
  [t−P_single, t]) → метрики = union wall-clock по классу. Детерминированная,
  без БД.
- Чистая bucketByDay(sessions, tz) (§6.3): union work/agent_only РАЗДЕЛЬНО, разрез
  по полуночи tz через нативный Intl (DST-корректно, без «+24ч»). Инвариант
  Σ activeMs == workMs держится по построению.
- PageHistoryRepo.findTimelineByPageId — лёгкая проекция без content, ASC.
- PageHistoryService.computeWorkTime + POST /pages/history/time, гейт
  validateCanView как у /history; атрибуция человек/агент по lastUpdatedSource.

Client:
- usePageWorkTime(pageId) шлёт tz зрителя (Intl локаль).
- Кликабельное число «≈ 4 ч 30 мин» в шапке страницы (порог в тултипе).
- Модалка-punch-card: 24-часовые дорожки по дням, окна work/agent разным цветом,
  одиночные (P_single) приглушены, сумма за день, пустой день «—», сворачивание
  длинных серий пустых дней, подпись tz + T_gap. i18n ru-RU + en-US.

Tests: 22 юнита на computeWorkTime/bucketByDay (§7-фикстура ≈1ч32м vs ≈60h наив,
закрытие последней сессии, коллапс/разрыв всплеска, idle-пульс, DST 23/25ч,
полуночный разрез, инвариант, union-без-задвоения); 5 юнитов гейта контроллера;
int-spec на реальном pg (проекция + совпадение с ядром); 6 клиентских на формат.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 06:32:12 +03:00
46 changed files with 2069 additions and 1267 deletions
@@ -1427,5 +1427,20 @@
"Boundary": "Boundary",
"Autosave": "Autosave",
"Only versions": "Only versions",
"No saved versions yet.": "No saved versions yet."
"No saved versions yet.": "No saved versions yet.",
"Time worked on this article": "Time worked on this article",
"Show time worked on this page": "Show time worked on this page",
"Estimated time worked (inactivity gap {{gap}} min)": "Estimated time worked (inactivity gap {{gap}} min)",
"Estimate · timezone {{tz}} · inactivity gap {{gap}} min": "Estimate · timezone {{tz}} · inactivity gap {{gap}} min",
"No editing activity recorded yet.": "No editing activity recorded yet.",
"× {{count}} days without edits": "× {{count}} days without edits",
"agent: {{value}}": "agent: {{value}}",
"Work": "Work",
"Agent": "Agent",
"≈ {{hours}}h {{minutes}}m": "≈ {{hours}}h {{minutes}}m",
"≈ {{hours}}h": "≈ {{hours}}h",
"≈ {{minutes}}m": "≈ {{minutes}}m",
"{{hours}}h {{minutes}}m": "{{hours}}h {{minutes}}m",
"{{hours}}h": "{{hours}}h",
"{{minutes}}m": "{{minutes}}m"
}
@@ -1442,5 +1442,20 @@
"Boundary": "Граница",
"Autosave": "Автосейв",
"Only versions": "Только версии",
"No saved versions yet.": "Пока нет сохранённых версий."
"No saved versions yet.": "Пока нет сохранённых версий.",
"Time worked on this article": "Время работы над статьёй",
"Show time worked on this page": "Показать время работы над страницей",
"Estimated time worked (inactivity gap {{gap}} min)": "Оценка времени работы (порог паузы {{gap}} мин)",
"Estimate · timezone {{tz}} · inactivity gap {{gap}} min": "Оценка · таймзона {{tz}} · порог паузы {{gap}} мин",
"No editing activity recorded yet.": "Правок пока нет.",
"× {{count}} days without edits": "× {{count}} дн. без правок",
"agent: {{value}}": "агент: {{value}}",
"Work": "Работа",
"Agent": "Агент",
"≈ {{hours}}h {{minutes}}m": "≈ {{hours}} ч {{minutes}} мин",
"≈ {{hours}}h": "≈ {{hours}} ч",
"≈ {{minutes}}m": "≈ {{minutes}} мин",
"{{hours}}h {{minutes}}m": "{{hours}} ч {{minutes}} м",
"{{hours}}h": "{{hours}} ч",
"{{minutes}}m": "{{minutes}} м"
}
@@ -0,0 +1,45 @@
import { describe, it, expect } from "vitest";
import {
formatHeadline,
formatDayTotal,
formatGapMinutes,
} from "./format-work-time";
const MIN = 60 * 1000;
// Fake translator: renders the key with {{tokens}} substituted, so the tests
// assert the rounding + branch selection without depending on the i18n catalogue.
const t = (key: string, opts?: Record<string, unknown>) =>
key.replace(/\{\{(\w+)\}\}/g, (_, k) => String(opts?.[k] ?? ""));
describe("formatHeadline", () => {
it("prefixes ≈ and rounds to a 5-minute step", () => {
expect(formatHeadline(4 * 60 * MIN + 27 * MIN, t)).toBe("≈ 4h 25m");
expect(formatHeadline(90 * MIN, t)).toBe("≈ 1h 30m");
});
it("shows hours only / minutes only cleanly", () => {
expect(formatHeadline(120 * MIN, t)).toBe("≈ 2h");
expect(formatHeadline(35 * MIN, t)).toBe("≈ 35m");
});
it("floors a tiny non-zero estimate to 5m, never 0", () => {
expect(formatHeadline(2 * MIN, t)).toBe("≈ 5m");
});
it("empty string for zero (widget hidden)", () => {
expect(formatHeadline(0, t)).toBe("");
});
});
describe("formatDayTotal", () => {
it('renders "h m" and shows — for empty days', () => {
expect(formatDayTotal(3 * 60 * MIN + 17 * MIN, t)).toBe("3h 17m");
expect(formatDayTotal(0, t)).toBe("—");
});
});
describe("formatGapMinutes", () => {
it("converts the tGap ms threshold to whole minutes", () => {
expect(formatGapMinutes(15 * MIN)).toBe(15);
});
});
@@ -0,0 +1,45 @@
// #395 — display formatting for the work-time estimate. Pure functions that take
// a translator so ru-RU / en-US wording lives in the i18n catalogue and the
// rounding logic stays unit-testable.
type Translate = (key: string, opts?: Record<string, unknown>) => string;
const MIN = 60 * 1000;
function hm(totalMinutes: number): { hours: number; minutes: number } {
return {
hours: Math.floor(totalMinutes / 60),
minutes: totalMinutes % 60,
};
}
/**
* Headline number (§6.1): an ESTIMATE, so rounded to a coarse 5-minute step and
* prefixed with "≈". A non-zero-but-tiny estimate floors to 5m rather than
* rounding down to "0" (which would read as "no work"). Zero → empty string
* (the caller hides the widget).
*/
export function formatHeadline(workMs: number, t: Translate): string {
if (workMs <= 0) return "";
let minutes = Math.round(workMs / MIN / 5) * 5;
if (minutes === 0) minutes = 5;
const { hours, minutes: m } = hm(minutes);
if (hours > 0 && m > 0) return t("≈ {{hours}}h {{minutes}}m", { hours, minutes: m });
if (hours > 0) return t("≈ {{hours}}h", { hours });
return t("≈ {{minutes}}m", { minutes: m });
}
/** Per-day sum (§6.2), rounded to the minute. Zero → "—". */
export function formatDayTotal(activeMs: number, t: Translate): string {
if (activeMs <= 0) return "—";
const minutes = Math.max(1, Math.round(activeMs / MIN));
const { hours, minutes: m } = hm(minutes);
if (hours > 0 && m > 0) return t("{{hours}}h {{minutes}}m", { hours, minutes: m });
if (hours > 0) return t("{{hours}}h", { hours });
return t("{{minutes}}m", { minutes: m });
}
/** The inactivity threshold, for the "estimate · gap = N min" caption. */
export function formatGapMinutes(tGapMs: number): number {
return Math.round(tGapMs / MIN);
}
@@ -0,0 +1,25 @@
import { useQuery, UseQueryResult } from "@tanstack/react-query";
import { IPageWorkTime } from "./work-time.types";
import { getPageWorkTime, viewerTimezone } from "./work-time-service";
const WORK_TIME_STALE_TIME = 5 * 60 * 1000;
/**
* #395 — the "time worked on this article" estimate + per-day punch-card
* buckets. The buckets are computed server-side in the viewer's timezone (so a
* midnight-crossing session lands on the right calendar day for the reader).
* `enabled` is opt-in so the (cheap but non-trivial) projection query only fires
* when the number is actually shown.
*/
export function usePageWorkTime(
pageId: string,
enabled = true,
): UseQueryResult<IPageWorkTime, Error> {
const tz = viewerTimezone();
return useQuery({
queryKey: ["page-work-time", pageId, tz],
queryFn: () => getPageWorkTime(pageId, tz),
enabled: enabled && !!pageId,
staleTime: WORK_TIME_STALE_TIME,
});
}
@@ -0,0 +1,171 @@
import { Group, Stack, Text } from "@mantine/core";
import { useTranslation } from "react-i18next";
import { useMemo } from "react";
import { IPageWorkTime, IPerDay, IDayWindow } from "./work-time.types";
import {
formatDayTotal,
formatGapMinutes,
formatHeadline,
} from "./format-work-time";
import classes from "./work-time.module.css";
const DAY_MS = 24 * 60 * 60 * 1000;
// Collapse a run of this many (or more) consecutive edit-free days into a single
// "× N days" separator (§6.2 long-range) — the row is still always one day.
const EMPTY_RUN_COLLAPSE = 8;
type Row =
| { type: "day"; day: IPerDay }
| { type: "gap"; count: number };
function collapseEmptyRuns(perDay: IPerDay[]): Row[] {
const rows: Row[] = [];
let emptyRun: IPerDay[] = [];
const flush = () => {
if (emptyRun.length >= EMPTY_RUN_COLLAPSE) {
rows.push({ type: "gap", count: emptyRun.length });
} else {
for (const d of emptyRun) rows.push({ type: "day", day: d });
}
emptyRun = [];
};
for (const d of perDay) {
if (d.activeMs === 0 && d.agentMs === 0) {
emptyRun.push(d);
} else {
flush();
rows.push({ type: "day", day: d });
}
}
flush();
return rows;
}
function dayHeading(day: number): string {
return new Date(day).toLocaleDateString(undefined, {
weekday: "short",
day: "numeric",
month: "short",
});
}
function DayTrack({
day,
pSingle,
}: {
day: IPerDay;
pSingle: number;
}) {
const { t } = useTranslation();
const ticks = [6, 12, 18];
return (
<div className={classes.row}>
<span className={classes.dayLabel}>{dayHeading(day.day)}</span>
<div className={classes.track}>
{ticks.map((h) => (
<div
key={h}
className={classes.hourTick}
style={{ left: `${(h / 24) * 100}%` }}
/>
))}
{day.windows.map((w: IDayWindow, i) => {
const leftPct = ((w.start - day.day) / DAY_MS) * 100;
const widthPct = ((w.end - w.start) / DAY_MS) * 100;
const isSingle = w.end - w.start <= pSingle;
const cls = [
classes.window,
w.class === "work" ? classes.windowWork : classes.windowAgent,
isSingle ? classes.windowSingle : "",
].join(" ");
return (
<div
key={i}
className={cls}
style={{
left: `${Math.max(0, Math.min(100, leftPct))}%`,
width: `${Math.max(0, Math.min(100, widthPct))}%`,
}}
/>
);
})}
</div>
<span className={classes.daySum}>
{formatDayTotal(day.activeMs, t)}
</span>
</div>
);
}
interface Props {
data: IPageWorkTime;
}
export default function WorkTimePunchCard({ data }: Props) {
const { t } = useTranslation();
const rows = useMemo(() => collapseEmptyRuns(data.perDay), [data.perDay]);
const gapMin = formatGapMinutes(data.config.tGap);
if (data.workMs <= 0 && data.agentOnlyMs <= 0) {
return (
<Text size="sm" c="dimmed" py="md">
{t("No editing activity recorded yet.")}
</Text>
);
}
return (
<Stack gap="xs">
<Group gap="lg">
<Text size="sm" fw={500}>
{formatHeadline(data.workMs, t)}
</Text>
{data.agentOnlyMs > 0 && (
<Text size="xs" c="dimmed">
{t("agent: {{value}}", { value: formatHeadline(data.agentOnlyMs, t) })}
</Text>
)}
</Group>
<Group gap="md">
<Text size="xs" c="dimmed">
<span
className={`${classes.legendSwatch} ${classes.windowWork}`}
style={{ marginRight: 4 }}
/>
{t("Work")}
</Text>
<Text size="xs" c="dimmed">
<span
className={`${classes.legendSwatch} ${classes.windowAgent}`}
style={{ marginRight: 4 }}
/>
{t("Agent")}
</Text>
</Group>
<div>
{rows.map((row, i) =>
row.type === "day" ? (
<DayTrack
key={row.day.dayISO}
day={row.day}
pSingle={data.config.pSingle}
/>
) : (
<div key={`gap-${i}`} className={classes.gapRow}>
{t("× {{count}} days without edits", { count: row.count })}
</div>
),
)}
</div>
<Text size="xs" c="dimmed" mt="xs">
{t("Estimate · timezone {{tz}} · inactivity gap {{gap}} min", {
tz: data.tz,
gap: gapMin,
})}
</Text>
</Stack>
);
}
@@ -0,0 +1,23 @@
import api from "@/lib/api-client";
import { IPageWorkTime } from "./work-time.types";
/** The viewer's IANA timezone (browser locale) — the punch-card lays days out
* in "my evenings", per §6.3/§10. Falls back to UTC if the runtime hides it. */
export function viewerTimezone(): string {
try {
return Intl.DateTimeFormat().resolvedOptions().timeZone || "UTC";
} catch {
return "UTC";
}
}
export async function getPageWorkTime(
pageId: string,
tz: string,
): Promise<IPageWorkTime> {
const req = await api.post<IPageWorkTime>("/pages/history/time", {
pageId,
tz,
});
return req.data;
}
@@ -0,0 +1,69 @@
import { Modal, Text, Tooltip, UnstyledButton } from "@mantine/core";
import { useDisclosure } from "@mantine/hooks";
import { IconClockHour4 } from "@tabler/icons-react";
import { useTranslation } from "react-i18next";
import { usePageWorkTime } from "./use-page-work-time";
import { formatGapMinutes, formatHeadline } from "./format-work-time";
import WorkTimePunchCard from "./work-time-punch-card";
interface Props {
pageId: string;
}
/**
* #395 — the clickable "time worked on this article" headline (§6.1). Renders
* the `work` estimate with a "≈" sign and the inactivity threshold in a tooltip
* (it is an estimate, not a stopwatch). Clicking opens the daily punch-card
* (§6.2). Renders nothing until there is a non-zero human OR agent estimate, so a
* brand-new / never-edited page shows no widget. For an agent-only-edited page
* (workMs===0, agentOnlyMs>0) the headline shows the agent estimate (labelled
* `agent:`, matching the punch-card) so the punch-card stays reachable (#395:
* "how much a HUMAN and separately the AGENT").
*/
export default function WorkTimeStat({ pageId }: Props) {
const { t } = useTranslation();
const [opened, { open, close }] = useDisclosure(false);
const { data } = usePageWorkTime(pageId);
if (!data || (data.workMs <= 0 && data.agentOnlyMs <= 0)) return null;
const agentOnly = data.workMs <= 0;
const label = agentOnly
? t("agent: {{value}}", { value: formatHeadline(data.agentOnlyMs, t) })
: formatHeadline(data.workMs, t);
const gapMin = formatGapMinutes(data.config.tGap);
return (
<>
<Tooltip
label={t("Estimated time worked (inactivity gap {{gap}} min)", {
gap: gapMin,
})}
position="bottom"
>
<UnstyledButton
onClick={open}
aria-label={t("Show time worked on this page")}
>
<Text
size="xs"
c="dimmed"
style={{ display: "inline-flex", alignItems: "center", gap: 4 }}
>
<IconClockHour4 size={14} />
{label}
</Text>
</UnstyledButton>
</Tooltip>
<Modal
opened={opened}
onClose={close}
title={t("Time worked on this article")}
size="lg"
>
<WorkTimePunchCard data={data} />
</Modal>
</>
);
}
@@ -0,0 +1,82 @@
/* #395 — 24h × days punch-card. Custom CSS segments on a fixed 24-hour track
(position = offset-in-day / 24h, width = duration / 24h), no chart library. */
.row {
display: grid;
grid-template-columns: 96px 1fr 64px;
align-items: center;
gap: 12px;
padding: 3px 0;
}
.dayLabel {
font-size: var(--mantine-font-size-xs);
color: var(--mantine-color-dimmed);
white-space: nowrap;
}
.track {
position: relative;
height: 16px;
border-radius: 4px;
background-color: light-dark(
var(--mantine-color-gray-1),
var(--mantine-color-dark-6)
);
overflow: hidden;
}
/* Faint hour grid so the eye can read "morning vs evening". */
.hourTick {
position: absolute;
top: 0;
bottom: 0;
width: 1px;
background-color: light-dark(
var(--mantine-color-gray-3),
var(--mantine-color-dark-4)
);
}
.window {
position: absolute;
top: 2px;
bottom: 2px;
border-radius: 3px;
min-width: 3px;
}
.windowWork {
background-color: var(--mantine-color-blue-5);
}
.windowAgent {
background-color: var(--mantine-color-grape-5);
}
/* A lone single-sample (P_single) window: minimal + dimmed, so it neither
vanishes nor fakes dense work (§6.2). */
.windowSingle {
opacity: 0.5;
}
.daySum {
font-size: var(--mantine-font-size-xs);
text-align: right;
white-space: nowrap;
}
.gapRow {
padding: 6px 0 6px 108px;
font-size: var(--mantine-font-size-xs);
color: var(--mantine-color-dimmed);
font-style: italic;
}
.legendSwatch {
display: inline-block;
width: 12px;
height: 12px;
border-radius: 3px;
vertical-align: middle;
}
@@ -0,0 +1,37 @@
// #395 — client-side mirror of the server work-time payload
// (apps/server/src/core/page/work-time). Shapes returned by POST /pages/history/time.
export type WorkSessionClass = "work" | "agent_only";
export interface IDayWindow {
start: number;
end: number;
class: WorkSessionClass;
}
export interface IPerDay {
day: number;
dayISO: string;
activeMs: number;
agentMs: number;
windows: IDayWindow[];
}
export interface IWorkTimeConfig {
tGap: number;
agentTGap: number;
pIn: number;
pOut: number;
pSingle: number;
excludeGit: boolean;
burstCapMs?: number;
dedupRoundMs: number;
}
export interface IPageWorkTime {
workMs: number;
agentOnlyMs: number;
perDay: IPerDay[];
config: IWorkTimeConfig;
tz: string;
}
@@ -51,6 +51,7 @@ import {
import { formattedDate } from "@/lib/time.ts";
import { PageEditModeToggle } from "@/features/user/components/page-state-pref.tsx";
import MovePageModal from "@/features/page/components/move-page-modal.tsx";
import WorkTimeStat from "@/features/page-history/work-time/work-time-stat.tsx";
import { useTimeAgo } from "@/hooks/use-time-ago.tsx";
import {
useFavoriteIds,
@@ -265,6 +266,8 @@ function PageActionMenu({ readOnly, onSaveVersion }: PageActionMenuProps) {
return (
<>
{page?.id && <WorkTimeStat pageId={page.id} />}
<Menu
shadow="xl"
position="bottom-end"
@@ -141,57 +141,7 @@ export function htmlToJson(html: string) {
}
}
/**
* Deterministic text-serializer overrides for the `format:"text"` page read
* (#502). Non-text nodes render to a STABLE placeholder instead of their
* (structure-dependent) inner text, so a machine diff of two text reads is
* driven only by the page's actual prose — output stability across package
* versions IS the contract (pinned by a snapshot test). Returning a string from
* a `textSerializer` also stops `generateText` descending into the node, so a
* table renders as ONE token rather than its flattened cell text.
*
* Only nodes with no meaningful flat-text form are overridden; every other node
* (paragraph/heading/list/code/blockquote/callout/…) keeps its natural text so
* a config written as markdown reads back byte-identical.
*/
const TEXT_READ_SERIALIZERS: Record<string, (props: { node: any }) => string> =
{
// Image atom: no inner text -> a fixed placeholder.
image: () => '[image]',
// Table: `[table RxC]` where R = row count, C = the first row's cell count
// (a table's columns are uniform per the schema). Computed from the PM node,
// so it is independent of cell contents.
table: ({ node }) => {
const rows = node?.childCount ?? 0;
const cols = rows > 0 ? (node.child(0)?.childCount ?? 0) : 0;
return `[table ${rows}x${cols}]`;
},
};
/**
* Serialize a ProseMirror/TipTap document to plain text.
*
* Default (no options): the long-standing search-index behavior — bare
* concatenated node text with `generateText`'s default `\n\n` block separator.
* This feeds the page `textContent` tsvector and MUST NOT change.
*
* `deterministic:true` (#502 `format:"text"` page read): a flat, machine-diffable
* rendering — one line per block (`\n` block separator; `hardBreak` already
* serializes to `\n`), inline marks/anchors/autoformat dropped, and non-text
* nodes replaced by the stable placeholders above (`[image]`, `[table RxC]`).
*/
export function jsonToText(
// `any` (like jsonToHtml/jsonToMarkdown) so a loosely-typed DB `page.content`
// (JsonValue) can be passed straight through, as the controller does.
tiptapJson: any,
options?: { deterministic?: boolean },
) {
if (options?.deterministic) {
return generateText(tiptapJson, tiptapExtensions, {
blockSeparator: '\n',
textSerializers: TEXT_READ_SERIALIZERS,
});
}
export function jsonToText(tiptapJson: JSONContent) {
return generateText(tiptapJson, tiptapExtensions);
}
@@ -1,116 +0,0 @@
import { jsonToText } from './collaboration.util';
// #502 READ contract: `jsonToText(json, { deterministic: true })` — the flat,
// machine-diffable text rendering behind getPage `format:"text"`. Block-per-line
// (`\n` separator), inline marks/anchors dropped, hardBreak -> `\n`, and non-text
// nodes replaced by STABLE placeholders (`[image]`, `[table RxC]`). Output
// stability across package versions IS the contract, so it is pinned by a
// snapshot below. The DEFAULT (no options) path is the search-index serializer
// and MUST be unchanged — asserted separately.
const doc = (...content: any[]) => ({ type: 'doc', content });
const para = (...content: any[]) => ({ type: 'paragraph', content });
const text = (t: string, marks?: any[]) =>
marks ? { type: 'text', text: t, marks } : { type: 'text', text: t };
describe('jsonToText — default (search index) behavior is unchanged', () => {
it('uses the `\\n\\n` block separator and drops non-text nodes to empty', () => {
const d = doc(para(text('alpha')), para(text('beta')));
expect(jsonToText(d)).toBe('alpha\n\nbeta');
});
it('an image contributes no text in the default (tsvector) mode', () => {
const d = doc(para(text('cap')), { type: 'image', attrs: { src: 's' } });
// No `[image]` placeholder leaks into the search index.
expect(jsonToText(d)).not.toContain('[image]');
});
});
describe('jsonToText — deterministic:true (getPage format:"text")', () => {
it('renders one line per block with `\\n` separators, marks dropped', () => {
const d = doc(
{ type: 'heading', attrs: { level: 2 }, content: [text('Title')] },
para(
text('hello ', [{ type: 'bold' }]),
text('world', [{ type: 'italic' }]),
),
);
expect(jsonToText(d, { deterministic: true })).toBe('Title\nhello world');
});
it('a hardBreak renders as a newline', () => {
const d = doc(para(text('a'), { type: 'hardBreak' }, text('b')));
expect(jsonToText(d, { deterministic: true })).toBe('a\nb');
});
it('an image node -> the stable `[image]` placeholder', () => {
const d = doc(para(text('before')), { type: 'image', attrs: { src: 's' } });
expect(jsonToText(d, { deterministic: true })).toBe('before\n[image]');
});
it('a table -> `[table RxC]` (rows x columns) and its cell text is NOT flattened in', () => {
const cell = (t: string) => ({
type: 'tableCell',
content: [para(text(t))],
});
const row = (...cells: any[]) => ({ type: 'tableRow', content: cells });
const table = {
type: 'table',
content: [
row(cell('CELLONE'), cell('CELLTWO'), cell('CELLTHREE')),
row(cell('CELLFOUR'), cell('CELLFIVE'), cell('CELLSIX')),
],
};
const d = doc(para(text('grid:')), table);
const out = jsonToText(d, { deterministic: true });
expect(out).toBe('grid:\n[table 2x3]');
expect(out).not.toContain('CELL'); // cell text is not flattened into the read
});
it('SNAPSHOT: a mixed document renders to a stable, deterministic string', () => {
const cell = (t: string) => ({
type: 'tableCell',
content: [para(text(t))],
});
const d = doc(
{ type: 'heading', attrs: { level: 1 }, content: [text('Config')] },
para(text('key = ', [{ type: 'bold' }]), text('value')),
{
type: 'bulletList',
content: [
{ type: 'listItem', content: [para(text('one'))] },
{ type: 'listItem', content: [para(text('two'))] },
],
},
{ type: 'image', attrs: { src: 's' } },
{
type: 'table',
content: [{ type: 'tableRow', content: [cell('x'), cell('y')] }],
},
);
expect(jsonToText(d, { deterministic: true })).toMatchInlineSnapshot(`
"Config
key = value
one
two
[image]
[table 1x2]"
`);
});
it('SCENARIO: a config written as a code block reads back byte-identical (diff empty)', () => {
const config =
'export TICKET_LIFETIME=$2592000\nservers:\n - www.internal.host';
const d = doc({
type: 'codeBlock',
attrs: { language: 'yaml' },
content: [text(config)],
});
// The code block is one block; its text (dollars, bare domain, newlines) is
// preserved verbatim, so a read-as-text of a stored config diffs empty.
expect(jsonToText(d, { deterministic: true })).toBe(config);
});
});
@@ -10,12 +10,6 @@ import { Transform } from 'class-transformer';
export type ContentFormat = 'json' | 'markdown' | 'html';
// READ-only rendering formats for `getPage` (#502). A superset of the writable
// `ContentFormat` with `text` — a flat, deterministic, machine-diffable text
// rendering — added. Kept SEPARATE from `ContentFormat` so the write path
// (createPage/updatePage `parseProsemirrorContent`) can never be handed `text`.
export type PageReadFormat = ContentFormat | 'text';
export class CreatePageDto {
@IsOptional()
@IsString()
+15 -3
View File
@@ -5,10 +5,11 @@ import {
IsOptional,
IsString,
IsUUID,
MaxLength,
} from 'class-validator';
import { Transform } from 'class-transformer';
import { PageReadFormat } from './create-page.dto';
import { ContentFormat } from './create-page.dto';
import { IsPageIdOrSlugId } from './page-identity.validator';
export class PageIdDto {
@@ -43,8 +44,19 @@ export class PageInfoDto extends PageIdDto {
@IsOptional()
@Transform(({ value }) => value?.toLowerCase())
@IsIn(['json', 'markdown', 'html', 'text'])
format?: PageReadFormat;
@IsIn(['json', 'markdown', 'html'])
format?: ContentFormat;
}
export class PageWorkTimeDto extends PageIdDto {
// Viewer IANA timezone for the per-day punch-card buckets (§6.3). Optional —
// falls back to UTC server-side. Length-capped so a bogus value cannot bloat
// the request; the value is only ever handed to Intl.DateTimeFormat, which
// throws on an unknown zone (caught by the controller → 400).
@IsOptional()
@IsString()
@MaxLength(64)
tz?: string;
}
export class DeletePageDto extends PageIdDto {
@@ -1,76 +0,0 @@
import { NotFoundException } from '@nestjs/common';
import { PageController } from './page.controller';
import { jsonToText, jsonToMarkdown } from '../../collaboration/collaboration.util';
// #502 READ: getPage `format:"text"` routes through the deterministic jsonToText
// path (placeholders for non-text nodes, block-per-line), while `format:"json"`
// (or none) returns the raw content and `format:"markdown"` still converts. This
// pins the CONTROLLER wiring with lightweight mocks (no DB needed — the jsonToText
// output contract itself is pinned in json-to-text-deterministic.spec.ts).
const CONTENT = {
type: 'doc',
content: [
{ type: 'heading', attrs: { level: 1 }, content: [{ type: 'text', text: 'Config' }] },
{
type: 'paragraph',
content: [
{ type: 'text', text: 'key=', marks: [{ type: 'bold' }] },
{ type: 'text', text: 'value' },
],
},
{ type: 'image', attrs: { src: 's' } },
],
};
function makeController(page: any): PageController {
const pageRepo = { findById: jest.fn().mockResolvedValue(page) } as any;
const pageAccessService = {
validateCanViewWithPermissions: jest
.fn()
.mockResolvedValue({ canEdit: true, hasRestriction: false }),
} as any;
// Only pageRepo + pageAccessService are exercised by getPage; the rest are
// never touched on this path, so undefined placeholders are fine.
return new PageController(
undefined as any, // pageService
pageRepo,
undefined as any, // pageHistoryService
undefined as any, // spaceAbility
pageAccessService,
undefined as any, // backlinkService
undefined as any, // labelService
undefined as any, // auditService
);
}
const user = { id: 'u1' } as any;
describe('PageController.getPage — format:"text" (#502)', () => {
it('returns deterministic flat text with placeholders for non-text nodes', async () => {
const controller = makeController({ id: 'p1', content: CONTENT });
const res: any = await controller.getPage({ pageId: 'p1', format: 'text' } as any, user);
expect(res.content).toBe(jsonToText(CONTENT, { deterministic: true }));
expect(res.content).toBe('Config\nkey=value\n[image]');
expect(res.permissions).toEqual({ canEdit: true, hasRestriction: false });
});
it('markdown format still converts to markdown (unchanged)', async () => {
const controller = makeController({ id: 'p1', content: CONTENT });
const res: any = await controller.getPage({ pageId: 'p1', format: 'markdown' } as any, user);
expect(res.content).toBe(jsonToMarkdown(CONTENT));
});
it('json / no format returns the raw ProseMirror content object', async () => {
const controller = makeController({ id: 'p1', content: CONTENT });
const res: any = await controller.getPage({ pageId: 'p1' } as any, user);
expect(res.content).toEqual(CONTENT); // untouched object, not a string
});
it('missing page -> NotFoundException', async () => {
const controller = makeController(null);
await expect(
controller.getPage({ pageId: 'nope', format: 'text' } as any, user),
).rejects.toBeInstanceOf(NotFoundException);
});
});
@@ -1,3 +1,8 @@
import {
BadRequestException,
ForbiddenException,
NotFoundException,
} from '@nestjs/common';
import { PageController } from './page.controller';
// Direct instantiation with stub deps. The Test.createTestingModule form failed
@@ -22,4 +27,88 @@ describe('PageController', () => {
it('should be defined', () => {
expect(controller).toBeDefined();
});
// #395 — the work-time endpoint must be gated exactly like /history.
describe('getPageWorkTime', () => {
const user = { id: 'u1' } as any;
function build(overrides: {
page?: any;
validate?: jest.Mock;
compute?: jest.Mock;
}) {
const pageRepo = { findById: jest.fn().mockResolvedValue(overrides.page) };
const pageAccessService = {
validateCanView: overrides.validate ?? jest.fn().mockResolvedValue(undefined),
};
const pageHistoryService = {
computeWorkTime:
overrides.compute ?? jest.fn().mockResolvedValue({ workMs: 0 }),
};
const c = new PageController(
{} as any,
pageRepo as any,
pageHistoryService as any,
{} as any,
pageAccessService as any,
{} as any,
{} as any,
{} as any,
);
return { c, pageRepo, pageAccessService, pageHistoryService };
}
it('404s when the page does not exist', async () => {
const { c } = build({ page: null });
await expect(
c.getPageWorkTime({ pageId: 'p1' } as any, user),
).rejects.toBeInstanceOf(NotFoundException);
});
it('enforces validateCanView before computing, then delegates with tz', async () => {
const validate = jest.fn().mockResolvedValue(undefined);
const compute = jest.fn().mockResolvedValue({ workMs: 42 });
const { c } = build({ page: { id: 'pg' }, validate, compute });
const out = await c.getPageWorkTime(
{ pageId: 'pg', tz: 'Europe/Moscow' } as any,
user,
);
expect(validate).toHaveBeenCalledWith({ id: 'pg' }, user);
expect(compute).toHaveBeenCalledWith('pg', 'Europe/Moscow');
expect(out).toEqual({ workMs: 42 });
});
it('propagates a denied view gate and does NOT reach compute (security)', async () => {
// If validateCanView is moved AFTER computeWorkTime, the timeline of a page
// the caller may not see would be read/estimated before the gate — this
// locks the order: a rejecting gate must short-circuit before any compute.
const validate = jest.fn().mockRejectedValue(new ForbiddenException());
const compute = jest.fn().mockResolvedValue({ workMs: 1 });
const { c, pageHistoryService } = build({
page: { id: 'pg' },
validate,
compute,
});
await expect(
c.getPageWorkTime({ pageId: 'pg' } as any, user),
).rejects.toBeInstanceOf(ForbiddenException);
expect(pageHistoryService.computeWorkTime).not.toHaveBeenCalled();
});
it('maps an unknown-timezone RangeError to a 400', async () => {
const compute = jest.fn().mockRejectedValue(new RangeError('bad tz'));
const { c } = build({ page: { id: 'pg' }, compute });
await expect(
c.getPageWorkTime({ pageId: 'pg', tz: 'X/Y' } as any, user),
).rejects.toBeInstanceOf(BadRequestException);
});
it('does not swallow a non-RangeError from the service', async () => {
const compute = jest.fn().mockRejectedValue(new Error('db down'));
const { c } = build({ page: { id: 'pg' }, compute });
await expect(
c.getPageWorkTime({ pageId: 'pg' } as any, user),
).rejects.toThrow('db down');
});
});
});
+31 -11
View File
@@ -21,6 +21,7 @@ import {
PageHistoryIdDto,
PageIdDto,
PageInfoDto,
PageWorkTimeDto,
} from './dto/page.dto';
import { PageHistoryService } from './services/page-history.service';
import { AuthUser } from '../../common/decorators/auth-user.decorator';
@@ -49,7 +50,6 @@ import { AddLabelsDto, RemoveLabelDto } from '../label/dto/label.dto';
import {
jsonToHtml,
jsonToMarkdown,
jsonToText,
} from '../../collaboration/collaboration.util';
import { AuditEvent, AuditResource } from '../../common/events/audit-events';
import {
@@ -94,16 +94,10 @@ export class PageController {
const permissions = { canEdit, hasRestriction };
if (dto.format && dto.format !== 'json' && page.content) {
let contentOutput: string;
if (dto.format === 'markdown') {
contentOutput = jsonToMarkdown(page.content);
} else if (dto.format === 'text') {
// #502: flat, deterministic, machine-diffable text (block-per-line,
// inline marks/anchors dropped, non-text nodes -> stable placeholders).
contentOutput = jsonToText(page.content, { deterministic: true });
} else {
contentOutput = jsonToHtml(page.content);
}
const contentOutput =
dto.format === 'markdown'
? jsonToMarkdown(page.content)
: jsonToHtml(page.content);
return {
...page,
content: contentOutput,
@@ -531,6 +525,32 @@ export class PageController {
return this.pageHistoryService.findHistoryByPageId(page.id, pagination);
}
@HttpCode(HttpStatus.OK)
@Post('/history/time')
async getPageWorkTime(
@Body() dto: PageWorkTimeDto,
@AuthUser() user: User,
) {
const page = await this.pageRepo.findById(dto.pageId);
if (!page) {
throw new NotFoundException('Page not found');
}
// Same view gate as /history and /history/info.
await this.pageAccessService.validateCanView(page, user);
try {
return await this.pageHistoryService.computeWorkTime(page.id, dto.tz);
} catch (e) {
// Intl.DateTimeFormat throws RangeError on an unknown IANA zone; surface
// it as a 400 rather than a 500.
if (e instanceof RangeError) {
throw new BadRequestException('Invalid timezone');
}
throw e;
}
}
@HttpCode(HttpStatus.OK)
@Post('/history/info')
async getPageHistoryInfo(
@@ -3,6 +3,23 @@ import { PageHistoryRepo } from '@docmost/db/repos/page/page-history.repo';
import { PageHistory } from '@docmost/db/types/entity.types';
import { PaginationOptions } from '@docmost/db/pagination/pagination-options';
import { CursorPaginationResult } from '@docmost/db/pagination/cursor-pagination';
import {
computeWorkTime,
bucketByDay,
DEFAULT_WORK_TIME_CONFIG,
WorkTimeConfig,
PerDay,
} from '../work-time';
export interface PageWorkTime {
workMs: number;
agentOnlyMs: number;
perDay: PerDay[];
/** the config actually used, so the UI can show "≈" + the T_gap threshold. */
config: WorkTimeConfig;
/** the tz the per-day buckets were computed in (echoed back for the label). */
tz: string;
}
@Injectable()
export class PageHistoryService {
@@ -23,4 +40,33 @@ export class PageHistoryService {
paginationOptions,
);
}
/**
* #395 — estimate time worked on a page (§5) and bucket it into the viewer's
* calendar days for the punch-card (§6.3). Reads only the cheap history
* projection (no `content`); the estimate itself is a pure, deterministic
* function so it is unit-tested exhaustively without a DB.
*
* `tz` is the viewer's IANA zone (browser locale) — it moves which day a
* session lands in and where its windows sit, but never the total (§10).
*/
async computeWorkTime(
pageId: string,
tz = 'UTC',
config?: Partial<WorkTimeConfig>,
): Promise<PageWorkTime> {
const rows = await this.pageHistoryRepo.findTimelineByPageId(pageId);
const result = computeWorkTime(rows, config);
const usedConfig: WorkTimeConfig = { ...DEFAULT_WORK_TIME_CONFIG, ...config };
// `bucketByDay` consumes the pure core's un-bucketed sessions here; the
// full session list is NOT shipped on the response (no client reads it).
const perDay = bucketByDay(result.sessions, tz);
return {
workMs: result.workMs,
agentOnlyMs: result.agentOnlyMs,
perDay,
config: usedConfig,
tz,
};
}
}
@@ -0,0 +1,129 @@
import { bucketByDay, zonedDayStart } from './bucket-by-day';
import { computeWorkTime } from './compute-work-time';
import { WorkSession, TimelineSample } from './work-time.types';
const MIN = 60 * 1000;
const HOUR = 60 * MIN;
function work(start: number, end: number): WorkSession {
return { start, end, class: 'work' };
}
function agent(start: number, end: number): WorkSession {
return { start, end, class: 'agent_only' };
}
function sumActive(perDay: ReturnType<typeof bucketByDay>): number {
return perDay.reduce((a, d) => a + d.activeMs, 0);
}
describe('bucketByDay', () => {
it('Σ activeMs == workMs — the §6.3 consistency invariant', () => {
const rows: TimelineSample[] = [
{ createdAt: '2026-07-04T03:40:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
{ createdAt: '2026-07-04T03:49:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
{ createdAt: '2026-07-04T18:11:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
{ createdAt: '2026-07-06T15:34:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
];
const r = computeWorkTime(rows);
const perDay = bucketByDay(r.sessions, 'UTC');
expect(sumActive(perDay)).toBe(r.workMs);
});
it('empty input → no days', () => {
expect(bucketByDay([], 'UTC')).toEqual([]);
});
it('midnight-crossing session splits across two days, sum preserved (§9#9)', () => {
const start = Date.UTC(2026, 0, 10, 23, 14);
const end = Date.UTC(2026, 0, 11, 0, 40);
const perDay = bucketByDay([work(start, end)], 'UTC');
expect(perDay).toHaveLength(2);
expect(perDay[0].dayISO).toBe('2026-01-10');
expect(perDay[1].dayISO).toBe('2026-01-11');
expect(perDay[0].activeMs).toBe(46 * MIN); // 23:14 → 24:00
expect(perDay[1].activeMs).toBe(40 * MIN); // 00:00 → 00:40
expect(sumActive(perDay)).toBe(end - start);
});
it('empty days between active days are emitted, not skipped (§9#12)', () => {
const d1 = work(Date.UTC(2026, 0, 10, 10, 0), Date.UTC(2026, 0, 10, 11, 0));
const d3 = work(Date.UTC(2026, 0, 12, 10, 0), Date.UTC(2026, 0, 12, 11, 0));
const perDay = bucketByDay([d1, d3], 'UTC');
expect(perDay.map((d) => d.dayISO)).toEqual([
'2026-01-10',
'2026-01-11',
'2026-01-12',
]);
expect(perDay[1].activeMs).toBe(0);
expect(perDay[1].windows).toEqual([]);
});
it('agent_only windows are drawn but excluded from activeMs', () => {
const w = work(Date.UTC(2026, 0, 10, 9, 0), Date.UTC(2026, 0, 10, 10, 0));
const a = agent(Date.UTC(2026, 0, 10, 14, 0), Date.UTC(2026, 0, 10, 14, 30));
const perDay = bucketByDay([w, a], 'UTC');
expect(perDay).toHaveLength(1);
expect(perDay[0].activeMs).toBe(1 * HOUR);
expect(perDay[0].agentMs).toBe(30 * MIN);
expect(perDay[0].windows.map((x) => x.class)).toEqual(['work', 'agent_only']);
});
it('work and agent_only are unioned SEPARATELY (agent does not swallow work)', () => {
// Overlapping work + agent windows on the same day.
const w = work(Date.UTC(2026, 0, 10, 9, 0), Date.UTC(2026, 0, 10, 11, 0));
const a = agent(Date.UTC(2026, 0, 10, 10, 0), Date.UTC(2026, 0, 10, 12, 0));
const perDay = bucketByDay([w, a], 'UTC');
expect(perDay[0].activeMs).toBe(2 * HOUR);
expect(perDay[0].agentMs).toBe(2 * HOUR);
});
it('overlapping same-class sessions are UNIONed, not summed (no double-count)', () => {
// Two work sessions that overlap 10:00–10:30 on one day.
const a = work(Date.UTC(2026, 0, 10, 9, 0), Date.UTC(2026, 0, 10, 10, 30));
const b = work(Date.UTC(2026, 0, 10, 10, 0), Date.UTC(2026, 0, 10, 11, 0));
const perDay = bucketByDay([a, b], 'UTC');
expect(perDay).toHaveLength(1);
// Union 09:00–11:00 = 2h, NOT 90m + 60m = 150m.
expect(perDay[0].activeMs).toBe(2 * HOUR);
// The drawn windows are also merged to one, so the punch-card cannot render
// an overlapping double bar.
expect(perDay[0].windows).toHaveLength(1);
expect(perDay[0].windows[0].start).toBe(a.start);
expect(perDay[0].windows[0].end).toBe(b.end);
});
it('DST fall-back: a full 25-hour day still balances (§9#14)', () => {
// America/New_York ends DST 2026-11-01 (25h day).
const tz = 'America/New_York';
const dayStart = zonedDayStart(Date.UTC(2026, 10, 1, 12, 0), tz);
const nextStart = zonedDayStart(dayStart + 26 * HOUR, tz);
expect(nextStart - dayStart).toBe(25 * HOUR);
const perDay = bucketByDay([work(dayStart, nextStart)], tz);
expect(perDay).toHaveLength(1);
expect(perDay[0].dayISO).toBe('2026-11-01');
expect(perDay[0].activeMs).toBe(25 * HOUR);
expect(sumActive(perDay)).toBe(nextStart - dayStart);
});
it('DST spring-forward: a full 23-hour day still balances (§9#14)', () => {
// America/New_York starts DST 2026-03-08 (23h day).
const tz = 'America/New_York';
const dayStart = zonedDayStart(Date.UTC(2026, 2, 8, 12, 0), tz);
const nextStart = zonedDayStart(dayStart + 26 * HOUR, tz);
expect(nextStart - dayStart).toBe(23 * HOUR);
const perDay = bucketByDay([work(dayStart, nextStart)], tz);
expect(perDay).toHaveLength(1);
expect(perDay[0].activeMs).toBe(23 * HOUR);
expect(sumActive(perDay)).toBe(nextStart - dayStart);
});
it('tz changes the day a session lands in but not the total', () => {
const start = Date.UTC(2026, 0, 10, 2, 0); // 02:00 UTC
const end = Date.UTC(2026, 0, 10, 3, 0);
const utc = bucketByDay([work(start, end)], 'UTC');
const ny = bucketByDay([work(start, end)], 'America/New_York'); // 21:00 prev day
expect(utc[0].dayISO).toBe('2026-01-10');
expect(ny[0].dayISO).toBe('2026-01-09');
expect(sumActive(utc)).toBe(sumActive(ny));
});
});
@@ -0,0 +1,180 @@
import { WorkSession, PerDay, DayWindow } from './work-time.types';
/**
* Merge intervals into a disjoint, sorted union. Overlapping OR touching
* intervals are joined. Empty input → [].
*/
function union(intervals: Array<[number, number]>): Array<[number, number]> {
if (intervals.length === 0) return [];
const sorted = [...intervals].sort((a, b) => a[0] - b[0]);
const out: Array<[number, number]> = [];
let [curStart, curEnd] = sorted[0];
for (let i = 1; i < sorted.length; i++) {
const [s, e] = sorted[i];
if (s <= curEnd) {
if (e > curEnd) curEnd = e;
} else {
out.push([curStart, curEnd]);
curStart = s;
curEnd = e;
}
}
out.push([curStart, curEnd]);
return out;
}
// Cache one Intl formatter per tz — constructing them is comparatively costly.
const fmtCache = new Map<string, Intl.DateTimeFormat>();
function partsFmt(tz: string): Intl.DateTimeFormat {
let fmt = fmtCache.get(tz);
if (!fmt) {
fmt = new Intl.DateTimeFormat('en-US', {
timeZone: tz,
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
hour12: false,
});
fmtCache.set(tz, fmt);
}
return fmt;
}
interface WallParts {
year: number;
month: number;
day: number;
hour: number;
minute: number;
second: number;
}
/** Wall-clock parts of an instant in `tz` (DST-correct, via Intl). */
function wallParts(ms: number, tz: string): WallParts {
const parts = partsFmt(tz).formatToParts(new Date(ms));
const get = (type: string) =>
Number(parts.find((p) => p.type === type)?.value ?? '0');
let hour = get('hour');
// Intl emits "24" for midnight under some engines/locales; normalize to 0.
if (hour === 24) hour = 0;
return {
year: get('year'),
month: get('month'),
day: get('day'),
hour,
minute: get('minute'),
second: get('second'),
};
}
/** tz offset (wall − real) at an instant, in ms. */
function offset(ms: number, tz: string): number {
const p = wallParts(ms, tz);
const asUTC = Date.UTC(p.year, p.month - 1, p.day, p.hour, p.minute, p.second);
return asUTC - ms;
}
/**
* Epoch-ms of the local-midnight day start of `ms` in `tz`. DST-correct: takes
* the calendar day of the instant, its wall-midnight, then converts back with
* the offset that actually applies AT that midnight (refined once). The rare
* tz-with-a-DST-transition-exactly-at-midnight case is a documented edge (§9#14).
*/
export function zonedDayStart(ms: number, tz: string): number {
const p = wallParts(ms, tz);
const wallMidnightAsUTC = Date.UTC(p.year, p.month - 1, p.day, 0, 0, 0);
let start = wallMidnightAsUTC - offset(ms, tz);
// Refine with the offset at the computed midnight (DST may differ from `ms`).
start = wallMidnightAsUTC - offset(start, tz);
return start;
}
/** The next local midnight after `dayStart` (handles 23/25h DST days). */
function nextDayStart(dayStart: number, tz: string): number {
// +26h always lands inside the NEXT calendar day (day length ∈ [23h,25h]),
// never two days ahead; startOf('day') of it is the next midnight.
return zonedDayStart(dayStart + 26 * 60 * 60 * 1000, tz);
}
function isoDay(dayStart: number, tz: string): string {
const p = wallParts(dayStart, tz);
const pad = (n: number) => String(n).padStart(2, '0');
return `${p.year}-${pad(p.month)}-${pad(p.day)}`;
}
/** Clip a union to [lo, hi) and emit windows of `class`. */
function clip(
merged: Array<[number, number]>,
lo: number,
hi: number,
cls: DayWindow['class'],
): DayWindow[] {
const out: DayWindow[] = [];
for (const [s, e] of merged) {
const start = Math.max(s, lo);
const end = Math.min(e, hi);
if (end > start) out.push({ start, end, class: cls });
}
return out;
}
/**
* #395 §6.3 — bucket sessions into calendar days of `tz` for the punch-card.
* Pure and deterministic. `work` and `agent_only` are unioned SEPARATELY (else
* agent windows would swallow work windows on overlap), then each union is split
* at tz midnight boundaries (`startOf('day')` in tz, NOT "+24h" — DST-safe §9#14)
* and clipped to each day.
*
* By construction Σ perDay.activeMs == workMs: the days are a partition of the
* `work` union — no loss, no dup, even on 23/25h DST days. `agent_only` windows
* are drawn but NOT in activeMs. Empty days between the first and last active day
* are emitted (empty track + "—") so the rhythm/pauses stay visible.
*/
export function bucketByDay(sessions: WorkSession[], tz: string): PerDay[] {
const uWork = union(
sessions.filter((s) => s.class === 'work').map((s) => [s.start, s.end]),
);
const uAgent = union(
sessions
.filter((s) => s.class === 'agent_only')
.map((s) => [s.start, s.end]),
);
if (uWork.length === 0 && uAgent.length === 0) return [];
const minStart = Math.min(
uWork.length ? uWork[0][0] : Infinity,
uAgent.length ? uAgent[0][0] : Infinity,
);
const maxEnd = Math.max(
uWork.length ? uWork[uWork.length - 1][1] : -Infinity,
uAgent.length ? uAgent[uAgent.length - 1][1] : -Infinity,
);
const perDay: PerDay[] = [];
let dayStart = zonedDayStart(minStart, tz);
// Guard against a pathological non-advancing boundary.
let guard = 0;
while (dayStart < maxEnd && guard < 100000) {
guard++;
const dayEnd = nextDayStart(dayStart, tz);
const workWin = clip(uWork, dayStart, dayEnd, 'work');
const agentWin = clip(uAgent, dayStart, dayEnd, 'agent_only');
const activeMs = workWin.reduce((a, w) => a + (w.end - w.start), 0);
const agentMs = agentWin.reduce((a, w) => a + (w.end - w.start), 0);
const windows = [...workWin, ...agentWin].sort((a, b) => a.start - b.start);
perDay.push({
day: dayStart,
dayISO: isoDay(dayStart, tz),
activeMs,
agentMs,
windows,
});
dayStart = dayEnd;
}
return perDay;
}
@@ -0,0 +1,358 @@
import { computeWorkTime } from './compute-work-time';
import { bucketByDay } from './bucket-by-day';
import { TimelineSample, WorkSession } from './work-time.types';
const MIN = 60 * 1000;
/** Union wall-clock of a set of intervals (touching intervals merge). */
function unionMs(intervals: Array<[number, number]>): number {
if (intervals.length === 0) return 0;
const sorted = [...intervals].sort((a, b) => a[0] - b[0]);
let total = 0;
let [cs, ce] = sorted[0];
for (let i = 1; i < sorted.length; i++) {
const [s, e] = sorted[i];
if (s <= ce) {
if (e > ce) ce = e;
} else {
total += ce - cs;
cs = s;
ce = e;
}
}
return total + (ce - cs);
}
const ivsOf = (sessions: WorkSession[], cls?: string): Array<[number, number]> =>
sessions
.filter((x) => cls == null || x.class === cls)
.map((x) => [x.start, x.end] as [number, number]);
function s(
iso: string,
opts: {
source?: string | null;
chat?: string | null;
kind?: string | null;
by?: string | null;
} = {},
): TimelineSample {
return {
createdAt: `${iso}Z`,
lastUpdatedById: opts.by ?? 'human-1',
lastUpdatedSource: opts.source === undefined ? 'user' : opts.source,
lastUpdatedAiChatId: opts.chat ?? null,
kind: opts.kind ?? null,
};
}
// §7 config: T_gap=30m, P_in+P_out=10m, P_single=2m.
const S7 = { tGap: 30 * MIN, agentTGap: 30 * MIN, pIn: 5 * MIN, pOut: 5 * MIN, pSingle: 2 * MIN };
describe('computeWorkTime', () => {
it('§7 fixture — sessionizes 20-ish samples to ≈1h32m, not the ≈60h naive span', () => {
const rows: TimelineSample[] = [
// S1: multi-sample morning session
s('2026-07-04T03:40:00'),
s('2026-07-04T03:45:00'),
s('2026-07-04T03:49:00'),
// S2: agent burst (one run) then human supervising → class work
s('2026-07-04T15:43:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T15:47:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T15:50:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T16:13:00'),
// S3: single
s('2026-07-04T18:11:00'),
// S4: multi-sample evening session
s('2026-07-04T19:38:00'),
s('2026-07-04T19:44:00'),
s('2026-07-04T19:54:00'),
// S5 / S6: two singles two days later, 44m apart → two sessions at T_gap=30
s('2026-07-06T15:34:00'),
s('2026-07-06T16:18:00'),
];
const r = computeWorkTime(rows, S7);
// 19 + 40 + 2 + 26 + 2 + 2 = 91 minutes.
expect(r.workMs).toBe(91 * MIN);
expect(r.agentOnlyMs).toBe(0);
expect(r.sessions).toHaveLength(6);
expect(r.sessions.every((x) => x.class === 'work')).toBe(true);
const naiveSpan =
new Date('2026-07-06T16:18:00Z').getTime() -
new Date('2026-07-04T03:40:00Z').getTime();
expect(naiveSpan).toBeGreaterThan(60 * 60 * MIN); // ≈60h
expect(r.workMs).toBeLessThan(naiveSpan / 30); // dramatically smaller
});
it('n=0 → zero, no sessions', () => {
const r = computeWorkTime([]);
expect(r).toEqual({ workMs: 0, agentOnlyMs: 0, sessions: [] });
});
it('n=1 human → one P_single work session', () => {
const r = computeWorkTime([s('2026-07-04T10:00:00')], S7);
expect(r.sessions).toHaveLength(1);
expect(r.sessions[0].class).toBe('work');
expect(r.workMs).toBe(2 * MIN);
expect(r.agentOnlyMs).toBe(0);
// pre-roll only: [t − P_single, t]
expect(r.sessions[0].end).toBe(new Date('2026-07-04T10:00:00Z').getTime());
});
it('n=1 agent → one P_single agent_only session, work=0 (§9#2)', () => {
const r = computeWorkTime(
[s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' })],
S7,
);
expect(r.sessions).toHaveLength(1);
expect(r.sessions[0].class).toBe('agent_only');
expect(r.workMs).toBe(0);
expect(r.agentOnlyMs).toBe(2 * MIN);
});
it('MUST close the last session — the newest session is not lost (§9#1)', () => {
// Two singles a day apart: without the post-loop close, the 2nd is dropped.
const rows = [s('2026-07-04T10:00:00'), s('2026-07-05T10:00:00')];
const r = computeWorkTime(rows, S7);
expect(r.sessions).toHaveLength(2);
const lastStart = Math.max(...r.sessions.map((x) => x.start));
expect(lastStart).toBe(
new Date('2026-07-05T10:00:00Z').getTime() - 2 * MIN,
);
expect(r.workMs).toBe(4 * MIN);
});
it('agent-burst collapse: density does not inflate — length = wall-clock', () => {
const span = ['00', '01', '02', '03', '04', '05', '06'];
const dense: TimelineSample[] = span.map((sec) =>
s(`2026-07-04T10:00:${sec}`, { source: 'agent', chat: 'c1', kind: 'agent' }),
);
const sparse: TimelineSample[] = [
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:00:06', { source: 'agent', chat: 'c1', kind: 'agent' }),
];
const rDense = computeWorkTime(dense, S7);
const rSparse = computeWorkTime(sparse, S7);
// Same 6-second wall-clock span → same estimate regardless of snapshot count.
expect(rDense.agentOnlyMs).toBe(rSparse.agentOnlyMs);
expect(rDense.sessions).toHaveLength(1);
expect(rDense.sessions[0].class).toBe('agent_only');
});
it('supervisory agent time inside a human session counts as work, not agent', () => {
const rows = [
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:05:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:12:00'), // human within T_gap
];
const r = computeWorkTime(rows, S7);
expect(r.sessions).toHaveLength(1);
expect(r.sessions[0].class).toBe('work');
expect(r.agentOnlyMs).toBe(0);
expect(r.workMs).toBeGreaterThan(0);
});
it('a DIFFERENT aiChatId breaks the burst — two agent runs, idle gap excluded', () => {
// Run c1 ends 10:05, run c2 starts 10:20 (15m > agentTGap 7m) → two sessions.
const rows = [
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:05:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:20:00', { source: 'agent', chat: 'c2', kind: 'agent' }),
s('2026-07-04T10:25:00', { source: 'agent', chat: 'c2', kind: 'agent' }),
];
const r = computeWorkTime(rows); // default agentTGap = 7m
expect(r.sessions).toHaveLength(2);
expect(r.sessions.every((x) => x.class === 'agent_only')).toBe(true);
// The 15m idle gap between the two runs is NOT counted.
const run1 = 5 * MIN + 5 * MIN + 5 * MIN; // pIn + span + pOut
expect(r.agentOnlyMs).toBe(2 * run1);
});
it('idle pulse (same/null run) is a full activity sample that continues a burst', () => {
const rows = [
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
// idle flush 4m later, null run id → continues the burst, not a new one
s('2026-07-04T10:04:00', { source: 'agent', chat: null, kind: 'idle' }),
s('2026-07-04T10:08:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
];
const r = computeWorkTime(rows);
expect(r.sessions).toHaveLength(1);
// burst span 10:00→10:08 (+pIn/pOut) = 8 + 10 = 18m
expect(r.agentOnlyMs).toBe(18 * MIN);
});
it('a USER-sourced idle breaks an agent burst → session is work, not agent_only', () => {
// A human supervision idle inherits source=user (aiChatId:null) and must NOT
// be swallowed into the agent burst. Δ=3m is within the default agentTGap so
// the two samples stay one session — but its class flips to `work`.
const rows = [
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:03:00', { source: 'user', chat: null, kind: 'idle' }),
];
const r = computeWorkTime(rows);
expect(r.sessions).toHaveLength(1);
expect(r.sessions[0].class).toBe('work');
expect(r.workMs).toBeGreaterThan(0);
// The human idle is NOT captured as agent_only time.
expect(r.agentOnlyMs).toBe(0);
// Σ over `work` sessions == workMs and Σ over `agent_only` == agentOnlyMs.
const sum = (cls: string) =>
r.sessions
.filter((x) => x.class === cls)
.reduce((acc, x) => acc + (x.end - x.start), 0);
expect(sum('work')).toBe(r.workMs);
expect(sum('agent_only')).toBe(r.agentOnlyMs);
});
it('idle pulse keeps a human writing session visible (not excluded)', () => {
const rows = [
s('2026-07-04T10:00:00'),
s('2026-07-04T10:08:00', { kind: 'idle' }), // pulse within T_gap
s('2026-07-04T10:15:00'),
];
const r = computeWorkTime(rows);
expect(r.sessions).toHaveLength(1);
expect(r.sessions[0].class).toBe('work');
// span 10:00→10:15 + pIn/pOut = 15 + 10 = 25m
expect(r.workMs).toBe(25 * MIN);
});
it('git-source samples are excluded (§10 excludeGit)', () => {
const rows = [
s('2026-07-04T10:00:00', { source: 'git', kind: 'boundary' }),
s('2026-07-04T10:01:00', { source: 'git', kind: 'boundary' }),
];
expect(computeWorkTime(rows).workMs).toBe(0);
// ...but honoured off:
expect(
computeWorkTime(rows, { excludeGit: false }).workMs,
).toBeGreaterThan(0);
});
it('rejects an invalid config (tGap < pIn + pOut)', () => {
expect(() =>
computeWorkTime([s('2026-07-04T10:00:00')], {
tGap: 5 * MIN,
pIn: 5 * MIN,
pOut: 5 * MIN,
}),
).toThrow(/tGap/);
});
it('rejects an invalid config (2·agentTGap < pIn + pOut)', () => {
// tGap (default 15m) still ≥ pIn+pOut, so only the 2·agentTGap guard trips.
// Without it a short session of one class between two of the other could
// produce a NON-adjacent cross-class overlap the adjacent-only clip misses.
expect(() =>
computeWorkTime([s('2026-07-04T10:00:00')], {
agentTGap: 2 * MIN,
pIn: 5 * MIN,
pOut: 5 * MIN,
}),
).toThrow(/agentTGap/);
});
// F1 — cross-class double-count. On the DEFAULT config agentTGap (7m) < pIn+pOut
// (10m), so a `work` session ending in an agent segment and a nearby separate
// `agent_only` run (gap in (7m,10m]) used to produce OVERLAPPING padded
// intervals — the same wall-clock counted into BOTH workMs and agentOnlyMs. The
// cross-class padding clip must make the two per-class unions disjoint.
it('does NOT double-count wall-clock across work/agent_only (§F1)', () => {
// user@0s ; agent(chatX)@60s (breaks into a work session with the human) ;
// agent(chatY)@560s,590s (a separate agent_only run). Raw gap between the work
// session (ends 60s) and the agent run (starts 560s) is 500s ∈ (agentTGap,
// pIn+pOut] once padded — the classic overlap window.
const rows: TimelineSample[] = [
s('2026-07-04T00:00:00'), // user @ 0s
s('2026-07-04T00:01:00', { source: 'agent', chat: 'cX', kind: 'agent' }), // @ 60s
s('2026-07-04T00:09:20', { source: 'agent', chat: 'cY', kind: 'agent' }), // @ 560s
s('2026-07-04T00:09:50', { source: 'agent', chat: 'cY', kind: 'agent' }), // @ 590s
];
const r = computeWorkTime(rows); // DEFAULT config
// Both classes present.
expect(r.workMs).toBeGreaterThan(0);
expect(r.agentOnlyMs).toBeGreaterThan(0);
// Per-class metrics are exactly their own union (union, not Σ).
expect(r.workMs).toBe(unionMs(ivsOf(r.sessions, 'work')));
expect(r.agentOnlyMs).toBe(unionMs(ivsOf(r.sessions, 'agent_only')));
// The F1 invariant: work-union and agent-union are cross-class-disjoint, so
// the union of ALL padded intervals equals workMs + agentOnlyMs (no overlap).
// With the clip disabled this fails (union < sum by the 100s overlap).
expect(unionMs(ivsOf(r.sessions))).toBe(r.workMs + r.agentOnlyMs);
});
// F1 property/fuzz — random timelines across several timezones must uphold the
// work-time invariants. Backs the (corrected) PR claim of a real fuzz test.
it('property: random timelines uphold union & cross-class-disjoint invariants', () => {
// Deterministic LCG (numerical-recipes constants) so a failure is reproducible.
let seed = 0x9e3779b9 >>> 0;
const rand = () => {
seed = (Math.imul(seed, 1664525) + 1013904223) >>> 0;
return seed / 0x100000000;
};
const pick = <T>(arr: T[]): T => arr[Math.floor(rand() * arr.length)];
const tzs = [
'UTC',
'America/New_York',
'Europe/Moscow',
'Australia/Lord_Howe', // 30-min DST offset — a nasty bucket stress
];
const base = Date.UTC(2026, 5, 1, 0, 0, 0); // 2026-06-01Z
const chats = ['c1', 'c2', 'c3'];
for (let iter = 0; iter < 250; iter++) {
const tz = pick(tzs);
const n = 2 + Math.floor(rand() * 18); // 2..19 rows
const rows: TimelineSample[] = [];
// Walk time forward by a random inter-sample gap. The gap distribution is
// centred on the DANGEROUS band — a bit under to a bit over pIn+pOut (10m)
// AND straddling agentTGap (7m) — so adjacent samples routinely split into
// separate sessions whose ±P padding would overlap if a class boundary sits
// there. Mixing user/agent classes at these gaps reliably manufactures the
// work-ending-in-agent → agent_only cross-class boundary F1 is about, plus
// dense within-class runs (occasional 0–2m gaps) that exercise the union.
let t = base + Math.floor(rand() * 60 * MIN);
for (let i = 0; i < n; i++) {
const roll = rand();
const gap =
roll < 0.25
? Math.floor(rand() * 2 * MIN) // dense burst (same-class union)
: roll < 0.85
? 5 * MIN + Math.floor(rand() * 8 * MIN) // 5–13m: the split band
: 20 * MIN + Math.floor(rand() * 40 * MIN); // long idle → new day-ish
t += gap;
const iso = new Date(t).toISOString().slice(0, 19); // 'YYYY-MM-DDTHH:MM:SS'
const isAgent = rand() < 0.5;
rows.push(
isAgent
? s(iso, { source: 'agent', chat: pick(chats), kind: 'agent' })
: s(iso, { source: 'user', kind: rand() < 0.3 ? 'idle' : 'manual' }),
);
}
const r = computeWorkTime(rows); // DEFAULT config
const workIvs = ivsOf(r.sessions, 'work');
const agentIvs = ivsOf(r.sessions, 'agent_only');
// (1) each metric is exactly its per-class union (catches a union→Σ regress).
expect(r.workMs).toBe(unionMs(workIvs));
expect(r.agentOnlyMs).toBe(unionMs(agentIvs));
// (2) NO cross-class overlap: union(all) == workMs + agentOnlyMs (F1).
expect(unionMs(ivsOf(r.sessions))).toBe(r.workMs + r.agentOnlyMs);
// (3) bucket invariant: Σ per-day activeMs == workMs (§6.3).
const perDay = bucketByDay(r.sessions, tz);
const sumActive = perDay.reduce((a, d) => a + d.activeMs, 0);
expect(sumActive).toBe(r.workMs);
}
});
});
@@ -0,0 +1,274 @@
import {
TimelineSample,
WorkSession,
WorkTimeResult,
} from './work-time.types';
import { WorkTimeConfig, resolveWorkTimeConfig } from './work-time.config';
/** A normalized activity sample (one history row), createdAt as epoch-ms. */
interface NormSample {
t: number;
isAgent: boolean;
aiChatId: string | null;
kind: string | null;
}
/**
* A collapsed segment: either a scalar sample (t_start == t_end) or an
* agent-burst spanning several agent samples of one run (§5.1). It participates
* in sessionization as a single "sample".
*/
interface Segment {
tStart: number;
tEnd: number;
isAgent: boolean;
}
function toMs(v: Date | string | number): number {
if (v instanceof Date) return v.getTime();
if (typeof v === 'number') return v;
return new Date(v).getTime();
}
/**
* Normalize raw rows → sorted, deduped activity samples. `git` is dropped when
* configured; every other kind (incl. `idle` — the continuous-work pulse §3) is
* a real activity sample. Sort is by createdAt ASC; samples whose timestamps
* fall in the same `dedupRoundMs` bucket collapse to one (§9#7: a synchronous
* boundary row + the immediate agent snapshot can share a createdAt). A merged
* sample is human unless EVERY member is an agent, so supervision never gets
* mis-attributed to the agent.
*/
function normalize(
rows: TimelineSample[],
config: WorkTimeConfig,
): NormSample[] {
const samples: NormSample[] = [];
for (const row of rows) {
const source = row.lastUpdatedSource;
if (config.excludeGit && source === 'git') continue;
samples.push({
t: toMs(row.createdAt),
isAgent: source === 'agent',
aiChatId: row.lastUpdatedAiChatId ?? null,
kind: row.kind ?? null,
});
}
samples.sort((a, b) => a.t - b.t);
if (config.dedupRoundMs <= 0 || samples.length < 2) return samples;
const deduped: NormSample[] = [];
for (const s of samples) {
const prev = deduped[deduped.length - 1];
if (prev && s.t - prev.t < config.dedupRoundMs) {
// Merge into the previous sample. Human wins the class; keep the earliest
// t; keep a non-null aiChatId if either has one (so a bare boundary row
// does not erase the run id).
prev.isAgent = prev.isAgent && s.isAgent;
prev.aiChatId = prev.aiChatId ?? s.aiChatId;
// Prefer the more specific kind (a real kind over a null/boundary) only
// matters for burst continuation; keep prev.kind (earliest) as-is.
continue;
}
deduped.push({ ...s });
}
return deduped;
}
/**
* Collapse consecutive same-run agent samples into one burst segment (§5.1) so a
* dense burst (8 snapshots in 7 minutes) contributes its wall-clock, not a count
* × block. A burst is broken by any sample NOT continuing the same aiChatId
* agent run: a non-agent sample, a `boundary` (actor transition), or a DIFFERENT
* aiChatId. Only an AGENT-sourced `idle` pulse with the SAME or a null aiChatId
* continues the burst (its label lags the real edit ≤ maxWait, well within
* rounding); a user-sourced `idle` (a human supervision pulse) breaks it.
*/
function collapse(samples: NormSample[], config: WorkTimeConfig): Segment[] {
const segments: Segment[] = [];
let burst: { chatId: string | null; tStart: number; tEnd: number } | null =
null;
const flush = () => {
if (!burst) return;
let tEnd = burst.tEnd;
if (config.burstCapMs != null && tEnd - burst.tStart > config.burstCapMs) {
tEnd = burst.tStart + config.burstCapMs;
}
segments.push({ tStart: burst.tStart, tEnd, isAgent: true });
burst = null;
};
for (const s of samples) {
// An agent-sourced idle pulse continues the current agent burst (same or
// null run id). A user-sourced idle (human supervision) must NOT be swallowed
// here — it falls through to the human branch so the session flips to `work`.
if (
burst &&
s.kind === 'idle' &&
s.isAgent &&
(s.aiChatId === burst.chatId || s.aiChatId == null)
) {
burst.tEnd = s.t;
continue;
}
if (s.isAgent && s.kind !== 'boundary') {
if (burst && burst.chatId === s.aiChatId) {
burst.tEnd = s.t;
} else {
flush();
burst = { chatId: s.aiChatId, tStart: s.t, tEnd: s.t };
}
continue;
}
// A human sample, a boundary, or an agent-boundary: breaks the burst and is
// itself a zero-width segment (its class follows its own source).
flush();
segments.push({ tStart: s.t, tEnd: s.t, isAgent: s.isAgent });
}
flush();
return segments;
}
function gapThreshold(
a: Segment,
b: Segment,
config: WorkTimeConfig,
): number {
return a.isAgent && b.isAgent ? config.agentTGap : config.tGap;
}
/** Merge intervals; overlapping OR touching intervals are unioned. */
function unionDuration(intervals: Array<[number, number]>): number {
if (intervals.length === 0) return 0;
const sorted = [...intervals].sort((a, b) => a[0] - b[0]);
let total = 0;
let [curStart, curEnd] = sorted[0];
for (let i = 1; i < sorted.length; i++) {
const [s, e] = sorted[i];
if (s <= curEnd) {
if (e > curEnd) curEnd = e;
} else {
total += curEnd - curStart;
curStart = s;
curEnd = e;
}
}
total += curEnd - curStart;
return total;
}
/**
* #395 core — estimate time worked on a page from its history timeline (§5).
* Pure and deterministic: no DB, no clock, no I/O.
*
* Pipeline: normalize+dedup → collapse agent bursts → ONE sessionization pass
* over all segments (threshold depends on the pair: both-agent → agentTGap, else
* tGap; the last session is ALWAYS closed after the loop) → class per finished
* session (all-agent → agent_only, else work) → pad each session (multi-sample
* → [first−P_in, last+P_out]; lone scalar → [t−P_single, t]) → clip padding of
* adjacent DIFFERENT-class sessions at the raw-gap midpoint (so work/agent_only
* never overlap) → metrics are the union wall-clock within each class (union, not
* Σ, so overlaps never double, and cross-class-disjoint by the clip above).
*/
export function computeWorkTime(
rows: TimelineSample[],
config?: Partial<WorkTimeConfig>,
): WorkTimeResult {
const cfg = resolveWorkTimeConfig(config);
const samples = normalize(rows, cfg);
const segments = collapse(samples, cfg);
// Sessionize — one pass over ALL segments.
const rawSessions: Segment[][] = [];
let cur: Segment[] | null = null;
for (const seg of segments) {
if (cur == null) {
cur = [seg];
} else {
const last = cur[cur.length - 1];
if (seg.tStart - last.tEnd <= gapThreshold(last, seg, cfg)) {
cur.push(seg);
} else {
rawSessions.push(cur);
cur = [seg];
}
}
}
if (cur != null) rawSessions.push(cur); // MUST close the last session (§5, §9#1)
// A finished session with BOTH its raw (unpadded) span and its padded bounds.
// `rawSessions` are already in ascending time order, so `built` is too.
interface BuiltSession {
rawStart: number;
rawEnd: number;
padStart: number;
padEnd: number;
cls: WorkSession['class'];
}
const built: BuiltSession[] = [];
for (const segs of rawSessions) {
const first = segs[0];
const last = segs[segs.length - 1];
const cls = segs.every((s) => s.isAgent) ? 'agent_only' : 'work';
let padStart: number;
let padEnd: number;
if (segs.length === 1 && first.tStart === first.tEnd) {
// Lone single-instant session (one scalar, or a one-snapshot agent run):
// pre-roll only, no invented "future" work (§5).
padStart = first.tStart - cfg.pSingle;
padEnd = first.tStart;
} else {
padStart = first.tStart - cfg.pIn;
padEnd = last.tEnd + cfg.pOut;
}
built.push({
rawStart: first.tStart,
rawEnd: last.tEnd,
padStart,
padEnd,
cls,
});
}
// Clip cross-class padding so a `work` and an `agent_only` session that abut
// never claim the same wall-clock. For each ADJACENT pair of DIFFERENT classes,
// cap the earlier session's trailing pad and the later session's leading pad at
// the MIDPOINT of the raw (unpadded) inactivity gap between them: the earlier
// padded interval then ends ≤ midpoint and the later one starts ≥ midpoint, so
// the two are disjoint (they touch at most at the midpoint). This makes the
// per-class unions (workMs / agentOnlyMs) cross-class-disjoint BY CONSTRUCTION
// — closing the double-count where a work session ending in an agent segment
// and a nearby agent_only session (gap in (agentTGap, pIn+pOut]) overlapped and
// were counted into both metrics (§5, §9). Within-class adjacency is left
// untouched: `unionDuration` already dedups it, and clipping there could perturb
// the per-class metric value.
for (let i = 1; i < built.length; i++) {
const a = built[i - 1];
const b = built[i];
if (a.cls === b.cls) continue;
const midpoint = (a.rawEnd + b.rawStart) / 2;
if (a.padEnd > midpoint) a.padEnd = midpoint;
if (b.padStart < midpoint) b.padStart = midpoint;
}
const sessions: WorkSession[] = [];
const workIvs: Array<[number, number]> = [];
const agentIvs: Array<[number, number]> = [];
for (const s of built) {
sessions.push({ start: s.padStart, end: s.padEnd, class: s.cls });
(s.cls === 'work' ? workIvs : agentIvs).push([s.padStart, s.padEnd]);
}
sessions.sort((a, b) => a.start - b.start);
return {
workMs: unionDuration(workIvs),
agentOnlyMs: unionDuration(agentIvs),
sessions,
};
}
@@ -0,0 +1,15 @@
export { computeWorkTime } from './compute-work-time';
export { bucketByDay, zonedDayStart } from './bucket-by-day';
export {
DEFAULT_WORK_TIME_CONFIG,
resolveWorkTimeConfig,
} from './work-time.config';
export type { WorkTimeConfig } from './work-time.config';
export type {
TimelineSample,
WorkSession,
WorkTimeResult,
SessionClass,
DayWindow,
PerDay,
} from './work-time.types';
@@ -0,0 +1,102 @@
import {
IDLE_MAX_WAIT_USER,
IDLE_MAX_WAIT_AGENT,
} from '../../../collaboration/constants';
/**
* #395 — tunables for the work-time estimate (§10). Defaults are calibrated off
* #374's idle-pulse ceilings: after #374 a continuous editing session leaves a
* history row at least every ~IDLE_MAX_WAIT (10m user / 5m agent), so a gap
* WIDER than that ceiling contains un-pulsed idle time = (partial) inactivity.
* `tGap` therefore sits a little above the user ceiling, `agentTGap` a little
* above the agent ceiling — a gap within the threshold is pulse-backed and
* counts as work.
*/
export interface WorkTimeConfig {
/** user inactivity timeout: gap ≤ tGap between samples = continuous work. */
tGap: number;
/** timeout for a pair of consecutive agent samples (tighter than tGap). */
agentTGap: number;
/** pre-roll padding for a multi-sample session (work began before sample 1). */
pIn: number;
/** post-roll padding for a multi-sample session (work continued after last). */
pOut: number;
/** block for a lone single-sample session (pre-roll only, no invented future). */
pSingle: number;
/** drop `git`-source samples (they are not human/agent article work). */
excludeGit: boolean;
/** optional cap on one collapsed agent-burst segment's wall-clock (§9#3). */
burstCapMs?: number;
/** samples whose createdAt round to the same bucket dedup to one (§9#7). */
dedupRoundMs: number;
}
export const DEFAULT_WORK_TIME_CONFIG: WorkTimeConfig = {
// ~15m: IDLE_MAX_WAIT_USER (10m) + headroom. Empirically backcast on a real
// 307-snapshot article (≈24h at 15m matched the owner's estimate; 30/45m
// over-counted). See #395 §10.
tGap: 15 * 60 * 1000,
// ~7m: IDLE_MAX_WAIT_AGENT (5m) + headroom.
agentTGap: 7 * 60 * 1000,
pIn: 5 * 60 * 1000,
pOut: 5 * 60 * 1000,
pSingle: 2 * 60 * 1000,
excludeGit: true,
burstCapMs: undefined,
dedupRoundMs: 1000,
};
// Compile-time cross-check that the defaults really are pulse-anchored — if a
// future edit moves the #374 ceilings, this reminds us to re-calibrate.
void IDLE_MAX_WAIT_USER;
void IDLE_MAX_WAIT_AGENT;
/**
* Fill a partial config with defaults and validate it. Cross-class metric
* disjointness is guaranteed jointly by `computeWorkTime`'s adjacent-pair padding
* clip (it caps the padding of adjacent DIFFERENT-class sessions at the raw-gap
* midpoint) AND the two bounds enforced below (§5):
* - `tGap ≥ pIn + pOut`: a session's own padding never exceeds its inactivity
* window.
* - `2·agentTGap ≥ pIn + pOut`: makes the adjacent-only clip provably COMPLETE.
* A NON-adjacent (i, i+2) cross-class overlap could only arise from two
* same-class sessions separated by a full intervening session of the other
* class; that separation spans at least two inter-session gaps, each strictly
* `> agentTGap`, so it is `> 2·agentTGap`. Requiring `2·agentTGap ≥ pIn + pOut`
* means even the widest padded reach (pIn + pOut) cannot bridge it — so the
* only cross-class overlaps possible are between ADJACENT sessions, which the
* clip handles. `workMs`/`agentOnlyMs` are therefore disjoint by construction.
*/
export function resolveWorkTimeConfig(
partial?: Partial<WorkTimeConfig>,
): WorkTimeConfig {
const config = { ...DEFAULT_WORK_TIME_CONFIG, ...(partial ?? {}) };
for (const key of [
'tGap',
'agentTGap',
'pIn',
'pOut',
'pSingle',
'dedupRoundMs',
] as const) {
const value = config[key];
if (!Number.isFinite(value) || value < 0) {
throw new Error(`work-time config: ${key} must be a non-negative number`);
}
}
if (config.burstCapMs != null && config.burstCapMs <= 0) {
throw new Error('work-time config: burstCapMs must be > 0 when set');
}
if (config.tGap < config.pIn + config.pOut) {
throw new Error(
"work-time config: tGap must be ≥ pIn + pOut (a session's padding may not exceed its inactivity window)",
);
}
if (2 * config.agentTGap < config.pIn + config.pOut) {
throw new Error(
'work-time config: 2·agentTGap must be ≥ pIn + pOut (so non-adjacent cross-class padding cannot overlap)',
);
}
return config;
}
@@ -0,0 +1,73 @@
/**
* #395 — "time worked on an article" domain types.
*
* The estimate is built by sessionizing a page's `page_history` timeline on
* inactivity gaps (WakaTime-style), NOT by taking the span between the first and
* last edit (which over-counts sleep / lunch / idle days). See the design doc in
* issue #395 §5–§6.3 for the normative algorithm.
*/
/**
* A single `page_history` row projected for the work-time computation — the
* cheap columns only (no `content`). Produced by
* `PageHistoryRepo.findTimelineByPageId`. `createdAt` is whatever the DB driver
* hands back (Date); the pure core normalizes it to epoch-ms itself so it stays
* deterministic and DB-free.
*/
export interface TimelineSample {
createdAt: Date | string | number;
lastUpdatedById: string | null;
/** 'user' | 'agent' | 'git' | null (legacy autosave = human). */
lastUpdatedSource: string | null;
lastUpdatedAiChatId: string | null;
/** #370 tier: 'manual' | 'agent' | 'idle' | 'boundary' | null (legacy). */
kind: string | null;
}
/** A finished session's class (§5.1). */
export type SessionClass = 'work' | 'agent_only';
/**
* A finished session: absolute wall-clock bounds already padded with P_in/P_out
* (multi-sample) or P_single (single scalar), plus its class. This is enough for
* both the metrics and the per-day punch-card colouring.
*/
export interface WorkSession {
/** epoch-ms, inclusive lower bound (already P-padded). */
start: number;
/** epoch-ms, exclusive upper bound (already P-padded). */
end: number;
class: SessionClass;
}
/** Output of {@link computeWorkTime}. */
export interface WorkTimeResult {
/** union wall-clock of `work` sessions, ms (the headline metric). */
workMs: number;
/** union wall-clock of `agent_only` sessions, ms (secondary). */
agentOnlyMs: number;
sessions: WorkSession[];
}
/** One activity window inside a calendar day (already clipped to the day). */
export interface DayWindow {
/** epoch-ms. */
start: number;
/** epoch-ms. */
end: number;
class: SessionClass;
}
/** One calendar day of the punch-card (§6.3). */
export interface PerDay {
/** epoch-ms of the local-midnight day start in the requested tz. */
day: number;
/** 'YYYY-MM-DD' in the requested tz — stable, tz-independent label. */
dayISO: string;
/** Σ of `work` windows this day, ms. Σ over days == workMs (invariant §6.3). */
activeMs: number;
/** Σ of `agent_only` windows this day, ms (drawn, NOT in activeMs). */
agentMs: number;
/** both classes, clipped to the day, sorted by start (for drawing). */
windows: DayWindow[];
}
@@ -157,6 +157,44 @@ export class PageHistoryRepo {
return { ...result, items: result.items.map(attachPageHistoryAgent) };
}
/**
* #395 — cheap projection of a page's FULL history timeline for the work-time
* estimate: only the columns the sessionizer needs, no heavy `content`, sorted
* oldest→newest. The secondary `id` tie-break keeps rows sharing a `createdAt`
* (e.g. a synchronous pre-agent boundary row + the immediate agent snapshot)
* in a deterministic order.
*/
async findTimelineByPageId(
pageId: string,
trx?: KyselyTransaction,
): Promise<
Array<
Pick<
PageHistory,
| 'createdAt'
| 'lastUpdatedById'
| 'lastUpdatedSource'
| 'lastUpdatedAiChatId'
| 'kind'
>
>
> {
const db = dbOrTx(this.db, trx);
return db
.selectFrom('pageHistory')
.select([
'createdAt',
'lastUpdatedById',
'lastUpdatedSource',
'lastUpdatedAiChatId',
'kind',
])
.where('pageId', '=', pageId)
.orderBy('createdAt', 'asc')
.orderBy('id', 'asc')
.execute();
}
async findPageLastHistory(
pageId: string,
opts?: {
@@ -85,14 +85,6 @@ export class ImportController {
throw new BadRequestException('spaceId is required');
}
// #502: optional multipart field. Only the MCP agent `createPage` path sends
// `disableMarkdownExtensions=true` (its body is agent-authored plain prose /
// config, so a `$…$` span must stay literal and a bare `www.host` must not
// autolink). A HUMAN file upload omits the field, so it stays false and math
// + autolink remain ON for human imports. Settable ONLY via this API param.
const disableMarkdownExtensions =
file.fields?.disableMarkdownExtensions?.value === 'true';
const ability = await this.spaceAbility.createForUser(user, spaceId);
if (ability.cannot(SpaceCaslAction.Edit, SpaceCaslSubject.Page)) {
throw new ForbiddenException();
@@ -103,7 +95,6 @@ export class ImportController {
user.id,
spaceId,
workspace.id,
disableMarkdownExtensions,
);
const ext = path.extname(file.filename).toLowerCase();
@@ -1,67 +0,0 @@
// Importing ImportService transitively loads import-formatter.ts, which imports
// the ESM-only @sindresorhus/slugify (not in jest's transform allowlist). It is
// irrelevant to this path, so mock it to keep the module graph loadable (mirrors
// the sibling import.service specs).
jest.mock('@sindresorhus/slugify', () => ({
__esModule: true,
default: (input: string) => String(input),
}));
import { ImportService } from './import.service';
// #502 BLOCKER 1: the server markdown import path (`/pages/import`) now accepts an
// optional `disableMarkdownExtensions` flag threaded into `processMarkdown`.
// - MCP agent `createPage` sends it TRUE -> extensions OFF (a `$…$` config span
// stays literal, a schemeless `www.host` is not autolinked).
// - a HUMAN file upload omits it (default FALSE) -> extensions ON, so a real
// `$x^2$` still becomes a formula (human imports unaffected).
// `processMarkdown` only uses the imported converter (no injected deps on this
// path), so the service is constructed with null deps for this focused unit test.
function makeService(): ImportService {
return new ImportService(null as any, null as any, null as any, null as any);
}
function findAll(node: any, type: string, acc: any[] = []): any[] {
if (!node || typeof node !== 'object') return acc;
if (node.type === type) acc.push(node);
if (Array.isArray(node.content)) for (const c of node.content) findAll(c, type, acc);
return acc;
}
function hasLink(node: any): boolean {
return findAll(node, 'text').some((t: any) =>
t.marks?.some((m: any) => m.type === 'link'),
);
}
describe('ImportService.processMarkdown — #502 disableMarkdownExtensions', () => {
it('disableMarkdownExtensions=true (MCP createPage): `$…$` stays literal, no math', async () => {
const svc = makeService();
const doc = await svc.processMarkdown('export A=$FOO and B=$BAR done', true);
expect(findAll(doc, 'mathInline')).toHaveLength(0);
});
it('disableMarkdownExtensions=true: a schemeless www is NOT autolinked', async () => {
const svc = makeService();
const doc = await svc.processMarkdown('see www.example.com here', true);
expect(hasLink(doc)).toBe(false);
});
it('disableMarkdownExtensions=true: an explicit https:// STILL links', async () => {
const svc = makeService();
const doc = await svc.processMarkdown('see https://example.com here', true);
expect(hasLink(doc)).toBe(true);
});
it('DEFAULT (human upload): a real `$x^2$` DOES become a math node', async () => {
const svc = makeService();
const doc = await svc.processMarkdown('$x^2$');
expect(findAll(doc, 'mathInline')).toHaveLength(1);
});
it('DEFAULT (human upload): a schemeless www IS autolinked', async () => {
const svc = makeService();
const doc = await svc.processMarkdown('see www.example.com here');
expect(hasLink(doc)).toBe(true);
});
});
@@ -52,12 +52,6 @@ export class ImportService {
userId: string,
spaceId: string,
workspaceId: string,
// #502: when true, the markdown importer runs with the two layered
// extensions OFF (a `$…$` span stays literal text; a schemeless `www.host` is
// NOT autolinked). ONLY the MCP agent `createPage` path sets this; a HUMAN
// file upload never passes it, so it defaults false and math/autolink stay ON
// for human imports (their `$x^2$` still becomes a formula).
disableMarkdownExtensions = false,
) {
const file = await filePromise;
const fileBuffer = await file.toBuffer();
@@ -72,10 +66,7 @@ export class ImportService {
try {
if (fileExtension.endsWith('.md')) {
prosemirrorState = await this.processMarkdown(
fileContent,
disableMarkdownExtensions,
);
prosemirrorState = await this.processMarkdown(fileContent);
} else if (fileExtension.endsWith('.html')) {
prosemirrorState = await this.processHTML(fileContent);
}
@@ -147,12 +138,7 @@ export class ImportService {
return createdPage;
}
async processMarkdown(
markdownInput: string,
// #502: forwarded to the importer. DEFAULT false keeps math + fuzzy autolink
// ON (human uploads unaffected); the MCP agent `createPage` path passes true.
disableMarkdownExtensions = false,
): Promise<any> {
async processMarkdown(markdownInput: string): Promise<any> {
// Canonical markdown -> ProseMirror JSON directly via
// `@docmost/prosemirror-markdown` (issue #345) — no HTML intermediate and no
// second editor-ext markdown layer. Foreign markdown surfaces the strict
@@ -161,12 +147,7 @@ export class ImportService {
// The HTML-cleanup pass (`normalizeImportHtml`) is intentionally skipped here:
// it targets foreign *HTML* (Notion/XWiki), which only ever arrives on the
// `.html` path (`processHTML`), never as canonical markdown.
return markdownToProseMirror(
normalizeForeignMarkdown(markdownInput),
disableMarkdownExtensions
? { parseMath: false, fuzzyLinkify: false }
: undefined,
);
return markdownToProseMirror(normalizeForeignMarkdown(markdownInput));
}
async processHTML(htmlInput: string): Promise<any> {
@@ -0,0 +1,137 @@
import { randomUUID } from 'node:crypto';
import { Kysely } from 'kysely';
import { PageHistoryRepo } from '../../src/database/repos/page/page-history.repo';
import { PageHistoryService } from '../../src/core/page/services/page-history.service';
import { computeWorkTime } from '../../src/core/page/work-time';
import {
getTestDb,
destroyTestDb,
createWorkspace,
createSpace,
createPage,
createUser,
createChat,
} from './db';
/**
* #395 — real-Postgres coverage for the work-time timeline projection and the
* service that computes the estimate. The pure sessionizer is unit-tested
* exhaustively (compute-work-time.spec.ts); this asserts the SQL projection
* (right rows, ASC, no `content`) and that the service's numbers agree with the
* pure core over the exact rows the DB returns.
*/
describe('PageHistory work-time [integration]', () => {
let db: Kysely<any>;
let repo: PageHistoryRepo;
let service: PageHistoryService;
let workspaceId: string;
let spaceId: string;
let pageId: string;
let userId: string;
let chatId: string;
const MIN = 60 * 1000;
beforeAll(async () => {
db = getTestDb();
repo = new PageHistoryRepo(db as any);
service = new PageHistoryService(repo);
workspaceId = (await createWorkspace(db)).id;
spaceId = (await createSpace(db, workspaceId)).id;
pageId = (await createPage(db, { workspaceId, spaceId })).id;
userId = (await createUser(db, workspaceId)).id;
chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
});
afterAll(async () => {
await destroyTestDb();
});
async function insertHistory(rows: Array<{
createdAt: string;
source: string | null;
chat?: string | null;
kind?: string | null;
content?: unknown;
}>) {
for (const r of rows) {
await db
.insertInto('pageHistory')
.values({
id: randomUUID(),
pageId,
spaceId,
workspaceId,
title: 'x',
content: r.content ?? { type: 'doc', content: [] },
lastUpdatedById: userId,
lastUpdatedSource: r.source,
lastUpdatedAiChatId: r.chat ?? null,
kind: r.kind ?? null,
createdAt: new Date(r.createdAt),
})
.execute();
}
}
it('findTimelineByPageId projects the cheap columns, ASC, without content', async () => {
await insertHistory([
{ createdAt: '2026-07-04T19:54:00Z', source: 'user', kind: 'manual' },
{ createdAt: '2026-07-04T03:40:00Z', source: 'user', kind: null },
{ createdAt: '2026-07-04T15:43:00Z', source: 'agent', chat: chatId, kind: 'agent' },
]);
const timeline = await repo.findTimelineByPageId(pageId);
expect(timeline).toHaveLength(3);
// Sorted oldest → newest.
const times = timeline.map((r) => new Date(r.createdAt).getTime());
expect(times).toEqual([...times].sort((a, b) => a - b));
// Projection carries exactly the sessionizer's inputs, and NO content.
for (const row of timeline) {
expect(row).toHaveProperty('createdAt');
expect(row).toHaveProperty('lastUpdatedById');
expect(row).toHaveProperty('lastUpdatedSource');
expect(row).toHaveProperty('lastUpdatedAiChatId');
expect(row).toHaveProperty('kind');
expect(row).not.toHaveProperty('content');
}
// Agent row keeps its provenance.
const agent = timeline.find((r) => r.lastUpdatedSource === 'agent');
expect(agent?.lastUpdatedAiChatId).toBe(chatId);
});
it('service estimate matches the pure core and satisfies Σ perDay == workMs', async () => {
const rows = await repo.findTimelineByPageId(pageId);
const pure = computeWorkTime(rows);
const result = await service.computeWorkTime(pageId, 'UTC');
expect(result.workMs).toBe(pure.workMs);
expect(result.agentOnlyMs).toBe(pure.agentOnlyMs);
expect(result.tz).toBe('UTC');
expect(result.config.tGap).toBe(15 * MIN);
const sumActive = result.perDay.reduce((a, d) => a + d.activeMs, 0);
expect(sumActive).toBe(result.workMs);
// The 3 seeded rows sessionize into ≤ their span; not the naive span.
const naive =
new Date('2026-07-04T19:54:00Z').getTime() -
new Date('2026-07-04T03:40:00Z').getTime();
expect(result.workMs).toBeGreaterThan(0);
expect(result.workMs).toBeLessThan(naive);
});
it('an unknown timezone surfaces as a RangeError (controller maps to 400)', async () => {
await expect(
service.computeWorkTime(pageId, 'Not/AZone'),
).rejects.toBeInstanceOf(RangeError);
});
it('a page with no history → zeros, no days', async () => {
const emptyPage = (await createPage(db, { workspaceId, spaceId })).id;
const result = await service.computeWorkTime(emptyPage, 'UTC');
expect(result.workMs).toBe(0);
expect(result.agentOnlyMs).toBe(0);
expect(result.perDay).toEqual([]);
});
});
+1 -4
View File
@@ -118,10 +118,7 @@ All 41 tools, grouped by what you'd reach for them.
- **`getPage`** — A page's content as clean **Markdown** (canonical for text; drops only
block ids, resolved-comment anchors, and a fixed no-Markdown-representation attr set —
table spans/colwidth/background, indent, `callout.icon`, `orderedList.type`, and link
`internal`/`target`/`rel`/`class`; use `getPageJson` when you need those). Pass
`format:"text"` for a flat, deterministic plain-text rendering (one line per block,
marks dropped, stable `[image]`/`[table RxC]` placeholders) to machine-diff what you
wrote against what was stored.
`internal`/`target`/`rel`/`class`; use `getPageJson` when you need those).
- **`getPageJson`** — A page's **lossless ProseMirror/TipTap JSON**, including every
block's `attrs.id` and the `slugId` used in URLs. This is what the per-block editing
tools consume.
+1 -4
View File
@@ -123,10 +123,7 @@ Docmost-MCP не сочетают:
лишь id блоков, якоря разрешённых комментариев и фиксированный набор атрибутов без
markdown-представления — спаны/colwidth/фон ячеек таблиц, отступы (indent),
`callout.icon`, `orderedList.type` и `internal`/`target`/`rel`/`class` у ссылок;
используйте `getPageJson`, когда они нужны). Передайте `format:"text"` для плоского
детерминированного plain-text рендера (одна строка на блок, маркировка убрана,
стабильные плейсхолдеры `[image]`/`[table RxC]`) — чтобы machine-diff'ом сверить то,
что вы записали, с тем, что сохранилось.
используйте `getPageJson`, когда они нужны).
- **`getPageJson`** — **Lossless ProseMirror/TipTap JSON** страницы, включая `attrs.id`
каждого блока и `slugId`, используемый в URL. Именно его потребляют инструменты
поблочного редактирования.
+3 -13
View File
@@ -701,20 +701,10 @@ export abstract class DocmostClientContext {
}
/**
* Raw page info including the ProseMirror JSON content and slugId.
*
* With `format:"text"` (#502) the server instead renders `content` as a flat,
* deterministic text string (its `jsonToText` path — the SAME serializer that
* feeds search), so the MCP text read reuses the server's ONE serializer
* rather than shipping a second one. Every other caller omits `format` and
* gets the JSON content unchanged.
*/
async getPageRaw(pageId: string, format?: "text") {
/** Raw page info including the ProseMirror JSON content and slugId. */
async getPageRaw(pageId: string) {
await this.ensureAuthenticated();
const body: Record<string, unknown> = { pageId };
if (format) body.format = format;
const response = await this.client.post("/pages/info", body);
const response = await this.client.post("/pages/info", { pageId });
return response.data?.data ?? response.data;
}
-7
View File
@@ -93,13 +93,6 @@ export function PagesMixin<TBase extends GConstructor<DocmostClientContext>>(Bas
const buildForm = () => {
const form = new FormData();
form.append("spaceId", spaceId);
// #502: this is an AGENT-authored body (plain prose / config), so tell the
// server import path to run the markdown importer with the two layered
// extensions OFF — a `$…$` span stays literal text (real math via
// `update_page_json`) and a schemeless `www.host`/email is not autolinked
// (an explicit `https://…` still links). A human file upload never sends
// this field, so human imports keep math + autolink ON.
form.append("disableMarkdownExtensions", "true");
form.append("file", fileContent, {
filename: `${title || "import"}.md`,
contentType: "text/markdown",
+2 -28
View File
@@ -72,7 +72,7 @@ export interface IReadMixin {
getTree(spaceId: string, rootPageId?: string, maxDepth?: number): any;
getPageContext(pageId: string): any;
listSidebarPages(spaceId: string, pageId?: string): any;
getPage(pageId: string, format?: "markdown" | "text"): any;
getPage(pageId: string): any;
getPageJson(pageId: string): any;
getOutline(pageId: string): any;
getNode(pageId: string, nodeId: string, format?: "markdown" | "json"): any;
@@ -420,34 +420,8 @@ export function ReadMixin<TBase extends GConstructor<DocmostClientContext>>(Base
return convertProseMirrorToMarkdown(content, options);
}
async getPage(pageId: string, format: "markdown" | "text" = "markdown") {
async getPage(pageId: string) {
await this.ensureAuthenticated();
// #502 `format:"text"`: a flat, deterministic, machine-diffable rendering
// (block-per-line, inline marks/comment anchors dropped, non-text nodes ->
// stable placeholders like `[image]` / `[table RxC]`). The server produces
// it via its `jsonToText` path (the SAME serializer that feeds search), so
// there is no second serializer here — we just request it and pass the
// string through. Distinct from the markdown default: no PM->markdown walk,
// no markdown conversion cache, and no `{{SUBPAGES}}` substitution (the text
// renderer emits no such placeholder). Use it to diff a page you wrote as a
// config/prose against what was stored.
if (format === "text") {
const textData = await this.getPageRaw(pageId, "text");
let subpages: any[] = [];
try {
subpages = await this.listSidebarPages(textData.spaceId, textData.id);
} catch (e: any) {
console.warn("Failed to fetch subpages:", e);
}
const textContent =
typeof textData.content === "string" ? textData.content : "";
return {
data: filterPage(textData, textContent, subpages),
success: true,
};
}
const resultData = await this.getPageRaw(pageId);
// Agent read: hide resolved-comment anchors so the agent sees only active
+2 -28
View File
@@ -14,7 +14,6 @@ import {
markdownToProseMirror,
normalizeAgentMarkdown,
} from "@docmost/prosemirror-markdown";
import type { MarkdownImportOptions } from "@docmost/prosemirror-markdown";
import { docmostExtensions, docmostSchema } from "./docmost-schema.js";
import { withPageLock } from "./page-lock.js";
import type { PageId } from "./page-id.js";
@@ -112,31 +111,16 @@ global.WebSocket = WebSocket;
* horizontalRule the serializer emitted, and stripping it would silently drop the
* page's leading content (#493 review). The front-matter strip stays on the
* server FILE-import boundary only (`normalizeForeignMarkdown`).
*
* #502 IMPORTANT — the two layered markdown extensions (`$…$` math, schemeless
* fuzzy autolink) are NOT decided here; they are the CALLER's choice via
* `options`, because the two callers of this wrapper need OPPOSITE behavior:
* - AGENT-authored plain markdown (`updatePageMarkdown`) → both OFF, so a
* `$…$` config span stays literal and a bare `www.host` is not autolinked
* (real math is authored via `update_page_json`).
* - FULL-FILE round-trip import (`import_page_markdown`, #328 lossless) →
* DEFAULTS (both ON), because the exporter serializes a math node as readable
* `$x^2$`; re-importing with math OFF would degrade it to literal text and
* BREAK the lossless export→import pair.
* So `options` DEFAULTS to `undefined` → the package importer's defaults (ON),
* which is the safe round-trip behavior; the agent-write caller opts OUT
* explicitly. (The fragment path `importMarkdownFragment` opts out on its own.)
*/
export async function markdownToProseMirrorCanonical(
markdownContent: string,
options?: MarkdownImportOptions,
): Promise<any> {
// #419: normalize + merge glyph-forked footnote definitions BEFORE
// canonicalizing, so the canonicalizer re-hangs references and drops the
// now-orphaned duplicate definitions.
return canonicalizeFootnotes(
normalizeAndMergeFootnotes(
await markdownToProseMirror(normalizeAgentMarkdown(markdownContent), options),
await markdownToProseMirror(normalizeAgentMarkdown(markdownContent)),
),
);
}
@@ -374,17 +358,7 @@ export async function updatePageContentRealtime(
): Promise<MutationResult> {
// PAGE write: canonicalize footnotes (markdown import builds the bottom list in
// definition order; numbering is reference-ordered).
//
// #502: this is the AGENT-authored `updatePageMarkdown` body — plain prose /
// config — so the two layered markdown extensions are turned OFF: a `$…$` span
// stays literal text (real math via `update_page_json`) and a SCHEMELESS
// `www.host`/email is not autolinked (an explicit `https://…` still links).
// Contrast `import_page_markdown`, which keeps DEFAULTS for the #328 lossless
// round-trip.
const tiptapJson = await markdownToProseMirrorCanonical(markdownContent, {
parseMath: false,
fuzzyLinkify: false,
});
const tiptapJson = await markdownToProseMirrorCanonical(markdownContent);
return await mutatePageContent(
pageId,
collabToken,
+1 -11
View File
@@ -135,17 +135,7 @@ export interface MarkdownFragment {
export async function importMarkdownFragment(
markdown: string,
): Promise<MarkdownFragment> {
// #502: the fragment path is an MCP agent WRITE (patch_node/insert_node
// markdown), so it uses the SAME extensions-OFF importer options as the
// full-page write (markdownToProseMirrorCanonical): a `$…$` span stays literal
// and a schemeless domain/email is not autolinked. This keeps a block written
// via markdown canonically identical to the same content in a full-page write
// (no "second canon"). Explicit `https://…` links and block structure are
// unaffected.
const doc = await markdownToProseMirror(markdown, {
parseMath: false,
fuzzyLinkify: false,
});
const doc = await markdownToProseMirror(markdown);
const content: any[] = Array.isArray(doc?.content) ? doc.content : [];
const blocks: any[] = [];
+18 -52
View File
@@ -469,11 +469,7 @@ export const SHARED_TOOL_SPECS = {
'(markdown). The fragment may be SEVERAL blocks (a "1 → N" splice: rewrite a ' +
'whole section in one call) — the first block inherits this block id, the ' +
'rest get fresh ids. `^[...]` footnotes are supported (their definitions ' +
"merge into the page's footnote list). Markdown is taken LITERALLY — " +
'`$...$`/`$$...$$` are NOT parsed as math and schemeless `www.host` / bare ' +
'emails are NOT auto-linked (an explicit `https://` URL still links); for a ' +
'real formula pass a `mathInline`/`mathBlock` ProseMirror node via `node` ' +
'(or updatePageJson). REJECTED when the target is a table ' +
"merge into the page's footnote list). REJECTED when the target is a table " +
'cell with attributes markdown cannot represent (merged/colored/fixed-width) ' +
'— use the table tools or `node`. ' +
'`node` (for precise attr/mark work): a raw ProseMirror node, e.g. a ' +
@@ -539,10 +535,6 @@ export const SHARED_TOOL_SPECS = {
'Provide EXACTLY ONE of `markdown` or `node`. ' +
'`markdown` (RECOMMENDED): a canonical markdown fragment — may be SEVERAL ' +
'blocks, inserted in order at the anchor; `^[...]` footnotes supported. ' +
'Markdown is taken LITERALLY — `$...$`/`$$...$$` are NOT parsed as math and ' +
'schemeless `www.host` / bare emails are NOT auto-linked (an explicit ' +
'`https://` URL still links); for a real formula pass a ' +
'`mathInline`/`mathBlock` ProseMirror node via `node` (or updatePageJson). ' +
'`node` (for precise attr/mark work OR table structure): a raw ProseMirror ' +
'node. Table structure is JSON-only (not expressible in markdown): to add a ' +
'tableRow, pass a tableRow node with position before/after and anchor INSIDE ' +
@@ -921,46 +913,28 @@ export const SHARED_TOOL_SPECS = {
inAppKey: 'getPage',
writeClass: 'readOnly',
description:
'Fetch a single page by its id. Returns the page title and its content. ' +
'format:"markdown" (DEFAULT) returns canonical Markdown (round-trips text ' +
'and block structure), sufficient for text edits; use the page-JSON read ' +
'tool only when you need what Markdown cannot carry. The Markdown drops ' +
'exactly: (1) block ids (not visible in Markdown); (2) resolved-comment ' +
'anchors (hidden here; only active <span data-comment-id> anchors remain); ' +
'(3) a fixed set of attributes with no Markdown representation — table-cell ' +
'colspan/rowspan/colwidth/backgroundColor/backgroundColorName, ' +
'heading/paragraph indent, callout.icon, orderedList.type, and link ' +
'internal/target/rel/class. Inline <span data-comment-id> tags in the ' +
'markdown are comment highlight anchors — treat them as markup, not page ' +
'text. format:"text" returns a FLAT, DETERMINISTIC plain-text rendering ' +
'for machine diffing: one line per block, ALL inline marks/formatting and ' +
'comment anchors dropped, a hardBreak is a newline, and non-text nodes ' +
'become STABLE placeholders — an image is "[image]" and a table is ' +
'"[table RxC]" (R rows x C columns). This output is stable across versions ' +
'(pinned by a snapshot test); use it to diff a config/prose you wrote ' +
'against what was stored (an empty diff means it was stored verbatim).',
'Fetch a single page as Markdown by its id. Returns the page title and ' +
'its Markdown content. The converter is canonical (round-trips text and ' +
'block structure), so this is sufficient for text edits; use the ' +
'page-JSON read tool only when you need what Markdown cannot carry. The ' +
'Markdown drops exactly: (1) block ids (not visible in Markdown); ' +
'(2) resolved-comment anchors (hidden here; only active <span ' +
'data-comment-id> anchors remain); (3) a fixed set of attributes with no ' +
'Markdown representation — table-cell colspan/rowspan/colwidth/' +
'backgroundColor/backgroundColorName, heading/paragraph indent, ' +
'callout.icon, orderedList.type, and link internal/target/rel/class. ' +
'Inline <span data-comment-id> tags in the markdown are comment highlight ' +
'anchors — treat them as markup, not page text.',
tier: 'core',
catalogLine:
'getPage — fetch a page by its id (format:"markdown" default, or "text" for a flat machine-diffable read).',
catalogLine: 'getPage — fetch a page as Markdown by its id.',
// Reconciled: MCP's stricter .min(1) kept; in-app's more-informative
// "(or slugId)" describe kept.
buildShape: (z) => ({
pageId: z.string().min(1).describe('The id (or slugId) of the page.'),
format: z
.enum(['markdown', 'text'])
.optional()
.describe(
'Output format: "markdown" (default, canonical round-trippable) or ' +
'"text" (flat deterministic plain text for machine diffing).',
),
}),
// MCP wraps the raw `{ data, success }` as JSON. The in-app host instead
// projects a token-efficient `{ title, markdown }` (its long-standing shape).
execute: (client, { pageId, format }) =>
client.getPage(
pageId as string,
(format as 'markdown' | 'text' | undefined) ?? 'markdown',
),
execute: (client, { pageId }) => client.getPage(pageId as string),
inAppExecute: async (client, { pageId }) => {
// getPage(pageId) -> { data: filterPage(page, markdown), success }.
const result = (await client.getPage(pageId as string)) as {
@@ -1103,12 +1077,8 @@ export const SHARED_TOOL_SPECS = {
description:
'Create a new page with a Markdown body in a space, optionally under a ' +
'parent page (omit parentPageId to create at the space root). Returns ' +
'the new page id and title. Body text is taken LITERALLY — ' +
'`$...$`/`$$...$$` are NOT parsed into a math formula and schemeless ' +
'`www.host` / bare emails are NOT auto-linked (an explicit `https://` URL ' +
'still links); for a real formula use updatePageJson with ' +
'`mathInline`/`mathBlock` nodes instead. Reversible: a page can be moved ' +
'to trash later.',
'the new page id and title. Reversible: a page can be moved to trash ' +
'later.',
tier: 'deferred',
catalogLine: 'createPage — create a new page with a Markdown body in a space.',
// Reconciled schema DRIFT: the MCP copy pinned `content` to .min(1) while
@@ -1367,11 +1337,7 @@ export const SHARED_TOOL_SPECS = {
'title). The whole body is re-imported from the markdown (block ids ' +
'regenerate — for surgical or id-preserving edits use the find/replace, ' +
'node-patch or page-JSON tools instead). Docmost-flavoured markdown is ' +
'parsed, including `^[...]` inline footnotes. Text is taken LITERALLY — ' +
'`$...$`/`$$...$$` are NOT parsed into a math formula and schemeless ' +
'`www.host` / bare emails are NOT auto-linked (an explicit `https://` URL ' +
'still links); for a real formula use updatePageJson with ' +
'`mathInline`/`mathBlock` nodes instead. Reversible: the previous ' +
'parsed, including `^[...]` inline footnotes. Reversible: the previous ' +
'version is kept in page history.',
tier: 'deferred',
catalogLine:
@@ -1,83 +0,0 @@
// #502 BLOCKER 1 (client half): the MCP `createPage` tool builds its body as an
// AGENT-authored markdown file and POSTs it to the server `/pages/import`
// endpoint. It must send the `disableMarkdownExtensions=true` multipart field so
// the SERVER importer runs with math + fuzzy-autolink OFF (a human file upload
// omits the field and keeps them ON). This mock http server captures the raw
// multipart body and asserts the field is present.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { DocmostClient } from "../../build/client.js";
function sendJson(res, status, obj, extra = {}) {
res.writeHead(status, { "Content-Type": "application/json", ...extra });
res.end(JSON.stringify(obj));
}
function readRaw(req) {
return new Promise((resolve) => {
const chunks = [];
req.on("data", (c) => chunks.push(c));
req.on("end", () => resolve(Buffer.concat(chunks).toString("utf-8")));
});
}
const openServers = [];
after(async () => {
await Promise.all(openServers.map((s) => new Promise((r) => s.close(r))));
});
const NEW_ID = "00000000-0000-4000-8000-000000000042";
const SPACE = "00000000-0000-4000-8000-0000000000aa";
function spawn(state) {
return new Promise((resolve) => {
const server = http.createServer(async (req, res) => {
const raw = await readRaw(req);
if (req.url === "/api/auth/login") {
return sendJson(res, 200, { success: true }, { "Set-Cookie": "authToken=t; Path=/; HttpOnly" });
}
if (req.url === "/api/pages/import") {
state.importBody = raw; // the raw multipart payload
return sendJson(res, 200, { data: { id: NEW_ID } });
}
if (req.url === "/api/pages/update") {
return sendJson(res, 200, { data: { id: NEW_ID } });
}
if (req.url === "/api/pages/info") {
return sendJson(res, 200, {
data: {
id: NEW_ID,
slugId: "slugnew1234",
title: "T",
spaceId: SPACE,
updatedAt: "2026-01-01T00:00:00Z",
content: { type: "doc", content: [{ type: "paragraph", content: [{ type: "text", text: "x" }] }] },
},
});
}
if (req.url === "/api/pages/sidebar-pages") {
return sendJson(res, 200, { data: { items: [], meta: { hasNextPage: false, nextCursor: null } } });
}
return sendJson(res, 404, { message: "not found" });
});
server.listen(0, "127.0.0.1", () => {
openServers.push(server);
resolve(`http://127.0.0.1:${server.address().port}/api`);
});
});
}
test("createPage sends disableMarkdownExtensions=true in the /pages/import multipart", async () => {
const state = {};
const baseURL = await spawn(state);
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await client.createPage("My Config", "ticket $x=1$ at www.host.com", SPACE);
assert.ok(state.importBody, "the import endpoint received a body");
// The multipart payload carries the field name and its "true" value.
assert.match(state.importBody, /name="disableMarkdownExtensions"/);
assert.match(state.importBody, /name="disableMarkdownExtensions"[\s\S]*?\r?\n\r?\ntrue\r?\n/);
// The agent body itself is still sent as the file part.
assert.match(state.importBody, /ticket \$x=1\$ at www\.host\.com/);
});
@@ -1,116 +0,0 @@
// #502 READ: the getPage tool gains a `format:"text"` mode. It requests the
// server's flat, deterministic text rendering (the server's jsonToText path — the
// SAME serializer that feeds search), passing it through unchanged. This mock
// stands up a local http server (same harness style as getpage-conversion-cache)
// and asserts end-to-end that:
// - format:"text" sends `format:"text"` in the /pages/info body and returns the
// server's text string verbatim (no client-side markdown conversion);
// - the DEFAULT (no format) still returns markdown-converted content;
// - the text read resolves page + subpages like the markdown read.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { DocmostClient } from "../../build/client.js";
function readBody(req) {
return new Promise((resolve) => {
let raw = "";
req.on("data", (c) => (raw += c));
req.on("end", () => resolve(raw));
});
}
function sendJson(res, status, obj, extra = {}) {
res.writeHead(status, { "Content-Type": "application/json", ...extra });
res.end(JSON.stringify(obj));
}
const openServers = [];
after(async () => {
await Promise.all(openServers.map((s) => new Promise((r) => s.close(r))));
});
const PAGE_UUID = "00000000-0000-4000-8000-000000000010";
const SPACE_UUID = "00000000-0000-4000-8000-0000000000aa";
const CHILD_UUID = "00000000-0000-4000-8000-0000000000bb";
// The deterministic text the SERVER's jsonToText would produce for this page.
const SERVER_TEXT = "Title line\nsecond block\n[image]\n[table 2x3]";
function makeDoc(text) {
return { type: "doc", content: [{ type: "paragraph", content: [{ type: "text", text }] }] };
}
function spawn(state) {
return new Promise((resolve) => {
const server = http.createServer(async (req, res) => {
const body = await readBody(req);
if (req.url === "/api/auth/login") {
return sendJson(res, 200, { success: true }, { "Set-Cookie": "authToken=t; Path=/; HttpOnly" });
}
if (req.url === "/api/pages/info") {
const parsed = body ? JSON.parse(body) : {};
state.lastInfoBody = parsed;
// Emulate the server: when format:"text" is requested, `content` is the
// flat text string; otherwise it is the raw ProseMirror JSON.
const content = parsed.format === "text" ? SERVER_TEXT : makeDoc("Title line");
return sendJson(res, 200, {
success: true,
data: {
id: PAGE_UUID,
slugId: "slug123456",
title: "Text Page",
parentPageId: null,
spaceId: SPACE_UUID,
updatedAt: "2026-01-01T00:00:00Z",
content,
},
});
}
if (req.url === "/api/pages/sidebar-pages") {
return sendJson(res, 200, {
success: true,
data: {
items: [{ id: CHILD_UUID, title: "Child", hasChildren: false }],
meta: { hasNextPage: false, nextCursor: null },
},
});
}
return sendJson(res, 404, { message: "not found" });
});
server.listen(0, "127.0.0.1", () => {
openServers.push(server);
resolve(`http://127.0.0.1:${server.address().port}/api`);
});
});
}
function makeClient(baseURL) {
return new DocmostClient({ apiUrl: baseURL, getToken: async () => "access" });
}
test('getPage(format:"text") requests text and returns the server text verbatim', async () => {
const state = {};
const client = makeClient(await spawn(state));
const result = await client.getPage(PAGE_UUID, "text");
assert.equal(state.lastInfoBody.format, "text", "the info request carried format:text");
assert.equal(result.success, true);
assert.equal(result.data.content, SERVER_TEXT, "returns the server's flat text unchanged");
// Deterministic placeholders are surfaced to the agent.
assert.ok(result.data.content.includes("[image]"));
assert.ok(result.data.content.includes("[table 2x3]"));
// No client-side markdown artifacts leaked in.
assert.ok(!result.data.content.includes("{{SUBPAGES}}"));
// Subpages still resolve for context.
assert.deepEqual(result.data.subpages, [{ id: CHILD_UUID, title: "Child" }]);
});
test('getPage default (no format) still returns markdown, not text', async () => {
const state = {};
const client = makeClient(await spawn(state));
const result = await client.getPage(PAGE_UUID);
assert.equal(state.lastInfoBody.format, undefined, "default read sends no format");
assert.ok(result.data.content.includes("Title line"), "markdown content converted client-side");
assert.notEqual(result.data.content, SERVER_TEXT);
});
@@ -1,181 +0,0 @@
// #502 caller-WIRING: end-to-end through a live Hocuspocus collab stack (same
// harness style as markdown-patch-insert), driving the REAL client methods and
// reading the persisted document back, so the test proves each write tool passes
// the RIGHT importer options — not just that the shared wrapper can:
// - updatePageMarkdown (client.updatePage) -> extensions OFF (`$…$` literal, www not linked)
// - import_page_markdown (client.importPageMarkdown) -> DEFAULTS (`$x^2$` -> math node, #328)
// Mutating either caller's option flips the matching assertion (see the coder's
// mutation note), so this file guards the wiring, not only the wrapper.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { WebSocketServer } from "ws";
import { Hocuspocus } from "@hocuspocus/server";
import { DocmostClient } from "../../build/client.js";
import { buildYDoc } from "../../build/lib/collaboration.js";
import { serializeDocmostMarkdown } from "@docmost/prosemirror-markdown";
const PAGE = "11111111-1111-4111-8111-111111111111";
function findAll(node, type, acc = []) {
if (!node || typeof node !== "object") return acc;
if (node.type === type) acc.push(node);
if (Array.isArray(node.content)) for (const c of node.content) findAll(c, type, acc);
return acc;
}
function allText(node, acc = []) {
if (!node || typeof node !== "object") return acc.join("");
if (node.type === "text" && typeof node.text === "string") acc.push(node.text);
if (Array.isArray(node.content)) for (const c of node.content) allText(c, acc);
return acc.join("");
}
function hasLink(node) {
return findAll(node, "text").some((t) => t.marks?.some((m) => m.type === "link"));
}
function fragmentToJson(frag) {
const decodeNode = (el) => {
if (el.constructor.name === "YXmlText") {
const delta = el.toDelta();
return delta.map((d) => {
const node = { type: "text", text: d.insert };
if (d.attributes && Object.keys(d.attributes).length) {
node.marks = Object.entries(d.attributes).map(([type, attrs]) =>
attrs && typeof attrs === "object" && Object.keys(attrs).length
? { type, attrs }
: { type },
);
}
return node;
});
}
const node = { type: el.nodeName };
const attrs = el.getAttributes();
if (attrs && Object.keys(attrs).length) node.attrs = attrs;
const children = [];
for (const child of el.toArray()) {
const decoded = decodeNode(child);
if (Array.isArray(decoded)) children.push(...decoded);
else children.push(decoded);
}
if (children.length) node.content = children;
return node;
};
const content = [];
for (const child of frag.toArray()) content.push(decodeNode(child));
return { type: "doc", content };
}
const openStacks = [];
after(async () => {
await Promise.all(
openStacks.map(
({ server, hocuspocus }) =>
new Promise((resolve) => {
server.close(() => {
Promise.resolve(hocuspocus.destroy?.()).finally(resolve);
});
}),
),
);
});
async function spawnCollabStack(seedDoc) {
const state = { lastDoc: null };
const hocuspocus = new Hocuspocus({
quiet: true,
async onLoadDocument() {
return buildYDoc(seedDoc);
},
async onChange(data) {
try {
state.lastDoc = fragmentToJson(data.document.getXmlFragment("default"));
} catch {
/* ignore teardown-race decode errors */
}
},
});
const wss = new WebSocketServer({ noServer: true });
const server = http.createServer((req, res) => {
let raw = "";
req.on("data", (c) => (raw += c));
req.on("end", () => {
if (req.url === "/api/auth/login") {
res.writeHead(200, {
"Content-Type": "application/json",
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
});
res.end(JSON.stringify({ success: true }));
return;
}
if (req.url === "/api/auth/collab-token") {
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ data: { token: "collab-jwt" } }));
return;
}
res.writeHead(404, { "Content-Type": "application/json" });
res.end(JSON.stringify({ message: "not found" }));
});
});
server.on("upgrade", (request, socket, head) => {
if (!request.url || !request.url.startsWith("/collab")) {
socket.destroy();
return;
}
wss.handleUpgrade(request, socket, head, (ws) => {
hocuspocus.handleConnection(ws, request);
});
});
const baseURL = await new Promise((resolve) => {
server.listen(0, "127.0.0.1", () => {
resolve(`http://127.0.0.1:${server.address().port}/api`);
});
});
openStacks.push({ server, hocuspocus });
return { state, baseURL };
}
function seed() {
return {
type: "doc",
content: [
{ type: "paragraph", attrs: { id: "p-id" }, content: [{ type: "text", text: "seed" }] },
],
};
}
test("updatePageMarkdown wiring: extensions OFF — `$…$` literal, www not linked, https links", async () => {
const { state, baseURL } = await spawnCollabStack(seed());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
await client.updatePage(PAGE, "cfg $x=1$ and www.host.com and https://ex.com");
assert.ok(state.lastDoc, "a document was persisted");
assert.equal(findAll(state.lastDoc, "mathInline").length, 0, "no phantom math from an agent write");
assert.ok(allText(state.lastDoc).includes("$x=1$"), "literal dollars preserved");
assert.ok(allText(state.lastDoc).includes("www.host.com"), "bare domain preserved as text");
// The explicit https URL still links (only the schemeless autolink is off).
const links = findAll(state.lastDoc, "text").filter((t) =>
t.marks?.some((m) => m.type === "link"),
);
assert.ok(links.some((t) => t.text?.includes("ex.com")), "explicit https still links");
});
test("import_page_markdown wiring: DEFAULTS — exported `$x^2$` re-imports AS a math node (#328)", async () => {
const { state, baseURL } = await spawnCollabStack(seed());
const client = new DocmostClient(baseURL, "e@x.com", "pw");
// A self-contained docmost markdown file whose body carries a math span (as the
// exporter emits it). import_page_markdown must import it with math ON.
const meta = { version: 1, pageId: PAGE, slugId: "s", title: "T", spaceId: "sp", parentPageId: null };
const fullMd = serializeDocmostMarkdown(meta, "energy is $x^2$ here", []);
await client.importPageMarkdown(PAGE, fullMd);
assert.ok(state.lastDoc, "a document was persisted");
assert.equal(
findAll(state.lastDoc, "mathInline").length,
1,
"the lossless round-trip is intact: math survives import_page_markdown",
);
});
@@ -1,108 +0,0 @@
// #502: the two layered markdown extensions (`$…$` math, schemeless fuzzy
// autolink) are decided PER CALLER of the shared write importer, not hardcoded in
// the wrapper — because the callers need OPPOSITE behavior:
// - AGENT-authored plain markdown (updatePageMarkdown, patch_node/insert_node)
// -> extensions OFF: a `$…$` config span stays literal, a bare `www.host` is
// not autolinked, an explicit `https://…` still links.
// - FULL-FILE round-trip import (import_page_markdown, #328 lossless) ->
// DEFAULTS (extensions ON): an exported math node's `$x^2$` re-imports AS a
// math node, so the export→import pair is NOT broken.
// This unit file pins the importer contract at each of those semantics. The
// caller-WIRING (that each client method passes the right options) is pinned by
// the collab-backed test in mock/write-path-extensions-wiring.test.mjs.
import { test } from "node:test";
import assert from "node:assert/strict";
import { markdownToProseMirrorCanonical } from "../../build/lib/collaboration.js";
import { importMarkdownFragment } from "../../build/lib/markdown-fragment.js";
import {
markdownToProseMirror,
convertProseMirrorToMarkdown,
} from "@docmost/prosemirror-markdown";
const OFF = { parseMath: false, fuzzyLinkify: false };
function findAll(node, type, acc = []) {
if (!node || typeof node !== "object") return acc;
if (node.type === type) acc.push(node);
if (Array.isArray(node.content)) for (const c of node.content) findAll(c, type, acc);
return acc;
}
function allText(node, acc = []) {
if (!node || typeof node !== "object") return acc.join("");
if (node.type === "text" && typeof node.text === "string") acc.push(node.text);
if (Array.isArray(node.content)) for (const c of node.content) allText(c, acc);
return acc.join("");
}
function hasLink(node) {
return findAll(node, "text").some((t) => t.marks?.some((m) => m.type === "link"));
}
// --- AGENT-write semantics (updatePageMarkdown): extensions OFF -----------------
test("agent-write importer (OFF): `$…$` config stays literal, no math node", async () => {
const doc = await markdownToProseMirrorCanonical("export A=$FOO and B=$BAR done", OFF);
assert.equal(findAll(doc, "mathInline").length, 0);
assert.equal(allText(doc), "export A=$FOO and B=$BAR done");
});
test("agent-write importer (OFF): schemeless www NOT linked; explicit https STILL linked", async () => {
const bare = await markdownToProseMirrorCanonical("see www.example.com here", OFF);
assert.equal(hasLink(bare), false);
assert.equal(allText(bare), "see www.example.com here");
const explicit = await markdownToProseMirrorCanonical("see https://example.com here", OFF);
assert.equal(hasLink(explicit), true);
});
test("agent-write importer (OFF): heading + list structure preserved", async () => {
const doc = await markdownToProseMirrorCanonical("## Heading\n\n- one\n- two", OFF);
assert.equal(findAll(doc, "heading").length, 1);
assert.equal(findAll(doc, "bulletList").length, 1);
});
test("fragment importer (patch_node/insert_node) is OFF: `$x=1$` literal, https links", async () => {
const { blocks } = await importMarkdownFragment("cfg $x=1$ and https://ex.com");
const doc = { type: "doc", content: blocks };
assert.equal(findAll(doc, "mathInline").length, 0);
assert.equal(hasLink(doc), true);
assert.ok(allText(doc).includes("$x=1$"));
});
// --- FULL-FILE import semantics (import_page_markdown): DEFAULTS (math ON) ------
test("import_page_markdown importer (DEFAULTS): `$x^2$` DOES create a math node", async () => {
// markdownToProseMirrorCanonical WITHOUT options == what import_page_markdown
// passes. Math must survive (this is the REAL importer, not the package default).
const doc = await markdownToProseMirrorCanonical("$x^2$");
assert.equal(findAll(doc, "mathInline").length, 1);
assert.equal(findAll(doc, "mathInline")[0].attrs.text, "x^2");
});
test("import_page_markdown (DEFAULTS): #328 lossless export->import keeps math (round-trip)", async () => {
// The exporter serializes a math node as readable `$x^2$`; re-importing through
// the REAL import_page_markdown importer (markdownToProseMirrorCanonical, no
// options) must yield a math node again and be byte-stable. Under the BUGGY code
// (canonical hardcoded parseMath:false) doc2 would be literal text -> this test
// would REDDEN.
const source = {
type: "doc",
content: [{ type: "paragraph", content: [{ type: "mathInline", attrs: { text: "x^2" } }] }],
};
const md1 = convertProseMirrorToMarkdown(source);
const doc2 = await markdownToProseMirrorCanonical(md1); // real import path, defaults
assert.equal(findAll(doc2, "mathInline").length, 1, "math survives the round-trip import");
const md2 = convertProseMirrorToMarkdown(doc2);
assert.equal(md2, md1, "export is byte-stable across the round-trip");
});
test("import_page_markdown (DEFAULTS): a schemeless www IS autolinked", async () => {
const doc = await markdownToProseMirrorCanonical("see www.example.com here");
assert.equal(hasLink(doc), true);
});
// --- Package default importer (editor/file-import/git-sync) is UNCHANGED --------
test("PACKAGE default importer keeps math ON (editor/file/git-sync path)", async () => {
const doc = await markdownToProseMirror("$x^2$");
assert.equal(findAll(doc, "mathInline").length, 1);
});
@@ -25,7 +25,6 @@ export {
markdownToProseMirror,
markdownToProseMirrorSync,
} from "./markdown-to-prosemirror.js";
export type { MarkdownImportOptions } from "./markdown-to-prosemirror.js";
// Foreign-markdown normalizer (#493): the input-liberal pre-pass that rewrites
// GFM `[^id]` reference footnotes to canonical inline `^[body]`. Two variants:
@@ -7,7 +7,7 @@
* natively through the collab gateway, so no websocket/Yjs write-path lives
* here.
*/
import { Marked, Tokenizer } from "marked";
import { Marked } from "marked";
import { parseHtmlDocument, generateJsonWith } from "./dom-parser.js";
import type { TokenizerExtension, RendererExtension } from "marked";
import { docmostExtensions } from "./docmost-schema.js";
@@ -230,88 +230,19 @@ function escapeFootnoteAttr(value: string): string {
return String(value).replace(/&/g, "&amp;").replace(/"/g, "&quot;");
}
/**
* Options controlling which of the two *layered* markdown extensions the
* canonical importer applies. Both default to `true`, so the human editor,
* file-import and git-sync paths keep their existing behavior byte-for-byte;
* ONLY the MCP markdown-write path opts OUT (see #502).
*
* The extensions are optional because they are the SOURCE of two silent
* corruptions when an AGENT writes plain prose/config as markdown:
* - `parseMath`: a `$…$` span becomes a `mathInline` node. An agent writing a
* config like `export A=$FOO and B=$BAR` gets `$FOO and B=$` silently turned
* into a formula. With `parseMath:false` the `$` stays literal text (real
* formulas go through `update_page_json` with `mathInline`/`mathBlock`).
* - `fuzzyLinkify`: marked's GFM autolinker turns a SCHEMELESS `www.foo.com`
* (and email) into a link. With `fuzzyLinkify:false` a schemeless domain
* stays literal text; an EXPLICIT `https://…` STILL becomes a link (only the
* fuzzy, schemeless autolink is suppressed).
*/
export interface MarkdownImportOptions {
/** Apply the `$…$` / `$$…$$` math extensions (default true). */
parseMath?: boolean;
/** Apply marked's GFM schemeless (fuzzy) autolinker (default true). */
fuzzyLinkify?: boolean;
}
/**
* `fuzzyLinkify:false` override of marked's built-in GFM `url` inline tokenizer.
*
* The stock tokenizer autolinks THREE shapes: a schemeless `www.host` domain, a
* bare email, and an EXPLICIT `scheme://…` URL. #502 wants only the last kept —
* a schemeless domain/email an agent typed as prose must stay literal text, but
* a deliberate `https://…` still links. We delegate to the original tokenizer
* and, when it matched, DROP the token (returning `undefined`, so the run stays
* literal text) unless the matched RAW text carries an explicit `scheme:` prefix.
* `www.`/email matches have no scheme in their raw text, so they are dropped;
* `https://…`/`ftp://…` keep their link.
*/
const SCHEME_PREFIX_RE = /^[a-zA-Z][a-zA-Z0-9+.-]*:/;
const noFuzzyUrlTokenizer = {
url(this: any, src: string) {
const token = Tokenizer.prototype.url.call(this, src) as any;
if (!token) return token;
// Keep only explicit-scheme URLs; schemeless `www.`/email matches -> literal.
return SCHEME_PREFIX_RE.test(token.raw) ? token : undefined;
},
};
/**
* Build a dedicated `marked` instance for a given extension combination: default
* (GFM) options plus the `==` highlight and `^[…]` footnote inline extensions
* ALWAYS, the `$…$`/`$$…$$` math extensions only when `parseMath`, and the
* schemeless-autolink suppressor only when `!fuzzyLinkify`. Built on a private
* `Marked` instance so nothing leaks into the global `marked` singleton.
*/
function buildMarkedInstance(parseMath: boolean, fuzzyLinkify: boolean): Marked {
const extensions: (TokenizerExtension & RendererExtension)[] = [
// Dedicated marked instance: default (GFM) options plus the `==` highlight
// inline extension, the `$…$` / `$$…$$` math extensions (#293 canon #6), and the
// `^[…]` inline-footnote extension (#293 canon #2). Constructed once at module
// load so the extensions are registered exactly once and never mutate the global
// `marked` singleton.
const markedInstance = new Marked().use({
extensions: [
highlightMarkExtension,
mathInlineExtension,
mathBlockExtension,
footnoteInlineExtension,
];
if (parseMath) {
extensions.push(mathInlineExtension, mathBlockExtension);
}
const instance = new Marked().use({ extensions });
if (!fuzzyLinkify) {
instance.use({ tokenizer: noFuzzyUrlTokenizer as any });
}
return instance;
}
// Memoize one instance per (parseMath, fuzzyLinkify) combination so the
// extensions are registered exactly once per combo (never on the global
// singleton). The default `(true, true)` instance preserves the pre-#502
// behavior exactly for the editor/file-import/git-sync paths.
const markedInstanceCache = new Map<string, Marked>();
function getMarkedInstance(parseMath: boolean, fuzzyLinkify: boolean): Marked {
const key = `${parseMath}:${fuzzyLinkify}`;
let instance = markedInstanceCache.get(key);
if (!instance) {
instance = buildMarkedInstance(parseMath, fuzzyLinkify);
markedInstanceCache.set(key, instance);
}
return instance;
}
],
});
// NOTE: this module no longer installs a module-level `global.window`/`document`
// jsdom shim. The HTML->DOM passes below (bridgeTaskLists / applyCommentDirectives
@@ -370,7 +301,7 @@ const CODE_FENCE_RE = /^(\s*)(`{3,}|~{3,})/;
// preprocess is sync. Keeping it sync lets a sync converter entry
// (`markdownToProseMirrorSync`, used by the client's chat renderer which must
// stay synchronous) share this exact logic with the async entry.
function preprocessCallouts(markdown: string, markedInstance: Marked): string {
function preprocessCallouts(markdown: string): string {
// Defensive cap: skip preprocessing for pathologically large inputs.
if (markdown.length > MAX_CALLOUT_PREPROCESS_BYTES) {
return markdown;
@@ -1024,7 +955,7 @@ function applyCommentDirectives(html: string): string {
// sups stay inert) rather than hang.
const MAX_FOOTNOTE_ROUNDS = 10000;
function assembleFootnotes(html: string, markedInstance: Marked): string {
function assembleFootnotes(html: string): string {
// Cheap early-out: nothing carries a footnote body -> nothing to assemble.
if (!html.includes("data-fn-text")) return html;
const document = parseHtmlDocument(html);
@@ -1150,18 +1081,8 @@ function stripEmptyParagraphs(node: any): any {
* for every existing Node consumer). A sync entry is REQUIRED by the client's
* chat renderer, which runs inside a React render/useMemo and cannot await.
*/
export function markdownToProseMirrorSync(
markdownContent: string,
options?: MarkdownImportOptions,
): any {
// Select the marked instance for this call's extension combination. Defaults
// (math + fuzzy autolink ON) preserve the editor/file-import/git-sync paths;
// the MCP markdown-write path passes both false (#502).
const markedInstance = getMarkedInstance(
options?.parseMath ?? true,
options?.fuzzyLinkify ?? true,
);
const withCallouts = preprocessCallouts(markdownContent, markedInstance);
export function markdownToProseMirrorSync(markdownContent: string): any {
const withCallouts = preprocessCallouts(markdownContent);
const html = markedInstance.parse(withCallouts) as string;
// Materialize comment directives (#293 #9 attached textAlign; #5 standalone
// subpages/pageBreak) while the comment nodes still exist, before generateJSON
@@ -1170,7 +1091,7 @@ export function markdownToProseMirrorSync(
// #293 canon #2: assemble the doc-level footnote list from the `<sup
// data-fn-text>` markers (from `^[…]` or the raw-HTML column form) before
// generateJSON, so references + definitions materialize into the schema model.
const withFootnotes = assembleFootnotes(withAttrs, markedInstance);
const withFootnotes = assembleFootnotes(withAttrs);
const bridged = bridgeTaskLists(withFootnotes);
const doc = generateJsonWith(bridged, docmostExtensions);
return stripEmptyParagraphs(doc);
@@ -1183,7 +1104,6 @@ export function markdownToProseMirrorSync(
*/
export async function markdownToProseMirror(
markdownContent: string,
options?: MarkdownImportOptions,
): Promise<any> {
return markdownToProseMirrorSync(markdownContent, options);
return markdownToProseMirrorSync(markdownContent);
}
@@ -1,170 +0,0 @@
import { describe, expect, it } from 'vitest';
// Import DIRECTLY from src (like math.test.ts) so we exercise the real
// converter and its module-load jsdom setup.
import { convertProseMirrorToMarkdown } from '../src/lib/markdown-converter.js';
import { markdownToProseMirror } from '../src/lib/markdown-to-prosemirror.js';
// ---------------------------------------------------------------------------
// #502: the canonical importer is parameterized with `{ parseMath, fuzzyLinkify }`
// (both DEFAULT true). The MCP markdown-WRITE path passes both false so an agent's
// plain prose/config is imported LITERALLY:
// - parseMath:false -> a `$…$` span stays literal text (no phantom mathInline)
// - fuzzyLinkify:false -> a SCHEMELESS `www.host`/email stays literal text; an
// EXPLICIT `https://…` STILL links (only the fuzzy autolink is suppressed).
// The DEFAULTS (editor/file-import/git-sync) keep math + fuzzy autolink ON, so
// this file also pins that the defaults are UNCHANGED.
// ---------------------------------------------------------------------------
const OFF = { parseMath: false, fuzzyLinkify: false } as const;
// Collect the first paragraph's inline children (the common assertion target).
function firstParaInline(doc: any): any[] {
const p = doc.content?.find((n: any) => n.type === 'paragraph');
return p?.content ?? [];
}
function findAll(node: any, type: string, acc: any[] = []): any[] {
if (!node || typeof node !== 'object') return acc;
if (node.type === type) acc.push(node);
if (Array.isArray(node.content)) {
for (const c of node.content) findAll(c, type, acc);
}
return acc;
}
// Flatten every text node's text (ignoring marks/structure) into one string.
function allText(node: any, acc: string[] = []): string {
if (!node || typeof node !== 'object') return acc.join('');
if (node.type === 'text' && typeof node.text === 'string') acc.push(node.text);
if (Array.isArray(node.content)) {
for (const c of node.content) allText(c, acc);
}
return acc.join('');
}
describe('#502 importer options — extensions OFF (MCP write path)', () => {
it('a `$…$` config span stays literal text (no mathInline node)', async () => {
const md = 'export A=$FOO and B=$BAR done';
const doc = await markdownToProseMirror(md, OFF);
expect(findAll(doc, 'mathInline')).toHaveLength(0);
expect(allText(doc)).toBe('export A=$FOO and B=$BAR done');
});
it('a real-looking `$x=1$` span stays literal text', async () => {
const doc = await markdownToProseMirror('$x=1$', OFF);
expect(findAll(doc, 'mathInline')).toHaveLength(0);
expect(allText(doc)).toBe('$x=1$');
});
it('the reported `($ticket_lifetime=2592000)` config stays literal', async () => {
const doc = await markdownToProseMirror('($ticket_lifetime=2592000)', OFF);
expect(findAll(doc, 'mathInline')).toHaveLength(0);
expect(allText(doc)).toBe('($ticket_lifetime=2592000)');
});
it('a `$$…$$` block stays literal (no mathBlock node)', async () => {
const doc = await markdownToProseMirror('$$\nx^2\n$$', OFF);
expect(findAll(doc, 'mathBlock')).toHaveLength(0);
expect(allText(doc)).toContain('x^2');
});
it('a SCHEMELESS `www.example.com` is NOT autolinked', async () => {
const doc = await markdownToProseMirror('see www.example.com here', OFF);
const inline = firstParaInline(doc);
expect(inline.some((n: any) => n.marks?.some((m: any) => m.type === 'link'))).toBe(false);
expect(allText(doc)).toBe('see www.example.com here');
});
it('a bare dotted domain `gitea.vvzvlad.xyz` stays literal text', async () => {
const doc = await markdownToProseMirror('see gitea.vvzvlad.xyz here', OFF);
expect(findAll(doc, 'text').every((t: any) => !t.marks?.some((m: any) => m.type === 'link'))).toBe(true);
});
it('an EXPLICIT `https://…` STILL becomes a link', async () => {
const doc = await markdownToProseMirror('see https://example.com here', OFF);
const linked = firstParaInline(doc).find((n: any) =>
n.marks?.some((m: any) => m.type === 'link'),
);
expect(linked?.text).toBe('https://example.com');
const link = linked.marks.find((m: any) => m.type === 'link');
expect(link.attrs.href).toBe('https://example.com');
});
it('`foo_bar_baz` is not italicized (CommonMark, unaffected by options)', async () => {
const doc = await markdownToProseMirror('foo_bar_baz', OFF);
expect(findAll(doc, 'text').some((t: any) => t.marks?.some((m: any) => m.type === 'italic' || m.type === 'em'))).toBe(false);
expect(allText(doc)).toBe('foo_bar_baz');
});
it('block STRUCTURE (headings, lists, code fence) is preserved with extensions off', async () => {
const md = '## Heading\n\n- one\n- two\n\n```\ncode $x$ here\n```';
const doc = await markdownToProseMirror(md, OFF);
expect(findAll(doc, 'heading')).toHaveLength(1);
expect(findAll(doc, 'bulletList')).toHaveLength(1);
expect(findAll(doc, 'codeBlock')).toHaveLength(1);
// The `$x$` inside the code fence never becomes math regardless.
expect(findAll(doc, 'mathInline')).toHaveLength(0);
});
});
describe('#502 importer options — DEFAULTS unchanged (editor/file/git-sync)', () => {
it('DEFAULT: `$x^2$` DOES create a mathInline node (file-import unaffected)', async () => {
const doc = await markdownToProseMirror('$x^2$');
expect(findAll(doc, 'mathInline')).toHaveLength(1);
expect(findAll(doc, 'mathInline')[0].attrs.text).toBe('x^2');
});
it('DEFAULT: a schemeless `www.example.com` IS autolinked', async () => {
const doc = await markdownToProseMirror('see www.example.com here');
const linked = firstParaInline(doc).find((n: any) =>
n.marks?.some((m: any) => m.type === 'link'),
);
expect(linked?.text).toBe('www.example.com');
});
it('DEFAULT: explicitly passing {parseMath:true, fuzzyLinkify:true} equals no-options', async () => {
const a = await markdownToProseMirror('$x^2$ and www.foo.com');
const b = await markdownToProseMirror('$x^2$ and www.foo.com', {
parseMath: true,
fuzzyLinkify: true,
});
expect(JSON.stringify(b)).toBe(JSON.stringify(a));
});
it('round-trip export->import at the PACKAGE-DEFAULT layer keeps math (file-import / #328)', async () => {
// The package DEFAULT importer is what the server file-import path uses. A
// page holding real math is exported, then re-imported with DEFAULTS (math
// ON): the mathInline survives. (The tool-level import_page_markdown round-
// trip, which goes through mcp's markdownToProseMirrorCanonical, is pinned
// authoritatively in @docmost/mcp's mcp-write-extensions-off test.)
const source = {
type: 'doc',
content: [
{ type: 'paragraph', content: [{ type: 'mathInline', attrs: { text: 'x^2' } }] },
],
};
const md1 = convertProseMirrorToMarkdown(source);
const doc2 = await markdownToProseMirror(md1); // DEFAULTS -> math on
expect(findAll(doc2, 'mathInline')).toHaveLength(1);
const md2 = convertProseMirrorToMarkdown(doc2);
expect(md2).toBe(md1); // byte-stable
});
});
describe('#502 mutation guard', () => {
// If a future change silently flipped the MCP write path back to parseMath:true,
// the OFF assertion below would go RED — this pins the discriminating behavior.
it('with parseMath:true the same span DOES become math (proves the flag drives it)', async () => {
const on = await markdownToProseMirror('$x=1$', { parseMath: true, fuzzyLinkify: false });
expect(findAll(on, 'mathInline')).toHaveLength(1);
const off = await markdownToProseMirror('$x=1$', OFF);
expect(findAll(off, 'mathInline')).toHaveLength(0);
});
it('with fuzzyLinkify:true the same www domain DOES link (proves the flag drives it)', async () => {
const on = await markdownToProseMirror('www.example.com', { parseMath: false, fuzzyLinkify: true });
expect(findAll(on, 'text').some((t: any) => t.marks?.some((m: any) => m.type === 'link'))).toBe(true);
const off = await markdownToProseMirror('www.example.com', OFF);
expect(findAll(off, 'text').some((t: any) => t.marks?.some((m: any) => m.type === 'link'))).toBe(false);
});
});