Compare commits

..

8 Commits

Author SHA1 Message Date
agent_coder 79b2da686b fix(work-time): clip cross-class padding; gate & payload fixes (#395)
F1: pad-clip adjacent different-class sessions at the raw-gap midpoint so
workMs and agentOnlyMs can never double-count the same wall-clock (default
agentTGap 7m < pIn+pOut 10m made a work-session-ending-in-agent overlap a
nearby agent_only run). Reword the config guard/comment: cross-class
disjointness is now structural, tGap≥pIn+pOut is a kept sanity bound. Add a
cross-class no-double-count test and a real seeded property/fuzz test (250
random timelines × 4 tz) asserting per-class union, cross-class disjointness,
and Σ per-day activeMs == workMs.

F2: page.controller.spec — add a ForbiddenException view-gate reject test that
asserts the rejection propagates and computeWorkTime is NOT reached, locking
validateCanView before compute.

F3: WorkTimeStat renders for agent-only pages (workMs==0, agentOnlyMs>0) with
an `agent:` headline so the punch-card stays reachable.

F4: drop the dead un-bucketed `sessions` list from the PageWorkTime response
and the client work-time types (no client reads it; bucketByDay still consumes
the core sessions internally).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 08:31:42 +03:00
agent_coder 3e81f64415 fix(work-time): user idle breaks agent burst; cap tz length (#395)
Only an agent-sourced idle pulse extends an agent burst; a user idle
(human supervision) now falls through to the human branch so the session
is classified `work`, not `agent_only`. Add @MaxLength(64) to tz to match
its comment.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 06:53:59 +03:00
agent_coder c674db2b2f feat(page-history): «время работы над статьёй» — число в UI + суточный punch-card (#395)
Оценка времени работы над страницей как сессионизация истории page_history по
паузам бездействия (WakaTime-подобно), а не span между крайними правками, плюс
drill-down в суточный таймлайн 24ч×дни. Зависит от #374 (kind + idle-пульс).

Server:
- Чистая computeWorkTime(rows, config) (§5): нормализация+дедуп → коллапс
  агентских всплесков по aiChatId → ОДИН проход сессионизации (порог зависит от
  пары: оба agent → agentTGap, иначе tGap; последняя сессия обязательно
  закрывается) → класс готовой сессии (все agent → agent_only, иначе work) →
  добивка (много-сэмпл → [first−P_in, last+P_out], одиночный скаляр →
  [t−P_single, t]) → метрики = union wall-clock по классу. Детерминированная,
  без БД.
- Чистая bucketByDay(sessions, tz) (§6.3): union work/agent_only РАЗДЕЛЬНО, разрез
  по полуночи tz через нативный Intl (DST-корректно, без «+24ч»). Инвариант
  Σ activeMs == workMs держится по построению.
- PageHistoryRepo.findTimelineByPageId — лёгкая проекция без content, ASC.
- PageHistoryService.computeWorkTime + POST /pages/history/time, гейт
  validateCanView как у /history; атрибуция человек/агент по lastUpdatedSource.

Client:
- usePageWorkTime(pageId) шлёт tz зрителя (Intl локаль).
- Кликабельное число «≈ 4 ч 30 мин» в шапке страницы (порог в тултипе).
- Модалка-punch-card: 24-часовые дорожки по дням, окна work/agent разным цветом,
  одиночные (P_single) приглушены, сумма за день, пустой день «—», сворачивание
  длинных серий пустых дней, подпись tz + T_gap. i18n ru-RU + en-US.

Tests: 22 юнита на computeWorkTime/bucketByDay (§7-фикстура ≈1ч32м vs ≈60h наив,
закрытие последней сессии, коллапс/разрыв всплеска, idle-пульс, DST 23/25ч,
полуночный разрез, инвариант, union-без-задвоения); 5 юнитов гейта контроллера;
int-spec на реальном pg (проекция + совпадение с ядром); 6 клиентских на формат.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 06:32:12 +03:00
agent_vscode 03eafa6c68 Merge remote-tracking branch 'gitea/develop' into develop 2026-07-12 05:16:51 +03:00
agent_vscode a42f1ead48 test(ai-chat): align #332 deferred-tool test with #490 cross-turn persistence
The #332 integration test (ai-chat-stream.int-spec.ts) asserted a deferred
tool activated in one turn does NOT leak into the next ("cold start per
turn"). #490 (f5bbfdb2) deliberately reversed that: it persists the
activation set into chat metadata.activatedTools and seeds the next turn
from it, so the model need not re-run loadTools. The test only surfaced now
because the token-estimate CI fix let the integration step run again.

Adopt #490 persistence as the intended behavior:
- flip the turn-2 assertion to expect createPage IS active on the fresh
  turn's first step (seeded from metadata); keep turn-1 cold-start assertions.
- rewrite the test docstring, describe/it titles and comments accordingly.
- fix two stale "not persisted / per-turn" comments in ai-chat.service.ts
  (prepareAgentStep + the streaming-loop activation block); no logic change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 05:16:47 +03:00
vvzvlad b7a3ec227d Merge pull request 'fix(drawio): content= — entity-XML вместо base64 (кириллица в редакторе) (#507)' (#521) from fix/507-drawio-cyrillic into develop
Reviewed-on: #521
2026-07-12 05:08:57 +03:00
agent_coder 846341d7d4 fix(drawio): encode literal tab/newline/CR in content= as numeric char-refs (#507 review)
Review follow-up to the base64→entity-XML content= switch. The whole mxfile XML
now lives in one content="..." attribute; XML attribute-value normalization
collapses a LITERAL tab/newline/CR to a single space on DOM read (jsdom and the
real draw.io editor alike), silently flattening multi-line labels and
tab-bearing values that the old base64 form stored verbatim.

Finding 1 (data-loss): both encode paths — buildDrawioSvg's xmlEscape (mcp) and
the import service's escape — now append &#x9;/&#xa;/&#xd; after the four
&<>" replaces (numeric char-refs survive normalization, as draw.io's own export
does). The extractContentAttr regex fallback now decodes those char-refs (hex
case-insensitive plus decimal &#9;/&#10;/&#13;) so it agrees with the DOM path;
&amp; stays decoded last so an escaped &amp;#x9; reads back as literal text.

Finding 2 (dedup): the server's private xmlEscapeAttr is replaced by the shared
htmlEscape helper (& < > " ' — a strict superset, the extra ' is harmless in a
"-delimited value) wrapped in xmlEscapeContent, which adds the three control-char
char-refs on top (htmlEscape does not escape them).

Finding 3 (docs): narrow the CHANGELOG healing claim — only a diagram still
holding its original correct-UTF-8 base64 (not yet opened/autosaved) is
recoverable; one already opened in the editor persisted mojibake at rest and its
text is lost.

Tests: new mcp round-trip test with literal tab/newline/CR in a value (DOM path,
byte-stable) plus a fallback-branch test forcing a malformed wrapper so both
decode paths are proven to agree; new server spec asserting char-ref encoding.
Mutation-checked: dropping the encode replaces reddens both new mcp tests;
dropping only the fallback decode reddens just the fallback test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 05:05:52 +03:00
agent_coder 32e10ca6d3 fix(drawio): content= пишется entity-XML, не base64 — кириллица не мойбейкает в редакторе (#507)
Реviewer-filed баг: draw.io-редактор декодит base64 content= как Latin-1
(atob-семантика) → кириллица разваливается в мойбейк, автосейв редактора
персистит порчу и убивает превью. Нативная форма draw.io — entity-encoded
mxfile-XML (content="&lt;mxfile…"), DOM-декодится как UTF-8.

Фикс двух write-путей: buildDrawioSvg (mcp, все create/update) и
createDrawioSvg (server Confluence-импорт) теперь XML-эскейпят content=
вместо base64. Декодер уже различает startsWith("&lt;") vs base64 — обе формы
читаются, старые base64-файлы открываются (back-compat), byte-stable
round-trip. CHANGELOG + заметка про лечение старых диаграмм (drawioGet→Update).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-12 05:05:52 +03:00
37 changed files with 2379 additions and 597 deletions
+19
View File
@@ -392,6 +392,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`@docmost/token-estimate` package) so they can never diverge. Deferred-tool
activation is also cached in the chat metadata to avoid re-resolving it each
turn. (#490)
- **Cyrillic (and any non-ASCII) draw.io labels no longer turn into mojibake
when a diagram is opened in the draw.io editor.** Agent-created diagrams
(`drawioCreate`) and Confluence-imported diagrams stored their model in the
SVG's `content=` attribute as base64; the draw.io editor decodes that via
Latin-1 `atob` (no UTF-8 step), so every non-ASCII char (e.g. `Старт-бит`,
`ё`, ``) split into garbage and the editor's autosave then persisted the
corrupted model, breaking the page preview too. Both write paths
(`buildDrawioSvg`, the import service's `createDrawioSvg`) now write `content=`
as XML-entity-escaped mxfile XML — draw.io's own native form, decoded by the
DOM as UTF-8 — so labels open intact. The decoder reads both the new
entity-encoded form and the old base64 form, so existing diagrams still open.
*Healing pre-fix diagrams:* only a diagram that still holds its original
(correct-UTF-8) base64 — i.e. one not yet opened/autosaved in the draw.io
editor — can be repaired in place by `drawioGet` → `drawioUpdate` with the
same XML (rewrites the attachment in the new form); no migration script is
needed. A diagram that was already opened in the editor persisted the
mojibake at rest, so `drawioGet` reads the already-corrupted text and
`drawioUpdate` faithfully rewrites it — that text is lost and is not
recoverable by a rewrite. (#507)
- **A chat with one malformed message part no longer 500s on every turn, and a
failed send no longer duplicates the user's message.** Incoming client parts
are now whitelisted to `text` (a forged tool-result part can no longer reach
@@ -1427,5 +1427,20 @@
"Boundary": "Boundary",
"Autosave": "Autosave",
"Only versions": "Only versions",
"No saved versions yet.": "No saved versions yet."
"No saved versions yet.": "No saved versions yet.",
"Time worked on this article": "Time worked on this article",
"Show time worked on this page": "Show time worked on this page",
"Estimated time worked (inactivity gap {{gap}} min)": "Estimated time worked (inactivity gap {{gap}} min)",
"Estimate · timezone {{tz}} · inactivity gap {{gap}} min": "Estimate · timezone {{tz}} · inactivity gap {{gap}} min",
"No editing activity recorded yet.": "No editing activity recorded yet.",
"× {{count}} days without edits": "× {{count}} days without edits",
"agent: {{value}}": "agent: {{value}}",
"Work": "Work",
"Agent": "Agent",
"≈ {{hours}}h {{minutes}}m": "≈ {{hours}}h {{minutes}}m",
"≈ {{hours}}h": "≈ {{hours}}h",
"≈ {{minutes}}m": "≈ {{minutes}}m",
"{{hours}}h {{minutes}}m": "{{hours}}h {{minutes}}m",
"{{hours}}h": "{{hours}}h",
"{{minutes}}m": "{{minutes}}m"
}
@@ -1442,5 +1442,20 @@
"Boundary": "Граница",
"Autosave": "Автосейв",
"Only versions": "Только версии",
"No saved versions yet.": "Пока нет сохранённых версий."
"No saved versions yet.": "Пока нет сохранённых версий.",
"Time worked on this article": "Время работы над статьёй",
"Show time worked on this page": "Показать время работы над страницей",
"Estimated time worked (inactivity gap {{gap}} min)": "Оценка времени работы (порог паузы {{gap}} мин)",
"Estimate · timezone {{tz}} · inactivity gap {{gap}} min": "Оценка · таймзона {{tz}} · порог паузы {{gap}} мин",
"No editing activity recorded yet.": "Правок пока нет.",
"× {{count}} days without edits": "× {{count}} дн. без правок",
"agent: {{value}}": "агент: {{value}}",
"Work": "Работа",
"Agent": "Агент",
"≈ {{hours}}h {{minutes}}m": "≈ {{hours}} ч {{minutes}} мин",
"≈ {{hours}}h": "≈ {{hours}} ч",
"≈ {{minutes}}m": "≈ {{minutes}} мин",
"{{hours}}h {{minutes}}m": "{{hours}} ч {{minutes}} м",
"{{hours}}h": "{{hours}} ч",
"{{minutes}}m": "{{minutes}} м"
}
@@ -0,0 +1,45 @@
import { describe, it, expect } from "vitest";
import {
formatHeadline,
formatDayTotal,
formatGapMinutes,
} from "./format-work-time";
const MIN = 60 * 1000;
// Fake translator: renders the key with {{tokens}} substituted, so the tests
// assert the rounding + branch selection without depending on the i18n catalogue.
const t = (key: string, opts?: Record<string, unknown>) =>
key.replace(/\{\{(\w+)\}\}/g, (_, k) => String(opts?.[k] ?? ""));
describe("formatHeadline", () => {
it("prefixes ≈ and rounds to a 5-minute step", () => {
expect(formatHeadline(4 * 60 * MIN + 27 * MIN, t)).toBe("≈ 4h 25m");
expect(formatHeadline(90 * MIN, t)).toBe("≈ 1h 30m");
});
it("shows hours only / minutes only cleanly", () => {
expect(formatHeadline(120 * MIN, t)).toBe("≈ 2h");
expect(formatHeadline(35 * MIN, t)).toBe("≈ 35m");
});
it("floors a tiny non-zero estimate to 5m, never 0", () => {
expect(formatHeadline(2 * MIN, t)).toBe("≈ 5m");
});
it("empty string for zero (widget hidden)", () => {
expect(formatHeadline(0, t)).toBe("");
});
});
describe("formatDayTotal", () => {
it('renders "h m" and shows — for empty days', () => {
expect(formatDayTotal(3 * 60 * MIN + 17 * MIN, t)).toBe("3h 17m");
expect(formatDayTotal(0, t)).toBe("—");
});
});
describe("formatGapMinutes", () => {
it("converts the tGap ms threshold to whole minutes", () => {
expect(formatGapMinutes(15 * MIN)).toBe(15);
});
});
@@ -0,0 +1,45 @@
// #395 — display formatting for the work-time estimate. Pure functions that take
// a translator so ru-RU / en-US wording lives in the i18n catalogue and the
// rounding logic stays unit-testable.
type Translate = (key: string, opts?: Record<string, unknown>) => string;
const MIN = 60 * 1000;
function hm(totalMinutes: number): { hours: number; minutes: number } {
return {
hours: Math.floor(totalMinutes / 60),
minutes: totalMinutes % 60,
};
}
/**
* Headline number (§6.1): an ESTIMATE, so rounded to a coarse 5-minute step and
* prefixed with "≈". A non-zero-but-tiny estimate floors to 5m rather than
* rounding down to "0" (which would read as "no work"). Zero empty string
* (the caller hides the widget).
*/
export function formatHeadline(workMs: number, t: Translate): string {
if (workMs <= 0) return "";
let minutes = Math.round(workMs / MIN / 5) * 5;
if (minutes === 0) minutes = 5;
const { hours, minutes: m } = hm(minutes);
if (hours > 0 && m > 0) return t("≈ {{hours}}h {{minutes}}m", { hours, minutes: m });
if (hours > 0) return t("≈ {{hours}}h", { hours });
return t("≈ {{minutes}}m", { minutes: m });
}
/** Per-day sum (§6.2), rounded to the minute. Zero → "—". */
export function formatDayTotal(activeMs: number, t: Translate): string {
if (activeMs <= 0) return "—";
const minutes = Math.max(1, Math.round(activeMs / MIN));
const { hours, minutes: m } = hm(minutes);
if (hours > 0 && m > 0) return t("{{hours}}h {{minutes}}m", { hours, minutes: m });
if (hours > 0) return t("{{hours}}h", { hours });
return t("{{minutes}}m", { minutes: m });
}
/** The inactivity threshold, for the "estimate · gap = N min" caption. */
export function formatGapMinutes(tGapMs: number): number {
return Math.round(tGapMs / MIN);
}
@@ -0,0 +1,25 @@
import { useQuery, UseQueryResult } from "@tanstack/react-query";
import { IPageWorkTime } from "./work-time.types";
import { getPageWorkTime, viewerTimezone } from "./work-time-service";
const WORK_TIME_STALE_TIME = 5 * 60 * 1000;
/**
* #395 the "time worked on this article" estimate + per-day punch-card
* buckets. The buckets are computed server-side in the viewer's timezone (so a
* midnight-crossing session lands on the right calendar day for the reader).
* `enabled` is opt-in so the (cheap but non-trivial) projection query only fires
* when the number is actually shown.
*/
export function usePageWorkTime(
pageId: string,
enabled = true,
): UseQueryResult<IPageWorkTime, Error> {
const tz = viewerTimezone();
return useQuery({
queryKey: ["page-work-time", pageId, tz],
queryFn: () => getPageWorkTime(pageId, tz),
enabled: enabled && !!pageId,
staleTime: WORK_TIME_STALE_TIME,
});
}
@@ -0,0 +1,171 @@
import { Group, Stack, Text } from "@mantine/core";
import { useTranslation } from "react-i18next";
import { useMemo } from "react";
import { IPageWorkTime, IPerDay, IDayWindow } from "./work-time.types";
import {
formatDayTotal,
formatGapMinutes,
formatHeadline,
} from "./format-work-time";
import classes from "./work-time.module.css";
const DAY_MS = 24 * 60 * 60 * 1000;
// Collapse a run of this many (or more) consecutive edit-free days into a single
// "× N days" separator (§6.2 long-range) — the row is still always one day.
const EMPTY_RUN_COLLAPSE = 8;
type Row =
| { type: "day"; day: IPerDay }
| { type: "gap"; count: number };
function collapseEmptyRuns(perDay: IPerDay[]): Row[] {
const rows: Row[] = [];
let emptyRun: IPerDay[] = [];
const flush = () => {
if (emptyRun.length >= EMPTY_RUN_COLLAPSE) {
rows.push({ type: "gap", count: emptyRun.length });
} else {
for (const d of emptyRun) rows.push({ type: "day", day: d });
}
emptyRun = [];
};
for (const d of perDay) {
if (d.activeMs === 0 && d.agentMs === 0) {
emptyRun.push(d);
} else {
flush();
rows.push({ type: "day", day: d });
}
}
flush();
return rows;
}
function dayHeading(day: number): string {
return new Date(day).toLocaleDateString(undefined, {
weekday: "short",
day: "numeric",
month: "short",
});
}
function DayTrack({
day,
pSingle,
}: {
day: IPerDay;
pSingle: number;
}) {
const { t } = useTranslation();
const ticks = [6, 12, 18];
return (
<div className={classes.row}>
<span className={classes.dayLabel}>{dayHeading(day.day)}</span>
<div className={classes.track}>
{ticks.map((h) => (
<div
key={h}
className={classes.hourTick}
style={{ left: `${(h / 24) * 100}%` }}
/>
))}
{day.windows.map((w: IDayWindow, i) => {
const leftPct = ((w.start - day.day) / DAY_MS) * 100;
const widthPct = ((w.end - w.start) / DAY_MS) * 100;
const isSingle = w.end - w.start <= pSingle;
const cls = [
classes.window,
w.class === "work" ? classes.windowWork : classes.windowAgent,
isSingle ? classes.windowSingle : "",
].join(" ");
return (
<div
key={i}
className={cls}
style={{
left: `${Math.max(0, Math.min(100, leftPct))}%`,
width: `${Math.max(0, Math.min(100, widthPct))}%`,
}}
/>
);
})}
</div>
<span className={classes.daySum}>
{formatDayTotal(day.activeMs, t)}
</span>
</div>
);
}
interface Props {
data: IPageWorkTime;
}
export default function WorkTimePunchCard({ data }: Props) {
const { t } = useTranslation();
const rows = useMemo(() => collapseEmptyRuns(data.perDay), [data.perDay]);
const gapMin = formatGapMinutes(data.config.tGap);
if (data.workMs <= 0 && data.agentOnlyMs <= 0) {
return (
<Text size="sm" c="dimmed" py="md">
{t("No editing activity recorded yet.")}
</Text>
);
}
return (
<Stack gap="xs">
<Group gap="lg">
<Text size="sm" fw={500}>
{formatHeadline(data.workMs, t)}
</Text>
{data.agentOnlyMs > 0 && (
<Text size="xs" c="dimmed">
{t("agent: {{value}}", { value: formatHeadline(data.agentOnlyMs, t) })}
</Text>
)}
</Group>
<Group gap="md">
<Text size="xs" c="dimmed">
<span
className={`${classes.legendSwatch} ${classes.windowWork}`}
style={{ marginRight: 4 }}
/>
{t("Work")}
</Text>
<Text size="xs" c="dimmed">
<span
className={`${classes.legendSwatch} ${classes.windowAgent}`}
style={{ marginRight: 4 }}
/>
{t("Agent")}
</Text>
</Group>
<div>
{rows.map((row, i) =>
row.type === "day" ? (
<DayTrack
key={row.day.dayISO}
day={row.day}
pSingle={data.config.pSingle}
/>
) : (
<div key={`gap-${i}`} className={classes.gapRow}>
{t("× {{count}} days without edits", { count: row.count })}
</div>
),
)}
</div>
<Text size="xs" c="dimmed" mt="xs">
{t("Estimate · timezone {{tz}} · inactivity gap {{gap}} min", {
tz: data.tz,
gap: gapMin,
})}
</Text>
</Stack>
);
}
@@ -0,0 +1,23 @@
import api from "@/lib/api-client";
import { IPageWorkTime } from "./work-time.types";
/** The viewer's IANA timezone (browser locale) the punch-card lays days out
* in "my evenings", per §6.3/§10. Falls back to UTC if the runtime hides it. */
export function viewerTimezone(): string {
try {
return Intl.DateTimeFormat().resolvedOptions().timeZone || "UTC";
} catch {
return "UTC";
}
}
export async function getPageWorkTime(
pageId: string,
tz: string,
): Promise<IPageWorkTime> {
const req = await api.post<IPageWorkTime>("/pages/history/time", {
pageId,
tz,
});
return req.data;
}
@@ -0,0 +1,69 @@
import { Modal, Text, Tooltip, UnstyledButton } from "@mantine/core";
import { useDisclosure } from "@mantine/hooks";
import { IconClockHour4 } from "@tabler/icons-react";
import { useTranslation } from "react-i18next";
import { usePageWorkTime } from "./use-page-work-time";
import { formatGapMinutes, formatHeadline } from "./format-work-time";
import WorkTimePunchCard from "./work-time-punch-card";
interface Props {
pageId: string;
}
/**
* #395 the clickable "time worked on this article" headline (§6.1). Renders
* the `work` estimate with a "≈" sign and the inactivity threshold in a tooltip
* (it is an estimate, not a stopwatch). Clicking opens the daily punch-card
* (§6.2). Renders nothing until there is a non-zero human OR agent estimate, so a
* brand-new / never-edited page shows no widget. For an agent-only-edited page
* (workMs===0, agentOnlyMs>0) the headline shows the agent estimate (labelled
* `agent:`, matching the punch-card) so the punch-card stays reachable (#395:
* "how much a HUMAN and separately the AGENT").
*/
export default function WorkTimeStat({ pageId }: Props) {
const { t } = useTranslation();
const [opened, { open, close }] = useDisclosure(false);
const { data } = usePageWorkTime(pageId);
if (!data || (data.workMs <= 0 && data.agentOnlyMs <= 0)) return null;
const agentOnly = data.workMs <= 0;
const label = agentOnly
? t("agent: {{value}}", { value: formatHeadline(data.agentOnlyMs, t) })
: formatHeadline(data.workMs, t);
const gapMin = formatGapMinutes(data.config.tGap);
return (
<>
<Tooltip
label={t("Estimated time worked (inactivity gap {{gap}} min)", {
gap: gapMin,
})}
position="bottom"
>
<UnstyledButton
onClick={open}
aria-label={t("Show time worked on this page")}
>
<Text
size="xs"
c="dimmed"
style={{ display: "inline-flex", alignItems: "center", gap: 4 }}
>
<IconClockHour4 size={14} />
{label}
</Text>
</UnstyledButton>
</Tooltip>
<Modal
opened={opened}
onClose={close}
title={t("Time worked on this article")}
size="lg"
>
<WorkTimePunchCard data={data} />
</Modal>
</>
);
}
@@ -0,0 +1,82 @@
/* #395 24h × days punch-card. Custom CSS segments on a fixed 24-hour track
(position = offset-in-day / 24h, width = duration / 24h), no chart library. */
.row {
display: grid;
grid-template-columns: 96px 1fr 64px;
align-items: center;
gap: 12px;
padding: 3px 0;
}
.dayLabel {
font-size: var(--mantine-font-size-xs);
color: var(--mantine-color-dimmed);
white-space: nowrap;
}
.track {
position: relative;
height: 16px;
border-radius: 4px;
background-color: light-dark(
var(--mantine-color-gray-1),
var(--mantine-color-dark-6)
);
overflow: hidden;
}
/* Faint hour grid so the eye can read "morning vs evening". */
.hourTick {
position: absolute;
top: 0;
bottom: 0;
width: 1px;
background-color: light-dark(
var(--mantine-color-gray-3),
var(--mantine-color-dark-4)
);
}
.window {
position: absolute;
top: 2px;
bottom: 2px;
border-radius: 3px;
min-width: 3px;
}
.windowWork {
background-color: var(--mantine-color-blue-5);
}
.windowAgent {
background-color: var(--mantine-color-grape-5);
}
/* A lone single-sample (P_single) window: minimal + dimmed, so it neither
vanishes nor fakes dense work (§6.2). */
.windowSingle {
opacity: 0.5;
}
.daySum {
font-size: var(--mantine-font-size-xs);
text-align: right;
white-space: nowrap;
}
.gapRow {
padding: 6px 0 6px 108px;
font-size: var(--mantine-font-size-xs);
color: var(--mantine-color-dimmed);
font-style: italic;
}
.legendSwatch {
display: inline-block;
width: 12px;
height: 12px;
border-radius: 3px;
vertical-align: middle;
}
@@ -0,0 +1,37 @@
// #395 — client-side mirror of the server work-time payload
// (apps/server/src/core/page/work-time). Shapes returned by POST /pages/history/time.
export type WorkSessionClass = "work" | "agent_only";
export interface IDayWindow {
start: number;
end: number;
class: WorkSessionClass;
}
export interface IPerDay {
day: number;
dayISO: string;
activeMs: number;
agentMs: number;
windows: IDayWindow[];
}
export interface IWorkTimeConfig {
tGap: number;
agentTGap: number;
pIn: number;
pOut: number;
pSingle: number;
excludeGit: boolean;
burstCapMs?: number;
dedupRoundMs: number;
}
export interface IPageWorkTime {
workMs: number;
agentOnlyMs: number;
perDay: IPerDay[];
config: IWorkTimeConfig;
tz: string;
}
@@ -51,6 +51,7 @@ import {
import { formattedDate } from "@/lib/time.ts";
import { PageEditModeToggle } from "@/features/user/components/page-state-pref.tsx";
import MovePageModal from "@/features/page/components/move-page-modal.tsx";
import WorkTimeStat from "@/features/page-history/work-time/work-time-stat.tsx";
import { useTimeAgo } from "@/hooks/use-time-ago.tsx";
import {
useFavoriteIds,
@@ -265,6 +266,8 @@ function PageActionMenu({ readOnly, onSaveVersion }: PageActionMenuProps) {
return (
<>
{page?.id && <WorkTimeStat pageId={page.id} />}
<Menu
shadow="xl"
position="bottom-end"
@@ -189,10 +189,11 @@ export function stepBudgetWarning(stepNumber: number): string {
//
// `system` is the in-scope system prompt; we CONCATENATE so the original
// persona/context is preserved — a bare `system` override would REPLACE the
// whole system prompt for the step. `activatedTools` is PER-TURN mutable state
// owned by the streaming loop (a closure Set grown by loadTools); it is passed
// in (not module-global, not persisted) so this stays a pure function of its
// arguments.
// whole system prompt for the step. `activatedTools` is a closure Set grown by
// loadTools and owned by the streaming loop; the caller seeds it from and
// persists it to the chat's metadata across turns (#490), but this function only
// READS the Set it is handed, so it stays a pure function of its arguments (not
// module-global).
//
// NOTE: at AI SDK v7 the per-step `system` field is renamed to `instructions`.
// On v6 (`^6.0.134`) `system` is the correct field — adjust when bumping.
@@ -1410,10 +1411,11 @@ export class AiChatService implements OnModuleInit, OnModuleDestroy {
const baseTools = { ...external.tools, ...docmostTools };
// Deferred tool loading state (#332), scoped to THIS streaming loop:
// - `activatedTools` is per-TURN mutable state — a fresh closure Set created
// per streamText call, NOT module-global and NOT persisted, so a new turn
// starts cold. loadTools.execute adds to it; prepareAgentStep reads it to
// widen `activeTools` on the NEXT step.
// - `activatedTools` is a fresh closure Set per streamText call (not
// module-global), SEEDED from the chat's persisted metadata.activatedTools
// (#490, just below) so activation carries across turns. loadTools.execute
// adds to it; prepareAgentStep reads it to widen `activeTools` on the NEXT
// step; turn end persists it back.
// - `validDeferredNames` = every tool that is NOT core (the in-app deferred
// tools + ALL external MCP tools), computed from the ACTUAL toolset so an
// external tool is loadable by its namespaced name. loadTools rejects any
+12
View File
@@ -5,6 +5,7 @@ import {
IsOptional,
IsString,
IsUUID,
MaxLength,
} from 'class-validator';
import { Transform } from 'class-transformer';
@@ -47,6 +48,17 @@ export class PageInfoDto extends PageIdDto {
format?: ContentFormat;
}
export class PageWorkTimeDto extends PageIdDto {
// Viewer IANA timezone for the per-day punch-card buckets (§6.3). Optional —
// falls back to UTC server-side. Length-capped so a bogus value cannot bloat
// the request; the value is only ever handed to Intl.DateTimeFormat, which
// throws on an unknown zone (caught by the controller → 400).
@IsOptional()
@IsString()
@MaxLength(64)
tz?: string;
}
export class DeletePageDto extends PageIdDto {
@IsOptional()
@IsBoolean()
@@ -1,3 +1,8 @@
import {
BadRequestException,
ForbiddenException,
NotFoundException,
} from '@nestjs/common';
import { PageController } from './page.controller';
// Direct instantiation with stub deps. The Test.createTestingModule form failed
@@ -22,4 +27,88 @@ describe('PageController', () => {
it('should be defined', () => {
expect(controller).toBeDefined();
});
// #395 — the work-time endpoint must be gated exactly like /history.
describe('getPageWorkTime', () => {
const user = { id: 'u1' } as any;
function build(overrides: {
page?: any;
validate?: jest.Mock;
compute?: jest.Mock;
}) {
const pageRepo = { findById: jest.fn().mockResolvedValue(overrides.page) };
const pageAccessService = {
validateCanView: overrides.validate ?? jest.fn().mockResolvedValue(undefined),
};
const pageHistoryService = {
computeWorkTime:
overrides.compute ?? jest.fn().mockResolvedValue({ workMs: 0 }),
};
const c = new PageController(
{} as any,
pageRepo as any,
pageHistoryService as any,
{} as any,
pageAccessService as any,
{} as any,
{} as any,
{} as any,
);
return { c, pageRepo, pageAccessService, pageHistoryService };
}
it('404s when the page does not exist', async () => {
const { c } = build({ page: null });
await expect(
c.getPageWorkTime({ pageId: 'p1' } as any, user),
).rejects.toBeInstanceOf(NotFoundException);
});
it('enforces validateCanView before computing, then delegates with tz', async () => {
const validate = jest.fn().mockResolvedValue(undefined);
const compute = jest.fn().mockResolvedValue({ workMs: 42 });
const { c } = build({ page: { id: 'pg' }, validate, compute });
const out = await c.getPageWorkTime(
{ pageId: 'pg', tz: 'Europe/Moscow' } as any,
user,
);
expect(validate).toHaveBeenCalledWith({ id: 'pg' }, user);
expect(compute).toHaveBeenCalledWith('pg', 'Europe/Moscow');
expect(out).toEqual({ workMs: 42 });
});
it('propagates a denied view gate and does NOT reach compute (security)', async () => {
// If validateCanView is moved AFTER computeWorkTime, the timeline of a page
// the caller may not see would be read/estimated before the gate — this
// locks the order: a rejecting gate must short-circuit before any compute.
const validate = jest.fn().mockRejectedValue(new ForbiddenException());
const compute = jest.fn().mockResolvedValue({ workMs: 1 });
const { c, pageHistoryService } = build({
page: { id: 'pg' },
validate,
compute,
});
await expect(
c.getPageWorkTime({ pageId: 'pg' } as any, user),
).rejects.toBeInstanceOf(ForbiddenException);
expect(pageHistoryService.computeWorkTime).not.toHaveBeenCalled();
});
it('maps an unknown-timezone RangeError to a 400', async () => {
const compute = jest.fn().mockRejectedValue(new RangeError('bad tz'));
const { c } = build({ page: { id: 'pg' }, compute });
await expect(
c.getPageWorkTime({ pageId: 'pg', tz: 'X/Y' } as any, user),
).rejects.toBeInstanceOf(BadRequestException);
});
it('does not swallow a non-RangeError from the service', async () => {
const compute = jest.fn().mockRejectedValue(new Error('db down'));
const { c } = build({ page: { id: 'pg' }, compute });
await expect(
c.getPageWorkTime({ pageId: 'pg' } as any, user),
).rejects.toThrow('db down');
});
});
});
@@ -21,6 +21,7 @@ import {
PageHistoryIdDto,
PageIdDto,
PageInfoDto,
PageWorkTimeDto,
} from './dto/page.dto';
import { PageHistoryService } from './services/page-history.service';
import { AuthUser } from '../../common/decorators/auth-user.decorator';
@@ -524,6 +525,32 @@ export class PageController {
return this.pageHistoryService.findHistoryByPageId(page.id, pagination);
}
@HttpCode(HttpStatus.OK)
@Post('/history/time')
async getPageWorkTime(
@Body() dto: PageWorkTimeDto,
@AuthUser() user: User,
) {
const page = await this.pageRepo.findById(dto.pageId);
if (!page) {
throw new NotFoundException('Page not found');
}
// Same view gate as /history and /history/info.
await this.pageAccessService.validateCanView(page, user);
try {
return await this.pageHistoryService.computeWorkTime(page.id, dto.tz);
} catch (e) {
// Intl.DateTimeFormat throws RangeError on an unknown IANA zone; surface
// it as a 400 rather than a 500.
if (e instanceof RangeError) {
throw new BadRequestException('Invalid timezone');
}
throw e;
}
}
@HttpCode(HttpStatus.OK)
@Post('/history/info')
async getPageHistoryInfo(
@@ -3,6 +3,23 @@ import { PageHistoryRepo } from '@docmost/db/repos/page/page-history.repo';
import { PageHistory } from '@docmost/db/types/entity.types';
import { PaginationOptions } from '@docmost/db/pagination/pagination-options';
import { CursorPaginationResult } from '@docmost/db/pagination/cursor-pagination';
import {
computeWorkTime,
bucketByDay,
DEFAULT_WORK_TIME_CONFIG,
WorkTimeConfig,
PerDay,
} from '../work-time';
export interface PageWorkTime {
workMs: number;
agentOnlyMs: number;
perDay: PerDay[];
/** the config actually used, so the UI can show "≈" + the T_gap threshold. */
config: WorkTimeConfig;
/** the tz the per-day buckets were computed in (echoed back for the label). */
tz: string;
}
@Injectable()
export class PageHistoryService {
@@ -23,4 +40,33 @@ export class PageHistoryService {
paginationOptions,
);
}
/**
* #395 estimate time worked on a page (§5) and bucket it into the viewer's
* calendar days for the punch-card (§6.3). Reads only the cheap history
* projection (no `content`); the estimate itself is a pure, deterministic
* function so it is unit-tested exhaustively without a DB.
*
* `tz` is the viewer's IANA zone (browser locale) it moves which day a
* session lands in and where its windows sit, but never the total (§10).
*/
async computeWorkTime(
pageId: string,
tz = 'UTC',
config?: Partial<WorkTimeConfig>,
): Promise<PageWorkTime> {
const rows = await this.pageHistoryRepo.findTimelineByPageId(pageId);
const result = computeWorkTime(rows, config);
const usedConfig: WorkTimeConfig = { ...DEFAULT_WORK_TIME_CONFIG, ...config };
// `bucketByDay` consumes the pure core's un-bucketed sessions here; the
// full session list is NOT shipped on the response (no client reads it).
const perDay = bucketByDay(result.sessions, tz);
return {
workMs: result.workMs,
agentOnlyMs: result.agentOnlyMs,
perDay,
config: usedConfig,
tz,
};
}
}
@@ -0,0 +1,129 @@
import { bucketByDay, zonedDayStart } from './bucket-by-day';
import { computeWorkTime } from './compute-work-time';
import { WorkSession, TimelineSample } from './work-time.types';
const MIN = 60 * 1000;
const HOUR = 60 * MIN;
function work(start: number, end: number): WorkSession {
return { start, end, class: 'work' };
}
function agent(start: number, end: number): WorkSession {
return { start, end, class: 'agent_only' };
}
function sumActive(perDay: ReturnType<typeof bucketByDay>): number {
return perDay.reduce((a, d) => a + d.activeMs, 0);
}
describe('bucketByDay', () => {
it('Σ activeMs == workMs — the §6.3 consistency invariant', () => {
const rows: TimelineSample[] = [
{ createdAt: '2026-07-04T03:40:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
{ createdAt: '2026-07-04T03:49:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
{ createdAt: '2026-07-04T18:11:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
{ createdAt: '2026-07-06T15:34:00Z', lastUpdatedById: 'h', lastUpdatedSource: 'user', lastUpdatedAiChatId: null, kind: null },
];
const r = computeWorkTime(rows);
const perDay = bucketByDay(r.sessions, 'UTC');
expect(sumActive(perDay)).toBe(r.workMs);
});
it('empty input → no days', () => {
expect(bucketByDay([], 'UTC')).toEqual([]);
});
it('midnight-crossing session splits across two days, sum preserved (§9#9)', () => {
const start = Date.UTC(2026, 0, 10, 23, 14);
const end = Date.UTC(2026, 0, 11, 0, 40);
const perDay = bucketByDay([work(start, end)], 'UTC');
expect(perDay).toHaveLength(2);
expect(perDay[0].dayISO).toBe('2026-01-10');
expect(perDay[1].dayISO).toBe('2026-01-11');
expect(perDay[0].activeMs).toBe(46 * MIN); // 23:14 → 24:00
expect(perDay[1].activeMs).toBe(40 * MIN); // 00:00 → 00:40
expect(sumActive(perDay)).toBe(end - start);
});
it('empty days between active days are emitted, not skipped (§9#12)', () => {
const d1 = work(Date.UTC(2026, 0, 10, 10, 0), Date.UTC(2026, 0, 10, 11, 0));
const d3 = work(Date.UTC(2026, 0, 12, 10, 0), Date.UTC(2026, 0, 12, 11, 0));
const perDay = bucketByDay([d1, d3], 'UTC');
expect(perDay.map((d) => d.dayISO)).toEqual([
'2026-01-10',
'2026-01-11',
'2026-01-12',
]);
expect(perDay[1].activeMs).toBe(0);
expect(perDay[1].windows).toEqual([]);
});
it('agent_only windows are drawn but excluded from activeMs', () => {
const w = work(Date.UTC(2026, 0, 10, 9, 0), Date.UTC(2026, 0, 10, 10, 0));
const a = agent(Date.UTC(2026, 0, 10, 14, 0), Date.UTC(2026, 0, 10, 14, 30));
const perDay = bucketByDay([w, a], 'UTC');
expect(perDay).toHaveLength(1);
expect(perDay[0].activeMs).toBe(1 * HOUR);
expect(perDay[0].agentMs).toBe(30 * MIN);
expect(perDay[0].windows.map((x) => x.class)).toEqual(['work', 'agent_only']);
});
it('work and agent_only are unioned SEPARATELY (agent does not swallow work)', () => {
// Overlapping work + agent windows on the same day.
const w = work(Date.UTC(2026, 0, 10, 9, 0), Date.UTC(2026, 0, 10, 11, 0));
const a = agent(Date.UTC(2026, 0, 10, 10, 0), Date.UTC(2026, 0, 10, 12, 0));
const perDay = bucketByDay([w, a], 'UTC');
expect(perDay[0].activeMs).toBe(2 * HOUR);
expect(perDay[0].agentMs).toBe(2 * HOUR);
});
it('overlapping same-class sessions are UNIONed, not summed (no double-count)', () => {
// Two work sessions that overlap 10:00–10:30 on one day.
const a = work(Date.UTC(2026, 0, 10, 9, 0), Date.UTC(2026, 0, 10, 10, 30));
const b = work(Date.UTC(2026, 0, 10, 10, 0), Date.UTC(2026, 0, 10, 11, 0));
const perDay = bucketByDay([a, b], 'UTC');
expect(perDay).toHaveLength(1);
// Union 09:00–11:00 = 2h, NOT 90m + 60m = 150m.
expect(perDay[0].activeMs).toBe(2 * HOUR);
// The drawn windows are also merged to one, so the punch-card cannot render
// an overlapping double bar.
expect(perDay[0].windows).toHaveLength(1);
expect(perDay[0].windows[0].start).toBe(a.start);
expect(perDay[0].windows[0].end).toBe(b.end);
});
it('DST fall-back: a full 25-hour day still balances (§9#14)', () => {
// America/New_York ends DST 2026-11-01 (25h day).
const tz = 'America/New_York';
const dayStart = zonedDayStart(Date.UTC(2026, 10, 1, 12, 0), tz);
const nextStart = zonedDayStart(dayStart + 26 * HOUR, tz);
expect(nextStart - dayStart).toBe(25 * HOUR);
const perDay = bucketByDay([work(dayStart, nextStart)], tz);
expect(perDay).toHaveLength(1);
expect(perDay[0].dayISO).toBe('2026-11-01');
expect(perDay[0].activeMs).toBe(25 * HOUR);
expect(sumActive(perDay)).toBe(nextStart - dayStart);
});
it('DST spring-forward: a full 23-hour day still balances (§9#14)', () => {
// America/New_York starts DST 2026-03-08 (23h day).
const tz = 'America/New_York';
const dayStart = zonedDayStart(Date.UTC(2026, 2, 8, 12, 0), tz);
const nextStart = zonedDayStart(dayStart + 26 * HOUR, tz);
expect(nextStart - dayStart).toBe(23 * HOUR);
const perDay = bucketByDay([work(dayStart, nextStart)], tz);
expect(perDay).toHaveLength(1);
expect(perDay[0].activeMs).toBe(23 * HOUR);
expect(sumActive(perDay)).toBe(nextStart - dayStart);
});
it('tz changes the day a session lands in but not the total', () => {
const start = Date.UTC(2026, 0, 10, 2, 0); // 02:00 UTC
const end = Date.UTC(2026, 0, 10, 3, 0);
const utc = bucketByDay([work(start, end)], 'UTC');
const ny = bucketByDay([work(start, end)], 'America/New_York'); // 21:00 prev day
expect(utc[0].dayISO).toBe('2026-01-10');
expect(ny[0].dayISO).toBe('2026-01-09');
expect(sumActive(utc)).toBe(sumActive(ny));
});
});
@@ -0,0 +1,180 @@
import { WorkSession, PerDay, DayWindow } from './work-time.types';
/**
* Merge intervals into a disjoint, sorted union. Overlapping OR touching
* intervals are joined. Empty input [].
*/
function union(intervals: Array<[number, number]>): Array<[number, number]> {
if (intervals.length === 0) return [];
const sorted = [...intervals].sort((a, b) => a[0] - b[0]);
const out: Array<[number, number]> = [];
let [curStart, curEnd] = sorted[0];
for (let i = 1; i < sorted.length; i++) {
const [s, e] = sorted[i];
if (s <= curEnd) {
if (e > curEnd) curEnd = e;
} else {
out.push([curStart, curEnd]);
curStart = s;
curEnd = e;
}
}
out.push([curStart, curEnd]);
return out;
}
// Cache one Intl formatter per tz — constructing them is comparatively costly.
const fmtCache = new Map<string, Intl.DateTimeFormat>();
function partsFmt(tz: string): Intl.DateTimeFormat {
let fmt = fmtCache.get(tz);
if (!fmt) {
fmt = new Intl.DateTimeFormat('en-US', {
timeZone: tz,
year: 'numeric',
month: '2-digit',
day: '2-digit',
hour: '2-digit',
minute: '2-digit',
second: '2-digit',
hour12: false,
});
fmtCache.set(tz, fmt);
}
return fmt;
}
interface WallParts {
year: number;
month: number;
day: number;
hour: number;
minute: number;
second: number;
}
/** Wall-clock parts of an instant in `tz` (DST-correct, via Intl). */
function wallParts(ms: number, tz: string): WallParts {
const parts = partsFmt(tz).formatToParts(new Date(ms));
const get = (type: string) =>
Number(parts.find((p) => p.type === type)?.value ?? '0');
let hour = get('hour');
// Intl emits "24" for midnight under some engines/locales; normalize to 0.
if (hour === 24) hour = 0;
return {
year: get('year'),
month: get('month'),
day: get('day'),
hour,
minute: get('minute'),
second: get('second'),
};
}
/** tz offset (wall − real) at an instant, in ms. */
function offset(ms: number, tz: string): number {
const p = wallParts(ms, tz);
const asUTC = Date.UTC(p.year, p.month - 1, p.day, p.hour, p.minute, p.second);
return asUTC - ms;
}
/**
* Epoch-ms of the local-midnight day start of `ms` in `tz`. DST-correct: takes
* the calendar day of the instant, its wall-midnight, then converts back with
* the offset that actually applies AT that midnight (refined once). The rare
* tz-with-a-DST-transition-exactly-at-midnight case is a documented edge (§9#14).
*/
export function zonedDayStart(ms: number, tz: string): number {
const p = wallParts(ms, tz);
const wallMidnightAsUTC = Date.UTC(p.year, p.month - 1, p.day, 0, 0, 0);
let start = wallMidnightAsUTC - offset(ms, tz);
// Refine with the offset at the computed midnight (DST may differ from `ms`).
start = wallMidnightAsUTC - offset(start, tz);
return start;
}
/** The next local midnight after `dayStart` (handles 23/25h DST days). */
function nextDayStart(dayStart: number, tz: string): number {
// +26h always lands inside the NEXT calendar day (day length ∈ [23h,25h]),
// never two days ahead; startOf('day') of it is the next midnight.
return zonedDayStart(dayStart + 26 * 60 * 60 * 1000, tz);
}
function isoDay(dayStart: number, tz: string): string {
const p = wallParts(dayStart, tz);
const pad = (n: number) => String(n).padStart(2, '0');
return `${p.year}-${pad(p.month)}-${pad(p.day)}`;
}
/** Clip a union to [lo, hi) and emit windows of `class`. */
function clip(
merged: Array<[number, number]>,
lo: number,
hi: number,
cls: DayWindow['class'],
): DayWindow[] {
const out: DayWindow[] = [];
for (const [s, e] of merged) {
const start = Math.max(s, lo);
const end = Math.min(e, hi);
if (end > start) out.push({ start, end, class: cls });
}
return out;
}
/**
* #395 §6.3 bucket sessions into calendar days of `tz` for the punch-card.
* Pure and deterministic. `work` and `agent_only` are unioned SEPARATELY (else
* agent windows would swallow work windows on overlap), then each union is split
* at tz midnight boundaries (`startOf('day')` in tz, NOT "+24h" DST-safe §9#14)
* and clipped to each day.
*
* By construction Σ perDay.activeMs == workMs: the days are a partition of the
* `work` union no loss, no dup, even on 23/25h DST days. `agent_only` windows
* are drawn but NOT in activeMs. Empty days between the first and last active day
* are emitted (empty track + "—") so the rhythm/pauses stay visible.
*/
export function bucketByDay(sessions: WorkSession[], tz: string): PerDay[] {
const uWork = union(
sessions.filter((s) => s.class === 'work').map((s) => [s.start, s.end]),
);
const uAgent = union(
sessions
.filter((s) => s.class === 'agent_only')
.map((s) => [s.start, s.end]),
);
if (uWork.length === 0 && uAgent.length === 0) return [];
const minStart = Math.min(
uWork.length ? uWork[0][0] : Infinity,
uAgent.length ? uAgent[0][0] : Infinity,
);
const maxEnd = Math.max(
uWork.length ? uWork[uWork.length - 1][1] : -Infinity,
uAgent.length ? uAgent[uAgent.length - 1][1] : -Infinity,
);
const perDay: PerDay[] = [];
let dayStart = zonedDayStart(minStart, tz);
// Guard against a pathological non-advancing boundary.
let guard = 0;
while (dayStart < maxEnd && guard < 100000) {
guard++;
const dayEnd = nextDayStart(dayStart, tz);
const workWin = clip(uWork, dayStart, dayEnd, 'work');
const agentWin = clip(uAgent, dayStart, dayEnd, 'agent_only');
const activeMs = workWin.reduce((a, w) => a + (w.end - w.start), 0);
const agentMs = agentWin.reduce((a, w) => a + (w.end - w.start), 0);
const windows = [...workWin, ...agentWin].sort((a, b) => a.start - b.start);
perDay.push({
day: dayStart,
dayISO: isoDay(dayStart, tz),
activeMs,
agentMs,
windows,
});
dayStart = dayEnd;
}
return perDay;
}
@@ -0,0 +1,358 @@
import { computeWorkTime } from './compute-work-time';
import { bucketByDay } from './bucket-by-day';
import { TimelineSample, WorkSession } from './work-time.types';
const MIN = 60 * 1000;
/** Union wall-clock of a set of intervals (touching intervals merge). */
function unionMs(intervals: Array<[number, number]>): number {
if (intervals.length === 0) return 0;
const sorted = [...intervals].sort((a, b) => a[0] - b[0]);
let total = 0;
let [cs, ce] = sorted[0];
for (let i = 1; i < sorted.length; i++) {
const [s, e] = sorted[i];
if (s <= ce) {
if (e > ce) ce = e;
} else {
total += ce - cs;
cs = s;
ce = e;
}
}
return total + (ce - cs);
}
const ivsOf = (sessions: WorkSession[], cls?: string): Array<[number, number]> =>
sessions
.filter((x) => cls == null || x.class === cls)
.map((x) => [x.start, x.end] as [number, number]);
function s(
iso: string,
opts: {
source?: string | null;
chat?: string | null;
kind?: string | null;
by?: string | null;
} = {},
): TimelineSample {
return {
createdAt: `${iso}Z`,
lastUpdatedById: opts.by ?? 'human-1',
lastUpdatedSource: opts.source === undefined ? 'user' : opts.source,
lastUpdatedAiChatId: opts.chat ?? null,
kind: opts.kind ?? null,
};
}
// §7 config: T_gap=30m, P_in+P_out=10m, P_single=2m.
const S7 = { tGap: 30 * MIN, agentTGap: 30 * MIN, pIn: 5 * MIN, pOut: 5 * MIN, pSingle: 2 * MIN };
describe('computeWorkTime', () => {
it('§7 fixture — sessionizes 20-ish samples to ≈1h32m, not the ≈60h naive span', () => {
const rows: TimelineSample[] = [
// S1: multi-sample morning session
s('2026-07-04T03:40:00'),
s('2026-07-04T03:45:00'),
s('2026-07-04T03:49:00'),
// S2: agent burst (one run) then human supervising → class work
s('2026-07-04T15:43:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T15:47:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T15:50:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T16:13:00'),
// S3: single
s('2026-07-04T18:11:00'),
// S4: multi-sample evening session
s('2026-07-04T19:38:00'),
s('2026-07-04T19:44:00'),
s('2026-07-04T19:54:00'),
// S5 / S6: two singles two days later, 44m apart → two sessions at T_gap=30
s('2026-07-06T15:34:00'),
s('2026-07-06T16:18:00'),
];
const r = computeWorkTime(rows, S7);
// 19 + 40 + 2 + 26 + 2 + 2 = 91 minutes.
expect(r.workMs).toBe(91 * MIN);
expect(r.agentOnlyMs).toBe(0);
expect(r.sessions).toHaveLength(6);
expect(r.sessions.every((x) => x.class === 'work')).toBe(true);
const naiveSpan =
new Date('2026-07-06T16:18:00Z').getTime() -
new Date('2026-07-04T03:40:00Z').getTime();
expect(naiveSpan).toBeGreaterThan(60 * 60 * MIN); // ≈60h
expect(r.workMs).toBeLessThan(naiveSpan / 30); // dramatically smaller
});
it('n=0 → zero, no sessions', () => {
const r = computeWorkTime([]);
expect(r).toEqual({ workMs: 0, agentOnlyMs: 0, sessions: [] });
});
it('n=1 human → one P_single work session', () => {
const r = computeWorkTime([s('2026-07-04T10:00:00')], S7);
expect(r.sessions).toHaveLength(1);
expect(r.sessions[0].class).toBe('work');
expect(r.workMs).toBe(2 * MIN);
expect(r.agentOnlyMs).toBe(0);
// pre-roll only: [t − P_single, t]
expect(r.sessions[0].end).toBe(new Date('2026-07-04T10:00:00Z').getTime());
});
it('n=1 agent → one P_single agent_only session, work=0 (§9#2)', () => {
const r = computeWorkTime(
[s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' })],
S7,
);
expect(r.sessions).toHaveLength(1);
expect(r.sessions[0].class).toBe('agent_only');
expect(r.workMs).toBe(0);
expect(r.agentOnlyMs).toBe(2 * MIN);
});
it('MUST close the last session — the newest session is not lost (§9#1)', () => {
// Two singles a day apart: without the post-loop close, the 2nd is dropped.
const rows = [s('2026-07-04T10:00:00'), s('2026-07-05T10:00:00')];
const r = computeWorkTime(rows, S7);
expect(r.sessions).toHaveLength(2);
const lastStart = Math.max(...r.sessions.map((x) => x.start));
expect(lastStart).toBe(
new Date('2026-07-05T10:00:00Z').getTime() - 2 * MIN,
);
expect(r.workMs).toBe(4 * MIN);
});
it('agent-burst collapse: density does not inflate — length = wall-clock', () => {
const span = ['00', '01', '02', '03', '04', '05', '06'];
const dense: TimelineSample[] = span.map((sec) =>
s(`2026-07-04T10:00:${sec}`, { source: 'agent', chat: 'c1', kind: 'agent' }),
);
const sparse: TimelineSample[] = [
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:00:06', { source: 'agent', chat: 'c1', kind: 'agent' }),
];
const rDense = computeWorkTime(dense, S7);
const rSparse = computeWorkTime(sparse, S7);
// Same 6-second wall-clock span → same estimate regardless of snapshot count.
expect(rDense.agentOnlyMs).toBe(rSparse.agentOnlyMs);
expect(rDense.sessions).toHaveLength(1);
expect(rDense.sessions[0].class).toBe('agent_only');
});
it('supervisory agent time inside a human session counts as work, not agent', () => {
const rows = [
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:05:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:12:00'), // human within T_gap
];
const r = computeWorkTime(rows, S7);
expect(r.sessions).toHaveLength(1);
expect(r.sessions[0].class).toBe('work');
expect(r.agentOnlyMs).toBe(0);
expect(r.workMs).toBeGreaterThan(0);
});
it('a DIFFERENT aiChatId breaks the burst — two agent runs, idle gap excluded', () => {
// Run c1 ends 10:05, run c2 starts 10:20 (15m > agentTGap 7m) → two sessions.
const rows = [
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:05:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:20:00', { source: 'agent', chat: 'c2', kind: 'agent' }),
s('2026-07-04T10:25:00', { source: 'agent', chat: 'c2', kind: 'agent' }),
];
const r = computeWorkTime(rows); // default agentTGap = 7m
expect(r.sessions).toHaveLength(2);
expect(r.sessions.every((x) => x.class === 'agent_only')).toBe(true);
// The 15m idle gap between the two runs is NOT counted.
const run1 = 5 * MIN + 5 * MIN + 5 * MIN; // pIn + span + pOut
expect(r.agentOnlyMs).toBe(2 * run1);
});
it('idle pulse (same/null run) is a full activity sample that continues a burst', () => {
const rows = [
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
// idle flush 4m later, null run id → continues the burst, not a new one
s('2026-07-04T10:04:00', { source: 'agent', chat: null, kind: 'idle' }),
s('2026-07-04T10:08:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
];
const r = computeWorkTime(rows);
expect(r.sessions).toHaveLength(1);
// burst span 10:00→10:08 (+pIn/pOut) = 8 + 10 = 18m
expect(r.agentOnlyMs).toBe(18 * MIN);
});
it('a USER-sourced idle breaks an agent burst → session is work, not agent_only', () => {
// A human supervision idle inherits source=user (aiChatId:null) and must NOT
// be swallowed into the agent burst. Δ=3m is within the default agentTGap so
// the two samples stay one session — but its class flips to `work`.
const rows = [
s('2026-07-04T10:00:00', { source: 'agent', chat: 'c1', kind: 'agent' }),
s('2026-07-04T10:03:00', { source: 'user', chat: null, kind: 'idle' }),
];
const r = computeWorkTime(rows);
expect(r.sessions).toHaveLength(1);
expect(r.sessions[0].class).toBe('work');
expect(r.workMs).toBeGreaterThan(0);
// The human idle is NOT captured as agent_only time.
expect(r.agentOnlyMs).toBe(0);
// Σ over `work` sessions == workMs and Σ over `agent_only` == agentOnlyMs.
const sum = (cls: string) =>
r.sessions
.filter((x) => x.class === cls)
.reduce((acc, x) => acc + (x.end - x.start), 0);
expect(sum('work')).toBe(r.workMs);
expect(sum('agent_only')).toBe(r.agentOnlyMs);
});
it('idle pulse keeps a human writing session visible (not excluded)', () => {
const rows = [
s('2026-07-04T10:00:00'),
s('2026-07-04T10:08:00', { kind: 'idle' }), // pulse within T_gap
s('2026-07-04T10:15:00'),
];
const r = computeWorkTime(rows);
expect(r.sessions).toHaveLength(1);
expect(r.sessions[0].class).toBe('work');
// span 10:00→10:15 + pIn/pOut = 15 + 10 = 25m
expect(r.workMs).toBe(25 * MIN);
});
it('git-source samples are excluded (§10 excludeGit)', () => {
const rows = [
s('2026-07-04T10:00:00', { source: 'git', kind: 'boundary' }),
s('2026-07-04T10:01:00', { source: 'git', kind: 'boundary' }),
];
expect(computeWorkTime(rows).workMs).toBe(0);
// ...but honoured off:
expect(
computeWorkTime(rows, { excludeGit: false }).workMs,
).toBeGreaterThan(0);
});
it('rejects an invalid config (tGap < pIn + pOut)', () => {
expect(() =>
computeWorkTime([s('2026-07-04T10:00:00')], {
tGap: 5 * MIN,
pIn: 5 * MIN,
pOut: 5 * MIN,
}),
).toThrow(/tGap/);
});
it('rejects an invalid config (2·agentTGap < pIn + pOut)', () => {
// tGap (default 15m) still ≥ pIn+pOut, so only the 2·agentTGap guard trips.
// Without it a short session of one class between two of the other could
// produce a NON-adjacent cross-class overlap the adjacent-only clip misses.
expect(() =>
computeWorkTime([s('2026-07-04T10:00:00')], {
agentTGap: 2 * MIN,
pIn: 5 * MIN,
pOut: 5 * MIN,
}),
).toThrow(/agentTGap/);
});
// F1 — cross-class double-count. On the DEFAULT config agentTGap (7m) < pIn+pOut
// (10m), so a `work` session ending in an agent segment and a nearby separate
// `agent_only` run (gap in (7m,10m]) used to produce OVERLAPPING padded
// intervals — the same wall-clock counted into BOTH workMs and agentOnlyMs. The
// cross-class padding clip must make the two per-class unions disjoint.
it('does NOT double-count wall-clock across work/agent_only (§F1)', () => {
// user@0s ; agent(chatX)@60s (breaks into a work session with the human) ;
// agent(chatY)@560s,590s (a separate agent_only run). Raw gap between the work
// session (ends 60s) and the agent run (starts 560s) is 500s ∈ (agentTGap,
// pIn+pOut] once padded — the classic overlap window.
const rows: TimelineSample[] = [
s('2026-07-04T00:00:00'), // user @ 0s
s('2026-07-04T00:01:00', { source: 'agent', chat: 'cX', kind: 'agent' }), // @ 60s
s('2026-07-04T00:09:20', { source: 'agent', chat: 'cY', kind: 'agent' }), // @ 560s
s('2026-07-04T00:09:50', { source: 'agent', chat: 'cY', kind: 'agent' }), // @ 590s
];
const r = computeWorkTime(rows); // DEFAULT config
// Both classes present.
expect(r.workMs).toBeGreaterThan(0);
expect(r.agentOnlyMs).toBeGreaterThan(0);
// Per-class metrics are exactly their own union (union, not Σ).
expect(r.workMs).toBe(unionMs(ivsOf(r.sessions, 'work')));
expect(r.agentOnlyMs).toBe(unionMs(ivsOf(r.sessions, 'agent_only')));
// The F1 invariant: work-union and agent-union are cross-class-disjoint, so
// the union of ALL padded intervals equals workMs + agentOnlyMs (no overlap).
// With the clip disabled this fails (union < sum by the 100s overlap).
expect(unionMs(ivsOf(r.sessions))).toBe(r.workMs + r.agentOnlyMs);
});
// F1 property/fuzz — random timelines across several timezones must uphold the
// work-time invariants. Backs the (corrected) PR claim of a real fuzz test.
it('property: random timelines uphold union & cross-class-disjoint invariants', () => {
// Deterministic LCG (numerical-recipes constants) so a failure is reproducible.
let seed = 0x9e3779b9 >>> 0;
const rand = () => {
seed = (Math.imul(seed, 1664525) + 1013904223) >>> 0;
return seed / 0x100000000;
};
const pick = <T>(arr: T[]): T => arr[Math.floor(rand() * arr.length)];
const tzs = [
'UTC',
'America/New_York',
'Europe/Moscow',
'Australia/Lord_Howe', // 30-min DST offset — a nasty bucket stress
];
const base = Date.UTC(2026, 5, 1, 0, 0, 0); // 2026-06-01Z
const chats = ['c1', 'c2', 'c3'];
for (let iter = 0; iter < 250; iter++) {
const tz = pick(tzs);
const n = 2 + Math.floor(rand() * 18); // 2..19 rows
const rows: TimelineSample[] = [];
// Walk time forward by a random inter-sample gap. The gap distribution is
// centred on the DANGEROUS band — a bit under to a bit over pIn+pOut (10m)
// AND straddling agentTGap (7m) — so adjacent samples routinely split into
// separate sessions whose ±P padding would overlap if a class boundary sits
// there. Mixing user/agent classes at these gaps reliably manufactures the
// work-ending-in-agent → agent_only cross-class boundary F1 is about, plus
// dense within-class runs (occasional 0–2m gaps) that exercise the union.
let t = base + Math.floor(rand() * 60 * MIN);
for (let i = 0; i < n; i++) {
const roll = rand();
const gap =
roll < 0.25
? Math.floor(rand() * 2 * MIN) // dense burst (same-class union)
: roll < 0.85
? 5 * MIN + Math.floor(rand() * 8 * MIN) // 5–13m: the split band
: 20 * MIN + Math.floor(rand() * 40 * MIN); // long idle → new day-ish
t += gap;
const iso = new Date(t).toISOString().slice(0, 19); // 'YYYY-MM-DDTHH:MM:SS'
const isAgent = rand() < 0.5;
rows.push(
isAgent
? s(iso, { source: 'agent', chat: pick(chats), kind: 'agent' })
: s(iso, { source: 'user', kind: rand() < 0.3 ? 'idle' : 'manual' }),
);
}
const r = computeWorkTime(rows); // DEFAULT config
const workIvs = ivsOf(r.sessions, 'work');
const agentIvs = ivsOf(r.sessions, 'agent_only');
// (1) each metric is exactly its per-class union (catches a union→Σ regress).
expect(r.workMs).toBe(unionMs(workIvs));
expect(r.agentOnlyMs).toBe(unionMs(agentIvs));
// (2) NO cross-class overlap: union(all) == workMs + agentOnlyMs (F1).
expect(unionMs(ivsOf(r.sessions))).toBe(r.workMs + r.agentOnlyMs);
// (3) bucket invariant: Σ per-day activeMs == workMs (§6.3).
const perDay = bucketByDay(r.sessions, tz);
const sumActive = perDay.reduce((a, d) => a + d.activeMs, 0);
expect(sumActive).toBe(r.workMs);
}
});
});
@@ -0,0 +1,274 @@
import {
TimelineSample,
WorkSession,
WorkTimeResult,
} from './work-time.types';
import { WorkTimeConfig, resolveWorkTimeConfig } from './work-time.config';
/** A normalized activity sample (one history row), createdAt as epoch-ms. */
interface NormSample {
t: number;
isAgent: boolean;
aiChatId: string | null;
kind: string | null;
}
/**
* A collapsed segment: either a scalar sample (t_start == t_end) or an
* agent-burst spanning several agent samples of one run (§5.1). It participates
* in sessionization as a single "sample".
*/
interface Segment {
tStart: number;
tEnd: number;
isAgent: boolean;
}
function toMs(v: Date | string | number): number {
if (v instanceof Date) return v.getTime();
if (typeof v === 'number') return v;
return new Date(v).getTime();
}
/**
* Normalize raw rows sorted, deduped activity samples. `git` is dropped when
* configured; every other kind (incl. `idle` the continuous-work pulse §3) is
* a real activity sample. Sort is by createdAt ASC; samples whose timestamps
* fall in the same `dedupRoundMs` bucket collapse to one (§9#7: a synchronous
* boundary row + the immediate agent snapshot can share a createdAt). A merged
* sample is human unless EVERY member is an agent, so supervision never gets
* mis-attributed to the agent.
*/
function normalize(
rows: TimelineSample[],
config: WorkTimeConfig,
): NormSample[] {
const samples: NormSample[] = [];
for (const row of rows) {
const source = row.lastUpdatedSource;
if (config.excludeGit && source === 'git') continue;
samples.push({
t: toMs(row.createdAt),
isAgent: source === 'agent',
aiChatId: row.lastUpdatedAiChatId ?? null,
kind: row.kind ?? null,
});
}
samples.sort((a, b) => a.t - b.t);
if (config.dedupRoundMs <= 0 || samples.length < 2) return samples;
const deduped: NormSample[] = [];
for (const s of samples) {
const prev = deduped[deduped.length - 1];
if (prev && s.t - prev.t < config.dedupRoundMs) {
// Merge into the previous sample. Human wins the class; keep the earliest
// t; keep a non-null aiChatId if either has one (so a bare boundary row
// does not erase the run id).
prev.isAgent = prev.isAgent && s.isAgent;
prev.aiChatId = prev.aiChatId ?? s.aiChatId;
// Prefer the more specific kind (a real kind over a null/boundary) only
// matters for burst continuation; keep prev.kind (earliest) as-is.
continue;
}
deduped.push({ ...s });
}
return deduped;
}
/**
* Collapse consecutive same-run agent samples into one burst segment (§5.1) so a
* dense burst (8 snapshots in 7 minutes) contributes its wall-clock, not a count
* × block. A burst is broken by any sample NOT continuing the same aiChatId
* agent run: a non-agent sample, a `boundary` (actor transition), or a DIFFERENT
* aiChatId. Only an AGENT-sourced `idle` pulse with the SAME or a null aiChatId
* continues the burst (its label lags the real edit maxWait, well within
* rounding); a user-sourced `idle` (a human supervision pulse) breaks it.
*/
function collapse(samples: NormSample[], config: WorkTimeConfig): Segment[] {
const segments: Segment[] = [];
let burst: { chatId: string | null; tStart: number; tEnd: number } | null =
null;
const flush = () => {
if (!burst) return;
let tEnd = burst.tEnd;
if (config.burstCapMs != null && tEnd - burst.tStart > config.burstCapMs) {
tEnd = burst.tStart + config.burstCapMs;
}
segments.push({ tStart: burst.tStart, tEnd, isAgent: true });
burst = null;
};
for (const s of samples) {
// An agent-sourced idle pulse continues the current agent burst (same or
// null run id). A user-sourced idle (human supervision) must NOT be swallowed
// here — it falls through to the human branch so the session flips to `work`.
if (
burst &&
s.kind === 'idle' &&
s.isAgent &&
(s.aiChatId === burst.chatId || s.aiChatId == null)
) {
burst.tEnd = s.t;
continue;
}
if (s.isAgent && s.kind !== 'boundary') {
if (burst && burst.chatId === s.aiChatId) {
burst.tEnd = s.t;
} else {
flush();
burst = { chatId: s.aiChatId, tStart: s.t, tEnd: s.t };
}
continue;
}
// A human sample, a boundary, or an agent-boundary: breaks the burst and is
// itself a zero-width segment (its class follows its own source).
flush();
segments.push({ tStart: s.t, tEnd: s.t, isAgent: s.isAgent });
}
flush();
return segments;
}
function gapThreshold(
a: Segment,
b: Segment,
config: WorkTimeConfig,
): number {
return a.isAgent && b.isAgent ? config.agentTGap : config.tGap;
}
/** Merge intervals; overlapping OR touching intervals are unioned. */
function unionDuration(intervals: Array<[number, number]>): number {
if (intervals.length === 0) return 0;
const sorted = [...intervals].sort((a, b) => a[0] - b[0]);
let total = 0;
let [curStart, curEnd] = sorted[0];
for (let i = 1; i < sorted.length; i++) {
const [s, e] = sorted[i];
if (s <= curEnd) {
if (e > curEnd) curEnd = e;
} else {
total += curEnd - curStart;
curStart = s;
curEnd = e;
}
}
total += curEnd - curStart;
return total;
}
/**
* #395 core estimate time worked on a page from its history timeline (§5).
* Pure and deterministic: no DB, no clock, no I/O.
*
* Pipeline: normalize+dedup collapse agent bursts ONE sessionization pass
* over all segments (threshold depends on the pair: both-agent agentTGap, else
* tGap; the last session is ALWAYS closed after the loop) class per finished
* session (all-agent agent_only, else work) pad each session (multi-sample
* [firstP_in, last+P_out]; lone scalar [tP_single, t]) clip padding of
* adjacent DIFFERENT-class sessions at the raw-gap midpoint (so work/agent_only
* never overlap) metrics are the union wall-clock within each class (union, not
* Σ, so overlaps never double, and cross-class-disjoint by the clip above).
*/
export function computeWorkTime(
rows: TimelineSample[],
config?: Partial<WorkTimeConfig>,
): WorkTimeResult {
const cfg = resolveWorkTimeConfig(config);
const samples = normalize(rows, cfg);
const segments = collapse(samples, cfg);
// Sessionize — one pass over ALL segments.
const rawSessions: Segment[][] = [];
let cur: Segment[] | null = null;
for (const seg of segments) {
if (cur == null) {
cur = [seg];
} else {
const last = cur[cur.length - 1];
if (seg.tStart - last.tEnd <= gapThreshold(last, seg, cfg)) {
cur.push(seg);
} else {
rawSessions.push(cur);
cur = [seg];
}
}
}
if (cur != null) rawSessions.push(cur); // MUST close the last session (§5, §9#1)
// A finished session with BOTH its raw (unpadded) span and its padded bounds.
// `rawSessions` are already in ascending time order, so `built` is too.
interface BuiltSession {
rawStart: number;
rawEnd: number;
padStart: number;
padEnd: number;
cls: WorkSession['class'];
}
const built: BuiltSession[] = [];
for (const segs of rawSessions) {
const first = segs[0];
const last = segs[segs.length - 1];
const cls = segs.every((s) => s.isAgent) ? 'agent_only' : 'work';
let padStart: number;
let padEnd: number;
if (segs.length === 1 && first.tStart === first.tEnd) {
// Lone single-instant session (one scalar, or a one-snapshot agent run):
// pre-roll only, no invented "future" work (§5).
padStart = first.tStart - cfg.pSingle;
padEnd = first.tStart;
} else {
padStart = first.tStart - cfg.pIn;
padEnd = last.tEnd + cfg.pOut;
}
built.push({
rawStart: first.tStart,
rawEnd: last.tEnd,
padStart,
padEnd,
cls,
});
}
// Clip cross-class padding so a `work` and an `agent_only` session that abut
// never claim the same wall-clock. For each ADJACENT pair of DIFFERENT classes,
// cap the earlier session's trailing pad and the later session's leading pad at
// the MIDPOINT of the raw (unpadded) inactivity gap between them: the earlier
// padded interval then ends ≤ midpoint and the later one starts ≥ midpoint, so
// the two are disjoint (they touch at most at the midpoint). This makes the
// per-class unions (workMs / agentOnlyMs) cross-class-disjoint BY CONSTRUCTION
// — closing the double-count where a work session ending in an agent segment
// and a nearby agent_only session (gap in (agentTGap, pIn+pOut]) overlapped and
// were counted into both metrics (§5, §9). Within-class adjacency is left
// untouched: `unionDuration` already dedups it, and clipping there could perturb
// the per-class metric value.
for (let i = 1; i < built.length; i++) {
const a = built[i - 1];
const b = built[i];
if (a.cls === b.cls) continue;
const midpoint = (a.rawEnd + b.rawStart) / 2;
if (a.padEnd > midpoint) a.padEnd = midpoint;
if (b.padStart < midpoint) b.padStart = midpoint;
}
const sessions: WorkSession[] = [];
const workIvs: Array<[number, number]> = [];
const agentIvs: Array<[number, number]> = [];
for (const s of built) {
sessions.push({ start: s.padStart, end: s.padEnd, class: s.cls });
(s.cls === 'work' ? workIvs : agentIvs).push([s.padStart, s.padEnd]);
}
sessions.sort((a, b) => a.start - b.start);
return {
workMs: unionDuration(workIvs),
agentOnlyMs: unionDuration(agentIvs),
sessions,
};
}
@@ -0,0 +1,15 @@
export { computeWorkTime } from './compute-work-time';
export { bucketByDay, zonedDayStart } from './bucket-by-day';
export {
DEFAULT_WORK_TIME_CONFIG,
resolveWorkTimeConfig,
} from './work-time.config';
export type { WorkTimeConfig } from './work-time.config';
export type {
TimelineSample,
WorkSession,
WorkTimeResult,
SessionClass,
DayWindow,
PerDay,
} from './work-time.types';
@@ -0,0 +1,102 @@
import {
IDLE_MAX_WAIT_USER,
IDLE_MAX_WAIT_AGENT,
} from '../../../collaboration/constants';
/**
* #395 tunables for the work-time estimate (§10). Defaults are calibrated off
* #374's idle-pulse ceilings: after #374 a continuous editing session leaves a
* history row at least every ~IDLE_MAX_WAIT (10m user / 5m agent), so a gap
* WIDER than that ceiling contains un-pulsed idle time = (partial) inactivity.
* `tGap` therefore sits a little above the user ceiling, `agentTGap` a little
* above the agent ceiling a gap within the threshold is pulse-backed and
* counts as work.
*/
export interface WorkTimeConfig {
/** user inactivity timeout: gap ≤ tGap between samples = continuous work. */
tGap: number;
/** timeout for a pair of consecutive agent samples (tighter than tGap). */
agentTGap: number;
/** pre-roll padding for a multi-sample session (work began before sample 1). */
pIn: number;
/** post-roll padding for a multi-sample session (work continued after last). */
pOut: number;
/** block for a lone single-sample session (pre-roll only, no invented future). */
pSingle: number;
/** drop `git`-source samples (they are not human/agent article work). */
excludeGit: boolean;
/** optional cap on one collapsed agent-burst segment's wall-clock (§9#3). */
burstCapMs?: number;
/** samples whose createdAt round to the same bucket dedup to one (§9#7). */
dedupRoundMs: number;
}
export const DEFAULT_WORK_TIME_CONFIG: WorkTimeConfig = {
// ~15m: IDLE_MAX_WAIT_USER (10m) + headroom. Empirically backcast on a real
// 307-snapshot article (≈24h at 15m matched the owner's estimate; 30/45m
// over-counted). See #395 §10.
tGap: 15 * 60 * 1000,
// ~7m: IDLE_MAX_WAIT_AGENT (5m) + headroom.
agentTGap: 7 * 60 * 1000,
pIn: 5 * 60 * 1000,
pOut: 5 * 60 * 1000,
pSingle: 2 * 60 * 1000,
excludeGit: true,
burstCapMs: undefined,
dedupRoundMs: 1000,
};
// Compile-time cross-check that the defaults really are pulse-anchored — if a
// future edit moves the #374 ceilings, this reminds us to re-calibrate.
void IDLE_MAX_WAIT_USER;
void IDLE_MAX_WAIT_AGENT;
/**
* Fill a partial config with defaults and validate it. Cross-class metric
* disjointness is guaranteed jointly by `computeWorkTime`'s adjacent-pair padding
* clip (it caps the padding of adjacent DIFFERENT-class sessions at the raw-gap
* midpoint) AND the two bounds enforced below (§5):
* - `tGap ≥ pIn + pOut`: a session's own padding never exceeds its inactivity
* window.
* - `2·agentTGap ≥ pIn + pOut`: makes the adjacent-only clip provably COMPLETE.
* A NON-adjacent (i, i+2) cross-class overlap could only arise from two
* same-class sessions separated by a full intervening session of the other
* class; that separation spans at least two inter-session gaps, each strictly
* `> agentTGap`, so it is `> 2·agentTGap`. Requiring `2·agentTGap ≥ pIn + pOut`
* means even the widest padded reach (pIn + pOut) cannot bridge it so the
* only cross-class overlaps possible are between ADJACENT sessions, which the
* clip handles. `workMs`/`agentOnlyMs` are therefore disjoint by construction.
*/
export function resolveWorkTimeConfig(
partial?: Partial<WorkTimeConfig>,
): WorkTimeConfig {
const config = { ...DEFAULT_WORK_TIME_CONFIG, ...(partial ?? {}) };
for (const key of [
'tGap',
'agentTGap',
'pIn',
'pOut',
'pSingle',
'dedupRoundMs',
] as const) {
const value = config[key];
if (!Number.isFinite(value) || value < 0) {
throw new Error(`work-time config: ${key} must be a non-negative number`);
}
}
if (config.burstCapMs != null && config.burstCapMs <= 0) {
throw new Error('work-time config: burstCapMs must be > 0 when set');
}
if (config.tGap < config.pIn + config.pOut) {
throw new Error(
"work-time config: tGap must be ≥ pIn + pOut (a session's padding may not exceed its inactivity window)",
);
}
if (2 * config.agentTGap < config.pIn + config.pOut) {
throw new Error(
'work-time config: 2·agentTGap must be ≥ pIn + pOut (so non-adjacent cross-class padding cannot overlap)',
);
}
return config;
}
@@ -0,0 +1,73 @@
/**
* #395 "time worked on an article" domain types.
*
* The estimate is built by sessionizing a page's `page_history` timeline on
* inactivity gaps (WakaTime-style), NOT by taking the span between the first and
* last edit (which over-counts sleep / lunch / idle days). See the design doc in
* issue #395 §5§6.3 for the normative algorithm.
*/
/**
* A single `page_history` row projected for the work-time computation the
* cheap columns only (no `content`). Produced by
* `PageHistoryRepo.findTimelineByPageId`. `createdAt` is whatever the DB driver
* hands back (Date); the pure core normalizes it to epoch-ms itself so it stays
* deterministic and DB-free.
*/
export interface TimelineSample {
createdAt: Date | string | number;
lastUpdatedById: string | null;
/** 'user' | 'agent' | 'git' | null (legacy autosave = human). */
lastUpdatedSource: string | null;
lastUpdatedAiChatId: string | null;
/** #370 tier: 'manual' | 'agent' | 'idle' | 'boundary' | null (legacy). */
kind: string | null;
}
/** A finished session's class (§5.1). */
export type SessionClass = 'work' | 'agent_only';
/**
* A finished session: absolute wall-clock bounds already padded with P_in/P_out
* (multi-sample) or P_single (single scalar), plus its class. This is enough for
* both the metrics and the per-day punch-card colouring.
*/
export interface WorkSession {
/** epoch-ms, inclusive lower bound (already P-padded). */
start: number;
/** epoch-ms, exclusive upper bound (already P-padded). */
end: number;
class: SessionClass;
}
/** Output of {@link computeWorkTime}. */
export interface WorkTimeResult {
/** union wall-clock of `work` sessions, ms (the headline metric). */
workMs: number;
/** union wall-clock of `agent_only` sessions, ms (secondary). */
agentOnlyMs: number;
sessions: WorkSession[];
}
/** One activity window inside a calendar day (already clipped to the day). */
export interface DayWindow {
/** epoch-ms. */
start: number;
/** epoch-ms. */
end: number;
class: SessionClass;
}
/** One calendar day of the punch-card (§6.3). */
export interface PerDay {
/** epoch-ms of the local-midnight day start in the requested tz. */
day: number;
/** 'YYYY-MM-DD' in the requested tz — stable, tz-independent label. */
dayISO: string;
/** Σ of `work` windows this day, ms. Σ over days == workMs (invariant §6.3). */
activeMs: number;
/** Σ of `agent_only` windows this day, ms (drawn, NOT in activeMs). */
agentMs: number;
/** both classes, clipped to the day, sorted by start (for drawing). */
windows: DayWindow[];
}
@@ -157,6 +157,44 @@ export class PageHistoryRepo {
return { ...result, items: result.items.map(attachPageHistoryAgent) };
}
/**
* #395 cheap projection of a page's FULL history timeline for the work-time
* estimate: only the columns the sessionizer needs, no heavy `content`, sorted
* oldestnewest. The secondary `id` tie-break keeps rows sharing a `createdAt`
* (e.g. a synchronous pre-agent boundary row + the immediate agent snapshot)
* in a deterministic order.
*/
async findTimelineByPageId(
pageId: string,
trx?: KyselyTransaction,
): Promise<
Array<
Pick<
PageHistory,
| 'createdAt'
| 'lastUpdatedById'
| 'lastUpdatedSource'
| 'lastUpdatedAiChatId'
| 'kind'
>
>
> {
const db = dbOrTx(this.db, trx);
return db
.selectFrom('pageHistory')
.select([
'createdAt',
'lastUpdatedById',
'lastUpdatedSource',
'lastUpdatedAiChatId',
'kind',
])
.where('pageId', '=', pageId)
.orderBy('createdAt', 'asc')
.orderBy('id', 'asc')
.execute();
}
async findPageLastHistory(
pageId: string,
opts?: {
@@ -1,70 +0,0 @@
import { INTERNAL_LINK_REGEX } from './utils';
import { isInternalPagePath } from '@docmost/prosemirror-markdown';
/**
* Cross-package DRIFT GUARD for the internal-link subset invariant (#522).
*
* The client-side `isInternalPagePath`
* (`packages/prosemirror-markdown/src/lib/internal-links.ts`) promotes a markdown
* link to `internal: true` on import. Every link it marks internal MUST be one
* the server would backlink and export-rewrite i.e. the client matcher MUST be
* a STRICT SUBSET of the server's canonical `INTERNAL_LINK_REGEX`
* (`./utils.ts`). If the client ever accepts a path the server rejects, that link
* is stored internal but silently dropped from the backlink graph and broken on
* export the exact bug #522 fixed.
*
* This spec is the load-bearing guard: it imports the LIVE server regex AND the
* LIVE `isInternalPagePath` (no hand-copied regex on either side). A narrowing of
* EITHER most dangerously the server regex reddens here. The in-package
* accept/reject test documents the client's behaviour but cannot see the server
* regex; this top-layer spec is what makes the subset relation mechanical
* (AGENTS.md rule #7: a CI test that fails on drift of the source of truth).
*/
describe('internal-link subset parity (client isInternalPagePath ⊆ server INTERNAL_LINK_REGEX)', () => {
// Every path the CLIENT accepts. Kept deliberately broad across the risky
// dimensions — the slug charset (digits, hyphens, mixed case, leading/trailing
// hyphen), the space charset, and the optional trailing slash — so a narrowing
// of the server regex on any of them reddens the subset assertion below.
const CLIENT_ACCEPTS = [
'/s/eng/p/abc123',
'/s/eng/p/abc123/', // trailing slash
'/s/My-Space/p/A-b-C-9', // hyphens + mixed case in both segments
'/s/x/p/z', // shortest
'/s/eng/p/my-page-abc123', // slug with hyphens (extractPageSlugId shape)
'/s/eng/p/-lead', // leading hyphen in slug
'/s/eng/p/trail-', // trailing hyphen in slug
'/s/eng/p/0123456789', // all-digit slug
'/s/a.b/p/abc', // dot in the SPACE segment (space charset is [^/]+)
'/s/space with space/p/abc', // space char in the SPACE segment
];
it('every corpus path is actually client-accepted (guards the corpus itself)', () => {
// If a path here stopped being client-accepted the subset test would pass
// vacuously; assert acceptance up front so the corpus stays meaningful. The
// filter-to-empty form names the offending paths on failure.
const notAccepted = CLIENT_ACCEPTS.filter((h) => !isInternalPagePath(h));
expect(notAccepted).toEqual([]);
});
it('every client-accepted path also matches the LIVE server regex (subset)', () => {
// The mechanical drift guard: narrow the server INTERNAL_LINK_REGEX and at
// least one hyphen/charset/structure case appears here.
const notInServer = CLIENT_ACCEPTS.filter((h) => !INTERNAL_LINK_REGEX.test(h));
expect(notInServer).toEqual([]);
});
it('the client rejects forbidden-slug-char paths the server also rejects', () => {
// Documents the subset BOUNDARY: correct shape, forbidden slug char. The
// client and the LIVE server regex must agree on rejection.
const forbidden = [
'/s/eng/p/abc.def',
'/s/eng/p/abc_def',
'/s/eng/p/abc%20',
'/s/eng/p/abc~x',
];
const clientAccepts = forbidden.filter((h) => isInternalPagePath(h));
const serverAccepts = forbidden.filter((h) => INTERNAL_LINK_REGEX.test(h));
expect(clientAccepts).toEqual([]);
expect(serverAccepts).toEqual([]);
});
});
@@ -0,0 +1,131 @@
// p-limit and @sindresorhus/slugify are ESM-only and not in jest's transform
// allowlist; both are irrelevant to createDrawioSvg (a pure fs + string method),
// so they are mocked out to keep the module graph loadable under ts-jest.
jest.mock('p-limit', () => ({
__esModule: true,
default: () => (fn: () => unknown) => fn(),
}));
jest.mock('@sindresorhus/slugify', () => ({
__esModule: true,
default: (input: string) => String(input),
}));
import { promises as fs } from 'fs';
import * as os from 'os';
import * as path from 'path';
import { ImportAttachmentService } from './import-attachment.service';
/**
* Unit test for ImportAttachmentService.createDrawioSvg (issue #507).
*
* The Confluence import wraps a `.drawio` file into a `.drawio.svg` attachment.
* The `content=` payload MUST be the mxfile XML entity-escaped (draw.io's native
* form), NOT base64 draw.io's editor decodes a base64 content= via Latin-1
* atob, mangling every non-ASCII char into mojibake. createDrawioSvg touches no
* injected dependency, so the service is built with placeholder deps.
*/
describe('ImportAttachmentService.createDrawioSvg (#507)', () => {
const service = new ImportAttachmentService(
{} as any,
{} as any,
{} as any,
);
const call = (p: string): Promise<Buffer> =>
(service as any).createDrawioSvg(p);
let tmpDir: string;
beforeAll(async () => {
tmpDir = await fs.mkdtemp(path.join(os.tmpdir(), 'drawio-507-'));
});
afterAll(async () => {
await fs.rm(tmpDir, { recursive: true, force: true });
});
const writeDrawio = async (name: string, xml: string): Promise<string> => {
const p = path.join(tmpDir, name);
await fs.writeFile(p, xml, 'utf-8');
return p;
};
it('writes content= as entity-encoded XML, not base64', async () => {
const drawio =
'<mxfile host="Confluence"><diagram name="Схема — ёж">' +
'<mxGraphModel><root><mxCell id="0"/>' +
'<mxCell id="2" value="Старт-бит" vertex="1" parent="0"/>' +
'</root></mxGraphModel></diagram></mxfile>';
const p = await writeDrawio('cyrillic.drawio', drawio);
const svg = (await call(p)).toString('utf-8');
const content = /content="([^"]*)"/.exec(svg)?.[1];
expect(content).toBeDefined();
// Entity-encoded XML form, starting with &lt;mxfile — never a base64 blob.
expect(content).toMatch(/^&lt;mxfile/);
expect(content).toContain('&lt;');
// Non-ASCII survives as raw UTF-8, with no Latin-1 mojibake.
expect(content).toContain('Старт-бит');
expect(content).toContain('Схема — ёж');
expect(content).not.toContain('Ð');
// Decoding the attribute (un-escaping) yields the original drawio file.
const decoded = content!
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&quot;/g, '"')
.replace(/&amp;/g, '&');
expect(decoded).toBe(drawio);
});
it('encodes literal tab/newline/CR as numeric char-refs, not literal control chars (#507 F1)', async () => {
// A literal tab/newline/CR inside the mxfile XML would be collapsed to a
// single space by XML attribute-value normalization when the draw.io editor
// reads content=, silently flattening multi-line labels and tab-bearing
// values. They must be emitted as numeric char-refs instead.
const drawio =
'<mxfile><diagram name="p">' +
'<mxGraphModel><root><mxCell id="0"/>' +
'<mxCell id="2" value="col1\tcol2" style="html=1;\nshadow=0" vertex="1" parent="0"/>' +
'<mxCell id="3" value="Line1\nLine2\rLine3" vertex="1" parent="0"/>' +
'</root></mxGraphModel></diagram></mxfile>';
const p = await writeDrawio('ctrl.drawio', drawio);
const svg = (await call(p)).toString('utf-8');
const content = /content="([^"]*)"/.exec(svg)?.[1];
expect(content).toBeDefined();
// No literal control chars survive in the attribute value.
expect(content).not.toMatch(/[\t\n\r]/);
// They round-trip as numeric char-refs.
expect(content).toContain('&#x9;');
expect(content).toContain('&#xa;');
expect(content).toContain('&#xd;');
// Decoding (char-refs back to literal, entities back) recovers the file.
const decoded = content!
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'")
.replace(/&#x9;/gi, '\t')
.replace(/&#xa;/gi, '\n')
.replace(/&#xd;/gi, '\r')
.replace(/&amp;/g, '&');
expect(decoded).toBe(drawio);
});
it('escapes XML metacharacters in the drawio payload', async () => {
const drawio = '<mxfile><diagram name="a &amp; b">"q" &lt;x&gt;</diagram></mxfile>';
const p = await writeDrawio('meta.drawio', drawio);
const svg = (await call(p)).toString('utf-8');
const content = /content="([^"]*)"/.exec(svg)?.[1];
expect(content).toBeDefined();
// The attribute value must contain no bare `<`, `>` or `"` that would break
// out of the content="..." attribute or the SVG element.
expect(content).not.toMatch(/[<>"]/);
const decoded = content!
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&quot;/g, '"')
.replace(/&amp;/g, '&');
expect(decoded).toBe(drawio);
});
});
@@ -8,6 +8,7 @@ import { createReadStream } from 'node:fs';
import { promises as fs } from 'fs';
import { Readable } from 'stream';
import { getMimeType, sanitizeFileName } from '../../../common/helpers';
import { htmlEscape } from '../../../common/helpers/html-escaper';
import { v7 } from 'uuid';
import { FileTask } from '@docmost/db/types/entity.types';
import { getAttachmentFolderPath } from '../../../core/attachment/attachment.utils';
@@ -849,7 +850,12 @@ export class ImportAttachmentService {
): Promise<Buffer> {
try {
const drawioContent = await fs.readFile(drawioPath, 'utf-8');
const drawioBase64 = Buffer.from(drawioContent).toString('base64');
// Write the mxfile XML XML-entity-escaped (draw.io's native content= form),
// NOT base64. draw.io's editor decodes a base64 content= via Latin-1 atob
// (no UTF-8 step), turning every non-ASCII char (Cyrillic, ё, —) into
// mojibake; the entity-encoded form is decoded by the DOM as UTF-8 and
// opens intact. Docmost's own decoder reads both forms.
const drawioEscaped = this.xmlEscapeContent(drawioContent);
let imageElement = '';
// If we have a PNG, include it in the SVG
@@ -875,7 +881,7 @@ export class ImportAttachmentService {
width="600"
height="400"
viewBox="0 0 600 400"
content="${drawioBase64}">${imageElement}</svg>`;
content="${drawioEscaped}">${imageElement}</svg>`;
return Buffer.from(svgContent, 'utf-8');
} catch (error) {
@@ -884,6 +890,24 @@ export class ImportAttachmentService {
}
}
/**
* Escape a string so it is safe as the value of a double-quoted XML attribute
* (the `content=` payload of a `.drawio.svg`). The shared `htmlEscape` covers
* `& < > " '` (a strict superset of what this attribute needs; the extra `'`
* escape is harmless in a `"`-delimited value). On top of that, the numeric
* char-refs for tab/newline/CR are required: a literal tab/newline/CR inside
* an attribute value is collapsed to a single space by XML attribute-value
* normalization on DOM read (both our decoder and the real draw.io editor),
* silently flattening multi-line labels and tab-bearing values. Char-refs
* survive that normalization (#507).
*/
private xmlEscapeContent(s: string): string {
return htmlEscape(s)
.replace(/\t/g, '&#x9;')
.replace(/\n/g, '&#xa;')
.replace(/\r/g, '&#xd;');
}
private async uploadWithRetry(opts: {
abs: string;
storageFilePath: string;
@@ -322,20 +322,21 @@ describe('AiChatService.stream [integration]', () => {
});
/**
* #332 deferred tool loading, the ON path. The riskiest property is that the
* per-turn `activatedTools` Set is created FRESH inside each stream() call, so a
* tool a previous turn activated via loadTools is NOT still active when the next
* turn starts the new turn begins "cold" (CORE + loadTools only). The unit
* tests only exercise pure prepareAgentStep with hand-fed Sets; this pins the
* real wiring end-to-end (loadTools.execute -> activatedTools -> prepareStep ->
* per-step activeTools) against the real streamText loop, and proves there is no
* cross-turn leak. We drive a MockLanguageModelV3 whose step 1 calls
* loadTools(['createPage']) and assert, via the model's recorded per-step
* CallOptions.tools (the AI SDK filters the provider tool list by activeTools),
* that the deferred tool becomes active on the SAME turn's next step but NOT on a
* fresh turn's first step.
* #332 + #490 deferred tool loading, the ON path. Turn 1 starts COLD (CORE +
* loadTools only) and activates a deferred tool via loadTools; that activation
* is PERSISTED into the chat's metadata.activatedTools (#490) so the NEXT turn
* SEEDS from it and the tool is active from the fresh turn's FIRST step the
* model never re-runs loadTools to re-activate the same tool. The unit tests
* only exercise pure prepareAgentStep with hand-fed Sets; this pins the real
* wiring end-to-end (loadTools.execute -> activatedTools -> persist -> next-turn
* seed -> prepareStep -> per-step activeTools) against the real streamText loop.
* We drive a MockLanguageModelV3 whose step 1 calls loadTools(['createPage'])
* and assert, via the model's recorded per-step CallOptions.tools (the AI SDK
* filters the provider tool list by activeTools), that the deferred tool becomes
* active on the SAME turn's next step AND, seeded from metadata, on the next
* turn's first step.
*/
describe('deferred tool loading ON — per-turn activation, no leak (#332)', () => {
describe('deferred tool loading ON — cross-turn activation persistence (#332 + #490)', () => {
// A stub deferred (non-core) tool the agent can activate. Its execute is never
// called — the model only needs to SEE it become active — but it must be a
// valid AI-SDK tool so the SDK includes it in a step's tool list once active.
@@ -451,7 +452,7 @@ describe('AiChatService.stream [integration]', () => {
} as any);
}
it('activates a deferred tool for the SAME turn, and a NEW turn starts cold (no leak)', async () => {
it('activates a deferred tool for the SAME turn, and a NEW turn SEEDS it from persisted chat metadata (#490)', async () => {
const chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
// --- Turn 1: loadTools(createPage) on step 1, then answer on step 2. ---
@@ -474,7 +475,7 @@ describe('AiChatService.stream [integration]', () => {
// Step 2 of the SAME turn sees the just-activated deferred tool.
expect(step2Tools).toContain('createPage');
// --- Turn 2 on the SAME chat: must start cold again. ---
// --- Turn 2 on the SAME chat: seeds the persisted activation (#490). ---
const model2 = new MockLanguageModelV3({
doStream: async () => ({ stream: successStream() }),
} as any);
@@ -485,9 +486,10 @@ describe('AiChatService.stream [integration]', () => {
const nextTurnFirstStep = toolNames(model2.doStreamCalls[0]);
expect(nextTurnFirstStep).toContain('loadTools');
// The activated set is per-turn: the prior turn's createPage did NOT leak,
// so the fresh turn's first step sees it deferred again.
expect(nextTurnFirstStep).not.toContain('createPage');
// #490: activation PERSISTS across turns — turn 1 wrote createPage into the
// chat's metadata.activatedTools, so the next turn seeds from it and the
// deferred tool is active from the FIRST step (no need to re-run loadTools).
expect(nextTurnFirstStep).toContain('createPage');
});
});
});
@@ -0,0 +1,137 @@
import { randomUUID } from 'node:crypto';
import { Kysely } from 'kysely';
import { PageHistoryRepo } from '../../src/database/repos/page/page-history.repo';
import { PageHistoryService } from '../../src/core/page/services/page-history.service';
import { computeWorkTime } from '../../src/core/page/work-time';
import {
getTestDb,
destroyTestDb,
createWorkspace,
createSpace,
createPage,
createUser,
createChat,
} from './db';
/**
* #395 real-Postgres coverage for the work-time timeline projection and the
* service that computes the estimate. The pure sessionizer is unit-tested
* exhaustively (compute-work-time.spec.ts); this asserts the SQL projection
* (right rows, ASC, no `content`) and that the service's numbers agree with the
* pure core over the exact rows the DB returns.
*/
describe('PageHistory work-time [integration]', () => {
let db: Kysely<any>;
let repo: PageHistoryRepo;
let service: PageHistoryService;
let workspaceId: string;
let spaceId: string;
let pageId: string;
let userId: string;
let chatId: string;
const MIN = 60 * 1000;
beforeAll(async () => {
db = getTestDb();
repo = new PageHistoryRepo(db as any);
service = new PageHistoryService(repo);
workspaceId = (await createWorkspace(db)).id;
spaceId = (await createSpace(db, workspaceId)).id;
pageId = (await createPage(db, { workspaceId, spaceId })).id;
userId = (await createUser(db, workspaceId)).id;
chatId = (await createChat(db, { workspaceId, creatorId: userId })).id;
});
afterAll(async () => {
await destroyTestDb();
});
async function insertHistory(rows: Array<{
createdAt: string;
source: string | null;
chat?: string | null;
kind?: string | null;
content?: unknown;
}>) {
for (const r of rows) {
await db
.insertInto('pageHistory')
.values({
id: randomUUID(),
pageId,
spaceId,
workspaceId,
title: 'x',
content: r.content ?? { type: 'doc', content: [] },
lastUpdatedById: userId,
lastUpdatedSource: r.source,
lastUpdatedAiChatId: r.chat ?? null,
kind: r.kind ?? null,
createdAt: new Date(r.createdAt),
})
.execute();
}
}
it('findTimelineByPageId projects the cheap columns, ASC, without content', async () => {
await insertHistory([
{ createdAt: '2026-07-04T19:54:00Z', source: 'user', kind: 'manual' },
{ createdAt: '2026-07-04T03:40:00Z', source: 'user', kind: null },
{ createdAt: '2026-07-04T15:43:00Z', source: 'agent', chat: chatId, kind: 'agent' },
]);
const timeline = await repo.findTimelineByPageId(pageId);
expect(timeline).toHaveLength(3);
// Sorted oldest → newest.
const times = timeline.map((r) => new Date(r.createdAt).getTime());
expect(times).toEqual([...times].sort((a, b) => a - b));
// Projection carries exactly the sessionizer's inputs, and NO content.
for (const row of timeline) {
expect(row).toHaveProperty('createdAt');
expect(row).toHaveProperty('lastUpdatedById');
expect(row).toHaveProperty('lastUpdatedSource');
expect(row).toHaveProperty('lastUpdatedAiChatId');
expect(row).toHaveProperty('kind');
expect(row).not.toHaveProperty('content');
}
// Agent row keeps its provenance.
const agent = timeline.find((r) => r.lastUpdatedSource === 'agent');
expect(agent?.lastUpdatedAiChatId).toBe(chatId);
});
it('service estimate matches the pure core and satisfies Σ perDay == workMs', async () => {
const rows = await repo.findTimelineByPageId(pageId);
const pure = computeWorkTime(rows);
const result = await service.computeWorkTime(pageId, 'UTC');
expect(result.workMs).toBe(pure.workMs);
expect(result.agentOnlyMs).toBe(pure.agentOnlyMs);
expect(result.tz).toBe('UTC');
expect(result.config.tGap).toBe(15 * MIN);
const sumActive = result.perDay.reduce((a, d) => a + d.activeMs, 0);
expect(sumActive).toBe(result.workMs);
// The 3 seeded rows sessionize into ≤ their span; not the naive span.
const naive =
new Date('2026-07-04T19:54:00Z').getTime() -
new Date('2026-07-04T03:40:00Z').getTime();
expect(result.workMs).toBeGreaterThan(0);
expect(result.workMs).toBeLessThan(naive);
});
it('an unknown timezone surfaces as a RangeError (controller maps to 400)', async () => {
await expect(
service.computeWorkTime(pageId, 'Not/AZone'),
).rejects.toBeInstanceOf(RangeError);
});
it('a page with no history → zeros, no days', async () => {
const emptyPage = (await createPage(db, { workspaceId, spaceId })).id;
const result = await service.computeWorkTime(emptyPage, 'UTC');
expect(result.workMs).toBe(0);
expect(result.agentOnlyMs).toBe(0);
expect(result.perDay).toEqual([]);
});
});
+36 -9
View File
@@ -178,10 +178,11 @@ function sliceModel(xml: string): string | null {
// --- decode chain ----------------------------------------------------------
/**
* Read the `content=` attribute out of a `.drawio.svg` string. Docmost stores a
* base64 payload there (createDrawioSvg); draw.io's own SVG export may store the
* XML entity-encoded instead. The DOM decodes entities for us, so the caller
* only has to distinguish "starts with '<'" (raw XML) from base64.
* Read the `content=` attribute out of a `.drawio.svg` string. Docmost writes
* the mxfile XML entity-encoded there (buildDrawioSvg / createDrawioSvg), which
* is also how draw.io's own SVG export stores it; older attachments stored a
* base64 payload instead. The DOM decodes entities for us, so the caller only
* has to distinguish "starts with '<'" (raw XML) from base64.
*/
export function extractContentAttr(svg: string): string {
const { doc, error } = parseXml(svg);
@@ -195,12 +196,23 @@ export function extractContentAttr(svg: string): string {
// value itself never contains a double-quote (base64 / entity-encoded XML).
const m = /content="([^"]*)"/.exec(svg);
if (m) {
// Decode the handful of XML entities a raw regex would leave encoded.
// Decode the handful of XML entities a raw regex would leave encoded. The
// numeric char-refs for tab/newline/CR MUST be decoded here too: the DOM
// path above turns them back into the literal control chars, so this
// regex fallback has to agree or the two decode paths diverge (#507).
// `&amp;` is decoded last so an escaped `&amp;#x9;` reads back as the
// literal text `&#x9;`, not a tab.
return m[1]
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&#39;/g, "'")
.replace(/&#x9;/gi, "\t")
.replace(/&#9;/g, "\t")
.replace(/&#xa;/gi, "\n")
.replace(/&#10;/g, "\n")
.replace(/&#xd;/gi, "\r")
.replace(/&#13;/g, "\r")
.replace(/&amp;/g, "&");
}
throw new Error("drawio: SVG has no content= attribute to decode");
@@ -307,9 +319,16 @@ export function encodeDrawioFile(modelXml: string, title = "Page-1"): string {
/**
* Build the `diagram.drawio.svg` attachment. Mirrors the import service's
* createDrawioSvg contract exactly:
* <svg xmlns= xmlns:xlink= content="${base64(drawioFile)}">${inner}</svg>
* <svg xmlns= xmlns:xlink= content="${xmlEscape(drawioFile)}">${inner}</svg>
* plus width/height/viewBox from the diagram bounding box and the schematic
* preview as the visible children (`inner`).
*
* The `content=` value is the mxfile XML XML-entity-escaped (draw.io's own
* native form), NOT base64. draw.io's editor decodes a base64 content= via
* Latin-1 atob (no UTF-8 step), turning every non-ASCII char (e.g. Cyrillic,
* ё, ) into mojibake; the entity-encoded form is decoded by the DOM as UTF-8
* and opens intact. Our decoder (decodeDrawioSvg) reads both forms, so old
* base64 attachments still round-trip.
*/
export function buildDrawioSvg(
modelXml: string,
@@ -318,14 +337,14 @@ export function buildDrawioSvg(
title = "Page-1",
): string {
const file = encodeDrawioFile(modelXml, title);
const base64 = Buffer.from(file, "utf-8").toString("base64");
const content = xmlEscape(file);
const w = Math.max(1, Math.round(bbox.width));
const h = Math.max(1, Math.round(bbox.height));
return (
`<svg xmlns="http://www.w3.org/2000/svg" ` +
`xmlns:xlink="http://www.w3.org/1999/xlink" ` +
`width="${w}" height="${h}" viewBox="0 0 ${w} ${h}" ` +
`content="${base64}">${inner}</svg>`
`content="${content}">${inner}</svg>`
);
}
@@ -334,7 +353,15 @@ function xmlEscape(s: string): string {
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;");
.replace(/"/g, "&quot;")
// A literal tab/newline/CR inside an attribute value is collapsed to a
// single space by XML attribute-value normalization on DOM read (both jsdom
// here and the real draw.io editor), silently flattening multi-line labels
// and tab-bearing values. Numeric char-refs survive that normalization, so
// emit them the way draw.io's own native export does (#507).
.replace(/\t/g, "&#x9;")
.replace(/\n/g, "&#xa;")
.replace(/\r/g, "&#xd;");
}
// --- normalization + hash --------------------------------------------------
+123 -2
View File
@@ -51,6 +51,14 @@ const hasRule = (issues, rule, cellId) =>
(i) => i.rule === rule && (cellId === undefined || i.cellId === cellId),
);
// Reverse the attribute-value XML escaping used in the `content=` payload.
const unescapeAttr = (s) =>
s
.replace(/&lt;/g, "<")
.replace(/&gt;/g, ">")
.replace(/&quot;/g, '"')
.replace(/&amp;/g, "&");
// --- style parsing ---------------------------------------------------------
test("parseStyle: base stylename + key=value pairs", () => {
@@ -335,16 +343,20 @@ test("encode/build: a title with < > \" & round-trips without corrupting the SVG
// The outer content="..." attribute must not be broken by the title: the raw
// title metacharacters never appear literally in the SVG markup (they are
// base64-encoded inside content=, and escaped inside the file XML).
// entity-escaped inside content=, doubly so where the file XML already escaped
// them inside name="...").
const contentMatch = /content="([^"]*)"/.exec(svg);
assert.ok(contentMatch, "SVG has a single well-formed content= attribute");
// The diagram model still decodes losslessly despite the exotic title.
assert.equal(decodeDrawioSvg(svg), model);
// content= is now entity-encoded XML (draw.io's native form), never base64.
assert.match(contentMatch[1], /^&lt;mxfile/);
// The file XML is well-formed: the title lives in name="..." as escaped
// entities, so unescaping recovers the original title byte-for-byte.
const fileXml = Buffer.from(contentMatch[1], "base64").toString("utf-8");
const fileXml = unescapeAttr(contentMatch[1]);
const nameMatch = /<diagram id="[^"]*" name="([^"]*)">/.exec(fileXml);
assert.ok(nameMatch, "the diagram name attribute is intact and quote-safe");
const decodedTitle = nameMatch[1]
@@ -358,3 +370,112 @@ test("encode/build: a title with < > \" & round-trips without corrupting the SVG
const file = encodeDrawioFile(model, title);
assert.match(file, /name="A &lt; B &gt; C &quot; D &amp; E">/);
});
// --- #507: content= is entity-encoded XML, never base64 --------------------
// A model whose cell values carry Cyrillic, ё and an em dash — exactly the
// characters draw.io's Latin-1 atob mangles when content= is base64.
const CYRILLIC_MODEL =
"<mxGraphModel><root>" +
'<mxCell id="0"/><mxCell id="1" parent="0"/>' +
'<mxCell id="2" value="Старт-бит — ёж" style="rounded=1;html=1;" vertex="1" parent="1">' +
'<mxGeometry x="100" y="100" width="120" height="60" as="geometry"/></mxCell>' +
"</root></mxGraphModel>";
test("#507: buildDrawioSvg writes content= as entity-encoded XML (not base64)", () => {
const model = normalizeXml(CYRILLIC_MODEL);
const svg = buildDrawioSvg(model, "<g/>", { width: 200, height: 120 }, "Диаграмма");
const contentMatch = /content="([^"]*)"/.exec(svg);
assert.ok(contentMatch, "SVG has a single well-formed content= attribute");
const content = contentMatch[1];
// The content= value is the entity-encoded mxfile XML — starts with `&lt;mxfile`.
assert.match(content, /^&lt;mxfile/, "content= is entity-encoded mxfile XML");
// It must NOT be a base64 blob: base64 has no XML entities and no literal `<`.
assert.ok(content.includes("&lt;"), "content= carries XML entities, not base64");
// Cyrillic / ё / — survive verbatim in the attribute (raw UTF-8, not atob-mangled).
assert.ok(content.includes("Старт-бит — ёж"), "non-ASCII value is raw UTF-8 in content=");
assert.ok(content.includes("Диаграмма"), "non-ASCII title is raw UTF-8 in content=");
// The mojibake that base64+atob would have produced must be absent.
assert.ok(!content.includes("Ð"), "no Latin-1 mojibake in content=");
});
test("#507: Cyrillic model round-trips byte-stable through buildDrawioSvg -> decodeDrawioSvg", () => {
const model = normalizeXml(CYRILLIC_MODEL);
const svg = buildDrawioSvg(model, "<g/>", { width: 200, height: 120 }, "Заголовок — ё");
assert.equal(decodeDrawioSvg(svg), model);
});
test("#507 back-compat: an OLD base64-form .drawio.svg still decodes losslessly", () => {
const model = normalizeXml(CYRILLIC_MODEL);
// Reproduce the pre-fix write path: encodeDrawioFile -> base64 in content=.
const file = encodeDrawioFile(model, "Старая диаграмма");
const base64 = Buffer.from(file, "utf-8").toString("base64");
const svg =
`<svg xmlns="http://www.w3.org/2000/svg" content="${base64}"><g/></svg>`;
// No XML entities, purely base64 alphabet — this is the legacy form.
assert.ok(!base64.includes("<") && !base64.includes("&"));
assert.equal(decodeDrawioSvg(svg), model);
});
test("#507 negative: non-ASCII in a cell value AND in the title round-trip clean", () => {
const model = normalizeXml(CYRILLIC_MODEL);
const title = "Тест — ёмкость № 5";
const svg = buildDrawioSvg(model, "<g/>", { width: 200, height: 120 }, title);
// Model recovered byte-for-byte.
assert.equal(decodeDrawioSvg(svg), model);
// Title lands in the <diagram name="..."> of the decoded file XML, intact.
const content = /content="([^"]*)"/.exec(svg)[1];
const fileXml = unescapeAttr(content);
const nameMatch = /<diagram id="[^"]*" name="([^"]*)">/.exec(fileXml);
assert.ok(nameMatch, "diagram name attribute present");
assert.equal(unescapeAttr(nameMatch[1]), title);
});
// --- #507 F1: literal tab/newline/CR in an attribute value survive the
// content= round-trip. The whole mxfile XML lives in one content="..." attr;
// XML attribute-value normalization collapses a LITERAL tab/newline/CR to a
// single space on DOM read, so the escape must emit numeric char-refs instead.
const CTRL_MODEL =
"<mxGraphModel><root>" +
'<mxCell id="0"/><mxCell id="1" parent="0"/>' +
'<mxCell id="2" value="col1\tcol2" style="html=1;\nshadow=0" vertex="1" parent="1">' +
'<mxGeometry x="10" y="10" width="120" height="60" as="geometry"/></mxCell>' +
'<mxCell id="3" value="Line1\nLine2\rLine3" vertex="1" parent="1">' +
'<mxGeometry x="10" y="100" width="120" height="60" as="geometry"/></mxCell>' +
"</root></mxGraphModel>";
test("#507 F1: literal tab/newline/CR in a value round-trip byte-stable (DOM decode)", () => {
const svg = buildDrawioSvg(CTRL_MODEL, "<g/>", { width: 200, height: 200 });
const content = /content="([^"]*)"/.exec(svg)[1];
// The tab/newline/CR must be emitted as numeric char-refs, never as literal
// control chars (which DOM attribute-value normalization would eat).
assert.ok(
!/[\t\n\r]/.test(content),
"no literal tab/newline/CR survive in the content= attribute",
);
assert.ok(
content.includes("&#x9;") &&
content.includes("&#xa;") &&
content.includes("&#xd;"),
"tab/newline/CR are emitted as numeric char-refs",
);
// Full DOM decode recovers the model byte-for-byte, control chars intact.
assert.equal(decodeDrawioSvg(svg), CTRL_MODEL);
});
test("#507 F1: regex fallback decodes tab/newline/CR char-refs (agrees with DOM path)", () => {
const svg = buildDrawioSvg(CTRL_MODEL, "<g/>", { width: 200, height: 200 });
const content = /content="([^"]*)"/.exec(svg)[1];
// A bare `&` makes the SVG wrapper malformed, forcing extractContentAttr onto
// its regex fallback branch. That branch must decode the tab/newline/CR
// char-refs exactly like the DOM path, or the two decoders diverge.
const malformedSvg = `<svg content="${content}">&</svg>`;
assert.equal(decodeDrawioSvg(malformedSvg), CTRL_MODEL);
// The well-formed (DOM) path yields the identical result.
assert.equal(decodeDrawioSvg(svg), CTRL_MODEL);
});
@@ -25,22 +25,13 @@
* drop any attr whose value equals its known schema default. A non-default
* value (e.g. `orderedList.start: 5`) is NOT a default, so it is KEPT.
*
* Every entry below with the single documented exception of the `link.internal`
* marker (see its own bullet) was read from `packages/docmost-client/src/lib/
* Every entry below was read from `packages/docmost-client/src/lib/
* docmost-schema.ts` (the line refs are the exact `default:` declarations) and
* confirmed to be materialized by an exportimportexport round-trip:
* - mark `link` target / rel DocmostAttributes + StarterKit link.
* StarterKit's link extension defaults `target: "_blank"` and
* `rel: "noopener noreferrer nofollow"`; both materialize on import
* (empirically confirmed) even when the source had only `href`.
* - mark `link` internal (#522) the ONE editor-sourced, NOT-import-
* materialized default here. Its source is `editor-ext/src/lib/link.ts`
* (default `internal: false`), not docmost-schema, and import does NOT
* materialize it (an imported external link leaves `internal` absent/null,
* never `false`). It is listed so that the editor's stored `internal:false`
* normalizes to the same "external" canon as absent/null (`false ≡ absent`),
* keeping a stored external link canonically equal to its re-import. The
* load-bearing `internal:true` is NON-default and therefore KEPT.
* - mark `comment` resolved docmost-schema.ts L213-214 (`default: false`).
* - node `orderedList` start provided by StarterKit's orderedList
* (`default: 1`); materializes on import (empirically confirmed).
@@ -65,14 +56,6 @@ const KNOWN_DEFAULTS: Record<string, Record<string, unknown>> = {
link: {
target: "_blank",
rel: "noopener noreferrer nofollow",
// Editor-authored EXTERNAL links store `internal: false` (editor-ext link
// default `packages/editor-ext/src/lib/link.ts`), while an imported external
// link leaves `internal` absent/null. Both mean "external", so `internal:
// false` must normalize away exactly like `null`/absent — otherwise a stored
// `internal:false` link diverges from its re-import under
// `docsCanonicallyEqual` (false !== null). The internal marker `internal:true`
// is NON-default, so it is KEPT and survives canonicalization (#522 §11).
internal: false,
},
comment: {
resolved: false,
@@ -38,10 +38,6 @@ export {
normalizeForeignMarkdown,
normalizeAgentMarkdown,
} from "./foreign-markdown.js";
// Pure primitive: detect the unambiguously-internal wiki-page link path
// (`/s/<space>/p/<slug>`) and promote such link marks to their native internal
// form during import (#522). A strict subset of the server's INTERNAL_LINK_REGEX.
export { isInternalPagePath, markInternalLinks } from "./internal-links.js";
// The Docmost tiptap schema mirror. Exposed so consumers (and the sync
// engine's schema-validity regression tests) can build the exact ProseMirror
@@ -1,112 +0,0 @@
/**
* Detect and mark unambiguously-internal wiki-page links during markdown import.
*
* WHY THIS EXISTS
* ---------------
* The markdown converter (`markdownToProseMirrorSync`) materializes every link
* with StarterKit's external defaults `internal: null`, `target: "_blank"`,
* `rel: "noopener noreferrer nofollow"`. A markdown link that points at an
* internal wiki page written in its host-less, root-relative form
* (`[text](/s/<space>/p/<slugId>)`) is therefore stored as EXTERNAL: it opens in
* a new tab, gets no hover-preview, and is invisible to the backlink graph
* (`extractInternalLinkSlugIds` only counts links whose mark carries
* `internal: true`). This module supplies the pure primitive that a post-walk in
* the converter uses to promote such links to their native internal form.
*
* WHERE THE CANON LIVES (a strict subset, on purpose)
* ---------------------------------------------------
* The authoritative definition of "what is an internal link path" is the
* server's `INTERNAL_LINK_REGEX`
* (`apps/server/src/integrations/export/utils.ts`):
* /^(https?:\/\/)?([^\/]+)?(\/s\/([^\/]+)\/)?p\/([a-zA-Z0-9-]+)\/?$/
* This package is a lower layer than the server app and cannot import it, so the
* matcher below is a DOCUMENTED, STRICT SUBSET of that regex: it accepts only the
* space-qualified, root-relative page path `/s/<space>/p/<slug>` (with optional
* trailing slash). Because it is a subset, every link we mark internal here is
* guaranteed to also satisfy the server regex, hence guaranteed backlink-able and
* export-rewritable. A unit test pins the exact accept/reject set as the guard
* against drift.
*
* FAIL-TOWARD-EXTERNAL
* --------------------
* We mark a link internal ONLY on an unambiguous anchored match. Any ambiguity
* (a scheme/host, `#anchor`, `?query`, a space-less `/p/<slug>`, a relative
* `p/<slug>`, a non-string href) leaves the link external. A false-external is a
* soft degradation (a new tab); a false-internal would produce broken SPA
* navigation, so "external" is the conservative default.
*
* Precedent for the target internal shape: the file importer already promotes
* internal anchors (`apps/server/src/integrations/import/utils/import-formatter.ts`
* `$a.attr('data-internal','true')`) and editor-ext reads it
* (`packages/editor-ext/src/lib/link.ts`).
*/
/**
* Matches ONLY the space-qualified, root-relative internal page path
* `/s/<space>/p/<slug>` (with optional trailing slash). A STRICT SUBSET of the
* server's `INTERNAL_LINK_REGEX`:
* - `<space>` = `[^/]+` (any non-slash segment, as in the server's group 4)
* - `<slug>` = `[a-zA-Z0-9-]+` (as in the server's group 5 / `extractPageSlugId`)
* Anchored on both ends so a scheme/host, a trailing `#anchor`/`?query`, or any
* extra path segment fails to match.
*/
const INTERNAL_PAGE_PATH = /^\/s\/[^/]+\/p\/[a-zA-Z0-9-]+\/?$/;
/**
* True iff `href` is the unambiguous, space-qualified, root-relative internal
* page path. Pure and side-effect-free. Non-string input returns false.
*/
export function isInternalPagePath(href: unknown): boolean {
return typeof href === "string" && INTERNAL_PAGE_PATH.test(href);
}
/**
* The native internal-link attribute overrides applied to a link mark whose href
* is an internal page path. Mirrors the manual JSON-patch form
* (`{internal:true, target:null, rel:null}`) and the editor-ext/file-importer
* precedent: internal links carry no `target`/`rel` (same-tab SPA navigation).
*/
const INTERNAL_LINK_ATTRS = { internal: true, target: null, rel: null } as const;
/**
* In-place post-walk of a finished ProseMirror doc that promotes every
* unambiguously-internal link mark to its native internal form.
*
* A link that spans several text nodes (e.g. `[**bold** word](/s/x/p/abc)`)
* stores an equivalent link mark on EACH covered text node and a covered node
* may carry nested marks (bold/italic) alongside the link. We therefore walk the
* entire tree and rewrite EVERY `link` mark whose href passes
* `isInternalPagePath`, so no covered segment is left external.
*
* Pure of external effects and IDEMPOTENT: re-running on an already-marked doc
* leaves it unchanged (an internal-path href always maps to the same attrs).
* External links are never touched their `target:_blank`/`rel:noopener…` stay.
*
* The doc is mutated in place (the converter owns the freshly-built doc and
* returns it directly); the same node reference is returned for convenience.
*/
export function markInternalLinks<T>(node: T): T {
walk(node);
return node;
}
function walk(node: any): void {
if (!node || typeof node !== "object") return;
if (Array.isArray(node)) {
for (const child of node) walk(child);
return;
}
if (Array.isArray(node.marks)) {
for (const mark of node.marks) {
if (
mark &&
mark.type === "link" &&
mark.attrs &&
isInternalPagePath(mark.attrs.href)
) {
mark.attrs = { ...mark.attrs, ...INTERNAL_LINK_ATTRS };
}
}
}
if (Array.isArray(node.content)) walk(node.content);
}
@@ -12,7 +12,6 @@ import { parseHtmlDocument, generateJsonWith } from "./dom-parser.js";
import type { TokenizerExtension, RendererExtension } from "marked";
import { docmostExtensions } from "./docmost-schema.js";
import { parseAttachedComment } from "./attached-comment.js";
import { markInternalLinks } from "./internal-links.js";
import { splitFootnoteParagraphs } from "./footnote.js";
import {
decodeInlineMathLatex,
@@ -1095,12 +1094,7 @@ export function markdownToProseMirrorSync(markdownContent: string): any {
const withFootnotes = assembleFootnotes(withAttrs);
const bridged = bridgeTaskLists(withFootnotes);
const doc = generateJsonWith(bridged, docmostExtensions);
// Promote unambiguously-internal wiki-page links (`[t](/s/<space>/p/<slug>)`)
// to their native internal form (`internal:true, target:null, rel:null`) so
// they get same-tab SPA navigation, hover-preview, and backlink participation.
// Every markdown import path funnels through here, so all of them are fixed at
// once; external links are left untouched (#522).
return markInternalLinks(stripEmptyParagraphs(doc));
return stripEmptyParagraphs(doc);
}
/**
@@ -1,345 +0,0 @@
import { describe, expect, it } from 'vitest';
import {
isInternalPagePath,
markInternalLinks,
} from '../src/lib/internal-links.js';
import { markdownToProseMirrorSync } from '../src/lib/markdown-to-prosemirror.js';
import {
canonicalizeContent,
docsCanonicallyEqual,
} from '../src/lib/canonicalize.js';
// A LOCAL, illustrative copy of the server's INTERNAL_LINK_REGEX
// (apps/server/src/integrations/export/utils.ts) used only to document the
// subset boundary WITHIN this package (this layer cannot import the server).
// It is NOT the cross-package drift guard: a hand copy can silently go stale if
// the server regex is later narrowed. The real, mechanical drift guard lives in
// the top layer, `apps/server/src/integrations/export/internal-link-parity.spec.ts`,
// which imports BOTH the LIVE server `INTERNAL_LINK_REGEX` and the LIVE
// `isInternalPagePath` and reddens if the client ever ceases to be a subset.
const SERVER_INTERNAL_LINK_REGEX =
/^(https?:\/\/)?([^\/]+)?(\/s\/([^\/]+)\/)?p\/([a-zA-Z0-9-]+)\/?$/;
// Walk a doc collecting every link mark (across all text nodes).
const linkMarks = (doc: any): any[] => {
const out: any[] = [];
const walk = (n: any): void => {
if (!n || typeof n !== 'object') return;
if (Array.isArray(n)) return n.forEach(walk);
if (Array.isArray(n.marks))
for (const m of n.marks) if (m?.type === 'link') out.push(m);
if (Array.isArray(n.content)) walk(n.content);
};
walk(doc);
return out;
};
describe('isInternalPagePath', () => {
const ACCEPT = [
'/s/eng/p/abc123',
'/s/eng/p/abc123/', // trailing slash
'/s/My-Space/p/A-b-C-9', // hyphens + mixed case in both segments
'/s/x/p/z',
];
const REJECT = [
'https://example.com/p/x', // scheme + host
'http://host/s/eng/p/abc', // scheme + host, even on the internal shape
'//host/s/eng/p/abc', // protocol-relative host
'/p/abc123', // space-less form (server does NOT backlink it)
'p/abc123', // relative
's/eng/p/abc123', // missing leading slash
'/s/eng/p/abc#section', // anchor
'/s/eng/p/abc?q=1', // query
'/s/eng/p/abc/extra', // extra segment
'/s//p/abc', // empty space segment
'/s/eng/p/', // empty slug
'/s/eng/p', // no slug at all
'/api/pages/abc', // other internal route
'/s/eng/x/abc', // wrong middle segment
'/s/a/b/p/abc', // space contains slash -> extra segment
'',
];
// Structurally VALID `/s/<space>/p/<slug>` shapes whose ONLY defect is a
// forbidden character in the slug segment. These pin the slug charset
// `[a-zA-Z0-9-]` itself (not just the surrounding structure): drop them and a
// mutation widening the charset (e.g. adding `.`) survives the suite. Each is
// ALSO rejected by the server regex — documenting that the subset boundary
// holds precisely on the charset, not just on structure.
const REJECT_SLUG_CHARSET = [
'/s/eng/p/abc.def', // dot
'/s/eng/p/abc_def', // underscore
'/s/eng/p/abc%20', // percent-encoded space
'/s/eng/p/abc~x', // tilde
];
it('accepts the space-qualified root-relative page path (+trailing slash)', () => {
for (const href of ACCEPT)
expect(isInternalPagePath(href), href).toBe(true);
});
it('rejects external URLs, ambiguous, and non-page forms', () => {
for (const href of REJECT)
expect(isInternalPagePath(href), href).toBe(false);
});
it('rejects a correctly-shaped path with a forbidden slug character', () => {
// Pins the slug charset itself: a mutation widening `[a-zA-Z0-9-]` reddens.
for (const href of REJECT_SLUG_CHARSET)
expect(isInternalPagePath(href), href).toBe(false);
});
it('the forbidden-slug-char paths are rejected by the server regex too', () => {
// The subset boundary holds on the charset, not just the structure: none of
// these match the server regex, so the client must not accept them either.
for (const href of REJECT_SLUG_CHARSET)
expect(SERVER_INTERNAL_LINK_REGEX.test(href), href).toBe(false);
});
it('rejects non-string input (fail-toward-external)', () => {
for (const v of [null, undefined, 123, {}, [], true])
expect(isInternalPagePath(v as unknown)).toBe(false);
});
it('is a STRICT SUBSET of the server INTERNAL_LINK_REGEX', () => {
// Every accepted href must also satisfy the server regex (so it is
// guaranteed backlink-able / export-rewritable).
for (const href of ACCEPT)
expect(SERVER_INTERNAL_LINK_REGEX.test(href), href).toBe(true);
});
});
describe('markInternalLinks', () => {
const linkNode = (text: string, href: string, extraMarks: any[] = []) => ({
type: 'text',
text,
marks: [
{
type: 'link',
attrs: {
href,
internal: null,
title: null,
target: '_blank',
rel: 'noopener noreferrer nofollow',
class: null,
},
},
...extraMarks,
],
});
it('marks an internal-path link internal:true, target:null, rel:null', () => {
const doc = {
type: 'doc',
content: [
{ type: 'paragraph', content: [linkNode('hi', '/s/eng/p/abc123')] },
],
};
markInternalLinks(doc);
const m = linkMarks(doc)[0];
expect(m.attrs.internal).toBe(true);
expect(m.attrs.target).toBeNull();
expect(m.attrs.rel).toBeNull();
expect(m.attrs.href).toBe('/s/eng/p/abc123'); // href untouched
});
it('leaves external links untouched', () => {
const doc = {
type: 'doc',
content: [
{
type: 'paragraph',
content: [linkNode('ext', 'https://example.com/p/x')],
},
],
};
markInternalLinks(doc);
const m = linkMarks(doc)[0];
expect(m.attrs.internal).toBeNull();
expect(m.attrs.target).toBe('_blank');
expect(m.attrs.rel).toBe('noopener noreferrer nofollow');
});
it('marks EVERY text node a multi-node link spans (incl. nested bold/italic)', () => {
// A link `[**bold** plain *ital*](/s/x/p/abc)` stores the link mark on each
// covered text node; some also carry bold/italic. All must become internal.
const bold = { type: 'bold' };
const italic = { type: 'italic' };
const doc = {
type: 'doc',
content: [
{
type: 'paragraph',
content: [
linkNode('bold', '/s/x/p/abc', [bold]),
linkNode(' plain ', '/s/x/p/abc'),
linkNode('ital', '/s/x/p/abc', [italic]),
],
},
],
};
markInternalLinks(doc);
const marks = linkMarks(doc);
expect(marks).toHaveLength(3);
for (const m of marks) {
expect(m.attrs.internal).toBe(true);
expect(m.attrs.target).toBeNull();
expect(m.attrs.rel).toBeNull();
}
// Nested bold/italic marks are preserved alongside the promoted link.
const nested = doc.content[0].content;
expect(nested[0].marks.some((m: any) => m.type === 'bold')).toBe(true);
expect(nested[2].marks.some((m: any) => m.type === 'italic')).toBe(true);
});
it('is idempotent', () => {
const doc = {
type: 'doc',
content: [
{ type: 'paragraph', content: [linkNode('hi', '/s/eng/p/abc')] },
],
};
markInternalLinks(doc);
const once = JSON.stringify(doc);
markInternalLinks(doc);
expect(JSON.stringify(doc)).toBe(once);
});
});
describe('markdown import (full converter) — #522 acceptance', () => {
it('promotes an internal link and leaves an external one external', () => {
const doc = markdownToProseMirrorSync(
'[t](/s/eng/p/abc123) and [ext](https://example.com/p/x)',
);
const marks = linkMarks(doc);
const internal = marks.find((m) => m.attrs.href === '/s/eng/p/abc123');
const external = marks.find(
(m) => m.attrs.href === 'https://example.com/p/x',
);
expect(internal.attrs).toMatchObject({
internal: true,
target: null,
rel: null,
});
expect(external.attrs).toMatchObject({
internal: null,
target: '_blank',
rel: 'noopener noreferrer nofollow',
});
});
it('does NOT promote the space-less /p/<slug> form (documented limit)', () => {
const doc = markdownToProseMirrorSync('[t](/p/abc123)');
const m = linkMarks(doc)[0];
expect(m.attrs.internal).toBeNull();
expect(m.attrs.target).toBe('_blank');
});
it('the promoted link is backlink-extractable (matches server regex + true)', () => {
// Mirrors extractInternalLinkSlugIds: internal flag AND server-regex match.
const doc = markdownToProseMirrorSync('[t](/s/eng/p/my-page-abc123)');
const m = linkMarks(doc)[0];
expect(m.attrs.internal).toBe(true);
const match = m.attrs.href.match(SERVER_INTERNAL_LINK_REGEX);
expect(match).not.toBeNull();
// group 5 is the slug segment the server feeds to extractPageSlugId.
expect(match![5]).toBe('my-page-abc123');
});
it('comment-body markdown gets the same treatment (shared converter path)', () => {
// Comment bodies go through the same markdownToProseMirrorSync; a spot-check
// that the shared path (not a comment-only branch) does the promotion.
const doc = markdownToProseMirrorSync('see [here](/s/team/p/xyz789)');
const m = linkMarks(doc).find((x) => x.attrs.href === '/s/team/p/xyz789');
expect(m.attrs.internal).toBe(true);
});
});
describe('canonicalize — internal:false ≡ external (#522 §11)', () => {
it('drops editor-authored internal:false so external stays canonically equal', () => {
// Editor stores external links with internal:false; import leaves it absent.
const stored = {
type: 'doc',
content: [
{
type: 'paragraph',
content: [
{
type: 'text',
text: 'x',
marks: [
{
type: 'link',
attrs: {
href: 'https://example.com',
internal: false,
target: '_blank',
rel: 'noopener noreferrer nofollow',
},
},
],
},
],
},
],
};
const reimport = {
type: 'doc',
content: [
{
type: 'paragraph',
content: [
{
type: 'text',
text: 'x',
marks: [
{
type: 'link',
attrs: {
href: 'https://example.com',
internal: null, // import default
target: '_blank',
rel: 'noopener noreferrer nofollow',
},
},
],
},
],
},
],
};
expect(docsCanonicallyEqual(stored, reimport)).toBe(true);
// internal:false / target / rel all drop as defaults; only the non-default
// href survives.
const canon = canonicalizeContent(stored);
const mark = canon.content[0].content[0].marks[0];
expect(mark.attrs).toEqual({ href: 'https://example.com' });
});
it('keeps internal:true through canonicalization (non-default, load-bearing)', () => {
const doc = {
type: 'doc',
content: [
{
type: 'paragraph',
content: [
{
type: 'text',
text: 'x',
marks: [
{
type: 'link',
attrs: { href: '/s/x/p/abc', internal: true },
},
],
},
],
},
],
};
const canon = canonicalizeContent(doc);
const mark = canon.content[0].content[0].marks[0];
expect(mark.attrs.internal).toBe(true);
expect(mark.attrs.href).toBe('/s/x/p/abc');
});
});