Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 70a9e2a9cb | |||
| 3e945305c8 | |||
| 6d7dba970c | |||
| 512bcba5f3 | |||
| 5c1ab9c7b5 | |||
| 363f20ab75 | |||
| 3411bda2d1 | |||
| e670f7498a |
@@ -5,6 +5,139 @@ repository. It has two layers: **how to run a task end-to-end** (the
|
||||
sections below), and **how the codebase is built** (the technical sections
|
||||
further down, formerly in `CLAUDE.md`).
|
||||
|
||||
## ARCHITECTURAL INVARIANTS — NON-NEGOTIABLE
|
||||
|
||||
THE TEN RULES BELOW ARE HARD CONSTRAINTS. Each one was paid for with a real
|
||||
production incident or a multi-PR bug chain in THIS repository (cited inline).
|
||||
They override convenience, deadlines and "it's just a small feature". A PR that
|
||||
violates any of them MUST be rejected in review regardless of how good the rest
|
||||
of it is. If a task genuinely seems to require breaking one — STOP and raise it
|
||||
with the owner; do not code around it.
|
||||
|
||||
### 1. EVERY BUFFER, CACHE, HISTORY AND PAYLOAD HAS AN EXPLICIT SIZE BUDGET
|
||||
|
||||
Nothing accumulates unboundedly. A row/item cap is NOT a byte cap. Anything
|
||||
replayed to a model, buffered in memory, persisted per step, or refetched by a
|
||||
poll must state its budget in bytes/tokens and enforce it. Rewriting a growing
|
||||
structure in full on every increment is FORBIDDEN — append or diff instead;
|
||||
O(n²) write/serialize patterns do not pass review.
|
||||
(Paid for by: full-row rewrite on every agent step — hundreds of MB of Postgres
|
||||
writes per 50-step run, with every tool output serialized twice; unbounded
|
||||
history replay killing long chats on the provider context window; 32 MB replay
|
||||
buffers per active run.)
|
||||
|
||||
### 2. EVERYTHING LONG-RUNNING TERMINATES BY CONSTRUCTION
|
||||
|
||||
Every run / row / session / lease / subscriber / queue entry must define AT
|
||||
DESIGN TIME: its owner; every terminal state; who writes the terminal state on
|
||||
EVERY path (success, error, abort, disconnect in each phase, process restart);
|
||||
retries for the terminal write; and a periodic sweeper that does not depend on
|
||||
a reboot. A best-effort terminal write with no retry and no sweep is FORBIDDEN.
|
||||
(Paid for by: assistant rows stuck 'streaming' forever; runs stuck 'running'
|
||||
409-locking their chat until a restart — the #183/#184 follow-up chain.)
|
||||
|
||||
### 3. EVERY AWAIT IS CANCELLABLE AND DEADLINED; NEVER BLOCK THE EVENT LOOP
|
||||
|
||||
Every async step inside a request or agent turn honors the turn's AbortSignal
|
||||
AND a wall-clock deadline — including in-app tools, lock queues and pagination
|
||||
loops, not just external calls. Synchronous CPU work beyond ~50 ms goes to a
|
||||
worker_thread. Promise.race DOES NOT cancel synchronous work — using it as a
|
||||
"timeout" for sync computation is forbidden (the timer only fires after the
|
||||
event loop is free again, i.e. after the damage is done).
|
||||
(Paid for by: in-app tools ignoring abortSignal and writing pages AFTER Stop;
|
||||
the synchronous ELK layout freezing every SSE stream in the process; the
|
||||
step-0 MCP handshake hang — #397.)
|
||||
|
||||
### 4. ONE SOURCE OF TRUTH; EVERYTHING ELSE IS A REBUILDABLE CACHE
|
||||
|
||||
Postgres is the authoritative state. Every in-memory structure (registries,
|
||||
caches, client stores) must be reconstructible from the DB and treated as
|
||||
lossy. The client renders SERVER-DECLARED state — "a run is active" is a server
|
||||
fact delivered as data, never inferred from side signals (204 vs 2xx, the
|
||||
flavor of a disconnect). A new feature must name the owner of each piece of
|
||||
state before implementation starts.
|
||||
(Paid for by: the strip/restore resume machinery, silently frozen UIs and
|
||||
ghost sends after unmount — the #381→#432→#456 chain.)
|
||||
|
||||
### 5. STATE MACHINES ARE EXPLICIT — ONE-SHOT FLAGS ARE FORBIDDEN
|
||||
|
||||
A complex lifecycle (chat thread, resume/reconnect, run) lives in a named-state
|
||||
automaton (reducer / enum) where every state has an owner and a rendered
|
||||
representation — including the failure states. Adding a boolean ref that one
|
||||
callback arms and another reads-and-clears is FORBIDDEN in the AI-chat client.
|
||||
New behavior = a new named state + explicit transitions, and the interruption
|
||||
matrix (disconnect in each phase × restart × stop × supersede) is enumerated at
|
||||
design time, not discovered one incident at a time.
|
||||
(Paid for by: 26 one-shot useRef flags in chat-thread.tsx and the drip of
|
||||
"one more missing transition" across #381→#386/#389→#432→#456.)
|
||||
|
||||
### 6. NO NEW MODE FORKS; A FLAG IS FOR ROLLOUT, THEN IT DIES
|
||||
|
||||
A behavior flag that forks a code path must ship with a written sunset
|
||||
condition; stacking a new flag onto the existing matrix without deleting or
|
||||
scheduling an old one is forbidden. While a temporary fork exists, BOTH sides
|
||||
must share identical lifecycle handling (abort semantics, error listeners,
|
||||
concurrency gates) — asymmetric forks are outlawed.
|
||||
(Paid for by: legacy vs autonomous divergence — the one-active-run gate and
|
||||
the socket 'error' listener each existing on only ONE side; 2^4 flag
|
||||
combinations each with different abort semantics.)
|
||||
|
||||
### 7. NO HAND-SYNCED MIRRORS — CODEGEN OR A CI PARITY TEST, NOTHING LESS
|
||||
|
||||
Two copies of the same knowledge (schema, tool registry, glyph map, probe
|
||||
body, hash/normalize algorithm, label list) require either generation from a
|
||||
single source or a CI test that FAILS on drift. A "mirror this change over
|
||||
there" comment is NOT a guard and does not pass review.
|
||||
(Paid for by: #293 — three drifting converter copies losing data; #447 —
|
||||
REGISTRY_STAMP covering only one of the mirrored files; ~10 still-unguarded
|
||||
mirrors across the MCP layer.)
|
||||
|
||||
### 8. CACHES, HEADERS, BUFFERS AND FSM TRANSITIONS GET AN INTEGRATION TEST OF THE OBSERVABLE PROPERTY
|
||||
|
||||
A unit test of a pure helper DOES NOT COUNT for these. Test the real header on
|
||||
the real HTTP response, the real cache hit under real token sources, the real
|
||||
transition under a really-killed socket. If the observable property cannot be
|
||||
tested, the design is wrong — fix the design, not the test.
|
||||
(Paid for by: #431→#439 — a cache keyed on a fresh-per-call JWT, so it NEVER
|
||||
hit and became prod incident #435 while its unit tests stayed green; and by
|
||||
the #352→#455 immutable-cache header silently overwritten by a framework
|
||||
default AFTER the unit-tested code ran.)
|
||||
|
||||
### 9. CLIENT INPUT IS HOSTILE UNTIL VALIDATED — ALSO BEFORE PERSISTENCE
|
||||
|
||||
Anything from the browser (message parts, ids, titles, selections, flags) is
|
||||
validated/sanitized BEFORE it is persisted into a row that will later be
|
||||
replayed into a prompt, a converter or another subsystem. A poisoned row must
|
||||
never be able to permanently brick a chat or a page on every subsequent read.
|
||||
(Paid for by: unvalidated UIMessage parts persisted verbatim — one bad row
|
||||
500s the chat on every later turn; #159 client-spoofed page titles; #388
|
||||
selection re-sanitized server-side for the same reason.)
|
||||
|
||||
### 10. FAILURES ARE LOUD AND SPECIFIC; SILENT DEGRADATION IS FORBIDDEN
|
||||
|
||||
Extends the error convention below: a fire-and-forget write is allowed ONLY
|
||||
with a metric or a greppable ERROR log; a degraded mode (dead cached MCP
|
||||
client, stopped poll, exhausted retries, evicted buffer) must be VISIBLE to
|
||||
the user or the operator. A feature that can quietly stop working — a frozen
|
||||
"streaming…" UI, a poll that silently gives up, a cache serving corpses — does
|
||||
not pass review.
|
||||
(Paid for by: the degraded poll's silent 10-minute death leaving a forever-
|
||||
"streaming" answer; dead MCP clients served from cache while every external
|
||||
tool call failed; #435 being caught in minutes ONLY because metrics — #403 —
|
||||
existed.)
|
||||
|
||||
## Default skill for feature design
|
||||
|
||||
For any feature-design request — the user hands over a raw feature idea, asks
|
||||
to design or think through a feature, or to draft an issue («спроектируй»,
|
||||
«продумай фичу», «составь ишью», "design X", "write an issue for X") — invoke
|
||||
the `orchestrator-feature-designer` skill (Skill tool) BEFORE any other work.
|
||||
It is the default operating mode for design work in this repository: research
|
||||
→ design checklist (R1–R10) → forks resolved with the human → adversarial
|
||||
self-attack → filed PR-sized issues. Do not design features or write issues
|
||||
ad-hoc while this skill is available. This does not apply to non-design work
|
||||
(bug fixes, reviews, retrospectives, refactors already specified by an issue).
|
||||
|
||||
## Task lifecycle
|
||||
|
||||
### 1. Start: sync with develop
|
||||
|
||||
@@ -45,6 +45,11 @@ COPY --from=builder /app/packages/editor-ext/dist /app/packages/editor-ext/dist
|
||||
COPY --from=builder /app/packages/editor-ext/package.json /app/packages/editor-ext/package.json
|
||||
COPY --from=builder /app/packages/mcp/build /app/packages/mcp/build
|
||||
COPY --from=builder /app/packages/mcp/package.json /app/packages/mcp/package.json
|
||||
# The mcp package reads its data files (drawio-presets.json, drawio-shape-index.json.gz)
|
||||
# at runtime via `new URL("../../data/…", import.meta.url)` relative to build/lib/*.js,
|
||||
# i.e. from packages/mcp/data/. tsc emits only build/, so ship data/ explicitly or
|
||||
# drawioFromGraph and the shape catalog die with ENOENT on packages/mcp/data/*.
|
||||
COPY --from=builder /app/packages/mcp/data /app/packages/mcp/data
|
||||
# mcp now depends on @docmost/prosemirror-markdown (workspace:*) and eager-imports
|
||||
# it at runtime (the in-app ai-chat DocmostClient loads build/index.js -> lib/
|
||||
# markdown-converter.js). Ship the built package + its manifest, or the prod
|
||||
|
||||
@@ -27,10 +27,12 @@ import type { DocmostClientLike } from './docmost-client.loader';
|
||||
*/
|
||||
|
||||
describe('tool tier metadata (#332)', () => {
|
||||
it('core set is the documented 13 + searchInPage + insertFootnote (15)', () => {
|
||||
expect(CORE_TOOL_KEYS).toHaveLength(15);
|
||||
it('core set is the documented 13 + searchInPage + insertFootnote + getTree + getPageContext (17, #443)', () => {
|
||||
expect(CORE_TOOL_KEYS).toHaveLength(17);
|
||||
expect(CORE_TOOL_SET.has('searchInPage')).toBe(true); // #330, promoted to core
|
||||
expect(CORE_TOOL_SET.has('insertFootnote')).toBe(true); // #410, promoted to core
|
||||
expect(CORE_TOOL_SET.has('getTree')).toBe(true); // #443, promoted to core
|
||||
expect(CORE_TOOL_SET.has('getPageContext')).toBe(true); // #443, promoted to core
|
||||
// loadTools is a meta-tool, not a normal core key.
|
||||
expect(CORE_TOOL_SET.has(LOAD_TOOLS_NAME)).toBe(false);
|
||||
});
|
||||
|
||||
@@ -39,12 +39,14 @@ export interface ToolCatalogEntry {
|
||||
|
||||
/**
|
||||
* CORE (always-active) in-app tool keys — 13 frequent/tiny tools + `searchInPage`
|
||||
* (#330) + `insertFootnote` (#410). `searchInPage` is core because it is frequent
|
||||
* for the editorial roles this feature targets; `insertFootnote` is core so the
|
||||
* footnote tool is NOT hidden while its natural sibling `editPageText` is always
|
||||
* active (that asymmetry is exactly what pushed the agent to write literal
|
||||
* `^[...]`). `loadTools` is active too but is not a normal tool key (it is added
|
||||
* to activeTools separately).
|
||||
* (#330) + `insertFootnote` (#410) + `getTree`/`getPageContext` (#443).
|
||||
* `searchInPage` is core because it is frequent for the editorial roles this
|
||||
* feature targets; `insertFootnote` is core so the footnote tool is NOT hidden
|
||||
* while its natural sibling `editPageText` is always active (that asymmetry is
|
||||
* exactly what pushed the agent to write literal `^[...]`). `getTree` and
|
||||
* `getPageContext` are the single-call navigation/lookup tools — core so the
|
||||
* agent never has to loadTools just to orient itself. `loadTools` is active too
|
||||
* but is not a normal tool key (it is added to activeTools separately).
|
||||
*/
|
||||
export const CORE_TOOL_KEYS = [
|
||||
'searchPages',
|
||||
@@ -66,6 +68,11 @@ export const CORE_TOOL_KEYS = [
|
||||
// #410 insertFootnote — core so pinpoint citations to already-written text
|
||||
// don't degrade into literal `^[...]`; kept symmetric with editPageText.
|
||||
'insertFootnote',
|
||||
// #443 getTree + getPageContext — cheap single-call navigation/lookup tools
|
||||
// (the core listPages even points to getTree); core so the agent never has
|
||||
// to loadTools just to orient itself.
|
||||
'getTree',
|
||||
'getPageContext',
|
||||
] as const;
|
||||
|
||||
/** O(1) membership test for the core tier. */
|
||||
|
||||
@@ -34,6 +34,8 @@ import {
|
||||
isMetricsEnabled,
|
||||
observeMcpTool,
|
||||
incConnectTimeout,
|
||||
incGetPageCacheHit,
|
||||
incGetPageCacheMiss,
|
||||
} from '../metrics/metrics.registry';
|
||||
|
||||
// Minimal shape of the embedded MCP HTTP handler exported by @docmost/mcp/http.
|
||||
@@ -357,6 +359,10 @@ export class McpService implements OnModuleDestroy {
|
||||
observeMcpTool(labels?.tool ?? 'other', value);
|
||||
} else if (name === 'collab_connect_timeouts_total') {
|
||||
incConnectTimeout();
|
||||
} else if (name === 'mcp_getpage_cache_hits_total') {
|
||||
incGetPageCacheHit();
|
||||
} else if (name === 'mcp_getpage_cache_misses_total') {
|
||||
incGetPageCacheMiss();
|
||||
}
|
||||
}
|
||||
: undefined,
|
||||
|
||||
@@ -25,6 +25,15 @@ export const METRIC_COLLAB_CONNECT_TIMEOUTS_TOTAL =
|
||||
export const METRIC_COLLAB_AUTH_DURATION = 'collab_auth_duration_seconds';
|
||||
export const METRIC_MCP_TOOL_DURATION = 'mcp_tool_duration_seconds';
|
||||
|
||||
// #479 — getPage PM→Markdown conversion cache hit/miss counters. Emitted by the
|
||||
// MCP package via its dependency-neutral onMetric sink and routed onto these two
|
||||
// prom counters by the mcp.service onMetric callback; a >50% hit-rate is the
|
||||
// success signal for the getPage perf work. Same "do not rename" contract.
|
||||
export const METRIC_MCP_GETPAGE_CACHE_HITS_TOTAL =
|
||||
'mcp_getpage_cache_hits_total';
|
||||
export const METRIC_MCP_GETPAGE_CACHE_MISSES_TOTAL =
|
||||
'mcp_getpage_cache_misses_total';
|
||||
|
||||
// Histogram buckets (seconds). Chosen to give useful p50/p95/p99 resolution
|
||||
// for typical web/DB latencies without exploding series cardinality.
|
||||
export const HTTP_BUCKETS = [
|
||||
|
||||
@@ -24,6 +24,8 @@ import {
|
||||
METRIC_DB_QUERY_DURATION,
|
||||
METRIC_HTTP_REQUEST_DURATION,
|
||||
METRIC_MCP_TOOL_DURATION,
|
||||
METRIC_MCP_GETPAGE_CACHE_HITS_TOTAL,
|
||||
METRIC_MCP_GETPAGE_CACHE_MISSES_TOTAL,
|
||||
sizeBucket,
|
||||
} from './metrics.constants';
|
||||
|
||||
@@ -61,6 +63,9 @@ let connectTimeoutsCounter: Counter | null = null;
|
||||
let collabConnectHist: Histogram | null = null;
|
||||
let collabAuthHist: Histogram | null = null;
|
||||
let mcpToolHist: Histogram<'tool'> | null = null;
|
||||
// #479 — getPage conversion-cache hit/miss counters.
|
||||
let getPageCacheHitsCounter: Counter | null = null;
|
||||
let getPageCacheMissesCounter: Counter | null = null;
|
||||
|
||||
// #402 — read-on-scrape source for collab_docs_open. The gauge is NEVER
|
||||
// inc/dec'd (that drifts under crashes/handoffs); instead its collect() callback
|
||||
@@ -175,6 +180,18 @@ function init(): void {
|
||||
buckets: MCP_TOOL_BUCKETS,
|
||||
registers: [registry],
|
||||
});
|
||||
|
||||
getPageCacheHitsCounter = new Counter({
|
||||
name: METRIC_MCP_GETPAGE_CACHE_HITS_TOTAL,
|
||||
help: 'Total getPage PM→Markdown conversions served from the cache (skipped)',
|
||||
registers: [registry],
|
||||
});
|
||||
|
||||
getPageCacheMissesCounter = new Counter({
|
||||
name: METRIC_MCP_GETPAGE_CACHE_MISSES_TOTAL,
|
||||
help: 'Total getPage PM→Markdown conversions computed (cache misses)',
|
||||
registers: [registry],
|
||||
});
|
||||
}
|
||||
|
||||
// Runs once when this module is first imported. Safe to call again (idempotent).
|
||||
@@ -247,6 +264,14 @@ export function observeCollabAuth(seconds: number): void {
|
||||
collabAuthHist?.observe(seconds);
|
||||
}
|
||||
|
||||
export function incGetPageCacheHit(): void {
|
||||
getPageCacheHitsCounter?.inc();
|
||||
}
|
||||
|
||||
export function incGetPageCacheMiss(): void {
|
||||
getPageCacheMissesCounter?.inc();
|
||||
}
|
||||
|
||||
export function observeMcpTool(tool: string, seconds: number): void {
|
||||
// `tool` MUST be a bounded, registration-derived MCP tool name (the caller
|
||||
// guarantees it comes from the registered-tool set) — never free-form input —
|
||||
|
||||
@@ -10,6 +10,8 @@ import {
|
||||
incConnectTimeout,
|
||||
incDocLoad,
|
||||
incDocUnload,
|
||||
incGetPageCacheHit,
|
||||
incGetPageCacheMiss,
|
||||
isMetricsEnabled,
|
||||
observeCollabAuth,
|
||||
observeCollabConnect,
|
||||
@@ -197,6 +199,8 @@ describe('metrics helpers are safe no-ops when METRICS_PORT is unset', () => {
|
||||
incDocLoad();
|
||||
incDocUnload();
|
||||
incConnectTimeout();
|
||||
incGetPageCacheHit();
|
||||
incGetPageCacheMiss();
|
||||
// Registering a source must not create the gauge or invoke the fn.
|
||||
registerDocsOpenSource(() => {
|
||||
throw new Error('docsOpenSource must NOT be called when disabled');
|
||||
|
||||
@@ -23,6 +23,7 @@ import { acquireCollabSession } from "../lib/collab-session.js";
|
||||
import { withPageLock, isUuid } from "../lib/page-lock.js";
|
||||
import { getCollabToken, performLogin } from "../lib/auth-utils.js";
|
||||
import { formatDocmostAxiosError } from "./errors.js";
|
||||
import { GetPageConversionCache } from "./getpage-cache.js";
|
||||
|
||||
// A generic mixin base constructor (issue #450). Each domain mixin is a factory
|
||||
// `<T extends GConstructor<DocmostClientContext>>(Base: T) => class extends Base`
|
||||
@@ -159,6 +160,13 @@ export abstract class DocmostClientContext {
|
||||
// bypassed on a forced refresh (the 401/403 reauth path). null = no token yet.
|
||||
protected collabTokenCache: { token: string; mintedAt: number } | null = null;
|
||||
|
||||
// Content-addressed conversion cache for getPage (issue #479). Keyed on
|
||||
// (canonical pageId, updatedAt, optionsHash) -> the converted Markdown, so a
|
||||
// re-read of an UNCHANGED page skips the expensive convertProseMirrorToMarkdown
|
||||
// tree walk. Per-instance (a DocmostClient is built per user / per chat), so a
|
||||
// cached conversion can never leak across identities. See getpage-cache.ts.
|
||||
protected getPageCache = new GetPageConversionCache();
|
||||
|
||||
// Two construction forms:
|
||||
// - new DocmostClient(config) // discriminated union (current)
|
||||
// - new DocmostClient(baseURL, email, password) // legacy positional creds
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
// Content-addressed LRU cache for the PM->Markdown conversion in getPage
|
||||
// (issue #479). getPage is the dominant agent op (812 calls / 2h, p95 840ms);
|
||||
// the bulk of its cost is convertProseMirrorToMarkdown — a full ProseMirror-tree
|
||||
// walk over the page content (hundreds of KB of JSON on large pages) run on
|
||||
// EVERY read. Since agents re-read far more than they write (812 reads vs 28
|
||||
// writes in the sample), most conversions re-produce the SAME markdown from
|
||||
// UNCHANGED content. This cache skips the recomputation on a hit.
|
||||
//
|
||||
// KEY = (pageId, updatedAt, optionsHash):
|
||||
// - pageId: the page's CANONICAL UUID (resultData.id), not the agent-supplied
|
||||
// slugId — so a slugId read and a UUID read of the same page share one entry.
|
||||
// - updatedAt: comes from the SAME /pages/info response as `content`, so the
|
||||
// two are mutually consistent; a changed page yields a new updatedAt -> a new
|
||||
// key -> automatic, precise invalidation (no stale markdown is ever served).
|
||||
// - optionsHash: a stable hash of the conversion options. getPage passes
|
||||
// `{dropResolvedCommentAnchors:true}` while exportPageMarkdown passes the
|
||||
// defaults (#328) — DIFFERENT output for the same content, so the options
|
||||
// MUST be part of the key or a hit would serve the wrong variant.
|
||||
//
|
||||
// BOUNDS: evict the LEAST-recently-used entry when EITHER the entry count OR the
|
||||
// total stored bytes would exceed its cap. Large pages are hundreds of KB, so a
|
||||
// byte cap (not just a count cap) is what actually bounds memory. A Map iterates
|
||||
// in insertion order, so the first key is the LRU entry; a hit re-inserts its key
|
||||
// to move it to the most-recently-used end.
|
||||
//
|
||||
// This module is dependency-neutral (no axios/client/prom-client): a plain class
|
||||
// the shared client context owns one instance of, so the cache persists across
|
||||
// getPage calls on a single DocmostClient instance (built per user / per chat).
|
||||
|
||||
/** A stable, order-insensitive hash of the conversion options object. */
|
||||
export function hashConvertOptions(options: unknown): string {
|
||||
// JSON.stringify with SORTED keys makes the hash independent of key order, so
|
||||
// {a:1,b:2} and {b:2,a:1} collapse to one entry. undefined/null options -> a
|
||||
// fixed empty-object key, matching a caller that passes no options at all.
|
||||
if (options === undefined || options === null) return "{}";
|
||||
return stableStringify(options);
|
||||
}
|
||||
|
||||
function stableStringify(value: any): string {
|
||||
if (value === null || typeof value !== "object") return JSON.stringify(value);
|
||||
if (Array.isArray(value)) return `[${value.map(stableStringify).join(",")}]`;
|
||||
const keys = Object.keys(value).sort();
|
||||
return `{${keys
|
||||
.map((k) => `${JSON.stringify(k)}:${stableStringify(value[k])}`)
|
||||
.join(",")}}`;
|
||||
}
|
||||
|
||||
interface CacheEntry {
|
||||
markdown: string;
|
||||
bytes: number;
|
||||
}
|
||||
|
||||
export interface GetPageCacheOptions {
|
||||
/** Max number of entries before LRU eviction. Default 50. */
|
||||
maxEntries?: number;
|
||||
/** Max total stored bytes before LRU eviction. Default 10 MB. */
|
||||
maxBytes?: number;
|
||||
}
|
||||
|
||||
export class GetPageConversionCache {
|
||||
private readonly maxEntries: number;
|
||||
private readonly maxBytes: number;
|
||||
// Insertion-ordered: the FIRST key is the least-recently-used entry.
|
||||
private readonly map = new Map<string, CacheEntry>();
|
||||
private totalBytes = 0;
|
||||
|
||||
constructor(opts: GetPageCacheOptions = {}) {
|
||||
// A non-positive/NaN cap is treated as "use the default", never as an
|
||||
// unbounded (or always-empty) cache — a silently unbounded cache would leak
|
||||
// memory, and an always-empty one would defeat the whole optimization.
|
||||
this.maxEntries =
|
||||
Number.isFinite(opts.maxEntries) && (opts.maxEntries as number) > 0
|
||||
? Math.floor(opts.maxEntries as number)
|
||||
: 50;
|
||||
this.maxBytes =
|
||||
Number.isFinite(opts.maxBytes) && (opts.maxBytes as number) > 0
|
||||
? Math.floor(opts.maxBytes as number)
|
||||
: 10 * 1024 * 1024;
|
||||
}
|
||||
|
||||
/** Compose the content-addressed key from its three parts. */
|
||||
static key(pageId: string, updatedAt: string, optionsHash: string): string {
|
||||
// A space separates the parts so no combination of values can collide by
|
||||
// concatenation: a canonical UUID and an ISO updatedAt never contain a
|
||||
// space, so the boundaries between the three parts are unambiguous.
|
||||
return `${pageId} ${updatedAt} ${optionsHash}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Return the cached markdown for `key`, or undefined on a miss. A hit moves
|
||||
* the entry to the most-recently-used end (delete + re-set) so the LRU order
|
||||
* reflects real access, not just insertion.
|
||||
*/
|
||||
get(key: string): string | undefined {
|
||||
const entry = this.map.get(key);
|
||||
if (entry === undefined) return undefined;
|
||||
this.map.delete(key);
|
||||
this.map.set(key, entry);
|
||||
return entry.markdown;
|
||||
}
|
||||
|
||||
/**
|
||||
* Store `markdown` under `key`, then evict LRU entries until BOTH caps hold.
|
||||
* Re-storing an existing key refreshes its value and recency (its old bytes
|
||||
* are subtracted first, so totalBytes stays exact).
|
||||
*/
|
||||
set(key: string, markdown: string): void {
|
||||
// Byte size of the stored string (UTF-8). A single entry larger than the
|
||||
// whole byte cap is still stored (so getPage always gets a hit next time),
|
||||
// then the eviction loop below simply cannot shrink below it — accepted:
|
||||
// one oversized page is bounded by the page itself, not a cache leak.
|
||||
const bytes = Buffer.byteLength(markdown, "utf8");
|
||||
const existing = this.map.get(key);
|
||||
if (existing !== undefined) {
|
||||
this.totalBytes -= existing.bytes;
|
||||
this.map.delete(key);
|
||||
}
|
||||
this.map.set(key, { markdown, bytes });
|
||||
this.totalBytes += bytes;
|
||||
this.evict();
|
||||
}
|
||||
|
||||
/** Evict the LRU entry until both the count and byte caps are satisfied. */
|
||||
private evict(): void {
|
||||
while (
|
||||
this.map.size > this.maxEntries ||
|
||||
(this.totalBytes > this.maxBytes && this.map.size > 1)
|
||||
) {
|
||||
// The first key in insertion order is the least-recently-used.
|
||||
const oldest = this.map.keys().next().value as string | undefined;
|
||||
if (oldest === undefined) break;
|
||||
const entry = this.map.get(oldest);
|
||||
this.map.delete(oldest);
|
||||
if (entry) this.totalBytes -= entry.bytes;
|
||||
}
|
||||
}
|
||||
|
||||
/** Current entry count (test/introspection). */
|
||||
get size(): number {
|
||||
return this.map.size;
|
||||
}
|
||||
|
||||
/** Current total stored bytes (test/introspection). */
|
||||
get bytes(): number {
|
||||
return this.totalBytes;
|
||||
}
|
||||
}
|
||||
@@ -10,7 +10,14 @@ import {
|
||||
filterComment,
|
||||
filterSearchResult,
|
||||
} from "../lib/filters.js";
|
||||
import { convertProseMirrorToMarkdown } from "../lib/markdown-converter.js";
|
||||
import {
|
||||
convertProseMirrorToMarkdown,
|
||||
type ConvertProseMirrorToMarkdownOptions,
|
||||
} from "../lib/markdown-converter.js";
|
||||
import {
|
||||
GetPageConversionCache,
|
||||
hashConvertOptions,
|
||||
} from "./getpage-cache.js";
|
||||
import {
|
||||
collectInternalFileNodes,
|
||||
normalizeFileUrl,
|
||||
@@ -395,6 +402,19 @@ export function ReadMixin<TBase extends GConstructor<DocmostClientContext>>(Base
|
||||
|
||||
/** Raw page info including the ProseMirror JSON content and slugId. */
|
||||
|
||||
/**
|
||||
* Overridable seam over convertProseMirrorToMarkdown (issue #479). Production
|
||||
* just delegates; it exists as a method so a unit test can spy on it and
|
||||
* assert the conversion is genuinely SKIPPED on a getPage cache HIT (the whole
|
||||
* point of the cache) — an ESM named import cannot be intercepted otherwise.
|
||||
*/
|
||||
protected convertPageMarkdown(
|
||||
content: any,
|
||||
options: ConvertProseMirrorToMarkdownOptions,
|
||||
): string {
|
||||
return convertProseMirrorToMarkdown(content, options);
|
||||
}
|
||||
|
||||
async getPage(pageId: string) {
|
||||
await this.ensureAuthenticated();
|
||||
const resultData = await this.getPageRaw(pageId);
|
||||
@@ -403,13 +423,55 @@ export function ReadMixin<TBase extends GConstructor<DocmostClientContext>>(Base
|
||||
// discussions. Active anchors are kept. (The lossless exportPageMarkdown
|
||||
// round-trip deliberately does NOT pass this flag — resolved anchors there
|
||||
// must be preserved.)
|
||||
let content = resultData.content
|
||||
? convertProseMirrorToMarkdown(resultData.content, {
|
||||
dropResolvedCommentAnchors: true,
|
||||
})
|
||||
: "";
|
||||
//
|
||||
// Content-addressed conversion cache (issue #479): the PM->Markdown walk is
|
||||
// the dominant cost of this hot read op. Key on the page's canonical UUID +
|
||||
// updatedAt (both from THIS /pages/info response, so mutually consistent) +
|
||||
// a hash of the conversion options. A hit returns the cached markdown and
|
||||
// skips the walk; a miss converts and stores. The cached value is the
|
||||
// conversion output BEFORE the {{SUBPAGES}} substitution below, which uses
|
||||
// live subpage data and stays outside the cache — so the final result is
|
||||
// byte-identical to the uncached path.
|
||||
const convertOptions = { dropResolvedCommentAnchors: true };
|
||||
let content = "";
|
||||
if (resultData.content) {
|
||||
// Only cache when we have a stable identity+version for the key. Both come
|
||||
// from the same response; if either is missing (unexpected server shape),
|
||||
// fall back to converting uncached rather than keying on a partial tuple.
|
||||
const cacheable =
|
||||
typeof resultData.id === "string" &&
|
||||
typeof resultData.updatedAt === "string";
|
||||
const cacheKey = cacheable
|
||||
? GetPageConversionCache.key(
|
||||
resultData.id,
|
||||
resultData.updatedAt,
|
||||
hashConvertOptions(convertOptions),
|
||||
)
|
||||
: null;
|
||||
|
||||
// Always fetch subpages to provide context to the agent
|
||||
const cached = cacheKey ? this.getPageCache.get(cacheKey) : undefined;
|
||||
if (cached !== undefined) {
|
||||
content = cached;
|
||||
this.onMetricFn?.("mcp_getpage_cache_hits_total", 1);
|
||||
} else {
|
||||
// Goes through the convertPageMarkdown seam (not the raw import) so a
|
||||
// test can assert the conversion is SKIPPED on a hit (issue #479 F2).
|
||||
content = this.convertPageMarkdown(resultData.content, convertOptions);
|
||||
if (cacheKey) this.getPageCache.set(cacheKey, content);
|
||||
// A non-cacheable page (missing id/updatedAt) is still a genuine
|
||||
// conversion, so it counts as a miss for an honest hit-rate.
|
||||
this.onMetricFn?.("mcp_getpage_cache_misses_total", 1);
|
||||
}
|
||||
}
|
||||
|
||||
// Always fetch subpages to provide context to the agent.
|
||||
//
|
||||
// NOT parallelizable with the page fetch (issue #479 asked to check): the
|
||||
// sidebar-pages endpoint REQUIRES spaceId in its POST body, and spaceId is
|
||||
// only known FROM this page fetch's response (resolvePageId yields the UUID
|
||||
// but never the spaceId). So `Promise.all([pageFetch, subpagesFetch])` would
|
||||
// have to invent a spaceId it does not have — the two calls are inherently
|
||||
// sequential. Correctness wins; the conversion cache above is the real speedup.
|
||||
let subpages: any[] = [];
|
||||
try {
|
||||
// `pageId` may be a slugId, but the sidebar-pages endpoint requires the
|
||||
|
||||
@@ -316,7 +316,8 @@ async function main() {
|
||||
const [idA, idB, idC] = seedIds;
|
||||
|
||||
// patchNode: replace the middle paragraph; siblings' ids must be unchanged.
|
||||
await client.patchNode(nid, idB, mkPara(idB, "Bravo PATCHED."));
|
||||
// #413 XOR input: the raw ProseMirror node goes under the `node` key.
|
||||
await client.patchNode(nid, idB, { node: mkPara(idB, "Bravo PATCHED.") });
|
||||
await new Promise((r) => setTimeout(r, 16000));
|
||||
const afterPatch = (await client.getPageJson(nid)).content;
|
||||
const patchText = JSON.stringify(afterPatch);
|
||||
@@ -327,7 +328,7 @@ async function main() {
|
||||
// insertNode: place a new block after the first paragraph.
|
||||
await client.insertNode(
|
||||
nid,
|
||||
mkPara("nodeops-ins", "Inserted paragraph."),
|
||||
{ node: mkPara("nodeops-ins", "Inserted paragraph.") },
|
||||
{ position: "after", anchorNodeId: idA },
|
||||
);
|
||||
await new Promise((r) => setTimeout(r, 16000));
|
||||
|
||||
@@ -0,0 +1,251 @@
|
||||
// Mock-HTTP integration tests for the getPage conversion cache (issue #479).
|
||||
// A local http.createServer stands in for Docmost (same harness style as
|
||||
// get-page-context.test.mjs) so everything is deterministic and offline.
|
||||
//
|
||||
// Verifies end-to-end through the real client that:
|
||||
// - the FIRST getPage of a page is a MISS (mcp_getpage_cache_misses_total)
|
||||
// and converts the content (the server's convert-representative counter);
|
||||
// - a SECOND getPage of the same (pageId, updatedAt) is a HIT
|
||||
// (mcp_getpage_cache_hits_total) and returns BYTE-IDENTICAL output while
|
||||
// skipping the conversion;
|
||||
// - a changed updatedAt is a fresh key -> MISS again;
|
||||
// - the returned shape still resolves page + subpages.
|
||||
import { test, after, mock } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import http from "node:http";
|
||||
import { DocmostClient } from "../../build/client.js";
|
||||
|
||||
function readBody(req) {
|
||||
return new Promise((resolve) => {
|
||||
let raw = "";
|
||||
req.on("data", (chunk) => (raw += chunk));
|
||||
req.on("end", () => resolve(raw));
|
||||
});
|
||||
}
|
||||
|
||||
function sendJson(res, status, obj, extraHeaders = {}) {
|
||||
res.writeHead(status, { "Content-Type": "application/json", ...extraHeaders });
|
||||
res.end(JSON.stringify(obj));
|
||||
}
|
||||
|
||||
const openServers = [];
|
||||
after(async () => {
|
||||
await Promise.all(openServers.map((s) => new Promise((r) => s.close(r))));
|
||||
});
|
||||
|
||||
const PAGE_UUID = "00000000-0000-4000-8000-000000000010";
|
||||
const SPACE_UUID = "00000000-0000-4000-8000-0000000000aa";
|
||||
const CHILD_UUID = "00000000-0000-4000-8000-0000000000bb";
|
||||
|
||||
// A small ProseMirror doc so the converter produces non-trivial markdown.
|
||||
function makeDoc(text) {
|
||||
return {
|
||||
type: "doc",
|
||||
content: [
|
||||
{
|
||||
type: "paragraph",
|
||||
content: [{ type: "text", text }],
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
// state.info counts /pages/info hits; state.updatedAt / state.text drive the
|
||||
// content+version returned; state.sidebar counts sidebar-pages hits;
|
||||
// state.subpages (when set) drives the child list the sidebar endpoint returns,
|
||||
// so a test can vary the live subpages across two reads of the same page.
|
||||
function spawn(state) {
|
||||
return new Promise((resolve) => {
|
||||
const server = http.createServer(async (req, res) => {
|
||||
await readBody(req);
|
||||
if (req.url === "/api/auth/login") {
|
||||
return sendJson(res, 200, { success: true }, {
|
||||
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
|
||||
});
|
||||
}
|
||||
if (req.url === "/api/pages/info") {
|
||||
state.info++;
|
||||
return sendJson(res, 200, {
|
||||
success: true,
|
||||
data: {
|
||||
id: PAGE_UUID,
|
||||
slugId: "slug123456",
|
||||
title: "Cached Page",
|
||||
parentPageId: null,
|
||||
spaceId: SPACE_UUID,
|
||||
updatedAt: state.updatedAt,
|
||||
content: makeDoc(state.text),
|
||||
},
|
||||
});
|
||||
}
|
||||
if (req.url === "/api/pages/sidebar-pages") {
|
||||
state.sidebar++;
|
||||
const items = state.subpages ?? [
|
||||
{ id: CHILD_UUID, title: "Child", hasChildren: false },
|
||||
];
|
||||
return sendJson(res, 200, {
|
||||
success: true,
|
||||
data: {
|
||||
items,
|
||||
meta: { hasNextPage: false, nextCursor: null },
|
||||
},
|
||||
});
|
||||
}
|
||||
return sendJson(res, 404, { message: "not found" });
|
||||
});
|
||||
server.listen(0, "127.0.0.1", () => {
|
||||
openServers.push(server);
|
||||
resolve(`http://127.0.0.1:${server.address().port}/api`);
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function makeClient(baseURL, metrics) {
|
||||
return new DocmostClient({
|
||||
apiUrl: baseURL,
|
||||
getToken: async () => "access",
|
||||
onMetric: (name, value) => {
|
||||
metrics[name] = (metrics[name] ?? 0) + value;
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
test("first read MISS, second read HIT with byte-identical output; convert runs once", async () => {
|
||||
const state = { info: 0, sidebar: 0, updatedAt: "2026-01-01T00:00:00Z", text: "Hello world" };
|
||||
const baseURL = await spawn(state);
|
||||
const metrics = {};
|
||||
const client = makeClient(baseURL, metrics);
|
||||
|
||||
const first = await client.getPage(PAGE_UUID);
|
||||
assert.equal(metrics["mcp_getpage_cache_misses_total"], 1, "first read is a miss");
|
||||
assert.equal(metrics["mcp_getpage_cache_hits_total"] ?? 0, 0, "no hit yet");
|
||||
|
||||
const second = await client.getPage(PAGE_UUID);
|
||||
assert.equal(metrics["mcp_getpage_cache_hits_total"], 1, "second read is a hit");
|
||||
assert.equal(metrics["mcp_getpage_cache_misses_total"], 1, "still one miss");
|
||||
|
||||
// BYTE-IDENTICAL: the cache only skips recomputation, never changes output.
|
||||
assert.deepEqual(second, first, "cached result is identical to the uncached one");
|
||||
assert.equal(
|
||||
JSON.stringify(second),
|
||||
JSON.stringify(first),
|
||||
"serialized output is byte-identical",
|
||||
);
|
||||
|
||||
// The page fetch + subpages fetch still happen every call (only the CPU
|
||||
// conversion is cached); both reads hit /pages/info and sidebar-pages.
|
||||
assert.equal(state.info, 2, "both reads still fetch /pages/info");
|
||||
assert.equal(state.sidebar, 2, "both reads still fetch subpages");
|
||||
|
||||
// Shape sanity: content present, subpages resolved.
|
||||
assert.equal(typeof second.data.content, "string");
|
||||
assert.ok(second.data.content.includes("Hello world"));
|
||||
assert.deepEqual(second.data.subpages, [{ id: CHILD_UUID, title: "Child" }]);
|
||||
});
|
||||
|
||||
test("a changed updatedAt is a fresh key -> MISS again, with the NEW content", async () => {
|
||||
const state = { info: 0, sidebar: 0, updatedAt: "2026-01-01T00:00:00Z", text: "Version one" };
|
||||
const baseURL = await spawn(state);
|
||||
const metrics = {};
|
||||
const client = makeClient(baseURL, metrics);
|
||||
|
||||
const a = await client.getPage(PAGE_UUID); // miss
|
||||
const b = await client.getPage(PAGE_UUID); // hit
|
||||
assert.equal(metrics["mcp_getpage_cache_misses_total"], 1);
|
||||
assert.equal(metrics["mcp_getpage_cache_hits_total"], 1);
|
||||
assert.ok(a.data.content.includes("Version one"));
|
||||
|
||||
// The page changes: new updatedAt AND new content.
|
||||
state.updatedAt = "2026-02-02T00:00:00Z";
|
||||
state.text = "Version two";
|
||||
|
||||
const c = await client.getPage(PAGE_UUID); // miss on the new key
|
||||
assert.equal(metrics["mcp_getpage_cache_misses_total"], 2, "changed version -> miss");
|
||||
assert.equal(metrics["mcp_getpage_cache_hits_total"], 1, "no stale hit");
|
||||
assert.ok(c.data.content.includes("Version two"), "the NEW content is served");
|
||||
assert.ok(!c.data.content.includes("Version one"), "no stale markdown");
|
||||
|
||||
const d = await client.getPage(PAGE_UUID); // hit on the new key
|
||||
assert.equal(metrics["mcp_getpage_cache_hits_total"], 2, "the new snapshot caches too");
|
||||
});
|
||||
|
||||
test("a slugId read and a UUID read of the same page share one cache entry", async () => {
|
||||
// resolvePageId maps the slugId -> UUID via /pages/info; the cache keys on the
|
||||
// canonical UUID (resultData.id), so both inputs land on the same entry.
|
||||
const state = { info: 0, sidebar: 0, updatedAt: "2026-01-01T00:00:00Z", text: "Shared" };
|
||||
const baseURL = await spawn(state);
|
||||
const metrics = {};
|
||||
const client = makeClient(baseURL, metrics);
|
||||
|
||||
await client.getPage(PAGE_UUID); // miss (keyed on UUID)
|
||||
await client.getPage("slug123456"); // the server returns the same id -> HIT
|
||||
assert.equal(metrics["mcp_getpage_cache_misses_total"], 1, "one conversion total");
|
||||
assert.equal(metrics["mcp_getpage_cache_hits_total"], 1, "slugId read hits the UUID entry");
|
||||
});
|
||||
|
||||
test("on a conversion HIT, the {{SUBPAGES}} block reflects the LIVE subpages, not the cached ones", async () => {
|
||||
// The whole byte-identity guarantee: the cache stores the conversion output
|
||||
// BEFORE the {{SUBPAGES}} substitution, so a re-read of an UNCHANGED page still
|
||||
// splices the FRESH subpage list. The page body itself contains {{SUBPAGES}}
|
||||
// (converts to a literal placeholder); getPage replaces it with the live list.
|
||||
const CHILD_A = "00000000-0000-4000-8000-0000000000a1";
|
||||
const CHILD_B = "00000000-0000-4000-8000-0000000000b2";
|
||||
const state = {
|
||||
info: 0,
|
||||
sidebar: 0,
|
||||
updatedAt: "2026-01-01T00:00:00Z", // FIXED across both reads -> conversion cache HIT
|
||||
text: "Body before {{SUBPAGES}} body after",
|
||||
subpages: [{ id: CHILD_A, title: "Alpha", hasChildren: false }],
|
||||
};
|
||||
const baseURL = await spawn(state);
|
||||
const metrics = {};
|
||||
const client = makeClient(baseURL, metrics);
|
||||
|
||||
// Read 1: MISS (converts). The substitution runs with list A.
|
||||
const first = await client.getPage(PAGE_UUID);
|
||||
assert.equal(metrics["mcp_getpage_cache_misses_total"], 1, "first read converts (miss)");
|
||||
assert.ok(first.data.content.includes("[Alpha](page:" + CHILD_A + ")"), "list A spliced in");
|
||||
assert.ok(!first.data.content.includes("{{SUBPAGES}}"), "placeholder consumed");
|
||||
|
||||
// The subpages change while the PAGE CONTENT/updatedAt do NOT: same conversion
|
||||
// cache key -> a HIT that skips the CPU walk, but the live substitution must
|
||||
// still run on the NEW list B.
|
||||
state.subpages = [{ id: CHILD_B, title: "Beta", hasChildren: false }];
|
||||
|
||||
const second = await client.getPage(PAGE_UUID);
|
||||
assert.equal(metrics["mcp_getpage_cache_hits_total"], 1, "second read is a conversion HIT");
|
||||
assert.equal(metrics["mcp_getpage_cache_misses_total"], 1, "no second conversion");
|
||||
|
||||
// The cache did NOT freeze the subpages block: list B is present, list A gone.
|
||||
assert.ok(second.data.content.includes("[Beta](page:" + CHILD_B + ")"), "live list B spliced in on a HIT");
|
||||
assert.ok(!second.data.content.includes("Alpha"), "stale list A is NOT frozen into the output");
|
||||
assert.deepEqual(second.data.subpages, [{ id: CHILD_B, title: "Beta" }], "subpages field reflects list B");
|
||||
});
|
||||
|
||||
test("a cache HIT SKIPS the convertProseMirrorToMarkdown CPU walk (called once across MISS+HIT)", async () => {
|
||||
// The single reason the cache exists: on a hit the expensive PM-tree walk must
|
||||
// NOT run. The miss counter alone can't prove this — a broken hit branch that
|
||||
// re-converted (same output, misses=1) would leave every other assert green.
|
||||
// So spy directly on the conversion seam and assert the call COUNT.
|
||||
const state = { info: 0, sidebar: 0, updatedAt: "2026-01-01T00:00:00Z", text: "Body text" };
|
||||
const baseURL = await spawn(state);
|
||||
const metrics = {};
|
||||
const client = makeClient(baseURL, metrics);
|
||||
|
||||
// Spy on the seam that wraps convertProseMirrorToMarkdown; it still delegates,
|
||||
// so output stays real and byte-identical — we only count invocations.
|
||||
const spy = mock.method(client, "convertPageMarkdown");
|
||||
|
||||
await client.getPage(PAGE_UUID); // MISS -> converts once
|
||||
assert.equal(spy.mock.callCount(), 1, "the miss converts exactly once");
|
||||
|
||||
await client.getPage(PAGE_UUID); // HIT -> must NOT convert again
|
||||
assert.equal(
|
||||
spy.mock.callCount(),
|
||||
1,
|
||||
"the hit skips the conversion: still exactly one call across MISS+HIT",
|
||||
);
|
||||
assert.equal(metrics["mcp_getpage_cache_hits_total"], 1, "and it was recorded as a hit");
|
||||
|
||||
spy.mock.restore();
|
||||
});
|
||||
@@ -0,0 +1,113 @@
|
||||
// Unit tests for the getPage content-addressed conversion cache (issue #479).
|
||||
// Exercises the GetPageConversionCache class in isolation: key composition,
|
||||
// hit/miss, LRU recency on read, and eviction by BOTH the count cap and the
|
||||
// byte cap. The getPage integration (counter emission, byte-identical output)
|
||||
// is covered separately in test/mock/getpage-conversion-cache.test.mjs.
|
||||
import { test } from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import {
|
||||
GetPageConversionCache,
|
||||
hashConvertOptions,
|
||||
} from "../../build/client/getpage-cache.js";
|
||||
|
||||
test("key: identical (pageId, updatedAt, optionsHash) collapse to one entry", () => {
|
||||
const c = new GetPageConversionCache();
|
||||
const k1 = GetPageConversionCache.key("uuid-1", "2026-01-01T00:00:00Z", "{}");
|
||||
const k2 = GetPageConversionCache.key("uuid-1", "2026-01-01T00:00:00Z", "{}");
|
||||
assert.equal(k1, k2, "same parts -> same key");
|
||||
c.set(k1, "MD-A");
|
||||
assert.equal(c.get(k2), "MD-A", "hit on the identical key");
|
||||
assert.equal(c.size, 1);
|
||||
});
|
||||
|
||||
test("miss after updatedAt changes (precise invalidation)", () => {
|
||||
const c = new GetPageConversionCache();
|
||||
const oldK = GetPageConversionCache.key("uuid-1", "v1", "{}");
|
||||
const newK = GetPageConversionCache.key("uuid-1", "v2", "{}");
|
||||
c.set(oldK, "OLD-MD");
|
||||
assert.equal(c.get(newK), undefined, "a new updatedAt is a fresh key -> miss");
|
||||
assert.equal(c.get(oldK), "OLD-MD", "the old snapshot is still addressable");
|
||||
});
|
||||
|
||||
test("miss after options change (dropResolvedCommentAnchors true vs false)", () => {
|
||||
const c = new GetPageConversionCache();
|
||||
const hDrop = hashConvertOptions({ dropResolvedCommentAnchors: true });
|
||||
const hKeep = hashConvertOptions({ dropResolvedCommentAnchors: false });
|
||||
assert.notEqual(hDrop, hKeep, "different options hash to different keys");
|
||||
const kDrop = GetPageConversionCache.key("uuid-1", "v1", hDrop);
|
||||
const kKeep = GetPageConversionCache.key("uuid-1", "v1", hKeep);
|
||||
c.set(kDrop, "AGENT-MD");
|
||||
assert.equal(
|
||||
c.get(kKeep),
|
||||
undefined,
|
||||
"the export variant must NOT be served the agent variant",
|
||||
);
|
||||
});
|
||||
|
||||
test("hashConvertOptions is order-insensitive and handles empty", () => {
|
||||
assert.equal(
|
||||
hashConvertOptions({ a: 1, b: 2 }),
|
||||
hashConvertOptions({ b: 2, a: 1 }),
|
||||
"key order does not change the hash",
|
||||
);
|
||||
assert.equal(hashConvertOptions(undefined), "{}");
|
||||
assert.equal(hashConvertOptions(null), "{}");
|
||||
assert.equal(hashConvertOptions({}), "{}");
|
||||
});
|
||||
|
||||
test("LRU eviction by COUNT cap evicts the least-recently-used entry", () => {
|
||||
const c = new GetPageConversionCache({ maxEntries: 2, maxBytes: 10 * 1024 * 1024 });
|
||||
c.set("k1", "a");
|
||||
c.set("k2", "b");
|
||||
c.set("k3", "c"); // over the count cap -> evict k1 (oldest)
|
||||
assert.equal(c.size, 2);
|
||||
assert.equal(c.get("k1"), undefined, "k1 evicted");
|
||||
assert.equal(c.get("k2"), "b");
|
||||
assert.equal(c.get("k3"), "c");
|
||||
});
|
||||
|
||||
test("a read refreshes recency so the OTHER entry is evicted next", () => {
|
||||
const c = new GetPageConversionCache({ maxEntries: 2, maxBytes: 10 * 1024 * 1024 });
|
||||
c.set("k1", "a");
|
||||
c.set("k2", "b");
|
||||
// Touch k1 so k2 becomes the least-recently-used.
|
||||
assert.equal(c.get("k1"), "a");
|
||||
c.set("k3", "c"); // evicts the LRU, which is now k2 (not k1)
|
||||
assert.equal(c.get("k2"), undefined, "k2 was LRU and got evicted");
|
||||
assert.equal(c.get("k1"), "a", "k1 survived because it was read");
|
||||
assert.equal(c.get("k3"), "c");
|
||||
});
|
||||
|
||||
test("LRU eviction by BYTE cap evicts oldest until under the cap", () => {
|
||||
// Byte cap of 100; each value is 40 bytes -> at most 2 fit (80 < 100 < 120).
|
||||
const big = "x".repeat(40);
|
||||
const c = new GetPageConversionCache({ maxEntries: 50, maxBytes: 100 });
|
||||
c.set("k1", big); // 40
|
||||
c.set("k2", big); // 80
|
||||
assert.equal(c.size, 2);
|
||||
c.set("k3", big); // 120 > 100 -> evict k1 -> 80
|
||||
assert.equal(c.size, 2, "byte cap forced an eviction despite the count cap");
|
||||
assert.equal(c.get("k1"), undefined, "oldest evicted by bytes");
|
||||
assert.equal(c.get("k2"), big);
|
||||
assert.equal(c.get("k3"), big);
|
||||
assert.ok(c.bytes <= 100, "total bytes stays within the cap");
|
||||
});
|
||||
|
||||
test("re-setting an existing key updates value+recency and keeps bytes exact", () => {
|
||||
const c = new GetPageConversionCache({ maxEntries: 5, maxBytes: 10 * 1024 * 1024 });
|
||||
c.set("k1", "short");
|
||||
const b1 = c.bytes;
|
||||
assert.equal(b1, Buffer.byteLength("short", "utf8"));
|
||||
c.set("k1", "a much longer value");
|
||||
assert.equal(c.size, 1, "no duplicate entry");
|
||||
assert.equal(c.bytes, Buffer.byteLength("a much longer value", "utf8"));
|
||||
assert.equal(c.get("k1"), "a much longer value");
|
||||
});
|
||||
|
||||
test("an oversized single entry is still stored (never a permanent miss)", () => {
|
||||
const c = new GetPageConversionCache({ maxEntries: 5, maxBytes: 10 });
|
||||
const big = "y".repeat(1000);
|
||||
c.set("k1", big);
|
||||
assert.equal(c.get("k1"), big, "the page bigger than the cap is still served");
|
||||
assert.equal(c.size, 1);
|
||||
});
|
||||
Reference in New Issue
Block a user