fix(#345): replace id-alternation regex with a fixed generic scanner + line-anchor frontmatter (review round 2)

F7 [CRITICAL] The round-1 F2(a) fix built ONE alternation regex over all
definition ids (`(id1|id2|...)`). On prefix-chain ids (a, aa, aaa, ...) V8's
regex compiler blows its stack with a fatal, UNCATCHABLE 'RegExpCompiler
Allocation failed' that kills the whole process — strictly worse than the
original per-def thread-hang, and its match cost was still O(text x defs).
Replaced with a single FIXED generic scanner `/\[\^([^\]]+)\]/g` plus a map
lookup in the replacer: genuinely O(total text), no per-document regex
compilation, cannot blow up. Output is identical (only real def ids are inlined).

F8 [WARNING] The frontmatter strip regex was not line-anchored: it closed on the
FIRST `---` anywhere, so a value containing a triple-dash (e.g.
'title: Q1 --- Q2') truncated the frontmatter and leaked the rest into the body.
Replaced with the line-anchored shape the canonical parser already uses
(page-file.ts): open on `---\n`, close on a `\n---` line.

Adds tests: 4000 prefix-chain ids do not crash and stay fast; a frontmatter
value containing '---' is stripped whole.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
agent_coder
2026-07-05 05:18:44 +03:00
parent e17d5bc060
commit 80fc30633b
2 changed files with 52 additions and 17 deletions
@@ -126,6 +126,35 @@ describe('normalizeForeignMarkdown — GFM reference footnotes', () => {
normalizeForeignMarkdown(doc);
expect(Date.now() - t0).toBeLessThan(2000);
});
it('does not crash or slow down on thousands of prefix-chain definition ids', () => {
// F7: the rewrite must use a FIXED generic scanner, not an alternation built
// from the ids. A `(a|aa|aaa|…)` alternation over prefix-chain ids blows the
// V8 regex compiler (FATAL RegExpCompiler Allocation failed — uncatchable,
// kills the process). A fixed scanner has no id-dependent compilation cost.
const N = 4000;
const ids = Array.from({ length: N }, (_, i) => 'a'.repeat(i + 1));
const defs = ids.map((id) => `[^${id}]: body ${id.length}`).join('\n');
const doc = `ref[^${ids[0]}] and[^${ids[N - 1]}] end\n\n${defs}`;
const t0 = Date.now();
const out = normalizeForeignMarkdown(doc);
expect(Date.now() - t0).toBeLessThan(2000);
// Prefix disambiguation is correct: [^a] and [^aaaa...] inline their OWN body.
expect(out).toContain('^[body 1]');
expect(out).toContain(`^[body ${N}]`);
});
it('strips front-matter whose value contains a triple-dash (line-anchored)', () => {
// F8: the block must close only on a `\n---` LINE, not the first inline
// `---`. A value like `title: Q1 --- Q2` must not truncate the front-matter
// and leak the rest (author/closing ---) into the body.
const out = normalizeForeignMarkdown(
'---\ntitle: Q1 --- Q2 results\nauthor: bob\n---\n\nReal body.',
);
expect(out).toBe('Real body.');
expect(out).not.toContain('author: bob');
expect(out).not.toContain('Q2 results');
});
});
describe('foreign markdown import acceptance (normalizer + canonical parser)', () => {