A 10-agent red-team pass on the two-way Docmost<->git sync surfaced 16 ranked findings (9 others triaged out as already-defended). Wrote a reproduction test per finding (each asserts the CORRECT behavior, so it fails on the bug), then fixed the production code so every repro goes green. All confirmed bugs: Round-trip data loss (markdown-converter.ts + docmost-schema.ts mirror): - #1 editor-ext node types silently dropped on export — ported the 8 missing canon nodes (footnoteReference/footnotesList/footnoteDefinition, htmlEmbed, status, pageEmbed, transclusionSource/Reference) into the git-sync schema mirror and added converter cases that emit their schema-matching HTML instead of flattening unknown nodes to '' (this was the critical data-loss flagged in review #1679: footnotes/htmlEmbed lost on sync). Snapshot surface updated. - #2 top-level image lost width/height/align/attachmentId — now emits an HTML <img> (like video/diagrams) when it carries layout attrs; bare images stay . Image node parses width/height as strings so they re-import. - #3 code block containing a ``` fence corrupted on round-trip — outer fence is now widened to (longest-inner-backtick-run + 1). - #16 deep nesting threw RangeError (page never synced) — added a depth guard (MAX_NODE_DEPTH=400) so the converter never overflows the stack. Push/layout/cycle (engine): - #4 disambiguation ' ~slugId' suffix corrupted Docmost titles + order-dependent layout — deterministic, order-independent sibling disambiguation; suffix is stripped from a path-derived title ONLY when the new name is exactly the old title plus the suffix (never a genuine retitle ending in ' ~token'). - #6 retry-adopt by (parent,title) clobbered the wrong duplicate-title sibling — ambiguous (parent,title) is no longer adopted (falls back to fresh create). - #12 a new child under a new parent was created at ROOT — creates are ordered parent-before-child with an in-memory created-id map for parent resolution. - #13 git conflict markers could reach Docmost — bodies are scanned and the marker lines stripped (a '=======' line is only treated as a conflict separator inside a <<<<<<< ... >>>>>>> block, so setext headings are safe). - #15 a divergent `docmost` mirror was escalated by runPush but dropped by runCycle — RunCycleResult now forwards divergentDocmost to the orchestrator. Server (merge / lock / provenance): - #9 3-way merge lost a human's block edit when git inserted an adjacent block — finer-grained diff3 region merge (via lcs) preserves non-overlapping human edits; genuine same-block conflicts still resolve git-wins. - #10 single-writer race — module-static liveLocks closes the same-process TOCTOU window, and a heartbeat refresh that cannot confirm the lock now aborts the cycle at its next write checkpoint (cooperative AbortSignal threaded through runCycle). Cross-process fencing tokens remain a follow-up. - #14 sticky-agent provenance overrode an explicit actor='git-sync' write, blinding the listener loop-guard — resolveSource now lets an explicit actor win over the sticky-agent fallback (explicit agent still wins). Verified: git-sync vitest 617 pass (+1 expected-fail), server unit jest 1541 pass, server tsc clean. A review pass over the fixes caught and corrected a title-suffix over-strip, an inert abort signal, a document-wide conflict-marker strip, and two leaf-atom content-holes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
197 lines
7.4 KiB
TypeScript
197 lines
7.4 KiB
TypeScript
import { VaultGit } from "./git.js";
|
|
import { GitSyncClient } from "./client.types.js";
|
|
import { Settings } from "./settings.js";
|
|
import { readExisting, computePullActions, applyPullActions } from "./pull.js";
|
|
import { runPush } from "./push.js";
|
|
|
|
/**
|
|
* Absolute-path filesystem primitives the cycle needs. Injected (not imported)
|
|
* so the engine stays IO-free and unit-testable. `mkdir` is recursive; `rm` is
|
|
* force (a missing file is a no-op).
|
|
*/
|
|
export interface CycleFs {
|
|
readFile: (absPath: string) => Promise<string>;
|
|
writeFile: (absPath: string, text: string) => Promise<void>;
|
|
mkdir: (absDir: string) => Promise<void>;
|
|
rm: (absPath: string) => Promise<void>;
|
|
}
|
|
|
|
export interface RunCycleDeps {
|
|
spaceId: string;
|
|
/** The Docmost seam (reads for pull, writes for push). */
|
|
client: GitSyncClient;
|
|
/** The per-space git vault (a real working repo). */
|
|
vault: VaultGit;
|
|
/** Engine settings; `vaultPath` roots the relPath -> absolute-path mapping. */
|
|
settings: Settings;
|
|
fs: CycleFs;
|
|
log: (line: string) => void;
|
|
/**
|
|
* Optional cooperative-abort signal. The caller (orchestrator) wires this to
|
|
* the per-space lock: if a heartbeat refresh cannot CONFIRM the lock is still
|
|
* held (CAS-miss / Redis error), the signal is aborted and the cycle bails at
|
|
* its next checkpoint (before the pull-apply and before the push-apply — the
|
|
* two destructive write phases) instead of writing blind after a possible
|
|
* lock loss. This is a COARSE best-effort guard; a fully fenced cross-process
|
|
* single-writer still needs the fencing-token redesign (follow-up).
|
|
*/
|
|
signal?: AbortSignal;
|
|
/**
|
|
* Delete-cap hook (the ONLY caller-specific policy). Called with the push
|
|
* dry-run's planned delete count (`Number.POSITIVE_INFINITY` when the dry-run
|
|
* itself failed, so the hook can fail safe) and the live client; returns the
|
|
* client to use for the REAL apply. The default (omitted) applies every op
|
|
* unmodified. gitmost uses it to neutralize deletes when over its cap.
|
|
*
|
|
* When omitted, NO dry-run is performed (one fewer push planning pass).
|
|
*/
|
|
resolveApplyClient?: (
|
|
plannedDeletes: number,
|
|
client: GitSyncClient,
|
|
) => GitSyncClient;
|
|
}
|
|
|
|
export interface RunCycleResult {
|
|
ran: boolean;
|
|
/** Set when the cycle short-circuited without running pull/push. */
|
|
skipped?: "merge-in-progress";
|
|
pull?: { written: number; deleted: number; conflict: boolean };
|
|
push?: { mode: string; failures: number };
|
|
/**
|
|
* Forwarded from the push result: `true` when the push REFUSED to fast-forward
|
|
* a divergent `docmost` mirror (the §5 invariant — `docmost` mirrors what
|
|
* Docmost contains — is broken). Surfaced here so a caller driving `runCycle`
|
|
* can detect the breach without scraping logs (red-team #15).
|
|
*/
|
|
divergentDocmost?: boolean;
|
|
}
|
|
|
|
/**
|
|
* Run ONE full reconcile cycle for a space: PULL (Docmost -> vault) then PUSH
|
|
* (vault -> Docmost), under the engine's required branch choreography. This is
|
|
* the single entry point the app drives — it owns the staging order so it can
|
|
* never drift from the engine it ships with.
|
|
*
|
|
* Staging (the ⭐ data-loss-critical order, SPEC §6/§9):
|
|
* 1. assertGitAvailable + ensureRepo (the git state store must exist).
|
|
* 2. refuse on an unresolved merge (a prior conflicting pull); next checkout
|
|
* would fail otherwise.
|
|
* 3. ensureBranch('docmost','main') + checkout('docmost'). Pull writes MUST
|
|
* land on `docmost`, not `main`: applyPullActions commits on `docmost`,
|
|
* then checks out `main` and merges docmost -> main. Writing Docmost
|
|
* content straight onto `main` would clobber local file edits before push
|
|
* can diff them.
|
|
* 4. PULL: readExisting -> listSpaceTree -> computePullActions -> apply.
|
|
* 5. PUSH: optional dry-run to feed the delete-cap hook, then the real apply.
|
|
*
|
|
* Lock + cap POLICY live in the caller; this owns only the mechanics.
|
|
*/
|
|
export async function runCycle(deps: RunCycleDeps): Promise<RunCycleResult> {
|
|
const { spaceId, client, vault, settings, fs, log, resolveApplyClient, signal } =
|
|
deps;
|
|
const vaultRoot = settings.vaultPath;
|
|
const abs = (relPath: string) => `${vaultRoot}/${relPath}`;
|
|
|
|
// 1. The engine state store is git: make sure the repo + branches exist
|
|
// before any tracked-file listing or diff.
|
|
await vault.assertGitAvailable();
|
|
await vault.ensureRepo();
|
|
|
|
// 2. Refuse to run on top of an unresolved merge (SPEC §9): a prior
|
|
// conflicting pull leaves the vault mid-merge; the next checkout would fail.
|
|
if (await vault.isMergeInProgress()) {
|
|
log(
|
|
`vault has an unresolved merge — resolve it (or 'git merge --abort') ` +
|
|
`and re-run (SPEC §9); skipping cycle.`,
|
|
);
|
|
return { ran: false, skipped: "merge-in-progress" };
|
|
}
|
|
|
|
// 3. Pull writes happen on `docmost`; be on it BEFORE applying (see docstring).
|
|
await vault.ensureBranch("docmost", "main");
|
|
await vault.checkout("docmost");
|
|
|
|
// 4. PULL --------------------------------------------------------------------
|
|
const existing = await readExisting({
|
|
listTracked: () => vault.listTrackedFiles("*.md"),
|
|
readFile: (relPath) => fs.readFile(abs(relPath)),
|
|
});
|
|
|
|
const tree = await client.listSpaceTree(spaceId);
|
|
const pullActions = computePullActions({
|
|
pages: tree.pages,
|
|
treeComplete: tree.complete,
|
|
existing,
|
|
});
|
|
|
|
// Bail before the first destructive write phase if the lock was lost.
|
|
signal?.throwIfAborted();
|
|
|
|
const pullResult = await applyPullActions(
|
|
{
|
|
client,
|
|
git: vault,
|
|
writeFile: (absPath, text) => fs.writeFile(absPath, text),
|
|
mkdir: (absDir) => fs.mkdir(absDir),
|
|
rm: (absPath) => fs.rm(absPath),
|
|
log,
|
|
},
|
|
pullActions,
|
|
vaultRoot,
|
|
);
|
|
|
|
// 5. PUSH --------------------------------------------------------------------
|
|
const pushDeps = {
|
|
settings,
|
|
git: vault,
|
|
makeClient: () => client,
|
|
readFile: (relPath: string) => fs.readFile(abs(relPath)),
|
|
writeFile: (relPath: string, text: string) => fs.writeFile(abs(relPath), text),
|
|
log,
|
|
};
|
|
|
|
let applyClient = client;
|
|
if (resolveApplyClient) {
|
|
// Plan the push as a DRY-RUN first to read the delete count, then let the
|
|
// caller decide the apply client (e.g. neutralize deletes over a cap). A
|
|
// failed dry-run yields Infinity so the hook can fail safe.
|
|
let plannedDeletes: number;
|
|
try {
|
|
const dry = await runPush(pushDeps, { dryRun: true });
|
|
plannedDeletes = dry.planned?.deletes ?? 0;
|
|
} catch (err) {
|
|
log(
|
|
`push dry-run planning failed (${
|
|
err instanceof Error ? err.message : String(err)
|
|
}); deferring deletion policy to the cap hook (fail-safe).`,
|
|
);
|
|
plannedDeletes = Number.POSITIVE_INFINITY;
|
|
}
|
|
applyClient = resolveApplyClient(plannedDeletes, client);
|
|
}
|
|
|
|
// Bail before pushing to Docmost if the lock was lost during pull.
|
|
signal?.throwIfAborted();
|
|
|
|
const pushResult = await runPush(
|
|
{ ...pushDeps, makeClient: () => applyClient },
|
|
{ dryRun: false },
|
|
);
|
|
|
|
return {
|
|
ran: true,
|
|
pull: {
|
|
written: pullResult.written,
|
|
deleted: pullResult.deleted,
|
|
conflict: pullResult.merge.conflict,
|
|
},
|
|
push: {
|
|
mode: pushResult.mode,
|
|
failures: pushResult.failures?.length ?? 0,
|
|
},
|
|
// Forward a divergent-`docmost` escalation so the caller can act on the §5
|
|
// invariant breach without scraping logs (red-team #15).
|
|
divergentDocmost: pushResult.divergentDocmost ?? false,
|
|
};
|
|
}
|