vvzvlad/gitmost
1
0
Fork 0
Code Issues 14 Pull Requests 12 Packages Projects Releases Activity

fix(git-sync): push 503 starvation + concurrent-edit marker leak/silent loss

Browse Source 
Bug #1 (push 503 starvation): an external receive-pack that briefly overlapped
a poll cycle immediately 503'd because the per-space single-writer lock was
held. Add a BOUNDED retry-acquire on the PUSH path only (SpaceLockService
.withSpaceLock acquireRetry: capped exponential backoff up to ~5s); a transient
overlap now waits and succeeds, a genuinely stuck cycle still 503s after the
bound. The poll cycle passes no retry (immediate skip). Push result stays
deterministic: the receive-pack only runs once the lock is held, so a 503 never
leaves a half-applied ref.

Bug #2 (concurrent-edit marker leak + silent same-block loss):
- Marker leak (a): the push UPDATE path stripped markers for the body sent to
  Docmost but left raw <<<<<<</>>>>>>> committed on the published `main` vault
  forever (autoMergeConflicts ON). Now the cleaned body is written back to the
  vault file + recorded in writtenBack so runPush commits it on `main` and the
  vault converges to clean bytes.
- Marker leak (b): pin merge.conflictStyle=merge in ensureRepo and teach
  stripConflictMarkers/hasConflictMarkers about the diff3 `|||||||` base section
  (drop the marker AND the stale base region) so diff3/zdiff3 conflicts can
  never leak `|||||||` + base content into a page. Also scrub the 3-way merge
  BASE markdown.
- Silent same-block loss: the block 3-way merge still resolves same-block
  conflicts deterministically to git, but it is no longer silent: diff3Plan now
  reports a conflict count (mergeXmlFragments3WayWithStats), gitSyncWriteBody
  logs it, and the persistence boundary-snapshot now fires for git-sync writes
  over a non-git-sync baseline so the human's pre-merge content is preserved in
  page history (recoverable). Full both-preserved persisted-conflict UI remains
  the deferred redesign.

Tests: space-lock bounded-retry (success/stuck/poll-immediate); push vault-clean
+ diff3 |||||||  strip; ensureRepo conflictStyle pin; diff3Plan/3-way conflict
counts; persistence git-sync boundary snapshot. Server tsc clean; git-sync
vitest + server collaboration/git-sync jest all green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
claude code agent 227
2026-06-28 20:03:21 +03:00
parent 906733b5c8
commit b7e5cb6970
15 changed files with 567 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
packages/git-sync/src/engine/git.ts
View File

@@ -220,6 +220,13 @@ export class VaultGit {
// that core.autocrlf=false does not cover). POSIX-only path, which is
// fine: the daemon runs on Linux (Docker) / macOS. A system
// /etc/gitattributes remains the host admin's domain (out of scope).
// - merge.conflictStyle=merge — CRITICAL (SPEC §9, conflict-marker leak):
// a global `merge.conflictStyle=diff3`/`zdiff3` makes a conflicting merge
// emit an EXTRA `|||||||` base-marker section. The conflict-marker
// scrub on the push side (`stripConflictMarkers`) handles `|||||||` too,
// but pinning the classic `merge` style keeps the markers the engine
// produces to the canonical three (`<<<<<<<`/`=======`/`>>>>>>>`) so
// behavior is deterministic regardless of the operator's global config.
// NOTE: these stay PERSISTED LOCAL config (not `-c` flags) on purpose — a
// human running git by hand in the vault must inherit the same neutralized
// behavior; a transient `-c` would not persist. (core.quotepath, by
@@ -230,6 +237,7 @@ export class VaultGit {
await this.run(["config", "core.safecrlf", "false"]);
await this.run(["config", "commit.gpgsign", "false"]);
await this.run(["config", "core.attributesFile", "/dev/null"]);
await this.run(["config", "merge.conflictStyle", "merge"]);
} catch (err: unknown) {
const detail = err instanceof Error ? err.message : String(err);
throw new Error(

74
packages/git-sync/src/engine/push.ts
View File

@@ -699,19 +699,39 @@ export async function applyPushActions(
});
continue;
}
const conflicted = hasConflictMarkers(rawBody);
const body = stripConflictMarkers(rawBody);
// The last-synced version of this file (pre-image) is the common ancestor
// for a 3-way merge against the live page, so concurrent human edits are
// not clobbered (review #5). Null when the file is new at last-pushed. Its
// body is stripped the SAME way so the merge compares body-to-body.
// body is stripped the SAME way (frontmatter AND conflict markers) so the
// merge compares clean body-to-body: a base that itself carried markers
// (from a prior conflict commit) must never reintroduce marker syntax or a
// stale diff3 base region into the 3-way merge.
const baseFull = await deps.git.showFileAtRef(LAST_PUSHED_REF, u.path);
const baseMarkdown = baseFull === null ? null : parsePageFile(baseFull).body;
const baseMarkdown =
baseFull === null
? null
: stripConflictMarkers(parsePageFile(baseFull).body);
const result = await client.importPageMarkdown(
u.pageId,
body,
baseMarkdown,
);
updated++;
// CONFLICT VAULT-CLEAN (autoMergeConflicts ON, SPEC §9 marker leak). On ON
// a conflicted page is auto-merged INTO Docmost (the clean `body` above),
// but the file on `main` still carries the raw `<<<<<<<`/`>>>>>>>` markers
// the pull-side `commitMerge` committed. Left as-is they would (1) stay in
// the PUBLISHED vault forever (external clones see raw markers) and (2)
// re-conflict every cycle. So write the CLEAN body back to the vault file
// and record it in `writtenBack` — `runPush` step 7a commits it on `main`
// and re-advances the refs, so the published vault converges to the merged
// content. Only conflicted files are rewritten (no churn for clean updates).
if (conflicted) {
await deps.writeFile(u.path, serializePageFile(u.pageId, body));
writtenBack.push({ path: u.path, pageId: u.pageId });
}
// §10 loop-guard data: hash the BODY we pushed + capture `updatedAt`.
pushed.push({
pageId: u.pageId,
@@ -1083,13 +1103,23 @@ export function isPageFile(path: string): boolean {
* Docmost). A body is treated as conflicted only when it carries BOTH a begin
* (`<<<<<<<`) and an end (`>>>>>>>`) marker line, so a legitimate Markdown setext
* heading underline (`=======`) is not mistaken for a conflict. When conflicted,
* the three marker line types are removed while BOTH sides' content is preserved
* (no data loss): the marker SYNTAX never reaches Docmost, but the human's content
* does — where the conflict is visible and fixable rather than silently dropped.
* every marker line type is removed while the human-visible content is preserved
* (no data loss): the marker SYNTAX never reaches Docmost, but the content does —
* where the conflict is visible and fixable rather than silently dropped.
*
* `diff3`/`zdiff3` style: a conflict in that style adds a `|||||||` base section
* (`|||||||` line + the merge-BASE content + `=======`). `ensureRepo` pins
* `merge.conflictStyle=merge` so the engine never produces it, but a vault that
* predates the pin — or content arriving via an external push that a human
* committed in diff3 style — could still carry it. So we ALSO recognize the
* `|||||||` marker and DROP the stale base region it introduces (between
* `|||||||` and `=======`): the base text is neither side's current content, so
* keeping it would inject obsolete lines AND leak a raw `|||||||` marker.
*/
const CONFLICT_BEGIN_RE = /^<{7}/m;
const CONFLICT_END_RE = /^>{7}/m;
const CONFLICT_BEGIN_LINE_RE = /^<{7}/;
const CONFLICT_BASE_LINE_RE = /^\|{7}/;
const CONFLICT_SEP_LINE_RE = /^={7}/;
const CONFLICT_END_LINE_RE = /^>{7}/;
@@ -1099,23 +1129,37 @@ export function hasConflictMarkers(body: string): boolean {
function stripConflictMarkers(body: string): string {
if (!hasConflictMarkers(body)) return body;
// Remove ONLY the three marker line types, and treat a `=======` line as a
// conflict separator ONLY when we are between a `<<<<<<<` begin and a `>>>>>>>`
// end — so a legitimate Markdown setext heading underline (`=======`) outside a
// conflict block is preserved (review finding). Both conflict sides' content is
// kept; only the marker SYNTAX is dropped.
let inBlock = false;
// Track where we are inside a conflict block so a `=======` line is treated as
// a conflict separator ONLY between a `<<<<<<<` begin and a `>>>>>>>` end — a
// legitimate Markdown setext heading underline (`=======`) outside a conflict
// block is preserved (review finding). State machine over the block:
// 'no' — outside any conflict block.
// 'ours' — after `<<<<<<<`, before `|||||||`/`=======` (our side: KEEP).
// 'base' — after `|||||||`, before `=======` (diff3 base region: DROP).
// 'theirs' — after `=======`, before `>>>>>>>` (their side: KEEP).
// Every marker LINE itself is dropped; only the base region's content is also
// dropped (it is stale and not part of either current side).
let state: "no" | "ours" | "base" | "theirs" = "no";
const out: string[] = [];
for (const line of body.split("\n")) {
if (CONFLICT_BEGIN_LINE_RE.test(line)) {
inBlock = true;
state = "ours";
continue;
}
if (CONFLICT_END_LINE_RE.test(line)) {
inBlock = false;
if (state !== "no" && CONFLICT_END_LINE_RE.test(line)) {
state = "no";
continue;
}
if (inBlock && CONFLICT_SEP_LINE_RE.test(line)) {
if (state === "ours" && CONFLICT_BASE_LINE_RE.test(line)) {
state = "base";
continue;
}
if ((state === "ours" || state === "base") && CONFLICT_SEP_LINE_RE.test(line)) {
state = "theirs";
continue;
}
// Drop the diff3 base region's content (stale, neither current side).
if (state === "base") {
continue;
}
out.push(line);

4
packages/git-sync/test/git.test.ts
View File

@@ -162,6 +162,10 @@ describe('VaultGit (integration; temp repo)', () => {
expect(await localConfig('commit.gpgsign')).toBe('false');
expect(await localConfig('core.safecrlf')).toBe('false');
expect(await localConfig('core.attributesFile')).toBe('/dev/null');
// merge.conflictStyle=merge keeps conflict markers to the canonical three
// (no diff3 `|||||||` base section) regardless of the operator's global
// config (bug #2 marker-leak determinism, SPEC §9).
expect(await localConfig('merge.conflictStyle')).toBe('merge');
// Idempotent: a second run leaves the same single values (no duplicates).
await git.ensureRepo();

73
packages/git-sync/test/redteam-push-cycle.test.ts
View File

@@ -145,6 +145,79 @@ describe('#13 conflict markers reach Docmost', () => {
expect(pushedBody).toContain('their line');
});
it('autoMergeConflicts on: rewrites the vault file with the CLEAN body so raw markers do not stay in the published vault (bug #2 marker-leak)', async () => {
// Previously the UPDATE path stripped markers for the body SENT to Docmost but
// left the file on `main` carrying raw `<<<<<<<`/`>>>>>>>` forever — the
// published vault external clients clone kept the markers and the page
// re-conflicted every cycle. The fix writes the cleaned body back + records it
// in writtenBack so runPush commits it on `main`.
const { deps, importPageMarkdown } = makeConflictDeps({
...makeSettings(),
autoMergeConflicts: true,
});
const res = await runPush(deps, { dryRun: false });
expect(res.mode).toBe('apply');
// The clean body was imported into Docmost (no markers).
const pushedBody: string = importPageMarkdown.mock.calls[0][1] as any;
expect(pushedBody).not.toMatch(/[<>=]{7}/);
// The vault file was rewritten with the cleaned content (no raw markers).
const writeCalls = (deps.writeFile as any).mock.calls as [string, string][];
const docWrite = writeCalls.find(([p]) => p === 'Doc.md');
expect(docWrite).toBeDefined();
expect(docWrite![1]).not.toMatch(/[<>=]{7}/);
expect(docWrite![1]).toContain('my line');
expect(docWrite![1]).toContain('their line');
// It is recorded for the follow-up commit so `main` converges to clean bytes.
expect(res.applied?.writtenBack).toEqual(
expect.arrayContaining([
expect.objectContaining({ path: 'Doc.md', pageId: 'p-1' }),
]),
);
});
it('autoMergeConflicts on: strips diff3-style ||||||| base markers + base content (defense-in-depth)', async () => {
// A vault created before `merge.conflictStyle=merge` was pinned (or content a
// human committed in diff3 style) can carry a `||||||| base` section. The
// scrub must drop the `|||||||` marker AND the stale base region, keeping only
// the two live sides — otherwise `|||||||` + obsolete base lines leak into the
// Docmost page.
const diff3Body =
'<<<<<<< HEAD\nmy line\n||||||| base\nold base line\n=======\ntheir line\n>>>>>>> feature\n';
const file = serializePageFile('p-1', diff3Body);
const { git } = makePushGit({ changes: [{ status: 'M', path: 'Doc.md' }] });
const importPageMarkdown = vi.fn(async () => ({ success: true }));
const client = {
listSpaceTree: vi.fn(async () => ({ pages: [], complete: true })),
importPageMarkdown,
createPage: vi.fn(),
deletePage: vi.fn(),
movePage: vi.fn(),
renamePage: vi.fn(),
};
const deps: PushDeps = {
settings: { ...makeSettings(), autoMergeConflicts: true },
git,
makeClient: () => client as any,
readFile: vi.fn(async (p: string) => {
if (p === 'Doc.md') return file;
throw new Error(`no such file: ${p}`);
}),
writeFile: vi.fn(async () => {}),
log: () => {},
};
await runPush(deps, { dryRun: false });
const pushedBody: string = importPageMarkdown.mock.calls[0][1] as any;
expect(pushedBody).not.toContain('|||||||');
expect(pushedBody).not.toContain('old base line'); // stale base dropped
expect(pushedBody).toContain('my line');
expect(pushedBody).toContain('their line');
});
it('CREATE branch (autoMergeConflicts off): does NOT create a page from a conflicted NEW file; records a create failure', async () => {
// The conflict-markers guard is DUPLICATED on the CREATE path (a brand-new
// .md with NO gitmost_id, status 'A') and was previously untested — only the