fix(git-sync): kill spurious marker-leaking conflict, concurrent-edit loss, flapping HEAD

Three more git-sync QA defects from the 2nd live pass on PR #119, plus a callout-fidelity nit: 1. SPURIOUS conflict leaked raw markers into canonical main (root cause). On an ordinary round-trip the only difference between the docmost mirror (normalize- on-write) and a user's raw push is trailing/empty-line normalization, which made git's line-based docmost->main merge CONFLICT, and the wedge fix then committed the file WITH literal <<<<<<< / ======= / >>>>>>> markers onto main (git and the DB silently diverged for cycles). Fix: on a conflict, normalize trailing/empty lines on BOTH sides (showStage :2:/:3:) before comparing — a trailing-only diff is recognized as spurious and resolved to the clean normalized form. A GENUINE same-block conflict is auto-resolved to OURS (git wins, mirroring the live-doc 3-way rule); the docmost side stays on the `docmost` branch + page history. Raw markers NEVER reach main again. 2. Concurrent UI<->git edit silently lost the UI side. The git->Docmost 3-way merge ran against a live Y.Doc that hadn't yet received the user's debounced in-flight edit, so git clean-applied (no conflict detected) and the edit vanished even on a different block. Fix: flush the pending debounced store before the merge so the in-flight edit is drained into the live doc first — a different-block edit is merged, a same-block one is detected and pinned to history (recoverable). 3. Smart-HTTP HEAD flapped to the read-only `docmost` mirror (~1/4 of clones). The engine transiently checks out `docmost` mid-pull and the host advertises whatever HEAD resolves to. Fix: VaultGit.pinHeadToMain(); the cycle restores HEAD->main in a finally; and the upload-pack ref advertisement is served HEAD-pinned under the per-space lock so it can never observe a mid-cycle HEAD. 4. (callout) clampCalloutType now mirrors the editor's GITHUB_ALERT_TYPE_MAP for non-schema aliases (tip->success, caution->danger, important->info) instead of flatly collapsing to info. The editor schema genuinely supports only the six banner types, so unknown types still fall back to info (by design). Tests: deterministic real-git trailing-blank round-trip (no conflict, no markers, in sync over 2 cycles) + genuine-conflict no-marker-leak; HEAD advertisement stability; pre/post-flush concurrent-edit survival; serveReadAdvertisement lock pin; widened callout-alias coverage. Engine vitest + server tsc + collaboration / git-http / orchestrator specs all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 22:05:32 +03:00
parent b7e5cb6970
commit b47751349f
16 changed files with 948 additions and 106 deletions
@@ -46,7 +46,10 @@ interface Built {
  abilityFactory: { createForUser: AnyMock };
  abilityCan: AnyMock;
  vaultRegistry: { ensureServable: AnyMock };
-  orchestrator: { ingestExternalPush: AnyMock };
+  orchestrator: {
+    ingestExternalPush: AnyMock;
+    serveReadAdvertisement: AnyMock;
+  };
  backend: { run: AnyMock };
 }

@@ -88,7 +91,14 @@ function build(opts: BuildOptions = {}): Built {
  };

  const vaultRegistry = { ensureServable: jest.fn(async () => undefined) };
-  const orchestrator = { ingestExternalPush: jest.fn(async () => undefined) };
+  const orchestrator = {
+    ingestExternalPush: jest.fn(async () => undefined),
+    // The read-advertisement wrapper pins HEAD under the lock then serves; the
+    // mock just runs the serve callback so the read path still hits backend.run.
+    serveReadAdvertisement: jest.fn(
+      async (_spaceId: string, serve: () => Promise<void>) => serve(),
+    ),
+  };
  const backend = { run: jest.fn(async () => undefined) };

  const service = new GitHttpService(
@@ -231,6 +241,48 @@ describe('GitHttpService.handle', () => {
    expect(built.orchestrator.ingestExternalPush).not.toHaveBeenCalled();
  });

+  it('upload-pack ref advertisement is served HEAD-pinned via serveReadAdvertisement (bug #3)', async () => {
+    // GET info/refs?service=git-upload-pack carries the HEAD symref a clone reads
+    // for its default branch, so it must be served with HEAD pinned to `main`
+    // (under the lock) — not streamed raw — or a clone racing a mid-pull cycle
+    // would default to the read-only `docmost` mirror.
+    const built = build({ abilityCan: true });
+    const { reply } = fakeReply();
+    const req = fakeRequest({
+      url: '/git/space-1.git/info/refs?service=git-upload-pack',
+      method: 'GET',
+      authorization: basic('dev@example.com', 'pw'),
+    });
+
+    await built.service.handle(req, reply);
+
+    expect(built.orchestrator.serveReadAdvertisement).toHaveBeenCalledTimes(1);
+    expect(built.orchestrator.serveReadAdvertisement.mock.calls[0][0]).toBe(
+      'space-1',
+    );
+    // The wrapper still streams the backend (the mock runs the serve callback).
+    expect(built.backend.run).toHaveBeenCalledTimes(1);
+    expect(built.orchestrator.ingestExternalPush).not.toHaveBeenCalled();
+  });
+
+  it('a POST git-upload-pack pack fetch streams directly (no HEAD-pin needed, resolved by SHA)', async () => {
+    // The pack negotiation is object-SHA based; only the ref advertisement carries
+    // the HEAD symref, so the pack POST streams the backend directly (no lock).
+    const built = build({ abilityCan: true });
+    const { reply } = fakeReply();
+    const req = fakeRequest({
+      url: '/git/space-1.git/git-upload-pack',
+      method: 'POST',
+      authorization: basic('dev@example.com', 'pw'),
+    });
+
+    await built.service.handle(req, reply);
+
+    expect(built.orchestrator.serveReadAdvertisement).not.toHaveBeenCalled();
+    expect(built.backend.run).toHaveBeenCalledTimes(1);
+    expect(built.orchestrator.ingestExternalPush).not.toHaveBeenCalled();
+  });
+
  it('cloud deployment resolves the workspace by the host subdomain', async () => {
    const built = build({ selfHosted: false });
    const { reply } = fakeReply();
@@ -376,7 +376,25 @@ export class GitHttpService implements OnModuleDestroy {
    const isReceivePack =
      req.method === 'POST' && parsedPath.subpath === 'git-receive-pack';
    if (serviceKind === 'read' || !isReceivePack) {
-      await this.backend.run(backendRequest, rawReq, rawRes);
+      // The clone's default branch comes from the HEAD symref advertised by the
+      // upload-pack ref advertisement (or a dumb `GET HEAD`). The engine
+      // transiently checks out the read-only `docmost` mirror mid-cycle, so serve
+      // THAT advertisement with HEAD pinned to `main` under the per-space lock so
+      // a clone never defaults to `docmost` (bug #3). Pack streaming and every
+      // other read are resolved by object SHA and need no pin, so they stream
+      // directly (no lock) as before.
+      const isReadAdvertise =
+        req.method === 'GET' &&
+        ((parsedPath.subpath === 'info/refs' &&
+          service === 'git-upload-pack') ||
+          parsedPath.subpath === 'HEAD');
+      if (isReadAdvertise) {
+        await this.orchestrator.serveReadAdvertisement(spaceId, () =>
+          this.backend.run(backendRequest, rawReq, rawRes),
+        );
+      } else {
+        await this.backend.run(backendRequest, rawReq, rawRes);
+      }
      return;
    }

@@ -118,6 +118,7 @@ function build(opts: BuildOptions = {}): Built {
    ensureBranch: jest.fn(async () => undefined),
    checkout: jest.fn(async () => undefined),
    listTrackedFiles: jest.fn(async () => []),
+    pinHeadToMain: jest.fn(async () => undefined),
    ...(vaultOverrides as Record<string, AnyMock>),
  };
  const vaultRegistry = {
@@ -380,6 +381,11 @@ describe('GitSyncOrchestrator', () => {
      expect(order).toEqual(['receive-pack', 'cycle']);
    });

+    // Explicit timeout: ingestExternalPush exhausts the full bounded
+    // acquire-retry budget (GIT_SYNC_PUSH_LOCK_RETRY_TOTAL_MS = 5_000ms) before it
+    // gives up and throws, which races jest's DEFAULT 5_000ms test timeout — flaky
+    // on a loaded/slow runner. Give it headroom so it deterministically observes
+    // the eventual LockHeldError instead of timing out first.
    it('throws GitSyncLockHeldError and does NOT run the receive-pack when the lock is held', async () => {
      const built = build();
      built.redis.set.mockResolvedValue(null); // acquire fails → lock-held
@@ -392,7 +398,7 @@ describe('GitSyncOrchestrator', () => {
      // We must never write to the working tree concurrently with a cycle.
      expect(runReceivePack).not.toHaveBeenCalled();
      expect(runCycleMock).not.toHaveBeenCalled();
-    });
+    }, 15_000);

    it('swallows a post-push cycle error (the push is durable; poll retries)', async () => {
      jest.spyOn(Logger.prototype, 'error').mockImplementation(() => undefined);
@@ -444,6 +450,37 @@ describe('GitSyncOrchestrator', () => {
    });
  });

+  describe('serveReadAdvertisement (bug #3 — stable advertised HEAD)', () => {
+    it('pins HEAD to main and serves under the space lock', async () => {
+      const built = build();
+      const serve = jest.fn(async () => undefined);
+
+      await built.orchestrator.serveReadAdvertisement('space-1', serve);
+
+      // The lock was taken (redis SET NX) and released (CAS eval).
+      expect(built.redis.set).toHaveBeenCalledTimes(1);
+      expect(built.redis.eval).toHaveBeenCalled();
+      // HEAD pinned BEFORE serving, on the right vault.
+      expect(built.vaultRegistry.getVault).toHaveBeenCalledWith('space-1');
+      expect(built.vault.pinHeadToMain).toHaveBeenCalledTimes(1);
+      expect(serve).toHaveBeenCalledTimes(1);
+      const pinOrder = built.vault.pinHeadToMain.mock.invocationCallOrder[0];
+      const serveOrder = serve.mock.invocationCallOrder[0];
+      expect(pinOrder).toBeLessThan(serveOrder);
+    });
+
+    it('serves WITHOUT a pin/lock when git-sync is globally disabled', async () => {
+      const built = build({ enabled: false });
+      const serve = jest.fn(async () => undefined);
+
+      await built.orchestrator.serveReadAdvertisement('space-1', serve);
+
+      expect(serve).toHaveBeenCalledTimes(1);
+      expect(built.redis.set).not.toHaveBeenCalled();
+      expect(built.vault.pinHeadToMain).not.toHaveBeenCalled();
+    });
+  });
+
  describe('module lifecycle', () => {
    it('registers exactly one interval on init and tears it down idempotently on destroy', () => {
      const built = build();
@@ -305,6 +305,53 @@ export class GitSyncOrchestrator implements OnModuleInit, OnModuleDestroy {
    }
  }

+  /**
+   * Serve a git smart-HTTP READ ADVERTISEMENT (`GET info/refs?service=git-upload-pack`
+   * or a dumb `GET HEAD`) with the repo's symbolic `HEAD` deterministically pinned
+   * to `main` (bug #3). The advertised `HEAD` symref decides a clone's default
+   * branch; the engine transiently checks out the read-only `docmost` mirror during
+   * a cycle, so an unsynchronized advertisement could route a clone to `docmost`
+   * (~1/4 of clones under continuous syncing).
+   *
+   * Running the pin + the advertisement under the SAME per-space lock the cycle
+   * uses guarantees no cycle is mid-flight while we pin (HEAD cannot flap) and that
+   * the pin never corrupts a cycle's checkout. The advertisement is cheap (a ref
+   * listing, no pack stream), so holding the lock for it is fine. A bounded
+   * retry-acquire absorbs a brief overlap with a cycle; if the lock still cannot be
+   * taken (a long cycle), we fall back to serving WITHOUT the pin — the cycle's
+   * finally-restore leaves HEAD on `main` between cycles, so the advertisement is
+   * still almost always correct (degrades only under sustained contention).
+   */
+  async serveReadAdvertisement(
+    spaceId: string,
+    serve: () => Promise<void>,
+  ): Promise<void> {
+    if (!this.environmentService.isGitSyncEnabled()) {
+      await serve();
+      return;
+    }
+    const result = await this.spaceLock.withSpaceLock(
+      spaceId,
+      async () => {
+        const vault = await this.vaultRegistry.getVault(spaceId);
+        await vault.pinHeadToMain();
+        await serve();
+      },
+      {
+        acquireRetry: {
+          timeoutMs: GIT_SYNC_PUSH_LOCK_RETRY_TOTAL_MS,
+          baseMs: GIT_SYNC_PUSH_LOCK_RETRY_BASE_MS,
+          maxMs: GIT_SYNC_PUSH_LOCK_RETRY_MAX_MS,
+        },
+      },
+    );
+    // Lock contended for the whole budget (in-progress / another replica): serve
+    // anyway. `serve` (backend.run) never ran inside the lock in this case.
+    if (typeof result === 'object' && result !== null && 'skipped' in result) {
+      await serve();
+    }
+  }
+
  /**
   * Drive ONE reconcile cycle for a space. The PULL->PUSH branch choreography
   * lives in the engine's `runCycle` (so it can never drift from the engine it