7d9cf99f6c9c073ce0139a104c6e864027629bb0
49 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 claude code agent 227
|
8f7da77939 |
fix(git-sync): correct stale S2-gate comment + drop vestigial currentPage narrowing (#119 F1)
The N1-D1 fix added an early `return {}` when `currentPage == null` in
importPageMarkdown. `currentPage` is a const, never reassigned, so from that
guard onward it is provably non-null — which made the cross-space (S2) gate's
comment false ("a not-found page still proceeds as before": a not-found page
now returns early above) and left dead null-handling around it.
- Rewrite the S2-gate comment: the null case is handled by the N1-D1 guard
above; here currentPage is guaranteed non-null. Confused-deputy / cross-space /
mirror-deletePage explanation kept intact.
- Drop the dead `currentPage &&` conjunct from the S2 condition (always true).
- Collapse downstream vestigial `currentPage?.` / `currentPage!` / the
`currentPage ? … : undefined` ternary to plain `currentPage.` — all
behavior-preserving (currentPage non-null after the guard). The unrelated
`page ? … : undefined` ternary (fresh findById that can be null) is untouched.
No runtime behavior change. jest gitmost-datasource.service.spec.ts: 34 passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
 agent_qa
|
e6a861bdaf |
fix(git-sync): a valid-but-nonexistent gitmost_id no longer wedges a space's sync (N1-D1)
A vault file whose `gitmost_id` is a WELL-FORMED UUID that matches no page (a stale id from a restored-from-backup file, or a copied/foreign id) fell through importPageMarkdown to writeBody() on a non-existent page, throwing "Page … not found". The push apply recorded that as a per-cycle failure that never cleared — refs never advance, so the whole space's sync looped on the failure indefinitely (observed live: a leftover orphan file kept a space stuck at "1 failure" every ~5s). Same user-visible impact as C9-D1, but the id is a valid uuid so the 22P02 guard does not catch it. Add the missing `currentPage == null` branch in importPageMarkdown: skip the unknown id as an inert no-op so the cycle succeeds and the rest of the space keeps syncing. Verified on the stand: pushing a valid-but-nonexistent gitmost_id now stays at 0 failures (was 1/cycle forever), logs a skip warn, and a concurrent legit edit still syncs. Unit test added; server suite green (2146). NOTE (separate design follow-up, not this commit): the reconcile still cleans the orphan file (it maps to no live page). ADOPTING such a file as a fresh page (the restore-from-backup use case, preserving the git-authored content) needs the title from the filename, which lives in the engine classifier, not this method. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
5b835fc185 |
fix(git-sync): coerce malformed parentPageId to root in createPage/movePage (#119 F1)
A non-UUID gitmost_id on a parent folder-note, used as parentPageId for a git-sync createPage (or as a movePage destination), wedged the entire space: the throw landed in `failures`, and push only advances refs when failures.length === 0, so the space re-attempted forever. createPage was the only user-influenced-uuid op left unguarded. The throw is a NotFoundException, not a 22P02 error: PageRepo.findById falls back to a slugId lookup for non-UUID input, finds no row, and PageService.create raises NotFoundException — so skipIfMalformedId (22P02-only) would NOT have caught it. Coerce-to-root is the correct fix: a non-UUID parentPageId is rewritten to root (undefined/null) so the page is created/moved at the space root instead of wedging. No data loss (page still created) and no duplication (push.ts writes the assigned id back to frontmatter, so the next sync matches by id, and the retry-adopt map re-parents once the vault id is fixed). Applied to both createPage and movePage (the move destination is reachable via two paths, one 22P02-swallowed-but-mislogged and one NotFound-wedging). The child pageId stays guarded by skipIfMalformedId. F2: softened the skipIfMalformedId comment (parentPageId is a second user-influenced uuid in create/move) and made the swallow log op-generic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
agent_qa
|
f36a2def73 |
fix(git-sync): a malformed non-UUID gitmost_id no longer wedges a space's sync (C9-D1)
A vault file with a broken/hand-edited `gitmost_id` frontmatter (e.g. `gitmost_id: [unclosed` or a non-uuid token) fed that value into a Postgres `uuid` predicate (page update/delete), throwing 22P02 "invalid input syntax for type uuid". The push apply recorded it as a per-cycle failure that never cleared — refs never advance when failures>0, so the WHOLE space's sync looped on the same failure indefinitely and no further legitimate change synced (found via web-test). Wrap the id-scoped write ops (import/delete/move/rename/restore) at the bind() seam: swallow exactly the 22P02 as an inert no-op so the cycle succeeds and the rest of the space keeps syncing; re-throw anything else. pageId is the only user-influenced uuid in these ops, so a 22P02 there unambiguously means it. Verified on the stand: pushing a non-UUID gitmost_id now logs a skip warn and the space stays at 0 failures (was 1 failure/cycle forever); a concurrent legit edit to another page still syncs. Unit tests: import/delete swallow 22P02, non-22P02 re-throws. Full server suite green (2145). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 agent_coder
|
5d45f5a85e |
fix(git-sync): close #119 blockers — dead edit-revert guard, cross-space guard, red suite (F5/S2/G1/A1/F7)
F5 (HIGH data-loss): guard #2 (GS-EDIT-REVERT) called a local key-sorting equality that never matched a real page (block ids + materialized defaults differ), so the guard was dead and a web edit on a git-sync space was silently reverted within one poll cycle. Use the package's authoritative docsCanonicallyEqual (strips block id + normalizes KNOWN_DEFAULTS), wired through the git-sync loader like sanitizeTitle; delete the dead local canonicalize/canonicalJsonEqual. S2 (security): importPageMarkdown targeted a page by the vault-file id without a spaceId check (deletePage had one) — a space-A vault file carrying space-B's page id could resurrect/overwrite/clear B's page. Mirror deletePage's guard: skip when the loaded page lives in a different space than ctx.spaceId. G1 (jest green): add sanitizeTitle + docsCanonicallyEqual to the loadGitSync mock; update the converter-gate + package golden expectations to the genuinely-fixed output (paragraph textAlign now round-trips, multi-block table cells emit HTML tables); fix the orchestrator spec's stale mock so the per-space enabled gate (added later) is satisfied. A1: the converter dropped heading textAlign on export (bare '## text'); emit a styled <hN> when aligned, symmetric to paragraphs — round-trips losslessly (level + align), no churn for unaligned headings. F7 (docs): reword the false 'single choke point' title-strip comment; correct push.ts docstrings that still described the removed standalone-CLI/daemon model. Adds regression tests: the F5 acceptance test (canonically-equal content with real uuids => writePageBody NOT called), the S2 cross-space import guard, and the A1 heading round-trip. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 claude-stand
|
67fa0d1a28 |
fix(git-sync): a git-revert of a page delete restores the page (review warning)
A git-side revert of a delete commit re-adds the page's .md; the push classifier saw an add carrying a known pageId and emitted an UPDATE, writing the body to the still-trashed page. It stayed in Trash and the next pull re-deleted the file, so the revert was silently nullified (permanent vault<->Docmost divergence). In importPageMarkdown, if the target page is soft-deleted, restorePage() it first (restorePage was already in the client seam but never called), then apply the body — so a git revert actually brings the page back. Verified on stand: git rm -> page trashed; git revert -> deleted_at cleared (page restored). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-stand
|
91d674fea6 |
chore(git-sync): remove dead Settings REST fields, warn on inert REMOTE_TEMPLATE, fix stale docstrings (review)
Dead code / doc-vs-code cleanup from review #4404: - Drop the unused Settings.docmostApiUrl/Email/Password fields (the native in-process datasource never reads them; the engine only used docmostSpaceId). Removed from the type, buildSettings, and the 4 engine test suites that fabricated them. - Warn ONCE at startup when GIT_SYNC_REMOTE_TEMPLATE is set — remote push is deferred (SPEC §7) so the value is currently inert; the operator now gets a log line instead of a silent no-op. - Correct stale docstrings that claimed live-destructive code was 'FAKES only / NEXT increment / no live wiring' (applyPushActions runs LIVE via runCycle->orchestrator); that importPageMarkdown receives a 'self-contained file (meta+body)' (it receives the stripped body); and that the Yjs body merge is '2-way' (3-way runs end-to-end when the base is present). No behavior change (except the new warning); build + git-sync smoke verified. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-stand
|
c838fdeebe |
fix(git-sync): review #4404 batch — sanitize-title echo, per-space gate, move-echo, merge-agreement, fence-aware conflict scan, e2e asserts
Addresses reviewer comment #4404 (critical + blocking): - Critical #2: renamePage skips the echo where the incoming title equals sanitizeTitle(current title) — a Docmost title with FS-hostile chars (: / " |, newlines, double-space, >120) was pulled to a sanitized stem then written back, permanently corrupting the real title. (datasource) - Blocking #3: runOnce enforces per-space settings.gitSync.enabled (the event path bypassed opt-in; any edited space would git-init + export). (orchestrator) - Blocking #6: movePage no-ops the position-less same-parent echo that clobbered the user's chosen sibling order. (datasource) - Blocking #9: hasConflictMarkers is fence-aware — '<<<<<<< HEAD' inside a code block (git-tutorial page) no longer trips the all-or-nothing gate that froze the whole space's refs. (push.ts) - Blocking #11: three-way tryMergeRegion short-circuits when live==target (diff3 agreement) instead of logging a false 'same-block conflict resolved to git' — the echo noise that masked real data-loss signals. (three-way-merge) - Blocking #12/#13: e2e-advanced — drop the delete-cap block (no such feature; failed with a scary '(data loss!)'); non-member assert now expects 404 (existence not leaked), not 403. Verified on stand: sanitized-title rename preserves DB title (vault file sanitized); non-enabled space creates no vault; fenced conflict markers ingest without jamming; build clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-stand
|
f2d12fd2cd |
fix(git-sync): make body ingest idempotent to stop idle churn + silent edit-revert (GS-EDIT-REVERT)
On a git-sync space, a page's web edit was silently reverted within ~1 poll,
and idle spaces showed dozens of 'update' actions per cycle with no real change.
Root cause: the vault->Docmost body ingest (importPageMarkdown) is re-run every
poll for pages the upstream change-detection mis-flags as changed (the
markdown<->ProseMirror round-trip is not byte-stable: JSON key order / default
attrs differ though the content is identical). Each call re-imports the SAME body
into the live collab doc -- a no-op at idle, but it CLOBBERS a concurrent human
edit still in the debounced (not-yet-flushed) Yjs doc.
Fix: skip the ingest when it is genuinely a no-op --
1) baseMarkdown byte-identical to the current file (vault unchanged), or
2) the parsed incoming body is canonically-JSON-equal (key-order-insensitive)
to the page's current Docmost content.
A real git-side change is neither, so legitimate git->Docmost ingests still apply.
Verified: idle churn 38 update/cycle -> 0; web edit on an affected page 0/3 -> 3/3
persisted; genuine git-side edit still ingests. Found by autonomous QA
(web-test-orchestrator) + independent verifier.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
agent_coder
|
fcc9ae0c24 |
test(git-sync): fix #119 review F1-F4 (bind assert, cross-move + strip tests, comment)
F1: git-sync.orchestrator.spec bind assertion now includes spaceId ('space-1'),
matching driveCycle's dataSource.bind({workspaceId,userId,spaceId}).
F2: add 4 non-vacuous tests for the cross-space move data-loss guard in deletePage
(CTX_SPACE with spaceId): move-out skips removePage (returns skipped:'moved-to-other-space');
same-space / not-found / already-deleted all still call removePage.
F3: add 2 tests for the ~<slugId> title-strip guard in renamePage (own slugId stripped;
a foreign ~<slugId> tail left intact).
F4: reword the gitmost-datasource 'single choke point' comment — the strip covers the
rename/update path, not every git-sync title write (createPage's filename-derived title
does not funnel through here).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
claude-stand
|
c7e034cab9 |
fix(git-sync): don't trash a page on cross-space move (move-to-space data loss)
A page moved to another space with git-sync enabled was sent to Trash and vanished from BOTH vaults. The source space's push phase sees the moved-away page's file gone from its vault and calls deletePage -> soft-delete, even though the page still lives in the destination space. Thread the reconciling spaceId into the bind context and, in deletePage, skip the soft-delete when the page's CURRENT space differs from the space being reconciled (a move-out): only the vault file is dropped, the page is preserved. Genuine in-space deletions are unaffected (space matches). Found by autonomous QA (web-test-orchestrator). Control: with git-sync OFF the move keeps deleted_at NULL; with it ON the page was trashed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude-stand
|
123e981808 |
fix(git-sync): strip cosmetic ~<slugId> disambiguation suffix from ingested title
Two sibling pages that share a title collide on one vault filename, so the layout appends a cosmetic ` ~<slugId>` suffix (engine disambiguate()). That suffix is a local filesystem artifact and must never become the page's real Docmost title, but on ingest the filename-derived title carried it back into the DB on some paths (observed: intermittent same-title collision left a page permanently titled "Title ~<slugId>"). Strip it in renamePage() — the single choke point every git-sync title write funnels through — but only when the trailing token equals THIS page's own slugId, so a genuine user title that legitimately ends in ` ~token` is never corrupted (slugId is a random nanoid). Repro: create two pages with the same title; ~1 in 4 the second page's title is permanently polluted. After fix: 0/6. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
b47751349f |
fix(git-sync): kill spurious marker-leaking conflict, concurrent-edit loss, flapping HEAD
Three more git-sync QA defects from the 2nd live pass on PR #119, plus a callout-fidelity nit: 1. SPURIOUS conflict leaked raw markers into canonical main (root cause). On an ordinary round-trip the only difference between the docmost mirror (normalize- on-write) and a user's raw push is trailing/empty-line normalization, which made git's line-based docmost->main merge CONFLICT, and the wedge fix then committed the file WITH literal <<<<<<< / ======= / >>>>>>> markers onto main (git and the DB silently diverged for cycles). Fix: on a conflict, normalize trailing/empty lines on BOTH sides (showStage :2:/:3:) before comparing — a trailing-only diff is recognized as spurious and resolved to the clean normalized form. A GENUINE same-block conflict is auto-resolved to OURS (git wins, mirroring the live-doc 3-way rule); the docmost side stays on the `docmost` branch + page history. Raw markers NEVER reach main again. 2. Concurrent UI<->git edit silently lost the UI side. The git->Docmost 3-way merge ran against a live Y.Doc that hadn't yet received the user's debounced in-flight edit, so git clean-applied (no conflict detected) and the edit vanished even on a different block. Fix: flush the pending debounced store before the merge so the in-flight edit is drained into the live doc first — a different-block edit is merged, a same-block one is detected and pinned to history (recoverable). 3. Smart-HTTP HEAD flapped to the read-only `docmost` mirror (~1/4 of clones). The engine transiently checks out `docmost` mid-pull and the host advertises whatever HEAD resolves to. Fix: VaultGit.pinHeadToMain(); the cycle restores HEAD->main in a finally; and the upload-pack ref advertisement is served HEAD-pinned under the per-space lock so it can never observe a mid-cycle HEAD. 4. (callout) clampCalloutType now mirrors the editor's GITHUB_ALERT_TYPE_MAP for non-schema aliases (tip->success, caution->danger, important->info) instead of flatly collapsing to info. The editor schema genuinely supports only the six banner types, so unknown types still fall back to info (by design). Tests: deterministic real-git trailing-blank round-trip (no conflict, no markers, in sync over 2 cycles) + genuine-conflict no-marker-leak; HEAD advertisement stability; pre/post-flush concurrent-edit survival; serveReadAdvertisement lock pin; widened callout-alias coverage. Engine vitest + server tsc + collaboration / git-http / orchestrator specs all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
b7e5cb6970 |
fix(git-sync): push 503 starvation + concurrent-edit marker leak/silent loss
Bug #1 (push 503 starvation): an external receive-pack that briefly overlapped a poll cycle immediately 503'd because the per-space single-writer lock was held. Add a BOUNDED retry-acquire on the PUSH path only (SpaceLockService .withSpaceLock acquireRetry: capped exponential backoff up to ~5s); a transient overlap now waits and succeeds, a genuinely stuck cycle still 503s after the bound. The poll cycle passes no retry (immediate skip). Push result stays deterministic: the receive-pack only runs once the lock is held, so a 503 never leaves a half-applied ref. Bug #2 (concurrent-edit marker leak + silent same-block loss): - Marker leak (a): the push UPDATE path stripped markers for the body sent to Docmost but left raw <<<<<<</>>>>>>> committed on the published `main` vault forever (autoMergeConflicts ON). Now the cleaned body is written back to the vault file + recorded in writtenBack so runPush commits it on `main` and the vault converges to clean bytes. - Marker leak (b): pin merge.conflictStyle=merge in ensureRepo and teach stripConflictMarkers/hasConflictMarkers about the diff3 `|||||||` base section (drop the marker AND the stale base region) so diff3/zdiff3 conflicts can never leak `|||||||` + base content into a page. Also scrub the 3-way merge BASE markdown. - Silent same-block loss: the block 3-way merge still resolves same-block conflicts deterministically to git, but it is no longer silent: diff3Plan now reports a conflict count (mergeXmlFragments3WayWithStats), gitSyncWriteBody logs it, and the persistence boundary-snapshot now fires for git-sync writes over a non-git-sync baseline so the human's pre-merge content is preserved in page history (recoverable). Full both-preserved persisted-conflict UI remains the deferred redesign. Tests: space-lock bounded-retry (success/stuck/poll-immediate); push vault-clean + diff3 ||||||| strip; ensureRepo conflictStyle pin; diff3Plan/3-way conflict counts; persistence git-sync boundary snapshot. Server tsc clean; git-sync vitest + server collaboration/git-sync jest all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
906733b5c8 |
fix(git-sync): address PR #119 review #4 — symlink guard, dead-code cull, changelog + warnings/suggestions
Blocking (review id 2514): - [security] Forbid symlinks in vaults. ensureServable now sets core.symlinks=false in each vault's local git config (a pushed symlink is checked out as a plain file, never a real link), and the engine cycle wraps every read/write/mkdir in an lstat/realpath guard (new path-guard.ts) that refuses a path that is — or traverses — a symlink, or whose realpath escapes the vault root. Prevents a writer from publishing /etc/passwd or the server .env, or writing outside the vault. Adds unit tests (path-guard.test.ts) + a read-guard integration test (cycle.test.ts) + real lstat/realpath in the roundtrip integration test. - [simplification] Delete dead lib/diff.ts + test/diff.test.ts and drop the now-unused @fellow/prosemirror-recreate-transform dependency. - [documentation] Add a CHANGELOG [Unreleased] → Added entry for git-sync. Warnings: - [test-coverage] Cover the CREATE-branch conflict-markers guard (a new .md with markers and no gitmost_id is recorded as a create failure, never created). Suggestions: - [stability] Bound each `git config` in ensureServable with a timeout. - [authz] Trigger endpoint resolves spaceId workspace-scoped and 404s a foreign space before any vault directory is created. - [stability] Attribute git-initiated moves to the service account (lastUpdatedById), via an optional actor param on PageService.movePage. - [documentation] Document the per-space autoMergeConflicts toggle in AGENTS.md. - [test-coverage] Cover the unterminated `:::` callout fence fallback. - [simplification] Move test-only roundtrip-helpers.ts out of src/ into test/. Architecture: - Move the Yjs/ProseMirror merge primitives (yjs-body-merge, three-way-merge, lcs + specs) into collaboration/merge/, breaking the collaboration → integrations/git-sync dependency cycle this PR introduced. - Port the schema-surface drift gate to packages/mcp (the mcp schema mirror had none); pins 52 entries. Deferred (with rationale in the review thread): the incremental-pull perf warning (correctness-neutral; needs a high-water-mark design + its own tests on the data-loss-critical path) and the redis-sync rolling-deploy mixed-version edge (the deficient behavior is in already-released old-instance code; the new code is correct on both sides; impact is a transient rollout-window artifact). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
 a
|
f020739bfd |
refactor(git-sync): address PR #119 review #3 — honest gitRemote scaffolding comments, env example, shared ESM bridge
1. gitRemote is NOT yet consumed (the vendored engine has no remote-push path,
SPEC §7). Corrected the buildSettings docstring (it wrongly called gitRemote
"load-bearing") and marked the env -> validation -> getter -> buildSettings
chain as inert SCAFFOLDING for the deferred remote-push feature at all three
sites. Kept the wiring (harmless; removing only churns).
2. .env.example: document that GIT_SYNC_REMOTE_TEMPLATE substitutes the literal
"{spaceId}" per-space (with the example), so an operator doesn't point every
space at one remote.
3. Extracted the copy-pasted CJS->ESM dynamic-import bridge
(`new Function('s','return import(s)')`) into one shared
common/helpers/esm-import.ts; git-sync.loader, docmost-client.loader and
mcp.service now import it and keep their own typed loadX() wrappers.
Deferred (notes only, not implemented):
- lcs.ts + three-way-merge.ts could move into packages/git-sync, but that engine
is vendored (manual re-sync) — added a one-line note at three-way-merge.ts to
revisit once the re-sync story is settled.
- schema-core single source + BullMQ/fencing remain documented from prior rounds.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
a
|
22e3fcdeba |
fix(git-sync): address PR #119 review #2 — throttle /git Basic auth, fix mcp schema drift + warnings/tests
Must-fix:
- Throttle the raw /git HTTP-Basic path: it bypasses Nest/ThrottlerGuard, so
verifyUserCredentials (bcrypt) ran unthrottled. Wrap it in the SAME
FailedLoginLimiter the /mcp path uses (5/60s; per-IP, per-IP+email, global
per-email keys; atomic tryReserve BEFORE bcrypt; success resets, non-credential
errors release). The (threshold+1)-th attempt now gets 429 pre-bcrypt. Sweep
timer + onModuleDestroy mirror McpService.
- Fix the mcp schema mirror drift: packages/mcp details `open` attr now reads via
hasAttribute (matches editor-ext canon + git-sync copy); getAttribute dropped a
bare `<details open>` state. (build/ is gitignored — rebuilt locally.)
Tests added:
- /git brute-force throttle: pre-bcrypt 429 on the 6th failure; success resets;
non-credential error releases the budget.
- git-http-backend lost-lock AbortSignal: already-aborted -> no spawn + 500;
live abort mid-request -> SIGTERM + response closed.
- orchestrator divergentDocmost -> WARN + flag surfaced in status (+ clean case).
- pollTick re-entrancy guard skips an overlapping tick.
- datasource NotFound early-throws (getPageJson/move/rename) + updatedAt:undefined
stale-read branch (importPageMarkdown/createPage).
Suggestions:
- space.repo updateGitSyncSettings: parameterize the jsonb key (`${prefKey}::text`)
instead of sql.raw (latent-injection footgun); value stays sql.lit. Spec updated.
- pollTick re-entrancy guard (private `polling` flag).
- page-change.listener docstring: honest about the move/rename/delete over-skip
(loop-guard keys only on lastUpdatedSource) -> ~poll-interval latency, not loss.
- AGENTS.md: document the root /git smart-HTTP route + GitSyncModule.
- Remove redundant redteam-provenance.spec.ts (covered e2e in
persistence.extension.spec.ts:145).
- Extract the duplicated SIGTERM->SIGKILL+finish block (watchdog + abort) into
terminateChild; centralize watchdog-timer teardown in done().
Architecture (deferred, documented): mcp schema header now carries the three-copy
keep-in-sync + schema-core note; the editor-ext contract test documents that the
mcp copy and attribute-behaviour drift (details `open`) are not mechanically
covered yet.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
a
|
7179f8a5b2 |
fix(git-sync): address PR #119 review — close 403/404 space-existence leak + warnings/tests/arch
Security (must-fix):
- /git smart-HTTP gate: an authenticated NON-member of a git-sync space now gets
404 (not 403), so the 403<->404 difference can no longer be used to brute-force
which spaces exist / have git-sync enabled. 403 is reserved for a MEMBER who
lacks the required role (existence already known). New gate input
userIsSpaceMember; decision-table + service specs extended.
Config (must-fix):
- Remove the dead GIT_SYNC_SSH_KEY_PATH knob (getter + validation field + two
.env.example lines) — it had zero consumers and advertised a nonexistent push
capability.
Stability/docs (warnings):
- Wire the lost-lock AbortSignal into runReceivePack -> git http-backend so the
receive-pack child is killed if the per-space lock lapses mid-write.
- Raise the divergent-`docmost` (invariant §5) push refusal from info -> warn and
surface divergentDocmost in the run status (/status).
- Comment the stale read-after-debounced-collab-write updatedAt in
importPageMarkdown (deferred §10 loop-guard must not trust it).
- Fix the Dockerfile comment: the loader uses require.resolve + dynamic import(),
it deliberately does NOT require('@docmost/git-sync').
- Merge the two near-identical space toggle handlers into one parameterized
handler; add the 2 missing en-US i18n keys for the auto-merge switch (ru-RU not
maintained for these git-sync strings, mirrored).
Tests:
- isGitSyncHttpEnabled() default-branch (unset -> isGitSyncEnabled fallback).
- agentSourceFields 'git-sync' case (source stamped, chat key omitted).
- editor-ext name-level schema contract (vendored mirror superset of editor-ext
node/mark types) + the new shared resolver + non-member 404 gate cases.
Architecture:
- Extract resolveRequestWorkspace shared by DomainMiddleware + GitHttpService
(the two real self-hosted/cloud copies; McpService has no cloud branch).
- Document the in-process setInterval multi-replica limitation + BullMQ/fencing
future direction (deferred, not implemented).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
a
|
fe4adf23a0 |
fix(git-sync): unwedge per-page conflicts, preserve callout types, flush collab on disconnect
Addresses QA findings on PR #119 (issues #235/#236). SYNC-WEDGE (HIGH): one same-line conflict on one page froze sync for the WHOLE space in both directions forever. The pull's docmost->main merge left the vault mid-merge, so every later cycle's isMergeInProgress() check returned skipped:"merge-in-progress" and skipped the entire space with no recovery. - pull.ts now COMMITS a conflicting merge with markers in place (commitMerge): cleanly-merged pages land, the conflicted page carries its markers on main and is isolated by the existing push-side conflict-marker skip (markers never reach Docmost), and the next cycle is no longer wedged. conflictedPaths is surfaced. - cycle.ts now RECOVERS a vault left mid-merge by a prior/pre-fix cycle: it aborts the stale merge (merge --abort, hard-reset fallback) and continues, instead of skipping the space forever. - git.ts: listUnmergedPaths / commitMerge / abortMerge / resetHardToHead. CALLOUT TYPE FIDELITY: git-sync's CALLOUT_TYPES was missing "note" and "default" (editor-canonical types), so [!note]/[!default] callouts flattened to [!info] on every round-trip. Aligned the list with @docmost/editor-ext getValidCalloutType. LOSS-ON-FAST-CLOSE: editing a page then closing the tab inside the collab debounce window (~3-18s) lost the edit, because with unloadImmediately:false Hocuspocus does not flush the debounced onStoreDocument on the last-client disconnect. PersistenceExtension.onDisconnect now flushes the pending store (debouncer.executeNow) on the last disconnect only, with no redundant write. DUPLICATION re-verify (#1): the schema-default merge-key normalization is intact; faithful toYdoc-based reproduction shows callout + rich content resync with 0 ops and no growth/strip across cycles -> the re-report was leftover vault data, not a live regression. Locked with a callout regression spec. Tests: git-sync 688 pass (incl. real-VaultGit wedge-recovery integration); server git-sync+collaboration 285 pass; new callout merge/fidelity + onDisconnect-flush specs. tsc --noEmit clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
e48d7720e9 |
fix(git-sync): propagate nested details open; drop dead delete-cap wiring; cover lost-lock abort + lose-prone atom round-trips
Addresses review 1863 (delta) on PR #119. MUST-FIX: - detailsToHtml (the raw-HTML path used for a details nested inside columns/spanned cells) now emits `<details${open}>`, mirroring the top-level case, so `open` no longer silently drops every round trip. - Remove the dead `resolveApplyClient` delete-cap hook from the engine `runCycle`: the orchestrator stopped passing it, so the hook + its dry-run pass were inert. Deletes are soft (Trash) + always logged and engine convergence is the guard, so no cap is re-added — just the dead wiring removed. TEST COVERAGE: - space-lock: heartbeat refresh CAS-miss (eval -> 0) and Redis-error (eval throws) both abort the in-flight fn's signal. - cycle: a pre-aborted signal (and an abort during the pull read) throws before the push apply / first destructive phase. - converter: htmlEmbed source VALUE + height survive; encode/decode UTF-8 symmetry and '' -> ''; footnote definition body + ref/def id match; transclusionReference both ids survive; fix the bad transclusionSource fixture (wrong `pageId` attr + empty content -> schema `id` + a block child); nested details `open` parity test. - orchestrator: autoMergeConflicts:true reaches engine settings; default false on a missing settings row. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
42e618ec7f |
fix(git-sync): normalize merge key against schema defaults — cover all node/mark default-attr duplication triggers (image, link, highlight, …)
The point-fix (
|
||
|
claude code agent 227
|
857a0064f7 |
fix(git-sync): make reconcile import truly idempotent — stop runaway whole-body duplication
The live Yjs document materializes the editor-schema default `indent: 0` on
every paragraph/heading (and on the paragraph inside every list item, callout,
and table cell), but a body re-imported from git — parsed from clean markdown —
carries no indent attribute. So every live block's merge key differed from the
same block coming back from git: the three-way merge could anchor on nothing,
and the trailing unit that git's export already contained (but the merge could
not match against the byte-identical live tail) was re-appended on every
reconcile cycle. Each grown export then diverged from the last-pushed base by one
more unit — a self-sustaining, unbounded whole-body duplication loop with no
client connected.
The prior fix (
|
||
|
claude code agent 227
|
daf6c9ea16 |
fix(git-sync): propagate remote custom-event handler errors instead of 30s timeout
When a git-sync body write (gitSyncWriteBody) is routed to the collab instance
that owns the doc, the handler runs remotely inside handleRedisMessage and CAN
throw (markdown->ProseMirror transform). Previously the throw was uncaught: the
customEventComplete reply was never published, so the origin's writePageBody
promise only rejected after customEventTTL (~30s) as a generic 'TIMEOUT', and an
unhandledRejection escaped the async messageBuffer listener on the owning
instance.
Now the owner wraps handleEventLocally in try/catch and, on throw, publishes a
customEventComplete carrying an `error` field on the same correlation channel.
The origin's pendingReplies holds {resolve, reject} and rejects promptly with the
real Error. The TTL TIMEOUT remains as the fallback for a genuinely lost reply.
The no-throw and local (same-instance) paths are unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
claude code agent 227
|
9e69d917ee |
fix(git-sync): converge git-ingest with open editor sessions — stop silent revert/data-loss on live pages
A git push to a page with an OPEN editor was silently reverted: the git commit landed and the DB body updated, but the page in the browser stayed on the old content and the editor's next autosave overwrote the git change. Root cause (distributed, not in the merge): writeBody applied the body merge via collabGateway.openDirectConnection on whichever instance/process runs git-sync (the api/worker). When an editor is connected to a DIFFERENT collab instance/process, that opens a SEPARATE, detached Y.Doc. The merge landed in the detached doc + DB, but the live editor's Y.Doc never received the Yjs update; its debounced autosave then persisted its STALE state over the DB, reverting the git change (and, for concurrent edits to different paragraphs, losing the git side). In one process the bug is invisible because the direct connection already shares the editor's doc. Fix: route the body write through the existing custom-event channel (the same mechanism comment-marks and updatePageContent use) so the merge runs on the instance that OWNS the live doc. Its update is then broadcast to every connection (Document.handleUpdate) and the editor's CRDT converges on the merged result. New CollaborationGateway.writePageBody dispatches to a new gitSyncWriteBody handler (builds incoming/base docs before opening the connection — crash-safe — then 3-way/2-way merges into the live fragment); without redis it runs locally on the single (owning) instance. writeBody now just forwards the converted ProseMirror bodies + service userId. Evidence: - git-ingest-convergence.spec.ts: deterministic two-Y.Doc repro. PATH B (undelivered update) asserts the LOSS (the bug); PATH A (update delivered, as the owner-routed write does) asserts the git change SURVIVES and that concurrent edits to different paragraphs both survive. - collaboration.handler.git-sync.spec.ts: exercises the real gitSyncWriteBody against a shared doc wired to a connected "editor" doc (models the owning-instance broadcast) — editor converges, concurrent edit preserved, crash-safe on transform failure. - gitmost-datasource.service.spec.ts: writeBody now routes via writePageBody (RED before this change — it called openDirectConnection). Honest scope: the failure is cross-instance; full multi-instance convergence needs a live Hocuspocus + redis and is not provable in a unit test, so the convergence invariant is captured at the Yjs update-exchange level. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
2594828758 |
fix(git-sync): idempotent first-block reconciliation — stop start-of-doc content duplicating every sync cycle
The block-level body merge keyed each block by its full attribute set, including the per-block UniqueID the editor stamps on every heading/paragraph. A body arriving from git is parsed from clean markdown and carries no block ids, so a live block (id present) never matched the same block coming from git (no id). The three-way merge's LCS could not anchor on it, and an incoming block with no matching anchor — content inserted at the TOP of the page — was re-added on every push/pull cycle: a non-convergent, unbounded duplication loop. Exclude the volatile 'id' attribute from the block comparison key (serializeXmlNode) so blocks compare by content across the git round-trip. The merge keeps the live block INSTANCE (and its id, and any in-flight edit) for an anchor — picks are by index, not key — so identity is preserved while reconciliation becomes idempotent. Mirrors canonicalize.ts, which already strips the regenerated block id from the round-trip idempotency comparison. Adds a RED-before-fix repro modelling the live-id vs git-no-id asymmetry and asserting no block growth across cycles. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
e777ebcf4f |
feat(git-sync): remove the per-cycle delete cap; deletes apply + are logged every cycle
The delete cap (GIT_SYNC_MAX_DELETES_PER_CYCLE, default 5) was a defense-in-depth guard that SUPPRESSED a cycle's deletions when the planned count exceeded the limit. In practice it was a crutch over engine correctness that also blocked legitimate deletes: deleting a folder with many child pages is a normal action, and git-sync deletes are SOFT (Trash, reversible), so a blocking limit has little upside and real downside. There is also no user-facing surface to "confirm" a large delete from a background sync — the only channel is the operator log. So: drop the cap entirely. Deletes apply unconditionally; every cycle already logs its full push plan, per-action `delete: <pageId>` lines, and completion counts through the engine `log`, so what was deleted (and what was skipped) is always recorded. Engine correctness (the reconcile/layout/round-trip tests) is what prevents phantom deletions — not a blocking cap. Removed: orchestrator `resolveApplyClient` cap hook + `maxDeletes`, `getGitSyncMaxDeletesPerCycle`, the `GIT_SYNC_MAX_DELETES_PER_CYCLE` env/validation/.env.example, and the cap tests. (The engine's generic optional `resolveApplyClient` hook is left as an unused extension point.) server tsc clean, git-sync + environment jest 174. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
5125296bfa |
fix(git-sync): subpages round-trips (was {{SUBPAGES}} literal) + exhaustive all-node round-trip test
subpages exported to the literal `{{SUBPAGES}}`, which has no markdown/HTML
inverse, so on re-import it came back as a plain paragraph holding the visible
text "{{SUBPAGES}}" — the embed rendered as that literal string on the page
after a sync (round-trip data loss, seen live). It now emits the schema-matching
`<div data-type="subpages">` like every other embed node, so the schema's
parseHTML rebuilds the subpages node. Also dropped the leaf-atom content-hole
in the subpages renderHTML.
New committed regression coverage:
- packages/git-sync/test/roundtrip-all-nodes.test.ts — exhaustive serialize ->
deserialize round trip for ALL 40 node/mark types; each asserts the node/mark
survives and no `{{...}}` literal leaks. This is the test that caught subpages.
- §13.1 gate (git-sync-converter-gate.spec.ts): subpages added to the green
corpus (round-trips through the REAL server schema).
- Corrected two PR-authored tests that asserted the old {{SUBPAGES}} loss as
"by design" — they now assert the fixed round trip.
Also folds in review #1679 coverage-gap tests (no prod change): orchestrator
pollTick/enabledSpaces, datasource 3-way merge dispatch, page.repo
last_updated_source provenance SQL.
git-sync vitest 659 (+1 expected-fail), server tsc clean, server specs green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
||
|
claude code agent 227
|
a40a00d5c5 |
feat(git-sync): per-space toggle for conflict-marker handling on push (#13)
Red-team #13 (conflict markers reaching Docmost) is now a per-space policy exposed as a UI toggle, instead of a hardcoded behavior. New boolean `gitSync.autoMergeConflicts` (default FALSE), mirroring the existing per-space `gitSync.enabled` flag end-to-end (jsonb space settings -> update-space DTO -> space.service -> client types -> space settings form switch): - OFF (default, safe): a page whose committed body still has unresolved git conflict markers is NOT pushed — it is recorded as a per-page push FAILURE ("unresolved conflict markers — resolve in git first"). Recording a failure (not a soft skip) deliberately HOLDS refs/docmost/last-pushed so the conflict commit is never marked pushed and a later pull cannot clobber the user's in-progress resolution; the page retries until the conflict is resolved in git. - ON: the marker lines are stripped and both sides' content is pushed (the prior behavior), so the conflict becomes visible/fixable inside Docmost. The engine Settings carries `autoMergeConflicts`; runPush threads it into the update AND create paths. The orchestrator's buildSettings reads the per-space flag from jsonb (strict opt-in like `enabled`, default false). Tests: redteam-push-cycle #13 rewritten (default -> not pushed + failure + refs held; ON -> strip-and-push); space.service + edit-space-form + orchestrator specs extended. git-sync vitest 618, server jest space+git-sync 163, client edit-space-form 11, server/client tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
81c0226be7 |
docs(git-sync): document GIT_SYNC_BACKEND_TIMEOUT_MS, drop dead consts, fix dangling plan refs
Address the non-red-team documentation/cleanup items from review #1679: - Document the GIT_SYNC_BACKEND_TIMEOUT_MS watchdog (git http-backend) in .env.example and add it to the environment validation schema — it was used (getGitSyncBackendTimeoutMs, default 120000) but undocumented/unvalidated. - Remove the dead GIT_SYNC_DEBOUNCE_MS_DEFAULT / GIT_SYNC_POLL_INTERVAL_MS_DEFAULT exports (never imported; environment.service is the single source of defaults). - Redirect the dangling `plan §X.Y` comment references to issue #194 (the git-sync spec moved there when docs/git-sync-plan.md was deleted by this PR). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude code agent 227
|
d5079aa1d8 |
fix(git-sync): red-team hardening — 12 confirmed sync-breaking bugs + regression tests
A 10-agent red-team pass on the two-way Docmost<->git sync surfaced 16 ranked findings (9 others triaged out as already-defended). Wrote a reproduction test per finding (each asserts the CORRECT behavior, so it fails on the bug), then fixed the production code so every repro goes green. All confirmed bugs: Round-trip data loss (markdown-converter.ts + docmost-schema.ts mirror): - #1 editor-ext node types silently dropped on export — ported the 8 missing canon nodes (footnoteReference/footnotesList/footnoteDefinition, htmlEmbed, status, pageEmbed, transclusionSource/Reference) into the git-sync schema mirror and added converter cases that emit their schema-matching HTML instead of flattening unknown nodes to '' (this was the critical data-loss flagged in review #1679: footnotes/htmlEmbed lost on sync). Snapshot surface updated. - #2 top-level image lost width/height/align/attachmentId — now emits an HTML <img> (like video/diagrams) when it carries layout attrs; bare images stay . Image node parses width/height as strings so they re-import. - #3 code block containing a ``` fence corrupted on round-trip — outer fence is now widened to (longest-inner-backtick-run + 1). - #16 deep nesting threw RangeError (page never synced) — added a depth guard (MAX_NODE_DEPTH=400) so the converter never overflows the stack. Push/layout/cycle (engine): - #4 disambiguation ' ~slugId' suffix corrupted Docmost titles + order-dependent layout — deterministic, order-independent sibling disambiguation; suffix is stripped from a path-derived title ONLY when the new name is exactly the old title plus the suffix (never a genuine retitle ending in ' ~token'). - #6 retry-adopt by (parent,title) clobbered the wrong duplicate-title sibling — ambiguous (parent,title) is no longer adopted (falls back to fresh create). - #12 a new child under a new parent was created at ROOT — creates are ordered parent-before-child with an in-memory created-id map for parent resolution. - #13 git conflict markers could reach Docmost — bodies are scanned and the marker lines stripped (a '=======' line is only treated as a conflict separator inside a <<<<<<< ... >>>>>>> block, so setext headings are safe). - #15 a divergent `docmost` mirror was escalated by runPush but dropped by runCycle — RunCycleResult now forwards divergentDocmost to the orchestrator. Server (merge / lock / provenance): - #9 3-way merge lost a human's block edit when git inserted an adjacent block — finer-grained diff3 region merge (via lcs) preserves non-overlapping human edits; genuine same-block conflicts still resolve git-wins. - #10 single-writer race — module-static liveLocks closes the same-process TOCTOU window, and a heartbeat refresh that cannot confirm the lock now aborts the cycle at its next write checkpoint (cooperative AbortSignal threaded through runCycle). Cross-process fencing tokens remain a follow-up. - #14 sticky-agent provenance overrode an explicit actor='git-sync' write, blinding the listener loop-guard — resolveSource now lets an explicit actor win over the sticky-agent fallback (explicit agent still wins). Verified: git-sync vitest 617 pass (+1 expected-fail), server unit jest 1541 pass, server tsc clean. A review pass over the fixes caught and corrected a title-suffix over-strip, an inert abort signal, a document-wide conflict-marker strip, and two leaf-atom content-holes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
claude_code
|
28d2560dfd |
fix(git-sync): address PR #119 review (#1571)
Resolve the code-review findings from comment #1571 on PR #119. Engine (packages/git-sync): - Idempotent CREATE on retry: before createPage, look the page up in the live Docmost tree by (parentPageId, title) and ADOPT it instead of duplicating when a prior cycle created it but failed to persist the pageId back to disk. Only trust a COMPLETE tree for the lookup; fall back to createPage otherwise. Covered by new tests incl. a complete=false regression-lock. - Route applyPullActions diagnostics through an injected logger instead of bare console (thread log from the cycle). - Add a timeout to the git execFile chokepoint (runRaw) so a hung git subprocess cannot wedge a sync cycle. - Translate remaining Russian code comments to English. - Remove dead standalone-CLI code (parseArgs/PushParsedArgs, parseSettings/envSchema, loadSettingsOrExit + config-errors.ts) and the matching index exports/specs; keep the Settings type. - Fix the dangling docs link in package.json. - Add a schema-surface snapshot guard so any drift in the vendored document schema is a loud, must-review CI failure (+ provenance header). Server (apps/server): - Add a configurable watchdog timeout to the spawned git http-backend so a stalled push cannot hold the per-space lock forever (GIT_SYNC_BACKEND_TIMEOUT_MS). - Close the in-process TOCTOU window in SpaceLockService.withSpaceLock by reserving the slot synchronously before acquire. - Add tests: removePage git-sync provenance (both branches), ensureServable force-push-protection git configs, and the phase-B+ datasource methods. Docs / build: - AGENTS.md: list git-sync as the fifth workspace package and note the three schema mirrors; fix the dangling git-sync-plan.md backlog link. - pnpm-lock.yaml: add the missing @docmost/git-sync workspace link so pnpm install --frozen-lockfile (CI default) succeeds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
5da12e89f9 |
refactor(git-sync): internalize the engine — first-class ESM, no vendoring bridge (#119 review)
Closes the architecture item from the #119 review: drop the "vendored from docmost-sync" framing and the CJS↔ESM `Function('import()')` bridge so the engine is a normal first-class gitmost package. Part 1 — vendoring markers removed (prose only, zero behavior change): reworded "VENDORED into gitmost" / "vendored from docmost-sync" / "Engine LOGIC is byte-identical" / "it's a port" comments across the engine. Behavior-bearing strings are untouched: BOT_AUTHOR_NAME/EMAIL and the `Docmost-Sync-Source:` provenance trailers (changing them would break git authorship + the loop-guard). Part 2 — the package is now ESM (matching the sibling @docmost/mcp): `type: module`, tsconfig Node16, `.js` extensions on relative imports, and a static `import { marked }` replacing the `new Function('return import(...)')` / `loadMarked` hack — the bridge is GONE from the package. The CommonJS NestJS server loads the now-ESM engine via a new `git-sync.loader.ts` that mirrors the existing `docmost-client.loader.ts` mcp loader exactly (Function-indirected dynamic import + cached promise + retry-on-reject). The 4 server consumers (orchestrator/datasource/vault-registry/git-http-backend) call `await loadGitSync()` for value exports; types stay `import type` (erased). The converter-gate spec — which needs the real converter — loads the package's TS source via a jest moduleNameMapper + isolatedModules (documented in that spec); the other git-sync specs mock the loader. Verified: engine builds pure ESM (no Function/require leftover), vitest 614, editor-ext build, server + client tsc, full server jest 1397/0. Live stand smoke-test: server starts clean on the ESM engine (no ERR_REQUIRE_ESM), a real sync cycle runs through the loader, and the basic e2e suite is 12/12 (clone via git-http-backend, push, pull, delete, 3-way merge — all through the new loader). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
71375e25ee |
chore(git-sync): drop now-unused dirname import (PR #119 review)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
dc7a0ec9f5 |
refactor(git-sync): move the PULL->PUSH cycle into the engine as runCycle (PR #119 review, arch #1)
The reconcile choreography (ensureRepo -> merge-check -> ensureBranch ->
checkout('docmost') -> pull -> push) was hand-rolled in the app orchestrator's
driveCycle, duplicating an order the vendored engine owns and could drift from on
upgrade — the failure mode is data clobber. Lift it into @docmost/git-sync as a
single entry point, `runCycle(deps)`. The orchestrator now calls runCycle and
keeps only the lock (its caller) and the gitmost-specific delete-cap POLICY,
injected as the `resolveApplyClient` hook (the engine does the dry-run, hands the
hook the planned delete count — Infinity if planning failed — and uses whatever
client it returns for the apply). driveCycle drops from ~150 lines to ~30.
Tests:
- engine test/cycle.test.ts: composition (merge-in-progress short-circuit;
ensureRepo->ensureBranch->checkout staging order before the pull; the cap hook
is consulted with the planned count; no dry-run when no hook).
- engine test/cycle-roundtrip.test.ts: runCycle against a REAL VaultGit in a temp
repo with a faked Docmost client — a git-originated CREATE flows pull->push and
the assigned pageId is written back; an unresolved merge short-circuits before
any client call.
- orchestrator spec rewired to mock runCycle and assert the wiring + the
resolveApplyClient cap policy (the engine-internal cycle-order/merge tests moved
to the engine).
Validated end to end on a live stand (real Postgres/Redis + server): a git clone
-> edit -> push over the /git remote round-trips the change into the Docmost page
through the refactored cycle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
claude code agent 227
|
085a30575f |
test(git-sync): cover ingestExternalPush in the orchestrator spec (PR #119 review)
Closes the test-coverage warning that the smart-HTTP push ingest path was unexercised. Adds 5 cases: receive-pack streams BEFORE the Docmost cycle; a held lock throws GitSyncLockHeldError and runs neither the receive-pack nor the cycle; a post-push cycle error is swallowed (the push is durable, poll retries) while the lock is still released; a missing service user runs the receive-pack but skips the immediate cycle; and a globally-disabled git-sync refuses without touching the lock. (The 503/Retry-After mapping in git-http.service is the sibling warning; its spec is in the repo's pre-existing set of jest suites that can't load locally via the react-dom/tiptap transform chain, so that case is left for CI.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
95bc9fe98d |
refactor(git-sync): extract SpaceLockService from the orchestrator (PR #119 review, arch #2)
The per-space single-writer lock — Redis CAS leader lock (SET NX PX, DEL-CAS and PEXPIRE-CAS Lua), the in-process mutex, the per-process instanceId and the heartbeat — lived inline in GitSyncOrchestrator. Extract it into a dedicated @Injectable() SpaceLockService exposing one narrow surface, withSpaceLock(spaceId, fn), so the lock is the orchestrator's only Redis-lock touch-point and is testable in isolation. The orchestrator now injects SpaceLockService and both consumers (runOnce, ingestExternalPush) go through spaceLock.withSpaceLock — behavior unchanged (same sentinel returns, same 503-on-lock-held contract). Orchestrator drops 591→472 lines. Adds space-lock.service.spec.ts asserting the lock SEMANTICS against a fake Redis (the test-coverage warning from the review): the SET NX/PX args, the DEL-CAS and PEXPIRE-CAS Lua + ARGV[1]=instanceId, plus the lock-held / in-progress / throw- still-releases paths. The orchestrator spec is unchanged in count and stays green (it now builds the real SpaceLockService over its mock Redis). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
cca0bfe306 |
docs(git-sync): remove dangling references to the deleted git-sync-plan doc (PR #119 review)
The implementation spec docs/git-sync-plan.md was removed as completed, but ~44 code comments still cited it as "plan §N". Strip those citations (comments only), keeping each comment grammatical. The vendored engine's own "SPEC §N" references point at a different, still-present spec and are left untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
fb357cd52e |
refactor(git-sync): extract shared buildLcsTable for the two block diffs (PR #119 review)
The two-way block diff (yjs-body-merge.diffBlocks) and the three-way merge planner (three-way-merge.lcsPairs) built the identical backward-filled LCS DP table inline. Extract it to lcs.ts (buildLcsTable); each caller keeps its own traceback. No behavior change — merge specs unchanged and green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
177d8a31d4 |
fix(git-sync): hold refs on suppressed deletes + stamp delete/restore provenance (PR #119 review)
Two stability warnings from the #119 review: 1. delete-cap no longer drops deletions forever. When planned deletes exceed GIT_SYNC_MAX_DELETES_PER_CYCLE the apply client's deletePage now THROWS instead of resolving to a no-op. A throw is recorded by the engine as a per-page failure, so `refs/docmost/last-pushed` is NOT advanced past the commit that dropped the files — the next cycle re-diffs from the un-advanced ref and re-plans the same deletes (a transient over-cap is retried, not silently dropped and then recreated by the next pull). Previously a resolving no-op let the engine count `deleted++` with no failure, advance the ref, and never replay the deletions. 2. git-sync soft-delete and restore now stamp provenance. deletePage routes GIT_SYNC_PROVENANCE through pageService.removePage, and restorePage stamps lastUpdatedSource='git-sync' on the restore update — so the page-change listener's loop-guard (skip when lastUpdatedSource==='git-sync') recognizes both as its own writes instead of scheduling a wasted echo cycle. Done via a backward-compatible optional `lastUpdatedSource` param on pageRepo.removePage/restorePage (omitted for ordinary user deletes/restores). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
8fa32e8438 |
docs(git-sync): document GIT_SYNC_* env vars; fix stale/non-English comments (PR #119 review)
Addresses the documentation/convention warnings from the #119 review: - .env.example: add the GIT-SYNC block (9 GIT_SYNC_* vars with defaults), noting GIT_SYNC_SERVICE_USER_ID is required when sync is enabled. - yjs-body-merge.ts: translate the Russian review note in the docstring to English (comments-only-in-English rule). - persistence.extension.ts: correct the stale "git-sync writes are full-body replaces" rationale — a git-sync write is now a block-level merge into the live doc, which is why it is debounced like a human edit rather than snapshotted. - history-item.tsx: the GitSyncBadge version is created on the PUSH path (writing the git body back into the doc), not by the pull — fix the comment. - edit-space-form.tsx: log the raw error in the git-sync toggle catch instead of swallowing it (AGENTS.md). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
fa89cba023 |
feat(git-sync): three-way body merge using the last-synced base (no edit loss)
Upgrades the 2-way body merge to a real diff3 three-way merge (review #5), so a block ONLY the human changed is KEPT when git changed a DIFFERENT block — the 2-way merge would revert it to git's stale version. Engine: the push update loop reads the last-synced pre-image (`git.showFileAtRef(refs/docmost/last-pushed, path)`) and passes it as the optional `baseMarkdown` to `client.importPageMarkdown` (the common ancestor). Server: gitmost-datasource converts base+incoming, and writeBody runs a block- level diff3 (new three-way-merge.ts `diff3Plan`): live-only change -> keep live, git-only change -> take git, both-changed -> git wins (conflict policy), inserts/ deletes from either side preserved. Without a base (createPage) it falls back to the 2-way merge. Crash-safety unchanged (docs built before the connection opens). Tests: three-way-merge.spec.ts (14 — every diff3 case incl. the cross-block preservation and conflict policy), yjs-body-merge 3-way (real Y.Docs: human's block instance preserved while git's block is applied), plus an engine test that the base is forwarded from showFileAtRef. Existing push assertions updated for the new base arg. git-sync 589 pass; server merge/datasource/gate 62 pass; typecheck clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
3386bf2865 |
fix(git-sync): merge git body into the live doc block-by-block (no clobber)
Supersedes the active-session "defer" guard with a real merge (review #5 — "запись делать через мерж", not skip-while-editing). writeBody no longer does delete-all + re-insert (which discarded a concurrent editor's in-flight changes on every sync). It now diffs the live body against the incoming git body at TOP-LEVEL BLOCK granularity (LCS over a canonical structural serialization) and applies only the minimal inserts/deletes: - a block a human is editing is left UNTOUCHED when git changed a DIFFERENT block; - an unchanged resync is a complete 0-op write; - Yjs CRDT-merges the minimal ops with concurrent edits. New yjs-body-merge.ts (mergeXmlFragments + cloneXmlNode + diffBlocks) is pure-Yjs and unit-tested with real Y.Docs (8 tests): identical->0 ops, edit-one-block keeps the other block instances, append/delete keep neighbours, marks survive the cross-doc clone. Crash-safety kept: the incoming doc is built before the connection opens, so a transform failure can't empty the body. Removed: the ActiveEditSessionError defer path and the now-unused CollaborationGateway.getActiveEditorCount. Honest limitation: this is a 2-way merge — for a block BOTH sides changed since the last sync, git wins (no common ancestor to decide). A full 3-way merge would need the last-synced base plumbed from the engine; the dominant cases (unchanged resync, edits to different blocks) are now lossless. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
181a8330f3 |
fix(git-sync): don't clobber pages with a live editing session; crash-safe body write
Review finding #5: the git -> page body write (writeBody) did a full-body replace
(delete-all + re-insert) on the shared Yjs doc. Applied while a human is editing
the page, it discarded their in-flight changes; and TiptapTransformer.toYdoc ran
AFTER the fragment was cleared, so a conversion failure could leave the page with
an empty body.
Fixes:
- Active-session guard: CollaborationGateway.getActiveEditorCount(documentName)
reports live human (websocket) editor sessions for a doc, excluding server-side
direct connections. writeBody now throws ActiveEditSessionError when an editor
is connected. The engine's push loop already isolates each importPageMarkdown in
try/catch and does not advance the loop-guard on failure, so the write is simply
retried on the next poll once the editor disconnects — never a clobber.
- Crash-safe conversion: build the replacement Yjs update BEFORE opening the
connection / clearing the fragment, so a transform failure can never leave the
body empty.
Also updates the server-side converter gate spec to the corrected round-trip
shape: the block-image hoist no longer leaves a leading empty paragraph (the
git-sync converter fix in
|
||
|
claude_code
|
04032ae677 |
feat(git-sync): serve spaces over smart-HTTP (gitmost as a two-way git host)
Expose each git-sync-enabled space as a clonable/pushable git repo over HTTP, so `git clone https://<user>:<pass>@<host>/git/<spaceId>.git` works and external pushes flow back into Docmost pages — gitmost itself acts as the git host (no external GitHub/Gitea, no SSH). Transport: shell out to `git http-backend` (CGI; git is already in the runtime image) which implements the full smart-HTTP protocol (info/refs, upload-pack, receive-pack, protocol v2). A raw Fastify route `/git/*` (mounted at the root, outside the `/api` prefix) bridges the request/response to the CGI; passthrough content-type parsers for the git media types stream the raw body to stdin. Reuse the existing engine: clients push the vault's `main` branch, whose commits beyond `refs/docmost/last-pushed` the engine already reconciles into Docmost. - http/git-http.service.ts — auth (HTTP Basic -> AuthService.verifyUserCredentials), self-resolved workspace (DomainMiddleware does not run for this raw route), per-space gating (global + per-space gitSync flags, 404 hides existence), CASL authz (Read=fetch, Manage=push), dispatch. - http/git-http-backend.service.ts — spawn `git http-backend`, binary-safe CGI response parsing (Status/headers/body), stream to the socket. - http/git-http.helpers.ts — pure path parse, service->kind mapping, gate decision (unit-tested); rejects literal and percent-encoded path traversal. - orchestrator: extract reusable withSpaceLock (CAS-guarded lock heartbeat so a long push cannot let the lock expire mid-cycle) and add ingestExternalPush (receive-pack + Docmost cycle under one lock; 503 on contention). - vault-registry: ensureServable() — ensureRepo + idempotent receive.denyCurrentBranch =updateInstead / denyNonFastForwards / http.receivepack / http.uploadpack. - env: GIT_SYNC_HTTP_ENABLED (defaults to GIT_SYNC_ENABLED) + validation. - main.ts: register the /git/* route and the git content-type parsers. Tests: pure helpers, CGI parsing, and the GitHttpService handler (auth/gate/authz + workspace resolution). Server tsc + git-sync/env suites green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude_code
|
d9d1d54aaa |
test(git-sync): add reviewer-requested coverage across engine, server, client
Implements the test cases called out in the PR #119 review threads (code-review, test-strategy report, red-team) — TESTS ONLY, no production code changes. packages/git-sync (vitest): - lib converter/markdown gaps: pageBreak data-loss (it.fails repro), subpages lossy round-trip, nested/fenced callouts, ol->taskList bridge, column.width number<->string drift, empty details. - engine units: parentFolderFile, planReconciliation swap/chained move, buildVaultLayout last-resort-by-id, firstDivergence, applyPushActions / applyPullActions failure isolation. - real temp-git integration: diffNameStatus -z rename+add/modify alignment, copy-line behavior, per-invocation committer identity (no leak into repo/global config). - ENFORCED type-level GitSyncClient contract via vitest typecheck over a *.test-d.ts file (tsconfig.vitest.json; build tsconfig untouched). apps/server (jest): - orchestrator: delete-cap neutralization + fail-safe, Redis lock / mutex skip ladder + release-on-throw, merge guard, pull/push order, remote template substitution, poll lifecycle. - page-change listener: loop-guard, debounce coalescing, id resolution, error swallowing. - vault registry, controller authz (trigger + status), env validation/getters, page.service git-sync provenance stamping, persistence precedence (agent > git-sync > user) + no boundary snapshot, space.service audit-delta, space.repo jsonb-merge, converter-gate corpus extension (mention/math/details/marks). apps/client (vitest + testing-library): - history-item git-sync badge: render gating + non-clickable. - edit-space-form toggle: initial state, optimistic payload, rollback on error, disabled states. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
593f181bbc |
fix(git-sync): address review — configurable poll, always-on loop-guard, cleanup
Comprehensive-review follow-ups (APPROVE WITH SUGGESTIONS; no critical issues):
- poll interval is now actually configurable: replaced the hardcoded
@Interval('git-sync-poll', 15000) with a dynamic SchedulerRegistry interval
registered in onModuleInit from getGitSyncPollIntervalMs() (cleared in
onModuleDestroy); /status and the real cadence now share one config source.
Boots logging 'poll interval registered (Nms)'.
- loop-guard now ALWAYS applies: the lastUpdatedSource==='git-sync' skip was
nested inside the !spaceId/!workspaceId branch, so structural self-writes
(CREATE/MOVE/RESTORE/SOFT_DELETE, which carry spaceId+workspaceId) bypassed it
and re-triggered cycles. Fetch the page row once, guard unconditionally, then
resolve space/workspace.
- remove the dead PAGE_CONTENT_UPDATED subscription (it's a BullMQ job, never an
EventEmitter event; body edits arrive via PAGE_UPDATED).
- fix the stale datasource comment (PageService DOES stamp 'git-sync' now).
- env getters: parseInt radix 10 + NaN/<=0 fallback for poll/debounce (+ max
deletes), with 6 new environment.service.spec tests.
tsc clean; jest 723 pass; live cycle re-verified post-refactor (ran, push
applied, unflagged 92-page space untouched).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
claude code agent 227
|
8373360a67 |
fix(git-sync): branch choreography + strict scoping + delete cap (Phase B hardening)
Fixes found by the live pull/push e2e: - CRITICAL: driveCycle never checked out the 'docmost' branch before applyPullActions, so Docmost content was written straight onto 'main', clobbering local file edits before push could diff them. Now checkout 'docmost' before pull (applyPullActions commits there then checks out main + merges) — mirrors the engine's pull main(). Round-trip now works both ways. - add an unresolved-merge guard (SPEC §9): skip the cycle if the vault is mid-merge instead of failing on checkout. - SAFETY: enabledSpaces() is now STRICT opt-in — only spaces with settings.gitSync.enabled===true; removed the all-spaces fallback that synced every space (incl. a 92-page one) the moment GIT_SYNC_ENABLED flipped. - SAFETY: per-cycle delete cap (GIT_SYNC_MAX_DELETES_PER_CYCLE, default 5): dry-run the push, and if planned deletes exceed the cap, run the apply with deletePage neutralized — phantom absence-deletions from a non-convergent vault can't soft-delete real pages. Fails safe if the dry-run throws. - fix manual trigger: TriggerGitSyncDto.spaceId needs @IsUUID or the global whitelist ValidationPipe strips it (arrived undefined -> vault 'undefined'). Live-verified on an isolated flagged space: push (vault file edit -> Docmost content, stamped lastUpdatedSource='git-sync') and pull (Docmost rename -> vault file + meta) both work; an unrelated 92-page space stayed untouched throughout. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
e2493cafa9 |
feat(git-sync): GitSyncModule orchestrator + config + listener (Phase A.4b/B)
Control plane wiring (plan §5-§11): - PageService create/update/movePage now honor provenance actor 'git-sync' (stamp lastUpdatedSource='git-sync'), closing the A.4a gap. - EnvironmentService: GIT_SYNC_ENABLED / DATA_DIR / REMOTE_TEMPLATE / POLL_INTERVAL_MS / DEBOUNCE_MS / SERVICE_USER_ID (required-if-enabled) / SSH_KEY_PATH + validation. - VaultRegistryService: per-space vault path + cached VaultGit. - GitSyncOrchestrator: per-space Redis leader-lock (SET NX PX + CAS-Lua release, randomUUID instanceId) + in-process mutex; runOnce drives the vendored engine PULL (readExisting->computePullActions->applyPullActions) then PUSH (runPush) with the bound native GitSyncClient + VaultGit; @Interval poll-safety gated on GIT_SYNC_ENABLED; imports plain ScheduleModule (TelemetryModule owns forRoot). - PageChangeListener: @OnEvent PAGE_* -> per-space debounce -> runOnce, with a best-effort lastUpdatedSource==='git-sync' loop-guard. - GitSyncController: admin POST /api/git-sync/trigger + GET /status (ops/e2e). - GitSyncModule registered in app.module. Enabled-space enumeration uses settings.gitSync.enabled, falling back to all live spaces until Phase C writes the flag (master gate = GIT_SYNC_ENABLED). tsc clean; 713 tests/71 suites pass; dev server hot-reloaded the module (route live, DI graph boots). Live pull/push round-trip verified next. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
claude code agent 227
|
5a4d9f84d7 |
feat(git-sync): native GitmostDataSource + 'git-sync' provenance (Phase A.4a)
Native data plane for git-sync (plan §3, §8.1):
- provenance: widen actor to 'user'|'agent'|'git-sync' (jwt-payload,
auth-provenance decorator); PersistenceExtension resolves lastUpdatedSource
with precedence agent > git-sync > user, debounced history (like a human edit,
not the agent's immediate snapshot).
- GitmostDataSourceService implements @docmost/git-sync's GitSyncClient natively:
reads via PageRepo/SpaceRepo (listSpaceTree complete:true, getPageJson), writes
via PageService (create/removePage soft-delete/movePage with computed fractional
position/update-rename/restore) + the writeBody linchpin through collab
openDirectConnection('page.'+id, {actor:'git-sync'}) mirroring
collaboration.handler withYdocConnection 'replace'. bind({workspaceId,userId})
returns the context-bound client for the orchestrator.
- 10 unit/contract tests (mapping + soft-delete + move-position), tsc clean.
Known gap (closed in A.4b): PageService.create/update/movePage only branch on
actor==='agent'; git-sync provenance is already passed through so the row source
marker propagates once PageService honors 'git-sync'. Module/orchestrator/config
come next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|