feat(git-sync): двусторонний Docmost↔git синк — тестовая фаза на каноническом конвертере (продолжение #119) #359
Open
agent_vscode
wants to merge 111 commits from
feat/git-sync-2 into develop
pull from: feat/git-sync-2
merge into: vvzvlad:develop
vvzvlad:main
vvzvlad:test/351-generative-converter
vvzvlad:feat/371-roles-catalog
vvzvlad:feat/370-page-versioning
vvzvlad:refactor/345-server-converter
vvzvlad:feat/196-multi-cursor
vvzvlad:refactor/294-spec-registry-cont
vvzvlad:fix/363-migration-order
vvzvlad:perf/348-backend-lowhanging
vvzvlad:fix/362-metrics-route-cardinality
vvzvlad:fix/ai-sdk-partial-output-oom
vvzvlad:perf/344-background-rerenders
vvzvlad:develop
vvzvlad:perf/342-code-splitting
vvzvlad:feat/355-perf-metrics
vvzvlad:perf/346-compression-cache
vvzvlad:perf/343-typing-latency
vvzvlad:fix/e2e-callout-and-gate-build
vvzvlad:fix/docker-re2-toolchain
vvzvlad:feat/git-sync
vvzvlad:fix/media-roundtrip-stability
vvzvlad:fix/340-comment-panel-perf
vvzvlad:fix/332-deferred-tools
vvzvlad:fix/329-ephemeral-suggestions
vvzvlad:fix/330-search-in-page
vvzvlad:fix/328-resolved-anchor-spam
vvzvlad:fix/331-intraline-diff
vvzvlad:fix/324-coverage-gate
vvzvlad:fix/325-mobile-390
vvzvlad:feat/293-A-git-sync-package
vvzvlad:feat/300-avatar-oklch
vvzvlad:fix/321-banner-mobile
vvzvlad:feat/300-avatar-colors
vvzvlad:feat/315-comment-suggestions
vvzvlad:feat/scroll-restore-stable-wait
vvzvlad:feat/300-agent-avatar-stack
vvzvlad:feat/300-avatar-polish
vvzvlad:refactor/294-tool-spec-registry
vvzvlad:feat/scroll-restore-ux
vvzvlad:fix/responsive-tablet-sidebar
vvzvlad:feature/ai-chat-page-change-observability
vvzvlad:feature/offline-sync
vvzvlad:image-inline-center
vvzvlad:fix/283-short-remap-title
vvzvlad:fix/283-slash-layout
vvzvlad:image-inline-row
vvzvlad:feat/276-ai-chat-dock
vvzvlad:fix/269-table-menu-refocus
vvzvlad:docs/dev-stand-guide
vvzvlad:feat/266-scroll-position
vvzvlad:fix/260-collab-docname-slugid
vvzvlad:test/244-phase2-tail
vvzvlad:fix/262-reindex-progress-realtime
vvzvlad:fix/258-changelog-compare-links
vvzvlad:fix/244-dataloss-bugs
vvzvlad:feat/246-spoiler
vvzvlad:feat/221-image-captions
vvzvlad:test/244-part-b
vvzvlad:feat/251-intentional-clear
vvzvlad:fix/embeddings-reindex-progress
vvzvlad:refactor/193-tool-spec-registry
vvzvlad:fix/255-ws-redis-adapter-leak
vvzvlad:fix/252-e2e-open-handles
vvzvlad:feat/229-catalog-yaml
vvzvlad:feat/243-blob-sandbox
vvzvlad:feat/228-inline-footnotes
vvzvlad:fix/qa-ui-bugs-216-218
vvzvlad:feature/agent-roles-catalog
vvzvlad:fix/share-alias-rename
vvzvlad:fix/ai-chat-empty-render
vvzvlad:feat/191-chat-doc-binding
vvzvlad:feat/201-temporary-notes
vvzvlad:feat/198-interrupt-agent
vvzvlad:feat/ai-chat-full-history
vvzvlad:feat/199-ai-generate-title
vvzvlad:feat/205-share-aliases
vvzvlad:batch/issues-189-187-170
vvzvlad:feat/170-mcp-test-button
vvzvlad:feat/189-context-badge
vvzvlad:feat/198-interrupt-agent-send-now
vvzvlad:fix/issues-190-159
vvzvlad:fix/ai-chat-new-chat-during-stream
vvzvlad:fix/ai-chat-stream-perf
vvzvlad:batch/issues-2026-06-25
vvzvlad:feat/ai-chat-persistent-history
vvzvlad:fix/ai-chat-copy-chat-wysiwyg
vvzvlad:fix/ai-stream-reset-resilience
vvzvlad:fix/ai-stream-undici-timeout
vvzvlad:fix/footnote-review-1227-followup
vvzvlad:fix/ai-chat-token-counter-realtime
vvzvlad:docs/manual-qa-test-plan
No Reviewers
Labels
Clear labels
epic
needs-human
review/approved
review/changes-requested
review/needs
Large multi-phase effort spanning many changes
эскалация: нужно решение человека
в последнем ревью нет открытых blocking-находок
последнее ревью оставило открытые blocking-находки
head не ревьюился (head != reviewed_head)
No Label
review/approved
Milestone
No items
No Milestone
Projects
Clear projects
No project
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: vvzvlad/gitmost#359
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "feat/git-sync-2"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Что это
Продолжение и замена #119 (закрыт в пользу этого PR: там 69 комментариев и 14 раундов ревью — полная история по ссылке). Ветка
feat/git-sync-2= головаfeat/git-sync(e46e89b5, approved round 14) + свежий merge develop, которым уже поглощён #350.Статус вливания: ЗАМОРОЖЕНО (#326, шаг 6b). Этот PR — площадка тестирования синка до стабилизации; мерж — только по явной команде мейнтейнера.
Краткое содержание предыдущих серий
PageChangeListenerс loop-guard, git smart-HTTP хост (git clone/push http://…/git/<spaceId>.git, Basic-auth + CASL, 404-не-403), per-space opt-in тоггл, провенансgit-sync, admin-trigger.@docmost/prosemirror-markdown+ канонические форматы (решения №1–№9: inline-сноски^[…],<!--img {…}-->, Obsidian-callouts,$…$,==…==, комментарии-дискриминаторы медиа,<!--subpages-->); #350 — класс «""≡absent». С шага 6a (#326) ветка живёт на пакете, собственной копии конвертера у синка больше нет — диф PR похудел с 305 до ~119 файлов.Оговорки для тестирующих (важно)
alt: ""→nullдифф — ожидаем.pnpm install --frozen-lockfileв чистом чекауте ДО push, вывод прикладывается (класс «merge ломает lock» стрелял дважды).Известные хвосты движка (осознанно открыты, из финала #119)
D3-N2 (corrupt loose object → re-init), D2-D1 (fencing tokens / SIGKILL-лок до TTL), valid-but-nonexistent PARENT uuid wedge (4-й вариант id-класса), orphan .md при опустошении спейса (SPEC §8 — осознанный guard, решение мейнтейнера), инкрементальный pull (перф, «с бенчем, не вслепую»).
Верификация этой головы
pnpm install --frozen-lockfile→ Done (guardrail №6);@docmost/git-syncvitest → 268/268;git-sync-converter-gate.spec.ts→ 29/29 (на дереве с поглощённым #350).Голова сдвинулась относительно approved
e46e89b5(merge-коммит) — нужен review-round на текущую голову.Ссылки
#119 — полная история · #326 — план (6a ✅ / 6b заморожен) · канон: #293 (comment) · #345/#347 — editor-ext md-слой · #351 — порождающее тестирование
Closes #194
Make @docmost/git-sync natively consumable by the CommonJS server (and jest): build to CommonJS (tsconfig module CommonJS, drop type:module, strip .js from relative imports), and lazy-load the only ESM-only dep (marked) via the dynamic Function('import()') trick (mirrors docmost-client.loader.ts) with a require() fallback so vitest's evaluator works too. git-sync tests stay green (314 pass, 3 expected fail). Add the §13.1 idempotency gate (apps/server .../git-sync-converter-gate.spec.ts): 13 editor-ext docs (paragraphs/headings, marks, links, bullet/ordered/task lists, blockquote, callouts, code block, hr, table, nested mix) round-trip content(editor-ext) -> convertProseMirrorToMarkdown -> markdownToProseMirror -> TiptapTransformer.toYdoc/fromYdoc(tiptapExtensions) -> canonicalize and assert docsCanonicallyEqual. All green => the vendored converter's docmost-schema is schema-compatible with editor-ext (no node/mark/attr loss), which the plan §13.1 requires before Phase B. The one intrinsic markdown-image lossiness (width/height /align can't ride plain ) is isolated in a KNOWN DIVERGENCE block, not hidden. Server tsc clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>Native data plane for git-sync (plan §3, §8.1): - provenance: widen actor to 'user'|'agent'|'git-sync' (jwt-payload, auth-provenance decorator); PersistenceExtension resolves lastUpdatedSource with precedence agent > git-sync > user, debounced history (like a human edit, not the agent's immediate snapshot). - GitmostDataSourceService implements @docmost/git-sync's GitSyncClient natively: reads via PageRepo/SpaceRepo (listSpaceTree complete:true, getPageJson), writes via PageService (create/removePage soft-delete/movePage with computed fractional position/update-rename/restore) + the writeBody linchpin through collab openDirectConnection('page.'+id, {actor:'git-sync'}) mirroring collaboration.handler withYdocConnection 'replace'. bind({workspaceId,userId}) returns the context-bound client for the orchestrator. - 10 unit/contract tests (mapping + soft-delete + move-position), tsc clean. Known gap (closed in A.4b): PageService.create/update/movePage only branch on actor==='agent'; git-sync provenance is already passed through so the row source marker propagates once PageService honors 'git-sync'. Module/orchestrator/config come next. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>Comprehensive-review follow-ups (APPROVE WITH SUGGESTIONS; no critical issues): - poll interval is now actually configurable: replaced the hardcoded @Interval('git-sync-poll', 15000) with a dynamic SchedulerRegistry interval registered in onModuleInit from getGitSyncPollIntervalMs() (cleared in onModuleDestroy); /status and the real cadence now share one config source. Boots logging 'poll interval registered (Nms)'. - loop-guard now ALWAYS applies: the lastUpdatedSource==='git-sync' skip was nested inside the !spaceId/!workspaceId branch, so structural self-writes (CREATE/MOVE/RESTORE/SOFT_DELETE, which carry spaceId+workspaceId) bypassed it and re-triggered cycles. Fetch the page row once, guard unconditionally, then resolve space/workspace. - remove the dead PAGE_CONTENT_UPDATED subscription (it's a BullMQ job, never an EventEmitter event; body edits arrive via PAGE_UPDATED). - fix the stale datasource comment (PageService DOES stamp 'git-sync' now). - env getters: parseInt radix 10 + NaN/<=0 fallback for poll/debounce (+ max deletes), with 6 new environment.service.spec tests. tsc clean; jest 723 pass; live cycle re-verified post-refactor (ran, push applied, unflagged 92-page space untouched). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>The reconcile choreography (ensureRepo -> merge-check -> ensureBranch -> checkout('docmost') -> pull -> push) was hand-rolled in the app orchestrator's driveCycle, duplicating an order the vendored engine owns and could drift from on upgrade — the failure mode is data clobber. Lift it into @docmost/git-sync as a single entry point, `runCycle(deps)`. The orchestrator now calls runCycle and keeps only the lock (its caller) and the gitmost-specific delete-cap POLICY, injected as the `resolveApplyClient` hook (the engine does the dry-run, hands the hook the planned delete count — Infinity if planning failed — and uses whatever client it returns for the apply). driveCycle drops from ~150 lines to ~30. Tests: - engine test/cycle.test.ts: composition (merge-in-progress short-circuit; ensureRepo->ensureBranch->checkout staging order before the pull; the cap hook is consulted with the planned count; no dry-run when no hook). - engine test/cycle-roundtrip.test.ts: runCycle against a REAL VaultGit in a temp repo with a faked Docmost client — a git-originated CREATE flows pull->push and the assigned pageId is written back; an unresolved merge short-circuits before any client call. - orchestrator spec rewired to mock runCycle and assert the wiring + the resolveApplyClient cap policy (the engine-internal cycle-order/merge tests moved to the engine). Validated end to end on a live stand (real Postgres/Redis + server): a git clone -> edit -> push over the /git remote round-trips the change into the Docmost page through the refactored cycle. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>subpages exported to the literal `{{SUBPAGES}}`, which has no markdown/HTML inverse, so on re-import it came back as a plain paragraph holding the visible text "{{SUBPAGES}}" — the embed rendered as that literal string on the page after a sync (round-trip data loss, seen live). It now emits the schema-matching `<div data-type="subpages">` like every other embed node, so the schema's parseHTML rebuilds the subpages node. Also dropped the leaf-atom content-hole in the subpages renderHTML. New committed regression coverage: - packages/git-sync/test/roundtrip-all-nodes.test.ts — exhaustive serialize -> deserialize round trip for ALL 40 node/mark types; each asserts the node/mark survives and no `{{...}}` literal leaks. This is the test that caught subpages. - §13.1 gate (git-sync-converter-gate.spec.ts): subpages added to the green corpus (round-trips through the REAL server schema). - Corrected two PR-authored tests that asserted the old {{SUBPAGES}} loss as "by design" — they now assert the fixed round trip. Also folds in review #1679 coverage-gap tests (no prod change): orchestrator pollTick/enabledSpaces, datasource 3-way merge dispatch, page.repo last_updated_source provenance SQL. git-sync vitest 659 (+1 expected-fail), server tsc clean, server specs green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>> [!type]) instead of:::typeb5ce63a956When a git-sync body write (gitSyncWriteBody) is routed to the collab instance that owns the doc, the handler runs remotely inside handleRedisMessage and CAN throw (markdown->ProseMirror transform). Previously the throw was uncaught: the customEventComplete reply was never published, so the origin's writePageBody promise only rejected after customEventTTL (~30s) as a generic 'TIMEOUT', and an unhandledRejection escaped the async messageBuffer listener on the owning instance. Now the owner wraps handleEventLocally in try/catch and, on throw, publishes a customEventComplete carrying an `error` field on the same correlation channel. The origin's pendingReplies holds {resolve, reject} and rejects promptly with the real Error. The TTL TIMEOUT remains as the fallback for a genuinely lost reply. The no-throw and local (same-instance) paths are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>open; drop dead delete-cap wiring; cover lost-lock abort + lose-prone atom round-trips e48d7720e9openas a boolean so open state survives render/round-trip 32e99c6e42Security (must-fix): - /git smart-HTTP gate: an authenticated NON-member of a git-sync space now gets 404 (not 403), so the 403<->404 difference can no longer be used to brute-force which spaces exist / have git-sync enabled. 403 is reserved for a MEMBER who lacks the required role (existence already known). New gate input userIsSpaceMember; decision-table + service specs extended. Config (must-fix): - Remove the dead GIT_SYNC_SSH_KEY_PATH knob (getter + validation field + two .env.example lines) — it had zero consumers and advertised a nonexistent push capability. Stability/docs (warnings): - Wire the lost-lock AbortSignal into runReceivePack -> git http-backend so the receive-pack child is killed if the per-space lock lapses mid-write. - Raise the divergent-`docmost` (invariant §5) push refusal from info -> warn and surface divergentDocmost in the run status (/status). - Comment the stale read-after-debounced-collab-write updatedAt in importPageMarkdown (deferred §10 loop-guard must not trust it). - Fix the Dockerfile comment: the loader uses require.resolve + dynamic import(), it deliberately does NOT require('@docmost/git-sync'). - Merge the two near-identical space toggle handlers into one parameterized handler; add the 2 missing en-US i18n keys for the auto-merge switch (ru-RU not maintained for these git-sync strings, mirrored). Tests: - isGitSyncHttpEnabled() default-branch (unset -> isGitSyncEnabled fallback). - agentSourceFields 'git-sync' case (source stamped, chat key omitted). - editor-ext name-level schema contract (vendored mirror superset of editor-ext node/mark types) + the new shared resolver + non-member 404 gate cases. Architecture: - Extract resolveRequestWorkspace shared by DomainMiddleware + GitHttpService (the two real self-hosted/cloud copies; McpService has no cloud branch). - Document the in-process setInterval multi-replica limitation + BullMQ/fencing future direction (deferred, not implemented). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>Must-fix: - Throttle the raw /git HTTP-Basic path: it bypasses Nest/ThrottlerGuard, so verifyUserCredentials (bcrypt) ran unthrottled. Wrap it in the SAME FailedLoginLimiter the /mcp path uses (5/60s; per-IP, per-IP+email, global per-email keys; atomic tryReserve BEFORE bcrypt; success resets, non-credential errors release). The (threshold+1)-th attempt now gets 429 pre-bcrypt. Sweep timer + onModuleDestroy mirror McpService. - Fix the mcp schema mirror drift: packages/mcp details `open` attr now reads via hasAttribute (matches editor-ext canon + git-sync copy); getAttribute dropped a bare `<details open>` state. (build/ is gitignored — rebuilt locally.) Tests added: - /git brute-force throttle: pre-bcrypt 429 on the 6th failure; success resets; non-credential error releases the budget. - git-http-backend lost-lock AbortSignal: already-aborted -> no spawn + 500; live abort mid-request -> SIGTERM + response closed. - orchestrator divergentDocmost -> WARN + flag surfaced in status (+ clean case). - pollTick re-entrancy guard skips an overlapping tick. - datasource NotFound early-throws (getPageJson/move/rename) + updatedAt:undefined stale-read branch (importPageMarkdown/createPage). Suggestions: - space.repo updateGitSyncSettings: parameterize the jsonb key (`${prefKey}::text`) instead of sql.raw (latent-injection footgun); value stays sql.lit. Spec updated. - pollTick re-entrancy guard (private `polling` flag). - page-change.listener docstring: honest about the move/rename/delete over-skip (loop-guard keys only on lastUpdatedSource) -> ~poll-interval latency, not loss. - AGENTS.md: document the root /git smart-HTTP route + GitSyncModule. - Remove redundant redteam-provenance.spec.ts (covered e2e in persistence.extension.spec.ts:145). - Extract the duplicated SIGTERM->SIGKILL+finish block (watchdog + abort) into terminateChild; centralize watchdog-timer teardown in done(). Architecture (deferred, documented): mcp schema header now carries the three-copy keep-in-sync + schema-core note; the editor-ext contract test documents that the mcp copy and attribute-behaviour drift (details `open`) are not mechanically covered yet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1. gitRemote is NOT yet consumed (the vendored engine has no remote-push path, SPEC §7). Corrected the buildSettings docstring (it wrongly called gitRemote "load-bearing") and marked the env -> validation -> getter -> buildSettings chain as inert SCAFFOLDING for the deferred remote-push feature at all three sites. Kept the wiring (harmless; removing only churns). 2. .env.example: document that GIT_SYNC_REMOTE_TEMPLATE substitutes the literal "{spaceId}" per-space (with the example), so an operator doesn't point every space at one remote. 3. Extracted the copy-pasted CJS->ESM dynamic-import bridge (`new Function('s','return import(s)')`) into one shared common/helpers/esm-import.ts; git-sync.loader, docmost-client.loader and mcp.service now import it and keep their own typed loadX() wrappers. Deferred (notes only, not implemented): - lcs.ts + three-way-merge.ts could move into packages/git-sync, but that engine is vendored (manual re-sync) — added a one-line note at three-way-merge.ts to revisit once the re-sync story is settled. - schema-core single source + BullMQ/fencing remain documented from prior rounds. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>F4: a rename/move + body edit in one diff used to lose the edit (renamed pages went only into renamesMoves, never updates). Now computePushActions also emits an updates entry for renames, AND threads the OLD path via a new UpdateAction.basePath so applyPushActions resolves the 3-way merge base from the pre-rename file. Without it the base lookup at the new path returns null and degrades to a 2-way merge that rolls back concurrent Docmost edits; with it the edited block wins while a concurrent edit to another block survives. A plain (status M) update carries no basePath and is byte-identical to before. F5: test the CREATE path stripping conflict markers (autoMergeConflicts on). F6: .env.example documents GIT_SYNC_REMOTE_TEMPLATE as deferred/inert scaffolding. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>F8: extract emitMoveWithBody helper (renamesMoves + body update with basePath=oldPath) and call it at all three rename emission sites (ghost-move A, ghost-move M, R/C) — byte-identical behavior, single F4 rationale. Helper placed above computePushActions so the planner JSDoc stays attached. F7: add an M-side ghost-move test (D+M same pageId) asserting the move and the body update carry basePath=oldPath — the previously-untested branch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>F1: git-sync.orchestrator.spec bind assertion now includes spaceId ('space-1'), matching driveCycle's dataSource.bind({workspaceId,userId,spaceId}). F2: add 4 non-vacuous tests for the cross-space move data-loss guard in deletePage (CTX_SPACE with spaceId): move-out skips removePage (returns skipped:'moved-to-other-space'); same-space / not-found / already-deleted all still call removePage. F3: add 2 tests for the ~<slugId> title-strip guard in renamePage (own slugId stripped; a foreign ~<slugId> tail left intact). F4: reword the gitmost-datasource 'single choke point' comment — the strip covers the rename/update path, not every git-sync title write (createPage's filename-derived title does not funnel through here). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>On a git-sync space, a page's web edit was silently reverted within ~1 poll, and idle spaces showed dozens of 'update' actions per cycle with no real change. Root cause: the vault->Docmost body ingest (importPageMarkdown) is re-run every poll for pages the upstream change-detection mis-flags as changed (the markdown<->ProseMirror round-trip is not byte-stable: JSON key order / default attrs differ though the content is identical). Each call re-imports the SAME body into the live collab doc -- a no-op at idle, but it CLOBBERS a concurrent human edit still in the debounced (not-yet-flushed) Yjs doc. Fix: skip the ingest when it is genuinely a no-op -- 1) baseMarkdown byte-identical to the current file (vault unchanged), or 2) the parsed incoming body is canonically-JSON-equal (key-order-insensitive) to the page's current Docmost content. A real git-side change is neither, so legitimate git->Docmost ingests still apply. Verified: idle churn 38 update/cycle -> 0; web edit on an affected page 0/3 -> 3/3 persisted; genuine git-side edit still ingests. Found by autonomous QA (web-test-orchestrator) + independent verifier. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>5d45f5a8) 67dca8c10eThe N1-D1 fix added an early `return {}` when `currentPage == null` in importPageMarkdown. `currentPage` is a const, never reassigned, so from that guard onward it is provably non-null — which made the cross-space (S2) gate's comment false ("a not-found page still proceeds as before": a not-found page now returns early above) and left dead null-handling around it. - Rewrite the S2-gate comment: the null case is handled by the N1-D1 guard above; here currentPage is guaranteed non-null. Confused-deputy / cross-space / mirror-deletePage explanation kept intact. - Drop the dead `currentPage &&` conjunct from the S2 condition (always true). - Collapse downstream vestigial `currentPage?.` / `currentPage!` / the `currentPage ? … : undefined` ternary to plain `currentPage.` — all behavior-preserving (currentPage non-null after the guard). The unrelated `page ? … : undefined` ternary (fresh findById that can be null) is untouched. No runtime behavior change. jest gitmost-datasource.service.spec.ts: 34 passed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>An interrupted git operation (a hard crash / OOM-kill / abrupt container stop mid `git add`/`commit`/`checkout`) leaves a `.git/index.lock` (or a ref `*.lock`). Git then refuses EVERY subsequent operation ("Unable to create '…/index.lock': File exists"), so every poll cycle failed and the space's sync wedged INDEFINITELY with no self-heal — the whole space stopped syncing until a human ran `rm` on the lock (found via web-test restart/corruption charter, reproduced deterministically). The daemon holds the per-space Redis lock and is the vault's ONLY writer, so any `*.lock` reaching a fresh cycle is necessarily stale (no live git process holds it). Add `VaultGit.clearStaleGitLocks()` and call it in the cycle preflight, right after ensureRepo and before the mid-merge recovery — clearing index/HEAD/config/packed-refs/ MERGE_HEAD/ORIG_HEAD and the engine's ref locks (best-effort, missing = no-op). Verified on the stand: a planted stale index.lock is now cleared and the space recovers (edit reaches the vault, 0 "File exists" errors) — was wedged forever. Unit test (real temp repo: index.lock blocks git add -> clear -> git add works); git-sync suite green (707). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>Ref-store damage (a deleted refs/heads/main, an interrupted ref update) can leave an existing vault repo without a 'main' branch. The cycle's ensureBranch('docmost','main') + checkout then throw every poll ("pathspec 'main' did not match"), wedging the space forever with no self-heal — ensureRepo only creates branches on a FRESH git init (found via web-test corruption charter, reproduced deterministically). Add VaultGit.ensureMainBranch() and call it in the cycle preflight (after clearStaleGitLocks, before the branch setup): if 'main' is missing, re-create it from the 'docmost' mirror branch (they track each other) else from HEAD. Same wedge-forever family as D3-N3. Verified on the stand: deleting refs/heads/main now self-heals (main restored, the edit reaches the vault, 0 pathspec errors) — was wedged forever. Unit test (real temp repo: delete main -> ensureMainBranch restores it from docmost). git-sync suite green (708). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>clearStaleGitLocks() removed a hardcoded list of 9 git lock files UNCONDITIONALLY — despite its name it never checked staleness. In a multi-replica TTL-lapse window (a documented engine limitation), replica B's preflight could rm a LIVE index.lock held by replica A mid `git add`/`commit`, turning a safe fail-fast ("index.lock: File exists") into concurrent index/ref writes and corruption. F1: gate each removal by file age. A live git op is bounded by GIT_EXEC_TIMEOUT_MS (120s), after which git is killed and the lock's mtime freezes — so a lock older than 3× that (STALE_LOCK_MIN_AGE_MS = 360s) provably has no live holder and is a genuine crash-leftover. Fresh locks (mtime within the window) are preserved; missing locks are a no-op. Strictly safer than the prior unconditional rm (the gate can only prevent removals, never add them). F4: move clearStaleGitLocks (doc + body) below isMergeInProgress so the mid-merge jsdoc re-attaches to its real owner; soften the doc (it clears a fixed list of the engine's own index/ref locks, not "any *.lock"). F2: cycle.test.ts pins the preflight order ensureRepo < clearStaleGitLocks < ensureMainBranch < ensureBranch (a refactor that moved clearStaleGitLocks before ensureRepo — deleting ensureRepo's own transient lock — would now fail the test). F3: git.test.ts covers ensureMainBranch's HEAD-fallback branch (both main and docmost gone → main recreated from HEAD) and the no-commit no-op. Also: the D3-N3 test now backdates the planted lock's mtime so it is stale (and still removed), plus a new test asserts a FRESH lock is PRESERVED (the corruption-safety property — this would fail against the old code). cycle.ts's preflight comment softened to match the mtime gating. git-sync vitest: 711 passed | 1 expected-fail (+4 new tests). tsc clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>Ревью — #359 (двусторонний git-sync, тестовая фаза, продолжение #119), голова
69565639. Вердикт: PASS ✅Merge-verify: голова = одобренный round-14 #119 (
e46e89b5) ⊕ свежий merge develop (5336f06d, поглощает #350). Сверил, что одобренный контент сохранён и develop вобран чисто.Сверено (по коду):
e46e89b5(approved #119) +5336f06d(develop tip) — чистый merge-коммит, не squash; approved-голова — предок (контент сохранён).@docmost/git-syncна месте в importer'еapps/serverлока,pnpm install --frozen-lockfile→ EXIT 0. Домёрж develop на этот раз лок не сломал.packages/prosemirror-markdown/src== develop (пустой дифф) — #350 (стабилизация «пустая строка ≡ отсутствие») вобран корректно.Объективка зелёная (мой прогон, CI-условия, чистое дерево, реальные сборки): frozen install 0; сборка editor-ext/prosemirror-markdown/git-sync/mcp 0; пакетные тесты — prosemirror-markdown 682 (вкл. #350), git-sync 268, mcp node --test 480/0; server tsc 0; адаптированные git-sync-спеки (converter-gate + schema-attribute-contract) 32/32. Одобренный git-sync ⊕ develop-#350 даёт зелёное дерево.
ℹ️ Инфо (не блокер)
mergeable: FALSE— но это против таргет-веткиf665f6fd(не develop; голова #359 УЖЕ содержит develop как родителя). PR ЗАМОРОЖЕН (#326 шаг 6b — площадка тестирования синка, мерж только по явной команде мейнтейнера), поэтому конфликт со stale-таргетом не гейтит PASS. PASS = «одобренный контент + чистый develop-merge, дерево CI-зелёное», а не «вливать сейчас».🐛 BUG (engine → this PR) — silent data loss: a freshly git-added
.mdwith an unknowngitmost_idis DELETED from the vault + remoteRoute: engine —
@docmost/git-syncpush classifier / reconcile (not the converter package).Severity: HIGH (silent loss of user-authored git content; no failure, no wedge).
Confidence: high, deterministic ≥2× (valid-but-nonexistent UUID, and a non-UUID token).
Repro (RU)
WT-ghost.mdс фронтматтеромgitmost_id: 019f2500-dead-7000-8000-000000000009(корректный по формату UUID, но НЕ соответствует ни одной странице; либо не-UUID токен[unclosed) и телом.git push origin HEAD:main. Подождать ~2 цикла (~16с).git fetch— файл исчез: движок его удалил и запушил удаление.Expected
Файл с неизвестным/несуществующим
gitmost_id— это НЕ страница, удалённая в Docmost (её никогда не было). Его нельзя молча удалять. Ожидается: пропустить (skip + лог) или адоптировать как новую страницу — контент git-автора сохраняется.Actual
Импорт корректно скипается (guard «no page with that id» — регрессия #119 НЕ произошла, спейс не виснет ✅). Но push-классификатор видит файл с id, которого нет среди live-страниц Docmost, и классифицирует его как delete → удаляет файл из vault и пушит удаление в remote. Контент потерян молча.
Root cause
Push absence-delete не различает: (a) файл, id которого БЫЛ в
refs/docmost/last-pushed(реально синхронизированная страница, удалённая в Docmost → удалять корректно) vs (b) СВЕЖЕ добавленный файл с мусорным/чужим id, которого в last-pushed никогда не было (→ удалять нельзя, это ADD). Классификатор удаляет по факту «id есть в vault, нет в live», без проверки, что id ранее синхронизировался.Suggested fix
Absence-delete должен срабатывать только для id, присутствовавшего в last-pushed (ранее синхронизированного). Файл, чей id впервые появляется в рабочем дереве (нет в last-pushed) и не матчит live-страницу → skip/adopt, не delete. (Это добивает 4-й вариант id-класса из хвостов #119, но со стороны own id и с симптомом «тихое удаление файла», а не wedge.)
Найдено полным веб-прогоном (charter P3). Проверено детерминированно на чистом сиде.
🐛 BUGS (converter →
@docmost/prosemirror-markdown, fixtures-first) — round-trip fidelity gaps found in the canonical converterRoute per guardrail #2: these are in the shared converter package (md↔pm), not the branch. Found via full web-test (charter P6), each verified deterministically by calling the built converter directly (
markdownToProseMirror/convertProseMirrorToMarkdown).P6-B [MED] — inline HTML comment in running prose is SWALLOWED (content deleted, adjacent text joined)
The converter uses HTML comments as canonical data carriers (
<!--attrs-->,<!--img-->,<!--subpages-->), so a literal comment is silently erased and the tokens on either side are merged with no boundary. For a vault whose value prop is lossless round-trip, a literal comment should survive (or at least not merge adjacent content). Deterministic 3×.P6-A [MED] — Obsidian titled callout
> [!info] Titledrops the title on import> [!type] titleis valid Obsidian syntax; the title text is silently lost (Docmost's callout node has no title slot, canonical emit is bare> [!info]). Since the feature advertises Obsidian-native callouts, dropping the title is data loss for an Obsidian author. Suggest folding the title into the callout body (e.g. a leading bold paragraph) rather than discarding. Systematic, 6+×.P6-H [LOW] — highlight
==a == b==mis-tokenized, non-round-trip==x = y = z==(equals not adjacent to==) is fine. Ambiguous adjacent-==authoring, but it changes content on round trip. LOW. 2×.P6-img [LOW / likely unsupported] — standalone
<!--img {…}-->block (no![]()prefix) dropped entirelyThe engine only ever emits
<!--img-->as a suffix to an![]()image, so a bare hand-authored<!--img-->isn't a produced canon form. Reporting for completeness; probably wontfix.Not bugs (disconfirmed, for the record): code-block literal markers stay literal ✅; in-body
>[!warning]-looking text does NOT false-trigger a callout ✅; unicode/emoji/CJK/RTL/8k-line intact ✅; the converter is a fixpoint on its own canonical output (idempotent, churn=0) for every construct ✅ — these gaps are all first-pass PARSE loss on hand-authored input, not oscillation.I'll mirror local fixtures-first fixes for the clear ones (P6-B, P6-A) in my working copy to keep the test loop converging; final fix belongs in a package PR on develop per the routing rule.
D-P3-1 follow-up — fix analysis (I explored two local fixes; BOTH have edge cases → the correct fix needs a page-existence check)
I tried to fix this locally (my working copy only, not pushed) and want to save the maintainer time by documenting what does and doesn't work. The root is the pull's absence-reconcile (
planReconciliationinengine/reconcile.ts): a tracked file whose pageId is absent fromliveis deleted. "Absent from live" has FOUR causes — only one should be preserved:Attempt 1 — gate on "id was in
refs/docmost/last-pushed" (previously synced): genuine deletes work, ghost survives ONE cycle — but once the ghost file is committed to the vault it enters last-pushed, so the next cycle sees it as "previously synced" and deletes it. ❌ (ghost-pollution)Attempt 2 — gate on "id is in Trash" (
listTrash): ghost never enters Trash, so it's correctly preserved forever; genuine trash-deletes work. BUT a cross-space move leaves the moved page live in space B (not in A's Trash) → A's source file is no longer absence-deleted → orphan.mdin A. ❌ (move-cleanup regression, same class as the known orphan-on-empty tail)Conclusion — the correct signal is "does this pageId correspond to ANY real page row (any space, any state)?": ghost → no row → keep; deleted/moved/trashed → row exists → delete the source file. This needs a bulk existence check (e.g. a new client method
pageIdsExist(ids) → Set=SELECT id FROM pages WHERE id = ANY($ids)incl. deleted, any space), gating only the small candidate-delete set. That's a clean but non-trivial change in the data-loss-critical reconcile path, in the same id-class family as the already-tracked open tails — so I'm leaving it to the maintainer rather than shipping a half-correct guard. Both of my attempts are reverted; the bug report above stands. Happy to implementpageIdsExist+ the gate if you want it.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.