Compare commits

..

76 Commits

Author SHA1 Message Date
claude code agent 227
fbaaa84419 test(git-sync): accurate null-edge docstring + fill round placeholder (F2/F3)
F2: the real-git modify/delete null-edge test docstring overclaimed it
caught loss of the `?? theirs` fallback end-to-end. git itself leaves
theirs in the working tree (stage 3) so commitMerge's `git add -A` would
stage it even with the bug — the assertions pass on broken logic. Reword
to state it verifies the clean-merge happy path; the real F1 regression
guard lives in the fake-fs apply-pull-actions.test.ts.

F3: fill the `round-?` placeholder with `round-2` in both new blocks to
match the file convention (header: 'QA #119 round-2').

Comment-only; no production or test-logic changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 02:04:47 +03:00
claude code agent 227
32cb9eb1e3 test(git-sync): cover null-edge conflict resolution in applyPullActions (F1)
The genuine-conflict branch in applyPullActions resolves to `ours ?? theirs`,
but the two stages where a side is ABSENT had NO test — the existing conflict
tests only fed stages where both ours and theirs are non-null. This is the
data-preservation core on the published `main`: a regression (dropping the
`?? theirs`, or wrongly writing on both-null) would silently lose a surviving
Docmost edit or resurrect a both-deleted page.

Adds four tests:
- apply-pull-actions.test.ts (fake-git, controlled stages): modify/delete
  (ours=null, theirs!=null -> keep THEIRS) and delete/delete (both null ->
  write nothing, deletion staged by commitMerge's `git add -A`).
- pull-conflict-normalize.test.ts (real-git 3-way): modify/delete built by
  deleting on main + modifying on docmost (stage 2 absent -> theirs kept,
  committed clean, no markers); delete/delete built via a rename/rename(1to2)
  on the shared base file, which records the original path as both-deleted
  (stages 2 AND 3 absent -> nothing written, deletion committed off main).

Production logic at pull.ts:487-497 held — pure test-coverage fix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 00:23:55 +03:00
claude code agent 227
b47751349f fix(git-sync): kill spurious marker-leaking conflict, concurrent-edit loss, flapping HEAD
Three more git-sync QA defects from the 2nd live pass on PR #119, plus a
callout-fidelity nit:

1. SPURIOUS conflict leaked raw markers into canonical main (root cause). On an
   ordinary round-trip the only difference between the docmost mirror (normalize-
   on-write) and a user's raw push is trailing/empty-line normalization, which made
   git's line-based docmost->main merge CONFLICT, and the wedge fix then committed
   the file WITH literal <<<<<<< / ======= / >>>>>>> markers onto main (git and the
   DB silently diverged for cycles). Fix: on a conflict, normalize trailing/empty
   lines on BOTH sides (showStage :2:/:3:) before comparing — a trailing-only diff
   is recognized as spurious and resolved to the clean normalized form. A GENUINE
   same-block conflict is auto-resolved to OURS (git wins, mirroring the live-doc
   3-way rule); the docmost side stays on the `docmost` branch + page history. Raw
   markers NEVER reach main again.

2. Concurrent UI<->git edit silently lost the UI side. The git->Docmost 3-way merge
   ran against a live Y.Doc that hadn't yet received the user's debounced in-flight
   edit, so git clean-applied (no conflict detected) and the edit vanished even on a
   different block. Fix: flush the pending debounced store before the merge so the
   in-flight edit is drained into the live doc first — a different-block edit is
   merged, a same-block one is detected and pinned to history (recoverable).

3. Smart-HTTP HEAD flapped to the read-only `docmost` mirror (~1/4 of clones). The
   engine transiently checks out `docmost` mid-pull and the host advertises whatever
   HEAD resolves to. Fix: VaultGit.pinHeadToMain(); the cycle restores HEAD->main in
   a finally; and the upload-pack ref advertisement is served HEAD-pinned under the
   per-space lock so it can never observe a mid-cycle HEAD.

4. (callout) clampCalloutType now mirrors the editor's GITHUB_ALERT_TYPE_MAP for
   non-schema aliases (tip->success, caution->danger, important->info) instead of
   flatly collapsing to info. The editor schema genuinely supports only the six
   banner types, so unknown types still fall back to info (by design).

Tests: deterministic real-git trailing-blank round-trip (no conflict, no markers,
in sync over 2 cycles) + genuine-conflict no-marker-leak; HEAD advertisement
stability; pre/post-flush concurrent-edit survival; serveReadAdvertisement lock
pin; widened callout-alias coverage. Engine vitest + server tsc + collaboration /
git-http / orchestrator specs all green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 22:05:32 +03:00
claude code agent 227
b7e5cb6970 fix(git-sync): push 503 starvation + concurrent-edit marker leak/silent loss
Bug #1 (push 503 starvation): an external receive-pack that briefly overlapped
a poll cycle immediately 503'd because the per-space single-writer lock was
held. Add a BOUNDED retry-acquire on the PUSH path only (SpaceLockService
.withSpaceLock acquireRetry: capped exponential backoff up to ~5s); a transient
overlap now waits and succeeds, a genuinely stuck cycle still 503s after the
bound. The poll cycle passes no retry (immediate skip). Push result stays
deterministic: the receive-pack only runs once the lock is held, so a 503 never
leaves a half-applied ref.

Bug #2 (concurrent-edit marker leak + silent same-block loss):
- Marker leak (a): the push UPDATE path stripped markers for the body sent to
  Docmost but left raw <<<<<<</>>>>>>> committed on the published `main` vault
  forever (autoMergeConflicts ON). Now the cleaned body is written back to the
  vault file + recorded in writtenBack so runPush commits it on `main` and the
  vault converges to clean bytes.
- Marker leak (b): pin merge.conflictStyle=merge in ensureRepo and teach
  stripConflictMarkers/hasConflictMarkers about the diff3 `|||||||` base section
  (drop the marker AND the stale base region) so diff3/zdiff3 conflicts can
  never leak `|||||||` + base content into a page. Also scrub the 3-way merge
  BASE markdown.
- Silent same-block loss: the block 3-way merge still resolves same-block
  conflicts deterministically to git, but it is no longer silent: diff3Plan now
  reports a conflict count (mergeXmlFragments3WayWithStats), gitSyncWriteBody
  logs it, and the persistence boundary-snapshot now fires for git-sync writes
  over a non-git-sync baseline so the human's pre-merge content is preserved in
  page history (recoverable). Full both-preserved persisted-conflict UI remains
  the deferred redesign.

Tests: space-lock bounded-retry (success/stuck/poll-immediate); push vault-clean
+ diff3 |||||||  strip; ensureRepo conflictStyle pin; diff3Plan/3-way conflict
counts; persistence git-sync boundary snapshot. Server tsc clean; git-sync
vitest + server collaboration/git-sync jest all green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 20:03:21 +03:00
claude code agent 227
906733b5c8 fix(git-sync): address PR #119 review #4 — symlink guard, dead-code cull, changelog + warnings/suggestions
Blocking (review id 2514):
- [security] Forbid symlinks in vaults. ensureServable now sets
  core.symlinks=false in each vault's local git config (a pushed symlink is
  checked out as a plain file, never a real link), and the engine cycle wraps
  every read/write/mkdir in an lstat/realpath guard (new path-guard.ts) that
  refuses a path that is — or traverses — a symlink, or whose realpath escapes
  the vault root. Prevents a writer from publishing /etc/passwd or the server
  .env, or writing outside the vault. Adds unit tests (path-guard.test.ts) +
  a read-guard integration test (cycle.test.ts) + real lstat/realpath in the
  roundtrip integration test.
- [simplification] Delete dead lib/diff.ts + test/diff.test.ts and drop the
  now-unused @fellow/prosemirror-recreate-transform dependency.
- [documentation] Add a CHANGELOG [Unreleased] → Added entry for git-sync.

Warnings:
- [test-coverage] Cover the CREATE-branch conflict-markers guard (a new .md with
  markers and no gitmost_id is recorded as a create failure, never created).

Suggestions:
- [stability] Bound each `git config` in ensureServable with a timeout.
- [authz] Trigger endpoint resolves spaceId workspace-scoped and 404s a foreign
  space before any vault directory is created.
- [stability] Attribute git-initiated moves to the service account
  (lastUpdatedById), via an optional actor param on PageService.movePage.
- [documentation] Document the per-space autoMergeConflicts toggle in AGENTS.md.
- [test-coverage] Cover the unterminated `:::` callout fence fallback.
- [simplification] Move test-only roundtrip-helpers.ts out of src/ into test/.

Architecture:
- Move the Yjs/ProseMirror merge primitives (yjs-body-merge, three-way-merge,
  lcs + specs) into collaboration/merge/, breaking the collaboration →
  integrations/git-sync dependency cycle this PR introduced.
- Port the schema-surface drift gate to packages/mcp (the mcp schema mirror had
  none); pins 52 entries.

Deferred (with rationale in the review thread): the incremental-pull perf
warning (correctness-neutral; needs a high-water-mark design + its own tests on
the data-loss-critical path) and the redis-sync rolling-deploy mixed-version
edge (the deficient behavior is in already-released old-instance code; the new
code is correct on both sides; impact is a transient rollout-window artifact).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:39:12 +03:00
a
f020739bfd refactor(git-sync): address PR #119 review #3 — honest gitRemote scaffolding comments, env example, shared ESM bridge
1. gitRemote is NOT yet consumed (the vendored engine has no remote-push path,
   SPEC §7). Corrected the buildSettings docstring (it wrongly called gitRemote
   "load-bearing") and marked the env -> validation -> getter -> buildSettings
   chain as inert SCAFFOLDING for the deferred remote-push feature at all three
   sites. Kept the wiring (harmless; removing only churns).

2. .env.example: document that GIT_SYNC_REMOTE_TEMPLATE substitutes the literal
   "{spaceId}" per-space (with the example), so an operator doesn't point every
   space at one remote.

3. Extracted the copy-pasted CJS->ESM dynamic-import bridge
   (`new Function('s','return import(s)')`) into one shared
   common/helpers/esm-import.ts; git-sync.loader, docmost-client.loader and
   mcp.service now import it and keep their own typed loadX() wrappers.

Deferred (notes only, not implemented):
- lcs.ts + three-way-merge.ts could move into packages/git-sync, but that engine
  is vendored (manual re-sync) — added a one-line note at three-way-merge.ts to
  revisit once the re-sync story is settled.
- schema-core single source + BullMQ/fencing remain documented from prior rounds.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
a
22e3fcdeba fix(git-sync): address PR #119 review #2 — throttle /git Basic auth, fix mcp schema drift + warnings/tests
Must-fix:
- Throttle the raw /git HTTP-Basic path: it bypasses Nest/ThrottlerGuard, so
  verifyUserCredentials (bcrypt) ran unthrottled. Wrap it in the SAME
  FailedLoginLimiter the /mcp path uses (5/60s; per-IP, per-IP+email, global
  per-email keys; atomic tryReserve BEFORE bcrypt; success resets, non-credential
  errors release). The (threshold+1)-th attempt now gets 429 pre-bcrypt. Sweep
  timer + onModuleDestroy mirror McpService.
- Fix the mcp schema mirror drift: packages/mcp details `open` attr now reads via
  hasAttribute (matches editor-ext canon + git-sync copy); getAttribute dropped a
  bare `<details open>` state. (build/ is gitignored — rebuilt locally.)

Tests added:
- /git brute-force throttle: pre-bcrypt 429 on the 6th failure; success resets;
  non-credential error releases the budget.
- git-http-backend lost-lock AbortSignal: already-aborted -> no spawn + 500;
  live abort mid-request -> SIGTERM + response closed.
- orchestrator divergentDocmost -> WARN + flag surfaced in status (+ clean case).
- pollTick re-entrancy guard skips an overlapping tick.
- datasource NotFound early-throws (getPageJson/move/rename) + updatedAt:undefined
  stale-read branch (importPageMarkdown/createPage).

Suggestions:
- space.repo updateGitSyncSettings: parameterize the jsonb key (`${prefKey}::text`)
  instead of sql.raw (latent-injection footgun); value stays sql.lit. Spec updated.
- pollTick re-entrancy guard (private `polling` flag).
- page-change.listener docstring: honest about the move/rename/delete over-skip
  (loop-guard keys only on lastUpdatedSource) -> ~poll-interval latency, not loss.
- AGENTS.md: document the root /git smart-HTTP route + GitSyncModule.
- Remove redundant redteam-provenance.spec.ts (covered e2e in
  persistence.extension.spec.ts:145).
- Extract the duplicated SIGTERM->SIGKILL+finish block (watchdog + abort) into
  terminateChild; centralize watchdog-timer teardown in done().

Architecture (deferred, documented): mcp schema header now carries the three-copy
keep-in-sync + schema-core note; the editor-ext contract test documents that the
mcp copy and attribute-behaviour drift (details `open`) are not mechanically
covered yet.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
a
7179f8a5b2 fix(git-sync): address PR #119 review — close 403/404 space-existence leak + warnings/tests/arch
Security (must-fix):
- /git smart-HTTP gate: an authenticated NON-member of a git-sync space now gets
  404 (not 403), so the 403<->404 difference can no longer be used to brute-force
  which spaces exist / have git-sync enabled. 403 is reserved for a MEMBER who
  lacks the required role (existence already known). New gate input
  userIsSpaceMember; decision-table + service specs extended.

Config (must-fix):
- Remove the dead GIT_SYNC_SSH_KEY_PATH knob (getter + validation field + two
  .env.example lines) — it had zero consumers and advertised a nonexistent push
  capability.

Stability/docs (warnings):
- Wire the lost-lock AbortSignal into runReceivePack -> git http-backend so the
  receive-pack child is killed if the per-space lock lapses mid-write.
- Raise the divergent-`docmost` (invariant §5) push refusal from info -> warn and
  surface divergentDocmost in the run status (/status).
- Comment the stale read-after-debounced-collab-write updatedAt in
  importPageMarkdown (deferred §10 loop-guard must not trust it).
- Fix the Dockerfile comment: the loader uses require.resolve + dynamic import(),
  it deliberately does NOT require('@docmost/git-sync').
- Merge the two near-identical space toggle handlers into one parameterized
  handler; add the 2 missing en-US i18n keys for the auto-merge switch (ru-RU not
  maintained for these git-sync strings, mirrored).

Tests:
- isGitSyncHttpEnabled() default-branch (unset -> isGitSyncEnabled fallback).
- agentSourceFields 'git-sync' case (source stamped, chat key omitted).
- editor-ext name-level schema contract (vendored mirror superset of editor-ext
  node/mark types) + the new shared resolver + non-member 404 gate cases.

Architecture:
- Extract resolveRequestWorkspace shared by DomainMiddleware + GitHttpService
  (the two real self-hosted/cloud copies; McpService has no cloud branch).
- Document the in-process setInterval multi-replica limitation + BullMQ/fencing
  future direction (deferred, not implemented).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
a
fe4adf23a0 fix(git-sync): unwedge per-page conflicts, preserve callout types, flush collab on disconnect
Addresses QA findings on PR #119 (issues #235/#236).

SYNC-WEDGE (HIGH): one same-line conflict on one page froze sync for the
WHOLE space in both directions forever. The pull's docmost->main merge left
the vault mid-merge, so every later cycle's isMergeInProgress() check returned
skipped:"merge-in-progress" and skipped the entire space with no recovery.
- pull.ts now COMMITS a conflicting merge with markers in place (commitMerge):
  cleanly-merged pages land, the conflicted page carries its markers on main and
  is isolated by the existing push-side conflict-marker skip (markers never reach
  Docmost), and the next cycle is no longer wedged. conflictedPaths is surfaced.
- cycle.ts now RECOVERS a vault left mid-merge by a prior/pre-fix cycle: it
  aborts the stale merge (merge --abort, hard-reset fallback) and continues,
  instead of skipping the space forever.
- git.ts: listUnmergedPaths / commitMerge / abortMerge / resetHardToHead.

CALLOUT TYPE FIDELITY: git-sync's CALLOUT_TYPES was missing "note" and "default"
(editor-canonical types), so [!note]/[!default] callouts flattened to [!info] on
every round-trip. Aligned the list with @docmost/editor-ext getValidCalloutType.

LOSS-ON-FAST-CLOSE: editing a page then closing the tab inside the collab
debounce window (~3-18s) lost the edit, because with unloadImmediately:false
Hocuspocus does not flush the debounced onStoreDocument on the last-client
disconnect. PersistenceExtension.onDisconnect now flushes the pending store
(debouncer.executeNow) on the last disconnect only, with no redundant write.

DUPLICATION re-verify (#1): the schema-default merge-key normalization is intact;
faithful toYdoc-based reproduction shows callout + rich content resync with 0 ops
and no growth/strip across cycles -> the re-report was leftover vault data, not a
live regression. Locked with a callout regression spec.

Tests: git-sync 688 pass (incl. real-VaultGit wedge-recovery integration); server
git-sync+collaboration 285 pass; new callout merge/fidelity + onDisconnect-flush
specs. tsc --noEmit clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
eefe17600c test(git-sync): mock db.transaction in movePage provenance specs
After rebasing onto develop, movePage runs its cycle-check + UPDATE inside
executeTx(this.db) (develop #207 advisory-lock/atomic cycle-guard). The
git-sync provenance specs still passed a bare `{}` db, so executeTx hit
`db.transaction is not a function`. Reuse the same trxStub Proxy + transaction
mock the develop movePage specs use so both the advisory-lock `sql.execute(trx)`
and updatePage resolve. Production movePage keeps BOTH develop's lock/cycle
guard AND git-sync's provenance stamping; this only updates the test harness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
32e99c6e42 fix(editor,git-sync): parse details open as a boolean so open state survives render/round-trip
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
e48d7720e9 fix(git-sync): propagate nested details open; drop dead delete-cap wiring; cover lost-lock abort + lose-prone atom round-trips
Addresses review 1863 (delta) on PR #119.

MUST-FIX:
- detailsToHtml (the raw-HTML path used for a details nested inside
  columns/spanned cells) now emits `<details${open}>`, mirroring the
  top-level case, so `open` no longer silently drops every round trip.
- Remove the dead `resolveApplyClient` delete-cap hook from the engine
  `runCycle`: the orchestrator stopped passing it, so the hook + its
  dry-run pass were inert. Deletes are soft (Trash) + always logged and
  engine convergence is the guard, so no cap is re-added — just the dead
  wiring removed.

TEST COVERAGE:
- space-lock: heartbeat refresh CAS-miss (eval -> 0) and Redis-error
  (eval throws) both abort the in-flight fn's signal.
- cycle: a pre-aborted signal (and an abort during the pull read) throws
  before the push apply / first destructive phase.
- converter: htmlEmbed source VALUE + height survive; encode/decode
  UTF-8 symmetry and '' -> ''; footnote definition body + ref/def id
  match; transclusionReference both ids survive; fix the bad
  transclusionSource fixture (wrong `pageId` attr + empty content ->
  schema `id` + a block child); nested details `open` parity test.
- orchestrator: autoMergeConflicts:true reaches engine settings; default
  false on a missing settings row.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
42e618ec7f fix(git-sync): normalize merge key against schema defaults — cover all node/mark default-attr duplication triggers (image, link, highlight, …)
The point-fix (7a7b840e) excluded only `indent: 0` via a hardcoded one-attribute
denylist (`DEFAULT_KEY_ATTRS`) applied solely to ELEMENT attributes. The same
divergence recurs for every attribute whose editor-ext (server) schema default the
LIVE Yjs doc materializes (`TiptapTransformer.toYdoc(tiptapExtensions)`) but the
git round-trip does not: the engine's `markdownToProseMirror` emits those attrs as
explicit `null` (verified live: link mark `internal: null`, heading/paragraph
`indent: null`), which `y-prosemirror` then drops — so the same block keys
differently on the two sides, the three-way merge anchors on nothing, and the body
is re-appended every reconcile cycle (unbounded, no client connected). The denylist
also could not reach MARK attributes at all (marks are serialized raw in the
XmlText delta), so the link mark's `internal` mismatch survived.

Replace the denylist with a normalization derived from the ACTUAL ProseMirror
schema (`getSchema(tiptapExtensions)`, memoized): in `serializeXmlNode`, drop any
ELEMENT attribute whose value equals its node's schema default (or is
null/undefined), and normalize each XmlText delta op's MARK attributes the same way
against `schema.marks[name].spec.attrs`. The volatile block `id` stays excluded and
genuine non-default values (a real `indent: 2`, `align: "left"`, `link.href`,
highlight color) stay in the key. This is general — it covers indent, image.align,
link.internal, highlight.colorName, youtube/pdf and any future node/mark — not
another per-attribute denylist. Schema build is wrapped so a degenerate test stub
(`tiptapExtensions: []`) degrades to dropping only null/undefined.

Tests: new `yjs-body-merge.schema-defaults.spec.ts` models image/link/highlight
both hand-built and through the REAL `TiptapTransformer.toYdoc` materialization
(live defaults vs engine-style explicit nulls, base stale-by-one) — RED before
(4 ops / growth), GREEN after (0 ops). Existing idempotency + open-editor
convergence suites still pass (261 server collab+git-sync tests, tsc clean).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
857a0064f7 fix(git-sync): make reconcile import truly idempotent — stop runaway whole-body duplication
The live Yjs document materializes the editor-schema default `indent: 0` on
every paragraph/heading (and on the paragraph inside every list item, callout,
and table cell), but a body re-imported from git — parsed from clean markdown —
carries no indent attribute. So every live block's merge key differed from the
same block coming back from git: the three-way merge could anchor on nothing,
and the trailing unit that git's export already contained (but the merge could
not match against the byte-identical live tail) was re-appended on every
reconcile cycle. Each grown export then diverged from the last-pushed base by one
more unit — a self-sustaining, unbounded whole-body duplication loop with no
client connected.

The prior fix (0c7b73f7) excluded the volatile block `id` from the key, which
was necessary but not sufficient: `indent: 0` is a CONTENT attribute the editor
stamps as a default, so it was never stripped. Normalize editor-materialized
schema defaults (`indent: 0`) out of the block key — only the default value, so a
genuine `indent: 2` still diffs and lands — so a live block compares equal to its
git-round-tripped twin and the resync is a true no-op.

Regression test (yjs-body-merge.idempotency.spec.ts) encodes the invariant on a
body of byte-identical units (heading + paragraph + callout + table with empty
cells): a live fragment carrying indent:0 + ids merged against the git-derived
fragment (neither) with a stale-by-one base applies 0 ops and does not grow — RED
before, GREEN after.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
daf6c9ea16 fix(git-sync): propagate remote custom-event handler errors instead of 30s timeout
When a git-sync body write (gitSyncWriteBody) is routed to the collab instance
that owns the doc, the handler runs remotely inside handleRedisMessage and CAN
throw (markdown->ProseMirror transform). Previously the throw was uncaught: the
customEventComplete reply was never published, so the origin's writePageBody
promise only rejected after customEventTTL (~30s) as a generic 'TIMEOUT', and an
unhandledRejection escaped the async messageBuffer listener on the owning
instance.

Now the owner wraps handleEventLocally in try/catch and, on throw, publishes a
customEventComplete carrying an `error` field on the same correlation channel.
The origin's pendingReplies holds {resolve, reject} and rejects promptly with the
real Error. The TTL TIMEOUT remains as the fallback for a genuinely lost reply.
The no-throw and local (same-instance) paths are unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
9e69d917ee fix(git-sync): converge git-ingest with open editor sessions — stop silent revert/data-loss on live pages
A git push to a page with an OPEN editor was silently reverted: the git
commit landed and the DB body updated, but the page in the browser stayed
on the old content and the editor's next autosave overwrote the git change.

Root cause (distributed, not in the merge): writeBody applied the body
merge via collabGateway.openDirectConnection on whichever instance/process
runs git-sync (the api/worker). When an editor is connected to a DIFFERENT
collab instance/process, that opens a SEPARATE, detached Y.Doc. The merge
landed in the detached doc + DB, but the live editor's Y.Doc never received
the Yjs update; its debounced autosave then persisted its STALE state over
the DB, reverting the git change (and, for concurrent edits to different
paragraphs, losing the git side). In one process the bug is invisible
because the direct connection already shares the editor's doc.

Fix: route the body write through the existing custom-event channel (the
same mechanism comment-marks and updatePageContent use) so the merge runs
on the instance that OWNS the live doc. Its update is then broadcast to
every connection (Document.handleUpdate) and the editor's CRDT converges on
the merged result. New CollaborationGateway.writePageBody dispatches to a
new gitSyncWriteBody handler (builds incoming/base docs before opening the
connection — crash-safe — then 3-way/2-way merges into the live fragment);
without redis it runs locally on the single (owning) instance. writeBody
now just forwards the converted ProseMirror bodies + service userId.

Evidence:
- git-ingest-convergence.spec.ts: deterministic two-Y.Doc repro. PATH B
  (undelivered update) asserts the LOSS (the bug); PATH A (update delivered,
  as the owner-routed write does) asserts the git change SURVIVES and that
  concurrent edits to different paragraphs both survive.
- collaboration.handler.git-sync.spec.ts: exercises the real gitSyncWriteBody
  against a shared doc wired to a connected "editor" doc (models the
  owning-instance broadcast) — editor converges, concurrent edit preserved,
  crash-safe on transform failure.
- gitmost-datasource.service.spec.ts: writeBody now routes via writePageBody
  (RED before this change — it called openDirectConnection).

Honest scope: the failure is cross-instance; full multi-instance convergence
needs a live Hocuspocus + redis and is not provable in a unit test, so the
convergence invariant is captured at the Yjs update-exchange level.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
2594828758 fix(git-sync): idempotent first-block reconciliation — stop start-of-doc content duplicating every sync cycle
The block-level body merge keyed each block by its full attribute set,
including the per-block UniqueID the editor stamps on every heading/paragraph.
A body arriving from git is parsed from clean markdown and carries no block
ids, so a live block (id present) never matched the same block coming from git
(no id). The three-way merge's LCS could not anchor on it, and an incoming
block with no matching anchor — content inserted at the TOP of the page — was
re-added on every push/pull cycle: a non-convergent, unbounded duplication loop.

Exclude the volatile 'id' attribute from the block comparison key
(serializeXmlNode) so blocks compare by content across the git round-trip.
The merge keeps the live block INSTANCE (and its id, and any in-flight edit)
for an anchor — picks are by index, not key — so identity is preserved while
reconciliation becomes idempotent. Mirrors canonicalize.ts, which already
strips the regenerated block id from the round-trip idempotency comparison.

Adds a RED-before-fix repro modelling the live-id vs git-no-id asymmetry and
asserting no block growth across cycles.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
b5ce63a956 feat(git-sync): Obsidian-native callouts (> [!type]) instead of :::type
Callouts now export as Obsidian's blockquote-callout syntax — `> [!type]` opener
plus a `>`-prefixed body — so they render as real callouts when the vault is
opened in Obsidian, instead of `:::type` (Docusaurus-style) which Obsidian shows
as a plain blockquote.

- Export (markdown-converter `case "callout"`): `> [!type]` + each body line
  blockquote-prefixed (a blank line becomes a bare `>` so the callout is not
  split). Nested callouts naturally become `> > [!type]`.
- Import (preprocessCallouts): a new branch recognizes `> [!type]` openers and
  the contiguous `>`-prefixed body, strips one blockquote level and recurses (so
  nested callouts work), emitting the same callout div the `:::` path produces.
  The legacy `:::type` parser is KEPT so existing vaults keep importing. A plain
  blockquote (no `[!type]`) stays a blockquote.

Tests: 4 converter golden tests updated to the new `> [!type]` output; 4 new
import tests (simple, nested, round-trip, plain-blockquote-untouched). The §13.1
gate still round-trips callout losslessly through the real server schema.
git-sync vitest 675 (+1 expected-fail), gate 27.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
e777ebcf4f feat(git-sync): remove the per-cycle delete cap; deletes apply + are logged every cycle
The delete cap (GIT_SYNC_MAX_DELETES_PER_CYCLE, default 5) was a defense-in-depth
guard that SUPPRESSED a cycle's deletions when the planned count exceeded the
limit. In practice it was a crutch over engine correctness that also blocked
legitimate deletes: deleting a folder with many child pages is a normal action,
and git-sync deletes are SOFT (Trash, reversible), so a blocking limit has little
upside and real downside. There is also no user-facing surface to "confirm" a
large delete from a background sync — the only channel is the operator log.

So: drop the cap entirely. Deletes apply unconditionally; every cycle already
logs its full push plan, per-action `delete: <pageId>` lines, and completion
counts through the engine `log`, so what was deleted (and what was skipped) is
always recorded. Engine correctness (the reconcile/layout/round-trip tests) is
what prevents phantom deletions — not a blocking cap.

Removed: orchestrator `resolveApplyClient` cap hook + `maxDeletes`,
`getGitSyncMaxDeletesPerCycle`, the `GIT_SYNC_MAX_DELETES_PER_CYCLE` env/validation/.env.example,
and the cap tests. (The engine's generic optional `resolveApplyClient` hook is
left as an unused extension point.)

server tsc clean, git-sync + environment jest 174.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
abd6e3948b fix(git-sync): preserve subpages.recursive and details.open on round trip
Found proactively by deepening the round-trip test from node-TYPE survival to
ATTRIBUTE fidelity (distinctive attr values per node). Two real losses (the
other 3 candidates — mathInline/mathBlock/pageEmbed — were verified to be
correct; the probe had used wrong attr names):

- subpages `recursive`: the converter emitted a bare div and the schema mirror
  didn't model the attr, so a recursive subpages reverted to non-recursive on a
  round trip. Now emits `data-recursive="true"` and the mirror parses it back
  (matching @docmost/editor-ext).
- details `open`: the `open` (collapsed/expanded) state lives on the details
  node, but the converter emitted the `<details>` wrapper from the summary case
  without it, so the state was dropped. The wrapper now carries `open`.

The round-trip test now also asserts attribute fidelity (12 cases) so these are
locked. Schema-surface snapshot updated for the new subpages attr.

git-sync vitest 671 (+1 expected-fail), §13.1 gate 27.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
5125296bfa fix(git-sync): subpages round-trips (was {{SUBPAGES}} literal) + exhaustive all-node round-trip test
subpages exported to the literal `{{SUBPAGES}}`, which has no markdown/HTML
inverse, so on re-import it came back as a plain paragraph holding the visible
text "{{SUBPAGES}}" — the embed rendered as that literal string on the page
after a sync (round-trip data loss, seen live). It now emits the schema-matching
`<div data-type="subpages">` like every other embed node, so the schema's
parseHTML rebuilds the subpages node. Also dropped the leaf-atom content-hole
in the subpages renderHTML.

New committed regression coverage:
- packages/git-sync/test/roundtrip-all-nodes.test.ts — exhaustive serialize ->
  deserialize round trip for ALL 40 node/mark types; each asserts the node/mark
  survives and no `{{...}}` literal leaks. This is the test that caught subpages.
- §13.1 gate (git-sync-converter-gate.spec.ts): subpages added to the green
  corpus (round-trips through the REAL server schema).
- Corrected two PR-authored tests that asserted the old {{SUBPAGES}} loss as
  "by design" — they now assert the fixed round trip.

Also folds in review #1679 coverage-gap tests (no prod change): orchestrator
pollTick/enabledSpaces, datasource 3-way merge dispatch, page.repo
last_updated_source provenance SQL.

git-sync vitest 659 (+1 expected-fail), server tsc clean, server specs green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
452a752264 fix(git-sync): don't run a Docmost cycle on receive-pack info/refs (fixes deterministic push 503)
A git push is a two-request exchange: GET info/refs?service=git-receive-pack
(ref advertisement) then POST git-receive-pack (the pack). The git-HTTP host
classified BOTH as serviceKind 'write' and routed both through
ingestExternalPush, which takes the per-space lock and runs a FULL Docmost
reconcile cycle. So the read-only info/refs advertisement held the lock while a
cycle ran, and the client's immediately-following POST git-receive-pack collided
with that still-running cycle and got 503 — deterministically, every push (and
Obsidian Git's "scan" failed for the same reason, since it probes push
capability via the same receive-pack info/refs).

Fix: only the actual pack-receiving write (POST git-receive-pack) runs under the
lock + cycle. Everything else streams the http-backend directly with no lock and
no cycle — a fetch/clone (read) AND the write-AUTHORIZED but read-only
info/refs?service=git-receive-pack advertisement. Authz is unchanged (the gate
still requires write permission for receive-pack refs); only the side effect of
running a cycle on a read-only request is removed.

Verified end-to-end on a live stand: clone, then `git push` of a new file lands
the page in Docmost (was 503 on every push before). Regression test added.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
a40a00d5c5 feat(git-sync): per-space toggle for conflict-marker handling on push (#13)
Red-team #13 (conflict markers reaching Docmost) is now a per-space policy
exposed as a UI toggle, instead of a hardcoded behavior. New boolean
`gitSync.autoMergeConflicts` (default FALSE), mirroring the existing per-space
`gitSync.enabled` flag end-to-end (jsonb space settings -> update-space DTO ->
space.service -> client types -> space settings form switch):

- OFF (default, safe): a page whose committed body still has unresolved git
  conflict markers is NOT pushed — it is recorded as a per-page push FAILURE
  ("unresolved conflict markers — resolve in git first"). Recording a failure
  (not a soft skip) deliberately HOLDS refs/docmost/last-pushed so the conflict
  commit is never marked pushed and a later pull cannot clobber the user's
  in-progress resolution; the page retries until the conflict is resolved in git.
- ON: the marker lines are stripped and both sides' content is pushed (the prior
  behavior), so the conflict becomes visible/fixable inside Docmost.

The engine Settings carries `autoMergeConflicts`; runPush threads it into the
update AND create paths. The orchestrator's buildSettings reads the per-space
flag from jsonb (strict opt-in like `enabled`, default false).

Tests: redteam-push-cycle #13 rewritten (default -> not pushed + failure + refs
held; ON -> strip-and-push); space.service + edit-space-form + orchestrator
specs extended. git-sync vitest 618, server jest space+git-sync 163, client
edit-space-form 11, server/client tsc clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
81c0226be7 docs(git-sync): document GIT_SYNC_BACKEND_TIMEOUT_MS, drop dead consts, fix dangling plan refs
Address the non-red-team documentation/cleanup items from review #1679:
- Document the GIT_SYNC_BACKEND_TIMEOUT_MS watchdog (git http-backend) in
  .env.example and add it to the environment validation schema — it was used
  (getGitSyncBackendTimeoutMs, default 120000) but undocumented/unvalidated.
- Remove the dead GIT_SYNC_DEBOUNCE_MS_DEFAULT / GIT_SYNC_POLL_INTERVAL_MS_DEFAULT
  exports (never imported; environment.service is the single source of defaults).
- Redirect the dangling `plan §X.Y` comment references to issue #194 (the
  git-sync spec moved there when docs/git-sync-plan.md was deleted by this PR).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
d5079aa1d8 fix(git-sync): red-team hardening — 12 confirmed sync-breaking bugs + regression tests
A 10-agent red-team pass on the two-way Docmost<->git sync surfaced 16 ranked
findings (9 others triaged out as already-defended). Wrote a reproduction test
per finding (each asserts the CORRECT behavior, so it fails on the bug), then
fixed the production code so every repro goes green. All confirmed bugs:

Round-trip data loss (markdown-converter.ts + docmost-schema.ts mirror):
- #1 editor-ext node types silently dropped on export — ported the 8 missing
  canon nodes (footnoteReference/footnotesList/footnoteDefinition, htmlEmbed,
  status, pageEmbed, transclusionSource/Reference) into the git-sync schema
  mirror and added converter cases that emit their schema-matching HTML instead
  of flattening unknown nodes to '' (this was the critical data-loss flagged in
  review #1679: footnotes/htmlEmbed lost on sync). Snapshot surface updated.
- #2 top-level image lost width/height/align/attachmentId — now emits an HTML
  <img> (like video/diagrams) when it carries layout attrs; bare images stay
  ![](src). Image node parses width/height as strings so they re-import.
- #3 code block containing a ``` fence corrupted on round-trip — outer fence is
  now widened to (longest-inner-backtick-run + 1).
- #16 deep nesting threw RangeError (page never synced) — added a depth guard
  (MAX_NODE_DEPTH=400) so the converter never overflows the stack.

Push/layout/cycle (engine):
- #4 disambiguation ' ~slugId' suffix corrupted Docmost titles + order-dependent
  layout — deterministic, order-independent sibling disambiguation; suffix is
  stripped from a path-derived title ONLY when the new name is exactly the old
  title plus the suffix (never a genuine retitle ending in ' ~token').
- #6 retry-adopt by (parent,title) clobbered the wrong duplicate-title sibling —
  ambiguous (parent,title) is no longer adopted (falls back to fresh create).
- #12 a new child under a new parent was created at ROOT — creates are ordered
  parent-before-child with an in-memory created-id map for parent resolution.
- #13 git conflict markers could reach Docmost — bodies are scanned and the
  marker lines stripped (a '=======' line is only treated as a conflict
  separator inside a <<<<<<< ... >>>>>>> block, so setext headings are safe).
- #15 a divergent `docmost` mirror was escalated by runPush but dropped by
  runCycle — RunCycleResult now forwards divergentDocmost to the orchestrator.

Server (merge / lock / provenance):
- #9 3-way merge lost a human's block edit when git inserted an adjacent block —
  finer-grained diff3 region merge (via lcs) preserves non-overlapping human
  edits; genuine same-block conflicts still resolve git-wins.
- #10 single-writer race — module-static liveLocks closes the same-process TOCTOU
  window, and a heartbeat refresh that cannot confirm the lock now aborts the
  cycle at its next write checkpoint (cooperative AbortSignal threaded through
  runCycle). Cross-process fencing tokens remain a follow-up.
- #14 sticky-agent provenance overrode an explicit actor='git-sync' write,
  blinding the listener loop-guard — resolveSource now lets an explicit actor
  win over the sticky-agent fallback (explicit agent still wins).

Verified: git-sync vitest 617 pass (+1 expected-fail), server unit jest 1541
pass, server tsc clean. A review pass over the fixes caught and corrected a
title-suffix over-strip, an inert abort signal, a document-wide conflict-marker
strip, and two leaf-atom content-holes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
b536a41ad3 chore(git-sync): drop stray build/ artifacts re-introduced during rebase
build/ is gitignored and compiled in CI/Docker; a few files leaked back into
the tree while replaying commits onto develop. Remove them so the package keeps
a single source of truth (src/).
2026-06-28 15:10:10 +03:00
claude_code
28d2560dfd fix(git-sync): address PR #119 review (#1571)
Resolve the code-review findings from comment #1571 on PR #119.

Engine (packages/git-sync):
- Idempotent CREATE on retry: before createPage, look the page up in the
  live Docmost tree by (parentPageId, title) and ADOPT it instead of
  duplicating when a prior cycle created it but failed to persist the
  pageId back to disk. Only trust a COMPLETE tree for the lookup; fall
  back to createPage otherwise. Covered by new tests incl. a complete=false
  regression-lock.
- Route applyPullActions diagnostics through an injected logger instead of
  bare console (thread log from the cycle).
- Add a timeout to the git execFile chokepoint (runRaw) so a hung git
  subprocess cannot wedge a sync cycle.
- Translate remaining Russian code comments to English.
- Remove dead standalone-CLI code (parseArgs/PushParsedArgs,
  parseSettings/envSchema, loadSettingsOrExit + config-errors.ts) and the
  matching index exports/specs; keep the Settings type.
- Fix the dangling docs link in package.json.
- Add a schema-surface snapshot guard so any drift in the vendored
  document schema is a loud, must-review CI failure (+ provenance header).

Server (apps/server):
- Add a configurable watchdog timeout to the spawned git http-backend so a
  stalled push cannot hold the per-space lock forever
  (GIT_SYNC_BACKEND_TIMEOUT_MS).
- Close the in-process TOCTOU window in SpaceLockService.withSpaceLock by
  reserving the slot synchronously before acquire.
- Add tests: removePage git-sync provenance (both branches), ensureServable
  force-push-protection git configs, and the phase-B+ datasource methods.

Docs / build:
- AGENTS.md: list git-sync as the fifth workspace package and note the
  three schema mirrors; fix the dangling git-sync-plan.md backlog link.
- pnpm-lock.yaml: add the missing @docmost/git-sync workspace link so
  pnpm install --frozen-lockfile (CI default) succeeds.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
52959de2f3 chore(mcp): drop build/ + node_modules leftovers after rebase
These files (build/lib/footnote-analyze.js, build/lib/footnote-lex.js from the
merged footnote work, and the y-prosemirror node_modules symlink) survived the
rebase because this branch's earlier "stop committing build/ and node_modules"
commit predated them. They are gitignored (packages/mcp/build/) and generated /
symlinked, so untrack them to keep the branch consistent with that decision.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
5da12e89f9 refactor(git-sync): internalize the engine — first-class ESM, no vendoring bridge (#119 review)
Closes the architecture item from the #119 review: drop the "vendored from
docmost-sync" framing and the CJS↔ESM `Function('import()')` bridge so the engine
is a normal first-class gitmost package.

Part 1 — vendoring markers removed (prose only, zero behavior change): reworded
"VENDORED into gitmost" / "vendored from docmost-sync" / "Engine LOGIC is
byte-identical" / "it's a port" comments across the engine. Behavior-bearing
strings are untouched: BOT_AUTHOR_NAME/EMAIL and the `Docmost-Sync-Source:`
provenance trailers (changing them would break git authorship + the loop-guard).

Part 2 — the package is now ESM (matching the sibling @docmost/mcp): `type: module`,
tsconfig Node16, `.js` extensions on relative imports, and a static
`import { marked }` replacing the `new Function('return import(...)')` /
`loadMarked` hack — the bridge is GONE from the package. The CommonJS NestJS
server loads the now-ESM engine via a new `git-sync.loader.ts` that mirrors the
existing `docmost-client.loader.ts` mcp loader exactly (Function-indirected
dynamic import + cached promise + retry-on-reject). The 4 server consumers
(orchestrator/datasource/vault-registry/git-http-backend) call `await loadGitSync()`
for value exports; types stay `import type` (erased). The converter-gate spec —
which needs the real converter — loads the package's TS source via a jest
moduleNameMapper + isolatedModules (documented in that spec); the other git-sync
specs mock the loader.

Verified: engine builds pure ESM (no Function/require leftover), vitest 614,
editor-ext build, server + client tsc, full server jest 1397/0. Live stand
smoke-test: server starts clean on the ESM engine (no ERR_REQUIRE_ESM), a real
sync cycle runs through the loader, and the basic e2e suite is 12/12 (clone via
git-http-backend, push, pull, delete, 3-way merge — all through the new loader).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
3a91e0eca9 test(git-sync): add missing DTO/User imports for the rebased git-sync provenance spec block
The rebase folded develop's agent-provenance PageService spec and the git-sync
provenance spec into one file; the appended git-sync block needs CreatePageDto /
UpdatePageDto / User imports that develop's spec (which used inline `as any`) did
not have. Server tsc + the suite (158 tests, both provenance blocks) green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
2e83c9cebf fix(git-sync): git-http stream error handlers + close test gaps (#119 review)
Addresses the stability + test-coverage warnings from the #119 review:

- git-http-backend.service.ts: add `'error'` handlers to child.stdout/stderr. An
  EventEmitter 'error' with no listener (e.g. EPIPE when the client aborts
  mid-response) is rethrown by Node as an uncaught exception and crashes the
  process; now swallowed + logged (never echoed to the client).
- TEST INFRA: a jest setupFile shims `navigator`/`MessageChannel` for the `node`
  testEnvironment. react-dom@18 reads `navigator` at module-init (pulled in via
  @docmost/editor-ext -> @tiptap/react), so every spec transitively importing the
  conversion engine — including git-http.service.spec.ts — previously FAILED TO
  LOAD ("navigator is not defined") and ran ZERO tests. With the shim those specs
  now run (git-sync integration: 11 suites / 133 tests green).
- git-http.service.spec.ts: cover the 503 lock-held push path — `ingestExternalPush`
  rejecting `GitSyncLockHeldError` -> 503 + Retry-After + "git-sync busy, retry",
  no double header write (+ the already-headers-sent no-rewrite path).
- git-http-backend.service.spec.ts: unit-test run() — child 'error'/'close' before
  headers -> 500; normal CGI parse+stream; stdout/stderr 'error' (EPIPE) swallowed;
  synchronous spawn throw -> 500.
- page-change.listener.ts: implement OnModuleDestroy to clearTimeout all pending
  debounce timers on shutdown (+ test).
- .env.example: vaults are non-bare working repos, not "bare repos".

(Docs deleted by the stray commit were restored in 9cdbce54.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
f6d22a59a6 fix(git-sync): screen non-page files out of PUSH (CRITICAL — review)
Self-review of phase 3 caught a data-corruption regression: nativeMeta always
supplies the run's spaceId, so the planner's 'create-without-spaceId' skip — which
had doubled as the only filter for non-page files — went dead. An ADDED
.obsidian/*.json, attachment, or dotfile (committed to the vault, no .gitignore)
would then be classified as a CREATE: a junk Docmost page, plus a gitmost_id
frontmatter written INTO the file, corrupting it.

Fix: isPageFile(path) — a .md file with NO dot-segment anywhere — and filter the
diff to page files at the very top of computePushActions, BEFORE any
classification, so non-page A/M/D/R are ignored (design §Адопция). 2 unit tests
pin it (.obsidian/json, attachment, dotfile, dot-segment, .md dotfile all ignored;
real pages still created). 614 engine tests green.

Also: refreshed stale docmost:meta comments to gitmost_id (review SUGGESTION), and
documented the deferred adoption frontmatter-preservation gap (review WARNING) in
page-file.ts + the design doc (do NOT roll native onto a real vault with Obsidian
properties until phase 4 round-trips them).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
6baad935f9 docs(git-sync): mark thin-meta phases 2 + 3 done in the plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
d255afa611 feat(git-sync): phase 3 — PUSH reads native gitmost_id + derives title/parent from path
PUSH now consumes the native-Obsidian format end-to-end:
- identity from the gitmost_id frontmatter (parsePageFile), not docmost:meta;
- title from the FILENAME, parentPageId from the enclosing folder's folder-note
  (parentFolderFile is now FOLDER-NOTE aware: a child's parent is dir/dir.md, and
  a folder-note's own parent is one level up), spaceId from the run (every vault
  file belongs to the vault's space);
- CREATE derives title/parent/space from path + run and writes the assigned
  pageId back as gitmost_id frontmatter (serializePageFile);
- UPDATE pushes the STRIPPED body (current + 3-way-merge base), so the frontmatter
  never leaks into Docmost content; the loop-guard hashes the body.

The PURE delete-sensitive classifier (computePushActions/classifyRenameMoves) is
UNCHANGED — only the injected IO resolvers (metaAt, parent, create write-back)
switched source. nativeMeta always carries the run spaceId, so the legacy
'create-without-spaceId' skip no longer fires through runPush.

Tests rewritten to native fixtures + folder-note parent paths; the noop case is
now a child under a renamed parent folder (filename=title, so a path-only-noop
needs an ancestor rename). parentFolderFile tests cover leaf/folder-note/nested/
dotted. 612 engine tests green; engine rebuilt.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
73c5c44301 feat(git-sync): phase 2b — PULL writes native gitmost_id frontmatter
PULL now serializes each page as the native-Obsidian format (serializePageFile:
a minimal gitmost_id frontmatter + the fixpoint markdown body) instead of the
heavy docmost:meta envelope. title/parent/space are derived (filename / folder /
repo), so only the pageId is persisted. readExisting recovers identity from the
gitmost_id frontmatter (parsePageFile) instead of docmost:meta.

Extracted stabilizePageBody() (the export->import->export fixpoint, no meta) so
the native writer and the legacy serializer share the same deterministic body —
re-pulls of an unchanged page stay byte-identical (loop-guard).

Tests: read-existing fixtures rewritten to gitmost_id; apply-pull asserts the
written text is native frontmatter and carries NO docmost:meta (regression
guard). 611 engine tests green.

NOTE: PUSH still reads docmost:meta — the end-to-end cycle is intentionally NOT
runnable until phase 3 (PUSH reads frontmatter + derives title/parent from path)
lands; no vault is wiped/deployed until then.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
8c42c4f0d6 feat(git-sync): phase 2a — folder-note layout (parent -> Folder/Folder.md)
Native-Obsidian structure: a page WITH children now lives at its folder-note
<name>/<name>.md (LostPaul Folder Notes convention) with its children alongside;
a leaf stays <name>.md. Folder-notes claim their canonical path before a
same-named child, so the child (a leaf) is the one disambiguated, never the
folder-note — a folder X/ always contains its own note X.

Format-agnostic and safe in isolation: only the destination PATH changes, the
file content/serialization is untouched, so an existing parent relocates via the
move-by-id path (no delete). The frontmatter format flip (pull+push) is next.

6 new layout unit tests (leaf / parent / nested / child-named-as-parent /
twin-parents / childless). 611 engine tests green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
071eae4e2a feat(git-sync): drop legacy docmost:meta back-compat (vaults wipe+rebuild)
Per owner: test data, no migration. parsePageFile no longer reads the old
docmost:meta block — a file without a gitmost_id frontmatter is simply un-tracked
(adopt). Vaults are a cache: rm -rf on the transition, rebuilt native from
Docmost. Simplifies the format work (no fallback). Doc updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
a91405632e feat(git-sync): native-Obsidian format — phase 1 = page-file (frontmatter gitmost_id)
Pivot the thin-meta design to "the vault IS a native Obsidian vault": clean
markdown + a minimal YAML frontmatter `gitmost_id:` (the durable pageId, travels
with the file so identity survives any move); folders mirror the page tree with
the parent's body as a folder-note `<Folder>/<Folder>.md` (LostPaul Folder Notes
convention); links as `[[wikilinks]]` (basename-resolved → reparent never breaks a
link, only retitle does); collisions disambiguated Obsidian-style; `.obsidian/`
and non-page files left untouched (no .gitignore). Verified the conventions
against the Obsidian/Folder-Notes docs.

Replaces the abandoned `.gitmost/index.json` sidecar (path-keyed → fragile to
git-undetected renames; the in-file id is self-sufficient): removes vault-index.ts.
Adds lib/page-file.ts — parsePageFile/serializePageFile (frontmatter id + clean
body) with a LEGACY `docmost:meta` fallback for migration. 6 unit tests; engine
suite green. Not yet wired into pull/push — no behavior change. Design doc
rewritten to the native-Obsidian format.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
5d4eb8ede2 docs(git-sync): thin-meta design — identity must travel with the file (B/C) + link-sync phase
Captures the design discussion: a path-keyed sidecar is NOT a safe source of
truth (a git-undetected rename loses the page), so the id must travel WITH the
file — either as a slugId suffix in the filename (B) or a minimal YAML frontmatter
`id:` (C); both robust, B/C is the open UX decision (author leans C for clean
names). The sidecar may remain an optional path->id cache. Adds phase 6 — link
sync between notes: Docmost links are by pageId (survive rename), vault markdown
links are by path (rewrite on rename, Obsidian-style); independent of B/C and the
format phases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
aa1ee64b7a feat(git-sync): thin-meta phase 1 — the .gitmost/index.json sidecar module
Pure read/write/lookup for the vault sidecar index that will hold page identity
(pageId) + collision token (slugId) keyed by file path, so the .md files can be
clean markdown. parseVaultIndex is tolerant (missing/garbage/bad entries degrade
to empty/skipped — never crashes a cycle); serializeVaultIndex is deterministic
(sorted keys -> stable diffs, no churn). Lookups (pageIdAt, pathForPageId reverse,
trackedPageIds) + mutations (set/remove/move). NOT wired into pull/push yet — no
behavior change. 5 unit tests; engine suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
53febfd5b9 docs(git-sync): design for thin meta + third-party-editor support
All service metadata moves into a single `.gitmost/index.json` sidecar; the `.md`
files become clean markdown (Obsidian & any editor work directly). The page tree
mirrors the folder structure (folder = parent page; the parent's body lives in
`<Folder>/index.md`); collisions disambiguate by a `~<slugId>` filename suffix
with identity tracked by pageId in the index (safe renames, never delete+create —
backed by 5133bb34). Bare files/folders from a third-party editor are adopted into
pages. Includes the migration path off the current `docmost:meta`-in-file format
and a phased plan (each phase gated by engine unit tests + the browser e2e +
isolated shell e2e). Agreed with the owner 2026-06-24.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
a2ac08c04c test(git-sync): e2e suites provision a throwaway space — never touch real data
The shell e2e suites defaulted to the General space and created/edited pages
there, polluting real content (and, when several enabled spaces raised poll
contention, flaking on 503s). Now each suite creates its OWN throwaway,
git-sync-enabled space at setup, runs everything against it, and deletes the
space (+ its vault) on exit. Set SPACE_ID explicitly to opt into an existing
space. Also gives the basic suite the 503-retry push helper the advanced one
already had. Verified isolated: basic 12/12, advanced 23/23, no spaces/users/
pages left behind, the real space untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
40ca04eb08 fix(git-sync): never trash a page whose pageId still exists in the tree (cross-cycle move) + browser e2e
Follow-up to 4376c5a6, found by a real BROWSER e2e (the flow the in-diff fix
missed). When the layout reshuffle's two halves land in SEPARATE sync cycles, the
later cycle's diff has only the DELETE of the old path — the matching add was
already pushed — so in-diff D+A coalescing can't see it, and the live page was
still trashed.

Robust fix on the identity invariant the reviewer (and the user) called out: a
page EXISTS iff its pageId is in the vault, regardless of filename. runPush now
collects the pageIds present at ANY path in the current `main` tree and passes
them to computePushActions; a deleted file whose pageId is still tracked
elsewhere is a MOVE, never a deletion. (Built only when the diff has deletes.)

Adds apps/server/test/git-sync-browser-e2e.cjs — a Playwright test that drives the
REAL Docmost web UI: log in, create several untitled pages, type a title, sync,
assert NOTHING is trashed. Reproduced the data loss before this fix; 5/5 green and
stable after. Engine suite 600 green (+2 computePushActions cases:
pageId-still-present -> skip; pageId-gone -> real delete).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
393875d910 test(git-sync): e2e guard for the untitled-page + retitle data-loss reshuffle
Reproduces the browser bug at the API level: create several untitled pages (all
collapse to the `_` fallback name), retitle one, sync — assert NO page is
trashed and all survive. Caught the data-loss bug fixed in 4376c5a6.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
c3dbee9fbf fix(git-sync): never trash a page that only MOVED (pageId-identity, not git rename heuristics) — data loss
CRITICAL data-loss bug: creating pages in Docmost (which start UNTITLED) and then
typing a title could soft-delete OTHER pages. Untitled pages all serialize to the
`_` fallback filename; the layout disambiguates them (`_.md`, `_ ~slug.md`).
Retitling one frees the bare `_` and another untitled page's file relocates into
it. git's rename detection (`-M`) can't see the move (the tiny meta-only files are
too dissimilar), so `git diff` reports it as DELETE(old) + ADD/MODIFY(new). The
push took the DELETE literally and trashed a live page.

Root cause is that the push trusted git's path-level rename heuristic for page
IDENTITY. Identity is the pageId. Fix: before emitting any delete, coalesce by
pageId — a pageId that is BOTH deleted (pre-image) AND present on the surviving
side (current meta of an ADD or a MODIFY, since a relocation into an occupied path
shows as M) is one page that MOVED, classified as a rename/move and NEVER a delete.

Reproduced + verified on a live stand: 4 untitled pages + retitle one trashed a
different page before; after the fix, retitling one (and stress-retitling all)
trashes nothing. Engine suite 598 green; 3 new computePushActions cases (ghost
D+A move -> rename; real delete still deletes; unrelated D+A stay delete+update).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
ea1f8da906 test(git-sync): basic e2e operates on a dedicated page + cleans up (no real-page pollution)
The push / 3-way-merge cases edited the FIRST real `.md` in the vault, leaving
`E2E-PUSH-*` / `E2E-MERGE-*` marker headings accumulating in a real page, and the
Docmost->git case left its created page in the Trash. Now the suite creates a
dedicated `E2E-SyncTarget-*` page and targets only that, and a teardown
hard-deletes every `E2E-*` fixture page and converges the vault on exit — so runs
never mutate real content and leave the stand clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
9baaf1ea58 test(git-sync): add advanced e2e suite — authz, protocol hardening, concurrency, data-loss guard
Output of a generate→critique subagent pass on "what the feature's tests do NOT
cover", implemented + verified against the live stand (20/20). Complements the
basic two-way suite. Covers:

- protocol shape: unknown service subpath -> 400; unknown content-type -> 415
  (global allowlist); PUT/DELETE on pack endpoints -> 400;
- path-traversal: `..%2f..`, `%2e%2e%2f`, bare `.git` space-id -> 400/404, no
  escape, never a file leak;
- authz boundaries: a gitSync-DISABLED space -> 404 (existence hidden) and flips
  to 200 when enabled; a READER member can fetch (200) but is FORBIDDEN to push
  (403); a NON-member of an enabled space gets 403 (NOT 404 — the critic caught a
  wrong generator assumption here; pinned as a contract);
- concurrency: a push while the per-space Redis lock is held -> 503 + Retry-After,
  and the receive-pack does NOT mutate the vault;
- idempotency: repeated no-op cycles never churn `main` / `refs/docmost/last-pushed`;
- data-loss guard (PR #119): deleting MORE than GIT_SYNC_MAX_DELETES_PER_CYCLE is
  HELD — none trashed AND last-pushed does not advance past the delete commit
  (retry-safe, not silently dropped).

Auto-creates/tears down its fixtures (reader/non-member users, a 2nd space) and
resets the vault cache on exit so re-runs and the basic suite stay green. Needs
the vault dir + Redis container reachable (see header). A structural rename/move
case was intentionally left to the engine unit suite (git rename-similarity on
meta-only fixture pages is a fixture artifact, not a feature bug).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
71375e25ee chore(git-sync): drop now-unused dirname import (PR #119 review)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
e528988d71 test(git-sync): add a live two-way smart-HTTP e2e suite
A runnable end-to-end suite that drives a LIVE git-sync stand over the real /git
remote — the integration counterpart to the unit tests. 10 checks across the full
feature:
- the auth/authz gate: no creds -> 401, wrong password -> 401, unknown space ->
  404 (existence never revealed), valid creds on a sync space -> 200;
- fetch: git clone over HTTP returns the vault markdown;
- push: a git-side edit propagates into the Docmost page;
- Docmost -> git: a page created via the API materializes as a vault file;
- delete: `git rm` + push soft-deletes the Docmost page (Trash);
- 3-way merge: a new git edit is added without clobbering prior page content.

Parameterized via env (SERVER/SPACE_ID/EMAIL/PASSWORD/DB_CONTAINER) and isolates
its own test page. It boots nothing — see the header for the stand prerequisites
(GIT_SYNC_ENABLED + a per-space gitSync flag + a service user). This is the suite
that caught the smart-HTTP PATH_INFO 404 bug.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
dc7a0ec9f5 refactor(git-sync): move the PULL->PUSH cycle into the engine as runCycle (PR #119 review, arch #1)
The reconcile choreography (ensureRepo -> merge-check -> ensureBranch ->
checkout('docmost') -> pull -> push) was hand-rolled in the app orchestrator's
driveCycle, duplicating an order the vendored engine owns and could drift from on
upgrade — the failure mode is data clobber. Lift it into @docmost/git-sync as a
single entry point, `runCycle(deps)`. The orchestrator now calls runCycle and
keeps only the lock (its caller) and the gitmost-specific delete-cap POLICY,
injected as the `resolveApplyClient` hook (the engine does the dry-run, hands the
hook the planned delete count — Infinity if planning failed — and uses whatever
client it returns for the apply). driveCycle drops from ~150 lines to ~30.

Tests:
- engine test/cycle.test.ts: composition (merge-in-progress short-circuit;
  ensureRepo->ensureBranch->checkout staging order before the pull; the cap hook
  is consulted with the planned count; no dry-run when no hook).
- engine test/cycle-roundtrip.test.ts: runCycle against a REAL VaultGit in a temp
  repo with a faked Docmost client — a git-originated CREATE flows pull->push and
  the assigned pageId is written back; an unresolved merge short-circuits before
  any client call.
- orchestrator spec rewired to mock runCycle and assert the wiring + the
  resolveApplyClient cap policy (the engine-internal cycle-order/merge tests moved
  to the engine).

Validated end to end on a live stand (real Postgres/Redis + server): a git clone
-> edit -> push over the /git remote round-trips the change into the Docmost page
through the refactored cycle.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
969c00aaf1 fix(git-sync): drop the .git suffix from git http-backend PATH_INFO (smart-HTTP 404)
The /git smart-HTTP host 404'd EVERY fetch and push: PATH_INFO was built as
`/<spaceId>.git/<subpath>`, so `git http-backend` resolved the repo at
`<GIT_PROJECT_ROOT>/<spaceId>.git` — which does not exist. The vault is a NON-bare
working repo (the engine needs a working tree) at `<dataDir>/<spaceId>`, so the
CGI repo path must be `<spaceId>` (git http-backend serves the `.git` inside).
The URL's conventional `.git` suffix is already stripped to `spaceId` by
parseGitPath; re-appending it for PATH_INFO was the bug.

Found by standing up a full e2e stand (real Postgres/Redis + server + a real git
clone/push over the /git remote): clone and push both 404'd until this fix, after
which a clone → edit → push round-trips the change all the way into the Docmost
page.

Also extracts the CGI-env construction into a pure, exported `buildGitBackendCgiEnv`
and adds unit tests (the env build was previously untested — the gap this bug hid
in): a regression guard pinning PATH_INFO to `/<spaceId>/<subpath>` (no `.git`),
plus method/query/content-type/remote-user forwarding and the conditional
GIT_PROTOCOL.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
085a30575f test(git-sync): cover ingestExternalPush in the orchestrator spec (PR #119 review)
Closes the test-coverage warning that the smart-HTTP push ingest path was
unexercised. Adds 5 cases: receive-pack streams BEFORE the Docmost cycle; a
held lock throws GitSyncLockHeldError and runs neither the receive-pack nor the
cycle; a post-push cycle error is swallowed (the push is durable, poll retries)
while the lock is still released; a missing service user runs the receive-pack
but skips the immediate cycle; and a globally-disabled git-sync refuses without
touching the lock.

(The 503/Retry-After mapping in git-http.service is the sibling warning; its spec
is in the repo's pre-existing set of jest suites that can't load locally via the
react-dom/tiptap transform chain, so that case is left for CI.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
95bc9fe98d refactor(git-sync): extract SpaceLockService from the orchestrator (PR #119 review, arch #2)
The per-space single-writer lock — Redis CAS leader lock (SET NX PX, DEL-CAS and
PEXPIRE-CAS Lua), the in-process mutex, the per-process instanceId and the
heartbeat — lived inline in GitSyncOrchestrator. Extract it into a dedicated
@Injectable() SpaceLockService exposing one narrow surface, withSpaceLock(spaceId,
fn), so the lock is the orchestrator's only Redis-lock touch-point and is testable
in isolation. The orchestrator now injects SpaceLockService and both consumers
(runOnce, ingestExternalPush) go through spaceLock.withSpaceLock — behavior
unchanged (same sentinel returns, same 503-on-lock-held contract). Orchestrator
drops 591→472 lines.

Adds space-lock.service.spec.ts asserting the lock SEMANTICS against a fake Redis
(the test-coverage warning from the review): the SET NX/PX args, the DEL-CAS and
PEXPIRE-CAS Lua + ARGV[1]=instanceId, plus the lock-held / in-progress / throw-
still-releases paths. The orchestrator spec is unchanged in count and stays green
(it now builds the real SpaceLockService over its mock Redis).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
cca0bfe306 docs(git-sync): remove dangling references to the deleted git-sync-plan doc (PR #119 review)
The implementation spec docs/git-sync-plan.md was removed as completed, but ~44
code comments still cited it as "plan §N". Strip those citations (comments only),
keeping each comment grammatical. The vendored engine's own "SPEC §N" references
point at a different, still-present spec and are left untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
0dbf85b129 refactor(git-sync): drop dead DebounceEntry.workspaceId field (PR #119 review)
The debounce map value carried `workspaceId`, but the scheduled cycle closes over
the `workspaceId` argument directly — the field was written and never read.
Replace the entry struct with `Map<string, NodeJS.Timeout>` (the timer handle is
all the map tracks). No behavior change. (page-change.listener.spec is in the
repo's pre-existing set of jest suites that can't load locally via the
react-dom/tiptap transform chain — unaffected by this change; tsc clean.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
fb357cd52e refactor(git-sync): extract shared buildLcsTable for the two block diffs (PR #119 review)
The two-way block diff (yjs-body-merge.diffBlocks) and the three-way merge
planner (three-way-merge.lcsPairs) built the identical backward-filled LCS DP
table inline. Extract it to lcs.ts (buildLcsTable); each caller keeps its own
traceback. No behavior change — merge specs unchanged and green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
177d8a31d4 fix(git-sync): hold refs on suppressed deletes + stamp delete/restore provenance (PR #119 review)
Two stability warnings from the #119 review:

1. delete-cap no longer drops deletions forever. When planned deletes exceed
   GIT_SYNC_MAX_DELETES_PER_CYCLE the apply client's deletePage now THROWS
   instead of resolving to a no-op. A throw is recorded by the engine as a
   per-page failure, so `refs/docmost/last-pushed` is NOT advanced past the
   commit that dropped the files — the next cycle re-diffs from the un-advanced
   ref and re-plans the same deletes (a transient over-cap is retried, not
   silently dropped and then recreated by the next pull). Previously a resolving
   no-op let the engine count `deleted++` with no failure, advance the ref, and
   never replay the deletions.

2. git-sync soft-delete and restore now stamp provenance. deletePage routes
   GIT_SYNC_PROVENANCE through pageService.removePage, and restorePage stamps
   lastUpdatedSource='git-sync' on the restore update — so the page-change
   listener's loop-guard (skip when lastUpdatedSource==='git-sync') recognizes
   both as its own writes instead of scheduling a wasted echo cycle. Done via a
   backward-compatible optional `lastUpdatedSource` param on
   pageRepo.removePage/restorePage (omitted for ordinary user deletes/restores).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
8fa32e8438 docs(git-sync): document GIT_SYNC_* env vars; fix stale/non-English comments (PR #119 review)
Addresses the documentation/convention warnings from the #119 review:
- .env.example: add the GIT-SYNC block (9 GIT_SYNC_* vars with defaults), noting
  GIT_SYNC_SERVICE_USER_ID is required when sync is enabled.
- yjs-body-merge.ts: translate the Russian review note in the docstring to
  English (comments-only-in-English rule).
- persistence.extension.ts: correct the stale "git-sync writes are full-body
  replaces" rationale — a git-sync write is now a block-level merge into the live
  doc, which is why it is debounced like a human edit rather than snapshotted.
- history-item.tsx: the GitSyncBadge version is created on the PUSH path (writing
  the git body back into the doc), not by the pull — fix the comment.
- edit-space-form.tsx: log the raw error in the git-sync toggle catch instead of
  swallowing it (AGENTS.md).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
807ff1f5f5 chore(mcp): stop committing build/ and node_modules; build in CI/Docker
Same hygiene fix as git-sync (review #2), applied to packages/mcp which had the
identical pre-existing problem: committed build/ (20 files) + node_modules (28,
pnpm symlinks with a baked /home/claude store path).

- git rm --cached packages/mcp/{build,node_modules}.
- .gitignore: add packages/mcp/build/ (packages/*/node_modules/ already covers it).
- Build where consumed: apps/server `pretest` and the CI Test workflow now build
  @docmost/mcp too. The Dockerfile builder already runs `pnpm build` (nx builds
  mcp) and already COPYs packages/mcp/build into the runtime image.

Verified: wiped build/, rebuilt via `pnpm --filter @docmost/mcp build`; the mcp
server suites (96 tests) pass against the freshly-built, non-committed output.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:10:10 +03:00
claude code agent 227
fa89cba023 feat(git-sync): three-way body merge using the last-synced base (no edit loss)
Upgrades the 2-way body merge to a real diff3 three-way merge (review #5), so a
block ONLY the human changed is KEPT when git changed a DIFFERENT block — the
2-way merge would revert it to git's stale version.

Engine: the push update loop reads the last-synced pre-image
(`git.showFileAtRef(refs/docmost/last-pushed, path)`) and passes it as the
optional `baseMarkdown` to `client.importPageMarkdown` (the common ancestor).

Server: gitmost-datasource converts base+incoming, and writeBody runs a block-
level diff3 (new three-way-merge.ts `diff3Plan`): live-only change -> keep live,
git-only change -> take git, both-changed -> git wins (conflict policy), inserts/
deletes from either side preserved. Without a base (createPage) it falls back to
the 2-way merge. Crash-safety unchanged (docs built before the connection opens).

Tests: three-way-merge.spec.ts (14 — every diff3 case incl. the cross-block
preservation and conflict policy), yjs-body-merge 3-way (real Y.Docs: human's
block instance preserved while git's block is applied), plus an engine test that
the base is forwarded from showFileAtRef. Existing push assertions updated for the
new base arg. git-sync 589 pass; server merge/datasource/gate 62 pass; typecheck
clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
3386bf2865 fix(git-sync): merge git body into the live doc block-by-block (no clobber)
Supersedes the active-session "defer" guard with a real merge (review #5 —
"запись делать через мерж", not skip-while-editing).

writeBody no longer does delete-all + re-insert (which discarded a concurrent
editor's in-flight changes on every sync). It now diffs the live body against the
incoming git body at TOP-LEVEL BLOCK granularity (LCS over a canonical structural
serialization) and applies only the minimal inserts/deletes:
- a block a human is editing is left UNTOUCHED when git changed a DIFFERENT block;
- an unchanged resync is a complete 0-op write;
- Yjs CRDT-merges the minimal ops with concurrent edits.

New yjs-body-merge.ts (mergeXmlFragments + cloneXmlNode + diffBlocks) is pure-Yjs
and unit-tested with real Y.Docs (8 tests): identical->0 ops, edit-one-block keeps
the other block instances, append/delete keep neighbours, marks survive the
cross-doc clone. Crash-safety kept: the incoming doc is built before the
connection opens, so a transform failure can't empty the body.

Removed: the ActiveEditSessionError defer path and the now-unused
CollaborationGateway.getActiveEditorCount.

Honest limitation: this is a 2-way merge — for a block BOTH sides changed since the
last sync, git wins (no common ancestor to decide). A full 3-way merge would need
the last-synced base plumbed from the engine; the dominant cases (unchanged
resync, edits to different blocks) are now lossless.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
98253cf614 chore(git-sync): stop committing build/ and node_modules; build in CI/Docker
Review finding #2: packages/git-sync/build/ (the COMPILED engine) and the
package's node_modules/ were committed. Prod executed the committed build/ while
CI/tests ran src/ and never rebuilt it — so a fix in src/ could pass tests while
stale compiled code shipped (a silent src/prod skew). The committed node_modules
were pnpm symlinks with a baked machine-local store path (/home/claude/...),
useless and misleading for everyone else.

- git rm --cached packages/git-sync/{build,node_modules} (42 + 31 files).
- .gitignore: ignore packages/*/node_modules/ and packages/git-sync/build/.
- Build the package where it is actually consumed: apps/server `pretest` now
  builds @docmost/git-sync (its suite imports the built build/index.js), and the
  CI Test workflow gains an explicit "Build git-sync" step. The Dockerfile builder
  already runs `pnpm build` (nx builds the package) and now COPYs the fresh build/.

Verified: wiped build/, rebuilt via `pnpm --filter @docmost/git-sync build`, then
the server converter gate (26/26, imports the rebuilt package) and the git-sync
suite (588 passed) both pass against the freshly-built, non-committed output.

NOTE: packages/mcp/ has the same committed-build/node_modules pattern (pre-existing,
out of this PR's scope) and should get the same treatment in a follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
181a8330f3 fix(git-sync): don't clobber pages with a live editing session; crash-safe body write
Review finding #5: the git -> page body write (writeBody) did a full-body replace
(delete-all + re-insert) on the shared Yjs doc. Applied while a human is editing
the page, it discarded their in-flight changes; and TiptapTransformer.toYdoc ran
AFTER the fragment was cleared, so a conversion failure could leave the page with
an empty body.

Fixes:
- Active-session guard: CollaborationGateway.getActiveEditorCount(documentName)
  reports live human (websocket) editor sessions for a doc, excluding server-side
  direct connections. writeBody now throws ActiveEditSessionError when an editor
  is connected. The engine's push loop already isolates each importPageMarkdown in
  try/catch and does not advance the loop-guard on failure, so the write is simply
  retried on the next poll once the editor disconnects — never a clobber.
- Crash-safe conversion: build the replacement Yjs update BEFORE opening the
  connection / clearing the fragment, so a transform failure can never leave the
  body empty.

Also updates the server-side converter gate spec to the corrected round-trip
shape: the block-image hoist no longer leaves a leading empty paragraph (the
git-sync converter fix in 7d39c16b, now reaching the built package).

A true merge of git content into a live Yjs session is out of scope (it needs a
real 3-way text merge with no shared update lineage); deferring the write while a
page is being edited is the safe, owner-approved minimum.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
02daccc453 fix(docker): ship packages/git-sync into the runtime image
The server requires @docmost/git-sync (main: ./build/index.js) at runtime, but
the installer stage copied only editor-ext and mcp — so the image built fine and
then crashed on startup with `Cannot find module '@docmost/git-sync'`. Copy the
package's freshly-built build/ + package.json, mirroring the mcp/editor-ext COPY
lines. (Addresses review finding #1 on PR #119.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
d06cf97ed6 test(git-sync): exhaustive converter coverage + fix 3 round-trip data-loss bugs
Coder↔reviewer design loop (9 rounds, reviewer verdict: exhaustive) produced
92 specs; implemented +123 tests (465 -> 588 passing). The new round-trip
coverage exposed three genuine data-loss bugs in the Markdown<->ProseMirror
converter, all now FIXED (round-trip is lossless for these):

1. pageBreak was lost on export (no converter case -> rendered to "" and the
   node vanished). Now emits <div data-type="pageBreak"></div>, which the schema
   parses back -> round-trips.
2. A block image between blocks left an empty <p> artifact after import-hoisting,
   producing a phantom blank-gap diff on every sync. markdownToProseMirror now
   strips content-less paragraphs after generateJSON — with a schema-validity
   guard that keeps the obligatory single empty paragraph in `content: "block+"`
   containers (tableCell/tableHeader/blockquote/column/callout/doc), so empty
   cells/quotes never become an invalid `content: []`.
3. The `code` mark combined with another mark was not byte-stable (emitted nested
   HTML that the schema's `code` `excludes:"_"` collapsed on import). The
   converter now emits code-only when `code` co-occurs, matching the editor.

New coverage spans media/diagram/details/columns/math/mention attribute
round-trips, converter emission branches, git error paths, and engine decision
branches. A dedicated test pins the empty-container schema validity (the review
catch on the bug-2 fix).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude_code
04032ae677 feat(git-sync): serve spaces over smart-HTTP (gitmost as a two-way git host)
Expose each git-sync-enabled space as a clonable/pushable git repo over HTTP,
so `git clone https://<user>:<pass>@<host>/git/<spaceId>.git` works and external
pushes flow back into Docmost pages — gitmost itself acts as the git host (no
external GitHub/Gitea, no SSH).

Transport: shell out to `git http-backend` (CGI; git is already in the runtime
image) which implements the full smart-HTTP protocol (info/refs, upload-pack,
receive-pack, protocol v2). A raw Fastify route `/git/*` (mounted at the root,
outside the `/api` prefix) bridges the request/response to the CGI; passthrough
content-type parsers for the git media types stream the raw body to stdin.

Reuse the existing engine: clients push the vault's `main` branch, whose commits
beyond `refs/docmost/last-pushed` the engine already reconciles into Docmost.

- http/git-http.service.ts — auth (HTTP Basic -> AuthService.verifyUserCredentials),
  self-resolved workspace (DomainMiddleware does not run for this raw route),
  per-space gating (global + per-space gitSync flags, 404 hides existence),
  CASL authz (Read=fetch, Manage=push), dispatch.
- http/git-http-backend.service.ts — spawn `git http-backend`, binary-safe CGI
  response parsing (Status/headers/body), stream to the socket.
- http/git-http.helpers.ts — pure path parse, service->kind mapping, gate decision
  (unit-tested); rejects literal and percent-encoded path traversal.
- orchestrator: extract reusable withSpaceLock (CAS-guarded lock heartbeat so a
  long push cannot let the lock expire mid-cycle) and add ingestExternalPush
  (receive-pack + Docmost cycle under one lock; 503 on contention).
- vault-registry: ensureServable() — ensureRepo + idempotent receive.denyCurrentBranch
  =updateInstead / denyNonFastForwards / http.receivepack / http.uploadpack.
- env: GIT_SYNC_HTTP_ENABLED (defaults to GIT_SYNC_ENABLED) + validation.
- main.ts: register the /git/* route and the git content-type parsers.

Tests: pure helpers, CGI parsing, and the GitHttpService handler (auth/gate/authz
+ workspace resolution). Server tsc + git-sync/env suites green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude_code
d9d1d54aaa test(git-sync): add reviewer-requested coverage across engine, server, client
Implements the test cases called out in the PR #119 review threads
(code-review, test-strategy report, red-team) — TESTS ONLY, no production
code changes.

packages/git-sync (vitest):
- lib converter/markdown gaps: pageBreak data-loss (it.fails repro),
  subpages lossy round-trip, nested/fenced callouts, ol->taskList bridge,
  column.width number<->string drift, empty details.
- engine units: parentFolderFile, planReconciliation swap/chained move,
  buildVaultLayout last-resort-by-id, firstDivergence, applyPushActions /
  applyPullActions failure isolation.
- real temp-git integration: diffNameStatus -z rename+add/modify
  alignment, copy-line behavior, per-invocation committer identity (no
  leak into repo/global config).
- ENFORCED type-level GitSyncClient contract via vitest typecheck over a
  *.test-d.ts file (tsconfig.vitest.json; build tsconfig untouched).

apps/server (jest):
- orchestrator: delete-cap neutralization + fail-safe, Redis lock / mutex
  skip ladder + release-on-throw, merge guard, pull/push order, remote
  template substitution, poll lifecycle.
- page-change listener: loop-guard, debounce coalescing, id resolution,
  error swallowing.
- vault registry, controller authz (trigger + status), env
  validation/getters, page.service git-sync provenance stamping,
  persistence precedence (agent > git-sync > user) + no boundary snapshot,
  space.service audit-delta, space.repo jsonb-merge, converter-gate corpus
  extension (mention/math/details/marks).

apps/client (vitest + testing-library):
- history-item git-sync badge: render gating + non-clickable.
- edit-space-form toggle: initial state, optimistic payload, rollback on
  error, disabled states.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
593f181bbc fix(git-sync): address review — configurable poll, always-on loop-guard, cleanup
Comprehensive-review follow-ups (APPROVE WITH SUGGESTIONS; no critical issues):
- poll interval is now actually configurable: replaced the hardcoded
  @Interval('git-sync-poll', 15000) with a dynamic SchedulerRegistry interval
  registered in onModuleInit from getGitSyncPollIntervalMs() (cleared in
  onModuleDestroy); /status and the real cadence now share one config source.
  Boots logging 'poll interval registered (Nms)'.
- loop-guard now ALWAYS applies: the lastUpdatedSource==='git-sync' skip was
  nested inside the !spaceId/!workspaceId branch, so structural self-writes
  (CREATE/MOVE/RESTORE/SOFT_DELETE, which carry spaceId+workspaceId) bypassed it
  and re-triggered cycles. Fetch the page row once, guard unconditionally, then
  resolve space/workspace.
- remove the dead PAGE_CONTENT_UPDATED subscription (it's a BullMQ job, never an
  EventEmitter event; body edits arrive via PAGE_UPDATED).
- fix the stale datasource comment (PageService DOES stamp 'git-sync' now).
- env getters: parseInt radix 10 + NaN/<=0 fallback for poll/debounce (+ max
  deletes), with 6 new environment.service.spec tests.

tsc clean; jest 723 pass; live cycle re-verified post-refactor (ran, push
applied, unflagged 92-page space untouched).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
582e1976cc feat(git-sync): client 'Git sync' provenance badge + git in runtime image (Phase D)
- page-history history-item: a lastUpdatedSource==='git-sync' version renders a
  neutral gray 'Git sync' badge (git-merge icon), NOT the agent badge/deep-link
  (it is not an agent edit). +2 i18n keys.
- Dockerfile: install git in the installer (runtime) stage — VaultGit shells out
  to git, so assertGitAvailable() needs the binary at runtime.
Client tsc clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
e0e01157c2 feat(git-sync): per-space 'Enable Git sync' toggle (Phase C, §7.1)
UI opt-in for git-sync, mirroring the existing sharing/comments settings pattern
(no new endpoint, no new mechanism; orchestrator read query untouched):
- UpdateSpaceDto.gitSyncEnabled?: boolean.
- SpaceRepo.updateGitSyncSettings: jsonb-merge into settings.gitSync.<key>
  (COALESCE || jsonb_build_object — never clobbers sibling sharing/comments);
  stored as a real jsonb boolean so the orchestrator's
  settings->'gitSync'->>'enabled' = 'true' matches.
- SpaceService.updateSpace handles the flag (audit diff) via the existing
  CASL-guarded space update path (Manage/Settings).
- client: Switch in edit-space-form (optimistic mutate + revert-on-error,
  readOnly-aware) + space types + 2 i18n keys.
- space.service.spec extended (calls updateGitSyncSettings; no-op when undefined).
tsc clean (server+client); jest src/core/space 4 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
8373360a67 fix(git-sync): branch choreography + strict scoping + delete cap (Phase B hardening)
Fixes found by the live pull/push e2e:
- CRITICAL: driveCycle never checked out the 'docmost' branch before
  applyPullActions, so Docmost content was written straight onto 'main',
  clobbering local file edits before push could diff them. Now checkout
  'docmost' before pull (applyPullActions commits there then checks out main +
  merges) — mirrors the engine's pull main(). Round-trip now works both ways.
- add an unresolved-merge guard (SPEC §9): skip the cycle if the vault is
  mid-merge instead of failing on checkout.
- SAFETY: enabledSpaces() is now STRICT opt-in — only spaces with
  settings.gitSync.enabled===true; removed the all-spaces fallback that synced
  every space (incl. a 92-page one) the moment GIT_SYNC_ENABLED flipped.
- SAFETY: per-cycle delete cap (GIT_SYNC_MAX_DELETES_PER_CYCLE, default 5):
  dry-run the push, and if planned deletes exceed the cap, run the apply with
  deletePage neutralized — phantom absence-deletions from a non-convergent vault
  can't soft-delete real pages. Fails safe if the dry-run throws.
- fix manual trigger: TriggerGitSyncDto.spaceId needs @IsUUID or the global
  whitelist ValidationPipe strips it (arrived undefined -> vault 'undefined').

Live-verified on an isolated flagged space: push (vault file edit -> Docmost
content, stamped lastUpdatedSource='git-sync') and pull (Docmost rename -> vault
file + meta) both work; an unrelated 92-page space stayed untouched throughout.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
e2493cafa9 feat(git-sync): GitSyncModule orchestrator + config + listener (Phase A.4b/B)
Control plane wiring (plan §5-§11):
- PageService create/update/movePage now honor provenance actor 'git-sync'
  (stamp lastUpdatedSource='git-sync'), closing the A.4a gap.
- EnvironmentService: GIT_SYNC_ENABLED / DATA_DIR / REMOTE_TEMPLATE /
  POLL_INTERVAL_MS / DEBOUNCE_MS / SERVICE_USER_ID (required-if-enabled) /
  SSH_KEY_PATH + validation.
- VaultRegistryService: per-space vault path + cached VaultGit.
- GitSyncOrchestrator: per-space Redis leader-lock (SET NX PX + CAS-Lua release,
  randomUUID instanceId) + in-process mutex; runOnce drives the vendored engine
  PULL (readExisting->computePullActions->applyPullActions) then PUSH (runPush)
  with the bound native GitSyncClient + VaultGit; @Interval poll-safety gated on
  GIT_SYNC_ENABLED; imports plain ScheduleModule (TelemetryModule owns forRoot).
- PageChangeListener: @OnEvent PAGE_* -> per-space debounce -> runOnce, with a
  best-effort lastUpdatedSource==='git-sync' loop-guard.
- GitSyncController: admin POST /api/git-sync/trigger + GET /status (ops/e2e).
- GitSyncModule registered in app.module. Enabled-space enumeration uses
  settings.gitSync.enabled, falling back to all live spaces until Phase C writes
  the flag (master gate = GIT_SYNC_ENABLED).

tsc clean; 713 tests/71 suites pass; dev server hot-reloaded the module (route
live, DI graph boots). Live pull/push round-trip verified next.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
5a4d9f84d7 feat(git-sync): native GitmostDataSource + 'git-sync' provenance (Phase A.4a)
Native data plane for git-sync (plan §3, §8.1):
- provenance: widen actor to 'user'|'agent'|'git-sync' (jwt-payload,
  auth-provenance decorator); PersistenceExtension resolves lastUpdatedSource
  with precedence agent > git-sync > user, debounced history (like a human edit,
  not the agent's immediate snapshot).
- GitmostDataSourceService implements @docmost/git-sync's GitSyncClient natively:
  reads via PageRepo/SpaceRepo (listSpaceTree complete:true, getPageJson), writes
  via PageService (create/removePage soft-delete/movePage with computed fractional
  position/update-rename/restore) + the writeBody linchpin through collab
  openDirectConnection('page.'+id, {actor:'git-sync'}) mirroring
  collaboration.handler withYdocConnection 'replace'. bind({workspaceId,userId})
  returns the context-bound client for the orchestrator.
- 10 unit/contract tests (mapping + soft-delete + move-position), tsc clean.

Known gap (closed in A.4b): PageService.create/update/movePage only branch on
actor==='agent'; git-sync provenance is already passed through so the row source
marker propagates once PageService honors 'git-sync'. Module/orchestrator/config
come next.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
70bd0dba4d feat(git-sync): vendor IO engine (pull/push/git/settings) with GitSyncClient seam (Phase A.3)
Vendor the IO engine from docmost-sync into packages/git-sync/src/engine:
- git.ts (VaultGit, execFile shell-out — verbatim)
- pull.ts (readExisting, computePullActions, applyPullActions)
- push.ts (classifyRenameMoves, computePushActions, applyPushActions, runPush)
- settings.ts adapted (pure parseSettings + Settings type; no process.env binding
  — the server builds Settings from EnvironmentService later), config-errors.ts.
CLI main()/import.meta entrypoints dropped (server drives in-process).

Client seam: new engine/client.types.ts defines GitSyncClient; pull.ts/push.ts
now use Pick<GitSyncClient, ...> instead of the non-vendored DocmostClient. Engine
logic byte-identical except a zod4-compat fix in config-errors (zod4 dropped the
issue.received==='undefined' signal; match /received undefined/ on the message).

Ported the engine unit tests (compute/apply pull+push actions, classify-rename-
moves, run-push, settings, config-errors) incl. real-git temp-repo tests: 431
pass / 3 expected-fail (was 314/3). REST/CLI-coupled upstream tests skipped
(noted). CJS build clean. No apps/server wiring yet (next step).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
b0cd4bd6cf feat(git-sync): CommonJS build + §13.1 editor-ext idempotency gate (Phase A.2)
Make @docmost/git-sync natively consumable by the CommonJS server (and jest):
build to CommonJS (tsconfig module CommonJS, drop type:module, strip .js from
relative imports), and lazy-load the only ESM-only dep (marked) via the dynamic
Function('import()') trick (mirrors docmost-client.loader.ts) with a require()
fallback so vitest's evaluator works too. git-sync tests stay green (314 pass,
3 expected fail).

Add the §13.1 idempotency gate (apps/server .../git-sync-converter-gate.spec.ts):
13 editor-ext docs (paragraphs/headings, marks, links, bullet/ordered/task lists,
blockquote, callouts, code block, hr, table, nested mix) round-trip
content(editor-ext) -> convertProseMirrorToMarkdown -> markdownToProseMirror ->
TiptapTransformer.toYdoc/fromYdoc(tiptapExtensions) -> canonicalize and assert
docsCanonicallyEqual. All green => the vendored converter's docmost-schema is
schema-compatible with editor-ext (no node/mark/attr loss), which the plan §13.1
requires before Phase B. The one intrinsic markdown-image lossiness (width/height
/align can't ride plain ![](src)) is isolated in a KNOWN DIVERGENCE block, not
hidden. Server tsc clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
claude code agent 227
56ab17fbc2 feat(git-sync): vendor pure converter + engine into @docmost/git-sync (Phase A.1)
First step of docs/git-sync-plan.md. New workspace package @docmost/git-sync
vendoring the PURE parts from docmost-sync (HEAD b03eb35):
- lib: markdown-converter, markdown-document, canonicalize, docmost-schema,
  node-ops, diff, and an extracted markdown-to-prosemirror (only the pure
  marked->HTML->generateJSON path from upstream collaboration.ts; no websocket).
- engine (pure, no IO): reconcile, layout, sanitize, stabilize, loop-guard.
Ported the upstream pure-module + round-trip corpus tests (vitest): 314 pass,
3 expected upstream known-limitation fails. tsc clean. No server wiring yet.

docmost-schema inlines getStyleProperty (as packages/mcp does — @tiptap/core
3.20.4 doesn't export it). IO engine (pull/push/git/settings) deferred to later
Phase A/B steps; the editor-ext idempotency gate (plan §13.1) is the next step.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 15:09:57 +03:00
371 changed files with 36978 additions and 24644 deletions

View File

@@ -92,19 +92,6 @@ IFRAME_EMBED_ALLOWED=false
# Example: https://intranet.example.com,https://portal.example.com
IFRAME_ALLOWED_ORIGINS=
# Comma-separated list of additional origins allowed to call the API via CORS.
# The APP_URL origin and native mobile (Capacitor) origins are always allowed.
# Leave empty for a same-origin (web-only) deployment.
CORS_ALLOWED_ORIGINS=
# Expose OpenAPI/Swagger docs at /api/docs (development/debugging aid only).
SWAGGER_ENABLED=false
# Capacitor (mobile shell): hosted client URL loaded by the iOS shell so the
# AGPL web client is NOT bundled into the .ipa (see docs/mobile-app-plan.md §9).
# Leave empty for Android bundled mode / local development.
CAP_SERVER_URL=
# Enable debug logging in production (default: false)
DEBUG_MODE=false
@@ -216,3 +203,42 @@ MCP_DOCMOST_PASSWORD=
# FAILS CLOSED if Redis is unavailable (default: 1,000,000 tokens per workspace
# per rolling day).
# SHARE_AI_WORKSPACE_TOKEN_BUDGET_PER_DAY=1000000
# --- GIT-SYNC (native two-way Docmost <-> git Markdown sync) ---
# Master switch. Off by default. When 'true', GIT_SYNC_SERVICE_USER_ID below is
# REQUIRED (the service account that git-originated create/move/rename/delete are
# attributed to) — the server refuses to boot with sync enabled and no user id.
# GIT_SYNC_ENABLED=false
#
# Serve the per-space vaults over smart-HTTP (the /git host). Defaults to
# GIT_SYNC_ENABLED when unset.
# GIT_SYNC_HTTP_ENABLED=false
#
# REQUIRED when GIT_SYNC_ENABLED=true: id of the user that git-originated page
# operations (create / move / rename / delete) are attributed to.
# GIT_SYNC_SERVICE_USER_ID=
#
# Where the per-space working vaults live (non-bare repos; the engine needs a
# working tree).
# Defaults to "<DATA_DIR or ./data>/git-sync".
# GIT_SYNC_DATA_DIR=
#
# Optional remote URL template to mirror each space's vault to (e.g. a git host).
# The literal "{spaceId}" is substituted per-space, so each space mirrors to its
# OWN remote — e.g. git@host:vault-{spaceId}.git. Without the placeholder every
# space would point at one remote. Leave unset to keep vaults local-only.
# GIT_SYNC_REMOTE_TEMPLATE=
#
# Poll-safety interval in ms — the cadence of the background reconcile cycle
# (default: 15000).
# GIT_SYNC_POLL_INTERVAL_MS=15000
#
# Debounce window in ms for collapsing bursts of page edits into one sync cycle
# (default: 2000).
# GIT_SYNC_DEBOUNCE_MS=2000
#
# Watchdog timeout in ms for the spawned `git http-backend` process serving a
# git smart-HTTP push (default: 120000). A stalled/hung receive-pack is killed
# after this deadline so it cannot hold the per-space lock forever.
# GIT_SYNC_BACKEND_TIMEOUT_MS=120000
#

View File

@@ -68,6 +68,13 @@ jobs:
- name: Build editor-ext
run: pnpm --filter @docmost/editor-ext build
# git-sync and mcp are no longer committed in built form (build/ is
# gitignored), so CI must compile them: the server resolves both via their
# built build/index.js. The server pretest also builds them, but building
# here keeps it explicit and independent of pnpm lifecycle ordering.
- name: Build git-sync and mcp
run: pnpm --filter @docmost/git-sync build && pnpm --filter @docmost/mcp build
- name: Run unit tests
run: pnpm -r test

11
.gitignore vendored
View File

@@ -5,6 +5,12 @@ data
# compiled output
/dist
/node_modules
# workspace package node_modules (pnpm symlinks — never commit; they bake
# machine-local store paths) and the git-sync compiled output (built in CI/Docker
# via `pnpm build`, never committed, so src/ and prod can never silently diverge).
packages/*/node_modules/
packages/git-sync/build/
packages/mcp/build/
# Logs
logs
@@ -49,8 +55,3 @@ lerna-debug.log*
# Self-hosted VAD / onnxruntime-web assets (copied from node_modules at dev/build time)
apps/client/public/vad/
# Capacitor native platform projects (generated locally via 'npx cap add ios|android')
/ios
/android
.capacitor

View File

@@ -182,7 +182,7 @@ tea issues create --repo vvzvlad/gitmost --labels feature \
## Monorepo layout
pnpm workspace (`pnpm@10.4.0`) orchestrated by **Nx**. Four workspace packages:
pnpm workspace (`pnpm@10.4.0`) orchestrated by **Nx**. Five workspace packages:
| Path | Name | Stack | Role |
| --- | --- | --- | --- |
@@ -190,6 +190,7 @@ pnpm workspace (`pnpm@10.4.0`) orchestrated by **Nx**. Four workspace packages:
| `apps/client` | `client` | React 18 + Vite + Mantine 8 + TanStack Query + Jotai | SPA frontend |
| `packages/editor-ext` | `@docmost/editor-ext` | Tiptap/ProseMirror | Shared Tiptap node/mark extensions, imported by both the client and the server |
| `packages/mcp` | `@docmost/mcp` | MCP SDK, Tiptap, Yjs | Standalone MCP server, also bundled into the server at `/mcp`. Does **not** import `editor-ext` — it keeps its own vendored mirror of the schema in `packages/mcp/src/lib/` |
| `packages/git-sync` | `@docmost/git-sync` | Tiptap/ProseMirror, Yjs, git | Pure ProseMirror↔Markdown converter plus the two-way Docmost↔git Markdown sync engine. Bundled into the server (loaded over the ESM bridge), built in CI and the Dockerfile. Does **not** import `editor-ext` — it keeps its own vendored mirror of the document schema (kept in sync with `editor-ext`). |
`build` targets are Nx-cached and dependency-ordered (`dependsOn: ["^build"]`), so `editor-ext` builds before the apps. `nx.json` sets `affected.defaultBase: main`.
@@ -243,8 +244,10 @@ Migration files live in `apps/server/src/database/migrations/` and are named `YY
The API server is a Fastify app with a global `/api` prefix (`main.ts` excludes `robots.txt`, public share pages, and `mcp` from the prefix). A `preHandler` hook enforces that a resolved `workspaceId` exists for most `/api` routes (multi-tenant by hostname/subdomain via `DomainMiddleware`). Auth is JWT (cookie + bearer); authorization is **CASL** (`core/casl`) — every data access is scoped to the user's abilities.
Two routes are mounted **outside** the `/api` prefix at the root, as raw Fastify routes that bypass the Nest pipeline (so neither `DomainMiddleware` nor `ThrottlerGuard` runs for them — each resolves the workspace and throttles itself): `/mcp` (the embedded MCP server, see below) and `/git/<spaceId>.git/...` (the git-sync smart-HTTP host, see below). Both share `mcp-auth.helpers.ts` (HTTP-Basic parsing, `FailedLoginLimiter`, `clientIp`) and the common `resolveRequestWorkspace` helper.
### Module structure (server)
`AppModule` wires integration modules (`integrations/*`: storage [local/S3/Azure], mail, queue [BullMQ on Redis], security, telemetry, throttle, `mcp`, `ai`) plus `CoreModule`, `DatabaseModule`, and `CollaborationModule`. `CoreModule` (`core/*`) holds the domain modules: `page`, `space`, `comment`, `workspace`, `user`, `auth`, `group`, `attachment`, `search`, `share`, `ai-chat`, etc. Each domain module follows NestJS controller → service → repo layering; DB repos live under `database/repos` and are injected app-wide from the global `DatabaseModule`.
`AppModule` wires integration modules (`integrations/*`: storage [local/S3/Azure], mail, queue [BullMQ on Redis], security, telemetry, throttle, `mcp`, `ai`, `git-sync`) plus `CoreModule`, `DatabaseModule`, and `CollaborationModule`. `CoreModule` (`core/*`) holds the domain modules: `page`, `space`, `comment`, `workspace`, `user`, `auth`, `group`, `attachment`, `search`, `share`, `ai-chat`, etc. Each domain module follows NestJS controller → service → repo layering; DB repos live under `database/repos` and are injected app-wide from the global `DatabaseModule`.
**EE removal artifact:** `app.module.ts` still contains a `try/require('./ee/ee.module')` stub. That path no longer exists, so the require fails and is swallowed (it only hard-exits when `CLOUD === 'true'`). Treat EE as gone — do not add code that depends on it.
@@ -260,10 +263,16 @@ The API server is a Fastify app with a global `/api` prefix (`main.ts` excludes
- `core/ai-chat/embedding/` — RAG indexer + a BullMQ consumer on `AI_QUEUE` that embeds pages into `page_embeddings` (vector search), complementing Postgres full-text search. Pages are (re)indexed on edit; `AI_EMBEDDING_TIMEOUT_MS` bounds a hung embeddings endpoint.
- `core/ai-chat/external-mcp/` — admins can attach external MCP servers (e.g. Tavily) to give the agent web access. **`ssrf-guard.ts` validates outbound MCP URLs against SSRF** — keep that guard in the path when touching external-MCP connection logic.
### Git-sync (native two-way Docmost ↔ git Markdown sync)
`integrations/git-sync/` (`GitSyncModule`) + the vendored pure engine in `packages/git-sync`. Off by default; gated by the `GIT_SYNC_ENABLED` master switch (and `GIT_SYNC_SERVICE_USER_ID`, the account git-originated writes are attributed to). Per-space opt-in via `space.settings.gitSync.enabled`, with a second per-space toggle `space.settings.gitSync.autoMergeConflicts` that changes PUSH behavior for a still-conflicted page (one carrying `<<<<<<<`/`>>>>>>>` markers): **off (the safe default)** records a per-page failure and holds the refs so the user resolves the git conflict first (markers never reach Docmost); **on** strips the marker lines and pushes both sides' content. Each enabled space gets an on-disk working "vault" repo; the `GitSyncOrchestrator` runs a debounced + poll-backstop reconcile cycle (PULL Docmost→vault, PUSH vault→Docmost) under a per-space Redis leader lock + in-process mutex (`SpaceLockService`). Writes go through the collaboration layer (so concurrent human edits aren't clobbered) and are stamped `lastUpdatedSource = 'git-sync'` for the listener loop-guard. The in-process `setInterval` orchestration + best-effort lock (no fencing tokens) is a known multi-replica limitation — BullMQ + fencing is the documented future direction.
- **`/git` smart-HTTP host** (`integrations/git-sync/http/`, gated additionally by `GIT_SYNC_HTTP_ENABLED`, which defaults to `GIT_SYNC_ENABLED`): a raw root-mounted Fastify route `/git/<spaceId>.git/...` (registered in `main.ts`, NOT under `/api`) that bridges `git clone`/`fetch`/`push` to `git http-backend`. It authenticates HTTP Basic against `AuthService` (throttled by a `FailedLoginLimiter` mirroring the `/mcp` path), authorizes via `SpaceAbilityFactory` (read = fetch, Manage = push), and gates existence so a non-member gets the SAME 404 as a missing/sync-disabled space (never 403 — that would leak space existence). A push runs the receive-pack under the space lock, then a reconcile cycle.
- **Schema mirror:** `packages/git-sync/src/lib/docmost-schema.ts` is one of the **three** hand-synced copies of the Tiptap document schema (see Client structure) — keep it in lockstep with `editor-ext` (canonical) and `packages/mcp`.
### Client structure
Vite SPA. Code is organized by feature under `apps/client/src/features/*` (mirrors the server domains: `page`, `space`, `comment`, `ai-chat`, `editor`, …). Conventions:
- **TanStack Query** for server state (one `queries/` file per feature), **Jotai** atoms for local/shared UI state, **Mantine 8** + CSS modules (`*.module.css`) + `postcss-preset-mantine` for UI.
- The editor is Tiptap; shared node/mark extensions live in `packages/editor-ext` and are imported by **both the client and the server** (collaboration, import/export) — editor schema changes often need to be made in `editor-ext`, not just the client. Note `packages/mcp` does *not* depend on `editor-ext`; it carries its own mirrored copy of the schema, so keep the two in sync manually when the document schema changes.
- The editor is Tiptap; shared node/mark extensions live in `packages/editor-ext` and are imported by **both the client and the server** (collaboration, import/export) — editor schema changes often need to be made in `editor-ext`, not just the client. Note neither `packages/mcp` nor `packages/git-sync` depends on `editor-ext`; each carries its own mirrored copy of the schema. There are now **three** independent copies (`editor-ext` is canonical, plus `packages/mcp` and `packages/git-sync`), so keep all three in sync manually when the document schema changes.
- API access goes through `apps/client/src/lib/api-client.ts` (axios). The `@` alias maps to `apps/client/src`.
- Runtime config is injected at build time by `vite.config.ts` via `define` (`APP_URL`, `COLLAB_URL`, `APP_VERSION`, …) — these come from the root `.env`, not from `import.meta.env`.

View File

@@ -12,6 +12,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
- **Native two-way Docmost ↔ git Markdown sync.** Opt-in per space (Space
settings → a git-sync toggle, plus an `autoMergeConflicts` toggle that controls
whether a still-conflicted page is held back or pushed with its conflict
markers stripped): each enabled space is mirrored to an on-disk git "vault" of
Markdown files and reconciled in both directions (Docmost → vault and vault →
Docmost) on a debounced + poll-backstop cycle, under a per-space lock, writing
through the collaboration layer so concurrent human edits aren't clobbered.
Git-originated changes are attributed to a configurable service account and
carry a "git-sync" provenance badge in page history. Optionally exposes a `/git`
smart-HTTP host so you can `git clone`/`fetch`/`push` a space directly (HTTP
Basic auth, space-permission authorized). Off by default and configured via the
`GIT_SYNC_*` environment variables, including `GIT_SYNC_ENABLED`,
`GIT_SYNC_SERVICE_USER_ID`, and `GIT_SYNC_HTTP_ENABLED` (see `.env.example`).
(#119)
- **Quick-create regular and temporary notes from the Home and Space screens.**
The Home screen now shows a second action next to "New note" that creates a
*temporary* note (one that auto-moves to Trash after the workspace lifetime),
@@ -67,33 +81,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
toggle. Previously the create call defaulted to including sub-pages, silently
exposing every child of a freshly shared page. (#216)
- **Offline reading support**: opened pages, their sidebar tree, breadcrumb
children, and comments are cached in IndexedDB (TanStack Query persister plus
`y-indexeddb` for the page's Yjs document), and a PWA service worker
(vite-plugin-pwa) serves an app shell so previously opened pages stay readable
offline. The two offline stores (the persisted query cache and the Yjs page
documents) are cleared on logout AND on sign-in so a previous user's private
data does not remain in the browser; the same purge also defensively drops any
legacy service-worker `api-get-cache` left by older clients (current builds
serve `/api` as NetworkOnly, so there is no active service-worker API cache).
- **Mobile bootstrap**: a `returnToken` opt-in on login so native/mobile clients
can request the access JWT in the response body (`data.authToken`) in addition
to the httpOnly cookie (the web client stays cookie-only); an optional
OpenAPI/Swagger UI at `/api/docs` gated by `SWAGGER_ENABLED` (off by default);
and new env vars `CORS_ALLOWED_ORIGINS`, `SWAGGER_ENABLED`, `CAP_SERVER_URL`.
### Changed
- **CORS is now an explicit allowlist** (replaces the previous unconfigured
`app.enableCors()`). The same-origin web client is unaffected, but any
separately-hosted cross-domain client must now be listed in
`CORS_ALLOWED_ORIGINS` (native Capacitor/Ionic/localhost WebView origins are
allowed automatically). Requests with no `Origin` header (server-to-server)
are still allowed. **Upgrade note:** the old bare `app.enableCors()` reflected
*any* origin (with `credentials:false`), so any previously-working cross-domain
REST/browser client is now rejected until its origin is added to
`CORS_ALLOWED_ORIGINS` (see `.env.example`).
### Fixed
- **Internal links in exported Markdown no longer lose their visible text.** A

View File

@@ -17,8 +17,9 @@ RUN pnpm build
FROM base AS installer
# git: required by the git-sync VaultGit (shells out to git)
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl bash \
&& apt-get install -y --no-install-recommends curl bash git \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
@@ -38,6 +39,14 @@ COPY --from=builder /app/packages/editor-ext/dist /app/packages/editor-ext/dist
COPY --from=builder /app/packages/editor-ext/package.json /app/packages/editor-ext/package.json
COPY --from=builder /app/packages/mcp/build /app/packages/mcp/build
COPY --from=builder /app/packages/mcp/package.json /app/packages/mcp/package.json
# git-sync: the server loads @docmost/git-sync at runtime via the loader
# (git-sync.loader.ts), which deliberately does NOT `require()` it — the package is
# ESM-only, so the loader uses `require.resolve` + a dynamic `import()`. Without
# these copied build artifacts that resolve/import fails and the server crashes on
# first use. Built fresh by the builder's `pnpm build` (nx builds the package's tsc
# `build` target).
COPY --from=builder /app/packages/git-sync/build /app/packages/git-sync/build
COPY --from=builder /app/packages/git-sync/package.json /app/packages/git-sync/package.json
# Copy root package files
COPY --from=builder /app/package.json /app/package.json

View File

@@ -10,7 +10,6 @@
<meta name="theme-color" content="#0E1117" media="(prefers-color-scheme: dark)" />
<meta name="theme-color" content="#f6f7f9" media="(prefers-color-scheme: light)" />
<link rel="manifest" href="/manifest.json" />
<link rel="apple-touch-icon" href="/icons/app-icon-192x192.png" />
<meta name="mobile-web-app-capable" content="yes" />
<meta name="apple-touch-fullscreen" content="yes" />
<meta name="apple-mobile-web-app-title" content="Gitmost" />

View File

@@ -33,9 +33,7 @@
"@slidoapp/emoji-mart-data": "1.2.4",
"@slidoapp/emoji-mart-react": "1.1.5",
"@tabler/icons-react": "3.40.0",
"@tanstack/query-async-storage-persister": "5.90.17",
"@tanstack/react-query": "5.90.17",
"@tanstack/react-query-persist-client": "5.90.17",
"@tanstack/react-virtual": "3.13.24",
"ai": "6.0.207",
"alfaaz": "1.1.0",
@@ -47,7 +45,6 @@
"highlightjs-sap-abap": "0.3.0",
"i18next": "25.10.1",
"i18next-http-backend": "3.0.6",
"idb-keyval": "6.2.5",
"jotai": "2.18.1",
"jotai-optics": "0.4.0",
"js-cookie": "3.0.7",
@@ -98,7 +95,6 @@
"typescript": "5.9.3",
"typescript-eslint": "8.57.1",
"vite": "8.0.5",
"vite-plugin-pwa": "1.3.0",
"vitest": "4.1.6"
}
}

View File

@@ -464,18 +464,6 @@
"Move page": "Move page",
"Move page to a different space.": "Move page to a different space.",
"Real-time editor connection lost. Retrying...": "Real-time editor connection lost. Retrying...",
"Offline — changes are saved locally and will sync when you reconnect": "Offline — changes are saved locally and will sync when you reconnect",
"Syncing changes…": "Syncing changes…",
"All changes synced": "All changes synced",
"Update available": "Update available",
"Reload": "Reload",
"Make available offline": "Make available offline",
"Saving page for offline use...": "Saving page for offline use...",
"Page is now available offline": "Page is now available offline",
"Failed to make page available offline": "Failed to make page available offline",
"You're offline": "You're offline",
"This page hasn't been saved for offline use, so it can't be loaded right now. Reconnect to the internet and try again.": "This page hasn't been saved for offline use, so it can't be loaded right now. Reconnect to the internet and try again.",
"Retry": "Retry",
"Table of contents": "Table of contents",
"Add headings (H1, H2, H3) to generate a table of contents.": "Add headings (H1, H2, H3) to generate a table of contents.",
"Share": "Share",
@@ -1229,6 +1217,8 @@
"Ran tool {{name}}": "Ran tool {{name}}",
"AI-agent": "AI-agent",
"Edited by AI agent on behalf of {{name}}": "Edited by AI agent on behalf of {{name}}",
"Git sync": "Git sync",
"Synced from Git on behalf of {{name}}": "Synced from Git on behalf of {{name}}",
"Endpoints": "Endpoints",
"where we fetch models": "where we fetch models",
"All endpoints are OpenAI-compatible. Point the Base URL at OpenAI, OpenRouter, a local Ollama, or any self-hosted server.": "All endpoints are OpenAI-compatible. Point the Base URL at OpenAI, OpenRouter, a local Ollama, or any self-hosted server.",
@@ -1253,6 +1243,10 @@
"MCP server": "MCP server",
"expose the workspace": "expose the workspace",
"Enable MCP server": "Enable MCP server",
"Enable Git sync": "Enable Git sync",
"Sync this space's pages to a Git repository.": "Sync this space's pages to a Git repository.",
"Auto-merge conflicts on push": "Auto-merge conflicts on push",
"When off (recommended), a page whose content still has unresolved Git conflict markers is skipped on push until you resolve the conflict in Git. When on, the markers are stripped and both sides' content is pushed.": "When off (recommended), a page whose content still has unresolved Git conflict markers is skipped on push until you resolve the conflict in Git. When on, the markers are stripped and both sides' content is pushed.",
"Exposes the workspace as an MCP server at /mcp — this provides a capability, it doesn't consume a model.": "Exposes the workspace as an MCP server at /mcp — this provides a capability, it doesn't consume a model.",
"Resolves to {{url}}": "Resolves to {{url}}",
"Model": "Model",

View File

@@ -474,18 +474,6 @@
"Move page": "Переместить страницу",
"Move page to a different space.": "Переместите страницу в другое пространство.",
"Real-time editor connection lost. Retrying...": "Соединение с редактором в реальном времени потеряно. Повторная попытка...",
"Offline — changes are saved locally and will sync when you reconnect": "Нет сети — изменения сохраняются локально и синхронизируются при восстановлении соединения",
"Syncing changes…": "Синхронизация изменений…",
"All changes synced": "Все изменения синхронизированы",
"Update available": "Доступно обновление",
"Reload": "Перезагрузить",
"Make available offline": "Сделать доступным офлайн",
"Saving page for offline use...": "Сохраняем страницу для офлайн-доступа…",
"Page is now available offline": "Страница доступна офлайн",
"Failed to make page available offline": "Не удалось сделать страницу доступной офлайн",
"You're offline": "Вы офлайн",
"This page hasn't been saved for offline use, so it can't be loaded right now. Reconnect to the internet and try again.": "Эта страница не была сохранена для офлайн-доступа, поэтому её нельзя загрузить сейчас. Подключитесь к интернету и попробуйте снова.",
"Retry": "Повторить",
"Table of contents": "Оглавление",
"Add headings (H1, H2, H3) to generate a table of contents.": "Добавьте заголовки (H1, H2, H3), чтобы создать оглавление.",
"Share": "Поделиться",

View File

@@ -1,19 +1,30 @@
{
"id": "/",
"name": "Gitmost",
"short_name": "Gitmost",
"description": "Gitmost - open-source collaborative documentation and knowledge base.",
"lang": "en",
"start_url": "/",
"scope": "/",
"display": "standalone",
"orientation": "any",
"background_color": "#0E1117",
"theme_color": "#0E1117",
"icons": [
{ "src": "icons/favicon-16x16.png", "type": "image/png", "sizes": "16x16" },
{ "src": "icons/favicon-32x32.png", "type": "image/png", "sizes": "32x32" },
{ "src": "icons/app-icon-192x192.png", "type": "image/png", "sizes": "192x192", "purpose": "any" },
{ "src": "icons/app-icon-512x512.png", "type": "image/png", "sizes": "512x512", "purpose": "any" }
{
"src": "icons/favicon-16x16.png",
"type": "image/png",
"sizes": "16x16"
},
{
"src": "icons/favicon-32x32.png",
"type": "image/png",
"sizes": "32x32"
},
{
"src": "icons/app-icon-192x192.png",
"type": "image/png",
"sizes": "180x180 192x192"
},
{
"src": "icons/app-icon-512x512.png",
"type": "image/png",
"sizes": "512x512"
}
]
}

View File

@@ -0,0 +1,37 @@
import { Badge, Tooltip } from "@mantine/core";
import { IconGitMerge } from "@tabler/icons-react";
import { useTranslation } from "react-i18next";
interface GitSyncBadgeProps {
authorName?: string;
}
/**
* Badge marking a version produced by git-sync (provenance §8.1). The history
* version is created on the PUSH path — when an incoming git body is written back
* into the Docmost doc — not by the pull itself. Like {@link AiAgentBadge} it is
* ADDITIVE — shown next to the human author, never replacing them — but a git-sync
* edit is NOT an agent edit and has no chat to deep-link into, so it is a small,
* neutral, non-clickable label.
*/
export function GitSyncBadge({ authorName }: GitSyncBadgeProps) {
const { t } = useTranslation();
const tooltip = t("Synced from Git on behalf of {{name}}", {
name: authorName ?? "",
});
return (
<Tooltip label={tooltip} withArrow>
<Badge
size="sm"
variant="light"
color="gray"
radius="sm"
leftSection={<IconGitMerge size={12} stroke={2} />}
>
{t("Git sync")}
</Badge>
</Tooltip>
);
}

View File

@@ -1,154 +0,0 @@
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { renderHook, act } from "@testing-library/react";
// react-i18next: identity t() so the hook renders without an i18n provider.
vi.mock("react-i18next", () => ({
useTranslation: () => ({ t: (k: string) => k }),
}));
// react-router-dom: only useNavigate is used by the hook.
const navigateMock = vi.fn();
vi.mock("react-router-dom", () => ({
useNavigate: () => navigateMock,
}));
// The auth service is the network boundary; stub login/logout per test.
const loginMock = vi.fn();
const logoutMock = vi.fn();
vi.mock("@/features/auth/services/auth-service", () => ({
login: (...args: unknown[]) => loginMock(...args),
logout: (...args: unknown[]) => logoutMock(...args),
forgotPassword: vi.fn(),
passwordReset: vi.fn(),
setupWorkspace: vi.fn(),
verifyUserToken: vi.fn(),
}));
vi.mock("@/features/workspace/services/workspace-service.ts", () => ({
acceptInvitation: vi.fn(),
}));
// The offline cache purge is the unit under test — assert it is invoked.
const clearOfflineCacheMock = vi.fn();
vi.mock("@/features/offline/clear-offline-cache", () => ({
clearOfflineCache: () => clearOfflineCacheMock(),
}));
// app-route helpers are pure config; provide deterministic values.
vi.mock("@/lib/app-route.ts", () => ({
default: { AUTH: { LOGIN: "/login" }, HOME: "/home" },
getPostLoginRedirect: () => "/home",
}));
// Mantine notifications: avoid touching the DOM-bound notification system.
vi.mock("@mantine/notifications", () => ({
notifications: { show: vi.fn() },
}));
import useAuth from "./use-auth";
beforeEach(() => {
navigateMock.mockReset();
loginMock.mockReset();
loginMock.mockResolvedValue(undefined);
logoutMock.mockReset();
logoutMock.mockResolvedValue(undefined);
clearOfflineCacheMock.mockReset();
clearOfflineCacheMock.mockResolvedValue(undefined);
});
describe("useAuth.handleSignIn", () => {
it("clears the offline cache BEFORE logging in (cross-user leak guard)", async () => {
const order: string[] = [];
clearOfflineCacheMock.mockImplementation(async () => {
order.push("clear");
});
loginMock.mockImplementation(async () => {
order.push("login");
});
const { result } = renderHook(() => useAuth());
await act(async () => {
await result.current.signIn({ email: "b@x", password: "pw" } as any);
});
expect(clearOfflineCacheMock).toHaveBeenCalledTimes(1);
expect(loginMock).toHaveBeenCalledTimes(1);
// The purge must run before the new session's login resolves.
expect(order).toEqual(["clear", "login"]);
expect(navigateMock).toHaveBeenCalledWith("/home");
});
it("does not block sign-in when the cache purge throws (best-effort)", async () => {
clearOfflineCacheMock.mockRejectedValue(new Error("idb unavailable"));
const { result } = renderHook(() => useAuth());
await act(async () => {
await result.current.signIn({ email: "b@x", password: "pw" } as any);
});
// Login still proceeds despite the cleanup failure.
expect(loginMock).toHaveBeenCalledTimes(1);
expect(navigateMock).toHaveBeenCalledWith("/home");
});
});
describe("useAuth.handleLogout", () => {
const replaceMock = vi.fn();
let originalLocation: Location;
beforeEach(() => {
replaceMock.mockReset();
// window.location.replace is the post-logout redirect. jsdom's real `replace`
// is a non-configurable method that warns "not implemented", so swap the
// whole location object for one whose `replace` we can capture.
originalLocation = window.location;
Object.defineProperty(window, "location", {
configurable: true,
writable: true,
value: { replace: replaceMock },
});
});
afterEach(() => {
Object.defineProperty(window, "location", {
configurable: true,
writable: true,
value: originalLocation,
});
});
it("purges the offline cache exactly once BEFORE redirecting (cross-user leak guard)", async () => {
const order: string[] = [];
clearOfflineCacheMock.mockImplementation(async () => {
order.push("clear");
});
replaceMock.mockImplementation((url: string) => {
order.push(`replace:${url}`);
});
const { result } = renderHook(() => useAuth());
await act(async () => {
await result.current.logout();
});
expect(clearOfflineCacheMock).toHaveBeenCalledTimes(1);
// Purge must complete before the redirect (which would otherwise interrupt
// the async cleanup).
expect(order).toEqual(["clear", "replace:/login?logout=1"]);
});
it("still redirects when the cache purge throws (best-effort, never blocks logout)", async () => {
clearOfflineCacheMock.mockRejectedValue(new Error("idb unavailable"));
const { result } = renderHook(() => useAuth());
await act(async () => {
await result.current.logout();
});
// The thrown purge error is swallowed and the redirect still fires.
expect(clearOfflineCacheMock).toHaveBeenCalledTimes(1);
expect(replaceMock).toHaveBeenCalledTimes(1);
expect(replaceMock).toHaveBeenCalledWith("/login?logout=1");
});
});

View File

@@ -23,7 +23,6 @@ import { acceptInvitation } from "@/features/workspace/services/workspace-servic
import APP_ROUTE, { getPostLoginRedirect } from "@/lib/app-route.ts";
import { RESET } from "jotai/utils";
import { useTranslation } from "react-i18next";
import { clearOfflineCache } from "@/features/offline/clear-offline-cache";
export default function useAuth() {
const { t } = useTranslation();
@@ -34,20 +33,6 @@ export default function useAuth() {
const handleSignIn = async (data: ILogin) => {
setIsLoading(true);
// Purge any previous user's offline data BEFORE signing in (mirrors logout).
// On a shared/kiosk device the prior session may have ended WITHOUT an
// explicit logout (cookie/JWT expiry, tab close, force-quit), leaving user
// A's persisted query cache (gitmost-rq-cache) and Yjs page bodies
// (page.<id>) in IndexedDB. Without this purge user B would briefly read A's
// cached currentUser/pages/comments on first render (UserProvider serves the
// cached user) and A's page bodies would stay readable offline. Best-effort:
// never block sign-in on cache cleanup.
try {
await clearOfflineCache();
} catch {
// best-effort: never block sign-in on cache cleanup
}
try {
await login(data);
setIsLoading(false);
@@ -138,13 +123,6 @@ export default function useAuth() {
const handleLogout = async () => {
setCurrentUser(RESET);
await logout();
// Purge the previous user's offline data while the page is still alive —
// window.location.replace below would otherwise interrupt async cleanup.
try {
await clearOfflineCache();
} catch {
// best-effort: never block logout on cache cleanup
}
window.location.replace(`${APP_ROUTE.AUTH.LOGIN}?logout=1`);
};

View File

@@ -1,43 +0,0 @@
import { describe, it, expect } from "vitest";
import { AxiosError } from "axios";
import { collabTokenRetry } from "./auth-query";
// Regression for the offline white-screen (#237/#238): offline the collab-token
// POST rejects as an axios NETWORK error (isAxiosError === true but
// error.response === undefined). The old predicate read `error.response.status`
// without a guard and threw an uncaught TypeError inside the React Query retryer
// BEFORE React mounted, blanking the whole app. The predicate must stay total.
describe("collabTokenRetry", () => {
it("does NOT throw and returns a retryable value for a network error with no response (offline)", () => {
// An axios error with no `response` is exactly the offline/network-failure shape.
const networkError = new AxiosError("Network Error");
expect(networkError.response).toBeUndefined();
let result: boolean | number = false;
expect(() => {
result = collabTokenRetry(0, networkError);
}).not.toThrow();
// Network failures stay retryable (truthy), matching the original intent.
expect(result).toBe(true);
});
it("returns false (no retry) for a real 404 response", () => {
const notFound = new AxiosError("Not Found");
notFound.response = { status: 404 } as AxiosError["response"];
expect(collabTokenRetry(0, notFound)).toBe(false);
});
it("retries for a non-404 response (e.g. 500)", () => {
const serverError = new AxiosError("Server Error");
serverError.response = { status: 500 } as AxiosError["response"];
expect(collabTokenRetry(0, serverError)).toBe(true);
});
it("does not throw and retries for a non-axios error", () => {
let result: boolean | number = false;
expect(() => {
result = collabTokenRetry(0, new Error("boom"));
}).not.toThrow();
expect(result).toBe(true);
});
});

View File

@@ -3,27 +3,6 @@ import { getCollabToken, verifyUserToken } from "../services/auth-service";
import { ICollabToken, IVerifyUserToken } from "../types/auth.types";
import { isAxiosError } from "axios";
/**
* Retry predicate for the collab-token query.
*
* Offline (or any network failure) the POST rejects as an axios NETWORK error:
* `isAxiosError(error) === true` but `error.response === undefined`. Reading
* `error.response.status` without a guard threw an uncaught TypeError inside the
* React Query retryer BEFORE React mounted, white-screening the whole app on an
* offline cold boot (#237/#238). Optional-chaining `error.response?.status`
* keeps the predicate total: a network error (no response) is retryable, a real
* 404 is not. Extracted (and exported) so it can be unit-tested in isolation.
*/
export function collabTokenRetry(
_failureCount: number,
error: Error,
): boolean {
if (isAxiosError(error) && error.response?.status === 404) {
return false;
}
return true;
}
export function useVerifyUserTokenQuery(
verify: IVerifyUserToken,
): UseQueryResult<any, Error> {
@@ -43,7 +22,13 @@ export function useCollabToken(): UseQueryResult<ICollabToken, Error> {
//refetchInterval: 12 * 60 * 60 * 1000, // 12hrs
//refetchIntervalInBackground: true,
refetchOnMount: true,
retry: collabTokenRetry,
//@ts-ignore
retry: (failureCount, error) => {
if (isAxiosError(error) && error.response.status === 404) {
return false;
}
return 10;
},
retryDelay: (retryAttempt) => {
// Exponential backoff: 5s, 10s, 20s, etc.
return 5000 * Math.pow(2, retryAttempt - 1);

View File

@@ -20,7 +20,6 @@ import { notifications } from "@mantine/notifications";
import { IPagination } from "@/lib/types.ts";
import { useTranslation } from "react-i18next";
import { useEffect, useMemo } from "react";
import { offlineMutationKeys } from "@/features/offline/offline-mutations";
export const RQ_KEY = (pageId: string) => ["comments", pageId];
@@ -61,9 +60,6 @@ export function useCreateCommentMutation() {
const { t } = useTranslation();
return useMutation<IComment, Error, Partial<IComment>>({
// Stable key so a paused comment-create restored from IndexedDB after an
// offline reload finds its default mutationFn and is replayed on reconnect.
mutationKey: offlineMutationKeys.createComment,
mutationFn: (data) => createComment(data),
onSuccess: (newComment) => {
const cache = queryClient.getQueryData(

View File

@@ -10,12 +10,6 @@ export const readOnlyEditorAtom = atom<Editor | null>(null);
export const yjsConnectionStatusAtom = atom<string>("");
// Local (IndexedDB) persistence sync state for the current page's Y.Doc.
export const isLocalSyncedAtom = atom<boolean>(false);
// Remote (Hocuspocus) sync state for the current page's Y.Doc.
export const isRemoteSyncedAtom = atom<boolean>(false);
export const showLinkMenuAtom = atom(false);
// Current page's edit mode — initialized from the user's saved preference on

View File

@@ -1,21 +0,0 @@
import { createContext, useContext } from "react";
import type { HocuspocusProvider } from "@hocuspocus/provider";
import type * as Y from "yjs";
// Shared collaboration providers lifted above the title/body editors so that
// both siblings bind to the SAME Y.Doc and HocuspocusProvider. The title lives
// in a dedicated 'title' fragment of the same doc as the body.
export interface EditorProvidersContextValue {
ydoc: Y.Doc;
remote: HocuspocusProvider;
providersReady: boolean;
}
export const EditorProvidersContext =
createContext<EditorProvidersContextValue | null>(null);
// Returns the shared providers, or null when rendered outside of a provider.
// Consumers must be null-safe (the body editor falls back to a non-collab mode).
export function useEditorProviders(): EditorProvidersContextValue | null {
return useContext(EditorProvidersContext);
}

View File

@@ -34,8 +34,6 @@ import {
} from "@/features/editor/atoms/editor-atoms.ts";
import { DictationGroup } from "@/features/editor/components/fixed-toolbar/groups/dictation-group";
import { GenerateTitleGroup } from "@/features/editor/components/fixed-toolbar/groups/generate-title-group";
import { usePageCollabProviders } from "@/features/editor/hooks/use-page-collab-providers";
import { EditorProvidersContext } from "@/features/editor/contexts/editor-providers-context";
const MemoizedTitleEditor = React.memo(TitleEditor);
const MemoizedPageEditor = React.memo(PageEditor);
@@ -82,24 +80,16 @@ export function FullEditor({
// AI title generation is gated by the general AI chat flag (the same toggle
// that enables the chat agent); the server enforces it too (#199).
const isTitleGenEnabled = workspace?.settings?.ai?.chat === true;
// `user` can momentarily be null during logout teardown (the currentUser atom
// is reset before this subtree unmounts). Optional-chain every access so the
// teardown render does not throw "Cannot read properties of null (reading
// 'settings')".
const fullPageWidth = user?.settings?.preferences?.fullPageWidth;
const fullPageWidth = user.settings?.preferences?.fullPageWidth;
const editorToolbarEnabled =
user?.settings?.preferences?.editorToolbar ?? false;
user.settings?.preferences?.editorToolbar ?? false;
const [currentPageEditMode, setCurrentPageEditMode] = useAtom(
currentPageEditModeAtom,
);
const userPageEditMode =
user?.settings?.preferences?.pageEditMode ?? PageEditMode.Edit;
user.settings?.preferences?.pageEditMode ?? PageEditMode.Edit;
const isEditMode = currentPageEditMode === PageEditMode.Edit;
// Single shared Y.Doc + HocuspocusProvider for both the title and body
// editors (title lives in the 'title' fragment of the same doc).
const { ydoc, remote, providersReady } = usePageCollabProviders(pageId);
// Apply the user's saved preference only once on initial load, not on every
// page navigation — so the mode sticks across navigations within a session.
useEffect(() => {
@@ -120,32 +110,28 @@ export function FullEditor({
)}
<MemoizedDeletedPageBanner slugId={slugId} />
<MemoizedTemporaryNoteBanner slugId={slugId} />
<EditorProvidersContext.Provider
value={ydoc && remote ? { ydoc, remote, providersReady } : null}
>
<MemoizedTitleEditor
pageId={pageId}
slugId={slugId}
title={title}
spaceSlug={spaceSlug}
editable={editable}
/>
<PageByline
pageId={pageId}
creator={creator}
contributors={contributors}
editable={editable}
isEditMode={isEditMode}
isDictationEnabled={isDictationEnabled}
isTitleGenEnabled={isTitleGenEnabled}
/>
<MemoizedPageEditor
pageId={pageId}
editable={editable}
content={content}
canComment={canComment}
/>
</EditorProvidersContext.Provider>
<MemoizedTitleEditor
pageId={pageId}
slugId={slugId}
title={title}
spaceSlug={spaceSlug}
editable={editable}
/>
<PageByline
pageId={pageId}
creator={creator}
contributors={contributors}
editable={editable}
isEditMode={isEditMode}
isDictationEnabled={isDictationEnabled}
isTitleGenEnabled={isTitleGenEnabled}
/>
<MemoizedPageEditor
pageId={pageId}
editable={editable}
content={content}
canComment={canComment}
/>
</Container>
);
}

View File

@@ -1,48 +0,0 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
// jwt-decode is mocked so we can drive the four token states deterministically
// (decode success with a chosen exp, or a thrown decode error).
const decodeMock = vi.hoisted(() => vi.fn());
vi.mock("jwt-decode", () => ({
jwtDecode: decodeMock,
}));
import { collabTokenNeedsRefresh } from "./collab-token";
const NOW_MS = 1_000_000_000; // fixed "now" in ms (so NOW_MS/1000 seconds)
beforeEach(() => {
decodeMock.mockReset();
});
describe("collabTokenNeedsRefresh", () => {
it("returns true when there is no token (fetch a fresh one)", () => {
expect(collabTokenNeedsRefresh(undefined, NOW_MS)).toBe(true);
// jwtDecode must not even be called for a missing token.
expect(decodeMock).not.toHaveBeenCalled();
});
it("returns true when the token is malformed (jwtDecode throws)", () => {
decodeMock.mockImplementation(() => {
throw new Error("invalid token");
});
expect(collabTokenNeedsRefresh("garbage", NOW_MS)).toBe(true);
});
it("returns false for a valid, not-yet-expired token (no reconnect)", () => {
// exp is in the future relative to NOW.
decodeMock.mockReturnValue({ exp: NOW_MS / 1000 + 60 });
expect(collabTokenNeedsRefresh("good", NOW_MS)).toBe(false);
});
it("returns true for a valid but expired token (refresh + reconnect)", () => {
// exp is in the past relative to NOW.
decodeMock.mockReturnValue({ exp: NOW_MS / 1000 - 60 });
expect(collabTokenNeedsRefresh("expired", NOW_MS)).toBe(true);
});
it("treats exp exactly equal to now as expired (>= boundary)", () => {
decodeMock.mockReturnValue({ exp: NOW_MS / 1000 });
expect(collabTokenNeedsRefresh("boundary", NOW_MS)).toBe(true);
});
});

View File

@@ -1,26 +0,0 @@
import { jwtDecode } from "jwt-decode";
/**
* Decide whether a collab token must be refreshed before reconnecting after an
* onAuthenticationFailed event. Pure and side-effect free so the four token
* states can be unit-tested directly:
* - no token -> true (fetch a fresh one and reconnect)
* - undecodable/malformed -> true (jwtDecode throws -> refresh)
* - valid, not expired -> false (token is still good; do NOT reconnect)
* - valid, expired -> true (refresh + reconnect)
*
* `nowMs` is injectable for deterministic tests; it defaults to `Date.now()`.
*/
export function collabTokenNeedsRefresh(
token: string | undefined,
nowMs: number = Date.now(),
): boolean {
if (!token) return true;
try {
const payload = jwtDecode<{ exp: number }>(token);
return nowMs / 1000 >= payload.exp;
} catch {
// malformed/undecodable token -> refresh
return true;
}
}

View File

@@ -139,7 +139,7 @@ describe("useGeneratePageTitle", () => {
);
});
it("happy path: applies the title, refreshes cache, broadcasts, and does NOT write the editor", async () => {
it("happy path: applies the title, refreshes cache, writes the field, broadcasts", async () => {
const store = createStore();
const titleEditor = makeTitleEditor();
store.set(pageEditorAtom as never, makePageEditor("pageA"));
@@ -157,11 +157,9 @@ describe("useGeneratePageTitle", () => {
title: "Generated Title",
});
expect(updatePageDataMock).toHaveBeenCalledWith(PAGE_A);
// The title editor is bound to the Yjs `title` fragment; the server REST
// update reseeds that fragment and the reseed reaches the bound editor on
// its own. Writing here too would double/garble the title, so the hook must
// NOT touch the editor (regression guard for the Yjs duplication trap).
expect(titleEditor.commands.setContent).not.toHaveBeenCalled();
expect(titleEditor.commands.setContent).toHaveBeenCalledWith(
"Generated Title",
);
expect(localEmitMock).toHaveBeenCalled();
expect(emitMock).toHaveBeenCalled();
expect(notificationsShowMock).toHaveBeenCalledWith(
@@ -169,7 +167,7 @@ describe("useGeneratePageTitle", () => {
);
});
it("keeps the DB write keyed by the captured pageId and still broadcasts after navigation", async () => {
it("does NOT write the visible title field when the user navigated away during generation", async () => {
const store = createStore();
const titleEditor = makeTitleEditor(); // persistent across navigation
store.set(pageEditorAtom as never, makePageEditor("pageA"));
@@ -205,9 +203,55 @@ describe("useGeneratePageTitle", () => {
pageId: "pageA",
title: "Generated Title",
});
// ...the hook never writes the editor regardless of navigation...
// ...but we must NOT stamp page A's title into page B's visible field.
expect(titleEditor.commands.setContent).not.toHaveBeenCalled();
// ...and the change is still broadcast to other clients.
// The change is still broadcast to other clients.
expect(emitMock).toHaveBeenCalled();
});
it("does NOT write the visible title field when the title editor is focused", async () => {
const store = createStore();
const titleEditor = makeTitleEditor();
store.set(pageEditorAtom as never, makePageEditor("pageA"));
store.set(titleEditorAtom as never, titleEditor);
// Resolve generation under our control so we can mark the live title editor
// as focused before the post-generation write runs.
let resolveTitle!: (t: string) => void;
generatePageTitleMock.mockReturnValue(
new Promise<string>((res) => {
resolveTitle = res;
}),
);
updateTitleMock.mockResolvedValue(PAGE_A);
const { result } = setup("pageA", store);
let pending!: Promise<void>;
act(() => {
pending = result.current.mutateAsync();
});
// The user clicked into the title field while the model ran — overwriting it
// now would clobber what they are actively typing.
act(() => {
(titleEditor as { isFocused: boolean }).isFocused = true;
});
await act(async () => {
resolveTitle("Generated Title");
await pending;
});
// The DB write still persists the value...
expect(updateTitleMock).toHaveBeenCalledWith({
pageId: "pageA",
title: "Generated Title",
});
expect(updatePageDataMock).toHaveBeenCalledWith(PAGE_A);
// ...but the visible field is left alone while it is focused.
expect(titleEditor.commands.setContent).not.toHaveBeenCalled();
// The change is still broadcast to other clients.
expect(localEmitMock).toHaveBeenCalled();
expect(emitMock).toHaveBeenCalled();
});

View File

@@ -1,9 +1,13 @@
import { useRef } from "react";
import { useMutation } from "@tanstack/react-query";
import { useAtomValue } from "jotai";
import { notifications } from "@mantine/notifications";
import { useTranslation } from "react-i18next";
import { htmlToMarkdown } from "@docmost/editor-ext";
import { pageEditorAtom } from "@/features/editor/atoms/editor-atoms.ts";
import {
pageEditorAtom,
titleEditorAtom,
} from "@/features/editor/atoms/editor-atoms.ts";
import {
updatePageData,
useUpdateTitlePageMutation,
@@ -29,9 +33,18 @@ const MAX_CONTENT_CHARS = 20000;
export function useGeneratePageTitle(pageId: string) {
const { t } = useTranslation();
const pageEditor = useAtomValue(pageEditorAtom);
const titleEditor = useAtomValue(titleEditorAtom);
const { mutateAsync: updateTitle } = useUpdateTitlePageMutation();
const emit = useQueryEmit();
// The page/title editors come from GLOBAL atoms that re-point when the user
// navigates to another page. The mutation below awaits the model for 1-3s, and
// its closure captures the editors from the render that started it. Keep a live
// reference so the post-generation write targets whatever page is on screen
// *now*, not the page the generation was started from.
const editorsRef = useRef({ pageEditor, titleEditor });
editorsRef.current = { pageEditor, titleEditor };
return useMutation<void, Error, void>({
mutationFn: async () => {
if (!pageEditor || pageEditor.isDestroyed) return;
@@ -57,15 +70,33 @@ export function useGeneratePageTitle(pageId: string) {
const page = await updateTitle({ pageId, title }); // POST /pages/update
updatePageData(page); // refresh the react-query cache
// Do NOT write the title into the editor here. The title editor is bound to
// the Yjs `title` fragment and Yjs is the source of truth. The server REST
// /pages/update reseeds that fragment (writePageTitle → writeTitleFragment,
// a full clear+replace) and the reseed reaches the bound title editor on
// its own as a remote provider update. The old REST-era setContent here
// would race that reseed and double/garble the title (the "Yjs duplication
// trap"), so it is intentionally omitted. The DB write above is keyed by
// the captured `pageId`, so it stays correct even if the user navigated
// away during generation.
// Reflect the new title in the field immediately. The button lives in the
// byline, so the title editor is not focused — setContent is safe and stays
// undoable through its History extension (Ctrl/Cmd+Z reverts the change).
//
// Guard against navigation during generation: if the user switched pages
// while the model ran, the (persistent) title editor now shows ANOTHER
// page, so writing here would drop page A's title into page B's visible
// field. page-editor.tsx stamps the live page editor with its pageId
// (`editor.storage.pageId`), mirroring TitleEditor's `activePageId !==
// pageId` guard — bail the visible write unless that live editor still
// belongs to the page this title was generated for. The DB write above is
// already correct (keyed by the captured `pageId`), and the broadcast below
// still propagates page A's change to other clients.
const livePageEditor = editorsRef.current.pageEditor;
const liveTitleEditor = editorsRef.current.titleEditor;
// `storage.pageId` is stamped untyped in page-editor.tsx's onCreate.
const livePageId = (livePageEditor?.storage as { pageId?: string })
?.pageId;
const stillOnPage = livePageId === pageId;
if (
stillOnPage &&
liveTitleEditor &&
!liveTitleEditor.isDestroyed &&
!liveTitleEditor.isFocused
) {
liveTitleEditor.commands.setContent(page.title);
}
// Broadcast to other clients, mirroring TitleEditor.saveTitle's event shape.
const event: UpdateEvent = {

View File

@@ -1,189 +0,0 @@
import { useEffect, useRef, useState } from "react";
import { IndexeddbPersistence } from "y-indexeddb";
import * as Y from "yjs";
import {
HocuspocusProvider,
onStatusParameters,
WebSocketStatus,
HocuspocusProviderWebsocket,
onSyncedParameters,
onStatelessParameters,
} from "@hocuspocus/provider";
import { useAtom, useSetAtom } from "jotai";
import useCollaborationUrl from "@/features/editor/hooks/use-collaboration-url";
import {
isLocalSyncedAtom,
isRemoteSyncedAtom,
yjsConnectionStatusAtom,
} from "@/features/editor/atoms/editor-atoms";
import { useCollabToken } from "@/features/auth/queries/auth-query.tsx";
import { useDocumentVisibility } from "@mantine/hooks";
import { useIdle } from "@/hooks/use-idle.ts";
import { queryClient } from "@/main.tsx";
import { IPage } from "@/features/page/types/page.types.ts";
import { useParams } from "react-router-dom";
import { extractPageSlugId } from "@/lib";
import { FIVE_MINUTES } from "@/lib/constants.ts";
import { collabTokenNeedsRefresh } from "@/features/editor/hooks/collab-token";
import { pageYdocName } from "@/features/editor/page-ydoc-name";
import { pageKeys } from "@/features/page/queries/page-query";
export interface PageCollabProviders {
ydoc: Y.Doc | null;
remote: HocuspocusProvider | null;
socket: HocuspocusProviderWebsocket | null;
providersReady: boolean;
}
/**
* Owns the full collaboration provider lifecycle for a page so that the title
* and body editors can share a single Y.Doc + HocuspocusProvider. The behavior
* is relocated verbatim from page-editor.tsx: it creates the providers once per
* pageId, connects/disconnects on idle/visibility, attaches each render,
* destroys on unmount, refreshes the collab token on auth failure, and applies
* the onStateless 'page.updated' cache update.
*/
export function usePageCollabProviders(pageId: string): PageCollabProviders {
const collaborationURL = useCollaborationUrl();
const [yjsConnectionStatus, setYjsConnectionStatus] = useAtom(
yjsConnectionStatusAtom,
);
const setIsLocalSyncedAtom = useSetAtom(isLocalSyncedAtom);
const setIsRemoteSyncedAtom = useSetAtom(isRemoteSyncedAtom);
const { data: collabQuery, refetch: refetchCollabToken } = useCollabToken();
// The provider-creating effect runs only once per pageId, so any token read
// inside its handlers would be captured STALE (the old token at first render).
// Mirror the latest token into a ref the auth-failure handler can read live.
const collabTokenRef = useRef<string | undefined>(undefined);
useEffect(() => {
collabTokenRef.current = collabQuery?.token;
}, [collabQuery?.token]);
const { isIdle, resetIdle } = useIdle(FIVE_MINUTES, { initialState: false });
const documentState = useDocumentVisibility();
const { pageSlug } = useParams();
const slugId = extractPageSlugId(pageSlug);
// Providers only created once per pageId
const providersRef = useRef<{
ydoc: Y.Doc;
local: IndexeddbPersistence;
remote: HocuspocusProvider;
socket: HocuspocusProviderWebsocket;
} | null>(null);
const [providersReady, setProvidersReady] = useState(false);
useEffect(() => {
if (!providersRef.current) {
const documentName = pageYdocName(pageId);
const ydoc = new Y.Doc();
const local = new IndexeddbPersistence(documentName, ydoc);
const socket = new HocuspocusProviderWebsocket({
url: collaborationURL,
});
const onLocalSyncedHandler = () => {
setIsLocalSyncedAtom(true);
};
const onStatusHandler = (event: onStatusParameters) => {
setYjsConnectionStatus(event.status);
};
const onSyncedHandler = (event: onSyncedParameters) => {
setIsRemoteSyncedAtom(event.state);
};
const onStatelessHandler = ({ payload }: onStatelessParameters) => {
try {
const message = JSON.parse(payload);
if (message?.type !== "page.updated" || !message.updatedAt) return;
const pageData = queryClient.getQueryData<IPage>(
pageKeys.detail(slugId),
);
if (pageData) {
queryClient.setQueryData(pageKeys.detail(slugId), {
...pageData,
updatedAt: message.updatedAt,
...(message.lastUpdatedBy && {
lastUpdatedBy: message.lastUpdatedBy,
}),
});
}
} catch {
// ignore unrelated stateless messages
}
};
const onAuthenticationFailedHandler = () => {
// Read the token from the ref, not the closed-over `collabQuery`: this
// handler is created once and would otherwise decode a stale token after
// a refetch. A missing/malformed token must NOT crash the handler —
// jwtDecode(undefined) throws — so treat any decode failure as "needs
// refresh" and proceed to refetch + reconnect instead of getting stuck.
if (!collabTokenNeedsRefresh(collabTokenRef.current)) return;
refetchCollabToken().then((result) => {
if (result.data?.token) {
socket.disconnect();
setTimeout(() => {
remote.configuration.token = result.data.token;
socket.connect();
}, 100);
}
});
};
const remote = new HocuspocusProvider({
websocketProvider: socket,
name: documentName,
document: ydoc,
token: collabQuery?.token,
onAuthenticationFailed: onAuthenticationFailedHandler,
onStatus: onStatusHandler,
onSynced: onSyncedHandler,
onStateless: onStatelessHandler,
});
local.on("synced", onLocalSyncedHandler);
providersRef.current = { ydoc, socket, local, remote };
setProvidersReady(true);
} else {
setProvidersReady(true);
}
// Only destroy on final unmount
return () => {
providersRef.current?.socket.destroy();
providersRef.current?.remote.destroy();
providersRef.current?.local.destroy();
providersRef.current = null;
// Reset shared sync state on page change/unmount.
setIsLocalSyncedAtom(false);
setIsRemoteSyncedAtom(false);
};
}, [pageId]);
// Only connect/disconnect on tab/idle, not destroy
useEffect(() => {
if (!providersReady || !providersRef.current) return;
const socket = providersRef.current.socket;
if (
isIdle &&
documentState === "hidden" &&
yjsConnectionStatus === WebSocketStatus.Connected
) {
socket.disconnect();
return;
}
if (
documentState === "visible" &&
yjsConnectionStatus === WebSocketStatus.Disconnected
) {
resetIdle();
socket.connect();
}
}, [isIdle, documentState, providersReady, resetIdle]);
// Attach here, to make sure the connection gets properly established
providersRef.current?.remote.attach();
return {
ydoc: providersRef.current?.ydoc ?? null,
remote: providersRef.current?.remote ?? null,
socket: providersRef.current?.socket ?? null,
providersReady,
};
}

View File

@@ -6,7 +6,16 @@ import React, {
useRef,
useState,
} from "react";
import { WebSocketStatus } from "@hocuspocus/provider";
import { IndexeddbPersistence } from "y-indexeddb";
import * as Y from "yjs";
import {
HocuspocusProvider,
onStatusParameters,
WebSocketStatus,
HocuspocusProviderWebsocket,
onSyncedParameters,
onStatelessParameters,
} from "@hocuspocus/provider";
import {
Editor,
EditorContent,
@@ -19,15 +28,13 @@ import {
mainExtensions,
} from "@/features/editor/extensions/extensions";
import { useAtom, useAtomValue } from "jotai";
import useCollaborationUrl from "@/features/editor/hooks/use-collaboration-url";
import { currentUserAtom } from "@/features/user/atoms/current-user-atom";
import {
currentPageEditModeAtom,
isLocalSyncedAtom,
isRemoteSyncedAtom,
pageEditorAtom,
yjsConnectionStatusAtom,
} from "@/features/editor/atoms/editor-atoms";
import { useEditorProviders } from "@/features/editor/contexts/editor-providers-context";
import { asideStateAtom } from "@/components/layouts/global/hooks/atoms/sidebar-atom";
import {
activeCommentIdAtom,
@@ -51,8 +58,10 @@ import {
} from "@/features/editor/components/common/editor-paste-handler.tsx";
import ExcalidrawMenu from "./components/excalidraw/excalidraw-menu-lazy";
import DrawioMenu from "./components/drawio/drawio-menu";
import { useCollabToken } from "@/features/auth/queries/auth-query.tsx";
import SearchAndReplaceDialog from "@/features/editor/components/search-and-replace/search-and-replace-dialog.tsx";
import { useDebouncedCallback } from "@mantine/hooks";
import { useDebouncedCallback, useDocumentVisibility } from "@mantine/hooks";
import { useIdle } from "@/hooks/use-idle.ts";
import { queryClient } from "@/main.tsx";
import { IPage } from "@/features/page/types/page.types.ts";
import { useParams } from "react-router-dom";
@@ -63,7 +72,9 @@ import {
GitmostInsertRecordingResult,
gitmostInsertRecordingIntoEditor,
} from "@/features/editor/gitmost/gitmost-recording.ts";
import { FIVE_MINUTES } from "@/lib/constants.ts";
import { PageEditMode } from "@/features/user/types/user.types.ts";
import { jwtDecode } from "jwt-decode";
import { searchSpotlight } from "@/features/search/constants.ts";
import { useEditorScroll } from "./hooks/use-editor-scroll";
import { EditorLinkMenu } from "@/features/editor/components/link/link-menu";
@@ -92,6 +103,7 @@ export default function PageEditor({
canComment,
}: PageEditorProps) {
const { t } = useTranslation();
const collaborationURL = useCollaborationUrl();
const isComponentMounted = useRef(false);
const editorRef = useRef<Editor | null>(null);
@@ -105,10 +117,22 @@ export default function PageEditor({
const [, setActiveCommentId] = useAtom(activeCommentIdAtom);
const [showCommentPopup, setShowCommentPopup] = useAtom(showCommentPopupAtom);
const [showReadOnlyCommentPopup] = useAtom(showReadOnlyCommentPopupAtom);
const [isLocalSynced, setIsLocalSynced] = useState(false);
const [isRemoteSynced, setIsRemoteSynced] = useState(false);
const [yjsConnectionStatus, setYjsConnectionStatus] = useAtom(
yjsConnectionStatusAtom,
);
const menuContainerRef = useRef(null);
const { data: collabQuery, refetch: refetchCollabToken } = useCollabToken();
// Always holds the latest collab token. The provider effect below runs once
// per pageId, so a handler created inside it would otherwise close over a
// stale `collabQuery`. Reading the ref gives the current token instead.
const collabTokenRef = useRef<string | undefined>(undefined);
useEffect(() => {
collabTokenRef.current = collabQuery?.token;
}, [collabQuery?.token]);
const { isIdle, resetIdle } = useIdle(FIVE_MINUTES, { initialState: false });
const documentState = useDocumentVisibility();
const { pageSlug } = useParams();
const slugId = extractPageSlugId(pageSlug);
const currentPageEditMode = useAtomValue(currentPageEditModeAtom);
@@ -117,27 +141,141 @@ export default function PageEditor({
[isComponentMounted],
);
const { handleScrollTo } = useEditorScroll({ canScroll });
// Providers only created once per pageId
const providersRef = useRef<{
local: IndexeddbPersistence;
remote: HocuspocusProvider;
socket: HocuspocusProviderWebsocket;
} | null>(null);
const [providersReady, setProvidersReady] = useState(false);
// Shared providers + Y.Doc lifted into full-editor via context. The provider
// lifecycle (creation, idle/visibility connect, attach, destroy, token
// refresh) lives in usePageCollabProviders. Null-safe when rendered without
// the context (defensive) — in practice full-editor always provides it.
const editorProviders = useEditorProviders();
const remote = editorProviders?.remote ?? null;
const providersReady = editorProviders?.providersReady ?? false;
const isLocalSynced = useAtomValue(isLocalSyncedAtom);
const isRemoteSynced = useAtomValue(isRemoteSyncedAtom);
useEffect(() => {
if (!providersRef.current) {
const documentName = `page.${pageId}`;
const ydoc = new Y.Doc();
const local = new IndexeddbPersistence(documentName, ydoc);
const socket = new HocuspocusProviderWebsocket({
url: collaborationURL,
});
const onLocalSyncedHandler = () => {
setIsLocalSynced(true);
};
const onStatusHandler = (event: onStatusParameters) => {
setYjsConnectionStatus(event.status);
};
const onSyncedHandler = (event: onSyncedParameters) => {
setIsRemoteSynced(event.state);
};
const onStatelessHandler = ({ payload }: onStatelessParameters) => {
try {
const message = JSON.parse(payload);
if (message?.type !== "page.updated" || !message.updatedAt) return;
const pageData = queryClient.getQueryData<IPage>(["pages", slugId]);
if (pageData) {
queryClient.setQueryData(["pages", slugId], {
...pageData,
updatedAt: message.updatedAt,
...(message.lastUpdatedBy && {
lastUpdatedBy: message.lastUpdatedBy,
}),
});
}
} catch {
// ignore unrelated stateless messages
}
};
const onAuthenticationFailedHandler = () => {
// Read the latest token via the ref (the closure-captured `collabQuery`
// may be stale). Guard the decode: a missing or unparseable token must
// not throw "Invalid token specified" and should trigger a refresh so
// the editor reconnects even when the initial token fetch failed.
const token = collabTokenRef.current;
let needsRefresh = true; // no/unparseable token -> fetch a fresh one and reconnect
if (token) {
try {
// A token that decodes but lacks a numeric `exp` must be treated as
// expired (`Date.now()/1000 >= undefined` is `false`, which would
// otherwise skip the reconnect), so refresh on any missing/non-number exp.
const exp = jwtDecode<{ exp?: number }>(token).exp;
needsRefresh = typeof exp !== "number" || Date.now() / 1000 >= exp;
} catch {
needsRefresh = true;
}
}
if (!needsRefresh) return;
refetchCollabToken().then((result) => {
if (result.data?.token) {
socket.disconnect();
setTimeout(() => {
remote.configuration.token = result.data.token;
socket.connect();
}, 100);
}
});
};
const remote = new HocuspocusProvider({
websocketProvider: socket,
name: documentName,
document: ydoc,
token: collabQuery?.token,
onAuthenticationFailed: onAuthenticationFailedHandler,
onStatus: onStatusHandler,
onSynced: onSyncedHandler,
onStateless: onStatelessHandler,
});
local.on("synced", onLocalSyncedHandler);
providersRef.current = { socket, local, remote };
setProvidersReady(true);
} else {
setProvidersReady(true);
}
// Only destroy on final unmount
return () => {
providersRef.current?.socket.destroy();
providersRef.current?.remote.destroy();
providersRef.current?.local.destroy();
providersRef.current = null;
};
}, [pageId]);
// Only connect/disconnect on tab/idle, not destroy
useEffect(() => {
if (!providersReady || !providersRef.current) return;
const socket = providersRef.current.socket;
if (
isIdle &&
documentState === "hidden" &&
yjsConnectionStatus === WebSocketStatus.Connected
) {
socket.disconnect();
return;
}
if (
documentState === "visible" &&
yjsConnectionStatus === WebSocketStatus.Disconnected
) {
resetIdle();
socket.connect();
}
}, [isIdle, documentState, providersReady, resetIdle]);
// Attach here, to make sure the connection gets properly established
providersRef.current?.remote.attach();
const extensions = useMemo(() => {
if (!providersReady || !remote || !currentUser?.user) {
if (!providersReady || !providersRef.current || !currentUser?.user) {
return mainExtensions;
}
const remoteProvider = providersRef.current.remote;
return [
...mainExtensions,
...collabExtensions(remote, currentUser?.user),
...collabExtensions(remoteProvider, currentUser?.user),
];
}, [providersReady, remote, currentUser?.user]);
}, [providersReady, currentUser?.user]);
const editor = useEditor(
{
@@ -412,7 +550,7 @@ export default function PageEditor({
{editor &&
!editorIsEditable &&
(editable || canComment) &&
remote && <ReadonlyBubbleMenu editor={editor} />}
providersRef.current && <ReadonlyBubbleMenu editor={editor} />}
{showCommentPopup && (
<CommentDialog editor={editor} pageId={pageId} />
)}

View File

@@ -1,14 +0,0 @@
/**
* Single source of truth for the IndexedDB / Hocuspocus document name of a
* page's collaborative Yjs doc.
*
* The `page.<id>` convention is shared knowledge across three call sites: the
* live editor providers (`use-page-collab-providers`), the offline warm path
* (`make-offline`), and the offline purge (`clear-offline-cache`, which matches
* the databases to delete by this prefix). Centralizing it here stops those
* sites from silently drifting apart.
*/
export const PAGE_YDOC_NAME_PREFIX = "page.";
export const pageYdocName = (pageId: string): string =>
`${PAGE_YDOC_NAME_PREFIX}${pageId}`;

View File

@@ -1,33 +0,0 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
// isChangeOrigin is mocked so we can simulate local vs remote/collab-origin
// transactions without constructing a real ProseMirror/Yjs transaction.
const isChangeOriginMock = vi.hoisted(() => vi.fn());
vi.mock("@tiptap/extension-collaboration", () => ({
isChangeOrigin: isChangeOriginMock,
}));
import { shouldPropagateTitleChange } from "./title-collab";
beforeEach(() => {
isChangeOriginMock.mockReset();
});
describe("shouldPropagateTitleChange", () => {
it("propagates a genuine local edit (isChangeOrigin false)", () => {
isChangeOriginMock.mockReturnValue(false);
expect(shouldPropagateTitleChange({ local: true })).toBe(true);
expect(isChangeOriginMock).toHaveBeenCalledWith({ local: true });
});
it("skips a remote/collab-origin update (isChangeOrigin true)", () => {
isChangeOriginMock.mockReturnValue(true);
expect(shouldPropagateTitleChange({ remote: true })).toBe(false);
});
it("propagates when there is no transaction (treated as local)", () => {
expect(shouldPropagateTitleChange(undefined)).toBe(true);
// isChangeOrigin must not be called for a missing transaction.
expect(isChangeOriginMock).not.toHaveBeenCalled();
});
});

View File

@@ -1,19 +0,0 @@
import { isChangeOrigin } from "@tiptap/extension-collaboration";
/**
* Whether a TitleEditor `onUpdate` should drive URL + tree propagation.
*
* Only genuine LOCAL edits propagate. Remote/collab-origin Yjs updates
* (detected via `isChangeOrigin`) are skipped so a remote title change is not
* re-broadcast back, which would create a feedback loop. A missing transaction
* is treated as a local edit (propagate).
*
* Extracted as a pure helper so the skip decision is unit-testable without
* mounting the full collaborative editor.
*/
export function shouldPropagateTitleChange(transaction: unknown): boolean {
return !(
transaction &&
isChangeOrigin(transaction as Parameters<typeof isChangeOrigin>[0])
);
}

View File

@@ -1,87 +0,0 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { render, screen } from "@testing-library/react";
// Drive the fallback-vs-collaborative switch (titleReady = providersReady &&
// !!ydoc) by controlling what the editor-providers context returns.
const editorProvidersValue: { ydoc: unknown; providersReady: boolean } = {
ydoc: null,
providersReady: false,
};
vi.mock("@/features/editor/contexts/editor-providers-context", () => ({
useEditorProviders: () => editorProvidersValue,
}));
// Mock the tiptap React bindings so the test does not mount a real editor:
// useEditor returns a minimal stub and EditorContent renders a marker.
vi.mock("@tiptap/react", () => ({
useEditor: () => ({
isInitialized: true,
commands: { focus: vi.fn() },
setEditable: vi.fn(),
getText: () => "",
}),
EditorContent: () => <div data-testid="collab-editor" />,
}));
vi.mock("react-i18next", () => ({
useTranslation: () => ({ t: (k: string) => k }),
}));
const navigateMock = vi.fn();
vi.mock("react-router-dom", () => ({
useNavigate: () => navigateMock,
}));
vi.mock("@/features/websocket/use-query-emit.ts", () => ({
useQueryEmit: () => vi.fn(),
}));
// page-query transitively imports @/main.tsx; mock it to a pure stub.
vi.mock("@/features/page/queries/page-query", () => ({
updatePageData: vi.fn(),
}));
vi.mock("@/main.tsx", () => ({
queryClient: { getQueryData: vi.fn(), setQueryData: vi.fn() },
}));
import { TitleEditor } from "./title-editor";
const baseProps = {
pageId: "p1",
slugId: "slug-1",
title: "My Page Title",
spaceSlug: "space",
editable: true,
};
beforeEach(() => {
navigateMock.mockReset();
editorProvidersValue.ydoc = null;
editorProvidersValue.providersReady = false;
});
describe("TitleEditor fallback vs collaborative switch", () => {
it("renders a static <h1> with the title before the shared doc is ready", () => {
editorProvidersValue.ydoc = null;
editorProvidersValue.providersReady = false;
render(<TitleEditor {...baseProps} />);
const heading = screen.getByRole("heading", { level: 1 });
expect(heading.textContent).toBe("My Page Title");
// The collaborative editor must NOT mount until the doc is ready.
expect(screen.queryByTestId("collab-editor")).toBeNull();
});
it("renders the collaborative editor once the shared doc is ready", () => {
editorProvidersValue.ydoc = {}; // truthy shared doc
editorProvidersValue.providersReady = true;
render(<TitleEditor {...baseProps} />);
expect(screen.getByTestId("collab-editor")).toBeDefined();
// The static fallback <h1> is gone — Yjs is the single source of truth and
// the prop is never seeded into the collaborative editor.
expect(screen.queryByRole("heading", { level: 1 })).toBeNull();
});
});

View File

@@ -1,5 +1,5 @@
import "@/features/editor/styles/index.css";
import { useEffect } from "react";
import React, { useCallback, useEffect, useState } from "react";
import { EditorContent, useEditor } from "@tiptap/react";
import { Document } from "@tiptap/extension-document";
import { Heading } from "@tiptap/extension-heading";
@@ -11,11 +11,14 @@ import {
pageEditorAtom,
titleEditorAtom,
} from "@/features/editor/atoms/editor-atoms";
import { pageKeys, updatePageData } from "@/features/page/queries/page-query";
import {
updatePageData,
useUpdateTitlePageMutation,
} from "@/features/page/queries/page-query";
import { useDebouncedCallback, getHotkeyHandler } from "@mantine/hooks";
import { useAtom } from "jotai";
import { Collaboration } from "@tiptap/extension-collaboration";
import { shouldPropagateTitleChange } from "@/features/editor/title-collab";
import { useQueryEmit } from "@/features/websocket/use-query-emit.ts";
import { History } from "@tiptap/extension-history";
import { buildPageUrl } from "@/features/page/page.utils.ts";
import { useNavigate } from "react-router-dom";
import { useTranslation } from "react-i18next";
@@ -25,9 +28,6 @@ import localEmitter from "@/lib/local-emitter.ts";
import { PageEditMode } from "@/features/user/types/user.types.ts";
import { searchSpotlight } from "@/features/search/constants.ts";
import { platformModifierKey } from "@/lib";
import { useEditorProviders } from "@/features/editor/contexts/editor-providers-context";
import { queryClient } from "@/main.tsx";
import { IPage } from "@/features/page/types/page.types.ts";
export interface TitleEditorProps {
pageId: string;
@@ -45,82 +45,65 @@ export function TitleEditor({
editable,
}: TitleEditorProps) {
const { t } = useTranslation();
const { mutateAsync: updateTitlePageMutationAsync } =
useUpdateTitlePageMutation();
const pageEditor = useAtomValue(pageEditorAtom);
const [, setTitleEditor] = useAtom(titleEditorAtom);
const emit = useQueryEmit();
const navigate = useNavigate();
const [activePageId, setActivePageId] = useState(pageId);
const currentPageEditMode = useAtomValue(currentPageEditModeAtom);
// Shared Y.Doc (title lives in its own 'title' fragment of the same doc as
// the body). Yjs is the source of truth for the title content.
const editorProviders = useEditorProviders();
const ydoc = editorProviders?.ydoc ?? null;
const providersReady = editorProviders?.providersReady ?? false;
// Until the shared doc is ready, the collaborative editor binds nothing and
// would render an empty heading until the Yjs 'title' fragment hydrates. Show
// a non-editable static <h1> with the `title` prop in the meantime. The prop
// is NEVER fed into the collaborative editor (Yjs stays the single source of
// truth — seeding it would duplicate the title).
const titleReady = providersReady && !!ydoc;
const titleEditor = useEditor(
{
extensions: [
Document.extend({
content: "heading",
}),
Heading.configure({
levels: [1],
}),
Text,
Placeholder.configure({
placeholder: t("Untitled"),
showOnlyWhenEditable: false,
}),
// Bind the title to the dedicated 'title' fragment of the shared doc.
// Collaboration also manages undo/redo, so the History extension is
// intentionally omitted (it would conflict with Yjs). When the doc is
// not ready yet the editor renders empty until the doc arrives.
...(ydoc
? [Collaboration.configure({ document: ydoc, field: "title" })]
: []),
EmojiCommand,
],
onCreate({ editor }) {
if (editor) {
// @ts-ignore
setTitleEditor(editor);
}
const titleEditor = useEditor({
extensions: [
Document.extend({
content: "heading",
}),
Heading.configure({
levels: [1],
}),
Text,
Placeholder.configure({
placeholder: t("Untitled"),
showOnlyWhenEditable: false,
}),
History.configure({
depth: 20,
}),
EmojiCommand,
],
onCreate({ editor }) {
if (editor) {
// @ts-ignore
setTitleEditor(editor);
setActivePageId(pageId);
}
},
onUpdate({ editor }) {
debounceUpdate();
},
editable: editable,
content: title,
immediatelyRender: true,
shouldRerenderOnTransaction: false,
editorProps: {
attributes: {
"aria-label": t("Page title"),
},
onUpdate({ editor, transaction }) {
// Drive URL + tree propagation only on genuine local edits; skip
// remote/collab-origin Yjs updates to avoid feedback loops.
if (!shouldPropagateTitleChange(transaction)) return;
debouncedPropagateTitle(editor.getText());
},
editable: editable,
immediatelyRender: true,
shouldRerenderOnTransaction: false,
editorProps: {
attributes: {
"aria-label": t("Page title"),
},
handleDOMEvents: {
keydown: (_view, event) => {
if (platformModifierKey(event) && event.code === "KeyS") {
event.preventDefault();
return true;
}
if (platformModifierKey(event) && event.code === "KeyK") {
searchSpotlight.open();
return true;
}
},
handleDOMEvents: {
keydown: (_view, event) => {
if (platformModifierKey(event) && event.code === "KeyS") {
event.preventDefault();
return true;
}
if (platformModifierKey(event) && event.code === "KeyK") {
searchSpotlight.open();
return true;
}
},
},
},
[pageId, ydoc],
);
});
useEffect(() => {
const anchorId = window.location.hash
@@ -130,45 +113,59 @@ export function TitleEditor({
navigate(pageSlug, { replace: true });
}, [title]);
// On a local title change: update the URL slug and propagate the change to
// the live tree/breadcrumbs for online users. No REST round-trip — the title
// itself is persisted through Yjs. Offline this simply no-ops the socket
// emit and the title syncs on reconnect.
const debouncedPropagateTitle = useDebouncedCallback((titleText: string) => {
const anchorId = window.location.hash
? window.location.hash.substring(1)
: undefined;
navigate(buildPageUrl(spaceSlug, slugId, titleText, anchorId), {
replace: true,
const saveTitle = useCallback(() => {
if (!titleEditor || activePageId !== pageId) return;
if (
titleEditor.getText() === title ||
(titleEditor.getText() === "" && title === null)
) {
return;
}
updateTitlePageMutationAsync({
pageId: pageId,
title: titleEditor.getText(),
}).then((page) => {
const event: UpdateEvent = {
operation: "updateOne",
spaceId: page.spaceId,
entity: ["pages"],
id: page.id,
payload: {
title: page.title,
slugId: page.slugId,
parentPageId: page.parentPageId,
icon: page.icon,
},
};
if (page.title !== titleEditor.getText()) return;
updatePageData(page);
localEmitter.emit("message", event);
emit(event);
});
}, [pageId, title, titleEditor]);
const page =
queryClient.getQueryData<IPage>(pageKeys.detail(slugId)) ??
queryClient.getQueryData<IPage>(pageKeys.detail(pageId));
if (!page) return;
const debounceUpdate = useDebouncedCallback(saveTitle, 500);
const updatedPage: IPage = { ...page, title: titleText };
const event: UpdateEvent = {
operation: "updateOne",
spaceId: page.spaceId,
entity: ["pages"],
id: page.id,
payload: {
title: titleText,
slugId: page.slugId,
parentPageId: page.parentPageId,
icon: page.icon,
},
};
updatePageData(updatedPage);
// Drive the local (same-tab) tree/breadcrumb update. The cross-user tree
// refresh is handled server-side: the collab process extracts the renamed
// 'title' Yjs fragment and broadcasts a treeUpdate. The previous socket
// `emit(event)` here was a no-op (the gateway ignores it) and was removed.
localEmitter.emit("message", event);
}, 500);
useEffect(() => {
// Do not overwrite the title while the user is actively editing it. The
// server rebroadcasts PAGE_UPDATED to the author too, and that echo can
// carry a title that lags behind what the user has just typed; resetting
// content from it here would drop in-progress characters and jump the
// cursor. Apply external title changes only when the field is not focused.
if (
titleEditor &&
!titleEditor.isDestroyed &&
!titleEditor.isFocused &&
title !== titleEditor.getText()
) {
titleEditor.commands.setContent(title);
}
}, [pageId, title, titleEditor]);
useEffect(() => {
setTimeout(() => {
@@ -178,6 +175,13 @@ export function TitleEditor({
}, 300);
}, [titleEditor]);
useEffect(() => {
return () => {
// force-save title on navigation
saveTitle();
};
}, [pageId]);
useEffect(() => {
if (!titleEditor) return;
titleEditor.setEditable(editable && currentPageEditMode === PageEditMode.Edit);
@@ -244,22 +248,16 @@ export function TitleEditor({
return (
<div className="page-title">
{titleReady ? (
<EditorContent
editor={titleEditor}
onKeyDown={(event) => {
// First handle the search hotkey
getHotkeyHandler([["mod+F", openSearchDialog]])(event);
<EditorContent
editor={titleEditor}
onKeyDown={(event) => {
// First handle the search hotkey
getHotkeyHandler([["mod+F", openSearchDialog]])(event);
// Then handle other key events
handleTitleKeyDown(event);
}}
/>
) : (
// Static, non-editable fallback so the title is visible before Yjs
// hydrates the 'title' fragment. Not wired into the collaborative editor.
<h1>{title}</h1>
)}
// Then handle other key events
handleTitleKeyDown(event);
}}
/>
</div>
);
}

View File

@@ -3,8 +3,6 @@ import { IconHourglass, IconPlus } from "@tabler/icons-react";
import { ReactNode } from "react";
import { useNavigate } from "react-router-dom";
import { useTranslation } from "react-i18next";
import { onlineManager } from "@tanstack/react-query";
import { notifications } from "@mantine/notifications";
import { useGetSpacesQuery } from "@/features/space/queries/space-query.ts";
import { useCreatePageMutation } from "@/features/page/queries/page-query.ts";
import { buildPageUrl } from "@/features/page/page.utils.ts";
@@ -38,39 +36,21 @@ function CreateNoteButton({
const createPageMutation = useCreatePageMutation();
const createNote = async (space: ISpace) => {
// `spaceId`/`temporary` are accepted by the create-page endpoint but are
// not part of the shared `IPageInput` type; cast to satisfy the mutation
// signature.
const variables = {
spaceId: space.id,
...(temporary ? { temporary: true } : {}),
} as any;
if (!onlineManager.isOnline()) {
// Offline: the create is PAUSED and queued — its promise will not resolve
// until we are back online, so awaiting it here would spin the button
// forever. Fire it without awaiting (it persists and replays on reconnect)
// and tell the user it was saved offline instead of leaving a dead spinner.
createPageMutation.mutate(variables);
notifications.show({
color: "blue",
message: t("You're offline. This note will be created once you reconnect."),
});
return;
}
try {
const createdPage = await createPageMutation.mutateAsync(variables);
// `spaceId`/`temporary` are accepted by the create-page endpoint but are
// not part of the shared `IPageInput` type; cast to satisfy the mutation
// signature.
const createdPage = await createPageMutation.mutateAsync({
spaceId: space.id,
...(temporary ? { temporary: true } : {}),
} as any);
navigate(buildPageUrl(space.slug, createdPage.slugId, createdPage.title));
} catch {
// useCreatePageMutation already surfaces a red notification on error.
}
};
// A paused (offline) mutation stays `isPending`, so gate the spinner on it NOT
// being paused — otherwise the button would spin forever after an offline
// create. The offline path above gives its own "saved offline" feedback.
const isPending = createPageMutation.isPending && !createPageMutation.isPaused;
const isPending = createPageMutation.isPending;
// Exactly one writable space → create directly, no picker needed.
if (writableSpaces.length === 1) {

View File

@@ -1,107 +0,0 @@
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
// vi.mock factories are hoisted above imports, so the spies they reference must
// be declared via vi.hoisted (also hoisted). These are inspected by assertions.
const h = vi.hoisted(() => ({
clear: vi.fn(),
del: vi.fn(),
}));
// The module under test imports the app entry at load time — it must be mocked.
vi.mock("@/main.tsx", () => ({
queryClient: { clear: h.clear },
}));
vi.mock("idb-keyval", () => ({
del: h.del,
}));
import { clearOfflineCache } from "./clear-offline-cache";
import { OFFLINE_CACHE_KEY } from "./query-persister";
// jsdom does not provide indexedDB.databases() or Cache Storage, so the browser
// globals are stubbed per-test. We restore them afterwards.
const originalIndexedDB = (globalThis as any).indexedDB;
const originalCaches = (globalThis as any).caches;
beforeEach(() => {
h.clear.mockClear();
h.del.mockClear();
});
afterEach(() => {
(globalThis as any).indexedDB = originalIndexedDB;
(globalThis as any).caches = originalCaches;
vi.restoreAllMocks();
});
describe("clearOfflineCache", () => {
it("resolves without throwing when the browser globals are absent", async () => {
(globalThis as any).indexedDB = undefined;
delete (globalThis as any).caches;
await expect(clearOfflineCache()).resolves.toBeUndefined();
// The two store-agnostic steps still run.
expect(h.clear).toHaveBeenCalledTimes(1);
expect(h.del).toHaveBeenCalledWith(OFFLINE_CACHE_KEY);
});
it("deletes only `page.*` IndexedDB databases and only `api-get-cache` caches", async () => {
const deleteDatabase = vi.fn((_name: string) => {
const request: any = {};
// Resolve the deletion on the next microtask, like a real IDBRequest.
queueMicrotask(() => request.onsuccess && request.onsuccess());
return request;
});
(globalThis as any).indexedDB = {
databases: vi
.fn()
.mockResolvedValue([
{ name: "page.aaa" },
{ name: "page.bbb" },
{ name: "keyval-store" },
{ name: undefined },
]),
deleteDatabase,
};
const cacheDelete = vi.fn().mockResolvedValue(true);
(globalThis as any).caches = {
keys: vi
.fn()
.mockResolvedValue([
"workbox-runtime-https://app/api-get-cache",
"other-cache",
]),
delete: cacheDelete,
};
await expect(clearOfflineCache()).resolves.toBeUndefined();
// Only the two page.* databases are deleted.
expect(deleteDatabase).toHaveBeenCalledTimes(2);
expect(deleteDatabase).toHaveBeenCalledWith("page.aaa");
expect(deleteDatabase).toHaveBeenCalledWith("page.bbb");
// Only the api-get-cache entry is deleted.
expect(cacheDelete).toHaveBeenCalledTimes(1);
expect(cacheDelete).toHaveBeenCalledWith(
"workbox-runtime-https://app/api-get-cache",
);
});
it("never throws even if a step rejects (best-effort)", async () => {
h.del.mockRejectedValueOnce(new Error("idb boom"));
(globalThis as any).indexedDB = {
databases: vi.fn().mockRejectedValue(new Error("databases boom")),
deleteDatabase: vi.fn(),
};
(globalThis as any).caches = {
keys: vi.fn().mockRejectedValue(new Error("caches boom")),
delete: vi.fn(),
};
await expect(clearOfflineCache()).resolves.toBeUndefined();
expect(h.clear).toHaveBeenCalledTimes(1);
});
});

View File

@@ -1,112 +0,0 @@
import { del } from "idb-keyval";
import { queryClient } from "@/main.tsx";
import {
OFFLINE_CACHE_KEY,
freezeOfflinePersistence,
unfreezeOfflinePersistence,
} from "./query-persister";
import { PAGE_YDOC_NAME_PREFIX } from "@/features/editor/page-ydoc-name";
/**
* Best-effort purge of all of the current user's offline data from the browser.
*
* On logout the previous user's private data would otherwise linger locally and
* be readable by the next person on the device. This clears the three offline
* stores the app writes:
* 1. the in-memory + IndexedDB-persisted TanStack Query cache (idb-keyval key
* `OFFLINE_CACHE_KEY`),
* 2. the Yjs page documents (IndexedDB databases named `page.<id>` created by
* y-indexeddb in make-offline.ts), and
* 3. any legacy service worker `api-get-cache` Cache Storage entry. The
* Workbox runtime no longer creates this cache (the GET /api NetworkFirst
* rule was removed — offline reads come from the persisted RQ cache), so
* this is now a defensive cleanup for caches left by older app versions.
*
* Fully best-effort: every step is isolated so a single failure neither blocks
* the remaining steps nor throws to the caller (logout must never be blocked on
* cache cleanup). Callers may ignore the resolved value.
*
* Limitations:
* - Deleting the Yjs page databases relies on `indexedDB.databases()`, which
* is unavailable in some browsers (notably Firefox). There we skip silently;
* those `page.<id>` databases are then left in place.
* - Cache Storage clearing only runs where `caches` exists (secure contexts /
* service-worker-capable browsers).
*/
export async function clearOfflineCache(): Promise<void> {
// Freeze the throttled persister BEFORE touching the cache so the
// queryClient.clear() below cannot trigger a late re-write of the (still
// nearly-full) dehydrated snapshot after we del() the key — which would
// otherwise resurrect the previous user's persisted data in IndexedDB.
// Re-enabled in `finally` so the next (sign-in) session persists normally.
freezeOfflinePersistence();
try {
// 1a. Drop the in-memory query cache immediately.
try {
queryClient.clear();
} catch {
// best-effort: ignore in-memory cache reset failures
}
// 1b. Delete the persisted RQ cache from IndexedDB.
try {
await del(OFFLINE_CACHE_KEY);
} catch {
// best-effort: ignore persisted-cache deletion failures
}
// 2. Delete the Yjs page IndexedDB databases (`page.<id>`).
// `indexedDB.databases()` is not implemented everywhere (e.g. Firefox); when
// it is missing we cannot enumerate the page databases, so we skip silently.
try {
if (
typeof indexedDB !== "undefined" &&
typeof indexedDB.databases === "function"
) {
const dbs = await indexedDB.databases();
for (const db of dbs) {
const name = db?.name;
if (typeof name !== "string" || !name.startsWith(PAGE_YDOC_NAME_PREFIX))
continue;
try {
// Fire-and-forget delete; await a thin wrapper so a slow delete does
// not race the page teardown, but never reject on it.
await new Promise<void>((resolve) => {
const request = indexedDB.deleteDatabase(name);
request.onsuccess = () => resolve();
request.onerror = () => resolve();
request.onblocked = () => resolve();
});
} catch {
// best-effort per database
}
}
}
} catch {
// best-effort: ignore enumeration/deletion failures
}
// 3. Clear any legacy service worker API cache. Current builds no longer
// create it, but an older client may have left an "api-get-cache" entry
// (Workbox may prefix the name), so match by substring rather than exact name.
try {
if ("caches" in window) {
const keys = await caches.keys();
await Promise.all(
keys
.filter((key) => key.includes("api-get-cache"))
.map((key) => caches.delete(key)),
);
}
} catch {
// best-effort: ignore Cache Storage failures
}
} finally {
// Re-enable persistence for the next session (sign-in continues running in
// the same tab; logout reloads via window.location.replace, so this is a
// harmless no-op there).
unfreezeOfflinePersistence();
}
}

View File

@@ -1,373 +0,0 @@
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
// vi.mock factories are hoisted above imports, so any spy they reference must be
// declared with vi.hoisted (which is hoisted as well). These shared spies are
// inspected by the assertions below.
const h = vi.hoisted(() => ({
ydocDestroy: vi.fn(),
idbDestroy: vi.fn(),
providerOn: vi.fn(),
providerOff: vi.fn(),
providerDestroy: vi.fn(),
}));
// The module under test imports the app entry at load time — it must be mocked.
vi.mock("@/main.tsx", () => ({
queryClient: { setQueryData: vi.fn(), prefetchQuery: vi.fn() },
}));
vi.mock("@/features/page/services/page-service", () => ({
getPageById: vi.fn(),
getPageBreadcrumbs: vi.fn(),
getSidebarPages: vi.fn(),
getAllSidebarPages: vi.fn(),
}));
vi.mock("@/features/space/services/space-service.ts", () => ({
getSpaceById: vi.fn(),
}));
vi.mock("@/features/comment/services/comment-service", () => ({
getPageComments: vi.fn(),
}));
// Use the `function` form (not an arrow) so Vitest binds the constructor return
// value when the module under test calls `new Y.Doc()` etc.
vi.mock("yjs", () => ({
Doc: vi.fn(function () {
return { destroy: h.ydocDestroy };
}),
}));
vi.mock("y-indexeddb", () => ({
IndexeddbPersistence: vi.fn(function () {
return { destroy: h.idbDestroy };
}),
}));
vi.mock("@hocuspocus/provider", () => ({
HocuspocusProvider: vi.fn(function () {
return { on: h.providerOn, off: h.providerOff, destroy: h.providerDestroy };
}),
}));
import {
warmInfiniteAll,
warmPageYdoc,
makePageAvailableOffline,
} from "./make-offline";
import { queryClient } from "@/main.tsx";
import {
getPageById,
getPageBreadcrumbs,
getSidebarPages,
} from "@/features/page/services/page-service";
import { getPageComments } from "@/features/comment/services/comment-service";
const setQueryData = (queryClient as any).setQueryData as ReturnType<
typeof vi.fn
>;
const prefetchQuery = (queryClient as any).prefetchQuery as ReturnType<
typeof vi.fn
>;
beforeEach(() => {
// Clear call history WITHOUT wiping the mock implementations the vi.mock
// factories installed (vi.clearAllMocks would drop the constructor return
// objects and break the provider/idb/yjs spies).
setQueryData.mockClear();
prefetchQuery.mockReset();
prefetchQuery.mockResolvedValue(undefined);
(getPageById as ReturnType<typeof vi.fn>).mockReset();
(getPageBreadcrumbs as ReturnType<typeof vi.fn>).mockReset();
(getSidebarPages as ReturnType<typeof vi.fn>).mockReset();
(getPageComments as ReturnType<typeof vi.fn>).mockReset();
h.ydocDestroy.mockClear();
h.idbDestroy.mockClear();
h.providerOn.mockClear();
h.providerOff.mockClear();
h.providerDestroy.mockClear();
});
describe("warmInfiniteAll", () => {
it("warms a single page and writes the InfiniteData cache shape", async () => {
const res = { items: [{ id: 1 }], meta: { nextCursor: null } };
const fetchPage = vi.fn().mockResolvedValue(res);
await warmInfiniteAll(["comments", "p1"], fetchPage);
expect(fetchPage).toHaveBeenCalledTimes(1);
expect(fetchPage).toHaveBeenCalledWith(undefined);
expect(setQueryData).toHaveBeenCalledTimes(1);
expect(setQueryData).toHaveBeenCalledWith(["comments", "p1"], {
pages: [res],
pageParams: [undefined],
});
});
it("walks the cursor chain across multiple pages", async () => {
const r0 = { items: [], meta: { nextCursor: "c1" } };
const r1 = { items: [], meta: { nextCursor: "c2" } };
const r2 = { items: [], meta: { nextCursor: null } };
const fetchPage = vi
.fn()
.mockResolvedValueOnce(r0)
.mockResolvedValueOnce(r1)
.mockResolvedValueOnce(r2);
await warmInfiniteAll(["comments", "p1"], fetchPage);
expect(fetchPage).toHaveBeenCalledTimes(3);
expect(fetchPage.mock.calls.map((c) => c[0])).toEqual([
undefined,
"c1",
"c2",
]);
const payload = setQueryData.mock.calls[0][1];
expect(payload.pages).toEqual([r0, r1, r2]);
expect(payload.pageParams).toEqual([undefined, "c1", "c2"]);
});
it("caps pagination at maxPages and reports the truncation (returns false)", async () => {
// Always returns a non-null cursor — the cap is the only thing that stops it.
const fetchPage = vi
.fn()
.mockResolvedValue({ items: [], meta: { nextCursor: "more" } });
const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {});
// Hitting maxPages with a cursor still pending is a truncated warm: the
// (partial) cache is still written, but the result is reported as false.
await expect(
warmInfiniteAll(["comments", "p1"], fetchPage, 2),
).resolves.toBe(false);
expect(fetchPage).toHaveBeenCalledTimes(2);
const payload = setQueryData.mock.calls[0][1];
expect(payload.pages).toHaveLength(2);
expect(errorSpy).toHaveBeenCalled();
errorSpy.mockRestore();
});
it("returns true on success", async () => {
const fetchPage = vi
.fn()
.mockResolvedValue({ items: [], meta: { nextCursor: null } });
await expect(
warmInfiniteAll(["comments", "p1"], fetchPage),
).resolves.toBe(true);
});
it("reports errors (returns false) and never writes the cache on failure", async () => {
const fetchPage = vi.fn().mockRejectedValue(new Error("network"));
const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {});
await expect(
warmInfiniteAll(["comments", "p1"], fetchPage),
).resolves.toBe(false);
expect(setQueryData).not.toHaveBeenCalled();
expect(errorSpy).toHaveBeenCalled();
errorSpy.mockRestore();
});
});
describe("makePageAvailableOffline", () => {
const okPage = {
id: "uuid-1",
slugId: "slug-1",
space: { slug: "space-slug" },
};
it("returns ok:true with no failures when every step succeeds", async () => {
(getPageById as ReturnType<typeof vi.fn>).mockResolvedValue(okPage);
(getPageBreadcrumbs as ReturnType<typeof vi.fn>).mockResolvedValue([]);
(getSidebarPages as ReturnType<typeof vi.fn>).mockResolvedValue({
items: [],
meta: { nextCursor: null },
});
(getPageComments as ReturnType<typeof vi.fn>).mockResolvedValue({
items: [],
meta: { nextCursor: null },
});
const result = await makePageAvailableOffline({
pageId: "uuid-1",
spaceId: "space-uuid",
});
expect(result).toEqual({ ok: true, failed: [] });
});
it("returns ok:false with the failed step label when a warm step fails", async () => {
const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {});
(getPageById as ReturnType<typeof vi.fn>).mockResolvedValue(okPage);
(getPageBreadcrumbs as ReturnType<typeof vi.fn>).mockResolvedValue([]);
(getSidebarPages as ReturnType<typeof vi.fn>).mockResolvedValue({
items: [],
meta: { nextCursor: null },
});
// Comments warm fails -> labeled "comments".
(getPageComments as ReturnType<typeof vi.fn>).mockRejectedValue(
new Error("network"),
);
const result = await makePageAvailableOffline({
pageId: "uuid-1",
spaceId: "space-uuid",
});
expect(result.ok).toBe(false);
expect(result.failed).toContain("comments");
expect(errorSpy).toHaveBeenCalled();
errorSpy.mockRestore();
});
// Helper: the page-ids passed to the sidebar-children warm (its query key is
// ["sidebar-pages", { pageId, spaceId }]) — i.e. which nodes were prefetched.
const warmedSidebarIds = () =>
prefetchQuery.mock.calls
.map((c) => c[0])
.filter((opts: any) => opts?.queryKey?.[0] === "sidebar-pages")
.map((opts: any) => opts.queryKey[1]?.pageId);
it("warms the page + every ancestor's children once and skips the self-ancestor guard", async () => {
(getPageById as ReturnType<typeof vi.fn>).mockResolvedValue(okPage);
// Breadcrumbs include two real ancestors, the page's OWN id (must be skipped
// by the ancestorId === pageId guard so it is not warmed twice), and a
// malformed entry with no id (also skipped).
(getPageBreadcrumbs as ReturnType<typeof vi.fn>).mockResolvedValue([
{ id: "anc-1" },
{ id: "uuid-1" }, // === pageId -> guard
{ id: "anc-2" },
{}, // no id -> skipped
]);
(getSidebarPages as ReturnType<typeof vi.fn>).mockResolvedValue({
items: [],
meta: { nextCursor: null },
});
(getPageComments as ReturnType<typeof vi.fn>).mockResolvedValue({
items: [],
meta: { nextCursor: null },
});
const result = await makePageAvailableOffline({
pageId: "uuid-1",
spaceId: "space-uuid",
});
const ids = warmedSidebarIds();
// The page's own children (warmSidebarChildren(pageId)) plus each real
// ancestor — exactly once each. The self-ancestor (uuid-1 in breadcrumbs) is
// NOT a second warm: uuid-1 appears once (from the page's own children call).
expect(ids).toEqual(["uuid-1", "anc-1", "anc-2"]);
expect(ids.filter((id: string) => id === "uuid-1")).toHaveLength(1);
expect(result).toEqual({ ok: true, failed: [] });
});
it("dedupes repeated tree failures into a single 'tree' label", async () => {
const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {});
(getPageById as ReturnType<typeof vi.fn>).mockResolvedValue(okPage);
(getPageBreadcrumbs as ReturnType<typeof vi.fn>).mockResolvedValue([
{ id: "anc-1" },
{ id: "anc-2" },
]);
(getSidebarPages as ReturnType<typeof vi.fn>).mockResolvedValue({
items: [],
meta: { nextCursor: null },
});
(getPageComments as ReturnType<typeof vi.fn>).mockResolvedValue({
items: [],
meta: { nextCursor: null },
});
// Fail ONLY the sidebar-children prefetches (page-own + both ancestors = 3
// failures); the currentUser/space prefetches still resolve.
prefetchQuery.mockImplementation(async (opts: any) => {
if (opts?.queryKey?.[0] === "sidebar-pages") throw new Error("network");
return undefined;
});
const result = await makePageAvailableOffline({
pageId: "uuid-1",
spaceId: "space-uuid",
});
// Three node warms failed but the contract collapses them to one "tree".
expect(result.ok).toBe(false);
expect(result.failed).toEqual(["tree"]);
expect(errorSpy).toHaveBeenCalled();
errorSpy.mockRestore();
});
it("records 'breadcrumbs' (not 'tree') when the breadcrumbs lookup rejects", async () => {
const errorSpy = vi.spyOn(console, "error").mockImplementation(() => {});
(getPageById as ReturnType<typeof vi.fn>).mockResolvedValue(okPage);
// Ancestor discovery fails -> the ancestor-walk is recorded as "breadcrumbs".
(getPageBreadcrumbs as ReturnType<typeof vi.fn>).mockRejectedValue(
new Error("network"),
);
(getSidebarPages as ReturnType<typeof vi.fn>).mockResolvedValue({
items: [],
meta: { nextCursor: null },
});
(getPageComments as ReturnType<typeof vi.fn>).mockResolvedValue({
items: [],
meta: { nextCursor: null },
});
const result = await makePageAvailableOffline({
pageId: "uuid-1",
spaceId: "space-uuid",
});
// The page's own children still warmed fine (prefetch resolves), so the only
// failure is the breadcrumbs lookup.
expect(result.ok).toBe(false);
expect(result.failed).toEqual(["breadcrumbs"]);
expect(errorSpy).toHaveBeenCalled();
errorSpy.mockRestore();
});
});
describe("warmPageYdoc", () => {
afterEach(() => {
vi.useRealTimers();
});
it("resolves on synced, detaches the listener once, and tears everything down (settle-once)", async () => {
const promise = warmPageYdoc("p1", "ws://x");
// Grab the synced handler the provider registered.
expect(h.providerOn).toHaveBeenCalledWith("synced", expect.any(Function));
const handler = h.providerOn.mock.calls.find(
(c) => c[0] === "synced",
)![1] as () => void;
handler();
await expect(promise).resolves.toBeUndefined();
// Listener detached and everything cleaned up.
expect(h.providerOff).toHaveBeenCalledWith("synced", expect.any(Function));
expect(h.providerDestroy).toHaveBeenCalledTimes(1);
expect(h.idbDestroy).toHaveBeenCalledTimes(1);
expect(h.ydocDestroy).toHaveBeenCalledTimes(1);
// Firing the handler again must NOT re-run cleanup (settled guard).
handler();
expect(h.providerDestroy).toHaveBeenCalledTimes(1);
expect(h.idbDestroy).toHaveBeenCalledTimes(1);
expect(h.ydocDestroy).toHaveBeenCalledTimes(1);
});
it("resolves and cleans up after the timeout when synced never fires", async () => {
vi.useFakeTimers();
const promise = warmPageYdoc("p1", "ws://x");
// Do not fire "synced"; let the 8s safety timeout settle it.
await vi.advanceTimersByTimeAsync(8000);
await expect(promise).resolves.toBeUndefined();
expect(h.providerDestroy).toHaveBeenCalledTimes(1);
expect(h.idbDestroy).toHaveBeenCalledTimes(1);
expect(h.ydocDestroy).toHaveBeenCalledTimes(1);
});
});

View File

@@ -1,315 +0,0 @@
import * as Y from "yjs";
import { IndexeddbPersistence } from "y-indexeddb";
import { HocuspocusProvider } from "@hocuspocus/provider";
import { queryClient } from "@/main.tsx";
import {
getPageById,
getPageBreadcrumbs,
getSidebarPages,
} from "@/features/page/services/page-service";
import {
pageKeys,
sidebarPagesQueryOptions,
} from "@/features/page/queries/page-query";
import { spaceByIdQueryOptions } from "@/features/space/queries/space-query";
import { RQ_KEY } from "@/features/comment/queries/comment-query";
import { getPageComments } from "@/features/comment/services/comment-service";
import { getMyInfo } from "@/features/user/services/user-service";
import { userKeys } from "@/features/user/hooks/use-current-user";
import { IPage } from "@/features/page/types/page.types";
import { IPagination } from "@/lib/types.ts";
import { pageYdocName } from "@/features/editor/page-ydoc-name";
/**
* Fully paginate an infinite query and write the @tanstack InfiniteData cache
* shape ({ pages, pageParams }) that the matching useInfiniteQuery hook reads.
*
* The default prefetchInfiniteQuery only warms the FIRST page, which leaves
* hooks that treat hasNextPage as still-loading (e.g. the comments panel)
* spinning forever offline, and silently truncates large lists. This walks the
* cursor chain until it runs out (or hits maxPages) so the whole list is cached.
*
* Best-effort: a failure does not throw (a partial/failed warm is still useful),
* but it is reported — the error is logged with context and `false` is returned
* so the caller can record the failed step instead of silently succeeding.
*
* Returns true ONLY if the cursor chain was fully exhausted and written. If the
* walk stops because it hit `maxPages` while a `nextCursor` is still pending,
* the cached list is truncated AND its last page keeps a nextCursor that cannot
* be re-fetched offline (hooks that gate on hasNextPage would spin forever), so
* that case is logged and returns false too — the caller records it as a failed
* warm instead of a silent truncated success. The (partial) cache is still
* written so what we did fetch is usable.
*
* Exported for unit testing of the cursor-walk / cache-write behavior.
*/
export async function warmInfiniteAll<T>(
queryKey: readonly unknown[],
fetchPage: (cursor: string | undefined) => Promise<IPagination<T>>,
maxPages = 50,
): Promise<boolean> {
try {
const pages: IPagination<T>[] = [];
const pageParams: (string | undefined)[] = [];
let cursor: string | undefined = undefined;
let exhausted = false;
for (let i = 0; i < maxPages; i++) {
const res = await fetchPage(cursor);
pages.push(res);
pageParams.push(cursor);
cursor = res?.meta?.nextCursor ?? undefined;
if (!cursor) {
exhausted = true;
break;
}
}
queryClient.setQueryData(queryKey, { pages, pageParams });
if (!exhausted) {
// Stopped at maxPages with a cursor still pending: the list is truncated
// and the last cached page's nextCursor is un-fetchable offline. Report it
// as a failed warm rather than a silent truncated success.
console.error("warmInfiniteAll truncated at maxPages", {
queryKey,
maxPages,
});
return false;
}
return true;
} catch (error) {
console.error("warmInfiniteAll failed", { queryKey, error });
return false;
}
}
export interface MakePageAvailableOfflineParams {
pageId: string;
spaceId?: string;
}
/**
* Outcome of {@link makePageAvailableOffline}. `ok` is true only when every warm
* step succeeded; `failed` lists the labels of the steps that failed (a subset
* of: "currentUser", "page", "space", "tree", "breadcrumbs", "comments").
*/
export interface MakePageAvailableOfflineResult {
ok: boolean;
failed: string[];
}
/**
* Best-effort prefetch of a page's read queries so they get persisted to
* IndexedDB and become readable offline.
*
* Each step is isolated and this function does NOT throw — a partial warm is
* still useful. Instead of silently succeeding, every failed step is logged
* with a label and recorded in the returned result: `{ ok, failed }` where
* `ok` is true only if no step failed and `failed` lists the failed step
* labels. Only meaningful while online (the underlying requests must succeed).
*/
export async function makePageAvailableOffline({
pageId,
spaceId,
}: MakePageAvailableOfflineParams): Promise<MakePageAvailableOfflineResult> {
const failed: string[] = [];
// Warm the current user (['currentUser']) so the auth-gated <Layout> can
// hydrate offline. UserProvider blanks the whole app while useCurrentUser has
// no data, and the offline POST /api/users/me fails as a network error, so
// without a persisted user a pinned page still white-screens after relaunch
// (#238). Persisted via OFFLINE_PERSIST_ROOTS; warmed here so the persisted
// cache actually has an entry to restore.
try {
await queryClient.prefetchQuery({
queryKey: userKeys.currentUser(),
queryFn: () => getMyInfo(),
});
} catch (error) {
console.error("makePageAvailableOffline: currentUser step failed", {
pageId,
error,
});
failed.push("currentUser");
}
// Fetch the page document ONCE and write it under BOTH cache keys, exactly
// like usePageQuery's onData effect. Every page consumer reads
// pageKeys.detail(slugId) (usePageQuery keys on the slugId for routed reads),
// so warming only the uuid key would leave the offline page blank.
let page: IPage | undefined;
try {
page = await getPageById({ pageId });
queryClient.setQueryData(pageKeys.detail(page.slugId), page);
queryClient.setQueryData(pageKeys.detail(page.id), page);
} catch (error) {
console.error("makePageAvailableOffline: page step failed", {
pageId,
error,
});
failed.push("page");
}
// Warm the space — page.tsx renders nothing until the space query resolves
// (useGetSpaceBySlugQuery). Awaited (not the fire-and-forget prefetchSpace) so
// the space is actually persisted before the caller fires its toast. Shares
// spaceByIdQueryOptions so the key/fn cannot drift from the hook.
try {
const spaceSlug = page?.space?.slug;
if (spaceSlug) {
await queryClient.prefetchQuery(spaceByIdQueryOptions(spaceSlug));
}
} catch (error) {
console.error("makePageAvailableOffline: space step failed", {
pageId,
error,
});
failed.push("space");
}
// Warm the sidebar tree root so the WHOLE root level renders offline (matches
// useGetRootSidebarPagesQuery's pageKeys.rootSidebar(spaceId) infinite cache).
// Fully paginated so large root levels are not truncated at 100.
if (spaceId) {
const ok = await warmInfiniteAll(pageKeys.rootSidebar(spaceId), (cursor) =>
getSidebarPages({ spaceId, cursor, limit: 100 }),
);
if (!ok) failed.push("tree");
}
// Warm the children of the page and of every ancestor so the path to this
// page is expandable offline. We MIRROR fetchAllAncestorChildren exactly via
// sidebarPagesQueryOptions — same pageKeys.sidebar({ pageId, spaceId }) key,
// same getAllSidebarPages fn (which aggregates ALL children pages, so nothing
// is truncated at 100), same 30min staleTime — otherwise the warmed cache
// would never be read by the offline tree.
const warmSidebarChildren = async (id: string): Promise<boolean> => {
try {
// Keep EXACTLY { pageId, spaceId } so the key hashes identically to
// fetchAllAncestorChildren's (no parentPageId, no extra fields).
const params = { pageId: id, spaceId };
await queryClient.prefetchQuery(sidebarPagesQueryOptions(params));
return true;
} catch (error) {
console.error("makePageAvailableOffline: tree node step failed", {
pageId: id,
error,
});
return false;
}
};
// The page's own children.
if (!(await warmSidebarChildren(pageId))) failed.push("tree");
// Each ancestor's children. Use the breadcrumbs endpoint ONLY to discover the
// ancestor ids — we intentionally do NOT cache the breadcrumbs themselves
// (the UI derives the path from the tree).
try {
const ancestors = (await getPageBreadcrumbs(pageId)) as
| Array<{ id?: string }>
| undefined;
for (const ancestor of ancestors ?? []) {
const ancestorId = ancestor?.id;
if (!ancestorId || ancestorId === pageId) continue;
if (!(await warmSidebarChildren(ancestorId))) failed.push("tree");
}
} catch (error) {
console.error("makePageAvailableOffline: breadcrumbs step failed", {
pageId,
error,
});
failed.push("breadcrumbs");
}
// Comments (matches useCommentsQuery's RQ_KEY(pageId) infinite cache).
// useCommentsQuery reports isLoading while hasNextPage is true, so warming
// only the first page leaves the offline comments panel spinning forever on
// pages with >100 comments. Fully paginate so the last cached page has no
// nextCursor and the panel settles offline.
const commentsOk = await warmInfiniteAll(RQ_KEY(pageId), (cursor) =>
getPageComments({ pageId, cursor, limit: 100 }),
);
if (!commentsOk) failed.push("comments");
// Dedupe — the tree label can be recorded once per failed node/ancestor.
const uniqueFailed = [...new Set(failed)];
return { ok: uniqueFailed.length === 0, failed: uniqueFailed };
}
/**
* Best-effort warm-up of the page's Yjs document into IndexedDB so the editor
* can open offline.
*
* Opens a local IndexeddbPersistence plus a transient HocuspocusProvider to
* pull the server state into IndexedDB, then tears both down once synced (or
* after a timeout). Entirely wrapped in try/catch — NEVER throws.
*
* Only meaningful when online at warm time; offline it is a no-op that resolves.
*/
export async function warmPageYdoc(
pageId: string,
collabUrl: string,
token?: string,
): Promise<void> {
let ydoc: Y.Doc | null = null;
let local: IndexeddbPersistence | null = null;
let remote: HocuspocusProvider | null = null;
try {
const documentName = pageYdocName(pageId);
ydoc = new Y.Doc();
local = new IndexeddbPersistence(documentName, ydoc);
remote = new HocuspocusProvider({
url: collabUrl,
name: documentName,
document: ydoc,
token,
});
const provider = remote;
await new Promise<void>((resolve) => {
let settled = false;
let timeoutId: ReturnType<typeof setTimeout> | undefined;
const finish = () => {
if (settled) return;
settled = true;
// Clear the pending timeout and detach the listener so neither leaks
// after we resolve.
if (timeoutId !== undefined) clearTimeout(timeoutId);
try {
provider.off("synced", finish);
} catch {
// best-effort
}
resolve();
};
// Resolve once the server state has synced into the local doc...
provider.on("synced", finish);
// ...or give up after a short timeout so we never hang.
timeoutId = setTimeout(finish, 8000);
});
} catch {
// best-effort
} finally {
try {
remote?.destroy();
} catch {
// best-effort
}
try {
local?.destroy();
} catch {
// best-effort
}
try {
ydoc?.destroy();
} catch {
// best-effort
}
}
}

View File

@@ -1,45 +0,0 @@
import { Button, Container, Group, Stack, Text, Title } from "@mantine/core";
import { Helmet } from "react-helmet-async";
import { useTranslation } from "react-i18next";
import { getAppName } from "@/lib/config";
/**
* Shown when the authenticated app shell cannot hydrate because the current
* user is unavailable AND there is no cached user to fall back on (e.g. an
* offline cold boot of a page that was never warmed for offline).
*
* Previously UserProvider returned a bare `<></>` in this situation, which
* white-screened the whole app on any offline reload (#237/#238). Rendering an
* explicit "you're offline" state with a retry instead gives the user a clear,
* non-blank fallback and a way to recover once the network returns.
*/
export function OfflineFallback() {
const { t } = useTranslation();
return (
<>
<Helmet>
<title>
{t("You're offline")} - {getAppName()}
</title>
</Helmet>
<Container size="sm" py={80}>
<Stack align="center" gap="md">
<Title order={2} ta="center">
{t("You're offline")}
</Title>
<Text c="dimmed" size="lg" ta="center">
{t(
"This page hasn't been saved for offline use, so it can't be loaded right now. Reconnect to the internet and try again.",
)}
</Text>
<Group justify="center">
<Button onClick={() => window.location.reload()} variant="subtle">
{t("Retry")}
</Button>
</Group>
</Stack>
</Container>
</>
);
}

View File

@@ -1,114 +0,0 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { QueryClient, hydrate, dehydrate } from "@tanstack/react-query";
// Stub the network services so a replayed mutation hits a spy, not the network.
const h = vi.hoisted(() => ({
createPage: vi.fn(),
movePage: vi.fn(),
createComment: vi.fn(),
}));
vi.mock("@/features/page/services/page-service", () => ({
createPage: h.createPage,
movePage: h.movePage,
}));
vi.mock("@/features/comment/services/comment-service", () => ({
createComment: h.createComment,
}));
// page-query pulls in the app entry (queryClient) and a lot of UI deps via its
// cache helpers; we only need invalidateOnCreatePage to be a no-op here.
vi.mock("@/features/page/queries/page-query", () => ({
invalidateOnCreatePage: vi.fn(),
}));
import {
offlineMutationKeys,
registerOfflineMutationDefaults,
} from "./offline-mutations";
beforeEach(() => {
h.createPage.mockReset().mockResolvedValue({ id: "new-page" });
h.movePage.mockReset().mockResolvedValue(undefined);
h.createComment.mockReset().mockResolvedValue({ id: "new-comment" });
});
describe("registerOfflineMutationDefaults", () => {
it("registers a default mutationFn for every offline mutation key", () => {
const qc = new QueryClient();
registerOfflineMutationDefaults(qc);
for (const key of Object.values(offlineMutationKeys)) {
const defaults = qc.getMutationDefaults(key);
expect(typeof defaults?.mutationFn).toBe("function");
}
});
// The headline durability guarantee: a paused mutation dehydrated into
// IndexedDB while offline must, after a reload, have a mutationFn so
// resumePausedMutations() actually replays the write on reconnect.
it("makes a rehydrated paused create replayable by resumePausedMutations", async () => {
// 1) Simulate the offline tab: a paused create mutation gets dehydrated.
const offlineClient = new QueryClient();
const observer = offlineClient.getMutationCache().build(offlineClient, {
mutationKey: offlineMutationKeys.createPage,
});
// Force the dehydrate-worthy paused state (offline = isPaused) with the
// payload the user submitted before losing connectivity.
observer.state.isPaused = true;
observer.state.status = "pending";
observer.state.variables = { spaceId: "s1", title: "Offline page" };
const dehydrated = dehydrate(offlineClient, {
shouldDehydrateMutation: () => true,
});
expect(dehydrated.mutations).toHaveLength(1);
// The dehydrated mutation carries NO mutationFn (functions aren't
// serializable) — only its key + variables survive the reload.
expect((dehydrated.mutations[0] as any).mutationFn).toBeUndefined();
// 2) Simulate the fresh page after reload: register defaults, then hydrate
// the persisted paused mutation back in.
const freshClient = new QueryClient();
registerOfflineMutationDefaults(freshClient);
hydrate(freshClient, dehydrated);
expect(freshClient.getMutationCache().getAll()).toHaveLength(1);
// 3) Reconnect: replay the paused mutations.
await freshClient.resumePausedMutations();
// The default mutationFn ran with the persisted variables — the write is
// NOT silently dropped.
expect(h.createPage).toHaveBeenCalledTimes(1);
expect(h.createPage).toHaveBeenCalledWith({
spaceId: "s1",
title: "Offline page",
});
});
it("makes a rehydrated paused move replayable by resumePausedMutations", async () => {
const offlineClient = new QueryClient();
const observer = offlineClient.getMutationCache().build(offlineClient, {
mutationKey: offlineMutationKeys.movePage,
});
observer.state.isPaused = true;
observer.state.status = "pending";
observer.state.variables = { pageId: "p1", parentPageId: null, position: "a" };
const dehydrated = dehydrate(offlineClient, {
shouldDehydrateMutation: () => true,
});
const freshClient = new QueryClient();
registerOfflineMutationDefaults(freshClient);
hydrate(freshClient, dehydrated);
await freshClient.resumePausedMutations();
expect(h.movePage).toHaveBeenCalledTimes(1);
expect(h.movePage).toHaveBeenCalledWith({
pageId: "p1",
parentPageId: null,
position: "a",
});
});
});

View File

@@ -1,64 +0,0 @@
import type { QueryClient } from "@tanstack/react-query";
import { createPage, movePage } from "@/features/page/services/page-service";
import { createComment } from "@/features/comment/services/comment-service";
import { invalidateOnCreatePage } from "@/features/page/queries/page-query";
import type {
IMovePage,
IPage,
IPageInput,
} from "@/features/page/types/page.types";
import type { IComment } from "@/features/comment/types/comment.types";
/**
* Stable mutation keys for the offline-relevant structural mutations.
*
* When the browser goes offline, React Query PAUSES these mutations and the
* PersistQueryClientProvider dehydrates the paused mutation into IndexedDB. On a
* reload-while-offline the mutation is restored, but a restored mutation has NO
* observer (no component is mounted) — so its replay relies entirely on the
* `mutationFn` registered via `setMutationDefaults` for its `mutationKey`.
* Without that, `resumePausedMutations()` finds a paused mutation with no
* `mutationFn` and silently no-ops, dropping the offline create/move/comment
* (#237/#238). Each offline mutation hook tags itself with the matching key so
* the rehydrated paused mutation can find its default `mutationFn` and replay.
*/
export const offlineMutationKeys = {
createPage: ["create-page"] as const,
movePage: ["move-page"] as const,
createComment: ["create-comment"] as const,
};
/**
* Register default `mutationFn`s (and the minimal success side effects safe to
* run without a mounted component) for the offline-relevant mutation keys, so a
* paused mutation restored from IndexedDB after an offline reload is replayable
* by `resumePausedMutations()` on reconnect.
*
* Called once when the QueryClient is created (see main.tsx). The hooks still
* carry their own inline `mutationFn`/`onSuccess` for the live in-session path;
* these defaults only take over for a rehydrated paused mutation that lost its
* observer across the reload.
*/
export function registerOfflineMutationDefaults(queryClient: QueryClient): void {
queryClient.setMutationDefaults(offlineMutationKeys.createPage, {
mutationFn: (data: Partial<IPageInput>) => createPage(data),
// Re-converge the sidebar tree / recent-changes from the authoritative
// create response. Pure cache writes — safe with no component mounted.
onSuccess: (data: IPage) => {
invalidateOnCreatePage(data);
},
});
queryClient.setMutationDefaults(offlineMutationKeys.movePage, {
// Replay the server-side move. The tree re-converges from the next online
// sidebar fetch / websocket `moveTreeNode` echo, so no cache write is
// needed here (the optimistic tree state was local-only anyway).
mutationFn: (data: IMovePage) => movePage(data),
});
queryClient.setMutationDefaults(offlineMutationKeys.createComment, {
// Replay the server-side comment create. The comments list refetches on the
// online reload, so the replay only needs to persist the write.
mutationFn: (data: Partial<IComment>) => createComment(data),
});
}

View File

@@ -1,128 +0,0 @@
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import { QueryClient, onlineManager } from "@tanstack/react-query";
import {
persistQueryClientRestore,
persistQueryClientSave,
} from "@tanstack/react-query-persist-client";
// Stub the network services so a replayed mutation hits a spy, not the network.
const h = vi.hoisted(() => ({
createPage: vi.fn(),
movePage: vi.fn(),
createComment: vi.fn(),
}));
vi.mock("@/features/page/services/page-service", () => ({
createPage: h.createPage,
movePage: h.movePage,
}));
vi.mock("@/features/comment/services/comment-service", () => ({
createComment: h.createComment,
}));
vi.mock("@/features/page/queries/page-query", () => ({
invalidateOnCreatePage: vi.fn(),
}));
// In-memory idb-keyval so the REAL queryPersister round-trips through a fake
// store (the actual persist -> reload -> restore path, not a hand-built blob).
const store = new Map<string, string>();
vi.mock("idb-keyval", () => ({
get: vi.fn((k: string) => Promise.resolve(store.get(k) ?? undefined)),
set: vi.fn((k: string, v: string) => {
store.set(k, v);
return Promise.resolve();
}),
del: vi.fn((k: string) => {
store.delete(k);
return Promise.resolve();
}),
}));
import { queryPersister } from "./query-persister";
import {
offlineMutationKeys,
registerOfflineMutationDefaults,
} from "./offline-mutations";
const BUSTER = "test-buster";
beforeEach(() => {
store.clear();
h.createPage.mockReset().mockResolvedValue({ id: "new-page" });
h.movePage.mockReset().mockResolvedValue(undefined);
h.createComment.mockReset().mockResolvedValue({ id: "new-comment" });
});
afterEach(() => {
// onlineManager is a global singleton; leave it in the default online state.
onlineManager.setOnline(true);
});
describe("offline paused-mutation resume across a reload", () => {
// This is the #120 silent-data-loss reproduction: a paused mutation persisted
// to IndexedDB while offline, then the tab RELOADS while still offline, must
// resume on reconnect. It exercises the real persister round-trip plus the two
// boot-time fixes the app wiring relies on:
// (a) onlineManager seeded to the real offline state so the later reconnect
// is a true offline->online transition that auto-resumes, and
// (b) resumePausedMutations() called after the persister restores (what the
// PersistQueryClientProvider onSuccess does), with mutation defaults
// registered BEFORE the resume so the rehydrated mutation has a fn.
it("replays a rehydrated paused create on reconnect (mutationFn fires)", async () => {
// --- Tab 1, OFFLINE: user creates a page; it pauses and gets persisted. ---
onlineManager.setOnline(false); // (a) boot seeded offline
const client1 = new QueryClient();
registerOfflineMutationDefaults(client1);
const observer = client1.getMutationCache().build(client1, {
mutationKey: offlineMutationKeys.createPage,
});
observer.state.isPaused = true;
observer.state.status = "pending";
observer.state.variables = { spaceId: "s1", title: "Offline page" };
await persistQueryClientSave({
// Cast: persist-client-core and react-query may resolve to different
// @tanstack/query-core copies whose QueryClient brands are nominally
// incompatible (see query-persister.ts). Structurally identical at runtime.
queryClient: client1 as any,
persister: queryPersister,
buster: BUSTER,
dehydrateOptions: { shouldDehydrateMutation: () => true },
});
// The paused mutation is now in the persisted store.
expect(store.size).toBe(1);
// --- RELOAD while still offline: fresh client restores from the SAME
// persister. Defaults are registered BEFORE restore/resume. ---
const client2 = new QueryClient();
registerOfflineMutationDefaults(client2);
client2.mount(); // subscribes to onlineManager (auto-resume on reconnect)
await persistQueryClientRestore({
queryClient: client2 as any,
persister: queryPersister,
buster: BUSTER,
});
expect(client2.getMutationCache().getAll()).toHaveLength(1);
// (b) onSuccess wiring resumes after restore — but we are still OFFLINE, so
// the mutation must stay paused and NOT fire yet.
await client2.resumePausedMutations();
expect(h.createPage).not.toHaveBeenCalled();
// --- RECONNECT: the offline->online transition auto-resumes the paused
// mutation and its registered default mutationFn finally fires. ---
onlineManager.setOnline(true);
await vi.waitFor(() => {
expect(h.createPage).toHaveBeenCalledTimes(1);
});
expect(h.createPage).toHaveBeenCalledWith({
spaceId: "s1",
title: "Offline page",
});
client2.unmount();
});
});

View File

@@ -1,48 +0,0 @@
import { describe, it, expect } from "vitest";
// The query modules transitively import the app entry (@/main.tsx) for the
// shared queryClient; mock it so importing the key factories has no side effects.
import { vi } from "vitest";
vi.mock("@/main.tsx", () => ({
queryClient: { setQueryData: vi.fn(), getQueryData: vi.fn() },
}));
import { OFFLINE_PERSIST_ROOTS } from "./query-persister";
import { pageKeys } from "@/features/page/queries/page-query";
import { spaceKeys } from "@/features/space/queries/space-query";
import { RQ_KEY } from "@/features/comment/queries/comment-query";
import { userKeys } from "@/features/user/hooks/use-current-user";
/**
* Architecture guard (#13): every string persisted via OFFLINE_PERSIST_ROOTS
* must be the ROOT (queryKey[0]) of some exported query-key factory. If a
* factory's root is renamed without updating the persist registry — or vice
* versa — offline persist/warm silently breaks (persisted keys never match the
* live queries). This turns that silent regression into a red build.
*
* Each factory is invoked with throwaway args; only queryKey[0] is inspected.
*/
function rootOf(key: readonly unknown[]): string {
return String(key[0]);
}
const FACTORY_ROOTS = new Set<string>([
rootOf(pageKeys.detail("x")),
rootOf(pageKeys.sidebar({})),
rootOf(pageKeys.rootSidebar("x")),
rootOf(pageKeys.breadcrumbs("x")),
rootOf(pageKeys.recentChanges("x")),
rootOf(spaceKeys.detail("x")),
rootOf(spaceKeys.list()),
rootOf(RQ_KEY("x")),
rootOf(userKeys.currentUser()),
]);
describe("OFFLINE_PERSIST_ROOTS is backed by real query-key factories", () => {
it("maps every persisted root to an exported factory root", () => {
const unbacked = [...OFFLINE_PERSIST_ROOTS].filter(
(root) => !FACTORY_ROOTS.has(root),
);
expect(unbacked).toEqual([]);
});
});

View File

@@ -1,128 +0,0 @@
import { describe, it, expect, vi, afterEach } from "vitest";
// In-memory idb-keyval so we can observe whether the persister actually writes.
const h = vi.hoisted(() => ({
get: vi.fn(() => Promise.resolve(undefined)),
set: vi.fn(() => Promise.resolve()),
del: vi.fn(() => Promise.resolve()),
}));
vi.mock("idb-keyval", () => h);
import {
shouldDehydrateOfflineQuery,
OFFLINE_PERSIST_ROOTS,
queryPersister,
freezeOfflinePersistence,
unfreezeOfflinePersistence,
} from "./query-persister";
// Small helper to build the structural query shape the predicate reads.
const makeQuery = (status: string, queryKey: readonly unknown[]) =>
({ state: { status }, queryKey }) as any;
describe("shouldDehydrateOfflineQuery", () => {
it("returns true for a successful query whose root is in the allowlist", () => {
expect(shouldDehydrateOfflineQuery(makeQuery("success", ["pages", "abc"]))).toBe(
true,
);
expect(
shouldDehydrateOfflineQuery(
makeQuery("success", ["sidebar-pages", { pageId: "p", spaceId: "s" }]),
),
).toBe(true);
expect(
shouldDehydrateOfflineQuery(makeQuery("success", ["comments", "p1"])),
).toBe(true);
expect(
shouldDehydrateOfflineQuery(makeQuery("success", ["space", "s"])),
).toBe(true);
expect(
shouldDehydrateOfflineQuery(makeQuery("success", ["recent-changes"])),
).toBe(true);
// currentUser is persisted so the auth-gated Layout can hydrate offline.
expect(
shouldDehydrateOfflineQuery(makeQuery("success", ["currentUser"])),
).toBe(true);
});
it("returns false when the status is not success (status gate)", () => {
expect(
shouldDehydrateOfflineQuery(makeQuery("pending", ["pages", "abc"])),
).toBe(false);
expect(
shouldDehydrateOfflineQuery(makeQuery("error", ["pages", "abc"])),
).toBe(false);
});
it("returns false for a successful query whose root is NOT in the allowlist (privacy gate)", () => {
expect(
shouldDehydrateOfflineQuery(makeQuery("success", ["collab-token", "ws"])),
).toBe(false);
expect(
shouldDehydrateOfflineQuery(makeQuery("success", ["trash", "s"])),
).toBe(false);
expect(
shouldDehydrateOfflineQuery(makeQuery("success", ["unknown"])),
).toBe(false);
});
it("returns false for an empty/undefined queryKey", () => {
// String(undefined) is not a member of the allowlist.
expect(shouldDehydrateOfflineQuery(makeQuery("success", []))).toBe(false);
expect(
shouldDehydrateOfflineQuery(makeQuery("success", undefined as any)),
).toBe(false);
});
});
describe("OFFLINE_PERSIST_ROOTS", () => {
it("contains exactly the expected 9 navigation/read roots", () => {
const expected = [
"pages",
"sidebar-pages",
"root-sidebar-pages",
"breadcrumbs",
"comments",
"space",
"spaces",
"recent-changes",
"currentUser",
];
expect(OFFLINE_PERSIST_ROOTS.size).toBe(9);
for (const root of expected) {
expect(OFFLINE_PERSIST_ROOTS.has(root)).toBe(true);
}
});
it("does NOT contain volatile/auth keys", () => {
expect(OFFLINE_PERSIST_ROOTS.has("collab-token")).toBe(false);
expect(OFFLINE_PERSIST_ROOTS.has("trash")).toBe(false);
});
});
describe("freeze/unfreeze persistence (logout no-late-write guard)", () => {
const dummyClient = {
timestamp: Date.now(),
buster: "",
clientState: { mutations: [], queries: [] },
} as any;
afterEach(() => {
// Always leave persistence enabled so other tests/sessions persist normally.
unfreezeOfflinePersistence();
h.set.mockClear();
});
it("does NOT write to storage while frozen", async () => {
freezeOfflinePersistence();
await queryPersister.persistClient(dummyClient);
expect(h.set).not.toHaveBeenCalled();
});
it("resumes writing to storage once unfrozen", async () => {
freezeOfflinePersistence();
unfreezeOfflinePersistence();
await queryPersister.persistClient(dummyClient);
expect(h.set).toHaveBeenCalledTimes(1);
});
});

View File

@@ -1,84 +0,0 @@
import { get, set, del } from "idb-keyval";
import { createAsyncStoragePersister } from "@tanstack/query-async-storage-persister";
// Structural subset of a TanStack Query we read when deciding what to persist.
// We avoid importing the branded `Query` class because the persist-client and
// react-query may resolve to different `@tanstack/query-core` copies, whose
// `Query` types are nominally incompatible (private brand). This structural
// shape stays assignable to whichever copy the persister expects.
type DehydratableQuery = {
state: { status: string };
queryKey: readonly unknown[];
};
// idb-keyval key under which TanStack Query persists its dehydrated cache.
// Exported so the logout cache-clear logic deletes the exact same key (no
// magic-string drift between persist and purge).
export const OFFLINE_CACHE_KEY = "gitmost-rq-cache";
// IndexedDB-backed storage adapter for TanStack Query's async persister.
const idbStorage = {
getItem: (key: string) => get<string>(key).then((v) => v ?? null),
setItem: (key: string, value: string) => set(key, value),
removeItem: (key: string) => del(key),
};
const basePersister = createAsyncStoragePersister({
storage: idbStorage,
key: OFFLINE_CACHE_KEY,
throttleTime: 1000,
});
// When frozen, persistClient becomes a no-op so no new dehydrated snapshot is
// written to IndexedDB. This closes a logout data-leak race: clearing the cache
// (queryClient.clear()) fires `removed` cache events, each of which the persist
// subscription turns into a throttled persistClient call. The FIRST such call
// dehydrates a still-nearly-full snapshot and its async write can land AFTER the
// del() that clears the key, resurrecting the previous user's data (~180KB) in
// IndexedDB. Freezing before clear()/del() prevents any such rewrite. Re-enabled
// afterwards so the next (sign-in) session persists normally. See
// clear-offline-cache.ts.
let persistFrozen = false;
export function freezeOfflinePersistence(): void {
persistFrozen = true;
}
export function unfreezeOfflinePersistence(): void {
persistFrozen = false;
}
export const queryPersister = {
persistClient: (persistedClient: Parameters<typeof basePersister.persistClient>[0]) =>
persistFrozen ? Promise.resolve() : basePersister.persistClient(persistedClient),
restoreClient: () => basePersister.restoreClient(),
removeClient: () => basePersister.removeClient(),
};
// Only navigation/read query roots are persisted for offline reading.
// Volatile/auth queries (collab tokens, trash lists) are intentionally excluded.
//
// `currentUser` IS persisted: UserProvider gates the entire <Layout> subtree on
// useCurrentUser(), and offline the POST /api/users/me fails as a no-response
// network error. Without the persisted/hydrated user the gate blanked every
// authenticated route on an offline cold boot (#237/#238). It is the logged-in
// user's own profile (already mirrored to localStorage["currentUser"]), so
// persisting it to IndexedDB leaks nothing new while unlocking offline reads.
export const OFFLINE_PERSIST_ROOTS = new Set<string>([
"pages",
"sidebar-pages",
"root-sidebar-pages",
"breadcrumbs",
"comments",
"space",
"spaces",
"recent-changes",
"currentUser",
]);
export function shouldDehydrateOfflineQuery(query: DehydratableQuery): boolean {
return (
query.state.status === "success" &&
OFFLINE_PERSIST_ROOTS.has(String(query.queryKey?.[0]))
);
}

View File

@@ -0,0 +1,227 @@
import { describe, it, expect, vi, afterEach, beforeAll } from "vitest";
import { render, screen, cleanup, within } from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
// Mantine Tooltip mounts its label lazily on hover via Floating UI, which is
// flaky under jsdom. Replace ONLY the Tooltip with a thin wrapper that renders
// the label inline (keeping Badge/Switch/etc. real), so the provenance label —
// the contract we care about — is deterministically queryable.
vi.mock("@mantine/core", async () => {
const actual =
await vi.importActual<typeof import("@mantine/core")>("@mantine/core");
const Tooltip = ({
label,
children,
}: {
label?: React.ReactNode;
children?: React.ReactNode;
}) => (
<>
{children}
<span data-testid="tooltip-label">{label}</span>
</>
);
Tooltip.Group = ({ children }: { children?: React.ReactNode }) => (
<>{children}</>
);
return { ...actual, Tooltip };
});
// jsdom lacks matchMedia, which MantineProvider's color-scheme hook needs.
beforeAll(() => {
if (!window.matchMedia) {
window.matchMedia = (query: string) =>
({
matches: false,
media: query,
onchange: null,
addListener: () => {},
removeListener: () => {},
addEventListener: () => {},
removeEventListener: () => {},
dispatchEvent: () => false,
}) as unknown as MediaQueryList;
}
});
// --- Mocks for the heavy / networked module graph ---------------------------
// HistoryItem pulls in i18n, jotai atoms (ai-chat / history), a config-backed
// avatar and a time formatter. The provenance-badge contract is the unit under
// test, so we stub everything else down to inert, deterministic renders and
// keep the real Mantine Badge/Tooltip so role/label queries are meaningful.
// i18n: interpolate {{name}} so the git-sync tooltip carries the author name,
// letting us assert provenance attribution without a real i18n backend.
vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (key: string, vars?: Record<string, unknown>) =>
vars && typeof vars.name !== "undefined"
? key.replace("{{name}}", String(vars.name))
: key,
}),
}));
// jotai setters: the badges call useSetAtom; return inert setters so a click on
// the (deep-linkable) AiAgentBadge would fire these — proving the git-sync badge
// does NOT wire any of them.
const setAiChatWindowOpen = vi.fn();
const setActiveChatId = vi.fn();
const setDraft = vi.fn();
const setHistoryModalOpen = vi.fn();
vi.mock("jotai", async () => {
const actual = await vi.importActual<typeof import("jotai")>("jotai");
return {
...actual,
useSetAtom: (atom: unknown) => {
switch (atom) {
case aiChatWindowOpenAtom:
return setAiChatWindowOpen;
case activeAiChatIdAtom:
return setActiveChatId;
case aiChatDraftAtom:
return setDraft;
case historyAtoms:
return setHistoryModalOpen;
default:
return vi.fn();
}
},
};
});
// Atoms are imported only as identity tokens for the useSetAtom switch above.
vi.mock("@/features/ai-chat/atoms/ai-chat-atom.ts", () => ({
activeAiChatIdAtom: { __tag: "activeAiChatIdAtom" },
aiChatWindowOpenAtom: { __tag: "aiChatWindowOpenAtom" },
aiChatDraftAtom: { __tag: "aiChatDraftAtom" },
}));
vi.mock("@/features/page-history/atoms/history-atoms.ts", () => ({
historyAtoms: { __tag: "historyAtoms" },
}));
// Avatar reaches into config (getAvatarUrl) — stub to a plain element.
vi.mock("@/components/ui/custom-avatar.tsx", () => ({
CustomAvatar: ({ name }: { name?: string }) => (
<span data-testid="avatar">{name}</span>
),
}));
// Deterministic, locale-free date string.
vi.mock("@/lib/time", () => ({
formattedDate: () => "2026-06-21",
}));
import HistoryItem from "./history-item";
import {
activeAiChatIdAtom,
aiChatWindowOpenAtom,
aiChatDraftAtom,
} from "@/features/ai-chat/atoms/ai-chat-atom.ts";
import { historyAtoms } from "@/features/page-history/atoms/history-atoms.ts";
import type { IPageHistory } from "@/features/page-history/types/page.types";
function makeItem(overrides: Partial<IPageHistory> = {}): IPageHistory {
return {
id: "h1",
pageId: "p1",
title: "Title",
slug: "slug",
icon: "",
coverPhoto: "",
version: 1,
lastUpdatedById: "u1",
workspaceId: "w1",
createdAt: "2026-06-21T00:00:00.000Z",
updatedAt: "2026-06-21T00:00:00.000Z",
lastUpdatedBy: { id: "u1", name: "Alice", avatarUrl: "" },
...overrides,
};
}
function renderItem(item: IPageHistory) {
return render(
<MantineProvider>
<HistoryItem
historyItem={item}
index={0}
onSelect={vi.fn()}
isActive={false}
/>
</MantineProvider>,
);
}
afterEach(() => {
cleanup();
vi.clearAllMocks();
});
describe("HistoryItem git-sync provenance badge", () => {
// Test 1: the git-sync badge renders ONLY for lastUpdatedSource === 'git-sync'.
it("renders the Git sync badge only when lastUpdatedSource is 'git-sync'", () => {
renderItem(makeItem({ lastUpdatedSource: "git-sync" }));
expect(screen.getByText("Git sync")).toBeTruthy();
});
it.each([
["agent", "agent"],
["user", "user"],
["undefined", undefined],
])(
"does NOT render the Git sync badge when lastUpdatedSource is %s",
(_label, source) => {
renderItem(makeItem({ lastUpdatedSource: source }));
expect(screen.queryByText("Git sync")).toBeNull();
},
);
// Test 2: provenance attribution + the git-sync badge is NOT interactive.
it("attributes the git-sync provenance to the correct author and is not clickable", () => {
renderItem(
makeItem({
lastUpdatedSource: "git-sync",
lastUpdatedBy: { id: "u2", name: "Bob", avatarUrl: "" },
}),
);
const badge = screen.getByText("Git sync");
// Provenance attribution: the tooltip label carries the author name (the
// git-sync badge passes authorName -> "Synced from Git on behalf of {{name}}").
expect(screen.getByText("Synced from Git on behalf of Bob")).toBeTruthy();
// The git-sync badge must NOT behave like AiAgentBadge: the badge element
// itself is not a button, carries no role=button and no tabIndex, and
// clicking it must not trigger any ai-chat deep-link. (The surrounding
// history-row IS an UnstyledButton — that is the row's own select affordance,
// not the badge — so we scope these checks to the badge element.)
const badgeRoot = (badge.closest("[class*='mantine-Badge-root']") ??
badge) as HTMLElement;
expect(badgeRoot.getAttribute("role")).not.toBe("button");
expect(badgeRoot.getAttribute("tabindex")).toBeNull();
expect(badgeRoot.tagName.toLowerCase()).not.toBe("button");
// No interactive descendant button lives inside the badge itself.
expect(within(badgeRoot).queryByRole("button")).toBeNull();
badgeRoot.dispatchEvent(new MouseEvent("click", { bubbles: true }));
expect(setActiveChatId).not.toHaveBeenCalled();
expect(setAiChatWindowOpen).not.toHaveBeenCalled();
expect(setDraft).not.toHaveBeenCalled();
expect(setHistoryModalOpen).not.toHaveBeenCalled();
});
// Sanity contrast: the agent badge (the copy-paste source) IS interactive when
// it carries an aiChatId — proving the not-clickable assertion above is real.
it("contrast: the AI-agent badge is a deep-link button when it has an aiChatId", () => {
renderItem(
makeItem({
lastUpdatedSource: "agent",
lastUpdatedAiChatId: "chat-1",
}),
);
const agentBadge = screen.getByText("AI-agent");
const root = agentBadge.closest("[role='button']");
expect(root).not.toBeNull();
within(root as HTMLElement).getByText("AI-agent");
});
});

View File

@@ -1,6 +1,7 @@
import { Text, Group, UnstyledButton, Avatar, Tooltip } from "@mantine/core";
import { CustomAvatar } from "@/components/ui/custom-avatar.tsx";
import { AiAgentBadge } from "@/components/ui/ai-agent-badge.tsx";
import { GitSyncBadge } from "@/components/ui/git-sync-badge.tsx";
import { formattedDate } from "@/lib/time";
import classes from "./css/history.module.css";
import clsx from "clsx";
@@ -41,6 +42,7 @@ const HistoryItem = memo(function HistoryItem({
const contributors = historyItem.contributors;
const hasContributors = contributors && contributors.length > 0;
const isAgentEdit = historyItem.lastUpdatedSource === "agent";
const isGitSyncEdit = historyItem.lastUpdatedSource === "git-sync";
return (
<UnstyledButton
@@ -108,6 +110,10 @@ const HistoryItem = memo(function HistoryItem({
onActivate={() => setHistoryModalOpen(false)}
/>
)}
{isGitSyncEdit && (
<GitSyncBadge authorName={historyItem.lastUpdatedBy?.name} />
)}
</Group>
</UnstyledButton>
);

View File

@@ -12,8 +12,6 @@ import {
IconList,
IconMarkdown,
IconPrinter,
IconCloud,
IconCloudCheck,
IconStar,
IconStarFilled,
IconTrash,
@@ -41,8 +39,6 @@ import { Trans, useTranslation } from "react-i18next";
import ExportModal from "@/components/common/export-modal";
import { htmlToMarkdown } from "@docmost/editor-ext";
import {
isLocalSyncedAtom,
isRemoteSyncedAtom,
pageEditorAtom,
yjsConnectionStatusAtom,
} from "@/features/editor/atoms/editor-atoms.ts";
@@ -415,16 +411,14 @@ function PageActionMenu({ readOnly }: PageActionMenuProps) {
function ConnectionWarning() {
const { t } = useTranslation();
const yjsConnectionStatus = useAtomValue(yjsConnectionStatusAtom);
const isLocalSynced = useAtomValue(isLocalSyncedAtom);
const isRemoteSynced = useAtomValue(isRemoteSyncedAtom);
const [showWarning, setShowWarning] = useState(false);
const timeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
const isDisconnected = ["disconnected", "connecting"].includes(
yjsConnectionStatus,
);
useEffect(() => {
const isDisconnected = ["disconnected", "connecting"].includes(
yjsConnectionStatus,
);
if (isDisconnected) {
if (!timeoutRef.current) {
timeoutRef.current = setTimeout(() => setShowWarning(true), 5000);
@@ -436,7 +430,7 @@ function ConnectionWarning() {
}
setShowWarning(false);
}
}, [isDisconnected]);
}, [yjsConnectionStatus]);
// Cleanup only on unmount
useEffect(() => {
@@ -447,59 +441,22 @@ function ConnectionWarning() {
};
}, []);
// State (1): offline/disconnected — changes are kept locally. Preserve the
// existing >5s debounce before surfacing this state.
if (isDisconnected) {
if (!showWarning) return null;
if (!showWarning) return null;
const offlineLabel = t(
"Offline — changes are saved locally and will sync when you reconnect",
);
return (
<Tooltip label={offlineLabel} openDelay={250} withArrow>
<ThemeIcon
variant="default"
c="red"
role="status"
aria-label={offlineLabel}
style={{ border: "none" }}
>
<IconWifiOff size={20} stroke={2} />
</ThemeIcon>
</Tooltip>
);
}
// State (2): connected but the remote replica is not fully caught up yet.
if (!isRemoteSynced || !isLocalSynced) {
const syncingLabel = t("Syncing changes…");
return (
<Tooltip label={syncingLabel} openDelay={250} withArrow>
<ThemeIcon
variant="default"
c="dimmed"
role="status"
aria-label={syncingLabel}
style={{ border: "none" }}
>
<IconCloud size={20} stroke={2} />
</ThemeIcon>
</Tooltip>
);
}
// State (3): fully synced — subtle confirmation indicator.
const syncedLabel = t("All changes synced");
return (
<Tooltip label={syncedLabel} openDelay={250} withArrow>
<Tooltip
label={t("Real-time editor connection lost. Retrying...")}
openDelay={250}
withArrow
>
<ThemeIcon
variant="default"
c="dimmed"
c="red"
role="status"
aria-label={syncedLabel}
aria-label={t("Real-time editor connection lost. Retrying...")}
style={{ border: "none" }}
>
<IconCloudCheck size={20} stroke={2} />
<IconWifiOff size={20} stroke={2} />
</ThemeIcon>
</Tooltip>
);

View File

@@ -1,7 +1,6 @@
import {
InfiniteData,
QueryKey,
queryOptions,
useInfiniteQuery,
UseInfiniteQueryResult,
useMutation,
@@ -43,38 +42,12 @@ import { treeModel } from "@/features/page/tree/model/tree-model";
import { SpaceTreeNode } from "@/features/page/tree/types";
import { useQueryEmit } from "@/features/websocket/use-query-emit";
import { moveToTrashNotificationMessage } from "@/features/page/components/move-to-trash-notification";
import { offlineMutationKeys } from "@/features/offline/offline-mutations";
/**
* Centralized React Query key factories for page queries. The hooks below and
* the offline warm path (features/offline/make-offline.ts) share these so the
* runtime keys can never silently drift apart.
*/
export const pageKeys = {
detail: (idOrSlug: string) => ["pages", idOrSlug] as const,
sidebar: (data: unknown) => ["sidebar-pages", data] as const,
rootSidebar: (spaceId: string) => ["root-sidebar-pages", spaceId] as const,
breadcrumbs: (pageId: string) => ["breadcrumbs", pageId] as const,
recentChanges: (spaceId?: string) => ["recent-changes", spaceId] as const,
};
/**
* Shared queryOptions for the sidebar-pages (ancestor children) query. Both
* fetchAllAncestorChildren and the offline warm path consume this so the key,
* queryFn and staleTime stay identical.
*/
export const sidebarPagesQueryOptions = (params: SidebarPagesParams) =>
queryOptions({
queryKey: pageKeys.sidebar(params),
queryFn: () => getAllSidebarPages(params),
staleTime: 30 * 60 * 1000,
});
export function usePageQuery(
pageInput: Partial<IPageInput>,
): UseQueryResult<IPage, Error> {
const query = useQuery({
queryKey: pageKeys.detail(pageInput.pageId),
queryKey: ["pages", pageInput.pageId],
queryFn: () => getPageById(pageInput),
enabled: !!pageInput.pageId,
staleTime: 5 * 60 * 1000,
@@ -83,9 +56,9 @@ export function usePageQuery(
useEffect(() => {
if (query.data) {
if (isValidUuid(pageInput.pageId)) {
queryClient.setQueryData(pageKeys.detail(query.data.slugId), query.data);
queryClient.setQueryData(["pages", query.data.slugId], query.data);
} else {
queryClient.setQueryData(pageKeys.detail(query.data.id), query.data);
queryClient.setQueryData(["pages", query.data.id], query.data);
}
}
}, [query.data]);
@@ -96,10 +69,6 @@ export function usePageQuery(
export function useCreatePageMutation() {
const { t } = useTranslation();
return useMutation<IPage, Error, Partial<IPageInput>>({
// Stable key so a paused create restored from IndexedDB after an offline
// reload finds its default mutationFn (registerOfflineMutationDefaults) and
// is replayed by resumePausedMutations() on reconnect instead of being lost.
mutationKey: offlineMutationKeys.createPage,
mutationFn: (data) => createPage(data),
onSuccess: (data) => {
invalidateOnCreatePage(data);
@@ -111,20 +80,18 @@ export function useCreatePageMutation() {
}
export function updatePageData(data: IPage) {
const pageBySlug = queryClient.getQueryData<IPage>(
pageKeys.detail(data.slugId),
);
const pageById = queryClient.getQueryData<IPage>(pageKeys.detail(data.id));
const pageBySlug = queryClient.getQueryData<IPage>(["pages", data.slugId]);
const pageById = queryClient.getQueryData<IPage>(["pages", data.id]);
if (pageBySlug) {
queryClient.setQueryData(pageKeys.detail(data.slugId), {
queryClient.setQueryData(["pages", data.slugId], {
...pageBySlug,
...data,
});
}
if (pageById) {
queryClient.setQueryData(pageKeys.detail(data.id), { ...pageById, ...data });
queryClient.setQueryData(["pages", data.id], { ...pageById, ...data });
}
invalidateOnUpdatePage(
@@ -178,11 +145,11 @@ export function useRemovePageMutation() {
});
// Stamp deletedAt so a re-visit shows the trash banner, not stale state.
const cached = queryClient.getQueryData<IPage>(pageKeys.detail(pageId));
const cached = queryClient.getQueryData<IPage>(["pages", pageId]);
if (cached) {
const stamped = { ...cached, deletedAt: new Date() };
queryClient.setQueryData(pageKeys.detail(cached.id), stamped);
queryClient.setQueryData(pageKeys.detail(cached.slugId), stamped);
queryClient.setQueryData(["pages", cached.id], stamped);
queryClient.setQueryData(["pages", cached.slugId], stamped);
}
invalidateOnDeletePage(pageId);
@@ -221,9 +188,6 @@ export function useDeletePageMutation() {
export function useMovePageMutation() {
return useMutation<void, Error, IMovePage>({
// Stable key so a paused move restored from IndexedDB after an offline
// reload finds its default mutationFn and is replayed on reconnect.
mutationKey: offlineMutationKeys.movePage,
mutationFn: (data) => movePage(data),
});
}
@@ -303,11 +267,8 @@ export function useRestorePageMutation() {
// Replace would strip space/permissions/content and break the editor.
const merge = (cached: IPage | undefined) =>
cached ? { ...cached, ...restoredPage } : cached;
queryClient.setQueryData<IPage>(pageKeys.detail(restoredPage.id), merge);
queryClient.setQueryData<IPage>(
pageKeys.detail(restoredPage.slugId),
merge,
);
queryClient.setQueryData<IPage>(["pages", restoredPage.id], merge);
queryClient.setQueryData<IPage>(["pages", restoredPage.slugId], merge);
},
onError: (error) => {
notifications.show({
@@ -322,7 +283,7 @@ export function useGetSidebarPagesQuery(
data: SidebarPagesParams | null,
): UseInfiniteQueryResult<InfiniteData<IPagination<IPage>, unknown>> {
return useInfiniteQuery({
queryKey: pageKeys.sidebar(data),
queryKey: ["sidebar-pages", data],
enabled: !!data?.pageId || !!data?.spaceId,
queryFn: ({ pageParam }) =>
getSidebarPages({ ...data, cursor: pageParam, limit: 100 }),
@@ -333,7 +294,7 @@ export function useGetSidebarPagesQuery(
export function useGetRootSidebarPagesQuery(data: SidebarPagesParams) {
return useInfiniteQuery({
queryKey: pageKeys.rootSidebar(data.spaceId),
queryKey: ["root-sidebar-pages", data.spaceId],
queryFn: async ({ pageParam }) => {
return getSidebarPages({
spaceId: data.spaceId,
@@ -359,7 +320,7 @@ export function usePageBreadcrumbsQuery(
pageId: string,
): UseQueryResult<Partial<IPage[]>, Error> {
return useQuery({
queryKey: pageKeys.breadcrumbs(pageId),
queryKey: ["breadcrumbs", pageId],
queryFn: () => getPageBreadcrumbs(pageId),
enabled: !!pageId,
});
@@ -371,12 +332,10 @@ export async function fetchAllAncestorChildren(
// refresh (#159 #8), which must NOT receive the 30-min-cached children.
opts?: { fresh?: boolean },
) {
// not using a hook here, so we can call it inside a useEffect hook. Reuse the
// shared sidebarPagesQueryOptions (key + queryFn) so the offline warm path and
// this fetch never drift, but override staleTime for the `fresh` reconnect
// refresh (#159 #8), which must force a server refetch (staleTime 0).
// not using a hook here, so we can call it inside a useEffect hook
const response = await queryClient.fetchQuery({
...sidebarPagesQueryOptions(params),
queryKey: ["sidebar-pages", params],
queryFn: () => getAllSidebarPages(params),
staleTime: opts?.fresh ? 0 : 30 * 60 * 1000,
});
@@ -386,7 +345,7 @@ export async function fetchAllAncestorChildren(
export function useRecentChangesQuery(spaceId?: string) {
return useInfiniteQuery({
queryKey: pageKeys.recentChanges(spaceId),
queryKey: ["recent-changes", spaceId],
queryFn: ({ pageParam }) =>
getRecentChanges({ spaceId, cursor: pageParam, limit: 15 }),
initialPageParam: undefined as string | undefined,
@@ -457,12 +416,12 @@ export function invalidateOnCreatePage(data: Partial<IPage>) {
let queryKey: QueryKey = null;
if (data.parentPageId === null) {
queryKey = pageKeys.rootSidebar(data.spaceId);
queryKey = ["root-sidebar-pages", data.spaceId];
} else {
queryKey = pageKeys.sidebar({
pageId: data.parentPageId,
spaceId: data.spaceId,
});
queryKey = [
"sidebar-pages",
{ pageId: data.parentPageId, spaceId: data.spaceId },
];
}
//update all sidebar pages
@@ -522,7 +481,7 @@ export function invalidateOnCreatePage(data: Partial<IPage>) {
//update root sidebar pages haschildern
const rootSideBarMatches = queryClient.getQueriesData({
queryKey: pageKeys.rootSidebar(data.spaceId),
queryKey: ["root-sidebar-pages", data.spaceId],
exact: false,
});
@@ -546,7 +505,7 @@ export function invalidateOnCreatePage(data: Partial<IPage>) {
//update recent changes
queryClient.invalidateQueries({
queryKey: pageKeys.recentChanges(data.spaceId),
queryKey: ["recent-changes", data.spaceId],
});
}
@@ -560,9 +519,9 @@ export function invalidateOnUpdatePage(
invalidatePageTree();
let queryKey: QueryKey = null;
if (parentPageId === null) {
queryKey = pageKeys.rootSidebar(spaceId);
queryKey = ["root-sidebar-pages", spaceId];
} else {
queryKey = pageKeys.sidebar({ pageId: parentPageId, spaceId: spaceId });
queryKey = ["sidebar-pages", { pageId: parentPageId, spaceId: spaceId }];
}
//update all sidebar pages
queryClient.setQueryData<InfiniteData<IPagination<IPage>>>(
@@ -585,7 +544,7 @@ export function invalidateOnUpdatePage(
//update recent changes
queryClient.invalidateQueries({
queryKey: pageKeys.recentChanges(spaceId),
queryKey: ["recent-changes", spaceId],
});
}
@@ -600,8 +559,8 @@ export function updateCacheOnMovePage(
// Remove page from old parent's cache
const oldQueryKey =
oldParentId === null
? pageKeys.rootSidebar(spaceId)
: pageKeys.sidebar({ pageId: oldParentId, spaceId });
? ["root-sidebar-pages", spaceId]
: ["sidebar-pages", { pageId: oldParentId, spaceId }];
queryClient.setQueryData<InfiniteData<IPagination<IPage>>>(
oldQueryKey,
@@ -621,7 +580,7 @@ export function updateCacheOnMovePage(
if (oldParentId !== null) {
const oldParentCache = queryClient.getQueryData<
InfiniteData<IPagination<IPage>>
>(pageKeys.sidebar({ pageId: oldParentId, spaceId }));
>(["sidebar-pages", { pageId: oldParentId, spaceId }]);
const remainingChildren =
oldParentCache?.pages.flatMap((p) => p.items).length ?? 0;
@@ -659,8 +618,8 @@ export function updateCacheOnMovePage(
// Add page to new parent's cache
const newQueryKey =
newParentId === null
? pageKeys.rootSidebar(spaceId)
: pageKeys.sidebar({ pageId: newParentId, spaceId });
? ["root-sidebar-pages", spaceId]
: ["sidebar-pages", { pageId: newParentId, spaceId }];
queryClient.setQueryData<InfiniteData<IPagination<Partial<IPage>>>>(
newQueryKey,

View File

@@ -7,7 +7,6 @@ import { notifications } from "@mantine/notifications";
import {
IconArrowRight,
IconClockHour4,
IconCloudDownload,
IconCopy,
IconDotsVertical,
IconFileExport,
@@ -36,12 +35,6 @@ import {
useToggleTemplateMutation,
useToggleTemporaryMutation,
} from "@/features/page-embed/queries/page-embed-query";
import { useCollabToken } from "@/features/auth/queries/auth-query.tsx";
import { getCollaborationUrl } from "@/lib/config.ts";
import {
makePageAvailableOffline,
warmPageYdoc,
} from "@/features/offline/make-offline";
import { treeDataAtom } from "@/features/page/tree/atoms/tree-data-atom.ts";
import { treeModel } from "@/features/page/tree/model/tree-model";
import { pageToTreeNode } from "@/features/page/tree/utils";
@@ -79,46 +72,6 @@ export function NodeMenu({ node, canEdit }: NodeMenuProps) {
const isTemplate = !!node.isTemplate;
const toggleTemporary = useToggleTemporaryMutation();
const isTemporary = !!node.temporaryExpiresAt;
const { data: collabQuery } = useCollabToken();
const handleMakeAvailableOffline = async () => {
notifications.show({ message: t("Saving page for offline use...") });
try {
// Prefetch read queries so they get persisted to IndexedDB. The result
// reports whether every warm step succeeded.
const result = await makePageAvailableOffline({
pageId: node.id,
spaceId: node.spaceId,
});
// Best-effort: warm the page's Yjs document into IndexedDB.
await warmPageYdoc(node.id, getCollaborationUrl(), collabQuery?.token);
if (result.ok) {
notifications.show({ message: t("Page is now available offline") });
} else {
// Partial warm — the page may still be partly usable offline, but some
// queries failed to cache, so surface it as an error rather than a
// silent success. Name the failed step(s) (AGENTS.md: errors must be
// specific, never a bare generic string); `result.failed` carries them.
notifications.show({
message: `${t("Failed to make page available offline")}: ${result.failed.join(", ")}`,
color: "red",
});
}
} catch (err) {
// makePageAvailableOffline no longer throws, but warmPageYdoc and other
// unexpected failures stay guarded here. Log the raw error and surface the
// real cause to the user instead of a bare generic string (AGENTS.md).
console.error("handleMakeAvailableOffline failed", err);
const reason =
(err as { response?: { data?: { message?: string } } })?.response?.data
?.message ?? (err instanceof Error ? err.message : String(err));
notifications.show({
message: `${t("Failed to make page available offline")}: ${reason}`,
color: "red",
});
}
};
const handleToggleTemplate = async () => {
const next = !isTemplate;
@@ -275,17 +228,6 @@ export function NodeMenu({ node, canEdit }: NodeMenuProps) {
{t("Export")}
</Menu.Item>
<Menu.Item
leftSection={<IconCloudDownload size={16} />}
onClick={(e) => {
e.preventDefault();
e.stopPropagation();
handleMakeAvailableOffline();
}}
>
{t("Make available offline")}
</Menu.Item>
{canEdit && (
<>
<Menu.Item

View File

@@ -15,6 +15,7 @@ import {
useCreatePageMutation,
useRemovePageMutation,
useMovePageMutation,
useUpdatePageMutation,
updateCacheOnMovePage,
} from "@/features/page/queries/page-query.ts";
import { buildPageUrl } from "@/features/page/page.utils.ts";
@@ -26,6 +27,7 @@ export type UseTreeMutation = {
parentId: string | null,
opts?: { temporary?: boolean },
) => Promise<void>;
handleRename: (id: string, name: string) => Promise<void>;
handleDelete: (id: string) => Promise<void>;
};
@@ -37,6 +39,7 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
// children) and then immediately invokes a handler.
const store = useStore();
const createPageMutation = useCreatePageMutation();
const updatePageMutation = useUpdatePageMutation();
const removePageMutation = useRemovePageMutation();
const movePageMutation = useMovePageMutation();
const navigate = useNavigate();
@@ -202,6 +205,20 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
[spaceId, createPageMutation, setData, store, navigate, spaceSlug],
);
const handleRename = useCallback(
async (id: string, name: string) => {
setData((prev) =>
treeModel.update(prev, id, { name } as Partial<SpaceTreeNode>),
);
try {
await updatePageMutation.mutateAsync({ pageId: id, title: name });
} catch (error) {
console.error("Error updating page title:", error);
}
},
[updatePageMutation, setData],
);
const handleDelete = useCallback(
async (id: string) => {
const node = treeModel.find(
@@ -247,7 +264,7 @@ export function useTreeMutation(spaceId: string): UseTreeMutation {
[removePageMutation, setData, store, pageSlug, navigate, spaceSlug],
);
return { handleMove, handleCreate, handleDelete };
return { handleMove, handleCreate, handleRename, handleDelete };
}
function isPageInNode(node: SpaceTreeNode, pageSlug: string): boolean {

View File

@@ -0,0 +1,240 @@
import {
describe,
it,
expect,
vi,
beforeAll,
afterEach,
} from "vitest";
import {
render,
screen,
cleanup,
fireEvent,
waitFor,
} from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
// --- Mocks for the heavy / networked module graph ---------------------------
// EditSpaceForm wires the "Enable Git sync" Switch to a TanStack-Query mutation
// (useUpdateSpaceMutation). We mock ONLY that hook so the test fully controls
// mutateAsync (resolve / reject) and isPending, and stub i18n. The real Mantine
// Switch is rendered so the checkbox role / disabled state is meaningful.
// i18n: identity translator — labels stay as their English keys for queries.
vi.mock("react-i18next", () => ({
useTranslation: () => ({ t: (key: string) => key }),
}));
// Mutation hook: a controllable mutateAsync plus a togglable isPending.
const mutateAsync = vi.fn();
let isPending = false;
vi.mock("@/features/space/queries/space-query.ts", () => ({
useUpdateSpaceMutation: () => ({
mutateAsync,
get isPending() {
return isPending;
},
}),
}));
// jsdom lacks matchMedia, which MantineProvider's color-scheme hook needs.
beforeAll(() => {
if (!window.matchMedia) {
window.matchMedia = (query: string) =>
({
matches: false,
media: query,
onchange: null,
addListener: () => {},
removeListener: () => {},
addEventListener: () => {},
removeEventListener: () => {},
dispatchEvent: () => false,
}) as unknown as MediaQueryList;
}
});
import { EditSpaceForm } from "./edit-space-form";
import type { ISpace } from "@/features/space/types/space.types.ts";
function makeSpace(overrides: Partial<ISpace> = {}): ISpace {
return {
id: "space-1",
name: "Engineering",
description: "",
slug: "eng",
hostname: "host",
creatorId: "u1",
createdAt: new Date("2026-01-01"),
updatedAt: new Date("2026-01-01"),
...overrides,
} as ISpace;
}
function renderForm(props: { space: ISpace; readOnly?: boolean }) {
return render(
<MantineProvider>
<EditSpaceForm space={props.space} readOnly={props.readOnly} />
</MantineProvider>,
);
}
// The form now renders TWO switches (git-sync enable + auto-merge-conflicts) in
// that DOM order. Mantine renders each as an <input type="checkbox"
// role="switch"> but does NOT expose its label as the accessible name, so we
// disambiguate by DOM order (index 0 = enable, 1 = auto-merge) and assert the
// human-readable label text is present alongside.
function getToggle(): HTMLInputElement {
screen.getByText("Enable Git sync");
return screen.getAllByRole("switch")[0] as HTMLInputElement;
}
function getAutoMergeToggle(): HTMLInputElement {
screen.getByText("Auto-merge conflicts on push");
return screen.getAllByRole("switch")[1] as HTMLInputElement;
}
afterEach(() => {
cleanup();
mutateAsync.mockReset();
isPending = false;
});
describe("EditSpaceForm git-sync toggle", () => {
// Test 3: initial checked state derives from settings.gitSync.enabled ?? false.
it("derives initial checked state from space.settings.gitSync.enabled (true -> checked)", () => {
renderForm({
space: makeSpace({ settings: { gitSync: { enabled: true } } }),
});
expect(getToggle().checked).toBe(true);
});
it("defaults to unchecked when gitSync settings are missing", () => {
renderForm({ space: makeSpace() });
expect(getToggle().checked).toBe(false);
});
// Test 4: toggling fires the mutation with { spaceId, gitSyncEnabled } and
// optimistically flips the switch.
it("fires the mutation with the correct payload and optimistically flips on", async () => {
mutateAsync.mockResolvedValue(undefined);
renderForm({ space: makeSpace() });
const toggle = getToggle();
expect(toggle.checked).toBe(false);
fireEvent.click(toggle);
// Optimistic update: the switch reflects the new state immediately.
expect(toggle.checked).toBe(true);
expect(mutateAsync).toHaveBeenCalledTimes(1);
expect(mutateAsync).toHaveBeenCalledWith({
spaceId: "space-1",
gitSyncEnabled: true,
});
// Resolution leaves the toggle on.
await waitFor(() => expect(toggle.checked).toBe(true));
});
// Test 5: rollback on mutation error — the most valuable test.
it("rolls back the toggle to its prior state when the mutation rejects", async () => {
mutateAsync.mockRejectedValue(new Error("network"));
renderForm({
space: makeSpace({ settings: { gitSync: { enabled: false } } }),
});
const toggle = getToggle();
expect(toggle.checked).toBe(false);
fireEvent.click(toggle);
// Optimistically flips on before the rejection lands.
expect(toggle.checked).toBe(true);
expect(mutateAsync).toHaveBeenCalledWith({
spaceId: "space-1",
gitSyncEnabled: true,
});
// After the rejected promise settles, the component reverts to OFF so the
// user is not misled into believing sync is enabled.
await waitFor(() => expect(toggle.checked).toBe(false));
});
// Test 6: disabled when readOnly and when the mutation is pending.
it("disables the toggle when readOnly", () => {
renderForm({ space: makeSpace(), readOnly: true });
expect(getToggle().disabled).toBe(true);
});
it("disables the toggle while the mutation is pending", () => {
isPending = true;
renderForm({ space: makeSpace() });
expect(getToggle().disabled).toBe(true);
});
});
describe("EditSpaceForm auto-merge-conflicts toggle", () => {
it("derives initial checked state from space.settings.gitSync.autoMergeConflicts (true -> checked)", () => {
renderForm({
space: makeSpace({
settings: { gitSync: { autoMergeConflicts: true } },
}),
});
expect(getAutoMergeToggle().checked).toBe(true);
});
it("defaults to unchecked when autoMergeConflicts is missing (SAFE default)", () => {
renderForm({ space: makeSpace() });
expect(getAutoMergeToggle().checked).toBe(false);
});
it("fires the mutation with { spaceId, autoMergeConflicts } and optimistically flips on", async () => {
mutateAsync.mockResolvedValue(undefined);
renderForm({ space: makeSpace() });
const toggle = getAutoMergeToggle();
expect(toggle.checked).toBe(false);
fireEvent.click(toggle);
// Optimistic update.
expect(toggle.checked).toBe(true);
expect(mutateAsync).toHaveBeenCalledTimes(1);
expect(mutateAsync).toHaveBeenCalledWith({
spaceId: "space-1",
autoMergeConflicts: true,
});
await waitFor(() => expect(toggle.checked).toBe(true));
});
it("rolls back to its prior state when the mutation rejects", async () => {
mutateAsync.mockRejectedValue(new Error("network"));
renderForm({
space: makeSpace({
settings: { gitSync: { autoMergeConflicts: false } },
}),
});
const toggle = getAutoMergeToggle();
expect(toggle.checked).toBe(false);
fireEvent.click(toggle);
expect(toggle.checked).toBe(true);
expect(mutateAsync).toHaveBeenCalledWith({
spaceId: "space-1",
autoMergeConflicts: true,
});
await waitFor(() => expect(toggle.checked).toBe(false));
});
it("disables the toggle when readOnly", () => {
renderForm({ space: makeSpace(), readOnly: true });
expect(getAutoMergeToggle().disabled).toBe(true);
});
});

View File

@@ -1,5 +1,14 @@
import { Group, Box, Button, TextInput, Stack, Textarea } from "@mantine/core";
import React from "react";
import {
Group,
Box,
Button,
TextInput,
Stack,
Textarea,
Divider,
Switch,
} from "@mantine/core";
import React, { useState } from "react";
import { useForm } from "@mantine/form";
import { zod4Resolver } from "mantine-form-zod-resolver";
import { z } from "zod/v4";
@@ -29,6 +38,37 @@ export function EditSpaceForm({ space, readOnly }: EditSpaceFormProps) {
const { t } = useTranslation();
const updateSpaceMutation = useUpdateSpaceMutation();
const [gitSyncEnabled, setGitSyncEnabled] = useState<boolean>(
space?.settings?.gitSync?.enabled ?? false,
);
const [autoMergeConflicts, setAutoMergeConflicts] = useState<boolean>(
space?.settings?.gitSync?.autoMergeConflicts ?? false,
);
// One parameterized handler for both git-sync space toggles: they differ only by
// the local state setter, the mutation payload field, and the error label. The
// update is optimistic and reverts the local state on failure (the mutation
// surfaces a toast via onError; the raw error is still logged per AGENTS.md).
const handleToggle = async (
field: "gitSyncEnabled" | "autoMergeConflicts",
value: boolean,
previous: boolean,
setLocal: (next: boolean) => void,
errorLabel: string,
) => {
setLocal(value); // optimistic update
try {
await updateSpaceMutation.mutateAsync({
spaceId: space.id,
[field]: value,
});
} catch (err) {
setLocal(previous); // revert on failure
console.error(errorLabel, err);
}
};
const form = useForm<FormValues>({
validate: zod4Resolver(formSchema),
initialValues: {
@@ -104,6 +144,43 @@ export function EditSpaceForm({ space, readOnly }: EditSpaceFormProps) {
</Group>
)}
</form>
<Divider my="lg" />
<Switch
label={t("Enable Git sync")}
description={t("Sync this space's pages to a Git repository.")}
checked={gitSyncEnabled}
disabled={readOnly || updateSpaceMutation.isPending}
onChange={(event) =>
handleToggle(
"gitSyncEnabled",
event.currentTarget.checked,
gitSyncEnabled,
setGitSyncEnabled,
"Failed to toggle git-sync for space",
)
}
/>
<Switch
mt="md"
label={t("Auto-merge conflicts on push")}
description={t(
"When off (recommended), a page whose content still has unresolved Git conflict markers is skipped on push until you resolve the conflict in Git. When on, the markers are stripped and both sides' content is pushed.",
)}
checked={autoMergeConflicts}
disabled={readOnly || updateSpaceMutation.isPending}
onChange={(event) =>
handleToggle(
"autoMergeConflicts",
event.currentTarget.checked,
autoMergeConflicts,
setAutoMergeConflicts,
"Failed to toggle git-sync auto-merge-conflicts",
)
}
/>
</Box>
</>
);

View File

@@ -1,6 +1,5 @@
import {
keepPreviousData,
queryOptions,
useInfiniteQuery,
useMutation,
useQuery,
@@ -32,37 +31,11 @@ import { getRecentChanges } from "@/features/page/services/page-service.ts";
import { useEffect } from "react";
import { validate as isValidUuid } from "uuid";
/**
* Centralized React Query key factories for space queries. The hooks below and
* the offline warm path (features/offline/make-offline.ts) share these so the
* runtime keys can never silently drift apart.
*/
export const spaceKeys = {
detail: (idOrSlug: string) => ["space", idOrSlug] as const,
list: (params?: QueryParams) => ["spaces", params] as const,
members: (spaceId: string, query?: string) =>
["spaceMembers", spaceId, query] as const,
};
/**
* Shared queryOptions for fetching a space by id/slug. Both
* useGetSpaceBySlugQuery and the offline warm path consume this so the key,
* queryFn and staleTime stay identical. (`enabled` is intentionally omitted —
* prefetchQuery ignores it anyway and the warm path always passes a real id;
* the hook reapplies `enabled` itself.)
*/
export const spaceByIdQueryOptions = (spaceId: string) =>
queryOptions({
queryKey: spaceKeys.detail(spaceId),
queryFn: () => getSpaceById(spaceId),
staleTime: 5 * 60 * 1000,
});
export function useGetSpacesQuery(
params?: QueryParams,
): UseQueryResult<IPagination<ISpace>, Error> {
return useQuery({
queryKey: spaceKeys.list(params),
queryKey: ["spaces", params],
queryFn: () => getSpaces(params),
placeholderData: keepPreviousData,
refetchOnMount: true,
@@ -71,16 +44,16 @@ export function useGetSpacesQuery(
export function useSpaceQuery(spaceId: string): UseQueryResult<ISpace, Error> {
const query = useQuery({
queryKey: spaceKeys.detail(spaceId),
queryKey: ["space", spaceId],
queryFn: () => getSpaceById(spaceId),
enabled: !!spaceId,
});
useEffect(() => {
if (query.data) {
if (isValidUuid(spaceId)) {
queryClient.setQueryData(spaceKeys.detail(query.data.slug), query.data);
queryClient.setQueryData(["space", query.data.slug], query.data);
} else {
queryClient.setQueryData(spaceKeys.detail(query.data.id), query.data);
queryClient.setQueryData(["space", query.data.id], query.data);
}
}
}, [query.data]);
@@ -89,11 +62,8 @@ export function useSpaceQuery(spaceId: string): UseQueryResult<ISpace, Error> {
}
export const prefetchSpace = (spaceSlug: string, spaceId?: string) => {
// Note: intentionally NOT using spaceByIdQueryOptions here — that factory sets
// a 5min staleTime which would let this prefetch skip fetching fresh data;
// prefetchSpace must always refetch (default staleTime: 0).
queryClient.prefetchQuery({
queryKey: spaceKeys.detail(spaceSlug),
queryKey: ["space", spaceSlug],
queryFn: () => getSpaceById(spaceSlug),
});
@@ -130,8 +100,10 @@ export function useGetSpaceBySlugQuery(
spaceId: string,
): UseQueryResult<ISpace, Error> {
return useQuery({
...spaceByIdQueryOptions(spaceId),
queryKey: ["space", spaceId],
queryFn: () => getSpaceById(spaceId),
enabled: !!spaceId,
staleTime: 5 * 60 * 1000,
});
}
@@ -144,16 +116,14 @@ export function useUpdateSpaceMutation() {
onSuccess: (data, variables) => {
notifications.show({ message: t("Space updated successfully") });
const space = queryClient.getQueryData(
spaceKeys.detail(variables.spaceId),
) as ISpace;
const space = queryClient.getQueryData([
"space",
variables.spaceId,
]) as ISpace;
if (space) {
const updatedSpace = { ...space, ...data };
queryClient.setQueryData(
spaceKeys.detail(variables.spaceId),
updatedSpace,
);
queryClient.setQueryData(spaceKeys.detail(data.slug), updatedSpace);
queryClient.setQueryData(["space", variables.spaceId], updatedSpace);
queryClient.setQueryData(["space", data.slug], updatedSpace);
}
queryClient.invalidateQueries({
@@ -178,7 +148,7 @@ export function useDeleteSpaceMutation() {
if (variables.slug) {
queryClient.removeQueries({
queryKey: spaceKeys.detail(variables.slug),
queryKey: ["space", variables.slug],
exact: true,
});
}
@@ -186,7 +156,7 @@ export function useDeleteSpaceMutation() {
// Remove space-specific queries
if (variables.id) {
queryClient.removeQueries({
queryKey: spaceKeys.detail(variables.id),
queryKey: ["space", variables.id],
exact: true,
});
@@ -226,7 +196,7 @@ export function useSpaceMembersInfiniteQuery(
query?: string,
) {
return useInfiniteQuery({
queryKey: spaceKeys.members(spaceId, query),
queryKey: ["spaceMembers", spaceId, query],
queryFn: ({ pageParam }) =>
getSpaceMembers(spaceId, { cursor: pageParam, limit: 50, query }),
enabled: !!spaceId,

View File

@@ -13,9 +13,15 @@ export interface ISpaceCommentsSettings {
allowViewerComments?: boolean;
}
export interface ISpaceGitSyncSettings {
enabled?: boolean;
autoMergeConflicts?: boolean;
}
export interface ISpaceSettings {
sharing?: ISpaceSharingSettings;
comments?: ISpaceCommentsSettings;
gitSync?: ISpaceGitSyncSettings;
}
export interface ISpace {
@@ -35,6 +41,8 @@ export interface ISpace {
// for updates
disablePublicSharing?: boolean;
allowViewerComments?: boolean;
gitSyncEnabled?: boolean;
autoMergeConflicts?: boolean;
}
interface IMembership {

View File

@@ -2,19 +2,9 @@ import { useQuery, UseQueryResult } from "@tanstack/react-query";
import { getMyInfo } from "@/features/user/services/user-service";
import { ICurrentUser } from "@/features/user/types/user.types";
/**
* Centralized React Query key factory for current-user queries. This hook and
* the offline warm path (features/offline/make-offline.ts) share it so the
* runtime key can never silently drift, and the OFFLINE_PERSIST_ROOTS guard
* test can assert the persisted "currentUser" root maps to a real factory.
*/
export const userKeys = {
currentUser: () => ["currentUser"] as const,
};
export default function useCurrentUser(): UseQueryResult<ICurrentUser> {
return useQuery({
queryKey: userKeys.currentUser(),
queryKey: ["currentUser"],
queryFn: async () => {
return await getMyInfo();
},

View File

@@ -1,118 +0,0 @@
import { describe, it, expect, vi, beforeEach } from "vitest";
import { render, screen } from "@testing-library/react";
import { MantineProvider } from "@mantine/core";
import { MemoryRouter } from "react-router-dom";
import { HelmetProvider } from "react-helmet-async";
// Control useCurrentUser per test; stub the rest of UserProvider's network/
// socket dependencies so we only exercise its render-gating logic.
const h = vi.hoisted(() => ({ useCurrentUser: vi.fn() }));
vi.mock("@/features/user/hooks/use-current-user", () => ({
default: h.useCurrentUser,
}));
vi.mock("@/features/auth/queries/auth-query.tsx", () => ({
useCollabToken: () => ({ data: undefined }),
}));
vi.mock("@/features/websocket/use-query-subscription.ts", () => ({
useQuerySubscription: () => {},
}));
vi.mock("@/features/websocket/use-tree-socket.ts", () => ({
useTreeSocket: () => {},
}));
vi.mock("@/features/notification/hooks/use-notification-socket.ts", () => ({
useNotificationSocket: () => {},
}));
vi.mock("@/main.tsx", () => ({ queryClient: {} }));
vi.mock("@/features/user/connect-resync.ts", () => ({
makeConnectHandler: () => () => {},
}));
vi.mock("socket.io-client", () => ({
io: () => ({ on: vi.fn(), disconnect: vi.fn() }),
}));
vi.mock("react-i18next", () => ({
useTranslation: () => ({
t: (k: string) => k,
i18n: {
changeLanguage: vi.fn(),
language: "en-US",
resolvedLanguage: "en-US",
},
}),
}));
import { UserProvider } from "./user-provider";
const networkError = { message: "Network Error" }; // axios network error: no `response`
function renderProvider() {
return render(
<HelmetProvider>
<MemoryRouter>
<MantineProvider>
<UserProvider>
<div data-testid="app-child">app content</div>
</UserProvider>
</MantineProvider>
</MemoryRouter>
</HelmetProvider>,
);
}
beforeEach(() => {
h.useCurrentUser.mockReset();
});
describe("UserProvider offline render-gating", () => {
it("renders the app (cached children) when useCurrentUser errors offline but a cached user exists", () => {
// Offline reload: the persisted ['currentUser'] cache hydrates `data`, but
// the background POST /api/users/me refetch fails as a network error.
h.useCurrentUser.mockReturnValue({
data: {
user: { id: "u1", locale: "en" },
workspace: { id: "w1" },
},
isLoading: false,
error: networkError,
isError: true,
});
renderProvider();
// The cached app must render — NOT a blank fragment (#237/#238).
expect(screen.getByTestId("app-child")).toBeDefined();
expect(screen.queryByText("You're offline")).toBeNull();
});
it("renders the offline fallback (not a blank fragment) when erroring with no cached user", () => {
h.useCurrentUser.mockReturnValue({
data: undefined,
isLoading: false,
error: networkError,
isError: true,
});
const { container } = renderProvider();
// Previously this returned `<></>` — a blank white screen. Now it must show
// an explicit offline fallback.
expect(screen.getByText("You're offline")).toBeDefined();
expect(screen.queryByTestId("app-child")).toBeNull();
expect(container.textContent?.length).toBeGreaterThan(0);
});
it("renders the app normally on a successful currentUser load", () => {
h.useCurrentUser.mockReturnValue({
data: {
user: { id: "u1", locale: "en" },
workspace: { id: "w1" },
},
isLoading: false,
error: null,
isError: false,
});
renderProvider();
expect(screen.getByTestId("app-child")).toBeDefined();
});
});

View File

@@ -11,7 +11,6 @@ import { useTreeSocket } from "@/features/websocket/use-tree-socket.ts";
import { useNotificationSocket } from "@/features/notification/hooks/use-notification-socket.ts";
import { useCollabToken } from "@/features/auth/queries/auth-query.tsx";
import { Error404 } from "@/components/ui/error-404.tsx";
import { OfflineFallback } from "@/features/offline/offline-fallback.tsx";
import { queryClient } from "@/main.tsx";
import { makeConnectHandler } from "@/features/user/connect-resync.ts";
@@ -71,30 +70,14 @@ export function UserProvider({ children }: React.PropsWithChildren) {
document.documentElement.lang = i18n.resolvedLanguage || i18n.language || "en-US";
}, [i18n.language, i18n.resolvedLanguage]);
// First load with no cached user yet: render nothing briefly while the
// persisted ['currentUser'] cache hydrates (avoids flashing the offline
// fallback before restore). Once we have a user we render the app even if a
// refetch is still in flight.
if (isLoading && !data) return <></>;
if (isLoading) return <></>;
if (isError && error?.["response"]?.status === 404) {
return <Error404 />;
}
// We have a (possibly cached/stale) user — render the app. Offline, the
// POST /api/users/me refetch fails as a network error, but the persisted/
// hydrated user is enough to render the cached UI. Previously `if (error)
// return <></>` blanked every authenticated route on an offline reload even
// though the cached data was present (#237/#238).
if (data) {
return <>{children}</>;
}
// No user AND an error (offline cold boot of a page never warmed for offline,
// or no persisted cache to restore): show an explicit offline fallback rather
// than a blank white screen.
if (error) {
return <OfflineFallback />;
return <></>;
}
return <>{children}</>;

View File

@@ -11,8 +11,7 @@ import { MantineProvider } from "@mantine/core";
import { BrowserRouter } from "react-router-dom";
import { ModalsProvider } from "@mantine/modals";
import { Notifications } from "@mantine/notifications";
import { QueryClient, onlineManager } from "@tanstack/react-query";
import { PersistQueryClientProvider } from "@tanstack/react-query-persist-client";
import { QueryClient, QueryClientProvider } from "@tanstack/react-query";
import { HelmetProvider } from "react-helmet-async";
import "./i18n";
import { PostHogProvider } from "posthog-js/react";
@@ -22,13 +21,6 @@ import {
isCloud,
isPostHogEnabled,
} from "@/lib/config.ts";
import {
queryPersister,
shouldDehydrateOfflineQuery,
} from "@/features/offline/query-persister";
import { registerOfflineMutationDefaults } from "@/features/offline/offline-mutations";
import { PwaUpdatePrompt } from "@/pwa/pwa-update-prompt";
import { isCapacitorNativePlatform } from "@/pwa/is-capacitor";
import posthog from "posthog-js";
export const queryClient = new QueryClient({
@@ -38,30 +30,10 @@ export const queryClient = new QueryClient({
refetchOnWindowFocus: false,
retry: false,
staleTime: 5 * 60 * 1000,
// Keep cached read data around long enough to be persisted/restored for offline use.
gcTime: 1000 * 60 * 60 * 24,
},
},
});
// Register default mutationFns for the offline-relevant structural mutations so
// a paused mutation restored from IndexedDB after an offline reload still has a
// mutationFn and is replayed by resumePausedMutations() on reconnect (instead
// of silently no-op'ing and dropping the offline create/move/comment). MUST run
// before any resumePausedMutations() so rehydrated paused mutations have a fn.
registerOfflineMutationDefaults(queryClient);
// Seed TanStack Query's onlineManager from the REAL connectivity state at boot.
// It defaults to `online: true` and only flips on window online/offline events,
// so a tab that COLD-BOOTS offline would wrongly believe it is online: paused
// mutations restored from IndexedDB would never get a later offline->online
// transition to trigger their replay, and the offline UI affordances could not
// tell they are offline. Seeding here makes the first real `online` event a true
// transition that auto-resumes the rehydrated paused mutations (#120 data loss).
if (typeof navigator !== "undefined" && "onLine" in navigator) {
onlineManager.setOnline(navigator.onLine);
}
if (isCloud() && isPostHogEnabled) {
posthog.init(getPostHogKey(), {
api_host: getPostHogHost(),
@@ -78,44 +50,15 @@ root.render(
<BrowserRouter>
<MantineProvider theme={theme} cssVariablesResolver={mantineCssResolver}>
<ModalsProvider>
<PersistQueryClientProvider
client={queryClient}
persistOptions={{
persister: queryPersister,
maxAge: 1000 * 60 * 60 * 24,
buster: APP_VERSION,
dehydrateOptions: {
shouldDehydrateQuery: shouldDehydrateOfflineQuery,
},
}}
// After the persister finishes rehydrating, replay any paused
// mutations restored from IndexedDB. If we are back online this fires
// them immediately; if still offline they stay paused and TanStack's
// onlineManager auto-resumes them on the next online transition (which
// is now a true transition thanks to the onlineManager seeding above).
// Without this, a paused mutation persisted while offline and then
// reloaded would never resume and the user's work would be lost (#120).
onSuccess={() => {
queryClient.resumePausedMutations();
}}
>
<QueryClientProvider client={queryClient}>
<Notifications position="bottom-center" limit={3} zIndex={10000} />
{/* Skip SW registration inside the Capacitor native WebView — the
native shell serves assets itself; a browser SW would conflict. */}
{!isCapacitorNativePlatform() && <PwaUpdatePrompt />}
<HelmetProvider>
<PostHogProvider client={posthog}>
<App />
</PostHogProvider>
</HelmetProvider>
</PersistQueryClientProvider>
</QueryClientProvider>
</ModalsProvider>
</MantineProvider>
</BrowserRouter>,
);
// Service worker registration is owned by <PwaUpdatePrompt /> above (via
// vite-plugin-pwa's useRegisterSW: Workbox precache + prompt-based updates,
// and skipped inside the Capacitor native WebView). The earlier hand-written
// /sw.js registration from the mobile bootstrap was removed here to avoid a
// double registration / competing service worker.

View File

@@ -9,7 +9,6 @@ import { useGetSpaceBySlugQuery } from "@/features/space/queries/space-query.ts"
import { useTranslation } from "react-i18next";
import React from "react";
import { EmptyState } from "@/components/ui/empty-state.tsx";
import { OfflineFallback } from "@/features/offline/offline-fallback.tsx";
import { IconAlertTriangle, IconFileOff } from "@tabler/icons-react";
import { Button } from "@mantine/core";
import { Link } from "react-router-dom";
@@ -63,19 +62,7 @@ function PageContent({ pageSlug }: { pageSlug: string | undefined }) {
}
if (isError || !page) {
// An offline fetch of a page that was never saved for offline use yields a
// network error with NO HTTP status (status is undefined), which would
// otherwise fall through to the generic "Error fetching page data." state.
// When we are offline (or the failure is a network error with no status),
// show the dedicated "You're offline — this page isn't saved for offline"
// fallback instead, so the user understands why the page won't load.
const httpStatus = error?.["status"];
const isOffline =
typeof navigator !== "undefined" && navigator.onLine === false;
if (isOffline || (isError && httpStatus == null)) {
return <OfflineFallback />;
}
if ([401, 403, 404].includes(httpStatus)) {
if ([401, 403, 404].includes(error?.["status"])) {
return (
<EmptyState
icon={IconFileOff}

View File

@@ -1,39 +0,0 @@
import { describe, it, expect, afterEach } from "vitest";
import { isCapacitorNativePlatform } from "./is-capacitor";
describe("isCapacitorNativePlatform", () => {
afterEach(() => {
// Keep tests isolated from each other and from the rest of the suite.
delete (globalThis as any).Capacitor;
});
it("returns false when Capacitor is undefined", () => {
expect(isCapacitorNativePlatform()).toBe(false);
});
it("uses isNativePlatform() when it is a function", () => {
(globalThis as any).Capacitor = { isNativePlatform: () => true };
expect(isCapacitorNativePlatform()).toBe(true);
(globalThis as any).Capacitor = { isNativePlatform: () => false };
expect(isCapacitorNativePlatform()).toBe(false);
});
it("falls back to the boolean property when isNativePlatform is not a function", () => {
(globalThis as any).Capacitor = { isNativePlatform: true };
expect(isCapacitorNativePlatform()).toBe(true);
(globalThis as any).Capacitor = { isNativePlatform: false };
expect(isCapacitorNativePlatform()).toBe(false);
});
it("returns false when reading Capacitor throws (try/catch)", () => {
Object.defineProperty(globalThis, "Capacitor", {
configurable: true,
get() {
throw new Error("boom");
},
});
expect(isCapacitorNativePlatform()).toBe(false);
});
});

View File

@@ -1,23 +0,0 @@
/**
* Detects whether the client is running inside a Capacitor native WebView
* (native iOS/Android shell from the feature/mobile-app-bootstrap branch).
*
* This is a pure runtime check against the global `Capacitor` object that the
* native bridge injects — no `@capacitor/*` dependency is added. On the plain
* browser / installed-PWA path `window.Capacitor` is undefined, so this returns
* false and the Workbox service worker registers normally.
*
* Inside the native WebView the SW must NOT register: it would layer a redundant
* (and conflicting) cache over Capacitor's own asset serving and interfere with
* the native auth/CORS flow.
*/
export function isCapacitorNativePlatform(): boolean {
try {
const cap = (globalThis as any)?.Capacitor;
return !!(cap && typeof cap.isNativePlatform === "function"
? cap.isNativePlatform()
: cap?.isNativePlatform);
} catch {
return false;
}
}

View File

@@ -1,59 +0,0 @@
import { useEffect } from "react";
import { Button } from "@mantine/core";
import { notifications } from "@mantine/notifications";
import { useTranslation } from "react-i18next";
import { useRegisterSW } from "virtual:pwa-register/react";
// Stable notification id so we can show/hide a single update prompt.
const UPDATE_NOTIFICATION_ID = "pwa-update-available";
/**
* Listens for a waiting service worker and surfaces a Mantine notification
* prompting the user to reload into the new version.
*
* Must be mounted inside the Mantine provider subtree (Notifications must be
* available). Renders nothing itself.
*/
export function PwaUpdatePrompt() {
const { t } = useTranslation();
const {
needRefresh: [needRefresh],
updateServiceWorker,
} = useRegisterSW({
onRegisterError(error) {
// Best-effort: a failed registration must not break the app.
console.error("Service worker registration error:", error);
},
});
useEffect(() => {
if (!needRefresh) return;
notifications.show({
id: UPDATE_NOTIFICATION_ID,
title: t("Update available"),
message: (
<Button
size="xs"
variant="light"
mt="xs"
onClick={() => updateServiceWorker(true)}
>
{t("Reload")}
</Button>
),
autoClose: false,
withCloseButton: true,
});
// Hide the notification when the prompt is no longer needed / on cleanup.
return () => {
notifications.hide(UPDATE_NOTIFICATION_ID);
};
}, [needRefresh, t, updateServiceWorker]);
return null;
}
export default PwaUpdatePrompt;

View File

@@ -1,32 +0,0 @@
import { describe, it, expect } from "vitest";
import { isApiPath, isCollabOrSocketPath } from "./sw-strategy";
describe("isApiPath", () => {
it("matches the /api segment and its subtree", () => {
expect(isApiPath("/api")).toBe(true);
expect(isApiPath("/api/")).toBe(true);
expect(isApiPath("/api/pages")).toBe(true);
});
it("does not over-match sibling paths", () => {
expect(isApiPath("/apidocs")).toBe(false);
expect(isApiPath("/apixyz")).toBe(false);
expect(isApiPath("/")).toBe(false);
expect(isApiPath("/pages")).toBe(false);
});
});
describe("isCollabOrSocketPath", () => {
it("matches the /collab and /socket.io segments and their subtrees", () => {
expect(isCollabOrSocketPath("/collab")).toBe(true);
expect(isCollabOrSocketPath("/collab/x")).toBe(true);
expect(isCollabOrSocketPath("/socket.io")).toBe(true);
expect(isCollabOrSocketPath("/socket.io/abc")).toBe(true);
});
it("does not over-match sibling paths", () => {
expect(isCollabOrSocketPath("/collaborators")).toBe(false);
expect(isCollabOrSocketPath("/collabx")).toBe(false);
expect(isCollabOrSocketPath("/socket.iox")).toBe(false);
});
});

View File

@@ -1,32 +0,0 @@
/**
* Canonical service-worker routing predicates.
*
* IMPORTANT: With vite-plugin-pwa using Workbox `generateSW`, the
* `runtimeCaching[].urlPattern` functions are serialized standalone into the
* generated service worker and CANNOT reference imported symbols. The matching
* logic is therefore duplicated as inline regex literals in
* apps/client/vite.config.ts. This module is the testable source of truth, and
* the two MUST be kept in sync. This duplication is intentional and is the
* documented Workbox limitation.
*
* Matching is anchored to a path SEGMENT boundary (`^/<seg>(/|$)`) so that
* sibling paths like `/apidocs`, `/collaborators`, `/socket.iox` are NOT
* wrongly treated as API/realtime traffic.
*/
/**
* True when `pathname` is the `/api` segment or anything beneath it.
* `/api` and `/api/...` -> true; `/apidocs`, `/apixyz` -> false.
*/
export function isApiPath(pathname: string): boolean {
return /^\/api(\/|$)/.test(pathname);
}
/**
* True when `pathname` is the `/collab` or `/socket.io` segment (or beneath it).
* `/collab`, `/collab/x`, `/socket.io`, `/socket.io/abc` -> true;
* `/collaborators`, `/collabx`, `/socket.iox` -> false.
*/
export function isCollabOrSocketPath(pathname: string): boolean {
return /^\/(collab|socket\.io)(\/|$)/.test(pathname);
}

View File

@@ -1,4 +1,2 @@
/// <reference types="vite/client" />
/// <reference types="vite-plugin-pwa/react" />
/// <reference types="vite-plugin-pwa/info" />
declare const APP_VERSION: string

View File

@@ -1,6 +1,5 @@
import { defineConfig, loadEnv } from "vite";
import react from "@vitejs/plugin-react";
import { VitePWA } from "vite-plugin-pwa";
import * as path from "path";
import { execSync } from "node:child_process";
@@ -54,55 +53,7 @@ export default defineConfig(({ mode }) => {
},
APP_VERSION: JSON.stringify(resolveAppVersion(envPath)),
},
plugins: [
react(),
VitePWA({
registerType: "prompt",
injectRegister: null,
strategies: "generateSW",
manifest: false,
workbox: {
globPatterns: ["**/*.{js,css,html,svg,png,ico,woff2,json}"],
navigateFallback: "index.html",
// Segment-anchored (`^/<seg>(/|$)`) so navigation requests to these
// segments are consistently excluded from the SPA fallback, mirroring
// the runtimeCaching urlPattern regexes below.
//
// `/share`, `/mcp`, `/l`, and `/robots.txt` mirror the server
// static-serve exclude list (apps/server/src/main.ts setGlobalPrefix
// `exclude`): robots.txt, the SEO/OG/analytics-injected public share
// HTML, the embedded MCP endpoint, and the `l/:alias` vanity short-link
// (a server 302 to a share page) are served by server controllers, so
// the SW must never shadow them with the precached index.html app shell.
// For `/l/:alias` the client router has NO matching route, so serving
// the app shell would dead-end on Error404 and break the public link;
// it must reach the server to perform the redirect.
navigateFallbackDenylist: [
/^\/api(\/|$)/,
/^\/collab(\/|$)/,
/^\/socket\.io(\/|$)/,
/^\/share(\/|$)/,
/^\/mcp(\/|$)/,
/^\/l(\/|$)/,
/^\/robots\.txt$/,
],
cleanupOutdatedCaches: true,
clientsClaim: true,
// The urlPattern regexes below mirror apps/client/src/pwa/sw-strategy.ts
// and MUST be kept in sync with it. Workbox `generateSW` serializes these
// functions standalone into the generated service worker, so they cannot
// import the module — the matching logic is intentionally duplicated as
// self-contained inline regex literals anchored to a path segment boundary.
runtimeCaching: [
{ urlPattern: ({ url }) => /^\/(collab|socket\.io)(\/|$)/.test(url.pathname), handler: "NetworkOnly" },
// All /api stays network-only; offline reads come from the persisted
// React Query cache (IndexedDB) + y-indexeddb, not the SW HTTP cache.
{ urlPattern: ({ url }) => /^\/api(\/|$)/.test(url.pathname), handler: "NetworkOnly" },
],
},
devOptions: { enabled: false },
}),
],
plugins: [react()],
build: {
rolldownOptions: {
output: {

View File

@@ -23,7 +23,7 @@
"migration:reset": "tsx src/database/migrate.ts down-to NO_MIGRATIONS",
"migration:codegen": "kysely-codegen --dialect=postgres --camel-case --env-file=../../.env --out-file=./src/database/types/db.d.ts",
"lint": "eslint \"{src,apps,libs,test}/**/*.ts\" --fix",
"pretest": "pnpm --filter @docmost/editor-ext build",
"pretest": "pnpm --filter @docmost/editor-ext build && pnpm --filter @docmost/git-sync build && pnpm --filter @docmost/mcp build",
"test": "jest",
"test:int": "jest --config test/jest-integration.json",
"test:watch": "jest --watch",
@@ -41,6 +41,7 @@
"@aws-sdk/s3-request-presigner": "3.1050.0",
"@azure/storage-blob": "12.31.0",
"@clickhouse/client": "^1.18.2",
"@docmost/git-sync": "workspace:*",
"@docmost/mcp": "workspace:*",
"@docmost/pdf-inspector": "1.9.6",
"@fastify/cookie": "^11.0.2",
@@ -64,7 +65,6 @@
"@nestjs/platform-fastify": "^11.1.19",
"@nestjs/platform-socket.io": "^11.1.19",
"@nestjs/schedule": "^6.1.3",
"@nestjs/swagger": "^11.2.0",
"@nestjs/terminus": "^11.1.1",
"@nestjs/throttler": "^6.5.0",
"@nestjs/websockets": "^11.1.19",
@@ -189,7 +189,12 @@
]
}
],
"^.+\\.(t|j)sx?$": "ts-jest"
"^.+\\.(t|j)sx?$": [
"ts-jest",
{
"isolatedModules": true
}
]
},
"transformIgnorePatterns": [
"/node_modules/(?!(\\.pnpm/)?(nanoid|uuid|image-dimensions|marked|happy-dom|lib0)(@|/))"
@@ -199,11 +204,17 @@
],
"coverageDirectory": "../coverage",
"testEnvironment": "node",
"setupFiles": [
"<rootDir>/../test/jest.setup.ts"
],
"moduleNameMapper": {
"^@docmost/db/(.*)$": "<rootDir>/database/$1",
"^@docmost/transactional/(.*)$": "<rootDir>/integrations/transactional/$1",
"^@docmost/ee/(.*)$": "<rootDir>/ee/$1",
"^src/(.*)$": "<rootDir>/$1"
"^src/(.*)$": "<rootDir>/$1",
"^@docmost/git-sync$": "<rootDir>/../../../packages/git-sync/src/index.ts",
"^@docmost/git-sync/(.*)$": "<rootDir>/../../../packages/git-sync/src/$1",
"^(\\.{1,2}/.*)\\.js$": "$1"
}
}
}

View File

@@ -28,6 +28,7 @@ import { ClsModule } from 'nestjs-cls';
import { NoopAuditModule } from './integrations/audit/audit.module';
import { ThrottleModule } from './integrations/throttle/throttle.module';
import { McpModule } from './integrations/mcp/mcp.module';
import { GitSyncModule } from './integrations/git-sync/git-sync.module';
import { AiModule } from './integrations/ai/ai.module';
import { AiChatModule } from './core/ai-chat/ai-chat.module';
@@ -89,6 +90,7 @@ try {
TelemetryModule,
ThrottleModule,
McpModule,
GitSyncModule,
AiModule,
AiChatModule,
...enterpriseModules,

View File

@@ -24,10 +24,7 @@ import { CollabWsAdapter } from './adapter/collab-ws.adapter';
import {
CollaborationHandler,
CollabEventHandlers,
writeTitleFragment,
} from './collaboration.handler';
import { User } from '@docmost/db/types/entity.types';
import * as Y from 'yjs';
@Injectable()
export class CollaborationGateway {
@@ -153,67 +150,42 @@ export class CollaborationGateway {
}
/**
* Write a new page title INTO the page's Yjs 'title' fragment, Redis-INDEPENDENT.
* Write a git-originated body into a page, applying the merge on the instance
* that OWNS the live Y.Doc so a connected editor CONVERGES on the change.
*
* Unlike the Redis-routed `handleYjsEvent` path — which routes through
* `redisSync?.handleEvent` and SILENTLY no-ops when Redis is disabled
* (COLLAB_DISABLE_REDIS=true → redisSync === null) — this goes straight
* through the local Hocuspocus `openDirectConnection`. The title sync
* therefore works in BOTH single-process (no Redis) and Redis-clustered
* deployments.
* git-sync must NOT use openDirectConnection directly for this: that opens the
* document on whichever instance/process runs git-sync (the API/worker). When
* an editor is connected to a DIFFERENT collab instance/process, that is a
* SEPARATE, detached Y.Doc — the merge lands in the detached doc and the DB,
* but the live editor never receives the Yjs update; its next debounced
* autosave then overwrites the DB with its stale state and SILENTLY REVERTS
* the git change (the data-loss bug). Routing through the custom-event channel
* runs the merge on the owning instance's shared Document, whose update is
* broadcast to every connection (handleUpdate), so the editor's CRDT converges
* on the merged result.
*
* openDirectConnection loads the doc from persistence when no editor is
* connected, so this works whether or not an editor is currently open: the
* clear+reseed lands on the loaded doc and is persisted by onStoreDocument.
*
* Provenance: when the caller is the agent, the actor/aiChatId are threaded
* into the connection `context` so onStoreDocument sees `context.actor ===
* 'agent'` for the resulting title store (mirrors the body/REST path). The
* resulting title store is usually a no-op anyway — PageService already wrote
* the same title to the page.title column, so onStoreDocument's
* `titleText !== page.title` guard skips the column write — but we wire the
* context for correctness regardless.
* Without redis there is a single instance, so the write runs locally — which
* is already the owning (and only) instance the editor is connected to.
*/
async writePageTitle(
pageId: string,
title: string,
context?: { user?: User; actor?: string; aiChatId?: string },
async writePageBody(
documentName: string,
payload: {
prosemirrorJson: unknown;
baseProsemirrorJson?: unknown;
userId: string;
},
): Promise<void> {
const documentName = `page.${pageId}`;
const connection = await this.hocuspocus.openDirectConnection(
documentName,
context ?? {},
);
try {
// Write the new title into the in-memory 'title' fragment AND capture the
// resulting full doc state so we can persist it directly below.
let ydocState: Buffer | null = null;
await connection.transact((doc) => {
writeTitleFragment(doc, title);
ydocState = Buffer.from(Y.encodeStateAsUpdate(doc));
});
// F1 (variant C): persist the 'title' fragment to `page.ydoc` DIRECTLY,
// bypassing onStoreDocument. PageService.update already wrote the new title
// to the page.title COLUMN before calling this, so onStoreDocument's no-op
// fast-path (titleText === column) would NOT persist the in-memory fragment
// on disconnect — leaving the stored ydoc with the OLD title, which a later
// body edit would then revert the column back to. Writing the ydoc here
// makes BOTH column and persisted fragment consistent (NEW = NEW).
//
// Safe with or without a live editor: the write is idempotent and carries
// no tree snapshot (no double broadcast); when an editor is connected, the
// normal onStoreDocument flow still persists the (superset) state later and
// the live clients receive the title change through the transact above.
if (ydocState) {
await this.persistenceExtension.persistTitleFragmentYdoc(
pageId,
ydocState,
);
}
} finally {
await connection.disconnect();
if (this.redisSync) {
await this.handleYjsEvent(
'gitSyncWriteBody',
documentName,
payload as any,
);
return;
}
await this.collabEventsService
.getHandlers(this.hocuspocus)
.gitSyncWriteBody(documentName, payload as any);
}
/*

View File

@@ -0,0 +1,262 @@
// Exercises the REAL `gitSyncWriteBody` collab handler (the owner-routed body
// write the data-loss fix introduces). The handler imports the editor graph via
// collaboration.util / yjs.util (tiptapExtensions -> editor-ext -> react-dom,
// unloadable under jest's node env, same coupling noted in
// gitmost-datasource.service.spec.ts), so we stub those + the transformer. The
// stubbed toYdoc builds paragraph blocks straight from the ProseMirror JSON so
// we can assert convergence on real text.
jest.mock('./collaboration.util', () => ({
tiptapExtensions: [],
getPageId: (name: string) => name.replace(/^page\./, ''),
prosemirrorNodeToYElement: jest.fn(),
}));
jest.mock('./yjs.util', () => ({
setYjsMark: jest.fn(),
updateYjsMarkAttribute: jest.fn(),
}));
jest.mock('@hocuspocus/transformer', () => {
const Yjs = require('yjs');
return {
TiptapTransformer: {
toYdoc: (json: any) => {
if (json?.__throw) throw new Error('boom: malformed doc');
const d = new Yjs.Doc();
const frag = d.getXmlFragment('default');
const blocks = (json?.content ?? []).map((node: any) => {
const el = new Yjs.XmlElement(node.type || 'paragraph');
const text = (node.content ?? [])
.map((t: any) => t.text ?? '')
.join('');
const t = new Yjs.XmlText();
if (text) t.insert(0, text);
el.insert(0, [t]);
return el;
});
if (blocks.length) frag.insert(0, blocks);
return d;
},
},
};
});
import * as Y from 'yjs';
import { CollaborationHandler } from './collaboration.handler';
const pmDoc = (...paras: string[]) => ({
type: 'doc',
content: paras.map((text) => ({
type: 'paragraph',
content: text ? [{ type: 'text', text }] : [],
})),
});
const texts = (frag: Y.XmlFragment): string[] =>
frag.toArray().map((el) =>
(el as Y.XmlElement)
.toArray()
.map((c) => (c as Y.XmlText).toString())
.join(''),
);
// Build a fake Hocuspocus whose openDirectConnection yields a DirectConnection
// over a REAL shared Document, with a connected "editor" doc that receives the
// shared doc's updates (modelling Document.handleUpdate's broadcast on the
// OWNING instance). Initial content carries live block ids; the editor starts
// fully synced with the shared doc.
function fakeHocuspocus(initial: { text: string; id: string }[]) {
const shared = new Y.Doc();
const frag = shared.getXmlFragment('default');
shared.transact(() => {
frag.insert(
0,
initial.map((s) => {
const el = new Y.XmlElement('paragraph');
el.setAttribute('id', s.id);
const t = new Y.XmlText();
if (s.text) t.insert(0, s.text);
el.insert(0, [t]);
return el;
}),
);
});
const editor = new Y.Doc();
Y.applyUpdate(editor, Y.encodeStateAsUpdate(shared));
// Broadcast relay: server-originated updates flow to the connected editor.
shared.on('update', (u: Uint8Array, origin: any) => {
if (origin !== 'editor') Y.applyUpdate(editor, u, 'server');
});
const openDirectConnection = jest.fn(async () => ({
// DirectConnection.transact runs the fn directly against the Document (no
// wrapping Y transaction), exactly like @hocuspocus/server.
transact: async (fn: (doc: Y.Doc) => void) => fn(shared),
disconnect: jest.fn(async () => undefined),
}));
return { hocuspocus: { openDirectConnection } as any, shared, editor };
}
describe('CollaborationHandler.gitSyncWriteBody (owner-routed body write)', () => {
it('converges a connected editor on the git change (no silent revert)', async () => {
const { hocuspocus, shared, editor } = fakeHocuspocus([
{ text: 'alpha', id: 'p1' },
{ text: 'beta', id: 'p2' },
]);
const handler = new CollaborationHandler();
const handlers = handler.getHandlers(hocuspocus);
// git changed block 1 beta -> beta2; base is the pre-change content.
await handlers.gitSyncWriteBody('page.x', {
prosemirrorJson: pmDoc('alpha', 'beta2'),
baseProsemirrorJson: pmDoc('alpha', 'beta'),
userId: 'svc-user',
});
// The shared (owning-instance) doc holds the merge...
expect(texts(shared.getXmlFragment('default'))).toEqual(['alpha', 'beta2']);
// ...and the connected editor CONVERGED via the broadcast (the bug would
// leave it on 'beta' and revert the page on its next autosave).
expect(texts(editor.getXmlFragment('default'))).toEqual(['alpha', 'beta2']);
});
it('preserves a concurrent edit to a DIFFERENT block (3-way, finding #2)', async () => {
const { hocuspocus, shared, editor } = fakeHocuspocus([
{ text: 'alpha', id: 'p1' },
{ text: 'beta', id: 'p2' },
]);
// The editor is actively editing block 0 while the push arrives.
const eFrag = editor.getXmlFragment('default');
editor.transact(
() => (eFrag.get(0) as Y.XmlElement).get(0) instanceof Y.XmlText &&
((eFrag.get(0) as Y.XmlElement).get(0) as Y.XmlText).insert(5, ' EDIT'),
'editor',
);
Y.applyUpdate(shared, Y.encodeStateAsUpdate(editor), 'editor');
const handler = new CollaborationHandler();
await handler.getHandlers(hocuspocus).gitSyncWriteBody('page.x', {
prosemirrorJson: pmDoc('alpha', 'beta2'),
baseProsemirrorJson: pmDoc('alpha', 'beta'),
userId: 'svc-user',
});
// Human's block-0 edit AND git's block-1 change both survive on the editor.
expect(texts(editor.getXmlFragment('default'))).toEqual([
'alpha EDIT',
'beta2',
]);
});
it('FLUSHES the pending debounced store BEFORE merging so an in-flight edit survives (finding #2)', async () => {
// QA #119 finding #2: the 3-way merge must run against the latest live-doc
// state. A concurrent UI edit that is still in-flight (the store is debounced)
// must be drained into the live doc BEFORE git merges, or git clean-applies and
// the edit is silently dropped — even on a DIFFERENT block. Model the drain via
// the pending-store flush: when it runs, the in-flight block-0 edit lands.
const shared = new Y.Doc();
const frag = shared.getXmlFragment('default');
shared.transact(() => {
frag.insert(
0,
[
{ text: 'alpha', id: 'p1' },
{ text: 'beta', id: 'p2' },
].map((s) => {
const el = new Y.XmlElement('paragraph');
el.setAttribute('id', s.id);
const t = new Y.XmlText();
t.insert(0, s.text);
el.insert(0, [t]);
return el;
}),
);
});
const order: string[] = [];
const debouncer = {
isDebounced: jest.fn(() => true),
executeNow: jest.fn(async () => {
order.push('flush');
// The in-flight client edit to block 0 only lands once the pending store
// is flushed (i.e. the event loop is drained) — BEFORE the merge.
shared.transact(() =>
((frag.get(0) as Y.XmlElement).get(0) as Y.XmlText).insert(5, ' EDIT'),
);
}),
};
const openDirectConnection = jest.fn(async () => ({
transact: async (fn: (doc: Y.Doc) => void) => {
order.push('merge');
fn(shared);
},
disconnect: jest.fn(async () => undefined),
}));
const hocuspocus = { openDirectConnection, debouncer } as any;
const handler = new CollaborationHandler();
await handler.getHandlers(hocuspocus).gitSyncWriteBody('page.x', {
prosemirrorJson: pmDoc('alpha', 'beta2'), // git changes block 1
baseProsemirrorJson: pmDoc('alpha', 'beta'),
userId: 'svc-user',
});
// The flush ran, and it ran BEFORE the merge transaction.
expect(debouncer.executeNow).toHaveBeenCalledTimes(1);
expect(order).toEqual(['flush', 'merge']);
// Both the in-flight block-0 edit and git's block-1 change survive — the
// pre-flush bug would have produced ['alpha', 'beta2'] (UI edit dropped).
expect(texts(shared.getXmlFragment('default'))).toEqual([
'alpha EDIT',
'beta2',
]);
});
it('does not flush when no store is pending (isDebounced false)', async () => {
const { hocuspocus, shared } = fakeHocuspocus([{ text: 'a', id: 'p1' }]);
const executeNow = jest.fn();
(hocuspocus as any).debouncer = {
isDebounced: jest.fn(() => false),
executeNow,
};
const handler = new CollaborationHandler();
await handler.getHandlers(hocuspocus).gitSyncWriteBody('page.x', {
prosemirrorJson: pmDoc('a', 'b'),
userId: 'svc-user',
});
expect(executeNow).not.toHaveBeenCalled();
expect(texts(shared.getXmlFragment('default'))).toEqual(['a', 'b']);
});
it('crash-safe: a transform failure never opens the connection or mutates the live doc', async () => {
const { hocuspocus, shared } = fakeHocuspocus([{ text: 'alpha', id: 'p1' }]);
const before = texts(shared.getXmlFragment('default'));
const handler = new CollaborationHandler();
await expect(
handler.getHandlers(hocuspocus).gitSyncWriteBody('page.x', {
prosemirrorJson: { __throw: true } as any,
userId: 'svc-user',
}),
).rejects.toThrow('boom');
// The incoming doc is built BEFORE opening the connection, so the throw
// happens first: the live doc is untouched and no connection was opened.
expect(hocuspocus.openDirectConnection).not.toHaveBeenCalled();
expect(texts(shared.getXmlFragment('default'))).toEqual(before);
});
it('falls back to a 2-way merge when no base is supplied', async () => {
const { hocuspocus, shared, editor } = fakeHocuspocus([
{ text: 'alpha', id: 'p1' },
]);
const handler = new CollaborationHandler();
await handler.getHandlers(hocuspocus).gitSyncWriteBody('page.x', {
prosemirrorJson: pmDoc('alpha', 'gamma'),
userId: 'svc-user',
});
expect(texts(shared.getXmlFragment('default'))).toEqual(['alpha', 'gamma']);
expect(texts(editor.getXmlFragment('default'))).toEqual(['alpha', 'gamma']);
});
});

View File

@@ -1,155 +0,0 @@
import * as Y from 'yjs';
import { TiptapTransformer } from '@hocuspocus/transformer';
import { writeTitleFragment } from './collaboration.handler';
import { CollaborationGateway } from './collaboration.gateway';
import {
buildTitleSeedYdoc,
jsonToText,
tiptapExtensions,
} from './collaboration.util';
// Read the plain text held in the doc's 'title' XmlFragment, the same way
// PersistenceExtension.onStoreDocument extracts it before writing page.title.
const readTitleText = (doc: Y.Doc): string => {
const titleJson = TiptapTransformer.fromYdoc(doc, 'title');
return titleJson ? jsonToText(titleJson).trim() : '';
};
describe('writeTitleFragment — the clear+seed title write (Bug 1)', () => {
it('replaces an OLD title fragment with EXACTLY the new title (no duplication)', () => {
// Seed the doc's 'title' fragment with an OLD title, like a real page.
const doc = new Y.Doc();
Y.applyUpdate(doc, Y.encodeStateAsUpdate(buildTitleSeedYdoc('Old Title')));
expect(readTitleText(doc)).toBe('Old Title');
writeTitleFragment(doc, 'New Title');
// The fragment must contain EXACTLY the new title — not "Old TitleNew Title"
// (append) or "New TitleNew Title" (duplication). A single heading node.
expect(readTitleText(doc)).toBe('New Title');
const titleJson = TiptapTransformer.fromYdoc(doc, 'title') as any;
expect(titleJson.content).toHaveLength(1);
expect(titleJson.content[0].type).toBe('heading');
});
it('seeds the title fragment when it started empty', () => {
const doc = new Y.Doc();
// Force the 'title' fragment to exist but be empty.
doc.getXmlFragment('title');
expect(readTitleText(doc)).toBe('');
writeTitleFragment(doc, 'First Title');
expect(readTitleText(doc)).toBe('First Title');
});
it('does not corrupt the body when rewriting the title', () => {
// A doc with both a body and an old title; the body must survive untouched.
const doc = new Y.Doc();
const bodyDoc = TiptapTransformer.toYdoc(
{
type: 'doc',
content: [
{ type: 'paragraph', content: [{ type: 'text', text: 'body text' }] },
],
},
'default',
tiptapExtensions,
);
Y.applyUpdate(doc, Y.encodeStateAsUpdate(bodyDoc));
Y.applyUpdate(doc, Y.encodeStateAsUpdate(buildTitleSeedYdoc('Old')));
writeTitleFragment(doc, 'New');
expect(readTitleText(doc)).toBe('New');
const bodyJson = TiptapTransformer.fromYdoc(doc, 'default');
expect(jsonToText(bodyJson)).toContain('body text');
});
});
describe('CollaborationGateway.writePageTitle — Redis-independent path', () => {
// Build a gateway with only its hocuspocus.openDirectConnection stubbed; the
// method must drive the clear+seed through that direct connection (NOT through
// redisSync), so the title write survives COLLAB_DISABLE_REDIS.
const makeGateway = (doc: Y.Doc) => {
const disconnect = jest.fn().mockResolvedValue(undefined);
const transact = jest.fn(async (fn: (d: Y.Doc) => void) => {
fn(doc);
});
const openDirectConnection = jest
.fn()
.mockResolvedValue({ transact, disconnect });
const gateway = Object.create(CollaborationGateway.prototype);
// redisSync is intentionally null — this is the no-Redis scenario.
gateway.redisSync = null;
gateway.hocuspocus = { openDirectConnection } as any;
// F1 (variant C): writePageTitle persists the 'title' fragment directly so a
// later body edit can't revert the rename (see title-rename-durability.spec).
const persistTitleFragmentYdoc = jest.fn().mockResolvedValue(undefined);
gateway.persistenceExtension = { persistTitleFragmentYdoc } as any;
return {
gateway,
openDirectConnection,
transact,
disconnect,
persistTitleFragmentYdoc,
};
};
it('writes the new title via openDirectConnection and disconnects', async () => {
const doc = new Y.Doc();
Y.applyUpdate(doc, Y.encodeStateAsUpdate(buildTitleSeedYdoc('Old Title')));
const { gateway, openDirectConnection, disconnect, persistTitleFragmentYdoc } =
makeGateway(doc);
await gateway.writePageTitle('page-1', 'New Title', { user: { id: 'u1' } });
expect(openDirectConnection).toHaveBeenCalledWith(
'page.page-1',
expect.objectContaining({ user: { id: 'u1' } }),
);
expect(readTitleText(doc)).toBe('New Title');
// The renamed fragment is persisted directly to page.ydoc (F1 variant C).
expect(persistTitleFragmentYdoc).toHaveBeenCalledWith(
'page-1',
expect.any(Buffer),
);
expect(disconnect).toHaveBeenCalledTimes(1);
});
it('threads agent provenance into the connection context', async () => {
const doc = new Y.Doc();
const { gateway, openDirectConnection } = makeGateway(doc);
await gateway.writePageTitle('page-1', 'Agent Title', {
user: { id: 'u1' },
actor: 'agent',
aiChatId: 'chat-1',
});
expect(openDirectConnection).toHaveBeenCalledWith(
'page.page-1',
expect.objectContaining({ actor: 'agent', aiChatId: 'chat-1' }),
);
});
it('disconnects even when the transaction throws', async () => {
const disconnect = jest.fn().mockResolvedValue(undefined);
const openDirectConnection = jest.fn().mockResolvedValue({
transact: jest.fn().mockRejectedValue(new Error('boom')),
disconnect,
});
const gateway = Object.create(CollaborationGateway.prototype);
gateway.redisSync = null;
gateway.hocuspocus = { openDirectConnection } as any;
await expect(
gateway.writePageTitle('page-1', 'X', {}),
).rejects.toThrow('boom');
expect(disconnect).toHaveBeenCalledTimes(1);
});
});

View File

@@ -2,47 +2,21 @@ import { Injectable, Logger } from '@nestjs/common';
import { Hocuspocus, Document } from '@hocuspocus/server';
import { TiptapTransformer } from '@hocuspocus/transformer';
import {
buildTitleSeedYdoc,
prosemirrorNodeToYElement,
tiptapExtensions,
} from './collaboration.util';
import { setYjsMark, updateYjsMarkAttribute, YjsSelection } from './yjs.util';
import * as Y from 'yjs';
import { User } from '@docmost/db/types/entity.types';
import {
mergeXmlFragments,
mergeXmlFragments3WayWithStats,
} from './merge/yjs-body-merge';
export type CollabEventHandlers = ReturnType<
CollaborationHandler['getHandlers']
>;
/**
* Clear+reseed the 'title' XmlFragment of `doc` so it holds EXACTLY `title`.
*
* Used by the gateway's direct `writePageTitle` method to write a new page
* title INTO the page's Yjs 'title' fragment. The title lives in the same
* Y.Doc as the body; onStoreDocument extracts it on every save, so a REST/MCP
* rename that only updated the page.title DB column would be reverted on the
* next collaborative save unless the Yjs 'title' fragment is kept in sync.
* The whole fragment is replaced (no merge/append),
* mirroring the 'replace' body path: the new title fully supersedes the old.
*
* DELIBERATE TRADE-OFF: because this does a FULL clear+replace of the 'title'
* fragment, a REST/MCP rename arriving while a user is actively editing the
* title in an open editor WILL overwrite that in-progress edit. This is
* acceptable — the title is a short, rarely-concurrently-edited field — and is
* preferable to leaving a stale Yjs title that onStoreDocument would revert the
* DB column to on the next save.
*/
export function writeTitleFragment(doc: Y.Doc, title: string): void {
const titleFragment = doc.getXmlFragment('title');
if (titleFragment.length > 0) {
titleFragment.delete(0, titleFragment.length);
}
const newTitleDoc = buildTitleSeedYdoc(title);
Y.applyUpdate(doc, Y.encodeStateAsUpdate(newTitleDoc));
}
@Injectable()
export class CollaborationHandler {
private readonly logger = new Logger(CollaborationHandler.name);
@@ -142,9 +116,130 @@ export class CollaborationHandler {
},
);
},
/**
* Git-sync body write, applied as a block-level MERGE into the LIVE doc on
* the instance that OWNS it (routed here via the custom-event channel —
* see CollaborationGateway.writePageBody). Running on the owning instance
* is what makes a connected editor CONVERGE: the merge mutates the shared
* Document, whose update is broadcast to every connection, so the editor's
* CRDT applies the git change instead of silently reverting it on its next
* autosave (the data-loss bug this fixes).
*
* With a `baseProsemirrorJson` (the last-synced common ancestor) it does a
* THREE-WAY merge — a block only the human changed is kept, a block only
* git changed is taken (conflicts -> git). Without a base it falls back to
* the 2-way merge.
*/
gitSyncWriteBody: async (
documentName: string,
payload: {
prosemirrorJson: any;
baseProsemirrorJson?: any;
userId: string;
},
) => {
const { prosemirrorJson, baseProsemirrorJson, userId } = payload;
// Build the incoming (and base) Yjs docs BEFORE opening the connection /
// touching the live doc. If a transform throws (a malformed/unsupported
// doc) we must NOT have mutated the live body — otherwise a conversion
// failure could leave the page empty (crash-safe conversion).
const targetDoc = TiptapTransformer.toYdoc(
prosemirrorJson,
'default',
tiptapExtensions,
);
const baseDoc =
baseProsemirrorJson != null
? TiptapTransformer.toYdoc(
baseProsemirrorJson,
'default',
tiptapExtensions,
)
: null;
// CONCURRENT-EDIT FLUSH (QA #119, finding #2). The 3-way merge below runs
// against the LIVE Y.Doc, so a concurrent UI edit is only preserved if it
// is already part of that doc. A user's edit is debounced before it lands
// (the editor batches; the collab store is debounced up to 10s), so the
// merge could otherwise run against a PRE-EDIT doc: git would then
// clean-apply (no same-block conflict detected) and the in-flight UI edit
// — even on a DIFFERENT block — would be silently dropped.
//
// Flushing the pending debounced store here (a) drains the event loop so a
// just-arrived client Yjs update is applied to the live doc BEFORE we
// merge, and (b) persists the live doc so the merge baseline is current
// even on the doc-reload-from-DB path. After the flush the merge sees the
// latest state, so an edit on a different block is MERGED (not overwritten)
// and a genuine same-block edit is detected as a conflict -> the
// boundary-snapshot in PersistenceExtension pins it to page history
// (recoverable) instead of vanishing silently.
await this.flushPendingStore(hocuspocus, documentName);
// actor:'git-sync' + the service user flow into PersistenceExtension
// (lastUpdatedSource='git-sync', lastUpdatedById=userId).
await this.withYdocConnection(
hocuspocus,
documentName,
{ actor: 'git-sync', user: { id: userId } },
(doc) => {
const liveFrag = doc.getXmlFragment('default');
const targetFrag = targetDoc.getXmlFragment('default');
if (baseDoc) {
const { conflicts } = mergeXmlFragments3WayWithStats(
liveFrag,
targetFrag,
baseDoc.getXmlFragment('default'),
);
// SAME-BLOCK conflict contract (SPEC §9): a block both the human
// and git changed resolves to GIT (deterministic). Make that
// OBSERVABLE rather than silent — log it. The losing human content
// is NOT destroyed: the persistence extension's boundary snapshot
// pins the pre-merge page state to history on this user->git-sync
// transition, so it stays recoverable.
if (conflicts > 0) {
this.logger.warn(
`git-sync merge for ${documentName}: ${conflicts} same-block ` +
`conflict(s) resolved to the git version; the prior page ` +
`state is preserved in page history (recoverable).`,
);
}
} else {
mergeXmlFragments(liveFrag, targetFrag);
}
},
);
},
};
}
/**
* Flush any pending DEBOUNCED store for `documentName` so the live Y.Doc and the
* DB are current BEFORE a git-sync merge reads them (QA #119, finding #2 —
* concurrent UI edit silently lost). Mirrors the PersistenceExtension.onDisconnect
* flush: only acts when a store is actually pending (`isDebounced`), runs the
* SAME scheduled payload (`executeNow`, preserving the edit's context/actor), and
* never throws — a flush failure must not abort the git-sync write. Awaiting it
* also drains the event loop, so a client Yjs update sitting in the socket buffer
* is applied to the live doc before the merge transaction runs.
*/
private async flushPendingStore(
hocuspocus: Hocuspocus,
documentName: string,
): Promise<void> {
const debounceId = `onStoreDocument-${documentName}`;
try {
const debouncer = (hocuspocus as any)?.debouncer;
if (!debouncer?.isDebounced?.(debounceId)) return;
await debouncer.executeNow(debounceId);
} catch (err) {
this.logger.warn(
`git-sync pre-merge flush failed for ${documentName}: ` +
(err instanceof Error ? err.message : String(err)),
);
}
}
async withYdocConnection(
hocuspocus: Hocuspocus,
documentName: string,

View File

@@ -1,12 +1,9 @@
import * as Y from 'yjs';
import { TiptapTransformer } from '@hocuspocus/transformer';
import {
getPageId,
isEmptyParagraphDoc,
jsonToNode,
prosemirrorNodeToYElement,
buildTitleSeedYdoc,
jsonToText,
} from './collaboration.util';
import { Node } from '@tiptap/pm/model';
@@ -244,43 +241,3 @@ describe('prosemirrorNodeToYElement', () => {
expect(element.get(1).get(0).toString()).toBe('two');
});
});
describe('buildTitleSeedYdoc', () => {
it('builds a level-1 heading carrying the title text', () => {
const doc = buildTitleSeedYdoc('Hello World');
const json: any = TiptapTransformer.fromYdoc(doc, 'title');
const first = json.content?.[0];
expect(first.type).toBe('heading');
expect(first.attrs.level).toBe(1);
expect(jsonToText(json).trim()).toBe('Hello World');
});
it('produces a non-empty title fragment for a non-empty title', () => {
const doc = buildTitleSeedYdoc('Some Title');
expect(doc.get('title', Y.XmlFragment).length).toBeGreaterThan(0);
});
it('produces a heading with no text child for an empty title', () => {
const doc = buildTitleSeedYdoc('');
const json: any = TiptapTransformer.fromYdoc(doc, 'title');
const first = json.content?.[0];
expect(first.type).toBe('heading');
// No text content for an empty title.
expect(first.content ?? []).toHaveLength(0);
expect(jsonToText(json).trim()).toBe('');
});
it('round-trips a title through build -> extract -> build -> extract', () => {
const title = 'Round Trip Title';
const doc1 = buildTitleSeedYdoc(title);
const text1 = jsonToText(TiptapTransformer.fromYdoc(doc1, 'title')).trim();
const doc2 = buildTitleSeedYdoc(text1);
const text2 = jsonToText(TiptapTransformer.fromYdoc(doc2, 'title')).trim();
expect(text1).toBe(title);
expect(text2).toBe(text1);
});
});

View File

@@ -59,7 +59,6 @@ import { generateHTML, generateJSON } from '../common/helpers/prosemirror/html';
import { Node, Schema } from '@tiptap/pm/model';
import * as Y from 'yjs';
import { Logger } from '@nestjs/common';
import { TiptapTransformer } from '@hocuspocus/transformer';
export const tiptapExtensions = [
StarterKit.configure({
@@ -144,34 +143,6 @@ export function jsonToText(tiptapJson: JSONContent) {
return generateText(tiptapJson, tiptapExtensions);
}
/**
* Build a standalone Y.Doc that holds ONLY the page title, in a dedicated Yjs
* fragment named exactly 'title' (the collaborative title-editor contract with
* the client). The ProseMirror shape is a doc with a single level-1 heading
* whose text is the title (empty title => heading with no text child).
*
* The encoded state of the returned doc can be merged into a body doc via
* `Y.applyUpdate(doc, Y.encodeStateAsUpdate(titleSeed))` to seed the title
* fragment for legacy pages. Seeding MUST be guarded by an emptiness check on
* the existing 'title' fragment to avoid the Yjs duplication trap.
*/
export function buildTitleSeedYdoc(title: string): Y.Doc {
return TiptapTransformer.toYdoc(
{
type: 'doc',
content: [
{
type: 'heading',
attrs: { level: 1 },
content: title ? [{ type: 'text', text: title }] : [],
},
],
},
'title',
tiptapExtensions,
);
}
export function jsonToNode(tiptapJson: JSONContent) {
const schema = getSchema(tiptapExtensions);
try {

View File

@@ -1,9 +1,3 @@
export const HISTORY_INTERVAL = 5 * 60 * 1000;
export const HISTORY_FAST_INTERVAL = 60 * 1000;
export const HISTORY_FAST_THRESHOLD = 5 * 60 * 1000;
// Redis pub/sub channel that bridges a PAGE_UPDATED tree snapshot (a title/icon
// rename) from the standalone collab process to the API process, which is the
// single broadcast authority. Imported by both halves of the bridge:
// PageTreeBridgePublisher (collab process) and PageTreeBridgeSubscriber (API process).
export const COLLAB_TREE_UPDATE_CHANNEL = 'collab:tree-update';

View File

@@ -0,0 +1,89 @@
import { PersistenceExtension } from './persistence.extension';
/**
* Regression for the QA #119 "loss-on-fast-close" data loss: editing a page then
* closing the tab within the collab debounce window (~3-18s) lost the edit
* because, with `unloadImmediately: false`, Hocuspocus does NOT flush the
* debounced onStoreDocument on a last-client disconnect. PersistenceExtension
* now flushes the pending store on the LAST disconnect (and only then).
*/
describe('PersistenceExtension.onDisconnect flush (loss-on-fast-close)', () => {
function makeExt(): PersistenceExtension {
// onDisconnect touches none of the injected deps; pass casts.
return new PersistenceExtension(
null as any,
null as any,
null as any,
null as any,
null as any,
null as any,
null as any,
null as any,
);
}
function makeData(opts: {
clientsCount: number;
isDebounced: boolean;
isLoading?: boolean;
}) {
const executeNow = jest.fn(async () => undefined);
const isDebounced = jest.fn(() => opts.isDebounced);
return {
executeNow,
isDebounced,
payload: {
clientsCount: opts.clientsCount,
context: {},
document: { isLoading: opts.isLoading ?? false } as any,
documentName: 'page.abc',
instance: { debouncer: { isDebounced, executeNow } } as any,
requestHeaders: {},
requestParameters: new URLSearchParams(),
socketId: 's',
} as any,
};
}
it('flushes the pending store when the LAST client disconnects', async () => {
const ext = makeExt();
const { executeNow, payload } = makeData({
clientsCount: 0,
isDebounced: true,
});
await ext.onDisconnect(payload);
expect(executeNow).toHaveBeenCalledTimes(1);
expect(executeNow).toHaveBeenCalledWith('onStoreDocument-page.abc');
});
it('does NOT flush while other editors remain connected', async () => {
const ext = makeExt();
const { executeNow, payload } = makeData({
clientsCount: 2,
isDebounced: true,
});
await ext.onDisconnect(payload);
expect(executeNow).not.toHaveBeenCalled();
});
it('does NOT write when nothing is pending (already persisted)', async () => {
const ext = makeExt();
const { executeNow, payload } = makeData({
clientsCount: 0,
isDebounced: false,
});
await ext.onDisconnect(payload);
expect(executeNow).not.toHaveBeenCalled();
});
it('does NOT flush a doc that is still loading (load error guard)', async () => {
const ext = makeExt();
const { executeNow, payload } = makeData({
clientsCount: 0,
isDebounced: true,
isLoading: true,
});
await ext.onDisconnect(payload);
expect(executeNow).not.toHaveBeenCalled();
});
});

View File

@@ -1,483 +1,223 @@
// Stub collaboration.util so importing the extension does not drag in the
// editor-ext -> @tiptap/react -> react-dom graph (unloadable under jest's node
// env, same coupling the gitmost-datasource / mcp specs document). The
// extension only calls getPageId, jsonToText and isEmptyParagraphDoc from it on
// the store path; tiptapExtensions is unused by onStoreDocument.
jest.mock('../collaboration.util', () => ({
tiptapExtensions: [],
getPageId: (name: string) => name.replace(/^page\./, ''),
jsonToText: () => 'text',
isEmptyParagraphDoc: () => false,
// The post-write mention extraction walks the doc via jsonToNode().descendants;
// return a node-like stub with no descendants so no mentions are produced
// (mention handling is out of scope here — we only assert provenance).
jsonToNode: () => ({ descendants: () => undefined }),
}));
// Control the Yjs<->JSON bridge: fromYdoc returns the "incoming" doc the writer
// is storing. We keep it distinct from the page's persisted content so the
// no-op guard (isDeepStrictEqual) never short-circuits the write.
const INCOMING_JSON = { type: 'doc', content: [{ type: 'paragraph' }, { t: 1 }] };
jest.mock('@hocuspocus/transformer', () => ({
TiptapTransformer: {
fromYdoc: jest.fn(() => INCOMING_JSON),
toYdoc: jest.fn(),
},
}));
// Run the executeTx callback inline with a passthrough trx.
jest.mock('@docmost/db/utils', () => ({
executeTx: jest.fn(async (_db: any, cb: any) => cb({} as any)),
}));
import * as Y from 'yjs';
import { TiptapTransformer } from '@hocuspocus/transformer';
import { PersistenceExtension } from './persistence.extension';
import { buildTitleSeedYdoc, tiptapExtensions } from '../collaboration.util';
import {
onChangePayload,
onStoreDocumentPayload,
} from '@hocuspocus/server';
// Direct instantiation with stub deps, mirroring the auth/env unit specs.
const bodyJson = {
type: 'doc',
content: [{ type: 'paragraph', content: [{ type: 'text', text: 'hello' }] }],
};
/**
* Provenance-precedence coverage for PersistenceExtension.onStoreDocument
* (test-strategy Module 4 / item #2): the contract `agent > git-sync > user`,
* plus the negative that a git-sync store does NOT pin a boundary history
* snapshot. We drive the precedence through the real public method (onChange to
* arm the sticky agent marker, then onStoreDocument), mocking the repos / db /
* Yjs bridge so no real database or collab server is needed. The store's
* persisted `lastUpdatedSource` and the saveHistory call are the observable
* outputs.
*/
describe('PersistenceExtension.onStoreDocument — provenance precedence (#2)', () => {
const DOCUMENT_NAME = 'page.page-1';
const PAGE_ID = 'page-1';
// Build a body Y.Doc with a known JSON, plus a monkey-patched broadcastStateless
// (the real Hocuspocus Document supplies it; a bare Y.Doc does not).
const buildDoc = () => {
const d: any = TiptapTransformer.toYdoc(bodyJson, 'default', tiptapExtensions);
d.broadcastStateless = jest.fn();
return d;
};
// `page.content` differs from INCOMING_JSON so the write is never skipped.
const persistedPage = (overrides?: { lastUpdatedSource?: string }) => ({
id: PAGE_ID,
slugId: 'slug-1',
spaceId: 'space-1',
workspaceId: 'ws-1',
creatorId: 'creator-1',
contributorIds: ['creator-1'],
content: { type: 'doc', content: [{ type: 'paragraph', content: [] }] },
lastUpdatedSource: overrides?.lastUpdatedSource ?? 'user',
createdAt: new Date(),
});
const cloneOut = (doc: any) =>
JSON.parse(JSON.stringify(TiptapTransformer.fromYdoc(doc, 'default')));
const addTitleFragment = (doc: any, title: string) =>
Y.applyUpdate(doc, Y.encodeStateAsUpdate(buildTitleSeedYdoc(title)));
describe('PersistenceExtension', () => {
let pageRepo: any;
let pageHistoryRepo: any;
let trx: any;
let db: any;
let aiQueue: any;
let historyQueue: any;
let notificationQueue: any;
let collabHistory: any;
let transclusionService: any;
let ext: PersistenceExtension;
beforeEach(() => {
pageRepo = {
findById: jest.fn(),
updatePage: jest.fn().mockResolvedValue(undefined),
const build = (pageOverrides?: { lastUpdatedSource?: string }) => {
const pageRepo = {
findById: jest.fn().mockResolvedValue(persistedPage(pageOverrides)),
updatePage: jest.fn().mockResolvedValue({ numUpdatedRows: 1n }),
};
pageHistoryRepo = {
findPageLastHistory: jest.fn(),
saveHistory: jest.fn(),
const pageHistoryRepo = {
// No prior snapshot -> humanBaselineMissing is true, so the ONLY thing
// gating the boundary snapshot in these tests is the source precedence.
findPageLastHistory: jest.fn().mockResolvedValue(null),
saveHistory: jest.fn().mockResolvedValue(undefined),
};
trx = {};
db = { transaction: () => ({ execute: (fn: any) => fn(trx) }) };
aiQueue = { add: jest.fn().mockResolvedValue(undefined) };
historyQueue = { add: jest.fn().mockResolvedValue(undefined) };
notificationQueue = { add: jest.fn().mockResolvedValue(undefined) };
collabHistory = { addContributors: jest.fn().mockResolvedValue(undefined) };
transclusionService = {
const aiQueue = { add: jest.fn().mockResolvedValue(undefined) };
const historyQueue = { add: jest.fn().mockResolvedValue(undefined) };
const notificationQueue = { add: jest.fn().mockResolvedValue(undefined) };
const collabHistory = {
addContributors: jest.fn().mockResolvedValue(undefined),
};
const transclusionService = {
syncPageTransclusions: jest.fn().mockResolvedValue(undefined),
syncPageReferences: jest.fn().mockResolvedValue(undefined),
syncPageTemplateReferences: jest.fn().mockResolvedValue(undefined),
};
ext = new PersistenceExtension(
const ext = new PersistenceExtension(
pageRepo as any,
pageHistoryRepo as any,
db as any,
{} as any, // db
aiQueue as any,
historyQueue as any,
notificationQueue as any,
collabHistory as any,
transclusionService as any,
);
return { ext, pageRepo, pageHistoryRepo, historyQueue };
};
// A real Y.Doc is required for Y.encodeStateAsUpdate(document); broadcastStateless
// is a no-op spy. The fromYdoc bridge is mocked, so the doc's contents are
// irrelevant to the JSON path.
const makeStorePayload = (context: any): onStoreDocumentPayload =>
({
documentName: DOCUMENT_NAME,
document: Object.assign(new Y.Doc(), {
broadcastStateless: jest.fn(),
}),
context,
}) as any;
const makeChangePayload = (actor: string): onChangePayload =>
({
documentName: DOCUMENT_NAME,
context: { user: { id: 'user-1' }, actor },
}) as any;
const sourceOf = (pageRepo: { updatePage: jest.Mock }) =>
pageRepo.updatePage.mock.calls[0][0].lastUpdatedSource;
it("tags 'user' for a plain write (no agent touch, no git-sync actor)", async () => {
const { ext, pageRepo } = build();
await ext.onStoreDocument(
makeStorePayload({ user: { id: 'user-1' }, actor: 'user' }),
);
expect(sourceOf(pageRepo)).toBe('user');
});
describe('seedTitleFragment', () => {
it('returns false for empty/whitespace/null titles', () => {
const doc = new Y.Doc();
expect((ext as any).seedTitleFragment(doc, '')).toBe(false);
expect((ext as any).seedTitleFragment(doc, ' ')).toBe(false);
expect((ext as any).seedTitleFragment(doc, null)).toBe(false);
});
it("tags 'git-sync' when the writer's actor is 'git-sync' and no agent touched the window", async () => {
const { ext, pageRepo } = build();
it('does NOT re-seed an existing non-empty title fragment', () => {
const doc = new Y.Doc();
addTitleFragment(doc, 'Existing');
await ext.onStoreDocument(
makeStorePayload({ user: { id: 'svc-user' }, actor: 'git-sync' }),
);
expect((ext as any).seedTitleFragment(doc, 'Other')).toBe(false);
const text = TiptapTransformer.fromYdoc(doc, 'title');
expect(JSON.stringify(text)).toContain('Existing');
expect(JSON.stringify(text)).not.toContain('Other');
});
it('seeds an empty fragment from a non-empty title and returns true', () => {
const doc = new Y.Doc();
expect((ext as any).seedTitleFragment(doc, 'Hello')).toBe(true);
const json: any = TiptapTransformer.fromYdoc(doc, 'title');
expect(JSON.stringify(json)).toContain('Hello');
});
it('returns false (defensive) when reading the fragment throws', () => {
const fakeDoc = {
get: () => {
throw new Error('boom');
},
};
expect((ext as any).seedTitleFragment(fakeDoc as any, 'X')).toBe(false);
});
expect(sourceOf(pageRepo)).toBe('git-sync');
});
describe('onStoreDocument', () => {
const basePage = (overrides: any) => ({
id: 'PAGE_ID',
slugId: 'slug',
spaceId: 'space',
parentPageId: null,
creatorId: 'creator',
contributorIds: ['creator'],
workspaceId: 'ws',
title: 'whatever',
content: null,
lastUpdatedSource: 'user',
createdAt: new Date().toISOString(),
...overrides,
});
it("keeps 'git-sync' for an explicit git-sync store even with a sticky agent marker (#14 loop-guard)", async () => {
const { ext, pageRepo } = build();
const context = { user: { id: 'u1', name: 'U', avatarUrl: null } };
// An agent edit landed earlier in the coalescing window (sticky marker),
// then a git-sync writer performs the store. Red-team finding #14: an
// EXPLICIT current-write actor is authoritative for THIS write, so the
// store must stay 'git-sync' — otherwise the PageChangeListener loop-guard
// (keyed on lastUpdatedSource === 'git-sync') fails to recognize git-sync's
// own write and re-exports it. Explicit 'agent' still wins (see below); the
// sticky marker only promotes a plain human writer to 'agent'.
await ext.onChange(makeChangePayload('agent'));
await ext.onStoreDocument(
makeStorePayload({ user: { id: 'svc-user' }, actor: 'git-sync' }),
);
it('no-op when neither body nor title changed', async () => {
const document = buildDoc();
const page = basePage({
content: cloneOut(document),
title: 'hello title',
});
pageRepo.findById.mockResolvedValue(page);
await ext.onStoreDocument({
documentName: 'page.PAGE_ID',
document,
context,
} as any);
expect(pageRepo.updatePage).not.toHaveBeenCalled();
expect(document.broadcastStateless).not.toHaveBeenCalled();
expect(collabHistory.addContributors).not.toHaveBeenCalled();
expect(transclusionService.syncPageTransclusions).not.toHaveBeenCalled();
expect(aiQueue.add).not.toHaveBeenCalled();
expect(historyQueue.add).not.toHaveBeenCalled();
});
it('title-only change persists the title without body side-effects', async () => {
const document = buildDoc();
addTitleFragment(document, 'New Title');
const page = basePage({
content: cloneOut(document),
title: 'Old Title',
});
pageRepo.findById.mockResolvedValue(page);
await ext.onStoreDocument({
documentName: 'page.PAGE_ID',
document,
context,
} as any);
expect(pageRepo.updatePage).toHaveBeenCalledTimes(1);
const call = pageRepo.updatePage.mock.calls[0];
expect(call[0].title).toBe('New Title');
expect(call[0].ydoc).toBeDefined();
expect(call[0].contributorIds).toBeDefined();
expect('content' in call[0]).toBe(false);
// Title-only must not touch the body-authorship provenance.
expect('lastUpdatedSource' in call[0]).toBe(false);
expect(call[1]).toBe('PAGE_ID');
expect(call[3].treeUpdate.title).toBe('New Title');
expect(collabHistory.addContributors).toHaveBeenCalledTimes(1);
expect(collabHistory.addContributors).toHaveBeenCalledWith(
'PAGE_ID',
expect.any(Array),
);
expect(document.broadcastStateless).toHaveBeenCalledTimes(1);
expect(transclusionService.syncPageTransclusions).not.toHaveBeenCalled();
expect(aiQueue.add).not.toHaveBeenCalled();
expect(historyQueue.add).not.toHaveBeenCalled();
});
it('an EMPTY title fragment does NOT overwrite a non-empty page.title (anti-corruption guard, Bug 2)', async () => {
// The client can momentarily seed the 'title' fragment as an EMPTY heading
// (hasTitleFragment true, extracted text '') before the real title syncs.
// Body is unchanged here, so the only candidate write is the title -> the
// guard must turn this into a full no-op (no updatePage, no broadcast).
const document = buildDoc();
addTitleFragment(document, ''); // empty heading: length > 0 but text ''
const page = basePage({
content: cloneOut(document),
title: 'Real Title',
});
pageRepo.findById.mockResolvedValue(page);
await ext.onStoreDocument({
documentName: 'page.PAGE_ID',
document,
context,
} as any);
// No write at all: the empty title is not authoritative and the body is
// unchanged, so onStoreDocument must take the no-op fast path.
expect(pageRepo.updatePage).not.toHaveBeenCalled();
expect(document.broadcastStateless).not.toHaveBeenCalled();
});
it('an EMPTY title fragment alongside a body change persists the body but NOT an empty title (anti-corruption guard, Bug 2)', async () => {
const document = buildDoc();
addTitleFragment(document, ''); // empty title fragment
const page = basePage({
content: { type: 'doc', content: [] }, // different body -> bodyChanged
title: 'Real Title',
});
pageRepo.findById.mockResolvedValue(page);
await ext.onStoreDocument({
documentName: 'page.PAGE_ID',
document,
context,
} as any);
expect(pageRepo.updatePage).toHaveBeenCalledTimes(1);
const call = pageRepo.updatePage.mock.calls[0];
// Body is persisted, but the title is NOT included (empty == not
// authoritative) and no tree update is broadcast for the title.
expect(call[0].content).toBeTruthy();
expect('title' in call[0]).toBe(false);
expect(call[3]).toBeUndefined();
});
it('body + title change persists both with full body side-effects', async () => {
const document = buildDoc();
addTitleFragment(document, 'New Title');
const page = basePage({
content: { type: 'doc', content: [] },
title: 'Old Title',
});
pageRepo.findById.mockResolvedValue(page);
await ext.onStoreDocument({
documentName: 'page.PAGE_ID',
document,
context,
} as any);
expect(pageRepo.updatePage).toHaveBeenCalledTimes(1);
const call = pageRepo.updatePage.mock.calls[0];
expect(call[0].content).toBeTruthy();
expect(call[0].title).toBe('New Title');
expect(call[0].ydoc).toBeDefined();
expect(call[0].lastUpdatedSource).toBe('user');
expect(call[3].treeUpdate).toBeDefined();
expect(transclusionService.syncPageTransclusions).toHaveBeenCalled();
expect(aiQueue.add).toHaveBeenCalled();
expect(historyQueue.add).toHaveBeenCalled();
expect(collabHistory.addContributors).toHaveBeenCalled();
expect(document.broadcastStateless).toHaveBeenCalled();
});
it('body-only change persists the body without a tree update', async () => {
const document = buildDoc();
const page = basePage({
content: { type: 'doc', content: [] },
title: 'whatever',
});
pageRepo.findById.mockResolvedValue(page);
await ext.onStoreDocument({
documentName: 'page.PAGE_ID',
document,
context,
} as any);
expect(pageRepo.updatePage).toHaveBeenCalledTimes(1);
const call = pageRepo.updatePage.mock.calls[0];
expect(call[0].content).toBeTruthy();
expect('title' in call[0]).toBe(false);
// No treeUpdate for a body-only save.
expect(call[3]).toBeUndefined();
expect(transclusionService.syncPageTransclusions).toHaveBeenCalled();
expect(aiQueue.add).toHaveBeenCalled();
expect(historyQueue.add).toHaveBeenCalled();
expect(document.broadcastStateless).toHaveBeenCalled();
});
expect(sourceOf(pageRepo)).toBe('git-sync');
});
describe('onLoadDocument', () => {
it('returns early (no DB read) when the document is not empty', async () => {
const document = { isEmpty: () => false };
const result = await ext.onLoadDocument({
documentName: 'page.PAGE_ID',
document,
} as any);
it("tags 'agent' when the storing writer itself is the agent (no prior onChange)", async () => {
const { ext, pageRepo } = build();
expect(result).toBeUndefined();
expect(pageRepo.findById).not.toHaveBeenCalled();
});
await ext.onStoreDocument(
makeStorePayload({ user: { id: 'agent-user' }, actor: 'agent' }),
);
it('returns undefined and does not persist when the page is null', async () => {
const document = { isEmpty: () => true };
pageRepo.findById.mockResolvedValue(null);
expect(sourceOf(pageRepo)).toBe('agent');
});
const result = await ext.onLoadDocument({
documentName: 'page.PAGE_ID',
document,
} as any);
// --- boundary snapshot for a git-sync store over a HUMAN baseline -----------
// SPEC §9 observable-loss guard (bug #2): a git-sync body write is a block-level
// 3-way merge whose same-block rule is "git wins". To keep a concurrent human
// edit RECOVERABLE rather than silently overwritten, a git-sync store over a
// prior NON-git-sync baseline pins that prior state to page history first —
// exactly like the agent path. So saveHistory MUST be called here.
it('DOES pin a boundary snapshot for a git-sync store over a prior human state', async () => {
const { ext, pageHistoryRepo } = build({ lastUpdatedSource: 'user' });
expect(result).toBeUndefined();
expect(pageRepo.updatePage).not.toHaveBeenCalled();
});
await ext.onStoreDocument(
makeStorePayload({ user: { id: 'svc-user' }, actor: 'git-sync' }),
);
it('seeds + persists under a lock when the persisted ydoc lacks a title fragment', async () => {
const src = TiptapTransformer.toYdoc(bodyJson, 'default', tiptapExtensions);
const page = {
id: 'PAGE_ID',
title: 'Legacy Title',
ydoc: Buffer.from(Y.encodeStateAsUpdate(src)),
content: null,
};
// Both the cheap pre-check and the locked re-read return the same row.
pageRepo.findById.mockResolvedValue(page);
expect(pageHistoryRepo.saveHistory).toHaveBeenCalledTimes(1);
});
const document = { isEmpty: () => true };
const result = await ext.onLoadDocument({
documentName: 'page.PAGE_ID',
document,
} as any);
// --- negative: a git-sync store over a git-sync baseline does NOT re-pin -----
// The boundary is pinned once on the transition INTO git-sync; a subsequent
// git-sync store over an already-git-sync baseline must not churn history.
it('does NOT re-pin a boundary snapshot for a git-sync store over a git-sync baseline', async () => {
const { ext, pageHistoryRepo } = build({ lastUpdatedSource: 'git-sync' });
// The locked re-read must take the row lock inside the tx.
const lockedReadCall = pageRepo.findById.mock.calls.find(
(c: any[]) => c[1]?.withLock,
);
expect(lockedReadCall).toBeDefined();
expect(lockedReadCall[1].trx).toBe(trx);
await ext.onStoreDocument(
makeStorePayload({ user: { id: 'svc-user' }, actor: 'git-sync' }),
);
expect(pageRepo.updatePage).toHaveBeenCalledTimes(1);
const call = pageRepo.updatePage.mock.calls[0];
expect(Buffer.isBuffer(call[0].ydoc)).toBe(true);
expect(call[1]).toBe('PAGE_ID');
// Persist must run inside the transaction.
expect(call[2]).toBe(trx);
expect(result).toBeTruthy();
});
expect(pageHistoryRepo.saveHistory).not.toHaveBeenCalled();
});
it('does NOT lock or persist when the ydoc already has a title fragment', async () => {
const src = TiptapTransformer.toYdoc(bodyJson, 'default', tiptapExtensions);
Y.applyUpdate(src, Y.encodeStateAsUpdate(buildTitleSeedYdoc('Has Title')));
const page = {
id: 'PAGE_ID',
title: 'Has Title',
ydoc: Buffer.from(Y.encodeStateAsUpdate(src)),
content: null,
};
pageRepo.findById.mockResolvedValue(page);
it('DOES pin a boundary snapshot for an agent store over a prior human state (control)', async () => {
// Confirms the negative above is meaningful: under the SAME mocks, an agent
// store over a 'user' baseline DOES trigger the boundary snapshot.
const { ext, pageHistoryRepo } = build({ lastUpdatedSource: 'user' });
const document = { isEmpty: () => true };
const result = await ext.onLoadDocument({
documentName: 'page.PAGE_ID',
document,
} as any);
await ext.onStoreDocument(
makeStorePayload({ user: { id: 'agent-user' }, actor: 'agent' }),
);
// Hot path: only the cheap lock-free read, no locked re-read, no write.
expect(pageRepo.findById).toHaveBeenCalledTimes(1);
expect(pageRepo.findById.mock.calls[0][1]?.withLock).toBeFalsy();
expect(pageRepo.updatePage).not.toHaveBeenCalled();
expect(result).toBeTruthy();
});
expect(pageHistoryRepo.saveHistory).toHaveBeenCalledTimes(1);
});
it('converts legacy content -> ydoc inside a tx and persists a {ydoc} Buffer', async () => {
const page = {
id: 'PAGE_ID',
title: 'T',
ydoc: null,
content: bodyJson,
};
pageRepo.findById.mockResolvedValue(page);
it('does NOT pin a boundary snapshot for a plain user store', async () => {
const { ext, pageHistoryRepo } = build({ lastUpdatedSource: 'user' });
const document = { isEmpty: () => true };
const result = await ext.onLoadDocument({
documentName: 'page.PAGE_ID',
document,
} as any);
await ext.onStoreDocument(
makeStorePayload({ user: { id: 'user-1' }, actor: 'user' }),
);
const lockedReadCall = pageRepo.findById.mock.calls.find(
(c: any[]) => c[1]?.withLock,
);
expect(lockedReadCall).toBeDefined();
expect(lockedReadCall[1].trx).toBe(trx);
expect(pageRepo.updatePage).toHaveBeenCalledTimes(1);
const call = pageRepo.updatePage.mock.calls[0];
expect(Buffer.isBuffer(call[0].ydoc)).toBe(true);
expect(call[2]).toBe(trx);
// The rebuilt doc carries the body.
expect(JSON.stringify(cloneOut(result))).toContain('hello');
});
it('SKIPS rebuild when the locked re-read shows the ydoc was already healed', async () => {
// Simulate a concurrent process: the cheap pre-check sees ydoc=null (legacy
// rebuild path), but by the time we hold the lock another process has
// already persisted a healthy ydoc. We must adopt it, not rebuild/clobber.
const healed = TiptapTransformer.toYdoc(
{ type: 'doc', content: [{ type: 'paragraph', content: [{ type: 'text', text: 'healed' }] }] },
'default',
tiptapExtensions,
);
Y.applyUpdate(healed, Y.encodeStateAsUpdate(buildTitleSeedYdoc('Healed Title')));
const healedYdoc = Buffer.from(Y.encodeStateAsUpdate(healed));
const preCheck = { id: 'PAGE_ID', title: 'T', ydoc: null, content: bodyJson };
const lockedRow = {
id: 'PAGE_ID',
title: 'Healed Title',
ydoc: healedYdoc,
content: bodyJson,
};
pageRepo.findById
.mockResolvedValueOnce(preCheck) // cheap pre-check
.mockResolvedValueOnce(lockedRow); // locked re-read
const document = { isEmpty: () => true };
const result = await ext.onLoadDocument({
documentName: 'page.PAGE_ID',
document,
} as any);
// The healthy ydoc had a title fragment already, so nothing was rebuilt or
// seeded -> no clobbering write.
expect(pageRepo.updatePage).not.toHaveBeenCalled();
// The returned doc is the healed body, NOT a fresh rebuild of bodyJson.
expect(JSON.stringify(cloneOut(result))).toContain('healed');
});
it('REJECTS the load when the rebuild persist fails (does not return an unpersisted doc)', async () => {
const page = { id: 'PAGE_ID', title: 'T', ydoc: null, content: bodyJson };
pageRepo.findById.mockResolvedValue(page);
pageRepo.updatePage.mockRejectedValue(new Error('db down'));
const errSpy = jest
.spyOn((ext as any).logger, 'error')
.mockImplementation(() => undefined);
const document = { isEmpty: () => true };
await expect(
ext.onLoadDocument({
documentName: 'page.PAGE_ID',
document,
} as any),
).rejects.toThrow('db down');
expect(errSpy).toHaveBeenCalled();
});
it('seed-only persist FAILURE returns the doc from the existing ydoc (no throw)', async () => {
const src = TiptapTransformer.toYdoc(bodyJson, 'default', tiptapExtensions);
const page = {
id: 'PAGE_ID',
title: 'Legacy Title',
ydoc: Buffer.from(Y.encodeStateAsUpdate(src)),
content: null,
};
pageRepo.findById.mockResolvedValue(page);
pageRepo.updatePage.mockRejectedValue(new Error('db down'));
const errSpy = jest
.spyOn((ext as any).logger, 'error')
.mockImplementation(() => undefined);
const document = { isEmpty: () => true };
const result = await ext.onLoadDocument({
documentName: 'page.PAGE_ID',
document,
} as any);
// Non-fatal: we fall back to the doc loaded from the existing page.ydoc.
expect(result).toBeTruthy();
expect(JSON.stringify(cloneOut(result))).toContain('hello');
expect(errSpy).toHaveBeenCalled();
});
expect(pageHistoryRepo.saveHistory).not.toHaveBeenCalled();
});
});

View File

@@ -2,6 +2,7 @@ import {
afterUnloadDocumentPayload,
Extension,
onChangePayload,
onDisconnectPayload,
onLoadDocumentPayload,
onStoreDocumentPayload,
} from '@hocuspocus/server';
@@ -9,7 +10,6 @@ import * as Y from 'yjs';
import { Injectable, Logger } from '@nestjs/common';
import { TiptapTransformer } from '@hocuspocus/transformer';
import {
buildTitleSeedYdoc,
getPageId,
isEmptyParagraphDoc,
jsonToText,
@@ -53,7 +53,17 @@ export function resolveSource(
stickyTouched: boolean,
contextActor?: string,
): ProvenanceSource {
return stickyTouched || contextActor === 'agent' ? 'agent' : 'user';
// An EXPLICIT current-write actor is authoritative for THIS write and wins
// over the sticky-agent fallback. Order: explicit 'agent' > explicit
// 'git-sync' > sticky agent marker > plain human 'user'. The git-sync case
// must NOT be masked by the sticky marker, or the PageChangeListener
// loop-guard (which keys on lastUpdatedSource === 'git-sync') would re-export
// git-sync's own writes (#14). Explicit agent still wins so a window that
// mixed an agent edit stays tagged 'agent'.
if (contextActor === 'agent') return 'agent';
if (contextActor === 'git-sync') return 'git-sync';
if (stickyTouched) return 'agent';
return 'user';
}
/**
@@ -117,10 +127,6 @@ export class PersistenceExtension implements Extension {
return;
}
// Cheap, lock-free pre-check (hot path stays lock-free). It tells us whether
// any heal (legacy rebuild and/or title seed) is needed; the heal itself
// re-reads the row FOR UPDATE and re-validates inside a transaction so it
// runs exactly once (see healUnderLock).
const page = await this.pageRepo.findById(pageId, {
includeContent: true,
includeYdoc: true,
@@ -132,70 +138,27 @@ export class PersistenceExtension implements Extension {
}
if (page.ydoc) {
this.logger.debug(`ydoc loaded from db: ${pageId}`);
const doc = new Y.Doc();
Y.applyUpdate(doc, new Uint8Array(page.ydoc));
const dbState = new Uint8Array(page.ydoc);
// Legacy pages persisted their title only in the `page.title` column; the
// ydoc has no 'title' fragment. Decide cheaply (no lock) whether a seed is
// needed by inspecting the loaded doc's 'title' fragment. A seed is needed
// only when that fragment is empty AND there is a non-empty column title.
let titleSeedNeeded = false;
try {
const titleFrag = doc.get('title', Y.XmlFragment);
titleSeedNeeded = titleFrag.length === 0 && !!page.title?.trim();
} catch (err) {
// A malformed title fragment must not break loading; skip the seed.
this.logger.warn(`failed to inspect title fragment: ${err?.['message']}`);
titleSeedNeeded = false;
}
if (!titleSeedNeeded) {
// Fully healthy: a ydoc with a title fragment (or nothing to seed).
this.logger.debug(`ydoc loaded from db: ${pageId}`);
return doc;
}
// SEED-ONLY heal: a valid page.ydoc already exists; we only need to add the
// title fragment. If the persist fails we must NOT hand out an unpersisted
// fresh-client-id seed (it could later duplicate the title), so we fall
// back to the healthy doc loaded from the EXISTING page.ydoc, without the
// seed. The title just won't render until a later successful heal —
// non-fatal, non-corrupting.
try {
return await this.healUnderLock(pageId);
} catch (err) {
this.logger.error(
`Failed to persist seeded ydoc for page ${pageId}; serving existing ydoc without title seed`,
err,
);
return doc;
}
Y.applyUpdate(doc, dbState);
return doc;
}
// NOTE (offline-sync M1, Goal 2): this per-load self-heal converts +
// title-seeds + persists every legacy page (content set, ydoc null) on its
// first open, which neutralizes the duplication trap incrementally. A
// proactive one-shot BATCH migration over all such pages could be added
// later, but it requires the tiptap schema + TiptapTransformer (Node/Yjs),
// which a Kysely SQL migration cannot run; no runnable-task/CLI convention
// exists in this repo yet, so we deliberately avoid a fragile migration.
//
// If no ydoc state in db, REBUILD a Y.Doc from the JSON in page.content under
// a row lock (see healUnderLock).
// if no ydoc state in db convert json in page.content to Ydoc.
if (page.content) {
// REBUILD heal: surface failures. If the persist fails we REFUSE the load
// (re-throw) rather than hand out an unpersisted fresh-client-id rebuild —
// returning it would re-arm the duplication trap. A transient DB failure
// means the client reconnects and retries: correctness over availability.
try {
return await this.healUnderLock(pageId);
} catch (err) {
this.logger.error(
`Failed to persist rebuilt ydoc for page ${pageId}; refusing load`,
err,
);
throw err;
}
this.logger.debug(`converting json to ydoc: ${pageId}`);
const ydoc = TiptapTransformer.toYdoc(
page.content,
'default',
tiptapExtensions,
);
Y.encodeStateAsUpdate(ydoc);
return ydoc;
}
this.logger.debug(`creating fresh ydoc: ${pageId}`);
@@ -203,122 +166,39 @@ export class PersistenceExtension implements Extension {
}
/**
* Serialize the legacy self-heal (rebuild from page.content and/or seed the
* title fragment, then persist) so it runs exactly ONCE per page, closing the
* Yjs duplication trap. Both TiptapTransformer.toYdoc and buildTitleSeedYdoc
* mint FRESH Yjs client-ids every call, so two concurrent rebuilds (the API
* process via openDirectConnection AND the standalone collab process both
* seeing `ydoc IS NULL`) could each persist a different-client-id state and let
* a long-offline client merge-and-duplicate. We prevent that by re-reading the
* row FOR UPDATE inside a transaction and re-validating state under the lock:
* whoever wins the lock heals; the loser observes the healthy `ydoc` and adopts
* it instead of rebuilding. The persist happens IN THE SAME TX, so a failed
* write rolls back and propagates out (the caller then decides refuse vs.
* fall-back).
*/
private async healUnderLock(pageId: string): Promise<Y.Doc> {
return executeTx(this.db, async (trx) => {
const locked = await this.pageRepo.findById(pageId, {
withLock: true,
includeContent: true,
includeYdoc: true,
trx,
});
const doc = new Y.Doc();
let rebuilt = false;
if (locked?.ydoc) {
// Another process already healed (or the page always had a ydoc): adopt
// the healthy persisted state, do NOT rebuild.
Y.applyUpdate(doc, new Uint8Array(locked.ydoc));
} else if (locked?.content) {
this.logger.debug(`converting json to ydoc: ${pageId}`);
const built = TiptapTransformer.toYdoc(
locked.content,
'default',
tiptapExtensions,
);
Y.applyUpdate(doc, Y.encodeStateAsUpdate(built));
rebuilt = true;
}
// else: no ydoc and no content -> a fresh empty doc.
// Idempotent, emptiness-guarded title seed (safe to call always).
const seeded = this.seedTitleFragment(doc, locked?.title ?? null);
if (rebuilt || seeded) {
// Persist IN THE SAME TX. If this throws, the tx rolls back and the
// error propagates out of executeTx to the caller.
await this.pageRepo.updatePage(
{ ydoc: Buffer.from(Y.encodeStateAsUpdate(doc)) },
pageId,
trx,
);
this.logger.debug(`persisted rebuilt/seeded ydoc: ${pageId}`);
}
return doc;
});
}
/**
* Seed the 'title' fragment of `doc` from the `page.title` column for legacy
* pages whose persisted ydoc has no title fragment yet.
* LOSS-ON-FAST-CLOSE FIX (QA #119). When the LAST editor disconnects, FLUSH any
* pending (debounced) store to the DB IMMEDIATELY instead of waiting out the
* up-to-10s `debounce` window.
*
* Guarded STRICTLY by emptiness: we only seed when the existing 'title'
* fragment is empty AND there is a non-empty column title. Seeding a non-empty
* fragment would re-introduce the Yjs duplication trap, so we never do it.
* Returns true when a seed was applied (so the caller can persist).
* Defensive: a malformed title must not break document loading.
* The collab server runs with `unloadImmediately: false` (collaboration.gateway),
* so on a last-client disconnect Hocuspocus does NOT flush the debounced
* onStoreDocument — it relies on the timer firing later. A quick edit-then-close
* (closing the tab within the debounce window, ~3-18s) therefore left the edit
* only in the soon-to-be-unloaded in-memory Y.Doc; meanwhile git-sync mirrored
* the STALE/empty DB body to the vault (the reported "59-byte frontmatter-only"
* data loss). Running the already-scheduled store now closes that window.
*
* Gated tightly so it never adds a redundant write: only on the LAST disconnect
* (`clientsCount === 0`), only for a fully-loaded doc, and only when a store is
* actually pending (`isDebounced`). `executeNow` runs the SAME payload Hocuspocus
* scheduled (preserving the edit's context/actor) and clears the timer.
*/
private seedTitleFragment(doc: Y.Doc, title: string | null): boolean {
const trimmed = (title ?? '').trim();
if (!trimmed) return false;
async onDisconnect(data: onDisconnectPayload) {
const { instance, document, documentName, clientsCount } = data;
if (clientsCount > 0) return;
if (!document || document.isLoading) return;
const debounceId = `onStoreDocument-${documentName}`;
if (!instance?.debouncer?.isDebounced(debounceId)) return;
try {
const titleFrag = doc.get('title', Y.XmlFragment);
if (titleFrag.length !== 0) return false;
const titleSeed = buildTitleSeedYdoc(title);
Y.applyUpdate(doc, Y.encodeStateAsUpdate(titleSeed));
this.logger.debug('seeded title fragment from page.title column');
return true;
await instance.debouncer.executeNow(debounceId);
} catch (err) {
this.logger.warn(`failed to seed title fragment: ${err?.['message']}`);
return false;
this.logger.error(
`onDisconnect flush failed for ${documentName}: ` +
(err instanceof Error ? err.message : String(err)),
);
}
}
/**
* Persist an already-encoded Y.Doc state directly to `page.ydoc`, mirroring the
* `pageRepo.updatePage({ ydoc })` write that onStoreDocument uses.
*
* Used by the gateway's writePageTitle (F1, variant C). A REST/MCP/agent rename
* with no live editor writes the new title into the in-memory 'title' fragment,
* but onStoreDocument's no-op fast-path (page.title column already equals the
* new title) does NOT persist that in-memory fragment, so the stored `page.ydoc`
* keeps the OLD title — and a later body edit then reverts the rename (loads the
* OLD fragment, sees it differs from the column, overwrites the column back to
* OLD). Writing the ydoc here keeps the persisted fragment consistent with the
* column so the rename survives.
*
* Broadcast-safe / no double broadcast: this carries no `treeUpdate`, so the
* tree WS + redis listeners (which gate on `treeUpdate`) do NOT re-broadcast the
* rename — only PageService.update's own PAGE_UPDATED does. The only extra
* side-effect is an idempotent search reindex.
*
* Idempotent and lock-free, so it is safe whether or not a live editor is
* connected: Yjs state is cumulative, so a concurrent onStoreDocument simply
* persists a superset of this state later.
*/
async persistTitleFragmentYdoc(
pageId: string,
ydocState: Buffer,
): Promise<void> {
await this.pageRepo.updatePage({ ydoc: ydocState }, pageId);
}
async onStoreDocument(data: onStoreDocumentPayload) {
const { documentName, document, context } = data;
@@ -336,38 +216,16 @@ export class PersistenceExtension implements Extension {
this.logger.warn('jsonToText' + err?.['message']);
}
// Title lives in the SAME Y.Doc as the body, in a dedicated 'title' fragment
// (the collaborative title-editor contract with the client). Extract it
// defensively: a malformed title fragment must NOT crash the document store.
// `hasTitleFragment` distinguishes "the doc actually carries a title
// fragment" from "legacy doc with no title fragment" — only the former may
// write page.title, so a legacy doc never clobbers the column with ''.
let titleText = '';
let hasTitleFragment = false;
try {
const titleFrag = document.get('title', Y.XmlFragment);
hasTitleFragment = !!titleFrag && titleFrag.length > 0;
if (hasTitleFragment) {
const titleJson = TiptapTransformer.fromYdoc(document, 'title');
titleText = titleJson ? jsonToText(titleJson).trim() : '';
}
} catch (err) {
this.logger.warn('title extraction: ' + err?.['message']);
hasTitleFragment = false;
}
let page: Page = null;
// Tracks whether the BODY ('default') changed in this store. The heavy
// body-only side-effects (transclusion sync, mentions, RAG, history) stay
// gated on this so a title-only change does not trigger them.
let bodyChanged = false;
// Tracks a successful title-only persist so the post-tx contributor folding
// (collabHistory.addContributors) runs for the title-only case too.
let titleOnlyPersisted = false;
const editingUserIds = this.consumeContributors(documentName);
// Sticky agent marker: 'agent' if any agent edit landed in this window, OR
// if the current writer is the agent (covers a store with no prior onChange
// agent event in the same window). §15 H2.
// Provenance precedence: agent > git-sync > user (see resolveSource). A
// 'git-sync' store is NOT given an immediate history snapshot — it is
// debounced like a human edit (a git-sync write is a block-level merge into
// the live doc, so it reads like an incremental human edit, not a bulk
// import that would warrant its own immediate snapshot).
const lastUpdatedSource = resolveSource(
this.consumeAgentTouched(documentName),
context?.actor,
@@ -397,80 +255,11 @@ export class PersistenceExtension implements Extension {
return;
}
bodyChanged = !isDeepStrictEqual(tiptapJson, page.content);
// Only a populated 'title' fragment may update page.title; compare
// against the current column value (treat null as '').
//
// ANTI-CORRUPTION GUARD (Bug 2): the client's collaborative title-editor
// can momentarily initialize the 'title' fragment as an EMPTY heading
// (so `hasTitleFragment` is true, but the extracted `titleText` is '')
// BEFORE the server's real-title seed has synced. Writing that '' would
// silently wipe a non-empty page.title to "untitled". A wiki page is
// never legitimately retitled to empty via this path, so we treat an
// empty extracted title as "not authoritative" and never persist it.
// The `titleText.length > 0` clause makes this guard apply to BOTH the
// title-only branch and the body+title branch below.
//
// DELIBERATE: this intentionally makes it impossible to retitle a page
// to EMPTY via the collab path — a wiki page is never legitimately
// empty-titled. If a non-empty-title rule ever needs relaxing or
// enforcing differently, the REST UpdatePageDto is the place to validate
// the title, not this collab guard.
const titleChanged =
hasTitleFragment &&
titleText.length > 0 &&
titleText !== (page.title ?? '');
// No-op fast path: neither body nor title changed.
if (!bodyChanged && !titleChanged) {
if (isDeepStrictEqual(tiptapJson, page.content)) {
page = null;
return;
}
// Title-only change: the body is unchanged, so skip the heavy body
// history/contributor logic and persist just the new title and the
// ydoc (the title fragment edit lives in the same ydoc). The early-skip
// used to drop this case entirely, losing the title change.
if (!bodyChanged) {
// Fold the window's editing users into contributors the same way the
// body branch does, so a user who edited ONLY the title is not dropped
// from page.contributorIds.
const contributorIds = Array.from(
new Set([
...(page.contributorIds || []),
...editingUserIds,
page.creatorId,
]),
);
await this.pageRepo.updatePage(
{
title: titleText,
ydoc: ydocState,
lastUpdatedById: context.user.id,
contributorIds,
// A title-only change is not a body-authorship transition; leave
// lastUpdatedSource/aiChatId untouched so the user->agent history
// boundary in the body branch is not bypassed.
},
pageId,
trx,
// Mirror PageService.update's tree snapshot so a collaborative rename
// propagates to other users' sidebar/breadcrumbs like the REST rename.
{
treeUpdate: {
id: pageId,
slugId: page.slugId,
spaceId: page.spaceId,
parentPageId: page.parentPageId ?? null,
title: titleText,
},
},
);
this.logger.debug(`Page title updated: ${pageId} - SlugId: ${page.slugId}`);
titleOnlyPersisted = true;
return;
}
let contributorIds = undefined;
try {
const existingContributors = page.contributorIds || [];
@@ -485,25 +274,41 @@ export class PersistenceExtension implements Extension {
//this.logger.debug('Contributors error:' + err?.['message']);
}
// Approach A — boundary snapshot before the agent's first edit.
// When this store is the agent's and the page's currently persisted
// state was authored by a human, pin that human state as its own
// history version BEFORE the agent overwrites it. `page` still holds the
// OLD content/provenance here, so saveHistory(page) captures the
// pre-agent state tagged 'user'. The agent's new content is snapshotted
// later by the debounced PAGE_HISTORY job ('agent'). Skip if the prior
// state is already agent-authored (boundary already pinned on the
// user->agent transition), if the page is effectively empty, or if the
// latest existing snapshot already equals this human state (avoid
// duplicates).
if (lastUpdatedSource === 'agent' && page.lastUpdatedSource !== 'agent') {
// Approach A — boundary snapshot before a MACHINE write overwrites a
// human (or other-source) baseline. When this store is from a machine
// source — the AGENT or GIT-SYNC — and the page's currently persisted
// state was authored by a DIFFERENT source, pin that prior state as its
// own history version BEFORE the machine write overwrites it. `page`
// still holds the OLD content/provenance here, so saveHistory(page)
// captures the pre-write state. The machine's new content is snapshotted
// later by the debounced PAGE_HISTORY job.
//
// For GIT-SYNC this is the OBSERVABLE-LOSS guard (SPEC §9 conflict
// contract): a git-sync body write is a block-level 3-way merge whose
// same-block rule is "git wins". Without this pin, a concurrent human
// edit to a block git also changed would be overwritten with NO trace.
// Pinning the pre-merge state here means the human's content is always
// RECOVERABLE via page history rather than silently lost — git still
// wins the live doc deterministically, but nothing is destroyed.
//
// Skip if the prior state was already authored by THIS machine source
// (boundary already pinned on the transition into it), if the page is
// effectively empty, or if the latest existing snapshot already equals
// the prior state (avoid duplicates).
const isMachineWrite =
lastUpdatedSource === 'agent' || lastUpdatedSource === 'git-sync';
if (isMachineWrite && page.lastUpdatedSource !== lastUpdatedSource) {
const lastHistory = await this.pageHistoryRepo.findPageLastHistory(
pageId,
{ includeContent: true, trx },
);
const humanBaselineMissing =
!lastHistory || !isDeepStrictEqual(lastHistory.content, page.content);
if (!isEmptyParagraphDoc(page.content as any) && humanBaselineMissing) {
!lastHistory ||
!isDeepStrictEqual(lastHistory.content, page.content);
if (
!isEmptyParagraphDoc(page.content as any) &&
humanBaselineMissing
) {
await this.pageHistoryRepo.saveHistory(page, {
contributorIds: page.contributorIds ?? undefined,
trx,
@@ -521,27 +326,9 @@ export class PersistenceExtension implements Extension {
lastUpdatedSource,
lastUpdatedAiChatId: context?.aiChatId ?? null,
contributorIds: contributorIds,
// Persist the title in the SAME transaction when the title fragment
// changed alongside the body.
...(titleChanged ? { title: titleText } : {}),
},
pageId,
trx,
// Mirror PageService.update's tree snapshot so a collaborative rename
// propagates to other users' sidebar/breadcrumbs like the REST rename.
// Only attach when the title actually changed; a body-only save must
// not trigger a tree broadcast.
titleChanged
? {
treeUpdate: {
id: pageId,
slugId: page.slugId,
spaceId: page.spaceId,
parentPageId: page.parentPageId ?? null,
title: titleText,
},
}
: undefined,
);
this.logger.debug(`Page updated: ${pageId} - SlugId: ${page.slugId}`);
@@ -562,8 +349,6 @@ export class PersistenceExtension implements Extension {
}
}
// `page` is truthy whenever anything was persisted (body OR title-only), so
// the page.updated broadcast fires for a title-only change too.
if (page) {
document.broadcastStateless(
JSON.stringify({
@@ -581,20 +366,11 @@ export class PersistenceExtension implements Extension {
: undefined,
}),
);
}
// Record the window's editing users in collab history for a title-only
// change too (the body branch does this below, gated on bodyChanged).
if (page && titleOnlyPersisted) {
await this.collabHistory.addContributors(pageId, editingUserIds);
}
// Body-only side-effects: skip them for a title-only change (body unchanged).
if (page && bodyChanged) {
await this.syncTransclusion(pageId, page.workspaceId, tiptapJson);
}
if (page && bodyChanged) {
if (page) {
await this.collabHistory.addContributors(pageId, editingUserIds);
const mentions = extractMentions(tiptapJson);

View File

@@ -0,0 +1,208 @@
// Regression coverage for the custom-event request/reply protocol in the
// RedisSyncExtension. git-sync routes its body write through a custom event
// (`gitSyncWriteBody`) which, when the target doc is owned by a DIFFERENT collab
// instance, runs REMOTELY inside `handleRedisMessage` on the owning instance. The
// remote handler can THROW (markdown->ProseMirror transform on a malformed body).
//
// Before the fix the throw was uncaught: (1) no `customEventComplete` reply was
// published, so the origin's awaiting promise only rejected after `customEventTTL`
// (~30s) as a generic 'TIMEOUT', and (2) an unhandledRejection escaped the async
// `messageBuffer` listener on the owning instance. These tests assert the throw is
// turned into an error-carrying reply that rejects the origin PROMPTLY with the
// real message, with the no-throw and local paths unchanged.
import { RedisSyncExtension } from './redis-sync.extension';
type Listener = (channel: Buffer, message: Buffer) => unknown;
// Minimal in-memory pub/sub + lock store shared across FakeRedis duplicates,
// modelling the two-instance topology (origin + owner) over one Redis.
class FakeRedisBus {
instances: FakeRedis[] = [];
locks = new Map<string, string>();
published: { channel: string; message: Buffer }[] = [];
register(inst: FakeRedis) {
this.instances.push(inst);
}
publish(channel: string, message: Buffer) {
this.published.push({ channel, message });
for (const inst of this.instances) {
if (!inst.subscribed.has(channel)) continue;
for (const listener of inst.messageListeners) {
// ioredis delivers async; `void` mirrors the production listener
// registration (`sub.on('messageBuffer', ...)`), whose rejection would
// surface as an unhandledRejection if the handler did not catch.
void listener(Buffer.from(channel), message);
}
}
}
}
class FakeRedis {
subscribed = new Set<string>();
messageListeners: Listener[] = [];
constructor(private bus: FakeRedisBus) {
bus.register(this);
}
duplicate() {
return new FakeRedis(this.bus);
}
subscribe(...channels: string[]) {
for (const c of channels) this.subscribed.add(c);
return Promise.resolve();
}
on(event: string, cb: any) {
if (event === 'messageBuffer') this.messageListeners.push(cb as Listener);
return this;
}
publish(channel: string, message: Buffer) {
this.bus.publish(channel, message);
return Promise.resolve(1);
}
// Models `SET key val PX ttl NX GET`: only writes when absent (NX); returns the
// previous value (GET) so the origin observes the owner already holding the lock.
set(key: string, val: string, ...args: any[]) {
const hasNX = args.includes('NX');
const hasGET = args.includes('GET');
const old = this.bus.locks.get(key) ?? null;
if (!hasNX || old === null) this.bus.locks.set(key, val);
return Promise.resolve(hasGET ? old : 'OK');
}
del(key: string) {
this.bus.locks.delete(key);
return Promise.resolve(1);
}
disconnect() {}
}
const pack = (m: any) => Buffer.from(JSON.stringify(m));
const unpack = (b: Buffer) => JSON.parse(b.toString());
function makeExtension(
bus: FakeRedisBus,
serverId: string,
customEvents: Record<string, (doc: string, payload: any) => Promise<any>>,
) {
const ext = new RedisSyncExtension({
redis: new FakeRedis(bus) as any,
pack: pack as any,
unpack: unpack as any,
serverId,
customEvents: customEvents as any,
customEventTTL: 30_000,
});
// Doc is NOT loaded on this instance -> handleEvent takes the remote/proxy path.
(ext as any).instance = { documents: new Map() };
return ext;
}
describe('RedisSyncExtension custom-event error propagation', () => {
let unhandled: unknown[];
let onUnhandled: (e: unknown) => void;
beforeEach(() => {
// Fake timers so the 30s TTL fallback timer never fires (and never dangles).
jest.useFakeTimers();
unhandled = [];
onUnhandled = (e) => unhandled.push(e);
process.on('unhandledRejection', onUnhandled);
});
afterEach(() => {
process.off('unhandledRejection', onUnhandled);
jest.useRealTimers();
});
const flush = async () => {
for (let i = 0; i < 10; i++) await Promise.resolve();
};
it('owner publishes an error-carrying reply (no unhandledRejection) when the remote handler throws', async () => {
const bus = new FakeRedisBus();
const owner = makeExtension(bus, 'owner', {
boom: async () => {
throw new Error('kaboom');
},
});
// Drive the remote branch directly, as if the origin's customEventStart arrived.
await (owner as any).handleRedisMessage(
Buffer.from('collabMsg:owner'),
pack({
type: 'customEventStart',
documentName: 'page.x',
eventName: 'boom',
payload: {},
replyTo: 'collabMsg:origin',
replyId: 7,
}),
);
await flush();
const replies = bus.published
.filter((p) => p.channel === 'collabMsg:origin')
.map((p) => unpack(p.message));
expect(replies).toHaveLength(1);
expect(replies[0]).toMatchObject({
type: 'customEventComplete',
replyId: 7,
error: 'kaboom',
});
expect(unhandled).toHaveLength(0);
});
it('origin rejects PROMPTLY with the real error (not a TTL TIMEOUT) when the remote handler throws', async () => {
const bus = new FakeRedisBus();
// Owner already holds the document lock.
bus.locks.set('collabLock:page.x', 'owner');
makeExtension(bus, 'owner', {
boom: async () => {
throw new Error('kaboom');
},
});
const origin = makeExtension(bus, 'origin', {
boom: async () => undefined,
});
const promise = (origin as any).handleEvent('boom', 'page.x', { foo: 1 });
// Attach a catch immediately so a rejection is never momentarily unhandled.
const settled = promise.then(
() => ({ ok: true as const }),
(e: unknown) => ({ ok: false as const, error: e }),
);
await flush();
// Resolves WITHOUT advancing any timer -> the 30s TIMEOUT fallback did not fire.
const result = await settled;
expect(result.ok).toBe(false);
expect((result as any).error).toBeInstanceOf(Error);
expect(((result as any).error as Error).message).toBe('kaboom');
expect(unhandled).toHaveLength(0);
});
it('origin resolves with the payload when the remote handler succeeds (unchanged behavior)', async () => {
const bus = new FakeRedisBus();
bus.locks.set('collabLock:page.x', 'owner');
makeExtension(bus, 'owner', {
ok: async (_doc: string, payload: any) => ({ echoed: payload }),
});
const origin = makeExtension(bus, 'origin', {
ok: async () => undefined,
});
const promise = (origin as any).handleEvent('ok', 'page.x', { foo: 1 });
await flush();
await expect(promise).resolves.toEqual({ echoed: { foo: 1 } });
expect(unhandled).toHaveLength(0);
});
});

View File

@@ -51,9 +51,15 @@ export class RedisSyncExtension<TCE extends CustomEvents> implements Extension {
private instance!: Hocuspocus;
private readonly customEvents: TCE;
private replyIdCounter: number = 0;
// @ts-ignore
private pendingReplies: Record<number, PromiseWithResolvers<any>['resolve']> =
{};
private pendingReplies: Record<
number,
{
// @ts-ignore
resolve: PromiseWithResolvers<any>['resolve'];
// @ts-ignore
reject: PromiseWithResolvers<any>['reject'];
}
> = {};
constructor(configuration: Configuration<TCE>) {
const {
@@ -176,25 +182,45 @@ export class RedisSyncExtension<TCE extends CustomEvents> implements Extension {
}
if (type === 'customEventStart') {
const { documentName, eventName, payload, replyTo, replyId } = msg;
const res = await this.handleEventLocally(
eventName as Extract<keyof TCE, string>,
documentName,
payload,
);
const reply: RSAMessageCustomEventComplete = {
type: 'customEventComplete',
replyId,
payload: res,
};
let reply: RSAMessageCustomEventComplete;
try {
const res = await this.handleEventLocally(
eventName as Extract<keyof TCE, string>,
documentName,
payload,
);
reply = {
type: 'customEventComplete',
replyId,
payload: res,
};
} catch (err) {
// The remote handler threw (e.g. the markdown->ProseMirror transform in
// gitSyncWriteBody can throw on a malformed body). Reply with the error on
// the SAME correlation channel so the origin rejects promptly with the real
// message instead of waiting out customEventTTL as a generic 'TIMEOUT'.
// Catching here also keeps the throw from escaping this async messageBuffer
// listener as an unhandledRejection on the owning instance.
reply = {
type: 'customEventComplete',
replyId,
payload: undefined,
error: err instanceof Error ? err.message : String(err),
};
}
this.pub.publish(`${replyTo}`, this.pack(reply));
return;
}
if (type === 'customEventComplete') {
const { replyId, payload } = msg;
const resolveFn = this.pendingReplies[replyId];
if (!resolveFn) return;
const { replyId, payload, error } = msg;
const pending = this.pendingReplies[replyId];
if (!pending) return;
delete this.pendingReplies[replyId];
resolveFn(payload);
if (error !== undefined) {
pending.reject(new Error(error));
} else {
pending.resolve(payload);
}
return;
}
const { socketId } = msg;
@@ -273,11 +299,22 @@ export class RedisSyncExtension<TCE extends CustomEvents> implements Extension {
};
const msg = this.pack(proxyMessage);
this.pub.publish(`${this.msgChannel}:${proxyTo}`, msg);
// @ts-ignore
const { promise, resolve, reject } = Promise.withResolvers();
this.pendingReplies[replyId] = resolve;
// Manual deferred (no Promise.withResolvers) so this runs on Node < 22 too.
let resolve!: (v: unknown) => void;
let reject!: (e: unknown) => void;
const promise = new Promise((res, rej) => {
resolve = res;
reject = rej;
});
this.pendingReplies[replyId] = { resolve, reject };
setTimeout(() => {
reject('TIMEOUT');
// Fallback for a genuinely lost reply. A handler that threw now rejects
// promptly via the error-carrying customEventComplete above; this TIMEOUT
// only fires when no reply ever comes back.
if (this.pendingReplies[replyId]) {
delete this.pendingReplies[replyId];
reject('TIMEOUT');
}
}, this.customEventTTL);
return promise as Promise<ReturnType<TCE[TName]>>;
}

View File

@@ -72,6 +72,10 @@ export type RSAMessageCustomEventComplete = {
type: 'customEventComplete';
replyId: number;
payload: unknown;
// When the remote handler THREW, the owner sends back the error message here
// instead of a payload, so the origin can reject its awaiting promise promptly
// (with the real error) rather than waiting out the customEventTTL timeout.
error?: string;
};
export type RSAMessage =

View File

@@ -0,0 +1,535 @@
/**
* JEST CONFIG NOTE (#119 ESM refactor): this is the one spec that needs the REAL
* `@docmost/git-sync` converter (not a mock). The package is now ESM, which jest
* cannot `require()` nor `import()` without --experimental-vm-modules, so the
* server jest config `moduleNameMapper`s `@docmost/git-sync` to its TS SOURCE and
* strips the ESM `.js` import suffixes. ts-jest then type-checks that source under
* the server's (looser) tsconfig and trips a benign narrowing; the global
* `isolatedModules: true` on the ts-jest transform (apps/server/package.json)
* makes it transpile-only so this spec loads. Full type-checking of the package
* is still enforced by its own `tsc`/vitest gates and the server `tsc --noEmit`.
*
* §13.1 IDEMPOTENCY GATE — the blocking gate for git-sync Phase B.
*
* Proves the `@docmost/git-sync` pure converter is schema-compatible
* with the server's REAL editor-ext document schema: a representative corpus of
* editor-ext ProseMirror documents must survive a full round trip through the
* actual server write path without losing any node / mark / attribute.
*
* Pipeline per document (issue #194 §13.1):
* 1. md = convertProseMirrorToMarkdown(content) // git-sync export
* 2. doc = await markdownToProseMirror(md) // git-sync import
* 3. push `doc` through the REAL editor-ext Yjs write path the server uses:
* ydoc = TiptapTransformer.toYdoc(doc, 'default', tiptapExtensions)
* normalized = TiptapTransformer.fromYdoc(ydoc, 'default')
* This is exactly what PersistenceExtension does on store
* (apps/server/src/collaboration/extensions/persistence.extension.ts:96/115)
* with the same `tiptapExtensions` (collaboration.util.ts) and the same
* `@hocuspocus/transformer`, so the gate exercises the real schema
* validation that runs on a git-sync write (issue #194 §3.3).
* 4. assert docsCanonicallyEqual(canon(original), canon(normalized)) === true
*
* Any node / mark / attr that editor-ext drops (because the git-sync
* docmost-schema named it differently, or declares a different default) makes
* the gate FAIL for that document — exactly the schema-divergence issue #194 §3.3 /
* §13.1 warn about. Genuine, irreducible divergences are isolated into the
* clearly-named `KNOWN DIVERGENCE` block at the bottom (never silently hidden).
*
* Requires the workspace packages built first:
* pnpm --filter @docmost/editor-ext build
* pnpm --filter @docmost/git-sync build
*/
import { TiptapTransformer } from '@hocuspocus/transformer';
// Import the server's real schema FIRST so `@docmost/editor-ext` resolves to its
// built CJS `dist` (its `main`). The ESM-only `@docmost/git-sync` package is
// mapped to its TS SOURCE by the jest `moduleNameMapper` (the built ESM cannot
// be `require()`d nor dynamically `import()`ed under jest's node VM), so ts-jest
// transpiles the real converter to CJS here — exercising the actual converter
// the server ships, not a stub.
import { tiptapExtensions } from './collaboration.util';
import {
convertProseMirrorToMarkdown,
markdownToProseMirror,
canonicalizeContent,
docsCanonicallyEqual,
} from '@docmost/git-sync';
/**
* Run a single editor-ext document through the full gate pipeline and return
* the canonical original vs the canonical doc as it lands after the real Yjs
* write path, plus the intermediate markdown for diagnostics.
*/
async function runGate(original: any): Promise<{
md: string;
imported: any;
normalized: any;
canonOriginal: any;
canonNormalized: any;
}> {
// 1) editor-ext JSON -> markdown (git-sync export).
const md = convertProseMirrorToMarkdown(original);
// 2) markdown -> ProseMirror JSON (git-sync import, docmost-schema).
const imported = await markdownToProseMirror(md);
// 3) push through the REAL editor-ext schema via the server's Yjs write path.
// toYdoc validates `imported` against tiptapExtensions (throws on an
// unknown node, drops unknown attrs); fromYdoc reads it back as the
// normalized editor-ext JSON the server would persist.
const ydoc = TiptapTransformer.toYdoc(imported, 'default', tiptapExtensions);
const normalized = TiptapTransformer.fromYdoc(ydoc, 'default');
return {
md,
imported,
normalized,
canonOriginal: canonicalizeContent(original),
canonNormalized: canonicalizeContent(normalized),
};
}
const doc = (...content: any[]) => ({ type: 'doc', content });
const text = (t: string, marks?: any[]) =>
marks ? { type: 'text', text: t, marks } : { type: 'text', text: t };
const para = (...content: any[]) => ({ type: 'paragraph', content });
// ---------------------------------------------------------------------------
// Corpus: editor-ext ProseMirror documents covering the common node/mark types.
// Node / mark / attr names and DEFAULTS are taken from the real schema —
// editor-ext (packages/editor-ext/src) + the server's tiptapExtensions
// (collaboration.util.ts) — NOT guessed. Where editor-ext materializes a
// non-null default on import (e.g. image.align="center", callout.type, list
// start) the fixture pre-authors that materialized value so the round trip is
// already at its fixpoint (matches how the engine normalizes-on-write, SPEC §11).
// ---------------------------------------------------------------------------
const CORPUS: Record<string, any> = {
'paragraphs + headings (h1-h3)': doc(
{ type: 'heading', attrs: { level: 1 }, content: [text('Heading one')] },
{ type: 'heading', attrs: { level: 2 }, content: [text('Heading two')] },
{ type: 'heading', attrs: { level: 3 }, content: [text('Heading three')] },
para(text('A plain paragraph of text.')),
para(text('Second paragraph.')),
),
'inline marks (bold/italic/strike/code)': doc(
para(
text('normal '),
text('bold', [{ type: 'bold' }]),
text(' '),
text('italic', [{ type: 'italic' }]),
text(' '),
text('struck', [{ type: 'strike' }]),
text(' '),
text('code', [{ type: 'code' }]),
),
),
'links': doc(
para(
text('see '),
text('the site', [
{ type: 'link', attrs: { href: 'https://example.com' } },
]),
text(' for more'),
),
),
'bullet list': doc({
type: 'bulletList',
content: [
{ type: 'listItem', content: [para(text('first'))] },
{ type: 'listItem', content: [para(text('second'))] },
{ type: 'listItem', content: [para(text('third'))] },
],
}),
'ordered list': doc({
type: 'orderedList',
attrs: { start: 1 },
content: [
{ type: 'listItem', content: [para(text('one'))] },
{ type: 'listItem', content: [para(text('two'))] },
],
}),
'task list (checkbox)': doc({
type: 'taskList',
content: [
{
type: 'taskItem',
attrs: { checked: true },
content: [para(text('done item'))],
},
{
type: 'taskItem',
attrs: { checked: false },
content: [para(text('todo item'))],
},
],
}),
'blockquote': doc({
type: 'blockquote',
content: [para(text('a quoted line')), para(text('second quoted line'))],
}),
'callout (info)': doc({
type: 'callout',
attrs: { type: 'info' },
content: [para(text('an informational callout'))],
}),
'callout (warning)': doc({
type: 'callout',
attrs: { type: 'warning' },
content: [para(text('a warning callout'))],
}),
'code block (with language)': doc({
type: 'codeBlock',
attrs: { language: 'typescript' },
// A fenced code block's body is stored with a trailing newline (the form a
// markdown ``` fence round-trips to: marked normalizes the code text to end
// in "\n"). Authoring the fixture at that fixpoint mirrors how the engine
// normalizes-on-write (SPEC §11): codeBlock + `language` round-trip exactly.
content: [text('const a: number = 1;\nconsole.log(a);\n')],
}),
'horizontal rule': doc(
para(text('before')),
{ type: 'horizontalRule' },
para(text('after')),
),
'table (header row + cells)': doc({
type: 'table',
content: [
{
type: 'tableRow',
content: [
{
type: 'tableHeader',
attrs: { colspan: 1, rowspan: 1, colwidth: null },
content: [para(text('Name'))],
},
{
type: 'tableHeader',
attrs: { colspan: 1, rowspan: 1, colwidth: null },
content: [para(text('Value'))],
},
],
},
{
type: 'tableRow',
content: [
{
type: 'tableCell',
attrs: { colspan: 1, rowspan: 1, colwidth: null },
content: [para(text('alpha'))],
},
{
type: 'tableCell',
attrs: { colspan: 1, rowspan: 1, colwidth: null },
content: [para(text('1'))],
},
],
},
],
}),
// --- editor-ext nodes/marks beyond the original corpus (item #7) ----------
// Each of these was verified to round-trip CLEANLY through the real gate
// (export -> markdown -> import -> editor-ext Yjs write path). Fixtures are
// pre-authored at the engine's normalize-on-write fixpoint (SPEC §11), e.g.
// details carries the materialized `open:false`, and color marks use the
// `rgb(...)` form the HTML re-parser normalizes to.
'mention (user)': doc(
para(
text('hi '),
{
type: 'mention',
attrs: {
id: 'user-123',
label: 'Alice',
entityType: 'user',
entityId: 'user-123',
creatorId: 'creator-1',
},
},
text(' there'),
),
),
'inline math': doc(
para(
text('inline '),
{ type: 'mathInline', attrs: { text: 'x^2' } },
text(' math'),
),
),
'block math': doc({ type: 'mathBlock', attrs: { text: 'x^2 + y^2 = z^2' } }),
'details (collapsible)': doc({
type: 'details',
// `open:false` is the value editor-ext materializes on import; pre-authoring
// it puts the fixture at its round-trip fixpoint.
attrs: { open: false },
content: [
{ type: 'detailsSummary', content: [text('Summary line')] },
{ type: 'detailsContent', content: [para(text('hidden body'))] },
],
}),
'highlight (mark, no color)': doc(
para(
text('a '),
text('highlighted', [{ type: 'highlight' }]),
text(' word'),
),
),
'highlight (mark, with color)': doc(
para(
text('a '),
text('red', [{ type: 'highlight', attrs: { color: 'rgb(255, 0, 0)' } }]),
text(' word'),
),
),
'subscript': doc(
para(text('H'), text('2', [{ type: 'subscript' }]), text('O')),
),
'superscript': doc(
para(text('E=mc'), text('2', [{ type: 'superscript' }])),
),
'text color (textStyle)': doc(
// The HTML re-parser normalizes CSS colors to the `rgb(...)` form, so the
// fixture pre-authors that form; a `#hex` color would round-trip to the
// equivalent rgb() and is therefore a value-normalization divergence (see
// the KNOWN DIVERGENCE block below).
para(text('green', [{ type: 'textStyle', attrs: { color: 'rgb(0, 255, 0)' } }])),
),
'nested / mixed document': doc(
{ type: 'heading', attrs: { level: 1 }, content: [text('Mixed')] },
para(
text('intro with '),
text('bold', [{ type: 'bold' }]),
text(' and a '),
text('link', [{ type: 'link', attrs: { href: 'https://example.com' } }]),
text('.'),
),
{
type: 'bulletList',
content: [
{
type: 'listItem',
content: [
para(text('item with '), text('code', [{ type: 'code' }])),
],
},
{
type: 'listItem',
content: [
para(text('item with sublist')),
{
type: 'bulletList',
content: [
{ type: 'listItem', content: [para(text('nested a'))] },
{ type: 'listItem', content: [para(text('nested b'))] },
],
},
],
},
],
},
{
type: 'callout',
attrs: { type: 'success' },
content: [
para(text('callout body')),
{ type: 'codeBlock', attrs: { language: 'bash' }, content: [text('echo hi\n')] },
],
},
{
type: 'blockquote',
content: [para(text('quote at the end'))],
},
),
// Atom embeds that carry no inline text: they must round-trip via their
// schema-matching HTML (data-type div), NOT a literal that re-imports as plain
// text. `subpages` used to export as the literal "{{SUBPAGES}}" and came back
// as visible text on the page (red-team round-trip data loss) — this locks it.
// editor-ext materializes the `recursive: false` default on import, so the
// fixture pre-authors it to sit at the round-trip fixpoint (matches the other
// default-materializing fixtures above).
'subpages embed': doc({ type: 'subpages', attrs: { recursive: false } }),
};
describe('git-sync converter §13.1 idempotency gate (editor-ext schema)', () => {
for (const [name, original] of Object.entries(CORPUS)) {
it(`round-trips losslessly: ${name}`, async () => {
const { md, canonOriginal, canonNormalized } = await runGate(original);
const equal = docsCanonicallyEqual(original, canonNormalized);
if (!equal) {
// Surface a readable diff so a real divergence is actionable.
// eslint-disable-next-line no-console
console.error(
`\n[GATE FAIL] ${name}\n--- markdown ---\n${md}\n` +
`--- canonical original ---\n${JSON.stringify(canonOriginal, null, 2)}\n` +
`--- canonical round-tripped ---\n${JSON.stringify(canonNormalized, null, 2)}\n`,
);
}
expect(equal).toBe(true);
});
}
});
// ---------------------------------------------------------------------------
// KNOWN DIVERGENCE — images (isolated so it does NOT silently weaken the gate).
//
// This is NOT a schema-name divergence: the `image` NODE itself round-trips
// through editor-ext fine (it survives toYdoc under the real tiptapExtensions).
// The loss is intrinsic to MARKDOWN, the on-disk transport format git-sync uses:
//
// 1. `convertProseMirrorToMarkdown` emits a standard `![alt](src)` image
// (markdown-converter.ts case "image"). Standard markdown image syntax has
// no way to express `width` / `height` / `align`, so those attrs are
// DROPPED on export and cannot be recovered on import.
// 2. A block-level image is hoisted out of its line by the HTML re-parser,
// leaving a leading EMPTY paragraph (the same block-image-hoist limitation
// documented in packages/git-sync/test/fixtures/known-limitations).
//
// The gate documents the EXACT lossy shape below. If the converter is ever
// taught to preserve image dimensions (e.g. by emitting an HTML <img> with
// data-* attrs, as it already does for video/diagrams), these assertions flip
// and the image fixture should be promoted into the green CORPUS above.
// ---------------------------------------------------------------------------
describe('git-sync converter §13.1 image dimensions preserved (was KNOWN DIVERGENCE)', () => {
const imageDoc = doc({
type: 'image',
attrs: {
src: 'https://example.com/pic.png',
width: 640,
height: 480,
align: 'center',
},
});
it('preserves width/height/align by exporting an HTML <img> (PR #119 round-trip fix)', async () => {
const { md, canonNormalized } = await runGate(imageDoc);
// A top-level image carrying layout attrs is now exported as a schema-
// matching HTML <img> (the same path video/diagrams already use), so the
// dimensions and alignment survive the round trip instead of collapsing to
// bare `![](src)`.
expect(md.trim()).toBe(
'<img src="https://example.com/pic.png" width="640" height="480" align="center">',
);
// The round-tripped image keeps src + the layout attrs. width/height are
// re-imported as strings (matching the video/audio/pdf string convention),
// so assert the values rather than the JS type.
const imgAttrs = (canonNormalized as any).content[0].attrs;
expect((canonNormalized as any).content[0].type).toBe('image');
expect(imgAttrs.src).toBe('https://example.com/pic.png');
expect(imgAttrs.align).toBe('center');
expect(String(imgAttrs.width)).toBe('640');
expect(String(imgAttrs.height)).toBe('480');
});
});
// ---------------------------------------------------------------------------
// KNOWN DIVERGENCE — text alignment (item #7; isolated, not silently dropped).
//
// editor-ext registers TextAlign for heading+paragraph, and the SERVER schema
// fully supports it — the loss is intrinsic to the MARKDOWN transport:
//
// • A paragraph's `textAlign` is EXPORTED as `<div align="...">text</div>`
// (markdown-converter case "paragraph"), but on import the converter's
// docmost-schema declares `textAlign` WITHOUT a parseHTML mapping, so the
// `align` attribute is never recovered -> it imports as `textAlign:null`
// and canonicalizes away. A heading's alignment is not even exported.
// • Therefore any non-default alignment is dropped on a full round trip.
//
// If the converter is ever taught to parse `align`/`text-align` back onto the
// block, this assertion flips and an aligned-paragraph fixture should be
// promoted into the green CORPUS above.
// ---------------------------------------------------------------------------
describe('git-sync converter §13.1 KNOWN DIVERGENCE (text alignment dropped)', () => {
it('drops a paragraph textAlign on the markdown round trip', async () => {
const alignedDoc = doc({
type: 'paragraph',
attrs: { textAlign: 'center' },
content: [text('centered')],
});
const { canonNormalized } = await runGate(alignedDoc);
// The round-tripped paragraph carries no alignment.
expect(canonNormalized).toEqual({
type: 'doc',
content: [{ type: 'paragraph', content: [{ type: 'text', text: 'centered' }] }],
});
expect(docsCanonicallyEqual(alignedDoc, canonNormalized)).toBe(false);
});
it('drops a heading textAlign (headings do not export alignment at all)', async () => {
const alignedHeading = doc({
type: 'heading',
attrs: { level: 2, textAlign: 'center' },
content: [text('centered heading')],
});
const { md, canonNormalized } = await runGate(alignedHeading);
// Export is a plain markdown heading — no alignment syntax.
expect(md.trim()).toBe('## centered heading');
expect(docsCanonicallyEqual(alignedHeading, canonNormalized)).toBe(false);
});
});
// ---------------------------------------------------------------------------
// KNOWN DIVERGENCE — textStyle color is VALUE-NORMALIZED, not lost (item #7).
//
// The textStyle/color mark itself round-trips (the green CORPUS has the rgb()
// form). But a `#hex` color is normalized to the equivalent `rgb(...)` string
// by the HTML re-parser on import, and canonicalize.ts does NOT normalize color
// formats — so a `#hex` original is not STRING-identical to its round trip even
// though the color is semantically preserved. Locked here so the boundary is
// explicit: author color fixtures in rgb() form to stay in the green corpus.
// ---------------------------------------------------------------------------
describe('git-sync converter §13.1 KNOWN DIVERGENCE (textStyle color #hex -> rgb)', () => {
it('normalizes a #hex text color to rgb() (semantically preserved, string-divergent)', async () => {
const hexDoc = doc(
para(text('green', [{ type: 'textStyle', attrs: { color: '#00ff00' } }])),
);
const { canonNormalized } = await runGate(hexDoc);
// Color survives, but as the normalized rgb() string.
expect(canonNormalized).toEqual({
type: 'doc',
content: [
{
type: 'paragraph',
content: [
{
type: 'text',
text: 'green',
marks: [{ type: 'textStyle', attrs: { color: 'rgb(0, 255, 0)' } }],
},
],
},
],
});
// Not string-identical to the #hex original.
expect(docsCanonicallyEqual(hexDoc, canonNormalized)).toBe(false);
});
});

View File

@@ -1,81 +0,0 @@
import { Test, TestingModule } from '@nestjs/testing';
import { RedisService } from '@nestjs-labs/nestjs-ioredis';
import { PageTreeBridgePublisher } from './page-tree-bridge.publisher';
import { COLLAB_TREE_UPDATE_CHANNEL } from '../constants';
import {
PageEvent,
TreeUpdateSnapshot,
} from '../../database/listeners/page.listener';
const treeUpdate: TreeUpdateSnapshot = {
id: 'page-1',
slugId: 'slug-1',
spaceId: 'space-1',
parentPageId: null,
title: 'Renamed',
icon: '🚀',
};
describe('PageTreeBridgePublisher', () => {
let publisher: PageTreeBridgePublisher;
let redis: { publish: jest.Mock };
beforeEach(async () => {
redis = { publish: jest.fn().mockResolvedValue(1) };
const redisService = { getOrThrow: () => redis } as unknown as RedisService;
const module: TestingModule = await Test.createTestingModule({
providers: [
PageTreeBridgePublisher,
{ provide: RedisService, useValue: redisService },
],
}).compile();
publisher = module.get<PageTreeBridgePublisher>(PageTreeBridgePublisher);
});
it('WITH a `treeUpdate`: publishes the JSON snapshot on the channel', async () => {
const event: PageEvent = {
pageIds: ['page-1'],
workspaceId: 'ws-1',
treeUpdate,
};
await publisher.onPageUpdated(event);
expect(redis.publish).toHaveBeenCalledTimes(1);
expect(redis.publish).toHaveBeenCalledWith(
COLLAB_TREE_UPDATE_CHANNEL,
JSON.stringify(treeUpdate),
);
});
it('content-only save (NO `treeUpdate`): does NOT publish', async () => {
const event: PageEvent = {
pageIds: ['page-1'],
workspaceId: 'ws-1',
};
await publisher.onPageUpdated(event);
expect(redis.publish).not.toHaveBeenCalled();
});
it('a publish rejection is caught (no throw)', async () => {
redis.publish.mockRejectedValueOnce(new Error('redis down'));
const errorSpy = jest
.spyOn(publisher['logger'], 'error')
.mockImplementation(() => undefined);
const event: PageEvent = {
pageIds: ['page-1'],
workspaceId: 'ws-1',
treeUpdate,
};
await expect(publisher.onPageUpdated(event)).resolves.toBeUndefined();
expect(errorSpy).toHaveBeenCalledTimes(1);
errorSpy.mockRestore();
});
});

View File

@@ -1,55 +0,0 @@
import { Injectable, Logger } from '@nestjs/common';
import { OnEvent } from '@nestjs/event-emitter';
import { RedisService } from '@nestjs-labs/nestjs-ioredis';
import type { Redis } from 'ioredis';
import { EventName } from '../../common/events/event.contants';
import { PageEvent } from '../../database/listeners/page.listener';
import { COLLAB_TREE_UPDATE_CHANNEL } from '../constants';
/**
* Collab-process half of the cross-process tree-update bridge.
*
* The standalone collab process bootstraps `CollabAppModule`, which does NOT
* import `WsModule`/`PageWsListener`. So when a collaborative title/icon rename
* persists and emits `EventName.PAGE_UPDATED` with a `treeUpdate` snapshot, there
* is no listener in this process to broadcast it — the live tree update would be
* lost for 2-process (COLLAB_URL set) deployments.
*
* This publisher fills that gap: it forwards the `treeUpdate` snapshot over a
* Redis pub/sub channel to the API process, which re-broadcasts it via
* `WsTreeService` (the single broadcast authority).
*
* It is registered ONLY in `CollabAppModule.providers`, so it never runs in the
* API process (where `PageWsListener` already broadcasts the same event locally).
* That module placement is what prevents a double broadcast. In single-process
* mode `CollabAppModule` is not loaded at all, so this publisher never runs.
*/
@Injectable()
export class PageTreeBridgePublisher {
private readonly logger = new Logger(PageTreeBridgePublisher.name);
private readonly redis: Redis;
constructor(private readonly redisService: RedisService) {
this.redis = this.redisService.getOrThrow();
}
@OnEvent(EventName.PAGE_UPDATED)
async onPageUpdated(event: PageEvent): Promise<void> {
// Mirror PageWsListener's gating: only title/icon changes carry a snapshot.
// Content-only saves leave `treeUpdate` undefined and are ignored.
if (!event.treeUpdate) return;
try {
await this.redis.publish(
COLLAB_TREE_UPDATE_CHANNEL,
JSON.stringify(event.treeUpdate),
);
} catch (err) {
// A Redis publish failure must not break the store path.
this.logger.error(
`Failed to publish tree update to ${COLLAB_TREE_UPDATE_CHANNEL}`,
err instanceof Error ? err.stack : String(err),
);
}
}
}

View File

@@ -0,0 +1,26 @@
/**
* Backward-filled LCS length table for sequences `a` and `b`: `dp[i][j]` is the
* length of the longest common subsequence of the suffixes `a[i:]` and `b[j:]`.
* O(n*m) time/space — fine for page block counts.
*
* Shared by the two-way block diff (`yjs-body-merge.diffBlocks`) and the
* three-way merge planner (`three-way-merge.lcsPairs`) so the (identical) table
* construction lives in ONE place; each caller does its own traceback over the
* returned table.
*/
export function buildLcsTable(a: string[], b: string[]): number[][] {
const n = a.length;
const m = b.length;
const dp: number[][] = Array.from({ length: n + 1 }, () =>
new Array(m + 1).fill(0),
);
for (let i = n - 1; i >= 0; i--) {
for (let j = m - 1; j >= 0; j--) {
dp[i][j] =
a[i] === b[j]
? dp[i + 1][j + 1] + 1
: Math.max(dp[i + 1][j], dp[i][j + 1]);
}
}
return dp;
}

View File

@@ -0,0 +1,20 @@
import { diff3Plan, type Pick } from './three-way-merge';
// Materialize a plan into the merged key sequence for assertion.
function apply(plan: Pick[], live: string[], target: string[]): string[] {
return plan.map((p) => (p.src === 'live' ? live[p.index] : target[p.index]));
}
const merge = (o: string[], a: string[], b: string[]): string[] =>
apply(diff3Plan(o, a, b), a, b);
describe('diff3Plan red-team #9 (human edit + adjacent git insert)', () => {
it('keeps human block-2 edit AND applies git insert of 2.5', () => {
// base: 1 2 3
// live: 1 H 3 (human rewrote block 2)
// target: 1 2 2.5 3 (git inserted 2.5 after block 2)
expect(
merge(['1', '2', '3'], ['1', 'H', '3'], ['1', '2', '2.5', '3']),
).toEqual(['1', 'H', '2.5', '3']);
});
});

View File

@@ -0,0 +1,159 @@
import {
diff3Plan,
diff3PlanWithConflicts,
type Pick,
} from './three-way-merge';
// Materialize a plan into the merged key sequence for assertion.
function apply(plan: Pick[], live: string[], target: string[]): string[] {
return plan.map((p) => (p.src === 'live' ? live[p.index] : target[p.index]));
}
const merge = (o: string[], a: string[], b: string[]): string[] =>
apply(diff3Plan(o, a, b), a, b);
describe('diff3Plan (block-level three-way merge)', () => {
it('identical on all three sides -> unchanged (all from live)', () => {
const plan = diff3Plan(['1', '2', '3'], ['1', '2', '3'], ['1', '2', '3']);
expect(plan.every((p) => p.src === 'live')).toBe(true);
expect(apply(plan, ['1', '2', '3'], ['1', '2', '3'])).toEqual(['1', '2', '3']);
});
it('git changed a block the human did not -> takes git', () => {
expect(merge(['1', '2', '3'], ['1', '2', '3'], ['1', '9', '3'])).toEqual([
'1',
'9',
'3',
]);
});
it('human changed a block git did not -> KEEPS the human edit (the core 3-way win)', () => {
expect(merge(['1', '2', '3'], ['1', 'H', '3'], ['1', '2', '3'])).toEqual([
'1',
'H',
'3',
]);
});
// Bug #2 observability: diff3PlanWithConflicts reports SAME-BLOCK conflicts so
// the caller can surface the "git wins" loss (log + history pin) instead of
// dropping the human side silently.
describe('diff3PlanWithConflicts (same-block conflict reporting)', () => {
it('reports 0 conflicts when sides changed DIFFERENT blocks (clean merge)', () => {
const r = diff3PlanWithConflicts(
['1', '2', '3'],
['H', '2', '3'],
['1', '2', 'G'],
);
expect(r.conflicts).toBe(0);
expect(apply(r.picks, ['H', '2', '3'], ['1', '2', 'G'])).toEqual([
'H',
'2',
'G',
]);
});
it('reports 1 conflict and git wins when BOTH rewrote the SAME block', () => {
const r = diff3PlanWithConflicts(
['1', '2', '3'],
['1', 'H', '3'], // human rewrote block 2
['1', 'G', '3'], // git rewrote block 2
);
expect(r.conflicts).toBe(1);
// Git wins the contested block; the human 'H' is NOT in the picks.
expect(apply(r.picks, ['1', 'H', '3'], ['1', 'G', '3'])).toEqual([
'1',
'G',
'3',
]);
});
it('does NOT count a git-only region (no human content to lose) as a conflict', () => {
const r = diff3PlanWithConflicts(
['1', '2', '3'],
['1', '2', '3'], // human unchanged
['1', '9', '3'], // git rewrote block 2
);
expect(r.conflicts).toBe(0);
});
});
it('human and git changed DIFFERENT blocks -> both preserved', () => {
// human rewrote block 1, git rewrote block 3.
expect(merge(['1', '2', '3'], ['H', '2', '3'], ['1', '2', 'G'])).toEqual([
'H',
'2',
'G',
]);
});
it('human inserted a block AND git changed a different block -> both preserved', () => {
expect(
merge(['1', '2', '3'], ['1', '1.5', '2', '3'], ['1', '2', 'G']),
).toEqual(['1', '1.5', '2', 'G']);
});
it('both changed the SAME block -> conflict resolves to git', () => {
expect(merge(['1', '2', '3'], ['1', 'H', '3'], ['1', 'G', '3'])).toEqual([
'1',
'G',
'3',
]);
});
it('both made the SAME edit -> that edit (no duplication)', () => {
expect(merge(['1', '2', '3'], ['1', 'X', '3'], ['1', 'X', '3'])).toEqual([
'1',
'X',
'3',
]);
});
it('human deleted a block git left alone -> deletion preserved', () => {
expect(merge(['1', '2', '3'], ['1', '3'], ['1', '2', '3'])).toEqual([
'1',
'3',
]);
});
it('git deleted a block the human left alone -> deletion applied', () => {
expect(merge(['1', '2', '3'], ['1', '2', '3'], ['1', '3'])).toEqual([
'1',
'3',
]);
});
it('both deleted the same block -> gone (no conflict)', () => {
expect(merge(['1', '2', '3'], ['1', '3'], ['1', '3'])).toEqual(['1', '3']);
});
it('git appended a trailing block -> appended', () => {
expect(merge(['1', '2'], ['1', '2'], ['1', '2', '3'])).toEqual([
'1',
'2',
'3',
]);
});
it('human appended a trailing block git did not -> kept', () => {
expect(merge(['1', '2'], ['1', '2', '3'], ['1', '2'])).toEqual([
'1',
'2',
'3',
]);
});
it('empty base, git provides content (brand-new page body) -> git content', () => {
expect(merge([], [], ['1', '2'])).toEqual(['1', '2']);
});
it('git changed block 1, human edited block 3, far apart -> both kept', () => {
expect(
merge(
['a', 'b', 'c', 'd', 'e'],
['a', 'b', 'c', 'd', 'E'],
['A', 'b', 'c', 'd', 'e'],
),
).toEqual(['A', 'b', 'c', 'd', 'E']);
});
});

View File

@@ -0,0 +1,274 @@
/**
* Pure block-level THREE-WAY merge planner (diff3) over arrays of opaque block
* keys. Used by the git-sync body write to merge an incoming git body into the
* live page using the last-synced version as the common ancestor (review #5):
*
* - a block only the human changed (live != base, git == base) -> keep LIVE
* - a block only git changed (git != base, live == base) -> take GIT
* - a block both sides changed (a real conflict) -> GIT wins
* - inserts/deletes from either side are preserved when unambiguous
*
* Content-agnostic: it works on string keys and returns the merged block order as
* picks ({ src: 'live'|'target', index }) — the caller (the Yjs applier)
* materializes them — so the whole algorithm is unit-testable on plain arrays.
*
* Algorithm: anchor on base blocks present (unchanged) in BOTH live and target
* (their LCS-with-base intersection). Between consecutive anchors lies one region
* the human and/or git rewrote; resolve each region three-way. Stable anchor
* blocks are emitted from LIVE so the applier keeps the existing Yjs block
* instances (and the human's in-flight edits) in place.
*
* LOCATION (deferred): this and its `lcs.ts` sibling are pure, framework-free and
* could conceptually live in `packages/git-sync` (the engine). They are kept in
* the server integration on purpose: `packages/git-sync` is a VENDORED engine
* (pinned upstream, manually re-synced), so adding first-party files there
* complicates the re-sync story, and the only consumer today is the server. Move
* them into the engine only once the vendoring re-sync story is settled.
*/
import { buildLcsTable } from './lcs';
/** Matched index pairs of the longest common subsequence of `a` and `b`. */
function lcsPairs(a: string[], b: string[]): Array<[number, number]> {
const n = a.length;
const m = b.length;
const dp = buildLcsTable(a, b);
const pairs: Array<[number, number]> = [];
let i = 0;
let j = 0;
while (i < n && j < m) {
if (a[i] === b[j]) {
pairs.push([i, j]);
i++;
j++;
} else if (dp[i + 1][j] >= dp[i][j + 1]) {
i++;
} else {
j++;
}
}
return pairs;
}
/** o-index -> matched index in the other side (only for LCS-matched blocks). */
function matchMap(pairs: Array<[number, number]>): Map<number, number> {
const m = new Map<number, number>();
for (const [o, x] of pairs) m.set(o, x);
return m;
}
/**
* One change `side` made to `base` within a region: base blocks `[oStart,oEnd)`
* were replaced by the side's blocks listed in `content` (region-local indices).
* A pure insert has `oStart === oEnd`; a pure delete has empty `content`.
*/
interface Hunk {
oStart: number;
oEnd: number;
content: number[];
}
/**
* Diff `o` against one side as a list of non-overlapping hunks (the base spans
* the side rewrote/inserted/deleted), derived from their LCS alignment.
*/
function buildHunks(o: string[], side: string[]): Hunk[] {
const pairs = lcsPairs(o, side); // [oIdx, sideIdx] kept (unchanged) blocks
const hunks: Hunk[] = [];
let prevO = -1;
let prevS = -1;
const flush = (curO: number, curS: number): void => {
const oStart = prevO + 1;
const oEnd = curO;
const content: number[] = [];
for (let s = prevS + 1; s < curS; s++) content.push(s);
if (oEnd > oStart || content.length > 0) hunks.push({ oStart, oEnd, content });
};
for (const [oIdx, sIdx] of pairs) {
flush(oIdx, sIdx);
prevO = oIdx;
prevS = sIdx;
}
flush(o.length, side.length);
return hunks;
}
/**
* Do two hunks (one per side) touch the same base region? Pure inserts only
* collide when nested strictly inside the other hunk's base span (or, for two
* inserts, at the same gap); changes sitting at a shared boundary do not.
*/
function hunksOverlap(a: Hunk, b: Hunk): boolean {
const aIns = a.oStart === a.oEnd;
const bIns = b.oStart === b.oEnd;
if (aIns && bIns) return a.oStart === b.oStart;
if (aIns) return b.oStart < a.oStart && a.oStart < b.oEnd;
if (bIns) return a.oStart < b.oStart && b.oStart < a.oEnd;
return Math.max(a.oStart, b.oStart) < Math.min(a.oEnd, b.oEnd);
}
interface LocalPick {
src: 'live' | 'target';
local: number;
}
/**
* Fine-grained three-way merge of ONE inter-anchor region. Combines the human's
* and git's NON-overlapping hunks (e.g. a human edit to one block plus a git
* insert/delete of OTHER blocks in the same region) so neither change is lost.
* Returns the merged region as region-local picks, or `null` when the two sides
* changed the SAME base block — a genuine conflict the caller resolves by the
* original all-or-nothing rule (git wins the whole region).
*/
function tryMergeRegion(
o: string[],
a: string[],
b: string[],
): LocalPick[] | null {
const aHunks = buildHunks(o, a);
const bHunks = buildHunks(o, b);
// Any overlap between a human hunk and a git hunk is a real conflict; bail so
// the caller falls back to git-wins (preserving the original behavior).
for (const ah of aHunks) {
for (const bh of bHunks) {
if (hunksOverlap(ah, bh)) return null;
}
}
// Disjoint: live index of each base block that BOTH sides kept (stable).
const aKept = matchMap(lcsPairs(o, a)); // base index -> live index
const out: LocalPick[] = [];
let pa = 0;
let pb = 0;
let oi = 0;
while (oi < o.length || pa < aHunks.length || pb < bHunks.length) {
const ah = pa < aHunks.length ? aHunks[pa] : null;
const bh = pb < bHunks.length ? bHunks[pb] : null;
const nextStart = Math.min(
ah ? ah.oStart : o.length,
bh ? bh.oStart : o.length,
);
// Emit stable base blocks (kept by both) until the next hunk, from LIVE.
while (oi < nextStart) {
out.push({ src: 'live', local: aKept.get(oi) as number });
oi++;
}
if (!ah && !bh) break;
// Apply the hunk at oi. When both sides act here they are disjoint, so the
// pure-insert (oEnd === oi) is emitted before the side that consumes base oi.
const aHere = ah !== null && ah.oStart === oi;
const bHere = bh !== null && bh.oStart === oi;
let useA: boolean;
if (aHere && bHere) {
useA = ah!.oEnd === oi; // insert side first; otherwise either order is fine
} else {
useA = aHere;
}
const h = (useA ? ah : bh) as Hunk;
const src: 'live' | 'target' = useA ? 'live' : 'target';
for (const idx of h.content) out.push({ src, local: idx });
oi = h.oEnd;
if (useA) pa++;
else pb++;
}
return out;
}
export interface Pick {
src: 'live' | 'target';
index: number;
}
/**
* The merged block order PLUS how many regions resolved as a genuine SAME-BLOCK
* conflict (both sides rewrote the same base block — `tryMergeRegion` returned
* null and git won the whole region, so the live/human version of those blocks
* is NOT in `picks`). `conflicts > 0` is the OBSERVABLE signal the caller uses to
* surface "git won a concurrent same-block edit" (log it + pin the human
* baseline to page history) instead of dropping the human side silently.
*/
export interface Diff3Result {
picks: Pick[];
conflicts: number;
}
/**
* Three-way merge of base `o`, live `a`, target `b` (arrays of block keys).
* Returns the merged block order as picks from live/target. Thin wrapper over
* `diff3PlanWithConflicts` (kept for the existing pure-array callers/tests).
*/
export function diff3Plan(o: string[], a: string[], b: string[]): Pick[] {
return diff3PlanWithConflicts(o, a, b).picks;
}
/**
* Like `diff3Plan` but also reports the SAME-BLOCK conflict count (see
* `Diff3Result`). A region where both the human and git rewrote the same base
* block cannot be merged automatically; the rule is deterministic — GIT WINS the
* whole region — but the human's version of those blocks is then absent from the
* picks, so we count it so the caller can make the loss observable/recoverable
* rather than silent (the documented conflict contract).
*/
export function diff3PlanWithConflicts(
o: string[],
a: string[],
b: string[],
): Diff3Result {
const oToA = matchMap(lcsPairs(o, a));
const oToB = matchMap(lcsPairs(o, b));
const res: Pick[] = [];
let conflicts = 0;
let oi = 0;
let ai = 0;
let bi = 0;
for (;;) {
// Next anchor: a base block present (unchanged) in BOTH live and target.
let anchor = oi;
while (anchor < o.length && !(oToA.has(anchor) && oToB.has(anchor))) {
anchor++;
}
const aEnd = anchor < o.length ? (oToA.get(anchor) as number) : a.length;
const bEnd = anchor < o.length ? (oToB.get(anchor) as number) : b.length;
// Resolve the region [oi,anchor) that one or both sides rewrote/inserted.
// Try a fine-grained three-way merge first so a human block-edit survives a
// git insert/delete of OTHER blocks in the same region; only a genuine
// same-block conflict (null) falls back to the original git-wins rule.
const merged = tryMergeRegion(
o.slice(oi, anchor),
a.slice(ai, aEnd),
b.slice(bi, bEnd),
);
if (merged) {
for (const p of merged) {
res.push(
p.src === 'live'
? { src: 'live', index: ai + p.local }
: { src: 'target', index: bi + p.local },
);
}
} else {
// SAME-BLOCK CONFLICT: count it ONLY when the human side actually had
// content in this region that git's win discards (live region non-empty).
// A region only git rewrote (live region empty) is not a human loss.
if (aEnd > ai) conflicts++;
for (let k = bi; k < bEnd; k++) res.push({ src: 'target', index: k });
}
if (anchor >= o.length) break;
// Emit the stable anchor block from LIVE, then advance past it on all sides.
res.push({ src: 'live', index: aEnd });
ai = aEnd + 1;
bi = bEnd + 1;
oi = anchor + 1;
}
return { picks: res, conflicts };
}

View File

@@ -0,0 +1,171 @@
import { TiptapTransformer } from '@hocuspocus/transformer';
import * as Y from 'yjs';
import {
markdownToProseMirror,
convertProseMirrorToMarkdown,
} from '@docmost/git-sync';
import { tiptapExtensions } from '../collaboration.util';
import { mergeXmlFragments, mergeXmlFragments3Way } from './yjs-body-merge';
/**
* Regression for the QA #119 callout findings (body-duplication re-verify +
* "callout strips the whole body"). These reproduce the ACTUAL live merge path:
*
* live = TiptapTransformer.toYdoc(editor JSON, tiptapExtensions) (the
* collaboration server's materialization — schema defaults stamped)
* git = toYdoc(markdownToProseMirror(convertProseMirrorToMarkdown(editor)))
* (the engine round-trip the push side feeds into writePageBody)
*
* A page containing a callout (with a neighbouring heading + paragraphs) must:
* - merge with ZERO ops on an unchanged resync (no duplication — bug #1), and
* - NEVER lose blocks / collapse to empty (no strip — bug #2),
* across repeated cycles, for every editor-canonical callout type.
*/
const toYdoc = (content: unknown[]) =>
TiptapTransformer.toYdoc(
{ type: 'doc', content },
'default',
tiptapExtensions as any,
);
const blockTypes = (f: Y.XmlFragment) =>
f.toArray().map((n: any) => n.nodeName);
function editorPage(calloutType: string) {
return [
{
type: 'heading',
attrs: { id: 'h1', level: 1 },
content: [{ type: 'text', text: 'Title here' }],
},
{
type: 'paragraph',
attrs: { id: 'p1' },
content: [{ type: 'text', text: 'Para before callout' }],
},
{
type: 'callout',
attrs: { type: calloutType },
content: [
{
type: 'paragraph',
attrs: { id: 'pc' },
content: [{ type: 'text', text: 'Inside the callout' }],
},
],
},
{
type: 'paragraph',
attrs: { id: 'p2' },
content: [{ type: 'text', text: 'Para after callout' }],
},
];
}
async function gitRoundTrip(content: unknown[]): Promise<any[]> {
const md = await convertProseMirrorToMarkdown({ type: 'doc', content });
const json = await markdownToProseMirror(md);
return json.content;
}
describe('git-sync callout merge is idempotent + non-destructive (QA #119)', () => {
for (const type of ['info', 'note', 'warning', 'danger', 'success', 'default']) {
it(`callout(${type}) resyncs with 0 ops and never strips the body`, async () => {
const editor = editorPage(type);
const gitContent = await gitRoundTrip(editor);
const liveDoc = toYdoc(editor);
const live = liveDoc.getXmlFragment('default');
const before = live.toArray().length;
expect(before).toBe(4);
// 2-way: live vs the git round-trip -> no-op (no dup, no strip).
let applied = -1;
liveDoc.transact(() => {
applied = mergeXmlFragments(live, toYdoc(gitContent).getXmlFragment('default'));
});
expect(applied).toBe(0);
expect(live.toArray().length).toBe(before);
// 3-way across 4 cycles with base == git (the steady-state) -> stable.
for (let cycle = 0; cycle < 4; cycle++) {
let a = -1;
liveDoc.transact(() => {
a = mergeXmlFragments3Way(
live,
toYdoc(gitContent).getXmlFragment('default'),
toYdoc(gitContent).getXmlFragment('default'),
);
});
expect(a).toBe(0);
expect(live.toArray().length).toBe(before);
expect(blockTypes(live)).toEqual([
'heading',
'paragraph',
'callout',
'paragraph',
]);
}
});
}
it('3-way with a stale base (callout JUST added) keeps the callout + neighbours', async () => {
// base = the previously-synced version WITHOUT the callout (git round-trip);
// the human just inserted the callout -> the merge must KEEP everything.
const prev = [
{ type: 'heading', attrs: { id: 'h1', level: 1 }, content: [{ type: 'text', text: 'Title here' }] },
{ type: 'paragraph', attrs: { id: 'p1' }, content: [{ type: 'text', text: 'Para before callout' }] },
{ type: 'paragraph', attrs: { id: 'p2' }, content: [{ type: 'text', text: 'Para after callout' }] },
];
const editor = editorPage('info');
const baseContent = await gitRoundTrip(prev);
const gitContent = await gitRoundTrip(editor);
const liveDoc = toYdoc(editor);
const live = liveDoc.getXmlFragment('default');
liveDoc.transact(() => {
mergeXmlFragments3Way(
live,
toYdoc(gitContent).getXmlFragment('default'),
toYdoc(baseContent).getXmlFragment('default'),
);
});
// Body survives in full — NOT stripped to empty / a lone paragraph.
expect(blockTypes(live)).toEqual([
'heading',
'paragraph',
'callout',
'paragraph',
]);
});
});
describe('git-sync callout type fidelity (QA "callout type -> [!info]")', () => {
for (const type of ['info', 'note', 'warning', 'danger', 'success', 'default']) {
it(`preserves callout type "${type}" across the engine round-trip`, async () => {
const content = editorPage(type);
const gitContent = await gitRoundTrip(content);
const co = gitContent.find((b: any) => b.type === 'callout');
expect(co?.attrs?.type).toBe(type);
});
}
it('maps a known GitHub/Obsidian alias to the editor banner (tip -> success)', async () => {
// `tip` is not a schema callout type — it is an input alias the editor itself
// maps onto the supported set (GITHUB_ALERT_TYPE_MAP: tip -> success). git-sync
// mirrors that so the ingest lands on the closest banner instead of flatly info.
const content = editorPage('tip');
const gitContent = await gitRoundTrip(content);
const co = gitContent.find((b: any) => b.type === 'callout');
expect(co?.attrs?.type).toBe('success');
});
it('flattens a genuinely unknown callout type to info', async () => {
const content = editorPage('banana'); // not a type and not a known alias
const gitContent = await gitRoundTrip(content);
const co = gitContent.find((b: any) => b.type === 'callout');
expect(co?.attrs?.type).toBe('info');
});
});

View File

@@ -0,0 +1,198 @@
import * as Y from 'yjs';
import { mergeXmlFragments, mergeXmlFragments3Way } from './yjs-body-merge';
/**
* Regression for the HIGH-severity runaway whole-body duplication: a page body
* was RE-APPENDED in full on every git-sync reconcile cycle, unbounded, with NO
* client connected.
*
* ROOT CAUSE (confirmed in-process against the real failing page): the LIVE Yjs
* document materializes the editor-schema default `indent: 0` on every
* paragraph/heading (and on the paragraph inside every list item, callout, and
* table cell), but a body re-imported from git — parsed from clean markdown —
* carries NO indent attribute. So every live block's comparison key differed from
* the same block coming back from git; the three-way merge could anchor on
* NOTHING, and the trailing unit that git's export already contained (but the
* merge could not match against the byte-identical live tail) was re-appended
* each cycle. Each grown export then diverged from the last-pushed base by one
* more unit — a self-sustaining loop.
*
* The fix normalizes the materialized default (`indent: 0`) out of the block key
* (the schema-derived `serializeXmlNode` normalization in yjs-body-merge.ts drops
* every attr equal to its ProseMirror-schema default; `indent: 0` is one such),
* so a live block compares equal to its git-round-tripped twin and the resync is
* a true no-op. The sibling `yjs-body-merge.schema-defaults.spec.ts` covers the
* rest of the bug class (image.align, link mark internal, …).
*
* These tests model that EXACTLY at the Yjs level: a LIVE fragment whose blocks
* carry `indent: 0` + block ids, versus a git-derived fragment of the SAME
* content with neither — for a body built from BYTE-IDENTICAL units that each
* contain a heading, a paragraph, a callout, and a table with empty cells (the
* trigger). RED before the fix (the merge applies > 0 ops and the body grows),
* GREEN after (0 ops, no growth).
*/
type Attrs = Record<string, string | number>;
function el(
name: string,
attrs: Attrs,
children: (Y.XmlElement | Y.XmlText)[],
) {
const e = new Y.XmlElement(name);
for (const [k, v] of Object.entries(attrs)) e.setAttribute(k, v as string);
if (children.length) e.insert(0, children);
return e;
}
function text(s: string): Y.XmlText {
const t = new Y.XmlText();
if (s) t.insert(0, s);
return t;
}
/**
* One byte-identical content unit (heading / paragraph / callout / table-with-
* empty-cells). `live` toggles the two things that exist ONLY in the live Yjs
* doc and NOT in a git round-trip: the materialized `indent: 0` default and the
* per-block `id`. `n` makes each unit's ids unique (as the editor would stamp)
* while keeping the visible CONTENT byte-identical across units.
*/
function unit(
live: boolean,
n: number,
headingText = 'Big Heading',
): Y.XmlElement[] {
const ind: Attrs = live ? { indent: 0 } : {};
const id = (base: string): Attrs => (live ? { id: `${base}${n}` } : {});
const para = (attrs: Attrs, s: string) =>
el('paragraph', { ...attrs, ...ind }, [text(s)]);
const cell = (name: string) =>
el(name, { colspan: 1, rowspan: 1 }, [para({}, '')]);
return [
el('heading', { ...id('h'), level: 1, ...ind }, [text(headingText)]),
para(id('p'), 'Para with the same words'),
el('callout', { type: 'info' }, [para(id('c'), 'CalloutText here')]),
el('table', {}, [
el('tableRow', {}, [cell('tableHeader'), cell('tableHeader')]),
el('tableRow', {}, [cell('tableCell'), cell('tableCell')]),
]),
];
}
function fragmentOf(units: Y.XmlElement[][]): {
doc: Y.Doc;
frag: Y.XmlFragment;
} {
const doc = new Y.Doc();
const frag = doc.getXmlFragment('default');
const blocks = units.flat();
if (blocks.length) frag.insert(0, blocks);
return { doc, frag };
}
const blockCount = (frag: Y.XmlFragment): number => frag.toArray().length;
describe('git-sync reconcile import is idempotent (no whole-body duplication)', () => {
const UNITS = 3;
it('3-way: identical content, live carries indent:0, base stale-by-one -> 0 ops, no growth', () => {
// LIVE: the editor-stamped Yjs doc (indent:0 + ids on every block).
const { doc: liveDoc, frag: live } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => unit(true, i)),
);
// INCOMING (git export -> re-import): same content, NO indent / ids.
const { frag: incoming } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => unit(false, i)),
);
// BASE = last-pushed file, lagging by ONE unit (the realistic divergence
// that drives the trailing insert-vs-insert).
const { frag: base } = fragmentOf(
Array.from({ length: UNITS - 1 }, (_, i) => unit(false, i)),
);
const before = blockCount(live);
let applied = -1;
liveDoc.transact(() => {
applied = mergeXmlFragments3Way(live, incoming, base);
});
expect(applied).toBe(0);
expect(blockCount(live)).toBe(before);
});
it('3-way is a fixpoint across repeated cycles (does not grow)', () => {
const { doc: liveDoc, frag: live } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => unit(true, i)),
);
const incomingUnits = () =>
fragmentOf(Array.from({ length: UNITS }, (_, i) => unit(false, i))).frag;
const baseUnits = () =>
fragmentOf(Array.from({ length: UNITS - 1 }, (_, i) => unit(false, i)))
.frag;
const before = blockCount(live);
for (let cycle = 0; cycle < 5; cycle++) {
let applied = -1;
liveDoc.transact(() => {
applied = mergeXmlFragments3Way(live, incomingUnits(), baseUnits());
});
expect(applied).toBe(0);
expect(blockCount(live)).toBe(before);
}
});
it('2-way: identical content, live carries indent:0 -> 0 ops, no growth', () => {
const { doc: liveDoc, frag: live } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => unit(true, i)),
);
const { frag: incoming } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => unit(false, i)),
);
const before = blockCount(live);
let applied = -1;
liveDoc.transact(() => {
applied = mergeXmlFragments(live, incoming);
});
expect(applied).toBe(0);
expect(blockCount(live)).toBe(before);
});
it('does NOT regress real edits: a git change to one block still lands', () => {
const { doc: liveDoc, frag: live } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => unit(true, i)),
);
const base = fragmentOf(
Array.from({ length: UNITS }, (_, i) => unit(false, i)),
).frag;
// git edits the heading text of the LAST unit.
const incoming = fragmentOf(
Array.from({ length: UNITS }, (_, i) =>
unit(false, i, i === UNITS - 1 ? 'EDITED Heading' : 'Big Heading'),
),
).frag;
const before = blockCount(live);
liveDoc.transact(() => {
mergeXmlFragments3Way(live, incoming, base);
});
// The edit landed, and the body did NOT grow (one block changed in place).
const headings = live
.toArray()
.filter((b) => (b as Y.XmlElement).nodeName === 'heading')
.map((b) =>
(b as Y.XmlElement)
.toArray()
.map((c) => (c as Y.XmlText).toString())
.join(''),
);
expect(headings).toContain('EDITED Heading');
expect(blockCount(live)).toBe(before);
});
});

View File

@@ -0,0 +1,316 @@
import { TiptapTransformer } from '@hocuspocus/transformer';
import * as Y from 'yjs';
import { tiptapExtensions } from '../collaboration.util';
import { mergeXmlFragments, mergeXmlFragments3Way } from './yjs-body-merge';
/**
* Regression for the BUG CLASS behind the runaway whole-body duplication: the
* point-fix (7a7b840e) only normalized `indent: 0`, but the SAME divergence
* recurs for every attribute whose editor-ext (server) schema default the live
* Yjs doc MATERIALIZES while the git round-trip — which comes through the engine
* schema (different, usually null, defaults) plus `y-prosemirror`'s null-attr
* dropping — does NOT carry. Confirmed triggers beyond `indent`:
*
* - `image.align` : editor-ext default "center" (materialized) vs engine
* default null (dropped) -> element-attr divergence.
* - link mark `internal`: editor-ext default false (materialized) vs engine
* default null -> MARK-attr divergence (the prior denylist
* could not reach marks at all — they are serialized raw in
* the XmlText delta).
*
* `highlight.colorName` is normalized too (defense-in-depth); it is NOT a strong
* real-world trigger because BOTH schemas default it to null, but the schema-
* derived normalization handles it for free and stays idempotent.
*
* The fix derives the defaults from the ACTUAL ProseMirror schema (getSchema of
* the server tiptapExtensions) and drops any element- OR mark-attribute equal to
* its schema default (or null/undefined) from the block comparison key — so a
* live block compares equal to its git-round-tripped twin and an unchanged
* resync applies 0 ops. RED before the fix (keys diverge -> ops > 0 / growth),
* GREEN after.
*/
type Attrs = Record<string, unknown>;
function el(
name: string,
attrs: Attrs,
children: (Y.XmlElement | Y.XmlText)[],
): Y.XmlElement {
const e = new Y.XmlElement(name);
for (const [k, v] of Object.entries(attrs)) e.setAttribute(k, v as string);
if (children.length) e.insert(0, children);
return e;
}
/** Text carrying marks, as the live Yjs doc stores them (XmlText format ops). */
function markedText(s: string, marks: Record<string, unknown>): Y.XmlText {
const t = new Y.XmlText();
t.insert(0, s, marks);
return t;
}
/**
* One byte-identical RICH unit: a paragraph with a LINK, a top-level IMAGE, and
* a paragraph with a HIGHLIGHT. `live` toggles exactly what the editor
* materializes but a git round-trip does not: block `id`, `indent: 0`,
* `image.align: "center"`, the link mark's `internal: false`, and the
* highlight's `colorName: null`.
*/
function richUnit(live: boolean, n: number): Y.XmlElement[] {
const ind: Attrs = live ? { indent: 0 } : {};
const id = (base: string): Attrs => (live ? { id: `${base}${n}` } : {});
const linkMarks = live
? {
link: {
href: 'https://example.com',
target: '_blank',
rel: 'noopener noreferrer nofollow',
class: null,
title: null,
internal: false, // editor-ext default, materialized
},
}
: {
link: {
href: 'https://example.com',
target: '_blank',
rel: 'noopener noreferrer nofollow',
internal: null, // engine default
},
};
const hlMarks = live
? { highlight: { color: '#ffd43b', colorName: null } }
: { highlight: { color: '#ffd43b' } };
const imageAttrs: Attrs = live
? { src: 'https://img.example.com/a.png', align: 'center' } // materialized
: { src: 'https://img.example.com/a.png' }; // align:null dropped on git side
return [
el('paragraph', { ...id('lp'), ...ind }, [
markedText('click here', linkMarks),
]),
el('image', imageAttrs, []),
el('paragraph', { ...id('hp'), ...ind }, [markedText('hot', hlMarks)]),
];
}
function fragmentOf(units: Y.XmlElement[][]): {
doc: Y.Doc;
frag: Y.XmlFragment;
} {
const doc = new Y.Doc();
const frag = doc.getXmlFragment('default');
const blocks = units.flat();
if (blocks.length) frag.insert(0, blocks);
return { doc, frag };
}
const blockCount = (frag: Y.XmlFragment): number => frag.toArray().length;
describe('git-sync reconcile is idempotent for schema-default attrs (image/link/highlight)', () => {
const UNITS = 3;
it('3-way: live carries image.align/link.internal/indent defaults, base stale-by-one -> 0 ops', () => {
const { doc: liveDoc, frag: live } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => richUnit(true, i)),
);
const { frag: incoming } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => richUnit(false, i)),
);
const { frag: base } = fragmentOf(
Array.from({ length: UNITS - 1 }, (_, i) => richUnit(false, i)),
);
const before = blockCount(live);
let applied = -1;
liveDoc.transact(() => {
applied = mergeXmlFragments3Way(live, incoming, base);
});
expect(applied).toBe(0);
expect(blockCount(live)).toBe(before);
});
it('2-way: live carries the materialized defaults -> 0 ops, no growth', () => {
const { doc: liveDoc, frag: live } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => richUnit(true, i)),
);
const { frag: incoming } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => richUnit(false, i)),
);
const before = blockCount(live);
let applied = -1;
liveDoc.transact(() => {
applied = mergeXmlFragments(live, incoming);
});
expect(applied).toBe(0);
expect(blockCount(live)).toBe(before);
});
it('is a fixpoint across repeated cycles (does not grow)', () => {
const { doc: liveDoc, frag: live } = fragmentOf(
Array.from({ length: UNITS }, (_, i) => richUnit(true, i)),
);
const incoming = () =>
fragmentOf(Array.from({ length: UNITS }, (_, i) => richUnit(false, i)))
.frag;
const base = () =>
fragmentOf(
Array.from({ length: UNITS - 1 }, (_, i) => richUnit(false, i)),
).frag;
const before = blockCount(live);
for (let cycle = 0; cycle < 5; cycle++) {
let applied = -1;
liveDoc.transact(() => {
applied = mergeXmlFragments3Way(live, incoming(), base());
});
expect(applied).toBe(0);
expect(blockCount(live)).toBe(before);
}
});
it('does NOT regress a genuine non-default value (a real link.href / image.align:left still diffs)', () => {
const { doc: liveDoc, frag: live } = fragmentOf([richUnit(true, 0)]);
const base = fragmentOf([richUnit(false, 0)]).frag;
// git genuinely changes the image alignment to a NON-default value.
const incomingUnit = richUnit(false, 0);
(incomingUnit[1] as Y.XmlElement).setAttribute('align', 'left');
const incoming = fragmentOf([incomingUnit]).frag;
liveDoc.transact(() => {
mergeXmlFragments3Way(live, incoming, base);
});
const img = live
.toArray()
.find((b) => (b as Y.XmlElement).nodeName === 'image') as Y.XmlElement;
expect(img.getAttribute('align')).toBe('left');
});
});
/**
* FAITHFUL end-to-end proof through the REAL server transformer: build the live
* doc the way the collaboration server does (defaults omitted in the JSON ->
* TiptapTransformer.toYdoc MATERIALIZES image.align:"center", link.internal:false,
* indent:0) versus the git-derived doc (engine-style: defaults emitted as
* explicit null, no block ids). An unchanged resync must apply 0 ops.
*/
describe('git-sync reconcile is idempotent through the real toYdoc materialization', () => {
const liveContent = [
{
type: 'paragraph',
attrs: { id: 'p1' },
content: [
{
type: 'text',
text: 'click here',
marks: [{ type: 'link', attrs: { href: 'https://example.com' } }],
},
],
},
{ type: 'image', attrs: { src: 'https://img.example.com/a.png' } },
{
type: 'paragraph',
attrs: { id: 'p2' },
content: [
{
type: 'text',
text: 'hot',
marks: [{ type: 'highlight', attrs: { color: '#ffd43b' } }],
},
],
},
];
// git/engine-style: explicit nulls for the engine-default attrs, no ids.
const gitContent = [
{
type: 'paragraph',
content: [
{
type: 'text',
text: 'click here',
marks: [
{
type: 'link',
attrs: {
href: 'https://example.com',
target: '_blank',
rel: 'noopener noreferrer nofollow',
class: null,
title: null,
internal: null,
},
},
],
},
],
},
{
type: 'image',
attrs: { src: 'https://img.example.com/a.png', align: null },
},
{
type: 'paragraph',
content: [
{
type: 'text',
text: 'hot',
marks: [
{ type: 'highlight', attrs: { color: '#ffd43b', colorName: null } },
],
},
],
},
];
const toYdoc = (content: unknown[]) =>
TiptapTransformer.toYdoc(
{ type: 'doc', content },
'default',
tiptapExtensions as any,
);
it('3-way: materialized-default live vs engine-style git, base stale-by-one -> 0 ops', () => {
const liveDoc = toYdoc(liveContent);
const targetDoc = toYdoc(gitContent);
const baseDoc = toYdoc(gitContent.slice(0, gitContent.length - 1));
const live = liveDoc.getXmlFragment('default');
const before = live.toArray().length;
let applied = -1;
liveDoc.transact(() => {
applied = mergeXmlFragments3Way(
live,
targetDoc.getXmlFragment('default'),
baseDoc.getXmlFragment('default'),
);
});
expect(applied).toBe(0);
expect(live.toArray().length).toBe(before);
});
it('2-way: materialized-default live vs engine-style git -> 0 ops', () => {
const liveDoc = toYdoc(liveContent);
const targetDoc = toYdoc(gitContent);
const live = liveDoc.getXmlFragment('default');
const before = live.toArray().length;
let applied = -1;
liveDoc.transact(() => {
applied = mergeXmlFragments(live, targetDoc.getXmlFragment('default'));
});
expect(applied).toBe(0);
expect(live.toArray().length).toBe(before);
});
});

View File

@@ -0,0 +1,373 @@
import * as Y from 'yjs';
import {
mergeXmlFragments,
mergeXmlFragments3Way,
mergeXmlFragments3WayWithStats,
cloneXmlNode,
diffBlocks,
} from './yjs-body-merge';
// Build a Y.XmlFragment('default') in `doc` from a list of paragraph specs.
// Each spec is the paragraph's plain text (a single XmlText child).
function buildFragment(doc: Y.Doc, paragraphs: string[]): Y.XmlFragment {
const frag = doc.getXmlFragment('default');
const blocks = paragraphs.map((text) => {
const el = new Y.XmlElement('paragraph');
const t = new Y.XmlText();
if (text) t.insert(0, text);
el.insert(0, [t]);
return el;
});
if (blocks.length) frag.insert(0, blocks);
return frag;
}
function texts(frag: Y.XmlFragment): string[] {
return frag.toArray().map((el) => (el as Y.XmlElement).toArray()
.map((c) => (c as Y.XmlText).toString())
.join(''));
}
describe('yjs-body-merge', () => {
describe('diffBlocks (LCS edit script)', () => {
it('identical sequences produce only keeps (no edits)', () => {
const ops = diffBlocks(['a', 'b', 'c'], ['a', 'b', 'c']);
expect(ops.every((o) => o.op === 'keep')).toBe(true);
});
it('a single changed middle element is one del + one ins', () => {
const ops = diffBlocks(['a', 'b', 'c'], ['a', 'B', 'c']);
expect(ops.filter((o) => o.op === 'del')).toHaveLength(1);
expect(ops.filter((o) => o.op === 'ins')).toHaveLength(1);
expect(ops.filter((o) => o.op === 'keep')).toHaveLength(2);
});
});
describe('mergeXmlFragments', () => {
it('identical content is a complete no-op (0 ops) — never clobbers an unchanged resync', () => {
const live = new Y.Doc();
const target = new Y.Doc();
const liveFrag = buildFragment(live, ['one', 'two', 'three']);
const targetFrag = buildFragment(target, ['one', 'two', 'three']);
// Capture block identities to prove they are left untouched.
const before = liveFrag.toArray();
let applied = -1;
live.transact(() => {
applied = mergeXmlFragments(liveFrag, targetFrag);
});
expect(applied).toBe(0);
// Same Y.XmlElement instances — nothing was deleted/recreated.
expect(liveFrag.toArray()).toEqual(before);
expect(texts(liveFrag)).toEqual(['one', 'two', 'three']);
});
it('a human edit to one block survives a git change to a DIFFERENT block', () => {
// Live: the human has the doc open; block 0 holds their edit. Git changed
// only block 2. The merge must touch ONLY block 2 and leave block 0 (and
// its in-flight edit) exactly as-is.
const live = new Y.Doc();
const target = new Y.Doc();
const liveFrag = buildFragment(live, ['HUMAN EDIT', 'shared', 'old tail']);
const targetFrag = buildFragment(target, [
'HUMAN EDIT',
'shared',
'new tail from git',
]);
const block0Before = liveFrag.get(0); // the human's block instance
const block1Before = liveFrag.get(1);
let applied = -1;
live.transact(() => {
applied = mergeXmlFragments(liveFrag, targetFrag);
});
// Only block 2 was replaced: one del + one ins.
expect(applied).toBe(2);
// The human's block and the shared block are the SAME instances (untouched).
expect(liveFrag.get(0)).toBe(block0Before);
expect(liveFrag.get(1)).toBe(block1Before);
// Block 2 now carries git's content.
expect(texts(liveFrag)).toEqual([
'HUMAN EDIT',
'shared',
'new tail from git',
]);
});
it('appends a new trailing block without disturbing existing ones', () => {
const live = new Y.Doc();
const target = new Y.Doc();
const liveFrag = buildFragment(live, ['a', 'b']);
const targetFrag = buildFragment(target, ['a', 'b', 'c']);
const a = liveFrag.get(0);
const b = liveFrag.get(1);
let applied = -1;
live.transact(() => {
applied = mergeXmlFragments(liveFrag, targetFrag);
});
expect(applied).toBe(1); // single insert
expect(liveFrag.get(0)).toBe(a);
expect(liveFrag.get(1)).toBe(b);
expect(texts(liveFrag)).toEqual(['a', 'b', 'c']);
});
it('deletes a removed block, keeping its neighbours', () => {
const live = new Y.Doc();
const target = new Y.Doc();
const liveFrag = buildFragment(live, ['a', 'b', 'c']);
const targetFrag = buildFragment(target, ['a', 'c']);
const a = liveFrag.get(0);
let applied = -1;
live.transact(() => {
applied = mergeXmlFragments(liveFrag, targetFrag);
});
expect(applied).toBe(1); // single delete
expect(liveFrag.get(0)).toBe(a);
expect(texts(liveFrag)).toEqual(['a', 'c']);
});
it('a fully different body is replaced (and stays valid)', () => {
const live = new Y.Doc();
const target = new Y.Doc();
const liveFrag = buildFragment(live, ['x', 'y']);
const targetFrag = buildFragment(target, ['p', 'q', 'r']);
live.transact(() => mergeXmlFragments(liveFrag, targetFrag));
expect(texts(liveFrag)).toEqual(['p', 'q', 'r']);
});
});
describe('mergeXmlFragments3Way', () => {
it('keeps a human edit to one block while applying a git change to another (3-way)', () => {
// base (last synced): [a, b, c]. Human edited block 0 in the live doc; git
// changed block 2 in the incoming file. 3-way must keep BOTH — the 2-way
// merge would instead revert the human's block 0 to git's stale version.
const base = new Y.Doc();
const live = new Y.Doc();
const target = new Y.Doc();
const baseFrag = buildFragment(base, ['a', 'b', 'c']);
const liveFrag = buildFragment(live, ['HUMAN', 'b', 'c']);
const targetFrag = buildFragment(target, ['a', 'b', 'GIT']);
const humanBlock = liveFrag.get(0); // the human's live instance
live.transact(() =>
mergeXmlFragments3Way(liveFrag, targetFrag, baseFrag),
);
// Human's block preserved as the SAME instance; git's change applied.
expect(liveFrag.get(0)).toBe(humanBlock);
expect(texts(liveFrag)).toEqual(['HUMAN', 'b', 'GIT']);
});
it('a block both sides changed resolves to git (conflict policy)', () => {
const base = new Y.Doc();
const live = new Y.Doc();
const target = new Y.Doc();
const baseFrag = buildFragment(base, ['a', 'b', 'c']);
const liveFrag = buildFragment(live, ['a', 'HUMAN', 'c']);
const targetFrag = buildFragment(target, ['a', 'GIT', 'c']);
live.transact(() =>
mergeXmlFragments3Way(liveFrag, targetFrag, baseFrag),
);
expect(texts(liveFrag)).toEqual(['a', 'GIT', 'c']);
});
// Bug #2 observability: the stats variant reports the same-block conflict so
// the handler can log it + the persistence layer can pin the human baseline.
it('reports the same-block conflict count via mergeXmlFragments3WayWithStats', () => {
const base = new Y.Doc();
const live = new Y.Doc();
const target = new Y.Doc();
const baseFrag = buildFragment(base, ['a', 'b', 'c']);
const liveFrag = buildFragment(live, ['a', 'HUMAN', 'c']);
const targetFrag = buildFragment(target, ['a', 'GIT', 'c']);
let result!: { applied: number; conflicts: number };
live.transact(() => {
result = mergeXmlFragments3WayWithStats(liveFrag, targetFrag, baseFrag);
});
expect(result.conflicts).toBe(1);
expect(texts(liveFrag)).toEqual(['a', 'GIT', 'c']);
});
it('reports 0 conflicts for a clean different-block 3-way merge', () => {
const base = new Y.Doc();
const live = new Y.Doc();
const target = new Y.Doc();
const baseFrag = buildFragment(base, ['a', 'b', 'c']);
const liveFrag = buildFragment(live, ['HUMAN', 'b', 'c']);
const targetFrag = buildFragment(target, ['a', 'b', 'GIT']);
let result!: { applied: number; conflicts: number };
live.transact(() => {
result = mergeXmlFragments3WayWithStats(liveFrag, targetFrag, baseFrag);
});
expect(result.conflicts).toBe(0);
expect(texts(liveFrag)).toEqual(['HUMAN', 'b', 'GIT']);
});
it('git change with no concurrent human edit (live == base) applies cleanly', () => {
const base = new Y.Doc();
const live = new Y.Doc();
const target = new Y.Doc();
const baseFrag = buildFragment(base, ['a', 'b']);
const liveFrag = buildFragment(live, ['a', 'b']);
const targetFrag = buildFragment(target, ['a', 'B2']);
live.transact(() =>
mergeXmlFragments3Way(liveFrag, targetFrag, baseFrag),
);
expect(texts(liveFrag)).toEqual(['a', 'B2']);
});
});
// Regression: start-of-document content duplicating on every two-way sync.
//
// The LIVE Docmost doc stamps a per-block UniqueID on every heading/paragraph;
// a body arriving FROM git is parsed from clean markdown and carries NO block
// ids. If the merge comparison key includes that `id`, an unchanged live block
// never matches the SAME block coming from git, so the three-way merge cannot
// anchor on it — and an incoming block with no anchor (content inserted at the
// TOP of the page) is RE-ADDED on every cycle, an unbounded duplication loop.
// These tests model that exact id-asymmetry and assert the reconciliation is
// IDEMPOTENT (no block growth). They are RED before excluding `id` from the
// key in `serializeXmlNode`.
describe('idempotent reconciliation with live block ids (start-of-doc dup)', () => {
// Build a fragment from block specs. `id` is set only when provided, mirroring
// the live doc (ids present) vs a git-parsed body (ids absent).
type Spec = { tag: 'heading' | 'paragraph'; text: string; id?: string };
function buildDoc(doc: Y.Doc, specs: Spec[]): Y.XmlFragment {
const frag = doc.getXmlFragment('default');
const blocks = specs.map((s) => {
const el = new Y.XmlElement(s.tag);
if (s.id) el.setAttribute('id', s.id);
if (s.tag === 'heading') el.setAttribute('level', '2');
const t = new Y.XmlText();
if (s.text) t.insert(0, s.text);
el.insert(0, [t]);
return el;
});
if (blocks.length) frag.insert(0, blocks);
return frag;
}
const textsOf = (frag: Y.XmlFragment): string[] =>
frag.toArray().map((el) =>
(el as Y.XmlElement)
.toArray()
.map((c) => (c as Y.XmlText).toString())
.join(''),
);
it('re-merging the SAME git body does NOT re-add the top block (idempotent)', () => {
// last-synced base (from git markdown): NO block ids.
const base = new Y.Doc();
const baseFrag = buildDoc(base, [
{ tag: 'heading', text: 'Title' },
{ tag: 'paragraph', text: 'Some paragraph.' },
{ tag: 'paragraph', text: 'End block.' },
]);
// live Docmost doc: SAME content, but every block carries a UniqueID.
const live = new Y.Doc();
const liveFrag = buildDoc(live, [
{ tag: 'heading', text: 'Title', id: 'ida' },
{ tag: 'paragraph', text: 'Some paragraph.', id: 'idb' },
{ tag: 'paragraph', text: 'End block.', id: 'idc' },
]);
// incoming git body: the user inserted a heading at the very TOP.
const buildTarget = (): Y.XmlFragment =>
buildDoc(new Y.Doc(), [
{ tag: 'heading', text: 'TOPDUP' },
{ tag: 'heading', text: 'Title' },
{ tag: 'paragraph', text: 'Some paragraph.' },
{ tag: 'paragraph', text: 'End block.' },
]);
// First sync: the top block is added once.
live.transact(() =>
mergeXmlFragments3Way(liveFrag, buildTarget(), baseFrag),
);
expect(textsOf(liveFrag)).toEqual([
'TOPDUP',
'Title',
'Some paragraph.',
'End block.',
]);
// Subsequent sync of the SAME git body against the SAME base must be a
// NO-OP — not a second copy of the top block. Before the fix this re-adds
// 'TOPDUP', growing the doc on every cycle.
live.transact(() =>
mergeXmlFragments3Way(liveFrag, buildTarget(), baseFrag),
);
expect(textsOf(liveFrag)).toEqual([
'TOPDUP',
'Title',
'Some paragraph.',
'End block.',
]);
expect(textsOf(liveFrag).filter((t) => t === 'TOPDUP')).toHaveLength(1);
});
it('an unchanged git body (live ids, none in git) is a complete no-op', () => {
// base == git body (no pending git change); live is the same content with
// ids. With `id` in the key the whole body looks rewritten; the merge must
// still leave live byte-identical (block instances untouched).
const base = new Y.Doc();
const baseFrag = buildDoc(base, [
{ tag: 'heading', text: 'Title' },
{ tag: 'paragraph', text: 'Body.' },
]);
const live = new Y.Doc();
const liveFrag = buildDoc(live, [
{ tag: 'heading', text: 'Title', id: 'ida' },
{ tag: 'paragraph', text: 'Body.', id: 'idb' },
]);
const before = liveFrag.toArray();
let applied = -1;
live.transact(() => {
applied = mergeXmlFragments3Way(
liveFrag,
buildDoc(new Y.Doc(), [
{ tag: 'heading', text: 'Title' },
{ tag: 'paragraph', text: 'Body.' },
]),
baseFrag,
);
});
expect(applied).toBe(0);
// Same live block instances (ids preserved) — nothing recreated.
expect(liveFrag.toArray()).toEqual(before);
});
});
describe('cloneXmlNode', () => {
it('preserves text marks (XmlText delta) across docs', () => {
const src = new Y.Doc();
const srcFrag = src.getXmlFragment('default');
const el = new Y.XmlElement('paragraph');
const t = new Y.XmlText();
t.insert(0, 'plain ');
t.insert(6, 'bold', { bold: true });
el.insert(0, [t]);
srcFrag.insert(0, [el]);
const dst = new Y.Doc();
const dstFrag = dst.getXmlFragment('default');
dstFrag.insert(0, [cloneXmlNode(srcFrag.get(0) as Y.XmlElement)]);
const clonedText = (dstFrag.get(0) as Y.XmlElement).get(0) as Y.XmlText;
expect(clonedText.toDelta()).toEqual([
{ insert: 'plain ' },
{ insert: 'bold', attributes: { bold: true } },
]);
});
});
});

View File

@@ -0,0 +1,369 @@
import * as Y from 'yjs';
import { getSchema } from '@tiptap/core';
import type { Schema } from '@tiptap/pm/model';
import { tiptapExtensions } from '../collaboration.util';
import { diff3PlanWithConflicts } from './three-way-merge';
import { buildLcsTable } from './lcs';
/**
* Block-level merge of an incoming (git) page body into a LIVE Yjs document,
* replacing the previous full-body "delete everything + re-insert" write that
* clobbered concurrent human edits on every sync (review #5 — "do the write as a
* merge").
*
* Strategy: diff the two documents at TOP-LEVEL BLOCK granularity (an LCS over a
* canonical structural serialization of each block) and apply only the minimal
* insert/delete operations. Blocks that are byte-identical on both sides are
* left UNTOUCHED in the live doc — so a human editing one paragraph is unaffected
* when git changes a different paragraph, and an unchanged re-sync is a complete
* no-op (zero Yjs operations). Yjs then CRDT-merges the minimal ops with any
* concurrent edits.
*
* Limitation (honest): this is a 2-way merge (live vs incoming). For a block that
* BOTH sides changed since the last sync it cannot tell which is newer without a
* common ancestor, so the incoming (git) version wins for that one block. A full
* 3-way merge would need the last-synced base plumbed from the engine; the common
* cases — unchanged resync, and edits to DIFFERENT blocks — are handled losslessly.
*/
type XmlNode = Y.XmlElement | Y.XmlText | Y.XmlHook;
/**
* Node attributes that are VOLATILE identity (not content) and so must be
* excluded from the block comparison key.
*
* `id` is the per-block UniqueID the editor stamps on every heading/paragraph
* (and transclusionSource). It exists ONLY in the live Yjs document — a body
* arriving from git is parsed from clean markdown, which carries no block ids
* (`markdownToProseMirror` materializes `id: null`, which the Yjs transform then
* drops). If `id` were part of the key, an UNCHANGED live block (id "abc123")
* would never match the SAME block coming from git (no id), so the three-way
* merge's LCS could not anchor on it. The merge would then treat every live
* block as deleted-and-reinserted and, when an incoming block has no matching
* anchor (e.g. content inserted at the very TOP of the page), RE-ADD a copy of
* it on every sync cycle — a non-convergent, unbounded duplication loop
* (start-of-document content duplicating each push/pull cycle).
*
* Excluding `id` makes blocks compare by CONTENT, so an unchanged block matches
* across the git round-trip and the reconciliation is idempotent. Block identity
* is still preserved in the merged output: `diff3Plan` keeps the LIVE block
* INSTANCE (with its id) for an anchor — picks are by index, not by key — so the
* stable Yjs block (and any in-flight human edit on it) stays put. This mirrors
* `canonicalize.ts`, which already strips the regenerated block `id` from the
* round-trip idempotency comparison for exactly the same reason.
*
* Known limitation (accepted trade-off of content-based matching): two GENUINELY
* DISTINCT blocks whose content is byte-identical now collapse to the same content
* key, so when git deletes one of the duplicates the LCS may drop the OTHER live
* instance instead. The visible result is identical (one copy removed, one kept),
* but a concurrent in-flight human edit on the dropped instance could be lost.
*/
const VOLATILE_KEY_ATTRS = new Set(['id']);
/**
* The editor (ProseMirror) schema, built ONCE from the same `tiptapExtensions`
* the collaboration server uses to materialize Yjs docs. Memoized: building the
* schema is non-trivial and the block key is computed per block per cycle.
*
* Why the schema (not a hardcoded denylist): the LIVE Yjs document is produced by
* `TiptapTransformer.toYdoc(pm, 'default', tiptapExtensions)`, which STAMPS every
* schema-default attribute onto every node and mark — `indent: 0` on every
* paragraph/heading, `image.align: "center"`, the link mark's `internal: false`,
* `highlight.colorName: null`, and so on for youtube/pdf/any future node. A body
* re-imported from git comes through the engine's `markdownToProseMirror`, whose
* schema declares those attrs with DIFFERENT (usually null) defaults; the
* resulting null/absent element attrs are then DROPPED by `y-prosemirror`'s
* toYdoc. So the SAME block carries materialized defaults on the live side and
* nothing on the git side, its key diverges, the three-way merge anchors on
* NOTHING, and the whole body is RE-APPENDED every reconcile cycle — an unbounded
* duplication loop with no client connected.
*
* Deriving the defaults from the actual schema normalizes ALL such attributes
* generally (it is not another per-attribute denylist): any attribute whose value
* equals the schema default — or is null/undefined — is dropped from the key, on
* BOTH element attributes and the mark attributes inside each XmlText delta, so a
* live block compares equal to its git-round-tripped twin and an unchanged resync
* applies zero ops. Genuinely non-default values (a real `indent: 2`, an
* `align: "left"`, a real `link.href`, a real highlight color) are content and
* stay in the key, so real edits still diff and land.
*/
let memoSchema: Schema | null = null;
let memoSchemaTried = false;
function getMergeSchema(): Schema | null {
if (!memoSchemaTried) {
memoSchemaTried = true;
try {
memoSchema = getSchema(tiptapExtensions as any);
} catch {
// Defensive: if the schema can't be built (e.g. a degenerate extension
// set in a unit test that stubs `tiptapExtensions`), fall back to dropping
// only null/undefined attrs. The real server always builds it fine.
memoSchema = null;
}
}
return memoSchema;
}
/** True if `value` is the schema default for `attrName` of `attrSpecs`, or is
* null/undefined (which a git round-trip drops). Such attributes are excluded
* from the comparison key. `attrSpecs` is a ProseMirror node/mark spec attr map
* (`{ [name]: { default } }`); a missing map (unknown node/mark) only drops
* null/undefined. (A non-null value matching an attr declared without a default
* cannot occur — `spec.default === value` is then `undefined === value`, false.) */
function isDefaultAttr(
attrSpecs: Record<string, any> | undefined | null,
attrName: string,
value: unknown,
): boolean {
if (value === null || value === undefined) return true;
const spec = attrSpecs?.[attrName];
return !!spec && spec.default === value;
}
/**
* Normalize one XmlText delta op's mark attributes: drop every mark-attr whose
* value equals the mark's schema default (or is null/undefined), so the link
* mark's materialized `internal: false`/`target: "_blank"` and a highlight's
* `colorName: null` no longer diverge from a git round-trip that carries neither.
* The text (op.insert) and genuinely-set mark attrs (a real `href`, a real
* highlight color) are preserved verbatim. `attributes` maps markName -> mark
* attrs object (or `true`/boolean for attr-less marks); each is handled safely.
*/
function normalizeDelta(delta: any[]): any[] {
const schema = getMergeSchema();
return delta.map((op) => {
if (!op || op.attributes == null || typeof op.attributes !== 'object') {
return op;
}
const marks: Record<string, unknown> = {};
for (const markName of Object.keys(op.attributes).sort()) {
const markVal = op.attributes[markName];
if (markVal === null || markVal === undefined) continue;
if (typeof markVal !== 'object') {
// attr-less mark stored as a primitive (e.g. `true`) — keep as-is.
marks[markName] = markVal;
continue;
}
const markSpec = schema?.marks[markName]?.spec.attrs as
| Record<string, any>
| undefined;
const cleaned: Record<string, unknown> = {};
for (const ak of Object.keys(markVal as object).sort()) {
const av = (markVal as Record<string, unknown>)[ak];
if (isDefaultAttr(markSpec, ak, av)) continue;
cleaned[ak] = av;
}
marks[markName] = cleaned;
}
return { ...op, attributes: marks };
});
}
/**
* Canonical, comparable serialization of a Yjs XML node (structure + text +
* marks + attributes), with attribute keys sorted so equal blocks always produce
* an identical string regardless of attribute insertion order. The volatile
* block `id` (see `VOLATILE_KEY_ATTRS`) and every schema-default attribute (see
* `getMergeSchema`) are excluded at every level — on element attributes AND on
* the mark attributes inside each XmlText delta — so a block compares equal by
* CONTENT across the git round-trip (which materializes neither), keeping the
* merge anchor-able and idempotent.
*/
export function serializeXmlNode(node: unknown): unknown {
if (node instanceof Y.XmlText) {
return { t: normalizeDelta(node.toDelta()) };
}
if (node instanceof Y.XmlElement) {
const attrs = node.getAttributes() as Record<string, unknown>;
const attrSpecs = getMergeSchema()?.nodes[node.nodeName]?.spec.attrs as
| Record<string, any>
| undefined;
const sorted: Record<string, unknown> = {};
for (const k of Object.keys(attrs).sort()) {
if (VOLATILE_KEY_ATTRS.has(k)) continue;
if (isDefaultAttr(attrSpecs, k, attrs[k])) continue;
sorted[k] = attrs[k];
}
return {
n: node.nodeName,
a: sorted,
c: node.toArray().map(serializeXmlNode),
};
}
// XmlHook / unknown: fall back to a stable string so it compares by identity
// of its serialized form (these do not occur in the Docmost block schema).
return { u: String(node) };
}
const key = (node: unknown): string => JSON.stringify(serializeXmlNode(node));
/**
* Deep-clone a detached/owned Yjs XML node into a fresh node that can be inserted
* into ANOTHER document (Yjs types are bound to their doc, so cross-doc moves are
* impossible — we rebuild). Preserves nodeName, attributes, text+marks (via the
* XmlText delta) and the full child subtree.
*/
export function cloneXmlNode(node: XmlNode): Y.XmlElement | Y.XmlText {
if (node instanceof Y.XmlText) {
const t = new Y.XmlText();
const delta = node.toDelta();
if (delta.length) t.applyDelta(delta);
return t;
}
if (node instanceof Y.XmlElement) {
const el = new Y.XmlElement(node.nodeName);
const attrs = node.getAttributes() as Record<string, unknown>;
for (const k of Object.keys(attrs)) el.setAttribute(k, attrs[k] as string);
const kids = node.toArray().map((c) => cloneXmlNode(c as XmlNode));
if (kids.length) el.insert(0, kids);
return el;
}
// Best-effort for any other node type (XmlHook — does not occur in the
// Docmost block schema): an empty paragraph so the merge never crashes.
return new Y.XmlElement('paragraph');
}
type Op = { op: 'keep' } | { op: 'del' } | { op: 'ins'; bi: number };
/**
* LCS-based edit script turning sequence `a` (live block keys) into `b` (incoming
* block keys): a run of keep/del/ins ops. O(n*m) table — fine for page block
* counts.
*/
export function diffBlocks(a: string[], b: string[]): Op[] {
const n = a.length;
const m = b.length;
const dp = buildLcsTable(a, b);
const ops: Op[] = [];
let i = 0;
let j = 0;
while (i < n && j < m) {
if (a[i] === b[j]) {
ops.push({ op: 'keep' });
i++;
j++;
} else if (dp[i + 1][j] >= dp[i][j + 1]) {
ops.push({ op: 'del' });
i++;
} else {
ops.push({ op: 'ins', bi: j });
j++;
}
}
while (i < n) {
ops.push({ op: 'del' });
i++;
}
while (j < m) {
ops.push({ op: 'ins', bi: j });
j++;
}
return ops;
}
/**
* Merge `target` block children into `live`, mutating `live` in place with the
* minimal set of inserts/deletes. MUST be called inside a Yjs transaction.
* Returns the number of block operations applied (0 == content already identical).
*/
export function mergeXmlFragments(
live: Y.XmlFragment,
target: Y.XmlFragment,
): number {
const liveKids = live.toArray();
const targetKids = target.toArray();
const liveKeys = liveKids.map(key);
const targetKeys = targetKids.map(key);
const ops = diffBlocks(liveKeys, targetKeys);
let cursor = 0; // index into the LIVE fragment as we mutate it
let applied = 0;
for (const op of ops) {
if (op.op === 'keep') {
cursor++;
} else if (op.op === 'del') {
live.delete(cursor, 1); // remove the live block at the cursor; do not advance
applied++;
} else {
live.insert(cursor, [cloneXmlNode(targetKids[op.bi] as XmlNode)]);
cursor++;
applied++;
}
}
return applied;
}
/** Outcome of a 3-way block merge: ops applied + same-block conflict count. */
export interface Merge3WayResult {
/** Number of block insert/delete operations spliced into `live`. */
applied: number;
/**
* Regions where the human AND git rewrote the SAME base block. The rule is
* deterministic (GIT WINS the region), so the human's version of those blocks
* is dropped from the live doc. `conflicts > 0` is the OBSERVABLE signal the
* caller uses to LOG the loss and pin the human baseline to page history (so it
* is recoverable), instead of the edit vanishing silently.
*/
conflicts: number;
}
/**
* THREE-WAY block merge: reconcile `live` toward `target` using `base` (the
* last-synced common ancestor) so a block only the human changed is KEPT and a
* block only git changed is taken — instead of git's version always winning
* (review #5). Conflicts (both changed the same block) resolve to git.
*
* Implementation: diff3Plan computes the merged block ORDER (picks from live or
* target); we materialize that as a virtual target fragment and reuse the 2-way
* `mergeXmlFragments` to splice it into `live` minimally (so untouched live block
* instances — and their in-flight edits — stay put). MUST be called inside a Yjs
* transaction. Returns the number of block operations applied. (Use
* `mergeXmlFragments3WayWithStats` when the SAME-BLOCK conflict count is needed.)
*/
export function mergeXmlFragments3Way(
live: Y.XmlFragment,
target: Y.XmlFragment,
base: Y.XmlFragment,
): number {
return mergeXmlFragments3WayWithStats(live, target, base).applied;
}
/**
* As `mergeXmlFragments3Way`, but also returns the SAME-BLOCK conflict count so
* the caller can make a "git won a concurrent same-block edit" event OBSERVABLE
* (the documented conflict contract: git wins deterministically, but the losing
* human content is never destroyed silently — it is logged and recoverable via
* page history).
*/
export function mergeXmlFragments3WayWithStats(
live: Y.XmlFragment,
target: Y.XmlFragment,
base: Y.XmlFragment,
): Merge3WayResult {
const liveKids = live.toArray();
const targetKids = target.toArray();
const liveKeys = liveKids.map(key);
const targetKeys = targetKids.map(key);
const baseKeys = base.toArray().map(key);
const { picks: plan, conflicts } = diff3PlanWithConflicts(
baseKeys,
liveKeys,
targetKeys,
);
// Build the merged block sequence in a throwaway doc, cloning from whichever
// side each pick came from, then 2-way merge it back into the live fragment.
const merged = new Y.Doc();
const mergedFrag = merged.getXmlFragment('default');
const nodes = plan.map((p) =>
cloneXmlNode(
(p.src === 'live' ? liveKids[p.index] : targetKids[p.index]) as XmlNode,
),
);
if (nodes.length) mergedFrag.insert(0, nodes);
return { applied: mergeXmlFragments(live, mergedFrag), conflicts };
}

View File

@@ -20,7 +20,6 @@ import { CaslModule } from '../../core/casl/casl.module';
import { ThrottleModule } from '../../integrations/throttle/throttle.module';
import { CacheModule } from '@nestjs/cache-manager';
import KeyvRedis from '@keyv/redis';
import { PageTreeBridgePublisher } from '../listeners/page-tree-bridge.publisher';
@Module({
imports: [
@@ -55,6 +54,6 @@ import { PageTreeBridgePublisher } from '../listeners/page-tree-bridge.publisher
? [CollaborationController]
: []),
],
providers: [AppService, PageTreeBridgePublisher],
providers: [AppService],
})
export class CollabAppModule {}

View File

@@ -1,187 +0,0 @@
import * as Y from 'yjs';
import { TiptapTransformer } from '@hocuspocus/transformer';
import { CollaborationGateway } from './collaboration.gateway';
import { PersistenceExtension } from './extensions/persistence.extension';
import {
buildTitleSeedYdoc,
jsonToText,
tiptapExtensions,
} from './collaboration.util';
/**
* F1 (variant C) — rename durability for a page with an already-persisted Yjs
* 'title' fragment and NO live editor (the REST/MCP/agent rename path).
*
* The bug: PageService.update writes the NEW title to the `page.title` COLUMN,
* then calls gateway.writePageTitle, which loads the page's ydoc (fragment =
* OLD) and overwrites it to NEW in memory. On disconnect, onStoreDocument sees
* titleText(NEW) === column(NEW) → no-op fast-path → it does NOT persist the
* in-memory fragment. So `page.ydoc` keeps the OLD title, and a LATER body edit
* loads the OLD fragment, sees it differs from the column, and silently reverts
* the column back to OLD.
*
* The fix: writePageTitle persists the 'title' fragment to `page.ydoc` DIRECTLY
* (via PersistenceExtension.persistTitleFragmentYdoc) after the transact, so the
* persisted fragment and the column stay consistent.
*
* This test drives the REAL writePageTitle + the REAL onStoreDocument against an
* in-memory page row, so it FAILS on the pre-fix no-op behaviour and PASSES after.
*/
const PAGE_ID = '550e8400-e29b-41d4-a716-446655440000';
const USER_ID = 'user-1';
const OLD_TITLE = 'Old Title';
const NEW_TITLE = 'Renamed Title';
const bodyJson = (text: string) => ({
type: 'doc',
content: [{ type: 'paragraph', content: [{ type: 'text', text }] }],
});
// Build the initial persisted ydoc carrying BOTH a 'title' fragment and a body.
const makeInitialYdoc = (title: string, body: any): Buffer => {
const doc = new Y.Doc();
Y.applyUpdate(doc, Y.encodeStateAsUpdate(buildTitleSeedYdoc(title)));
Y.applyUpdate(
doc,
Y.encodeStateAsUpdate(TiptapTransformer.toYdoc(body, 'default', tiptapExtensions)),
);
return Buffer.from(Y.encodeStateAsUpdate(doc));
};
// Load a doc from a persisted buffer (mirrors openDirectConnection loading from
// persistence when no editor is connected). hocuspocus augments the live doc
// with broadcastStateless(); a bare Y.Doc lacks it, so stub it.
const loadDoc = (buf: Buffer): Y.Doc => {
const doc = new Y.Doc();
if (buf) Y.applyUpdate(doc, new Uint8Array(buf));
(doc as any).broadcastStateless = jest.fn();
return doc;
};
// Read the 'title' fragment text from a persisted buffer.
const readTitle = (buf: Buffer): string => {
const doc = loadDoc(buf);
const titleJson = TiptapTransformer.fromYdoc(doc, 'title');
return titleJson ? jsonToText(titleJson).trim() : '';
};
describe('rename durability (F1 variant C): persisted title fragment survives a body edit', () => {
it('persists the renamed title into page.ydoc so a later body edit does not revert it', async () => {
// In-memory page row = the DB.
const row: any = {
id: PAGE_ID,
slugId: 'slug-1',
spaceId: 'space-1',
workspaceId: 'ws-1',
creatorId: 'creator-1',
contributorIds: ['creator-1'],
createdAt: new Date('2020-01-01T00:00:00Z'),
lastUpdatedSource: 'user',
title: OLD_TITLE,
// content column mirrors the normalized body in the ydoc.
content: TiptapTransformer.fromYdoc(
loadDoc(makeInitialYdoc(OLD_TITLE, bodyJson('BODY V1'))),
'default',
),
ydoc: makeInitialYdoc(OLD_TITLE, bodyJson('BODY V1')),
};
const pageRepo = {
findById: jest.fn(async () => ({ ...row })),
updatePage: jest.fn(async (data: any, _pageId?: string) => {
Object.assign(row, data, { updatedAt: new Date() });
}),
};
const pageHistoryRepo = {
saveHistory: jest.fn().mockResolvedValue(undefined),
findPageLastHistory: jest.fn().mockResolvedValue(null),
};
const noopQueue = { add: jest.fn().mockResolvedValue(undefined) };
const collabHistory = { addContributors: jest.fn().mockResolvedValue(undefined) };
const transclusionService = {
syncPageTransclusions: jest.fn().mockResolvedValue(undefined),
syncPageReferences: jest.fn().mockResolvedValue(undefined),
syncPageTemplateReferences: jest.fn().mockResolvedValue(undefined),
};
// db whose transaction().execute(fn) runs fn with a trx stub (drives the
// real executeTx helper without a database).
const db = {
transaction: () => ({
execute: (fn: (trx: any) => Promise<any>) => fn({ __trx: true }),
}),
};
const ext = new PersistenceExtension(
pageRepo as any,
pageHistoryRepo as any,
db as any,
noopQueue as any,
noopQueue as any,
noopQueue as any,
collabHistory as any,
transclusionService as any,
);
jest.spyOn(ext['logger'], 'debug').mockImplementation(() => undefined);
jest.spyOn(ext['logger'], 'warn').mockImplementation(() => undefined);
jest.spyOn(ext['logger'], 'error').mockImplementation(() => undefined);
const documentName = `page.${PAGE_ID}`;
// Fake hocuspocus: openDirectConnection loads a doc from the CURRENT persisted
// ydoc (no live editor) and, on disconnect, runs the real onStoreDocument —
// exactly the no-live-editor unload path.
const fakeHocuspocus = {
openDirectConnection: jest.fn(async (name: string, context: any) => {
const liveDoc = loadDoc(row.ydoc);
return {
transact: async (fn: (doc: Y.Doc) => void) => fn(liveDoc),
disconnect: async () => {
await ext.onStoreDocument({
documentName: name,
document: liveDoc,
context,
} as any);
},
};
}),
};
const gateway: CollaborationGateway = Object.create(
CollaborationGateway.prototype,
);
(gateway as any).hocuspocus = fakeHocuspocus;
(gateway as any).persistenceExtension = ext;
// --- REST/service rename (no live editor) ---
// 1) PageService.update writes the NEW title to the column.
await pageRepo.updatePage({ title: NEW_TITLE }, PAGE_ID);
// 2) PageService.update syncs the Yjs 'title' fragment.
await gateway.writePageTitle(PAGE_ID, NEW_TITLE, {
user: { id: USER_ID } as any,
});
// Reload the persisted ydoc: the 'title' fragment must now be NEW.
// (Pre-fix this is still OLD — writePageTitle did not persist the fragment.)
expect(readTitle(row.ydoc)).toBe(NEW_TITLE);
// --- a later body edit must NOT revert the title ---
const editDoc = loadDoc(row.ydoc);
const frag = editDoc.getXmlFragment('default');
const p = new Y.XmlElement('paragraph');
const t = new Y.XmlText();
t.insert(0, 'appended');
p.insert(0, [t]);
frag.insert(frag.length, [p]);
await ext.onStoreDocument({
documentName,
document: editDoc,
context: { user: { id: USER_ID } },
} as any);
// The body edit was persisted, and the title stayed NEW in BOTH the column
// and the persisted ydoc fragment (pre-fix the column reverts to OLD).
expect(row.title).toBe(NEW_TITLE);
expect(readTitle(row.ydoc)).toBe(NEW_TITLE);
});
});

View File

@@ -73,6 +73,32 @@ describe('agentSourceFields', () => {
).toEqual({ lastUpdatedSource: 'agent', lastUpdatedAiChatId: null });
});
it("stamps ONLY the source column 'git-sync' (no chat key) for a git-sync write", () => {
// The git-sync data plane (issue #194 §8.1) has no internal ai_chats row, so
// it stamps the *Source column 'git-sync' and OMITS the chat key entirely
// (unlike the agent branch, which also writes aiChatId). Pinned directly here
// because the page.service.spec only exercises it indirectly.
expect(
agentSourceFields(
{ actor: 'git-sync', aiChatId: null },
'lastUpdatedSource',
'lastUpdatedAiChatId',
),
).toEqual({ lastUpdatedSource: 'git-sync' });
});
it("ignores any aiChatId on a git-sync write (chat key never written)", () => {
// Even if a non-null aiChatId is present, the git-sync branch must not emit
// the chat key.
expect(
agentSourceFields(
{ actor: 'git-sync', aiChatId: 'should-be-ignored' },
'createdSource',
'aiChatId',
),
).toEqual({ createdSource: 'git-sync' });
});
it('returns {} for a user write so the column keeps its default', () => {
expect(
agentSourceFields(

View File

@@ -9,6 +9,8 @@ import { ProvenanceSource } from '../../core/auth/dto/jwt-payload';
* cannot fake an 'agent' marker.
*/
export interface AuthProvenanceData {
// ProvenanceSource includes 'git-sync' — set by the in-process git-sync data
// plane (issue #194 §8.1) when it drives PageService writes; never from a request token.
actor: ProvenanceSource;
aiChatId: string | null;
}
@@ -60,6 +62,14 @@ export function agentSourceFields<S extends string, C extends string>(
sourceKey: S,
chatKey: C,
): Partial<Record<S, ProvenanceSource> & Record<C, string | null>> {
// git-sync data-plane write (issue #194 §8.1): stamp the source 'git-sync' with NO
// aiChatId (it has no internal ai_chats row). Mirrors the agent branch; each
// write has a single actor, so precedence is irrelevant here.
if (provenance?.actor === 'git-sync') {
return { [sourceKey]: 'git-sync' } as Partial<
Record<S, ProvenanceSource> & Record<C, string | null>
>;
}
if (provenance?.actor !== 'agent') return {};
return {
[sourceKey]: 'agent',

View File

@@ -0,0 +1,18 @@
/**
* Dynamic ESM import bridge for a CommonJS build.
*
* The server compiles with `module: commonjs`, and TypeScript downlevels a
* literal `import()` expression to `require()` — which cannot load an ESM-only
* package (`@docmost/mcp`, `@docmost/git-sync`). Indirecting through `new
* Function` hides the `import()` from the TS downleveler so the REAL dynamic
* `import()` survives to runtime and can load ESM from CommonJS.
*
* This is the single shared copy of that bridge. The per-package typed loaders
* (git-sync.loader.ts, docmost-client.loader.ts, mcp.service.ts) import this and
* keep their own typed `loadX()` wrappers (require.resolve + pathToFileURL +
* memoization) on top.
*/
export const esmImport = new Function(
'specifier',
'return import(specifier)',
) as (specifier: string) => Promise<unknown>;

View File

@@ -0,0 +1,71 @@
import { resolveRequestWorkspace } from './resolve-request-workspace';
// Unit tests for the shared self-hosted/cloud workspace resolver deduplicated out
// of DomainMiddleware + GitHttpService (architecture #11). They must behave
// identically, so this pins the single source of truth.
type AnyMock = jest.Mock;
function build(opts: {
selfHosted: boolean;
first?: { id: string } | null;
byHostname?: { id: string } | null;
}) {
const env = {
isSelfHosted: jest.fn(() => opts.selfHosted),
isCloud: jest.fn(() => !opts.selfHosted),
};
const repo = {
findFirst: jest.fn(async () => opts.first ?? null) as AnyMock,
findByHostname: jest.fn(async () => opts.byHostname ?? null) as AnyMock,
};
return { env, repo };
}
describe('resolveRequestWorkspace', () => {
it('self-hosted: returns the first/default workspace, ignoring the host', async () => {
const { env, repo } = build({ selfHosted: true, first: { id: 'ws-1' } });
const ws = await resolveRequestWorkspace(
env as any,
repo as any,
'anything.example.com',
);
expect(ws).toEqual({ id: 'ws-1' });
expect(repo.findFirst).toHaveBeenCalledTimes(1);
expect(repo.findByHostname).not.toHaveBeenCalled();
});
it('self-hosted: returns null when no workspace is configured', async () => {
const { env, repo } = build({ selfHosted: true, first: null });
expect(await resolveRequestWorkspace(env as any, repo as any, 'h')).toBeNull();
});
it('cloud: resolves by the host-header subdomain', async () => {
const { env, repo } = build({
selfHosted: false,
byHostname: { id: 'ws-acme' },
});
const ws = await resolveRequestWorkspace(
env as any,
repo as any,
'acme.example.com',
);
expect(ws).toEqual({ id: 'ws-acme' });
expect(repo.findByHostname).toHaveBeenCalledWith('acme');
expect(repo.findFirst).not.toHaveBeenCalled();
});
it('cloud: returns null for a blank/missing host (no throw)', async () => {
const { env, repo } = build({ selfHosted: false, byHostname: { id: 'x' } });
expect(await resolveRequestWorkspace(env as any, repo as any, undefined)).toBeNull();
expect(await resolveRequestWorkspace(env as any, repo as any, '')).toBeNull();
expect(repo.findByHostname).not.toHaveBeenCalled();
});
it('cloud: returns null when the subdomain matches no workspace', async () => {
const { env, repo } = build({ selfHosted: false, byHostname: null });
expect(
await resolveRequestWorkspace(env as any, repo as any, 'ghost.example.com'),
).toBeNull();
});
});

View File

@@ -0,0 +1,35 @@
import { WorkspaceRepo } from '@docmost/db/repos/workspace/workspace.repo';
import { Workspace } from '@docmost/db/types/entity.types';
import { EnvironmentService } from '../../integrations/environment/environment.service';
/**
* The ONE canonical way to resolve the workspace for an incoming request:
* - self-hosted (single workspace) -> the first/default workspace;
* - cloud (multi-tenant) -> resolved by the host-header subdomain.
* Returns null when none resolves (no workspace configured, or a blank/unknown
* subdomain on cloud). `isSelfHosted()` is `!isCloud()`, so exactly one branch is
* always taken.
*
* Extracted so the self-hosted/cloud branch is not hand-duplicated. Shared by
* `DomainMiddleware` (the normal /api request path) and `GitHttpService` (the raw
* root-mounted /git smart-HTTP host, which Nest middleware does NOT run for) so
* the two cannot drift.
*
* This helper does NOT catch DB errors — callers decide: DomainMiddleware lets a
* throw bubble (as before); GitHttpService wraps it to log + treat as
* unresolvable (-> 404). A blank/missing host on cloud resolves to null rather
* than throwing.
*/
export async function resolveRequestWorkspace(
environmentService: EnvironmentService,
workspaceRepo: WorkspaceRepo,
hostHeader: string | undefined,
): Promise<Workspace | null> {
if (environmentService.isSelfHosted()) {
return (await workspaceRepo.findFirst()) ?? null;
}
// Cloud (isSelfHosted === !isCloud, so this is the only remaining branch).
const subdomain = hostHeader ? hostHeader.split('.')[0] : '';
if (!subdomain) return null;
return (await workspaceRepo.findByHostname(subdomain)) ?? null;
}

View File

@@ -1,7 +1,8 @@
import { Injectable, NestMiddleware, NotFoundException } from '@nestjs/common';
import { Injectable, NestMiddleware } from '@nestjs/common';
import { FastifyRequest, FastifyReply } from 'fastify';
import { EnvironmentService } from '../../integrations/environment/environment.service';
import { WorkspaceRepo } from '@docmost/db/repos/workspace/workspace.repo';
import { resolveRequestWorkspace } from '../helpers/resolve-request-workspace';
@Injectable()
export class DomainMiddleware implements NestMiddleware {
@@ -14,30 +15,19 @@ export class DomainMiddleware implements NestMiddleware {
res: FastifyReply['raw'],
next: () => void,
) {
if (this.environmentService.isSelfHosted()) {
const workspace = await this.workspaceRepo.findFirst();
if (!workspace) {
//throw new NotFoundException('Workspace not found');
(req as any).workspaceId = null;
return next();
}
// TODO: unify
(req as any).workspaceId = workspace.id;
(req as any).workspace = workspace;
} else if (this.environmentService.isCloud()) {
const header = req.headers.host;
const subdomain = header.split('.')[0];
const workspace = await this.workspaceRepo.findByHostname(subdomain);
if (!workspace) {
(req as any).workspaceId = null;
return next();
}
// Shared self-hosted/cloud resolution (the SAME branch the /git host uses),
// so the logic cannot drift between the two.
const workspace = await resolveRequestWorkspace(
this.environmentService,
this.workspaceRepo,
req.headers.host,
);
if (workspace) {
(req as any).workspaceId = workspace.id;
(req as any).workspace = workspace;
} else {
(req as any).workspaceId = null;
}
next();

Some files were not shown because too many files have changed in this diff Show More