Commit Graph

11 Commits

Author SHA1 Message Date
vvzvlad
1750058503 refactor(sync): testability seams for pull + collab; integration tests
Behavior-preserving refactors (R-Collab-1, R-Pull-1, R-Pull-2) to unblock testing,
plus the integration tests they enable.

- collaboration: extract applyTransformToYdoc from onSynced; onSynced stays
  synchronous (NO await between Yjs read and write — SPEC §2 atomicity preserved)
- pull: readExisting(deps) injectable IO; split main into pure computePullActions
  (plan + suppression/mass-delete decisions) + thin applyPullActions(deps) (IO);
  ordering and data-loss guards preserved bit-for-bit
- tests (+35): collaboration-apply (atomicity/null-abort/throw-no-partial),
  read-existing, compute/apply-pull-actions (move-write-fail keeps old path),
  git temp-repo 3-way non-FF merge
- transforms-extra property: constrain the generator to mutually-non-substring
  words (the domain where the renumber property holds) -> deterministic; document
  the inherited commentsToFootnotes substring-overlap comment-drop via it.fails
  (off the sync path, SPEC §3; backport-fix lives in docmost-mcp)
- 695 -> 731 green; build clean; corpus STABLE
2026-06-17 01:29:49 +03:00
vvzvlad
d9d8538846 test(sync): implement test-strategy Phase 1-2 (pure unit/golden/property), +102 tests
Work through test-strategy-report.md, high-ROI no-refactor subset (no regen).

- R-Infra: vitest resolve.alias docmost-client -> packages/docmost-client/src
  (fixes the dist-vs-src coverage artifact: canonicalize 0% -> real)
- R-Cfg-1: export parseArgs + tests
- canonicalize: align family / comment.resolved kept / link non-default +
  fixpoint & docsCanonicallyEqual reflexive/symmetric properties (0 -> 100%)
- markdown-converter golden matrix: columns/embed/audio/pdf, drawio data-align
  rule, inline-mark matrix, textAlign, escaping idempotence, table sanitization
  (61 -> 79%)
- schema parse-closures via generateJSON (TextStyle/comment/mention/Highlight/Column)
- node-ops (immutability, table edge cases, makeFreshId property), transforms
  (setCalloutRange/insertMarkerAfter/commentsToFootnotes + renumber property)
- stabilize normalize-on-write fixpoint (0 -> 100%); diff coarse-fallback;
  client-utils; firstDivergence; corpus fixtures details/columns/mention
- 593 -> 695 green; build clean; corpus STABLE

Deferred (Phase 3-4, refactor-gated): pull/collab/client-REST/git-merge integration.
2026-06-17 01:01:26 +03:00
vvzvlad
f68168e3c1 refactor(sync): unify git exec layer; fix non-ASCII paths + diagnostics (review)
Address a code review of the git-hardening changes.

- single runRaw primitive: every git invocation funnels through it; run() is a
  thin throw+trim wrapper; the two direct execFileAsync bypasses (commitRaw,
  assertGitAvailable) removed; one unified error format
- `-c core.quotepath=false` is now the argv baseline for ALL commands (was only
  listTrackedFiles) — removes the latent quoting asymmetry on ls-files -u /
  diff --name-only; persisted LOCAL config (autocrlf/safecrlf/gpgsign/
  attributesFile) kept as-is in ensureRepo
- preserve spawn-error message (ENOENT): use `||` not `??` (promisified execFile
  sets stderr to "" on spawn failure)
- contextual error when pinning vault git config; module/vaultGitEnv docs corrected
- README: require a system git binary on PATH for local runs
- tests: --no-verify honored (failing pre-commit hook), vaultGitEnv pins,
  core.attributesFile=/dev/null neutralization (593 green)
2026-06-17 00:32:37 +03:00
vvzvlad
ec0a3d47c7 fix(sync): robust git coupling — non-ASCII paths, config neutralization, runtime git
Address git-integration fragility (output is not parsed for control flow; we rely
on exit codes + plumbing — but porcelain BEHAVIOR is config-sensitive, and the
runtime image lacked git).

- listTrackedFiles: `git -c core.quotepath=false ls-files -z` + NUL split — fixes
  Cyrillic/UTF-8 vault filenames being returned octal-escaped/quoted
- Dockerfile: install git (node:22-slim ships none; the daemon shells out at runtime)
- VaultGit env: LC_ALL=C/LANG=C, GIT_PAGER=cat, GIT_TERMINAL_PROMPT=0; keep
  stripping GIT_DIR/GIT_WORK_TREE (cwd-isolation, §12)
- ensureRepo local config: core.autocrlf=false + core.safecrlf=false (protect §11
  byte-stability from a global autocrlf=true), commit.gpgsign=false, and
  core.attributesFile=/dev/null (neutralize a global clean/smudge filter that
  would rewrite the stored blob); commit uses --no-verify (skip injected hooks)
- assertGitAvailable() preflight: clear error if the git binary is missing
- tests: Cyrillic listTrackedFiles, LF byte-preservation of the stored blob,
  local-config neutralization incl. attributesFile (590+ green)
2026-06-17 00:15:17 +03:00
vvzvlad
531b320776 feat(sync): add git vault layer (§5) and the Docmost->vault pull cycle (§6)
Turn the read-only mirror into a git-backed pull cycle. Read-only toward Docmost.

- git.ts (VaultGit): system-git wrapper, all ops cwd=vaultPath (vault is its own
  repo under data/vault, never the source repo); ensureRepo/branches main+docmost,
  commit with provenance (author/committer identity + Docmost-Sync-Source trailer,
  §7.3), merge with conflict surfacing (no auto-resolve, §9), isMergeInProgress;
  GIT_DIR/GIT_WORK_TREE stripped from env (§12 cwd isolation)
- stabilize.ts: normalize-on-write (one export->import->export fixpoint pass, §11)
- reconcile.ts: pure planReconciliation (add/update/move/delete by pageId) +
  decideAbsenceDeletions gate
- pull.ts: write/commit on docmost -> merge into main; listSpaceTree completeness
  signal suppresses absence-deletions on a partial fetch (§8); mass-delete guard;
  merge-in-progress guard makes re-runs converge (§12); move old-path removal only
  on successful write
- docmost-client: listSpaceTree({pages, complete}) without touching the 1:1-copied
  enumerateSpacePages
- tests: reconcile planner + decideAbsenceDeletions, VaultGit incl. real temp-repo
  merge conflict, listSpaceTree completeness (586 green)

Push to a git remote and the FS->Docmost direction are deferred to the next increment.
2026-06-16 23:57:50 +03:00
vvzvlad
4b34f4d30a feat(sync): resolve §11 idempotency via canonical comparison + corpus harness
Close Задача №0 (SPEC §11) with the spec-sanctioned option (b): compare a
canonicalized ProseMirror form instead of raw bytes.

- canonicalize.ts: canonicalizeContent/docsCanonicallyEqual — strip node attrs.id,
  drop null/undefined attrs, and drop attrs equal to their type's known non-null
  schema default (KNOWN_DEFAULTS: link target/rel, comment.resolved, orderedList.start,
  diagram/media align) so "absent" ≡ "default"; comment anchors + meaningful attrs kept
- roundtrip.ts: assert markdown byte-stability AND canonical stability; add --corpus
  mode and mutually-exclusive-flag warning
- synthetic corpus (headings, marks, lists, table, callout, code w/ trailing \n,
  diagrams, textStyle/mention) + canonicalize/corpus tests (558 green)
- known converter asymmetries (block image after paragraph; embed width/height
  coercion) converge to a fixpoint after one export->import pass -> handled by
  normalize-on-write at vault-write time; isolated under it.fails
- SPEC §11: record the resolution and normalize-on-write strategy
2026-06-16 23:23:32 +03:00
vvzvlad
90d8f86fda test: add full test suite for docmost-client and remaining modules
Raise coverage from 2.6% to 68% statements by adding 19 test files (~480
tests) covering every module in test-strategy-report.md. No production code
changed — tests reach private logic via (client as any), mock HTTP with
axios-mock-adapter on the real axios instance (interceptors intact), and mock
the Hocuspocus provider with vi.mock + real yjs + fake timers.

Coverage: auth-utils/filters/page-lock/json-edit 100%, diff 99%, node-ops 96%,
transforms 95%, collaboration 86%, layout 91%, client.ts 41% (transport).

- node-ops/transforms/json-edit/page-lock/filters: pure tree/text ops,
  immutability + clone guarantees, throw-vs-noop contracts
- markdown-converter + markdown-document envelope + fast-check round-trip
  property test
- diff, docmost-schema (sanitizeCssColor/clampCalloutType security guards)
- collaboration: pure (buildCollabWsUrl/buildYDoc) + write-path (mutatePageContent
  read-transform-write, false-success suppression)
- client.ts: isSafeUrl/validateDoc* XSS guards, vm-sandbox, REST pagination,
  401 re-auth interceptor, login dedup, uploadImage/createPage multipart guards
- collectRecentSince edge cases; loadSettingsOrExit invalid-value branch
- env-gated E2E skeleton (DOCMOST_E2E)

Two genuine markdown round-trip non-idempotency bugs are documented as it.fails
(code-mark excludes other marks; block-image injects a blank line). Latent:
isSafeUrl allows file:// on link context.

Adds dev-deps: fast-check, @vitest/coverage-v8, axios-mock-adapter; adds the
"coverage" npm script.
2026-06-16 22:50:04 +03:00
vvzvlad
cc13c94f53 test(docmost-client): add unit tests for pure lib modules
Add 230 Vitest unit tests covering the dependency-light, pure modules of
packages/docmost-client/src/lib, imported directly from source:

- node-ops: tree addressing, immutability/clone guarantees, table ops,
  throw-vs-noop contracts (87)
- transforms: commentsToFootnotes reading-order renumbering, insertMarkerAfter
  mark-preserving split, setCalloutRange regex statefulness (43)
- json-edit: applyTextEdits literal $&/$1, error distinction, immutability (17)
- page-lock: async per-page mutex ordering and error isolation (6)
- filters: filterPage/filterComment truthiness traps, filterSearchResult (19)
- markdown-converter: per-node golden matrix + edge cases (41)
- markdown-document envelope: round-trip, CRLF, malformed-JSON throws (17)

No source files changed. The pre-existing test/markdown-document.test.ts is
left intact; new envelope coverage lives in markdown-document-envelope.test.ts.
Full suite: 16 files / 279 tests green.
2026-06-16 22:10:06 +03:00
vvzvlad
c6edd73324 refactor(pull): extract tested vault-layout module; harden pull; close review findings
Address the Increment-1 code review (3 warnings + suggestions).

- layout: new pure src/layout.ts (buildVaultLayout) — page-tree -> vault paths,
  sibling + full-path collision disambiguation (sanitized ~slugId suffix), parent
  cycle guard; pull.ts is now a thin I/O loop
- layout: resolve orphan/root collisions at the NAME stage so an orphan ancestor
  can't desync its children's folder segments (fixes review Major); covered by test
- pull: per-page try/catch (one bad page no longer aborts the mirror), bounded
  concurrency (6), progress logging, process.exitCode=1 on partial mirror
- security: filename disambiguation suffix now passes through sanitizeTitle
- docs: AGENTS.md -> Increment 1 status/structure/run targets; pull.ts meta-block
  comment; collectRecentSince JSDoc (lexicographic UTC-ISO precondition)
- tests: layout (9), markdown-document round-trip (no comments block, SPEC §3),
  firstDivergence; export firstDivergence. 49 tests green.
2026-06-16 21:09:40 +03:00
vvzvlad
447d2508ae feat(sync): scaffold monorepo, extract docmost-client, add Phase-0 harness + read-only pull
Lock the access-layer decision (REST only) and start implementation per SPEC.

- monorepo (npm workspaces): packages/docmost-client = DocmostClient + lib/*
  copied 1:1 from docmost-mcp/src (backport target), plus bannered sync methods
  (listTrash, restorePage, listAllSpacePages, exportPageBody, listRecentSince /
  collectRecentSince cursor scan)
- engine stays the root app per AGENTS.md (src/, test/, build/, data/, settings.ts);
  add roundtrip.ts (SPEC §11 idempotency harness), pull.ts (SPEC §6 read-only
  Docmost->FS mirror), sanitize.ts (SPEC §12 filenames, path-traversal-safe)
- Dockerfile builds the workspace lib before the app; vitest gates CI
- exportPageBody never touches /comments (SPEC §3); serializeDocmostMarkdownBody
  emits meta + body only
- SPEC: resolve access-layer (REST), reflect root-engine layout + REST pagination
- tests: sanitize (incl. dot-traversal), collectRecentSince (cutoff/dedup/cap),
  stripBlockIds, markdown round-trip byte-stability

Note: raw ProseMirror round-trip is byte-stable in Markdown but not yet attribute-
idempotent (SPEC §11 Задача №0, before Phase 2).
2026-06-16 20:20:20 +03:00
vvzvlad
ef223e13ff chore(scaffold): bootstrap docmost-sync Node/TS project skeleton
Set up the project structure per the new-project guide, adapted from the
Python skeleton to the Node/TS stack fixed in SPEC.md (reuses docmost-mcp).
Scaffold only — the sync engine is not implemented yet.

- src/settings.ts: single config layer on zod, schema keyed by real ENV
  names; credentials and own-service address have no default (fail fast).
- src/config-errors.ts: loadSettingsOrExit — clear startup message naming
  the missing/invalid env var instead of a raw stack trace; exit(1).
- src/index.ts: thin entry point that validates config and logs (stub).
- test/: vitest unit tests for settings parsing and config errors (10 tests).
- Makefile (install/env/build/test/run/dev/clean), strict tsconfig, vitest.
- Dockerfile (single-stage, no EXPOSE, prunes dev deps), docker-compose
  (daemon, volume on /app/data, watchtower), ghcr CI with build needs test.
- .env.example, .gitignore/.dockerignore, AGENTS.md, README.md.
- Pinned deps (dotenv, zod) + committed package-lock.json.
2026-06-16 18:54:29 +03:00