QA follow-up on 5d45f5a8: that commit taught the converter to export heading
textAlign (<hN style>) but left the converter-gate heading test still asserting
the OLD dropped behavior (expects a bare '## text'), so jest was RED — the G1
green-suite gate was not actually met. Two gaps closed:
1. Flip the heading KNOWN-DIVERGENCE gate test to assert the round trip now
PRESERVES alignment (exported as <h2 style="text-align:center"> and recovered
on import), matching the shipped converter behavior. Suite is green again.
2. blockToHtml (the nested-container path: heading/paragraph inside a
column/table/callout) still emitted bare <hN>/<p>, dropping textAlign for
nested blocks. Carry the style there too, symmetric with the processNode path.
Also add #7 (table inside a column) and #8 (multi-block table cell) to the
lossless round-trip CORPUS so both survive export->import through the real
editor-ext schema (columns widthMode pre-authored at its normalize fixpoint).
Verified: server jest 193 suites / 2142 tests green, git-sync vitest 704 green,
no type errors.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
F5 (HIGH data-loss): guard #2 (GS-EDIT-REVERT) called a local key-sorting equality that
never matched a real page (block ids + materialized defaults differ), so the guard was
dead and a web edit on a git-sync space was silently reverted within one poll cycle. Use
the package's authoritative docsCanonicallyEqual (strips block id + normalizes
KNOWN_DEFAULTS), wired through the git-sync loader like sanitizeTitle; delete the dead
local canonicalize/canonicalJsonEqual.
S2 (security): importPageMarkdown targeted a page by the vault-file id without a spaceId
check (deletePage had one) — a space-A vault file carrying space-B's page id could
resurrect/overwrite/clear B's page. Mirror deletePage's guard: skip when the loaded page
lives in a different space than ctx.spaceId.
G1 (jest green): add sanitizeTitle + docsCanonicallyEqual to the loadGitSync mock; update
the converter-gate + package golden expectations to the genuinely-fixed output (paragraph
textAlign now round-trips, multi-block table cells emit HTML tables); fix the orchestrator
spec's stale mock so the per-space enabled gate (added later) is satisfied.
A1: the converter dropped heading textAlign on export (bare '## text'); emit a styled
<hN> when aligned, symmetric to paragraphs — round-trips losslessly (level + align), no
churn for unaligned headings.
F7 (docs): reword the false 'single choke point' title-strip comment; correct push.ts
docstrings that still described the removed standalone-CLI/daemon model.
Adds regression tests: the F5 acceptance test (canonically-equal content with real uuids
=> writePageBody NOT called), the S2 cross-space import guard, and the A1 heading
round-trip.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The existing name-level contract (packages/git-sync schema-editor-ext-contract)
only compares node/mark TYPE NAMES, so a new attribute added to an existing node
upstream slips through and is silently dropped on every git-sync round trip -- a
repeatedly-hit data-loss class (image caption #221, paragraph align #10).
This closes the attribute gap by comparing the RESOLVED ProseMirror Schema
objects (getSchema has already merged all addGlobalAttributes spreads into
concrete per-node attrs) of the server's canonical tiptapExtensions vs the
git-sync mirror, asserting equal attribute-key sets per shared node/mark modulo
a committed, self-checking allowlist of the 6 understood divergences. A forgotten
attribute now fails CI loudly instead of losing data. Comparing resolved schemas
(not raw extension configs) is what makes this stable rather than the fragile
config-shape compare the name-level test deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Clearing a page's body in git advanced the vault ref past the empty commit, but
the persistence store-side empty-guard rejected the empty write (reloading the
non-empty DB content) — so Docmost kept the old body while the vault held the
empty one, a permanent silent vault<->Docmost divergence that never re-detects.
A git-sync write is authoritative and its content IS the vault file, so an empty
incoming doc there is a DELIBERATE clear (no transient-glitch empties for a
file-sourced write). Allow it (lastUpdatedSource==='git-sync'), mirroring the
#251 intentional-clear allowance for the user-signalled source.
Verified on stand: a git-side body clear takes the page body 28 -> 0 (page not
trashed); previously it stayed 28 (diverged).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A git-side revert of a delete commit re-adds the page's .md; the push classifier
saw an add carrying a known pageId and emitted an UPDATE, writing the body to the
still-trashed page. It stayed in Trash and the next pull re-deleted the file, so
the revert was silently nullified (permanent vault<->Docmost divergence). In
importPageMarkdown, if the target page is soft-deleted, restorePage() it first
(restorePage was already in the client seam but never called), then apply the
body — so a git revert actually brings the page back.
Verified on stand: git rm -> page trashed; git revert -> deleted_at cleared
(page restored).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dead code / doc-vs-code cleanup from review #4404:
- Drop the unused Settings.docmostApiUrl/Email/Password fields (the native
in-process datasource never reads them; the engine only used docmostSpaceId).
Removed from the type, buildSettings, and the 4 engine test suites that
fabricated them.
- Warn ONCE at startup when GIT_SYNC_REMOTE_TEMPLATE is set — remote push is
deferred (SPEC §7) so the value is currently inert; the operator now gets a
log line instead of a silent no-op.
- Correct stale docstrings that claimed live-destructive code was 'FAKES only /
NEXT increment / no live wiring' (applyPushActions runs LIVE via
runCycle->orchestrator); that importPageMarkdown receives a 'self-contained
file (meta+body)' (it receives the stripped body); and that the Yjs body merge
is '2-way' (3-way runs end-to-end when the base is present).
No behavior change (except the new warning); build + git-sync smoke verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Review #4: forward HTTP_CONTENT_ENCODING to git http-backend so it inflates
gzip'd RPC bodies — a non-trivial `git pull` no longer fails with
`fatal: expected 'packfile'`. (git-http-backend + git-http.service)
- Review #5: the read-advertisement branch runs under the space lock AFTER
reply.hijack(); a reject there (e.g. Redis down) previously left the socket
open forever, hanging every clone/fetch. Mirror the push branch: catch, 500 if
unwritten, always end the socket. (git-http.service)
- GS-EXPORT-500 (QA): a page with an inline comment mark returned HTTP 500 on
Export/copy-as-markdown. The Comment mark's renderHTML took the imperative
document.createElement branch server-side (the DOM shim used by generateHTML
defines window/document), returning a live node with no content hole that
crashed prosemirror-model's DOMSerializer under happy-dom. Gate the imperative
branch on a real browser (navigator.userAgent contains 'Mozilla'); the server
now uses the static DOMOutputSpec form. Verified: export 200 (was 500).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses reviewer comment #4404 (critical + blocking):
- Critical #2: renamePage skips the echo where the incoming title equals
sanitizeTitle(current title) — a Docmost title with FS-hostile chars (: / " |,
newlines, double-space, >120) was pulled to a sanitized stem then written back,
permanently corrupting the real title. (datasource)
- Blocking #3: runOnce enforces per-space settings.gitSync.enabled (the event
path bypassed opt-in; any edited space would git-init + export). (orchestrator)
- Blocking #6: movePage no-ops the position-less same-parent echo that clobbered
the user's chosen sibling order. (datasource)
- Blocking #9: hasConflictMarkers is fence-aware — '<<<<<<< HEAD' inside a code
block (git-tutorial page) no longer trips the all-or-nothing gate that froze
the whole space's refs. (push.ts)
- Blocking #11: three-way tryMergeRegion short-circuits when live==target (diff3
agreement) instead of logging a false 'same-block conflict resolved to git' —
the echo noise that masked real data-loss signals. (three-way-merge)
- Blocking #12/#13: e2e-advanced — drop the delete-cap block (no such feature;
failed with a scary '(data loss!)'); non-member assert now expects 404 (existence
not leaked), not 403.
Verified on stand: sanitized-title rename preserves DB title (vault file
sanitized); non-enabled space creates no vault; fenced conflict markers ingest
without jamming; build clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On a git-sync space, a page's web edit was silently reverted within ~1 poll,
and idle spaces showed dozens of 'update' actions per cycle with no real change.
Root cause: the vault->Docmost body ingest (importPageMarkdown) is re-run every
poll for pages the upstream change-detection mis-flags as changed (the
markdown<->ProseMirror round-trip is not byte-stable: JSON key order / default
attrs differ though the content is identical). Each call re-imports the SAME body
into the live collab doc -- a no-op at idle, but it CLOBBERS a concurrent human
edit still in the debounced (not-yet-flushed) Yjs doc.
Fix: skip the ingest when it is genuinely a no-op --
1) baseMarkdown byte-identical to the current file (vault unchanged), or
2) the parsed incoming body is canonically-JSON-equal (key-order-insensitive)
to the page's current Docmost content.
A real git-side change is neither, so legitimate git->Docmost ingests still apply.
Verified: idle churn 38 update/cycle -> 0; web edit on an affected page 0/3 -> 3/3
persisted; genuine git-side edit still ingests. Found by autonomous QA
(web-test-orchestrator) + independent verifier.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
F1: git-sync.orchestrator.spec bind assertion now includes spaceId ('space-1'),
matching driveCycle's dataSource.bind({workspaceId,userId,spaceId}).
F2: add 4 non-vacuous tests for the cross-space move data-loss guard in deletePage
(CTX_SPACE with spaceId): move-out skips removePage (returns skipped:'moved-to-other-space');
same-space / not-found / already-deleted all still call removePage.
F3: add 2 tests for the ~<slugId> title-strip guard in renamePage (own slugId stripped;
a foreign ~<slugId> tail left intact).
F4: reword the gitmost-datasource 'single choke point' comment — the strip covers the
rename/update path, not every git-sync title write (createPage's filename-derived title
does not funnel through here).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A page moved to another space with git-sync enabled was sent to Trash and
vanished from BOTH vaults. The source space's push phase sees the moved-away
page's file gone from its vault and calls deletePage -> soft-delete, even though
the page still lives in the destination space.
Thread the reconciling spaceId into the bind context and, in deletePage, skip the
soft-delete when the page's CURRENT space differs from the space being reconciled
(a move-out): only the vault file is dropped, the page is preserved. Genuine
in-space deletions are unaffected (space matches).
Found by autonomous QA (web-test-orchestrator). Control: with git-sync OFF the
move keeps deleted_at NULL; with it ON the page was trashed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two sibling pages that share a title collide on one vault filename, so the
layout appends a cosmetic ` ~<slugId>` suffix (engine disambiguate()). That
suffix is a local filesystem artifact and must never become the page's real
Docmost title, but on ingest the filename-derived title carried it back into
the DB on some paths (observed: intermittent same-title collision left a page
permanently titled "Title ~<slugId>"). Strip it in renamePage() — the single
choke point every git-sync title write funnels through — but only when the
trailing token equals THIS page's own slugId, so a genuine user title that
legitimately ends in ` ~token` is never corrupted (slugId is a random nanoid).
Repro: create two pages with the same title; ~1 in 4 the second page's title
is permanently polluted. After fix: 0/6.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
F1: escape the collaborative page title before interpolating into
<page_changed page="..."> (and the pre-existing openedPage attr) — strip
<>" and collapse whitespace, so a crafted title can't break out of the
attribute into the system prompt (cross-user injection).
F2: neutralize <page_changed>/</page_changed> occurrences inside the diff body
so a crafted line can't close the block early.
F3: remove the dead content_hash column (written every turn, never read) —
migration, repo, service hashing + crypto import, db.d.ts, spec asserts.
F4: test the best-effort catch branches (detectPageChange / snapshotOpenPage
swallow errors and don't break the turn).
F5: soften the overstated 'diff cannot smuggle instructions' comment to
defense-in-depth framing referencing the F1/F2 mitigations + safety sandwich.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The agent rebuilds context from DB each turn and didn't know the user manually
edited the open page since its last response, so it could overwrite those edits.
Add a per-turn ephemeral <page_changed> note in the system prompt (twin of
INTERRUPT_NOTE, self-clearing) carrying a unified Markdown diff of what changed
since the END of the agent's previous turn.
- New ai_chat_page_snapshots table (migration + hand-declared db.d.ts/entity
types) storing the page Markdown per (chat,page) at each turn's end.
- Pure computePageChange util (whitespace-normalized unified diff via the
existing jsdiff dep, 6KB cap + getPage hint).
- Turn start: if the open page's updatedAt moved past the snapshot, diff current
vs snapshot; non-empty -> PAGE_CHANGED_NOTE in the safety sandwich.
- Turn end: upsert the snapshot on EVERY terminal path (onFinish/onError/onAbort,
once) so the agent's own edits are excluded by construction even on aborted
turns.
All best-effort (never breaks/latency-regresses a turn); fast path when updatedAt
is unchanged. Server-only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Real root cause of the silent MCP edit loss: the web editor always opens the
collaboration document by the page UUID (`page.${page.id}`), but the MCP
opened it by the agent-supplied id — usually a slugId — so `page.${pageId}`
became `page.<slugId>`. For one DB page that is TWO independent Yjs documents;
both persist to the same `pages` row (findById/updatePage resolve id or
slugId), so the human tab's debounced store overwrites the agent edit
(last-store-wins) — gone after reload, never shown live. The slugId doc also
made the server's transclusion sync + embedding reindex throw Postgres 22P02.
Fix:
- MCP (primary): resolvePageId(pageId) returns the canonical UUID — a UUID
short-circuits with no network call, a slugId resolves once via getPageRaw
and is cached both ways. Every collab-write path (mutatePageContent /
updatePageContentRealtime / replacePageContent and the mutate/replace/
unlocked seams) now opens by the resolved UUID, so the MCP and the editor
share ONE Yjs doc. replaceImage's whole-operation page lock also keys on the
UUID so it serializes against the other (now-UUID-keyed) writes.
- Server (defense + kills the 22P02 noise): onStoreDocument passes the resolved
page.id — not the raw doc-name id — to syncTransclusion, the embedding queue,
the mention-notification job, addContributors, and the in-tx history read.
Content store and the empty-guard are untouched.
Tests: a new MCP test stands up a real Hocuspocus server and asserts a slugId
input opens `page.<uuid>` (never `page.<slugId>`), with UUID short-circuit and
single-resolve caching; the server spec asserts the side-effects receive the
UUID for a `page.<slugId>` doc. closes#260
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an inline spoiler (Telegram/Discord-style hidden text): a TipTap mark
`spoiler` rendered as <span data-spoiler="true" class="spoiler">, blurred via
CSS and revealed on click (UI-only is-revealed class, never persisted).
- packages/editor-ext: the Spoiler mark (inclusive:false, set/toggle/unset
commands, ||text|| input rule), exported; a lossless turndown rule emitting
raw inline HTML; round-trip test.
- apps/client: SpoilerView mark-view (ReactMarkViewRenderer, Link pattern),
registration in extensions, bubble-menu toggle button (editable only), CSS
(blur + @media print reveal), en/ru i18n.
- apps/server: register Spoiler in collaboration.util tiptapExtensions so the
mark survives HTML<->JSON export/index/import/Yjs; a test proving the public
share keeps the spoiler (it isn't stripped with comments).
No keyboard shortcut: the proposed Mod-Shift-s collides with Strike (and
Mod-Shift-h with Highlight); the ||text|| input rule + the bubble-menu button
cover ergonomics.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
F17: the header's content-clause literal omitted the [[:space:]]* tolerance;
copy page.repo.ts's exact '"type"[[:space:]]*:[[:space:]]*"text"' (jsonb::text
renders a space after the colon, which is why the tolerance exists).
F16: remove expect(ttl).toBeGreaterThan(0) — the twin of the F15 removal;
expect(ttl).toBe(120) strictly subsumes it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
F14: the lockstep int-spec header still described the pre-F6 two-clause set with
'iff' — add the content-JSON text-node clause so it matches embeddablePredicate.
F15: remove the redundant expect(ttl).toBeLessThanOrEqual(120) that followed
expect(ttl).toBe(120).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
F11: PRE_SEED_TTL_SECONDS 45->120 (= client REINDEX_POLL_CAP_MS). At concurrency
1 a queued reindex can wait past the old 45s; if the pre-seed expired while
pending, getMasked fell back to the COUNT and reported done, so the client
stopped polling and missed the climb. Tie the pre-seed TTL to the client cap.
F12: extend the lockstep integration spec — insertPage takes content; a
text_content=null + text-node-content page is IN and a math-only page is OUT,
pinning the structural "type":"text" clause (and the jsonb space-after-colon).
F13: list all three embeddable clauses in the reindex JSDoc/inline comments.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
F6: extend embeddablePredicate to pages with body content but null text_content,
keyed on the text-node marker "type":"text" (not a bare "text": key, which
also matched math nodes' attrs.text and would leave math-only pages stuck
below 100%). Numerator and denominator share the predicate; tests assert the
compiled WHERE is byte-identical and a math-only doc is excluded.
F7: correct the start() JSDoc (both totals are the real page count).
F8: nextReindexPollInterval reuses isReindexComplete.
F9: getMasked reads progress first and skips the two COUNTs while a reindex is active.
F10: pre-seed the progress entry with a short 45s TTL so a deduped enqueue's
phantom "0 of N" expires quickly instead of sticking for the 1h TTL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
guardedFetch blocks loopback/private/link-local/metadata IPs and never calls
fetch; decryptHeaders fails open (returns undefined, warns once, no blob leak).
yjs.util setYjsMark/removeYjsMarkByAttribute/updateYjsMarkAttribute on real
Y.Docs. SHARED_TOOL_SPECS<->in-app parity (name/desc/input-schema; a dropped or
renamed wiring fails). Replace the tautological storage.service spec with
driver-delegation checks across every public method.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The pub/sub error handlers were `(err) => () => {}` — a noop returning an
inner arrow that never runs, so socket.io redis client errors were silently
swallowed. Log them via Nest Logger. Adjacent pre-existing bug surfaced in
review of #255.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The WsRedisIoAdapter creates two ioredis clients (pubClient/subClient) for
@socket.io/redis-adapter but never closed them, leaking their TCP handles on
application shutdown (#255). The redis-adapter does not own these clients'
lifecycle, and the adapter is instantiated from main.ts (not a DI provider),
so no Nest lifecycle hook applied to it.
Keep references to both clients and override dispose(), which Nest's
SocketModule.close() invokes exactly once during shutdown after all socket.io
servers are closed. Use disconnect(false) to mirror the sibling pub/sub pair
in collaboration/extensions/redis-sync (onDestroy): immediate close, no QUIT
round-trip, no auto-reconnect. Refs are nulled to guard against double-close.
Runtime behavior is unchanged; only the shutdown path is added.
Verified with a script that boots connectToRedis() against a real Redis:
2 sockets to :6379 open after connect, 0 remain after dispose().
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The store-side empty-guard consumed the per-document intentional-clear flag
INSIDE the bounded retry loop. consumeIntentionalClear always deletes the
in-memory Map entry, but a tx rollback cannot un-delete it: attempt 1
consumed the flag then updatePage threw a transient error and rolled back;
attempt 2 re-read the page non-empty, saw the flag gone, and the empty-guard
silently BLOCKED the write — dropping the user's deliberate clear and
defeating the retry guarantee for clears.
Hoist the decision out of the loop (like consumeContributors /
consumeAgentTouched): consume once into `allowIntentionalClear` before the
`for`, and only read that boolean on the empty-over-non-empty branch. The
single hoisted consume still drops a pending flag for a non-empty store
(the "cleared then retyped" case), since every store consumes regardless of
incoming emptiness.
Add a regression test: arm via the real onStateless transport, updatePage
throws once then succeeds, assert it is called twice and the retry writes the
empty doc (the clear survives). It fails on the old consume-in-loop ordering
(updatePage called once) and passes after the hoist.
Document the known fail-safe limitation near the TTL constant: if document
ownership transfers / a node crashes between the stateless signal and the
debounced store, the in-memory flag is lost and the clear is silently not
applied (the doc reloads non-empty) — fail-safe, content is never destroyed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The full-AppModule e2e (apps/server/test/app.e2e-spec.ts) passed but jest
never exited, burning CI to its timeout. Diagnosis (process._getActiveHandles
after app.close()) showed exactly two ioredis sockets to :6379 still open after
shutdown; everything else (BullMQ queues/workers, @nestjs/schedule intervals,
nestjs-ioredis, nestjs-kysely pg pool, @nestjs/cache-manager Keyv store,
hocuspocus pub/sub) already closes on app.close().
The two leaks were owned-but-never-closed clients:
1. ThrottleModule passed a pre-built `new Redis(...)` instance to
ThrottlerStorageRedisService. With an instance, the lib sets
disconnectRequired=false, so its onModuleDestroy never disconnects.
Pass ioredis options instead so the service owns + disconnects the client.
2. CollaborationGateway created a source `new RedisClient(...)` that
RedisSyncExtension only duplicates into pub/sub; the extension's onDestroy
disconnects those duplicates but not the source. Keep a reference and
disconnect it after the hocuspocus onDestroy hook in destroy().
Both are real lifecycle fixes (production shutdown is now clean too), so no
--forceExit is needed. Verified against real Postgres+Redis:
- test:e2e (no forceExit, --runInBand) exits 0 in ~18s (was: hung forever)
- --detectOpenHandles exits 0 with no open-handle report
- active handles after app.close(): none
CI timeout-minutes safety nets left untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The #248 store-side empty-guard (onStoreDocument) unconditionally refuses to
overwrite non-empty persisted content with an empty document, because a
momentarily-empty live Y.Doc is indistinguishable from a real clear at the
store layer. That correctly blocks glitches/bad-merges, but also blocks a user
who genuinely wants to empty a page. This re-introduces a WORKING, narrow,
non-spoofable exception (the dead context.intentionalClear hatch #248 removed
never had a real channel).
Definition of an intentional clear (client, IntentionalClear editor extension):
a LOCAL user transaction (docChanged, NOT a remote y-sync change — filtered via
isChangeOrigin) that reduces a non-empty doc to the empty single-paragraph
shape. This is exactly the select-all + Delete/Backspace keystroke path.
Transport (option b — hocuspocus stateless message): on that transition the
client sends a `{type:'intentional-clear'}` stateless message. The server
(PersistenceExtension.onStateless) records a short-lived (TTL 60s > 45s
maxDebounce), single-use "pending clear" flag keyed by the connection's
document. The next debounced onStoreDocument consumes it on the empty-guard
branch to let that one empty write through.
Why this is the right channel and non-spoofable:
- Yjs transaction origin/metadata does not survive to the server store; awareness
is per-connection and racy. A stateless message ties the signal to a specific
clear, survives the debounce, and rides the authenticated connection.
- The document is taken from the connection, never the payload, so a client
cannot target another page.
- The flag is read ONLY on the empty-over-non-empty branch, so the worst a forged
signal can do is clear a page the connection may already edit; it can never
force or alter a non-empty write. Read-only connections cannot arm it. Every
non-empty store drops a pending flag, so "cleared then retyped" leaves nothing
usable; the flag is single-use and TTL-bounded.
NOTE: #248 is not yet on develop, so the empty-guard block is included here as
the foundation this exception extends. If #248 lands first this rebases cleanly
(the guard logic is identical; the #251-unique additions are the exception,
onStateless, the pending-flag state, and the client extension).
Tests:
- Server (real transport path, not a hand-poke): onStateless sets the flag with
the exact client payload, then the debounced onStoreDocument persists the empty
doc; plus single-use consumption, read-only rejection, non-empty-store drops
the flag, and the unchanged #248 guard tests (empty-over-non-empty blocked,
empty-over-empty allowed).
- Client: a real Editor + the actual selectAll+deleteSelection command emits the
signal; typing / non-emptying edits / already-empty docs do not.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Apply review suggestions to the real-files block in
ai-agent-roles-catalog.provider.spec.ts (test-only):
1. Fix inaccurate comment: there are 5 content YAML files (index +
four per-bundle/lang files), not 6.
2. Improve isolation: read/parse the real index lazily inside tests
(via loadRealIndex) instead of in the describe body, so a broken
real file fails only these catalog tests, not collection of the
whole spec (incl. the unrelated mocked-remote provider tests).
3. Add the symmetric slug check: each language file's slug set must
equal the declared slug set (no undeclared/extra roles), matching
scripts/check.mjs's exact two-way correspondence.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The blob-sandbox feature (#243/#250) made AiChatToolsService.forUser()
eagerly call this.sandboxStore.asSink() while wiring the stash tool, but
the spec still passed an empty {} as the sandboxStore constructor arg.
That object has no asSink method, so all 19 tests in the suite failed in
CI with 'TypeError: this.sandboxStore.asSink is not a function'.
Replace the stale {} mock at all 4 constructor sites with a no-op sink
exposing asSink() -> { put, has, evict } (jest.fn()). These tests never
execute the stash tool, so a no-op sink is sufficient for forUser() to
wire successfully. Test-only change; production code is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Provider tests only exercised synthetic stringifyYaml fixtures, so a
hand-conversion error in one of the 6 real catalog files (index.yaml,
bundles/{editorial,research}/{en,ru}.yaml) — a stray quote/colon in a
description, a broken emoji/arrow, a block-scalar indent slip that
silently changes or drops instructions — was caught by no automated
test. scripts/check.mjs is the only other guard and is wired into no
CI/turbo/husky step.
Add a real-files test block that reads each shipped file off disk,
parses it with the SAME options the provider uses
(strict: true, maxAliasCount: 100), and validates it through the
provider's own exported type guards (isCatalogIndex / isCatalogBundleFile
/ isCatalogRole). It is driven from the real index so new bundles/langs
are auto-covered, asserts the editorial bundle still ships fact-checker,
and requires every declared role to be present with non-empty
instructions/name in each language file.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
F1: clear the "Reindex now" spinner once the poll cap fires. Gate the
reindexing part of the button's loading state on the active poll window
(reindexDeadline !== null) so a run that outlives the 120s cap no longer
leaves the button stuck-disabled with a stale `reindexing: true`; the
admin can restart.
F2: rewrite reindexWorkspace JSDoc to describe the EMBEDDABLE page set
(text OR existing embeddings), matching getEmbeddablePageIds /
countEmbeddablePages instead of the old "every non-deleted page".
F3: extract the shared embeddable-content predicate into a private
PageRepo.embeddablePredicate helper, called by both countEmbeddablePages
and getEmbeddablePageIds, removing the verbatim duplication. Behavior is
identical (lockstep int-spec stays green).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three more git-sync QA defects from the 2nd live pass on PR #119, plus a
callout-fidelity nit:
1. SPURIOUS conflict leaked raw markers into canonical main (root cause). On an
ordinary round-trip the only difference between the docmost mirror (normalize-
on-write) and a user's raw push is trailing/empty-line normalization, which made
git's line-based docmost->main merge CONFLICT, and the wedge fix then committed
the file WITH literal <<<<<<< / ======= / >>>>>>> markers onto main (git and the
DB silently diverged for cycles). Fix: on a conflict, normalize trailing/empty
lines on BOTH sides (showStage :2:/:3:) before comparing — a trailing-only diff
is recognized as spurious and resolved to the clean normalized form. A GENUINE
same-block conflict is auto-resolved to OURS (git wins, mirroring the live-doc
3-way rule); the docmost side stays on the `docmost` branch + page history. Raw
markers NEVER reach main again.
2. Concurrent UI<->git edit silently lost the UI side. The git->Docmost 3-way merge
ran against a live Y.Doc that hadn't yet received the user's debounced in-flight
edit, so git clean-applied (no conflict detected) and the edit vanished even on a
different block. Fix: flush the pending debounced store before the merge so the
in-flight edit is drained into the live doc first — a different-block edit is
merged, a same-block one is detected and pinned to history (recoverable).
3. Smart-HTTP HEAD flapped to the read-only `docmost` mirror (~1/4 of clones). The
engine transiently checks out `docmost` mid-pull and the host advertises whatever
HEAD resolves to. Fix: VaultGit.pinHeadToMain(); the cycle restores HEAD->main in
a finally; and the upload-pack ref advertisement is served HEAD-pinned under the
per-space lock so it can never observe a mid-cycle HEAD.
4. (callout) clampCalloutType now mirrors the editor's GITHUB_ALERT_TYPE_MAP for
non-schema aliases (tip->success, caution->danger, important->info) instead of
flatly collapsing to info. The editor schema genuinely supports only the six
banner types, so unknown types still fall back to info (by design).
Tests: deterministic real-git trailing-blank round-trip (no conflict, no markers,
in sync over 2 cycles) + genuine-conflict no-marker-leak; HEAD advertisement
stability; pre/post-flush concurrent-edit survival; serveReadAdvertisement lock
pin; widened callout-alias coverage. Engine vitest + server tsc + collaboration /
git-http / orchestrator specs all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mandatory (test-coverage):
- internal-file-urls.test: pin the SSRF/traversal ACCEPT path of
resolveInternalFilePath (the sole guard for content-controlled `src`): an
absolute/protocol-relative URL has its foreign host dropped and only an
/api/files/ pathname survives (http://evil.com/api/files/x/y.png -> /files/x/y.png),
while a host-dropped path that escapes /api/files/ (https://evil.com/api/auth/whoami)
or a backslash-traversal (/api/files\..\auth\whoami) is rejected. Locks the
behavior so a future prefix-only refactor cannot silently open a bypass.
Suggestions:
- index.ts: the stash_page MCP tool now returns structuredContent
{ uri, sha256, size, images } alongside the resource_link, so the MCP output
matches the documented shape (clients get the blob's sha256/ETag and the
mirror counts, not just the link). No outputSchema registered. Rebuilt build/.
- new stash-page-mcp-result.test: server round-trip via InMemoryTransport asserts
both the resource_link and the structuredContent mirror.
- internal-file-urls.test: cover the new URL parse-failure catch branch
(http://[ -> "Invalid internal file src").
- environment.service.spec: assert getPositiveIntEnv warns once per key and
independently across keys (the invalidPositiveIntWarned dedup).
Tests: packages/mcp 383 pass; apps/server sandbox/environment/mcp 235 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Must-fix:
- mcp.module: drop the now-dead EnvironmentModule import (and its stale
comment). McpService no longer injects EnvironmentService; EnvironmentModule
is @Global and imported at the app root, so DI still resolves.
Stability:
- environment.service: route getSandboxTtlMs + the three SANDBOX_MAX_*_BYTES
caps through a shared getPositiveIntEnv() helper that warns once per key and
falls back to the default on a non-integer or <= 0 value (previously the byte
caps did a bare parseInt, so SANDBOX_MAX_TOTAL_BYTES=0 made every stash_page
fail against a 0-byte cap). TTL behavior is unchanged.
Simplification:
- sandbox.controller: replace the homemade UUID_RE with the project's shared
`uuid` validator (import { validate as isValidUUID } from 'uuid'), matching
the attachment routes; update the spec fixtures to valid v4 UUIDs.
- mcp.service: inline the single-caller one-liner buildSandboxConfig() to
this.sandboxStore.asSink() at the wiring site.
Docs:
- CHANGELOG: add an [Unreleased] > Added entry for #243 (stash_page tool,
anonymous GET /api/sb/:id, five SANDBOX_* env vars).
- AGENTS.md: note that GET /api/sb/:id is in the workspace-gate preHandler's
excludedPaths and is fully tokenless, unlike /api/files/public/... which
still resolves a workspace and needs an attachment JWT.
Tests: cap-getter validation (0/-5/abc -> default, valid -> parsed), updated
UUID fixtures. apps/server jest sandbox/environment/mcp: 233 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Bug #1 (push 503 starvation): an external receive-pack that briefly overlapped
a poll cycle immediately 503'd because the per-space single-writer lock was
held. Add a BOUNDED retry-acquire on the PUSH path only (SpaceLockService
.withSpaceLock acquireRetry: capped exponential backoff up to ~5s); a transient
overlap now waits and succeeds, a genuinely stuck cycle still 503s after the
bound. The poll cycle passes no retry (immediate skip). Push result stays
deterministic: the receive-pack only runs once the lock is held, so a 503 never
leaves a half-applied ref.
Bug #2 (concurrent-edit marker leak + silent same-block loss):
- Marker leak (a): the push UPDATE path stripped markers for the body sent to
Docmost but left raw <<<<<<</>>>>>>> committed on the published `main` vault
forever (autoMergeConflicts ON). Now the cleaned body is written back to the
vault file + recorded in writtenBack so runPush commits it on `main` and the
vault converges to clean bytes.
- Marker leak (b): pin merge.conflictStyle=merge in ensureRepo and teach
stripConflictMarkers/hasConflictMarkers about the diff3 `|||||||` base section
(drop the marker AND the stale base region) so diff3/zdiff3 conflicts can
never leak `|||||||` + base content into a page. Also scrub the 3-way merge
BASE markdown.
- Silent same-block loss: the block 3-way merge still resolves same-block
conflicts deterministically to git, but it is no longer silent: diff3Plan now
reports a conflict count (mergeXmlFragments3WayWithStats), gitSyncWriteBody
logs it, and the persistence boundary-snapshot now fires for git-sync writes
over a non-git-sync baseline so the human's pre-merge content is preserved in
page history (recoverable). Full both-preserved persisted-conflict UI remains
the deferred redesign.
Tests: space-lock bounded-retry (success/stuck/poll-immediate); push vault-clean
+ diff3 ||||||| strip; ensureRepo conflictStyle pin; diff3Plan/3-way conflict
counts; persistence git-sync boundary snapshot. Server tsc clean; git-sync
vitest + server collaboration/git-sync jest all green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Security (must-fix):
- sandbox.controller: the anonymous GET /api/sb/:id response now sets
X-Content-Type-Options: nosniff, a restrictive CSP, and Content-Disposition=
attachment for any mime outside a raster-image allowlist (png/jpeg/gif/webp/
avif). entry.mime is attacker-controlled, so an evil.svg/evil.html could
otherwise execute script inline on the Docmost origin (stored XSS). Mirrors
the public attachment route's hardening.
Stability:
- client.stashPage: reconcile mirrors AFTER the final document put, not only
before it. The doc blob is the newest entry and FIFO eviction drops the
oldest = this stash's own images, so the stored doc could reference an
evicted blob (consumer 404) and over-report images.mirrored. A bounded loop
now reverts doc-put-evicted mirrors, drops the stale doc blob, and re-puts
until stable. Regenerated packages/mcp/build/.
- sandbox.controller: emit Cache-Control on the 304 branch too (ttlSeconds is
computed before the conditional check).
Docs:
- Bump the MCP tool count 39 -> 40 across all READMEs and AGENTS.md (the
registry now exposes exactly 40 tools).
Refactor:
- SandboxStore.asSink() centralizes the {put,has,evict} sink + uri<->id
mapping; the embedded-MCP and in-app agent-tools wiring sites share it.
Tests:
- security headers (inline vs attachment, nosniff, CSP), 304 Cache-Control,
putAndLink URL form, has()/remove(), asSink() round-trip, getSandboxPublicUrl
(trailing-slash trim + APP_URL fallback), and a stash test where the doc put
itself evicts a mirrored image.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Security:
- stash_page: reject path-traversal / percent-encoded srcs before the authed
loopback fetch (resolveInternalFilePath), closing an SSRF/exfiltration hole
where a crafted node.attrs.src could read an arbitrary internal GET endpoint
into the anonymous sandbox.
Stability:
- stash_page: revert + recount mirrors FIFO-evicted by a later put in the same
stash (no dangling sandbox refs, honest images.mirrored/failed); free image
blobs if the final document put throws.
- Reject/clamp non-positive SANDBOX_TTL_MS to the 1h default (warn once).
- Log mirror failures unconditionally (console.warn, no blob bodies).
Cleanup / architecture:
- Remove dead expiresAt from SandboxPutResult.
- Centralize the /api/sb route in SANDBOX_ROUTE_SEGMENT/SANDBOX_API_PATH and
move URL composition into SandboxStore.putAndLink; drop the duplicated sink
closures and the now-unused EnvironmentService injection from McpService and
AiChatToolsService.
- Un-export isInternalFileUrl; document the process-local (instance-bound)
sandbox limitation in the tool description and .env.example.
Docs/tests:
- README/README.ru: 38 -> 39 tools + stash_page entry.
- Add traversal/normalize/recursion unit tests, stash self-eviction +
doc-put-throw + empty/octet-stream mock tests, controller If-None-Match
(wildcard/weak/list) + Cache-Control tests, and SANDBOX_TTL_MS validation
tests. Regenerate packages/mcp/build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Blocking (review id 2514):
- [security] Forbid symlinks in vaults. ensureServable now sets
core.symlinks=false in each vault's local git config (a pushed symlink is
checked out as a plain file, never a real link), and the engine cycle wraps
every read/write/mkdir in an lstat/realpath guard (new path-guard.ts) that
refuses a path that is — or traverses — a symlink, or whose realpath escapes
the vault root. Prevents a writer from publishing /etc/passwd or the server
.env, or writing outside the vault. Adds unit tests (path-guard.test.ts) +
a read-guard integration test (cycle.test.ts) + real lstat/realpath in the
roundtrip integration test.
- [simplification] Delete dead lib/diff.ts + test/diff.test.ts and drop the
now-unused @fellow/prosemirror-recreate-transform dependency.
- [documentation] Add a CHANGELOG [Unreleased] → Added entry for git-sync.
Warnings:
- [test-coverage] Cover the CREATE-branch conflict-markers guard (a new .md with
markers and no gitmost_id is recorded as a create failure, never created).
Suggestions:
- [stability] Bound each `git config` in ensureServable with a timeout.
- [authz] Trigger endpoint resolves spaceId workspace-scoped and 404s a foreign
space before any vault directory is created.
- [stability] Attribute git-initiated moves to the service account
(lastUpdatedById), via an optional actor param on PageService.movePage.
- [documentation] Document the per-space autoMergeConflicts toggle in AGENTS.md.
- [test-coverage] Cover the unterminated `:::` callout fence fallback.
- [simplification] Move test-only roundtrip-helpers.ts out of src/ into test/.
Architecture:
- Move the Yjs/ProseMirror merge primitives (yjs-body-merge, three-way-merge,
lcs + specs) into collaboration/merge/, breaking the collaboration →
integrations/git-sync dependency cycle this PR introduced.
- Port the schema-surface drift gate to packages/mcp (the mcp schema mirror had
none); pins 52 entries.
Deferred (with rationale in the review thread): the incremental-pull perf
warning (correctness-neutral; needs a high-water-mark design + its own tests on
the data-loss-critical path) and the redis-sync rolling-deploy mixed-version
edge (the deficient behavior is in already-released old-instance code; the new
code is correct on both sides; impact is a transient rollout-window artifact).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an ephemeral, process-local blob store so the in-app agent (and the
embedded MCP) can hand a large page document and its images to an external
consumer WITHOUT routing the bytes through the model context or Docmost auth.
- SandboxStore (@Injectable singleton): Map<uuid,{buf,mime,sha256,expiresAt}>
in RAM only. put() picks a per-blob cap by mime (image vs doc), enforces a
total-bytes RAM guard with oldest-first eviction, and stamps a TTL; get()
lazily expires. sha256 computed at put() doubles as the strong ETag. An
unref'd sweep interval clears expired entries and is cleared on destroy.
- GET /api/sb/:uuid anonymous controller: serves raw bytes with Content-Type,
Content-Length and ETag=sha256; 404 on missing/expired/non-UUID (anti-
traversal), 304 on a matching If-None-Match. No tokens, no 401 — the
capability is the unguessable UUID + short TTL + TLS. Auth-exempt the same
way as /api/files/public (no JwtAuthGuard) plus an /api/sb entry in main.ts's
workspace-resolution preHandler so a remote consumer with no workspace host
is not rejected.
- stash_page tool in both layers (MCP resource_link + in-app {uri,size,sha256,
images}). client.stashPage serializes the get_page_json shape, mirrors every
INTERNAL file/image src (type-agnostic, covers drawio/excalidraw/video/file)
into the sandbox under Docmost auth and rewrites src to the sandbox URL;
external http(s) srcs are left untouched; dedup by src; a failed image fetch
is counted, never aborts the doc.
- SANDBOX_PUBLIC_URL / SANDBOX_TTL_MS / SANDBOX_MAX_BYTES /
SANDBOX_MAX_IMAGE_BYTES / SANDBOX_MAX_TOTAL_BYTES wired through the
environment service + validation + .env.example.
- SandboxModule (@Global) provides the shared store to the controller,
McpService and AiChatToolsService (same instance for put and get).
Tests: SandboxStore (round-trip, sha256, TTL lazy + sweep, caps, eviction),
SandboxController (200+ETag+CT+CL, 404 missing/expired/non-UUID, 304), and a
mock-HTTP stashPage test (mirror+rewrite internal, keep external, dedup, failed
image counted, returns only a link). Interoperates with the vvzvlad/habr-mcp
consumer's anonymous-GET + sha256-ETag + resource_link contract.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1. gitRemote is NOT yet consumed (the vendored engine has no remote-push path,
SPEC §7). Corrected the buildSettings docstring (it wrongly called gitRemote
"load-bearing") and marked the env -> validation -> getter -> buildSettings
chain as inert SCAFFOLDING for the deferred remote-push feature at all three
sites. Kept the wiring (harmless; removing only churns).
2. .env.example: document that GIT_SYNC_REMOTE_TEMPLATE substitutes the literal
"{spaceId}" per-space (with the example), so an operator doesn't point every
space at one remote.
3. Extracted the copy-pasted CJS->ESM dynamic-import bridge
(`new Function('s','return import(s)')`) into one shared
common/helpers/esm-import.ts; git-sync.loader, docmost-client.loader and
mcp.service now import it and keep their own typed loadX() wrappers.
Deferred (notes only, not implemented):
- lcs.ts + three-way-merge.ts could move into packages/git-sync, but that engine
is vendored (manual re-sync) — added a one-line note at three-way-merge.ts to
revisit once the re-sync story is settled.
- schema-core single source + BullMQ/fencing remain documented from prior rounds.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
agent_qa
agent_coder
claude-stand
claude code agent 227
vvzvlad
a
