First, safe step of docs/backlog/ai-chat-tool-definitions-duplicated.md: the
"node may be a JSON object OR a JSON string" quirk was hand-copied at 6 tool
sites. Extract it into a single parseNodeArg() helper per package and call it at
every site. Behavior-preserving — each site's throw message is byte-identical
(patch/insert: 'node was a string but not valid JSON'; update_page_json: 'content
was a string but not valid JSON'); no tool name/description/schema changed.
Two helper copies (packages/mcp/src/lib/parse-node-arg.ts and
apps/server/src/core/ai-chat/tools/parse-node-arg.ts) are intentional: the
ESM-only @docmost/mcp cannot be imported by the CommonJS server (it is loaded at
runtime via the Function('import()') trick), so runtime code cannot cross that
boundary by a normal import. Each copy is now the single source within its
package (6 inline copies -> 2 helpers). packages/mcp/build rebuilt in sync.
Tests: parse-node-arg.spec.ts (server, Jest) + parse-node-arg.test.mjs (mcp,
node:test) — object passthrough, valid-string parse, invalid-string throw with
the right message. Server tsc clean; mcp suite 254 pass; agent structural-edit
path verified live in-browser (agent inserted a node, persisted to the doc).
Deferred (documented for the record, since the backlog doc is removed with this
commit): the FULL transport-agnostic tool-spec registry (one name+schema+
description per tool shared by both transports) and deriving DocmostClientLike
from the real client type. Both are blocked by the current architecture, not by
effort: (1) @docmost/mcp ships no type declarations and is ESM-only, so a
type-only derivation needs declaration emission + tsconfig path wiring, and the
real client's precise return types break the in-app tool test stubs (attempted,
reverted to keep tsc green); (2) the two transports intentionally DIVERGE in tool
NAMES (snake_case x38 vs camelCase x41), membership (in-app adds getCurrentPage/
listSidebarPages, omits delete_comment/image tools) and model-facing
DESCRIPTIONS, so a unified registry would change behavior on BOTH the agent and
external MCP clients and needs its own verification pass. This is forward-looking
debt (the code is correct today), to be done incrementally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The PM<->Markdown converter and its lib are duplicated the same way as the
AI-chat tool definitions: a copy lives in packages/mcp/src/lib (without
canonicalize.ts), another in docmost-sync's docmost-client lib (with
canonicalize + the no-comment-threads markdown-document mode), and the
git-sync integration plan vendors a third copy into packages/git-sync.
Record the already-observed drift (collaboration.ts ~329 changed lines,
etc.) and the docmost-schema vs @docmost/editor-ext schema-divergence risk,
and tie it to the existing single-source-of-truth fix direction.
Follow-up fixes to the htmlEmbed-sandbox / trackerHead change:
- share-seo: inject trackerHead via a function replacer so `$`-sequences
($&, $', $`, $$) in the admin snippet are inserted literally instead of
being treated as String.replace substitution patterns; warn when the
</head> marker is absent instead of silently skipping injection.
- mcp: register a passthrough `htmlEmbed` node in the schema mirror so an
AI/MCP edit of a page containing an embed no longer throws
"Unknown node type: htmlEmbed" in TiptapTransformer.toYdoc.
- editor-ext + client: treat a non-finite `data-height` as auto (null) so a
crafted/corrupted height cannot disable auto-resize or yield a NaN iframe
height; extract a shared clampHeight helper.
- client: rename render-raw-html.{ts,test.ts} -> html-embed-sandbox.{...} and
shouldExecute -> shouldRender so the seam name matches the sandbox model.
- client: i18n the iframe title; surface the real error reason in
tracker-settings (console.error + err.response.data.message).
- docs: note hasHtmlEmbedNode is now a test-only helper; add an Unreleased
CHANGELOG entry; drop the dangling "arbitrary HTML embed" planning-doc ref.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keep the backlog focused on deferred TESTS; the related non-test gaps
(model-allow-list, restriction-cache invalidation, server embed-recursion
guard, collectPageEmbeds cycle guard, jest DI/lib0-ESM debt) are now
tracked as issues #52-#56 and only linked from the backlog.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Captures what PR #49 intentionally left out: DB-integration tests (need a
test Postgres), the public-share XFF e2e + real-Redis Lua check (need an
HTTP/Redis harness), the full AiChatService.stream integration (R1-stream
seam), and the related non-test findings (no server-side model allow-list,
unreferenced restriction-cache invalidation, client-only embed recursion
cap, missing cycle guard, and the pre-existing jest DI/lib0-ESM debt).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The admin-only raw HTML/JS embed is a deliberate stored-XSS surface, so gate the
whole feature behind a workspace toggle that is OFF by default; it only works
when a workspace admin explicitly enables it.
- settings.htmlEmbed (boolean, default false) + workspace-update field htmlEmbed,
persisted via WorkspaceRepo.updateSetting with an audit diff. Flipping it is
admin-only (same Manage Settings CASL as other workspace toggles).
- New gate htmlEmbedAllowed(featureEnabled, role) = featureEnabled && admin/owner.
All 7 server write paths (create, duplicate, collab onStoreDocument, REST/MCP/AI
updatePageContent, single + zip import, transclusion unsync) now read the
workspace's settings.htmlEmbed and strip unless (toggle ON AND admin). OFF
(default, or a failed/empty workspace lookup) strips htmlEmbed for EVERYONE
including admins -> existing embeds are cleaned up on next save, none persist.
- Client (defense-in-depth): the /html slash item is hidden unless toggle ON +
admin; the NodeView executes nothing and shows a 'disabled in this workspace'
placeholder when OFF; an admin Switch in Workspace Settings -> General with a
description of the behavior.
- docs/html-embed-admin.md documents the toggle + admin-only + fail-closed
coedit (a non-admin save strips an admin's embed) + execution semantics.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve conflicts with the independently-merged ai-agent-roles feature:
- ai-chat.module.ts: keep BOTH AiAgentRolesModule and the public-share
wiring (Share/Search modules, PublicShareChatController, services).
- ai.service.ts: take develop's getChatModel ChatModelOverride superset,
which already covers the public-share model-id-only override.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Integrate the already-merged step-limit work from develop. Only conflict was
ai-chat.service.spec.ts: both sides appended a describe block and edited the
import line. Resolved as a union — keep compactToolOutput + the assistantParts/
serializeSteps/rowToUiMessage suites (this branch) AND the prepareAgentStep
suite (develop), importing all symbols from ai-chat.service.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unit tests for the safety-critical paths: crypto secret-box (round-trip,
tamper detection, wrong key), the SSRF guard (blocked ranges + DNS-rebinding),
the ai-chat tools service, the page-embedding repo, and the
assistant-parts/serialization helpers. Those server helpers (assistantParts,
rowToUiMessage, serializeSteps) are exported ONLY for the tests — no runtime
change.
Also: keyboard a11y on the chat history header and conversation rows
(role/tabIndex/Enter+Space), and DRY refactors that move shared logic into one
place (isToolPart -> tool-parts util; buildInitialValues in the MCP form).
The behaviour-changing edits that previously rode along in this commit are
split out into the following two commits, per review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The configured x enabled status dot is implemented and merged via this
branch, so the backlog plan is no longer needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The in-field Clear for the API key fields is implemented and merged via
this branch, so the backlog plan is no longer needed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Delete the backlog documentation that described the removal of non‑functional DOCX, PDF, and Confluence import features now that the code changes have been merged.
The two catch blocks in importPage() threw an opaque "Error processing file
content" / "Failed to create imported page" BadRequest, hiding the real cause
from the HTTP response. This made a production 400 regression impossible to
diagnose without server log access, and violated the project convention that
errors must never be swallowed.
Extract `${err.name}: ${err.message}` into both the log (full err object kept
for the stack) and the thrown BadRequestException. Inner processMarkdown/
processHTML rethrowing catches and the EE processDocx/processPdf license
catches are left unchanged.
Local reproduction of the happy-dom 14->20 theory failed (full import chain
+ 22 edge cases pass on happy-dom@20.8.9), so the root cause is still pending
the now-visible reason from a recurring 400. Diagnostic script test-import.tsx
added; backlog doc updated with findings.
Add a dedicated section describing the licensing conflict between the AGPL‑3.0‑licensed web client and App Store DRM/usage rules. Explain why this is a non‑technical blocker, outline possible distribution approaches (server‑loaded client, OTA updates, PWA, sideload), and recommend confirming the chosen path before implementing any iOS wrapper code.
Add markdown files describing the per‑user authentication mechanism and the ability to expand or collapse all nodes in the page tree, improving guidance for developers working with the MCP backlog feature.
Add two new backlog documentation files:
- ai-chat-collapse-on-page-focus.md describing auto‑collapse behavior for the AI chat window.
- comments-panel-density.md outlining UI density improvements for the comments panel.
Add a backlog design note for making page-tree realtime updates
server-authoritative instead of client-relayed.
Problem: page content syncs via Yjs/Hocuspocus (server-authoritative),
but tree create/move/delete is broadcast by the originating browser only,
so non-UI creation paths (AI agent, MCP, REST API, import) and lost-event
races leave other clients' sidebars stale.
The note specifies a WsService.emitTreeEvent broadcaster, WsTreeService
broadcast helpers, a PageWsListener on PAGE_CREATED/SOFT_DELETED/DELETED/
MOVED/RESTORED, event-payload enrichment to avoid the in-transaction
re-fetch race, a dedicated PAGE_MOVED event, removal of the client relay,
plus edge cases, work breakdown, tests, alternatives and open questions.
Remove outdated process sections from several backlog markdown files and add new backlog items for AI chat step limits, endpoint status config, and API key field UI improvements.
Add docs/backlog/stt-providers-and-async.md: how to add new synchronous STT
request formats (Deepgram, native Gemini, ElevenLabs) via the explicit
sttApiStyle axis, which providers are inherently async and don't fit the
current sync model, and a target job-based async architecture (BullMQ job
table, sync+async unification, polling -> push -> live streaming) with the
migration path and security/cleanup considerations.
Add docs/streaming-dictation-plan.md — a design document for true
"text appears as you speak" dictation via the OpenAI Realtime API.
- Maps the current batch dictation flow (client MediaRecorder -> single
blob -> POST /ai-chat/transcribe) and why streaming is impossible there.
- Documents the Realtime API contract (transcription session, ephemeral
token, pcm16 audio, input_audio_buffer.append, input_audio_transcription
delta/completed events, server_vad).
- Recommends a server-side WS proxy transport (key stays server-side,
SSRF-guarded, provider-agnostic via sttBaseUrl) over direct browser
WebRTC, and a ProseMirror decoration for interim text with final-only
commit to avoid polluting Yjs collab/history.
- Covers config additions, AudioWorklet PCM16 capture, security per repo
conventions, edge cases, phased rollout, risks, and impacted files.
The README files now list Voice dictation as a completed feature (✅) instead of an upcoming one (🔭). Consequently, the detailed `voice-dictation-plan.md` documentation has been removed. This reflects that the feature is ready and no longer merely a plan.
Add a detailed design and implementation plan for an AI assistant that
operates on publicly shared document trees. The document outlines the
feature scope, architecture, security considerations, and remaining work,
providing context for future development.
Add docs/page-templates-plan.md describing a whole-page live
transclusion feature: pages flagged is_template, a new pageEmbed
node referencing a source page, a whole-page lookup endpoint reusing
the existing transclusion access-control and share paths, reference
sync, duplicate remap, and cycle/deletion/access/export edge cases.
Decision: separate pageEmbed node over extending transclusionReference.
