Add a lightweight "streaming" dictation mode as a simpler alternative to the
realtime-websocket path: detect speech with Silero VAD (@ricky0123/vad-web),
cut each segment on a pause and POST it to the existing /ai-chat/transcribe
endpoint, so text appears progressively. No server changes.
- new useStreamingDictation hook (same API as useDictation), lazy-loads VAD,
in-order seq emission, session-epoch guard against stop->start races
- new encodeWavPcm16 util (Float32 -> mono PCM16 WAV, accepted by the server)
- MicButton gains a `streaming` prop; enabled in the editor toolbar and chat
- VAD tuning: redemptionMs 640 / preSpeechPadMs 320 / minSpeechMs 96
- batch dictation kept as the fallback (streaming=false)
- deps: @ricky0123/vad-web@0.0.30, onnxruntime-web@1.27.0
Note: VAD assets load from the library CDN by default; for self-hosted/offline
set VAD_BASE_ASSET_PATH/VAD_ONNX_WASM_BASE_PATH and copy assets to public/vad/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add push-to-talk voice dictation that transcribes recorded audio on the
server via the workspace's OpenAI-compatible AI provider (Whisper /
gpt-4o-transcribe / self-hosted whisper), then inserts the text.
Backend:
- New `stt_api_key_enc` column + migration; STT creds parity with chat/
embeddings (sttModel/sttBaseUrl/sttApiKey, write-only key, fallbacks to
chat baseUrl/key). Both provider whitelists updated (service + repo).
- AiService.getTranscriptionModel + AiTranscriptionService.
- Gated POST /ai-chat/transcribe (dictation flag → 403, JWT + workspace
scope + throttle, 25MB cap, MIME whitelist, never logs audio/key).
- New `settings.ai.dictation` workspace flag (DTO + service + audit).
Frontend:
- Wire up the Voice/STT settings card (model/base URL/key) and the
Voice-dictation toggle.
- New `features/dictation`: useDictation (MediaRecorder state machine),
MicButton, transcribe service; integrated into the chat composer and a
new editor-toolbar dictation group, both gated by ai.dictation.
Add autoFocus to the chat composer Textarea so a freshly created chat
(window open, "New chat", chat switch — all remount ChatThread via key)
lands with the cursor ready in the input field, letting the user type
immediately without clicking into it.
Typing into the composer while the agent was streaming lost the draft once
the turn finished: on a brand-new chat, adopting the freshly created chat id
changes ChatThread's key and remounts it, wiping ChatInput's local state.
Lift the composer draft into a module-level jotai atom (aiChatDraftAtom) so it
survives the remount. Reset it only on deliberate chat switches — startNewChat,
selectChat, and the page-history "AI agent" badge deep-link — so a draft never
leaks between conversations, while adoption (which goes through a useEffect)
preserves it.
- Add reversible write tools to the per-user agent toolset (page create/update/
move/soft-delete; comment reply + resolve), exposed under the user's JWT and
enforced by Docmost CASL; no permanent/force delete (D3).
- Non-spoofable agent provenance: sign actor/aiChatId into the access and collab
tokens (TokenService), propagate via jwt.strategy onto the request, and set
pages.last_updated_source/last_updated_ai_chat_id on REST create/update/move and
comments.created_source/resolved_source/ai_chat_id.
- packages/mcp: add an optional getCollabToken provider (content-edit provenance)
and guard against empty tokens; service-account /mcp path unchanged.
Frontend:
- Admin 'AI / Models' settings section: provider/model/embedding/base URL, a
write-only API key field, system prompt, and Test connection.
- AI chat panel (useChat + DefaultChatTransport): conversation list, streamed
messages, tool-call action log and page citations; header entry point gated on
settings.ai.chat.
Compile-verified (server nest build + client tsc/vite); not yet live-tested.
Known gaps: history 'AI agent' badge (C3), vector RAG (D), external MCP (E);
chat tool-card citation links pending a fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
