Commit Graph

99 Commits

Author SHA1 Message Date
vvzvlad
a945b47749 fix(mcp): verifiable mutation results + refuse formatting edits in edit_page_text
edit_page_text reported "success" when asked to change formatting (e.g. remove
strikethrough): the markdown-strip fallback matched the bare text, the replace
preserved marks, and the tool returned success — so the agent believed it had
fixed something that never changed.

Two fixes, both in the shared @docmost/mcp DocmostClient so they reach BOTH the
standalone MCP server and the in-app AI chat (which loads @docmost/mcp):

- Verifiable result for every content mutator: mutatePageContent now computes a
  `verify` change-report (text inserted/deleted, blocks changed, per-mark-type
  delta, integrity/structure delta) via summarizeChange() and returns it on all
  mutators (incl. replaceImage via mutateLiveContentUnlocked). diffDocs is
  text-only, so the mark/structure delta is what surfaces formatting changes.
- edit_page_text hard-refuses formatting edits: applyTextEdits rejects an edit
  whose find/replace differ only in markdown markers (via stripBalancedWrappers,
  which strips balanced wrappers/links without trimming whitespace/emoji, so
  plain-text edits like trailing-space trims, snake_case, math are NOT refused).
  A fully-refused batch errors instead of silently succeeding.

Also updated the model-facing edit_page_text descriptions in BOTH tool layers
(packages/mcp/src/index.ts and ai-chat-tools.service.ts) to drop the misleading
"strip-and-retry tolerated" wording and point formatting changes to patch_node.

New unit tests: test/unit/diff-verify.test.mjs, test/unit/json-edit-refuse.test.mjs.
2026-06-18 05:46:13 +03:00
vvzvlad
334a50f003 feat(mcp): fetch insert_image/replace_image sources from web URLs
The insert_image and replace_image MCP tools previously uploaded only
local files (filePath), which an AI MCP client cannot provide — it has
no access to the server filesystem. Replace filePath with a required
imageUrl and download the image over http(s).

- client.ts: add fetchRemoteImage(url, maxBytes) — http/https-only scheme
  allowlist, 20 MiB cap (maxContentLength + post-download length recheck),
  30s timeout, Content-Type→MIME resolution with URL-extension fallback,
  filename derivation with canonical extension
- client.ts: rewrite uploadImage(pageId, url) as URL-only; drop the
  local-file branch, imageMimeFromPath and the fs import; insertImage/
  replaceImage now take a url
- index.ts: drop filePath, add required imageUrl to both tools; update
  tool descriptions and SERVER_INSTRUCTIONS
- README: document the web-URL behaviour
2026-06-18 01:28:23 +03:00
vvzvlad
afd2248a75 feat(ai-chat): tolerate markdown in edit_page_text/insert_node locators
Locators (edit_page_text `find`, insert_node `anchorText`) are matched
against the document's plain text, so a model-supplied locator carrying
markdown wrappers (**bold**, *italic*, `code`, [t](url)) or trailing emoji
never matched and the edit/insert failed. Add stripInlineMarkdown() and a
fallback: try the locator verbatim first (exact match wins, so literal
asterisks/underscores still work), and only on zero matches retry with a
markdown-stripped form. The ambiguity guard runs on the post-fallback count,
and `replace` / inserted node content are never stripped, so no formatting is
lost. Failed edits gain an atom-aware reason plus a bounded "closest block
text" hint; the insert_node "anchor not found" error now points at plain-text
anchors / anchorNodeId.

New packages/mcp/src/lib/text-normalize.ts (+ unit tests); wired into
json-edit.ts and node-ops.ts; tool descriptions updated. Tests: 212 pass.
2026-06-17 15:44:19 +03:00
vvzvlad
fc9088b74d fix(ai-chat): cross-mark text edits, partial batches, JSON-string node parity
edit_page_text (applyTextEdits) now matches at the inline-block level instead of
per text node, so a find/replace may cross bold/italic/link boundaries; the
replacement inherits marks from the unchanged common prefix/suffix via a diff
splice. Atom (non-text inline) slots can never be part of a match, making the
U+FFFC placeholder collision-safe, and inserted text never inherits an atom's
marks.

The edit batch is no longer all-or-nothing: applyTextEdits returns
{ doc, results, failed } and applies what it can; editPageText writes only on a
real change (no spurious history version for a no-op) and throws an aggregated,
actionable error only when nothing applied.

The AI-chat insert_node / patch_node / update_page_json tools now JSON.parse a
node/content argument that arrives as a string, matching the standalone MCP
server (this is what made insert_node fail under OpenAI tool calls).

Tool descriptions gain concrete ProseMirror examples and reflect the new
edit_page_text behavior. Adds/updates json-edit unit tests (183 pass).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 06:57:58 +03:00
vvzvlad
a4b7919753 fix(ai-chat): OpenAI Chat Completions for multi-turn + provider settings, stream UX & errors" -m "Live-stand fixes (OpenRouter / OpenAI-compatible):
- openai provider: use .chat() (Chat Completions) instead of the default callable
  (Responses API), which gateways reject on multi-turn -> 400.
- updateAiProviderSettings: assemble settings.ai.provider via jsonb_build_object
  with ::text-cast bound params + jsonb_typeof self-heal (postgres.js was
  double-encoding it into an array; the ::text cast avoids 'could not determine
  data type of parameter').
- chat agent: drop the hard maxOutputTokens cap (truncated complex tool calls);
  keep a tiny cap only on the test-connection ping.
- testConnection + chat stream: surface the real provider error (statusCode+message)
  to logs and the UI instead of generic masks; never log the API key.
- chat UI: typing indicator, incremental streaming render, tool 'running' status, Stop.

Also bundled (prior uncommitted ai-chat work):
- history 'AI agent' provenance badge; vector RAG (pgvector image + page_embeddings
  + AI_QUEUE indexer + space-scoped semanticSearch); external MCP servers backend
  (@ai-sdk/mcp client, SSRF IP-pinning, encrypted headers, admin CRUD/Test);
  yjs duplicate-instance fix via pnpm patch (single CJS instance server-side).
2026-06-17 04:28:29 +03:00
vvzvlad
44b340dc1a feat(ai-chat): agent write tools, provenance wiring, chat panel + provider settings UI" -m "Backend:
- Add reversible write tools to the per-user agent toolset (page create/update/
  move/soft-delete; comment reply + resolve), exposed under the user's JWT and
  enforced by Docmost CASL; no permanent/force delete (D3).
- Non-spoofable agent provenance: sign actor/aiChatId into the access and collab
  tokens (TokenService), propagate via jwt.strategy onto the request, and set
  pages.last_updated_source/last_updated_ai_chat_id on REST create/update/move and
  comments.created_source/resolved_source/ai_chat_id.
- packages/mcp: add an optional getCollabToken provider (content-edit provenance)
  and guard against empty tokens; service-account /mcp path unchanged.

Frontend:
- Admin 'AI / Models' settings section: provider/model/embedding/base URL, a
  write-only API key field, system prompt, and Test connection.
- AI chat panel (useChat + DefaultChatTransport): conversation list, streamed
  messages, tool-call action log and page citations; header entry point gated on
  settings.ai.chat.

Compile-verified (server nest build + client tsc/vite); not yet live-tested.
Known gaps: history 'AI agent' badge (C3), vector RAG (D), external MCP (E);
chat tool-card citation links pending a fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 02:39:26 +03:00
vvzvlad
683da7a4c5 feat(ai-chat): per-user AI agent backend — LLM config, read-only agent, provenance schema
WIP checkpoint of the gitmost AI-chat backend (plan stages A + B1 + B3a).
The agent acts under the requesting user's JWT (Docmost CASL enforces page
access); the external service-account /mcp endpoint is untouched.

LLM provider config (A2-A4):
- integrations/crypto: AES-256-GCM SecretBoxService (key derived from APP_SECRET,
  per-record salt/iv; clear error on rotation instead of crashing).
- ai_provider_credentials table/repo/types: encrypted API key stored outside
  workspace settings/baseFields, write-only (never returned by any endpoint).
- integrations/ai: per-workspace AI SDK v6 provider driver (openai/gemini/ollama),
  admin-gated GET(masked)/PATCH(write-only key)/Test endpoints; settings.ai.provider
  holds non-secret config incl. systemPrompt. Removed unused AI_* env getters (DB is
  the single source of truth).

Chat module (A1, A5-A8):
- ai_chats/ai_chat_messages repos (workspace-scoped, soft-delete, tsv never selected).
- core/ai-chat: CRUD + POST /ai-chat/stream (Fastify hijack + AI SDK v6
  pipeUIMessageStreamToResponse, abort on disconnect, persist user/assistant msgs).
- Agent loop: streamText + stepCountIs(8); read tools searchPages/getPage via a
  per-request DocmostClient over loopback REST under the user's minted access token.
- Gate settings.ai.chat (+ 503 when provider unconfigured); buildSystemPrompt with a
  non-removable safety/anti-prompt-injection framework. Per-user rate limit.

Per-user auth (B1):
- @docmost/mcp DocmostClient gains an additive getToken variant (carry a user JWT,
  re-fetch on 401) and exports DocmostClient; the email/password service-account path
  (external /mcp, stdio) is unchanged.

Agent-edit provenance backbone (B3a):
- Migration: pages/page_history (last_updated_source, last_updated_ai_chat_id) and
  comments (created_source, ai_chat_id, resolved_source).
- Signed actor/aiChatId claim in the collab token; onAuthenticate propagates it,
  onStoreDocument writes it with a sticky agent marker, saveHistory copies it.

Migrations auto-run on boot (additive). Write tools, frontend, RAG and external MCP
servers are not in this checkpoint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 01:36:41 +03:00
vvzvlad
1f5987d6b0 feat(mcp): serve embedded community MCP server at /mcp
Replace the removed enterprise EE MCP (private apps/server/src/ee submodule,
license-gated /mcp route) with our docmost-mcp, vendored as an isolated ESM
workspace package and served by the server over HTTP — no enterprise license.

Backend:
- Add packages/mcp (@docmost/mcp): vendored docmost-mcp refactored into a
  side-effect-free createDocmostMcpServer() factory (38 tools preserved),
  stdio entry kept in stdio.ts, Streamable-HTTP session manager in http.ts.
- Add apps/server McpModule: @Post/@Get/@Delete('mcp') (served at /mcp via the
  existing global-prefix exclude), @SkipTransform + reply.hijack to bridge raw
  Fastify req/res into the SDK transport. The module dynamically imports the
  ESM-only package from CommonJS via a Function-indirected import resolved with
  require.resolve + file:// URL. Gated by the workspace ai.mcp toggle, a
  service-account (MCP_DOCMOST_EMAIL/PASSWORD/API_URL) and optional MCP_TOKEN;
  per-session idle eviction (MCP_SESSION_IDLE_MS).
- Drop the enterprise license check on mcpEnabled in workspace.service.
- Dockerfile: copy packages/mcp into the production image.
- .env.example: document MCP_DOCMOST_*, MCP_TOKEN, MCP_SESSION_IDLE_MS.

Frontend:
- Recreate the community "AI & MCP" workspace-settings panel (mcp-settings.tsx):
  admin-only toggle on settings.ai.mcp with optimistic update, copyable
  ${APP_URL}/mcp URL; wired into workspace-settings page. Reuses existing i18n.

Fixes:
- Pin packages/mcp tiptap deps to 3.20.4 (matching the client) and inline
  getStyleProperty, preventing a duplicate @tiptap/core@3.26.1 from leaking into
  the client editor via pnpm shamefully-hoist (was breaking apps/client tsc).
2026-06-16 23:54:53 +03:00
vvzvlad
a88b3f776c feat(fork)!: remove all frontend Enterprise-Edition code (community build)
Strip the proprietary client EE so the fork ships a clean community/AGPL
edition, mirroring Forkmost. Delete apps/client/src/ee (201 files) and
packages/ee, and patch every consumer that imported from @/ee/*.

- gate-out EE features (useHasFeature -> false): API keys, SSO, MFA, SCIM,
  audit logs, AI / AI-chat, templates, page permissions, page verification,
  comment resolution, trash retention, viewer comments
- drop cloud/billing/trial/entitlement/posthog flows; sign-in is now
  email+password only (no SSO/LDAP/cloud)
- remove EE routes from App.tsx and EE entries from sidebars/settings nav
- restore the community page-share button (ShareModal) that the EE
  PageShareModal used to provide
- remove the dead "Attachments" search filter, dead MFA navigation and
  orphaned route constants

Client type-checks clean; full `pnpm build` is green for all three projects.
2026-06-16 22:32:44 +03:00
Philip Okugbe
92c0e36e46 fix(a11y): WCAG 2.1 AA fixes (#2219) 2026-05-20 16:47:25 +01:00
Olivier Lambert
1c166c4736 feat(editor): add alt text support for images (#2097)
* feat(editor): add alt text support for images
* feat:  extend alt text support to videos and diagrams

---------
Co-authored-by: Philipinho <16838612+Philipinho@users.noreply.github.com>
2026-05-20 16:45:59 +01:00
Philip Okugbe
b7b99cb3b2 fix: code splitting and editor fixes (#2211)
* fix table

* fix code splitting

* fix: editor ready check

* fix codeblock/mermaid gap cursor

* fix callout
2026-05-15 02:46:54 +01:00
Philip Okugbe
f4af4c3fc0 feat(editor): add page break node (#2202) 2026-05-14 03:48:13 +01:00
Philip Okugbe
cea9be7926 feat: table enhancement (#2191) 2026-05-14 00:37:44 +01:00
Philip Okugbe
537e45bc11 feat: page details section and backlinks (#2186)
* feat: page details section and backlinks
2026-05-09 17:03:08 +01:00
Philip Okugbe
2d8b470495 feat(editor): indentation (#2174)
* switch to default codeblock tab handling

* feat(editor): indentation
2026-05-08 21:40:37 +01:00
Philip Okugbe
de60aa7e61 feat: synced blocks (transclusion) (#2163)
* feat: synced blocks (transclusion)

* fix:remove name

* make placeholders smaller

* feat: enforce strict transclusion schema

* fix: scope synced blocks to workspace, gate unsync on edit permission

* fix collab module error
2026-05-08 13:23:16 +01:00
Philip Okugbe
d6068310b4 Merge commit from fork
Refactor link.ts to simplify HTML parsing and rendering logic.
2026-04-13 01:09:36 +01:00
Philip Okugbe
57efb91bd3 feat(ee): ai chat (#2098)
* feat: ai chat

* feat: ai chat

* sync

* cleanup

* view space button
2026-04-10 19:23:47 +01:00
Philipinho
bca85a49d6 pin marked version 2026-03-29 03:03:35 +01:00
Philip Okugbe
412962204c fix: editor fixes (#2067)
* autojoiner

* fix marked

* return clipboardTextSerializer as markdown

* fix clipboardTextSerializer for single lines

* cleanup two preceeding spaces in ordered lists item

* fix extra paragraph in task list

* don't zip sinple page exports
2026-03-29 02:19:09 +01:00
Philip Okugbe
7981ef462e feat(editor): audio and PDF nodes (#2064)
* use local resizable

* feat: aduio

* support audio imports

* feat: use confluence real file names

* cleanup

* error handling

* hide notice

* add audio

* fix pulse

* Fix import and export

* unify pulse

* hide in readonly mode

* keywords

* keyword

* translations

* better sort

* feat: PDF embed

* cleanup

* remove audio menu

* open active

* hide focus on readonly mode

* increase iframe default dimension
2026-03-28 17:33:29 +00:00
Philipinho
6d6f3a8a8e merge commit 2026-03-24 10:52:09 +00:00
Philip Okugbe
6683c515cf fix: make codeblock language detection performant (#2032)
* fix: make codeblock language detection performant
* lint
2026-03-17 20:40:22 +00:00
Philip Okugbe
89b94e5d19 feat: refactor link menu (#2025)
* link markview - WIP

* WIP

* feat: refactor links

* cleanup
2026-03-15 17:08:59 +00:00
Philip Okugbe
b4f009513e fix: resize handle clipping (#1990) 2026-03-04 12:24:46 +00:00
Philipinho
fcffa3dfa0 fix media 2026-03-04 12:08:08 +00:00
Philip Okugbe
bea1637519 fix: image fallback regression (#1989)
* fix: image fallback regression

* fix image preview on upload

* fix image loading
2026-03-04 11:51:43 +00:00
Olivier Lambert
f5d794220e fix: resolve keystroke input being swallowed after link in Firefox (#1922)
* fix: resolve keystroke input being swallowed after link in Firefox

In Firefox, when the cursor is at the right boundary of a link mark,
contenteditable inserts new text inside the <a> element. ProseMirror
then rejects the DOM mutation because the link mark has inclusive: false,
causing keystrokes to be silently swallowed. Unlike Chrome, Firefox also
does not fire ProseMirror's handleTextInput callback in this state.

This adds a ProseMirror plugin that intercepts printable character
keydowns at link mark boundaries and programmatically inserts the text
without the link mark, bypassing Firefox's native contenteditable
behavior entirely.

Fixes #1773

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve keystroke input being swallowed before a link in Firefox

Extend the linkBoundaryInput plugin to also handle the left boundary
of links, where the cursor is just before a link (e.g. at the start
of a line). Firefox inserts text inside the <a> element in this case
too, causing ProseMirror to reject the mutation.

Fixes #1748
2026-03-03 17:19:03 +00:00
Philip Okugbe
fc0997fd90 feat: editor attachment paste handling (#1975)
* reupload attachments if uploaded to a different page
* use image dimensions on paste/DnD

* tooltips withinPortal:false

* isolating attribute
2026-02-28 01:24:19 +00:00
Philip Okugbe
ea44468fad feat: editor inline status node (#1973)
* inline status node

* fix alignment

* fix

* typed storage

* fix math block popup on select all
2026-02-27 01:34:03 +00:00
Philipinho
22f33bab7c cleanups 2026-02-25 22:41:54 +00:00
Philipinho
e0a8521566 enhance columns 2026-02-25 22:31:01 +00:00
Philip Okugbe
ef87210b3d feat: editor UI refresh and enhancements (#1968)
* feat: new image menu
* switch to resizable side handles
* use pixels

* refactor excalidraw and drawio menu

* support image resize undo

* video resize

* callout menu refresh

* refresh table menus

* fix color scheme

* fix: patch @tiptap/core ResizableNodeView to prevent resize sticking after mouseup

* feat: columns

* notes callout

* focus on first column

* capture tab key in column

* fix print

* hide columns menu when some nodes are focused

* fix print

* fix columns

* selective placeholder

* fix blockquote

* quote

* fix callout in columns
2026-02-24 15:22:37 +00:00
Julien Fontanet
03a70d768a fix: allow deleting last character in headings (#1954)
The copy-link decoration widget (contentEditable="false") injected
inside headings prevented browsers from deleting the last remaining
character via Backspace or Delete keys. Only show the widget when the
heading has more than one character of content.
2026-02-18 13:48:15 +00:00
Philipinho
25f4b8c2b4 fix 2026-02-11 17:47:30 -08:00
Philip Okugbe
7879e1f600 fix: add execCommand fallback for clipboard (#1927)
* fix: add execCommand fallback for clipboard
2026-02-09 14:44:27 -08:00
Philip Okugbe
4878850b25 fix: attachment bugs in safari(#1908)
* use widely available arrayBuffer
* fix stream fails in safari
* fix hasFocus bug
* fix safari upload bug
* feat: add HTTP range request support for file serving
2026-02-05 07:47:03 -08:00
Philip Okugbe
5506eb194b feat: page history diff (#1891)
* Show actual history changes
* V2 - WIP
* feat: page history diff
* fix: exclude content from history listing

---------

Co-authored-by: Jason Norwood-Young <jason@10layer.com>
2026-02-03 11:55:20 -08:00
Pleasure1234
3178cad796 fix: handle empty replace term in search and replace functionality (#1562)
- Fix 'Empty text nodes are not allowed' error when replace field is empty
- Update both replace() and replaceAll() functions to check for empty replaceTerm
2026-01-30 22:37:22 +00:00
Philip Okugbe
74e915546b feat: collab redis extension with server affinity (#1873)
* feat(collab): better redis extension
* move types to own file
* debug logging
* fix: graceful collab shutdown
* rename default prefix
* pass wsAdapter to gateway
* expose event handler
* unique collab serverId generation
* uninstall @hocuspocus/extension-redis package
* expose more functions
* sync with latest
* cleanup
* fastify router options
* cleanup type
2026-01-27 17:05:05 +00:00
Philipinho
5dbf0027bd Add isomorphic basename utility 2026-01-25 00:08:02 +00:00
Philipinho
55b8128829 Fix Google sheets regex 2026-01-24 23:35:04 +00:00
Philip Okugbe
aa6a046aa6 feat(export): add export loading state and copy as markdown (#1867)
* feat: add loading state to export

* feat: copy as markdown

* preserve taskList comment
2026-01-24 23:30:17 +00:00
Philip Okugbe
657fdf8cb7 feat: Tiptap V3 migration (#1854)
* Tiptap3 migration - WIP

* fix collaboration

* remove unused code

* fix flicker

* disable duplicate extensions

* update tiptap version

* Switch to useEditorState
- Set shouldRerenderOnTransaction to false

* fix editable state

* add tippyoptions for reference

* merge main

* tiptap 3.6.1

* fix bubble menu

* fix converter

* fix menus

* fix collaboration caret css

* fix: Set `isInitialized` to force immediate react node view rendering

* feat: Migrate tippy.js menus to Floating UI

* feat: Update collaboration connection for HocusPocus v3

* fix: Connect/disconnect websocketProvider

* cleanup

* cleanup

* feat: Improved placeholder and upload handling for images

* feat: Improved placeholder and upload handling for videos

* refactor: Image node and view clean-up

* feat: Improved placeholder and upload handling for attachments

* fix: Video view styles

* fix: Transaction handling on asset upload

* fix: Use imageDimensionsFromStream

* feat: Multiple file upload, improved placeholders, local previews

* fix: Drag & drop, paste upload

* fix: Allow media as attachment

* * add skeleton pulse animation
* add translation strings
* fix attachment view responsiveness

* fix collab connection status display

* Tiptap v3.17.0

* fix suggestion menu exit bug

* fix search shortcut

* fix history editor css

* tiptap 3.17.1

---------

Co-authored-by: Arek Nawo <areknawo@areknawo.com>
2026-01-24 20:41:08 +00:00
Philipinho
d59539f197 fix ai streaming 2025-12-13 14:15:41 +00:00
Philip Okugbe
6e350f6746 fix nodeview dragging (#1775) 2025-12-11 19:32:18 +00:00
Philip Okugbe
d2629afff2 feat: anchor links (#1765)
* feat: add heading extension with unique ID support and scroll functionality
* Added unique id for heading
* remove baseUrl heading storage
* move heading to extensions package
* WIP
* support anchors in mentions
* enhance scrolling functionality
* nodeId function
* fix nanoid import
* Bring unique-id extension local
* fixes
* fix internal link scroll in public pages
* add unique id server side
* rename mention anchor to anchorId
* capture first anchorId on paste

---------

Co-authored-by: Romik <40670677+RomikMakavana@users.noreply.github.com>
2025-12-06 14:46:54 +00:00
Philip Okugbe
8014ba3ab7 feat: Text background highlight (#1754)
* #1196/feat: add text background highlight

* unify text color

* dark mode support
* unify text color and highlight

* dark mode support for color selector trigger

* fix see through in color selector dark mode

* fix selection highlight in dark mode

* brown color

* clean up

---------

Co-authored-by: sanua356 <sanek.pankratov356@gmail.com>
2025-12-01 11:34:35 +00:00
Philip Okugbe
9ac180f719 fix: enhance page import (#1570)
* change import process

* fix processor

* fix page name in notion import

* preserve confluence table bg color

* sync
2025-09-17 23:50:27 +01:00