Compare commits

..

13 Commits

Author SHA1 Message Date
claude code agent 227
f9b58a0e3d test(server): SSRF guardedFetch, decryptHeaders fail-open, yjs.util, tool-spec parity, storage delegation
guardedFetch blocks loopback/private/link-local/metadata IPs and never calls
fetch; decryptHeaders fails open (returns undefined, warns once, no blob leak).
yjs.util setYjsMark/removeYjsMarkByAttribute/updateYjsMarkAttribute on real
Y.Docs. SHARED_TOOL_SPECS<->in-app parity (name/desc/input-schema; a dropped or
renamed wiring fails). Replace the tautological storage.service spec with
driver-delegation checks across every public method.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 04:49:56 +03:00
claude code agent 227
388894c257 fix(client): stop findBreadcrumbPath mutating the live tree + tests
findBreadcrumbPath set node.name='Untitled' in place, mutating the shared
sidebar tree (treeData passed from resolveBreadcrumbNodes). Surface 'Untitled'
via a shallow copy on the returned chain only; input nodes stay untouched.
Add tests for the non-mutation invariant plus applyUpdateOne reducer,
formatRelativeTime buckets, and the pure tree mappers (sortPositionKeys,
pageToTreeNode).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 04:49:48 +03:00
claude code agent 227
e2b7ff10d9 test(mcp): media round-trip attrs, cookie parsing, anchor apply, recreate drift
Extract pure extractAuthTokenFromSetCookie from performLogin (behavior-identical)
so cookie parsing is unit-testable without a network login. Add round-trip
coverage for media attrs (width/height/align/drawio/escaping) the existing
suite omitted; applyAnchorInDoc selection/ambiguity/atom-break cases; and a
cross-copy drift guard proving the vendored editor-ext recreate-transform and
the @fellow npm copy used by diff.ts emit identical steps (apply(diff)==target).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 04:49:41 +03:00
claude code agent 227
683a62a547 test(editor-ext): cover recreateTransform invariant, table move/selection, unique-id
recreateTransform: apply(diff)==target round-trip across text/mark/structural
edits and complexSteps/wordDiffs options. moveRow/moveColumn drive real PM
tables (reorder preserves content, self-move/no-table -> false, CellSelection
on select). getSelectionRangeInColumn: single/multi-column + colspan + range
guard. addUniqueIdsToDoc: only configured types, nested targets, idempotency.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 04:49:31 +03:00
4a72ee1681 Merge pull request 'refactor(agent-roles-catalog): YAML catalog with block-scalar instructions (#229)' (#231) from feat/229-catalog-yaml into develop
Reviewed-on: #231
2026-06-29 01:20:40 +03:00
claude_code
82c41ccec6 ci: add timeout limits to CI jobs
Set explicit `timeout-minutes` for develop and test workflows to prevent jobs from running indefinitely and to cap resource usage. This includes a hard‑cap for the e2e‑server job, which can leak open handles and cause hangs.
2026-06-29 00:06:14 +03:00
claude_code
62eb7d082f test(ai-chat): stub sandboxStore.asSink in AiChatToolsService spec
The blob-sandbox feature (#243/#250) made AiChatToolsService.forUser()
eagerly call this.sandboxStore.asSink() while wiring the stash tool, but
the spec still passed an empty {} as the sandboxStore constructor arg.
That object has no asSink method, so all 19 tests in the suite failed in
CI with 'TypeError: this.sandboxStore.asSink is not a function'.

Replace the stale {} mock at all 4 constructor sites with a no-op sink
exposing asSink() -> { put, has, evict } (jest.fn()). These tests never
execute the stash tool, so a no-op sink is sufficient for forUser() to
wire successfully. Test-only change; production code is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 23:45:06 +03:00
6daa10db67 Merge pull request 'feat(#243): in-RAM blob sandbox (anonymous GET by UUID, TTL, ETag) + stash_page tool with image mirroring' (#250) from feat/243-blob-sandbox into develop
Reviewed-on: #250
2026-06-28 21:01:12 +03:00
claude_code
204cf9dfe7 test(sandbox): address PR #250 round-4 review — SSRF accept-path tests, MCP structuredContent (#243)
Mandatory (test-coverage):
- internal-file-urls.test: pin the SSRF/traversal ACCEPT path of
  resolveInternalFilePath (the sole guard for content-controlled `src`): an
  absolute/protocol-relative URL has its foreign host dropped and only an
  /api/files/ pathname survives (http://evil.com/api/files/x/y.png -> /files/x/y.png),
  while a host-dropped path that escapes /api/files/ (https://evil.com/api/auth/whoami)
  or a backslash-traversal (/api/files\..\auth\whoami) is rejected. Locks the
  behavior so a future prefix-only refactor cannot silently open a bypass.

Suggestions:
- index.ts: the stash_page MCP tool now returns structuredContent
  { uri, sha256, size, images } alongside the resource_link, so the MCP output
  matches the documented shape (clients get the blob's sha256/ETag and the
  mirror counts, not just the link). No outputSchema registered. Rebuilt build/.
- new stash-page-mcp-result.test: server round-trip via InMemoryTransport asserts
  both the resource_link and the structuredContent mirror.
- internal-file-urls.test: cover the new URL parse-failure catch branch
  (http://[ -> "Invalid internal file src").
- environment.service.spec: assert getPositiveIntEnv warns once per key and
  independently across keys (the invalidPositiveIntWarned dedup).

Tests: packages/mcp 383 pass; apps/server sandbox/environment/mcp 235 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 20:58:36 +03:00
claude_code
aff58646d1 refactor(sandbox): address PR #250 round-3 review — dead import, env validation, uuid validator, docs (#243)
Must-fix:
- mcp.module: drop the now-dead EnvironmentModule import (and its stale
  comment). McpService no longer injects EnvironmentService; EnvironmentModule
  is @Global and imported at the app root, so DI still resolves.

Stability:
- environment.service: route getSandboxTtlMs + the three SANDBOX_MAX_*_BYTES
  caps through a shared getPositiveIntEnv() helper that warns once per key and
  falls back to the default on a non-integer or <= 0 value (previously the byte
  caps did a bare parseInt, so SANDBOX_MAX_TOTAL_BYTES=0 made every stash_page
  fail against a 0-byte cap). TTL behavior is unchanged.

Simplification:
- sandbox.controller: replace the homemade UUID_RE with the project's shared
  `uuid` validator (import { validate as isValidUUID } from 'uuid'), matching
  the attachment routes; update the spec fixtures to valid v4 UUIDs.
- mcp.service: inline the single-caller one-liner buildSandboxConfig() to
  this.sandboxStore.asSink() at the wiring site.

Docs:
- CHANGELOG: add an [Unreleased] > Added entry for #243 (stash_page tool,
  anonymous GET /api/sb/:id, five SANDBOX_* env vars).
- AGENTS.md: note that GET /api/sb/:id is in the workspace-gate preHandler's
  excludedPaths and is fully tokenless, unlike /api/files/public/... which
  still resolves a workspace and needs an attachment JWT.

Tests: cap-getter validation (0/-5/abc -> default, valid -> parsed), updated
UUID fixtures. apps/server jest sandbox/environment/mcp: 233 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 20:21:31 +03:00
claude_code
8842bc8bf3 fix(sandbox): address PR #250 follow-up review — XSS hardening, eviction reconcile, doc sync (#243)
Security (must-fix):
- sandbox.controller: the anonymous GET /api/sb/:id response now sets
  X-Content-Type-Options: nosniff, a restrictive CSP, and Content-Disposition=
  attachment for any mime outside a raster-image allowlist (png/jpeg/gif/webp/
  avif). entry.mime is attacker-controlled, so an evil.svg/evil.html could
  otherwise execute script inline on the Docmost origin (stored XSS). Mirrors
  the public attachment route's hardening.

Stability:
- client.stashPage: reconcile mirrors AFTER the final document put, not only
  before it. The doc blob is the newest entry and FIFO eviction drops the
  oldest = this stash's own images, so the stored doc could reference an
  evicted blob (consumer 404) and over-report images.mirrored. A bounded loop
  now reverts doc-put-evicted mirrors, drops the stale doc blob, and re-puts
  until stable. Regenerated packages/mcp/build/.
- sandbox.controller: emit Cache-Control on the 304 branch too (ttlSeconds is
  computed before the conditional check).

Docs:
- Bump the MCP tool count 39 -> 40 across all READMEs and AGENTS.md (the
  registry now exposes exactly 40 tools).

Refactor:
- SandboxStore.asSink() centralizes the {put,has,evict} sink + uri<->id
  mapping; the embedded-MCP and in-app agent-tools wiring sites share it.

Tests:
- security headers (inline vs attachment, nosniff, CSP), 304 Cache-Control,
  putAndLink URL form, has()/remove(), asSink() round-trip, getSandboxPublicUrl
  (trailing-slash trim + APP_URL fallback), and a stash test where the doc put
  itself evicts a mirrored image.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 19:08:06 +03:00
claude_code
6eb335d5e3 fix(sandbox): address PR #250 review — SSRF guard, eviction safety, cleanup (#243)
Security:
- stash_page: reject path-traversal / percent-encoded srcs before the authed
  loopback fetch (resolveInternalFilePath), closing an SSRF/exfiltration hole
  where a crafted node.attrs.src could read an arbitrary internal GET endpoint
  into the anonymous sandbox.

Stability:
- stash_page: revert + recount mirrors FIFO-evicted by a later put in the same
  stash (no dangling sandbox refs, honest images.mirrored/failed); free image
  blobs if the final document put throws.
- Reject/clamp non-positive SANDBOX_TTL_MS to the 1h default (warn once).
- Log mirror failures unconditionally (console.warn, no blob bodies).

Cleanup / architecture:
- Remove dead expiresAt from SandboxPutResult.
- Centralize the /api/sb route in SANDBOX_ROUTE_SEGMENT/SANDBOX_API_PATH and
  move URL composition into SandboxStore.putAndLink; drop the duplicated sink
  closures and the now-unused EnvironmentService injection from McpService and
  AiChatToolsService.
- Un-export isInternalFileUrl; document the process-local (instance-bound)
  sandbox limitation in the tool description and .env.example.

Docs/tests:
- README/README.ru: 38 -> 39 tools + stash_page entry.
- Add traversal/normalize/recursion unit tests, stash self-eviction +
  doc-put-throw + empty/octet-stream mock tests, controller If-None-Match
  (wildcard/weak/list) + Cache-Control tests, and SANDBOX_TTL_MS validation
  tests. Regenerate packages/mcp/build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 18:02:46 +03:00
claude code agent 227
2fe4ca8537 feat(sandbox): in-RAM blob sandbox for out-of-band page transfer (#243)
Add an ephemeral, process-local blob store so the in-app agent (and the
embedded MCP) can hand a large page document and its images to an external
consumer WITHOUT routing the bytes through the model context or Docmost auth.

- SandboxStore (@Injectable singleton): Map<uuid,{buf,mime,sha256,expiresAt}>
  in RAM only. put() picks a per-blob cap by mime (image vs doc), enforces a
  total-bytes RAM guard with oldest-first eviction, and stamps a TTL; get()
  lazily expires. sha256 computed at put() doubles as the strong ETag. An
  unref'd sweep interval clears expired entries and is cleared on destroy.
- GET /api/sb/:uuid anonymous controller: serves raw bytes with Content-Type,
  Content-Length and ETag=sha256; 404 on missing/expired/non-UUID (anti-
  traversal), 304 on a matching If-None-Match. No tokens, no 401 — the
  capability is the unguessable UUID + short TTL + TLS. Auth-exempt the same
  way as /api/files/public (no JwtAuthGuard) plus an /api/sb entry in main.ts's
  workspace-resolution preHandler so a remote consumer with no workspace host
  is not rejected.
- stash_page tool in both layers (MCP resource_link + in-app {uri,size,sha256,
  images}). client.stashPage serializes the get_page_json shape, mirrors every
  INTERNAL file/image src (type-agnostic, covers drawio/excalidraw/video/file)
  into the sandbox under Docmost auth and rewrites src to the sandbox URL;
  external http(s) srcs are left untouched; dedup by src; a failed image fetch
  is counted, never aborts the doc.
- SANDBOX_PUBLIC_URL / SANDBOX_TTL_MS / SANDBOX_MAX_BYTES /
  SANDBOX_MAX_IMAGE_BYTES / SANDBOX_MAX_TOTAL_BYTES wired through the
  environment service + validation + .env.example.
- SandboxModule (@Global) provides the shared store to the controller,
  McpService and AiChatToolsService (same instance for put and get).

Tests: SandboxStore (round-trip, sha256, TTL lazy + sweep, caps, eviction),
SandboxController (200+ETag+CT+CL, 404 missing/expired/non-UUID, 304), and a
mock-HTTP stashPage test (mirror+rewrite internal, keep external, dedup, failed
image counted, returns only a link). Interoperates with the vvzvlad/habr-mcp
consumer's anonymous-GET + sha256-ETag + resource_link contract.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:13:11 +03:00
58 changed files with 4688 additions and 77 deletions

View File

@@ -124,6 +124,26 @@ MCP_DOCMOST_PASSWORD=
# MCP_TOKEN=
# MCP_SESSION_IDLE_MS=1800000
#
# BLOB SANDBOX (stash_page). An in-RAM, process-local store that hands large page
# content + images to an external consumer WITHOUT bloating the model context or
# requiring Docmost auth. The stash_page tool serializes a page, mirrors its
# internal images into the store, and returns ONLY a short anonymous URL; the
# consumer fetches blobs via `GET /api/sb/<uuid>` (no token — the capability is
# the unguessable UUID + short TTL + TLS). Blobs are RAM-only and cleared on
# restart. ETag = the blob's sha256 (integrity check).
# SANDBOX_PUBLIC_URL is the base used to build those URLs; it MUST be reachable
# by the consumer (do NOT use a loopback address if the consumer is remote).
# Defaults to APP_URL when unset.
# NOTE: the store is process-local — blobs live only on the instance that
# created them. Behind a multi-replica load balancer WITHOUT sticky sessions a
# consumer may hit a different instance and get a 404 (indistinguishable from an
# expired blob). Single-host deployments are unaffected.
# SANDBOX_PUBLIC_URL=https://docs.example.com
# SANDBOX_TTL_MS=3600000
# SANDBOX_MAX_BYTES=8388608
# SANDBOX_MAX_IMAGE_BYTES=20971520
# SANDBOX_MAX_TOTAL_BYTES=134217728
#
# AI-AGENT ATTRIBUTION (comments/pages written via MCP are badged as "AI"):
# attribution is driven by a per-user `is_agent` flag on the users row. There is
# NO admin UI/API for it — set it out-of-band with SQL. Use a DEDICATED service

View File

@@ -25,6 +25,7 @@ jobs:
build:
needs: test
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v4
@@ -65,6 +66,8 @@ jobs:
# deploy block.
e2e-server:
runs-on: ubuntu-latest
# Hard cap: the full-AppModule e2e leaks open handles and hung jest to the 6h max.
timeout-minutes: 15
env:
DATABASE_URL: postgresql://docmost:docmost@localhost:5432/docmost
REDIS_URL: redis://localhost:6379
@@ -123,6 +126,7 @@ jobs:
# a red run plus GitHub's email to the pusher is the notification mechanism.
e2e-mcp:
runs-on: ubuntu-latest
timeout-minutes: 20
env:
DATABASE_URL: postgresql://docmost:docmost@localhost:5432/docmost
REDIS_URL: redis://localhost:6379

View File

@@ -15,6 +15,7 @@ permissions:
jobs:
test:
runs-on: ubuntu-latest
timeout-minutes: 20
# Real Postgres + Redis so the server integration suite (`*.int-spec.ts`,
# behind `pnpm --filter server test:int`) runs in CI (red-team finding #7).
# Without it, cost-cap / FK-cascade / jsonb-round-trip / real-apply tests

View File

@@ -241,7 +241,7 @@ Migration files live in `apps/server/src/database/migrations/` and are named `YY
- **API server** — `dist/main` (`apps/server/src/main.ts`), the Fastify HTTP app (`AppModule`).
- **Collaboration server** — `dist/collaboration/server/collab-main` (`pnpm collab`), a Hocuspocus/Yjs WebSocket server (`apps/server/src/collaboration/`) handling real-time document editing, persistence, and page-history snapshots. It listens on `COLLAB_PORT` (default `3001`), separate from the API server's `PORT` (default `3000`), and shares state with the API server through Redis.
The API server is a Fastify app with a global `/api` prefix (`main.ts` excludes `robots.txt`, public share pages, and `mcp` from the prefix). A `preHandler` hook enforces that a resolved `workspaceId` exists for most `/api` routes (multi-tenant by hostname/subdomain via `DomainMiddleware`). Auth is JWT (cookie + bearer); authorization is **CASL** (`core/casl`) — every data access is scoped to the user's abilities.
The API server is a Fastify app with a global `/api` prefix (`main.ts` excludes `robots.txt`, public share pages, and `mcp` from the prefix). A `preHandler` hook enforces that a resolved `workspaceId` exists for most `/api` routes (multi-tenant by hostname/subdomain via `DomainMiddleware`). `GET /api/sb/:id` (the anonymous blob-sandbox read route) is listed in that preHandler's `excludedPaths`, so it is exempt from workspace resolution and carries no session auth at all (its capability is the unguessable UUID + TTL + TLS) — unlike `/api/files/public/...`, which still resolves a workspace and requires a workspace-bound attachment JWT. Auth is JWT (cookie + bearer); authorization is **CASL** (`core/casl`) — every data access is scoped to the user's abilities.
### Module structure (server)
`AppModule` wires integration modules (`integrations/*`: storage [local/S3/Azure], mail, queue [BullMQ on Redis], security, telemetry, throttle, `mcp`, `ai`) plus `CoreModule`, `DatabaseModule`, and `CollaborationModule`. `CoreModule` (`core/*`) holds the domain modules: `page`, `space`, `comment`, `workspace`, `user`, `auth`, `group`, `attachment`, `search`, `share`, `ai-chat`, etc. Each domain module follows NestJS controller → service → repo layering; DB repos live under `database/repos` and are injected app-wide from the global `DatabaseModule`.
@@ -254,7 +254,7 @@ The API server is a Fastify app with a global `/api` prefix (`main.ts` excludes
- **Redis** backs caching, the BullMQ queues, the WebSocket Socket.IO adapter, and collaboration sync.
### The two AI subsystems (the main fork additions)
1. **Embedded MCP server** (`integrations/mcp/` + `packages/mcp`). The standalone `@docmost/mcp` server (39 agent-native tools: per-block patch/insert/delete by id, scripted `(doc)=>doc` transforms with dry-run diff, table editing, version diff/restore, comments, images, shares) is bundled and served over HTTP at `/mcp`. It writes through Docmost's real-time-collaboration layer so concurrent human edits aren't clobbered. Each request authenticates **per-user** via the `Authorization` header — either HTTP Basic (`base64(email:password)`, the user's own Docmost login, validated through `AuthService`) or a Bearer access JWT (the user's `authToken`) — and the session acts under that user's permissions. `MCP_DOCMOST_EMAIL` / `MCP_DOCMOST_PASSWORD` are an **optional service-account fallback**, used only when a request carries neither Basic nor Bearer credentials (back-compat for CI/scripts). An admin enables MCP with a workspace toggle (Workspace settings → AI). Optionally protected by a shared `MCP_TOKEN`: when set, every `/mcp` request must carry a matching `X-MCP-Token` header (its own header, separate from `Authorization`, which now carries the per-user Basic/Bearer credentials). Note: this changed from the older `Authorization: Bearer <MCP_TOKEN>` scheme — see `.env.example` and the CHANGELOG Breaking Changes entry.
1. **Embedded MCP server** (`integrations/mcp/` + `packages/mcp`). The standalone `@docmost/mcp` server (40 agent-native tools: per-block patch/insert/delete by id, scripted `(doc)=>doc` transforms with dry-run diff, table editing, version diff/restore, comments, images, shares) is bundled and served over HTTP at `/mcp`. It writes through Docmost's real-time-collaboration layer so concurrent human edits aren't clobbered. Each request authenticates **per-user** via the `Authorization` header — either HTTP Basic (`base64(email:password)`, the user's own Docmost login, validated through `AuthService`) or a Bearer access JWT (the user's `authToken`) — and the session acts under that user's permissions. `MCP_DOCMOST_EMAIL` / `MCP_DOCMOST_PASSWORD` are an **optional service-account fallback**, used only when a request carries neither Basic nor Bearer credentials (back-compat for CI/scripts). An admin enables MCP with a workspace toggle (Workspace settings → AI). Optionally protected by a shared `MCP_TOKEN`: when set, every `/mcp` request must carry a matching `X-MCP-Token` header (its own header, separate from `Authorization`, which now carries the per-user Basic/Bearer credentials). Note: this changed from the older `Authorization: Bearer <MCP_TOKEN>` scheme — see `.env.example` and the CHANGELOG Breaking Changes entry.
2. **AI agent chat** (`core/ai-chat/` server + `apps/client/src/features/ai-chat/` client). A built-in agent over the wiki using the Vercel **AI SDK** (`ai`, `@ai-sdk/*`) against any OpenAI-compatible provider configured per workspace (`integrations/ai/` — credentials encrypted at rest via `integrations/crypto`, stored in `ai_provider_credentials`). Key pieces:
- `core/ai-chat/tools/` — the agent's ~40 read+write tools. Every tool runs under the **calling user's** CASL permissions via a per-user loopback access token (`docmost-client.loader.ts`), so the agent can never exceed what the user could do. Only **reversible** operations are exposed (page history + trash; no permanent delete). Agent edits get an "AI agent" provenance badge in page history (`20260616T130000-agent-provenance` migration).
- `core/ai-chat/embedding/` — RAG indexer + a BullMQ consumer on `AI_QUEUE` that embeds pages into `page_embeddings` (vector search), complementing Postgres full-text search. Pages are (re)indexed on edit; `AI_EMBEDDING_TIMEOUT_MS` bounds a hung embeddings endpoint.

View File

@@ -58,6 +58,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
append/prepend fragments, nor to COMMENT bodies — a comment may legitimately
contain a standalone footnote definition, which canonicalization would drop.
(#228)
- **Out-of-band page transfer via an in-RAM blob sandbox (`stash_page`).** A
new MCP tool serializes a whole page (its full ProseMirror JSON, with every
internal image/file mirrored) into an ephemeral in-RAM blob and returns only
a short anonymous URL, so a large page can be handed to an external consumer
without flooding the model context. Blobs are served by unguessable UUID over
a new anonymous `GET /api/sb/:id` route (strong sha256 ETag, short TTL,
`nosniff` + restrictive CSP + attachment disposition for non-image mimes) and
are RAM-only, bound to the instance that created them. Tunable via five
`SANDBOX_*` env vars (see `.env.example`). (#243)
### Changed

View File

@@ -34,7 +34,7 @@ The goal of the fork is a **100% open, AGPL-only build with no Enterprise-Editio
| --- | --- |
| **EE code removed** | Stripped all client and server Enterprise-Edition code; ships as a clean community/AGPL build with no license checks. |
| **Comment resolution** | Re-implemented from scratch as a community feature (resolve / re-open with Open/Resolved tabs). No EE code reused, available to anyone who can comment. |
| **Embedded MCP server** | A community MCP server (`@docmost/mcp`, 39 tools) is served over HTTP at `/mcp` — no enterprise license required. Replaces the removed license-gated EE MCP. |
| **Embedded MCP server** | A community MCP server (`@docmost/mcp`, 40 tools) is served over HTTP at `/mcp` — no enterprise license required. Replaces the removed license-gated EE MCP. |
| **AI agent chat** | Built-in AI agent chat over your wiki, written from scratch as a community feature — no enterprise license. The agent reads and edits pages on your behalf (scoped to your permissions), with full-text + vector (RAG) search and optional web access via external MCP servers. |
| **Rebranding** | App logo / name changed from *Docmost* to *Gitmost*. |
| **Compact page tree** | Default page-tree indentation reduced from 16px to 8px per nesting level. |
@@ -44,7 +44,7 @@ The goal of the fork is a **100% open, AGPL-only build with no Enterprise-Editio
### Embedded MCP server
Gitmost has **our own MCP server** — [docmost-mcp](https://github.com/vvzvlad/docmost-mcp),
which we wrote — **built directly into the app** and served at `/mcp`. It exposes **39
which we wrote — **built directly into the app** and served at `/mcp`. It exposes **40
agent-native tools**: surgical per-block edits (patch / insert / delete by id),
structure-preserving find/replace, scripted `(doc) => doc` transforms with a dry-run diff,
structured table editing, version history with diff / restore, comments, images and share
@@ -60,7 +60,7 @@ every little fix. And it needs no enterprise license.
| | **Gitmost `/mcp` (our docmost-mcp)** | Docmost's built-in MCP |
| --- | :---: | :---: |
| **Enterprise license** | Not required | Required |
| **Tools** | 39, agent-native | Coarse (read Markdown, page CRUD, replace whole page) |
| **Tools** | 40, agent-native | Coarse (read Markdown, page CRUD, replace whole page) |
| **Per-block edits / find-replace / scripted transforms** | ✅ | — |
| **Structured table editing, version diff / restore** | ✅ | — |
| **Comments, images, share links** | ✅ | — |

View File

@@ -33,7 +33,7 @@
| --- | --- |
| **Удалён EE-код** | Вырезан весь код Enterprise-редакции на клиенте и сервере; это чистая community/AGPL-сборка без лицензионных проверок. |
| **Резолв комментариев** | Переписан с нуля как community-функция (резолв / переоткрытие с вкладками «Открытые» / «Решённые»). EE-код не используется, доступно любому, кто может комментировать. |
| **Встроенный MCP-сервер** | Community MCP-сервер (`@docmost/mcp`, 39 инструментов) отдаётся по HTTP на `/mcp` — без enterprise-лицензии. Заменяет удалённый лицензируемый EE MCP. |
| **Встроенный MCP-сервер** | Community MCP-сервер (`@docmost/mcp`, 40 инструментов) отдаётся по HTTP на `/mcp` — без enterprise-лицензии. Заменяет удалённый лицензируемый EE MCP. |
| **Чат с AI-агентом** | Встроенный чат с AI-агентом по содержимому вики, написанный с нуля как community-функция — без enterprise-лицензии. Агент читает и редактирует страницы от вашего имени (в рамках ваших прав), с полнотекстовым + векторным (RAG) поиском и опциональным доступом в интернет через внешние MCP-серверы. |
| **Ребрендинг** | Логотип / название приложения изменены с *Docmost* на *Gitmost*. |
| **Компактное дерево страниц** | Отступ дерева страниц по умолчанию уменьшен с 16px до 8px на уровень вложенности. |
@@ -44,7 +44,7 @@
В Gitmost есть **наш собственный MCP-сервер** — [docmost-mcp](https://github.com/vvzvlad/docmost-mcp),
который мы написали сами, — **встроенный прямо в приложение** и доступный на `/mcp`. Он даёт
**39 agent-native инструментов**: точечное редактирование по блокам (patch / insert / delete
**40 agent-native инструментов**: точечное редактирование по блокам (patch / insert / delete
по id), find/replace с сохранением структуры, скриптовые трансформации `(doc) => doc` с
предпросмотром диффа, структурное редактирование таблиц, история версий с диффом /
восстановлением, комментарии, изображения и ссылки на шаринг — всё применяется через слой
@@ -60,7 +60,7 @@ real-time-коллаборации Docmost, поэтому запись нико
| | **`/mcp` в Gitmost (наш docmost-mcp)** | Родной MCP у Docmost |
| --- | :---: | :---: |
| **Enterprise-лицензия** | Не нужна | Нужна |
| **Инструменты** | 39, agent-native | Примитивные (Markdown, CRUD страниц, замена целиком) |
| **Инструменты** | 40, agent-native | Примитивные (Markdown, CRUD страниц, замена целиком) |
| **Правки по блокам / find-replace / скриптовые трансформации** | ✅ | — |
| **Структурное редактирование таблиц, дифф / восстановление версий** | ✅ | — |
| **Комментарии, изображения, ссылки на шаринг** | ✅ | — |

View File

@@ -1,5 +1,7 @@
import { describe, it, expect, vi, beforeEach, afterEach } from "vitest";
import i18n from "@/i18n.ts";
import {
formatRelativeTime,
getTimeGroup,
groupNotificationsByTime,
} from "@/features/notification/notification.utils.ts";
@@ -132,3 +134,59 @@ describe("groupNotificationsByTime", () => {
expect(groupNotificationsByTime([], labels)).toEqual([]);
});
});
describe("formatRelativeTime — relative buckets and absolute-date fallback", () => {
// Distinct fixed clock for the relative formatter (uses Date.now via `new
// Date()`), so the bucket boundaries are deterministic under fake timers.
const NOW = new Date("2026-06-15T12:00:00.000Z");
const MIN = 60_000;
beforeEach(() => {
vi.setSystemTime(NOW);
});
// ISO string `ms` milliseconds before NOW.
function ago(ms: number): string {
return new Date(NOW.getTime() - ms).toISOString();
}
it("returns the i18n 'now' label for anything under a minute", () => {
expect(formatRelativeTime(ago(0))).toBe(i18n.t("now"));
expect(formatRelativeTime(ago(59_000))).toBe(i18n.t("now"));
});
it("crosses into the minutes bucket exactly at 1 minute", () => {
expect(formatRelativeTime(ago(MIN - 1000))).toBe(i18n.t("now"));
expect(formatRelativeTime(ago(MIN))).toBe("1m");
expect(formatRelativeTime(ago(5 * MIN))).toBe("5m");
expect(formatRelativeTime(ago(59 * MIN))).toBe("59m");
});
it("crosses into the hours bucket exactly at 60 minutes", () => {
expect(formatRelativeTime(ago(60 * MIN - 1000))).toBe("59m");
expect(formatRelativeTime(ago(HOUR))).toBe("1h");
expect(formatRelativeTime(ago(23 * HOUR))).toBe("23h");
});
it("crosses into the days bucket exactly at 24 hours", () => {
expect(formatRelativeTime(ago(24 * HOUR - 1000))).toBe("23h");
expect(formatRelativeTime(ago(DAY))).toBe("1d");
expect(formatRelativeTime(ago(6 * DAY))).toBe("6d");
});
it("falls back to an absolute short date once >= 7 days old", () => {
// 6d -> still relative; 7d -> absolute date (no longer N[mhd], and equal to
// the localized short-date of the source timestamp).
expect(formatRelativeTime(ago(6 * DAY))).toBe("6d");
const sevenDaysAgo = ago(7 * DAY);
const result = formatRelativeTime(sevenDaysAgo);
expect(result).not.toMatch(/^\d+[mhd]$/);
expect(result).not.toBe(i18n.t("now"));
const expected = new Intl.DateTimeFormat(i18n.language, {
month: "short",
day: "numeric",
}).format(new Date(sevenDaysAgo));
expect(result).toBe(expected);
});
});

View File

@@ -0,0 +1,79 @@
import { describe, it, expect } from "vitest";
import { findBreadcrumbPath } from "./utils";
import type { SpaceTreeNode } from "@/features/page/tree/types.ts";
// findBreadcrumbPath walks the live, SHARED sidebar tree. The high-value
// invariant: when a node has no usable name it must surface "Untitled" ONLY on
// the returned breadcrumb chain via a shallow copy — never by mutating the input
// node (which would silently rename the node in the sidebar). Also covers normal
// ancestor-chain resolution, the not-found case, and nested children.
function node(id: string, over: Partial<SpaceTreeNode> = {}): SpaceTreeNode {
return {
id,
slugId: `slug-${id}`,
name: id.toUpperCase(),
icon: undefined,
position: "a0",
spaceId: "space-1",
parentPageId: null as unknown as string,
hasChildren: false,
children: [],
...over,
};
}
describe("findBreadcrumbPath", () => {
it("does NOT mutate the input tree when a node has an empty/whitespace name", () => {
// A whitespace-only-named node nested under a blank-named root.
const target = node("target", { name: " " });
const root = node("root", { name: "", hasChildren: true, children: [target] });
const tree = [root];
const result = findBreadcrumbPath(tree, "target");
expect(result).not.toBeNull();
// The RETURNED chain shows "Untitled" for both blank nodes.
expect(result!.map((n) => n.name)).toEqual(["Untitled", "Untitled"]);
// The original input nodes are untouched (still blank).
expect(root.name).toBe("");
expect(target.name).toBe(" ");
// The renamed breadcrumb entries are fresh copies, not the input objects.
expect(result![0]).not.toBe(root);
expect(result![1]).not.toBe(target);
});
it("returns the SAME node reference (no copy) when the name is non-empty", () => {
// No rename needed -> the node is passed through by reference (cheap path).
const target = node("target", { name: "Real Title" });
const result = findBreadcrumbPath([target], "target");
expect(result![0]).toBe(target);
expect(result![0].name).toBe("Real Title");
});
it("resolves the full ancestor chain ending at the target", () => {
const target = node("c");
const mid = node("b", { hasChildren: true, children: [target] });
const root = node("a", { hasChildren: true, children: [mid] });
const result = findBreadcrumbPath([root], "c");
expect(result!.map((n) => n.id)).toEqual(["a", "b", "c"]);
});
it("finds a target nested under a deeper sibling branch", () => {
// Two root branches; the target lives inside the second branch's child.
const target = node("deep");
const branch2 = node("r2", {
hasChildren: true,
children: [node("x"), node("y", { hasChildren: true, children: [target] })],
});
const branch1 = node("r1", { hasChildren: true, children: [node("z")] });
const result = findBreadcrumbPath([branch1, branch2], "deep");
expect(result!.map((n) => n.id)).toEqual(["r2", "y", "deep"]);
});
it("returns null when the page id is not present in the tree", () => {
const root = node("root", { hasChildren: true, children: [node("child")] });
expect(findBreadcrumbPath([root], "missing")).toBeNull();
expect(findBreadcrumbPath([], "anything")).toBeNull();
});
});

View File

@@ -8,6 +8,8 @@ import {
closeIds,
mergeRootTrees,
loadedOpenBranchIds,
sortPositionKeys,
pageToTreeNode,
} from "./utils";
import type { IPage } from "@/features/page/types/page.types.ts";
import type { SpaceTreeNode } from "@/features/page/tree/types.ts";
@@ -60,6 +62,82 @@ function treeNode(id: string, children: SpaceTreeNode[] = []): SpaceTreeNode {
};
}
describe("sortPositionKeys", () => {
it("orders items ascending by their fractional `position` string", () => {
const items = [
{ id: "c", position: "a5" },
{ id: "a", position: "a1" },
{ id: "b", position: "a3" },
];
expect(sortPositionKeys(items).map((i) => i.id)).toEqual(["a", "b", "c"]);
});
it("is a stable sort: equal positions keep their input order", () => {
const items = [
{ id: "x", position: "a1" },
{ id: "y", position: "a1" },
{ id: "z", position: "a1" },
];
expect(sortPositionKeys(items).map((i) => i.id)).toEqual(["x", "y", "z"]);
});
});
describe("pageToTreeNode", () => {
function pageRow(over: Partial<IPage> = {}): IPage {
return {
id: "p1",
slugId: "slug-p1",
title: "My Page",
icon: "📄",
position: "a1",
hasChildren: true,
spaceId: "space-1",
parentPageId: null as unknown as string,
...over,
} as IPage;
}
it("maps page.title -> node.name and copies the core fields", () => {
const node = pageToTreeNode(pageRow());
// The non-trivial transform: a page's `title` becomes the tree node's `name`.
expect(node.name).toBe("My Page");
expect(node.id).toBe("p1");
expect(node.slugId).toBe("slug-p1");
expect(node.icon).toBe("📄");
expect(node.position).toBe("a1");
expect(node.spaceId).toBe("space-1");
expect(node.hasChildren).toBe(true);
// Always materialized with an empty children array.
expect(node.children).toEqual([]);
});
it("derives canEdit from page.permissions.canEdit when the flat field is absent", () => {
const node = pageToTreeNode(
pageRow({ canEdit: undefined, permissions: { canEdit: true } } as Partial<IPage>),
);
expect(node.canEdit).toBe(true);
});
it("prefers the flat page.canEdit over permissions.canEdit", () => {
const node = pageToTreeNode(
pageRow({ canEdit: false, permissions: { canEdit: true } } as Partial<IPage>),
);
expect(node.canEdit).toBe(false);
});
it("carries temporaryExpiresAt straight off the page", () => {
const expiresAt = "2026-06-27T21:00:00.000Z";
expect(pageToTreeNode(pageRow({ temporaryExpiresAt: expiresAt })).temporaryExpiresAt).toBe(
expiresAt,
);
});
it("applies overrides on top of the mapped fields (e.g. optimistic blank name)", () => {
const node = pageToTreeNode(pageRow(), { name: "" });
expect(node.name).toBe("");
});
});
describe("buildTree", () => {
it("builds one node per unique page", () => {
const tree = buildTree([page("a", "a1"), page("b", "a2")]);

View File

@@ -70,18 +70,22 @@ export function findBreadcrumbPath(
path: SpaceTreeNode[] = [],
): SpaceTreeNode[] | null {
for (const node of tree) {
if (!node.name || node.name.trim() === "") {
node.name = "Untitled";
}
// Never mutate the input tree (it is the live, shared sidebar tree state).
// When a node has no usable name, surface "Untitled" via a shallow copy that
// only the returned breadcrumb chain sees — the source node stays untouched.
const displayNode: SpaceTreeNode =
!node.name || node.name.trim() === ""
? { ...node, name: "Untitled" }
: node;
if (node.id === pageId) {
return [...path, node];
return [...path, displayNode];
}
if (node.children) {
const newPath = findBreadcrumbPath(node.children, pageId, [
...path,
node,
displayNode,
]);
if (newPath) {
return newPath;

View File

@@ -3,6 +3,7 @@ import {
applyAddTreeNode,
applyMoveTreeNode,
applyDeleteTreeNode,
applyUpdateOne,
} from "./tree-socket-reducers";
import { treeModel } from "@/features/page/tree/model/tree-model";
import { SpaceTreeNode } from "@/features/page/tree/types.ts";
@@ -338,3 +339,76 @@ describe("applyAddTreeNode", () => {
expect(treeModel.find(next, "temp")?.temporaryExpiresAt).toBe(expiresAt);
});
});
describe("applyUpdateOne", () => {
// A loaded two-level tree so we can patch both a root and a nested node.
const buildTree = (): SpaceTreeNode[] => [
node("root", {
position: "a0",
name: "Root",
icon: "📁",
hasChildren: true,
children: [node("child", { position: "a1", parentPageId: "root", name: "Child", icon: "📄" })],
}),
];
// Build the UpdateEvent envelope; only `id`/`payload` matter to the reducer.
const ev = (id: string, payload: Record<string, unknown>) =>
({
operation: "updateOne",
spaceId: "space-1",
entity: ["pages"],
id,
payload,
}) as unknown as Parameters<typeof applyUpdateOne>[1];
it("applies a title-only update to the node's name (icon untouched)", () => {
const tree = buildTree();
const next = applyUpdateOne(tree, ev("child", { title: "Renamed" }));
const child = treeModel.find(next, "child");
expect(child?.name).toBe("Renamed");
// Icon is left as it was.
expect(child?.icon).toBe("📄");
});
it("applies an icon-only update to the node's icon (name untouched)", () => {
const tree = buildTree();
const next = applyUpdateOne(tree, ev("root", { icon: "🔥" }));
const root = treeModel.find(next, "root");
expect(root?.icon).toBe("🔥");
expect(root?.name).toBe("Root");
});
it("applies a combined title + icon update", () => {
const tree = buildTree();
const next = applyUpdateOne(tree, ev("child", { title: "Both", icon: "⭐" }));
const child = treeModel.find(next, "child");
expect(child?.name).toBe("Both");
expect(child?.icon).toBe("⭐");
});
it("returns prev UNCHANGED (same reference) when the id is not loaded", () => {
const tree = buildTree();
const next = applyUpdateOne(tree, ev("ghost", { title: "Nope" }));
expect(next).toBe(tree);
});
it("returns prev UNCHANGED (same reference) for a no-op payload (no title/icon)", () => {
// The node exists, but the payload carries neither title nor icon -> nothing
// to patch, so the reducer must hand back the same array reference.
const tree = buildTree();
const next = applyUpdateOne(tree, ev("child", {}));
expect(next).toBe(tree);
});
it("treats an explicit null icon/title as a value to apply (undefined check, not truthiness)", () => {
// The reducer guards on `!== undefined`, so a clearing null IS applied.
const tree = buildTree();
const next = applyUpdateOne(tree, ev("child", { title: "", icon: null }));
const child = treeModel.find(next, "child");
expect(child?.name).toBe("");
expect(child?.icon).toBeNull();
// And it did change something -> a fresh reference, not prev.
expect(next).not.toBe(tree);
});
});

View File

@@ -28,6 +28,7 @@ import { ClsModule } from 'nestjs-cls';
import { NoopAuditModule } from './integrations/audit/audit.module';
import { ThrottleModule } from './integrations/throttle/throttle.module';
import { McpModule } from './integrations/mcp/mcp.module';
import { SandboxModule } from './integrations/sandbox/sandbox.module';
import { AiModule } from './integrations/ai/ai.module';
import { AiChatModule } from './core/ai-chat/ai-chat.module';
@@ -89,6 +90,7 @@ try {
TelemetryModule,
ThrottleModule,
McpModule,
SandboxModule,
AiModule,
AiChatModule,
...enterpriseModules,

View File

@@ -0,0 +1,278 @@
import * as Y from 'yjs';
import { getSchema } from '@tiptap/core';
import {
initProseMirrorDoc,
absolutePositionToRelativePosition,
prosemirrorJSONToYDoc,
} from '@tiptap/y-tiptap';
import { tiptapExtensions } from './collaboration.util';
import {
setYjsMark,
removeYjsMarkByAttribute,
updateYjsMarkAttribute,
type YjsSelection,
} from './yjs.util';
/**
* Unit tests for the server-side Yjs mark helpers used by the collaboration
* handler to set/resolve/delete comment marks directly on the shared Y.Doc
* (collaboration.handler.ts: setCommentMark / resolveCommentMark).
*
* The fragment shape mirrors production exactly: a `default` XmlFragment whose
* children are block XmlElements (paragraph) holding XmlText runs. For setYjsMark
* the selection is a pair of Yjs RelativePosition JSONs (what the client sends);
* we synthesize them from known ProseMirror absolute positions via
* absolutePositionToRelativePosition so the marked range is deterministic.
*/
const schema = getSchema(tiptapExtensions);
// Build a real Y.Doc from ProseMirror JSON (same path the collab handler uses
// via TiptapTransformer) and return the doc + its `default` fragment.
function buildFromPm(pmJson: unknown) {
const ydoc = prosemirrorJSONToYDoc(
schema,
pmJson as never,
'default',
) as unknown as Y.Doc;
const fragment = ydoc.getXmlFragment('default');
return { ydoc, fragment };
}
// Make a YjsSelection (anchor/head RelativePosition JSON) for two ProseMirror
// absolute positions in `fragment`.
function selectionFor(
fragment: Y.XmlFragment,
anchorPos: number,
headPos: number,
): YjsSelection {
const { mapping } = initProseMirrorDoc(fragment, schema);
const anchor = absolutePositionToRelativePosition(
anchorPos,
fragment as never,
mapping,
);
const head = absolutePositionToRelativePosition(
headPos,
fragment as never,
mapping,
);
return {
anchor: Y.relativePositionToJSON(anchor),
head: Y.relativePositionToJSON(head),
};
}
// The XmlText run of the i-th top-level paragraph.
function paragraphText(fragment: Y.XmlFragment, index = 0): Y.XmlText {
const para = fragment.get(index) as Y.XmlElement;
return para.get(0) as Y.XmlText;
}
// --- raw fragment builder for the remove/update tests (no schema needed) ---
//
// removeYjsMarkByAttribute / updateYjsMarkAttribute only read item.toDelta() and
// call item.format(); they never touch the ProseMirror schema. Build the runs
// directly so we control which segment carries which comment attrs.
function buildWithComments(
segments: Array<{
text: string;
comment?: { commentId: string; resolved: boolean };
}>,
): { fragment: Y.XmlFragment; text: Y.XmlText } {
const ydoc = new Y.Doc();
const fragment = ydoc.getXmlFragment('default');
const para = new Y.XmlElement('paragraph');
fragment.insert(0, [para]);
const text = new Y.XmlText();
para.insert(0, [text]);
let offset = 0;
for (const seg of segments) {
text.insert(offset, seg.text);
if (seg.comment) {
text.format(offset, seg.text.length, { comment: seg.comment });
}
offset += seg.text.length;
}
return { fragment, text };
}
describe('setYjsMark', () => {
it('applies the mark over exactly the selected sub-range (PM pos 1..6 = "Hello")', () => {
const { ydoc, fragment } = buildFromPm({
type: 'doc',
content: [
{ type: 'paragraph', content: [{ type: 'text', text: 'Hello world' }] },
],
});
// PM pos 1 = start of the paragraph text; pos 6 = just after "Hello".
const sel = selectionFor(fragment, 1, 6);
setYjsMark(ydoc as never, fragment, sel, 'comment', {
commentId: 'c1',
resolved: false,
});
// The run splits: "Hello" carries the comment mark, " world" stays clean.
expect(paragraphText(fragment).toDelta()).toEqual([
{
insert: 'Hello',
attributes: { comment: { commentId: 'c1', resolved: false } },
},
{ insert: ' world' },
]);
});
it('normalizes a reversed selection (head before anchor) to the same range', () => {
const { ydoc, fragment } = buildFromPm({
type: 'doc',
content: [
{ type: 'paragraph', content: [{ type: 'text', text: 'Hello world' }] },
],
});
// anchor=6, head=1 — reversed; setYjsMark takes min/max so it marks "Hello".
const sel = selectionFor(fragment, 6, 1);
setYjsMark(ydoc as never, fragment, sel, 'comment', {
commentId: 'c2',
resolved: false,
});
expect(paragraphText(fragment).toDelta()).toEqual([
{
insert: 'Hello',
attributes: { comment: { commentId: 'c2', resolved: false } },
},
{ insert: ' world' },
]);
});
it('marks across two paragraphs (range spans an element boundary)', () => {
const { ydoc, fragment } = buildFromPm({
type: 'doc',
content: [
{ type: 'paragraph', content: [{ type: 'text', text: 'aaa' }] },
{ type: 'paragraph', content: [{ type: 'text', text: 'bbb' }] },
],
});
// PM positions: "aaa" = 1..4; the </p><p> boundary consumes pos 4 and 5, so
// "bbb" starts at pos 6 (chars at 6,7,8). Select pos 2 (inside "aaa") to pos
// 8 (after the second "b").
const sel = selectionFor(fragment, 2, 8);
setYjsMark(ydoc as never, fragment, sel, 'comment', {
commentId: 'c3',
resolved: false,
});
// First paragraph: "a" clean, "aa" marked.
expect(paragraphText(fragment, 0).toDelta()).toEqual([
{ insert: 'a' },
{
insert: 'aa',
attributes: { comment: { commentId: 'c3', resolved: false } },
},
]);
// Second paragraph: "bb" marked, "b" clean.
expect(paragraphText(fragment, 1).toDelta()).toEqual([
{
insert: 'bb',
attributes: { comment: { commentId: 'c3', resolved: false } },
},
{ insert: 'b' },
]);
});
});
describe('removeYjsMarkByAttribute', () => {
it('removes only the run whose attribute value matches, leaving others', () => {
const { fragment, text } = buildWithComments([
{ text: 'AAA', comment: { commentId: 'c1', resolved: false } },
{ text: 'BBB', comment: { commentId: 'c2', resolved: false } },
]);
removeYjsMarkByAttribute(fragment, 'comment', 'commentId', 'c1');
// c1's run loses the mark; c2's run is untouched.
expect(text.toDelta()).toEqual([
{ insert: 'AAA' },
{
insert: 'BBB',
attributes: { comment: { commentId: 'c2', resolved: false } },
},
]);
});
it('does nothing when no run carries the requested value (no-match branch)', () => {
const { fragment, text } = buildWithComments([
{ text: 'AAA', comment: { commentId: 'c1', resolved: false } },
]);
const before = text.toDelta();
removeYjsMarkByAttribute(fragment, 'comment', 'commentId', 'does-not-exist');
expect(text.toDelta()).toEqual(before);
});
it('leaves a different mark type alone', () => {
// A run carrying only `bold` must survive a comment removal pass.
const ydoc = new Y.Doc();
const fragment = ydoc.getXmlFragment('default');
const para = new Y.XmlElement('paragraph');
fragment.insert(0, [para]);
const text = new Y.XmlText();
para.insert(0, [text]);
text.insert(0, 'XYZ');
text.format(0, 3, { bold: true });
removeYjsMarkByAttribute(fragment, 'comment', 'commentId', 'c1');
expect(text.toDelta()).toEqual([
{ insert: 'XYZ', attributes: { bold: true } },
]);
});
});
describe('updateYjsMarkAttribute', () => {
it('merges new attributes into the matching run, preserving the rest', () => {
const { fragment, text } = buildWithComments([
{ text: 'AAA', comment: { commentId: 'c1', resolved: false } },
{ text: 'BBB', comment: { commentId: 'c2', resolved: false } },
]);
updateYjsMarkAttribute(
fragment,
'comment',
{ name: 'commentId', value: 'c1' },
{ resolved: true },
);
// c1's run flips resolved=true (commentId preserved via merge); c2 untouched.
expect(text.toDelta()).toEqual([
{
insert: 'AAA',
attributes: { comment: { commentId: 'c1', resolved: true } },
},
{
insert: 'BBB',
attributes: { comment: { commentId: 'c2', resolved: false } },
},
]);
});
it('does nothing when no run matches (no-match branch)', () => {
const { fragment, text } = buildWithComments([
{ text: 'AAA', comment: { commentId: 'c1', resolved: false } },
]);
const before = text.toDelta();
updateYjsMarkAttribute(
fragment,
'comment',
{ name: 'commentId', value: 'nope' },
{ resolved: true },
);
expect(text.toDelta()).toEqual(before);
});
});

View File

@@ -0,0 +1,166 @@
import { McpClientsService } from './mcp-clients.service';
/**
* Unit tests for the two security-critical surfaces of McpClientsService that the
* sibling specs (ssrf-guard / validate-resolved-addresses / lease) do NOT cover:
*
* 1. `decryptHeaders` (private) — FAIL-OPEN behavior. A decrypt/parse failure
* (e.g. APP_SECRET rotated, tampered blob) must NEVER throw and must NEVER
* log the blob: it returns `undefined` so the connect proceeds WITHOUT the
* now-unreadable auth headers (which then 401s and the server is skipped),
* rather than crashing the whole turn.
*
* 2. `this.guardedFetch` (private, bound to the SSRF-pinned dispatcher) — the
* per-request DNS-rebinding guard. A blocked host (private/loopback/metadata
* IP literal, or an unparseable URL) must REJECT before any socket is opened;
* a public host is allowed through to the real `fetch` with the pinned
* dispatcher attached.
*
* No network and no DB: the repo + secretBox deps are stubbed, and global `fetch`
* is mocked for the single allow-path assertion.
*/
// Build the service with a SecretBoxService stub whose decryptSecret is supplied
// per-test. The repo dep is unused by the methods under test.
function buildService(decryptSecret: (blob: string) => string) {
const secretBox = { decryptSecret: jest.fn(decryptSecret) };
const service = new McpClientsService({} as never, secretBox as never);
return { service, secretBox };
}
describe('McpClientsService.decryptHeaders', () => {
// Reach the private method via the as-any pattern common in these NestJS specs.
const callDecrypt = (
service: McpClientsService,
blob: string | null,
): Record<string, string> | undefined =>
(
service as unknown as {
decryptHeaders: (b: string | null) => Record<string, string> | undefined;
}
).decryptHeaders(blob);
it('returns undefined for a null blob without decrypting', () => {
const { service, secretBox } = buildService(() => '{}');
expect(callDecrypt(service, null)).toBeUndefined();
expect(secretBox.decryptSecret).not.toHaveBeenCalled();
});
it('decrypts a valid blob and keeps only string-valued headers', () => {
const { service } = buildService(() =>
JSON.stringify({
Authorization: 'Bearer abc',
'X-Api-Key': 'k',
// Non-string values must be dropped, not coerced.
count: 5,
flag: true,
nested: { a: 1 },
}),
);
expect(callDecrypt(service, 'cipher')).toEqual({
Authorization: 'Bearer abc',
'X-Api-Key': 'k',
});
});
it('returns undefined when the decrypted object has no string headers', () => {
const { service } = buildService(() => JSON.stringify({ count: 5 }));
// No usable headers -> undefined (connect with no auth header), not {}.
expect(callDecrypt(service, 'cipher')).toBeUndefined();
});
it('FAILS OPEN: a decrypt error returns undefined instead of throwing', () => {
const { service } = buildService(() => {
throw new Error('Failed to decrypt secret — APP_SECRET may have changed');
});
const warnSpy = jest
.spyOn(
(service as unknown as { logger: { warn: (...a: unknown[]) => void } })
.logger,
'warn',
)
.mockImplementation(() => undefined);
let result: unknown;
expect(() => {
result = callDecrypt(service, 'tampered-blob');
}).not.toThrow();
expect(result).toBeUndefined();
// It warns (so ops sees degradation) but never logs the blob itself.
expect(warnSpy).toHaveBeenCalledTimes(1);
expect(String(warnSpy.mock.calls[0]?.[0])).not.toContain('tampered-blob');
});
it('FAILS OPEN: malformed JSON (decrypts to non-JSON) returns undefined', () => {
const { service } = buildService(() => 'not-json{');
jest
.spyOn(
(service as unknown as { logger: { warn: (...a: unknown[]) => void } })
.logger,
'warn',
)
.mockImplementation(() => undefined);
expect(callDecrypt(service, 'cipher')).toBeUndefined();
});
});
describe('McpClientsService.guardedFetch (SSRF per-request guard)', () => {
// The bound guardedFetch closure lives on the instance as a private field.
const guardedFetchOf = (service: McpClientsService) =>
(service as unknown as { guardedFetch: typeof fetch }).guardedFetch;
let fetchSpy: jest.SpiedFunction<typeof fetch>;
beforeEach(() => {
// Any reachable real fetch would be a network call; assert per-test that the
// blocked paths never reach it, and stub a Response for the allow path.
fetchSpy = jest
.spyOn(global, 'fetch')
.mockResolvedValue(new Response('ok', { status: 200 }));
});
afterEach(() => {
jest.restoreAllMocks();
});
const blocked: Array<[string, string]> = [
['loopback IPv4', 'http://127.0.0.1/mcp'],
['private 10/8', 'http://10.0.0.5/mcp'],
['private 192.168/16', 'http://192.168.1.1/mcp'],
['cloud metadata link-local', 'http://169.254.169.254/latest/meta-data/'],
['loopback IPv6 (bracketed)', 'http://[::1]:8080/mcp'],
];
it.each(blocked)(
'rejects a request to %s without opening a socket',
async (_label, url) => {
const { service } = buildService(() => '{}');
await expect(guardedFetchOf(service)(url)).rejects.toThrow(
/blocked request/,
);
expect(fetchSpy).not.toHaveBeenCalled();
},
);
it('rejects an unparseable URL as a blocked request', async () => {
const { service } = buildService(() => '{}');
await expect(
guardedFetchOf(service)('::: not a url :::'),
).rejects.toThrow('blocked request: invalid URL');
expect(fetchSpy).not.toHaveBeenCalled();
});
it('allows a public IP literal and forwards through the pinned dispatcher', async () => {
const { service } = buildService(() => '{}');
const res = await guardedFetchOf(service)('http://8.8.8.8/mcp');
expect(res.status).toBe(200);
expect(fetchSpy).toHaveBeenCalledTimes(1);
// The init MUST carry the SSRF-pinned undici dispatcher (the rebinding pin);
// dropping it would let undici do a second, unchecked DNS resolution.
const init = fetchSpy.mock.calls[0][1] as RequestInit & {
dispatcher?: unknown;
};
expect(init.dispatcher).toBeDefined();
});
});

View File

@@ -63,6 +63,12 @@ describe('AiChatToolsService deletePage guardrail (H4)', () => {
{} as never,
{} as never,
{} as never,
// sandboxStore: forUser() eagerly calls asSink() to wire the stash tool,
// even though these tests never execute it — return a no-op sink so the
// tool wiring in forUser() succeeds.
{
asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }),
} as never,
);
});
@@ -175,6 +181,12 @@ describe('AiChatToolsService expanded toolset guardrails', () => {
{} as never,
{} as never,
{} as never,
// sandboxStore: forUser() eagerly calls asSink() to wire the stash tool,
// even though these tests never execute it — return a no-op sink so the
// tool wiring in forUser() succeeds.
{
asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }),
} as never,
);
});
@@ -290,6 +302,12 @@ describe('AiChatToolsService node-arg JSON-string coercion', () => {
{} as never,
{} as never,
{} as never,
// sandboxStore: forUser() eagerly calls asSink() to wire the stash tool,
// even though these tests never execute it — return a no-op sink so the
// tool wiring in forUser() succeeds.
{
asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }),
} as never,
);
});
@@ -440,6 +458,12 @@ describe('AiChatToolsService model-friendly input validation (#190)', () => {
{} as never,
{} as never,
{} as never,
// sandboxStore: forUser() eagerly calls asSink() to wire the stash tool,
// even though these tests never execute it — return a no-op sink so the
// tool wiring in forUser() succeeds.
{
asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }),
} as never,
);
});

View File

@@ -16,6 +16,7 @@ import {
import { resolveCurrentPageResult } from './current-page.util';
import { parseNodeArg } from './parse-node-arg';
import { modelFriendlyInput } from './model-friendly-input';
import { SandboxStore } from '../../../integrations/sandbox/sandbox.store';
/**
* Per-user, per-request adapter that exposes Docmost READ operations to the
@@ -41,6 +42,8 @@ export class AiChatToolsService {
private readonly pageEmbeddingRepo: PageEmbeddingRepo,
private readonly spaceMemberRepo: SpaceMemberRepo,
private readonly pagePermissionRepo: PagePermissionRepo,
// Shared singleton in-RAM blob store backing the stash tool.
private readonly sandboxStore: SandboxStore,
) {}
async forUser(
@@ -86,11 +89,17 @@ export class AiChatToolsService {
aiChatId,
});
// Bind the stash tool to the shared in-RAM SandboxStore. The store owns the
// anonymous-URL composition (putAndLink) and the live/evict probes the MCP
// package needs to keep its mirror counts honest under FIFO eviction (the
// package never touches env or the store). asSink() centralizes the uri↔id
// mapping next to putAndLink, shared with the embedded-MCP wiring site.
const { DocmostClient, sharedToolSpecs } = await loadDocmostMcp();
const client: DocmostClientLike = new DocmostClient({
apiUrl,
getToken,
getCollabToken,
sandbox: this.sandboxStore.asSink(),
});
// Build an ai-SDK tool from a shared, zod-agnostic spec. The spec owns the
@@ -625,6 +634,14 @@ export class AiChatToolsService {
async ({ pageId, edits }) => await client.editPageText(pageId, edits),
),
// Returns ONLY the short link object — never the document body — so a
// large page can be handed to an external consumer without bloating
// context.
stashPage: sharedTool(
sharedToolSpecs.stashPage,
async ({ pageId }) => await client.stashPage(pageId),
),
patchNode: tool({
description:
'Replace a single content block (by id) with a new ProseMirror ' +

View File

@@ -154,6 +154,14 @@ export interface DocmostClientLike {
commentId: string,
resolved: boolean,
): Promise<Record<string, unknown>>;
// Serialize a page + mirror its internal images into the blob sandbox; returns
// ONLY a short anonymous URL (the body never enters the model context).
stashPage(pageId: string): Promise<{
uri: string;
sha256: string;
size: number;
images: { mirrored: number; failed: number };
}>;
}
export type DocmostClientConfig = {
@@ -161,6 +169,18 @@ export type DocmostClientConfig = {
getToken: () => Promise<string>;
// Provenance collab-token provider for content mutations (signed agent claim).
getCollabToken?: () => Promise<string>;
// Optional blob-sandbox sink for the stash tool. `put` stores a blob in the
// host's in-RAM SandboxStore and returns the anonymous read URL + integrity.
// The optional `has`/`evict` probes let stashPage keep its mirror counts
// honest under the store's FIFO eviction (mirror of the package's sink type).
sandbox?: {
put: (
buf: Buffer,
mime: string,
) => { uri: string; sha256: string; size: number };
has?: (uri: string) => boolean;
evict?: (uri: string) => void;
};
};
export interface DocmostClientCtor {

View File

@@ -0,0 +1,124 @@
import { z } from 'zod';
import { AiChatToolsService } from './ai-chat-tools.service';
import * as loader from './docmost-client.loader';
import type { DocmostClientLike } from './docmost-client.loader';
// The real zod-agnostic registry, imported from source so the contract is checked
// against exactly what the @docmost/mcp package ships (no hand-stub).
import { SHARED_TOOL_SPECS } from '../../../../../../packages/mcp/src/tool-specs';
/**
* CONTRACT: SHARED_TOOL_SPECS <-> in-app tool wiring parity.
*
* `packages/mcp/src/tool-specs.ts` is the single source of truth for the tools
* that are intentionally IDENTICAL across the standalone MCP server (zod v3) and
* the in-app AI-SDK service (zod v4). The in-app service builds each one via
* `sharedTool(sharedToolSpecs.<key>, execute)`, keyed by the spec's `inAppKey`.
*
* This test fails the build if a spec is added to the registry but never wired
* in-app, if an `inAppKey` is renamed without updating the service, if the
* description drifts between the registry and the exposed tool, if the
* snake_case `mcpName` <-> camelCase `inAppKey` convention is broken, or if the
* exposed tool's input-schema keys diverge from the spec's `buildShape`.
*
* It does NOT need @docmost/mcp built: the registry is imported from TS source,
* and the ESM loader is mocked so `forUser()` never dynamically imports the
* package.
*/
describe('SHARED_TOOL_SPECS contract parity', () => {
// Empty fake client: no tool is executed here — every assertion is on tool
// presence / metadata / schema, so the client methods are never called.
const fakeClient: Partial<DocmostClientLike> = {};
const tokenServiceStub = {
generateAccessToken: jest.fn().mockResolvedValue('access-token'),
generateCollabToken: jest.fn().mockResolvedValue('collab-token'),
};
let tools: Record<string, unknown>;
beforeAll(async () => {
jest.spyOn(loader, 'loadDocmostMcp').mockResolvedValue({
DocmostClient: function () {
return fakeClient as DocmostClientLike;
} as unknown as loader.DocmostClientCtor,
// Feed the service the SAME registry this test asserts against.
sharedToolSpecs: SHARED_TOOL_SPECS as unknown as Record<
string,
loader.SharedToolSpec
>,
});
const service = new AiChatToolsService(
tokenServiceStub as never,
{} as never,
{} as never,
{} as never,
{} as never,
{ asSink: () => ({ put: jest.fn(), has: jest.fn(), evict: jest.fn() }) } as never,
);
tools = (await service.forUser(
{ id: 'user-1', email: 'u@example.com', workspaceId: 'ws-1' } as never,
'session-1',
'ws-1',
'chat-1',
)) as unknown as Record<string, unknown>;
});
afterAll(() => jest.restoreAllMocks());
// camelCase -> snake_case, matching the registry's mcpName convention.
const toSnake = (s: string) =>
s.replace(/[A-Z]/g, (c) => `_${c.toLowerCase()}`);
// Type as the (optional-buildShape) SharedToolSpec; the `satisfies` literal
// above otherwise narrows to a union where some members lack buildShape.
const specEntries = Object.entries(SHARED_TOOL_SPECS) as Array<
[string, loader.SharedToolSpec]
>;
// Sanity: the registry is non-empty, so the per-spec table below is not vacuous.
it('registry is non-empty', () => {
expect(specEntries.length).toBeGreaterThan(0);
});
describe.each(specEntries)('spec "%s"', (registryKey, spec) => {
it('registry key equals its inAppKey', () => {
// The service indexes the registry by property name; a key != inAppKey
// would wire the wrong (or no) tool.
expect(spec.inAppKey).toBe(registryKey);
});
it('mcpName is the snake_case form of inAppKey', () => {
expect(spec.mcpName).toBe(toSnake(spec.inAppKey));
});
it('is exposed in-app under its inAppKey', () => {
// Fails if a spec is added to the registry but never wired in forUser().
expect(tools[spec.inAppKey]).toBeDefined();
});
it("exposed tool's description matches the registry description", () => {
const tool = tools[spec.inAppKey] as { description: string };
expect(tool.description).toBe(spec.description);
});
it("exposed tool's input-schema keys match buildShape (incl. required)", () => {
const tool = tools[spec.inAppKey] as {
inputSchema: { jsonSchema: { properties?: Record<string, unknown>; required?: string[] } };
};
const json = tool.inputSchema.jsonSchema;
const actualKeys = Object.keys(json.properties ?? {}).sort();
// Derive the spec's declared shape with THIS layer's zod (v4) — the same
// call the service makes — then compare key sets and required-ness.
const shape = spec.buildShape ? spec.buildShape(z) : {};
const expectedKeys = Object.keys(shape).sort();
expect(actualKeys).toEqual(expectedKeys);
// A non-.optional() field must surface as required in the advertised schema.
const expectedRequired = Object.entries(shape)
.filter(([, field]) => !(field as z.ZodTypeAny).isOptional?.())
.map(([k]) => k)
.sort();
expect((json.required ?? []).slice().sort()).toEqual(expectedRequired);
});
});
});

View File

@@ -14,4 +14,148 @@ describe('EnvironmentService', () => {
it('should be defined', () => {
expect(service).toBeDefined();
});
describe('getSandboxTtlMs', () => {
// ConfigService stub: get(key, def) returns the configured value for the key
// (falling back to def), matching the @nestjs/config contract the service
// calls with (key, default).
const build = (sandboxTtl?: string) =>
new EnvironmentService({
get: (key: string, def?: string) =>
key === 'SANDBOX_TTL_MS' ? (sandboxTtl ?? def) : def,
} as any);
it.each(['0', '-5', 'abc'])(
'falls back to the 3600000 default for invalid value %s',
(value) => {
expect(build(value).getSandboxTtlMs()).toBe(3_600_000);
},
);
it('returns the parsed value for a valid positive integer', () => {
expect(build('120000').getSandboxTtlMs()).toBe(120_000);
});
it('uses the 3600000 default when SANDBOX_TTL_MS is unset', () => {
expect(build(undefined).getSandboxTtlMs()).toBe(3_600_000);
});
});
// The three byte caps share the same getPositiveIntEnv() helper as the TTL,
// so a non-integer / non-positive value ('0'/'-5'/'abc') falls back to the
// documented default and a valid positive integer is returned parsed. Note
// parseInt truncates '1.5' -> 1 (a valid positive integer), so that value is
// accepted, not rejected — same as the pre-existing TTL getter.
describe.each([
{
name: 'getSandboxMaxBytes',
key: 'SANDBOX_MAX_BYTES',
def: 8_388_608,
getter: (s: EnvironmentService) => s.getSandboxMaxBytes(),
},
{
name: 'getSandboxMaxImageBytes',
key: 'SANDBOX_MAX_IMAGE_BYTES',
def: 20_971_520,
getter: (s: EnvironmentService) => s.getSandboxMaxImageBytes(),
},
{
name: 'getSandboxMaxTotalBytes',
key: 'SANDBOX_MAX_TOTAL_BYTES',
def: 134_217_728,
getter: (s: EnvironmentService) => s.getSandboxMaxTotalBytes(),
},
])('$name', ({ key, def, getter }) => {
// ConfigService stub: get(k, d) returns the configured value for THIS cap's
// key (falling back to d), and the default for every other key.
const build = (value?: string) =>
new EnvironmentService({
get: (k: string, d?: string) =>
k === key ? (value ?? d) : d,
} as any);
it.each(['0', '-5', 'abc'])(
`falls back to the ${def} default for invalid value %s`,
(value) => {
expect(getter(build(value))).toBe(def);
},
);
it('returns the parsed value for a valid positive integer', () => {
expect(getter(build('4096'))).toBe(4096);
});
it('truncates a non-integer like "1.5" to 1 via parseInt (not rejected)', () => {
expect(getter(build('1.5'))).toBe(1);
});
it(`uses the ${def} default when the env is unset`, () => {
expect(getter(build(undefined))).toBe(def);
});
});
// getPositiveIntEnv keeps a one-shot `invalidPositiveIntWarned` set so a bad
// value is logged ONCE per key (not on every getter call, which the sandbox
// hits per-put). These tests pin that dedup so a regression to per-call logging
// would fail loudly.
describe('invalid-value warn dedup', () => {
it('warns only once per key across repeated getter calls', () => {
const service = new EnvironmentService({
get: (k: string, d?: string) =>
k === 'SANDBOX_MAX_TOTAL_BYTES' ? '-5' : d,
} as any);
const warnSpy = jest
.spyOn((service as any).logger, 'warn')
.mockImplementation(() => undefined);
service.getSandboxMaxTotalBytes();
service.getSandboxMaxTotalBytes();
expect(warnSpy).toHaveBeenCalledTimes(1);
});
it('warns independently per key (dedup is per-key, not global)', () => {
// Two DIFFERENT SANDBOX_* keys are both invalid -> each warns once, so two
// warns total. This proves the dedup set is keyed, not a single global flag.
const service = new EnvironmentService({
get: (k: string, d?: string) =>
k === 'SANDBOX_MAX_BYTES' || k === 'SANDBOX_MAX_TOTAL_BYTES'
? '-5'
: d,
} as any);
const warnSpy = jest
.spyOn((service as any).logger, 'warn')
.mockImplementation(() => undefined);
service.getSandboxMaxBytes();
service.getSandboxMaxTotalBytes();
expect(warnSpy).toHaveBeenCalledTimes(2);
});
});
describe('getSandboxPublicUrl', () => {
// Stub that resolves BOTH keys the public-url logic consults.
const build = (vals: { sandboxUrl?: string; appUrl?: string }) =>
new EnvironmentService({
get: (key: string, def?: string) =>
key === 'SANDBOX_PUBLIC_URL'
? (vals.sandboxUrl ?? def)
: key === 'APP_URL'
? (vals.appUrl ?? def)
: def,
} as any);
it('uses SANDBOX_PUBLIC_URL and trims a trailing slash', () => {
expect(
build({ sandboxUrl: 'https://docs.example.com/' }).getSandboxPublicUrl(),
).toBe('https://docs.example.com');
});
it('falls back to APP_URL (origin) when SANDBOX_PUBLIC_URL is unset', () => {
expect(
build({ appUrl: 'https://app.example.com' }).getSandboxPublicUrl(),
).toBe('https://app.example.com');
});
});
});

View File

@@ -1,9 +1,15 @@
import { Injectable } from '@nestjs/common';
import { Injectable, Logger } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import ms, { StringValue } from 'ms';
@Injectable()
export class EnvironmentService {
private readonly logger = new Logger(EnvironmentService.name);
// Env keys already warned about for an invalid value (one-shot per key, so a
// bad SANDBOX_* value is not logged on every blob put). Mirrors the original
// sandboxTtlWarned guard, generalized across the TTL + the three byte caps.
private readonly invalidPositiveIntWarned = new Set<string>();
constructor(private configService: ConfigService) {}
getNodeEnv(): string {
@@ -332,4 +338,63 @@ export class EnvironmentService {
.map((o) => o.trim())
.filter(Boolean);
}
// --- Blob sandbox (in-RAM ephemeral blob transfer; see SandboxModule) ---
// Base URL the sandbox `uri` is built from. It MUST be reachable over the
// network by the external consumer that fetches the blobs (not a loopback
// address if that consumer is remote). Falls back to APP_URL when unset so a
// single-host deployment works out of the box; set it explicitly when the
// consumer lives on another host.
getSandboxPublicUrl(): string {
const raw =
this.configService.get<string>('SANDBOX_PUBLIC_URL') || this.getAppUrl();
// Drop any trailing slash so `${base}/api/sb/${id}` never doubles up.
return raw.replace(/\/+$/, '');
}
// Parse a REQUIRED positive-integer env (TTL in ms or a byte cap). A
// non-integer or <= 0 value would break the sandbox silently (instant expiry,
// or every put failing against a 0-byte cap), so warn once and fall back to
// the default instead. Blob bodies are never logged.
private getPositiveIntEnv(key: string, def: number): number {
const parsed = parseInt(
this.configService.get<string>(key, String(def)),
10,
);
if (!Number.isInteger(parsed) || parsed <= 0) {
if (!this.invalidPositiveIntWarned.has(key)) {
this.invalidPositiveIntWarned.add(key);
this.logger.warn(
`Invalid ${key} (must be a positive integer); falling back to the ${def} default`,
);
}
return def;
}
return parsed;
}
// Blob time-to-live. Default 1h. The unguessable UUID + this short TTL + TLS
// are the whole capability model (no tokens). A non-positive or non-integer
// value would make every blob expire instantly (silent 404s), so reject it and
// fall back to the 1h default (warned about once to avoid per-put log spam).
getSandboxTtlMs(): number {
return this.getPositiveIntEnv('SANDBOX_TTL_MS', 3_600_000);
}
// Per-blob cap for non-image blobs (the serialized document). Default 8 MiB.
getSandboxMaxBytes(): number {
return this.getPositiveIntEnv('SANDBOX_MAX_BYTES', 8_388_608);
}
// Per-blob cap for mirrored image blobs. Default 20 MiB.
getSandboxMaxImageBytes(): number {
return this.getPositiveIntEnv('SANDBOX_MAX_IMAGE_BYTES', 20_971_520);
}
// RAM guard: total bytes the whole store may hold. Default 128 MiB. On
// overflow the store evicts oldest entries to make room.
getSandboxMaxTotalBytes(): number {
return this.getPositiveIntEnv('SANDBOX_MAX_TOTAL_BYTES', 134_217_728);
}
}

View File

@@ -2,6 +2,7 @@ import {
IsIn,
IsNotEmpty,
IsNotIn,
IsNumberString,
IsOptional,
IsString,
IsUrl,
@@ -170,6 +171,35 @@ export class EnvironmentVariables {
},
)
CLICKHOUSE_URL: string;
// --- Blob sandbox (in-RAM ephemeral blob transfer; see SandboxModule) ---
@IsOptional()
@ValidateIf((obj) => obj.SANDBOX_PUBLIC_URL != '' && obj.SANDBOX_PUBLIC_URL != null)
@IsUrl(
{ protocols: ['http', 'https'], require_tld: false },
{
message:
'SANDBOX_PUBLIC_URL must be a valid http(s) URL reachable by the external blob consumer',
},
)
SANDBOX_PUBLIC_URL: string;
@IsOptional()
@IsNumberString({}, { message: 'SANDBOX_TTL_MS must be an integer (milliseconds)' })
SANDBOX_TTL_MS: string;
@IsOptional()
@IsNumberString({}, { message: 'SANDBOX_MAX_BYTES must be an integer (bytes)' })
SANDBOX_MAX_BYTES: string;
@IsOptional()
@IsNumberString({}, { message: 'SANDBOX_MAX_IMAGE_BYTES must be an integer (bytes)' })
SANDBOX_MAX_IMAGE_BYTES: string;
@IsOptional()
@IsNumberString({}, { message: 'SANDBOX_MAX_TOTAL_BYTES must be an integer (bytes)' })
SANDBOX_MAX_TOTAL_BYTES: string;
}
export function validate(config: Record<string, any>) {

View File

@@ -131,10 +131,25 @@ export class FailedLoginLimiter {
}
// The per-session DocmostMcpConfig shape understood by @docmost/mcp: either the
// service-account credentials variant OR the per-user getToken variant.
export type DocmostMcpConfig =
// service-account credentials variant OR the per-user getToken variant. The
// optional `sandbox` sink (blob store for the stash tool) is common to both and
// injected by McpService after the auth decision.
export type DocmostMcpConfig = (
| { apiUrl: string; email: string; password: string }
| { apiUrl: string; getToken: () => Promise<string> };
| { apiUrl: string; getToken: () => Promise<string> }
) & {
sandbox?: {
put: (
buf: Buffer,
mime: string,
) => { uri: string; sha256: string; size: number };
// Optional live/evict probes the package uses to keep stash_page's mirror
// counts honest under the store's FIFO eviction (mirror of the package's
// sink type); older bindings omit them.
has?: (uri: string) => boolean;
evict?: (uri: string) => void;
};
};
export interface ResolvedMcpAuth {
config: DocmostMcpConfig;

View File

@@ -109,13 +109,13 @@ function makeService(opts: {
};
const service = new McpService(
undefined as never, // environmentService
undefined as never, // workspaceRepo
undefined as never, // authService
undefined as never, // tokenService
undefined as never, // userRepo
undefined as never, // userSessionRepo
moduleRef as never, // moduleRef (read by the MFA branch)
undefined as never, // sandboxStore (unused by the login-gate path)
);
// Stop the constructor's unref'd sweep timer leaking across tests.
service.onModuleDestroy();

View File

@@ -2,17 +2,15 @@ import { Module } from '@nestjs/common';
import { McpController } from './mcp.controller';
import { McpService } from './mcp.service';
import { DatabaseModule } from '@docmost/db/database.module';
import { EnvironmentModule } from '../environment/environment.module';
import { AuthModule } from '../../core/auth/auth.module';
import { TokenModule } from '../../core/auth/token.module';
// Community MCP feature: the server itself serves the Model Context Protocol
// over HTTP at /mcp. DatabaseModule (global) provides WorkspaceRepo and
// EnvironmentModule (global) provides EnvironmentService. AuthModule supplies
// AuthService (per-user HTTP-Basic login validation) and TokenModule supplies
// TokenService (Bearer access-JWT verification for the token fallback).
// over HTTP at /mcp. DatabaseModule (global) provides WorkspaceRepo. AuthModule
// supplies AuthService (per-user HTTP-Basic login validation) and TokenModule
// supplies TokenService (Bearer access-JWT verification for the token fallback).
@Module({
imports: [DatabaseModule, EnvironmentModule, AuthModule, TokenModule],
imports: [DatabaseModule, AuthModule, TokenModule],
controllers: [McpController],
providers: [McpService],
})

View File

@@ -8,7 +8,6 @@ import { ModuleRef } from '@nestjs/core';
import { pathToFileURL } from 'node:url';
import { IncomingMessage } from 'node:http';
import { FastifyReply, FastifyRequest } from 'fastify';
import { EnvironmentService } from '../environment/environment.service';
import { WorkspaceRepo } from '@docmost/db/repos/workspace/workspace.repo';
import { UserRepo } from '@docmost/db/repos/user/user.repo';
import { UserSessionRepo } from '@docmost/db/repos/session/user-session.repo';
@@ -30,6 +29,7 @@ import {
DocmostMcpConfig,
ResolvedMcpAuth,
} from './mcp-auth.helpers';
import { SandboxStore } from '../sandbox/sandbox.store';
// Minimal shape of the embedded MCP HTTP handler exported by @docmost/mcp/http.
interface McpHttpHandler {
@@ -92,13 +92,14 @@ export class McpService implements OnModuleDestroy {
private readonly sweepTimer: NodeJS.Timeout;
constructor(
private readonly environmentService: EnvironmentService,
private readonly workspaceRepo: WorkspaceRepo,
private readonly authService: AuthService,
private readonly tokenService: TokenService,
private readonly userRepo: UserRepo,
private readonly userSessionRepo: UserSessionRepo,
private readonly moduleRef: ModuleRef,
// Shared singleton in-RAM blob store backing the stash tool.
private readonly sandboxStore: SandboxStore,
) {
this.sweepTimer = setInterval(() => {
try {
@@ -326,7 +327,11 @@ export class McpService implements OnModuleDestroy {
// Should never happen: handle() always stashes before delegating.
throw new UnauthorizedException('MCP authentication missing.');
}
return resolved.config;
// Inject the blob-sandbox sink after the auth decision so stash_page
// can store blobs in the shared in-RAM store regardless of which
// credential variant resolved. The sink (put/has/evict + uri↔id
// mapping) is owned by SandboxStore.asSink().
return { ...resolved.config, sandbox: this.sandboxStore.asSink() };
},
{
identify: (req: IncomingMessage) => {

View File

@@ -0,0 +1,6 @@
// Single source of truth for the anonymous blob-sandbox route. The controller
// is mounted under the global `/api` prefix, so its decorator uses the bare
// segment while the public URL and the workspace-gate exclusion need the full
// path — derive the latter from the former so the two never drift.
export const SANDBOX_ROUTE_SEGMENT = 'sb';
export const SANDBOX_API_PATH = `/api/${SANDBOX_ROUTE_SEGMENT}`;

View File

@@ -0,0 +1,265 @@
import { SandboxController } from './sandbox.controller';
import { SandboxEntry } from './sandbox.store';
// Capturing fake of the FastifyReply surface the controller uses:
// status()/header()/headers()/send(), all chainable.
function makeRes() {
const sent: { status: number; headers: Record<string, any>; body: any } = {
status: 200,
headers: {},
body: undefined,
};
const res: any = {
status(code: number) {
sent.status = code;
return res;
},
header(key: string, value: any) {
sent.headers[key.toLowerCase()] = value;
return res;
},
headers(obj: Record<string, any>) {
for (const k of Object.keys(obj)) sent.headers[k.toLowerCase()] = obj[k];
return res;
},
send(body?: any) {
sent.body = body;
return res;
},
_sent: sent,
};
return res;
}
function makeReq(headers: Record<string, any> = {}) {
return { headers } as any;
}
// A syntactically valid v4 UUID (version nibble 4, variant nibble 8). The
// shared `uuid` validator is stricter than a bare hex-shape regex, so the id
// must carry a real version/variant.
const VALID_ID = 'aaaaaaaa-bbbb-4ccc-8ddd-eeeeeeeeeeee';
function entry(buf: Buffer, mime: string, sha256: string): SandboxEntry {
return { buf, mime, sha256, expiresAt: Date.now() + 60_000 };
}
describe('SandboxController', () => {
it('serves 200 with body, Content-Type, Content-Length and sha256 ETag', async () => {
const buf = Buffer.from('{"ok":true}', 'utf8');
const sha = 'a'.repeat(64);
const store = { get: jest.fn().mockReturnValue(entry(buf, 'application/json', sha)) };
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq(), res);
expect(store.get).toHaveBeenCalledWith(VALID_ID);
expect(res._sent.status).toBe(200);
expect(res._sent.headers['content-type']).toBe('application/json');
expect(res._sent.headers['content-length']).toBe(buf.length);
expect(res._sent.headers['etag']).toBe(`"${sha}"`);
expect(res._sent.body).toBe(buf);
});
it('returns 404 for a missing/expired blob', async () => {
const store = { get: jest.fn().mockReturnValue(undefined) };
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq(), res);
expect(res._sent.status).toBe(404);
expect(res._sent.body).toBeUndefined();
});
it('returns 404 for a non-UUID id WITHOUT touching the store (anti-traversal)', async () => {
const store = { get: jest.fn() };
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get('../../etc/passwd', makeReq(), res);
expect(store.get).not.toHaveBeenCalled();
expect(res._sent.status).toBe(404);
});
it('returns 304 (no body) when If-None-Match matches the ETag', async () => {
const sha = 'b'.repeat(64);
const store = {
get: jest.fn().mockReturnValue(entry(Buffer.from('x'), 'application/json', sha)),
};
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq({ 'if-none-match': `"${sha}"` }), res);
expect(res._sent.status).toBe(304);
expect(res._sent.body).toBeUndefined();
expect(res._sent.headers['etag']).toBe(`"${sha}"`);
});
it('accepts a bare (unquoted) sha256 in If-None-Match too', async () => {
const sha = 'c'.repeat(64);
const store = {
get: jest.fn().mockReturnValue(entry(Buffer.from('x'), 'application/json', sha)),
};
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq({ 'if-none-match': sha }), res);
expect(res._sent.status).toBe(304);
});
it('serves 200 when If-None-Match does NOT match', async () => {
const sha = 'd'.repeat(64);
const store = {
get: jest.fn().mockReturnValue(entry(Buffer.from('x'), 'application/json', sha)),
};
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq({ 'if-none-match': '"stale"' }), res);
expect(res._sent.status).toBe(200);
});
it('returns 304 for a wildcard "*" If-None-Match', async () => {
const sha = 'e'.repeat(64);
const store = {
get: jest.fn().mockReturnValue(entry(Buffer.from('x'), 'application/json', sha)),
};
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq({ 'if-none-match': '*' }), res);
expect(res._sent.status).toBe(304);
});
it('returns 304 for a weak validator W/"<sha>"', async () => {
const sha = 'f'.repeat(64);
const store = {
get: jest.fn().mockReturnValue(entry(Buffer.from('x'), 'application/json', sha)),
};
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq({ 'if-none-match': `W/"${sha}"` }), res);
expect(res._sent.status).toBe(304);
});
it('returns 304 when a comma-separated If-None-Match list contains the sha', async () => {
const sha = '1'.repeat(64);
const store = {
get: jest.fn().mockReturnValue(entry(Buffer.from('x'), 'application/json', sha)),
};
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(
VALID_ID,
makeReq({ 'if-none-match': `"other", "${sha}"` }),
res,
);
expect(res._sent.status).toBe(304);
});
it('sets a private, immutable Cache-Control with a max-age within the TTL on 200', async () => {
const sha = '2'.repeat(64);
// Known TTL: ~30s out, so the floored max-age must land within [0, 60].
const e: SandboxEntry = {
buf: Buffer.from('x'),
mime: 'application/json',
sha256: sha,
expiresAt: Date.now() + 30_000,
};
const store = { get: jest.fn().mockReturnValue(e) };
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq(), res);
expect(res._sent.status).toBe(200);
const cc = res._sent.headers['cache-control'] as string;
expect(cc).toMatch(/^private, max-age=\d+, immutable$/);
const maxAge = Number(cc.match(/max-age=(\d+)/)![1]);
expect(maxAge).toBeGreaterThanOrEqual(0);
expect(maxAge).toBeLessThanOrEqual(60);
});
it('emits Cache-Control alongside ETag on the 304 branch', async () => {
const sha = '3'.repeat(64);
const store = {
get: jest.fn().mockReturnValue(entry(Buffer.from('x'), 'application/json', sha)),
};
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq({ 'if-none-match': `"${sha}"` }), res);
expect(res._sent.status).toBe(304);
expect(res._sent.headers['cache-control']).toMatch(
/^private, max-age=\d+, immutable$/,
);
});
it('sets nosniff + restrictive CSP and serves an allowlisted image inline', async () => {
const sha = '4'.repeat(64);
const store = {
get: jest.fn().mockReturnValue(entry(Buffer.from('x'), 'image/png', sha)),
};
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq(), res);
expect(res._sent.status).toBe(200);
expect(res._sent.headers['x-content-type-options']).toBe('nosniff');
expect(res._sent.headers['content-security-policy']).toBe(
"base-uri 'none'; object-src 'self'; default-src 'self';",
);
expect(res._sent.headers['content-disposition']).toBe('inline');
});
it('forces an SVG to download (attachment) while keeping nosniff + CSP', async () => {
const sha = '5'.repeat(64);
const store = {
get: jest.fn().mockReturnValue(entry(Buffer.from('<svg/>'), 'image/svg+xml', sha)),
};
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq(), res);
expect(res._sent.status).toBe(200);
expect(res._sent.headers['content-disposition']).toBe('attachment');
expect(res._sent.headers['x-content-type-options']).toBe('nosniff');
expect(res._sent.headers['content-security-policy']).toBe(
"base-uri 'none'; object-src 'self'; default-src 'self';",
);
});
it('forces text/html to download (attachment) while keeping nosniff + CSP', async () => {
const sha = '6'.repeat(64);
const store = {
get: jest
.fn()
.mockReturnValue(entry(Buffer.from('<h1>x</h1>'), 'text/html', sha)),
};
const controller = new SandboxController(store as any);
const res = makeRes();
await controller.get(VALID_ID, makeReq(), res);
expect(res._sent.status).toBe(200);
expect(res._sent.headers['content-disposition']).toBe('attachment');
expect(res._sent.headers['x-content-type-options']).toBe('nosniff');
expect(res._sent.headers['content-security-policy']).toBe(
"base-uri 'none'; object-src 'self'; default-src 'self';",
);
});
});

View File

@@ -0,0 +1,130 @@
import { Controller, Get, Param, Req, Res } from '@nestjs/common';
import { FastifyReply, FastifyRequest } from 'fastify';
import { validate as isValidUUID } from 'uuid';
import { SandboxStore } from './sandbox.store';
import { SANDBOX_ROUTE_SEGMENT } from './sandbox.constants';
// MIME types safe to render inline in a browser. SVG is deliberately EXCLUDED
// (it can carry script), as are text/html and the JSON document blob — anything
// not on this list is served as an attachment so an attacker-controlled mime can
// never execute script on this origin (the route is anonymous + same-origin).
const INLINE_SAFE_MIME = new Set([
'image/png',
'image/jpeg',
'image/gif',
'image/webp',
'image/avif',
]);
/**
* Anonymous read endpoint for the in-RAM blob sandbox.
*
* Mounted under the global `/api` prefix as `GET /api/sb/:id`. It carries NO
* `@UseGuards(JwtAuthGuard)`, so — exactly like the public attachment route
* `GET /api/files/public/...` — it is exempt from Docmost session auth. The
* route is ALSO listed in the workspace-resolution preHandler's excludedPaths
* in main.ts so a request from a remote consumer (which carries no workspace
* host) is not rejected with "Workspace not found".
*
* It only ever serves blobs looked up from the SandboxStore by a validated
* UUID; `:id` is never used as a filesystem path, so there is no traversal
* surface. Never returns tokens, never 401s.
*
* Anti-XSS hardening mirrors the public attachment route: every response sets
* `X-Content-Type-Options: nosniff` and a restrictive CSP, and serves any mime
* NOT on the inline-safe allowlist (svg/html/the JSON document blob) as an
* attachment, so an attacker-controlled `entry.mime` can never execute script
* on this same-origin anonymous route.
*/
@Controller(SANDBOX_ROUTE_SEGMENT)
export class SandboxController {
constructor(private readonly store: SandboxStore) {}
@Get(':id')
async get(
@Param('id') id: string,
@Req() req: FastifyRequest,
@Res() res: FastifyReply,
): Promise<void> {
// Validate `:id` as a real UUID via the shared `uuid` validator (same as the
// attachment routes). This is anti-traversal / input hygiene (so `:id` can
// never be a path like `../...`), NOT authorization — the capability is the
// unguessable id itself plus the short TTL plus TLS. A non-UUID id (including
// any traversal attempt) → 404 before touching the store; no stack trace
// leaks out.
if (!isValidUUID(id)) {
res.status(404).send();
return;
}
const entry = this.store.get(id);
if (!entry) {
// Missing or expired — indistinguishable to the caller, by design.
res.status(404).send();
return;
}
// Strong validator: quoted sha256, no W/ weak prefix. Same value computed
// at put() time, so an external consumer can detect a truncated/corrupted
// body — the original bug this whole channel exists to fix.
const etag = `"${entry.sha256}"`;
// Compute freshness BEFORE the conditional check: a 304 conditional
// revalidation must not lose the Cache-Control freshness directives, or a
// revalidating client would forget how long the blob stays fresh.
const ttlSeconds = Math.max(
0,
Math.floor((entry.expiresAt - Date.now()) / 1000),
);
// Capability URL — keep it out of shared caches; immutable for its TTL.
const cacheControl = `private, max-age=${ttlSeconds}, immutable`;
// Conditional request: an exact ETag match → 304 with no body. The blob is
// immutable, so the validator is stable for the blob's whole lifetime.
if (this.ifNoneMatchMatches(req.headers['if-none-match'], entry.sha256)) {
res
.status(304)
.header('ETag', etag)
.header('Cache-Control', cacheControl)
.send();
return;
}
// Non-allowlisted mimes (svg/html/the JSON blob) are forced to download so
// an attacker-controlled mime can never run script inline on this origin.
const disposition = INLINE_SAFE_MIME.has(entry.mime)
? 'inline'
: 'attachment';
// Use @Res() + res.send(Buffer) with an explicit Content-Type so the binary
// body bypasses the global JSON response transform/serializer.
res
.status(200)
.headers({
'Content-Type': entry.mime,
'Content-Length': entry.buf.length,
ETag: etag,
'Cache-Control': cacheControl,
'X-Content-Type-Options': 'nosniff',
'Content-Security-Policy':
"base-uri 'none'; object-src 'self'; default-src 'self';",
'Content-Disposition': disposition,
})
.send(entry.buf);
}
// Accept the consumer's If-None-Match whether it sends the quoted ETag, a bare
// sha256, a weak "W/"-prefixed validator, or a comma-separated list.
private ifNoneMatchMatches(
header: string | string[] | undefined,
sha256: string,
): boolean {
if (!header) return false;
const raw = Array.isArray(header) ? header.join(',') : header;
if (raw.trim() === '*') return true;
return raw
.split(',')
.map((t) => t.trim().replace(/^W\//, '').replace(/^"|"$/g, ''))
.some((t) => t === sha256);
}
}

View File

@@ -0,0 +1,19 @@
import { Global, Module } from '@nestjs/common';
import { SandboxController } from './sandbox.controller';
import { SandboxStore } from './sandbox.store';
/**
* In-RAM blob sandbox: a SINGLE shared SandboxStore (the @Injectable singleton)
* is written to by the stash tool (via McpService / AiChatToolsService) and read
* back by the anonymous SandboxController. Marked @Global so the same store
* instance is injectable everywhere without import churn — put() and get() MUST
* hit the same Map. EnvironmentService (caps/TTL/public URL) is provided by the
* global EnvironmentModule.
*/
@Global()
@Module({
controllers: [SandboxController],
providers: [SandboxStore],
exports: [SandboxStore],
})
export class SandboxModule {}

View File

@@ -0,0 +1,163 @@
import { createHash } from 'node:crypto';
import { validate as isValidUUID } from 'uuid';
import { SandboxStore } from './sandbox.store';
// Build a minimal EnvironmentService stub with overridable caps/TTL.
function makeEnv(
overrides: Partial<{
ttlMs: number;
maxBytes: number;
maxImageBytes: number;
maxTotalBytes: number;
}> = {},
) {
const cfg = {
ttlMs: 3_600_000,
maxBytes: 8_388_608,
maxImageBytes: 20_971_520,
maxTotalBytes: 134_217_728,
...overrides,
};
return {
getSandboxTtlMs: () => cfg.ttlMs,
getSandboxMaxBytes: () => cfg.maxBytes,
getSandboxMaxImageBytes: () => cfg.maxImageBytes,
getSandboxMaxTotalBytes: () => cfg.maxTotalBytes,
getSandboxPublicUrl: () => 'https://example.test',
} as any;
}
describe('SandboxStore', () => {
let store: SandboxStore;
afterEach(() => {
// Clear the unref'd sweep interval so it never leaks across tests.
store?.onModuleDestroy();
jest.useRealTimers();
});
it('put/get round-trips the exact bytes + mime and returns a UUID id', () => {
store = new SandboxStore(makeEnv());
const buf = Buffer.from('{"type":"doc","content":[]}', 'utf8');
const res = store.put(buf, 'application/json');
expect(isValidUUID(res.id)).toBe(true);
expect(res.size).toBe(buf.length);
const entry = store.get(res.id);
expect(entry).toBeDefined();
expect(entry!.buf.equals(buf)).toBe(true);
expect(entry!.mime).toBe('application/json');
});
it('computes sha256 over the body (matches a manual digest)', () => {
store = new SandboxStore(makeEnv());
const buf = Buffer.from('hello sandbox', 'utf8');
const expected = createHash('sha256').update(buf).digest('hex');
const res = store.put(buf, 'text/plain');
expect(res.sha256).toBe(expected);
expect(store.get(res.id)!.sha256).toBe(expected);
});
it('returns undefined for a missing id', () => {
store = new SandboxStore(makeEnv());
expect(store.get('11111111-1111-1111-1111-111111111111')).toBeUndefined();
});
it('lazily expires entries past the TTL (get returns undefined)', () => {
jest.useFakeTimers();
jest.setSystemTime(new Date('2026-01-01T00:00:00Z'));
store = new SandboxStore(makeEnv({ ttlMs: 1000 }));
const res = store.put(Buffer.from('x'), 'text/plain');
expect(store.get(res.id)).toBeDefined();
jest.setSystemTime(new Date('2026-01-01T00:00:02Z')); // +2s > 1s TTL
expect(store.get(res.id)).toBeUndefined();
// Eviction also frees the byte accounting.
expect(store.bytes).toBe(0);
});
it('background sweep drops expired entries without a get()', () => {
jest.useFakeTimers();
jest.setSystemTime(new Date('2026-01-01T00:00:00Z'));
store = new SandboxStore(makeEnv({ ttlMs: 1000 }));
store.put(Buffer.from('x'), 'text/plain');
expect(store.size).toBe(1);
jest.setSystemTime(new Date('2026-01-01T00:01:30Z')); // past TTL
jest.advanceTimersByTime(60_000); // fire the sweep interval
expect(store.size).toBe(0);
});
it('rejects a non-image blob over SANDBOX_MAX_BYTES', () => {
store = new SandboxStore(makeEnv({ maxBytes: 16 }));
expect(() => store.put(Buffer.alloc(17), 'application/json')).toThrow(
/per-blob cap/,
);
});
it('uses the larger image cap for image/* blobs', () => {
// 100 bytes exceeds the doc cap (16) but fits the image cap (1024).
store = new SandboxStore(makeEnv({ maxBytes: 16, maxImageBytes: 1024 }));
expect(() => store.put(Buffer.alloc(100), 'image/png')).not.toThrow();
// SVG counts as an image too.
expect(() => store.put(Buffer.alloc(100), 'image/svg+xml')).not.toThrow();
});
it('evicts oldest entries when the total cap would be exceeded', () => {
// Total cap 250 bytes; each blob 100 bytes -> only 2 fit at a time.
store = new SandboxStore(
makeEnv({ maxTotalBytes: 250, maxBytes: 1024 }),
);
const a = store.put(Buffer.alloc(100), 'application/json');
const b = store.put(Buffer.alloc(100), 'application/json');
const c = store.put(Buffer.alloc(100), 'application/json'); // evicts a
expect(store.get(a.id)).toBeUndefined(); // oldest evicted
expect(store.get(b.id)).toBeDefined();
expect(store.get(c.id)).toBeDefined();
expect(store.bytes).toBeLessThanOrEqual(250);
});
it('rejects a single blob larger than the whole total cap', () => {
store = new SandboxStore(
makeEnv({ maxTotalBytes: 50, maxBytes: 1024 }),
);
expect(() => store.put(Buffer.alloc(100), 'application/json')).toThrow(
/total store cap/,
);
});
it('putAndLink composes the anonymous /api/sb/<id> url with matching integrity', () => {
store = new SandboxStore(makeEnv());
const buf = Buffer.from('hello link', 'utf8');
const expected = createHash('sha256').update(buf).digest('hex');
const res = store.putAndLink(buf, 'image/png');
expect(res.uri).toMatch(/^https:\/\/example\.test\/api\/sb\/[0-9a-f-]{36}$/);
expect(res.sha256).toBe(expected);
expect(res.size).toBe(buf.length);
});
it('has()/remove() report and free a blob by id', () => {
store = new SandboxStore(makeEnv());
const { id } = store.put(Buffer.from('x'), 'text/plain');
expect(store.has(id)).toBe(true);
store.remove(id);
expect(store.has(id)).toBe(false);
expect(store.bytes).toBe(0);
});
it('asSink() round-trips put/has/evict through the anonymous uri', () => {
store = new SandboxStore(makeEnv());
const sink = store.asSink();
const buf = Buffer.from('sink bytes', 'utf8');
const r = sink.put(buf, 'image/png');
expect(sink.has(r.uri)).toBe(true);
sink.evict(r.uri);
expect(sink.has(r.uri)).toBe(false);
});
});

View File

@@ -0,0 +1,178 @@
import { Injectable, Logger, OnModuleDestroy } from '@nestjs/common';
import { createHash, randomUUID } from 'node:crypto';
import { EnvironmentService } from '../environment/environment.service';
import { SANDBOX_API_PATH } from './sandbox.constants';
// In-RAM, process-local blob store. No disk, no DB. Ephemeral by design: a
// restart empties it. A blob is addressed by an unguessable randomUUID() which
// IS the read capability — there are NO tokens. Each blob is immutable (its id
// never maps to changing content), so its sha256 is a perfect strong ETag.
export interface SandboxEntry {
buf: Buffer;
mime: string;
sha256: string;
expiresAt: number;
}
export interface SandboxPutResult {
id: string;
sha256: string;
size: number;
}
@Injectable()
export class SandboxStore implements OnModuleDestroy {
private readonly logger = new Logger(SandboxStore.name);
// Map preserves insertion order, so the first key is the oldest entry — used
// for FIFO eviction when the total-bytes RAM guard is exceeded.
private readonly map = new Map<string, SandboxEntry>();
private totalBytes = 0;
// Background sweep clears expired entries so never-fetched blobs do not linger
// until the next get(). unref()'d so it never holds the event loop open;
// cleared on module destroy. Mirrors the sweepTimer pattern in
// integrations/mcp/mcp.service.ts and packages/mcp/src/http.ts.
private readonly sweepIntervalMs = 60_000;
private readonly sweepTimer: NodeJS.Timeout;
constructor(private readonly environmentService: EnvironmentService) {
this.sweepTimer = setInterval(() => {
try {
this.sweep();
} catch (err) {
this.logger.error('Sandbox sweep failed', err as Error);
}
}, this.sweepIntervalMs);
this.sweepTimer.unref?.();
}
onModuleDestroy(): void {
clearInterval(this.sweepTimer);
}
/**
* Store a blob and return its read capability id + integrity metadata. The
* per-blob cap is chosen by mime (images get the larger image cap), and the
* total-store RAM guard evicts oldest entries to make room. Throws a clear
* error when a single blob cannot fit even after eviction. Blob bodies are
* never logged.
*/
put(buf: Buffer, mime: string): SandboxPutResult {
const perBlobCap = mime.startsWith('image/')
? this.environmentService.getSandboxMaxImageBytes()
: this.environmentService.getSandboxMaxBytes();
if (buf.length > perBlobCap) {
throw new Error(
`Sandbox blob of ${buf.length} bytes exceeds the ${perBlobCap}-byte per-blob cap`,
);
}
const maxTotal = this.environmentService.getSandboxMaxTotalBytes();
if (buf.length > maxTotal) {
throw new Error(
`Sandbox blob of ${buf.length} bytes exceeds the total store cap of ${maxTotal} bytes`,
);
}
// Drop expired entries first, then evict oldest until the new blob fits.
this.sweep();
while (this.totalBytes + buf.length > maxTotal && this.map.size > 0) {
const oldest = this.map.keys().next().value as string;
this.evict(oldest);
}
const id = randomUUID();
const sha256 = createHash('sha256').update(buf).digest('hex');
const expiresAt = Date.now() + this.environmentService.getSandboxTtlMs();
this.map.set(id, { buf, mime, sha256, expiresAt });
this.totalBytes += buf.length;
return { id, sha256, size: buf.length };
}
/**
* Store a blob and return its anonymous read URL plus integrity metadata.
* Owns the single sandbox-URL composition (`${publicBase}${SANDBOX_API_PATH}/
* <id>`) so callers never hand-build the route; the raw put() stays public for
* tests/low-level callers. sha256 is also the blob's strong ETag.
*/
putAndLink(
buf: Buffer,
mime: string,
): { uri: string; sha256: string; size: number } {
const stored = this.put(buf, mime);
const base = this.environmentService.getSandboxPublicUrl();
return {
uri: `${base}${SANDBOX_API_PATH}/${stored.id}`,
sha256: stored.sha256,
size: stored.size,
};
}
/**
* Adapter to the package's blob-sandbox sink contract `{ put, has, evict }`.
* The sink speaks anonymous `uri`s while the store is keyed by `id`, so this is
* the ONE place that maps a sandbox uri back to its id (the last path segment).
* Both wiring sites (embedded MCP + in-app agent tools) use this so the uri↔id
* mapping and URL composition live next to putAndLink, not copy-pasted.
*/
asSink(): {
put: (buf: Buffer, mime: string) => { uri: string; sha256: string; size: number };
has: (uri: string) => boolean;
evict: (uri: string) => void;
} {
const idOf = (uri: string) => uri.substring(uri.lastIndexOf('/') + 1);
return {
put: (buf, mime) => this.putAndLink(buf, mime),
has: (uri) => this.has(idOf(uri)),
evict: (uri) => this.remove(idOf(uri)),
};
}
/** True if the blob is still live (not evicted/expired). */
has(id: string): boolean {
return this.get(id) !== undefined;
}
/** Drop a blob by id (public wrapper over the private FIFO evict). */
remove(id: string): void {
this.evict(id);
}
/** Returns the entry, or undefined if missing OR expired (lazy expiry). */
get(id: string): SandboxEntry | undefined {
const entry = this.map.get(id);
if (!entry) return undefined;
if (entry.expiresAt <= Date.now()) {
this.evict(id);
return undefined;
}
return entry;
}
/** Current number of live entries (test/diagnostic helper). */
get size(): number {
return this.map.size;
}
/** Current total bytes held (test/diagnostic helper). */
get bytes(): number {
return this.totalBytes;
}
private evict(id: string): void {
const entry = this.map.get(id);
if (entry) {
this.totalBytes -= entry.buf.length;
this.map.delete(id);
}
}
private sweep(): void {
const now = Date.now();
for (const [id, entry] of this.map) {
if (entry.expiresAt <= now) {
this.evict(id);
}
}
}
}

View File

@@ -1,18 +1,110 @@
import { Readable } from 'stream';
import { StorageService } from './storage.service';
import type { StorageDriver } from './interfaces';
// Direct instantiation with a stub driver. The Test.createTestingModule form
// failed to resolve the STORAGE_DRIVER_TOKEN at compile(); this smoke test only
// needs the service to construct.
describe('StorageService', () => {
/**
* StorageService is a thin facade over the injected StorageDriver: each public
* method must forward to the driver with the SAME arguments and return/await the
* driver's result unchanged (the read paths return it; the write paths await it).
* A mock driver lets us assert that delegation exactly, with no real S3/disk IO.
*/
describe('StorageService delegation', () => {
// Every driver method is a jest mock so we can assert call args + return passing.
function buildDriver(): jest.Mocked<StorageDriver> {
return {
upload: jest.fn().mockResolvedValue(undefined),
uploadStream: jest.fn().mockResolvedValue(undefined),
copy: jest.fn().mockResolvedValue(undefined),
read: jest.fn(),
readStream: jest.fn(),
readRangeStream: jest.fn(),
exists: jest.fn(),
getUrl: jest.fn(),
getSignedUrl: jest.fn(),
delete: jest.fn().mockResolvedValue(undefined),
getDriver: jest.fn(),
getDriverName: jest.fn(),
getConfig: jest.fn(),
} as unknown as jest.Mocked<StorageDriver>;
}
let driver: jest.Mocked<StorageDriver>;
let service: StorageService;
beforeEach(() => {
service = new StorageService(
{} as any, // storageDriver
);
driver = buildDriver();
service = new StorageService(driver as unknown as StorageDriver);
});
it('should be defined', () => {
expect(service).toBeDefined();
it('upload forwards path + content to the driver', async () => {
const buf = Buffer.from('data');
await service.upload('a/b.png', buf);
expect(driver.upload).toHaveBeenCalledWith('a/b.png', buf);
});
it('uploadStream forwards path, stream and options', async () => {
const stream = Readable.from(['x']);
await service.uploadStream('a/b.bin', stream, { recreateClient: true });
expect(driver.uploadStream).toHaveBeenCalledWith('a/b.bin', stream, {
recreateClient: true,
});
});
it('copy forwards both paths', async () => {
await service.copy('from.txt', 'to.txt');
expect(driver.copy).toHaveBeenCalledWith('from.txt', 'to.txt');
});
it('read returns the driver buffer unchanged', async () => {
const buf = Buffer.from('content');
driver.read.mockResolvedValue(buf);
await expect(service.read('f.txt')).resolves.toBe(buf);
expect(driver.read).toHaveBeenCalledWith('f.txt');
});
it('readStream returns the driver stream unchanged', async () => {
const stream = Readable.from(['y']);
driver.readStream.mockResolvedValue(stream);
await expect(service.readStream('f.bin')).resolves.toBe(stream);
expect(driver.readStream).toHaveBeenCalledWith('f.bin');
});
it('readRangeStream forwards the range object and returns the stream', async () => {
const stream = Readable.from(['z']);
driver.readRangeStream.mockResolvedValue(stream);
const range = { start: 0, end: 99 };
await expect(service.readRangeStream('f.bin', range)).resolves.toBe(stream);
expect(driver.readRangeStream).toHaveBeenCalledWith('f.bin', range);
});
it('exists returns the driver boolean', async () => {
driver.exists.mockResolvedValue(false);
await expect(service.exists('missing')).resolves.toBe(false);
expect(driver.exists).toHaveBeenCalledWith('missing');
});
it('getSignedUrl forwards path + expiry and returns the signed url', async () => {
driver.getSignedUrl.mockResolvedValue('https://signed/url');
await expect(service.getSignedUrl('f.png', 600)).resolves.toBe(
'https://signed/url',
);
expect(driver.getSignedUrl).toHaveBeenCalledWith('f.png', 600);
});
it('getUrl returns the driver url synchronously', () => {
driver.getUrl.mockReturnValue('https://cdn/f.png');
expect(service.getUrl('f.png')).toBe('https://cdn/f.png');
expect(driver.getUrl).toHaveBeenCalledWith('f.png');
});
it('delete forwards the path', async () => {
await service.delete('old.txt');
expect(driver.delete).toHaveBeenCalledWith('old.txt');
});
it('getDriverName returns the driver name', () => {
driver.getDriverName.mockReturnValue('s3');
expect(service.getDriverName()).toBe('s3');
expect(driver.getDriverName).toHaveBeenCalledTimes(1);
});
});

View File

@@ -13,6 +13,7 @@ import fastifyCookie from '@fastify/cookie';
import fastifyIp from 'fastify-ip';
import { InternalLogFilter } from './common/logger/internal-log-filter';
import { EnvironmentService } from './integrations/environment/environment.service';
import { SANDBOX_API_PATH } from './integrations/sandbox/sandbox.constants';
import { resolveFrameHeader } from './common/helpers';
import { resolveTrustProxy } from './integrations/environment/trust-proxy.util';
@@ -126,6 +127,10 @@ async function bootstrap() {
'/api/workspace/create',
'/api/workspace/joined',
'/api/workspace/find-by-email',
// Anonymous in-RAM blob sandbox: a remote consumer fetches blobs by an
// unguessable UUID without any workspace host context, so the
// workspace-resolution gate must not apply.
SANDBOX_API_PATH,
];
if (

View File

@@ -0,0 +1,133 @@
import { describe, it, expect } from "vitest";
import { schema } from "@tiptap/pm/schema-basic";
import type { Node as PMNode } from "@tiptap/pm/model";
import { Transform } from "@tiptap/pm/transform";
import { recreateTransform } from "./recreateTransform";
/**
* recreateTransform diffs two documents and produces ProseMirror steps that turn
* `fromDoc` into `toDoc`. It is the backbone of collaborative/version diffing, so
* THE invariant that matters is: replaying the produced steps on `fromDoc` must
* reproduce `toDoc` exactly. Every test below re-applies the steps onto a fresh
* Transform seeded from `fromDoc` (not just trusting `tr.doc`) and asserts node
* equality with `.eq()`. If a regression makes any step wrong, the round-trip
* breaks and the test fails.
*/
// Real ProseMirror schema (the standard basic schema) with paragraph/heading +
// strong/em marks — the same primitives the editor diffs in production.
const doc = (...c: PMNode[]) => schema.node("doc", null, c);
const p = (...c: PMNode[]) =>
schema.node("paragraph", null, c.length ? c : undefined);
const h = (level: number, ...c: PMNode[]) =>
schema.node("heading", { level }, c);
const t = (text: string, ...marks: any[]) =>
schema.text(text, marks.length ? marks : undefined);
const strong = schema.marks.strong.create();
const em = schema.marks.em.create();
// Replay the diff's steps onto a fresh Transform built from `fromDoc`. This is
// the faithful "apply(diff) == target" check — it exercises the actual Step
// objects rather than the transform's internal accumulated doc.
function applyDiff(fromDoc: PMNode, toDoc: PMNode, options?: any): PMNode {
const tr = recreateTransform(fromDoc, toDoc, options);
const replay = new Transform(fromDoc);
tr.steps.forEach((s) => {
const result = replay.maybeStep(s);
if (result.failed) throw new Error(`step failed: ${result.failed}`);
});
return replay.doc;
}
describe("recreateTransform round-trip (apply(diff) == target)", () => {
it("reconstructs the target on plain text insertion", () => {
// Inserting " world" must yield exactly the target paragraph.
const from = doc(p(t("hello")));
const to = doc(p(t("hello world")));
expect(applyDiff(from, to).eq(to)).toBe(true);
});
it("reconstructs the target on text deletion", () => {
// Deleting a trailing word is the inverse of insertion and must round-trip.
const from = doc(p(t("hello world")));
const to = doc(p(t("hello")));
expect(applyDiff(from, to).eq(to)).toBe(true);
});
it("reconstructs the target when a word is replaced mid-string", () => {
// A char-level replace in the middle must not corrupt the surrounding text.
const from = doc(p(t("the quick brown fox")));
const to = doc(p(t("the slow brown fox")));
expect(applyDiff(from, to).eq(to)).toBe(true);
});
it("reconstructs the target when a mark is added (complexSteps path)", () => {
// Mark-only changes are diffed in a separate pass; the bolded run must match.
const from = doc(p(t("hello")));
const to = doc(p(t("hello", strong)));
const out = applyDiff(from, to);
expect(out.eq(to)).toBe(true);
// Sanity: the produced doc actually carries the strong mark.
expect(out.firstChild!.firstChild!.marks.length).toBe(1);
});
it("reconstructs the target when a mark is removed", () => {
// Removing the only mark must leave the same text with no marks.
const from = doc(p(t("hello", strong)));
const to = doc(p(t("hello")));
const out = applyDiff(from, to);
expect(out.eq(to)).toBe(true);
expect(out.firstChild!.firstChild!.marks.length).toBe(0);
});
it("reconstructs the target on a paragraph split into two blocks", () => {
// Structural change (one block -> two) must replay as valid replace steps.
const from = doc(p(t("hello world")));
const to = doc(p(t("hello")), p(t("world")));
const out = applyDiff(from, to);
expect(out.eq(to)).toBe(true);
expect(out.childCount).toBe(2);
});
it("reconstructs the target on a node-type change (paragraph -> heading)", () => {
// Type/attrs changes drive the setNodeMarkup branch; the node must become a
// heading while keeping its text.
const from = doc(p(t("hello")));
const to = doc(h(1, t("hello")));
const out = applyDiff(from, to);
expect(out.eq(to)).toBe(true);
expect(out.firstChild!.type.name).toBe("heading");
});
it("reconstructs a combined structural + mark change", () => {
// Several diff kinds at once (new block + italic run) still round-trips.
const from = doc(p(t("alpha")));
const to = doc(p(t("alpha")), p(t("beta", em)));
const out = applyDiff(from, to);
expect(out.eq(to)).toBe(true);
});
it("produces an empty step list for identical documents", () => {
// No diff => no work; spurious steps would mean wasted/incorrect history.
const from = doc(p(t("same")));
const to = doc(p(t("same")));
const tr = recreateTransform(from, to);
expect(tr.steps.length).toBe(0);
expect(tr.doc.eq(to)).toBe(true);
});
it("round-trips with complexSteps:false (marks diffed as replaces)", () => {
// With complexSteps off, mark changes are folded into replace steps rather
// than dedicated mark steps — the result must still equal the target.
const from = doc(p(t("hello")));
const to = doc(p(t("hello", strong)));
expect(applyDiff(from, to, { complexSteps: false }).eq(to)).toBe(true);
});
it("round-trips with wordDiffs:true (whole-word text diffing)", () => {
// wordDiffs changes the granularity of the text diff, not the outcome.
const from = doc(p(t("the quick brown fox")));
const to = doc(p(t("the quick red fox")));
expect(applyDiff(from, to, { wordDiffs: true }).eq(to)).toBe(true);
});
});

View File

@@ -0,0 +1,108 @@
import { describe, it, expect } from "vitest";
import { Schema } from "@tiptap/pm/model";
import type { Node as PMNode } from "@tiptap/pm/model";
import { tableNodes } from "@tiptap/pm/tables";
import { EditorState, Selection } from "@tiptap/pm/state";
import { getSelectionRangeInColumn } from "./get-selection-range-in-column";
/**
* getSelectionRangeInColumn computes the rectangular column range (the set of
* column indexes, plus anchor/head cell positions) that a drag-reorder or
* column-select operation should act on, accounting for merged (colspan) cells.
* It keys off the table found from the current selection, so we drive it with a
* real EditorState whose selection sits inside the table.
*/
// Real ProseMirror table schema (same primitives the editor uses) so TableMap /
// cellsInRect behave exactly as in production.
const tNodes = tableNodes({
tableGroup: "block",
cellContent: "inline*",
cellAttributes: {},
});
const schema = new Schema({
nodes: {
doc: { content: "block+" },
paragraph: { group: "block", content: "inline*", toDOM: () => ["p", 0] },
text: { group: "inline" },
...tNodes,
},
marks: {},
});
const cell = (txt: string, attrs?: Record<string, unknown>): PMNode =>
schema.nodes.table_cell.createChecked(attrs ?? null, schema.text(txt));
const row = (...cells: PMNode[]): PMNode =>
schema.nodes.table_row.createChecked(null, cells);
const table = (...rows: PMNode[]): PMNode =>
schema.nodes.table.createChecked(null, rows);
const doc = (...content: PMNode[]): PMNode =>
schema.nodes.doc.createChecked(null, content);
// Build a transaction whose selection is inside the table (the function locates
// the table via `tr.selection.$from`).
const trFor = (d: PMNode) =>
EditorState.create({ doc: d, selection: Selection.atStart(d) }).tr;
// A 2-row x 3-col grid; each column is identifiable by its top-row letter.
const grid3x2 = () =>
doc(
table(
row(cell("a"), cell("b"), cell("c")),
row(cell("d"), cell("e"), cell("f")),
),
);
describe("getSelectionRangeInColumn", () => {
it("returns a single-column range for a single index", () => {
// Asking for column 1 yields exactly indexes [1].
const tr = trFor(grid3x2());
const range = getSelectionRangeInColumn(tr, 1);
expect(range).toBeTruthy();
expect(range!.indexes).toEqual([1]);
});
it("anchor/head resolve to the top and bottom cells OF the requested column", () => {
// $head must point at the column's first (top) cell and $anchor at its last
// (bottom) cell — pinning that the returned positions belong to column 1,
// not some other column.
const tr = trFor(grid3x2());
const range = getSelectionRangeInColumn(tr, 1)!;
expect(tr.doc.nodeAt(range.$head.pos)?.textContent).toBe("b"); // top of col 1
expect(tr.doc.nodeAt(range.$anchor.pos)?.textContent).toBe("e"); // bottom of col 1
});
it("returns the inclusive span of columns for a multi-column request", () => {
// A 0..2 request must enumerate every covered column, in order.
const tr = trFor(grid3x2());
const range = getSelectionRangeInColumn(tr, 0, 2);
expect(range!.indexes).toEqual([0, 1, 2]);
});
it("returns a two-column span for an adjacent pair", () => {
const tr = trFor(grid3x2());
const range = getSelectionRangeInColumn(tr, 1, 2);
expect(range!.indexes).toEqual([1, 2]);
});
it("expands the range to cover a horizontally merged (colspan) cell", () => {
// Row 0 col 0 spans 2 columns. Requesting just column 0 must pull column 1
// into the range because they are merged together in the top row.
const d = doc(
table(
row(cell("ab", { colspan: 2 }), cell("c")),
row(cell("d"), cell("e"), cell("f")),
),
);
const tr = trFor(d);
const range = getSelectionRangeInColumn(tr, 0);
expect(range!.indexes).toEqual([0, 1]);
});
it("throws when the requested column is entirely out of range", () => {
// No cells exist at column 5 of a 3-wide table, so the function cannot pick
// an anchor cell and dereferences undefined — pin this as the current
// (caller-guarded) contract so a silent behavior change is caught.
const tr = trFor(grid3x2());
expect(() => getSelectionRangeInColumn(tr, 5)).toThrow();
});
});

View File

@@ -0,0 +1,156 @@
import { describe, it, expect } from "vitest";
import { Schema } from "@tiptap/pm/model";
import type { Node as PMNode } from "@tiptap/pm/model";
import { tableNodes, CellSelection } from "@tiptap/pm/tables";
import { EditorState, Selection } from "@tiptap/pm/state";
import { moveColumn } from "./move-column";
import { convertTableNodeToArrayOfRows } from "./convert-table-node-to-array-of-rows";
import { findTable } from "./query";
/**
* moveColumn reorders whole columns of a real ProseMirror table by mutating a
* Transaction (transpose -> move row -> transpose back -> replace). The invariant
* is that after the call each column appears at its new position with every
* cell's content preserved and nothing dropped or duplicated.
*/
const tNodes = tableNodes({
tableGroup: "block",
cellContent: "inline*",
cellAttributes: {},
});
const schema = new Schema({
nodes: {
doc: { content: "block+" },
paragraph: { group: "block", content: "inline*", toDOM: () => ["p", 0] },
text: { group: "inline" },
...tNodes,
},
marks: {},
});
const cell = (txt: string): PMNode =>
schema.nodes.table_cell.createChecked(null, schema.text(txt));
const row = (...cells: PMNode[]): PMNode =>
schema.nodes.table_row.createChecked(null, cells);
const table = (...rows: PMNode[]): PMNode =>
schema.nodes.table.createChecked(null, rows);
const doc = (...content: PMNode[]): PMNode =>
schema.nodes.doc.createChecked(null, content);
const grid = (tr: any): string[][] => {
const t = findTable(tr.doc.resolve(tr.selection.from))!;
return convertTableNodeToArrayOfRows(t.node).map((r) =>
r.map((c) => (c ? c.textContent : "")),
);
};
// 2-row x 3-col table; column k is (rowX-col-k). Columns: 0=(a,d) 1=(b,e) 2=(c,f).
const grid3x2 = () =>
doc(
table(
row(cell("a"), cell("b"), cell("c")),
row(cell("d"), cell("e"), cell("f")),
),
);
const stateFor = (d: PMNode) =>
EditorState.create({ doc: d, selection: Selection.atStart(d) });
describe("moveColumn", () => {
it("moves the first column to the last index, preserving column content", () => {
// origin 0 -> target 2 sends column (a,d) to the right: cols become 1,2,0.
const state = stateFor(grid3x2());
const tr = state.tr;
const ok = moveColumn({
tr,
originIndex: 0,
targetIndex: 2,
select: false,
pos: state.selection.from,
});
expect(ok).toBe(true);
expect(grid(tr)).toEqual([
["b", "c", "a"],
["e", "f", "d"],
]);
});
it("moves a later column to the first index", () => {
// origin 2 -> target 0 pulls column (c,f) to the front: cols become 2,0,1.
const state = stateFor(grid3x2());
const tr = state.tr;
const ok = moveColumn({
tr,
originIndex: 2,
targetIndex: 0,
select: false,
pos: state.selection.from,
});
expect(ok).toBe(true);
expect(grid(tr)).toEqual([
["c", "a", "b"],
["f", "d", "e"],
]);
});
it("never drops or duplicates cells when reordering columns", () => {
const state = stateFor(grid3x2());
const tr = state.tr;
moveColumn({
tr,
originIndex: 1,
targetIndex: 2,
select: false,
pos: state.selection.from,
});
expect(grid(tr).flat().sort()).toEqual(
["a", "b", "c", "d", "e", "f"].sort(),
);
expect(grid(tr)[0].length).toBe(3);
});
it("returns false (no-op) when target equals origin", () => {
const state = stateFor(grid3x2());
const tr = state.tr;
const before = grid(tr);
const ok = moveColumn({
tr,
originIndex: 1,
targetIndex: 1,
select: false,
pos: state.selection.from,
});
expect(ok).toBe(false);
expect(grid(tr)).toEqual(before);
});
it("returns false when pos is not inside a table", () => {
const d = doc(
schema.nodes.paragraph.createChecked(null, schema.text("plain")),
);
const state = stateFor(d);
const tr = state.tr;
const ok = moveColumn({
tr,
originIndex: 0,
targetIndex: 1,
select: false,
pos: state.selection.from,
});
expect(ok).toBe(false);
});
it("installs a CellSelection on the moved column when select is true", () => {
const state = stateFor(grid3x2());
const tr = state.tr;
const ok = moveColumn({
tr,
originIndex: 0,
targetIndex: 2,
select: true,
pos: state.selection.from,
});
expect(ok).toBe(true);
expect(tr.selection instanceof CellSelection).toBe(true);
});
});

View File

@@ -0,0 +1,167 @@
import { describe, it, expect } from "vitest";
import { Schema } from "@tiptap/pm/model";
import type { Node as PMNode } from "@tiptap/pm/model";
import { tableNodes, CellSelection } from "@tiptap/pm/tables";
import { EditorState, Selection } from "@tiptap/pm/state";
import { moveRow } from "./move-row";
import { convertTableNodeToArrayOfRows } from "./convert-table-node-to-array-of-rows";
import { findTable } from "./query";
/**
* moveRow reorders whole rows of a real ProseMirror table by mutating a
* Transaction: it locates the table, computes origin/target row ranges, rebuilds
* the table with rows reordered, and replaces it in the doc. The invariant is
* that after the call the table's rows appear in the new order with every cell's
* content preserved, and no rows are dropped or duplicated.
*/
const tNodes = tableNodes({
tableGroup: "block",
cellContent: "inline*",
cellAttributes: {},
});
const schema = new Schema({
nodes: {
doc: { content: "block+" },
paragraph: { group: "block", content: "inline*", toDOM: () => ["p", 0] },
text: { group: "inline" },
...tNodes,
},
marks: {},
});
const cell = (txt: string): PMNode =>
schema.nodes.table_cell.createChecked(null, schema.text(txt));
const row = (...cells: PMNode[]): PMNode =>
schema.nodes.table_row.createChecked(null, cells);
const table = (...rows: PMNode[]): PMNode =>
schema.nodes.table.createChecked(null, rows);
const doc = (...content: PMNode[]): PMNode =>
schema.nodes.doc.createChecked(null, content);
// Read the table's content as a grid of cell texts (rows x cols) from whatever
// table currently lives in `tr.doc`.
const grid = (tr: any): string[][] => {
const t = findTable(tr.doc.resolve(tr.selection.from))!;
return convertTableNodeToArrayOfRows(t.node).map((r) =>
r.map((c) => (c ? c.textContent : "")),
);
};
// 3-row x 2-col table; each row identifiable by its cells.
const grid2x3 = () =>
doc(
table(
row(cell("r0a"), cell("r0b")),
row(cell("r1a"), cell("r1b")),
row(cell("r2a"), cell("r2b")),
),
);
const stateFor = (d: PMNode) =>
EditorState.create({ doc: d, selection: Selection.atStart(d) });
describe("moveRow", () => {
it("moves the first row down to the last index, preserving content", () => {
// origin 0 -> target 2 makes row 0 land after the other rows: [r1, r2, r0].
const state = stateFor(grid2x3());
const tr = state.tr;
const ok = moveRow({
tr,
originIndex: 0,
targetIndex: 2,
select: false,
pos: state.selection.from,
});
expect(ok).toBe(true);
expect(grid(tr)).toEqual([
["r1a", "r1b"],
["r2a", "r2b"],
["r0a", "r0b"],
]);
});
it("moves a lower row up to an earlier index", () => {
// origin 2 -> target 0 lifts the last row above the rest: [r2, r0, r1].
const state = stateFor(grid2x3());
const tr = state.tr;
const ok = moveRow({
tr,
originIndex: 2,
targetIndex: 0,
select: false,
pos: state.selection.from,
});
expect(ok).toBe(true);
expect(grid(tr)).toEqual([
["r2a", "r2b"],
["r0a", "r0b"],
["r1a", "r1b"],
]);
});
it("never drops or duplicates rows when reordering", () => {
// The full multiset of cell texts is invariant under any valid move.
const state = stateFor(grid2x3());
const tr = state.tr;
moveRow({
tr,
originIndex: 1,
targetIndex: 2,
select: false,
pos: state.selection.from,
});
const flat = grid(tr).flat().sort();
expect(flat).toEqual(
["r0a", "r0b", "r1a", "r1b", "r2a", "r2b"].sort(),
);
expect(grid(tr).length).toBe(3);
});
it("returns false (no-op) when target equals origin", () => {
// Moving a row onto itself is rejected and leaves the table unchanged.
const state = stateFor(grid2x3());
const tr = state.tr;
const before = grid(tr);
const ok = moveRow({
tr,
originIndex: 1,
targetIndex: 1,
select: false,
pos: state.selection.from,
});
expect(ok).toBe(false);
expect(grid(tr)).toEqual(before);
});
it("returns false when pos is not inside a table", () => {
// Without a table at `pos`, the function bails out instead of throwing.
const d = doc(
schema.nodes.paragraph.createChecked(null, schema.text("plain")),
);
const state = stateFor(d);
const tr = state.tr;
const ok = moveRow({
tr,
originIndex: 0,
targetIndex: 1,
select: false,
pos: state.selection.from,
});
expect(ok).toBe(false);
});
it("installs a CellSelection on the moved row when select is true", () => {
// With select:true the moved row at the target index is selected.
const state = stateFor(grid2x3());
const tr = state.tr;
const ok = moveRow({
tr,
originIndex: 0,
targetIndex: 2,
select: true,
pos: state.selection.from,
});
expect(ok).toBe(true);
expect(tr.selection instanceof CellSelection).toBe(true);
});
});

View File

@@ -100,4 +100,51 @@ describe("addUniqueIdsToDoc", () => {
const [id] = ids(out);
expect(id).toBeTruthy();
});
it("only assigns ids to configured node types, not to others", () => {
// `types` is ["heading","paragraph"]; a codeBlock is NOT addressed, so it
// must come back without an id while the sibling paragraph is filled. (The
// UniqueID attribute only exists on configured types in the schema.)
const doc = {
type: "doc",
content: [
{ type: "codeBlock", content: [{ type: "text", text: "x = 1" }] },
para(undefined, "after"),
],
};
const out = addUniqueIdsToDoc(doc, extensions);
const [codeId, paraId] = ids(out);
expect(codeId).toBeUndefined();
expect(paraId).toBeTruthy();
});
it("assigns ids to target nodes nested inside non-target containers", () => {
// findChildren walks the whole tree: a paragraph inside a blockquote still
// gets an id, while the (non-target) blockquote wrapper does not.
const doc = {
type: "doc",
content: [
{ type: "blockquote", content: [para(undefined, "quoted")] },
],
};
const out = addUniqueIdsToDoc(doc, extensions) as any;
const blockquote = out.content[0];
const nestedPara = blockquote.content[0];
expect(blockquote.attrs?.id).toBeUndefined();
expect(nestedPara.attrs.id).toBeTruthy();
});
it("is idempotent: a second pass keeps every already-unique id unchanged", () => {
// Once ids are assigned and unique, re-running must be a fixed point — no
// churn that would invalidate stored MCP anchors on every save.
const doc = {
type: "doc",
content: [para(undefined, "a"), para(undefined, "b"), para(undefined, "c")],
};
const once = addUniqueIdsToDoc(doc, extensions);
const twice = addUniqueIdsToDoc(once, extensions);
expect(ids(twice)).toEqual(ids(once));
// And all three are distinct, so the second pass had real ids to preserve.
expect(new Set(ids(once)).size).toBe(3);
});
});

View File

@@ -16,7 +16,7 @@ license.
> that interface. Other Docmost MCPs are human-shaped — they expose "open the page" and
> "replace the page"; this one exposes the editing primitives a model is good at.
It exposes **38 tools** built around three ideas that the other Docmost MCPs do not
It exposes **40 tools** built around three ideas that the other Docmost MCPs do not
combine:
1. **Surgical, token-cheap edits.** Address a single block by id and patch it, or run
@@ -106,7 +106,7 @@ There are several Docmost MCPs. Here is a capability-by-capability comparison.
## Tools
All 38 tools, grouped by what you'd reach for them.
All 40 tools, grouped by what you'd reach for them.
### Exploration & retrieval
@@ -203,6 +203,14 @@ All 38 tools, grouped by what you'd reach for them.
node referencing the old attachment (recursively, including callouts/tables) via the
live document, preserving comments, alignment and alt text. (In-place overwrite is
deliberately avoided — some Docmost versions corrupt the attachment on overwrite.)
- **`stash_page`** — Serialize a whole page (its full ProseMirror JSON) into an ephemeral
in-RAM blob and return ONLY a short anonymous URL — the body never enters the model
context, so it is the way to hand a large page (and its images) to an external consumer
without truncation. Every internal file/image attachment is mirrored into the same
sandbox and its `src` rewritten to a sandbox URL; external http(s) images are left
untouched. Returns `{ uri, size, sha256, images:{ mirrored, failed } }` (`sha256` is also
the blob's ETag). Blobs are RAM-only, expire after a short TTL (~1h) and are bound to the
server instance that created them.
### Comments

View File

@@ -17,7 +17,7 @@
> «открыть страницу» и «заменить страницу»; этот даёт примитивы редактирования, в которых
> модель сильна.
Сервер предоставляет **38 инструментов**, построенных вокруг трёх идей, которые другие
Сервер предоставляет **40 инструментов**, построенных вокруг трёх идей, которые другие
Docmost-MCP не сочетают:
1. **Точечные, экономичные по токенам правки.** Адресуйте отдельный блок по id и патчите
@@ -109,7 +109,7 @@ Docmost-MCP не сочетают:
## Инструменты
Все 38 инструментов, сгруппированы по задачам, для которых вы их возьмёте.
Все 40 инструментов, сгруппированы по задачам, для которых вы их возьмёте.
### Чтение и поиск
@@ -209,6 +209,15 @@ Docmost-MCP не сочетают:
коллауты/таблицы), через живой документ, сохраняя комментарии, выравнивание и alt-текст.
(Перезапись «по месту» намеренно не используется — некоторые версии Docmost портят
вложение при перезаписи.)
- **`stash_page`** — Сериализовать страницу целиком (её полный ProseMirror JSON) в
эфемерный blob в оперативной памяти и вернуть ТОЛЬКО короткий анонимный URL — тело
никогда не попадает в контекст модели, поэтому это способ передать большую страницу
(вместе с её изображениями) внешнему потребителю без усечения. Каждое внутреннее
файловое/графическое вложение зеркалируется в тот же sandbox, а его `src` переписывается
на URL sandbox; внешние http(s)-изображения остаются нетронутыми. Возвращает
`{ uri, size, sha256, images:{ mirrored, failed } }` (`sha256` — это также ETag blob'а).
Blob'ы хранятся только в оперативной памяти, истекают через короткий TTL (~1 ч) и
привязаны к тому экземпляру сервера, который их создал.
### Комментарии

View File

@@ -7,6 +7,7 @@ import { TiptapTransformer } from "@hocuspocus/transformer";
import * as Y from "yjs";
import WebSocket from "ws";
import { convertProseMirrorToMarkdown } from "./lib/markdown-converter.js";
import { collectInternalFileNodes, normalizeFileUrl, resolveInternalFilePath, } from "./lib/internal-file-urls.js";
import { updatePageContentRealtime, replacePageContent, markdownToProseMirror, markdownToProseMirrorCanonical, mutatePageContent, buildCollabWsUrl, assertYjsEncodable, applyDocToFragment, } from "./lib/collaboration.js";
import { footnoteWarningsField } from "./lib/footnote-analyze.js";
import { buildPageTree } from "./lib/tree.js";
@@ -51,6 +52,13 @@ export class DocmostClient {
// its token instead of calling POST /auth/collab-token; on a 401/403 it is
// re-invoked once. Used by the internal agent to carry signed provenance.
getCollabTokenFn = null;
// Optional blob-sandbox sink for the stash tool. Null when not configured.
sandboxPut = null;
// Optional probes paired with the sink. `has` lets stashPage detect a blob
// FIFO-evicted by a LATER put in the same stash; `evict` lets it free this
// op's image blobs if the final doc put throws. Null when the sink omits them.
sandboxHas = null;
sandboxEvict = null;
// In-flight login dedup: when the token expires, the 401 interceptor,
// ensureAuthenticated, getCollabTokenWithReauth and the two multipart retries
// can all call login() at once. Memoizing a single promise collapses that
@@ -77,6 +85,11 @@ export class DocmostClient {
if (config.getCollabToken) {
this.getCollabTokenFn = config.getCollabToken;
}
if (config.sandbox) {
this.sandboxPut = config.sandbox.put;
this.sandboxHas = config.sandbox.has ?? null;
this.sandboxEvict = config.sandbox.evict ?? null;
}
this.client = axios.create({
baseURL: this.apiUrl,
// Default request timeout so a hung connection cannot wedge a per-page
@@ -605,6 +618,181 @@ export class DocmostClient {
content: data.content || { type: "doc", content: [] },
};
}
/**
* Fetch an INTERNAL Docmost file (authed loopback) for sandbox mirroring.
* `src` is normalized to `/api/files/<id>/<file>`; `this.client.baseURL`
* already ends in `/api`, so we strip the leading `/api` and request the
* relative path with the client's Authorization header. Returns the raw bytes
* and the response Content-Type (mime), defaulting to octet-stream.
*
* The fetch is size-bounded (hard 64 MiB ceiling) purely to protect memory;
* the authoritative per-blob cap is enforced by the sandbox `put`. The path is
* resolved via resolveInternalFilePath, which REJECTS (throws) any traversal
* or percent-encoded src that would let an attacker-controlled `attrs.src`
* escape `/api/files/` and reach another internal endpoint (SSRF). That throw
* happens before this.client.get, so a malicious src is counted as a failed
* mirror — it never reaches the network.
*/
async fetchInternalFile(src) {
const HARD_CEILING = 64 * 1024 * 1024; // 64 MiB memory guard
const relPath = resolveInternalFilePath(src);
const response = await this.client.get(relPath, {
responseType: "arraybuffer",
timeout: 30000,
maxContentLength: HARD_CEILING,
maxBodyLength: HARD_CEILING,
});
const buffer = Buffer.from(response.data);
if (buffer.length === 0) {
throw new Error(`Empty file response from "${src}"`);
}
const rawCt = response.headers?.["content-type"];
const mime = typeof rawCt === "string" && rawCt.length > 0
? rawCt.split(";")[0].trim().toLowerCase()
: "application/octet-stream";
return { buffer, mime };
}
/**
* Stash a page's full content into the in-RAM blob sandbox and return ONLY a
* short anonymous URL — the body never enters the model context (this is the
* whole point: ~30KB+ ProseMirror docs blow the model context if passed as a
* tool argument). Every INTERNAL file/image src (the type-agnostic criterion,
* so drawio/excalidraw/video/file nodes are covered too) is mirrored into the
* sandbox and its `src` rewritten to the sandbox URL, so an external consumer
* can fetch the images anonymously. External http(s) srcs are left untouched.
*
* Blobs live in RAM with a short TTL and are cleared on restart — consume the
* URLs within the TTL and one uptime. A failed image fetch never aborts the
* doc: the original src is kept and the failure counted.
*
* Returns { uri, sha256, size, images:{mirrored, failed} }. `uri` and `sha256`
* are for the document blob; `sha256` is also the blob's ETag (integrity).
*/
async stashPage(pageId) {
if (!this.sandboxPut) {
throw new Error("stash_page is unavailable: the blob sandbox is not configured on this server");
}
await this.ensureAuthenticated();
// Stash the SAME shape get_page_json returns (id/title/.../content), with a
// deep clone so the rewrite never mutates anything shared.
const pageJson = await this.getPageJson(pageId);
const cloned = structuredClone(pageJson);
// Group internal-file nodes by normalized src so each unique resource is
// fetched + stored ONCE (dedup), and every node sharing that src points at
// the one sandbox blob. Capture each node's ORIGINAL raw src per-node:
// dedup groups nodes whose normalized src is equal even when their raw srcs
// differ (e.g. `/api/files/...` vs the bare `/files/...`), so on a revert we
// must restore each node's own original value, not the group key.
const bySrc = new Map();
for (const node of collectInternalFileNodes(cloned.content)) {
const origSrc = String(node.attrs.src);
const src = normalizeFileUrl(origSrc);
const entry = { node, origSrc };
const group = bySrc.get(src);
if (group)
group.push(entry);
else
bySrc.set(src, [entry]);
}
let mirrored = 0;
let failed = 0;
// Record every successful mirror so it can be (a) reverted if its blob gets
// FIFO-evicted by a LATER put in this same stash, and (b) freed if the final
// doc put throws.
const mirrors = [];
const MAX_CONCURRENCY = 5;
const groups = [...bySrc.entries()];
for (let i = 0; i < groups.length; i += MAX_CONCURRENCY) {
const batch = groups.slice(i, i + MAX_CONCURRENCY);
await Promise.all(batch.map(async ([src, entries]) => {
try {
const { buffer, mime } = await this.fetchInternalFile(src);
// put may throw if the blob exceeds the per-blob/total caps.
const stored = this.sandboxPut(buffer, mime);
for (const entry of entries)
entry.node.attrs.src = stored.uri;
mirrors.push({ uri: stored.uri, entries });
mirrored++;
}
catch (err) {
// One bad/oversized image (or a rejected traversal src) must not
// abort the document. Logged unconditionally (never the blob body),
// matching the package's ungated console.warn convention.
failed++;
console.warn(`stash_page: failed to mirror "${src}": ${err instanceof Error ? err.message : String(err)}`);
}
}));
}
// Revert one mirror's nodes to their original internal srcs and re-count it
// as failed (its blob was FIFO-evicted before the doc could reference it
// safely).
const revertMirror = (mirror) => {
for (const entry of mirror.entries)
entry.node.attrs.src = entry.origSrc;
mirrored--;
failed++;
console.warn(`stash_page: mirrored blob ${mirror.uri} was evicted before the doc ` +
`could safely reference it; reverted its src and counted it as failed`);
};
// Pre-put reconciliation: an image put earlier in THIS stash can FIFO-evict
// an even-earlier image of the same stash. Drop those from the live set
// first so the first serialized doc is already mostly correct.
let liveMirrors = mirrors;
if (this.sandboxHas) {
liveMirrors = [];
for (const mirror of mirrors) {
if (this.sandboxHas(mirror.uri))
liveMirrors.push(mirror);
else
revertMirror(mirror);
}
}
// Put the document, then reconcile against eviction caused by the doc put
// ITSELF (the doc is newest, FIFO drops oldest = this stash's images). Each
// iteration reverts >=1 mirror, so the loop terminates (worst case: all
// images reverted and the doc references no sandbox image URLs).
let stored;
for (;;) {
const docBuf = Buffer.from(JSON.stringify(cloned), "utf8");
let docStored;
try {
docStored = this.sandboxPut(docBuf, "application/json");
}
catch (err) {
// The doc put failed (e.g. doc exceeds the cap). Free this op's image
// blobs instead of leaking them in RAM for the whole TTL, then
// re-throw.
if (this.sandboxEvict) {
for (const mirror of liveMirrors)
this.sandboxEvict(mirror.uri);
}
throw err;
}
if (!this.sandboxHas) {
stored = docStored;
break;
}
const evictedNow = liveMirrors.filter((m) => !this.sandboxHas(m.uri));
if (evictedNow.length === 0) {
stored = docStored;
break;
}
// The doc we just stored references now-dead blobs. Revert those nodes,
// drop the stale doc blob, and loop to re-serialize + re-put the
// corrected doc.
for (const mirror of evictedNow)
revertMirror(mirror);
liveMirrors = liveMirrors.filter((m) => this.sandboxHas(m.uri));
if (this.sandboxEvict)
this.sandboxEvict(docStored.uri);
}
return {
uri: stored.uri,
sha256: stored.sha256,
size: stored.size,
images: { mirrored, failed },
};
}
/**
* Compact outline of a page's top-level blocks (no full document body).
* Cheap way to locate sections/tables and grab block ids before drilling in

View File

@@ -285,6 +285,38 @@ export function createDocmostMcpServer(config) {
const result = await docmostClient.editPageText(pageId, edits);
return jsonContent(result);
});
// Tool: stash_page — returns a resource_link (NOT embedded text) so the doc
// body never enters the model context. Registered directly (not via
// registerShared) because that helper only emits text content. Also returns
// `structuredContent` carrying the full documented `{uri, sha256, size, images}`
// shape alongside the resource_link, so MCP clients receive the blob's sha256
// (its ETag, for integrity) and mirror counts, not just the link.
server.registerTool(SHARED_TOOL_SPECS.stashPage.mcpName, {
description: SHARED_TOOL_SPECS.stashPage.description,
inputSchema: SHARED_TOOL_SPECS.stashPage.buildShape(z),
}, async ({ pageId }) => {
const result = await docmostClient.stashPage(pageId);
return {
content: [
{
type: "resource_link",
uri: result.uri,
name: "page.json",
mimeType: "application/json",
size: result.size,
},
],
// Mirror the full documented result shape ({ uri, size, sha256, images })
// as structuredContent so MCP clients get the blob's sha256 (its ETag, for
// integrity) and the mirror counts, not just the resource_link.
structuredContent: {
uri: result.uri,
sha256: result.sha256,
size: result.size,
images: result.images,
},
};
});
// Tool: patch_node
server.registerTool("patch_node", {
description: "Replaces a single block identified by its attrs.id WITHOUT resending the " +

View File

@@ -29,6 +29,41 @@ export async function getCollabToken(baseUrl, apiToken) {
throw error;
}
}
/**
* Pure cookie-parsing helper extracted from `performLogin` so the parsing logic
* can be unit-tested without performing the login network request. Given the
* raw `Set-Cookie` header array from the login response, return the `authToken`
* cookie's value.
*
* Behavior (kept identical to the original inline logic):
* - throws if there is no Set-Cookie header at all;
* - matches the cookie NAME exactly (`authToken`), so a future
* `authTokenRefresh=...` cookie is NOT picked up (a `startsWith` would be);
* - returns everything after the FIRST `=` up to the first `;`, so a base64
* value containing `=` padding is preserved (a naive `split("=")` would
* truncate it);
* - cookie attributes after the first `;` (Path, HttpOnly, Expires, …) are
* ignored;
* - throws if no `authToken` cookie is present.
*/
export function extractAuthTokenFromSetCookie(cookies) {
if (!cookies) {
throw new Error("No Set-Cookie header found in login response");
}
// Match the cookie name exactly to avoid matching a future
// authTokenRefresh cookie (startsWith would catch it).
const authCookie = cookies.find((c) => {
const kv = c.split(";")[0];
return kv.slice(0, kv.indexOf("=")) === "authToken";
});
if (!authCookie) {
throw new Error("No authToken cookie found in login response");
}
// Take everything after the FIRST "=" up to the first ";".
// Splitting on "=" would truncate base64 values containing "=" padding.
const kv = authCookie.split(";")[0];
return kv.slice(kv.indexOf("=") + 1);
}
export async function performLogin(baseUrl, email, password) {
try {
const response = await axios.post(`${baseUrl}/auth/login`, {
@@ -36,24 +71,7 @@ export async function performLogin(baseUrl, email, password) {
password,
});
// Extract token from Set-Cookie header
const cookies = response.headers["set-cookie"];
if (!cookies) {
throw new Error("No Set-Cookie header found in login response");
}
// Match the cookie name exactly to avoid matching a future
// authTokenRefresh cookie (startsWith would catch it).
const authCookie = cookies.find((c) => {
const kv = c.split(";")[0];
return kv.slice(0, kv.indexOf("=")) === "authToken";
});
if (!authCookie) {
throw new Error("No authToken cookie found in login response");
}
// Take everything after the FIRST "=" up to the first ";".
// Splitting on "=" would truncate base64 values containing "=" padding.
const kv = authCookie.split(";")[0];
const token = kv.slice(kv.indexOf("=") + 1);
return token;
return extractAuthTokenFromSetCookie(response.headers["set-cookie"]);
}
catch (error) {
// Avoid leaking the full server response body by default; log only the

View File

@@ -0,0 +1,110 @@
// Detection + collection of INTERNAL Docmost file URLs inside a ProseMirror doc.
//
// An internal file URL is a relative path served by Docmost's authenticated
// attachment route (`GET /api/files/:fileId/:fileName`). It is useless to an
// external consumer (relative + needs a Docmost session), so the stash tool
// mirrors every such resource into the blob sandbox and rewrites its `src`.
//
// The criterion is "internal file URL", NOT the node TYPE: image, drawio,
// excalidraw, video and file nodes all carry such a `src`, so a type-agnostic
// walker covers them all. External http(s) srcs (CDNs) are left untouched.
//
// Mirrors editor-ext's isInternalFileUrl / normalizeFileUrl (kept as a local
// dup so the ESM mcp package does not depend on the editor-ext build).
function isInternalFileUrl(url) {
if (typeof url !== "string")
return false;
const normalized = url.trim();
return (normalized.startsWith("/api/files/") || normalized.startsWith("/files/"));
}
/** Normalize a bare `/files/...` src to the canonical `/api/files/...` form. */
export function normalizeFileUrl(src) {
const trimmed = src.trim();
if (trimmed.startsWith("/files/"))
return "/api" + trimmed;
return trimmed;
}
/**
* Resolve a page-content `src` into the safe, `/api`-relative path the stash
* tool may fetch over the authenticated loopback client — or THROW.
*
* SECURITY (SSRF / path-traversal): `src` comes from page content and is fully
* attacker-controllable. The mirroring fetch runs through the AUTHENTICATED
* loopback axios client whose baseURL ends in `/api`, so a naive
* `src.replace(/^\/api/, "")` lets a crafted value like
* `/api/files/../auth/whoami` collapse (via axios/WHATWG URL `..` resolution)
* into an ARBITRARY internal GET endpoint, whose authed response would then be
* stored in the anonymous sandbox (SSRF + data exfiltration). A prefix-only
* `startsWith("/api/files/")` check does NOT defend against this because the
* `..` segments are still present in the raw string and resolved later.
*
* This function defeats that by resolving the canonical pathname FIRST and only
* then asserting it still lives under `/api/files/`:
* - it rejects any percent-encoded dot/slash (`%2e` / `%2f`): the WHATWG URL
* parser collapses LITERAL `../` but does NOT decode `%2f` separators, so a
* content-controlled src must never be allowed to smuggle those past the
* canonicalization;
* - it resolves `new URL(trimmed, "http://internal.invalid").pathname`, which
* normalizes `..`/`.` segments (e.g. `/api/files/../auth/whoami` →
* `/api/auth/whoami`);
* - it then requires the canonical pathname to start with `/api/files/`, so a
* traversal that escaped that subtree is rejected.
*
* Returns the path RELATIVE to the `/api` base (e.g. `/files/<id>/<name>`),
* ready to hand to the loopback client. The throw happens BEFORE any network
* call, so a rejected src is counted as a failed mirror and its original src is
* kept (the per-image try/catch in stashPage never aborts the whole document).
*/
export function resolveInternalFilePath(src) {
const trimmed = src.trim();
// Percent-encoded dot/slash must never reach the URL canonicalizer: the
// WHATWG parser does NOT decode `%2f` into a path separator, so an encoded
// `..%2fauth` would survive canonicalization and still escape /api/files/.
if (/%2e|%2f/i.test(trimmed)) {
throw new Error(`Refusing internal file src with percent-encoded path segment: "${src}"`);
}
let pathname;
try {
// The base host is irrelevant (never contacted); it only lets the parser
// resolve a relative `src` and normalize `..`/`.` segments.
pathname = new URL(trimmed, "http://internal.invalid").pathname;
}
catch {
throw new Error(`Invalid internal file src: "${src}"`);
}
if (!pathname.startsWith("/api/files/")) {
throw new Error(`Refusing internal file src that escapes /api/files/: "${src}"`);
}
// Strip the `/api` base prefix; the loopback client's baseURL already ends
// in `/api`, so it expects the path relative to that (e.g. /files/<id>/<f>).
return pathname.replace(/^\/api/, "");
}
/**
* Recursively collect every node whose `attrs.src` is an internal file URL.
* Returns references to the live nodes (so the caller can rewrite `attrs.src`
* in place on its clone). Descends `content` arrays, covering callouts, tables,
* details and any other nested container.
*/
export function collectInternalFileNodes(doc) {
const out = [];
const visit = (node) => {
if (!node)
return;
if (Array.isArray(node)) {
for (const child of node)
visit(child);
return;
}
if (typeof node !== "object")
return;
if (node.attrs && isInternalFileUrl(node.attrs.src)) {
out.push(node);
}
if (Array.isArray(node.content)) {
for (const child of node.content)
visit(child);
}
};
visit(doc);
return out;
}

View File

@@ -209,4 +209,27 @@ export const SHARED_TOOL_SPECS = {
.describe('List of find/replace operations, applied in order'),
}),
},
// --- hand a large page to an external consumer without bloating context ---
stashPage: {
mcpName: 'stash_page',
inAppKey: 'stashPage',
description: 'Serialize a whole page (the full ProseMirror JSON, as get_page_json ' +
'returns) into an ephemeral in-memory blob and return ONLY a short ' +
'anonymous URL to it — the body NEVER enters the model context, so this ' +
'is the way to hand a large page (or its images) to an external consumer ' +
'without truncation. Every internal file/image attachment is mirrored ' +
'into the same sandbox and its src rewritten to a sandbox URL, so the ' +
'consumer can fetch the images anonymously too; external http(s) images ' +
'are left untouched. Returns { uri, size, sha256, images:{mirrored, ' +
'failed} }. Integrity: the blob is served with ETag = its sha256, so a ' +
'truncated/corrupted fetch is detectable. Blobs are RAM-only: they expire ' +
'after a short TTL (~1h) and are cleared on restart — consume the URL ' +
'within the TTL and one uptime, or re-stash. A blob is bound to the ' +
'server instance that created it: in a multi-replica deployment without ' +
'sticky sessions a blob stored on one instance is not retrievable via the ' +
'sandbox URL on another (it 404s like an expired one).',
buildShape: (z) => ({
pageId: z.string().min(1),
}),
},
};

View File

@@ -13,6 +13,11 @@ import { TiptapTransformer } from "@hocuspocus/transformer";
import * as Y from "yjs";
import WebSocket from "ws";
import { convertProseMirrorToMarkdown } from "./lib/markdown-converter.js";
import {
collectInternalFileNodes,
normalizeFileUrl,
resolveInternalFilePath,
} from "./lib/internal-file-urls.js";
import {
updatePageContentRealtime,
replacePageContent,
@@ -102,6 +107,14 @@ const MIME_TO_EXT: Record<string, string> = {
* Housed here (not in index.ts) so client.ts has no type dependency on index.ts;
* index.ts re-exports it for the package's public surface.
*/
// Sink the stash tool writes blobs into. The host app binds this to its in-RAM
// SandboxStore and composes the public `uri` (the package never sees the store
// or any env). `put` returns the anonymous read URL plus integrity metadata.
export type SandboxPut = (
buf: Buffer,
mime: string,
) => { uri: string; sha256: string; size: number };
export type DocmostMcpConfig = { apiUrl: string } & (
| { email: string; password: string }
| { getToken: () => Promise<string> } // returns a BARE JWT; the client adds "Bearer "
@@ -109,6 +122,15 @@ export type DocmostMcpConfig = { apiUrl: string } & (
// Optional collab-token provider (returns a ready collab JWT). Common to
// both branches; see the type doc above.
getCollabToken?: () => Promise<string>;
// Optional blob sandbox sink. Present only where the stash tool is wired;
// when absent, stash_page throws a clear "not configured" error. The
// optional `has`/`evict` probes let stashPage keep its mirror counts honest
// under the store's FIFO eviction (see stashPage); older sinks omit them.
sandbox?: {
put: SandboxPut;
has?: (uri: string) => boolean;
evict?: (uri: string) => void;
};
};
export class DocmostClient {
@@ -126,6 +148,13 @@ export class DocmostClient {
// its token instead of calling POST /auth/collab-token; on a 401/403 it is
// re-invoked once. Used by the internal agent to carry signed provenance.
private getCollabTokenFn: (() => Promise<string>) | null = null;
// Optional blob-sandbox sink for the stash tool. Null when not configured.
private sandboxPut: SandboxPut | null = null;
// Optional probes paired with the sink. `has` lets stashPage detect a blob
// FIFO-evicted by a LATER put in the same stash; `evict` lets it free this
// op's image blobs if the final doc put throws. Null when the sink omits them.
private sandboxHas: ((uri: string) => boolean) | null = null;
private sandboxEvict: ((uri: string) => void) | null = null;
// In-flight login dedup: when the token expires, the 401 interceptor,
// ensureAuthenticated, getCollabTokenWithReauth and the two multipart retries
// can all call login() at once. Memoizing a single promise collapses that
@@ -165,6 +194,11 @@ export class DocmostClient {
if (config.getCollabToken) {
this.getCollabTokenFn = config.getCollabToken;
}
if (config.sandbox) {
this.sandboxPut = config.sandbox.put;
this.sandboxHas = config.sandbox.has ?? null;
this.sandboxEvict = config.sandbox.evict ?? null;
}
this.client = axios.create({
baseURL: this.apiUrl,
// Default request timeout so a hung connection cannot wedge a per-page
@@ -767,6 +801,203 @@ export class DocmostClient {
};
}
/**
* Fetch an INTERNAL Docmost file (authed loopback) for sandbox mirroring.
* `src` is normalized to `/api/files/<id>/<file>`; `this.client.baseURL`
* already ends in `/api`, so we strip the leading `/api` and request the
* relative path with the client's Authorization header. Returns the raw bytes
* and the response Content-Type (mime), defaulting to octet-stream.
*
* The fetch is size-bounded (hard 64 MiB ceiling) purely to protect memory;
* the authoritative per-blob cap is enforced by the sandbox `put`. The path is
* resolved via resolveInternalFilePath, which REJECTS (throws) any traversal
* or percent-encoded src that would let an attacker-controlled `attrs.src`
* escape `/api/files/` and reach another internal endpoint (SSRF). That throw
* happens before this.client.get, so a malicious src is counted as a failed
* mirror — it never reaches the network.
*/
private async fetchInternalFile(
src: string,
): Promise<{ buffer: Buffer; mime: string }> {
const HARD_CEILING = 64 * 1024 * 1024; // 64 MiB memory guard
const relPath = resolveInternalFilePath(src);
const response = await this.client.get(relPath, {
responseType: "arraybuffer",
timeout: 30000,
maxContentLength: HARD_CEILING,
maxBodyLength: HARD_CEILING,
});
const buffer = Buffer.from(response.data);
if (buffer.length === 0) {
throw new Error(`Empty file response from "${src}"`);
}
const rawCt = response.headers?.["content-type"];
const mime =
typeof rawCt === "string" && rawCt.length > 0
? rawCt.split(";")[0].trim().toLowerCase()
: "application/octet-stream";
return { buffer, mime };
}
/**
* Stash a page's full content into the in-RAM blob sandbox and return ONLY a
* short anonymous URL — the body never enters the model context (this is the
* whole point: ~30KB+ ProseMirror docs blow the model context if passed as a
* tool argument). Every INTERNAL file/image src (the type-agnostic criterion,
* so drawio/excalidraw/video/file nodes are covered too) is mirrored into the
* sandbox and its `src` rewritten to the sandbox URL, so an external consumer
* can fetch the images anonymously. External http(s) srcs are left untouched.
*
* Blobs live in RAM with a short TTL and are cleared on restart — consume the
* URLs within the TTL and one uptime. A failed image fetch never aborts the
* doc: the original src is kept and the failure counted.
*
* Returns { uri, sha256, size, images:{mirrored, failed} }. `uri` and `sha256`
* are for the document blob; `sha256` is also the blob's ETag (integrity).
*/
async stashPage(pageId: string): Promise<{
uri: string;
sha256: string;
size: number;
images: { mirrored: number; failed: number };
}> {
if (!this.sandboxPut) {
throw new Error(
"stash_page is unavailable: the blob sandbox is not configured on this server",
);
}
await this.ensureAuthenticated();
// Stash the SAME shape get_page_json returns (id/title/.../content), with a
// deep clone so the rewrite never mutates anything shared.
const pageJson = await this.getPageJson(pageId);
const cloned: any = structuredClone(pageJson);
// Group internal-file nodes by normalized src so each unique resource is
// fetched + stored ONCE (dedup), and every node sharing that src points at
// the one sandbox blob. Capture each node's ORIGINAL raw src per-node:
// dedup groups nodes whose normalized src is equal even when their raw srcs
// differ (e.g. `/api/files/...` vs the bare `/files/...`), so on a revert we
// must restore each node's own original value, not the group key.
const bySrc = new Map<string, Array<{ node: any; origSrc: string }>>();
for (const node of collectInternalFileNodes(cloned.content)) {
const origSrc = String(node.attrs.src);
const src = normalizeFileUrl(origSrc);
const entry = { node, origSrc };
const group = bySrc.get(src);
if (group) group.push(entry);
else bySrc.set(src, [entry]);
}
let mirrored = 0;
let failed = 0;
// Record every successful mirror so it can be (a) reverted if its blob gets
// FIFO-evicted by a LATER put in this same stash, and (b) freed if the final
// doc put throws.
const mirrors: Array<{
uri: string;
entries: Array<{ node: any; origSrc: string }>;
}> = [];
const MAX_CONCURRENCY = 5;
const groups = [...bySrc.entries()];
for (let i = 0; i < groups.length; i += MAX_CONCURRENCY) {
const batch = groups.slice(i, i + MAX_CONCURRENCY);
await Promise.all(
batch.map(async ([src, entries]) => {
try {
const { buffer, mime } = await this.fetchInternalFile(src);
// put may throw if the blob exceeds the per-blob/total caps.
const stored = this.sandboxPut!(buffer, mime);
for (const entry of entries) entry.node.attrs.src = stored.uri;
mirrors.push({ uri: stored.uri, entries });
mirrored++;
} catch (err) {
// One bad/oversized image (or a rejected traversal src) must not
// abort the document. Logged unconditionally (never the blob body),
// matching the package's ungated console.warn convention.
failed++;
console.warn(
`stash_page: failed to mirror "${src}": ${
err instanceof Error ? err.message : String(err)
}`,
);
}
}),
);
}
// Revert one mirror's nodes to their original internal srcs and re-count it
// as failed (its blob was FIFO-evicted before the doc could reference it
// safely).
const revertMirror = (mirror: {
uri: string;
entries: Array<{ node: any; origSrc: string }>;
}) => {
for (const entry of mirror.entries) entry.node.attrs.src = entry.origSrc;
mirrored--;
failed++;
console.warn(
`stash_page: mirrored blob ${mirror.uri} was evicted before the doc ` +
`could safely reference it; reverted its src and counted it as failed`,
);
};
// Pre-put reconciliation: an image put earlier in THIS stash can FIFO-evict
// an even-earlier image of the same stash. Drop those from the live set
// first so the first serialized doc is already mostly correct.
let liveMirrors = mirrors;
if (this.sandboxHas) {
liveMirrors = [];
for (const mirror of mirrors) {
if (this.sandboxHas(mirror.uri)) liveMirrors.push(mirror);
else revertMirror(mirror);
}
}
// Put the document, then reconcile against eviction caused by the doc put
// ITSELF (the doc is newest, FIFO drops oldest = this stash's images). Each
// iteration reverts >=1 mirror, so the loop terminates (worst case: all
// images reverted and the doc references no sandbox image URLs).
let stored: { uri: string; sha256: string; size: number };
for (;;) {
const docBuf = Buffer.from(JSON.stringify(cloned), "utf8");
let docStored: { uri: string; sha256: string; size: number };
try {
docStored = this.sandboxPut(docBuf, "application/json");
} catch (err) {
// The doc put failed (e.g. doc exceeds the cap). Free this op's image
// blobs instead of leaking them in RAM for the whole TTL, then
// re-throw.
if (this.sandboxEvict) {
for (const mirror of liveMirrors) this.sandboxEvict(mirror.uri);
}
throw err;
}
if (!this.sandboxHas) {
stored = docStored;
break;
}
const evictedNow = liveMirrors.filter((m) => !this.sandboxHas!(m.uri));
if (evictedNow.length === 0) {
stored = docStored;
break;
}
// The doc we just stored references now-dead blobs. Revert those nodes,
// drop the stale doc blob, and loop to re-serialize + re-put the
// corrected doc.
for (const mirror of evictedNow) revertMirror(mirror);
liveMirrors = liveMirrors.filter((m) => this.sandboxHas!(m.uri));
if (this.sandboxEvict) this.sandboxEvict(docStored.uri);
}
return {
uri: stored.uri,
sha256: stored.sha256,
size: stored.size,
images: { mirrored, failed },
};
}
/**
* Compact outline of a page's top-level blocks (no full document body).
* Cheap way to locate sections/tables and grab block ids before drilling in

View File

@@ -408,6 +408,43 @@ registerShared(SHARED_TOOL_SPECS.editPageText, async ({ pageId, edits }) => {
return jsonContent(result);
});
// Tool: stash_page — returns a resource_link (NOT embedded text) so the doc
// body never enters the model context. Registered directly (not via
// registerShared) because that helper only emits text content. Also returns
// `structuredContent` carrying the full documented `{uri, sha256, size, images}`
// shape alongside the resource_link, so MCP clients receive the blob's sha256
// (its ETag, for integrity) and mirror counts, not just the link.
server.registerTool(
SHARED_TOOL_SPECS.stashPage.mcpName,
{
description: SHARED_TOOL_SPECS.stashPage.description,
inputSchema: SHARED_TOOL_SPECS.stashPage.buildShape!(z),
},
async ({ pageId }: { pageId: string }) => {
const result = await docmostClient.stashPage(pageId);
return {
content: [
{
type: "resource_link" as const,
uri: result.uri,
name: "page.json",
mimeType: "application/json",
size: result.size,
},
],
// Mirror the full documented result shape ({ uri, size, sha256, images })
// as structuredContent so MCP clients get the blob's sha256 (its ETag, for
// integrity) and the mirror counts, not just the resource_link.
structuredContent: {
uri: result.uri,
sha256: result.sha256,
size: result.size,
images: result.images,
},
};
},
);
// Tool: patch_node
server.registerTool(
"patch_node",

View File

@@ -38,6 +38,45 @@ export async function getCollabToken(
}
}
/**
* Pure cookie-parsing helper extracted from `performLogin` so the parsing logic
* can be unit-tested without performing the login network request. Given the
* raw `Set-Cookie` header array from the login response, return the `authToken`
* cookie's value.
*
* Behavior (kept identical to the original inline logic):
* - throws if there is no Set-Cookie header at all;
* - matches the cookie NAME exactly (`authToken`), so a future
* `authTokenRefresh=...` cookie is NOT picked up (a `startsWith` would be);
* - returns everything after the FIRST `=` up to the first `;`, so a base64
* value containing `=` padding is preserved (a naive `split("=")` would
* truncate it);
* - cookie attributes after the first `;` (Path, HttpOnly, Expires, …) are
* ignored;
* - throws if no `authToken` cookie is present.
*/
export function extractAuthTokenFromSetCookie(
cookies: string[] | undefined,
): string {
if (!cookies) {
throw new Error("No Set-Cookie header found in login response");
}
// Match the cookie name exactly to avoid matching a future
// authTokenRefresh cookie (startsWith would catch it).
const authCookie = cookies.find((c: string) => {
const kv = c.split(";")[0];
return kv.slice(0, kv.indexOf("=")) === "authToken";
});
if (!authCookie) {
throw new Error("No authToken cookie found in login response");
}
// Take everything after the FIRST "=" up to the first ";".
// Splitting on "=" would truncate base64 values containing "=" padding.
const kv = authCookie.split(";")[0];
return kv.slice(kv.indexOf("=") + 1);
}
export async function performLogin(
baseUrl: string,
email: string,
@@ -50,25 +89,7 @@ export async function performLogin(
});
// Extract token from Set-Cookie header
const cookies = response.headers["set-cookie"];
if (!cookies) {
throw new Error("No Set-Cookie header found in login response");
}
// Match the cookie name exactly to avoid matching a future
// authTokenRefresh cookie (startsWith would catch it).
const authCookie = cookies.find((c: string) => {
const kv = c.split(";")[0];
return kv.slice(0, kv.indexOf("=")) === "authToken";
});
if (!authCookie) {
throw new Error("No authToken cookie found in login response");
}
// Take everything after the FIRST "=" up to the first ";".
// Splitting on "=" would truncate base64 values containing "=" padding.
const kv = authCookie.split(";")[0];
const token = kv.slice(kv.indexOf("=") + 1);
return token;
return extractAuthTokenFromSetCookie(response.headers["set-cookie"]);
} catch (error: any) {
// Avoid leaking the full server response body by default; log only the
// HTTP status. Log the verbose body only when DEBUG is set.

View File

@@ -0,0 +1,113 @@
// Detection + collection of INTERNAL Docmost file URLs inside a ProseMirror doc.
//
// An internal file URL is a relative path served by Docmost's authenticated
// attachment route (`GET /api/files/:fileId/:fileName`). It is useless to an
// external consumer (relative + needs a Docmost session), so the stash tool
// mirrors every such resource into the blob sandbox and rewrites its `src`.
//
// The criterion is "internal file URL", NOT the node TYPE: image, drawio,
// excalidraw, video and file nodes all carry such a `src`, so a type-agnostic
// walker covers them all. External http(s) srcs (CDNs) are left untouched.
//
// Mirrors editor-ext's isInternalFileUrl / normalizeFileUrl (kept as a local
// dup so the ESM mcp package does not depend on the editor-ext build).
function isInternalFileUrl(url: unknown): boolean {
if (typeof url !== "string") return false;
const normalized = url.trim();
return (
normalized.startsWith("/api/files/") || normalized.startsWith("/files/")
);
}
/** Normalize a bare `/files/...` src to the canonical `/api/files/...` form. */
export function normalizeFileUrl(src: string): string {
const trimmed = src.trim();
if (trimmed.startsWith("/files/")) return "/api" + trimmed;
return trimmed;
}
/**
* Resolve a page-content `src` into the safe, `/api`-relative path the stash
* tool may fetch over the authenticated loopback client — or THROW.
*
* SECURITY (SSRF / path-traversal): `src` comes from page content and is fully
* attacker-controllable. The mirroring fetch runs through the AUTHENTICATED
* loopback axios client whose baseURL ends in `/api`, so a naive
* `src.replace(/^\/api/, "")` lets a crafted value like
* `/api/files/../auth/whoami` collapse (via axios/WHATWG URL `..` resolution)
* into an ARBITRARY internal GET endpoint, whose authed response would then be
* stored in the anonymous sandbox (SSRF + data exfiltration). A prefix-only
* `startsWith("/api/files/")` check does NOT defend against this because the
* `..` segments are still present in the raw string and resolved later.
*
* This function defeats that by resolving the canonical pathname FIRST and only
* then asserting it still lives under `/api/files/`:
* - it rejects any percent-encoded dot/slash (`%2e` / `%2f`): the WHATWG URL
* parser collapses LITERAL `../` but does NOT decode `%2f` separators, so a
* content-controlled src must never be allowed to smuggle those past the
* canonicalization;
* - it resolves `new URL(trimmed, "http://internal.invalid").pathname`, which
* normalizes `..`/`.` segments (e.g. `/api/files/../auth/whoami` →
* `/api/auth/whoami`);
* - it then requires the canonical pathname to start with `/api/files/`, so a
* traversal that escaped that subtree is rejected.
*
* Returns the path RELATIVE to the `/api` base (e.g. `/files/<id>/<name>`),
* ready to hand to the loopback client. The throw happens BEFORE any network
* call, so a rejected src is counted as a failed mirror and its original src is
* kept (the per-image try/catch in stashPage never aborts the whole document).
*/
export function resolveInternalFilePath(src: string): string {
const trimmed = src.trim();
// Percent-encoded dot/slash must never reach the URL canonicalizer: the
// WHATWG parser does NOT decode `%2f` into a path separator, so an encoded
// `..%2fauth` would survive canonicalization and still escape /api/files/.
if (/%2e|%2f/i.test(trimmed)) {
throw new Error(
`Refusing internal file src with percent-encoded path segment: "${src}"`,
);
}
let pathname: string;
try {
// The base host is irrelevant (never contacted); it only lets the parser
// resolve a relative `src` and normalize `..`/`.` segments.
pathname = new URL(trimmed, "http://internal.invalid").pathname;
} catch {
throw new Error(`Invalid internal file src: "${src}"`);
}
if (!pathname.startsWith("/api/files/")) {
throw new Error(
`Refusing internal file src that escapes /api/files/: "${src}"`,
);
}
// Strip the `/api` base prefix; the loopback client's baseURL already ends
// in `/api`, so it expects the path relative to that (e.g. /files/<id>/<f>).
return pathname.replace(/^\/api/, "");
}
/**
* Recursively collect every node whose `attrs.src` is an internal file URL.
* Returns references to the live nodes (so the caller can rewrite `attrs.src`
* in place on its clone). Descends `content` arrays, covering callouts, tables,
* details and any other nested container.
*/
export function collectInternalFileNodes(doc: unknown): any[] {
const out: any[] = [];
const visit = (node: any): void => {
if (!node) return;
if (Array.isArray(node)) {
for (const child of node) visit(child);
return;
}
if (typeof node !== "object") return;
if (node.attrs && isInternalFileUrl(node.attrs.src)) {
out.push(node);
}
if (Array.isArray(node.content)) {
for (const child of node.content) visit(child);
}
};
visit(doc);
return out;
}

View File

@@ -266,4 +266,29 @@ export const SHARED_TOOL_SPECS = {
.describe('List of find/replace operations, applied in order'),
}),
},
// --- hand a large page to an external consumer without bloating context ---
stashPage: {
mcpName: 'stash_page',
inAppKey: 'stashPage',
description:
'Serialize a whole page (the full ProseMirror JSON, as get_page_json ' +
'returns) into an ephemeral in-memory blob and return ONLY a short ' +
'anonymous URL to it — the body NEVER enters the model context, so this ' +
'is the way to hand a large page (or its images) to an external consumer ' +
'without truncation. Every internal file/image attachment is mirrored ' +
'into the same sandbox and its src rewritten to a sandbox URL, so the ' +
'consumer can fetch the images anonymously too; external http(s) images ' +
'are left untouched. Returns { uri, size, sha256, images:{mirrored, ' +
'failed} }. Integrity: the blob is served with ETag = its sha256, so a ' +
'truncated/corrupted fetch is detectable. Blobs are RAM-only: they expire ' +
'after a short TTL (~1h) and are cleared on restart — consume the URL ' +
'within the TTL and one uptime, or re-stash. A blob is bound to the ' +
'server instance that created it: in a multi-replica deployment without ' +
'sticky sessions a blob stored on one instance is not retrievable via the ' +
'sandbox URL on another (it 404s like an expired one).',
buildShape: (z) => ({
pageId: z.string().min(1),
}),
},
} satisfies Record<string, SharedToolSpec>;

View File

@@ -0,0 +1,155 @@
// Server round-trip test for the stash_page MCP tool result shape. The in-app
// path returns the full documented `{ uri, size, sha256, images }` object, but
// the MCP transport must deliver the SAME shape: a resource_link (primary
// payload) PLUS a `structuredContent` mirror carrying sha256 + image counts.
// This connects a real MCP Client to the server over a linked in-memory
// transport pair and asserts both halves of the result, end to end.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { createHash } from "node:crypto";
import { createDocmostMcpServer } from "../../build/index.js";
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js";
function readBody(req) {
return new Promise((resolve) => {
let raw = "";
req.on("data", (c) => (raw += c));
req.on("end", () => resolve(raw));
});
}
function startServer(handler) {
return new Promise((resolve) => {
const server = http.createServer(handler);
server.listen(0, "127.0.0.1", () => {
const { port } = server.address();
resolve({ server, baseURL: `http://127.0.0.1:${port}/api` });
});
});
}
const openServers = [];
async function spawn(handler) {
const { server, baseURL } = await startServer(handler);
openServers.push(server);
return baseURL;
}
after(async () => {
await Promise.all(openServers.map((s) => new Promise((r) => s.close(r))));
});
// Minimal in-memory sandbox sink: store the blob and return a uri + sha256 +
// size, with has/evict probes the client's reconciliation may call.
function makeSandbox() {
const live = new Map();
const idOf = (uri) => uri.substring(uri.lastIndexOf("/") + 1);
let n = 0;
return {
put(buf) {
const sha256 = createHash("sha256").update(buf).digest("hex");
const id = `id-${n++}`;
live.set(id, buf.length);
return { uri: `https://sb.test/api/sb/${id}`, sha256, size: buf.length };
},
has(uri) {
return live.has(idOf(uri));
},
evict(uri) {
live.delete(idOf(uri));
},
};
}
const IMAGE_BYTES = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a]);
// One internal image (so images.mirrored === 1) inside a normal page doc.
function pageDoc() {
return {
type: "doc",
content: [
{
type: "image",
attrs: { src: "/api/files/att-1/pic.png", attachmentId: "att-1" },
},
],
};
}
// Mock Docmost: login, page info, internal file bytes — same pattern as
// stash-page.test.mjs.
async function buildBaseURL() {
return spawn(async (req, res) => {
await readBody(req);
if (req.url === "/api/auth/login") {
res.writeHead(200, {
"Content-Type": "application/json",
"Set-Cookie": "authToken=tok; HttpOnly",
});
res.end(JSON.stringify({ token: "tok" }));
return;
}
if (req.url === "/api/pages/info") {
res.writeHead(200, { "Content-Type": "application/json" });
res.end(
JSON.stringify({ data: { id: "page-1", title: "T", content: pageDoc() } }),
);
return;
}
if (req.url.startsWith("/api/files/")) {
res.writeHead(200, { "Content-Type": "image/png" });
res.end(IMAGE_BYTES);
return;
}
res.writeHead(404);
res.end();
});
}
test("stash_page MCP tool returns a resource_link AND a structuredContent mirror", async () => {
const baseURL = await buildBaseURL();
const sandbox = makeSandbox();
const server = createDocmostMcpServer({
apiUrl: baseURL,
email: "u@example.com",
password: "pw",
sandbox,
});
const client = new Client({ name: "test-client", version: "0.0.0" });
const [a, b] = InMemoryTransport.createLinkedPair();
await server.connect(b);
await client.connect(a);
try {
const res = await client.callTool({
name: "stash_page",
arguments: { pageId: "page-1" },
});
// Primary payload: a resource_link pointing at the sandbox doc blob.
const link = res.content[0];
assert.equal(link.type, "resource_link");
assert.match(link.uri, /^https:\/\/sb\.test\/api\/sb\//);
// structuredContent mirrors the full documented shape.
const sc = res.structuredContent;
assert.equal(typeof sc, "object");
assert.equal(sc.uri, link.uri); // same blob as the link
assert.match(sc.sha256, /^[0-9a-f]{64}$/); // 64-hex ETag
assert.equal(typeof sc.size, "number");
assert.deepEqual(sc.images, { mirrored: 1, failed: 0 });
// Deep-equal the whole structured payload against what the mock implies.
assert.deepEqual(sc, {
uri: link.uri,
sha256: sc.sha256,
size: sc.size,
images: { mirrored: 1, failed: 0 },
});
} finally {
await client.close();
await server.close();
}
});

View File

@@ -0,0 +1,378 @@
// Mock-HTTP test for DocmostClient.stashPage: a local http server stands in for
// Docmost so the whole flow stays deterministic and offline. Asserts the tool
// (1) serializes the page into the sandbox and returns ONLY a link (uri + sha256
// + size), never the body; (2) mirrors INTERNAL image srcs into the sandbox and
// rewrites them to the sandbox uri; (3) leaves EXTERNAL http(s) srcs untouched;
// (4) de-duplicates a repeated internal src to a single blob; (5) counts a
// failed image fetch without aborting the document.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { createHash } from "node:crypto";
import { DocmostClient } from "../../build/client.js";
function readBody(req) {
return new Promise((resolve) => {
let raw = "";
req.on("data", (c) => (raw += c));
req.on("end", () => resolve(raw));
});
}
function startServer(handler) {
return new Promise((resolve) => {
const server = http.createServer(handler);
server.listen(0, "127.0.0.1", () => {
const { port } = server.address();
resolve({ server, baseURL: `http://127.0.0.1:${port}/api` });
});
});
}
const openServers = [];
async function spawn(handler) {
const { server, baseURL } = await startServer(handler);
openServers.push(server);
return baseURL;
}
after(async () => {
await Promise.all(openServers.map((s) => new Promise((r) => s.close(r))));
});
// In-memory sandbox sink mirroring the host binding: store the blob, return a
// uri + sha256 + size. Records every put so the test can inspect what was
// stashed (and verify the doc body never leaves via the return value). Models
// the real store's FIFO eviction + cap + the has/evict probes so B1 (self-
// eviction reconciliation and doc-put-throw cleanup) is testable. Default
// maxTotal is effectively unlimited so the happy-path tests behave as before.
//
// `throwOnJson` forces the final document put to throw, standing in for "doc
// exceeds the cap".
function makeSandbox({ maxTotal = Infinity, throwOnJson = false } = {}) {
const puts = [];
const evicted = [];
// id -> size, in insertion order (Map preserves it) so the oldest is first.
const live = new Map();
let total = 0;
const idOf = (uri) => uri.substring(uri.lastIndexOf("/") + 1);
return {
puts,
evicted,
put(buf, mime) {
if (throwOnJson && mime === "application/json") {
throw new Error("doc blob exceeds the sandbox cap");
}
const sha256 = createHash("sha256").update(buf).digest("hex");
const id = `id-${puts.length}`;
puts.push({ buf, mime, sha256, id });
live.set(id, buf.length);
total += buf.length;
// FIFO-evict the oldest live blobs until this put fits under the cap.
while (total > maxTotal && live.size > 0) {
const oldest = live.keys().next().value;
if (oldest === id) break; // never evict the blob we just stored
total -= live.get(oldest);
live.delete(oldest);
evicted.push(oldest);
}
return { uri: `https://sb.test/api/sb/${id}`, sha256, size: buf.length };
},
has(uri) {
return live.has(idOf(uri));
},
evict(uri) {
const id = idOf(uri);
if (live.has(id)) {
total -= live.get(id);
live.delete(id);
}
evicted.push(id);
},
};
}
const IMAGE_BYTES = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a]); // "PNG" header-ish
function pageDoc() {
return {
type: "doc",
content: [
{
type: "image",
attrs: { src: "/api/files/att-1/pic.png", attachmentId: "att-1", width: 100 },
},
// Same internal src again -> must dedup to ONE blob, both rewritten.
{
type: "image",
attrs: { src: "/api/files/att-1/pic.png", attachmentId: "att-1", width: 50 },
},
// External CDN image -> must be left untouched.
{
type: "image",
attrs: { src: "https://cdn.example.com/remote.png" },
},
],
};
}
// Build a client wired to a server that logs in, serves the page, and serves the
// internal file bytes. `fileStatus` lets a test force the file fetch to fail;
// `doc` overrides the served page; `fileBytes`/`fileHeaders` shape the file
// response (used by the empty-body / missing-Content-Type branch tests).
async function buildClient(
sandbox,
{
fileStatus = 200,
doc = pageDoc(),
fileBytes = IMAGE_BYTES,
fileHeaders = { "Content-Type": "image/png" },
} = {},
) {
const baseURL = await spawn(async (req, res) => {
await readBody(req);
if (req.url === "/api/auth/login") {
res.writeHead(200, {
"Content-Type": "application/json",
"Set-Cookie": "authToken=tok; HttpOnly",
});
res.end(JSON.stringify({ token: "tok" }));
return;
}
if (req.url === "/api/pages/info") {
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ data: { id: "page-1", title: "T", content: doc } }));
return;
}
if (req.url.startsWith("/api/files/")) {
if (fileStatus !== 200) {
res.writeHead(fileStatus);
res.end();
return;
}
res.writeHead(200, fileHeaders);
res.end(fileBytes);
return;
}
res.writeHead(404);
res.end();
});
return new DocmostClient({
apiUrl: baseURL,
email: "u@example.com",
password: "pw",
sandbox: {
put: (buf, mime) => sandbox.put(buf, mime),
has: (uri) => sandbox.has(uri),
evict: (uri) => sandbox.evict(uri),
},
});
}
// A page with several DISTINCT internal images (each a unique attachment id) so
// each is its own sandbox blob — needed to exercise FIFO self-eviction.
function multiImageDoc(n) {
return {
type: "doc",
content: Array.from({ length: n }, (_, i) => ({
type: "image",
attrs: { src: `/api/files/att-${i}/pic.png`, attachmentId: `att-${i}` },
})),
};
}
test("stashPage stores the doc + mirrors/rewrites internal images, returns only a link", async () => {
const sandbox = makeSandbox();
const client = await buildClient(sandbox);
const result = await client.stashPage("page-1");
// Returns ONLY a link shape — never the document body.
assert.equal(typeof result.uri, "string");
assert.match(result.uri, /^https:\/\/sb\.test\/api\/sb\//);
assert.equal(typeof result.sha256, "string");
assert.equal(typeof result.size, "number");
assert.ok(!("doc" in result) && !("content" in result) && !("body" in result));
assert.deepEqual(result.images, { mirrored: 1, failed: 0 });
// One image blob (dedup) + one doc blob = 2 puts.
assert.equal(sandbox.puts.length, 2);
const imagePut = sandbox.puts[0];
const docPut = sandbox.puts[1];
assert.equal(imagePut.mime, "image/png");
assert.ok(imagePut.buf.equals(IMAGE_BYTES));
assert.equal(docPut.mime, "application/json");
// The returned uri/sha256 are the DOCUMENT blob's.
assert.equal(result.sha256, docPut.sha256);
// Inspect the stashed document: internal srcs rewritten, external untouched.
const stashed = JSON.parse(docPut.buf.toString("utf8"));
const imgs = stashed.content.content.filter((n) => n.type === "image");
assert.equal(imgs[0].attrs.src, "https://sb.test/api/sb/id-0");
assert.equal(imgs[1].attrs.src, "https://sb.test/api/sb/id-0"); // same blob (dedup)
assert.equal(imgs[2].attrs.src, "https://cdn.example.com/remote.png"); // external kept
});
test("stashPage counts a failed image fetch without aborting the document", async () => {
const sandbox = makeSandbox();
const client = await buildClient(sandbox, { fileStatus: 500 });
const result = await client.stashPage("page-1");
assert.deepEqual(result.images, { mirrored: 0, failed: 1 });
// Only the doc blob was stored (image fetch failed).
assert.equal(sandbox.puts.length, 1);
assert.equal(sandbox.puts[0].mime, "application/json");
// The failed internal src is LEFT as-is so nothing is silently dropped.
const stashed = JSON.parse(sandbox.puts[0].buf.toString("utf8"));
const imgs = stashed.content.content.filter((n) => n.type === "image");
assert.equal(imgs[0].attrs.src, "/api/files/att-1/pic.png");
});
test("stashPage throws a clear error when no sandbox is configured", async () => {
const baseURL = await spawn(async (req, res) => {
await readBody(req);
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({}));
});
const client = new DocmostClient({
apiUrl: baseURL,
email: "u@example.com",
password: "pw",
});
await assert.rejects(() => client.stashPage("page-1"), /not configured/);
});
test("stashPage reverts a FIFO-evicted image and counts it as failed (B1)", async () => {
// 3 distinct images of S=4000 bytes each; doc JSON is far smaller than one
// image. With a cap of 4500: storing img1 evicts img0, storing img2 evicts
// img1 — so only img2 survives the loop (img0 + img1 reverted). The doc
// (4000 + a few hundred bytes <= 4500) then fits alongside the survivor, so it
// does NOT trigger further eviction. The stored doc must therefore reference
// exactly one live blob and revert the other two to their internal srcs.
const BIG = Buffer.alloc(4000, 0x41);
const sandbox = makeSandbox({ maxTotal: 4500 });
const client = await buildClient(sandbox, {
doc: multiImageDoc(3),
fileBytes: BIG,
});
const result = await client.stashPage("page-1");
// Two images were evicted before the doc was stored -> counted as failed.
assert.deepEqual(result.images, { mirrored: 1, failed: 2 });
// Inspect the stashed doc: no node may point at an evicted (now-dead) blob,
// and every reverted node carries its ORIGINAL internal src again.
const docPut = sandbox.puts.find((p) => p.mime === "application/json");
const stashed = JSON.parse(docPut.buf.toString("utf8"));
const imgs = stashed.content.content.filter((n) => n.type === "image");
let live = 0;
let reverted = 0;
for (const img of imgs) {
const src = img.attrs.src;
if (src.startsWith("https://sb.test/api/sb/")) {
assert.ok(sandbox.has(src), `doc references evicted blob ${src}`);
live++;
} else {
// Reverted to the original internal src.
assert.match(src, /^\/api\/files\/att-\d+\/pic\.png$/);
reverted++;
}
}
assert.equal(live, 1);
assert.equal(reverted, 2);
});
test("stashPage reverts an image evicted by the DOC put itself (after-put reconcile, B1)", async () => {
// Both images (1000 bytes each) survive the image phase: total 2000 <= cap
// 2500. The doc, however, serializes large (a node with a ~700-byte string
// attr), so putting it (newest) tips total over the cap and FIFO-evicts the
// OLDEST image (img0) — an eviction caused by the doc put itself, which only
// the after-put reconciliation can catch. The loop then reverts img0, drops
// the stale doc blob, and re-puts the corrected doc (now total = img1 +
// docSize <= cap, so img1 survives).
const BIG = Buffer.alloc(1000, 0x41);
const sandbox = makeSandbox({ maxTotal: 2500 });
const doc = {
type: "doc",
content: [
{ type: "image", attrs: { src: "/api/files/att-0/pic.png", attachmentId: "att-0" } },
{ type: "image", attrs: { src: "/api/files/att-1/pic.png", attachmentId: "att-1" } },
// Bulk the doc JSON up so the doc put crosses the cap on its own. Stays in
// the doc across reverts, so each re-serialization is similarly large.
{ type: "paragraph", attrs: { filler: "x".repeat(700) }, content: [] },
],
};
const client = await buildClient(sandbox, { doc, fileBytes: BIG });
const result = await client.stashPage("page-1");
// The doc put evicted exactly one image -> reverted + counted as failed.
assert.deepEqual(result.images, { mirrored: 1, failed: 1 });
// Use the LAST json put: the first (stale) doc referenced the now-dead blob
// and was itself evicted; the corrected re-put is the one that stands.
const docPut = sandbox.puts.filter((p) => p.mime === "application/json").at(-1);
const stashed = JSON.parse(docPut.buf.toString("utf8"));
const imgs = stashed.content.content.filter((n) => n.type === "image");
let live = 0;
let reverted = 0;
for (const img of imgs) {
const src = img.attrs.src;
if (src.startsWith("https://sb.test/api/sb/")) {
assert.ok(sandbox.has(src), `final doc references evicted blob ${src}`);
live++;
} else {
assert.match(src, /^\/api\/files\/att-\d+\/pic\.png$/);
reverted++;
}
}
assert.equal(live, 1);
assert.equal(reverted, 1);
});
test("stashPage frees image blobs when the doc put throws (B1)", async () => {
// Two distinct images mirror fine; the final JSON doc put throws (doc exceeds
// cap). stashPage must reject AND evict every image blob it stored this op.
const sandbox = makeSandbox({ throwOnJson: true });
const client = await buildClient(sandbox, { doc: multiImageDoc(2) });
await assert.rejects(() => client.stashPage("page-1"));
// Both image blobs were stored, then evicted on the doc-put failure.
const imagePuts = sandbox.puts.filter((p) => p.mime === "image/png");
assert.equal(imagePuts.length, 2);
for (const p of imagePuts) {
assert.ok(sandbox.evicted.includes(p.id), `image ${p.id} was not freed`);
}
});
test("stashPage counts an empty file response as failed (B1/fetchInternalFile)", async () => {
const sandbox = makeSandbox();
const client = await buildClient(sandbox, {
fileBytes: Buffer.alloc(0),
fileHeaders: { "Content-Type": "image/png", "Content-Length": "0" },
});
const result = await client.stashPage("page-1");
// The single internal image (deduped) yielded an empty body -> failed.
assert.deepEqual(result.images, { mirrored: 0, failed: 1 });
// Only the doc blob was stored.
assert.equal(sandbox.puts.filter((p) => p.mime === "image/png").length, 0);
});
test("stashPage mirrors a file with no Content-Type as octet-stream (fetchInternalFile)", async () => {
const sandbox = makeSandbox();
// No Content-Type header at all -> fetchInternalFile defaults to octet-stream.
const client = await buildClient(sandbox, { fileHeaders: {} });
const result = await client.stashPage("page-1");
assert.equal(result.images.mirrored, 1);
const imagePut = sandbox.puts.find((p) => p.mime !== "application/json");
assert.ok(imagePut, "expected an image put");
assert.equal(imagePut.mime, "application/octet-stream");
});

View File

@@ -0,0 +1,93 @@
// Cookie parsing for the login flow.
//
// `performLogin` in auth-utils.ts does a real network POST and then extracts the
// auth token from the response's Set-Cookie header. The cookie-parsing logic was
// extracted into the pure, exported helper `extractAuthTokenFromSetCookie` so it
// can be tested without network I/O; `performLogin` now delegates to it, so these
// tests cover the exact parsing path the login uses.
import { test } from "node:test";
import assert from "node:assert/strict";
import { extractAuthTokenFromSetCookie } from "../../build/lib/auth-utils.js";
// ---------------------------------------------------------------------------
// Happy path: a single authToken cookie with attributes.
// ---------------------------------------------------------------------------
test("extracts the authToken value, ignoring trailing attributes", () => {
const cookies = [
"authToken=abc123; Path=/; HttpOnly; Secure; SameSite=Lax",
];
assert.equal(extractAuthTokenFromSetCookie(cookies), "abc123");
});
// ---------------------------------------------------------------------------
// A base64/JWT value containing "=" padding must NOT be truncated: only the
// FIRST "=" separates name from value.
// ---------------------------------------------------------------------------
test("preserves an '=' inside the value (base64 padding is not truncated)", () => {
const jwt = "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxIn0=";
const cookies = [`authToken=${jwt}; Path=/`];
assert.equal(extractAuthTokenFromSetCookie(cookies), jwt);
});
// ---------------------------------------------------------------------------
// Exact-name match: a different cookie whose name merely STARTS WITH "authToken"
// (e.g. authTokenRefresh) must not be picked up; the real authToken wins.
// ---------------------------------------------------------------------------
test("matches the cookie name exactly, not by prefix (authTokenRefresh ignored)", () => {
const cookies = [
"authTokenRefresh=refreshvalue; Path=/; HttpOnly",
"authToken=realtoken; Path=/; HttpOnly",
];
assert.equal(extractAuthTokenFromSetCookie(cookies), "realtoken");
});
// ---------------------------------------------------------------------------
// Picks the authToken out of several unrelated cookies regardless of order.
// ---------------------------------------------------------------------------
test("selects authToken among multiple unrelated cookies", () => {
const cookies = [
"session=xyz; Path=/",
"authToken=tok-7; Path=/; HttpOnly",
"theme=dark",
];
assert.equal(extractAuthTokenFromSetCookie(cookies), "tok-7");
});
// ---------------------------------------------------------------------------
// An empty value is valid and returns "".
// ---------------------------------------------------------------------------
test("returns an empty string when authToken has an empty value", () => {
assert.equal(extractAuthTokenFromSetCookie(["authToken=; Path=/"]), "");
});
// ---------------------------------------------------------------------------
// Missing Set-Cookie header -> documented error.
// ---------------------------------------------------------------------------
test("throws when there is no Set-Cookie header", () => {
assert.throws(
() => extractAuthTokenFromSetCookie(undefined),
/No Set-Cookie header/,
);
});
// ---------------------------------------------------------------------------
// Set-Cookie present but no authToken cookie -> documented error.
// ---------------------------------------------------------------------------
test("throws when no authToken cookie is present", () => {
assert.throws(
() => extractAuthTokenFromSetCookie(["session=xyz; Path=/", "theme=dark"]),
/No authToken cookie/,
);
});
// ---------------------------------------------------------------------------
// An empty cookie array also yields the "no authToken" error (header exists but
// is empty), distinct from the "no Set-Cookie header" case above.
// ---------------------------------------------------------------------------
test("throws 'no authToken' (not 'no header') for an empty cookie array", () => {
assert.throws(
() => extractAuthTokenFromSetCookie([]),
/No authToken cookie/,
);
});

View File

@@ -0,0 +1,111 @@
// applyAnchorInDoc — first-match / ambiguity / boundary behavior.
//
// comment-anchor.test.mjs already covers the core apply paths (single-node
// match, spanning adjacent text nodes, code/italic boundary mark preservation,
// smart-quote normalization, no-match-no-mutation, pre-existing comment mark
// replacement, nested-list DFS). This file focuses on the SELECTION/RESOLUTION
// behavior those tests don't pin down: which occurrence/block wins when a
// selection appears more than once, sub-word ranges, and the run boundary
// created by a non-text inline node.
import { test } from "node:test";
import assert from "node:assert/strict";
import { applyAnchorInDoc, canAnchorInDoc } from "../../build/lib/comment-anchor.js";
const commentMark = (node) =>
(Array.isArray(node.marks) ? node.marks : []).find((m) => m && m.type === "comment") || null;
const paragraphDoc = (content) => ({ type: "doc", content: [{ type: "paragraph", content }] });
// ---------------------------------------------------------------------------
// Document order: when two separate blocks both contain the selection, only the
// FIRST block (DFS document order) is anchored; the second is left untouched.
// ---------------------------------------------------------------------------
test("anchors only the FIRST block when the selection occurs in two blocks", () => {
const doc = {
type: "doc",
content: [
{ type: "paragraph", content: [{ type: "text", text: "first target here" }] },
{ type: "paragraph", content: [{ type: "text", text: "second target here" }] },
],
};
assert.equal(applyAnchorInDoc(doc, "target", "C"), true);
const marked0 = doc.content[0].content.filter((p) => commentMark(p));
const marked1 = doc.content[1].content.filter((p) => commentMark(p));
assert.equal(marked0.length, 1, "first block is anchored");
assert.equal(marked0[0].text, "target");
assert.equal(marked1.length, 0, "second block is left untouched");
});
// ---------------------------------------------------------------------------
// Ambiguity within one block: indexOf finds the FIRST occurrence, so only the
// first "ab" is marked; the later occurrences stay in one unmarked fragment.
// ---------------------------------------------------------------------------
test("anchors only the FIRST occurrence within a block (ambiguous selection)", () => {
const doc = paragraphDoc([{ type: "text", text: "ab ab ab" }]);
assert.equal(applyAnchorInDoc(doc, "ab", "C"), true);
const parts = doc.content[0].content;
assert.equal(parts.length, 2, "split into [marked, rest]");
assert.equal(parts[0].text, "ab");
assert.ok(commentMark(parts[0]), "first occurrence is marked");
assert.equal(parts[1].text, " ab ab");
assert.equal(commentMark(parts[1]), null, "later occurrences are not marked");
});
// ---------------------------------------------------------------------------
// Sub-word range: a selection that is a substring inside a single text node is
// spliced into before / marked / after, marking exactly the matched characters.
// ---------------------------------------------------------------------------
test("anchors a sub-word range inside a single text node", () => {
const doc = paragraphDoc([{ type: "text", text: "Hello" }]);
assert.equal(applyAnchorInDoc(doc, "ell", "C"), true);
const parts = doc.content[0].content;
assert.deepEqual(parts.map((p) => p.text), ["H", "ell", "o"]);
assert.equal(commentMark(parts[0]), null);
assert.ok(commentMark(parts[1]), "only the matched substring is marked");
assert.equal(commentMark(parts[2]), null);
});
// ---------------------------------------------------------------------------
// A non-text inline node (hardBreak) breaks the matching run: a selection that
// would span the break cannot match, but one wholly inside a run still does.
// ---------------------------------------------------------------------------
test("a non-text inline node breaks the run: cross-break selection does not match", () => {
const make = () =>
paragraphDoc([
{ type: "text", text: "foo" },
{ type: "hardBreak" },
{ type: "text", text: "bar" },
]);
// "foobar" straddles the hardBreak -> no match, no mutation.
const docA = make();
const before = JSON.stringify(docA);
assert.equal(canAnchorInDoc(docA, "foobar"), false);
assert.equal(applyAnchorInDoc(docA, "foobar", "C"), false);
assert.equal(JSON.stringify(docA), before, "failed match must not mutate");
// "foo" lives entirely in the first run -> matches and is marked; the
// hardBreak node is preserved untouched.
const docB = make();
assert.equal(applyAnchorInDoc(docB, "foo", "C"), true);
const parts = docB.content[0].content;
assert.equal(parts[0].text, "foo");
assert.ok(commentMark(parts[0]));
assert.equal(parts[1].type, "hardBreak", "the inline atom is preserved");
assert.equal(parts[2].text, "bar");
assert.equal(commentMark(parts[2]), null);
});
// ---------------------------------------------------------------------------
// A whitespace-only selection normalizes to empty and never anchors.
// ---------------------------------------------------------------------------
test("a whitespace-only selection does not anchor and does not mutate", () => {
const doc = paragraphDoc([{ type: "text", text: "hello world" }]);
const before = JSON.stringify(doc);
assert.equal(canAnchorInDoc(doc, " "), false);
assert.equal(applyAnchorInDoc(doc, " ", "C"), false);
assert.equal(JSON.stringify(doc), before);
});

View File

@@ -0,0 +1,101 @@
// Unit tests for the internal-file URL helpers the stash tool relies on. The
// critical case is resolveInternalFilePath, whose whole job is to REJECT a
// content-controlled `src` that tries to escape /api/files/ (SSRF / traversal)
// before it ever reaches the authenticated loopback client.
import { test } from "node:test";
import assert from "node:assert/strict";
import {
resolveInternalFilePath,
normalizeFileUrl,
collectInternalFileNodes,
} from "../../build/lib/internal-file-urls.js";
test("resolveInternalFilePath accepts a normal internal src", () => {
assert.equal(
resolveInternalFilePath("/api/files/att-1/pic.png"),
"/files/att-1/pic.png",
);
});
test("resolveInternalFilePath rejects traversal / encoded variants (SSRF guard)", () => {
// `..` collapses to /api/auth/whoami -> outside /api/files/ -> rejected.
assert.throws(() => resolveInternalFilePath("/api/files/../auth/whoami"));
// Escapes the /api base entirely.
assert.throws(() => resolveInternalFilePath("/api/files/../../internal"));
// Percent-encoded dot -> rejected before canonicalization.
assert.throws(() => resolveInternalFilePath("/api/files/%2e%2e/x"));
// Percent-encoded slash separator -> rejected before canonicalization.
assert.throws(() => resolveInternalFilePath("/api/files/..%2fauth"));
});
test("resolveInternalFilePath drops a foreign host and keeps only the /api/files/ pathname (SSRF accept-path)", () => {
// ACCEPT path: an absolute URL has its host dropped; only the canonical
// pathname survives, and it must still start with /api/files/. This is SAFE
// because the loopback axios client ignores any host in `src` and uses its own
// /api baseURL — so a foreign host like evil.com is never contacted. This is
// the SOLE SSRF/traversal guard for content-controlled `src`, so it must be
// pinned: a future refactor to a prefix-only check would silently open a
// bypass with no failing test.
assert.equal(
resolveInternalFilePath("http://evil.com/api/files/x/y.png"),
"/files/x/y.png",
);
// Protocol-relative URL: host likewise dropped, pathname kept.
assert.equal(
resolveInternalFilePath("//evil.com/api/files/x/y.png"),
"/files/x/y.png",
);
});
test("resolveInternalFilePath rejects a foreign-host src whose pathname escapes /api/files/", () => {
// Even though the host is dropped, the canonical pathname /api/auth/whoami
// does NOT start with /api/files/, so it is rejected.
assert.throws(() =>
resolveInternalFilePath("https://evil.com/api/auth/whoami"),
);
// The WHATWG URL parser converts backslashes to `/` for http(s), so this
// collapses to /api/auth/whoami and escapes the /api/files/ subtree.
assert.throws(() => resolveInternalFilePath("/api/files\\..\\auth\\whoami"));
});
test("resolveInternalFilePath wraps a new URL parse failure in a clear error", () => {
// `http://[` has no %2e/%2f so it passes the first guard, then fails the
// `new URL(...)` parse — exercising the catch branch that re-throws with a
// clear message.
assert.throws(
() => resolveInternalFilePath("http://["),
/Invalid internal file src/,
);
});
test("normalizeFileUrl rewrites the bare /files/ branch and leaves /api/files/ alone", () => {
assert.equal(
normalizeFileUrl("/files/att-1/pic.png"),
"/api/files/att-1/pic.png",
);
assert.equal(
normalizeFileUrl("/api/files/att-1/pic.png"),
"/api/files/att-1/pic.png",
);
});
test("collectInternalFileNodes recurses into nested content containers", () => {
// The internal image is buried inside a callout's content array, so a
// regression on the recursion (e.g. a shallow .filter()) would miss it.
const nested = {
type: "image",
attrs: { src: "/api/files/att-9/deep.png", attachmentId: "att-9" },
};
const doc = {
type: "doc",
content: [
{
type: "callout",
content: [{ type: "paragraph", content: [nested] }],
},
],
};
const found = collectInternalFileNodes(doc);
assert.equal(found.length, 1);
assert.equal(found[0], nested);
});

View File

@@ -0,0 +1,135 @@
// Extra media round-trip coverage (issue #244), complementing
// media-roundtrip.test.mjs.
//
// The existing media-roundtrip.test.mjs already asserts that video, youtube,
// embed, excalidraw, audio and pdf SURVIVE a PM -> markdown -> PM round-trip and
// keeps their identifying src / provider / name / attachmentId. It does NOT,
// however, exercise:
// * the `drawio` node (a distinct schema node that shares the excalidraw
// converter case) — not covered at all;
// * the dimension / layout attributes (width, height, align) that ride in
// data-* attributes — exactly where a converter<->schema mismatch silently
// drops a value while the node itself survives;
// * attribute escaping for a src containing `"` (escapeAttr) — a malformed
// value here would either break the round-trip or inject HTML.
//
// These are the gaps this file locks down.
import { test } from "node:test";
import assert from "node:assert/strict";
import { convertProseMirrorToMarkdown } from "../../build/lib/markdown-converter.js";
import { markdownToProseMirror } from "../../build/lib/collaboration.js";
const doc = (...content) => ({ type: "doc", content });
const findAll = (node, type, acc = []) => {
if (!node || typeof node !== "object") return acc;
if (node.type === type) acc.push(node);
for (const c of node.content || []) findAll(c, type, acc);
return acc;
};
// PM node -> markdown -> PM; return both the markdown and the matching nodes.
const roundtrip = async (node, type) => {
const md = convertProseMirrorToMarkdown(doc(node));
const pm = await markdownToProseMirror(md);
return { md, found: findAll(pm, type) };
};
// ---------------------------------------------------------------------------
// drawio: a separate schema node sharing the excalidraw converter case. Not
// covered by the existing file at all, so guard its full round-trip here.
// ---------------------------------------------------------------------------
test("round-trip: drawio diagram survives with src, title, dimensions, align, attachmentId", async () => {
const { md, found } = await roundtrip(
{
type: "drawio",
attrs: {
src: "/api/files/d.drawio",
title: "Flow",
width: 400,
height: 300,
align: "left",
attachmentId: "dz1",
},
},
"drawio",
);
// The converter must emit the schema-matching div[data-type="drawio"].
assert.match(md, /data-type="drawio"/);
assert.equal(found.length, 1, "drawio node must survive the round-trip");
const a = found[0].attrs;
assert.equal(a.src, "/api/files/d.drawio");
assert.equal(a.title, "Flow");
assert.equal(a.attachmentId, "dz1");
assert.equal(a.align, "left");
// Numeric dimensions come back as strings via the schema parseHTML.
assert.equal(String(a.width), "400");
assert.equal(String(a.height), "300");
});
// ---------------------------------------------------------------------------
// Dimension + align attrs ride in data-* (or width/height) attributes. The
// existing file checks only src/provider/name/attachmentId, so a dropped
// width/height/align would pass there but fail here.
// ---------------------------------------------------------------------------
test("round-trip: youtube preserves width/height/align (data-* attrs)", async () => {
const { found } = await roundtrip(
{ type: "youtube", attrs: { src: "https://youtube.com/watch?v=x", width: 560, height: 315, align: "left" } },
"youtube",
);
assert.equal(found.length, 1);
const a = found[0].attrs;
assert.equal(String(a.width), "560");
assert.equal(String(a.height), "315");
assert.equal(a.align, "left");
});
test("round-trip: embed preserves provider, width/height and align", async () => {
const { found } = await roundtrip(
{ type: "embed", attrs: { src: "https://e.com/x", provider: "iframe", width: 600, height: 480, align: "right" } },
"embed",
);
assert.equal(found.length, 1);
const a = found[0].attrs;
assert.equal(a.provider, "iframe");
assert.equal(String(a.width), "600");
assert.equal(String(a.height), "480");
assert.equal(a.align, "right");
});
test("round-trip: video preserves width/height and align (data-align)", async () => {
const { found } = await roundtrip(
{ type: "video", attrs: { src: "/api/files/v.mp4", attachmentId: "att1", width: 640, height: 360, align: "right" } },
"video",
);
assert.equal(found.length, 1);
const a = found[0].attrs;
assert.equal(String(a.width), "640");
assert.equal(String(a.height), "360");
assert.equal(a.align, "right");
});
test("round-trip: pdf preserves width/height (standard attrs) plus name", async () => {
const { found } = await roundtrip(
{ type: "pdf", attrs: { src: "/api/files/x.pdf", name: "x.pdf", attachmentId: "a4", width: 700, height: 900 } },
"pdf",
);
assert.equal(found.length, 1);
const a = found[0].attrs;
assert.equal(a.name, "x.pdf");
assert.equal(String(a.width), "700");
assert.equal(String(a.height), "900");
});
// ---------------------------------------------------------------------------
// Escaping: a src containing a double quote must survive the attribute-quoted
// HTML emission (escapeAttr) and re-parse to the exact original value, with no
// node loss and no HTML injection.
// ---------------------------------------------------------------------------
test("round-trip: a src containing a double quote is escaped and recovered intact", async () => {
const tricky = 'https://e.com/x?a="b"&c=1';
const { found } = await roundtrip({ type: "youtube", attrs: { src: tricky } }, "youtube");
assert.equal(found.length, 1, "node must survive a quote-bearing src");
assert.equal(found[0].attrs.src, tricky, "the exact src is recovered");
});

View File

@@ -0,0 +1,139 @@
// CONTRACT / DRIFT GUARD: mcp diff vs the vendored editor-ext recreate-transform.
//
// packages/mcp/src/lib/diff.ts computes its document diff with
// `recreateTransform` from the published @fellow/prosemirror-recreate-transform
// package. Docmost's in-app history editor computes the SAME diff with its own
// vendored copy at
// packages/editor-ext/src/lib/recreate-transform/recreateTransform.ts.
// diff.ts's header comment claims the two are "identical" — if they ever drift,
// the headless mcp diff would stop matching what a user sees in the app.
//
// This test guards that claim two ways, on representative doc pairs, using the
// EXACT options diff.ts passes (complexSteps:false, wordDiffs:true,
// simplifyDiff:true):
// 1. invariant: each implementation's transform reproduces the target doc
// (apply(diff) == target);
// 2. cross-copy parity: both implementations emit the SAME step sequence, so a
// behavioral divergence between the two copies fails this test.
//
// The vendored copy is TypeScript, so it is transpiled to CommonJS at test time
// and required directly — the test runs the ACTUAL vendored source, not a stand-in.
import { test, before } from "node:test";
import assert from "node:assert/strict";
import ts from "typescript";
import fs from "node:fs";
import path from "node:path";
import { createRequire } from "node:module";
import { fileURLToPath } from "node:url";
import { recreateTransform as fellowRecreate } from "@fellow/prosemirror-recreate-transform";
import { Node } from "@tiptap/pm/model";
import { docmostSchema } from "../../build/lib/docmost-schema.js";
const require = createRequire(import.meta.url);
const HERE = path.dirname(fileURLToPath(import.meta.url));
// .../packages/mcp/test/unit -> repo packages root.
const PACKAGES = path.resolve(HERE, "..", "..", "..");
const VENDOR_SRC = path.join(
PACKAGES,
"editor-ext",
"src",
"lib",
"recreate-transform",
);
// Emit transpiled CJS under mcp/build so Node resolves the hoisted deps
// (@tiptap/pm, rfc6902, diff) up the directory tree exactly as diff.js does.
const VENDOR_OUT = path.resolve(HERE, "..", "..", "build", "_vendored_editor_ext");
// The exact options the mcp diff pipeline uses (diff.ts).
const DIFF_OPTS = { complexSteps: false, wordDiffs: true, simplifyDiff: true };
let vendoredRecreate;
before(() => {
assert.ok(
fs.existsSync(VENDOR_SRC),
`vendored recreate-transform sources missing at ${VENDOR_SRC}`,
);
fs.rmSync(VENDOR_OUT, { recursive: true, force: true });
fs.mkdirSync(VENDOR_OUT, { recursive: true });
// Mark the output as CommonJS so relative `require("./x")` resolves to x.js.
fs.writeFileSync(
path.join(VENDOR_OUT, "package.json"),
JSON.stringify({ type: "commonjs" }),
);
for (const f of fs.readdirSync(VENDOR_SRC)) {
if (!f.endsWith(".ts")) continue;
const code = fs.readFileSync(path.join(VENDOR_SRC, f), "utf8");
const out = ts.transpileModule(code, {
compilerOptions: {
module: ts.ModuleKind.CommonJS,
target: ts.ScriptTarget.ES2020,
},
});
fs.writeFileSync(path.join(VENDOR_OUT, f.replace(/\.ts$/, ".js")), out.outputText);
}
vendoredRecreate = require(path.join(VENDOR_OUT, "index.js")).recreateTransform;
assert.equal(typeof vendoredRecreate, "function", "vendored recreateTransform loaded");
});
// ---------------------------------------------------------------------------
// Builders + representative doc pairs covering the diff shapes diff.ts handles.
// ---------------------------------------------------------------------------
const t = (text, marks) => (marks ? { type: "text", text, marks } : { type: "text", text });
const para = (...c) => ({ type: "paragraph", content: c });
const doc = (...c) => ({ type: "doc", content: c });
const PAIRS = [
// word inserted mid-sentence
["insert word", doc(para(t("Hello world"))), doc(para(t("Hello brave world")))],
// whole block deleted
["delete block", doc(para(t("keep this")), para(t("remove this"))), doc(para(t("keep this")))],
// word removed mid-sentence
["delete word", doc(para(t("one two three"))), doc(para(t("one three")))],
// pure mark addition (complexSteps:false treats it as a content step)
["add mark", doc(para(t("plain"))), doc(para(t("plain", [{ type: "bold" }])))],
// two blocks swapped (reorder)
["reorder blocks", doc(para(t("a")), para(t("b"))), doc(para(t("b")), para(t("a")))],
// structural insert: an image node appears
[
"insert image",
doc(para(t("caption"))),
doc(para(t("caption")), { type: "image", attrs: { src: "/api/files/a.png", attachmentId: "i1" } }),
],
];
const stepsJSON = (tr) => JSON.stringify(tr.steps.map((s) => s.toJSON()));
for (const [label, fromJSON, toJSON] of PAIRS) {
test(`invariant: @fellow recreateTransform reproduces the target (${label})`, () => {
const from = Node.fromJSON(docmostSchema, fromJSON);
const to = Node.fromJSON(docmostSchema, toJSON);
const tr = fellowRecreate(from, to, DIFF_OPTS);
// apply(diff) == target, comparing schema-normalized JSON on both sides.
assert.equal(JSON.stringify(tr.doc.toJSON()), JSON.stringify(to.toJSON()));
});
test(`drift: @fellow and vendored editor-ext emit identical steps (${label})`, () => {
const mk = () => [
Node.fromJSON(docmostSchema, fromJSON),
Node.fromJSON(docmostSchema, toJSON),
];
const [fA, tA] = mk();
const [fB, tB] = mk();
const trFellow = fellowRecreate(fA, tA, DIFF_OPTS);
const trVendor = vendoredRecreate(fB, tB, DIFF_OPTS);
// Both must reach the same target...
const target = JSON.stringify(tA.toJSON());
assert.equal(JSON.stringify(trFellow.doc.toJSON()), target, "fellow reaches target");
assert.equal(JSON.stringify(trVendor.doc.toJSON()), target, "vendored reaches target");
// ...and, critically, via the SAME step sequence. A divergence in the two
// recreate-transform copies' algorithm would change the steps and fail here.
assert.equal(
stepsJSON(trVendor),
stepsJSON(trFellow),
`vendored editor-ext drifted from @fellow on "${label}"`,
);
});
}