Commit Graph

40 Commits

Author SHA1 Message Date
claude_code b17e1d4a02 fix(offline,server,docs): apply PR #116 review findings to offline-sync
Carries the still-applicable findings from the PR #116 review into PR #120,
since #120 includes the mobile-bootstrap commit. CORS hardening (removing the
unconditional localhost/capacitor origins) is intentionally left out of scope.

Service worker routing (latent bug fix + testability):
- vite.config.ts: anchor Workbox path matching to a segment boundary
  (^/<seg>(/|$)) instead of startsWith, so siblings like /apidocs,
  /collaborators, /socket.iox are no longer mis-routed as API/realtime and
  forced NetworkOnly; align navigateFallbackDenylist with the same anchors.
- new apps/client/src/pwa/sw-strategy.ts holds the canonical predicates
  (isApiPath, isCollabOrSocketPath) + unit tests; the vite.config regexes
  mirror it inline (Workbox generateSW serializes urlPattern fns standalone,
  so they cannot import the module).

Server CORS (R1 extraction + coverage):
- extract buildCorsAllowlist / isOriginAllowed into cors.util.ts with unit
  tests (evil-origin rejected, WebView/no-Origin allowed); main.ts rewired to
  use them with byte-for-byte identical behavior.

Privacy — clear offline cache on logout:
- new clear-offline-cache.ts purges the persisted query cache
  (idb-keyval gitmost-rq-cache), the Yjs page.* IndexedDB databases, and the
  service-worker api-get-cache; wired into handleLogout (best-effort, before
  the redirect) so a previous user's private data does not linger locally.

Conventions & docs:
- prettier fixes on main.ts and login.dto.ts.
- CHANGELOG: document offline reading, returnToken opt-in, optional Swagger,
  new env vars, logout cache-clear, and the CORS open->allowlist breaking
  change.
- docs/mobile-app-plan.md: correct the now-false §2.4 claims and update the
  §12 checklist (native cap add ios left unchecked — generated locally,
  gitignored).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 04:27:10 +03:00
claude_code f9d74a8701 feat(mobile): bootstrap mobile app (PWA + Capacitor + backend auth/CORS)
Implements the §12 bootstrap from docs/mobile-app-plan.md.

Backend (§6):
- auth: optional returnToken flag on login returns the JWT in the body
  (data.authToken) for native Keychain/Keystore + Bearer; web cookie flow
  unchanged.
- main.ts: explicit CORS allowlist (APP_URL + CORS_ALLOWED_ORIGINS env +
  Capacitor WebView origins), credentials enabled, replaces open enableCors().
- optional OpenAPI/Swagger at /api/docs behind SWAGGER_ENABLED.
- env: CORS_ALLOWED_ORIGINS, SWAGGER_ENABLED, CAP_SERVER_URL.

PWA:
- manifest metadata, hand-rolled service worker (network-first nav, SWR
  assets, never intercepts /api,/socket.io,/collab), prod-only registration,
  apple-touch-icon.

Capacitor:
- capacitor.config.ts (webDir apps/client/dist; iOS via CAP_SERVER_URL to
  avoid bundling the AGPL client in the .ipa, see plan §9), cap:* scripts,
  deps, .gitignore for native dirs.
- docs/mobile-bootstrap.md documenting what is done and the remaining manual
  steps (cap add ios/android, APNs/FCM, stores).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-06 04:25:39 +03:00
agent_coder d3209b5aab fix(#355 review E1=B + F1-F8): gate client telemetry OFF by default + throttler/lifecycle/overflow fixes
Maintainer resolved E1 as variant B: the public vitals sink + client collection
must be OFF by default (else client_metrics grows unbounded on a self-host deploy
with no external pruner, via an unauthenticated public endpoint).

- F1: new operator flag CLIENT_TELEMETRY_ENABLED (default OFF), SEPARATE from
  METRICS_PORT (Grafana reads the table directly, independent of the scrape port).
  ClientTelemetryModule.register() provides VitalsController ONLY when the flag is
  true (route absent otherwise); the flag reaches the client via window.CONFIG
  (config.ts isClientTelemetryEnabled), and initVitals() early-returns when off.
- F2/F3 [throttler]: this repo's ThrottlerGuard applies EVERY named throttler to
  every guarded route unless skipped. The new VITALS bucket therefore (a) newly
  bound collab-token → 429 behind shared/NAT IPs, and (b) the vitals route didn't
  skip the stricter public-share-ai (5/min) bucket → effective 5/min not 120.
  Fix (additive, global config unchanged): vitals.controller @SkipThrottle the
  other buckets + @Throttle VITALS 120/min; collab-token adds VITALS_THROTTLER to
  its existing @SkipThrottle (restoring its prior effectively-unthrottled state).
- F4: metrics node:http server is closed on shutdown (MetricsServerLifecycle
  OnModuleDestroy → closeMetricsServer(), fired by enableShutdownHooks).
- F5: docSize outside [0, int4-max] drops to null (keeping the event) instead of
  overflowing int4 and failing the WHOLE batch insert (+ 2 tests).
- F6: .env.example documents METRICS_PORT (no default — unset = subsystem OFF) +
  CLIENT_TELEMETRY_ENABLED; fixed the inaccurate "default 9464" wording.
- F7: disabled/non-sampled sessions install ZERO observers — isVitalsActive()
  (enabled && sampled) gates reportClientMetric AND the page-editor
  measurePageOpen + dispatchTransaction wrapping.
- F8: kept db.d.ts hand-added (wontfix) — this repo HAND-CURATES db.d.ts (verified
  across recent fork migrations a32fba63/8c5b57eb/fdeede00); codegen would be the
  deviation. The ClientMetrics interface maps the migration 1:1.

Gate: server tsc 0, client tsc 0, server metrics/vitals/telemetry/throttle 21
tests, client route-template 5. No new deps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 00:00:03 +03:00
agent_coder b9f3de80f5 feat(observability): dev-side perf metrics — /metrics + client vitals (#355)
The metrics INFRA is already deployed (VictoriaMetrics scraping docmost:9464,
Grafana dashboards, alerts) with a target `gitmost-app` that is red because the
app half didn't exist. This is that half. The contract (metric names, port,
table, endpoint) is FIXED by the deployed infra and matched exactly.

Server (prom-client):
- A bare node:http `/metrics` server on METRICS_PORT (default 9464), SEPARATE
  from the Fastify :3000 listener so /metrics never exists publicly; the whole
  subsystem is OFF when METRICS_PORT is unset.
- collectDefaultMetrics() + http_request_duration_seconds{method,route,status}
  via a Fastify onResponse hook using the ROUTE TEMPLATE (req.routeOptions.url,
  never the raw URL — bounded cardinality; 404 -> "unknown"), EXCLUDING SSE/
  streaming responses (would record the connection lifetime and poison p95).
- db_query_duration_seconds (Kysely log callback, labelled by the leading SQL
  token), bullmq_queue_depth{queue} (getJobCounts every 15s) +
  bullmq_job_duration_seconds{queue} (worker completed/failed),
  collab_store_duration_seconds (around onStoreDocument).
- POST /api/telemetry/vitals — PUBLIC (sendBeacon) but IP-throttled; ~16KB body
  cap, <=50 events/batch, metric-name + rating whitelist, attr truncated to 120
  chars, batch insert; malformed/foreign/oversized silently dropped and 200'd (no
  browser retry). New migration `client_metrics` (schema byte-identical to the
  contract, both indexes, conditional grafana_ro GRANT; no app-side retention —
  the maintenance container prunes >90d).

Client (web-vitals):
- initVitals() decides sampling ONCE per session (25%, sessionStorage) BEFORE
  subscribing; onINP/onLCP/onCLS/onTTFB (attribution) buffered + flushed via
  navigator.sendBeacon on visibilitychange:hidden and a timer (not fetch-per-
  metric). Custom: editor_tx_ms (dispatchTransaction sync-part timer, >8ms, with
  doc_size), page_open_ms, longtask_ms. Route labels are templates only; no
  titles/slugs/text.

Gate: server + client tsc 0, frozen install 0 (added prom-client + web-vitals +
regenerated the lock), server metrics/vitals tests 11, client route-template 5,
and the migration verified valid against real Postgres.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-04 23:10:29 +03:00
claude_code 6eb335d5e3 fix(sandbox): address PR #250 review — SSRF guard, eviction safety, cleanup (#243)
Security:
- stash_page: reject path-traversal / percent-encoded srcs before the authed
  loopback fetch (resolveInternalFilePath), closing an SSRF/exfiltration hole
  where a crafted node.attrs.src could read an arbitrary internal GET endpoint
  into the anonymous sandbox.

Stability:
- stash_page: revert + recount mirrors FIFO-evicted by a later put in the same
  stash (no dangling sandbox refs, honest images.mirrored/failed); free image
  blobs if the final document put throws.
- Reject/clamp non-positive SANDBOX_TTL_MS to the 1h default (warn once).
- Log mirror failures unconditionally (console.warn, no blob bodies).

Cleanup / architecture:
- Remove dead expiresAt from SandboxPutResult.
- Centralize the /api/sb route in SANDBOX_ROUTE_SEGMENT/SANDBOX_API_PATH and
  move URL composition into SandboxStore.putAndLink; drop the duplicated sink
  closures and the now-unused EnvironmentService injection from McpService and
  AiChatToolsService.
- Un-export isInternalFileUrl; document the process-local (instance-bound)
  sandbox limitation in the tool description and .env.example.

Docs/tests:
- README/README.ru: 38 -> 39 tools + stash_page entry.
- Add traversal/normalize/recursion unit tests, stash self-eviction +
  doc-put-throw + empty/octet-stream mock tests, controller If-None-Match
  (wildcard/weak/list) + Cache-Control tests, and SANDBOX_TTL_MS validation
  tests. Regenerate packages/mcp/build.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 18:02:46 +03:00
claude code agent 227 2fe4ca8537 feat(sandbox): in-RAM blob sandbox for out-of-band page transfer (#243)
Add an ephemeral, process-local blob store so the in-app agent (and the
embedded MCP) can hand a large page document and its images to an external
consumer WITHOUT routing the bytes through the model context or Docmost auth.

- SandboxStore (@Injectable singleton): Map<uuid,{buf,mime,sha256,expiresAt}>
  in RAM only. put() picks a per-blob cap by mime (image vs doc), enforces a
  total-bytes RAM guard with oldest-first eviction, and stamps a TTL; get()
  lazily expires. sha256 computed at put() doubles as the strong ETag. An
  unref'd sweep interval clears expired entries and is cleared on destroy.
- GET /api/sb/:uuid anonymous controller: serves raw bytes with Content-Type,
  Content-Length and ETag=sha256; 404 on missing/expired/non-UUID (anti-
  traversal), 304 on a matching If-None-Match. No tokens, no 401 — the
  capability is the unguessable UUID + short TTL + TLS. Auth-exempt the same
  way as /api/files/public (no JwtAuthGuard) plus an /api/sb entry in main.ts's
  workspace-resolution preHandler so a remote consumer with no workspace host
  is not rejected.
- stash_page tool in both layers (MCP resource_link + in-app {uri,size,sha256,
  images}). client.stashPage serializes the get_page_json shape, mirrors every
  INTERNAL file/image src (type-agnostic, covers drawio/excalidraw/video/file)
  into the sandbox under Docmost auth and rewrites src to the sandbox URL;
  external http(s) srcs are left untouched; dedup by src; a failed image fetch
  is counted, never aborts the doc.
- SANDBOX_PUBLIC_URL / SANDBOX_TTL_MS / SANDBOX_MAX_BYTES /
  SANDBOX_MAX_IMAGE_BYTES / SANDBOX_MAX_TOTAL_BYTES wired through the
  environment service + validation + .env.example.
- SandboxModule (@Global) provides the shared store to the controller,
  McpService and AiChatToolsService (same instance for put and get).

Tests: SandboxStore (round-trip, sha256, TTL lazy + sweep, caps, eviction),
SandboxController (200+ETag+CT+CL, 404 missing/expired/non-UUID, 304), and a
mock-HTTP stashPage test (mirror+rewrite internal, keep external, dedup, failed
image counted, returns only a link). Interoperates with the vvzvlad/habr-mcp
consumer's anonymous-GET + sha256-ETag + resource_link contract.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-28 15:13:11 +03:00
claude code agent 227 fdeede003b feat(share): custom /l/:alias pretty links (share_aliases table) (#205)
Add a retargetable, human-readable vanity link namespace /l/<alias> that
sits alongside the untouched /share/... routes.

- New share_aliases table (workspace-scoped, UNIQUE(workspace_id, alias),
  page_id nullable ON DELETE SET NULL so the address outlives its target).
- ShareAliasRepo + ShareAliasService (create / no-op / 409 reassign guard /
  availability / request-time readable-target resolution through the single
  existing share boundary).
- Public ShareAliasRedirectController (GET /l/:alias) issues a 302 (never 301,
  the target is mutable) to the canonical /share/:key/p/:slug page; unknown /
  dangling / no-longer-readable aliases serve the SPA index with no leak.
  'l/:alias' excluded from the global /api prefix.
- Authenticated ShareAliasController (set/remove/availability/for-page).
- Shared ASCII-only normalize/validate util (server + client copies).
- Client: Custom address block in the share modal (live normalize + debounced
  availability + copy + reassign confirmation dialog).
- Unit tests: util, repo SQL-shape, service semantics, migration/entity sanity
  (server jest) + client alias util (vitest).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 06:28:26 +03:00
claude code agent 227 ec4622a1b8 test(security): export + unit-test resolveTrustProxy (#105)
Relocate resolveTrustProxy from main.ts (untestable — bootstraps on import) to
integrations/environment/trust-proxy.util.ts and import it back. Unit-test every
branch (empty/undefined -> safe loopback/private default; true/false; hop count;
trim; CIDR/negative passthrough) so a regression can't silently re-open the XFF
spoofing hole (#61).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 05:52:15 +03:00
claude code agent 227 5215913533 fix(security): env-configurable trustProxy with a safe default (#61)
trustProxy was unconditionally true, so req.ip came from a client-forgeable
X-Forwarded-For and the per-IP throttles (share-AI, /mcp brute-force) were
spoofable. Make it env-configurable (TRUST_PROXY) with a safe default that
trusts XFF only from loopback/private proxies, documented in .env.example.
NOTE: this changes the default from trust-all; deployments whose proxy is on a
public IP must set TRUST_PROXY (caveat documented).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 03:17:37 +03:00
Philipinho 0d6538ab1a feat: iframe configuration 2026-05-18 22:02:31 +01:00
Philipinho ec51ca7815 fix request ip 2026-05-07 22:09:32 +01:00
Philip Okugbe 641ce142df feat(ee): SCIM (#1347)
* SCIM - init (EE)

* accept db transaction

* sync

* Content parser support for scim+json

* patch scimmy

* sync

* return early if userIds is empty

* sync

* SCIM db table

* fixes

* scim tokens

* backfill

* feat(audit): add scim token events

* rename scim migration

* fix

* fix translation

* cleanup
2026-05-01 14:53:30 +01:00
Philip Okugbe c180d0e487 feat: ratelimits (#2073)
* feat: rate limits

* ip
2026-03-30 15:38:44 +01:00
Philip Okugbe 97c459be67 feat(cloud): add find-workspace and email verification endpoints (#2020)
* feat: add find-workspace and email verification endpoints
* sync
2026-03-14 13:36:30 +00:00
Philipinho ee6b98edaa * enhance ai menu
* remove api prefix from mcp
2026-03-02 03:31:52 +00:00
Philip Okugbe 05b3c65b0f feat: notifications (#1947)
* feat: notifications
* feat: watchers

* improvements

* handle page move for watchers

* make watchers non-blocking

* more
2026-02-14 20:00:38 -08:00
Philip Okugbe 96ed98619f feat: add IPv6 support via configurable HOST binding (#1885) 2026-01-30 00:33:10 +00:00
Philip Okugbe 60501de992 fix: missing logs on OnApplicationBootstrap hook (#1882)
* - fix: set default Nest logger and bufferLogs to false for pino compatibility
- handle redis error event

* fix collab server logging too
2026-01-29 09:25:23 +00:00
Philip Okugbe 918f4508d2 feat: switch to pino for logs (#1855)
- switch to json logs in production
- add option to support http logging
2026-01-21 01:23:50 +00:00
Philipinho b061df7f7d Use new fastify router options 2025-12-13 14:15:06 +00:00
Philip Okugbe 6d024fc3de feat: bulk page imports (#1219)
* refactor imports - WIP

* Add readstream

* WIP

* fix attachmentId render

* fix attachmentId render

* turndown video tag

* feat: add stream upload support and improve file handling

- Add stream upload functionality to storage drivers\n- Improve ZIP file extraction with better encoding handling\n- Fix attachment ID rendering issues\n- Add AWS S3 upload stream support\n- Update dependencies for better compatibility

* WIP

* notion formatter

* move embed parser to editor-ext package

* import embeds

* utility files

* cleanup

* Switch from happy-dom to cheerio
* Refine code

* WIP

* bug fixes and UI

* sync

* WIP

* sync

* keep import modal mounted

* Show modal during upload

* WIP

* WIP
2025-06-09 04:29:27 +01:00
Philip Okugbe e8847bd9cd fix: handle unhandled exceptions (#1116)
* Handle unhandled exceptions
* cleanup
2025-04-29 23:29:00 +01:00
Philip Okugbe 6c422011ac feat: public page sharing (#1012)
* Share - WIP

* - public attachment links
- WIP

* WIP

* WIP

* Share - WIP

* WIP

* WIP

* include userRole in space object

* WIP

* Server render shared page meta tags

* disable user select

* Close Navbar on outside click on mobile

* update shared page spaceId

* WIP

* fix

* close sidebar on click

* close sidebar

* defaults

* update copy

* Store share key in lowercase

* refactor page breadcrumbs

* Change copy

* add link ref

* open link button

* add meta og:title

* add twitter tags

* WIP

* make shares/info endpoint public

* fix

* * add /p/ segment to share urls
* minore fixes

* change mobile breadcrumb icon
2025-04-22 20:37:32 +01:00
Philip Okugbe b81c9ee10c feat: cloud and ee (#805)
* stripe init
git submodules for enterprise modules

* * Cloud billing UI - WIP
* Proxy websockets in dev mode
* Separate workspace login and creation for cloud
* Other fixes

* feat: billing (cloud)

* * add domain service
* prepare links from workspace hostname

* WIP

* Add exchange token generation
* Validate JWT token type during verification

* domain service

* add SkipTransform decorator

* * updates (server)
* add new packages
* new sso migration file

* WIP

* Fix hostname generation

* WIP

* WIP

* Reduce input error font-size
* set max password length

* jwt package

* license page - WIP

* * License management UI
* Move license key store to db

* add reflector

* SSO enforcement

* * Add default plan
* Add usePlan hook

* * Fix auth container margin in mobile
* Redirect login and home to select page in cloud

* update .gitignore

* Default to yearly

* * Trial messaging
* Handle ended trials

* Don't set to readonly on collab disconnect (Cloud)

* Refine trial (UI)
* Fix bug caused by using jotai optics atom in AppHeader component

* configurable database maximum pool

* Close SSO form on save

* wip

* sync

* Only show sign-in in cloud

* exclude base api part from workspaceId check

* close db connection beforeApplicationShutdown

* Add health/live endpoint

* clear cookie on hostname change

* reset currentUser atom

* Change text

* return 401 if workspace does not match

* feat: show user workspace list in cloud login page

* sync

* Add home path

* Prefetch to speed up queries

* * Add robots.txt
* Disallow login and forgot password routes

* wildcard user-agent

* Fix space query cache

* fix

* fix

* use space uuid for recent pages

* prefetch billing plans

* enhance license page

* sync
2025-03-06 13:38:37 +00:00
Philip Okugbe 72f64e7b10 revert sentry (#808)
* revert sentry
* remove sentry env
2025-02-27 15:58:32 +00:00
Philip Okugbe 0065f29634 feat: sentry (#802) 2025-02-26 15:42:19 +00:00
Philipinho 7d034e8a8b enable trustProxy 2025-02-26 13:16:11 +00:00
Philipinho ad1571b902 Log APP_URL on startup 2025-02-26 11:49:58 +00:00
Philip Okugbe 85b3073681 Merge pull request #57 from will2hew/will/url-setup
CORS setup and Vite devServer proxy
2024-07-07 09:42:51 +01:00
Philip Okugbe c4c169b17a Merge pull request #64 from docmost/feat/health
Add health check
2024-07-06 01:06:17 +01:00
Philipinho 35dcd5f254 refactor health module 2024-07-05 18:59:26 +01:00
Will H 5ace7616d0 cors setup and dev server proxy 2024-07-05 16:25:00 +12:00
Philipinho 13f26f9c31 make page attachments private 2024-07-04 16:01:35 +01:00
Philipinho a3b690ac50 update dependencies 2024-06-22 03:47:42 +01:00
Philipinho 2aec5a3fe7 fix maxParamLength 2024-06-20 22:57:39 +01:00
Philipinho d4eefa48a8 restructure directories
* set log level based on env
2024-06-09 15:57:52 +01:00
Philipinho 38ef610e5e fixes
* integrate websocket redis adapter
* use APP_SECRET for jwt signing
* auto migrate database on startup in production
* add updatedAt to update db operations
* create enterprise ee package directory
* fix comment editor focus
* other fixes
2024-06-07 17:29:34 +01:00
Philipinho 9c7c2f1163 updates and fixes
* seo friendly urls
* custom client serve-static module
* database fixes
* fix recent pages
* other fixes
2024-05-18 03:19:42 +01:00
Philipinho 41ecf2d082 pagination - wip 2024-02-28 16:36:41 +00:00
Philipinho 093e634c0b switch to nx monorepo 2024-01-09 18:58:26 +01:00