Commit Graph

23 Commits

Author SHA1 Message Date
claude code agent 227
032df2cf31 feat(git-sync): remove the per-cycle delete cap; deletes apply + are logged every cycle
The delete cap (GIT_SYNC_MAX_DELETES_PER_CYCLE, default 5) was a defense-in-depth
guard that SUPPRESSED a cycle's deletions when the planned count exceeded the
limit. In practice it was a crutch over engine correctness that also blocked
legitimate deletes: deleting a folder with many child pages is a normal action,
and git-sync deletes are SOFT (Trash, reversible), so a blocking limit has little
upside and real downside. There is also no user-facing surface to "confirm" a
large delete from a background sync — the only channel is the operator log.

So: drop the cap entirely. Deletes apply unconditionally; every cycle already
logs its full push plan, per-action `delete: <pageId>` lines, and completion
counts through the engine `log`, so what was deleted (and what was skipped) is
always recorded. Engine correctness (the reconcile/layout/round-trip tests) is
what prevents phantom deletions — not a blocking cap.

Removed: orchestrator `resolveApplyClient` cap hook + `maxDeletes`,
`getGitSyncMaxDeletesPerCycle`, the `GIT_SYNC_MAX_DELETES_PER_CYCLE` env/validation/.env.example,
and the cap tests. (The engine's generic optional `resolveApplyClient` hook is
left as an unused extension point.)

server tsc clean, git-sync + environment jest 174.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 03:59:12 +03:00
claude code agent 227
b751c4bdc5 docs(git-sync): document GIT_SYNC_BACKEND_TIMEOUT_MS, drop dead consts, fix dangling plan refs
Address the non-red-team documentation/cleanup items from review #1679:
- Document the GIT_SYNC_BACKEND_TIMEOUT_MS watchdog (git http-backend) in
  .env.example and add it to the environment validation schema — it was used
  (getGitSyncBackendTimeoutMs, default 120000) but undocumented/unvalidated.
- Remove the dead GIT_SYNC_DEBOUNCE_MS_DEFAULT / GIT_SYNC_POLL_INTERVAL_MS_DEFAULT
  exports (never imported; environment.service is the single source of defaults).
- Redirect the dangling `plan §X.Y` comment references to issue #194 (the
  git-sync spec moved there when docs/git-sync-plan.md was deleted by this PR).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 01:34:14 +03:00
claude_code
66bd039f8f feat(git-sync): serve spaces over smart-HTTP (gitmost as a two-way git host)
Expose each git-sync-enabled space as a clonable/pushable git repo over HTTP,
so `git clone https://<user>:<pass>@<host>/git/<spaceId>.git` works and external
pushes flow back into Docmost pages — gitmost itself acts as the git host (no
external GitHub/Gitea, no SSH).

Transport: shell out to `git http-backend` (CGI; git is already in the runtime
image) which implements the full smart-HTTP protocol (info/refs, upload-pack,
receive-pack, protocol v2). A raw Fastify route `/git/*` (mounted at the root,
outside the `/api` prefix) bridges the request/response to the CGI; passthrough
content-type parsers for the git media types stream the raw body to stdin.

Reuse the existing engine: clients push the vault's `main` branch, whose commits
beyond `refs/docmost/last-pushed` the engine already reconciles into Docmost.

- http/git-http.service.ts — auth (HTTP Basic -> AuthService.verifyUserCredentials),
  self-resolved workspace (DomainMiddleware does not run for this raw route),
  per-space gating (global + per-space gitSync flags, 404 hides existence),
  CASL authz (Read=fetch, Manage=push), dispatch.
- http/git-http-backend.service.ts — spawn `git http-backend`, binary-safe CGI
  response parsing (Status/headers/body), stream to the socket.
- http/git-http.helpers.ts — pure path parse, service->kind mapping, gate decision
  (unit-tested); rejects literal and percent-encoded path traversal.
- orchestrator: extract reusable withSpaceLock (CAS-guarded lock heartbeat so a
  long push cannot let the lock expire mid-cycle) and add ingestExternalPush
  (receive-pack + Docmost cycle under one lock; 503 on contention).
- vault-registry: ensureServable() — ensureRepo + idempotent receive.denyCurrentBranch
  =updateInstead / denyNonFastForwards / http.receivepack / http.uploadpack.
- env: GIT_SYNC_HTTP_ENABLED (defaults to GIT_SYNC_ENABLED) + validation.
- main.ts: register the /git/* route and the git content-type parsers.

Tests: pure helpers, CGI parsing, and the GitHttpService handler (auth/gate/authz
+ workspace resolution). Server tsc + git-sync/env suites green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 00:17:24 +03:00
claude code agent 227
8201e76c66 fix(git-sync): branch choreography + strict scoping + delete cap (Phase B hardening)
Fixes found by the live pull/push e2e:
- CRITICAL: driveCycle never checked out the 'docmost' branch before
  applyPullActions, so Docmost content was written straight onto 'main',
  clobbering local file edits before push could diff them. Now checkout
  'docmost' before pull (applyPullActions commits there then checks out main +
  merges) — mirrors the engine's pull main(). Round-trip now works both ways.
- add an unresolved-merge guard (SPEC §9): skip the cycle if the vault is
  mid-merge instead of failing on checkout.
- SAFETY: enabledSpaces() is now STRICT opt-in — only spaces with
  settings.gitSync.enabled===true; removed the all-spaces fallback that synced
  every space (incl. a 92-page one) the moment GIT_SYNC_ENABLED flipped.
- SAFETY: per-cycle delete cap (GIT_SYNC_MAX_DELETES_PER_CYCLE, default 5):
  dry-run the push, and if planned deletes exceed the cap, run the apply with
  deletePage neutralized — phantom absence-deletions from a non-convergent vault
  can't soft-delete real pages. Fails safe if the dry-run throws.
- fix manual trigger: TriggerGitSyncDto.spaceId needs @IsUUID or the global
  whitelist ValidationPipe strips it (arrived undefined -> vault 'undefined').

Live-verified on an isolated flagged space: push (vault file edit -> Docmost
content, stamped lastUpdatedSource='git-sync') and pull (Docmost rename -> vault
file + meta) both work; an unrelated 92-page space stayed untouched throughout.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 00:17:24 +03:00
claude code agent 227
901147a224 feat(git-sync): GitSyncModule orchestrator + config + listener (Phase A.4b/B)
Control plane wiring (plan §5-§11):
- PageService create/update/movePage now honor provenance actor 'git-sync'
  (stamp lastUpdatedSource='git-sync'), closing the A.4a gap.
- EnvironmentService: GIT_SYNC_ENABLED / DATA_DIR / REMOTE_TEMPLATE /
  POLL_INTERVAL_MS / DEBOUNCE_MS / SERVICE_USER_ID (required-if-enabled) /
  SSH_KEY_PATH + validation.
- VaultRegistryService: per-space vault path + cached VaultGit.
- GitSyncOrchestrator: per-space Redis leader-lock (SET NX PX + CAS-Lua release,
  randomUUID instanceId) + in-process mutex; runOnce drives the vendored engine
  PULL (readExisting->computePullActions->applyPullActions) then PUSH (runPush)
  with the bound native GitSyncClient + VaultGit; @Interval poll-safety gated on
  GIT_SYNC_ENABLED; imports plain ScheduleModule (TelemetryModule owns forRoot).
- PageChangeListener: @OnEvent PAGE_* -> per-space debounce -> runOnce, with a
  best-effort lastUpdatedSource==='git-sync' loop-guard.
- GitSyncController: admin POST /api/git-sync/trigger + GET /status (ops/e2e).
- GitSyncModule registered in app.module. Enabled-space enumeration uses
  settings.gitSync.enabled, falling back to all live spaces until Phase C writes
  the flag (master gate = GIT_SYNC_ENABLED).

tsc clean; 713 tests/71 suites pass; dev server hot-reloaded the module (route
live, DI graph boots). Live pull/push round-trip verified next.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-26 00:17:24 +03:00
Philip Okugbe
4295ea09f6 feat(storage): add Azure Blob Storage driver (#2222) 2026-05-21 12:18:58 +01:00
Philip Okugbe
9aaa6c731c feat: add AI_EMBEDDING_SUPPORTS_MRL env var to decouple pgvector dimensions from model API (#2079)
Some embedding models don't accept a `dimensions` parameter. This adds
an optional env var that controls whether the dimension is sent to the
model API, while always using it for pgvector indexing. Preset models
have this handled automatically; the env var allows explicit override
for custom models.
2026-03-31 19:39:49 +01:00
Philip Okugbe
97c459be67 feat(cloud): add find-workspace and email verification endpoints (#2020)
* feat: add find-workspace and email verification endpoints
* sync
2026-03-14 13:36:30 +00:00
Philip Okugbe
69d7532c6c feat(ee): audit logs (#1977)
feat: clickhouse driver
* sync
* updates
2026-03-01 01:29:03 +00:00
Philipinho
0ce74d34de env validation 2026-02-16 04:11:19 +00:00
Philipinho
bf89eff5e7 sync 2026-01-16 13:20:31 +00:00
Philipinho
d59539f197 fix ai streaming 2025-12-13 14:15:41 +00:00
Philip Okugbe
9fb16bc842 feat(EE): AI vector search (#1691)
* WIP

* AI module - init

* WIP

* sync

* WIP

* refactor naming

* new columns

* sync

* sync

* fix search bug

* stream response

* WIP

* feat embeddings sync

* refine

* Add workspaceId to page events

* refine

* WIP

* add translation string

* sync

* reset ai answer on query change

* hide AI search in cloud

* capture streaming error

* sync
2025-12-01 11:50:25 +00:00
Philip Okugbe
bf8cf6254f feat: Typesense search driver (EE) (#1664)
* feat: typesense driver (EE) - WIP

* feat: typesense driver (EE) - WIP

* feat: typesense

* sync

* fix
2025-10-07 17:34:32 +01:00
Philip Okugbe
b81c9ee10c feat: cloud and ee (#805)
* stripe init
git submodules for enterprise modules

* * Cloud billing UI - WIP
* Proxy websockets in dev mode
* Separate workspace login and creation for cloud
* Other fixes

* feat: billing (cloud)

* * add domain service
* prepare links from workspace hostname

* WIP

* Add exchange token generation
* Validate JWT token type during verification

* domain service

* add SkipTransform decorator

* * updates (server)
* add new packages
* new sso migration file

* WIP

* Fix hostname generation

* WIP

* WIP

* Reduce input error font-size
* set max password length

* jwt package

* license page - WIP

* * License management UI
* Move license key store to db

* add reflector

* SSO enforcement

* * Add default plan
* Add usePlan hook

* * Fix auth container margin in mobile
* Redirect login and home to select page in cloud

* update .gitignore

* Default to yearly

* * Trial messaging
* Handle ended trials

* Don't set to readonly on collab disconnect (Cloud)

* Refine trial (UI)
* Fix bug caused by using jotai optics atom in AppHeader component

* configurable database maximum pool

* Close SSO form on save

* wip

* sync

* Only show sign-in in cloud

* exclude base api part from workspaceId check

* close db connection beforeApplicationShutdown

* Add health/live endpoint

* clear cookie on hostname change

* reset currentUser atom

* Change text

* return 401 if workspace does not match

* feat: show user workspace list in cloud login page

* sync

* Add home path

* Prefetch to speed up queries

* * Add robots.txt
* Disallow login and forgot password routes

* wildcard user-agent

* Fix space query cache

* fix

* fix

* use space uuid for recent pages

* prefetch billing plans

* enhance license page

* sync
2025-03-06 13:38:37 +00:00
Philip Okugbe
4b9ab4f63c feat: standalone collab server (#767)
* feat: standalone collab server

* * custom collab server port env
* fix collab start script command

* * API prefix
* Log startup PORT

* Tweak collab debounce
2025-02-25 13:15:51 +00:00
Philip Okugbe
040d6625df fix: enforce 32-character minimum length for APP_SECRET (#702)
* Enforce 32 characters minimum APP_SECRET length

* update APP_SECRET comment
2025-02-06 17:46:32 +00:00
Philip Okugbe
4a2a5a7a4d fix: postgres and redis url validation (#548) 2024-12-09 14:56:15 +00:00
Philipinho
bc7cd033f2 more env validations 2024-06-27 22:47:59 +01:00
Philipinho
a582d4786d make env validation errors clear
* modify mail smtp variable names
2024-06-27 17:55:17 +01:00
Philipinho
38ef610e5e fixes
* integrate websocket redis adapter
* use APP_SECRET for jwt signing
* auto migrate database on startup in production
* add updatedAt to update db operations
* create enterprise ee package directory
* fix comment editor focus
* other fixes
2024-06-07 17:29:34 +01:00
Philipinho
eefe63d1cd implement new invitation system
* fix comments on the frontend
* move jwt token service to its own module
* other fixes and updates
2024-05-14 22:55:11 +01:00
Philipinho
186c4f5f5c move environment module to integrations 2024-03-21 20:09:25 +00:00