Commit Graph

37 Commits

Author SHA1 Message Date
claude_code e9e9f74ec6 Merge remote-tracking branch 'gitea/develop' into feat/page-templates
# Conflicts:
#	apps/server/src/integrations/throttle/throttle.module.ts
#	apps/server/src/integrations/throttle/throttler-names.ts
2026-06-20 20:18:42 +03:00
claude_code 46688074d8 Merge pull request 'feat(tree): server-authoritative realtime tree updates' (#15) from feat/realtime-tree-server into develop 2026-06-20 19:48:36 +03:00
claude code agent 227 39ae89264d feat(editor): page templates - live whole-page embed (MVP)
Embed another page's LIVE content into a host page (it updates when the source
changes, not a static copy). A page can be flagged a template for discovery in
the picker; any accessible page can be embedded.

Server:
- migrations: pages.is_template (+ partial index) and page_template_references
  (whole-page back-refs); db.d.ts/entity types hand-merged (db.d.ts is curated).
- POST /pages/toggle-template (CASL Edit) flips is_template; is_template is
  returned by findById + the sidebar tree select so the tree menu label
  reflects state. Search suggestions gain an onlyTemplates filter for the picker.
- POST /pages/template/lookup ({sourcePageIds[]}, <=50): returns each accessible
  source's {title, icon, slugId, content, sourceUpdatedAt} with comment marks
  stripped (same access path as transclusion: filterViewerAccessiblePageIds;
  inaccessible -> no_access, missing -> not_found; error path -> not_found, never
  raw content).
- reference sync (collectPageEmbedsFromPmJson + syncPageTemplateReferences) on
  the Yjs save hook; duplicatePage remaps pageEmbed.sourcePageId + inserts refs.
  Known MVP gap: REST content updates don't resync refs (lookup uses in-doc ids).

Client:
- pageEmbed node (editor-ext, registered in BOTH client + server schemas);
  read-only NodeView with a batching lookup; '/Embed page' slash + template
  picker (self-embed prevented); 'Make/Unset template' in the tree node menu.
- Cycle guard: an ancestry-chain context + depth cap (5) render a 'circular
  embed' placeholder instead of recursing.
- Public shares show a placeholder (no public lookup in MVP).

MVP excludes (follow-ups): public-share lookup, unsync->static copy, server-side
expansion for export/RAG, MCP schema mirror, point-in-time snapshots.

Implements docs/page-templates-plan.md (MVP, variant A).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 10:05:00 +03:00
claude code agent 227 046132afc7 feat(tree): server-authoritative realtime tree updates
The sidebar page tree only updated on other clients when a change was made
via the UI tree, in an open tab, within a ~50ms client relay window — API/MCP/
AI/import changes never propagated. Move the source of truth to the server.

Server:
- Enrich PageEvent with thin TreeNodeSnapshot(s) so the WS listener never reads
  the DB (avoids the in-transaction visibility race). insertPage fills the
  create snapshot from its returning() row; removePage ships only the deleted
  subtree ROOT (client treeModel.remove drops descendants); restorePage carries
  spaceId.
- New PAGE_MOVED event from movePage with old/new parent + position + snapshot
  (generic PAGE_UPDATED stays for content/rename).
- WsService.emitTreeEvent mirrors emitCommentEvent (per-space restriction gate:
  spaceHasRestrictions -> hasRestrictedAncestor -> broadcastToAuthorizedUsers);
  author NOT excluded so non-UI creators see their own page (receiver is
  idempotent).
- WsTreeService.broadcastPageCreated/Deleted/Moved + broadcastRefetchRoot;
  new PageWsListener (create/delete/move/restore) registered in WsModule.

Client:
- Remove the client relay (emit + setTimeout(50)) from create/move/delete;
  keep optimistic local updates. Make the optimistic create insert id-idempotent
  (find-then-skip) so the now-fast server addTreeNode broadcast can't race it
  into a duplicate row. addTreeNode inserts by fractional position among loaded
  siblings (consistent order across clients).

Restore uses refetchRootTreeNodeEvent (robust for subtree re-attach). Rename/icon
updateOne and cross-space move realtime are deferred (commented as follow-ups).

Implements docs/backlog/realtime-tree-server-authoritative.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 08:27:56 +03:00
claude code agent 227 b81819ef63 feat(tree): Expand all / Collapse all for the space page tree
Adds a server-authoritative whole-tree endpoint and sidebar menu commands
so a deep space tree can be expanded in one request instead of a per-level
BFS storm.

Server:
- POST /pages/tree (SidebarPageTreeDto: spaceId | pageId), same CASL space
  scoping as /sidebar-pages. Returns the whole space tree / subtree as a flat
  list in the sidebar item shape (id, slugId, title, icon, position,
  parentPageId, spaceId, hasChildren, canEdit), ordered by position
  (collate C byte order), content never fetched.
- page.service.getSidebarPagesTree reproduces getSidebarPages' two-branch
  permission model: open space -> spaceCanEdit; restricted space -> seed the
  full descendant set then prune via filterAccessibleTreePages +
  filterAccessiblePageIdsWithPermissions (keeps restricted-but-granted pages,
  prunes inaccessible subtrees). hasChildren is derived from the final
  filtered set so it can never reveal inaccessible children.
- page.repo.getSpaceDescendants: recursive CTE seeded by space roots.

Client:
- SpaceTree is forwardRef exposing expandAll/collapseAll/isExpanding;
  expandAll fetches the whole tree once, replaces current-space nodes, opens
  every branch (current space only), aborts on space switch, surfaces real
  errors; collapseAll collapses only current-space ids (shared open-map).
- SpaceMenu gains Expand all / Collapse all items (no admin gate).

Implements docs/backlog/tree-expand-collapse-all.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-20 05:31:34 +03:00
vvzvlad 91a63f0b2c fix(ai): stop RAG coverage bar sticking below 100% on empty pages
"Indexed N of M pages" stayed at e.g. "27 of 34" forever even after a
successful full reindex. The numerator counted pages that have embeddings
while the denominator counted ALL non-deleted pages, so empty / text-less
pages (which legitimately store zero embeddings) could never be reached.

Add PageRepo.countEmbeddablePages: counts non-deleted pages that have
non-empty textContent OR already have a stored embedding row, and use it as
the totalPages denominator in AiSettingsService.getMasked. The "has
embeddings" clause covers pages indexed from the content JSON (null
textContent) and guarantees indexedPages <= totalPages. No DB migration.
2026-06-18 03:33:38 +03:00
vvzvlad 52e19fe678 feat(ai): wire up workspace RAG bulk reindex + manual "Reindex now"
The WORKSPACE_CREATE_EMBEDDINGS / WORKSPACE_DELETE_EMBEDDINGS jobs were
enqueued (on AI Search enable/disable) but had no AI_QUEUE handler, so
existing pages were never indexed ("Indexed 0 of N pages") and disabling
never purged embeddings.

- EmbeddingProcessor: handle WORKSPACE_CREATE_EMBEDDINGS (bulk reindex all
  live pages) and WORKSPACE_DELETE_EMBEDDINGS (purge workspace embeddings)
- EmbeddingIndexerService: add reindexWorkspace() (skips when embeddings
  unconfigured; per-page error isolation) and removeWorkspace()
- PageRepo.getIdsByWorkspace(), PageEmbeddingRepo.deleteByWorkspace()
- AiSettingsService.reindex() + admin-only POST /workspace/ai-settings/reindex
- Frontend: "Reindex now" button, service call and mutation
- Stable per-workspace jobId with remove-before-add so a stale job can't
  block future reindexes; cancel the delayed purge on enable/reindex so it
  can't wipe freshly-built embeddings
2026-06-18 02:15:18 +03:00
vvzvlad 1f2d20244e feat(ai-chat): show RAG indexing coverage in AI settings
Display "Indexed N of M pages" on the AI provider settings page so admins
can see how much of the wiki is covered by vector-RAG semantic search.

- page-embedding.repo: add countIndexedPages() — distinct non-deleted pages
  that have stored embeddings in the workspace
- page.repo: add countByWorkspace() — total non-deleted pages
- ai-settings.service: compute both counts in getMasked() (Promise.all) and
  return them with the masked settings; inject PageEmbeddingRepo + PageRepo
- MaskedAiSettings / IAiSettings: add indexedPages + totalPages
- ai-provider-settings: render a dimmed coverage line under "Embedding model"
- i18n: add the "Indexed {{indexed}} of {{total}} pages" key (en-US, ru-RU)
2026-06-17 23:18:51 +03:00
vvzvlad 0a9788e89a feat(collab): separate agent edits from human edits in page history
Page-history snapshots are debounced/coalesced (one per 1–5 min window,
jobId=page.id). A human edit followed by an agent edit in the same window
collapsed into a single snapshot, losing both the pre-agent human state and
a deterministic record of the agent's result.

Two provenance-aware boundaries now bracket an agent intervention:
- Before: on a user->agent transition, onStoreDocument synchronously pins the
  current (pre-agent) human content as its own history version tagged 'user',
  inside the page-write transaction, before the agent overwrites it.
- After: agent stores enqueue an immediate (delay 0), source-keyed history job
  (jobId=`${pageId}:agent`) so the agent's result snapshots deterministically
  as 'agent' and a later human edit (jobId=page.id) cannot coalesce/retag it.

Also add an `id desc` tie-break to findPageLastHistory so "last history" stays
deterministic when two snapshots share a created_at, consistent with
findPageHistoryByPageId.

Known trade-offs (Variant 1): the delay-0 worker re-reads the row, leaving a
millisecond mis-tag window; multiple agent edits in one turn may yield multiple
versions. The reverse agent->human boundary is intentionally out of scope.
2026-06-17 06:40:28 +03:00
vvzvlad 683da7a4c5 feat(ai-chat): per-user AI agent backend — LLM config, read-only agent, provenance schema
WIP checkpoint of the gitmost AI-chat backend (plan stages A + B1 + B3a).
The agent acts under the requesting user's JWT (Docmost CASL enforces page
access); the external service-account /mcp endpoint is untouched.

LLM provider config (A2-A4):
- integrations/crypto: AES-256-GCM SecretBoxService (key derived from APP_SECRET,
  per-record salt/iv; clear error on rotation instead of crashing).
- ai_provider_credentials table/repo/types: encrypted API key stored outside
  workspace settings/baseFields, write-only (never returned by any endpoint).
- integrations/ai: per-workspace AI SDK v6 provider driver (openai/gemini/ollama),
  admin-gated GET(masked)/PATCH(write-only key)/Test endpoints; settings.ai.provider
  holds non-secret config incl. systemPrompt. Removed unused AI_* env getters (DB is
  the single source of truth).

Chat module (A1, A5-A8):
- ai_chats/ai_chat_messages repos (workspace-scoped, soft-delete, tsv never selected).
- core/ai-chat: CRUD + POST /ai-chat/stream (Fastify hijack + AI SDK v6
  pipeUIMessageStreamToResponse, abort on disconnect, persist user/assistant msgs).
- Agent loop: streamText + stepCountIs(8); read tools searchPages/getPage via a
  per-request DocmostClient over loopback REST under the user's minted access token.
- Gate settings.ai.chat (+ 503 when provider unconfigured); buildSystemPrompt with a
  non-removable safety/anti-prompt-injection framework. Per-user rate limit.

Per-user auth (B1):
- @docmost/mcp DocmostClient gains an additive getToken variant (carry a user JWT,
  re-fetch on 401) and exports DocmostClient; the email/password service-account path
  (external /mcp, stdio) is unchanged.

Agent-edit provenance backbone (B3a):
- Migration: pages/page_history (last_updated_source, last_updated_ai_chat_id) and
  comments (created_source, ai_chat_id, resolved_source).
- Signed actor/aiChatId claim in the collab token; onAuthenticate propagates it,
  onStoreDocument writes it with a sticky agent marker, saveHistory copies it.

Migrations auto-run on boot (additive). Write tools, frontend, RAG and external MCP
servers are not in this checkpoint.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-17 01:36:41 +03:00
Peter Tripp 932c1ad5b7 Better trash (#2190)
* Better trash

I recently lost a bunch of time editing and searching for pages that were actually in the Trash. Docmost intentionally tries to not link to Trashed pages, but the url of that Trashed page and any inbound links still work.  This makes it clearer when a page you are interacting with is in the Trash.

- /trash
  - Refactored banner into `trash-banner.tsx`
  - Refactored "Restore" modal into `use-restore-page-modal.tsx`
- Page (when isDeleted)
  - Add: `trash-banner.tsx`
  - Add breadcrumbs: `Parent / Child / Page (Deleted)`
  - Change: Deleted Pages are read-only
  - Replace "Move to Trash" with "Restore" in page menu (invokes `use-restore-page-modal`)

I tried very hard to keep this simple and re-use existing translation strings wherever possible.

* cleanup

---------

Co-authored-by: Philipinho <16838612+Philipinho@users.noreply.github.com>
2026-05-14 14:41:10 +01:00
Philip Okugbe f758091b2a perf(permissions): cache space role and page edit lookups (#2208) 2026-05-14 13:11:28 +01:00
Philip Okugbe de60aa7e61 feat: synced blocks (transclusion) (#2163)
* feat: synced blocks (transclusion)

* fix:remove name

* make placeholders smaller

* feat: enforce strict transclusion schema

* fix: scope synced blocks to workspace, gate unsync on edit permission

* fix collab module error
2026-05-08 13:23:16 +01:00
Philip Okugbe d42091ccb1 feat: favorites (#2103)
* feat: favorites and templates(ee)

* rename migrations

* fix sidebar

* cleanup tabs

* fix

* turn off templates

* cleanup

* uuid validation
2026-04-12 22:06:25 +01:00
Philip Okugbe 59e945562d feat(ee): page-level access/permissions (#1971)
* Add page_hierarchy table

* feat(ee): page-level permissions

* pagination

* rename migration
fixes

* fix

* tabs

* fix theme

* cleanup

* sync

* page permissions notification
* other fixes

* sharing disbled

* fix column nodes

* toggle error handling
2026-02-26 19:49:10 +00:00
Philip Okugbe 0f02261ee6 feat: page version history improvements (#1925)
* Refactor: use queue for page history

* feat: save multiple version contributors

* display contributor avatars in history list

* fix interval
2026-02-09 18:25:35 -08:00
Philip Okugbe 5506eb194b feat: page history diff (#1891)
* Show actual history changes
* V2 - WIP
* feat: page history diff
* fix: exclude content from history listing

---------

Co-authored-by: Jason Norwood-Young <jason@10layer.com>
2026-02-03 11:55:20 -08:00
Philip Okugbe 78b1c1a453 feat: switch to cursor pagination (#1884)
* add cursor pagination function

* support custom order modifier
* refactor returned object

* feat(db): migrate paginated endpoints to cursor-based pagination

* sync

* support hasPrevPage boolean

* feat(client): migrate pagination from offset to cursor-based

* support beforeCursor/prevCursor

* wrap search results in items array for API consistency
2026-01-30 19:28:54 +00:00
Philipinho 3523600f40 add timestamps 2026-01-27 16:49:22 +00:00
Philip Okugbe 47097969a0 fix: use subquery (#1833)
- enhance file tasks list endpoint
2026-01-13 15:58:26 +00:00
Philip Okugbe 9fb16bc842 feat(EE): AI vector search (#1691)
* WIP

* AI module - init

* WIP

* sync

* WIP

* refactor naming

* new columns

* sync

* sync

* fix search bug

* stream response

* WIP

* feat embeddings sync

* refine

* Add workspaceId to page events

* refine

* WIP

* add translation string

* sync

* reset ai answer on query change

* hide AI search in cloud

* capture streaming error

* sync
2025-12-01 11:50:25 +00:00
Philip Okugbe bf8cf6254f feat: Typesense search driver (EE) (#1664)
* feat: typesense driver (EE) - WIP

* feat: typesense driver (EE) - WIP

* feat: typesense

* sync

* fix
2025-10-07 17:34:32 +01:00
Philip Okugbe aa58e272d6 fix: exclude deleted pages (#1494) 2025-08-31 09:11:33 +01:00
Philip Okugbe 1cb2535de3 fix trash in search (#1439)
- delete share if page is trashed
2025-08-02 00:14:00 +01:00
Eddy Oyieko ec12e80423 feat: trash for deleted pages in space (#325)
* initial commit

* added recycle bin modal, updated api routes

* updated page service & controller, recycle bin modal

* updated page-query.ts, use-tree-mutation.ts, recycled-pages.ts

* removed quotes from openRestorePageModal prompt

* Updated page.repo.ts

* move button to space menu

* fix react issues

* opted to reload to enact changes in the client

* lint

* hide deleted pages in recents, handle restore child page

* fix null check

* WIP

* WIP

* feat: implement dedicated trash page
- Replace modal-based trash view with dedicated route `/s/:spaceSlug/trash`
- Add pagination support for deleted pages
- Other improvements

* fix translation

* trash cleanup cron

* cleanup

---------

Co-authored-by: Philipinho <16838612+Philipinho@users.noreply.github.com>
2025-07-29 21:20:49 +01:00
Philip Okugbe 7993532111 fix page export (#1081) 2025-04-24 23:18:54 +01:00
Philip Okugbe 6c422011ac feat: public page sharing (#1012)
* Share - WIP

* - public attachment links
- WIP

* WIP

* WIP

* Share - WIP

* WIP

* WIP

* include userRole in space object

* WIP

* Server render shared page meta tags

* disable user select

* Close Navbar on outside click on mobile

* update shared page spaceId

* WIP

* fix

* close sidebar on click

* close sidebar

* defaults

* update copy

* Store share key in lowercase

* refactor page breadcrumbs

* Change copy

* add link ref

* open link button

* add meta og:title

* add twitter tags

* WIP

* make shares/info endpoint public

* fix

* * add /p/ segment to share urls
* minore fixes

* change mobile breadcrumb icon
2025-04-22 20:37:32 +01:00
Philipinho 3559358d14 fix pagination issue where user is not part of any space 2025-04-07 19:09:02 +01:00
Philip Okugbe 17ce3bab8a feat: move page between spaces (#988)
* feat: Move the page to another space

- The ability to move a page to another space has been added

* feat: Move the page to another space
* feat: Move the page to another space

- Correction of the visibility attribute of elements that extend beyond the boundaries of the space selection modal window

* feat: Move the page to another space

- Added removal of query keys when moving pages

* feat: Move the page to another space

- Fix locales

* feat: Move the page to another space
* feat: Move the page to another space

- Fix docker compose

* feat: Move the page to another space

* feat: Move the page to another space

- Some refactor

* feat: Move the page to another space

- Attachments update

* feat: Move the page to another space

- The function of searching for attachments by page ID and updating attachments has been combined

* feat: Move the page to another space

- Fix variable name

* feat: Move the page to another space

- Move current space to parameter of component SpaceSelectionModal

* refactor ui

---------

Co-authored-by: plekhanov <astecom@mail.ru>
2025-04-04 23:44:18 +01:00
Philip Okugbe 64f0531093 feat: keep track of page contributors (#959)
* WIP

* feat: store and retrieve page contributors
2025-04-04 13:03:57 +01:00
Philip Okugbe f45d9dc5a0 feat: add page stats to page menu (#876) 2025-03-13 14:54:18 +00:00
Philip Okugbe e209aaa272 feat: internal page links and mentions (#604)
* Work on mentions

* fix: properly parse page slug

* fix editor suggestion bugs

* mentions must start with whitespace

* add icon to page mention render

* feat: backlinks - WIP

* UI - WIP

* permissions check
* use FTS for page suggestion

* cleanup

* WIP

* page title fallback

* feat: handle internal link paste

* link styling

* WIP

* Switch back to LIKE operator for search suggestion

* WIP
* scope to workspaceId
* still create link for pages not found

* select necessary columns

* cleanups
2025-02-14 15:36:44 +00:00
Philipinho a25cf84671 fix: add spaceId 2024-11-30 20:29:13 +00:00
Philip Okugbe fe83557767 feat: space export (#506)
* wip

* Space export
* option to export pages with children
* include attachments in exports
* unified export UI

* cleanup

* fix: change export icon

* add export button to space settings

* cleanups

* export name
2024-11-30 19:47:22 +00:00
Philipinho 38ef610e5e fixes
* integrate websocket redis adapter
* use APP_SECRET for jwt signing
* auto migrate database on startup in production
* add updatedAt to update db operations
* create enterprise ee package directory
* fix comment editor focus
* other fixes
2024-06-07 17:29:34 +01:00
Philipinho 06d854a7d2 refactor layout
* ui polishing
* frontend and backend fixes
2024-05-31 21:51:44 +01:00
Philipinho 9c7c2f1163 updates and fixes
* seo friendly urls
* custom client serve-static module
* database fixes
* fix recent pages
* other fixes
2024-05-18 03:19:42 +01:00