Compare commits

...

5 Commits

Author SHA1 Message Date
vvzvlad 443cc0e88b Merge pull request 'feat(mcp): getPageContext — «где я / что вокруг» по pageId (#443, часть 3/3, closes #443)' (#470) from feat/443-get-page-context into feat/443-get-tree
Reviewed-on: #470
2026-07-10 23:45:38 +03:00
agent_coder 5dc7a2703f feat(mcp): getPageContext — «где я / что вокруг» по pageId одним вызовом (#443, часть 3/3)
Финальная из трёх частей #443. Есть pageId (из поиска, из ссылки) — нужно понять
местоположение и окружение. Раньше это get_page + полное дерево. Только чтение,
только метаданные (без контента). closes #443.

- client.ts getPageContext(pageId): resolvePageId (slugId→uuid, для uuid без
  round-trip) → POST /pages/breadcrumbs + listSidebarPages(spaceId, uuid). Два
  запроса для uuid-входа (spaceId берётся из ответа breadcrumbs), +1 для slugId.
  Выход {page:{pageId,title,spaceId}, breadcrumbs:[{pageId,title}] (root→parent;
  [] для корня), children:[{pageId,title,hasChildren}]}.
- Порядок breadcrumbs выверен по серверному источнику (getPageBreadCrumbs: CTE
  сидится на childPageId, идёт вверх, .reverse() → root→page; страница —
  ПОСЛЕДНИЙ элемент). page = chain[last], breadcrumbs = chain.slice(0,-1); корень
  → []. Проекция явная — наружу только pageId (UUID), slugId/icon/position не
  утекают. Пустой chain / 404 / 403 → явная ошибка, не пустой объект.
- children — прямые дети через listSidebarPages (cursor-пагинация #442/#451,
  страница с >20 детьми отдаёт всех без дублей, порядок по position, hasChildren).
- Общий реестр: getPageContext на ОБОИХ хостах через цикл спеков (mcpName===
  inAppKey==="getPageContext", camelCase); DocmostClientLike/DocmostClientMethod
  += it; compile-time contract; TOOL_FAMILY READ + routing-проза.

Тесты: get-page-context.test.mjs +6 (3-й уровень split, no-slugId-leak + 2
запроса, корень→[], slugId-resolve, >20 детей без cap/дублей, плохой id→ошибка,
пустой chain→throw), tool-specs +1. mcp node --test 725/725, tsc чисто; server
jest (contract + ai-chat) 265/265.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 22:19:25 +03:00
agent_coder bfb4c8d8d0 feat(mcp): getTree — иерархия пространства/поддерева одним вызовом (#443, часть 2/3)
Раньше единственный способ получить дерево — listPages tree:true: BFS по
sidebar-эндпоинту, десятки-сотни HTTP-вызовов и молчаливая потеря страниц
(#442). Бэкенд форка уже отдаёт всё одним POST /pages/tree (getSidebarPagesTree,
merged #442/#451), так что это MCP-сторона + общий реестр. Только чтение.

- lib/tree.ts: buildPageTree(nodes) аддитивно расширен до buildPageTree(nodes,
  options?) с {shape?: "lean"|"getTree"; maxDepth?}. Дефолт/{} — байт-идентичный
  lean {id,slugId,title,children?} (существующие вызыватели listPages tree:true
  и subtree-BFS не тронуты, есть регрессионный тест на точный набор полей).
  shape:"getTree" проецирует {pageId, title, children?, hasChildren?} (id→pageId;
  slugId/icon/position/parentPageId не утекают ни на одной глубине).
- client.ts: getTree(spaceId, rootPageId?, maxDepth?) — тот же код-путь, что
  listPages tree:true: один enumerateSpacePages(spaceId, rootPageId) (единичный
  /pages/tree + cursor-BFS фолбэк для stock upstream) → buildPageTree(pages,
  {shape:"getTree", maxDepth}). listPages tree:true помечен DEPRECATED в JSDoc
  (BFS-фолбэк на месте, поведение не тронуто).
- maxDepth/hasChildren: полное дерево строится, потом обрезается. Корни = глубина
  1; maxDepth:1 → только корни; hasChildren:true ТОЛЬКО на срезанном узле с
  серверным hasChildren (source of truth), опущен у листьев и раскрытых узлов.
  Схема тула min(1) отбраковывает 0/отрицательные.
- tool-specs.ts: shared-спек getTree (оба хоста через цикл реестра),
  DocmostClientLike Pick += getTree, listPages desc/catalogLine — про депрекацию.
  server-instructions.ts: getTree:"READ" + routing-проза. Loader
  DocmostClientMethod += getTree, compile-time client-call contract += getTree.

Циклы/self-ref безопасны: project рекурсирует только от корней, циклический
компонент недостижим из корней (нет бесконечной рекурсии). Orphan (родитель
отфильтрован пермишенами) всплывает как корень.

Тесты: tree.test.mjs +9 (nesting+порядок+no-leak, maxDepth:1/2 + hasChildren,
orphan→root, seeded rootPageId, ≤0/нефинитный = без среза, lean-байт-идентичность),
tool-specs.test.mjs (схема getTree + депрекация listPages). mcp node --test
718/718, tsc чисто; server jest (shared-tool-specs.contract + ai-chat) 263/263.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 21:58:32 +03:00
agent_coder f794ac6d6c fix(search): оживить trgm-индекс — убрать coalesce из LIKE-предикатов (#443, ревью)
Ревьюер поймал EXPLAIN'ом: lookup-query фильтровал по
LOWER(f_unaccent(coalesce(col,''))), а GIN-trgm-индексы — на coalesce-free
LOWER(f_unaccent(col)). Postgres берёт функциональный индекс только при точном
совпадении выражения, так что обёртка coalesce отключала индекс → Seq Scan по
pages на КАЖДЫЙ lookup (MCP-клиент всегда шлёт substring:true). Зелёный
int-гейт это не ловил (проверял корректность на крошечном датасете, не
использование индекса).

- search.service.ts: убрал coalesce из двух index-driving LIKE-предикатов
  (title + text_content). Семантически эквивалентно: NULL LIKE '%q%' = NULL
  (falsy), NULL-страница просто не матчится — как пустая строка не матчит %q%.
  SELECT/ORDER BY оставлены с coalesce (индекс не выбирают, Node гардит NULL).
- Миграция: убран избыточный ре-ассерт idx_pages_title_trgm — его создаёт #348
  на том же coalesce-free выражении, теперь title-предикат его использует.
  idx_pages_text_content_trgm без изменений (уже coalesce-free), поправлены
  вводящие в заблуждение комментарии. down() дропает только text-индекс.
- Новый тест search-lookup-explain.int-spec.ts (3): title→idx_pages_title_trgm,
  text→idx_pages_text_content_trgm (оба ассертят отсутствие Seq Scan on pages),
  + негативный контроль (coalesce-обёртка не может использовать индекс) против
  тихой ре-регрессии. Дискриминатор enable_seqscan=off.
- CHANGELOG: две записи (opt-in substring lookup mode + смена shape MCP search).

EXPLAIN на реальном PG: обе fixed-ветки → Bitmap Index Scan on trgm; buggy
coalesce → Seq Scan (Disabled:true). Гейт: server jest src/core/search 27/27
(16 исходных int без изменений + 3 EXPLAIN + 8 unit), mcp node --test 708/708,
tsc чисто.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 21:57:16 +03:00
agent_coder e4e788f151 feat(search): агентский lookup-режим — substring по точкам/дефисам/цифрам, path, snippet, scope (#443, часть 1/3)
MCP-сервером пользуются LLM-агенты; основной паттерн — lookup: найти страницу по
обрывку технической строки и сразу понять, где она лежит и что в ней. Раньше на
такой вопрос уходило 3–5 вызовов. Эта часть закрывает `search` (часть 1 из 3;
get_tree и get_page_context — следующими PR стопкой). Только чтение.

Серверная часть — opt-in, веб-UI байт-в-байт неизменен:
- SearchDTO: опциональные substring/parentPageId/titleOnly (default-off; без
  флагов путь FTS не тронут — guard `if (substring) return searchPageLookup`).
- searchPageLookup: один скан pages, WHERE = title LIKE '%q%' OR (если не
  titleOnly) text LIKE '%q%' OR (если tsquery непустой) tsv @@ ... . LIKE-
  метасимволы %/_/\ экранируются (escapeLikePattern, ESCAPE '\') — `%`/`_` не
  матчат всё; substring-ветка работает даже при пустом tsquery (кейс 10.0.12).
- Ранжирование тирами (TITLE_EXACT > TITLE_SUBSTRING > TEXT), вторичный сигнал
  ts_rank / позиция; score∈(0,1] только для сортировки одной выдачи (формула в
  комментарии). 200-cap упорядочен по SQL-прокси тира ДО среза (иначе Postgres
  отдаёт произвольные 200 и сильный хит мог выпасть). Пермишен-фильтр к
  merged-набору ДО limit. path — одна рекурсивная CTE на все хиты (не N+1).
- snippet оконный в SQL (~500 символов вокруг первого совпадения). Позиция и
  срез в ОДНОМ пространстве LOWER(f_unaccent(...)) — f_unaccent не length-
  preserving (ß→ss, лигатуры, …→...), иначе окно смещалось/пустело. titleOnly →
  пустой snippet. Компромисс задокументирован.
- Миграция: GIN gin_trgm_ops по LOWER(f_unaccent(text_content)); title-trgm
  индекс #348 переиспользован (IF NOT EXISTS), down() дропает только новый.

MCP: схема search (spaceId/parentPageId/titleOnly/limit 1–50, default 10),
client.search прокидывает substring:true, filterSearchResult → {pageId, title,
path, snippet, score}. Инвариант: наружу только pageId (UUID), slugId/id
никогда. Комментарии про намеренное расхождение с in-app hybrid-RRF (не тронут)
и про деградацию на stock-upstream/Typesense (substring→plain FTS, без path/
snippet).

Проверка на реальном Postgres: server integration 16/16 (вся acceptance-таблица
#443 + регрессии на смещение snippet и cap-200), server unit 27/27, mcp
node --test 708/708, tsc чисто.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-10 21:34:53 +03:00
21 changed files with 2265 additions and 45 deletions
+16
View File
@@ -251,6 +251,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
by physical key position and matched against the commands; genuine Cyrillic
search terms keep priority over remapped candidates, and short wrong-layout
prefixes match by command title. (#283, #285, #287)
- **Opt-in substring "lookup" search mode for agents.** `/api/search` gains an
additive, opt-in mode (guarded by a new `substring` flag) that matches literal
substrings of page titles and body text — so technical tokens the full-text
tokenizer mangles (`backup-srv.local`, `10.0.12.5`, `WB-MGE-30D86B`) are found
even when the FTS query is empty. It returns a location `path`, a windowed
`snippet` and a per-response relevance `score`, supports `titleOnly` and a
`parentPageId` subtree scope, and applies the page-level permission filter
before the limit. The web UI never sets `substring`, so its full-text search
behaviour is byte-for-byte unchanged. The leading-wildcard `LIKE` predicates
are backed by GIN trigram indexes on `LOWER(f_unaccent(title))` and
`LOWER(f_unaccent(text_content))` so lookups use a bitmap index scan instead of
a sequential scan. (#443)
- **MCP `search` tool returns richer, agent-oriented results.** The external MCP
`search` response shape changes for the agent surface: each hit now carries
`pageId` (renamed from `id`), plus `path`, `snippet` and `score`; the
UI-oriented `spaceId`, `rank` and `highlight` fields are dropped. (#443)
### Changed
@@ -59,6 +59,8 @@ function __assertClientCallContract(client: DocmostClientLike): void {
void client.getWorkspace();
void client.getSpaces();
void client.listPages(s, n, true);
void client.getTree(s, s, n);
void client.getPageContext(s);
void client.listSidebarPages(s, s);
void client.getOutline(s);
void client.getPageJson(s);
@@ -23,6 +23,8 @@ type DocmostClientMethod =
| 'getWorkspace'
| 'getSpaces'
| 'listPages'
| 'getTree'
| 'getPageContext'
| 'listSidebarPages'
| 'getOutline'
| 'getPageJson'
@@ -12,3 +12,22 @@ export class SearchResponseDto {
updatedAt: Date;
space: Partial<Space>;
}
// Response shape for the opt-in agent-lookup mode (#443, `substring: true`).
// Additive to the FTS response: carries the location (`path`), a windowed
// `snippet` around the first match and a per-response sort `score`. The MCP
// layer maps `id → pageId`; `slugId` is never exposed.
export class SearchLookupResponseDto {
id: string;
slugId: string;
title: string;
parentPageId: string | null;
// Ancestor titles from the space root down to the direct parent; [] for a
// root page.
path: string[];
// ~300–500 chars around the first match (or a leading text window / extended
// ts_headline fallback).
snippet: string;
// 0..1 float, meaningful ONLY for sorting within one response.
score: number;
}
@@ -30,6 +30,31 @@ export class SearchDTO {
@IsOptional()
@IsNumber()
offset?: number;
// --- Opt-in agent-lookup mode (#443). ------------------------------------
// These fields are ADDITIVE and default-off: a web client that sends none of
// them gets byte-identical FTS behaviour and result shape. They are only read
// by the substring/path/snippet code path in SearchService.searchPage.
//
// NOTE (standalone stdio vs stock upstream): stock upstream validates this DTO
// with `whitelist: true`, so an older server silently strips these unknown
// fields and the request degrades gracefully to the plain FTS behaviour.
// Enables the hybrid substring branch (title + text_content LIKE) merged with
// the existing FTS branch, plus tiered ranking, path and windowed snippet.
@IsOptional()
@IsBoolean()
substring?: boolean;
// Restrict the search to a page and all of its descendants (inclusive).
@IsOptional()
@IsString()
parentPageId?: string;
// Match titles only; do not scan text_content.
@IsOptional()
@IsBoolean()
titleOnly?: boolean;
}
export class SearchShareDTO extends SearchDTO {
@@ -60,6 +60,12 @@ export class SearchController {
}
}
// #443 graceful degradation: on EE/Typesense instances the request routes to
// the Typesense backend, which does NOT implement the opt-in agent-lookup
// mode. The `substring`/`parentPageId`/`titleOnly` fields are silently ignored
// and the response carries no `path`/`snippet`/`score` and no substring/tier
// ranking — it degrades to plain Typesense FTS. The native lookup mode below
// is Postgres-search-driver only.
if (this.environmentService.getSearchDriver() === 'typesense') {
return this.searchTypesense(searchDto, {
userId: user.id,
@@ -0,0 +1,95 @@
import {
computeLookupScore,
escapeLikePattern,
SearchLookupTier,
} from './search.service';
/**
* Pure-function coverage for the #443 agent-lookup helpers:
* - escapeLikePattern: LIKE-metacharacter escaping so `%`/`_`/`\` are literals
* (the acceptance-table requirement that a query of `%` or `_` does NOT match
* everything);
* - computeLookupScore: the tiered 0..1 ranking score, where a stronger tier
* always outranks a weaker one regardless of the in-tier secondary signal.
*
* The DB-touching branch (substring UNION FTS, path CTE, snippet window) is
* covered by the integration spec against the real schema.
*/
describe('escapeLikePattern', () => {
it('escapes the LIKE metacharacters % _ and \\', () => {
expect(escapeLikePattern('%')).toBe('\\%');
expect(escapeLikePattern('_')).toBe('\\_');
expect(escapeLikePattern('\\')).toBe('\\\\');
});
it('escapes the backslash FIRST so it does not double-escape %/_', () => {
// Input `\%` must become `\\` + `\%` = `\\\%`, not `\\%`.
expect(escapeLikePattern('\\%')).toBe('\\\\\\%');
});
it('leaves ordinary technical chars (. - / digits) untouched', () => {
expect(escapeLikePattern('backup-srv.local')).toBe('backup-srv.local');
expect(escapeLikePattern('10.0.12')).toBe('10.0.12');
expect(escapeLikePattern('WB-MGE-30D86B')).toBe('WB-MGE-30D86B');
expect(escapeLikePattern('a/b')).toBe('a/b');
});
it('escapes only the metacharacters in a mixed string', () => {
expect(escapeLikePattern('50%_off.zip')).toBe('50\\%\\_off.zip');
});
it('is null/undefined-safe', () => {
expect(escapeLikePattern(undefined as any)).toBe('');
expect(escapeLikePattern(null as any)).toBe('');
});
});
describe('computeLookupScore', () => {
it('keeps every score within (0, 1]', () => {
for (const tier of [
SearchLookupTier.TITLE_EXACT,
SearchLookupTier.TITLE_SUBSTRING,
SearchLookupTier.TEXT,
]) {
for (const secondary of [0, 0.001, 1, 100, 1e6]) {
const s = computeLookupScore({ tier, secondary });
expect(s).toBeGreaterThan(0);
expect(s).toBeLessThanOrEqual(1);
}
}
});
it('a stronger tier ALWAYS outranks a weaker tier, whatever the secondary', () => {
// Weak tier with a huge secondary must still lose to a strong tier with a
// tiny secondary — tiers dominate.
const strongLowSecondary = computeLookupScore({
tier: SearchLookupTier.TITLE_EXACT,
secondary: 0,
});
const weakHighSecondary = computeLookupScore({
tier: SearchLookupTier.TEXT,
secondary: 1e9,
});
expect(strongLowSecondary).toBeGreaterThan(weakHighSecondary);
});
it('within a tier a larger secondary sorts higher', () => {
const lo = computeLookupScore({
tier: SearchLookupTier.TEXT,
secondary: 0.1,
});
const hi = computeLookupScore({
tier: SearchLookupTier.TEXT,
secondary: 5,
});
expect(hi).toBeGreaterThan(lo);
});
it('treats a negative/absent secondary as 0', () => {
const zero = computeLookupScore({ tier: SearchLookupTier.TEXT, secondary: 0 });
expect(computeLookupScore({ tier: SearchLookupTier.TEXT })).toBe(zero);
expect(
computeLookupScore({ tier: SearchLookupTier.TEXT, secondary: -5 }),
).toBe(zero);
});
});
+401 -2
View File
@@ -1,6 +1,9 @@
import { Injectable } from '@nestjs/common';
import { SearchDTO, SearchSuggestionDTO } from './dto/search.dto';
import { SearchResponseDto } from './dto/search-response.dto';
import {
SearchLookupResponseDto,
SearchResponseDto,
} from './dto/search-response.dto';
import { InjectKysely } from 'nestjs-kysely';
import { KyselyDB } from '@docmost/db/types/kysely.types';
import { sql } from 'kysely';
@@ -34,6 +37,53 @@ export function buildTsQuery(raw: string): string {
return tsquery(cleaned + '*');
}
// Escape the LIKE metacharacters (`%`, `_`, `\`) in a raw user query so every
// character — including `.`, `-`, `_`, `%`, `/` — is matched LITERALLY by a
// `col LIKE '%' || q || '%'` predicate. Without this, a query of `%` or `_`
// would match every row (see the #443 acceptance table). The backslash is the
// escape char (Postgres LIKE default), so it must be escaped first.
export function escapeLikePattern(raw: string): string {
return (raw ?? '')
.replace(/\\/g, '\\\\')
.replace(/%/g, '\\%')
.replace(/_/g, '\\_');
}
// Ranking tiers for the agent-lookup mode (#443), highest first. A hit's tier
// is the strongest way it matched; ties inside a tier break on a secondary
// signal (FTS rank, or first-match position). The numeric `score` returned to
// the caller is derived from (tier, secondary) and is meaningful ONLY for
// ordering within a single response.
export enum SearchLookupTier {
// Title equals the query, case-insensitively.
TITLE_EXACT = 3,
// Query is a substring of the title.
TITLE_SUBSTRING = 2,
// Query matched in the text (substring or FTS).
TEXT = 1,
}
export interface RankableHit {
tier: SearchLookupTier;
// Secondary in-tier signal, higher = better (e.g. ts_rank, or a
// position-derived closeness score). Defaults to 0.
secondary?: number;
}
// Map (tier, secondary) → a 0..1 float used ONLY to sort one response.
//
// Formula: score = (tier + squash(secondary)) / (maxTier + 1), where
// squash(x) = x / (1 + x) maps any non-negative secondary into [0, 1)
// so a stronger tier ALWAYS outranks a weaker one regardless of the secondary
// value, and within a tier a larger secondary sorts higher. maxTier is the top
// enum value (TITLE_EXACT = 3), so the divisor keeps the result in (0, 1].
export function computeLookupScore(hit: RankableHit): number {
const maxTier = SearchLookupTier.TITLE_EXACT;
const secondary = Math.max(0, hit.secondary ?? 0);
const squashed = secondary / (1 + secondary);
return (hit.tier + squashed) / (maxTier + 1);
}
@Injectable()
export class SearchService {
constructor(
@@ -50,12 +100,19 @@ export class SearchService {
userId?: string;
workspaceId: string;
},
): Promise<{ items: SearchResponseDto[] }> {
): Promise<{ items: SearchResponseDto[] | SearchLookupResponseDto[] }> {
const { query } = searchParams;
if (query.length < 1) {
return { items: [] };
}
// Opt-in agent-lookup mode (#443). Guarded by the `substring` flag so the
// web-UI (which never sets it) keeps byte-identical FTS behaviour below.
if (searchParams.substring) {
return this.searchPageLookup(searchParams, opts);
}
const searchQuery = buildTsQuery(query);
let queryResults = this.db
@@ -175,6 +232,348 @@ export class SearchService {
return { items: searchResults };
}
/**
* Agent-lookup search (#443, opt-in via `SearchDTO.substring`).
*
* ADDITIVE to the FTS path: runs a substring branch (title + optionally
* text_content, LIKE with metacharacters escaped) MERGED with the existing
* FTS branch, so technical tokens that the `english` tokenizer mangles
* (`backup-srv.local`, `10.0.12.5`, `WB-MGE-30D86B`) are still found even
* when `buildTsQuery()` returns '' for a dotted/numeric query. Results carry a
* location (`path`), a windowed `snippet` and a per-response `score`.
*
* The whole method is only reached when `substring: true`; the web-UI never
* sets it, so its behaviour is unchanged.
*/
private async searchPageLookup(
searchParams: SearchDTO,
opts: { userId?: string; workspaceId: string },
): Promise<{ items: SearchLookupResponseDto[] }> {
const rawQuery = searchParams.query.trim();
if (!rawQuery) {
return { items: [] };
}
const limit = Math.min(Math.max(searchParams.limit || 10, 1), 50);
// Normalize the query the same way as the FTS / suggest path: f_unaccent +
// lower, done in SQL. `q` is the escaped LIKE pattern body (literal chars).
const likeBody = escapeLikePattern(rawQuery);
// Compare against `LOWER(f_unaccent(col))`; unaccent+lower the needle too.
const needle = sql<string>`LOWER(f_unaccent(${rawQuery}))`;
const likePattern = sql<string>`LOWER(f_unaccent(${'%' + likeBody + '%'}))`;
const tsQuery = buildTsQuery(rawQuery);
const hasTsQuery = tsQuery.length > 0;
// --- Resolve the space scope. ---------------------------------------------
// Mirrors searchPage: explicit spaceId, else the authenticated user's member
// spaces. The share path is not exposed to this opt-in mode.
let spaceIds: string[] = [];
if (searchParams.spaceId) {
spaceIds = [searchParams.spaceId];
} else if (opts.userId) {
spaceIds = await this.spaceMemberRepo.getUserSpaceIds(opts.userId);
} else {
return { items: [] };
}
if (spaceIds.length === 0) {
return { items: [] };
}
// --- Optional parentPageId subtree scope (inclusive). ---------------------
// Reuse the same recursive-descendants pattern used for share-scope.
let descendantIds: string[] | null = null;
if (searchParams.parentPageId) {
const descendants = await this.pageRepo.getPageAndDescendants(
searchParams.parentPageId,
{ includeContent: false },
);
descendantIds = descendants.map((p: any) => p.id);
if (descendantIds.length === 0) {
return { items: [] };
}
}
// --- Candidate query: substring (title + text) UNION FTS. -----------------
// We compute everything the ranker needs in SQL and pull only small columns
// (never the whole text_content) into Node:
// - titleExact / titleSub: tier signals
// - textMatchPos: 1-based position of the first text match (0 = none)
// - ftsRank: ts_rank for the FTS secondary signal (0 when no tsquery)
// - snippet: windowed ~500 chars around the first text match, or a leading
// text window (title-only hit), or an extended ts_headline fallback.
const N_BEFORE = 60; // chars of context before the first match
const SNIPPET_LEN = 500;
let candidates = this.db
.selectFrom('pages')
.select([
'pages.id as id',
'pages.slugId as slugId',
'pages.title as title',
'pages.parentPageId as parentPageId',
// Tier signals.
sql<boolean>`LOWER(f_unaccent(coalesce(pages.title, ''))) = ${needle}`.as(
'titleExact',
),
sql<boolean>`LOWER(f_unaccent(coalesce(pages.title, ''))) LIKE ${likePattern} ESCAPE '\\'`.as(
'titleSub',
),
// 1-based position of the first text match (0 = no text match).
sql<number>`strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle})`.as(
'textMatchPos',
),
// FTS secondary signal (0 when the tsquery is empty).
hasTsQuery
? sql<number>`ts_rank(pages.tsv, to_tsquery('english', f_unaccent(${tsQuery})))`.as(
'ftsRank',
)
: sql<number>`0`.as('ftsRank'),
// Windowed snippet, computed entirely in SQL. Priority:
// 1. window around the first text match;
// 2. otherwise (titleOnly: no snippet; else) a leading window of the
// page text (title-only hit);
// 3. otherwise an extended ts_headline for pure-FTS hits.
//
// #443 snippet-position fix: the match position (`strpos`) is computed in
// the LOWER(f_unaccent(...)) space, but f_unaccent is NOT length-
// preserving (ß→ss, æ→ae, …→..., ½→ 1/2, full-width forms), so slicing
// the ORIGINAL text at that position was misaligned — a single expanding
// char before the match shifted the window (or ran it past end → empty).
// We now slice from the SAME LOWER(f_unaccent(...)) string so position
// and slice share one coordinate space. DELIBERATE trade-off: the snippet
// loses original case/diacritics — acceptable for an agent-facing snippet
// (position accuracy over original-glyph fidelity). The ts_headline branch
// matches over the ORIGINAL text itself, so it is unaffected and kept as-is.
searchParams.titleOnly
? sql<string>`''`.as('snippet')
: sql<string>`
coalesce(
case
when strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) > 0
then substring(
LOWER(f_unaccent(coalesce(pages.text_content, '')))
from greatest(1, strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) - ${N_BEFORE})
for ${SNIPPET_LEN}
)
when coalesce(pages.text_content, '') <> ''
then substring(LOWER(f_unaccent(pages.text_content)) from 1 for 300)
${
hasTsQuery
? sql`else ts_headline('english', coalesce(pages.text_content, ''), to_tsquery('english', f_unaccent(${tsQuery})), 'MinWords=25, MaxWords=40, MaxFragments=3')`
: sql``
}
end,
''
)
`.as('snippet'),
])
.where('pages.deletedAt', 'is', null)
.where('pages.spaceId', 'in', spaceIds);
if (descendantIds) {
candidates = candidates.where('pages.id', 'in', descendantIds);
}
// Match predicate: title substring OR (unless titleOnly) text substring OR
// (unless titleOnly) FTS. The substring branch runs even when the tsquery is
// empty — that is the dotted/numeric-token case the FTS path misses.
//
// #443 dead-index fix: these two LIKE predicates MUST match the GIN trgm
// index expressions EXACTLY for Postgres to use them. The indexes are on the
// coalesce-FREE expressions `LOWER(f_unaccent(title))` (#348's
// idx_pages_title_trgm) and `LOWER(f_unaccent(text_content))` (this PR's
// idx_pages_text_content_trgm). A `coalesce(col,'')` wrapper here would make
// the query expression differ from the index expression and force a Seq Scan
// on pages for every lookup. Dropping coalesce is SEMANTICALLY EQUIVALENT:
// `NULL LIKE '%q%'` is NULL (falsy), so a NULL title/text simply doesn't
// match — exactly as an empty string wouldn't match `%q%`.
candidates = candidates.where((eb) => {
const ors = [
eb(
sql`LOWER(f_unaccent(pages.title))`,
'like',
sql`${likePattern} ESCAPE '\\'`,
),
];
if (!searchParams.titleOnly) {
ors.push(
eb(
sql`LOWER(f_unaccent(pages.text_content))`,
'like',
sql`${likePattern} ESCAPE '\\'`,
),
);
if (hasTsQuery) {
ors.push(
sql<boolean>`pages.tsv @@ to_tsquery('english', f_unaccent(${tsQuery}))` as any,
);
}
}
return eb.or(ors);
});
// Pull a generous candidate set (before permission filtering + limit).
// Cap it so a pathological match set cannot blow up memory; 200 >> limit
// (max 50) leaves ample headroom for the post-permission truncation.
//
// #443 cap-ordering fix: the 200-cap MUST be deterministic and relevance-
// biased. Without an ORDER BY, Postgres returns an ARBITRARY 200 rows, so on
// a broad match set (common word / short substring) a strong TITLE_EXACT hit
// could be among the dropped rows while 200 low-tier TEXT hits fill the cap.
// We order by the SAME SQL tier proxies the Node ranker uses — title-exact,
// then title-substring, then fts-rank (nulls last), then earliest text-match
// position — so the cap keeps the strongest candidates. The Node-side final
// tier sort + slice(0, limit) below still runs and stays authoritative; this
// ORDER BY only decides WHICH candidates survive the 200-cap.
// NB: a BARE integer literal in ORDER BY is read by Postgres as an ordinal
// column position (`ORDER BY 0` → "position 0 is not in select list"), so the
// no-tsquery fallback is `0::float`, not `0`.
const ftsRankExpr = hasTsQuery
? sql`ts_rank(pages.tsv, to_tsquery('english', f_unaccent(${tsQuery})))`
: sql`0::float`;
const candidatesCapped = candidates
// Raw-SQL ORDER BY expressions: pass the full `<expr> <dir>` as ONE arg
// (the two-arg form treats a raw-SQL second arg as an ORDER BY position).
.orderBy(
sql`(LOWER(f_unaccent(coalesce(pages.title, ''))) = ${needle}) desc`,
)
.orderBy(
sql`(LOWER(f_unaccent(coalesce(pages.title, ''))) LIKE ${likePattern} ESCAPE '\\') desc`,
)
.orderBy(sql`${ftsRankExpr} desc nulls last`)
// Earlier text match first; strpos returns 0 for "no match", which would
// sort BEFORE a real (>=1) position under plain ASC, so push 0 to the end.
.orderBy(
sql`case when strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) = 0 then 2147483647 else strpos(LOWER(f_unaccent(coalesce(pages.text_content, ''))), ${needle}) end asc`,
);
let rows: any[] = await candidatesCapped.limit(200).execute();
if (rows.length === 0) {
return { items: [] };
}
// --- Permissions BEFORE limit. --------------------------------------------
// Apply the existing page-level post-filter to the MERGED set, then rank and
// only THEN truncate to `limit` — never lose the permission filter.
if (opts.userId) {
const accessibleIds =
await this.pagePermissionRepo.filterAccessiblePageIds({
pageIds: rows.map((r) => r.id),
userId: opts.userId,
spaceId: searchParams.spaceId,
workspaceId: opts.workspaceId,
});
const accessibleSet = new Set(accessibleIds);
rows = rows.filter((r) => accessibleSet.has(r.id));
}
if (rows.length === 0) {
return { items: [] };
}
// --- Tiered ranking + dedup. ----------------------------------------------
// Rows are already unique by id (single pages scan), so no cross-branch
// dedup is needed here; the tier captures the strongest match reason.
const ranked = rows.map((r) => {
let tier: SearchLookupTier;
let secondary: number;
if (r.titleExact) {
tier = SearchLookupTier.TITLE_EXACT;
secondary = Number(r.ftsRank) || 0;
} else if (r.titleSub) {
tier = SearchLookupTier.TITLE_SUBSTRING;
secondary = Number(r.ftsRank) || 0;
} else {
tier = SearchLookupTier.TEXT;
// Prefer earlier text matches; map position → closeness in (0, 1].
const pos = Number(r.textMatchPos) || 0;
secondary =
pos > 0 ? 1 / (1 + (pos - 1) / 100) : Number(r.ftsRank) || 0;
}
return { row: r, tier, score: computeLookupScore({ tier, secondary }) };
});
ranked.sort((a, b) => b.score - a.score);
const top = ranked.slice(0, limit);
// --- Batch ancestor path (ONE recursive CTE, not N+1). --------------------
const pathById = await this.buildAncestorPaths(top.map((t) => t.row.id));
const items: SearchLookupResponseDto[] = top.map((t) => ({
id: t.row.id,
slugId: t.row.slugId,
title: t.row.title,
parentPageId: t.row.parentPageId ?? null,
path: pathById.get(t.row.id) ?? [],
snippet: (t.row.snippet ?? '')
.replace(/\r\n|\r|\n/g, ' ')
.replace(/\s+/g, ' ')
.trim(),
score: t.score,
}));
return { items };
}
/**
* Batch ancestor-titles helper (#443): ONE recursive CTE seeded with ALL hit
* ids, walking UP parentPageId. Returns a map hitId ancestor titles ordered
* root direct parent (the hit's own title is excluded). Root pages map to
* an empty array. Avoids the N+1 of a per-page breadcrumb call.
*/
private async buildAncestorPaths(
hitIds: string[],
): Promise<Map<string, string[]>> {
const result = new Map<string, string[]>();
if (hitIds.length === 0) return result;
// ancestry(hit_id, page_id, title, parent_page_id, depth): seed one row per
// hit at depth 0 (the hit itself), then walk to parents (increasing depth).
const rows = await this.db
.withRecursive('ancestry', (db) =>
db
.selectFrom('pages')
.select([
'pages.id as hitId',
'pages.id as pageId',
'pages.title as title',
'pages.parentPageId as parentPageId',
sql<number>`0`.as('depth'),
])
.where('pages.id', 'in', hitIds)
.unionAll((exp) =>
exp
.selectFrom('pages as p')
.innerJoin('ancestry as a', 'p.id', 'a.parentPageId')
.select([
'a.hitId as hitId',
'p.id as pageId',
'p.title as title',
'p.parentPageId as parentPageId',
sql<number>`a.depth + 1`.as('depth'),
]),
),
)
.selectFrom('ancestry')
.select(['hitId', 'title', 'depth'])
// depth 0 is the hit itself — excluded from the path.
.where('depth', '>', 0)
.orderBy('hitId')
// Larger depth = closer to the space root. Ordering DESC gives
// root → parent once collected.
.orderBy('depth', 'desc')
.execute();
for (const r of rows as any[]) {
const list = result.get(r.hitId) ?? [];
list.push(r.title);
result.set(r.hitId, list);
}
return result;
}
async searchSuggestions(
suggestion: SearchSuggestionDTO,
userId: string,
@@ -0,0 +1,55 @@
import { type Kysely, sql } from 'kysely';
/**
* #443 trigram indexes for the opt-in agent-lookup search mode.
*
* The lookup mode adds a substring branch that runs leading-wildcard
* `LOWER(f_unaccent(col)) LIKE '%q%'` predicates on pages.title and
* pages.text_content. A leading wildcard cannot use a b-tree index, so without a
* GIN trigram index each such predicate is a sequential scan.
*
* - TITLE: the lookup-mode title predicate is `LOWER(f_unaccent(title)) LIKE
* '%q%'` (coalesce-free, so it can use a functional index), which is IDENTICAL
* to the one added for /search/suggest (#348). #348's perf-indexes migration
* already created `idx_pages_title_trgm` on `(LOWER(f_unaccent(title)))
* gin_trgm_ops`, so the title predicate is already covered — we do NOT
* re-create that index here (it would be redundant).
*
* - TEXT_CONTENT: NEW. The substring branch scans text_content when the query
* is not titleOnly. text_content is the large column, so a GIN trigram index
* on it is the meaningful acceleration for the lookup mode. The lookup search
* is ALWAYS space-scoped (spaceId or the user's member spaces), so on small
* instances a per-space sequential scan is tolerable but the index turns the
* `%q%` text predicate into a Bitmap Index Scan and removes the only
* unbounded-per-space cost of the feature. We add it. The trade-off is disk +
* write amplification on page edits (GIN trigram indexes are larger and slower
* to update than b-trees); on the small instances this fork targets that cost
* is acceptable and the read win on agent lookups is the priority.
*
* DEPLOY-TIME LOCK WARNING: plain (non-CONCURRENT) CREATE INDEX Kysely runs
* each migration in a transaction, so CONCURRENTLY is impossible. The build takes
* a SHARE lock that BLOCKS writes on `pages` for its duration. The text_content
* GIN build is the slow one and can take minutes on a large tenant. For large
* installations, run this in a maintenance window or build the index out-of-band
* with CREATE INDEX CONCURRENTLY before deploying (then `IF NOT EXISTS` no-ops
* here). Small/typical tenants are unaffected.
*/
export async function up(db: Kysely<any>): Promise<void> {
// The title predicate is served by #348's idx_pages_title_trgm — see header.
// Only the text_content index is introduced here.
// text_content trigram index. Its expression is coalesce-free —
// `LOWER(f_unaccent(text_content))` — to EXACTLY match the coalesce-free
// lookup-mode text substring predicate in search.service.ts, so Postgres can
// use it (a `coalesce(...)` mismatch would silently fall back to a Seq Scan).
await sql`
CREATE INDEX IF NOT EXISTS idx_pages_text_content_trgm
ON pages USING gin ((LOWER(f_unaccent(text_content))) gin_trgm_ops)
`.execute(db);
}
export async function down(db: Kysely<any>): Promise<void> {
// Only drop the index this migration introduced. idx_pages_title_trgm is owned
// by the #348 perf-indexes migration, so leave it for that migration's down().
await sql`DROP INDEX IF EXISTS idx_pages_text_content_trgm`.execute(db);
}
@@ -0,0 +1,123 @@
import { randomUUID } from 'node:crypto';
import { Kysely, sql } from 'kysely';
import {
getTestDb,
destroyTestDb,
createWorkspace,
createSpace,
} from './db';
/**
* #443 dead-index guard EXPLAIN on the REAL DB.
*
* The lookup mode's substring predicates run a leading-wildcard
* `LOWER(f_unaccent(col)) LIKE '%q%'`. Those are only fast when Postgres uses
* the GIN trigram indexes:
* - idx_pages_title_trgm on (LOWER(f_unaccent(title))) [#348]
* - idx_pages_text_content_trgm on (LOWER(f_unaccent(text_content))) [#443]
*
* Postgres uses a functional index ONLY when the query expression matches the
* index expression EXACTLY. The original lookup query wrapped the columns in
* `coalesce(col,'')`, which differs from the coalesce-FREE index expression and
* silently forced a Seq Scan on pages for EVERY lookup (the MCP client always
* sends substring:true). This test locks that in.
*
* Discriminator: `SET enable_seqscan = off` asks the planner "CAN this predicate
* use the index at all?" which is exactly what the coalesce bug breaks. With
* seqscan disabled:
* - the coalesce-FREE (fixed) predicate plans a Bitmap Index Scan on the trgm
* index (no Seq Scan on pages);
* - the coalesce-WRAPPED (buggy) predicate cannot use the index and falls back
* to a Seq Scan on pages even though seqscan is disabled.
* We assert both to prove the fix and to keep the regression from silently
* returning.
*/
describe('SearchService agent-lookup EXPLAIN — trgm index is live [integration]', () => {
let db: Kysely<any>;
let workspaceId: string;
let spaceId: string;
async function insertPage(title: string, textContent: string): Promise<void> {
const id = randomUUID();
await db
.insertInto('pages')
.values({
id,
slugId: `slug-${id.slice(0, 12)}`,
title,
textContent,
spaceId,
workspaceId,
})
.execute();
}
// Run EXPLAIN (no ANALYZE — we only inspect the chosen plan) and return the
// concatenated plan text.
async function explain(query: string): Promise<string> {
const rows = await sql<{ 'QUERY PLAN': string }>`EXPLAIN ${sql.raw(query)}`.execute(
db,
);
return (rows.rows as any[]).map((r) => r['QUERY PLAN']).join('\n');
}
beforeAll(async () => {
db = getTestDb();
workspaceId = (await createWorkspace(db)).id;
spaceId = (await createSpace(db, workspaceId)).id;
// Seed enough rows that a trigram index is a plausible plan. The content is
// varied so the '%needle%' pattern is selective.
for (let i = 0; i < 200; i++) {
await insertPage(
`seed-title-${i}`,
`seed body content number ${i} lorem ipsum dolor sit amet ${i}`,
);
}
await insertPage('backup-srv.local', 'the needle-token-xyz lives here');
// Keep the trgm indexes' stats fresh so the planner costs them correctly.
await sql`ANALYZE pages`.execute(db);
});
afterAll(async () => {
await destroyTestDb();
});
// Force the planner to answer "can the index be used?" rather than "is it
// cheaper than a seq scan on this size?". Restored after each test.
beforeEach(async () => {
await sql`SET enable_seqscan = off`.execute(db);
});
afterEach(async () => {
await sql`RESET enable_seqscan`.execute(db);
});
it('title predicate (coalesce-FREE, as fixed) uses idx_pages_title_trgm, not a Seq Scan', async () => {
const plan = await explain(
`SELECT id FROM pages WHERE LOWER(f_unaccent(title)) LIKE '%srv.local%'`,
);
expect(plan).toContain('idx_pages_title_trgm');
expect(plan).not.toMatch(/Seq Scan on pages/i);
});
it('text_content predicate (coalesce-FREE, as fixed) uses idx_pages_text_content_trgm, not a Seq Scan', async () => {
const plan = await explain(
`SELECT id FROM pages WHERE LOWER(f_unaccent(text_content)) LIKE '%needle-token%'`,
);
expect(plan).toContain('idx_pages_text_content_trgm');
expect(plan).not.toMatch(/Seq Scan on pages/i);
});
// Negative control: the OLD coalesce-wrapped predicate must NOT be able to use
// the index — even with seqscan disabled it can only Seq Scan pages. If this
// ever stops seq-scanning, the coalesce/index expressions have re-aligned and
// the guard above is no longer meaningful.
it('coalesce-WRAPPED text predicate (the bug) cannot use the index — falls to Seq Scan', async () => {
const plan = await explain(
`SELECT id FROM pages WHERE LOWER(f_unaccent(coalesce(text_content,''))) LIKE '%needle-token%'`,
);
expect(plan).not.toContain('idx_pages_text_content_trgm');
expect(plan).toMatch(/Seq Scan on pages/i);
});
});
@@ -0,0 +1,462 @@
import { randomUUID } from 'node:crypto';
import { Kysely } from 'kysely';
import { SearchService } from 'src/core/search/search.service';
import { PageRepo } from '@docmost/db/repos/page/page.repo';
import {
getTestDb,
destroyTestDb,
createWorkspace,
createSpace,
} from './db';
/**
* #443 agent-lookup search mode, acceptance on the REAL DB schema.
*
* Exercises SearchService.searchPage(..., { substring: true }) against a
* migrated Postgres: substring matching of technical tokens the FTS tokenizer
* mangles (backup-srv.local, 10.0.12.5, WB-MGE-30D86B, "Теги: Docker"), the
* populated path + snippet, parentPageId subtree scoping, titleOnly, the empty
* result, LIKE-metacharacter escaping (`%`/`_` must NOT match everything), the
* permission post-filter applied BEFORE the limit, and the web-UI path staying
* on the legacy FTS shape when `substring` is absent.
*
* The tsv column is populated by the pages_tsvector_trigger on insert, so the
* FTS branch is exercised too.
*/
describe('SearchService agent-lookup mode [integration]', () => {
let db: Kysely<any>;
let service: SearchService;
let workspaceId: string;
let spaceId: string;
// Direct page insert (the shared createPage seeder omits text_content /
// parent_page_id, both of which this mode depends on). Returns the id.
async function insertPage(args: {
title: string;
textContent?: string;
parentPageId?: string | null;
spaceId?: string;
}): Promise<string> {
const id = randomUUID();
await db
.insertInto('pages')
.values({
id,
slugId: `slug-${id.slice(0, 12)}`,
title: args.title,
textContent: args.textContent ?? null,
parentPageId: args.parentPageId ?? null,
spaceId: args.spaceId ?? spaceId,
workspaceId,
})
.execute();
return id;
}
// Build a SearchService wired to the real DB + a real PageRepo (only its
// recursive-descendants method is used by this mode, and it needs only `db`),
// with lightweight stubs for the space-membership and permission repos so a
// test can drive scope + the permission post-filter explicitly.
function buildService(opts?: {
userSpaceIds?: string[];
// ids to KEEP after the permission post-filter; undefined = keep all.
accessibleIds?: string[];
}): SearchService {
const pageRepo = new PageRepo(db as any, null as any, null as any);
const spaceMemberRepo = {
getUserSpaceIds: async () => opts?.userSpaceIds ?? [spaceId],
};
const pagePermissionRepo = {
filterAccessiblePageIds: async ({ pageIds }: { pageIds: string[] }) =>
opts?.accessibleIds
? pageIds.filter((id) => opts.accessibleIds!.includes(id))
: pageIds,
};
return new SearchService(
db as any,
pageRepo as any,
{} as any, // shareRepo — unused by the lookup path
spaceMemberRepo as any,
pagePermissionRepo as any,
);
}
beforeAll(async () => {
db = getTestDb();
workspaceId = (await createWorkspace(db)).id;
spaceId = (await createSpace(db, workspaceId)).id;
service = buildService();
});
afterAll(async () => {
await destroyTestDb();
});
it('finds `backup-srv.local` by the fragment `srv.local`', async () => {
const pageId = await insertPage({
title: 'backup-srv.local',
textContent: 'A backup server node.',
});
const { items } = (await service.searchPage(
{ query: 'srv.local', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
expect(items.map((i: any) => i.id)).toContain(pageId);
const hit = items.find((i: any) => i.id === pageId);
expect(hit.title).toBe('backup-srv.local');
// slugId must never be part of the server response shape.
expect('slugId' in hit).toBe(true); // server carries it; MCP strips it
});
it('finds a page whose TEXT contains `10.0.12.5` by the fragment `10.0.12` (empty-tsquery case)', async () => {
const pageId = await insertPage({
title: 'Server inventory',
textContent: 'The backup box lives at IP: 10.0.12.5. Debian 12, backups.',
});
const { items } = (await service.searchPage(
{ query: '10.0.12', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// The windowed snippet must include the matched text.
expect(hit.snippet).toContain('10.0.12.5');
});
it('finds `WB-MGE-30D86B` (alphanumeric token with dashes) by title', async () => {
const pageId = await insertPage({
title: 'WB-MGE-30D86B',
textContent: 'Device page.',
});
const { items } = (await service.searchPage(
{ query: 'WB-MGE-30D86B', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// Exact title match → top tier (TITLE_EXACT=3) → score in [0.75, 1].
expect(hit.score).toBeGreaterThanOrEqual(0.75);
// And it is the top-ranked hit of its own result set.
expect(items[0].id).toBe(pageId);
});
it('finds every page whose text literally contains `Теги: Docker`', async () => {
const a = await insertPage({
title: 'Container host A',
textContent: 'Some notes.\nТеги: Docker, compose\nmore.',
});
const b = await insertPage({
title: 'Container host B',
textContent: 'Prelude.\nТеги: Docker\nepilogue.',
});
const noise = await insertPage({
title: 'Unrelated',
textContent: 'Теги: Kubernetes',
});
const { items } = (await service.searchPage(
{ query: 'Теги: Docker', spaceId, substring: true, limit: 50 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(a);
expect(ids).toContain(b);
expect(ids).not.toContain(noise);
});
it('populates a non-empty `path` for a nested hit and `[]` for a root hit', async () => {
const root = await insertPage({ title: 'Infrastructure' });
const mid = await insertPage({ title: 'Datacenter A', parentPageId: root });
const leaf = await insertPage({
title: 'unique-nested-host',
parentPageId: mid,
});
const { items } = (await service.searchPage(
{ query: 'unique-nested-host', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === leaf);
expect(hit.path).toEqual(['Infrastructure', 'Datacenter A']);
const rootHits = (await service.searchPage(
{ query: 'Infrastructure', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const rootHit = rootHits.items.find((i: any) => i.id === root);
expect(rootHit.path).toEqual([]);
});
it('scopes to a subtree with parentPageId (cutting off sibling branches)', async () => {
const branchA = await insertPage({ title: 'BranchA-root' });
const inA = await insertPage({
title: 'scoped-target-xyz',
parentPageId: branchA,
});
const branchB = await insertPage({ title: 'BranchB-root' });
const inB = await insertPage({
title: 'scoped-target-xyz',
parentPageId: branchB,
});
const { items } = (await service.searchPage(
{
query: 'scoped-target-xyz',
spaceId,
substring: true,
parentPageId: branchA,
} as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(inA);
expect(ids).not.toContain(inB);
});
it('includes the parent page itself in the parentPageId subtree', async () => {
const parent = await insertPage({ title: 'self-included-parent' });
await insertPage({ title: 'child-of-self', parentPageId: parent });
const { items } = (await service.searchPage(
{
query: 'self-included-parent',
spaceId,
substring: true,
parentPageId: parent,
} as any,
{ workspaceId },
)) as any;
expect(items.map((i: any) => i.id)).toContain(parent);
});
it('titleOnly does NOT match on text_content', async () => {
const pageId = await insertPage({
title: 'Plain title',
textContent: 'body mentions the-secret-token here',
});
const withText = (await service.searchPage(
{ query: 'the-secret-token', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
expect(withText.items.map((i: any) => i.id)).toContain(pageId);
const titleOnly = (await service.searchPage(
{
query: 'the-secret-token',
spaceId,
substring: true,
titleOnly: true,
} as any,
{ workspaceId },
)) as any;
expect(titleOnly.items.map((i: any) => i.id)).not.toContain(pageId);
});
// #443 Fix #1 regression: f_unaccent is NOT length-preserving, so an
// expanding char (ß→ss, …→...) BEFORE the match shifted the strpos position
// relative to the ORIGINAL text and the snippet slice ran past end → empty.
// The position and the slice now share the LOWER(f_unaccent(...)) space, so
// the window is aligned and always contains the matched (unaccented) token.
it('returns a populated snippet when an unaccent-EXPANDING char precedes the match', async () => {
// 300 × `ß` (each f_unaccent-expands to `ss`) before the needle. Under the
// old code strpos returned a position ~593 in the expanded space but the
// slice ran over the ORIGINAL (~360 char) text → empty snippet, match lost.
const prefix = 'ß'.repeat(300);
const pageId = await insertPage({
title: 'Expanding-unaccent page',
textContent: `${prefix} needle-token-xyz trailing.`,
});
const { items } = (await service.searchPage(
{ query: 'needle-token-xyz', spaceId, substring: true } as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// Snippet must be non-empty AND contain the matched token (unaccented form).
expect(hit.snippet.length).toBeGreaterThan(0);
expect(hit.snippet).toContain('needle-token-xyz');
});
// #443 Fix #2 regression: >200 matching pages for a broad substring, with
// exactly ONE exact-title hit. Without an ORDER BY on the 200-cap the exact
// hit could be among the arbitrarily-dropped rows; the ORDER BY keeps the
// strongest candidates so it must survive the cap and rank at the top.
it('keeps an exact-title hit through the 200-cap on a >200-row match set', async () => {
const isoSpace = (await createSpace(db, workspaceId)).id;
const svc = buildService({ userSpaceIds: [isoSpace] });
// 250 low-tier TEXT hits: the shared substring `capword` appears only in the
// body, never the title, so each is a TEXT-tier match (weakest tier).
for (let i = 0; i < 250; i++) {
await insertPage({
title: `filler-page-${i}`,
textContent: `body contains capword here #${i}`,
spaceId: isoSpace,
});
}
// Exactly one EXACT-title hit for the same query token.
const exact = await insertPage({
title: 'capword',
textContent: 'unrelated body text',
spaceId: isoSpace,
});
const { items } = (await svc.searchPage(
{ query: 'capword', spaceId: isoSpace, substring: true, limit: 10 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
// The exact-title hit must survive the 200-cap and appear in the top `limit`.
expect(ids).toContain(exact);
// And, being TITLE_EXACT, it must be the single strongest hit.
expect(items[0].id).toBe(exact);
});
// #443 Fix #3: titleOnly matches only the title, so it must not leak the page
// body as the snippet (the old "first 300 chars of text_content" fallback).
it('titleOnly does NOT return a text-body snippet', async () => {
const pageId = await insertPage({
title: 'titleonly-snippet-page',
textContent: 'SECRET-BODY-CONTENT-NOT-IN-TITLE that must not leak.',
});
const { items } = (await service.searchPage(
{
query: 'titleonly-snippet-page',
spaceId,
substring: true,
titleOnly: true,
} as any,
{ workspaceId },
)) as any;
const hit = items.find((i: any) => i.id === pageId);
expect(hit).toBeDefined();
// The body text must not appear in the snippet; titleOnly → empty snippet.
expect(hit.snippet).not.toContain('SECRET-BODY-CONTENT-NOT-IN-TITLE');
expect(hit.snippet).toBe('');
});
it('returns [] (not an error) for a query that matches nothing', async () => {
const { items } = (await service.searchPage(
{
query: 'zzz-no-such-string-anywhere-42',
spaceId,
substring: true,
} as any,
{ workspaceId },
)) as any;
expect(items).toEqual([]);
});
it('a `%` query does NOT match everything (LIKE metacharacter escaped)', async () => {
// Fresh space so we can assert on total counts without cross-test noise.
const isoSpace = (await createSpace(db, workspaceId)).id;
const svc = buildService({ userSpaceIds: [isoSpace] });
await insertPage({ title: 'alpha', spaceId: isoSpace });
await insertPage({ title: 'beta', spaceId: isoSpace });
const literal = await insertPage({
title: '100%-coverage',
spaceId: isoSpace,
});
const { items } = (await svc.searchPage(
{ query: '%', spaceId: isoSpace, substring: true, limit: 50 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
// `%` is a literal → matches only the page that actually contains '%'.
expect(ids).toContain(literal);
expect(ids).not.toContain(
items.find((i: any) => i.title === 'alpha')?.id,
);
expect(items.length).toBe(1);
});
it('an `_` query does NOT match everything (LIKE metacharacter escaped)', async () => {
const isoSpace = (await createSpace(db, workspaceId)).id;
const svc = buildService({ userSpaceIds: [isoSpace] });
await insertPage({ title: 'gamma', spaceId: isoSpace });
const literal = await insertPage({
title: 'snake_case_name',
spaceId: isoSpace,
});
const { items } = (await svc.searchPage(
{ query: '_', spaceId: isoSpace, substring: true, limit: 50 } as any,
{ workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(literal);
expect(items.length).toBe(1);
});
it('applies the permission post-filter to the MERGED set BEFORE the limit', async () => {
const isoSpace = (await createSpace(db, workspaceId)).id;
const keep = await insertPage({
title: 'perm-visible-target',
spaceId: isoSpace,
});
const hidden = await insertPage({
title: 'perm-hidden-target',
spaceId: isoSpace,
});
// Authenticated (userId set) so the permission filter runs; only `keep` is
// accessible. limit 1 must NOT be able to select `hidden`.
const svc = buildService({
userSpaceIds: [isoSpace],
accessibleIds: [keep],
});
const { items } = (await svc.searchPage(
{
query: 'perm-',
spaceId: isoSpace,
substring: true,
limit: 1,
} as any,
{ userId: 'user-1', workspaceId },
)) as any;
const ids = items.map((i: any) => i.id);
expect(ids).toContain(keep);
expect(ids).not.toContain(hidden);
});
it('web-UI path (no `substring` flag) keeps the legacy FTS response shape', async () => {
await insertPage({
title: 'legacy shape page',
textContent: 'searchable legacyword content',
});
const { items } = (await service.searchPage(
{ query: 'legacyword', spaceId } as any,
{ userId: 'user-1', workspaceId },
)) as any;
// Legacy hits carry rank + highlight + space, and NO path/snippet/score.
const hit = items[0];
expect(hit).toBeDefined();
expect('rank' in hit).toBe(true);
expect('highlight' in hit).toBe(true);
expect('path' in hit).toBe(false);
expect('snippet' in hit).toBe(false);
expect('score' in hit).toBe(false);
});
});
+128 -9
View File
@@ -811,13 +811,17 @@ export class DocmostClient {
* large instances, so a single bounded page of results is returned (default
* 50, max 100) via the `/pages/recent` feed.
*
* Tree (`tree` true): the space's FULL page hierarchy as a nested tree (each
* node has a `children` array). This mode REQUIRES `spaceId` (a page tree is
* Tree (`tree` true): DEPRECATED prefer `getTree`, which shares this exact
* code path (a single `/pages/tree` request via `enumerateSpacePages` +
* `buildPageTree`) but returns the compact `{pageId, title, children?,
* hasChildren?}` shape and supports `rootPageId`/`maxDepth`. This tree mode is
* kept for backward compatibility; it REQUIRES `spaceId` (a page tree is
* scoped to one space) and IGNORES `limit` the whole hierarchy is returned.
* It fetches the tree via `enumerateSpacePages`, which on the fork server
* resolves to a single `/pages/tree` request returning the whole
* permission-filtered flat page set (soft-deleted pages excluded
* server-side).
* server-side); the cursor-BFS in `enumerateSpacePages` is only a fallback for
* stock upstream servers that lack `/pages/tree`.
*/
async listPages(spaceId?: string, limit: number = 50, tree: boolean = false) {
await this.ensureAuthenticated();
@@ -841,6 +845,107 @@ export class DocmostClient {
return items.map((page: any) => filterPage(page));
}
/**
* Fetch a space's page hierarchy (or one subtree) as a nested tree in a SINGLE
* request the #443 `getTree` tool. Shares its whole code path with
* `listPages(tree:true)`: `enumerateSpacePages` issues one `POST /pages/tree`
* (with the cursor-BFS only as a fallback for stock upstream servers that lack
* the endpoint), then `buildPageTree` nests the flat, permission-filtered,
* position-ordered list. No second tree fetch, no per-node BFS.
*
* - `rootPageId` restrict to that page's subtree; the server seeds the CTE
* with the page itself, so the result is exactly ONE root (the page and its
* descendants). Omit it for the whole space.
* - `maxDepth` trim the response to that many levels (roots = depth 1) to
* save tokens; the server still returns everything in one request, the cut
* is applied in `buildPageTree` AFTER the full tree is built. A node whose
* children were cut carries `hasChildren: true` (source of truth = the flat
* item's server `hasChildren`) so the caller can descend with a follow-up
* `getTree(spaceId, rootPageId=that node)` call.
*
* Output nodes are `{pageId, title, children?, hasChildren?}` only the UUID
* `pageId` is exposed (never `slugId`/`icon`/`position`). Requires `spaceId`
* (a page tree is scoped to one space).
*/
async getTree(spaceId: string, rootPageId?: string, maxDepth?: number) {
await this.ensureAuthenticated();
if (!spaceId) {
throw new Error(
"getTree: spaceId is required (a page tree is scoped to one space).",
);
}
const { pages } = await this.enumerateSpacePages(spaceId, rootPageId);
return buildPageTree(pages, { shape: "getTree", maxDepth });
}
/**
* "Where am I / what's around" for a single page the #443 `getPageContext`
* tool. Metadata only (no page content), using exactly TWO server requests:
*
* 1. `POST /pages/breadcrumbs` a recursive CTE that walks UP from the page.
* The server returns the chain root->page order (it `.reverse()`s the
* child-first walk before responding), INCLUDING the page itself as the
* LAST element. So the last element is the page and everything before it
* is the ancestor chain root->parent. This carries the page's own title
* and spaceId, so no extra page-info fetch is needed for a UUID input.
* 2. `listSidebarPages(spaceId, pageId)` the page's DIRECT children,
* cursor-paginated (a page with >20 children returns ALL of them, no
* dupes) and in sidebar `position` order, each carrying `hasChildren`.
*
* The input may be a slugId (agents copy them from URLs); it is run through
* `resolvePageId` first, exactly like the other page tools. A UUID input adds
* no request there (short-circuit), keeping the total at two; a slugId input
* adds one unavoidable resolve round-trip.
*
* INVARIANT: only the UUID `pageId` is exposed anywhere server `id` is
* mapped to `pageId` and `slugId` is never leaked. A nonexistent/inaccessible
* pageId makes the server 404/403, which propagates as a clear tool error
* (never a hollow empty object).
*/
async getPageContext(pageId: string) {
await this.ensureAuthenticated();
// Resolve a possibly-slugId input to the canonical UUID (no round-trip for a
// UUID). Errors here (bad/inaccessible id) propagate as a clear tool error.
const pageUuid = await this.resolvePageId(pageId);
// Request 1: the ancestor chain, root->page, page included as the LAST item.
const response = await this.client.post("/pages/breadcrumbs", {
pageId: pageUuid,
});
const chain: any[] = (response.data?.data ?? response.data) ?? [];
if (!Array.isArray(chain) || chain.length === 0) {
// The endpoint always includes the page itself, so an empty chain means
// the page is gone/inaccessible — surface a clear error, not {}.
throw new Error(`getPageContext: page "${pageId}" not found or inaccessible`);
}
// Split: the last element is the page, the rest (root->parent) are the
// breadcrumbs. A root page has no ancestors -> breadcrumbs is [].
const self = chain[chain.length - 1];
const ancestors = chain.slice(0, -1);
const page = {
pageId: self.id,
title: self.title,
spaceId: self.spaceId,
};
const breadcrumbs = ancestors.map((n: any) => ({
pageId: n.id,
title: n.title,
}));
// Request 2: direct children in sidebar order, each with hasChildren.
const childItems = await this.listSidebarPages(self.spaceId, pageUuid);
const children = childItems.map((c: any) => ({
pageId: c.id,
title: c.title,
hasChildren: Boolean(c.hasChildren),
}));
return { page, breadcrumbs, children };
}
/**
* List sidebar pages for a space. With no pageId the request returns the
* space ROOT pages; with a pageId it returns the direct CHILDREN of that
@@ -2830,14 +2935,28 @@ export class DocmostClient {
return { success: true, removedShareId: share.shareId, pageId };
}
async search(query: string, spaceId?: string, limit?: number) {
async search(
query: string,
spaceId?: string,
limit?: number,
opts: { parentPageId?: string; titleOnly?: boolean } = {},
) {
await this.ensureAuthenticated();
const payload: Record<string, any> = { query, spaceId };
// Clamp an optional caller-supplied limit into a sane 1..100 range before
// forwarding it to the server; omit it entirely when not provided so the
// server applies its own default.
// Opt into the #443 agent-lookup mode: `substring: true` turns on the hybrid
// substring + FTS branch that returns path + snippet + score. A stock
// upstream server strips these unknown DTO fields (whitelist:true) and
// silently degrades to plain FTS — see the tool-registration comment.
const payload: Record<string, any> = {
query,
spaceId,
substring: true,
};
if (opts.parentPageId) payload.parentPageId = opts.parentPageId;
if (opts.titleOnly) payload.titleOnly = true;
// Clamp an optional caller-supplied limit into the lookup range (1..50)
// before forwarding; omit it when not provided so the server default applies.
if (limit !== undefined) {
payload.limit = Math.max(1, Math.min(100, limit));
payload.limit = Math.max(1, Math.min(50, limit));
}
const response = await this.client.post("/search", payload);
+44 -11
View File
@@ -435,30 +435,63 @@ server.registerTool(
// Tool: search
// INTENTIONAL per-transport divergence (not shared): the in-app `searchPages`
// runs a semantic + keyword hybrid (RRF) with in-process access control and a
// different schema (limit 1-20); this transport is a plain REST full-text search
// (limit up to 100). Different behaviour AND schema, so kept per-layer.
// different schema; this transport is the #443 agent-lookup search — a hybrid
// substring + full-text search that also returns each hit's location (`path`)
// and a windowed `snippet`, so one call answers "where is it and what's in it".
// The in-app hybrid-RRF search is deliberately NOT touched. Different behaviour
// AND schema, so kept per-layer.
//
// STANDALONE-vs-STOCK-UPSTREAM: the client sends the opt-in `substring`/
// `parentPageId`/`titleOnly` DTO fields. A stock upstream server validates the
// DTO with `whitelist: true` and silently strips these unknown fields, so the
// request degrades gracefully to plain FTS (no path/snippet, current shape).
//
// EE/TYPESENSE DEGRADATION (#443): on an instance whose SEARCH_DRIVER is
// `typesense`, the server routes this request to the Typesense backend, which
// does NOT implement agent-lookup — the substring/path/snippet/tiering is
// ignored and the response degrades to plain Typesense FTS. The rich lookup
// shape is only produced by the native Postgres search driver.
server.registerTool(
"search",
{
description:
"Full-text search for pages and content across the whole workspace. " +
"Results are bounded by `limit` (1-100; when omitted the server applies " +
"its own default).",
"Find pages by a fragment of a technical string (hostnames, IPs, IDs " +
"like `srv.local`, `10.0.12`, `WB-MGE-30D86B`) — one call returns each " +
"hit's location (`path`: ancestor titles root→parent) and a `snippet` " +
"around the first match, so you rarely need a follow-up get_page. " +
"Matches substrings literally (dots/dashes/digits are not tokenized) as " +
"well as full-text. Returns `{ pageId, title, path, snippet, score }` " +
"sorted by `score` (a per-response relevance float).",
inputSchema: {
query: z.string().min(1).describe("Search query"),
spaceId: z
.string()
.optional()
.describe("Restrict the search to a single space"),
parentPageId: z
.string()
.optional()
.describe(
"Restrict to a page and all its descendants (the page itself included)",
),
titleOnly: z
.boolean()
.optional()
.describe("Match page titles only; skip page text"),
limit: z
.number()
.int()
.min(1)
.max(100)
.max(50)
.optional()
.describe("Max results to return (max 100)"),
.describe("Max results to return (1-50, default 10)"),
},
},
async ({ query, limit }) => {
// The tool exposes no spaceId filter, so pass undefined for the client's
// optional spaceId parameter and forward limit into its correct slot.
const result = await docmostClient.search(query, undefined, limit);
async ({ query, spaceId, parentPageId, titleOnly, limit }) => {
const result = await docmostClient.search(query, spaceId, limit, {
parentPageId,
titleOnly,
});
return jsonContent(result);
},
);
+24 -8
View File
@@ -83,16 +83,32 @@ export function filterComment(comment: any, markdownContent?: string) {
};
}
// Map one server search hit to the MCP output contract (#443):
// { pageId, title, path, snippet, score }
//
// INVARIANT: the only page identifier exposed is `pageId` (the server `id`
// UUID). The server also carries `slugId` — it is NEVER surfaced.
//
// GRACEFUL DEGRADATION: against a stock upstream server the opt-in lookup DTO
// fields are stripped, so the response is the legacy FTS shape (no path/snippet/
// score, a `highlight` + `rank` instead). We synthesize the contract from
// whatever is present: `snippet` falls back to the FTS `highlight`, `score` to
// the FTS `rank`, and `path` to [] (upstream has no path). This keeps the tool
// usable even when the server has not been upgraded.
export function filterSearchResult(result: any) {
return {
id: result.id,
pageId: result.id,
title: result.title,
parentPageId: result.parentPageId,
createdAt: result.createdAt,
updatedAt: result.updatedAt,
rank: result.rank,
highlight: result.highlight,
spaceId: result.space?.id,
spaceName: result.space?.name,
path: Array.isArray(result.path) ? result.path : [],
snippet:
typeof result.snippet === "string"
? result.snippet
: (result.highlight ?? ""),
score:
typeof result.score === "number"
? result.score
: typeof result.rank === "number"
? result.rank
: 0,
};
}
+85 -8
View File
@@ -1,11 +1,30 @@
/**
* Options for `buildPageTree`. Fully OPTIONAL so the existing call form
* `buildPageTree(nodes)` keeps its historic behaviour (lean `{id, slugId,
* title, children?}` output, no depth cut) unchanged.
*
* - `shape: "getTree"` emit the #443 `getTree` output node shape
* `{pageId, title, children?, hasChildren?}` instead of the lean
* `{id, slugId, title, children?}` shape. `slugId`/`icon`/`position` are
* never exposed (INVARIANT: only the UUID `pageId` leaves the MCP layer).
* - `maxDepth` trim the built tree to this many levels (root nodes are
* depth 1). Only meaningful together with `shape: "getTree"` (the lean shape
* has no `hasChildren` to signal a cut). See the depth logic below.
*/
export interface BuildPageTreeOptions {
shape?: "lean" | "getTree";
maxDepth?: number;
}
/**
* Pure tree-builder: turn a flat array of sidebar-style page nodes (as produced
* by `enumerateSpacePages`) into a nested tree.
*
* Input: a flat array of nodes. Each node is expected to carry at least
* { id, slugId, title, position, parentPageId } (extra fields are ignored).
* { id, slugId, title, position, parentPageId } (extra fields are ignored),
* plus a server `hasChildren` boolean used by the `getTree` shape below.
*
* Output: an array of ROOT nodes, each shaped as
* Output (default / `shape: "lean"`): an array of ROOT nodes, each shaped as
* { id, slugId, title, children? }
* where `children` is the array of child nodes (same shape, recursively). The
* `children` key is OMITTED entirely when a node has no children consistent
@@ -13,6 +32,14 @@
* lean (nesting alone conveys the structure; parentPageId/position/hasChildren
* are intentionally dropped from the output).
*
* Output (`shape: "getTree"`, the #443 tool shape): each node is
* { pageId, title, children?, hasChildren? }
* the server `id` is exposed as `pageId` (never `slugId`/`icon`/`position`).
* `children` is omitted for leaves and for nodes trimmed by `maxDepth`.
* `hasChildren: true` is set ONLY on a node whose children exist on the server
* (per the flat item's `hasChildren`) but were CUT by `maxDepth`; on leaves and
* on fully-expanded interior nodes the field is omitted (see `maxDepth` below).
*
* Linking rule: a node is attached as a child of `parentPageId` only when that
* parent id is actually present in the input. Otherwise including a null /
* undefined `parentPageId`, or a parent that was capped out of the bounded walk
@@ -26,18 +53,42 @@
* fractional-index ASCII keys (e.g. "a0", "a1"). Nodes with a missing/undefined
* `position` sort last.
*
* maxDepth (getTree shape only): the tree is built in FULL first, then trimmed
* on the way out. Root nodes are depth 1. `maxDepth: N` keeps nodes at depth
* <= N and drops the `children` of any node AT depth N. A node whose children
* were dropped this way gets `hasChildren: true` when it actually had children
* in the flat input (source of truth = the server `hasChildren` flag), so the
* caller knows it can descend further with a follow-up `rootPageId` call. An
* absent/undefined `maxDepth` means no cut (whole tree). `maxDepth <= 0` is
* treated as "no cut" (defensive; the tool schema clamps to >= 1).
*
* Pure: no I/O, no network, deterministic.
*/
export function buildPageTree(nodes: any[]): any[] {
type OutputNode = {
export function buildPageTree(
nodes: any[],
options: BuildPageTreeOptions = {},
): any[] {
const getTreeShape = options.shape === "getTree";
// A finite, positive cut only; anything else means "no cut".
const maxDepth =
typeof options.maxDepth === "number" &&
Number.isFinite(options.maxDepth) &&
options.maxDepth > 0
? Math.floor(options.maxDepth)
: undefined;
type InternalNode = {
id: string;
// Retained internally for shaping; never all emitted at once.
slugId: any;
title: any;
children?: OutputNode[];
hasServerChildren: boolean;
children?: InternalNode[];
};
// Map id -> output node. Build the lean output shape up front.
const byId = new Map<string, OutputNode>();
// Map id -> internal node. Build up front; the output shape is projected at
// the very end so the maxDepth cut can consult `hasServerChildren`.
const byId = new Map<string, InternalNode>();
// Preserve the original position string for sorting (kept off the output).
const positionById = new Map<string, string | undefined>();
@@ -49,6 +100,7 @@ export function buildPageTree(nodes: any[]): any[] {
id: node.id,
slugId: node.slugId,
title: node.title,
hasServerChildren: node.hasChildren === true,
});
positionById.set(node.id, node.position);
}
@@ -90,5 +142,30 @@ export function buildPageTree(nodes: any[]): any[] {
}
roots.sort(byPosition);
return roots.map((id) => byId.get(id)!);
const rootNodes = roots.map((id) => byId.get(id)!);
// Project the internal nodes into the requested OUTPUT shape, applying the
// maxDepth cut for the getTree shape. `depth` is 1-based (roots = depth 1).
const project = (node: InternalNode, depth: number): any => {
if (getTreeShape) {
const out: any = { pageId: node.id, title: node.title };
const atCut = maxDepth !== undefined && depth >= maxDepth;
if (!atCut && node.children && node.children.length > 0) {
out.children = node.children.map((c) => project(c, depth + 1));
} else if (atCut && node.hasServerChildren) {
// Children exist on the server but were trimmed by maxDepth: signal it
// so the caller can descend with a follow-up rootPageId call.
out.hasChildren = true;
}
return out;
}
// Lean (historic) shape: cycle-safe, no depth cut, no hasChildren.
const out: any = { id: node.id, slugId: node.slugId, title: node.title };
if (node.children && node.children.length > 0) {
out.children = node.children.map((c) => project(c, depth + 1));
}
return out;
};
return rootNodes.map((n) => project(n, 1));
}
+4 -2
View File
@@ -40,7 +40,7 @@ import { SHARED_TOOL_SPECS, SharedToolSpec } from "./tool-specs.js";
*/
export const ROUTING_PROSE =
"Docmost editing guide — choose the tool by intent. The <tool_inventory> at the end lists every tool with a one-line purpose; the notes below are the routing hints for WHEN to reach for each.\n" +
"READ: find a page -> search (workspace-wide full-text); list -> listPages / listSpaces. Locate blocks and their ids CHEAPLY -> getOutline (compact top-level map; start here, not getPageJson). One block, for editing -> getNode (by attrs.id, or \"#<index>\" for tables, which carry no id) — returns MARKDOWN by default (comment anchors kept for safe write-back); pass format:\"json\" for the raw ProseMirror subtree. Find every occurrence of a string/regex ON a page (and where each is) -> searchInPage, NOT block-by-block getNode — it returns each hit's node ref + block index + context for a targeted comment. Whole page -> getPage (Markdown, canonical for text; drops only block ids, resolved-comment anchors, and a fixed no-md-representation attr set: table spans/colwidth/bg, indent, callout.icon, orderedList.type, link internal/target/rel/class; inline <span data-comment-id> tags are comment anchors — markup, not text) or getPageJson (full ProseMirror with block ids, for those dropped attrs). Hand a huge page (with images) to an external consumer without pulling it through the model context -> stashPage (returns a short-lived anonymous URL).\n" +
"READ: find a page by a fragment of a technical string (hostname/IP/ID like srv.local, 10.0.12, WB-MGE-30D86B) -> search — hybrid substring + full-text, returns each hit's location (path: root->parent titles) and a snippet around the match, so you rarely need a follow-up getPage; scope with spaceId or parentPageId (a subtree), titleOnly to match titles only. A space's page HIERARCHY (or one subtree) -> getTree (one request, complete, `{pageId,title,children?}`; rootPageId for a subtree, maxDepth to trim depth — a trimmed node gets hasChildren:true); prefer it over listPages tree:true (deprecated). Have a pageId, need WHERE-AM-I / what's around it (its breadcrumbs + direct children, metadata only) -> getPageContext (one call; parent = last breadcrumb, [] for a root page). list -> listPages / listSpaces. Locate blocks and their ids CHEAPLY -> getOutline (compact top-level map; start here, not getPageJson). One block, for editing -> getNode (by attrs.id, or \"#<index>\" for tables, which carry no id) — returns MARKDOWN by default (comment anchors kept for safe write-back); pass format:\"json\" for the raw ProseMirror subtree. Find every occurrence of a string/regex ON a page (and where each is) -> searchInPage, NOT block-by-block getNode — it returns each hit's node ref + block index + context for a targeted comment. Whole page -> getPage (Markdown, canonical for text; drops only block ids, resolved-comment anchors, and a fixed no-md-representation attr set: table spans/colwidth/bg, indent, callout.icon, orderedList.type, link internal/target/rel/class; inline <span data-comment-id> tags are comment anchors — markup, not text) or getPageJson (full ProseMirror with block ids, for those dropped attrs). Hand a huge page (with images) to an external consumer without pulling it through the model context -> stashPage (returns a short-lived anonymous URL).\n" +
"EDIT: fix wording/typos/numbers -> editPageText (find/replace inside blocks, no node id needed). Edit a block -> getNode(markdown) -> edit the markdown -> patchNode(markdown) (by attrs.id from getOutline; the markdown fragment may be several blocks — a 1->N section rewrite in one call, the first block keeps the id). Reach for patchNode's `node`-JSON only for fine attr/mark work; a table cell with spans/colors/fixed width -> the table tools (patchNode markdown refuses it). Add a block -> insertNode (markdown, before/after a block by attrs.id or by anchor text, or append; `node` for raw JSON or bare table structure). Remove a block -> deleteNode (by attrs.id). Tables -> tableGet / tableUpdateCell / tableInsertRow / tableDeleteRow (address by \"#<index>\" from getOutline; table nodes have no attrs.id). Images -> insertImage (add from a web URL) / replaceImage (swap an existing image). Draw.io diagrams -> drawioCreate (create from mxGraph XML and insert), drawioGet (read a diagram as mxGraph XML + a hash), drawioUpdate (replace a diagram; pass the hash from drawioGet as baseHash for optimistic locking); before authoring a diagram, drawioShapes (look up verified stencil style-strings so a shape name never renders as an empty box) and drawioGuide (on-demand authoring reference: skeleton/layout/containers/icons-aws/icons-azure), and pass layout:\"elk\" to drawioCreate/drawioUpdate to auto-place nodes. Footnotes -> insertFootnote. Bulk/structural rewrite -> updatePageJson (full ProseMirror replace) or updatePageMarkdown (full plain-Markdown body replace, re-imported — block ids regenerate); prefer the granular tools above to avoid resending the whole ~100KB+ document. Complex/scripted rewrite (multiple coordinated edits, renumbering) -> docmostTransform: write a JS `(doc, ctx) => doc` transform, preview the diff with dryRun (default), then apply with dryRun:false; ctx.helpers includes commentsToFootnotes for turning inline comments into numbered footnotes.\n" +
"PAGES: new -> createPage (Markdown). Rename (title only) -> renamePage. Move -> movePage. Delete -> deletePage (SOFT delete — the page goes to trash and is restorable; nothing is permanent). Copy/replace a page's whole content from another page (server-side, no document through the model) -> copyPageContent. Sharing -> sharePage / unsharePage / listShares; sharePage makes the page PUBLICLY accessible — do it only when explicitly asked.\n" +
"COMMENTS: createComment is always inline and requires an EXACT selection — contiguous text from a single block, <=250 chars (fails rather than leaving an unanchored comment); reply to a thread via parentCommentId. Propose a concrete text fix for one-click human approval -> createComment with suggestedText (the exact plain-text replacement for the selection; the selection must then be UNIQUE in the page — extend it with context if needed); prefer this over editing directly when the change is subjective or needs the author's sign-off. Manage -> listComments, updateComment, resolveComment (resolve/reopen, reversible — prefer over delete to close), deleteComment, checkNewComments.\n" +
@@ -82,6 +82,8 @@ const TOOL_FAMILY: Record<string, Family> = {
// READ
search: "READ",
listPages: "READ",
getTree: "READ",
getPageContext: "READ",
listSpaces: "READ",
getOutline: "READ",
getNode: "READ",
@@ -152,7 +154,7 @@ export const INLINE_MCP_INVENTORY: ToolInventoryLine[] = [
{
name: "search",
purpose:
"full-text search for pages and content across the whole workspace.",
"find pages by a fragment of a technical string (hybrid substring + full-text); returns each hit's path and a snippet.",
},
{
name: "docmostTransform",
+81 -4
View File
@@ -63,6 +63,8 @@ export type DocmostClientLike = Pick<
| 'getSpaces'
| 'listShares'
| 'listPages'
| 'getTree'
| 'getPageContext'
| 'getPage'
| 'getPageJson'
| 'getOutline'
@@ -917,11 +919,13 @@ export const SHARED_TOOL_SPECS = {
description:
'List the most recent pages (ordered by updatedAt, descending), ' +
'optionally scoped to a single space. Returns a bounded list (default ' +
'50, max 100) — use search for lookups in large spaces. Pass tree:true ' +
"(with spaceId) to instead get the space's full page hierarchy as a " +
'nested tree.',
'50, max 100) — use search for lookups in large spaces. tree:true (with ' +
"spaceId) returns the space's full page hierarchy as a nested tree, but " +
'is DEPRECATED — use getTree instead (leaner nodes, plus rootPageId / ' +
'maxDepth).',
tier: 'core',
catalogLine: "listPages — list recent pages, or a space's full page tree.",
catalogLine:
"listPages — list recent pages (tree:true is deprecated; use getTree for the hierarchy).",
buildShape: (z) => ({
spaceId: z
.string()
@@ -954,6 +958,79 @@ export const SHARED_TOOL_SPECS = {
),
},
getTree: {
mcpName: 'getTree',
inAppKey: 'getTree',
description:
"Get a space's page hierarchy (or one subtree) as a nested tree in a " +
'SINGLE request — completely and without loss. Each node is ' +
'`{ pageId, title, children? }`; children are ordered as in the sidebar. ' +
'Pass rootPageId to return only that page and its descendants (exactly ' +
'one root). Pass maxDepth to trim depth and save tokens (root nodes are ' +
'depth 1, so maxDepth:1 returns only the roots); a node whose children ' +
'were trimmed carries `hasChildren:true` so you can descend later with ' +
'getTree(rootPageId=that page). Prefer this over listPages tree:true.',
tier: 'core',
catalogLine:
"getTree — a space's page hierarchy (or a subtree) as a nested tree in one request.",
buildShape: (z) => ({
spaceId: z
.string()
.min(1)
.describe('The id of the space whose page tree to return.'),
rootPageId: z
.string()
.optional()
.describe(
'Optional page id: return only this page and its descendants (one root).',
),
maxDepth: z
.number()
.int()
.min(1)
.optional()
.describe(
'Optional depth cap (roots are depth 1). maxDepth:1 returns only the ' +
'roots; trimmed nodes carry hasChildren:true.',
),
}),
execute: (client, { spaceId, rootPageId, maxDepth }) =>
client.getTree(
spaceId as string,
rootPageId as string | undefined,
maxDepth as number | undefined,
),
},
getPageContext: {
mcpName: 'getPageContext',
inAppKey: 'getPageContext',
description:
'Given a pageId, get its LOCATION and immediate surroundings (metadata ' +
'only, no page content) in one call — answers "where am I / what is ' +
"around this page\". Returns `{ page: { pageId, title, spaceId }, " +
'breadcrumbs: [{ pageId, title }], children: [{ pageId, title, ' +
'hasChildren }] }`. `breadcrumbs` is the ancestor chain from the space ' +
'root down to the PARENT (the parent is its last element; a root page ' +
'has `breadcrumbs: []`). `children` are the direct children in sidebar ' +
'order, each flagged `hasChildren` so you know which can be expanded ' +
'(descend with getTree(rootPageId=that child) or another getPageContext). ' +
'Ids, titles and child order are consistent with getTree.',
tier: 'core',
catalogLine:
'getPageContext — a page’s breadcrumbs + direct children (where-am-I) in one call.',
buildShape: (z) => ({
pageId: z
.string()
.min(1)
.describe(
'The id of the page to locate (a pageId/UUID, or a slugId from a URL).',
),
}),
execute: (client, { pageId }) =>
client.getPageContext(pageId as string),
},
createPage: {
mcpName: 'createPage',
inAppKey: 'createPage',
@@ -0,0 +1,375 @@
// Mock-HTTP tests for DocmostClient.getPageContext — the #443 "where am I /
// what's around" read tool. A local http.createServer stands in for Docmost
// (same harness style as pagination-cursor.test.mjs) so everything is
// deterministic and offline.
//
// Contract pinned here:
// - Two requests: POST /pages/breadcrumbs (ancestor chain root->page, page
// INCLUDED as the LAST element) + listSidebarPages (direct children).
// - Split: last chain element -> `page`; the rest (root->parent) ->
// `breadcrumbs`. A ROOT page (chain length 1) -> breadcrumbs: [].
// - children: {pageId, title, hasChildren} in sidebar order.
// - INVARIANT: only the UUID `pageId` is exposed, never `slugId`.
// - A slugId input is resolved via /pages/info first (adds one request); a
// UUID input short-circuits (stays at two requests).
// - A bad/inaccessible pageId throws a CLEAR error, not an empty object.
import { test, after } from "node:test";
import assert from "node:assert/strict";
import http from "node:http";
import { DocmostClient } from "../../build/client.js";
function readBody(req) {
return new Promise((resolve) => {
let raw = "";
req.on("data", (chunk) => {
raw += chunk;
});
req.on("end", () => resolve(raw));
});
}
function startServer(handler) {
return new Promise((resolve) => {
const server = http.createServer(handler);
server.listen(0, "127.0.0.1", () => {
const { port } = server.address();
resolve({ server, baseURL: `http://127.0.0.1:${port}/api` });
});
});
}
function closeServer(server) {
return new Promise((resolve) => server.close(resolve));
}
function sendJson(res, status, obj, extraHeaders = {}) {
res.writeHead(status, { "Content-Type": "application/json", ...extraHeaders });
res.end(JSON.stringify(obj));
}
const openServers = [];
async function spawn(handler) {
const { server, baseURL } = await startServer(handler);
openServers.push(server);
return { server, baseURL };
}
after(async () => {
await Promise.all(openServers.map((s) => closeServer(s)));
});
function handleLogin(req, res) {
if (req.url === "/api/auth/login") {
sendJson(res, 200, { success: true }, {
"Set-Cookie": "authToken=t; Path=/; HttpOnly",
});
return true;
}
return false;
}
// Two real UUIDs so resolvePageId short-circuits (no /pages/info round-trip).
const ROOT_UUID = "00000000-0000-4000-8000-000000000001";
const MID_UUID = "00000000-0000-4000-8000-000000000002";
const PAGE_UUID = "00000000-0000-4000-8000-000000000003";
const CHILD_A = "00000000-0000-4000-8000-00000000000a";
const CHILD_B = "00000000-0000-4000-8000-00000000000b";
// Build a breadcrumbs response as the server sends it: root->page order, page
// LAST, wrapped in the {data,success} envelope. slugId/icon/position are present
// on the wire (they must NOT leak into the tool output).
function breadcrumbsEnvelope(chain) {
return { success: true, data: chain };
}
// -----------------------------------------------------------------------------
// 1) 3rd-level page: page = last chain element; breadcrumbs = the two ancestors
// root->parent; children mapped {pageId,title,hasChildren} in order; no leak;
// exactly two requests for a UUID input.
// -----------------------------------------------------------------------------
test("getPageContext: 3rd-level page splits chain, maps children, no slugId leak, 2 requests", async () => {
let breadcrumbReqs = 0;
let sidebarReqs = 0;
let infoReqs = 0;
let breadcrumbBody = null;
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/info") {
infoReqs++;
sendJson(res, 404, {});
return;
}
if (req.url === "/api/pages/breadcrumbs") {
breadcrumbReqs++;
breadcrumbBody = JSON.parse(raw || "{}");
// root -> parent -> page (page LAST). slugId/icon/position on the wire.
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", position: "a", icon: null, parentPageId: null, hasChildren: true },
{ id: MID_UUID, slugId: "midSlug", title: "Datacenter A", spaceId: "sp1", position: "a", icon: null, parentPageId: ROOT_UUID, hasChildren: true },
{ id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1", position: "b", icon: null, parentPageId: MID_UUID, hasChildren: true },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
sidebarReqs++;
const body = JSON.parse(raw || "{}");
assert.equal(body.pageId, PAGE_UUID, "children scoped to the page UUID");
assert.equal(body.spaceId, "sp1", "children scoped to the page's space");
sendJson(res, 200, {
success: true,
data: {
items: [
{ id: CHILD_A, slugId: "aSlug", title: "Servers", parentPageId: PAGE_UUID, hasChildren: true, position: "a" },
{ id: CHILD_B, slugId: "bSlug", title: "Network", parentPageId: PAGE_UUID, hasChildren: false, position: "b" },
],
meta: { hasNextPage: false, nextCursor: null },
},
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext(PAGE_UUID);
assert.equal(infoReqs, 0, "UUID input short-circuits resolvePageId (no /pages/info)");
assert.equal(breadcrumbReqs, 1, "exactly one breadcrumbs request");
assert.equal(sidebarReqs, 1, "exactly one sidebar request");
assert.deepEqual(breadcrumbBody, { pageId: PAGE_UUID }, "breadcrumbs posts the UUID");
// page = the LAST chain element.
assert.deepEqual(result.page, {
pageId: PAGE_UUID,
title: "Rack 12",
spaceId: "sp1",
});
// breadcrumbs = root->parent (the chain minus the page itself).
assert.deepEqual(result.breadcrumbs, [
{ pageId: ROOT_UUID, title: "Infrastructure" },
{ pageId: MID_UUID, title: "Datacenter A" },
]);
// children mapped in order, hasChildren coerced to boolean.
assert.deepEqual(result.children, [
{ pageId: CHILD_A, title: "Servers", hasChildren: true },
{ pageId: CHILD_B, title: "Network", hasChildren: false },
]);
// No slugId anywhere in the output.
const dump = JSON.stringify(result);
assert.ok(!dump.includes("Slug"), "no slugId leaks into the output");
assert.ok(!/\bslugId\b/.test(dump), "no slugId key in the output");
});
// -----------------------------------------------------------------------------
// 2) ROOT page: chain has ONE element (the page itself) -> breadcrumbs: [].
// -----------------------------------------------------------------------------
test("getPageContext: a root page has breadcrumbs: []", async () => {
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
// A root page: the CTE returns only the page itself.
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", parentPageId: null, hasChildren: false },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
sendJson(res, 200, {
success: true,
data: { items: [], meta: { hasNextPage: false, nextCursor: null } },
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext(ROOT_UUID);
assert.deepEqual(result.page, {
pageId: ROOT_UUID,
title: "Infrastructure",
spaceId: "sp1",
});
assert.deepEqual(result.breadcrumbs, [], "root page: no ancestors");
assert.deepEqual(result.children, [], "no children");
});
// -----------------------------------------------------------------------------
// 3) A slugId input is resolved via /pages/info first (one extra request), then
// breadcrumbs/sidebar use the resolved UUID.
// -----------------------------------------------------------------------------
test("getPageContext: a slugId input is resolved via /pages/info", async () => {
let infoReqs = 0;
let infoBody = null;
let breadcrumbBody = null;
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/info") {
infoReqs++;
infoBody = JSON.parse(raw || "{}");
// getPageRaw: slugId -> canonical UUID.
sendJson(res, 200, {
success: true,
data: { id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1" },
});
return;
}
if (req.url === "/api/pages/breadcrumbs") {
breadcrumbBody = JSON.parse(raw || "{}");
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: ROOT_UUID, slugId: "rootSlug", title: "Infrastructure", spaceId: "sp1", parentPageId: null },
{ id: PAGE_UUID, slugId: "pageSlug", title: "Rack 12", spaceId: "sp1", parentPageId: ROOT_UUID },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
sendJson(res, 200, {
success: true,
data: { items: [], meta: { hasNextPage: false, nextCursor: null } },
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext("pageSlug");
assert.equal(infoReqs, 1, "slugId resolved via one /pages/info");
assert.deepEqual(infoBody, { pageId: "pageSlug" }, "resolve posts the raw slugId");
assert.deepEqual(
breadcrumbBody,
{ pageId: PAGE_UUID },
"breadcrumbs posts the RESOLVED uuid, not the slugId",
);
assert.equal(result.page.pageId, PAGE_UUID, "page.pageId is the UUID");
assert.deepEqual(result.breadcrumbs, [
{ pageId: ROOT_UUID, title: "Infrastructure" },
]);
});
// -----------------------------------------------------------------------------
// 4) >20 children: cursor pagination returns ALL of them, no dupes (regression
// on the #442 bug class — getPageContext must not re-introduce a cap).
// -----------------------------------------------------------------------------
test("getPageContext: a page with >20 children returns ALL of them (no cap, no dupes)", async () => {
// 45 children spread over three cursor pages.
const all = Array.from({ length: 45 }, (_, i) => ({
id: `child-${i}`,
slugId: `slug-${i}`,
title: `Child ${i}`,
parentPageId: PAGE_UUID,
hasChildren: i % 2 === 0,
}));
const PAGES = {
"": { items: all.slice(0, 20), nextCursor: "c1" },
c1: { items: all.slice(20, 40), nextCursor: "c2" },
c2: { items: all.slice(40), nextCursor: null },
};
const { baseURL } = await spawn(async (req, res) => {
const raw = await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
sendJson(
res,
200,
breadcrumbsEnvelope([
{ id: PAGE_UUID, slugId: "pageSlug", title: "Big Parent", spaceId: "sp1", parentPageId: null },
]),
);
return;
}
if (req.url === "/api/pages/sidebar-pages") {
const body = JSON.parse(raw || "{}");
const page = PAGES[body.cursor ?? ""] ?? { items: [], nextCursor: null };
sendJson(res, 200, {
success: true,
data: {
items: page.items,
meta: { hasNextPage: page.nextCursor != null, nextCursor: page.nextCursor },
},
});
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
const result = await client.getPageContext(PAGE_UUID);
assert.equal(result.children.length, 45, "all 45 children returned");
const ids = result.children.map((c) => c.pageId);
assert.equal(new Set(ids).size, 45, "no duplicate children");
assert.deepEqual(ids, all.map((c) => c.id), "children in server order across cursor pages");
assert.equal(result.children[0].hasChildren, true, "hasChildren preserved (child 0)");
assert.equal(result.children[1].hasChildren, false, "hasChildren preserved (child 1)");
});
// -----------------------------------------------------------------------------
// 5) A nonexistent / inaccessible pageId -> a CLEAR error, NOT an empty object.
// -----------------------------------------------------------------------------
test("getPageContext: a bad/inaccessible pageId throws a clear error (not {})", async () => {
const { baseURL } = await spawn(async (req, res) => {
await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
// Server rejects an unknown/forbidden page.
sendJson(res, 404, { message: "Page not found" });
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects(
() => client.getPageContext(PAGE_UUID),
(err) => {
assert.ok(err instanceof Error, "throws an Error");
return true;
},
"a 404 from breadcrumbs propagates as a thrown error, not a hollow {}",
);
});
// -----------------------------------------------------------------------------
// 6) An empty breadcrumbs chain (should never happen — the endpoint always
// includes the page itself) is treated as not-found, not a hollow {page:...}.
// -----------------------------------------------------------------------------
test("getPageContext: an empty breadcrumbs chain throws (defensive)", async () => {
const { baseURL } = await spawn(async (req, res) => {
await readBody(req);
if (handleLogin(req, res)) return;
if (req.url === "/api/pages/breadcrumbs") {
sendJson(res, 200, breadcrumbsEnvelope([]));
return;
}
sendJson(res, 404, {});
});
const client = new DocmostClient(baseURL, "user@example.com", "pw");
await assert.rejects(
() => client.getPageContext(PAGE_UUID),
/not found or inaccessible/,
"an empty chain is a clear error, not {}",
);
});
+93 -1
View File
@@ -1,7 +1,11 @@
import { test } from "node:test";
import assert from "node:assert/strict";
import { filterComment, filterPage } from "../../build/lib/filters.js";
import {
filterComment,
filterPage,
filterSearchResult,
} from "../../build/lib/filters.js";
test("filterComment includes resolvedAt/resolvedById as null when absent", () => {
const result = filterComment({
@@ -171,3 +175,91 @@ test("filterPage includes both content and subpages together", () => {
assert.equal(result.content, "body");
assert.deepEqual(result.subpages, [{ id: "s1", title: "Sub" }]);
});
// --- filterSearchResult (#443 agent-lookup contract) -------------------------
test("filterSearchResult maps the lookup shape to {pageId,title,path,snippet,score}", () => {
const result = filterSearchResult({
id: "0199aa-uuid",
slugId: "slug-secret",
title: "backup-srv.local",
parentPageId: "0199pp",
path: ["Infrastructure", "Datacenter A", "Servers"],
snippet: "…IP: 10.0.12.5. Debian 12…",
score: 0.92,
});
assert.deepEqual(result, {
pageId: "0199aa-uuid",
title: "backup-srv.local",
path: ["Infrastructure", "Datacenter A", "Servers"],
snippet: "…IP: 10.0.12.5. Debian 12…",
score: 0.92,
});
});
test("filterSearchResult NEVER exposes slugId (pageId is the only identifier)", () => {
const result = filterSearchResult({
id: "uuid-1",
slugId: "slug-1",
title: "t",
path: [],
snippet: "s",
score: 0.1,
});
assert.equal("slugId" in result, false);
assert.equal("id" in result, false);
assert.equal(result.pageId, "uuid-1");
});
test("filterSearchResult root page yields path: []", () => {
const result = filterSearchResult({
id: "uuid-root",
title: "Root",
path: [],
snippet: "s",
score: 0.5,
});
assert.deepEqual(result.path, []);
});
test("filterSearchResult degrades a legacy FTS hit (no lookup fields)", () => {
// Stock upstream stripped the opt-in DTO fields → legacy shape with
// highlight + rank and no path/snippet/score.
const result = filterSearchResult({
id: "uuid-legacy",
slugId: "slug-legacy",
title: "Legacy",
parentPageId: null,
rank: 0.37,
highlight: "…matched <b>text</b>…",
space: { id: "sp1", name: "Space" },
});
assert.equal(result.pageId, "uuid-legacy");
assert.equal(result.title, "Legacy");
// snippet falls back to highlight, score to rank, path to [].
assert.equal(result.snippet, "…matched <b>text</b>…");
assert.equal(result.score, 0.37);
assert.deepEqual(result.path, []);
assert.equal("slugId" in result, false);
});
test("filterSearchResult is null-safe on missing snippet/score/path", () => {
const result = filterSearchResult({ id: "u", title: "t" });
assert.equal(result.pageId, "u");
assert.equal(result.snippet, "");
assert.equal(result.score, 0);
assert.deepEqual(result.path, []);
});
test("filterSearchResult ignores a non-array path", () => {
const result = filterSearchResult({
id: "u",
title: "t",
path: "not-an-array",
snippet: "s",
score: 1,
});
assert.deepEqual(result.path, []);
});
@@ -165,6 +165,70 @@ test("insertNode spec exists, describes markdown+node XOR, builds the full ancho
);
});
// #443: getTree — a space's page hierarchy (or a subtree) in one request.
test("getTree spec exists on both hosts, builds { spaceId, rootPageId?, maxDepth? }", () => {
const spec = SHARED_TOOL_SPECS.getTree;
assert.ok(spec, "getTree spec missing");
assert.equal(spec.mcpName, "getTree");
assert.equal(spec.inAppKey, "getTree");
// Shared spec: registered on BOTH hosts.
assert.notEqual(spec.inAppOnly, true);
assert.notEqual(spec.mcpOnly, true);
const shape = spec.buildShape(z);
assert.deepEqual(Object.keys(shape).sort(), ["maxDepth", "rootPageId", "spaceId"]);
const schema = z.object(shape);
// spaceId required; rootPageId + maxDepth optional.
assert.doesNotThrow(() => schema.parse({ spaceId: "sp1" }));
assert.throws(() => schema.parse({}));
assert.doesNotThrow(() =>
schema.parse({ spaceId: "sp1", rootPageId: "p1", maxDepth: 2 }),
);
// maxDepth is an integer >= 1.
assert.throws(() => schema.parse({ spaceId: "sp1", maxDepth: 0 }));
assert.throws(() => schema.parse({ spaceId: "sp1", maxDepth: 1.5 }));
// The description advertises the output node shape, rootPageId, maxDepth, and
// steers away from the deprecated listPages tree:true.
assert.match(spec.description, /pageId/);
assert.match(spec.description, /rootPageId/);
assert.match(spec.description, /maxDepth/);
assert.match(spec.description, /hasChildren/);
assert.match(spec.description, /listPages tree:true/);
});
// #443: getPageContext — a page's breadcrumbs + direct children in one call.
test("getPageContext spec exists on both hosts, builds { pageId }", () => {
const spec = SHARED_TOOL_SPECS.getPageContext;
assert.ok(spec, "getPageContext spec missing");
assert.equal(spec.mcpName, "getPageContext");
assert.equal(spec.inAppKey, "getPageContext");
// Shared spec: registered on BOTH hosts.
assert.notEqual(spec.inAppOnly, true);
assert.notEqual(spec.mcpOnly, true);
const shape = spec.buildShape(z);
assert.deepEqual(Object.keys(shape).sort(), ["pageId"]);
const schema = z.object(shape);
// pageId required.
assert.doesNotThrow(() => schema.parse({ pageId: "p1" }));
assert.throws(() => schema.parse({}));
// The description advertises the output shape (page/breadcrumbs/children) and
// the root-page empty-breadcrumbs contract.
assert.match(spec.description, /breadcrumbs/);
assert.match(spec.description, /children/);
assert.match(spec.description, /hasChildren/);
assert.match(spec.description, /getTree/);
});
// #443: listPages tree:true is deprecated in favour of getTree.
test("listPages description deprecates tree:true and points at getTree", () => {
const spec = SHARED_TOOL_SPECS.listPages;
assert.match(spec.description, /DEPRECATED/i);
assert.match(spec.description, /getTree/);
});
test("no-arg specs (getWorkspace/listSpaces/listShares) omit buildShape", () => {
for (const key of ["getWorkspace", "listSpaces", "listShares"]) {
assert.equal(SHARED_TOOL_SPECS[key].buildShape, undefined, `${key} should be no-arg`);
+161
View File
@@ -137,3 +137,164 @@ test("buildPageTree output shape is lean (drops position/parentPageId/hasChildre
assert.equal("hasChildren" in node, false);
assert.equal("spaceId" in node, false);
});
// ---------------------------------------------------------------------------
// #443 getTree output shape: { pageId, title, children?, hasChildren? }
// ---------------------------------------------------------------------------
// A small representative space used across the getTree tests:
// r1 (Infrastructure)
// c1 (Datacenter A)
// g1 (Servers) [leaf]
// c2 (Datacenter B) [leaf]
// r2 (Notes) [leaf]
const SAMPLE = [
{ id: "r2", slugId: "s-r2", title: "Notes", position: "a1", icon: "📝", hasChildren: false },
{ id: "r1", slugId: "s-r1", title: "Infrastructure", position: "a0", icon: "🏢", hasChildren: true },
{ id: "c2", slugId: "s-c2", title: "Datacenter B", position: "b1", parentPageId: "r1", icon: "🅱️", hasChildren: false },
{ id: "c1", slugId: "s-c1", title: "Datacenter A", position: "b0", parentPageId: "r1", icon: "🅰️", hasChildren: true },
{ id: "g1", slugId: "s-g1", title: "Servers", position: "c0", parentPageId: "c1", icon: "🖥️", hasChildren: false },
];
test("getTree shape: correct nesting + order-by-position, only {pageId,title,children?}, no leak", () => {
const tree = buildPageTree(SAMPLE, { shape: "getTree" });
// Roots sorted by position: r1 (a0) before r2 (a1).
assert.deepEqual(
tree.map((n) => n.pageId),
["r1", "r2"],
);
// r1's children sorted by position: c1 (b0) before c2 (b1).
assert.deepEqual(
tree[0].children.map((n) => n.pageId),
["c1", "c2"],
);
// Deep nesting: g1 under c1.
assert.deepEqual(
tree[0].children[0].children.map((n) => n.pageId),
["g1"],
);
// No slugId/icon/position/parentPageId/hasChildren leak on any node.
const walk = (nodes) => {
for (const n of nodes) {
assert.deepEqual(
Object.keys(n).sort(),
n.children ? ["children", "pageId", "title"] : ["pageId", "title"],
`unexpected keys on ${n.pageId}: ${Object.keys(n)}`,
);
assert.equal("slugId" in n, false);
assert.equal("icon" in n, false);
assert.equal("position" in n, false);
assert.equal("parentPageId" in n, false);
// Fully-expanded tree (no maxDepth): hasChildren never set.
assert.equal("hasChildren" in n, false);
if (n.children) walk(n.children);
}
};
walk(tree);
});
test("getTree maxDepth:1 returns roots only, each with hasChildren from the flat item", () => {
const tree = buildPageTree(SAMPLE, { shape: "getTree", maxDepth: 1 });
assert.deepEqual(
tree.map((n) => n.pageId),
["r1", "r2"],
);
// No children arrays at depth 1 when maxDepth:1.
for (const n of tree) assert.equal("children" in n, false);
// r1 has children on the server -> hasChildren:true; r2 is a leaf -> omitted.
assert.equal(tree[0].hasChildren, true);
assert.equal("hasChildren" in tree[1], false);
});
test("getTree maxDepth:2 cuts grandchildren; hasChildren only on the cut interior node", () => {
const tree = buildPageTree(SAMPLE, { shape: "getTree", maxDepth: 2 });
const r1 = tree[0];
// Depth-1 node r1 was EXPANDED (its children are present) -> no hasChildren.
assert.equal("hasChildren" in r1, false);
assert.equal(r1.children.length, 2);
const [c1, c2] = r1.children;
// c1 is at depth 2 (the cut) and has children on the server -> hasChildren:true,
// and its grandchild g1 is NOT present.
assert.equal(c1.pageId, "c1");
assert.equal("children" in c1, false);
assert.equal(c1.hasChildren, true);
// c2 is at depth 2 but is a leaf on the server -> hasChildren omitted.
assert.equal(c2.pageId, "c2");
assert.equal("children" in c2, false);
assert.equal("hasChildren" in c2, false);
// r2 is a depth-1 leaf -> no hasChildren, no children.
assert.equal("hasChildren" in tree[1], false);
});
test("getTree hasChildren is set ONLY on depth-cut nodes (not leaves, not expanded interior nodes)", () => {
// Full tree (no cut): NO node anywhere carries hasChildren.
const full = buildPageTree(SAMPLE, { shape: "getTree" });
const anyHasChildren = (nodes) =>
nodes.some((n) => "hasChildren" in n || (n.children && anyHasChildren(n.children)));
assert.equal(anyHasChildren(full), false);
});
test("getTree orphan (parent filtered out) surfaces as a root, not dropped", () => {
const tree = buildPageTree(
[
{ id: "root", slugId: "s-root", title: "Root", position: "a0", hasChildren: true },
// parentPageId points at an id NOT in the flat list (parent filtered by perms).
{ id: "orphan", slugId: "s-orphan", title: "Orphan", position: "a1", parentPageId: "gone", hasChildren: false },
],
{ shape: "getTree" },
);
assert.deepEqual(
tree.map((n) => n.pageId).sort(),
["orphan", "root"],
);
const orphan = tree.find((n) => n.pageId === "orphan");
assert.equal("children" in orphan, false);
assert.equal("hasChildren" in orphan, false);
});
test("getTree rootPageId path: a seeded single-root subtree keeps the getTree shape", () => {
// Simulate the server seeding the CTE with the subtree root c1: the flat list
// it returns contains c1 (now a root, parent absent) + its descendant g1.
const subtree = [
{ id: "c1", slugId: "s-c1", title: "Datacenter A", position: "b0", hasChildren: true },
{ id: "g1", slugId: "s-g1", title: "Servers", position: "c0", parentPageId: "c1", hasChildren: false },
];
const tree = buildPageTree(subtree, { shape: "getTree" });
assert.equal(tree.length, 1);
assert.equal(tree[0].pageId, "c1");
assert.deepEqual(
tree[0].children.map((n) => n.pageId),
["g1"],
);
assert.equal("slugId" in tree[0], false);
});
test("getTree maxDepth<=0 / non-finite is treated as no cut (whole tree)", () => {
for (const bad of [0, -3, NaN, Infinity, undefined]) {
const tree = buildPageTree(SAMPLE, { shape: "getTree", maxDepth: bad });
// Grandchild g1 present -> no cut applied.
assert.deepEqual(
tree[0].children[0].children.map((n) => n.pageId),
["g1"],
`maxDepth=${bad} should not cut`,
);
}
});
test("buildPageTree() with no options is byte-identical to the historic lean call", () => {
// Guard the existing callers: buildPageTree(pages) must be unchanged by the
// additive options param.
const withoutOpts = buildPageTree(SAMPLE);
const withEmptyOpts = buildPageTree(SAMPLE, {});
assert.deepEqual(withoutOpts, withEmptyOpts);
// And it is the lean {id,slugId,title,children?} shape, not the getTree shape.
assert.deepEqual(Object.keys(withoutOpts[0]).sort(), ["children", "id", "slugId", "title"]);
});