Compare commits
6 Commits
feat/266-s
...
fix/283-sl
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0f4b03d89f | ||
|
|
d70b80c449 | ||
|
|
5f02b7c80e | ||
| 2524f39a36 | |||
|
|
ef173f022d | ||
| 38f9a7938a |
@@ -197,6 +197,12 @@ pnpm workspace (`pnpm@10.4.0`) orchestrated by **Nx**. Four workspace packages:
|
||||
|
||||
Run from the repo root unless noted. The dev workflow needs **Postgres (with the `pgvector` extension) and Redis** reachable per `.env` (copy `.env.example` → `.env`).
|
||||
|
||||
> **Bringing up a full local stand** (API + client + the separate realtime
|
||||
> collaboration process) has several non-obvious gotchas — a missing collab
|
||||
> server, `APP_SECRET` mismatch between processes, a stale `editor-ext` white-
|
||||
> screening the client, LAN exposure. See **[docs/dev-stand.md](docs/dev-stand.md)**
|
||||
> for the step-by-step and the traps.
|
||||
|
||||
```bash
|
||||
pnpm install # install all workspaces (uses pnpm patches; see package.json `pnpm.patchedDependencies`)
|
||||
pnpm dev # client (Vite) + server (Nest watch) concurrently — primary dev loop
|
||||
@@ -241,6 +247,8 @@ Migration files live in `apps/server/src/database/migrations/` and are named `YY
|
||||
- **API server** — `dist/main` (`apps/server/src/main.ts`), the Fastify HTTP app (`AppModule`).
|
||||
- **Collaboration server** — `dist/collaboration/server/collab-main` (`pnpm collab`), a Hocuspocus/Yjs WebSocket server (`apps/server/src/collaboration/`) handling real-time document editing, persistence, and page-history snapshots. It listens on `COLLAB_PORT` (default `3001`), separate from the API server's `PORT` (default `3000`), and shares state with the API server through Redis.
|
||||
|
||||
`pnpm dev` starts **only** the API server + client — the collaboration process is separate and must be started too, or the editor never connects. See **[docs/dev-stand.md](docs/dev-stand.md)** for running both locally (and why `APP_SECRET` must match between them).
|
||||
|
||||
The API server is a Fastify app with a global `/api` prefix (`main.ts` excludes `robots.txt`, public share pages, and `mcp` from the prefix). A `preHandler` hook enforces that a resolved `workspaceId` exists for most `/api` routes (multi-tenant by hostname/subdomain via `DomainMiddleware`). `GET /api/sb/:id` (the anonymous blob-sandbox read route) is listed in that preHandler's `excludedPaths`, so it is exempt from workspace resolution and carries no session auth at all (its capability is the unguessable UUID + TTL + TLS) — unlike `/api/files/public/...`, which still resolves a workspace and requires a workspace-bound attachment JWT. Auth is JWT (cookie + bearer); authorization is **CASL** (`core/casl`) — every data access is scoped to the user's abilities.
|
||||
|
||||
### Module structure (server)
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
import { describe, it, expect } from "vitest";
|
||||
import {
|
||||
buildLayoutCandidates,
|
||||
getSuggestionItems,
|
||||
} from "@/features/editor/components/slash-menu/menu-items.ts";
|
||||
|
||||
/**
|
||||
* `buildLayoutCandidates` maps a slash query across physical keyboard layouts
|
||||
* (RU ЙЦУКЕН <-> US QWERTY) so the menu matches Latin item titles/terms even
|
||||
* when typed with the wrong layout active, while keeping the original query so
|
||||
* genuine Cyrillic search terms still match. See bug #283.
|
||||
*/
|
||||
describe("buildLayoutCandidates", () => {
|
||||
it("remaps a RU-layout query to its US-QWERTY equivalent (сщву -> code)", () => {
|
||||
expect(buildLayoutCandidates("сщву")).toContain("code");
|
||||
});
|
||||
|
||||
it("remaps a US-layout query to its RU-ЙЦУКЕН equivalent (cyjcrf -> сноска)", () => {
|
||||
expect(buildLayoutCandidates("cyjcrf")).toContain("сноска");
|
||||
});
|
||||
|
||||
it("always includes the original query", () => {
|
||||
expect(buildLayoutCandidates("сщву")).toContain("сщву");
|
||||
expect(buildLayoutCandidates("cyjcrf")).toContain("cyjcrf");
|
||||
expect(buildLayoutCandidates("сноска")).toContain("сноска");
|
||||
});
|
||||
|
||||
it("leaves a query with no mappable keys as a single-element set", () => {
|
||||
// Digits are on neither layout map, so both remaps are no-ops and de-dup
|
||||
// back to one entry.
|
||||
expect(buildLayoutCandidates("123")).toEqual(["123"]);
|
||||
});
|
||||
});
|
||||
|
||||
/** Helper: flatten grouped suggestion items to a flat list of titles. */
|
||||
const titles = (groups: ReturnType<typeof getSuggestionItems>): string[] =>
|
||||
Object.values(groups).flatMap((items) => items.map((i) => i.title));
|
||||
|
||||
describe("getSuggestionItems layout-aware matching", () => {
|
||||
it("finds Code when 'code' is typed in RU layout (/сщву)", () => {
|
||||
expect(titles(getSuggestionItems({ query: "сщву" }))).toContain("Code");
|
||||
});
|
||||
|
||||
it("still finds Code for the plain /code query", () => {
|
||||
expect(titles(getSuggestionItems({ query: "code" }))).toContain("Code");
|
||||
});
|
||||
|
||||
it("still matches genuine Cyrillic search terms (/сноска -> Footnote)", () => {
|
||||
expect(titles(getSuggestionItems({ query: "сноска" }))).toContain(
|
||||
"Footnote",
|
||||
);
|
||||
});
|
||||
|
||||
it("finds Footnote when 'сноска' is typed in EN layout (/cyjcrf)", () => {
|
||||
expect(titles(getSuggestionItems({ query: "cyjcrf" }))).toContain(
|
||||
"Footnote",
|
||||
);
|
||||
});
|
||||
|
||||
it("does not surface Footnote for a short wrong-layout query (/cy)", () => {
|
||||
// "cy" EN->RU remaps to "сн", a substring of the "сноска" searchTerm, but
|
||||
// the gate blocks it because the remapped candidate is < 3 chars.
|
||||
expect(titles(getSuggestionItems({ query: "cy" }))).not.toContain(
|
||||
"Footnote",
|
||||
);
|
||||
});
|
||||
|
||||
it("does not surface Footnote for a single-char wrong-layout query (/b)", () => {
|
||||
// "b" EN->RU remaps to "и", a substring of the "примечание" searchTerm, but
|
||||
// the gate blocks it because the remapped candidate is < 3 chars.
|
||||
expect(titles(getSuggestionItems({ query: "b" }))).not.toContain(
|
||||
"Footnote",
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -35,6 +35,7 @@ import { PAGE_EMBED_PICKER_EVENT } from "@/features/editor/components/page-embed
|
||||
import {
|
||||
CommandProps,
|
||||
SlashMenuGroupedItemsType,
|
||||
SlashMenuItemType,
|
||||
} from "@/features/editor/components/slash-menu/types";
|
||||
import { uploadImageAction } from "@/features/editor/components/image/upload-image-action.tsx";
|
||||
import { uploadVideoAction } from "@/features/editor/components/video/upload-video-action.tsx";
|
||||
@@ -835,6 +836,49 @@ export function isHtmlEmbedFeatureEnabled(): boolean {
|
||||
}
|
||||
}
|
||||
|
||||
// Russian ЙЦУКЕН -> US QWERTY by physical key position (lowercase; callers
|
||||
// lowercase first). Lets the slash menu match Latin item titles/terms even when
|
||||
// a command is typed with the wrong keyboard layout active (e.g. "/сщву" while
|
||||
// ЙЦУКЕН is on physically types the same keys as "/code").
|
||||
const RU_TO_EN_LAYOUT: Record<string, string> = {
|
||||
й: "q", ц: "w", у: "e", к: "r", е: "t", н: "y", г: "u", ш: "i", щ: "o",
|
||||
з: "p", х: "[", ъ: "]",
|
||||
ф: "a", ы: "s", в: "d", а: "f", п: "g", р: "h", о: "j", л: "k", д: "l",
|
||||
ж: ";", э: "'",
|
||||
я: "z", ч: "x", с: "c", м: "v", и: "b", т: "n", ь: "m", б: ",", ю: ".",
|
||||
ё: "`",
|
||||
};
|
||||
// Inverse map: US QWERTY -> Russian ЙЦУКЕН by physical key position. Handles the
|
||||
// mirror case (e.g. "cyjcrf" typed with EN layout on == "сноска" == Footnote).
|
||||
const EN_TO_RU_LAYOUT: Record<string, string> = Object.fromEntries(
|
||||
Object.entries(RU_TO_EN_LAYOUT).map(([ru, en]) => [en, ru]),
|
||||
);
|
||||
|
||||
function translitByLayout(text: string, map: Record<string, string>): string {
|
||||
let out = "";
|
||||
for (const ch of text) out += map[ch] ?? ch;
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the list of search strings to try for a given query: the original
|
||||
* query first, followed by its RU->EN and EN->RU physical-layout remappings.
|
||||
* Keeping the original first preserves genuine Cyrillic search terms (e.g.
|
||||
* "сноска"/"примечание" for Footnote) and lets callers treat the original
|
||||
* differently from the remapped candidates. De-duplication only collapses the
|
||||
* list to one element when nothing is remappable (e.g. digits/spaces), so a
|
||||
* typical ASCII query still yields multiple candidates.
|
||||
*/
|
||||
export function buildLayoutCandidates(search: string): string[] {
|
||||
return [
|
||||
...new Set([
|
||||
search,
|
||||
translitByLayout(search, RU_TO_EN_LAYOUT),
|
||||
translitByLayout(search, EN_TO_RU_LAYOUT),
|
||||
]),
|
||||
];
|
||||
}
|
||||
|
||||
export const getSuggestionItems = ({
|
||||
query,
|
||||
excludeItems,
|
||||
@@ -843,6 +887,18 @@ export const getSuggestionItems = ({
|
||||
excludeItems?: Set<string>;
|
||||
}): SlashMenuGroupedItemsType => {
|
||||
const search = query.toLowerCase();
|
||||
const candidates = buildLayoutCandidates(search);
|
||||
// Only the original query is allowed to match via a short substring. Remapped
|
||||
// (wrong-layout) candidates must be at least REMAP_MIN_LEN chars and differ
|
||||
// from the original before they can match, so a 1-2 char ASCII query does not
|
||||
// spuriously substring-match unrelated Cyrillic search terms (e.g. "/cy" ->
|
||||
// "сн" hitting "сноска", "/b" -> "и" hitting "примечание").
|
||||
const REMAP_MIN_LEN = 3;
|
||||
const [originalCandidate, ...remapped] = candidates;
|
||||
const remappedCandidates = remapped.filter(
|
||||
(candidate) =>
|
||||
candidate.length >= REMAP_MIN_LEN && candidate !== originalCandidate,
|
||||
);
|
||||
const filteredGroups: SlashMenuGroupedItemsType = {};
|
||||
const htmlEmbedFeatureEnabled = isHtmlEmbedFeatureEnabled();
|
||||
|
||||
@@ -856,24 +912,42 @@ export const getSuggestionItems = ({
|
||||
return false;
|
||||
};
|
||||
|
||||
const candidateMatchesItem = (
|
||||
candidate: string,
|
||||
item: SlashMenuItemType,
|
||||
description: string,
|
||||
) =>
|
||||
fuzzyMatch(candidate, item.title) ||
|
||||
description.includes(candidate) ||
|
||||
(item.searchTerms != null &&
|
||||
item.searchTerms.some((term: string) => term.includes(candidate)));
|
||||
|
||||
for (const [group, items] of Object.entries(CommandGroups)) {
|
||||
const filteredItems = items.filter((item) => {
|
||||
if (excludeItems?.has(item.title)) return false;
|
||||
// Hide the HTML embed item unless the workspace master toggle is ON.
|
||||
if (item.requiresHtmlEmbedFeature && !htmlEmbedFeatureEnabled)
|
||||
return false;
|
||||
const description = item.description.toLowerCase();
|
||||
return (
|
||||
fuzzyMatch(search, item.title) ||
|
||||
item.description.toLowerCase().includes(search) ||
|
||||
(item.searchTerms &&
|
||||
item.searchTerms.some((term: string) => term.includes(search)))
|
||||
candidateMatchesItem(originalCandidate, item, description) ||
|
||||
remappedCandidates.some((candidate) =>
|
||||
candidateMatchesItem(candidate, item, description),
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
if (filteredItems.length) {
|
||||
const titleMatchesAnyCandidate = (title: string) => {
|
||||
const lower = title.toLowerCase();
|
||||
return (
|
||||
lower.includes(originalCandidate) ||
|
||||
remappedCandidates.some((candidate) => lower.includes(candidate))
|
||||
);
|
||||
};
|
||||
filteredGroups[group] = filteredItems.sort((a, b) => {
|
||||
const aTitle = a.title.toLowerCase().includes(search) ? 0 : 1;
|
||||
const bTitle = b.title.toLowerCase().includes(search) ? 0 : 1;
|
||||
const aTitle = titleMatchesAnyCandidate(a.title) ? 0 : 1;
|
||||
const bTitle = titleMatchesAnyCandidate(b.title) ? 0 : 1;
|
||||
return aTitle - bTitle;
|
||||
});
|
||||
}
|
||||
|
||||
135
docs/dev-stand.md
Normal file
135
docs/dev-stand.md
Normal file
@@ -0,0 +1,135 @@
|
||||
# Running a local dev stand
|
||||
|
||||
How to bring up a working local instance (API + client + realtime collaboration)
|
||||
and the non-obvious gotchas that will otherwise eat an hour. Written from real
|
||||
setup pain — read the **Gotchas** section before you start.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Node 20+ / pnpm 10+.**
|
||||
- **Postgres with pgvector.** Use the `pgvector/pgvector` image (e.g.
|
||||
`pgvector/pgvector:pg18`). The stock `postgres` image will FAIL the
|
||||
`CREATE EXTENSION vector` migration — the RAG feature stores embeddings in
|
||||
`page_embeddings`.
|
||||
- **Redis** — backs caching, BullMQ queues, the Socket.IO adapter, and collab
|
||||
sync.
|
||||
|
||||
## 1. Environment (`.env`)
|
||||
|
||||
The client (`apps/client/vite.config.ts`) and both server processes read env via
|
||||
`envPath` → the **workspace root `.env`**. Keep a single source of truth. Minimum:
|
||||
|
||||
```dotenv
|
||||
APP_URL=http://localhost:3000
|
||||
PORT=3000
|
||||
APP_SECRET=<one long secret — SAME value everywhere, see gotcha #3>
|
||||
DATABASE_URL="postgresql://<user>:<pass>@localhost:5432/<db>?schema=public"
|
||||
REDIS_URL=redis://127.0.0.1:6379
|
||||
COLLAB_URL=http://localhost:3001 # where the CLIENT connects for realtime
|
||||
COLLAB_PORT=3001 # where the COLLAB server listens
|
||||
STORAGE_DRIVER=local
|
||||
DISABLE_TELEMETRY=true
|
||||
```
|
||||
|
||||
> If you also keep an `apps/server/.env`, its `APP_SECRET` **must match** the
|
||||
> root one (see gotcha #3).
|
||||
|
||||
## 2. Migrations
|
||||
|
||||
Migrations do **not** auto-run in local dev. After a fresh checkout or switching
|
||||
branches, apply them yourself or endpoints touching a new column/table will 500:
|
||||
|
||||
```bash
|
||||
pnpm --filter server migration:latest
|
||||
```
|
||||
|
||||
## 3. Bring it up — THREE processes, not two
|
||||
|
||||
`pnpm dev` starts only the **API server** (Nest, `:3000`) and the **client**
|
||||
(Vite). Realtime collaboration is a **separate process** and `pnpm dev` does NOT
|
||||
start it. You need all three:
|
||||
|
||||
```bash
|
||||
# 1) API + client (from the repo root)
|
||||
pnpm dev
|
||||
# → API http://localhost:3000
|
||||
# → client http://localhost:5173 (Vite; localhost-only by default)
|
||||
|
||||
# 2) Collaboration server — SEPARATE process. Build first (see gotcha #2), then:
|
||||
pnpm --filter server build # produces dist/collaboration/server/collab-main.js
|
||||
pnpm collab:dev # node dist/.../collab-main → listens on :3001 (0.0.0.0)
|
||||
```
|
||||
|
||||
Without step 2 the editor shows **"Real-time editor connection lost. Retrying…"**,
|
||||
stays in read-only *static* mode, and anything that only mounts in the *live*
|
||||
editor won't appear.
|
||||
|
||||
## Seeding a login
|
||||
|
||||
Register through the UI, or reset an existing user's password directly in the DB
|
||||
(the server hashes with `bcrypt`):
|
||||
|
||||
```js
|
||||
// node -e '...' with pg + bcrypt from the repo's node_modules
|
||||
const bcrypt = require("bcrypt");
|
||||
const { Client } = require("pg");
|
||||
(async () => {
|
||||
const hash = await bcrypt.hash("demopass", 10);
|
||||
const c = new Client({ /* DATABASE_URL parts */ });
|
||||
await c.connect();
|
||||
await c.query("update users set password=$1 where email=$2", [hash, "admin@example.com"]);
|
||||
await c.end();
|
||||
})();
|
||||
```
|
||||
|
||||
> **Use a simple one-word password with no special characters** (e.g. `demopass`,
|
||||
> not `Str0ng!Pass@2026`). Demo/test credentials get passed through shells, JSON
|
||||
> payloads, and URLs by scripts and automation, where `!` `@` `$` `&` etc. get
|
||||
> mangled or need escaping — a plain alphanumeric word avoids a whole class of
|
||||
> "wrong password" confusion.
|
||||
|
||||
## Gotchas (the грабли)
|
||||
|
||||
1. **Collaboration is a third process.** `pnpm dev` runs API + client only.
|
||||
Start `pnpm collab:dev` (on `:3001`) separately or the live editor never
|
||||
connects. The client connects to `COLLAB_URL` directly (default
|
||||
`http://localhost:3001`), NOT through the Vite `/collab` proxy — the API
|
||||
server on `:3000` does **not** serve the collab websocket.
|
||||
|
||||
2. **The collab server must be built — you can't run it from source.**
|
||||
`collab:dev` runs `node dist/collaboration/server/collab-main.js`, so run
|
||||
`pnpm --filter server build` first. Running the entry via `tsx`/`ts-node`
|
||||
fails with a NestJS DI error ("dependency … appears to be undefined at
|
||||
runtime") because direct TS execution doesn't emit the decorator metadata the
|
||||
built output has.
|
||||
|
||||
3. **`APP_SECRET` must be identical for the API server and the collab server.**
|
||||
The API issues a collab-token (JWT signed with `APP_SECRET`); the collab
|
||||
server validates it with `APP_SECRET`. If they load different values (e.g. a
|
||||
root `.env` and an `apps/server/.env` with different secrets), every realtime
|
||||
connection is rejected with **`[onAuthenticate] Invalid collab token`** and
|
||||
the editor shows "connection lost". Keep one secret everywhere.
|
||||
|
||||
4. **Vite binds localhost only.** To reach the stand from another machine on the
|
||||
LAN, start the client with `--host` (`pnpm --filter client exec vite --host`)
|
||||
and use the box's LAN IP. The `/api`, `/socket.io`, and `/collab` Vite proxies
|
||||
forward to `APP_URL`, so the API just works over the LAN; realtime needs
|
||||
`COLLAB_URL` reachable from the browser (point it at the LAN IP:3001, and run
|
||||
collab on `0.0.0.0` — it does by default).
|
||||
|
||||
5. **A stale `@docmost/editor-ext` white-screens the client.** The client imports
|
||||
from `@docmost/editor-ext` (a workspace package). If that package's source is
|
||||
behind (missing a newer export, e.g. `Spoiler`), the client dies at load with
|
||||
*"The requested module … does not provide an export named 'Spoiler'"* → blank
|
||||
page. Make sure the workspace `packages/editor-ext` is current for the branch
|
||||
you're running (a stale sibling checkout resolved through a shared
|
||||
`node_modules` symlink is the usual cause).
|
||||
|
||||
6. **pgvector, not stock postgres** (see Prerequisites) — the `vector` extension
|
||||
migration fails otherwise.
|
||||
|
||||
7. **Migrations don't auto-run in dev** — run `migration:latest` after every pull
|
||||
or branch switch.
|
||||
|
||||
See also the **Commands** and **Architecture → Two server processes** sections in
|
||||
[`AGENTS.md`](../AGENTS.md).
|
||||
Reference in New Issue
Block a user