refactor(git-sync): internalize the engine — first-class ESM, no vendoring bridge (#119 review)
Closes the architecture item from the #119 review: drop the "vendored from docmost-sync" framing and the CJS↔ESM `Function('import()')` bridge so the engine is a normal first-class gitmost package. Part 1 — vendoring markers removed (prose only, zero behavior change): reworded "VENDORED into gitmost" / "vendored from docmost-sync" / "Engine LOGIC is byte-identical" / "it's a port" comments across the engine. Behavior-bearing strings are untouched: BOT_AUTHOR_NAME/EMAIL and the `Docmost-Sync-Source:` provenance trailers (changing them would break git authorship + the loop-guard). Part 2 — the package is now ESM (matching the sibling @docmost/mcp): `type: module`, tsconfig Node16, `.js` extensions on relative imports, and a static `import { marked }` replacing the `new Function('return import(...)')` / `loadMarked` hack — the bridge is GONE from the package. The CommonJS NestJS server loads the now-ESM engine via a new `git-sync.loader.ts` that mirrors the existing `docmost-client.loader.ts` mcp loader exactly (Function-indirected dynamic import + cached promise + retry-on-reject). The 4 server consumers (orchestrator/datasource/vault-registry/git-http-backend) call `await loadGitSync()` for value exports; types stay `import type` (erased). The converter-gate spec — which needs the real converter — loads the package's TS source via a jest moduleNameMapper + isolatedModules (documented in that spec); the other git-sync specs mock the loader. Verified: engine builds pure ESM (no Function/require leftover), vitest 614, editor-ext build, server + client tsc, full server jest 1397/0. Live stand smoke-test: server starts clean on the ESM engine (no ERR_REQUIRE_ESM), a real sync cycle runs through the loader, and the basic e2e suite is 12/12 (clone via git-http-backend, push, pull, delete, 3-way merge — all through the new loader). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -1,24 +1,21 @@
|
||||
/**
|
||||
* The client seam. Upstream `pull.ts`/`push.ts` reached into the
|
||||
* REST `DocmostClient` from the `docmost-client` package via `Pick<DocmostClient,
|
||||
* ...>` subsets. That package is NOT vendored here (the gitmost server writes
|
||||
* NATIVELY — through repositories + collab `openDirectConnection`),
|
||||
* so the engine must depend on a narrow STRUCTURAL interface instead.
|
||||
* The client seam. `pull.ts`/`push.ts` depend on a narrow STRUCTURAL interface
|
||||
* rather than any concrete client, because the gitmost server writes NATIVELY —
|
||||
* through repositories + collab `openDirectConnection`.
|
||||
*
|
||||
* `GitSyncClient` is that interface: the native datasource (server side, a later
|
||||
* step) implements it, and the vendored engine only ever uses `Pick<GitSyncClient,
|
||||
* ...>` subsets of it. The signatures below MIRROR exactly the methods the
|
||||
* vendored `pull.ts`/`push.ts` actually call (arg shapes + the fields the engine
|
||||
* reads off each result) — verified against the upstream `DocmostClient`
|
||||
* (packages/docmost-client/src/client.ts) so a real REST client is still
|
||||
* structurally assignable, and so the native adapter has a precise contract.
|
||||
* `GitSyncClient` is that interface: the native datasource (server side)
|
||||
* implements it, and the engine only ever uses `Pick<GitSyncClient, ...>`
|
||||
* subsets of it. The signatures below MIRROR exactly the methods the engine's
|
||||
* `pull.ts`/`push.ts` actually call (arg shapes + the fields the engine reads
|
||||
* off each result), so a REST-style client is still structurally assignable and
|
||||
* the native adapter has a precise contract.
|
||||
*/
|
||||
|
||||
/**
|
||||
* A page node as returned by `listSpaceTree` (the sidebar/tree walk, no body).
|
||||
* The engine layout (`buildVaultLayout`) consumes `PageNode` from `./layout`,
|
||||
* which only requires `id` (+ optional `title`/`slugId`/`parentPageId`); this
|
||||
* lite shape documents the fields the tree walk surfaces. Upstream nodes also
|
||||
* lite shape documents the fields the tree walk surfaces. Real tree nodes also
|
||||
* carry `position`, `icon`, `hasChildren` — kept open via the index signature.
|
||||
*/
|
||||
export interface GitSyncPageNodeLite {
|
||||
@@ -27,7 +24,7 @@ export interface GitSyncPageNodeLite {
|
||||
title?: string;
|
||||
parentPageId?: string | null;
|
||||
hasChildren?: boolean;
|
||||
/** Upstream `listSpaceTree` nodes carry extra fields (position, icon, …). */
|
||||
/** `listSpaceTree` nodes carry extra fields (position, icon, …). */
|
||||
[key: string]: unknown;
|
||||
}
|
||||
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
import { VaultGit } from "./git";
|
||||
import { GitSyncClient } from "./client.types";
|
||||
import { Settings } from "./settings";
|
||||
import { readExisting, computePullActions, applyPullActions } from "./pull";
|
||||
import { runPush } from "./push";
|
||||
import { VaultGit } from "./git.js";
|
||||
import { GitSyncClient } from "./client.types.js";
|
||||
import { Settings } from "./settings.js";
|
||||
import { readExisting, computePullActions, applyPullActions } from "./pull.js";
|
||||
import { runPush } from "./push.js";
|
||||
|
||||
/**
|
||||
* Absolute-path filesystem primitives the cycle needs. Injected (not imported)
|
||||
|
||||
@@ -3,11 +3,10 @@
|
||||
*
|
||||
* IMPORTANT — VAULT-SCOPED: every operation here runs with `cwd = vaultPath`,
|
||||
* which is the vault's OWN git repository (default `data/vault`), SEPARATE from
|
||||
* the docmost-sync source repo. This module MUST NEVER run git against the
|
||||
* source repo. `data/` is gitignored by the source repo, so a nested repo under
|
||||
* `data/vault` is safe. The pull cycle is READ-ONLY toward Docmost; this module
|
||||
* only touches the local vault git, never a git remote (push is deferred, see
|
||||
* SPEC §7).
|
||||
* the gitmost application repo. This module MUST NEVER run git against the
|
||||
* application repo. `data/` is gitignored, so a nested repo under `data/vault`
|
||||
* is safe. The pull cycle is READ-ONLY toward Docmost; this module only touches
|
||||
* the local vault git, never a git remote (push is deferred, see SPEC §7).
|
||||
*
|
||||
* Implementation notes:
|
||||
* - We shell out via `node:child_process` `execFile` (promisified), passing
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
* lives in each file's meta block (pageId / slugId).
|
||||
*/
|
||||
|
||||
import { sanitizeTitle, disambiguate } from "./sanitize";
|
||||
import { sanitizeTitle, disambiguate } from "./sanitize.js";
|
||||
|
||||
/** Flat page node as returned by `listAllSpacePages` (no content). */
|
||||
export interface PageNode {
|
||||
|
||||
@@ -25,30 +25,29 @@
|
||||
* (read-only: listSpaceTree + getPageJson). All git operations run against
|
||||
* the vault repo (`cwd = vaultPath`), never the source repo (see ./git.ts).
|
||||
*
|
||||
* VENDORED into gitmost: the client seam is the native
|
||||
* `GitSyncClient` (`Pick<GitSyncClient, ...>`), not the upstream REST
|
||||
* `DocmostClient`; the upstream CLI `main()` entry point is dropped (the gitmost
|
||||
* server drives the engine in-process). Engine LOGIC is byte-identical.
|
||||
* The client seam is the native `GitSyncClient` (`Pick<GitSyncClient, ...>`);
|
||||
* the gitmost server drives the engine in-process (there is no standalone CLI
|
||||
* entry point).
|
||||
*/
|
||||
import { dirname } from "node:path";
|
||||
import { sep } from "node:path";
|
||||
import { parsePageFile, serializePageFile } from "../lib/page-file";
|
||||
import type { GitSyncClient } from "./client.types";
|
||||
import { buildVaultLayout, type PageNode } from "./layout";
|
||||
import { parsePageFile, serializePageFile } from "../lib/page-file.js";
|
||||
import type { GitSyncClient } from "./client.types.js";
|
||||
import { buildVaultLayout, type PageNode } from "./layout.js";
|
||||
import {
|
||||
VaultGit,
|
||||
BOT_AUTHOR_NAME,
|
||||
BOT_AUTHOR_EMAIL,
|
||||
DEFAULT_BRANCH,
|
||||
} from "./git";
|
||||
} from "./git.js";
|
||||
import {
|
||||
planReconciliation,
|
||||
decideAbsenceDeletions,
|
||||
type LiveEntry,
|
||||
type MovedEntry,
|
||||
type DeletionDecision,
|
||||
} from "./reconcile";
|
||||
import { stabilizePageBody } from "./stabilize";
|
||||
} from "./reconcile.js";
|
||||
import { stabilizePageBody } from "./stabilize.js";
|
||||
|
||||
// Engine-only mirror branch (SPEC §5): the engine writes here, humans never do.
|
||||
const DOCMOST_BRANCH = "docmost";
|
||||
|
||||
@@ -22,21 +22,20 @@
|
||||
* then calls `move_page` / `rename_page` (both for a reparent+retitle), or
|
||||
* records a NO-OP for a cosmetic local-only file-path rename.
|
||||
*
|
||||
* VENDORED into gitmost: the client seam is the native
|
||||
* `GitSyncClient` (`Pick<GitSyncClient, ...>`), not the upstream REST
|
||||
* `DocmostClient`; the upstream CLI `main()` entry point is dropped (the gitmost
|
||||
* server drives the engine in-process). Engine LOGIC is byte-identical.
|
||||
* The client seam is the native `GitSyncClient` (`Pick<GitSyncClient, ...>`);
|
||||
* the gitmost server drives the engine in-process (there is no standalone CLI
|
||||
* entry point).
|
||||
*/
|
||||
import { type DocmostMdMeta } from "../lib/index";
|
||||
import { parsePageFile, serializePageFile } from "../lib/page-file";
|
||||
import type { GitSyncClient } from "./client.types";
|
||||
import type { DiffEntry } from "./git";
|
||||
import { VaultGit, DEFAULT_BRANCH } from "./git";
|
||||
import { bodyHash } from "./loop-guard";
|
||||
import { type Settings } from "./settings";
|
||||
import { type DocmostMdMeta } from "../lib/index.js";
|
||||
import { parsePageFile, serializePageFile } from "../lib/page-file.js";
|
||||
import type { GitSyncClient } from "./client.types.js";
|
||||
import type { DiffEntry } from "./git.js";
|
||||
import { VaultGit, DEFAULT_BRANCH } from "./git.js";
|
||||
import { bodyHash } from "./loop-guard.js";
|
||||
import { type Settings } from "./settings.js";
|
||||
|
||||
// Re-export so callers/tests can import the diff row shape from either module.
|
||||
export type { DiffEntry } from "./git";
|
||||
export type { DiffEntry } from "./git.js";
|
||||
|
||||
/** A page to CREATE in Docmost (new local file, meta has no pageId yet). */
|
||||
export interface CreateAction {
|
||||
|
||||
@@ -1,9 +1,7 @@
|
||||
/**
|
||||
* Pure helpers extracted from the docmost-sync Phase-0 idempotency harness
|
||||
* (`src/roundtrip.ts`). Only the IO-free comparison utilities are vendored —
|
||||
* the CLI scaffold (`--fixture`/`--page`/`--corpus`, `loadSettings`, the
|
||||
* `DocmostClient` live path and `process.exit`) is NOT vendored (the roundtrip
|
||||
* harness moves into the package's tests, not the engine).
|
||||
* Pure, IO-free comparison helpers for the idempotency round-trip checks. The
|
||||
* round-trip harness that drives these lives in the package's tests, not in the
|
||||
* engine.
|
||||
*/
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,16 +1,14 @@
|
||||
/**
|
||||
* Engine settings (ADAPTED for vendoring).
|
||||
* Engine settings.
|
||||
*
|
||||
* Upstream this module also loaded `.env` (`dotenv`) and bound `parseSettings`
|
||||
* to `process.env` via a `loadSettings()` entry point. In gitmost the engine is
|
||||
* driven IN-PROCESS by the NestJS server, which builds the `Settings` object
|
||||
* from `EnvironmentService` — so the engine must NOT reach into
|
||||
* `process.env` here. We therefore vendor ONLY:
|
||||
* The engine is driven IN-PROCESS by the NestJS server, which builds the
|
||||
* `Settings` object from `EnvironmentService` — so this module must NOT reach
|
||||
* into `process.env`. It exposes only:
|
||||
* - the `Settings` type the engine consumes, and
|
||||
* - `parseSettings(env)` as a PURE function (validate a raw env object -> typed
|
||||
* `Settings`), kept for unit tests and for the server to reuse if it wants
|
||||
* to validate an env-shaped object.
|
||||
* The `loadSettings()` / `loadDotenv()` side-effecting entry point is dropped.
|
||||
* There is no `.env`-loading side-effecting entry point.
|
||||
*/
|
||||
import { z } from 'zod';
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ import {
|
||||
markdownToProseMirror,
|
||||
serializeDocmostMarkdownBody,
|
||||
type DocmostMdMeta,
|
||||
} from "../lib/index";
|
||||
} from "../lib/index.js";
|
||||
|
||||
/**
|
||||
* Meta object as `exportPageBody` builds it (SPEC §4). Kept byte-for-byte
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
/**
|
||||
* Public surface of `@docmost/git-sync`.
|
||||
*
|
||||
* Phase A vendors only the PURE converter + pure engine modules
|
||||
* from docmost-sync. Server integration (GitmostDataSource, orchestrator,
|
||||
* VaultGit, pull/push) is added in later steps.
|
||||
* Exposes the pure converter (markdown <-> ProseMirror, file envelope,
|
||||
* canonicalization) and the sync engine (reconcile planner, vault layout,
|
||||
* pull/push, the git wrapper, and the settings parser) that the gitmost server
|
||||
* drives in-process.
|
||||
*/
|
||||
|
||||
// Pure converter (markdown <-> ProseMirror, file envelope, canonicalization).
|
||||
@@ -15,8 +16,8 @@ export {
|
||||
markdownToProseMirror,
|
||||
canonicalizeContent,
|
||||
docsCanonicallyEqual,
|
||||
} from "./lib/index";
|
||||
export type { DocmostMdMeta } from "./lib/index";
|
||||
} from "./lib/index.js";
|
||||
export type { DocmostMdMeta } from "./lib/index.js";
|
||||
|
||||
// Pure engine (no IO): reconcile planner, vault layout, sanitize, stabilize,
|
||||
// loop-guard body hash.
|
||||
@@ -25,7 +26,7 @@ export {
|
||||
decideAbsenceDeletions,
|
||||
MASS_DELETE_MIN_EXISTING,
|
||||
MASS_DELETE_FRACTION,
|
||||
} from "./engine/reconcile";
|
||||
} from "./engine/reconcile.js";
|
||||
export type {
|
||||
LiveEntry,
|
||||
ExistingEntry,
|
||||
@@ -33,23 +34,23 @@ export type {
|
||||
MovedEntry,
|
||||
ReconciliationPlan,
|
||||
DeletionDecision,
|
||||
} from "./engine/reconcile";
|
||||
} from "./engine/reconcile.js";
|
||||
|
||||
export { buildVaultLayout } from "./engine/layout";
|
||||
export type { PageNode, VaultEntry } from "./engine/layout";
|
||||
export { buildVaultLayout } from "./engine/layout.js";
|
||||
export type { PageNode, VaultEntry } from "./engine/layout.js";
|
||||
|
||||
export { sanitizeTitle, disambiguate } from "./engine/sanitize";
|
||||
export { sanitizeTitle, disambiguate } from "./engine/sanitize.js";
|
||||
|
||||
export { stabilizePageFile } from "./engine/stabilize";
|
||||
export type { PageMeta } from "./engine/stabilize";
|
||||
export { stabilizePageFile } from "./engine/stabilize.js";
|
||||
export type { PageMeta } from "./engine/stabilize.js";
|
||||
|
||||
export { bodyHash } from "./engine/loop-guard";
|
||||
export { bodyHash } from "./engine/loop-guard.js";
|
||||
|
||||
// IO engine: the client seam, the VaultGit git wrapper, the
|
||||
// pull (Docmost->FS) + push (FS->Docmost) planners/appliers, and the (pure)
|
||||
// settings parser. The engine consumes the native `GitSyncClient` seam (server
|
||||
// implements it) — the upstream REST `DocmostClient` is NOT vendored.
|
||||
export type { GitSyncClient, GitSyncPageNodeLite } from "./engine/client.types";
|
||||
// settings parser. The engine consumes the native `GitSyncClient` seam (the
|
||||
// server implements it) rather than any REST client.
|
||||
export type { GitSyncClient, GitSyncPageNodeLite } from "./engine/client.types.js";
|
||||
|
||||
export {
|
||||
VaultGit,
|
||||
@@ -58,21 +59,21 @@ export {
|
||||
BOT_AUTHOR_NAME,
|
||||
BOT_AUTHOR_EMAIL,
|
||||
DEFAULT_BRANCH,
|
||||
} from "./engine/git";
|
||||
export type { DiffEntry, MergeResult, CommitOptions } from "./engine/git";
|
||||
} from "./engine/git.js";
|
||||
export type { DiffEntry, MergeResult, CommitOptions } from "./engine/git.js";
|
||||
|
||||
export {
|
||||
readExisting,
|
||||
computePullActions,
|
||||
applyPullActions,
|
||||
} from "./engine/pull";
|
||||
} from "./engine/pull.js";
|
||||
export type {
|
||||
ReadExistingDeps,
|
||||
PullActionsInput,
|
||||
PullActions,
|
||||
ApplyPullActionsDeps,
|
||||
ApplyResult,
|
||||
} from "./engine/pull";
|
||||
} from "./engine/pull.js";
|
||||
|
||||
export {
|
||||
classifyRenameMoves,
|
||||
@@ -86,7 +87,7 @@ export {
|
||||
LOCAL_AUTHOR_NAME,
|
||||
LOCAL_AUTHOR_EMAIL,
|
||||
LOCAL_SOURCE_TRAILER,
|
||||
} from "./engine/push";
|
||||
} from "./engine/push.js";
|
||||
export type {
|
||||
CreateAction,
|
||||
UpdateAction,
|
||||
@@ -106,18 +107,18 @@ export type {
|
||||
PushDeps,
|
||||
PushRunResult,
|
||||
PushParsedArgs,
|
||||
} from "./engine/push";
|
||||
} from "./engine/push.js";
|
||||
|
||||
export { parseSettings, envSchema } from "./engine/settings";
|
||||
export type { Settings } from "./engine/settings";
|
||||
export { parseSettings, envSchema } from "./engine/settings.js";
|
||||
export type { Settings } from "./engine/settings.js";
|
||||
|
||||
export { loadSettingsOrExit } from "./engine/config-errors";
|
||||
export { loadSettingsOrExit } from "./engine/config-errors.js";
|
||||
|
||||
export { runCycle } from "./engine/cycle";
|
||||
export { runCycle } from "./engine/cycle.js";
|
||||
export type {
|
||||
RunCycleDeps,
|
||||
RunCycleResult,
|
||||
CycleFs,
|
||||
} from "./engine/cycle";
|
||||
} from "./engine/cycle.js";
|
||||
|
||||
export { parsePageFile, serializePageFile } from "./lib/page-file";
|
||||
export { parsePageFile, serializePageFile } from "./lib/page-file.js";
|
||||
|
||||
@@ -1,9 +1,7 @@
|
||||
/**
|
||||
* docmost-sync ADDITION (not present in docmost-mcp).
|
||||
*
|
||||
* Semantic canonicalization of ProseMirror/TipTap documents for the Phase-0
|
||||
* round-trip idempotency check (SPEC §11, "Задача №0", option (б): compare a
|
||||
* CANONICALIZED form rather than raw bytes).
|
||||
* Semantic canonicalization of ProseMirror/TipTap documents for the round-trip
|
||||
* idempotency check (SPEC §11, "Задача №0", option (б): compare a CANONICALIZED
|
||||
* form rather than raw bytes).
|
||||
*
|
||||
* `markdownToProseMirror` reconstructs schema DEFAULT attributes (e.g.
|
||||
* `indent: null` where the source omitted it) and regenerates per-block ids on
|
||||
@@ -12,8 +10,7 @@
|
||||
* normalizes a document so that two semantically-equal docs compare deep-equal
|
||||
* regardless of block ids and absent-vs-explicit-default-null attributes.
|
||||
*
|
||||
* This file is intentionally a NEW, self-contained module so it is trivial to
|
||||
* backport into docmost-mcp without touching existing code.
|
||||
* It is a self-contained module with no external dependencies.
|
||||
*/
|
||||
|
||||
/**
|
||||
|
||||
@@ -21,7 +21,7 @@ import { getSchema } from "@tiptap/core";
|
||||
import { Node } from "@tiptap/pm/model";
|
||||
import { ChangeSet, simplifyChanges } from "@tiptap/pm/changeset";
|
||||
import { recreateTransform } from "@fellow/prosemirror-recreate-transform";
|
||||
import { docmostExtensions } from "./docmost-schema";
|
||||
import { docmostExtensions } from "./docmost-schema.js";
|
||||
|
||||
/** A single inserted/deleted change with its containing-block context. */
|
||||
export interface DiffChange {
|
||||
|
||||
@@ -1,28 +1,26 @@
|
||||
/**
|
||||
* Public surface of the vendored pure converter (the `lib/` half of the
|
||||
* docmost-sync `docmost-client` package). This barrel re-exports only the
|
||||
* Public surface of the pure converter (`lib/`). This barrel re-exports the
|
||||
* PURE, IO-free pieces the sync engine needs: the self-contained markdown
|
||||
* (de)serializers, the lossless ProseMirror <-> Markdown converter, the
|
||||
* markdown -> ProseMirror import path, and semantic canonicalization for the
|
||||
* round-trip idempotency check (SPEC §11).
|
||||
*
|
||||
* The REST client, websocket/collab write-path, auth-utils and page-lock from
|
||||
* the upstream package are deliberately NOT vendored (the gitmost server writes
|
||||
* natively).
|
||||
* There is no REST client, websocket/collab write-path, auth-utils or page-lock
|
||||
* here — the gitmost server writes natively.
|
||||
*/
|
||||
export {
|
||||
serializeDocmostMarkdown,
|
||||
parseDocmostMarkdown,
|
||||
serializeDocmostMarkdownBody,
|
||||
} from "./markdown-document";
|
||||
export type { DocmostMdMeta } from "./markdown-document";
|
||||
} from "./markdown-document.js";
|
||||
export type { DocmostMdMeta } from "./markdown-document.js";
|
||||
|
||||
export { convertProseMirrorToMarkdown } from "./markdown-converter";
|
||||
export { convertProseMirrorToMarkdown } from "./markdown-converter.js";
|
||||
|
||||
export { markdownToProseMirror } from "./markdown-to-prosemirror";
|
||||
export { markdownToProseMirror } from "./markdown-to-prosemirror.js";
|
||||
|
||||
export {
|
||||
canonicalizeContent,
|
||||
docsCanonicallyEqual,
|
||||
} from "./canonicalize";
|
||||
export { parsePageFile, serializePageFile } from "./page-file";
|
||||
} from "./canonicalize.js";
|
||||
export { parsePageFile, serializePageFile } from "./page-file.js";
|
||||
|
||||
@@ -135,11 +135,9 @@ export function parseDocmostMarkdown(full: string): {
|
||||
return { meta, body, comments };
|
||||
}
|
||||
|
||||
// --- docmost-sync addition (backport target: docmost-mcp/src/lib/markdown-document.ts) ---
|
||||
|
||||
/**
|
||||
* Serialize a self-contained markdown file with the meta block + body ONLY —
|
||||
* NO trailing `docmost:comments` block. The docmost-sync engine never touches
|
||||
* NO trailing `docmost:comments` block. The sync engine never touches
|
||||
* `/comments` (SPEC §3): the synced file carries just page identity (meta) and
|
||||
* the body, where comment threads survive only as inline `<span
|
||||
* data-comment-id>` anchor marks inside the body.
|
||||
|
||||
@@ -1,67 +1,16 @@
|
||||
/**
|
||||
* Pure markdown -> ProseMirror conversion (extracted from docmost-sync's
|
||||
* `packages/docmost-client/src/lib/collaboration.ts`).
|
||||
* Pure markdown -> ProseMirror conversion.
|
||||
*
|
||||
* Only the PURE converter path is vendored here: `markdownToProseMirror`
|
||||
* (marked -> HTML -> generateJSON) plus the two pre/post processors it needs
|
||||
* (`preprocessCallouts`, `bridgeTaskLists`). The collaboration/websocket
|
||||
* write-path (Hocuspocus, Yjs, `ws`, `withPageLock`, `sanitizeForYjs`) that
|
||||
* lives in the same upstream file is intentionally NOT vendored — the gitmost
|
||||
* server writes page bodies natively through the collab gateway.
|
||||
* The converter path is `markdownToProseMirror` (marked -> HTML ->
|
||||
* generateJSON) plus the two pre/post processors it needs (`preprocessCallouts`,
|
||||
* `bridgeTaskLists`). The gitmost server writes the resulting page bodies
|
||||
* natively through the collab gateway, so no websocket/Yjs write-path lives
|
||||
* here.
|
||||
*/
|
||||
import { generateJSON } from "@tiptap/html";
|
||||
import { JSDOM } from "jsdom";
|
||||
import { docmostExtensions } from "./docmost-schema";
|
||||
|
||||
/**
|
||||
* Structural type for the bits of the `marked` ESM module we use: just the
|
||||
* `marked` named export's `parse` method (markdown -> HTML string).
|
||||
*/
|
||||
interface MarkedModule {
|
||||
marked: { parse(markdown: string): string | Promise<string> };
|
||||
}
|
||||
|
||||
// `marked` is ESM-only. Under this package's CommonJS build TS would otherwise
|
||||
// downlevel a literal `import()` to `require()`, which cannot load an ESM-only
|
||||
// module. Indirect through `Function` so the real dynamic `import()` survives
|
||||
// compilation and loads ESM from CommonJS at runtime in Node (same trick as
|
||||
// apps/server/src/core/ai-chat/tools/docmost-client.loader.ts).
|
||||
const esmImport = new Function(
|
||||
"specifier",
|
||||
"return import(specifier)",
|
||||
) as (specifier: string) => Promise<unknown>;
|
||||
|
||||
// Memoize the in-flight/loaded module so the dynamic import runs at most once.
|
||||
let markedPromise: Promise<MarkedModule> | null = null;
|
||||
|
||||
/**
|
||||
* Lazily load the ESM-only `marked` module (cached).
|
||||
*
|
||||
* In the built CommonJS package (Node, jest with ts-jest) the `esmImport`
|
||||
* Function trick performs a real dynamic `import()` of the ESM module. Under
|
||||
* vitest, however, the transformed module is evaluated without a dynamic-import
|
||||
* callback, so `new Function('return import(...)')` throws "A dynamic import
|
||||
* callback was not specified"; there `require('marked')` succeeds because the
|
||||
* test runner's loader interops ESM. We therefore try the Function import first
|
||||
* and fall back to `require` so BOTH runtimes resolve `marked` transparently.
|
||||
*/
|
||||
async function loadMarked(): Promise<MarkedModule["marked"]> {
|
||||
if (!markedPromise) {
|
||||
markedPromise = (esmImport("marked") as Promise<MarkedModule>)
|
||||
.catch(() => {
|
||||
// Function-trick import is unavailable (e.g. under vitest's evaluator):
|
||||
// fall back to require, which the test runner can interop for ESM.
|
||||
// eslint-disable-next-line @typescript-eslint/no-var-requires
|
||||
return require("marked") as MarkedModule;
|
||||
})
|
||||
.catch((err) => {
|
||||
// Do not cache a rejected import — allow the next call to retry.
|
||||
markedPromise = null;
|
||||
throw err;
|
||||
});
|
||||
}
|
||||
return (await markedPromise).marked;
|
||||
}
|
||||
import { marked } from "marked";
|
||||
import { docmostExtensions } from "./docmost-schema.js";
|
||||
|
||||
// Setup DOM environment for Tiptap HTML parsing in Node.js
|
||||
const dom = new JSDOM("<!DOCTYPE html><html><body></body></html>");
|
||||
@@ -110,8 +59,6 @@ async function preprocessCallouts(markdown: string): Promise<string> {
|
||||
return markdown;
|
||||
}
|
||||
|
||||
const marked = await loadMarked();
|
||||
|
||||
// Recursively transform a slice of lines, converting top-level callouts in
|
||||
// that slice into <div> blocks and rendering their inner content (which may
|
||||
// itself contain nested callouts) through this same function.
|
||||
@@ -379,7 +326,6 @@ function stripEmptyParagraphs(node: any): any {
|
||||
export async function markdownToProseMirror(
|
||||
markdownContent: string,
|
||||
): Promise<any> {
|
||||
const marked = await loadMarked();
|
||||
const withCallouts = await preprocessCallouts(markdownContent);
|
||||
const html = await marked.parse(withCallouts);
|
||||
const bridged = bridgeTaskLists(html);
|
||||
|
||||
Reference in New Issue
Block a user