fix(ai): raise reindex pre-seed TTL to the client poll cap; cover predicate clause; align docs (F11-F13)

F11: PRE_SEED_TTL_SECONDS 45->120 (= client REINDEX_POLL_CAP_MS). At concurrency
     1 a queued reindex can wait past the old 45s; if the pre-seed expired while
     pending, getMasked fell back to the COUNT and reported done, so the client
     stopped polling and missed the climb. Tie the pre-seed TTL to the client cap.
F12: extend the lockstep integration spec — insertPage takes content; a
     text_content=null + text-node-content page is IN and a math-only page is OUT,
     pinning the structural "type":"text" clause (and the jsonb space-after-colon).
F13: list all three embeddable clauses in the reindex JSDoc/inline comments.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
claude code agent 227
2026-06-29 16:12:36 +03:00
parent 91f24fc062
commit d0eae69086
4 changed files with 99 additions and 37 deletions
@@ -182,7 +182,11 @@ describe('AiSettingsService.reindex progress seed', () => {
);
const ttl = reindexProgress.start.mock.calls[0][2];
expect(ttl).toBeGreaterThan(0);
expect(ttl).toBeLessThanOrEqual(60); // short, not the full 1h record TTL
// Short pre-seed TTL, distinct from the full 1h (3600s) record TTL, but
// pinned to the client poll cap (120s) so a still-pending run can't expire
// into a false "done" while the client is still polling (F11).
expect(ttl).toBe(120);
expect(ttl).toBeLessThanOrEqual(120);
expect(aiQueue.add).toHaveBeenCalledTimes(1);
// Seed must precede the enqueue so the first poll already reports done=0.
expect(order).toEqual(['start', 'add']);
@@ -33,14 +33,27 @@ export function parsePositiveInt(raw: unknown): number | undefined {
/**
* TTL (seconds) for the enqueue-time progress PRE-SEED written by `reindex()`
* before the worker starts. Deliberately SHORT: if `aiQueue.add()` de-duplicates
* against a job that is just finishing (the worker's finally already ran
* `clear()` but removeOnComplete hasn't yet removed the job), no new worker runs
* to overwrite/clear this seed — so a short TTL lets the phantom "reindexing:
* 0 of N" expire in seconds instead of sticking for the full 1h record TTL. A
* worker that DOES start re-seeds with the full TTL, so a real run is unaffected.
* before the worker starts. Deliberately SHORT relative to the full 1h record
* TTL: if `aiQueue.add()` de-duplicates against a job that is just finishing
* (the worker's finally already ran `clear()` but removeOnComplete hasn't yet
* removed the job), no new worker runs to overwrite/clear this seed — so this
* shorter TTL lets the phantom "reindexing: 0 of N" expire instead of sticking
* for the full 1h record TTL. A worker that DOES start re-seeds with the full
* TTL, so a real run is unaffected.
*
* It MUST be >= the client poll cap (REINDEX_POLL_CAP_MS = 120000ms in
* ai-provider-settings.tsx) though: the AI_QUEUE worker runs at concurrency 1
* and shares the queue with page-level embedding jobs, so a queued reindex can
* wait well beyond a few dozen seconds before the worker re-seeds with the full
* TTL. If the pre-seed expired while the job is still pending, `get()` returns
* null and getMasked() falls back to the steady-state COUNT (indexedPages ==
* totalPages, reindexing=false) — the client reads that as "done & fully
* indexed", clears its deadline and STOPS polling, so the admin never sees the
* real climb. Pinning the pre-seed TTL to the client cap means a deduped phantom
* is bounded to ~120s — the same window the client already polls — and a genuine
* pending run never expires-into-"done" inside that window.
*/
const PRE_SEED_TTL_SECONDS = 45;
const PRE_SEED_TTL_SECONDS = 120;
/**
* Shape of the partial update accepted by `update`. Mirrors the validated
@@ -128,10 +141,13 @@ export class AiSettingsService {
let seeded = false;
if ((await this.reindexProgress.get(workspaceId)) === null) {
const totalPages = await this.pageRepo.countEmbeddablePages(workspaceId);
// Short TTL: if add() below de-duplicates against a just-finishing job
// whose worker already clear()ed but isn't removed yet, no worker runs to
// clear this seed — the short TTL expires the phantom record in seconds
// rather than leaving a stuck "reindexing: 0 of N" for the full record TTL.
// Short TTL (vs the full 1h record TTL): if add() below de-duplicates
// against a just-finishing job whose worker already clear()ed but isn't
// removed yet, no worker runs to clear this seed — the shorter TTL expires
// the phantom record rather than leaving a stuck "reindexing: 0 of N" for
// the full record TTL. It is kept >= the client poll cap (120s) so a
// genuine but still-pending run never expires into a false "done" while
// the client is still polling (see PRE_SEED_TTL_SECONDS).
await this.reindexProgress.start(
workspaceId,
totalPages,