fix(ai): raise reindex pre-seed TTL to the client poll cap; cover predicate clause; align docs (F11-F13)
F11: PRE_SEED_TTL_SECONDS 45->120 (= client REINDEX_POLL_CAP_MS). At concurrency
1 a queued reindex can wait past the old 45s; if the pre-seed expired while
pending, getMasked fell back to the COUNT and reported done, so the client
stopped polling and missed the climb. Tie the pre-seed TTL to the client cap.
F12: extend the lockstep integration spec — insertPage takes content; a
text_content=null + text-node-content page is IN and a math-only page is OUT,
pinning the structural "type":"text" clause (and the jsonb space-after-colon).
F13: list all three embeddable clauses in the reindex JSDoc/inline comments.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -182,7 +182,11 @@ describe('AiSettingsService.reindex progress seed', () => {
|
||||
);
|
||||
const ttl = reindexProgress.start.mock.calls[0][2];
|
||||
expect(ttl).toBeGreaterThan(0);
|
||||
expect(ttl).toBeLessThanOrEqual(60); // short, not the full 1h record TTL
|
||||
// Short pre-seed TTL, distinct from the full 1h (3600s) record TTL, but
|
||||
// pinned to the client poll cap (120s) so a still-pending run can't expire
|
||||
// into a false "done" while the client is still polling (F11).
|
||||
expect(ttl).toBe(120);
|
||||
expect(ttl).toBeLessThanOrEqual(120);
|
||||
expect(aiQueue.add).toHaveBeenCalledTimes(1);
|
||||
// Seed must precede the enqueue so the first poll already reports done=0.
|
||||
expect(order).toEqual(['start', 'add']);
|
||||
|
||||
@@ -33,14 +33,27 @@ export function parsePositiveInt(raw: unknown): number | undefined {
|
||||
|
||||
/**
|
||||
* TTL (seconds) for the enqueue-time progress PRE-SEED written by `reindex()`
|
||||
* before the worker starts. Deliberately SHORT: if `aiQueue.add()` de-duplicates
|
||||
* against a job that is just finishing (the worker's finally already ran
|
||||
* `clear()` but removeOnComplete hasn't yet removed the job), no new worker runs
|
||||
* to overwrite/clear this seed — so a short TTL lets the phantom "reindexing:
|
||||
* 0 of N" expire in seconds instead of sticking for the full 1h record TTL. A
|
||||
* worker that DOES start re-seeds with the full TTL, so a real run is unaffected.
|
||||
* before the worker starts. Deliberately SHORT relative to the full 1h record
|
||||
* TTL: if `aiQueue.add()` de-duplicates against a job that is just finishing
|
||||
* (the worker's finally already ran `clear()` but removeOnComplete hasn't yet
|
||||
* removed the job), no new worker runs to overwrite/clear this seed — so this
|
||||
* shorter TTL lets the phantom "reindexing: 0 of N" expire instead of sticking
|
||||
* for the full 1h record TTL. A worker that DOES start re-seeds with the full
|
||||
* TTL, so a real run is unaffected.
|
||||
*
|
||||
* It MUST be >= the client poll cap (REINDEX_POLL_CAP_MS = 120000ms in
|
||||
* ai-provider-settings.tsx) though: the AI_QUEUE worker runs at concurrency 1
|
||||
* and shares the queue with page-level embedding jobs, so a queued reindex can
|
||||
* wait well beyond a few dozen seconds before the worker re-seeds with the full
|
||||
* TTL. If the pre-seed expired while the job is still pending, `get()` returns
|
||||
* null and getMasked() falls back to the steady-state COUNT (indexedPages ==
|
||||
* totalPages, reindexing=false) — the client reads that as "done & fully
|
||||
* indexed", clears its deadline and STOPS polling, so the admin never sees the
|
||||
* real climb. Pinning the pre-seed TTL to the client cap means a deduped phantom
|
||||
* is bounded to ~120s — the same window the client already polls — and a genuine
|
||||
* pending run never expires-into-"done" inside that window.
|
||||
*/
|
||||
const PRE_SEED_TTL_SECONDS = 45;
|
||||
const PRE_SEED_TTL_SECONDS = 120;
|
||||
|
||||
/**
|
||||
* Shape of the partial update accepted by `update`. Mirrors the validated
|
||||
@@ -128,10 +141,13 @@ export class AiSettingsService {
|
||||
let seeded = false;
|
||||
if ((await this.reindexProgress.get(workspaceId)) === null) {
|
||||
const totalPages = await this.pageRepo.countEmbeddablePages(workspaceId);
|
||||
// Short TTL: if add() below de-duplicates against a just-finishing job
|
||||
// whose worker already clear()ed but isn't removed yet, no worker runs to
|
||||
// clear this seed — the short TTL expires the phantom record in seconds
|
||||
// rather than leaving a stuck "reindexing: 0 of N" for the full record TTL.
|
||||
// Short TTL (vs the full 1h record TTL): if add() below de-duplicates
|
||||
// against a just-finishing job whose worker already clear()ed but isn't
|
||||
// removed yet, no worker runs to clear this seed — the shorter TTL expires
|
||||
// the phantom record rather than leaving a stuck "reindexing: 0 of N" for
|
||||
// the full record TTL. It is kept >= the client poll cap (120s) so a
|
||||
// genuine but still-pending run never expires into a false "done" while
|
||||
// the client is still polling (see PRE_SEED_TTL_SECONDS).
|
||||
await this.reindexProgress.start(
|
||||
workspaceId,
|
||||
totalPages,
|
||||
|
||||
Reference in New Issue
Block a user