[bug][ai-search] "Reindex now" counter does not update (stale until reload) — reindex runs async but the UI reads coverage once and never polls #134

New Issue

Ghost · 2026-06-22T16:29:50+03:00

Ghost commented

2026-06-22 16:29:50 +03:00

Severity: medium (UI/UX) — reindexing works, but the coverage counter does not reflect it until a manual page reload.

Symptom

After a base index exists, clicking "Reindex now" (AI settings → Embeddings) does not change the "Indexed X of Y pages" counter / the unindexed count never drops — it looks like reindex does nothing.

What's actually happening (reproduced + isolated on a fresh stand)

Reindex does work — it just runs as a background BullMQ job that indexes pages one-by-one via the embedding provider, while the UI reads the counter once, immediately, before the job has done anything, and never refreshes.

Reproduction (OpenRouter text-embedding-3-large, fresh DB):

Put the workspace in a state with unindexed embeddable pages (e.g. create content pages while the embedding key is unset so auto-index no-ops, then set the key). State: indexed=44, total=84 → unindexed=40, stable.
POST /api/workspace/ai-settings/reindex:
- HTTP response time: 0.02s, and the returned counts are stale: indexed 44 / total 84 (unindexed 40 — unchanged).

The background job then indexes slowly (one page per embedding call):

+3s  db_indexed_pages=44/84
+12s 45 ... +30s 47 ... +48s 53 ... +60s 63 ... → 84/84 after ~150s

A fresh GET (i.e. a manual page reload) afterwards correctly shows indexed=84 / total=84 → unindexed=0.

So the indexing is correct; only the displayed counter is stale between the click and a manual reload.

Root cause

apps/server/src/integrations/ai/ai-settings.controller.ts:88-90 — the reindex endpoint enqueues the async job and immediately returns getMasked() (which reads the current, pre-job counts). The code comment "Return refreshed masked settings so the client can update the counter" is misleading: the job has not run yet, so the counts are stale.
apps/client/src/features/workspace/queries/ai-settings-query.ts (reindex mutation) — invalidates/refetches the coverage query once on success (job barely started) and does not poll while the reindex job is in progress. The counter therefore stays at the old value until the user manually reloads the settings page.

The effect is masked on small workspaces / fast providers (the job can finish within the request round-trip), which is why it's intermittent.

Suggested fix

While a reindex job is in progress, the client should poll the coverage endpoint until it stabilizes (or the endpoint should expose an "indexing in progress" flag + remaining count, and the client polls until it clears). At minimum, show a "indexing in background…" state so the static counter isn't read as "reindex did nothing".

Found while investigating a user report; reproduced live on develop @ v0.93.0-64-gb60190ff with a fresh DB and an OpenRouter embedding provider.

**Severity:** medium (UI/UX) — reindexing works, but the coverage counter does not reflect it until a manual page reload. ### Symptom After a base index exists, clicking **"Reindex now"** (AI settings → Embeddings) does not change the "Indexed X of Y pages" counter / the unindexed count never drops — it looks like reindex does nothing. ### What's actually happening (reproduced + isolated on a fresh stand) Reindex **does** work — it just runs as a background BullMQ job that indexes pages one-by-one via the embedding provider, while the UI reads the counter **once, immediately**, before the job has done anything, and never refreshes. Reproduction (OpenRouter `text-embedding-3-large`, fresh DB): 1. Put the workspace in a state with unindexed embeddable pages (e.g. create content pages while the embedding key is unset so auto-index no-ops, then set the key). State: `indexed=44, total=84` → **unindexed=40**, stable. 2. `POST /api/workspace/ai-settings/reindex`: - **HTTP response time: 0.02s**, and the returned counts are **stale**: `indexed 44 / total 84` (unindexed 40 — unchanged). 3. The background job then indexes slowly (one page per embedding call): ``` +3s db_indexed_pages=44/84 +12s 45 ... +30s 47 ... +48s 53 ... +60s 63 ... → 84/84 after ~150s ``` 4. A fresh `GET` (i.e. a manual page reload) afterwards correctly shows `indexed=84 / total=84` → **unindexed=0**. So the indexing is correct; only the **displayed counter is stale** between the click and a manual reload. ### Root cause - `apps/server/src/integrations/ai/ai-settings.controller.ts:88-90` — the `reindex` endpoint enqueues the async job and **immediately** returns `getMasked()` (which reads the current, pre-job counts). The code comment "Return refreshed masked settings so the client can update the counter" is misleading: the job has not run yet, so the counts are stale. - `apps/client/src/features/workspace/queries/ai-settings-query.ts` (reindex mutation) — invalidates/refetches the coverage query **once** on success (job barely started) and does **not poll** while the reindex job is in progress. The counter therefore stays at the old value until the user manually reloads the settings page. The effect is masked on small workspaces / fast providers (the job can finish within the request round-trip), which is why it's intermittent. ### Suggested fix While a reindex job is in progress, the client should poll the coverage endpoint until it stabilizes (or the endpoint should expose an "indexing in progress" flag + remaining count, and the client polls until it clears). At minimum, show a "indexing in background…" state so the static counter isn't read as "reindex did nothing". --- _Found while investigating a user report; reproduced live on `develop` @ `v0.93.0-64-gb60190ff` with a fresh DB and an OpenRouter embedding provider._

Ghost referenced a pull request that will close this issue

2026-06-22 20:48:21 +03:00

fix(qa): resolve QA-pass issues #122–#134 #135

Ghost referenced this issue from a commit

2026-06-22 20:48:23 +03:00

fix(qa): resolve QA-pass issues #122–#134

vvzvlad closed this issue

2026-06-22 21:07:19 +03:00

Sign in to join this conversation.