a14560c7c9
Long research turns failed mid-task with "Lost connection to the AI provider".
Node's global fetch (undici) defaults BOTH headersTimeout and bodyTimeout to
300_000ms, and the chat provider + the external-MCP dispatcher both ran on it
with no override, so:
- the z.ai chat stream dropped when a late step's huge accumulated context
pushed the model's time-to-first-token past 5 min (the model reasons
server-side with NO streamed reasoning, so the connection is silent until the
first answer token — reproduced: even a trivial glm-5.2 query has a ~4-8s
first-chunk gap; a long run reaches 400k+-token steps), or a reasoning model
paused >5 min between chunks (bodyTimeout);
- the crawl4ai SSE transport, held open across the whole turn, dropped when it
idled >5 min between tool calls.
Fix: a dedicated undici dispatcher whose stream timeouts are raised to a
generous-but-FINITE silence timeout (default 15 min, AI_STREAM_TIMEOUT_MS) on
each path. NOT disabled (0): that would let a genuinely hung provider — with the
client still connected — hang forever, since the turn's abortSignal only fires on
client disconnect. The timeout bounds SILENCE (time-to-first-byte and the gap
BETWEEN chunks), NOT total turn duration, so an arbitrarily long turn that keeps
streaming is never cut; only a stream quiet for >15 min is treated as a hang.
- ai-streaming-fetch.ts: createStreamingFetch() + streamTimeoutMs() /
streamingDispatcherOptions() (the shared, configurable timeout).
- ai.service: the chat provider fetch is createStreamingFetch(), wrapped by the
existing passive ECONNRESET telemetry (createDiagnosticFetch gained an
optional baseFetch) so the telemetry observes the SAME transport.
- mcp-clients: the SSRF-pinned Agent uses streamingDispatcherOptions().
Investigation: reproduced the transport mechanism against the real z.ai endpoint
(a 1ms headersTimeout throws UND_ERR_HEADERS_TIMEOUT — the exact drop) and ran
the actual research agent to a ~428k-token context. Verified the fixed path
streams cleanly live (glm-5.2 turns finish; telemetry confirms the streaming
fetch is in use).
Tests: ai-streaming-fetch.spec (default 15m + env override + invalid fallback +
both-timeouts + streams a delayed response); ai-http-diagnostics + ai/mcp specs
green. server tsc clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
79 lines
2.6 KiB
TypeScript
79 lines
2.6 KiB
TypeScript
import * as http from 'node:http';
|
|
import {
|
|
createStreamingFetch,
|
|
streamTimeoutMs,
|
|
streamingDispatcherOptions,
|
|
} from './ai-streaming-fetch';
|
|
|
|
/**
|
|
* #175: undici's default 300s headers/body timeouts severed long agent turns.
|
|
* The streaming fetch raises them to a generous-but-FINITE silence timeout (not
|
|
* 0 — a true hang must still break). We pin: the configured value + env override,
|
|
* that both dispatcher timeouts use it, and that a delayed response streams.
|
|
*/
|
|
describe('streamTimeoutMs', () => {
|
|
const ORIG = process.env.AI_STREAM_TIMEOUT_MS;
|
|
afterEach(() => {
|
|
if (ORIG === undefined) delete process.env.AI_STREAM_TIMEOUT_MS;
|
|
else process.env.AI_STREAM_TIMEOUT_MS = ORIG;
|
|
});
|
|
|
|
it('defaults to a generous-but-finite 15 minutes', () => {
|
|
delete process.env.AI_STREAM_TIMEOUT_MS;
|
|
expect(streamTimeoutMs()).toBe(900_000);
|
|
// Finite — NOT disabled (0 would let a hung provider leak forever).
|
|
expect(streamTimeoutMs()).toBeGreaterThan(0);
|
|
expect(Number.isFinite(streamTimeoutMs())).toBe(true);
|
|
});
|
|
|
|
it('honours a positive AI_STREAM_TIMEOUT_MS override', () => {
|
|
process.env.AI_STREAM_TIMEOUT_MS = '120000';
|
|
expect(streamTimeoutMs()).toBe(120000);
|
|
});
|
|
|
|
it('ignores an invalid / non-positive override (falls back to default)', () => {
|
|
for (const bad of ['0', '-5', 'abc', '']) {
|
|
process.env.AI_STREAM_TIMEOUT_MS = bad;
|
|
expect(streamTimeoutMs()).toBe(900_000);
|
|
}
|
|
});
|
|
|
|
it('applies the timeout to BOTH undici stream timeouts', () => {
|
|
delete process.env.AI_STREAM_TIMEOUT_MS;
|
|
expect(streamingDispatcherOptions()).toEqual({
|
|
headersTimeout: 900_000,
|
|
bodyTimeout: 900_000,
|
|
});
|
|
});
|
|
});
|
|
|
|
describe('createStreamingFetch — against a delayed server', () => {
|
|
let server: http.Server;
|
|
let url: string;
|
|
// The server waits before sending ANY byte (a long time-to-first-token).
|
|
const DELAY = 400;
|
|
|
|
beforeAll(async () => {
|
|
server = http.createServer((_req, res) => {
|
|
setTimeout(() => {
|
|
res.writeHead(200, { 'Content-Type': 'text/plain' });
|
|
res.end('ok');
|
|
}, DELAY);
|
|
});
|
|
await new Promise<void>((resolve) => server.listen(0, '127.0.0.1', resolve));
|
|
const addr = server.address() as import('node:net').AddressInfo;
|
|
url = `http://127.0.0.1:${addr.port}/`;
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await new Promise<void>((resolve) => server.close(() => resolve()));
|
|
});
|
|
|
|
it('streams the delayed response instead of timing out', async () => {
|
|
const streamingFetch = createStreamingFetch();
|
|
const res = await streamingFetch(url);
|
|
expect(res.status).toBe(200);
|
|
expect(await res.text()).toBe('ok');
|
|
});
|
|
});
|