Стабилизация: статика без флаки + устранение зависаний (стадии 1–7) #19
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: vvzvlad/pyrogram-bridge#19
Reference in New Issue
Block a user
Delete Branch "fix/stability"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Собранный результат всех семи стадий плана стабилизации (plans/2026-07-05-stability-fix-plan.md), закрывает #1 #2 #3 #4 #5 #6 #7.
Что внутри
tg_rpc_bounded: гейт снаружи,asyncio.timeoutвнутри), устойчивый фоновый воркер (task_doneтолько после успешногоget(),put_nowait, FloodWait-backoff), супервизия фоновых задач..part.{hex}+os.rename(инвариант «финальное имя = полный файл»), масштабируемый по размеру таймаут, in-flight дедуп в detached-таске, FloodWait→429 (+Retry-After), семафор с ожиданием ≤30с (иначе 503), debounced touch mtime дляtemp_*.FileResponse(Range/ETag/206/416 из коробки, без per-64KB threadpool-хопов), чистый ASGI-логгер вместо BaseHTTPMiddleware, executor 32 потока.raw_message, ровно один sanitize на выходную границу (fail-closed: при падении bleach —html.escape, не сырой HTML), батч-сбор media file-id без create_task из потока./media./pingбез TG RPC (по возрасту watchdog-probe), healthcheck в compose переведён на него.Проверено перед мержем стека
После мержа в main
/ping(пример уже в dockercompose.yml).diag_semaphore_wait,diag_download_timing,diag_sanitize_slow,watchdog: heartbeat, отсутствие 404-всплесков на/mediaпри флуде.🤖 Generated with Claude Code
Eliminates the app-wide hangs where one stuck Telegram RPC holds the single tg_rpc gate permit forever (freezing every RSS/HTML request) and where the background download worker dies silently. 1.1 Every RPC under the global gate is now bounded by asyncio.wait_for (config tg_rpc_timeout, env TG_RPC_TIMEOUT, default 60). The wait_for wraps ONLY the RPC body, never the gate acquire, so queue backpressure stays legitimate; the paginated get_chat_history is collected in an inner coroutine so the async-for can be bounded. A timeout propagates out of the gate so its permit is released (no leak). 1.2 The other live RPC paths get the same treatment: _reply_enrichment's get_messages now runs under the gate + timeout; PostParser.get_post is bounded at 30s (and a stray print removed); /health's get_me at 10s. 1.3 background_download_worker: queue.get() moved out of the try so task_done() in finally balances exactly one get (no ValueError that killed the worker); FloodWait is caught BEFORE the generic Exception (sleeps, does not flood); download_new_files uses put_nowait so a full queue no longer blocks the sweeper forever. 1.4 Both background tasks run under a _supervised() wrapper: a crash or an unexpected return is logged CRITICAL and restarted (restarts rate-limited to once per 60s so a hard-failing task can't spin), while CancelledError is propagated to the child for a clean shutdown. Tests (tests/test_stage1_hangs.py): gate timeout releases the permit + a second call succeeds; gate cancel mid-spacing releases the permit; the worker survives Exception and FloodWait with balanced task_done; and _supervised restarts a crashing task (rate-limited) and an unexpected return, and propagates cancellation to its child. 180 passed (174 baseline + 6). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>F1 [WARNING] pytest-asyncio was not declared, so the 6 new async stage-1 tests ERROR on a clean checkout (async def not natively supported) — zero regression protection, masked by a globally-installed plugin. Added pytest-asyncio to requirements.txt + asyncio_mode=auto to tests/pytest.ini. Verified on a fresh venv from requirements alone: the tests collect and run (180 passed). F2 [WARNING] The /raw_json endpoint's get_messages was the one remaining live RPC without a timeout, violating the stage-1 DoD ('every Telegram RPC is bounded'). Wrapped it in asyncio.wait_for(..., 30) mirroring PostParser.get_post. (It is not under the tg_rpc gate, so its blast radius was one request, not the app.) F3 [WARNING] The worker test globally no-op'd asyncio.sleep, so the dedicated except FloodWait branch was indistinguishable from the generic handler — deleting it kept the test green. The sleep stub now records delays and the test asserts the FloodWait backoff of 6 (=min(1+5,900)), distinct from the success path's 2. F5 [low] The tricky 'gate outside, timeout inside' nesting was open-coded at 3 sites (each re-deriving the invariant). Extracted tg_rpc_bounded(timeout) into tg_throttle (using asyncio.timeout()); the 3 sites now use it, so a future call site cannot silently wrap the gate entry and reopen the hang-under-backpressure. F4 [low] Documented TG_RPC_TIMEOUT in the dockercompose.yml environment block next to the other TG_RPC_* knobs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>Fixes flaky static media serving: partial files served as 'ready', truncated stubs living for an hour, and app-wide hangs under a saturated download path. 2.1 _download_atomic(file_id, final_path, timeout): downloads to a unique {final}.part.{hex}, validates size>0, and publishes solely via os.rename (atomic on POSIX); a finally ALWAYS removes the partial (timeout/cancel/ zero-size/race-loser). Every downloader call routes through it, so a file at a FINAL name ({fid} / temp_{fid}) is GUARANTEED complete (grep-proven: the only safe_download_media caller passes a .part. path). Big-video timeout scales with size (min/max/min-speed knobs, documented in dockercompose.yml). The sweeper regex now matches both .part. and legacy .tmp. stubs. 2.2 In-flight dedup registry: the first request for a key runs the download in a DETACHED task sharing a Future; the task's finally sets the Future AND pops the key. Waiters await asyncio.shield(fut) so a client disconnect / waiter timeout cancels only the waiter, never the download — no hung waiters, no stuck key, a failed download frees the key for retry. 2.3 FloodWait->429: handler before except RPCError (FloodWait subclasses it), Retry-After = min(value + rand(1,30), 300); propagates from the detached task through the Future. 2.4 Serving a temp_* file touches its mtime so the 1h sweeper can't delete a video out from under a viewer. 2.5 The HTTP download semaphore acquire is bounded (wait_for 30 -> 503 + Retry-After); the permit is released only if the acquire succeeded. The request-scoped-permit trade-off (a disconnect can transiently exceed the download count) is documented inline for stage-7 prod observation. Tests (tests/test_stage2_static.py, 13): atomic publish/clean on every exit incl. FloodWait-through-the-finally; concurrent big-video serves no partial; dedup runs one download, a cancelled waiter doesn't hang others, a failed download frees the key; FloodWait -> 429; mtime touch; sweeper cleans .part./.tmp./stale temp_ but keeps fresh files. 193 passed (180 baseline + 13). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>Replaces the hand-rolled media streaming with Starlette FileResponse, drops the BaseHTTPMiddleware, and enlarges the default threadpool. 3.1 prepare_file_response now returns FileResponse (handles Range/If-Range/206/ 416/multipart, sets Accept-Ranges/ETag/Last-Modified, reads efficiently — no per-64KB to_thread hop that starved the pool). Kept: the early 404 pre-check, the MIME logic (python-magic + SQLite cache), and every stage-2 behavior — the temp_* mtime touch (now DEBOUNCED, see below), delete_after -> BackgroundTask (passed as FileResponse background=), the media_key MIME cache. Removed the manual Range parsing, file_chunk_generator, and hand-built headers; Content-Disposition is formed by FileResponse from filename= (no double-set). 206 slices are byte-identical to the old code; accepted RFC-7233 deltas documented in the tests. 3.2 RequestLoggingMiddleware rewritten as a pure-ASGI class (wraps only send to observe the status line, never buffers the body, passes non-http scopes through) — the streaming body flows untouched. 3.3 lifespan sets a larger default executor (ThreadPoolExecutor, IO_THREAD_POOL_SIZE default 32) and shuts it down on exit. Review round-1 fixes folded in: the temp_* mtime touch is DEBOUNCED (TEMP_MTIME_REFRESH_INTERVAL=300s) so FileResponse's mtime-derived ETag stays stable across a resume/seek session (an every-serve touch broke If-Range resume); starlette pinned to 0.45.3; the io executor is shut down on lifespan exit; the ASGI logger includes the query string. Tests (tests/test_stage3_fileresponse.py, 18): the Range matrix vs FileResponse with every delta documented; temp_* mtime refreshed when stale AND stable when fresh (ETag identical); delete_after background runs and removes the file; media_key MIME cache hit/miss; the ASGI middleware passes the body and logs. 213 passed (195 baseline + 18). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>Feed generation blocked the loop (deepcopy of hundreds of Messages + per-message bleach x4 + str(message) per post). This moves the CPU work off the loop and cuts it down. 4.1 raw_message is lazy: process_message(include_raw=False) omits str(message) entirely; only /json and debug-HTML compute it. 4.2 (done BEFORE 4.3) _save_media_file_ids no longer touches asyncio/DB — it appends to the per-request PostParser._pending_media_ids, and the caller flushes once via to_thread(upsert_media_file_ids_bulk_sync) (a new executemany fn in file_io.py). Removed _persist_pending_count + the create_task machinery. This had to precede 4.3 or get_running_loop() inside the render thread would raise and silently kill media-id persistence. 4.3 The render pipeline (_create_time_based_media_groups, _create_messages_groups, _trim_messages_groups, _render_messages_groups) is now plain-sync and runs in ONE asyncio.to_thread(_render_pipeline) from both feed generators; deepcopy moved into the thread. The render path is verified free of asyncio primitives. 4.4 Sanitize is now ONE pass per output boundary (removed the 4 internal per-fragment bleach passes): RSS/HTML feeds sanitize once at the whole-feed pass; /html and /json sanitize body+footer once in process_message; the debug <pre> raw dump is html.escape'd. Media embeds in body, reactions in footer, so both are covered by the boundary pass. XSS tests (<script>/onerror=/javascript:) confirm all four outputs are clean. Review round-1: documented that flags are now extracted from the pre-sanitize body (a 4.4 consequence — non-security, legitimate links unaffected); added media-caption XSS tests for the exact fragments whose internal passes were removed (adversarially validated: neutering the sanitizer makes them fail); the media-id flush is now in a finally so a partial render still persists what it collected. 230 passed (214 baseline + 16). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>