vvzvlad/pyrogram-bridge
1
0
Fork 0
Code Issues 5 Pull Requests Packages Projects Releases Wiki Activity

fix(stability): stage 2 — atomic media downloads, in-flight dedup, FloodWait->429, bounded semaphore

Browse Source 
Fixes flaky static media serving: partial files served as 'ready', truncated
stubs living for an hour, and app-wide hangs under a saturated download path.

2.1 _download_atomic(file_id, final_path, timeout): downloads to a unique
    {final}.part.{hex}, validates size>0, and publishes solely via os.rename
    (atomic on POSIX); a finally ALWAYS removes the partial (timeout/cancel/
    zero-size/race-loser). Every downloader call routes through it, so a file at a
    FINAL name ({fid} / temp_{fid}) is GUARANTEED complete (grep-proven: the only
    safe_download_media caller passes a .part. path). Big-video timeout scales with
    size (min/max/min-speed knobs, documented in dockercompose.yml). The sweeper
    regex now matches both .part. and legacy .tmp. stubs.
2.2 In-flight dedup registry: the first request for a key runs the download in a
    DETACHED task sharing a Future; the task's finally sets the Future AND pops the
    key. Waiters await asyncio.shield(fut) so a client disconnect / waiter timeout
    cancels only the waiter, never the download — no hung waiters, no stuck key, a
    failed download frees the key for retry.
2.3 FloodWait->429: handler before except RPCError (FloodWait subclasses it),
    Retry-After = min(value + rand(1,30), 300); propagates from the detached task
    through the Future.
2.4 Serving a temp_* file touches its mtime so the 1h sweeper can't delete a
    video out from under a viewer.
2.5 The HTTP download semaphore acquire is bounded (wait_for 30 -> 503 +
    Retry-After); the permit is released only if the acquire succeeded. The
    request-scoped-permit trade-off (a disconnect can transiently exceed the
    download count) is documented inline for stage-7 prod observation.

Tests (tests/test_stage2_static.py, 13): atomic publish/clean on every exit incl.
FloodWait-through-the-finally; concurrent big-video serves no partial; dedup runs
one download, a cancelled waiter doesn't hang others, a failed download frees the
key; FloodWait -> 429; mtime touch; sweeper cleans .part./.tmp./stale temp_ but
keeps fresh files. 193 passed (180 baseline + 13).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
claude code agent
2026-07-05 07:40:47 +03:00
parent 03d1de2954
commit 444bd3e42f
6 changed files with 523 additions and 45 deletions
Download Patch File Download Diff File Expand all files Collapse all files
dockercompose.yml
+3
View File
@@ -25,6 +25,9 @@ services:
# TG_RPC_CONCURRENCY: 1 # Max concurrent live Telegram RPC calls — global throttle (default: 1)
# TG_RPC_MIN_INTERVAL_MS: 500 # Minimum gap between live Telegram RPC starts, ms (default: 500)
# TG_RPC_TIMEOUT: 60 # Max seconds a single live Telegram RPC may run before timing out (default: 60)
# MEDIA_DOWNLOAD_TIMEOUT_MIN: 120 # Min per-download timeout, seconds — also the timeout for regular (non-large) files (default: 120)
# MEDIA_DOWNLOAD_TIMEOUT_MAX: 1800 # Max per-download timeout, seconds — cap for the largest videos (default: 1800)
# MEDIA_DOWNLOAD_MIN_SPEED: 262144 # Assumed floor download speed, bytes/s — large-video timeout ≈ file_size / this, clamped to [MIN,MAX] (default: 262144 = 256 KB/s)
PYROGRAM_BRIDGE_URL: https://pgbridge.example.com
API_PORT: 80
TOKEN: ХХХ