Commit Graph

20 Commits

Author SHA1 Message Date
vvzvlad f9550d8330 feat: upgrade to Kurigram 2.2.23 and render new Telegram media types
Docker Image CI / build (pull_request) Has been cancelled
Upgrade Kurigram 2.2.22 -> 2.2.23 (no breaking API changes; whole suite
green on the new version unchanged).

New in 2.2.23 and now rendered by the bridge:
- LIVE_PHOTO: rendered as a looping video via the /media pipeline
- Poll media (description_media): photo/video/animation/sticker attached
  to a poll is rendered above the poll block; find_file_id_in_message
  searches poll description_media AND explanation_media instead of
  early-returning None for polls

Previously silent media types now rendered:
- STORY: reposted story photo/video via the /media pipeline (render and
  URL derive from the same helper-selected object)
- GIVEAWAY / GIVEAWAY_WINNERS: info blocks with quantity/prize/date
- PAID_MEDIA: info block with star amount (content is not downloadable)
- CHECKLIST: title + tasks with done marks
- CONTACT, LOCATION, VENUE (with OSM links), DICE, GAME, INVOICE,
  UNSUPPORTED: info blocks, all user-controlled strings html-escaped

Supporting changes:
- titles for all new types in _media_message_title
- no_image flag covers info-block types; poll with media is not no_image;
  LIVE_PHOTO counts as video for the video flag
- _save_media_file_ids collects live_photo/story/poll media for the
  background cache warmer, honoring the >100MB video guard; paid media
  is never collected
- info-block types no longer emit an empty message-media div
- 53 new tests (media URLs with digest, download symmetry, XSS escaping,
  flags, size guard); 314 total

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 17:58:34 +03:00
claude code agent 870e0a40d8 fix(stability): stage 3 — serve via FileResponse, pure-ASGI logging, bigger executor
Replaces the hand-rolled media streaming with Starlette FileResponse, drops the
BaseHTTPMiddleware, and enlarges the default threadpool.

3.1 prepare_file_response now returns FileResponse (handles Range/If-Range/206/
    416/multipart, sets Accept-Ranges/ETag/Last-Modified, reads efficiently — no
    per-64KB to_thread hop that starved the pool). Kept: the early 404 pre-check,
    the MIME logic (python-magic + SQLite cache), and every stage-2 behavior — the
    temp_* mtime touch (now DEBOUNCED, see below), delete_after -> BackgroundTask
    (passed as FileResponse background=), the media_key MIME cache. Removed the
    manual Range parsing, file_chunk_generator, and hand-built headers;
    Content-Disposition is formed by FileResponse from filename= (no double-set).
    206 slices are byte-identical to the old code; accepted RFC-7233 deltas
    documented in the tests.
3.2 RequestLoggingMiddleware rewritten as a pure-ASGI class (wraps only send to
    observe the status line, never buffers the body, passes non-http scopes
    through) — the streaming body flows untouched.
3.3 lifespan sets a larger default executor (ThreadPoolExecutor, IO_THREAD_POOL_SIZE
    default 32) and shuts it down on exit.

Review round-1 fixes folded in: the temp_* mtime touch is DEBOUNCED
(TEMP_MTIME_REFRESH_INTERVAL=300s) so FileResponse's mtime-derived ETag stays
stable across a resume/seek session (an every-serve touch broke If-Range resume);
starlette pinned to 0.45.3; the io executor is shut down on lifespan exit; the
ASGI logger includes the query string.

Tests (tests/test_stage3_fileresponse.py, 18): the Range matrix vs FileResponse
with every delta documented; temp_* mtime refreshed when stale AND stable when
fresh (ETag identical); delete_after background runs and removes the file;
media_key MIME cache hit/miss; the ASGI middleware passes the body and logs.
213 passed (195 baseline + 18).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 08:26:35 +03:00
claude code agent 03d1de2954 fix(stability): declare pytest-asyncio, bound /raw_json RPC, assert FloodWait backoff, dedupe gate+timeout (review round 1)
F1 [WARNING] pytest-asyncio was not declared, so the 6 new async stage-1 tests
ERROR on a clean checkout (async def not natively supported) — zero regression
protection, masked by a globally-installed plugin. Added pytest-asyncio to
requirements.txt + asyncio_mode=auto to tests/pytest.ini. Verified on a fresh venv
from requirements alone: the tests collect and run (180 passed).

F2 [WARNING] The /raw_json endpoint's get_messages was the one remaining live RPC
without a timeout, violating the stage-1 DoD ('every Telegram RPC is bounded').
Wrapped it in asyncio.wait_for(..., 30) mirroring PostParser.get_post. (It is not
under the tg_rpc gate, so its blast radius was one request, not the app.)

F3 [WARNING] The worker test globally no-op'd asyncio.sleep, so the dedicated
except FloodWait branch was indistinguishable from the generic handler — deleting
it kept the test green. The sleep stub now records delays and the test asserts the
FloodWait backoff of 6 (=min(1+5,900)), distinct from the success path's 2.

F5 [low] The tricky 'gate outside, timeout inside' nesting was open-coded at 3
sites (each re-deriving the invariant). Extracted tg_rpc_bounded(timeout) into
tg_throttle (using asyncio.timeout()); the 3 sites now use it, so a future call
site cannot silently wrap the gate entry and reopen the hang-under-backpressure.

F4 [low] Documented TG_RPC_TIMEOUT in the dockercompose.yml environment block
next to the other TG_RPC_* knobs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 07:12:10 +03:00
vvzvlad 79b127d406 feat(client): add watchdog and disconnect flap handling
Introduce an active watchdog that probes the Telegram client to detect
zombie sessions and restart them in‑process. Add configurable disconnect
flap detection with a sliding window to trigger restarts after repeated
disconnects. New environment variables and config entries are added, and
the Kurigram dependency is now version‑pinned.
2026-06-04 19:02:09 +03:00
vvzvlad c9b0d2ffc9 feat(db): replace media file ID JSON store with SQLite
Switch persistence of media file identifiers from a JSON file to a
SQLite database. Introduce DB_PATH, init_db_sync, upsert, update, and
removal functions. Remove json-repair dependency and related locking
logic. Update api_server and post_parser to use the new database
helpers.

BREAKING CHANGE: media_file_ids.json is no longer used; existing
data must be migrated to the new SQLite database.
2026-04-05 20:20:37 +03:00
vvzvlad b7a8f4f60d refactor requirements and rss_generator: replace direct get_chat calls with cached_get_chat for improved performance and update requirements.txt to uncomment Kurigram 2025-06-10 02:40:23 +03:00
vvzvlad 6aba094a6c add types support 2025-04-18 19:42:54 +03:00
vvzvlad 46ce824ed0 update requirements.txt to include Kurigram from GitHub and comment out the previous entry 2025-04-14 17:35:12 +03:00
vvzvlad a45678a852 Add JSON repair functionality 2025-03-26 15:04:24 +03:00
vvzvlad 583a4c4b3c Add uvloop support for asyncio in api_server and telegram_client; update requirements.txt 2025-03-21 14:29:57 +03:00
vvzvlad 10434bbbd3 Remove beautifulsoup4 dependency from requirements 2025-02-09 06:02:18 +03:00
vvzvlad 1512c09f1b BIG REFACTORING post parsing and rendering with modular HTML generation and improved media group handling 2025-02-09 02:29:49 +03:00
vvzvlad 594b1a8c57 Add HTML sanitization and improve channel ID handling in PostParser 2025-02-05 23:37:39 +03:00
vvzvlad 86689b47a7 Add BeautifulSoup4 to project dependencies 2025-02-05 23:13:53 +03:00
vvzvlad b95cf63b3c Improve MIME type detection using python-magic 2025-02-03 04:08:48 +03:00
vvzvlad 17ef3b0b09 Update requirements.txt with specific package versions 2025-02-02 18:53:14 +03:00
vvzvlad f360f1167c BIG refactoring 2025-02-02 18:51:50 +03:00
vvzvlad 9bb0d8a021 Add RSS feed generation endpoint for Telegram channels 2025-02-02 01:59:14 +03:00
vvzvlad 20221c78e1 add html support 2025-02-01 17:52:23 +03:00
vvzvlad 6e367b3ac9 first work version 2025-02-01 16:47:46 +03:00