Commit Graph

190 Commits

Author SHA1 Message Date
vvzvlad f0e38b9776 perf(core): improve async handling and caching performance
Convert delayed_delete_file to async and use asyncio.sleep.
Add pre‑semaphore cache hit to serve cached files without acquiring the download semaphore.
Set SQLite busy_timeout to reduce lock errors.
Introduce a diagnostics counter for pending _persist_media_file_id_async tasks.
Refactor flag extraction to accept pre‑generated HTML and compute html body once.
Batch reply enrichment in RSS generator to minimize API calls.
Store cached messages directly in tg_cache and add fallback for legacy double‑pickle format.

BREAKING CHANGE: delayed_delete_file is now async and must be awaited.
2026-05-18 00:08:46 +03:00
vvzvlad ab8f15d49d feat(api): add atomic temp file handling for media downloads
- Use unique temporary file paths with UUID to avoid race conditions during concurrent downloads.
- Perform atomic rename of the temp file to the final cache location and handle existing concurrent downloads.
- Clean up zero‑size and stale temporary files, including new “.tmp.<hex>” pattern.
- Extend cleanup logic to detect and remove race‑condition temp files.
- Validate and convert environment variables (TG_API_ID, TG_PROXY_PORT, API_PORT) with clear error messages and proper exit handling.
- Flush prints and replace `os._exit` with `sys.exit` for graceful termination.
- Fix URL regex in `PostParser` and guard against missing channel usernames when generating media URLs.
- Deep‑copy message lists in RSS generator to prevent mutation across calls.
- Propagate `FloodWait` exceptions to be handled by the API server instead of retrying internally.
- Enhance `TelegramClient` disconnect handling with a brief pause, shutdown check, and reconnection reset.
- Refine auth retry logic to check for actual `KeyError` instances.
2026-05-17 23:49:18 +03:00
vvzvlad dd5999b51f fix: guard against missing messages and reaction issues
Added a guard in `api_server.download_media_file` to detect `None` or empty messages and return a 404 error, preventing downstream failures.

Enhanced `PostParser._extract_reactions` to handle paid reactions, custom emojis, and unknown types, aggregating counts correctly. Updated `_extract_flags` to skip paid and custom emoji reactions, ensuring flag detection works reliably.
2026-05-17 23:12:11 +03:00
vvzvlad 705310da28 fix(parser): log channel arg to avoid UnboundLocalError
Use the original `channel` argument in the error log instead of
`prepared_channel_id` to prevent an UnboundLocalError when an
exception is raised during post parsing.
2026-05-17 22:30:01 +03:00
vvzvlad cd4e6f0c82 fix(api): use MessageMediaType enum for poll checks
Replace string comparisons with the MessageMediaType.POLL enum and add the
necessary import. This ensures poll messages are correctly identified and
skipped during media processing.
2026-04-16 03:46:03 +03:00
vvzvlad 9f9e952c4e fix(parser): handle poll question/option objects with .text
Update poll handling to extract the `text` attribute from question and option objects when present, falling back to string conversion otherwise. This ensures proper rendering of poll content from Telegram objects that expose a `text` field.
2026-04-16 02:43:53 +03:00
vvzvlad f19e2bfe6d fix(parser): cast poll question and option texts to string
Convert poll.question and option.text to strings before processing to avoid type errors when they are not already strings.
2026-04-16 02:40:35 +03:00
vvzvlad c9b0d2ffc9 feat(db): replace media file ID JSON store with SQLite
Switch persistence of media file identifiers from a JSON file to a
SQLite database. Introduce DB_PATH, init_db_sync, upsert, update, and
removal functions. Remove json-repair dependency and related locking
logic. Update api_server and post_parser to use the new database
helpers.

BREAKING CHANGE: media_file_ids.json is no longer used; existing
data must be migrated to the new SQLite database.
2026-04-05 20:20:37 +03:00
vvzvlad a38a729137 perf(api): add timing diagnostics for media ops
Add monotonic timing instrumentation around media download, semaphore
wait, HTML sanitization, and media file persistence. Log warnings when
operations exceed defined thresholds and provide diagnostic info for
task queue size. This aids performance monitoring without altering
behaviour.
2026-03-16 01:14:24 +03:00
vvzvlad 5257c50243 fix(post_parser): correct content order when caption display flag is toggled
The changes ensure that when `show_caption_above` is true, text content is rendered before media, and when false, media is rendered before text. This aligns the actual rendering order with the semantic meaning of the flag, fixing a bug where content ordering did not match user expectations.
2026-01-02 05:38:30 +03:00
vvzvlad 76e163098b feat(post_parser): support show caption above media attribute for message rendering
Add support for Telegram's show_caption_above_media message attribute to conditionally order caption and media content in HTML output. When enabled, media is rendered before the text caption instead of the default text-before-media order.
2026-01-02 05:27:18 +03:00
vvzvlad 5543c0514a feat(post_parser): allow strikethrough tags in HTML sanitization
Add 's' and 'del' tags to the allowed HTML tags list in the sanitization
method to support strikethrough and deleted text formatting.
2026-01-02 02:33:37 +03:00
vvzvlad 68a2dbfa04 refactor api_server and post_parser: implement asynchronous JSON file operations with locking mechanism to ensure thread safety, and streamline media file ID persistence logic 2025-09-13 20:47:16 +03:00
vvzvlad 45e9ac7a1e refactor post_parser: add line breaks for improved readability in post content 2025-04-27 13:06:39 +04:00
vvzvlad e0bb640a73 refactor post_parser: enhance post text and poll formatting, improve HTML structure 2025-04-26 13:15:03 +04:00
vvzvlad 508be049d9 refactor post_parser: streamline content body construction for improved readability 2025-04-26 12:55:38 +04:00
vvzvlad 36236d4407 refactor post_parser: clean up whitespace and improve PDF placeholder handling 2025-04-26 12:52:18 +04:00
vvzvlad 1bd36b32d8 refactor post_parser: wrap media content in a div for better HTML structure 2025-04-26 12:51:37 +04:00
vvzvlad c340d22fc4 improve error logging 2025-04-26 12:46:22 +04:00
vvzvlad 721302d2e6 oops 2025-04-26 12:42:27 +04:00
vvzvlad b9d5e839df refactor post_parser and rss_generator:move media in body render 2025-04-26 12:38:52 +04:00
vvzvlad db0d1a6dc2 refactor _generate_html_body to streamline reply info handling and improve logging 2025-04-20 19:07:30 +03:00
vvzvlad 80d6cc5d5a add line break in debug HTML 2025-04-19 04:33:47 +03:00
vvzvlad 65a69ed37a refactoring 2025-04-19 04:32:44 +03:00
vvzvlad 4e1987983f refactoring 2025-04-19 04:11:02 +03:00
vvzvlad 2f7f524e83 remove unnecessary line breaks in forwarded message HTML output 2025-04-19 04:03:14 +03:00
vvzvlad c2bbf72b07 refactor _format_forward_info to handle multiple forward cases and improve logging 2025-04-19 04:01:52 +03:00
vvzvlad 5d4951aecd update comment in post_parser.py to clarify main cut logic 2025-04-19 03:55:03 +03:00
vvzvlad 1e283c4249 fix bug 2025-04-19 03:50:33 +03:00
vvzvlad 97ad18e9d4 remove unused 2025-04-19 00:01:21 +03:00
vvzvlad 26744e3699 refactoring 2025-04-18 23:02:29 +03:00
vvzvlad d2870bc0f6 add youtube title 2025-04-18 22:58:25 +03:00
vvzvlad d0584d4525 more logging 2025-04-18 21:52:17 +03:00
vvzvlad ba20f2ee02 add logs 2025-04-18 21:43:56 +03:00
vvzvlad 87822c0d51 add message id debug 2025-04-18 21:34:41 +03:00
vvzvlad 045360d497 debug print 2025-04-18 21:28:15 +03:00
vvzvlad 5aa5783672 format fix 2025-04-18 21:25:45 +03:00
vvzvlad 1760ec3485 add types-2 2025-04-18 20:39:57 +03:00
vvzvlad 6aba094a6c add types support 2025-04-18 19:42:54 +03:00
vvzvlad ec2f439e56 format 2025-04-18 18:02:31 +03:00
vvzvlad cb419c97af update pylint and pylance disable comments 2025-04-18 18:00:02 +03:00
vvzvlad 6dcec780ac improve error logging 2025-04-18 17:27:55 +03:00
vvzvlad 34701a4c50 refactor PostParser 2025-04-18 17:05:51 +03:00
vvzvlad fb95636da5 add detailed error logging for reactions views parsing in PostParser 2025-04-18 16:58:15 +03:00
vvzvlad 46aadb93d9 revert commit 2025-04-18 05:41:31 +03:00
vvzvlad 70c8e7aaf0 enhance forward info 2025-04-18 05:33:13 +03:00
vvzvlad 64a505cc71 change log level 2025-04-18 05:26:31 +03:00
vvzvlad c31eddf211 refactor forward info handling in PostParser to streamline title generation and improve debug logging 2025-04-18 05:23:53 +03:00
vvzvlad 5549890e6a fix bug 2025-04-18 05:10:59 +03:00
vvzvlad 65d5952241 add audio flag to PostParser for messages with AUDIO or VOICE media 2025-04-18 05:08:28 +03:00