Commit Graph

68 Commits

Author SHA1 Message Date
vvzvlad f0e38b9776 perf(core): improve async handling and caching performance
Convert delayed_delete_file to async and use asyncio.sleep.
Add pre‑semaphore cache hit to serve cached files without acquiring the download semaphore.
Set SQLite busy_timeout to reduce lock errors.
Introduce a diagnostics counter for pending _persist_media_file_id_async tasks.
Refactor flag extraction to accept pre‑generated HTML and compute html body once.
Batch reply enrichment in RSS generator to minimize API calls.
Store cached messages directly in tg_cache and add fallback for legacy double‑pickle format.

BREAKING CHANGE: delayed_delete_file is now async and must be awaited.
2026-05-18 00:08:46 +03:00
vvzvlad ab8f15d49d feat(api): add atomic temp file handling for media downloads
- Use unique temporary file paths with UUID to avoid race conditions during concurrent downloads.
- Perform atomic rename of the temp file to the final cache location and handle existing concurrent downloads.
- Clean up zero‑size and stale temporary files, including new “.tmp.<hex>” pattern.
- Extend cleanup logic to detect and remove race‑condition temp files.
- Validate and convert environment variables (TG_API_ID, TG_PROXY_PORT, API_PORT) with clear error messages and proper exit handling.
- Flush prints and replace `os._exit` with `sys.exit` for graceful termination.
- Fix URL regex in `PostParser` and guard against missing channel usernames when generating media URLs.
- Deep‑copy message lists in RSS generator to prevent mutation across calls.
- Propagate `FloodWait` exceptions to be handled by the API server instead of retrying internally.
- Enhance `TelegramClient` disconnect handling with a brief pause, shutdown check, and reconnection reset.
- Refine auth retry logic to check for actual `KeyError` instances.
2026-05-17 23:49:18 +03:00
vvzvlad 64fe6327d8 fix(rss): handle missing dates in media grouping and RSS
Add deterministic fallback sorting for messages without a date to avoid
TypeError during grouping. Use current time as fallback when calculating
time differences and when generating RSS entries. Sort rendered posts
with epoch fallback for None dates and emit a warning when a post lacks a
date. These changes make the RSS generation robust against messages that
do not have timestamps.
2026-05-17 22:37:20 +03:00
vvzvlad 6e2c1f79d3 fix(rss): stop popping oldest group in _trim_messages_groups
Removed the pop operation that discarded the oldest message group during trimming, preventing unintended loss of messages.
2026-05-17 22:28:28 +03:00
vvzvlad 9bfef3bf2d fix(rss): remove filter for NEW_CHAT_TITLE messages
Previously NEW_CHAT_TITLE events were ignored in the RSS generator,
preventing title updates from being included. The filter line has been
removed so these messages are now processed correctly.
2026-04-16 04:02:10 +03:00
vvzvlad fef63c1224 chore(rss_generator): add logging for RSS feed date ranges and entry timestamps
Logs the date range (oldest/newest) and total count of posts being added to RSS feeds, plus debug-level logging for each entry's timestamp conversion. This improves observability for feed generation and helps diagnose date-related issues.
2026-01-01 15:02:36 +03:00
vvzvlad 337cabe180 fix(rss_generator): handle Telegram flood wait errors with automatic retry logic
Add flood wait exception handling in both RSS and HTML generation functions.
The bot now automatically waits for the specified duration and retries the
operation when encountering rate limiting from Telegram's API.
2026-01-01 14:57:16 +03:00
vvzvlad deacb8b9a0 refactor rss_generator: offload HTML concatenation and sanitization to asyncio.to_thread to improve CPU performance and prevent event loop blocking 2025-09-15 12:16:53 +03:00
vvzvlad 14081a3d44 refactor rss_generator and tg_cache: replace cached_get_chat with direct client.get_chat calls, update message caching logic to handle limits, and improve cache file naming for message history 2025-06-10 03:48:08 +03:00
vvzvlad b7a8f4f60d refactor requirements and rss_generator: replace direct get_chat calls with cached_get_chat for improved performance and update requirements.txt to uncomment Kurigram 2025-06-10 02:40:23 +03:00
vvzvlad 0999e86f2b add timing logs for RSS and HTML generation in api_server and rss_generator 2025-04-27 18:40:31 +04:00
vvzvlad b9d5e839df refactor post_parser and rss_generator:move media in body render 2025-04-26 12:38:52 +04:00
vvzvlad 5c34d62c11 EPIC: support REPLY_TO in rss 2025-04-19 04:49:01 +03:00
vvzvlad ce39431998 move import 2025-04-18 23:34:26 +03:00
vvzvlad 386e7e8ce4 improve logging 2025-04-18 23:33:47 +03:00
vvzvlad 26744e3699 refactoring 2025-04-18 23:02:29 +03:00
vvzvlad 27f39a3948 add processed_message_to_tg_message for merge flags 2025-04-18 21:08:20 +03:00
vvzvlad 3f253b7a0e add template 2025-04-18 21:05:16 +03:00
vvzvlad 8321844a3e refactoring 2025-04-18 20:57:48 +03:00
vvzvlad 1760ec3485 add types-2 2025-04-18 20:39:57 +03:00
vvzvlad 1b2cc65899 fix bug 2025-04-18 20:31:37 +03:00
vvzvlad 6aba094a6c add types support 2025-04-18 19:42:54 +03:00
vvzvlad cb419c97af update pylint and pylance disable comments 2025-04-18 18:00:02 +03:00
vvzvlad 7621235baa todo 2025-04-18 05:38:48 +03:00
vvzvlad c9d5f6b2f6 refactor PostParser and RSS generator to remove header generation and adjust body rendering 2025-04-18 05:06:26 +03:00
vvzvlad 9e394b28e2 add merged flag to processed messages in RSS generator 2025-04-18 05:02:09 +03:00
vvzvlad 5dc5783b86 Revert "Enhance RSS description generation"
This reverts commit ef5206a0c8.
2025-04-16 06:18:45 +03:00
vvzvlad ef5206a0c8 Enhance RSS description generation 2025-04-16 03:18:47 +03:00
vvzvlad 77d85ad3be Update generate_html_footer to accept optional flags_list for improved flexibility in HTML generation 2025-04-15 19:06:24 +03:00
vvzvlad 8deb96ce38 Refactor PostParser and RSS generator to improve flag handling and HTML footer generation 2025-04-15 18:56:12 +03:00
vvzvlad 1098770c53 Collect unique flags from processed messages in _render_messages_groups 2025-04-15 18:14:43 +03:00
vvzvlad 1871843b17 small refactoring 2025-04-15 02:43:26 +03:00
vvzvlad 0600ce4307 add more logging 2025-04-14 17:06:25 +03:00
vvzvlad defba4cdaf Enhance PostParser to handle service messages 2025-04-13 19:14:44 +03:00
vvzvlad 6c8debbff6 Add condition to ignore VIDEO_CHAT_STARTED messages in message grouping 2025-04-12 19:57:50 +03:00
vvzvlad 422e2adc2d Add shebang and encoding declaration to multiple Python files 2025-04-03 01:48:34 +03:00
vvzvlad 3d54230169 Update PostParser to classify unknown posts with a new emoji and enhance RSS generator to ignore new chat title events 2025-04-01 02:22:59 +03:00
vvzvlad 93698e0d4e adding UNICODE flag to regex pattern 2025-03-22 15:11:30 +03:00
vvzvlad d3fe2c1b71 exclude_text filtering now use regex 2025-03-22 14:46:36 +03:00
vvzvlad ac2c06fe22 Update limit validation in generate_channel_rss to allow a maximum of 200 instead of 100 2025-03-22 13:55:04 +03:00
vvzvlad a17a4aa860 Update media section formatting in _render_messages_groups to include line breaks for better HTML presentation 2025-03-21 15:32:20 +03:00
vvzvlad e2701d7979 Skip service messages about new chat photos 2025-03-11 13:47:51 +03:00
vvzvlad 716fa7ee42 Add text exclusion filter 2025-03-04 03:15:26 +10:00
vvzvlad 2dc1fecc66 Improve error logging and increase HTML generation limit 2025-02-10 05:54:28 +03:00
vvzvlad 44dbeabe1d Add docstrings and improve HTML generation in RSS generator 2025-02-10 05:53:11 +03:00
vvzvlad 60b253c1c1 Refactor session handling and configuration management 2025-02-10 05:09:29 +03:00
vvzvlad 67d52b507a Add time-based media group merging for RSS and HTML generation 2025-02-09 21:44:55 +03:00
vvzvlad 232a2d3a0a Add message group trimming to RSS and HTML generation 2025-02-09 20:30:01 +03:00
vvzvlad 893bb02a40 Fix channel history retrieval parameter in RSS generator 2025-02-09 06:04:27 +03:00
vvzvlad 7c9fd859d7 Refactor PostParser: Improve code style and method visibility 2025-02-09 06:04:01 +03:00