8bcce02368
feat(api): add diagnostic monitoring and semaphore tracking
...
Introduce a periodic event‑loop monitor that logs active tasks, semaphore
state, queue size and thread‑pool activity. Track the number of coroutine
waiters for the download semaphore and emit detailed diagnostic logs in
download handling and the background worker. This aids debugging of
concurrency and resource‑usage issues.
2026-03-16 01:00:31 +03:00
3c2b4ce544
perf(downloads): implement concurrent download queue with semaphore limiting and retry logic
...
Add queue-based background download system to improve performance and reliability:
- Introduce DOWNLOAD_SEMAPHORE to limit concurrent downloads to 3
- Add asyncio.Queue (maxsize 100) with dedicated worker for background processing
- Implement safe_get_messages and safe_download_media wrappers with timeout protection (30s and 120s)
- Add retry logic for KeyError auth failures with 5s backoff
- Replace synchronous sequential downloads with asynchronous queued processing
- Prevent event loop blocking by queuing files instead of immediate download
2026-01-01 14:14:26 +03:00
68a2dbfa04
refactor api_server and post_parser: implement asynchronous JSON file operations with locking mechanism to ensure thread safety, and streamline media file ID persistence logic
2025-09-13 20:47:16 +03:00
1464ea326e
refactor api_server: streamline RSS feed generation by consolidating error handling and removing redundant loop structure
2025-09-13 00:06:36 +03:00
778aa04904
refactor api_server: implement asynchronous file I/O using asyncio.to_thread for JSON operations, add global python-magic instance for MIME type detection, and enhance delayed file deletion documentation
2025-09-12 23:50:52 +03:00
43516d6dab
refactor api_server and TelegramClient: remove force_exit function and replace with sys.exit for process termination, update restart logic to use SIGTERM instead of SIGKILL for graceful shutdown
2025-06-10 15:03:09 +03:00
521c3dc394
implement forceful termination in api_server and TelegramClient: add force_exit function for immediate process shutdown, update SIGTERM handler for forceful exit, and adjust restart logic in connection_handler to use SIGKILL
2025-06-10 14:28:51 +03:00
d6c3725b02
implement graceful shutdown in api_server: add SIGTERM handler for proper termination and enhance error handling for server startup issues
2025-06-10 14:06:56 +03:00
0999e86f2b
add timing logs for RSS and HTML generation in api_server and rss_generator
2025-04-27 18:40:31 +04:00
f499dc56a7
refactor api_server: comment out redundant log for cached media files
2025-04-27 18:22:31 +04:00
422a3925ff
fix bug
2025-04-25 16:59:21 +04:00
386e7e8ce4
improve logging
2025-04-18 23:33:47 +03:00
fe1fee242d
fix local req logic
2025-04-18 21:15:35 +03:00
8c0a88cb84
small fix
2025-04-18 21:10:59 +03:00
1760ec3485
add types-2
2025-04-18 20:39:57 +03:00
6aba094a6c
add types support
2025-04-18 19:42:54 +03:00
cb419c97af
update pylint and pylance disable comments
2025-04-18 18:00:02 +03:00
d520e8c2e8
Update get_raw_post_json
2025-04-13 03:38:00 +03:00
96419265a3
Update get_raw_post_json to using json.dumps
2025-04-13 03:31:01 +03:00
ff97d1190b
Refactor get_raw_post_json
2025-04-13 03:25:28 +03:00
0b75b5a9b9
Add endpoint to retrieve raw JSON post data
2025-04-13 03:19:10 +03:00
e76fb8606d
Remove unnecessary pass statement in ZeroSizeFileError exception class
2025-04-11 18:13:57 +03:00
55d2b7331a
fix linter err
2025-04-11 18:12:56 +03:00
d21a68471d
Refactor zero-size file handling
2025-04-11 18:10:21 +03:00
fb73f6b868
fix zero size bug
2025-04-11 18:00:30 +03:00
f42e159ba6
Add flags endpoint
2025-04-11 02:04:58 +03:00
997b49b4df
removal of invalid entries from media_file_ids.json
2025-04-04 22:21:22 +03:00
129b07abda
add logging
2025-04-04 22:09:52 +03:00
de5c3f1fa1
fix bug Invalid digest : None, expected: None
2025-04-04 22:07:47 +03:00
422e2adc2d
Add shebang and encoding declaration to multiple Python files
2025-04-03 01:48:34 +03:00
a45678a852
Add JSON repair functionality
2025-03-26 15:04:24 +03:00
750c01776a
Update API endpoints to include request parameter for token validation, enhancing security by checking client host against allowed addresses.
2025-03-26 15:00:28 +03:00
bbaf2314d5
Remove base64 and URL decoding
2025-03-22 15:26:55 +03:00
5f5128fed0
Decode base64 parameters if they are encoded
2025-03-22 15:03:53 +03:00
710ad700e0
URL decoding for exclude_text
2025-03-22 15:01:02 +03:00
d6df00ff8c
Refactor RequestLoggingMiddleware to ensure proper logging of request and response details, including error handling during request processing
2025-03-22 14:54:05 +03:00
f5f80487ce
Add RequestLoggingMiddleware to log request and response details in API server
2025-03-22 14:50:59 +03:00
c8ad7f3e58
Remove commented-out logging for valid token in get_rss_feed function to improve code clarity
2025-03-22 14:41:10 +03:00
8723540389
Refactor flood wait handling to return 429 status with Retry-After header instead of sleeping,
2025-03-22 14:28:12 +03:00
e5ed6151e0
Enhance post retrieval functions to include optional debug parameter for logging raw messages; update README.md with usage example for debug mode.
2025-03-21 14:46:57 +03:00
583a4c4b3c
Add uvloop support for asyncio in api_server and telegram_client; update requirements.txt
2025-03-21 14:29:57 +03:00
716fa7ee42
Add text exclusion filter
2025-03-04 03:15:26 +10:00
c4aed732af
Refactor sensitive data masking and configuration logging
2025-02-10 05:33:32 +03:00
29ff31d136
Enhance health check endpoint with masked configuration details
2025-02-09 22:06:00 +03:00
c9c1bacc49
Add server configuration logging with sensitive data masking
2025-02-09 22:04:18 +03:00
67d52b507a
Add time-based media group merging for RSS and HTML generation
2025-02-09 21:44:55 +03:00
7c9fd859d7
Refactor PostParser: Improve code style and method visibility
2025-02-09 06:04:01 +03:00
344052e00e
EPIC: Add content filtering via message flags
2025-02-09 03:53:52 +03:00
9207056df8
Improve temporary file cleanup by removing associated media file entries
2025-02-07 22:51:42 +03:00
77cc68e4e1
Add temp file skip mechanism for large video downloads
2025-02-07 03:17:53 +03:00
vvzvlad