19 Commits

Author	SHA1	Message	Date
yangdx	b3328542c7	refactor: migrate synchronous locks to async locks for improved concurrency ... • Add UnifiedLock wrapper class • Convert with blocks to async with	2025-03-01 02:22:35 +08:00
yangdx	a721421bd8	Add async support and update flag mechanism for shared storage ... • Use asyncio.Lock instead of thread lock for single process mode • Add storage update notification system	2025-03-01 01:49:26 +08:00
yangdx	731d820bcc	Remove redundancy set_logger function and related calls	2025-02-28 21:46:45 +08:00
yangdx	8cd45161f2	feat: add history_messages to track pipeline processing progress ... • Add shared history_messages list • Track pipeline progress with messages	2025-02-28 13:53:40 +08:00
yangdx	b090a22be7	Add concurrency check for auto scan task to prevent duplicate scans ... • Add pipeline status check before scan • Add storage lock protection • Add latest_message to status tracking • Add helpful log message at startup	2025-02-28 12:22:20 +08:00
yangdx	b2da69b7f1	Add pipeline status control for concurrent document indexing processes ... • Add shared pipeline status namespace • Implement concurrent process control • Add request queuing for pending jobs	2025-02-28 11:52:42 +08:00
yangdx	cd7648791a	Fix linting	2025-02-28 01:25:59 +08:00
yangdx	3dcfa561d7	Remove debug logging	2025-02-28 01:15:12 +08:00
yangdx	291e0c1b14	revert vector and graph use local data(single process)	2025-02-28 01:14:25 +08:00
yangdx	05d03638ec	Clean up logging output and remove redundant log messages	2025-02-27 20:17:28 +08:00
yangdx	64f22966a3	Fix linting	2025-02-27 19:05:51 +08:00
yangdx	946095ef80	Fix multiprocess dict creation logic, add process safety locks for namespace creation.	2025-02-27 19:03:53 +08:00
yangdx	e881bc0709	simplify process state management by removing redundant multiprocess flag	2025-02-27 15:36:12 +08:00
yangdx	03d05b094d	Improve Gunicorn support and cleanup shared storage initialization ... • Move Gunicorn check before other startup • Improve startup flow organization	2025-02-27 14:13:42 +08:00
yangdx	7aec78833c	Implement Gunicorn+Uvicorn integration for shared data preloading ... - Create run_with_gunicorn.py script to properly initialize shared data in the main process before forking worker processes - Revert unvicorn to single process mode only, and let gunicorn do all the multi-process jobs	2025-02-27 13:25:22 +08:00
yangdx	7c237920b1	Refactor shared storage to support both single and multi-process modes ... • Initialize storage based on worker count • Remove redundant global variable checks • Add explicit mutex initialization • Centralize shared storage initialization • Fix process/thread lock selection logic	2025-02-27 08:48:33 +08:00
yangdx	7436c06f6c	Fix linting	2025-02-26 18:11:16 +08:00
yangdx	7d12715f09	Refactor shared storage to safely handle multi-process initialization and data sharing ... • Add namespace initialization check • Use atomic operations for shared data	2025-02-26 18:11:02 +08:00
yangdx	2752a764ae	Refactor storage implementations to support both single and multi-process modes ... • Add shared storage management module • Support process/thread lock based on mode	2025-02-26 05:38:38 +08:00