yujunjun/LightRAG
1
0
Fork 0
mirror of https://github.com/HKUDS/LightRAG.git synced 2025-07-03 23:19:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,179 Commits 3 Branches 39 Tags
Commit Graph

222 Commits

Author SHA1 Message Date
yangdx
8d6960f280 Fix linting 2025-03-02 00:13:11 +08:00
yangdx
f76cf98dbd Add automatic dependency checking and installation for server startup 
• Added check_and_install_dependencies()
• Install missing dependencies automatically
 2025-03-02 00:04:59 +08:00
yangdx
40e9e26edb feat: add update flags status to API health endpoint 2025-03-01 14:58:26 +08:00
yangdx
c07a5039b7 Refactor shared storage locks to separate pipeline, storage and internal locks for deadlock preventing 2025-03-01 10:48:55 +08:00
yangdx
fd76e00c6a Refactor storage initialization to separate object creation from data loading 
• Split __post_init__ and initialize()
• Move data loading to initialize()
• Add FastAPI lifespan integration
 2025-03-01 03:48:19 +08:00
yangdx
b3328542c7 refactor: migrate synchronous locks to async locks for improved concurrency 
• Add UnifiedLock wrapper class
• Convert with blocks to async with
 2025-03-01 02:22:35 +08:00
yangdx
aac1bdd9e6 feat: add configurable log rotation settings via environment variables 
• Add LOG_DIR env var for log file location
• Add LOG_MAX_BYTES for max log file size
• Add LOG_BACKUP_COUNT for backup count
 2025-02-28 23:21:14 +08:00
yangdx
c973498c34 Fix linting 2025-02-28 21:35:04 +08:00
yangdx
c37b1e8aa7 Align Gunicorn configuration with Uvicorn 
- centralize config in gunicorn_config.py
- fix log level handling in Gunicorn
 2025-02-28 20:41:11 +08:00
yangdx
f588cdc5df Optimize logging config & worker handling for different server modes 
• Separate logging config for uvicorn/gunicorn
• Force workers=1 in uvicorn mode
• Add warning for worker count in uvicorn
 2025-02-28 16:50:54 +08:00
yangdx
81f6f6e343 Fix lightrag logger initailization problem, fix gunicorn acccess log missing 2025-02-28 16:07:33 +08:00
yangdx
157ec862ae Enhance logging system with file rotation and unified configuration 
• Unify logging across Gunicorn and Uvicorn
• Add rotating file handlers
 2025-02-28 14:57:25 +08:00
yangdx
b090a22be7 Add concurrency check for auto scan task to prevent duplicate scans 
• Add pipeline status check before scan
• Add storage lock protection
• Add latest_message to status tracking
• Add helpful log message at startup
 2025-02-28 12:22:20 +08:00
yangdx
64f22966a3 Fix linting 2025-02-27 19:05:51 +08:00
yangdx
03d05b094d Improve Gunicorn support and cleanup shared storage initialization 
• Move Gunicorn check before other startup
• Improve startup flow organization
 2025-02-27 14:13:42 +08:00
yangdx
7aec78833c Implement Gunicorn+Uvicorn integration for shared data preloading 
- Create run_with_gunicorn.py script to properly initialize shared data in the
  main process before forking worker processes
- Revert unvicorn to single process mode only, and let gunicorn do all the multi-process jobs
 2025-02-27 13:25:22 +08:00
yangdx
7c237920b1 Refactor shared storage to support both single and multi-process modes 
• Initialize storage based on worker count
• Remove redundant global variable checks
• Add explicit mutex initialization
• Centralize shared storage initialization
• Fix process/thread lock selection logic
 2025-02-27 08:48:33 +08:00
yangdx
7436c06f6c Fix linting 2025-02-26 18:11:16 +08:00
yangdx
41f5d208a9 fix: shared data intitialization failed for multi-worker 2025-02-26 13:32:15 +08:00
yangdx
15a6a9cf7c fix: log filtering void when uvicorn wokers is greater than 1 
- Centralize logging setup
- Fix logger propagation issues
 2025-02-26 12:23:35 +08:00
yangdx
2752a764ae Refactor storage implementations to support both single and multi-process modes 
• Add shared storage management module
• Support process/thread lock based on mode
 2025-02-26 05:38:38 +08:00
yangdx
d74a23d2cc Add multiple workers support for API Server 2025-02-25 09:37:00 +08:00
yangdx
2e13def95c Remove unused global_top_k variable and related configurations. 2025-02-24 18:20:39 +08:00
yangdx
f5efe5977b Merge branch 'clear-text-before-insert' into simplify-cli-arguments 2025-02-23 17:06:39 +08:00
yangdx
df95f251dc Move server ready message to lifespan 2025-02-23 16:42:31 +08:00
yangdx
637d6756b8 Optimize RAG initialization for openai-ollama 2025-02-23 11:54:36 +08:00
yangdx
e8efcc335d Add access log filtering to reduce noise from high-frequency API endpoints 2025-02-21 17:53:01 +08:00
yangdx
5d884f6d3e refactor: centralize configuration and utility functions 
- Move parse_args and display_splash_screen functions from lightrag_server.py to utils_api.py
- Move OllamaServerInfos class and instance from ollama_api.py to utils_api.py
 2025-02-20 12:12:20 +08:00
yangdx
a8abcf14ac Fit linting 2025-02-20 04:12:21 +08:00
yangdx
3c080a9ebf Enhance webui mounting with root endpoint and directory check. 
- Added FileResponse for webui root endpoint
- Enabled directory check in StaticFiles mount
- Improved webui static file handling
- Ensured webui directory existence
- Simplified webui access with root endpoint
 2025-02-20 04:04:54 +08:00
yangdx
c0c87edc45 split lightrag_servery.py to smaller files 2025-02-20 03:26:39 +08:00
Yannick Stephan
c79b15c1cc
Merge pull request #844 from danielaskdd/add-duplicate-check 
Fix office file indexing problem
 2025-02-19 08:34:40 +01:00
yangdx
32bfcbb332 Revert "Improved document enqueue logic with existence checks" 2025-02-19 13:39:50 +08:00
ArnoChen
ca170ce783 rename is_managed_by_server to auto_manage_storages_states 2025-02-19 05:27:38 +08:00
ArnoChen
f3b030e1a3 revert temp changes 2025-02-19 04:53:33 +08:00
ArnoChen
6d8e627f85 improve MongoDB client management and storage init 2025-02-19 04:30:52 +08:00
ArnoChen
e194e04226 refactor database connection management and improve storage lifecycle handling 
update
 2025-02-19 03:49:05 +08:00
yangdx
8196df83f8 Fix linting 2025-02-19 01:39:25 +08:00
yangdx
d34dbc5717 Improved document enqueue logic with existence checks. 
- Added return status to `apipeline_enqueue_documents`
- Enhanced logging for duplicate documents
 2025-02-19 01:35:34 +08:00
yangdx
cac93424d9 Added support for reading .xlsx files in LightRAG. 
- Install openpyxl if not present
- Load .xlsx file using openpyxl
- Extract sheet titles and content
- Format rows with tab-separated values
- Append sheet content to overall text
 2025-02-19 01:34:51 +08:00
yangdx
9c45824e78 Optimize file handling for DOCX and PPTX processing. 
- Removed redundant file content reading.
- Directly passed file to BytesIO.
- Simplified DOCX content extraction.
- Streamlined PPTX slide processing.
- Reduced memory usage in file handling.
 2025-02-19 01:34:25 +08:00
yangdx
89c01c686f Fix casting dateime it to str in needed 
- Added datetime formatting utility and standardized timestamp handling in DocStatusResponse
 2025-02-18 17:46:28 +08:00
yangdx
5acd054180 Fix linting 2025-02-18 16:33:57 +08:00
yangdx
75ee4592b8 refactor: Implement dynamic database module imports 
- Consolidate database instance management
- Improve database management and error handling
- Enhance error handling and logging
 2025-02-18 16:14:11 +08:00
Yannick Stephan
ad5c5bba31 fix merge 2025-02-17 08:48:53 +01:00
Yannick Stephan
ef92a3f1f4 Merge remote-tracking branch 'origin/main' into api_improvment 
# Conflicts:
#	lightrag/api/lightrag_server.py
 2025-02-17 08:48:15 +01:00
Yannick Stephan
e9e786b75c update pr 2025-02-17 08:44:37 +01:00
zrguo
fc6dff04ff fix logger level 2025-02-17 15:10:15 +08:00
zrguo
fce24f7611
Merge pull request #797 from danielaskdd/add-env-settings 
Add the token size truncation for local query and token size setting by env
 2025-02-17 15:00:07 +08:00
Yannick Stephan
efec27c0b8 fixed stream 2025-02-16 20:10:28 +01:00