yangdx
81f6f6e343
Fix lightrag logger initailization problem, fix gunicorn acccess log missing
2025-02-28 16:07:33 +08:00
yangdx
157ec862ae
Enhance logging system with file rotation and unified configuration
...
• Unify logging across Gunicorn and Uvicorn
• Add rotating file handlers
2025-02-28 14:57:25 +08:00
yangdx
8cd45161f2
feat: add history_messages to track pipeline processing progress
...
• Add shared history_messages list
• Track pipeline progress with messages
2025-02-28 13:53:40 +08:00
yangdx
b090a22be7
Add concurrency check for auto scan task to prevent duplicate scans
...
• Add pipeline status check before scan
• Add storage lock protection
• Add latest_message to status tracking
• Add helpful log message at startup
2025-02-28 12:22:20 +08:00
yangdx
04bd5413c9
Add API endpoint to retrieve document indexing pipeline status
...
• GET /pipeline_status endpoint added
• Returns current pipeline processing state
2025-02-28 12:21:50 +08:00
yangdx
b4bcd76599
Remove useless scan progress tracking functionality and related code
2025-02-28 10:53:36 +08:00
yangdx
db2a902fcb
Rename get_scan_lock to get_storage_lock
2025-02-28 00:34:33 +08:00
yangdx
64f22966a3
Fix linting
2025-02-27 19:05:51 +08:00
yangdx
92ecb0da97
Refactor document scanning progress share variable initialization
2025-02-27 16:07:00 +08:00
yangdx
03d05b094d
Improve Gunicorn support and cleanup shared storage initialization
...
• Move Gunicorn check before other startup
• Improve startup flow organization
2025-02-27 14:13:42 +08:00
yangdx
7aec78833c
Implement Gunicorn+Uvicorn integration for shared data preloading
...
- Create run_with_gunicorn.py script to properly initialize shared data in the
main process before forking worker processes
- Revert unvicorn to single process mode only, and let gunicorn do all the multi-process jobs
2025-02-27 13:25:22 +08:00
yangdx
7c237920b1
Refactor shared storage to support both single and multi-process modes
...
• Initialize storage based on worker count
• Remove redundant global variable checks
• Add explicit mutex initialization
• Centralize shared storage initialization
• Fix process/thread lock selection logic
2025-02-27 08:48:33 +08:00
yangdx
7436c06f6c
Fix linting
2025-02-26 18:11:16 +08:00
yangdx
4eb069d1d6
Initialize scan_progress with default values if not already set
2025-02-26 17:42:49 +08:00
yangdx
41f5d208a9
fix: shared data intitialization failed for multi-worker
2025-02-26 13:32:15 +08:00
yangdx
15a6a9cf7c
fix: log filtering void when uvicorn wokers is greater than 1
...
- Centralize logging setup
- Fix logger propagation issues
2025-02-26 12:23:35 +08:00
yangdx
2752a764ae
Refactor storage implementations to support both single and multi-process modes
...
• Add shared storage management module
• Support process/thread lock based on mode
2025-02-26 05:38:38 +08:00
yangdx
7262f61b0e
add redis configuration and update workers default value
2025-02-25 10:47:27 +08:00
yangdx
04fc5ce604
Remove unspported endpoint from splash mesages
2025-02-25 09:45:14 +08:00
yangdx
ddc366b672
Optimize display_splash_screen function
...
- Merge System Configuration into Server Configuration section
- Add Workers parameter display after Port parameter
2025-02-25 09:44:17 +08:00
yangdx
d74a23d2cc
Add multiple workers support for API Server
2025-02-25 09:37:00 +08:00
yangdx
2e13def95c
Remove unused global_top_k variable and related configurations.
2025-02-24 18:20:39 +08:00
yangdx
facf7c11fe
Reduce knowledge graph max_depth from 100 to 3
2025-02-24 03:34:44 +08:00
yangdx
f29628125b
Fix typo in parameter name from 'nodel_label' to 'node_label'
2025-02-24 02:36:36 +08:00
yangdx
f5efe5977b
Merge branch 'clear-text-before-insert' into simplify-cli-arguments
2025-02-23 17:06:39 +08:00
yangdx
78e0ca7835
Change defautl timeout for LLM to 150s
2025-02-23 17:03:35 +08:00
yangdx
57884f2fb8
Refine LLM settings in env sample file
2025-02-23 16:52:41 +08:00
yangdx
df95f251dc
Move server ready message to lifespan
2025-02-23 16:42:31 +08:00
yangdx
dbeda8a9ff
Change scanning logs from INFO to DEBUG level
2025-02-23 16:12:08 +08:00
yangdx
460bc3a6aa
update README.md and .env.example
2025-02-23 13:26:38 +08:00
yangdx
637d6756b8
Optimize RAG initialization for openai-ollama
2025-02-23 11:54:36 +08:00
yangdx
d84b90bcd4
Handle special case of CLI argument openai-ollama
2025-02-22 15:27:19 +08:00
yangdx
f9780830ad
Revert: get llm-binding and embedding-binding from cli
2025-02-22 10:46:54 +08:00
yangdx
2fd997ca4f
Merge branch 'clear-text-before-insert' into simplify-cmdline-arguments
2025-02-22 10:07:46 +08:00
yangdx
3c866eec16
Merge branch 'refactor-api-server' into clear-text-before-insert
2025-02-22 10:04:56 +08:00
yangdx
dff07e50a4
Merge branch 'main' into refactor-api-server
2025-02-21 21:12:02 +08:00
yangdx
b144e0c3b0
Sync modifications from main branch
2025-02-21 21:07:37 +08:00
yangdx
2fdbbc2062
Reorder and improve command line argument definitions for better organization
...
- Change verbose to store_true action
- Move verbose flag to logging section
- Move auto-scan flag to end of options
2025-02-21 20:01:43 +08:00
yangdx
a848884a7b
Remove unnesessary CLI arguments, reduce CLI arguments complexity
...
• Move storage config from CLI
• Move LLM and embedding binding config from CLI
• Remove chunk config from CLI
2025-02-21 19:34:17 +08:00
yangdx
9ba12a4f31
refactor: remove redundant top_k checks in query routes
...
The top_k parameter already has a default value set in the QueryParam class (base.py), making these checks unnecessary. This change simplifies the code while maintaining the same functionality.
Changes:
Remove top_k check in query_text function
Remove top_k check in query_text_stream function
2025-02-21 18:53:43 +08:00
yangdx
e8efcc335d
Add access log filtering to reduce noise from high-frequency API endpoints
2025-02-21 17:53:01 +08:00
yangdx
1749678384
Fix linting
2025-02-21 12:16:04 +08:00
Saifeddine ALOUI
ed9dc799b0
Merge branch 'HKUDS:main' into main
2025-02-20 18:15:55 +01:00
Yannick Stephan
678e0f9aea
Revert "Cleanup of code"
2025-02-20 15:09:43 +01:00
Yannick Stephan
439685e69c
Revert "removed get_knowledge_graph"
2025-02-20 14:29:36 +01:00
Saifeddine ALOUI
c3c3c87494
Added instructions on how to install age for postgresql
2025-02-20 11:55:09 +01:00
Saifeddine ALOUI
167fa82363
Fixed linting
2025-02-20 11:38:07 +01:00
Saifeddine ALOUI
3f3e9ef367
Upgraded webui to show more upload options and added building without using bun to the webui
2025-02-20 11:25:57 +01:00
Saifeddine ALOUI
3627b4485f
Upgraded webui to support more files
2025-02-20 11:14:48 +01:00
Saifeddine ALOUI
38b62d58ad
Added extra extentions
2025-02-20 10:59:07 +01:00
