yangdx
6d5c04d561
Reduce max_depth and update edge type in NetworkXStorage.
...
- Decreased max_depth from 5 to 3
- Changed edge type from "RELATED" to "DIRECTED"
2025-03-06 20:54:02 +08:00
yangdx
5b29e760f0
Add dynamic parameter handling for storage
2025-03-06 14:29:48 +08:00
yangdx
252d1f57e0
Merge branch 'main' into add-graph-search-mode
2025-03-06 10:23:38 +08:00
yangdx
0ca7c0e230
Fix linting
2025-03-05 15:07:11 +08:00
yangdx
4e2a9b1994
Apply min_degree filter for full graph query
2025-03-05 13:21:17 +08:00
yangdx
ca9e958cad
Fix subgraph filtering bugs
2025-03-05 13:13:46 +08:00
yangdx
5f12aa3ee4
Fix linting
2025-03-05 12:32:23 +08:00
yangdx
5e40e4107d
Added min_degree exception for connected nodes
2025-03-05 11:48:04 +08:00
yangdx
1fddc8552e
Added minimum degree filter for graph queries
...
- Introduced min_degree parameter in graph query
- Updated UI to include minimum degree setting
- Modified API to handle min_degree parameter
- Updated graph query logic in LightRAG
2025-03-05 11:37:55 +08:00
zrguo
1ee6c23a53
fix linting
2025-03-04 16:12:27 +08:00
zrguo
4ebaf8026b
Update oracle_impl.py
2025-03-04 16:11:13 +08:00
yangdx
002948d342
Added search mode and min degree filtering for NetworkX
...
- Implemented exact and inclusive search modes
- Added min degree filtering for nodes
- Updated API to parse label for search options
2025-03-04 16:08:05 +08:00
zrguo
81568f3bad
fix linting
2025-03-04 15:53:20 +08:00
zrguo
3a2a636862
Implement the missing methods.
2025-03-04 15:50:53 +08:00
zrguo
0679ca4055
Update neo4j_impl.py
2025-03-04 14:20:55 +08:00
zrguo
fd9f71e0ee
fix delete_by_doc_id
2025-03-04 13:22:33 +08:00
yangdx
11fdb60fe5
Remove Chinese comments
2025-03-03 01:30:41 +08:00
yangdx
465737efed
Fix linting
2025-03-02 17:32:25 +08:00
yangdx
68bf02abb6
refactor: improve graph querying with label substring matching and security fixes
2025-03-02 16:20:37 +08:00
yangdx
0f1eb42c8d
Add node limit and prioritization for knowledge graph retrieval
...
• Add MAX_GRAPH_NODES limit from env var
• Prioritize nodes by label match & connection
2025-03-02 15:39:14 +08:00
yangdx
1ca6837219
Add max nodes limit for graph retrieval of networkX
...
• Set MAX_GRAPH_NODES env var (default 1000)
• Change edge type to "RELATED"
2025-03-02 12:52:25 +08:00
yangdx
c0b22a8ae2
Merge branch 'main' into add-multi-worker-support
2025-03-02 02:54:57 +08:00
yangdx
7cd25fe5ab
Improve shared storage cleanup and clarify initialization in multi-worker setup
2025-03-02 01:00:27 +08:00
yangdx
e8d0d065f3
fix: Improve async handling and FAISS storage reliability
...
- Add async context manager support
- Fix embedding data type conversion
- Improve error handling in FAISS ops
- Add multiprocess storage sync
2025-03-01 23:35:09 +08:00
yangdx
9aef112d51
Fix incorrect comment about update flag behavior in FAISS implementation
2025-03-01 22:27:12 +08:00
zrguo
4219454fab
fix format
2025-03-01 17:45:06 +08:00
yangdx
e3a40c2fdb
Fix linting
2025-03-01 16:23:34 +08:00
yangdx
d18eb52ccc
Add type ignore comments for asyncpg imports to suppress mypy errors
2025-03-01 15:38:39 +08:00
yangdx
40e9e26edb
feat: add update flags status to API health endpoint
2025-03-01 14:58:26 +08:00
yangdx
41eff2ca2f
Fix data persistence issue in NanoVectorDBStorage
2025-03-01 13:35:00 +08:00
yangdx
35bcfca28f
feat: add multi-process support for FAISS vector storage
...
• Add storage update flag and locks
• Support cross-process index reload
• Add async initialize method
2025-03-01 12:42:30 +08:00
yangdx
d4f6dcfd54
Improve multi-process data synchronization and persistence in storage implementations
...
• Remove _get_client() or _get_graph() from index_done_callback
• Add return value for index_done_callback
2025-03-01 12:41:30 +08:00
yangdx
c07a5039b7
Refactor shared storage locks to separate pipeline, storage and internal locks for deadlock preventing
2025-03-01 10:48:55 +08:00
yangdx
d3de57c1e4
Add multi-process support for vector database and graph storage with lock flags
...
• Implement storage lock mechanism
• Add update flag handling
• Add cross-process reload detection
2025-03-01 10:37:05 +08:00
yangdx
d704512139
Refactor shared storage module to improve async handling and naming consistency
...
• Add async support for get_namespace_data
• Rename get_update_flags to get_update_flag
• Rename set_update_flag to set_all_update_flags
• Update docstrings for clarity
• Fix typos in log messages
2025-03-01 05:01:26 +08:00
yangdx
fd76e00c6a
Refactor storage initialization to separate object creation from data loading
...
• Split __post_init__ and initialize()
• Move data loading to initialize()
• Add FastAPI lifespan integration
2025-03-01 03:48:19 +08:00
yangdx
b3328542c7
refactor: migrate synchronous locks to async locks for improved concurrency
...
• Add UnifiedLock wrapper class
• Convert with blocks to async with
2025-03-01 02:22:35 +08:00
yangdx
a721421bd8
Add async support and update flag mechanism for shared storage
...
• Use asyncio.Lock instead of thread lock for single process mode
• Add storage update notification system
2025-03-01 01:49:26 +08:00
yangdx
731d820bcc
Remove redundancy set_logger function and related calls
2025-02-28 21:46:45 +08:00
yangdx
8cd45161f2
feat: add history_messages to track pipeline processing progress
...
• Add shared history_messages list
• Track pipeline progress with messages
2025-02-28 13:53:40 +08:00
yangdx
b090a22be7
Add concurrency check for auto scan task to prevent duplicate scans
...
• Add pipeline status check before scan
• Add storage lock protection
• Add latest_message to status tracking
• Add helpful log message at startup
2025-02-28 12:22:20 +08:00
yangdx
b2da69b7f1
Add pipeline status control for concurrent document indexing processes
...
• Add shared pipeline status namespace
• Implement concurrent process control
• Add request queuing for pending jobs
2025-02-28 11:52:42 +08:00
yangdx
cd7648791a
Fix linting
2025-02-28 01:25:59 +08:00
yangdx
3dcfa561d7
Remove debug logging
2025-02-28 01:15:12 +08:00
yangdx
291e0c1b14
revert vector and graph use local data(single process)
2025-02-28 01:14:25 +08:00
yangdx
05d03638ec
Clean up logging output and remove redundant log messages
2025-02-27 20:17:28 +08:00
yangdx
05cf029bcc
fix: convert multiprocessing managed dict to normal dict before JSON dump
2025-02-27 20:16:53 +08:00
yangdx
64f22966a3
Fix linting
2025-02-27 19:05:51 +08:00
yangdx
946095ef80
Fix multiprocess dict creation logic, add process safety locks for namespace creation.
2025-02-27 19:03:53 +08:00
yangdx
e881bc0709
simplify process state management by removing redundant multiprocess flag
2025-02-27 15:36:12 +08:00
