yangdx
efc359c411
Update webui assets
2025-07-13 00:57:41 +08:00
yangdx
0e3aaa318f
Feat: Add keyed lock cleanup and status monitoring
2025-07-13 00:09:00 +08:00
yangdx
e4bf4d19a0
Optimize knowledge graph rebuild with parallel processing
...
- Add parallel processing for KG rebuild
- Implement keyed locks for data consistency
2025-07-12 13:22:56 +08:00
yangdx
a85d7054d4
fix: move node existence check inside lock to prevent race condition
...
Move knowledge_graph_inst.has_node check inside get_storage_keyed_lock
in _merge_edges_then_upsert to ensure atomic check-then-act operations
and prevent duplicate node creation during concurrent updates.
2025-07-12 12:22:32 +08:00
yangdx
2ade3067f8
Refac: Generalize keyed lock with namespace support
...
Refactored the `KeyedUnifiedLock` to be generic and support dynamic namespaces. This decouples the locking mechanism from a specific "GraphDB" implementation, allowing it to be reused across different components and workspaces safely.
Key changes:
- `KeyedUnifiedLock` now takes a `namespace` parameter on lock acquisition.
- Renamed `_graph_db_lock_keyed` to a more generic _storage_keyed_lock`
- Replaced `get_graph_db_lock_keyed` with get_storage_keyed_lock` to support namespaces
2025-07-12 12:10:12 +08:00
yangdx
f2d875f8ab
Update comments
2025-07-12 11:05:25 +08:00
yangdx
943ead8b1d
Bump api version to 0181
2025-07-12 05:59:13 +08:00
yangdx
5ee509e671
Fix linting
2025-07-12 05:17:44 +08:00
yangdx
964293f21b
Optimize lock cleanup with time tracking and intervals
...
- Add cleanup time tracking variables
- Implement minimum cleanup intervals
- Track earliest cleanup times
- Handle time rollback cases
- Improve cleanup logging
2025-07-12 04:34:26 +08:00
yangdx
39965d7ded
Move merging stage back controled by max parallel insert semhore
2025-07-12 03:32:08 +08:00
yangdx
7490a18481
Optimize lock cleanup parameters
2025-07-12 03:10:03 +08:00
yangdx
3d8e6924bc
Show lock clean up message
2025-07-12 02:58:05 +08:00
yangdx
22c36f2fd2
Optimize log messages
2025-07-12 02:41:31 +08:00
yangdx
a64c767298
optimize: improve lock cleanup performance with threshold-based strategy
...
- Add CLEANUP_THRESHOLD constant (100) to control cleanup frequency
- Modify _release_shared_raw_mp_lock to only scan when cleanup list exceeds threshold
- Modify _release_async_lock to only scan when cleanup list exceeds threshold
2025-07-11 23:43:40 +08:00
yangdx
ad99d9ba5a
Improve code organization and comments
2025-07-11 22:13:02 +08:00
yangdx
c52c451cf7
Fix linting
2025-07-11 20:40:50 +08:00
yangdx
3afdd1b67c
Fix initial count error for multi-process lock with key
2025-07-11 20:39:08 +08:00
yangdx
c47747da9e
Merge branch 'main' into merge_lock_with_key
2025-07-11 16:37:10 +08:00
yangdx
ef4870fda5
Combined entity and edge processing tasks and optimize merging with semaphore
2025-07-11 16:34:54 +08:00
zrguo
b0479c078a
fix process_chunks_unified()
2025-07-09 15:55:38 +08:00
yangdx
9aa2ed0837
Merge branch 'main' into rerank
2025-07-09 15:33:39 +08:00
zrguo
e1541caea9
Update webui setting
2025-07-09 12:10:06 +08:00
yangdx
207f0a7f2a
Merge branch 'main' into merge_lock_with_key
2025-07-09 09:25:28 +08:00
yangdx
cb3bfc0e5b
Release semphore before merge stage
2025-07-09 09:24:44 +08:00
Anton Vice
b192f8c9a3
Fix: Handle NoneType error when processing documents without a file path
...
The document processing pipeline would crash with a TypeError when a document was submitted as raw text via the API, as the file_path attribute would be None. This change adds a check to handle the None case gracefully, preventing the crash and allowing text-based documents to be indexed correctly.
2025-07-08 19:35:22 -03:00
yangdx
4705a22861
Bump core version to 1.4.0
2025-07-09 04:43:20 +08:00
yangdx
2056c3c809
Increase default CHUNK_TOP_K from 5 to 15
2025-07-09 04:41:51 +08:00
yangdx
e9c3503f77
Update logger info
2025-07-09 04:36:52 +08:00
yangdx
5d4484882a
Merge branch 'main' into rerank
2025-07-09 03:59:04 +08:00
yangdx
14d51518dd
Merge branch 'add-Memgraph-graph-db' into memgraph
2025-07-09 03:38:07 +08:00
DavIvek
08eb68b8ed
run pre-commit
2025-07-08 20:21:20 +02:00
yangdx
75ce636084
Merge branch 'main' into add-Memgraph-graph-db
2025-07-09 02:09:35 +08:00
DavIvek
4438897b6b
add changes based on review
2025-07-08 16:28:06 +02:00
zrguo
d4651d59c1
Add rerank to server
2025-07-08 21:44:20 +08:00
yangdx
b6ab69e25d
Merge branch 'main' into fix-issue-1746
2025-07-08 18:20:02 +08:00
yangdx
2a0cff3ed6
Fix linting
2025-07-08 18:17:21 +08:00
zrguo
71cb3adb4f
Merge branch 'main' into rerank
2025-07-08 15:10:23 +08:00
zrguo
c295d355a0
fix chunk_top_k limiting
2025-07-08 15:05:30 +08:00
SLKun
5f330ec11a
remove <think> tag for entities and keywords extraction
2025-07-08 14:59:15 +08:00
zrguo
04a57445da
update chunks truncation method
2025-07-08 13:31:05 +08:00
Molion Surya
8cbba6e9db
Fix #1746 : [openai.py logic for streaming complete]
2025-07-08 13:25:52 +08:00
yangdx
56d43de58a
Merge branch 'main' into merge_lock_with_key
2025-07-08 12:46:31 +08:00
zrguo
f5c80d7cde
Simplify Configuration
2025-07-08 11:16:34 +08:00
yangdx
9b7b2a9b0f
Reduce default embedding batch size from 32 to 10
2025-07-08 11:00:09 +08:00
yangdx
2670f8dc98
Merge branch 'main' into add-Memgraph-graph-db
2025-07-08 00:31:46 +08:00
yangdx
a1bbf367ad
Update webui assets
2025-07-08 00:22:14 +08:00
zrguo
75dd4f3498
add rerank model
2025-07-07 22:44:59 +08:00
yangdx
ef79088f60
Move max_graph_nodes to global config
2025-07-07 21:53:57 +08:00
yangdx
cb14ce6ff3
Bump api version to 0180
2025-07-07 18:14:31 +08:00
yangdx
f417118e27
Center banner text dynamically
2025-07-07 17:28:59 +08:00
