yangdx
b7f8c20e61
fix(postgres): use correct table for vector queries
...
Change SQL templates from LIGHTRAG_DOC_CHUNKS to LIGHTRAG_VDB_CHUNKS
to fix "content_vector does not exist" error in vector operations.
2025-06-28 15:36:54 +08:00
yangdx
2c47367975
Fix linting
2025-06-28 14:37:55 +08:00
yangdx
95c7a7d038
feat(db): Add data migration from LIGHTRAG_DOC_CHUNKS to LIGHTRAG_VDB_CHUNKS
2025-06-28 14:37:47 +08:00
yangdx
3a8a99b73d
feat(postgres): Implement text_chunks upsert for PGKVStorage
2025-06-28 14:37:35 +08:00
yangdx
0f51ec48f1
fix: streaming error when only_need_context=True returns empty results
...
Prevents NoneType async iteration error by handling None responses
in stream_generator and ensuring kg_query returns valid strings.
2025-06-28 09:18:06 +08:00
yangdx
a506753548
Fix linting
2025-06-27 02:33:20 +08:00
yangdx
60777d535b
fix: prevent Path Traversal vulnerability in upload endpoint
...
- Add sanitize_filename() function to validate and clean uploaded filenames
- Remove path separators, traversal sequences, and control characters
- Verify final paths stay within input directory using Path.resolve()
- Return HTTP 400 errors for unsafe filenames
- Prevents directory traversal attacks like ../../../etc/passwd
2025-06-27 02:33:05 +08:00
yangdx
44daf51501
Fix edge_degree to handle non-existent nodes
2025-06-27 01:37:10 +08:00
Daniel.y
c740401b7f
Merge pull request #1654 from a-bruhn/azure-env-vars
...
Clean up azure env vars
2025-06-26 19:11:20 +08:00
yangdx
6e1ca8f461
Merge branch 'patch-1'
2025-06-26 18:45:23 +08:00
yangdx
fb3006b44d
Update requirements.txt
2025-06-26 18:45:01 +08:00
yangdx
687ccd4923
fix: optimize MongoDB aggregation pipeline to prevent memory limit errors
...
- Move $limit operation early in pipeline for "*" queries to reduce memory usage
- Remove memory-intensive $sort operation for large dataset queries
- Add fallback mechanism for memory limit errors with simple query
- Implement additional safety checks to enforce max_nodes limit
- Improve error handling and logging for memory-related issues
2025-06-26 14:37:04 +08:00
yangdx
d8b544ab6f
Fix linting
2025-06-26 14:15:11 +08:00
yangdx
c51079335e
Optimize node label retrieval with aggregation
...
- Enable allowDiskUse for large datasets
2025-06-26 14:14:52 +08:00
yangdx
d60db573dc
Add allowDiskUse flag to MongoDB aggregations
...
- Enable disk use for large aggregations
- Fix cursor handling for list_search_indexes
- Improve query performance for big datasets
- Update vector search index check
- Set proper length for to_list results
2025-06-26 13:51:53 +08:00
yangdx
71565f4794
Add get_all method to MongoKVStorage
2025-06-26 13:51:15 +08:00
yangdx
d512db26e4
Fix MongoDB set handling in delete operations
2025-06-26 13:50:19 +08:00
yangdx
6364d076aa
Enable MongoGraphStorage
2025-06-26 13:47:50 +08:00
yangdx
d031cc7cdf
Merge branch 'main' into kenspirit/main
2025-06-26 09:20:52 +08:00
yangdx
8fb1c09b08
Refac: pipelinge message
2025-06-26 01:00:54 +08:00
Ken Chen
a3865caaea
Implement get_nodes_by_chunk_ids and get_edges_by_chunk_ids,
2025-06-25 22:17:17 +08:00
Ken Chen
81cff6e97f
Merge branch 'HKUDS:main' into main
2025-06-25 20:35:55 +08:00
yangdx
88c528ec94
Update webui assetes
2025-06-25 19:26:24 +08:00
yangdx
bdcd55a871
Feat: Add delete upload file option to document deletion
2025-06-25 19:02:46 +08:00
yangdx
6a1737784d
Bump api version to 0176
2025-06-25 17:17:56 +08:00
yangdx
f440461810
Update webui assets
2025-06-25 17:16:59 +08:00
yangdx
51bb0471cd
Change the API for deleting documents to support deleting multiple documents at once.
2025-06-25 16:19:49 +08:00
yangdx
495d6c8cce
Improve the pipeline status message for document deletetion
2025-06-25 15:46:58 +08:00
yangdx
2aaa6d5f7d
Fix linting
2025-06-25 14:59:45 +08:00
yangdx
49baeb7318
Change document deletion API to async
2025-06-25 14:59:10 +08:00
yangdx
8a365533d7
Add comprehensive error handling for document deletion
2025-06-25 14:58:41 +08:00
yangdx
922484915b
Remove deprecated API endpoint.
2025-06-25 13:55:47 +08:00
yangdx
72384f87c4
Remove deprecated code from Postgres_impl.py
...
- Stop filtering out 'base' node labels
- Match any edge type in query to improve performance
2025-06-25 12:53:07 +08:00
yangdx
492269ac44
Bump core version to 1.3.10
2025-06-25 12:39:57 +08:00
yangdx
109c2b48be
Fix linting
2025-06-25 12:39:43 +08:00
yangdx
da46b341dc
feat: Optimize document deletion performance
...
- To enhance performance during document deletion, new batch-get methods, `get_nodes_by_chunk_ids` and `get_edges_by_chunk_ids`, have been added to the graph storage layer (`BaseGraphStorage` and its implementations). The [`adelete_by_doc_id`](lightrag/lightrag.py:1681) function now leverages these methods to avoid unnecessary iteration over the entire knowledge graph, significantly improving efficiency.
- Graph storage updated: Networkx, Neo4j, Postgres AGE
2025-06-25 12:37:57 +08:00
yangdx
ebe5b1e0d2
Bump api version to 0175
2025-06-24 22:16:06 +08:00
Ken Chen
12054fa8d9
Merge branch 'HKUDS:main' into main
2025-06-24 20:20:54 +08:00
yangdx
2946bbdb71
Add TODO: There is performance when iterating get_all_labels
2025-06-24 11:32:28 +08:00
yangdx
8b6dcfb6eb
Pls do not use /delete_document API endpoint
2025-06-24 11:26:38 +08:00
yangdx
5ae945c1e5
Improved error handling for document deletion
...
Added HTTPException for not_found status
Added HTTPException for fail status
2025-06-24 01:12:25 +08:00
yangdx
fd69c895cd
Remove debug looging from Neo4j
2025-06-24 01:03:02 +08:00
yangdx
64d0df0caa
Remove debug logging from Neo4J operations
...
- Delete node upsert debug log
- Remove edge upsert debug messages
2025-06-24 00:45:17 +08:00
yangdx
cc12460b38
Fix: Silence PostgreSQL logs during idempotent graph initialization
2025-06-23 23:08:56 +08:00
yangdx
c18065a912
Disable document deletion when LLM cache for extraction is off
2025-06-23 22:41:27 +08:00
yangdx
1973c80dca
Feat: Add entity and relation deletion endpoints
2025-06-23 22:14:50 +08:00
yangdx
e6baffe10c
Add retrun status to entity and relation delete operations
2025-06-23 21:39:45 +08:00
yangdx
bd487dd252
Unify document APIs returen status string
2025-06-23 21:38:47 +08:00
yangdx
ce50135efb
Improved docstring for document deletion method
2025-06-23 21:08:51 +08:00
yangdx
1697ea4bf6
Fix linting
2025-06-23 19:20:35 +08:00
