yangdx
a506753548
Fix linting
2025-06-27 02:33:20 +08:00
yangdx
60777d535b
fix: prevent Path Traversal vulnerability in upload endpoint
...
- Add sanitize_filename() function to validate and clean uploaded filenames
- Remove path separators, traversal sequences, and control characters
- Verify final paths stay within input directory using Path.resolve()
- Return HTTP 400 errors for unsafe filenames
- Prevents directory traversal attacks like ../../../etc/passwd
2025-06-27 02:33:05 +08:00
yangdx
44daf51501
Fix edge_degree to handle non-existent nodes
2025-06-27 01:37:10 +08:00
Daniel.y
c740401b7f
Merge pull request #1654 from a-bruhn/azure-env-vars
...
Clean up azure env vars
2025-06-26 19:11:20 +08:00
yangdx
6e1ca8f461
Merge branch 'patch-1'
2025-06-26 18:45:23 +08:00
yangdx
fb3006b44d
Update requirements.txt
2025-06-26 18:45:01 +08:00
zrguo
5b6ac84cd0
Update README
2025-06-26 18:08:21 +08:00
zrguo
082a338df3
Update README.md
2025-06-26 17:52:52 +08:00
zrguo
afba3df01a
Update README
2025-06-26 16:20:39 +08:00
zrguo
fc7a0329df
Update README
2025-06-26 16:17:00 +08:00
zrguo
145e3a238b
Update README.md
2025-06-26 16:12:20 +08:00
zrguo
1d788c3e97
Update RAGAnything related
2025-06-26 16:08:14 +08:00
yangdx
687ccd4923
fix: optimize MongoDB aggregation pipeline to prevent memory limit errors
...
- Move $limit operation early in pipeline for "*" queries to reduce memory usage
- Remove memory-intensive $sort operation for large dataset queries
- Add fallback mechanism for memory limit errors with simple query
- Implement additional safety checks to enforce max_nodes limit
- Improve error handling and logging for memory-related issues
2025-06-26 14:37:04 +08:00
yangdx
d8b544ab6f
Fix linting
2025-06-26 14:15:11 +08:00
yangdx
c51079335e
Optimize node label retrieval with aggregation
...
- Enable allowDiskUse for large datasets
2025-06-26 14:14:52 +08:00
yangdx
778ad4f23a
Merge branch 'kenspirit/main'
2025-06-26 13:52:42 +08:00
yangdx
d60db573dc
Add allowDiskUse flag to MongoDB aggregations
...
- Enable disk use for large aggregations
- Fix cursor handling for list_search_indexes
- Improve query performance for big datasets
- Update vector search index check
- Set proper length for to_list results
2025-06-26 13:51:53 +08:00
yangdx
71565f4794
Add get_all method to MongoKVStorage
2025-06-26 13:51:15 +08:00
yangdx
d512db26e4
Fix MongoDB set handling in delete operations
2025-06-26 13:50:19 +08:00
yangdx
6364d076aa
Enable MongoGraphStorage
2025-06-26 13:47:50 +08:00
yangdx
d031cc7cdf
Merge branch 'main' into kenspirit/main
2025-06-26 09:20:52 +08:00
yangdx
dc182bfaca
Merge branch 'delete_doc'
2025-06-26 01:21:25 +08:00
yangdx
8fb1c09b08
Refac: pipelinge message
2025-06-26 01:00:54 +08:00
Ken Chen
a3865caaea
Implement get_nodes_by_chunk_ids and get_edges_by_chunk_ids,
2025-06-25 22:17:17 +08:00
Ken Chen
81cff6e97f
Merge branch 'HKUDS:main' into main
2025-06-25 20:35:55 +08:00
Daniel.y
394b5870c6
Merge pull request #1698 from HKUDS/delete_doc
...
Feat: Implement document deletion functionality in the WebUI
2025-06-25 19:37:21 +08:00
yangdx
88c528ec94
Update webui assetes
2025-06-25 19:26:24 +08:00
yangdx
bdcd55a871
Feat: Add delete upload file option to document deletion
2025-06-25 19:02:46 +08:00
yangdx
6a1737784d
Bump api version to 0176
2025-06-25 17:17:56 +08:00
yangdx
f440461810
Update webui assets
2025-06-25 17:16:59 +08:00
yangdx
be41af3749
Update i18n files
2025-06-25 17:16:21 +08:00
yangdx
0f392dfa78
Fix linting
2025-06-25 17:09:41 +08:00
yangdx
8ea7d7ad85
Feat: Add document deletion for WebUI
2025-06-25 17:09:10 +08:00
yangdx
51bb0471cd
Change the API for deleting documents to support deleting multiple documents at once.
2025-06-25 16:19:49 +08:00
yangdx
495d6c8cce
Improve the pipeline status message for document deletetion
2025-06-25 15:46:58 +08:00
yangdx
2aaa6d5f7d
Fix linting
2025-06-25 14:59:45 +08:00
yangdx
49baeb7318
Change document deletion API to async
2025-06-25 14:59:10 +08:00
yangdx
8a365533d7
Add comprehensive error handling for document deletion
2025-06-25 14:58:41 +08:00
yangdx
922484915b
Remove deprecated API endpoint.
2025-06-25 13:55:47 +08:00
Daniel.y
c6a28bfe4b
Merge pull request #1697 from HKUDS/delete_doc
...
feat: Optimize document deletion performance
2025-06-25 13:16:20 +08:00
yangdx
72384f87c4
Remove deprecated code from Postgres_impl.py
...
- Stop filtering out 'base' node labels
- Match any edge type in query to improve performance
2025-06-25 12:53:07 +08:00
yangdx
492269ac44
Bump core version to 1.3.10
2025-06-25 12:39:57 +08:00
yangdx
109c2b48be
Fix linting
2025-06-25 12:39:43 +08:00
yangdx
da46b341dc
feat: Optimize document deletion performance
...
- To enhance performance during document deletion, new batch-get methods, `get_nodes_by_chunk_ids` and `get_edges_by_chunk_ids`, have been added to the graph storage layer (`BaseGraphStorage` and its implementations). The [`adelete_by_doc_id`](lightrag/lightrag.py:1681) function now leverages these methods to avoid unnecessary iteration over the entire knowledge graph, significantly improving efficiency.
- Graph storage updated: Networkx, Neo4j, Postgres AGE
2025-06-25 12:37:57 +08:00
yangdx
ebe5b1e0d2
Bump api version to 0175
2025-06-24 22:16:06 +08:00
Ken Chen
12054fa8d9
Merge branch 'HKUDS:main' into main
2025-06-24 20:20:54 +08:00
Daniel.y
3acdeeb871
Merge pull request #1693 from HKUDS/delete_doc
...
Feat: Add comprehensive document deletion feature with integrated knowledge graph reconstruction
2025-06-24 11:33:20 +08:00
yangdx
2946bbdb71
Add TODO: There is performance when iterating get_all_labels
2025-06-24 11:32:28 +08:00
yangdx
8b6dcfb6eb
Pls do not use /delete_document API endpoint
2025-06-24 11:26:38 +08:00
yangdx
5ae945c1e5
Improved error handling for document deletion
...
Added HTTPException for not_found status
Added HTTPException for fail status
2025-06-24 01:12:25 +08:00
