zrguo
479865a271
Add max_gleaning to env
2025-07-01 17:13:33 +08:00
SLKun
4e88ee3662
update ollama compatible api
2025-06-30 10:41:35 +08:00
yangdx
4c2b4b4b6b
Revert "Fix LLM cache handling for Redis to address document deletion scenarios."
...
This reverts commit 14cda93988dc110c11cf1fe8286ded7b5db7b8fb.
2025-06-29 22:35:40 +08:00
yangdx
10cd9c90e7
Revert "Fix linting"
...
This reverts commit abd9de2a63b39021320855fab9f5c3b332dee162.
2025-06-29 22:35:26 +08:00
yangdx
75ea0bc38d
Remove default get_nodes_by_chunk_ids implementation
...
- Deletes inefficient default implementation
- Makes method purely abstract
2025-06-29 22:05:48 +08:00
yangdx
e70f5a35e5
Refac: Add KG rebuild logging with pipeline status
...
- Logs detailed progress, including warnings and failures, to the pipeline status.
- Adds counters to report the total number of successfully rebuilt entities and relationships upon completion.
2025-06-29 21:27:12 +08:00
yangdx
abd9de2a63
Fix linting
2025-06-29 15:15:49 +08:00
yangdx
14cda93988
Fix LLM cache handling for Redis to address document deletion scenarios.
...
- Implements bulk scan for "extract" cache entries
- Maintains backward compatibility for normal IDs
2025-06-29 15:13:42 +08:00
yangdx
e2824b721e
Fix LLM cache handling for MongoKVStorage to address document deletion scenarios.
...
- Support fetching all "default_" prefixed documents
- Maintain original behavior for other IDs
- Return dictionary of documents for "default"
- Keep backward compatibility
2025-06-29 15:03:57 +08:00
yangdx
37bf341a69
Fix LLM cache handling for PGKVStorage to address document deletion scenarios.
...
- Add dynamic cache_type field
- Support mode parameter for LLM cache
- Maintain backward compatibility
2025-06-29 14:39:50 +08:00
Moddy Surala
319d12bb6a
Fix GRAPH_FIELD_SEP import typo
2025-06-29 01:28:39 +05:00
yangdx
6d5e73a251
Merge branch 'kenspirit/main'
2025-06-29 00:30:58 +08:00
yangdx
28aedd8b3c
Update comments
2025-06-29 00:30:39 +08:00
Ken Chen
4a953d6829
As Graph edges should be treated undirectional, fix incorrect upsert_edge method in MongoDBGraph
2025-06-28 21:03:54 +08:00
Ken Chen
5116d61eaa
As Graph edges should be treated undirectional, fix incorrect has_edge method in MongoDBGraph
2025-06-28 20:48:30 +08:00
Ken Chen
73cc86662a
Add two BFS subgraph search support for MongoDBGraph
2025-06-28 20:00:13 +08:00
yangdx
8522bfc9dc
Optimied logger info
2025-06-28 19:27:36 +08:00
Ken Chen
5739f52d29
Rewrite get_knowledge_graph with label * by degree
2025-06-28 17:10:39 +08:00
Ken Chen
d0f4eee404
Fix accidentally hardcoded edge collection name in searching upstream nodes
2025-06-28 16:25:44 +08:00
yangdx
b7f8c20e61
fix(postgres): use correct table for vector queries
...
Change SQL templates from LIGHTRAG_DOC_CHUNKS to LIGHTRAG_VDB_CHUNKS
to fix "content_vector does not exist" error in vector operations.
2025-06-28 15:36:54 +08:00
yangdx
2c47367975
Fix linting
2025-06-28 14:37:55 +08:00
yangdx
95c7a7d038
feat(db): Add data migration from LIGHTRAG_DOC_CHUNKS to LIGHTRAG_VDB_CHUNKS
2025-06-28 14:37:47 +08:00
yangdx
3a8a99b73d
feat(postgres): Implement text_chunks upsert for PGKVStorage
2025-06-28 14:37:35 +08:00
Ken Chen
6574dfb7ea
Fix accidentally hardcode max depth in searching upstream nodes
2025-06-28 11:40:39 +08:00
Ken Chen
b586bdc02f
Fix accidentally hardcode label in searching upstream nodes
2025-06-28 10:50:56 +08:00
yangdx
0f51ec48f1
fix: streaming error when only_need_context=True returns empty results
...
Prevents NoneType async iteration error by handling None responses
in stream_generator and ensuring kg_query returns valid strings.
2025-06-28 09:18:06 +08:00
Ken Chen
7c8f65d020
Add search on neighbor nodes which are source to selected one
2025-06-28 08:50:32 +08:00
yangdx
a506753548
Fix linting
2025-06-27 02:33:20 +08:00
yangdx
60777d535b
fix: prevent Path Traversal vulnerability in upload endpoint
...
- Add sanitize_filename() function to validate and clean uploaded filenames
- Remove path separators, traversal sequences, and control characters
- Verify final paths stay within input directory using Path.resolve()
- Return HTTP 400 errors for unsafe filenames
- Prevents directory traversal attacks like ../../../etc/passwd
2025-06-27 02:33:05 +08:00
yangdx
44daf51501
Fix edge_degree to handle non-existent nodes
2025-06-27 01:37:10 +08:00
Ken Chen
f40bc43d5e
Fix nodes & edges are missing when retrieving knowledge subgraph by selecting particular node_id
2025-06-26 23:11:31 +08:00
Daniel.y
c740401b7f
Merge pull request #1654 from a-bruhn/azure-env-vars
...
Clean up azure env vars
2025-06-26 19:11:20 +08:00
yangdx
6e1ca8f461
Merge branch 'patch-1'
2025-06-26 18:45:23 +08:00
yangdx
fb3006b44d
Update requirements.txt
2025-06-26 18:45:01 +08:00
yangdx
687ccd4923
fix: optimize MongoDB aggregation pipeline to prevent memory limit errors
...
- Move $limit operation early in pipeline for "*" queries to reduce memory usage
- Remove memory-intensive $sort operation for large dataset queries
- Add fallback mechanism for memory limit errors with simple query
- Implement additional safety checks to enforce max_nodes limit
- Improve error handling and logging for memory-related issues
2025-06-26 14:37:04 +08:00
yangdx
d8b544ab6f
Fix linting
2025-06-26 14:15:11 +08:00
yangdx
c51079335e
Optimize node label retrieval with aggregation
...
- Enable allowDiskUse for large datasets
2025-06-26 14:14:52 +08:00
yangdx
d60db573dc
Add allowDiskUse flag to MongoDB aggregations
...
- Enable disk use for large aggregations
- Fix cursor handling for list_search_indexes
- Improve query performance for big datasets
- Update vector search index check
- Set proper length for to_list results
2025-06-26 13:51:53 +08:00
yangdx
71565f4794
Add get_all method to MongoKVStorage
2025-06-26 13:51:15 +08:00
yangdx
d512db26e4
Fix MongoDB set handling in delete operations
2025-06-26 13:50:19 +08:00
yangdx
6364d076aa
Enable MongoGraphStorage
2025-06-26 13:47:50 +08:00
yangdx
d031cc7cdf
Merge branch 'main' into kenspirit/main
2025-06-26 09:20:52 +08:00
yangdx
8fb1c09b08
Refac: pipelinge message
2025-06-26 01:00:54 +08:00
Ken Chen
a3865caaea
Implement get_nodes_by_chunk_ids and get_edges_by_chunk_ids,
2025-06-25 22:17:17 +08:00
Ken Chen
81cff6e97f
Merge branch 'HKUDS:main' into main
2025-06-25 20:35:55 +08:00
yangdx
88c528ec94
Update webui assetes
2025-06-25 19:26:24 +08:00
yangdx
bdcd55a871
Feat: Add delete upload file option to document deletion
2025-06-25 19:02:46 +08:00
yangdx
6a1737784d
Bump api version to 0176
2025-06-25 17:17:56 +08:00
yangdx
f440461810
Update webui assets
2025-06-25 17:16:59 +08:00
yangdx
51bb0471cd
Change the API for deleting documents to support deleting multiple documents at once.
2025-06-25 16:19:49 +08:00
