31 Commits

Author SHA1 Message Date
Saifeddine ALOUI
06c9e4e454 Fixed missing imports bug and fixed linting 2025-01-25 00:55:07 +01:00
Saifeddine ALOUI
34018cb1e0 Separated llms from the main llm.py file and fixed some deprication bugs 2025-01-25 00:11:00 +01:00
hyb
ff71952c8c feat: 新增ini文件读取数据库配置方式,方便生产环境,修改Lightrag ainsert方法_add_doc_keys获取方式,原来只过滤存在的,但这会让失败的文档无法再次存储,新增--chunk_size和--chunk_overlap_size方便生产环境,新增llm_binding:openai-ollama 方便用openai的同时使用ollama embedding 2025-01-23 22:58:57 +08:00
zrguo
d489d9dec0 fix linting errors 2024-12-31 17:32:04 +08:00
zrguo
cee5b2fbb0 add delete by doc id 2024-12-31 17:15:57 +08:00
Magic_yuan
7271ac69b7 修复断点续传bug,过滤已经存在的文档是只过滤状态是已经处理完成的 2024-12-30 20:32:10 +08:00
Magic_yuan
7b91dc7fd8 feat: 增强知识图谱关系的时序性支持
- 为关系和向量数据增加时间戳支持,记录知识获取的时间
- 优化混合查询策略,同时考虑语义相关性和时间顺序
- 增强提示词模板,指导LLM在处理冲突信息时考虑时间因素
2024-12-29 15:37:34 +08:00
Magic_yuan
4c950cf4ce feat: 增强知识图谱关系的时序性支持
- 为关系和向量数据增加时间戳支持,记录知识获取的时间
- 优化混合查询策略,同时考虑语义相关性和时间顺序
- 增强提示词模板,指导LLM在处理冲突信息时考虑时间因素
2024-12-29 15:25:57 +08:00
Magic_yuan
650b8e38b7 feat(lightrag): Add document status tracking and checkpoint support
功能(lightrag): 添加文档状态跟踪和断点续传支持

- Add DocStatus enum and DocProcessingStatus class for document processing state management
- 添加 DocStatus 枚举和 DocProcessingStatus 类用于文档处理状态管理

- Implement JsonDocStatusStorage for persistent status storage
- 实现 JsonDocStatusStorage 用于持久化状态存储

- Add document-level deduplication in batch processing
- 在批处理中添加文档级别的去重功能

- Add checkpoint support in ainsert method for resumable document processing
- 在 ainsert 方法中添加断点续传支持,实现可恢复的文档处理

- Add status query methods for monitoring processing progress
- 添加状态查询方法用于监控处理进度

- Update LightRAG initialization to support document status tracking
- 更新 LightRAG 初始化以支持文档状态跟踪
2024-12-28 00:11:25 +08:00
LarFii
b7a2d336e6 Update __version__ 2024-12-13 20:15:49 +08:00
billvsme
a788c78197 keep tqdm_async work 2024-12-13 16:48:22 +08:00
billvsme
cf0278ca43 The asyncio.as_completed() function does not guarantee that the results are ordered 2024-12-13 15:42:41 +08:00
david
d0a4ef252e fix: rare embedding issue. 2024-12-10 09:00:22 +08:00
Larfii
8562ecdebc Add a progress bar 2024-11-25 15:04:38 +08:00
LarFii
b49f73181c update 2024-11-11 17:54:22 +08:00
LarFii
4c0352ee2b Add delete method 2024-11-11 17:48:40 +08:00
Ken Wiltshire
3d5d083f42 fix event loop conflict 2024-11-06 11:18:14 -05:00
Ken Wiltshire
f375620992 cleaning code for pull 2024-11-01 16:11:19 -04:00
Ken Wiltshire
e966a14418 set kg by start param, defaults to networkx 2024-11-01 08:47:52 -04:00
Ken Wiltshire
e4509327dd index to neo4j working and tested. check queires next. 2024-10-30 17:48:14 -04:00
Ken Wiltshire
01b7df7afa inference running locally. use neo4j next 2024-10-27 15:37:41 -04:00
Ken Wiltshire
cc45ea7310 use seperate module 2024-10-26 19:29:45 -04:00
Ken Wiltshire
43943e7168 adding neo4j 2024-10-26 05:57:56 -04:00
Ken Wiltshire
6e216abee6 adding neo4j 2024-10-25 17:45:59 -04:00
Ken Wiltshire
4ef705b13f adding neo4j integration 2024-10-25 11:28:41 -04:00
Sanketh Kumar
32464fab4e chore: added pre-commit-hooks and ruff formatting for commit-hooks 2024-10-19 09:43:17 +05:30
LarFii
3746eabf06 update 2024-10-10 15:02:30 +08:00
LarFii
130f7f391f Revert "first commit" 2024-10-10 15:00:31 +08:00
LarFii
0c2640f4e2 update 2024-10-10 14:58:21 +08:00
LarFii
74c2f04146 Revert "first commit" 2024-10-10 14:57:32 +08:00
Larfii
25df09a8ac first commit 2024-10-07 17:23:54 +08:00