40 Commits

Author SHA1 Message Date
Saifeddine ALOUI
16d1ae77ee
fixed csv_string_to_list when data contains null 2025-01-27 10:15:30 +01:00
Saifeddine ALOUI
315f0bf5f9
Added escaping to list_of_list_to_csv 2025-01-27 10:13:06 +01:00
Saifeddine ALOUI
06c9e4e454 Fixed missing imports bug and fixed linting 2025-01-25 00:55:07 +01:00
Saifeddine ALOUI
34018cb1e0 Separated llms from the main llm.py file and fixed some deprication bugs 2025-01-25 00:11:00 +01:00
Magic_yuan
5719aa8882 支持多轮对话 2025-01-24 19:03:36 +08:00
Magic_yuan
f6d29e1793 修复缓存处理的运行时错误
].func  # ["func"]
 ^^^^
AttributeError: 'dict' object has no attribute 'func'
2025-01-24 11:01:32 +08:00
jin
6ae8647285 support pipeline mode 2025-01-16 12:58:15 +08:00
jin
d5ae6669ea support pipeline mode 2025-01-16 12:52:37 +08:00
jin
85331e3fa2 update Oracle support
add cache support, fix bug
2025-01-10 11:36:28 +08:00
Samuel Chan
6ae27d8f06 Some enhancements:
- Enable the llm_cache storage to support get_by_mode_and_id, to improve the performance for using real KV server
- Provide an option for the developers to cache the LLM response when extracting entities for a document. Solving the paint point that sometimes the process failed, the processed chunks we need to call LLM again, money and time wasted. With the new option (by default not enabled) enabling, we can cache that result, can significantly save the time and money for beginners.
2025-01-06 12:50:05 +08:00
zrguo
a1493f56d6
Update utils.py 2024-12-26 22:09:46 +08:00
GG
04293466ad fix(utils): 修复缓存未正确启用的问题
- 在 handle_cache 函数中添加了对 global_config 中 enable_l`lm_cache 设置的检查
- 如果配置禁用了缓存,则直接返回 None
-这个修改确保了在不需要缓存的情况下,函数能够正确地跳过缓存处理
2024-12-26 22:03:45 +08:00
david
f6eeedb050 add concurrent embedding limit 2024-12-10 09:01:21 +08:00
david
3210c8f5bd fix unicode_escape. 2024-12-09 19:14:27 +08:00
partoneplay
a7fcb653e3 Merge remote-tracking branch 'origin/main' and fix syntax 2024-12-09 12:36:55 +08:00
Magic_yuan
ccf44dc334 feat(cache): 增加 LLM 相似性检查功能并优化缓存机制
- 在 embedding 缓存配置中添加 use_llm_check 参数
- 实现 LLM 相似性检查逻辑,作为缓存命中的二次验证- 优化 naive 模式的缓存处理流程
- 调整缓存数据结构,移除不必要的 model 字段
2024-12-08 17:35:52 +08:00
Magic_yuan
dc2a45004a 缓存计算函数迁移到工具类 2024-12-08 10:37:55 +08:00
magicyuan876
786aa92b48 fix(utils): 修复 JSON 日志编码问题
- 在 json.dumps 中添加 ensure_ascii=False 参数,以支持非 ASCII 字符编码
-这个修改确保了包含中文等非 ASCII 字符的日志信息能够正确处理和显示
2024-12-06 14:32:41 +08:00
magicyuan876
e619b09c8a 重构缓存处理逻辑
- 提取通用缓存处理逻辑到新函数 handle_cache 和 save_to_cache
- 使用 CacheData 类统一缓存数据结构
- 优化嵌入式缓存和常规缓存的处理流程
- 添加模式参数以支持不同查询模式的缓存策略
- 重构 get_best_cached_response 函数,提高缓存查询效率
2024-12-06 14:29:16 +08:00
magicyuan876
2ecdab2f18 整理了一下手误的注释 2024-12-06 09:11:22 +08:00
magicyuan876
d48c6e4588 feat(lightrag): 添加 查询时使用embedding缓存功能
- 在 LightRAG 类中添加 embedding_cache_config配置项
- 实现基于 embedding 相似度的缓存查询和存储
- 添加量化和反量化函数,用于压缩 embedding 数据
- 新增示例演示 embedding 缓存的使用
2024-12-06 08:17:20 +08:00
b10902118
753c1e6714 support JSON output for ollama and openai 2024-11-29 21:41:37 +08:00
jin
26ae240c65 Logic Optimization 2024-11-25 13:40:38 +08:00
jin
1dbe803521 Merge branch 'main' of https://github.com/jin38324/LightRAG 2024-11-25 13:32:33 +08:00
jin
89c2de54a2 Optimization logic 2024-11-25 13:29:55 +08:00
jin
662303f605 use oracle bind variables to avoid error 2024-11-15 12:57:01 +08:00
LarFii
f8b15df74c fix cache bug 2024-11-14 15:59:37 +08:00
Ken Wiltshire
3d5d083f42 fix event loop conflict 2024-11-06 11:18:14 -05:00
zrguo
3f7ae11962 Update utils.py 2024-11-03 17:53:53 +08:00
gogoswift
b91d3912cc Update utils.py 2024-10-31 14:31:26 +08:00
gogoswift
f0524598c8 Update utils.py
混合检索的合并函数
2024-10-31 11:34:01 +08:00
Sanketh Kumar
a157e8e0a2 Manually reformatted files 2024-10-25 13:32:25 +05:30
LarFii
bcd81e3a50 Add visualization methods 2024-10-20 23:08:26 +08:00
Sanketh Kumar
32464fab4e chore: added pre-commit-hooks and ruff formatting for commit-hooks 2024-10-19 09:43:17 +05:30
LarFii
b6ec084097 update utils.py 2024-10-11 11:24:42 +08:00
LarFii
3746eabf06 update 2024-10-10 15:02:30 +08:00
LarFii
130f7f391f Revert "first commit" 2024-10-10 15:00:31 +08:00
LarFii
0c2640f4e2 update 2024-10-10 14:58:21 +08:00
LarFii
74c2f04146 Revert "first commit" 2024-10-10 14:57:32 +08:00
Larfii
25df09a8ac first commit 2024-10-07 17:23:54 +08:00