yangdx
b45ae1567c
Refactor LLM cache handling and entity extraction
...
- Removed custom LLM function in entity extraction
- Simplified cache handling logic
- Added `force_llm_cache` parameter
- Updated cache handling conditions
2025-02-02 01:28:46 +08:00
yangdx
c9481c81b9
Add cache type "extract" for entity extraction
2025-02-01 23:05:02 +08:00
yangdx
2d387fa6de
Save keywords to cache only when it's no empty
2025-02-01 22:54:23 +08:00
yangdx
b22a8b216c
Fix linting
2025-02-01 10:36:25 +08:00
yangdx
2a010c985e
Add logging for chunk truncation in mix_kg_vector_query
2025-02-01 10:36:25 +08:00
yangdx
b0d87b2e29
Fix linting
2025-01-31 15:33:50 +08:00
yangdx
cc50ade14e
Fix concurrent problem on extract_entities function.
...
- Abandon the approach of temporarily replacing the global llm_model_func configuration
- Introduce custom_llm function with new_config for handle_cache while extracting entities
- Update handle_cache to accept custom_llm
2025-01-30 02:45:33 +08:00
yangdx
90c765c724
Fix linting
2025-01-29 22:14:18 +08:00
yangdx
4302c65a04
Avoid unnecessary LLM response generation
2025-01-29 21:01:35 +08:00
yangdx
c423f37f62
Lower log level for keyword outputs
2025-01-29 21:00:42 +08:00
MdNazishArmanShorthillsAI
f0b2024667
Query with your custom prompts
2025-01-27 10:32:22 +05:30
yangdx
86282ba434
Fix history_context handling error in kg_query
2025-01-25 22:54:12 +08:00
yangdx
06673b6fb2
Add logging for high and low-level keywords
2025-01-25 20:11:52 +08:00
yangdx
cf2f83377c
fix linting
2025-01-25 16:57:47 +08:00
yangdx
d1969515d3
Add conversation history to naive_query prompt
2025-01-25 16:29:18 +08:00
zrguo
13cc0e662f
Merge pull request #642 from dimatill/main
...
asyncio optimizations
2025-01-25 01:52:24 +08:00
Dmytro Til
f7b66d2c22
asyncio optimizations
2025-01-24 16:06:04 +01:00
Magic_yuan
8c121f50e9
支持多轮对话
2025-01-24 19:21:03 +08:00
Magic_yuan
5719aa8882
支持多轮对话
2025-01-24 19:03:36 +08:00
zrguo
b84aab5cd0
Merge pull request #590 from jin38324/main
...
Enhance Robustness of insert Method with Pipeline Processing and Caching Mechanisms
2025-01-16 14:20:08 +08:00
jin
6ae8647285
support pipeline mode
2025-01-16 12:58:15 +08:00
jin
d5ae6669ea
support pipeline mode
2025-01-16 12:52:37 +08:00
jin
17a2ec2bc4
Merge branch 'HKUDS:main' into main
2025-01-16 09:59:27 +08:00
Gurjot Singh
bc79f6650e
Fix linting errors
2025-01-14 22:23:14 +05:30
Gurjot Singh
ef61ffe444
Add custom function with separate keyword extraction for user's query and a separate prompt
2025-01-14 22:10:47 +05:30
jin
85331e3fa2
update Oracle support
...
add cache support, fix bug
2025-01-10 11:36:28 +08:00
adikalra
acde4ed173
Add custom chunking function.
2025-01-09 17:20:24 +05:30
zrguo
b93203804c
Merge branch 'main' into main
2025-01-09 15:28:57 +08:00
童石渊
dd213c95be
增加仅字符分割参数，如果开启，仅采用字符分割，不开启，在分割完以后如果chunk过大，会继续根据token size分割，更新测试文件
2025-01-09 11:55:49 +08:00
zrguo
6c78c96854
fix linting errors
2025-01-07 22:02:34 +08:00
zrguo
fe7f7086b1
Merge pull request #547 from n3A87/main
...
Fix:Optimized logic for automatic switching modes when keywords do not exist
2025-01-07 21:51:51 +08:00
童石渊
6b19401dc6
chunk split retry
2025-01-07 16:26:12 +08:00
童石渊
536d6f2283
添加字符分割功能，在“insert”函数中如果增加参数split_by_character，则会按照split_by_character进行字符分割，此时如果每个分割后的chunk的tokens大于max_token_size，则会继续按token_size分割（todo：考虑字符分割后过短的chunk处理）
2025-01-07 00:28:15 +08:00
xYLiuuuuuu
79646fced8
Fix:Optimized logic for automatic switching modes when keywords do not exist
2025-01-06 16:54:53 +08:00
Samuel Chan
6ae27d8f06
Some enhancements:
...
- Enable the llm_cache storage to support get_by_mode_and_id, to improve the performance for using real KV server
- Provide an option for the developers to cache the LLM response when extracting entities for a document. Solving the paint point that sometimes the process failed, the processed chunks we need to call LLM again, money and time wasted. With the new option (by default not enabled) enabling, we can cache that result, can significantly save the time and money for beginners.
2025-01-06 12:50:05 +08:00
Magic_yuan
7b91dc7fd8
feat: 增强知识图谱关系的时序性支持
...
- 为关系和向量数据增加时间戳支持,记录知识获取的时间
- 优化混合查询策略,同时考虑语义相关性和时间顺序
- 增强提示词模板,指导LLM在处理冲突信息时考虑时间因素
2024-12-29 15:37:34 +08:00
Magic_yuan
4c950cf4ce
feat: 增强知识图谱关系的时序性支持
...
- 为关系和向量数据增加时间戳支持,记录知识获取的时间
- 优化混合查询策略,同时考虑语义相关性和时间顺序
- 增强提示词模板,指导LLM在处理冲突信息时考虑时间因素
2024-12-29 15:25:57 +08:00
Magic_yuan
aaaf617451
feat(lightrag): Implement mix search mode combining knowledge graph and vector retrieval
...
- Add 'mix' mode to QueryParam for hybrid search functionality
- Implement mix_kg_vector_query to combine knowledge graph and vector search results
- Update LightRAG class to handle 'mix' mode queries
- Enhance README with examples and explanations for the new mix search mode
- Introduce new prompt structure for generating responses based on combined search results
2024-12-28 11:56:28 +08:00
zrguo
b7552f35aa
Merge pull request #461 from tjyiiuan/main
...
fix: update operate.py
2024-12-13 15:10:53 +08:00
Jiyu Tian
aac26b086e
fix: update operate.py
...
1. 避免变量在赋值之前就被引用
2. 解决未找到entity返回None导致的unpack问题
2024-12-12 15:47:57 -05:00
chenzihong
e9107a67c3
fix: fix variable name(entitiy->entity)
2024-12-12 23:59:40 +08:00
Magic_yuan
b89041b5b3
feat(operate): 添加实体类型配置并优化提示生成
...
- 在全局配置中添加 entity_types 参数，用于自定义实体类型
- 在生成实体提取和关系提取的提示时，使用配置的实体类型替代默认值
- 优化了提示生成逻辑，提高了代码的可配置性和灵活性
2024-12-11 13:53:05 +08:00
Magic_yuan
316c4df949
更新日志描述
2024-12-10 14:15:43 +08:00
Magic_yuan
58c0f94346
fix(lightrag): 修复只有实体没有关系的chunk处理逻辑
...
- 只有实体没有关系时，继续处理，而不是直接return
- 当只有实体而没有关系的图片在高阶查询关系时会返回空，这里优化返回，当没有关系时降级为local查询
2024-12-10 14:13:11 +08:00
Larfii
2ba20910bb
fix naive_query
2024-12-09 17:45:01 +08:00
zrguo
71af34196f
Merge branch 'main' into fix-entity-name-string
2024-12-09 17:30:40 +08:00
Magic_yuan
865e76a083
修复bug
...
https://github.com/HKUDS/LightRAG/issues/306
主要修改包括：
在存储文本块数据时增加了验证，确保只存储有效的数据
在处理文本块之前增加了空列表检查
在截断文本块之前过滤掉无效的数据
增加了更多的日志警告信息
查询的修改：
添加了对 chunks 的有效性检查,过滤掉无效的 chunks:
2024-12-09 15:08:30 +08:00
Magic_yuan
ccf44dc334
feat(cache): 增加 LLM 相似性检查功能并优化缓存机制
...
- 在 embedding 缓存配置中添加 use_llm_check 参数
- 实现 LLM 相似性检查逻辑，作为缓存命中的二次验证- 优化 naive 模式的缓存处理流程
- 调整缓存数据结构，移除不必要的 model 字段
2024-12-08 17:35:52 +08:00
Saujanya Verma
5a33ce1c1a
Fix: Ensure entity_or_relation_name is a string in _handle_entity_relation_summary
2024-12-06 20:54:01 +05:30
magicyuan876
8924d2b8fc
Merge remote-tracking branch 'origin/main'
...
# Conflicts:
# lightrag/llm.py
# lightrag/operate.py
2024-12-06 15:06:00 +08:00
