yujunjun/LightRAG
1
0
Fork 0
mirror of https://github.com/HKUDS/LightRAG.git synced 2025-11-22 13:06:10 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,753 Commits 5 Branches 72 Tags
Commit Graph

3070 Commits

Author SHA1 Message Date
yangdx
99e28e815b fix: prevent document processing failures from UTF-8 surrogate characters 
- Change sanitize_text_for_encoding to fail-fast instead of returning error placeholders
- Add strict UTF-8 cleaning pipeline to entity/relationship extraction
- Skip problematic entities/relationships instead of corrupting data

Fixes document processing crashes when encountering surrogate characters (U+D800-U+DFFF)
 2025-08-27 23:52:39 +08:00
yangdx
6a2a592224 Fix linting 2025-08-27 12:51:50 +08:00
yangdx
8a0d06e557 Restore default entity types 2025-08-27 12:51:18 +08:00
yangdx
28e07c89f9 Fix linting 2025-08-27 12:35:51 +08:00
yangdx
2ccc39de9a Fix language fallback in summarize error 2025-08-27 12:34:27 +08:00
yangdx
0be4f0144b Merge branch 'entityTypesServerSupport' 2025-08-27 12:23:58 +08:00
yangdx
ff0a18e08c Unify SUMMARY_LANGUANGE and ENTITY_TYPES implementation method 2025-08-27 12:23:22 +08:00
LinkinPony
45da0385eb
Merge branch 'HKUDS:main' into main 2025-08-27 09:22:39 +08:00
Thibo Rosemplatt
c3aabfc251 Merge branch 'main' into entityTypesServerSupport 2025-08-26 21:48:20 +02:00
yangdx
c259b8f22c Update webui assets and bump aip verion to 0208 2025-08-26 23:05:00 +08:00
yangdx
d3623cc9ae fix: resolve infinite loop risk in _handle_entity_relation_summary 
- Ensure oversized descriptions are force-merged with subsequent ones
- Add len(current_list) <= 2 termination condition to guarantee convergence
- Implement token-based truncation in _summarize_descriptions to prevent overflow
 2025-08-26 21:58:31 +08:00
yangdx
e0a755e42c Refactor prompt instructions to emphasize depth and completeness 2025-08-26 18:28:57 +08:00
yangdx
79e0226b2b Refactor: move force_llm_summary_on_merge to global_config access 
- Remove parameter from function signature
- Access from global_config instead
- Improve code consistency
 2025-08-26 18:02:39 +08:00
yangdx
01a2c79f29 Standardize prompt formatting and section headers across templates 
- Remove hash delimiters
- Consistent section headers
- Add "Output:" labels
- Clean up example formatting
 2025-08-26 14:42:52 +08:00
yangdx
6bcfe696ee feat: add output length recommendation and description type to LLM summary 
- Add SUMMARY_LENGTH_RECOMMENDED parameter (600 tokens)
- Optimize prompt temple for LLM summary
 2025-08-26 14:41:12 +08:00
LinkinPony
ff4c747a2a fix mismatch of 'error' and 'error_msg' in MongoDB 2025-08-26 10:43:56 +08:00
yangdx
025f70089a Simplify status messages in knowledge rebuild operations 2025-08-26 04:26:15 +08:00
yangdx
84416d104d Increase default LLM summary merge threshold from 4 to 8 for reducing summary trigger frequency 2025-08-26 03:57:35 +08:00
yangdx
9eb2be79b8 feat: track actual LLM usage in entity/relation merging 
- Modified _handle_entity_relation_summary to return tuple[str, bool]
- Updated merge functions to log "LLMmerg" vs "Merging" based on actual LLM usage
- Replaced hardcoded fragment count prediction with real-time LLM usage tracking
 2025-08-26 03:56:18 +08:00
yangdx
cb0fe38b9a Fix linting 2025-08-26 02:22:34 +08:00
yangdx
de2daf6565 refac: Rename summary_max_tokens to summary_context_size, comprehensive parameter validation for summary configuration 
- Update algorithm logic in operate.py for better token management
- Fix health endpoint to use correct parameter names
 2025-08-26 01:35:50 +08:00
yangdx
91767ffcee Improve warning message formatting in entity/relationship rebuild 2025-08-25 21:55:29 +08:00
yangdx
15cdd0dd8f fix: Sort cached extraction results by the create_time within each chunk 
This ensures the KG rebuilds maintain the original creation order of the first extraction result for each chunk.
 2025-08-25 21:41:33 +08:00
yangdx
882d6857d8 feat: Implement map-reduce summarization to handle large humber of description merging 2025-08-25 21:03:16 +08:00
yangdx
0b1b264a5d refactor: optimize graph lock scope in document deletion 
- Move dependency analysis outside graph database lock
- Add persistence call before lock release to prevent dirty reads
 2025-08-25 17:46:32 +08:00
yangdx
cac8e189e7 Remove redundant entity vector deletion before upsert 2025-08-25 17:18:51 +08:00
yangdx
9b6de7512d Optimize the stability of description merging order 2025-08-25 17:10:51 +08:00
yangdx
31f4f96944 Exclude conversation history from context length calculation 2025-08-25 12:43:34 +08:00
yangdx
f688e95f56 Add warning for vector chunks missing chunk_id 2025-08-25 12:42:25 +08:00
yangdx
b6aedba7ae Add logging for empty naive query results in vector context 2025-08-25 12:21:31 +08:00
yangdx
f1ff5cf93f fix: initialize truncated_chunks variable in _build_query_context 
Prevents local variable 'truncated_chunks'referenced before assignment
 2025-08-25 11:56:56 +08:00
Thibo Rosemplatt
f5938f76bc Azure OpenAI requires import of OpenAILLMOptions (missing) 2025-08-24 00:28:49 +02:00
Thibo Rosemplatt
d054ec5d00 Added entity_types as a user defined variable (via .env) 2025-08-23 20:16:11 +02:00
yangdx
49ea9a79a7 Update rerank doc in README 2025-08-23 23:06:10 +08:00
yangdx
3d5e6226a9 Refactored rerank_example file to utilize the updated rerank function. 2025-08-23 22:51:41 +08:00
yangdx
9bc349ddd6 Improve Empty Keyword Handling logic 2025-08-23 11:50:58 +08:00
yangdx
1be9a54c8d Rename ENABLE_RERANK to RERANK_BY_DEFAULT and update default to true 2025-08-23 09:46:51 +08:00
yangdx
d29272e6a5 Merge branch 'add-diagnostic-tool' 2025-08-23 02:39:28 +08:00
yangdx
8a293a2c07 Fix linting 2025-08-23 02:39:12 +08:00
yangdx
4595e4b82e Merge branch 'improve-initialization-error-messages' 2025-08-23 02:34:59 +08:00
yangdx
059003c906 Rename allow_create to first_initialization for clarity 2025-08-23 02:34:39 +08:00
yangdx
7f404bbecb Update webui assets and bump api version to 0207 2025-08-23 02:05:09 +08:00
yangdx
47485b130d refac(ui): Show rerank binding info on status card 
- Remove separate ENABLE_RERANK flag in favor of rerank_binding="null"
- Change default rerank binding from "cohere" to "null" (disabled)
- Update UI to display both rerank binding and model information
 2025-08-23 02:04:14 +08:00
yangdx
bf43e1b8c1 fix: Resolve default rerank config problem when env var missing 
- Read config from selected_rerank_func when env var missing
- Make api_key optional for rerank function
- Add response format validation with proper error handling
- Update Cohere rerank default to official API endpoint
 2025-08-23 01:07:59 +08:00
yangdx
580cb7906c feat: Add multiple rerank provider support to LightRAG Server by adding new env vars and cli params 
- Add --enable-rerank CLI argument and ENABLE_RERANK env var
- Simplify rerank configuration logic to only check enable flag and binding
- Update health endpoint to show enable_rerank and rerank_configured status
- Improve logging messages for rerank enable/disable states
- Maintain backward compatibility with default value True
 2025-08-22 19:29:45 +08:00
Albert Gil López
3fca3be09b fix: Fix server startup issue with PipelineNotInitializedError 
- Add allow_create parameter to get_namespace_data() to permit internal initialization
- initialize_pipeline_status() now uses allow_create=True to create the namespace
- External calls still get the error if pipeline_status is not initialized
- This maintains the improved error messages while allowing proper server startup

Fixes server startup failure reported in PR #1978
 2025-08-22 10:55:56 +00:00
Albert Gil López
c66fc3483a fix: Implement PipelineNotInitializedError usage in get_namespace_data 
- Add PipelineNotInitializedError import to shared_storage.py
- Raise PipelineNotInitializedError when accessing uninitialized pipeline_status namespace
- This provides clear error messages to users about initialization requirements
- Other namespaces continue to be created dynamically as before

Addresses review feedback from PR #1978 about unused exception class
 2025-08-22 02:52:51 +00:00
Albert Gil López
3a64b267cb Merge upstream/main and resolve conflicts 2025-08-21 16:56:11 +00:00
Albert Gil López
ba7b3cedb5 Merge remote-tracking branch 'upstream/main' into add-diagnostic-tool 2025-08-21 16:52:09 +00:00
yangdx
16a1ef1178 Update summary_max_tokens default from 10k to 30k tokens 2025-08-21 23:16:07 +08:00