yujunjun/LightRAG
1
0
Fork 0
mirror of https://github.com/HKUDS/LightRAG.git synced 2025-11-16 01:55:08 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,753 Commits 5 Branches 72 Tags
Commit Graph

4753 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
yangdx
99e28e815b fix: prevent document processing failures from UTF-8 surrogate characters 
- Change sanitize_text_for_encoding to fail-fast instead of returning error placeholders
- Add strict UTF-8 cleaning pipeline to entity/relationship extraction
- Skip problematic entities/relationships instead of corrupting data

Fixes document processing crashes when encountering surrogate characters (U+D800-U+DFFF)
 2025-08-27 23:52:39 +08:00
yangdx
4dfbe5e2db Rename workflow and remove latest tag from Docker build 
• Rename docker-build-main to manual
• Remove latest tag from metadata
 2025-08-27 15:14:23 +08:00
yangdx
6a2a592224 Fix linting 2025-08-27 12:51:50 +08:00
yangdx
8a0d06e557 Restore default entity types 2025-08-27 12:51:18 +08:00
yangdx
28e07c89f9 Fix linting 2025-08-27 12:35:51 +08:00
yangdx
2ccc39de9a Fix language fallback in summarize error 2025-08-27 12:34:27 +08:00
yangdx
0be4f0144b Merge branch 'entityTypesServerSupport' 2025-08-27 12:23:58 +08:00
yangdx
ff0a18e08c Unify SUMMARY_LANGUANGE and ENTITY_TYPES implementation method 2025-08-27 12:23:22 +08:00
Daniel.y
4edb0011b9
Merge pull request #2009 from LinkinPony/main 
fix mismatch of 'error' and 'error_msg' in MongoDB
 2025-08-27 11:45:45 +08:00
yangdx
194f46f239 Add json_repair dependency to project requirements v1.4.7rc2 2025-08-27 11:14:09 +08:00
yangdx
cb0a035076 Update env.example 2025-08-27 11:12:52 +08:00
LinkinPony
45da0385eb
Merge branch 'HKUDS:main' into main 2025-08-27 09:22:39 +08:00
Thibo Rosemplatt
c3aabfc251 Merge branch 'main' into entityTypesServerSupport 2025-08-26 21:48:20 +02:00
Daniel.y
c975263fbc
Merge pull request #2013 from danielaskdd/fix-file-order 
fix(webui): resolve document status grouping issue in DocumentManager
 2025-08-26 23:57:21 +08:00
Daniel.y
82f72521f5
Merge pull request #2006 from danielaskdd/optimize-merge-stage 
refac: Refactor LLM Summary Generation Algorithm
 2025-08-26 23:56:56 +08:00
yangdx
c259b8f22c Update webui assets and bump aip verion to 0208 2025-08-26 23:05:00 +08:00
yangdx
7db788aa66 fix(webui): resolve document status grouping issue in DocumentManager 
- Fix documents being grouped by status after pagination and sorting
- Use backend-sorted data directly from currentPageDocs instead of re-grouping
- Preserve backend sort order to prevent status-based grouping
- Maintain backward compatibility with legacy docs structure
- Ensure all sorting fields (file name, dates, ID) work correctly without status grouping

The issue occurred because the frontend was re-grouping already-sorted data
from the backend by status, breaking the intended sort order. Now documents
are displayed in the exact order returned by the backend API.

Fixes: Document list sorting by file name was grouping by status instead of
maintaining proper sort order across all documents.
 2025-08-26 23:03:41 +08:00
yangdx
d3623cc9ae fix: resolve infinite loop risk in _handle_entity_relation_summary 
- Ensure oversized descriptions are force-merged with subsequent ones
- Add len(current_list) <= 2 termination condition to guarantee convergence
- Implement token-based truncation in _summarize_descriptions to prevent overflow
 2025-08-26 21:58:31 +08:00
yangdx
e0a755e42c Refactor prompt instructions to emphasize depth and completeness 2025-08-26 18:28:57 +08:00
yangdx
79e0226b2b Refactor: move force_llm_summary_on_merge to global_config access 
- Remove parameter from function signature
- Access from global_config instead
- Improve code consistency
 2025-08-26 18:02:39 +08:00
yangdx
01a2c79f29 Standardize prompt formatting and section headers across templates 
- Remove hash delimiters
- Consistent section headers
- Add "Output:" labels
- Clean up example formatting
 2025-08-26 14:42:52 +08:00
yangdx
6bcfe696ee feat: add output length recommendation and description type to LLM summary 
- Add SUMMARY_LENGTH_RECOMMENDED parameter (600 tokens)
- Optimize prompt temple for LLM summary
 2025-08-26 14:41:12 +08:00
LinkinPony
ff4c747a2a fix mismatch of 'error' and 'error_msg' in MongoDB 2025-08-26 10:43:56 +08:00
yangdx
025f70089a Simplify status messages in knowledge rebuild operations 2025-08-26 04:26:15 +08:00
yangdx
84416d104d Increase default LLM summary merge threshold from 4 to 8 for reducing summary trigger frequency 2025-08-26 03:57:35 +08:00
yangdx
9eb2be79b8 feat: track actual LLM usage in entity/relation merging 
- Modified _handle_entity_relation_summary to return tuple[str, bool]
- Updated merge functions to log "LLMmerg" vs "Merging" based on actual LLM usage
- Replaced hardcoded fragment count prediction with real-time LLM usage tracking
 2025-08-26 03:56:18 +08:00
yangdx
cb0fe38b9a Fix linting 2025-08-26 02:22:34 +08:00
yangdx
de2daf6565 refac: Rename summary_max_tokens to summary_context_size, comprehensive parameter validation for summary configuration 
- Update algorithm logic in operate.py for better token management
- Fix health endpoint to use correct parameter names
 2025-08-26 01:35:50 +08:00
yangdx
91767ffcee Improve warning message formatting in entity/relationship rebuild 2025-08-25 21:55:29 +08:00
yangdx
15cdd0dd8f fix: Sort cached extraction results by the create_time within each chunk 
This ensures the KG rebuilds maintain the original creation order of the first extraction result for each chunk.
 2025-08-25 21:41:33 +08:00
yangdx
882d6857d8 feat: Implement map-reduce summarization to handle large humber of description merging 2025-08-25 21:03:16 +08:00
yangdx
0b1b264a5d refactor: optimize graph lock scope in document deletion 
- Move dependency analysis outside graph database lock
- Add persistence call before lock release to prevent dirty reads
 2025-08-25 17:46:32 +08:00
yangdx
cac8e189e7 Remove redundant entity vector deletion before upsert 2025-08-25 17:18:51 +08:00
yangdx
9b6de7512d Optimize the stability of description merging order 2025-08-25 17:10:51 +08:00
yangdx
31f4f96944 Exclude conversation history from context length calculation 2025-08-25 12:43:34 +08:00
yangdx
f688e95f56 Add warning for vector chunks missing chunk_id 2025-08-25 12:42:25 +08:00
yangdx
b6aedba7ae Add logging for empty naive query results in vector context 2025-08-25 12:21:31 +08:00
yangdx
f1ff5cf93f fix: initialize truncated_chunks variable in _build_query_context 
Prevents local variable 'truncated_chunks'referenced before assignment
 2025-08-25 11:56:56 +08:00
yangdx
b5682b15cb Remove json-repair from core deps, add missing api deps 2025-08-25 07:23:41 +08:00
Daniel.y
2adbafd6b0
Merge pull request #1999 from thiborose/missingDependencyAOAI 
Added import of OpenAILLMOptions when using azure_openai
 2025-08-24 10:38:09 +08:00
Thibo Rosemplatt
f5938f76bc Azure OpenAI requires import of OpenAILLMOptions (missing) 2025-08-24 00:28:49 +02:00
Thibo Rosemplatt
d054ec5d00 Added entity_types as a user defined variable (via .env) 2025-08-23 20:16:11 +02:00
yangdx
4e79550725 Merge remote-tracking branch 'origin/main' 2025-08-24 01:08:48 +08:00
yangdx
540a83ea1c Add --use-pep517 flag to all pip install commands in Dockerfile 2025-08-24 01:07:23 +08:00
yangdx
eebc8938ed Update action name 2025-08-24 00:02:57 +08:00
yangdx
a82f126418 Rename github action name 2025-08-23 23:58:33 +08:00
yangdx
8b1e54c8b4 Update Docker workflow name to clarify release trigger 
• Clarified workflow purpose
• Added "on Release" to name
 2025-08-23 23:55:27 +08:00
yangdx
9a66c944e2 Add Docker build workflow for main branch with manual trigger 
- Manual workflow dispatch trigger
- Multi-platform build support
- GHCR registry integration
- Git tag-based versioning
- GitHub Actions cache optimization
 2025-08-23 23:50:06 +08:00
yangdx
b815e47f7c Upgrade Python to 3.12 and update pip/setuptools in Dockerfile 
• Upgrade base image to Python 3.12
• Update pip, setuptools, wheel
 2025-08-23 23:38:39 +08:00
yangdx
bac946fd98 Merge branch 'fix-rerank-example' 2025-08-23 23:07:38 +08:00