ragflow

mirror of https://github.com/infiniflow/ragflow.git synced 2025-06-26 22:19:57 +00:00

History

Refa: GraphRAG and explaining GraphRAG stalling behavior on large files (#8223 )

### What problem does this PR solve?

This PR investigates the cause of #7957.

TL;DR: Incorrect similarity calculations lead to too many candidates.
Since candidate selection involves interaction with the LLM, this causes
significant delays in the program.

What this PR does:

1. **Fix similarity calculation**:
When processing a 64 pages government document, the corrected similarity
calculation reduces the number of candidates from over 100,000 to around
16,000. With a default batch size of 100 pairs per LLM call, this fix
reduces unnecessary LLM interactions from over 1,000 calls to around
160, a roughly 10x improvement.
2. **Add concurrency and timeout limits**: 
Up to 5 entity types are processed in "parallel", each with a 180-second
timeout. These limits may be configurable in future updates.
3. **Improve logging**:
The candidate resolution process now reports progress in real time.
4. **Mitigates potential concurrency risks**


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

2025-06-12 19:09:50 +08:00

general

Refa: GraphRAG and explaining GraphRAG stalling behavior on large files (#8223 )

2025-06-12 19:09:50 +08:00

light

Optimize graphrag again (#6513 )

2025-03-26 15:34:42 +08:00

__init__.py

Cache the result from llm for graphrag and raptor (#4051 )

2024-12-17 09:48:03 +08:00

entity_resolution_prompt.py

Add graphrag (#1793 )