1051 Commits

Author SHA1 Message Date
Thibo Rosemplatt
f5938f76bc Azure OpenAI requires import of OpenAILLMOptions (missing) 2025-08-24 00:28:49 +02:00
yangdx
49ea9a79a7 Update rerank doc in README 2025-08-23 23:06:10 +08:00
yangdx
3d5e6226a9 Refactored rerank_example file to utilize the updated rerank function. 2025-08-23 22:51:41 +08:00
yangdx
1be9a54c8d Rename ENABLE_RERANK to RERANK_BY_DEFAULT and update default to true 2025-08-23 09:46:51 +08:00
yangdx
7f404bbecb Update webui assets and bump api version to 0207 2025-08-23 02:05:09 +08:00
yangdx
47485b130d refac(ui): Show rerank binding info on status card
- Remove separate ENABLE_RERANK flag in favor of rerank_binding="null"
- Change default rerank binding from "cohere" to "null" (disabled)
- Update UI to display both rerank binding and model information
2025-08-23 02:04:14 +08:00
yangdx
bf43e1b8c1 fix: Resolve default rerank config problem when env var missing
- Read config from selected_rerank_func when env var missing
- Make api_key optional for rerank function
- Add response format validation with proper error handling
- Update Cohere rerank default to official API endpoint
2025-08-23 01:07:59 +08:00
yangdx
580cb7906c feat: Add multiple rerank provider support to LightRAG Server by adding new env vars and cli params
- Add --enable-rerank CLI argument and ENABLE_RERANK env var
- Simplify rerank configuration logic to only check enable flag and binding
- Update health endpoint to show enable_rerank and rerank_configured status
- Improve logging messages for rerank enable/disable states
- Maintain backward compatibility with default value True
2025-08-22 19:29:45 +08:00
yangdx
105fb43a54 Updat webui assets and bump api version to 0206 2025-08-21 22:56:44 +08:00
yangdx
8c6b5f4a3a Update README 2025-08-21 18:14:27 +08:00
yangdx
62cdc7d7eb Update documentation with LLM selection guidelines and API improvements 2025-08-21 13:59:14 +08:00
yangdx
0e67ead8fa Rename MAX_TOKENS to SUMMARY_MAX_TOKENS for clarity 2025-08-21 10:15:20 +08:00
yangdx
aa22772721 Refactor LLM temperature handling to be provider-specific
• Remove global temperature parameter
• Add provider-specific temp configs
• Update env example with new settings
• Fix Bedrock temperature handling
• Clean up splash screen display
2025-08-20 23:52:33 +08:00
yangdx
df7bcb1e3d Add LLM_TIMEOUT configuration for all LLM providers
- Add LLM_TIMEOUT env variable
- Apply timeout to all LLM bindings
2025-08-20 23:50:57 +08:00
yangdx
4c556d8aae Set default TIMEOUT value to 150, and gunicorn timeout to TIMEOUT+30 2025-08-20 22:04:32 +08:00
yangdx
9b7ed84e05 Improve document deletion error handling and message consistency
- Standardize deletion log messages
- Add try-catch for file operations
- Improve enqueued file error handling
2025-08-20 11:01:24 +08:00
yangdx
2603e99005 Enhance file deletion to remove files from both input and enqueued dirs 2025-08-19 17:13:58 +08:00
yangdx
1f86543772 Update i18n translation and webui assets 2025-08-19 16:23:05 +08:00
yangdx
0aa1bc8bf9 Update webui assets and bump api version to 0205 2025-08-19 15:11:34 +08:00
yangdx
9ed5b93467 Add [File Extraction] prefix to error messages and logs 2025-08-19 11:33:28 +08:00
yangdx
368d2b00d6 Update webui assets and bump api version to 0204 2025-08-18 19:33:46 +08:00
yangdx
377f1a022e fix: reset PROCESSING/FAILED docs to PENDING at the beginging of document processing pipeline
- Reset documents with PROCESSING/FAILED status to PENDING when they pass consistency checks
- Update doc_status storage and clear error messages/metadata on reset
2025-08-18 00:49:52 +08:00
yangdx
add8b07a21 Improve logging messages for document processing clarity 2025-08-18 00:22:04 +08:00
yangdx
14e083a1a6 fix: replace pyuca with pypinyin for Chinese pinyin sorting and add file_path sort 2025-08-17 15:21:24 +08:00
yangdx
61469c0a56 Add Chinese pinyin sorting support across document operations
• Replace pyuca with centralized utils function
• Add pinyin sort keys for file paths
• Update MongoDB indexes with zh collation
• Migrate existing indexes for compatibility
• Support Chinese chars in Redis/JSON storage
• Keep PostgreSQL sorting order controled by Database Collate order
2025-08-17 12:45:48 +08:00
yangdx
6196bab00a Update webui assets and bump api version to 0203 2025-08-17 10:39:16 +08:00
yangdx
3a7310873c Merge branch 'bedrock-support' 2025-08-17 02:23:44 +08:00
yangdx
da7e4b79e5 Update documentation in README files 2025-08-17 02:23:14 +08:00
yangdx
1ed77a2e53 Remove openai-ollama binding from LightRAG level args 2025-08-17 02:13:50 +08:00
yangdx
301acfc274 Update webui assets 2025-08-17 01:54:39 +08:00
yangdx
45365ff6ef Bump api version to 0202 2025-08-16 23:53:01 +08:00
yangdx
cceb46b320 fix: subdirectories are no longer processed during file scans
• Change rglob to glob for file scanning
• Simplify error logging messages
2025-08-16 23:46:33 +08:00
yangdx
f5b0c3d38c feat: Recording file extraction error status to document pipeline
- Add apipeline_enqueue_error_documents function to LightRAG class for recording file processing errors in doc_status storage
- Enhance pipeline_enqueue_file with detailed error handling for all file processing stages:
  * File access errors (permissions, not found)
  * UTF-8 encoding errors
  * Format-specific processing errors (PDF, DOCX, PPTX, XLSX)
  * Content validation errors
  * Unsupported file type errors

This implementation ensures all file extraction failures are properly tracked and recorded in the doc_status storage system, providing better visibility into document processing issues and enabling improved error monitoring and debugging capabilities.
2025-08-16 23:08:52 +08:00
yangdx
5d00c4c7a8 feat: move processed files to __enqueued__ directory after processing with filename conflicts handling 2025-08-16 13:19:20 +08:00
SJ
f7ca9ae16a Ruff formatted 2025-08-15 22:21:34 +00:00
yangdx
dc7a6e1c5b Update README 2025-08-16 06:15:27 +08:00
SJ
3aa3332505
Merge pull request #1 from HKUDS/main
merge
2025-08-15 17:09:03 -05:00
yangdx
3bba5fc506 Fix linting 2025-08-14 13:03:23 +08:00
yangdx
772f981e7e fix: check and process queued docs even when upload directory is empty 2025-08-14 12:35:39 +08:00
yangdx
fd0ae4646f Fixes crash when processing files with UTF-8 encoding error
- Fix TypeError "cannot unpack non-iterable bool object" in document processing
- Change all error returns from `False` to `(False, "")` for consistency
- Ensure pipeline_enqueue_file always returns tuple (bool, str)
- Add missing return statement for no-content-extracted case
- Improve error handling for UTF-8 encoding issues and unsupported file types
2025-08-14 05:31:38 +08:00
yangdx
3ccd10f1e4 Update webui assets 2025-08-14 05:03:43 +08:00
yangdx
160a40dc04 Bump api version to 0201 2025-08-14 05:02:20 +08:00
yangdx
ae517181ad Bump api version to 0200 2025-08-14 05:01:13 +08:00
SJ
99643f01de
Enhancement: support aws bedrock as an LLm binding #1733 2025-08-13 02:08:13 -05:00
yangdx
2dab4e321d Bump api version to 0199 2025-08-06 01:03:35 +08:00
yangdx
c22315ea6d refactor: remove selective LLM cache clearing functionality
- Remove optional 'modes' parameter from aclear_cache() and clear_cache() methods
- Replace deprecated drop_cache_by_modes() with drop() method for complete cache clearing
- Update API endpoint to ignore mode-specific parameters and clear all cache
- Simplify frontend clearCache() function to send empty request body

This change ensures all LLM cache is cleared together.
2025-08-05 23:51:51 +08:00
yangdx
0b5c708660 Update storage implementation documentation
- Add detailed storage type descriptions
- Remove Chroma from vector storage options
- Include recommended PostgreSQL version
- Add Memgraph to graph storage options
- Update performance comparison notes
2025-08-05 18:03:51 +08:00
yangdx
4d492abf41 feat: implement temperature priority cascade for LLM bindings
- Add global --temperature command line argument with env fallback
- Implement temperature priority for Ollama LLM binding:
  1. --ollama-llm-temperature (highest)
  2. OLLAMA_LLM_TEMPERATURE env var
  3. --temperature command arg
  4. TEMPERATURE env var (lowest)
- Implement same priority logic for OpenAI/Azure OpenAI LLM binding
- Ensure command line args always override environment variables
- Maintain backward compatibility with existing configurations
2025-08-05 04:53:55 +08:00
yangdx
adf7ec8e35 feat: Add OpenAI LLM Options support with BindingOptions framework
- Add OpenAILLMOptions dataclass with full OpenAI API parameter support
- Integrate OpenAI options in config.py for automatic binding detection
- Update server functions to inject OpenAI options for openai/azure_openai bindings
2025-08-05 03:47:26 +08:00
yangdx
3099748668 Add temperature fallback for Ollama LLM binding
- Implement OLLAMA_LLM_TEMPERATURE env var
- Fallback to global TEMPERATURE if unset
- Remove redundant OllamaLLMOptions logic
- Update env.example with new setting
2025-08-05 01:50:09 +08:00