1077 Commits

Author SHA1 Message Date
Stephen Hu
0fccd1fef3
Fix:in the knowledge base operation file will result in an error (#8962)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/8941
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-25 19:26:31 +08:00
Yongteng Lei
5cc570f5e0
Refa: suppress DB migration error logs (#9043)
### What problem does this PR solve?

Suppress DB migration error logs.

### Type of change

- [x] Refactoring
2025-07-25 12:38:07 +08:00
Liu An
b5ffca332a
Refa: validation utils to use Pydantic v2 style models (#9037)
### What problem does this PR solve?

- Update BaseModel to use model_config instead of Config class
- Replace StrEnum with Literal types for method fields
- Convert Field declarations to Annotated style

### Type of change

- [x] Refactoring
2025-07-25 12:16:45 +08:00
Stephen Hu
bcaac061ac
Feature: Support set chats kbs to empty (#9038)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/9034

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-07-25 10:06:33 +08:00
Gifford Nowland
34c35cf8ae
fix: obfuscate additional server secrets values (#9014)
### What problem does this PR solve?

Obfuscates additional secrets values on ragflow_server startup to
prevent leakage:
* `secret` (azure)
* `client_secret` (oauth)
* `http_secret_key` (authentication)
* `sas_token` (azure)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Gifford R Nowland <gifford.r.nowland@aero.org>
2025-07-24 10:16:23 +08:00
Yongteng Lei
8345e92671
Feat: OpenAI-compatible-API supports references (#8997)
### What problem does this PR solve?

OpenAI-compatible-API supports references.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-07-23 18:10:05 +08:00
Liu An
b4b6d296ea
Fix: Increase timeouts for document parsing and model checks (#8996)
### What problem does this PR solve?

- Extended embedding model timeout from 3 to 10 seconds in api_utils.py
- Added more time for large file batches and concurrent parsing
operations to prevent test flakiness
- Import from #8940
- https://github.com/infiniflow/ragflow/actions/runs/16422052652

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-23 15:08:36 +08:00
Liu An
0020c50000
Fix: Refactor parser config handling and add GraphRAG defaults (#8778)
### What problem does this PR solve?

- Update `get_parser_config` to merge provided configs with defaults
- Add GraphRAG configuration defaults for all chunk methods
- Make raptor and graphrag fields non-nullable in ParserConfig schema
- Update related test cases to reflect config changes
- Ensure backward compatibility while adding new GraphRAG support
- #8396

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-23 09:29:37 +08:00
Yongteng Lei
f63ad6b725
Fix: correct cancel logic error (#8973)
### What problem does this PR solve?

Correct cancel logic error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2025-07-23 09:25:48 +08:00
Adrian Altermatt
6691532079
Feat: Add model editing functionality with improved UI labels (#8855)
### What problem does this PR solve?

Add edit button for local LLM models
<img width="1531" height="1428" alt="image"
src="https://github.com/user-attachments/assets/19d62255-59a6-4a7e-9772-8b8743101f78"
/>

<img width="1531" height="1428" alt="image"
src="https://github.com/user-attachments/assets/c3a0f77e-cc6b-4190-95a6-13835463428b"
/>



### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

---------

Co-authored-by: Liu An <asiro@qq.com>
2025-07-21 19:16:53 +08:00
Kevin Hu
0b487dee43
Fix: support cross language for API. (#8946)
### What problem does this PR solve?

Close #8943

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-21 17:25:28 +08:00
Kevin Hu
c783d90ba3
Perf: set timeout for building chunks. (#8940)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2025-07-21 15:56:45 +08:00
Kevin Hu
ab53a73768
Perf: limit embedding in KG. (#8917)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2025-07-18 19:51:14 +08:00
天海蒼灆
412a088008
Feat: Add knowledge graph http api (#8896)
### What problem does this PR solve?

Add knowledge graph http api

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-07-17 19:20:48 +08:00
Kevin Hu
9767c26535
Fix: wrong parameters. (#8900)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-17 18:19:13 +08:00
Kevin Hu
ecdb1701df
Perf: test llm before RAPTOR. (#8897)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2025-07-17 16:48:50 +08:00
湛露先生
fa1d6ed683
fix settings global docStoreConn for opensearch (#8885)
### What problem does this PR solve?
fix opensearch OSConnection init.
```
        docStoreConn = rag.utils.opensearch_conn.OSConnection()
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
2025-07-17 12:10:42 +08:00
Kevin Hu
fbd115773b
Perf: set timeout of some steps in KG. (#8873)
### What problem does this PR solve?

### Type of change


- [x] Performance Improvement
2025-07-16 18:06:03 +08:00
Kevin Hu
f2909ea0c4
Perf: retryable mysql connection. (#8858)
### What problem does this PR solve?

### Type of change

- [x] Performance Improvement
2025-07-15 19:05:48 +08:00
Kevin Hu
aa4a725529
Pref: use redis to check if canceled. (#8853)
### What problem does this PR solve?

### Type of change

- [x] Performance Improvement
2025-07-15 17:19:27 +08:00
Kevin Hu
24c41d2a61
Perf: make do_cancel quicker. (#8846)
### What problem does this PR solve?

### Type of change

- [x] Performance Improvement
2025-07-15 14:35:00 +08:00
Kevin Hu
c642dbefca
Perf: Enhance timeout handling. (#8826)
### What problem does this PR solve?


### Type of change

- [x] Performance Improvement
2025-07-15 09:36:45 +08:00
Yongteng Lei
237e59532b
Feat: refine create and list operations for MCP dashboard (#8823)
### What problem does this PR solve?

Refine MCP dashboard create and list operations.

### Type of change

- [x] Refactoring
2025-07-14 14:36:56 +08:00
Yongteng Lei
72c19b44c3
Refa: better MIME content type (#8801)
### What problem does this PR solve?

Better uniform MIME content type.

### Type of change

- [x] Refactoring
2025-07-11 18:47:19 +08:00
Yongteng Lei
e8aee8d720
Feat: change document status in bulk (#8777)
### What problem does this PR solve?
 
Change document status in bulk.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-07-11 10:38:59 +08:00
haol
512772c45a
Fix: Resolve typo in /list route function (#8769)
### What problem does this PR solve?

Fixes a function name typo for the `/list` route in
`api/apps/conversation_app.py`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-10 14:32:28 +08:00
He Wang
cedcd13204
fix: use tenant_id of kb to get index name in rm chunk func (#8760)
### What problem does this PR solve?

The rm function in chunk_app.py now takes the index name differently
than other functions, so there will be situations where users can create
and update a chunk but not delete it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-10 10:30:56 +08:00
Liu An
f8524462b0 Fix: Increase default chunk_token_num from 128 to 512 in parser config (#8753)
### What problem does this PR solve?

Updated the default `chunk_token_num` value in `api_utils.py` and
`validation_utils.py` to 512 to accommodate larger text chunks. Adjusted
corresponding test cases in HTTP and SDK API tests to reflect this
change.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-10 09:53:20 +08:00
Can Wang
779932dcb0
Fix: graphrag, raptor can be null for api created kb issue (#8743)
### What problem does this PR solve?

When knowledgebase/dataset created by API, graphrag and raptor can be
null, and will trigger NoneType error when reach to this code, causing
chunking task not able to finish.

![image](https://github.com/user-attachments/assets/998a63e9-611b-4301-8808-24839a05be8a)

Proposed solution will result in None and pass the condition check
without error.

![image](https://github.com/user-attachments/assets/184374fb-e06a-46e6-b8ac-d66a3fd93b59)


### Type of change

-   Bug Fix (non-breaking change which fixes an issue)
2025-07-09 17:12:42 +08:00
Yongteng Lei
c1f6e6f00e
Feat: add advanced document filter (#8723)
### What problem does this PR solve?

Add advanced document filter

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-07-09 09:33:11 +08:00
Liu An
addda5ccbe
Fix: Add validation for dialog name (#8722)
### What problem does this PR solve?

- Validate dialog name in `dialog_app.py` to ensure it is a non-empty
string and does not exceed 255 bytes in UTF-8 encoding.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-08 19:20:29 +08:00
Yongteng Lei
4d7bfd2ba3
Fix: typo process_duration (#8696)
### What problem does this PR solve?

Fix typo process_duration.

### Type of change

- [x] Documentation Update
- [x] Refactoring
2025-07-07 14:11:47 +08:00
Fee He
ae3683c346
fix task_service.py (#8687)
Fix the case where pages variable might be None

### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-07 09:48:51 +08:00
Yongteng Lei
1ac61c0f0f
Fix: secure canvas (#8670)
### What problem does this PR solve?

Secure canvas access.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-04 19:40:39 +08:00
Yongteng Lei
4243330d5c
Feat: add MCP server test endpoint (#8632)
### What problem does this PR solve?

Add MCP server test endpoint.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-07-02 18:52:24 +08:00
Can Wang
83c8af1b59
Fix: page_size can be None error (#8603)
### What problem does this PR solve?

Issue #8602

`parser_config.task_page_size` can be defaults to `None` when dataset is
created by API. This was not handled by the `task_executor.py` code thus
`page_size` could sometimes be `None` which will cause issue in line
351.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-02 18:38:48 +08:00
Kevin Hu
fffb7c0bba
Fix: anthropic llm issue. (#8633)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-02 18:37:34 +08:00
Liu An
0b40eb3e90
Test: Add tests for chunk API endpoints (#8616)
### What problem does this PR solve?

- Add comprehensive test suite for chunk operations including:
  - Test files for create, list, retrieve, update, and delete chunks
  - Authorization tests
  - Batch operations tests
- Update test configurations and common utilities
- Validate `important_kwd` and `question_kwd` fields are lists in
chunk_app.py
- Reorganize imports and clean up duplicate code

### Type of change

- [x] Add test cases
2025-07-02 09:49:08 +08:00
Kevin Hu
e3edcc3064
Trivals. (#8597)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-01 14:05:18 +08:00
天海蒼灆
d4da6dce6e
Feat: Add file management HTTP_API (#8395)
### What problem does this PR solve?

Add file management HTTP_API for operating files

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-07-01 09:51:53 +08:00
Yongteng Lei
8801de2772
Refa: change mcp_client module to rag/utils/conn (#8578)
### What problem does this PR solve?

Change mcp_client module to rag/utils/conn.

### Type of change

- [x] Refactoring
2025-07-01 09:29:19 +08:00
Yongteng Lei
0478f36e36
Feat: allow users to choose which MCP tools are enabled (#8519)
### What problem does this PR solve?

Allow users to choose which MCP tools are enabled.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-06-27 10:23:34 +08:00
Stephen Hu
7dbe06f7d8
Refactor: remove useless initialize logic in list_doc (#8523)
### What problem does this PR solve?

Remove useless logic in a loop for list_doc

### Type of change

- [x] Refactoring
- [x] Performance Improvement
2025-06-27 10:23:08 +08:00
Stephen Hu
938d8dd878
Fix: user_default_llm configuration doesn't work for OpenAI API compatible LLM factory (#8502)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/8467
when add llm the llm_name will like "llm1___OpenAI-API"
f09ca8e795/api/apps/llm_app.py (L173)
so we should not use llm1 to query


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-27 09:41:12 +08:00
zhanglei
daf6c82066
fix: list index out of range (#8518)
### What problem does this PR solve?

stack:

```
2025-06-26 17:22:24,739 ERROR    1609 list index out of range
Traceback (most recent call last):
  File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
  File "/ragflow/.venv/lib/python3.10/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
  File "/ragflow/api/utils/api_utils.py", line 298, in decorated_function
    return func(*args, **kwargs)
  File "/ragflow/api/apps/sdk/session.py", line 472, in list_session
    print(conv["reference"][message_num])
IndexError: list index out of range

```


![图片](https://github.com/user-attachments/assets/93fe90a8-0434-4842-ba9f-bb5a995b498a)


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-06-27 09:38:33 +08:00
Yongteng Lei
d768130204
Fix: chunk number error after re-parsing (#8513)
### What problem does this PR solve?

Fix chunk number error after re-parsing. #8503.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-26 17:46:53 +08:00
Liu An
d11cfd4e45
Fix: Add input validation to chunk creation endpoint (#8516)
### What problem does this PR solve?

- Include optional `tag_feas` field if present in request
- Add input validation for `important_kwd` and `question_kwd` to ensure
they are lists
- #8462

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-26 17:46:00 +08:00
Yongteng Lei
0eb90e73a5
Feat: add MCP dashboard functionalities list_tools and test_tool (#8505)
### What problem does this PR solve?

Add MCP dashboard functionalities list_tools and test_tool.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-06-26 13:52:01 +08:00
Liu An
dac5bcdf17
Fix: Enforce default embedding model in create_dataset / update_dataset (#8486)
### What problem does this PR solve?

Previous:
- Defaulted to hardcoded model 'BAAI/bge-large-zh-v1.5@BAAI'
- Did not respect user-configured default embedding_model

Now:
- Correctly prioritizes user-configured default embedding_model

Other:
- Make embedding_model optional in CreateDatasetReq with proper None
handling
- Add default embedding model fallback in dataset update when empty
- Enhance validation utils to handle None values and string
normalization
- Update SDK default embedding model to None to match API changes
- Adjust related test cases to reflect new validation rules

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-25 16:41:32 +08:00
Stephen Hu
8d9d2cc0a9
Fix: some cases Task return but not set progress (#8469)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/8466
I go through the codes, current logic:
When do_handle_task raises an exception, handle_task will set the
progress, but for some cases do_handle_task internal will just return
but not set the right progress, at this cases the redis stream will been
acked but the task is running.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-06-25 09:58:55 +08:00