1092 Commits

Author SHA1 Message Date
buua436
5674d762f7
Feat:check embedding model api (#10854)
### What problem does this PR solve?
change:
Randomly sample `check_num` chunks from knowledge base `kb_id`, re-embed
them using `embd_id`, and compare with stored vectors via cosine
similarity. If `avg_cos_sim > 0.99`, return success (`code=0`);
otherwise return business failure (`code=10`).

url:
`/v1/kb/check_embedding`

Request Body:
```
{
  "kb_id": "<dataset_id>",
  "embd_id": "BAAI/bge-m3@SILICONFLOW",
  "check_num": 5
}

```
Success Response:
```
{
  "code": 0,
  "message": "success",
  "data": {
    "summary": { "avg_cos_sim": 0.999999, "sampled": 5, "valid": 5, "max_cos_sim":0.999999,"min_cos_sim":0.999999,"model":"BAAI/bge-m3@SILICONFLOW" },
    "results": [ ... ]
  }
}
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-30 19:06:16 +08:00
Billy Bao
fa38aed01b
Fix: the input length exceeds the context length (#10895)
### What problem does this PR solve?

Fix: the input length exceeds the context length #10750

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-30 19:00:53 +08:00
Yongteng Lei
a3bb4aadcc
Fix: predictable token generation (#10868)
### What problem does this PR solve?

Fix predictable token generation.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-30 09:31:36 +08:00
Liu An
40b2c48957
Chore(config): remove Youdao and BAAI embedding model providers (#10873)
### What problem does this PR solve?

This commit removes the Youdao and BAAI entries from the LLM factories
configuration as they are no longer needed or supported.

### Type of change

- [x] Config update
2025-10-29 19:38:57 +08:00
Billy Bao
55eb525fdc
Feat: rename file to avoid package name conflict (#10863)
### What problem does this PR solve?

Feat: rename file to avoid package name conflict

### Type of change

- [x] Refactoring
2025-10-29 12:19:57 +08:00
Yongteng Lei
c0c2a10680
Feat: allow initialize Redis without password (#10856)
### What problem does this PR solve?

Allow initialize Redis without password.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-29 09:45:28 +08:00
Jin Hai
5a200f7652
Add time utils (#10849)
### What problem does this PR solve?

- Add time utilities and unit tests

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-28 19:09:14 +08:00
Jin Hai
766d900a41
Refactor: rename rmSpace to remove_redundant_spaces (#10796)
### What problem does this PR solve?

- rename rmSpace to remove_redundant_spaces
- move clean_markdown_block to common module
- add unit tests for remove_redundant_spaces and clean_markdown_block

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-28 09:46:32 +08:00
jiasu
c5ac571676
Fixed the bug that passing an empty array will not update (#10755)
### What problem does this PR solve?
Fixed the bug that the "dataset_ids" field will not be updated if an
empty array is passed when updating the assistant

### Type of change

- [*] Bug Fix (non-breaking change which fixes an issue)
2025-10-27 11:29:20 +08:00
Zhichang Yu
73144e278b
Don't release full image (#10654)
### What problem does this PR solve?

Introduced gpu profile in .env
Added Dockerfile_tei
fix datrie
Removed LIGHTEN flag

### Type of change

- [x] Documentation Update
- [x] Refactoring
2025-10-23 23:02:27 +08:00
Kevin Hu
5fb5a51b2e
Fix: create KB initial embedding. (#10751)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-23 16:17:43 +08:00
Kevin Hu
f24d464a53
Fix: video file suffix (#10740)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-23 11:13:09 +08:00
Kevin Hu
de4f75dcd8
Fix: add video parser (#10735)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-23 09:24:16 +08:00
Billy Bao
a82e9b3d91
Fix: can't upload image in ollama model #10447 (#10717)
### What problem does this PR solve?

Fix: can't upload image in ollama model #10447

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)


### Change all `image=[]` to `image = None`

Changing `image=[]` to `images=None` avoids Python’s mutable default
parameter issue.
If you keep `images=[]`, all calls share the same list, so modifying it
(e.g., images.append()) will affect later calls.
Using images=None and creating a new list inside the function ensures
each call is independent.
This change does not affect current behavior — it simply makes the code
safer and more predictable.


把 `images=[]` 改成 `images=None` 是为了避免 Python 默认参数的可变对象问题。
如果保留 `images=[]`,所有调用都会共用同一个列表,一旦修改就会影响后续调用。
改成 None 并在函数内部重新创建列表,可以确保每次调用都是独立的。
这个修改不会影响现有运行结果,只是让代码更安全、更可控。
2025-10-22 12:24:12 +08:00
Yongteng Lei
2d491188b8
Refa: improve flow of GraphRAG and RAPTOR (#10709)
### What problem does this PR solve?

Improve flow of GraphRAG and RAPTOR.

### Type of change

- [x] Refactoring
2025-10-22 09:29:20 +08:00
Billy Bao
cd75fa02b1
Feat: Make knowledge base renaming automatically reflected in agent discussions, solved #10597 (#10680)
### What problem does this PR solve?
Feat: Make knowledge base renaming automatically reflected in agent
discussions, solved #10597

### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2025-10-21 10:42:05 +08:00
Billy Bao
cfdd37820a
Feat: Support attribute filtering #8703 (#10670)
### What problem does this PR solve?

Feat: Support attribute filtering #8703

### Type of change

- [X] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
Co-authored-by: writinwaters <cai.keith@gmail.com>
2025-10-21 10:38:40 +08:00
Yongteng Lei
5b2e5dd334
Feat: Gemini supports video parsing (#10671)
### What problem does this PR solve?

Gemini supports video parsing.


![img_v3_02r8_adbd5adc-d665-4756-9a00-3ae0f12224fg](https://github.com/user-attachments/assets/30d8d296-c336-4b55-9823-803979e705ca)


![img_v3_02r8_ab60c046-1727-4029-ad2e-66097fd3ccbg](https://github.com/user-attachments/assets/441b1487-a970-427e-98b6-6e1e002f2bad)

Close: #10617

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-20 16:49:47 +08:00
Billy Bao
d956a442ce
Fix: Remove pdf embed support, update based on #10635 (#10663)
### What problem does this PR solve?

Fix: Remove pdf embed support, update based on  #10635

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-20 13:45:53 +08:00
Kevin Hu
5fc59a3132
Fix: retrieval test (#10662)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-20 11:37:18 +08:00
Lynn
c9b18cbe18
Feat:admin api (#10642)
### What problem does this PR solve?

Support frontend auth.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-18 16:09:48 +08:00
Billy Bao
8ee0b6ea54
File: Now parsing support all types of embedded documents, solved #10059 (#10635)
### What problem does this PR solve?

File: Now parsing support all types of embedded documents, solved #10059
Fix: Incomplete words in chat #10530
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-17 18:46:47 +08:00
Kevin Hu
43ea312144
Fix: search highlight. (#10616)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-16 18:45:43 +08:00
Billy Bao
0f62bfda21
Feat: add forgot password reset (update naming style), solve #8547 (#10606)
### What problem does this PR solve?

Feat: add forgot password reset (update naming style), solve #8547

### Type of change

- [X] New Feature (non-breaking change which adds functionality)
2025-10-16 17:48:20 +08:00
Andrea Bugeja
8a41057236
Fix: Add RetryingPooledPostgresqlDatabase to handle max_retries param (#10524)
## What problem does this PR solve?

Fixes the PostgreSQL connection error that prevents RAGFlow from
starting:

peewee.ProgrammingError: invalid dsn: invalid connection option
"max_retries"


## Problem Analysis

The `BaseDataBase` class in `api/db/db_models.py` adds `max_retries` and
`retry_delay` to the database configuration dict before passing it to
the database connection constructor.

- **MySQL**: Has `RetryingPooledMySQLDatabase` class that properly
extracts these custom parameters using `kwargs.pop()` before calling the
parent constructor
- **PostgreSQL**: Was using the base `PooledPostgresqlDatabase` class
which passes all parameters directly to `psycopg2.connect()`, which
doesn't recognize `max_retries` as a valid connection option

## Solution

Created `RetryingPooledPostgresqlDatabase` class that:
- Extracts `max_retries` and `retry_delay` parameters before
initialization
- Implements retry logic with exponential backoff for connection
failures
- Handles PostgreSQL-specific connection errors (connection refused,
server closed, etc.)
- Mirrors the existing `RetryingPooledMySQLDatabase` implementation

Updated the `PooledDatabase` enum to use the new retrying class for
PostgreSQL.

## Benefits

 Prevents invalid connection parameters from being passed to psycopg2  
 Adds automatic retry logic for PostgreSQL connection failures  
 Provides better error logging for PostgreSQL-specific issues  
 Maintains consistency between MySQL and PostgreSQL database handling  

## Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

## Testing

Tested with PostgreSQL database configuration and verified:
- Server starts without the "invalid dsn" error
- Database connections are established successfully
- Retry logic works correctly on connection failures

Co-authored-by: Andrea Bugeja <andrea.bugeja@gig.com>
2025-10-16 15:08:41 +08:00
Billy Bao
447041d265
Feat: add forgot password reset, solve #8547 (#10586)
### What problem does this PR solve?

Feat: add forgot password reset, solve #8547

### Type of change

- [X] New Feature (non-breaking change which adds functionality)
2025-10-16 15:07:49 +08:00
buua436
4e86ee4ff9
Feat: Support Specifying OpenRouter Model Provider (#10550)
### What problem does this PR solve?
issue:
[#5787](https://github.com/infiniflow/ragflow/issues/5787)
change:
Support Specifying OpenRouter Model Provider

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-16 09:39:59 +08:00
Yongteng Lei
86b254d214
Improve file management (#10577)
### What problem does this PR solve?

Improve file management. #10287.

Passed tests:

1. Create folder `A` and `B`.
2. Upload a file inside `A`, called `file`.
3. Create a KB, called `K`.
3. Link `file` to `K`.
4. Parse `file` inside of `K`. (OK)
5. Move `file` from `A` to `B`.
6. Parse `file` inside of `K`. (OK)
7. Move `file` from `B` to `A`.
8. Parse `file` inside of `K`. (OK)
9. Move entire folder `A` into `B`. (B -> A -> file)
10. Parse `file` inside of `K`. (OK)
11. Delete folder `B`.
12. All clear. (There is no document inside of `K`)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-16 09:38:25 +08:00
Jin Hai
8844826208
Refactor admin client for message prompts (#10583)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-15 16:22:07 +08:00
Kevin Hu
34c54cd459
Fix: agent templates... (#10564)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-15 10:31:30 +08:00
Kevin Hu
16b5feadb7
Fix: canvas list with team. (#10549)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-14 19:38:54 +08:00
Kevin Hu
5fb3d2f55c
Fix: update parser id for change_parser. (#10545)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-14 15:49:05 +08:00
Kevin Hu
f92a45dcc4
Feat: let toc run asynchronizly... (#10513)
### What problem does this PR solve?

#10436 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-14 14:14:52 +08:00
Yongteng Lei
87659dcd3a
Fix: unexpected Auth return code (#10539)
### What problem does this PR solve?

Fix unexpected Auth return code.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-14 14:13:10 +08:00
Yongteng Lei
f11d8af936
Fix: wrong Knowledgebase tasks_finish_at (#10521)
### What problem does this PR solve?

Wrong Knowledgebase tasks_finish_at.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-14 09:30:46 +08:00
buua436
4e6b84bb41
Feat: add trino support (#10512)
### What problem does this PR solve?
issue:
[#10296](https://github.com/infiniflow/ragflow/issues/10296)
change:
- ExeSQL: support connecting to Trino.
- Validation: password can be empty only when db_type === "trino";
  all other database types keep the existing requirement (non-empty).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-13 13:57:40 +08:00
Yongteng Lei
65c3f0406c
Fix: maintain backward compatibility for KB tasks (#10508)
### What problem does this PR solve?

Maintain backward compatibility for KB tasks

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-10-13 11:53:48 +08:00
Günter Lukas
0283e4098f
Fix #10408 (#10471)
### What problem does this PR solve?

Google Cloud model does not work correctly with gemini-2.5 models
Close #10408

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-10-10 19:18:24 +08:00
YngvarHuang
2cdba3d1e6
Fix: Fixed the issue where swagger apidocs could not be opened properly(#9522) (#10461)
### What problem does this PR solve?

Fix: Fixed the issue where swagger apidocs could not be opened
properly(#9522)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: virgilwong <hyhvirgil@gmail.com>
2025-10-10 18:36:20 +08:00
Kevin Hu
0d8791936e
Feat: TOC retrieval (#10456)
### What problem does this PR solve?

#10436

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-10 17:07:55 +08:00
Billy Bao
9b06734ced
Feat: add total in List dataset API (#10448)
### What problem does this PR solve?

Feat: add total in List dataset API,  solved #10360 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-10 11:20:55 +08:00
Jin Hai
d931c33ced
Fix typos: retrievaler -> retriever (#10372)
### What problem does this PR solve?

Fix typos

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-10 09:17:36 +08:00
Billy Bao
f04c9e2937
Fix: correctly update parser method & correct vllm pdf parser (#10441)
### What problem does this PR solve?

Fix: correctly update parser method

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2025-10-09 19:03:12 +08:00
Kevin Hu
cbf04ee470
Feat: Use data pipeline to visualize the parsing configuration of the knowledge base (#10423)
### What problem does this PR solve?

#9869

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: jinhai <haijin.chn@gmail.com>
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: chanx <1243304602@qq.com>
Co-authored-by: balibabu <cike8899@users.noreply.github.com>
Co-authored-by: Lynn <lynn_inf@hotmail.com>
Co-authored-by: 纷繁下的无奈 <zhileihuang@126.com>
Co-authored-by: huangzl <huangzl@shinemo.com>
Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
Co-authored-by: Wilmer <33392318@qq.com>
Co-authored-by: Adrian Weidig <adrianweidig@gmx.net>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Yongteng Lei <yongtengrey@outlook.com>
Co-authored-by: Liu An <asiro@qq.com>
Co-authored-by: buua436 <66937541+buua436@users.noreply.github.com>
Co-authored-by: BadwomanCraZY <511528396@qq.com>
Co-authored-by: cucusenok <31804608+cucusenok@users.noreply.github.com>
Co-authored-by: Russell Valentine <russ@coldstonelabs.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Billy Bao <newyorkupperbay@gmail.com>
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
Co-authored-by: TensorNull <129579691+TensorNull@users.noreply.github.com>
Co-authored-by: TensorNull <tensor.null@gmail.com>
Co-authored-by: TeslaZY <TeslaZY@outlook.com>
Co-authored-by: Ajay <160579663+aybanda@users.noreply.github.com>
Co-authored-by: AB <aj@Ajays-MacBook-Air.local>
Co-authored-by: 天海蒼灆 <huangaoqin@tecpie.com>
Co-authored-by: He Wang <wanghechn@qq.com>
Co-authored-by: Atsushi Hatakeyama <atu729@icloud.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Mohamed Mathari <155896313+melmathari@users.noreply.github.com>
Co-authored-by: Mohamed Mathari <nocodeventure@Mac-mini-van-Mohamed.fritz.box>
Co-authored-by: Stephen Hu <stephenhu@seismic.com>
Co-authored-by: Shaun Zhang <zhangwfjh@users.noreply.github.com>
Co-authored-by: zhimeng123 <60221886+zhimeng123@users.noreply.github.com>
Co-authored-by: mxc <mxc@example.com>
Co-authored-by: Dominik Novotný <50611433+SgtMarmite@users.noreply.github.com>
Co-authored-by: EVGENY M <168018528+rjohny55@users.noreply.github.com>
Co-authored-by: mcoder6425 <mcoder64@gmail.com>
Co-authored-by: lemsn <lemsn@msn.com>
Co-authored-by: lemsn <lemsn@126.com>
Co-authored-by: Adrian Gora <47756404+adagora@users.noreply.github.com>
Co-authored-by: Womsxd <45663319+Womsxd@users.noreply.github.com>
Co-authored-by: FatMii <39074672+FatMii@users.noreply.github.com>
2025-10-09 12:36:19 +08:00
Jin Hai
ef0aecea3b
Refactor: fix admin exception (#10400)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-09 11:15:33 +08:00
Lynn
fb950079ef
Feat/service manage (#10381)
### What problem does this PR solve?

- Admin service support SHOW SERVICE <id>.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

issue: #10241
2025-09-30 16:23:09 +08:00
Yongteng Lei
7c620bdc69
Fix: unexpected operation of document management (#10366)
### What problem does this PR solve?

 Unexpected operation of document management.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-09-30 15:20:38 +08:00
Jin Hai
d6eded1959
Refactor: move exceptions to common folder (#10383)
### What problem does this PR solve?

as title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-09-30 15:10:04 +08:00
Stephen Hu
0b759f559c
Fix: invalid user can login from OSS (#10348)
### What problem does this PR solve?

An invalid user can log in from OSS
https://github.com/infiniflow/ragflow/issues/10293

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-09-29 10:16:31 +08:00
Lynn
2d5d10ecbf
Feat/admin drop user (#10342)
### What problem does this PR solve?

- Admin client support drop user.

Issue: #10241 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-09-29 10:16:13 +08:00