67 Commits

Author SHA1 Message Date
Yongteng Lei
03656da4dd
Refa: upgrade MCP SDK to v1.9.4 (#8421)
### What problem does this PR solve?

Upgrade MCP SDK to v1.9.4 (latest).

### Type of change

- [x] Refactoring
2025-06-23 16:53:59 +08:00
Liu An
7e87eb2e23
Docs: Update version references to v0.19.1 in READMEs and docs (#8366)
### What problem does this PR solve?

- Update Docker image version badges and references from v0.19.0 to
v0.19.1
- Modify version mentions in all localized README files (id, ja, ko,
pt_br, tzh, zh)
- Update version in docker/README.md and related documentation files
- Includes updates to Helm values and Python SDK dependencies

### Type of change

- [x] Documentation Update
2025-06-19 14:39:27 +08:00
africa-worker
44287fb05f
Oss support opendal(including mysql) (#8204)
### What problem does this PR solve?

#8074
Oss support opendal(including mysql)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-06-12 11:37:42 +08:00
Liu An
52c814b89d
Refa: Move HTTP API tests to top-level test directory (#8042)
### What problem does this PR solve?

Move test cases only - CI still runs tests under sdk/python

### Type of change

- [x] Refactoring
2025-06-04 13:16:17 +08:00
Liu An
e702431fcb
Feat: sync test group to top pyproject.toml (#8015)
### What problem does this PR solve?

sync test group from sdk/python/pyproject.toml to top pyproject.toml

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-06-03 15:21:06 +08:00
He Wang
aaefc3f44c
update xgboost and dep scripts for local build on MacOS (#7857)
### What problem does this PR solve?

There are two main changes:

1. Update xgboost to 1.6.0 to build the project on MacOS with Apple
chips, this change refers to the issue:
https://github.com/infiniflow/ragflow/issues/5114.

2. When `use_china_mirrors` is set in `download_deps.py`, the names of
chrome files downloaded by the script will be different from the file
names used in Dockerfile, so I added the file name in `get_urls`
function to solve this problem.

I think it's better to add testing for Docker image
`infiniflow/ragflow_deps` to the test workflow, but since the workflow
is currently running on a self-hosted runner, I'm not sure how to modify
it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-27 09:28:52 +08:00
liu an
590b9dabab Docs: update for v0.19.0 (#7823)
### What problem does this PR solve?

update for v0.19.0

### Type of change

- [x] Documentation Update
2025-05-23 18:25:47 +08:00
Song Fuchang
a1f06a4fdc
Feat: Support tool calling in Generate component (#7572)
### What problem does this PR solve?

Hello, our use case requires LLM agent to invoke some tools, so I made a
simple implementation here.

This PR does two things:

1. A simple plugin mechanism based on `pluginlib`:

This mechanism lives in the `plugin` directory. It will only load
plugins from `plugin/embedded_plugins` for now.

A sample plugin `bad_calculator.py` is placed in
`plugin/embedded_plugins/llm_tools`, it accepts two numbers `a` and `b`,
then give a wrong result `a + b + 100`.

In the future, it can load plugins from external location with little
code change.

Plugins are divided into different types. The only plugin type supported
in this PR is `llm_tools`, which must implement the `LLMToolPlugin`
class in the `plugin/llm_tool_plugin.py`.
More plugin types can be added in the future.

2. A tool selector in the `Generate` component:

Added a tool selector to select one or more tools for LLM:


![image](https://github.com/user-attachments/assets/74a21fdf-9333-4175-991b-43df6524c5dc)

And with the `bad_calculator` tool, it results this with the `qwen-max`
model:


![image](https://github.com/user-attachments/assets/93aff9c4-8550-414a-90a2-1a15a5249d94)


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2025-05-16 16:32:19 +08:00
Gifford Nowland
0b4d366514
Fix: Setuptools project.license as a TOML table deprecation (#7652)
TOML-table-based project.license is deprecated as per PEP 639, see:
https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license-and-license-files

### What problem does this PR solve?

The following error when building project (e.g. `uv build`)

```
SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated
!!

        ********************************************************************************
        Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0).

        By 2026-Feb-18, you need to update your project and remove deprecated calls
        or your builds will no longer be supported.

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-15 12:23:29 +08:00
Gifford Nowland
e7a84bdac2
Fix: setuptools package definitions (#7654)
### What problem does this PR solve?
For `uv package`/`uv pip install ".[full]"`, bug introduced in #6370:

* Removes erroneous (non-package) directories (`helm`, `flask_session`)
* Adds `mcp.server` package
* Resolves "warning: package would be ignored" ambiguity by changing
`sdk` to `sdk.python.ragflow_sdk`
* Resolves "error: package directory 'intergrations' does not exist" by
including `intergrations.chatgpt-on-wechat.plugins` explicitly
* Also rearranges packages in alphabetical order, for DX.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2025-05-15 12:22:56 +08:00
pyyuhao
c8c3b756b0
Feat: Adds OpenSearch2.19.1 as the vector_database support (#7140)
### What problem does this PR solve?

This PR adds the support for latest OpenSearch2.19.1 as the store engine
& search engine option for RAGFlow.

### Main Benefit

1. OpenSearch2.19.1 is licensed under the [Apache v2.0 License] which is
much better than Elasticsearch
2. For search, OpenSearch2.19.1 supports full-text
search、vector_search、hybrid_search those are similar with Elasticsearch
on schema
3. For store, OpenSearch2.19.1 stores text、vector those are quite
simliar with Elasticsearch on schema

### Changes

- Support opensearch_python_connetor. I make a lot of adaptions since
the schema and api/method between ES and Opensearch differs in many
ways(especially the knn_search has a significant gap) :
rag/utils/opensearch_coon.py
- Support static config adaptions by changing:
conf/service_conf.yaml、api/settings.py、rag/settings.py
- Supprt some store&search schema changes between OpenSearch and ES:
conf/os_mapping.json
- Support OpenSearch python sdk : pyproject.toml
- Support docker config for OpenSearch2.19.1 :
docker/.env、docker/docker-compose-base.yml、docker/service_conf.yaml.template

### How to use
- I didn't change the priority that ES as the default doc/search engine.
Only if in docker/.env , we set DOC_ENGINE=${DOC_ENGINE:-opensearch}, it
will work.


### Others
Our team tested a lot of docs in our environment by using OpenSearch as
the vector database ,it works very well.
All the conifg for OpenSearch is necessary.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yongteng Lei <yongtengrey@outlook.com>
Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2025-04-24 16:03:31 +08:00
liu an
03672df691
Docs: update for v0.18.0 (#7223)
### What problem does this PR solve?

update for v0.18.0

### Type of change

- [x] Documentation Update
2025-04-23 12:02:50 +08:00
Yongteng Lei
68b9dae6c0
Feat: mcp server (#7084)
### What problem does this PR solve?

Add MCP support with a client example.

Issue link: #4344

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-04-21 09:43:20 +08:00
Zhichang Yu
6bf26e2a81
Optimize graphrag again (#6513)
### What problem does this PR solve?

Removed set_entity and set_relation to avoid accessing doc engine during
graph computation.
Introduced GraphChange to avoid writing unchanged chunks.

### Type of change

- [x] Performance Improvement
2025-03-26 15:34:42 +08:00
Yongteng Lei
85eb367ede
Feat: add basic Langfuse support for LLM module (#6443)
### What problem does this PR solve?

#6155

Add basic Langfuse support for LLM module.

A trace example:

<img width="755" alt="image"
src="https://github.com/user-attachments/assets/25c1f852-5116-486c-a47f-6097187142ca"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 13:18:47 +08:00
fansir
4091af4560
Fix: multiple top-level packages error in Python project (#6370)
### What problem does this PR solve?

This PR resolves the issue of multiple top-level packages being detected
in the Python project, which caused errors when using uv pip install.
The problem occurred because the project had multiple directories files
at the root level, leading to a flat-layout error.
To fix this, the pyproject.toml file was updated to explicitly list the
packages using the [tool.setuptools] section. This ensures that the
correct packages are included during installation, avoiding the
flat-layout error.
Type of change

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-21 18:44:49 +08:00
Yongteng Lei
674f94228b
Chore: unify Ruff config and enable async checks (ASYNC, TRIO) (#6351)
### What problem does this PR solve?

Unify Ruff config and enable async checks (ASYNC, TRIO)

### Type of change

- [x] CI/CD or tooling improvement
2025-03-20 22:31:18 +08:00
Kevin Hu
4df4bf68a2
DOCS: for release. (#6023)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update
2025-03-13 15:09:29 +08:00
Kevin Hu
d44739283c
Docs: prepare docs for release v0.17.1 (#5900)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update
2025-03-11 14:39:41 +08:00
Zhichang Yu
c813c1ff4c
Made task_executor async to speedup parsing (#5530)
### What problem does this PR solve?

Made task_executor async to speedup parsing

### Type of change

- [x] Performance Improvement
2025-03-03 18:59:49 +08:00
Kevin Hu
d6836444c9
DOC: for release. (#5472)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2025-03-02 18:47:06 +08:00
yihong
37aacb3960
Refa: drop useless fasttext (#5470)
### What problem does this PR solve?

This patch drop useless fastext which is seems useless in the code base 
and its very kind of hard install
should close #4498


### Type of change

- [x] Refactoring

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-02-28 14:30:56 +08:00
Kevin Hu
53b9e7b52f
Add tavily as web searh tool. (#5349)
### What problem does this PR solve?

#5198

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-26 10:21:04 +08:00
Kevin Hu
fe9e9a644f
Preparation for release. (#4739)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update
2025-02-06 15:15:13 +08:00
Zhichang Yu
3411d0a2ce
Added cuda_is_available (#4725)
### What problem does this PR solve?

Added cuda_is_available

### Type of change

- [x] Refactoring
2025-02-05 18:01:23 +08:00
Zhichang Yu
e1526846da
Fixed GPU detection on CPU only environment (#4711)
### What problem does this PR solve?

Fixed GPU detection on CPU only environment. Close #4692

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-05 12:02:43 +08:00
Zhichang Yu
c4b1c4e6f4
Fix onnxruntime-gpu marks (#4643)
### What problem does this PR solve?

Fix onnxruntime-gpu marks

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-26 09:37:59 +08:00
Zhichang Yu
3c2c8942d5
Removed onnxruntime (#4632)
### What problem does this PR solve?

Removed onnxruntime. It conflicts with the onnxruntime-gpu.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-24 23:41:52 +08:00
Zhichang Yu
8b49734241
Added onnxruntime-gpu (#4631)
### What problem does this PR solve?

Added onnxruntime-gpu

### Type of change

- [x] Refactoring
2025-01-24 14:33:21 +08:00
Kevin Hu
dd0ebbea35
Light GraphRAG (#4585)
### What problem does this PR solve?

#4543

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-22 19:43:14 +08:00
Zhichang Yu
2962284c79
Bump akshare (#4536)
### What problem does this PR solve?

Bump akshare. Close #4525 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-20 11:17:59 +08:00
Zhichang Yu
57b4e0c464
Bump infinity to v0.6.0-dev2 (#4497)
### What problem does this PR solve?

Bump infinity to v0.6.0-dev2. Close #4477 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-16 10:07:17 +08:00
Zhichang Yu
d3c07794b5
Replace poetry with uv (#4471)
### What problem does this PR solve?

Replace poetry with uv

### Type of change

- [x] Refactoring
2025-01-14 11:49:43 +08:00
Zhichang Yu
95261f17f6
Bump infinity to v0.6.0-dev1 (#4448)
### What problem does this PR solve?

Bump infinity to v0.6.0-dev1 and poetry to 2.0.1

### Type of change

- [x] Refactoring
2025-01-12 19:54:33 +08:00
Hayden Zhou
6acbd374d8
fix duckduckgo search subsection error (#4430)
### What problem does this PR solve?

duckduckgo search 6.3.0 still has error sometimes, need to update to
7.2.0, after updated, it works ok.
this PR is going to fix this issue
https://github.com/infiniflow/ragflow/issues/4396

### Type of change

 Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: xiaohzho <xiaohzho@cisco.com>
2025-01-10 09:34:20 +08:00
Hayden Zhou
ac89a2dda1
[Fix] fix duckduck go search 202 ratelimit failed (#4398)
this PR is going to fix issue
https://github.com/infiniflow/ragflow/issues/4396

### Type of change

 Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: xiaohzho <xiaohzho@cisco.com>
2025-01-07 18:40:46 +08:00
Jin Hai
439d20e41f
Use LTS polars to resolve some machines don't support AVX CPU flag (#4364)
### What problem does this PR solve?

Some old types of machine or virtual machine doesn't support AVX CPU
flag. This PR is to use lts polars module to avoid this fault.

fix issue: #4349

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

Signed-off-by: jinhai <haijin.chn@gmail.com>
2025-01-06 18:52:17 +08:00
Jin Hai
1160b58b6e
Update pyproject.toml to 0.15.1 (#4222)
### What problem does this PR solve?

Missing update information

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-12-25 14:06:00 +08:00
Zhichang Yu
b7a7413419
Bump infinity to 0.5.2 (#4207)
### What problem does this PR solve?

Bump infinity to 0.5.2

### Type of change

- [x] Refactoring
2024-12-24 15:17:37 +08:00
Zhichang Yu
4764ca5ef7
Bump infinity to 0.5.0 (#4088)
### What problem does this PR solve?

Bump infinity to 0.5.0

### Type of change

- [x] Refactoring
2024-12-18 14:39:09 +08:00
liuhua
db74a3ef34
Fix conversation bug in agent session (#4078)
### What problem does this PR solve?

Fix conversation bug in agent session

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
2024-12-18 11:49:25 +08:00
Zhichang Yu
0bca46ac3a
Migrate infinity at startup (#3858)
### What problem does this PR solve?

Migrate infinity at startup

#3809
https://github.com/infiniflow/infinity/issues/2321

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-12-13 13:43:56 +08:00
so95
d6c74ff131
Add mssql support (#3985)
some thing
-  execsql  add connection mssql
- fix bug duckduckgo-search rate limit
- update typo vi res

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-12-12 19:26:44 +08:00
Zhichang Yu
301f95837c
Try to reuse existing chunks (#3983)
### What problem does this PR solve?

Try to reuse existing chunks. Close #3793
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-12-12 16:38:03 +08:00
Kevin Hu
0d5486aa57
Add a dependency blincker. (#3882)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-12-05 18:14:39 +08:00
Zhichang Yu
6fb1a181aa
Added aspose on macosx/arm64 (#3686)
### What problem does this PR solve?

Added aspose on macosx/arm64. Close #3666 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-27 15:00:07 +08:00
Zhichang Yu
d61bbe6750
Use polars-lts-cpu on arm64 (#3667)
### What problem does this PR solve?

Use polars-lts-cpu on arm64

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-27 09:32:41 +08:00
Kevin Hu
49494d4e3c
Update version number for poetry (#3637)
### What problem does this PR solve?



### Type of change

- [x] Refactoring
2024-11-25 17:57:08 +08:00
Zhichang Yu
08ead81dde
Bump infinity to v0.5.0-dev5 (#3520)
### What problem does this PR solve?

Bump infinity to v0.5.0-dev5

### Type of change

- [x] Refactoring
2024-11-25 11:53:58 +08:00
Zhichang Yu
0cd5b64c3b
Changed requirement to python 3.10 (#3496)
### What problem does this PR solve?

Changed requirement to python 3.10.
Changed image base to Ubuntu 22.04 since it contains python 3.10.

### Type of change

- [x] Refactoring
2024-11-19 18:25:04 +08:00