### What problem does this PR solve?
Fix: Fixed the issue where swagger apidocs could not be opened
properly(#9522)
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Co-authored-by: virgilwong <hyhvirgil@gmail.com>
### What problem does this PR solve?
Feat: add total in List dataset API, solved #10360
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
- Admin service support SHOW SERVICE <id>.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
issue: #10241
### What problem does this PR solve?
Unexpected operation of document management.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
- Admin client support drop user.
Issue: #10241
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Fix unexpected operation of file management.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
issue：#5617
change：add IBM DB2 support in ExeSQL
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Move base64 related function to api/common/base64.py
### Type of change
- [x] Refactoring
---------
Signed-off-by: jinhai <haijin.chn@gmail.com>
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
### What problem does this PR solve?
Refactor import modules.
### Type of change
- [x] Refactoring
---------
Signed-off-by: jinhai <haijin.chn@gmail.com>
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
### What problem does this PR solve?
Fixed the issue where database connections were interrupted under high
concurrency
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: lemsn <lemsn@126.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
### What problem does this PR solve?
- Admin client support show user and create user command.
- Admin client support alter user password and active status.
- Admin client support list user datasets.
issue: #10241
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Move base64 related function to api/common/base64.py
### Type of change
- [x] Refactoring
---------
Signed-off-by: jinhai <haijin.chn@gmail.com>
### What problem does this PR solve?
1. Fix typos
2. Rename function
3. Use English to write comment
### Type of change
- [x] Refactoring
Signed-off-by: jinhai <haijin.chn@gmail.com>
…rsation_app.py
### What problem does this PR solve?
issue：
#10188
change：
This PR replaces traceback.print_exc() with logging.exception(e) in
conversation_app.py to ensure that full error tracebacks are captured by
the logging system instead of being written only to stderr.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
Revert back to chat.completions.
### Type of change
- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [x] Other (please describe):
Revert back to chat.completions.
### What problem does this PR solve?
Fix broken imports
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Signed-off-by: jinhai <haijin.chn@gmail.com>
### What problem does this PR solve?
Fix: resolve hash collisions by switching to UUID &correct logic for
always-true statements, solved: #10165
Feat: Update GPT api integration, solved: #10204
Feat: Support qianwen-deepresearch, solved: #10163
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Add tree_merge for law parsers, significantly outperforming
hierarchical_merge, solved: #8637
1. Add tree_merge for law parsers, include build_tree and get_tree by
dfs.
2. add Copyright statement for helath_utils
### Type of change
- [x] Documentation Update
- [x] Performance Improvement
### What problem does this PR solve?
Introduce new feature: RAGFlow system admin service and CLI
### Introduction
Admin Service is a dedicated management component designed to monitor,
maintain, and administrate the RAGFlow system. It provides comprehensive
tools for ensuring system stability, performing operational tasks, and
managing users and permissions efficiently.
The service offers monitoring of critical components, including the
RAGFlow server, Task Executor processes, and dependent services such as
MySQL, Infinity / Elasticsearch, Redis, and MinIO. It automatically
checks their health status, resource usage, and uptime, and performs
restarts in case of failures to minimize downtime.
For user and system management, it supports listing, creating,
modifying, and deleting users and their associated resources like
knowledge bases and Agents.
Built with scalability and reliability in mind, the Admin Service
ensures smooth system operation and simplifies maintenance workflows.
It consists of a server-side Service and a command-line client (CLI),
both implemented in Python. User commands are parsed using the Lark
parsing toolkit.
- **Admin Service**: A backend service that interfaces with the RAGFlow
system to execute administrative operations and monitor its status.
- **Admin CLI**: A command-line interface that allows users to connect
to the Admin Service and issue commands for system management.
### Starting the Admin Service
1. Before start Admin Service, please make sure RAGFlow system is
already started.
2. Run the service script:
```bash
python admin/admin_server.py
```
The service will start and listen for incoming connections from the CLI
on the configured port.
### Using the Admin CLI
1. Ensure the Admin Service is running.
2. Launch the CLI client:
```bash
python admin/admin_client.py -h 0.0.0.0 -p 9381
## Supported Commands
Commands are case-insensitive and must be terminated with a semicolon
(`;`).
### Service Management Commands
- [x] `LIST SERVICES;`
- Lists all available services within the RAGFlow system.
- [ ] `SHOW SERVICE <id>;`
- Shows detailed status information for the service identified by
`<id>`.
- [ ] `STARTUP SERVICE <id>;`
- Attempts to start the service identified by `<id>`.
- [ ] `SHUTDOWN SERVICE <id>;`
- Attempts to gracefully shut down the service identified by `<id>`.
- [ ] `RESTART SERVICE <id>;`
- Attempts to restart the service identified by `<id>`.
### User Management Commands
- [x] `LIST USERS;`
- Lists all users known to the system.
- [ ] `SHOW USER '<username>';`
- Shows details and permissions for the specified user. The username
must be enclosed in single or double quotes.
- [ ] `DROP USER '<username>';`
- Removes the specified user from the system. Use with caution.
- [ ] `ALTER USER PASSWORD '<username>' '<new_password>';`
- Changes the password for the specified user.
### Data and Agent Commands
- [ ] `LIST DATASETS OF '<username>';`
- Lists the datasets associated with the specified user.
- [ ] `LIST AGENTS OF '<username>';`
- Lists the agents associated with the specified user.
### Meta-Commands
Meta-commands are prefixed with a backslash (`\`).
- `\?` or `\help`
- Shows help information for the available commands.
- `\q` or `\quit`
- Exits the CLI application.
## Examples
```commandline
admin> list users;
+-------------------------------+------------------------+-----------+-------------+
| create_date | email | is_active | nickname |
+-------------------------------+------------------------+-----------+-------------+
| Fri, 22 Nov 2024 16:03:41 GMT | jeffery@infiniflow.org | 1 | Jeffery |
| Fri, 22 Nov 2024 16:10:55 GMT | aya@infiniflow.org | 1 | Waterdancer |
+-------------------------------+------------------------+-----------+-------------+
admin> list services;
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+
| extra | host | id | name | port | service_type |
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+
| {} | 0.0.0.0 | 0 | ragflow_0 | 9380 | ragflow_server |
| {'meta_type': 'mysql', 'password': 'infini_rag_flow', 'username': 'root'} | localhost | 1 | mysql | 5455 | meta_data |
| {'password': 'infini_rag_flow', 'store_type': 'minio', 'user': 'rag_flow'} | localhost | 2 | minio | 9000 | file_store |
| {'password': 'infini_rag_flow', 'retrieval_type': 'elasticsearch', 'username': 'elastic'} | localhost | 3 | elasticsearch | 1200 | retrieval |
| {'db_name': 'default_db', 'retrieval_type': 'infinity'} | localhost | 4 | infinity | 23817 | retrieval |
| {'database': 1, 'mq_type': 'redis', 'password': 'infini_rag_flow'} | localhost | 5 | redis | 6379 | message_queue |
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+
```
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Signed-off-by: jinhai <haijin.chn@gmail.com>
### What problem does this PR solve?
Support server health check. Solved issue: #10106
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Add file convert to document API just like file2document_app.py
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Add support for KB document basic info
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Support image recognition with image links in markdown files, solved
issue: #8755
Fixed log info error in code_exec, solved issue: #10064
### Type of change (8755)
- [x] New Feature (non-breaking change which adds functionality)
### Type of change (10064)
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
An exception happens if you give session_id to agent_open_ai completion.
Because session_id is being given as well as **req so it tries to send
session_id twice. But also the logic seemed odd on picking one of
session_id, id, metadata.id. So cleaned it up a little.
See #10111
### Type of change
- [X] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
This PR fixes incorrect naming for PostgreSQL usage by replacing all
instances of `postgresql` with the correct `postgres` in the `db_type`
field. This resolves potential configuration errors and ensures
consistency when specifying the database type.
Also fixed handling of None for `get_queue_length`
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Co-authored-by: cucusenok <BP-116: updated readme.md>
### What problem does this PR solve?
Refine dataflow and initialize dataflow app.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
-Added the metadata_dedition parameter in the document retrieval
interface to filter document metadata -Updated the API documentation and
added explanations for the metadata_dedition parameter
### What problem does this PR solve?
Make /api/v1/retrieval api also can use metadata filter
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Fix wrong chunk number while re-parsing document and keeping original
chunks
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
### What problem does this PR solve?
During the chat, the assistant's response cited documents outside the
current knowledge base。
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
Add `canvas_category` field for UserCanvas and CanvasTemplate.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- Add robust serialize_for_json() function to handle non-serializable
objects
- Update server_error_response() to safely serialize exception data
- Update get_json_result() with fallback error handling
- Handles ModelMetaclass, functions, and other problematic objects
- Maintains proper JSON response format instead of server crashes
Fixes#9797
### What problem does this PR solve?
Currently, error responses and certain result objects may include types
that are not JSON serializable (e.g., ModelMetaclass, functions). This
causes server crashes instead of returning valid JSON responses.
This PR introduces a robust serializer that converts unsupported types
into string representations, ensuring the server always returns a valid
JSON response.
### Type of change
- [] Bug Fix (non-breaking change which fixes an issue)
### What problem does this PR solve?
Unify reference format of agent completion and OpenAI-compatible
completion API.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Documentation Update
- [x] Refactoring
