123 Commits

Author SHA1 Message Date
Jin Hai
360f5c1179
Move token related functions to common (#10942)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 08:50:05 +08:00
Jin Hai
5a200f7652
Add time utils (#10849)
### What problem does this PR solve?

- Add time utilities and unit tests

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-28 19:09:14 +08:00
Jin Hai
766d900a41
Refactor: rename rmSpace to remove_redundant_spaces (#10796)
### What problem does this PR solve?

- rename rmSpace to remove_redundant_spaces
- move clean_markdown_block to common module
- add unit tests for remove_redundant_spaces and clean_markdown_block

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-28 09:46:32 +08:00
Kevin Hu
0d8791936e
Feat: TOC retrieval (#10456)
### What problem does this PR solve?

#10436

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-10-10 17:07:55 +08:00
Jin Hai
d931c33ced
Fix typos: retrievaler -> retriever (#10372)
### What problem does this PR solve?

Fix typos

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-10 09:17:36 +08:00
Lynn
2d5d10ecbf
Feat/admin drop user (#10342)
### What problem does this PR solve?

- Admin client support drop user.

Issue: #10241 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-09-29 10:16:13 +08:00
Jin Hai
4eb7659499
Fix bug: broken import from rag.prompts.prompts (#10217)
### What problem does this PR solve?

Fix broken imports

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2025-09-23 10:19:25 +08:00
Kevin Hu
63781bde3f
Refa: import issue. (#9958)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2025-09-05 19:26:15 +08:00
GrubbyLee
72bb79e8dd
During the chat, the assistant's response cited documents outside current chat's kbs (#9900)
### What problem does this PR solve?

During the chat, the assistant's response cited documents outside the
current knowledge base。

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-09-04 16:51:13 +08:00
Kevin Hu
8d8a5f73b6
Fix: meta data filter with AND logic operations. (#9687)
### What problem does this PR solve?

Close #9648

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-25 18:29:24 +08:00
Kevin Hu
d0dc56166c
Fix: no effect on retrieval_test in term of metadata filter. (#9566)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-19 18:57:35 +08:00
Kevin Hu
f123587538
Feat: add meta filter to search app. (#9554)
### What problem does this PR solve?


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-08-19 17:25:44 +08:00
Kevin Hu
b5b8032a56
Feat: Support metadata auto filer for Search. (#9524)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-08-19 10:27:24 +08:00
Yongteng Lei
188c0f614b
Refa: refine search app (#9536)
### What problem does this PR solve?

Refine search app.

### Type of change

- [x] Refactoring
2025-08-19 09:33:33 +08:00
Yongteng Lei
ffc095bd50
Feat: conversation completion can specify different model (#9485)
### What problem does this PR solve?

Conversation completion can specify different model

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-08-15 17:44:58 +08:00
Kevin Hu
5e8cd693a5
Refa: split services about llm. (#9450)
### What problem does this PR solve?

### Type of change

- [x] Refactoring
2025-08-13 16:41:01 +08:00
Kevin Hu
4b1b68c5fc
Fix: no doc hits after meta data filter. (#9435)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-13 12:43:31 +08:00
balibabu
9098efb8aa
Feat: Fixed the issue where some fields in the chat configuration could not be displayed #3221 (#9430)
### What problem does this PR solve?

Feat: Fixed the issue where some fields in the chat configuration could
not be displayed #3221
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-08-13 10:26:26 +08:00
Kevin Hu
153e430b00
Feat: add meta data filter. (#9405)
### What problem does this PR solve?

#8531 
#7417 
#6761 
#6573
#6477

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-08-12 14:12:56 +08:00
Yongteng Lei
465f7e036a
Feat: advanced list dialogs (#9256)
### What problem does this PR solve?

Advanced list dialogs

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-08-06 10:33:52 +08:00
Yongteng Lei
52a349349d
Fix: migrate deprecated Langfuse API from v2 to v3 (#9204)
### What problem does this PR solve?

Fix:

```bash
'Langfuse' object has no attribute 'trace'
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-08-04 14:45:43 +08:00
Kevin Hu
fbd115773b
Perf: set timeout of some steps in KG. (#8873)
### What problem does this PR solve?

### Type of change


- [x] Performance Improvement
2025-07-16 18:06:03 +08:00
Yongteng Lei
1b022116d5
Feat: wrap search app (#8320)
### What problem does this PR solve?

Wrap search app

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-06-18 16:45:42 +08:00
Kevin Hu
91804f28f1
Fix: issue for tavily only in a assistant. (#8076)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-06-05 13:00:43 +08:00
Yongteng Lei
0c562f0a9f
Refa: change citation mark as [ID:n] (#7923)
### What problem does this PR solve?

Change citation mark as [ID:n], it's easier for LLMs to follow the
instruction :) #7904

### Type of change

- [x] Refactoring
2025-05-29 10:03:51 +08:00
Yongteng Lei
453287b06b Feat: more robust fallbacks for citations (#7801)
### What problem does this PR solve?

Add more robust fallbacks for citations

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2025-05-23 18:24:55 +08:00
Yongteng Lei
42f4d4dbc8 Fix: wrong type hint (#7738)
### What problem does this PR solve?

Wrong hint type. #7729.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-23 18:21:06 +08:00
Yongteng Lei
e8e2a95165
Refa: more fallbacks for bad citation format (#7710)
### What problem does this PR solve?

More fallbacks for bad citation format

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2025-05-19 19:34:05 +08:00
Kevin Hu
2ccec93d71
Feat: support cross-lang search. (#7557)
### What problem does this PR solve?

#7376
#4503
#5710 
#7470

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-05-09 15:32:02 +08:00
Kevin Hu
75b24ba02a
Fix: chat solo issue. (#7479)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-05-06 19:30:00 +08:00
WhiteBear
2c62652ea8
<think> tag is missing. (#7256)
### What problem does this PR solve?

Some models force thinking, resulting in the absence of the think tag in
the returned content

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-24 11:44:10 +08:00
Kevin Hu
5af2d57086
Refa. (#7022)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2025-04-15 10:20:33 +08:00
Yongteng Lei
7a34159737
Fix: add fallback for bad citation output (#7014)
### What problem does this PR solve?

Add fallback for bad citation output. #6948

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-04-15 09:33:53 +08:00
Yongteng Lei
dc2c74b249
Feat: add primitive support for function calls (#6840)
### What problem does this PR solve?

This PR introduces ​**​primitive support for function calls​**​,
enabling the system to handle basic function call capabilities.
However, this feature is currently experimental and ​**​not yet enabled
for general use​**​, as it is only supported by a subset of models,
namely, Qwen and OpenAI models.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-04-08 16:09:03 +08:00
Kevin Hu
9ecc78feeb
Refa: copywriting refinement. (#6779)
### What problem does this PR solve?

Close #6762

### Type of change

- [x] Refactoring
2025-04-03 11:38:02 +08:00
Roccat
200b6f55c6
Fix: NameError: free variable 'langfuse_generation' referenced before assignment in enclosing scope (#6451)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: lizheng@ssc-hn.com <lizheng@ssc-hn.com>
2025-03-24 15:14:36 +08:00
Yongteng Lei
85eb367ede
Feat: add basic Langfuse support for LLM module (#6443)
### What problem does this PR solve?

#6155

Add basic Langfuse support for LLM module.

A trace example:

<img width="755" alt="image"
src="https://github.com/user-attachments/assets/25c1f852-5116-486c-a47f-6097187142ca"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-24 13:18:47 +08:00
Kevin Hu
42eb99554f
Feat: add token comsumption & speed to little lamp. (#6077)
### What problem does this PR solve?

#6059

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-14 13:37:31 +08:00
Kevin Hu
caecaa7562
Feat: apply LLM to optimize citations. (#5935)
### What problem does this PR solve?

#5905

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-03-11 19:56:21 +08:00
Kevin Hu
90d18143ba
Refa: add prompt to empty retrieved answwer. (#5892)
### What problem does this PR solve?

#5883

### Type of change

- [x] Refactoring
2025-03-11 13:11:14 +08:00
Kevin Hu
8e965040ce
Fix: rm <think> for ES sql generation. (#5881)
### What problem does this PR solve?

#5850

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-11 10:41:19 +08:00
Kevin Hu
ec68ab1c8c
Fix: search citation issue. (#5657)
### What problem does this PR solve?
#5649
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-05 17:25:47 +08:00
Yongteng Lei
f6dd2cd1af
Fix: fix may lose part of information of last stream chunck (#5584)
### What problem does this PR solve?

 Fix may lose part of information of last stream chunck

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-04 11:58:10 +08:00
Kevin Hu
7a81fa00e9
Optimize prompt. (#5541)
### What problem does this PR solve?

#5526

### Type of change

- [x] Performance Improvement
2025-03-03 13:12:38 +08:00
Kevin Hu
5fdfb8d465
Fix: rm think if stream is Flase. (#5458)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-28 10:05:18 +08:00
Kevin Hu
fa76974e24
Fix issue of ask API. (#5400)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-26 19:45:22 +08:00
Kevin Hu
4f40f685d9
Code refactor (#5371)
### What problem does this PR solve?

#5173

### Type of change

- [x] Refactoring
2025-02-26 15:40:52 +08:00
Kevin Hu
4e2afcd3b8
Fix FlagRerank max_length issue. (#5366)
### What problem does this PR solve?

#5352

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-26 11:01:13 +08:00
Kevin Hu
53b9e7b52f
Add tavily as web searh tool. (#5349)
### What problem does this PR solve?

#5198

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-26 10:21:04 +08:00
Kevin Hu
b3d579e2c1
Refine prompt of agentic search. (#5312)
### What problem does this PR solve?

#5173

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-02-25 09:21:52 +08:00