yujunjun/dify
1
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-01-06 06:51:03 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix text splitter (#22596)

Browse Source
This commit is contained in:
znn 2025-07-18 11:21:58 +05:30 committed by GitHub
parent d37b6716cd
commit ed263aed9f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
api/core/rag/splitter/fixed_text_splitter.py
View File

@ -102,6 +102,7 @@ class FixedRecursiveCharacterTextSplitter(EnhanceRecursiveCharacterTextSplitter)
splits = text.split()
else:
splits = text.split(separator)
splits = [item + separator if i < len(splits) else item for i, item in enumerate(splits)]
else:
splits = list(text)
splits = [s for s in splits if (s not in {"", "\n"})]