yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-08-04 14:48:51 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,688 Commits 73 Branches 248 Tags
Commit Graph

3 Commits

Author SHA1 Message Date
Silvano Cerza
ec9f898cd6
fix: Fix TextDocumentSplitter failing if run with empty list (#6081) 
* Fix TextDocumentSplitter failing if run with empty list

* Release notes

* Simplify check

* Enhance test
 2023-10-17 11:25:28 +02:00
Julian Risch
90ddeba579
fix: DocumentSplitter and DocumentCleaner copy id_hash_keys to newly created Documents (#6083) 
* copy id_hash_keys in splitter and cleaner

* reno
 2023-10-17 11:03:48 +02:00
Julian Risch
4413675e64
feat: Add TextDocumentSplitter that splits by word, sentence, passage (2.0) (#5870) 
* draft split by word, sentence, passage

* naive way to split sentences without nltk

* reno

* add tests

* make input list of docs, review feedback

* add source_id and more validation

* update docstrings

* add split delimiters back to strings

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2023-09-27 12:26:20 +02:00