yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-11-23 13:36:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
haystack/test/components/preprocessors
History
Julian Risch f687d49fec
feat: Add option to split by number of tokens to RecursiveDocumentSplitter (#9143) 
* add token split_unit

* fix overlap with fallback

* reno

* mark as integration tests

* use type ignore instead of assert

* Update releasenotes/notes/recursive-splitter-token-df56428887ac45bd.yaml

Co-authored-by: David S. Batista <dsbatista@gmail.com>

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-04-01 09:48:59 +02:00
..
__init__.py
chore: add license header to all modules (#7675)
2024-05-09 13:40:36 +00:00
test_csv_document_cleaner.py
feat: Update csv cleaner (#8828)
2025-02-07 14:29:53 +01:00
test_csv_document_splitter.py
feat: add split_by_row feature to CSVDocumentSplitter (#9031)
2025-03-19 12:18:44 +01:00
test_document_cleaner.py
refactor!: remove dataframe field from Document and ExtractedTableAnswer; make pandas optional (#8906)
2025-03-04 11:06:07 +00:00
test_document_splitter.py
fix: update meta data before initializing new Document in DocumentSplitter (#8745)
2025-01-20 09:51:47 +01:00
test_hierarchical_doc_splitter.py
feat: adding AutoMergingRetriever and HierarchicalDocumentSplitter (#9067)
2025-03-19 18:25:23 +00:00
test_recursive_splitter.py
feat: Add option to split by number of tokens to RecursiveDocumentSplitter (#9143)
2025-04-01 09:48:59 +02:00
test_sentence_tokenizer.py
fix: adding missing abbreviations files for SentenceSplitter (#8660)
2024-12-19 11:08:29 +01:00
test_text_cleaner.py
chore: add license header to all modules (#7675)
2024-05-09 13:40:36 +00:00