Dayenne Souza 2f2cfa7b70
Test and unify text splitter functionality (#1547)
* add text_splitting unit test

* change folder test text splitting

* fix chunk fn

* test new function

* run formatter

* run spell check

* run semver

* remove tiktoken mocked from tests

* change progress ticker

* fix ruff check
2025-01-13 18:42:44 -03:00

3 lines
77 B
Python

# Copyright (c) 2024 Microsoft Corporation.
# Licensed under the MIT License