2 Commits

Author SHA1 Message Date
giorgossideris
11a43421e3
Min tokens in token limiter (#2400)
* Add minimum token threshold in MessageHistoryLimiter

* Update transforms tests for the threshold

* Move min_threshold_tokens from Message to Token Limiter

* Optimize _check_tokens_threshold

Co-authored-by: Wael Karkoub <wael.karkoub96@gmail.com>

* Apply requested changes (renaming, phrasing, validations)

* Fix format

* Fix _check_tokens_threshold logic

* Update docs and notebook

* Improve phrasing

* Add min_tokens example in notebook

* Add min_tokens example in website docs

* Add min_tokens example in notebook

* Update website docs to be in sync with get_logs change

---------

Co-authored-by: Wael Karkoub <wael.karkoub96@gmail.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-04-29 02:11:16 +00:00
giorgossideris
90883904c5
Standardize printing of MessageTransforms (#2308)
* Standardize printing of MessageTransforms

* Fix pre-commit black failure

* Add test for transform_messages printing

* Return str instead of printing

* Rename to_print_stats to verbose

* Cleanup

* t i# This is a combination of 3 commits.

Update requirements

* Remove lazy-fixture

* Avoid calling apply_transform in two code paths

* Format

* Replace stats with logs

* Handle no content messages in TokenLimiter get_logs()

* Move tests from test_transform_messages to test_transforms

---------

Co-authored-by: Wael Karkoub <wael.karkoub96@gmail.com>
2024-04-14 00:48:24 +00:00