5 Commits

Author	SHA1	Message	Date
Gaoxiang Luo	972b4ed024	Fix message history limiter for tool call (#3178) ... * fix: message history limiter to support tool calls * add: pytest and docs for message history limiter for tool calls * Added keep_first_message for HistoryLimiter transform * Update to inbetween to between * Updated keep_first_message to non-optional, logic for history limiter * Update transforms.py * Update test_transforms to match utils introduction, add keep_first_message testing * Update test_transforms.py for pre-commit checks --------- Co-authored-by: Mark Sze <66362098+marklysze@users.noreply.github.com> Co-authored-by: Chi Wang <wang.chi@microsoft.com>	2024-08-09 03:14:34 +00:00
Wael Karkoub	4ebfb82186	Ignore Some Messages When Transforming (#2661) ... * works * spelling * returned old docstring * add cache fix * spelling? --------- Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>	2024-05-22 20:22:17 +00:00
Wael Karkoub	372ac1e794	Text Compression Transform (#2225) ... * adds implementation * handles optional import * cleanup * updates github workflows * skip test if dependencies not installed * skip test if dependencies not installed * use cpu * skip openai * unskip openai * adds protocol * better docstr * minor fixes * updates optional dependencies docs * wip * update docstrings * wip * adds back llmlingua requirement * finalized protocol * improve docstr * guide complete * improve docstr * fix FAQ * added cache support * improve cache key * cache key fix + faq fix * improve docs * improve guide * args -> params * spelling	2024-05-06 14:16:49 +00:00
giorgossideris	11a43421e3	Min tokens in token limiter (#2400) ... * Add minimum token threshold in MessageHistoryLimiter * Update transforms tests for the threshold * Move min_threshold_tokens from Message to Token Limiter * Optimize _check_tokens_threshold Co-authored-by: Wael Karkoub <wael.karkoub96@gmail.com> * Apply requested changes (renaming, phrasing, validations) * Fix format * Fix _check_tokens_threshold logic * Update docs and notebook * Improve phrasing * Add min_tokens example in notebook * Add min_tokens example in website docs * Add min_tokens example in notebook * Update website docs to be in sync with get_logs change --------- Co-authored-by: Wael Karkoub <wael.karkoub96@gmail.com> Co-authored-by: Chi Wang <wang.chi@microsoft.com>	2024-04-29 02:11:16 +00:00
giorgossideris	90883904c5	Standardize printing of MessageTransforms (#2308) ... * Standardize printing of MessageTransforms * Fix pre-commit black failure * Add test for transform_messages printing * Return str instead of printing * Rename to_print_stats to verbose * Cleanup * t i# This is a combination of 3 commits. Update requirements * Remove lazy-fixture * Avoid calling apply_transform in two code paths * Format * Replace stats with logs * Handle no content messages in TokenLimiter get_logs() * Move tests from test_transform_messages to test_transforms --------- Co-authored-by: Wael Karkoub <wael.karkoub96@gmail.com>	2024-04-14 00:48:24 +00:00