2 Commits

Author SHA1 Message Date
afourney
b29f9a90e8
Updated get_max_token_limit with latest models and numbers (#972)
* Updated with latest models and numbers

* Updated as per comments.

* Added common Azure deployment names for models.

---------

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2023-12-18 03:06:41 +00:00
Yiran Wu
8c3401dd6a
Add token_count_util (#421)
* add token_count_util

* remove token_count from retrieval util

* format

* update dependency

* update test
2023-10-27 12:57:35 +00:00