Farzad E 6eb251d1f0
fix: Support for gpt-4-32k (#4825)
* Add step to loook up tokenizers by prefix in openai_utils

* Updated tiktoken min version + openai_utils test

* Added test case for GPT-4 and Azure model naming

* Broken down tests

* Added default case

---------

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>
2023-05-12 19:02:12 +02:00
..
2023-05-09 20:26:59 +02:00
2023-02-08 15:34:43 +01:00
2023-05-12 19:02:12 +02:00