* Add isort
* Apply isort on py files
* Fix circular import
* Fix format for notebooks
* Fix format
---------
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
* Change defualt model for `lmm`
* Try to use PIL image for LMM's _oai_messages
* Update test cases and llava
* Remove redundant files
* Update the imports for lmm tests
* Test case fix
* Docstring update
* LMM notebook lint
* Typo correction for img_utils and its test
* Fix self.client bug for Dalle Agent
The self.client member variable is reserved for OpenAI's LLM-compatible
client, which requires usage summary support.
So, we rename self.client to self._dalle_client to resolve this issue
* Change defualt model for `lmm`
* Try to use PIL image for LMM's _oai_messages
* Update test cases and llava
* Remove redundant files
* Update the imports for lmm tests
* Test case fix
* Docstring update
* LMM notebook lint
* Typo correction for img_utils and its test
* Update test_llava.py
debug, reformat
---------
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Shaokun Zhang <shaokunzhang529@gmail.com>
Co-authored-by: Shaokun Zhang <shaokun.zhang@psu.edu>
* Validate the OpenAI API key format
Increase the amount of internal validation for OpenAI API keys. The intent is
to shorten the debugging loop in case of typos. The changes do *not* add
validation for Azure OpenAI API keys.
* Add the validation in `__init__` of `OpenAIClient`.
* Introduce the `MOCK_OPEN_AI_API_KEY` constant for testing.
* Add unit test coverage for the `is_valid_api_key` function.
* Validate the OpenAI API key format
Increase the amount of internal validation for OpenAI API keys. The intent is
to shorten the debugging loop in case of typos. The changes do *not* add
validation for Azure OpenAI API keys.
* Add the validation in `__init__` of `OpenAIClient`.
* Introduce the `MOCK_OPEN_AI_API_KEY` constant for testing.
*Add unit test coverage for the `is_valid_api_key` function.
* Log a warning when register a default client fails.
* Validate the OpenAI API key format
Increase the amount of internal validation for OpenAI API keys. The intent is
to shorten the debugging loop in case of typos. The changes do *not* add
validation for Azure OpenAI API keys.
* Add the validation in `__init__` of `OpenAIClient`. We'll log a
warning when the OpenAI API key isn't valid.
* Introduce the `MOCK_OPEN_AI_API_KEY` constant for testing.
* Add unit test coverage for the `is_valid_api_key` function.
* Check for OpenAI base_url before API key validation
---------
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
* fix: typo
* fix: typo
* fix: typo of function name
* fix: typo of function name of test file
* Update test_token_count.py
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
* LMM Code added
* LLaVA notebook update
* Test cases and Notebook modified for OpenAI v1
* Move LMM into contrib
To resolve test issues and deploy issues
In the future, we can install pillow by default, and then move back
LMM agents into agentchat
* LMM test setup update
* try...except... clause for LMM tests
* disable patch for llava agent test
To resolve dependencies issue for build
* Add LMM Blog
* Change docstring for LMM agents
* Docstring update patch
* llava: insert reply at position 1 now
So, it can still handle human_input_mode
and max_consecutive_reply
* Resolve comments
Fixing: typos, blogs, yml, and add OpenAIWrapper
* Signature typo fix for LMM agent: system_message
* Update LMM "content" from latest OpenAI release
Reference https://platform.openai.com/docs/guides/vision
* update LMM test according to latest OpenAI release
* Fully support GPT-4V now
1. Add a notebook for GPT-4V. LLava notebook also updated.
2. img_utils updated
3. GPT-4V formatter now return base64 image with mime type
4. Infer mime type directly from b64 image content (while loading
without suffix)
5. Test cases modified according to all the related changes.
* GPT-4V link updated in blog
---------
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
