149 Commits

Author SHA1 Message Date
Henry Tu
3fdf8dea22
Add Cerebras Integration (#3585)
* Cerebras Integration

* Address feedback

* Fix typo

* Run formatter
2024-09-30 21:14:55 +00:00
afourney
d2b750de5d
Removes EasyOCR dependency from mdconvert. (#3573)
* Removes EasyOCR dependency from mdconvert.

* Update mdconvert.py

---------

Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>
2024-09-26 21:36:03 +00:00
afourney
0d5163b78a
WebSurfer Updated (Selenium, Playwright, and support for many filetypes) (#1929)
* Feat/headless browser (retargeted) (#1832)

* Add headless browser to the WebSurferAgent, closes #1481

* replace soup.get_text() with markdownify.MarkdownConverter().convert_soup(soup)

* import HeadlessChromeBrowser

* implicitly wait for 10s

* inicrease max. wait time to 99s

* fix: trim trailing whitespace

* test: fix headless tests

* better bing query search

* docs: add example 3 for headless option

---------

Co-authored-by: Vijay Ramesh <vijay@regrello.com>

* Handle missing Selenium package.

* Added browser_chat.py example to simplify testing.

* Based browser on mdconvert. (#1847)

* Based browser on mdconvert.

* Updated web_surfer.

* Renamed HeadlessChromeBrowser to SeleniumChromeBrowser

* Added an initial POC with Playwright.

* Separated Bing search into it's own utility module.

* Simple browser now uses Bing tools.

* Updated Playwright browser to inherit from SimpleTextBrowser

* Got Selenium working too.

* Renamed classes and files for consistency.

* Added more instructions.

* Initial work to support other search providers.

* Added some basic behavior when the BING_API_KEY is missing.

* Cleaned up some search results.

* Moved to using the request.Sessions object. Moved Bing SERP paring to mdconvert to be more broadly useful.

* Added backward compatibility to WebSurferAgent

* Selenium and Playwright now grab the whole DOM, not jus the body, allowing the converters access to metadata.

* Fixed printing of page titles in Playwright.

* Moved installation of WebSurfer dependencies to contrib-tests.yml

* Fixing pre-commit issues.

* Reverting conversable_agent, which should not have been changed in prior commit.

* Added RequestMarkdownBrowser tests.

* Fixed a bug with Bing search, and added search test cases.

* Added tests for Bing search.

* Added tests for md_convert

* Added test files.

* Added missing pptx.

* Added more tests for WebSurfer coverage.

* Fixed guard on requests_markdown_browser test.

* Updated test coverage for mdconvert.

* Fix brwser_utils tests.

* Removed image test from browser, since exiftool isn't installed on test machine.

* Removed image test from browser, since exiftool isn't installed on test machine.

* Disable Selenium GPU and sandbox to ensure it runs headless in Docker.

* Added option for Bing API results to be interleaved (as Bing specifies), or presented in a categorized list (Web, News, Videos), etc

* Print more details when requests exceptions are thrown.

* Added additional documentation to markdown_search

* Added documentation to the selenium_markdown_browser.

* Added documentation to playwright_markdown_browser.py

* Added documentation to requests_markdown_browser

* Added documentation to mdconvert.py

* Updated agentchat_surfer notebook.

* Update .github/workflows/contrib-tests.yml

Co-authored-by: Davor Runje <davor@airt.ai>

* Merge main. Resolve conflicts.

* Resolve pre-commit checks.

* Removed offending LFS file.

* Re-added offending LFS file.

* Fixed browser_utils tests.

* Fixed style errors.

---------

Co-authored-by: Asapanna Rakesh <45640029+INF800@users.noreply.github.com>
Co-authored-by: Vijay Ramesh <vijay@regrello.com>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Davor Runje <davor@airt.ai>
2024-09-25 22:17:42 +00:00
Lokesh Goel
2e1f788293
Add Couchbase VectorDB Support (#3525)
* add couchbase vector store

* add unit test

* cleanup

* add couchbase to website

* fix code formatting

---------

Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>
2024-09-25 17:15:35 +00:00
HRUSHIKESH DOKALA
34b34d0203
Amazon Bedrock Client for AutoGen (#3232)
* intial commit for aws-bedrock

* format

* converse setup for model req-response

* Renamed to bedrock.py, updated parameter parsing, system message extraction, client class incorporation

* Established Bedrock class based on @astroalek and @ChristianT's code, added ability to disable system prompt separation

* Image parsing and removing access credential checks

* Added tests, added additional parameter support

* Amazon Bedrock documentation

* Moved client parameters to init, align parameter names with Anthropic, spelling fix, remove unnecessary imports, use base and additional parameters, update documentation

* Tidy up comments

* Minor typo fix

* Correct comment re aws_region

---------

Co-authored-by: Mark Sze <mark@sze.family>
Co-authored-by: Mark Sze <66362098+marklysze@users.noreply.github.com>
Co-authored-by: Li Jiang <bnujli@gmail.com>
2024-08-26 07:28:53 +00:00
Eric Zhu
d5068d9b6c
update contact information on the repo and release package (#3383)
* update contact information on the repo and release package

* update contact

* update

* fix format
2024-08-20 02:04:44 +00:00
Mark Sze
fa273961f6
Update Mistral client class to support new Mistral v1.0.1 package (#3356)
* Update Mistral client class to support new Mistral v1.0.1 package

* Remove comments

* Refactored assistant/system role order, tidied imports and comments

---------

Co-authored-by: HRUSHIKESH DOKALA <96101829+Hk669@users.noreply.github.com>
2024-08-16 20:55:36 +00:00
Fabian Valle
f9295c4c39
+mdb atlas vectordb [clean_final] (#3000)
* +mdb atlas

* Update test/agentchat/contrib/vectordb/test_mongodb.py

Co-authored-by: HRUSHIKESH DOKALA <96101829+Hk669@users.noreply.github.com>

* update test_mongodb.py; we dont need to do the assert .collection_name vs .name

* Try fix mongodb service

* Try fix mongodb service

* Update username and password

* Update autogen/agentchat/contrib/vectordb/mongodb.py

* closer --- but im not super thrilled about the solution...

* PYTHON-4506 Expanded tests and simplified vector search pipelines

* Update mongodb.py

* Update mongodb.py - Casey

* search_index_magic

index_name change; keeping track of lucene indexes is tricky

* Fix format

* Fix tests

* hacking trying to figure this out

* Streamline checks for indexes in construction and restructure tests

* Add tests for score_threshold, embedding inclusion, and multiple query tests

* refactored create_collection to meet base object requirements

* lint

* change the localhost port to 27017

* add test to check that no embedding is there unless explicitly provided

* Update logger

* Add test get docs with ids=None

* Rename and update notebook

* have index management include waiting behaviors

* Adds further optional waits or users and tests. Cleans up upsert.

* ensure the embedding size for multiple embedding inputs is equal to dimensions

* fix up tests and add configuration to ensure documents and indexes are READY for querying

* fix import failure

* adjust typing for 3.9

* fix up the notebook output

* changed language to communicate time taken on first init_chat call

* replace environment variable usage

---------

Co-authored-by: Fabian Valle <fabian.valle-simmons@mongodb.com>
Co-authored-by: HRUSHIKESH DOKALA <96101829+Hk669@users.noreply.github.com>
Co-authored-by: Li Jiang <bnujli@gmail.com>
Co-authored-by: Casey Clements <casey.clements@mongodb.com>
Co-authored-by: Jib <jib.adegunloye@mongodb.com>
Co-authored-by: Jib <Jibzade@gmail.com>
Co-authored-by: Cozypet <yanhan860711@gmail.com>
2024-07-25 23:11:19 +00:00
HRUSHIKESH DOKALA
b4a3f263b0
Cohere Client (#3004)
* initial setup for cohere client

* client update

* changes: ClintType added to the utils

* Revert "changes: ClintType added to the utils"

This reverts commit 80d61552287f2d2eff50b4c0f1a4adfd97233aa3.

* Message conversion to Cohere, Parameter handling, cost calculation, streaming, tool calling

* Changed Groq references.

* minor fix

* tests added

* ref fix

* added in the workflows

* Fixed bug on non-streaming text generation

* fix: formatting

* Support Cohere rule for last message not USER when tool_results exist

* Added Cohere to documentation

* fixed client.py merge, removed unnecessary comments in groq.py, updated Cohere documentation, added Groq documentation

* log: ignored params

* update: custom exception added

---------

Co-authored-by: Mark Sze <mark@sze.family>
Co-authored-by: Mark Sze <66362098+marklysze@users.noreply.github.com>
2024-07-03 14:33:03 +00:00
Li Jiang
62792478a4
Update qdrant dependency (#3064)
* Update qdrant dependency

* Update qdrant dependency
2024-07-02 16:00:31 +00:00
Mark Sze
23c1dec206
Groq Client (#3003)
* Groq Client Class - main class and setup, except tests

* Change pricing per K, added tests

* Streaming support, including with tool calling

* Used Groq retries instead of loop, thanks Gal-Gilor!

* Fixed bug when using logging.

---------

Co-authored-by: Qingyun Wu <qingyun0327@gmail.com>
2024-06-28 05:58:42 +00:00
Mark Sze
b1ec3ae545
Together AI Client (#2919)
* First pass together.ai client class

* Config handling, models and cost

* Added tests, moved param management to create function

* Tests, parameter, validation, logging updates

* Added use of client_utils PR 2949

* Updated to return OAI response

* Notebook example

* Improved function calling, updated tests, updated notebook with Chess example

* Tidied up together client class, better parameter handling, simpler exception capture, warning for no cost, reuse in tests, cleaner tests

* Update of documentation notebook, replacement of old version

* Fix of messages parameter for hide_tools function call

* Update autogen/oai/together.py

Co-authored-by: Qingyun Wu <qingyun0327@gmail.com>

* Update together.py to fix text

---------

Co-authored-by: Qingyun Wu <qingyun0327@gmail.com>
Co-authored-by: Yiran Wu <32823396+yiranwu0@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-06-21 17:14:44 +00:00
Li Jiang
3e3f2985c2
Fix qdrant version (#2984) 2024-06-21 09:51:17 +00:00
Mark Sze
27e3690378
Mistral Client (#2892)
* Initial commit of Mistral client class

* Updated to manage final system message for reflection_with_llm

* Add Mistral support to client class

* Add Mistral support across the board (based on Gemini changes)

* Test file for Mistral client

* Updated handling of config, added notebook for documentation

* Added support for additional API parameters

* Remove unneeded code, updated exception raising

* Updated handling of keywords, including type checks, defaults, warnings. Updated notebook example to remove logging warnings.

* Added class description.

* Updated tests to support new config handling.

* Moved parameter parsing to create function, minimised init, added parameter tests

* Refined parameter validation

* Correct spacing

* Fixed string concat in parameter validation

* Corrected upper/lower bound warning

* Use client_tools, tidy up Mistral create, better handle tool call response, tidy tests

* Update of documentation notebook, replacement of old version

* Update to handle multiple tool_call recommendations in a message

* Updated tests to accommodate multiple tool_calls as well as content in message

* Update autogen/oai/mistral.py comment

Co-authored-by: Qingyun Wu <qingyun0327@gmail.com>

* cleanup, rewrite mock

* update

---------

Co-authored-by: Qingyun Wu <qingyun0327@gmail.com>
Co-authored-by: kevin666aa <yrwu000627@gmail.com>
2024-06-21 05:42:16 +00:00
HRUSHIKESH DOKALA
32fa709e13
Anthropic Client (#2931)
* intial setup for the anthropic client with cost config

* update: other methods added

* fix: formatting

* fix: config unused

* update: changes made in the client

* update: test added to the workflow

* update: added tests to the anthropic client

* fix: errors in workflows and client

* fix

* fix: anthropic tools type

* update: notebook anthropic

* Nonetype fixed

* fix-tests config

* update: tests and client issues

* logger support

* remove sys path

* updated the functioning of the client

* update: type hints and stream

* skip tests- importerror

* fix: anthropic client and tests

* none fix

* Alternating roles, parameter keywords, cost on response,

* update: anthropic notebook

* update: notebook with more details

* devcontainer

* update: added validate_params from the client_utils

* fix: formatting

* fix: minor comment

---------

Co-authored-by: Mark Sze <mark@sze.family>
2024-06-19 06:15:12 +00:00
LeoLjl
c54e8a5db1
[Autobuild] improve robustness and reduce cost (#2907)
* Update Autobuild.

* merge main into autobuild

* update test for new autobuild

* update author info

* fix pre-commit

* Update autobuild notebook

* Update autobuild_agent_library.ipynb

* Update autobuild_agent_library.ipynb

* Fix pre-commit failures.

---------

Co-authored-by: Linxin Song <rm.social.song1@gmail.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-06-14 14:14:08 +00:00
Zoltan Lux
a959deeac6
Use Gemini without API key (#2805)
* google default auth and svc keyfile for Gemini

* [.Net] Release note for 0.0.14 (#2815)

* update release note

* update trigger

* [.Net] Update website for AutoGen.SemanticKernel and AutoGen.Ollama (#2814)
support vertex ai compute region

* [CAP] User supplied threads for agents (#2812)

* First pass:  message loop in main thread

* pypi version bump

* Fix readme

* Better example

* Fixed docs

* pre-commit fixes

* refactoring, minor fixes, update gemini demo ipynb

* add new deps again and reset line endings

* Docstring for the init function. Use private methods

* improve docstring

---------

Co-authored-by: Xiaoyun Zhang <bigmiao.zhang@gmail.com>
Co-authored-by: Rajan <rajan.chari@yahoo.com>
Co-authored-by: Zoltan Lux <z.lux@campus.tu-berlin.de>
2024-06-03 21:00:00 +00:00
Audel Rouhi
6604ca511b
PGVector Support for Custom Connection Object (#2566)
* Added fixes and tests for basic auth format

* User can provide their own connection object. Added test for it.

* Updated instructions on how to use. Fully tested all 3 authentication methods successfully.

* Get password from gitlab secrets.

* Hide passwords.

* Update notebook/agentchat_pgvector_RetrieveChat.ipynb

Co-authored-by: Li Jiang <bnujli@gmail.com>

* Hide passwords.

* Added connection_string test. 3 tests total for auth.

* Fixed quotes on db config params. No other changes found.

* Ran notebook

* Ran pre-commits and updated setup to include psycopg[binary] for windows and mac.

* Corrected list extension.

* Separate connection establishment function. Testing pending.

* Fixed pgvectordb auth

* Update agentchat_pgvector_RetrieveChat.ipynb

Added autocommit=True in example

* Rerun notebook

---------

Co-authored-by: Li Jiang <bnujli@gmail.com>
Co-authored-by: Li Jiang <lijiang1@microsoft.com>
2024-05-24 17:58:56 +00:00
Li Jiang
80b2dba6ab
Add packaging explicitly (#2780) 2024-05-24 14:52:56 +00:00
Wael Karkoub
372ac1e794
Text Compression Transform (#2225)
* adds implementation

* handles optional import

* cleanup

* updates github workflows

* skip test if dependencies not installed

* skip test if dependencies not installed

* use cpu

* skip openai

* unskip openai

* adds protocol

* better docstr

* minor fixes

* updates optional dependencies docs

* wip

* update docstrings

* wip

* adds back llmlingua requirement

* finalized protocol

* improve docstr

* guide complete

* improve docstr

* fix FAQ

* added cache support

* improve cache key

* cache key fix + faq fix

* improve docs

* improve guide

* args -> params

* spelling
2024-05-06 14:16:49 +00:00
Li Jiang
83f9f3e733
Fix chroma import error (#2557)
* Fix chroma import error

* fix format
2024-05-01 18:05:45 +00:00
Wael Karkoub
a9171211c7
Streamline Testing with pytest-cov and pytest Defaults (#2490)
* done

* update docs

* try fix

* update workflows

* undo minor fix

* resolve comments

* adds back pytest-asyncio

* minor fix

* add branch coverage

* restore pip install e.

* test with coverage

* fix mypy

* fix coverage + docker + windows combo

* fix bash command

* formatter

* formatter

* one last fix

* I lied, last fix

* fix

* fix retrieve chat test

* fix windows paths

* change cache seed

* down grade openai version

* fix openai mypy

* better error type

* fix image gen cache test

* fix

* experimenting

* fix lmm

* skip cosmos test

* remove cosmos db

* unused imports

* handle more cosmosdb skips

* fix flaky test
2024-04-29 00:27:34 +00:00
Audel Rouhi
1b8d65df0a
2447 fix pgvector tests and notebook (#2455)
* Re-added missing notebook

* Test installing postgres

* Error handle the connection.

* Fixed import.

* Fixed import.

* Fixed creation of collection without client.

* PGVector portion working. OpenAI untested.

* Fixed prints.

* Added output.

* Fixed pre-commits.

* Run pgvector notebook

* Improve efficiency of get_collection

* Fix delete_collection

* Fixed issues with pytests and validated functions.

* Validated pytests.

* Fixed pre-commits

* Separated extra_requires to allow more logic. Retrieve_chat base dependencies included on pgvector and qdrant.

* Fixed extra newline.

* Added username and password fields.

* URL Encode the connection string parameters to support symbols like %

* Fixed pre-commits.

* Added pgvector service

* pgvector doesn't have health intervals.

* Switched to colon based key values.

* Run on Ubuntu only. Linux is only option with container service support.

* Using default credentials instead.

* Fix postgres setup

* Fix postgres setup

* Don't skip tests on win and mac

* Fix command error

* Try apt install postgresql

* Assert table does not exist when deleted.

* Raise value error on a empty list or None value provided for IDs

* pre-commit

* Add install pgvector

* Add install pgvector

* Reorg test files, create a separate job for test pgvector

* Fix format

* Fix env format

* Simplify job name, enable test_retrieve_config

* Fix test_retrieve_config

* Corrected behavior for get_docs_by_ids with no ids returning all docs.

* Corrected behavior for get_docs_by_ids with no ids returning all docs.

* Fixed pre-commits.

* Added return values for all functions.

* Validated distance search is implemented correctly.

* Validated all pytests

* Removed print.

* Added default clause.

* Make ids optional

* Fix test, make it more robust

* Bump version of openai for the vector_store support

* Added support for choosing the sentence transformer model.

* Added error handling for model name entered.

* Updated model info.

* Added model_name db_config param.

* pre-commit fixes and last link fix.

* Use secrets password.

* fix: link fixed

* updated tests

* Updated config_list.

* pre-commit fix.

* Added chat_result to all output.
Unable to re-run notebooks.

* Pre-commit fix detected this requirement.

* Fix python 3.8 and 3.9 not supported for macos

* Fix python 3.8 and 3.9 not supported for macos

* Fix format

* Reran notebook with MetaLlama3Instruct7BQ4_k_M

* added gpt model.

* Reran notebook

---------

Co-authored-by: Li Jiang <bnujli@gmail.com>
Co-authored-by: Hk669 <hrushi669@gmail.com>
2024-04-28 13:43:02 +00:00
Ian
a41182a93f
Support openai assistant v2 API (#2466)
* adapted to openai assistant v2 api

* fix comments

* format code

* fix ci

* Update autogen/agentchat/contrib/gpt_assistant_agent.py

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2024-04-23 17:26:06 +00:00
Li Jiang
b7366b570f
Update RetrieveChat extra dependencies (#2449) 2024-04-20 18:32:57 +00:00
Audel Rouhi
ded2d612c3
[Re-Opened] Support for PGVector Database in Autogen (#2439)
* PGVector Contrib Initial Commit - KnucklesTeam:autogen:pgvector_contrib fork

* Update website/docs/ecosystem/pgvector.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Updated qdrant installation instructions.

* Fixed openai version.

* Added dependencies to install for qdrant and pgvector in contrib tests.

* Added dependencies to install for qdrant and pgvector in contrib tests.

* Cleaned up dependencies.

* Removed flaml out of setup.py. Used only for notebook example.

* Added PGVector notebook link

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-04-18 20:00:03 +00:00
Chi Wang
6636689cb9
speed up notebook in CI and fix CI failures (#2427)
* speed up notebook in CI

* improve notebook

* comment about filter

* bug fix for summary

* bump version to 0.2.26

* restrict openai version

* news update

* add tags in example
2024-04-18 16:43:34 +00:00
Beibin Li
0aaf30a8da
Merge "Gemini" feature into the main branch (#2360)
* Start Gemini integration: works ok with Text now

* Gemini notebook lint

* try catch "import" for Gemini

* Debug: id issue for chat completion in Gemini

* Add RAG example

* Update docs for RAG

* Fix missing pydash

* Remove temp folder

* Fix test error in runs/7206014032/job/19630042864

* Fix tqdm warning

* Fix notebook output

* Gemini's vision model is supported now

* Install instructions for the Gemini branch

* Catch and retry when see Interval Server Error 500

* Allow gemini to take more flexible messages
i.e., it can take messages where "user" is not the last role.

* Use int time for Gemini client

* Handle other exceptions in gemini call

* rename to "create" function for gemini

* GeminiClient compatible with ModelClient now

* Lint

* Update instructions in Gemini notebook

* Lint

* Remove empty blocks from Gemini notebook

* Add gemini into example page

* self.create instead of call

* Add py and Py into python execution

* Remove error code from merging

* Remove pydash dependency for gemini

* Add cloud-gemini doc

* Remove temp file

* cache import update

* Add test case for summary with mm input

* Lint: warnings instead of print

* Add test cases for gemini

* Gemini test config

* Disable default model for gemini

* Typo fix in gemini workflow

* Correct grammar in example notebook

* Raise if "model" is not provided in create(...)

* Move TODOs into a roadmap

* Update .github/workflows/contrib-tests.yml

Co-authored-by: Davor Runje <davor@airt.ai>

* Gemini test config update

* Update setup.py

Co-authored-by: Davor Runje <davor@airt.ai>

* Update test/oai/test_gemini.py

Co-authored-by: Davor Runje <davor@airt.ai>

* Update test/oai/test_gemini.py

Co-authored-by: Davor Runje <davor@airt.ai>

* Remove python 3.8 from gemini
No google's generativeai for Windows with Python 3.8

* Update import error handling for gemini

* Count tokens and cost for gemini

---------

Co-authored-by: Li Jiang <bnujli@gmail.com>
Co-authored-by: Davor Runje <davor@airt.ai>
2024-04-17 00:24:07 +00:00
William W Wang
689950e58e
Add extra_require for cosmosdb in setup.py (#2355) 2024-04-11 16:37:24 +00:00
Li Jiang
6b1376b04d
Add bs4 and overlap (#2271)
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-04-05 05:21:32 +00:00
Jack Gerrits
5ef2dfc104
Implement User Defined Functions for Local CLI Executor (#2102)
* Implement user defined functions feature for local cli exec, add docs

* add tests, update docs

* fixes

* fix test

* add pandas test dep

* install test

* provide template as func

* formatting

* undo change

* address comments

* add test deps

* formatting

* test only in 1 env

* formatting

* remove test for local only

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2024-03-27 23:45:17 +00:00
Davor Runje
78aa0eb220
Introducing IOStream protocol and adding support for websockets (#1551)
* Introducing IOStream

* bug fixing

* polishing

* refactoring

* refactoring

* refactoring

* wip: async tests

* websockets added

* wip

* merge with main

* notebook added

* FastAPI example added

* wip

* merge

* getter/setter to iostream added

* website/blog/2024-03-03-AutoGen-Update/img/dalle_gpt4v.png: convert to Git LFS

* website/blog/2024-03-03-AutoGen-Update/img/gaia.png: convert to Git LFS

* website/blog/2024-03-03-AutoGen-Update/img/teach.png: convert to Git LFS

* add SSL support

* wip

* wip

* exception handling added to on_connect()

* refactoring: default iostream is being set in a context manager

* test fix

* polishing

* polishing

* polishing

* fixed bug with new thread

* polishing

* a bit of refactoring and docs added

* notebook added to docs

* type checking added to CI

* CI fix

* CI fix

* CI fix

* polishing

* obsolete todo comment removed

* fixed precommit error

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2024-03-26 22:39:55 +00:00
Davor Runje
99e48dc024
Add mypy to pre-commit (#2068)
* Add mypy check in pre-commit

* Add mypy check in pre-commit

* bug fix

* CI fix

* add python version matrix for checking types in CI

* added 3.12 to CI

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2024-03-20 02:01:16 +00:00
Jack Gerrits
c2663330af
Require numpy be a 1.x release (#1966) 2024-03-12 15:05:53 +00:00
Chi Wang
0a49f2a256
bug fix and cleanup for 0.2.16 (#1827) 2024-03-01 04:31:49 +00:00
Jack Gerrits
76ef0789c0
Implement docker based jupyter executor (#1794)
* Implement docker based jupyter client

* formatting

* skip docker tests when asked

* feedback

* add log

* update build

* formatting

* structural changes

* update setup.py

* update tests

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-02-29 14:54:11 +00:00
Jack Gerrits
fbc2f6e911
Use jupyer-kernel-gateway for ipython executor (#1748)
* checkpoint async based

* Implement jupyter client and use jupyer gateway

* update deps

* address comments

* add missing parenthesis

* Update build.yml

* CI fixes

* change requirement name

* debug

* print stderr

* dont seek

* show token

* mitigaton for windows bug

* use hex token to avoid - in token

* formatting

* put back in place original while the windows bug exists

* lint

* Update autogen/coding/jupyter_code_executor.py

* Update jupyter_code_executor.py

* Update test_embedded_ipython_code_executor.py

* Update setup.py

* Update build.yml

* fix nameerror

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2024-02-23 19:09:59 +00:00
Eric Zhu
609ba7c649
Code executors (#1405)
* code executor

* test

* revert to main conversable agent

* prepare for pr

* kernel

* run open ai tests only when it's out of draft status

* update workflow file

* revert workflow changes

* ipython executor

* check kernel installed; fix tests

* fix tests

* fix tests

* update system prompt

* Update notebook, more tests

* notebook

* raise instead of return None

* allow user provided code executor.

* fixing types

* wip

* refactoring

* polishing

* fixed failing tests

* resolved merge conflict

* fixing failing test

* wip

* local command line executor and embedded ipython executor

* revert notebook

* fix format

* fix merged error

* fix lmm test

* fix lmm test

* move warning

* name and description should be part of the agent protocol, reset is not as it is only used for ConversableAgent; removing accidentally commited file

* version for dependency

* Update autogen/agentchat/conversable_agent.py

Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>

* ordering of protocol

* description

* fix tests

* make ipython executor dependency optional

* update document optional dependencies

* Remove exclude from Agent protocol

* Make ConversableAgent consistent with Agent

* fix tests

* add doc string

* add doc string

* fix notebook

* fix interface

* merge and update agents

* disable config usage in reply function

* description field setter

* customize system message update

* update doc

---------

Co-authored-by: Davor Runje <davor@airt.ai>
Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>
Co-authored-by: Aaron <aaronlaptop12@hotmail.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-02-10 04:52:16 +00:00
Jack Gerrits
8ece258c54
Add mypy config to pyproject.toml (#1543)
* Add mypy config to pyproject.toml

* put back follow_imports

* ignore math utils
2024-02-07 22:15:07 +00:00
Joshua Kim
c603ca434e
Graph group chat (#857)
* Move contrib-openai.yml

* Moved groupgroupchat

* From #753

* Removed local test references

* Added ignore=test/agentchat/contrib

* Trying to pass contrib-openai tests

* More specific in unit testing.

* Update .github/workflows/contrib-tests.yml

Co-authored-by: Li Jiang <lijiang1@microsoft.com>

* Remove coverage as it is included in test dependencies

* Improved docstring with overview of GraphGroupChat

* Iterate on feedback

* Precommit pass

* user just use pip install pyautogen[graphs]

* Pass precommit

* Pas precommit

* Graph utils an test completed

* Added inversion tests

* Added inversion util

* allow_repeat_speaker can be a list of Agents

* Remove unnessary imports

* Expect ValueError with 1 and 0 agents

* Check that main passes all tests

* Check main

* Pytest all in main

* All done

* pre-commit changes

* noqa E402

* precommit pass

* Removed bin

* Removed old unit test

* Test test_graph_utils

* minor cleanup

* restore tests

* Correct documentation

* Special case of only one agent remaining.

* Improved pytest

* precommit pass

* Delete OAI_CONFIG_LIST_sample copy

* Returns a filtered list for auto to work

* Rename var speaker_order_dict

* To write test cases

* Added check for a list of Agents to repeat

* precommit pass

* Update documentation

* Extract names in allow_repeat_speaker

* Post review changes

* hange "pull_request_target" into "pull_request" temporarily.

* 3 return values from main

* pre-commit changes

* PC edits

* docstr changes

* PC edits

* Rest of changes from main

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Remove unnecessary script files from tracking

* Non empty scripts files from main

* Revert changes in script files to match main branch

* Removed link from website as notebook is removed.

* test/test_graph_utils.py is tested as part of L52 of build.yml

* GroupChat ValueError check

* docstr update

* More clarification in docstr

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* Update autogen/agentchat/groupchat.py

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* 1.add commit to line138 in groupchat.py;2.fix bug if random choice [];3.return selected_agent if len(graph_eligible_agents) is 1;4.replace all speaker_order to speaker_transitions;5.format

* fix graph_modelling notebook in the last cell

* fix failure in test_groupchat.py

* fix agent out of group to initiate a chat like SocietyOfMind

* add a warning rule in graph_utils to check duplicates in any lists

* refactor allowed_or_disallowed_speaker_transitions to Dict[Agent, List[Agent]] and modify the tests and notebook

* delete Rule 4 in graph_utils and related test case. Add a test to resolve 993fd006e9 (r1460726831)

* fix as the final comments

* modify setup option from graphs to graph and add texts in optional-dependencies.md

* Update autogen/graph_utils.py

---------

Co-authored-by: Li Jiang <lijiang1@microsoft.com>
Co-authored-by: Beibin Li <BeibinLi@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
Co-authored-by: Qingyun Wu <qingyun0327@gmail.com>
Co-authored-by: Yishen Sun <freedeaths@FREEDEATHS-XPS>
Co-authored-by: freedeaths <register917@gmail.com>
2024-02-06 03:13:18 +00:00
Davor Runje
101d7fbf17
limited version of pytest to under 8.0.0 (#1437) 2024-01-28 00:52:18 +00:00
afourney
708eb4d884
Add a web surfer agent that can search and browse the web. (#1093)
* Initial commit of WebSurfer. Adds the browser_utils, and related tests. WebSurfer will be added in a subsequent commit.

* Added the web surfer agent, and related tests.

* Added a notebook to show how WebSurferAgent works.

* Fixed a typo.

* Updated test_web_surfer for compatibility with #1110.

* Updated skip_oai logic.

* Fixed code formatting.

* More pre-commit fixes.

* Added block to contrib-openai.yml

* Added block to contrib-openai.yml

* Added hook for BING_API_KEY

* Temporarily commented out other tests, per request.

* Fixed indentation (maybe?)

* Restoring contrib-openai.yml
2024-01-22 03:43:15 +00:00
Vijay Ramesh
ee6ad8d519
[Core] implement redis cache mode (#1222)
* implement redis cache mode, if redis_url is set in the llm_config then
it will try to use this.  also adds a test to validate both the existing
and the redis cache behavior.

* PR feedback, add unit tests

* more PR feedback, move the new style cache to a context manager

* Update agent_chat.md

* more PR feedback, remove tests from contrib and have them run with the normal jobs

* doc

* updated

* Update website/docs/Use-Cases/agent_chat.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* update docs

* update docs; let openaiwrapper to use cache object

* typo

* Update website/docs/Use-Cases/enhanced_inference.md

Co-authored-by: Chi Wang <wang.chi@microsoft.com>

* save previous client cache and reset it after send/a_send

* a_run_chat

---------

Co-authored-by: Vijay Ramesh <vijay@regrello.com>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-01-20 17:06:29 +00:00
olgavrou
a911d1c2ec
set use_docker to default to True (#1147)
* set use_docker to default to true

* black formatting

* centralize checking and add env variable option

* set docker env flag for contrib tests

* set docker env flag for contrib tests

* better error message and cleanup

* disable explicit docker tests

* docker is installed so can't check for that in test

* pr comments and fix test

* rename and fix function descriptions

* documentation

* update notebooks so that they can be run with change in default

* add unit tests for new code

* cache and restore env var

* skip on windows because docker is running in the CI but there are problems connecting the volume

* update documentation

* move header

* update contrib tests
2024-01-18 17:03:49 +00:00
Eric Zhu
22e36cbb10
Support for Python 3.12 (#1317)
* support for Python 3.12

* add python 3.12 to workflows

* version string fix

* retrieval chat

* teachability

* workflow

* redistribute notebook tests

* fix incorrect notebook introduction

* update banner; remove unused imports; update openai workflow

---------

Co-authored-by: Chi Wang <wang.chi@microsoft.com>
2024-01-18 07:15:06 +00:00
Linxin Song
e673500129
[AutoBuild] Supporting build agents from library; supporting generating agent descriptions (#1039)
* try to fix blog

* modify blog

* fix test error in #717; fix blog typo in installation; update blogs with output examples.

* pre-commit

* pre-commit

* Update website/blog/2023-11-26-Agent-AutoBuild/index.mdx

Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>

* add future work

* fix grammar

* update agent_builder

* solve #941; add detailed debug info; support json string config

* pre-commit

* solve #954

* pre-commit

* [new feature] build group chat agents from library.

* pre-commit

* add authors' info in notebook; add a new notebook for build_from_library; reduce prompt effort

* update test and example for build_from_library

* pre-commit

* add notebook; update docs

* change notebook name

* change description for notebook and doc

* remove default value for default_llm_config

* add embedding similarity agent selection

* pre-commit

* update test

* add dependency installation in github workflow

* update test

* pre-commit

* update notebook

* support directly json as library; support customize embedding model

* update test

* pre-commit

* update github test workflow

* Update autobuild_agent_library.ipynb

* add agent description

* refine prompt; update notebook

* pre-commit

* update test example

* update test

* update test

* update test

* change `config_path` to `config_path_or_env`; update test

* pre-commit

* update test

* update test

* update test: add config_file_location

* change `config_path_or_env` to `config_file_or_env`

* update test

* solve noqa

* fix import error for conftest

* fix test error

* pre-commit

* * update error message in `_create_agent`.
* replace `gpt-4-1106-preview` to `gpt-4` in test file.

* add comment on local server creation; modify notebook; update contrib-openai.yml for test; add autobuild option in setup.py; add autotest model name statement

* move import huggingface_hub to _create_agent

* pre-commit

* add uncover comment in the endpoint creation code block

* recover contrib-openai.yml for merge

---------

Co-authored-by: Jieyu Zhang <jieyuz2@cs.washington.edu>
Co-authored-by: Qingyun Wu <qingyun.wu@psu.edu>
2024-01-06 17:23:23 +00:00
Eric Zhu
14a9672032
Update setup.py (#1071)
Fix dependency error introduced when merging #1044
2023-12-27 18:58:19 +00:00
Davor Runje
cb6512dd5a
Add default value to the logprobs parameter (#1044)
* added default value to logprobs param

* fixed to work withversions of openai below 1.5.0

* rebase

* fixed openai version in comments

* polishing

* limit openai version to below 1.5.0

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
2023-12-25 16:36:58 +00:00
Davor Runje
4b5ec5a52f
Add decorator for function calling (#1018)
* add function decorator to converasble agent

* polishing

* polishing

* added function decorator to the notebook with async function calls

* added support for return type hint and JSON encoding of returned value if needed

* polishing

* polishing

* refactored async case

* Python 3.8 support added

* polishing

* polishing

* missing docs added

* refacotring and changes as requested

* getLogger

* documentation added

* test fix

* test fix

* added testing of agentchat_function_call_currency_calculator.ipynb to test_notebook.py

* added support for Pydantic parameters in function decorator

* polishing

* Update website/docs/Use-Cases/agent_chat.md

Co-authored-by: Li Jiang <bnujli@gmail.com>

* Update website/docs/Use-Cases/agent_chat.md

Co-authored-by: Li Jiang <bnujli@gmail.com>

* fixes problem with logprob parameter in openai.types.chat.chat_completion.Choice added by openai version 1.5.0

* get 100% code coverage on code added

* updated docs

* default values added to JSON schema

* serialization using json.dump() add for values not string or BaseModel

* added limit to openai version because of breaking changes in 1.5.0

* added line-by-line comments in docs to explain the process

* polishing

---------

Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Li Jiang <bnujli@gmail.com>
2023-12-25 16:07:20 +00:00
Chi Wang
a31b240100
bump version to 0.2.2 (#906)
* bump version to 0.2.2

* openai version
2023-12-10 03:16:18 +00:00