Introduce a sample chat application using AgentChat and FastAPI,
demonstrating single-agent and team chat functionalities, along with
state persistence and conversation history management.
Resolves#5423
---------
Co-authored-by: Victor Dibia <victor.dibia@gmail.com>
Co-authored-by: Victor Dibia <victordibia@microsoft.com>
Presently MagenticOne and the m1 CLI use the LocalCommandLineExecutor
(presumably copied from the agbench code, which already runs in Docker).
This pr defaults m1 to Docker, and adds a code_executor parameter to
MagenticOne, which defaults to local for now to maintain backward
compatibility -- but this behavior is immediately deprecated.
- Updated HumanEval template to use AgentChat
- Update templates to use config.yaml for model and other configuration
- Read environment from ENV.yaml (ENV.json still supported but
deprecated)
- Temporarily removed WebArena and AssistantBench. Neither had viable
Templates after `autogen_magentic_one` was removed. Templates need to be
update to AgentChat (in a future PR, but this PR is getting big enough
already)
A series of changes to the
`python/packages/autogen-ext/src/autogen_ext/agents/web_surfer/_multimodal_web_surfer.py`
file have been made to better support smaller models.
This includes changes to the prompts, state descriptions, and ordering
of messages.
Regression tasks with OpenAI models shows no change in GAIA scores,
while scores for Llama are significantly improved.
Get's SelectorGroupChat working for llama by:
1. Using a UserMessage rather than a SystemMessage
2. Normalizing how roles are presented (one agent per line)
3. Normalizing how the transcript is constructed (a blank line between
every message)
This PR fixes:
A prompting bug when no control had focus.
Awkward prompt phrasing.
Renamed page_down to scroll_down to better match other prompting and
agent descriptions.
Some agent descriptions were split over multiple lines in the M1
orchestrator. This PR ensures that each description appears on one, and
only one, line. This makes it easier for smaller models to understand.
The Tuple class is never used in CountDownAgent class.
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
<!-- For example: "Closes #1234" -->
## Checks
- [x] I've included any doc changes needed for
https://microsoft.github.io/autogen/. See
https://microsoft.github.io/autogen/docs/Contribute#documentation to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
Co-authored-by: Victor Dibia <victordibia@microsoft.com>
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
Currently the way to accomplish RAG behavior with agent chat,
specifically assistant agents is with the memory interface, however
there is no way to configure it via the declarative API.
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
<!-- For example: "Closes #1234" -->
## Checks
- [ ] I've included any doc changes needed for
https://microsoft.github.io/autogen/. See
https://microsoft.github.io/autogen/docs/Contribute#documentation to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
---------
Co-authored-by: Victor Dibia <chuvidi2003@gmail.com>
Just fix typo
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
<!-- For example: "Closes #1234" -->
## Checks
- [x] I've included any doc changes needed for
https://microsoft.github.io/autogen/. See
https://microsoft.github.io/autogen/docs/Contribute#documentation to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
<!-- Please give a short summary of the change and the problem this
solves. -->
To allow serialization of OAI Assistant Agent.
## Related issue number
<!-- For example: "Closes #1234" -->
Closes#5130
## Checks
- [ ] I've included any doc changes needed for
https://microsoft.github.io/autogen/. See
https://microsoft.github.io/autogen/docs/Contribute#documentation to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
Fix permission issue in ags windows (env files now stored in a user
directory)
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
<!-- For example: "Closes #1234" -->
Closes#5355
## Checks
- [ ] I've included any doc changes needed for
https://microsoft.github.io/autogen/. See
https://microsoft.github.io/autogen/docs/Contribute#documentation to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
Allow AssistantAgent to drop images when not equipped with a multi-modal model.
Adds a corresponding utility function, which can be used in autogen-ext and teams, to accomplish the same.
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
Wrong file names in the README.md
## Related issue number
## Checks
- [x] I've included any doc changes needed for
https://microsoft.github.io/autogen/. See
https://microsoft.github.io/autogen/docs/Contribute#documentation to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
Co-authored-by: Mohammad Mazraeh <Mazraeh.Mohammad@Gmail.com>
This PR does the following:
- Fix warning messages in AGS on launch.
- Improve Cli message to include app URL on startup from command line
- Minor improvements default gallery generator. (add more default tools)
- Improve new session behaviour.
## Related issue number
Closes#5097
## Checks
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
Fix bug where the `model_info` field is not serialized for the
`OpenAIChatCompletionClient` class. This was because the `_raw_config`
field was based on a version of the args that had been sanitized
(model_info removed). We need the full model info field for non-openai
models
```python
from autogen_ext.agents.web_surfer import MultimodalWebSurfer
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_core.models import ModelInfo
mistral_vllm_model = OpenAIChatCompletionClient(
model="TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
base_url="http://localhost:1234/v1",
api_key="empty",
model_info=ModelInfo(vision=False, function_calling=True, json_output=False, family="unkown"),
)
(mistral_vllm_model.dump_component().model_dump_json())
```
Before
```
{
"provider": "autogen_ext.models.openai.OpenAIChatCompletionClient",
"component_type": "model",
"version": 1,
"component_version": 1,
"description": "Chat completion client for OpenAI hosted models.",
"label": "OpenAIChatCompletionClient",
"config": {
"model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
"api_key": "empty",
"base_url": "http://localhost:1234/v1"
}
}
```
After
```
{
"provider": "autogen_ext.models.openai.OpenAIChatCompletionClient",
"component_type": "model",
"version": 1,
"component_version": 1,
"description": "Chat completion client for OpenAI hosted models.",
"label": "OpenAIChatCompletionClient",
"config": {
"model": "TheBloke/Mistral-7B-Instruct-v0.2-GGUF",
"api_key": "empty",
"model_info": {
"vision": false,
"function_calling": true,
"json_output": false,
"family": "unkown"
},
"base_url": "http://localhost:1234/v1"
}
}
```
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
<!-- For example: "Closes #1234" -->
## Checks
- [ ] I've included any doc changes needed for
https://microsoft.github.io/autogen/. See
https://microsoft.github.io/autogen/docs/Contribute#documentation to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
## Why are these changes needed?
This pull request introduces the 'o3' model family and adds support for
the 'o3-mini' model.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
<!-- Please give a short summary of the change and the problem this
solves. -->
We are seeing this issue more often now, probably related to the load on
the API servers. Hence this PR:
1. Demotes the the `max_consecutive_empty_chunk_tolerance` parameter
from function to inline threshold
2. Change exception to a one time warning
## Related issue number
<!-- For example: "Closes #1234" -->
## Checks
- [x] I've included any doc changes needed for
https://microsoft.github.io/autogen/. See
https://microsoft.github.io/autogen/docs/Contribute#documentation to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
---------
Signed-off-by: Mohammad Mazraeh <mazraeh.mohammad@gmail.com>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
This PR adds a method that approximately extracts the text visible in
the viewport of the web browser (as opposed to always printing the first
50 lines, or relying entirely on OCR).
This PR removes the older `autogen_magentic_one` package, and directs
people to use the new AgentChat implementation.
Hopefully this eases confusion.
---------
Co-authored-by: Jack Gerrits <jack@jackgerrits.com>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
* New sample of chess playing showing R1's thought process in streaming
mode
* Modify existing samples to use `model_config.yml` instead of JSON
configs for better clarity.
---------
Co-authored-by: Mohammad Mazraeh <Mazraeh.Mohammad@Gmail.com>
* Separate agent and team examples
* Add streaming output
* Refactor to better use the chainlit API
* Removed the user proxy example -- this needs a bit more work to
improve the presentation on the ChainLit interface.
---------
Co-authored-by: Victor Dibia <victordibia@microsoft.com>
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
This PR updates AGS to use the declarative config serialization native
to AgentChat.
The effect? You can build your teams/artifacts directly in python, run
`team.dump_component()` and immediately run it in AGS.
Some change details:
- Removes ComponentFactory. Instead TeamManager just loads team specs
directly using `Team.load_component`.
- Some fixes to the UI to simplify drag and drop experience.
- Improve layout of nodes...
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
<!-- For example: "Closes #1234" -->
Closes#4439Closes#5172
## Checks
- [ ] I've included any doc changes needed for
https://microsoft.github.io/autogen/. See
https://microsoft.github.io/autogen/docs/Contribute#documentation to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
cc @EItanya @nour-bouzid
