<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
These changes are needed to expand AutoGen's memory capabilities with a
robust, production-ready integration with Mem0.ai.
<!-- Please give a short summary of the change and the problem this
solves. -->
This PR adds a new memory component for AutoGen that integrates with
Mem0.ai, providing a robust memory solution that supports both cloud and
local backends. The Mem0Memory class enables agents to store and
retrieve information persistently across conversation sessions.
## Key Features
- Seamless integration with Mem0.ai memory system
- Support for both cloud-based and local storage backends
- Robust error handling with detailed logging
- Full implementation of AutoGen's Memory interface
- Context updating for enhanced agent conversations
- Configurable search parameters for memory retrieval
## Related issue number
<!-- For example: "Closes #1234" -->
## Checks
- [x] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
---------
Co-authored-by: Victor Dibia <victordibia@microsoft.com>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Ricky Loynd <riloynd@microsoft.com>
## Why are these changes needed?
This PR adds support for configurable embedding functions in
ChromaDBVectorMemory, addressing the need for users to customize how
embeddings are generated for vector similarity search. Currently,
ChromaDB memory is limited to default embedding functions, which
restricts flexibility for different use cases that may require specific
embedding models or custom embedding logic.
The implementation allows users to:
- Use different SentenceTransformer models for domain-specific
embeddings
- Integrate with OpenAI's embedding API for consistent embedding
generation
- Define custom embedding functions for specialized requirements
- Maintain backward compatibility with existing default behavior
## Related issue number
Closes#6267
## Checks
- [x] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests corresponding to the changes introduced in this
PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Victor Dibia <victordibia@microsoft.com>
Co-authored-by: Victor Dibia <victor.dibia@gmail.com>
Add OTel GenAI traces:
- `create_agent`
- `invoke_agnet`
- `execute_tool`
Introduces context manager helpers to create these traces. The helpers
also serve as instrumentation points for other instrumentation
libraries.
Resolves#6644
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
MCP Python-sdk has started to support a new transport protocol named
`Streamble HTTP` since
[v1.8.0](https://github.com/modelcontextprotocol/python-sdk/releases/tag/v1.8.0)
last month. I heard it supersedes the SSE transport. Therefore, AutoGen
have to support it as soon as possible.
## Related issue number
https://github.com/microsoft/autogen/discussions/6517
## Checks
- [x] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Victor Dibia <victordibia@microsoft.com>
Co-authored-by: Victor Dibia <victor.dibia@gmail.com>
## Why are these changes needed?
This PR introduces a new `OpenAIAgent` implementation that uses the
[OpenAI Response
API](https://platform.openai.com/docs/guides/responses-vs-chat-completions)
as its backend. The OpenAI Assistant API will be deprecated in 2026, and
the Response API is its successor. This change ensures our codebase is
future-proof and aligned with OpenAI’s latest platform direction.
### Motivation
- **Deprecation Notice:** The OpenAI Assistant API will be deprecated in
2026.
- **Future-Proofing:** The Response API is the recommended replacement
and offers improved capabilities for stateful, multi-turn, and
tool-augmented conversations.
- **AgentChat Compatibility:** The new agent is designed to conform to
the behavior and expectations of `AssistantAgent` in AgentChat, but is
implemented directly on top of the OpenAI Response API.
### Key Changes
- **New Agent:** Adds `OpenAIAgent`, a stateful agent that interacts
with the OpenAI Response API.
- **Stateful Design:** The agent maintains conversation state, tool
usage, and other metadata as required by the Response API.
- **AssistantAgent Parity:** The new agent matches the interface and
behavior of `AssistantAgent` in AgentChat, ensuring a smooth migration
path.
- **Direct OpenAI Integration:** Uses the official `openai` Python
library for all API interactions.
- **Extensible:** Designed to support future enhancements, such as
advanced tool use, function calling, and multi-modal capabilities.
### Migration Path
- Existing users of the Assistant API should migrate to the new
`OpenAIAgent` to ensure long-term compatibility.
- Documentation and examples will be updated to reflect the new agent
and its usage patterns.
### References
- [OpenAI: Responses vs. Chat
Completions](https://platform.openai.com/docs/guides/responses-vs-chat-completions)
- [OpenAI Deprecation
Notice](https://platform.openai.com/docs/guides/responses-vs-chat-completions#deprecation-timeline)
---
## Related issue number
Closes#6032
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [X] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [X] I've made sure all auto checks have passed.
Co-authored-by: Griffin Bassman <griffinbassman@gmail.com>
## Why are these changes needed?
Enables usage statistics for streaming responses by default.
There is a similar bug in the AzureAI client. Theoretically adding the
parameter
```
model_extras={"stream_options": {"include_usage": True}}
```
should fix the problem, but I'm currently unable to test that workflow
## Related issue number
closes https://github.com/microsoft/autogen/issues/6548
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
There have been updates to the azure ai agent foundry sdk
(azure-ai-project). This PR updates the autogen `AzureAIAgent` which
wraps the azure ai agent. A list of some changes
- Update docstring samples to use `endpoint` (instead of connection
string previously)
- Update imports and arguments e.g, from `azure.ai.agents` etc
- Add a guide in ext docs showing Bing Search Grounding tool example.
<img width="1423" alt="image"
src="https://github.com/user-attachments/assets/0b7c8fa6-8aa5-4c20-831b-b525ac8243b7"
/>
## Why are these changes needed?
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
Closes#6601
<!-- For example: "Closes #1234" -->
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
## Why are these changes needed?
I added `created_at` to both BaseChatMessage and BaseAgentEvent classes
that store the time these Pydantic model instances are generated. And
then users will be able to use `created_at` to build up a customized
external persisting state management layer for their case.
## Related issue number
https://github.com/microsoft/autogen/discussions/6169#discussioncomment-13151540
## Checks
- [x] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Jack Gerrits <jackgerrits@users.noreply.github.com>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
<!-- Please give a short summary of the change and the problem this
solves. -->
The `LocalCommandLineCodeExecutor `creates temporary files for each code
execution, which can accumulate over time and clutter the filesystem -
especially when a temporary working directory is not used. These changes
introduce an option to automatically delete temporary files after
execution, helping to prevent file system debris, reduce disk usage, and
ensure cleaner runtime environments in long-running or repeated
execution scenarios.
## Related issue number
Closes#4380
## Checks
- [x] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
## Why are these changes needed?
Simplified the azure ai search tool and fixed bugs in the code
## Related issue number
"Closes #6430 "
## Checks
- [X] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [X] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [X] I've made sure all auto checks have passed.
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
## Why are these changes needed?
FIX/mistral could not recive name field, so add model transformer for
mistral
## Related issue number
Closes#6147
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
…ized properly
## Why are these changes needed?
The exceptions thrown by MCP server tools weren't being serialized
properly - the user would see `[{}, {}, ... {}]` instead of an actual
error/exception message.
## Related issue number
Fixes#6481
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Signed-off-by: Peter Jausovec <peter.jausovec@solo.io>
Co-authored-by: Victor Dibia <victordibia@microsoft.com>
Co-authored-by: Victor Dibia <victor.dibia@gmail.com>
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
Nice to have functionality
## Related issue number
Closes#6060
## Checks
- [x] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
As-is: Deleting `McpWorkbench` does not close the `McpSession`.
To-be: Deleting `McpWorkbench` now properly closes the `McpSession`.
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
<!-- For example: "Closes #1234" -->
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
Some fixes with the AnthropicBedrockChatCompletionClient
- Ensure `AnthropicBedrockChatCompletionClient` exported and can be
imported.
- Update the BedrockInfo keys serialization - client argument can be
string (similar to api key in this ) but exported config should be
Secret
- Replace `AnthropicBedrock` with `AsyncAnthropicBedrock` : client
should be async to work with the ag stack and the BaseAnthropicClient it
inherits from
- Improve `AnthropicBedrockChatCompletionClient` docstring to use the
correct client arguments rather than serialized dict format.
Expect
```python
from autogen_ext.models.anthropic import AnthropicBedrockChatCompletionClient, BedrockInfo
from autogen_core.models import UserMessage, ModelInfo
async def main():
anthropic_client = AnthropicBedrockChatCompletionClient(
model="anthropic.claude-3-5-sonnet-20240620-v1:0",
temperature=0.1,
model_info=ModelInfo(vision=False, function_calling=True,
json_output=False, family="unknown", structured_output=True),
bedrock_info=BedrockInfo(
aws_access_key="<aws_access_key>",
aws_secret_key="<aws_secret_key>",
aws_session_token="<aws_session_token>",
aws_region="<aws_region>",
),
)
# type: ignore
result = await anthropic_client.create([UserMessage(content="What is the capital of France?", source="user")])
print(result)
await main()
```
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
<!-- For example: "Closes #1234" -->
Closes#6483
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
## Why are these changes needed?
Our McpWorkbench required properties however mongodb-lens's some tools
do not has it. I will fix it from when properties is None, -> {}
Our McpWorkbench now does not have stop routine with without async with
McpWorkbench(params) as workbench: and lazy init. So, I will adding def
__del__: pass just insert that, It could show error.
## Related issue number
Closes#6425
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
## Why are these changes needed?
Multimodal message fill context with other routine. However current
`_set_empty_to_whitespace` is fill with context.
So, error occured.
And, I checked `multimodal_user_transformer_funcs` and I found it, in
this routine, context must not be empty.
Now remove the `_set_empty_to_whitespace` when multimodal message,
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
Closes#6439
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
## Why are these changes needed?
Current autogen-ext's test is too slow.
So, I will search slow test case and makes more fast.
[init docker executor function to module
180s->140s](a3cf70bcf8)
[reuse executor at some tests
140s->120s](ca15938afa)
[Remove unnecessary start of docker
120s->110s](61247611e0)
## Related issue number
<!-- For example: "Closes #1234" -->
Part of #6376
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Adding support for Bing grounding citations to the AzureAIAgent.
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
<!-- Please give a short summary of the change and the problem this
solves. -->
## Related issue number
<!-- For example: "Closes #1234" -->
## Checks
- [X] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [X] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [X] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Co-authored-by: Dheeraj Bandaru <BandaruDheeraj@users.noreply.github.com>
## Why are these changes needed?
*Problem*
Previously, in `DockerCommandLineCodeExecutor`, cancellation tasks were
added directly to `self._cancellation_tasks` using
`asyncio.create_task()`:
```python
self._cancellation_tasks.append(asyncio.create_task(self._kill_running_command(command)))
```
This caused issues when cancellation tasks were created from multiple
event loops, leading to loop mismatch errors during executor shutdown.
*Solution*
This PR fixes the issue by introducing a dedicated internal event loop
for managing cancellation tasks.
Cancellation tasks are now scheduled in a fixed event loop using
`asyncio.run_coroutine_threadsafe()`:
```python
future: ConcurrentFuture[None] = asyncio.run_coroutine_threadsafe(
self._kill_running_command(command), self._loop
)
self._cancellation_futures.append(future)
```
*Additional Changes*
- Added detailed logging for easier debugging.
- Ensured clean shutdown of the internal event loop and associated
thread.
*Note*
This change ensures that all cancellation tasks are handled consistently
in a single loop, preventing cross-loop conflicts and improving executor
stability in multi-threaded environments.
## Related issue number
<!-- For example: "Closes #1234" -->
Closes#6395
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
## Why are these changes needed?
- Add return_value_as_string for formating result from MCP tool
## Related issue number
- Opened Issue on #6368
## Checks
- [x] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
## Why are these changes needed?
> The pytest tests test_local_executor_with_custom_venv and
test_local_executor_with_custom_venv_in_local_relative_path located in
packages/autogen-ext/tests/code_executors/test_commandline_code_executor.py
fail when run on macOS (aarch64) using a Python interpreter managed by
uv (following the project's recommended development setup).
>
> The failure occurs during the creation of a nested virtual environment
using Python's standard venv.EnvBuilder. Specifically, the attempt to
run ensurepip inside the newly created venv fails immediately with a
SIGABRT signal. The root cause appears to be a dynamic library loading
error (dyld error) where the Python executable inside the newly created
venv cannot find its required libpythonX.Y.dylib shared library.
So, when MacOS + uv case, skipping that test.
And, adding uv-venv case
## Related issue number
Closes#6341
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
This PR introduces `WorkBench`.
A workbench provides a group of tools that share the same resource and
state. For example, `McpWorkbench` provides the underlying tools on the
MCP server. A workbench allows tools to be managed together and abstract
away the lifecycle of individual tools under a single entity. This makes
it possible to create agents with stateful tools from serializable
configuration (component configs), and it also supports dynamic tools:
tools change after each execution.
Here is how a workbench may be used with AssistantAgent (not included in
this PR):
```python
workbench = McpWorkbench(server_params)
agent = AssistantAgent("assistant", tools=workbench)
result = await agent.run(task="do task...")
```
TODOs:
1. In a subsequent PR, update `AssistantAgent` to use workbench as an
alternative in the `tools` parameter. Use `StaticWorkbench` to manage
individual tools.
2. In another PR, add documentation on workbench.
---------
Co-authored-by: EeS <chiyoung.song@motov.co.kr>
Co-authored-by: Minh Đăng <74671798+perfogic@users.noreply.github.com>
## Why are these changes needed?
| Package | Test time-Origin (Sec) | Test time-Edited (Sec) |
|-------------------------|------------------|-----------------------------------------------|
| autogen-studio | 1.64 | 1.64 |
| autogen-core | 6.03 | 6.17 |
| autogen-ext | 387.15 | 373.40 |
| autogen-agentchat | 54.20 | 20.67 |
## Related issue number
Related #6361
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
- Added the support Azure AI Agent. The new agent is named AzureAIAgent.
- The agent supports Bing search, file search, and Azure search tools.
- Added a Jupiter notebook to demonstrate the usage of the AzureAIAgent.
## What's missing?
- AzureAIAgent support only text message responses
- Parallel execution for the custom functions.
## Related issue number
[5545](https://github.com/microsoft/autogen/issues/5545#event-16626859772)
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
## Why are these changes needed?
`convert_tools` failed if Optional args were used in tools (the `type`
field doesn't exist in that case and `anyOf` must be used).
This uses the `anyOf` field to pick the first non-null type to use.
## Related issue number
Fixes#6323
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Signed-off-by: Peter Jausovec <peter.jausovec@solo.io>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
The current implementation of consecutive `SystemMessage` merging
applies only to models where `model_info.family` starts with
`"gemini-"`.
Since PR #6327 introduced the `multiple_system_messages` field in
`model_info`, we can now generalize this logic by checking whether the
field is explicitly set to `False`.
This change replaces the hardcoded family check with a conditional that
merges consecutive `SystemMessage` blocks whenever
`multiple_system_messages` is set to `False`.
Test cases that previously depended on the `"gemini"` model family have
been updated to reflect this configuration flag, and renamed accordingly
for clarity.
In addition, for consistency across conditional logic, a follow-up PR is
planned to refactor the Claude-specific transformation condition
(currently implemented via `create_args.get("model",
"unknown").startswith("claude-")`)
to instead use the existing `is_claude()`.
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
<!-- Thank you for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->
## Why are these changes needed?
This is an initial exploration of what could be a solution for #6214 .
It implements a simple text canvas using difflib and also a memory
component and a tool component for interacting with the canvas. Still in
early testing but would love feedback on the design.
## Related issue number
<!-- For example: "Closes #1234" -->
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [ ] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ ] I've made sure all auto checks have passed.
---------
Co-authored-by: Leonardo Pinheiro <lpinheiro@microsoft.com>
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
This PR introduces a safer and more controllable execution environment
for LLM code execution in version 0.4 by enabling the use of Jupyter
inside a container. This enhancement addresses security concerns and
provides a more robust execution context. In particular, it allows:
Isolation of code execution via containerized Jupyter environments.
Persistent memory of variables and their values throughout the
conversation.
Memory of code execution results to support more advanced reasoning and
follow-up tasks.
These improvements help build a more interactive and stateful LLM-agent
programming experience, especially for iterative code generation and
debugging scenarios.
## Related issue number
Open #6153
## Checks
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Resolves#6232, #6198
This PR introduces an optional parameter `session` to `mcp_server_tools`
to support reuse of the same session.
```python
import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.conditions import TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.ui import Console
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_ext.tools.mcp import StdioServerParams, create_mcp_server_session, mcp_server_tools
async def main() -> None:
model_client = OpenAIChatCompletionClient(model="gpt-4o", parallel_tool_calls=False) # type: ignore
params = StdioServerParams(
command="npx",
args=["@playwright/mcp@latest"],
read_timeout_seconds=60,
)
async with create_mcp_server_session(params) as session:
await session.initialize()
tools = await mcp_server_tools(server_params=params, session=session)
print(f"Tools: {[tool.name for tool in tools]}")
agent = AssistantAgent(
name="Assistant",
model_client=model_client,
tools=tools, # type: ignore
)
termination = TextMentionTermination("TERMINATE")
team = RoundRobinGroupChat([agent], termination_condition=termination)
await Console(
team.run_stream(
task="Go to https://ekzhu.com/, visit the first link in the page, then tell me about the linked page."
)
)
asyncio.run(main())
```
Based on discussion in this thread: #6284, we will consider
serialization and deserialization of MCP server tools when used in this
manner in a separate issue.
This PR also replaces the `json_schema_to_pydantic` dependency with
built-in utils.
## Why are these changes needed?
bug fix : add get_embedding() implementation
## Related issue number
"Closes #6240 " -->
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [X] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [X] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
This pull request introduces a new feature to the
`DockerCommandLineCodeExecutor` class, which allows temporary files
generated by code execution to be deleted after code execution. The most
important changes include adding a new configuration option, updating
the class to handle this option, and adding tests to verify the new
functionality.
### New Feature: Temporary File Deletion
*
[`python/packages/autogen-ext/src/autogen_ext/code_executors/docker/_docker_code_executor.py`](diffhunk://#diff-8ef47c21141ed8b0a757b0e6f9d1491561fc31684756d22ed0253edbfcfcdf91R81):
Added `delete_tmp_files` attribute to the
`DockerCommandLineCodeExecutorConfig` class and updated the
`DockerCommandLineCodeExecutor` class to handle this attribute. This
includes initializing the attribute, adding it to the configuration
methods, and implementing the file deletion logic in the
`_execute_code_dont_check_setup` method.
[[1]](diffhunk://#diff-8ef47c21141ed8b0a757b0e6f9d1491561fc31684756d22ed0253edbfcfcdf91R81)
[[2]](diffhunk://#diff-8ef47c21141ed8b0a757b0e6f9d1491561fc31684756d22ed0253edbfcfcdf91R128)
[[3]](diffhunk://#diff-8ef47c21141ed8b0a757b0e6f9d1491561fc31684756d22ed0253edbfcfcdf91R177)
[[4]](diffhunk://#diff-8ef47c21141ed8b0a757b0e6f9d1491561fc31684756d22ed0253edbfcfcdf91R231)
[[5]](diffhunk://#diff-8ef47c21141ed8b0a757b0e6f9d1491561fc31684756d22ed0253edbfcfcdf91R318)
[[6]](diffhunk://#diff-8ef47c21141ed8b0a757b0e6f9d1491561fc31684756d22ed0253edbfcfcdf91R346-R352)
[[7]](diffhunk://#diff-8ef47c21141ed8b0a757b0e6f9d1491561fc31684756d22ed0253edbfcfcdf91R527)
[[8]](diffhunk://#diff-8ef47c21141ed8b0a757b0e6f9d1491561fc31684756d22ed0253edbfcfcdf91R547)
### Testing
*
[`python/packages/autogen-ext/tests/code_executors/test_docker_commandline_code_executor.py`](diffhunk://#diff-635dbdcdeca161e620283399d5cd43ca756ec0f88d4429f059ee4f6b346874e4R318-R363):
Added a new test `test_delete_tmp_files` to verify the behavior of the
`delete_tmp_files` attribute. This test checks that temporary files are
correctly deleted or retained based on the configuration.<!-- Thank you
for your contribution! Please review
https://microsoft.github.io/autogen/docs/Contribute before opening a
pull request. -->
This PR improves fallback safety when an invalid `model_family` is
supplied to `get_transformer()`. Previously, if a user passed an
arbitrary or incorrect `family` string in `model_info`, the lookup could
fail without falling back to `ModelFamily.UNKNOWN`.
Now, we explicitly check whether `model_family` is a valid value in
`ModelFamily.ANY`. If not, we fallback to `_find_model_family()` as
intended.
## Related issue number
Related #6011#issuecomment-2779957730
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Messages sent as part of `LLMCallEvent` for Anthropic were not fully serializable
The example below shows TextBlock and ToolUseBlocks inside the content of messages - these throw downsteam errors in apps like AGS (or event sinks) that expect serializable dicts inside the LLMCallEvent.
```
[
{'role': 'user', 'content': 'What is the weather in New York?'},
{'role': 'assistant', 'content': [TextBlock(citations=None, text='I can help you find the weather in New York. Let me check that for you.', type='text'), ToolUseBlock(id='toolu_016W8g55GejYGBzRRrcsnt7M', input={'city': 'New York'}, name='get_weather', type='tool_use')]},
{'role': 'user', 'content': [{'type': 'tool_result', 'tool_use_id': 'toolu_016W8g55GejYGBzRRrcsnt7M', 'content': 'The weather in New York is 73 degrees and Sunny.'}]}
]
```
This PR attempts to first serialize content of anthropic messages before they are passed to `LLMCallEvent`
```
[
{'role': 'user', 'content': 'What is the weather in New York?'},
{'role': 'assistant', 'content': [{'citations': None, 'text': 'I can help you find the weather in New York. Let me check that for you.', 'type': 'text'}, {'id': 'toolu_016W8g55GejYGBzRRrcsnt7M', 'input': {'city': 'New York'}, 'name': 'get_weather', 'type': 'tool_use'}]},
{'role': 'user', 'content': [{'type': 'tool_result', 'tool_use_id': 'toolu_016W8g55GejYGBzRRrcsnt7M', 'content': 'The weather in New York is 73 degrees and Sunny.'}]}
]
```
# Azure AI Search Tool Implementation
This PR adds a new tool for Azure AI Search integration to autogen-ext,
enabling agents to search and retrieve information from Azure AI Search
indexes.
## Why Are These Changes Needed?
AutoGen currently lacks native integration with Azure AI Search, which
is a powerful enterprise search service that supports semantic, vector,
and hybrid search capabilities. This integration enables agents to:
1. Retrieve relevant information from large document collections
2. Perform semantic search with AI-powered ranking
3. Execute vector similarity search using embeddings
4. Combine text and vector approaches for optimal results
This tool complements existing retrieval capabilities and provides a
seamless way to integrate with Azure's search infrastructure.
## Features
- **Multiple Search Types**: Support for text, semantic, vector, and
hybrid search
- **Flexible Configuration**: Customizable search parameters and fields
- **Robust Error Handling**: User-friendly error messages with
actionable guidance
- **Performance Optimizations**: Configurable caching and retry
mechanisms
- **Vector Search Support**: Built-in embedding generation with
extensibility
## Usage Example
```python
from autogen_ext.tools.azure import AzureAISearchTool
from azure.core.credentials import AzureKeyCredential
from autogen import AssistantAgent, UserProxyAgent
# Create the search tool
search_tool = AzureAISearchTool.load_component({
"provider": "autogen_ext.tools.azure.AzureAISearchTool",
"config": {
"name": "DocumentSearch",
"description": "Search for information in the knowledge base",
"endpoint": "https://your-service.search.windows.net",
"index_name": "your-index",
"credential": {"api_key": "your-api-key"},
"query_type": "semantic",
"semantic_config_name": "default"
}
})
# Create an agent with the search tool
assistant = AssistantAgent(
"assistant",
llm_config={"tools": [search_tool]}
)
# Create a user proxy agent
user_proxy = UserProxyAgent(
"user_proxy",
human_input_mode="TERMINATE",
max_consecutive_auto_reply=10,
code_execution_config={"work_dir": "coding"}
)
# Start the conversation
user_proxy.initiate_chat(
assistant,
message="What information do we have about quantum computing in our knowledge base?"
)
```
## Testing
- Added unit tests for all search types (text, semantic, vector, hybrid)
- Added tests for error handling and cancellation
- All tests pass locally
## Documentation
- Added comprehensive docstrings with examples
- Included warnings about placeholder embedding implementation
- Added links to Azure AI Search documentation
## Related issue number
Closes#5419
## Checks
- [x] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Just simple change.
```python
messages: List[LLMMessage] = [UserMessage(content="Call the pass tool with input 'task'", source="user")]
```
to
```python
messages: List[LLMMessage] = [UserMessage(content="Call the pass tool with input 'task' and talk result", source="user")]
```
And, now.
Anthropic model could pass that test case
`test_model_client_with_function_calling`.
-> Yup. Before, claude could not pass that test case.
With this change, Claude (Anthropic) models are now able to pass the
test case successfully.
Before this fix, Claude failed to interpret the intent correctly. Now,
it can infer both tool usage and follow-up generation.
This change is backward-compatible with other models (e.g., GPT-4) and
improves cross-model consistency for function-calling tests.
This PR improves how model_family is resolved when selecting a
transformer from the registry.
Previously, model families were inferred using a simple prefix-based
match like:
```
if model.startswith(family): ...
```
This works for cleanly prefixed models (e.g., `gpt-4o`, `claude-3`) but
fails for models like `mistral-large-latest`, `codestral-latest`, etc.,
where prefix-based matching is ambiguous or misleading.
To address this:
• model_family can now be passed explicitly (e.g., via ModelInfo)
• _find_model_family() is only used as a fallback when the value is
"unknown"
• Transformer lookup is now more robust and predictable
• Example integration in to_oai_type() demonstrates this pattern using
self._model_info["family"]
This change is required for safe support of models like Mistral and
other future models that do not follow standard naming conventions.
Linked to discussion in
[#6151](https://github.com/microsoft/autogen/issues/6151)
Related : #6011
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
The initializer for ACADynamicSessionsCodeExecutor creates a new GUID to
use as the session ID for dynamic sessions.
In some scenarios it is desirable to be able to re-create the agent
group chat from saved state. In this case, the
ACADynamicSessionsCodeExecutor needs to be associated with a previous
instance (so that any execution state is still valid)
This PR adds a new argument to the initializer to allow a session ID to
be passed in (defaulting to the current behaviour of creating a GUID if
absent).
Closes#6119
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
## Why are these changes needed?
This PR fixes a `400 - invalid_request_error` that occurs when using
Anthropic models and the **final message is from the assistant and ends
with trailing whitespace**.
Example error:
```
Error code: 400 - {'error': {'code': 'invalid_request_error', 'message': 'messages: final assistant content cannot end with trailing whitespace', ...}}
```
To unblock ongoing internal usage, this patch introduces an **ad-hoc
fix** that strips trailing whitespace if the model is Anthropic and the
last message is from the assistant.
## Related issue number
Ad-hoc fix for issue discussed here:
https://github.com/microsoft/autogen/issues/6167
Follow-up structural proposal here:
https://github.com/microsoft/autogen/issues/6167https://github.com/microsoft/autogen/issues/6167#issuecomment-2768592840
## Why are these changes needed?
Changed default working directory of code executors, from the current
directory `"."` to Python's
[`tempfile`](https://docs.python.org/3/library/tempfile.html#tempfile.TemporaryDirectory).
These changes simplify file cleanup and prevent the model from accessing
code files or other sensitive data that should not be accessible.
These changes simplify file cleanup and prevent the model from accessing
code files or other sensitive data that should not be accessible.
Changes made:
- The default `work_dir` parameter in code executors is changed to
`None`; when invoking the `start()` method, if not `work_dir` was
specified (`None`) a temporary directory is created.
- The `start()` and `stop()` methods of code executors handle the
creation and cleanup of the working directory, for the default temporary
directory.
- For maintaining backward compatibility:
- A `DeprecationWarning` is emitted when the current dir, `"."`, is used
as `work_dir` as it is in the current code executor implementation. The
deprecation warning is tested in `test_deprecated_warning()`.
- For existing implementation that do not call the `start()` method and
do not specify a `work_dir`, the executors will continue using the
current directory `"."` as the working directory, mantaining backward
compatibility.
- Updated test suites:
- Added tests to confirm that by default code executors use a temporary
directory as their working directory: `test_default_work_dir_is_temp()`;
- Implemented test to ensure that a `DeprecationWarning` is raised when
the current directory is used as the default directory:
`test_deprecated_warning()`;
- Added tests to ensure that errors arise when invalid paths (doesn't
exist or user has not the right permissions) are provided:
`test_error_wrong_path()`.
Feel free to suggest any additions or improvements!
## Related issue number
Close#6041
## Checks
- [x] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [x] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
## Why are these changes needed?
This change addresses a compatibility issue when using Google Gemini
models with AutoGen. Specifically, Gemini returns a 400 INVALID_ARGUMENT
error when receiving a response with an empty "text" parameter.
The root cause is that Gemini does not accept empty string values (e.g.,
"") as valid inputs in the history of the conversation.
To fix this, if the content field is falsy (e.g., None, "", etc.), it is
explicitly replaced with a single whitespace (" "), which prevents the
Gemini model from rejecting the request.
- **Gemini API compatibility:** Gemini models reject empty assistant
messages (e.g., `""`), causing runtime errors. This PR ensures such
messages are safely replaced with whitespace where appropriate.
- **Avoiding regressions:** Applying the empty content workaround **only
to Gemini**, and **only to valid message types**, avoids breaking OpenAI
or other models.
- **Reducing duplication:** Previously, message transformation logic was
scattered and repeated across different message types and models.
Modularizing this pipeline removes that redundancy.
- **Improved maintainability:** With future model variants likely to
introduce more constraints, this modular structure makes it easier to
adapt transformations without writing ad-hoc code each time.
- **Testing for correctness:** The new structure is verified with tests,
ensuring the bug fix is effective and non-intrusive.
## Summary
This PR introduces a **modular transformer pipeline** for message
conversion and **fixes a Gemini-specific bug** related to empty
assistant message content.
### Key Changes
- **[Refactor]** Extracted message transformation logic into a unified
pipeline to:
- Reduce code duplication
- Improve maintainability
- Simplify debugging and extension for future model-specific logic
- **[BugFix]** Gemini models do not accept empty assistant message
content.
- Introduced `_set_empty_to_whitespace` transformer to replace empty
strings with `" "` only where needed
- Applied it **only** to `"text"` and `"thought"` message types, not to
`"tools"` to avoid serialization errors
- **Improved structure for model-specific handling**
- Transformer functions are now grouped and conditionally applied based
on message type and model family
- This design makes it easier to support future models or combinations
(e.g., Gemini + R1)
- **Test coverage added**
- Added dedicated tests to verify that empty assistant content causes
errors for Gemini
- Ensured the fix resolves the issue without affecting OpenAI models
---
## Motivation
Originally, Gemini-compatible endpoints would fail when receiving
assistant messages with empty content (`""`).
This issue required special handling without introducing brittle, ad-hoc
patches.
In addressing this, I also saw an opportunity to **modularize** the
message transformation logic across models.
This improves clarity, avoids duplication, and simplifies future
adaptations (e.g., different constraints across model families).
---
## 📘 AutoGen Modular Message Transformer: Design & Usage Guide
This document introduces the **new modular transformer system** used in
AutoGen for converting `LLMMessage` instances to SDK-specific message
formats (e.g., OpenAI-style `ChatCompletionMessageParam`).
The design improves **reusability, extensibility**, and
**maintainability** across different model families.
---
### 🚀 Overview
Instead of scattering model-specific message conversion logic across the
codebase, the new design introduces:
- Modular transformer **functions** for each message type
- Per-model **transformer maps** (e.g., for OpenAI-compatible models)
- Optional **conditional transformers** for multimodal/text hybrid
models
- Clear separation between **message adaptation logic** and
**SDK-specific builder** (e.g., `ChatCompletionUserMessageParam`)
---
### 🧱 1. Define Transform Functions
Each transformer function takes:
- `LLMMessage`: a structured AutoGen message
- `context: dict`: metadata passed through the builder pipeline
And returns:
- A dictionary of keyword arguments for the target message constructor
(e.g., `{"content": ..., "name": ..., "role": ...}`)
```python
def _set_thought_as_content_gemini(message: LLMMessage, context: Dict[str, Any]) -> Dict[str, str | None]:
assert isinstance(message, AssistantMessage)
return {"content": message.thought or " "}
```
---
### 🪢 2. Compose Transformer Pipelines
Multiple transformer functions are composed into a pipeline using
`build_transformer_func()`:
```python
base_user_transformer_funcs: List[Callable[[LLMMessage, Dict[str, Any]], Dict[str, Any]]] = [
_assert_valid_name,
_set_name,
_set_role("user"),
]
user_transformer = build_transformer_func(
funcs=base_user_transformer_funcs,
message_param_func=ChatCompletionUserMessageParam
)
```
- The `message_param_func` is the actual constructor for the target
message class (usually from the SDK).
- The pipeline is **ordered** — each function adds or overrides keys in
the builder kwargs.
---
### 🗂️ 3. Register Transformer Map
Each model family maintains a `TransformerMap`, which maps `LLMMessage`
types to transformers:
```python
__BASE_TRANSFORMER_MAP: TransformerMap = {
SystemMessage: system_transformer,
UserMessage: user_transformer,
AssistantMessage: assistant_transformer,
}
register_transformer("openai", model_name_or_family, __BASE_TRANSFORMER_MAP)
```
- `"openai"` is currently required (as only OpenAI-compatible format is
supported now).
- Registration ensures AutoGen knows how to transform each message type
for that model.
---
### 🔁 4. Conditional Transformers (Optional)
When message construction depends on runtime conditions (e.g., `"text"`
vs. `"multimodal"`), use:
```python
conditional_transformer = build_conditional_transformer_func(
funcs_map=user_transformer_funcs_claude,
message_param_func_map=user_transformer_constructors,
condition_func=user_condition,
)
```
Where:
- `funcs_map`: maps condition label → list of transformer functions
```python
user_transformer_funcs_claude = {
"text": text_transformers + [_set_empty_to_whitespace],
"multimodal": multimodal_transformers + [_set_empty_to_whitespace],
}
```
- `message_param_func_map`: maps condition label → message builder
```python
user_transformer_constructors = {
"text": ChatCompletionUserMessageParam,
"multimodal": ChatCompletionUserMessageParam,
}
```
- `condition_func`: determines which transformer to apply at runtime
```python
def user_condition(message: LLMMessage, context: Dict[str, Any]) -> str:
if isinstance(message.content, str):
return "text"
return "multimodal"
```
---
### 🧪 Example Flow
```python
llm_message = AssistantMessage(name="a", thought="let’s go")
model_family = "openai"
model_name = "claude-3-opus"
transformer = get_transformer(model_family, model_name, type(llm_message))
sdk_message = transformer(llm_message, context={})
```
---
### 🎯 Design Benefits
| Feature | Benefit |
|--------|---------|
| 🧱 Function-based modular design | Easy to compose and test |
| 🧩 Per-model registry | Clean separation across model families |
| ⚖️ Conditional support | Allows multimodal / dynamic adaptation |
| 🔄 Reuse-friendly | Shared logic (e.g., `_set_name`) is DRY |
| 📦 SDK-specific | Keeps message adaptation aligned to builder interface
|
---
### 🔮 Future Direction
- Support more SDKs and formats by introducing new message_param_func
- Global registry integration (currently `"openai"`-scoped)
- Class-based transformer variant if complexity grows
---
## Related issue number
Closes#5762
## Checks
- [ ] I've included any doc changes needed for
<https://microsoft.github.io/autogen/>. See
<https://github.com/microsoft/autogen/blob/main/CONTRIBUTING.md> to
build and test documentation locally.
- [x] I've added tests (if relevant) corresponding to the changes
introduced in this PR.
- [ v ] I've made sure all auto checks have passed.
---------
Co-authored-by: Eric Zhu <ekzhu@users.noreply.github.com>
Rename the `ChatMessage` and `AgentEvent` base classes to `BaseChatMessage` and `BaseAgentEvent`.
Bring back the `ChatMessage` and `AgentEvent` as union of built-in concrete types to avoid breaking existing applications that depends on Pydantic serialization.
Why?
Many existing code uses containers like this:
```python
class AppMessage(BaseModel):
name: str
message: ChatMessage
# Serialization is this:
m = AppMessage(...)
m.model_dump_json()
# Fields like HandoffMessage.target will be lost because it is now treated as a base class without content or target fields.
```
The assumption on `ChatMessage` or `AgentEvent` to be a union of concrete types could be in many existing code bases. So this PR brings back the union types, while keep method type hints such as those on `on_messages` to use the `BaseChatMessage` and `BaseAgentEvent` base classes for flexibility.
