483532180a
1. Add host network support in Docker and remove unused requirements
from argument check.
2. Use Pandas to simplify summary statistic calculations.
3. Add running time to summary statistics
```
Using tabulation method defined in '/home/ekzhu/autogen/python/packages/agbench/benchmarks/HumanEval/Scripts/custom_tabulate.py'
Task Id Trial 0 Success Trial 0 Time
-- ------------ ----------------- --------------
0 HumanEval_0 True 3
1 HumanEval_1 False 15
2 HumanEval_2 True 2
3 HumanEval_3 True 11
4 HumanEval_4 True 4
5 HumanEval_5 True 2
6 HumanEval_6 False 18
7 HumanEval_7 True 2
8 HumanEval_8 True 2
9 HumanEval_9 True 12
10 HumanEval_10 False 11
11 HumanEval_11 True 2
12 HumanEval_12 True 3
13 HumanEval_13 True 1
14 HumanEval_14 True 4
15 HumanEval_15 True 1
16 HumanEval_16 True 2
17 HumanEval_17 False 76
18 HumanEval_18 True 4
19 HumanEval_19 True 3
20 HumanEval_20 True 5
21 HumanEval_21 True 3
22 HumanEval_22 True 1
23 HumanEval_23 True 2
24 HumanEval_24 nan
Summary Statistics
Successes Failures Missing Total Average Success Rate Average Time Total Time
------- ----------- ---------- --------- ------- ---------------------- -------------- ------------
Trial 0 20 4 1 25 0.8 7.875 189
CAUTION: 'autogenbench tabulate' is in early preview and is not thoroughly tested.
Please do not cite values from these calculations in academic work without first inspecting and verifying the results in the run logs yourself.
```
Now the default tabulate output looks like this
---------
Co-authored-by: Ryan Sweet <rysweet@microsoft.com>
AutoGen Python packages
This directory works as a single uv workspace containing all project packages. See packages to discover all project packages.
Migrating from 0.2.x?
Please refer to the migration guide for how to migrate your code from 0.2.x to 0.4.x.
Development
TL;DR, run all checks with:
uv sync --all-extras
source .venv/bin/activate
poe check
Setup
uv is a package manager that assists in creating the necessary environment and installing packages to run AutoGen.
Note: To prevent incompatibilities between versions the same UV version as is running in CI should be used. Check the version in CI by looking the setup-uv action, here for example.
For example, to change your version to 0.5.18, run:
uv self update 0.5.18
Virtual Environment
During development, you may need to test changes made to any of the packages.
To do so, create a virtual environment where the AutoGen packages are installed based on the current state of the directory.
Run the following commands at the root level of the Python directory:
uv sync --all-extras
source .venv/bin/activate
uv sync --all-extraswill create a.venvdirectory at the current level and install packages from the current directory along with any other dependencies. Theall-extrasflag adds optional dependencies.source .venv/bin/activateactivates the virtual environment.
Common Tasks
To create a pull request (PR), ensure the following checks are met. You can run each check individually:
- Format:
poe format - Lint:
poe lint - Test:
poe test - Mypy:
poe mypy - Pyright:
poe pyright - Build docs:
poe --directory ./packages/autogen-core/ docs-build - Auto rebuild+serve docs:
poe --directory ./packages/autogen-core/ docs-serve - Check samples in
python/samples:poe samples-code-checkAlternatively, you can run all the checks with: poe check
Note
These need to be run in the virtual environment.
Syncing Dependencies
When you pull new changes, you may need to update the dependencies.
To do so, first make sure you are in the virtual environment, and then in the python directory, run:
uv sync --all-extras
This will update the dependencies in the virtual environment.
Creating a New Package
To create a new package, similar to autogen-core or autogen-chat, use the following:
uv sync --python 3.12
source .venv/bin/activate
cookiecutter ./templates/new-package/