878aa4c3fc
This pull request introduces a new linting feature to the benchmark configuration in the `agbench` package. The main changes include adding a new command to the CLI, implementing the linter functionality, and integrating it with the existing codebase. ### New Linting Feature: * [`python/packages/agbench/src/agbench/cli.py`](diffhunk://#diff-0eafed70ad5e99e6f7319927bf92ee3ce4787d156dd2775b10a61baad7ec1799R10): Added `lint_cli` import and integrated the new "lint" command into the `main` function. [[1]](diffhunk://#diff-0eafed70ad5e99e6f7319927bf92ee3ce4787d156dd2775b10a61baad7ec1799R10) [[2]](diffhunk://#diff-0eafed70ad5e99e6f7319927bf92ee3ce4787d156dd2775b10a61baad7ec1799R37-R41) ### Linter Implementation: * [`python/packages/agbench/src/agbench/linter/__init__.py`](diffhunk://#diff-45842e728e3daad063b3cf84d5857a4fdfe14e6d977fb2054f284eb9f5bb5272R1-R4): Added necessary imports to initialize the linter module. * [`python/packages/agbench/src/agbench/linter/_base.py`](diffhunk://#diff-f7ea2f6706232406b6c727fda6d71f09c568b4573f070af79bb7f3da3514e364R1-R81): Defined core classes such as `Document`, `Code`, `CodeExample`, `CodedDocument`, and the `BaseQualitativeCoder` protocol. * [`python/packages/agbench/src/agbench/linter/cli.py`](diffhunk://#diff-e6ad1e14dc0df2c10fe62fede5a06d83865ad1961f99ec2d78f9052feb4d663bR1-R86): Implemented the `lint_cli` function, which includes loading log files, coding them, and printing the results. * [`python/packages/agbench/src/agbench/linter/coders/oai_coder.py`](diffhunk://#diff-5059129410822c8a214f797a6167cbfcfbe31bd6a3b1efcb65a2dd703ef9b331R1-R212): Implemented the `OAIQualitativeCoder` class to interact with OpenAI for coding documents and caching results. Example usage: <img width="997" alt="image" src="https://github.com/user-attachments/assets/6718688e-9917-4a43-a2f1-1105b030528d" /> <img width="999" alt="image" src="https://github.com/user-attachments/assets/7fcb9c43-70f2-4fe7-ae29-5ad6a4ef2a16" /> > If you are in VSCode Terminal, you can click on the links in the terminal output to jump to the exact error. --------- Co-authored-by: afourney <adamfo@microsoft.com>
AutoGen Python packages
This directory works as a single uv workspace containing all project packages. See packages to discover all project packages.
Migrating from 0.2.x?
Please refer to the migration guide for how to migrate your code from 0.2.x to 0.4.x.
Development
TL;DR, run all checks with:
uv sync --all-extras
source .venv/bin/activate
poe check
Setup
uv is a package manager that assists in creating the necessary environment and installing packages to run AutoGen.
Note: To prevent incompatibilities between versions the same UV version as is running in CI should be used. Check the version in CI by looking the setup-uv action, here for example.
For example, to change your version to 0.5.18, run:
uv self update 0.5.18
Virtual Environment
During development, you may need to test changes made to any of the packages.
To do so, create a virtual environment where the AutoGen packages are installed based on the current state of the directory.
Run the following commands at the root level of the Python directory:
uv sync --all-extras
source .venv/bin/activate
uv sync --all-extraswill create a.venvdirectory at the current level and install packages from the current directory along with any other dependencies. Theall-extrasflag adds optional dependencies.source .venv/bin/activateactivates the virtual environment.
Common Tasks
To create a pull request (PR), ensure the following checks are met. You can run each check individually:
- Format:
poe format - Lint:
poe lint - Test:
poe test - Mypy:
poe mypy - Pyright:
poe pyright - Build docs:
poe --directory ./packages/autogen-core/ docs-build - Auto rebuild+serve docs:
poe --directory ./packages/autogen-core/ docs-serve - Check samples in
python/samples:poe samples-code-checkAlternatively, you can run all the checks with: poe check
Note
These need to be run in the virtual environment.
Syncing Dependencies
When you pull new changes, you may need to update the dependencies.
To do so, first make sure you are in the virtual environment, and then in the python directory, run:
uv sync --all-extras
This will update the dependencies in the virtual environment.
Creating a New Package
To create a new package, similar to autogen-core or autogen-chat, use the following:
uv sync --python 3.12
source .venv/bin/activate
cookiecutter ./templates/new-package/