38 lines
1.4 KiB
Python
Raw Permalink Normal View History

Support Component Validation API in AGS (#5503) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? It is useful to rapidly validate any changes to a team structure as teams are built either via drag and drop or by modifying the underlying spec You can now “validate” your team. The key ideas are as follows - Each team is based on some Component Config specification which is a pedantic model underneath. - Validation is 3 pronged based on a ValidatorService class - Data model validation (validate component schema) - Instantiation validation (validate component can be instantiated) - Provider validation, component_type validation (validate that provider exists and can be imported) - UX: each time a component is **loaded or saved**, it is automatically validated and any errors shown (via a server endpoint). This way, the developer immediately knows if updates to the configuration is wrong or has errors. > Note: this is different from actually running the component against a task. Currently you can run the entire team. In a separate PR we will implement ability to run/test other components. <img width="1360" alt="image" src="https://github.com/user-attachments/assets/d61095b7-0b07-463a-b4b2-5c50ded750f6" /> <img width="1368" alt="image" src="https://github.com/user-attachments/assets/09a1677e-76e8-44a4-9749-15c27457efbb" /> <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number Closes #4616 <!-- For example: "Closes #1234" --> ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
2025-02-12 09:29:43 -08:00
# api/routes/validation.py
from fastapi import APIRouter
Support Component Validation API in AGS (#5503) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? It is useful to rapidly validate any changes to a team structure as teams are built either via drag and drop or by modifying the underlying spec You can now “validate” your team. The key ideas are as follows - Each team is based on some Component Config specification which is a pedantic model underneath. - Validation is 3 pronged based on a ValidatorService class - Data model validation (validate component schema) - Instantiation validation (validate component can be instantiated) - Provider validation, component_type validation (validate that provider exists and can be imported) - UX: each time a component is **loaded or saved**, it is automatically validated and any errors shown (via a server endpoint). This way, the developer immediately knows if updates to the configuration is wrong or has errors. > Note: this is different from actually running the component against a task. Currently you can run the entire team. In a separate PR we will implement ability to run/test other components. <img width="1360" alt="image" src="https://github.com/user-attachments/assets/d61095b7-0b07-463a-b4b2-5c50ded750f6" /> <img width="1368" alt="image" src="https://github.com/user-attachments/assets/09a1677e-76e8-44a4-9749-15c27457efbb" /> <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number Closes #4616 <!-- For example: "Closes #1234" --> ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
2025-02-12 09:29:43 -08:00
from ...validation.component_test_service import ComponentTestRequest, ComponentTestResult, ComponentTestService
from ...validation.validation_service import ValidationError, ValidationRequest, ValidationResponse, ValidationService
Support Component Validation API in AGS (#5503) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? It is useful to rapidly validate any changes to a team structure as teams are built either via drag and drop or by modifying the underlying spec You can now “validate” your team. The key ideas are as follows - Each team is based on some Component Config specification which is a pedantic model underneath. - Validation is 3 pronged based on a ValidatorService class - Data model validation (validate component schema) - Instantiation validation (validate component can be instantiated) - Provider validation, component_type validation (validate that provider exists and can be imported) - UX: each time a component is **loaded or saved**, it is automatically validated and any errors shown (via a server endpoint). This way, the developer immediately knows if updates to the configuration is wrong or has errors. > Note: this is different from actually running the component against a task. Currently you can run the entire team. In a separate PR we will implement ability to run/test other components. <img width="1360" alt="image" src="https://github.com/user-attachments/assets/d61095b7-0b07-463a-b4b2-5c50ded750f6" /> <img width="1368" alt="image" src="https://github.com/user-attachments/assets/09a1677e-76e8-44a4-9749-15c27457efbb" /> <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number Closes #4616 <!-- For example: "Closes #1234" --> ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
2025-02-12 09:29:43 -08:00
router = APIRouter()
Support Component Validation API in AGS (#5503) <!-- Thank you for your contribution! Please review https://microsoft.github.io/autogen/docs/Contribute before opening a pull request. --> <!-- Please add a reviewer to the assignee section when you create a PR. If you don't have the access to it, we will shortly find a reviewer and assign them to your PR. --> ## Why are these changes needed? It is useful to rapidly validate any changes to a team structure as teams are built either via drag and drop or by modifying the underlying spec You can now “validate” your team. The key ideas are as follows - Each team is based on some Component Config specification which is a pedantic model underneath. - Validation is 3 pronged based on a ValidatorService class - Data model validation (validate component schema) - Instantiation validation (validate component can be instantiated) - Provider validation, component_type validation (validate that provider exists and can be imported) - UX: each time a component is **loaded or saved**, it is automatically validated and any errors shown (via a server endpoint). This way, the developer immediately knows if updates to the configuration is wrong or has errors. > Note: this is different from actually running the component against a task. Currently you can run the entire team. In a separate PR we will implement ability to run/test other components. <img width="1360" alt="image" src="https://github.com/user-attachments/assets/d61095b7-0b07-463a-b4b2-5c50ded750f6" /> <img width="1368" alt="image" src="https://github.com/user-attachments/assets/09a1677e-76e8-44a4-9749-15c27457efbb" /> <!-- Please give a short summary of the change and the problem this solves. --> ## Related issue number Closes #4616 <!-- For example: "Closes #1234" --> ## Checks - [ ] I've included any doc changes needed for https://microsoft.github.io/autogen/. See https://microsoft.github.io/autogen/docs/Contribute#documentation to build and test documentation locally. - [ ] I've added tests (if relevant) corresponding to the changes introduced in this PR. - [ ] I've made sure all auto checks have passed.
2025-02-12 09:29:43 -08:00
@router.post("/")
async def validate_component(request: ValidationRequest) -> ValidationResponse:
"""Validate a component configuration"""
try:
return ValidationService.validate(request.component)
except Exception as e:
return ValidationResponse(
is_valid=False, errors=[ValidationError(field="validation", error=str(e))], warnings=[]
)
@router.post("/test")
async def test_component(request: ComponentTestRequest) -> ComponentTestResult:
"""Test a component functionality with appropriate inputs based on type"""
# First validate the component configuration
validation_result = ValidationService.validate(request.component)
# Only proceed with testing if the component is valid
if not validation_result.is_valid:
return ComponentTestResult(
status=False, message="Component validation failed", logs=[e.error for e in validation_result.errors]
)
# If validation passed, run the functional test
return await ComponentTestService.test_component(
component=request.component, timeout=request.timeout if request.timeout else 60
)