mirror of
https://github.com/microsoft/autogen.git
synced 2025-07-19 06:53:17 +00:00
9b92093b00
* update markdown hyperlinks to stable urls * update notebook images and text * re-write observability section * Updated section * update wording * added newline * update styling in image tags to be jsx compatible * added text * update link * simplified text --------- Co-authored-by: Braelyn Boynton <bboynton97@gmail.com>
38 lines
2.0 KiB
Markdown
38 lines
2.0 KiB
Markdown
# Agent Observability
|
|
|
|
AutoGen supports advanced LLM agent observability and monitoring through built-in logging and partner providers.
|
|
|
|
## AutoGen Observability Integrations
|
|
|
|
### Built-In Logging
|
|
AutoGen's SQLite and File Logger - [Tutorial Notebook](/docs/notebooks/agentchat_logging)
|
|
|
|
### Full-Service Partner Integrations
|
|
AutoGen partners with [AgentOps](https://agentops.ai) to provide multi-agent tracking, metrics, and monitoring - [Tutorial Notebook](/docs/notebooks/agentchat_agentops)
|
|
|
|
|
|
## What is Observability?
|
|
Observability provides developers with the necessary insights to understand and improve the internal workings of their agents. Observability is necessary for maintaining reliability, tracking costs, and ensuring AI safety.
|
|
|
|
**Without observability tools, developers face significant hurdles:**
|
|
|
|
- Tracking agent activities across sessions becomes a complex, error-prone task.
|
|
- Manually sifting through verbose terminal outputs to understand LLM interactions is inefficient.
|
|
- Pinpointing the exact moments of tool invocations is often like finding a needle in a haystack.
|
|
|
|
|
|
**Key Features of Observability Dashboards:**
|
|
- Human-readable overview analytics and replays of agent activities.
|
|
- LLM cost, prompt, completion, timestamp, and metadata tracking for performance monitoring.
|
|
- Tool invocation, events, and agent-to-agent interactions for workflow monitoring.
|
|
- Error flagging and notifications for faster debugging.
|
|
- Access to a wealth of data for developers using supported agent frameworks, such as environments, SDK versions, and more.
|
|
|
|
### Compliance
|
|
|
|
Observability is not just a development convenience—it's a compliance necessity, especially in regulated industries:
|
|
- It offers insights into AI decision-making processes, fostering trust and transparency.
|
|
- Anomalies and unintended behaviors are detected promptly, reducing various risks.
|
|
- Ensuring adherence to data privacy regulations, thereby safeguarding sensitive information.
|
|
- Compliance violations are quickly identified and addressed, enhancing incident management.
|