"AutoGen has [built-in support for tracing](https://microsoft.github.io/autogen/dev/user-guide/core-user-guide/framework/telemetry.html) and observability for collecting comprehensive records on the execution of your application. This feature is useful for debugging, performance analysis, and understanding the flow of your application.\n",
"\n",
"This capability is powered by the [OpenTelemetry](https://opentelemetry.io/) library, which means you can use any OpenTelemetry-compatible backend to collect and analyze traces.\n",
"\n",
"AutoGen follows the [OpenTelemetry Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/) for tracing, for agents and tools.\n",
"It also follows the [Semantic Conventions for GenAI Systems](https://opentelemetry.io/docs/specs/semconv/gen-ai/) currently under development.\n",
"\n",
"## Setup\n",
"\n",
"To begin, you need to install the OpenTelemetry Python package. You can do this using pip:\n",
"Once you have the SDK installed, the simplest way to set up tracing in AutoGen is to:\n",
"\n",
"1. Configure an OpenTelemetry tracer provider\n",
"2. Set up an exporter to send traces to your backend\n",
"3. Connect the tracer provider to the AutoGen runtime\n",
"\n",
"## Telemetry Backend\n",
"\n",
"To collect and view traces, you need to set up a telemetry backend. Several open-source options are available, including Jaeger, Zipkin. For this example, we will use Jaeger as our telemetry backend.\n",
"\n",
"For a quick start, you can run Jaeger locally using Docker:\n",
"\n",
"```bash\n",
"docker run -d --name jaeger \\\n",
" -e COLLECTOR_OTLP_ENABLED=true \\\n",
" -p 16686:16686 \\\n",
" -p 4317:4317 \\\n",
" -p 4318:4318 \\\n",
" jaegertracing/all-in-one:latest\n",
"```\n",
"\n",
"This command starts a Jaeger instance that listens on port 16686 for the Jaeger UI and port 4317 for the OpenTelemetry collector. You can access the Jaeger UI at `http://localhost:16686`.\n",
"\n",
"## Tracing an AgentChat Team\n",
"\n",
"In the following section, we will review how to enable tracing with an AutoGen GroupChat team. The AutoGen runtime already supports open telemetry (automatically logging message metadata). To begin, we will create a tracing service that will be used to instrument the AutoGen runtime. "
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Overriding of current TracerProvider is not allowed\n",
"Attempting to instrument while already instrumented\n"
"Additionally, you can set the environment variable `AUTOGEN_DISABLE_RUNTIME_TRACING` to `true` to disable the agent runtime telemetry if you don't have access to the runtime constructor. For example, if you are using `ComponentConfig`.\n",
" selector_prompt = \"\"\"Select an agent to perform task.\n",
"\n",
" {roles}\n",
"\n",
" Current conversation context:\n",
" {history}\n",
"\n",
" Read the above conversation, then select an agent from {participants} to perform the next task.\n",
" Make sure the planner agent has assigned tasks before other agents start working.\n",
" Only select one agent.\n",
" \"\"\"\n",
"\n",
" task = \"Who was the Miami Heat player with the highest points in the 2006-2007 season, and what was the percentage change in his total rebounds between the 2007-2008 and 2008-2009 seasons?\"\n",
"\n",
" runtime = SingleThreadedAgentRuntime(\n",
" tracer_provider=trace.NoOpTracerProvider(), # Disable telemetry for runtime.\n",
"Who was the Miami Heat player with the highest points in the 2006-2007 season, and what was the percentage change in his total rebounds between the 2007-2008 and 2008-2009 seasons?\n",
"[FunctionExecutionResult(content='Here are the total points scored by Miami Heat players in the 2006-2007 season:\\n Udonis Haslem: 844 points\\n Dwayne Wade: 1397 points\\n James Posey: 550 points\\n ...\\n ', name='search_web_tool', call_id='call_hS8yod9l6CYUllDveUffp58e', is_error=False)]\n",
"[FunctionCall(id='call_bUJxtpxUXFSxECDogye9WL0g', arguments='{\"query\":\"Dwyane Wade total rebounds in 2007-2008 season\"}', name='search_web_tool')]\n",
"[FunctionExecutionResult(content='The number of total rebounds for Dwayne Wade in the Miami Heat season 2007-2008 is 214.', name='search_web_tool', call_id='call_bUJxtpxUXFSxECDogye9WL0g', is_error=False)]\n",
"[FunctionCall(id='call_pgYNSDhhyodtteot56FRktxp', arguments='{\"query\":\"Dwyane Wade total rebounds in 2008-2009 season\"}', name='search_web_tool')]\n",
"[FunctionExecutionResult(content='The number of total rebounds for Dwayne Wade in the Miami Heat season 2008-2009 is 398.', name='search_web_tool', call_id='call_pgYNSDhhyodtteot56FRktxp', is_error=False)]\n",