autogen/notebook/agentchat_oai_code_interpreter.ipynb

286 lines
192 KiB
Plaintext
Raw Normal View History

{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Auto Generated Agent Chat: GPTAssistant with Code Interpreter\n",
"The latest released Assistants API by OpenAI allows users to build AI assistants within their own applications. The Assistants API currently supports three types of tools: Code Interpreter, Retrieval, and Function calling. In this notebook, we demonstrate how to enable `GPTAssistantAgent` to use code interpreter. \n",
"\n",
"## Requirements\n",
"\n",
"AutoGen requires `Python>=3.8`. To run this notebook example, please install:\n",
"```bash\n",
"pip install pyautogen\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Set your API Endpoint\n",
"\n",
"The [`config_list_from_json`](https://microsoft.github.io/autogen/docs/reference/oai/openai_utils#config_list_from_json) function loads a list of configurations from an environment variable or a json file."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"import io\n",
"\n",
"from IPython.display import display\n",
"from PIL import Image\n",
"\n",
"import autogen\n",
"from autogen.agentchat import AssistantAgent, UserProxyAgent\n",
"from autogen.agentchat.contrib.gpt_assistant_agent import GPTAssistantAgent\n",
"\n",
"config_list = autogen.config_list_from_json(\n",
" \"OAI_CONFIG_LIST\",\n",
" file_location=\".\",\n",
" filter_dict={\n",
" \"model\": [\"gpt-3.5-turbo\", \"gpt-35-turbo\", \"gpt-4\", \"gpt4\", \"gpt-4-32k\", \"gpt-4-turbo\"],\n",
" },\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It first looks for environment variable \"OAI_CONFIG_LIST\" which needs to be a valid json string. If that variable is not found, it then looks for a json file named \"OAI_CONFIG_LIST\". It filters the configs by models (you can filter by other keys as well).\n",
"\n",
"The config list looks like the following:\n",
"```python\n",
"config_list = [\n",
" {\n",
" \"model\": \"gpt-4\",\n",
" \"api_key\": \"<your OpenAI API key>\",\n",
" }, # OpenAI API endpoint for gpt-4\n",
"]\n",
"```\n",
"\n",
"Currently Azure OpenAi does not support assistant api. You can set the value of config_list in any way you prefer. Please refer to this [notebook](https://github.com/microsoft/autogen/blob/main/notebook/oai_openai_utils.ipynb) for full code examples of the different methods."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Perform Tasks Using Code Interpreter\n",
"\n",
"We demonstrate task solving using `GPTAssistantAgent` with code interpreter. Pass `code_interpreter` in `tools` parameter to enable `GPTAssistantAgent` with code interpreter. It will write code and automatically execute it in a sandbox. The agent will receive the results from the sandbox environment and act accordingly.\n",
"\n",
"### Example 1: Math Problem Solving\n",
"In this example, we demonstrate how to use code interpreter to solve math problems.\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:autogen.agentchat.contrib.gpt_assistant_agent:assistant_id was None, creating a new assistant\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[33muser_proxy\u001b[0m (to Coder Assistant):\n",
"\n",
"If $725x + 727y = 1500$ and $729x+ 731y = 1508$, what is the value of $x - y$ ?\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\u001b[33mCoder Assistant\u001b[0m (to user_proxy):\n",
"\n",
"The value of $x - y$ is $-48$. \n",
"\n",
"TERMINATE\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n"
]
}
],
"source": [
"# Initiate an agent equipped with code interpreter\n",
"gpt_assistant = GPTAssistantAgent(\n",
" name=\"Coder Assistant\",\n",
" llm_config={\n",
" \"tools\": [{\"type\": \"code_interpreter\"}],\n",
" \"config_list\": config_list,\n",
" },\n",
" instructions=\"You are an expert at solving math questions. Write code and run it to solve math problems. Reply TERMINATE when the task is solved and there is no problem.\",\n",
")\n",
"\n",
"user_proxy = UserProxyAgent(\n",
" name=\"user_proxy\",\n",
" is_termination_msg=lambda msg: \"TERMINATE\" in msg[\"content\"],\n",
" code_execution_config={\n",
" \"work_dir\": \"coding\",\n",
" \"use_docker\": False, # Please set use_docker=True if docker is available to run the generated code. Using docker is safer than running the generated code directly.\n",
" },\n",
" human_input_mode=\"NEVER\",\n",
")\n",
"\n",
"# When all is set, initiate the chat.\n",
"user_proxy.initiate_chat(\n",
" gpt_assistant, message=\"If $725x + 727y = 1500$ and $729x+ 731y = 1508$, what is the value of $x - y$ ?\"\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Example 2: Plotting with Code Interpreter\n",
"\n",
"Code Interpreter can outputs files, such as generating image diagrams. In this example, we demonstrate how to draw figures and download it."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:autogen.agentchat.contrib.gpt_assistant_agent:assistant_id was None, creating a new assistant\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[33muser_proxy\u001b[0m (to Coder Assistant):\n",
"\n",
"Draw a line chart to show the population trend in US. Show how you solved it with code.\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\u001b[33mCoder Assistant\u001b[0m (to user_proxy):\n",
"\n",
"To draw a line chart showing the population trend in the US, we need some population data over a range of years. Normally, this data can be obtained from sources like the United States Census Bureau or other datasets available online.\n",
"\n",
"Since I don't have internet access to download the latest data directly, you can provide the data if you have it. The data should ideally consist of two columns: one for the years and one for the corresponding US population for each year.\n",
"\n",
"If you don't have the data, I can demonstrate how to generate the line chart with some dummy data to illustrate the process. Would you like to provide the data or should I use dummy data for the demonstration?\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\u001b[33muser_proxy\u001b[0m (to Coder Assistant):\n",
"\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n",
"\u001b[33mCoder Assistant\u001b[0m (to user_proxy):\n",
"\n",
"It seems there was no response regarding the data, so let's proceed with some dummy data to demonstrate the process. I'll create a Python script that will generate a line chart showing the hypothetical population trend in the US from the year 1900 to 2000. I'll assume a simple linear growth in population for the sake of this example.\n",
"\n",
"Let's begin by generating the dummy data and plotting the line chart.\n",
"\n",
"\n",
"Recieved file id=file-sAG7r763XR9jiuKYiQeoHRFU\n",
"\n",
"Here is a line chart showing the hypothetical population trend of the US from the year 1900 to 2000. The data is dummy and assumes linear growth in the population.\n",
"\n",
"If you have actual data or would like to see a different representation, please let me know. Otherwise, if everything looks good with this example chart, let me know if there is anything else I can help with. \n",
"\n",
"TERMINATE\n",
"\n",
"\n",
"--------------------------------------------------------------------------------\n"
]
}
],
"source": [
"gpt_assistant = GPTAssistantAgent(\n",
" name=\"Coder Assistant\",\n",
" llm_config={\n",
" \"tools\": [{\"type\": \"code_interpreter\"}],\n",
" \"config_list\": config_list,\n",
" },\n",
" instructions=\"You are an expert at writing python code to solve problems. Reply TERMINATE when the task is solved and there is no problem.\",\n",
")\n",
"\n",
"user_proxy.initiate_chat(\n",
" gpt_assistant,\n",
" message=\"Draw a line chart to show the population trend in US. Show how you solved it with code.\",\n",
" clear_history=True,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we have the file id. We can download and display it."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAB7sAAAPUCAYAAADc4V37AAEAAElEQVR4AezdCXwU9f3/8Q9HADEqRAjBqFiVwwtKIF5QtLZBRanaWqzalqO1l7b2X/1pW1sRelhqL2u11raCth6g1ioeralWKR4gh6BVMR4gRsJhRA03yD/vobPMTmZ3Z8/s8fo+HmFnZ77znZnnzH53mc98v98OO1uTkRBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEECgggY4FtK/sKgIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAo4AwW4uBAQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQACBghMg2F1wp4wdRgABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBAg2M01gAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCBQcAIEuwvulLHDCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIEu7kGEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQKToBgd8GdMnYYAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQIBgN9cAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggEDBCRDsLrhTxg4jgAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCBDs5hpAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEECg4AYLdBXfK2GEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAYLdXAMIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAgUnQLC74E4ZO4wAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAgggQLCbawABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAoOAECHYX3CljhxFAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEECHZzDSCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIFJwAwe6CO2XsMAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAwW6uAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQACBghMg2F1wp4wdRgABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBDoDAECCCCAAAIIIIAAAtkQGD58eFSxCxYsiHrPG7OrrrrKHnjggQjF5MmTbezYsZH3xT5R6sev87t582Z7+eWX7a233rIPPvjANmzYYF27drU99tjD9t13X+vbt68deOCB1r1792K/HDi+FAVUt37ta1+LrF1TU2M33XRT5H0xTLz99tv2qU99KnIo+lzMnj078p4JBIpZoL2/K//yl7/Ytdde6xB369bN/v73v1uvXr3ynnz9+vW2dOlSW7dunWm6rKzM2e/+/fvboYcempX9b2pqshdeeMH0qu93fXdXV1fb4MGDrWfPnhnfZktLiy1ZssRWrlzp/H7o0qWL9e7d2w4//HDnt0PGN1ggBW7fvt3efPNNe/311+2dd96J2Oyzzz6233772RFHHGG6lrOR3n33Xee6a2xstI0bNzrb0XeWtllVVZXxTeo603W+YsUK53dkp06dnOt84MCBWbnOP/zwQ3vppZfs1VdfNR3rzp07Ta4HH3ywHXnkkda5c+ekj1G/fc8880ynPK08YcIEu+iii5IuhxUQQAABBEpbIPlvoNL24ugRQAABBBBAIIaA/0Zcujfb/Tfvtdn777/fuUERYxeYjQACCAQK/OEPf7A//vGPkWWZCJT5H+ZI5kGFrVu32r/+9S+79957nRuUO3bsiOxb0ETHjh2tX79+zo3SY4891vTXo0ePoKwZmec/tliFKihfXl7u3MAfNGiQHXXUUfaJT3wiq/sWa1+YjwACbQX8v83a5sjcnEzUq5nbG0rKlIACxX/6058ixZ1//vlJBboVGFPA8cUXX7T//ve/zmtDQ4MpGOmmTF47Crw99thjpgC9tqntByVt8zOf+Yyde+65zgNmQXmSmff444/bLbfcYs8//3zgavoer62ttS996Uum/yOlm1555RXnd81//vOfKEtvuQo+fuELX7DTTz/dOnTo4F1UlNNvvPGGPfHEE/bss8/ac889Z1u2bIl5nAoI6zyMGzfOTjjhBNP5STctWrTI/vznPzvbj3Xd6XfS+PHj7cQTT0x3c6Zguj6bjzzySMxjVWD/nHPOcf5SCUJ7d1KBe32u7rnnHmtubvYuikzrN6Ee2p00aVJSD3fsueeezmfjF7/4hVPWbbfd5jzgpoc9SQgggAACCIQVINgdVop8CCCAAAIIIIBAiQgsW7bMdNPOTWoZkImbMm55xfCqVobeVoW6mVRKLbKL4Ry21zGo9c3UqVNt+fLloXdBN011E1d/6glAN2Xnzp1rasHVnkk3kvWnVlNq4aN9+9WvfmVjxoyxb33rW7bXXnu15+6x7TwRUF2pOtNNqitVZ5IQQCD/BdSiW60uldR6U8HTRGnt2rV2++23O8FttQDdtGlTolUyslyB+csuu8x5iCxRgatWrbLf/e53Tiv1adOmmX7rppJ0bFOmTHEeYIu3vr7H582b5/x97nOfs29/+9sptYDVNqZPn2433nijJXpQTg8ZaN/03fyzn/0sqeBjvGPJt2UKbOv49DskbJKdguL6U6t7/S7bf//9w64elU8Pbvz617+2mTNnRs0PeqOHIS699FKrq6uzK6+80unFJyhfonk6p1dffXXMILe7vr57tW/6HlYgOdVj1IMj+mypx4J4ST0N3HHHHfbQQw/Zj370Izv++OPjZY9apodPFOTWZ3Pbtm3OfmvfSQgggAACCIQVSP/RtbBbIh8CCCCAAAIIIIBAQQi4rUXUElZ/3sB3QRxADnZSN49cH716A9852DybKFCBRx991C644IKYgW61NFKAWC1c4iXdNFfrtXxMCn6rxbpu5qt7dhICqh+99aU38I0OAgjkr4CCh//85z8jO6hWsGq5mSipO2W1AFVL11wFuvUAmVrM6oEyf1KrZn23BrVs1RAiX/nKV5yux/3rJXqv7qMvvvjiwEC3thnL6s4777Qf/vCHKX2P6+GD66+/PjDQrd8OQS2UFy5c6ByjunIvxqTrNFGgW+dCvdEEJV0z6rEgld8s+j2mcxkr0B3rGqivr3eunXitz4P2VfNmzZplV111VWCgW93m67ekP8nny1/+sjNkjn9Zovfqll9DpQQFuvXQZZDre++9Z//v//0/p6V9ovLd5RpmQOfBTeq1QA8ykBBAAAEEEAgrQMvusFLkQwABBBBAAAEEEEAAAQRSFNCYmldccUWbG9TqRlMtXT/60Y86LW7crkbVUkitW9Ttq272/fvf/3ZaUKe4+bRWO+2005wuyv2F6Eb/6tWrTTfS9ZCMN2m+Wq7NmDEjK2NUerfFNAIIBAuo+2KNoRomKTjx+9//PirreeedF3rc30QP6UQVzJuCEND1oGCeksY3VnfI+ZjUvfL//d//Od9H3v0bNmyYffGLXzQNzaGAnB4Sc3tIUetTtR5VUsv1Sy65xGlVmsxY5Gopq4C+N+m7XF2VDx061DFT2U899ZTT3fRrr70Wyapgp8YOV3fPYdPDDz/sPETgza9eMlSGhhBRQF/HpN8N+u5VDzBu0nH/4Ac/sOuuu67ouzRXAPZjH/uYHX300aZrQEZuTzjqiebpp5+2W2+91ele3/XRedIY0WpZ3KdPH3d2wle1ste59CaNB6/Aslo1K/is30r6nXTzzTc746u7eXXt6BrSb8OwSev88pe/jMquseD1oId61amoqHC6tVdwW70rqIW1m9TzgT4nujaCAtRuPu+rxuRWS3R9xtykYLoefDn77LPtgAMOcK4nBcLvu+8+x899wEWt5/UggB5
"text/plain": [
"<PIL.PngImagePlugin.PngImageFile image mode=RGBA size=1979x980>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"api_response = gpt_assistant.openai_client.files.with_raw_response.retrieve_content(\"file-sAG7r763XR9jiuKYiQeoHRFU\")\n",
"\n",
"if api_response.status_code == 200:\n",
" content = api_response.content\n",
" image_data_bytes = io.BytesIO(content)\n",
" image = Image.open(image_data_bytes)\n",
" display(image)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.13"
}
},
"nbformat": 4,
"nbformat_minor": 2
}