autogen/notebook/agentchat_function_call.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ae1f50ec",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/microsoft/autogen/blob/main/notebook/agentchat_function_call.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9a71fa36",
   "metadata": {},
   "source": [
    "# Auto Generated Agent Chat: Task Solving with Provided Tools as Functions\n",
    "\n",
    "AutoGen offers conversable agents powered by LLM, tool, or human, which can be used to perform tasks collectively via automated chat. This framework allows tool use and human participation through multi-agent conversation. Please find documentation about this feature [here](https://microsoft.github.io/autogen/docs/Use-Cases/agent_chat).\n",
    "\n",
    "In this notebook, we demonstrate how to use `AssistantAgent` and `UserProxyAgent` to make function calls with the new feature of OpenAI models (in model version 0613). A specified prompt and function configs must be passed to `AssistantAgent` to initialize the agent. The corresponding functions must be passed to `UserProxyAgent`, which will execute any function calls made by `AssistantAgent`. Besides this requirement of matching descriptions with functions, we recommend checking the system message in the `AssistantAgent` to ensure the instructions align with the function call descriptions.\n",
    "\n",
    "## Requirements\n",
    "\n",
    "AutoGen requires `Python>=3.8`. To run this notebook example, please install `pyautogen`:\n",
    "```bash\n",
    "pip install pyautogen\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2b803c17",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %pip install \"pyautogen~=0.2.0b2\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "5ebd2397",
   "metadata": {},
   "source": [
    "## Set your API Endpoint\n",
    "\n",
    "The [`config_list_from_json`](https://microsoft.github.io/autogen/docs/reference/oai/openai_utils#config_list_from_json) function loads a list of configurations from an environment variable or a json file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "dca301a4",
   "metadata": {},
   "outputs": [],
   "source": [
    "import autogen\n",
    "\n",
    "config_list = autogen.config_list_from_json(\n",
    "    \"OAI_CONFIG_LIST\",\n",
    "    filter_dict={\n",
    "        \"model\": [\"gpt-4\", \"gpt-3.5-turbo\", \"gpt-3.5-turbo-16k\"],\n",
    "    },\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "92fde41f",
   "metadata": {},
   "source": [
    "It first looks for environment variable \"OAI_CONFIG_LIST\" which needs to be a valid json string. If that variable is not found, it then looks for a json file named \"OAI_CONFIG_LIST\". It filters the configs by models (you can filter by other keys as well). Only the models with matching names are kept in the list based on the filter condition.\n",
    "\n",
    "The config list looks like the following:\n",
    "```python\n",
    "config_list = [\n",
    "    {\n",
    "        'model': 'gpt-4',\n",
    "        'api_key': '<your OpenAI API key here>',\n",
    "    },\n",
    "    {\n",
    "        'model': 'gpt-3.5-turbo',\n",
    "        'api_key': '<your Azure OpenAI API key here>',\n",
    "        'base_url': '<your Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-08-01-preview',\n",
    "    },\n",
    "    {\n",
    "        'model': 'gpt-3.5-turbo-16k',\n",
    "        'api_key': '<your Azure OpenAI API key here>',\n",
    "        'base_url': '<your Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-08-01-preview',\n",
    "    },\n",
    "]\n",
    "```\n",
    "\n",
    "You can set the value of config_list in any way you prefer. Please refer to this [notebook](https://github.com/microsoft/autogen/blob/main/notebook/oai_openai_utils.ipynb) for full code examples of the different methods."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "2b9526e7",
   "metadata": {},
   "source": [
    "## Making Function Calls\n",
    "\n",
    "In this example, we demonstrate function call execution with `AssistantAgent` and `UserProxyAgent`. With the default system prompt of `AssistantAgent`, we allow the LLM assistant to perform tasks with code, and the `UserProxyAgent` would extract code blocks from the LLM response and execute them. With the new \"function_call\" feature, we define functions and specify the description of the function in the OpenAI config for the `AssistantAgent`. Then we register the functions in `UserProxyAgent`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9fb85afb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33muser_proxy\u001b[0m (to chatbot):\n",
      "\n",
      "Draw two agents chatting with each other with an example dialog. Don't add plt.show().\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mchatbot\u001b[0m (to user_proxy):\n",
      "\n",
      "\u001b[32m***** Suggested function Call: python *****\u001b[0m\n",
      "Arguments: \n",
      "{\n",
      "  \"cell\": \"import matplotlib.pyplot as plt\\n\n",
      "# Initialize an empty figure and axis\\n\n",
      "fig, ax = plt.subplots()\\n\n",
      "# Create the chatboxes for messages\\n\n",
      "ax.text(0.5, 0.6, 'Agent1: Hi!', bbox=dict(facecolor='red', alpha=0.5))\\n\n",
      "ax.text(0.5, 0.5, 'Agent2: Hello!', bbox=dict(facecolor='blue', alpha=0.5))\\n\n",
      "plt.axis('off')\"\n",
      "}\n",
      "\u001b[32m*******************************************\u001b[0m\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[35m\n",
      ">>>>>>>> EXECUTING FUNCTION python...\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAGFCAYAAABg2vAPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAUuklEQVR4nO3dfZBVhZ3n4W/zotCoKLQY2sUYXwqJEUIQjGWhZLRYiDOzSbRiZWISqmICyZjZZK1Z45g3zIs6GUeTsqLUDpFyMo6aGRNTEV0lhjdfojINRhTUlAJBQLrBhk6roPT+IdNrj0kEBDv6e56q/qPPuefc371UcT997rnnNnR1dXUFACirT28PAAD0LjEAAMWJAQAoTgwAQHFiAACKEwMAUJwYAIDixAAAFCcGAKA4MQAAxYkBAChODABAcWIAAIoTAwBQnBgAgOLEAAAUJwYAoDgxAADFiQEAKE4MAEBxYgAAihMDAFCcGACA4sQAABQnBgCgODEAAMWJAQAoTgwAQHFiAACKEwMAUJwYAIDixAAAFCcGAKA4MQAAxYkBAChODABAcWIAAIoTAwBQnBgAgOLEAAAUJwYAoDgxAADFiQEAKE4MAEBxYgAAihMDAFCcGACA4sQAABQnBgCgODEAAMWJAQAoTgwAQHFiAACKEwMAUJwYAIDixAAAFCcGAKA4MQAAxYkBAChODABAcWIAAIoTAwBQnBgAgOLEAAAUJwYAoDgxAADFiQEAKE4MAEBxYgAAihMDAFCcGACA4sQAABQnBgCgODEAAMWJAQAoTgwAQHFiAACKEwMAUJwYAIDixAAAFCcGAKA4MQAAxYkBAChODABAcWIAAIoTAwBQnBgAgOLEAAAUJwYAoDgxAADFiQEAKE4MAEBxYgAAihMDAFCcGACA4sQAABQnBgCgODEAAMWJAQAoTgwAQHFiAACKEwMAUJwYAIDixAAAFCcGAKA4MQAAxYkBAChODABAcWIAAIoTAwBQnBgAgOLEAAAUJwYAoDgxAADFiQEAKE4MAEBxYgAAihMDAFCcGACA4sQAABQnBgCgODEAAMWJAQAoTgwAQHFiAACKEwMAUJwYAIDixAAAFCcGAKA4MQAAxYkBAChODABAcWIAAIoTAwBQnBgAgOLEAAAUJwYAoDgxAADFiQEAKK5fbw8AvHna29vT2dnZ22O87TU2Nmbw4MG9PQbsMjEARbS3t+fqb34z21tbe3uUt73+TU05/6tfFQS8ZYgBKKKzszPbW1vzkYEDc2hjY2+P87a1sbMzt7S2prOzUwzwliEGoJhDGxsz/MADe3uMt7fnn+/tCWC3OIEQAIoTAwBQnBgAgOLEAMAfMGfp0hx82WW9PQbsc2IA6OG+NWvS95JLcuYNN/TaDE8/91waZs7M0vXreyxf/uyzOevmm3PkVVelYebMXHX//Xu0/2k//Wk+dOONr1k+/+mn0zBzZp574YUkyTnHH5/Hv/CFHrf5xvz5ee+11+7R/cKfKjEA9DC7pSVfmDAhC1etyjNbt/b2OD10bt+eow4+OJedcUbeccAB+/z+Bvbvn2GDBu3z+4He5qOFQLeObdty0/Lleegzn8n6jo7MWbo0fzdxYo/b/Gzlylxw551Z096ek0eMyLQxYzLt1luz+cILc/CAAUmSxatX56Jf/CIPPfNMmhob8+Hjjsulp5+eQfvtlyQ58qqr8tlx4/Lkpk358aOP5pABA/KVU0/NZ8eNS5K863vfS5KMnTUrSXLaO9+Z+dOmZfzhh2f84YcnSb48b94+fz7mLF2aL95xR5778pf3+X1Bb3JkAOh28/LlOa6pKSObmnLu6NH5YUtLurq6utc/tXlzzr755nxo5MgsmzEj08eNy8V3391jH7/ZtClTfvSjnDVqVB6eMSM3nX12Fq9enfNvv73H7a64776c2NyclunT8/nx4/O5227Lyp1XR3zgvPOSJPM+8Ymsu+CC3HLOObv8GOYsXZqGmTP39CmAkhwZALrNbmnJuSeckCSZcswxaX/xxSxYtSqTjjwySTJryZKMbGrKdydPTpKMbGrKI88+m28vWtS9j0sXL87HTzghX3z/+5Mkxw4dmu9PnZrT5szJNWeemQH9Xvlv54PHHpvPjx+fJLnwlFNy5f3355dPP52RTU05dOeh+aGNjbv9dsDg/ffPyKFDX/d2P3/88Rzwne/0WPbyq8IHKhEDQJJkZWtrHli7Nj/Z+Vd4vz59cs7xx2d2S0t3DKxsa8v45uYe203Yedj+Py3bsCEPb9iQf/n1r7uXdSXZ0dWVpzZvzqhDD02SjB42rHt9Q0ND3nHAAXn2d797w4/jw6NG5cOjRr3u7T7wrnflmjPP7LHsV7/9bc79yU/e8AzwViMGgCSvHBV4aceONF9xRfeyriT79+2bq6dOzeCd5wO8no5t2zJ93Lj8zUknvWbdEa+6Vn//vn17rGvIK8HwZhnUv3+OGTKkx7Lfbtnypt0//CkRA0Be2rEj1y9blismT87ko4/use5DN96Yf33kkcw48cSMHDo0c594osf6B9eu7fH7+4YPz6MbN77mhXZ37LczFF7esWOP97GvfGPSpHxj0qTeHgP2KicQAvn5449n8wsv5NNjx+Y9w4b1+Dlr1KjMbmlJkkwfNy4rWltz4V135fG2tty8fHnmLFuW5JW/7JNX3v+/d82anD93bpauX58n2tpy64oVOX/u3F2eZ9igQRnYr1/uePLJbOjoSPvOz/1ve/nlLF2/PkvXr8+2l1/O2i1bsnT9+jy5aVP3tj957LEcd/XVe+eJ+T0umjcvH77ppn22f+gNYgDI7JaWnHHUUb/3rYCz3v3uPPTMM3l4w4a865BD8m8f/WhuWbEio6+5Jtc89FAu3vnRw/13nhg4+rDDsmDatDze1paJ112XsbNm5Wvz56d5N74psV+fPvn+1KmZtWRJmv/xH/M/dl4g6JmtWzN21qyMnTUr6zo68g/33Zexs2blvJ/9rHvb9hdfzMq2tjfydPxR6zo68tTmzfts/9AbGrq6nD4LFaxbty6zLroo04cO3atfYfzthQtz7ZIlWfOlL+21fb6Vrdu6NbPa2jL90kszfPjw3h4HdolzBoDd8oMHH8z45uYMbWzMPatX57v33pvzJ0zo7bGAN0AMALvliba2fGvhwmx6/vkcMXhwLjj55Fz0X65SCLy1iAFgt1w5ZUqunDKlt8cA9iInEAJAcWIAAIoTAwBQnHMGoJiNnZ29PcLbmueXtyIxAEU0Njamf1NTbmltTZ5/vrfHeVvr39SUxsbG3h4DdpmLDkEh7e3t6fSX6z7X2NiYwa/6Uib4UycGAKA4JxACQHFiAACKEwMAUJwYAIDixAAAFCcGAKA4MQAAxYkBACjO5YihEFcgfOtyVUP2JTEARbS3t+eb37w6ra3be3sU9kBTU/989avnCwL2CTEARXR2dqa1dXsGDvxIGhsP7e1x2A2dnRvT2npLOjs7xQD7hBiAYhobD82BBw7v7THYTb5okn3JCYQAUJwYAIDixAAAFCcGAN6AmTMbsmLFT5Mkzz33dGbObMj69Ut7dSbYXWIA6GHNmvtyySV9c8MNZ/baDH/oRXXJkv+T666bmMsvPySXX35Irr/+jKxd+8Bu73/OnEm5444vvmb50qVzctllB+/Z0G/AH5oH3ixiAOihpWV2Jkz4QlatWpitW5/p7XF6WLVqft7zno/lU5/6ZT796fsyePCI/PM/T86WLWt7ezR4S/PRQqDbtm0dWb78pnzmMw+lo2N9li6dk4kT/67HbVau/FnuvPOCtLevyYgRJ2fMmGm59dZpufDCzRkw4OAkyerVi/OLX1yUZ555KI2NTTnuuA/n9NMvzX77DUqSXHXVkRk37rPZtOnJPProjzNgwCE59dSvZNy4zyZJvve9dyVJZs0amyR55ztPy7Rp8/ORj/xLj1n+4i/+KY8++u956qlfZMyYT+6T52TFiluzYMHMbNz4aA48sDljxnwqp556cfr02bX/Pp9+ekHuuutvs2HDsgwcOCRjxnwqf/Zn39rl7eHN4MgA0G358pvT1HRcmppGZvToc9P
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33muser_proxy\u001b[0m (to chatbot):\n",
      "\n",
      "\u001b[32m***** Response from calling function \"python\" *****\u001b[0m\n",
      "(0.0, 1.0, 0.0, 1.0)\n",
      "\u001b[32m***************************************************\u001b[0m\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "llm_config = {\n",
    "    \"functions\": [\n",
    "        {\n",
    "            \"name\": \"python\",\n",
    "            \"description\": \"run cell in ipython and return the execution result.\",\n",
    "            \"parameters\": {\n",
    "                \"type\": \"object\",\n",
    "                \"properties\": {\n",
    "                    \"cell\": {\n",
    "                        \"type\": \"string\",\n",
    "                        \"description\": \"Valid Python cell to execute.\",\n",
    "                    }\n",
    "                },\n",
    "                \"required\": [\"cell\"],\n",
    "            },\n",
    "        },\n",
    "        {\n",
    "            \"name\": \"sh\",\n",
    "            \"description\": \"run a shell script and return the execution result.\",\n",
    "            \"parameters\": {\n",
    "                \"type\": \"object\",\n",
    "                \"properties\": {\n",
    "                    \"script\": {\n",
    "                        \"type\": \"string\",\n",
    "                        \"description\": \"Valid shell script to execute.\",\n",
    "                    }\n",
    "                },\n",
    "                \"required\": [\"script\"],\n",
    "            },\n",
    "        },\n",
    "    ],\n",
    "    \"config_list\": config_list,\n",
    "    \"timeout\": 120,\n",
    "}\n",
    "chatbot = autogen.AssistantAgent(\n",
    "    name=\"chatbot\",\n",
    "    system_message=\"For coding tasks, only use the functions you have been provided with. Reply TERMINATE when the task is done.\",\n",
    "    llm_config=llm_config,\n",
    ")\n",
    "\n",
    "# create a UserProxyAgent instance named \"user_proxy\"\n",
    "user_proxy = autogen.UserProxyAgent(\n",
    "    name=\"user_proxy\",\n",
    "    is_termination_msg=lambda x: x.get(\"content\", \"\") and x.get(\"content\", \"\").rstrip().endswith(\"TERMINATE\"),\n",
    "    human_input_mode=\"NEVER\",\n",
    "    max_consecutive_auto_reply=10,\n",
    "    code_execution_config={\"work_dir\": \"coding\"},\n",
    ")\n",
    "\n",
    "# define functions according to the function description\n",
    "from IPython import get_ipython\n",
    "\n",
    "def exec_python(cell):\n",
    "    ipython = get_ipython()\n",
    "    result = ipython.run_cell(cell)\n",
    "    log = str(result.result)\n",
    "    if result.error_before_exec is not None:\n",
    "        log += f\"\\n{result.error_before_exec}\"\n",
    "    if result.error_in_exec is not None:\n",
    "        log += f\"\\n{result.error_in_exec}\"\n",
    "    return log\n",
    "\n",
    "def exec_sh(script):\n",
    "    return user_proxy.execute_code_blocks([(\"sh\", script)])\n",
    "\n",
    "# register the functions\n",
    "user_proxy.register_function(\n",
    "    function_map={\n",
    "        \"python\": exec_python,\n",
    "        \"sh\": exec_sh,\n",
    "    }\n",
    ")\n",
    "\n",
    "# start the conversation\n",
    "user_proxy.initiate_chat(\n",
    "    chatbot,\n",
    "    message=\"Draw two agents chatting with each other with an example dialog. Don't add plt.show().\",\n",
    ")\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "flaml_dev",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}