autogen/notebook/agentchat_lmm_gpt-4v.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2c75da30",
   "metadata": {},
   "source": [
    "# Agent Chat with Multimodal Models: GPT-4V"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f51914c",
   "metadata": {},
   "source": [
    "### Before everything starts, install AutoGen with the `lmm` option\n",
    "```bash\n",
    "pip install \"pyautogen[lmm]>=0.2.3\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "67d45964",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "import random\n",
    "import time\n",
    "from typing import Any, Callable, Dict, List, Optional, Tuple, Type, Union\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import requests\n",
    "from PIL import Image\n",
    "from termcolor import colored\n",
    "\n",
    "import autogen\n",
    "from autogen import Agent, AssistantAgent, ConversableAgent, UserProxyAgent\n",
    "from autogen.agentchat.contrib.multimodal_conversable_agent import MultimodalConversableAgent"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e4faf59",
   "metadata": {},
   "source": [
    "<a id=\"app-1\"></a>\n",
    "## Application 1: Image Chat\n",
    "\n",
    "In this section, we present a straightforward dual-agent architecture to enable user to chat with a multimodal agent.\n",
    "\n",
    "\n",
    "First, we show this image and ask a question.\n",
    "![](https://th.bing.com/th/id/R.422068ce8af4e15b0634fe2540adea7a?rik=y4OcXBE%2fqutDOw&pid=ImgRaw&r=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3d5580e",
   "metadata": {},
   "source": [
    "Within the user proxy agent, we can decide to activate the human input mode or not (for here, we use human_input_mode=\"NEVER\" for conciseness). This allows you to interact with LMM in a multi-round dialogue, enabling you to provide feedback as the conversation unfolds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "b1db6f5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "config_list_4v = autogen.config_list_from_json(\n",
    "    \"OAI_CONFIG_LIST\",\n",
    "    filter_dict={\n",
    "        \"model\": [\"gpt-4-vision-preview\"],\n",
    "    },\n",
    ")\n",
    "\n",
    "\n",
    "config_list_gpt4 = autogen.config_list_from_json(\n",
    "    \"OAI_CONFIG_LIST\",\n",
    "    filter_dict={\n",
    "        \"model\": [\"gpt-4\", \"gpt-4-0314\", \"gpt4\", \"gpt-4-32k\", \"gpt-4-32k-0314\", \"gpt-4-32k-v0314\"],\n",
    "    },\n",
    ")\n",
    "\n",
    "gpt4_llm_config = {\"config_list\": config_list_gpt4, \"cache_seed\": 42}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "6868785c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 2, 4]\n"
     ]
    }
   ],
   "source": [
    "x = [1, 2, gpt4_llm_config]\n",
    "\n",
    "\n",
    "def replace(arr, x, y):\n",
    "    idx = arr.index(x)\n",
    "    if idx >= 0:\n",
    "        arr[idx] = y\n",
    "\n",
    "\n",
    "replace(x, gpt4_llm_config, 4)\n",
    "\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "67157629",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mUser_proxy\u001b[0m (to image-explainer):\n",
      "\n",
      "What's the breed of this dog here? \n",
      "<image>.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33mimage-explainer\u001b[0m (to User_proxy):\n",
      "\n",
      "The dog in the image appears to be a Poodle or a Poodle mix, possibly a Labradoodle or Goldendoodle, given its curly coat and overall facial structure. These types of dogs are known for their curly, hypoallergenic fur and friendly disposition. The exact breed can be difficult to determine from a photo alone, especially with mixed breeds, as they can inherit various traits from each parent breed.\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "image_agent = MultimodalConversableAgent(\n",
    "    name=\"image-explainer\",\n",
    "    max_consecutive_auto_reply=10,\n",
    "    llm_config={\"config_list\": config_list_4v, \"temperature\": 0.5, \"max_tokens\": 300},\n",
    ")\n",
    "\n",
    "user_proxy = autogen.UserProxyAgent(\n",
    "    name=\"User_proxy\",\n",
    "    system_message=\"A human admin.\",\n",
    "    human_input_mode=\"NEVER\",  # Try between ALWAYS or NEVER\n",
    "    max_consecutive_auto_reply=0,\n",
    "    code_execution_config={\n",
    "        \"use_docker\": False\n",
    "    },  # Please set use_docker=True if docker is available to run the generated code. Using docker is safer than running the generated code directly.\n",
    ")\n",
    "\n",
    "# Ask the question with an image\n",
    "user_proxy.initiate_chat(\n",
    "    image_agent,\n",
    "    message=\"\"\"What's the breed of this dog?\n",
    "<img https://th.bing.com/th/id/R.422068ce8af4e15b0634fe2540adea7a?rik=y4OcXBE%2fqutDOw&pid=ImgRaw&r=0>.\"\"\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f60521d",
   "metadata": {},
   "source": [
    "Now, input another image, and ask a followup question.\n",
    "\n",
    "![](https://th.bing.com/th/id/OIP.29Mi2kJmcHHyQVGe_0NG7QHaEo?pid=ImgDet&rs=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "73a2b234",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mUser_proxy\u001b[0m (to image-explainer):\n",
      "\n",
      "How about this breed? \n",
      "<image>\n",
      "\n",
      "Among the breeds, which one barks less?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33mimage-explainer\u001b[0m (to User_proxy):\n",
      "\n",
      "The dog in the image is a Siberian Husky, identifiable by its thick fur, erect triangular ears, and distinctive facial markings.\n",
      "\n",
      "Between Siberian Huskies and Poodles (or their mixes like Labradoodles and Goldendoodles), Siberian Huskies are generally known to be less prone to barking. They are more likely to howl or talk in other vocal ways. Poodles and their mixes can vary in their tendency to bark, but they are often more vocal than Huskies in terms of barking. However, individual temperaments can vary greatly within any breed, and training also plays a significant role in a dog's vocal behavior.\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "# Ask the question with an image\n",
    "user_proxy.send(\n",
    "    message=\"\"\"What is this breed?\n",
    "<img https://th.bing.com/th/id/OIP.29Mi2kJmcHHyQVGe_0NG7QHaEo?pid=ImgDet&rs=1>\n",
    "\n",
    "Among the breeds, which one barks less?\"\"\",\n",
    "    recipient=image_agent,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c40d0eb",
   "metadata": {},
   "source": [
    "<a id=\"app-2\"></a>\n",
    "## Application 2: Figure Creator\n",
    "\n",
    "Here, we define a `FigureCreator` agent, which contains three child agents: commander, coder, and critics.\n",
    "\n",
    "- Commander: interacts with users, runs code, and coordinates the flow between the coder and critics.\n",
    "- Coder: writes code for visualization.\n",
    "- Critics: LMM-based agent that provides comments and feedback on the generated image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e8eca993",
   "metadata": {},
   "outputs": [],
   "source": [
    "class FigureCreator(AssistantAgent):\n",
    "    def __init__(self, n_iters=2, **kwargs):\n",
    "        \"\"\"\n",
    "        Initializes a FigureCreator instance.\n",
    "\n",
    "        This agent facilitates the creation of visualizations through a collaborative effort among its child agents: commander, coder, and critics.\n",
    "\n",
    "        Parameters:\n",
    "            - n_iters (int, optional): The number of \"improvement\" iterations to run. Defaults to 2.\n",
    "            - **kwargs: keyword arguments for the parent AssistantAgent.\n",
    "        \"\"\"\n",
    "        super().__init__(**kwargs)\n",
    "        self.register_reply([Agent, None], reply_func=FigureCreator._reply_user, position=0)\n",
    "        self._n_iters = n_iters\n",
    "\n",
    "    def _reply_user(self, messages=None, sender=None, config=None):\n",
    "        if all((messages is None, sender is None)):\n",
    "            error_msg = f\"Either {messages=} or {sender=} must be provided.\"\n",
    "            logger.error(error_msg)  # noqa: F821\n",
    "            raise AssertionError(error_msg)\n",
    "\n",
    "        if messages is None:\n",
    "            messages = self._oai_messages[sender]\n",
    "\n",
    "        user_question = messages[-1][\"content\"]\n",
    "\n",
    "        ### Define the agents\n",
    "        commander = AssistantAgent(\n",
    "            name=\"Commander\",\n",
    "            human_input_mode=\"NEVER\",\n",
    "            max_consecutive_auto_reply=10,\n",
    "            system_message=\"Help me run the code, and tell other agents it is in the <img result.jpg> file location.\",\n",
    "            is_termination_msg=lambda x: x.get(\"content\", \"\").rstrip().endswith(\"TERMINATE\"),\n",
    "            code_execution_config={\"last_n_messages\": 3, \"work_dir\": \".\", \"use_docker\": False},\n",
    "            llm_config=self.llm_config,\n",
    "        )\n",
    "\n",
    "        critics = MultimodalConversableAgent(\n",
    "            name=\"Critics\",\n",
    "            system_message=\"\"\"Criticize the input figure. How to replot the figure so it will be better? Find bugs and issues for the figure.\n",
    "            Pay attention to the color, format, and presentation. Keep in mind of the reader-friendliness.\n",
    "            If you think the figures is good enough, then simply say NO_ISSUES\"\"\",\n",
    "            llm_config={\"config_list\": config_list_4v, \"max_tokens\": 300},\n",
    "            human_input_mode=\"NEVER\",\n",
    "            max_consecutive_auto_reply=1,\n",
    "            #     use_docker=False,\n",
    "        )\n",
    "\n",
    "        coder = AssistantAgent(\n",
    "            name=\"Coder\",\n",
    "            llm_config=self.llm_config,\n",
    "        )\n",
    "\n",
    "        coder.update_system_message(\n",
    "            coder.system_message\n",
    "            + \"ALWAYS save the figure in `result.jpg` file. Tell other agents it is in the <img result.jpg> file location.\"\n",
    "        )\n",
    "\n",
    "        # Data flow begins\n",
    "        commander.initiate_chat(coder, message=user_question)\n",
    "        img = Image.open(\"result.jpg\")\n",
    "        plt.imshow(img)\n",
    "        plt.axis(\"off\")  # Hide the axes\n",
    "        plt.show()\n",
    "\n",
    "        for i in range(self._n_iters):\n",
    "            commander.send(message=\"Improve <img result.jpg>\", recipient=critics, request_reply=True)\n",
    "\n",
    "            feedback = commander._oai_messages[critics][-1][\"content\"]\n",
    "            if feedback.find(\"NO_ISSUES\") >= 0:\n",
    "                break\n",
    "            commander.send(\n",
    "                message=\"Here is the feedback to your figure. Please improve! Save the result to `result.jpg`\\n\"\n",
    "                + feedback,\n",
    "                recipient=coder,\n",
    "                request_reply=True,\n",
    "            )\n",
    "            img = Image.open(\"result.jpg\")\n",
    "            plt.imshow(img)\n",
    "            plt.axis(\"off\")  # Hide the axes\n",
    "            plt.show()\n",
    "\n",
    "        return True, \"result.jpg\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "977b9017",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mUser\u001b[0m (to Figure Creator~):\n",
      "\n",
      "\n",
      "Plot a figure by using the data from here:\n",
      "https://raw.githubusercontent.com/vega/vega/main/docs/data/seattle-weather.csv\n",
      "\n",
      "I want to show both temperature high and low.\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mCommander\u001b[0m (to Coder):\n",
      "\n",
      "\n",
      "Plot a figure by using the data from here:\n",
      "https://raw.githubusercontent.com/vega/vega/main/docs/data/seattle-weather.csv\n",
      "\n",
      "I want to show both temperature high and low.\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mCoder\u001b[0m (to Commander):\n",
      "\n",
      "To achieve this, I will provide you with a Python script that uses the `pandas` library to read data from the CSV file located at the provided URL, and `matplotlib` to plot both the high and low temperatures. If you don't already have `pandas` and `matplotlib` installed, you will need to install them first by running `pip install pandas matplotlib`.\n",
      "\n",
      "Please execute the following Python code:\n",
      "\n",
      "```python\n",
      "# filename: plot_temperatures.py\n",
      "import pandas as pd\n",
      "import matplotlib.pyplot as plt\n",
      "\n",
      "# URL of the CSV file\n",
      "url = \"https://raw.githubusercontent.com/vega/vega/main/docs/data/seattle-weather.csv\"\n",
      "\n",
      "# Read the data from the URL\n",
      "data = pd.read_csv(url)\n",
      "\n",
      "# Extract the Date, Temperature High, and Temperature Low columns\n",
      "dates = pd.to_datetime(data['date'])\n",
      "temp_high = data['temp_max']\n",
      "temp_low = data['temp_min']\n",
      "\n",
      "# Plot the high and low temperatures\n",
      "plt.figure(figsize=(10, 5))\n",
      "plt.plot(dates, temp_high, label='High Temperature', color='r')\n",
      "plt.plot(dates, temp_low, label='Low Temperature', color='b')\n",
      "plt.xlabel('Date')\n",
      "plt.ylabel('Temperature (°C)')\n",
      "plt.title('High and Low Temperatures in Seattle')\n",
      "plt.legend()\n",
      "plt.grid(True)\n",
      "\n",
      "# Save the figure\n",
      "plt.savefig('result.jpg')\n",
      "\n",
      "# Show the plot\n",
      "plt.show()\n",
      "```\n",
      "\n",
      "After running this script, it will output a plot of the high and low temperatures and save the figure as `result.jpg` in the current directory. Open the `result.jpg` file to view the figure.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...\u001b[0m\n",
      "\u001b[33mCommander\u001b[0m (to Coder):\n",
      "\n",
      "exitcode: 0 (execution succeeded)\n",
      "Code output: \n",
      "Figure(1000x500)\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mCoder\u001b[0m (to Commander):\n",
      "\n",
      "The script executed successfully, and it created a figure of the high and low temperatures from the Seattle weather dataset. You should find the figure saved as `result.jpg` in the current directory.\n",
      "\n",
      "You can view this figure by opening the `result.jpg` file on your computer.\n",
      "\n",
      "If there's anything more you'd like to do with this plot or if you have any other requests, please let me know.\n",
      "\n",
      "TERMINATE\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAEMCAYAAABZZbUfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9e7xlV1XnDX/HnHPty7nVNZVU7gm5EEC5B8UGufdDEJVutX3VtvXj5dOt9tOvr/28vk8/bfdj24qPNvRFWxtREVEREQwCQQTBCwgiN0NCKrdKKlVJXU7VqXPde6815xzj/WOuvXMSkkBCQs6p2r98dursc/ZlrbnmmmPMMX7jN8TMjCmmmGKKKaaY4qyFe7IPYIoppphiiimmeHIxdQammGKKKaaY4izH1BmYYooppphiirMcU2dgiimmmGKKKc5yTJ2BKaaYYooppjjLMXUGpphiiimmmOIsx9QZmGKKKaaYYoqzHFNnYIoppphiiinOckydgSmmmGKKKaY4yzF1BqaYYoopppjiLMfUGZhiiimmmGKKsxxTZ2CKKaaYYoopznJMnYEppphiiimmOMsxdQammGKKKaaY4izH1BmYYooppphiirMcU2dgiimmmGKKKc5yTJ2BKaaYYooppjjLMXUGpphiiimmmOIsx9QZmGKKKaaYYoqzHFNnYIoppphiiinOckydgSmmmGKKKaY4yzF1BqaYYoopppjiLMfUGZhiiimmmGKKsxxTZ2CKKaaYYoopznJMnYEppphiiimmOMsxdQammGKKKaaY4izH1BmYYooppphiirMc4ck+gCmmeCJhZgAsLS0xOztLt9vFzFhcXGTv3r0MBgNEhNnZ2S95r6qyuLjIvn37EJHH5XhUldOnT7Nr1y6cK754zpnTp0+zZ8+eh/0eEZmcy8PBzLjvvvuIMU5eLyKce+659Hq9x+X4v1KMv3s0GuG9p6qqJ+R7Tp06xY4dOwghTL7zoZBzZmlpicFgQL/fZ8+ePXjvH/HYx1BVRqMRMzMzmBnD4ZBer4dzjuXlZXq9Hr1e7yGv0eM1b6aY4onGNDIwxRkNMyPGyOtf/3q+8IUvTBb2n/qpn+L48ePcfvvt3HXXXajq5BFjJKXE8vIy/+bf/BvW19cnv1NVUkrknIkxknOe/E5VyTkzGo0e8Pfx31SV1dVVfuZnfoZTp05NDMfJkyf56Z/+aTY2NiavH/+7+TPM7AEPKEYu5wxASon3ve99/P7v/z7f+73fy5ve9Cbe/e53s7i4iKpiZl9y3DFGmqahruvJcTZN84DzNbPJv+OfY4wPGLPN559Somkacs684x3v4O/+7u8eMAbjY37wY/M5Pvi7x8/H3zs+j09+8pOsra1R1/Xkdw/+jqZpuP766/m5n/s53vrWt/KGN7yBI0eOPOB1m6/r5t8BHD58mP/xP/4HMUaWl5f5pV/6JdbW1kgp8Wu/9mt84hOfmHzWg8driim2C6aRgSnOeIwX5pQSwMTQAayurtLv98k58773vY8DBw5w2WWX0e12+cZv/EYGgwHXX3899957L69+9at5+tOfjnMOVaWua2644QbuuOMOnvKUp/DqV7+apml429vexp49ezhx4gSvec1ruOyyyzhw4AA33HAD+/btY21tbfIZ3nvMjKZpJjtSVeXUqVP80R/9EadPn+bFL34xL3jBC3j729/O6173Om666SZuueUWfuiHfoh3vOMdvOxlL+Occ87BOccP//AP0zQNN910Ez/yIz/Czp07ueGGGzhy5AhPe9rTeNWrXsXf//3fc+uttzIYDFhYWODpT386H/3oR7nkkkt43etex6c//WluueUWVlZW6Ha7fPd3fzdzc3N86lOf4mMf+xgLCwt8x3d8B845fu/3fo9du3bR6/W45ppr+PCHPwzAy1/+ci688EL+9E//lF6vx80338y1117LiRMneNnLXsZdd93FgQMH+IZv+Abe8Y53sLCwQKfT4XnPex4f+MAH2NjY4NWvfjXPeMYz+PjHP84nPvEJqqriO7/zO7nwwguBsus+fvw4TdPwhS98gdtvv521tTVmZ2f5ru/6Lubm5ibX+33vex//+l//a572tKcB4L1nOBzygQ98gFtuuYUrr7yS6667jpQS119/PSdOnOBpT3saL3/5y/mLv/gL3vOe96CqXHbZZbz3ve+lrmte/epXT5zNGCN//ud/zo033sj+/fv5p//0n7KwsPC1nupTTPGYMY0MTHHGY7xTe8tb3sLP/dzP8YY3vIHDhw/jnOPAgQPceeedfPKTn+SGG27gNa95DTfffDN//dd/DcDp06eZm5vjBS94Ab/+678+2b2OQ/x79uzh2muv5ZOf/CQf+tCHWF1d5V3vehcXXXQRV1xxBW9+85tZXl7mF3/xF3nhC1/Inj17uPfeeye7+YfaPZoZ//2//3eqquK1r30tv/7rv87Bgwe56aabuPvuu/ngBz/IBz/4QQ4fPsyHP/xhQgiTz3POTY7NzPjN3/xNVlZW+JZv+Rb+7M/+jM997nPcfPPN/M3f/A0vf/nL+ehHP8r111/Pq171Kt7//vdz1113cfvtt3PDDTfwkpe8hNFoxNve9jZuuukm3vrWt3LdddexsLDAr/7qr7K6usof/dEfsX//fl74whfS7/d55jOfyYUXXsgb3vAGvPdcddVVvOpVr+K1r30tx44d48Ybb0REOHr0KJ/73OfY2Njgne98J+eddx7Pec5z+KVf+iUuv/xyXvziF/PGN76Ru+++mze96U0873nP4+UvfzkzMzOo6iT8/rd/+7esrq5yxx138MEPfpBXvOIVHDhwgE984hOT69TpdHjpS1/KL/zCL/Af/+N/5EMf+hAxRm644QY+//nP8+3f/u3cfPPNfOADH8A5x4UXXshznvMc3ve+9/G5z32Opz/96Vx22WV8z/d8D9/wDd/ApZdeyvd93/fx9V//9eSc8d7zV3/1V3z4wx/mda97HcvLy/zBH/zBNEUwxbbCNDIwxRmPcW74277t23jWs57FYDDgxhtvnOzEzYwDBw7wnOc8h2c84xmsrq7ynve8h6Zp2LVrF694xSuo65o3v/nN5Jzp9XqoKvfeey9vf/vbufzyy1lfX+fgwYM885nP5LzzzuMf/aN/xH333cf111/P4uIi3W6XF73oRSwtLfHud78bKBGLseHeDFXl7rvv5kd/9Ee54IILeNrTnsbBgwe59tpr+ehHP8r6+jrXXHMNH/nIRzj33HPZuXMnzjlyzhMDJCLEGPn4xz/OZZddxqlTp1heXub2229HRPimb/omrrnmGq655hquuOIKnvGMZ3DhhRdy9OhRRIRv/MZv5FnPehbz8/P84i/+IrOzsxw9epQ//dM/ZWNjg0OHDpFznpxrCIE3velNHDp0iHPOOWfyfQsLC+zZs4f9+/fzxS9+cTLm4wfAJZdcwjd90zdx5MgRbrnlFs4991y89ywuLlLXNd/4jd/I9ddfzwUXXMD3f//3T85vbOzHn/PiF7+Ya665hmc/+9kcOXJkMrbee/7ZP/tnvPSlL+Uzn/kMv/M7v8P6+jqf+cxnOHnyJO9+97s5duwYvV6PCy+8kHe+851ceumlLC0tcccdd3DttdcyOzvL+eefz9raGjMzM5x//vkTnomq8rd/+7ecOnWKd77znZw+fXqS7pg6BFNsF0ydgSnOaIzD7s459u3bx/79+xkOh3Q6nQeQ7Pbu3cunP/1pVldXOXDgACklQgh0Oh2g7Li99xOjO44qXHXVVfzIj/wIv/qrvzrZJXa73cl3O+eYnZ1lOBxy7NgxFhcXWVpawns/+f7xY319HeccKSV6vR4HDx5kYWGBu+++m+uuu45Op8Nv/MZv8KpXvYpnPvOZvPGNb+QHfuAHJqmGEMrtPD5f7z2XXXYZL3jBC3j2s59NSol9+/bx7ne/mxDC5DWbHYhxXv/gwYOcOnWKm2++mf3793PRRRdx5ZVX8k/+yT+ZjEmn06Hb7VJV1SRU/5M/+ZMA/PVf//XkmNbX19nY2GBmZobFxUXW1ta4+eabJ86Lc45er8fOnTu55JJLuO6
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mCommander\u001b[0m (to Critics):\n",
      "\n",
      "Improve <image>\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mCritics\u001b[0m (to Commander):\n",
      "\n",
      "The provided figure shows a time series plot of high and low temperatures in Seattle. Here are a few suggestions for improving the plot:\n",
      "\n",
      "1. **Resolution and Clarity**: The image appears to have low resolution, which makes the text and lines a bit blurry. Increasing the resolution would enhance clarity.\n",
      "\n",
      "2. **Color Contrast**: The colors chosen for high and low temperatures (red and blue) are good for contrast, but ensuring that they are distinguishable by all audiences, including those with color vision deficiencies, can be useful. Consider using colorblind-friendly palettes.\n",
      "\n",
      "3. **Line Thickness**: The lines look slightly thick, which can cause them to blur together when they cross or when viewed at smaller sizes. Thinner lines might provide better clarity.\n",
      "\n",
      "4. **Smoothing**: The data appears to be quite noisy. If the aim is to show trends, consider applying a smoothing filter or showing a moving average to make the general trends more apparent.\n",
      "\n",
      "5. **Legend**: The legend is clear, but it is positioned in such a way that it could potentially cover up data points. Consider placing the legend outside the main plot area or making it semi-transparent so that no data is obscured.\n",
      "\n",
      "6. **Axes Labels**: The font size for the axes labels and the title could be increased to improve legibility, especially if the image is scaled down.\n",
      "\n",
      "7. **Grid Lines**: The grid lines are helpful, but if they are too prominent, they can distract from the data. Lightening the grid\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mCommander\u001b[0m (to Coder):\n",
      "\n",
      "Here is the feedback to your figure. Please improve! Save the result to `result.jpg`\n",
      "The provided figure shows a time series plot of high and low temperatures in Seattle. Here are a few suggestions for improving the plot:\n",
      "\n",
      "1. **Resolution and Clarity**: The image appears to have low resolution, which makes the text and lines a bit blurry. Increasing the resolution would enhance clarity.\n",
      "\n",
      "2. **Color Contrast**: The colors chosen for high and low temperatures (red and blue) are good for contrast, but ensuring that they are distinguishable by all audiences, including those with color vision deficiencies, can be useful. Consider using colorblind-friendly palettes.\n",
      "\n",
      "3. **Line Thickness**: The lines look slightly thick, which can cause them to blur together when they cross or when viewed at smaller sizes. Thinner lines might provide better clarity.\n",
      "\n",
      "4. **Smoothing**: The data appears to be quite noisy. If the aim is to show trends, consider applying a smoothing filter or showing a moving average to make the general trends more apparent.\n",
      "\n",
      "5. **Legend**: The legend is clear, but it is positioned in such a way that it could potentially cover up data points. Consider placing the legend outside the main plot area or making it semi-transparent so that no data is obscured.\n",
      "\n",
      "6. **Axes Labels**: The font size for the axes labels and the title could be increased to improve legibility, especially if the image is scaled down.\n",
      "\n",
      "7. **Grid Lines**: The grid lines are helpful, but if they are too prominent, they can distract from the data. Lightening the grid\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mCoder\u001b[0m (to Commander):\n",
      "\n",
      "To address the feedback and improve the figure, I'll make several adjustments to the script. The improvements will include:\n",
      "\n",
      "1. Increase the resolution by specifying a higher DPI (dots per inch) when saving the figure.\n",
      "2. Use a colorblind-friendly palette for the high and low temperature lines.\n",
      "3. Decrease the line thickness for better clarity.\n",
      "4. Apply a rolling window to smooth the high and low temperature data.\n",
      "5. Move the legend to an optimal location to avoid covering data points and make it semi-transparent.\n",
      "6. Increase the font size for axes labels and title to improve legibility.\n",
      "7. Adjust grid line style to be less prominent.\n",
      "\n",
      "Please execute the updated Python script:\n",
      "\n",
      "```python\n",
      "# filename: plot_temperatures_improved.py\n",
      "import pandas as pd\n",
      "import matplotlib.pyplot as plt\n",
      "import seaborn as sns  # For colorblind-friendly palette\n",
      "\n",
      "# Ensure you have seaborn installed: pip install seaborn\n",
      "\n",
      "# URL of the CSV file\n",
      "url = \"https://raw.githubusercontent.com/vega/vega/main/docs/data/seattle-weather.csv\"\n",
      "\n",
      "# Read the data from the URL\n",
      "data = pd.read_csv(url)\n",
      "\n",
      "# Extract the Date, Temperature High, and Temperature Low columns\n",
      "dates = pd.to_datetime(data['date'])\n",
      "temp_high = data['temp_max'].rolling(window=7).mean()  # 7-day rolling average\n",
      "temp_low = data['temp_min'].rolling(window=7).mean()    # 7-day rolling average\n",
      "\n",
      "# Plot the high and low temperatures using seaborn's colorblind-friendly palette\n",
      "plt.figure(figsize=(10, 5), dpi=150)\n",
      "plt.plot(dates, temp_high, label='High Temperature', color=sns.color_palette('colorblind')[2], linewidth=1)\n",
      "plt.plot(dates, temp_low, label='Low Temperature', color=sns.color_palette('colorblind')[0], linewidth=1)\n",
      "plt.xlabel('Date', fontsize=12)\n",
      "plt.ylabel('Temperature (°C)', fontsize=12)\n",
      "plt.title('High and Low Temperatures in Seattle (7-day Rolling Average)', fontsize=14)\n",
      "\n",
      "# Adjusting the legend\n",
      "leg = plt.legend(loc='upper left', frameon=True)\n",
      "leg.get_frame().set_alpha(0.5)\n",
      "\n",
      "# Adjusting the grid\n",
      "plt.grid(True, linestyle='--', linewidth=0.5, color='grey', alpha=0.7)\n",
      "\n",
      "# Increase tick label size\n",
      "plt.tick_params(labelsize=10)\n",
      "\n",
      "# Save the figure with high resolution\n",
      "plt.savefig('result.jpg', format='jpg', dpi=150)\n",
      "\n",
      "# Show the plot\n",
      "plt.show()\n",
      "```\n",
      "\n",
      "This script should create a higher-quality figure with the suggested improvements. Once the script has run, please check the `result.jpg` file for the enhanced plot.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...\u001b[0m\n",
      "\u001b[33mCommander\u001b[0m (to Coder):\n",
      "\n",
      "exitcode: 0 (execution succeeded)\n",
      "Code output: \n",
      "Figure(1500x750)\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mCoder\u001b[0m (to Commander):\n",
      "\n",
      "The script executed successfully and created an improved figure with high resolution, colorblind-friendly colors, thinner lines, smoothed data, a better-positioned and semi-transparent legend, enhanced legibility for labels and title, and less prominent grid lines.\n",
      "\n",
      "The updated figure is saved as `result.jpg` in the current directory. You can view the figure by opening the `result.jpg` file on your computer.\n",
      "\n",
      "If there are any more changes you would like to make or if you need further assistance, feel free to let me know.\n",
      "\n",
      "TERMINATE\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAEMCAYAAABZZbUfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9d5xcxZnvj7+rTuo4WRrlgAABIgjBEmxEssnGkTVrHPY64h843b2GdVjjtVn2Onsd1r5f26zXGHudTbDJOecsISSSEsqTOp1Y9fvj9DnqGY2EwoxmxuqPXq3u6T6nTtVT6aknCq21pokmmmiiiSaa2Gchx7oCTTTRRBNNNNHE2KLJDDTRRBNNNNHEPo4mM9BEE0000UQT+ziazEATTTTRRBNN7ONoMgNNNNFEE000sY+jyQw00UQTTTTRxD6OJjPQRBNNNNFEE/s4msxAE0000UQTTezjaDIDTTTRRBNNNLGPo8kMNNFEE0000cQ+jiYz0EQTTTTRRBP7OJrMQBNNNNFEE03s42gyA0000UQTTTSxj6PJDDTRRBNNNNHEPo4mM9BEE0000UQT+ziazEATTTTRRBNN7ONoMgNNNNFEE000sY+jyQw00UQTTTTRxD6OJjPQRBNNNNFEE/s4msxAE0000UQTTezjaDIDTTTRRBNNNLGPo8kMNNFEE0000cQ+jiYz0EQTTTTRRBP7OJrMQBNNNNFEE03s42gyA0000UQTTTSxj6PJDDTRRBNNNNHEPg5zrCvQxJ5Ba52+9/b2YpomhUIBKSWe59Hb28vkyZN59dVX2bRpE8cee+yge4UQ6ecHH3yQqVOnMnfu3G1+F0Kkzxr6/MYykmuH+00pxU033cRJJ51EoVBIy/R9n56eHrq7u5FS7vRzEjTWLbkmiiLWrl2LUmpQGZZlMW3atG3alfw9XDuGPrPx+51BY3lD2yGEQCk1qMyh7Rnu83B1Ga7ejc9JaH3rrbfypje9iUwms91rk+uTe7Zs2UIURRSLRVpbW7f7nO3VaWidh7bpscceo62tjQMOOGC7dGysk1KK++67j8MPP5xMJsOGDRsGtbO7u5tcLrcNzZJr+vv7eeCBBzjjjDOQcufOREPng1KKjRs3UqvVEEJQLBZpa2tLx/DQftZa89BDDzF16lSmT5/OLbfcwimnnEK5XOa5557j5JNP3u44GI7ew32ntaa/v59qtcqUKVO2e9/Q9iR/N35+Pbo0tmt7dWoc4w8++CDz58+ns7NzEB2bGB9oMgN/A4iiCIDvfve77LfffvzjP/4jSileeOEFvvSlL/Hb3/6W3t5e1q1bly6mAEoppJTpZP3jH//ICSecwOzZs9PfYPCETTbXxoUk+a6xvOS+5Hlaa4Ig4Ac/+AGHHnoohUIhLfOll17ikksu4Xe/+x35fD5t03CLSvJ56HfJ34ZhpAvij3/8Y0qlEk8++SS5XI4DDzyQrq4uLrnkErLZ7KD6NW6ACT0SNH4/3GaZLP7b24yTspLvk+uT77/5zW9y9tlnc9hhh6XlRVE0iDFqLHt7C/DQfh1KKwDf93nxxRc58cQTsW07fV7CkAxdpHt7e7nsssvo7++nWCxSKpX4whe+wPz584fd6IfWJ2njSy+9xG9+8xs+//nPI4TgzjvvZO3atXzgAx9Aa811113HQQcdxP777z+IORpK86QtK1as4KqrruI//uM/WL16NT/60Y8IggDf97nrrrv49a9/zVFHHYVhGOk9SX2klGzYsIH//M//5E1vehOmaW7Tp1EUobXGNM1B9Gsc30EQ8E//9E94nsfkyZNZtWoVH/nIR3jrW986aHw2tuf3v/89J510Ep2dnfzwhz/kyCOPxHVdVq5cSRRFg+o79N7Gv4UQw27WYRjy1a9+lfvvv59rr72Wrq6utLyhY6JxLA5FY981zqvh5n1jvyRjqZHmyb1r1qzhnnvu4Z//+Z+bTMA4RJMZ+BtAsnm4rovv++n3Simq1SoALS0tdHd3A1Cr1bjjjjvo7+/n0EMPZcOGDZx22mnpJnrNNdcQBAGnnXYabW1tgyb7wMAA9913H+vWrWPu3Lm84Q1vwHEcnnzySaIoYtOmTQwMDHDqqacyadIklFI89dRTPPPMMxxyyCHbXXhc193mFLR582buvvtuPM/j+OOPZ86cOTz99NNIKTn88MO5//77UUqxePFili1bRqVSYdGiRQghaGlp4fLLL0cIwRe/+EVmzJjBxz/+ccrlMnfddRcbN27k8MMP54gjjsB1Xe644w5mzpzJ008/zcKFC5k9ezZ33nknAG9+85vJ5/M88MADZLNZXnrpJaSUnHbaabS0tBCGIcuWLePxxx+ntbWVk046iba2Nh5++GEsy+KVV15h6tSp7L///tx333309vZy+OGHs3DhQtatW8df//pXtmzZwoIFCzjnnHN46KGHOPXUU9NnJRKdW2+9lWnTpvHcc89x1FFHMWXKFO6++256e3s5+uijOfjgg1FK8dhjj7F06VKKxSKLFy9O+x3ANE3mzp2LlJKVK1fy8ssvp58XL16c/pbg4YcfZtOmTfzsZz8jk8lQqVRwHAetNX19fdx777309PRw9NFHs2DBAqIo4vHHH+f555+nWCxywgkn0NHRwc0338z1119PZ2cnBx98MNdffz0vv/wyrutyyimnDOr7arXK/fffz9q1azniiCM4/PDDsW170Jj5wx/+wEknnUQ+n2fevHl861vfQgjBQw89xKpVq5g/f346tpJ33/e57777eO2115g6dWpaVm9vL/fffz8bN25k/vz5HHPMMVSrVe6++27OPPNMLMtiYGCAu+++m7POOgvHcdJ7fd/nYx/7GCeffDI33XQTP/3pT1Npw+OPP87SpUuZPXs2b3jDG1IGtHGMCyHI5XLMnDkzrb9t2ylzcOqpp9Le3k4QBDz++OO88MILLFq0iJUrV3L66aeTy+UG0W7Dhg0sW7aMww8/nLvvvpt3vOMd3HnnnRx00EFMnz4drTW33347Bx10EFOnTuXZZ5/l8ccfZ9KkSZxyyink83nuvfdeWlpaWLFiBXPnzmXq1Kk88MADlEolFi1axGGHHYZhGFQqFe644w6q1SoLFixg48aNnHLKKSilePLJJ3n22WeZOXMmixcvJpPJcMopp3D11VfT09MzSDrQxPhA02bgbwTJAr5mzRqeeeYZnnnmGZYvX56esJ9++mmuueYaoiji29/+Nn/5y18QQnDFFVfwk5/8JD0V/eEPf6BcLvPII4/wf//v/x10QgZYtWoVK1eupFAo8Lvf/Y6f//znCCG44447+PznP8+mTZtYunQpl112WSoavOyyyzBNk+uuu47Vq1dvswgMJyrv7+/nk5/8JC+88ALlcpmLL76Yl19+mdWrV3PVVVcRBAHf+973+O53v4vv+1x99dWsXr0awzCQUmKaJqZppnSRUhIEAV/96ld55JFHyGQyfOc73+HOO++kVCrx5S9/mT/96U8IIfjsZz/L5ZdfTqVS4ZZbbuFnP/sZANdeey1f+MIXqFarPPLII3zxi1/E933uvvtuvva1r2EYBs8//zyXXXYZruty7bXX8vnPf55yuYxt2yxbtoy+vj6y2Wz6bMdxyGQydHV1MW3aNKIo4sc//jGVSiXdUG644QYAfvzjH/ONb3wjVQF9/vOfZ+nSpZimyeWXX85jjz3GI488wr//+7+TzWYplUqsWbNmkFTB932+//3v47ouy5Yt45/+6Z9YsmQJAwMDfOYzn6G/v3/Q5tLS0sKrr77KX/7yF1588UUsy8K2bSqVCl/84hdZsmRJ+vxHH32UcrnMU089RT6fZ8WKFXz+858nCALa2trIZDJMmzaN9vZ2isUixWKRadOmkc/nB6kxLr/8cu6//35yuRzf+973uPXWWweNF9/3eeyxxzjyyCORUiKlTE+f11xzDWeeeWYqYVJKoZRCa82VV17Jf//3f2OaJldffTXVahWtNa+88grr16+nUCjws5/9jD/96U/Yts0vf/lLli1bhhCC22+/ndtuuy09uQ8dv1prKpUKhUIBwzD44x//yDe/+U0ymQy/+tW
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mCommander\u001b[0m (to Critics):\n",
      "\n",
      "Improve <image>\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mCommander\u001b[0m (to Coder):\n",
      "\n",
      "Here is the feedback to your figure. Please improve! Save the result to `result.jpg`\n",
      "Improve <img result.jpg>\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mCoder\u001b[0m (to Commander):\n",
      "\n",
      "I'm sorry, but as an AI developed by OpenAI, I don't have the capability to view images directly or interpret visual feedback. My previous response provided an updated script intended to address the initial feedback points you provided.\n",
      "\n",
      "If there are specific aspects of the figure from the `result.jpg` file that you would like to further adjust, please provide clear textual feedback about what specific improvements you would like to see. For example, you can ask for:\n",
      "\n",
      "- Adjustments to the graphical elements such as line style or markers.\n",
      "- Changes to the axes or grid to improve visibility.\n",
      "- Modifications to the smoothing function or its parameters.\n",
      "- Any other specific visual modifications to the plot.\n",
      "\n",
      "Once I have your specific feedback, I can provide you with an updated code script to address those points.\n",
      "\n",
      "TERMINATE\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgMAAAEMCAYAAABZZbUfAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAEAAElEQVR4nOz9d5xcxZnvj7+rTuo4WRrlgAABIgjBEmxEssnGkTVrHPY64h843b2GdVjjtVn2Onsd1r5f26zXGHudTbDJOecsISSSEsqTOp1Y9fvj9DnqGY2EwoxmxuqPXq3u6T6nTtVT6aknCq21pokmmmiiiSaa2Gchx7oCTTTRRBNNNNHE2KLJDDTRRBNNNNHEPo4mM9BEE0000UQT+ziazEATTTTRRBNN7ONoMgNNNNFEE000sY+jyQw00UQTTTTRxD6OJjPQRBNNNNFEE/s4msxAE0000UQTTezjaDIDTTTRRBNNNLGPo8kMNNFEE0000cQ+jiYz0EQTTTTRRBP7OJrMQBNNNNFEE03s42gyA0000UQTTTSxj6PJDDTRRBNNNNHEPo4mM9BEE0000UQT+ziazEATTTTRRBNN7ONoMgNNNNFEE000sY+jyQw00UQTTTTRxD6OJjPQRBNNNNFEE/s4msxAE0000UQTTezjaDIDTTTRRBNNNLGPo8kMNNFEE0000cQ+jiYz0EQTTTTRRBP7OJrMQBNNNNFEE03s42gyA0000UQTTTSxj6PJDDTRRBNNNNHEPg5zrCvQxJ5Ba52+9/b2YpomhUIBKSWe59Hb28vkyZN59dVX2bRpE8cee+yge4UQ6ecHH3yQqVOnMnfu3G1+F0Kkzxr6/MYykmuH+00pxU033cRJJ51EoVBIy/R9n56eHrq7u5FS7vRzEjTWLbkmiiLWrl2LUmpQGZZlMW3atG3alfw9XDuGPrPx+51BY3lD2yGEQCk1qMyh7Rnu83B1Ga7ejc9JaH3rrbfypje9iUwms91rk+uTe7Zs2UIURRSLRVpbW7f7nO3VaWidh7bpscceo62tjQMOOGC7dGysk1KK++67j8MPP5xMJsOGDRsGtbO7u5tcLrcNzZJr+vv7eeCBBzjjjDOQcufOREPng1KKjRs3UqvVEEJQLBZpa2tLx/DQftZa89BDDzF16lSmT5/OLbfcwimnnEK5XOa5557j5JNP3u44GI7ew32ntaa/v59qtcqUKVO2e9/Q9iR/N35+Pbo0tmt7dWoc4w8++CDz58+ns7NzEB2bGB9oMgN/A4iiCIDvfve77LfffvzjP/4jSileeOEFvvSlL/Hb3/6W3t5e1q1bly6mAEoppJTpZP3jH//ICSecwOzZs9PfYPCETTbXxoUk+a6xvOS+5Hlaa4Ig4Ac/+AGHHnoohUIhLfOll17ikksu4Xe/+x35fD5t03CLSvJ56HfJ34ZhpAvij3/8Y0qlEk8++SS5XI4DDzyQrq4uLrnkErLZ7KD6NW6ACT0SNH4/3GaZLP7b24yTspLvk+uT77/5zW9y9tlnc9hhh6XlRVE0iDFqLHt7C/DQfh1KKwDf93nxxRc58cQTsW07fV7CkAxdpHt7e7nsssvo7++nWCxSKpX4whe+wPz584fd6IfWJ2njSy+9xG9+8xs+//nPI4TgzjvvZO3atXzgAx9Aa811113HQQcdxP777z+IORpK86QtK1as4KqrruI//uM/WL16NT/60Y8IggDf97nrrrv49a9/zVFHHYVhGOk9SX2klGzYsIH//M//5E1vehOmaW7Tp1EUobXGNM1B9Gsc30EQ8E//9E94nsfkyZNZtWoVH/nIR3jrW986aHw2tuf3v/89J510Ep2dnfzwhz/kyCOPxHVdVq5cSRRFg+o79N7Gv4UQw27WYRjy1a9+lfvvv59rr72Wrq6utLyhY6JxLA5FY981zqvh5n1jvyRjqZHmyb1r1qzhnnvu4Z//+Z+bTMA4RJMZ+BtAsnm4rovv++n3Simq1SoALS0tdHd3A1Cr1bjjjjvo7+/n0EMPZcOGDZx22mnpJnrNNdcQBAGnnXYabW1tgyb7wMAA9913H+vWrWPu3Lm84Q1vwHEcnnzySaIoYtOmTQwMDHDqqacyadIklFI89dRTPPPMMxxyyCHbXXhc193mFLR582buvvtuPM/j+OOPZ86cOTz99NNIKTn88MO5//77UUqxePFili1bRqVSYdGiRQghaGlp4fLLL0cIwRe/+EVmzJjBxz/+ccrlMnfddRcbN27k8MMP54gjjsB1Xe644w5mzpzJ008/zcKFC5k9ezZ33nknAG9+85vJ5/M88MADZLNZXnrpJaSUnHbaabS0tBCGIcuWLePxxx+ntbWVk046iba2Nh5++GEsy+KVV15h6tSp7L///tx333309vZy+OGHs3DhQtatW8df//pXtmzZwoIFCzjnnHN46KGHOPXUU9NnJRKdW2+9lWnTpvHcc89x1FFHMWXKFO6++256e3s5+uijOfjgg1FK8dhjj7F06VKKxSKLFy9O+x3ANE3mzp2LlJKVK1fy8ssvp58XL16c/pbg4YcfZtOmTfzsZz8jk8lQqVRwHAetNX19fdx777309PRw9NFHs2DBAqIo4vHHH+f555+nWCxywgkn0NHRwc0338z1119PZ2cnBx98MNdffz0vv/wyrutyyimnDOr7arXK/fffz9q1azniiCM4/PDDsW170Jj5wx/+wEknnUQ+n2fevHl861vfQgjBQw89xKpVq5g/f346tpJ33/e57777eO2115g6dWpaVm9vL/fffz8bN25k/vz5HHPMMVSrVe6++27OPPNMLMtiYGCAu+++m7POOgvHcdJ7fd/nYx/7GCeffDI33XQTP/3pT1Npw+OPP87SpUuZPXs2b3jDG1IGtHGMCyHI5XLMnDkzrb9t2ylzcOqpp9Le3k4QBDz++OO88MILLFq0iJUrV3L66aeTy+UG0W7Dhg0sW7aMww8/nLvvvpt3vOMd3HnnnRx00EFMnz4drTW33347Bx10EFOnTuXZZ5/l8ccfZ9KkSZxyyink83nuvfdeWlpaWLFiBXPnzmXq1Kk88MADlEolFi1axGGHHYZhGFQqFe644w6q1SoLFixg48aNnHLKKSilePLJJ3n22WeZOXMmixcvJpPJcMopp3D11VfT09MzSDrQxPhA02bgbwTJAr5mzRqeeeYZnnnmGZYvX56esJ9++mmuueYaoiji29/+Nn/5y18QQnDFFVfwk5/8JD0V/eEPf6BcLvPII4/wf//v/x10QgZYtWoVK1eupFAo8Lvf/Y6f//znCCG44447+PznP8+mTZtYunQpl112WSoavOyyyzBNk+uuu47Vq1dvswgMJyrv7+/nk5/8JC+88ALlcpmLL76Yl19+mdWrV3PVVVcRBAHf+973+O53v4vv+1x99dWsXr0awzCQUmKaJqZppnSRUhIEAV/96ld55JFHyGQyfOc73+HOO++kVCrx5S9/mT/96U8IIfjsZz/L5ZdfTqVS4ZZbbuFnP/sZANdeey1f+MIXqFarPPLII3zxi1/E933uvvtuvva1r2EYBs8//zyXXXYZruty7bXX8vnPf55yuYxt2yxbtoy+vj6y2Wz6bMdxyGQydHV1MW3aNKIo4sc//jGVSiXdUG644QYAfvzjH/ONb3wjVQF9/vOfZ+nSpZimyeWXX85jjz3GI488wr//+7+TzWYplUqsWbNmkFTB932+//3v47ouy5Yt45/+6Z9YsmQJAwMDfOYzn6G/v3/Q5tLS0sKrr77KX/7yF1588UUsy8K2bSqVCl/84hdZsmRJ+vxHH32UcrnMU089RT6fZ8WKFXz+858nCALa2trIZDJMmzaN9vZ2isUixWKRadOmkc/nB6kxLr/8cu6//35yuRzf+973uPXWWweNF9/3eeyxxzjyyCORUiKlTE+f11xzDWeeeWYqYVJKoZRCa82VV17Jf//3f2OaJldffTXVahWtNa+88grr16+nUCjws5/9jD/96U/Yts0vf/lLli1bhhCC22+/ndtuuy09uQ8dv1prKpUKhUIBwzD44x//yDe/+U0ymQy/+tW
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mFigure Creator~\u001b[0m (to User):\n",
      "\n",
      "result.jpg\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "creator = FigureCreator(name=\"Figure Creator~\", llm_config=gpt4_llm_config)\n",
    "\n",
    "user_proxy = autogen.UserProxyAgent(\n",
    "    name=\"User\", human_input_mode=\"NEVER\", max_consecutive_auto_reply=0, code_execution_config={\"use_docker\": False}\n",
    ")\n",
    "\n",
    "user_proxy.initiate_chat(\n",
    "    creator,\n",
    "    message=\"\"\"\n",
    "Plot a figure by using the data from:\n",
    "https://raw.githubusercontent.com/vega/vega/main/docs/data/seattle-weather.csv\n",
    "\n",
    "I want to show both temperature high and low.\n",
    "\"\"\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f0a58827",
   "metadata": {},
   "outputs": [],
   "source": [
    "if os.path.exists(\"result.jpg\"):\n",
    "    os.remove(\"result.jpg\")  # clean up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c6206648",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "a95d87c2",
   "metadata": {},
   "source": [
    "## Group Chat Example with Multimodal Agent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "56bd5742",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mUser_proxy\u001b[0m (to chat_manager):\n",
      "\n",
      "Describe the image:\n",
      "                        <img https://th.bing.com/th/id/R.422068ce8af4e15b0634fe2540adea7a?rik=y4OcXBE%2fqutDOw&pid=ImgRaw&r=0>.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33mimage-explainer-1\u001b[0m (to chat_manager):\n",
      "\n",
      "In the soft embrace of a quiet room, a curly-haired sentinel of joy rests upon the floor—a puppy, with fur the color of sunlit autumn leaves. Its innocent eyes, deep and dark as a twilight sky, gaze into the world with wonder and gentle curiosity. Adorned with a collar that boasts a bright, mosaic-like pattern, the young canine wears a name tag shaped like a heart, whispering the promise of unconditional love and companionship. In the background, a pair of black boots stands sentinel, hinting at the return of a beloved owner, while the puppy waits, the embodiment of warmth and affection in an often too brisk world.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33mUser_proxy\u001b[0m (to chat_manager):\n",
      "\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33mUser_proxy\u001b[0m (to chat_manager):\n",
      "\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33mUser_proxy\u001b[0m (to chat_manager):\n",
      "\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "agent1 = MultimodalConversableAgent(\n",
    "    name=\"image-explainer-1\",\n",
    "    max_consecutive_auto_reply=10,\n",
    "    llm_config={\"config_list\": config_list_4v, \"temperature\": 0.5, \"max_tokens\": 300},\n",
    "    system_message=\"Your image description is poetic and engaging.\",\n",
    ")\n",
    "agent2 = MultimodalConversableAgent(\n",
    "    name=\"image-explainer-2\",\n",
    "    max_consecutive_auto_reply=10,\n",
    "    llm_config={\"config_list\": config_list_4v, \"temperature\": 0.5, \"max_tokens\": 300},\n",
    "    system_message=\"Your image description is factual and to the point.\",\n",
    ")\n",
    "\n",
    "user_proxy = autogen.UserProxyAgent(\n",
    "    name=\"User_proxy\",\n",
    "    system_message=\"Ask both image explainer 1 and 2 for their description.\",\n",
    "    human_input_mode=\"TERMINATE\",  # Try between ALWAYS, NEVER, and TERMINATE\n",
    "    max_consecutive_auto_reply=10,\n",
    "    code_execution_config={\n",
    "        \"use_docker\": False\n",
    "    },  # Please set use_docker=True if docker is available to run the generated code. Using docker is safer than running the generated code directly.\n",
    ")\n",
    "\n",
    "# We set max_round to 5\n",
    "groupchat = autogen.GroupChat(agents=[agent1, agent2, user_proxy], messages=[], max_round=5)\n",
    "group_chat_manager = autogen.GroupChatManager(groupchat=groupchat, llm_config=gpt4_llm_config)\n",
    "\n",
    "user_proxy.initiate_chat(\n",
    "    group_chat_manager,\n",
    "    message=\"\"\"Describe the image:\n",
    "                        <img https://th.bing.com/th/id/R.422068ce8af4e15b0634fe2540adea7a?rik=y4OcXBE%2fqutDOw&pid=ImgRaw&r=0>.\"\"\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "00bd473d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "defaultdict(list,\n",
       "            {<autogen.agentchat.groupchat.GroupChatManager at 0x7f690545b820>: [{'content': [{'type': 'text',\n",
       "                 'text': 'Describe the image:\\n                        '},\n",
       "                {'type': 'image_url',\n",
       "                 'image_url': {'url': <PIL.Image.Image image mode=RGB size=1920x1080>}},\n",
       "                {'type': 'text', 'text': '.'}],\n",
       "               'name': 'User_proxy',\n",
       "               'role': 'user'},\n",
       "              {'content': [{'type': 'text',\n",
       "                 'text': 'In the soft embrace of a quiet room, a curly-haired sentinel of joy rests upon the floor—a puppy, with fur the color of sunlit autumn leaves. Its innocent eyes, deep and dark as a twilight sky, gaze into the world with wonder and gentle curiosity. Adorned with a collar that boasts a bright, mosaic-like pattern, the young canine wears a name tag shaped like a heart, whispering the promise of unconditional love and companionship. In the background, a pair of black boots stands sentinel, hinting at the return of a beloved owner, while the puppy waits, the embodiment of warmth and affection in an often too brisk world.'}],\n",
       "               'role': 'assistant'},\n",
       "              {'content': [{'type': 'text', 'text': ''}],\n",
       "               'name': 'User_proxy',\n",
       "               'role': 'user'},\n",
       "              {'content': [{'type': 'text', 'text': ''}],\n",
       "               'name': 'User_proxy',\n",
       "               'role': 'user'},\n",
       "              {'content': [{'type': 'text', 'text': ''}],\n",
       "               'name': 'User_proxy',\n",
       "               'role': 'user'}]})"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agent1._oai_messages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39d293fb",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}