autogen/notebook/autogen_agent_human_feedback.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/autogen_agent_human_feedback.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Interactive LLM Agent with Human Feedback\n",
    "\n",
    "FLAML offers an experimental feature of interactive LLM agents, which can be used to solve various tasks with human or automatic feedback, including tasks that require using tools via code.\n",
    "\n",
    "In this notebook, we demonstrate how to use `AssistantAgent` and `UserProxyAgent` to solve a challenging math problem with human feedback. Here `AssistantAgent` is an LLM-based agent that can write Python code (in a Python coding block) for a user to execute for a given task. `UserProxyAgent` is an agent which serves as a proxy for a user to execute the code written by `AssistantAgent`. By setting `human_input_mode` properly, the `UserProxyAgent` can also prompt the user for feedback to `AssistantAgent`. For example, when `human_input_mode` is set to \"ALWAYS\", the `UserProxyAgent` will always prompt the user for feedback. When user feedback is provided, the `UserProxyAgent` will directly pass the feedback to `AssistantAgent` without doing any additional steps. When no user feedback is provided, the `UserProxyAgent` will execute the code written by `AssistantAgent` directly and return the execution results (success or failure and corresponding outputs) to `AssistantAgent`.\n",
    "\n",
    "## Requirements\n",
    "\n",
    "FLAML requires `Python>=3.8`. To run this notebook example, please install flaml with the [autogen] option:\n",
    "```bash\n",
    "pip install flaml[autogen]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-13T23:40:52.317406Z",
     "iopub.status.busy": "2023-02-13T23:40:52.316561Z",
     "iopub.status.idle": "2023-02-13T23:40:52.321193Z",
     "shell.execute_reply": "2023-02-13T23:40:52.320628Z"
    }
   },
   "outputs": [],
   "source": [
    "# %pip install flaml[autogen]==2.0.0rc3"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set your API Endpoint\n",
    "\n",
    "The [`config_list_gpt4_gpt35`](https://microsoft.github.io/FLAML/docs/reference/autogen/oai/openai_utils#config_list_gpt4_gpt35) function tries to create a list of gpt-4 and gpt-3.5 configurations using Azure OpenAI endpoints and OpenAI endpoints. It assumes the api keys and api bases are stored in the corresponding environment variables or local txt files:\n",
    "\n",
    "- OpenAI API key: os.environ[\"OPENAI_API_KEY\"] or `openai_api_key_file=\"key_openai.txt\"`.\n",
    "- Azure OpenAI API key: os.environ[\"AZURE_OPENAI_API_KEY\"] or `aoai_api_key_file=\"key_aoai.txt\"`. Multiple keys can be stored, one per line.\n",
    "- Azure OpenAI API base: os.environ[\"AZURE_OPENAI_API_BASE\"] or `aoai_api_base_file=\"base_aoai.txt\"`. Multiple bases can be stored, one per line.\n",
    "\n",
    "It's OK to have only the OpenAI API key, or only the Azure OpenAI API key + base.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from flaml import oai\n",
    "\n",
    "config_list = oai.config_list_gpt4_gpt35()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The config list looks like the following:\n",
    "```python\n",
    "config_list = [\n",
    "    {\n",
    "        'model': 'gpt-4',\n",
    "        'api_key': '<your OpenAI API key here>',\n",
    "    },  # only if OpenAI API key is found\n",
    "    {\n",
    "        'model': 'gpt-4',\n",
    "        'api_key': '<your first Azure OpenAI API key here>',\n",
    "        'api_base': '<your first Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-03-15-preview',\n",
    "    },  # only if the at least one Azure OpenAI API key is found\n",
    "    {\n",
    "        'model': 'gpt-4',\n",
    "        'api_key': '<your second Azure OpenAI API key here>',\n",
    "        'api_base': '<your second Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-03-15-preview',\n",
    "    },  # only if the second Azure OpenAI API key is found\n",
    "    {\n",
    "        'model': 'gpt-3.5-turbo',\n",
    "        'api_key': '<your OpenAI API key here>',\n",
    "    },  # only if OpenAI API key is found\n",
    "    {\n",
    "        'model': 'gpt-3.5-turbo',\n",
    "        'api_key': '<your first Azure OpenAI API key here>',\n",
    "        'api_base': '<your first Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-03-15-preview',\n",
    "    },  # only if the at least one Azure OpenAI API key is found\n",
    "    {\n",
    "        'model': 'gpt-3.5-turbo',\n",
    "        'api_key': '<your second Azure OpenAI API key here>',\n",
    "        'api_base': '<your second Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-03-15-preview',\n",
    "    },  # only if the second Azure OpenAI API key is found\n",
    "]\n",
    "```\n",
    "\n",
    "You can directly override it if the above function returns an empty list, i.e., it doesn't find the keys in the specified locations."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Construct Agents\n",
    "\n",
    "We construct the assistant agent and the user proxy agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from flaml.autogen.agent import AssistantAgent, UserProxyAgent\n",
    "\n",
    "# create an AssistantAgent instance named \"assistant\"\n",
    "assistant = AssistantAgent(name=\"assistant\", request_timeout=600, seed=42, config_list=config_list)\n",
    "# create a UserProxyAgent instance named \"user\"\n",
    "user = UserProxyAgent(\n",
    "    name=\"user\",\n",
    "    human_input_mode=\"ALWAYS\",\n",
    "    is_termination_msg=lambda x: x.get(\"content\", \"\").rstrip().endswith(\"TERMINATE\"),\n",
    ")\n",
    "\n",
    "# the purpose of the following line is to log the conversation history\n",
    "oai.ChatCompletion.start_logging()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Perform a task\n",
    "\n",
    "We invoke the `receive()` method of the assistant agent to start the conversation. When you run the cell below, you will be prompted to provide feedback after receving a message from the assistant agent. If you don't provide any feedback (by pressing Enter directly), the user proxy agent will try to execute the code suggested by the assistant agent on behalf of you, or terminate if the assistant agent sends a \"TERMINATE\" signal in the end of the message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "user (to assistant):\n",
      "\n",
      "\n",
      "     Find $a + b + c$, given that $x+y \\neq -1$ and \n",
      "     \\begin{align}\n",
      "\tax + by + c & = x + 7,\\\n",
      "\ta + bx + cy & = 2x + 6y,\\\n",
      "\tay + b + cx & = 4x + y.\n",
      "\t\\end{align}.\n",
      "    \n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "Let's solve the system of equations step by step. First, we can solve the third equation for $a$, we get:\n",
      "\n",
      "$$a = \\frac{4x + y - b - cx}{y}$$\n",
      "\n",
      "Substituting this value of $a$ into the second equation gives:\n",
      "\n",
      "$$\\frac{4x + y - b - cx}{y} + bx + cy = 2x + 6y.$$\n",
      "\n",
      "Expanding and rearranging terms, we get:\n",
      "\n",
      "$$(4 - c)x + \\left(\\frac{1}{y} - b\\right)y - \\frac{b}{y} = 2x + 6y.$$\n",
      "\n",
      "Simplifying further, we have:\n",
      "\n",
      "$$(4 - c)x - b = 2x + 6y.$$\n",
      "\n",
      "Rearranging terms gives:\n",
      "\n",
      "$$(2 - c)x - b - 6y = 0. \\qquad\\qquad (4)$$\n",
      "\n",
      "Now, we can solve the first equation for $c$, we get:\n",
      "\n",
      "$$c = x + 7 - ax - by.$$\n",
      "\n",
      "Substituting this value of $c$ into the fourth equation gives:\n",
      "\n",
      "$$(2 - (x + 7 - ax - by))x - b - 6y = 0.$$\n",
      "\n",
      "Expanding and rearranging terms, we obtain:\n",
      "\n",
      "$$2x - 6y - x - 7 + ax + by - b - 6y = 0.$$\n",
      "\n",
      "Simplifying further, we have:\n",
      "\n",
      "$$2x - x - 7 + ax - 6y + by - b - 6y = 0.$$\n",
      "\n",
      "Rearranging terms gives:\n",
      "\n",
      "$$ax + by + (2 - 2x - 12y) - 7 - b = 0.$$\n",
      "\n",
      "Simplifying, we have:\n",
      "\n",
      "$$ax + by - 2x - 12y - b - 7 = 0. \\qquad\\qquad (5)$$\n",
      "\n",
      "Now, we can solve equations (4) and (5) simultaneously to find the values of $a$, $b$, $x$, and $y$. From equation (4), we have:\n",
      "\n",
      "$$(2 - c)x - b - 6y = 0.$$\n",
      "\n",
      "Substituting the value of $c$ from equation (5), we get:\n",
      "\n",
      "$$(2 - (ax + by - 2x - 12y - b - 7))x - b - 6y = 0.$$\n",
      "\n",
      "Expanding and rearranging terms, we obtain:\n",
      "\n",
      "$$2x - (2ax + 2by - 4x - 24y - 2b - 14)x - b - 6y = 0.$$\n",
      "\n",
      "Simplifying further, we have:\n",
      "\n",
      "$$2x - 2ax - 2by + 4x + 24y + 2b + 14x - b - 6y = 0.$$\n",
      "\n",
      "Rearranging terms gives:\n",
      "\n",
      "$$(2 + 4 + 14 - 2a)x + (-2b - 6 + 24 - b)y + (2 - 6 + 14) = 0.$$\n",
      "\n",
      "Simplifying, we have:\n",
      "\n",
      "$$(16 - 2a)x + (-3b + 18)y + 10 = 0. \\qquad\\qquad (6)$$\n",
      "\n",
      "Now, we can solve equations (5) and (6) simultaneously to find the values of $a$, $b$, $x$, and $y$. From equation (5), we have:\n",
      "\n",
      "$$ax + by - 2x - 12y - b - 7 = 0.$$\n",
      "\n",
      "Substituting the value of $ax + by$ from equation (6), we get:\n",
      "\n",
      "$$(16 - 2a)x + (-3b + 18)y + 10 - 2x - 12y - b - 7 = 0.$$\n",
      "\n",
      "Expanding and rearranging terms, we obtain:\n",
      "\n",
      "$$(16 - 2a - 2 + 2)x + (-3b + 18 - 12 - b)y + (10 - 7) = 0.$$\n",
      "\n",
      "Simplifying further, we have:\n",
      "\n",
      "$$(14 - 2a)x + (-4b + 6)y + 3 = 0.$$\n",
      "\n",
      "Rearranging terms gives:\n",
      "\n",
      "$$(14 - 2a)x + (-4b + 6)y + 3 = 0. \\qquad\\qquad (7)$$\n",
      "\n",
      "Now, we have a system of two equations:\n",
      "\n",
      "Equation (6): $$(16 - 2a)x + (-3b + 18)y + 10 = 0$$\n",
      "Equation (7): $$(14 - 2a)x + (-4b + 6)y + 3 = 0$$\n",
      "\n",
      "We can solve these equations simultaneously to find the values of $a$, $b$, $x$, and $y$.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "user (to assistant):\n",
      "\n",
      "The question is to find a+b+c\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "My apologies for the confusion. Let's solve the given system of linear equations and find the value of $a + b + c$. We can use the method of substitution to eliminate variables and find the solution step by step.\n",
      "\n",
      "```python\n",
      "from sympy import symbols, Eq, solve\n",
      "\n",
      "a, b, c, x, y = symbols('a b c x y')\n",
      "eq1 = Eq(a*x + b*y + c, x+7)\n",
      "eq2 = Eq(a + b*x + c*y, 2*x+6*y)\n",
      "eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
      "\n",
      "sol = solve((eq1, eq2, eq3), (a, b, c, x, y))\n",
      "sol_sum_a_b_c = sol[a] + sol[b] + sol[c]\n",
      "print(sol_sum_a_b_c)\n",
      "```\n",
      "\n",
      "This program will solve the given system of equations and output the sum $a + b + c$.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 1 (execution failed)\n",
      "Code output: \n",
      "Traceback (most recent call last):\n",
      "  File \"/workspaces/FLAML/flaml/autogen/extensions/tmp_code_653e3a5debe68e7daf371ec5f72b4100.py\", line 9, in <module>\n",
      "    sol_sum_a_b_c = sol[a] + sol[b] + sol[c]\n",
      "TypeError: list indices must be integers or slices, not Symbol\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "My apologies for the error. It seems that `solve` returned multiple solutions as a list instead of a single solution. Let's modify the code to handle this case and calculate the sum `a + b + c` for each solution:\n",
      "\n",
      "```python\n",
      "from sympy import symbols, Eq, solve\n",
      "\n",
      "a, b, c, x, y = symbols('a b c x y')\n",
      "eq1 = Eq(a*x + b*y + c, x+7)\n",
      "eq2 = Eq(a + b*x + c*y, 2*x+6*y)\n",
      "eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
      "\n",
      "solutions = solve((eq1, eq2, eq3), (a, b, c, x, y))\n",
      "sums_a_b_c = []\n",
      "for sol in solutions:\n",
      "    sum_a_b_c = sol[a] + sol[b] + sol[c]\n",
      "    sums_a_b_c.append(sum_a_b_c)\n",
      "\n",
      "print(sums_a_b_c)\n",
      "```\n",
      "\n",
      "This updated code will handle multiple solutions and output a list of sums `a + b + c` for each solution.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 1 (execution failed)\n",
      "Code output: \n",
      "Traceback (most recent call last):\n",
      "  File \"/workspaces/FLAML/flaml/autogen/extensions/tmp_code_10a52342307c73b854b79112bc22a7a6.py\", line 11, in <module>\n",
      "    sum_a_b_c = sol[a] + sol[b] + sol[c]\n",
      "TypeError: tuple indices must be integers or slices, not Symbol\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "Apologies again for the mistake. It seems that the solutions returned by `solve` are in the form of tuples instead of dictionaries. Let's update the code to correctly access the values and calculate the sum `a + b + c` for each solution:\n",
      "\n",
      "```python\n",
      "from sympy import symbols, Eq, solve\n",
      "\n",
      "a, b, c, x, y = symbols('a b c x y')\n",
      "eq1 = Eq(a*x + b*y + c, x+7)\n",
      "eq2 = Eq(a + b*x + c*y, 2*x+6*y)\n",
      "eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
      "\n",
      "solutions = solve((eq1, eq2, eq3), (a, b, c, x, y))\n",
      "sums_a_b_c = []\n",
      "for sol in solutions:\n",
      "    a_val = sol[0]\n",
      "    b_val = sol[1]\n",
      "    c_val = sol[2]\n",
      "    sum_a_b_c = a_val + b_val + c_val\n",
      "    sums_a_b_c.append(sum_a_b_c)\n",
      "\n",
      "print(sums_a_b_c)\n",
      "```\n",
      "\n",
      "This updated code will correctly handle the solutions in tuple form and calculate the sum `a + b + c` for each solution.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 0 (execution succeeded)\n",
      "Code output: \n",
      "[(2*x**2 - 3*x + y)/(x**2 - x*y - x + y**2 - y + 1) + (x**2 - 3*x*y + 2*x + y**2 - y)/(x**2 - x*y - x + y**2 - y + 1) + (4*x**2 - 4*x*y - 6*x + 6*y**2 - 7*y + 7)/(x**2 - x*y - x + y**2 - y + 1)]\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "I apologize for the confusion. It seems that the given system of equations does not have a unique solution, and the expressions for `a + b + c` involve variables `x` and `y`. \n",
      "\n",
      "To find the value of `a + b + c`, we need specific values for `x` and `y` to substitute into the expressions. Could you please provide the values of `x` and `y` that you would like to use?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "user (to assistant):\n",
      "\n",
      "x+y \\neq -1\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "Thank you for providing the additional constraint. To find the value of `a + b + c`, we will substitute the given values for `x` and `y` into the expressions we obtained previously.\n",
      "\n",
      "```python\n",
      "a_val = sol[0].subs({x: 1, y: 2})\n",
      "b_val = sol[1].subs({x: 1, y: 2})\n",
      "c_val = sol[2].subs({x: 1, y: 2})\n",
      "sum_a_b_c = a_val + b_val + c_val\n",
      "\n",
      "print(sum_a_b_c)\n",
      "```\n",
      "\n",
      "Please replace `1` and `2` with the specific values of `x` and `y` that you would like to use. This code will substitute the values of `x` and `y` into the expressions and output the value of `a + b + c`.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 1 (execution failed)\n",
      "Code output: \n",
      "Traceback (most recent call last):\n",
      "  File \"/workspaces/FLAML/flaml/autogen/extensions/tmp_code_6055b4e0fd6e14f6c218b3d95712ecc2.py\", line 1, in <module>\n",
      "    a_val = sol[0].subs({x: 1, y: 2})\n",
      "NameError: name 'sol' is not defined\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "Apologies for the confusion. It seems that there was an error in the code. Let me correct it for you:\n",
      "\n",
      "```python\n",
      "# Given system of equations\n",
      "eq1 = Eq(a*x + b*y + c, x+7)\n",
      "eq2 = Eq(a + b*x + c*y, 2*x+6*y)\n",
      "eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
      "\n",
      "# Solve the system of equations\n",
      "sol = solve((eq1, eq2, eq3), (a, b, c, x, y))\n",
      "\n",
      "# Check if the solution exists for x+y != -1\n",
      "if sol:\n",
      "    # Given values for x and y\n",
      "    x_val = 1\n",
      "    y_val = 2\n",
      "\n",
      "    # Substitute the values of x and y into the expressions for a, b, c\n",
      "    a_val = sol[0].subs({x: x_val, y: y_val})\n",
      "    b_val = sol[1].subs({x: x_val, y: y_val})\n",
      "    c_val = sol[2].subs({x: x_val, y: y_val})\n",
      "\n",
      "    # Calculate the sum a + b + c\n",
      "    sum_a_b_c = a_val + b_val + c_val\n",
      "\n",
      "    print(sum_a_b_c)\n",
      "else:\n",
      "    print(\"No solution exists for x + y != -1.\")\n",
      "```\n",
      "\n",
      "Please replace `1` and `2` with the specific values of `x` and `y` that satisfy `x + y != -1`. This updated code will check if a solution exists and, if so, substitute the values of `x` and `y`, calculate the sum `a + b + c`, and output the result.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 1 (execution failed)\n",
      "Code output: \n",
      "Traceback (most recent call last):\n",
      "  File \"/workspaces/FLAML/flaml/autogen/extensions/tmp_code_b69073202e5e41922bdc8f31c6eaca33.py\", line 2, in <module>\n",
      "    eq1 = Eq(a*x + b*y + c, x+7)\n",
      "NameError: name 'Eq' is not defined\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "My apologies for the error. It seems that the `Eq` function from the `sympy` library was not imported. Let's update the code to include the necessary import:\n",
      "\n",
      "```python\n",
      "from sympy import symbols, Eq, solve\n",
      "\n",
      "a, b, c, x, y = symbols('a b c x y')\n",
      "eq1 = Eq(a*x + b*y + c, x+7)\n",
      "eq2 = Eq(a + b*x + c*y, 2*x+6*y)\n",
      "eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
      "\n",
      "# Solve the system of equations\n",
      "sol = solve((eq1, eq2, eq3), (a, b, c, x, y))\n",
      "\n",
      "# Check if the solution exists for x+y != -1\n",
      "if sol:\n",
      "    # Given values for x and y\n",
      "    x_val = 1\n",
      "    y_val = 2\n",
      "\n",
      "    # Substitute the values of x and y into the expressions for a, b, c\n",
      "    a_val = sol[0].subs({x: x_val, y: y_val})\n",
      "    b_val = sol[1].subs({x: x_val, y: y_val})\n",
      "    c_val = sol[2].subs({x: x_val, y: y_val})\n",
      "\n",
      "    # Calculate the sum a + b + c\n",
      "    sum_a_b_c = a_val + b_val + c_val\n",
      "\n",
      "    print(sum_a_b_c)\n",
      "else:\n",
      "    print(\"No solution exists for x + y != -1.\")\n",
      "```\n",
      "\n",
      "Please replace `1` and `2` with the specific values of `x` and `y` that satisfy `x + y != -1`. This updated code will check if a solution exists, substitute the values of `x` and `y`, calculate the sum `a + b + c`, and output the result.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 1 (execution failed)\n",
      "Code output: \n",
      "Traceback (most recent call last):\n",
      "  File \"/workspaces/FLAML/flaml/autogen/extensions/tmp_code_b052e62fbfd6fffc723b5965949cfa2c.py\", line 18, in <module>\n",
      "    a_val = sol[0].subs({x: x_val, y: y_val})\n",
      "AttributeError: 'tuple' object has no attribute 'subs'\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "Apologies for the typo in the code. The variable `sol` is a tuple containing the values of `a`, `b`, and `c` as its elements. To correctly substitute the values of `x` and `y` into the expressions, we need to access the expressions for `a`, `b`, and `c` stored in the tuple. Let's update the code accordingly:\n",
      "\n",
      "```python\n",
      "from sympy import symbols, Eq, solve\n",
      "\n",
      "a, b, c, x, y = symbols('a b c x y')\n",
      "eq1 = Eq(a*x + b*y + c, x+7)\n",
      "eq2 = Eq(a + b*x + c*y, 2*x+6*y)\n",
      "eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
      "\n",
      "# Solve the system of equations\n",
      "sol = solve((eq1, eq2, eq3), (a, b, c, x, y))\n",
      "\n",
      "# Check if the solution exists for x+y != -1\n",
      "if sol:\n",
      "    # Given values for x and y\n",
      "    x_val = 1\n",
      "    y_val = 2\n",
      "\n",
      "    # Substitute the values of x and y into the expressions for a, b, c\n",
      "    a_val = sol[a].subs({x: x_val, y: y_val})\n",
      "    b_val = sol[b].subs({x: x_val, y: y_val})\n",
      "    c_val = sol[c].subs({x: x_val, y: y_val})\n",
      "\n",
      "    # Calculate the sum a + b + c\n",
      "    sum_a_b_c = a_val + b_val + c_val\n",
      "\n",
      "    print(sum_a_b_c)\n",
      "else:\n",
      "    print(\"No solution exists for x + y != -1.\")\n",
      "```\n",
      "\n",
      "Please replace `1` and `2` with the specific values of `x` and `y` that satisfy `x + y != -1`. This updated code will correctly substitute the values of `x` and `y`, calculate the sum `a + b + c`, and output the result.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 1 (execution failed)\n",
      "Code output: \n",
      "Traceback (most recent call last):\n",
      "  File \"/workspaces/FLAML/flaml/autogen/extensions/tmp_code_8a54c64accea719c88a7ec6969de55a9.py\", line 18, in <module>\n",
      "    a_val = sol[a].subs({x: x_val, y: y_val})\n",
      "TypeError: list indices must be integers or slices, not Symbol\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "I apologize for the confusion. It seems that I made another mistake in the previous code. Let's use a different approach for solving the system of equations and finding the sum `a + b + c`:\n",
      "\n",
      "```python\n",
      "from sympy import symbols, Eq, solve\n",
      "\n",
      "a, b, c, x, y = symbols('a b c x y')\n",
      "eq1 = Eq(a*x + b*y + c, x + 7)\n",
      "eq2 = Eq(a + b*x + c*y, 2 * x + 6 * y)\n",
      "eq3 = Eq(a*y + b + c*x, 4 * x + y)\n",
      "constraints = Eq(x + y, -1)\n",
      "\n",
      "solutions = solve((eq1, eq2, eq3, constraints), (a, b, c, x, y))\n",
      "\n",
      "# Calculate the sum a + b + c\n",
      "if solutions:\n",
      "    a_val = solutions[a]\n",
      "    b_val = solutions[b]\n",
      "    c_val = solutions[c]\n",
      "    sum_a_b_c = a_val + b_val + c_val\n",
      "    print(sum_a_b_c)\n",
      "else:\n",
      "    print(\"No solution exists for x + y != -1.\")\n",
      "```\n",
      "\n",
      "This code will solve the system of equations with the constraint `x + y != -1` and calculate the sum `a + b + c`.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 1 (execution failed)\n",
      "Code output: \n",
      "Traceback (most recent call last):\n",
      "  File \"/workspaces/FLAML/flaml/autogen/extensions/tmp_code_4033fe30cb02a3888743d43adba718e5.py\", line 9, in <module>\n",
      "    solutions = solve((eq1, eq2, eq3, constraints), (a, b, c, x, y))\n",
      "  File \"/home/vscode/.local/lib/python3.9/site-packages/sympy/solvers/solvers.py\", line 1114, in solve\n",
      "    solution = _solve_system(f, symbols, **flags)\n",
      "  File \"/home/vscode/.local/lib/python3.9/site-packages/sympy/solvers/solvers.py\", line 1886, in _solve_system\n",
      "    raise NotImplementedError('no valid subset found')\n",
      "NotImplementedError: no valid subset found\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "[flaml.autogen.oai.completion: 07-10 05:51:21] {211} INFO - retrying in 10 seconds...\n",
      "Traceback (most recent call last):\n",
      "  File \"/workspaces/FLAML/flaml/autogen/oai/completion.py\", line 203, in _get_response\n",
      "    response = openai_completion.create(**config)\n",
      "  File \"/home/vscode/.local/lib/python3.9/site-packages/openai/api_resources/chat_completion.py\", line 25, in create\n",
      "    return super().create(*args, **kwargs)\n",
      "  File \"/home/vscode/.local/lib/python3.9/site-packages/openai/api_resources/abstract/engine_api_resource.py\", line 153, in create\n",
      "    response, _, api_key = requestor.request(\n",
      "  File \"/home/vscode/.local/lib/python3.9/site-packages/openai/api_requestor.py\", line 298, in request\n",
      "    resp, got_stream = self._interpret_response(result, stream)\n",
      "  File \"/home/vscode/.local/lib/python3.9/site-packages/openai/api_requestor.py\", line 700, in _interpret_response\n",
      "    self._interpret_response_line(\n",
      "  File \"/home/vscode/.local/lib/python3.9/site-packages/openai/api_requestor.py\", line 743, in _interpret_response_line\n",
      "    raise error.ServiceUnavailableError(\n",
      "openai.error.ServiceUnavailableError: The server is overloaded or not ready yet.\n",
      "assistant (to user):\n",
      "\n",
      "I apologize for the confusion caused by my previous attempt. It turns out that the approach was incorrect. Let's try to find the sum `a + b + c` by solving the given system of linear equations without constraints:\n",
      "\n",
      "```python\n",
      "from sympy import symbols, Eq, solve\n",
      "\n",
      "a, b, c, x, y = symbols('a b c x y')\n",
      "eq1 = Eq(a*x + b*y + c, x + 7)\n",
      "eq2 = Eq(a + b*x + c*y, 2 * x + 6 * y)\n",
      "eq3 = Eq(a*y + b + c*x, 4 * x + y)\n",
      "\n",
      "solutions = solve((eq1, eq2, eq3), (a, b, c), dict=True)\n",
      "\n",
      "# Calculate the sum a + b + c\n",
      "sums_a_b_c = []\n",
      "for sol in solutions:\n",
      "    a_val = sol[a]\n",
      "    b_val = sol[b]\n",
      "    c_val = sol[c]\n",
      "    sum_a_b_c = a_val + b_val + c_val\n",
      "    if (a != -1):\n",
      "        sums_a_b_c.append(sum_a_b_c)\n",
      "\n",
      "print(sums_a_b_c)\n",
      "```\n",
      "\n",
      "This code will solve the system of equations and calculate the sum `a + b + c` for the given values of `x` and `y` without the constraint `x + y != -1`. Please let me know if you still need to enforce the constraint.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 0 (execution succeeded)\n",
      "Code output: \n",
      "[(2*x**2 - 3*x + y)/(x**2 - x*y - x + y**2 - y + 1) + (x**2 - 3*x*y + 2*x + y**2 - y)/(x**2 - x*y - x + y**2 - y + 1) + (4*x**2 - 4*x*y - 6*x + 6*y**2 - 7*y + 7)/(x**2 - x*y - x + y**2 - y + 1)]\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "The solution provided in the output is not a unique value, but rather an expression that depends on the values of `x` and `y`. The expression for `a + b + c` in terms of `x` and `y` is:\n",
      "\n",
      "```((2*x**2 - 3*x + y) + (x**2 - 3*x*y + 2*x + y**2 - y) + (4*x**2 - 4*x*y - 6*x + 6*y**2 - 7*y + 7)) / (x**2 - x*y - x + y**2 - y + 1)```\n",
      "\n",
      "If you have any specific values of `x` and `y`, you can simply plug them into this expression to find the corresponding value of `a + b + c`.\n",
      "\n",
      "However, note that the problem states that the relation `x + y != -1` must be satisfied. In this scenario, you would need to find an appropriate pair of values for `x` and `y` that fit this condition before substituting them into the expression to obtain the sum `a + b + c`.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "user (to assistant):\n",
      "\n",
      "simplify the answer\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "Let's simplify the expression for $a + b + c$ using Sympy library:\n",
      "\n",
      "```python\n",
      "from sympy import simplify\n",
      "\n",
      "expr = (2 * x ** 2 - 3 * x + y) + (x ** 2 - 3 * x * y + 2 * x + y ** 2 - y) + (4 * x ** 2 - 4 * x * y - 6 * x + 6 * y ** 2 - 7 * y + 7)\n",
      "denominator = x ** 2 - x * y - x + y ** 2 - y + 1\n",
      "simplified_expr = simplify(expr / denominator)\n",
      "\n",
      "print(simplified_expr)\n",
      "```\n",
      "\n",
      "This code will simplify the expression for $a + b + c$ and output the result.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 1 (execution failed)\n",
      "Code output: \n",
      "Traceback (most recent call last):\n",
      "  File \"/workspaces/FLAML/flaml/autogen/extensions/tmp_code_8789f3b4a30e6ef1b1f1a8750b93be85.py\", line 3, in <module>\n",
      "    expr = (2 * x ** 2 - 3 * x + y) + (x ** 2 - 3 * x * y + 2 * x + y ** 2 - y) + (4 * x ** 2 - 4 * x * y - 6 * x + 6 * y ** 2 - 7 * y + 7)\n",
      "NameError: name 'x' is not defined\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "I apologize, the code was missing the definitions of variables `x` and `y`. Here's the updated code with the missing definitions:\n",
      "\n",
      "```python\n",
      "from sympy import symbols, simplify\n",
      "\n",
      "x, y = symbols('x y')\n",
      "expr = (2 * x ** 2 - 3 * x + y) + (x ** 2 - 3 * x * y + 2 * x + y ** 2 - y) + (4 * x ** 2 - 4 * x * y - 6 * x + 6 * y ** 2 - 7 * y + 7)\n",
      "denominator = x ** 2 - x * y - x + y ** 2 - y + 1\n",
      "simplified_expr = simplify(expr / denominator)\n",
      "\n",
      "print(simplified_expr)\n",
      "```\n",
      "\n",
      "This code will simplify the expression for `a + b + c` and output the result.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "exitcode: 0 (execution succeeded)\n",
      "Code output: \n",
      "7\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "Now we have the simplified result. The sum of `a + b + c` is equal to 7.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
      "user (to assistant):\n",
      "\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "assistant (to user):\n",
      "\n",
      "Do you have any other question or task to solve? If you do, feel free to ask!\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "math_problem_to_solve = \"\"\"\n",
    "     Find $a + b + c$, given that $x+y \\\\neq -1$ and \n",
    "     \\\\begin{align}\n",
    "\tax + by + c & = x + 7,\\\\\n",
    "\ta + bx + cy & = 2x + 6y,\\\\\n",
    "\tay + b + cx & = 4x + y.\n",
    "\t\\\\end{align}.\n",
    "    \"\"\"\n",
    "\n",
    "# the assistant receives a message from the user, which contains the task description\n",
    "assistant.receive(math_problem_to_solve, user)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analyze the conversation\n",
    "\n",
    "The human user can provide feedback at each step. When the human user didn't provide feedback, the code was executed. The executed results and error messages are returned to the assistant, and the assistant was able to modify the code to rectify. In the end, the task is complete and a \"TERMINATE\" signal is sent from the assistant. The user skipped feedback in the end and the conversation is finished.\n",
    "\n",
    "After the conversation is finished, we can save the log of the conversation between the two agents. The log can be accessed from `oai.ChatCompletion.logged_history`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'[{\"content\": \"You are a helpful AI assistant.\\\\n    In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute. You must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can\\'t modify your code.\\\\n    1. When you need to ask the user for some info, use the code to output the info you need, for example, browse or search the web, download/read a file.\\\\n    2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly. Solve the task step by step if you need to.\\\\n    If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don\\'t include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use \\'print\\' function for the output when relevant. Check the execution result returned by the user.\\\\n    If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes.\\\\n    Verify your answer carefully. If a function for planning is provided, call the function to make plans and verify the execution.\\\\n    Reply \\\\\"TERMINATE\\\\\" in the end when everything is done.\\\\n    \", \"role\": \"system\"}, {\"content\": \"\\\\n     Find $a + b + c$, given that $x+y \\\\\\\\neq -1$ and \\\\n     \\\\\\\\begin{align}\\\\n\\\\tax + by + c & = x + 7,\\\\\\\\\\\\n\\\\ta + bx + cy & = 2x + 6y,\\\\\\\\\\\\n\\\\tay + b + cx & = 4x + y.\\\\n\\\\t\\\\\\\\end{align}.\\\\n    \", \"role\": \"user\"}, {\"content\": \"Let\\'s solve the system of equations step by step. First, we can solve the third equation for $a$, we get:\\\\n\\\\n$$a = \\\\\\\\frac{4x + y - b - cx}{y}$$\\\\n\\\\nSubstituting this value of $a$ into the second equation gives:\\\\n\\\\n$$\\\\\\\\frac{4x + y - b - cx}{y} + bx + cy = 2x + 6y.$$\\\\n\\\\nExpanding and rearranging terms, we get:\\\\n\\\\n$$(4 - c)x + \\\\\\\\left(\\\\\\\\frac{1}{y} - b\\\\\\\\right)y - \\\\\\\\frac{b}{y} = 2x + 6y.$$\\\\n\\\\nSimplifying further, we have:\\\\n\\\\n$$(4 - c)x - b = 2x + 6y.$$\\\\n\\\\nRearranging terms gives:\\\\n\\\\n$$(2 - c)x - b - 6y = 0. \\\\\\\\qquad\\\\\\\\qquad (4)$$\\\\n\\\\nNow, we can solve the first equation for $c$, we get:\\\\n\\\\n$$c = x + 7 - ax - by.$$\\\\n\\\\nSubstituting this value of $c$ into the fourth equation gives:\\\\n\\\\n$$(2 - (x + 7 - ax - by))x - b - 6y = 0.$$\\\\n\\\\nExpanding and rearranging terms, we obtain:\\\\n\\\\n$$2x - 6y - x - 7 + ax + by - b - 6y = 0.$$\\\\n\\\\nSimplifying further, we have:\\\\n\\\\n$$2x - x - 7 + ax - 6y + by - b - 6y = 0.$$\\\\n\\\\nRearranging terms gives:\\\\n\\\\n$$ax + by + (2 - 2x - 12y) - 7 - b = 0.$$\\\\n\\\\nSimplifying, we have:\\\\n\\\\n$$ax + by - 2x - 12y - b - 7 = 0. \\\\\\\\qquad\\\\\\\\qquad (5)$$\\\\n\\\\nNow, we can solve equations (4) and (5) simultaneously to find the values of $a$, $b$, $x$, and $y$. From equation (4), we have:\\\\n\\\\n$$(2 - c)x - b - 6y = 0.$$\\\\n\\\\nSubstituting the value of $c$ from equation (5), we get:\\\\n\\\\n$$(2 - (ax + by - 2x - 12y - b - 7))x - b - 6y = 0.$$\\\\n\\\\nExpanding and rearranging terms, we obtain:\\\\n\\\\n$$2x - (2ax + 2by - 4x - 24y - 2b - 14)x - b - 6y = 0.$$\\\\n\\\\nSimplifying further, we have:\\\\n\\\\n$$2x - 2ax - 2by + 4x + 24y + 2b + 14x - b - 6y = 0.$$\\\\n\\\\nRearranging terms gives:\\\\n\\\\n$$(2 + 4 + 14 - 2a)x + (-2b - 6 + 24 - b)y + (2 - 6 + 14) = 0.$$\\\\n\\\\nSimplifying, we have:\\\\n\\\\n$$(16 - 2a)x + (-3b + 18)y + 10 = 0. \\\\\\\\qquad\\\\\\\\qquad (6)$$\\\\n\\\\nNow, we can solve equations (5) and (6) simultaneously to find the values of $a$, $b$, $x$, and $y$. From equation (5), we have:\\\\n\\\\n$$ax + by - 2x - 12y - b - 7 = 0.$$\\\\n\\\\nSubstituting the value of $ax + by$ from equation (6), we get:\\\\n\\\\n$$(16 - 2a)x + (-3b + 18)y + 10 - 2x - 12y - 
     ]
    }
   ],
   "source": [
    "print(oai.ChatCompletion.logged_history)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "json.dump(oai.ChatCompletion.logged_history, open(\"conversations.json\", \"w\"), indent=2)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.15"
  },
  "vscode": {
   "interpreter": {
    "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1"
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "2d910cfd2d2a4fc49fc30fbbdc5576a7": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "454146d0f7224f038689031002906e6f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_e4ae2b6f5a974fd4bafb6abb9d12ff26",
        "IPY_MODEL_577e1e3cc4db4942b0883577b3b52755",
        "IPY_MODEL_b40bdfb1ac1d4cffb7cefcb870c64d45"
       ],
       "layout": "IPY_MODEL_dc83c7bff2f241309537a8119dfc7555",
       "tabbable": null,
       "tooltip": null
      }
     },
     "577e1e3cc4db4942b0883577b3b52755": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_2d910cfd2d2a4fc49fc30fbbdc5576a7",
       "max": 1,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_74a6ba0c3cbc4051be0a83e152fe1e62",
       "tabbable": null,
       "tooltip": null,
       "value": 1
      }
     },
     "6086462a12d54bafa59d3c4566f06cb2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "74a6ba0c3cbc4051be0a83e152fe1e62": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "7d3f3d9e15894d05a4d188ff4f466554": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "background": null,
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "b40bdfb1ac1d4cffb7cefcb870c64d45": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_f1355871cc6f4dd4b50d9df5af20e5c8",
       "placeholder": "​",
       "style": "IPY_MODEL_ca245376fd9f4354af6b2befe4af4466",
       "tabbable": null,
       "tooltip": null,
       "value": " 1/1 [00:00&lt;00:00, 44.69it/s]"
      }
     },
     "ca245376fd9f4354af6b2befe4af4466": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "background": null,
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "dc83c7bff2f241309537a8119dfc7555": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "e4ae2b6f5a974fd4bafb6abb9d12ff26": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_6086462a12d54bafa59d3c4566f06cb2",
       "placeholder": "​",
       "style": "IPY_MODEL_7d3f3d9e15894d05a4d188ff4f466554",
       "tabbable": null,
       "tooltip": null,
       "value": "100%"
      }
     },
     "f1355871cc6f4dd4b50d9df5af20e5c8": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}