autogen/notebook/autogen_agentchat_human_feedback.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/autogen_agentchat_human_feedback.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Auto Generated Agent Chat: Task Solving with Code Generation, Execution, Debugging & Human Feedback\n",
    "\n",
    "`flaml.autogen` offers conversable agents powered by LLM, tool or human, which can be used to perform tasks collectively via automated chat. This framwork allows tool use and human participance through multi-agent conversation.\n",
    "Please find documentation about this feature [here](https://microsoft.github.io/FLAML/docs/Use-Cases/Autogen#agents).\n",
    "\n",
    "In this notebook, we demonstrate how to use `AssistantAgent` and `UserProxyAgent` to solve a challenging math problem with human feedback. Here `AssistantAgent` is an LLM-based agent that can write Python code (in a Python coding block) for a user to execute for a given task. `UserProxyAgent` is an agent which serves as a proxy for a user to execute the code written by `AssistantAgent`. By setting `human_input_mode` properly, the `UserProxyAgent` can also prompt the user for feedback to `AssistantAgent`. For example, when `human_input_mode` is set to \"ALWAYS\", the `UserProxyAgent` will always prompt the user for feedback. When user feedback is provided, the `UserProxyAgent` will directly pass the feedback to `AssistantAgent`. When no user feedback is provided, the `UserProxyAgent` will execute the code written by `AssistantAgent` and return the execution results (success or failure and corresponding outputs) to `AssistantAgent`.\n",
    "\n",
    "## Requirements\n",
    "\n",
    "FLAML requires `Python>=3.8`. To run this notebook example, please install flaml with the [autogen] option:\n",
    "```bash\n",
    "pip install flaml[autogen]\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-02-13T23:40:52.317406Z",
     "iopub.status.busy": "2023-02-13T23:40:52.316561Z",
     "iopub.status.idle": "2023-02-13T23:40:52.321193Z",
     "shell.execute_reply": "2023-02-13T23:40:52.320628Z"
    }
   },
   "outputs": [],
   "source": [
    "# %pip install flaml[autogen]~=2.0.1"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set your API Endpoint\n",
    "\n",
    "The [`config_list_from_json`](https://microsoft.github.io/FLAML/docs/reference/autogen/oai/openai_utils#config_list_from_json) function loads a list of configurations from an environment variable or a json file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from flaml import autogen\n",
    "\n",
    "config_list = autogen.config_list_from_json(\"OAI_CONFIG_LIST\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It first looks for environment variable \"OAI_CONFIG_LIST\" which needs to be a valid json string. If that variable is not found, it then looks for a json file named \"OAI_CONFIG_LIST\".\n",
    "\n",
    "The config list looks like the following:\n",
    "```python\n",
    "config_list = [\n",
    "    {\n",
    "        'model': 'gpt-4',\n",
    "        'api_key': '<your OpenAI API key here>',\n",
    "    },  # OpenAI API endpoint for gpt-4\n",
    "    {\n",
    "        'model': 'gpt-4',\n",
    "        'api_key': '<your first Azure OpenAI API key here>',\n",
    "        'api_base': '<your first Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-06-01-preview',\n",
    "    },  # Azure OpenAI API endpoint for gpt-4\n",
    "    {\n",
    "        'model': 'gpt-4',\n",
    "        'api_key': '<your second Azure OpenAI API key here>',\n",
    "        'api_base': '<your second Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-06-01-preview',\n",
    "    },  # another Azure OpenAI API endpoint for gpt-4\n",
    "    {\n",
    "        'model': 'gpt-3.5-turbo',\n",
    "        'api_key': '<your OpenAI API key here>',\n",
    "    },  # OpenAI API endpoint for gpt-3.5-turbo\n",
    "    {\n",
    "        'model': 'gpt-3.5-turbo',\n",
    "        'api_key': '<your first Azure OpenAI API key here>',\n",
    "        'api_base': '<your first Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-06-01-preview',\n",
    "    },  # Azure OpenAI API endpoint for gpt-3.5-turbo\n",
    "    {\n",
    "        'model': 'gpt-3.5-turbo',\n",
    "        'api_key': '<your second Azure OpenAI API key here>',\n",
    "        'api_base': '<your second Azure OpenAI API base here>',\n",
    "        'api_type': 'azure',\n",
    "        'api_version': '2023-06-01-preview',\n",
    "    },  # another Azure OpenAI API endpoint for gpt-3.5-turbo\n",
    "]\n",
    "```\n",
    "\n",
    "If you open this notebook in colab, you can upload your files by clicking the file icon on the left panel and then choose \"upload file\" icon.\n",
    "\n",
    "You can set the value of config_list in other ways you prefer, e.g., loading from a YAML file."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Construct Agents\n",
    "\n",
    "We construct the assistant agent and the user proxy agent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create an AssistantAgent instance named \"assistant\"\n",
    "assistant = autogen.AssistantAgent(\n",
    "    name=\"assistant\",\n",
    "    llm_config={\n",
    "        \"seed\": 41,\n",
    "        \"config_list\": config_list,\n",
    "    }\n",
    ")\n",
    "# create a UserProxyAgent instance named \"user_proxy\"\n",
    "user_proxy = autogen.UserProxyAgent(\n",
    "    name=\"user_proxy\",\n",
    "    human_input_mode=\"ALWAYS\",\n",
    "    is_termination_msg=lambda x: x.get(\"content\", \"\").rstrip().endswith(\"TERMINATE\"),\n",
    ")\n",
    "\n",
    "# the purpose of the following line is to log the conversation history\n",
    "autogen.ChatCompletion.start_logging()\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Perform a task\n",
    "\n",
    "We invoke the `initiate_chat()` method of the user proxy agent to start the conversation. When you run the cell below, you will be prompted to provide feedback after receving a message from the assistant agent. If you don't provide any feedback (by pressing Enter directly), the user proxy agent will try to execute the code suggested by the assistant agent on behalf of you, or terminate if the assistant agent sends a \"TERMINATE\" signal in the end of the message."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33muser_proxy\u001b[0m (to assistant):\n",
      "\n",
      "\n",
      "Find $a + b + c$, given that $x+y \\neq -1$ and \n",
      "\\begin{align}\n",
      "\tax + by + c & = x + 7,\\\n",
      "\ta + bx + cy & = 2x + 6y,\\\n",
      "\tay + b + cx & = 4x + y.\n",
      "\\end{align}.\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33massistant\u001b[0m (to user_proxy):\n",
      "\n",
      "To solve this problem, we can follow these steps:\n",
      "\n",
      "1. Identify the equations: We have three equations:\n",
      "    1) ax + by + c = x + 7,\n",
      "    2) a + bx + cy = 2x + 6y,\n",
      "    3) ay + b + cx = 4x + y.\n",
      "2. Rearrange the equations to make them easy to solve:\n",
      "    1) ax + by + c - x = 7,\n",
      "    2) a + cx - 2x + by - 6y = 0,\n",
      "    3) ay - y + b + cx - 4x = 0.\n",
      "3. Since we need to find the sum a + b + c, substitute the expressions for a, b and c from the rearranged equations into the expression for the sum a + b + c.\n",
      "4. Simplify the equation to get a + b + c.\n",
      "\n",
      "The above steps involve both coding and language skills, the coding is used to solve the equations and obtain the values for a, b and c (which involves algebraic computation and symbolic computation). The language skill is used for explanation and clarification of the problem and the steps involved.\n",
      "\n",
      "We'll use Python's SymPy library to help us solve these system of equations. SymPy is a Python library for symbolic computation.\n",
      "\n",
      "Here's the Python code you can execute. We'll use SymPy to solve a system of equations and sympy.solve to find the solution.\n",
      "\n",
      "Please, execute the following Python code:\n",
      "\n",
      "```python\n",
      "# SymPy is a Python library for symbolic mathematics\n",
      "from sympy import symbols, Eq, solve\n",
      "\n",
      "# defining the symbols\n",
      "x, y, a, b, c = symbols('x y a b c')\n",
      "\n",
      "# defining the equations\n",
      "eq1 = Eq(a*x + b*y + c, x + 7)\n",
      "eq2 = Eq(a + b*x + c*y, 2*x + 6*y)\n",
      "eq3 = Eq(a*y + b + c*x , 4*x + y)\n",
      "\n",
      "# solve the system of equations\n",
      "sol = solve((eq1,eq2,eq3), (a, b, c))\n",
      "\n",
      "# Output the value of a + b + c\n",
      "print(sol[a] + sol[b] + sol[c])\n",
      "```\n",
      "\n",
      "This Python code block will solve the system of equations and print the value of a + b + c.\n",
      "\n",
      "We need to assure that x + y ≠ -1, because if x + y = -1, we will be dividing by zero when solving the equations. Hence, our solution is valid under the condition x + y ≠ -1.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED.\u001b[0m\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[31m\n",
      ">>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...\u001b[0m\n",
      "\u001b[33muser_proxy\u001b[0m (to assistant):\n",
      "\n",
      "exitcode: 0 (execution succeeded)\n",
      "Code output: \n",
      "(2*x**2 - 3*x + y)/(x**2 - x*y - x + y**2 - y + 1) + (x**2 - 3*x*y + 2*x + y**2 - y)/(x**2 - x*y - x + y**2 - y + 1) + (4*x**2 - 4*x*y - 6*x + 6*y**2 - 7*y + 7)/(x**2 - x*y - x + y**2 - y + 1)\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33massistant\u001b[0m (to user_proxy):\n",
      "\n",
      "The output is the solution for a + b + c as a function of x and y.\n",
      "\n",
      "However, since we have this constraint that x + y ≠ -1, we must state this clearly. Keep in mind that if x + y equals -1, the denominator of the solution is equal to zero, which makes the solution undefined.\n",
      "\n",
      "So, the solution is:\n",
      "\n",
      "a + b + c = (2x² - 3x + y) / (x² - xy - x + y² - y + 1) + (x² - 3xy + 2x + y² - y) / (x² - xy - x + y² - y + 1) + (4x² - 4xy - 6x + 6y² - 7y + 7) / (x² - xy - x + y² - y + 1)\n",
      "\n",
      "Under the condition that x + y ≠ -1.\n",
      "\n",
      "If you want to find the specific value of a + b + c for certain values of x and y (which do not sum to -1), you can substitute those values in place of x and y in the above equation.\n",
      "\n",
      "Now that we've successfully found a+b+c, we can conclude this task. \n",
      "\n",
      "TERMINATE\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33muser_proxy\u001b[0m (to assistant):\n",
      "\n",
      "simplify the answer\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33massistant\u001b[0m (to user_proxy):\n",
      "\n",
      "Sure, I'll simplify the final expression for a + b + c.\n",
      "\n",
      "The Python's sympy library has a function called \"simplify\" that can be used to simplify expressions. This function applies various simplification algorithms to the given expression and returns a simpler form of the expression. \n",
      "\n",
      "Please, execute the following Python code:\n",
      "\n",
      "```python\n",
      "from sympy import simplify\n",
      "\n",
      "print(simplify(sol[a] + sol[b] + sol[c]))\n",
      "```\n",
      "\n",
      "The block of code will print the simplified value for a + b + c. The answer will be a function of x and y. Remember that x, y ≠ -1.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED.\u001b[0m\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[31m\n",
      ">>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...\u001b[0m\n",
      "\u001b[33muser_proxy\u001b[0m (to assistant):\n",
      "\n",
      "exitcode: 1 (execution failed)\n",
      "Code output: \n",
      "Traceback (most recent call last):\n",
      "  File \"\", line 3, in <module>\n",
      "    print(simplify(sol[a] + sol[b] + sol[c]))\n",
      "NameError: name 'sol' is not defined\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33massistant\u001b[0m (to user_proxy):\n",
      "\n",
      "I apologize for the inconvenience. The error occurred because the 'sol' variable, which stored the solution for the system of equations, was not defined in this new Python execution. Since each execution is like a new Python session, previous variables and definitions are not retained.\n",
      "\n",
      "Let's correct this by combining the entire solution - from defining the equations to simplifying the sum a+b+c - into one block of python code.\n",
      "\n",
      "Please, execute the following Python code:\n",
      "\n",
      "```python\n",
      "from sympy import symbols, Eq, solve, simplify\n",
      "\n",
      "# defining the symbols\n",
      "x, y, a, b, c = symbols('x y a b c')\n",
      "\n",
      "# defining the equations\n",
      "eq1 = Eq(a*x + b*y + c, x + 7)\n",
      "eq2 = Eq(a + b*x + c*y, 2*x + 6*y)\n",
      "eq3 = Eq(a*y + b + c*x , 4*x + y)\n",
      "\n",
      "# solve the system of equations and simplify the sum a + b + c\n",
      "sol = solve((eq1,eq2,eq3), (a, b, c))\n",
      "print(simplify(sol[a] + sol[b] + sol[c]))\n",
      "```\n",
      "\n",
      "This Python code block will solve the system of equations, find the sum a + b + c, simplify it, and print the simplified result. Remember to consider the condition that x + y ≠ -1.\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED.\u001b[0m\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[31m\n",
      ">>>>>>>> EXECUTING CODE BLOCK 0 (inferred language is python)...\u001b[0m\n",
      "\u001b[33muser_proxy\u001b[0m (to assistant):\n",
      "\n",
      "exitcode: 0 (execution succeeded)\n",
      "Code output: \n",
      "7\n",
      "\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33massistant\u001b[0m (to user_proxy):\n",
      "\n",
      "Great! The solution for a + b + c is 7, given that x + y ≠ -1. \n",
      "\n",
      "Now, the solution is quite simplified. We have a definite numeric value for a + b + c regardless of what x and y are, as long as x + y ≠ -1.\n",
      "\n",
      "We've successfully found and simplified a + b + c. We can conclude this task.\n",
      "\n",
      "TERMINATE\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> NO HUMAN INPUT RECEIVED.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "math_problem_to_solve = \"\"\"\n",
    "Find $a + b + c$, given that $x+y \\\\neq -1$ and \n",
    "\\\\begin{align}\n",
    "\tax + by + c & = x + 7,\\\\\n",
    "\ta + bx + cy & = 2x + 6y,\\\\\n",
    "\tay + b + cx & = 4x + y.\n",
    "\\\\end{align}.\n",
    "\"\"\"\n",
    "\n",
    "# the assistant receives a message from the user, which contains the task description\n",
    "user_proxy.initiate_chat(assistant, message=math_problem_to_solve)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analyze the conversation\n",
    "\n",
    "The human user can provide feedback at each step. When the human user didn't provide feedback, the code was executed. The executed results and error messages are returned to the assistant, and the assistant was able to modify the code based on the feedback. In the end, the task is complete and a \"TERMINATE\" signal is sent from the assistant. The user skipped feedback in the end and the conversation is finished.\n",
    "\n",
    "After the conversation is finished, we can save the log of the conversation between the two agents. The log can be accessed from `autogen.ChatCompletion.logged_history`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'[{\"content\": \"You are a helpful AI assistant.\\\\n    Solve tasks using your coding and language skills.\\\\n    If a plan is not provided, explain the plan first. Be clear which step uses code, and which step uses your language skill.\\\\n    In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute.\\\\n    1. When you need to collect info, use the code to output the info you need, for example, browse or search the web, download/read a file, print the content of a webpage or a file, get the current date/time. After sufficient info is printed and the task is ready to be solved based on your language skill, you can solve the task by yourself.\\\\n    2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly. Solve the task step by step if you need to.\\\\n    You must indicate the script type in the code block. The user cannot provide any other feedback or perform any other action beyond executing the code you suggest. The user can\\'t modify your code. So do not suggest incomplete code which requires users to modify. Don\\'t use a code block if it\\'s not intended to be executed by the user.\\\\n    If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don\\'t include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use \\'print\\' function for the output when relevant. Check the execution result returned by the user.\\\\n    If the result indicates there is an error, fix the error and output the code again. Suggest the full code instead of partial code or code changes. If the error can\\'t be fixed or if the task is not solved even after the code is executed successfully, analyze the problem, revisit your assumption, collect additional info you need, and think of a different approach to try.\\\\n    When you find an answer, verify the answer carefully. If a function for planning is provided, call the function to make plans and verify the execution.\\\\n    Reply \\\\\"TERMINATE\\\\\" in the end when everything is done.\\\\n    \", \"role\": \"system\"}, {\"content\": \"\\\\nFind $a + b + c$, given that $x+y \\\\\\\\neq -1$ and \\\\n\\\\\\\\begin{align}\\\\n\\\\tax + by + c & = x + 7,\\\\\\\\\\\\n\\\\ta + bx + cy & = 2x + 6y,\\\\\\\\\\\\n\\\\tay + b + cx & = 4x + y.\\\\n\\\\\\\\end{align}.\\\\n\", \"role\": \"user\"}, {\"content\": \"To solve this problem, we can follow these steps:\\\\n\\\\n1. Identify the equations: We have three equations:\\\\n    1) ax + by + c = x + 7,\\\\n    2) a + bx + cy = 2x + 6y,\\\\n    3) ay + b + cx = 4x + y.\\\\n2. Rearrange the equations to make them easy to solve:\\\\n    1) ax + by + c - x = 7,\\\\n    2) a + cx - 2x + by - 6y = 0,\\\\n    3) ay - y + b + cx - 4x = 0.\\\\n3. Since we need to find the sum a + b + c, substitute the expressions for a, b and c from the rearranged equations into the expression for the sum a + b + c.\\\\n4. Simplify the equation to get a + b + c.\\\\n\\\\nThe above steps involve both coding and language skills, the coding is used to solve the equations and obtain the values for a, b and c (which involves algebraic computation and symbolic computation). The language skill is used for explanation and clarification of the problem and the steps involved.\\\\n\\\\nWe\\'ll use Python\\'s SymPy library to help us solve these system of equations. SymPy is a Python library for symbolic computation.\\\\n\\\\nHere\\'s the Python code you can execute. We\\'ll use SymPy to solve a system of equations and sympy.solve to find the solution.\\\\n\\\\nPlease, execute the following Python code:\\\\n\\\\n```python\\\\n# SymPy is a Python library for symbolic mathematics\\\\nfrom sympy import symbols, Eq, solve\\\\n\\\\n# defining the symbols\\\\nx, y, a, b, c = symbols(\\'x y a b c\\')\\\\n\\\\n# defining the equations\\\\neq1 = Eq(a*x + b*y + c, x + 7)\\\\neq2 = Eq(a + b*x + c*y, 2*x + 6*y)\\\\
     ]
    }
   ],
   "source": [
    "print(autogen.ChatCompletion.logged_history)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "json.dump(autogen.ChatCompletion.logged_history, open(\"conversations.json\", \"w\"), indent=2)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.17"
  },
  "vscode": {
   "interpreter": {
    "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1"
   }
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "2d910cfd2d2a4fc49fc30fbbdc5576a7": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "454146d0f7224f038689031002906e6f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_e4ae2b6f5a974fd4bafb6abb9d12ff26",
        "IPY_MODEL_577e1e3cc4db4942b0883577b3b52755",
        "IPY_MODEL_b40bdfb1ac1d4cffb7cefcb870c64d45"
       ],
       "layout": "IPY_MODEL_dc83c7bff2f241309537a8119dfc7555",
       "tabbable": null,
       "tooltip": null
      }
     },
     "577e1e3cc4db4942b0883577b3b52755": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_2d910cfd2d2a4fc49fc30fbbdc5576a7",
       "max": 1,
       "min": 0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_74a6ba0c3cbc4051be0a83e152fe1e62",
       "tabbable": null,
       "tooltip": null,
       "value": 1
      }
     },
     "6086462a12d54bafa59d3c4566f06cb2": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "74a6ba0c3cbc4051be0a83e152fe1e62": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "7d3f3d9e15894d05a4d188ff4f466554": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "background": null,
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "b40bdfb1ac1d4cffb7cefcb870c64d45": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_f1355871cc6f4dd4b50d9df5af20e5c8",
       "placeholder": "​",
       "style": "IPY_MODEL_ca245376fd9f4354af6b2befe4af4466",
       "tabbable": null,
       "tooltip": null,
       "value": " 1/1 [00:00&lt;00:00, 44.69it/s]"
      }
     },
     "ca245376fd9f4354af6b2befe4af4466": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "background": null,
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "dc83c7bff2f241309537a8119dfc7555": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "e4ae2b6f5a974fd4bafb6abb9d12ff26": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_6086462a12d54bafa59d3c4566f06cb2",
       "placeholder": "​",
       "style": "IPY_MODEL_7d3f3d9e15894d05a4d188ff4f466554",
       "tabbable": null,
       "tooltip": null,
       "value": "100%"
      }
     },
     "f1355871cc6f4dd4b50d9df5af20e5c8": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}