"<a href=\"https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/autogen_agent_human_feedback.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Interactive LLM Agent with Human Feedback\n",
"\n",
"FLAML offers an experimental feature of interactive LLM agents, which can be used to solve various tasks with human or automatic feedback, including tasks that require using tools via code.\n",
"\n",
"In this notebook, we demonstrate how to use `AssistantAgent` and `UserProxyAgent` to solve a challenging math problem with human feedback. Here `AssistantAgent` is an LLM-based agent that can write Python code (in a Python coding block) for a user to execute for a given task. `UserProxyAgent` is an agent which serves as a proxy for a user to execute the code written by `AssistantAgent`. By setting `human_input_mode` properly, the `UserProxyAgent` can also prompt the user for feedback to `AssistantAgent`. For example, when `human_input_mode` is set to \"ALWAYS\", the `UserProxyAgent` will always prompt the user for feedback. When user feedback is provided, the `UserProxyAgent` will directly pass the feedback to `AssistantAgent` without doing any additional steps. When no user feedback is provided, the `UserProxyAgent` will execute the code written by `AssistantAgent` directly and return the execution results (success or failure and corresponding outputs) to `AssistantAgent`.\n",
"\n",
"## Requirements\n",
"\n",
"FLAML requires `Python>=3.7`. To run this notebook example, please install flaml with the [autogen] option:\n",
"The [`config_list_gpt4_gpt35`](https://microsoft.github.io/FLAML/docs/reference/autogen/oai/openai_utils#config_list_gpt4_gpt35) function tries to create a list of gpt-4 and gpt-3.5 configurations using Azure OpenAI endpoints and OpenAI endpoints. It assumes the api keys and api bases are stored in the corresponding environment variables or local txt files:\n",
"\n",
"- OpenAI API key: os.environ[\"OPENAI_API_KEY\"] or `openai_api_key_file=\"key_openai.txt\"`.\n",
"- Azure OpenAI API key: os.environ[\"AZURE_OPENAI_API_KEY\"] or `aoai_api_key_file=\"key_aoai.txt\"`. Multiple keys can be stored, one per line.\n",
"- Azure OpenAI API base: os.environ[\"AZURE_OPENAI_API_BASE\"] or `aoai_api_base_file=\"base_aoai.txt\"`. Multiple bases can be stored, one per line.\n",
"\n",
"It's OK to have only the OpenAI API key, or only the Azure OpenAI API key + base.\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"from flaml import oai\n",
"\n",
"config_list = oai.config_list_gpt4_gpt35()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The config list looks like the following:\n",
"```python\n",
"config_list = [\n",
" {\n",
" 'model': 'gpt-4',\n",
" 'api_key': '<your OpenAI API key here>',\n",
" }, # only if OpenAI API key is found\n",
" {\n",
" 'model': 'gpt-4',\n",
" 'api_key': '<your first Azure OpenAI API key here>',\n",
" 'api_base': '<your first Azure OpenAI API base here>',\n",
" 'api_type': 'azure',\n",
" 'api_version': '2023-03-15-preview',\n",
" }, # only if the at least one Azure OpenAI API key is found\n",
" {\n",
" 'model': 'gpt-4',\n",
" 'api_key': '<your second Azure OpenAI API key here>',\n",
" 'api_base': '<your second Azure OpenAI API base here>',\n",
" 'api_type': 'azure',\n",
" 'api_version': '2023-03-15-preview',\n",
" }, # only if the second Azure OpenAI API key is found\n",
" {\n",
" 'model': 'gpt-3.5-turbo',\n",
" 'api_key': '<your OpenAI API key here>',\n",
" }, # only if OpenAI API key is found\n",
" {\n",
" 'model': 'gpt-3.5-turbo',\n",
" 'api_key': '<your first Azure OpenAI API key here>',\n",
" 'api_base': '<your first Azure OpenAI API base here>',\n",
" 'api_type': 'azure',\n",
" 'api_version': '2023-03-15-preview',\n",
" }, # only if the at least one Azure OpenAI API key is found\n",
" {\n",
" 'model': 'gpt-3.5-turbo',\n",
" 'api_key': '<your second Azure OpenAI API key here>',\n",
" 'api_base': '<your second Azure OpenAI API base here>',\n",
" 'api_type': 'azure',\n",
" 'api_version': '2023-03-15-preview',\n",
" }, # only if the second Azure OpenAI API key is found\n",
"]\n",
"```\n",
"\n",
"You can directly override it if the above function returns an empty list, i.e., it doesn't find the keys in the specified locations."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Construct Agents\n",
"\n",
"We construct the assistant agent and the user proxy agent."
"# the purpose of the following line is to log the conversation history\n",
"oai.ChatCompletion.start_logging()\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Perform a task\n",
"\n",
"We invoke the `receive()` method of the assistant agent to start the conversation. When you run the cell below, you will be prompted to provide feedback after receving a message from the assistant agent. If you don't provide any feedback (by pressing Enter directly), the user proxy agent will try to execute the code suggested by the assistant agent on behalf of you, or terminate if the assistant agent sends a \"TERMINATE\" signal in the end of the message."
"To find the values of $a$, $b$, and $c$, we can use the system of linear equations and solve them using a Python script. Let's use the numpy library to solve the system of linear equations.\n",
"\n",
"```python\n",
"import numpy as np\n",
"\n",
"# Define the coefficients matrix (A) and the constant matrix (B)\n",
"My apologies for the misunderstanding. I mistakenly assumed x = y = 1 in the previous code, which should not have been the case. I will provide the corrected Python code to solve the given system of linear equations without any assumptions.\n",
"\n",
"```python\n",
"import sympy as sp\n",
"\n",
"a, b, c, x, y = sp.symbols('a b c x y')\n",
"\n",
"eq1 = a * x + b * y + c - (x + 7)\n",
"eq2 = a + b * x + c * y - (2 * x + 6 * y)\n",
"eq3 = a * y + b + c * x - (4 * x + y)\n",
"\n",
"# Solve the system of linear equations using sympy\n",
"I apologize for the confusion in my previous response. I will simplify the sum of $a + b + c$ for you. Here is the updated Python code with the simplified result:\n",
"\n",
"```python\n",
"import sympy as sp\n",
"\n",
"a, b, c, x, y = sp.symbols('a b c x y')\n",
"\n",
"eq1 = a * x + b * y + c - (x + 7)\n",
"eq2 = a + b * x + c * y - (2 * x + 6 * y)\n",
"eq3 = a * y + b + c * x - (4 * x + y)\n",
"\n",
"# Solve the system of linear equations using sympy\n",
"Great! The simplified sum of $a + b + c$ is 7.\n",
"\n",
"TERMINATE\n"
]
}
],
"source": [
"math_problem_to_solve = \"\"\"\n",
" Find $a + b + c$, given that $x+y \\\\neq -1$ and \n",
" \\\\begin{align}\n",
"\tax + by + c & = x + 7,\\\\\n",
"\ta + bx + cy & = 2x + 6y,\\\\\n",
"\tay + b + cx & = 4x + y.\n",
"\t\\\\end{align}.\n",
" \"\"\"\n",
"\n",
"# the assistant receives a message from the user, which contains the task description\n",
"assistant.receive(math_problem_to_solve, user)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Analyze the conversation\n",
"\n",
"The human user can provide feedback at each step. When the human user didn't provide feedback, the code was executed. The executed results and error messages are returned to the assistant, and the assistant was able to modify the code to rectify. In the end, the task is complete and a \"TERMINATE\" signal is sent from the assistant. The user skipped feedback in the end and the conversation is finished.\n",
"\n",
"After the conversation is finished, we can save the log of the conversation between the two agents. The log can be accessed from `oai.ChatCompletion.logged_history`."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'[{\"content\": \"You are a helpful AI assistant.\\\\n In the following cases, suggest python code (in a python coding block) or shell script (in a sh coding block) for the user to execute. You must indicate the script type in the code block.\\\\n 1. When you need to ask the user for some info, use the code to output the info you need, for example, browse or search the web, download/read a file.\\\\n 2. When you need to perform some task with code, use the code to perform the task and output the result. Finish the task smartly. Solve the task step by step if you need to.\\\\n If you want the user to save the code in a file before executing it, put # filename: <filename> inside the code block as the first line. Don\\'t include multiple code blocks in one response. Do not ask users to copy and paste the result. Instead, use \\'print\\' function for the output when relevant. Check the execution result returned by the user.\\\\n If the result indicates there is an error, fix the error and output the code again. Suggeset the full code instead of partial code or code changes.\\\\n Reply \\\\\"TERMINATE\\\\\" in the end when everything is done.\\\\n \", \"role\": \"system\"}, {\"content\": \"\\\\n Find $a + b + c$, given that $x+y \\\\\\\\neq -1$ and \\\\n \\\\\\\\begin{align}\\\\n\\\\tax + by + c & = x + 7,\\\\\\\\\\\\n\\\\ta + bx + cy & = 2x + 6y,\\\\\\\\\\\\n\\\\tay + b + cx & = 4x + y.\\\\n\\\\t\\\\\\\\end{align}.\\\\n \", \"role\": \"user\"}, {\"content\": \"To find the values of $a$, $b$, and $c$, we can use the system of linear equations and solve them using a Python script. Let\\'s use the numpy library to solve the system of linear equations.\\\\n\\\\n```python\\\\nimport numpy as np\\\\n\\\\n# Define the coefficients matrix (A) and the constant matrix (B)\\\\nA = np.array([[1, 1, 1], [1, 1, 1], [1, 1, 1]])\\\\nB = np.array([7, 6, 4])\\\\n\\\\n# Solve the system of linear equations using numpy\\\\nvariables = np.linalg.solve(A, B)\\\\na, b, c = variables\\\\n\\\\n# Print the values of a, b, and c\\\\nprint(\\\\\"a =\\\\\", a, \\\\\"b =\\\\\", b, \\\\\"c =\\\\\", c)\\\\n\\\\n# Calculate and print the sum of a, b, and c\\\\nsum_abc = a + b + c\\\\nprint(\\\\\"a + b + c =\\\\\", sum_abc)\\\\n```\\\\n\\\\nPlease run this Python code to calculate the values of $a$, $b$, $c$, and the sum of $a + b + c$.\", \"role\": \"assistant\"}, {\"content\": \"Are you assuming x=y=1?\", \"role\": \"user\"}, {\"content\": \"My apologies for the misunderstanding. I mistakenly assumed x = y = 1 in the previous code, which should not have been the case. I will provide the corrected Python code to solve the given system of linear equations without any assumptions.\\\\n\\\\n```python\\\\nimport sympy as sp\\\\n\\\\na, b, c, x, y = sp.symbols(\\'a b c x y\\')\\\\n\\\\neq1 = a * x + b * y + c - (x + 7)\\\\neq2 = a + b * x + c * y - (2 * x + 6 * y)\\\\neq3 = a * y + b + c * x - (4 * x + y)\\\\n\\\\n# Solve the system of linear equations using sympy\\\\nsolutions = sp.solve([eq1, eq2, eq3], (a, b, c))\\\\n\\\\n# Display the solutions for a, b, and c\\\\nprint(\\\\\"a =\\\\\", solutions[a], \\\\\"b =\\\\\", solutions[b], \\\\\"c =\\\\\", solutions[c])\\\\n\\\\n# Calculate and print the sum of a, b, and c\\\\nsum_abc = solutions[a] + solutions[b] + solutions[c]\\\\nprint(\\\\\"a + b + c =\\\\\", sum_abc)\\\\n```\\\\n\\\\nPlease run this Python code to calculate the values of $a$, $b$, $c$, and the sum of $a + b + c$ without any incorrect assumptions.\", \"role\": \"assistant\"}, {\"content\": \"exitcode: 0 (execution succeeded)\\\\nCode output: \\\\na = (x**2 - 3*x*y + 2*x + y**2 - y)/(x**2 - x*y - x + y**2 - y + 1) b = (2*x**2 - 3*x + y)/(x**2 - x*y - x + y**2 - y + 1) c = (4*x**2 - 4*x*y - 6*x + 6*y**2 - 7*y + 7)/(x**2 - x*y - x + y**2 - y + 1)\\\\na + b + c = (2*x**2 - 3*x + y)/(x**2 - x*y - x + y**2 - y + 1) + (x**2 - 3*x*y + 2*x + y**2 - y)/(x**2 - x*y - x + y**2 - y + 1) + (4*x**2 - 4*x*y - 6*x + 6*y**2 - 7*y + 7)/(x**2 - x*y - x + y**2 - y + 1)\\\\n\", \"role\": \"user\"}, {\"content\": \"The solutions
