"<a href=\"https://colab.research.google.com/github/microsoft/FLAML/blob/main/notebook/autogen_agent_two_users.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"# Multi-Agent Human-in-the-loop Application\n",
"\n",
"FLAML offers an experimental feature of interactive LLM agents, which can be used to solve various tasks with human or automatic feedback, including tasks that require using tools via code.\n",
"\n",
"In this notebook, we demonstrate an application involving multiple agents and human users to work together and accomplish a task. `AssistantAgent` is an LLM-based agent that can write Python code (in a Python coding block) for a user to execute for a given task. `UserProxyAgent` is an agent which serves as a proxy for a user to execute the code written by `AssistantAgent`. We create multiple `UserProxyAgent` instances which can represent different human users.\n",
"\n",
"## Requirements\n",
"\n",
"FLAML requires `Python>=3.8`. To run this notebook example, please install flaml with the [autogen] option:\n",
"* The [`config_list_openai_aoai`](https://microsoft.github.io/FLAML/docs/reference/autogen/oai/openai_utils#config_list_openai_aoai) function tries to create a list of configurations using Azure OpenAI endpoints and OpenAI endpoints. It assumes the api keys and api bases are stored in the corresponding environment variables or local txt files:\n",
" - OpenAI API key: os.environ[\"OPENAI_API_KEY\"] or `openai_api_key_file=\"key_openai.txt\"`.\n",
" - Azure OpenAI API key: os.environ[\"AZURE_OPENAI_API_KEY\"] or `aoai_api_key_file=\"key_aoai.txt\"`. Multiple keys can be stored, one per line.\n",
" - Azure OpenAI API base: os.environ[\"AZURE_OPENAI_API_BASE\"] or `aoai_api_base_file=\"base_aoai.txt\"`. Multiple bases can be stored, one per line.\n",
"* The [`config_list_from_json`](https://microsoft.github.io/FLAML/docs/reference/autogen/oai/openai_utils#config_list_from_json) function loads a list of configurations from an environment variable or a json file. It first looks for environment variable `env_or_file` which needs to be a valid json string. If that variable is not found, it then looks for a json file with the same name. It filters the configs by filter_dict.\n",
"\n",
"It's OK to have only the OpenAI API key, or only the Azure OpenAI API key + base. If you open this notebook in colab, you can upload your files by clicking the file icon on the left panel and then choose \"upload file\" icon.\n"
" 'api_key': '<your Azure OpenAI API key here>',\n",
" 'api_base': '<your Azure OpenAI API base here>',\n",
" 'api_type': 'azure',\n",
" 'api_version': '2023-06-01-preview',\n",
" }, # Azure OpenAI API endpoint for gpt-4\n",
" {\n",
" 'model': 'gpt-4-32k',\n",
" 'api_key': '<your Azure OpenAI API key here>',\n",
" 'api_base': '<your Azure OpenAI API base here>',\n",
" 'api_type': 'azure',\n",
" 'api_version': '2023-06-01-preview',\n",
" }, # Azure OpenAI API endpoint for gpt-4-32k\n",
"]\n",
"```\n",
"\n",
"If you open this notebook in colab, you can upload your files by clicking the file icon on the left panel and then choose \"upload file\" icon.\n",
"\n",
"You can set the value of config_list in other ways you prefer, e.g., loading from a YAML file.\n",
"\n",
"## Construct Agents\n",
"\n",
"We define `ask_expert` function to start a conversation between two agents and return a summary of the result. We construct an assistant agent named \"assistant_for_expert\" and a user proxy agent named \"expert\". We specify `human_input_mode` as \"ALWAYS\" in the user proxy agent, which will always ask for feedback from the expert user."
"We construct another assistant agent named \"assistant_for_student\" and a user proxy agent named \"student\". We specify `human_input_mode` as \"TERMINATE\" in the user proxy agent, which will ask for feedback when it receives a \"TERMINATE\" signal from the assistant agent. We set the `functions` in `AssistantAgent` and `function_map` in `UserProxyAgent` to use the created `ask_expert` function.\n",
"\n",
"For simplicity, the `ask_expert` function is defined to run locally. For real applications, the function should run remotely to interact with an expert user."
" \"model\": \"gpt-4-0613\", # make sure the endpoint you use supports the model\n",
" \"temperature\": 0,\n",
" \"functions\": [\n",
" {\n",
" \"name\": \"ask_expert\",\n",
" \"description\": \"ask expert when you can't solve the problem satisfactorily.\",\n",
" \"parameters\": {\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
" \"message\": {\n",
" \"type\": \"string\",\n",
" \"description\": \"question to ask expert. Make sure the question include enough context, such as the code and the execution result. The expert does not know the conversation between you and the user, unless you share the conversation with the expert.\",\n",
"We invoke the `initiate_chat()` method of the student proxy agent to start the conversation. When you run the cell below, you will be prompted to provide feedback after the assistant agent sends a \"TERMINATE\" signal in the end of the message. If you don't provide any feedback (by pressing Enter directly), the conversation will finish. Before the \"TERMINATE\" signal, the student proxy agent will try to execute the code suggested by the assistant agent on behalf of the user."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"student (to assistant_for_student):\n",
"\n",
"Find $a + b + c$, given that $x+y \\neq -1$ and \n",
"This is a system of linear equations. We can solve it using Python's sympy library, which provides a method to solve systems of equations. Here is the Python code to solve it:\n",
"The result you got is a symbolic expression in terms of x and y. This is because the system of equations is underdetermined, meaning there are more variables (a, b, c, x, y) than there are equations. Therefore, the solution is not unique and depends on the values of x and y.\n",
"\n",
"If you have specific values for x and y, you can substitute them into the expression to get a numerical result. Here is how you can do it:\n",
"I apologize for the confusion. The variable `result` was defined in the previous code block and is not available in the current one. We need to combine the two code blocks into one. Here is the corrected code:\n",
"Great! The sum of a, b, and c for the given values of x and y is 7. If you have different values for x and y, you can substitute them into the code to get the corresponding sum of a, b, and c. \n",
"To verify the correctness of the solution for different values of x and y, you can substitute the values of a, b, and c back into the original equations and check if the equations hold true. Here is the Python code to do this:\n",
"\n",
"```python\n",
"# define the values for x and y\n",
"x_val = 1 # replace with your value\n",
"y_val = 2 # replace with your value\n",
"\n",
"# substitute the values of x and y into the solution\n",
"solution_xy = {var: val.subs({x: x_val, y: y_val}) for var, val in solution.items()}\n",
"\n",
"# substitute the values of a, b, c, x, and y into the original equations\n",
"This code will print `True` for each equation if the equation holds true for the given values of x and y, and `False` otherwise. Please replace `x_val` and `y_val` with your specific values and run the code to verify the solution.\n",
"I apologize for the confusion. The variable `solution` was defined in a previous code block and is not available in the current one. We need to combine the two code blocks into one. Here is the corrected code:\n",
"\n",
"```python\n",
"from sympy import symbols, Eq, solve\n",
"\n",
"# define the symbols\n",
"x, y, a, b, c = symbols('x y a b c')\n",
"\n",
"# define the equations\n",
"eq1 = Eq(a*x + b*y + c, x + 7)\n",
"eq2 = Eq(a + b*x + c*y, 2*x + 6*y)\n",
"eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
"\n",
"# solve the system of equations\n",
"solution = solve((eq1, eq2, eq3), (a, b, c))\n",
"\n",
"# define the values for x and y\n",
"x_val = 1 # replace with your value\n",
"y_val = 2 # replace with your value\n",
"\n",
"# substitute the values of x and y into the solution\n",
"solution_xy = {var: val.subs({x: x_val, y: y_val}) for var, val in solution.items()}\n",
"\n",
"# substitute the values of a, b, c, x, and y into the original equations\n",
"I apologize for the confusion. The `lhs` and `rhs` attributes are not available for the `BooleanTrue` object. We should use the `simplify()` function to simplify the difference between the left-hand side and the right-hand side of the equations. If the simplified result is zero, it means the equation holds true. Here is the corrected code:\n",
"\n",
"```python\n",
"from sympy import symbols, Eq, solve\n",
"\n",
"# define the symbols\n",
"x, y, a, b, c = symbols('x y a b c')\n",
"\n",
"# define the equations\n",
"eq1 = Eq(a*x + b*y + c, x + 7)\n",
"eq2 = Eq(a + b*x + c*y, 2*x + 6*y)\n",
"eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
"\n",
"# solve the system of equations\n",
"solution = solve((eq1, eq2, eq3), (a, b, c))\n",
"\n",
"# define the values for x and y\n",
"x_val = 1 # replace with your value\n",
"y_val = 2 # replace with your value\n",
"\n",
"# substitute the values of x and y into the solution\n",
"solution_xy = {var: val.subs({x: x_val, y: y_val}) for var, val in solution.items()}\n",
"\n",
"# substitute the values of a, b, c, x, and y into the original equations\n",
"***** Suggested function Call: ask_expert *****\n",
"Arguments: \n",
"{\n",
" \"message\": \"I tried to solve a system of equations and then verify the solution by substituting the solution back into the original equations. However, I encountered an error saying 'BooleanTrue' object has no attribute 'lhs'. Here is the code I used:\\n\\n```python\\nfrom sympy import symbols, Eq, solve\\n\\n# define the symbols\\nx, y, a, b, c = symbols('x y a b c')\\n\\n# define the equations\\neq1 = Eq(a*x + b*y + c, x + 7)\\neq2 = Eq(a + b*x + c*y, 2*x + 6*y)\\neq3 = Eq(a*y + b + c*x, 4*x + y)\\n\\n# solve the system of equations\\nsolution = solve((eq1, eq2, eq3), (a, b, c))\\n\\n# define the values for x and y\\nx_val = 1 # replace with your value\\ny_val = 2 # replace with your value\\n\\n# substitute the values of x and y into the solution\\nsolution_xy = {var: val.subs({x: x_val, y: y_val}) for var, val in solution.items()}\\n\\n# substitute the values of a, b, c, x, and y into the original equations\\neq1_xy = eq1.subs({**solution_xy, x: x_val, y: y_val})\\neq2_xy = eq2.subs({**solution_xy, x: x_val, y: y_val})\\neq3_xy = eq3.subs({**solution_xy, x: x_val, y: y_val})\\n\\n# check if the equations hold true\\nprint((eq1_xy.lhs - eq1_xy.rhs).simplify() == 0)\\nprint((eq2_xy.lhs - eq2_xy.rhs).simplify() == 0)\\nprint((eq3_xy.lhs - eq3_xy.rhs).simplify() == 0)\\n```\\n\\nThe error occurred at the line `print((eq1_xy.lhs - eq1_xy.rhs).simplify() == 0)`. Could you please help me understand what went wrong and how to fix it?\"\n",
">>>>>>>> NO HUMAN INPUT RECEIVED. USING AUTO REPLY FOR THE USER...\n",
"expert (to assistant_for_expert):\n",
"\n",
"I tried to solve a system of equations and then verify the solution by substituting the solution back into the original equations. However, I encountered an error saying 'BooleanTrue' object has no attribute 'lhs'. Here is the code I used:\n",
"\n",
"```python\n",
"from sympy import symbols, Eq, solve\n",
"\n",
"# define the symbols\n",
"x, y, a, b, c = symbols('x y a b c')\n",
"\n",
"# define the equations\n",
"eq1 = Eq(a*x + b*y + c, x + 7)\n",
"eq2 = Eq(a + b*x + c*y, 2*x + 6*y)\n",
"eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
"\n",
"# solve the system of equations\n",
"solution = solve((eq1, eq2, eq3), (a, b, c))\n",
"\n",
"# define the values for x and y\n",
"x_val = 1 # replace with your value\n",
"y_val = 2 # replace with your value\n",
"\n",
"# substitute the values of x and y into the solution\n",
"solution_xy = {var: val.subs({x: x_val, y: y_val}) for var, val in solution.items()}\n",
"\n",
"# substitute the values of a, b, c, x, and y into the original equations\n",
"The error occurred at the line `print((eq1_xy.lhs - eq1_xy.rhs).simplify() == 0)`. Could you please help me understand what went wrong and how to fix it?\n",
"The error message 'BooleanTrue' object has no attribute 'lhs' indicates that the object `eq1_xy` is not an equation but a boolean value `True`. This happens when the substitution in the line `eq1_xy = eq1.subs({**solution_xy, x: x_val, y: y_val})` results in an equation that is already satisfied, i.e., the left-hand side (lhs) equals the right-hand side (rhs). In such a case, the `subs` method simplifies the equation to `True`, and `True` does not have `lhs` or `rhs` attributes.\n",
"\n",
"To fix this issue, you can check if the result of the substitution is an instance of `Eq` before trying to access its `lhs` and `rhs` attributes. If it's not an instance of `Eq`, it means the equation is already satisfied, and you can print `True` directly. Here is the corrected code:\n",
"\n",
"```python\n",
"from sympy import symbols, Eq, solve\n",
"\n",
"# define the symbols\n",
"x, y, a, b, c = symbols('x y a b c')\n",
"\n",
"# define the equations\n",
"eq1 = Eq(a*x + b*y + c, x + 7)\n",
"eq2 = Eq(a + b*x + c*y, 2*x + 6*y)\n",
"eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
"\n",
"# solve the system of equations\n",
"solution = solve((eq1, eq2, eq3), (a, b, c))\n",
"\n",
"# define the values for x and y\n",
"x_val = 1 # replace with your value\n",
"y_val = 2 # replace with your value\n",
"\n",
"# substitute the values of x and y into the solution\n",
"solution_xy = {var: val.subs({x: x_val, y: y_val}) for var, val in solution.items()}\n",
"\n",
"# substitute the values of a, b, c, x, and y into the original equations\n",
"Great! The output `True` for each equation indicates that the solution satisfies all the equations. This means the system of equations was solved correctly and the solution was verified successfully. If you have any other questions or need further assistance, feel free to ask. Otherwise, if everything is done, we can terminate this session.\n",
"Great! The code has successfully calculated the sum of a, b, and c from the simplified solution. The result is:\n",
"\n",
"a + b + c = 7\n",
"\n",
"This means that the sum of the solutions to your system of equations is 7. If you have any other questions or need further assistance, feel free to ask. Otherwise, if everything is done, we can terminate this session.\n",
"1. You had a system of three equations with five variables (a, b, c, x, y). The equations were:\n",
"\n",
" a*x + b*y + c = x + 7\n",
"\n",
" a + b*x + c*y = 2*x + 6*y\n",
"\n",
" a*y + b + c*x = 4*x + y\n",
"\n",
"2. We solved this system of equations for a, b, and c using sympy's `solve` function. The solution was:\n",
"\n",
" a = (x² - 3xy + 2x + y² - y) / (x² - xy - x + y² - y + 1)\n",
"\n",
" b = (2x² - 3x + y) / (x² - xy - x + y² - y + 1)\n",
"\n",
" c = (4x² - 4xy - 6x + 6y² - 7y + 7) / (x² - xy - x + y² - y + 1)\n",
"\n",
"3. We simplified the solution using sympy's `simplify` function.\n",
"\n",
"4. We calculated the sum of a, b, and c from the simplified solution and found that a + b + c = 7.\n",
"\n",
"This means that the sum of the solutions to your system of equations is 7, regardless of the values of x and y. If you have any other questions or need further assistance, feel free to ask. Otherwise, if everything is done, we can terminate this session.\n",
"***** Response from calling function \"ask_expert\" *****\n",
"Sure, here is the summary of the solution:\n",
"\n",
"1. You had a system of three equations with five variables (a, b, c, x, y). The equations were:\n",
"\n",
" a*x + b*y + c = x + 7\n",
"\n",
" a + b*x + c*y = 2*x + 6*y\n",
"\n",
" a*y + b + c*x = 4*x + y\n",
"\n",
"2. We solved this system of equations for a, b, and c using sympy's `solve` function. The solution was:\n",
"\n",
" a = (x² - 3xy + 2x + y² - y) / (x² - xy - x + y² - y + 1)\n",
"\n",
" b = (2x² - 3x + y) / (x² - xy - x + y² - y + 1)\n",
"\n",
" c = (4x² - 4xy - 6x + 6y² - 7y + 7) / (x² - xy - x + y² - y + 1)\n",
"\n",
"3. We simplified the solution using sympy's `simplify` function.\n",
"\n",
"4. We calculated the sum of a, b, and c from the simplified solution and found that a + b + c = 7.\n",
"\n",
"This means that the sum of the solutions to your system of equations is 7, regardless of the values of x and y. If you have any other questions or need further assistance, feel free to ask. Otherwise, if everything is done, we can terminate this session.\n",
"I apologize for the confusion. The error occurred because the equations were already simplified to `True` when the values of a, b, c, x, and y were substituted. This means that the equations hold true for the given values of x and y. Therefore, we don't need to subtract the right-hand side from the left-hand side and simplify the result. Instead, we can directly print the simplified equations. Here is the corrected code:\n",
"\n",
"```python\n",
"from sympy import symbols, Eq, solve\n",
"\n",
"# define the symbols\n",
"x, y, a, b, c = symbols('x y a b c')\n",
"\n",
"# define the equations\n",
"eq1 = Eq(a*x + b*y + c, x + 7)\n",
"eq2 = Eq(a + b*x + c*y, 2*x + 6*y)\n",
"eq3 = Eq(a*y + b + c*x, 4*x + y)\n",
"\n",
"# solve the system of equations\n",
"solution = solve((eq1, eq2, eq3), (a, b, c))\n",
"\n",
"# define the values for x and y\n",
"x_val = 1 # replace with your value\n",
"y_val = 2 # replace with your value\n",
"\n",
"# substitute the values of x and y into the solution\n",
"solution_xy = {var: val.subs({x: x_val, y: y_val}) for var, val in solution.items()}\n",
"\n",
"# substitute the values of a, b, c, x, and y into the original equations\n",
"This code will print `True` for each equation if the equation holds true for the given values of x and y. Please replace `x_val` and `y_val` with your specific values and run the code to verify the solution.\n",
"Great! The output `True` for each equation indicates that the equations hold true for the given values of x and y. Therefore, the solution is correct. \n",
"\n",
"If you have different values for x and y, you can substitute them into the code to verify the solution for those values.\n",
"# the assistant receives a message from the student, which contains the task description\n",
"student.initiate_chat(\n",
" assistant_for_student,\n",
" message=\"\"\"Find $a + b + c$, given that $x+y \\\\neq -1$ and \n",
" \\\\begin{align}\n",
"\tax + by + c & = x + 7,\\\\\n",
"\ta + bx + cy & = 2x + 6y,\\\\\n",
"\tay + b + cx & = 4x + y.\n",
"\t\\\\end{align}.\n",
"\"\"\",\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"When the assistant needs to consult the expert, it suggests a function call to `ask_expert`. When this happens, a line like the following will be displayed:\n",
"\n",
"***** Suggested function Call: ask_expert *****\n"
