"FLAML offers an experimental feature of interactive LLM agents, which can be used to solve various tasks with human or automatic feedback, including tasks that require using tools via code. Please find documentation about this feature [here](https://microsoft.github.io/FLAML/docs/Use-Cases/Auto-Generation#agents-experimental).\n",
"In this notebook, we demonstrate how to use multiple agents to work together and accomplish a task which requires finding info from the web and coding. `AssistantAgent` is an LLM-based agent that can write and debug Python code (in a Python coding block) for a user to execute for a given task. `UserProxyAgent` is an agent which serves as a proxy for a user to execute the code written by `AssistantAgent`. We further create a planning agent for the assistant agent to consult. The planning agent is a variation of the LLM-based `AssistantAgent` with a different system message.\n",
"* The [`config_list_openai_aoai`](https://microsoft.github.io/FLAML/docs/reference/autogen/oai/openai_utils#config_list_openai_aoai) function tries to create a list of configurations using Azure OpenAI endpoints and OpenAI endpoints. It assumes the api keys and api bases are stored in the corresponding environment variables or local txt files:\n",
" - OpenAI API key: os.environ[\"OPENAI_API_KEY\"] or `openai_api_key_file=\"key_openai.txt\"`.\n",
" - Azure OpenAI API key: os.environ[\"AZURE_OPENAI_API_KEY\"] or `aoai_api_key_file=\"key_aoai.txt\"`. Multiple keys can be stored, one per line.\n",
" - Azure OpenAI API base: os.environ[\"AZURE_OPENAI_API_BASE\"] or `aoai_api_base_file=\"base_aoai.txt\"`. Multiple bases can be stored, one per line.\n",
"* The [`config_list_from_json`](https://microsoft.github.io/FLAML/docs/reference/autogen/oai/openai_utils#config_list_from_json) function loads a list of configurations from an environment variable or a json file. It first looks for environment variable `env_or_file` which needs to be a valid json string. If that variable is not found, it then looks for a json file with the same name. It filters the configs by filter_dict.\n",
"It's OK to have only the OpenAI API key, or only the Azure OpenAI API key + base. If you open this notebook in colab, you can upload your files by clicking the file icon on the left panel and then choose \"upload file\" icon.\n"
"We construct the planning agent named \"planner\" and a user proxy agent for the planner named \"planner_user\". We specify `human_input_mode` as \"NEVER\" in the user proxy agent, which will never ask for human feedback. We define `ask_planner` function to send a message to planner and return the suggestion from the planner."
" system_message=\"You are a helpful AI assistant. You suggest coding and reasoning steps for another AI assistant to accomplish a task. Do not suggest concrete code. For any action beyond writing code or reasoning, convert it to a step which can be implemented by writing code. For example, the action of browsing the web can be implemented by writing code which reads and prints the content of a web page. Finally, inspect the execution result. If the plan is not good, suggest a better plan. If the execution is wrong, analyze the error and suggest a fix.\"\n",
"We construct the assistant agent and the user proxy agent. We specify `human_input_mode` as \"TERMINATE\" in the user proxy agent, which will ask for feedback when it receives a \"TERMINATE\" signal from the assistant agent. We set the `functions` in `AssistantAgent` and `function_map` in `UserProxyAgent` to use the created `ask_planner` function."
" \"description\": \"ask planner to: 1. get a plan for finishing a task, 2. verify the execution result of the plan and potentially suggest new plan.\",\n",
" \"description\": \"question to ask planner. Make sure the question include enough context, such as the code and the execution result. The planner does not know the conversation between you and the user, unless you share the conversation with the planner.\",\n",
"We invoke the `initiate_chat()` method of the user proxy agent to start the conversation. When you run the cell below, you will be prompted to provide feedback after the assistant agent sends a \"TERMINATE\" signal in the end of the message. If you don't provide any feedback (by pressing Enter directly), the conversation will finish. Before the \"TERMINATE\" signal, the user proxy agent will try to execute the code suggested by the assistant agent on behalf of the user."
"To suggest a fix to an open good first issue of FLAML, we first need to fetch the list of open issues labeled as \"good first issue\" from the FLAML GitHub repository. We can do this using the GitHub API. Here is a Python script that fetches and prints the list of such issues.\n",
"Please save this script as `fetch_issues.py` and run it. It will print the list of open issues labeled as \"good first issue\" in the FLAML repository.\n",
" \"message\": \"Here are the open good first issues of FLAML: \\n\\nIssue #1120: use_label_encoder warning with xgboost\\nIssue #1034: Use appropriate wait time for retry based on the error message.\\nIssue #1029: Issues with Adding Custom APIs in Auto Generation\\nIssue #981: Running flaml[tune] using \\\"-O\\\" flag for python interpreter (optimization - disables assertions) crashes\\nIssue #903: Conditional parameter flow2 crash\\nIssue #884: indentation space\\nIssue #882: Check if openml version is required\\nIssue #834: Adjust the indent\\nIssue #821: pip install flaml FAIL\\nIssue #807: Isolate the ensemble part and expose it to users\\nIssue #805: how to pass categorical features names or indices to learner\\nIssue #785: Flaml/LightGBM - Shouldn't I found better/faster or equal results from FLAML than direct LightGBM?\\nIssue #764: Add an announcement of the discord channel\\nIssue #748: Documentation about small budget\\nIssue #737: Make zero-shot automl more discoverable\\nIssue #509: New HCrystalBall release\\nIssue #429: samples about conversion to ONNX\\nIssue #413: support anomaly detection\\nIssue #304: CatBoost Fails with Keyword 'groups'\\n\\nPlease suggest a fix for one of these issues.\"\n",
"Without certain details about the issues, it's a bit challenging to suggest explicit fixes. However, I'll give you a roadmap to tackle some of these:\n",
"Plan: Update the call to the XGBoost module in FLAML to either stop using the label encoder or specify explicitly that the label encoder should be used, based on how FLAML is currently set up.\n",
"Plan: Identify the minimum required version of the `openml` library that FLAML can operate with. If there's no minimum version, then there's no issue. If there is, update the documentation and the package requirements to show this.\n",
"Plan: Investigate the specific errors when trying to install via pip. If dependencies are missing or incompatible, update the package requirements. If there are issues with the package, fix those issues.\n",
"Reasoning: It seems users are facing difficulty in passing categorical features. This could be due to unclear or incorrect documentation, or missing functionality.\n",
"Plan: Inspect the current documentation and functionality. Update the documentation to clarify how to pass these features, and if the functionality is missing, implement the functionality.\n",
"Remember to thoroughly inspect the execution result for each issue to ensure your fix is effective. If the plan does not appear to fix the issue, adjust your plan based on the error message or unexpected behavior.\n",
"***** Response from calling function \"ask_planner\" *****\n",
"Without certain details about the issues, it's a bit challenging to suggest explicit fixes. However, I'll give you a roadmap to tackle some of these:\n",
"Reasoning: It appears like the `use_label_encoder` warning stems from deprecated usage in the XGBoost module that FLAML utilizes. \n",
"\n",
"Plan: Update the call to the XGBoost module in FLAML to either stop using the label encoder or specify explicitly that the label encoder should be used, based on how FLAML is currently set up.\n",
"Plan: Identify the minimum required version of the `openml` library that FLAML can operate with. If there's no minimum version, then there's no issue. If there is, update the documentation and the package requirements to show this.\n",
"Plan: Investigate the specific errors when trying to install via pip. If dependencies are missing or incompatible, update the package requirements. If there are issues with the package, fix those issues.\n",
"Reasoning: It seems users are facing difficulty in passing categorical features. This could be due to unclear or incorrect documentation, or missing functionality.\n",
"Plan: Inspect the current documentation and functionality. Update the documentation to clarify how to pass these features, and if the functionality is missing, implement the functionality.\n",
"\n",
"Remember to thoroughly inspect the execution result for each issue to ensure your fix is effective. If the plan does not appear to fix the issue, adjust your plan based on the error message or unexpected behavior.\n",
"The warning is likely due to a deprecated usage in the XGBoost module that FLAML utilizes. To fix this, we need to update the call to the XGBoost module in FLAML to either stop using the label encoder or specify explicitly that the label encoder should be used, based on how FLAML is currently set up.\n",
"1. Clone the FLAML repository to your local machine.\n",
"2. Navigate to the file where XGBoost is being called.\n",
"3. Look for the `use_label_encoder` parameter in the XGBoost function call.\n",
"4. If `use_label_encoder` is set to True, you can try setting it to False to see if the warning disappears.\n",
"5. If the warning persists, it might be necessary to refactor the code to avoid using label encoding in XGBoost, or to handle label encoding before data is passed to XGBoost.\n",
"6. After making changes, run the tests to make sure your changes do not break anything.\n",
"7. If everything is fine, commit your changes and create a pull request.\n",
"Please note that without the actual codebase and the specific details of the issue, this is a general approach and might need to be adjusted based on the actual code and setup of FLAML. \n",
"When the assistant needs to consult the planner, it suggests a function call to `ask_planner`. When this happens, a line like the following will be displayed:\n",
"\n",
"***** Suggested function Call: ask_planner *****\n"
