autogen/website/docs/tutorial/human-in-the-loop.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Allowing Human Feedback in Agents\n",
    "\n",
    "In the last two chapters we introduced the `ConversableAgent` class and showed how you can use it to create autonomous (`human_input_mode=NEVER`) agents that can accomplish tasks. We also showed how to properly terminate a conversation between agents.\n",
    "\n",
    "But many applications may require putting humans in-the-loop with agents. For example, to allow human feedback to steer agents in the right direction, specify goals, etc. In this chapter, we will show how AutoGen supports human intervention."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Human Input Modes\n",
    "\n",
    "Currently AutoGen supports three modes for human input. The mode is specified through\n",
    "the `human_input_mode` argument of the `ConversableAgent`. The three modes are:\n",
    "\n",
    "1. `NEVER`: human input is never requested.\n",
    "2. `TERMINATE` (default): human input is only requested when a termination condition is\n",
    "    met. Note that in this mode if the human chooses to intercept and reply, the conversation continues\n",
    "    and the counter used by `max_consectuive_auto_reply` is reset.\n",
    "3. `ALWAYS`: human input is always requested and the human can choose to skip and trigger an auto-reply,\n",
    "    intercept and provide feedback, or terminate the conversation. Note that in this mode\n",
    "    termination based on `max_consecutive_auto_reply` is ignored.\n",
    "\n",
    "The previous chapters already showed many examples of the cases when `human_input_mode` is `NEVER`. \n",
    "Below we show one such example again and then show the differences when this mode is set to `ALWAYS` and `NEVER` instead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from autogen import ConversableAgent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Human Input Mode = `NEVER`\n",
    "\n",
    "In this mode, human input is never requested and the termination conditions\n",
    "are used to terminate.\n",
    "This mode is useful when you want your agents to act fully autonomously.\n",
    "\n",
    "Here is an example of using this mode to run a simple guess-a-number game between\n",
    "two agents, with the maximum number of guesses set to 5 through\n",
    "`max_consecutive_auto_reply`, and the termination message set to check for the \n",
    "number that is the correct guess."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "I have a number between 1 and 100. Guess it!\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "50\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "75\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "62\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "56\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "53\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "agent_with_number = ConversableAgent(\n",
    "    \"agent_with_number\",\n",
    "    system_message=\"You are playing a game of guess-my-number. You have the \"\n",
    "    \"number 53 in your mind, and I will try to guess it. \"\n",
    "    \"If I guess too high, say 'too high', if I guess too low, say 'too low'. \",\n",
    "    llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
    "    is_termination_msg=lambda msg: \"53\" in msg[\"content\"],  # terminate if the number is guessed by the other agent\n",
    "    human_input_mode=\"NEVER\",  # never ask for human input\n",
    ")\n",
    "\n",
    "agent_guess_number = ConversableAgent(\n",
    "    \"agent_guess_number\",\n",
    "    system_message=\"I have a number in my mind, and you will try to guess it. \"\n",
    "    \"If I say 'too high', you should guess a lower number. If I say 'too low', \"\n",
    "    \"you should guess a higher number. \",\n",
    "    llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
    "    human_input_mode=\"NEVER\",\n",
    ")\n",
    "\n",
    "result = agent_with_number.initiate_chat(\n",
    "    agent_guess_number,\n",
    "    message=\"I have a number between 1 and 100. Guess it!\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Yay! The game is over. The guessing agent got the number correctly in exactly\n",
    "5 guesses using binary search -- very efficient!\n",
    "You can see that the conversation was terminated after the 5th guess which\n",
    "triggered both the counter-based and message-based termination conditions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Human Input Mode = `ALWAYS`\n",
    "\n",
    "In this mode, human input is always requested and the human can choose to skip,\n",
    "intersecpt, or terminate the conversation.\n",
    "Let us see this mode in action by playing the same game as before with the agent with the number, but this time\n",
    "participating in the game as a human.\n",
    "We will be the agent that is guessing the number, and play against the agent\n",
    "with the number from before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
      "\n",
      "10\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
      "\n",
      "Too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
      "\n",
      "70\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
      "\n",
      "50\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
      "\n",
      "Too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
      "\n",
      "53\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "human_proxy = ConversableAgent(\n",
    "    \"human_proxy\",\n",
    "    llm_config=False,  # no LLM used for human proxy\n",
    "    human_input_mode=\"ALWAYS\",  # always ask for human input\n",
    ")\n",
    "\n",
    "# Start a chat with the agent with number with an initial guess.\n",
    "result = human_proxy.initiate_chat(\n",
    "    agent_with_number,  # this is the same agent with the number as before\n",
    "    message=\"10\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you run the code above, you will be prompt to enter a response\n",
    "each time it is your turn to speak. You can see the human in the conversation\n",
    "was not very good at guessing the number... but hey the agent was nice enough\n",
    "to give out the number in the end."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Human Input Mode = `TERMINATE`\n",
    "\n",
    "In this mode, human input is only requested when a termination condition is\n",
    "met. **If the human choose to intercept and reply, the counter will be reset**; if \n",
    "the human choose to skip, automatic reply mechanism will be used; if the human\n",
    "choose to terminate, the conversation will be terminated.\n",
    "\n",
    "Let us see this mode in action by playing the same game again, but this time\n",
    "the agent with the number will choose a new number and the game will restart."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "I have a number between 1 and 100. Guess it!\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "50\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "75\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "62\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "56\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "53\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Correct! You've guessed my number.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Great! This was a fun game. Let's play again sometime.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "10\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Your guess is too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Okay, let's try 5.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Your guess is too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Okay, how about 2?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Your guess is too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Alright, then it must be 3.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Yes, correct! The number was 3. Good job guessing it!\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Thank you! That was indeed a fun game. I look forward to playing again with you.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "I'm glad you enjoyed it! I'm always here for a good game. Don't hesitate to come back when you want to play again.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Surely, I will. This was fun. Thanks for playing with me. See you next time!\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "You're welcome! I'm glad you had fun. Looking forward to our next game. See you next time!\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Lets play again\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Sure! I'll think of a number between 1 and 100. Let's start guessing!\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "I'll start with 50.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "The guess is too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Okay, let's try 75.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "The guess is too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "How about 62?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "The guess is too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Then, let's try 68.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "The guess is too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Okay, how about 65?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Yes, correct! The number was 65. Good job guessing it!\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "agent_with_number = ConversableAgent(\n",
    "    \"agent_with_number\",\n",
    "    system_message=\"You are playing a game of guess-my-number. \"\n",
    "    \"In the first game, you have the \"\n",
    "    \"number 53 in your mind, and I will try to guess it. \"\n",
    "    \"If I guess too high, say 'too high', if I guess too low, say 'too low'. \",\n",
    "    llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
    "    max_consecutive_auto_reply=5,\n",
    "    human_input_mode=\"TERMINATE\",  # ask for human input until the game is terminated\n",
    ")\n",
    "\n",
    "agent_guess_number = ConversableAgent(\n",
    "    \"agent_guess_number\",\n",
    "    system_message=\"I have a number in my mind, and you will try to guess it. \"\n",
    "    \"If I say 'too high', you should guess a lower number. If I say 'too low', \"\n",
    "    \"you should guess a higher number. \",\n",
    "    llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
    "    human_input_mode=\"NEVER\",\n",
    ")\n",
    "\n",
    "result = agent_with_number.initiate_chat(\n",
    "    agent_guess_number,\n",
    "    message=\"I have a number between 1 and 100. Guess it!\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous conversation, we were asked to provide human input first time\n",
    "when the counter-based termination condition was met. We intercepted the\n",
    "message and replied \"Let's play again\". The conversation continued and the\n",
    "counter was reset. The game continued until the counter-based termination\n",
    "condition was met again, and this time we chose to skip. The conversation\n",
    "continued again but human input was requested at every turn since, and we\n",
    "chose to skip each time until the game was over."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "In this chapter, we showed you how to use the human-in-the-loop component\n",
    "to provide human feedback to agent and to terminate conversation.\n",
    "We also showed you the different human input modes and how they affect\n",
    "the behavior of the human-in-the-loop component.\n",
    "\n",
    "The next chapter will be all about code executor -- the most powerful\n",
    "component second only to LLMs."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "autogen",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}