autogen/website/docs/tutorial/human-in-the-loop.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Allowing Human Feedback in Agents\n",
    "\n",
    "In the last two chapters we introduced the `ConversableAgent` class and showed how you can use it to create autonomous (`human_input_mode=NEVER`) agents that can accomplish tasks. We also showed how to properly terminate a conversation between agents.\n",
    "\n",
    "But many applications may require putting humans in-the-loop with agents. For example, to allow human feedback to steer agents in the right direction, specify goals, etc. In this chapter, we will show how AutoGen supports human intervention.\n",
    "\n",
    "In AutoGen's `ConversableAgent`, the human-the-loop component sits in front\n",
    "of the auto-reply components. It can intercept the incoming messages and\n",
    "decide whether to pass them to the auto-reply components or to provide\n",
    "human feedback. The figure below illustrates the design.\n",
    "\n",
    "```{=mdx}\n",
    "![Human in the loop](./assets/human-in-the-loop.png)\n",
    "```\n",
    "\n",
    "The human-in-the-loop component can be customized through the `human_input_mode` parameter.\n",
    "We will show you how to use it in the following sections."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Human Input Modes\n",
    "\n",
    "Currently AutoGen supports three modes for human input. The mode is specified through\n",
    "the `human_input_mode` argument of the `ConversableAgent`. The three modes are:\n",
    "\n",
    "1. `NEVER`: human input is never requested.\n",
    "2. `TERMINATE` (default): human input is only requested when a termination condition is\n",
    "    met. Note that in this mode if the human chooses to intercept and reply, the conversation continues\n",
    "    and the counter used by `max_consectuive_auto_reply` is reset.\n",
    "3. `ALWAYS`: human input is always requested and the human can choose to skip and trigger an auto-reply,\n",
    "    intercept and provide feedback, or terminate the conversation. Note that in this mode\n",
    "    termination based on `max_consecutive_auto_reply` is ignored.\n",
    "\n",
    "The previous chapters already showed many examples of the cases when `human_input_mode` is `NEVER`. \n",
    "Below we show one such example again and then show the differences when this mode is set to `ALWAYS` and `NEVER` instead."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Human Input Mode = `NEVER`\n",
    "\n",
    "In this mode, human input is never requested and the termination conditions\n",
    "are used to terminate.\n",
    "This mode is useful when you want your agents to act fully autonomously.\n",
    "\n",
    "Here is an example of using this mode to run a simple guess-a-number game between\n",
    "two agents, the termination message is set to check for the \n",
    "number that is the correct guess."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "I have a number between 1 and 100. Guess it!\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 50?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 75?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 63?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 57?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 54?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 52?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 53?\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from autogen import ConversableAgent\n",
    "\n",
    "agent_with_number = ConversableAgent(\n",
    "    \"agent_with_number\",\n",
    "    system_message=\"You are playing a game of guess-my-number. You have the \"\n",
    "    \"number 53 in your mind, and I will try to guess it. \"\n",
    "    \"If I guess too high, say 'too high', if I guess too low, say 'too low'. \",\n",
    "    llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
    "    is_termination_msg=lambda msg: \"53\" in msg[\"content\"],  # terminate if the number is guessed by the other agent\n",
    "    human_input_mode=\"NEVER\",  # never ask for human input\n",
    ")\n",
    "\n",
    "agent_guess_number = ConversableAgent(\n",
    "    \"agent_guess_number\",\n",
    "    system_message=\"I have a number in my mind, and you will try to guess it. \"\n",
    "    \"If I say 'too high', you should guess a lower number. If I say 'too low', \"\n",
    "    \"you should guess a higher number. \",\n",
    "    llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
    "    human_input_mode=\"NEVER\",\n",
    ")\n",
    "\n",
    "result = agent_with_number.initiate_chat(\n",
    "    agent_guess_number,\n",
    "    message=\"I have a number between 1 and 100. Guess it!\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Yay! The game is over. The guessing agent got the number correctly \n",
    "using binary search -- very efficient!\n",
    "You can see that the conversation was terminated after the guessing agent\n",
    "said the correct number, which triggered \n",
    "the message-based termination condition."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Human Input Mode = `ALWAYS`\n",
    "\n",
    "In this mode, human input is always requested and the human can choose to skip,\n",
    "intersecpt, or terminate the conversation.\n",
    "Let us see this mode in action by playing the same game as before with the agent with the number, but this time\n",
    "participating in the game as a human.\n",
    "We will be the agent that is guessing the number, and play against the agent\n",
    "with the number from before."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
      "\n",
      "10\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
      "\n",
      "Too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
      "\n",
      "79\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
      "\n",
      "76\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
      "\n",
      "I give up\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
      "\n",
      "That's okay! The number I was thinking of was 53.\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "human_proxy = ConversableAgent(\n",
    "    \"human_proxy\",\n",
    "    llm_config=False,  # no LLM used for human proxy\n",
    "    human_input_mode=\"ALWAYS\",  # always ask for human input\n",
    ")\n",
    "\n",
    "# Start a chat with the agent with number with an initial guess.\n",
    "result = human_proxy.initiate_chat(\n",
    "    agent_with_number,  # this is the same agent with the number as before\n",
    "    message=\"10\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you run the code above, you will be prompt to enter a response\n",
    "each time it is your turn to speak. You can see the human in the conversation\n",
    "was not very good at guessing the number... but hey the agent was nice enough\n",
    "to give out the number in the end."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Human Input Mode = `TERMINATE`\n",
    "\n",
    "In this mode, human input is only requested when a termination condition is\n",
    "met. **If the human choose to intercept and reply, the counter will be reset**; if \n",
    "the human choose to skip, automatic reply mechanism will be used; if the human\n",
    "choose to terminate, the conversation will be terminated.\n",
    "\n",
    "Let us see this mode in action by playing the same game again, but this time\n",
    "the guessing agent will only have two chances to guess the number, and if it \n",
    "fails, the human will be asked to provide feedback,\n",
    "and the guessing agent gets two more chances.\n",
    "If the correct number is guessed eventually, the conversation will be terminated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "I have a number between 1 and 100. Guess it!\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 50?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 75?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "It is too high my friend. \n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 60?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too high.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 55?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "still too high, but you are very close.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 52?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[31m\n",
      ">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Too low.\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 54?\n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
      "\n",
      "Almost there! \n",
      "\n",
      "--------------------------------------------------------------------------------\n",
      "\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
      "\n",
      "Is it 53?\n",
      "\n",
      "--------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "agent_with_number = ConversableAgent(\n",
    "    \"agent_with_number\",\n",
    "    system_message=\"You are playing a game of guess-my-number. \"\n",
    "    \"In the first game, you have the \"\n",
    "    \"number 53 in your mind, and I will try to guess it. \"\n",
    "    \"If I guess too high, say 'too high', if I guess too low, say 'too low'. \",\n",
    "    llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
    "    max_consecutive_auto_reply=1,  # maximum number of consecutive auto-replies before asking for human input\n",
    "    is_termination_msg=lambda msg: \"53\" in msg[\"content\"],  # terminate if the number is guessed by the other agent\n",
    "    human_input_mode=\"TERMINATE\",  # ask for human input until the game is terminated\n",
    ")\n",
    "\n",
    "agent_guess_number = ConversableAgent(\n",
    "    \"agent_guess_number\",\n",
    "    system_message=\"I have a number in my mind, and you will try to guess it. \"\n",
    "    \"If I say 'too high', you should guess a lower number. If I say 'too low', \"\n",
    "    \"you should guess a higher number. \",\n",
    "    llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
    "    human_input_mode=\"NEVER\",\n",
    ")\n",
    "\n",
    "result = agent_with_number.initiate_chat(\n",
    "    agent_guess_number,\n",
    "    message=\"I have a number between 1 and 100. Guess it!\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous conversation,\n",
    "\n",
    "1. When the agent guessed \"74\", the human said \"It is too high my friend.\"\n",
    "2. When the agent guessed \"55\", the human said \"still too high, but you are very close.\"\n",
    "3. When the agent guessed \"54\", the human said \"Almost there!\"\n",
    "\n",
    "Each time after one auto-reply from the agent with the number,\n",
    "the human was asked to provide feedback.\n",
    "Once the human provided feedback, the counter was reset.\n",
    "The conversation was terminated after the agent correctly guessed \"53\"."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary\n",
    "\n",
    "In this chapter, we showed you how to use the human-in-the-loop component\n",
    "to provide human feedback to agent and to terminate conversation.\n",
    "We also showed you the different human input modes and how they affect\n",
    "the behavior of the human-in-the-loop component.\n",
    "\n",
    "The next chapter will be all about code executor -- the most powerful\n",
    "component second only to LLMs."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "autogen",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}