mirror of
https://github.com/microsoft/autogen.git
synced 2025-07-19 06:53:17 +00:00
475 lines
18 KiB
Plaintext
475 lines
18 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Allowing Human Feedback in Agents\n",
|
|
"\n",
|
|
"In the last two chapters we introduced the `ConversableAgent` class and showed how you can use it to create autonomous (`human_input_mode=NEVER`) agents that can accomplish tasks. We also showed how to properly terminate a conversation between agents.\n",
|
|
"\n",
|
|
"But many applications may require putting humans in-the-loop with agents. For example, to allow human feedback to steer agents in the right direction, specify goals, etc. In this chapter, we will show how AutoGen supports human intervention.\n",
|
|
"\n",
|
|
"In AutoGen's `ConversableAgent`, the human-in-the-loop component sits in front\n",
|
|
"of the auto-reply components. It can intercept the incoming messages and\n",
|
|
"decide whether to pass them to the auto-reply components or to provide\n",
|
|
"human feedback. The figure below illustrates the design.\n",
|
|
"\n",
|
|
"```{=mdx}\n",
|
|
"\n",
|
|
"```\n",
|
|
"\n",
|
|
"The human-in-the-loop component can be customized through the `human_input_mode` parameter.\n",
|
|
"We will show you how to use it in the following sections."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Human Input Modes\n",
|
|
"\n",
|
|
"Currently AutoGen supports three modes for human input. The mode is specified through\n",
|
|
"the `human_input_mode` argument of the `ConversableAgent`. The three modes are:\n",
|
|
"\n",
|
|
"1. `NEVER`: human input is never requested.\n",
|
|
"2. `TERMINATE` (default): human input is only requested when a termination condition is\n",
|
|
" met. Note that in this mode if the human chooses to intercept and reply, the conversation continues\n",
|
|
" and the counter used by `max_consecutive_auto_reply` is reset.\n",
|
|
"3. `ALWAYS`: human input is always requested and the human can choose to skip and trigger an auto-reply,\n",
|
|
" intercept and provide feedback, or terminate the conversation. Note that in this mode\n",
|
|
" termination based on `max_consecutive_auto_reply` is ignored.\n",
|
|
"\n",
|
|
"The previous chapters already showed many examples of the cases when `human_input_mode` is `NEVER`. \n",
|
|
"Below we show one such example again and then show the differences when this mode is set to `ALWAYS` and `NEVER` instead."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Human Input Mode = `NEVER`\n",
|
|
"\n",
|
|
"In this mode, human input is never requested and the termination conditions\n",
|
|
"are used to terminate.\n",
|
|
"This mode is useful when you want your agents to act fully autonomously.\n",
|
|
"\n",
|
|
"Here is an example of using this mode to run a simple guess-a-number game between\n",
|
|
"two agents, the termination message is set to check for the \n",
|
|
"number that is the correct guess."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 3,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"I have a number between 1 and 100. Guess it!\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 50?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"Too low.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 75?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"Too high.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 63?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"Too high.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 57?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"Too high.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 54?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"Too high.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 52?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"Too low.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 53?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"import os\n",
|
|
"\n",
|
|
"from autogen import ConversableAgent\n",
|
|
"\n",
|
|
"agent_with_number = ConversableAgent(\n",
|
|
" \"agent_with_number\",\n",
|
|
" system_message=\"You are playing a game of guess-my-number. You have the \"\n",
|
|
" \"number 53 in your mind, and I will try to guess it. \"\n",
|
|
" \"If I guess too high, say 'too high', if I guess too low, say 'too low'. \",\n",
|
|
" llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
|
|
" is_termination_msg=lambda msg: \"53\" in msg[\"content\"], # terminate if the number is guessed by the other agent\n",
|
|
" human_input_mode=\"NEVER\", # never ask for human input\n",
|
|
")\n",
|
|
"\n",
|
|
"agent_guess_number = ConversableAgent(\n",
|
|
" \"agent_guess_number\",\n",
|
|
" system_message=\"I have a number in my mind, and you will try to guess it. \"\n",
|
|
" \"If I say 'too high', you should guess a lower number. If I say 'too low', \"\n",
|
|
" \"you should guess a higher number. \",\n",
|
|
" llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
|
|
" human_input_mode=\"NEVER\",\n",
|
|
")\n",
|
|
"\n",
|
|
"result = agent_with_number.initiate_chat(\n",
|
|
" agent_guess_number,\n",
|
|
" message=\"I have a number between 1 and 100. Guess it!\",\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Yay! The game is over. The guessing agent got the number correctly \n",
|
|
"using binary search -- very efficient!\n",
|
|
"You can see that the conversation was terminated after the guessing agent\n",
|
|
"said the correct number, which triggered \n",
|
|
"the message-based termination condition."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Human Input Mode = `ALWAYS`\n",
|
|
"\n",
|
|
"In this mode, human input is always requested and the human can choose to skip,\n",
|
|
"intercept , or terminate the conversation.\n",
|
|
"Let us see this mode in action by playing the same game as before with the agent with the number, but this time\n",
|
|
"participating in the game as a human.\n",
|
|
"We will be the agent that is guessing the number, and play against the agent\n",
|
|
"with the number from before."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 4,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"10\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
|
|
"\n",
|
|
"Too low.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"79\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
|
|
"\n",
|
|
"Too high.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"76\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
|
|
"\n",
|
|
"Too high.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33mhuman_proxy\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"I give up\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to human_proxy):\n",
|
|
"\n",
|
|
"That's okay! The number I was thinking of was 53.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"human_proxy = ConversableAgent(\n",
|
|
" \"human_proxy\",\n",
|
|
" llm_config=False, # no LLM used for human proxy\n",
|
|
" human_input_mode=\"ALWAYS\", # always ask for human input\n",
|
|
")\n",
|
|
"\n",
|
|
"# Start a chat with the agent with number with an initial guess.\n",
|
|
"result = human_proxy.initiate_chat(\n",
|
|
" agent_with_number, # this is the same agent with the number as before\n",
|
|
" message=\"10\",\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"If you run the code above, you will be prompt to enter a response\n",
|
|
"each time it is your turn to speak. You can see the human in the conversation\n",
|
|
"was not very good at guessing the number... but hey the agent was nice enough\n",
|
|
"to give out the number in the end."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Human Input Mode = `TERMINATE`\n",
|
|
"\n",
|
|
"In this mode, human input is only requested when a termination condition is\n",
|
|
"met. **If the human chooses to intercept and reply, the counter will be reset**; if \n",
|
|
"the human chooses to skip, the automatic reply mechanism will be used; if the human\n",
|
|
"chooses to terminate, the conversation will be terminated.\n",
|
|
"\n",
|
|
"Let us see this mode in action by playing the same game again, but this time\n",
|
|
"the guessing agent will only have two chances to guess the number, and if it \n",
|
|
"fails, the human will be asked to provide feedback,\n",
|
|
"and the guessing agent gets two more chances.\n",
|
|
"If the correct number is guessed eventually, the conversation will be terminated."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": 10,
|
|
"metadata": {},
|
|
"outputs": [
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"I have a number between 1 and 100. Guess it!\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 50?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[31m\n",
|
|
">>>>>>>> USING AUTO REPLY...\u001b[0m\n"
|
|
]
|
|
},
|
|
{
|
|
"name": "stdout",
|
|
"output_type": "stream",
|
|
"text": [
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"Too low.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 75?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"It is too high my friend. \n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 60?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[31m\n",
|
|
">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"Too high.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 55?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"still too high, but you are very close.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 52?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[31m\n",
|
|
">>>>>>>> USING AUTO REPLY...\u001b[0m\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"Too low.\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 54?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_with_number\u001b[0m (to agent_guess_number):\n",
|
|
"\n",
|
|
"Almost there! \n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n",
|
|
"\u001b[33magent_guess_number\u001b[0m (to agent_with_number):\n",
|
|
"\n",
|
|
"Is it 53?\n",
|
|
"\n",
|
|
"--------------------------------------------------------------------------------\n"
|
|
]
|
|
}
|
|
],
|
|
"source": [
|
|
"agent_with_number = ConversableAgent(\n",
|
|
" \"agent_with_number\",\n",
|
|
" system_message=\"You are playing a game of guess-my-number. \"\n",
|
|
" \"In the first game, you have the \"\n",
|
|
" \"number 53 in your mind, and I will try to guess it. \"\n",
|
|
" \"If I guess too high, say 'too high', if I guess too low, say 'too low'. \",\n",
|
|
" llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
|
|
" max_consecutive_auto_reply=1, # maximum number of consecutive auto-replies before asking for human input\n",
|
|
" is_termination_msg=lambda msg: \"53\" in msg[\"content\"], # terminate if the number is guessed by the other agent\n",
|
|
" human_input_mode=\"TERMINATE\", # ask for human input until the game is terminated\n",
|
|
")\n",
|
|
"\n",
|
|
"agent_guess_number = ConversableAgent(\n",
|
|
" \"agent_guess_number\",\n",
|
|
" system_message=\"I have a number in my mind, and you will try to guess it. \"\n",
|
|
" \"If I say 'too high', you should guess a lower number. If I say 'too low', \"\n",
|
|
" \"you should guess a higher number. \",\n",
|
|
" llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n",
|
|
" human_input_mode=\"NEVER\",\n",
|
|
")\n",
|
|
"\n",
|
|
"result = agent_with_number.initiate_chat(\n",
|
|
" agent_guess_number,\n",
|
|
" message=\"I have a number between 1 and 100. Guess it!\",\n",
|
|
")"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In the previous conversation,\n",
|
|
"\n",
|
|
"1. When the agent guessed \"74\", the human said \"It is too high my friend.\"\n",
|
|
"2. When the agent guessed \"55\", the human said \"still too high, but you are very close.\"\n",
|
|
"3. When the agent guessed \"54\", the human said \"Almost there!\"\n",
|
|
"\n",
|
|
"Each time after one auto-reply from the agent with the number,\n",
|
|
"the human was asked to provide feedback.\n",
|
|
"Once the human provided feedback, the counter was reset.\n",
|
|
"The conversation was terminated after the agent correctly guessed \"53\"."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Summary\n",
|
|
"\n",
|
|
"In this chapter, we showed you how to use the human-in-the-loop component\n",
|
|
"to provide human feedback to agent and to terminate conversation.\n",
|
|
"We also showed you the different human input modes and how they affect\n",
|
|
"the behavior of the human-in-the-loop component.\n",
|
|
"\n",
|
|
"The next chapter will be all about code executor -- the most powerful\n",
|
|
"component second only to LLMs."
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "autogen",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.11.5"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|