From ecc459f8c5e01c8243db4f49cd25e79eb105fcfb Mon Sep 17 00:00:00 2001 From: Eric Zhu Date: Wed, 20 Mar 2024 12:31:58 -0700 Subject: [PATCH] Add tool use chapter to tutorial (#2079) * Add tool use chapter * fix format * typo * Edits to tool-use chapter * update --------- Co-authored-by: Jack Gerrits --- website/docs/topics/task_decomposition.ipynb | 2 + website/docs/tutorial/code-executors.ipynb | 4 +- website/docs/tutorial/tool-use.ipynb | 756 +++++++++++++++++++ website/docs/tutorial/what-next.md | 5 +- website/sidebars.js | 5 + 5 files changed, 769 insertions(+), 3 deletions(-) create mode 100644 website/docs/tutorial/tool-use.ipynb diff --git a/website/docs/topics/task_decomposition.ipynb b/website/docs/topics/task_decomposition.ipynb index 23c5dee91..d6278c9c7 100644 --- a/website/docs/topics/task_decomposition.ipynb +++ b/website/docs/topics/task_decomposition.ipynb @@ -110,6 +110,8 @@ ")\n", "\n", "# register the function task_decomposition for execution\n", + "\n", + "\n", "@user_proxy.register_for_execution()\n", "@assistant.register_for_llm(\n", " name=\"task_decomposition\",\n", diff --git a/website/docs/tutorial/code-executors.ipynb b/website/docs/tutorial/code-executors.ipynb index 774ed20a3..6a8d43987 100644 --- a/website/docs/tutorial/code-executors.ipynb +++ b/website/docs/tutorial/code-executors.ipynb @@ -806,8 +806,8 @@ "\n", "In this chapter, we introduced code executors, how to set up Docker and local\n", "execution, and how to use code execution in a conversation to solve tasks.\n", - "In the next chapter, we will introduce conversation patterns, and show\n", - "how to use the result of a conversation." + "In the next chapter, we will introduce tool use, which is similar to code\n", + "executors but restricts what code agent can execute." ] } ], diff --git a/website/docs/tutorial/tool-use.ipynb b/website/docs/tutorial/tool-use.ipynb new file mode 100644 index 000000000..bcc161d5a --- /dev/null +++ b/website/docs/tutorial/tool-use.ipynb @@ -0,0 +1,756 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Tool Use\n", + "\n", + "In the previous chapter, we explored code executors which give\n", + "agents the super power of programming.\n", + "Agents writing arbitrary code is useful, however,\n", + "controlling what code an agent writes can be challenging. \n", + "This is where tools come in.\n", + "\n", + "Tools are pre-defined functions that agents can use. Instead of writing arbitrary\n", + "code, agents can call tools to perform actions, such as searching the web,\n", + "performing calculations, reading files, or calling remote APIs.\n", + "Because you can control what tools are available to an agent, you can control what\n", + "actions an agent can perform.\n", + "\n", + "````mdx-code-block\n", + ":::note\n", + "Tool use is currently only available for LLMs\n", + "that support OpenAI-comaptible tool call API.\n", + ":::\n", + "````" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Creating Tools\n", + "\n", + "Tools can be created as regular Python functions.\n", + "For example, let's create a calculator tool which\n", + "can only perform a single operation at a time." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "from typing import Annotated, Literal\n", + "\n", + "Operator = Literal[\"+\", \"-\", \"*\", \"/\"]\n", + "\n", + "\n", + "def calculator(a: int, b: int, operator: Annotated[Operator, \"operator\"]) -> int:\n", + " if operator == \"+\":\n", + " return a + b\n", + " elif operator == \"-\":\n", + " return a - b\n", + " elif operator == \"*\":\n", + " return a * b\n", + " elif operator == \"/\":\n", + " return int(a / b)\n", + " else:\n", + " raise ValueError(\"Invalid operator\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The above function takes three arguments:\n", + "`a` and `b` are the integer numbers to be operated on;\n", + "`operator` is the operation to be performed.\n", + "We used type hints to define the types of the arguments and the return value.\n", + "\n", + "````mdx-code-block\n", + ":::tip\n", + "Always use type hints to define the types of the arguments and the return value\n", + "as they provide helpful hints to the agent about the tool's usage.\n", + ":::\n", + "````" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Registering Tools\n", + "\n", + "Once you have created a tool, you can register it with the agents that\n", + "are involved in conversation." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "from autogen import ConversableAgent\n", + "\n", + "# Let's first define the assistant agent that suggests tool calls.\n", + "assistant = ConversableAgent(\n", + " name=\"Assistant\",\n", + " system_message=\"You are a helpful AI assistant. \"\n", + " \"You can help with simple calculations. \"\n", + " \"Return 'TERMINATE' when the task is done.\",\n", + " llm_config={\"config_list\": [{\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]}]},\n", + ")\n", + "\n", + "# The user proxy agent is used for interacting with the assistant agent\n", + "# and executes tool calls.\n", + "user_proxy = ConversableAgent(\n", + " name=\"User\",\n", + " llm_config=False,\n", + " is_termination_msg=lambda msg: msg.get(\"content\") is not None and \"TERMINATE\" in msg[\"content\"],\n", + " human_input_mode=\"NEVER\",\n", + ")\n", + "\n", + "# Register the tool signature with the assistant agent.\n", + "assistant.register_for_llm(name=\"calculator\", description=\"A simple calculator\")(calculator)\n", + "\n", + "# Register the tool function with the user proxy agent.\n", + "user_proxy.register_for_execution(name=\"calculator\")(calculator)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the above code, we registered the `calculator` function as a tool with\n", + "the assistant and user proxy agents. We also provide a name and a description\n", + "for the tool for the assistant agent to understand its usage.\n", + "\n", + "````mdx-code-block\n", + ":::tip\n", + "Always provide a clear and concise description for the tool as it helps the\n", + "agent's underlying LLM to understand the tool's usage.\n", + ":::\n", + "````\n", + "\n", + "Similar to code executors, a tool must be registered with at least two agents\n", + "for it to be useful in conversation. \n", + "The agent registered with the tool's signature\n", + "through \n", + "[`register_for_llm`](/docs/reference/agentchat/conversable_agent#register_for_llm)\n", + "can call the tool;\n", + "the agent registered with the tool's function object through \n", + "[`register_for_execution`](/docs/reference/agentchat/conversable_agent#register_for_execution)\n", + "can execute the tool's function." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Alternatively, you can use \n", + "[`autogen.register_function`](/docs/reference/agentchat/conversable_agent#register_function-1)\n", + "function to register a tool with both agents at once." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [], + "source": [ + "from autogen import register_function\n", + "\n", + "# Register the calculator function to the two agents.\n", + "register_function(\n", + " calculator,\n", + " caller=assistant, # The assistant agent can suggest calls to the calculator.\n", + " executor=user_proxy, # The user proxy agent can execute the calculator calls.\n", + " name=\"calculator\", # By default, the function name is used as the tool name.\n", + " description=\"A simple calculator\", # A description of the tool.\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Using Tool\n", + "\n", + "Once the tool is registered, we can use it in conversation.\n", + "In the code below, we ask the assistant to perform some arithmetic\n", + "calculation using the `calculator` tool." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "What is (44232 + 13312 / (232 - 32)) * 5?\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "\u001b[32m***** Suggested tool Call (call_bACquf0OreI0VHh7rWiP6ZE7): calculator *****\u001b[0m\n", + "Arguments: \n", + "{\n", + " \"a\": 13312,\n", + " \"b\": 232 - 32,\n", + " \"operator\": \"/\"\n", + "}\n", + "\u001b[32m***************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[32m***** Response from calling tool \"call_bACquf0OreI0VHh7rWiP6ZE7\" *****\u001b[0m\n", + "Error: Expecting ',' delimiter: line 1 column 26 (char 25)\n", + " You argument should follow json format.\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "\u001b[32m***** Suggested tool Call (call_2c0H5gzX9SWsJ05x7nEOVbav): calculator *****\u001b[0m\n", + "Arguments: \n", + "{\n", + " \"a\": 13312,\n", + " \"b\": 200,\n", + " \"operator\": \"/\"\n", + "}\n", + "\u001b[32m***************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[32m***** Response from calling tool \"call_2c0H5gzX9SWsJ05x7nEOVbav\" *****\u001b[0m\n", + "66\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "\u001b[32m***** Suggested tool Call (call_ioceLhuKMpfU131E7TSQ8wCD): calculator *****\u001b[0m\n", + "Arguments: \n", + "{\n", + " \"a\": 44232,\n", + " \"b\": 66,\n", + " \"operator\": \"+\"\n", + "}\n", + "\u001b[32m***************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[32m***** Response from calling tool \"call_ioceLhuKMpfU131E7TSQ8wCD\" *****\u001b[0m\n", + "44298\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "\u001b[32m***** Suggested tool Call (call_0rhx9vrbigcbqLssKLh4sS7j): calculator *****\u001b[0m\n", + "Arguments: \n", + "{\n", + " \"a\": 44298,\n", + " \"b\": 5,\n", + " \"operator\": \"*\"\n", + "}\n", + "\u001b[32m***************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[32m***** Response from calling tool \"call_0rhx9vrbigcbqLssKLh4sS7j\" *****\u001b[0m\n", + "221490\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "The result of the calculation (44232 + 13312 / (232 - 32)) * 5 is 221490. \n", + "\n", + "TERMINATE\n", + "\n", + "--------------------------------------------------------------------------------\n" + ] + } + ], + "source": [ + "chat_result = user_proxy.initiate_chat(assistant, message=\"What is (44232 + 13312 / (232 - 32)) * 5?\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's verify the answer:" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "221490" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "(44232 + int(13312 / (232 - 32))) * 5" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The answer is correct.\n", + "You can see that the assistant is able to understand the tool's usage\n", + "and perform calculation correctly." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Tool Schema\n", + "\n", + "If you are familiar with [OpenAI's tool use API](https://platform.openai.com/docs/guides/function-calling), \n", + "you might be wondering\n", + "why we didn't create a tool schema.\n", + "In fact, the tool schema is automatically generated from the function signature\n", + "and the type hints.\n", + "You can see the tool schema by inspecting the `llm_config` attribute of the\n", + "agent." + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'type': 'function',\n", + " 'function': {'description': 'A simple calculator',\n", + " 'name': 'calculator',\n", + " 'parameters': {'type': 'object',\n", + " 'properties': {'a': {'type': 'integer', 'description': 'a'},\n", + " 'b': {'type': 'integer', 'description': 'b'},\n", + " 'operator': {'enum': ['+', '-', '*', '/'],\n", + " 'type': 'string',\n", + " 'description': 'operator'}},\n", + " 'required': ['a', 'b', 'operator']}}}]" + ] + }, + "execution_count": 6, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "assistant.llm_config[\"tools\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can see the tool schema has been automatically generated from the function\n", + "signature and the type hints, as well as the description.\n", + "This is why it is important to use type hints and provide a clear description\n", + "for the tool as the LLM uses them to understand the tool's usage." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can also use Pydantic model for the type hints to provide more complex\n", + "type schema. In the example below, we use a Pydantic model to define the\n", + "calculator input." + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "metadata": {}, + "outputs": [], + "source": [ + "from pydantic import BaseModel, Field\n", + "\n", + "\n", + "class CalculatorInput(BaseModel):\n", + " a: Annotated[int, Field(description=\"The first number.\")]\n", + " b: Annotated[int, Field(description=\"The second number.\")]\n", + " operator: Annotated[Operator, Field(description=\"The operator.\")]\n", + "\n", + "\n", + "def calculator(input: Annotated[CalculatorInput, \"Input to the calculator.\"]) -> int:\n", + " if input.operator == \"+\":\n", + " return input.a + input.b\n", + " elif input.operator == \"-\":\n", + " return input.a - input.b\n", + " elif input.operator == \"*\":\n", + " return input.a * input.b\n", + " elif input.operator == \"/\":\n", + " return int(input.a / input.b)\n", + " else:\n", + " raise ValueError(\"Invalid operator\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Same as before, we register the tool with the agents using the name `\"calculator\"`.\n", + "\n", + "````mdx-code-block\n", + ":::tip\n", + "Registering tool to the same name will override the previous tool.\n", + ":::\n", + "````" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "assistant.register_for_llm(name=\"calculator\", description=\"A calculator tool that accepts nested expression as input\")(\n", + " calculator\n", + ")\n", + "user_proxy.register_for_execution(name=\"calculator\")(calculator)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You can see the tool schema has been updated to reflect the new type schema." + ] + }, + { + "cell_type": "code", + "execution_count": 28, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'type': 'function',\n", + " 'function': {'description': 'A calculator tool that accepts nested expression as input',\n", + " 'name': 'calculator',\n", + " 'parameters': {'type': 'object',\n", + " 'properties': {'input': {'properties': {'a': {'description': 'The first number.',\n", + " 'title': 'A',\n", + " 'type': 'integer'},\n", + " 'b': {'description': 'The second number.',\n", + " 'title': 'B',\n", + " 'type': 'integer'},\n", + " 'operator': {'description': 'The operator.',\n", + " 'enum': ['+', '-', '*', '/'],\n", + " 'title': 'Operator',\n", + " 'type': 'string'}},\n", + " 'required': ['a', 'b', 'operator'],\n", + " 'title': 'CalculatorInput',\n", + " 'type': 'object',\n", + " 'description': 'Input to the calculator.'}},\n", + " 'required': ['input']}}}]" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "assistant.llm_config[\"tools\"]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's use the tool in conversation." + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "What is (1423 - 123) / 3 + (32 + 23) * 5?\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "\u001b[32m***** Suggested tool Call (call_t9By3vewGRoSLWsvdTR7p8Zo): calculator *****\u001b[0m\n", + "Arguments: \n", + "{\n", + " \"input\": {\n", + " \"a\": 1423,\n", + " \"b\": 123,\n", + " \"operator\": \"-\"\n", + " }\n", + "}\n", + "\u001b[32m***************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[32m***** Response from calling tool \"call_t9By3vewGRoSLWsvdTR7p8Zo\" *****\u001b[0m\n", + "1300\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "\u001b[32m***** Suggested tool Call (call_rhecyhVCo0Y8HPL193xOUPE6): calculator *****\u001b[0m\n", + "Arguments: \n", + "{\n", + " \"input\": {\n", + " \"a\": 1300,\n", + " \"b\": 3,\n", + " \"operator\": \"/\"\n", + " }\n", + "}\n", + "\u001b[32m***************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[32m***** Response from calling tool \"call_rhecyhVCo0Y8HPL193xOUPE6\" *****\u001b[0m\n", + "433\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "\u001b[32m***** Suggested tool Call (call_zDpq9J5MYAsL7uS8cobOwa7S): calculator *****\u001b[0m\n", + "Arguments: \n", + "{\n", + " \"input\": {\n", + " \"a\": 32,\n", + " \"b\": 23,\n", + " \"operator\": \"+\"\n", + " }\n", + "}\n", + "\u001b[32m***************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[32m***** Response from calling tool \"call_zDpq9J5MYAsL7uS8cobOwa7S\" *****\u001b[0m\n", + "55\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "\u001b[32m***** Suggested tool Call (call_mjDuVMojOIdaxmvDUIF4QtVi): calculator *****\u001b[0m\n", + "Arguments: \n", + "{\n", + " \"input\": {\n", + " \"a\": 55,\n", + " \"b\": 5,\n", + " \"operator\": \"*\"\n", + " }\n", + "}\n", + "\u001b[32m***************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[32m***** Response from calling tool \"call_mjDuVMojOIdaxmvDUIF4QtVi\" *****\u001b[0m\n", + "275\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "\u001b[32m***** Suggested tool Call (call_hpirkAGKOewZstsDOxL2sYNW): calculator *****\u001b[0m\n", + "Arguments: \n", + "{\n", + " \"input\": {\n", + " \"a\": 433,\n", + " \"b\": 275,\n", + " \"operator\": \"+\"\n", + " }\n", + "}\n", + "\u001b[32m***************************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[35m\n", + ">>>>>>>> EXECUTING FUNCTION calculator...\u001b[0m\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[33mUser\u001b[0m (to Assistant):\n", + "\n", + "\u001b[32m***** Response from calling tool \"call_hpirkAGKOewZstsDOxL2sYNW\" *****\u001b[0m\n", + "708\n", + "\u001b[32m**********************************************************************\u001b[0m\n", + "\n", + "--------------------------------------------------------------------------------\n", + "\u001b[31m\n", + ">>>>>>>> USING AUTO REPLY...\u001b[0m\n", + "\u001b[33mAssistant\u001b[0m (to User):\n", + "\n", + "The result of the calculation is 708. \n", + "\n", + "TERMINATE\n", + "\n", + "--------------------------------------------------------------------------------\n" + ] + } + ], + "source": [ + "chat_result = user_proxy.initiate_chat(assistant, message=\"What is (1423 - 123) / 3 + (32 + 23) * 5?\")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's verify the answer:" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "708" + ] + }, + "execution_count": 31, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "int((1423 - 123) / 3) + (32 + 23) * 5" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Again, the answer is correct. You can see that the assistant is able to understand\n", + "the new tool schema and perform calculation correctly." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Summary\n", + "\n", + "In this chapter, we showed you how to create, register and use tools.\n", + "Tools allows agents to perform actions without writing arbitrary code.\n", + "In the next chapter, we will introduce conversation patterns, and show\n", + "how to use the result of a conversation." + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "autogen", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.5" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/website/docs/tutorial/what-next.md b/website/docs/tutorial/what-next.md index 7e3820cad..74a28364a 100644 --- a/website/docs/tutorial/what-next.md +++ b/website/docs/tutorial/what-next.md @@ -6,7 +6,9 @@ topics: 1. **Chat with LLMs**: In [Human in the Loop](./human-in-the-loop) we covered the basic human-in-the-loop usage. You can try to hook up different LLMs - using proxy servers like [Ollama](https://github.com/ollama/ollama), and + using local model servers like + [Ollama](https://github.com/ollama/ollama) + and [LM Studio](https://lmstudio.ai/), and chat with them using the human-in-the-loop component of your human proxy agent. 2. **Prompt Engineering**: In [Code Executors](./code-executors) we @@ -18,6 +20,7 @@ topics: 3. **Complex Tasks**: In [ConversationPatterns](./conversation-patterns) we covered the basic conversation patterns. You can try to find other tasks that can be decomposed into these patterns, and leverage the code executors + and tools to make the agents more powerful. ## Dig Deeper diff --git a/website/sidebars.js b/website/sidebars.js index 1abe16d68..44829e607 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -46,6 +46,11 @@ id: 'tutorial/code-executors', label: 'Code Executors', }, + { + type: 'doc', + id: 'tutorial/tool-use', + label: 'Tool Use', + }, { type: 'doc', id: 'tutorial/conversation-patterns',