yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-12-28 15:38:36 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs: Update Agent docstrings + add api docs (#4296)

Browse Source 
* Update docstrings + add api docs

* Update with reviewer's changes

* Fix category id and blackify

* make max iterations test more robust

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
This commit is contained in:
Agnieszka Marzec 2023-03-15 18:26:35 +02:00 committed by GitHub
parent d87b310f01
commit 374d7c9c4f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 87 additions and 49 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
docs/pydoc/config/agent.yml Normal file
View File

@ -0,0 +1,26 @@
loaders:
- type: python
search_path: [../../../haystack/agents/agent]
modules: ['agent']
ignore_when_discovered: ['__init__']
processors:
- type: filter
expression:
documented_only: true
do_not_filter_modules: false
skip_empty_modules: true
- type: smart
- type: crossref
renderer:
type: renderers.ReadmeRenderer
excerpt: Uses a large language model to answer complex queries that require multiple steps to find the correct answer.
category: haystack-classes
title: Agent API
slug: agent-api
order: 0
markdown:
descriptive_class_title: false
descriptive_module_title: true
add_method_class_prefix: true
add_member_class_prefix: false
filename: agent_api.md

106
haystack/agents/base.py
View File

@ -24,16 +24,18 @@ logger = logging.getLogger(__name__)
class Tool:
"""
A tool is a pipeline or node that also has a name and description. When you add a Tool to an Agent, the Agent can
invoke the underlying pipeline or node to answer questions. The wording of the description is important for the
Agent to decide which tool is most useful for a given question.
Agent uses tools to find the best answer. A tool is a pipeline or a node. When you add a tool to an Agent, the Agent can
invoke the underlying pipeline or node to answer questions.
You must provide a name and a description for each tool. The name should be short and should indicate what the tool can do. The description should explain what the tool is useful for. The Agent uses the description to decide when to use a tool, so the wording you use is important.
:param name: The name of the tool. The Agent uses this name to refer to the tool in the text the Agent generates.
The name should be short, ideally one token, and a good description of what the tool can do, for example
"Calculator" or "Search". Use only letters (a-z, A-Z), digits (0-9) and underscores (_).".
:param pipeline_or_node: The pipeline or node to run when this tool is invoked by an Agent.
:param description: A description of what the tool is useful for. An Agent can use this description for the decision
when to use which tool. For example, a tool for calculations can be described by "useful for when you need to
The name should be short, ideally one token, and a good description of what the tool can do, for example:
"Calculator" or "Search". Use only letters (a-z, A-Z), digits (0-9) and underscores (_)."
:param pipeline_or_node: The pipeline or node to run when the Agent invokes this tool.
:param description: A description of what the tool is useful for. The Agent uses this description to decide
when to use which tool. For example, you can describe a tool for calculations by "useful for when you need to
answer questions about math".
"""
@ -98,9 +100,12 @@ class Tool:
class Agent:
"""
An Agent answers queries by choosing between different tools, which are pipelines or nodes. It uses a large
language model (LLM) to generate a thought based on the query, choose a tool, and generate the input for the
tool. Based on the result returned by the tool the Agent can either stop if it now knows the answer or repeat the
An Agent answers queries using the tools you give to it. The tools are pipelines or nodes. The Agent uses a large
language model (LLM) through the PromptNode you initialize it with. To answer a query, the Agent follows this sequence:
1. It generates a thought based on the query.
2. It decides which tool to use.
3. It generates the input for the tool.
4. Based on the output it gets from the tool, the Agent can either stop if it now knows the answer or repeat the
process of 1) generate thought, 2) choose tool, 3) generate input.
Agents are useful for questions containing multiple subquestions that can be answered step-by-step (Multihop QA)
@ -120,12 +125,12 @@ class Agent:
Creates an Agent instance.
:param prompt_node: The PromptNode that the Agent uses to decide which tool to use and what input to provide to it in each iteration.
:param prompt_template: The name of a PromptTemplate supported by the PromptNode or a new PromptTemplate. It is used for generating thoughts and running Tools to answer queries step-by-step.
:param tools: A List of Tools that the Agent can choose to run. If no Tools are provided, they need to be added with `add_tool()` before you can use the Agent.
:param max_iterations: The number of times the Agent can run a tool plus then once infer it knows the final answer.
Set at least to 2 so that the Agent can run one Tool and then infer it knows the final answer. Default 5.
:param tool_pattern: A regular expression to extract the name of the Tool and the corresponding input from the text generated by the Agent.
:param final_answer_pattern: A regular expression to extract final answer from the text generated by the Agent.
:param prompt_template: The name of a PromptTemplate for the PromptNode. It's used for generating thoughts and choosing tools to answer queries step-by-step. You can use the default `zero-shot-react` template or create a new template in a similar format.
:param tools: A list of tools the Agent can run. If you don't specify any tools here, you must add them with `add_tool()` before running the Agent.
:param max_iterations: The number of times the Agent can run a tool +1 to let it infer it knows the final answer.
Set it to at least 2, so that the Agent can run one a tool once and then infer it knows the final answer. The default is 5.
:param tool_pattern: A regular expression to extract the name of the tool and the corresponding input from the text the Agent generated.
:param final_answer_pattern: A regular expression to extract the final answer from the text the Agent generated.
"""
self.prompt_node = prompt_node
self.prompt_template = (
@ -143,9 +148,16 @@ class Agent:
def add_tool(self, tool: Tool):
"""
Add the provided tool to the Agent and update the template for the Agent's PromptNode.
Add a tool to the Agent. This also updates the PromptTemplate for the Agent's PromptNode with the tool name.
:param tool: The Tool to add to the Agent. Any previously added tool with the same name will be overwritten.
:param tool: The tool to add to the Agent. Any previously added tool with the same name will be overwritten. Example:
`agent.add_tool(
Tool(
name="Calculator",
pipeline_or_node=calculator
description="Useful when you need to answer questions about math"
)
)
"""
self.tools[tool.name] = tool
self.tool_names = ", ".join(self.tools.keys())
@ -155,9 +167,9 @@ class Agent:
def has_tool(self, tool_name: str):
"""
Check whether the Agent has a Tool registered under the provided tool name.
Check whether the Agent has a tool with the name you provide.
:param tool_name: The name of the Tool for which to check whether the Agent has it.
:param tool_name: The name of the tool for which you want to check whether the Agent has it.
"""
return tool_name in self.tools
@ -166,26 +178,26 @@ class Agent:
) -> Dict[str, Union[str, List[Answer]]]:
"""
Runs the Agent given a query and optional parameters to pass on to the tools used. The result is in the
same format as a pipeline's result: a dictionary with a key `answers` containing a List of Answers
same format as a pipeline's result: a dictionary with a key `answers` containing a list of answers.
:param query: The search query.
:param max_iterations: The number of times the Agent can run a tool plus then once infer it knows the final answer.
If set it should be at least 2 so that the Agent can run one tool and then infer it knows the final answer.
:param params: A dictionary of parameters that you want to pass to those tools that are pipelines.
To pass a parameter to all nodes in those pipelines, use: `{"top_k": 10}`.
To pass a parameter to targeted nodes in those pipelines, use:
:param max_iterations: The number of times the Agent can run a tool +1 to infer it knows the final answer.
If you want to set it, make it at least 2 so that the Agent can run a tool once and then infer it knows the final answer.
:param params: A dictionary of parameters you want to pass to the tools that are pipelines.
To pass a parameter to all nodes in those pipelines, use the format: `{"top_k": 10}`.
To pass a parameter to targeted nodes in those pipelines, use the format:
`{"Retriever": {"top_k": 10}, "Reader": {"top_k": 3}}`.
Parameters can only be passed to tools that are pipelines but not nodes.
You can only pass parameters to tools that are pipelines, but not nodes.
"""
if not self.tools:
raise AgentError(
"Agents without tools cannot be run. Add at least one tool using `add_tool()` or set the parameter `tools` when initializing the Agent."
"An Agent needs tools to run. Add at least one tool using `add_tool()` or set the parameter `tools` when initializing the Agent."
)
if max_iterations is None:
max_iterations = self.max_iterations
if max_iterations < 2:
raise AgentError(
f"max_iterations was set to {max_iterations} but it should be at least 2 so that the Agent can run one tool and then infer it knows the final answer."
f"max_iterations must be at least 2 to let the Agent use a tool once and then infer it knows the final answer. It was set to {max_iterations}."
)
transcript = self._get_initial_transcript(query=query)
@ -194,7 +206,7 @@ class Agent:
for _ in range(max_iterations):
preds = self.prompt_node(transcript)
if not preds:
raise AgentError(f"No output was generated by the Agent. Transcript:\n{transcript}")
raise AgentError(f"The Agent generated no output. Transcript:\n{transcript}")
# Try to extract final answer or tool name and input from the generated text and update the transcript
final_answer = self._extract_final_answer(pred=preds[0])
@ -206,8 +218,8 @@ class Agent:
transcript += f"{preds[0]}\nObservation: {observation}\nThought:"
logger.warning(
"Maximum number of iterations (%s) reached for query (%s). Increase max_iterations "
"or no answer can be provided for this query.",
"The Agent reached the maximum number of iterations (%s) for query (%s). Increase the max_iterations parameter "
"or the Agent won't be able to provide an answer to this query.",
max_iterations,
query,
)
@ -217,16 +229,16 @@ class Agent:
self, queries: List[str], max_iterations: Optional[int] = None, params: Optional[dict] = None
) -> Dict[str, str]:
"""
Runs the Agent in a batch mode
Runs the Agent in a batch mode.
:param queries: List of search queries.
:param max_iterations: The number of times the Agent can run a tool plus then once infer it knows the final answer.
If set it should be at least 2 so that the Agent can run one tool and then infer it knows the final answer.
:param params: A dictionary of parameters that you want to pass to those tools that are pipelines.
To pass a parameter to all nodes in those pipelines, use: `{"top_k": 10}`.
To pass a parameter to targeted nodes in those pipelines, use:
:param max_iterations: The number of times the Agent can run a tool +1 to infer it knows the final answer.
If you want to set it, make it at least 2 so that the Agent can run a tool once and then infer it knows the final answer.
:param params: A dictionary of parameters you want to pass to the tools that are pipelines.
To pass a parameter to all nodes in those pipelines, use the format: `{"top_k": 10}`.
To pass a parameter to targeted nodes in those pipelines, use the format:
`{"Retriever": {"top_k": 10}, "Reader": {"top_k": 3}}`.
Parameters can only be passed to tools that are pipelines but not nodes.
You can only pass parameters to tools that are pipelines but not nodes.
"""
results: Dict = {"queries": [], "answers": [], "transcripts": []}
for query in queries:
@ -249,7 +261,7 @@ class Agent:
)
if not self.has_tool(tool_name):
raise AgentError(
f"Cannot use the tool {tool_name} because it is not in the list of added tools {self.tools.keys()}."
f"The tool {tool_name} wasn't added to the Agent tools: {self.tools.keys()}."
"Add the tool using `add_tool()` or include it in the parameter `tools` when initializing the Agent."
f"Transcript:\n{transcript}"
)
@ -259,7 +271,7 @@ class Agent:
"""
Parse the tool name and the tool input from the prediction output of the Agent's PromptNode.
:param pred: Prediction output of the Agent's PromptNode from which to parse the tool and tool input
:param pred: Prediction output of the Agent's PromptNode from which to parse the tool and tool input.
"""
tool_match = re.search(self.tool_pattern, pred)
if tool_match:
@ -272,7 +284,7 @@ class Agent:
"""
Parse the final answer from the prediction output of the Agent's PromptNode.
:param pred: Prediction output of the Agent's PromptNode from which to parse the final answer
:param pred: Prediction output of the Agent's PromptNode from which to parse the final answer.
"""
final_answer_match = re.search(self.final_answer_pattern, pred)
if final_answer_match:
@ -282,18 +294,18 @@ class Agent:
def _format_answer(self, query: str, answer: str, transcript: str) -> Dict[str, Union[str, List[Answer]]]:
"""
Formats an answer as a dict containing `query` and `answers` similar to the output of a Pipeline.
Formats an answer as a dict containing `query` and `answers`, similar to the output of a Pipeline.
The full transcript based on the Agent's initial prompt template and the text it generated during execution.
:param query: The search query.
:param answer: The final answer returned by the Agent. An empty string corresponds to no answer.
:param transcript: The text generated by the Agent and the initial filled template for debug purposes.
:param answer: The final answer the Agent returned. An empty string corresponds to no answer.
:param transcript: The text the Agent generated and the initial, filled template for debug purposes.
"""
return {"query": query, "answers": [Answer(answer=answer, type="generative")], "transcript": transcript}
def _get_initial_transcript(self, query: str):
"""
Fills the Agent's PromptTemplate with the query, tool names and descriptions
Fills the Agent's PromptTemplate with the query, tool names, and descriptions.
:param query: The search query.
"""

4
test/agents/test_agent.py
View File

@ -94,13 +94,13 @@ def test_max_iterations(caplog, monkeypatch):
with caplog.at_level(logging.WARN, logger="haystack.agents"):
result = agent.run("Where does Christelle live?")
assert result["answers"] == [Answer(answer="", type="generative")]
assert "Maximum number of iterations (3) reached" in caplog.text
assert "maximum number of iterations (3)" in caplog.text.lower()
# Setting max_iterations in the Agent's run method
with caplog.at_level(logging.WARN, logger="haystack.agents"):
result = agent.run("Where does Christelle live?", max_iterations=2)
assert result["answers"] == [Answer(answer="", type="generative")]
assert "Maximum number of iterations (2) reached" in caplog.text
assert "maximum number of iterations (2)" in caplog.text.lower()
@pytest.mark.unit