mirror of
https://github.com/deepset-ai/haystack.git
synced 2026-01-08 13:06:29 +00:00
90 lines
3.7 KiB
Plaintext
90 lines
3.7 KiB
Plaintext
---
|
||
title: "AnthropicGenerator"
|
||
id: anthropicgenerator
|
||
slug: "/anthropicgenerator"
|
||
description: "This component enables text completions using Anthropic large language models (LLMs)."
|
||
---
|
||
|
||
# AnthropicGenerator
|
||
|
||
This component enables text completions using Anthropic large language models (LLMs).
|
||
|
||
<div className="key-value-table">
|
||
|
||
| | |
|
||
| --- | --- |
|
||
| **Most common position in a pipeline** | After a [PromptBuilder](../builders/promptbuilder.mdx) |
|
||
| **Mandatory init variables** | `api_key`: An Anthropic API key. Can be set with `ANTHROPIC_API_KEY` env var. |
|
||
| **Mandatory run variables** | `prompt`: A string containing the prompt for the LLM |
|
||
| **Output variables** | `replies`: A list of strings with all the replies generated by the LLM <br /> <br />`meta`: A list of dictionaries with the metadata associated with each reply, such as token count, finish reason, and so on |
|
||
| **API reference** | [Anthropic](/reference/integrations-anthropic) |
|
||
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/anthropic |
|
||
|
||
</div>
|
||
|
||
## Overview
|
||
|
||
This integration supports Anthropic models such as `claude-3-5-sonnet-20240620`,`claude-3-opus-20240229`, `claude-3-haiku-20240307`, and similar. Although these LLMs are called chat models, the main prompt interface works with the string prompts. Check out the most recent full list in the [Anthropic documentation](https://docs.anthropic.com/en/docs/about-claude/models).
|
||
|
||
### Parameters
|
||
|
||
`AnthropicGenerator` needs an Anthropic API key to work. You can provide this key in:
|
||
|
||
- The `ANTHROPIC_API_KEY` environment variable (recommended)
|
||
- The `api_key` init parameter and Haystack [Secret](../../concepts/secret-management.mdx) API: `Secret.from_token("your-api-key-here")`
|
||
|
||
Set your preferred Anthropic model in the `model` parameter when initializing the component.
|
||
|
||
`AnthropicGenerator` requires a prompt to generate text, but you can pass any text generation parameters available in the Anthropic [Messaging API](https://docs.anthropic.com/en/api/messages) method directly to this component using the `generation_kwargs` parameter, both at initialization and to `run()` method. For more details on the parameters supported by the Anthropic API, see [Anthropic documentation](https://docs.anthropic.com).
|
||
|
||
Finally, the component run method requires a single string prompt to generate text.
|
||
|
||
### Streaming
|
||
|
||
This Generator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly in output. To do so, pass a function to the `streaming_callback` init parameter.
|
||
|
||
## Usage
|
||
|
||
Install the `anthropic-haystack` package to use the `AnthropicGenerator`:
|
||
|
||
```shell
|
||
pip install anthropic-haystack
|
||
```
|
||
|
||
### On its own
|
||
|
||
```python
|
||
from haystack_integrations.components.generators.anthropic import AnthropicGenerator
|
||
|
||
generator = AnthropicGenerator()
|
||
print(generator.run("What's Natural Language Processing? Be brief."))
|
||
```
|
||
|
||
### In a pipeline
|
||
|
||
You can also use `AnthropicGenerator` with the Anthropic models in your pipeline.
|
||
|
||
```python
|
||
from haystack import Pipeline
|
||
from haystack.components.builders import PromptBuilder
|
||
from haystack_integrations.components.generators.anthropic import AnthropicGenerator
|
||
from haystack.utils import Secret
|
||
|
||
template = """
|
||
You are an assistant giving out valuable information to language learners.
|
||
Answer this question, be brief.
|
||
|
||
Question: {{ query }}?
|
||
"""
|
||
|
||
pipe = Pipeline()
|
||
pipe.add_component("prompt_builder", PromptBuilder(template))
|
||
pipe.add_component("llm", AnthropicGenerator(Secret.from_env_var("ANTHROPIC_API_KEY")))
|
||
pipe.connect("prompt_builder", "llm")
|
||
|
||
query = "What language is spoke in Germany?"
|
||
res = pipe.run(data={"prompt_builder": {"query": {query}}})
|
||
print(res)
|
||
|
||
```
|