---
title: "STACKITChatGenerator"
id: stackitchatgenerator
slug: "/stackitchatgenerator"
description: "This component enables chat completions using the STACKIT API."
---

# STACKITChatGenerator

This component enables chat completions using the STACKIT API.

<div className="key-value-table">

|  |  |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | `model`: The model used through the STACKIT API |
| **Mandatory run variables** | `messages`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx)  objects |
| **Output variables** | `replies`: A list of [`ChatMessage`](../../concepts/data-classes/chatmessage.mdx) objects  <br /> <br />`meta`: A list of dictionaries with the metadata associated with each reply (such as token count, finish reason, and so on) |
| **API reference** | [STACKIT](/reference/integrations-stackit) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit |

</div>

## Overview

`STACKITChatGenerator` enables text generation models served by STACKIT through their API.

### Parameters

To use the `STACKITChatGenerator`, ensure you have set a `STACKIT_API_KEY` as an environment variable. Alternatively, provide the API key as another environment variable or a token by setting
`api_key` and using Haystack’s [secret management](../../concepts/secret-management.mdx).

Set your preferred supported model with the `model` parameter when initializing the component. See the full list of all supported models on the [STACKIT website](https://docs.stackit.cloud/stackit/en/models-licenses-319914532.html).

Optionally, you can change the default `api_base_url`, which is `"https://api.openai-compat.model-serving.eu01.onstackit.cloud/v1"`.

You can pass any text generation parameters valid for the STACKIT Chat Completion API directly to this component with the `generation_kwargs` parameter in the init or run methods.

The component needs a list of `ChatMessage` objects to run. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata. Find out more about it [ChatMessage documentation](../../concepts/data-classes/chatmessage.mdx).

### Streaming

This ChatGenerator supports [streaming](guides-to-generators/choosing-the-right-generator.mdx#streaming-support) the tokens from the LLM directly into the output. To do so, pass a function to the `streaming_callback` init parameter.

## Usage

Install the `stackit-haystack` package to use the `STACKITChatGenerator`:

```shell
pip install stackit-haystack
```

### On its own

```python
from haystack_integrations.components.generators.stackit import STACKITChatGenerator
from haystack.dataclasses import ChatMessage

generator = STACKITChatGenerator(model="neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8")

result = generator.run([ChatMessage.from_user("Tell me a joke.")])
print(result)
```

With multimodal inputs:

```python
from haystack.dataclasses import ChatMessage, ImageContent
from haystack_integrations.components.generators.stackit import STACKITChatGenerator

llm = STACKITChatGenerator(model="meta-llama/Llama-3.2-11B-Vision-Instruct")

image = ImageContent.from_file_path("apple.jpg")
user_message = ChatMessage.from_user(content_parts=[
	"What does the image show? Max 5 words.",
	image
	])

response = llm.run([user_message])["replies"][0].text
print(response)

# Red apple on straw.
```

### In a pipeline

You can also use `STACKITChatGenerator` in your pipeline.

```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage

from haystack_integrations.components.generators.stackit import STACKITChatGenerator

prompt_builder = ChatPromptBuilder()
llm = STACKITChatGenerator(model="neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8")

messages = [ChatMessage.from_user("Question: {{question}} \\n")]

pipeline = Pipeline()
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("llm", llm)

pipeline.connect("prompt_builder.prompt", "llm.messages")

result = pipeline.run({"prompt_builder": {"template_variables": {"question": "Tell me a joke."}, "template": messages}})

print(result)
```

For an example of streaming in a pipeline, refer to the examples in the STACKIT integration [repository](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit/examples) and on its dedicated [integration page](https://haystack.deepset.ai/integrations/stackit).