---
title: "STACKITChatGenerator"
id: stackitchatgenerator
slug: "/stackitchatgenerator"
description: "This component enables chat completions using the STACKIT API."
---

# STACKITChatGenerator

This component enables chat completions using the STACKIT API.

|  |  |
| --- | --- |
| **Most common position in a pipeline** | After a [`ChatPromptBuilder`](../builders/chatpromptbuilder.mdx) |
| **Mandatory init variables** | "model": The model used through the STACKIT API |
| **Mandatory run variables** | “messages”: A list of [`ChatMessage`](/docs/chatmessage)  objects |
| **Output variables** | "replies": A list of [`ChatMessage`](/docs/chatmessage) objects  <br /> <br />”meta”: A list of dictionaries with the metadata associated with each reply (such as token count, finish reason, and so on) |
| **API reference** | [STACKIT](/reference/integrations-stackit) |
| **GitHub link** | https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit |

## Overview

`STACKITChatGenerator` enables text generation models served by STACKIT through their API.

### Parameters

To use the `STACKITChatGenerator`, ensure you have set a `STACKIT_API_KEY` as an environment variable. Alternatively, provide the API key as another environment variable or a token by setting
`api_key` and using Haystack’s [secret management](/docs/secret-management).

Set your preferred supported model with the `model` parameter when initializing the component. See the full list of all supported models on the [STACKIT website](https://docs.stackit.cloud/stackit/en/models-licenses-319914532.html).

Optionally, you can change the default `api_base_url`, which is `"https://api.openai-compat.model-serving.eu01.onstackit.cloud/v1"`.

You can pass any text generation parameters valid for the STACKIT Chat Completion API directly to this component with the `generation_kwargs` parameter in the init or run methods.

The component needs a list of `ChatMessage` objects to run. `ChatMessage` is a data class that contains a message, a role (who generated the message, such as `user`, `assistant`, `system`, `function`), and optional metadata. Find out more about it [ChatMessage documentation](/docs/chatmessage).

### Streaming

This ChatGenerator supports [streaming](/docs/choosing-the-right-generator#streaming-support) the tokens from the LLM directly into the output. To do so, pass a function to the `streaming_callback` init parameter.

## Usage

Install the `stackit-haystack` package to use the `STACKITChatGenerator`:

```shell
pip install stackit-haystack
```

### On its own

```python
from haystack_integrations.components.generators.stackit import STACKITChatGenerator
from haystack.dataclasses import ChatMessage

generator = STACKITChatGenerator(model="neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8")

result = generator.run([ChatMessage.from_user("Tell me a joke.")])
print(result)
```

### In a pipeline

You can also use `STACKITChatGenerator` in your pipeline.

```python
from haystack import Pipeline
from haystack.components.builders import ChatPromptBuilder
from haystack.dataclasses import ChatMessage

from haystack_integrations.components.generators.stackit import STACKITChatGenerator

prompt_builder = ChatPromptBuilder()
llm = STACKITChatGenerator(model="neuralmagic/Meta-Llama-3.1-70B-Instruct-FP8")

messages = [ChatMessage.from_user("Question: {{question}} \\n")]

pipeline = Pipeline()
pipeline.add_component("prompt_builder", prompt_builder)
pipeline.add_component("llm", llm)

pipeline.connect("prompt_builder.prompt", "llm.messages")

result = pipeline.run({"prompt_builder": {"template_variables": {"question": "Tell me a joke."}, "template": messages}})

print(result)
```

For an example of streaming in a pipeline, refer to the examples in the STACKIT integration [repository](https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/stackit/examples) and on its dedicated [integration page](https://haystack.deepset.ai/integrations/stackit).