mirror of
https://github.com/deepset-ai/haystack.git
synced 2026-01-01 01:27:28 +00:00
44 lines
2.1 KiB
Plaintext
44 lines
2.1 KiB
Plaintext
---
|
|
title: "Enabling GPU Acceleration"
|
|
id: enabling-gpu-acceleration
|
|
slug: "/enabling-gpu-acceleration"
|
|
description: "Speed up your Haystack application by engaging the GPU."
|
|
---
|
|
|
|
import ClickableImage from "@site/src/components/ClickableImage";
|
|
|
|
# Enabling GPU Acceleration
|
|
|
|
Speed up your Haystack application by engaging the GPU.
|
|
|
|
The Transformer models used in Haystack are designed to be run on GPU-accelerated hardware. The steps for GPU acceleration setup depend on the environment that you're working in.
|
|
|
|
Once you have GPU enabled on your machine, you can set the `device` on which a given model for a component is loaded.
|
|
|
|
For example, to load a model for the `HuggingFaceLocalGenerator`, set `device="ComponentDevice.from_single(Device.gpu(id=0))` or `device = ComponentDevice.from_str("cuda:0")` when initializing.
|
|
|
|
You can find more information on the [Device management](../concepts/device-management.mdx) page.
|
|
|
|
### Enabling the GPU in Linux
|
|
|
|
1. Ensure that you have a fitting version of NVIDIA CUDA installed. To learn how to install CUDA, see the [NVIDIA CUDA Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html).
|
|
|
|
2. Run the `nvidia-smi`in the command line to check if the GPU is enabled. If the GPU is enabled, the output shows a list of available GPUs and their memory usage:
|
|
<ClickableImage src="/img/b44c7f4-gpu_enabled_cropped.png" alt="A screenshot of the command output with the name of the GPU device and its memory usage highlighted." />
|
|
|
|
### Enabling the GPU in Colab
|
|
|
|
1. In your Colab environment, select **Runtime>Change Runtime type**.
|
|
<ClickableImage src="/img/85079c7-68747470733a2f2f7261772e67697468756275736572636f6e74656e742e636f6d2f646565707365742d61692f686179737461636b2f6d61696e2f646f63732f696d672f636f6c61625f6770755f72756e74696d652e6a7067.jpeg" alt="Google Colab Runtime menu with Change runtime type option highlighted for selecting GPU acceleration" size="large" />
|
|
|
|
2. Choose **Hardware accelerator>GPU**.
|
|
3. To check if the GPU is enabled, run:
|
|
|
|
```python python
|
|
%%bash
|
|
|
|
nvidia-smi
|
|
```
|
|
|
|
The output should show the GPUs available and their usage.
|