mirror of
https://github.com/deepset-ai/haystack.git
synced 2025-12-30 00:30:09 +00:00
This PR changes the instance type of the public Haystack demo from p3.2xlarge to g4dn.2xlarge. g4dn.2xlarge has 1 GPU, 8 vCPUs, 32 GiB of memory p3.2xlarge had 1 GPU, 8 vCPUs, 61 GiB of memory which results in 75% lower costs with g4dn.2xlarge. I also tried out the even smaller g4dn.xlarge, which has 1 GPU, 4 vCPUs, 16 GiB of memory. However, the memory was not enough to run the demo. I tried out multiple requests at the same time and it worked well with g4dn.2xlarge. Requests are slightly slower as with the more powerful instance type but it's hard to notice.