mirror of
https://github.com/deepset-ai/haystack.git
synced 2025-10-10 23:48:14 +00:00
da2e8da561
* Added support for Multi-GPU inference to DPR including benchmark * fixed multi gpu * added batch size to benchmark to better reflect multi gpu capabilities * remove unnecessary entry in config.json * fixed typos * fixed config name * update benchmark to use DEVICES constant * changed multi gpu parameters and updated docstring * adds silent fallback on cpu * update doc string, warning and config Co-authored-by: Michel Bartels <kontakt@michelbartels.com> Co-authored-by: Malte Pietsch <malte.pietsch@deepset.ai>
Benchmarks
Run the benchmarks with the following command:
python run.py [--reader] [--retriever_index] [--retriever_query] [--ci] [--update-json]
You can specify which components and processes to benchmark with the following flags.
--reader will trigger the speed and accuracy benchmarks for the reader. Here we simply use the SQuAD dev set.
--retriever_index will trigger indexing benchmarks
--retriever_query will trigger querying benchmarks (embeddings will be loaded from file instead of being computed on the fly)
--ci will cause the the benchmarks to run on a smaller slice of each dataset and a smaller subset of Retriever / Reader / DocStores.
--update-json will cause the script to update the json files in docs/_src/benchmarks so that the website benchmarks will be updated.