yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-10-16 10:28:55 +00:00
Code Issues Packages Projects Releases Wiki Activity

add readme

Browse Source
This commit is contained in:
Branden Chan 2020-10-22 15:32:56 +02:00
parent 87e5f06fa8
commit b0483cfd99
1 changed files with 20 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
test/benchmarks/README.md Normal file
View File

@ -0,0 +1,20 @@
# Benchmarks
Run the benchmarks with the following command:
```
python run.py [--reader] [--retriever_index] [--retriever_query] [--ci] [--update-json]
```
You can specify which components and processes to benchmark with the following flags.
**--reader** will trigger the speed and accuracy benchmarks for the reader. Here we simply use the SQuAD dev set.
**--retriever_index** will trigger indexing benchmarks
**--retriever_query** will trigger querying benchmarks (embeddings will be loaded from file instead of being computed on the fly)
**--ci** will cause the the benchmarks to run on a smaller slice of each dataset and a smaller subset of Retriever / Reader / DocStores.
**--update-json** will cause the script to update the json files in docs/_src/benchmarks so that the website benchmarks will be updated.