yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2026-01-01 17:47:19 +00:00
Code Issues Packages Projects Releases Wiki Activity
haystack/test/benchmarks
History
Branden Chan 77d4c2ca1c
Benchmark milvus (#850) 
* Add milvus benchmarking support

* Add latest docstring and tutorial changes

* Edit config

* Disable docker interactive mode

* Add milvus index type support

* Adjust FAISS and Milvus node branching

* Remove duplicate in config

* Revert method for speedup

* Add latest docstring and tutorial changes

* Add latest benchmark run

* Add latest docstring and tutorial changes

* Add json files

* Revert "Add latest docstring and tutorial changes"

This reverts commit e2efa5f08aa4fb55bbeeed42aa76817d63fc8923.

* Add latest docstring and tutorial changes

* Revert "Add latest docstring and tutorial changes"

This reverts commit b085a679b9d5f175e91c2c59565e73c5dec1374b.

* Fix typo

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2021-04-13 14:54:15 +02:00
..
data_scripts
Create time and performance benchmarks for all readers and retrievers (#339)
2020-10-12 13:34:42 +02:00
config.json
Benchmark milvus (#850)
2021-04-13 14:54:15 +02:00
nq_to_squad.py
Create time and performance benchmarks for all readers and retrievers (#339)
2020-10-12 13:34:42 +02:00
reader_results.csv
Choose correct similarity fns during benchmark runs & re-run benchmarks (#773)
2021-02-03 11:45:18 +01:00
reader.py
Choose correct similarity fns during benchmark runs & re-run benchmarks (#773)
2021-02-03 11:45:18 +01:00
README.md
add readme
2020-10-22 15:32:56 +02:00
results_to_json.py
Benchmark milvus (#850)
2021-04-13 14:54:15 +02:00
retriever_index_results.csv
Benchmark milvus (#850)
2021-04-13 14:54:15 +02:00
retriever_query_results.csv
Benchmark milvus (#850)
2021-04-13 14:54:15 +02:00
retriever_query_results.md
Choose correct similarity fns during benchmark runs & re-run benchmarks (#773)
2021-02-03 11:45:18 +01:00
retriever.py
Integrate sentence transformers into benchmarks (#843)
2021-04-09 17:24:16 +02:00
run.py
Integrate sentence transformers into benchmarks (#843)
2021-04-09 17:24:16 +02:00
templates.py
fixed link to dpr (#962)
2021-04-13 09:45:04 +02:00
utils.py
Benchmark milvus (#850)
2021-04-13 14:54:15 +02:00

README.md

Benchmarks

Run the benchmarks with the following command:

python run.py [--reader] [--retriever_index] [--retriever_query] [--ci] [--update-json]

You can specify which components and processes to benchmark with the following flags.

--reader will trigger the speed and accuracy benchmarks for the reader. Here we simply use the SQuAD dev set.

--retriever_index will trigger indexing benchmarks

--retriever_query will trigger querying benchmarks (embeddings will be loaded from file instead of being computed on the fly)

--ci will cause the the benchmarks to run on a smaller slice of each dataset and a smaller subset of Retriever / Reader / DocStores.

--update-json will cause the script to update the json files in docs/_src/benchmarks so that the website benchmarks will be updated.