mirror of
https://github.com/deepset-ai/haystack.git
synced 2026-01-07 04:27:15 +00:00
f3a3b73d9b
* Adapt to new dataset_from_dicts return signature * rename fn * Align similarity fn in benchmark doc store * Better choice of similarity fn * Increase postgres wait time * Add more expected returned variables * update benchmark results * Fix typo * update all benchmark runs * multiply stats by 100 * Specify similarity fns for website Co-authored-by: Malte Pietsch <malte.pietsch@deepset.ai>
2.6 KiB
2.6 KiB
| 1 | retriever | doc_store | n_docs | n_queries | retrieve_time | queries_per_second | seconds_per_query | recall | map | top_k | date_time | error | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | 1 | dpr | elasticsearch | 1000 | 1064 | 34.67547423102587 | 30.684511851549132 | 0.0325897314201371 | 0.9915413533834586 | 0.9295105322830889 | 10 | 2021-02-01 11:27:43.048502 | |
| 3 | 5 | dpr | elasticsearch | 10000 | 5637 | 288.06125728895495 | 19.568754413737462 | 0.05110187285594375 | 0.9749866950505588 | 0.8987097014904354 | 10 | 2021-02-01 11:37:21.149887 | |
| 4 | 9 | dpr | elasticsearch | 100000 | 5637 | 1225.6274364620313 | 4.5992769354707805 | 0.2174254810115365 | 0.9579563597658329 | 0.8654564090434241 | 10 | 2021-02-01 12:15:52.757320 | |
| 5 | 13 | dpr | elasticsearch | 500000 | 5637 | 5339.008413678017 | 1.0558140319761546 | 0.947136493467805 | 0.9308142629058009 | 0.8086137228234089 | 10 | 2021-02-01 14:52:23.056230 | |
| 6 | 0 | elastic | elasticsearch | 1000 | 1064 | 4.046542626992959 | 262.9405144288997 | 0.0038031415667227064 | 0.8909774436090225 | 0.742044471297291 | 10 | 2021-02-01 11:26:04.346134 | |
| 7 | 4 | elastic | elasticsearch | 10000 | 5637 | 30.701430243001596 | 183.6070813438718 | 0.005446413028738974 | 0.8110697179350719 | 0.6620627317806674 | 10 | 2021-02-01 11:31:20.470092 | |
| 8 | 8 | elastic | elasticsearch | 100000 | 5637 | 34.705507238930295 | 162.42378943468643 | 0.006156733588598598 | 0.7193542664537875 | 0.5625959153101251 | 10 | 2021-02-01 11:50:36.048887 | |
| 9 | 12 | elastic | elasticsearch | 500000 | 5637 | 68.3838099470413 | 82.4317920333114 | 0.01213124178588634 | 0.6274614156466205 | 0.45594527090003406 | 10 | 2021-02-01 13:02:16.905187 | |
| 10 | 2 | dpr | faiss_flat | 1000 | 1064 | 30.053267333012627 | 35.40380445859966 | 0.028245552004711117 | 0.9915413533834586 | 0.9295105322830889 | 10 | 2021-02-01 11:28:55.544474 | |
| 11 | 6 | dpr | faiss_flat | 10000 | 5637 | 218.59436279792135 | 25.78749025294445 | 0.038778492602079356 | 0.9749866950505588 | 0.8987097014904354 | 10 | 2021-02-01 11:42:07.545869 | |
| 12 | 10 | dpr | faiss_flat | 100000 | 5637 | 865.7440476809643 | 6.511162294559943 | 0.15358241044544338 | 0.9579563597658329 | 0.8654606328368972 | 10 | 2021-02-01 12:34:29.493598 | |
| 13 | 14 | dpr | faiss_flat | 500000 | 5637 | 3717.9468668280497 | 1.5161593755666505 | 0.6595612678424783 | 0.9308142629058009 | 0.808613722823409 | 10 | 2021-02-01 16:12:52.804436 | |
| 14 | 3 | dpr | faiss_hnsw | 1000 | 1064 | 27.167708159968242 | 39.164142729117266 | 0.02553356030072203 | 0.9915413533834586 | 0.9295105322830889 | 10 | 2021-02-01 11:30:02.684535 | |
| 15 | 7 | dpr | faiss_hnsw | 10000 | 5637 | 167.55242089293097 | 33.6432023480111 | 0.02972368651639719 | 0.972503104488203 | 0.8969941373746582 | 10 | 2021-02-01 11:46:07.130588 | |
| 16 | 11 | dpr | faiss_hnsw | 100000 | 5637 | 167.48228578322596 | 33.65729082116796 | 0.029711244595214823 | 0.9402164271775767 | 0.8507984377595874 | 10 | 2021-02-01 12:43:21.697968 | |
| 17 | 15 | dpr | faiss_hnsw | 500000 | 5637 | 164.45566683610014 | 34.27671486454735 | 0.029174324434291313 | 0.8825616462657442 | 0.7691475821598232 | 10 | 2021-02-01 16:47:01.710072 |