mirror of
https://github.com/deepset-ai/haystack.git
synced 2025-11-01 18:29:32 +00:00
363be65a78
* Simplify ODES init * Add arguments to ES init and create script * Rename similarity_fn_name and add util fn * Create OpenSearchDocumentStore * Specify params of Open Search HNSW * Add better argument handling * Update opensearch index mapping * Edit opensearch default port * Fix HNSW mapping * Force small HNSW params * Implement auto start and stopping of document store services * Fix starting and stopping of ds service * Restore HNSW params * Add opensearch query benchmarks * Add write wait time * Revert wait time * Add timeout * Update benchmarks * Update benchmarks * Update benchmarks json * Update documentation * Update documentation * Fix similarity name * Improve argument passing * Improve stopping and starting of service
5.6 KiB
5.6 KiB
| 1 | retriever | doc_store | n_docs | n_queries | retrieve_time | queries_per_second | seconds_per_query | recall | map | top_k | date_time | error | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | 1 | dpr | elasticsearch | 1000 | 1064 | 31.085943690999557 | 34.22768858415144 | 0.029216112491540935 | 99.15413533834587 | 92.9510532283089 | 10 | 2021-06-01 10:41:39.916257 | |
| 3 | 8 | dpr | elasticsearch | 10000 | 5637 | 253.95221038600266 | 22.19708972578685 | 0.04505095092886334 | 97.49866950505589 | 89.87097014904354 | 10 | 2021-06-01 10:53:27.311560 | |
| 4 | 7 | elastic | elasticsearch | 10000 | 5637 | 44.34573464198456 | 127.11481826852271 | 0.007866903431255023 | 81.16019159127195 | 66.26543444531747 | 10 | 2021-06-01 10:48:00.160553 | |
| 5 | 6 | sentence_transformers | elasticsearch | 1000 | 1064 | 22.393676894006603 | 47.51341215808855 | 0.02104668881015658 | 98.7781954887218 | 90.06638620360428 | 10 | 2021-06-01 10:46:25.787857 | |
| 6 | 0 | sentence_transformers | elasticsearch | 10000 | 5637 | 189.50983109897788 | 29.74515869340777 | 0.033618916285076794 | 96.52297321270179 | 87.1125514246855 | 10 | 2021-06-02 08:34:52.614091 | |
| 7 | 4 | dpr | faiss_flat | 1000 | 1064 | 25.037448677994234 | 42.496342725813136 | 0.02353143672743819 | 99.15413533834587 | 92.9510532283089 | 10 | 2021-06-01 10:44:39.303106 | |
| 8 | 11 | dpr | faiss_flat | 10000 | 5637 | 203.61912121901514 | 27.68404050784983 | 0.03612189484105289 | 97.49866950505589 | 89.87097014904354 | 10 | 2021-06-01 11:05:18.461448 | |
| 9 | 5 | dpr | faiss_hnsw | 1000 | 1064 | 24.534864505002133 | 43.36685860983961 | 0.02305908318139298 | 99.15413533834587 | 92.9510532283089 | 10 | 2021-06-01 10:45:44.973819 | |
| 10 | 12 | dpr | faiss_hnsw | 10000 | 5637 | 134.7947145470116 | 41.819147130090286 | 0.023912491493172185 | 96.98421145999644 | 89.51337675393017 | 10 | 2021-06-01 11:09:08.851328 | |
| 11 | 2 | dpr | milvus_flat | 1000 | 1064 | 25.87419783899759 | 41.12204778755844 | 0.02431785511183984 | 99.15413533834587 | 92.9510532283089 | 10 | 2021-06-01 10:42:39.082235 | |
| 12 | 9 | dpr | milvus_flat | 10000 | 5637 | 148.85595429698787 | 37.86882443918513 | 0.026406945945891054 | 97.49866950505589 | 89.87097014904354 | 10 | 2021-06-01 10:57:04.267331 | |
| 13 | 3 | dpr | milvus_hnsw | 1000 | 1064 | 25.85785580699985 | 41.148036710451834 | 0.024302496059210386 | 99.15413533834587 | 92.9510532283089 | 10 | 2021-06-01 10:43:33.575115 | |
| 14 | 10 | dpr | milvus_hnsw | 10000 | 5637 | 140.6687312999984 | 40.07287154654294 | 0.024954538105374915 | 95.5472769203477 | 88.24421129104468 | 10 | 2021-06-01 11:00:28.134755 | |
| 15 | 1 | dpr | elasticsearch | 100000 | 5637 | 1085.631664579022 | 5.192368815242574 | 0.1925903254530818 | 95.7956359765833 | 86.54606328368976 | 10 | 2021-06-01 11:49:21.374639 | |
| 16 | 8 | dpr | elasticsearch | 500000 | 5637 | 5452.980152460905 | 1.0337466563959614 | 0.9673550030975527 | 93.0814262905801 | 80.8613722823409 | 10 | 2021-06-01 15:23:36.366798 | |
| 17 | 0 | elastic | elasticsearch | 100000 | 5637 | 54.681204182025795 | 103.0884393334727 | 0.009700408760338087 | 71.97090651055525 | 56.25299537353825 | 10 | 2021-06-01 11:26:20.664211 | |
| 18 | 7 | elastic | elasticsearch | 500000 | 5637 | 71.39928511296421 | 78.95037031647354 | 0.012666185047536669 | 62.746141564662054 | 45.59509026246653 | 10 | 2021-06-01 13:31:49.432201 | |
| 19 | 6 | sentence_transformers | elasticsearch | 100000 | 5637 | 882.7645091800478 | 6.385621466857457 | 0.15660182884159088 | 93.79102359411034 | 82.74686664920836 | 10 | 2021-06-01 13:12:26.587291 | |
| 20 | 13 | sentence_transformers | elasticsearch | 500000 | 5637 | 3976.5921420610975 | 1.417545425485426 | 0.7054447653115306 | 89.6398793684584 | 76.49564526892905 | 10 | 2021-06-01 21:25:08.210475 | |
| 21 | 4 | dpr | faiss_flat | 100000 | 5637 | 850.5991484880342 | 6.6270933964840415 | 0.15089571553805822 | 95.7956359765833 | 86.54606328368972 | 10 | 2021-06-01 12:30:30.144119 | |
| 22 | 11 | dpr | faiss_flat | 500000 | 5637 | 3661.5868466028005 | 1.5394964631878052 | 0.6495630382477915 | 93.0814262905801 | 80.86137228234091 | 10 | 2021-06-01 18:12:56.329460 | |
| 23 | 5 | dpr | faiss_hnsw | 100000 | 5637 | 141.69029460409365 | 39.78395285118659 | 0.025135762746867775 | 93.18786588610963 | 84.33419639513305 | 10 | 2021-06-01 12:43:24.110221 | |
| 24 | 12 | dpr | faiss_hnsw | 500000 | 5637 | 141.48467584206082 | 39.84177061191119 | 0.025099286117094347 | 86.8901898172787 | 75.73062475537202 | 10 | 2021-06-01 19:08:33.234845 | |
| 25 | 2 | dpr | milvus_flat | 100000 | 5637 | 197.28661122802168 | 28.57264344960955 | 0.03499851183750607 | 95.7956359765833 | 86.54606328368972 | 10 | 2021-06-01 11:59:02.307581 | |
| 26 | 9 | dpr | milvus_flat | 500000 | 5637 | 360.2868321948117 | 15.645867393099733 | 0.0639146411557232 | 93.0814262905801 | 80.8613722823409 | 10 | 2021-06-01 15:59:31.477921 | |
| 27 | 3 | dpr | milvus_hnsw | 100000 | 5637 | 145.2638032350751 | 38.80526238789059 | 0.0257697007690394 | 90.01241795281177 | 81.63864883662649 | 10 | 2021-06-01 12:07:43.734963 | |
| 28 | 10 | dpr | milvus_hnsw | 500000 | 5637 | 151.7069119141779 | 37.15717318924075 | 0.02691270390530032 | 84.1759801312755 | 73.57986207906387 | 10 | 2021-06-01 16:30:55.573386 | |
| 29 | 0 | elastic | elasticsearch | 1000 | 1064 | 3.760260104999361 | 282.95914917837337 | 0.003534079046052031 | 89.09774436090225 | 74.2044471297291 | 10 | 2021-06-02 08:27:07.187435 | |
| 30 | 0 | dpr | opensearch_flat | 1000 | 1064 | 36.61243656901206 | 29.061163356184426 | 0.034410184745312086 | 99.15413533834587 | 92.9510532283089 | 10 | 2021-07-19 15:25:29.480947 | |
| 31 | 2 | dpr | opensearch_flat | 10000 | 5637 | 226.98340490202918 | 24.834414667596725 | 0.040266703016148514 | 97.49866950505589 | 89.87097014904359 | 10 | 2021-07-19 15:32:29.913507 | |
| 32 | 4 | dpr | opensearch_flat | 100000 | 5637 | 368.2654070430117 | 15.306895223372486 | 0.06533003495529745 | 95.77789604399504 | 86.54014997282702 | 10 | 2021-07-19 15:49:18.090106 | |
| 33 | 1 | dpr | opensearch_hnsw | 1000 | 1064 | 36.55576791198473 | 29.106213896581007 | 0.03435692472930896 | 98.96616541353383 | 92.76308330349686 | 10 | 2021-07-19 15:27:04.463343 | |
| 34 | 3 | dpr | opensearch_hnsw | 10000 | 5637 | 209.3657621010234 | 26.924173004371312 | 0.03714134505961032 | 96.41653361717225 | 89.00403653862938 | 10 | 2021-07-19 15:37:40.060081 | |
| 35 | 5 | dpr | opensearch_hnsw | 100000 | 5637 | 225.86542887897758 | 24.95733865947408 | 0.040068374823306295 | 94.8199396842292 | 85.7342431384476 | 10 | 2021-07-19 15:58:37.854825 | |
| 36 | 0 | dpr | opensearch_flat | 500000 | 5637 | 497.4096126070708 | 11.33271222977541 | 0.0882401299640005 | 93.06368635799184 | 80.85588135082547 | 10 | 2021-07-21 13:12:13.891855 | |
| 37 | 1 | dpr | opensearch_hnsw | 500000 | 5637 | 233.5204362630284 | 24.139214923573974 | 0.04142636797286294 | 88.96576193010468 | 77.54264623476982 | 10 | 2021-07-21 13:39:28.533054 |