mirror of
https://github.com/deepset-ai/haystack.git
synced 2025-12-29 16:08:38 +00:00
7e79a48540
* quick fix benchmark runs to make them work with current haystack version * fix minor typo * update readme. fix minor things to make benchmarks run again * Update Documentation & Code Style * fix typo in readme * update result files for reader and retriever querying * reduce batch size for update embeddings to prevent xlarge bulk_update requests that exceed elastic's limits (happening in dense 500k runs) * change default memory allocation back to normal. add note to readme * add first indexing results * add memory to docker cmd * full benchmarks results on commit c5a2651fcbbeffca06ffa9036b10e62669bcc1b0 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
8 lines
2.0 KiB
CSV
8 lines
2.0 KiB
CSV
,EM,f1,top_n_accuracy,top_n,reader_time,seconds_per_query,EM_text_answer,f1_text_answer,top_n_accuracy_text_answer,top_n_EM_text_answer,top_n_f1_text_answer,Total_text_answer,EM_no_answer,f1_no_answer,top_n_accuracy_no_answer,Total_no_answer,passages_per_second,reader,error
|
|
0,76.1672004045171,78.21542223321491,97.41277599865161,5,104.290868674987,0.0087890501158762,61.491731353358084,65.59301387433817,94.81943975700304,81.01586230172123,90.50667071721769,5926,90.8080808080808,90.8080808080808,100.0,5940,118.41880460779026,deepset/roberta-base-squad2,
|
|
1,76.84982302376538,79.6479775678566,97.21894488454407,5,52.54729196999688,0.004428391367773207,65.89605129935876,71.49897094501964,94.4313196085049,79.96962537968275,89.70802565150441,5926,87.77777777777777,87.77777777777777,100.0,5940,235.02638360605766,deepset/minilm-uncased-squad2,
|
|
2,70.0825889094893,73.49806294434092,95.53345693578291,5,105.55706145399017,0.008895757749367114,56.59804252446844,63.43705955071704,91.05636179547756,72.1228484643942,84.47853540832811,5926,83.53535353535354,83.53535353535354,100.0,5940,116.99833085428466,deepset/bert-base-cased-squad2,
|
|
3,83.69290409573573,87.35038229973254,98.07854373841226,5,284.2086430560157,0.023951512140233923,75.19406007424907,82.51765716649112,96.15254809314884,82.33209584880188,91.89205932239825,5926,92.17171717171718,92.17171717171718,100.0,5940,43.45399164221017,deepset/roberta-large-squad2,
|
|
4,79.50446654306421,82.29388851439239,97.52233271532108,5,103.88512606802396,0.008754856402159445,71.110361120486,76.69579499017553,95.03881201484981,81.72460344245697,90.89098996208568,5926,87.87878787878788,87.87878787878788,100.0,5940,118.88131118898794,deepset/roberta-base-squad2-distilled,
|
|
5,38.53868194842407,43.340283031558755,95.43232765885725,5,62.23228176299017,0.005244588046771462,74.53594330070874,84.1504891077415,90.85386432669593,75.09281133985826,85.21259442024737,5926,2.6262626262626263,2.6262626262626263,100.0,5940,198.4500591997352,distilbert-base-uncased-distilled-squad,
|