mirror of
https://github.com/deepset-ai/haystack.git
synced 2025-12-30 08:37:20 +00:00
f3a3b73d9b
* Adapt to new dataset_from_dicts return signature * rename fn * Align similarity fn in benchmark doc store * Better choice of similarity fn * Increase postgres wait time * Add more expected returned variables * update benchmark results * Fix typo * update all benchmark runs * multiply stats by 100 * Specify similarity fns for website Co-authored-by: Malte Pietsch <malte.pietsch@deepset.ai>
1.0 KiB
1.0 KiB
| 1 | EM | f1 | top_n_accuracy | top_n | reader_time | seconds_per_query | passages_per_second | reader | error | |
|---|---|---|---|---|---|---|---|---|---|---|
| 2 | 0 | 0.7839204449688185 | 0.8258860575299658 | 0.9742120343839542 | 5 | 98.16358173700064 | 0.008272676701247315 | 125.81040525892847 | deepset/roberta-base-squad2 | |
| 3 | 1 | 0.7438058317883027 | 0.7887858491007042 | 0.9719366256531266 | 5 | 47.38258053499885 | 0.003993138423647299 | 260.6443097981493 | deepset/minilm-uncased-squad2 | |
| 4 | 2 | 0.6947581324793528 | 0.7431182400443286 | 0.9557559413450194 | 5 | 101.99811779300217 | 0.008595829916821352 | 121.08066567525722 | deepset/bert-base-cased-squad2 | |
| 5 | 3 | 0.7897353783920446 | 0.8326306774734308 | 0.976908815101972 | 5 | 292.51886408200517 | 0.024651851009776266 | 42.21949937744112 | deepset/bert-large-uncased-whole-word-masking-squad2 | |
| 6 | 4 | 0.8021237148154391 | 0.8450422699207468 | 0.974043485589078 | 5 | 293.53038741600176 | 0.024737096529243364 | 42.07400844838984 | deepset/xlm-roberta-large-squad2 | |
| 7 | 5 | 0.3729984830608461 | 0.4231925844723574 | 0.9539019046013821 | 5 | 55.403011280999635 | 0.004669055391960192 | 222.91207128366705 | distilbert-base-uncased-distilled-squad |