mirror of
https://github.com/deepset-ai/haystack.git
synced 2025-09-02 12:53:52 +00:00
363be65a78
* Simplify ODES init * Add arguments to ES init and create script * Rename similarity_fn_name and add util fn * Create OpenSearchDocumentStore * Specify params of Open Search HNSW * Add better argument handling * Update opensearch index mapping * Edit opensearch default port * Fix HNSW mapping * Force small HNSW params * Implement auto start and stopping of document store services * Fix starting and stopping of ds service * Restore HNSW params * Add opensearch query benchmarks * Add write wait time * Revert wait time * Add timeout * Update benchmarks * Update benchmarks * Update benchmarks json * Update documentation * Update documentation * Fix similarity name * Improve argument passing * Improve stopping and starting of service
Benchmarks
Run the benchmarks with the following command:
python run.py [--reader] [--retriever_index] [--retriever_query] [--ci] [--update-json]
You can specify which components and processes to benchmark with the following flags.
--reader will trigger the speed and accuracy benchmarks for the reader. Here we simply use the SQuAD dev set.
--retriever_index will trigger indexing benchmarks
--retriever_query will trigger querying benchmarks (embeddings will be loaded from file instead of being computed on the fly)
--ci will cause the the benchmarks to run on a smaller slice of each dataset and a smaller subset of Retriever / Reader / DocStores.
--update-json will cause the script to update the json files in docs/_src/benchmarks so that the website benchmarks will be updated.