2531c8e061
* add time and perf benchmark for es * Add retriever benchmarking * Add Reader benchmarking * add nq to squad conversion * add conversion stats * clean benchmarks * Add link to dataset * Update imports * add first support for neg psgs * Refactor test * set max_seq_len * cleanup benchmark * begin retriever speed benchmarking * Add support for retriever query index benchmarking * improve reader eval, retriever speed benchmarking * improve retriever speed benchmarking * Add retriever accuracy benchmark * Add neg doc shuffling * Add top_n * 3x speedup of SQL. add postgres docker run. make shuffle neg a param. add more logging * Add models to sweep * add option for faiss index type * remove unneeded line * change faiss to faiss_flat * begin automatic benchmark script * remove existing postgres docker for benchmarking * Add data processing scripts * Remove shuffle in script bc data already shuffled * switch hnsw setup from 256 to 128 * change es similarity to dot product by default * Error includes stack trace * Change ES default timeout * remove delete_docs() from timing for indexing * Add support for website export * update website on push to benchmarks * add complete benchmarks results * new json format * removed NaN as is not a valid json token * versioning for docs * unsaved changes * cleaning * cleaning * Edit format of benchmarks data * update also jsons in v0.4.0 Co-authored-by: brandenchan <brandenchan@icloud.com> Co-authored-by: deepset <deepset@Crenolape.localdomain> Co-authored-by: Malte Pietsch <malte.pietsch@deepset.ai>
1.7 KiB
Open roadmap
We believe open-source is more than open source code. It's a lot about people, collaboration, transparency and trust. Therefore, we decided to be as open as possible with our roadmap and sprint planning. In fact, you can see all of it in real-time on GitHub. We hope this helps to clarify the direction of the open-source project and inspires discussions in the community.
How to access it
Add Zenhub Plugin
We decided for Zenhub, as it allows a close integration with GitHub and real-time sharing of roadmaps and sprints. Once you have installed the browser plugin below, you will see additional tabs and infos on the Haystack GitHub page.
Zenhub Plugin: https://www.zenhub.com/extension
Roadmap
Zenhub Tab -> Roadmap (left menu)
Here you can find our most recent roadmap with the high-level projects that are planned for the upcoming quarters. We update it regularly and refine the projects as they come closer in time.
Board
Zenhub Tab -> Board (left menu)
If you are interested in the operational tasks and their status, you can find our agile board here.
Additional issue details & Releases
Right panel in regular Github issues
With Zenhub you can also see some additional tags in every GitHub issue. For those of you who wonder about the next release date: We aim for releases every ~ 4 weeks and will tag the issues that will need to be finished before a bit in advance.
