diff --git a/docs/_src/usage/usage/database.md b/docs/_src/usage/usage/database.md index 47d7dc3fd..b9f8c2e28 100644 --- a/docs/_src/usage/usage/database.md +++ b/docs/_src/usage/usage/database.md @@ -20,35 +20,55 @@ There are different DocumentStores in Haystack to fit different use cases and te Initialising a new Document Store is straight forward. -
-Elasticsearch FAISS In Memory SQL -
-
+
+ +
+ + +
```python document_store = ElasticsearchDocumentStore() ```
-
+
+ +
+ + +
```python document_store = FAISSDocumentStore() ```
-
+
+ +
+ + +
+ +```python +document_store = InMemoryDocumentStore() +``` + +
+
+ +
+ + +
```python document_store = SQLDocumentStore() ```
-
- -```python -document_store = InMemoryDocumentStore() -``` +
@@ -123,11 +143,12 @@ Having GPU acceleration will significantly speed this up. The Document stores have different characteristics. You should choose one depending on the maturity of your project, the use case and technical environment: +
-
-Elasticsearch FAISS In Memory SQL -
-
+
+ + +
**Pros:** - Fast & accurate sparse retrieval @@ -139,7 +160,12 @@ The Document stores have different characteristics. You should choose one depend - Slow for dense retrieval with more than ~ 1 Mio documents
-
+
+ +
+ + +
**Pros:** - Fast & accurate dense retrieval @@ -150,7 +176,12 @@ The Document stores have different characteristics. You should choose one depend - No efficient sparse retrieval
-
+
+ +
+ + +
**Pros:** - Simple @@ -162,7 +193,12 @@ The Document stores have different characteristics. You should choose one depend - Not recommended for production
-
+
+ +
+ + +
**Pros:** - Simple & fast to test @@ -172,6 +208,9 @@ The Document stores have different characteristics. You should choose one depend - Not scalable - Not persisting your data on disk +
+
+
#### Our recommendations diff --git a/docs/_src/usage/usage/get_started.md b/docs/_src/usage/usage/get_started.md index 4052f783d..ec85ac299 100644 --- a/docs/_src/usage/usage/get_started.md +++ b/docs/_src/usage/usage/get_started.md @@ -11,23 +11,28 @@ id: "get_startedmd" ## Installation -
-Basic Editable -
-
+
-The most straightforward way to install Haystack is through pip. +
+ + +
+The most straightforward way to install Haystack is through pip.

```python $ pip install farm-haystack ``` +
-
+
+ + +
If you’d like to run a specific, unreleased version of Haystack, or make edits to the way Haystack runs, you’ll want to install it using `git` and `pip --editable`. -This clones a copy of the repo to a local directory and runs Haystack from there. +This clones a copy of the repo to a local directory and runs Haystack from there.

```python $ git clone https://github.com/deepset-ai/haystack.git @@ -35,9 +40,9 @@ $ cd haystack $ pip install --editable . ``` -By default, this will give you the latest version of the master branch. -Use regular git commands to switch between different branches and commits. - +By default, this will give you the latest version of the master branch. Use regular git commands to switch between different branches and commits. +
+
diff --git a/docs/_src/usage/usage/reader.md b/docs/_src/usage/usage/reader.md index edaf06fa6..cd91cf753 100644 --- a/docs/_src/usage/usage/reader.md +++ b/docs/_src/usage/usage/reader.md @@ -25,10 +25,12 @@ Haystack’s Readers are: * state-of-the-art in QA tasks like SQuAD and Natural Questions -
-FARM Transformers -
-
+
+ +
+ + +
```python model = "deepset/roberta-base-squad2" @@ -36,8 +38,13 @@ reader = FARMReader(model, use_gpu=True) finder = Finder(reader, retriever) ``` +
-
+ +
+ + +
```python model = "deepset/roberta-base-squad2" @@ -45,6 +52,9 @@ reader = TransformersReader(model, use_gpu=1) finder = Finder(reader, retriever) ``` +
+
+
While these models can work on CPU, it is recommended that they are run using GPUs to keep query times low. @@ -58,12 +68,19 @@ and you have the option of using the QA pipeline from deepset FARM or HuggingFac Currently, there are a lot of different models out there and it can be rather overwhelming trying to pick the one that fits your use case. To get you started, we have a few recommendations for you to try out. -**FARM** +
-
-RoBERTa (base) MiniLM ALBERT (XXL) -
-
+
+ + +
+ +
+ +
+ + +
**An optimised variant of BERT and a great starting point.** @@ -71,14 +88,17 @@ To get you started, we have a few recommendations for you to try out. reader = FARMReader("deepset/roberta-base-squad2") ``` - * **Pro**: Strong all round model - * **Con**: There are other models that are either faster or more accurate
-
+
+ +
+ + +
**A cleverly distilled model that sacrifices a little accuracy for speed.** @@ -86,14 +106,17 @@ reader = FARMReader("deepset/roberta-base-squad2") reader = FARMReader("deepset/minilm-uncased-squad2") ``` - * **Pro**: Inference speed up to 50% faster than BERT base - * **Con**: Still doesn’t match the best base sized models in accuracy
-
+
+ +
+ + +
**Large, powerful, SotA model.** @@ -101,21 +124,29 @@ reader = FARMReader("deepset/minilm-uncased-squad2") reader = FARMReader("ahotrod/albert_xxlargev1_squad2_512") ``` - * **Pro**: Better accuracy than any other open source model in QA - * **Con**: The computational power needed make it impractical for most use cases
- - -**Transformers** - -
-RoBERTa (base) MiniLM ALBERT (XXL)
-
+ +
+ +
+
+ +
+ + +
+ +
+ +
+ + +
**An optimised variant of BERT and a great starting point.** @@ -123,14 +154,17 @@ reader = FARMReader("ahotrod/albert_xxlargev1_squad2_512") reader = TransformersReader("deepset/roberta-base-squad2") ``` - * **Pro**: Strong all round model - * **Con**: There are other models that are either faster or more accurate
-
+
+ +
+ + +
**A cleverly distilled model that sacrifices a little accuracy for speed.** @@ -138,14 +172,17 @@ reader = TransformersReader("deepset/roberta-base-squad2") reader = TransformersReader("deepset/minilm-uncased-squad2") ``` - * **Pro**: Inference speed up to 50% faster than BERT base - * **Con**: Still doesn’t match the best base sized models in accuracy
-
+
+ +
+ + +
**Large, powerful, SotA model.** @@ -153,12 +190,18 @@ reader = TransformersReader("deepset/minilm-uncased-squad2") reader = TransformersReader("ahotrod/albert_xxlargev1_squad2_512") ``` - * **Pro**: Better accuracy than any other open source model in QA - * **Con**: The computational power needed make it impractical for most use cases +
+
+ +
+ +
+
+
**All-rounder**: In the class of base sized models trained on SQuAD, **RoBERTa** has shown better performance than BERT @@ -183,59 +226,104 @@ While models are comparatively more performant on English, thanks to a wealth of available English training data, there are a couple QA models that are directly usable in Haystack. -**FARM** +
-
-French Italian Zero-shot -
-
+
+ + +
+ +
+ +
+ + +
```python reader = FARMReader("illuin/camembert-base-fquad") ```
-
+
+ +
+ + +
```python reader = FARMReader("mrm8488/bert-italian-finedtuned-squadv1-it-alfa") ```
-
+
+ +
+ + +
```python reader = FARMReader("deepset/xlm-roberta-large-squad2") ```
- - -**Transformers** - -
-French Italian Zero-shot
-
+ +
+ +
+
+ +
+ + +
+ +
+ +
+ + +
```python reader = TransformersReader("illuin/camembert-base-fquad") ```
-
+
+ +
+ + +
```python reader = TransformersReader("mrm8488/bert-italian-finedtuned-squadv1-it-alfa") ```
-
+
+ +
+ + +
```python reader = TransformersReader("deepset/xlm-roberta-large-squad2") ``` +
+
+ +
+ +
+
+
The **French** and **Italian models** are both monolingual language models trained on French and Italian versions of the SQuAD dataset @@ -317,22 +405,32 @@ This functions by slicing the document into overlapping passages of (approximate that are each offset by `doc_stride` number of tokens. These can be set when the Reader is initialized. -
-FARM Transformers -
-
+
+ +
+ + +
```python reader = FARMReader(... max_seq_len=384, doc_stride=128 ...) ``` +
-
+ +
+ + +
```python reader = TransformersReader(... max_seq_len=384, doc_stride=128 ... ``` +
+
+
Predictions are made on each individual passage and the process of aggregation picks the best candidates across all passages.