haystack/releasenotes/notes/add-v2-extractive-reader-a2158d38781803ec.yaml
MichelBartels f3dc9edd26
feat: initial ExtractiveReader implementation (#5553)
* initial ExtractiveReader implementation

* initial ExtractiveReader implementation

* fix mypy

* remove unused import

* Use AutoTokenizer

* rename reader to model

* combine no-answer logit

* support document slicing with proper probabilities

* add variable stride

* validate model

* fix typo

* make postprocessing easier to understand

* remove debug code

* set default reader

* add ExtractiveReader to __init__

* remove validation

* use new answer class

* add batching

* use v2 lazy imports

* move reader

* fix type hints

* add doc strings

* add nucleus sampling

* fix types

* fix doc string

* add no_answer parameter

* remove print statement

* fix gpu support

* turn into binary classification task

* change dataclass so document does not need to be provided for no answer

* add simple tests

* add unit tests

* rename reader folder to readers

* add integration tests

* fix type hints

* add release notes

* remove accidentally included test file

* remove unnecessary __init__ file

* revert __init__ file to main

* rename test script by adding test_ prefix

* undo accidentally moving of test script after renaming it

* remove use of bisect

* rename _flatten and _unflatten

* make variable name more intuitive

* remove type: ignore

* fix mypy issue

* refactor long tuple

* add doc strings

* explain HF test

* remove unnecessary top_k check

---------

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>
2023-09-21 12:16:51 +02:00

8 lines
306 B
YAML

---
preview:
- |
This adds an ExtractiveReader for v2. It should be a replacement where
FARMReader would have been used before for inference.
The confidence scores are calculated differently from FARMReader because
each span is considered to be an independent binary classification task.