Ashwin Mathur
393a7993c3
feat: Add Semantic Answer Similarity metric ( #6877 )
...
* Add SAS metric
* Add release notes
* Round similarity scores for precision consistency
* Add tolerance to tests
* Update haystack/evaluation/eval.py
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
* Add types for preprocess_text; Add additional types for f1 and em methods
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-02-02 17:07:52 +01:00
Ashwin Mathur
7217f9d9f0
feat: Add F1 metric ( #6822 )
...
* Add F1 metric
* Add release notes
2024-01-26 11:04:43 +01:00
Ashwin Mathur
a238c6dd51
feat: Add Exact Match metric ( #6696 )
...
* Add exact match metric
* Add release notes
* Cleanup comments in test_eval_exact_match.py
* Create separate preprocessing function; Add output_key parameter
* Update release note
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-01-22 09:57:04 +01:00
ZanSara
96c0b59aaa
feat!: Rename model_name_or_path to model in ExtractiveReader ( #6736 )
...
* rename model parameter and internam model attribute in ExtractiveReader
* fix tests for ExtractiveReader
* fix e2e
* reno
* another fix
* review feedback
* Update releasenotes/notes/rename-model-param-reader-b8cbb0d638e3b8c2.yaml
2024-01-15 14:48:33 +01:00
Massimiliano Pippi
e1ec4e5e4d
refact!: Remove symbols under the haystack.document_stores namespace ( #6714 )
...
* remove symbols under the haystack.document_stores namespace
* Update haystack/document_stores/types/protocol.py
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
* fix
* same for retrievers
* leftovers
* more leftovers
* add relnote
* leftovers
* one more
* fix examples
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-01-10 21:20:42 +01:00
Ashwin Mathur
374a937663
feat: Add calculate_metrics and MetricsResult ( #6680 )
...
* Add calculate_metrics, MetricsResult, Exact Match
* Add additional tests for metric calculation
* Add release notes
* Add docstring for Exact Match metric
* Remove Exact Match Implementation
* Update release notes
* Remove unnecessary metrics implementation
* Simplify logic to run supported metrics
* Add some evaluation tests
* Fix linting
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
2024-01-10 10:26:44 +01:00
Ashwin Mathur
46b395eec3
feat: Add Eval and EvaluationResult ( #6505 )
...
* Add initial implementation for Eval and EvaluationResult
* Add release notes
* Update files with suggestions from review
* Remove serialization
* Add eval e2e tests
* Update eval e2e tests
2023-12-18 11:29:09 +01:00
