David S. Batista
0c9dc008f0
fix: improve context relevancy metric ( #7964 )
...
* fixing tests
* fixing tests
* updating tests
* updating tests
* updating docstring
* adding release notes
* making the insufficient information more robust
* updating docstring and release notes
* empty list instead of informative string
* Update haystack/components/evaluators/context_relevance.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/evaluators/context_relevance.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* fixing tests
* Update haystack/components/evaluators/context_relevance.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* reverting commit
* reverting again commit
* fixing docstrings
* removing deprecation warning
* removing warning import
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-07-22 15:13:46 +02:00
David S. Batista
55513f7521
feat: EvaluationRunResult add parameter to specify columns to keep in the comparative Dataframe ( #7879 )
...
* adding param to explictily state which cols to keep
* adding param to explictily state which cols to keep
* adding param to explictily state which cols to keep
* updating tests
* adding release notes
* Update haystack/evaluation/eval_run_result.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update releasenotes/notes/add-keep-columns-to-EvalRunResult-comparative-be3e15ce45de3e0b.yaml
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* updating docstring
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-06-17 18:08:52 +02:00
David S. Batista
ce9b0ecb19
fix: EvaluationRunResult.score_report() is missing the metrics column ( #7817 )
...
* fixing the DataFrame with the aggregated scores
* fixing tests
2024-06-06 14:33:45 +02:00
Massimiliano Pippi
10c675d534
chore: add license header to all modules ( #7675 )
...
* add license header to modules
* check license header at linting time
2024-05-09 13:40:36 +00:00
Madeesh Kannan
a881451d3a
refactor: Refactor EvaluationResult into BaseEvaluationRunResult and EvaluationRunResult ( #7594 )
...
The new `EvaluationRunResult` has slightly different semantics - it separates the previous `data` parameter into `inputs` and `results`and expects aggregate scores to be provided in the latter.
2024-04-25 12:16:48 +02:00
Silvano Cerza
cf221a9701
Delete old eval API ( #6983 )
2024-02-14 17:11:08 +01:00
Silvano Cerza
36ab23d360
feat: Add StatisticalEvaluator component ( #6982 )
...
* Add StatisticalEvaluator component
* Remove F1 and Exact Metric from old API
* Add release notes
* Update docstrings
2024-02-14 16:48:03 +01:00
Silvano Cerza
9297fca520
feat: Add SASEvaluator component ( #6980 )
...
* Add SASEvaluator component
* Add release notes
* Delete old tests
* Remove SAS metric in old API
* Avoid importing whole numpy package
2024-02-14 16:16:22 +01:00
Ashwin Mathur
393a7993c3
feat: Add Semantic Answer Similarity metric ( #6877 )
...
* Add SAS metric
* Add release notes
* Round similarity scores for precision consistency
* Add tolerance to tests
* Update haystack/evaluation/eval.py
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
* Add types for preprocess_text; Add additional types for f1 and em methods
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-02-02 17:07:52 +01:00
Ashwin Mathur
7217f9d9f0
feat: Add F1 metric ( #6822 )
...
* Add F1 metric
* Add release notes
2024-01-26 11:04:43 +01:00
Ashwin Mathur
a238c6dd51
feat: Add Exact Match metric ( #6696 )
...
* Add exact match metric
* Add release notes
* Cleanup comments in test_eval_exact_match.py
* Create separate preprocessing function; Add output_key parameter
* Update release note
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-01-22 09:57:04 +01:00
Ashwin Mathur
374a937663
feat: Add calculate_metrics and MetricsResult ( #6680 )
...
* Add calculate_metrics, MetricsResult, Exact Match
* Add additional tests for metric calculation
* Add release notes
* Add docstring for Exact Match metric
* Remove Exact Match Implementation
* Update release notes
* Remove unnecessary metrics implementation
* Simplify logic to run supported metrics
* Add some evaluation tests
* Fix linting
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
2024-01-10 10:26:44 +01:00
