Silvano Cerza
685343d13f
feat: Add DocumentRecallEvaluator ( #7399 )
...
* Add DocumentRecallEvaluator
* Fix mypy error
* Simplify recall logic and change output for single hit mode
* Remove unused import
* Add comment for RecallMode fields
* Reword RecallMode comments
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-03-26 16:15:03 +01:00
Silvano Cerza
f398b29e7f
feat: Change outputs of AnswerExactMatchEvaluator ( #7390 )
...
* Change outputs of AnswerExactMatchEvaluator
* Changes scores to return the number of matches per question
* Revert "Changes scores to return the number of matches per question"
This reverts commit e4358720793d4584b0b961402d4557c50c4c2381.
* Change output names
2024-03-26 10:57:59 +01:00
Julian Risch
bfd0d3eacd
feat: Add new LLMEvaluator component ( #7401 )
...
* draft llm evaluator
* docstrings
* flexible inputs; validate inputs and outputs
* add tests
* add release note
* remove example
* docstrings
* make outputs parameter optional. default:
* validate init parameters
* linting
* remove mention of binary scores from template
* make examples and outputs params non-optional
* removed leftover from optional outputs param
* simplify building examples section for template
* validate inputs and outputs in examples are dict with str as key
* fix pylint too-many-boolean-expressions
* increase test coverage
2024-03-25 07:05:27 +01:00
Silvano Cerza
610ad6f6b2
Add AnswerExactMatchEvaluator ( #7381 )
...
* Add AnswerExactMatchEvaluator
* Add release notes
* Fix linting
* Update docstrings
* Update docstrings
* Remove to_dict and from_dict
* Fix linting
2024-03-19 16:58:01 +01:00
Silvano Cerza
0a7dfc1b32
Revert "Add AnswerExactMatchEvaluator ( #7050 )" ( #7075 )
...
This reverts commit b4011af8e9bc4ae2f72e51db254bfda69e20b651.
2024-02-23 14:05:57 +01:00
Silvano Cerza
b4011af8e9
Add AnswerExactMatchEvaluator ( #7050 )
...
* Add AnswerExactMatchEvaluator
* Add release notes
* Fix linting
* Update docstrings
2024-02-23 10:37:18 +01:00
Silvano Cerza
8ca4bf405b
Remove all evaluator components ( #7053 )
2024-02-21 18:24:14 +01:00
Ashwin Mathur
327c2d260d
feat: Add Mean Reciprocal Rank (MRR) metric to StatisticalEvaluator ( #7042 )
...
* Add MRR Metric
* Add release notes
* Update logic
2024-02-20 13:58:48 +01:00
Silvano Cerza
9215882779
Add Recall Multi Hit and Single Hit metric ( #7038 )
2024-02-19 18:00:39 +01:00
Silvano Cerza
6fe1d3b595
refactor: Clean eval components ( #7005 )
...
* Remove preprocess.py
* Rename eval components to evaluators
2024-02-15 17:17:59 +01:00
