David S. Batista
da60156174
chore: removing unused imports from tests ( #9446 )
2025-05-26 16:22:51 +00:00
Stefano Fiorucci
656fe6dc6e
chore: LLM Evaluators - remove deprecated parameters ( #9219 )
2025-04-15 09:26:31 +02:00
Stefano Fiorucci
c6df8d2c7a
test: monkeypatch OpenAI API key in some unit tests ( #9173 )
2025-04-04 13:33:22 +02:00
Stefano Fiorucci
adc3dfc5d2
refactor: LLM evaluators - introduce chat_generator init param; deprecate api, api_key and api_params ( #9122 )
...
* start
* progress
* tests for deserialize_chatgenerator_inplace
* progress on llmevaluator + tests
* update context relevance evaluator
* update faithfulness evaluator + tests
* release note
* rm unused import
* rm indentation
2025-03-31 15:35:03 +02:00
Stefano Fiorucci
e4cf460bf6
refactor!: use Chat Generator in LLM evaluators ( #9116 )
...
* use chatgenerator instead of generator
* rename generator to _chat_generator
* rm print
* Update releasenotes/notes/llm-evaluators-chat-generator-bf930fa6db019714.yaml
Co-authored-by: David S. Batista <dsbatista@gmail.com>
---------
Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-03-26 15:38:56 +01:00
Ajit Singh
6cf13e8b98
enhancement: reduced usage of numpy and substituted built-in libraries ( #8418 )
...
* reduced usage of numpy and substituted built-in libraries
* added release note
* edited expit function to support both float as well as list (this case was giving error CI)
* revert code , numpy can't be removed here
* more cleaning
* fix relnote
---------
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2024-10-18 15:42:19 +02:00
Madeesh Kannan
672bcf7e03
fix: Add constraints to set_input_type(s) based on run method ( #8358 )
...
* fix: Prevent the usage of `set_input_type(s)` when the `run` method doesn't have kwargs,
raise if `set_input_type(s)` overrides `run` method parameters
* fix: update components and tests
* reno
2024-09-12 15:58:16 +02:00
Ulises M
6f8834d036
feat: add and expose api_params for OpenAIGenerator in LLMEvaluator based classes ( #7987 )
...
* initial support for api_params
* add tests and reno
* resolve suggestions and add integration test
* fix mypy
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-07-11 13:14:03 +02:00
David S. Batista
186512459d
feat: LLM-based evaluators return meta info from OpenAI ( #7947 )
...
* LLM-Evaluator returns metadata from OpenAI
* adding tests
* adding release notes
* updating test
* updating release notes
* fixing live tests
* attending PR comments
* fixing tests
* Update releasenotes/notes/adding-metadata-info-from-OpenAI-f5309af5f59bb6a7.yaml
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* Update llm_evaluator.py
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-07-02 11:31:51 +02:00
Madeesh Kannan
fe60eedee9
fix: Fix deserialization of pipelines that contain LLMEvaluator subclasses ( #7891 )
2024-06-19 13:47:38 +02:00
David S. Batista
38747ff7a3
fix: failsafe for non-valid json and failed LLM calls ( #7723 )
...
* wip
* initial import
* adding tests
* adding params
* adding safeguards for nan in evaluators
* adding docstrings
* fixing tests
* removing unused imports
* adding tests to context and faithfullness evaluators
* fixing docstrings
* nit
* removing unused imports
* adding release notes
* attending PR comments
* fixing tests
* fixing tests
* adding types
* removing unused imports
* Update haystack/components/evaluators/context_relevance.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/evaluators/faithfulness.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* attending PR comments
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-05-23 15:41:29 +00:00
David S. Batista
798dc4a4a5
fix: avoid FaithfulnessEvaluator and ContextRelevanceEvaluator return Nan ( #7685 )
...
* initial import
* fixing tests
* relaxing condition
* adding safeguard for ContextRelevanceEvaluator as well
* adding release notes
2024-05-14 17:08:51 +02:00
Massimiliano Pippi
10c675d534
chore: add license header to all modules ( #7675 )
...
* add license header to modules
* check license header at linting time
2024-05-09 13:40:36 +00:00
Julian Risch
2509eeea7e
refactor: Rename FaithfulnessEvaluator input responses to predicted_answers ( #7621 )
2024-04-30 16:30:57 +02:00
Julian Risch
07307709ee
test: Make FaithfulnessEvaluator integration test more robust ( #7582 )
2024-04-23 15:44:00 +00:00
Julian Risch
d7638cfd4b
refactor: FaithfulnessEvaluator specifies inputs explicitly ( #7548 )
...
* specify inputs explicitly. move out examples
* Update haystack/components/evaluators/faithfulness.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-22 12:52:10 +00:00
Julian Risch
9d02dc607a
feat: Add FaithfulnessEvaluator component ( #7424 )
...
* draft FaithfulnessEvaluator
* reno
* calculate score per statement and aggregate
* Update release note
* update default values in tests and fix import path
* remove instructions, inputs, outputs params
* remove unused imports
* add expected format example to docstring
* remove name 'llm' from tests and docstring
2024-04-04 16:33:59 +00:00
