David S. Batista
0047cd115e
fix: EvaluationRunResult should compare only input keys ( #7603 )
...
* initial import
* Update haystack/evaluation/eval_run_result.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-26 13:44:18 +00:00
Stefano Fiorucci
704293d491
add pydoc config for evaluation ( #7602 )
2024-04-26 12:30:21 +02:00
Vladimir Blagojevic
36b9a05212
Rollback macos-latest to macos-12 ( #7597 )
2024-04-26 10:44:49 +02:00
Madeesh Kannan
a881451d3a
refactor: Refactor EvaluationResult into BaseEvaluationRunResult and EvaluationRunResult ( #7594 )
...
The new `EvaluationRunResult` has slightly different semantics - it separates the previous `data` parameter into `inputs` and `results`and expects aggregate scores to be provided in the latter.
2024-04-25 12:16:48 +02:00
Madeesh Kannan
ec0e22265a
feat: Expand Pipeline.inputs and Pipeline.outputs to include connected sockets ( #7586 )
2024-04-24 12:27:18 +02:00
Stefano Fiorucci
19a46af9da
add __eq__ method to SparseEmbedding ( #7574 )
...
* add __eq__ method to SparseEmbedding
* reno
* improve reno
2024-04-23 19:03:41 +02:00
David S. Batista
958f1eb3a3
doc: adding docstring linting based on ruff ( #7463 )
...
* wip: docstrings linting
* set ruff rules
2024-04-23 18:43:09 +02:00
Julian Risch
9c56dbe288
test: Make ContextRelevanceEvaluator integration test more robust ( #7584 )
2024-04-23 16:01:25 +00:00
David S. Batista
32db7eb532
initial import ( #7583 )
2024-04-23 17:59:13 +02:00
Julian Risch
07307709ee
test: Make FaithfulnessEvaluator integration test more robust ( #7582 )
2024-04-23 15:44:00 +00:00
David S. Batista
f3b0bd2e6f
Fixing remaining docstring issues ( #7579 )
...
* adding missing docstrings
* adding missing docstrings
* Update haystack/dataclasses/answer.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* reverting some docstrings due to pylint issue, adding a noqa for ruff
* reverting some docstrings due to pylint issue, adding a noqa for ruff
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-04-23 17:24:10 +02:00
David S. Batista
201db5b288
docs: fixing all D205 docstring issues ( #7577 )
...
* fixing all D205 issues
* Update haystack/components/embedders/hugging_face_api_document_embedder.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* Update haystack/components/embedders/hugging_face_api_text_embedder.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* Update haystack/components/generators/chat/hugging_face_api.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* Update haystack/components/generators/chat/hugging_face_local.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* Update haystack/components/generators/hugging_face_api.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* fixing 205 issues and attending PR comments
* fixing 205 issues and attending PR comments
* Update haystack/components/converters/azure.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Update haystack/components/converters/azure.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Update haystack/components/extractors/named_entity_extractor.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Update haystack/components/extractors/named_entity_extractor.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Update haystack/core/component/component.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Update haystack/components/evaluators/answer_exact_match.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Update haystack/core/pipeline/template.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Update haystack/core/serialization.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Update haystack/core/serialization.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Update haystack/core/pipeline/draw.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Update haystack/components/generators/azure.py
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* Apply suggestions from code review
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2024-04-23 13:42:46 +00:00
Stefano Fiorucci
081757c6b9
test: replace mistral-7b with zephyr-7b-beta in tests ( #7576 )
...
* replace mistral-7b with gemma-2b-it in tests
* rm wrong comment
* change model
2024-04-23 13:56:07 +02:00
Julian Risch
d7638cfd4b
refactor: FaithfulnessEvaluator specifies inputs explicitly ( #7548 )
...
* specify inputs explicitly. move out examples
* Update haystack/components/evaluators/faithfulness.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-22 12:52:10 +00:00
Julian Risch
b12e0db134
feat: Add ContextRelevanceEvaluator component ( #7519 )
...
* feat: Add ContextRelevanceEvaluator component
* reno
* fix expected inputs and example docstring
* remove responses parameter from tests
* specify inputs explicitly
* add new evaluator to api reference docs
2024-04-22 14:10:00 +02:00
Massimiliano Pippi
5d0ccfe7d4
fix hatch scripts ( #7546 )
2024-04-12 18:04:18 +02:00
dependabot[bot]
3d0f7affed
chore(deps): bump actions/add-to-project from 1.0.0 to 1.0.1 ( #7547 )
...
Bumps [actions/add-to-project](https://github.com/actions/add-to-project ) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/actions/add-to-project/releases )
- [Commits](https://github.com/actions/add-to-project/compare/v1.0.0...v1.0.1 )
---
updated-dependencies:
- dependency-name: actions/add-to-project
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-12 16:33:24 +02:00
Massimiliano Pippi
3a80c866c9
fix: do not use reserved attributes in the logger ( #7545 )
...
* avoid using reserved keywords in the logger
* make the tests independent from the log level
* relnotes
2024-04-12 14:07:18 +00:00
Daria Fokina
a5f6571cfb
docs: add evaluators component reference ( #7532 )
2024-04-12 12:51:39 +02:00
Massimiliano Pippi
5fce2e90cd
chore: fix linting errors ( #7541 )
...
* fix linting errors
* ellipsis are actually needed
2024-04-12 11:38:59 +02:00
Massimiliano Pippi
2bad5bcb96
refactor: AnswerExactMatchEvaluator component inputs ( #7536 )
...
* refactor component inputs
* release notes
* Update class docstring
* pylint
* update existing note instead of creating a new one
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-04-12 06:59:16 +00:00
Massimiliano Pippi
e90ffafb47
chore: forward hatch command args to pytest ( #7537 )
2024-04-11 21:30:34 +02:00
Massimiliano Pippi
9f33178163
chore: fix docstrings in logging module ( #7539 )
...
* fix docstrings in logging module
* remove unused ellipsis
2024-04-11 17:28:32 +00:00
Massimiliano Pippi
722919c452
use the latest Python image ( #7540 )
2024-04-11 19:05:03 +02:00
David S. Batista
4058882e2e
fixing some more docstrings ( #7530 )
...
* some more docstring fixes
* some more docstring fixes
2024-04-11 18:42:13 +02:00
Massimiliano Pippi
e4013c6dcc
chore: Update CODEOWNERS ( #7538 )
2024-04-11 18:41:43 +02:00
Silvano Cerza
6a8834e43e
fix: Fix corner case when running Pipeline that causes it to get stuck in a loop ( #7531 )
...
* Fix corner case when running Pipeline that causes it to get stuck in a loop
* Update haystack/core/pipeline/pipeline.py
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-04-11 16:39:38 +02:00
David S. Batista
b90a005b85
some more docstring fixes ( #7527 )
2024-04-11 07:42:06 +00:00
Madeesh Kannan
b1760add56
feat: Add support for pipeline deserialization callbacks ( #7518 )
...
* feat: Add support for deserialization callbacks
* Lint
* Fix type hint for older Python versions
* Apply suggestions from code review
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
* Lint
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-04-10 17:47:14 +02:00
Madeesh Kannan
fd84cd5f9a
feat: Add support for returning intermediate outputs of pipeline components ( #7504 )
...
* feat: Add support for returning intermediate outputs of pipeline components
The `pipeline.run` method has been extended to accept a set of component
names whose inputs are returned in addition to the outputs of leaf components.
* Add reno
* Lint
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-04-10 17:16:00 +02:00
David S. Batista
9a9c8aa1c8
feat: implementing evalualtion results API ( #7520 )
...
* initial import
* adding tests
* attending PR comments
* fixing tests
* updating tests
* updating tests and code
* renaming
* fixing linting issues
* adding release notes
* adding docstrings
* latest fixes
2024-04-10 13:34:03 +00:00
Julian Risch
e974a23fa3
docs: Fix eval metric examples in docstrings ( #7505 )
...
* fix eval metric docstrings, change type of individual scores
* change import order
* change exactmatch docstring to single ground truth answer
* change exactmatch comment to single ground truth answer
* reverted changing docs to single ground truth
* add warm up in SASEvaluator example
* fix FaithfulnessEvaluator docstring example
* extend FaithfulnessEvaluator docstring example
* Update FaithfulnessEvaluator init docstring
* Remove outdated default from LLMEvaluator docstring
* Add examples param to LLMEvaluator docstring example
* Add import and print to LLMEvaluator docstring example
2024-04-10 11:00:20 +02:00
David S. Batista
932213e8db
some more docstring fixes ( #7524 )
2024-04-10 08:06:43 +00:00
Stefano Fiorucci
843376bb1b
forward declaration of AnalyzeResult ( #7523 )
2024-04-10 09:02:08 +02:00
Stefano Fiorucci
39be515ba6
skip HF integrations tests if running from fork ( #7517 )
2024-04-09 17:47:13 +02:00
David S. Batista
d9c861e223
Fixing docstrings ( #7513 )
...
* wip
* some more docstring fixes
* reverting to make type checker pass
2024-04-09 16:06:56 +02:00
Stefano Fiorucci
f5719ed12a
fix release notes generation on github release workflow ( #7515 )
2024-04-09 14:47:36 +02:00
Vladimir Blagojevic
988c360b6d
feat: Azure converter updates ( #7409 )
...
* Initial commit
* Remove old mock tests
* Fix current_last_page_number calculation
* Carry over unit tests from the other side
* Update pydocs, skip failing tests
* Fix pylint and mypy
* Minor adjustments
* Add release note
* Minor touch ups
* Resolve Document unique id issue by using custom id calculation
* Better hashing, add unit tests
* Small fixes
2024-04-09 09:45:06 +02:00
Massimiliano Pippi
174ac79b35
chore: add missing docstrings in document store base tests ( #7509 )
...
* add missing docstrings in document store base tests
* also tell pylint to ignore empty docstrings for legacy
2024-04-09 09:25:17 +02:00
Massimiliano Pippi
2dca53f69b
chore: set linting parameters to the minimum ( #7501 )
...
* set line-length to the minimum
* add more defaults
---------
Co-authored-by: David S. Batista <dsbatista@gmail.com>
2024-04-09 08:56:16 +02:00
Massimiliano Pippi
caa67a5fb8
fix docstring for the sample_components package ( #7499 )
2024-04-09 08:51:22 +02:00
David S. Batista
1b10a8304f
proposal: rag evaluation results presentation ( #7462 )
...
* adding files
* adding proposal in md
* renaming proposal number
* removing stuff
* cleaning up
* adding PR number and issue
* updating proposal
* updating proposal
* Update proposals/text/7462-rag-evaluation.md
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* changing name
* PR comments
* changing output to table format
* adding user stories
* Update proposals/text/7462-rag-evaluation.md
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* adding user stories
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-08 18:37:48 +02:00
Silvano Cerza
be77e85f43
Fix inaccuracy in PromptBuilder docstring ( #7503 )
2024-04-08 16:44:45 +02:00
Stefano Fiorucci
eff53a9131
feat: HuggingFaceAPIDocumentEmbedder ( #7485 )
...
* add HuggingFaceAPITextEmbedder
* add HuggingFaceAPITextEmbedder
* rm unneeded else
* wip
* small fixes
* deprecation; reno
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* make params mandatory
* changes requested
* fix test
* fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-08 15:06:26 +02:00
Stefano Fiorucci
c91bd49cae
feat: HuggingFaceAPITextEmbedder ( #7484 )
...
* add HuggingFaceAPITextEmbedder
* add HuggingFaceAPITextEmbedder
* rm unneeded else
* small fixes
* changes requested
* fix test
2024-04-08 14:22:54 +02:00
David S. Batista
3777f4342f
just some more quick fixes ( #7498 )
...
* just some more quick fixes
* Update haystack/utils/url_validation.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-08 09:11:11 +00:00
Massimiliano Pippi
b20a14d6fe
fix docstrints format for tracing package ( #7497 )
2024-04-08 10:51:25 +02:00
David S. Batista
aae2b31359
fix: typo in sas_evaluator arg ( #7486 )
...
* fixing typo on SAS arg
* fixing tests
* fixing tests
2024-04-08 10:21:37 +02:00
Stefano Fiorucci
0dbb98c0a0
feat: HuggingFaceAPIChatGenerator ( #7480 )
...
* draft
* docstrings and more tests
* deprecation; reno
* pydoc config
* better error messages
* wip
* add test
* better docstrings
* deprecation; reno
* pylint
* typo
* rm unneeded else
* rm unneeded else
* fixes from feedback
* docstring showing the enum
* improve docstring
* make params mandatory
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* document enum
* Update haystack/utils/hf.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* mandatory params
* fix test
* fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-05 18:48:34 +02:00
Stefano Fiorucci
1d083861ff
feat: HuggingFaceAPIGenerator ( #7464 )
...
* draft
* docstrings and more tests
* deprecation; reno
* pydoc config
* better error messages
* rm unneeded else
* make params mandatory
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* document enum
* Update haystack/utils/hf.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-05 18:48:13 +02:00
