3803 Commits

Author SHA1 Message Date
evanderiel
5de5619abd
Add instance argument to code samples in docstrings for component.py (#7622) 2024-04-30 16:04:06 +02:00
Vladimir Blagojevic
8cb3cecf34
feat: Trace pipeline run input/output data (#7590)
* Trace pipeline run

* Add reno note

* Update tracing tests to check input_data and output_data

* empty

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-04-29 17:29:27 +02:00
Tobias Wochinger
451fae880e
ci: fix catch-all (#7215)
* ci: trigger separate workflow

* ci: temporary use current branch

* ci: fix workflow name

* ci: try with same job name

* ci: try with dispatch

* Revert "ci: try with dispatch"

This reverts commit bd66e56c0697ae97fc2599eebaceff417d9be65c.

* Revert "ci: try with same job name"

This reverts commit 9e2ae5b402758c14a9f812c2e06f820bd3ece767.

* ci: try with workflow call in both cases

* ci: introduce change to trigger CI

* Revert "ci: introduce change to trigger CI"

This reverts commit e3ec07c5e26f114364babea69535183253c801b7.

* ci: add name

* Revert "Revert "ci: introduce change to trigger CI""

This reverts commit 6718585fd24069112e0f773e010056e1d96e3eee.

* ci: improve naming

* ci: further improve naming

* Unset reusable workflow version and use relative path

* Remove CI trigger

---------

Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
2024-04-29 14:54:12 +02:00
Bohan Qu
40360e44ff
feat: add required flag for prompt builder inputs (#7553) 2024-04-29 14:21:53 +02:00
Carlos Fernández
d2c87b2fd9
feat: add page_number to metadata in DocumentSplitter (#7599)
* Add the implementation for page counting used in the v1.25.x branch. It should work as expected in issue #6705.

* Add tests that reflect the desired behabiour. This behabiour is inffered from the one it had on Haystack 1.x
Solve some minor bugs spotted by tests.

* Update docstrings.

* Add reno.

* Update haystack/components/preprocessors/document_splitter.py

Update docstring from suggestion

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* solve suggestion to improve readability

* fragment tests

* Update haystack/components/preprocessors/document_splitter.py

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* Update .gitignore

* Update .gitignore

* Update add-page-number-to-document-splitter-162e9dc7443575f0.yaml

* blackening

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2024-04-29 12:51:18 +02:00
David S. Batista
8d04e530da
test: end2end evaluation tests (#7601)
* initial import

* wip

* cleaning up tests

* fixing tests

* adding context relevance

* reverting some wrong changes to due PyCharm error in refactoring

* building eval pipeline only once

* handling mypy issues
2024-04-26 14:07:05 +00:00
David S. Batista
0047cd115e
fix: EvaluationRunResult should compare only input keys (#7603)
* initial import

* Update haystack/evaluation/eval_run_result.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-26 13:44:18 +00:00
Stefano Fiorucci
704293d491
add pydoc config for evaluation (#7602) 2024-04-26 12:30:21 +02:00
Vladimir Blagojevic
36b9a05212
Rollback macos-latest to macos-12 (#7597) 2024-04-26 10:44:49 +02:00
Madeesh Kannan
a881451d3a
refactor: Refactor EvaluationResult into BaseEvaluationRunResult and EvaluationRunResult (#7594)
The new `EvaluationRunResult` has slightly different semantics - it separates the previous `data` parameter into `inputs` and `results`and expects aggregate scores to be provided in the latter.
2024-04-25 12:16:48 +02:00
Madeesh Kannan
ec0e22265a
feat: Expand Pipeline.inputs and Pipeline.outputs to include connected sockets (#7586) 2024-04-24 12:27:18 +02:00
Stefano Fiorucci
19a46af9da
add __eq__ method to SparseEmbedding (#7574)
* add __eq__ method to SparseEmbedding

* reno

* improve reno
2024-04-23 19:03:41 +02:00
David S. Batista
958f1eb3a3
doc: adding docstring linting based on ruff (#7463)
* wip: docstrings linting

* set ruff rules
2024-04-23 18:43:09 +02:00
Julian Risch
9c56dbe288
test: Make ContextRelevanceEvaluator integration test more robust (#7584) 2024-04-23 16:01:25 +00:00
David S. Batista
32db7eb532
initial import (#7583) 2024-04-23 17:59:13 +02:00
Julian Risch
07307709ee
test: Make FaithfulnessEvaluator integration test more robust (#7582) 2024-04-23 15:44:00 +00:00
David S. Batista
f3b0bd2e6f
Fixing remaining docstring issues (#7579)
* adding missing docstrings

* adding missing docstrings

* Update haystack/dataclasses/answer.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* reverting some docstrings due to pylint issue, adding a noqa for ruff

* reverting some docstrings due to pylint issue, adding a noqa for ruff

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-04-23 17:24:10 +02:00
David S. Batista
201db5b288
docs: fixing all D205 docstring issues (#7577)
* fixing all D205 issues

* Update haystack/components/embedders/hugging_face_api_document_embedder.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/embedders/hugging_face_api_text_embedder.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/generators/chat/hugging_face_api.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/generators/chat/hugging_face_local.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/generators/hugging_face_api.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* fixing 205 issues and attending PR comments

* fixing 205 issues and attending PR comments

* Update haystack/components/converters/azure.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/converters/azure.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/extractors/named_entity_extractor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/extractors/named_entity_extractor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/core/component/component.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/evaluators/answer_exact_match.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/core/pipeline/template.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/core/serialization.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/core/serialization.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/core/pipeline/draw.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/azure.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Apply suggestions from code review

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2024-04-23 13:42:46 +00:00
Stefano Fiorucci
081757c6b9
test: replace mistral-7b with zephyr-7b-beta in tests (#7576)
* replace mistral-7b with gemma-2b-it in tests

* rm wrong comment

* change model
2024-04-23 13:56:07 +02:00
Julian Risch
d7638cfd4b
refactor: FaithfulnessEvaluator specifies inputs explicitly (#7548)
* specify inputs explicitly. move out examples

* Update haystack/components/evaluators/faithfulness.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-22 12:52:10 +00:00
Julian Risch
b12e0db134
feat: Add ContextRelevanceEvaluator component (#7519)
* feat: Add ContextRelevanceEvaluator component

* reno

* fix expected inputs and example docstring

* remove responses parameter from tests

* specify inputs explicitly

* add new evaluator to api reference docs
2024-04-22 14:10:00 +02:00
Massimiliano Pippi
5d0ccfe7d4
fix hatch scripts (#7546) 2024-04-12 18:04:18 +02:00
dependabot[bot]
3d0f7affed
chore(deps): bump actions/add-to-project from 1.0.0 to 1.0.1 (#7547)
Bumps [actions/add-to-project](https://github.com/actions/add-to-project) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/actions/add-to-project/releases)
- [Commits](https://github.com/actions/add-to-project/compare/v1.0.0...v1.0.1)

---
updated-dependencies:
- dependency-name: actions/add-to-project
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-12 16:33:24 +02:00
Massimiliano Pippi
3a80c866c9
fix: do not use reserved attributes in the logger (#7545)
* avoid using reserved keywords in the logger

* make the tests independent from the log level

* relnotes
2024-04-12 14:07:18 +00:00
Daria Fokina
a5f6571cfb
docs: add evaluators component reference (#7532) 2024-04-12 12:51:39 +02:00
Massimiliano Pippi
5fce2e90cd
chore: fix linting errors (#7541)
* fix linting errors

* ellipsis are actually needed
2024-04-12 11:38:59 +02:00
Massimiliano Pippi
2bad5bcb96
refactor: AnswerExactMatchEvaluator component inputs (#7536)
* refactor component inputs

* release notes

* Update class docstring

* pylint

* update existing note instead of creating a new one

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-04-12 06:59:16 +00:00
Massimiliano Pippi
e90ffafb47
chore: forward hatch command args to pytest (#7537) 2024-04-11 21:30:34 +02:00
Massimiliano Pippi
9f33178163
chore: fix docstrings in logging module (#7539)
* fix docstrings in logging module

* remove unused ellipsis
2024-04-11 17:28:32 +00:00
Massimiliano Pippi
722919c452
use the latest Python image (#7540) 2024-04-11 19:05:03 +02:00
David S. Batista
4058882e2e
fixing some more docstrings (#7530)
* some more docstring fixes

* some more docstring fixes
2024-04-11 18:42:13 +02:00
Massimiliano Pippi
e4013c6dcc
chore: Update CODEOWNERS (#7538) 2024-04-11 18:41:43 +02:00
Silvano Cerza
6a8834e43e
fix: Fix corner case when running Pipeline that causes it to get stuck in a loop (#7531)
* Fix corner case when running Pipeline that causes it to get stuck in a loop

* Update haystack/core/pipeline/pipeline.py

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>

---------

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-04-11 16:39:38 +02:00
David S. Batista
b90a005b85
some more docstring fixes (#7527) 2024-04-11 07:42:06 +00:00
Madeesh Kannan
b1760add56
feat: Add support for pipeline deserialization callbacks (#7518)
* feat: Add support for deserialization callbacks

* Lint

* Fix type hint for older Python versions

* Apply suggestions from code review

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* Lint

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-04-10 17:47:14 +02:00
Madeesh Kannan
fd84cd5f9a
feat: Add support for returning intermediate outputs of pipeline components (#7504)
* feat: Add support for returning intermediate outputs of pipeline components

The `pipeline.run` method has been extended to accept a set of component
names whose inputs are returned in addition to the outputs of leaf components.

* Add reno

* Lint

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-04-10 17:16:00 +02:00
David S. Batista
9a9c8aa1c8
feat: implementing evalualtion results API (#7520)
* initial import

* adding tests

* attending PR comments

* fixing tests

* updating tests

* updating tests and code

* renaming

* fixing linting issues

* adding release notes

* adding docstrings

* latest fixes
2024-04-10 13:34:03 +00:00
Julian Risch
e974a23fa3
docs: Fix eval metric examples in docstrings (#7505)
* fix eval metric docstrings, change type of individual scores

* change import order

* change exactmatch docstring to single ground truth answer

* change exactmatch comment to single ground truth answer

* reverted changing docs to single ground truth

* add warm up in SASEvaluator example

* fix FaithfulnessEvaluator docstring example

* extend FaithfulnessEvaluator docstring example

* Update FaithfulnessEvaluator init docstring

* Remove outdated default from LLMEvaluator docstring

* Add examples param to LLMEvaluator docstring example

* Add import and print to LLMEvaluator docstring example
2024-04-10 11:00:20 +02:00
David S. Batista
932213e8db
some more docstring fixes (#7524) 2024-04-10 08:06:43 +00:00
Stefano Fiorucci
843376bb1b
forward declaration of AnalyzeResult (#7523) 2024-04-10 09:02:08 +02:00
Stefano Fiorucci
39be515ba6
skip HF integrations tests if running from fork (#7517) 2024-04-09 17:47:13 +02:00
David S. Batista
d9c861e223
Fixing docstrings (#7513)
* wip

* some more docstring fixes

* reverting to make type checker pass
2024-04-09 16:06:56 +02:00
Stefano Fiorucci
f5719ed12a
fix release notes generation on github release workflow (#7515) 2024-04-09 14:47:36 +02:00
Vladimir Blagojevic
988c360b6d
feat: Azure converter updates (#7409)
* Initial commit

* Remove old mock tests

* Fix current_last_page_number calculation

* Carry over unit tests from the other side

* Update pydocs, skip failing tests

* Fix pylint and mypy

* Minor adjustments

* Add release note

* Minor touch ups

* Resolve Document unique id issue by using custom id calculation

* Better hashing, add unit tests

* Small fixes
2024-04-09 09:45:06 +02:00
Massimiliano Pippi
174ac79b35
chore: add missing docstrings in document store base tests (#7509)
* add missing docstrings in document store base tests

* also tell pylint to ignore empty docstrings for legacy
2024-04-09 09:25:17 +02:00
Massimiliano Pippi
2dca53f69b
chore: set linting parameters to the minimum (#7501)
* set line-length to the minimum

* add more defaults

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2024-04-09 08:56:16 +02:00
Massimiliano Pippi
caa67a5fb8
fix docstring for the sample_components package (#7499) 2024-04-09 08:51:22 +02:00
David S. Batista
1b10a8304f
proposal: rag evaluation results presentation (#7462)
* adding files

* adding proposal in md

* renaming proposal number

* removing stuff

* cleaning up

* adding PR number and issue

* updating proposal

* updating proposal

* Update proposals/text/7462-rag-evaluation.md

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* changing name

* PR comments

* changing output to table format

* adding user stories

* Update proposals/text/7462-rag-evaluation.md

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* adding user stories

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-08 18:37:48 +02:00
Silvano Cerza
be77e85f43
Fix inaccuracy in PromptBuilder docstring (#7503) 2024-04-08 16:44:45 +02:00
Stefano Fiorucci
eff53a9131
feat: HuggingFaceAPIDocumentEmbedder (#7485)
* add HuggingFaceAPITextEmbedder

* add HuggingFaceAPITextEmbedder

* rm unneeded else

* wip

* small fixes

* deprecation; reno

* Apply suggestions from code review

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* make params mandatory

* changes requested

* fix test

* fix test

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-08 15:06:26 +02:00