Madeesh Kannan
fd84cd5f9a
feat: Add support for returning intermediate outputs of pipeline components ( #7504 )
...
* feat: Add support for returning intermediate outputs of pipeline components
The `pipeline.run` method has been extended to accept a set of component
names whose inputs are returned in addition to the outputs of leaf components.
* Add reno
* Lint
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-04-10 17:16:00 +02:00
David S. Batista
9a9c8aa1c8
feat: implementing evalualtion results API ( #7520 )
...
* initial import
* adding tests
* attending PR comments
* fixing tests
* updating tests
* updating tests and code
* renaming
* fixing linting issues
* adding release notes
* adding docstrings
* latest fixes
2024-04-10 13:34:03 +00:00
Julian Risch
e974a23fa3
docs: Fix eval metric examples in docstrings ( #7505 )
...
* fix eval metric docstrings, change type of individual scores
* change import order
* change exactmatch docstring to single ground truth answer
* change exactmatch comment to single ground truth answer
* reverted changing docs to single ground truth
* add warm up in SASEvaluator example
* fix FaithfulnessEvaluator docstring example
* extend FaithfulnessEvaluator docstring example
* Update FaithfulnessEvaluator init docstring
* Remove outdated default from LLMEvaluator docstring
* Add examples param to LLMEvaluator docstring example
* Add import and print to LLMEvaluator docstring example
2024-04-10 11:00:20 +02:00
David S. Batista
932213e8db
some more docstring fixes ( #7524 )
2024-04-10 08:06:43 +00:00
Stefano Fiorucci
843376bb1b
forward declaration of AnalyzeResult ( #7523 )
2024-04-10 09:02:08 +02:00
Stefano Fiorucci
39be515ba6
skip HF integrations tests if running from fork ( #7517 )
2024-04-09 17:47:13 +02:00
David S. Batista
d9c861e223
Fixing docstrings ( #7513 )
...
* wip
* some more docstring fixes
* reverting to make type checker pass
2024-04-09 16:06:56 +02:00
Stefano Fiorucci
f5719ed12a
fix release notes generation on github release workflow ( #7515 )
2024-04-09 14:47:36 +02:00
Vladimir Blagojevic
988c360b6d
feat: Azure converter updates ( #7409 )
...
* Initial commit
* Remove old mock tests
* Fix current_last_page_number calculation
* Carry over unit tests from the other side
* Update pydocs, skip failing tests
* Fix pylint and mypy
* Minor adjustments
* Add release note
* Minor touch ups
* Resolve Document unique id issue by using custom id calculation
* Better hashing, add unit tests
* Small fixes
2024-04-09 09:45:06 +02:00
Massimiliano Pippi
174ac79b35
chore: add missing docstrings in document store base tests ( #7509 )
...
* add missing docstrings in document store base tests
* also tell pylint to ignore empty docstrings for legacy
2024-04-09 09:25:17 +02:00
Massimiliano Pippi
2dca53f69b
chore: set linting parameters to the minimum ( #7501 )
...
* set line-length to the minimum
* add more defaults
---------
Co-authored-by: David S. Batista <dsbatista@gmail.com>
2024-04-09 08:56:16 +02:00
Massimiliano Pippi
caa67a5fb8
fix docstring for the sample_components package ( #7499 )
2024-04-09 08:51:22 +02:00
David S. Batista
1b10a8304f
proposal: rag evaluation results presentation ( #7462 )
...
* adding files
* adding proposal in md
* renaming proposal number
* removing stuff
* cleaning up
* adding PR number and issue
* updating proposal
* updating proposal
* Update proposals/text/7462-rag-evaluation.md
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* changing name
* PR comments
* changing output to table format
* adding user stories
* Update proposals/text/7462-rag-evaluation.md
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* adding user stories
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-08 18:37:48 +02:00
Silvano Cerza
be77e85f43
Fix inaccuracy in PromptBuilder docstring ( #7503 )
2024-04-08 16:44:45 +02:00
Stefano Fiorucci
eff53a9131
feat: HuggingFaceAPIDocumentEmbedder ( #7485 )
...
* add HuggingFaceAPITextEmbedder
* add HuggingFaceAPITextEmbedder
* rm unneeded else
* wip
* small fixes
* deprecation; reno
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* make params mandatory
* changes requested
* fix test
* fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-08 15:06:26 +02:00
Stefano Fiorucci
c91bd49cae
feat: HuggingFaceAPITextEmbedder ( #7484 )
...
* add HuggingFaceAPITextEmbedder
* add HuggingFaceAPITextEmbedder
* rm unneeded else
* small fixes
* changes requested
* fix test
2024-04-08 14:22:54 +02:00
David S. Batista
3777f4342f
just some more quick fixes ( #7498 )
...
* just some more quick fixes
* Update haystack/utils/url_validation.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-08 09:11:11 +00:00
Massimiliano Pippi
b20a14d6fe
fix docstrints format for tracing package ( #7497 )
2024-04-08 10:51:25 +02:00
David S. Batista
aae2b31359
fix: typo in sas_evaluator arg ( #7486 )
...
* fixing typo on SAS arg
* fixing tests
* fixing tests
2024-04-08 10:21:37 +02:00
Stefano Fiorucci
0dbb98c0a0
feat: HuggingFaceAPIChatGenerator ( #7480 )
...
* draft
* docstrings and more tests
* deprecation; reno
* pydoc config
* better error messages
* wip
* add test
* better docstrings
* deprecation; reno
* pylint
* typo
* rm unneeded else
* rm unneeded else
* fixes from feedback
* docstring showing the enum
* improve docstring
* make params mandatory
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* document enum
* Update haystack/utils/hf.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* mandatory params
* fix test
* fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-05 18:48:34 +02:00
Stefano Fiorucci
1d083861ff
feat: HuggingFaceAPIGenerator ( #7464 )
...
* draft
* docstrings and more tests
* deprecation; reno
* pydoc config
* better error messages
* rm unneeded else
* make params mandatory
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* document enum
* Update haystack/utils/hf.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-05 18:48:13 +02:00
Silvano Cerza
ff269db12d
Fix unit tests failing if HF_API_TOKEN is set ( #7491 )
2024-04-05 18:05:43 +02:00
Silvano Cerza
65705a8fdc
Add HF_API_TOKEN env var in tests.yml ( #7490 )
2024-04-05 17:02:26 +02:00
David S. Batista
5592622c93
linting some docstrings ( #7483 )
...
* fixing docstrings
* fixing docstrings
2024-04-05 14:45:49 +00:00
Bilge Yücel
b8fc86eb6a
Update Docker README.md ( #7369 )
...
* Update Docker README.md
* mention 1.x/2.0
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-04-05 16:20:56 +02:00
Vladimir Blagojevic
c3b96392fd
feat: Use all HTMLToDocument extractors until content is extracted ( #7452 )
...
* Use all HTMLToDocument extractors until content is extracted
* Add release note
* Minor doc update
* Improvements, unit test fixes
* Add try_others init param, update tests
* Update haystack/components/converters/html.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* PR feedback - Stefano
* Improve reno release note, add reference
* little fixes
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-04-05 16:02:34 +02:00
Julian Risch
9d02dc607a
feat: Add FaithfulnessEvaluator component ( #7424 )
...
* draft FaithfulnessEvaluator
* reno
* calculate score per statement and aggregate
* Update release note
* update default values in tests and fix import path
* remove instructions, inputs, outputs params
* remove unused imports
* add expected format example to docstring
* remove name 'llm' from tests and docstring
2024-04-04 16:33:59 +00:00
Silvano Cerza
189dfaf640
Export evaluators ( #7469 )
2024-04-04 17:39:35 +02:00
Julian Risch
8ef6062748
refactor: Remove name 'llm' from LLMEvaluator output ( #7479 )
2024-04-04 15:19:30 +00:00
Silvano Cerza
8b8a93bc0d
refactor: Rename DocumentMeanAveragePrecision and DocumentMeanReciprocalRank ( #7470 )
...
* Rename DocumentMeanAveragePrecision and DocumentMeanReciprocalRank
* Update releasenotes
* Simplify names
2024-04-04 17:04:59 +02:00
David S. Batista
bf8453e48e
fixing some docstrings ( #7465 )
2024-04-04 16:54:33 +02:00
Silvano Cerza
bdc25ca2a0
feat: Add DocumentMeanReciprocalRank ( #7468 )
...
* Add DocumentMeanReciprocalRank
* Fix float precision error
2024-04-04 14:55:37 +02:00
Silvano Cerza
7799909069
feat: Add DocumentMeanAveragePrecision ( #7461 )
...
* Add DocumentMeanAveragePrecision
* Remove questions input
* Update docstrings
* Update haystack/components/evaluators/document_map.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-04 14:15:45 +02:00
Silvano Cerza
dc87f51759
refactor: Remove questions inputs from evaluators ( #7466 )
...
* Remove questions input from AnswerExactMatchEvaluator
* Remove questions input from DocumentRecallEvaluator
2024-04-04 14:14:18 +02:00
Silvano Cerza
12acb3f12e
feat: Add SASEvaluator ( #7428 )
...
* Add SASEvaluator
* Add release notes
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Simplify similarity calculation with bi-encoders models
* Fix linting
* Update docstrings
* Move tensor to CPU after calculating cosine similarity
* Fix CI failing
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-04 10:10:41 +02:00
Ashwin Mathur
1c7d1618d8
Add truncate and normalize parameters to TEI Embedders ( #7460 )
2024-04-03 16:41:30 +02:00
Silvano Cerza
1ce12c7a6a
Remove example ( #7458 )
2024-04-03 14:27:43 +02:00
Vladimir Blagojevic
d83af92270
feat: Update searchapi format, default to Google, allow search engine selection ( #7453 )
...
* Update searchapi payload
* Add release note
* PR feedback - Stefano
* Adjust unit test for mandatory engine search_param field
2024-04-03 10:48:50 +02:00
Nicola Procopio
42c5b7af32
feat: added dimensions parameters to Azure OpenAI Embedders ( #7449 )
...
* added dimensions parameter to AzureOpenAIEmbedders
* created releasenote
* update release note
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-04-02 14:04:16 +02:00
Silvano Cerza
6e289698e9
fix: Fix Pipeline.run() getting stuck in a loop even though there are components that can run ( #7434 )
2024-03-28 12:31:36 +01:00
dependabot[bot]
6fcb62ae34
chore(deps): bump actions/add-to-project from 0.6.1 to 1.0.0 ( #7436 )
...
Bumps [actions/add-to-project](https://github.com/actions/add-to-project ) from 0.6.1 to 1.0.0.
- [Release notes](https://github.com/actions/add-to-project/releases )
- [Commits](https://github.com/actions/add-to-project/compare/v0.6.1...v1.0.0 )
---
updated-dependencies:
- dependency-name: actions/add-to-project
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-28 12:23:39 +01:00
Massimiliano Pippi
47f38c340e
Ensure test_comparison_in checks an actual subset of documents ( #7427 )
...
* fix test_comparison_in
* relnotes
2024-03-27 17:31:05 +01:00
Vladimir Blagojevic
ce8e114769
feat: DynamicChatPromptBuilder add templating to all user/system messages ( #7423 )
2024-03-27 15:34:50 +01:00
dependabot[bot]
7894024e6f
chore(deps): bump tj-actions/changed-files from 43 to 44 ( #7435 )
...
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files ) from 43 to 44.
- [Release notes](https://github.com/tj-actions/changed-files/releases )
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md )
- [Commits](https://github.com/tj-actions/changed-files/compare/v43...v44 )
---
updated-dependencies:
- dependency-name: tj-actions/changed-files
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-27 15:26:56 +01:00
Vladimir Blagojevic
eb7974e78f
Add TransformersZeroShotTextRouter to docs ( #7433 )
2024-03-27 13:20:34 +01:00
Stefano Fiorucci
69bd112f3e
ci: notify e2e nightly failures ( #7429 )
...
* try e2e notification
* rm code used to test
2024-03-27 11:12:54 +01:00
Silvano Cerza
58d91b64dc
Fix: Fix Pipeline.run() running components with only defaults in the wrong order ( #7426 )
...
* Fix Pipeline.run() running components with only defaults in the wrong order
* Add release notes
2024-03-26 16:55:31 +01:00
Silvano Cerza
685343d13f
feat: Add DocumentRecallEvaluator ( #7399 )
...
* Add DocumentRecallEvaluator
* Fix mypy error
* Simplify recall logic and change output for single hit mode
* Remove unused import
* Add comment for RecallMode fields
* Reword RecallMode comments
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-03-26 16:15:03 +01:00
Stefano Fiorucci
e26ee0f1db
refactor!: make TGI generators compatible with huggingface_hub>=0.22.0 ( #7425 )
...
* progress
* progress
* better lazy imports
* fixes
* reno
2024-03-26 16:10:06 +01:00
David S. Batista
fcd48d662c
test: HuggingFaceLocalGenerator test stopwords ( #7416 )
...
* initial import
* Update test/components/generators/test_hugging_face_local_generator.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* attending PR comments
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-03-26 12:39:02 +01:00
