Stefano Fiorucci
39be515ba6
skip HF integrations tests if running from fork ( #7517 )
2024-04-09 17:47:13 +02:00
Vladimir Blagojevic
988c360b6d
feat: Azure converter updates ( #7409 )
...
* Initial commit
* Remove old mock tests
* Fix current_last_page_number calculation
* Carry over unit tests from the other side
* Update pydocs, skip failing tests
* Fix pylint and mypy
* Minor adjustments
* Add release note
* Minor touch ups
* Resolve Document unique id issue by using custom id calculation
* Better hashing, add unit tests
* Small fixes
2024-04-09 09:45:06 +02:00
Stefano Fiorucci
eff53a9131
feat: HuggingFaceAPIDocumentEmbedder ( #7485 )
...
* add HuggingFaceAPITextEmbedder
* add HuggingFaceAPITextEmbedder
* rm unneeded else
* wip
* small fixes
* deprecation; reno
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* make params mandatory
* changes requested
* fix test
* fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-08 15:06:26 +02:00
Stefano Fiorucci
c91bd49cae
feat: HuggingFaceAPITextEmbedder ( #7484 )
...
* add HuggingFaceAPITextEmbedder
* add HuggingFaceAPITextEmbedder
* rm unneeded else
* small fixes
* changes requested
* fix test
2024-04-08 14:22:54 +02:00
David S. Batista
aae2b31359
fix: typo in sas_evaluator arg ( #7486 )
...
* fixing typo on SAS arg
* fixing tests
* fixing tests
2024-04-08 10:21:37 +02:00
Stefano Fiorucci
0dbb98c0a0
feat: HuggingFaceAPIChatGenerator ( #7480 )
...
* draft
* docstrings and more tests
* deprecation; reno
* pydoc config
* better error messages
* wip
* add test
* better docstrings
* deprecation; reno
* pylint
* typo
* rm unneeded else
* rm unneeded else
* fixes from feedback
* docstring showing the enum
* improve docstring
* make params mandatory
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* document enum
* Update haystack/utils/hf.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* mandatory params
* fix test
* fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-05 18:48:34 +02:00
Stefano Fiorucci
1d083861ff
feat: HuggingFaceAPIGenerator ( #7464 )
...
* draft
* docstrings and more tests
* deprecation; reno
* pydoc config
* better error messages
* rm unneeded else
* make params mandatory
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* document enum
* Update haystack/utils/hf.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-05 18:48:13 +02:00
Silvano Cerza
ff269db12d
Fix unit tests failing if HF_API_TOKEN is set ( #7491 )
2024-04-05 18:05:43 +02:00
Vladimir Blagojevic
c3b96392fd
feat: Use all HTMLToDocument extractors until content is extracted ( #7452 )
...
* Use all HTMLToDocument extractors until content is extracted
* Add release note
* Minor doc update
* Improvements, unit test fixes
* Add try_others init param, update tests
* Update haystack/components/converters/html.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* PR feedback - Stefano
* Improve reno release note, add reference
* little fixes
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-04-05 16:02:34 +02:00
Julian Risch
9d02dc607a
feat: Add FaithfulnessEvaluator component ( #7424 )
...
* draft FaithfulnessEvaluator
* reno
* calculate score per statement and aggregate
* Update release note
* update default values in tests and fix import path
* remove instructions, inputs, outputs params
* remove unused imports
* add expected format example to docstring
* remove name 'llm' from tests and docstring
2024-04-04 16:33:59 +00:00
Julian Risch
8ef6062748
refactor: Remove name 'llm' from LLMEvaluator output ( #7479 )
2024-04-04 15:19:30 +00:00
Silvano Cerza
8b8a93bc0d
refactor: Rename DocumentMeanAveragePrecision and DocumentMeanReciprocalRank ( #7470 )
...
* Rename DocumentMeanAveragePrecision and DocumentMeanReciprocalRank
* Update releasenotes
* Simplify names
2024-04-04 17:04:59 +02:00
Silvano Cerza
bdc25ca2a0
feat: Add DocumentMeanReciprocalRank ( #7468 )
...
* Add DocumentMeanReciprocalRank
* Fix float precision error
2024-04-04 14:55:37 +02:00
Silvano Cerza
7799909069
feat: Add DocumentMeanAveragePrecision ( #7461 )
...
* Add DocumentMeanAveragePrecision
* Remove questions input
* Update docstrings
* Update haystack/components/evaluators/document_map.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-04 14:15:45 +02:00
Silvano Cerza
dc87f51759
refactor: Remove questions inputs from evaluators ( #7466 )
...
* Remove questions input from AnswerExactMatchEvaluator
* Remove questions input from DocumentRecallEvaluator
2024-04-04 14:14:18 +02:00
Silvano Cerza
12acb3f12e
feat: Add SASEvaluator ( #7428 )
...
* Add SASEvaluator
* Add release notes
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Simplify similarity calculation with bi-encoders models
* Fix linting
* Update docstrings
* Move tensor to CPU after calculating cosine similarity
* Fix CI failing
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-04-04 10:10:41 +02:00
Ashwin Mathur
1c7d1618d8
Add truncate and normalize parameters to TEI Embedders ( #7460 )
2024-04-03 16:41:30 +02:00
Vladimir Blagojevic
d83af92270
feat: Update searchapi format, default to Google, allow search engine selection ( #7453 )
...
* Update searchapi payload
* Add release note
* PR feedback - Stefano
* Adjust unit test for mandatory engine search_param field
2024-04-03 10:48:50 +02:00
Nicola Procopio
42c5b7af32
feat: added dimensions parameters to Azure OpenAI Embedders ( #7449 )
...
* added dimensions parameter to AzureOpenAIEmbedders
* created releasenote
* update release note
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-04-02 14:04:16 +02:00
Silvano Cerza
6e289698e9
fix: Fix Pipeline.run() getting stuck in a loop even though there are components that can run ( #7434 )
2024-03-28 12:31:36 +01:00
Vladimir Blagojevic
ce8e114769
feat: DynamicChatPromptBuilder add templating to all user/system messages ( #7423 )
2024-03-27 15:34:50 +01:00
Silvano Cerza
58d91b64dc
Fix: Fix Pipeline.run() running components with only defaults in the wrong order ( #7426 )
...
* Fix Pipeline.run() running components with only defaults in the wrong order
* Add release notes
2024-03-26 16:55:31 +01:00
Silvano Cerza
685343d13f
feat: Add DocumentRecallEvaluator ( #7399 )
...
* Add DocumentRecallEvaluator
* Fix mypy error
* Simplify recall logic and change output for single hit mode
* Remove unused import
* Add comment for RecallMode fields
* Reword RecallMode comments
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-03-26 16:15:03 +01:00
Stefano Fiorucci
e26ee0f1db
refactor!: make TGI generators compatible with huggingface_hub>=0.22.0 ( #7425 )
...
* progress
* progress
* better lazy imports
* fixes
* reno
2024-03-26 16:10:06 +01:00
David S. Batista
fcd48d662c
test: HuggingFaceLocalGenerator test stopwords ( #7416 )
...
* initial import
* Update test/components/generators/test_hugging_face_local_generator.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* attending PR comments
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-03-26 12:39:02 +01:00
Silvano Cerza
f398b29e7f
feat: Change outputs of AnswerExactMatchEvaluator ( #7390 )
...
* Change outputs of AnswerExactMatchEvaluator
* Changes scores to return the number of matches per question
* Revert "Changes scores to return the number of matches per question"
This reverts commit e4358720793d4584b0b961402d4557c50c4c2381.
* Change output names
2024-03-26 10:57:59 +01:00
Stefano Fiorucci
6925e3a2e1
refactor!: Improve PyPDFToDocument ( #7362 )
...
* first draft
* rm kwargs from protocol
* Simplify
* no breaking changes
* reno
* one more test of the deprecated registry
2024-03-26 10:09:29 +01:00
Julian Risch
bfd0d3eacd
feat: Add new LLMEvaluator component ( #7401 )
...
* draft llm evaluator
* docstrings
* flexible inputs; validate inputs and outputs
* add tests
* add release note
* remove example
* docstrings
* make outputs parameter optional. default:
* validate init parameters
* linting
* remove mention of binary scores from template
* make examples and outputs params non-optional
* removed leftover from optional outputs param
* simplify building examples section for template
* validate inputs and outputs in examples are dict with str as key
* fix pylint too-many-boolean-expressions
* increase test coverage
2024-03-25 07:05:27 +01:00
Stefano Fiorucci
c789f905bc
refactor: pass a role string to OpenAI API ( #7404 )
...
* draft
* rm unused imports
2024-03-22 09:36:56 +01:00
Vladimir Blagojevic
e779d43384
feat: Add streaming to HuggingFaceLocalGenerator ( #7377 )
...
* Inital streaming impl
* Add unit tests
* Add release note
2024-03-21 15:49:18 +01:00
Stefano Fiorucci
6e69d4f188
fix: Pipeline - disable autoshow on Jupyter ( #7397 )
...
* try
* fix docstring
* simplify tests
* add release note
2024-03-21 12:55:06 +01:00
Stefano Fiorucci
dbfd351da7
feat: introduce SparseEmbedding ( #7382 )
...
* introduce SparseEmbedding
* reno
* add to pydoc config
2024-03-19 18:04:16 +01:00
Silvano Cerza
610ad6f6b2
Add AnswerExactMatchEvaluator ( #7381 )
...
* Add AnswerExactMatchEvaluator
* Add release notes
* Fix linting
* Update docstrings
* Update docstrings
* Remove to_dict and from_dict
* Fix linting
2024-03-19 16:58:01 +01:00
Christopher Keibel
f69c3e5cd2
refactor: default for max_new_tokens to 512 in Hugging Face generators ( #7370 )
...
* set default for max_new_tokens to 512 in Hugging Face generators
* add release notes
* fix tests
* remove issues from release note
---------
Co-authored-by: christopherkeibel <christopher.keibel@karakun.com>
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-03-19 08:47:53 +01:00
Mohit Lal
280719339c
bug: run parameter "ranking_mode" does not override init param in meta field ranker ( #7375 )
...
* bug: run parameter ranking_mode does not override init param in metafield ranker
* Added a release note
* Used pytest.approx for comparing floating point numbers in unit test
2024-03-19 07:53:26 +01:00
Sebastian Husch Lee
85c1e39fab
feat: Add Zero Shot Transformers Text Router ( #7018 )
...
* Starting to add TransformersTextRouter
* First pass at a TextRouter based off of the zero shot classification model on HuggingFace
* Fix pylint
* Remove unneeded imports
* Update documentation example
* Update error message strings
* Starting to add unit tests
* Release notes
* Fix pylint
* Add tests for to dict and from dict
* Update patches in tests to be correct with respect to changes
* Doc strings and fixes
* Adding more tests
* Change name
* Adding to init
* Use Haystack logger
* Beef up docstrings
* Make example runnable
* Rename to huggingface_pipeline_kwargs
* Fix example
2024-03-15 13:56:07 +01:00
Stefano Fiorucci
abda78c122
unpin OpenAI and fix problem with mock ( #7364 )
2024-03-15 08:32:28 +01:00
Vladimir Blagojevic
2aae8472e7
feat: Add trust_remote_code init param to SentenceTransformer embedders ( #7356 )
...
* Add trust_remote_code init param to SentenceTransformer embedders
* Add release note
* Go with no kwargs solution
* Update haystack/components/embedders/sentence_transformers_document_embedder.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* Pydoc fix
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-03-14 11:14:04 +01:00
Silvano Cerza
de4fca4526
ci: Skip collection of test_json_schema.py to fix CI failures ( #7353 )
...
* Skip collection of test_json_schema.py to fix CI failures
* mock chroma instance
* revert
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-03-13 16:59:26 +01:00
Yudhajit Sinha
41dbbdb3fc
feat: Add support for matching mime types using regex ( #7303 )
...
* feat: Add support for matching mime types using regex
---------
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
2024-03-11 14:58:08 +01:00
Ashwin Mathur
38b3472bb2
feat: Add SentenceTransformersDiversityRanker ( #7095 )
...
* Add Diversity Ranker
* Update tests
* Add separate suffix, prefix params for query and documents; allow empty query
* Update docstrings
* Make changes based on review
* Add additional tests
* Add test for warm up
* Update release notes
---------
Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
2024-03-11 13:14:59 +01:00
Ashwin Mathur
8d7a58347d
fix: HuggingFaceTEITextEmbedder returning embedding of incorrect shape when used with Docker endpoint ( #7319 )
...
* Fix HuggingFaceTEITextEmbedder
* Update haystack/components/embedders/hugging_face_tei_text_embedder.py
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* Improve imports; Add additional tests
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-03-07 16:23:57 +01:00
Stefano Fiorucci
3dbde84a28
test: monkeypatch some env vars in Predefined Pipelines tests ( #7321 )
...
* ci: skip some tests if the OPENAI API key is not set
* better idea: monkeypatch the env var
2024-03-07 08:52:25 +01:00
Tobias Wochinger
23c65c250f
chore: migrate ExtractiveReader to use secret management ( #7309 )
...
* chore: migrate `ExtractiveReader` to use secret management
* docs: add release notes
2024-03-05 13:04:53 +01:00
Julian Risch
50ad1fa2c4
fix: Remove pipeline serialization from telemetry code ( #7289 )
...
* remove pipeline serialization from telemetry
* simplify getting component instance from pipeline
* reno
* add unit test with non-serializable component
* generate qualified class names
* added pipeline.walk()
* fix imports
* sort Iterator import
* remove bfs
* add test for pipeline.walk() with cycles
* Apply suggestions from code review
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* raise TypeError if telemetry_data is no dict
* Update haystack/telemetry/_telemetry.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-03-05 12:45:53 +01:00
Tobias Wochinger
655d4a1a8d
test: test for missing dependencies ( #7278 )
...
* tests: import test for missing libraries
* build: add missing dependencies
* refactor: use glob instead of tree walk
* test: extract constants + more documentation
2024-03-05 12:14:10 +01:00
Stefano Fiorucci
38a80b0235
fix: MetaFieldRanker - use weight if passed in the run method ( #7305 )
...
* fix: - use if passed in the method
* reno
2024-03-05 12:13:56 +01:00
Julian Risch
c1c0cbfde4
docs: Update docs of MetaFieldRanker, TransformersSimilarityRanker ( #7301 )
...
* docs: Update docstrings of MetaFieldRanker and TransformersSimilarityRanker
* add warm_up() call to usage example
* Apply suggestions from code review
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
* show result of usage example
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-03-05 10:20:18 +01:00
Julian Risch
9a0e2e58fd
docs: Added LostInTheMiddleRanker usage example and updated docstrings ( #7294 )
...
* docs: Added LostInTheMiddleRanker usage example
* remove to_dict test
* explain LITM in more detail
2024-03-04 15:42:51 +01:00
Vladimir Blagojevic
0e7c41be5e
feat: Improve OpenAPIServiceToFunctions signature ( #7257 )
...
* Convert OpenAPIServiceToFunctions run interface
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-03-04 14:38:58 +01:00
