Tobias Wochinger
f22d49944d
docs: review and normalize haystack.components.websearch ( #7236 )
...
* docs: review and normalize `haystack.components.websearch`
* fix: use correct type annotations
* refactor: use type from protocol
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
* Revert "refactor: use type from protocol"
This reverts commit 23d6f45cd763c39b98be1bff03639a90f2a01fac.
* docs: refactor according to comments
* build: correctly pin to 4.7
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-02-28 16:43:08 +01:00
Stefano Fiorucci
7b9704a93a
docs: review Routers docstrings ( #7234 )
...
* wip
* review routers
* small fixes
* Update haystack/components/routers/conditional_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/routers/conditional_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/routers/file_type_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/routers/file_type_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/routers/file_type_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/routers/file_type_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/routers/metadata_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/routers/metadata_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/routers/text_language_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/routers/text_language_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/components/routers/text_language_router.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-02-28 11:26:22 +01:00
Tobias Wochinger
ac4f458e2b
docs: review and normalize haystack.components.fetchers ( #7232 )
...
* docs: review and normalize `haystack.components.fetchers`
* docs: drop defaults
2024-02-28 11:24:12 +01:00
Tobias Wochinger
419009b495
fix: move sensitive log to debug mode ( #7230 )
2024-02-28 09:45:50 +01:00
Vladimir Blagojevic
d871bbbfbd
feat: Add complex types in OpenAPI support ( #7065 )
...
* Add complex types OpenAPI support
* Add release note
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-02-27 18:11:06 +01:00
Stefano Fiorucci
e194c08316
docs: review DocumentLanguageClassifier docstrings ( #7210 )
...
* review DocumentLanguageClassifier docstrings
* fix
* improve pydoc config
2024-02-27 16:02:53 +01:00
Silvano Cerza
0a7dfc1b32
Revert "Add AnswerExactMatchEvaluator ( #7050 )" ( #7075 )
...
This reverts commit b4011af8e9bc4ae2f72e51db254bfda69e20b651.
2024-02-23 14:05:57 +01:00
Silvano Cerza
b4011af8e9
Add AnswerExactMatchEvaluator ( #7050 )
...
* Add AnswerExactMatchEvaluator
* Add release notes
* Fix linting
* Update docstrings
2024-02-23 10:37:18 +01:00
Vladimir Blagojevic
49cad21a2e
chore: Adjust json_schema.py slightly ( #7055 )
...
* Slighly adjust json_schema.py
* Adjust test structures
2024-02-22 14:33:07 +01:00
Vladimir Blagojevic
cb6389d7a2
feat: Improve OpenAPI integration ( #7034 )
...
* Simplify and improve OpenAPIServiceConnector and OpenAPIServiceToFunctions, add unit tests
* Add reno note
* Add flask test dependency
* Initial PR feedback - Julian
* Remove indirection - Silvano
* Remove flask end-to-end tests
* Remove unused import
* Add mixed body unit test
* Update unit test, mock properly
2024-02-22 14:03:50 +01:00
Silvano Cerza
8ca4bf405b
Remove all evaluator components ( #7053 )
2024-02-21 18:24:14 +01:00
Varun Mathur
b335b5d723
feat: Add Lost In The Middle Ranker ( #6995 )
...
* add lost in the middle ranker
* update
* add release notes
* update release notes
* fix mypy
* Update
* fix mypy
* fix mypy [union-attr] for content.split
* remove e2e tests and negative topk param
* remove query param, validate params
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-02-20 19:55:41 +01:00
Ashwin Mathur
327c2d260d
feat: Add Mean Reciprocal Rank (MRR) metric to StatisticalEvaluator ( #7042 )
...
* Add MRR Metric
* Add release notes
* Update logic
2024-02-20 13:58:48 +01:00
Silvano Cerza
05af9c3439
test: Simplify OpenAPIServiceConnector run test ( #7043 )
...
* Simplify OpenAPIServiceConnector run test
* Fix linting
2024-02-20 11:54:51 +01:00
Silvano Cerza
9215882779
Add Recall Multi Hit and Single Hit metric ( #7038 )
2024-02-19 18:00:39 +01:00
Stefano Fiorucci
d00f171f8b
refactor!: Sentence Transformers Embedders - new devices mgmt ( #7033 )
...
* new device mgmt for Sentence Transformers embedders
* reno
2024-02-19 14:52:44 +01:00
Stefano Fiorucci
44b5ae291c
specify CPU device in warm_up test ( #7014 )
2024-02-16 13:01:57 +01:00
Stefano Fiorucci
0aa788facc
refactor!: LocalWhisperTranscriber - new devices mgmt ( #7008 )
...
* wip
* whisper local transcriber: use new device mgmt
* better from_dict + test
* reno
2024-02-16 11:25:53 +01:00
Silvano Cerza
a7209f6413
Mark OpenAPIServiceConnector integration test as flaky ( #7007 )
2024-02-15 19:33:34 +01:00
Tuana Çelik
e2cee468fc
fix: Adding api_base_url to OpenAITextEmbeder self assignments ( #7004 )
...
* assigning api_base_url
This fix resolves issues with the MistralTextEmbedder integration
* adding base url to `to_dict` and the tests
* adding release note
* Update fix-openai-base-url-assignment-0570a494d88fe365.yaml
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-15 17:35:28 +01:00
Silvano Cerza
6fe1d3b595
refactor: Clean eval components ( #7005 )
...
* Remove preprocess.py
* Rename eval components to evaluators
2024-02-15 17:17:59 +01:00
Silvano Cerza
2b8a606cb8
refactor: Refactor StatisticalEvaluator ( #6999 )
...
* Refactor StatisticalEvaluator
* Update StatisticalEvaluator
* Rename StatisticalMetric.from_string to from_str and change internal logic
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Fix tests
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-02-15 16:47:35 +01:00
Silvano Cerza
c82f787b41
feat: Add TextCleaner component ( #6997 )
...
* Add TextCleaner component
* Update docstrings and simplify run logic
* Update docstrings
2024-02-15 16:10:38 +01:00
Silvano Cerza
2a4e6a1de2
refactor: Refactor SASEvaluator ( #6998 )
...
* Remove preprocessing from SASEvaluator and add warm_up method
* Update docstrings
2024-02-15 16:05:43 +01:00
Vladimir Blagojevic
5a8d02064b
feat: Add JsonSchemaValidator ( #6937 )
...
* Add JsonSchemaValidator
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-15 14:07:01 +01:00
Silvano Cerza
36ab23d360
feat: Add StatisticalEvaluator component ( #6982 )
...
* Add StatisticalEvaluator component
* Remove F1 and Exact Metric from old API
* Add release notes
* Update docstrings
2024-02-14 16:48:03 +01:00
Silvano Cerza
9297fca520
feat: Add SASEvaluator component ( #6980 )
...
* Add SASEvaluator component
* Add release notes
* Delete old tests
* Remove SAS metric in old API
* Avoid importing whole numpy package
2024-02-14 16:16:22 +01:00
Vladimir Blagojevic
8d46a2883e
feat: Make system_messages optional in OpenAPIServiceToFunctions run ( #6825 )
...
* Make system_messages optional in OpenAPIServiceToFunctions run
* Adjust unit test
* PR feedback Massi
2024-02-14 16:04:35 +01:00
Vladimir Blagojevic
6a776e672f
Add OutputAdapter sede for custom filters ( #6985 )
2024-02-13 16:56:43 +01:00
Sebastian Husch Lee
ea7275955d
feat: Meta field ranker add meta_value_type ( #6977 )
...
* Update MetaFieldRanker to parse string meta values based on meta_value_type
* Add some unit tests
* Add another unit test
* Add release notes
* Fix mypy
* Fix pylint
* Add more unit tests
* Update release notes
* Update docs
* Further improve doc strings
2024-02-13 13:08:35 +01:00
Vladimir Blagojevic
97a0df66d2
feat: Add OutputAdapter ( #6936 )
...
* Add OutputAdapter component
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-13 13:03:50 +01:00
Vladimir Blagojevic
a311d82593
feat: Externalize callable serialization so it can be reused ( #6979 )
...
* Callback (de)serialization
* Add unit tests
* Replace callback handler sede with callable sede
* Remove unused functions
---------
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
2024-02-13 13:00:49 +01:00
Vladimir Blagojevic
37d9de3c4e
feat: Add service_credentials to OpenAPIServiceConnector run ( #6962 )
...
* Add service_credentials to OpenAPIServiceConnector run
* PR feedback Silvano
2024-02-09 16:03:27 +01:00
Bijay Gurung
74683fe74d
Feat: Add FilterRetriever ( #6836 )
...
* Add FilterRetriever draft
* Implement FilterRetriever and add tests
* Update comparison to compare whole docs instead of just contents
* Expose FilterRetriever at the retrievers level
* Update docstring (add example usage)
* Add filter_retriever in the API reference docs config
Update retriever search path to start one dir level higher
* simplify _documents_equal
* improve usage example
---------
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2024-02-08 08:48:46 +01:00
Vladimir Blagojevic
9e6a2e3cf9
fix: HuggingFaceTGIGenerator gets stuck when model is not supported ( #6915 )
...
* HuggingFaceTGIGenerator/HuggingFaceTGIChatGenerator check if model is deployed on free-tier
2024-02-06 16:55:06 +01:00
ZanSara
1182c08daf
fix: Dont filter negative scores when using BM25Okapi and scale_score=False ( #6889 )
...
* dont filter negatives for unscaled Okapi
* change BM25 algorithm default to BM25L
* Update haystack/document_stores/in_memory/document_store.py
* improve comment
2024-02-06 11:07:27 +01:00
Massimiliano Pippi
7d29ddba42
chore: merge hf utils modules into one ( #6921 )
...
* merge hf utils modules
* relnotes
* lint
* Update releasenotes/notes/merge-hf-utils-modules-5c16e04025123568.yaml
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-06 09:59:25 +01:00
Silvano Cerza
0191b1e6e4
feat: Change Component's I/O dunder type ( #6916 )
...
* Add Pipeline.get_component_name() method
* Add utility class to ease discoverability of Component I/O
* Move InputOutput in component package
* Rename InputOutput to _InputOutput
* Raise if inputs or outputs field already exist
* Fix tests
* Add release notes
* Move InputSocket and OutputSocket in types package
* Move _InputOutput in socket package
* Rename _InputOutput class to Sockets
* Simplify Sockets class
* Dictch I/O dunder fields in favour of inputs and outputs fields
* Update Sockets docstrings
* Update release notes
* Fix mypy
* Remove unnecessary assignment
* Remove unused logging
* Change SocketsType to SocketsIOType to avoid confusion
* Change sockets type and name
* Change Sockets.__repr__ to return component instance
* Fix linting
* Fix sockets tests
* Revert to dunder fields for Component IO
* Use singular in IO dunder fields
* Delete release notes
* Update haystack/core/component/types.py
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-05 17:46:45 +01:00
sahusiddharth
3bd6ba93ca
feat:Add dimensions parameter to OpenAI Embedders to fully support th… ( #6841 )
...
* feat:Add dimensions parameter to OpenAI Embedders to fully support the new models
* fixed linting
* changed != None to is not None
2024-02-05 16:20:46 +01:00
Madeesh Kannan
27d1af3068
feat!: Use Secret for passing authentication secrets to components ( #6887 )
...
* feat!: Use `Secret` for passing authentication secrets to components
* Add comment to clarify type ignore
2024-02-05 13:17:01 +01:00
ZanSara
9af6c7e442
add some tolerance to Roberta test ( #6880 )
2024-01-31 17:19:07 +01:00
Sebastian Husch Lee
ceda4cd655
feat: Add support for device_map ( #6679 )
...
* Getting device_map working to support 8bit loading and multi device inference
* Update to take account the device specified by the user
* add release notes
* Add device_map support for ExtractiveReader
* Update test
* Update to model that doesn't have issues
* Update test
* Update pytest approx
* Update release notes
* Start supporting device map
* Update ExtractiveReader to use new ComponentDevice
* Update similarity ranker to follow extractive reader implementation
* Fixing pylint
* Make mypy mostly happy
* Add new unit test to test device_map
* Adding unit tests
* Some refactoring
* Add more tests
* Add more tests
* Add another unit test
* Update first_device property to return a ComponentDevice to be able to use the to methods
* Updating tests for test_device
* Update tests and now explicitly modify device_map in model_kwargs
* Update haystack/utils/hf.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Make mypy happy
* mypy
* Remove unneeded optional flag
* Update ExtractiveReader with new logic
* Update ranker to follow new logic
* Removing unneeded code
* Make mypy happy
* fxi pylint
* Fix test
* Adding unit tests for device_map="auto"
* Add unit tests for ranker
* PR comments
* Make util method
* Adding unit tests
* Fix type annotation
* Fix pylint
* Fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-01-30 13:47:57 +01:00
Silvano Cerza
f5e61338ba
chore: Remove all mentions of Canals ( #6844 )
...
* Remove unnecessary Connection class
* Remove all mentions of canals
* Add release notes
2024-01-29 17:26:11 +01:00
Massimiliano Pippi
acf4cd502f
refact: Rename helper function ( #6831 )
...
* change function name
* add api docs
* release notes
2024-01-26 16:00:02 +01:00
Sebastian Husch Lee
3bea3b1714
feat: Add query and document prefix options for the TransformerSimilarityRanker ( #6826 )
...
* Add query and doc prefix
* Fix some tests
* add release notes
2024-01-25 15:29:19 +01:00
Rob Pasternak
7358b910d7
feat: Weights and score normalization for DocumentJoiner with reciprocal rank fusion ( #6735 )
...
* Add weighting and score normalization for DocumentJoiner w/ reciprocal rank fusion (fix trailing whitespace)
* Add release notes
* Add unit test
* Update release note
---------
Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
2024-01-24 15:45:53 +01:00
Vladimir Blagojevic
6e86f4e26a
Update embedding integration tests ( #6823 )
2024-01-24 15:22:47 +01:00
Vladimir Blagojevic
0b177b3bc6
feat: Improve OpenAPIServiceConnector service response serialization ( #6772 )
...
* Better service response json -> str serialization
* Add unit test
2024-01-18 16:49:48 +01:00
Vladimir Blagojevic
fea1428e84
feat: Add HuggingFaceLocalChatGenerator ( #6751 )
2024-01-18 15:53:12 +01:00
Madeesh Kannan
5d66d040cc
feat: Add serde methods to HTMLToDocument ( #6758 )
2024-01-18 10:02:01 +01:00
