Silvano Cerza
c82f787b41
feat: Add TextCleaner component ( #6997 )
...
* Add TextCleaner component
* Update docstrings and simplify run logic
* Update docstrings
2024-02-15 16:10:38 +01:00
Silvano Cerza
2a4e6a1de2
refactor: Refactor SASEvaluator ( #6998 )
...
* Remove preprocessing from SASEvaluator and add warm_up method
* Update docstrings
2024-02-15 16:05:43 +01:00
Silvano Cerza
06a9349095
Change mermaid integration test to avoid comparing received image ( #7002 )
2024-02-15 15:01:04 +01:00
Vladimir Blagojevic
5a8d02064b
feat: Add JsonSchemaValidator ( #6937 )
...
* Add JsonSchemaValidator
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-15 14:07:01 +01:00
Silvano Cerza
cf221a9701
Delete old eval API ( #6983 )
2024-02-14 17:11:08 +01:00
Silvano Cerza
cb42ac19ae
docs: Add eval Components docs config ( #6984 )
...
* Add eval components docs config
* Update order
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
---------
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2024-02-14 16:57:01 +01:00
Silvano Cerza
36ab23d360
feat: Add StatisticalEvaluator component ( #6982 )
...
* Add StatisticalEvaluator component
* Remove F1 and Exact Metric from old API
* Add release notes
* Update docstrings
2024-02-14 16:48:03 +01:00
Silvano Cerza
9297fca520
feat: Add SASEvaluator component ( #6980 )
...
* Add SASEvaluator component
* Add release notes
* Delete old tests
* Remove SAS metric in old API
* Avoid importing whole numpy package
2024-02-14 16:16:22 +01:00
Vladimir Blagojevic
8d46a2883e
feat: Make system_messages optional in OpenAPIServiceToFunctions run ( #6825 )
...
* Make system_messages optional in OpenAPIServiceToFunctions run
* Adjust unit test
* PR feedback Massi
2024-02-14 16:04:35 +01:00
Vladimir Blagojevic
0e044a88aa
Reintroduce serialize_callback_handler and deserialize_callback_handler ( #6988 )
2024-02-14 14:57:17 +01:00
Vladimir Blagojevic
6a776e672f
Add OutputAdapter sede for custom filters ( #6985 )
2024-02-13 16:56:43 +01:00
Sebastian Husch Lee
ea7275955d
feat: Meta field ranker add meta_value_type ( #6977 )
...
* Update MetaFieldRanker to parse string meta values based on meta_value_type
* Add some unit tests
* Add another unit test
* Add release notes
* Fix mypy
* Fix pylint
* Add more unit tests
* Update release notes
* Update docs
* Further improve doc strings
2024-02-13 13:08:35 +01:00
Vladimir Blagojevic
97a0df66d2
feat: Add OutputAdapter ( #6936 )
...
* Add OutputAdapter component
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-13 13:03:50 +01:00
Vladimir Blagojevic
a311d82593
feat: Externalize callable serialization so it can be reused ( #6979 )
...
* Callback (de)serialization
* Add unit tests
* Replace callback handler sede with callable sede
* Remove unused functions
---------
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
2024-02-13 13:00:49 +01:00
Silvano Cerza
f96eb3847f
refactor: Merge Pipelines definition in core package ( #6973 )
...
* Move marshalling functions in core Pipeline
* Move telemetry gathering in core Pipeline
* Move run logic in core Pipeline
* Update root Pipeline import
* Add release notes
* Update Pipeline docs path
* Update releasenotes/notes/merge-pipeline-definitions-1da80e9803e2a8bb.yaml
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-12 18:25:28 +01:00
Silvano Cerza
549021d2fc
Use better cache key for dependencies in tests.yml ( #6966 )
2024-02-09 17:49:06 +01:00
David S. Batista
55785d553b
fixing typo ( #6967 )
2024-02-09 17:18:22 +01:00
Silvano Cerza
21488ac6ca
fix: Fix Pipeline.draw() failing cause of missing socket ( #6965 )
...
* Fix Pipeline.draw() issue
* Mark draw test as flaky
2024-02-09 16:10:48 +01:00
Vladimir Blagojevic
37d9de3c4e
feat: Add service_credentials to OpenAPIServiceConnector run ( #6962 )
...
* Add service_credentials to OpenAPIServiceConnector run
* PR feedback Silvano
2024-02-09 16:03:27 +01:00
Silvano Cerza
d2d01f9fe1
feat: Enhance Pipeline.__repr__() ( #6963 )
...
* Enhance Pipeline.draw() to show image directly in Jupyter notebook
* Add util method to check if we're in a Jupyter notebook
* Split Pipeline.draw() in two methods
* Update tests
* Update releasenotes
* Enhance Pipeline.__repr__
* Simplify Pipeline.__repr__
* Update release notes
2024-02-09 14:44:34 +01:00
Silvano Cerza
a7f36fdd32
feat: Enhance Pipeline.draw() to show image directly in Jupyter notebook ( #6961 )
...
* Enhance Pipeline.draw() to show image directly in Jupyter notebook
* Add util method to check if we're in a Jupyter notebook
* Split Pipeline.draw() in two methods
* Update tests
* Update releasenotes
2024-02-09 14:44:24 +01:00
Vladimir Blagojevic
d2497d54e8
Update to use the default Secret.from_env_var(OPENAI_API_KEY) approach ( #6941 )
2024-02-09 14:15:45 +01:00
Madeesh Kannan
24b23a2cc7
docs: Add missing docstrings to NamedEntityExtractor ( #6958 )
...
* docs: Add missing docstrings to `NamedEntityExtractor`
* Update named_entity_extractor.py
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-08 13:39:45 +01:00
Silvano Cerza
2f965fb176
feat: Add __repr__ method to all Components ( #6927 )
...
* Add __repr__ to show Component I/O
* Add release notes
* Change Component repr to show full module path and name in Pipeline
* Fix linting
2024-02-08 11:46:10 +01:00
Bijay Gurung
74683fe74d
Feat: Add FilterRetriever ( #6836 )
...
* Add FilterRetriever draft
* Implement FilterRetriever and add tests
* Update comparison to compare whole docs instead of just contents
* Expose FilterRetriever at the retrievers level
* Update docstring (add example usage)
* Add filter_retriever in the API reference docs config
Update retriever search path to start one dir level higher
* simplify _documents_equal
* improve usage example
---------
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2024-02-08 08:48:46 +01:00
Stefano Fiorucci
a771d7f015
bump version ( #6935 )
2024-02-07 16:40:52 +01:00
Silvano Cerza
b49e86d007
fix: Change PromptBuilder to have default values for all inputs ( #6934 )
...
* Change PromptBuilder to have default values for all inputs
* Add release notes
2024-02-07 16:03:17 +01:00
dependabot[bot]
929baf3961
chore(deps): bump aws-actions/configure-aws-credentials ( #6933 )
...
Bumps [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials ) from 4.0.1 to 4.0.2.
- [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases )
- [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md )
- [Commits](010d0da01d...e3dd6a429d
2024-02-07 15:28:02 +01:00
Vladimir Blagojevic
9e6a2e3cf9
fix: HuggingFaceTGIGenerator gets stuck when model is not supported ( #6915 )
...
* HuggingFaceTGIGenerator/HuggingFaceTGIChatGenerator check if model is deployed on free-tier
2024-02-06 16:55:06 +01:00
Silvano Cerza
b875eda4af
refactor: Rename Pipeline.connect() arguments ( #6919 )
...
* Rename Pipeline.connect() arguments
* Add release notes
2024-02-06 11:31:53 +01:00
Vladimir Blagojevic
95dce5b0ec
Change HuggingFaceTGIChatGenerator default model to HuggingFaceH4/zephyr-7b-beta ( #6920 )
2024-02-06 11:15:28 +01:00
ZanSara
1182c08daf
fix: Dont filter negative scores when using BM25Okapi and scale_score=False ( #6889 )
...
* dont filter negatives for unscaled Okapi
* change BM25 algorithm default to BM25L
* Update haystack/document_stores/in_memory/document_store.py
* improve comment
2024-02-06 11:07:27 +01:00
Silvano Cerza
3d5c329949
Change Pipeline.connect() to return Pipeline instance ( #6922 )
2024-02-06 11:03:18 +01:00
Silvano Cerza
289aa44aec
Add flaky as dev dependency ( #6924 )
2024-02-06 10:48:39 +01:00
Massimiliano Pippi
7d29ddba42
chore: merge hf utils modules into one ( #6921 )
...
* merge hf utils modules
* relnotes
* lint
* Update releasenotes/notes/merge-hf-utils-modules-5c16e04025123568.yaml
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-06 09:59:25 +01:00
Silvano Cerza
b9d7a98359
Rework Pipeline.connect() tests and structure ( #6917 )
2024-02-05 18:16:02 +01:00
Silvano Cerza
0191b1e6e4
feat: Change Component's I/O dunder type ( #6916 )
...
* Add Pipeline.get_component_name() method
* Add utility class to ease discoverability of Component I/O
* Move InputOutput in component package
* Rename InputOutput to _InputOutput
* Raise if inputs or outputs field already exist
* Fix tests
* Add release notes
* Move InputSocket and OutputSocket in types package
* Move _InputOutput in socket package
* Rename _InputOutput class to Sockets
* Simplify Sockets class
* Dictch I/O dunder fields in favour of inputs and outputs fields
* Update Sockets docstrings
* Update release notes
* Fix mypy
* Remove unnecessary assignment
* Remove unused logging
* Change SocketsType to SocketsIOType to avoid confusion
* Change sockets type and name
* Change Sockets.__repr__ to return component instance
* Fix linting
* Fix sockets tests
* Revert to dunder fields for Component IO
* Use singular in IO dunder fields
* Delete release notes
* Update haystack/core/component/types.py
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-05 17:46:45 +01:00
sahusiddharth
3bd6ba93ca
feat:Add dimensions parameter to OpenAI Embedders to fully support th… ( #6841 )
...
* feat:Add dimensions parameter to OpenAI Embedders to fully support the new models
* fixed linting
* changed != None to is not None
2024-02-05 16:20:46 +01:00
Bilge Yücel
0fbb0655f0
Create breaking-change-proposal.md issue template ( #6892 )
...
* Create breaking-change-proposal.md issue template
* Update breaking-change-proposal.md
2024-02-05 14:54:33 +01:00
Madeesh Kannan
c3a9dac196
chore: Tick version to 2.0.0-beta.6 ( #6914 )
2024-02-05 13:51:45 +01:00
Madeesh Kannan
27d1af3068
feat!: Use Secret for passing authentication secrets to components ( #6887 )
...
* feat!: Use `Secret` for passing authentication secrets to components
* Add comment to clarify type ignore
2024-02-05 13:17:01 +01:00
Ashwin Mathur
393a7993c3
feat: Add Semantic Answer Similarity metric ( #6877 )
...
* Add SAS metric
* Add release notes
* Round similarity scores for precision consistency
* Add tolerance to tests
* Update haystack/evaluation/eval.py
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
* Add types for preprocess_text; Add additional types for f1 and em methods
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-02-02 17:07:52 +01:00
Silvano Cerza
461556cca2
fix: Fix language servers never working with Components ( #6893 )
...
* Fix language servers never working with Components
* Add release notes
2024-02-02 15:59:05 +01:00
Massimiliano Pippi
27d0b28d06
chore: rename categories in the API docs ( #6885 )
...
* rename API categories
* fix
* update slugs
* rename files for consistency
* fix category ID
* try getting the right version
2024-02-01 16:47:26 +01:00
ZanSara
1039c73553
feat: Allow setting metadata for ByteStream when created from file or from string ( #6857 )
...
* add params
* add tests
* reno
* add default
* defreeze
2024-02-01 12:50:11 +01:00
ZanSara
9af6c7e442
add some tolerance to Roberta test ( #6880 )
2024-01-31 17:19:07 +01:00
Madeesh Kannan
b772c1127c
feat: Implement Secret for structured authentication ( #6855 )
...
* feat: Implement `AuthPolicy` for structured authentication
* Rename `AuthPolicy` to `Secret`
* Update release notes, fix typo
2024-01-31 12:51:14 +01:00
Stefano Fiorucci
537107ba6e
ci: bump transformers to 4.37.2 in test_requirements ( #6848 )
...
* bump transformers to 4.37.1 in test_requirements
* use 4.37.2
2024-01-30 17:19:22 +01:00
Sebastian Husch Lee
ceda4cd655
feat: Add support for device_map ( #6679 )
...
* Getting device_map working to support 8bit loading and multi device inference
* Update to take account the device specified by the user
* add release notes
* Add device_map support for ExtractiveReader
* Update test
* Update to model that doesn't have issues
* Update test
* Update pytest approx
* Update release notes
* Start supporting device map
* Update ExtractiveReader to use new ComponentDevice
* Update similarity ranker to follow extractive reader implementation
* Fixing pylint
* Make mypy mostly happy
* Add new unit test to test device_map
* Adding unit tests
* Some refactoring
* Add more tests
* Add more tests
* Add another unit test
* Update first_device property to return a ComponentDevice to be able to use the to methods
* Updating tests for test_device
* Update tests and now explicitly modify device_map in model_kwargs
* Update haystack/utils/hf.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Make mypy happy
* mypy
* Remove unneeded optional flag
* Update ExtractiveReader with new logic
* Update ranker to follow new logic
* Removing unneeded code
* Make mypy happy
* fxi pylint
* Fix test
* Adding unit tests for device_map="auto"
* Add unit tests for ranker
* PR comments
* Make util method
* Adding unit tests
* Fix type annotation
* Fix pylint
* Fix test
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-01-30 13:47:57 +01:00
Silvano Cerza
76d324a149
feat: Change Pipeline.add_component to fail when reusing Component instances ( #6847 )
...
* Change Pipeline.add_component to fail when reusing Component instances
* Change variable name and store Pipeline instance in it
* Fix tests
2024-01-30 11:15:26 +01:00
