Tobias Wochinger
6d0d373def
feat: opentelemetry tracer ( #7052 )
...
* feat: add opentelemetry tracer
* feat: implement auto enable for `OpenTelemetry`
* docs: add release notes
* style: fix linting
* tests: more test coverage
* refactor: fix old method call
* docs: fix duplicate `the`
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-22 14:30:58 +01:00
Vladimir Blagojevic
cb6389d7a2
feat: Improve OpenAPI integration ( #7034 )
...
* Simplify and improve OpenAPIServiceConnector and OpenAPIServiceToFunctions, add unit tests
* Add reno note
* Add flask test dependency
* Initial PR feedback - Julian
* Remove indirection - Silvano
* Remove flask end-to-end tests
* Remove unused import
* Add mixed body unit test
* Update unit test, mock properly
2024-02-22 14:03:50 +01:00
Tobias Wochinger
6e580e4430
feat: implement pipeline tracing ( #7046 )
...
* feat: implement pipeline tracing
* tests: improve test setup for spying tracer
* feat: implement util for type coercion
* fix: trace a after checking pipeline output
* docs: add release notes
* docs: drop unused imports
* refactor: simplify getting raw span
* refactor: implement `ProxyTracer`
2024-02-22 12:52:04 +01:00
Silvano Cerza
8ca4bf405b
Remove all evaluator components ( #7053 )
2024-02-21 18:24:14 +01:00
Massimiliano Pippi
f3be576b5c
refact: remove the concept of TemplateSource from the public API ( #7051 )
...
* remove unused field
* hide the TemplateSource abstraction
* amend release notes
* revert
2024-02-21 12:02:04 +01:00
Varun Mathur
b335b5d723
feat: Add Lost In The Middle Ranker ( #6995 )
...
* add lost in the middle ranker
* update
* add release notes
* update release notes
* fix mypy
* Update
* fix mypy
* fix mypy [union-attr] for content.split
* remove e2e tests and negative topk param
* remove query param, validate params
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-02-20 19:55:41 +01:00
Ashwin Mathur
327c2d260d
feat: Add Mean Reciprocal Rank (MRR) metric to StatisticalEvaluator ( #7042 )
...
* Add MRR Metric
* Add release notes
* Update logic
2024-02-20 13:58:48 +01:00
Silvano Cerza
05af9c3439
test: Simplify OpenAPIServiceConnector run test ( #7043 )
...
* Simplify OpenAPIServiceConnector run test
* Fix linting
2024-02-20 11:54:51 +01:00
Silvano Cerza
9215882779
Add Recall Multi Hit and Single Hit metric ( #7038 )
2024-02-19 18:00:39 +01:00
Stefano Fiorucci
d00f171f8b
refactor!: Sentence Transformers Embedders - new devices mgmt ( #7033 )
...
* new device mgmt for Sentence Transformers embedders
* reno
2024-02-19 14:52:44 +01:00
Vladimir Blagojevic
cb01cb4207
feat: Add PipelineTemplate for ready made pipelines ( #7001 )
...
* Initial working version
* More experiments
* Use jinja2 tempplate extensions
* Remove for_type static method
* Add ternary filter, with_kwargs
* Add indexing pipeline
* Rename with_kwargs -> with_parameters, allow cummulative invocation
* Remove ternary filter, use another approach to conditionals in default component configuration
* Massi feedback, add pydocs
* PipelineTemplateBuilder -> PipelineTemplate
* Add unit tests
* Add reno note
* Fix pylint, mypy
* PR review - Madeesh
* Add list_variables
* PR review feedback - Madeesh
* Update haystack/templates/pipelines.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Add timeout, fix pylint
* Use full import path
* Update release note
* Fix failing windows test
* Update haystack/templates/source.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/templates/source.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* PR feedback Madeesh
* Fix pylint
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-02-19 13:59:48 +01:00
Silvano Cerza
f1a6b2a78a
feat: Add is_greedy argument in @component decorator ( #7016 )
...
* Add is_greedy argument in @component decorator
* Log warning if Component is greedy and non variadic
2024-02-19 12:43:40 +01:00
Silvano Cerza
5f97e08feb
feat: Reintroduce max_loops_allowed check in Pipeline.run() ( #7010 )
...
* Reintroduce max_loops_allowed check in Pipeline.run()
* Add release notes
2024-02-19 10:05:35 +01:00
Vladimir Blagojevic
3ce6b9768e
feat: Add ByteStream to_string method ( #7009 )
2024-02-17 12:57:42 +01:00
Stefano Fiorucci
3f85a63468
fix: deepcopy the inputs of components ( #6987 )
...
* deepcopy inputs
* reno
* simplify test
2024-02-16 17:07:52 +01:00
Stefano Fiorucci
44b5ae291c
specify CPU device in warm_up test ( #7014 )
2024-02-16 13:01:57 +01:00
Madeesh Kannan
b552b0b37c
fix: Ensure Secret types are immutable ( #6994 )
2024-02-16 12:46:38 +01:00
Stefano Fiorucci
0aa788facc
refactor!: LocalWhisperTranscriber - new devices mgmt ( #7008 )
...
* wip
* whisper local transcriber: use new device mgmt
* better from_dict + test
* reno
2024-02-16 11:25:53 +01:00
Silvano Cerza
a7209f6413
Mark OpenAPIServiceConnector integration test as flaky ( #7007 )
2024-02-15 19:33:34 +01:00
Tuana Çelik
e2cee468fc
fix: Adding api_base_url to OpenAITextEmbeder self assignments ( #7004 )
...
* assigning api_base_url
This fix resolves issues with the MistralTextEmbedder integration
* adding base url to `to_dict` and the tests
* adding release note
* Update fix-openai-base-url-assignment-0570a494d88fe365.yaml
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-15 17:35:28 +01:00
Silvano Cerza
6fe1d3b595
refactor: Clean eval components ( #7005 )
...
* Remove preprocess.py
* Rename eval components to evaluators
2024-02-15 17:17:59 +01:00
Silvano Cerza
2b8a606cb8
refactor: Refactor StatisticalEvaluator ( #6999 )
...
* Refactor StatisticalEvaluator
* Update StatisticalEvaluator
* Rename StatisticalMetric.from_string to from_str and change internal logic
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Fix tests
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-02-15 16:47:35 +01:00
Silvano Cerza
c82f787b41
feat: Add TextCleaner component ( #6997 )
...
* Add TextCleaner component
* Update docstrings and simplify run logic
* Update docstrings
2024-02-15 16:10:38 +01:00
Silvano Cerza
2a4e6a1de2
refactor: Refactor SASEvaluator ( #6998 )
...
* Remove preprocessing from SASEvaluator and add warm_up method
* Update docstrings
2024-02-15 16:05:43 +01:00
Silvano Cerza
06a9349095
Change mermaid integration test to avoid comparing received image ( #7002 )
2024-02-15 15:01:04 +01:00
Vladimir Blagojevic
5a8d02064b
feat: Add JsonSchemaValidator ( #6937 )
...
* Add JsonSchemaValidator
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-15 14:07:01 +01:00
Silvano Cerza
cf221a9701
Delete old eval API ( #6983 )
2024-02-14 17:11:08 +01:00
Silvano Cerza
36ab23d360
feat: Add StatisticalEvaluator component ( #6982 )
...
* Add StatisticalEvaluator component
* Remove F1 and Exact Metric from old API
* Add release notes
* Update docstrings
2024-02-14 16:48:03 +01:00
Silvano Cerza
9297fca520
feat: Add SASEvaluator component ( #6980 )
...
* Add SASEvaluator component
* Add release notes
* Delete old tests
* Remove SAS metric in old API
* Avoid importing whole numpy package
2024-02-14 16:16:22 +01:00
Vladimir Blagojevic
8d46a2883e
feat: Make system_messages optional in OpenAPIServiceToFunctions run ( #6825 )
...
* Make system_messages optional in OpenAPIServiceToFunctions run
* Adjust unit test
* PR feedback Massi
2024-02-14 16:04:35 +01:00
Vladimir Blagojevic
6a776e672f
Add OutputAdapter sede for custom filters ( #6985 )
2024-02-13 16:56:43 +01:00
Sebastian Husch Lee
ea7275955d
feat: Meta field ranker add meta_value_type ( #6977 )
...
* Update MetaFieldRanker to parse string meta values based on meta_value_type
* Add some unit tests
* Add another unit test
* Add release notes
* Fix mypy
* Fix pylint
* Add more unit tests
* Update release notes
* Update docs
* Further improve doc strings
2024-02-13 13:08:35 +01:00
Vladimir Blagojevic
97a0df66d2
feat: Add OutputAdapter ( #6936 )
...
* Add OutputAdapter component
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-13 13:03:50 +01:00
Vladimir Blagojevic
a311d82593
feat: Externalize callable serialization so it can be reused ( #6979 )
...
* Callback (de)serialization
* Add unit tests
* Replace callback handler sede with callable sede
* Remove unused functions
---------
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
2024-02-13 13:00:49 +01:00
Silvano Cerza
f96eb3847f
refactor: Merge Pipelines definition in core package ( #6973 )
...
* Move marshalling functions in core Pipeline
* Move telemetry gathering in core Pipeline
* Move run logic in core Pipeline
* Update root Pipeline import
* Add release notes
* Update Pipeline docs path
* Update releasenotes/notes/merge-pipeline-definitions-1da80e9803e2a8bb.yaml
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-12 18:25:28 +01:00
Silvano Cerza
21488ac6ca
fix: Fix Pipeline.draw() failing cause of missing socket ( #6965 )
...
* Fix Pipeline.draw() issue
* Mark draw test as flaky
2024-02-09 16:10:48 +01:00
Vladimir Blagojevic
37d9de3c4e
feat: Add service_credentials to OpenAPIServiceConnector run ( #6962 )
...
* Add service_credentials to OpenAPIServiceConnector run
* PR feedback Silvano
2024-02-09 16:03:27 +01:00
Silvano Cerza
d2d01f9fe1
feat: Enhance Pipeline.__repr__() ( #6963 )
...
* Enhance Pipeline.draw() to show image directly in Jupyter notebook
* Add util method to check if we're in a Jupyter notebook
* Split Pipeline.draw() in two methods
* Update tests
* Update releasenotes
* Enhance Pipeline.__repr__
* Simplify Pipeline.__repr__
* Update release notes
2024-02-09 14:44:34 +01:00
Silvano Cerza
a7f36fdd32
feat: Enhance Pipeline.draw() to show image directly in Jupyter notebook ( #6961 )
...
* Enhance Pipeline.draw() to show image directly in Jupyter notebook
* Add util method to check if we're in a Jupyter notebook
* Split Pipeline.draw() in two methods
* Update tests
* Update releasenotes
2024-02-09 14:44:24 +01:00
Silvano Cerza
2f965fb176
feat: Add __repr__ method to all Components ( #6927 )
...
* Add __repr__ to show Component I/O
* Add release notes
* Change Component repr to show full module path and name in Pipeline
* Fix linting
2024-02-08 11:46:10 +01:00
Bijay Gurung
74683fe74d
Feat: Add FilterRetriever ( #6836 )
...
* Add FilterRetriever draft
* Implement FilterRetriever and add tests
* Update comparison to compare whole docs instead of just contents
* Expose FilterRetriever at the retrievers level
* Update docstring (add example usage)
* Add filter_retriever in the API reference docs config
Update retriever search path to start one dir level higher
* simplify _documents_equal
* improve usage example
---------
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2024-02-08 08:48:46 +01:00
Vladimir Blagojevic
9e6a2e3cf9
fix: HuggingFaceTGIGenerator gets stuck when model is not supported ( #6915 )
...
* HuggingFaceTGIGenerator/HuggingFaceTGIChatGenerator check if model is deployed on free-tier
2024-02-06 16:55:06 +01:00
ZanSara
1182c08daf
fix: Dont filter negative scores when using BM25Okapi and scale_score=False ( #6889 )
...
* dont filter negatives for unscaled Okapi
* change BM25 algorithm default to BM25L
* Update haystack/document_stores/in_memory/document_store.py
* improve comment
2024-02-06 11:07:27 +01:00
Silvano Cerza
3d5c329949
Change Pipeline.connect() to return Pipeline instance ( #6922 )
2024-02-06 11:03:18 +01:00
Massimiliano Pippi
7d29ddba42
chore: merge hf utils modules into one ( #6921 )
...
* merge hf utils modules
* relnotes
* lint
* Update releasenotes/notes/merge-hf-utils-modules-5c16e04025123568.yaml
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-06 09:59:25 +01:00
Silvano Cerza
b9d7a98359
Rework Pipeline.connect() tests and structure ( #6917 )
2024-02-05 18:16:02 +01:00
Silvano Cerza
0191b1e6e4
feat: Change Component's I/O dunder type ( #6916 )
...
* Add Pipeline.get_component_name() method
* Add utility class to ease discoverability of Component I/O
* Move InputOutput in component package
* Rename InputOutput to _InputOutput
* Raise if inputs or outputs field already exist
* Fix tests
* Add release notes
* Move InputSocket and OutputSocket in types package
* Move _InputOutput in socket package
* Rename _InputOutput class to Sockets
* Simplify Sockets class
* Dictch I/O dunder fields in favour of inputs and outputs fields
* Update Sockets docstrings
* Update release notes
* Fix mypy
* Remove unnecessary assignment
* Remove unused logging
* Change SocketsType to SocketsIOType to avoid confusion
* Change sockets type and name
* Change Sockets.__repr__ to return component instance
* Fix linting
* Fix sockets tests
* Revert to dunder fields for Component IO
* Use singular in IO dunder fields
* Delete release notes
* Update haystack/core/component/types.py
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-05 17:46:45 +01:00
sahusiddharth
3bd6ba93ca
feat:Add dimensions parameter to OpenAI Embedders to fully support th… ( #6841 )
...
* feat:Add dimensions parameter to OpenAI Embedders to fully support the new models
* fixed linting
* changed != None to is not None
2024-02-05 16:20:46 +01:00
Madeesh Kannan
27d1af3068
feat!: Use Secret for passing authentication secrets to components ( #6887 )
...
* feat!: Use `Secret` for passing authentication secrets to components
* Add comment to clarify type ignore
2024-02-05 13:17:01 +01:00
Ashwin Mathur
393a7993c3
feat: Add Semantic Answer Similarity metric ( #6877 )
...
* Add SAS metric
* Add release notes
* Round similarity scores for precision consistency
* Add tolerance to tests
* Update haystack/evaluation/eval.py
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
* Add types for preprocess_text; Add additional types for f1 and em methods
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-02-02 17:07:52 +01:00
