Tobias Wochinger
419009b495
fix: move sensitive log to debug mode ( #7230 )
2024-02-28 09:45:50 +01:00
Tobias Wochinger
14f65d5a68
feat: check tty / jupyter availability for logging config detection ( #7226 )
...
* feat: check tty availability for logging config detection
* chore: support jupyter + lower env variable
* chore: also use jupyter util for check
* test: set right mode
2024-02-27 18:31:51 +01:00
Vladimir Blagojevic
d871bbbfbd
feat: Add complex types in OpenAPI support ( #7065 )
...
* Add complex types OpenAPI support
* Add release note
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-02-27 18:11:06 +01:00
Massimiliano Pippi
93acdc1d3a
refactor: Refactor pipeline templates ( #7213 )
...
* merge template source into pipeline template
* docstrings
* rework pipeline templates using Jinja features
* amend release notes
* linting
* Apply suggestions from code review
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
* specify QA is generative
* amend docstring
* review feedback
* forgot one comment
* add template source to the error message for easy debugging
* fix tests
* try
* annotate return type of build function
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-02-27 17:39:23 +01:00
Tobias Wochinger
6fce2dbb11
feat: log trace correlation ( #7206 )
...
* feat: implement support for trace-log correlation
* docs: add release notes
2024-02-27 10:06:53 +01:00
Tobias Wochinger
2a591280ab
feat: implement support for structured logging ( #7126 )
...
* feat: implement support for structured logging
* docs: add release notes
* style: add explanatory comment
* chore: test + import fixes
* tests: fix windows tests
2024-02-27 09:15:01 +01:00
Julian Risch
22e9def2cd
chore: Remove deprecated GPTGenerator and GPTChatGenerator ( #7125 )
...
* remove deprecated GPTGenerator and GPTChatGenerator
* remove unused import
2024-02-26 16:06:26 +01:00
Stefano Fiorucci
3b8af365e3
restore transparent background for Pipeline images ( #7086 )
2024-02-26 09:25:07 +01:00
Silvano Cerza
0a7dfc1b32
Revert "Add AnswerExactMatchEvaluator ( #7050 )" ( #7075 )
...
This reverts commit b4011af8e9bc4ae2f72e51db254bfda69e20b651.
2024-02-23 14:05:57 +01:00
Tobias Wochinger
41b93108c0
feat: allow content tracing ( #7066 )
...
* feat: implement content tracing capability
* feat: trace component input/output
* chore: extend type serialization to classes with `to_dict`
* docs: add option for content tracing
* style: remove unused import
* chore: add more tags
* style: ignore typing
* fix: fix tagging of types
2024-02-23 11:52:54 +01:00
Silvano Cerza
b4011af8e9
Add AnswerExactMatchEvaluator ( #7050 )
...
* Add AnswerExactMatchEvaluator
* Add release notes
* Fix linting
* Update docstrings
2024-02-23 10:37:18 +01:00
Tobias Wochinger
bc8a48cc3c
feat: datadog tracer ( #7058 )
...
* feat: implement datadog tracer
* feat: autoenable for ddtrace
* docs: add release notes
* ci: add missing test dependency
* chore: use forward references
2024-02-23 09:18:32 +01:00
Tobias Wochinger
6d0d373def
feat: opentelemetry tracer ( #7052 )
...
* feat: add opentelemetry tracer
* feat: implement auto enable for `OpenTelemetry`
* docs: add release notes
* style: fix linting
* tests: more test coverage
* refactor: fix old method call
* docs: fix duplicate `the`
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-22 14:30:58 +01:00
Vladimir Blagojevic
cb6389d7a2
feat: Improve OpenAPI integration ( #7034 )
...
* Simplify and improve OpenAPIServiceConnector and OpenAPIServiceToFunctions, add unit tests
* Add reno note
* Add flask test dependency
* Initial PR feedback - Julian
* Remove indirection - Silvano
* Remove flask end-to-end tests
* Remove unused import
* Add mixed body unit test
* Update unit test, mock properly
2024-02-22 14:03:50 +01:00
Tobias Wochinger
6e580e4430
feat: implement pipeline tracing ( #7046 )
...
* feat: implement pipeline tracing
* tests: improve test setup for spying tracer
* feat: implement util for type coercion
* fix: trace a after checking pipeline output
* docs: add release notes
* docs: drop unused imports
* refactor: simplify getting raw span
* refactor: implement `ProxyTracer`
2024-02-22 12:52:04 +01:00
ZanSara
34edaa1105
fix formatting ( #7059 )
2024-02-22 10:39:02 +01:00
Massimiliano Pippi
f3be576b5c
refact: remove the concept of TemplateSource from the public API ( #7051 )
...
* remove unused field
* hide the TemplateSource abstraction
* amend release notes
* revert
2024-02-21 12:02:04 +01:00
Varun Mathur
b335b5d723
feat: Add Lost In The Middle Ranker ( #6995 )
...
* add lost in the middle ranker
* update
* add release notes
* update release notes
* fix mypy
* Update
* fix mypy
* fix mypy [union-attr] for content.split
* remove e2e tests and negative topk param
* remove query param, validate params
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-02-20 19:55:41 +01:00
Ashwin Mathur
327c2d260d
feat: Add Mean Reciprocal Rank (MRR) metric to StatisticalEvaluator ( #7042 )
...
* Add MRR Metric
* Add release notes
* Update logic
2024-02-20 13:58:48 +01:00
Stefano Fiorucci
d00f171f8b
refactor!: Sentence Transformers Embedders - new devices mgmt ( #7033 )
...
* new device mgmt for Sentence Transformers embedders
* reno
2024-02-19 14:52:44 +01:00
Vladimir Blagojevic
cb01cb4207
feat: Add PipelineTemplate for ready made pipelines ( #7001 )
...
* Initial working version
* More experiments
* Use jinja2 tempplate extensions
* Remove for_type static method
* Add ternary filter, with_kwargs
* Add indexing pipeline
* Rename with_kwargs -> with_parameters, allow cummulative invocation
* Remove ternary filter, use another approach to conditionals in default component configuration
* Massi feedback, add pydocs
* PipelineTemplateBuilder -> PipelineTemplate
* Add unit tests
* Add reno note
* Fix pylint, mypy
* PR review - Madeesh
* Add list_variables
* PR review feedback - Madeesh
* Update haystack/templates/pipelines.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Add timeout, fix pylint
* Use full import path
* Update release note
* Fix failing windows test
* Update haystack/templates/source.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* Update haystack/templates/source.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
* PR feedback Madeesh
* Fix pylint
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-02-19 13:59:48 +01:00
Silvano Cerza
f1a6b2a78a
feat: Add is_greedy argument in @component decorator ( #7016 )
...
* Add is_greedy argument in @component decorator
* Log warning if Component is greedy and non variadic
2024-02-19 12:43:40 +01:00
Silvano Cerza
5f97e08feb
feat: Reintroduce max_loops_allowed check in Pipeline.run() ( #7010 )
...
* Reintroduce max_loops_allowed check in Pipeline.run()
* Add release notes
2024-02-19 10:05:35 +01:00
Stefano Fiorucci
3f85a63468
fix: deepcopy the inputs of components ( #6987 )
...
* deepcopy inputs
* reno
* simplify test
2024-02-16 17:07:52 +01:00
Stefano Fiorucci
0aa788facc
refactor!: LocalWhisperTranscriber - new devices mgmt ( #7008 )
...
* wip
* whisper local transcriber: use new device mgmt
* better from_dict + test
* reno
2024-02-16 11:25:53 +01:00
Tuana Çelik
e2cee468fc
fix: Adding api_base_url to OpenAITextEmbeder self assignments ( #7004 )
...
* assigning api_base_url
This fix resolves issues with the MistralTextEmbedder integration
* adding base url to `to_dict` and the tests
* adding release note
* Update fix-openai-base-url-assignment-0570a494d88fe365.yaml
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-15 17:35:28 +01:00
Silvano Cerza
c82f787b41
feat: Add TextCleaner component ( #6997 )
...
* Add TextCleaner component
* Update docstrings and simplify run logic
* Update docstrings
2024-02-15 16:10:38 +01:00
Vladimir Blagojevic
5a8d02064b
feat: Add JsonSchemaValidator ( #6937 )
...
* Add JsonSchemaValidator
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-15 14:07:01 +01:00
Silvano Cerza
cf221a9701
Delete old eval API ( #6983 )
2024-02-14 17:11:08 +01:00
Silvano Cerza
36ab23d360
feat: Add StatisticalEvaluator component ( #6982 )
...
* Add StatisticalEvaluator component
* Remove F1 and Exact Metric from old API
* Add release notes
* Update docstrings
2024-02-14 16:48:03 +01:00
Silvano Cerza
9297fca520
feat: Add SASEvaluator component ( #6980 )
...
* Add SASEvaluator component
* Add release notes
* Delete old tests
* Remove SAS metric in old API
* Avoid importing whole numpy package
2024-02-14 16:16:22 +01:00
Sebastian Husch Lee
ea7275955d
feat: Meta field ranker add meta_value_type ( #6977 )
...
* Update MetaFieldRanker to parse string meta values based on meta_value_type
* Add some unit tests
* Add another unit test
* Add release notes
* Fix mypy
* Fix pylint
* Add more unit tests
* Update release notes
* Update docs
* Further improve doc strings
2024-02-13 13:08:35 +01:00
Vladimir Blagojevic
97a0df66d2
feat: Add OutputAdapter ( #6936 )
...
* Add OutputAdapter component
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-13 13:03:50 +01:00
Silvano Cerza
f96eb3847f
refactor: Merge Pipelines definition in core package ( #6973 )
...
* Move marshalling functions in core Pipeline
* Move telemetry gathering in core Pipeline
* Move run logic in core Pipeline
* Update root Pipeline import
* Add release notes
* Update Pipeline docs path
* Update releasenotes/notes/merge-pipeline-definitions-1da80e9803e2a8bb.yaml
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2024-02-12 18:25:28 +01:00
Vladimir Blagojevic
37d9de3c4e
feat: Add service_credentials to OpenAPIServiceConnector run ( #6962 )
...
* Add service_credentials to OpenAPIServiceConnector run
* PR feedback Silvano
2024-02-09 16:03:27 +01:00
Silvano Cerza
d2d01f9fe1
feat: Enhance Pipeline.__repr__() ( #6963 )
...
* Enhance Pipeline.draw() to show image directly in Jupyter notebook
* Add util method to check if we're in a Jupyter notebook
* Split Pipeline.draw() in two methods
* Update tests
* Update releasenotes
* Enhance Pipeline.__repr__
* Simplify Pipeline.__repr__
* Update release notes
2024-02-09 14:44:34 +01:00
Silvano Cerza
a7f36fdd32
feat: Enhance Pipeline.draw() to show image directly in Jupyter notebook ( #6961 )
...
* Enhance Pipeline.draw() to show image directly in Jupyter notebook
* Add util method to check if we're in a Jupyter notebook
* Split Pipeline.draw() in two methods
* Update tests
* Update releasenotes
2024-02-09 14:44:24 +01:00
Silvano Cerza
2f965fb176
feat: Add __repr__ method to all Components ( #6927 )
...
* Add __repr__ to show Component I/O
* Add release notes
* Change Component repr to show full module path and name in Pipeline
* Fix linting
2024-02-08 11:46:10 +01:00
Bijay Gurung
74683fe74d
Feat: Add FilterRetriever ( #6836 )
...
* Add FilterRetriever draft
* Implement FilterRetriever and add tests
* Update comparison to compare whole docs instead of just contents
* Expose FilterRetriever at the retrievers level
* Update docstring (add example usage)
* Add filter_retriever in the API reference docs config
Update retriever search path to start one dir level higher
* simplify _documents_equal
* improve usage example
---------
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2024-02-08 08:48:46 +01:00
Silvano Cerza
b49e86d007
fix: Change PromptBuilder to have default values for all inputs ( #6934 )
...
* Change PromptBuilder to have default values for all inputs
* Add release notes
2024-02-07 16:03:17 +01:00
Vladimir Blagojevic
9e6a2e3cf9
fix: HuggingFaceTGIGenerator gets stuck when model is not supported ( #6915 )
...
* HuggingFaceTGIGenerator/HuggingFaceTGIChatGenerator check if model is deployed on free-tier
2024-02-06 16:55:06 +01:00
Silvano Cerza
b875eda4af
refactor: Rename Pipeline.connect() arguments ( #6919 )
...
* Rename Pipeline.connect() arguments
* Add release notes
2024-02-06 11:31:53 +01:00
Silvano Cerza
3d5c329949
Change Pipeline.connect() to return Pipeline instance ( #6922 )
2024-02-06 11:03:18 +01:00
Massimiliano Pippi
7d29ddba42
chore: merge hf utils modules into one ( #6921 )
...
* merge hf utils modules
* relnotes
* lint
* Update releasenotes/notes/merge-hf-utils-modules-5c16e04025123568.yaml
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
---------
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-02-06 09:59:25 +01:00
sahusiddharth
3bd6ba93ca
feat:Add dimensions parameter to OpenAI Embedders to fully support th… ( #6841 )
...
* feat:Add dimensions parameter to OpenAI Embedders to fully support the new models
* fixed linting
* changed != None to is not None
2024-02-05 16:20:46 +01:00
Madeesh Kannan
27d1af3068
feat!: Use Secret for passing authentication secrets to components ( #6887 )
...
* feat!: Use `Secret` for passing authentication secrets to components
* Add comment to clarify type ignore
2024-02-05 13:17:01 +01:00
Ashwin Mathur
393a7993c3
feat: Add Semantic Answer Similarity metric ( #6877 )
...
* Add SAS metric
* Add release notes
* Round similarity scores for precision consistency
* Add tolerance to tests
* Update haystack/evaluation/eval.py
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
* Add types for preprocess_text; Add additional types for f1 and em methods
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-02-02 17:07:52 +01:00
Silvano Cerza
461556cca2
fix: Fix language servers never working with Components ( #6893 )
...
* Fix language servers never working with Components
* Add release notes
2024-02-02 15:59:05 +01:00
ZanSara
1039c73553
feat: Allow setting metadata for ByteStream when created from file or from string ( #6857 )
...
* add params
* add tests
* reno
* add default
* defreeze
2024-02-01 12:50:11 +01:00
Madeesh Kannan
b772c1127c
feat: Implement Secret for structured authentication ( #6855 )
...
* feat: Implement `AuthPolicy` for structured authentication
* Rename `AuthPolicy` to `Secret`
* Update release notes, fix typo
2024-01-31 12:51:14 +01:00
