4166 Commits

Author SHA1 Message Date
Amna Mubashar
720cc19d7d
feat: add serialization to State / move State to agents.state (#9345)
* Add serialization to State

* Add release notes

* Deprecate State in dataclasses

* Fix tests

* Remove state_utils test

* Fix linting

* Fix formating

* Update tests and remove old state utils

* Update agents test

* Update deserilaization per review

* Linting

* Add tests for edge case (custom class types)

* Fix type serialization

* PR comments

* Move State to agents

* Fix tests

* Update utils init

* Improve seriliaztion/deser

* Update the release notes

* Minor fix in docstrings

* PR comments

* Add deprecation warnign for state utils

* Recreate the serialization methods to use schema

* Update key names

* Make serialization methods private
2025-05-23 11:04:15 +02:00
David S. Batista
3342f17f01
feat: draw/show SuperComponents in detail, expand it and show it's internal components in the visualisation diagram (#9389)
* initial import

* small fixes

* adding tests

* adding tests

* refactoring merge graphs

* updating tests

* docstrings

* adding release notes

* adding SuperComponent name to extended components

* adding colours and legend to different super components

* adding missed docstring parameter

* fixing tests and type checking

* Update haystack/core/pipeline/base.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* forcing keyword arguments for draw() and show()

* adding wrapper function and a deprecation warning

* adding pylint disable - this will be removed soon

* wip

* adding a decorator function to test if another function is being called with positional arguments

* adding a decorator function to test if another function is being called with positional arguments

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2025-05-23 10:21:44 +02:00
David S. Batista
ba41696bba
chore: removing unused fixtures in test functions 2025-05-23 09:43:01 +02:00
Mohammed Abdul Razak Wahab
4a5e4d3e65
feat: return common type in SuperComponent type compatibility check (#9275)
* feat: return common type in SuperComponent type compatibility check

* fix test_utils

* address review comments

* update tests

* use typing module types

* refactor

* refactor

* unenforce type check

* refactor

---------

Co-authored-by: Michele Pangrazzi <xmikex83@gmail.com>
2025-05-22 12:35:31 +00:00
Vladimir Blagojevic
167229f328
feat: Extend AnswerBuilder for Agent (#9406)
* Extend AnswerBuilder for Agent

* Update tests

* Add reno note

* PR feedback

* Add a better unit test

* Update haystack/components/builders/answer_builder.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/builders/answer_builder.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* PR feedback

* Remove copy

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-05-22 14:32:36 +02:00
Sebastian Husch Lee
e6a53b9dca
fix: Add missing timeout and max_retries to OpenAITextEmbedder and OpenAIDocumentEmbedder (#9421)
* Add missing params to to_dict for OpenAI embedders

* add reno

* Track variable internally instead of using client
2025-05-22 09:19:14 +00:00
Stefano Fiorucci
17432f710d
feat: introduce SentenceTransformersSimilarityRanker (#9415)
* new component + tests

* soft deprecation of TransformersSimilarityRanker + reno

* add comp files to slow workflow

* Apply suggestions from code review

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* self.model -> self._cross_encoder

* recommend installing sentence-transformers>=4.1.0

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2025-05-21 10:52:46 +02:00
Amna Mubashar
995fa18607
feat: stream ToolResult from run_async in Agent (#9407)
* Add async run

* Add release notes

* Update the run async

* Fixes

* Fix linting

* Add tests

* Fix tests

* Remove changes from Tool

* Linting updates

* Update haystack/components/tools/tool_invoker.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* Updates tests based on comments

* Update release notes

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2025-05-21 10:22:38 +02:00
Vladimir Blagojevic
f253db3e14
feat: ComponentTool - preserve docstrings from underlying pipeline components (#9403)
* ComponentTool - preserve docstrings from underlying pipeline components

* PR feedback, adjustments
2025-05-20 13:11:49 +02:00
Stefano Fiorucci
6ad23f822f
ci: add timeout-minutes to relevant CI jobs (#9409) 2025-05-20 09:23:42 +02:00
Jan Trienes
83b087caf4
feat: add local_files_only to sentence-transformers embedders (#9400)
* feat: add  to sentence-transformers embedders

* add release note

* Fix wording

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-05-19 16:11:49 +00:00
Stefano Fiorucci
fab9e0d792
chore: run mypy --check-untyped-defs and fix some errors (#9398)
* first batch of fixes

* more fixes

* more fixes

* revert HuggingFaceLocalChatGenerator
2025-05-19 17:51:24 +02:00
Sebastian Husch Lee
707573d967
feat: Streamline using Agent as a ComponentTool (#9388)
* Make agent as a tool more streamlined

* Add reno

* fix mypy
2025-05-16 13:11:43 +02:00
dependabot[bot]
3e28ec207a
chore(deps): bump fossas/fossa-action from 1.6.0 to 1.7.0 (#9392)
Bumps [fossas/fossa-action](https://github.com/fossas/fossa-action) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/fossas/fossa-action/releases)
- [Commits](https://github.com/fossas/fossa-action/compare/v1.6.0...v1.7.0)

---
updated-dependencies:
- dependency-name: fossas/fossa-action
  dependency-version: 1.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-16 11:22:30 +02:00
Denis Washington
954b0d1033
fix: Improve typing of @component and allow TypedDict output types (#9396)
- Add type annotations to the `_Component.__call__` function that
  backs `@component`. This fixes "decorator ignored" warnings of
  Pyright in "strict" type-checking mode.

- Type-check that a `@component` class conforms to the `Component`
  protocol (i.e., that it has a `run()` method).

- Relax the `run()` return type required by the `Component` protocol
  from `Dict[str, Any]` to `Mapping[str, Any]`. The former is
  incompatible with `TypedDict`, which makes it impossible to
  type a component's precise outputs. `Mapping` fixes this because
  it is covariant, i.e. the value type may be more specific than
  `Any`.
2025-05-16 09:15:20 +02:00
Daria Fokina
2a64cd4e9a
docs: update ChatMessage examples (#9394) 2025-05-15 17:23:37 +02:00
David S. Batista
d8ff508c44
chore: removing warning and using permute to reverse the dimensions of a tensor (#9393)
* using permute to reverse the dimensions of a tensor
2025-05-15 16:56:09 +02:00
Sebastian Husch Lee
af073852d0
feat: Add usage when using HuggingFaceAPIChatGenerator with streaming (#9371)
* Small fix and update tests

* Add usage support to streaming for HuggingFaceAPIChatGenerator

* Add reno

* try using provider='auto'

* Undo provider

* Fix unit tests

* Update releasenotes/notes/add-usage-hf-api-chat-streaming-91fd04705f45d5b3.yaml

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2025-05-15 13:09:36 +02:00
Sebastian Husch Lee
9ae76e1653
Fix component tool parameters (#9342)
* Starting property schema refactor

* Adding more tests

* More tests

* Handle null type explicitly

* More updates of tests to accomodate Optional properly

* Fix more tests

* Remove unecessary check

* Some cleanup

* Update test

* Add reno

* Fix typing

* Add license header

* Use docstrings of dataclasses in parameter spec generation

* More tests of Haystack dataclass types

* Properly handle Sequence

* Fix license header

* Update OpenAI tests to add more complicated tool parameter signature

* Properly set required for dataclasses

* Add integration test for azure that includes additionalProperties

* Add more complicated integration test for HuggingFaceAPIChatGenerator

* Alternate approach using pydantic like we do in from_function.py

* Cleanup and fix other affected tests

* Fix mypy

* PR comments

* PR comment

* Remove test from HF API

* Update reno

* Update reno
2025-05-15 07:51:06 +00:00
David S. Batista
42b378950f
fix: DocumentRecallEvaluator changing division and adding checks for emptiness of documents (#9380)
* changing division and adding checks for emptiness of documents

* adding release notes

* adding tests

* Update releasenotes/notes/updated-doc-recall-eval-uniqueness-59b09082cf8e7593.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* attending PR comments

* Update releasenotes/notes/updated-doc-recall-eval-uniqueness-59b09082cf8e7593.yaml

* Update releasenotes/notes/updated-doc-recall-eval-uniqueness-59b09082cf8e7593.yaml

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update haystack/components/evaluators/document_recall.py

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update haystack/components/evaluators/document_recall.py

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update haystack/components/evaluators/document_recall.py

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update haystack/components/evaluators/document_recall.py

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* adding tests

* linting

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2025-05-14 11:37:47 +02:00
Daria Fokina
aeea3b2d39
dC to dAP (#9383) 2025-05-13 18:03:51 +02:00
Stefano Fiorucci
1541d93670
chore: remove deprecated deserialize tools inplace function (#9379)
* rm deserialize_tools_inplace + clean up

* release note
2025-05-13 09:27:36 +02:00
Sebastian Husch Lee
9f2c0679d4
Small fix and update tests (#9370) 2025-05-12 22:02:26 +02:00
Amna Mubashar
da5fc0f16c
Fix template (#9377) 2025-05-12 15:48:26 +02:00
David S. Batista
f233e06f0a
feat : adding a new Protocol for TextEmbedder (#9353)
* initial import

* removing unused imports

* adding an Embbeder Protocol

* adding tests

* adding tests

* adding release notes

* renaming dir

* removing dir

* cleaning

* adding clean tests

* dealing eith elipsis and pylint

* wip: extending tests

* cleaning extended tests

* adding an invalid TextEmbedder
2025-05-12 12:35:09 +02:00
Sebastian Husch Lee
2ccdba3e99
fix: Remove old reno and update reno (#9357)
* Remove old reno and update reno

* Update releasenotes/notes/dont-deepcopy-components-tools-b1be7ef7b4365116.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-05-08 13:09:09 +00:00
Sebastian Husch Lee
6bef2c36bb
perf: Don't deepcopy Components, Tools, or Toolsets (#9356)
* Don't copy components

* Use deepcopy_with_fallback in more places and don't deepcopy Components, Tools or Toolsets

* Slight change

* Slightly update tests

* Refactor function based on PR feedback

* Add reno

* Fix lint

* Simplify tests, rename function, PR comments

* Fix mypy

* Undo typing
2025-05-08 12:48:08 +00:00
Stefano Fiorucci
4b4b0f0041
fix: HuggingFaceAPIChatGenerator - make tool conversion compatible with huggingface_hub>=0.31.0 (#9354)
* fix: HuggingFaceAPIChatGenerator - make tool conversion compatible with huggingface_hub>=0.31.0

* relnote
2025-05-07 18:37:05 +02:00
Stefano Fiorucci
de5c7ea3d2
feat: add py.typed; adjust Component protocol (#9329)
* experimenting with py.typed

* try changing run method in protocol

* Trigger Build

* better docstring + release note

* remove type:ignore where possible

* Removed a few more type: ignores

---------

Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
2025-05-07 09:34:31 +02:00
Sebastian Husch Lee
4ce6934dd9
fix: Update deepcopying in Pipeline to have a fallback in case of error (#9346)
* First pass at fix for deepcopying inputs and outputs

* Add reno

* Add recursion for dict objects

* Bump recursion depth

* More tests and some improvments

* Fix unit tests

* PR comments
2025-05-06 11:49:45 +00:00
Amna Mubashar
64f384b52d
feat: enable streaming ToolCall/Result from Agent (#9290)
* Testing solutions for streaming

* Remove unused methods

* Add fixes

* Update docstrings

* add release notes and test

* PR comments

* add a new util function

* Adjust emit_tool_info

* PR comments

* Remove emit function, add streaming for tool_call


---------

Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
2025-05-05 16:23:44 +02:00
Stefano Fiorucci
7db719981d
fix: Component decorator type hinting (#9339)
* progress

* fix: component decorator type hinting

* improve

* rel note
2025-05-02 18:31:32 +02:00
David S. Batista
0f00c1882e
fix: make SentenceSplitter QUOTE_SPANS_RE regex ReDoS-safe (#9338)
* fix: make QUOTE_SPANS_RE regex ReDoS-safe

* Removing the capture of leading non-character on double quotes, allowing quote with new lines, adding tests

* cleaning

* fixing release notes

* changing import

* adding test for Regex Denial of Service (ReDoS)

* reducing the size/time of tests

* Update test/components/preprocessors/test_sentence_tokenizer.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update test/components/preprocessors/test_sentence_tokenizer.py

---------

Co-authored-by: Waivey <waivey@proton.me>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-05-02 15:40:17 +00:00
Mo Sriha
e5255d9061
feat: add visualization capabilities to SuperComponent (#9336)
* feat: add visualization methods to SuperComponent for pipeline representation

* refactor: update show and draw methods in SuperComponent to return None

* test: add unit tests

* add release note

* chore: update copyright year

* test: move unit tests to test_super_component

* Update releasenotes/notes/add-pipeline-viz-to-supercomponent-80165756cc777056.yaml

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-05-02 08:46:08 -05:00
Stefano Fiorucci
e3f9da13d0
test: fix test incorrectly marked as async (#9327)
* test: fix test incorrectly marked as async

* fix inmemory async tests
2025-04-30 14:07:30 +00:00
David S. Batista
201becd400
fix: RecursiveSplitter bug in the case when the recursive chunking is triggered (#9316)
* initial import

* adding release notes

* Update fixing-bug-recursive-splitter-88d5714529f84e4e.yaml
2025-04-30 13:03:23 +02:00
Stefano Fiorucci
f8eead3adc
ci: disable caching on Windows tests (#9318)
* ci: try skipping cache on Windows

* trigger

* remove trigger

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-04-29 18:21:34 +02:00
David S. Batista
04e4701a17
chore: cleaning unused imports from core tests 2025-04-29 18:09:11 +02:00
David S. Batista
07f4bf5522
chore: cleaning imports and unused variables in Component tests 2025-04-29 16:27:34 +02:00
David S. Batista
d61f9f7f68
feat: validation function for run() and run_async() parameters signature for (custom) components (#9322)
* adding tests

* adding release notes

* small improvements
2025-04-29 13:53:24 +00:00
Yassin Nouh
ed6176a8cb
fix: make HuggingFaceAPIChatGenerator convert Tool Call arguments from string (#9303)
* fix: sort imports in hugging_face_api.py

* fix: import logging in hugging_face_api.py

* fix: refactor HuggingFace API tool call handling for improved argument conversion

* Update haystack/components/generators/chat/hugging_face_api.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* refinements + tests + relnote

* simplify

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-04-28 15:36:19 +02:00
Mohammed Abdul Razak Wahab
53308a6294
feat: Add sanitization for Meta field during serialization (#9272)
* feat: Add sanitization for Meta field during serialization

* Revert "feat: Add sanitization for Meta field during serialization"

This reverts commit c529f7c25b69aed626bb2072c8bf171815b591cc.

* feat: add nested serialization in openai usage object

* add reno

* add nested serialization in OpenAiChatGenerator

* Update releasenotes/notes/nested-serialization-openai-usage-object-3817b07342999edf.yaml

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* merge tests

* Adjust the test

---------

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
2025-04-26 15:04:02 +05:00
Sebastian Husch Lee
0fdb88424b
fix: Fix Azure test on forks (#9312)
* Fix unit test

* Fix test
2025-04-25 11:10:59 +02:00
Stefano Fiorucci
38c39a49de
test: review integration tests (#9306)
* AzureOCR: convert integration test to unit test and simplify

* clean up HuggingFaceAPITextEmbedder

* clean up LinkContentFetcher

* simplify HuggingFaceLocalGenerator

* clean up OpenAIGenerator

* OpenAIChatGenerator

* SentenceTransformersDiversityRanker

* TransformersSimilarityRanker

* ChatMessage: rm outdated tests

* fail fast false

* typo
2025-04-25 09:07:57 +02:00
Mohammed Abdul Razak Wahab
f97472329f
feat: Add support for multiple outputs in ConditionalRouter (#9271)
* feat: Add support for multiple outputs in ConditionalRouter

* Update haystack/components/routers/conditional_router.py

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>

* add additional route

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
2025-04-24 16:17:06 +02:00
Michele Pangrazzi
4a908d075e
Fix OpenAIGenerator and OpenAIChatGenerator to allow wrapped streaming objects usage (#9304)
* Fix for handling wrapped ChatCompletion instances in streaming (used by tools like weave)

* Add release note

* Applied same fix to OpenAIGenerator ; Refactoring ; Update release note

* Fix integration test error after refactoring
2025-04-24 16:16:41 +02:00
Sebastian Husch Lee
d806ccf22a
fix: Update to ComponentTool __deepcopy__ (#9302)
* Hopefully a fix

* Add reno
2025-04-24 14:26:45 +02:00
Sebastian Husch Lee
7617304077
fix: Move deserialize_tools_inplace to original import path (#9301)
* Fix for preserving import path

* Move deserialize_tools_inplace back to original import path

* Add reno

* Make mypy happy

* Remove unused import
2025-04-24 11:59:16 +00:00
Stefano Fiorucci
c3bce46663
docs: update Contribution guidelines with a section about Slow/Unstable tests (#9300)
* docs: update Contribution guidelines with a section about Slow/Unstable tests

* Apply suggestions from code review

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2025-04-24 12:23:43 +02:00
Stefano Fiorucci
e3d4e21237
test: mark more tests as slow (#9296)
* test: mark tests as slow

* alphabetical order; install xet

* revert pyproject

* Trigger Build

* simplify tests as suggested

* add comment to workflow
2025-04-24 10:25:13 +02:00