1775 Commits

Author SHA1 Message Date
Stefano Fiorucci
646eedf26a
chore: reenable HF API Embedders tests + improve HFAPIChatGenerator docstrings (#9589)
* chore: reenable some HF API tests + improve docstrings

* revert deletion
2025-07-04 09:39:43 +02:00
Amna Mubashar
050c987946
chore: remove backward compatibility for State deserialization (#9585)
* remove backward compatability

* Fix linting
2025-07-03 13:20:34 +02:00
Sebastian Husch Lee
85258f0654
fix: Fix types and formatting pipeline test_run.py (#9575)
* Fix types in test_run.py

* Get test_run.py to pass fmt-check

* Add test_run to mypy checks

* Update test folder to pass ruff linting

* Fix merge

* Fix HF tests

* Fix hf test

* Try to fix tests

* Another attempt

* minor fix

* fix SentenceTransformersDiversityRanker

* skip integrations tests due to model unavailable on HF inference

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-07-03 09:49:09 +02:00
Sebastian Husch Lee
16fc41cd95
feat: Relax requirement for creating a ToolCallDelta dataclass (#9582)
* Relax our requirement for ToolCallDelta to better match ChoiceDeltaToolCall and ChoiceDeltaToolCallFunction from OpenAI

* Add reno

* Update tests
2025-07-03 08:50:29 +02:00
Amna Mubashar
9fd552f906
chore: remove deprecated async_executor param from ToolInvoker (#9571)
* Remove async executor

* Add release notes

* Linting

* update release notes
2025-07-02 14:02:51 +02:00
Amna Mubashar
adb2759d00
chore: remove deprecated State from haystack.dataclasses (#9578)
* Remove deprecated class

* Remove state from pydocs
2025-07-02 12:19:06 +02:00
Stefano Fiorucci
848115c65e
fix: fix print_streaming_chunk + add tests (#9579)
* fix: fix print_streaming_chunk + add tests

* rel note
2025-07-01 16:49:18 +02:00
Sebastian Husch Lee
3aaa201ed6
feat: Add tool_invoker_kwargs to Agent (#9574)
* Add new param to agent to pass any kwargs to tool invoker

* Add reno
2025-07-01 11:09:58 +02:00
Amna Mubashar
f11870b212
fix: adjust the async executor in ToolInvoker (#9562)
* Fix bug

* Add a new test

* PR comments

* Add another test

* Small fix

* Fix linting

* Update tests
2025-06-30 15:13:01 +02:00
Sebastian Husch Lee
97e72b9693
feat: Add to_dict and from_dict to ByteStream (#9568)
* Add to_dict and from_dict to ByteStream

* Add reno

* Add unit tests

* Fix and expand tests

* Fix typing

* PR comments
2025-06-30 11:57:22 +00:00
Sebastian Husch Lee
fc64884819
fix: Fix _convert_streaming_chunks_to_chat_message (#9566)
* Fix conversion

* Add reno

* Add unit test
2025-06-30 11:51:25 +02:00
Amna Mubashar
1cd0a128d0
feat: enable parallel tool execution in ToolInvoker (#9530)
* Enable parallel tool execution in ToolInvoker

* Update handling of errors

* Small fixes

* Small fixes

* Adapt number of executors

* Add release notes

* Add parallel tool calling to sync run

* Deprecate async_executor

* Deprecate async_executor

* Add thread lock

* extract methods

* Update release notes

* Update release notes

* Updates

* Add new tests

* Add test for async

* PR comments
2025-06-25 13:32:11 +02:00
Vladimir Blagojevic
91094e1038
feat: Add finish_reason field to StreamingChunk (#9536)
* Initial commit

* Update deprecation version

* Improve comment

* Minor simplification

* Add reno note

* Remove deprecation warning

* Remove fallback in haystack/components/generators/utils.py

* FinishReason alphabetical import

* Add tool_call_results finish reason, adapt codebase

* Define finish_reason to be Optional[FinishReason]

* Add StreamingChunk finish_reason in HF generators

* Update reno note

* Repair merge issue

* Update tests for finish_reason

* Resolve mypy issues

* Lint issue

* Enhance HF finish_reason translation

* Remove irrlevant test

* PR comments

---------

Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
2025-06-25 09:06:01 +00:00
Stefano Fiorucci
0d0a66b4f5
feat: add LLMMessagesRouter, a component to route Chat Messages using LLMs (#9540)
* llmmessagesrouter - draft

* serde methods

* refinements, tests and release note

* Apply suggestions from code review

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2025-06-24 14:54:20 +02:00
Amna Mubashar
9ed0b9b0bc
fix: Update the de/serialization with schema utils (#9526)
* Update the util methods

* Update tests

* fix tests

* schema fix

* Add json schema for tuples and sets

* Add proper conversion for sets and tuples

* Adjust typing

* PR comments

* Linting

* Optimize deserialization

* remove TODO

* PR comments

* PR comments

* Update tests and deserialization error

* Support legacy deserialization

* Update deprecating warning

* Update test
2025-06-24 13:10:12 +02:00
Stefano Fiorucci
d14f5dca0e
feat: add trust_remote_code parameter to SentenceTransformersSimilarityRanker (#9546) 2025-06-24 11:39:59 +02:00
Sebastian Husch Lee
ec371387f0
refactor: Update to StreamingChunk, better index setting and change tool_call to tool_calls (#9525)
* Fixes to setting StreamingChunk.index properly and refactoring tests for conversion

* Make _convert_chat_completion_chunk_to_streaming_chunk a member of OpenAIChatGenerator so we can overwrite it in integrations that inherit from it

* Fixes

* Modify streaming chunk to accept a list of tool call deltas.

* Fix tests

* Fix mypy and update original reno

* Undo change

* Update conversion to return a single streaming chunk

* update to print streaming chunk

* Fix types

* PR comments
2025-06-23 08:14:25 +00:00
Ahmad Zidan
f911459647
feat: add resource name for Haystack Component Datadog spans (#9337)
* feat: add resource name for Haystack Component Datadog spans

* fest: format resource name

Signed-off-by: Ahmad Zidan <ahmad.zidan@traveloka.com>

* feat: add release notes

Signed-off-by: Ahmad Zidan <ahmad.zidan@traveloka.com>

---------

Signed-off-by: Ahmad Zidan <ahmad.zidan@traveloka.com>
Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2025-06-18 09:15:15 +00:00
Sebastian Husch Lee
3784889e5d
fix: Fix Tool and ComponentTool serialization when specifying outputs_to_string (#9524)
* Fix serialization of outputs_to_string in Tool and ComponentTool

* Add reno

* Fix mypy, simplify logic

* fix pylint

* Fix test

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-06-18 11:00:46 +02:00
Stefano Fiorucci
a16ee96003
fix: fix SuperComponent class serialization/deserialization for async Pipelines (#9527)
* draft

* better test + release note

* improve test
2025-06-18 08:17:52 +00:00
Sriniketh J
6198f0cba9
feat: adding support for torch xpu device (#9470)
* feat: add support for torch xpu device support

* test: xpu based tests ci/cd

* test: add xpu code device support

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-06-17 14:15:19 +02:00
baki gul
7dbac5b3c9
Fixes incorrect ID generation for identical chunks in RecursiveDocumentSplitter (#9517)
* fix(preprocessor): ensure RecursiveDocumentSplitter generates unique chunk IDs

* fix: update meta handling in RecursiveDocumentSplitter to ensure correct overlap information

---------

Co-authored-by: Michele Pangrazzi <xmikex83@gmail.com>
2025-06-16 21:49:00 +02:00
Sebastian Husch Lee
a1484cb91c
Add unit test (#9519) 2025-06-16 13:14:02 +02:00
Sebastian Husch Lee
379df4ab84
feat: Warn users if Agent is called with only system messages (#9514)
* Add warning message and raise error in agent run method

* Add tests

* Add reno

* Updates

* Updates
2025-06-13 14:58:50 +02:00
Mohammed Abdul Razak Wahab
a28b2851d9
feat: Add async streaming support in HuggingFaceLocalChatGenerator (#9405)
* feat: Add async streaming support in hugging face generator

* enforce streamingcallback to be async

* refactor

* fix: schedule and await async task in Event Loop

* unenforce typecheck

* add integration test

* After merge fixes:
- fix breaking tests
- added component_info to AsyncHFTokenStreamingHandler

* fix integration test

* refactor: improve async handling in HuggingFaceLocalChatGenerator and update tests

* fix typo

* address review comments

* refactors

* typo

* refactor
2025-06-11 14:50:25 +00:00
Stefano Fiorucci
f8155e1b77
chore: clean up (#9504) 2025-06-11 11:05:05 +02:00
Sebastian Husch Lee
54c5057e0b
feat: (and fix) Add enable_streaming_passthrough to ToolInvoker and add missing params to to_dict (#9498)
* Fixes and tests

* Add reno

* Change variable name

* Add test and fix for passing streaming_callback to a component tool

* Add unit test

* Remove unused import

* Fix reno
2025-06-06 14:16:05 +02:00
Amna Mubashar
1d6a9f652a
fix: serialization of nested ChatMessage in GeneratedAnswerdataclass (#9497)
* Fix serialization

* small fix

* fix the erros

* Fix tests

* PR comments
2025-06-06 11:46:24 +02:00
Sebastian Husch Lee
b61886b138
feat: Update streaming chunk (#9424)
* Start expanding StreamingChunk

* First pass at expanding Streaming Chunk

* Working version!

* Some tweaks and also make ToolInvoker stream a chunk with a finish reason

* Properly update test

* Change to tool_name, remove kw_only since its python 3.10 only and update HuggingFaceAPIChatGenerator to start following new StreamingChunk

* Add reno

* Some cleanup

* Fix unit tests

* Fix mypy and integration test

* Fix pylint

* Start refactoring huggingface local api

* Refactor openai generator and chat generator to reuse util methods

* Did some reorg

* Reusue utility method in HuggingFaceAPI

* Get rid of unneeded default values in tests

* Update conversion of streaming chunks to chat message to not rely on openai dataclass anymore

* Fix tests and loosen check in StreamingChunk post_init

* Fixes

* Fix license header

* Add start and index to HFAPIGenerator

* Fix mypy

* Clean up

* Update haystack/components/generators/utils.py

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update haystack/components/generators/utils.py

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Change StreamingChunk.start to only a bool

* PR comments

* Fix unit test

* PR comment

* Fix test

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2025-06-06 08:17:02 +00:00
Stefano Fiorucci
f85ce19a32
test: replace tool calling model in tests with Qwen2.5-72B-Instruct (#9500) 2025-06-06 08:42:46 +02:00
Sebastian Husch Lee
8e21c501df
fix: Fix serialization and deserialization of ConditionalRouter with multiple outputs (#9490)
* Fix sede of ConditionalRouter with multiple outputs

* Add reno
2025-06-05 15:57:24 +02:00
Vladimir Blagojevic
853a32f8da
feat: Improve ChatMessage _deserialize_content ValueError - make it more LLM friendly (#9484)
* Improve ChatMessage _deserialize_content ValueError - make it more LLM friendly

* Add unit test

* Add reno note

* Add descriptive ValueError for missing role

* Update haystack/dataclasses/chat_message.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update releasenotes/notes/improve-chatmessage-error-messages-llm-agents-a1b2c3d4e5f6g7h8.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Add role check in ChatMessage

* fixes + refinements

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-06-04 15:14:05 +00:00
Sebastian Husch Lee
ff56363db1
fix: In set_output_types check that the decorator @component.output_types is not present on the run_async method (#9485)
* Fix

* Add reno
2025-06-04 12:17:47 +02:00
Stefano Fiorucci
1e2214a1a0
feat: ChatMessage.to_openai_dict_format - add require_tool_call_ids parameter (#9481) 2025-06-03 16:55:13 +02:00
Sebastian Husch Lee
ce0917e586
feat: Add raise_on_failure boolean parameter to OpenAIDocumentEmbedder and AzureOpenAIDocumentEmbedder (#9474)
* Add raise_on_failure to OpenAIDocumentEmbedder

* Add reno

* Add parameter to Azure Doc embedder as well

* Fix bug

* Update reno

* PR comments

* update reno
2025-06-03 10:22:34 +00:00
Sebastian Husch Lee
5fcd7c4732
feat: Allow passing of additional parameters to HF Inference clients in HuggingFaceAPIChatGenerator and HuggingFaceAPIGenerator (#9457)
* Fix tests by allowing passing of provider

* Add reno

* Fix mypy

* Update release note
2025-06-03 10:21:51 +00:00
Sebastian Husch Lee
12e3de364a
Fix test (#9475) 2025-06-03 08:00:10 +00:00
David S. Batista
b85c8e3382
feat: adding deserialize_component_inplace() (#9459)
* adding tests

* adding release notes

* deserialize_chatgenerator_inplace uses deserialize_component_inplace

* removing tests
2025-06-02 09:40:35 +02:00
Stefano Fiorucci
2616d4d55b
test: speed up some tests + minor refactorings (#9451)
* this is an integration test

* more improvements

* rm redundant comments
2025-05-29 09:49:11 +02:00
Sebastian Husch Lee
81c0cefa41
refactor: Refactor hf api chat generator (#9449)
* Refactor HFAPI Chat Generator

* Add component info to generators

* Fix type hint

* Add reno

* Fix unit tests

* Remove incorrect dev comment

* Move _convert_streaming_chunks_to_chat_message to utils file
2025-05-27 15:55:06 +02:00
atopx
3deaa20cb6
feat: Add HuggingFace API (text-embeddings-inference for rerank model) for component.rankers (#9414)
* feat(component.rankers): Add HuggingFace API (text-embeddings-inference for rerank) ranker component

* update test flow & doc loaders

* Support run_async for HuggingFaceAPIRanker

* Add release note for HuggingFace API support in component.rankers

* Add release note for HuggingFace API support in component.rankers

* Add release note for HuggingFace API support in component.rankers

* Add release note for HuggingFace API support in component.rankers

* fix:
1. `hugging_face_api.HuggingFaceAPIRanker` rename to `hugging_face_tei.HuggingFaceAPIRanker`
2. HuggingFaceAPIRanker: use our Secret API for token
3. add the missing modules for `docs/pydoc/config/rankers_api.yml`
4. added function `async_request_with_retry` for `haystack/utils/requests_utils.py` and added unittest on `test/utils/test_requests_utils.py`
4. HuggingFaceAPIRanker: refactor the retry function to support configuration based on attempts and status code.
5. HuggingFaceAPIRanker: refactor the test into unit tests using mocks

* fix(HuggingFaceTEIRanker): change the token check logic to use the resolve_value method.

* fix(format): run `hatch run format`

* fix:
- Force keyword-only arguments in __init__ method by adding *,
- Clarify token docstring that it's not always required
- Copy documents to avoid modifying original objects
- Remove test file from slow workflow
- Add monkeypatch eånvironment variable cleanup in tests
- Fix missing module in rankers_api.yml and sort modules alphabetically
- Remove unnecessary test info from release notes

* fix HuggingFaceTEIRanker:
- "None" of "Optional[Secret]" has no attribute "resolve_value"
- run/run_async: too many parameters

* fix(HuggingFaceTEIRanker) :Revise the docstring of the HuggingFaceTEIRanker, improve the parameter descriptions, ensure consistency and clarity. Add error handling information to enhance the readability of the API response.

* fix:unit test for HuggingFaceTEIRanker raise message

* fix fmt

* minor refinements

* refine release note

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-05-27 12:44:54 +02:00
Sebastian Husch Lee
db3d95b12a
refactor: Refactor openai generator (#9445)
* Refactor openai generator and chat generator to reusue same util methods

* Start fixing tests

* More fixes

* Fix mypy

* Fix
2025-05-27 12:44:17 +02:00
Amna Mubashar
64def6d41b
feat: add component name and type to StreamingChunk (#9426)
* Stream component name in openai

* Fix type

* PR comments

* Update huggingface gen

* Typing fix

* Update huggingfacelocal gen

* Fix errors

* Remove model changes

* Fix minor errors

* Update releasenotes/notes/add-component-info-dataclass-be115dee2fa50abd.yaml

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* PR comments

* update annotation

* Update hf files

* Fix linting

* Add a from_component method

* use add_component

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2025-05-27 12:23:40 +02:00
Stefano Fiorucci
085c3add41
ci: prevent DocumentWriter tests from blocking CI (#9448) 2025-05-27 12:10:21 +02:00
Stefano Fiorucci
d8487c4d8d
chore: make mypy run with --check-untyped-defs; fix some errors (#9447)
* chore: make mypy run with --check-untyped-defs; fix some errors

* small fixes

* use HfPipeline

* fix license error
2025-05-27 07:35:25 +00:00
David S. Batista
da60156174
chore: removing unused imports from tests (#9446) 2025-05-26 16:22:51 +00:00
David S. Batista
2092bedb90
chore: removing unused imports from tests (#9444) 2025-05-26 13:41:36 +00:00
David S. Batista
c82a3377f2
chore: cleaning up tests (#9443) 2025-05-26 15:12:19 +02:00
Sebastian Husch Lee
0d23fe7f0d
fix: Fix invoker to work when using dataclass with from_dict but dataclass… (#9434)
* Fix invoker to work when using dataclass with from_dict but dataclass is already given

* add reno

* Add unit test

* Remove line
2025-05-26 09:40:09 +02:00
Seth Peters
f025501792
fix: LLMMetadataExtractor bug in handling Document objects with no content
* test(extractors): Add unit test for LLMMetadataExtractor with no content

Adds a new unit test `test_run_with_document_content_none` to `TestLLMMetadataExtractor`.

This test verifies that `LLMMetadataExtractor` correctly handles documents where `document.content` is None or an empty string.

It ensures that:

- Such documents are added to the `failed_documents` list.

- The correct error message ("Document has no content, skipping LLM call.") is present in their metadata.

- No actual LLM call is attempted for these documents.

This test provides coverage for the fix that prevents an AttributeError when processing documents with no content.

* chore: update comment to reflect new behavior in _run_on_thread method

* docs: Add release note for LLMMetadataExtractor no content fix

* Update releasenotes/notes/fix-llm-metadata-extractor-no-content-910067ea72094f18.yaml

* Update fix-llm-metadata-extractor-no-content-910067ea72094f18.yaml

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-05-23 18:57:39 +02:00