59 Commits

Author SHA1 Message Date
Amna Mubashar
071d2f18b9
feat: Add extra field in ToolCall and ToolCallDelta (#10018)
* Add extra to Toolcall

* Add release notes

* Update tests
2025-11-05 13:35:32 +01:00
Sebastian Husch Lee
7443efe2f8
feat: Add reasoning content to streaming chunk (#9777)
* Add reasoning content to streaming chunk

* Add reno

* Update print_streaming_chunk
2025-09-15 13:44:12 +02:00
Stefano Fiorucci
95dafdc20b
fix: reintroduce helpful error message in ChatMessage deserialization (#9748)
* fix: reintroduce helpful error message in  deserialization

* fix fmt
2025-08-28 15:27:09 +02:00
Stefano Fiorucci
35e69369dc
feat: add ReasoningContent to ChatMessage (#9696)
* feat: add ReasoningContent to ChatMessage

* more tests

* release note

* Update haystack/dataclasses/chat_message.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2025-08-11 10:01:31 +02:00
Stefano Fiorucci
441b487147
fix: ChatMessage bugfixes (#9682)
* fix: ChatMessage bugfixes

* fix + release note

* better condition
2025-08-06 09:29:05 +02:00
Stefano Fiorucci
f8a71ecf39
feat: add serde methods to ImageContent and TextContent (#9679)
* ChatMessage - minor serde refactoring

* simplifications + more tests

* more tests + release notes
2025-08-05 11:17:08 +02:00
Stefano Fiorucci
1d96e6e4af
fix: ChatMessage.from_user - raise error if text and content_parts are None; pin ddtrace (#9657)
* fix: allow empty text in ChatMessage.from_user

* pin ddtrace<3.11.0
2025-07-29 12:39:27 +02:00
Srishti Gureja
e5810005ed
feat: add to_dict and from_dict to StreamingChunk (#9608)
* add to_dict and from_dict to StreamingChunk

* bugfix: set index field

* release notes + minor bugfix in tests

* ensure full serialization + update tests

* support de/serialization for all involved dataclasses

* remove unnecessary import

* code cleanup + only allow dicts in from_dict

* update release notes

* add more tests
2025-07-22 08:40:58 +00:00
Stefano Fiorucci
4d75ff42b4
feat: add ImageContent class methods (#9632) 2025-07-22 09:46:47 +02:00
Kane Norman
9420492798
feat: allow mime_type to be guessed for ByteStream (#9573)
* feat(bytestream): add guess_mime_type parameter

* refactor(FileTypeRouter): refactor guess mimetype

* feat(bytestream): add guess_mime_type to util

* style(ruff): add trailing whitespace

* fix: fix type annotation

* test(file_type_router): add test for additional_mimetypes param

* fix(file_type_router): non-existent file behavior

* feat(file_type_router): add release notes

* fix(file_type_router): remove unused logger

* style: fix ruff formatting magic values

* test(bytestream): handle windows/unix mimetype differences

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2025-07-22 07:43:22 +00:00
Stefano Fiorucci
6a591bd027
feat: add ImageContent dataclass to include images in ChatMessage + OpenAI support (#9626) 2025-07-21 14:39:31 +02:00
Sebastian Husch Lee
85258f0654
fix: Fix types and formatting pipeline test_run.py (#9575)
* Fix types in test_run.py

* Get test_run.py to pass fmt-check

* Add test_run to mypy checks

* Update test folder to pass ruff linting

* Fix merge

* Fix HF tests

* Fix hf test

* Try to fix tests

* Another attempt

* minor fix

* fix SentenceTransformersDiversityRanker

* skip integrations tests due to model unavailable on HF inference

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-07-03 09:49:09 +02:00
Sebastian Husch Lee
16fc41cd95
feat: Relax requirement for creating a ToolCallDelta dataclass (#9582)
* Relax our requirement for ToolCallDelta to better match ChoiceDeltaToolCall and ChoiceDeltaToolCallFunction from OpenAI

* Add reno

* Update tests
2025-07-03 08:50:29 +02:00
Amna Mubashar
adb2759d00
chore: remove deprecated State from haystack.dataclasses (#9578)
* Remove deprecated class

* Remove state from pydocs
2025-07-02 12:19:06 +02:00
Sebastian Husch Lee
97e72b9693
feat: Add to_dict and from_dict to ByteStream (#9568)
* Add to_dict and from_dict to ByteStream

* Add reno

* Add unit tests

* Fix and expand tests

* Fix typing

* PR comments
2025-06-30 11:57:22 +00:00
Vladimir Blagojevic
91094e1038
feat: Add finish_reason field to StreamingChunk (#9536)
* Initial commit

* Update deprecation version

* Improve comment

* Minor simplification

* Add reno note

* Remove deprecation warning

* Remove fallback in haystack/components/generators/utils.py

* FinishReason alphabetical import

* Add tool_call_results finish reason, adapt codebase

* Define finish_reason to be Optional[FinishReason]

* Add StreamingChunk finish_reason in HF generators

* Update reno note

* Repair merge issue

* Update tests for finish_reason

* Resolve mypy issues

* Lint issue

* Enhance HF finish_reason translation

* Remove irrlevant test

* PR comments

---------

Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
2025-06-25 09:06:01 +00:00
Sebastian Husch Lee
ec371387f0
refactor: Update to StreamingChunk, better index setting and change tool_call to tool_calls (#9525)
* Fixes to setting StreamingChunk.index properly and refactoring tests for conversion

* Make _convert_chat_completion_chunk_to_streaming_chunk a member of OpenAIChatGenerator so we can overwrite it in integrations that inherit from it

* Fixes

* Modify streaming chunk to accept a list of tool call deltas.

* Fix tests

* Fix mypy and update original reno

* Undo change

* Update conversion to return a single streaming chunk

* update to print streaming chunk

* Fix types

* PR comments
2025-06-23 08:14:25 +00:00
Amna Mubashar
1d6a9f652a
fix: serialization of nested ChatMessage in GeneratedAnswerdataclass (#9497)
* Fix serialization

* small fix

* fix the erros

* Fix tests

* PR comments
2025-06-06 11:46:24 +02:00
Sebastian Husch Lee
b61886b138
feat: Update streaming chunk (#9424)
* Start expanding StreamingChunk

* First pass at expanding Streaming Chunk

* Working version!

* Some tweaks and also make ToolInvoker stream a chunk with a finish reason

* Properly update test

* Change to tool_name, remove kw_only since its python 3.10 only and update HuggingFaceAPIChatGenerator to start following new StreamingChunk

* Add reno

* Some cleanup

* Fix unit tests

* Fix mypy and integration test

* Fix pylint

* Start refactoring huggingface local api

* Refactor openai generator and chat generator to reuse util methods

* Did some reorg

* Reusue utility method in HuggingFaceAPI

* Get rid of unneeded default values in tests

* Update conversion of streaming chunks to chat message to not rely on openai dataclass anymore

* Fix tests and loosen check in StreamingChunk post_init

* Fixes

* Fix license header

* Add start and index to HFAPIGenerator

* Fix mypy

* Clean up

* Update haystack/components/generators/utils.py

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update haystack/components/generators/utils.py

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Change StreamingChunk.start to only a bool

* PR comments

* Fix unit test

* PR comment

* Fix test

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2025-06-06 08:17:02 +00:00
Vladimir Blagojevic
853a32f8da
feat: Improve ChatMessage _deserialize_content ValueError - make it more LLM friendly (#9484)
* Improve ChatMessage _deserialize_content ValueError - make it more LLM friendly

* Add unit test

* Add reno note

* Add descriptive ValueError for missing role

* Update haystack/dataclasses/chat_message.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update releasenotes/notes/improve-chatmessage-error-messages-llm-agents-a1b2c3d4e5f6g7h8.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Add role check in ChatMessage

* fixes + refinements

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-06-04 15:14:05 +00:00
Stefano Fiorucci
1e2214a1a0
feat: ChatMessage.to_openai_dict_format - add require_tool_call_ids parameter (#9481) 2025-06-03 16:55:13 +02:00
Amna Mubashar
64def6d41b
feat: add component name and type to StreamingChunk (#9426)
* Stream component name in openai

* Fix type

* PR comments

* Update huggingface gen

* Typing fix

* Update huggingfacelocal gen

* Fix errors

* Remove model changes

* Fix minor errors

* Update releasenotes/notes/add-component-info-dataclass-be115dee2fa50abd.yaml

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* PR comments

* update annotation

* Update hf files

* Fix linting

* Add a from_component method

* use add_component

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2025-05-27 12:23:40 +02:00
David S. Batista
da60156174
chore: removing unused imports from tests (#9446) 2025-05-26 16:22:51 +00:00
Amna Mubashar
720cc19d7d
feat: add serialization to State / move State to agents.state (#9345)
* Add serialization to State

* Add release notes

* Deprecate State in dataclasses

* Fix tests

* Remove state_utils test

* Fix linting

* Fix formating

* Update tests and remove old state utils

* Update agents test

* Update deserilaization per review

* Linting

* Add tests for edge case (custom class types)

* Fix type serialization

* PR comments

* Move State to agents

* Fix tests

* Update utils init

* Improve seriliaztion/deser

* Update the release notes

* Minor fix in docstrings

* PR comments

* Add deprecation warnign for state utils

* Recreate the serialization methods to use schema

* Update key names

* Make serialization methods private
2025-05-23 11:04:15 +02:00
Stefano Fiorucci
38c39a49de
test: review integration tests (#9306)
* AzureOCR: convert integration test to unit test and simplify

* clean up HuggingFaceAPITextEmbedder

* clean up LinkContentFetcher

* simplify HuggingFaceLocalGenerator

* clean up OpenAIGenerator

* OpenAIChatGenerator

* SentenceTransformersDiversityRanker

* TransformersSimilarityRanker

* ChatMessage: rm outdated tests

* fail fast false

* typo
2025-04-25 09:07:57 +02:00
Stefano Fiorucci
c67d1bf0e9
fix: make ChatMessage.from_dict handle cases where optional fields are missing (#9232)
* fix: make ChatMessage.from_dict handle cases where optional fields are missing

* one more test
2025-04-14 14:53:08 +02:00
Mohammed Abdul Razak Wahab
859e90cc61
fix: Document field precedence in to_dict() (#9227)
* fix: Document field precedence in to_dict()

* add test

* add release note
2025-04-14 13:53:12 +02:00
Sebastian Husch Lee
7bb9c6934b
fix: Set messages in state_schema at init time of Agent (#9195)
* Fix issue and set messages in state_schema at init time

* Add reno

* Small changes and add more tests for state

* Add comment

* PR comments
2025-04-10 06:39:17 +00:00
Sebastian Husch Lee
bde2d77df0
feat: Move storing of messages into State in Agent (#9150)
* Update messages to be stored in State so users can control how they are stored through a handler in the schema

* Fix test

* Add test

* Add reno

* Fix docstring
2025-04-01 11:29:44 +02:00
Julian Risch
657d09d7f1
feat: integrate updates of Tool, ToolInvoker, State, create_tool_from_function, ComponentTool from haystack-experimental (#9113)
* update Tool,ToolInvoker,ComponentTool,create_tool_from_function

* add State and its utils

* add tests for State and its utils

* update tests for Tool etc.

* reno

* fix circular imports

* update experimental imports in tests

* fix unit tests

* fix ChatGenerator unit tests

* mypy

* add State to init and pydoc

* explain State in more detail in release note

* add test from #8913

* re-add _check_duplicate_tool_names and refactor imports

* rename inputs and outputs
2025-03-28 10:49:23 +01:00
Stefano Fiorucci
aa82adf9a2
refactor!: ChatMessage serialization-deserialization updates (#9069)
* chatmessage serde updates

* improvements and relnotes

* improve relnotes

* simplification

* warning proposal
2025-03-21 11:59:26 +01:00
Stefano Fiorucci
4c1facdfab
fix: add dataframe to legacy fields for Document (#9026)
* fix: add dataframe to legacy fields for Document

* fmt

* small fixes
2025-03-12 13:01:03 +01:00
Mohammed Abdul Razak Wahab
e33a9e46ed
fix: add chat message name field (#8969)
* add-chat-message-name-field

* add release notes

* Update add-chat-message-name-field-a8ae96fb9ff13f7b.yaml

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-03-05 09:29:36 +00:00
Stefano Fiorucci
f3c44be904
refactor!: remove dataframe field from Document and ExtractedTableAnswer; make pandas optional (#8906)
* remove dataframe

* release note

* small fix

* group imports

* Update pyproject.toml

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update pyproject.toml

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* address feedback

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2025-03-04 11:06:07 +00:00
Stefano Fiorucci
62ac27c947
chore: remove deprecated function ChatRole and from_function class method in ChatMessage (#8725)
* rm deprecated function role and from_function class method in chatmessage

* release note
2025-01-15 18:55:22 +01:00
Stefano Fiorucci
3f15f38c51
refactor: move Tool to a separate package; refactor serde (#8690)
* move tool to separate package; refactor serde

* release note

* rm unused import
2025-01-09 12:30:13 +01:00
tstadel
e6059e632e
fix: truncate ByteStream string representation (#8673)
* fix: truncate ByteStream string representation

* add reno

* better reno

* add test

* Update test_byte_stream.py

* apply feedback

* update reno
2025-01-07 19:00:52 +01:00
Stefano Fiorucci
7b4d9ba86e
feat: introduce class method to create ChatMessage from the OpenAI dictionary format (#8670)
* add ChatMessage.from_openai_dict_format

* remove print

* release note

* improve docstring

* separate validation logic

* rm obvious comment
2025-01-02 10:34:41 +00:00
Stefano Fiorucci
188b2a7f06
feat: support for tools in OpenAIChatGenerator (#8666)
* move chatmsg>openai conversion to chatmsg dataclass

* implementation and tests cleanup

* release note

* try fixing azure chat generator

* add serde test for toolinvoker

* small fix
2024-12-20 14:20:54 +00:00
Stefano Fiorucci
2bc58d2987
feat: support for tools in HuggingFaceAPIChatGenerator (#8661)
* message conversion function

* hfapi w tools

* right test file + hf_hub version

* release note

* feedback
2024-12-19 15:04:37 +01:00
Stefano Fiorucci
96b4a1d2fd
feat: Tool dataclass - unified abstraction to represent tools (#8652)
* draft

* del HF token in tests

* adaptations

* progress

* fix type

* import sorting

* more control on deserialization

* release note

* improvements

* support name field

* fix chatpromptbuilder test

* port Tool from experimental

* release note

* docs upd

* Update tool.py

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2024-12-18 11:36:44 +00:00
Stefano Fiorucci
ea3602643a
feat!: new ChatMessage (#8640)
* draft

* del HF token in tests

* adaptations

* progress

* fix type

* import sorting

* more control on deserialization

* release note

* improvements

* support name field

* fix chatpromptbuilder test

* Update chat_message.py

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2024-12-17 17:02:04 +01:00
Stefano Fiorucci
c8685aa141
refactor: update components to access ChatMessage.text instead of content (#8589)
* introduce text property and deprecate content

* release note

* use chatmessage.text

* release note

* linting
2024-11-28 10:16:07 +00:00
Stefano Fiorucci
fb1baf4921
refactor: ChatMessage - introduce text property and deprecate content (#8588)
* introduce text property and deprecate content

* release note

* minor test refactoring

---------

Co-authored-by: Michele Pangrazzi <xmikex83@gmail.com>
2024-11-28 09:53:02 +00:00
Stefano Fiorucci
51c1390426
chore: use class methods to create ChatMessage (#8581)
* use class methods to build messages

* fix failing format
2024-11-28 09:35:24 +00:00
Madeesh Kannan
cf5fd2a821
chore: Remove deprecated ChatMessage.to_openai_format (#8242)
* chore: Remove deprecated `ChatMessage.to_openai_format`

* lint
2024-08-16 10:34:44 +02:00
Carlos Fernández
57c1d47c7d
fix: ChatPromptBuilder Fails to JSON Serialize (#7849)
* implement serialization for chat messages and add tests

* implement serialization for ChatPromptBuilder and test it

* add reno

* solve mypy type error

* solve mypy type error

* remove flattening parameter in to_dict

* simplify to jus non-flat metadata

* try to fix linting issue

* solve format issues

* update test for ChatPromptBuilder

* remove unused import

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2024-06-20 13:20:52 +02:00
Massimiliano Pippi
3a03fce71c
ci: Add code formatting checks (#7882)
* ruff settings

enable ruff format and re-format outdated files

feat: `EvaluationRunResult` add parameter to specify columns to keep in the comparative `Dataframe`  (#7879)

* adding param to explictily state which cols to keep

* adding param to explictily state which cols to keep

* adding param to explictily state which cols to keep

* updating tests

* adding release notes

* Update haystack/evaluation/eval_run_result.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update releasenotes/notes/add-keep-columns-to-EvalRunResult-comparative-be3e15ce45de3e0b.yaml

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* updating docstring

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

add format-check

fail on format and linting failures

fix string formatting

reformat long lines

fix tests

fix typing

linter

pull from main

* reformat

* lint -> check

* lint -> check
2024-06-18 15:52:46 +00:00
Massimiliano Pippi
10c675d534
chore: add license header to all modules (#7675)
* add license header to modules
* check license header at linting time
2024-05-09 13:40:36 +00:00
Stefano Fiorucci
19a46af9da
add __eq__ method to SparseEmbedding (#7574)
* add __eq__ method to SparseEmbedding

* reno

* improve reno
2024-04-23 19:03:41 +02:00