4252 Commits

Author SHA1 Message Date
Daria Fokina
f8d3a82997
Your commit message here (#9692) 2025-08-11 14:41:16 +02:00
Sebastian Husch Lee
af9aac2b99
chore!: Update finish reason in output of HuggingFaceAPIChatGenerator to match between stream and non-stream modes (#9686)
* Update finish reason

* Fix unit test

* Add reno

* Update releasenotes/notes/update-finish-reason-hf-api-chat-gen-c700042a079733e8.yaml

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Update async as well

* Fix unit test

---------

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
2025-08-11 13:52:16 +02:00
Abdelrahman Kaseb
03d9f0fd74
fix: prevent in-place mutation of documents in Document Embedders (#9693)
* fix: prevent in-place mutation of documents after embeddings by using deepcopy

* Add tests

* use from dataclasses import replace instead of deepcopy

* Address PR comments
2025-08-11 12:21:09 +02:00
Stefano Fiorucci
35e69369dc
feat: add ReasoningContent to ChatMessage (#9696)
* feat: add ReasoningContent to ChatMessage

* more tests

* release note

* Update haystack/dataclasses/chat_message.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2025-08-11 10:01:31 +02:00
Amna Mubashar
683c935b38
feat: Update MetadataRouter to support ByteStream (#9688)
* Start changes for updating metadata router

* Update the router

* releasenotes/notes/add-bytestream-support-metadata-router-1ee5149745894f42.yaml

* Add release notes

* Fix error

* Update typing

* Update the typing

* PR comments

* Update param name

* Update type name

* Fix typo

* Remove type var

* Update typing

* Update typing

* Add overloads for all filter methods

* Use type ignore

* Remove unused imports

---------

Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
2025-08-08 16:21:27 +02:00
Stefano Fiorucci
95f00210e8
chore: OpenAI - explictly filter function tools (#9697) 2025-08-08 11:37:11 +02:00
Stefano Fiorucci
47508bc1e6
fix: fix OpenAI tests for openai==1.99.3 (#9694)
* fix: fix OpenAI tests for openai==1.99.3

* fix async tests
2025-08-08 06:40:57 +00:00
Stefano Fiorucci
c05d3f0051
chore: remove unused type:ignore and cast (#9690)
* chore: remove unused type:ignore and casts

* rm unused imports
2025-08-07 15:41:00 +02:00
Abdelrahman Kaseb
5f3c37d287
chore: adopt PEP 585 type hints (#9678)
* chore(lint): enforce and apply PEP 585 type hinting

* Run fmt fixes

* Fix all typing imports using some regex

* Fix all typing written in string in tests

* undo changes in the e2e tests

* make e2e test use list instead of List

* type fixes

* remove type:ignore

* pylint

* Remove typing from Usage example comments

* Remove typing from most of comments

* try to fix e2e tests on comm PRs

* fix

* Add tests typing.List in to adjust test compatiplity
- test/components/agents/test_state_class.py
- test/components/converters/test_output_adapter.py
- test/components/joiners/test_list_joiner.py

* simplify pyproject

* improve relnote

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-08-07 10:23:14 +02:00
Chinmay Bansal
4b9fb20bab
feat: add image support to HuggingFaceAPIChatGenerator (#9680)
* feat(huggingface-api): #9671 add image support to HuggingFaceAPIChatGenerator

* docs: add release notes for image support in HuggingFaceAPIChatGenerator

* Fixed comments on PR: implementation, testing, default value for validation

* refinements

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-08-06 16:35:32 +02:00
Stefano Fiorucci
441b487147
fix: ChatMessage bugfixes (#9682)
* fix: ChatMessage bugfixes

* fix + release note

* better condition
2025-08-06 09:29:05 +02:00
Bilge Yücel
323274e174
Add Haystack Enterprise section (#9674)
* Update README.md

* add a full stop

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Change the enterprise link

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2025-08-05 15:20:22 +02:00
Stefano Fiorucci
f8a71ecf39
feat: add serde methods to ImageContent and TextContent (#9679)
* ChatMessage - minor serde refactoring

* simplifications + more tests

* more tests + release notes
2025-08-05 11:17:08 +02:00
Abdelrahman Kaseb
d0de78ec0a
fix: ensure sentence_transformers_similarity score is a float to not np.float (#9665)
* fix: ensure sentence_transformers_similarity score is a float to prevent serialization issues

* solve PR comments
2025-08-04 11:28:05 +02:00
Daria Fokina
f2012a4521
docs-fix-syntax (#9670) 2025-07-31 16:26:00 +00:00
Sara Calla
61866ba9aa
chore: make the lazy import error message clearer (#9667)
* made the lazy import error message clearer

* add header
2025-07-31 16:10:14 +02:00
Stefano Fiorucci
ed48e9e965
test: fix Datadog tests and unpin ddtrace test dependency (#9659)
* test: fix Datadog tests and unpin ddtrace test dependency

* we need less env vars
2025-07-30 17:20:15 +02:00
Stefano Fiorucci
834a624cdc
chore: re-export missing symbols in __init__.py (#9662) 2025-07-30 10:06:38 +02:00
Sebastian Husch Lee
8efbfc0da6
Update documentation codeowners (#9658) 2025-07-29 17:06:04 +02:00
Stefano Fiorucci
1d96e6e4af
fix: ChatMessage.from_user - raise error if text and content_parts are None; pin ddtrace (#9657)
* fix: allow empty text in ChatMessage.from_user

* pin ddtrace<3.11.0
2025-07-29 12:39:27 +02:00
Sebastian Husch Lee
09476094fa
feat: Add support for the union operator | added in python 3.10 (#9650)
* Add support for | operator

* Add reno

* Use type(None) instead of NoneType

* Make custom UnionType private

* Add check that test should only run if python is 3.10+

* PR comments

* Update test/utils/test_type_serialization.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Fix formatting

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-07-28 08:38:53 +00:00
Luca Rolshoven
f72ab7f63f
fix(embeddings): add encoding_format keyword argument when calling OpenAI's client.embeddings.create (#9655)
* fix(embeddings): add `encoding_format` keyword argument when calling OpenAI's `client.embeddings.create`.

* fix mypy

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-07-28 08:26:39 +00:00
Haystack Bot
5c42256059
Update unstable version to 2.17.0-rc0 (#9653)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-07-24 13:06:40 +02:00
Stefano Fiorucci
23237f76d2
docs: fix DocumentLengthRouter usage example (#9652)
Co-authored-by: David S. Batista <dsbatista@gmail.com>
v2.17.0-rc0
2025-07-24 10:45:05 +00:00
David S. Batista
8af4cf8b01
chore: removing Pipeline.draw() deprecation warnings (#9651)
* cleaning up tests

* adding release notes
2025-07-24 12:35:19 +02:00
Stefano Fiorucci
d059cf2c23
feat: add skip_empty_documents init parameter to DocumentSplitter (#9649)
* feat: add skip_empty_documents init parameter to DocumentSplitter

* improve test

* fix + relnote
2025-07-24 11:26:11 +02:00
David S. Batista
3b9b1ae802
feat: adding debugging breakpoints to Pipeline and Agent (#9611)
* wip: fixing tests

* wip: fixing tests

* wip: fixing tests

* wip: fixing tests

* fixing circular imports

* decoupling resume and initial run() for agent

* adding release notes

* re-raising BreakPointException from pipeline.run()

* fixing imports

* refactor: Refactor suggestions for Pipeline breakpoints (#9614)

* Refactoring

* Start adding debug_path into Breakpoint class

* Fully move debug_path into Breakpoint dataclass

* Simplifications in pipeline run logic

* More simplification

* lint

* More simplification

* Updates

* Rename resume_state to pipeline_snapshot

* PR comments

* Missed renaming of state in a few more places

* feat: Add dataclasses to represent a `PipelineSnapshot` and refactored to use it (#9619)

* Refactor to use dataclasses for PipelineSnapshot and AgentSnapshot

* Fix integration tests

* Mypy

* Fix mypy

* Fix lint

* Refactor AgentSnapshot to only contain needed info

* Fix mypy

* More refactoring

* removing unused import

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* feat: saving include_outputs_from intermediate results to `PipelineState` object (#9629)

* saving intermediate components results in include_outputs_from into the PipelineSnaptshot

* cleaning up

* fixing tests

* fixing tests

* extending tests

* Update haystack/dataclasses/breakpoints.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* Update haystack/dataclasses/breakpoints.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* linting

* moving intermediate results to pipeline state and adding pipeline outputs to state

* moving ordered_component_names and include_outputs_from to PipelineSnapshot

* moving original_input_data to PipelineSnapshot

* simplifying saving the intermediate results

* Update haystack/dataclasses/breakpoints.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* Update haystack/dataclasses/breakpoints.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* Update haystack/dataclasses/breakpoints.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* Update haystack/dataclasses/breakpoints.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* linting

* cleaning up

* avoiding creating PipelineSnapshot for every component run

* removing unecessary code

* Update checks in Agent to not unecessarily create AgentSnapshot when not needed.

* Update haystack/components/agents/agent.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* Update haystack/components/agents/agent.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* cleaning up tests

* linting

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
2025-07-24 08:54:23 +00:00
Sebastian Husch Lee
3f5c894afa
Fix example in DocumentToImageContent (#9648) 2025-07-24 08:45:34 +00:00
dependabot[bot]
90ec214589
chore(deps): bump pilosus/action-pip-license-checker from 2 to 3 (#9644)
* chore(deps): bump pilosus/action-pip-license-checker from 2 to 3

Bumps [pilosus/action-pip-license-checker](https://github.com/pilosus/action-pip-license-checker) from 2 to 3.
- [Release notes](https://github.com/pilosus/action-pip-license-checker/releases)
- [Changelog](https://github.com/pilosus/action-pip-license-checker/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pilosus/action-pip-license-checker/compare/v2...v3)

---
updated-dependencies:
- dependency-name: pilosus/action-pip-license-checker
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* trigger check

* try

* fix

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-07-23 16:36:31 +02:00
Stefano Fiorucci
33e8bd5ef6
chore: update SentenceTransformersEmbeddingBackend.embed type hint to include images (#9643)
* chore: update SentenceTransformersEmbeddingBackend type hint to include images

* fix test

* linting

* simplify
2025-07-23 15:57:43 +02:00
Amna Mubashar
8e792a3d12
fix: update _convert_streaming_chunks_to_chat_message to handle tool calls with empty arguments (#9639)
* Update util function

* Add a new test

* PR comments
2025-07-23 13:28:05 +02:00
JohnKagunda
59403de1f0
feat: added return_embedding attr in in_memory/document_store (#9622)
* feat: added  to init

* feat: added return_embedding in to_dict

* feat: added  return_embedding to filter_documents

* feat: added return_embedding to  bm25_retrieval

* refactor: embedding_retrieval to use return_embedding attribute rather than parameter passed

* docs: added releasenote

* fix: pop from doc_fields instead of changing return_documents attr to none

* fix: made return_embedding an optional field and removed deprecation warning

* fix: give return_embedding a higher priority than self.return_embedding

* feat: changed default behaviour of return_embedding to True

* chore: update tests after InMemory Document store update

* Update releasenotes/notes/update-in-memory-document-store-17f555695caf9d52.yaml

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* chore: update docs

* chore: enhanced clarity and redability of expression

* test: return_embedding is set to false during initialization

* test: overriding  return_embedding inside

* fix: changed the use of self.filter_documents to actual implementation inside `embedding_retrieval`

Signed-off-by: rafaeljohn9 <rafaeljohb@gmail.com>

---------

Signed-off-by: rafaeljohn9 <rafaeljohb@gmail.com>
Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2025-07-23 10:48:14 +00:00
Sebastian Husch Lee
b9b1652fd4
feat: Add LLMDocumentContentExtractor (#9637)
* Add LLMDocumentContentExtractor

* Remove file

* Remove from slow
2025-07-23 11:16:37 +02:00
Stefano Fiorucci
c9e43c9ca2
feat: add DocumentLengthRouter (#9636) 2025-07-22 14:59:28 +02:00
Stefano Fiorucci
868ea41698
feat: add SentenceTransformersDocumentImageEmbedder (#9635) 2025-07-22 13:10:33 +02:00
Sebastian Husch Lee
f801171191
feat: Add DocumentTypeRouter (#9634)
* Add DocumentTypeRouter

* PR comments

* Turn off isort for one line so pylint will pass
2025-07-22 11:08:24 +00:00
Amna Mubashar
b3971ff574
fix: update the deserialization for tool decorator (#9618)
* Fix linting

* Add tests

* Small fixes

* check for Tool instance

* Remove unnecessary update

* PR comments

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-07-22 12:35:29 +02:00
Srishti Gureja
e5810005ed
feat: add to_dict and from_dict to StreamingChunk (#9608)
* add to_dict and from_dict to StreamingChunk

* bugfix: set index field

* release notes + minor bugfix in tests

* ensure full serialization + update tests

* support de/serialization for all involved dataclasses

* remove unnecessary import

* code cleanup + only allow dicts in from_dict

* update release notes

* add more tests
2025-07-22 08:40:58 +00:00
Stefano Fiorucci
976cb868d6
chore: HuggingFaceAPIChatGenerator - check if ChatCompletionOutput.choices is None (#9633) 2025-07-22 10:33:54 +02:00
Stefano Fiorucci
4d75ff42b4
feat: add ImageContent class methods (#9632) 2025-07-22 09:46:47 +02:00
Kane Norman
9420492798
feat: allow mime_type to be guessed for ByteStream (#9573)
* feat(bytestream): add guess_mime_type parameter

* refactor(FileTypeRouter): refactor guess mimetype

* feat(bytestream): add guess_mime_type to util

* style(ruff): add trailing whitespace

* fix: fix type annotation

* test(file_type_router): add test for additional_mimetypes param

* fix(file_type_router): non-existent file behavior

* feat(file_type_router): add release notes

* fix(file_type_router): remove unused logger

* style: fix ruff formatting magic values

* test(bytestream): handle windows/unix mimetype differences

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2025-07-22 07:43:22 +00:00
Stefano Fiorucci
b9fa70610f
feat: extend ChatPromptBuilder to support string templates (#9631) 2025-07-22 09:36:51 +02:00
Sebastian Husch Lee
7414ef6823
feat: Add image converters (#9628)
* Add image converters

* Fix tests

* Update haystack/components/converters/image/__init__.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-07-21 15:46:14 +00:00
Stefano Fiorucci
6a591bd027
feat: add ImageContent dataclass to include images in ChatMessage + OpenAI support (#9626) 2025-07-21 14:39:31 +02:00
Amna Mubashar
f5c0c1a9ca
Update README (#9621) 2025-07-18 11:03:45 +02:00
Krishna Somani
8ae24ca268
upgraded README.md (#9613)
* upgraded README.md

Added table of contents and made minor grammatical fixes with enhancing the readability of the file

* update README.md

Removal of capitalization

* Update README.md
2025-07-16 10:47:10 +02:00
Sebastian Husch Lee
7a63559faf
feat: Raise warning if the pipeline is unable to continue running due to a blocked component (#9569)
* First pass at alerting a user that the pipeline is blocked

* Change approach and add change to async pipeline

* Fix check in run_async

* Another fix

* Somehow also fixed the max_runs_per_component

* Align sync run and async run component

* Update output type of component outputs to Mapping to align with our protocol

* linting

* ruff

* Make it a core test

* Add reno

* Some refactoring

* Linting

* Fix mypy

* Converting to warning

* Small changes

* Update release note

* More cleanup

* PR comment

* PR comments
2025-07-15 14:02:39 +00:00
Sebastian Husch Lee
393a1bd293
feat: Update sentence window retriever to add source_id_meta_field, split_id_meta_field , and raise_on_missing_meta_fields (#9610)
* Update sentence window retriever

* Improvements and more tests

* Add new variable raise_on_failure

* Update reno
2025-07-14 13:18:05 +02:00
Sebastian Husch Lee
34f871d6e3
feat: Explicitly test support for builtin python 3.9+ types (#9606)
* Updating tests

* More tests

* More tests

* More tests

* More tests

* Get bare types to work with builtins

* Add reno

* PR comments

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-07-14 09:14:44 +00:00
Bilge Yücel
7660c80efa
docs: Fix the deepset platform logo (#9609) 2025-07-14 08:56:06 +02:00