89 Commits

Author SHA1 Message Date
Amna Mubashar
28db039bca
feat: add run_async to HuggingfaceAPIChatGenerator (#8943)
* add run_async

* add release notes

* Add integration test
2025-03-03 16:51:30 +01:00
tstadel
13968cc15b
fix: in OpenAIChatGenerator set additionalProperties to False when tools_strict=True (#8913)
* fix: set ComponentTool addtionalProperties for OpenAI tools_strict=True

* add reno

* Move the additionalProperties into the OpenAIChatGenerator

* Remove

* Put additionalProperties into the correct place

* Fix test

* Update releasenotes/notes/fix-componenttool-for-openai-tools_strict-998e5cd7ebc6ec19.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Sebastian Husch Lee <sebastian.lee@deepset.ai>
Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-03-03 16:23:24 +01:00
Sebastian Husch Lee
296e31c182
feat: Add Type Validation parameter for Pipeline Connections (#8875)
* Starting to refactor type util tests to be more systematic

* refactoring

* Expand tests

* Update to type utils

* Add missing subclass check

* Expand and refactor tests, introduce type_validation Literal

* More test refactoring

* Test refactoring, adding type validation variable to pipeline base

* Update relaxed version of type checking to pass all newly added tests

* trim whitespace

* Add tests

* cleanup

* Updates docstrings

* Add reno

* docs

* Fix mypy and add docstrings

* Changes based on advice from Tobi

* Remove unused imports

* Doc strings

* Add connection type validation to to_dict and from_dict

* Update tests

* Fix test

* Also save connection_type_validation at global pipeline level

* Fix tests

* Remove connection type validation from the connect level, only keep at pipeline level

* Formatting

* Fix tests

* formatting
2025-03-03 16:00:22 +01:00
Sebastian Husch Lee
00fe4d157d
feat: Add run async for AzureOpenAIChatGenerator (#8948)
* Add tests for run_async

* Add reno

* Add async client

* Add init test

* Add comment

* Fix test

* Update releasenotes/notes/run-async-azure-54450f0c2495f5c8.yaml

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

---------

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
2025-03-03 14:17:18 +00:00
Michele Pangrazzi
209e6d5ff0
remove duplicate test (#8944) 2025-02-28 13:27:43 +00:00
Michele Pangrazzi
d1e503e5c7
skip HF API integration test (#8938) 2025-02-26 12:10:54 +01:00
Julian Risch
6652dd7550
Revert "test: skip HF API live integration tests (#8889)" (#8914)
* Revert "test: skip HF API live integration tests (#8889)"

This reverts commit 56a3a9bd61b7391ae91e3d8179b3b33918ef4932.

* Replace zephyr-7b-beta model with SmolLM2-1.7B-Instruct

* Use zephyr-7b-beta model but extend instructions

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-02-25 09:03:20 +01:00
Michele Pangrazzi
44fb20c2d5
Add run_async to OpenAIChatGenerator (#8880)
* Implememntation of run_async (wip)

* Add missing tests ; Move async tests to test_openai_async.py

* Add release note

* Update docstring

* Alignments with haystack-experimental implementation

* Lint: removed unused imports

* Update haystack/components/generators/chat/openai.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-02-20 16:51:46 +00:00
Stefano Fiorucci
56a3a9bd61
test: skip HF API live integration tests (#8889)
* skip HF API integration tests

* better wording
2025-02-20 16:38:57 +00:00
Sebastian Husch Lee
52909a0c81
fix: Fix OpenAIChatGenerator + tools + streaming (#8879)
* Fix chat generator + tools + streaming

* Add reno

* Update docs

* Remove unused import

* add doc

* Fix test

* small cleanup

* PR comments

* fix test

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-02-20 08:40:22 +01:00
mathislucka
8c54f06a19
fix: component checks failing for components that return dataframes (#8873)
* fix: use is not to compare to sentinel value

* chore: release notes

* Update releasenotes/notes/fix-component-checks-with-ambiguous-truth-values-949c447b3702e427.yaml

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* fix: another sentinel value

* test: also test base class

* add pandas as test dependency

* format

* Trigger CI

* mark test with xfail strict=False

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-02-19 09:10:48 +00:00
Sebastian Husch Lee
0c62087dd7
Make openai test more robust (#8872) 2025-02-18 11:38:16 +01:00
Vladimir Blagojevic
a7c1661f13
fix: Look through all streaming chunks for tools calls (#8829)
* Look through all streaming chunks for tools calls

* Add reno note

* mypy fixes

* Improve robustness

* Don't concatenate, use the last value

* typing

* Update releasenotes/notes/improve-tool-call-chunk-search-986474e814af17a7.yaml

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* Small refactoring

* isort

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-02-11 13:25:39 +01:00
Vladimir Blagojevic
73bfc08b71
feat: HuggingFaceLocalChatGenerator unified support for tools (#8827)
* Add tools to HuggingFaceLocalChatGenerator

* Add reno

* Fix types

* Small post merge fix

* Add unit tests

* Add tools serde and tests

* PR feedback

* PR feedback
2025-02-10 09:44:51 +01:00
Stefano Fiorucci
1f257944a6
chore: fix Hugging Face components for mypy 1.15.0 (#8822)
* chore: fix Hugging Face components for mypy 1.15.0

* small fixes

* fix test

* rm print

* use cast and be more permissive
2025-02-06 16:25:59 +00:00
Stefano Fiorucci
5ae94886b2
fix: fix test failures with Transformers models in PRs from forks (#8809)
* trigger

* try pinning sentence transformers

* make integr tests run right away

* pin transformers instead

* older transformers version

* rm transformers pin

* try ignoring cache

* change ubuntu version

* try removing token

* try again

* more HF_API_TOKEN local deletions

* restore test priority

* rm leftover

* more deletions

* moreee

* more

* deletions

* restore jobs order
2025-02-04 19:08:37 +01:00
tstadel
bf79f04932
feat: support streaming_callback as run param for HF Chat generators (#8763)
* feat: support streaming_callback as run param for HF Chat generators

* add tests
2025-01-23 12:14:32 +01:00
Stefano Fiorucci
c3d0643511
feat: AzureOpenAIChatGenerator - support for tools (#8757)
* feat: AzureOpenAIChatGenerator - support for tools

* release note

* feedback
2025-01-23 09:24:04 +00:00
Stefano Fiorucci
f96839e139
chore: update transformers test dependency (#8752)
* update transformers test dependency

* add pad_token_id to the mock tokenizer

* fix HFLocal test + new test
2025-01-21 14:43:27 +01:00
Stefano Fiorucci
424bce2783
test: fix HF API flaky live test with tools (#8744)
* test: fix HF API flaky live test with tools

* rm print
2025-01-17 12:36:07 +00:00
Vladimir Blagojevic
21dd03d3e7
feat: Add completion start time timestamp to relevant generators (#8728)
* OpenAIChatGenerator - add completion_start_time

* HuggingFaceAPIChatGenerator - add completion_start_time

* Add tests

* Add reno note

* Relax condition for cached responses

* Add completion_start_time timestamping to non-chat generators

* Update haystack/components/generators/chat/hugging_face_api.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* PR feedback

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-01-17 09:58:45 +01:00
David S. Batista
425ce9b98f
test: updating HuggingFaceAPIChatGenerator tests 2025-01-14 16:47:29 +01:00
Stefano Fiorucci
741ce5df50
fix: OpenAIChatGenerator - do not pass tools to the OpenAI client when none are provided (#8702)
* do not pass tools to OpenAI client if None

* release note

* fix release note
2025-01-10 14:46:41 +01:00
Stefano Fiorucci
3f15f38c51
refactor: move Tool to a separate package; refactor serde (#8690)
* move tool to separate package; refactor serde

* release note

* rm unused import
2025-01-09 12:30:13 +01:00
Stefano Fiorucci
5539f6c33f
refactor: improve serialization/deserialization of callables (to handle class methods and static methods) (#8683)
* progress

* refinements

* tidy up

* release note
2025-01-08 11:28:00 +01:00
Stefano Fiorucci
188b2a7f06
feat: support for tools in OpenAIChatGenerator (#8666)
* move chatmsg>openai conversion to chatmsg dataclass

* implementation and tests cleanup

* release note

* try fixing azure chat generator

* add serde test for toolinvoker

* small fix
2024-12-20 14:20:54 +00:00
Stefano Fiorucci
f4d9c2bb91
fix: Make the HuggingFaceLocalChatGenerator compatible with the new ChatMessage; serialize chat_template (#8663)
* message conversion function

* hfapi w tools

* right test file + hf_hub version

* release note

* fix for new chatmessage; serialize chat_template

* feedback
2024-12-19 15:12:12 +01:00
Stefano Fiorucci
2bc58d2987
feat: support for tools in HuggingFaceAPIChatGenerator (#8661)
* message conversion function

* hfapi w tools

* right test file + hf_hub version

* release note

* feedback
2024-12-19 15:04:37 +01:00
Stefano Fiorucci
ea3602643a
feat!: new ChatMessage (#8640)
* draft

* del HF token in tests

* adaptations

* progress

* fix type

* import sorting

* more control on deserialization

* release note

* improvements

* support name field

* fix chatpromptbuilder test

* Update chat_message.py

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2024-12-17 17:02:04 +01:00
Stefano Fiorucci
f2b5f123b3
del HF token in tests (#8634) 2024-12-13 09:50:23 +01:00
Stefano Fiorucci
c8685aa141
refactor: update components to access ChatMessage.text instead of content (#8589)
* introduce text property and deprecate content

* release note

* use chatmessage.text

* release note

* linting
2024-11-28 10:16:07 +00:00
Silvano Cerza
3ef8c081be
fix: OpenAIChatGenerator and OpenAIGenerator crashing when streaming with usage tracking (#8558)
* Fix OpenAIGenerator crashing with tracking usage with streaming enabled

* Fix OpenAIChatGenerator crashing with tracking usage with streaming enabled

* Add release notes

* Fix linting
2024-11-20 10:27:22 +01:00
Sebastian Husch Lee
e45d3329a1
feat: Adding DALLE image generator (#8448)
* First pass at adding DALLE image generator

* Add missing header

* Fix tests

* Add tests

* Fix mypy

* Make mypy happy

* More unit tests

* Adding release notes

* Add a test for run

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* Fix pylint

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2024-11-14 16:19:49 +01:00
David S. Batista
e5a80722c2
feat: adding metadata grouper component (#8512)
* initial import

* making tests more readable; adding docstring

* adding release notes

* adding LICENSE header

* Update test/components/rankers/test_metadata_grouper.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* refactoring

* fixing docstring

* fixing types

* test docstrings

* renaming test

* handling too-many-arguments

* liting

* Update haystack/components/rankers/metadata_grouper.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* changing name

* Update haystack/components/rankers/metadata_grouper.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/rankers/metadata_grouper.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* assiging value inside function for re-use

* improving docstring

* updating name to MetaFieldGroupingRanker

* adding to pydocs

* fixing imports

* adding output docstring

* Update haystack/components/rankers/meta_field_grouper_ranker.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/rankers/__init__.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update releasenotes/notes/add-metadata-grouper-21ec05fd4a307425.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update test/components/rankers/test_metadata_grouper.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* update docstring tests

* fixing imports

* rename modules for consistency

* fix pydocs

* simplification + more tests

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
2024-11-12 16:01:53 +01:00
David S. Batista
852900d5e3
lowercase comparision (#8532) 2024-11-11 16:33:54 +01:00
Bohan Qu
2595e68050
feat: Add TTFT support in OpenAI chat generator (#8444)
* feat: Add TTFT support in OpenAI generators

* pylint fixes

* correct disable

---------

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
2024-10-31 16:56:17 +01:00
tstadel
d430833f8f
feat: streaming_callback as run param from HF generators (#8406)
* feat: streaming_callback as run param from HF generators

* apply feedback

* add reno

* fix test

* fix test

* fix mypy

* fix excessive linting rule
2024-10-29 15:32:06 +01:00
David S. Batista
3a50d35f06
feat: allow Generators to run with a system prompt defined at run time (#8423)
* initial import

* Update haystack/components/generators/openai.py

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>

* docs: fixing

* supporting the three use cases: no system prompt, using system prompt defined at init, using system prompt defined at run time

* renaming 'run_time_system_prompt' to 'system_prompt'

* adding tests, converting methods to static

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
2024-10-22 11:21:10 +02:00
Stefano Fiorucci
dfd339ca2d
ensure compatibility with huggingface_hub==0.26.0 (#8464) 2024-10-18 08:38:48 +00:00
Vladimir Blagojevic
09b95746a2
feat: HuggingFaceAPIChatGenerator add token usage data (#8375)
* Ensure HuggingFaceAPIChatGenerator has token usage data

* Add reno note

* Fix release note
2024-09-23 15:40:50 +02:00
David S. Batista
97126eb544
fix: changing default model to gpt-4o-mini on OpenAI API calls (#8360)
* chaning default model to gpt-4o-mini

* adding release notes

* fixing some missed tests

* fixing some more missed tests

* fixing one last missed test

* fixing linting issues

* making pylint happy about an end2end test

* chaning if test to walruss operator

* fixing azure embedder from ada to text-embedding-ada-002
2024-09-17 10:36:42 +02:00
Ulises M
145ca89a3f
feat: Expose default_headers and add kwargs for Azure Client (#8244)
* default_headers and azure_kwargs added

* update docstrings

* dont forget about chat generator

* Remove azure_kwargs argument

---------

Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
2024-09-10 10:29:56 +00:00
Stefano Fiorucci
e17d0c4192
chore: deprecate to_openai_format and create similar utility functions (#8146)
* deprecate and add new specific functions

* reno
2024-08-02 16:47:17 +02:00
Amna Mubashar
b374c528b2
Assign streaming_callback to OpenAIGenerator and OpenAIChatGenerator in run() method (#8054)
* Add optional parameter for streaming_callback in run() method
2024-07-24 15:49:19 +02:00
Vladimir Blagojevic
a59de1d7b3
chore: Combined main unblock (#8045)
* Pin structlog to 24.2.0 due to unit test failures

* Remove object init parameter in huggingface_hub unit tests

* Use less restrictive structlog pin

* Add release note
2024-07-19 10:39:10 +02:00
Nitanshu Vashistha
376336686b
feat: Configure max_retries and timeout for AzureOpenAIChatGenerator (#7988)
* feat: Configure max_retries & timeout for AzureOpenAIChatGenerator

max_retries: if not set is read from the OPENAI_MAX_RETRIES
env variable or set to 5.

timeout: if not set is read from the OPENAI_TIMEOUT
env variable or set to 30.

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>

* Update haystack/components/generators/chat/azure.py

* Update haystack/components/generators/chat/azure.py

* Update max_retries-for-AzureOpenAIChatGenerator-9e49b4c7bec5c72b.yaml

---------

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
2024-07-08 22:34:51 +02:00
Nitanshu Vashistha
167e886f2c
feat: Configure max_retries & timeout for AzureOpenAIGenerator (#7983)
max_retries: if not set is read from the OPENAI_MAX_RETRIES
env variable or set to 5.

timeout: if not set is read from the OPENAI_TIMEOUT
env variable or set to 30.

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>
2024-07-08 11:16:26 +02:00
Vladimir Blagojevic
535a281eec
feat: Add option to use HF_TOKEN as env var for authentication across all HF components (#7942)
* Read both HF_API_TOKEN and HF_TOKEN env vars in all HF related components

* Add reno note

* Test fixes

* More test updates

* More test updates
2024-06-27 10:31:58 +02:00
Amna Mubashar
866e6c8fc2
Add the missing parameter for serialization (#7929)
* Add the missing parameter for serialization

* Updated test

---------

Co-authored-by: Amna Mubashar <amna.mubashar@Amnas-MBP.fritz.box>
2024-06-26 11:07:00 +02:00
Stefano Fiorucci
c59ad95f42
chore: remove deprecated TGI generators (#7908)
* remove deprecated TGI generators

* rm unused import
2024-06-21 11:15:13 +02:00