yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-12-04 10:58:45 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,081 Commits 105 Branches 266 Tags
Commit Graph

480 Commits

Author SHA1 Message Date
David S. Batista
201becd400
fix: RecursiveSplitter bug in the case when the recursive chunking is triggered (#9316) 
* initial import

* adding release notes

* Update fixing-bug-recursive-splitter-88d5714529f84e4e.yaml
 2025-04-30 13:03:23 +02:00
Yassin Nouh
ed6176a8cb
fix: make HuggingFaceAPIChatGenerator convert Tool Call arguments from string (#9303) 
* fix: sort imports in hugging_face_api.py

* fix: import logging in hugging_face_api.py

* fix: refactor HuggingFace API tool call handling for improved argument conversion

* Update haystack/components/generators/chat/hugging_face_api.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* refinements + tests + relnote

* simplify

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-04-28 15:36:19 +02:00
Mohammed Abdul Razak Wahab
53308a6294
feat: Add sanitization for Meta field during serialization (#9272) 
* feat: Add sanitization for Meta field during serialization

* Revert "feat: Add sanitization for Meta field during serialization"

This reverts commit c529f7c25b69aed626bb2072c8bf171815b591cc.

* feat: add nested serialization in openai usage object

* add reno

* add nested serialization in OpenAiChatGenerator

* Update releasenotes/notes/nested-serialization-openai-usage-object-3817b07342999edf.yaml

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* merge tests

* Adjust the test

---------

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
 2025-04-26 15:04:02 +05:00
Sebastian Husch Lee
0fdb88424b
fix: Fix Azure test on forks (#9312) 
* Fix unit test

* Fix test
 2025-04-25 11:10:59 +02:00
Stefano Fiorucci
38c39a49de
test: review integration tests (#9306) 
* AzureOCR: convert integration test to unit test and simplify

* clean up HuggingFaceAPITextEmbedder

* clean up LinkContentFetcher

* simplify HuggingFaceLocalGenerator

* clean up OpenAIGenerator

* OpenAIChatGenerator

* SentenceTransformersDiversityRanker

* TransformersSimilarityRanker

* ChatMessage: rm outdated tests

* fail fast false

* typo
 2025-04-25 09:07:57 +02:00
Mohammed Abdul Razak Wahab
f97472329f
feat: Add support for multiple outputs in ConditionalRouter (#9271) 
* feat: Add support for multiple outputs in ConditionalRouter

* Update haystack/components/routers/conditional_router.py

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>

* add additional route

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
 2025-04-24 16:17:06 +02:00
Michele Pangrazzi
4a908d075e
Fix OpenAIGenerator and OpenAIChatGenerator to allow wrapped streaming objects usage (#9304) 
* Fix for handling wrapped ChatCompletion instances in streaming (used by tools like weave)

* Add release note

* Applied same fix to OpenAIGenerator ; Refactoring ; Update release note

* Fix integration test error after refactoring
 2025-04-24 16:16:41 +02:00
Stefano Fiorucci
e3d4e21237
test: mark more tests as slow (#9296) 
* test: mark tests as slow

* alphabetical order; install xet

* revert pyproject

* Trigger Build

* simplify tests as suggested

* add comment to workflow
 2025-04-24 10:25:13 +02:00
Stefano Fiorucci
df662daaef
test: improve some slow tests (#9297) 
* test: improve slow tests

* rm leftover and improve test
 2025-04-24 08:50:36 +02:00
Stefano Fiorucci
9ae7da8df3
test: workflow for slow/unstable integration tests (#9267) 
* workflow for slow integration tests

* try changing skipper

* Trigger Build

* better names

* fix

* mv tika to slow

* try skipping slow workflow

* retry paths-ignore

* remove skipper

* Revert "remove skipper"

This reverts commit 302ed2f07f36b33fa61fde0843b5590d79b98d74.

* better skipper

* retry

* Revert "retry"

This reverts commit fe5dff68f496645cc45292d74fcd8d043e868392.

* try using one workflow

* trigger

* try to see if it fails

* cosmetic changes

* improvements

* try matrix

* retry

* fix

* clean up

* simplify datadog monitoring and trigger

* send event to datadog for nightly failures

* tests should run if: manual trigger, scheduled, PR has label, release branch, or relevant files changed

* clarify slow marker

* improve comments

* labels
 2025-04-23 10:36:44 +02:00
Mohammed Abdul Razak Wahab
ddd7318ae8
fix: use coerce_tag_value in LoggingTracer to serialize tag values (#9251) 
* fix: use coerce_tag_value in LoggingTracer to serialize tag values

* add rn

* fix tests

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
 2025-04-22 16:18:24 +02:00
Sebastian Husch Lee
d5ae46bc93
feat: Add Toolset to Agent (#9284) 
* Add Toolset to Agent

* Add reno
 2025-04-22 14:08:34 +02:00
Grig Alex
14669419f2
feat: Allow OpenAI client config in other components (#9270) 
* Add http config to generators

* Add http config to RemoteWhisperTranscriber

* Add http config to embedders

* Add notes of http config

* disable linter too-many-positional-arguments

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
 2025-04-22 09:44:55 +00:00
Sebastian Husch Lee
114b4568ba
Fix state_schema serialization for agent tracing (#9278) 2025-04-22 09:39:41 +02:00
Sebastian Husch Lee
0f374e0563
Fix from_dict and update test (#9277) 2025-04-22 06:59:03 +00:00
Sebastian Husch Lee
19cf220136
feat: integrate two ready-made SuperComponents from haystack-experimental (#9235) 
* Add super component decorator

* Add reno

* MultiFileConverter

* Add DocumentPreprocessor

* Add reno

* Add tests and change doc preprocessor to split first then clean

* Remove code from merge

* Add to pydoc and missing test file

* PR comments

* Lint fix

* Fix mypy

* Fix mypy

* Add comment

* PR comments

* Update haystack/components/converters/multi_file_converter.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/converters/multi_file_converter.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* PR comments

* PR comment

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2025-04-17 10:02:26 +00:00
Amna Mubashar
498637788a
feat: Allow OpenAI client config in OpenAIChatGenerator and AzureOpenAIChatGenerator (#9215) 
* Allow OpenAI client config in chat generator

* Add init_http_client as a util method

* Update azure chat gen

* Fix linting
 2025-04-16 18:32:13 +02:00
Sebastian Husch Lee
cdc53cae78
fix: Add batch_size to to_dict of TransformersSimilarityRanker (#9248) 
* Add missing batch_size to to_dict of similarity ranker

* Add reno
 2025-04-16 12:16:59 +02:00
MetroCat69
f7ac4b35cb
feat: add run_async for HuggingFaceAPIDocumentEmbedder (#9226) 
* added async support for HuggingFaceAPIDocumentEmbedder

* added type anotations, removed unused import

* Trigger mark test complited

* Apply suggestions from code review

* utility function

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-04-16 09:54:36 +02:00
Sebastian Husch Lee
f46bf14851
fix: Allow Agent to run with no tools (#9230) 
* Fix

* Add reno

* Add test

* Update docstring and warning message

* Update docstring
 2025-04-16 07:53:21 +02:00
Sebastian Husch Lee
185e1c79c9
feat: Agent tracing (#9240) 
* Agent tracing

* Small changes

* Some changes and refactoring

* Refactoring to reuse code

* Fix

* Add reno

* Fix tests

* Fix tests

* Fix linting

* Refactor and add tracing support to run_async of Agent

* Reduce duplicate code

* Remove finalize_run

* Use break instead of copying code three times

* Adding a test

* Add tracing unit tests

* Make async tracing test actually run async

* Increase test coverage

* Unit test for traces in pipeline

* Add cleanup

* Fix proper indentation

* PR comments

* PR comments and new test

* Update warning message

* Update warning message

---------

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
 2025-04-15 15:58:26 +02:00
Stefano Fiorucci
656fe6dc6e
chore: LLM Evaluators - remove deprecated parameters (#9219) 2025-04-15 09:26:31 +02:00
David S. Batista
d860a73ddb
chore: cleaning duplicated import (#9234) 2025-04-15 09:25:19 +02:00
Julian Risch
13780cfcc4
feat: Add run_async to Agent (#9239) 
* add run_async

* refactor with _check_exit_conditions

* add run_async tests

* reno

* fix linting issues
 2025-04-14 19:01:59 +00:00
Stefano Fiorucci
dcba774e30
chore: LLMMetadataExtractor - remove deprecated parameters (#9218) 2025-04-11 15:50:52 +02:00
Stefano Fiorucci
8bf41a8510
test: create e2e environment; stop testing spacy in unit tests (#9212) 
* ci: create e2e environment; stop testing spacy in unit tests

* try fix

* fix yml

* exclude test python files

* self-referential environment

* do not use self-referential environment
 2025-04-11 10:28:53 +00:00
David S. Batista
45aa9608b5
removing async test for non-existant model (#9208) 2025-04-10 12:38:35 +02:00
Arseniy Shkunkov
bac29d9337
feat: add run_async for HuggingFaceAPITextEmbedder (#9204) 
* Initial commit

* adding release notes

* adding async integrations tests

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2025-04-10 11:44:40 +02:00
Sebastian Husch Lee
7bb9c6934b
fix: Set messages in state_schema at init time of Agent (#9195) 
* Fix issue and set messages in state_schema at init time

* Add reno

* Small changes and add more tests for state

* Add comment

* PR comments
 2025-04-10 06:39:17 +00:00
Stefano Fiorucci
45cd6f43d6
feat: make AzureOpenAIDocumentEmbedder inherit from OpenAIDocumentEmbedder - async support (#9189) 
* feat: make AzureOpenAIDocumentEmbedder inherit from OpenAIDocumentEmbedder - async support

* fix type

* rm unused import

* do not replace newlines

* fix test
 2025-04-08 12:51:45 +02:00
Stefano Fiorucci
6f4e70050f
feat: make AzureOpenAITexttEmbedder inherit from OpenAITextEmbedder - async support (#9188) 
* draft

* updates

* relnote
 2025-04-08 12:51:34 +02:00
Francesco Nuzzo
c539ffa4c3
feat: add run_async for OpenAITextEmbedder (#9084) 
* feat: add run_async for OpenAITextEmbedder

* fix: typing

* fix: avoid replacing newlines with spaces.

Also fix kwargs "input" field to include prefix and suffix

* ci: add release notes

* expand release notes; unit tests

---------

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
Co-authored-by: Michele Pangrazzi <xmikex83@gmail.com>
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
 2025-04-08 07:20:10 +00:00
Vladimir Blagojevic
e1e797206d
feat: Add Toolset support in ChatGenerator(s) (#9177) 
* Add Toolset support in ChatGenerator(s)

* Add reno note

* Update azure test

* Updates

* Minor fix

* Add more tests

* Remove some integration tests

* PR feedback

* rm unused fixture

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
 2025-04-07 14:12:09 +00:00
Vladimir Blagojevic
c81d68402c
feat: Add Toolset to tooling architecture (#9161) 
* Add Toolset abstraction

* Add reno note

* More pydoc improvements

* Update test

* Simplify, Toolset is a dataclass

* Wrap toolset instance with list

* Add example

* Toolset pydoc serde enhancement

* Toolset as init param

* Fix types

* Linting

* Minor updates

* PR feedback

* Add to pydoc config, minor import fixes

* Improve pydoc example

* Improve coverage for test_toolset.py

* Improve test_toolset.py, test custom toolset serde properly

* Update haystack/utils/misc.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Rework Toolset pydoc

* Another minor pydoc improvement

* Prevent single Tool instantiating Toolset

* Reduce number of integration tests

* Remove some toolset tests from openai

* Rework tests

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-04-04 16:09:46 +02:00
Mohammed Abdul Razak Wahab
a2f73d134d
feat(embedders): Add async support for OpenAI document embedder (#9140) 
* feat(embedders): Add async support for OpenAI document embedder

* add release notes

* resolve review comments

* Update releasenotes/notes/openai-document-embedder-async-support-b46f1e84043da366.yaml

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Update openai-document-embedder-async-support-b46f1e84043da366.yaml

---------

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2025-04-04 11:55:59 +00:00
Stefano Fiorucci
c6df8d2c7a
test: monkeypatch OpenAI API key in some unit tests (#9173) 2025-04-04 13:33:22 +02:00
Bilge Yücel
d977b262b6
replace all gpt-3.5-turbo with gpt-4o-mini (#9165) 2025-04-04 12:07:55 +02:00
Stefano Fiorucci
d1db061058
test: make Azure embedders correctly run on PRs from forks (#9168) 2025-04-04 11:16:58 +02:00
Amna Mubashar
dd6ff10d3b
feat: allow OpenAI client config in AzureOpenAI embedders (#9136) 
* Allow OpenAI client config
 2025-04-02 16:50:48 +02:00
Sebastian Husch Lee
fce80715e0
feat: Update agent exit condition check (#9151) 
* Update agent to properly check all llm messages instead of just first one

* reno

* fix release note

* PR comments

* Fix unit test

* PR comments

* Updated warning message
 2025-04-01 14:51:53 +02:00
Sebastian Husch Lee
d15f1b9043
feat: Add outputs_to_string to Tool and ComponentTool (#9152) 
* Add outputs_to_string to Tool and ComponentTool

* Doc string and fix tests

* Add reno

* Fix mypy
 2025-04-01 14:04:50 +02:00
Sebastian Husch Lee
bde2d77df0
feat: Move storing of messages into State in Agent (#9150) 
* Update messages to be stored in State so users can control how they are stored through a handler in the schema

* Fix test

* Add test

* Add reno

* Fix docstring
 2025-04-01 11:29:44 +02:00
Julian Risch
f687d49fec
feat: Add option to split by number of tokens to RecursiveDocumentSplitter (#9143) 
* add token split_unit

* fix overlap with fallback

* reno

* mark as integration tests

* use type ignore instead of assert

* Update releasenotes/notes/recursive-splitter-token-df56428887ac45bd.yaml

Co-authored-by: David S. Batista <dsbatista@gmail.com>

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2025-04-01 09:48:59 +02:00
Stefano Fiorucci
b12af1e6a9
chore: use deserialize_chatgenerator_inplace utility function in Agent (#9149) 2025-03-31 16:00:59 +00:00
Stefano Fiorucci
adc3dfc5d2
refactor: LLM evaluators - introduce chat_generator init param; deprecate api, api_key and api_params (#9122) 
* start

* progress

* tests for deserialize_chatgenerator_inplace

* progress on llmevaluator + tests

* update context relevance evaluator

* update faithfulness evaluator + tests

* release note

* rm unused import

* rm indentation
 2025-03-31 15:35:03 +02:00
Julian Risch
fc33382b48
feat: Agent checks that its chat_generator supports tools (#9144) 
* check that chat_generator run accepts tools

* reno
 2025-03-31 14:47:38 +02:00
Julian Risch
c8918e43ba
feat: support multiple exit conditions in Agent and rename init param (#9132) 
* rename to exit_conditions and accept list of str

* reno

* Update haystack/components/agents/agent.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-03-31 09:02:25 +00:00
Julian Risch
e483ec6f56
feat: integrate Agent from haystack-experimental (#9112) 
* add Agent

* add Agent

* update imports

* add state tests

* reno

* remove State, its utils, and tests

* add pydoc yml for agents

* fix module path in serialization test

* fix mypy error and use ChatGenerator protocol

* remove unused import

* address review feedback

* remove unused _load_component
 2025-03-28 14:23:39 +01:00
Stefano Fiorucci
637dcb4599
fix: DALLEImageGenerator - ensure max_retries is correctly set when 0 (#9131) 
* fix: DALLEImageGenerator - ensure max_retries correctly set when 0

* other small fixes

* wording
 2025-03-28 14:21:07 +01:00
scara
18367203a8
fix: manage max_retries=0 in AzureOpenAIGenerator, AzureOpenAIChatGenerator, AzureOpenAITextEmbedder, AzureOpenAIDocumentEmbedder (#9128) 
* fix: manage max_retries=0 in AzureOpenAIGenerator and AzureOpenAIChatGenerator

* fix: manage max_retries=0 in AzureOpenAITextEmbedder and AzureOpenAIDocumentEmbedder
 2025-03-28 13:11:09 +01:00