yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-11-28 07:57:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,135 Commits 104 Branches 264 Tags
Commit Graph

1736 Commits

Author SHA1 Message Date
Sebastian Husch Lee
114b4568ba
Fix state_schema serialization for agent tracing (#9278) 2025-04-22 09:39:41 +02:00
Sebastian Husch Lee
0f374e0563
Fix from_dict and update test (#9277) 2025-04-22 06:59:03 +00:00
Sebastian Husch Lee
19cf220136
feat: integrate two ready-made SuperComponents from haystack-experimental (#9235) 
* Add super component decorator

* Add reno

* MultiFileConverter

* Add DocumentPreprocessor

* Add reno

* Add tests and change doc preprocessor to split first then clean

* Remove code from merge

* Add to pydoc and missing test file

* PR comments

* Lint fix

* Fix mypy

* Fix mypy

* Add comment

* PR comments

* Update haystack/components/converters/multi_file_converter.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/converters/multi_file_converter.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* PR comments

* PR comment

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2025-04-17 10:02:26 +00:00
Sebastian Husch Lee
c5684b64a6
fix: Fix datadog tracer tests (#9253) 
* Fix tests

* Make tests work in old versions of datadog
 2025-04-17 09:26:27 +02:00
Sebastian Husch Lee
5154d1c7eb
feat: Add super component decorator (#9233) 
* Add super component decorator

* Add reno

* Update tests
 2025-04-16 16:47:07 +00:00
Amna Mubashar
498637788a
feat: Allow OpenAI client config in OpenAIChatGenerator and AzureOpenAIChatGenerator (#9215) 
* Allow OpenAI client config in chat generator

* Add init_http_client as a util method

* Update azure chat gen

* Fix linting
 2025-04-16 18:32:13 +02:00
Mohammed Abdul Razak Wahab
c4689f16c9
feat: allow SuperComponent to Include outputs from non leaf pipeline components (#9242) 
* allow non leaf outputs in supercomponents

* add rn

* add output default fallback

* Update releasenotes/notes/allow-non-leaf-outputs-in-supercomponents-outputs-adf29d68636c23ba.yaml

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
 2025-04-16 16:59:43 +02:00
Sebastian Husch Lee
cdc53cae78
fix: Add batch_size to to_dict of TransformersSimilarityRanker (#9248) 
* Add missing batch_size to to_dict of similarity ranker

* Add reno
 2025-04-16 12:16:59 +02:00
MetroCat69
f7ac4b35cb
feat: add run_async for HuggingFaceAPIDocumentEmbedder (#9226) 
* added async support for HuggingFaceAPIDocumentEmbedder

* added type anotations, removed unused import

* Trigger mark test complited

* Apply suggestions from code review

* utility function

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-04-16 09:54:36 +02:00
Sebastian Husch Lee
f46bf14851
fix: Allow Agent to run with no tools (#9230) 
* Fix

* Add reno

* Add test

* Update docstring and warning message

* Update docstring
 2025-04-16 07:53:21 +02:00
Sebastian Husch Lee
185e1c79c9
feat: Agent tracing (#9240) 
* Agent tracing

* Small changes

* Some changes and refactoring

* Refactoring to reuse code

* Fix

* Add reno

* Fix tests

* Fix tests

* Fix linting

* Refactor and add tracing support to run_async of Agent

* Reduce duplicate code

* Remove finalize_run

* Use break instead of copying code three times

* Adding a test

* Add tracing unit tests

* Make async tracing test actually run async

* Increase test coverage

* Unit test for traces in pipeline

* Add cleanup

* Fix proper indentation

* PR comments

* PR comments and new test

* Update warning message

* Update warning message

---------

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
 2025-04-15 15:58:26 +02:00
Stefano Fiorucci
656fe6dc6e
chore: LLM Evaluators - remove deprecated parameters (#9219) 2025-04-15 09:26:31 +02:00
David S. Batista
d860a73ddb
chore: cleaning duplicated import (#9234) 2025-04-15 09:25:19 +02:00
Julian Risch
13780cfcc4
feat: Add run_async to Agent (#9239) 
* add run_async

* refactor with _check_exit_conditions

* add run_async tests

* reno

* fix linting issues
 2025-04-14 19:01:59 +00:00
Stefano Fiorucci
c67d1bf0e9
fix: make ChatMessage.from_dict handle cases where optional fields are missing (#9232) 
* fix: make ChatMessage.from_dict handle cases where optional fields are missing

* one more test
 2025-04-14 14:53:08 +02:00
Mohammed Abdul Razak Wahab
859e90cc61
fix: Document field precedence in to_dict() (#9227) 
* fix: Document field precedence in to_dict()

* add test

* add release note
 2025-04-14 13:53:12 +02:00
Stefano Fiorucci
dcba774e30
chore: LLMMetadataExtractor - remove deprecated parameters (#9218) 2025-04-11 15:50:52 +02:00
Stefano Fiorucci
8bf41a8510
test: create e2e environment; stop testing spacy in unit tests (#9212) 
* ci: create e2e environment; stop testing spacy in unit tests

* try fix

* fix yml

* exclude test python files

* self-referential environment

* do not use self-referential environment
 2025-04-11 10:28:53 +00:00
Stefano Fiorucci
81fbe546cb
feat: ChatGenerator protocol - do not require to_dict and from_dict methods (#9213) 
* minimize protocol

* progress

* rm unneeded test changes

* reno

* use keywords arguments for clarity
 2025-04-11 10:30:48 +02:00
David S. Batista
45aa9608b5
removing async test for non-existant model (#9208) 2025-04-10 12:38:35 +02:00
Arseniy Shkunkov
bac29d9337
feat: add run_async for HuggingFaceAPITextEmbedder (#9204) 
* Initial commit

* adding release notes

* adding async integrations tests

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2025-04-10 11:44:40 +02:00
Sebastian Husch Lee
7bb9c6934b
fix: Set messages in state_schema at init time of Agent (#9195) 
* Fix issue and set messages in state_schema at init time

* Add reno

* Small changes and add more tests for state

* Add comment

* PR comments
 2025-04-10 06:39:17 +00:00
Sebastian Husch Lee
6dbc1ff27b
enhancement: Add attributes to PipelineRuntimeError (#9182) 
* Start refactoring PipelineRuntimeError

* Slight change

* Fix test and remove test that had no asserts

* Change back to pipeline runtime error

* PR comments

* Add reno

* PR comments

* Update test
 2025-04-09 08:18:50 +02:00
Stefano Fiorucci
45cd6f43d6
feat: make AzureOpenAIDocumentEmbedder inherit from OpenAIDocumentEmbedder - async support (#9189) 
* feat: make AzureOpenAIDocumentEmbedder inherit from OpenAIDocumentEmbedder - async support

* fix type

* rm unused import

* do not replace newlines

* fix test
 2025-04-08 12:51:45 +02:00
Stefano Fiorucci
6f4e70050f
feat: make AzureOpenAITexttEmbedder inherit from OpenAITextEmbedder - async support (#9188) 
* draft

* updates

* relnote
 2025-04-08 12:51:34 +02:00
Vladimir Blagojevic
2665d048b8
chore: Rename deserialize_tools_inplace -> deserialize_tools_or_toolset_inplace (#9190) 
* Rename deserialize_tools_inplace -> deserialize_tools_or_toolset_inplace

* Add reno note
 2025-04-08 10:45:17 +02:00
Francesco Nuzzo
c539ffa4c3
feat: add run_async for OpenAITextEmbedder (#9084) 
* feat: add run_async for OpenAITextEmbedder

* fix: typing

* fix: avoid replacing newlines with spaces.

Also fix kwargs "input" field to include prefix and suffix

* ci: add release notes

* expand release notes; unit tests

---------

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
Co-authored-by: Michele Pangrazzi <xmikex83@gmail.com>
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
 2025-04-08 07:20:10 +00:00
Stefano Fiorucci
f6fceb1b56
refactor: reorganize Tool serde utility functions (#9185) 
* refactor: reorganize Tool serde utility functions

* license header

* rm unused import

* HF local update
 2025-04-08 08:09:54 +02:00
Vladimir Blagojevic
e1e797206d
feat: Add Toolset support in ChatGenerator(s) (#9177) 
* Add Toolset support in ChatGenerator(s)

* Add reno note

* Update azure test

* Updates

* Minor fix

* Add more tests

* Remove some integration tests

* PR feedback

* rm unused fixture

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
 2025-04-07 14:12:09 +00:00
Vladimir Blagojevic
c81d68402c
feat: Add Toolset to tooling architecture (#9161) 
* Add Toolset abstraction

* Add reno note

* More pydoc improvements

* Update test

* Simplify, Toolset is a dataclass

* Wrap toolset instance with list

* Add example

* Toolset pydoc serde enhancement

* Toolset as init param

* Fix types

* Linting

* Minor updates

* PR feedback

* Add to pydoc config, minor import fixes

* Improve pydoc example

* Improve coverage for test_toolset.py

* Improve test_toolset.py, test custom toolset serde properly

* Update haystack/utils/misc.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Rework Toolset pydoc

* Another minor pydoc improvement

* Prevent single Tool instantiating Toolset

* Reduce number of integration tests

* Remove some toolset tests from openai

* Rework tests

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-04-04 16:09:46 +02:00
Mohammed Abdul Razak Wahab
a2f73d134d
feat(embedders): Add async support for OpenAI document embedder (#9140) 
* feat(embedders): Add async support for OpenAI document embedder

* add release notes

* resolve review comments

* Update releasenotes/notes/openai-document-embedder-async-support-b46f1e84043da366.yaml

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Update openai-document-embedder-async-support-b46f1e84043da366.yaml

---------

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2025-04-04 11:55:59 +00:00
Stefano Fiorucci
c6df8d2c7a
test: monkeypatch OpenAI API key in some unit tests (#9173) 2025-04-04 13:33:22 +02:00
Bilge Yücel
d977b262b6
replace all gpt-3.5-turbo with gpt-4o-mini (#9165) 2025-04-04 12:07:55 +02:00
Stefano Fiorucci
d1db061058
test: make Azure embedders correctly run on PRs from forks (#9168) 2025-04-04 11:16:58 +02:00
Amna Mubashar
dd6ff10d3b
feat: allow OpenAI client config in AzureOpenAI embedders (#9136) 
* Allow OpenAI client config
 2025-04-02 16:50:48 +02:00
David S. Batista
108fc3b4a2
test: adding Pipeline component name checks - cannot have . (dot characters) (#9155) 
* adding component name checks + tests

* fixes
 2025-04-01 16:45:04 +02:00
Julian Risch
c7bc5b54de
feat: integrate SuperComponent from haystack-experimental (#9134) 
* add SuperComponent with utils and tests

* fix tests, circular imports

* use __all__

* reno and walrus operator

* add InvalidMappingValueError, InvalidMappingTypeError

* add run_async

* add usage example
 2025-04-01 14:52:19 +02:00
Sebastian Husch Lee
fce80715e0
feat: Update agent exit condition check (#9151) 
* Update agent to properly check all llm messages instead of just first one

* reno

* fix release note

* PR comments

* Fix unit test

* PR comments

* Updated warning message
 2025-04-01 14:51:53 +02:00
Sebastian Husch Lee
d15f1b9043
feat: Add outputs_to_string to Tool and ComponentTool (#9152) 
* Add outputs_to_string to Tool and ComponentTool

* Doc string and fix tests

* Add reno

* Fix mypy
 2025-04-01 14:04:50 +02:00
Sebastian Husch Lee
bde2d77df0
feat: Move storing of messages into State in Agent (#9150) 
* Update messages to be stored in State so users can control how they are stored through a handler in the schema

* Fix test

* Add test

* Add reno

* Fix docstring
 2025-04-01 11:29:44 +02:00
Julian Risch
f687d49fec
feat: Add option to split by number of tokens to RecursiveDocumentSplitter (#9143) 
* add token split_unit

* fix overlap with fallback

* reno

* mark as integration tests

* use type ignore instead of assert

* Update releasenotes/notes/recursive-splitter-token-df56428887ac45bd.yaml

Co-authored-by: David S. Batista <dsbatista@gmail.com>

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2025-04-01 09:48:59 +02:00
Stefano Fiorucci
b12af1e6a9
chore: use deserialize_chatgenerator_inplace utility function in Agent (#9149) 2025-03-31 16:00:59 +00:00
Stefano Fiorucci
adc3dfc5d2
refactor: LLM evaluators - introduce chat_generator init param; deprecate api, api_key and api_params (#9122) 
* start

* progress

* tests for deserialize_chatgenerator_inplace

* progress on llmevaluator + tests

* update context relevance evaluator

* update faithfulness evaluator + tests

* release note

* rm unused import

* rm indentation
 2025-03-31 15:35:03 +02:00
Julian Risch
fc33382b48
feat: Agent checks that its chat_generator supports tools (#9144) 
* check that chat_generator run accepts tools

* reno
 2025-03-31 14:47:38 +02:00
Julian Risch
c8918e43ba
feat: support multiple exit conditions in Agent and rename init param (#9132) 
* rename to exit_conditions and accept list of str

* reno

* Update haystack/components/agents/agent.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-03-31 09:02:25 +00:00
Julian Risch
e483ec6f56
feat: integrate Agent from haystack-experimental (#9112) 
* add Agent

* add Agent

* update imports

* add state tests

* reno

* remove State, its utils, and tests

* add pydoc yml for agents

* fix module path in serialization test

* fix mypy error and use ChatGenerator protocol

* remove unused import

* address review feedback

* remove unused _load_component
 2025-03-28 14:23:39 +01:00
Stefano Fiorucci
637dcb4599
fix: DALLEImageGenerator - ensure max_retries is correctly set when 0 (#9131) 
* fix: DALLEImageGenerator - ensure max_retries correctly set when 0

* other small fixes

* wording
 2025-03-28 14:21:07 +01:00
scara
18367203a8
fix: manage max_retries=0 in AzureOpenAIGenerator, AzureOpenAIChatGenerator, AzureOpenAITextEmbedder, AzureOpenAIDocumentEmbedder (#9128) 
* fix: manage max_retries=0 in AzureOpenAIGenerator and AzureOpenAIChatGenerator

* fix: manage max_retries=0 in AzureOpenAITextEmbedder and AzureOpenAIDocumentEmbedder
 2025-03-28 13:11:09 +01:00
Stefano Fiorucci
01957b106a
refactor: use token inSentenceTransformersDiversityRanker (#9126) 
* chore: use token in SASEvaluator and SentenceTransformersDiversityRanker

* directly pass token to CrossEncoder

* restrict scope to SentenceTransformersDiversityRanker
 2025-03-28 12:14:11 +01:00
Julian Risch
657d09d7f1
feat: integrate updates of Tool, ToolInvoker, State, create_tool_from_function, ComponentTool from haystack-experimental (#9113) 
* update Tool,ToolInvoker,ComponentTool,create_tool_from_function

* add State and its utils

* add tests for State and its utils

* update tests for Tool etc.

* reno

* fix circular imports

* update experimental imports in tests

* fix unit tests

* fix ChatGenerator unit tests

* mypy

* add State to init and pydoc

* explain State in more detail in release note

* add test from #8913

* re-add _check_duplicate_tool_names and refactor imports

* rename inputs and outputs
 2025-03-28 10:49:23 +01:00