4166 Commits

Author SHA1 Message Date
MaChi
1fd4dfddcd
Merge branch 'main' into feature/chinese-document-splitter 2025-06-05 17:37:55 +08:00
Vladimir Blagojevic
b69d261280
chore: Make docstring-parser core dep (#9477)
* Make docstring-parser core dep

* Add reno note
2025-06-05 11:28:18 +02:00
David S. Batista
a1dea7f7ae fixing linting issues 2025-06-05 11:14:17 +02:00
David S. Batista
67c1d43b40 adding LICENSE header 2025-06-05 11:09:21 +02:00
David S. Batista
d5d0606de4 wip 2025-06-05 11:07:28 +02:00
David S. Batista
e8ec98c876 adding LICENSE header to tests 2025-06-05 11:04:50 +02:00
David S. Batista
efb1aef485 wip 2025-06-05 10:59:31 +02:00
mc112611
10ddc6edc0 Add test script for ChineseDocumentSplitter, remove Chinese comments, and fix lint issues 2025-06-05 16:08:21 +08:00
David S. Batista
7b2d038098 fixing lazy import 2025-06-04 18:00:57 +02:00
David S. Batista
32cd95c602 adding hanlp dependency 2025-06-04 17:45:26 +02:00
David S. Batista
9bc0fc0835
Merge branch 'main' into feature/chinese-document-splitter 2025-06-04 16:37:13 +01:00
David S. Batista
34552f2e5d cleaning up 2025-06-04 17:36:55 +02:00
Vladimir Blagojevic
853a32f8da
feat: Improve ChatMessage _deserialize_content ValueError - make it more LLM friendly (#9484)
* Improve ChatMessage _deserialize_content ValueError - make it more LLM friendly

* Add unit test

* Add reno note

* Add descriptive ValueError for missing role

* Update haystack/dataclasses/chat_message.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update releasenotes/notes/improve-chatmessage-error-messages-llm-agents-a1b2c3d4e5f6g7h8.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Add role check in ChatMessage

* fixes + refinements

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-06-04 15:14:05 +00:00
David S. Batista
20ea23409d cleaning release notes 2025-06-04 15:40:30 +02:00
David S. Batista
089d207f17 cleaning release notes 2025-06-04 15:39:16 +02:00
David S. Batista
eda0f79150
Merge branch 'main' into feature/chinese-document-splitter 2025-06-04 14:20:20 +01:00
Sebastian Husch Lee
db359cff40
Add state to agent pydocs (#9486) 2025-06-04 14:01:58 +02:00
Sebastian Husch Lee
ff56363db1
fix: In set_output_types check that the decorator @component.output_types is not present on the run_async method (#9485)
* Fix

* Add reno
2025-06-04 12:17:47 +02:00
Stefano Fiorucci
1e2214a1a0
feat: ChatMessage.to_openai_dict_format - add require_tool_call_ids parameter (#9481) 2025-06-03 16:55:13 +02:00
mc112611
7b6d05a3f7 Fix according to review:
- Removed notebook and original_pipeline.png
- Added release note YAML file in notes/
- Reverted config.yaml
- Implemented lazy import for hanlp
- Removed main guard block from module
2025-06-03 20:09:15 +08:00
MaChi
4331b49eda
Merge branch 'main' into feature/chinese-document-splitter 2025-06-03 18:45:53 +08:00
Sebastian Husch Lee
ce0917e586
feat: Add raise_on_failure boolean parameter to OpenAIDocumentEmbedder and AzureOpenAIDocumentEmbedder (#9474)
* Add raise_on_failure to OpenAIDocumentEmbedder

* Add reno

* Add parameter to Azure Doc embedder as well

* Fix bug

* Update reno

* PR comments

* update reno
2025-06-03 10:22:34 +00:00
Sebastian Husch Lee
5fcd7c4732
feat: Allow passing of additional parameters to HF Inference clients in HuggingFaceAPIChatGenerator and HuggingFaceAPIGenerator (#9457)
* Fix tests by allowing passing of provider

* Add reno

* Fix mypy

* Update release note
2025-06-03 10:21:51 +00:00
Sebastian Husch Lee
12e3de364a
Fix test (#9475) 2025-06-03 08:00:10 +00:00
MaChi
a25d171d3c
Merge branch 'main' into feature/chinese-document-splitter 2025-06-03 08:05:38 +08:00
David S. Batista
b85c8e3382
feat: adding deserialize_component_inplace() (#9459)
* adding tests

* adding release notes

* deserialize_chatgenerator_inplace uses deserialize_component_inplace

* removing tests
2025-06-02 09:40:35 +02:00
MaChi
78d10d3cfb
Merge branch 'main' into feature/chinese-document-splitter 2025-06-01 07:02:25 +08:00
Sebastian Husch Lee
25c8d7ef9a
fix: In State schema validation use != instead of is not for checking the type of messages (#9454)
* Use != instead of is not

* Add reno

* Use more == instead of is

* Fix mypy
2025-05-30 10:07:37 +02:00
mc112611
307f8340b2 fix: update tests and release notes 2025-05-29 17:20:49 +08:00
mc112611
3ffa3b70a6 Add Chinese DocumentSplitter support with examples 2025-05-29 16:36:33 +08:00
Stefano Fiorucci
2616d4d55b
test: speed up some tests + minor refactorings (#9451)
* this is an integration test

* more improvements

* rm redundant comments
2025-05-29 09:49:11 +02:00
Sebastian Husch Lee
81c0cefa41
refactor: Refactor hf api chat generator (#9449)
* Refactor HFAPI Chat Generator

* Add component info to generators

* Fix type hint

* Add reno

* Fix unit tests

* Remove incorrect dev comment

* Move _convert_streaming_chunks_to_chat_message to utils file
2025-05-27 15:55:06 +02:00
atopx
3deaa20cb6
feat: Add HuggingFace API (text-embeddings-inference for rerank model) for component.rankers (#9414)
* feat(component.rankers): Add HuggingFace API (text-embeddings-inference for rerank) ranker component

* update test flow & doc loaders

* Support run_async for HuggingFaceAPIRanker

* Add release note for HuggingFace API support in component.rankers

* Add release note for HuggingFace API support in component.rankers

* Add release note for HuggingFace API support in component.rankers

* Add release note for HuggingFace API support in component.rankers

* fix:
1. `hugging_face_api.HuggingFaceAPIRanker` rename to `hugging_face_tei.HuggingFaceAPIRanker`
2. HuggingFaceAPIRanker: use our Secret API for token
3. add the missing modules for `docs/pydoc/config/rankers_api.yml`
4. added function `async_request_with_retry` for `haystack/utils/requests_utils.py` and added unittest on `test/utils/test_requests_utils.py`
4. HuggingFaceAPIRanker: refactor the retry function to support configuration based on attempts and status code.
5. HuggingFaceAPIRanker: refactor the test into unit tests using mocks

* fix(HuggingFaceTEIRanker): change the token check logic to use the resolve_value method.

* fix(format): run `hatch run format`

* fix:
- Force keyword-only arguments in __init__ method by adding *,
- Clarify token docstring that it's not always required
- Copy documents to avoid modifying original objects
- Remove test file from slow workflow
- Add monkeypatch eånvironment variable cleanup in tests
- Fix missing module in rankers_api.yml and sort modules alphabetically
- Remove unnecessary test info from release notes

* fix HuggingFaceTEIRanker:
- "None" of "Optional[Secret]" has no attribute "resolve_value"
- run/run_async: too many parameters

* fix(HuggingFaceTEIRanker) :Revise the docstring of the HuggingFaceTEIRanker, improve the parameter descriptions, ensure consistency and clarity. Add error handling information to enhance the readability of the API response.

* fix:unit test for HuggingFaceTEIRanker raise message

* fix fmt

* minor refinements

* refine release note

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2025-05-27 12:44:54 +02:00
Sebastian Husch Lee
db3d95b12a
refactor: Refactor openai generator (#9445)
* Refactor openai generator and chat generator to reusue same util methods

* Start fixing tests

* More fixes

* Fix mypy

* Fix
2025-05-27 12:44:17 +02:00
Amna Mubashar
64def6d41b
feat: add component name and type to StreamingChunk (#9426)
* Stream component name in openai

* Fix type

* PR comments

* Update huggingface gen

* Typing fix

* Update huggingfacelocal gen

* Fix errors

* Remove model changes

* Fix minor errors

* Update releasenotes/notes/add-component-info-dataclass-be115dee2fa50abd.yaml

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* PR comments

* update annotation

* Update hf files

* Fix linting

* Add a from_component method

* use add_component

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
2025-05-27 12:23:40 +02:00
Stefano Fiorucci
085c3add41
ci: prevent DocumentWriter tests from blocking CI (#9448) 2025-05-27 12:10:21 +02:00
Stefano Fiorucci
d8487c4d8d
chore: make mypy run with --check-untyped-defs; fix some errors (#9447)
* chore: make mypy run with --check-untyped-defs; fix some errors

* small fixes

* use HfPipeline

* fix license error
2025-05-27 07:35:25 +00:00
David S. Batista
da60156174
chore: removing unused imports from tests (#9446) 2025-05-26 16:22:51 +00:00
David S. Batista
2092bedb90
chore: removing unused imports from tests (#9444) 2025-05-26 13:41:36 +00:00
Sebastian Husch Lee
b8dff93200
chore: Fix Streaming Callback types (#9441)
* Fix types

* Add select_streaming_callback
2025-05-26 15:39:59 +02:00
David S. Batista
c82a3377f2
chore: cleaning up tests (#9443) 2025-05-26 15:12:19 +02:00
Stefano Fiorucci
802328e292
chore: drop python>=3.9 check in type serialization (#9439)
Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-05-26 09:40:54 +00:00
Denis Washington
eefda0452d
chore: Make the Haystack core "type complete" (#9438)
* chore: Make the Haystack core "type complete"

For libraries with a `py.typed` marker, it is [recommended][1] to
make all public interfaces "type complete", i.e. to explicitly
annotate all function parameters and return types. Doing so has the
following benefits:

- It maximizes the type information available to users and IDEs.
- It ensures that the argument and return types are the intended ones.
- It sidesteps differences in type inference between the different
  type checker implementations.

This change makes a first step towards type completeness by enabling
the Mypy `disallow_incomplete_defs` for the core modules (excluding
`haystack.components.*` and `haystack.testing.*`) and fixing the
resulting errors.

[1]: https://typing.python.org/en/latest/guides/libraries.html#how-much-of-my-library-needs-types

* chore: Add `python_version = 3.9` to Mypy config

This catches type constructs that are only supported in later Python
versions.

* Remove unused import

* try to fix linting

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2025-05-26 11:00:22 +02:00
Sebastian Husch Lee
0d23fe7f0d
fix: Fix invoker to work when using dataclass with from_dict but dataclass… (#9434)
* Fix invoker to work when using dataclass with from_dict but dataclass is already given

* add reno

* Add unit test

* Remove line
2025-05-26 09:40:09 +02:00
Seth Peters
f025501792
fix: LLMMetadataExtractor bug in handling Document objects with no content
* test(extractors): Add unit test for LLMMetadataExtractor with no content

Adds a new unit test `test_run_with_document_content_none` to `TestLLMMetadataExtractor`.

This test verifies that `LLMMetadataExtractor` correctly handles documents where `document.content` is None or an empty string.

It ensures that:

- Such documents are added to the `failed_documents` list.

- The correct error message ("Document has no content, skipping LLM call.") is present in their metadata.

- No actual LLM call is attempted for these documents.

This test provides coverage for the fix that prevents an AttributeError when processing documents with no content.

* chore: update comment to reflect new behavior in _run_on_thread method

* docs: Add release note for LLMMetadataExtractor no content fix

* Update releasenotes/notes/fix-llm-metadata-extractor-no-content-910067ea72094f18.yaml

* Update fix-llm-metadata-extractor-no-content-910067ea72094f18.yaml

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-05-23 18:57:39 +02:00
Haystack Bot
1fb2477149
chore: Update unstable version to 2.15.0-rc0 (#9437)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-05-23 16:46:27 +02:00
David S. Batista
81425afe22
typing: typing for SuperComponents public methods (#9420)
* adding stub files for the supercomponents in haystack

* adding LICENSE to stub files

* fixing

* fixing no member issue

* fixing no member issue #2

* wip

* makying static analyzers happy

* adding alternative to stubs for all SuperComponents + tests

* Update haystack/components/converters/multi_file_converter.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/preprocessors/document_preprocessor.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* removing tests + left stub

* fixing run typing

* making the fake run keyword arguments only

* extend fake to all user callable methods

* fixing async run signature

* fixing async run signature

* removing async

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
v2.15.0-rc0
2025-05-23 13:57:05 +00:00
Sebastian Husch Lee
33115bcccb
fix: Copy context when using ThreadPoolExecutor in `AsyncPipeline.run_async (#9433)
* Copy context when using threadpoolexecutor in AsyncPipeline.run_async

* Add reno

* Add dev comment
2025-05-23 15:11:08 +02:00
David S. Batista
d8cc6f7339
chore: removing an invalid arg from pipeline.base.PipelineBase.connect() docstring 2025-05-23 14:48:41 +02:00
Sebastian Husch Lee
6f5b745d0d
feat: Improve formatting in print streaming chunk (#9431)
* Improve formatting in print streaming chunk

* Add reno

* Fix mypy

* PR comments
2025-05-23 12:22:03 +00:00