yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2026-01-07 12:37:27 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,959 Commits 100 Branches 267 Tags
Commit Graph

3959 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
David S. Batista
be2d1fb303
feat: adding AutoMergingRetriever and HierarchicalDocumentSplitter (#9067) 
* adding Auto-Merging-Retriever

* adding release notes

* updating tests

* adding renamed file

* Update haystack/components/preprocessors/hierarchical_document_splitter.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/retrievers/auto_merging_retriever.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* fixing tests and imports

* adding pydoc

* adding to type checking

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-03-19 18:25:23 +00:00
tstadel
9a046ed431
fix: don't break on IndexError in logging (#9068) 
* fix: don't break on IndexError in logging

* add reno

* Update releasenotes/notes/fix-logging-index-error-c58691db633542c5.yaml

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2025-03-19 17:46:35 +01:00
mathislucka
9fbfa9676f
feat: time extension for ChatPromptBuilder (#9001) 
* feat: time extension for ChatPromptBuilder

* chore: release notes

* Fix comment

---------

Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
 2025-03-19 15:38:55 +01:00
Amna Mubashar
3c101cdfd6
feat: add split_by_row feature to CSVDocumentSplitter (#9031) 
* Add split by row feature
 2025-03-19 12:18:44 +01:00
David S. Batista
ed931b4c2b
fix: adding pylint disable for EvalRunResult end2endtest (#9054) 2025-03-18 11:20:11 +01:00
dependabot[bot]
75a9441e84
build(deps): bump tj-actions/changed-files from 45 to 46 (#9049) 
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 45 to 46.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v45...v46)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-03-18 11:10:10 +01:00
David S. Batista
de76d20f12
fix: updating end2end evaluation tests (#9053) 
* updating tests

* fixing tests, default now is JSON object and no longer dataframe

* cleaning up leftovers
 2025-03-18 10:52:05 +01:00
Sebastian Husch Lee
783b74bd21
Add reno (#9051) 2025-03-18 10:12:26 +01:00
David S. Batista
c06a5434fc
docs: updating docstring BranchJoiner 2025-03-17 17:34:55 +01:00
Stefano Fiorucci
b0e7411205
fix: use eager imports in haystack/__init__.py (#9042) 2025-03-17 16:47:38 +01:00
David S. Batista
3b1918c967
chore: removing EvalRunResult deprecation warnings (#9047) 
* removing warnings

* adding release notes
 2025-03-17 13:50:49 +00:00
Sebastian Husch Lee
6f98cc2ac1
refactor: Update how we look for finish_reason (#9046) 
* Update how we look for finish_Reason

* Additional change

* Add unit test and integration test

* Refactor

* Use correct mock

* PR comments
 2025-03-17 13:25:00 +01:00
Stefano Fiorucci
c5cde40d3a
unpin ruff and update code (#9040) 2025-03-14 14:53:25 +00:00
Sebastian Husch Lee
6366f6577e
chore: Use thread safe import in import_class_by_name utility function (#9028) 
* Use thread safe import

* fix debug string
 2025-03-14 12:31:06 +01:00
Sebastian Husch Lee
3d7d65a260
Pin ruff (#9038) 2025-03-14 12:00:21 +01:00
Sebastian Husch Lee
363ac504dc
feat: Add warning to ChatPromptBuilder and PromptBuilder if they have variables, but required_variables is not set (#9027) 
* Add warning to ChatPromptBuilder and PromptBuilder if they have variables, but required variables is not set.

* Add reno
 2025-03-12 15:35:19 +01:00
Sebastian Husch Lee
4edefe3e56
Feat: Support Azure Workload Identity Credential (#9012) 
* Start adding support for passing callable to Azure components

* Add to chat version

* Fix test

* Add reno

* Add support to azure doc and text embedder

* Rename

* update llm metadata extractor

* Add tests for text embedder

* Update tests

* Remove unused fixture and import

* Update reno
 2025-03-12 13:45:40 +01:00
Stefano Fiorucci
4c1facdfab
fix: add dataframe to legacy fields for Document (#9026) 
* fix: add dataframe to legacy fields for Document

* fmt

* small fixes
 2025-03-12 13:01:03 +01:00
Sebastian Husch Lee
9905e9fa17
fix: Fix logging test (#9024) 
* Pin structlog

* Fix test
 2025-03-12 10:54:52 +01:00
Julian Risch
195d4031b9
chore: remove mention of 2.0 from banner (#9010) 
* chore: remove mention of 2.0 from  banner

* Update img alt text in readme

* Change the banner

---------

Co-authored-by: bilgeyucel <bilgeyucel96@gmail.com>
 2025-03-11 09:53:51 +01:00
Mohammed Abdul Razak Wahab
7291134680
feat: Improve type validation for bare types (#8997) 
* feat: Improve type validation for bare types

* added release notes

* refactor

* resolve review comments

* address review comments

* Update haystack/core/type_utils.py

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
 2025-03-11 08:48:26 +01:00
Sebastian Husch Lee
3d41c04130
fix: Fix type serialization and deserialization (#8993) 
* Expand tests

* New version of type serialization

* Adding more tests

* More tests

* Fix type serialization when using python 3.9

* Deserialization works with Optional now and we don't require 'typing.' to be present anymore

* Don't worry about Literal

* Add reno

* Fix mypy

* Pylint

* Add additional test

* Simplify

* Add back comment

* Fix types

* Fix
 2025-03-07 11:10:16 +01:00
David S. Batista
672ab09477
fix: cleaning up InMemoryDocumentStore executor when created inside the class (#8994) 
* cleaning up executor when created inside the class

* adding missed tests
 2025-03-07 11:01:29 +01:00
David S. Batista
c037052581
feat: adding function to detect unmapped CID characters in PDFMinerToDocument (#8992) 
* adding function to detect unmapped CID characters

* adding release notes

* adding test for logs
 2025-03-06 15:44:06 +00:00
David S. Batista
4c9d08add5
feat: async support for the HuggingFaceLocalChatGenerator (#8981) 
* adding async run method

* passing an optional ThreadExecutor

* adding tests

* adding release notes

* nit: license

* fixing linting

* Update releasenotes/notes/adding-async-huggingface-local-chat-generator-962512f52282d12d.yaml

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Use Phi isntead (#8982)

* build: drop Python 3.8 support (#8978)

* draft

* readd typing_extensions

* small fix + release note

* remove ruff target-version

* Update releasenotes/notes/drop-python-3.8-868710963e794c83.yaml

Co-authored-by: David S. Batista <dsbatista@gmail.com>

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* Update unstable version to 2.12.0-rc0 (#8983)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* fix: allow support for `include_usage` in streaming using OpenAIChatGenerator (#8968)

* fix error in handling usage completion chunk

* ci: improve release notes format checking (#8984)

* chore: fix invalid release note

* try improving relnote linting

* add relnotes path

* fix bad release note

* improve reno config

* fix: handle async tests in`HuggingFaceAPIChatGenerator` to prevent error (#8986)

* add missing asyncio

* explicitly close connection in the test

* Fix tests (#8990)

* docs: Update docstrings of `BranchJoiner` (#8988)

* Update docstrings

* Add a bit more explanatory text

* Add reno

* Update haystack/components/joiners/branch.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/joiners/branch.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/joiners/branch.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/joiners/branch.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Fix formatting

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* PR comments

* destroying ThreadPoolExecutor when the generator instance is being destroyied, only if it was not passed externally

* fixing bug in streaming_callback

* PR comments

---------

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: Haystack Bot <73523382+HaystackBot@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2025-03-06 15:57:11 +01:00
Sebastian Husch Lee
c4fafd9b04
docs: Update docstrings of BranchJoiner (#8988) 
* Update docstrings

* Add a bit more explanatory text

* Add reno

* Update haystack/components/joiners/branch.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/joiners/branch.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/joiners/branch.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/joiners/branch.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Fix formatting

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2025-03-06 13:08:07 +01:00
Sebastian Husch Lee
24084e6431
Fix tests (#8990) 2025-03-06 11:17:57 +01:00
Amna Mubashar
ae26e7580b
fix: handle async tests inHuggingFaceAPIChatGenerator to prevent error (#8986) 
* add missing asyncio

* explicitly close connection in the test
 2025-03-06 10:55:01 +01:00
Stefano Fiorucci
40798bc4f2
ci: improve release notes format checking (#8984) 
* chore: fix invalid release note

* try improving relnote linting

* add relnotes path

* fix bad release note

* improve reno config
 2025-03-05 19:07:46 +01:00
Amna Mubashar
13c3768d49
fix: allow support for include_usage in streaming using OpenAIChatGenerator (#8968) 
* fix error in handling usage completion chunk
 2025-03-05 18:30:26 +01:00
Haystack Bot
ab67a76ecd
Update unstable version to 2.12.0-rc0 (#8983) 
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
 2025-03-05 16:15:09 +01:00
Stefano Fiorucci
c04c900f26
build: drop Python 3.8 support (#8978) 
* draft

* readd typing_extensions

* small fix + release note

* remove ruff target-version

* Update releasenotes/notes/drop-python-3.8-868710963e794c83.yaml

Co-authored-by: David S. Batista <dsbatista@gmail.com>

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
v2.12.0-rc0 		2025-03-05 14:59:56 +00:00
Sebastian Husch Lee
4a87ceb0ed
Use Phi isntead (#8982) 2025-03-05 15:53:26 +01:00
Sebastian Husch Lee
f741df88df
fix: Update flaky HugginFace Generator tests to use more reliable model and add instruction tokens (#8980) 
* Fix test

* Make other HF tests more reliable

* Add back test
 2025-03-05 15:26:17 +01:00
Stefano Fiorucci
ec97f4d991
update transformers test dependency to 4.48.3 (#8979) 2025-03-05 14:49:34 +01:00
Julian Risch
b77f2bad79
feat: Add async run to DocumentWriter (#8962) 
* add async run to DocumentWriter

* reno
 2025-03-05 11:53:35 +01:00
Stefano Fiorucci
bb0e36f712
feat: increase Mermaid timeout and make it configurable (#8973) 
* increase Mermaid timeout and make it configurable

* rm e2e trigger

* simplify test
 2025-03-05 10:49:34 +00:00
David S. Batista
9581fea3bc
feat: adding async version of InMemoryDocumentStore and associated retrievers (#8963) 
* adding classes from experimental

* adding release notes

* adding tests

* merging all into a single class

* adding async retriever methods

* Update haystack/document_stores/in_memory/document_store.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* adding missed tests

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-03-05 11:36:24 +01:00
Stefano Fiorucci
9da6696a45
chore: make openapi-llm an optional dependency (#8958) 
* openapi-llm should be and optional dependency

* rm empty line
 2025-03-05 11:15:19 +01:00
Stefano Fiorucci
10f11d40d4
build: support python 3.13 (#8965) 
* support python 3.13

* release note

* add python version info to contributing guide

* better explanation
 2025-03-05 09:49:10 +00:00
Mohammed Abdul Razak Wahab
e33a9e46ed
fix: add chat message name field (#8969) 
* add-chat-message-name-field

* add release notes

* Update add-chat-message-name-field-a8ae96fb9ff13f7b.yaml

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-03-05 09:29:36 +00:00
Julian Risch
830e7497c3
chore: update EvalRunResult deprecation warning to 2.12 (#8957) 
* chore: update EvalRunResult deprecation warning to 2.12

* Update warning message

* format
 2025-03-04 11:38:18 +00:00
Stefano Fiorucci
f3c44be904
refactor!: remove dataframe field from Document and ExtractedTableAnswer; make pandas optional (#8906) 
* remove dataframe

* release note

* small fix

* group imports

* Update pyproject.toml

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* Update pyproject.toml

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* address feedback

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2025-03-04 11:06:07 +00:00
Mohammed Abdul Razak Wahab
0d65b4caa7
feat: Enhance error handling in Azure document embedder (#8941) 
* feat: Enhance error handling in Azure document embedder

* add release notes

* address review comments

* Update releasenotes/notes/add-azure-embedder-exception-handler-c10ea46fb536de3b.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* more alignment with OpenAI impl

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-03-04 11:16:08 +01:00
Amna Mubashar
28db039bca
feat: add run_async to HuggingfaceAPIChatGenerator (#8943) 
* add run_async

* add release notes

* Add integration test
 2025-03-03 16:51:30 +01:00
Amna Mubashar
1b2053b358
Small fix in the docstring example (#8950) 2025-03-03 16:26:42 +01:00
tstadel
13968cc15b
fix: in OpenAIChatGenerator set additionalProperties to False when tools_strict=True (#8913) 
* fix: set ComponentTool addtionalProperties for OpenAI tools_strict=True

* add reno

* Move the additionalProperties into the OpenAIChatGenerator

* Remove

* Put additionalProperties into the correct place

* Fix test

* Update releasenotes/notes/fix-componenttool-for-openai-tools_strict-998e5cd7ebc6ec19.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Sebastian Husch Lee <sebastian.lee@deepset.ai>
Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2025-03-03 16:23:24 +01:00
Sebastian Husch Lee
296e31c182
feat: Add Type Validation parameter for Pipeline Connections (#8875) 
* Starting to refactor type util tests to be more systematic

* refactoring

* Expand tests

* Update to type utils

* Add missing subclass check

* Expand and refactor tests, introduce type_validation Literal

* More test refactoring

* Test refactoring, adding type validation variable to pipeline base

* Update relaxed version of type checking to pass all newly added tests

* trim whitespace

* Add tests

* cleanup

* Updates docstrings

* Add reno

* docs

* Fix mypy and add docstrings

* Changes based on advice from Tobi

* Remove unused imports

* Doc strings

* Add connection type validation to to_dict and from_dict

* Update tests

* Fix test

* Also save connection_type_validation at global pipeline level

* Fix tests

* Remove connection type validation from the connect level, only keep at pipeline level

* Formatting

* Fix tests

* formatting
 2025-03-03 16:00:22 +01:00
Sebastian Husch Lee
00fe4d157d
feat: Add run async for AzureOpenAIChatGenerator (#8948) 
* Add tests for run_async

* Add reno

* Add async client

* Add init test

* Add comment

* Fix test

* Update releasenotes/notes/run-async-azure-54450f0c2495f5c8.yaml

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

---------

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
 2025-03-03 14:17:18 +00:00
Sebastian Husch Lee
52a028251c
refactor!: update AzureOCRDocumentConverter to not use the dataframe field for tabular Documents (#8885) 
* Save document as a csv table now

* Fix tests

* Fix tests

* Add reno
 2025-03-03 12:45:02 +00:00