yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2026-01-07 04:27:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,314 Commits 101 Branches 267 Tags
Commit Graph

591 Commits

Author SHA1 Message Date
Stefano Fiorucci
75e1018f4e
ci: update Hatch to 1.14.2 (#9823) 2025-09-25 08:56:02 +02:00
Arseniy Shkunkov
1fb76ec7e4
feat: add Sparse Embedders based on Sentence Transformers (#9588) 
* Added backend class for SparseEncoder and also SentenceTransformersSparseTextEmbedder

* Added SentenceTransformersSparseDocumentEmbedder

* Created a separate _SentenceTransformersSparseEmbeddingBackendFactory and added tests

* Remove unused parameter

* Wrapped output into SparseEmbedding dataclass + fix tests

* Return correct SparseEmbedding, imports and tests

* fix fmt

* Style changes and fixes

* Added a test for embed function

* Added integration test and fixed some other tests

* Add lint fixes

* Fixed positional arguments

* fix types, simplify and more

* fix

* token fixes

* pydocs, small model in test, cache improvement

* try 3.9 for docs

* better to pin click

* release note

* small fix

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
 2025-09-19 14:00:13 +00:00
dependabot[bot]
e3d4e9e945
chore(deps): bump tj-actions/changed-files from 46 to 47 (#9788) 
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 46 to 47.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v46...v47)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-version: '47'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-15 16:41:45 +02:00
dependabot[bot]
542214cc19
chore(deps): bump actions/labeler from 5 to 6 (#9768) 
Bumps [actions/labeler](https://github.com/actions/labeler) from 5 to 6.
- [Release notes](https://github.com/actions/labeler/releases)
- [Commits](https://github.com/actions/labeler/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/labeler
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-05 16:09:27 +02:00
dependabot[bot]
329bcbb71c
chore(deps): bump actions/setup-python from 5 to 6 (#9764) 
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5 to 6.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-05 10:11:53 +02:00
dependabot[bot]
ce50022a18
chore(deps): bump actions/setup-go from 5 to 6 (#9765) 
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5 to 6.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-05 10:11:31 +02:00
dependabot[bot]
9db408e941
chore(deps): bump actions/stale from 9 to 10 (#9762) 
Bumps [actions/stale](https://github.com/actions/stale) from 9 to 10.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/stale/compare/v9...v10)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-version: '10'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-09-04 16:45:12 +02:00
Stefano Fiorucci
8bb8b67f8a
ci: avoid coveralls blocking CI (#9713) 
* ci: avoid coveralls blocking CI

* trigger unit tests

* remove trigger
 2025-08-14 16:22:55 +02:00
dependabot[bot]
33b2b83ce8
chore(deps): bump actions/checkout from 4 to 5 (#9709) 
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '5'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-08-13 09:28:45 +02:00
Daria Fokina
f8d3a82997
Your commit message here (#9692) 2025-08-11 14:41:16 +02:00
Abdelrahman Kaseb
5f3c37d287
chore: adopt PEP 585 type hints (#9678) 
* chore(lint): enforce and apply PEP 585 type hinting

* Run fmt fixes

* Fix all typing imports using some regex

* Fix all typing written in string in tests

* undo changes in the e2e tests

* make e2e test use list instead of List

* type fixes

* remove type:ignore

* pylint

* Remove typing from Usage example comments

* Remove typing from most of comments

* try to fix e2e tests on comm PRs

* fix

* Add tests typing.List in to adjust test compatiplity
- test/components/agents/test_state_class.py
- test/components/converters/test_output_adapter.py
- test/components/joiners/test_list_joiner.py

* simplify pyproject

* improve relnote

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
 2025-08-07 10:23:14 +02:00
Sebastian Husch Lee
8efbfc0da6
Update documentation codeowners (#9658) 2025-07-29 17:06:04 +02:00
dependabot[bot]
90ec214589
chore(deps): bump pilosus/action-pip-license-checker from 2 to 3 (#9644) 
* chore(deps): bump pilosus/action-pip-license-checker from 2 to 3

Bumps [pilosus/action-pip-license-checker](https://github.com/pilosus/action-pip-license-checker) from 2 to 3.
- [Release notes](https://github.com/pilosus/action-pip-license-checker/releases)
- [Changelog](https://github.com/pilosus/action-pip-license-checker/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pilosus/action-pip-license-checker/compare/v2...v3)

---
updated-dependencies:
- dependency-name: pilosus/action-pip-license-checker
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* trigger check

* try

* fix

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
 2025-07-23 16:36:31 +02:00
Stefano Fiorucci
868ea41698
feat: add SentenceTransformersDocumentImageEmbedder (#9635) 2025-07-22 13:10:33 +02:00
Michele Pangrazzi
d3c6c3e68e
chore: Workflow to sync docs with Docusaurus repo (#9597) 
* Remove state according to https://github.com/deepset-ai/haystack/pull/9578

* Add workflow to sync API docs with Docusaurus repo
 2025-07-08 11:48:44 +02:00
mathislucka
c54a68ab63
fix: files should not be passed as single string (#9559) 
* fix: files should not be passed as single string

* chore: we want word splitting in this case

* fix: place directive before command

* fix: find correct directive placement
 2025-06-27 11:17:42 +02:00
Stefano Fiorucci
c18f81283c
chore: fix deepset_sync.py for pylint + general linting improvements (#9558) 
* chore: fix deepset_sync.py for pylint

* check .github with ruff

* fix

* Update .github/utils/pyproject_to_requirements.py

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
 2025-06-27 07:54:22 +00:00
mathislucka
101e9cdc34
docs: sync code to deepset workspace (#9555) 
* docs: sync code to deepset workspace

* fix: naming

* fix: actionlint
 2025-06-27 07:51:59 +02:00
Stefano Fiorucci
bcaef53cbc
test: export HF_TOKEN env var in e2e environment (#9551) 
* try to fix e2e tests for private NER models

* explanatory comment

* extend skipif condition
 2025-06-25 15:00:28 +02:00
Michele Pangrazzi
3207a76d50
chore: Update pydoc-markdown.sh (#9547) 
* Make config path a $1 param ; Add usage in comment ; Add echo log

* Update sync command
 2025-06-24 14:01:51 +02:00
Sriniketh J
6198f0cba9
feat: adding support for torch xpu device (#9470) 
* feat: add support for torch xpu device support

* test: xpu based tests ci/cd

* test: add xpu code device support

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2025-06-17 14:15:19 +02:00
Stefano Fiorucci
12665ade14
chore: simplify Haystack Hatch scripts (#9491) 
* try unifying hatch scripts

* formatting

* simplify

* improve contributing guidelines

* fmt-check
 2025-06-06 10:43:02 +02:00
Stefano Fiorucci
d8487c4d8d
chore: make mypy run with --check-untyped-defs; fix some errors (#9447) 
* chore: make mypy run with --check-untyped-defs; fix some errors

* small fixes

* use HfPipeline

* fix license error
 2025-05-27 07:35:25 +00:00
Stefano Fiorucci
17432f710d
feat: introduce SentenceTransformersSimilarityRanker (#9415) 
* new component + tests

* soft deprecation of TransformersSimilarityRanker + reno

* add comp files to slow workflow

* Apply suggestions from code review

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>

* self.model -> self._cross_encoder

* recommend installing sentence-transformers>=4.1.0

---------

Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
 2025-05-21 10:52:46 +02:00
Stefano Fiorucci
6ad23f822f
ci: add timeout-minutes to relevant CI jobs (#9409) 2025-05-20 09:23:42 +02:00
dependabot[bot]
3e28ec207a
chore(deps): bump fossas/fossa-action from 1.6.0 to 1.7.0 (#9392) 
Bumps [fossas/fossa-action](https://github.com/fossas/fossa-action) from 1.6.0 to 1.7.0.
- [Release notes](https://github.com/fossas/fossa-action/releases)
- [Commits](https://github.com/fossas/fossa-action/compare/v1.6.0...v1.7.0)

---
updated-dependencies:
- dependency-name: fossas/fossa-action
  dependency-version: 1.7.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-05-16 11:22:30 +02:00
Amna Mubashar
da5fc0f16c
Fix template (#9377) 2025-05-12 15:48:26 +02:00
Stefano Fiorucci
f8eead3adc
ci: disable caching on Windows tests (#9318) 
* ci: try skipping cache on Windows

* trigger

* remove trigger

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2025-04-29 18:21:34 +02:00
Stefano Fiorucci
38c39a49de
test: review integration tests (#9306) 
* AzureOCR: convert integration test to unit test and simplify

* clean up HuggingFaceAPITextEmbedder

* clean up LinkContentFetcher

* simplify HuggingFaceLocalGenerator

* clean up OpenAIGenerator

* OpenAIChatGenerator

* SentenceTransformersDiversityRanker

* TransformersSimilarityRanker

* ChatMessage: rm outdated tests

* fail fast false

* typo
 2025-04-25 09:07:57 +02:00
Stefano Fiorucci
e3d4e21237
test: mark more tests as slow (#9296) 
* test: mark tests as slow

* alphabetical order; install xet

* revert pyproject

* Trigger Build

* simplify tests as suggested

* add comment to workflow
 2025-04-24 10:25:13 +02:00
Stefano Fiorucci
9ae7da8df3
test: workflow for slow/unstable integration tests (#9267) 
* workflow for slow integration tests

* try changing skipper

* Trigger Build

* better names

* fix

* mv tika to slow

* try skipping slow workflow

* retry paths-ignore

* remove skipper

* Revert "remove skipper"

This reverts commit 302ed2f07f36b33fa61fde0843b5590d79b98d74.

* better skipper

* retry

* Revert "retry"

This reverts commit fe5dff68f496645cc45292d74fcd8d043e868392.

* try using one workflow

* trigger

* try to see if it fails

* cosmetic changes

* improvements

* try matrix

* retry

* fix

* clean up

* simplify datadog monitoring and trigger

* send event to datadog for nightly failures

* tests should run if: manual trigger, scheduled, PR has label, release branch, or relevant files changed

* clarify slow marker

* improve comments

* labels
 2025-04-23 10:36:44 +02:00
Stefano Fiorucci
4279d7e149
ci: stop rerunning tests on PRs ready for review (#9254) 2025-04-17 10:10:45 +02:00
Stefano Fiorucci
e5dc4ef94d
ci: stop caching Hatch python environment on Ubuntu (#9249) 
* try setting hatch variables

* debug steps

* stop caching on ubuntu

* remove debug steps

* cache should work for mac and wind

* bring back cache and log verbose; temp skip win and mac

* try

* stop caching Hatch python environment for Ubuntu

* remove trigger
 2025-04-16 18:05:12 +02:00
Stefano Fiorucci
f478d9145f
fix: make mypy run + some type fixes on Agent (#9250) 
* make lint work + some mypy fixes on Agent

* Simple solution

* Remove unused import

---------

Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
 2025-04-16 17:00:51 +02:00
Stefano Fiorucci
36bbd27e8d
ci: update hatch to 1.14.1 (#9245) 
* try updating hatch to 1.14.1

* trigger tests

* better trigger

* remove trigger
 2025-04-16 11:25:57 +02:00
Stefano Fiorucci
407984f4d5
chore: update label name for stalebot (#9228) 2025-04-14 09:26:57 +02:00
Stefano Fiorucci
8bf41a8510
test: create e2e environment; stop testing spacy in unit tests (#9212) 
* ci: create e2e environment; stop testing spacy in unit tests

* try fix

* fix yml

* exclude test python files

* self-referential environment

* do not use self-referential environment
 2025-04-11 10:28:53 +00:00
Stefano Fiorucci
08fa1db98a
ci: fix lint job (#9217) 
* fix lint job

* remove trigger
 2025-04-11 11:01:03 +02:00
Stefano Fiorucci
77898762a6
ci: make lint workflow check pyproject.toml (#9199) 
* ci: make lint workflow rune when pyproject is modified

* remove trigger
 2025-04-09 12:05:24 +02:00
Stefano Fiorucci
cf97ba2a68
ci: review required jobs (#9124) 
* ci: review required jobs

* retrigger

* rm trigger
 2025-03-27 18:12:27 +01:00
Stefano Fiorucci
06512a8696
chore: skip typing_extensions license check (#9120) 
* chore: skip typing_extensions license check

* unrelated: remove unused imports to fix linting
 2025-03-27 11:59:58 +00:00
Stefano Fiorucci
593ca87645
ci: run mypy across the codebase (#9103) 
* ci: run mypy across the codebase

* trigger

* rm trigger

* actually rm trigger

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2025-03-25 12:14:40 +01:00
dependabot[bot]
3e435439d9
build(deps): bump fossas/fossa-action from 1.5.0 to 1.6.0 (#9089) 
Bumps [fossas/fossa-action](https://github.com/fossas/fossa-action) from 1.5.0 to 1.6.0.
- [Release notes](https://github.com/fossas/fossa-action/releases)
- [Commits](https://github.com/fossas/fossa-action/compare/v1.5.0...v1.6.0)

---
updated-dependencies:
- dependency-name: fossas/fossa-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-03-21 18:02:57 +01:00
dependabot[bot]
75a9441e84
build(deps): bump tj-actions/changed-files from 45 to 46 (#9049) 
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 45 to 46.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v45...v46)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-03-18 11:10:10 +01:00
Stefano Fiorucci
40798bc4f2
ci: improve release notes format checking (#8984) 
* chore: fix invalid release note

* try improving relnote linting

* add relnotes path

* fix bad release note

* improve reno config
 2025-03-05 19:07:46 +01:00
Stefano Fiorucci
3339097e99
ci: refactor job to check imports (#8892) 
* refactor

* Trigger CI

* run tests if this file changes

* show failure

* revert

* rm duplicate subdir and explain in comment
 2025-02-21 11:37:41 +01:00
Julian Risch
92c87805b8 Revert "build(deps): bump docker/bake-action from 5 to 6 (#8685)" 
This reverts commit 687f7593c705271dd4225ded9f1cdf6e00efca3f.
 2025-02-20 10:20:06 +01:00
dependabot[bot]
687f7593c7
build(deps): bump docker/bake-action from 5 to 6 (#8685) 
* build(deps): bump docker/bake-action from 5 to 6

Bumps [docker/bake-action](https://github.com/docker/bake-action) from 5 to 6.
- [Release notes](https://github.com/docker/bake-action/releases)
- [Commits](https://github.com/docker/bake-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/bake-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Remove checkout step

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2025-02-20 09:37:44 +01:00
dependabot[bot]
f1679f1dca
build(deps): bump fossas/fossa-action from 1.4.0 to 1.5.0 (#8771) 
Bumps [fossas/fossa-action](https://github.com/fossas/fossa-action) from 1.4.0 to 1.5.0.
- [Release notes](https://github.com/fossas/fossa-action/releases)
- [Commits](https://github.com/fossas/fossa-action/compare/v1.4.0...v1.5.0)

---
updated-dependencies:
- dependency-name: fossas/fossa-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2025-02-04 15:01:54 +01:00
Michele Pangrazzi
c192488bf6
Named entity extractor private models (#8658) 
* add 'token' support to NamedEntityExtractor to enable using private models on HF backend

* fix existing error message format

* add release note

* add HF_API_TOKEN to e2e workflow

* add informative comment

* Updated to_dict / from_dict to handle 'token' correctly ; Added tests

* Fix lint

* Revert unwanted change
 2024-12-20 11:15:55 +01:00