yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2026-01-05 11:38:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,777 Commits 101 Branches 267 Tags
Commit Graph

355 Commits

Author SHA1 Message Date
Massimiliano Pippi
f019896335
ci: Generate release notes in a Github workflow (#6211) 
* first try

* Update config.yaml

* Update github_release.yml

* set the rc0 tag more explicitly
 2023-11-08 12:29:37 +01:00
Massimiliano Pippi
58e357148e
ci: tag when branching off for a release (#6206) 
* tag when branching off

* change minor bump workflow

* Update .github/workflows/minor_version_release.yml

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* Update minor_version_release.yml

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
 2023-11-08 11:06:45 +01:00
Ashwin Mathur
6bf0b9dc7c
feat: Add MarkdownToTextDocument (v2) (#6159) 
* Add MarkdownToTextDocument

* Add release notes

* Update GitHub workflows

* Update GitHub workflows

* Refactor code with minimal dependencies

* Update docstrings

* Apply suggestions from code review

Co-authored-by: Daria Fokina <daria.f93@gmail.com>

* Update document with content and meta for backward compatibility

* Refactor Document Class for Backward Compatibility

Co-authored-by: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com>

* Update tests

* Improve test assertions

---------

Co-authored-by: Daria Fokina <daria.f93@gmail.com>
Co-authored-by: Stefano Fiorucci <44616784+anakin87@users.noreply.github.com>
 2023-10-31 18:28:13 +01:00
Massimiliano Pippi
209e349be3
do not run preview tests twice (#6204) 2023-10-31 13:13:32 +01:00
Nripesh Niketan
708d33a657
feat: add apple silicon GPU acceleration (#6151) 
* feat: add apple silicon GPU acceleration

* add release notes

* small fix

* Update utils.py

* Update utils.py

* ci fix mps

* Revert "ci fix mps"

This reverts commit 783ae503940d9ff8270a970a321549fb9e69dce7.

* mps fix

* Update experiment_tracking.py

* try removing upper watermark limit

* disable mps CI

* Use xl runner

* initialise env

* small fix

* black linting

---------

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
 2023-10-30 11:26:46 +01:00
dependabot[bot]
8a2a3c9a3f
build(deps): bump tj-actions/changed-files from 39 to 40 (#6175) 
Bumps [tj-actions/changed-files](https://github.com/tj-actions/changed-files) from 39 to 40.
- [Release notes](https://github.com/tj-actions/changed-files/releases)
- [Changelog](https://github.com/tj-actions/changed-files/blob/main/HISTORY.md)
- [Commits](https://github.com/tj-actions/changed-files/compare/v39...v40)

---
updated-dependencies:
- dependency-name: tj-actions/changed-files
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-27 16:58:38 +02:00
Julian Risch
db36d6277a
docs: Add readme sync for API docs 2.0 (#6173) 
* add sync docs for preview

* add example config for audio docs

* hardcode version in renderer

* use custom renderer for preview docs

* update comment and excerpt
 2023-10-27 14:53:03 +02:00
Grant Williams
1cf70d3dce
build: Upgrade transformers to the latest version 4.34.1 (#5994) 
* Upgrade transformers to the latest version 4.34.0 so that Haystack can support the new Mistral, Nougat, and other models.

* update release notes

* updated missing lazy import

* Update .github workflows imports

* bump more versions in .github workflows

* rever import sorting

* Update  to catch runtime errors to match haystack_hub changes

* add language parameter value to whisper test

* bump transformers version in linting preview workflow

* bump transformers version in linting preview workflow

* bump version to v4.34.1

* resolve mypy issue with reused variables

* install openai-whisper without dependencies

* remove audio extra, update whisper install instructions

* remove audio extra, update whisper install instructions

* keep audio extra but add version

* keep audio extra with no constraints

* remove audio extra

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2023-10-24 19:13:12 +02:00
Stefano Fiorucci
c4187eeebe
CI: make only test_preview run when preview e2e tests are changed (#6078) 
* make only test_preview workflow run when e2e tests are modified

* revert wrong changes to test_preview

* revert wrong order
 2023-10-17 10:06:39 +02:00
Stefano Fiorucci
167700de4d
CI: make only linting_preview run on preview e2e tests (#6077) 
* apply only linting_preview to preview e2e tests

* add paths to linting_skipper
 2023-10-16 18:18:17 +02:00
Silvano Cerza
c78e1a7eb3
Add a workflow to verify haystack.preview doesn't import non preview modules (#6053) 2023-10-16 09:36:45 +02:00
ZanSara
adf7e49af3
chore: review all extra (#6029) 2023-10-12 21:50:53 +02:00
Nicola Procopio
c102b152dc
fix: Run update_embeddings in examples (#6008) 
* added hybrid search example

Added an example about hybrid search for faq pipeline on covid dataset

* formatted with back formatter

* renamed document

* fixed

* fixed typos

* added test

added test for hybrid search

* fixed withespaces

* removed test for hybrid search

* fixed pylint

* commented logging

* updated hybrid search example

* release notes

* Update hybrid_search_faq_pipeline.py-815df846dca7e872.yaml

* Update hybrid_search_faq_pipeline.py

* mention hybrid search example in release notes

* reduce installed dependencies in examples test workflow

* do not install cuda dependencies

* skip models if API key not set; delete document indices

* skip models if API key not set; delete document indices

* skip models if API key not set; delete document indices

* keep roberta-base model and inference extra

* pylint

* disable pylint no-logging-basicconfig rule

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2023-10-10 16:38:52 +02:00
Vladimir Blagojevic
c05f564359
feat: Split linting preview into a separate file (#6017) 
* Split linting preview into seperate file

* Add not trigger paths in old workflow
 2023-10-10 14:54:27 +02:00
dependabot[bot]
a4beec3013
build(deps): bump aws-actions/configure-aws-credentials (#5968) 
Bumps [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases)
- [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md)
- [Commits](8c3f20df09...010d0da01d)

---
updated-dependencies:
- dependency-name: aws-actions/configure-aws-credentials
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-04 17:20:17 +02:00
dependabot[bot]
58192d35f1
build(deps): bump iterative/setup-cml from 1 to 2 (#5911) 
Bumps [iterative/setup-cml](https://github.com/iterative/setup-cml) from 1 to 2.
- [Release notes](https://github.com/iterative/setup-cml/releases)
- [Commits](https://github.com/iterative/setup-cml/compare/v1...v2)

---
updated-dependencies:
- dependency-name: iterative/setup-cml
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-03 17:39:22 +02:00
ZanSara
b844ab8e22
chore: remove matrix from Linux CI (#5955) 
* remove matrix

* workflow names
 2023-10-03 17:39:04 +02:00
Silvano Cerza
a933a42749 Fix release_notes.yml syntax 2023-10-02 13:24:08 -07:00
Zubeen
b8c3b68141
Update release_notes.yml (#5949) 
Ignoring release notes check for PRs of type doc/ci/test
 2023-10-02 22:17:55 +02:00
dependabot[bot]
69232612d0
build(deps): bump actions/checkout from 3 to 4 (#5928) 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-10-01 12:38:57 +02:00
ZanSara
81b2e83d04
feat: separate out preview tests (#5639) 
* add preview workflows

* feedback

* feedback

* use preview extra

* remove coverage and add separate e2e

* rename workflow file for consistency

* trigger ci

* undo trigger

* torch import in testing

* add deps to unit tests

* feedback

* run container instead of service

* comment

* add if statement

* fix tika version

* separate out win integration tests

* separate out all CIs

* try installing docker on macos

* exclude tika

* remove tika docker
 2023-09-29 13:16:08 +02:00
ZanSara
578f2b4bbf
feat: update canals to 0.8.1 (#5900) 
* Update canals to 0.8.1

* scale up runner
 2023-09-28 17:50:46 +02:00
Massimiliano Pippi
dfa48eece9
clean up the Slack integrations (#5908) 2023-09-28 15:49:19 +02:00
ZanSara
6665e8ec7f
Add preview extra to e2e tests (#5898) 2023-09-27 10:36:00 +02:00
bogdankostic
80192589b1
feat: Add AzureOCRDocumentConverter (2.0) (#5855) 
* Add AzureOCRDocumentConverter

* Add tests

* Add release note

* Formatting

* update docstrings

* Apply suggestions from code review

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>

* PR feedback

* PR feedback

* PR feedback

* Add secrets as environment variables

* Adapt test

* Add azure dependency to CI

* Add azure dependency to CI

---------

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2023-09-26 15:57:55 +02:00
ZanSara
6cb7d16e22
feat: preview extra (#5869) 
* copy the deps list over from haystack-ai

* fix lazyimport usage

* keep jinja and openai

* fix ci

* reno

* separate out preview unit tests

* fix import error message for tika

* tika

* add preview to all

* wrap torch

* remove comment

* unwrap openai and jinja
 2023-09-26 12:48:15 +02:00
Stefano Fiorucci
e9d34fc0e3
test: e2e tests for RAG Pipelines (#5876) 
* relax extractive reader integration tests

* force reader to CPU

* ensure integration tests reproducibility

* e2e rag tests

* move set_all_seeds to testing package

* refine rag tests

* Update e2e/preview/pipelines/test_rag_pipelines.py

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>

---------

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>
 2023-09-26 11:49:50 +02:00
bogdankostic
9a4373bf8e
feat: Add TikaDocumentConverter (2.0) (#5847) 
* Add TikaFileToDocument component

* Add tests

* Add tika service to CI

* Add release note

* Change name

* PR feedback

* Fix naming in tests

* Fix tika version in CI

* Update tests

---------

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>
 2023-09-25 11:47:21 +02:00
ZanSara
23fdef929e
chore: move GPT35Generator tests in the main test suite (#5844) 
* move tests

* fix no-test-found error from pytest

* missing self

---------

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
 2023-09-21 11:42:32 +02:00
ZanSara
28f5c4c780
fix: Whisper integration tests (#5851) 
* fix tests

* add ffmpeg

* apt update for ffmpeg

* not run on windows
 2023-09-21 00:14:07 +02:00
ZanSara
ea2a5595ca
add missing dependency (#5849) 2023-09-20 12:57:53 +02:00
bogdankostic
57d33ee6da
ci: Run preview integration tests in CI (#5843) 
* Run preview integration tests in CI

* Only install inference extra
 2023-09-20 11:54:41 +02:00
Christian Clauss
41126397d6
Revert "ci: Speed up pylint GitHub Action (#5828)" (#5832) 
This reverts commit d49c86c845ef9ba5bfc17909cd6cf456910516e1.
 2023-09-18 10:05:17 +02:00
Christian Clauss
d49c86c845
ci: Speed up pylint GitHub Action (#5828) 2023-09-16 16:30:13 +02:00
Massimiliano Pippi
c7971a809d
ci: skip mandatory release notes check when not needed (#5817) 2023-09-14 17:00:41 +02:00
Massimiliano Pippi
6fc12a2bd1
ci: run apt-get update (#5816) 
* run apt-get update

* run when changing the workflow file
 2023-09-14 16:37:42 +02:00
Silvano Cerza
c23cac3215
Try to send event to Datadog only if possible (#5795) 2023-09-13 14:10:30 +02:00
Stefano Fiorucci
784034ffc3
Revert "build(deps): bump readmeio/rdme from 8.3.1 to 8.6.6 (#5789)" (#5792) 
This reverts commit 55a2e7ab7fc16e4e311ea994b95553031711a506.
 2023-09-13 11:56:42 +02:00
dependabot[bot]
55a2e7ab7f
build(deps): bump readmeio/rdme from 8.3.1 to 8.6.6 (#5789) 
Bumps [readmeio/rdme](https://github.com/readmeio/rdme) from 8.3.1 to 8.6.6.
- [Release notes](https://github.com/readmeio/rdme/releases)
- [Changelog](https://github.com/readmeio/rdme/blob/next/CHANGELOG.md)
- [Commits](https://github.com/readmeio/rdme/compare/8.3.1...8.6.6)

---
updated-dependencies:
- dependency-name: readmeio/rdme
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-09-13 11:03:12 +02:00
Silvano Cerza
7e544d4f60
Fix license compliance workflow (#5791) 
* Formatting

* Try to send event to Datadog only if possible
 2023-09-13 10:43:06 +02:00
dependabot[bot]
e688d3dddb
build(deps): bump aws-actions/configure-aws-credentials (#5790) 
Bumps [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials) from 2.2.0 to 4.0.0.
- [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases)
- [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md)
- [Commits](5fd3084fc3...8c3f20df09)

---
updated-dependencies:
- dependency-name: aws-actions/configure-aws-credentials
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-09-13 10:25:54 +02:00
Christian Clauss
75dc60b0bb
ci: Upgrade GitHub Actions (#5787) 2023-09-13 09:58:47 +02:00
Christian Clauss
45cc40bf51
linting.yml: Upgrade GitHub Actions (#5752) 2023-09-11 20:49:20 +02:00
Vladimir Blagojevic
c5edb45c10
feat: Add SerperDevWebSearch Haystack 2.0 component (#5712) 
* Add SerperDev

* Add release note

* PR Feedback

* Simplify, remove one-liner

* Update haystack/preview/components/websearch/serper_dev.py

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>

* Update haystack/preview/components/websearch/serper_dev.py

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>

* Fix formatting

* PR feedback

* Fix tests

* Function rename

* Remove scoring, update tests

* PR feedback

* Fix return

* small adjustments

* fix tests

* add e2e test

* fix release notes

* fix tests

* fix e2e

---------

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>
 2023-09-06 17:31:42 +02:00
ZanSara
0bbc219a59
chore: enable e2e preview tests (#5730) 
* enable e2e preview tests

* fix transcriber test

* quotes

* add missing dep

* missing comma

* ffmpeg
 2023-09-06 16:48:45 +02:00
Timo Moeller
d540883469
Add api keys to CI workflows (#5722) 2023-09-05 16:21:17 +02:00
ZanSara
ce06268990
test: fix e2e test failures (#5685) 
* fix test errors

* fix pipeline yaml

* disable cache

* fix errors

* remove stray fixture
 2023-08-30 12:24:03 +02:00
ZanSara
1709be162c
auto trigger e2e workflow on PRs that affect it (#5684) 2023-08-30 10:25:47 +02:00
ZanSara
5985b6d358
chore: refactor pipeline tests for e2e testing (#5576) 
* enable pipeline filder in e2e

* merge standard pipeline tests with stanrdard pipeline batch tests

* merge summarization tests into standard pipelines tests

* Update test_standard_pipelines.py

* black
 2023-08-29 11:22:39 +02:00
Silvano Cerza
444edce126
Add workflow to trigger preview package release (#5631) 2023-08-25 17:10:28 +02:00