yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-12-01 01:16:09 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,135 Commits 103 Branches 264 Tags
Commit Graph

1736 Commits

Author SHA1 Message Date
Stefano Fiorucci
c789f905bc
refactor: pass a role string to OpenAI API (#7404) 
* draft

* rm unused imports
 2024-03-22 09:36:56 +01:00
Vladimir Blagojevic
e779d43384
feat: Add streaming to HuggingFaceLocalGenerator (#7377) 
* Inital streaming impl

* Add unit tests

* Add release note
 2024-03-21 15:49:18 +01:00
Stefano Fiorucci
6e69d4f188
fix: Pipeline - disable autoshow on Jupyter (#7397) 
* try

* fix docstring

* simplify tests

* add release note
 2024-03-21 12:55:06 +01:00
Stefano Fiorucci
dbfd351da7
feat: introduce SparseEmbedding (#7382) 
* introduce SparseEmbedding

* reno

* add to pydoc config
 2024-03-19 18:04:16 +01:00
Silvano Cerza
610ad6f6b2
Add AnswerExactMatchEvaluator (#7381) 
* Add AnswerExactMatchEvaluator

* Add release notes

* Fix linting

* Update docstrings

* Update docstrings

* Remove to_dict and from_dict

* Fix linting
 2024-03-19 16:58:01 +01:00
Christopher Keibel
f69c3e5cd2
refactor: default for max_new_tokens to 512 in Hugging Face generators (#7370) 
* set default for max_new_tokens to 512 in Hugging Face generators

* add release notes

* fix tests

* remove issues from release note

---------

Co-authored-by: christopherkeibel <christopher.keibel@karakun.com>
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2024-03-19 08:47:53 +01:00
Mohit Lal
280719339c
bug: run parameter "ranking_mode" does not override init param in meta field ranker (#7375) 
* bug: run parameter ranking_mode does not override init param in metafield ranker

* Added a release note

* Used pytest.approx for comparing floating point numbers in unit test
 2024-03-19 07:53:26 +01:00
Sebastian Husch Lee
85c1e39fab
feat: Add Zero Shot Transformers Text Router (#7018) 
* Starting to add TransformersTextRouter

* First pass at a TextRouter based off of the zero shot classification model on HuggingFace

* Fix pylint

* Remove unneeded imports

* Update documentation example

* Update error message strings

* Starting to add unit tests

* Release notes

* Fix pylint

* Add tests for to dict and from dict

* Update patches in tests to be correct with respect to changes

* Doc strings and fixes

* Adding more tests

* Change name

* Adding to init

* Use Haystack logger

* Beef up docstrings

* Make example runnable

* Rename to huggingface_pipeline_kwargs

* Fix example
 2024-03-15 13:56:07 +01:00
Stefano Fiorucci
abda78c122
unpin OpenAI and fix problem with mock (#7364) 2024-03-15 08:32:28 +01:00
Vladimir Blagojevic
2aae8472e7
feat: Add trust_remote_code init param to SentenceTransformer embedders (#7356) 
* Add trust_remote_code init param to SentenceTransformer embedders

* Add release note

* Go with no kwargs solution

* Update haystack/components/embedders/sentence_transformers_document_embedder.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Pydoc fix

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-03-14 11:14:04 +01:00
Silvano Cerza
de4fca4526
ci: Skip collection of test_json_schema.py to fix CI failures (#7353) 
* Skip collection of test_json_schema.py to fix CI failures

* mock chroma instance

* revert

---------

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
 2024-03-13 16:59:26 +01:00
Yudhajit Sinha
41dbbdb3fc
feat: Add support for matching mime types using regex (#7303) 
* feat: Add support for matching mime types using regex
---------

Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
 2024-03-11 14:58:08 +01:00
Ashwin Mathur
38b3472bb2
feat: Add SentenceTransformersDiversityRanker (#7095) 
* Add Diversity Ranker

* Update tests

* Add separate suffix, prefix params for query and documents; allow empty query

* Update docstrings

* Make changes based on review

* Add additional tests

* Add test for warm up

* Update release notes

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
 2024-03-11 13:14:59 +01:00
Ashwin Mathur
8d7a58347d
fix: HuggingFaceTEITextEmbedder returning embedding of incorrect shape when used with Docker endpoint (#7319) 
* Fix HuggingFaceTEITextEmbedder

* Update haystack/components/embedders/hugging_face_tei_text_embedder.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Improve imports; Add additional tests

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-03-07 16:23:57 +01:00
Stefano Fiorucci
3dbde84a28
test: monkeypatch some env vars in Predefined Pipelines tests (#7321) 
* ci: skip some tests if the OPENAI API key is not set

* better idea: monkeypatch the env var
 2024-03-07 08:52:25 +01:00
Tobias Wochinger
23c65c250f
chore: migrate ExtractiveReader to use secret management (#7309) 
* chore: migrate `ExtractiveReader` to use secret management

* docs: add release notes
 2024-03-05 13:04:53 +01:00
Julian Risch
50ad1fa2c4
fix: Remove pipeline serialization from telemetry code (#7289) 
* remove pipeline serialization from telemetry

* simplify getting component instance from pipeline

* reno

* add unit test with non-serializable component

* generate qualified class names

* added pipeline.walk()

* fix imports

* sort Iterator import

* remove bfs

* add test for pipeline.walk() with cycles

* Apply suggestions from code review

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* raise TypeError if telemetry_data is no dict

* Update haystack/telemetry/_telemetry.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-03-05 12:45:53 +01:00
Tobias Wochinger
655d4a1a8d
test: test for missing dependencies (#7278) 
* tests: import test for missing libraries

* build: add missing dependencies

* refactor: use glob instead of tree walk

* test: extract constants + more documentation
 2024-03-05 12:14:10 +01:00
Stefano Fiorucci
38a80b0235
fix: MetaFieldRanker - use weight if passed in the run method (#7305) 
* fix:  - use  if passed in the  method

* reno
 2024-03-05 12:13:56 +01:00
Julian Risch
c1c0cbfde4
docs: Update docs of MetaFieldRanker, TransformersSimilarityRanker (#7301) 
* docs: Update docstrings of MetaFieldRanker and TransformersSimilarityRanker

* add warm_up() call to usage example

* Apply suggestions from code review

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* show result of usage example

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-03-05 10:20:18 +01:00
Julian Risch
9a0e2e58fd
docs: Added LostInTheMiddleRanker usage example and updated docstrings (#7294) 
* docs: Added LostInTheMiddleRanker usage example

* remove to_dict test

* explain LITM in more detail
 2024-03-04 15:42:51 +01:00
Vladimir Blagojevic
0e7c41be5e
feat: Improve OpenAPIServiceToFunctions signature (#7257) 
* Convert OpenAPIServiceToFunctions run interface
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
 2024-03-04 14:38:58 +01:00
Stefano Fiorucci
721691c036
replace flaky with pytest-rerunfailures (#7298) 2024-03-04 12:26:40 +01:00
Silvano Cerza
72d776c390
fix: Fix run order of variadic greedy components in Pipeline.run() (#7258) 
* Fix run order of variadic greedy components in Pipeline.run()

* Add release notes
 2024-03-01 17:39:13 +01:00
Massimiliano Pippi
34dac5f86f
Update test_pipeline.py (#7284) 2024-03-01 14:20:15 +01:00
Silvano Cerza
d6597952a2
fix: Update Component protocol to fix some type checking issues (#7270) 
* Update Component protocol to fix some type checking issues

* Add release notes

* Fix logline in test

* Fix run type definition
 2024-03-01 10:56:47 +01:00
Tobias Wochinger
69d5438d6e
tests: make 3.12 compatible (#7265) 2024-02-29 16:11:14 +01:00
Tobias Wochinger
fe0ac5c4a2
chore: enforce kwarg logging (#7207) 
* chore: add logger which eases logging of extras

* chore: start migrating to key value

* fix: import fixes

* tests: temporarily comment out breaking test

* refactor: move to kwarg based logging

* style: fix import order

* chore: implement self-review comments

* test: drop failing test

* chore: fix more import orders

* docs: add changelog

* tests: fix broken tests

* chore: fix getting the frames

* chore: add comment

* chore: cleanup

* chore: adapt remaining `%s` usages
 2024-02-29 14:31:20 +01:00
Massimiliano Pippi
25a1a97be0
restore to_dict method (#7261) 2024-02-29 14:30:06 +01:00
Massimiliano Pippi
e7809b6fea
feat: Add from_template class method to Pipeline (#7240) 
* move templating code under the core package

* make from_predefined part of the Pipeline API

* add tests

* amend release notes

* import under haystack package

* Apply suggestions from code review

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* from_predefined -> from_template

* remove template inheritance for more readability

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2024-02-29 12:23:32 +01:00
Massimiliano Pippi
cf1e28431a
fix docstrings for the builder package (#7248) 
* fix docstrings for the builder package

* remove dead test

* Apply suggestions from code review

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* review feedback

* pylint

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-02-28 18:22:29 +01:00
Tobias Wochinger
e5f0e248b6
docs: review docstrings in haystack.components.validators (#7238) 
* chore: make private

* docs: review and normalize docstrings

* docs: fix format and unused import
 2024-02-28 17:46:30 +01:00
Tobias Wochinger
f22d49944d
docs: review and normalize haystack.components.websearch (#7236) 
* docs: review and normalize `haystack.components.websearch`

* fix: use correct type annotations

* refactor: use type from protocol

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* Revert "refactor: use type from protocol"

This reverts commit 23d6f45cd763c39b98be1bff03639a90f2a01fac.

* docs: refactor according to comments

* build: correctly pin to 4.7

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
 2024-02-28 16:43:08 +01:00
Massimiliano Pippi
f812048713
remove the override feature (#7227) 2024-02-28 11:33:40 +01:00
Stefano Fiorucci
7b9704a93a
docs: review Routers docstrings (#7234) 
* wip

* review routers

* small fixes

* Update haystack/components/routers/conditional_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/conditional_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/metadata_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/metadata_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/text_language_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/text_language_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/text_language_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-02-28 11:26:22 +01:00
Tobias Wochinger
ac4f458e2b
docs: review and normalize haystack.components.fetchers (#7232) 
* docs: review and normalize `haystack.components.fetchers`

* docs: drop defaults
 2024-02-28 11:24:12 +01:00
Tobias Wochinger
419009b495
fix: move sensitive log to debug mode (#7230) 2024-02-28 09:45:50 +01:00
Tobias Wochinger
14f65d5a68
feat: check tty / jupyter availability for logging config detection (#7226) 
* feat: check tty availability for logging config detection

* chore: support jupyter + lower env variable

* chore: also use jupyter util for check

* test: set right mode
 2024-02-27 18:31:51 +01:00
Vladimir Blagojevic
d871bbbfbd
feat: Add complex types in OpenAPI support (#7065) 
* Add complex types OpenAPI support

* Add release note
---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2024-02-27 18:11:06 +01:00
Massimiliano Pippi
93acdc1d3a
refactor: Refactor pipeline templates (#7213) 
* merge template source into pipeline template

* docstrings

* rework pipeline templates using Jinja features

* amend release notes

* linting

* Apply suggestions from code review

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* specify QA is generative

* amend docstring

* review feedback

* forgot one comment

* add template source to the error message for easy debugging

* fix tests

* try

* annotate return type of build function

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2024-02-27 17:39:23 +01:00
Stefano Fiorucci
e194c08316
docs: review DocumentLanguageClassifier docstrings (#7210) 
* review DocumentLanguageClassifier docstrings

* fix

* improve pydoc config
 2024-02-27 16:02:53 +01:00
Tobias Wochinger
6fce2dbb11
feat: log trace correlation (#7206) 
* feat: implement support for trace-log correlation

* docs: add release notes
 2024-02-27 10:06:53 +01:00
Tobias Wochinger
2a591280ab
feat: implement support for structured logging (#7126) 
* feat: implement support for structured logging

* docs: add release notes

* style: add explanatory comment

* chore: test + import fixes

* tests: fix windows tests
 2024-02-27 09:15:01 +01:00
Tobias Wochinger
ba49905eff
ci: unify dependency management + hatch scripts (#7079) 
* ci: unify dependency management + hatch scripts

* ci: migrate readme sync

* build: migrate snippets

* ci: pin hatch

* ci: make Python version more explicit + quote

* ci: add scripts with parameters to hatch

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
 2024-02-26 15:40:10 +01:00
Silvano Cerza
0a7dfc1b32
Revert "Add AnswerExactMatchEvaluator (#7050)" (#7075) 
This reverts commit b4011af8e9bc4ae2f72e51db254bfda69e20b651.
 2024-02-23 14:05:57 +01:00
Tobias Wochinger
41b93108c0
feat: allow content tracing (#7066) 
* feat: implement content tracing capability

* feat: trace component input/output

* chore: extend type serialization to classes with `to_dict`

* docs: add option for content tracing

* style: remove unused import

* chore: add more tags

* style: ignore typing

* fix: fix tagging of types
 2024-02-23 11:52:54 +01:00
Silvano Cerza
b4011af8e9
Add AnswerExactMatchEvaluator (#7050) 
* Add AnswerExactMatchEvaluator

* Add release notes

* Fix linting

* Update docstrings
 2024-02-23 10:37:18 +01:00
Tobias Wochinger
bc8a48cc3c
feat: datadog tracer (#7058) 
* feat: implement datadog tracer

* feat: autoenable for ddtrace

* docs: add release notes

* ci: add missing test dependency

* chore: use forward references
 2024-02-23 09:18:32 +01:00
Vladimir Blagojevic
08e97d874e
Export (de)serialize_callable and (de)serialize_type in haystack.utils (#7069) 
* Export (de)serialize_callable and (de)serialize_type in haystack.utils

* Group imports where possible
 2024-02-22 17:07:07 +01:00
Vladimir Blagojevic
49cad21a2e
chore: Adjust json_schema.py slightly (#7055) 
* Slighly adjust json_schema.py

* Adjust test structures
 2024-02-22 14:33:07 +01:00