yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-07-07 00:51:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,803 Commits 73 Branches 243 Tags
Commit Graph

1524 Commits

Author SHA1 Message Date
Ashwin Mathur
38b3472bb2
feat: Add SentenceTransformersDiversityRanker (#7095) 
* Add Diversity Ranker

* Update tests

* Add separate suffix, prefix params for query and documents; allow empty query

* Update docstrings

* Make changes based on review

* Add additional tests

* Add test for warm up

* Update release notes

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
 2024-03-11 13:14:59 +01:00
Ashwin Mathur
8d7a58347d
fix: HuggingFaceTEITextEmbedder returning embedding of incorrect shape when used with Docker endpoint (#7319) 
* Fix HuggingFaceTEITextEmbedder

* Update haystack/components/embedders/hugging_face_tei_text_embedder.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Improve imports; Add additional tests

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-03-07 16:23:57 +01:00
Stefano Fiorucci
3dbde84a28
test: monkeypatch some env vars in Predefined Pipelines tests (#7321) 
* ci: skip some tests if the OPENAI API key is not set

* better idea: monkeypatch the env var
 2024-03-07 08:52:25 +01:00
Tobias Wochinger
23c65c250f
chore: migrate ExtractiveReader to use secret management (#7309) 
* chore: migrate `ExtractiveReader` to use secret management

* docs: add release notes
 2024-03-05 13:04:53 +01:00
Julian Risch
50ad1fa2c4
fix: Remove pipeline serialization from telemetry code (#7289) 
* remove pipeline serialization from telemetry

* simplify getting component instance from pipeline

* reno

* add unit test with non-serializable component

* generate qualified class names

* added pipeline.walk()

* fix imports

* sort Iterator import

* remove bfs

* add test for pipeline.walk() with cycles

* Apply suggestions from code review

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* raise TypeError if telemetry_data is no dict

* Update haystack/telemetry/_telemetry.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-03-05 12:45:53 +01:00
Tobias Wochinger
655d4a1a8d
test: test for missing dependencies (#7278) 
* tests: import test for missing libraries

* build: add missing dependencies

* refactor: use glob instead of tree walk

* test: extract constants + more documentation
 2024-03-05 12:14:10 +01:00
Stefano Fiorucci
38a80b0235
fix: MetaFieldRanker - use weight if passed in the run method (#7305) 
* fix:  - use  if passed in the  method

* reno
 2024-03-05 12:13:56 +01:00
Julian Risch
c1c0cbfde4
docs: Update docs of MetaFieldRanker, TransformersSimilarityRanker (#7301) 
* docs: Update docstrings of MetaFieldRanker and TransformersSimilarityRanker

* add warm_up() call to usage example

* Apply suggestions from code review

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* show result of usage example

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-03-05 10:20:18 +01:00
Julian Risch
9a0e2e58fd
docs: Added LostInTheMiddleRanker usage example and updated docstrings (#7294) 
* docs: Added LostInTheMiddleRanker usage example

* remove to_dict test

* explain LITM in more detail
 2024-03-04 15:42:51 +01:00
Vladimir Blagojevic
0e7c41be5e
feat: Improve OpenAPIServiceToFunctions signature (#7257) 
* Convert OpenAPIServiceToFunctions run interface
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
 2024-03-04 14:38:58 +01:00
Stefano Fiorucci
721691c036
replace flaky with pytest-rerunfailures (#7298) 2024-03-04 12:26:40 +01:00
Silvano Cerza
72d776c390
fix: Fix run order of variadic greedy components in Pipeline.run() (#7258) 
* Fix run order of variadic greedy components in Pipeline.run()

* Add release notes
 2024-03-01 17:39:13 +01:00
Massimiliano Pippi
34dac5f86f
Update test_pipeline.py (#7284) 2024-03-01 14:20:15 +01:00
Silvano Cerza
d6597952a2
fix: Update Component protocol to fix some type checking issues (#7270) 
* Update Component protocol to fix some type checking issues

* Add release notes

* Fix logline in test

* Fix run type definition
 2024-03-01 10:56:47 +01:00
Tobias Wochinger
69d5438d6e
tests: make 3.12 compatible (#7265) 2024-02-29 16:11:14 +01:00
Tobias Wochinger
fe0ac5c4a2
chore: enforce kwarg logging (#7207) 
* chore: add logger which eases logging of extras

* chore: start migrating to key value

* fix: import fixes

* tests: temporarily comment out breaking test

* refactor: move to kwarg based logging

* style: fix import order

* chore: implement self-review comments

* test: drop failing test

* chore: fix more import orders

* docs: add changelog

* tests: fix broken tests

* chore: fix getting the frames

* chore: add comment

* chore: cleanup

* chore: adapt remaining `%s` usages
 2024-02-29 14:31:20 +01:00
Massimiliano Pippi
25a1a97be0
restore to_dict method (#7261) 2024-02-29 14:30:06 +01:00
Massimiliano Pippi
e7809b6fea
feat: Add from_template class method to Pipeline (#7240) 
* move templating code under the core package

* make from_predefined part of the Pipeline API

* add tests

* amend release notes

* import under haystack package

* Apply suggestions from code review

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* from_predefined -> from_template

* remove template inheritance for more readability

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2024-02-29 12:23:32 +01:00
Massimiliano Pippi
cf1e28431a
fix docstrings for the builder package (#7248) 
* fix docstrings for the builder package

* remove dead test

* Apply suggestions from code review

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* review feedback

* pylint

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-02-28 18:22:29 +01:00
Tobias Wochinger
e5f0e248b6
docs: review docstrings in haystack.components.validators (#7238) 
* chore: make private

* docs: review and normalize docstrings

* docs: fix format and unused import
 2024-02-28 17:46:30 +01:00
Tobias Wochinger
f22d49944d
docs: review and normalize haystack.components.websearch (#7236) 
* docs: review and normalize `haystack.components.websearch`

* fix: use correct type annotations

* refactor: use type from protocol

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* Revert "refactor: use type from protocol"

This reverts commit 23d6f45cd763c39b98be1bff03639a90f2a01fac.

* docs: refactor according to comments

* build: correctly pin to 4.7

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
 2024-02-28 16:43:08 +01:00
Massimiliano Pippi
f812048713
remove the override feature (#7227) 2024-02-28 11:33:40 +01:00
Stefano Fiorucci
7b9704a93a
docs: review Routers docstrings (#7234) 
* wip

* review routers

* small fixes

* Update haystack/components/routers/conditional_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/conditional_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/metadata_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/metadata_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/text_language_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/text_language_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/text_language_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-02-28 11:26:22 +01:00
Tobias Wochinger
ac4f458e2b
docs: review and normalize haystack.components.fetchers (#7232) 
* docs: review and normalize `haystack.components.fetchers`

* docs: drop defaults
 2024-02-28 11:24:12 +01:00
Tobias Wochinger
419009b495
fix: move sensitive log to debug mode (#7230) 2024-02-28 09:45:50 +01:00
Tobias Wochinger
14f65d5a68
feat: check tty / jupyter availability for logging config detection (#7226) 
* feat: check tty availability for logging config detection

* chore: support jupyter + lower env variable

* chore: also use jupyter util for check

* test: set right mode
 2024-02-27 18:31:51 +01:00
Vladimir Blagojevic
d871bbbfbd
feat: Add complex types in OpenAPI support (#7065) 
* Add complex types OpenAPI support

* Add release note
---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2024-02-27 18:11:06 +01:00
Massimiliano Pippi
93acdc1d3a
refactor: Refactor pipeline templates (#7213) 
* merge template source into pipeline template

* docstrings

* rework pipeline templates using Jinja features

* amend release notes

* linting

* Apply suggestions from code review

Co-authored-by: Julian Risch <julian.risch@deepset.ai>

* specify QA is generative

* amend docstring

* review feedback

* forgot one comment

* add template source to the error message for easy debugging

* fix tests

* try

* annotate return type of build function

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2024-02-27 17:39:23 +01:00
Stefano Fiorucci
e194c08316
docs: review DocumentLanguageClassifier docstrings (#7210) 
* review DocumentLanguageClassifier docstrings

* fix

* improve pydoc config
 2024-02-27 16:02:53 +01:00
Tobias Wochinger
6fce2dbb11
feat: log trace correlation (#7206) 
* feat: implement support for trace-log correlation

* docs: add release notes
 2024-02-27 10:06:53 +01:00
Tobias Wochinger
2a591280ab
feat: implement support for structured logging (#7126) 
* feat: implement support for structured logging

* docs: add release notes

* style: add explanatory comment

* chore: test + import fixes

* tests: fix windows tests
 2024-02-27 09:15:01 +01:00
Tobias Wochinger
ba49905eff
ci: unify dependency management + hatch scripts (#7079) 
* ci: unify dependency management + hatch scripts

* ci: migrate readme sync

* build: migrate snippets

* ci: pin hatch

* ci: make Python version more explicit + quote

* ci: add scripts with parameters to hatch

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
 2024-02-26 15:40:10 +01:00
Silvano Cerza
0a7dfc1b32
Revert "Add AnswerExactMatchEvaluator (#7050)" (#7075) 
This reverts commit b4011af8e9bc4ae2f72e51db254bfda69e20b651.
 2024-02-23 14:05:57 +01:00
Tobias Wochinger
41b93108c0
feat: allow content tracing (#7066) 
* feat: implement content tracing capability

* feat: trace component input/output

* chore: extend type serialization to classes with `to_dict`

* docs: add option for content tracing

* style: remove unused import

* chore: add more tags

* style: ignore typing

* fix: fix tagging of types
 2024-02-23 11:52:54 +01:00
Silvano Cerza
b4011af8e9
Add AnswerExactMatchEvaluator (#7050) 
* Add AnswerExactMatchEvaluator

* Add release notes

* Fix linting

* Update docstrings
 2024-02-23 10:37:18 +01:00
Tobias Wochinger
bc8a48cc3c
feat: datadog tracer (#7058) 
* feat: implement datadog tracer

* feat: autoenable for ddtrace

* docs: add release notes

* ci: add missing test dependency

* chore: use forward references
 2024-02-23 09:18:32 +01:00
Vladimir Blagojevic
08e97d874e
Export (de)serialize_callable and (de)serialize_type in haystack.utils (#7069) 
* Export (de)serialize_callable and (de)serialize_type in haystack.utils

* Group imports where possible
 2024-02-22 17:07:07 +01:00
Vladimir Blagojevic
49cad21a2e
chore: Adjust json_schema.py slightly (#7055) 
* Slighly adjust json_schema.py

* Adjust test structures
 2024-02-22 14:33:07 +01:00
Tobias Wochinger
6d0d373def
feat: opentelemetry tracer (#7052) 
* feat: add opentelemetry tracer

* feat: implement auto enable for `OpenTelemetry`

* docs: add release notes

* style: fix linting

* tests: more test coverage

* refactor: fix old method call

* docs: fix duplicate `the`

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>

---------

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
 2024-02-22 14:30:58 +01:00
Vladimir Blagojevic
cb6389d7a2
feat: Improve OpenAPI integration (#7034) 
* Simplify and improve OpenAPIServiceConnector and OpenAPIServiceToFunctions, add unit tests

* Add reno note

* Add flask test dependency

* Initial PR feedback - Julian

* Remove indirection - Silvano

* Remove flask end-to-end tests

* Remove unused import

* Add mixed body unit test

* Update unit test, mock properly
 2024-02-22 14:03:50 +01:00
Tobias Wochinger
6e580e4430
feat: implement pipeline tracing (#7046) 
* feat: implement pipeline tracing

* tests: improve test setup for spying tracer

* feat: implement util for type coercion

* fix: trace a after checking pipeline output

* docs: add release notes

* docs: drop unused imports

* refactor: simplify getting raw span

* refactor: implement `ProxyTracer`
 2024-02-22 12:52:04 +01:00
Silvano Cerza
8ca4bf405b
Remove all evaluator components (#7053) 2024-02-21 18:24:14 +01:00
Massimiliano Pippi
f3be576b5c
refact: remove the concept of TemplateSource from the public API (#7051) 
* remove unused field

* hide the TemplateSource abstraction

* amend release notes

* revert
 2024-02-21 12:02:04 +01:00
Varun Mathur
b335b5d723
feat: Add Lost In The Middle Ranker (#6995) 
* add lost in the middle ranker

* update

* add release notes

* update release notes

* fix mypy

* Update

* fix mypy

* fix mypy [union-attr] for content.split

* remove e2e tests and negative topk param

* remove query param, validate params

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2024-02-20 19:55:41 +01:00
Ashwin Mathur
327c2d260d
feat: Add Mean Reciprocal Rank (MRR) metric to StatisticalEvaluator (#7042) 
* Add MRR Metric

* Add release notes

* Update logic
 2024-02-20 13:58:48 +01:00
Silvano Cerza
05af9c3439
test: Simplify OpenAPIServiceConnector run test (#7043) 
* Simplify OpenAPIServiceConnector run test

* Fix linting
 2024-02-20 11:54:51 +01:00
Silvano Cerza
9215882779
Add Recall Multi Hit and Single Hit metric (#7038) 2024-02-19 18:00:39 +01:00
Stefano Fiorucci
d00f171f8b
refactor!: Sentence Transformers Embedders - new devices mgmt (#7033) 
* new device mgmt for Sentence Transformers embedders

* reno
 2024-02-19 14:52:44 +01:00
Vladimir Blagojevic
cb01cb4207
feat: Add PipelineTemplate for ready made pipelines (#7001) 
* Initial working version

* More experiments

* Use jinja2 tempplate extensions

* Remove for_type static method

* Add ternary filter, with_kwargs

* Add indexing pipeline

* Rename with_kwargs -> with_parameters, allow cummulative invocation

* Remove ternary filter, use another approach to conditionals in default component configuration

* Massi feedback, add pydocs

* PipelineTemplateBuilder -> PipelineTemplate

* Add unit tests

* Add reno note

* Fix pylint, mypy

* PR review - Madeesh

* Add list_variables

* PR review feedback - Madeesh

* Update haystack/templates/pipelines.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Add timeout, fix pylint

* Use full import path

* Update release note

* Fix failing windows test

* Update haystack/templates/source.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/templates/source.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* PR feedback Madeesh

* Fix pylint

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-02-19 13:59:48 +01:00
Silvano Cerza
f1a6b2a78a
feat: Add is_greedy argument in @component decorator (#7016) 
* Add is_greedy argument in @component decorator

* Log warning if Component is greedy and non variadic
 2024-02-19 12:43:40 +01:00