yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-11-30 08:56:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,855 Commits 103 Branches 264 Tags
Commit Graph

373 Commits

Author SHA1 Message Date
Stefano Fiorucci
abda78c122
unpin OpenAI and fix problem with mock (#7364) 2024-03-15 08:32:28 +01:00
Vladimir Blagojevic
2aae8472e7
feat: Add trust_remote_code init param to SentenceTransformer embedders (#7356) 
* Add trust_remote_code init param to SentenceTransformer embedders

* Add release note

* Go with no kwargs solution

* Update haystack/components/embedders/sentence_transformers_document_embedder.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Pydoc fix

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-03-14 11:14:04 +01:00
Yudhajit Sinha
41dbbdb3fc
feat: Add support for matching mime types using regex (#7303) 
* feat: Add support for matching mime types using regex
---------

Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
 2024-03-11 14:58:08 +01:00
Ashwin Mathur
38b3472bb2
feat: Add SentenceTransformersDiversityRanker (#7095) 
* Add Diversity Ranker

* Update tests

* Add separate suffix, prefix params for query and documents; allow empty query

* Update docstrings

* Make changes based on review

* Add additional tests

* Add test for warm up

* Update release notes

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
 2024-03-11 13:14:59 +01:00
Ashwin Mathur
8d7a58347d
fix: HuggingFaceTEITextEmbedder returning embedding of incorrect shape when used with Docker endpoint (#7319) 
* Fix HuggingFaceTEITextEmbedder

* Update haystack/components/embedders/hugging_face_tei_text_embedder.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Improve imports; Add additional tests

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-03-07 16:23:57 +01:00
Tobias Wochinger
23c65c250f
chore: migrate ExtractiveReader to use secret management (#7309) 
* chore: migrate `ExtractiveReader` to use secret management

* docs: add release notes
 2024-03-05 13:04:53 +01:00
Stefano Fiorucci
38a80b0235
fix: MetaFieldRanker - use weight if passed in the run method (#7305) 
* fix:  - use  if passed in the  method

* reno
 2024-03-05 12:13:56 +01:00
Julian Risch
c1c0cbfde4
docs: Update docs of MetaFieldRanker, TransformersSimilarityRanker (#7301) 
* docs: Update docstrings of MetaFieldRanker and TransformersSimilarityRanker

* add warm_up() call to usage example

* Apply suggestions from code review

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* show result of usage example

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-03-05 10:20:18 +01:00
Julian Risch
9a0e2e58fd
docs: Added LostInTheMiddleRanker usage example and updated docstrings (#7294) 
* docs: Added LostInTheMiddleRanker usage example

* remove to_dict test

* explain LITM in more detail
 2024-03-04 15:42:51 +01:00
Vladimir Blagojevic
0e7c41be5e
feat: Improve OpenAPIServiceToFunctions signature (#7257) 
* Convert OpenAPIServiceToFunctions run interface
---------
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
 2024-03-04 14:38:58 +01:00
Massimiliano Pippi
25a1a97be0
restore to_dict method (#7261) 2024-02-29 14:30:06 +01:00
Massimiliano Pippi
cf1e28431a
fix docstrings for the builder package (#7248) 
* fix docstrings for the builder package

* remove dead test

* Apply suggestions from code review

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* review feedback

* pylint

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-02-28 18:22:29 +01:00
Tobias Wochinger
e5f0e248b6
docs: review docstrings in haystack.components.validators (#7238) 
* chore: make private

* docs: review and normalize docstrings

* docs: fix format and unused import
 2024-02-28 17:46:30 +01:00
Tobias Wochinger
f22d49944d
docs: review and normalize haystack.components.websearch (#7236) 
* docs: review and normalize `haystack.components.websearch`

* fix: use correct type annotations

* refactor: use type from protocol

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* Revert "refactor: use type from protocol"

This reverts commit 23d6f45cd763c39b98be1bff03639a90f2a01fac.

* docs: refactor according to comments

* build: correctly pin to 4.7

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
 2024-02-28 16:43:08 +01:00
Stefano Fiorucci
7b9704a93a
docs: review Routers docstrings (#7234) 
* wip

* review routers

* small fixes

* Update haystack/components/routers/conditional_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/conditional_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/file_type_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/metadata_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/metadata_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/text_language_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/text_language_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/routers/text_language_router.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-02-28 11:26:22 +01:00
Tobias Wochinger
ac4f458e2b
docs: review and normalize haystack.components.fetchers (#7232) 
* docs: review and normalize `haystack.components.fetchers`

* docs: drop defaults
 2024-02-28 11:24:12 +01:00
Tobias Wochinger
419009b495
fix: move sensitive log to debug mode (#7230) 2024-02-28 09:45:50 +01:00
Vladimir Blagojevic
d871bbbfbd
feat: Add complex types in OpenAPI support (#7065) 
* Add complex types OpenAPI support

* Add release note
---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2024-02-27 18:11:06 +01:00
Stefano Fiorucci
e194c08316
docs: review DocumentLanguageClassifier docstrings (#7210) 
* review DocumentLanguageClassifier docstrings

* fix

* improve pydoc config
 2024-02-27 16:02:53 +01:00
Silvano Cerza
0a7dfc1b32
Revert "Add AnswerExactMatchEvaluator (#7050)" (#7075) 
This reverts commit b4011af8e9bc4ae2f72e51db254bfda69e20b651.
 2024-02-23 14:05:57 +01:00
Silvano Cerza
b4011af8e9
Add AnswerExactMatchEvaluator (#7050) 
* Add AnswerExactMatchEvaluator

* Add release notes

* Fix linting

* Update docstrings
 2024-02-23 10:37:18 +01:00
Vladimir Blagojevic
49cad21a2e
chore: Adjust json_schema.py slightly (#7055) 
* Slighly adjust json_schema.py

* Adjust test structures
 2024-02-22 14:33:07 +01:00
Vladimir Blagojevic
cb6389d7a2
feat: Improve OpenAPI integration (#7034) 
* Simplify and improve OpenAPIServiceConnector and OpenAPIServiceToFunctions, add unit tests

* Add reno note

* Add flask test dependency

* Initial PR feedback - Julian

* Remove indirection - Silvano

* Remove flask end-to-end tests

* Remove unused import

* Add mixed body unit test

* Update unit test, mock properly
 2024-02-22 14:03:50 +01:00
Silvano Cerza
8ca4bf405b
Remove all evaluator components (#7053) 2024-02-21 18:24:14 +01:00
Varun Mathur
b335b5d723
feat: Add Lost In The Middle Ranker (#6995) 
* add lost in the middle ranker

* update

* add release notes

* update release notes

* fix mypy

* Update

* fix mypy

* fix mypy [union-attr] for content.split

* remove e2e tests and negative topk param

* remove query param, validate params

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2024-02-20 19:55:41 +01:00
Ashwin Mathur
327c2d260d
feat: Add Mean Reciprocal Rank (MRR) metric to StatisticalEvaluator (#7042) 
* Add MRR Metric

* Add release notes

* Update logic
 2024-02-20 13:58:48 +01:00
Silvano Cerza
05af9c3439
test: Simplify OpenAPIServiceConnector run test (#7043) 
* Simplify OpenAPIServiceConnector run test

* Fix linting
 2024-02-20 11:54:51 +01:00
Silvano Cerza
9215882779
Add Recall Multi Hit and Single Hit metric (#7038) 2024-02-19 18:00:39 +01:00
Stefano Fiorucci
d00f171f8b
refactor!: Sentence Transformers Embedders - new devices mgmt (#7033) 
* new device mgmt for Sentence Transformers embedders

* reno
 2024-02-19 14:52:44 +01:00
Stefano Fiorucci
44b5ae291c
specify CPU device in warm_up test (#7014) 2024-02-16 13:01:57 +01:00
Stefano Fiorucci
0aa788facc
refactor!: LocalWhisperTranscriber - new devices mgmt (#7008) 
* wip

* whisper local transcriber: use new device mgmt

* better from_dict + test

* reno
 2024-02-16 11:25:53 +01:00
Silvano Cerza
a7209f6413
Mark OpenAPIServiceConnector integration test as flaky (#7007) 2024-02-15 19:33:34 +01:00
Tuana Çelik
e2cee468fc
fix: Adding api_base_url to OpenAITextEmbeder self assignments (#7004) 
* assigning api_base_url

This fix resolves issues with the MistralTextEmbedder integration

* adding base url to `to_dict` and the tests

* adding release note

* Update fix-openai-base-url-assignment-0570a494d88fe365.yaml

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-02-15 17:35:28 +01:00
Silvano Cerza
6fe1d3b595
refactor: Clean eval components (#7005) 
* Remove preprocess.py

* Rename eval components to evaluators
 2024-02-15 17:17:59 +01:00
Silvano Cerza
2b8a606cb8
refactor: Refactor StatisticalEvaluator (#6999) 
* Refactor StatisticalEvaluator

* Update StatisticalEvaluator

* Rename StatisticalMetric.from_string to from_str and change internal logic

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Fix tests

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-02-15 16:47:35 +01:00
Silvano Cerza
c82f787b41
feat: Add TextCleaner component (#6997) 
* Add TextCleaner component

* Update docstrings and simplify run logic

* Update docstrings
 2024-02-15 16:10:38 +01:00
Silvano Cerza
2a4e6a1de2
refactor: Refactor SASEvaluator (#6998) 
* Remove preprocessing from SASEvaluator and add warm_up method

* Update docstrings
 2024-02-15 16:05:43 +01:00
Vladimir Blagojevic
5a8d02064b
feat: Add JsonSchemaValidator (#6937) 
* Add JsonSchemaValidator
---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-02-15 14:07:01 +01:00
Silvano Cerza
36ab23d360
feat: Add StatisticalEvaluator component (#6982) 
* Add StatisticalEvaluator component

* Remove F1 and Exact Metric from old API

* Add release notes

* Update docstrings
 2024-02-14 16:48:03 +01:00
Silvano Cerza
9297fca520
feat: Add SASEvaluator component (#6980) 
* Add SASEvaluator component

* Add release notes

* Delete old tests

* Remove SAS metric in old API

* Avoid importing whole numpy package
 2024-02-14 16:16:22 +01:00
Vladimir Blagojevic
8d46a2883e
feat: Make system_messages optional in OpenAPIServiceToFunctions run (#6825) 
* Make system_messages optional in OpenAPIServiceToFunctions run

* Adjust unit test

* PR feedback Massi
 2024-02-14 16:04:35 +01:00
Vladimir Blagojevic
6a776e672f
Add OutputAdapter sede for custom filters (#6985) 2024-02-13 16:56:43 +01:00
Sebastian Husch Lee
ea7275955d
feat: Meta field ranker add meta_value_type (#6977) 
* Update MetaFieldRanker to parse string meta values based on meta_value_type

* Add some unit tests

* Add another unit test

* Add release notes

* Fix mypy

* Fix pylint

* Add more unit tests

* Update release notes

* Update docs

* Further improve doc strings
 2024-02-13 13:08:35 +01:00
Vladimir Blagojevic
97a0df66d2
feat: Add OutputAdapter (#6936) 
* Add OutputAdapter component
---------
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
 2024-02-13 13:03:50 +01:00
Vladimir Blagojevic
a311d82593
feat: Externalize callable serialization so it can be reused (#6979) 
* Callback (de)serialization

* Add unit tests

* Replace callback handler sede with callable sede

* Remove unused functions

---------

Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
 2024-02-13 13:00:49 +01:00
Vladimir Blagojevic
37d9de3c4e
feat: Add service_credentials to OpenAPIServiceConnector run (#6962) 
* Add service_credentials to OpenAPIServiceConnector run
* PR feedback Silvano
 2024-02-09 16:03:27 +01:00
Bijay Gurung
74683fe74d
Feat: Add FilterRetriever (#6836) 
* Add FilterRetriever draft

* Implement FilterRetriever and add tests

* Update comparison to compare whole docs instead of just contents

* Expose FilterRetriever at the retrievers level

* Update docstring (add example usage)

* Add filter_retriever in the API reference docs config

Update retriever search path to start one dir level higher

* simplify _documents_equal

* improve usage example

---------

Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
 2024-02-08 08:48:46 +01:00
Vladimir Blagojevic
9e6a2e3cf9
fix: HuggingFaceTGIGenerator gets stuck when model is not supported (#6915) 
* HuggingFaceTGIGenerator/HuggingFaceTGIChatGenerator check if model is deployed on free-tier
 2024-02-06 16:55:06 +01:00
ZanSara
1182c08daf
fix: Dont filter negative scores when using BM25Okapi and scale_score=False (#6889) 
* dont filter negatives for unscaled Okapi

* change BM25 algorithm default to BM25L

* Update haystack/document_stores/in_memory/document_store.py

* improve comment
 2024-02-06 11:07:27 +01:00
Massimiliano Pippi
7d29ddba42
chore: merge hf utils modules into one (#6921) 
* merge hf utils modules

* relnotes

* lint

* Update releasenotes/notes/merge-hf-utils-modules-5c16e04025123568.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-02-06 09:59:25 +01:00