yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-12-01 09:27:28 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,787 Commits 103 Branches 264 Tags
Commit Graph

62 Commits

Author SHA1 Message Date
Stefano Fiorucci
2bc58d2987
feat: support for tools in HuggingFaceAPIChatGenerator (#8661) 
* message conversion function

* hfapi w tools

* right test file + hf_hub version

* release note

* feedback
 2024-12-19 15:04:37 +01:00
Stefano Fiorucci
ea3602643a
feat!: new ChatMessage (#8640) 
* draft

* del HF token in tests

* adaptations

* progress

* fix type

* import sorting

* more control on deserialization

* release note

* improvements

* support name field

* fix chatpromptbuilder test

* Update chat_message.py

---------

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2024-12-17 17:02:04 +01:00
Stefano Fiorucci
f2b5f123b3
del HF token in tests (#8634) 2024-12-13 09:50:23 +01:00
Stefano Fiorucci
c8685aa141
refactor: update components to access ChatMessage.text instead of content (#8589) 
* introduce text property and deprecate content

* release note

* use chatmessage.text

* release note

* linting
 2024-11-28 10:16:07 +00:00
Silvano Cerza
3ef8c081be
fix: OpenAIChatGenerator and OpenAIGenerator crashing when streaming with usage tracking (#8558) 
* Fix OpenAIGenerator crashing with tracking usage with streaming enabled

* Fix OpenAIChatGenerator crashing with tracking usage with streaming enabled

* Add release notes

* Fix linting
 2024-11-20 10:27:22 +01:00
Sebastian Husch Lee
e45d3329a1
feat: Adding DALLE image generator (#8448) 
* First pass at adding DALLE image generator

* Add missing header

* Fix tests

* Add tests

* Fix mypy

* Make mypy happy

* More unit tests

* Adding release notes

* Add a test for run

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* Fix pylint

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/generators/openai_dalle.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2024-11-14 16:19:49 +01:00
David S. Batista
e5a80722c2
feat: adding metadata grouper component (#8512) 
* initial import

* making tests more readable; adding docstring

* adding release notes

* adding LICENSE header

* Update test/components/rankers/test_metadata_grouper.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* refactoring

* fixing docstring

* fixing types

* test docstrings

* renaming test

* handling too-many-arguments

* liting

* Update haystack/components/rankers/metadata_grouper.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* changing name

* Update haystack/components/rankers/metadata_grouper.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* Update haystack/components/rankers/metadata_grouper.py

Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>

* assiging value inside function for re-use

* improving docstring

* updating name to MetaFieldGroupingRanker

* adding to pydocs

* fixing imports

* adding output docstring

* Update haystack/components/rankers/meta_field_grouper_ranker.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update haystack/components/rankers/__init__.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update releasenotes/notes/add-metadata-grouper-21ec05fd4a307425.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update test/components/rankers/test_metadata_grouper.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* update docstring tests

* fixing imports

* rename modules for consistency

* fix pydocs

* simplification + more tests

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
 2024-11-12 16:01:53 +01:00
David S. Batista
852900d5e3
lowercase comparision (#8532) 2024-11-11 16:33:54 +01:00
Bohan Qu
2595e68050
feat: Add TTFT support in OpenAI chat generator (#8444) 
* feat: Add TTFT support in OpenAI generators

* pylint fixes

* correct disable

---------

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
 2024-10-31 16:56:17 +01:00
tstadel
d430833f8f
feat: streaming_callback as run param from HF generators (#8406) 
* feat: streaming_callback as run param from HF generators

* apply feedback

* add reno

* fix test

* fix test

* fix mypy

* fix excessive linting rule
 2024-10-29 15:32:06 +01:00
David S. Batista
3a50d35f06
feat: allow Generators to run with a system prompt defined at run time (#8423) 
* initial import

* Update haystack/components/generators/openai.py

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>

* docs: fixing

* supporting the three use cases: no system prompt, using system prompt defined at init, using system prompt defined at run time

* renaming 'run_time_system_prompt' to 'system_prompt'

* adding tests, converting methods to static

---------

Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
 2024-10-22 11:21:10 +02:00
Stefano Fiorucci
dfd339ca2d
ensure compatibility with huggingface_hub==0.26.0 (#8464) 2024-10-18 08:38:48 +00:00
Vladimir Blagojevic
09b95746a2
feat: HuggingFaceAPIChatGenerator add token usage data (#8375) 
* Ensure HuggingFaceAPIChatGenerator has token usage data

* Add reno note

* Fix release note
 2024-09-23 15:40:50 +02:00
David S. Batista
97126eb544
fix: changing default model to gpt-4o-mini on OpenAI API calls (#8360) 
* chaning default model to gpt-4o-mini

* adding release notes

* fixing some missed tests

* fixing some more missed tests

* fixing one last missed test

* fixing linting issues

* making pylint happy about an end2end test

* chaning if test to walruss operator

* fixing azure embedder from ada to text-embedding-ada-002
 2024-09-17 10:36:42 +02:00
Ulises M
145ca89a3f
feat: Expose default_headers and add kwargs for Azure Client (#8244) 
* default_headers and azure_kwargs added

* update docstrings

* dont forget about chat generator

* Remove azure_kwargs argument

---------

Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
 2024-09-10 10:29:56 +00:00
Stefano Fiorucci
e17d0c4192
chore: deprecate to_openai_format and create similar utility functions (#8146) 
* deprecate and add new specific functions

* reno
 2024-08-02 16:47:17 +02:00
Amna Mubashar
b374c528b2
Assign streaming_callback to OpenAIGenerator and OpenAIChatGenerator in run() method (#8054) 
* Add optional parameter for streaming_callback in run() method
 2024-07-24 15:49:19 +02:00
Vladimir Blagojevic
a59de1d7b3
chore: Combined main unblock (#8045) 
* Pin structlog to 24.2.0 due to unit test failures

* Remove object init parameter in huggingface_hub unit tests

* Use less restrictive structlog pin

* Add release note
 2024-07-19 10:39:10 +02:00
Nitanshu Vashistha
376336686b
feat: Configure max_retries and timeout for AzureOpenAIChatGenerator (#7988) 
* feat: Configure max_retries & timeout for AzureOpenAIChatGenerator

max_retries: if not set is read from the OPENAI_MAX_RETRIES
env variable or set to 5.

timeout: if not set is read from the OPENAI_TIMEOUT
env variable or set to 30.

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>

* Update haystack/components/generators/chat/azure.py

* Update haystack/components/generators/chat/azure.py

* Update max_retries-for-AzureOpenAIChatGenerator-9e49b4c7bec5c72b.yaml

---------

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
 2024-07-08 22:34:51 +02:00
Nitanshu Vashistha
167e886f2c
feat: Configure max_retries & timeout for AzureOpenAIGenerator (#7983) 
max_retries: if not set is read from the OPENAI_MAX_RETRIES
env variable or set to 5.

timeout: if not set is read from the OPENAI_TIMEOUT
env variable or set to 30.

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>
 2024-07-08 11:16:26 +02:00
Vladimir Blagojevic
535a281eec
feat: Add option to use HF_TOKEN as env var for authentication across all HF components (#7942) 
* Read both HF_API_TOKEN and HF_TOKEN env vars in all HF related components

* Add reno note

* Test fixes

* More test updates

* More test updates
 2024-06-27 10:31:58 +02:00
Amna Mubashar
866e6c8fc2
Add the missing parameter for serialization (#7929) 
* Add the missing parameter for serialization

* Updated test

---------

Co-authored-by: Amna Mubashar <amna.mubashar@Amnas-MBP.fritz.box>
 2024-06-26 11:07:00 +02:00
Stefano Fiorucci
c59ad95f42
chore: remove deprecated TGI generators (#7908) 
* remove deprecated TGI generators

* rm unused import
 2024-06-21 11:15:13 +02:00
Massimiliano Pippi
0ceeb733ba
chore: make warm_up() usage consistent (#7752) 
* make  usage consistent

* fix error type

* release notes

* pylint fix

* change of plan

* revert

* fix test

* revert

* fix HF tests

* Apply suggestions from code review

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* fix formatting

* reformat

* fix regex match with the new error message

* fix integration test

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-05-29 10:54:21 +02:00
Massimiliano Pippi
e3dccf4406
add timeout to AzureOpenAIGenerator (#7724) 
* add timeout to AzureOpenAIGenerator

* add to chat also

* Update azure-openai-generator-timeout-c39ecd6d4b0cdb4b.yaml
 2024-05-23 16:28:24 +02:00
Carlos Fernández
686a4999cf
feat: widen support of env vars in OpenAI components (#7653) 
* add enviroment variables to the _enviroment.py file

* add support for two of the three variables

* Add support for 'OPENAI_TIMEOUT' and 'OPENAI_MAX_RETRIES' on OpenAIDocument Ebedder.

* Replicate support for env vars in OpenAITextEmbedder.

* Add support for env vars in OpenAIGenerator..

* Add support for env vars in OpenAIChatGenerator.

* add docstrings and reno

* add params to __init__ in OpenAIDocumentEmbedder

* add params to __init__ in OpenAITextEmbedder

* make fully functional implementation of env vars and unit tests

* update reno

* Update haystack/components/embedders/openai_text_embedder.py

* reverse changes to telemetry/_enviroment.py

* Update haystack/components/embedders/openai_text_embedder.py

---------

Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
 2024-05-15 21:58:41 +00:00
David S. Batista
96b9d3e32a
fix: Adding missing component decorator to AzureOpenAIGenerator (#7698) 
* initial import

* adding release notes

* tests avoiding I/O operations

* Update fix-azure-generators-serialization-18fcdc9cbcb3732e.yaml
 2024-05-15 10:00:38 +02:00
Massimiliano Pippi
10c675d534
chore: add license header to all modules (#7675) 
* add license header to modules
* check license header at linting time
 2024-05-09 13:40:36 +00:00
Stefano Fiorucci
7c9532b200
fix broken serialization of HFAPI components (#7661) 2024-05-08 17:14:37 +02:00
Vladimir Blagojevic
5f813373eb
chore: Update huggingface_hub classes used after library upgrade (#7631) 
* Update huggingface_hub classes used after library upgrade

* Fix chat tests

* Update lazy import guard and other references to huggingface_hub>=0.23.0

* In huggingface_hub 0.23.0 TextGenerationOutput property details is now optional

* More fixes

* Add reno note
 2024-05-03 10:14:54 +02:00
Stefano Fiorucci
081757c6b9
test: replace mistral-7b with zephyr-7b-beta in tests (#7576) 
* replace mistral-7b with gemma-2b-it in tests

* rm wrong comment

* change model
 2024-04-23 13:56:07 +02:00
Massimiliano Pippi
3a80c866c9
fix: do not use reserved attributes in the logger (#7545) 
* avoid using reserved keywords in the logger

* make the tests independent from the log level

* relnotes
 2024-04-12 14:07:18 +00:00
Stefano Fiorucci
39be515ba6
skip HF integrations tests if running from fork (#7517) 2024-04-09 17:47:13 +02:00
Stefano Fiorucci
0dbb98c0a0
feat: HuggingFaceAPIChatGenerator (#7480) 
* draft

* docstrings and more tests

* deprecation; reno

* pydoc config

* better error messages

* wip

* add test

* better docstrings

* deprecation; reno

* pylint

* typo

* rm unneeded else

* rm unneeded else

* fixes from feedback

* docstring showing the enum

* improve docstring

* make params mandatory

* Apply suggestions from code review

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* document enum

* Update haystack/utils/hf.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* mandatory params

* fix test

* fix test

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-04-05 18:48:34 +02:00
Stefano Fiorucci
1d083861ff
feat: HuggingFaceAPIGenerator (#7464) 
* draft

* docstrings and more tests

* deprecation; reno

* pydoc config

* better error messages

* rm unneeded else

* make params mandatory

* Apply suggestions from code review

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* document enum

* Update haystack/utils/hf.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* fix test

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
 2024-04-05 18:48:13 +02:00
Silvano Cerza
ff269db12d
Fix unit tests failing if HF_API_TOKEN is set (#7491) 2024-04-05 18:05:43 +02:00
Stefano Fiorucci
e26ee0f1db
refactor!: make TGI generators compatible with huggingface_hub>=0.22.0 (#7425) 
* progress

* progress

* better lazy imports

* fixes

* reno
 2024-03-26 16:10:06 +01:00
David S. Batista
fcd48d662c
test: HuggingFaceLocalGenerator test stopwords (#7416) 
* initial import

* Update test/components/generators/test_hugging_face_local_generator.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* attending PR comments

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-03-26 12:39:02 +01:00
Vladimir Blagojevic
e779d43384
feat: Add streaming to HuggingFaceLocalGenerator (#7377) 
* Inital streaming impl

* Add unit tests

* Add release note
 2024-03-21 15:49:18 +01:00
Christopher Keibel
f69c3e5cd2
refactor: default for max_new_tokens to 512 in Hugging Face generators (#7370) 
* set default for max_new_tokens to 512 in Hugging Face generators

* add release notes

* fix tests

* remove issues from release note

---------

Co-authored-by: christopherkeibel <christopher.keibel@karakun.com>
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
 2024-03-19 08:47:53 +01:00
Sebastian Husch Lee
85c1e39fab
feat: Add Zero Shot Transformers Text Router (#7018) 
* Starting to add TransformersTextRouter

* First pass at a TextRouter based off of the zero shot classification model on HuggingFace

* Fix pylint

* Remove unneeded imports

* Update documentation example

* Update error message strings

* Starting to add unit tests

* Release notes

* Fix pylint

* Add tests for to dict and from dict

* Update patches in tests to be correct with respect to changes

* Doc strings and fixes

* Adding more tests

* Change name

* Adding to init

* Use Haystack logger

* Beef up docstrings

* Make example runnable

* Rename to huggingface_pipeline_kwargs

* Fix example
 2024-03-15 13:56:07 +01:00
Stefano Fiorucci
abda78c122
unpin OpenAI and fix problem with mock (#7364) 2024-03-15 08:32:28 +01:00
Vladimir Blagojevic
a311d82593
feat: Externalize callable serialization so it can be reused (#6979) 
* Callback (de)serialization

* Add unit tests

* Replace callback handler sede with callable sede

* Remove unused functions

---------

Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
 2024-02-13 13:00:49 +01:00
Vladimir Blagojevic
9e6a2e3cf9
fix: HuggingFaceTGIGenerator gets stuck when model is not supported (#6915) 
* HuggingFaceTGIGenerator/HuggingFaceTGIChatGenerator check if model is deployed on free-tier
 2024-02-06 16:55:06 +01:00
Massimiliano Pippi
7d29ddba42
chore: merge hf utils modules into one (#6921) 
* merge hf utils modules

* relnotes

* lint

* Update releasenotes/notes/merge-hf-utils-modules-5c16e04025123568.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
 2024-02-06 09:59:25 +01:00
Madeesh Kannan
27d1af3068
feat!: Use Secret for passing authentication secrets to components (#6887) 
* feat!: Use `Secret` for passing authentication secrets to components

* Add comment to clarify type ignore
 2024-02-05 13:17:01 +01:00
Massimiliano Pippi
acf4cd502f
refact: Rename helper function (#6831) 
* change function name

* add api docs

* release notes
 2024-01-26 16:00:02 +01:00
Vladimir Blagojevic
fea1428e84
feat: Add HuggingFaceLocalChatGenerator (#6751) 2024-01-18 15:53:12 +01:00
Madeesh Kannan
7376838922
feat!: Framework-agnostic device management (#6748) 
* feat: Framework-agnostic device management

* Add release note

* Linting

* Fix test

* Add `first_device` property, expand release notes, validate `ComponentDevice` state
 2024-01-17 10:41:34 +01:00
Vladimir Blagojevic
8cafff0645
refactor: Extract HF stop words handling in hf_utils.py (#6745) 
* Move StopWordsCriteria to hf_utils.py

* Raise ValueError for invalid StopWordsCriteria tokenizer

* StopWordsCriteria, make sure padding token exists

* Use proper torch types

* Update unit tests
 2024-01-15 17:42:29 +01:00