haystack/test/prompt/invocation_layer/test_invocation_layers.py
Vivek Silimkhan f998bf4a4f
feat: add Amazon Bedrock support (#6226)
* Add Bedrock

* Update supported models for Bedrock

* Fix supports and add extract response in Bedrock

* fix errors imports

* improve and refactor supports

* fix install

* fix mypy

* fix pylint

* fix existing tests

* Added Anthropic Bedrock

* fix tests

* fix sagemaker tests

* add default prompt handler, constructor and supports tests

* more tests

* invoke refactoring

* refactor model_kwargs

* fix mypy

* lstrip responses

* Add streaming support

* bump boto3 version

* add class docstrings, better exception names

* fix layer name

* add tests for anthropic and cohere model adapters

* update cohere params

* update ai21 args and add tests

* support cohere command light model

* add tital tests

* better class names

* support meta llama 2 model

* fix streaming support

* more future-proof model adapter selection

* fix import

* fix mypy

* fix pylint for preview

* add tests for streaming

* add release notes

* Apply suggestions from code review

Co-authored-by: Agnieszka Marzec <97166305+agnieszka-m@users.noreply.github.com>

* fix format

* fix tests after msg changes

* fix streaming for cohere

---------

Co-authored-by: tstadel <60758086+tstadel@users.noreply.github.com>
Co-authored-by: tstadel <thomas.stadelmann@deepset.ai>
Co-authored-by: Agnieszka Marzec <97166305+agnieszka-m@users.noreply.github.com>
2023-11-15 13:26:29 +01:00

20 lines
832 B
Python

import pytest
from haystack.nodes.prompt.prompt_model import PromptModelInvocationLayer
from haystack.nodes.prompt.invocation_layer import HFLocalInvocationLayer, HFInferenceEndpointInvocationLayer
@pytest.mark.unit
def test_invocation_layer_order():
"""
Checks that the huggingface invocation layer is positioned further down the list of providers
as they can time out or be slow to respond.
"""
invocation_layers = PromptModelInvocationLayer.invocation_layer_providers
assert HFLocalInvocationLayer in invocation_layers
assert HFInferenceEndpointInvocationLayer in invocation_layers
index_hf = invocation_layers.index(HFLocalInvocationLayer) + 1
index_hf_inference = invocation_layers.index(HFInferenceEndpointInvocationLayer) + 1
assert index_hf >= 7
assert index_hf_inference >= 7