3803 Commits

Author SHA1 Message Date
Julian Risch
ae9f384a97
feat: Add prompt_template to conversational agent init params (#4994) 2023-05-24 09:22:29 +02:00
Silvano Cerza
524d2cba36
Fix CohereInvocationLayer _ensure_token_limit not returning resized (#4978)
prompt
2023-05-23 17:58:01 +02:00
Massimiliano Pippi
00bee17b79
fix: remove old dependency from file-converters.yml (#4999) 2023-05-23 17:27:04 +02:00
Massimiliano Pippi
68924161df
chore: remove deprecated node PDFToTextOCRConverter (#4982)
* remove deprecated node

* remove related test
2023-05-23 16:55:54 +02:00
ZanSara
949b1b63b3
PromptHub integration in PromptNode (#4879)
* initial integration

* upgrade of prompthub

* fix get_prompt_template

* feedback

* add prompthub-py to dependencies

* tests

* mypy

* stray changes

* review feedback

* missing init

* fix test

* move logic in prompttemplate

* linting

* bugfixes

* fix unit tests

* fix cache

* simplify prompttemplate init

* remove unused function

* removing wrong params

* try remove all instances of prompt names

* more tests

* fix agent tests

* more tests

* fix tests

* pylint

* comma

* black

* fix test

* docstring

* review feedback

* review feedback

* fix mocks

* mypy

* fix mocks

* fix reference to missing templates

* feedback

* remove direct references to default template var

* tests

* Update haystack/nodes/prompt/prompt_node.py

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2023-05-23 15:22:58 +02:00
Julian Risch
9e4feb6bed
build: Remove tiktoken alternative (#4991)
* remove conditional statements from tiktoken

* remove count_openai_tokens method
2023-05-23 13:05:30 +02:00
Julian Risch
6747f1f0a6
build: Remove SPARQLWrapper and rdflib from generalimport (#4986)
Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>
2023-05-23 12:05:24 +02:00
Silvano Cerza
afc342b5fe
Pin typing_extensions to fix Pydantic issue (#4987) 2023-05-23 11:08:50 +02:00
dependabot[bot]
8a45be5733
build(deps): bump requests from 2.28.2 to 2.31.0 in /docs/pydoc (#4984)
Bumps [requests](https://github.com/psf/requests) from 2.28.2 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.28.2...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-23 08:26:38 +02:00
github-actions[bot]
b353b22fff
Update unstable version (#4981)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-05-22 18:43:17 +02:00
Silvano Cerza
0fb47b5fda
Fix request_with_retry kwargs (#4980) 2023-05-22 18:36:00 +02:00
ZanSara
f80ae01174
LocalWhisperTranscriber (v2) (#4909)
* original component

* remove remote parts

* unit tests

* polish docstrings

* fix unit tests

* fix e2e tests

* pylint

* remove check

* review feedback

* add type: ignore

* improve tests

* test stream handling

* upgrade canals and improve tests

* pylint
2023-05-22 18:30:35 +02:00
ZanSara
516db4cb52
RemoteWhisperTranscriber (v2) (#4910)
* original-component

* stub

* fix implementation

* fix tests

* review feedback

* review feedback

* upgrade canals

* upgrade canals

* upgrade canals to fix pipeline test

* remove requests_with_retry

* feedback
2023-05-22 16:02:58 +02:00
Vladimir Blagojevic
068a967e5b
feat: HFInferenceEndpointInvocationLayer streaming support (#4819)
* HFInferenceEndpointInvocationLayer streaming support

* Small fixes

* Add unit test

* PR feedback

* Alphabetically sort params

* Convert PromptNode tests to HFInferenceEndpointInvocationLayer invoke tests

* Rewrite streaming with sseclient

* More PR updates

* Implement and test _ensure_token_limit

* Further optimize DefaultPromptHandler

* Fix CohereInvocationLayer mistypes

* PR feedback

* Break up unit tests, simplify

* Simplify unit tests even further

* PR feedback on unit test simplification

* Proper code identation under patch context manager

* More unit tests, slight adjustments

* Remove unrelated CohereInvocationLayer change

This reverts commit 82337151e8328d982f738e5da9129ff99350ea0c.

* Revert "Further optimize DefaultPromptHandler"

This reverts commit 606a761b6e3333f27df51a304cfbd1906c806e05.

* lg update

mostly full stops at the end of docstrings

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
Co-authored-by: Darja Fokina <daria.f93@gmail.com>
2023-05-22 14:45:53 +02:00
Silvano Cerza
9398183447
Simplify PromptNode generation_kwargs tests (#4975) 2023-05-22 14:28:08 +02:00
Vladimir Blagojevic
361cb1d240
fix: Remove streaming LLM tracking; they are all streaming now (#4944)
* Remove streaming LLM tracking; they are all streaming now

* PR feedback
2023-05-22 12:33:47 +02:00
Fanli Lin
cd2ea4bc91
feat: enable passing generation_kwargs to the PromptNode in pipeline.run() (#4832)
* add generation_kwargs

* add documentation

* enable max_new_tokens  customization

* add code formatting

* add unit test

* fix formatting

* test with black

* add a new unit test

* remove doc and update tests

* unpack generation_kwargs

* ix comment

* update unit test

* remove generation_kwargs

* not pass `generation_kwargs`

* update tests

* add max_length

* fix formatting

* revert

* reformatting
2023-05-22 11:45:06 +02:00
Massimiliano Pippi
8228081e7a
chore: leftovers from removing knowledge graph support (#4974)
* leftovers from removing knowledge graph support

* more leftovers
2023-05-22 10:03:51 +02:00
bogdankostic
5c576df335
fix: Fix necessary extra for MarkdownConverter (#4947) 2023-05-22 09:51:09 +02:00
Massimiliano Pippi
917cea8af4
docs: Revamp the README file (#4971)
* revamp readme

* center the table

* more badges and cosmetics
2023-05-22 09:30:36 +02:00
Massimiliano Pippi
c6ea542b57
chore: remove BaseKnowledgeGraph (#4953)
* remove BaseKnowledgeGraph

* fix pylint
2023-05-21 10:42:02 +02:00
Tuana Çelik
5321d91f97
Update README.md (#4966)
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2023-05-19 17:43:46 +02:00
Silvano Cerza
f235d30af8
Add workflow name to Datadog event (#4968) 2023-05-19 17:42:33 +02:00
Tuana Çelik
b482f1d875
Update README.md (#4967)
Adding link to bricky from Lars at Lego
2023-05-19 17:41:20 +02:00
Silvano Cerza
ce4cf3bc55
Add workflow id to Datadog event tags (#4965) 2023-05-19 16:52:39 +02:00
Silvano Cerza
58bb5f09e4
Standardize workflows file names (#4964) 2023-05-19 16:41:56 +02:00
Massimiliano Pippi
4974bf7ab3
chore: remove deprecated MilvusDocumentStore (#4951)
* remove deprecated MilvusDocumentStore

* remove leftovers

* fix pylint
2023-05-19 16:37:38 +02:00
Tuana Çelik
b22cfb8206
Update README.md (#4963) 2023-05-19 16:26:12 +02:00
Massimiliano Pippi
85254fe9f6
leftover from merge conflict (#4962) 2023-05-19 16:10:26 +02:00
Silvano Cerza
d5cc6ff9a9
ci: Remove legacy tests (#4961)
* Remove legacy tests

* Remove unecessary env vars
2023-05-19 15:49:07 +02:00
Silvano Cerza
69bae2a3d6
Set calculator shell explicitly to handle Windows runs (#4960) 2023-05-19 15:15:18 +02:00
Silvano Cerza
dd9245531a
Add Datadog event send in examples tests workflow (#4959) 2023-05-19 15:15:10 +02:00
Silvano Cerza
2d76237508
Fix step failing to calculate Datadog event type (#4958) 2023-05-19 15:03:09 +02:00
Silvano Cerza
21ca24f70b
Send tests outcomes to Datadog instead of sending message to Slack (#4957) 2023-05-19 14:45:36 +02:00
Vladimir Blagojevic
eb9d14faeb
fix: Adjust tool pattern to support multi-line inputs (#4801)
* Add support for multi line tool input

* Fix failing agent test, additional test_tools_manager.py tests

* Allow empty tool input, add more tests

* More unit tests

* String formatting

* Small str fix
2023-05-18 16:39:31 +02:00
Massimiliano Pippi
58acef77c4
avoid importing the weaviate client directly (#4945) 2023-05-18 16:08:53 +02:00
Malte Pietsch
1f3debfd74
Add retrieval augmentation demo (#4938) 2023-05-18 16:06:32 +02:00
ZanSara
123ee55a5c
docstring (#4950) 2023-05-18 16:00:02 +02:00
Vladimir Blagojevic
edc328e21a
Remove instruction following warning (#4942) 2023-05-18 15:55:18 +02:00
Vladimir Blagojevic
5d7ee2e5e6
feat: Add max_tokens to BaseGenerator params (#4168)
* Add max_tokens to BaseGenerator params

* Make mypy happy

* Rebase and resolve conflicts

* Fix signature issues

* Update lg

* Add a mocked unit test method

* end-of-file-fixer corrected file

* Convert to unit test

* Mark test as integration

* make the test unit

---------

Co-authored-by: agnieszka-m <amarzec13@gmail.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2023-05-18 15:19:29 +02:00
Daria Fokina
401520b1d2
web.py docstring update (#4921)
Corrected spelling and added GoogleAPI
2023-05-18 14:20:51 +02:00
Massimiliano Pippi
df55ec5e61
Pin Weaviate client (#4952) 2023-05-18 12:22:16 +02:00
Shukri
ad162f2e65
feat: Support authentication using AuthBearerToken and AuthClientCredentials in Weaviate (#4028)
* refactor: make the scope param configurable

the scope parameter is used when authenticating using
AuthClientPassword and AuthClientCredentials

* feat: add support for AuthClientCredentials

add support for authenticating using the OIDC Client Credentials
authentication flow

* feat: add support for AuthBearerToken

Add support for authenticating using OIDC and bearer tokens

* Update lg

* refactor how client is built

Signed-off-by: hsm207 <hsm207@users.noreply.github.com>

* unit test the auth methods

Signed-off-by: hsm207 <hsm207@users.noreply.github.com>

* Update test_weaviate.py

* revert formatting change

* Fix type hints

---------

Signed-off-by: hsm207 <hsm207@users.noreply.github.com>
Co-authored-by: John Doe <johndoe@example.com>
Co-authored-by: agnieszka-m <amarzec13@gmail.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2023-05-18 10:17:11 +02:00
Massimiliano Pippi
3ea784464a
add test case for #4929 (#4936) 2023-05-18 09:12:03 +02:00
Julian Risch
8cfeed095d
build: Remove mmh3 dependency (#4896)
* build: Remove mmh3 dependency

* resolve circular import

* pylint

* make mmh3.py sibling of schema.py

* pylint import order

* pylint

* undo example changes

* increase coverage in modeling module

* increase coverage further

* rename new unit tests
2023-05-17 21:31:08 +02:00
bogdankostic
df46e7fadd
fix: Use AutoTokenizer instead of DPR specific tokenizer (#4898)
* Use AutoTokenizer instead of DPR specific tokenizer

* Adapt TableTextRetriever

* Adapt tests

* Adapt tests
2023-05-17 18:54:34 +02:00
Sebastian
34b7d1edb0
Small fix to PromptTemplate API docs (#4870) 2023-05-17 18:51:21 +02:00
Vladimir Blagojevic
9d52998b25
feat: Add conversational agent (#4931) 2023-05-17 15:19:09 +02:00
yuanwu2017
ca68601ec7
fix: shaper exception when retriever return 0 docs. (#4929)
* When retriever retrieves 0 documents from the documentStore,
  shaper will raise an exception.

Signed-off-by: root <root@aia-opa-clx-4017.jf.intel.com>
Co-authored-by: root <root@aia-opa-clx-4017.jf.intel.com>
2023-05-17 08:48:11 +02:00
tstadel
7625829684
fix: EvaluationResult serialization changes dataframes (#4906)
* fix nan and index values

* add test

* make test for None values after evalresult read explicit
2023-05-16 16:03:09 +02:00