2358 Commits

Author SHA1 Message Date
bogdankostic
19829da01b
refactor: Generate eval result in separate method (#5001)
Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2023-05-25 10:30:41 +02:00
ZanSara
44fd0cff7a
fix: fitz import switcher (#5012)
* fix pymupdf import switcher

* install pdf

* check after the import

* revert workflow change

* pylint

* pylint

* pylint again
2023-05-24 18:58:40 +02:00
Massimiliano Pippi
929b8d1fb0
ci: run Elasticsearch 8.6 in compatibility mode (#3853)
* bump ES version in CI

disable ssl

wait for service to start

set env vars

do not use choco to install ES

re-enable jobs deps

skip test on windows CI because of OOM

allocate more memory for ES

uniform ES installation and use default heap size

skip tests causing OOM

increase job timeout

restore memory limit for ES8

* Use latest elasticsearch version
2023-05-24 18:53:54 +02:00
Silvano Cerza
56d033e7e7
Add back hardcoded default templates (#4998) 2023-05-24 16:50:11 +02:00
bogdankostic
b85bc44c00
Mock request from prompt hub (#5011) 2023-05-24 12:23:49 +02:00
Julian Risch
ae9f384a97
feat: Add prompt_template to conversational agent init params (#4994) 2023-05-24 09:22:29 +02:00
Silvano Cerza
524d2cba36
Fix CohereInvocationLayer _ensure_token_limit not returning resized (#4978)
prompt
2023-05-23 17:58:01 +02:00
Massimiliano Pippi
00bee17b79
fix: remove old dependency from file-converters.yml (#4999) 2023-05-23 17:27:04 +02:00
Massimiliano Pippi
68924161df
chore: remove deprecated node PDFToTextOCRConverter (#4982)
* remove deprecated node

* remove related test
2023-05-23 16:55:54 +02:00
ZanSara
949b1b63b3
PromptHub integration in PromptNode (#4879)
* initial integration

* upgrade of prompthub

* fix get_prompt_template

* feedback

* add prompthub-py to dependencies

* tests

* mypy

* stray changes

* review feedback

* missing init

* fix test

* move logic in prompttemplate

* linting

* bugfixes

* fix unit tests

* fix cache

* simplify prompttemplate init

* remove unused function

* removing wrong params

* try remove all instances of prompt names

* more tests

* fix agent tests

* more tests

* fix tests

* pylint

* comma

* black

* fix test

* docstring

* review feedback

* review feedback

* fix mocks

* mypy

* fix mocks

* fix reference to missing templates

* feedback

* remove direct references to default template var

* tests

* Update haystack/nodes/prompt/prompt_node.py

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2023-05-23 15:22:58 +02:00
Julian Risch
9e4feb6bed
build: Remove tiktoken alternative (#4991)
* remove conditional statements from tiktoken

* remove count_openai_tokens method
2023-05-23 13:05:30 +02:00
Julian Risch
6747f1f0a6
build: Remove SPARQLWrapper and rdflib from generalimport (#4986)
Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>
2023-05-23 12:05:24 +02:00
Silvano Cerza
afc342b5fe
Pin typing_extensions to fix Pydantic issue (#4987) 2023-05-23 11:08:50 +02:00
dependabot[bot]
8a45be5733
build(deps): bump requests from 2.28.2 to 2.31.0 in /docs/pydoc (#4984)
Bumps [requests](https://github.com/psf/requests) from 2.28.2 to 2.31.0.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.28.2...v2.31.0)

---
updated-dependencies:
- dependency-name: requests
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-23 08:26:38 +02:00
github-actions[bot]
b353b22fff
Update unstable version (#4981)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-05-22 18:43:17 +02:00
Silvano Cerza
0fb47b5fda
Fix request_with_retry kwargs (#4980) 2023-05-22 18:36:00 +02:00
ZanSara
f80ae01174
LocalWhisperTranscriber (v2) (#4909)
* original component

* remove remote parts

* unit tests

* polish docstrings

* fix unit tests

* fix e2e tests

* pylint

* remove check

* review feedback

* add type: ignore

* improve tests

* test stream handling

* upgrade canals and improve tests

* pylint
2023-05-22 18:30:35 +02:00
ZanSara
516db4cb52
RemoteWhisperTranscriber (v2) (#4910)
* original-component

* stub

* fix implementation

* fix tests

* review feedback

* review feedback

* upgrade canals

* upgrade canals

* upgrade canals to fix pipeline test

* remove requests_with_retry

* feedback
2023-05-22 16:02:58 +02:00
Vladimir Blagojevic
068a967e5b
feat: HFInferenceEndpointInvocationLayer streaming support (#4819)
* HFInferenceEndpointInvocationLayer streaming support

* Small fixes

* Add unit test

* PR feedback

* Alphabetically sort params

* Convert PromptNode tests to HFInferenceEndpointInvocationLayer invoke tests

* Rewrite streaming with sseclient

* More PR updates

* Implement and test _ensure_token_limit

* Further optimize DefaultPromptHandler

* Fix CohereInvocationLayer mistypes

* PR feedback

* Break up unit tests, simplify

* Simplify unit tests even further

* PR feedback on unit test simplification

* Proper code identation under patch context manager

* More unit tests, slight adjustments

* Remove unrelated CohereInvocationLayer change

This reverts commit 82337151e8328d982f738e5da9129ff99350ea0c.

* Revert "Further optimize DefaultPromptHandler"

This reverts commit 606a761b6e3333f27df51a304cfbd1906c806e05.

* lg update

mostly full stops at the end of docstrings

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
Co-authored-by: Darja Fokina <daria.f93@gmail.com>
2023-05-22 14:45:53 +02:00
Silvano Cerza
9398183447
Simplify PromptNode generation_kwargs tests (#4975) 2023-05-22 14:28:08 +02:00
Vladimir Blagojevic
361cb1d240
fix: Remove streaming LLM tracking; they are all streaming now (#4944)
* Remove streaming LLM tracking; they are all streaming now

* PR feedback
2023-05-22 12:33:47 +02:00
Fanli Lin
cd2ea4bc91
feat: enable passing generation_kwargs to the PromptNode in pipeline.run() (#4832)
* add generation_kwargs

* add documentation

* enable max_new_tokens  customization

* add code formatting

* add unit test

* fix formatting

* test with black

* add a new unit test

* remove doc and update tests

* unpack generation_kwargs

* ix comment

* update unit test

* remove generation_kwargs

* not pass `generation_kwargs`

* update tests

* add max_length

* fix formatting

* revert

* reformatting
2023-05-22 11:45:06 +02:00
Massimiliano Pippi
8228081e7a
chore: leftovers from removing knowledge graph support (#4974)
* leftovers from removing knowledge graph support

* more leftovers
2023-05-22 10:03:51 +02:00
bogdankostic
5c576df335
fix: Fix necessary extra for MarkdownConverter (#4947) 2023-05-22 09:51:09 +02:00
Massimiliano Pippi
917cea8af4
docs: Revamp the README file (#4971)
* revamp readme

* center the table

* more badges and cosmetics
2023-05-22 09:30:36 +02:00
Massimiliano Pippi
c6ea542b57
chore: remove BaseKnowledgeGraph (#4953)
* remove BaseKnowledgeGraph

* fix pylint
2023-05-21 10:42:02 +02:00
Tuana Çelik
5321d91f97
Update README.md (#4966)
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2023-05-19 17:43:46 +02:00
Silvano Cerza
f235d30af8
Add workflow name to Datadog event (#4968) 2023-05-19 17:42:33 +02:00
Tuana Çelik
b482f1d875
Update README.md (#4967)
Adding link to bricky from Lars at Lego
2023-05-19 17:41:20 +02:00
Silvano Cerza
ce4cf3bc55
Add workflow id to Datadog event tags (#4965) 2023-05-19 16:52:39 +02:00
Silvano Cerza
58bb5f09e4
Standardize workflows file names (#4964) 2023-05-19 16:41:56 +02:00
Massimiliano Pippi
4974bf7ab3
chore: remove deprecated MilvusDocumentStore (#4951)
* remove deprecated MilvusDocumentStore

* remove leftovers

* fix pylint
2023-05-19 16:37:38 +02:00
Tuana Çelik
b22cfb8206
Update README.md (#4963) 2023-05-19 16:26:12 +02:00
Massimiliano Pippi
85254fe9f6
leftover from merge conflict (#4962) 2023-05-19 16:10:26 +02:00
Silvano Cerza
d5cc6ff9a9
ci: Remove legacy tests (#4961)
* Remove legacy tests

* Remove unecessary env vars
2023-05-19 15:49:07 +02:00
Silvano Cerza
69bae2a3d6
Set calculator shell explicitly to handle Windows runs (#4960) 2023-05-19 15:15:18 +02:00
Silvano Cerza
dd9245531a
Add Datadog event send in examples tests workflow (#4959) 2023-05-19 15:15:10 +02:00
Silvano Cerza
2d76237508
Fix step failing to calculate Datadog event type (#4958) 2023-05-19 15:03:09 +02:00
Silvano Cerza
21ca24f70b
Send tests outcomes to Datadog instead of sending message to Slack (#4957) 2023-05-19 14:45:36 +02:00
Vladimir Blagojevic
eb9d14faeb
fix: Adjust tool pattern to support multi-line inputs (#4801)
* Add support for multi line tool input

* Fix failing agent test, additional test_tools_manager.py tests

* Allow empty tool input, add more tests

* More unit tests

* String formatting

* Small str fix
2023-05-18 16:39:31 +02:00
Massimiliano Pippi
58acef77c4
avoid importing the weaviate client directly (#4945) 2023-05-18 16:08:53 +02:00
Malte Pietsch
1f3debfd74
Add retrieval augmentation demo (#4938) 2023-05-18 16:06:32 +02:00
ZanSara
123ee55a5c
docstring (#4950) 2023-05-18 16:00:02 +02:00
Vladimir Blagojevic
edc328e21a
Remove instruction following warning (#4942) 2023-05-18 15:55:18 +02:00
Vladimir Blagojevic
5d7ee2e5e6
feat: Add max_tokens to BaseGenerator params (#4168)
* Add max_tokens to BaseGenerator params

* Make mypy happy

* Rebase and resolve conflicts

* Fix signature issues

* Update lg

* Add a mocked unit test method

* end-of-file-fixer corrected file

* Convert to unit test

* Mark test as integration

* make the test unit

---------

Co-authored-by: agnieszka-m <amarzec13@gmail.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2023-05-18 15:19:29 +02:00
Daria Fokina
401520b1d2
web.py docstring update (#4921)
Corrected spelling and added GoogleAPI
2023-05-18 14:20:51 +02:00
Massimiliano Pippi
df55ec5e61
Pin Weaviate client (#4952) 2023-05-18 12:22:16 +02:00
Shukri
ad162f2e65
feat: Support authentication using AuthBearerToken and AuthClientCredentials in Weaviate (#4028)
* refactor: make the scope param configurable

the scope parameter is used when authenticating using
AuthClientPassword and AuthClientCredentials

* feat: add support for AuthClientCredentials

add support for authenticating using the OIDC Client Credentials
authentication flow

* feat: add support for AuthBearerToken

Add support for authenticating using OIDC and bearer tokens

* Update lg

* refactor how client is built

Signed-off-by: hsm207 <hsm207@users.noreply.github.com>

* unit test the auth methods

Signed-off-by: hsm207 <hsm207@users.noreply.github.com>

* Update test_weaviate.py

* revert formatting change

* Fix type hints

---------

Signed-off-by: hsm207 <hsm207@users.noreply.github.com>
Co-authored-by: John Doe <johndoe@example.com>
Co-authored-by: agnieszka-m <amarzec13@gmail.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2023-05-18 10:17:11 +02:00
Massimiliano Pippi
3ea784464a
add test case for #4929 (#4936) 2023-05-18 09:12:03 +02:00
Julian Risch
8cfeed095d
build: Remove mmh3 dependency (#4896)
* build: Remove mmh3 dependency

* resolve circular import

* pylint

* make mmh3.py sibling of schema.py

* pylint import order

* pylint

* undo example changes

* increase coverage in modeling module

* increase coverage further

* rename new unit tests
2023-05-17 21:31:08 +02:00