2539 Commits

Author SHA1 Message Date
Silvano Cerza
9398183447
Simplify PromptNode generation_kwargs tests (#4975) 2023-05-22 14:28:08 +02:00
Vladimir Blagojevic
361cb1d240
fix: Remove streaming LLM tracking; they are all streaming now (#4944)
* Remove streaming LLM tracking; they are all streaming now

* PR feedback
2023-05-22 12:33:47 +02:00
Fanli Lin
cd2ea4bc91
feat: enable passing generation_kwargs to the PromptNode in pipeline.run() (#4832)
* add generation_kwargs

* add documentation

* enable max_new_tokens  customization

* add code formatting

* add unit test

* fix formatting

* test with black

* add a new unit test

* remove doc and update tests

* unpack generation_kwargs

* ix comment

* update unit test

* remove generation_kwargs

* not pass `generation_kwargs`

* update tests

* add max_length

* fix formatting

* revert

* reformatting
2023-05-22 11:45:06 +02:00
Massimiliano Pippi
8228081e7a
chore: leftovers from removing knowledge graph support (#4974)
* leftovers from removing knowledge graph support

* more leftovers
2023-05-22 10:03:51 +02:00
bogdankostic
5c576df335
fix: Fix necessary extra for MarkdownConverter (#4947) 2023-05-22 09:51:09 +02:00
Massimiliano Pippi
917cea8af4
docs: Revamp the README file (#4971)
* revamp readme

* center the table

* more badges and cosmetics
2023-05-22 09:30:36 +02:00
Massimiliano Pippi
c6ea542b57
chore: remove BaseKnowledgeGraph (#4953)
* remove BaseKnowledgeGraph

* fix pylint
2023-05-21 10:42:02 +02:00
Tuana Çelik
5321d91f97
Update README.md (#4966)
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2023-05-19 17:43:46 +02:00
Silvano Cerza
f235d30af8
Add workflow name to Datadog event (#4968) 2023-05-19 17:42:33 +02:00
Tuana Çelik
b482f1d875
Update README.md (#4967)
Adding link to bricky from Lars at Lego
2023-05-19 17:41:20 +02:00
Silvano Cerza
ce4cf3bc55
Add workflow id to Datadog event tags (#4965) 2023-05-19 16:52:39 +02:00
Silvano Cerza
58bb5f09e4
Standardize workflows file names (#4964) 2023-05-19 16:41:56 +02:00
Massimiliano Pippi
4974bf7ab3
chore: remove deprecated MilvusDocumentStore (#4951)
* remove deprecated MilvusDocumentStore

* remove leftovers

* fix pylint
2023-05-19 16:37:38 +02:00
Tuana Çelik
b22cfb8206
Update README.md (#4963) 2023-05-19 16:26:12 +02:00
Massimiliano Pippi
85254fe9f6
leftover from merge conflict (#4962) 2023-05-19 16:10:26 +02:00
Silvano Cerza
d5cc6ff9a9
ci: Remove legacy tests (#4961)
* Remove legacy tests

* Remove unecessary env vars
2023-05-19 15:49:07 +02:00
Silvano Cerza
69bae2a3d6
Set calculator shell explicitly to handle Windows runs (#4960) 2023-05-19 15:15:18 +02:00
Silvano Cerza
dd9245531a
Add Datadog event send in examples tests workflow (#4959) 2023-05-19 15:15:10 +02:00
Silvano Cerza
2d76237508
Fix step failing to calculate Datadog event type (#4958) 2023-05-19 15:03:09 +02:00
Silvano Cerza
21ca24f70b
Send tests outcomes to Datadog instead of sending message to Slack (#4957) 2023-05-19 14:45:36 +02:00
Vladimir Blagojevic
eb9d14faeb
fix: Adjust tool pattern to support multi-line inputs (#4801)
* Add support for multi line tool input

* Fix failing agent test, additional test_tools_manager.py tests

* Allow empty tool input, add more tests

* More unit tests

* String formatting

* Small str fix
2023-05-18 16:39:31 +02:00
Massimiliano Pippi
58acef77c4
avoid importing the weaviate client directly (#4945) 2023-05-18 16:08:53 +02:00
Malte Pietsch
1f3debfd74
Add retrieval augmentation demo (#4938) 2023-05-18 16:06:32 +02:00
ZanSara
123ee55a5c
docstring (#4950) 2023-05-18 16:00:02 +02:00
Vladimir Blagojevic
edc328e21a
Remove instruction following warning (#4942) 2023-05-18 15:55:18 +02:00
Vladimir Blagojevic
5d7ee2e5e6
feat: Add max_tokens to BaseGenerator params (#4168)
* Add max_tokens to BaseGenerator params

* Make mypy happy

* Rebase and resolve conflicts

* Fix signature issues

* Update lg

* Add a mocked unit test method

* end-of-file-fixer corrected file

* Convert to unit test

* Mark test as integration

* make the test unit

---------

Co-authored-by: agnieszka-m <amarzec13@gmail.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2023-05-18 15:19:29 +02:00
Daria Fokina
401520b1d2
web.py docstring update (#4921)
Corrected spelling and added GoogleAPI
2023-05-18 14:20:51 +02:00
Massimiliano Pippi
df55ec5e61
Pin Weaviate client (#4952) 2023-05-18 12:22:16 +02:00
Shukri
ad162f2e65
feat: Support authentication using AuthBearerToken and AuthClientCredentials in Weaviate (#4028)
* refactor: make the scope param configurable

the scope parameter is used when authenticating using
AuthClientPassword and AuthClientCredentials

* feat: add support for AuthClientCredentials

add support for authenticating using the OIDC Client Credentials
authentication flow

* feat: add support for AuthBearerToken

Add support for authenticating using OIDC and bearer tokens

* Update lg

* refactor how client is built

Signed-off-by: hsm207 <hsm207@users.noreply.github.com>

* unit test the auth methods

Signed-off-by: hsm207 <hsm207@users.noreply.github.com>

* Update test_weaviate.py

* revert formatting change

* Fix type hints

---------

Signed-off-by: hsm207 <hsm207@users.noreply.github.com>
Co-authored-by: John Doe <johndoe@example.com>
Co-authored-by: agnieszka-m <amarzec13@gmail.com>
Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>
2023-05-18 10:17:11 +02:00
Massimiliano Pippi
3ea784464a
add test case for #4929 (#4936) 2023-05-18 09:12:03 +02:00
Julian Risch
8cfeed095d
build: Remove mmh3 dependency (#4896)
* build: Remove mmh3 dependency

* resolve circular import

* pylint

* make mmh3.py sibling of schema.py

* pylint import order

* pylint

* undo example changes

* increase coverage in modeling module

* increase coverage further

* rename new unit tests
2023-05-17 21:31:08 +02:00
bogdankostic
df46e7fadd
fix: Use AutoTokenizer instead of DPR specific tokenizer (#4898)
* Use AutoTokenizer instead of DPR specific tokenizer

* Adapt TableTextRetriever

* Adapt tests

* Adapt tests
2023-05-17 18:54:34 +02:00
Sebastian
34b7d1edb0
Small fix to PromptTemplate API docs (#4870) 2023-05-17 18:51:21 +02:00
Vladimir Blagojevic
9d52998b25
feat: Add conversational agent (#4931) 2023-05-17 15:19:09 +02:00
yuanwu2017
ca68601ec7
fix: shaper exception when retriever return 0 docs. (#4929)
* When retriever retrieves 0 documents from the documentStore,
  shaper will raise an exception.

Signed-off-by: root <root@aia-opa-clx-4017.jf.intel.com>
Co-authored-by: root <root@aia-opa-clx-4017.jf.intel.com>
2023-05-17 08:48:11 +02:00
tstadel
7625829684
fix: EvaluationResult serialization changes dataframes (#4906)
* fix nan and index values

* add test

* make test for None values after evalresult read explicit
2023-05-16 16:03:09 +02:00
Vladimir Blagojevic
37cadd702a
fix: Make sure summary memory is cumulative (#4932)
* Fix summary memory not being cummulative

* PR feedback - Julian
2023-05-16 13:35:19 +02:00
Stefano Fiorucci
6e0000732d
feat: add BLIP support in TransformersImageToText (#4912)
* add blip support

* fix typo

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2023-05-16 10:57:41 +02:00
Vladimir Blagojevic
4c9843017c
feat: Add agent memory (#4829) 2023-05-15 18:08:44 +02:00
Julian Risch
d4bbde2d9d
build: Upgrade transformers to 4.29.1 (#4886)
* Upgrade transformers to 4.29.0

* Upgrade transformers to 4.29.1
2023-05-15 17:11:17 +02:00
Ben Heckmann
099d0deb86
fix: Dynamic max_answers for SquadProcessor (fixes IndexError when max_answers is less than the number of answers in the dataset) (#4817)
* #4320 implemented dynamic max_answers for SquadProcessor, fixed IndexError when max_answers is less than the number of answers in the dataset

* #4320 added two unit tests for dataset_from_dicts testing default and manual max_answers

* apply suggestions from code review

Co-authored-by: bogdankostic <bogdankostic@web.de>

* simplify comment, fix mypy & pylint errors, fix old test

* adjust max_answers to each dataset individually

---------

Co-authored-by: bogdankostic <bogdankostic@web.de>
2023-05-15 14:34:23 +02:00
ZanSara
8fbfca9ebb
fix: Document v2 JSON serialization (#4863)
* fix json serialization

* add missing markers

* pylint

* fix decoder bug

* pylint

* add some more tests

* linting & windows

* windows

* windows

* windows paths again
2023-05-15 11:39:04 +02:00
ZanSara
bffe2d8c19
add base test class (#4908) 2023-05-15 10:36:55 +02:00
Farzad E
6eb251d1f0
fix: Support for gpt-4-32k (#4825)
* Add step to loook up tokenizers by prefix in openai_utils

* Updated tiktoken min version + openai_utils test

* Added test case for GPT-4 and Azure model naming

* Broken down tests

* Added default case

---------

Co-authored-by: ZanSara <sara.zanzottera@deepset.ai>
2023-05-12 19:02:12 +02:00
bogdankostic
179e9cea08
feat: Send pipeline config hash every 100 runs (#4884)
* Add since_last_run property

* Revert "Add since_last_run property"

This reverts commit c1c907ef58a696a97d964fb9c45fbee0c80365aa.

* Send pipeline config hash for each run

* Send event every 100 runs

* Merge branch 'main' into telemetry_since_last_run

* PR review

* Move constant
2023-05-12 18:51:26 +02:00
Vladimir Blagojevic
73380b194a
feat: Add Cohere PromptNode invocation layer (#4827)
* Add CohereInvocationLayer
---------

Co-authored-by: bogdankostic <bogdankostic@web.de>
2023-05-12 17:50:09 +02:00
Silvano Cerza
7e2b824bea
Add Datadog in Who Uses Haystack README.md section (#4894) 2023-05-12 11:37:22 +02:00
Massimiliano Pippi
428096733d
ci: add a job to vet license of direct dependencies only (#4885)
* add conversion script

* run job in CI

* typo

* invoke python

* install toml

* fix pylint error

* more exclusions

* add toml to dev dependencies

* fix exclusions list

* fix mypy and remove test clause
2023-05-12 11:20:48 +02:00
Massimiliano Pippi
d322beed6c
build: do not install 'dev' extras with 'all' (#4888)
* do not install 'dev' with 'all'

* some fixes around
2023-05-11 19:24:47 +02:00
ZanSara
618699eb52
fix: improve Document comparison (v2) (#4860)
* don't compare on content directly, use id as proxy

* stray change

* add more tests

* fix tests

* pylint

* black

* review feedback

* fix tests
2023-05-11 18:28:56 +02:00