3821 Commits

Author SHA1 Message Date
Amna Mubashar
499fbcc59f
Remove Multiplexer and related tests (#8020) 2024-07-16 15:39:40 +02:00
Silvano Cerza
0411cd938a
Fix bug in Pipeline.run() executing Components in a wrong and unexpected order (#8021)
* Fix bug in Pipeline.run() executing Components in a wrong and unexpected order

* Update haystack/core/pipeline/base.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-07-12 15:30:10 +00:00
Madeesh Kannan
94b806815c
refactor: Improve error messages shown during pipeline deserialization (#8016)
* refactor: Improve error messages shown during pipeline deserialization

* Add link to release notes

* Update release notes link
2024-07-12 14:47:00 +00:00
Anushree Bannadabhavi
1f05e633a9
refactor: refactor DocumentJoiner to follow enum pattern for join_mode parameter (#8010)
* refactor document joiner to follow enum pattern for join mode

* Added to_dict and from_dict
2024-07-12 11:29:44 +02:00
Silvano Cerza
0cec82e55e
refactor: Pipeline.run() (#8019)
* Move utility functions from _enqueue_next_runnable_component (#7895)

* Isolate logic to check if we're stuck in a loop

* Simplify for else

* Add missing return in docstring

* Emit warning when stuck in a loop

* Fix docstring

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Add utility function to move Components in queues

* Add function to find next Component to run

* Comment update

* Add missing break in loop

* Make _add_missing_input_defaults less error prone and add tests

* Fix tests

* Update docstring

* Simplify enqueue logic

* Remove unused _enqueue_next_runnable_component function

* Add method to find Component with lazy variadic input or all inputs with defaults

* Simplify _find_next_runnable_lazy_variadic_or_default_component

* Remove unnecessary type ignore

* Split _dequeue_components_that_received_no_input into separate functions

* Fix linting

* Simplify variadic check when running Component

* Simplify code

* Reorganize functions used by Pipeline.run

* Rename variables used in Pipeline.run() for clarity

* Add comment clarifying last_waiting_queue and before_last_waiting_queue

* Add functions to easily update waiting_queue

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-07-12 08:35:23 +00:00
David S. Batista
d02356fe7a
chore: normalise the use of importlib in getting an object from a qualified name string across the codebase (#8012)
* initial import

* cleaning up

* removing unused imports
2024-07-11 16:14:00 +02:00
Madeesh Kannan
8faa3fa465
Revert "fix: make PyPDF backward compatible (#7996)" (#8014)
This reverts commit 58b48e36eb56a896365133ab4a9d8e327989948c.
2024-07-11 13:06:08 +00:00
Ulises M
6f8834d036
feat: add and expose api_params for OpenAIGenerator in LLMEvaluator based classes (#7987)
* initial support for api_params

* add tests and reno

* resolve suggestions and add integration test

* fix mypy

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-07-11 13:14:03 +02:00
David S. Batista
ebfeb571d7
feat: add sentence window retrieval (#7997)
* initial import

* adding tests

* adding license and release notes

* adding missing release notes

* working with any type of doc store

* nit

* adding get_class_object to serialization package

* nit

* refactoring get_class_object()

* refactoring get_class_object()

* chaning type and var names

* more refactoring

* Update haystack/core/serialization.py

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>

* Update haystack/core/serialization.py

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>

* Update test/core/test_serialization.py

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>

* more refactoring

* more refactoring

* Pydoc syntax

---------

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
2024-07-10 13:13:46 +00:00
Sebastian Husch Lee
c121c86c4c
fix: Fix from_dict methods of components using HF models to work with default values (#8003)
* Fix from_dict to work if device isn't provided in init params

* Minor refactoring of from_dict for components that load HF models

* Add tests

* Update tests to test loading with all default parameters

* Add more tests

* Add release notes

* Add unit test for whisper local

* Update reno

* Add fix for ExtractiveReader

* Fix NamedEntityExtractor
2024-07-10 12:18:05 +02:00
Madeesh Kannan
f19131f13a
chore: Deprecate legacy document/metadata filters (#8004) 2024-07-09 16:18:38 +02:00
tstadel
7e35280d4f
fix: LinkContentFetcher html text encoding (#7975)
* fix: content encoding of LinkContentFetcher

* fix tests

* add reno

* only touch html
2024-07-09 15:28:49 +02:00
Sebastian Husch Lee
583eb8a293
fix: TransformersZeroShotTextRouter and TransformersTextRouter from_dict to work with default value for huggingface_pipeline_kwargs (#8002)
* Fix default value for huggingface_pipeline_kwargs

* Add reno note

* Update HuggingFaceLocalGenerator.from_dict to use the same logic as HuggingFaceLocalChatGenerator.from_dict

* Update tests slightly

* Update release note
2024-07-09 13:32:44 +02:00
Tobias Wochinger
58b48e36eb
fix: make PyPDF backward compatible (#7996)
* fix: make PyPDF backward compatible

* Add release note

---------

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
2024-07-09 10:08:37 +02:00
Nitanshu Vashistha
cd8a5b98fe
feat: Configure max_retries & timeout for AzureOpenAITextEmbedder (#7993)
max_retries: if not set is read from the OPENAI_MAX_RETRIES
env variable or set to 5.

timeout: if not set is read from the OPENAI_TIMEOUT
env variable or set to 30.

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>
2024-07-09 09:56:46 +02:00
Nitanshu Vashistha
f9d53c5ca8
feat: Configure max_retries and timeout for AzureOpenAIDocumentEmbedder (#7994)
* feat: Configure max_retries & timeout for AzureOpenAIDocumentEmbedder

max_retries: if not set is read from the OPENAI_MAX_RETRIES
env variable or set to 5.

timeout: if not set is read from the OPENAI_TIMEOUT
env variable or set to 30.

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>

* Update retries-and-timeout-for-AzureOpenAIDocumentEmbedder-006fd84204942e43.yaml

* Update haystack/components/embedders/azure_document_embedder.py

* Update haystack/components/embedders/azure_document_embedder.py

---------

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
2024-07-08 22:35:25 +02:00
Nitanshu Vashistha
376336686b
feat: Configure max_retries and timeout for AzureOpenAIChatGenerator (#7988)
* feat: Configure max_retries & timeout for AzureOpenAIChatGenerator

max_retries: if not set is read from the OPENAI_MAX_RETRIES
env variable or set to 5.

timeout: if not set is read from the OPENAI_TIMEOUT
env variable or set to 30.

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>

* Update haystack/components/generators/chat/azure.py

* Update haystack/components/generators/chat/azure.py

* Update max_retries-for-AzureOpenAIChatGenerator-9e49b4c7bec5c72b.yaml

---------

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
2024-07-08 22:34:51 +02:00
Haystack Bot
d7a7d9c1fb
Update unstable version to 2.4.0-rc0 (#7992)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-07-08 14:32:56 +02:00
Vladimir Blagojevic
b3b3f89302
feat: Add haystack-experimental dependency (#7921)
* Add haystack-experimental dependency

* Add reno note
v2.4.0-rc0
2024-07-08 14:07:15 +02:00
David S. Batista
ff75444f53
fix: adding deprecation warning ContextEvaluator (#7991)
* adding deprecation warning

* adding to release notes

* Update releasenotes/notes/add-deprecation-warning-context-relevance-937df7e807ac1a8d.yaml

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
2024-07-08 10:18:39 +00:00
Nitanshu Vashistha
167e886f2c
feat: Configure max_retries & timeout for AzureOpenAIGenerator (#7983)
max_retries: if not set is read from the OPENAI_MAX_RETRIES
env variable or set to 5.

timeout: if not set is read from the OPENAI_TIMEOUT
env variable or set to 30.

Signed-off-by: Nitanshu Vashistha <nitanshu.vzard@gmail.com>
2024-07-08 11:16:26 +02:00
Ulises M
e92a0e4beb
feat: Allow Connection of ChatGenerator to AnswerBuilder (#7897)
* initial implementation

* add support for meta and add ChatMessage tests

* explictly cast types for mypy and update reno

* leave inputs unchanged avoiding side effects

---------

Co-authored-by: Julian Risch <julian.risch@deepset.ai>
2024-07-05 08:21:53 +02:00
Vladimir Blagojevic
61de1dcc61
Fix project image and pypi url in slack configuration (#7980) 2024-07-04 17:27:28 +02:00
Silvano Cerza
06aafa18fc
Update pre-commit hooks versions (#7979) 2024-07-04 17:11:28 +02:00
Vladimir Blagojevic
0255422eb3
chore: Mark AzureOCRDocumentConverter test_run_with_pdf_file flaky (#7978)
* Disable AzureOCRDocumentConverter test_run_with_pdf_file on osx

* Mark test flaky instead

* Remove import
2024-07-04 16:36:32 +02:00
tstadel
aa46466894
fix: meta from ByteStream input for AzureOCRDocumentConverter (#7955)
* fix: meta from ByteStream input for AzureOCRDocumentConverter

* add test

* add reno

* fix test
2024-07-04 14:42:30 +02:00
Chris Pappalardo
7178aa0253
feat: add custom jinja filter handling to ConditionalRouter (#7957)
* add custom jinja filter handling to ConditionalRouter

* add release notes for custom filters

* align sede to existing patterns and update docstring example

* update sede unit test route condition to be more explicit

---------

Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
2024-07-04 10:08:12 +02:00
Nicola Procopio
cafcf51cb0
Fixed ZeroDivisionError in JoinDocuments (#7972)
* added new strategy DBRF

* fix hook

* fix typos

* added test for DBRF

* fix format

* new release note

* reformatted with black

* Update haystack/components/joiners/document_joiner.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* updated comments

* added type-hint and return type

* fix

* revert for lint problems

* fix

* fix

* fix

* fix

* another tentative

* dict out file

* only output

* fix output

* revert

* removed unused imports

* fix typing

* fixed ZeroDivisionError

* added test

* add release note

* removed try - except

* renamed test

* Update test/components/joiners/test_document_joiner.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* Update haystack/components/joiners/document_joiner.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* fix format error

* removed releasenotes/notes/release-note-9b2bc03a8a398078.yaml

* added comment

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2024-07-04 10:07:26 +02:00
Nicola Procopio
03d9057e64
Add Distribution based rank fusion mode (#7915)
* added new strategy DBRF

* fix hook

* fix typos

* added test for DBRF

* fix format

* new release note

* reformatted with black

* Update haystack/components/joiners/document_joiner.py

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>

* updated comments

* added type-hint and return type

* fix

* revert for lint problems

* fix

* fix

* fix

* fix

* another tentative

* dict out file

* only output

* fix output

* revert

* removed unused imports

* fix typing

---------

Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
2024-07-03 13:55:17 +02:00
David S. Batista
a67872e7a8
docs: correcting CONTRIBUTING hatch commands (#7963)
* fixing CONTRIBUTING hatch commands

* updating CONTRIBUTING.MD

* Update CONTRIBUTING.md

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-07-03 10:59:21 +02:00
David S. Batista
186512459d
feat: LLM-based evaluators return meta info from OpenAI (#7947)
* LLM-Evaluator returns metadata from OpenAI

* adding tests

* adding release notes

* updating test

* updating release notes

* fixing live tests

* attending PR comments

* fixing tests

* Update releasenotes/notes/adding-metadata-info-from-OpenAI-f5309af5f59bb6a7.yaml

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>

* Update llm_evaluator.py

---------

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
2024-07-02 11:31:51 +02:00
Vladimir Blagojevic
3068ea258b
Fix whisper test (#7959) 2024-07-01 10:10:19 +02:00
Silvano Cerza
a86bf963a0
ci: Fix workflows_linting.yml never running (#7941)
* Fix workflows_linting.yml never running

* Add setup-go step

* Ignore SC2102 rule
2024-06-28 11:01:52 +02:00
David S. Batista
5b9e989f9a
fix: adjusting code due to new ruff version enforcing more strict linting (#7948)
* initial import

* fixing if clause
2024-06-28 10:51:57 +02:00
David S. Batista
91f57015c0
feat : adding split_id and split_overlap to DocumentSplitter (#7933)
* wip: adding _split_overlapp

* fixing join issue for _split_overlap

* adding tests

* adding release notes

* cleaning and fixing tests

* making mypy happy

* Update haystack/components/preprocessors/document_splitter.py

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>

* adding docstrings

---------

Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com>
2024-06-27 15:07:43 +02:00
Vladimir Blagojevic
569b2a87cb
feat: Update LocalWhisperTranscriber, add tests (#7935)
* Update LocalWhisperTranscriber, add tests

* Final touches

* Update haystack/components/audio/whisper_local.py

Co-authored-by: David S. Batista <dsbatista@gmail.com>

* Fix prev commit

* Relax test for tiny model to work

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2024-06-27 12:53:41 +02:00
Stefano Fiorucci
1f7786d6dd
replace expired Discord invitation link (#7944) 2024-06-27 11:47:50 +02:00
Vladimir Blagojevic
c2ed275a2d
feat: Improve LinkContentFetcher content type handling (#7920)
* LinkContentFetcher: add more default content type handlers

* Update/add unit test

* Add reno note

* Add image content handler

* Update unit test
2024-06-27 11:45:20 +02:00
Vladimir Blagojevic
535a281eec
feat: Add option to use HF_TOKEN as env var for authentication across all HF components (#7942)
* Read both HF_API_TOKEN and HF_TOKEN env vars in all HF related components

* Add reno note

* Test fixes

* More test updates

* More test updates
2024-06-27 10:31:58 +02:00
Sebastian Husch Lee
6836079686
chore: Capitalize DOCX in DOCXToDocument converter (#7931)
* Capitalize DOCX in DOCXToDocument converter

* Update docstrings

* Update test class name

* add releease notes
2024-06-27 08:19:01 +02:00
Silvano Cerza
fd1a06d171
Disable tracing when running tests (#7934) 2024-06-26 12:32:05 +02:00
Amna Mubashar
866e6c8fc2
Add the missing parameter for serialization (#7929)
* Add the missing parameter for serialization

* Updated test

---------

Co-authored-by: Amna Mubashar <amna.mubashar@Amnas-MBP.fritz.box>
2024-06-26 11:07:00 +02:00
David S. Batista
8b9eddcd94
fix: explicitly tell ContextRelevanceEvaluator that each statement should be scored (#7904)
* initial import

* adding release notes

* adding pytest decorator for live test

* make examples more readable

* updating tests

* reverting progress_bar = False
2024-06-25 16:59:37 +02:00
dependabot[bot]
8583d8c6a6
chore(deps): bump actions/add-to-project from 1.0.1 to 1.0.2 (#7926)
Bumps [actions/add-to-project](https://github.com/actions/add-to-project) from 1.0.1 to 1.0.2.
- [Release notes](https://github.com/actions/add-to-project/releases)
- [Commits](https://github.com/actions/add-to-project/compare/v1.0.1...v1.0.2)

---
updated-dependencies:
- dependency-name: actions/add-to-project
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-25 16:27:43 +02:00
Amna Mubashar
fc011d7b04
bug: fix MRR and MAP calculations (#7841)
* bug: fix MRR and MAP calculations
2024-06-25 12:07:11 +02:00
Stefano Fiorucci
c51f8ffb86
PyPDFToDocument: remove deprecated converter_name and CONVERTERS_REGISTRY (#7910) 2024-06-21 16:52:03 +02:00
David Berenstein
08104e0042
feat: InMemoryDocumentStore serialization (#7888)
* Add: InMemoryDocumentStore serialization

* Add: additional chek to test if path exists

* Fix: failing test
2024-06-21 16:45:25 +02:00
Ulises M
9c45203a76
fix: check for None in SAS eval input (#7909)
* check for None in SAS input

* Update releasenotes/notes/check-for-None-SAS-eval-0b982ccc1491ee83.yaml

---------

Co-authored-by: David S. Batista <dsbatista@gmail.com>
2024-06-21 14:22:33 +02:00
Vedant Naik
f5a34d4d5c
fix: CacheChecker filters syntax (#7898)
* fix: cache_checker filters

* add release notes

* test: add test for cache checker filters syntax
2024-06-21 12:31:29 +02:00
Stefano Fiorucci
c59ad95f42
chore: remove deprecated TGI generators (#7908)
* remove deprecated TGI generators

* rm unused import
2024-06-21 11:15:13 +02:00