Silvano Cerza 8205724395
feat: Rework Pipeline.run() to better handle cycles (#8431)
* draft

* Enhance

* Almost works

* Simplify some parts and handle intermediate outputs

* Handle connections with default

* Handle cycles with multiple connections from two components

* Update distributed outputs at the correct time

* Remove Component inputs after it runs

* Add agent pipeline test case

* Fix infite loop test

* Handle some corner cases with loops checking and inputs deletion

* Fix tests

* Add new behavioral test

* Remove unused code in behavioural test

* Fix behavioural test

* Fix max run check

* Simplify outputs distribution

* Simplify subgraph run check

* Remove unused _init_run_queue function

* Remove commented code

* Add some missing type hints

* Simplify cycles breaking

* Fix _distribute_output test

* Fix _find_components_that_will_receive_no_input test

* Fix validation test

* Fix tracer losing Component inputs

* Fix some linting issues

* Remove ignore pylint rule

* Rename method that break cycles and make it raise

* Add docstring to _run_subgraph

* Update Pipeline.run() docstring

* Update comment to clarify cycles execution

* Remove SelfLoop sample Component

* Add behavioural test for unsupported cycles

* Rename behavioural test to be more specific

* Add new behavioural test

* Add release notes

* Remove commented out code and random pass

* Use more efficient function to find cycles

* Simplify _break_supported_cycles_in_graph by using defaultdict

* Stop breaking edges as soon as we make the graph acyclic

* Fix docstring and add some more comments

* Fix _distribute_output docstring

* Fix _find_receivers_from docstring

* More detailed release notes

* Minimize calls to networkx.is_directed_acyclic_graph

* Add some more info on edges keys

* Adjust components_in_cycles comment

* Add new Pipeline behavioural test

* Enhance _find_components_that_will_receive_no_input to cover more cases

* Explain why run_queue is reset after running a subgraph cycle

* Rename _init_inputs_state to _normalize_input_data

* Better explain the subgraph output distribution

* Remove for else

* Fix some comments and docstrings

* Fix linting

* Add missing return type

* Fix typo

* Rename _normalize_input_data to _normalize_varidiac_input_data and add more documentation

* Remove unused import

---------

Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
2024-10-29 15:43:16 +01:00

39 lines
1.3 KiB
Python

# SPDX-FileCopyrightText: 2022-present deepset GmbH <info@deepset.ai>
#
# SPDX-License-Identifier: Apache-2.0
from haystack.testing.sample_components.accumulate import Accumulate
from haystack.testing.sample_components.add_value import AddFixedValue
from haystack.testing.sample_components.concatenate import Concatenate
from haystack.testing.sample_components.double import Double
from haystack.testing.sample_components.fstring import FString
from haystack.testing.sample_components.greet import Greet
from haystack.testing.sample_components.hello import Hello
from haystack.testing.sample_components.joiner import StringJoiner, StringListJoiner
from haystack.testing.sample_components.parity import Parity
from haystack.testing.sample_components.remainder import Remainder
from haystack.testing.sample_components.repeat import Repeat
from haystack.testing.sample_components.subtract import Subtract
from haystack.testing.sample_components.sum import Sum
from haystack.testing.sample_components.text_splitter import TextSplitter
from haystack.testing.sample_components.threshold import Threshold
__all__ = [
"Concatenate",
"Subtract",
"Parity",
"Remainder",
"Accumulate",
"Threshold",
"AddFixedValue",
"Repeat",
"Sum",
"Greet",
"Double",
"StringJoiner",
"Hello",
"TextSplitter",
"StringListJoiner",
"FString",
]