haystack/test/core/pipeline/features/pipeline_run.feature
mathislucka e5b9bdeb66
feat: AsyncPipeline that can schedule components to run concurrently (#8812)
* add component checks

* pipeline should run deterministically

* add FIFOQueue

* add agent tests

* add order dependent tests

* run new tests

* remove code that is not needed

* test: intermediate from cycle outputs are available outside cycle

* add tests for component checks (Claude)

* adapt tests for component checks (o1 review)

* chore: format

* remove tests that aren't needed anymore

* add _calculate_priority tests

* revert accidental change in pyproject.toml

* test format conversion

* adapt to naming convention

* chore: proper docstrings and type hints for PQ

* format

* add more unit tests

* rm unneeded comments

* test input consumption

* lint

* fix: docstrings

* lint

* format

* format

* fix license header

* fix license header

* add component run tests

* fix: pass correct input format to tracing

* fix types

* format

* format

* types

* add defaults from Socket instead of signature

- otherwise components with dynamic inputs would fail

* fix test names

* still wait for optional inputs on greedy variadic sockets

- mirrors previous behavior

* fix format

* wip: warn for ambiguous running order

* wip: alternative warning

* fix license header

* make code more readable

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Introduce content tracing to a behavioral test

* Fixing linting

* Remove debug print statements

* Fix tracer tests

* remove print

* test: test for component inputs

* test: remove testing for run order

* chore: update component checks from experimental

* chore: update pipeline and base from experimental

* refactor: remove unused method

* refactor: remove unused method

* refactor: outdated comment

* refactor: inputs state is updated as side effect

- to prepare for AsyncPipeline implementation

* format

* test: add file conversion test

* format

* fix: original implementation deepcopies outputs

* lint

* fix: from_dict was updated

* fix: format

* fix: test

* test: add test for thread safety

* remove unused imports

* format

* test: FIFOPriorityQueue

* chore: add release note

* feat: add AsyncPipeline

* chore: Add release notes

* fix: format

* debug: switch run order to debug ubuntu and windows tests

* fix: consider priorities of other components while waiting for DEFER

* refactor: simplify code

* fix: resolve merge conflict with mermaid changes

* fix: format

* fix: remove unused import

* refactor: rename to avoid accidental conflicts

* fix: track pipeline type

* fix: and extend test

* fix: format

* style: sort alphabetically

* Update test/core/pipeline/features/conftest.py

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Update test/core/pipeline/features/conftest.py

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>

* Update releasenotes/notes/feat-async-pipeline-338856a142e1318c.yaml

* fix: indentation, do not close loop

* fix: use asyncio.run

* fix: format

---------

Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
2025-02-07 16:37:29 +01:00

66 lines
3.8 KiB
Gherkin

Feature: Pipeline running
Scenario Outline: Running a correct Pipeline
Given a pipeline <kind>
When I run the Pipeline
Then components are called with the expected inputs
And it should return the expected result
Examples:
| kind |
| that has no components |
| that is linear |
| that is really complex with lots of components, forks, and loops |
| that has a single component with a default input |
| that has two loops of identical lengths |
| that has two loops of different lengths |
| that has a single loop with two conditional branches |
| that has a component with dynamic inputs defined in init |
| that has two branches that don't merge |
| that has three branches that don't merge |
| that has two branches that merge |
| that has different combinations of branches that merge and do not merge |
| that has two branches, one of which loops back |
| that has a component with mutable input |
| that has a component with mutable output sent to multiple inputs |
| that has a greedy and variadic component after a component with default input |
| that has a component with only default inputs |
| that has a component with only default inputs as first to run and receives inputs from a loop |
| that has multiple branches that merge into a component with a single variadic input |
| that has multiple branches of different lengths that merge into a component with a single variadic input |
| that is linear and returns intermediate outputs |
| that has a loop and returns intermediate outputs from it |
| that is linear and returns intermediate outputs from multiple sockets |
| that has a component with default inputs that doesn't receive anything from its sender |
| that has a component with default inputs that doesn't receive anything from its sender but receives input from user |
| that has a loop and a component with default inputs that doesn't receive anything from its sender but receives input from user |
| that has multiple components with only default inputs and are added in a different order from the order of execution |
| that is linear with conditional branching and multiple joins |
| that is a simple agent |
| that has a variadic component that receives partial inputs |
| that has a variadic component that receives partial inputs in a different order |
| that has an answer joiner variadic component |
| that is linear and a component in the middle receives optional input from other components and input from the user |
| that has a loop in the middle |
| that has variadic component that receives a conditional input |
| that has a string variadic component |
| that is an agent that can use RAG |
| that has a feedback loop |
| created in a non-standard order that has a loop |
| that has an agent with a feedback cycle |
| that passes outputs that are consumed in cycle to outside the cycle |
| with a component that has dynamic default inputs |
| with a component that has variadic dynamic default inputs |
| that is a file conversion pipeline with two joiners |
Scenario Outline: Running a bad Pipeline
Given a pipeline <kind>
When I run the Pipeline
Then it must have raised <exception>
Examples:
| kind | exception |
| that has an infinite loop | PipelineMaxComponentRuns |
| that has a component that doesn't return a dictionary | PipelineRuntimeError |
| that has a cycle that would get it stuck | PipelineRuntimeError |