* initial import
* small fixes
* adding tests
* adding tests
* refactoring merge graphs
* updating tests
* docstrings
* adding release notes
* adding SuperComponent name to extended components
* adding colours and legend to different super components
* adding missed docstring parameter
* fixing tests and type checking
* Update haystack/core/pipeline/base.py
Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
* forcing keyword arguments for draw() and show()
* adding wrapper function and a deprecation warning
* adding pylint disable - this will be removed soon
* wip
* adding a decorator function to test if another function is being called with positional arguments
* adding a decorator function to test if another function is being called with positional arguments
---------
Co-authored-by: Sebastian Husch Lee <10526848+sjrl@users.noreply.github.com>
* First pass at fix for deepcopying inputs and outputs
* Add reno
* Add recursion for dict objects
* Bump recursion depth
* More tests and some improvments
* Fix unit tests
* PR comments
* Agent tracing
* Small changes
* Some changes and refactoring
* Refactoring to reuse code
* Fix
* Add reno
* Fix tests
* Fix tests
* Fix linting
* Refactor and add tracing support to run_async of Agent
* Reduce duplicate code
* Remove finalize_run
* Use break instead of copying code three times
* Adding a test
* Add tracing unit tests
* Make async tracing test actually run async
* Increase test coverage
* Unit test for traces in pipeline
* Add cleanup
* Fix proper indentation
* PR comments
* PR comments and new test
* Update warning message
* Update warning message
---------
Co-authored-by: Vladimir Blagojevic <dovlex@gmail.com>
* Start refactoring PipelineRuntimeError
* Slight change
* Fix test and remove test that had no asserts
* Change back to pipeline runtime error
* PR comments
* Add reno
* PR comments
* Update test
* Starting to refactor type util tests to be more systematic
* refactoring
* Expand tests
* Update to type utils
* Add missing subclass check
* Expand and refactor tests, introduce type_validation Literal
* More test refactoring
* Test refactoring, adding type validation variable to pipeline base
* Update relaxed version of type checking to pass all newly added tests
* trim whitespace
* Add tests
* cleanup
* Updates docstrings
* Add reno
* docs
* Fix mypy and add docstrings
* Changes based on advice from Tobi
* Remove unused imports
* Doc strings
* Add connection type validation to to_dict and from_dict
* Update tests
* Fix test
* Also save connection_type_validation at global pipeline level
* Fix tests
* Remove connection type validation from the connect level, only keep at pipeline level
* Formatting
* Fix tests
* formatting
* fix: only overwrite existing socket inputs when we provide a new value
* chore: add release notes
* Apply suggestions from code review
---------
Co-authored-by: Julian Risch <julian.risch@deepset.ai>
* fix: use is not to compare to sentinel value
* chore: release notes
* Update releasenotes/notes/fix-component-checks-with-ambiguous-truth-values-949c447b3702e427.yaml
Co-authored-by: David S. Batista <dsbatista@gmail.com>
* fix: another sentinel value
* test: also test base class
* add pandas as test dependency
* format
* Trigger CI
* mark test with xfail strict=False
---------
Co-authored-by: Sebastian Husch Lee <sjrl@users.noreply.github.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
Co-authored-by: anakin87 <stefanofiorucci@gmail.com>
* add component checks
* pipeline should run deterministically
* add FIFOQueue
* add agent tests
* add order dependent tests
* run new tests
* remove code that is not needed
* test: intermediate from cycle outputs are available outside cycle
* add tests for component checks (Claude)
* adapt tests for component checks (o1 review)
* chore: format
* remove tests that aren't needed anymore
* add _calculate_priority tests
* revert accidental change in pyproject.toml
* test format conversion
* adapt to naming convention
* chore: proper docstrings and type hints for PQ
* format
* add more unit tests
* rm unneeded comments
* test input consumption
* lint
* fix: docstrings
* lint
* format
* format
* fix license header
* fix license header
* add component run tests
* fix: pass correct input format to tracing
* fix types
* format
* format
* types
* add defaults from Socket instead of signature
- otherwise components with dynamic inputs would fail
* fix test names
* still wait for optional inputs on greedy variadic sockets
- mirrors previous behavior
* fix format
* wip: warn for ambiguous running order
* wip: alternative warning
* fix license header
* make code more readable
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
* Introduce content tracing to a behavioral test
* Fixing linting
* Remove debug print statements
* Fix tracer tests
* remove print
* test: test for component inputs
* test: remove testing for run order
* chore: update component checks from experimental
* chore: update pipeline and base from experimental
* refactor: remove unused method
* refactor: remove unused method
* refactor: outdated comment
* refactor: inputs state is updated as side effect
- to prepare for AsyncPipeline implementation
* format
* test: add file conversion test
* format
* fix: original implementation deepcopies outputs
* lint
* fix: from_dict was updated
* fix: format
* fix: test
* test: add test for thread safety
* remove unused imports
* format
* test: FIFOPriorityQueue
* chore: add release note
* feat: add AsyncPipeline
* chore: Add release notes
* fix: format
* debug: switch run order to debug ubuntu and windows tests
* fix: consider priorities of other components while waiting for DEFER
* refactor: simplify code
* fix: resolve merge conflict with mermaid changes
* fix: format
* fix: remove unused import
* refactor: rename to avoid accidental conflicts
* fix: track pipeline type
* fix: and extend test
* fix: format
* style: sort alphabetically
* Update test/core/pipeline/features/conftest.py
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
* Update test/core/pipeline/features/conftest.py
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
* Update releasenotes/notes/feat-async-pipeline-338856a142e1318c.yaml
* fix: indentation, do not close loop
* fix: use asyncio.run
* fix: format
---------
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
* add component checks
* pipeline should run deterministically
* add FIFOQueue
* add agent tests
* add order dependent tests
* run new tests
* remove code that is not needed
* test: intermediate from cycle outputs are available outside cycle
* add tests for component checks (Claude)
* adapt tests for component checks (o1 review)
* chore: format
* remove tests that aren't needed anymore
* add _calculate_priority tests
* revert accidental change in pyproject.toml
* test format conversion
* adapt to naming convention
* chore: proper docstrings and type hints for PQ
* format
* add more unit tests
* rm unneeded comments
* test input consumption
* lint
* fix: docstrings
* lint
* format
* format
* fix license header
* fix license header
* add component run tests
* fix: pass correct input format to tracing
* fix types
* format
* format
* types
* add defaults from Socket instead of signature
- otherwise components with dynamic inputs would fail
* fix test names
* still wait for optional inputs on greedy variadic sockets
- mirrors previous behavior
* fix format
* wip: warn for ambiguous running order
* wip: alternative warning
* fix license header
* make code more readable
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
* Introduce content tracing to a behavioral test
* Fixing linting
* Remove debug print statements
* Fix tracer tests
* remove print
* test: test for component inputs
* test: remove testing for run order
* chore: update component checks from experimental
* chore: update pipeline and base from experimental
* refactor: remove unused method
* refactor: remove unused method
* refactor: outdated comment
* refactor: inputs state is updated as side effect
- to prepare for AsyncPipeline implementation
* format
* test: add file conversion test
* format
* fix: original implementation deepcopies outputs
* lint
* fix: from_dict was updated
* fix: format
* fix: test
* test: add test for thread safety
* remove unused imports
* format
* test: FIFOPriorityQueue
* chore: add release note
* fix: resolve merge conflict with mermaid changes
* fix: format
* fix: remove unused import
* refactor: rename to avoid accidental conflicts
* chore: remove unused inputs, add missing license header
* chore: extend release notes
* Update releasenotes/notes/fix-pipeline-run-2fefeafc705a6d91.yaml
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
* fix: format
* fix: format
* Update release note
---------
Co-authored-by: Amna Mubashar <amnahkhan.ak@gmail.com>
Co-authored-by: David S. Batista <dsbatista@gmail.com>
* compress graph data to support pako endpoint
* support mermaid.ink parameters and custom servers
* dont try to resolve conflicts with the github web ui...
* avoid double graph copy
* fixing typing, improving docstrings and release notes
* reverting type
* nit - force type checker no cache
* nit - force type checker no cache
---------
Co-authored-by: Ulises M <ulises@lbux.org>
Co-authored-by: Ulises M <30765968+lbux@users.noreply.github.com>
* draft
* del HF token in tests
* adaptations
* progress
* fix type
* import sorting
* more control on deserialization
* release note
* improvements
* support name field
* fix chatpromptbuilder test
* Update chat_message.py
---------
Co-authored-by: Daria Fokina <daria.fokina@deepset.ai>
* draft
* Enhance
* Almost works
* Simplify some parts and handle intermediate outputs
* Handle connections with default
* Handle cycles with multiple connections from two components
* Update distributed outputs at the correct time
* Remove Component inputs after it runs
* Add agent pipeline test case
* Fix infite loop test
* Handle some corner cases with loops checking and inputs deletion
* Fix tests
* Add new behavioral test
* Remove unused code in behavioural test
* Fix behavioural test
* Fix max run check
* Simplify outputs distribution
* Simplify subgraph run check
* Remove unused _init_run_queue function
* Remove commented code
* Add some missing type hints
* Simplify cycles breaking
* Fix _distribute_output test
* Fix _find_components_that_will_receive_no_input test
* Fix validation test
* Fix tracer losing Component inputs
* Fix some linting issues
* Remove ignore pylint rule
* Rename method that break cycles and make it raise
* Add docstring to _run_subgraph
* Update Pipeline.run() docstring
* Update comment to clarify cycles execution
* Remove SelfLoop sample Component
* Add behavioural test for unsupported cycles
* Rename behavioural test to be more specific
* Add new behavioural test
* Add release notes
* Remove commented out code and random pass
* Use more efficient function to find cycles
* Simplify _break_supported_cycles_in_graph by using defaultdict
* Stop breaking edges as soon as we make the graph acyclic
* Fix docstring and add some more comments
* Fix _distribute_output docstring
* Fix _find_receivers_from docstring
* More detailed release notes
* Minimize calls to networkx.is_directed_acyclic_graph
* Add some more info on edges keys
* Adjust components_in_cycles comment
* Add new Pipeline behavioural test
* Enhance _find_components_that_will_receive_no_input to cover more cases
* Explain why run_queue is reset after running a subgraph cycle
* Rename _init_inputs_state to _normalize_input_data
* Better explain the subgraph output distribution
* Remove for else
* Fix some comments and docstrings
* Fix linting
* Add missing return type
* Fix typo
* Rename _normalize_input_data to _normalize_varidiac_input_data and add more documentation
* Remove unused import
---------
Co-authored-by: Sebastian Husch Lee <sjrl423@gmail.com>
* Added equality check for sender and receiver in connection function of pipeline
* Update base.py
irrelevant changes reverted
* added release note
* altered a walk with cycle test
* added a test to verify that pipeline raises PipelineConnectError when adding a component to itself
* Update release notes
* Remove self connection feature tests
* Tidy up connect unit test
---------
Co-authored-by: Silvano Cerza <silvanocerza@gmail.com>
* fix: Prevent the usage of `set_input_type(s)` when the `run` method doesn't have kwargs,
raise if `set_input_type(s)` overrides `run` method parameters
* fix: update components and tests
* reno
* Deprecate max_loops_allowed in favour of new argument max_runs_per_component
* Add missing test file
* Some enhancements
* Add version that will remove deprecate stuff
* Fix bug in Pipeline.run() executing Components in a wrong and unexpected order
* Update haystack/core/pipeline/base.py
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
---------
Co-authored-by: Madeesh Kannan <shadeMe@users.noreply.github.com>
