mirror of
https://github.com/deepset-ai/haystack.git
synced 2025-09-24 15:45:26 +00:00

* feat!(Crawler): Integrate Crawler in the Pipeline. +Output Documents +Optional file saving +Optional Document meta about file path * refactor: add Optional decl. * chore: dummy commit * chore: dummy commit * refactor: improve overwrite flow * refactor: change custom file path meta logic + add test * Update haystack/nodes/connector/crawler.py Co-authored-by: Massimiliano Pippi <mpippi@gmail.com> * Update haystack/nodes/connector/crawler.py Co-authored-by: Massimiliano Pippi <mpippi@gmail.com> * Update haystack/nodes/connector/crawler.py Co-authored-by: Massimiliano Pippi <mpippi@gmail.com> * Update haystack/nodes/connector/crawler.py Co-authored-by: Massimiliano Pippi <mpippi@gmail.com> * Update haystack/nodes/connector/crawler.py Co-authored-by: Massimiliano Pippi <mpippi@gmail.com> --------- Co-authored-by: Silvano Cerza <3314350+silvanocerza@users.noreply.github.com> Co-authored-by: Massimiliano Pippi <mpippi@gmail.com>