haystack/url-cache-checker-a0fb3d7ad0bdb8c2.yaml at new-output-types-in-conditional-router - haystack - Gitea: Git with a cup of tea

yujunjun/haystack

mirror of https://github.com/deepset-ai/haystack.git synced 2025-06-26 22:00:13 +00:00

ZanSara 454988672e

feat: UrlCacheChecker (#5841 )

* add UrlCacheChecker

* rename

* add tests

* reno

* pylint

* review feedback

2023-09-20 14:45:50 +02:00

7 lines

277 B

YAML

Raw Permalink Blame History

 ---
 preview:
   - |
      Add `UrlCacheChecker` to support Web retrieval pipelines.
      Check if documents coming from a given list of URLs are already present in the store and if so, returns them.
      All URLs with no matching documents are returned on a separate connection.