yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-10-03 03:56:30 +00:00
Code Issues Packages Projects Releases Wiki Activity
haystack/docs/pydoc/config/crawler.yml

27 lines
769 B
YAML
Raw Normal View History

Update Documentation (#976) * Add api pages * Add latest docstring and tutorial changes * First sweep of usage docs * Add link to conversion script * Add import statements * Add summarization page * Add web crawler documentation * Add confidence scores usage * Add crawler api docs * Regenerate api docs * Update summarizer and translator api * Add api pages * Add latest docstring and tutorial changes * First sweep of usage docs * Add link to conversion script * Add import statements * Add summarization page * Add web crawler documentation * Add confidence scores usage * Add crawler api docs * Regenerate api docs * Update summarizer and translator api * Add indentation (pydoc-markdown 3.10.1) * Comment out metadata * Remove Finder deprecation message * Remove Finder in FAQ * Update tutorial link * Incorporate reviewer feedback * Regen api docs * Add type annotations Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2021-04-22 16:45:29 +02:00
loaders:
- type: python
re-organize pydoc config files (#4042)
2023-02-03 12:51:10 +01:00
search_path: [../../../haystack/nodes/connector]
Update Documentation (#976) * Add api pages * Add latest docstring and tutorial changes * First sweep of usage docs * Add link to conversion script * Add import statements * Add summarization page * Add web crawler documentation * Add confidence scores usage * Add crawler api docs * Regenerate api docs * Update summarizer and translator api * Add api pages * Add latest docstring and tutorial changes * First sweep of usage docs * Add link to conversion script * Add import statements * Add summarization page * Add web crawler documentation * Add confidence scores usage * Add crawler api docs * Regenerate api docs * Update summarizer and translator api * Add indentation (pydoc-markdown 3.10.1) * Comment out metadata * Remove Finder deprecation message * Remove Finder in FAQ * Update tutorial link * Incorporate reviewer feedback * Regen api docs * Add type annotations Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2021-04-22 16:45:29 +02:00
modules: ['crawler']
ignore_when_discovered: ['__init__']
Upgrade `pydoc-markdown` & refactor GitHub Actions (#2117) * Upgrade pydoc-markdown and fix the YAMLs to work with it * Pin pydoc-markdown to major version * Generalize pydoc-markdown workflow * Make a single Action to perform all tasks that require committing into the local branch * Merge the code updates and the docs in the Linux CI to prevent the bot from always show the pipeline as green * Installing Jupyter deps for Black * Build cache before running generation tasks * Add check not to run the code generation on master * Simplify push action * Add more test deps in setup.cfg and remove from GH Action workflow * Remove forced upgrades on pip install Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-04 15:45:09 +01:00
processors:
Update Documentation (#976) * Add api pages * Add latest docstring and tutorial changes * First sweep of usage docs * Add link to conversion script * Add import statements * Add summarization page * Add web crawler documentation * Add confidence scores usage * Add crawler api docs * Regenerate api docs * Update summarizer and translator api * Add api pages * Add latest docstring and tutorial changes * First sweep of usage docs * Add link to conversion script * Add import statements * Add summarization page * Add web crawler documentation * Add confidence scores usage * Add crawler api docs * Regenerate api docs * Update summarizer and translator api * Add indentation (pydoc-markdown 3.10.1) * Comment out metadata * Remove Finder deprecation message * Remove Finder in FAQ * Update tutorial link * Incorporate reviewer feedback * Regen api docs * Add type annotations Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2021-04-22 16:45:29 +02:00
- type: filter
re-organize pydoc config files (#4042)
2023-02-03 12:51:10 +01:00
expression:
Upgrade `pydoc-markdown` & refactor GitHub Actions (#2117) * Upgrade pydoc-markdown and fix the YAMLs to work with it * Pin pydoc-markdown to major version * Generalize pydoc-markdown workflow * Make a single Action to perform all tasks that require committing into the local branch * Merge the code updates and the docs in the Linux CI to prevent the bot from always show the pipeline as green * Installing Jupyter deps for Black * Build cache before running generation tasks * Add check not to run the code generation on master * Simplify push action * Add more test deps in setup.cfg and remove from GH Action workflow * Remove forced upgrades on pip install Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-04 15:45:09 +01:00
documented_only: true
do_not_filter_modules: false
skip_empty_modules: true
- type: smart
- type: crossref
Update Documentation (#976) * Add api pages * Add latest docstring and tutorial changes * First sweep of usage docs * Add link to conversion script * Add import statements * Add summarization page * Add web crawler documentation * Add confidence scores usage * Add crawler api docs * Regenerate api docs * Update summarizer and translator api * Add api pages * Add latest docstring and tutorial changes * First sweep of usage docs * Add link to conversion script * Add import statements * Add summarization page * Add web crawler documentation * Add confidence scores usage * Add crawler api docs * Regenerate api docs * Update summarizer and translator api * Add indentation (pydoc-markdown 3.10.1) * Comment out metadata * Remove Finder deprecation message * Remove Finder in FAQ * Update tutorial link * Incorporate reviewer feedback * Regen api docs * Add type annotations Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2021-04-22 16:45:29 +02:00
renderer:
docs: sync Haystack API with Readme (#3223) * First pass at syncing Haystack API with Readme * Reapply changes * Regularize slugs * Regularize slugs * Regularize slugs * Set category id and regen * Trigger workflow * Delete old md files * Test sync * Undo test string * Incorporate reviewer feedback * Test on the fly API generation and sync * Test on the fly API generation and sync * Test on the fly API generation and sync * Test on the fly API generation and sync * Test on the fly API generation and sync * Change name of pydoc-markdown scripts * Test on the fly API generation and sync * Remove version tag * Test version tag * Test version tag * Test version tag * Revert test docstring * Revert md file changes * Revert md file changes * Revert script naming * Test on the fly generation and sync * Adjust for on the fly generation and sync * Revert test string * Remove old documentation workflow * Set workflow to work on main * Change readme version name
2022-09-21 17:18:34 +02:00
type: renderers.ReadmeRenderer
excerpt: The Crawler scrapes the text from a website, creates a Haystack Document object out of it, and saves it to a JSON file.
category: 6310ca73c622850ddd3875a2
title: Crawler API
slug: crawler-api
order: 10
markdown:
descriptive_class_title: false
descriptive_module_title: true
add_method_class_prefix: true
add_member_class_prefix: false
filename: crawler_api.md
Reference in New Issue Copy Permalink