13510aa753
* Files moved, imports all broken * Fix most imports and docstrings into * Fix the paths to the modules in the API docs * Add latest docstring and tutorial changes * Add a few pipelines that were lost in the inports * Fix a bunch of mypy warnings * Add latest docstring and tutorial changes * Create a file_classifier module * Add docs for file_classifier * Fixed most circular imports, now the REST API can start * Add latest docstring and tutorial changes * Tackling more mypy issues * Reintroduce from FARM and fix last mypy issues hopefully * Re-enable old-style imports * Fix some more import from the top-level package in an attempt to sort out circular imports * Fix some imports in tests to new-style to prevent failed class equalities from breaking tests * Change document_store into document_stores * Update imports in tutorials * Add latest docstring and tutorial changes * Probably fixes summarizer tests * Improve the old-style import allowing module imports (should work) * Try to fix the docs * Remove dedicated KnowledgeGraph page from autodocs * Remove dedicated GraphRetriever page from autodocs * Fix generate_docstrings.sh with an updated list of yaml files to look for * Fix some more modules in the docs * Fix the document stores docs too * Fix a small issue on Tutorial14 * Add latest docstring and tutorial changes * Add deprecation warning to old-style imports * Remove stray folder and import Dict into dense.py * Change import path for MLFlowLogger * Add old loggers path to the import path aliases * Fix debug output of convert_ipynb.py * Fix circular import on BaseRetriever * Missed one merge block * re-run tutorial 5 * Fix imports in tutorial 5 * Re-enable squad_to_dpr CLI from the root package and move get_batches_from_generator into document_stores.base * Add latest docstring and tutorial changes * Fix typo in utils __init__ * Fix a few more imports * Fix benchmarks too * New-style imports in test_knowledge_graph * Rollback setup.py * Rollback squad_to_dpr too Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
📒 Looking for the docs?
You find them here here:
https://haystack.deepset.ai/overview/intro
💻 How to update docs?
Overview and Usage
We move the Overview and Usage docs to the haystack-website repository. You will find the docs in the folder docs. Please make sure to only edit the newest version of the docs. We will release the docs together with the Haystack version.
We are open for contibutions to our documentation. Please make sure to check our Contribution Guidelines. You will find a step by step introduction to our docs here.
Tutorials
The Tutorials live in the folder tutorials. They are created as colab notebooks which can be used by users to explore new haystack features. To include tutorials into the docs website, markdowns files need to be generated from the notebook. This can be done by running the script /docs/_src/tutorials/tutorials/convert_ipynb.py. Just run python convert_ipynb.py and the script will update all existing notebooks. Furthermore, plaese make sure to update the headers.py file with headers for the new tutorials. These headers are important for the docs website workflow. After the markdown files are generated successfully, you can raise a PR. We will review it and as soons as the markdown file is merged to master, it can be added to our website. Please follow the steps described here under Tutorial & Reference Docs.
API Reference
We use Pydoc-Markdown to create markdown files from the docstrings in our code.
Update docstrings
Execute the following commands in /haystack/docs/_src/api/api:
pip install 'pydoc-markdown==3.11.0'
./generate_docstrings.sh
If you want to generate a new markdown file for a new haystack module, please create a .yml which is inline with the following configuration and a a new line to generate_docstrings.sh for the module. After you ran the generate_docstrings.sh again, there should be a new markdown file for the module. To include it into the docs website, push it to master and follow the steps described here under Tutorial & Reference Docs.
Configuration
Pydoc will read the configuration from a .yml file which is located in the current working directory. Our files contains three main sections:
- loader: A list of plugins that load API objects from python source files.
- type: Loader for python source files
- search_path: Location of source files
- modules: Module which are used for generating the markdown file
- ignore_when_discovered: Define which files should be ignored
- processor: A list of plugins that process API objects to modify their docstrings (e.g. to adapt them from a documentation format to Markdown or to remove items that should not be rendered into the documentation).
- type: filter: Filter for specific modules
- documented_only: Only documented API objects
- do_not_filter_modules: Do not filter module objects
- skip_empty_modules: Skip modules without content
- renderer: A plugin that produces the output files.
- type: Define the renderer which you want to use. We are using the Markdown renderer as it can be configured in very detail.
- descriptive_class_title: Remove the word "Object" from class titles.
- descriptive_module_title: Adding the word “Module” before the module name
- add_method_class_prefix: Add the class name as a prefix to method names
- add_member_class_prefix: Add the class name as a prefix to member names
- filename: file name of the generated file