mirror of
https://github.com/deepset-ai/haystack.git
synced 2026-01-15 08:36:56 +00:00
d38c07e0ee
* Add knowledge graph module * Fix type hint * Add graph retriver module * Change type annotations, change return format * Add graph retriever that executes questions as sparql queries * Linking only those entities that are in the knowledge graph * Added logging and using relations extracted from Knowledge graph for linking * Preventing entity linking from linking the same token to multiple entities * Pruning triples that have no variables for select and count queries * Support knowledge graphs with Pipelines * Add text2sparql * Entity linking and relation linking consider more special cases now based on evaluation on labelled data * Separating example code from KGQA implementation * Add eval on combined extarctive and kg questions * Remove references to hp-test * Add fields sparql_query and long_answer_list to metadata * Removing modular Question2SPARQL approach * Removing additional classes used for modular kgqa approach * preparing lcquad data * change graph db * Translating namespaces in knowledge graph queries * Creating graphdb index and loading triples from .ttl file * Fetching graph config files, triples and model from S3 * Fix incompatibility issues with BaseGraphRetriever and BaseComponent * Removing unused utility functions * Adding doc strings and tutorial header * Adding sparqlwrapper dependency * Moving tutorial header * Sorting tutorials by number within name of notebook * Add latest docstring and tutorial changes * Creating test cases for knowledge graph * Changing knowledge graph example to harry potter * Add latest docstring and tutorial changes * Adapting the tutorial notebook to harry potter example * Add GraphDB fixture for tests * Add latest docstring and tutorial changes * Added GraphDB docker launch to CI * Use correct GraphDB fixture * Check if GraphDB instance is already running * Renaming question/query and incorporating other feedback from Timo and Tanay * Removed type annotation * Add latest docstring and tutorial changes Co-authored-by: oryx1729 <oryx1729@protonmail.com> Co-authored-by: Timo Moeller <timo.moeller@deepset.ai> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
32 lines
734 B
Python
32 lines
734 B
Python
import re
|
|
|
|
from nbconvert import MarkdownExporter
|
|
import os
|
|
from pathlib import Path
|
|
from headers import headers
|
|
|
|
|
|
def atoi(text):
|
|
return int(text) if text.isdigit() else text
|
|
|
|
|
|
def natural_keys(text):
|
|
test = [ atoi(c) for c in re.split('(\d+)',text) ]
|
|
return test
|
|
|
|
|
|
dir = Path("../../../../tutorials")
|
|
|
|
notebooks = [x for x in os.listdir(dir) if x[-6:] == ".ipynb"]
|
|
# sort notebooks based on numbers within name of notebook
|
|
notebooks = sorted(notebooks, key=lambda x: natural_keys(x))
|
|
|
|
|
|
e = MarkdownExporter(exclude_output=True)
|
|
for i, nb in enumerate(notebooks):
|
|
body, resources = e.from_filename(dir / nb)
|
|
with open(str(i + 1) + ".md", "w") as f:
|
|
f.write(headers[i + 1] + "\n\n")
|
|
f.write(body)
|
|
|