yujunjun/haystack
1
0
Fork 0
mirror of https://github.com/deepset-ai/haystack.git synced 2025-07-20 15:31:15 +00:00
Code Issues Packages Projects Releases Wiki Activity
haystack/tutorials/Tutorial3_Basic_QA_Pipeline_without_Elasticsearch.ipynb
Branden Chan 783893c3d2
Tutorial update (#1166) 
* Add header / footer

* Add Milvus example

* Generate md files

* Fix mypy CI
2021-06-11 11:09:15 +02:00

459 lines
46 KiB
Plaintext
Raw Blame History

{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Build a QA System Without Elasticsearch\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/deepset-ai/haystack/blob/master/tutorials/Tutorial3_Basic_QA_Pipeline_without_Elasticsearch.ipynb)\n",
"\n",
"Haystack provides alternatives to Elasticsearch for developing quick prototypes.\n",
"\n",
"You can use an `InMemoryDocumentStore` or a `SQLDocumentStore`(with SQLite) as the document store.\n",
"\n",
"If you are interested in more feature-rich Elasticsearch, then please refer to the Tutorial 1. "
]
},
{
"cell_type": "markdown",
"source": [
"### Prepare environment\n",
"\n",
"#### Colab: Enable the GPU runtime\n",
"Make sure you enable the GPU runtime to experience decent speed in this tutorial.\n",
"**Runtime -> Change Runtime type -> Hardware accelerator -> GPU**\n",
"\n",
"<img src=\"https://raw.githubusercontent.com/deepset-ai/haystack/master/docs/_src/img/colab_gpu_runtime.jpg\">"
],
"metadata": {
"collapsed": false
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"# Make sure you have a GPU running\n",
"!nvidia-smi"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Install the latest release of Haystack in your own environment \n",
"#! pip install farm-haystack\n",
"\n",
"# Install the latest master of Haystack\n",
"!pip install grpcio-tools==1.34.1\n",
"!pip install git+https://github.com/deepset-ai/haystack.git\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"pycharm": {
"is_executing": false
}
},
"outputs": [],
"source": [
"from haystack import Finder\n",
"from haystack.preprocessor.cleaning import clean_wiki_text\n",
"from haystack.preprocessor.utils import convert_files_to_dicts, fetch_archive_from_http\n",
"from haystack.reader.farm import FARMReader\n",
"from haystack.reader.transformers import TransformersReader\n",
"from haystack.utils import print_answers"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Document Store\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"# In-Memory Document Store\n",
"from haystack.document_store.memory import InMemoryDocumentStore\n",
"document_store = InMemoryDocumentStore()"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"# SQLite Document Store\n",
"# from haystack.document_store.sql import SQLDocumentStore\n",
"# document_store = SQLDocumentStore(url=\"sqlite:///qa.db\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"pycharm": {
"name": "#%% md\n"
}
},
"source": [
"## Preprocessing of documents\n",
"\n",
"Haystack provides a customizable pipeline for:\n",
" - converting files into texts\n",
" - cleaning texts\n",
" - splitting texts\n",
" - writing them to a Document Store\n",
"\n",
"In this tutorial, we download Wikipedia articles on Game of Thrones, apply a basic cleaning function, and index them in Elasticsearch."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"04/28/2020 15:06:07 - INFO - haystack.indexing.io - Found data stored in `data/article_txt_got`. Delete this first if you really want to fetch new data.\n",
"04/28/2020 15:06:07 - INFO - haystack.indexing.io - Wrote 517 docs to DB\n"
]
}
],
"source": [
"# Let's first get some documents that we want to query\n",
"# Here: 517 Wikipedia articles for Game of Thrones\n",
"doc_dir = \"data/article_txt_got\"\n",
"s3_url = \"https://s3.eu-central-1.amazonaws.com/deepset.ai-farm-qa/datasets/documents/wiki_gameofthrones_txt.zip\"\n",
"fetch_archive_from_http(url=s3_url, output_dir=doc_dir)\n",
"\n",
"# convert files to dicts containing documents that can be indexed to our datastore\n",
"# You can optionally supply a cleaning function that is applied to each doc (e.g. to remove footers)\n",
"# It must take a str as input, and return a str.\n",
"dicts = convert_files_to_dicts(dir_path=doc_dir, clean_func=clean_wiki_text, split_paragraphs=True)\n",
"\n",
"# We now have a list of dictionaries that we can write to our document store.\n",
"# If your texts come from a different source (e.g. a DB), you can of course skip convert_files_to_dicts() and create the dictionaries yourself.\n",
"# The default format here is: {\"name\": \"<some-document-name>, \"text\": \"<the-actual-text>\"}\n",
"\n",
"# Let's have a look at the first 3 entries:\n",
"print(dicts[:3])\n",
"# Now, let's write the docs to our DB.\n",
"document_store.write_documents(dicts)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initalize Retriever, Reader, & Finder\n",
"\n",
"### Retriever\n",
"\n",
"Retrievers help narrowing down the scope for the Reader to smaller units of text where a given question could be answered. \n",
"\n",
"With InMemoryDocumentStore or SQLDocumentStore, you can use the TfidfRetriever. For more retrievers, please refer to the tutorial-1."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"pycharm": {
"is_executing": false,
"name": "#%%\n"
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"04/28/2020 15:07:34 - INFO - haystack.retriever.tfidf - Found 2811 candidate paragraphs from 517 docs in DB\n"
]
}
],
"source": [
"# An in-memory TfidfRetriever based on Pandas dataframes\n",
"\n",
"from haystack.retriever.sparse import TfidfRetriever\n",
"retriever = TfidfRetriever(document_store=document_store)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Reader\n",
"\n",
"A Reader scans the texts returned by retrievers in detail and extracts the k best answers. They are based\n",
"on powerful, but slower deep learning models.\n",
"\n",
"Haystack currently supports Readers based on the frameworks FARM and Transformers.\n",
"With both you can either load a local model or one from Hugging Face's model hub (https://huggingface.co/models).\n",
"\n",
"**Here:** a medium sized RoBERTa QA model using a Reader based on FARM (https://huggingface.co/deepset/roberta-base-squad2)\n",
"\n",
"**Alternatives (Reader):** TransformersReader (leveraging the `pipeline` of the Transformers package)\n",
"\n",
"**Alternatives (Models):** e.g. \"distilbert-base-uncased-distilled-squad\" (fast) or \"deepset/bert-large-uncased-whole-word-masking-squad2\" (good accuracy)\n",
"\n",
"**Hint:** You can adjust the model to return \"no answer possible\" with the no_ans_boost. Higher values mean the model prefers \"no answer possible\"\n",
"\n",
"#### FARMReader"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"pycharm": {
"is_executing": false
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"04/28/2020 15:07:40 - INFO - farm.utils - device: cpu n_gpu: 0, distributed training: False, automatic mixed precision training: None\n",
"04/28/2020 15:07:40 - INFO - farm.infer - Could not find `deepset/roberta-base-squad2` locally. Try to download from model hub ...\n",
"04/28/2020 15:07:45 - WARNING - farm.modeling.language_model - Could not automatically detect from language model name what language it is. \n",
"\t We guess it's an *ENGLISH* model ... \n",
"\t If not: Init the language model by supplying the 'language' param.\n",
"04/28/2020 15:07:49 - WARNING - farm.modeling.prediction_head - Some unused parameters are passed to the QuestionAnsweringHead. Might not be a problem. Params: {\"loss_ignore_index\": -1}\n",
"04/28/2020 15:07:54 - INFO - farm.utils - device: cpu n_gpu: 0, distributed training: False, automatic mixed precision training: None\n"
]
}
],
"source": [
"# Load a local model or any of the QA models on\n",
"# Hugging Face's model hub (https://huggingface.co/models)\n",
"\n",
"reader = FARMReader(model_name_or_path=\"deepset/roberta-base-squad2\", use_gpu=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### TransformersReader"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Alternative:\n",
"# reader = TransformersReader(model_name_or_path=\"distilbert-base-uncased-distilled-squad\", tokenizer=\"distilbert-base-uncased\", use_gpu=-1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Pipeline\n",
"\n",
"With a Haystack `Pipeline` you can stick together your building blocks to a search pipeline.\n",
"Under the hood, `Pipelines` are Directed Acyclic Graphs (DAGs) that you can easily customize for your own use cases.\n",
"To speed things up, Haystack also comes with a few predefined Pipelines. One of them is the `ExtractiveQAPipeline` that combines a retriever and a reader to answer our questions.\n",
"You can learn more about `Pipelines` in the [docs](https://haystack.deepset.ai/docs/latest/pipelinesmd)."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"pycharm": {
"is_executing": false
}
},
"outputs": [],
"source": [
"from haystack.pipeline import ExtractiveQAPipeline\n",
"pipe = ExtractiveQAPipeline(reader, retriever)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Voilà! Ask a question!"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"pycharm": {
"is_executing": false
}
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"04/28/2020 15:07:54 - INFO - haystack.retriever.tfidf - Identified 10 candidates via retriever:\n",
" paragraph_id document_id text\n",
" 2721 f258f55f534d1a8b3644c3b1bd2d9069 \\n===Arya Stark===\\n'''Arya Stark''' portrayed by Maisie Williams. Arya Stark of House Stark is the younger daughter and third child of Lord Eddard and Catelyn Stark of Winterfell. Ever the tomboy, Arya would rather be training to use weapons than sewing with a needle. She names her direwolf Nymeria, after a legendary warrior queen.\n",
" 2210 2d9538e229b8272de9b6f74bf8cdd270 \\n====Season 8====\\nArya reunites with Jon, Gendry, and the Hound, who have all journeyed to Winterfell with Daenerys Targaryen's forces to make a stand against the approaching White Walkers. Arya asks Gendry, who is forging dragonglass into weapons, to make her a special dragonglass staff. When Gendry gives it to Arya, he tells her he is the bastard son of Robert Baratheon. Aware of their chances of dying in the upcoming battle and Arya wanting to experience sex, Arya and Gendry sleep together. Later that night, Arya hears the signal alerting her that the White Walkers' army has arrived.\\nArya fights in the battle against the dead with Sandor Clegane and Beric Dondarrion. Beric sacrifices himself to allow Arya and the Hound to escape the wights. A battered Arya sprints through the corridors of Winterfell and encounters Melisandre, who suggests to Arya that she is meant to kill the Night King. In the Godswood, just as the Night King is about to kill Bran, Arya sneaks up and stabs the Night King with the Valyrian steel dagger Bran gave her. Upon killing the Night King, the White Walkers and wights are all destroyed. \\nIn the aftermath of the battle, Arya is proposed to by Gendry, who had just been legitimised as a Baratheon by Daenerys. Arya declines, as she does not want the life of a lady. Sansa and Arya tell Jon they don't trust Daenerys, but Jon defends her. Arya learns that Jon is the son of her aunt, Lyanna Stark, and Rhaegar Targaryen after Jon swears her and Sansa to secrecy about his true parentage.\\nArya journeys south to King's Landing with the Hound to kill Cersei. The two infiltrate the Red Keep with the civilians Cersei is using to deter Daenerys' attack. Despite the city's surrender to Daenerys, she begins laying waste to the populace atop of Drogon. In his mission for revenge against his brother, the Mountain, the Hound seeks out the Mountain but urges Arya to leave and give up her quest for revenge to avoid a life consumed by it. Arya thanks the Hound, calling him by his name for the first time. She tries and fails to save the smallfolk, narrowly avoiding being incinerated in Daenerys' attack on the city, but survives. In the aftermath, Arya is reunited with Jon and warns him that he and the Starks are not safe from Daenerys. Jon tries but is unable to dissuade Daenerys from further destruction and ultimately assassinates her. He is imprisoned. Weeks later, Arya joins the other lords and ladies of Westeros in a council to decide who shall lead the Seven Kingdoms. Bran is chosen as king, though Arya abstains from voting as Sansa declares the North's independence. Arya, Sansa, and Bran bid Jon farewell as he is exiled. \\nArya reveals that she is leaving Westeros to see what lies west of the continent. She embarks on her voyage aboard a Stark ship.\n",
" 2203 2d9538e229b8272de9b6f74bf8cdd270 \\n====Season 1====\\nArya accompanies her father Ned and her sister Sansa to King's Landing. Before their departure, Arya's half-brother Jon Snow gifts Arya a sword which she dubs \"Needle\". On the Kingsroad, Arya is sparring with a butcher's boy, Mycah, when Sansa's betrothed Prince Joffrey Baratheon attacks Mycah, prompting Arya's direwolf Nymeria to bite Joffrey. Arya shoos Nymeria away so she is not killed, but is furious when Sansa later refuses to support her version of events. Mycah is later killed by Joffrey's bodyguard Sandor \"The Hound\" Clegane, earning him Arya's hatred. Ned arranges for Arya to have sword lessons with the Braavosi Syrio Forel, who later defends her from Ser Meryn Trant after Joffrey ascends to the throne and kills the Stark household. Arya flees the Red Keep, accidentally killing a stable boy in her escape, hiding out as a beggar in the streets of King's Landing. Ned is eventually taken to the Great Sept of Baelor to face judgment; he spots Arya in the crowd, and alerts the Night's Watch recruiter Yoren to her presence. Yoren prevents Arya from witnessing Ned's execution and has her pose as a boy, \"Arry\", to avoid detection as she joins Yoren's recruits traveling north to Castle Black.\n",
" 546 5719fd2f8b3bafdc028f1e043301cd05 \\n===''A Game of Thrones''===\\nSansa Stark begins the novel by being betrothed to Crown Prince Joffrey Baratheon, believing Joffrey to be a gallant prince. While Joffrey and Sansa are walking through the woods, Joffrey notices Arya sparring with the butcher's boy, Mycah. A fight breaks out and Joffrey is attacked by Nymeria (Arya's direwolf) after Joffrey threatens to hurt Arya. Sansa lies to King Robert about the circumstances of the fight in order to protect both Joffrey and her sister Arya. Since Arya ran off with her wolf to save it, Sansa's wolf is killed instead, estranging the Stark daughters.\\nDuring the Tourney of the Hand to honour her father Lord Eddard Stark, Sansa Stark is enchanted by the knights performing in the event. At the request of his mother, Queen Cersei Lannister, Joffrey spends a portion of the tourney with Sansa, but near the end he commands his guard Sandor Clegane, better known as The Hound, to take her back to her quarters. Sandor explains how his older brother, Gregor, aka \"Mountain that Rides\" pushed his face into a brazier of hot coals, for playing with one of his wooden toys.\\nAfter Eddard discovers the truth of Joffrey's paternity, he tells Sansa that they will be heading back to Winterfell. Sansa is devastated and wishes to stay in King's Landing, so she runs off to inform Queen Cersei of her father's plans, unwittingly providing Cersei with the information needed to arrest her father. After Robert dies, Sansa begs Joffrey to show mercy on her father and he agrees, if Ned will swear an oath of loyalty, but executes him anyway, in front of Sansa. Sansa is now effectively a hostage in King's Landing and finally sees Joffrey's true nature, after he forces her to look at the tarred head of her now-deceased father.\n",
" 1435 54fa6e9a31fd8e0cac24b34d95085424 \\n===In Braavos===\\nLady Crane returns to her chambers to find a wounded Arya hiding inside, and helps stitch her wounds. She tells Arya that, thanks to her warning, she mutilated her would-be killer Bianca's face before kicking her out of the acting company. She then offers to have Arya join them, but she refuses, saying that she intends to travel west of Westeros to see the edge of the world. As Arya recovers, the Waif arrives and kills Lady Crane, intending to kill Arya as well. Arya flees through the streets of Braavos, but during the chase, Arya's wounds reopen and she limps back to her hideout with the Waif in pursuit. As the Waif closes in, Arya extinguishes the candle lighting the room; having trained while blinded for several weeks, Arya has the upper hand.\\nAt the House of Black and White, Jaqen follows a bloodtrail to the Hall of Faces, where he finds the Waif's face before being held at sword-point by Arya. Jaqen congratulates Arya for finally becoming No One. However, she rejects the title, asserting her identity as Arya Stark before turning and leaving, announcing that she is \"going home.\" Jaqen proudly watches on as she leaves.\n",
" 460 f72a43a5163da9b8ffc2672df09099be \\n== Characters ==\\nThe tale is told through the eyes of 9 recurring POV characters plus one prologue POV character:\\n* Prologue: Maester Cressen, maester at Dragonstone\\n* Tyrion Lannister, youngest son of Lord Tywin Lannister, a dwarf and a brother to Queen Cersei, and the acting Hand of the King\\n* Lady Catelyn Stark, of House Tully, widow of Eddard Stark, Lord of Winterfell\\n* Ser Davos Seaworth, a smuggler turned knight in the service of King Stannis Baratheon, often called the Onion Knight\\n* Sansa Stark, eldest daughter of Eddard Stark and Catelyn Stark, held captive by Queen Cersei at King's Landing\\n* Arya Stark, youngest daughter of Eddard Stark and Catelyn Stark, missing and presumed dead\\n* Bran Stark, second son of Eddard Stark and Catelyn Stark and heir to Winterfell and the King in the North\\n* Jon Snow, bastard son of Eddard Stark, and a man of the Night's Watch\\n* Theon Greyjoy, heir to the Seastone Chair and former ward of Lord Eddard Stark\\n* Queen Daenerys Targaryen, the Unburnt and Mother of Dragons, of the Targaryen dynasty\n",
" 2196 2d9538e229b8272de9b6f74bf8cdd270 \\n==== ''A Game of Thrones'' ====\\nArya adopts a direwolf cub, which she names Nymeria after a legendary warrior queen. She travels with her father, Eddard, to King's Landing when he is made Hand of the King. Before she leaves, her half-brother Jon Snow has a smallsword made for her as a parting gift, which she names \"Needle\" after her least favorite ladylike activity.\\nWhile taking a walk together, Prince Joffrey and her sister Sansa happen upon Arya and her friend, the low-born butcher apprentice Mycah, sparring in the woods with broomsticks. Arya defends Mycah from Joffrey's torments and her direwolf Nymeria helps Arya fight off Joffrey, wounding his arm in the process. Knowing that Nymeria will likely be killed in retribution, Arya chases her wolf away; but Sansa's direwolf Lady is killed in Nymeria's stead and Mycah is hunted down and killed by Sandor Clegane, Joffrey's bodyguard.\\nIn King's Landing, her father discovers Arya's possession of Needle, but instead of confiscating it he arranges for fencing lessons under the Braavosi swordmaster Syrio Forel, who teaches her the style of fighting known as \"water dancing\". After her father's arrest, Syrio is killed protecting her and Arya narrowly escapes capture. She later witnesses the public execution of her father before falling under the protection of the Night's Watch recruiter Yoren.\n",
" 2209 2d9538e229b8272de9b6f74bf8cdd270 \\n====Season 7====\\nTaking the face of Walder Frey, Arya gathers the men of House Frey for a feast before killing them all with poisoned wine. Arya then journeys south, intending to travel to King's Landing to assassinate Cersei (now Queen of the Seven Kingdoms following the extinction of House Baratheon). However, Arya changes her mind after learning from Hot Pie that Jon has ousted House Bolton from Winterfell and has been crowned King in the North, and decides to return to her ancestral home. Along the way she encounters a wolf pack led by her long-lost direwolf Nymeria. Nymeria recognizes Arya, but she has grown feral and ignores Arya when she asks her to come North with her.\\nArriving at Winterfell, Arya finds that Jon has traveled to Dragonstone but is reunited with Sansa and Bran. Bran reveals his knowledge of Arya's kill list through greenseeing and presents her with a Valyrian steel dagger, which had been given to him by Littlefinger. Arya is also reunited with Brienne, who continues to serve the Starks, and manages to equal the female warrior during sparring despite her smaller size.\\nLittlefinger seeks to increase his influence on Sansa by driving a wedge between the Stark sisters. To this end, he allows Arya to witness him receiving a confidential message obtained from Maester Luwin's records. Arya breaks into Littlefinger's quarters to steal the message, which is a plea sent by Sansa following Ned's imprisonment to Robb imploring him to bend the knee to Joffrey. Outraged, Arya confronts Sansa and is unconvinced by her explanation that she did so to try and save Ned's life. Later, Arya catches Sansa looking at her collection of faces and threatens Sansa before leaving. Some time later, Sansa summons Arya to the great hall and begins an accusation of treason and murder; however, the accusation is directed towards Littlefinger, whose crimes have been discovered by Bran's greenseeing. Despite Littlefinger's pleas for mercy, Sansa sentences Littlefinger to death and Arya cuts his throat with the Valyrian steel dagger. The Stark sisters later resolve their differences, and acknowledge that the Starks must stay together to survive winter.\n",
" 568 647298c3d183ab1dba03a447092e8763 \\n=== Arya Stark ===\\nArya Stark is the third child and younger daughter of Eddard and Catelyn Stark. She serves as a POV character for 33 chapters throughout ''A Game of Thrones'', ''A Clash of Kings'', ''A Storm of Swords'', ''A Feast for Crows'', and ''A Dance with Dragons''. So far she is the only character to appear in all 5 books as a POV character.\\nIn the HBO television adaptation, she is portrayed by Maisie Williams.\n",
" 311 e7adfa870b0e451f0cacdab756e231e3 \\n===On the Kingsroad===\\nCity Watchmen search the caravan for Gendry but are turned away by Yoren. Gendry tells Arya Stark that he knows she is a girl, and she reveals she is actually Arya Stark after learning that her father met Gendry before he was executed.\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"04/28/2020 15:07:54 - INFO - haystack.finder - Reader is looking for detailed answer in 12569 chars ...\n"
]
}
],
"source": [
"# You can configure how many candidates the reader and retriever shall return\n",
"# The higher top_k_retriever, the better (but also the slower) your answers.\n",
"prediction = pipe.run(query=\"Who is the father of Arya Stark?\", top_k_retriever=10, top_k_reader=5)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# prediction = pipe.run(query=\"Who created the Dothraki vocabulary?\", top_k_reader=5)\n",
"# prediction = pipe.run(query=\"Who is the sister of Sansa?\", top_k_reader=5)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"pycharm": {
"is_executing": false,
"name": "#%%\n"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[ { 'answer': 'Eddard',\n",
" 'context': 's Nymeria after a legendary warrior queen. She travels '\n",
" \"with her father, Eddard, to King's Landing when he is made \"\n",
" 'Hand of the King. Before she leaves,'},\n",
" { 'answer': 'Ned',\n",
" 'context': '\\n'\n",
" '====Season 1====\\n'\n",
" 'Arya accompanies her father Ned and her sister Sansa to '\n",
" \"King's Landing. Before their departure, Arya's \"\n",
" 'half-brother Jon Snow gifts A'},\n",
" { 'answer': 'Lord Eddard and Catelyn Stark',\n",
" 'context': 'k of House Stark is the younger daughter and third child '\n",
" 'of Lord Eddard and Catelyn Stark of Winterfell. Ever the '\n",
" 'tomboy, Arya would rather be trainin'},\n",
" { 'answer': 'Lord Eddard Stark',\n",
" 'context': 'ark daughters.\\n'\n",
" 'During the Tourney of the Hand to honour her father Lord '\n",
" 'Eddard Stark, Sansa Stark is enchanted by the knights '\n",
" 'performing in the event.'},\n",
" { 'answer': 'Robert Baratheon',\n",
" 'context': 'hen Gendry gives it to Arya, he tells her he is the '\n",
" 'bastard son of Robert Baratheon. Aware of their chances of '\n",
" 'dying in the upcoming battle and Arya w'}]\n"
]
}
],
"source": [
"print_answers(prediction, details=\"minimal\")"
]
},
{
"cell_type": "markdown",
"source": [
"## About us\n",
"\n",
"This [Haystack](https://github.com/deepset-ai/haystack/) notebook was made with love by [deepset](https://deepset.ai/) in Berlin, Germany\n",
"\n",
"We bring NLP to the industry via open source! \n",
"Our focus: Industry specific language models & large scale QA systems. \n",
" \n",
"Some of our other work: \n",
"- [German BERT](https://deepset.ai/german-bert)\n",
"- [GermanQuAD and GermanDPR](https://deepset.ai/germanquad)\n",
"- [FARM](https://github.com/deepset-ai/FARM)\n",
"\n",
"Get in touch:\n",
"[Twitter](https://twitter.com/deepset_ai) | [LinkedIn](https://www.linkedin.com/company/deepset-ai/) | [Slack](https://haystack.deepset.ai/community/join) | [GitHub Discussions](https://github.com/deepset-ai/haystack/discussions) | [Website](https://deepset.ai)\n",
"\n",
"By the way: [we're hiring!](https://apply.workable.com/deepset/) "
],
"metadata": {
"collapsed": false
}
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
},
"pycharm": {
"stem_cell": {
"cell_type": "raw",
"source": [],
"metadata": {
"collapsed": false
}
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}
Reference in New Issue View Git Blame Copy Permalink