![](\"https://raw.githubusercontent.com/deepset-ai/haystack/master/docs/_src/img/colab_gpu_runtime.jpg\")
"
],
"metadata": {
"collapsed": false,
"id": "lEKOjCS5U7so"
}
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"# Make sure you have a GPU running\n",
"!nvidia-smi"
],
"outputs": [],
"metadata": {
"pycharm": {
"name": "#%%\n"
},
"id": "xhFIMX_7U7ss",
"outputId": "285b2491-01e5-4bfd-cba9-c2279d4417c4",
"colab": {
"base_uri": "https://localhost:8080/"
}
}
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"# Install the latest release of Haystack in your own environment \n",
"#! pip install farm-haystack\n",
"\n",
"# Install the latest master of Haystack\n",
"!pip install grpcio-tools==1.34.1\n",
"!pip install git+https://github.com/deepset-ai/haystack.git\n",
"\n",
"# If you run this notebook on Google Colab, you might need to\n",
"# restart the runtime after installing haystack."
],
"outputs": [],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "vgmFOp82Oht_",
"pycharm": {
"name": "#%%\n"
},
"outputId": "5bbcbb42-3a90-43a9-ebfd-598a98fa7143"
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"from haystack.modeling.utils import initialize_device_settings\n",
"devices, n_gpu = initialize_device_settings(use_cuda=True)"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Start an Elasticsearch server\n",
"You can start Elasticsearch on your local machine instance using Docker. If Docker is not readily available in your environment (eg., in Colab notebooks), then you can manually download and execute Elasticsearch from source."
],
"metadata": {
"collapsed": false
}
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"# If Docker is available: Start Elasticsearch as docker container\n",
"# from haystack.utils import launch_es\n",
"# launch_es()\n",
"\n",
"# Alternative in Colab / No Docker environments: Start Elasticsearch from source\n",
"! wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.9.2-linux-x86_64.tar.gz -q\n",
"! tar -xzf elasticsearch-7.9.2-linux-x86_64.tar.gz\n",
"! chown -R daemon:daemon elasticsearch-7.9.2\n",
"\n",
"import os\n",
"from subprocess import Popen, PIPE, STDOUT\n",
"es_server = Popen(['elasticsearch-7.9.2/bin/elasticsearch'],\n",
" stdout=PIPE, stderr=STDOUT,\n",
" preexec_fn=lambda: os.setuid(1) # as daemon\n",
" )\n",
"# wait until ES has started\n",
"! sleep 30"
],
"outputs": [],
"metadata": {
"id": "tNoaWcDKOhuL",
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "markdown",
"source": [
"## Fetch, Store And Preprocess the Evaluation Dataset"
],
"metadata": {
"collapsed": false
}
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"from haystack.utils import fetch_archive_from_http\n",
"\n",
"# Download evaluation data, which is a subset of Natural Questions development set containing 50 documents with one question per document and multiple annotated answers\n",
"doc_dir = \"../data/nq\"\n",
"s3_url = \"https://s3.eu-central-1.amazonaws.com/deepset.ai-farm-qa/datasets/nq_dev_subset_v2.json.zip\"\n",
"fetch_archive_from_http(url=s3_url, output_dir=doc_dir)"
],
"outputs": [],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "tTXxr6TAOhuz",
"pycharm": {
"name": "#%%\n"
},
"outputId": "586d4775-4354-4ed9-a72c-c30bedcdfbee"
}
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"# make sure these indices do not collide with existing ones, the indices will be wiped clean before data is inserted\n",
"doc_index = \"tutorial5_docs\"\n",
"label_index = \"tutorial5_labels\""
],
"outputs": [],
"metadata": {
"id": "T-G7Ay2jU7s_"
}
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"# Connect to Elasticsearch\n",
"from haystack.document_stores import ElasticsearchDocumentStore\n",
"\n",
"# Connect to Elasticsearch\n",
"document_store = ElasticsearchDocumentStore(host=\"localhost\", username=\"\", password=\"\", index=doc_index,\n",
" label_index=label_index, embedding_field=\"emb\",\n",
" embedding_dim=768, excluded_meta_data=[\"emb\"])"
],
"outputs": [],
"metadata": {
"id": "B_NEtezLOhu5",
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"from haystack.nodes import PreProcessor\n",
"\n",
"# Add evaluation data to Elasticsearch Document Store\n",
"# We first delete the custom tutorial indices to not have duplicate elements\n",
"# and also split our documents into shorter passages using the PreProcessor\n",
"preprocessor = PreProcessor(\n",
" split_length=200,\n",
" split_overlap=0,\n",
" split_respect_sentence_boundary=False,\n",
" clean_empty_lines=False,\n",
" clean_whitespace=False\n",
")\n",
"document_store.delete_documents(index=doc_index)\n",
"document_store.delete_documents(index=label_index)\n",
"\n",
"# The add_eval_data() method converts the given dataset in json format into Haystack document and label objects. Those objects are then indexed in their respective document and label index in the document store. The method can be used with any dataset in SQuAD format.\n",
"document_store.add_eval_data(\n",
" filename=\"../data/nq/nq_dev_subset_v2.json\",\n",
" doc_index=doc_index,\n",
" label_index=label_index,\n",
" preprocessor=preprocessor\n",
")"
],
"outputs": [],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "bRFsQUAJOhu_",
"pycharm": {
"name": "#%%\n"
},
"outputId": "477031b9-5c2c-4128-ef5f-54db86259734"
}
},
{
"cell_type": "markdown",
"source": [
"## Initialize the Two Components of an ExtractiveQAPipeline: Retriever and Reader"
],
"metadata": {
"id": "gy8YwmSYOhvE",
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"# Initialize Retriever\n",
"from haystack.nodes import ElasticsearchRetriever\n",
"retriever = ElasticsearchRetriever(document_store=document_store)\n",
"# Alternative: Evaluate dense retrievers (DensePassageRetriever or EmbeddingRetriever)\n",
"# DensePassageRetriever uses two separate transformer based encoders for query and document.\n",
"# In contrast, EmbeddingRetriever uses a single encoder for both.\n",
"# Please make sure the \"embedding_dim\" parameter in the DocumentStore above matches the output dimension of your models!\n",
"# Please also take care that the PreProcessor splits your files into chunks that can be completely converted with\n",
"# the max_seq_len limitations of Transformers\n",
"# The SentenceTransformer model \"all-mpnet-base-v2\" generally works well with the EmbeddingRetriever on any kind of English text.\n",
"# For more information check out the documentation at: https://www.sbert.net/docs/pretrained_models.html\n",
"# from haystack.retriever import DensePassageRetriever, EmbeddingRetriever\n",
"# retriever = DensePassageRetriever(document_store=document_store,\n",
"# query_embedding_model=\"facebook/dpr-question_encoder-single-nq-base\",\n",
"# passage_embedding_model=\"facebook/dpr-ctx_encoder-single-nq-base\",\n",
"# use_gpu=True,\n",
"# max_seq_len_passage=256,\n",
"# embed_title=True)\n",
"# retriever = EmbeddingRetriever(document_store=document_store, model_format=\"sentence_transformers\",\n",
"# embedding_model=\"all-mpnet-base-v2\")\n",
"# document_store.update_embeddings(retriever, index=doc_index)"
],
"outputs": [],
"metadata": {
"id": "JkhaPMIJOhvF",
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"# Initialize Reader\n",
"from haystack.nodes import FARMReader\n",
"\n",
"reader = FARMReader(\"deepset/roberta-base-squad2\", top_k=4, return_no_answer=True)\n",
"\n",
"# Define a pipeline consisting of the initialized retriever and reader\n",
"from haystack.pipelines import ExtractiveQAPipeline\n",
"pipeline = ExtractiveQAPipeline(reader=reader, retriever=retriever)\n",
"\n",
"# The evaluation also works with any other pipeline.\n",
"# For example you could use a DocumentSearchPipeline as an alternative:\n",
"# from haystack.pipelines import DocumentSearchPipeline\n",
"# pipeline = DocumentSearchPipeline(retriever=retriever)"
],
"outputs": [],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 313,
"referenced_widgets": [
"118aedffeecd4f109ae04a4561baeb08",
"59efb57b419242a1aba4d20084e29d38",
"ddaf59cedca143c8b4fe005a51077323",
"25d7818d6f7b4b628ab3f83f2c2fa6a3",
"48431fa7696540fc9696799f75166680",
"a21f4958105f4b3ca32f0977bcfd7d48",
"3d211e6614f4451e9d14866cb3a8332d",
"3df3da3d04d5448c810bd00f66bd3a0e",
"8dfc1480100f43e0b4ea2fc2fb9279d3",
"0618a236cc14473f92257aebbc3d930d",
"8e0958b5dc27412e9f0332da3457ffdb",
"6f7ddc1720344ba9b939a8e4ac593d67",
"b480c97c1d9944b9a8dd09ed6e1e9bd3",
"ed683929822b4084ba33e89b23936b16",
"94c88d0fc3f949fbacfb6b4fcd99cc63",
"b262e92ff6484405a0e9364f6ecafb6a",
"bacf1704dbaf4176afbe2cbcc8e036ef",
"fe56b1d86ab84675b82781a1f8edd40a",
"d7e3c8e1e0424cec9dc1b97090b5af87",
"98da94a8d7b94fb4a08adcebea15e114",
"13b75146701145808315dc87d598b3f9",
"2bd3bd51ae644c1894a2ddca09d14e85",
"59039df74ce64c2f9e91663b6498c29c",
"9e130b97f0f4463f85df834d0f99d6ef",
"2c028a3f096344d68071d78387efa117",
"4d922fb301f944fbb0218335a28cf6e5",
"05d82995d5a94b5db39bf639d1cc05c2",
"76f4af76b42f460fa34d5f00a9656dc5",
"73d7fdd3f38349b4882124d8351eace5",
"ea439e2251ed467fb3a775f0c8e0c3bb",
"b6729cc6ba084677af55ac63c819b72f",
"fc011913e8464d439a97fe75ef5f9fa1",
"8a9f9b7bab8e40278430a35720066a61",
"5db857b352964db3a617568ff1dce86d",
"7752437041f745a4af4b9130df3fefa7",
"5f94d400ea884c1cadfc966e44849b3a",
"0d34710578ac4c1db6fe372b5d1215b4",
"994ae85181664e2e87a2ee18a7a237ba",
"368a61e33c3144bfa3cc94af10691146",
"ccfd1a0b6f494d8a9d78e7387261fba8",
"db57281b3d7a448fbd2d63d8f127ea3e",
"978b165c69dd4e14b8479ea7bd8cb1e5",
"6f028c7e888e4ae5ab5c1e42ff142b5f",
"c8ba8c2a210b45f6a9b5257589babac3",
"36f855f41cc1488f9d92ec34bb8d30b0",
"1d5d220bedc54dbdbacb9c43767bf64d",
"0523b10429d04f3d81d7078a13a12168",
"64cd5b6f0c4d4631a1049ee7ee50f063",
"eb11ea5785284bf6a15cc31ad643ed88",
"807e4eee3b2c440c8799afcc6344ff5d",
"6ca6dc2c6b4349fcb39ed8c44f65bdb0",
"4837b34ccb4d4688865dc24dc58a7c1e",
"d4dbfa5e89e7432dbed34606a786fd6f",
"7e058076836e438daf5399428eabac5e",
"a529dbbfdd6b469dbfe80cee993c9a33",
"69750fea2e7149eab8928282ba9bae29",
"08a1d1a6fb884c769d409170d6cda556",
"548ef6c85056414cb0ce79164a086d35",
"cf58b340496b4d62b610451cedbd709a",
"4dad7e58cc47436aafe38230514325a1",
"f36dc87a4af7481cb3c2cba23d57eb5a",
"2a9ef1f2f43d47b28cd0ff7ef4a21ade",
"caa374f7dc5045218c6f71f322d8e6be",
"e567fab4446544f795be2eb0a6705f9c",
"2b17ffac93b8406fac55b695d93d963b",
"3096cae7388e4b988df306be9cc58afd"
]
},
"id": "cW3Ypn_gOhvK",
"pycharm": {
"name": "#%%\n"
},
"outputId": "4b5feff7-ae9f-4cd8-de1e-944f0eb66f66"
}
},
{
"cell_type": "markdown",
"source": [
"## Evaluation of an ExtractiveQAPipeline\n",
"Here we evaluate retriever and reader in open domain fashion on the full corpus of documents i.e. a document is considered\n",
"correctly retrieved if it contains the gold answer string within it. The reader is evaluated based purely on the\n",
"predicted answer string, regardless of which document this came from and the position of the extracted span.\n",
"\n",
"The generation of predictions is seperated from the calculation of metrics. This allows you to run the computation-heavy model predictions only once and then iterate flexibly on the metrics or reports you want to generate.\n"
],
"metadata": {
"id": "7i84KXONOhvc",
"pycharm": {
"name": "#%% md\n"
}
}
},
{
"cell_type": "code",
"execution_count": null,
"outputs": [],
"source": [
"from haystack.schema import EvaluationResult, MultiLabel\n",
"\n",
"# We can load evaluation labels from the document store\n",
"eval_labels = document_store.get_all_labels_aggregated(drop_negative_labels=True, drop_no_answers=False)\n",
"\n",
"# Alternative: Define queries and labels directly\n",
"# from haystack.schema import Answer, Document, Label, Span\n",
"# eval_labels = [\n",
"# MultiLabel(labels=[Label(query=\"who is written in the book of life\",\n",
"# answer=Answer(answer=\"every person who is destined for Heaven or the World to Come\",\n",
"# offsets_in_context=[Span(374, 434)]),\n",
"# document=Document(id='1b090aec7dbd1af6739c4c80f8995877-0',\n",
"# content_type=\"text\",\n",
"# content='Book of Life - wikipedia Book of Life Jump to: navigation, search This article is about the book mentioned in Christian and Jewish religious teachings. For other uses, see The Book of Life. In Christianity and Judaism, the Book of Life (Hebrew: ספר החיים, transliterated Sefer HaChaim; Greek: βιβλίον τῆς ζωῆς Biblíon tēs Zōēs) is the book in which God records the names of every person who is destined for Heaven or the World to Come. According to the Talmud it is open on Rosh Hashanah, as is its analog for the wicked, the Book of the Dead. For this reason extra mention is made for the Book of Life during Amidah recitations during the Days of Awe, the ten days between Rosh Hashanah, the Jewish new year, and Yom Kippur, the day of atonement (the two High Holidays, particularly in the prayer Unetaneh Tokef). Contents (hide) 1 In the Hebrew Bible 2 Book of Jubilees 3 References in the New Testament 4 The eschatological or annual roll-call 5 Fundraising 6 See also 7 Notes 8 References In the Hebrew Bible(edit) In the Hebrew Bible the Book of Life - the book or muster-roll of God - records forever all people considered righteous before God'),\n",
"# is_correct_answer=True,\n",
"# is_correct_document=True,\n",
"# origin=\"gold-label\")])\n",
"# ]\n",
"\n",
"# Similar to pipeline.run() we can execute pipeline.eval()\n",
"eval_result = pipeline.eval(\n",
" labels=eval_labels,\n",
" params={\"Retriever\": {\"top_k\": 5}}\n",
")"
],
"metadata": {
"collapsed": false,
"pycharm": {
"name": "#%%\n"
}
}
},
{
"cell_type": "code",
"execution_count": 9,
"outputs": [
{
"data": {
"text/plain": " content \\\n0 people considered righteous before God. God has such a book, and to be blott... \n1 as adversaries (of God). Also, according to ib. xxxvi. 10, one who contrives... \n2 the citizens' registers. The life which the righteous participate in is to b... \n3 apostles' names are ``written in heaven'' (Luke x. 20), or ``the fellow-work... \n4 The Absolutely True Diary of a Part-Time Indian - wikipedia The Absolutely T... \n\n document_id type \\\n0 1b090aec7dbd1af6739c4c80f8995877-1 document \n1 1b090aec7dbd1af6739c4c80f8995877-2 document \n2 1b090aec7dbd1af6739c4c80f8995877-6 document \n3 1b090aec7dbd1af6739c4c80f8995877-3 document \n4 e9260cbbc129f4246ee8fcfbbe385822-0 document \n\n gold_document_ids \\\n0 [1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] \n1 [1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] \n2 [1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] \n3 [1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] \n4 [1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] \n\n gold_document_contents \\\n0 [Book of Life - wikipedia Book of Life Jump to: navigation, search This arti... \n1 [Book of Life - wikipedia Book of Life Jump to: navigation, search This arti... \n2 [Book of Life - wikipedia Book of Life Jump to: navigation, search This arti... \n3 [Book of Life - wikipedia Book of Life Jump to: navigation, search This arti... \n4 [Book of Life - wikipedia Book of Life Jump to: navigation, search This arti... \n\n gold_id_match answer_match gold_id_or_answer_match rank node \\\n0 0.0 0.0 0.0 1 Retriever \n1 0.0 0.0 0.0 2 Retriever \n2 0.0 0.0 0.0 3 Retriever \n3 0.0 0.0 0.0 4 Retriever \n4 0.0 0.0 0.0 5 Retriever \n\n query \n0 who is written in the book of life \n1 who is written in the book of life \n2 who is written in the book of life \n3 who is written in the book of life \n4 who is written in the book of life ",
"text/html": "| \n | content | \ndocument_id | \ntype | \ngold_document_ids | \ngold_document_contents | \ngold_id_match | \nanswer_match | \ngold_id_or_answer_match | \nrank | \nnode | \nquery | \n
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \npeople considered righteous before God. God has such a book, and to be blott... | \n1b090aec7dbd1af6739c4c80f8995877-1 | \ndocument | \n[1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] | \n[Book of Life - wikipedia Book of Life Jump to: navigation, search This arti... | \n0.0 | \n0.0 | \n0.0 | \n1 | \nRetriever | \nwho is written in the book of life | \n
| 1 | \nas adversaries (of God). Also, according to ib. xxxvi. 10, one who contrives... | \n1b090aec7dbd1af6739c4c80f8995877-2 | \ndocument | \n[1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] | \n[Book of Life - wikipedia Book of Life Jump to: navigation, search This arti... | \n0.0 | \n0.0 | \n0.0 | \n2 | \nRetriever | \nwho is written in the book of life | \n
| 2 | \nthe citizens' registers. The life which the righteous participate in is to b... | \n1b090aec7dbd1af6739c4c80f8995877-6 | \ndocument | \n[1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] | \n[Book of Life - wikipedia Book of Life Jump to: navigation, search This arti... | \n0.0 | \n0.0 | \n0.0 | \n3 | \nRetriever | \nwho is written in the book of life | \n
| 3 | \napostles' names are ``written in heaven'' (Luke x. 20), or ``the fellow-work... | \n1b090aec7dbd1af6739c4c80f8995877-3 | \ndocument | \n[1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] | \n[Book of Life - wikipedia Book of Life Jump to: navigation, search This arti... | \n0.0 | \n0.0 | \n0.0 | \n4 | \nRetriever | \nwho is written in the book of life | \n
| 4 | \nThe Absolutely True Diary of a Part-Time Indian - wikipedia The Absolutely T... | \ne9260cbbc129f4246ee8fcfbbe385822-0 | \ndocument | \n[1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] | \n[Book of Life - wikipedia Book of Life Jump to: navigation, search This arti... | \n0.0 | \n0.0 | \n0.0 | \n5 | \nRetriever | \nwho is written in the book of life | \n
| \n | answer | \ndocument_id | \noffsets_in_document | \ncontext | \ntype | \ngold_answers | \ngold_offsets_in_documents | \ngold_document_ids | \nexact_match | \nf1 | \nrank | \nnode | \nquery | \n
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n\n | None | \n[{'start': 0, 'end': 0}] | \nNone | \nanswer | \n[all people considered righteous before God, every person who is destined fo... | \n[[Span(start=1107, end=1149)], [Span(start=374, end=434)]] | \n[1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] | \n0.0 | \n0.000000 | \n1 | \nReader | \nwho is written in the book of life | \n
| 1 | \nthose whose names are written in the Book of Life from the foundation of the... | \n1b090aec7dbd1af6739c4c80f8995877-3 | \n[{'start': 576, 'end': 658}] | \nohn of Patmos. As described, only those whose names are written in the Book ... | \nanswer | \n[all people considered righteous before God, every person who is destined fo... | \n[[Span(start=1107, end=1149)], [Span(start=374, end=434)]] | \n[1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] | \n0.0 | \n0.083333 | \n2 | \nReader | \nwho is written in the book of life | \n
| 2 | \nonly the names of the righteous | \n1b090aec7dbd1af6739c4c80f8995877-1 | \n[{'start': 498, 'end': 529}] | \n. The Psalmist likewise speaks of the Book of Life in which only the names o... | \nanswer | \n[all people considered righteous before God, every person who is destined fo... | \n[[Span(start=1107, end=1149)], [Span(start=374, end=434)]] | \n[1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] | \n0.0 | \n0.200000 | \n3 | \nReader | \nwho is written in the book of life | \n
| 3 | \nthose who are found written in the book and who shall escape the troubles pr... | \n1b090aec7dbd1af6739c4c80f8995877-6 | \n[{'start': 135, 'end': 305}] | \nthose who are found written in the book and who shall escape the troubles pr... | \nanswer | \n[all people considered righteous before God, every person who is destined fo... | \n[[Span(start=1107, end=1149)], [Span(start=374, end=434)]] | \n[1b090aec7dbd1af6739c4c80f8995877-0, 1b090aec7dbd1af6739c4c80f8995877-0] | \n0.0 | \n0.111111 | \n4 | \nReader | \nwho is written in the book of life | \n
| 0 | \nher cousin | \n965a125f65658579529b39f8e4344969-3 | \n[{'start': 423, 'end': 433}] | \nng a story in the newspaper of a 12-year-old girl getting pregnant by her co... | \nanswer | \n[Ethel ``Edy'' Proctor] | \n[[Span(start=181, end=202)]] | \n[965a125f65658579529b39f8e4344969-3] | \n0.0 | \n0.000000 | \n1 | \nReader | \nwho was the girl in the video brenda got a baby | \n