{ "cells": [ { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "# Copyright (c) 2024 Microsoft Corporation.\n", "# Licensed under the MIT License." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "import os\n", "\n", "import pandas as pd\n", "import tiktoken\n", "\n", "from graphrag.query.indexer_adapters import (\n", " read_indexer_communities,\n", " read_indexer_entities,\n", " read_indexer_reports,\n", ")\n", "from graphrag.query.llm.oai.chat_openai import ChatOpenAI\n", "from graphrag.query.llm.oai.typing import OpenaiApiType\n", "from graphrag.query.structured_search.global_search.community_context import (\n", " GlobalCommunityContext,\n", ")\n", "from graphrag.query.structured_search.global_search.search import GlobalSearch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Global Search example\n", "\n", "Global search method generates answers by searching over all AI-generated community reports in a map-reduce fashion. This is a resource-intensive method, but often gives good responses for questions that require an understanding of the dataset as a whole (e.g. What are the most significant values of the herbs mentioned in this notebook?)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### LLM setup" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "api_key = os.environ[\"GRAPHRAG_API_KEY\"]\n", "llm_model = os.environ[\"GRAPHRAG_LLM_MODEL\"]\n", "\n", "llm = ChatOpenAI(\n", " api_key=api_key,\n", " model=llm_model,\n", " api_type=OpenaiApiType.OpenAI, # OpenaiApiType.OpenAI or OpenaiApiType.AzureOpenAI\n", " max_retries=20,\n", ")\n", "\n", "token_encoder = tiktoken.encoding_for_model(llm_model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Load community reports as context for global search\n", "\n", "- Load all community reports in the `community_reports` table from GraphRAG, to be used as context data for global search.\n", "- Load entities from the `entities` tables from GraphRAG, to be used for calculating community weights for context ranking. Note that this is optional (if no entities are provided, we will not calculate community weights and only use the rank attribute in the community reports table for context ranking)\n", "- Load all communities in the `communities` table from the GraphRAG, to be used to reconstruct the community graph hierarchy for dynamic community selection." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "# parquet files generated from indexing pipeline\n", "INPUT_DIR = \"./inputs/operation dulce\"\n", "COMMUNITY_TABLE = \"communities\"\n", "COMMUNITY_REPORT_TABLE = \"community_reports\"\n", "ENTITY_TABLE = \"entities\"\n", "\n", "# community level in the Leiden community hierarchy from which we will load the community reports\n", "# higher value means we use reports from more fine-grained communities (at the cost of higher computation cost)\n", "COMMUNITY_LEVEL = 2" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Total report count: 72\n", "Report count after filtering by community level 2: 56\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
idhuman_readable_idcommunityparentleveltitlesummaryfull_contentrankrank_explanationfindingsfull_content_jsonperiodsize
016949a5d17b740b2b4a6f787b0a637f14343102Ben Bloomberg and the Harmoniser ProjectThe community centers around Ben Bloomberg, a ...# Ben Bloomberg and the Harmoniser Project\\n\\n...7.5The impact severity rating is high due to the ...[{'explanation': 'Ben Bloomberg is a pivotal f...{\\n \"title\": \"Ben Bloomberg and the Harmoni...2025-01-1035
14ff756b7041f4dcab6612e016af2b14d4444102North Hampton and Influential MusiciansThe community centers around North Hampton, a ...# North Hampton and Influential Musicians\\n\\nT...6.5The impact severity rating is moderately high ...[{'explanation': 'North Hampton serves as the ...{\\n \"title\": \"North Hampton and Influential...2025-01-104
22d3df394272743a781606ad80ccb53124545102Prince of Monaco and MonacoThe community revolves around the Prince of Mo...# Prince of Monaco and Monaco\\n\\nThe community...4.0The impact severity rating is moderate due to ...[{'explanation': 'The Prince of Monaco is a ke...{\\n \"title\": \"Prince of Monaco and Monaco\",...2025-01-102
3becbd958973f42b0bd53cca9250feaf14646102Robot Opera and BroadwayThe community revolves around the Robot Opera,...# Robot Opera and Broadway\\n\\nThe community re...7.5The impact severity rating is high due to the ...[{'explanation': 'The Robot Opera is a notable...{\\n \"title\": \"Robot Opera and Broadway\",\\n ...2025-01-102
4f7d29921ae3e41a79ae7f88dae5848924747132Ben and Jacob's Fusion of Art and TechnologyThe community centers around Ben and Jacob, wh...# Ben and Jacob's Fusion of Art and Technology...7.5The impact severity rating is high due to the ...[{'explanation': 'Ben and Jacob are key collab...{\\n \"title\": \"Ben and Jacob's Fusion of Art...2025-01-105
\n", "
" ], "text/plain": [ " id human_readable_id community parent \\\n", "0 16949a5d17b740b2b4a6f787b0a637f1 43 43 10 \n", "1 4ff756b7041f4dcab6612e016af2b14d 44 44 10 \n", "2 2d3df394272743a781606ad80ccb5312 45 45 10 \n", "3 becbd958973f42b0bd53cca9250feaf1 46 46 10 \n", "4 f7d29921ae3e41a79ae7f88dae584892 47 47 13 \n", "\n", " level title \\\n", "0 2 Ben Bloomberg and the Harmoniser Project \n", "1 2 North Hampton and Influential Musicians \n", "2 2 Prince of Monaco and Monaco \n", "3 2 Robot Opera and Broadway \n", "4 2 Ben and Jacob's Fusion of Art and Technology \n", "\n", " summary \\\n", "0 The community centers around Ben Bloomberg, a ... \n", "1 The community centers around North Hampton, a ... \n", "2 The community revolves around the Prince of Mo... \n", "3 The community revolves around the Robot Opera,... \n", "4 The community centers around Ben and Jacob, wh... \n", "\n", " full_content rank \\\n", "0 # Ben Bloomberg and the Harmoniser Project\\n\\n... 7.5 \n", "1 # North Hampton and Influential Musicians\\n\\nT... 6.5 \n", "2 # Prince of Monaco and Monaco\\n\\nThe community... 4.0 \n", "3 # Robot Opera and Broadway\\n\\nThe community re... 7.5 \n", "4 # Ben and Jacob's Fusion of Art and Technology... 7.5 \n", "\n", " rank_explanation \\\n", "0 The impact severity rating is high due to the ... \n", "1 The impact severity rating is moderately high ... \n", "2 The impact severity rating is moderate due to ... \n", "3 The impact severity rating is high due to the ... \n", "4 The impact severity rating is high due to the ... \n", "\n", " findings \\\n", "0 [{'explanation': 'Ben Bloomberg is a pivotal f... \n", "1 [{'explanation': 'North Hampton serves as the ... \n", "2 [{'explanation': 'The Prince of Monaco is a ke... \n", "3 [{'explanation': 'The Robot Opera is a notable... \n", "4 [{'explanation': 'Ben and Jacob are key collab... \n", "\n", " full_content_json period size \n", "0 {\\n \"title\": \"Ben Bloomberg and the Harmoni... 2025-01-10 35 \n", "1 {\\n \"title\": \"North Hampton and Influential... 2025-01-10 4 \n", "2 {\\n \"title\": \"Prince of Monaco and Monaco\",... 2025-01-10 2 \n", "3 {\\n \"title\": \"Robot Opera and Broadway\",\\n ... 2025-01-10 2 \n", "4 {\\n \"title\": \"Ben and Jacob's Fusion of Art... 2025-01-10 5 " ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "community_df = pd.read_parquet(f\"{INPUT_DIR}/{COMMUNITY_TABLE}.parquet\")\n", "entity_df = pd.read_parquet(f\"{INPUT_DIR}/{ENTITY_TABLE}.parquet\")\n", "report_df = pd.read_parquet(f\"{INPUT_DIR}/{COMMUNITY_REPORT_TABLE}.parquet\")\n", "\n", "communities = read_indexer_communities(community_df, report_df)\n", "reports = read_indexer_reports(report_df, community_df, COMMUNITY_LEVEL)\n", "entities = read_indexer_entities(entity_df, community_df, COMMUNITY_LEVEL)\n", "\n", "print(f\"Total report count: {len(report_df)}\")\n", "print(\n", " f\"Report count after filtering by community level {COMMUNITY_LEVEL}: {len(reports)}\"\n", ")\n", "\n", "report_df.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Build global context based on community reports" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [], "source": [ "context_builder = GlobalCommunityContext(\n", " community_reports=reports,\n", " communities=communities,\n", " entities=entities, # default to None if you don't want to use community weights for ranking\n", " token_encoder=token_encoder,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Perform global search" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "context_builder_params = {\n", " \"use_community_summary\": False, # False means using full community reports. True means using community short summaries.\n", " \"shuffle_data\": True,\n", " \"include_community_rank\": True,\n", " \"min_community_rank\": 0,\n", " \"community_rank_name\": \"rank\",\n", " \"include_community_weight\": True,\n", " \"community_weight_name\": \"occurrence weight\",\n", " \"normalize_community_weight\": True,\n", " \"max_tokens\": 12_000, # change this based on the token limit you have on your model (if you are using a model with 8k limit, a good setting could be 5000)\n", " \"context_name\": \"Reports\",\n", "}\n", "\n", "map_llm_params = {\n", " \"max_tokens\": 1000,\n", " \"temperature\": 0.0,\n", " \"response_format\": {\"type\": \"json_object\"},\n", "}\n", "\n", "reduce_llm_params = {\n", " \"max_tokens\": 2000, # change this based on the token limit you have on your model (if you are using a model with 8k limit, a good setting could be 1000-1500)\n", " \"temperature\": 0.0,\n", "}" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "search_engine = GlobalSearch(\n", " llm=llm,\n", " context_builder=context_builder,\n", " token_encoder=token_encoder,\n", " max_data_tokens=12_000, # change this based on the token limit you have on your model (if you are using a model with 8k limit, a good setting could be 5000)\n", " map_llm_params=map_llm_params,\n", " reduce_llm_params=reduce_llm_params,\n", " allow_general_knowledge=False, # set this to True will add instruction to encourage the LLM to incorporate general knowledge in the response, which may increase hallucinations, but could be useful in some use cases.\n", " json_mode=True, # set this to False if your LLM model does not support JSON mode.\n", " context_builder_params=context_builder_params,\n", " concurrent_coroutines=32,\n", " response_type=\"multiple paragraphs\", # free form text describing the response type and format, can be anything, e.g. prioritized list, single paragraph, multiple paragraphs, multiple-page report\n", ")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "### Cosmic Vocalization: An Overview\n", "\n", "Cosmic Vocalization is a term coined by Jordan Hayes to describe a repeating sequence found in cryptic communications. This concept is pivotal in the realm of interstellar communication, serving as a common reference point for both humanity and extraterrestrial entities. The repeating sequence identified by Hayes is crucial for understanding and interpreting the signals exchanged during what is known as the Interstellar Duet [Data: Reports (65)].\n", "\n", "### Key Figures and Entities\n", "\n", "#### Jordan Hayes\n", "Jordan Hayes is a central figure in the development and understanding of Cosmic Vocalization. Hayes' work in identifying and describing the repeating sequence in cryptic communications has been instrumental in the ongoing efforts to interact with extraterrestrial intelligence. His contributions provide the foundational knowledge required to decode and engage with these alien signals [Data: Reports (65)].\n", "\n", "#### Paranormal Military Squad\n", "The Paranormal Military Squad plays a significant role in activities related to Cosmic Vocalization. Their responsibilities include responding to alien signals and participating in the Interstellar Duet. This squad is part of a broader initiative aimed at facilitating interstellar communication, ensuring that humanity can effectively engage with extraterrestrial entities [Data: Reports (65)].\n", "\n", "### Implications\n", "\n", "The concept of Cosmic Vocalization and the involvement of key figures like Jordan Hayes and the Paranormal Military Squad highlight the collaborative efforts required to advance interstellar communication. By establishing a common ground for understanding cryptic signals, these efforts may pave the way for more profound interactions with extraterrestrial intelligence, potentially leading to significant advancements in our knowledge and capabilities.\n", "\n", "In summary, Cosmic Vocalization is a critical concept in the field of interstellar communication, with Jordan Hayes and the Paranormal Military Squad being key contributors to its development and application [Data: Reports (65)].\n" ] } ], "source": [ "result = await search_engine.search(\n", " \"What is Cosmic Vocalization and who are involved in it?\"\n", ")\n", "\n", "print(result.response)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
idtitleoccurrence weightcontentrank
050Alex Mercer and the Dulce Base Team0.956522# Alex Mercer and the Dulce Base Team\\n\\nThe c...8.5
135Kevin Scott and Technology Development0.608696# Kevin Scott and Technology Development\\n\\nTh...7.5
253Dulce Base and Paranormal Military Squad0.565217# Dulce Base and Paranormal Military Squad\\n\\n...8.5
322Paranormal Military Squad and Technological Ex...0.434783# Paranormal Military Squad and Technological ...8.5
460First Contact with Extraterrestrial Civilization0.304348# First Contact with Extraterrestrial Civiliza...9.5
561Dulce Base Operations and Distress0.173913# Dulce Base Operations and Distress\\n\\nThe co...8.5
657Operation: Dulce in New Mexico0.130435# Operation: Dulce in New Mexico\\n\\nThe commun...7.5
748Jacob Collier and Ben Bloomberg's First Tour0.130435# Jacob Collier and Ben Bloomberg's First Tour...7.5
851Cosmic Translators and Alien Script0.086957# Cosmic Translators and Alien Script\\n\\nThe c...8.5
964Terminal and Deep Hum0.086957# Terminal and Deep Hum\\n\\nThe community revol...8.5
1029Paranormal Military Squad and Cosmic Dialogue0.086957# Paranormal Military Squad and Cosmic Dialogu...8.5
1139Extraterrestrial Signal Decryption Community0.086957# Extraterrestrial Signal Decryption Community...8.5
1234Growth Mindset and Stanford0.086957# Growth Mindset and Stanford\\n\\nThe community...7.5
1320Jacob Collier and Taylor Swift's Albums0.086957# Jacob Collier and Taylor Swift's Albums\\n\\nT...7.5
140Omberg and Jacob Collier Collaboration0.086957# Omberg and Jacob Collier Collaboration\\n\\nTh...7.5
1565Galactic Orchestra and Interstellar Duet0.043478# Galactic Orchestra and Interstellar Duet\\n\\n...8.5
1659Alien Intelligence and Interstellar Siren's Call0.043478# Alien Intelligence and Interstellar Siren's ...8.5
1716Jimmy Fallon Project on Primetime Television0.043478# Jimmy Fallon Project on Primetime Television...6.5
1842Decryption Process and Digital Soundscape0.043478# Decryption Process and Digital Soundscape\\n\\...6.5
195Jacob Collier's Video Production0.043478# Jacob Collier's Video Production\\n\\nThe comm...4.0
2014Ben Bloomberg's Phone System and House0.043478# Ben Bloomberg's Phone System and House\\n\\nTh...3.0
219Jacob Collier's Video Production0.043478# Jacob Collier's Video Production\\n\\nThe comm...3.0
2217Ben Bloomberg's Phone System and Parental Conc...0.043478# Ben Bloomberg's Phone System and Parental Co...3.0
2358Paranormal Military Squad and Alien Communicat...0.956522# Paranormal Military Squad and Alien Communic...7.5
2452Paranormal Military Squad at Dulce Base0.695652# Paranormal Military Squad at Dulce Base\\n\\nT...8.5
2511Jacob Collier and His Musical Collaborations0.565217# Jacob Collier and His Musical Collaborations...8.5
2654Dr. Jordan Hayes and the Paranormal Military S...0.347826# Dr. Jordan Hayes and the Paranormal Military...8.5
2755Operation: Dulce and Paranormal Military Squad0.260870# Operation: Dulce and Paranormal Military Squ...8.5
2868Earth's Interstellar Communication and Galacti...0.260870# Earth's Interstellar Communication and Galac...8.5
2970Paranormal Military Squad and Interstellar Com...0.130435# Paranormal Military Squad and Interstellar C...8.5
3071Threshold and Humankind's Communication with E...0.130435# Threshold and Humankind's Communication with...8.5
3156Dulce Military Base and Paranormal Operations0.130435# Dulce Military Base and Paranormal Operation...8.5
3269Paranormal Military Squad and Interstellar Com...0.130435# Paranormal Military Squad and Interstellar C...8.5
3333Behind the Tech and Microsoft Community0.130435# Behind the Tech and Microsoft Community\\n\\nT...7.5
3449Djesse Vol. 3 and Djesse Albums Series0.130435# Djesse Vol. 3 and Djesse Albums Series\\n\\nTh...7.5
3566Humanity and Cosmic Relationships0.086957# Humanity and Cosmic Relationships\\n\\nThe com...8.5
3619Pandemic and Its Impact on Work and Art0.086957# Pandemic and Its Impact on Work and Art\\n\\nT...8.5
3767Decryption and Understanding of Alien Signal0.043478# Decryption and Understanding of Alien Signal...8.5
3812Montreux Jazz Festival and Key Performers0.043478# Montreux Jazz Festival and Key Performers\\n\\...7.5
3946Robot Opera and Broadway0.043478# Robot Opera and Broadway\\n\\nThe community re...7.5
4021Taylor Swift's Albums and Documentary0.043478# Taylor Swift's Albums and Documentary\\n\\nThe...7.0
417Stage Equipment and Transportation Network0.043478# Stage Equipment and Transportation Network\\n...6.5
4237Jaron Lanier and His Collection of Musical Ins...0.043478# Jaron Lanier and His Collection of Musical I...4.5
4345Prince of Monaco and Monaco0.043478# Prince of Monaco and Monaco\\n\\nThe community...4.0
4462Paranormal Military Squad at Dulce Base1.000000# Paranormal Military Squad at Dulce Base\\n\\nT...8.5
4563Paranormal Military Squad and Operation: Dulce0.782609# Paranormal Military Squad and Operation: Dul...9.0
4643Ben Bloomberg and the Harmoniser Project0.478261# Ben Bloomberg and the Harmoniser Project\\n\\n...7.5
4747Ben and Jacob's Fusion of Art and Technology0.173913# Ben and Jacob's Fusion of Art and Technology...7.5
4828Mission to Uncover Dulce's Mysteries0.130435# Mission to Uncover Dulce's Mysteries\\n\\nThe ...8.5
4918Taylor Swift and Album of the Year0.130435# Taylor Swift and Album of the Year\\n\\nThe co...7.5
5040Conversation between Kevin Scott and Jacob Col...0.086957# Conversation between Kevin Scott and Jacob C...8.5
5141Humanity and the Unseen Partner0.043478# Humanity and the Unseen Partner\\n\\nThe commu...8.5
5215Jimmy Fallon Project on Primetime TV0.043478# Jimmy Fallon Project on Primetime TV\\n\\nThe ...7.5
5338Kevin Scott and the Engineering Mindset0.043478# Kevin Scott and the Engineering Mindset\\n\\nT...6.5
5444North Hampton and Influential Musicians0.043478# North Hampton and Influential Musicians\\n\\nT...6.5
5536Kevin Scott's Daughter and Her Fantasy Novel0.043478# Kevin Scott's Daughter and Her Fantasy Novel...2.0
\n", "
" ], "text/plain": [ " id title occurrence weight \\\n", "0 50 Alex Mercer and the Dulce Base Team 0.956522 \n", "1 35 Kevin Scott and Technology Development 0.608696 \n", "2 53 Dulce Base and Paranormal Military Squad 0.565217 \n", "3 22 Paranormal Military Squad and Technological Ex... 0.434783 \n", "4 60 First Contact with Extraterrestrial Civilization 0.304348 \n", "5 61 Dulce Base Operations and Distress 0.173913 \n", "6 57 Operation: Dulce in New Mexico 0.130435 \n", "7 48 Jacob Collier and Ben Bloomberg's First Tour 0.130435 \n", "8 51 Cosmic Translators and Alien Script 0.086957 \n", "9 64 Terminal and Deep Hum 0.086957 \n", "10 29 Paranormal Military Squad and Cosmic Dialogue 0.086957 \n", "11 39 Extraterrestrial Signal Decryption Community 0.086957 \n", "12 34 Growth Mindset and Stanford 0.086957 \n", "13 20 Jacob Collier and Taylor Swift's Albums 0.086957 \n", "14 0 Omberg and Jacob Collier Collaboration 0.086957 \n", "15 65 Galactic Orchestra and Interstellar Duet 0.043478 \n", "16 59 Alien Intelligence and Interstellar Siren's Call 0.043478 \n", "17 16 Jimmy Fallon Project on Primetime Television 0.043478 \n", "18 42 Decryption Process and Digital Soundscape 0.043478 \n", "19 5 Jacob Collier's Video Production 0.043478 \n", "20 14 Ben Bloomberg's Phone System and House 0.043478 \n", "21 9 Jacob Collier's Video Production 0.043478 \n", "22 17 Ben Bloomberg's Phone System and Parental Conc... 0.043478 \n", "23 58 Paranormal Military Squad and Alien Communicat... 0.956522 \n", "24 52 Paranormal Military Squad at Dulce Base 0.695652 \n", "25 11 Jacob Collier and His Musical Collaborations 0.565217 \n", "26 54 Dr. Jordan Hayes and the Paranormal Military S... 0.347826 \n", "27 55 Operation: Dulce and Paranormal Military Squad 0.260870 \n", "28 68 Earth's Interstellar Communication and Galacti... 0.260870 \n", "29 70 Paranormal Military Squad and Interstellar Com... 0.130435 \n", "30 71 Threshold and Humankind's Communication with E... 0.130435 \n", "31 56 Dulce Military Base and Paranormal Operations 0.130435 \n", "32 69 Paranormal Military Squad and Interstellar Com... 0.130435 \n", "33 33 Behind the Tech and Microsoft Community 0.130435 \n", "34 49 Djesse Vol. 3 and Djesse Albums Series 0.130435 \n", "35 66 Humanity and Cosmic Relationships 0.086957 \n", "36 19 Pandemic and Its Impact on Work and Art 0.086957 \n", "37 67 Decryption and Understanding of Alien Signal 0.043478 \n", "38 12 Montreux Jazz Festival and Key Performers 0.043478 \n", "39 46 Robot Opera and Broadway 0.043478 \n", "40 21 Taylor Swift's Albums and Documentary 0.043478 \n", "41 7 Stage Equipment and Transportation Network 0.043478 \n", "42 37 Jaron Lanier and His Collection of Musical Ins... 0.043478 \n", "43 45 Prince of Monaco and Monaco 0.043478 \n", "44 62 Paranormal Military Squad at Dulce Base 1.000000 \n", "45 63 Paranormal Military Squad and Operation: Dulce 0.782609 \n", "46 43 Ben Bloomberg and the Harmoniser Project 0.478261 \n", "47 47 Ben and Jacob's Fusion of Art and Technology 0.173913 \n", "48 28 Mission to Uncover Dulce's Mysteries 0.130435 \n", "49 18 Taylor Swift and Album of the Year 0.130435 \n", "50 40 Conversation between Kevin Scott and Jacob Col... 0.086957 \n", "51 41 Humanity and the Unseen Partner 0.043478 \n", "52 15 Jimmy Fallon Project on Primetime TV 0.043478 \n", "53 38 Kevin Scott and the Engineering Mindset 0.043478 \n", "54 44 North Hampton and Influential Musicians 0.043478 \n", "55 36 Kevin Scott's Daughter and Her Fantasy Novel 0.043478 \n", "\n", " content rank \n", "0 # Alex Mercer and the Dulce Base Team\\n\\nThe c... 8.5 \n", "1 # Kevin Scott and Technology Development\\n\\nTh... 7.5 \n", "2 # Dulce Base and Paranormal Military Squad\\n\\n... 8.5 \n", "3 # Paranormal Military Squad and Technological ... 8.5 \n", "4 # First Contact with Extraterrestrial Civiliza... 9.5 \n", "5 # Dulce Base Operations and Distress\\n\\nThe co... 8.5 \n", "6 # Operation: Dulce in New Mexico\\n\\nThe commun... 7.5 \n", "7 # Jacob Collier and Ben Bloomberg's First Tour... 7.5 \n", "8 # Cosmic Translators and Alien Script\\n\\nThe c... 8.5 \n", "9 # Terminal and Deep Hum\\n\\nThe community revol... 8.5 \n", "10 # Paranormal Military Squad and Cosmic Dialogu... 8.5 \n", "11 # Extraterrestrial Signal Decryption Community... 8.5 \n", "12 # Growth Mindset and Stanford\\n\\nThe community... 7.5 \n", "13 # Jacob Collier and Taylor Swift's Albums\\n\\nT... 7.5 \n", "14 # Omberg and Jacob Collier Collaboration\\n\\nTh... 7.5 \n", "15 # Galactic Orchestra and Interstellar Duet\\n\\n... 8.5 \n", "16 # Alien Intelligence and Interstellar Siren's ... 8.5 \n", "17 # Jimmy Fallon Project on Primetime Television... 6.5 \n", "18 # Decryption Process and Digital Soundscape\\n\\... 6.5 \n", "19 # Jacob Collier's Video Production\\n\\nThe comm... 4.0 \n", "20 # Ben Bloomberg's Phone System and House\\n\\nTh... 3.0 \n", "21 # Jacob Collier's Video Production\\n\\nThe comm... 3.0 \n", "22 # Ben Bloomberg's Phone System and Parental Co... 3.0 \n", "23 # Paranormal Military Squad and Alien Communic... 7.5 \n", "24 # Paranormal Military Squad at Dulce Base\\n\\nT... 8.5 \n", "25 # Jacob Collier and His Musical Collaborations... 8.5 \n", "26 # Dr. Jordan Hayes and the Paranormal Military... 8.5 \n", "27 # Operation: Dulce and Paranormal Military Squ... 8.5 \n", "28 # Earth's Interstellar Communication and Galac... 8.5 \n", "29 # Paranormal Military Squad and Interstellar C... 8.5 \n", "30 # Threshold and Humankind's Communication with... 8.5 \n", "31 # Dulce Military Base and Paranormal Operation... 8.5 \n", "32 # Paranormal Military Squad and Interstellar C... 8.5 \n", "33 # Behind the Tech and Microsoft Community\\n\\nT... 7.5 \n", "34 # Djesse Vol. 3 and Djesse Albums Series\\n\\nTh... 7.5 \n", "35 # Humanity and Cosmic Relationships\\n\\nThe com... 8.5 \n", "36 # Pandemic and Its Impact on Work and Art\\n\\nT... 8.5 \n", "37 # Decryption and Understanding of Alien Signal... 8.5 \n", "38 # Montreux Jazz Festival and Key Performers\\n\\... 7.5 \n", "39 # Robot Opera and Broadway\\n\\nThe community re... 7.5 \n", "40 # Taylor Swift's Albums and Documentary\\n\\nThe... 7.0 \n", "41 # Stage Equipment and Transportation Network\\n... 6.5 \n", "42 # Jaron Lanier and His Collection of Musical I... 4.5 \n", "43 # Prince of Monaco and Monaco\\n\\nThe community... 4.0 \n", "44 # Paranormal Military Squad at Dulce Base\\n\\nT... 8.5 \n", "45 # Paranormal Military Squad and Operation: Dul... 9.0 \n", "46 # Ben Bloomberg and the Harmoniser Project\\n\\n... 7.5 \n", "47 # Ben and Jacob's Fusion of Art and Technology... 7.5 \n", "48 # Mission to Uncover Dulce's Mysteries\\n\\nThe ... 8.5 \n", "49 # Taylor Swift and Album of the Year\\n\\nThe co... 7.5 \n", "50 # Conversation between Kevin Scott and Jacob C... 8.5 \n", "51 # Humanity and the Unseen Partner\\n\\nThe commu... 8.5 \n", "52 # Jimmy Fallon Project on Primetime TV\\n\\nThe ... 7.5 \n", "53 # Kevin Scott and the Engineering Mindset\\n\\nT... 6.5 \n", "54 # North Hampton and Influential Musicians\\n\\nT... 6.5 \n", "55 # Kevin Scott's Daughter and Her Fantasy Novel... 2.0 " ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# inspect the data used to build the context for the LLM responses\n", "result.context_data[\"reports\"]" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "LLM calls: 4. Prompt tokens: 33015. Output tokens: 655.\n" ] } ], "source": [ "# inspect number of LLM calls and tokens\n", "print(\n", " f\"LLM calls: {result.llm_calls}. Prompt tokens: {result.prompt_tokens}. Output tokens: {result.output_tokens}.\"\n", ")" ] } ], "metadata": { "kernelspec": { "display_name": "graphrag-ta_-cxM1-py3.10", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.12" } }, "nbformat": 4, "nbformat_minor": 2 }