yujunjun/graphrag
1
0
Fork 0
mirror of https://github.com/microsoft/graphrag.git synced 2025-11-03 11:20:38 +00:00
Code Issues Packages Projects Releases Wiki Activity

Deploying to gh-pages from @ microsoft/graphrag@dad2176b3c 🚀

Browse Source
This commit is contained in:
jgbradley1 2024-11-27 18:31:01 +00:00
parent b32f885be7
commit dfb59de794
9 changed files with 2553 additions and 2550 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

42
cli/index.html
View File

@ -1406,19 +1406,17 @@
<a id="__codelineno-3-7" name="__codelineno-3-7" href="#__codelineno-3-7"></a> --resume TEXT Resume a given indexing run
<a id="__codelineno-3-8" name="__codelineno-3-8" href="#__codelineno-3-8"></a> --reporter [rich|print|none] The progress reporter to use. [default:
<a id="__codelineno-3-9" name="__codelineno-3-9" href="#__codelineno-3-9"></a> rich]
<a id="__codelineno-3-10" name="__codelineno-3-10" href="#__codelineno-3-10"></a> --emit TEXT The data formats to emit, comma-separated.
<a id="__codelineno-3-11" name="__codelineno-3-11" href="#__codelineno-3-11"></a> [default: parquet]
<a id="__codelineno-3-12" name="__codelineno-3-12" href="#__codelineno-3-12"></a> --dry-run / --no-dry-run Run the indexing pipeline without executing
<a id="__codelineno-3-13" name="__codelineno-3-13" href="#__codelineno-3-13"></a> any steps to inspect and validate the
<a id="__codelineno-3-14" name="__codelineno-3-14" href="#__codelineno-3-14"></a> configuration. [default: no-dry-run]
<a id="__codelineno-3-15" name="__codelineno-3-15" href="#__codelineno-3-15"></a> --cache / --no-cache Use LLM cache. [default: cache]
<a id="__codelineno-3-16" name="__codelineno-3-16" href="#__codelineno-3-16"></a> --skip-validation / --no-skip-validation
<a id="__codelineno-3-17" name="__codelineno-3-17" href="#__codelineno-3-17"></a> Skip any preflight validation. Useful when
<a id="__codelineno-3-18" name="__codelineno-3-18" href="#__codelineno-3-18"></a> running no LLM steps. [default: no-skip-
<a id="__codelineno-3-19" name="__codelineno-3-19" href="#__codelineno-3-19"></a> validation]
<a id="__codelineno-3-20" name="__codelineno-3-20" href="#__codelineno-3-20"></a> --output PATH Indexing pipeline output directory.
<a id="__codelineno-3-21" name="__codelineno-3-21" href="#__codelineno-3-21"></a> Overrides storage.base_dir in the
<a id="__codelineno-3-22" name="__codelineno-3-22" href="#__codelineno-3-22"></a> configuration file.
<a id="__codelineno-3-10" name="__codelineno-3-10" href="#__codelineno-3-10"></a> --dry-run / --no-dry-run Run the indexing pipeline without executing
<a id="__codelineno-3-11" name="__codelineno-3-11" href="#__codelineno-3-11"></a> any steps to inspect and validate the
<a id="__codelineno-3-12" name="__codelineno-3-12" href="#__codelineno-3-12"></a> configuration. [default: no-dry-run]
<a id="__codelineno-3-13" name="__codelineno-3-13" href="#__codelineno-3-13"></a> --cache / --no-cache Use LLM cache. [default: cache]
<a id="__codelineno-3-14" name="__codelineno-3-14" href="#__codelineno-3-14"></a> --skip-validation / --no-skip-validation
<a id="__codelineno-3-15" name="__codelineno-3-15" href="#__codelineno-3-15"></a> Skip any preflight validation. Useful when
<a id="__codelineno-3-16" name="__codelineno-3-16" href="#__codelineno-3-16"></a> running no LLM steps. [default: no-skip-
<a id="__codelineno-3-17" name="__codelineno-3-17" href="#__codelineno-3-17"></a> validation]
<a id="__codelineno-3-18" name="__codelineno-3-18" href="#__codelineno-3-18"></a> --output PATH Indexing pipeline output directory.
<a id="__codelineno-3-19" name="__codelineno-3-19" href="#__codelineno-3-19"></a> Overrides storage.base_dir in the
<a id="__codelineno-3-20" name="__codelineno-3-20" href="#__codelineno-3-20"></a> configuration file.
</code></pre></div>
<h2 id="init">init</h2>
<p>Generate a default configuration file.</p>
@ -1512,16 +1510,14 @@
<a id="__codelineno-11-6" name="__codelineno-11-6" href="#__codelineno-11-6"></a> profiling [default: no-memprofile]
<a id="__codelineno-11-7" name="__codelineno-11-7" href="#__codelineno-11-7"></a> --reporter [rich|print|none] The progress reporter to use. [default:
<a id="__codelineno-11-8" name="__codelineno-11-8" href="#__codelineno-11-8"></a> rich]
<a id="__codelineno-11-9" name="__codelineno-11-9" href="#__codelineno-11-9"></a> --emit TEXT The data formats to emit, comma-separated.
<a id="__codelineno-11-10" name="__codelineno-11-10" href="#__codelineno-11-10"></a> [default: parquet]
<a id="__codelineno-11-11" name="__codelineno-11-11" href="#__codelineno-11-11"></a> --cache / --no-cache Use LLM cache. [default: cache]
<a id="__codelineno-11-12" name="__codelineno-11-12" href="#__codelineno-11-12"></a> --skip-validation / --no-skip-validation
<a id="__codelineno-11-13" name="__codelineno-11-13" href="#__codelineno-11-13"></a> Skip any preflight validation. Useful when
<a id="__codelineno-11-14" name="__codelineno-11-14" href="#__codelineno-11-14"></a> running no LLM steps. [default: no-skip-
<a id="__codelineno-11-15" name="__codelineno-11-15" href="#__codelineno-11-15"></a> validation]
<a id="__codelineno-11-16" name="__codelineno-11-16" href="#__codelineno-11-16"></a> --output PATH Indexing pipeline output directory.
<a id="__codelineno-11-17" name="__codelineno-11-17" href="#__codelineno-11-17"></a> Overrides storage.base_dir in the
<a id="__codelineno-11-18" name="__codelineno-11-18" href="#__codelineno-11-18"></a> configuration file.
<a id="__codelineno-11-9" name="__codelineno-11-9" href="#__codelineno-11-9"></a> --cache / --no-cache Use LLM cache. [default: cache]
<a id="__codelineno-11-10" name="__codelineno-11-10" href="#__codelineno-11-10"></a> --skip-validation / --no-skip-validation
<a id="__codelineno-11-11" name="__codelineno-11-11" href="#__codelineno-11-11"></a> Skip any preflight validation. Useful when
<a id="__codelineno-11-12" name="__codelineno-11-12" href="#__codelineno-11-12"></a> running no LLM steps. [default: no-skip-
<a id="__codelineno-11-13" name="__codelineno-11-13" href="#__codelineno-11-13"></a> validation]
<a id="__codelineno-11-14" name="__codelineno-11-14" href="#__codelineno-11-14"></a> --output PATH Indexing pipeline output directory.
<a id="__codelineno-11-15" name="__codelineno-11-15" href="#__codelineno-11-15"></a> Overrides storage.base_dir in the
<a id="__codelineno-11-16" name="__codelineno-11-16" href="#__codelineno-11-16"></a> configuration file.
</code></pre></div>

4
config/env_vars/index.html
View File

@ -1754,7 +1754,7 @@
<h1 id="default-configuration-mode-using-env-vars">Default Configuration Mode (using Env Vars)</h1>
<h2 id="text-embeddings-customization">Text-Embeddings Customization</h2>
<p>By default, the GraphRAG indexer will only emit embeddings required for our query methods. However, the model has embeddings defined for all plaintext fields, and these can be generated by setting the <code>GRAPHRAG_EMBEDDING_TARGET</code> environment variable to <code>all</code>.</p>
<p>By default, the GraphRAG indexer will only export embeddings required for our query methods. However, the model has embeddings defined for all plaintext fields, and these can be generated by setting the <code>GRAPHRAG_EMBEDDING_TARGET</code> environment variable to <code>all</code>.</p>
<p>If the embedding target is <code>all</code>, and you want to only embed a subset of these fields, you may specify which embeddings to skip using the <code>GRAPHRAG_EMBEDDING_SKIP</code> argument described below.</p>
<h3 id="embedded-fields">Embedded Fields</h3>
<ul>
@ -2440,7 +2440,7 @@
</tbody>
</table>
<h2 id="storage">Storage</h2>
<p>This section controls the storage mechanism used by the pipeline used for emitting output tables.</p>
<p>This section controls the storage mechanism used by the pipeline used for exporting output tables.</p>
<table>
<thead>
<tr>

14
config/yaml/index.html
View File

@ -1439,7 +1439,7 @@
<li><code>async_mode</code> (see Async Mode top-level config)</li>
<li><code>batch_size</code> <strong>int</strong> - The maximum batch size to use.</li>
<li><code>batch_max_tokens</code> <strong>int</strong> - The maximum batch # of tokens.</li>
<li><code>target</code> <strong>required|all|none</strong> - Determines which set of embeddings to emit.</li>
<li><code>target</code> <strong>required|all|none</strong> - Determines which set of embeddings to export.</li>
<li><code>skip</code> <strong>list[str]</strong> - Which embeddings to skip. Only useful if target=all to customize the list.</li>
<li><code>vector_store</code> <strong>dict</strong> - The vector store to use. Configured for lancedb by default.<ul>
<li><code>type</code> <strong>str</strong> - <code>lancedb</code> or <code>azure_ai_search</code>. Default=<code>lancedb</code></li>
@ -1566,7 +1566,7 @@
<h3 id="cluster_graph">cluster_graph</h3>
<h4 id="fields_13">Fields</h4>
<ul>
<li><code>max_cluster_size</code> <strong>int</strong> - The maximum cluster size to emit.</li>
<li><code>max_cluster_size</code> <strong>int</strong> - The maximum cluster size to export.</li>
<li><code>strategy</code> <strong>dict</strong> - Fully override the cluster_graph strategy.</li>
</ul>
<h3 id="embed_graph">embed_graph</h3>
@ -1588,11 +1588,11 @@
<h3 id="snapshots">snapshots</h3>
<h4 id="fields_16">Fields</h4>
<ul>
<li><code>embeddings</code> <strong>bool</strong> - Emit embeddings snapshots to parquet.</li>
<li><code>graphml</code> <strong>bool</strong> - Emit graph snapshots to GraphML.</li>
<li><code>raw_entities</code> <strong>bool</strong> - Emit raw entity snapshots to JSON.</li>
<li><code>top_level_nodes</code> <strong>bool</strong> - Emit top-level-node snapshots to JSON.</li>
<li><code>transient</code> <strong>bool</strong> - Emit transient workflow tables snapshots to parquet.</li>
<li><code>embeddings</code> <strong>bool</strong> - Export embeddings snapshots to parquet.</li>
<li><code>graphml</code> <strong>bool</strong> - Export graph snapshots to GraphML.</li>
<li><code>raw_entities</code> <strong>bool</strong> - Export raw entity snapshots to JSON.</li>
<li><code>top_level_nodes</code> <strong>bool</strong> - Export top-level-node snapshots to JSON.</li>
<li><code>transient</code> <strong>bool</strong> - Export transient workflow tables snapshots to parquet.</li>
</ul>
<h3 id="encoding_model">encoding_model</h3>
<p><strong>str</strong> - The text encoding model to use. Default=<code>cl100k_base</code>.</p>

4957
examples_notebooks/drift_search/index.html
View File

File diff suppressed because one or more lines are too long

6
examples_notebooks/global_search/index.html
View File

@ -2589,11 +2589,11 @@ Cosmic Vocalization is a phenomenon that has garnered significant attention from
2. **Alex Mercer**: Alex Mercer views Cosmic Vocalization as part of an interstellar duet. This perspective suggests a responsive approach to cosmic events, indicating that there may be an interactive or communicative aspect to the phenomenon [Data: Reports (6)].
3. **Taylor Cruz**: Taylor Cruz has raised concerns about Cosmic Vocalization, fearing it might be a homing tune. This adds a layer of urgency and potential threat, as it implies that the phenomenon could be attracting or guiding something or someone [Data: Reports (6)].
3. **Taylor Cruz**: Taylor Cruz has raised concerns about Cosmic Vocalization, fearing it might be a homing tune. This adds a layer of urgency and potential threat, as it implies that the phenomenon could be signaling or attracting attention from unknown entities or forces [Data: Reports (6)].
### Implications
The involvement of these diverse groups and individuals suggests that Cosmic Vocalization is not only a subject of scientific or academic interest but also a matter of strategic and security concern. The varying perspectives—from seeing it as a duet to fearing it as a homing signal—highlight the complexity and potential risks associated with this phenomenon. Further investigation and monitoring may be necessary to fully understand its implications and to develop appropriate responses.
The involvement of these diverse groups and individuals suggests that Cosmic Vocalization is not only a subject of scientific curiosity but also a matter of strategic and security interest. The concerns raised by Taylor Cruz, in particular, highlight the potential risks associated with the phenomenon, which may necessitate further investigation and monitoring.
</pre>
</div>
</div>
@ -2844,7 +2844,7 @@ print(
<div class="jp-OutputArea-child">
<div class="jp-OutputPrompt jp-OutputArea-prompt"></div>
<div class="jp-RenderedText jp-OutputArea-output" data-mime-type="text/plain" tabindex="0">
<pre>LLM calls: 2. Prompt tokens: 11238. Output tokens: 511.
<pre>LLM calls: 2. Prompt tokens: 11238. Output tokens: 493.
</pre>
</div>
</div>

32
examples_notebooks/global_search_with_dynamic_community_selection/index.html
View File

@ -2541,19 +2541,19 @@ print(result.response)</div>
<div class="jp-RenderedText jp-OutputArea-output" data-mime-type="text/plain" tabindex="0">
<pre>### Overview of Cosmic Vocalization
Cosmic Vocalization is a phenomenon that has captured the attention of various individuals and groups within the community. It is perceived as a significant event with potential implications for both cosmic and terrestrial activities. The concept of Cosmic Vocalization is central to the community's focus, suggesting its importance in ongoing discussions and strategic considerations.
Cosmic Vocalization is a phenomenon that has captured the attention of various individuals and groups, becoming a central focus within the community. It is perceived as a significant cosmic event, with implications that extend beyond mere observation. The phenomenon is interpreted in different ways by those involved, suggesting a complex and multifaceted nature.
### Perspectives and Concerns
### Key Participants and Perspectives
Alex Mercer views Cosmic Vocalization as part of an interstellar duet, indicating a belief that it may be a responsive or interactive event with cosmic origins [Data: Reports (6)]. This perspective suggests that Cosmic Vocalization could be part of a larger cosmic communication or interaction, which may have implications for how it is studied and understood.
1. **Paranormal Military Squad**: This group is actively engaged with Cosmic Vocalization, treating it as a strategic element in their security measures. Their involvement indicates the phenomenon's importance in broader security contexts. The squad metaphorically views the Universe as a concert hall, which reflects a unique perspective on how cosmic events are interpreted and responded to by human entities [Data: Reports (6)].
On the other hand, Taylor Cruz raises concerns about the potential implications of Cosmic Vocalization, fearing it might be a homing tune. This perspective adds a layer of urgency and potential threat, as it suggests that the phenomenon could be attracting attention or entities from beyond our immediate environment [Data: Reports (6)].
2. **Alex Mercer**: Alex Mercer perceives Cosmic Vocalization as part of an interstellar duet. This suggests a responsive and perhaps harmonious approach to the phenomenon, indicating that it may be part of a larger cosmic interaction or communication [Data: Reports (6)].
### Strategic and Security Implications
3. **Taylor Cruz**: Taylor Cruz raises concerns about the implications of Cosmic Vocalization, fearing it might be a homing tune. This perspective adds a layer of urgency and potential threat, suggesting that the phenomenon could have unintended or dangerous consequences [Data: Reports (6)].
The involvement of the Paranormal Military Squad highlights the strategic importance of Cosmic Vocalization. Their engagement indicates that the phenomenon is not only of scientific interest but also of security concern. The squad's involvement suggests that there are measures being taken to understand and possibly mitigate any risks associated with Cosmic Vocalization [Data: Reports (6)].
### Implications
In summary, Cosmic Vocalization is a multifaceted phenomenon that involves various stakeholders, each with their own perspectives and concerns. The community's focus on this event underscores its significance, while the involvement of security forces highlights the potential implications for safety and strategic planning.
The involvement of diverse groups and individuals with Cosmic Vocalization highlights its significance and the varied interpretations it inspires. The strategic engagement by the Paranormal Military Squad underscores its potential impact on security, while the differing views of Alex Mercer and Taylor Cruz illustrate the complexity and potential risks associated with the phenomenon. As such, Cosmic Vocalization remains a subject of intrigue and concern, warranting further investigation and understanding.
</pre>
</div>
</div>
@ -2655,10 +2655,10 @@ result.context_data["reports"]</div>
</tr>
<tr>
<th>4</th>
<td>19</td>
<td>Central Terminal and Viewing Monitors at Dulce...</td>
<td>5</td>
<td>Alien Script and First Contact Operations</td>
<td>0.02</td>
<td># Central Terminal and Viewing Monitors at Dul...</td>
<td># Alien Script and First Contact Operations\n\...</td>
<td>8.5</td>
</tr>
<tr>
@ -2671,6 +2671,14 @@ result.context_data["reports"]</div>
</tr>
<tr>
<th>6</th>
<td>19</td>
<td>Central Terminal and Viewing Monitors at Dulce...</td>
<td>0.02</td>
<td># Central Terminal and Viewing Monitors at Dul...</td>
<td>8.5</td>
</tr>
<tr>
<th>7</th>
<td>6</td>
<td>Cosmic Vocalization and Universe Interactions</td>
<td>0.02</td>
@ -2753,9 +2761,9 @@ print(
<div class="jp-OutputPrompt jp-OutputArea-prompt"></div>
<div class="jp-RenderedText jp-OutputArea-output" data-mime-type="text/plain" tabindex="0">
<pre>Build context (gpt-4o-mini)
LLM calls: 12. Prompt tokens: 8565. Output tokens: 1145.
LLM calls: 12. Prompt tokens: 8565. Output tokens: 1121.
Map-reduce (gpt-4o)
LLM calls: 2. Prompt tokens: 5864. Output tokens: 509.
LLM calls: 2. Prompt tokens: 6349. Output tokens: 589.
</pre>
</div>
</div>

34
examples_notebooks/local_search/index.html
View File

@ -3355,17 +3355,21 @@ Agent Alex Mercer is a prominent figure within the Paranormal Military Squad, pl
### Role and Responsibilities
Alex Mercer is deeply involved in the philosophical and strategic aspects of interstellar communication, viewing these interactions as a form of cosmic dialogue. His leadership style is characterized by a blend of determination, compliance with mission protocols, and a protective approach towards his team. He collaborates with team members like Jordan Hayes, exploring secured areas and engaging in high-stakes, secretive operations. Mercer emphasizes the importance of intuition and trust beyond protocol, particularly in his mentorship of Sam Rivera [Data: Entities (0)].
Mercer is deeply involved in the philosophical and strategic aspects of interstellar communication, viewing these interactions as a form of cosmic dialogue. His leadership style is characterized by a blend of determination, compliance with mission protocols, and a protective approach towards his team. He collaborates with team members like Jordan Hayes, exploring secured areas and engaging in high-stakes, secretive operations. Mercer is also known for his intellectual curiosity and deep involvement in the philosophical and strategic aspects of interstellar communication [Data: Entities (0)].
### Relationships and Influence
### Mentorship and Influence
Mercer is known for his intellectual curiosity and is depicted as a thoughtful mentor. His experiences during encounters with alien signals have led to profound changes, reinforcing his role as a key decision-maker and guardian in missions that transcend traditional boundaries. He shares a deep commitment to discovery with colleagues like Dr. Jordan Hayes, working together on alien technology during Operation: Dulce [Data: Entities (0); Relationships (336)].
As a mentor, particularly to Sam Rivera, Mercer emphasizes the importance of intuition and trust beyond protocol. His experiences during encounters with alien signals have led to profound changes, reinforcing his role as a key decision-maker and guardian in missions that transcend traditional boundaries. His mentorship has a significant impact on his team, influencing their actions and confidence [Data: Entities (0); Relationships (167)].
### Philosophical and Strategic Approach
### Involvement in Operation: Dulce
Agent Mercer harbors a subtle form of determination that resists easy submission to authority, as observed during the briefing for Operation: Dulce. He is involved in analyzing cosmic signals, strategizing contact efforts, and maintaining clarity in human-alien communication. His approach to leadership and strategy is marked by a balance between adhering to mission protocols and fostering an environment of trust and intuition among his team members [Data: Claims (1); Entities (0)].
Agent Mercer is a key participant in Operation: Dulce, deeply involved in its execution and implications. This operation marked the beginning of an interstellar odyssey and was pivotal as it led to the first contact with an alien race, initiating a dialogue that would expand the boundaries of human understanding of the universe [Data: Entities (0); Relationships (245); Sources (40, 42)].
In summary, Agent Alex Mercer is a pivotal figure in the Paranormal Military Squad, known for his strategic leadership, mentorship, and deep involvement in interstellar communication efforts. His role at Dulce Base highlights his commitment to understanding and engaging with extraterrestrial intelligence, making him a key player in the unfolding events of Operation: Dulce.
### Interpersonal Dynamics
Mercer shares a complex relationship with his colleagues, including Jordan Hayes and Taylor Cruz. While he and Jordan share a commitment to discovery, their interactions often reflect a competitive undercurrent against Cruz's authoritative approach. This dynamic is evident in their shared moments of philosophical reflection and strategic planning, which sometimes clash with Cruz's focus on actionable results and strict adherence to protocols [Data: Sources (40, 42)].
In summary, Agent Alex Mercer is a pivotal figure in the Paranormal Military Squad, known for his strategic leadership, mentorship, and deep involvement in interstellar communication efforts. His role in Operation: Dulce and his interactions with his team highlight his influence and the complexities of his responsibilities.
</pre>
</div>
</div>
@ -3411,21 +3415,21 @@ print(result.response)</div>
<div class="jp-RenderedText jp-OutputArea-output" data-mime-type="text/plain" tabindex="0">
<pre>### Overview of Dr. Jordan Hayes
Dr. Jordan Hayes is a prominent scientist at Dulce Base, playing a crucial role in the Paranormal Military Squad. Their work is primarily focused on the analysis and decryption of alien communications, which is vital for interstellar communication and potential interspecies interaction. Dr. Hayes is known for their expertise in deciphering alien codes and understanding extraterrestrial thought patterns, which are essential for leading communications with extraterrestrial entities [Data: Entities (17, 2); Relationships (201, 322)].
Dr. Jordan Hayes is a prominent scientist at Dulce Base, playing a crucial role in the Paranormal Military Squad's efforts to understand and communicate with extraterrestrial entities. Dr. Hayes's work is primarily focused on decoding and analyzing alien signals and codes, which are essential for interstellar communication and potentially interspecies interaction. This involves decrypting algorithms, interpreting cosmic signals, and analyzing the implications of alien society [Data: Entities (2, 17, 47); Relationships (201, 322, 283)].
### Role and Contributions
### Key Contributions and Expertise
Dr. Hayes's contributions are significant in the context of Operation: Dulce, where they are deeply involved in scientific exploration and the decryption of alien signals. Their work involves decrypting algorithms, interpreting cosmic signals, and analyzing the implications of alien society. This includes isolating and understanding complex alien signals that resemble human cognition, suggesting a tandem evolution with humanity [Data: Entities (2); Claims (60, 91, 134)].
Dr. Hayes is noted for their adaptability and skepticism, qualities that are essential given the uncertainties and unknown challenges of their mission. They are deeply involved in the scientific exploration aspects of Operation: Dulce, where their efforts are on the verge of a significant scientific breakthrough. Dr. Hayes leads efforts in isolating and understanding complex alien signals that resemble human cognition, suggesting that these signals are artificial and patterned, indicating a tandem evolution with humanity [Data: Entities (2, 17); Claims (60, 83, 91, 134)].
### Collaboration and Leadership
### Collaborative Efforts and Leadership
Dr. Hayes collaborates closely with other key members of the Paranormal Military Squad, such as Alex Mercer and Taylor Cruz. Despite some tension with Cruz due to differing leadership styles, Dr. Hayes's analytical skills and adaptability are highly valued by their colleagues. They are involved in setting up lab stations, operating the mainframe, and playing a crucial role in the command center at Dulce Base. Their leadership style emphasizes adaptability and skepticism, which are crucial given the uncertainties and unknown challenges of their mission [Data: Entities (2); Relationships (26, 82, 175); Claims (2, 13)].
Dr. Hayes works closely with colleagues like Alex Mercer and Taylor Cruz, engaging in thoughtful dialogue and showing analytical thinking about the mission's uncertainties. Despite some tension with Taylor Cruz due to differing leadership styles, Dr. Hayes continues to collaborate effectively with the team, contributing to the strategic discussions and decisions made in the briefing room. Their role in the command center at Dulce Base involves setting up lab stations, operating the mainframe, and playing a crucial role in the command center [Data: Entities (2, 17); Relationships (26, 82, 175, 322); Claims (2, 13)].
### Scientific Breakthroughs
### Scientific Breakthroughs and Challenges
Dr. Hayes is on the verge of significant scientific breakthroughs, as they lead efforts in isolating and understanding alien signals. Their work suggests that these signals are artificial and patterned, indicating a potential technological breakthrough. Dr. Hayes's efforts are crucial in crafting humanity's responses to cosmic alignments and responsive galactic signals, navigating the dark corridors of Dulce with a focus on the unknown variables and challenges beyond established protocols [Data: Entities (2); Claims (83, 91, 134)].
Dr. Hayes's work suggests that the alien signals they are studying are not just random but are structured and intentional, potentially indicating a form of sentient communication. This has led to the identification of new inviting signal patterns suggesting an intelligent confluence. Dr. Hayes's efforts in decrypting these signals are crucial for understanding and interpreting alien messages, aiding the squad in their paranormal operations [Data: Entities (2, 17); Claims (60, 83, 91, 134)].
In summary, Dr. Jordan Hayes is a central figure in the efforts to understand and communicate with extraterrestrial entities, contributing significantly to the scientific and strategic objectives of the Paranormal Military Squad at Dulce Base. Their work not only advances the understanding of alien communications but also holds the potential to redefine humanity's place in the cosmos.
In summary, Dr. Jordan Hayes is a central figure in the efforts to bridge the gap between human and extraterrestrial communication, leveraging their expertise in alien technology and signals to advance the mission's objectives. Their work is characterized by a blend of scientific rigor, adaptability, and a commitment to discovery, making them an invaluable asset to the Paranormal Military Squad [Data: Entities (2, 17); Relationships (201, 322, 283)].
</pre>
</div>
</div>
@ -3949,7 +3953,7 @@ print(candidate_questions.response)</div>
<div class="jp-OutputArea-child">
<div class="jp-OutputPrompt jp-OutputArea-prompt"></div>
<div class="jp-RenderedText jp-OutputArea-output" data-mime-type="text/plain" tabindex="0">
<pre>['- What is the role of the Paranormal Military Squad at Dulce Base?', '- How does Agent Alex Mercer contribute to the operations at Dulce Base?', '- What are the main objectives of Operation: Dulce at the Dulce Military Base?', '- How does the environment of Dulce Base affect the agents working there?', '- What are the challenges faced by the team at Dulce Base during their mission?']
<pre>['- What role does Agent Alex Mercer play at the Dulce Base?', '- How does the Paranormal Military Squad interact with extraterrestrial intelligence at Dulce?', '- What are the main objectives of Operation: Dulce at the Dulce Base?', '- How does the environment of Dulce Base affect the agents working there?', '- What is the significance of the Dulce Base in the context of interstellar communication?']
</pre>
</div>
</div>

12
index/default_dataflow/index.html
View File

@ -1947,8 +1947,8 @@ flowchart LR
<h3 id="entity-relationship-summarization">Entity &amp; Relationship Summarization</h3>
<p>Now that we have a graph of entities and relationships, each with a list of descriptions, we can summarize these lists into a single description per entity and relationship. This is done by asking the LLM for a short summary that captures all of the distinct information from each description. This allows all of our entities and relationships to have a single concise description.</p>
<h3 id="claim-extraction-emission">Claim Extraction &amp; Emission</h3>
<p>Finally, as an independent workflow, we extract claims from the source TextUnits. These claims represent positive factual statements with an evaluated status and time-bounds. These are emitted as a primary artifact called <strong>Covariates</strong>.</p>
<p>Note: claim extraction is <em>optional</em> and turned off by default. This is because claim extraction generally needs prompt tuning to be useful.</p>
<p>Finally, as an independent workflow, we extract claims from the source TextUnits. These claims represent positive factual statements with an evaluated status and time-bounds. These get exported as a primary artifact called <strong>Covariates</strong>.</p>
<p>Note: claim extraction is <em>optional</em> and turned off by default. This is because claim extraction generally requires prompt tuning to be useful.</p>
<h2 id="phase-3-graph-augmentation">Phase 3: Graph Augmentation</h2>
<p>Now that we have a usable graph of entities and relationships, we want to understand their community structure and augment the graph with additional information. This is done in two steps: <em>Community Detection</em> and <em>Graph Embedding</em>. These give us explicit (communities) and implicit (embeddings) ways of understanding the topological structure of our graph.</p>
<pre class="mermaid"><code>---
@ -1961,7 +1961,7 @@ flowchart LR
<h3 id="graph-embedding">Graph Embedding</h3>
<p>In this step, we generate a vector representation of our graph using the Node2Vec algorithm. This will allow us to understand the implicit structure of our graph and provide an additional vector-space in which to search for related concepts during our query phase.</p>
<h3 id="graph-tables-emission">Graph Tables Emission</h3>
<p>Once our graph augmentation steps are complete, the final <strong>Entities</strong> and <strong>Relationships</strong> tables are emitted after their text fields are text-embedded.</p>
<p>Once our graph augmentation steps are complete, the final <strong>Entities</strong> and <strong>Relationships</strong> tables are exported after their text fields are text-embedded.</p>
<h2 id="phase-4-community-summarization">Phase 4: Community Summarization</h2>
<pre class="mermaid"><code>---
title: Community Summarization
@ -1977,7 +1977,7 @@ flowchart LR
<h3 id="community-embedding">Community Embedding</h3>
<p>In this step, we generate a vector representation of our communities by generating text embeddings of the community report, the community report summary, and the title of the community report.</p>
<h3 id="community-tables-emission">Community Tables Emission</h3>
<p>At this point, some bookkeeping work is performed and we emit the <strong>Communities</strong> and <strong>CommunityReports</strong> tables.</p>
<p>At this point, some bookkeeping work is performed and we export the <strong>Communities</strong> and <strong>CommunityReports</strong> tables.</p>
<h2 id="phase-5-document-processing">Phase 5: Document Processing</h2>
<p>In this phase of the workflow, we create the <em>Documents</em> table for the knowledge model.</p>
<pre class="mermaid"><code>---
@ -1992,7 +1992,7 @@ flowchart LR
<h3 id="document-embedding">Document Embedding</h3>
<p>In this step, we generate a vector representation of our documents using an average embedding of document slices. We re-chunk documents without overlapping chunks, and then generate an embedding for each chunk. We create an average of these chunks weighted by token-count and use this as the document embedding. This will allow us to understand the implicit relationship between documents, and will help us generate a network representation of our documents.</p>
<h3 id="documents-table-emission">Documents Table Emission</h3>
<p>At this point, we can emit the <strong>Documents</strong> table into the knowledge Model.</p>
<p>At this point, we can export the <strong>Documents</strong> table into the knowledge Model.</p>
<h2 id="phase-6-network-visualization">Phase 6: Network Visualization</h2>
<p>In this phase of the workflow, we perform some steps to support network visualization of our high-dimensional vector spaces within our existing graphs. At this point there are two logical graphs at play: the <em>Entity-Relationship</em> graph and the <em>Document</em> graph.</p>
<pre class="mermaid"><code>---
@ -2000,7 +2000,7 @@ title: Network Visualization Workflows
---
flowchart LR
nv[Umap Documents] --&gt; ne[Umap Entities] --&gt; ng[Nodes Table Emission]</code></pre>
<p>For each of the logical graphs, we perform a UMAP dimensionality reduction to generate a 2D representation of the graph. This will allow us to visualize the graph in a 2D space and understand the relationships between the nodes in the graph. The UMAP embeddings are then emitted as a table of <em>Nodes</em>. The rows of this table include a discriminator indicating whether the node is a document or an entity, and the UMAP coordinates.</p>
<p>For each of the logical graphs, we perform a UMAP dimensionality reduction to generate a 2D representation of the graph. This will allow us to visualize the graph in a 2D space and understand the relationships between the nodes in the graph. The UMAP embeddings are then exported as a table of <em>Nodes</em>. The rows of this table include a discriminator indicating whether the node is a document or an entity, and the UMAP coordinates.</p>

2
search/search_index.json
View File

File diff suppressed because one or more lines are too long