yujunjun/datahub
2
0
Fork 0
mirror of https://github.com/datahub-project/datahub.git synced 2025-10-02 12:37:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix(docs) Update impact analysis docs to call out lightning cache bugs (#12918)

Browse Source
This commit is contained in:
Chris Collins 2025-04-03 12:16:40 -04:00 committed by GitHub
parent 497ac3c58b
commit c4866a959c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docs/act-on-metadata/impact-analysis.md
View File

@ -56,6 +56,14 @@ Follow these simple steps to understand the full dependency chain of your data e
<img width="70%" src="https://raw.githubusercontent.com/datahub-project/static-assets/main/imgs/impact-analysis-view-export-results.png"/>
</p>
### Known Issues
Impact Analysis is a powerful feature that can place significant demands on the system. To maintain high performance when handling large result sets, we've implemented "Lightning Cache" - an alternate processing path that delivers results more quickly. By default, this cache activates with simple queries when there are more than 300 assets in the result set. You can customize this threshold by setting the environment variable `CACHE_SEARCH_LINEAGE_LIGHTNING_THRESHOLD` in your GMS pod.
However, the Lightning Cache has a limitation: it may include assets that are soft-deleted or no longer exist in the DataHub database. This occurs because lineage references may contain "ghost entities" (URNs without associated data).
Note that when you download Impact Analysis results, our system properly filters out these soft-deleted and non-existent assets. As a result, you might notice differences between what appears in the UI and what appears in your downloaded results.
## Additional Resources
### Videos