Mayuri Nehate
f6627efe71
fix(ingest/snowflake): add more reporting for usage aggregation, handle lineage errors ( #10279 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-05-07 08:42:39 -07:00
Harshal Sheth
0d780e5f8f
feat(ingest): sql parsing aggregator ( #9786 )
2024-02-09 16:27:45 -05:00
Shubham Jagtap
3635c1c221
feat(ingestion/bigquery): Use sqlglot_lineage for usage and add more perf timers ( #9247 )
...
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-12-28 04:54:26 -05:00
Andrew Sikowitz
806f09ae23
test(ingest/unity): Unity catalog data generation ( #8949 )
2023-12-05 12:33:00 -05:00
Andrew Sikowitz
301d3e6b1c
test(ingest/unity): Add Unity Catalog memory performance testing ( #8932 )
2023-10-04 10:23:13 -04:00
Andrew Sikowitz
aa5e02d0ec
feat(ingest): Create zero usage aspects ( #8205 )
...
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-06-22 17:07:50 -04:00
Mayuri Nehate
b3d80e57e8
feat(ingest/bigquery): usage for views ( #8046 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-05-24 09:48:58 -07:00
Andrew Sikowitz
1ac1ccf26e
perf(ingest/bigquery): Improve bigquery usage disk usage and speed ( #7825 )
2023-04-14 18:09:43 -07:00
Andrew Sikowitz
087855f374
fix(ingest/bigquery): Support cross project usage using FileBackedDict ( #7663 )
...
Includes major refactor of bigquery usage ingestion, minor refactor of the source as a whole, and reporting cleanup.
Includes bigquery performance testing changes.
2023-04-07 12:18:26 -07:00
Andrew Sikowitz
54a372795b
test(ingest/bigquery): Add performance testing framework for bigquery usage ( #7690 )
...
- Creates metadata-ingestion/tests/performance directory
- Excludes metadata-ingestion/tests from docs generation
- Updates bigquery reporting around project state
2023-03-29 14:13:43 -07:00
