Sergio Gómez Villamor
dd371130c1
chore(ruff): enable some ignored rules ( #12815 )
2025-03-11 09:30:06 +01:00
Aseem Bansal
262dd76518
dev: remove black in favor of ruff for formatting ( #12378 )
2025-01-18 15:06:20 +05:30
Aseem Bansal
76e46b89db
dev(ingest): move modules from isort,flake8 to ruff ( #12373 )
2025-01-17 21:38:29 +05:30
Sergio Gómez Villamor
9d9a368dea
chore(tableau): set ingestion stage report and perftimers ( #12234 )
2025-01-09 19:04:37 +01:00
Mayuri Nehate
4d990b06bd
fix(ingest/snowflake): always ingest view and external table ddl lineage ( #12191 )
2024-12-24 18:14:51 +05:30
Harshal Sheth
5519a330e2
chore(ingest): bump black ( #11898 )
2024-11-20 13:33:54 -08:00
Mayuri Nehate
1a73c664f0
feat(ingest/databricks): add usage perf report ( #11480 )
2024-09-27 09:46:22 -07:00
Harshal Sheth
3755731f0e
chore(ingest): improve code formatting ( #11326 )
2024-09-11 10:48:57 -07:00
Harshal Sheth
401b787427
fix(ingest): add custom StrEnum type ( #11270 )
2024-09-04 08:09:36 -07:00
Mayuri Nehate
223650dd7a
feat(ingest): add bigquery-queries source ( #10994 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-25 22:51:00 -07:00
Mayuri Nehate
f6627efe71
fix(ingest/snowflake): add more reporting for usage aggregation, handle lineage errors ( #10279 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-05-07 08:42:39 -07:00
Harshal Sheth
0d780e5f8f
feat(ingest): sql parsing aggregator ( #9786 )
2024-02-09 16:27:45 -05:00
Shubham Jagtap
3635c1c221
feat(ingestion/bigquery): Use sqlglot_lineage for usage and add more perf timers ( #9247 )
...
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-12-28 04:54:26 -05:00
Andrew Sikowitz
806f09ae23
test(ingest/unity): Unity catalog data generation ( #8949 )
2023-12-05 12:33:00 -05:00
Andrew Sikowitz
301d3e6b1c
test(ingest/unity): Add Unity Catalog memory performance testing ( #8932 )
2023-10-04 10:23:13 -04:00
Andrew Sikowitz
aa5e02d0ec
feat(ingest): Create zero usage aspects ( #8205 )
...
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-06-22 17:07:50 -04:00
Mayuri Nehate
b3d80e57e8
feat(ingest/bigquery): usage for views ( #8046 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-05-24 09:48:58 -07:00
Andrew Sikowitz
1ac1ccf26e
perf(ingest/bigquery): Improve bigquery usage disk usage and speed ( #7825 )
2023-04-14 18:09:43 -07:00
Andrew Sikowitz
087855f374
fix(ingest/bigquery): Support cross project usage using FileBackedDict ( #7663 )
...
Includes major refactor of bigquery usage ingestion, minor refactor of the source as a whole, and reporting cleanup.
Includes bigquery performance testing changes.
2023-04-07 12:18:26 -07:00
Andrew Sikowitz
54a372795b
test(ingest/bigquery): Add performance testing framework for bigquery usage ( #7690 )
...
- Creates metadata-ingestion/tests/performance directory
- Excludes metadata-ingestion/tests from docs generation
- Updates bigquery reporting around project state
2023-03-29 14:13:43 -07:00
