15 Commits

Author SHA1 Message Date
Tamas Nemeth
3cb1678ecc
feat(ingest/bigquery): Add way to reference existing DataHub Tag from a bigquery label (#11544) 2024-10-09 17:01:34 +02:00
Mayuri Nehate
6a58493011
fix(ingest/bq): do not query PARTITIONS for biglake tables (#11463) 2024-09-27 16:46:37 +05:30
Mayuri Nehate
cf49f80e77
feat(ingest/sql): auto extract and use mode query user metadata (#11307)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-09 12:38:24 -07:00
Mayuri Nehate
08a5956e26
fix(bigquery): followups on bigquery queries v2 integration (#11291) 2024-09-04 07:42:35 -07:00
Mayuri Nehate
a7fc7f519a
feat(ingest/bq): integrate bigquery-queries into main source (#11247) 2024-08-30 18:16:45 -07:00
sid-acryl
627c5abfd6
feat(ingestion/bigquery): Add ability to filter GCP project ingestion based on project labels (#11169)
Co-authored-by: Alice Naghshineh <alice.naghshineh@nytimes.com>
Co-authored-by: Alice Naghshineh <45885699+anaghshineh@users.noreply.github.com>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-08-20 14:42:00 -04:00
Mayuri Nehate
ff1c6b895e
feat(ingest/BigQuery): refactor+parallelize dataset metadata extraction (#10884) 2024-07-16 11:46:42 -07:00
sagar-salvi-apptware
d69966074a
fix(ingest/bigquery): Map BigQuery policy tags to datahub column-level tags (#10669) 2024-06-14 16:43:12 +05:30
Harshal Sheth
3d5735cbc5
chore(ingest): run pyupgrade for python 3.8 (#10513) 2024-05-15 22:31:05 -07:00
Mayuri Nehate
77c72dad01
feat(ingest): add classification to bigquery, redshift (#10031) 2024-03-13 22:45:28 -07:00
Mayuri Nehate
cdb9f5ba62
feat(bigquery): add better timers around every API call (#8626) 2023-09-15 11:55:39 -07:00
Andrew Sikowitz
22c35f1a23
fix(ingest/bigquery): Add config option to create DataPlatformInstance, default off (#8659) 2023-08-24 14:46:06 +05:30
Andrew Sikowitz
c5cc53b99a
feat(ingest/bigquery_v2): enable platform instance using project id (#8216)
Co-authored-by: Adrián Pertíñez <khurzak92@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-14 09:50:21 -07:00
Andrew Sikowitz
369a04ae30
revert(ingest/bigquery): Do not emit DataPlatformInstance; remove references to platform_instance (#8196) 2023-06-09 13:44:24 +05:30
Adrián Pertíñez
743439c11d
feat(ingest/bigquery_v2): enable platform instance using project id (#8142)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-06-05 15:17:40 -07:00