32 Commits

Author SHA1 Message Date
Harshal Sheth
287f373a9c
fix(ingest/snowflake): fix previously broken tests (#13428) 2025-05-06 10:19:25 -07:00
Harshal Sheth
f48c6b53ee
feat(ingest/snowflake): show returned query row counts (#13246) 2025-04-21 14:41:40 -07:00
Harshal Sheth
9f7f3cb886
chore(ingest/snowflake): remove unused query code (#13245) 2025-04-21 10:31:36 -07:00
Mayuri Nehate
ac9997d970
feat(ingest/snowflake): ingest stored procedures (#12929)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-03-26 20:02:55 +05:30
Mayuri Nehate
fbed37b374
fix(ingest/snowflake): update logic to identify hybrid tables (#12672) 2025-02-20 11:23:57 +05:30
Brock Griffey
ac13f255e6
feat(ingestion/snowflake):adds streams as a new dataset with lineage and properties. (#12318)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2025-02-05 10:48:37 +05:30
Tamas Nemeth
4dc9bfcde7
fix(ingest/snowflake): Create all structured propery templates before assignation (#12469) 2025-01-30 13:49:20 -08:00
Mayuri Nehate
4d990b06bd
fix(ingest/snowflake): always ingest view and external table ddl lineage (#12191) 2024-12-24 18:14:51 +05:30
Mayuri Nehate
d38427a773
feat(ingest/snowflake): secure view lineage without owner permissions (#12123) 2024-12-16 17:49:49 +05:30
Harshal Sheth
5519a330e2
chore(ingest): bump black (#11898) 2024-11-20 13:33:54 -08:00
Alisa Aylward
675b225c5b
feat(ingest/snowflake): Add cluster formula to dataset properties (#11254) 2024-08-29 16:11:35 -07:00
Alisa Aylward
88b3893b5c
feat(ingest/snowflake): allow iceberg tables in lineage and access metadata (#10961) 2024-08-22 01:20:51 +00:00
Harshal Sheth
a4bce6af1c
feat(ingest): add snowflake-queries source (#10835) 2024-07-12 15:08:51 -07:00
Harshal Sheth
0d677e4992
fix(ingest/snowflake): fix column batcher (#10781) 2024-06-25 22:21:54 -07:00
Harshal Sheth
0dc0bc5761
feat(ingest/snowflake): performance improvements (#10746) 2024-06-25 14:46:55 -07:00
Harshal Sheth
2d727a960b
feat(ingest/snowflake): support more than 10k views in a db (#10718) 2024-06-18 07:37:39 +02:00
Harshal Sheth
3d5735cbc5
chore(ingest): run pyupgrade for python 3.8 (#10513) 2024-05-15 22:31:05 -07:00
Mayuri Nehate
f6627efe71
fix(ingest/snowflake): add more reporting for usage aggregation, handle lineage errors (#10279)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-05-07 08:42:39 -07:00
Mayuri Nehate
8047d0b514
feat(ingest/snowflake): generate query nodes for snowflake (#9966)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-06 15:11:37 -08:00
Tamas Nemeth
0ea6145a9d
fix(ingest/profiling): Add option to enable external table profiling (#9463) 2023-12-15 00:12:45 +01:00
Tamas Nemeth
f794a90515
fix(ingest/snowflake): Apply email filter on all usage metrics (#9269) 2023-11-23 09:45:25 +01:00
Harshal Sheth
ec9725026d
chore(ingest): remove legacy memory_leak_detector (#9158) 2023-11-02 09:39:08 -07:00
Mayuri Nehate
e02b9096bd
fix(ingest/snowflake): missing view downstream cll if platform instance is set (#8966) 2023-10-27 12:03:43 -07:00
Andrew Sikowitz
8141e2d649
remove(ingest/snowflake): Remove legacy snowflake lineage (#8653)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-08-23 15:57:46 -04:00
Mayuri Nehate
ddcd5109dc
feat(ingest): allow relative start time config (#8562)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-08-14 17:48:06 -07:00
Mayuri Nehate
e67f811034
feat(classification): allow parallelisation to reduce time (#8368) 2023-08-02 09:53:39 +05:30
Andrew Sikowitz
aa5e02d0ec
feat(ingest): Create zero usage aspects (#8205)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-06-22 17:07:50 -04:00
Mayuri Nehate
c845c75a2d
feat(ingest/snowflake): add config option to specify deny patterns for upstreams (#7962)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-05-08 14:13:57 -07:00
Mayuri Nehate
3212e74969
feat(ingest/snowflake): optionally emit all upstreams irrespective of recipe pattern (#7842) 2023-04-24 11:01:15 -07:00
Mayuri Nehate
1fda92441f
feat(snowflake): improve snowflake lineage perf and memory, push down to snowflake (#7710) 2023-04-07 11:06:06 -07:00
Fredrik Sannholm
e0aa812621
feat(ingest): allow extracting snowflake tags (#6500) 2023-01-04 16:05:23 -05:00
Mayuri Nehate
2129496c98
feat(ingest/snowflake): handle failures gracefully and raise permission failures (#6748) 2022-12-28 08:20:37 -08:00