54 Commits

Author SHA1 Message Date
Harshal Sheth
e609ff810d
feat(ingest/powerbi): improve reporting around m-query parser (#11763) 2024-10-31 16:27:45 -07:00
Harshal Sheth
6316e10d48
feat(ingest): check ordering in SqlParsingAggregator tests (#11735) 2024-10-29 17:50:37 +01:00
Mayuri Nehate
87fa5b89e8
feat: multi-query lineage for temp upstreams (#11708) 2024-10-25 16:56:55 +05:30
Mayuri Nehate
eab2ac7a2e
feat(ingest/snowflake): support lineage via rename and swap using que… (#11600) 2024-10-23 14:02:08 +05:30
Harshal Sheth
35f30b7d3c
feat(ingest): use mainline sqlglot (#11693) 2024-10-22 19:57:46 -07:00
Mayuri Nehate
179a6714a6
feat(ingest): ensure sqlite file delete on clean exit (#11612) 2024-10-18 10:33:28 +05:30
Harshal Sheth
6e3724b2da
perf(ingest): streamline CLL generation (#11645) 2024-10-17 17:50:59 -07:00
Mayuri Nehate
837d00d391
fix(ingest/bq): fix ordering of queries for use_queries_v2 (#11333) 2024-09-10 12:17:23 -07:00
Harshal Sheth
f4033707d4
chore(ingest): bump acryl-sqlglot (#11331) 2024-09-09 21:09:44 -07:00
Mayuri Nehate
cf49f80e77
feat(ingest/sql): auto extract and use mode query user metadata (#11307)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-09 12:38:24 -07:00
Mayuri Nehate
223650dd7a
feat(ingest): add bigquery-queries source (#10994)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-25 22:51:00 -07:00
sid-acryl
b1f16f9b11
fix(ingestion/lookml): fix for sql parsing error (#11079)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-09 15:06:42 -07:00
Harshal Sheth
840b15083a
fix(sql-parser): prevent bad urns from alter table lineage (#11092) 2024-08-08 14:05:55 -07:00
Harshal Sheth
8c3bfd996d
feat(ingest/bigquery): improve handling of information schema in sql parser (#10985) 2024-07-25 17:58:16 -07:00
Harshal Sheth
a4bce6af1c
feat(ingest): add snowflake-queries source (#10835) 2024-07-12 15:08:51 -07:00
Harshal Sheth
e842161849
feat(ingest): add fast query fingerprinting (#10619) 2024-06-05 13:47:44 -07:00
Harshal Sheth
2e14f70864
test(ingest/sql): refactor CLL generator + add tests (#10580) 2024-05-23 18:11:22 -07:00
Harshal Sheth
2b6c78b776
feat(ingest): bump acryl-sqlglot dep (#10554) 2024-05-21 23:52:33 +02:00
Harshal Sheth
187ef12182
fix(ingest/dbt): improve handling for CLL via ephemeral nodes (#10535) 2024-05-20 13:33:25 -07:00
Tamas Nemeth
897e648eae
fix(ingest/mode): Improve query lineage (#10284)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-05-07 22:02:37 -07:00
Harshal Sheth
08731055ba
feat(ingest): bump acryl-sqlglot dep (#10343) 2024-04-20 08:37:22 +02:00
Harshal Sheth
bdf2c9a5c1
feat(ingest/sql): normalize bigquery partitioned tables when parsing (#10224) 2024-04-07 17:17:28 +02:00
Harshal Sheth
a70e775a12
feat(ingest): emit platform for query entities (#10103) 2024-03-26 11:22:53 -07:00
alexs-101
e6e5c091ed
feat(tableau): ability to force extraction of table/column level linage from SQL queries (#9838) 2024-03-21 09:27:22 +01:00
Harshal Sheth
d4d175da1a
fix(ingest/redshift): tweak lineage v2 queries (#10045) 2024-03-15 12:16:18 -07:00
Harshal Sheth
b0163c4885
feat(ingest): utilities for query logs (#10036) 2024-03-12 23:20:46 -07:00
Tamas Nemeth
4535f2adfd
fix(ingest/redshift): Temp table lineage fix (#10008)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-03-12 14:31:00 -07:00
Harshal Sheth
7e2076e852
feat(ingest): add query formatting to sql aggregator (#10025) 2024-03-11 15:34:44 -07:00
Harshal Sheth
b6956f9a5c
feat(ingest): update sqlglot fork (#10022) 2024-03-11 15:22:30 +01:00
Harshal Sheth
1736edf8f5
feat(ingest): fix bugs in SqlParsingAggregator (#9926) 2024-02-28 15:06:33 +01:00
Harshal Sheth
392158809d
feat(ingest/redshift): redshift lineage v2 (#9904) 2024-02-23 16:32:51 -08:00
Harshal Sheth
b01416c11b
feat(ingest): add timeout to sqlglot parser (#9868) 2024-02-23 11:11:02 -08:00
Harshal Sheth
ef3a8148ec
fix(ingest): bump sqlglot + add some debug info to tests (#9867) 2024-02-21 13:07:59 -08:00
Harshal Sheth
f13ae77966
feat(ingest): add escape hatch methods to SqlParsingAggregator (#9860) 2024-02-21 11:33:05 -08:00
Harshal Sheth
0d780e5f8f
feat(ingest): sql parsing aggregator (#9786) 2024-02-09 16:27:45 -05:00
Harshal Sheth
19b76c3307
feat(ingest/sql-parser): test redshift temp tables (#9653) 2024-01-23 08:37:53 +01:00
Harshal Sheth
f14dc4ded2
feat(ingest): improve sql tests + dbt reporting (#9649) 2024-01-23 08:37:23 +01:00
Harshal Sheth
52687f3eea
feat(ingest): support CLL for redshift materialized views with auto refresh (#9508) 2023-12-22 08:18:22 +01:00
Adriano Vega Llobell
d0fa5de9f0
fix(sql-parser): convert platform instance to lowercase when building table urns (#9181) 2023-11-16 13:58:49 -05:00
Harshal Sheth
19aa215068
feat(ingest/dbt): dbt column-level lineage (#8991) 2023-11-13 19:00:30 -05:00
Harshal Sheth
55f14530a3
feat(ingest): bump sqlglot parser (#9155) 2023-11-01 12:42:52 +05:30
Harshal Sheth
ea1273281e
feat(ingest): sql parser perf + asyncio fixes (#9119) 2023-10-30 20:57:59 -07:00
Harshal Sheth
ce0f36b8bc
feat(sqlparser): support more update syntaxes + fix bug with subqueries (#9105) 2023-10-30 14:23:19 -07:00
Harshal Sheth
d13553f53a
feat(sqlparser): extract CLL from updates (#9078) 2023-10-24 12:24:50 -07:00
Harshal Sheth
8fb95e88a1
feat(sqlparser): parse create DDL statements (#9002) 2023-10-23 12:40:42 -07:00
Tamas Nemeth
a8f0080c08
feat(ingest/teradata): Teradata source (#8977) 2023-10-12 15:14:45 -07:00
Harshal Sheth
84bba4dc44
feat(ingest): add output schema inference for sql parser (#8989) 2023-10-11 22:31:17 -07:00
Harshal Sheth
a300b39f15
feat(ingest/airflow): airflow plugin v2 (#8853) 2023-10-04 16:23:15 +05:30
Harshal Sheth
2a0200b047
feat(ingest): bump acryl-sqlglot (#8882) 2023-09-21 14:28:51 -07:00
Harshal Sheth
4ffad4d9b9
chore(ingest): upgrade sqlglot fork (#8775)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-09-06 12:49:44 -07:00