78 Commits

Author SHA1 Message Date
Sergio Gómez Villamor
e7d8f2913c
fix(snowflake): fixes deduplication and fingerprint requirements for Hex (#13121) 2025-04-09 10:17:43 +02:00
Mayuri Nehate
1bf395bf55
fix(ingest): split merge statements correctly (#12989) 2025-03-27 14:24:50 +05:30
Mayuri Nehate
ac9997d970
feat(ingest/snowflake): ingest stored procedures (#12929)
Co-authored-by: Sergio Gómez Villamor <sgomezvillamor@gmail.com>
2025-03-26 20:02:55 +05:30
Sergio Gómez Villamor
29d05c214a
feat(hex): warehouse integration via Query enrichment (#12949) 2025-03-26 09:35:50 +01:00
Harshal Sheth
a6461853dc
feat(ingest): improve extract-sql-agg-log command (#12803) 2025-03-06 11:08:56 -08:00
Harshal Sheth
de60ca30e8
fix(ingest): enable fuzzy case resolution for oracle sql (#12778) 2025-03-05 10:28:29 -08:00
Aseem Bansal
9c3bd34995
dev: enable ruff rule (#12749) 2025-02-28 17:49:52 +05:30
Harshal Sheth
9759a58d04
docs(ingest): add more details on SQL parser limitations (#12669) 2025-02-21 16:16:07 -08:00
ttekampe
27e72782b9
fix(mssql): correctly split stored procs SQL (#12618)
Co-authored-by: Tobias Tekampe <tobias.tekampe@movingdots.com>
2025-02-18 12:09:06 -08:00
Harshal Sheth
7472c535d3
feat(ci): add pytest hooks for updating golden files (#12581) 2025-02-12 15:32:31 -08:00
Harshal Sheth
32c62e5a8d
feat(ingest/mssql): improve stored procedure splitting (#12563) 2025-02-07 15:49:14 -08:00
Sergio Gómez Villamor
4b79e7525f
fix(ingestion): groupby_unsorted (#12403) 2025-01-21 09:44:19 +01:00
Harshal Sheth
f06ad1a1d5
fix(ingest): simplify + fix ruff config (#12382) 2025-01-18 05:58:12 +05:30
Aseem Bansal
765bf80282
dev(ingest): move from isort to ruff (#12364) 2025-01-16 16:48:21 +05:30
Mayuri Nehate
fba09966f3
fix(ingest): consistent fingerprint for sql parsing aggregator (#12239) 2025-01-03 10:49:42 -08:00
sagar-salvi-apptware
2e544614f1
feat(ingest): add looker meta extractor support in sql parsing (#12062)
Co-authored-by: Mayuri N <mayuri.nehate@gslab.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2024-12-19 12:41:40 +05:30
Andrew Sikowitz
ff385edbb1
build(ingest/sqlglot): Bump pin to support snowflake CREATE ... WITH TAG (#12003) 2024-12-17 03:49:47 -08:00
Harshal Sheth
e730afdb68
feat(ingest): improve query fingerprinting (#12104) 2024-12-12 13:51:18 -08:00
sid-acryl
2ec9cb0536
fix(ingestion/lookml): resolve CLL issue caused by column name casing. (#11876) 2024-12-12 15:32:56 +05:30
Mayuri Nehate
c3f9a9206d
feat(ingest/mssql): include stored procedure lineage (#11912)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-11-22 20:32:24 +05:30
Harshal Sheth
1bfd4ee1d4
feat(ingest): handle mssql casing issues in lineage (#11920) 2024-11-21 17:16:04 -08:00
Harshal Sheth
5519a330e2
chore(ingest): bump black (#11898) 2024-11-20 13:33:54 -08:00
Harshal Sheth
7dbb3e60cb
chore(ingest): start using explicit exports (#11899) 2024-11-20 13:33:30 -08:00
Harshal Sheth
85c8e605be
fix(ingest): consider sql parsing fallback as failure (#11896) 2024-11-19 15:06:16 -08:00
Harshal Sheth
e609ff810d
feat(ingest/powerbi): improve reporting around m-query parser (#11763) 2024-10-31 16:27:45 -07:00
Harshal Sheth
6316e10d48
feat(ingest): check ordering in SqlParsingAggregator tests (#11735) 2024-10-29 17:50:37 +01:00
Mayuri Nehate
87fa5b89e8
feat: multi-query lineage for temp upstreams (#11708) 2024-10-25 16:56:55 +05:30
Mayuri Nehate
eab2ac7a2e
feat(ingest/snowflake): support lineage via rename and swap using que… (#11600) 2024-10-23 14:02:08 +05:30
Harshal Sheth
35f30b7d3c
feat(ingest): use mainline sqlglot (#11693) 2024-10-22 19:57:46 -07:00
Mayuri Nehate
179a6714a6
feat(ingest): ensure sqlite file delete on clean exit (#11612) 2024-10-18 10:33:28 +05:30
Harshal Sheth
6e3724b2da
perf(ingest): streamline CLL generation (#11645) 2024-10-17 17:50:59 -07:00
Mayuri Nehate
837d00d391
fix(ingest/bq): fix ordering of queries for use_queries_v2 (#11333) 2024-09-10 12:17:23 -07:00
Harshal Sheth
f4033707d4
chore(ingest): bump acryl-sqlglot (#11331) 2024-09-09 21:09:44 -07:00
Mayuri Nehate
cf49f80e77
feat(ingest/sql): auto extract and use mode query user metadata (#11307)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-09-09 12:38:24 -07:00
Mayuri Nehate
223650dd7a
feat(ingest): add bigquery-queries source (#10994)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-25 22:51:00 -07:00
sid-acryl
b1f16f9b11
fix(ingestion/lookml): fix for sql parsing error (#11079)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-08-09 15:06:42 -07:00
Harshal Sheth
840b15083a
fix(sql-parser): prevent bad urns from alter table lineage (#11092) 2024-08-08 14:05:55 -07:00
Harshal Sheth
8c3bfd996d
feat(ingest/bigquery): improve handling of information schema in sql parser (#10985) 2024-07-25 17:58:16 -07:00
Harshal Sheth
a4bce6af1c
feat(ingest): add snowflake-queries source (#10835) 2024-07-12 15:08:51 -07:00
Harshal Sheth
e842161849
feat(ingest): add fast query fingerprinting (#10619) 2024-06-05 13:47:44 -07:00
Harshal Sheth
2e14f70864
test(ingest/sql): refactor CLL generator + add tests (#10580) 2024-05-23 18:11:22 -07:00
Harshal Sheth
2b6c78b776
feat(ingest): bump acryl-sqlglot dep (#10554) 2024-05-21 23:52:33 +02:00
Harshal Sheth
187ef12182
fix(ingest/dbt): improve handling for CLL via ephemeral nodes (#10535) 2024-05-20 13:33:25 -07:00
Tamas Nemeth
897e648eae
fix(ingest/mode): Improve query lineage (#10284)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-05-07 22:02:37 -07:00
Harshal Sheth
08731055ba
feat(ingest): bump acryl-sqlglot dep (#10343) 2024-04-20 08:37:22 +02:00
Harshal Sheth
bdf2c9a5c1
feat(ingest/sql): normalize bigquery partitioned tables when parsing (#10224) 2024-04-07 17:17:28 +02:00
Harshal Sheth
a70e775a12
feat(ingest): emit platform for query entities (#10103) 2024-03-26 11:22:53 -07:00
alexs-101
e6e5c091ed
feat(tableau): ability to force extraction of table/column level linage from SQL queries (#9838) 2024-03-21 09:27:22 +01:00
Harshal Sheth
d4d175da1a
fix(ingest/redshift): tweak lineage v2 queries (#10045) 2024-03-15 12:16:18 -07:00
Harshal Sheth
b0163c4885
feat(ingest): utilities for query logs (#10036) 2024-03-12 23:20:46 -07:00