3975 Commits

Author SHA1 Message Date
Harshal Sheth
62e6b7ff78
fix(ingest/snowflake): fix error case in column lineage (#10808) 2024-06-28 12:47:59 -07:00
Harshal Sheth
65a1dd1d82
fix(ingest/snowflake): add limits on tables/columns/queries in lineage (#10804) 2024-06-27 18:54:34 -07:00
Harshal Sheth
f4be88d0a9
feat(ingest): set pipeline name in system metadata (#10190)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2024-06-27 15:00:35 -07:00
pie1nthesky
5e9afc6d28
fix(ingest/pipeline): catch pipeline exceptions (#10753)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-06-27 13:03:36 -07:00
Harshal Sheth
fa2ab1bcee
fix(ingest): add status aspect to dataProcessInstance (#10757) 2024-06-27 12:07:28 -07:00
Harshal Sheth
d63f25faa8
fix(ingest/looker): prevent bad input fields (#10785) 2024-06-27 11:12:01 -07:00
Ellie O'Neil
1ae5bfc1b8
refactor(tags): Use TagUrn class when generating urn (#10786)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-06-27 18:00:22 +00:00
Harshal Sheth
45a8cc9ecf
feat(ingest): bump sqlglot (#10770) 2024-06-26 14:31:57 -07:00
Harshal Sheth
0d677e4992
fix(ingest/snowflake): fix column batcher (#10781) 2024-06-25 22:21:54 -07:00
Harshal Sheth
724907b8f4
feat(ingest): add async batch mode to the rest sink (#10733) 2024-06-25 15:49:00 -07:00
Harshal Sheth
0dc0bc5761
feat(ingest/snowflake): performance improvements (#10746) 2024-06-25 14:46:55 -07:00
Harshal Sheth
c6f2839650
fix(ingest): use more aggressive errors with sqlglot (#10769) 2024-06-25 13:27:42 -07:00
John Joyce
0527a57617
feat(docs): Adding docs for custom assertion reporting APIs (WIP) (#10656)
Co-authored-by: John Joyce <john@ip-192-168-1-200.us-west-2.compute.internal>
Co-authored-by: John Joyce <john@Johns-MBP.lan>
2024-06-25 13:11:56 -07:00
Mayuri Nehate
5735d9ec87
feat(data quality): custom assertions models, graphql, sdk (#10761) 2024-06-25 09:16:13 -07:00
Hyejin Yoon
49d1233403
docs: merge cli guide (#10464)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-06-25 00:28:38 +00:00
Eric L (CCCS)
79ba0b1720
fix(ingest/iceberg): add support for nested dictionaries when configuring pyiceberg (#10762) 2024-06-21 14:38:01 -07:00
ethan-cartwright
b3dccb0b35
fix(bigquery): use get() instead of hassattr for view labels (#10756)
Co-authored-by: Ethan Cartwright <ethan.cartwright@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-06-20 13:56:07 -07:00
Aseem Bansal
f581f70c93
fix(ingest/tableau): warn with better error message (#10749) 2024-06-20 16:56:53 +05:30
pie1nthesky
dfd5bcf5dd
fix(ingest/logging): fix excessive ingestion logging (#10735) 2024-06-19 16:27:39 +05:30
Harshal Sheth
c2b6272730
feat(ingest/snowflake): log queries at info level (#10745) 2024-06-18 15:16:20 -07:00
Harshal Sheth
f1e5a4757c
fix(ingest/dbt): support emitting only model performance (#10714) 2024-06-18 10:31:19 -07:00
Pedro Silva
1e7fec421d
feat(cli): Make ingest deploy create recipe with urn if not exists (#10724) 2024-06-18 14:02:16 +05:30
Harshal Sheth
2d727a960b
feat(ingest/snowflake): support more than 10k views in a db (#10718) 2024-06-18 07:37:39 +02:00
david-leifker
333799c338
feat(strucutred-properties): structured properties delete and schema change support (#10711) 2024-06-17 19:53:54 -05:00
Harshal Sheth
07df5f872a
fix(ingest/looker): fix looker browse paths v2 (#10700) 2024-06-17 22:55:08 +02:00
Harshal Sheth
39081ece15
feat(ingest/bigquery): support using table read permission without profiling (#10699) 2024-06-17 22:54:28 +02:00
Harshal Sheth
0816e7590c
fix(ingest): pin numpy<2 for classification (#10725) 2024-06-17 22:50:08 +02:00
ethan-cartwright
c58be155f3
feat(ingest/bigquery): Support for View Labels (#10648)
Co-authored-by: Ethan Cartwright <ethan.cartwright@acryl.io>
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2024-06-17 18:36:41 +05:30
Shubham Jagtap
e6246c9d71
fix(ingestion/bigquery): user exceeded quota for concurrent project.lists requests (#10578)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2024-06-14 17:09:13 -07:00
Harshal Sheth
60b09c5dbe
feat(ingest): log http request retries (#10715) 2024-06-14 17:07:53 -07:00
Rajasekhar-Vuppala
a2d8a099d8
feat(ingest/vertica): use 3 part naming (#10636) 2024-06-14 17:04:55 -07:00
Alex Waldron
edb9cf61f7
fix(docs): formatting of transformers code blocks (#10670) 2024-06-14 17:02:34 -07:00
noggi
e66726b9cc
Expose get_entities_v2 endpoint in python client (#10694) 2024-06-14 13:46:39 -07:00
Harshal Sheth
62c6704f69
feat(ingest/snowflake): refactor + parallel schema extraction (#10653) 2024-06-14 13:23:07 -07:00
Harshal Sheth
402bf31938
feat(gms): add ingestProposalBatch endpoint (#10706) 2024-06-14 12:30:19 -07:00
Davi Arnaut
d29d2c3b90
docs(ingest): Add Oracle prerequisites (#10712) 2024-06-14 08:46:57 -10:00
Shubham Jagtap
3479d070fe
fix(ingestion/sigma): Fix multiple requests http errors (#10616) 2024-06-14 11:30:45 -07:00
sagar-salvi-apptware
cc8389e431
fix(ingest/kafka-connect): Add lineage extraction for BigQuery Sink Connector in Kafka Connect source (#10647) 2024-06-14 18:51:04 +05:30
sagar-salvi-apptware
d69966074a
fix(ingest/bigquery): Map BigQuery policy tags to datahub column-level tags (#10669) 2024-06-14 16:43:12 +05:30
Harshal Sheth
6329153e36
fix(ingest): fix redshift query urns + reduce memory usage (#10691) 2024-06-13 11:27:06 -07:00
Harshal Sheth
25d48d2d09
fix(ingest/fivetran): fix fivetran bigquery support (#10693) 2024-06-13 11:26:47 -07:00
Aseem Bansal
e00d7f172d
feat(ingest/cli): optionally show server config (#10676) 2024-06-12 10:34:22 -07:00
Harshal Sheth
894e25680b
feat(ingest): add snowflake-summary source (#10642) 2024-06-12 10:04:22 -07:00
Harshal Sheth
3a72d92493
feat(ingest/dbt): include package_name in dbt custom props (#10652) 2024-06-12 15:07:42 +02:00
Harshal Sheth
5da77df75d
feat(ingest/airflow): fix materialize_iolets bug (#10613) 2024-06-12 15:07:00 +02:00
Aseem Bansal
3ac9b72d2b
docs(cli/dataset): add dataset CLI get and upsert examples (#10688) 2024-06-12 18:26:41 +05:30
Aseem Bansal
9307ec0580
feat(cli/data product): add support for institutional memory (#10686) 2024-06-12 17:51:51 +05:30
Chris Collins
22fac3a811
feat(cli): add actors to forms yaml API (#10683) 2024-06-12 13:05:18 +05:30
Harshal Sheth
1d4977cbb7
feat(ingest): make query formatting more robust (#10678) 2024-06-12 12:27:36 +05:30
Shubham Jagtap
05aee03f3f
perf(ingestion/fivetran): Connector performance optimization (#10556) 2024-06-11 20:19:57 -07:00