3904 Commits

Author SHA1 Message Date
Mayuri Nehate
a488557962
feat(ingest/kafka-connect): allow setting platform_instance for kafka… (#8299) 2023-07-13 16:00:42 +05:30
Mayuri Nehate
60b9623675
fix(ingest/tableau): graceful handling of get all datasources failure (#8406) 2023-07-13 12:58:19 +05:30
Andrew Sikowitz
6ce157d0ff
docs(ingest): Add warning for Python 3.7 deprecation (#8411) 2023-07-13 12:57:52 +05:30
Andrew Sikowitz
851c5bab9e
test(ingest/minio): Configure delta lake minio tests for arm64 (#8364)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-13 02:41:07 -04:00
Đặng Minh Dũng
64bcc132a3
feat(ingest/airflow): able to set platform_instance in Dataset (#8313)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-07-12 19:37:36 -04:00
Tamas Nemeth
54c7aef1bc
feat(ingest/presto-on-hive): Extracting all the table properties from Hive Metastore (#8348)
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-12 15:56:13 -03:00
Mayuri Nehate
a2fc02294c
fix(ingest/snowflake): fix azure cloud region ids in external url (#8376)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-12 17:11:33 +05:30
Harshal Sheth
ae7e3c2080
feat(ingest/dbt-cloud): reduce graphql query complexity (#8390) 2023-07-12 17:11:13 +05:30
Andrew Sikowitz
ea3e119210
test(ingest/kafka-connect): Attempt to fix flaky test (#8404) 2023-07-12 00:46:10 -04:00
Harshal Sheth
ffd14eebf1
fix(ingest/bigquery): skip self-references when generating lineage (#8403) 2023-07-11 21:30:57 -04:00
Andrew Sikowitz
71f80b6207
test(ingest/testing-utils): Add back delta info ignore path (#8402) 2023-07-11 20:57:29 -04:00
Andrew Sikowitz
75b79e160a
build(ingest): Inline import testing utils for check cli (#8400) 2023-07-11 16:40:20 -04:00
Harshal Sheth
d4135d57b7
feat(ingest/bigquery): support column-level lineage (#8382) 2023-07-11 11:12:51 -07:00
Andrew Sikowitz
759d0c86af
test(ingest/airflow): Fix test for airflow 2.6.3 (#8393) 2023-07-11 12:55:32 -04:00
Andrew Sikowitz
2261531e31
test(ingest): Aspect level golden file comparison (#8310) 2023-07-11 10:39:47 -04:00
Segun Sogunle
d8888cf0b7
feat(ingest): Add disable flag for TopicRecordNameStrategy (#8224)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-10 11:51:48 -07:00
Tamas Nemeth
e4736d7b46
fix(ingest/s3): Deequ import rearragement (#8389) 2023-07-10 14:40:06 -04:00
Andrew Sikowitz
d92cbe99e0
test(ingest/kafka): Configure kafka connect tests for arm64 (#8362) 2023-07-10 12:27:29 -04:00
Harshal Sheth
3e47b3d228
feat(ingest): schema-aware SQL parsing for column-level lineage (#8334) 2023-07-07 16:24:35 -07:00
Andrew Sikowitz
1f84bf5b2b
fix(ingest/sql-common): Fix profile_table_level_only (#8331)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-07 19:05:50 -04:00
Andrew Sikowitz
8617e072fa
build(ingest): Pin pydeequ to unblock CI (#8381) 2023-07-07 16:18:52 -04:00
Andrew Sikowitz
74301ecb60
test(ingest/nifi): Configure nifi tests for arm64 (#8363)
Co-authored-by: Mayuri N <mayuri.nehate@gslab.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-07-07 12:43:01 -04:00
Harshal Sheth
4daee108c2
chore(ingest/delta-lake): cleanup import error handling (#8230)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-06 16:53:03 -04:00
Andrew Sikowitz
3a21c27f06
feat(ingest): Turn on browse path v2 creation (#8342)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-06 16:43:42 -04:00
Segun Sogunle
b8f5074763
fix(cli): local variable reference before assignment (#8222)
Co-authored-by: John Joyce <john@acryl.io>
2023-07-06 13:16:46 -07:00
Mayuri Nehate
8cf778dc9b
fix(ingest): update pydantic helpers to address unique name issue (#8324)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-07-06 13:16:07 -07:00
Dmitry Bryazgin
294857ab8d
feat(ingest): Add advanced configuration for LDAP manager ingestion (#7784)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-06 13:15:16 -07:00
Harshal Sheth
393b5a3039
ci(docs): add missing deps for lxml package for vercel (#8372) 2023-07-05 15:21:27 -07:00
Andrew Sikowitz
afbd52bdf0
test(ingest/mysql): Configure sql_server tests for arm64 (#8360)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-05 13:41:24 -04:00
Francesco Macagno
5901e94ee0
fix(ingest/airflow): Remove info log on import (#8246)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-05 11:56:59 +02:00
Tamas Nemeth
74ab1bea06
fix(ingest/s3): Fix for flaky s3 test - uploading s3 files in consistent order (#8367) 2023-07-04 19:19:39 +02:00
Ellie O'Neil
ac4af259f2
docs(lineage): add read lineage example (#8322) 2023-06-30 08:48:05 -07:00
Andrew Sikowitz
72a41ef9f6
test(ingest/trino): xfail test to unblock CI (#8340) 2023-06-30 17:51:50 +05:30
Harshal Sheth
08d4e904a8
feat(ingest): add YamlFileUpdater utility (#8266) 2023-06-29 13:15:34 -07:00
Gabe Lyons
55d75a9401
feat(tableau): add config knobs for excluding external links from tableau (#8314) 2023-06-28 10:46:17 -07:00
Mayuri Nehate
75d67b97bc
fix(ingest/postgres): fix profiling errors, skip json type column (#8291) 2023-06-28 10:59:31 -04:00
Mayuri Nehate
711efde2c0
feat(ingest/snowflake): snowpipe s3 lineage (#8262)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-28 10:59:01 -04:00
Andrew Sikowitz
1f0723fad1
fix(ingest/looker): Fix typo on ViewField creation for measures (#8318) 2023-06-28 11:13:45 +05:30
Andrew Sikowitz
8a198cd615
fix(ingest/unity): Pin databricks-sdk and update docs (#8293) 2023-06-27 13:38:55 -04:00
Gabe Lyons
d075bb4824
fix(embed): set embed url to false for tableau config (#8308) 2023-06-27 07:55:16 -07:00
Ellie O'Neil
8880b47ca1
docs(business glossary) Update business glossary docs (#8287) 2023-06-26 11:00:09 -07:00
Andrew Sikowitz
584366771d
refactor(unity): Remove databricks_cli and cleanup (#8249)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-06-23 18:01:05 +05:30
Mayuri Nehate
381d4d4afa
fix(ingest/snowflake): snowflake profiling geometry type (#8279) 2023-06-23 18:00:54 +05:30
Tamas Nemeth
726ee0f6b8
fix(ingest/csv-enricher): Adding extra check in csv enricher to ignore non-urn urns (#8169)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-06-23 12:42:50 +05:30
Ellie O'Neil
795b185f59
docs(ingest/lineage): Update fine grained file lineage docs (#8283) 2023-06-23 12:29:54 +05:30
Andrew Sikowitz
aa5e02d0ec
feat(ingest): Create zero usage aspects (#8205)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-06-22 17:07:50 -04:00
Serhii Dimchenko
5b9fd977eb
fix(ingest/dbt-athena): dbt-athena types mapping for complex types (#8264)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-22 11:40:27 +02:00
mohdsiddique
9c65715c5e
fix(ingestion/powerbi): use admin api resolver to fetch modified workspaces (#8273)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-06-22 11:37:38 +02:00
Tamas Nemeth
d3aed62778
feat(cli): Initial support for sending exceptions to Sentry (#7172) 2023-06-22 10:24:58 +02:00
Andrew Sikowitz
e7312db202
feat(ingest/delete-cli): Add configurable batch size; update docs (#8274) 2023-06-21 14:41:49 -04:00