3089 Commits

Author SHA1 Message Date
Aseem Bansal
9df70d7355
ingest(elasticsearch): add basic profiling (#8351) 2023-07-20 08:25:30 +05:30
Ellie O'Neil
d734b2849e
feat(ingest/mysql): Add estimate row count for mysql (#8420) 2023-07-20 08:06:26 +05:30
Harshal Sheth
695de42a8c
chore(ingest): hide ignore old/new state options (#8438) 2023-07-19 16:46:32 +05:30
JifeiMei
0def0e5a2e
fix(GreatExpections): AssertionRunEventClass does not match the examp… (#8243) 2023-07-19 16:08:00 +05:30
Harshal Sheth
addf76c849
fix(ingest/bigquery): make sql parsing more robust (#8450) 2023-07-19 16:06:42 +05:30
Arun Vasudevan
cc2dc342c6
fix(ingestion-redshift): Fix Redshift ingestion logs (#8454) 2023-07-18 22:24:54 -04:00
Andrew Sikowitz
48c1dc820e
build(ingest/boto3): Update boto3-stubs to fix CI (#8452) 2023-07-18 21:29:50 +00:00
Andrew Sikowitz
61611b7202
feat(ingest/snowflake): View CLL from sql parsing of view definition (#8419) 2023-07-14 18:50:32 -04:00
Andrew Sikowitz
f41f642eaf
build(ingest/boto3): Update boto3-stubs to fix CI (#8425) 2023-07-14 15:48:04 -07:00
Harshal Sheth
4fb77e4a25
fix(ingest): tweak ingestion exit codes (#8418) 2023-07-14 15:47:16 -07:00
Mayuri Nehate
a488557962
feat(ingest/kafka-connect): allow setting platform_instance for kafka… (#8299) 2023-07-13 16:00:42 +05:30
Mayuri Nehate
60b9623675
fix(ingest/tableau): graceful handling of get all datasources failure (#8406) 2023-07-13 12:58:19 +05:30
Andrew Sikowitz
6ce157d0ff
docs(ingest): Add warning for Python 3.7 deprecation (#8411) 2023-07-13 12:57:52 +05:30
Đặng Minh Dũng
64bcc132a3
feat(ingest/airflow): able to set platform_instance in Dataset (#8313)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-07-12 19:37:36 -04:00
Tamas Nemeth
54c7aef1bc
feat(ingest/presto-on-hive): Extracting all the table properties from Hive Metastore (#8348)
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-12 15:56:13 -03:00
Mayuri Nehate
a2fc02294c
fix(ingest/snowflake): fix azure cloud region ids in external url (#8376)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-12 17:11:33 +05:30
Harshal Sheth
ae7e3c2080
feat(ingest/dbt-cloud): reduce graphql query complexity (#8390) 2023-07-12 17:11:13 +05:30
Harshal Sheth
ffd14eebf1
fix(ingest/bigquery): skip self-references when generating lineage (#8403) 2023-07-11 21:30:57 -04:00
Andrew Sikowitz
71f80b6207
test(ingest/testing-utils): Add back delta info ignore path (#8402) 2023-07-11 20:57:29 -04:00
Andrew Sikowitz
75b79e160a
build(ingest): Inline import testing utils for check cli (#8400) 2023-07-11 16:40:20 -04:00
Harshal Sheth
d4135d57b7
feat(ingest/bigquery): support column-level lineage (#8382) 2023-07-11 11:12:51 -07:00
Andrew Sikowitz
759d0c86af
test(ingest/airflow): Fix test for airflow 2.6.3 (#8393) 2023-07-11 12:55:32 -04:00
Andrew Sikowitz
2261531e31
test(ingest): Aspect level golden file comparison (#8310) 2023-07-11 10:39:47 -04:00
Segun Sogunle
d8888cf0b7
feat(ingest): Add disable flag for TopicRecordNameStrategy (#8224)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-10 11:51:48 -07:00
Tamas Nemeth
e4736d7b46
fix(ingest/s3): Deequ import rearragement (#8389) 2023-07-10 14:40:06 -04:00
Harshal Sheth
3e47b3d228
feat(ingest): schema-aware SQL parsing for column-level lineage (#8334) 2023-07-07 16:24:35 -07:00
Andrew Sikowitz
1f84bf5b2b
fix(ingest/sql-common): Fix profile_table_level_only (#8331)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-07 19:05:50 -04:00
Harshal Sheth
4daee108c2
chore(ingest/delta-lake): cleanup import error handling (#8230)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-06 16:53:03 -04:00
Andrew Sikowitz
3a21c27f06
feat(ingest): Turn on browse path v2 creation (#8342)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-06 16:43:42 -04:00
Segun Sogunle
b8f5074763
fix(cli): local variable reference before assignment (#8222)
Co-authored-by: John Joyce <john@acryl.io>
2023-07-06 13:16:46 -07:00
Mayuri Nehate
8cf778dc9b
fix(ingest): update pydantic helpers to address unique name issue (#8324)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-07-06 13:16:07 -07:00
Dmitry Bryazgin
294857ab8d
feat(ingest): Add advanced configuration for LDAP manager ingestion (#7784)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-06 13:15:16 -07:00
Francesco Macagno
5901e94ee0
fix(ingest/airflow): Remove info log on import (#8246)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-05 11:56:59 +02:00
Harshal Sheth
08d4e904a8
feat(ingest): add YamlFileUpdater utility (#8266) 2023-06-29 13:15:34 -07:00
Gabe Lyons
55d75a9401
feat(tableau): add config knobs for excluding external links from tableau (#8314) 2023-06-28 10:46:17 -07:00
Mayuri Nehate
75d67b97bc
fix(ingest/postgres): fix profiling errors, skip json type column (#8291) 2023-06-28 10:59:31 -04:00
Mayuri Nehate
711efde2c0
feat(ingest/snowflake): snowpipe s3 lineage (#8262)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-28 10:59:01 -04:00
Andrew Sikowitz
1f0723fad1
fix(ingest/looker): Fix typo on ViewField creation for measures (#8318) 2023-06-28 11:13:45 +05:30
Gabe Lyons
d075bb4824
fix(embed): set embed url to false for tableau config (#8308) 2023-06-27 07:55:16 -07:00
Andrew Sikowitz
584366771d
refactor(unity): Remove databricks_cli and cleanup (#8249)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-06-23 18:01:05 +05:30
Mayuri Nehate
381d4d4afa
fix(ingest/snowflake): snowflake profiling geometry type (#8279) 2023-06-23 18:00:54 +05:30
Tamas Nemeth
726ee0f6b8
fix(ingest/csv-enricher): Adding extra check in csv enricher to ignore non-urn urns (#8169)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-06-23 12:42:50 +05:30
Andrew Sikowitz
aa5e02d0ec
feat(ingest): Create zero usage aspects (#8205)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-06-22 17:07:50 -04:00
Serhii Dimchenko
5b9fd977eb
fix(ingest/dbt-athena): dbt-athena types mapping for complex types (#8264)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-22 11:40:27 +02:00
mohdsiddique
9c65715c5e
fix(ingestion/powerbi): use admin api resolver to fetch modified workspaces (#8273)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-06-22 11:37:38 +02:00
Tamas Nemeth
d3aed62778
feat(cli): Initial support for sending exceptions to Sentry (#7172) 2023-06-22 10:24:58 +02:00
Andrew Sikowitz
e7312db202
feat(ingest/delete-cli): Add configurable batch size; update docs (#8274) 2023-06-21 14:41:49 -04:00
Aseem Bansal
af6973ebe0
fix(ingest/okta): Set default of okta connector to match OIDC defaults (#8272) 2023-06-21 19:15:31 +05:30
Mayuri Nehate
88ceac316b
fix(ingest/tableau): split table columns query from datasources query (#8217)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-06-21 11:50:35 +02:00
Andrew Sikowitz
02bc71dd84
fix(ingest/okta): Set default of okta_profile_to_username_attr to email (#8263) 2023-06-21 13:38:59 +05:30