3975 Commits

Author SHA1 Message Date
Aseem Bansal
c0dbea8363
feat(ingest/elastic): reduce number of calls made (#8477) 2023-07-23 17:12:31 +05:30
Aseem Bansal
8fb5912978
fix(mysql): getting table name correctly (#8476)
Co-authored-by: Ellie O'Neil <oneile729@gmail.com>
2023-07-23 17:10:54 +05:30
Harshal Sheth
056d3619f0
feat(sdk): easily generate container urns (#8198)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-07-21 21:14:06 +00:00
mohdsiddique
bec018257c
fix(ingestion/powerbi-report-srever): handle requests.exceptions.JSONDecodeError (#8442)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-07-21 14:01:57 -07:00
Aseem Bansal
f4c0ed3aab
ingest(mysql): add storage bytes information (#8294)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-07-20 17:31:06 -04:00
Mayuri Nehate
f4fde21168
feat(ingest/nifi): add support for basic auth in nifi (#8457) 2023-07-20 14:12:33 -04:00
Harshal Sheth
d733363bed
chore(ingest): drop bigquery-beta and snowflake-beta aliases (#8451) 2023-07-20 14:05:25 -04:00
Harshal Sheth
47616ff093
feat(ingest/lookml): fail when nothing was produced (#8464) 2023-07-20 14:04:30 -04:00
Aseem Bansal
9df70d7355
ingest(elasticsearch): add basic profiling (#8351) 2023-07-20 08:25:30 +05:30
Ellie O'Neil
d734b2849e
feat(ingest/mysql): Add estimate row count for mysql (#8420) 2023-07-20 08:06:26 +05:30
Harshal Sheth
695de42a8c
chore(ingest): hide ignore old/new state options (#8438) 2023-07-19 16:46:32 +05:30
JifeiMei
0def0e5a2e
fix(GreatExpections): AssertionRunEventClass does not match the examp… (#8243) 2023-07-19 16:08:00 +05:30
Harshal Sheth
addf76c849
fix(ingest/bigquery): make sql parsing more robust (#8450) 2023-07-19 16:06:42 +05:30
Arun Vasudevan
cc2dc342c6
fix(ingestion-redshift): Fix Redshift ingestion logs (#8454) 2023-07-18 22:24:54 -04:00
Andrew Sikowitz
48c1dc820e
build(ingest/boto3): Update boto3-stubs to fix CI (#8452) 2023-07-18 21:29:50 +00:00
Andrew Sikowitz
20b3adb7b1
fix(ingest/snowflake): Add sqlglot as snowflake dependency (#8427) 2023-07-14 21:31:24 -04:00
Andrew Sikowitz
61611b7202
feat(ingest/snowflake): View CLL from sql parsing of view definition (#8419) 2023-07-14 18:50:32 -04:00
Andrew Sikowitz
f41f642eaf
build(ingest/boto3): Update boto3-stubs to fix CI (#8425) 2023-07-14 15:48:04 -07:00
Harshal Sheth
4fb77e4a25
fix(ingest): tweak ingestion exit codes (#8418) 2023-07-14 15:47:16 -07:00
Tamas Nemeth
a91c78cf31
fix(ingest/s3): fix test flakiness (#8416) 2023-07-14 00:42:00 +02:00
mohdsiddique
cbbe083731
fix(ingestion/powerbi): increment msal version (#8385)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-07-13 17:33:19 +05:30
Mayuri Nehate
a488557962
feat(ingest/kafka-connect): allow setting platform_instance for kafka… (#8299) 2023-07-13 16:00:42 +05:30
Mayuri Nehate
60b9623675
fix(ingest/tableau): graceful handling of get all datasources failure (#8406) 2023-07-13 12:58:19 +05:30
Andrew Sikowitz
6ce157d0ff
docs(ingest): Add warning for Python 3.7 deprecation (#8411) 2023-07-13 12:57:52 +05:30
Andrew Sikowitz
851c5bab9e
test(ingest/minio): Configure delta lake minio tests for arm64 (#8364)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-13 02:41:07 -04:00
Đặng Minh Dũng
64bcc132a3
feat(ingest/airflow): able to set platform_instance in Dataset (#8313)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-07-12 19:37:36 -04:00
Tamas Nemeth
54c7aef1bc
feat(ingest/presto-on-hive): Extracting all the table properties from Hive Metastore (#8348)
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-12 15:56:13 -03:00
Mayuri Nehate
a2fc02294c
fix(ingest/snowflake): fix azure cloud region ids in external url (#8376)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-12 17:11:33 +05:30
Harshal Sheth
ae7e3c2080
feat(ingest/dbt-cloud): reduce graphql query complexity (#8390) 2023-07-12 17:11:13 +05:30
Andrew Sikowitz
ea3e119210
test(ingest/kafka-connect): Attempt to fix flaky test (#8404) 2023-07-12 00:46:10 -04:00
Harshal Sheth
ffd14eebf1
fix(ingest/bigquery): skip self-references when generating lineage (#8403) 2023-07-11 21:30:57 -04:00
Andrew Sikowitz
71f80b6207
test(ingest/testing-utils): Add back delta info ignore path (#8402) 2023-07-11 20:57:29 -04:00
Andrew Sikowitz
75b79e160a
build(ingest): Inline import testing utils for check cli (#8400) 2023-07-11 16:40:20 -04:00
Harshal Sheth
d4135d57b7
feat(ingest/bigquery): support column-level lineage (#8382) 2023-07-11 11:12:51 -07:00
Andrew Sikowitz
759d0c86af
test(ingest/airflow): Fix test for airflow 2.6.3 (#8393) 2023-07-11 12:55:32 -04:00
Andrew Sikowitz
2261531e31
test(ingest): Aspect level golden file comparison (#8310) 2023-07-11 10:39:47 -04:00
Segun Sogunle
d8888cf0b7
feat(ingest): Add disable flag for TopicRecordNameStrategy (#8224)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-10 11:51:48 -07:00
Tamas Nemeth
e4736d7b46
fix(ingest/s3): Deequ import rearragement (#8389) 2023-07-10 14:40:06 -04:00
Andrew Sikowitz
d92cbe99e0
test(ingest/kafka): Configure kafka connect tests for arm64 (#8362) 2023-07-10 12:27:29 -04:00
Harshal Sheth
3e47b3d228
feat(ingest): schema-aware SQL parsing for column-level lineage (#8334) 2023-07-07 16:24:35 -07:00
Andrew Sikowitz
1f84bf5b2b
fix(ingest/sql-common): Fix profile_table_level_only (#8331)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-07 19:05:50 -04:00
Andrew Sikowitz
8617e072fa
build(ingest): Pin pydeequ to unblock CI (#8381) 2023-07-07 16:18:52 -04:00
Andrew Sikowitz
74301ecb60
test(ingest/nifi): Configure nifi tests for arm64 (#8363)
Co-authored-by: Mayuri N <mayuri.nehate@gslab.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-07-07 12:43:01 -04:00
Harshal Sheth
4daee108c2
chore(ingest/delta-lake): cleanup import error handling (#8230)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-06 16:53:03 -04:00
Andrew Sikowitz
3a21c27f06
feat(ingest): Turn on browse path v2 creation (#8342)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-06 16:43:42 -04:00
Segun Sogunle
b8f5074763
fix(cli): local variable reference before assignment (#8222)
Co-authored-by: John Joyce <john@acryl.io>
2023-07-06 13:16:46 -07:00
Mayuri Nehate
8cf778dc9b
fix(ingest): update pydantic helpers to address unique name issue (#8324)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
2023-07-06 13:16:07 -07:00
Dmitry Bryazgin
294857ab8d
feat(ingest): Add advanced configuration for LDAP manager ingestion (#7784)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-06 13:15:16 -07:00
Harshal Sheth
393b5a3039
ci(docs): add missing deps for lxml package for vercel (#8372) 2023-07-05 15:21:27 -07:00
Andrew Sikowitz
afbd52bdf0
test(ingest/mysql): Configure sql_server tests for arm64 (#8360)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-05 13:41:24 -04:00