927 Commits

Author SHA1 Message Date
Andrew Sikowitz
439cf4d7dc
test(ingest/vertica): Skip integration test failing CI; support arm Macs (#8694) 2023-08-22 16:27:46 -04:00
Jinlin Yang
6748aecdc0
fix(ingest/s3): emit data_platform_instance aspect if the config has platform_instance (#8585) 2023-08-17 10:40:54 +05:30
kr_Deepankar
23ac9062fe
feat(ingestion/ldap): flag to ingest ldap users with email instead of username (#8606) 2023-08-16 10:33:18 +05:30
Mayuri Nehate
ddcd5109dc
feat(ingest): allow relative start time config (#8562)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-08-14 17:48:06 -07:00
alplatonov
11fdfcf956
Fix(ingestion/clickhouse) move to two tier sqlalchemy (#8300)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-08-11 16:11:40 -04:00
alplatonov
b58f9bb396
Feat(ingest/ldap)fix list index out of range error (#8525)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-08-09 13:13:27 -04:00
Mayuri Nehate
b4e104f190
fix(ingest/snowflake): maintain qualified name casing, do not lowercase (#8574) 2023-08-04 10:43:22 -07:00
mohdsiddique
6a36118b4f
feat(ingestion/snowflake): use user email-id in urn generation for top users stat (#8513)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-08-03 08:30:50 +05:30
mohdsiddique
05ef7db45e
fix(ingetion/mssql): convert dataset urns to lowercase (#8551)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-08-02 13:44:05 -07:00
TusharM
e2919cfd06
feat(ingest/kafka-connect): add support for Confluent S3 Sink Connector (#8298) 2023-08-02 15:09:50 +05:30
Harshal Sheth
08a39c4b0d
fix(ingest/presto): fix presto on hive test failures (#8548) 2023-08-02 12:46:39 +05:30
Mayuri Nehate
e67f811034
feat(classification): allow parallelisation to reduce time (#8368) 2023-08-02 09:53:39 +05:30
zheyu001
bf47d65412
feat(ingest/presto-on-hive): enable partition key for presto-on-hive (#8380) 2023-08-02 09:53:27 +05:30
Gabe Lyons
843f82b943
feat(presto-on-hive): allow v1 fieldpaths in the presto-on-hive source (#8474) 2023-08-01 14:05:50 -07:00
mohdsiddique
547e1f498e
feat(ingestion/tableau): support column level lineage for custom sql (#8466)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-08-01 19:35:18 +05:30
VISHAL KUMAR
ef3b9489aa
feat(ingest/vertica): performance improvement and bug fixes (#8328)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-08-01 19:34:35 +05:30
Mayuri Nehate
b9060db515
chore(ingest/tableau): miscellaneous cleanup refractor (#8417)
Co-authored-by: Andrew Sikowitz <andrew.sikowitz@acryl.io>
2023-07-27 17:02:48 +05:30
Andrew Sikowitz
bf9f380350
fix(ingest): Generate browse paths v2 for more sources; properly pass platform_instance (#8501) 2023-07-25 11:35:34 +05:30
Aseem Bansal
8fb5912978
fix(mysql): getting table name correctly (#8476)
Co-authored-by: Ellie O'Neil <oneile729@gmail.com>
2023-07-23 17:10:54 +05:30
mohdsiddique
bec018257c
fix(ingestion/powerbi-report-srever): handle requests.exceptions.JSONDecodeError (#8442)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-07-21 14:01:57 -07:00
Mayuri Nehate
f4fde21168
feat(ingest/nifi): add support for basic auth in nifi (#8457) 2023-07-20 14:12:33 -04:00
Ellie O'Neil
d734b2849e
feat(ingest/mysql): Add estimate row count for mysql (#8420) 2023-07-20 08:06:26 +05:30
Tamas Nemeth
a91c78cf31
fix(ingest/s3): fix test flakiness (#8416) 2023-07-14 00:42:00 +02:00
Mayuri Nehate
a488557962
feat(ingest/kafka-connect): allow setting platform_instance for kafka… (#8299) 2023-07-13 16:00:42 +05:30
Mayuri Nehate
60b9623675
fix(ingest/tableau): graceful handling of get all datasources failure (#8406) 2023-07-13 12:58:19 +05:30
Andrew Sikowitz
851c5bab9e
test(ingest/minio): Configure delta lake minio tests for arm64 (#8364)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-13 02:41:07 -04:00
Tamas Nemeth
54c7aef1bc
feat(ingest/presto-on-hive): Extracting all the table properties from Hive Metastore (#8348)
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-12 15:56:13 -03:00
Andrew Sikowitz
ea3e119210
test(ingest/kafka-connect): Attempt to fix flaky test (#8404) 2023-07-12 00:46:10 -04:00
Tamas Nemeth
e4736d7b46
fix(ingest/s3): Deequ import rearragement (#8389) 2023-07-10 14:40:06 -04:00
Andrew Sikowitz
d92cbe99e0
test(ingest/kafka): Configure kafka connect tests for arm64 (#8362) 2023-07-10 12:27:29 -04:00
Andrew Sikowitz
1f84bf5b2b
fix(ingest/sql-common): Fix profile_table_level_only (#8331)
Co-authored-by: Aseem Bansal <asmbansal2@gmail.com>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-07 19:05:50 -04:00
Andrew Sikowitz
74301ecb60
test(ingest/nifi): Configure nifi tests for arm64 (#8363)
Co-authored-by: Mayuri N <mayuri.nehate@gslab.com>
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-07-07 12:43:01 -04:00
Andrew Sikowitz
3a21c27f06
feat(ingest): Turn on browse path v2 creation (#8342)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-07-06 16:43:42 -04:00
Andrew Sikowitz
afbd52bdf0
test(ingest/mysql): Configure sql_server tests for arm64 (#8360)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-05 13:41:24 -04:00
Tamas Nemeth
74ab1bea06
fix(ingest/s3): Fix for flaky s3 test - uploading s3 files in consistent order (#8367) 2023-07-04 19:19:39 +02:00
Andrew Sikowitz
72a41ef9f6
test(ingest/trino): xfail test to unblock CI (#8340) 2023-06-30 17:51:50 +05:30
Mayuri Nehate
75d67b97bc
fix(ingest/postgres): fix profiling errors, skip json type column (#8291) 2023-06-28 10:59:31 -04:00
Gabe Lyons
d075bb4824
fix(embed): set embed url to false for tableau config (#8308) 2023-06-27 07:55:16 -07:00
Andrew Sikowitz
584366771d
refactor(unity): Remove databricks_cli and cleanup (#8249)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-06-23 18:01:05 +05:30
Andrew Sikowitz
aa5e02d0ec
feat(ingest): Create zero usage aspects (#8205)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-06-22 17:07:50 -04:00
Serhii Dimchenko
5b9fd977eb
fix(ingest/dbt-athena): dbt-athena types mapping for complex types (#8264)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-22 11:40:27 +02:00
Aseem Bansal
af6973ebe0
fix(ingest/okta): Set default of okta connector to match OIDC defaults (#8272) 2023-06-21 19:15:31 +05:30
Mayuri Nehate
88ceac316b
fix(ingest/tableau): split table columns query from datasources query (#8217)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-06-21 11:50:35 +02:00
Andrew Sikowitz
02bc71dd84
fix(ingest/okta): Set default of okta_profile_to_username_attr to email (#8263) 2023-06-21 13:38:59 +05:30
mohdsiddique
e7e07a73b4
feat(ingestion/powerbi): Ingest datasets not used in PowerBI visualization(tiles/pages) (#8212)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-06-15 14:04:40 -07:00
Andrew Sikowitz
66806a805e
feat(ingest/unity): Set external url for containers and datasets (#8238) 2023-06-15 09:05:49 +02:00
Andrew Sikowitz
c5cc53b99a
feat(ingest/bigquery_v2): enable platform instance using project id (#8216)
Co-authored-by: Adrián Pertíñez <khurzak92@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-14 09:50:21 -07:00
Mayuri Nehate
ab3fe0da81
refractor(classification): datahub classifier init (#8193) 2023-06-12 05:07:03 -07:00
Andrew Sikowitz
369a04ae30
revert(ingest/bigquery): Do not emit DataPlatformInstance; remove references to platform_instance (#8196) 2023-06-09 13:44:24 +05:30
mohdsiddique
45e592b7c6
fix(ingestion/looker): ingest looks not part of dashboard (#8140)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-06-08 12:54:14 +05:30