3975 Commits

Author SHA1 Message Date
Francesco Macagno
5901e94ee0
fix(ingest/airflow): Remove info log on import (#8246)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-07-05 11:56:59 +02:00
Tamas Nemeth
74ab1bea06
fix(ingest/s3): Fix for flaky s3 test - uploading s3 files in consistent order (#8367) 2023-07-04 19:19:39 +02:00
Ellie O'Neil
ac4af259f2
docs(lineage): add read lineage example (#8322) 2023-06-30 08:48:05 -07:00
Andrew Sikowitz
72a41ef9f6
test(ingest/trino): xfail test to unblock CI (#8340) 2023-06-30 17:51:50 +05:30
Harshal Sheth
08d4e904a8
feat(ingest): add YamlFileUpdater utility (#8266) 2023-06-29 13:15:34 -07:00
Gabe Lyons
55d75a9401
feat(tableau): add config knobs for excluding external links from tableau (#8314) 2023-06-28 10:46:17 -07:00
Mayuri Nehate
75d67b97bc
fix(ingest/postgres): fix profiling errors, skip json type column (#8291) 2023-06-28 10:59:31 -04:00
Mayuri Nehate
711efde2c0
feat(ingest/snowflake): snowpipe s3 lineage (#8262)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-28 10:59:01 -04:00
Andrew Sikowitz
1f0723fad1
fix(ingest/looker): Fix typo on ViewField creation for measures (#8318) 2023-06-28 11:13:45 +05:30
Andrew Sikowitz
8a198cd615
fix(ingest/unity): Pin databricks-sdk and update docs (#8293) 2023-06-27 13:38:55 -04:00
Gabe Lyons
d075bb4824
fix(embed): set embed url to false for tableau config (#8308) 2023-06-27 07:55:16 -07:00
Ellie O'Neil
8880b47ca1
docs(business glossary) Update business glossary docs (#8287) 2023-06-26 11:00:09 -07:00
Andrew Sikowitz
584366771d
refactor(unity): Remove databricks_cli and cleanup (#8249)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-06-23 18:01:05 +05:30
Mayuri Nehate
381d4d4afa
fix(ingest/snowflake): snowflake profiling geometry type (#8279) 2023-06-23 18:00:54 +05:30
Tamas Nemeth
726ee0f6b8
fix(ingest/csv-enricher): Adding extra check in csv enricher to ignore non-urn urns (#8169)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-06-23 12:42:50 +05:30
Ellie O'Neil
795b185f59
docs(ingest/lineage): Update fine grained file lineage docs (#8283) 2023-06-23 12:29:54 +05:30
Andrew Sikowitz
aa5e02d0ec
feat(ingest): Create zero usage aspects (#8205)
Co-authored-by: Mayuri Nehate <33225191+mayurinehate@users.noreply.github.com>
2023-06-22 17:07:50 -04:00
Serhii Dimchenko
5b9fd977eb
fix(ingest/dbt-athena): dbt-athena types mapping for complex types (#8264)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-22 11:40:27 +02:00
mohdsiddique
9c65715c5e
fix(ingestion/powerbi): use admin api resolver to fetch modified workspaces (#8273)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-06-22 11:37:38 +02:00
Tamas Nemeth
d3aed62778
feat(cli): Initial support for sending exceptions to Sentry (#7172) 2023-06-22 10:24:58 +02:00
Andrew Sikowitz
e7312db202
feat(ingest/delete-cli): Add configurable batch size; update docs (#8274) 2023-06-21 14:41:49 -04:00
Aseem Bansal
af6973ebe0
fix(ingest/okta): Set default of okta connector to match OIDC defaults (#8272) 2023-06-21 19:15:31 +05:30
Mayuri Nehate
88ceac316b
fix(ingest/tableau): split table columns query from datasources query (#8217)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2023-06-21 11:50:35 +02:00
Andrew Sikowitz
02bc71dd84
fix(ingest/okta): Set default of okta_profile_to_username_attr to email (#8263) 2023-06-21 13:38:59 +05:30
Andrew Sikowitz
2751a09284
fix(ingest): pass platform correctly to browse path v2 helper (#8244) 2023-06-15 20:10:15 -07:00
Shubham Jagtap
35a4434490
fix(sdk/dataflow): deprecate cluster and use env and platform_instance instead (#8201)
Co-authored-by: mohdsiddique <mohdsiddiquebagwan@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-15 14:06:28 -07:00
mohdsiddique
e7e07a73b4
feat(ingestion/powerbi): Ingest datasets not used in PowerBI visualization(tiles/pages) (#8212)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-06-15 14:04:40 -07:00
Tamas Nemeth
a3ceee46d0
fix(ingest/druid) Handling gracefully if no table returned in a schema (#8203) 2023-06-15 18:40:21 +05:30
mohdsiddique
25fa4ebbef
fix(ingestion/tableau): Remove unused field documentViewId (#8225)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-06-15 16:38:30 +05:30
Andrew Sikowitz
66806a805e
feat(ingest/unity): Set external url for containers and datasets (#8238) 2023-06-15 09:05:49 +02:00
RyanHolstien
f78f7f9609
feat(patch): support datajob input output (#8190)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-06-14 22:21:16 -07:00
Harshal Sheth
2d7692a245
feat(sdk): support patches as MCPs in file source (#8220)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-06-14 14:56:27 -07:00
Andrew Sikowitz
c5cc53b99a
feat(ingest/bigquery_v2): enable platform instance using project id (#8216)
Co-authored-by: Adrián Pertíñez <khurzak92@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-06-14 09:50:21 -07:00
Harshal Sheth
1528935d40
fix(cli): make quickstart robust to docker race conditions (#8233) 2023-06-13 23:28:03 -07:00
Harshal Sheth
e5f9ee178c
fix(cli/quickstart): handle docker hangs gracefully (#8211)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-06-13 14:34:06 -05:00
Mayuri Nehate
8e8f2f5e65
fix(ingest/delta-lake): fix CI issues due to delta lake version bump (#8215) 2023-06-12 08:02:04 -07:00
Mayuri Nehate
f3cf9b7d5a
fix(glue): fix typo in reported warning, report with flow_urn (#8138) 2023-06-12 05:08:20 -07:00
Mayuri Nehate
ab3fe0da81
refractor(classification): datahub classifier init (#8193) 2023-06-12 05:07:03 -07:00
Andrew Sikowitz
f2c66fd8d3
feat(ingest): Produce browse paths v2 on demand and with platform instance (#8173)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-06-09 10:35:54 -07:00
Andrew Sikowitz
369a04ae30
revert(ingest/bigquery): Do not emit DataPlatformInstance; remove references to platform_instance (#8196) 2023-06-09 13:44:24 +05:30
Tamas Nemeth
8d31a18e1f
fix(ingest/redshift): Adding env parameter where it was missing for urn generation (#8199) 2023-06-09 08:38:33 +02:00
Andrew Sikowitz
a04631bb5c
fix(ingest/bigquery): Swap log order to avoid confusion (#8197) 2023-06-08 21:05:06 +00:00
Harshal Sheth
2e1d31b6ae
fix(ingest/profiling): only apply monkeypatches once when profiling (#8160) 2023-06-08 12:59:17 +05:30
mohdsiddique
45e592b7c6
fix(ingestion/looker): ingest looks not part of dashboard (#8140)
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
2023-06-08 12:54:14 +05:30
Mayuri Nehate
ac06cf3d3f
feat(classification): configurable minimum values threshold (#8186) 2023-06-07 21:28:13 -07:00
Andrew Sikowitz
fd790e33d8
fix(ingest/kafka): Better error handling around topic and topic description extraction (#8183) 2023-06-07 12:52:38 -07:00
Tamas Nemeth
ea79a4b6af
feat(ingest/presto-on-hive): add support for extra properties and merge property capabilities (#8147)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-06-07 13:12:20 +02:00
Andrew Sikowitz
9fa8489cb8
feat(ingest/snowflake): Okta OAuth support; update docs (#8157) 2023-06-07 01:09:05 -07:00
Andrew Sikowitz
7041281bbe
build(ingest/feast): Pin feast to minor version (#8180) 2023-06-07 10:04:42 +02:00
Andrew Sikowitz
6bad15be5c
fix(ingest): Fix modeldocgen; bump feast to relax pyarrow constraint (#8178) 2023-06-06 13:12:10 -07:00