Shirshanka Das
f8171702f6
feat(ci): docker actions simplify, add vulnerability scanner, simplify smoke-tests ( #4881 )
...
co-authored-by: Dexter Lee <dexter@acryl.io>
2022-05-09 19:49:57 -07:00
Zach Bluhm
6ced69cf31
fix(bigquery-usage): dataset allow filter impl ( #4776 )
...
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-06 13:45:37 -07:00
Cristian Calugaru
2a19a85f97
ssl configuration support for elasticsearch source ( #4843 )
2022-05-06 11:22:33 -07:00
Ravindra Lanka
5c64e9d541
fix(ingestion): Allow profiling of only those tables that are allowed by the table_pattern. ( #4842 )
2022-05-06 11:07:31 +02:00
Tamas Nemeth
56ee4d9651
feat(ingest): s3 - add support for multiple pathspecs in one recipe ( #4777 )
2022-05-05 10:09:47 -07:00
mayurinehate
b2c82dcaa3
feat(ingest): great-expectations - add more logs ( #4832 )
2022-05-05 09:25:58 -07:00
Aseem Bansal
bddfd89002
fix(ingestion): bigquery - extract temp table prefix as config, fix reporting, logging ( #4766 )
2022-05-05 10:02:10 +02:00
mayurinehate
d3fb6ce026
fix(ingest): great-expectations - fix failure to serialize type Decimal ( #4763 )
2022-05-04 22:56:08 -07:00
Ravindra Lanka
842fb391eb
feat(ingestion): kafka - add protobuf schema support ( #4819 )
...
Co-authored-by: Luis Angel Vicente Sanchez <luis.vicentesanchez@aaqua.live>
2022-05-04 17:07:01 -07:00
Tamas Nemeth
075d19ef16
fix(doc): improving docs across multiple sources ( #4815 )
2022-05-04 12:57:26 -07:00
Aseem Bansal
3ff53b417b
fix(snowflake): passing connect args should not cause failures ( #4764 )
...
* fix(snowflake): passing connect args should not cause failures
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-03 05:20:11 -07:00
Aseem Bansal
6828dc3d4c
fix(snowflake): get external tables when there is default namespace ( #4803 )
2022-05-03 15:35:04 +05:30
Vladislavs Gaidass
8a24408cbf
fix(bigquery): improve handling of extracted audit log sql queries ( #4735 )
2022-05-03 14:43:23 +05:30
Ravindra Lanka
2b62ed5260
fix(ingest): avro - fix schema field type for avro logical types ( #4801 )
2022-05-02 17:43:42 -07:00
Aditya Radhakrishnan
c20a47f34c
feat(operation): display the reported time for last updated in the UI ( #4800 )
2022-05-02 16:00:29 -07:00
Shirshanka Das
a9ad138172
feat(ingest): docs - overhaul source connector docs to make it code driven ( #4798 )
...
Co-authored-by: MugdhaHardikar-GSLab <mugdha.hardikar@gslab.com>
2022-05-02 00:18:15 -07:00
mayurinehate
c34a1ba735
fix(s3): improved handling for corner cases ( #4774 )
2022-04-29 12:25:41 -07:00
vanmeete
74d6d35881
feat(ingestion): add Pulsar source ( #4721 )
2022-04-29 15:57:02 +05:30
Aseem Bansal
ee6413f0ab
feat(snowflake): add option to disable checking role grants ( #4760 )
...
* feat(ingest): add option to disable checking role grants
2022-04-29 10:14:05 +02:00
Aseem Bansal
37c1fce964
fix(snowflake): improve debug log for external tables ( #4772 )
2022-04-29 10:10:46 +02:00
Jordan Wolinsky
bbac4a7a11
feat(ingestion): glue/s3 - Ingest Tags from s3 bucket on an AWS Glue job and S3 Data Lake Ingest Job ( #4689 )
2022-04-29 10:09:06 +02:00
Shirshanka Das
d0eb772301
fix(ingest): fwk - datahub_api should be initialized by datahub-rest … ( #4786 )
2022-04-28 22:31:19 -07:00
John Joyce
91f166c60b
fix(ingest): fwk - fix how we import DataHub actions ( #4784 )
2022-04-28 21:18:15 -07:00
Aseem Bansal
9d1168317c
fix(lint): lib upgrade caused ( #4773 )
2022-04-28 20:35:20 +05:30
Aditya Radhakrishnan
a7d76e43b5
fix(azure_ad): make redirect and graph_url optional parameters and update docs ( #4754 )
2022-04-27 15:19:49 -07:00
mayurinehate
33d6842ab0
fix(tableau): miscellaneous tableau fixes for lineage, browse path, non-embedded datasets ( #4724 )
...
* fix(tableau): add config whether to emit aspects for external datasets
other changes:
- do not set browse path in absence of datasource or project name
- remove unused nodes from tableau metadata query
* fix(tableau): remove redundant (transitive) lineage edges between tables, datasource, sheet
other changes:
- update subtypes for datasource to be more specific
* fix(tableau): fix browse paths for custom sql and embedded datasource
other changes:
- do not set browse path if any intermediate folder level in browse path is empty
* docs(tableau): update tableau doc
2022-04-27 11:20:03 -07:00
Paweł Iwiński
810b6a4655
feat(ingestion): aws - Custom endpoint url and proxies in S3. ( #4708 )
...
This feature will allow user to specify custom endpoint url with custom proxies to connect to dedicated S3 bucket not associated with the amazon aws.
2022-04-27 14:13:08 +02:00
Jordan Wolinsky
c6424f7a8a
fix(ingest): aws - When using aws_profile, grab temporary credentials from the session. ( #4751 )
...
* allow for temporary credentials generated when using an aws_profile. Mostly used for SSO and temporary credentials
2022-04-27 11:48:57 +02:00
Danilo Peixoto
d2a6bc06dc
feat(ingest): feast - add support for Feast 0.18, deprecate older integration ( #4094 )
2022-04-26 14:35:02 -07:00
cccs-eric
abf8d62cf5
fix(azure_ad): silently discard other Azure AD object types ( #4693 ) ( #4704 )
2022-04-26 13:56:46 -07:00
Sebo Kim
958b52f2f4
fix(ingest): bigquery - Fix BigQuery Datetime/Timestamp type column partition table profile bug ( #4658 )
...
* fix BigQuery Datetime type column partition table profile bug
* inplace datetime replace
* extract out 'if' blocks and write a unit-test
* parse logic inside get_partition_range func
2022-04-26 16:54:19 +02:00
John Joyce
95487555c5
feat(cli): Eagerly load datahub actions CLI commands ( #4748 )
2022-04-26 19:00:41 +05:30
Eddie (Xiao Di) Li
01a5b13a7a
feat(Cassandra): Add Cassandra implementation of EntityService ( #3286 )
2022-04-25 17:29:24 -07:00
Shirshanka Das
7e9e9ddca0
fix(cli): telemetry - make config file processing more robust ( #4738 )
2022-04-25 11:05:24 -07:00
Tamas Nemeth
474b0ba61e
feat(ingest): dbt - add query tag mapping and match template ( #4744 )
2022-04-25 10:56:45 -07:00
Shirshanka Das
a55073036d
fix(cli): align default sink env variables across all cli commands ( #4739 )
2022-04-25 10:19:17 -07:00
Shirshanka Das
a518e3d13e
feat(cli): improve error reporting, make sink config optional ( #4718 )
2022-04-24 17:12:21 -07:00
John Joyce
b1b1898752
Revert "fix(timelineCli): fix naming for timeline cli ( #4729 )" ( #4731 )
...
This reverts commit 7bcc2d9f62a7513b8bd8bc74d6eea0c85f07e245.
2022-04-22 15:15:47 -07:00
RyanHolstien
7bcc2d9f62
fix(timelineCli): fix naming for timeline cli ( #4729 )
2022-04-22 13:33:17 -07:00
Shirshanka Das
3353451561
fix(transformers): add snapshot aspects from dataset into base_transformer ( #4719 )
2022-04-22 08:46:55 -07:00
Kevin Hu
df514cfd6e
feat(ingest): mysql - add decimal128 custom type ( #4624 )
2022-04-21 16:43:29 -07:00
mayurinehate
9e30a9cc81
fix(glue): fix error for custom connector if ignore_unsupported_conne… ( #4667 )
2022-04-21 11:33:04 -07:00
BZ
bbfc902950
fix(ingestion): glue - delete CatalogId parameter from get_jobs api call ( #4646 )
2022-04-21 09:30:01 +02:00
Aseem Bansal
c66ef7c1fe
fix(snowflake): deprecate config, update examples ( #4644 )
...
* fix(snowflake): deprecate config, update examples
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-04-20 15:21:09 -07:00
Aditya Radhakrishnan
15e90f6dd0
feat(ingest) - update identity sources to add flags for masking sensitive work units ( #4711 )
2022-04-20 14:21:08 -07:00
Tamas Nemeth
bb2b8515ff
fix(ingest) bigquery: Moving bigquery temporary credential deletion to atexit ( #4701 )
2022-04-20 13:35:16 -07:00
Tamas Nemeth
b217c8cd4e
fix(looker): Pydantic validation error for Looker TransportOptions on python 3.8 ( #4705 )
...
* Fix for pydantic validation error for Looker TransportOptions on python 3.8
2022-04-20 22:27:56 +02:00
Aseem Bansal
4b7f407e26
fix(bigquery): error due to not handling date properly ( #4702 )
2022-04-20 18:14:33 +02:00
Aseem Bansal
bb0a87ae74
fix(snowflake): remove extra lineage edges in reports, change badly named config variable ( #4595 )
...
* fix(snowflake): remove extra lineage edges in reports
2022-04-20 07:03:54 -07:00
Aseem Bansal
98d4fd4ea9
fix(cli): rest emitter should override config and env variables ( #4622 )
...
* fix(cli): rest emitter should override env variables
* fix(cli): change to not update env variables, small refactor
* fix bug
2022-04-18 07:31:01 -07:00
