Shirshanka Das
0ee2569d5c
feat(ingest): dbt - adding support for dbt tests ( #5201 )
2022-06-18 13:30:04 -07:00
Aseem Bansal
4541379024
feat(build): changes to decrease build time, cancel runs in case of multiple commits ( #5187 )
2022-06-17 18:05:10 +05:30
Tamas Nemeth
b0db17e237
feat(ingest): bigquery - option to send usage queries as well as operational metadata ( #5151 )
2022-06-16 16:00:35 -07:00
skrydal
503208beaa
feat(ingest): dbt - add platform instance for target platform ( #5129 )
2022-06-12 09:29:44 -07:00
cccs-eric
f25a5f043d
fix(iceberg): Change how MapType are mapped to Avro to support complex Map key type. ( #5060 )
2022-06-09 08:13:10 +05:30
Sebo Kim
da273802be
fix(ingest): looker - original view name should be used for explore_joins ( #4928 )
2022-06-08 18:42:55 -07:00
Gabe Lyons
538cfba526
fix(ingest): dbt - ensure platform_instance filled out only for dbt nodes ( #5115 )
2022-06-07 20:09:42 -07:00
Shirshanka Das
601ee9eae1
feat(looker): ensure explore name matches looker's display name ( #5111 )
2022-06-07 16:49:53 -07:00
John Joyce
5c501fba2c
fix(cli): Bumping avro-gen3 to 0.7.4 ( #5098 )
2022-06-07 12:56:06 -07:00
Gabe Lyons
3c7c83f447
tests(lookml): adding tests for model deny patterns ( #4934 )
2022-06-06 08:26:27 -07:00
Lal Rishav
4f82e29697
feat(ingest): snowflake using oauth for Azure ( #4647 )
2022-06-06 17:49:49 +05:30
Ravindra Lanka
96f923e007
fix(ingest): common - fix nullability determination for the AVRO fixed type. ( #5023 )
2022-05-29 09:06:08 -07:00
Gabe Lyons
e607410a40
dont set platform instances for sources ( #5028 )
2022-05-27 10:57:27 -07:00
Aseem Bansal
d926fa26f5
fix(parsing): incorrect parsing for commas ( #5027 )
2022-05-27 22:22:13 +05:30
Aseem Bansal
64bb918939
fix(parsing): improve sql parsing, some debugging redash ( #5025 )
2022-05-27 19:28:45 +05:30
Ravindra Lanka
26c0acf786
fix(build): fix for hana build failure for aarch64. ( #5019 )
2022-05-26 15:29:21 -07:00
Harshal Sheth
480fb2e770
test(ingestion): change class names to avoid unittest warnings ( #5005 )
2022-05-26 22:09:40 +05:30
Ravindra Lanka
108b492ed1
feat(ingestion): Add Iceberg source ( #5010 )
...
Co-authored-by: cccs-eric <eric.ladouceur@cyber.gc.ca>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-05-26 08:05:57 -07:00
buggythepirate
92338c7912
feat(ingest): Added new ingestion source SAP HANA ( #4376 )
...
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-26 03:42:50 -07:00
Ebu (えぶ)
2911e1ed1b
feat(ingest): Add Source from Vertica ( #4555 )
...
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-26 03:26:28 -07:00
John Joyce
1e428aacd8
feat(DataHub Operations): Adding GraphQL mutation for reporting Dataset operations ( #4988 )
2022-05-25 13:27:46 -07:00
mayurinehate
6f553d4644
fix(ingest): tableau - fix chart custom properties None key error, update docs ( #4931 )
2022-05-24 22:52:53 -07:00
Tamas Nemeth
be91e2341f
feat(ingest): s3 - speeding up ingestion with sampling ( #4927 )
2022-05-24 22:17:10 -07:00
Claudio Benfatto
bbd0ab823d
feat(ingestion): optionally disable some kafka schema warnings ( #4169 )
...
Co-authored-by: Claudio Benfatto <claudio.benfatto@adevinta.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-24 14:27:02 -07:00
Gabe Lyons
a8079ca163
feat(transformers): add transformers to provide tags & terms to schema fields based on regex patterns ( #4936 )
...
* add tag & term transformers for schemas
* added documentation
* lint fixes
* add clarification that only first set of matching terms is applied
2022-05-18 16:03:34 -07:00
Gabe Lyons
7b1cf6f8b2
feat(dbt): enable data platform instance on dbt ( #4926 )
2022-05-17 16:53:16 -07:00
Aseem Bansal
15438f62f1
fix(doc): update doc url to generated docs ( #4860 )
2022-05-13 10:19:46 +05:30
BZ
367fac6066
feat(ingestion): For all usage connectors, allow exclusion of top_n_queries from ingestion via a config param. ( #4839 )
...
* feat(redshift-usage): allow users to not ingest top_n_queries
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-12 14:26:03 -07:00
Aseem Bansal
b9f78026c8
revert(bigquery-usage): dataset allow filter impl ( #4901 )
...
* Revert "fix(ingestion): bigquery-usage: Fix biquery usage table deny pattern template (#4898 )"
2022-05-11 20:03:03 +02:00
Sebo Kim
f3df15d6dc
fix(ingestion): ElasticSearch when no properties from elastic_mappings, gracefully continue ( #4853 )
...
* when no properties from elastic_mappings, gracefully continue
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-11 09:07:18 -07:00
Shirshanka Das
8d281fc013
fix(ingest): lookml - add view definitions for all views ( #4875 )
2022-05-10 10:48:36 -07:00
Zach Bluhm
6ced69cf31
fix(bigquery-usage): dataset allow filter impl ( #4776 )
...
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-06 13:45:37 -07:00
Tamas Nemeth
56ee4d9651
feat(ingest): s3 - add support for multiple pathspecs in one recipe ( #4777 )
2022-05-05 10:09:47 -07:00
mayurinehate
d3fb6ce026
fix(ingest): great-expectations - fix failure to serialize type Decimal ( #4763 )
2022-05-04 22:56:08 -07:00
Ravindra Lanka
842fb391eb
feat(ingestion): kafka - add protobuf schema support ( #4819 )
...
Co-authored-by: Luis Angel Vicente Sanchez <luis.vicentesanchez@aaqua.live>
2022-05-04 17:07:01 -07:00
Aseem Bansal
3ff53b417b
fix(snowflake): passing connect args should not cause failures ( #4764 )
...
* fix(snowflake): passing connect args should not cause failures
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-03 05:20:11 -07:00
Vladislavs Gaidass
8a24408cbf
fix(bigquery): improve handling of extracted audit log sql queries ( #4735 )
2022-05-03 14:43:23 +05:30
Ravindra Lanka
df75eafcfc
fix(ci): fix presto_on_hive tests. ( #4802 )
2022-05-02 21:09:33 -07:00
Ravindra Lanka
2b62ed5260
fix(ingest): avro - fix schema field type for avro logical types ( #4801 )
2022-05-02 17:43:42 -07:00
Aditya Radhakrishnan
c20a47f34c
feat(operation): display the reported time for last updated in the UI ( #4800 )
2022-05-02 16:00:29 -07:00
Shirshanka Das
a9ad138172
feat(ingest): docs - overhaul source connector docs to make it code driven ( #4798 )
...
Co-authored-by: MugdhaHardikar-GSLab <mugdha.hardikar@gslab.com>
2022-05-02 00:18:15 -07:00
mayurinehate
c34a1ba735
fix(s3): improved handling for corner cases ( #4774 )
2022-04-29 12:25:41 -07:00
vanmeete
74d6d35881
feat(ingestion): add Pulsar source ( #4721 )
2022-04-29 15:57:02 +05:30
Jordan Wolinsky
bbac4a7a11
feat(ingestion): glue/s3 - Ingest Tags from s3 bucket on an AWS Glue job and S3 Data Lake Ingest Job ( #4689 )
2022-04-29 10:09:06 +02:00
Shirshanka Das
d0eb772301
fix(ingest): fwk - datahub_api should be initialized by datahub-rest … ( #4786 )
2022-04-28 22:31:19 -07:00
mayurinehate
33d6842ab0
fix(tableau): miscellaneous tableau fixes for lineage, browse path, non-embedded datasets ( #4724 )
...
* fix(tableau): add config whether to emit aspects for external datasets
other changes:
- do not set browse path in absence of datasource or project name
- remove unused nodes from tableau metadata query
* fix(tableau): remove redundant (transitive) lineage edges between tables, datasource, sheet
other changes:
- update subtypes for datasource to be more specific
* fix(tableau): fix browse paths for custom sql and embedded datasource
other changes:
- do not set browse path if any intermediate folder level in browse path is empty
* docs(tableau): update tableau doc
2022-04-27 11:20:03 -07:00
Danilo Peixoto
d2a6bc06dc
feat(ingest): feast - add support for Feast 0.18, deprecate older integration ( #4094 )
2022-04-26 14:35:02 -07:00
cccs-eric
abf8d62cf5
fix(azure_ad): silently discard other Azure AD object types ( #4693 ) ( #4704 )
2022-04-26 13:56:46 -07:00
Sebo Kim
958b52f2f4
fix(ingest): bigquery - Fix BigQuery Datetime/Timestamp type column partition table profile bug ( #4658 )
...
* fix BigQuery Datetime type column partition table profile bug
* inplace datetime replace
* extract out 'if' blocks and write a unit-test
* parse logic inside get_partition_range func
2022-04-26 16:54:19 +02:00
Tamas Nemeth
474b0ba61e
feat(ingest): dbt - add query tag mapping and match template ( #4744 )
2022-04-25 10:56:45 -07:00
