3089 Commits

Author SHA1 Message Date
Aseem Bansal
f81ead366d
feat(redash): add parallelism support for ingestion (#5061) 2022-06-01 19:36:30 +05:30
Felix Lüdin
2a373c21e5
fix(docs): hana - rename SAP HANA source and data platform (#5043) 2022-05-31 14:44:16 -07:00
Aseem Bansal
08a5fcfd01
fix(redash): fix bug with names, add option for page size, debugging info (#5045) 2022-05-31 18:10:34 +05:30
BZ
5b55f25b69
feat(ingest): glue - enable profiling (#4879) 2022-05-30 13:46:35 +02:00
Tamas Nemeth
f40c250e9f
fix(profiling): bigquery - Fix for Bigquery temp table creation on GE >= 0.15.3 (#5035) 2022-05-30 15:43:52 +05:30
Ravindra Lanka
96f923e007
fix(ingest): common - fix nullability determination for the AVRO fixed type. (#5023) 2022-05-29 09:06:08 -07:00
Aseem Bansal
1fd5549ae3
fix(cli): don't use env for container, add example (#5012) 2022-05-29 08:26:45 -07:00
RyanHolstien
2a2fd05345
feat: telemetry improvements (#5029) 2022-05-27 17:04:45 -07:00
Gabe Lyons
e607410a40
dont set platform instances for sources (#5028) 2022-05-27 10:57:27 -07:00
Aseem Bansal
d926fa26f5
fix(parsing): incorrect parsing for commas (#5027) 2022-05-27 22:22:13 +05:30
Aseem Bansal
64bb918939
fix(parsing): improve sql parsing, some debugging redash (#5025) 2022-05-27 19:28:45 +05:30
Aseem Bansal
912ce11821
fix(bigquery): reduce number of calls for details of partitioning (#5014) 2022-05-27 13:09:08 +05:30
Harshal Sheth
480fb2e770
test(ingestion): change class names to avoid unittest warnings (#5005) 2022-05-26 22:09:40 +05:30
Harshal Sheth
06a290fcd6
fix(ingestion): use raw strings for regexes (#5006) 2022-05-26 20:56:02 +05:30
Ravindra Lanka
108b492ed1
feat(ingestion): Add Iceberg source (#5010)
Co-authored-by: cccs-eric <eric.ladouceur@cyber.gc.ca>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-05-26 08:05:57 -07:00
Patrick Franco Braz
567bd0cf86
fix(bigquery-usage): fix audit metadata query template (#5001) 2022-05-26 14:07:55 +02:00
Aseem Bansal
21f8d4be03
fix(redash): improve logging for debugging, add validation for dataset urn, some refactoring (#4997) 2022-05-26 16:24:54 +05:30
buggythepirate
92338c7912
feat(ingest): Added new ingestion source SAP HANA (#4376)
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-26 03:42:50 -07:00
Ebu (えぶ)
2911e1ed1b
feat(ingest): Add Source from Vertica (#4555)
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-26 03:26:28 -07:00
Shirshanka Das
0aacec51c5
fix(cli): timeline - adjust for timeline API changes on server (#4998) 2022-05-25 13:14:07 -07:00
Tamas Nemeth
8ac2bd6b0e
fix(airflow): Fix for Airflow 1 support (#4995) 2022-05-25 20:59:05 +05:30
BZ
2eb7920fe0
feat(dbt): enable dbt read artifacts from s3 (#4935)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-05-25 14:27:02 +05:30
mayurinehate
6f553d4644
fix(ingest): tableau - fix chart custom properties None key error, update docs (#4931) 2022-05-24 22:52:53 -07:00
Aseem Bansal
34f099fbf7
refactor(redash): emit charts first and try with id based dashboard API first (#4991) 2022-05-25 11:00:19 +05:30
mohdsiddique
049db62a49
metabase chart are missing from dashboard (#4942) 2022-05-25 10:55:50 +05:30
Tamas Nemeth
be91e2341f
feat(ingest): s3 - speeding up ingestion with sampling (#4927) 2022-05-24 22:17:10 -07:00
Dexter Lee
669160a677
feat(run): Create a describe run endpoint for fetching aspects created by the ingestion run (#4964) 2022-05-24 14:46:36 -07:00
Claudio Benfatto
bbd0ab823d
feat(ingestion): optionally disable some kafka schema warnings (#4169)
Co-authored-by: Claudio Benfatto <claudio.benfatto@adevinta.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-24 14:27:02 -07:00
mayurinehate
822f0a0ec2
feat(great-expectations): allow DATAHUB_DEBUG env var to enable debug logs in GE Action (#4972) 2022-05-24 12:58:30 -07:00
Aseem Bansal
b77f381429
doc(ingestion): default boolean fix, broken bigquery docgen (#4984) 2022-05-24 23:17:27 +05:30
Aseem Bansal
bbfe500581
fix(ingest): remove new schema field usage (#4987) 2022-05-24 22:02:40 +05:30
Aseem Bansal
e1eb2e589a
fix(redash): use dashboard id if slug does not work (#4985) 2022-05-24 22:00:40 +05:30
Shirshanka Das
7dd1a6f17c
fix(ingest): mode - dashboards without creator info fails to process (#4983) 2022-05-23 21:13:27 -07:00
Aseem Bansal
a9ff203363
feat(bigquery): add partition key tag (#4974) 2022-05-23 21:27:13 +05:30
Aseem Bansal
b5e1ed739c
fix(bigquery): add rate limiting for api calls made (#4967) 2022-05-23 20:14:36 +05:30
Shirshanka Das
0f7ff79368
fix(cli): graph - get_aspect_v2 method fails to deserialize aspects correctly (#4971) 2022-05-22 13:28:39 -07:00
Aseem Bansal
2260118232
feat(bigquery): reduce logging (#4961)
* feat(bigquery): reduce logging

* doc: add entry for behaviour change
2022-05-20 09:42:55 -07:00
mayurinehate
ec356d3aab
feat(cli): raise error if get entity api fails (#4922) 2022-05-19 20:50:59 -07:00
Aseem Bansal
2bb2c5243c
fix(bigquery): add dataset_id for bigquery (#4932) 2022-05-19 10:43:06 +05:30
Gabe Lyons
a8079ca163
feat(transformers): add transformers to provide tags & terms to schema fields based on regex patterns (#4936)
* add tag & term transformers for schemas

* added documentation

* lint fixes

* add clarification that only first set of matching terms is applied
2022-05-18 16:03:34 -07:00
Gabe Lyons
7b1cf6f8b2
feat(dbt): enable data platform instance on dbt (#4926) 2022-05-17 16:53:16 -07:00
Aseem Bansal
9a97af1350
fix(lint): lint failure due to mypy upgrade (#4933)
* fix(lint): lint failure due to mypy upgrade
2022-05-17 21:49:41 +02:00
Aseem Bansal
1a8e3f5ae7
docs(ingest): remove incorrectly annotated lineage capability (#4914) 2022-05-13 09:11:30 -07:00
Pedro Silva
78c3ca039e
feat(gms): Add support for deleting reference pointers when deleting by urn (#4791) 2022-05-12 19:02:13 -07:00
BZ
367fac6066
feat(ingestion): For all usage connectors, allow exclusion of top_n_queries from ingestion via a config param. (#4839)
* feat(redshift-usage): allow users to not ingest top_n_queries

Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-12 14:26:03 -07:00
Aseem Bansal
b9f78026c8
revert(bigquery-usage): dataset allow filter impl (#4901)
* Revert "fix(ingestion): bigquery-usage: Fix biquery usage table deny pattern template (#4898)"
2022-05-11 20:03:03 +02:00
Sebo Kim
f3df15d6dc
fix(ingestion): ElasticSearch when no properties from elastic_mappings, gracefully continue (#4853)
* when no properties from elastic_mappings, gracefully continue

Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-11 09:07:18 -07:00
Tamas Nemeth
6627e6b70f
fix(ingestion): bigquery-usage: Fix biquery usage table deny pattern template (#4898) 2022-05-11 13:59:54 +02:00
Shirshanka Das
8d281fc013
fix(ingest): lookml - add view definitions for all views (#4875) 2022-05-10 10:48:36 -07:00
Aseem Bansal
d0cdadbb3f
fix(sql-parsing): improve error handling (#4862) 2022-05-10 12:18:54 +02:00