3904 Commits

Author SHA1 Message Date
Ravindra Lanka
108b492ed1
feat(ingestion): Add Iceberg source (#5010)
Co-authored-by: cccs-eric <eric.ladouceur@cyber.gc.ca>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-05-26 08:05:57 -07:00
Patrick Franco Braz
567bd0cf86
fix(bigquery-usage): fix audit metadata query template (#5001) 2022-05-26 14:07:55 +02:00
Aseem Bansal
21f8d4be03
fix(redash): improve logging for debugging, add validation for dataset urn, some refactoring (#4997) 2022-05-26 16:24:54 +05:30
buggythepirate
92338c7912
feat(ingest): Added new ingestion source SAP HANA (#4376)
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-26 03:42:50 -07:00
Ravindra Lanka
a8c1df083f
Fix pulsar source docs. (#5011) 2022-05-26 03:34:04 -07:00
Ebu (えぶ)
2911e1ed1b
feat(ingest): Add Source from Vertica (#4555)
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-26 03:26:28 -07:00
Aseem Bansal
c2bb02903c
fix(dbt): missing aws dependency (#5008) 2022-05-26 14:23:50 +05:30
Aseem Bansal
b976393780
fix(bigquery): restrict protobuf version (#5007) 2022-05-26 12:26:45 +05:30
John Joyce
1e428aacd8
feat(DataHub Operations): Adding GraphQL mutation for reporting Dataset operations (#4988) 2022-05-25 13:27:46 -07:00
Shirshanka Das
0aacec51c5
fix(cli): timeline - adjust for timeline API changes on server (#4998) 2022-05-25 13:14:07 -07:00
John Joyce
944be9a1da
feat(Tests): Metadata Tests Models + APIs + UI (Part 1) (#4989) 2022-05-25 08:55:08 -07:00
Tamas Nemeth
8ac2bd6b0e
fix(airflow): Fix for Airflow 1 support (#4995) 2022-05-25 20:59:05 +05:30
BZ
2eb7920fe0
feat(dbt): enable dbt read artifacts from s3 (#4935)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-05-25 14:27:02 +05:30
mayurinehate
6f553d4644
fix(ingest): tableau - fix chart custom properties None key error, update docs (#4931) 2022-05-24 22:52:53 -07:00
mayurinehate
65cf0053a3
feat(model): add created, lastModified auditstamps to SchemaField (#4943) 2022-05-24 22:51:26 -07:00
Aseem Bansal
34f099fbf7
refactor(redash): emit charts first and try with id based dashboard API first (#4991) 2022-05-25 11:00:19 +05:30
mohdsiddique
049db62a49
metabase chart are missing from dashboard (#4942) 2022-05-25 10:55:50 +05:30
Aseem Bansal
2b77468b7b
doc(ingest): update golden file command (#4992) 2022-05-24 22:22:19 -07:00
Tamas Nemeth
be91e2341f
feat(ingest): s3 - speeding up ingestion with sampling (#4927) 2022-05-24 22:17:10 -07:00
Dexter Lee
669160a677
feat(run): Create a describe run endpoint for fetching aspects created by the ingestion run (#4964) 2022-05-24 14:46:36 -07:00
Claudio Benfatto
bbd0ab823d
feat(ingestion): optionally disable some kafka schema warnings (#4169)
Co-authored-by: Claudio Benfatto <claudio.benfatto@adevinta.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-24 14:27:02 -07:00
mayurinehate
822f0a0ec2
feat(great-expectations): allow DATAHUB_DEBUG env var to enable debug logs in GE Action (#4972) 2022-05-24 12:58:30 -07:00
Aseem Bansal
b77f381429
doc(ingestion): default boolean fix, broken bigquery docgen (#4984) 2022-05-24 23:17:27 +05:30
Aseem Bansal
bbfe500581
fix(ingest): remove new schema field usage (#4987) 2022-05-24 22:02:40 +05:30
Aseem Bansal
e1eb2e589a
fix(redash): use dashboard id if slug does not work (#4985) 2022-05-24 22:00:40 +05:30
Shirshanka Das
7dd1a6f17c
fix(ingest): mode - dashboards without creator info fails to process (#4983) 2022-05-23 21:13:27 -07:00
Aseem Bansal
a9ff203363
feat(bigquery): add partition key tag (#4974) 2022-05-23 21:27:13 +05:30
Aseem Bansal
b5e1ed739c
fix(bigquery): add rate limiting for api calls made (#4967) 2022-05-23 20:14:36 +05:30
Shirshanka Das
0f7ff79368
fix(cli): graph - get_aspect_v2 method fails to deserialize aspects correctly (#4971) 2022-05-22 13:28:39 -07:00
Aseem Bansal
87ac8a32aa
doc(ingest): mysql - describe required grants (#4958) 2022-05-20 15:52:44 -07:00
Aseem Bansal
2260118232
feat(bigquery): reduce logging (#4961)
* feat(bigquery): reduce logging

* doc: add entry for behaviour change
2022-05-20 09:42:55 -07:00
Mert Tunç
b38cab0f7c
docs(datahub-kafka-sink): add topic_routes config to doc of datahub-kafka-sink (#4965) 2022-05-19 20:55:10 -07:00
mayurinehate
ec356d3aab
feat(cli): raise error if get entity api fails (#4922) 2022-05-19 20:50:59 -07:00
Aseem Bansal
2bb2c5243c
fix(bigquery): add dataset_id for bigquery (#4932) 2022-05-19 10:43:06 +05:30
Gabe Lyons
a8079ca163
feat(transformers): add transformers to provide tags & terms to schema fields based on regex patterns (#4936)
* add tag & term transformers for schemas

* added documentation

* lint fixes

* add clarification that only first set of matching terms is applied
2022-05-18 16:03:34 -07:00
Gabe Lyons
7b1cf6f8b2
feat(dbt): enable data platform instance on dbt (#4926) 2022-05-17 16:53:16 -07:00
Aseem Bansal
9a97af1350
fix(lint): lint failure due to mypy upgrade (#4933)
* fix(lint): lint failure due to mypy upgrade
2022-05-17 21:49:41 +02:00
Aseem Bansal
1a8e3f5ae7
docs(ingest): remove incorrectly annotated lineage capability (#4914) 2022-05-13 09:11:30 -07:00
mayurinehate
d316fc47ff
docs(transformer): update custom transform example to add missing super init (#4912) 2022-05-13 12:12:32 +05:30
Aseem Bansal
15438f62f1
fix(doc): update doc url to generated docs (#4860) 2022-05-13 10:19:46 +05:30
Pedro Silva
78c3ca039e
feat(gms): Add support for deleting reference pointers when deleting by urn (#4791) 2022-05-12 19:02:13 -07:00
BZ
367fac6066
feat(ingestion): For all usage connectors, allow exclusion of top_n_queries from ingestion via a config param. (#4839)
* feat(redshift-usage): allow users to not ingest top_n_queries

Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-12 14:26:03 -07:00
Aseem Bansal
39fcdd2855
doc(biqquery): add caveat for materialized view (#4859) 2022-05-12 12:48:59 -07:00
Aseem Bansal
5ac6ba58f8
fix(build): Fix breaking changes from GE 0.15.3 (#4905) 2022-05-12 08:46:23 -07:00
Aseem Bansal
b9f78026c8
revert(bigquery-usage): dataset allow filter impl (#4901)
* Revert "fix(ingestion): bigquery-usage: Fix biquery usage table deny pattern template (#4898)"
2022-05-11 20:03:03 +02:00
Sagar Tiwari
830f2b0d6f
fix(docs): Update developing.md to mention directory context (#4899) 2022-05-11 10:00:37 -07:00
Sebo Kim
f3df15d6dc
fix(ingestion): ElasticSearch when no properties from elastic_mappings, gracefully continue (#4853)
* when no properties from elastic_mappings, gracefully continue

Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-05-11 09:07:18 -07:00
Tamas Nemeth
6627e6b70f
fix(ingestion): bigquery-usage: Fix biquery usage table deny pattern template (#4898) 2022-05-11 13:59:54 +02:00
Shirshanka Das
8d281fc013
fix(ingest): lookml - add view definitions for all views (#4875) 2022-05-10 10:48:36 -07:00
Aseem Bansal
d0cdadbb3f
fix(sql-parsing): improve error handling (#4862) 2022-05-10 12:18:54 +02:00