3089 Commits

Author SHA1 Message Date
mayurinehate
e19b12030e
fix(GE): fix dependencies for GE DataHubValidationAction, logic for s… (#4347) 2022-03-08 11:20:09 -08:00
Aseem Bansal
beb51ebf59
fix(ingestion): add logging, make job more resilient to errors (#4331) 2022-03-07 14:32:44 -08:00
Salih Can
915798a5ad
fix(ingest): elasticsearch - connector should work with defaults for auth (#4329)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-03-07 13:16:05 -08:00
Swaroop Jagadish
35b187a8d4
feat(ingest): transformers - add support for processing MCP-s (#4337) 2022-03-07 13:14:29 -08:00
Ravindra Lanka
bd6603bec5
fix(ingest): logging - insulate 'datahub' and child loggers from external modules. (#4324) 2022-03-07 12:00:06 -08:00
mayurinehate
92b0e1c7c7
feat(tableau): emit workbook as container entity in tableau source, some minor fixes in tableau source (#4261) 2022-03-04 11:52:04 -08:00
John Joyce
9f1c5a8f75
feat(assertions): Adding Assertions Entity & Great Expectations BETA (#4305) 2022-03-04 11:51:31 -08:00
Aseem Bansal
9a9a5c3084
fix: revert changes to OpenApi casing (#4291) 2022-03-04 10:35:44 -08:00
Tamas Nemeth
a4c1d5d382
fix(ingest): bigquery - remove unneeded warning (#4317) 2022-03-04 09:11:30 -08:00
Arun Vasudevan
2bf648f98a
feat (ingest): kafka-connect: adding Auth to Kafka Connect API (#4298) 2022-03-03 20:43:20 -08:00
Aseem Bansal
53d1379a85
fix(cli): telemetry - failures should not cause the CLI to fail (#4302) 2022-03-03 09:40:01 -08:00
Ravindra Lanka
520968b247
fix(ingest): bigquery - fix stateful ingestion checkpoint reconstruction. (#4295) 2022-03-02 13:10:15 -08:00
Kevin Hu
b2b8826118
fix(ingest): clarify s3/s3a requirements and platform defaults (#4263) 2022-03-02 08:58:27 -08:00
Tamas Nemeth
2a5cf3dd07
feat(ingest): bigquery - ability to disable partition profiling (#4228) 2022-03-01 22:29:48 -08:00
Aseem Bansal
d52638a252
fix(ingest): lib - revert positional arg change, create new method (#4266)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-03-01 22:06:37 -08:00
Ravindra Lanka
ba0c912c88
fix(ingest): snowflake - fix view upstream lineages to eliminate false edges. (#4284) 2022-03-01 16:53:26 -08:00
MugdhaHardikar-GSLab
ec26b0d52a
fix(ingest): fix config parsing for $ (#4258) 2022-03-01 16:11:06 -08:00
Tamas Nemeth
329b0b34b2
feat(ingest): bigquery - supporting v2 audit metadata on Bigquery (#4233) 2022-03-01 16:09:19 -08:00
Aditya Radhakrishnan
1275379d7d
feat(ingest) - cli - always display CLI version (#4282) 2022-03-01 16:07:58 -08:00
mohdsiddique
e2f8db7926
feat: powerbi - add new source (#4201) 2022-02-28 17:37:22 -08:00
Ravindra Lanka
f12f653366
fix(ingest): snowflake - lineage + logging & reporting improvements. (#4276) 2022-02-28 17:34:48 -08:00
RyanHolstien
77a27359f2
fix(platform): timeline - fix semantic version ordering in API response and cli improvements (#4271) 2022-02-28 15:14:13 -08:00
Vishal Shah
93ff09517b
feat(ingest): add lineage_client_project_id field to the BigQuery config (#4138)
* feat(ingest): add lineage_client_project_id field to the bigquery config

* fix linting issues

* add type annotation for arguments
2022-02-28 11:19:23 -08:00
Aseem Bansal
c8a3e68202
fix(docs): doc build failing (#4267) 2022-02-28 10:53:22 -08:00
Vincenzo Lavorini
a113e4357e
fix(ingest): openapi - add support for user, password auth (#4086) 2022-02-24 23:29:01 -08:00
Kevin Hu
02fe05eb8f
feat(ingest): data-lake - remove spark requirement if not profiling (#4131) 2022-02-24 23:26:06 -08:00
Kevin Hu
3d6db93aa6
fix(ingest): fix mixpanel profile emits (#4253) 2022-02-24 21:26:41 -08:00
Ravindra Lanka
1ac20ee764
feat(platform): add timeline api for dataset entity (#4252) 2022-02-24 19:40:55 -08:00
Sunil Patil
16f3d4683a
feat(ingest): elasticsearch - add support for url_prefix in configuration (#4214)
Co-authored-by: Sunil Patil <spatil@twilio.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-24 17:06:38 -08:00
Edward Vaisman
6ff551cbcd
feat(ingest): lineage-file - add ability to provide lineage manually through a file (#4116) 2022-02-24 17:02:38 -08:00
David Sanchez
8f75c278a1
fix(ingest): tableau - fix sheets external url ingestion (#4231) 2022-02-24 15:44:31 -08:00
Arun Vasudevan
a3940962ba
fix(ingest): kafka-connect - pass the env variable as part of making dataset (#4244) 2022-02-24 13:58:52 -08:00
Kevin Hu
46701319dc
feat(ingest): switch telemetry endpoint to Mixpanel (#4238) 2022-02-24 12:35:48 -08:00
Xu Wang
aa3363bcc2
feat(ingest): lib - add better support for working with urns (#4172)
Co-authored-by: Xu Wang <xu.wang@grandrounds.com>
2022-02-22 19:39:24 -08:00
Ravindra Lanka
7f4cb87c57
Revert "fix(ingest): Use lower-case dataset names in the dataset urns for all SQL-styled datasets. (#4140)" (#4218)
This reverts commit 6c75185445bbb23974932ff64cb142ee6bf5b51b.
2022-02-22 16:21:40 -08:00
Ravindra Lanka
84005d3848
feat(ingest): kafka - add support for non-default schema registry subject name strategies (#4215) 2022-02-22 16:05:46 -08:00
Gabe Lyons
a9c1b7e6a3
fix(ingest): dbt - dont produce key aspects if the entity has no other aspects (#4217) 2022-02-22 16:02:58 -08:00
Alexander Chashnikov
c2065bd7fe
feat(ingest): clickhouse - add initial support (#4057)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-21 07:36:08 -08:00
Guido Turtu
3a6d1bde64
feat(ingest): airflow - capture airflow task description (#4147)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-21 00:07:08 -08:00
Kevin Hu
6b5fba882e
fix(ingest): sql-sources: add mapping for postgres types (#4179) 2022-02-20 19:32:15 -08:00
Tamas Nemeth
a4dc4137b7
feat(ingest): sql-sources - prevent hard failure on table/view ingestion exceptions (#4185) 2022-02-20 14:32:59 -08:00
Tamas Nemeth
3d02b5bec8
feat(ingest): bigquery - ignore temporary tables from lineage and connect edges directly (#4160) 2022-02-20 14:23:23 -08:00
Kevin Hu
c3f531fbf9
feat(ingest): profiling - record bucketed profiling runtimes (#4068) 2022-02-20 13:39:25 -08:00
Tamas Nemeth
c7ea6f0db2
fix(ingest): bigquery - profiling works with limit and offset using temp tables (#4161) 2022-02-19 19:20:26 -08:00
abiwill
8bbc66b3e6
fix(ingest): elasticsearch - http/https host config support (#4191)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-02-19 11:42:01 -08:00
Ravindra Lanka
ab633d0047
fix(ingest): snowflake - handle external S3 bucket lineage for "External Tables". (#4192) 2022-02-18 15:52:13 -08:00
Swaroop Jagadish
eaf7b02b2a
docs(model): auto-generated docs and hand-written docs for the metadata model (#4189) 2022-02-18 09:45:45 -08:00
Tamas Nemeth
585aad1aac
Fix for wrong containers on Athena (#4167) 2022-02-17 01:34:02 -08:00
Tamas Nemeth
2e7f3ae6f3
Removing sql_common dep from glue (#4168) 2022-02-17 01:33:29 -08:00
Harshal Sheth
75d2ec2a39
ci(ingestion): fix airflow 1 deps for tox (#4083) 2022-02-17 00:33:28 -08:00