Harshal Sheth
954397a078
fix(ingest): remove dbt disable_dbt_node_creation and load_schema options ( #5877 )
2022-09-09 14:07:55 -07:00
Marcin Szymański
57996049fb
fix(ingest): sql - add missing trino types ( #5870 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-09 14:07:00 -07:00
Harshal Sheth
2115d5bf1d
fix(ingest): remove dbt delete_tests_as_datasets option ( #5865 )
...
This option was added for backwards compat and can be dropped now.
2022-09-08 11:22:25 -07:00
Ravindra Lanka
228f3b50ea
feat(ingestion): send reports of ingestion runs to datahub ( #5639 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-19 09:08:17 -07:00
Piotr Sierkin
828a711684
feat(ingest): dbt - control over emitting test_results, test_definitions, etc. ( #5328 )
...
Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-06 21:42:53 -07:00
Marcin Szymański
fa42b59d9f
fix(ingestion): correct trino datatype handling ( #5541 )
...
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-08-03 20:55:14 -07:00
mohdsiddique
f1abdc91ee
fix(dbt): fix issue of assertion error when stateful ingestion is used with dbt tests ( #5540 )
...
* fix(dbt): fix issue of dbt stateful ingestion with tests
Co-authored-by: MugdhaHardikar-GSLab <mugdha.hardikar@gslab.com>
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-08-03 13:22:04 -07:00
Aezo
85a55ffac7
fix(ingest): dbt - add support for trino datatypes ( #5379 )
2022-07-26 17:02:00 -07:00
Gabe Lyons
efc5602493
feat(ingest): dbt - updating source lineage logic ( #5414 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-24 23:55:01 -07:00
Shirshanka Das
4b3135a0f8
feat(ingest): dbt - improving dbt_meta mapping ( #5237 )
2022-06-24 13:43:12 +02:00
Piotr Sierkin
9f2b3b9ba6
fix(ingest): dbt - make assertion display names consistent ( #5214 )
...
Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-06-22 22:07:00 -07:00
Shirshanka Das
0ee2569d5c
feat(ingest): dbt - adding support for dbt tests ( #5201 )
2022-06-18 13:30:04 -07:00
skrydal
503208beaa
feat(ingest): dbt - add platform instance for target platform ( #5129 )
2022-06-12 09:29:44 -07:00
Gabe Lyons
538cfba526
fix(ingest): dbt - ensure platform_instance filled out only for dbt nodes ( #5115 )
2022-06-07 20:09:42 -07:00
Gabe Lyons
e607410a40
dont set platform instances for sources ( #5028 )
2022-05-27 10:57:27 -07:00
Gabe Lyons
7b1cf6f8b2
feat(dbt): enable data platform instance on dbt ( #4926 )
2022-05-17 16:53:16 -07:00
Tamas Nemeth
474b0ba61e
feat(ingest): dbt - add query tag mapping and match template ( #4744 )
2022-04-25 10:56:45 -07:00
Arun Vasudevan
5aa3da5c9c
feat(ingestion) dbt: Fixing issue with strip_user_ids_from_email and adding owner_extraction_pattern ( #4587 )
...
* Fixing issue with strip_user_ids_from_email and adding owner_extraction_pattern
Co-authored-by: BZ <93607724+BoyuanZhangDE@users.noreply.github.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-04-13 16:58:36 -07:00
mohdsiddique
57002c766d
feat(stateful dbt): add stateful ingestion capability in dbt source ( #4456 )
...
* feat(stateful dbt): add stateful ingestion capability in dbt source
Co-authored-by: MohdSiddiqueBagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-03-30 18:09:02 -07:00
Shirshanka Das
a69eac8247
feat(ingest): dbt,looker,sql_common,kafka - moving sources to produce display names and subtypes more consistently ( #4496 )
2022-03-27 18:49:26 -05:00
Gabe Lyons
a9c1b7e6a3
fix(ingest): dbt - dont produce key aspects if the entity has no other aspects ( #4217 )
2022-02-22 16:02:58 -08:00
Swaroop Jagadish
7d986ec880
fix(ingest): populate system metadata for all metadata events (mcp, mcpw) ( #3900 )
2022-01-16 12:03:38 -08:00
Swaroop Jagadish
480ad24282
feat(ingest): dbt - add support for dbt meta mapping ( #3832 )
2022-01-05 16:32:05 -08:00
Sergio Gómez Villamor
c59c63e90d
feat: enables dbt metadata files to be loaded from URIs ( #3739 )
2021-12-15 09:11:39 -08:00
Remi
a9754ebe83
fix(ingest): fix dbt source platform when disable_dbt_node_creation is False ( #3592 )
2021-11-17 16:49:48 -08:00
varunbharill
417b555fef
feat(dbt): dbt enhancements - dbt nodes, lineage, subtype, etc. ( #3519 )
2021-11-05 09:27:41 -07:00
AndreasTA-AW
561c04bcf8
feat(dbt-ingestion): add ability to skip specific models ( #3340 )
2021-10-25 22:31:19 -07:00
AndreasTA-AW
61ba3cf578
feat(ingest): added tags and owner from dbt ( #3270 )
...
Co-authored-by: Andreas Törnström Andersson <andreas.tornstrom-andersson@forefront.se>
2021-10-05 07:31:58 -07:00
Gabe Lyons
ff527f4bed
feat(foreign keys): add foreign key models ( #3275 )
2021-09-22 10:29:27 -07:00
rslanka
8844240328
feat: Adding support for nested schemas in ingestion and visualization ( #3079 )
2021-08-11 15:47:18 -07:00
Kevin Hu
3701931363
feat(ingest): extract dbt versions into custom properties ( #3021 )
2021-08-03 12:13:49 -07:00
Gabe Lyons
aa253f5b3b
feat(deletes): add run commands (list, show, rollback) to datahub ingest ( #2960 )
2021-07-29 20:04:40 -07:00
Kevin Hu
f82ea3abef
feat(ingest): ingest descriptions from dbt models ( #2955 )
2021-07-26 10:47:25 -07:00
Kevin Hu
84a1486b89
feat(ingest): dbt aliases ( #2927 )
2021-07-22 07:57:14 -07:00
Kevin Hu
d6875b4f4f
feat(ingest): test dbt ingestion with and without schemas ( #2922 )
2021-07-21 12:56:31 -07:00
Kevin Hu
859fab8bd3
docs(ingest): update sample recipe and test input for dbt ( #2917 )
2021-07-20 13:28:37 -07:00
Kevin Hu
bc84c82a68
feat(ingest): extract dbt meta fields ( #2876 )
2021-07-13 14:58:25 -07:00
Kevin Hu
799b0634e1
fix(ingest): check for dbt materialization before proceeding ( #2842 )
2021-07-08 15:12:37 -07:00
Harshal Sheth
2f921d15e8
fix(ingest): avoid setting timestamps unless source system provides it ( #2843 )
2021-07-08 12:11:06 -07:00
Harshal Sheth
d66381451a
feat(ingest): refactor mce comparison and add pytest update golden files option ( #2812 )
2021-06-30 16:53:20 -07:00
Kevin Hu
22a2ed81e4
feat(ingest): ingest last-modified from dbt sources.json ( #2729 )
2021-06-23 13:56:20 -07:00
Kevin Hu
554e1637c5
fix(ingest): types for dbt ( #2716 )
2021-06-22 10:37:08 -07:00
Gary Lucas
af4f3b9683
fix(dbt): set target platform and load schema ( #2483 )
2021-05-17 12:22:52 -07:00
shakti-garg
8ed14a62e2
feat(business_glossary): add new entity business term and its relationship with dataset and its fields ( #2228 )
...
Co-authored-by: shubham.garg <shubham.garg@thoughtworks.com>
2021-05-10 13:20:23 -07:00
Gabe Lyons
851e00ba9f
feat(lineage): implement support for datasets, charts and dashboards downstream lineage fetching in a generic way ( #2397 )
...
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Brian <brianwebtek@gmail.com>
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2021-04-23 00:18:39 -07:00
Harshal Sheth
a11329d5b8
refactor(ingest): update test harness to use a compose file per test ( #2392 )
2021-04-13 17:30:24 -07:00
Gary Lucas
9d00e8bb2e
fix(test): dbt-manifest files ( #2330 )
2021-04-02 10:59:20 -07:00
Harshal Sheth
a921d0deae
feat(ingest): MongoDB ingestion source ( #2289 )
2021-03-23 20:15:44 -07:00
Gary Lucas
922307f8ec
feat(ingest): Add DBT source to metadata-ingestion ( #2247 )
2021-03-22 23:11:29 -07:00
