Harshal Sheth
9384a52b23
feat(ingest): skip ssh known_hosts verification for git clone ( #5945 )
2022-09-14 16:43:21 -07:00
Ravindra Lanka
ee68f09624
feat(ingestion): Refactor standard state-handling tasks into a common handler that are common across all stateful ingestion sources. ( #5766 )
2022-09-14 09:30:42 -07:00
Shirshanka Das
15a33fadc8
feat(ingest): lookml - support for git checkout ( #5924 )
2022-09-13 16:40:59 -07:00
Mayuri Nehate
aedf1522fb
feat(ingest): snowflake-beta - minor changes, tests ( #5910 )
2022-09-12 10:42:52 -07:00
Harshal Sheth
3cd8ea8316
test(ingest): use pytest parameterization for dbt integration tests ( #5879 )
2022-09-10 18:35:58 -07:00
Harshal Sheth
954397a078
fix(ingest): remove dbt disable_dbt_node_creation and load_schema options ( #5877 )
2022-09-09 14:07:55 -07:00
Marcin Szymański
57996049fb
fix(ingest): sql - add missing trino types ( #5870 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-09 14:07:00 -07:00
Harshal Sheth
2115d5bf1d
fix(ingest): remove dbt delete_tests_as_datasets option ( #5865 )
...
This option was added for backwards compat and can be dropped now.
2022-09-08 11:22:25 -07:00
Mayuri Nehate
a8c1397e5e
feat(ingest): tableau - improve lineage, workbooks query, fix pagination ( #5756 )
2022-09-05 23:19:22 -07:00
mohdsiddique
dbcd3099d4
fix(ingestion): looker - fix dashboard and chart-usage calculations ( #5791 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2022-08-31 16:22:11 -07:00
mohdsiddique
1f1056d7cf
feat(ingestion): looker - chart usage statistics ( #5652 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-30 00:21:31 -07:00
Harshal Sheth
fa0fd7b830
fix(ingest): tweak mongodb schema inference to fix test ( #5744 )
2022-08-26 11:15:16 -07:00
Mugdha Hardikar
e448bb8832
feat(ingest): mysql - support multiple database in single recipe ( #5684 )
2022-08-26 19:47:49 +02:00
Harshal Sheth
d7bdfc2b86
fix(ingest): redshift-usage: check full table names with AllowDenyPattern ( #5702 )
...
* fix(ingest): redshift-usage: check full table/schema names with AllowDenyPattern
* schema pattern does not take dot syntax
* update redshift filter test
2022-08-24 14:33:18 -07:00
mohdsiddique
7d532b8886
fix(ingestion): ingest tables from dba_tables in oracle source ( #5592 )
...
* inspector object wrapper for oracle inspector
* WIP
* before golden file creation
* fix(ingestion): ingest tables from dba_tables in oracle source
* code review comments
* address review comments
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2022-08-24 09:48:08 -07:00
Gabe Lyons
d9b2a45c5f
feat(looker): index looker charts and dashboards by business term ( #5649 )
2022-08-24 08:01:56 -07:00
Ravindra Lanka
2a56d2c276
fix(ci): ingest - fix nifi integration tests ( #5718 )
2022-08-23 21:46:01 -07:00
Peter Szalai
150a36e236
feat(ingest): presto-on-hive - Add mode option to presto-on-hive source ( #5659 )
2022-08-23 23:18:27 +02:00
Aditya Radhakrishnan
4e2bfdd236
fix(ingest): emit status aspect for entities ingested from okta and azure_ad ( #5700 )
2022-08-22 13:28:35 -07:00
Ravindra Lanka
228f3b50ea
feat(ingestion): send reports of ingestion runs to datahub ( #5639 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-19 09:08:17 -07:00
Aezo
8d6a9e08c9
feat(ingest): GE profiler Trino support ( #5361 )
...
* feat(ingest): GE Profile/Action Trino support
* bumping GE version
* updated golden test files
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-08-16 09:46:52 -07:00
Harshal Sheth
680c4a8f81
chore(ingest): drop data-lake source in favor of s3 source ( #5628 )
2022-08-15 22:30:40 -07:00
Mugdha Hardikar
a449e8ba7d
feat(mssql): add multi database ingest support ( #5516 )
...
* feat(mssql): add multi database ingest support
* Delete older golden file.
* Update s3.md
* fix test setup
2022-08-15 21:37:47 -07:00
Mayuri Nehate
dc08bedd6e
feat(ingest): snowflake - add snowflake-beta connector ( #5517 )
2022-08-15 20:54:02 -07:00
Aditya Radhakrishnan
b042ab1ab0
fix(ingest): fix indenting issue in azure ad connector ( #5627 )
2022-08-11 14:10:28 -07:00
Harshal Sheth
355c129c7c
chore(ingest): drop python 3.6 support ( #5521 )
2022-08-10 15:00:31 -07:00
liyuhui666
d33eecbad5
feat(ingest): clickhouse - add metadata modification time and data size ( #5330 )
...
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-08-08 14:00:02 -07:00
Jordan Wolinsky
3a86ff3485
Fix profiling when using {table}. ( #5531 )
...
* profiling fix for when using {table}
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-08-08 13:16:59 -07:00
mohdsiddique
0a3721cab3
feat(ingest): power-bi - make ownership ingestion optional ( #5335 )
...
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-08-08 11:26:00 -07:00
Mugdha Hardikar
b32a0723a1
feat(ingest): delta-lake - extract table history into operation aspect ( #5277 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-07 11:35:09 -07:00
Shirshanka Das
da0258c6ad
fix(ingest): moving delta-lake connector to be 3.7+ only ( #5584 )
2022-08-07 09:32:18 -07:00
Piotr Sierkin
828a711684
feat(ingest): dbt - control over emitting test_results, test_definitions, etc. ( #5328 )
...
Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-06 21:42:53 -07:00
Marcin Szymański
fa42b59d9f
fix(ingestion): correct trino datatype handling ( #5541 )
...
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-08-03 20:55:14 -07:00
mohdsiddique
f1abdc91ee
fix(dbt): fix issue of assertion error when stateful ingestion is used with dbt tests ( #5540 )
...
* fix(dbt): fix issue of dbt stateful ingestion with tests
Co-authored-by: MugdhaHardikar-GSLab <mugdha.hardikar@gslab.com>
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-08-03 13:22:04 -07:00
Harshal Sheth
b1ff0300e6
test(ingest): mark trino/hana tests as xfail due to flakes ( #5549 )
2022-08-03 10:30:36 +05:30
Alexey Kravtsov
c182fe7fca
feat(ingestion): add groups to ldap users ( #5470 )
2022-08-01 15:04:30 -07:00
Aditya Radhakrishnan
c8fc246319
feat(identity): update azure and okta connectors to emit Origin aspects ( #5495 )
2022-07-28 16:03:57 -07:00
Harshal Sheth
690443ce14
fix(ingest): cleanup unused flake8 noqa statements ( #5492 )
...
* fix(ingest): cleanup unused flake8 noqa statements
In the future, we can discover these using `flake8-noqa`.
* add back c901
2022-07-27 22:02:32 +05:30
Aezo
85a55ffac7
fix(ingest): dbt - add support for trino datatypes ( #5379 )
2022-07-26 17:02:00 -07:00
Mayuri Nehate
455ec182e5
feat(model, ingest): add user email in dashboard user usage counts ( #5471 )
2022-07-25 23:06:53 -07:00
Gabe Lyons
efc5602493
feat(ingest): dbt - updating source lineage logic ( #5414 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-24 23:55:01 -07:00
Mayuri Nehate
e6ce4324c9
fix(ingest): looker - pass transport options to all api calls ( #5417 )
2022-07-17 13:55:20 -07:00
Shirshanka Das
f387fa6149
fix(ingest): ensure upgrade checks run async ( #5383 )
2022-07-16 17:38:33 -07:00
Pedro Silva
bc96cf22e7
fix(metadata-ingestion) Fix broken csv enricher test ( #5406 )
2022-07-15 19:00:36 +05:30
Mayuri Nehate
2c48329810
feat(model): dashboard usage model, is_null condition added ( #5397 )
2022-07-15 15:37:06 +05:30
Pedro Silva
b2edd44b6a
Adds support for Domains in CSV source ( #5372 )
2022-07-15 14:20:41 +05:30
Felix Lüdin
a0303448ba
feat(dashboards): add datasets field to DashboardInfo aspect ( #5188 )
...
Co-authored-by: John Joyce <john@acryl.io>
2022-07-14 09:54:02 -07:00
Tamas Nemeth
4334248953
feat(airflow): Circuit breaker and python api for Assertion and Operation ( #5196 )
2022-07-13 10:17:38 -07:00
Shirshanka Das
860d475c5e
feat(ingest): improve domain ingestion usability ( #5366 )
2022-07-11 09:37:38 -07:00
Shirshanka Das
558a65a3c3
fix(ci): fix mysql test and attempt kafka-connect ingestion ( #5352 )
2022-07-07 08:28:34 -07:00
