312 Commits

Author SHA1 Message Date
Harshal Sheth
3e907ab0d1
feat(ingest): loosen sqlalchemy dep & support airflow 2.3+ (#6204)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-11 15:04:36 -05:00
Oleksandr
0a0cf70b6f
feat(ingest): power-bi - add support for on-prem PowerBI Report Server (#5369) 2022-11-01 16:05:42 -07:00
mohdsiddique
fbb1b7271c
feat(ingestion): business-glossary - Add values and relatedTerms support (#6148)
* WIP

* updated snapshot

* merge

* while creating BusinessGlossaryConfig object, the version field always get set to None

* WIP

* lint fix

* remove field

* doc update

* rename the fields

* review comments

* lintFix

* Fix the business glossary example link

* small fixes to examples and docs

* some small tweaks to the model annotations

Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2022-11-01 15:17:30 -07:00
Tamas Nemeth
0dc2d6af25
feat(ingest): databricks - Unity catalog source (#6292)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-29 21:44:41 -07:00
Kevin Neville
3eae087a30
chore(ci): update actions versions addressing warnings in builds. (#6308) 2022-10-28 17:10:30 -07:00
David Schmidt
5934a0e79f
fix(docs): fix custom transformer example interface (#6309) 2022-10-28 17:00:18 -07:00
Tamas Nemeth
94fae0a464
feat(ingest): bigquery - Promoting bigquery-beta to bigquery source (#6222)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-26 11:15:44 -07:00
Shirshanka Das
46453e4b10
docs(ingest): lookml - add docs for multi-project lookml ingestion (#6284) 2022-10-25 19:24:46 -07:00
Arne Laponin
bbdc185ac2
feat(ingest): add platform instance to tableau (#5978)
Co-authored-by: Arne Laponin <arne.laponin@thoughtworks.com>
Co-authored-by: Amanda Hernando <110099762+amanda-her@users.noreply.github.com>
Co-authored-by: amanda-her <912amandahernando@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-13 18:44:06 -07:00
Harshal Sheth
0427122f5d
fix(ingest): allow hiding some fields from the schema (#6077) 2022-10-10 20:46:59 -07:00
Tamas Nemeth
0f04cd6431
fix(ingest): bigquery-beta - eliminate the need for data.read permission for table schema (#6146)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-07 10:19:16 -07:00
Tamas Nemeth
6db0925b26
fix(ingest): bigquery-beta - fix for missing key error if dataset was empty (#6133) 2022-10-06 09:20:49 -07:00
Shirshanka Das
5fa8818dd3
docs(tableau): adding an ingestion video (#6124) 2022-10-05 14:50:44 -07:00
Harshal Sheth
7e08a05a22
docs(ingest): clarify bigquery-beta multiproject setup (#6071) 2022-09-27 23:52:55 -07:00
Mayuri Nehate
dc8b9d3c4d
docs(ingest): add path spec examples for s3 (#6050) 2022-09-26 18:06:54 -07:00
Harshal Sheth
036f7202dd
docs(ingest): add example of dbt column_meta_mapping (#6038) 2022-09-25 23:40:59 -07:00
Tamas Nemeth
722be5dc0c
fix(ingest): bigquery-beta - Project name quote and graceful lineage/usage failures (#6035) 2022-09-23 19:25:34 +02:00
Ravindra Lanka
b638bcf442
feat(ingestion): Documentation on adding stateful ingestion use-cases to new sources (#5985) 2022-09-21 09:02:50 -07:00
Harshal Sheth
8ffa7054e5
feat(ingest): dbt column-level meta mappings + add_terms operation (#5970)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-19 15:26:12 -07:00
Tamas Nemeth
663a659f66
feat(ingest): bigquery-v2 - Add full profiling capability (#5953) 2022-09-17 09:46:45 +02:00
Harshal Sheth
78f648a6e0
docs(ingest): add example of ssl with mysql (#5954) 2022-09-16 17:46:52 +05:30
Shirshanka Das
8bbe3f61f7
fix(docs): fix loom embed (#5956) 2022-09-15 13:19:46 -07:00
Mayuri Nehate
2558129391
refactor(snowflake): move snowflake-beta to certified snowflake source (#5923)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-15 09:53:54 -07:00
Shirshanka Das
14873775bd
feat(docs): lookml - add video for ui ingestion, remove secret step until we support ssh-keys as secrets (#5948) 2022-09-14 20:16:49 -07:00
Shirshanka Das
c5f8fc1907
fix(docs): lookml - clean up links (#5938) 2022-09-14 09:43:59 -07:00
Shirshanka Das
15a33fadc8
feat(ingest): lookml - support for git checkout (#5924) 2022-09-13 16:40:59 -07:00
Danilo Peixoto
d1cfdf1894
fix(docs): feast recipe documentation (#5917) 2022-09-13 10:46:47 -07:00
Harshal Sheth
954397a078
fix(ingest): remove dbt disable_dbt_node_creation and load_schema options (#5877) 2022-09-09 14:07:55 -07:00
Shirshanka Das
d911b15da5
docs(ingest): improve doc gen, docs for snowflake, looker (#5867) 2022-09-08 00:38:10 -07:00
mohdsiddique
2f65e2f226
feat(transformers): Add semantics & transform_aspect support in transformers (#5514)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-06 14:44:14 -07:00
Harshal Sheth
bee27e8468
fix(ingest): use AwsConnectionConfig instead of AwsSourceConfig (#5813) 2022-09-05 21:10:33 -07:00
David Sanchez
a092a770a3
docs(ingest): bigquery - add needed delete role to list (#5826) 2022-09-05 21:04:33 -07:00
Mayuri Nehate
4bf4236e29
feat(ingest): snowflake-beta - populate size, add table level profiles for large tables (#5774) 2022-09-04 22:37:57 -07:00
Gabe Lyons
570367ce4a
docs(lookml): updating github_info in lookml docs (#5779) 2022-08-31 10:23:10 -07:00
Harshal Sheth
680c4a8f81
chore(ingest): drop data-lake source in favor of s3 source (#5628) 2022-08-15 22:30:40 -07:00
Mugdha Hardikar
a449e8ba7d
feat(mssql): add multi database ingest support (#5516)
* feat(mssql): add multi database ingest support

* Delete older golden file.

* Update s3.md

* fix test setup
2022-08-15 21:37:47 -07:00
Mayuri Nehate
dc08bedd6e
feat(ingest): snowflake - add snowflake-beta connector (#5517) 2022-08-15 20:54:02 -07:00
Harshal Sheth
355c129c7c
chore(ingest): drop python 3.6 support (#5521) 2022-08-10 15:00:31 -07:00
mohdsiddique
0a3721cab3
feat(ingest): power-bi - make ownership ingestion optional (#5335)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-08-08 11:26:00 -07:00
Piotr Sierkin
828a711684
feat(ingest): dbt - control over emitting test_results, test_definitions, etc. (#5328)
Co-authored-by: Piotr Sierkin <piotr.sierkin@getindata.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-06 21:42:53 -07:00
xiphl
16ff4aaa94
feat(ingest) Allow ingestion of Elasticsearch index template (#5444)
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
2022-08-03 20:58:07 -07:00
Sergio Gómez Villamor
0455ae00d5
doc(bigquery): groups grants by requirements (#5468) 2022-07-22 18:49:37 +05:30
Mugdha Hardikar
1f4d3722e2
fix(ingest): hive - add column comment as a column description (#5449) 2022-07-20 12:32:02 -07:00
Mayuri Nehate
4b515e022b
feat(ingest): salesforce - add connector (#5104)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Vincent Koc <koconder@users.noreply.github.com>
2022-07-06 10:01:16 -07:00
Tamas Nemeth
f08c3f784f
fix(ingest): bigquery - Fix for bigquery error when there was no bigquery catalog specified (#5303) 2022-07-01 17:47:07 +02:00
Mugdha Hardikar
5455122267
feat(ingest): delta-lake: adding support for delta lake (#5259)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-06-27 16:16:33 -07:00
Shirshanka Das
4b3135a0f8
feat(ingest): dbt - improving dbt_meta mapping (#5237) 2022-06-24 13:43:12 +02:00
mayurinehate
022ef2f17c
feat(tableau): use pagination for all connection queries (#5204) 2022-06-22 08:53:31 -07:00
Shirshanka Das
0ee2569d5c
feat(ingest): dbt - adding support for dbt tests (#5201) 2022-06-18 13:30:04 -07:00
Aseem Bansal
a79d02abe1
doc(bigquery): fix missing permissions (#5041) 2022-05-30 14:09:12 +02:00