3975 Commits

Author SHA1 Message Date
Ravindra Lanka
b638bcf442
feat(ingestion): Documentation on adding stateful ingestion use-cases to new sources (#5985) 2022-09-21 09:02:50 -07:00
mohdsiddique
44a67b52cf
feat(ingestion): Tableau dashboard & chart usage stats (#5999)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-20 17:41:56 -07:00
Mayuri Nehate
b195b6c123
fix(ingest): encode reserved characters when creating dataset urn (#5977)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-20 16:59:02 -07:00
Tamas Nemeth
46494bc0ad
feat(ingest): presto-on-hive - Description and hive view support (#5993)
* Presto on Hive Improvements like:
 - Capturing table/column descriptions
 - Adding Hive view support
 - Various bug fixes
2022-09-21 00:40:10 +02:00
Shirshanka Das
8e8fb51ce3
feat(cli): extend put command to support platform creation (#5990) 2022-09-20 11:52:22 -07:00
Aezo
6c865d00c4
feat(ingest): stateful, catalog and pascalcase con (#5890) 2022-09-20 14:54:22 +02:00
Pedro Silva
7e5f44a481
feat(docs): Adds docs on K8s scheduled ingestion (#5984) 2022-09-20 15:04:42 +05:30
Harshal Sheth
8ffa7054e5
feat(ingest): dbt column-level meta mappings + add_terms operation (#5970)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-19 15:26:12 -07:00
John Joyce
df3633e7f3
refactor(looker): Migrating to new browse path format (#5688) 2022-09-19 14:43:35 -07:00
Mayuri Nehate
4062385d6b
fix(ingest): pin great-expectations dependency to fix profiling error (#5980) 2022-09-19 12:45:22 -07:00
Harshal Sheth
98ee621849
fix(ingest): handle uname missing on windows (#5981) 2022-09-19 10:36:16 -07:00
Tamas Nemeth
663a659f66
feat(ingest): bigquery-v2 - Add full profiling capability (#5953) 2022-09-17 09:46:45 +02:00
Harshal Sheth
c8cd7d5040
refactor(ingest): use typed objects in dbt ingestion (#5929) 2022-09-16 15:07:59 -07:00
Mayuri Nehate
a14617b6a4
fix(ingest): continue validation of s3 path_specs even if platform is set (#5951) 2022-09-16 12:03:57 -07:00
Harshal Sheth
937ab192c0
feat(ingest): add support for aliases in plugin registry (#5958) 2022-09-16 07:19:32 -07:00
Harshal Sheth
78f648a6e0
docs(ingest): add example of ssl with mysql (#5954) 2022-09-16 17:46:52 +05:30
skrydal
a026c84691
feat: qualifiedName support + populating glue ARN (#5952) 2022-09-15 21:15:03 -07:00
Shirshanka Das
f70bb201ce
fix(build): fix preflight script for m1 for lib postgresql (#5957) 2022-09-15 13:21:02 -07:00
Shirshanka Das
8bbe3f61f7
fix(docs): fix loom embed (#5956) 2022-09-15 13:19:46 -07:00
Mayuri Nehate
2558129391
refactor(snowflake): move snowflake-beta to certified snowflake source (#5923)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-15 09:53:54 -07:00
Mugdha Hardikar
93681d7a71
feat(ingest): delta-lake - support delta tables from minio (#5758)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-15 09:51:14 -07:00
Harshal Sheth
66985335d0
fix(ingest): add trailing newline to ssh keys (#5947) 2022-09-14 20:53:26 -07:00
Shirshanka Das
14873775bd
feat(docs): lookml - add video for ui ingestion, remove secret step until we support ssh-keys as secrets (#5948) 2022-09-14 20:16:49 -07:00
Harshal Sheth
9384a52b23
feat(ingest): skip ssh known_hosts verification for git clone (#5945) 2022-09-14 16:43:21 -07:00
Ravindra Lanka
cf2f59f85f
fix(ingest): Fix snowflake usage stateful ingestion backward compatibility check. (#5943) 2022-09-14 13:25:50 -07:00
skrydal
f61a040555
feat(ingestion) Add more info to glue entities (#5874)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-14 12:25:09 -07:00
Harshal Sheth
e23523a781
fix(ingest): fix type annotations on some pydantic fields (#5795) 2022-09-14 11:05:31 -07:00
Mayuri Nehate
8ead62dae5
fix(transformer): fix invalid lastModified.actor entry in transformers (#5906) 2022-09-14 09:59:17 -07:00
Harshal Sheth
a1e1d2fd0a
feat(ingest): add ConfigEnum type (#5734) 2022-09-14 09:57:42 -07:00
Shirshanka Das
c5f8fc1907
fix(docs): lookml - clean up links (#5938) 2022-09-14 09:43:59 -07:00
Ravindra Lanka
ee68f09624
feat(ingestion): Refactor standard state-handling tasks into a common handler that are common across all stateful ingestion sources. (#5766) 2022-09-14 09:30:42 -07:00
Harshal Sheth
8f5b773025
fix(airflow): prioritize conn_type over id-based type inference (#5911) 2022-09-14 18:30:03 +02:00
firas omrane
c606abdb40
fix(docker) Add platform to docker-compose command (#5683) 2022-09-14 17:56:42 +05:30
Harshal Sheth
d5d8a38caa
fix(ingest): move git requirement into lookml deps (#5932) 2022-09-14 14:47:07 +05:30
Harshal Sheth
68ea926770
fix(ingest): add dbt redshift type mappings (#5933) 2022-09-14 14:46:21 +05:30
Harshal Sheth
e3f8f29a31
feat(ingest): support version option in datahub client get_aspect_v2 (#5934) 2022-09-14 14:45:01 +05:30
Harshal Sheth
a8ac5f9e33
refactor(ingest): simplify DefaultSQLParser alias (#5935) 2022-09-14 14:44:22 +05:30
Shirshanka Das
15a33fadc8
feat(ingest): lookml - support for git checkout (#5924) 2022-09-13 16:40:59 -07:00
Danilo Peixoto
d1cfdf1894
fix(docs): feast recipe documentation (#5917) 2022-09-13 10:46:47 -07:00
Chris Collins
0be5c39802
feat(ingestion-ui) Display ingestion sources in UI more dynamically (#5789) 2022-09-13 13:27:12 -04:00
Harshal Sheth
dfeced8eee
fix(ingest): hide internal profiler.allow_deny_patterns from config (#5619) 2022-09-13 16:09:10 +05:30
Harshal Sheth
45979edd12
fix(ingest): handle lookml directory being in a symlink'd folder (#5916) 2022-09-13 13:42:44 +05:30
Harshal Sheth
b6d3fe88e5
fix(ingest): lookml - support relative paths in base_folder (#5914) 2022-09-12 13:59:48 -07:00
Harry Nguyen
46bc1fe367
feat(ingestion): schema inference for jsonlines in S3 (#5725)
Co-authored-by: Nguyen Tri Hieu <nhieu@fossil.com>
2022-09-12 10:49:46 -07:00
Mayuri Nehate
aedf1522fb
feat(ingest): snowflake-beta - minor changes, tests (#5910) 2022-09-12 10:42:52 -07:00
Ravindra Lanka
386719f15a
feat(platform): timeseries - Server & Client side changes to support timeseries aspect deletion & rollback. (#4756) 2022-09-11 11:27:46 -07:00
Harshal Sheth
e556bcb306
feat(ingest): add entity type inference to mcpw (#5880) 2022-09-10 20:36:10 -07:00
Harshal Sheth
220ae0b6c9
feat(ingest): make sink use type annotations (#5899) 2022-09-10 19:46:20 -07:00
Harshal Sheth
8f72de5d8b
refactor(ingest): move aspect maps to dedicated file (#5821) 2022-09-10 18:43:11 -07:00
Harshal Sheth
3cd8ea8316
test(ingest): use pytest parameterization for dbt integration tests (#5879) 2022-09-10 18:35:58 -07:00