Mayuri Nehate
a14617b6a4
fix(ingest): continue validation of s3 path_specs even if platform is set ( #5951 )
2022-09-16 12:03:57 -07:00
Harshal Sheth
937ab192c0
feat(ingest): add support for aliases in plugin registry ( #5958 )
2022-09-16 07:19:32 -07:00
skrydal
a026c84691
feat: qualifiedName support + populating glue ARN ( #5952 )
2022-09-15 21:15:03 -07:00
Mayuri Nehate
2558129391
refactor(snowflake): move snowflake-beta to certified snowflake source ( #5923 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-15 09:53:54 -07:00
Mugdha Hardikar
93681d7a71
feat(ingest): delta-lake - support delta tables from minio ( #5758 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-15 09:51:14 -07:00
Harshal Sheth
9384a52b23
feat(ingest): skip ssh known_hosts verification for git clone ( #5945 )
2022-09-14 16:43:21 -07:00
skrydal
f61a040555
feat(ingestion) Add more info to glue entities ( #5874 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-14 12:25:09 -07:00
Harshal Sheth
e23523a781
fix(ingest): fix type annotations on some pydantic fields ( #5795 )
2022-09-14 11:05:31 -07:00
Harshal Sheth
a1e1d2fd0a
feat(ingest): add ConfigEnum type ( #5734 )
2022-09-14 09:57:42 -07:00
Ravindra Lanka
ee68f09624
feat(ingestion): Refactor standard state-handling tasks into a common handler that are common across all stateful ingestion sources. ( #5766 )
2022-09-14 09:30:42 -07:00
Shirshanka Das
15a33fadc8
feat(ingest): lookml - support for git checkout ( #5924 )
2022-09-13 16:40:59 -07:00
Mayuri Nehate
aedf1522fb
feat(ingest): snowflake-beta - minor changes, tests ( #5910 )
2022-09-12 10:42:52 -07:00
Harshal Sheth
e556bcb306
feat(ingest): add entity type inference to mcpw ( #5880 )
2022-09-10 20:36:10 -07:00
Harshal Sheth
220ae0b6c9
feat(ingest): make sink use type annotations ( #5899 )
2022-09-10 19:46:20 -07:00
Harshal Sheth
3cd8ea8316
test(ingest): use pytest parameterization for dbt integration tests ( #5879 )
2022-09-10 18:35:58 -07:00
Harshal Sheth
954397a078
fix(ingest): remove dbt disable_dbt_node_creation and load_schema options ( #5877 )
2022-09-09 14:07:55 -07:00
Marcin Szymański
57996049fb
fix(ingest): sql - add missing trino types ( #5870 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-09 14:07:00 -07:00
Shirshanka Das
056add128d
fix(ingest): datahub-api - move instantiation to the right config class ( #5878 )
2022-09-09 13:34:21 -07:00
Harshal Sheth
6063484714
fix(ingest): avrogen handling for missing fields with default values ( #5844 )
2022-09-08 14:05:28 -07:00
Harshal Sheth
2115d5bf1d
fix(ingest): remove dbt delete_tests_as_datasets option ( #5865 )
...
This option was added for backwards compat and can be dropped now.
2022-09-08 11:22:25 -07:00
Harshal Sheth
08622f25ef
feat(ingest): add utility for converting MCEs to MCPs ( #5812 )
2022-09-06 15:25:48 -07:00
mohdsiddique
2f65e2f226
feat(transformers): Add semantics & transform_aspect support in transformers ( #5514 )
...
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-06 14:44:14 -07:00
Mayuri Nehate
a8c1397e5e
feat(ingest): tableau - improve lineage, workbooks query, fix pagination ( #5756 )
2022-09-05 23:19:22 -07:00
Shirshanka Das
9afda47085
feat(cli): add support for sampled reporting to keep logs manageable ( #5800 )
2022-09-01 14:47:28 -07:00
Harshal Sheth
c05f3970fd
feat(ingest): cli - add rewrite option for metadata file check ( #5763 )
2022-09-01 14:30:00 -07:00
mohdsiddique
dbcd3099d4
fix(ingestion): looker - fix dashboard and chart-usage calculations ( #5791 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2022-08-31 16:22:11 -07:00
mohdsiddique
1f1056d7cf
feat(ingestion): looker - chart usage statistics ( #5652 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-30 00:21:31 -07:00
Tamas Nemeth
4572c96d60
feat(ingestion): bigquery - Bigquery beta connector - first cut ( #5663 )
2022-08-30 07:33:24 +02:00
Harshal Sheth
eb87db9813
fix(ingest): proper null skip logic in serialization ( #5749 )
2022-08-29 16:34:58 -07:00
Ravindra Lanka
b23195d3df
Fix sqllineage parser to handle special tokens with a hyphens in the table and column names. ( #5748 )
2022-08-26 18:11:00 -07:00
Harshal Sheth
fa0fd7b830
fix(ingest): tweak mongodb schema inference to fix test ( #5744 )
2022-08-26 11:15:16 -07:00
Mugdha Hardikar
e448bb8832
feat(ingest): mysql - support multiple database in single recipe ( #5684 )
2022-08-26 19:47:49 +02:00
liyuhui666
08f5a44df0
feat(elasticsearch): Add nested type display ( #5524 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Ravindra Lanka <rslanka@gmail.com>
2022-08-26 09:07:03 -07:00
David Haglund
b830247727
fix(superset): do not crash when display_uri is not set ( #5711 )
2022-08-24 23:26:02 -07:00
Harshal Sheth
d7bdfc2b86
fix(ingest): redshift-usage: check full table names with AllowDenyPattern ( #5702 )
...
* fix(ingest): redshift-usage: check full table/schema names with AllowDenyPattern
* schema pattern does not take dot syntax
* update redshift filter test
2022-08-24 14:33:18 -07:00
mohdsiddique
7d532b8886
fix(ingestion): ingest tables from dba_tables in oracle source ( #5592 )
...
* inspector object wrapper for oracle inspector
* WIP
* before golden file creation
* fix(ingestion): ingest tables from dba_tables in oracle source
* code review comments
* address review comments
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2022-08-24 09:48:08 -07:00
Gabe Lyons
d9b2a45c5f
feat(looker): index looker charts and dashboards by business term ( #5649 )
2022-08-24 08:01:56 -07:00
Ravindra Lanka
2a56d2c276
fix(ci): ingest - fix nifi integration tests ( #5718 )
2022-08-23 21:46:01 -07:00
Peter Szalai
150a36e236
feat(ingest): presto-on-hive - Add mode option to presto-on-hive source ( #5659 )
2022-08-23 23:18:27 +02:00
Aditya Radhakrishnan
4e2bfdd236
fix(ingest): emit status aspect for entities ingested from okta and azure_ad ( #5700 )
2022-08-22 13:28:35 -07:00
Shirshanka Das
bb788ac317
feat(ingest): file - add support for folders, large files, improve co… ( #5692 )
2022-08-21 14:18:22 +05:30
Ravindra Lanka
228f3b50ea
feat(ingestion): send reports of ingestion runs to datahub ( #5639 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-19 09:08:17 -07:00
Kwanyoung Son
9143663f1f
fix(ingest): redash - fix redash dashboard url bug ( #5500 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-08-16 13:39:27 -07:00
Aezo
8d6a9e08c9
feat(ingest): GE profiler Trino support ( #5361 )
...
* feat(ingest): GE Profile/Action Trino support
* bumping GE version
* updated golden test files
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-08-16 09:46:52 -07:00
Harshal Sheth
680c4a8f81
chore(ingest): drop data-lake source in favor of s3 source ( #5628 )
2022-08-15 22:30:40 -07:00
Mugdha Hardikar
a449e8ba7d
feat(mssql): add multi database ingest support ( #5516 )
...
* feat(mssql): add multi database ingest support
* Delete older golden file.
* Update s3.md
* fix test setup
2022-08-15 21:37:47 -07:00
Mayuri Nehate
dc08bedd6e
feat(ingest): snowflake - add snowflake-beta connector ( #5517 )
2022-08-15 20:54:02 -07:00
Amanda Hernando
337087cac0
feat(ingest): glue - add stateful ingestion ( #5553 )
2022-08-15 20:50:45 -07:00
Aditya Radhakrishnan
b042ab1ab0
fix(ingest): fix indenting issue in azure ad connector ( #5627 )
2022-08-11 14:10:28 -07:00
Harshal Sheth
355c129c7c
chore(ingest): drop python 3.6 support ( #5521 )
2022-08-10 15:00:31 -07:00
