1604 Commits

Author SHA1 Message Date
Harshal Sheth
b4687ffceb
feat(ingest): drop plugin support for airflow 1.x (#6331) 2022-11-01 21:12:34 -07:00
Oleksandr
0a0cf70b6f
feat(ingest): power-bi - add support for on-prem PowerBI Report Server (#5369) 2022-11-01 16:05:42 -07:00
Tamas Nemeth
0dc2d6af25
feat(ingest): databricks - Unity catalog source (#6292)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-29 21:44:41 -07:00
Harshal Sheth
ef824bd082
feat(ingest): add fallthrough support to KeyValuePattern (#6302) 2022-10-28 11:07:47 +02:00
Mayuri Nehate
29b99052d7
feat(ingest): tableau - add support for column level lineage (#6264) 2022-10-26 19:42:51 -07:00
Mayuri Nehate
dc1e4d70e4
refactor(ingest): snowflake - update field name for column lineage (#6263) 2022-10-26 16:27:27 -07:00
Tamas Nemeth
94fae0a464
feat(ingest): bigquery - Promoting bigquery-beta to bigquery source (#6222)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-26 11:15:44 -07:00
BZ
64c28a2316
feat(ingest/kakfa-connect): support generic connectors (#6173)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-25 10:31:31 -07:00
Shirshanka Das
c5fc2ea798
fix(ingest): looker - deps, column level lineage fixes (#6271) 2022-10-24 08:31:48 +02:00
Shirshanka Das
26b4a9eec4
fix(ingest):lookml - better column-level lineage, hive urn generation… (#6254) 2022-10-20 16:39:11 -07:00
Tamas Nemeth
0545f3cb38
fix(ingest): presto-on-hive - Adding catalog name to the presto on hive urn (#6024)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-19 09:28:33 -07:00
Tamas Nemeth
9015a43f25
fix(ingest): bigquery-beta - Adding python 3.8 fix for memory footprint util (#6228) 2022-10-18 17:59:31 -07:00
Harshal Sheth
73fd35888b
build(ingest): remove markupsafe dep and bump pytest-docker (#6201) 2022-10-14 18:59:40 -07:00
Harshal Sheth
d08f5f7cdd
feat(ingest): replace base85's pickle with json (#6178) 2022-10-14 14:48:44 -07:00
Arne Laponin
bbdc185ac2
feat(ingest): add platform instance to tableau (#5978)
Co-authored-by: Arne Laponin <arne.laponin@thoughtworks.com>
Co-authored-by: Amanda Hernando <110099762+amanda-her@users.noreply.github.com>
Co-authored-by: amanda-her <912amandahernando@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-13 18:44:06 -07:00
Harshal Sheth
521e61d3bf
fix(ingest): Closeable as a context manager (#6067) 2022-10-13 12:49:50 -07:00
Harshal Sheth
8030ac83bc
feat(ingest/tableau): emit status aspects + streamline stateful ingestion (#6188) 2022-10-13 12:48:28 -07:00
Harshal Sheth
09616ee2b3
feat(ingest): include instance in container dataPlatform when provided (#6083) 2022-10-13 11:29:54 -07:00
Tamas Nemeth
6e34cd6001
feat(ingest): bigquery-beta - Parsing view ddl definition for lineage (#6187)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-12 18:24:04 -07:00
Harshal Sheth
c7cc0af6ae
feat(ingest/tableau): support dashboard tags (#6185) 2022-10-12 15:34:35 -07:00
Harshal Sheth
e70c0ac4b6
feat(ingest): include raw s3 paths if s3 source (#6168) 2022-10-11 15:55:00 -07:00
Amanda Hernando
ffa838e00e
feat(ingestion): Tableau stateful ingestion (#6094) 2022-10-11 15:52:15 -07:00
Harshal Sheth
95d1e01195
feat(ingest): infer aspect name from type in get_aspect (#6033) 2022-10-11 13:35:41 -07:00
Harshal Sheth
d569734193
refactor(ingest): Tableau cleanup (#6131)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-10 15:52:34 -07:00
Mayuri Nehate
7b88de89d5
fix(ingest): snowflake - allow profiling to work with geography type (#6162) 2022-10-10 08:05:09 -07:00
Shirshanka Das
e9c4c823d8
fix(ingest): bigquery-beta - ensure that status aspect is emitted for… (#6154)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-08 16:00:45 -07:00
Harshal Sheth
30014072ad
feat(ingest): lookml - add support for local/remote dependencies (#6150) 2022-10-07 13:14:21 -07:00
Mayuri Nehate
b8513ae492
fix(ingest): fetch workbook tags in workbooks graphql query (#6102) 2022-10-07 15:04:03 +02:00
Tamas Nemeth
2f79b50c24
fix(ingest): presto-on-hive - not failing on Hive type parsing error (#6118)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-04 20:54:38 -07:00
Ravindra Lanka
055e4082da
fix(ingestion): fix percent change computation in stale_entity_removal (#6121) 2022-10-04 20:40:59 -07:00
Harshal Sheth
feb9d59b4d
fix(ingest): remove unused mysql golden file (#6106) 2022-10-04 20:26:51 -07:00
Harshal Sheth
3c0f63c50a
fix(ingest): hide deprecated path_spec option from config (#5944) 2022-10-04 12:14:00 -07:00
Tamas Nemeth
a6930562fd
fix(ingest): presto-on-hive - Setting display name properly (#6065) 2022-09-29 16:20:17 +02:00
Shirshanka Das
74d9fa25a6
feat(ingest): looker - add support for simple column level lineage (#6084) 2022-09-28 20:41:30 -07:00
Tamas Nemeth
3b9e9793a7
fix(ingest): bigquery-beta - handling complex types properly (#6062) 2022-09-27 21:31:24 +02:00
Harshal Sheth
59a2228d13
fix(ingest): correct tableau browse paths (#6064) 2022-09-27 12:30:52 -07:00
Harshal Sheth
3f1d47c069
feat(ingest): list referenced env variables in recipe (#6043) 2022-09-26 23:16:18 -07:00
mohdsiddique
c062bf2201
fix(ingestion): Tableau test case execution fix (#6005) 2022-09-26 18:14:42 -07:00
Alexey Kravtsov
3c3ab64954
feat(ingest): implement compression for CheckpointState (#6007) 2022-09-26 10:18:42 -07:00
John Joyce
78454888b5
refactor(ingest): Browse Paths Upgrade Tableau (#6008) 2022-09-23 13:02:34 -07:00
Harshal Sheth
27f28019de
refactor(ingest): move common host_port validation (#6009) 2022-09-22 16:32:07 -07:00
Ravindra Lanka
b8941ab190
feat(ingestion): Add fail-safe stale entity removal via configurable 'fail_safe_threshold' param. (#6027) 2022-09-22 16:09:22 -07:00
Harshal Sheth
68db859ca1
refactor(ingest): streamline two-tier db config validation (#5986) 2022-09-21 10:45:37 -07:00
mohdsiddique
44a67b52cf
feat(ingestion): Tableau dashboard & chart usage stats (#5999)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-20 17:41:56 -07:00
Mayuri Nehate
b195b6c123
fix(ingest): encode reserved characters when creating dataset urn (#5977)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-20 16:59:02 -07:00
Tamas Nemeth
46494bc0ad
feat(ingest): presto-on-hive - Description and hive view support (#5993)
* Presto on Hive Improvements like:
 - Capturing table/column descriptions
 - Adding Hive view support
 - Various bug fixes
2022-09-21 00:40:10 +02:00
Aezo
6c865d00c4
feat(ingest): stateful, catalog and pascalcase con (#5890) 2022-09-20 14:54:22 +02:00
Harshal Sheth
8ffa7054e5
feat(ingest): dbt column-level meta mappings + add_terms operation (#5970)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-19 15:26:12 -07:00
John Joyce
df3633e7f3
refactor(looker): Migrating to new browse path format (#5688) 2022-09-19 14:43:35 -07:00
Harshal Sheth
c8cd7d5040
refactor(ingest): use typed objects in dbt ingestion (#5929) 2022-09-16 15:07:59 -07:00