927 Commits

Author SHA1 Message Date
Mayuri Nehate
9c1577da13
feat(ingest): add stateful ingestion support to looker and lookml source (#6443) 2022-11-21 13:44:34 -05:00
Harshal Sheth
d3174e75b7
fix(ingest): dbt - disable incremental lineage by default (#6467) 2022-11-17 09:41:43 +01:00
mohdsiddique
73bf7a81a9
feat(ingestion): powerbi - support for report (#6339)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2022-11-16 22:52:23 -08:00
Harshal Sheth
29c6c43cb3
feat(ingest): detect old or missing docker compose (#6466) 2022-11-16 22:38:00 -08:00
Mayuri Nehate
198c86a620
feat(ingest): support incremental lineage to dbt node from external platform (#6392) 2022-11-14 13:16:38 -05:00
Mayuri Nehate
443d15f346
fix(ingest): mysql - fix native data type for mysql set type (#6407) 2022-11-14 14:33:26 +01:00
Mayuri Nehate
e79b4e8c2b
feat(ingest): s3 - add status aspect for detected s3 datasets (#6402) 2022-11-13 17:29:42 -08:00
Mayuri Nehate
6c42064332
fix(ingest): snowflake - fix missing snowflake lineage when table_pattern is set (#6410) 2022-11-11 16:47:09 +01:00
Harshal Sheth
ae2ea524a4
fix(ingest): use profiler config settings correctly (#6354)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-09 19:03:22 -08:00
cburroughs
39c84c2f5b
feat(ingest): upgrade feast (#6186)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-11-03 11:23:19 -07:00
Mayuri Nehate
c01aa53aa7
fix(ingest): fix schema field urns with empty field path (#6338) 2022-11-02 15:02:13 +01:00
Oleksandr
0a0cf70b6f
feat(ingest): power-bi - add support for on-prem PowerBI Report Server (#5369) 2022-11-01 16:05:42 -07:00
Tamas Nemeth
0dc2d6af25
feat(ingest): databricks - Unity catalog source (#6292)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-29 21:44:41 -07:00
Mayuri Nehate
29b99052d7
feat(ingest): tableau - add support for column level lineage (#6264) 2022-10-26 19:42:51 -07:00
Mayuri Nehate
dc1e4d70e4
refactor(ingest): snowflake - update field name for column lineage (#6263) 2022-10-26 16:27:27 -07:00
Tamas Nemeth
94fae0a464
feat(ingest): bigquery - Promoting bigquery-beta to bigquery source (#6222)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-26 11:15:44 -07:00
BZ
64c28a2316
feat(ingest/kakfa-connect): support generic connectors (#6173)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-25 10:31:31 -07:00
Shirshanka Das
c5fc2ea798
fix(ingest): looker - deps, column level lineage fixes (#6271) 2022-10-24 08:31:48 +02:00
Shirshanka Das
26b4a9eec4
fix(ingest):lookml - better column-level lineage, hive urn generation… (#6254) 2022-10-20 16:39:11 -07:00
Tamas Nemeth
0545f3cb38
fix(ingest): presto-on-hive - Adding catalog name to the presto on hive urn (#6024)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-19 09:28:33 -07:00
Arne Laponin
bbdc185ac2
feat(ingest): add platform instance to tableau (#5978)
Co-authored-by: Arne Laponin <arne.laponin@thoughtworks.com>
Co-authored-by: Amanda Hernando <110099762+amanda-her@users.noreply.github.com>
Co-authored-by: amanda-her <912amandahernando@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-13 18:44:06 -07:00
Harshal Sheth
521e61d3bf
fix(ingest): Closeable as a context manager (#6067) 2022-10-13 12:49:50 -07:00
Harshal Sheth
8030ac83bc
feat(ingest/tableau): emit status aspects + streamline stateful ingestion (#6188) 2022-10-13 12:48:28 -07:00
Harshal Sheth
09616ee2b3
feat(ingest): include instance in container dataPlatform when provided (#6083) 2022-10-13 11:29:54 -07:00
Harshal Sheth
c7cc0af6ae
feat(ingest/tableau): support dashboard tags (#6185) 2022-10-12 15:34:35 -07:00
Harshal Sheth
e70c0ac4b6
feat(ingest): include raw s3 paths if s3 source (#6168) 2022-10-11 15:55:00 -07:00
Amanda Hernando
ffa838e00e
feat(ingestion): Tableau stateful ingestion (#6094) 2022-10-11 15:52:15 -07:00
Harshal Sheth
d569734193
refactor(ingest): Tableau cleanup (#6131)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-10 15:52:34 -07:00
Mayuri Nehate
7b88de89d5
fix(ingest): snowflake - allow profiling to work with geography type (#6162) 2022-10-10 08:05:09 -07:00
Shirshanka Das
e9c4c823d8
fix(ingest): bigquery-beta - ensure that status aspect is emitted for… (#6154)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-08 16:00:45 -07:00
Harshal Sheth
30014072ad
feat(ingest): lookml - add support for local/remote dependencies (#6150) 2022-10-07 13:14:21 -07:00
Mayuri Nehate
b8513ae492
fix(ingest): fetch workbook tags in workbooks graphql query (#6102) 2022-10-07 15:04:03 +02:00
Harshal Sheth
feb9d59b4d
fix(ingest): remove unused mysql golden file (#6106) 2022-10-04 20:26:51 -07:00
Harshal Sheth
3c0f63c50a
fix(ingest): hide deprecated path_spec option from config (#5944) 2022-10-04 12:14:00 -07:00
Tamas Nemeth
a6930562fd
fix(ingest): presto-on-hive - Setting display name properly (#6065) 2022-09-29 16:20:17 +02:00
Shirshanka Das
74d9fa25a6
feat(ingest): looker - add support for simple column level lineage (#6084) 2022-09-28 20:41:30 -07:00
Harshal Sheth
59a2228d13
fix(ingest): correct tableau browse paths (#6064) 2022-09-27 12:30:52 -07:00
mohdsiddique
c062bf2201
fix(ingestion): Tableau test case execution fix (#6005) 2022-09-26 18:14:42 -07:00
John Joyce
78454888b5
refactor(ingest): Browse Paths Upgrade Tableau (#6008) 2022-09-23 13:02:34 -07:00
Ravindra Lanka
b8941ab190
feat(ingestion): Add fail-safe stale entity removal via configurable 'fail_safe_threshold' param. (#6027) 2022-09-22 16:09:22 -07:00
mohdsiddique
44a67b52cf
feat(ingestion): Tableau dashboard & chart usage stats (#5999)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-20 17:41:56 -07:00
Mayuri Nehate
b195b6c123
fix(ingest): encode reserved characters when creating dataset urn (#5977)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-09-20 16:59:02 -07:00
Tamas Nemeth
46494bc0ad
feat(ingest): presto-on-hive - Description and hive view support (#5993)
* Presto on Hive Improvements like:
 - Capturing table/column descriptions
 - Adding Hive view support
 - Various bug fixes
2022-09-21 00:40:10 +02:00
Aezo
6c865d00c4
feat(ingest): stateful, catalog and pascalcase con (#5890) 2022-09-20 14:54:22 +02:00
Harshal Sheth
8ffa7054e5
feat(ingest): dbt column-level meta mappings + add_terms operation (#5970)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-19 15:26:12 -07:00
John Joyce
df3633e7f3
refactor(looker): Migrating to new browse path format (#5688) 2022-09-19 14:43:35 -07:00
Harshal Sheth
c8cd7d5040
refactor(ingest): use typed objects in dbt ingestion (#5929) 2022-09-16 15:07:59 -07:00
Mayuri Nehate
a14617b6a4
fix(ingest): continue validation of s3 path_specs even if platform is set (#5951) 2022-09-16 12:03:57 -07:00
Mayuri Nehate
2558129391
refactor(snowflake): move snowflake-beta to certified snowflake source (#5923)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-15 09:53:54 -07:00
Mugdha Hardikar
93681d7a71
feat(ingest): delta-lake - support delta tables from minio (#5758)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-15 09:51:14 -07:00