Harshal Sheth
1b889022f0
test(ingest/kafka-connect): make docker setup more reliable ( #6902 )
2022-12-30 11:31:33 +01:00
Tamas Nemeth
ead0074169
deprecate(ingest): bigquery - Removing bigquery-legacy source ( #6851 )
...
Co-authored-by: John Joyce <john@acryl.io>
2022-12-29 13:19:05 -08:00
Harshal Sheth
667ca8632d
feat(ingest): avoid embedding serialized json in metadata files ( #6742 )
2022-12-28 19:28:38 -05:00
Mayuri Nehate
2129496c98
feat(ingest/snowflake): handle failures gracefully and raise permission failures ( #6748 )
2022-12-28 08:20:37 -08:00
Mayuri Nehate
69a2347db1
feat(ingest): update profiling to fetch configurable number of sample values ( #6859 )
2022-12-27 08:57:26 +01:00
mohdsiddique
9daa8ed56f
feat(ingestion): Business Glossary# Add domain support in GlossaryTerm ingestion ( #6829 )
...
* lint fix
* domain in term
* domain in term
* review comments
* add todo
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-22 17:47:57 -05:00
Mayuri Nehate
a05c5c4069
feat(ingest): extract kafka topic config properties as customProperties ( #6783 )
2022-12-22 09:34:55 +01:00
John Joyce
2e3a25123d
refactor(ingestion): Browse Paths Upgrade V2 Feast & Sagemaker ( #6002 )
2022-12-21 08:02:59 -08:00
Harshal Sheth
e2b4a65a8e
refactor(ingest): clean up exception types ( #6818 )
2022-12-21 07:28:18 -08:00
Harshal Sheth
47be95689e
refactor(ingest/stateful): remove most remaining state classes ( #6791 )
2022-12-19 13:40:48 -05:00
Tamas Nemeth
e41b455e14
fix(ingest): bigquery - sharded table support improvements ( #6789 )
2022-12-19 18:57:37 +01:00
Mayuri Nehate
9716a49067
fix(ingest): correct external url for account identifier with account name ( #6715 )
2022-12-16 14:00:42 -05:00
Harshal Sheth
8a537b0559
feat(ingest): add datahub state inspect command ( #6763 )
2022-12-15 18:55:36 -05:00
Harshal Sheth
6152b5e9f7
feat(ingest): simplify more stateful ingestion state ( #6762 )
2022-12-15 11:33:29 -05:00
Harshal Sheth
2f95719dba
feat(ingest): remove source config from DatahubIngestionCheckpoint ( #6722 )
2022-12-14 12:39:21 -05:00
Harshal Sheth
68fd802881
fix(ingest/lookml): fix directory handling and a github_info resolution bug ( #6751 )
2022-12-14 14:55:38 +01:00
Harshal Sheth
cf3db168ac
feat(ingest): start simplifying stateful ingestion state ( #6740 )
2022-12-13 10:05:57 +01:00
Dmitry Bryazgin
551ef1b335
feat(ingest): add stateful ingestion to the ldap source ( #6127 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-13 01:13:39 -05:00
Harshal Sheth
85bb1f5030
test(ingest): make hive/trino test more reliable ( #6741 )
2022-12-12 21:02:52 -05:00
cccs-Dustin
2cc64742e0
feat(ingest/iceberg): add stateful ingestion ( #6344 )
2022-12-12 13:06:03 -05:00
Mayuri Nehate
65ba13d9aa
feat(ingest): snowflake - add separate config for include_column_lineage in snowflake ( #6712 )
2022-12-12 15:23:12 +01:00
Mayuri Nehate
5c99f20b7d
fix(ingest): mysql - fix mysql ingestion issue with non-lowercase database ( #6713 )
2022-12-12 10:48:01 +01:00
Harshal Sheth
b7735d5b21
fix(ingest): fix bug in auto_status_aspect ( #6705 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-09 12:24:39 -05:00
Harshal Sheth
acc79d7d0d
fix(ingest/tableau): support ssl_verify flag properly ( #6682 )
2022-12-08 14:58:31 -08:00
Mayuri Nehate
9e3267a0ec
feat(ingest): add timestamps for snowflake objects ( #6570 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-07 18:11:08 -05:00
mohdsiddique
c4dcd268a6
feat(ingest): support knowledge links in business glossary ( #6375 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-07 18:09:50 -05:00
Harshal Sheth
bf307a4bcf
feat(ingest): run profiler in more cardinality cases ( #6397 )
2022-12-07 12:20:06 -05:00
David Haglund
1a6677083e
fix(ingest/powerbi-report-server): deprecate unused graphql config ( #6630 )
2022-12-07 01:03:49 -05:00
Matthieu Blais
4e2dde84f6
feat(ingest/dbt): add support for latest DBT version 1.3 ( #6651 )
...
Co-authored-by: Matthieu Blais <matthieu.blais@tech.jago.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-06 19:03:24 -05:00
Harshal Sheth
fceef480a2
chore(ingest): remove feast-legacy ( #6661 )
2022-12-06 14:19:38 -08:00
Fredrik Sannholm
4dd66be654
feat(ingest/kafka-connect): support MongoSourceConnector ( #6416 )
...
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-05 16:09:58 -05:00
Mayuri Nehate
e5a823e0d8
feat(ingest/snowflake): support filtering by fully qualified schema_pattern ( #6611 )
2022-12-05 14:27:25 -05:00
Mayuri Nehate
1689212434
feat(ingest): add external url for snowflake objects ( #6580 )
2022-12-02 13:38:46 -05:00
Teppo Naakka
87312f85f5
feat(ingest): powerbi - scan all accessible workspaces ( #6441 )
2022-11-28 17:17:15 +01:00
Mayuri Nehate
37e95c9f91
fix(ingest): fix trino properties and tests ( #6518 )
2022-11-23 01:10:07 -05:00
Mayuri Nehate
22847a987a
feat(ingest): automated term classification for snowflake ( #6376 )
2022-11-23 00:43:30 -05:00
Mayuri Nehate
a1b3929b38
fix(ingest): fix typo in unique count profiling ( #6517 )
2022-11-22 14:23:01 -05:00
Mayuri Nehate
e085a9e7dc
feat(ingest): add config for ingesting delta table without files ( #6403 )
...
Closes undefined
2022-11-22 14:15:40 -05:00
Harshal Sheth
05a0f3e2a6
feat(ingest): dbt cloud integration ( #6323 )
2022-11-21 14:14:33 -05:00
Mayuri Nehate
9c1577da13
feat(ingest): add stateful ingestion support to looker and lookml source ( #6443 )
2022-11-21 13:44:34 -05:00
Harshal Sheth
d3174e75b7
fix(ingest): dbt - disable incremental lineage by default ( #6467 )
2022-11-17 09:41:43 +01:00
mohdsiddique
73bf7a81a9
feat(ingestion): powerbi - support for report ( #6339 )
...
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2022-11-16 22:52:23 -08:00
Harshal Sheth
29c6c43cb3
feat(ingest): detect old or missing docker compose ( #6466 )
2022-11-16 22:38:00 -08:00
Mayuri Nehate
198c86a620
feat(ingest): support incremental lineage to dbt node from external platform ( #6392 )
2022-11-14 13:16:38 -05:00
Mayuri Nehate
443d15f346
fix(ingest): mysql - fix native data type for mysql set type ( #6407 )
2022-11-14 14:33:26 +01:00
Mayuri Nehate
e79b4e8c2b
feat(ingest): s3 - add status aspect for detected s3 datasets ( #6402 )
2022-11-13 17:29:42 -08:00
Mayuri Nehate
6c42064332
fix(ingest): snowflake - fix missing snowflake lineage when table_pattern is set ( #6410 )
2022-11-11 16:47:09 +01:00
Harshal Sheth
ae2ea524a4
fix(ingest): use profiler config settings correctly ( #6354 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-09 19:03:22 -08:00
cburroughs
39c84c2f5b
feat(ingest): upgrade feast ( #6186 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-11-03 11:23:19 -07:00
Mayuri Nehate
c01aa53aa7
fix(ingest): fix schema field urns with empty field path ( #6338 )
2022-11-02 15:02:13 +01:00
