1646 Commits

Author SHA1 Message Date
cccs-Dustin
2cc64742e0
feat(ingest/iceberg): add stateful ingestion (#6344) 2022-12-12 13:06:03 -05:00
Mayuri Nehate
65ba13d9aa
feat(ingest): snowflake - add separate config for include_column_lineage in snowflake (#6712) 2022-12-12 15:23:12 +01:00
Harshal Sheth
fd911c9820
feat(ingest): redact configs reported in ingestion_run_summary (#6696) 2022-12-12 10:48:26 +01:00
Mayuri Nehate
5c99f20b7d
fix(ingest): mysql - fix mysql ingestion issue with non-lowercase database (#6713) 2022-12-12 10:48:01 +01:00
Harshal Sheth
b7735d5b21
fix(ingest): fix bug in auto_status_aspect (#6705)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-09 12:24:39 -05:00
Harshal Sheth
4dded454ff
fix(ingest): cleanup config extra usage (#6699) 2022-12-08 16:34:34 -08:00
Harshal Sheth
acc79d7d0d
fix(ingest/tableau): support ssl_verify flag properly (#6682) 2022-12-08 14:58:31 -08:00
Mayuri Nehate
9e3267a0ec
feat(ingest): add timestamps for snowflake objects (#6570)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-07 18:11:08 -05:00
mohdsiddique
c4dcd268a6
feat(ingest): support knowledge links in business glossary (#6375)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-07 18:09:50 -05:00
Harshal Sheth
bf307a4bcf
feat(ingest): run profiler in more cardinality cases (#6397) 2022-12-07 12:20:06 -05:00
Mayuri Nehate
eeb7a9dfe5
feat(ingest): snowflake - update snowflake docs, add simple validations (#6636) 2022-12-07 14:56:03 +01:00
Tamas Nemeth
9a1f78fc60
fix(ingest): profiling - Changing profiling defaults on Bigquery and Snowflake (#6640) 2022-12-07 10:33:10 +01:00
David Haglund
1a6677083e
fix(ingest/powerbi-report-server): deprecate unused graphql config (#6630) 2022-12-07 01:03:49 -05:00
Matthieu Blais
4e2dde84f6
feat(ingest/dbt): add support for latest DBT version 1.3 (#6651)
Co-authored-by: Matthieu Blais <matthieu.blais@tech.jago.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-06 19:03:24 -05:00
Harshal Sheth
fceef480a2
chore(ingest): remove feast-legacy (#6661) 2022-12-06 14:19:38 -08:00
Fredrik Sannholm
4dd66be654
feat(ingest/kafka-connect): support MongoSourceConnector (#6416)
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-05 16:09:58 -05:00
Mayuri Nehate
e5a823e0d8
feat(ingest/snowflake): support filtering by fully qualified schema_pattern (#6611) 2022-12-05 14:27:25 -05:00
Harshal Sheth
a1e62c723e
docs(ingest): add airflow docs that use the PythonVirtualenvOperator (#6604) 2022-12-02 19:56:17 +01:00
Harshal Sheth
44cfd21a65
chore(ingest): bump and pin mypy (#6584) 2022-12-02 19:53:28 +01:00
Mayuri Nehate
1689212434
feat(ingest): add external url for snowflake objects (#6580) 2022-12-02 13:38:46 -05:00
Teppo Naakka
87312f85f5
feat(ingest): powerbi - scan all accessible workspaces (#6441) 2022-11-28 17:17:15 +01:00
Tamas Nemeth
8d525d67a9
fix(ingest): kafka - properly picking doc from union type (#6472) 2022-11-23 20:42:21 +01:00
Mayuri Nehate
37e95c9f91
fix(ingest): fix trino properties and tests (#6518) 2022-11-23 01:10:07 -05:00
Mayuri Nehate
22847a987a
feat(ingest): automated term classification for snowflake (#6376) 2022-11-23 00:43:30 -05:00
Mayuri Nehate
a1b3929b38
fix(ingest): fix typo in unique count profiling (#6517) 2022-11-22 14:23:01 -05:00
Mayuri Nehate
e085a9e7dc
feat(ingest): add config for ingesting delta table without files (#6403)
Closes undefined
2022-11-22 14:15:40 -05:00
Harshal Sheth
74cc88f2df
fix(ingest): correctly handle transformer patch semantics (#6505) 2022-11-22 09:29:57 -08:00
Harshal Sheth
05a0f3e2a6
feat(ingest): dbt cloud integration (#6323) 2022-11-21 14:14:33 -05:00
Mayuri Nehate
9c1577da13
feat(ingest): add stateful ingestion support to looker and lookml source (#6443) 2022-11-21 13:44:34 -05:00
Harshal Sheth
d3174e75b7
fix(ingest): dbt - disable incremental lineage by default (#6467) 2022-11-17 09:41:43 +01:00
mohdsiddique
73bf7a81a9
feat(ingestion): powerbi - support for report (#6339)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
2022-11-16 22:52:23 -08:00
Harshal Sheth
29c6c43cb3
feat(ingest): detect old or missing docker compose (#6466) 2022-11-16 22:38:00 -08:00
Mayuri Nehate
198c86a620
feat(ingest): support incremental lineage to dbt node from external platform (#6392) 2022-11-14 13:16:38 -05:00
Mayuri Nehate
443d15f346
fix(ingest): mysql - fix native data type for mysql set type (#6407) 2022-11-14 14:33:26 +01:00
Mayuri Nehate
e79b4e8c2b
feat(ingest): s3 - add status aspect for detected s3 datasets (#6402) 2022-11-13 17:29:42 -08:00
Harshal Sheth
3e907ab0d1
feat(ingest): loosen sqlalchemy dep & support airflow 2.3+ (#6204)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-11 15:04:36 -05:00
Mayuri Nehate
6c42064332
fix(ingest): snowflake - fix missing snowflake lineage when table_pattern is set (#6410) 2022-11-11 16:47:09 +01:00
Harshal Sheth
ae2ea524a4
fix(ingest): use profiler config settings correctly (#6354)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-09 19:03:22 -08:00
Harshal Sheth
1a81d8de6a
feat(ingest): supports MCEs in domain transformer (#6364) 2022-11-05 11:41:43 -07:00
cburroughs
39c84c2f5b
feat(ingest): upgrade feast (#6186)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-11-03 11:23:19 -07:00
Harshal Sheth
0ca3383d30
feat(ingest): support reserved keywords in model codegen (#6351) 2022-11-02 22:26:56 -07:00
Mayuri Nehate
c01aa53aa7
fix(ingest): fix schema field urns with empty field path (#6338) 2022-11-02 15:02:13 +01:00
Harshal Sheth
b4687ffceb
feat(ingest): drop plugin support for airflow 1.x (#6331) 2022-11-01 21:12:34 -07:00
Oleksandr
0a0cf70b6f
feat(ingest): power-bi - add support for on-prem PowerBI Report Server (#5369) 2022-11-01 16:05:42 -07:00
Tamas Nemeth
0dc2d6af25
feat(ingest): databricks - Unity catalog source (#6292)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-29 21:44:41 -07:00
Harshal Sheth
ef824bd082
feat(ingest): add fallthrough support to KeyValuePattern (#6302) 2022-10-28 11:07:47 +02:00
Mayuri Nehate
29b99052d7
feat(ingest): tableau - add support for column level lineage (#6264) 2022-10-26 19:42:51 -07:00
Mayuri Nehate
dc1e4d70e4
refactor(ingest): snowflake - update field name for column lineage (#6263) 2022-10-26 16:27:27 -07:00
Tamas Nemeth
94fae0a464
feat(ingest): bigquery - Promoting bigquery-beta to bigquery source (#6222)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-26 11:15:44 -07:00
BZ
64c28a2316
feat(ingest/kakfa-connect): support generic connectors (#6173)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-10-25 10:31:31 -07:00