3975 Commits

Author SHA1 Message Date
Tamas Nemeth
43775ecd49
fix(ingest/bigquery): ignore complex types from profiling (#6613) 2022-12-02 13:26:53 -05:00
Harshal Sheth
308b4eae87
fix(ingest): clarify tableau auth error messages (#6600) 2022-12-01 19:33:10 -08:00
Harshal Sheth
d6dd8ccc51
fix(ingest): unify emit interface (#6592) 2022-12-01 23:02:50 +01:00
Harshal Sheth
6fe9ad4fbb
feat(ingest/bigquery): avoid creating/deleting tables for profiling (#6578) 2022-12-01 14:05:29 -05:00
Mayuri Nehate
f63c3e5222
fix(ingest): restrict snowflake-connector-python dependency (#6594) 2022-12-01 10:33:03 +01:00
Tamas Nemeth
72b95f2957
fix(ingest): profiling - Profiling failed if column cardinality threw an error #6582 2022-12-01 07:37:22 +01:00
Harshal Sheth
1366724097
fix(ingest): restrict snowflake's sqlalchemy dep (#6579) 2022-11-30 08:14:45 +01:00
Aseem Bansal
329ecb8958
feat(cli): remove inconsistency check command (#6569) 2022-11-29 13:23:21 -08:00
Bumsoo Kim
6dd6bfc795
refactor(airflow): remove verbose log from airflow plugin (#6516)
Co-authored-by: John Joyce <john@acryl.io>
2022-11-29 14:08:07 -05:00
Mayuri Nehate
fb2ffe459b
fix(ingest): clickhouse - fix types changes in clickhouse sqlalchemy 0.2.3 (#6572) 2022-11-29 16:00:45 +01:00
Mayuri Nehate
ec056211a8
fix(ingest): snowflake - graceful error handling in snowflake classification (#6568) 2022-11-29 12:24:24 +01:00
Harshal Sheth
7f93ee5f13
fix(ingest): set DataProcessInstance created ts to start time (#6566) 2022-11-28 20:26:40 -08:00
Mert Tunç
536218cb4b
docs(ingest/kafka): add field descriptions of kafka-related configs to pydantic (#6559) 2022-11-28 17:37:04 -05:00
Harshal Sheth
880d04246d
fix(ingest): handle docker-compose version v prefix (#6561) 2022-11-28 16:55:15 -05:00
Tamas Nemeth
28a61bc9f9
fix(ingest): bigquery - setting partition id for profiling data (#6558) 2022-11-28 18:18:51 +01:00
Teppo Naakka
87312f85f5
feat(ingest): powerbi - scan all accessible workspaces (#6441) 2022-11-28 17:17:15 +01:00
Tamas Nemeth
278c38cae4
fix(ingest): bigquery - Fixing querying non-date partition columns in profiling (#6554) 2022-11-26 18:48:33 +01:00
Tamas Nemeth
d424edde41
fix(ingest): bigquery - missing sqlalchemy dep and row count fix (#6553) 2022-11-25 22:33:14 +01:00
Mayuri Nehate
7a8e36d57d
feat(ingest): refactor classification mixin interface, support new info types (#6545) 2022-11-25 18:48:42 +05:30
Mayuri Nehate
a12f5daaf4
style(ingest): fix lint checks for superset (#6548) 2022-11-24 21:33:57 +05:30
Harshal Sheth
ce3f663a57
build(ingest): support flake8 6.0.0 (#6540) 2022-11-23 17:40:55 -05:00
Phong Vu
0eb54c6387
fix(ingest): handle groupby custom label case (#6456)
fix(ingest): handle groupby custom label case

Co-authored-by: Phong Vu <phongvt@meeyland.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-11-23 14:12:18 -08:00
Harshal Sheth
ebd685d40d
fix(ingest/airflow): respect enabled flag in airflow plugin (#6528) 2022-11-23 11:52:05 -08:00
Harshal Sheth
4db51b4c4f
feat(ingest): disable stateful_ingestion failsafe by default (#6537) 2022-11-23 11:51:56 -08:00
Tamas Nemeth
8d525d67a9
fix(ingest): kafka - properly picking doc from union type (#6472) 2022-11-23 20:42:21 +01:00
Mayuri Nehate
37e95c9f91
fix(ingest): fix trino properties and tests (#6518) 2022-11-23 01:10:07 -05:00
Bumsoo Kim
e41f42b9db
fix(ingest): fix typo in urn utilities (#6520)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-11-23 01:05:32 -05:00
Mayuri Nehate
22847a987a
feat(ingest): automated term classification for snowflake (#6376) 2022-11-23 00:43:30 -05:00
Harshal Sheth
2b9b123a85
fix(ingest): make kafka emit callback optional (#6525) 2022-11-22 16:57:14 -05:00
Mayuri Nehate
a1b3929b38
fix(ingest): fix typo in unique count profiling (#6517) 2022-11-22 14:23:01 -05:00
Mayuri Nehate
e085a9e7dc
feat(ingest): add config for ingesting delta table without files (#6403)
Closes undefined
2022-11-22 14:15:40 -05:00
Harshal Sheth
2cb1e6576d
fix(ingest): add alias for bigquery-beta (#6521) 2022-11-22 14:15:20 -05:00
Harshal Sheth
1229c1f60d
feat(airflow): support kafka sinks in the airflow plugin (#6508) 2022-11-22 10:49:00 -08:00
Harshal Sheth
3fdaf13f2c
feat(ingest/csv-enrich): handle BOM character (#6509)
This has come up a number of times e.g. https://datahubspace.slack.com/archives/C029A3M079U/p1669000226732159?thread_ts=1669000226.732159&amp;cid=C029A3M079U
2022-11-22 12:53:34 -05:00
Harshal Sheth
74cc88f2df
fix(ingest): correctly handle transformer patch semantics (#6505) 2022-11-22 09:29:57 -08:00
Harshal Sheth
10a31b1aa0
fix(ingest/snowflake): fix lineage allow/deny pattern typo (#6506) 2022-11-22 07:09:48 +01:00
Harshal Sheth
490097e532
fix(ingest): remove redundant types (#6486)
Possible since https://github.com/python/typeshed/pull/9220 was merged.

Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-21 23:08:05 +01:00
Harshal Sheth
fdb74b0524
fix(tableau): extra defensive error-handling (#6503) 2022-11-21 14:40:18 -05:00
Harshal Sheth
05a0f3e2a6
feat(ingest): dbt cloud integration (#6323) 2022-11-21 14:14:33 -05:00
Mayuri Nehate
9c1577da13
feat(ingest): add stateful ingestion support to looker and lookml source (#6443) 2022-11-21 13:44:34 -05:00
Lukasz Stefaniak
bc5e2d43af
fix(ingest/tableau): check tableName existence on datasource response (#6478)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-11-21 11:39:19 -05:00
Tamas Nemeth
250f7ce1a8
feat(ingest): presto - Adding presto source (#6459) 2022-11-18 12:02:48 +01:00
Harshal Sheth
0d474a1fc6
docs(ingest/tableau): required permissions + doc formatting (#6484) 2022-11-18 09:31:16 +01:00
Harshal Sheth
817406eadb
refactor(ingest): simplify stateful ingestion config (#6454)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-11-18 00:09:24 -05:00
Harshal Sheth
b7c03731c4
fix(ingest): fix athena and GE lint errors (#6482) 2022-11-17 19:07:05 -05:00
jakobhanna
3daeaccb2b
fix(docs): Ownership Type is deprecated (#6477)
The ownership type is deprecated (described in OwnershipTypeClass), thats why it should be changed from DATAOWNER to TECHNICAL_OWNER.
2022-11-17 12:20:40 -08:00
Aditya Radhakrishnan
738b6a28d0
fix(docs): update the Okta and Azure AD docs to clarify the point of ingesting users (#6465) 2022-11-17 11:09:41 -08:00
Dmytro Kulyk
496f61b608
build: remove Jinja2 dependency from superset (#6476) 2022-11-17 13:46:42 -05:00
Harshal Sheth
b73ebe69b5
revert: mssql - lowercase db name in mssql ingestion (#6481) 2022-11-17 19:42:38 +01:00
Harshal Sheth
94e86e9713
docs(ingest): trino - add example of http connection (#6461) 2022-11-17 16:00:09 +01:00