Harshal Sheth
4dded454ff
fix(ingest): cleanup config extra usage ( #6699 )
2022-12-08 16:34:34 -08:00
Felix Lüdin
e7acc8ef30
fix(config): unify the handling of boolean environment variables ( #6684 )
2022-12-08 15:00:09 -08:00
Harshal Sheth
acc79d7d0d
fix(ingest/tableau): support ssl_verify flag properly ( #6682 )
2022-12-08 14:58:31 -08:00
Tamas Nemeth
729e486b62
feat(ingest): bigquery - option to set on behalf project ( #6660 )
2022-12-08 15:25:22 -05:00
orlandine
b219f0848a
docs(ingest/salesforce): list required permissions ( #6610 )
2022-12-08 14:50:15 -05:00
Felix Lüdin
05e18a0ae7
feat(ingest): use entry point for registering transformers ( #6628 )
2022-12-07 23:08:08 -05:00
Mayuri Nehate
9e3267a0ec
feat(ingest): add timestamps for snowflake objects ( #6570 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-07 18:11:08 -05:00
İnanç Dokurel
996dabfcac
fix(ingestion/vertica): support columns with timestamp precision ( #6295 )
...
Co-authored-by: İnanç Dokurel <inancdokurel@users.noreply.github.com>
Fixes https://github.com/datahub-project/datahub/issues/5295
2022-12-07 18:10:37 -05:00
mohdsiddique
c4dcd268a6
feat(ingest): support knowledge links in business glossary ( #6375 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-07 18:09:50 -05:00
Harshal Sheth
bf307a4bcf
feat(ingest): run profiler in more cardinality cases ( #6397 )
2022-12-07 12:20:06 -05:00
Mayuri Nehate
eeb7a9dfe5
feat(ingest): snowflake - update snowflake docs, add simple validations ( #6636 )
2022-12-07 14:56:03 +01:00
Tamas Nemeth
9a1f78fc60
fix(ingest): profiling - Changing profiling defaults on Bigquery and Snowflake ( #6640 )
2022-12-07 10:33:10 +01:00
David Haglund
1a6677083e
fix(ingest/powerbi-report-server): deprecate unused graphql config ( #6630 )
2022-12-07 01:03:49 -05:00
Matthieu Blais
4e2dde84f6
feat(ingest/dbt): add support for latest DBT version 1.3 ( #6651 )
...
Co-authored-by: Matthieu Blais <matthieu.blais@tech.jago.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-06 19:03:24 -05:00
Harshal Sheth
c8969d9ba8
fix(ingest/snowflake): support domains for snowflake schema containers ( #6662 )
2022-12-06 14:24:07 -08:00
Harshal Sheth
fceef480a2
chore(ingest): remove feast-legacy ( #6661 )
2022-12-06 14:19:38 -08:00
Harshal Sheth
f0206baa8b
fix(ingest): issue warning correctly ( #6623 )
2022-12-06 14:17:14 -08:00
Tamas Nemeth
2373c707b8
feat(ingest): bigquery - Running lineage extraction after metadata extraction ( #6653 )
...
* Running lineage extraction after metadata extraction
Adding table creation/alter time to the datasetproperties
Fixing bigquery permissions doc
* Disabling by default to run sql parser in a separate process
Fixing adding views to the global view list
2022-12-06 23:04:27 +01:00
Harshal Sheth
71bfa98f89
fix(ingest): fix lingering demo-data source issues ( #6659 )
2022-12-06 16:10:21 -05:00
Aseem Bansal
43c566ee4f
feat(ingest): add dummy data source for automated testing ( #6550 )
2022-12-06 16:57:12 +05:30
Fredrik Sannholm
4dd66be654
feat(ingest/kafka-connect): support MongoSourceConnector ( #6416 )
...
Co-authored-by: John Joyce <john@acryl.io>
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-05 16:09:58 -05:00
Mayuri Nehate
e5a823e0d8
feat(ingest/snowflake): support filtering by fully qualified schema_pattern ( #6611 )
2022-12-05 14:27:25 -05:00
Mayuri Nehate
fdcb731e29
feat(ingest): snowflake - config variable for specifying a direct private key ( #6609 )
2022-12-05 19:09:08 +05:30
david-leifker
2de9d3d5bf
fix(logging): Remove lombok as source of slf4j-api, convert to compileOnly where possible ( #6616 )
2022-12-04 19:57:47 -08:00
djordje-mijatovic
99e6f3a87c
feat(ingest): print detailed GMS error messages ( #6519 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-02 18:20:09 -05:00
Harshal Sheth
a1e62c723e
docs(ingest): add airflow docs that use the PythonVirtualenvOperator ( #6604 )
2022-12-02 19:56:17 +01:00
Harshal Sheth
71466aab36
fix(ingest): only require github_info for lookml and not looker ( #6608 )
2022-12-02 19:54:24 +01:00
Harshal Sheth
44cfd21a65
chore(ingest): bump and pin mypy ( #6584 )
2022-12-02 19:53:28 +01:00
Mayuri Nehate
1689212434
feat(ingest): add external url for snowflake objects ( #6580 )
2022-12-02 13:38:46 -05:00
Tamas Nemeth
43775ecd49
fix(ingest/bigquery): ignore complex types from profiling ( #6613 )
2022-12-02 13:26:53 -05:00
Harshal Sheth
308b4eae87
fix(ingest): clarify tableau auth error messages ( #6600 )
2022-12-01 19:33:10 -08:00
Harshal Sheth
d6dd8ccc51
fix(ingest): unify emit interface ( #6592 )
2022-12-01 23:02:50 +01:00
Harshal Sheth
6fe9ad4fbb
feat(ingest/bigquery): avoid creating/deleting tables for profiling ( #6578 )
2022-12-01 14:05:29 -05:00
Mayuri Nehate
f63c3e5222
fix(ingest): restrict snowflake-connector-python dependency ( #6594 )
2022-12-01 10:33:03 +01:00
Tamas Nemeth
72b95f2957
fix(ingest): profiling - Profiling failed if column cardinality threw an error #6582
2022-12-01 07:37:22 +01:00
Harshal Sheth
1366724097
fix(ingest): restrict snowflake's sqlalchemy dep ( #6579 )
2022-11-30 08:14:45 +01:00
Aseem Bansal
329ecb8958
feat(cli): remove inconsistency check command ( #6569 )
2022-11-29 13:23:21 -08:00
Bumsoo Kim
6dd6bfc795
refactor(airflow): remove verbose log from airflow plugin ( #6516 )
...
Co-authored-by: John Joyce <john@acryl.io>
2022-11-29 14:08:07 -05:00
Mayuri Nehate
fb2ffe459b
fix(ingest): clickhouse - fix types changes in clickhouse sqlalchemy 0.2.3 ( #6572 )
2022-11-29 16:00:45 +01:00
Mayuri Nehate
ec056211a8
fix(ingest): snowflake - graceful error handling in snowflake classification ( #6568 )
2022-11-29 12:24:24 +01:00
Harshal Sheth
7f93ee5f13
fix(ingest): set DataProcessInstance created ts to start time ( #6566 )
2022-11-28 20:26:40 -08:00
Mert Tunç
536218cb4b
docs(ingest/kafka): add field descriptions of kafka-related configs to pydantic ( #6559 )
2022-11-28 17:37:04 -05:00
Harshal Sheth
880d04246d
fix(ingest): handle docker-compose version v prefix ( #6561 )
2022-11-28 16:55:15 -05:00
Tamas Nemeth
28a61bc9f9
fix(ingest): bigquery - setting partition id for profiling data ( #6558 )
2022-11-28 18:18:51 +01:00
Teppo Naakka
87312f85f5
feat(ingest): powerbi - scan all accessible workspaces ( #6441 )
2022-11-28 17:17:15 +01:00
Tamas Nemeth
278c38cae4
fix(ingest): bigquery - Fixing querying non-date partition columns in profiling ( #6554 )
2022-11-26 18:48:33 +01:00
Tamas Nemeth
d424edde41
fix(ingest): bigquery - missing sqlalchemy dep and row count fix ( #6553 )
2022-11-25 22:33:14 +01:00
Mayuri Nehate
7a8e36d57d
feat(ingest): refactor classification mixin interface, support new info types ( #6545 )
2022-11-25 18:48:42 +05:30
Mayuri Nehate
a12f5daaf4
style(ingest): fix lint checks for superset ( #6548 )
2022-11-24 21:33:57 +05:30
Harshal Sheth
ce3f663a57
build(ingest): support flake8 6.0.0 ( #6540 )
2022-11-23 17:40:55 -05:00
