Dago Romer
9cb1eed6e7
fix(ingest): fixed snowflake oauth ingestion not using role attribute from recipe ( #6825 )
2022-12-21 07:52:06 -08:00
Harshal Sheth
e2b4a65a8e
refactor(ingest): clean up exception types ( #6818 )
2022-12-21 07:28:18 -08:00
Aseem Bansal
2fb0b1ecfb
fix(ci): reduce flakiness views select test ( #6821 )
2022-12-21 20:53:58 +05:30
Harshal Sheth
8972ea4b04
fix(ingest): support patches in auto_status_aspect ( #6827 )
...
Patches generate a raw MCP because MCPW doesn't support patches right now, so we need to handle that correctly downstream.
2022-12-21 10:25:24 +01:00
Tamas Nemeth
a1970d2dce
feat(ingest/bigquery): add option to enable/disable legacy sharded table support ( #6822 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: John Joyce <john@acryl.io>
2022-12-20 23:29:46 -05:00
danielli-ziprecruiter
ca6c3b949b
fix(ingest): exclude ztsd from uber jar to prevent jni conflicts with spark ( #6787 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-20 21:54:56 -06:00
Harshal Sheth
2c911ccf7b
refactor(ingest): clean up pipeline init error handling ( #6817 )
2022-12-20 19:21:28 -08:00
Harshal Sheth
88e40a9069
feat(ingest): add failure/warning counts to ingest_stats ( #6823 )
2022-12-20 19:13:11 -08:00
djordje-mijatovic
e6c48e5f19
feat(kafka): expose default kafka producer mechanism ( #6381 )
...
* Expose Kafka Sender Retry Parameters
* Implement KafkaHealthChecker
* feat(kafka): expose default kafka producer mechanism
2022-12-20 14:41:24 -06:00
John Joyce
b58021ab0a
feat(ui): Adding DBT Cloud support for UI ingestion ( #6804 )
2022-12-20 09:22:03 -08:00
Harshal Sheth
137f4500b6
feat(ingest/stateful): remove platform_instance_id from state urn ( #6795 )
2022-12-20 12:12:19 -05:00
Aditya Radhakrishnan
4a0f798ab5
fix(ui): change caching to happen post server-response when creating a UI ingestion recipe ( #6815 )
2022-12-20 07:57:24 -08:00
Chris Collins
f559dcc282
feat(lineage) Adds ability to edit lineage manually from the UI ( #6816 )
2022-12-20 10:54:08 -05:00
Aditya Radhakrishnan
2475d2bbec
fix(policies): resolve the associated domain for a domain as the domain itself ( #6812 )
2022-12-20 07:13:30 -08:00
Aditya Radhakrishnan
80041d7586
fix(posts): add deletePost GraphQL endpoint ( #6813 )
2022-12-20 07:08:34 -08:00
Aditya Radhakrishnan
cd937e0c67
fix(ui): URL encode urns for ownership entity links ( #6814 )
2022-12-20 07:04:56 -08:00
Harshal Sheth
5584bfb469
refactor(ingest/stateful): remove get_last_state method ( #6794 )
2022-12-19 20:48:22 -05:00
raysaka
fcb3242983
chore(ingest): bump python package dependencies to resolve vulns ( #6384 )
...
Co-authored-by: John Joyce <john@acryl.io>
2022-12-19 18:12:56 -05:00
Harshal Sheth
e9d50ed992
refactor(ingest/stateful): remove IngestionJobStateProvider ( #6792 )
2022-12-19 17:03:54 -05:00
Navin Sharma
7c3872a2ed
Update Grafana Dashboard ( #6076 )
...
* Add Datasource as variable in dashboard
(cherry picked from commit e75b3f7333dd9436f7434eefa76ab9fbfb1babab)
* Update datahub_dashboard.json
(cherry picked from commit 701592697702a99ddf33f75d57826282df999c82)
* Bump docker compose version to 3.8
(cherry picked from commit ff6a97b1692ac21368eacc83a0daed30b0364d5d)
* Update grafana image tag from latest to 9.1.4
(cherry picked from commit 2c88e2a3041b9ddfa70bb92d13e80ca15be0ead2)
* Update old metric name in datahub_dashboard.json
(cherry picked from commit 21b502e25392f37c5053dcf00161fc5c7d35c94d)
* Add panel for new metrics
(cherry picked from commit 194452778bc825820dda87ab2e159b7064a360ef)
Co-authored-by: Peter Szalai <szalaipeti.vagyok@gmail.com>
2022-12-19 22:43:47 +01:00
Monica Senapati
5c366205f5
fix(bigquery-legacy): Fix for TypeError related failures in legacy plugin ( #6806 )
...
Co-authored-by: John Joyce <john@acryl.io>
2022-12-19 13:28:25 -08:00
Harshal Sheth
47be95689e
refactor(ingest/stateful): remove most remaining state classes ( #6791 )
2022-12-19 13:40:48 -05:00
Harshal Sheth
14a00f4098
chore(ingest): pin black version ( #6807 )
2022-12-19 19:35:49 +01:00
Tamas Nemeth
e41b455e14
fix(ingest): bigquery - sharded table support improvements ( #6789 )
2022-12-19 18:57:37 +01:00
John Joyce
807b50699d
refactor(ui): Miscellaneous caching improvements ( #6796 )
2022-12-19 09:55:56 -08:00
Harshal Sheth
54e04ba436
fix(ingest/dbt): remove unsupported usage indicator ( #6805 )
2022-12-19 09:34:49 -08:00
jx2lee
cdedc19c70
fix(docs): edit text to link ( #6798 )
2022-12-19 08:45:22 -08:00
danielli-ziprecruiter
7ca21a8e7c
fix(ingest): prevent NullPointerException when non-jdbc SaveIntoDataSourceCommand ( #6803 )
2022-12-19 08:41:06 -08:00
John Joyce
6f71c957d5
fix(ui): Fixing minor issues with Ingestion forms ( #6790 )
2022-12-19 07:50:14 -08:00
david-leifker
637b540fa6
fix(kafka-setup): fix return code check ( #6782 )
...
* fix(kafka-setup): parallelize topic creation
* Remove -setup from docker compose (not services)
2022-12-16 13:39:18 -06:00
Aditya Radhakrishnan
d48e44499e
fix(tutorial): skip getting steps if there is no user ( #6786 )
2022-12-16 11:18:43 -08:00
Mayuri Nehate
9716a49067
fix(ingest): correct external url for account identifier with account name ( #6715 )
2022-12-16 14:00:42 -05:00
Harshal Sheth
5c8a3659db
fix(smoke-test): remove stateful ingestion config check ( #6781 )
2022-12-16 13:04:48 -05:00
Shirshanka Das
b236d0958c
feat(ci): adding github security reporting to trivy scans ( #6773 )
2022-12-15 21:28:07 -08:00
Harshal Sheth
22081f5ecc
feat(ingest): lookml - add unreachable views to report ( #6779 )
2022-12-15 20:26:30 -08:00
david-leifker
cca3639848
Optimize kafka setup ( #6778 )
...
* fix(kafka-setup): parallelize topic creation
2022-12-15 20:09:29 -06:00
Maggie Hays
2d0188c7ed
docs(snowflake) Snowflake quick ingestion guide ( #6750 )
2022-12-15 17:01:38 -08:00
John Joyce
0215245aa3
refactor(ui): Caching Ingestion Secrets ( #6772 )
2022-12-15 16:37:07 -08:00
Harshal Sheth
8a537b0559
feat(ingest): add datahub state inspect command ( #6763 )
2022-12-15 18:55:36 -05:00
Harshal Sheth
798d82fe60
docs(ingest): fix error in custom tags transformer example ( #6767 )
2022-12-15 15:31:12 -08:00
Aseem Bansal
1350bb010c
docs(docker): add section for adding community images ( #6770 )
2022-12-15 15:26:31 -08:00
Tamas Nemeth
b7bc1e9116
fix(ingest): bigquery - handling custom sql errors as warning ( #6777 )
2022-12-15 23:40:32 +01:00
Harshal Sheth
6152b5e9f7
feat(ingest): simplify more stateful ingestion state ( #6762 )
2022-12-15 11:33:29 -05:00
Harshal Sheth
7d7da15742
docs(ingest/airflow): clarify Airflow 1.x docs for airflow plugin ( #6761 )
2022-12-14 23:48:06 +01:00
John Joyce
0e614d110b
fix(): Fix the datahub-web-react build ( #6764 )
2022-12-14 12:58:54 -08:00
Shirshanka Das
db182e4639
fix(python-sdk): DataHubGraph get_aspect should accept empty responses ( #6760 )
2022-12-14 10:40:16 -08:00
Harshal Sheth
2f95719dba
feat(ingest): remove source config from DatahubIngestionCheckpoint ( #6722 )
2022-12-14 12:39:21 -05:00
jakobhanna
8109b8b567
docs(): Added build failed yarn error ( #6757 )
...
docs: add build failed yarn error message and how to deal with it
I encountere this error and with the help of the community i could deal with it (https://datahubspace.slack.com/archives/CV2UVAPPG/p1670608619404699 ).
2022-12-14 09:30:45 -08:00
John Joyce
fd6d12f84a
refactor(ux): Misc UX Improvements (tutorial copy, caching, filters) ( #6743 )
2022-12-14 09:11:51 -08:00
Patrick Franco Braz
f0a371941e
refactor(ingest): bigquery-lineage - allow tables and datasets in uppercase ( #6739 )
2022-12-14 14:58:03 +01:00
