7252 Commits

Author SHA1 Message Date
Dago Romer
9cb1eed6e7
fix(ingest): fixed snowflake oauth ingestion not using role attribute from recipe (#6825) 2022-12-21 07:52:06 -08:00
Harshal Sheth
e2b4a65a8e
refactor(ingest): clean up exception types (#6818) 2022-12-21 07:28:18 -08:00
Aseem Bansal
2fb0b1ecfb
fix(ci): reduce flakiness views select test (#6821) 2022-12-21 20:53:58 +05:30
Harshal Sheth
8972ea4b04
fix(ingest): support patches in auto_status_aspect (#6827)
Patches generate a raw MCP because MCPW doesn't support patches right now, so we need to handle that correctly downstream.
2022-12-21 10:25:24 +01:00
Tamas Nemeth
a1970d2dce
feat(ingest/bigquery): add option to enable/disable legacy sharded table support (#6822)
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: John Joyce <john@acryl.io>
2022-12-20 23:29:46 -05:00
danielli-ziprecruiter
ca6c3b949b
fix(ingest): exclude ztsd from uber jar to prevent jni conflicts with spark (#6787)
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-20 21:54:56 -06:00
Harshal Sheth
2c911ccf7b
refactor(ingest): clean up pipeline init error handling (#6817) 2022-12-20 19:21:28 -08:00
Harshal Sheth
88e40a9069
feat(ingest): add failure/warning counts to ingest_stats (#6823) 2022-12-20 19:13:11 -08:00
djordje-mijatovic
e6c48e5f19
feat(kafka): expose default kafka producer mechanism (#6381)
* Expose Kafka Sender Retry Parameters

* Implement KafkaHealthChecker

* feat(kafka): expose default kafka producer mechanism
v0.9.4
2022-12-20 14:41:24 -06:00
John Joyce
b58021ab0a
feat(ui): Adding DBT Cloud support for UI ingestion (#6804) 2022-12-20 09:22:03 -08:00
Harshal Sheth
137f4500b6
feat(ingest/stateful): remove platform_instance_id from state urn (#6795) 2022-12-20 12:12:19 -05:00
Aditya Radhakrishnan
4a0f798ab5
fix(ui): change caching to happen post server-response when creating a UI ingestion recipe (#6815) 2022-12-20 07:57:24 -08:00
Chris Collins
f559dcc282
feat(lineage) Adds ability to edit lineage manually from the UI (#6816) 2022-12-20 10:54:08 -05:00
Aditya Radhakrishnan
2475d2bbec
fix(policies): resolve the associated domain for a domain as the domain itself (#6812) 2022-12-20 07:13:30 -08:00
Aditya Radhakrishnan
80041d7586
fix(posts): add deletePost GraphQL endpoint (#6813) 2022-12-20 07:08:34 -08:00
Aditya Radhakrishnan
cd937e0c67
fix(ui): URL encode urns for ownership entity links (#6814) 2022-12-20 07:04:56 -08:00
Harshal Sheth
5584bfb469
refactor(ingest/stateful): remove get_last_state method (#6794) 2022-12-19 20:48:22 -05:00
raysaka
fcb3242983
chore(ingest): bump python package dependencies to resolve vulns (#6384)
Co-authored-by: John Joyce <john@acryl.io>
2022-12-19 18:12:56 -05:00
Harshal Sheth
e9d50ed992
refactor(ingest/stateful): remove IngestionJobStateProvider (#6792) 2022-12-19 17:03:54 -05:00
Navin Sharma
7c3872a2ed
Update Grafana Dashboard (#6076)
* Add Datasource as variable in dashboard

(cherry picked from commit e75b3f7333dd9436f7434eefa76ab9fbfb1babab)

* Update datahub_dashboard.json

(cherry picked from commit 701592697702a99ddf33f75d57826282df999c82)

* Bump docker compose version to 3.8

(cherry picked from commit ff6a97b1692ac21368eacc83a0daed30b0364d5d)

* Update grafana image tag from latest to 9.1.4

(cherry picked from commit 2c88e2a3041b9ddfa70bb92d13e80ca15be0ead2)

* Update old metric name in datahub_dashboard.json

(cherry picked from commit 21b502e25392f37c5053dcf00161fc5c7d35c94d)

* Add panel for new metrics

(cherry picked from commit 194452778bc825820dda87ab2e159b7064a360ef)

Co-authored-by: Peter Szalai <szalaipeti.vagyok@gmail.com>
2022-12-19 22:43:47 +01:00
Monica Senapati
5c366205f5
fix(bigquery-legacy): Fix for TypeError related failures in legacy plugin (#6806)
Co-authored-by: John Joyce <john@acryl.io>
2022-12-19 13:28:25 -08:00
Harshal Sheth
47be95689e
refactor(ingest/stateful): remove most remaining state classes (#6791) 2022-12-19 13:40:48 -05:00
Harshal Sheth
14a00f4098
chore(ingest): pin black version (#6807) 2022-12-19 19:35:49 +01:00
Tamas Nemeth
e41b455e14
fix(ingest): bigquery - sharded table support improvements (#6789) 2022-12-19 18:57:37 +01:00
John Joyce
807b50699d
refactor(ui): Miscellaneous caching improvements (#6796) 2022-12-19 09:55:56 -08:00
Harshal Sheth
54e04ba436
fix(ingest/dbt): remove unsupported usage indicator (#6805) 2022-12-19 09:34:49 -08:00
jx2lee
cdedc19c70
fix(docs): edit text to link (#6798) 2022-12-19 08:45:22 -08:00
danielli-ziprecruiter
7ca21a8e7c
fix(ingest): prevent NullPointerException when non-jdbc SaveIntoDataSourceCommand (#6803) 2022-12-19 08:41:06 -08:00
John Joyce
6f71c957d5
fix(ui): Fixing minor issues with Ingestion forms (#6790) 2022-12-19 07:50:14 -08:00
david-leifker
637b540fa6
fix(kafka-setup): fix return code check (#6782)
* fix(kafka-setup): parallelize topic creation
* Remove -setup from docker compose (not services)
2022-12-16 13:39:18 -06:00
Aditya Radhakrishnan
d48e44499e
fix(tutorial): skip getting steps if there is no user (#6786) 2022-12-16 11:18:43 -08:00
Mayuri Nehate
9716a49067
fix(ingest): correct external url for account identifier with account name (#6715) 2022-12-16 14:00:42 -05:00
Harshal Sheth
5c8a3659db
fix(smoke-test): remove stateful ingestion config check (#6781) 2022-12-16 13:04:48 -05:00
Shirshanka Das
b236d0958c
feat(ci): adding github security reporting to trivy scans (#6773) 2022-12-15 21:28:07 -08:00
Harshal Sheth
22081f5ecc
feat(ingest): lookml - add unreachable views to report (#6779) 2022-12-15 20:26:30 -08:00
david-leifker
cca3639848
Optimize kafka setup (#6778)
* fix(kafka-setup): parallelize topic creation
2022-12-15 20:09:29 -06:00
Maggie Hays
2d0188c7ed
docs(snowflake) Snowflake quick ingestion guide (#6750) 2022-12-15 17:01:38 -08:00
John Joyce
0215245aa3
refactor(ui): Caching Ingestion Secrets (#6772) 2022-12-15 16:37:07 -08:00
Harshal Sheth
8a537b0559
feat(ingest): add datahub state inspect command (#6763) 2022-12-15 18:55:36 -05:00
Harshal Sheth
798d82fe60
docs(ingest): fix error in custom tags transformer example (#6767) 2022-12-15 15:31:12 -08:00
Aseem Bansal
1350bb010c
docs(docker): add section for adding community images (#6770) 2022-12-15 15:26:31 -08:00
Tamas Nemeth
b7bc1e9116
fix(ingest): bigquery - handling custom sql errors as warning (#6777) 2022-12-15 23:40:32 +01:00
Harshal Sheth
6152b5e9f7
feat(ingest): simplify more stateful ingestion state (#6762) 2022-12-15 11:33:29 -05:00
Harshal Sheth
7d7da15742
docs(ingest/airflow): clarify Airflow 1.x docs for airflow plugin (#6761) 2022-12-14 23:48:06 +01:00
John Joyce
0e614d110b
fix(): Fix the datahub-web-react build (#6764) 2022-12-14 12:58:54 -08:00
Shirshanka Das
db182e4639
fix(python-sdk): DataHubGraph get_aspect should accept empty responses (#6760) 2022-12-14 10:40:16 -08:00
Harshal Sheth
2f95719dba
feat(ingest): remove source config from DatahubIngestionCheckpoint (#6722) 2022-12-14 12:39:21 -05:00
jakobhanna
8109b8b567
docs(): Added build failed yarn error (#6757)
docs: add build failed yarn error message and how to deal with it 

I encountere this error and with the help of the community i could deal with it (https://datahubspace.slack.com/archives/CV2UVAPPG/p1670608619404699).
2022-12-14 09:30:45 -08:00
John Joyce
fd6d12f84a
refactor(ux): Misc UX Improvements (tutorial copy, caching, filters) (#6743) 2022-12-14 09:11:51 -08:00
Patrick Franco Braz
f0a371941e
refactor(ingest): bigquery-lineage - allow tables and datasets in uppercase (#6739) 2022-12-14 14:58:03 +01:00