david-leifker
10ea10ce85
fix(security): require signed/encrypted jwt tokens ( #6565 )
...
* fix(security): require unsigned/encrypted jwt tokens
* Add import
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 19:45:32 +00:00
Aseem Bansal
1bec1d87bd
fix(ci): reduce smoke test run time ( #6841 )
2022-12-27 00:08:01 +05:30
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) ( #6744 )
...
* fix(security): commons-text in frontend
* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint
* testing docker build
* Add retry configuration options for entity client
* Kafka debugging
* fix(kafka-setup): parallelize topic creation
* Adjust docker build
* Docker build updates
* WIP
* fix(lint): metadata-ingestion lint
* fix(gradle-docker): fix docker frontend dep
* fix(elastic): fix race condition between gms and mae for index creation
* Revert "fix(elastic): fix race condition between gms and mae for index creation"
This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.
* fix(test): fix datahub frontend test for clean/test cycle
* fix(test): datahub-frontend missing assets in test
* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer
* gitingore update
* fix(docker): remove platform on docker base image, set by buildx
* refactor(kafka-producer): update kafka producer tracking/logging
* updates per PR feedback
* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
Aseem Bansal
6fdbf6b42d
Revert "fix(ci): remove warnings due to deprecated action ( #6735 )" ( #6857 )
...
This reverts commit 1da27edcbc97d64d5e88321c8727fabf92a576f8.
2022-12-26 17:56:46 +05:30
John Joyce
cc0b6019fb
chore(0.9.5): Bump defaults for release v0.9.5 ( #6856 )
2022-12-26 15:40:26 +05:30
Harshal Sheth
392115b4c4
feat(ingest): add pydantic helper for removed fields ( #6853 )
2022-12-26 15:31:49 +05:30
John Joyce
c482ef0215
refactor(ui): Misc domains improvements ( #6850 )
2022-12-23 11:06:47 -08:00
Harshal Sheth
ea5ee6f761
fix(ingest/looker): handle missing label fields ( #6849 )
2022-12-22 19:43:44 -05:00
mohdsiddique
9daa8ed56f
feat(ingestion): Business Glossary# Add domain support in GlossaryTerm ingestion ( #6829 )
...
* lint fix
* domain in term
* domain in term
* review comments
* add todo
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2022-12-22 17:47:57 -05:00
RyanHolstien
53fe13ebed
docs(confluent): add details for actions pod for confluent ( #6810 )
2022-12-22 14:40:26 -08:00
david-leifker
bacc2f957b
fix(oidc): fix oidc authentication loop ( #6848 )
...
* fix(oidc): fix oidc authentication loop
2022-12-22 16:12:51 -06:00
Harshal Sheth
1d0c7852a7
feat(ingest): add db/schema properties hook to SQL common ( #6847 )
2022-12-22 13:38:59 -08:00
Chris Collins
2ef2ad05d0
fix(ingestion) Inject pipeline_name into recipes at runtime ( #6833 )
2022-12-22 13:16:51 -08:00
John Joyce
4cba09e97d
fix(ingest): Fixing lint ( #6844 )
2022-12-22 08:33:18 -08:00
wangsaisai
0f8e2d945e
fix(ingest): kafka ingest task hand up with error bootstrap server ( #6820 )
2022-12-22 07:39:30 -08:00
Aseem Bansal
bdd346db27
fix(ci): add labels based on more folders ( #6840 )
2022-12-22 07:33:02 -08:00
Teppo Naakka
b0a82ca7f4
feat(ui): sortable domain list ( #6736 )
2022-12-22 07:22:22 -08:00
Aseem Bansal
86ee0e58de
feat(ci): add cypress test ui based ingestion ( #6769 )
2022-12-22 20:37:29 +05:30
John Joyce
8976844635
feat(ui): Support adding custom id when creating term and term group ( #6830 )
2022-12-22 06:32:46 -08:00
danielli-ziprecruiter
a6470fc267
feat(ingestion): spark - support lineage for delta lake writes ( #6834 )
2022-12-22 12:13:32 +01:00
Mirko R
3b8686d012
docs: Incorrect import statement fixed in example ( #6838 )
2022-12-22 12:08:20 +01:00
Mayuri Nehate
a05c5c4069
feat(ingest): extract kafka topic config properties as customProperties ( #6783 )
2022-12-22 09:34:55 +01:00
david-leifker
2a182f4846
fix(pac4j-oidc): add verifier parameter ( #6835 )
...
* fix(pac4j-oidc): add verifier parameter
2022-12-21 20:11:11 -06:00
Chris Collins
517688d050
fix(lineage) Fix lineage viz with multiple siblings ( #6826 )
2022-12-21 14:36:09 -05:00
John Joyce
2e3a25123d
refactor(ingestion): Browse Paths Upgrade V2 Feast & Sagemaker ( #6002 )
2022-12-21 08:02:59 -08:00
Dago Romer
9cb1eed6e7
fix(ingest): fixed snowflake oauth ingestion not using role attribute from recipe ( #6825 )
2022-12-21 07:52:06 -08:00
Harshal Sheth
e2b4a65a8e
refactor(ingest): clean up exception types ( #6818 )
2022-12-21 07:28:18 -08:00
Aseem Bansal
2fb0b1ecfb
fix(ci): reduce flakiness views select test ( #6821 )
2022-12-21 20:53:58 +05:30
Harshal Sheth
8972ea4b04
fix(ingest): support patches in auto_status_aspect ( #6827 )
...
Patches generate a raw MCP because MCPW doesn't support patches right now, so we need to handle that correctly downstream.
2022-12-21 10:25:24 +01:00
Tamas Nemeth
a1970d2dce
feat(ingest/bigquery): add option to enable/disable legacy sharded table support ( #6822 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: John Joyce <john@acryl.io>
2022-12-20 23:29:46 -05:00
danielli-ziprecruiter
ca6c3b949b
fix(ingest): exclude ztsd from uber jar to prevent jni conflicts with spark ( #6787 )
...
Co-authored-by: Tamas Nemeth <treff7es@gmail.com>
2022-12-20 21:54:56 -06:00
Harshal Sheth
2c911ccf7b
refactor(ingest): clean up pipeline init error handling ( #6817 )
2022-12-20 19:21:28 -08:00
Harshal Sheth
88e40a9069
feat(ingest): add failure/warning counts to ingest_stats ( #6823 )
2022-12-20 19:13:11 -08:00
djordje-mijatovic
e6c48e5f19
feat(kafka): expose default kafka producer mechanism ( #6381 )
...
* Expose Kafka Sender Retry Parameters
* Implement KafkaHealthChecker
* feat(kafka): expose default kafka producer mechanism
2022-12-20 14:41:24 -06:00
John Joyce
b58021ab0a
feat(ui): Adding DBT Cloud support for UI ingestion ( #6804 )
2022-12-20 09:22:03 -08:00
Harshal Sheth
137f4500b6
feat(ingest/stateful): remove platform_instance_id from state urn ( #6795 )
2022-12-20 12:12:19 -05:00
Aditya Radhakrishnan
4a0f798ab5
fix(ui): change caching to happen post server-response when creating a UI ingestion recipe ( #6815 )
2022-12-20 07:57:24 -08:00
Chris Collins
f559dcc282
feat(lineage) Adds ability to edit lineage manually from the UI ( #6816 )
2022-12-20 10:54:08 -05:00
Aditya Radhakrishnan
2475d2bbec
fix(policies): resolve the associated domain for a domain as the domain itself ( #6812 )
2022-12-20 07:13:30 -08:00
Aditya Radhakrishnan
80041d7586
fix(posts): add deletePost GraphQL endpoint ( #6813 )
2022-12-20 07:08:34 -08:00
Aditya Radhakrishnan
cd937e0c67
fix(ui): URL encode urns for ownership entity links ( #6814 )
2022-12-20 07:04:56 -08:00
Harshal Sheth
5584bfb469
refactor(ingest/stateful): remove get_last_state method ( #6794 )
2022-12-19 20:48:22 -05:00
raysaka
fcb3242983
chore(ingest): bump python package dependencies to resolve vulns ( #6384 )
...
Co-authored-by: John Joyce <john@acryl.io>
2022-12-19 18:12:56 -05:00
Harshal Sheth
e9d50ed992
refactor(ingest/stateful): remove IngestionJobStateProvider ( #6792 )
2022-12-19 17:03:54 -05:00
Navin Sharma
7c3872a2ed
Update Grafana Dashboard ( #6076 )
...
* Add Datasource as variable in dashboard
(cherry picked from commit e75b3f7333dd9436f7434eefa76ab9fbfb1babab)
* Update datahub_dashboard.json
(cherry picked from commit 701592697702a99ddf33f75d57826282df999c82)
* Bump docker compose version to 3.8
(cherry picked from commit ff6a97b1692ac21368eacc83a0daed30b0364d5d)
* Update grafana image tag from latest to 9.1.4
(cherry picked from commit 2c88e2a3041b9ddfa70bb92d13e80ca15be0ead2)
* Update old metric name in datahub_dashboard.json
(cherry picked from commit 21b502e25392f37c5053dcf00161fc5c7d35c94d)
* Add panel for new metrics
(cherry picked from commit 194452778bc825820dda87ab2e159b7064a360ef)
Co-authored-by: Peter Szalai <szalaipeti.vagyok@gmail.com>
2022-12-19 22:43:47 +01:00
Monica Senapati
5c366205f5
fix(bigquery-legacy): Fix for TypeError related failures in legacy plugin ( #6806 )
...
Co-authored-by: John Joyce <john@acryl.io>
2022-12-19 13:28:25 -08:00
Harshal Sheth
47be95689e
refactor(ingest/stateful): remove most remaining state classes ( #6791 )
2022-12-19 13:40:48 -05:00
Harshal Sheth
14a00f4098
chore(ingest): pin black version ( #6807 )
2022-12-19 19:35:49 +01:00
Tamas Nemeth
e41b455e14
fix(ingest): bigquery - sharded table support improvements ( #6789 )
2022-12-19 18:57:37 +01:00
John Joyce
807b50699d
refactor(ui): Miscellaneous caching improvements ( #6796 )
2022-12-19 09:55:56 -08:00
