33 Commits

Author SHA1 Message Date
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) (#6744)
* fix(security): commons-text in frontend

* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint

* testing docker build

* Add retry configuration options for entity client

* Kafka debugging

* fix(kafka-setup): parallelize topic creation

* Adjust docker build

* Docker build updates

* WIP

* fix(lint): metadata-ingestion lint

* fix(gradle-docker): fix docker frontend dep

* fix(elastic): fix race condition between gms and mae for index creation

* Revert "fix(elastic): fix race condition between gms and mae for index creation"

This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.

* fix(test): fix datahub frontend test for clean/test cycle

* fix(test): datahub-frontend missing assets in test

* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer

* gitingore update

* fix(docker): remove platform on docker base image, set by buildx

* refactor(kafka-producer): update kafka producer tracking/logging

* updates per PR feedback

* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool

Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
Shirshanka Das
155a68386a
docs(actions): slack, teams (#6632) 2022-12-04 13:27:52 -08:00
Shirshanka Das
5937b35447
fix(actions): add missing datahub-gms-protocol env var (#6593) 2022-12-01 00:25:43 -08:00
Peter Szalai
f77117a9e0
fix(envs): remove DATASET_ENABLE_SCSI stale env var (#6546) 2022-11-24 21:34:53 +05:30
david-leifker
33fd876421
feat(elasticsearch): Updates to elasticsearch configuration, dao, and tests (#6269) 2022-11-15 18:03:11 -08:00
leifker
203a6ff57b
refactor(java11) - convert most modules to java 11 (#5836)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-25 08:39:22 -07:00
mohdsiddique
2b4f9400ca
fix(GMS): fix no such classes directory file:///etc/datahub/plugins/auth/resources (#5743) 2022-08-29 12:08:18 -07:00
mohdsiddique
cfaf7a0c20
fix(quickstart): fix for No such classes directory file:///etc/datahub/plugins/auth/resources (#5720)
Co-authored-by: MohdSiddique Bagwan <mohdsiddique.bagwan@gslab.com>
2022-08-24 09:26:13 -07:00
Aseem Bansal
252992f15e
fix(ci): intermittent failure in github actions (#5452) 2022-07-25 23:02:02 -07:00
Shirshanka Das
941770f3bc
feat(cli): quickstart - experimental support for backup restore (#5418) 2022-07-24 23:52:25 -07:00
Aseem Bansal
acb9879eb4
feat(cli,build): remove deprecated variables GMS_HOST/_PORT (#5451) 2022-07-20 20:54:43 +05:30
Shirshanka Das
024797df0d
feat(quickstart): moving to official confluent images for m1 (#5416) 2022-07-17 18:59:09 -07:00
Navin Sharma
f3e5afdba9
feat(quickstart): Adding env variables and cli options for customizing mapped ports in quickstart (#5353)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-07-11 22:33:24 -07:00
Aseem Bansal
1adcc994f3
feat(ingest): remove need for sink block in UI based ingestion (#5208) 2022-06-23 15:22:40 +05:30
Dexter Lee
d31c00939c
fix(docker): Fix mysql setup bug (#5071) 2022-06-02 10:01:36 +02:00
RyanHolstien
2a2fd05345
feat: telemetry improvements (#5029) 2022-05-27 17:04:45 -07:00
RyanHolstien
bb341f740c
fix(env): provide default for unset telemetry variable (#4937) 2022-05-18 14:01:37 +05:30
RyanHolstien
c3cfaf8e3c
feat(telemetry): add server side telemetry (#4925)
Co-authored-by: Kevin Hu <kevinhuwest@gmail.com>
2022-05-17 11:37:45 -07:00
John Joyce
d96ddec843
refactor(actions): Migrate to use new datahub-actions container (#4903) 2022-05-13 12:11:56 +05:30
John Joyce
c69310522b
feat(metadata service): Introducing Platform Events (#4477) 2022-03-29 18:32:04 -07:00
Shirshanka Das
f18f453200
moving to dockerhub for actions container (#4497) 2022-03-24 22:24:07 -07:00
Aseem Bansal
8c7f63e4f7
feat: change quickstart to use head tag for actions (#4357) 2022-03-09 10:29:10 -08:00
Aseem Bansal
e6a2615c08
feat(quickstart): restart actions pod in case of failures (#4342) 2022-03-08 11:55:21 -08:00
John Joyce
787a7e6dcd
fix(quickstart): Pin actions pod + add volume mount for datahub-frontend (#4318) 2022-03-04 14:05:14 -08:00
John Joyce
35ebca77ee
quickstart: Adding UI ingestion to quickstart YAML (#3994) 2022-01-27 10:47:11 -08:00
Swaroop Jagadish
90b7b9dddd
fix(ci): moving defaults back to head now that docker builds are green (#3993) 2022-01-27 09:48:01 -08:00
John Joyce
9366a47f88
fix(quickstart): Assign correct mysql-setup container for M1 and remove "head" default version. (#3987)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-01-26 18:15:54 -08:00
John Joyce
aeabeec133
fix(quickstart): Pin Quickstart version to v0.8.23. (#3983) 2022-01-26 11:36:26 -08:00
Sergio Gómez Villamor
c59c63e90d
feat: enables dbt metadata files to be loaded from URIs (#3739) 2021-12-15 09:11:39 -08:00
Gabe Lyons
3fd3313544
Revert "feat(graph): Make Dgraph a proper Neo4j alternative (#3578)" (#3740) 2021-12-14 10:49:03 -08:00
Enrico Minack
a6deaabfcf
feat(graph): Make Dgraph a proper Neo4j alternative (#3578) 2021-12-13 12:37:59 -08:00
Swaroop Jagadish
50122cbd98
fix(docs): small fixes to docs and docker images for custom metadata … (#3640) 2021-11-30 00:12:17 -08:00
Tamas Nemeth
d41c5eba35
feat(docker): add multiplatform docker support for arm64 (m1) (#3537) 2021-11-09 11:50:06 -08:00