Aseem Bansal
867007f209
chore: remove obsolete attribute from docker ( #11573 )
2024-10-09 20:52:50 +05:30
david-leifker
4296373c5f
config(rest-api): enable authentication and api authorization by default ( #11484 )
2024-09-27 11:31:25 -05:00
david-leifker
edb9a87b84
fix(elasticsearch): refactor idHashAlgo setting ( #11193 )
2024-08-16 14:41:44 -05:00
Pinaki Bhattacharjee
a5a33f074b
feat(config): support alternate hashing algorithm for doc id ( #10423 )
...
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
Co-authored-by: John Joyce <john@acryl.io>
2024-08-06 13:50:38 -05:00
Brandon Brown
9c5bea34cd
feat(docker/quickstart): Adding in support for overriding the conflue… ( #10533 )
2024-07-01 16:11:50 +00:00
Davi Arnaut
995af4e980
fix(quickstart): remove unneeded init.sql ( #10266 )
2024-04-10 09:05:20 -07:00
david-leifker
77c4629ccf
refactor(docker): move to acryldata repo for all images ( #9459 )
2024-04-02 09:36:44 -05:00
RyanHolstien
ef637ccb37
fix(docker): fix versioning for compose file post release ( #10176 )
2024-04-01 15:01:09 -05:00
Harshal Sheth
5793467e29
feat(quickstart): remove container_name from compose files ( #9809 )
2024-02-09 20:45:30 -05:00
RyanHolstien
131c8f878c
feat(kafka): add health indicator for kafka ( #9662 )
2024-01-19 13:44:48 -06:00
RyanHolstien
55cb56821c
fix(kafka): fix infinite deserialization logging ( #9494 )
2023-12-21 10:33:25 -06:00
david-leifker
3777730d78
fix(quickstart): force strings for mysql version ( #9485 )
2023-12-19 15:30:47 -06:00
david-leifker
3a840371cc
fix(docker): docker compose health checks port fix ( #9326 )
2023-11-28 21:21:15 -06:00
RyanHolstien
a62d52e6b1
feat(mysql): upgrade to version 8.2 for quickstart ( #9241 )
...
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-11-22 13:54:12 -06:00
Xuelei Li
179f103412
fix(metadata-io): in Neo4j service use proper algorithm to get lineage ( #8687 )
...
Co-authored-by: RyanHolstien <RyanHolstien@users.noreply.github.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-11-10 10:58:38 -06:00
deepgarg-visa
399e032dfa
feat(neo4j): Allow datahub to connect to specific neo4j database ( #9179 )
...
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-11-08 11:02:13 -06:00
david-leifker
353584c10a
feat(browsepathv2): Allow system-update to reprocess browse paths v2 ( #9200 )
2023-11-07 18:22:18 -06:00
Harshal Sheth
b565a657d2
feat(quickstart): fix broker InconsistentClusterIdException issues ( #9148 )
2023-10-30 23:35:12 -07:00
david-leifker
3f4ab44a91
feat(kafka): increase kafka message size and enable compression ( #9038 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-10-29 16:26:05 -05:00
david-leifker
21eb4dfc12
feat(search): update to support OpenSearch 2.x ( #8852 )
2023-09-21 13:01:55 -05:00
david-leifker
40caf35812
feat(datahub-ingestion): refactor datahub ingestion slim images ( #8515 )
2023-08-17 10:40:17 +05:30
Harshal Sheth
b8681a7129
feat(quickstart): bump quickstart start periods more ( #8573 )
2023-08-04 15:17:16 +05:30
RyanHolstien
9f791a3b6e
feat(browseV2): add browseV2 logic to system update ( #8506 )
...
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-08-01 12:48:02 -05:00
david-leifker
3640291493
config(ebean): add ebean retry configuration ( #8500 )
2023-07-28 14:09:44 -05:00
Harshal Sheth
2a3f6c42d4
ci: workarounds for pyyaml installation ( #8435 )
2023-07-17 15:26:43 -07:00
Harshal Sheth
e5f9ee178c
fix(cli/quickstart): handle docker hangs gracefully ( #8211 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-06-13 14:34:06 -05:00
Jinlin Yang
166dc90896
Upgrade kafka and its dependencies to 3.4 in docker compose ( #8161 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-06-13 13:57:54 +01:00
david-leifker
43d37ff442
chore(cp-schema-registry): bump minor version ( #8192 )
2023-06-09 11:53:18 +01:00
Guillaume Cernier
ef1ada118d
fix(docker-compose): make startup more robust with deterministic services' dependencies ( #7880 )
2023-05-08 16:42:15 -05:00
david-leifker
cd05f5b174
feat(schema-registry): replace confluent schema registry ( #7930 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
Co-authored-by: Ryan Holstien <ryan@acryl.io>
2023-05-01 13:18:41 -05:00
Harshal Sheth
8f8c11081d
fix(cli): remove duplicate labels from quickstart files ( #7886 )
2023-04-25 14:35:34 -07:00
Harshal Sheth
7cf3d378a8
feat(quickstart): move quickstart back to master ( #7697 )
2023-03-28 15:10:57 +02:00
Harshal Sheth
81ec296b06
feat(cli): switch default quickstart to v0.10.0 ( #7567 )
2023-03-13 21:42:05 -07:00
Peter Szalai
fd1bd51541
Docs update ( #7517 )
2023-03-08 13:43:46 +01:00
Peter Szalai
1d33392761
feat(cli): introduce remote config for quickstart ( #7424 )
...
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
2023-03-07 13:14:24 +01:00
Peter Szalai
e9f060dae3
Add setup job labels to compose files ( #7473 )
2023-03-02 15:05:16 +01:00
Shirshanka Das
516aee0aaa
fix(docker): add env variable to control mysql setup image, sort dock… ( #7266 )
2023-02-07 00:56:08 -08:00
david-leifker
39920bb00f
feat(elasticsearch): Elasticsearch improvements ( #6894 )
2023-01-31 18:44:37 -06:00
Peter Szalai
8a2230aea4
fix(quickstart): adding back kafka-setup ( #7181 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-01-31 00:34:36 +01:00
Pedro Silva
bef59b0928
feat(quickstart): Remove kafka-setup as a hard deployment requirement ( #7073 )
2023-01-24 16:12:57 +00:00
david-leifker
9277101a5f
config(quickstart): enable auto-reindex for quickstart ( #6983 )
2023-01-09 17:58:29 -06:00
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) ( #6744 )
...
* fix(security): commons-text in frontend
* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint
* testing docker build
* Add retry configuration options for entity client
* Kafka debugging
* fix(kafka-setup): parallelize topic creation
* Adjust docker build
* Docker build updates
* WIP
* fix(lint): metadata-ingestion lint
* fix(gradle-docker): fix docker frontend dep
* fix(elastic): fix race condition between gms and mae for index creation
* Revert "fix(elastic): fix race condition between gms and mae for index creation"
This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.
* fix(test): fix datahub frontend test for clean/test cycle
* fix(test): datahub-frontend missing assets in test
* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer
* gitingore update
* fix(docker): remove platform on docker base image, set by buildx
* refactor(kafka-producer): update kafka producer tracking/logging
* updates per PR feedback
* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
Navin Sharma
7c3872a2ed
Update Grafana Dashboard ( #6076 )
...
* Add Datasource as variable in dashboard
(cherry picked from commit e75b3f7333dd9436f7434eefa76ab9fbfb1babab)
* Update datahub_dashboard.json
(cherry picked from commit 701592697702a99ddf33f75d57826282df999c82)
* Bump docker compose version to 3.8
(cherry picked from commit ff6a97b1692ac21368eacc83a0daed30b0364d5d)
* Update grafana image tag from latest to 9.1.4
(cherry picked from commit 2c88e2a3041b9ddfa70bb92d13e80ca15be0ead2)
* Update old metric name in datahub_dashboard.json
(cherry picked from commit 21b502e25392f37c5053dcf00161fc5c7d35c94d)
* Add panel for new metrics
(cherry picked from commit 194452778bc825820dda87ab2e159b7064a360ef)
Co-authored-by: Peter Szalai <szalaipeti.vagyok@gmail.com>
2022-12-19 22:43:47 +01:00
david-leifker
637b540fa6
fix(kafka-setup): fix return code check ( #6782 )
...
* fix(kafka-setup): parallelize topic creation
* Remove -setup from docker compose (not services)
2022-12-16 13:39:18 -06:00
mohdsiddique
f2a83f3074
feat(gms): Pluggable Authentication & Authorization Framework ( #6634 )
2022-12-06 10:22:41 -08:00
Shirshanka Das
155a68386a
docs(actions): slack, teams ( #6632 )
2022-12-04 13:27:52 -08:00
Shirshanka Das
4876fdd084
feat(quickstart): add support for env vars for Slack, Teams actions ( #6591 )
2022-12-03 23:00:50 -08:00
Shirshanka Das
5937b35447
fix(actions): add missing datahub-gms-protocol env var ( #6593 )
2022-12-01 00:25:43 -08:00
david-leifker
a2dc229c62
fix(tests): Misc updates for tests, auth log level, and quickstart ( #6491 )
2022-11-29 08:44:55 -08:00
Peter Szalai
f77117a9e0
fix(envs): remove DATASET_ENABLE_SCSI stale env var ( #6546 )
2022-11-24 21:34:53 +05:30
