david-leifker
b61c38ab05
refactor(misc): testngJava fix, systemrestli client, cache key fix, e… ( #8926 )
2023-09-30 22:47:59 -05:00
david-leifker
aef49b8fb2
feat(entity-client): enable client side cache for entity-client and usage-client ( #8877 )
2023-09-21 22:00:14 -05:00
david-leifker
21eb4dfc12
feat(search): update to support OpenSearch 2.x ( #8852 )
2023-09-21 13:01:55 -05:00
ukayani
0f3819b511
feat(upgrade): add ability to provide a startingOffset for RestoreIndices ( #8539 )
2023-09-11 10:13:55 -05:00
david-leifker
1b79142d9e
feat(EntityService): batched transactions and ebean updates ( #8456 )
2023-09-02 19:25:44 -05:00
david-leifker
7dd6e09ac5
refactor(build): upgrade to gradle 7 & guava update ( #8745 )
2023-09-01 19:36:01 +05:30
david-leifker
40caf35812
feat(datahub-ingestion): refactor datahub ingestion slim images ( #8515 )
2023-08-17 10:40:17 +05:30
RyanHolstien
9f791a3b6e
feat(browseV2): add browseV2 logic to system update ( #8506 )
...
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-08-01 12:48:02 -05:00
RyanHolstien
93fde6bf93
feat(io): refactor metadata-io module ( #8306 )
2023-07-20 06:39:14 +05:30
david-leifker
217151ea55
fix(schema-registry): fix internal schema reg with custom duhe topic … ( #8371 )
2023-07-06 14:15:58 -05:00
david-leifker
81fdf9df55
fix(vulnerabilities)/vulnerabilities_fixes_datahub ( #8075 ) ( #8189 )
...
Co-authored-by: Sejal-NucleusTeq <109514187+Sejal-NucleusTeq@users.noreply.github.com>
2023-06-07 18:42:19 -05:00
david-leifker
22e77710f2
fix(nocode): fix DeleteLegacyGraphRelationshipsStep for Elasticsearch ( #8181 )
...
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-06-07 09:36:03 -05:00
Aseem Bansal
0b0e4997bf
chore(log): fix log as error instead of info ( #8146 )
2023-05-31 15:30:50 +05:30
RyanHolstien
530b9a6540
fix(jackson): increase max serialized string length default ( #8053 )
2023-05-23 11:38:58 -05:00
david-leifker
b97e7a03e8
feat(system-update): remove datahub-update requirement on schema reg ( #7999 )
2023-05-17 10:32:23 -05:00
RyanHolstien
d061d8782f
feat(metrics): add metrics for upgrade steps ( #7992 )
2023-05-10 11:59:14 -05:00
Aseem Bansal
1f88dab6f9
chore(log): change sout to log ( #7931 )
2023-05-06 17:50:26 +05:30
david-leifker
2ffa39f009
Logging and test models fixes ( #7884 )
2023-04-21 15:45:20 -05:00
RyanHolstien
cbf0809c4c
chore(snakeyaml): upgrade to snakeyaml 2 ( #7786 )
...
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-04-13 12:01:51 -05:00
Felipe Ribeiro
6f5f8de99a
feat(upgrade): add gms protocol variable ( #7752 )
...
Co-authored-by: Felipe Ribeiro <fribeiro@fanatics.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-04-12 20:47:55 -05:00
Aseem Bansal
b039ad1176
refactor: misc fixes logging, annotations ( #7579 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-03-16 09:49:35 +05:30
david-leifker
72198f98c0
feat(search): searchAcrossEntities/Lineage improvements ( #7550 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-03-12 18:18:51 -07:00
david-leifker
7b939ab2a4
feat(elasticsearch): validate index.blocks.write setting ( #7478 )
2023-03-02 18:04:12 -06:00
david-leifker
eea95be4fc
chore(bump): bump hadoop client and fix exclusion name ( #7386 )
2023-02-20 19:53:59 -06:00
david-leifker
bd11575d5f
chore(version): additional version bumps & suppressions ( #7382 )
2023-02-20 14:39:27 -06:00
david-leifker
1df806d177
chore(versions): bump versions ( #7358 )
2023-02-20 11:32:05 +05:30
david-leifker
267472ccbb
gradle(development): add additional commands for development ( #7321 )
2023-02-16 10:41:42 -06:00
John Joyce
32b55e7caa
refactor(upgrade): Trim upgrade name before executing ( #7343 )
2023-02-15 08:59:06 -08:00
david-leifker
2e339fe008
fix(datahub-upgrade): allow registry override ( #7258 )
2023-02-10 21:09:09 +05:30
david-leifker
39920bb00f
feat(elasticsearch): Elasticsearch improvements ( #6894 )
2023-01-31 18:44:37 -06:00
david-leifker
122c024bab
refactor(gradle): move the local docker registry to common location ( #6881 )
2022-12-29 15:08:36 +05:30
david-leifker
a5b5abf270
fix(kafka): datahub-upgrade job ( #6864 )
2022-12-27 12:35:48 -06:00
david-leifker
ecc01b9a46
refactor(restli-mce-consumer) ( #6744 )
...
* fix(security): commons-text in frontend
* refactor(restli): set threads based on cpu cores
feat(mce-consumers): hit local restli endpoint
* testing docker build
* Add retry configuration options for entity client
* Kafka debugging
* fix(kafka-setup): parallelize topic creation
* Adjust docker build
* Docker build updates
* WIP
* fix(lint): metadata-ingestion lint
* fix(gradle-docker): fix docker frontend dep
* fix(elastic): fix race condition between gms and mae for index creation
* Revert "fix(elastic): fix race condition between gms and mae for index creation"
This reverts commit 9629d12c3bdb3c0dab87604d409ca4c642c9c6d3.
* fix(test): fix datahub frontend test for clean/test cycle
* fix(test): datahub-frontend missing assets in test
* fix(security): set protobuf lib datahub-upgrade & mce/mae-consumer
* gitingore update
* fix(docker): remove platform on docker base image, set by buildx
* refactor(kafka-producer): update kafka producer tracking/logging
* updates per PR feedback
* Add documentation around mce standalone consumer
Kafka consumer concurrency to follow thread count for restli & sql connection pool
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-26 16:09:08 +00:00
david-leifker
1e5d434501
fix(security): commons-text in frontend, hadoop-commons in datahub-upgrade ( #6723 )
2022-12-09 14:11:12 -06:00
david-leifker
2de9d3d5bf
fix(logging): Remove lombok as source of slf4j-api, convert to compileOnly where possible ( #6616 )
2022-12-04 19:57:47 -08:00
Pedro Silva
5fd5866a03
feat(upgrade): Make restore from backup logic generic ( #6536 )
2022-11-24 21:49:24 +05:30
Parham Ghazanfari
497c5eee74
feat(datahub-upgrade): Added msk IAM auth as a build dependency. ( #6439 )
2022-11-22 15:36:53 -06:00
John Joyce
d15518f0d9
fix(browse): Fixing browse path to remove requirement for simple name suffix ( #5634 )
2022-09-07 13:32:38 -07:00
Aseem Bansal
b643b34d30
feat(restore-indices): add endpoint for restore indices, add basic check for graph ( #5805 )
2022-09-02 15:24:40 +05:30
Aseem Bansal
157363586e
feat(restore-indices): add timing info ( #5773 )
2022-08-30 17:38:20 +05:30
Aseem Bansal
b477047b3b
feat(restore-indices): add urn like filter ( #5770 )
...
* feat(restore-indices): add urn like filter
* change logging
2022-08-30 15:16:48 +05:30
Aseem Bansal
67a68b9705
feat(restore-indices): add multithreading and add aspectName, urn filter ( #5712 )
...
* feat(upgrade): improve logging to get idea of progress
* division by zero check
* division by zero fix
* wip
* temp work
* feat(datahub-upgrade): add multi-threading and aspect filter
* revert change on local
* fix bugs
* add back batchDelayMs
* remove redundant format
* report stats finally, shutdown executor
* Add time metrics
* clarify logging
* improve logging
2022-08-29 11:43:39 -05:00
RyanHolstien
ba21efc7dd
feat(bootstrap): add bootstrap step to clear out unknown aspect rows from the database ( #5148 )
2022-06-21 16:00:16 -07:00
RyanHolstien
92d6b043e1
fix(upgrade): fix upgrade when s3 path has = ( #5220 )
2022-06-21 15:25:46 -07:00
Dexter Lee
b890ba0d12
fix(restore-indices): Do not fail on MAE row count diff ( #5165 )
2022-06-14 10:12:10 -07:00
Dexter Lee
59d33ca28b
fix(restore-indices): Do not fail while working with each row ( #5084 )
2022-06-03 14:11:12 -07:00
RyanHolstien
259a63ab6f
fix(datahub-upgrade): exclude unnecessary configuration from standalone applications ( #5047 )
2022-05-31 22:59:31 -07:00
John Joyce
1a31f7888a
fix(restore): Add RESTATE ChangeType to MCL / MCP to permit restore indices ( #5022 )
2022-05-30 10:04:51 -07:00
RyanHolstien
a39a8b26a2
fix(telemetry): exclude configuration from standalone apps ( #5034 )
2022-05-27 17:16:53 -07:00
Justin Marozas
97bce80fc9
refactor(metadata-io): introduce a storage-independent in-memory entity aspect model ( #4957 )
2022-05-24 12:40:42 -07:00
