61 Commits

Author SHA1 Message Date
david-leifker
b61c38ab05
refactor(misc): testngJava fix, systemrestli client, cache key fix, e… (#8926) 2023-09-30 22:47:59 -05:00
david-leifker
aef49b8fb2
feat(entity-client): enable client side cache for entity-client and usage-client (#8877) 2023-09-21 22:00:14 -05:00
david-leifker
21eb4dfc12
feat(search): update to support OpenSearch 2.x (#8852) 2023-09-21 13:01:55 -05:00
ukayani
0f3819b511
feat(upgrade): add ability to provide a startingOffset for RestoreIndices (#8539) 2023-09-11 10:13:55 -05:00
david-leifker
1b79142d9e
feat(EntityService): batched transactions and ebean updates (#8456) 2023-09-02 19:25:44 -05:00
RyanHolstien
9f791a3b6e
feat(browseV2): add browseV2 logic to system update (#8506)
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-08-01 12:48:02 -05:00
RyanHolstien
93fde6bf93
feat(io): refactor metadata-io module (#8306) 2023-07-20 06:39:14 +05:30
david-leifker
217151ea55
fix(schema-registry): fix internal schema reg with custom duhe topic … (#8371) 2023-07-06 14:15:58 -05:00
david-leifker
22e77710f2
fix(nocode): fix DeleteLegacyGraphRelationshipsStep for Elasticsearch (#8181)
Co-authored-by: Pedro Silva <pedro@acryl.io>
2023-06-07 09:36:03 -05:00
Aseem Bansal
0b0e4997bf
chore(log): fix log as error instead of info (#8146) 2023-05-31 15:30:50 +05:30
RyanHolstien
530b9a6540
fix(jackson): increase max serialized string length default (#8053) 2023-05-23 11:38:58 -05:00
david-leifker
b97e7a03e8
feat(system-update): remove datahub-update requirement on schema reg (#7999) 2023-05-17 10:32:23 -05:00
RyanHolstien
d061d8782f
feat(metrics): add metrics for upgrade steps (#7992) 2023-05-10 11:59:14 -05:00
Aseem Bansal
1f88dab6f9
chore(log): change sout to log (#7931) 2023-05-06 17:50:26 +05:30
Felipe Ribeiro
6f5f8de99a
feat(upgrade): add gms protocol variable (#7752)
Co-authored-by: Felipe Ribeiro <fribeiro@fanatics.com>
Co-authored-by: david-leifker <114954101+david-leifker@users.noreply.github.com>
2023-04-12 20:47:55 -05:00
Aseem Bansal
b039ad1176
refactor: misc fixes logging, annotations (#7579)
Co-authored-by: John Joyce <john@acryl.io>
2023-03-16 09:49:35 +05:30
david-leifker
72198f98c0
feat(search): searchAcrossEntities/Lineage improvements (#7550)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-03-12 18:18:51 -07:00
david-leifker
7b939ab2a4
feat(elasticsearch): validate index.blocks.write setting (#7478) 2023-03-02 18:04:12 -06:00
John Joyce
32b55e7caa
refactor(upgrade): Trim upgrade name before executing (#7343) 2023-02-15 08:59:06 -08:00
david-leifker
39920bb00f
feat(elasticsearch): Elasticsearch improvements (#6894) 2023-01-31 18:44:37 -06:00
david-leifker
a5b5abf270
fix(kafka): datahub-upgrade job (#6864) 2022-12-27 12:35:48 -06:00
Pedro Silva
5fd5866a03
feat(upgrade): Make restore from backup logic generic (#6536) 2022-11-24 21:49:24 +05:30
John Joyce
d15518f0d9
fix(browse): Fixing browse path to remove requirement for simple name suffix (#5634) 2022-09-07 13:32:38 -07:00
Aseem Bansal
b643b34d30
feat(restore-indices): add endpoint for restore indices, add basic check for graph (#5805) 2022-09-02 15:24:40 +05:30
Aseem Bansal
157363586e
feat(restore-indices): add timing info (#5773) 2022-08-30 17:38:20 +05:30
Aseem Bansal
b477047b3b
feat(restore-indices): add urn like filter (#5770)
* feat(restore-indices): add urn like filter

* change logging
2022-08-30 15:16:48 +05:30
Aseem Bansal
67a68b9705
feat(restore-indices): add multithreading and add aspectName, urn filter (#5712)
* feat(upgrade): improve logging to get idea of progress

* division by zero check

* division by zero fix

* wip

* temp work

* feat(datahub-upgrade): add multi-threading and aspect filter

* revert change on local

* fix bugs

* add back batchDelayMs

* remove redundant format

* report stats finally, shutdown executor

* Add time metrics

* clarify logging

* improve logging
2022-08-29 11:43:39 -05:00
RyanHolstien
ba21efc7dd
feat(bootstrap): add bootstrap step to clear out unknown aspect rows from the database (#5148) 2022-06-21 16:00:16 -07:00
RyanHolstien
92d6b043e1
fix(upgrade): fix upgrade when s3 path has = (#5220) 2022-06-21 15:25:46 -07:00
Dexter Lee
b890ba0d12
fix(restore-indices): Do not fail on MAE row count diff (#5165) 2022-06-14 10:12:10 -07:00
Dexter Lee
59d33ca28b
fix(restore-indices): Do not fail while working with each row (#5084) 2022-06-03 14:11:12 -07:00
RyanHolstien
259a63ab6f
fix(datahub-upgrade): exclude unnecessary configuration from standalone applications (#5047) 2022-05-31 22:59:31 -07:00
John Joyce
1a31f7888a
fix(restore): Add RESTATE ChangeType to MCL / MCP to permit restore indices (#5022) 2022-05-30 10:04:51 -07:00
RyanHolstien
a39a8b26a2
fix(telemetry): exclude configuration from standalone apps (#5034) 2022-05-27 17:16:53 -07:00
Justin Marozas
97bce80fc9
refactor(metadata-io): introduce a storage-independent in-memory entity aspect model (#4957) 2022-05-24 12:40:42 -07:00
Eddie (Xiao Di) Li
01a5b13a7a
feat(Cassandra): Add Cassandra implementation of EntityService (#3286) 2022-04-25 17:29:24 -07:00
John Joyce
f387901ca8
feat(gms): Entity change events v1 (Platform Event) (#4687) 2022-04-19 20:04:14 -07:00
Dexter Lee
b8cdc7ebed
feat(vulnerability): Upgrade spring libraries to latest version (#4698) 2022-04-19 16:42:41 -07:00
Dexter Lee
18dd5b6c13
feat(Impact Analysis): Support impact analysis to check all downstreams of given entity (#4322) 2022-03-04 16:10:25 -08:00
Aseem Bansal
a56f0661d5
perf(ingest): changes to improve ingest performance a bit (#3837) 2022-01-06 08:07:16 -08:00
RyanHolstien
34c27f076b
feat(removeGMA): remove all dependencies on gma libraries (#3835) 2022-01-05 17:32:31 -08:00
Dexter Lee
b00fe8426e
fix(datahub-upgrade): Fix Spring injection issue with datahub-upgrade (#3688) 2021-12-09 21:31:33 -08:00
John Joyce
f49666a230
feat(auth): Metadata Service Authentication! (#3598) 2021-11-22 16:33:14 -08:00
John Joyce
27a5f6dc3b
feat(gms): Adding yaml configuration for metadata-service (#3514) 2021-11-09 12:16:15 -08:00
Gabe Lyons
b87efa43b5
feat(deletes): support deletion of non-snapshot aspects (#3518)
Co-authored-by: Dexter Lee <dexter@acryl.io>
2021-11-08 16:22:24 -08:00
John Joyce
a92ab66a3a
refactor(nocode): Final part of No-Code cleanup (#3477) 2021-10-31 22:06:36 -07:00
Dexter Lee
87d344faa3
fix(backup-restore): fix backup restore jobs usage of system-metadata (#3445) 2021-10-25 14:43:04 -07:00
John Joyce
8ef273423c
feat(metadata-service): Introducing EntityClient interface to avoid unnecessary HTTP calls. (#3421) 2021-10-19 15:53:48 -07:00
Dexter Lee
9b54242a6e
fix(datahub-upgrade): Update datahub upgrade to use MCL instead of MAE (#3411) 2021-10-18 22:52:48 -07:00
Dexter Lee
d0548408b4
feat(search): redesign search page - left filter pane (#3337) 2021-10-07 11:41:29 -07:00