40 Commits

Author SHA1 Message Date
Chris Collins
68fb0d8f06
feat(schema) Show last observed timestamp in the schema tab (#5348) 2022-07-07 12:31:01 -07:00
Pedro Silva
bb3208d734
fix(gms) Remove auto-creating status aspects if not present when ingesting metadata (#5315) 2022-07-01 15:45:09 -07:00
John Joyce
11356e37ba
feat(ui) Adding support for deleting Tags and Domains via the UI (#5280)
* Adding support for deleting tags and domains via the UI

* Fixing tests
2022-06-29 19:41:41 -07:00
RyanHolstien
ba21efc7dd
feat(bootstrap): add bootstrap step to clear out unknown aspect rows from the database (#5148) 2022-06-21 16:00:16 -07:00
Justin Marozas
97bce80fc9
refactor(metadata-io): introduce a storage-independent in-memory entity aspect model (#4957) 2022-05-24 12:40:42 -07:00
RyanHolstien
71c7dc455f
fix(metadata-service): telemetry - fix hardcoded aspect name, suppress errors when producing MAE (#4981) 2022-05-24 09:18:20 -07:00
RyanHolstien
c3cfaf8e3c
feat(telemetry): add server side telemetry (#4925)
Co-authored-by: Kevin Hu <kevinhuwest@gmail.com>
2022-05-17 11:37:45 -07:00
RyanHolstien
65fb49f4b4
feat(versionedDataset): adds a versionStamp to timeline response & adds versionStamp param to dataset graphql (#4727) 2022-04-29 12:04:38 -07:00
Eddie (Xiao Di) Li
01a5b13a7a
feat(Cassandra): Add Cassandra implementation of EntityService (#3286) 2022-04-25 17:29:24 -07:00
John Joyce
f387901ca8
feat(gms): Entity change events v1 (Platform Event) (#4687) 2022-04-19 20:04:14 -07:00
John Joyce
c69310522b
feat(metadata service): Introducing Platform Events (#4477) 2022-03-29 18:32:04 -07:00
RyanHolstien
b843014c5e
Oss/urn validation (#4511) 2022-03-29 16:44:58 -07:00
Dexter Lee
df69e76289
feat(policy): Add domain based and view based policies (#4516)
* Add domain based policy and view based policy

* Add new files

* Fix issue

* Fix comment on markdown

* Fix to John's comments

* Update policies.md

* Remove comments

* Fix to John's comments

* Fix policy details page

* Add back notebook privileges

* Fix graphql

Co-authored-by: John Joyce <john@acryl.io>
2022-03-29 12:02:22 -07:00
Pedro Silva
e8f6c4cabd
feat(cli) Changes rollback behaviour to apply soft deletes by default (#4358)
* Changes rollback behaviour to apply soft deletes by default

Summary:
Addresses feature request: Flag in delete command to only delete aspects touched by an ingestion run; add flag to nuke everything by modifying the default behaviour of a rollback operation which will not by default delete an entity if a keyAspect is being rolled-back.

Instead the key aspect is kept and a StatusAspect is upserted with removed=true, effectively making a soft delete.
Another PR will follow to perform garbage collection on these soft deleted entities.

To keep old behaviour, a new parameter to the cli ingest rollback endpoint: --hard-delete was added.

* Adds restli specs

* Fixes deleteAspect endpoint & adds support for nested transactions

* Enable regression test & fix docker-compose for local development

* Add generated quickstart

* Fix quickstart generation script

* Adds missing var env to docker-compose-without-neo4j

* Sets status removed=true when ingesting resources

* Adds soft deletes for ElasticSearch + soft delete flags across ingestion sub-commands

* Makes elastic search consistent

* Update tests with new behaviour

* apply review comments

* apply review comment

* Forces Elastic search to add documents with status removed false when ingesting

* Reset gradle properties to default

* Fix tests
2022-03-15 12:05:52 -07:00
RyanHolstien
a251d589ed
Feature/oss/update to v2 endpoints (#4128) 2022-02-15 14:37:05 -08:00
RyanHolstien
d46c55733e
fix(platform): prevent invalid urns during ingestion (#4045)
Co-authored-by: Ryan Holstien <ryan@acryl.io>
2022-02-02 22:58:44 -08:00
Aseem Bansal
a56f0661d5
perf(ingest): changes to improve ingest performance a bit (#3837) 2022-01-06 08:07:16 -08:00
RyanHolstien
34c27f076b
feat(removeGMA): remove all dependencies on gma libraries (#3835) 2022-01-05 17:32:31 -08:00
Dexter Lee
9d04aa1255
feat(mae/mcl): Make ingestAspect produce both MCLs and MAEs (#3737) 2022-01-04 18:24:30 -08:00
Dexter Lee
76f9423581
feat(nocode): Add get entities v2 endpoint that can get without snapshot (#3738) 2022-01-03 07:19:46 -08:00
Dexter Lee
8e4769f40c
feat(retention): Add retention to Local DB (#3715) 2021-12-13 18:18:02 -08:00
Shirshanka Das
842c05980b
feat(metadata-service): support for custom model extensions without forks (#3630) 2021-11-28 21:06:27 -08:00
John Joyce
f49666a230
feat(auth): Metadata Service Authentication! (#3598) 2021-11-22 16:33:14 -08:00
Gabe Lyons
b87efa43b5
feat(deletes): support deletion of non-snapshot aspects (#3518)
Co-authored-by: Dexter Lee <dexter@acryl.io>
2021-11-08 16:22:24 -08:00
John Joyce
a92ab66a3a
refactor(nocode): Final part of No-Code cleanup (#3477) 2021-10-31 22:06:36 -07:00
Gabe Lyons
7dbd10f072
fix(browse): fix browse for entities without default browse logic (#3422) 2021-10-21 11:15:10 -07:00
Dexter Lee
9b54242a6e
fix(datahub-upgrade): Update datahub upgrade to use MCL instead of MAE (#3411) 2021-10-18 22:52:48 -07:00
Dexter Lee
d0548408b4
feat(search): redesign search page - left filter pane (#3337) 2021-10-07 11:41:29 -07:00
Gabe Lyons
6bf237b38e
feat(graphql): Adding write side validation and tests for add+remove API (#3259) 2021-09-20 11:58:31 -07:00
Gabe Lyons
2de4ea9566
feat(business glossary): Add support to add & remove glossary terms from the UI (#3213) 2021-09-13 09:16:37 -07:00
John Joyce
ccb09a6966
feat(access control): Fine-Grained Access Control M1 (#3182) 2021-09-02 19:05:13 -07:00
John Joyce
f3fc0970f3
refactor(build): Remove unnecessary ext modules. (#3074) 2021-08-10 22:48:06 -07:00
John Joyce
352a0abf8d
Introducing TimeSeries Aspects + Dataset Profile (Stats) Aspect (#2983)
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-07-30 17:41:03 -07:00
Gabe Lyons
aa253f5b3b
feat(deletes): add run commands (list, show, rollback) to datahub ingest (#2960) 2021-07-29 20:04:40 -07:00
Dexter Lee
8f0f322279
feat(backup): Add restore indices and restore backup tasks (#2779) 2021-06-30 16:49:02 -07:00
John Joyce
09cbc548a4
feat(logs): improve logging in GMS and datahub-frontend (#2761) 2021-06-25 10:56:45 -07:00
John Joyce
550a9deab0
fix(noCode): Improving efficiency of EntityService "listLatestAspects" API (#2711) 2021-06-17 19:52:50 -07:00
Gabe Lyons
523c3bf1d4
feat(aspects): support fetching of versioned aspects (#2677) 2021-06-16 10:03:21 -07:00
John Joyce
d26903f160
fix(gms): Return empty snapshots when one does not exist (#2646) 2021-06-04 11:28:53 -07:00
John Joyce
97e9660037
feat: No Code Metadata Modeling (#2629)
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Gabe Lyons <itsgabelyons@gmail.com>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-06-03 13:24:33 -07:00