Egemen Berk Galatali
7dad712b1d
fix(ingestion): Readd batchDelayMs ( #7559 )
2023-03-15 13:26:02 -07:00
Pedro Silva
4f8a5c3800
feat(timeBasedLineage): add feature flag for always producing MCL ( #7407 )
2023-02-22 15:52:51 +00:00
John Joyce
92cd2b2c1b
fix(lineage): Fix Upstream + Downstream Count in presence of Soft-Deleted / Non-Existent references ( #7374 )
2023-02-20 14:00:14 -08:00
Pedro Silva
bec5a27a7e
fix(async ingest): Fix async ingest path ( #7269 )
2023-02-07 18:39:52 +00:00
Aditya Radhakrishnan
c13e7e09f8
feat(lineage): add time filtering for lineage edges ( #7159 )
2023-02-02 15:30:49 -08:00
Pedro Silva
4732694780
fix(gms): Corrects MCP generation in async mode ( #7214 )
...
Co-authored-by: John Joyce <john@acryl.io>
2023-02-02 11:45:44 -08:00
david-leifker
c845866521
test(misc): misc test updates ( #6890 )
2022-12-29 11:26:42 -06:00
david-leifker
d8af0b43bd
fix(runid): always update runid, except when queued ( #6876 )
2022-12-28 13:09:30 -06:00
david-leifker
cb6314c51d
fix(elasticsearch): fixes out of order runId writes ( #6845 )
...
Co-authored-by: leifker <dleifker@gmail.com>
Co-authored-by: Pedro Silva <pedro@acryl.io>
2022-12-27 17:50:11 -06:00
RyanHolstien
bfb903cfb8
feat(ingest): add async option to ingest proposal endpoint ( #6097 )
...
* feat(ingest): add async option to ingest proposal endpoint
* small tweak to validate before write to K, also keep existing path for timeseries aspects
* avoid double convert
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-10-03 19:56:19 -05:00
Aseem Bansal
79575b2815
feat(retention): change default policy, add API to apply retention ( #6088 )
2022-09-30 16:11:39 +05:30
RyanHolstien
0728656fbf
feat(patch): initial support of json patch style semantics in MCPs ( #5901 )
...
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-22 23:52:18 -07:00
Parham Ghazanfari
2712dd1903
fix(patch-entity-registry): Use AspectSpec to retrieve aspect class in order to support custom entity ingestion via patch mechanism. ( #5936 )
...
* fix(patch-entity-registry): Remove exception for entities with key aspects.
* test(patch-entity-registry): Fix failing unit test for entities with key aspect
* test(patch-entity-registry): Fix checkstyle for PatchEntityRegistryTest.java
* fix(patch-entity-registry): Remove duplicate semicolon.
* fix(patch-entity-registry): Use AspectSpec to retrieve aspect class in order to support custom entity ingestion via patch mechanism.
* fix(patch-entity-registry): Fixed incorrect merge conflict resolution in BrowsePathUtils.java
* fix(patch-entity-registry): Remove extraneous import from BrowsePathUtils.java
Co-authored-by: Parham Ghazanfari <pghazanfari53@yahoo.com>
2022-09-19 10:47:20 -05:00
John Joyce
d15518f0d9
fix(browse): Fixing browse path to remove requirement for simple name suffix ( #5634 )
2022-09-07 13:32:38 -07:00
Aseem Bansal
b643b34d30
feat(restore-indices): add endpoint for restore indices, add basic check for graph ( #5805 )
2022-09-02 15:24:40 +05:30
Chris Collins
68fb0d8f06
feat(schema) Show last observed timestamp in the schema tab ( #5348 )
2022-07-07 12:31:01 -07:00
Pedro Silva
bb3208d734
fix(gms) Remove auto-creating status aspects if not present when ingesting metadata ( #5315 )
2022-07-01 15:45:09 -07:00
John Joyce
11356e37ba
feat(ui) Adding support for deleting Tags and Domains via the UI ( #5280 )
...
* Adding support for deleting tags and domains via the UI
* Fixing tests
2022-06-29 19:41:41 -07:00
RyanHolstien
ba21efc7dd
feat(bootstrap): add bootstrap step to clear out unknown aspect rows from the database ( #5148 )
2022-06-21 16:00:16 -07:00
Justin Marozas
97bce80fc9
refactor(metadata-io): introduce a storage-independent in-memory entity aspect model ( #4957 )
2022-05-24 12:40:42 -07:00
RyanHolstien
71c7dc455f
fix(metadata-service): telemetry - fix hardcoded aspect name, suppress errors when producing MAE ( #4981 )
2022-05-24 09:18:20 -07:00
RyanHolstien
c3cfaf8e3c
feat(telemetry): add server side telemetry ( #4925 )
...
Co-authored-by: Kevin Hu <kevinhuwest@gmail.com>
2022-05-17 11:37:45 -07:00
RyanHolstien
65fb49f4b4
feat(versionedDataset): adds a versionStamp to timeline response & adds versionStamp param to dataset graphql ( #4727 )
2022-04-29 12:04:38 -07:00
Eddie (Xiao Di) Li
01a5b13a7a
feat(Cassandra): Add Cassandra implementation of EntityService ( #3286 )
2022-04-25 17:29:24 -07:00
John Joyce
f387901ca8
feat(gms): Entity change events v1 (Platform Event) ( #4687 )
2022-04-19 20:04:14 -07:00
John Joyce
c69310522b
feat(metadata service): Introducing Platform Events ( #4477 )
2022-03-29 18:32:04 -07:00
RyanHolstien
b843014c5e
Oss/urn validation ( #4511 )
2022-03-29 16:44:58 -07:00
Dexter Lee
df69e76289
feat(policy): Add domain based and view based policies ( #4516 )
...
* Add domain based policy and view based policy
* Add new files
* Fix issue
* Fix comment on markdown
* Fix to John's comments
* Update policies.md
* Remove comments
* Fix to John's comments
* Fix policy details page
* Add back notebook privileges
* Fix graphql
Co-authored-by: John Joyce <john@acryl.io>
2022-03-29 12:02:22 -07:00
Pedro Silva
e8f6c4cabd
feat(cli) Changes rollback behaviour to apply soft deletes by default ( #4358 )
...
* Changes rollback behaviour to apply soft deletes by default
Summary:
Addresses feature request: Flag in delete command to only delete aspects touched by an ingestion run; add flag to nuke everything by modifying the default behaviour of a rollback operation which will not by default delete an entity if a keyAspect is being rolled-back.
Instead the key aspect is kept and a StatusAspect is upserted with removed=true, effectively making a soft delete.
Another PR will follow to perform garbage collection on these soft deleted entities.
To keep old behaviour, a new parameter to the cli ingest rollback endpoint: --hard-delete was added.
* Adds restli specs
* Fixes deleteAspect endpoint & adds support for nested transactions
* Enable regression test & fix docker-compose for local development
* Add generated quickstart
* Fix quickstart generation script
* Adds missing var env to docker-compose-without-neo4j
* Sets status removed=true when ingesting resources
* Adds soft deletes for ElasticSearch + soft delete flags across ingestion sub-commands
* Makes elastic search consistent
* Update tests with new behaviour
* apply review comments
* apply review comment
* Forces Elastic search to add documents with status removed false when ingesting
* Reset gradle properties to default
* Fix tests
2022-03-15 12:05:52 -07:00
RyanHolstien
a251d589ed
Feature/oss/update to v2 endpoints ( #4128 )
2022-02-15 14:37:05 -08:00
RyanHolstien
d46c55733e
fix(platform): prevent invalid urns during ingestion ( #4045 )
...
Co-authored-by: Ryan Holstien <ryan@acryl.io>
2022-02-02 22:58:44 -08:00
Aseem Bansal
a56f0661d5
perf(ingest): changes to improve ingest performance a bit ( #3837 )
2022-01-06 08:07:16 -08:00
RyanHolstien
34c27f076b
feat(removeGMA): remove all dependencies on gma libraries ( #3835 )
2022-01-05 17:32:31 -08:00
Dexter Lee
9d04aa1255
feat(mae/mcl): Make ingestAspect produce both MCLs and MAEs ( #3737 )
2022-01-04 18:24:30 -08:00
Dexter Lee
76f9423581
feat(nocode): Add get entities v2 endpoint that can get without snapshot ( #3738 )
2022-01-03 07:19:46 -08:00
Dexter Lee
8e4769f40c
feat(retention): Add retention to Local DB ( #3715 )
2021-12-13 18:18:02 -08:00
Shirshanka Das
842c05980b
feat(metadata-service): support for custom model extensions without forks ( #3630 )
2021-11-28 21:06:27 -08:00
John Joyce
f49666a230
feat(auth): Metadata Service Authentication! ( #3598 )
2021-11-22 16:33:14 -08:00
Gabe Lyons
b87efa43b5
feat(deletes): support deletion of non-snapshot aspects ( #3518 )
...
Co-authored-by: Dexter Lee <dexter@acryl.io>
2021-11-08 16:22:24 -08:00
John Joyce
a92ab66a3a
refactor(nocode): Final part of No-Code cleanup ( #3477 )
2021-10-31 22:06:36 -07:00
Gabe Lyons
7dbd10f072
fix(browse): fix browse for entities without default browse logic ( #3422 )
2021-10-21 11:15:10 -07:00
Dexter Lee
9b54242a6e
fix(datahub-upgrade): Update datahub upgrade to use MCL instead of MAE ( #3411 )
2021-10-18 22:52:48 -07:00
Dexter Lee
d0548408b4
feat(search): redesign search page - left filter pane ( #3337 )
2021-10-07 11:41:29 -07:00
Gabe Lyons
6bf237b38e
feat(graphql): Adding write side validation and tests for add+remove API ( #3259 )
2021-09-20 11:58:31 -07:00
Gabe Lyons
2de4ea9566
feat(business glossary): Add support to add & remove glossary terms from the UI ( #3213 )
2021-09-13 09:16:37 -07:00
John Joyce
ccb09a6966
feat(access control): Fine-Grained Access Control M1 ( #3182 )
2021-09-02 19:05:13 -07:00
John Joyce
f3fc0970f3
refactor(build): Remove unnecessary ext modules. ( #3074 )
2021-08-10 22:48:06 -07:00
John Joyce
352a0abf8d
Introducing TimeSeries Aspects + Dataset Profile (Stats) Aspect ( #2983 )
...
Co-authored-by: Dexter Lee <dexter@acryl.io>
Co-authored-by: Harshal Sheth <hsheth2@gmail.com>
Co-authored-by: Ravindra Lanka <rlanka@acryl.io>
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2021-07-30 17:41:03 -07:00
Gabe Lyons
aa253f5b3b
feat(deletes): add run commands (list, show, rollback) to datahub ingest ( #2960 )
2021-07-29 20:04:40 -07:00
Dexter Lee
8f0f322279
feat(backup): Add restore indices and restore backup tasks ( #2779 )
2021-06-30 16:49:02 -07:00
