15 Commits

Author SHA1 Message Date
david-leifker
68b188a011
build(gradle): Update gradle.properties (#11458) 2024-09-23 17:38:20 -05:00
Harshal Sheth
4532733093
build: allow gradle daemon (#11437) 2024-09-19 18:05:22 -05:00
david-leifker
731c29e675
feat(graph-retriever): implement graph retriever (#10241) 2024-04-16 10:12:48 -05:00
david-leifker
ed10a8d8cc
feat(search): search access controls (#9892)
Co-authored-by: Chris Collins <chriscollins3456@gmail.com>
2024-02-28 16:57:26 -06:00
Harshal Sheth
9f79f44dd6
build: enable gradle caching (#9525) 2023-12-27 20:05:17 -05:00
Harshal Sheth
d231497603
build: increase gradle retries (#9091) 2023-10-31 21:37:11 -07:00
Harsha Mandadi
bf36c935fa
feat(ingest/s3): support path_specs of different S3 buckets in the same recipe (#7514) 2023-03-14 21:55:57 -07:00
mohdsiddique
c31e380489
feat(apache-ranger): Apache Ranger Authorizer support in datahub-gms (#4999) 2022-08-01 09:47:08 -07:00
Shirshanka Das
bfed6b8029
fix(ci): split out expensive build steps, increase memory (#4825) 2022-05-04 22:21:56 -07:00
Pedro Silva
e8f6c4cabd
feat(cli) Changes rollback behaviour to apply soft deletes by default (#4358)
* Changes rollback behaviour to apply soft deletes by default

Summary:
Addresses feature request: Flag in delete command to only delete aspects touched by an ingestion run; add flag to nuke everything by modifying the default behaviour of a rollback operation which will not by default delete an entity if a keyAspect is being rolled-back.

Instead the key aspect is kept and a StatusAspect is upserted with removed=true, effectively making a soft delete.
Another PR will follow to perform garbage collection on these soft deleted entities.

To keep old behaviour, a new parameter to the cli ingest rollback endpoint: --hard-delete was added.

* Adds restli specs

* Fixes deleteAspect endpoint & adds support for nested transactions

* Enable regression test & fix docker-compose for local development

* Add generated quickstart

* Fix quickstart generation script

* Adds missing var env to docker-compose-without-neo4j

* Sets status removed=true when ingesting resources

* Adds soft deletes for ElasticSearch + soft delete flags across ingestion sub-commands

* Makes elastic search consistent

* Update tests with new behaviour

* apply review comments

* apply review comment

* Forces Elastic search to add documents with status removed false when ingesting

* Reset gradle properties to default

* Fix tests
2022-03-15 12:05:52 -07:00
Swaroop Jagadish
d23a5bb725
feat(spark-lineage): simplified jars, config, auto publish to maven (#3924) 2022-01-20 00:48:09 -08:00
Harshal Sheth
91e5fd457a
fix(build): increase retries for dependency fetches (#2867) 2021-07-13 09:11:42 -07:00
Jyoti Wadhwani
ad6f1653e1
metadata-models 62.0.3 -> 72.0.8 (#1693) 2020-06-11 10:21:51 -07:00
Mars Lan
4a77576d31
build: pegasus plugin doesn't work well with gradle caching (#1622)
Snapshot & IDL will not be regenerated consistently when gradle caching is enabled.
2020-04-02 19:27:24 -07:00
Kerem Sahin
3765c1d234 Add gradle.properties & enable parallel Gradle build 2020-03-11 18:36:12 -07:00