16 Commits

Author SHA1 Message Date
John Joyce
d96ddec843
refactor(actions): Migrate to use new datahub-actions container (#4903) 2022-05-13 12:11:56 +05:30
Shirshanka Das
f18f453200
moving to dockerhub for actions container (#4497) 2022-03-24 22:24:07 -07:00
Pedro Silva
e8f6c4cabd
feat(cli) Changes rollback behaviour to apply soft deletes by default (#4358)
* Changes rollback behaviour to apply soft deletes by default

Summary:
Addresses feature request: Flag in delete command to only delete aspects touched by an ingestion run; add flag to nuke everything by modifying the default behaviour of a rollback operation which will not by default delete an entity if a keyAspect is being rolled-back.

Instead the key aspect is kept and a StatusAspect is upserted with removed=true, effectively making a soft delete.
Another PR will follow to perform garbage collection on these soft deleted entities.

To keep old behaviour, a new parameter to the cli ingest rollback endpoint: --hard-delete was added.

* Adds restli specs

* Fixes deleteAspect endpoint & adds support for nested transactions

* Enable regression test & fix docker-compose for local development

* Add generated quickstart

* Fix quickstart generation script

* Adds missing var env to docker-compose-without-neo4j

* Sets status removed=true when ingesting resources

* Adds soft deletes for ElasticSearch + soft delete flags across ingestion sub-commands

* Makes elastic search consistent

* Update tests with new behaviour

* apply review comments

* apply review comment

* Forces Elastic search to add documents with status removed false when ingesting

* Reset gradle properties to default

* Fix tests
2022-03-15 12:05:52 -07:00
Aseem Bansal
8c7f63e4f7
feat: change quickstart to use head tag for actions (#4357) 2022-03-09 10:29:10 -08:00
Aseem Bansal
e6a2615c08
feat(quickstart): restart actions pod in case of failures (#4342) 2022-03-08 11:55:21 -08:00
John Joyce
787a7e6dcd
fix(quickstart): Pin actions pod + add volume mount for datahub-frontend (#4318) 2022-03-04 14:05:14 -08:00
John Joyce
271784c9c1
feat(ui): UI-based ingestion (as featured in Dec Townhall) (#3975) 2022-01-27 10:33:12 -08:00
Swaroop Jagadish
90b7b9dddd
fix(ci): moving defaults back to head now that docker builds are green (#3993) 2022-01-27 09:48:01 -08:00
John Joyce
aeabeec133
fix(quickstart): Pin Quickstart version to v0.8.23. (#3983) 2022-01-26 11:36:26 -08:00
Gabe Lyons
3fd3313544
Revert "feat(graph): Make Dgraph a proper Neo4j alternative (#3578)" (#3740) 2021-12-14 10:49:03 -08:00
Enrico Minack
a6deaabfcf
feat(graph): Make Dgraph a proper Neo4j alternative (#3578) 2021-12-13 12:37:59 -08:00
Dexter Lee
2264547a5b
fix(docker): remove unnecessary components from docker-compose (#3073) 2021-08-10 22:46:12 -07:00
Harshal Sheth
8c3d266cf5
fix(quickstart): update compose spec version (#2873)
Also update the defaults to point at the head tag instead of latest.
2021-07-13 12:02:05 -07:00
John Joyce
c7ce817bb3
fix(quickstart): Fixing manual mysql quickstart (#2811) 2021-06-30 15:45:02 -07:00
Gabe Lyons
f4ff5409e3
fix(elastic-as-graph): adding elasticsearch setup back in (#2752) 2021-06-23 18:59:56 -07:00
Gabe Lyons
3f5e371851
feat(graph): support using elasticsearch as graph backend. (#2726) 2021-06-22 16:08:54 -07:00