26 Commits

Author SHA1 Message Date
Guillaume Cernier
ef1ada118d
fix(docker-compose): make startup more robust with deterministic services' dependencies (#7880) 2023-05-08 16:42:15 -05:00
Peter Szalai
e9f060dae3
Add setup job labels to compose files (#7473) 2023-03-02 15:05:16 +01:00
Peter Szalai
0a43d250d0
fix(dev.sh): Add context to kafka-setup build (#7234) 2023-02-03 17:45:42 +05:30
david-leifker
39920bb00f
feat(elasticsearch): Elasticsearch improvements (#6894) 2023-01-31 18:44:37 -06:00
Peter Szalai
8a2230aea4
fix(quickstart): adding back kafka-setup (#7181)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2023-01-31 00:34:36 +01:00
Pedro Silva
bef59b0928
feat(quickstart): Remove kafka-setup as a hard deployment requirement (#7073) 2023-01-24 16:12:57 +00:00
david-leifker
637b540fa6
fix(kafka-setup): fix return code check (#6782)
* fix(kafka-setup): parallelize topic creation
* Remove -setup from docker compose (not services)
2022-12-16 13:39:18 -06:00
leifker
203a6ff57b
refactor(java11) - convert most modules to java 11 (#5836)
Co-authored-by: Shirshanka Das <shirshanka@apache.org>
2022-09-25 08:39:22 -07:00
Harshal Sheth
13f20ad2c3
fix(docker): add missing port mappings for non-neo4j quickstart (#5799)
* fix(docker): add missing port mappings for non-neo4j quickstart

* undo cassandra
2022-09-05 15:32:19 +05:30
Aseem Bansal
252992f15e
fix(ci): intermittent failure in github actions (#5452) 2022-07-25 23:02:02 -07:00
John Joyce
d96ddec843
refactor(actions): Migrate to use new datahub-actions container (#4903) 2022-05-13 12:11:56 +05:30
Shirshanka Das
f18f453200
moving to dockerhub for actions container (#4497) 2022-03-24 22:24:07 -07:00
Pedro Silva
e8f6c4cabd
feat(cli) Changes rollback behaviour to apply soft deletes by default (#4358)
* Changes rollback behaviour to apply soft deletes by default

Summary:
Addresses feature request: Flag in delete command to only delete aspects touched by an ingestion run; add flag to nuke everything by modifying the default behaviour of a rollback operation which will not by default delete an entity if a keyAspect is being rolled-back.

Instead the key aspect is kept and a StatusAspect is upserted with removed=true, effectively making a soft delete.
Another PR will follow to perform garbage collection on these soft deleted entities.

To keep old behaviour, a new parameter to the cli ingest rollback endpoint: --hard-delete was added.

* Adds restli specs

* Fixes deleteAspect endpoint & adds support for nested transactions

* Enable regression test & fix docker-compose for local development

* Add generated quickstart

* Fix quickstart generation script

* Adds missing var env to docker-compose-without-neo4j

* Sets status removed=true when ingesting resources

* Adds soft deletes for ElasticSearch + soft delete flags across ingestion sub-commands

* Makes elastic search consistent

* Update tests with new behaviour

* apply review comments

* apply review comment

* Forces Elastic search to add documents with status removed false when ingesting

* Reset gradle properties to default

* Fix tests
2022-03-15 12:05:52 -07:00
Aseem Bansal
8c7f63e4f7
feat: change quickstart to use head tag for actions (#4357) 2022-03-09 10:29:10 -08:00
Aseem Bansal
e6a2615c08
feat(quickstart): restart actions pod in case of failures (#4342) 2022-03-08 11:55:21 -08:00
John Joyce
787a7e6dcd
fix(quickstart): Pin actions pod + add volume mount for datahub-frontend (#4318) 2022-03-04 14:05:14 -08:00
John Joyce
271784c9c1
feat(ui): UI-based ingestion (as featured in Dec Townhall) (#3975) 2022-01-27 10:33:12 -08:00
Swaroop Jagadish
90b7b9dddd
fix(ci): moving defaults back to head now that docker builds are green (#3993) 2022-01-27 09:48:01 -08:00
John Joyce
aeabeec133
fix(quickstart): Pin Quickstart version to v0.8.23. (#3983) 2022-01-26 11:36:26 -08:00
Gabe Lyons
3fd3313544
Revert "feat(graph): Make Dgraph a proper Neo4j alternative (#3578)" (#3740) 2021-12-14 10:49:03 -08:00
Enrico Minack
a6deaabfcf
feat(graph): Make Dgraph a proper Neo4j alternative (#3578) 2021-12-13 12:37:59 -08:00
Dexter Lee
2264547a5b
fix(docker): remove unnecessary components from docker-compose (#3073) 2021-08-10 22:46:12 -07:00
Harshal Sheth
8c3d266cf5
fix(quickstart): update compose spec version (#2873)
Also update the defaults to point at the head tag instead of latest.
2021-07-13 12:02:05 -07:00
John Joyce
c7ce817bb3
fix(quickstart): Fixing manual mysql quickstart (#2811) 2021-06-30 15:45:02 -07:00
Gabe Lyons
f4ff5409e3
fix(elastic-as-graph): adding elasticsearch setup back in (#2752) 2021-06-23 18:59:56 -07:00
Gabe Lyons
3f5e371851
feat(graph): support using elasticsearch as graph backend. (#2726) 2021-06-22 16:08:54 -07:00