* add job info as aspect of a dataset
* add job urn def., aspect and entity
* job entity with upstream and downstream lineage
* use job urn in upstream & downstream
* add Job entity rest APIs
* rest.li api, impl and factory for job entity
* code cleanup
* use pdl; onboard data process entity
* add es index json
* fix gradlew build ignored tasks
* add a comment about data process info field
* fix style warning issues
* update content based on PR
* checked in generated snapshot json
* updated based on PR feedback
* update data process data format
* updated based on code review feedback
* revert back gms & mce-job docker image
* delete temp files
* update based pr feedback
* file name and a typo
* format with linkedin style
Co-authored-by: Liangjun <liajiang@expediagroup.com>
* build(docker): refactor docker build scripts
- add "build" option to docker-compose files to simplify rebuilding of images
- create "start.sh" script so it's easier to override "command" in the quickstart's docker-compose file
- use dockerize to wait for requisite services to start up
- add a dedicated Dockerfile for kafka-setup
This fixes https://github.com/linkedin/datahub/issues/1549 & https://github.com/linkedin/datahub/issues/1550
- add "build" option to docker-compose file to simplify rebuilding of images
- move command from docker-compose.yml to Dockerfile
- add ingestion.sh script to simplify quickstart instruction and to reduce confusion
As we're now utilize GitHub Actions to build & publish docker images to docker hub under linkedin org
Also allow overriding image tags via DATAHUB_VERSION environment variable
Slack has separate links for joining (invitation) and logging in. Slack seems to log people out after some time, so we should have both links to make it easier to log back in for the people that don't remember that our slack space is called "datahubspace".
* feat(urn): add AzkabanFlow and AzkabanJob urn
Add AzkabanFlow urn to represent an azkaban flow defined on a specific cluster
and inside a project.
Add AzkabanJob urn to represent an azkaban job defined as part of an azkaban
flow.
For more details on Azkaban flow and job see - https://azkaban.readthedocs.io/en/latest/
