b8e18b0b5d
* Make docker files easier to use during development. During development it quite nice to have docker work with locally built code. This allows you to launch all services very quickly, with your changes, and optionally with debugging support. Changes made to docker files: - Removed all redundant docker-compose files. We now have 1 giant file, and smaller files to use as overrides. - Remove redundant README files that provided little information. - Rename docker/<dir> to match the service name in the docker-compose file for clarity. - Move environment variables to .env files. We only provide dev / the default environment for quickstart. - Add debug options to docker files using multistage build to build minimal images with the idea that built files will be mounted instead. - Add a docker/dev.sh script + compose file to easily use the dev override images (separate tag; images never published; uses debug docker files; mounts binaries to image). - Added docs/docker documentation for this.
3.1 KiB
Using Docker Images During Development
We've created a special docker-compose.dev.yml override file that should configure docker images to be easier to use
during development.
Normally, you'd rebuild your images from scratch with docker-compose build (or docker-compose up --build). However,
this takes way too long for development. It has to copy the entire repo to each image and rebuild it there.
The docker-compose.dev.yml file bypasses this problem by mounting binaries, startup scripts, and other data to
special, slimmed down images (of which the Dockerfile is usually defined in <service>/debug/Dockerfile). Mounts work
both ways, so they should also try to mount log directories on the container, so that they are easy to read on your
local machine without needing to inspect the running container (especially if the app crashes and the container stops!).
We highly recommend you just invoke the docker/dev.sh script we've included. It is pretty small if you want to read it
to see what it does, but it ends up using our docker-compose.dev.yml file.
Debugging
The default dev images, while set up to use your local code, do not enable debugging by default. To enable debugging, you need to make two small edits (don't check these changes in!).
- Add the JVM debug flags to the environment file for the service.
- Assign the port in the docker-compose file.
For example, to debug dathaub-gms:
# Add this line to docker/datahub-gms/env/dev.env. You can change the port and/or change suspend=n to y.
JAVA_TOOL_OPTIONS -agentlib:jdwp=transport=dt_socket,address=5005,server=y,suspend=n
# Change the definition in docker/docker-compose.dev.yml to this
datahub-gms:
image: linkedin/datahub-gms:debug
build:
context: datahub-gms/debug
dockerfile: Dockerfile
ports: # <--- Add this line
- "5005:5005" # <--- And this line. Must match port from environment file.
volumes:
- ./datahub-gms/start.sh:/datahub/datahub-gms/scripts/start.sh
- ../gms/war/build/libs/:/datahub/datahub-gms/bin
Tips for People New To Docker
Conflicting containers
If you ran docker/quickstart.sh before, your machine may already have a container for DataHub. If you want to run
docker/dev.sh instead, ensure that the old container is removed by running docker container prune. The opposite also
applies.
Note this only removes containers, not images. Should still be fast to switch between these once you've launched both at least once.
Running a specific service
docker-compose up will launch all services in the configuration, including dependencies, unless they're already
running. If you, for some reason, wish to change this behavior, check out these example commands.
docker-compose -p datahub -f docker-compose.yml -f docker-compose.overrides.yml -f docker-compose.dev.yml up datahub-gms
Will only start datahub-gms and its dependencies.
docker-compose -p datahub -f docker-compose.yml -f docker-compose.overrides.yml -f docker-compose.dev.yml up --no-deps datahub-gms
Will only start datahub-gms, without dependencies.