Deploying OpenMetadata in Docker is a great start!
Before starting with the deployment make sure you follow all the below Prerequisites.
## Prerequisites
### Docker (version 20.10.0 or greater)
[Docker](https://docs.docker.com/get-started/overview/) is an open-source platform for developing, shipping, and running applications. It enables you to separate your applications from your infrastructure, so you can deliver software quickly using OS-level virtualization. It helps deliver software in packages called Containers.
To check what version of Docker you have, please use the following command.
```commandline
docker --version
```
If you need to install Docker, please visit [Get Docker](https://docs.docker.com/get-docker/).
### Docker Compose (version v2.2.3 or greater)
The Docker compose package enables you to define and run multi-container Docker applications. The compose command integrates compose functions into the Docker platform, making them available from the Docker command-line interface ( CLI). The Python packages you will install in the procedure below use compose to deploy OpenMetadata.
- **MacOS X**: Docker on MacOS X ships with compose already available in the Docker CLI.
- **Linux**: To install compose on Linux systems, please visit the Docker CLI command documentation and follow the
instructions.
To verify that the docker compose command is installed and accessible on your system, run the following command.
```commandline
docker compose version
```
Upon running this command you should see output similar to the following.
```commandline
Docker Compose version v2.2.3
```
### Install Docker Compose Version 2 on Linux
Follow the instructions [here](https://docs.docker.com/compose/cli-command/#install-on-linux) to install docker compose version 2
1. Run the following command to download the current stable release of Docker Compose
- First download the docker-compose.yml file from the release page [here](https://github.com/open-metadata/OpenMetadata/releases/latest). The latest version is at the top of the page
In a few seconds, you should be able to access the OpenMetadata UI at [http://localhost:8585](http://localhost:8585)
## Port Mapping / Port Forwarding
### For OpenMetadata-Server
We are shipping the OpenMetadata server and UI at `8585`, and the ingestion container (Airflow) at `8080`. You can
change the port number's according to your requirement. As an example, You could
update the ports to serve OpenMetadata Server and UI at port `80`
To achieve this
- You just have to update the ports mapping of the openmetadata-server in the `docker-compose.yml` file under `openmetadata-server` docker service section.
```yaml
ports:
- "80:8585"
```
- Once the port is updated if there are any containers running remove them first using `docker compose down` command and then recreate the containers once again by below command
```commandline
docker compose up --build -d
```
### For Ingestion-Server
We are shipping the OpenMetadata server and UI at `8585`, and the ingestion container (Airflow) at `8080`. You can
change the port number's according to your requirement. As an example, You could
update the ports to serve Ingestion Server and UI at port `80`
To achieve this
- You just have to update the ports mapping of the openmetadata-server in the `docker-compose.yml` file under `ingestion-server` docker service section.
```yaml
ports:
- "80:8080"
```
- Also update the Airflow environment variable in openmetadata-server section
```commandline
AIRFLOW_HOST: '<AIRFLOW_HOST:-<AIRFLOW_HOST:80>'
```
- Once the port is updated if there are any containers running remove them first using `docker compose down` command and then recreate the containers once again by below command
```commandline
docker compose up --build -d
```
## PROD Deployment of OpenMetadata Using Docker
If you are planning on going to PROD, we recommend to validate below points:
- MySQL and OpenSearch (ElasticSearch) are available.
- OpenMetadata-Server require the minimum configuration of 2vCPU and 6Memory (GiB)
- OpenMetadata-Ingestion require the minimum configuration of 2vCPU and 8Memory (GiB)
- We also recommend to bind Docker Volumes for data persistence. Minimum disk space required would be 128 Gib. Learn how to do so [here](/deployment/docker/volumes).
### Steps for Deploying Ingestion
- Download the docker-compose.yml file from the release page [here](https://github.com/open-metadata/OpenMetadata/releases).
- Update the environment variables below for OpenMetadata-Ingestion Docker Compose backed systems to connect with Database.
```
# MySQL Environment Variables for ingestion service
Once the environment variables values with the RDS are updated then provide this environment variable file as part of docker compose command.
```
docker compose --env-file ./config/.env.prod up -d openmetadata_ingestion
```
### Steps for Deploying OpenMetadata-Server
- Download the docker-compose.yml file from the release page [here](https://github.com/open-metadata/OpenMetadata/releases).
- Update the environment variables below for OpenMetadata-Ingestion Docker Compose backed systems to connect with Database and ElasticSearch and Ingestion.
Once the environment variables values with the RDS are updated then provide this environment variable file as part of docker compose command.
```
docker compose --env-file ./config/.env.prod up -d openmetadata_server
```
## Run OpenMetadata with AWS Services
If you are running OpenMetadata in AWS, it is recommended to use [Amazon RDS](https://docs.aws.amazon.com/rds/index.html) and [Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/?id=docs_gateway).
For Production Systems, we recommend Amazon RDS to be in Multiple Availibility Zones. For Amazon OpenSearch (or ElasticSearch) Service, we recommend Multiple Availibility Zones with minimum 3 Master Nodes.
Once you have the RDS and OpenSearch Services Setup, you can update the environment variables below for OpenMetadata Docker Compose backed systems to connect with Database and ElasticSearch.
```
# MySQL Environment Variables
DB_DRIVER_CLASS='com.mysql.cj.jdbc.Driver'
DB_SCHEME='mysql'
DB_USE_SSL='true'
DB_USER_PASSWORD='<DATABASE_USER_PASSWORD>'
DB_HOST='<DATABASE_HOST_NAME>'
DB_USER='<DATABASE_USER_NAME>'
OM_DATABASE='<DATABASE_NAME>'
DB_PORT='<DATABASE_PORT>'
# ElasticSearch Environment Variables
ELASTICSEARCH_SOCKET_TIMEOUT_SECS='60'
ELASTICSEARCH_USER='<ELASTICSEARCH_USERNAME>'
ELASTICSEARCH_CONNECTION_TIMEOUT_SECS='5'
ELASTICSEARCH_PORT='443'
ELASTICSEARCH_SCHEME='https'
ELASTICSEARCH_BATCH_SIZE='10'
ELASTICSEARCH_HOST='<ELASTICSEARCH_HOST_URL>'
ELASTICSEARCH_PASSWORD='<ELASTICSEARCH_PASSWORD>'
```
Replace the environment variables values with the RDS and OpenSearch Service ones and then provide this environment variable file as part of docker compose command.
```
docker compose --env-file ./config/.env.prod up -d openmetadata_server
If your openmetadata Docker Compose logs speaks about the below issue -
```
Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "AsyncAppender-Worker-async-file-appender"
Exception in thread "pool-5-thread-1" java.lang.OutOfMemoryError: Java heap space
Exception in thread "AsyncAppender-Worker-async-file-appender" java.lang.OutOfMemoryError: Java heap space
Exception in thread "dw-46" java.lang.OutOfMemoryError: Java heap space
Exception in thread "AsyncAppender-Worker-async-console-appender" java.lang.OutOfMemoryError: Java heap space
```
This is due to the default JVM Heap Space configuration (1 GiB) being not enough for your workloads. In order to resolve this issue, head over to your custom openmetadata environment variable file and append the below environment variable
```
#environment variable file
OPENMETADATA_HEAP_OPTS="-Xmx2G -Xms2G"
```
The flag `Xmx` specifies the maximum memory allocation pool for a Java virtual machine (JVM), while `Xms` specifies the initial memory allocation pool.
Restart the OpenMetadata Docker Compose Application using `docker compose --env-file my-env-file -f docker-compose.yml up -d` which will recreate the containers with new environment variable values you have provided.
