OpenMetadata/openmetadata-docs/content/v1.0.0/deployment/upgrade/docker.md

title	slug
Upgrade on Docker	/deployment/upgrade/docker

Upgrade on Docker

To run OpenMetadata with Docker, you can simply download the docker-compose.yml file. Optionally, we added some Named Volumes to handle data persistence.

{% note %}

You can find more details about Docker deployment here

{% /note %}

Below we have highlighted the steps needed to upgrade to the latest version with Docker. Make sure to also look here for the specific details related to upgrading to 1.0.0

{% note noteType="Warning" %}

It is advised to go through openmetadata release notes before starting the upgrade process.

{% /note %}

Upgrade from 0.13 to 1.0.0

Your production deployment should go from stable version to stable version. This translated to moving from 0.13 to 1.0.0 to get the latest stable OpenMetadata release.

Let's go through the required steps:

1. Backup Existing OpenMetadata Data using Metadata CLI

• Make sure your instance is connected to the Database server
• Create a virtual environment to install an upgraded metadata version to run the backup command:

   python -m venv venv
   source venv/bin/activate
   pip install openmetadata-ingestion~=1.0.0
  

• Validate the installed metadata version with python -m metadata --version, which should tell us that we are indeed at 1.0.0. Notice the python -m metadata vs. metadata.
• Run the backup using the updated metadata CLI:

  python -m metadata backup -u openmetadata_user -p openmetadata_password -H mysql -d openmetadata_db --port 3306
  

  if using Postgres:

  python -m metadata backup -u openmetadata_user -p openmetadata_password -H postgresql -d openmetadata_db --port 5432 -s public
  

• This will generate the .sql file which can be used for the backup In our case, the backup file was named openmetadata_202212201528_backup.sql. You can copy the name from the backup command output.

2. Replace the docker compose file

• Stop the running compose deployment with below command

docker compose down

• Download the Docker Compose Service File from OpenMetadata Github Release page here
• Replace the existing Docker Compose Service File with the one downloaded from the above step

{% note %}

Please make sure to go through breaking changes and release highlights.

{% /note %}

• Start the Docker Compose Service with the below command

docker compose -f docker-compose.yml up -d

3. Re-index all your metadata

Go to Settings -> OpenMetadata -> Search

{% image src="/images/v1.0.0/deployment/upgrade/elasticsearch-re-index.png" alt="create-project" caption="Reindex" /%}

Click on reindex all.

In the dialog box choose Recreate Indexes to All.

{% image src="/images/v1.0.0/deployment/upgrade/reindex-ES.png" alt="create-project" caption="Reindex" /%}

---

Upgrade ingestion patch versions

During the release lifespan we may publish new patch versions of openmetadata-ingestion. If you deployed the ingestion container and require one of the fixes or improvements from a new patch release, there's usually no need to re-deploy the full ingestion container.

{% note %}

Note that this process will only work if we are moving from PATCH versions. For example: 0.13.1.1 -> 0.13.1.2.

This method won't work when upgrading from 0.13.1.X -> 0.13.2.X, as that will also require to upgrade the server version.

{% /note %}

The steps to follow are:

• Connect to the ingestion container. If using our docker compose files or metadata docker CLI, this translates to

    docker exec -it openmetadata_ingestion bash
  

• Validate your metadata version via metadata --version. You will get back something like:

  metadata 0.13.1.5 from /home/airflow/.local/lib/python3.9 (python 3.9)
  

• Upgrade the openmetadata-ingestion package via pip install "openmetadata-ingestion==0.13.1.X",for example,

  pip install "openmetadata-ingestion==0.13.1.7" 
  

  You can find the list of all released versions of the openmetadata-ingestion package here.
• Exit the container by typing exit.
• Restart the ingestion container with docker restart openmetadata_ingestion. This will need a few minutes to to stop the container and start it again. Now, Airflow will start with the upgraded metadata version.
• Connect to the ingestion container and validate the metadata version:

  docker exec -it openmetadata_ingestion bash
  

  • metadata version: where we expect to get the same version that was previously installed.

Troubleshooting

Permission Denied when running metadata openmetadata-imports-migration

If you have a Permission Denied error thrown when running metadata openmetadata-imports-migration --change-config-file-path you might need to change the permission on the /opt/airflow/dags folder. SSH into the ingestion container and check the permission on the folder running the below commands

ls -l /opt/airflow

ls -l /opt/airflow/dags

both the dags folder and the files inside dags/ should have airflow root permission. if this is not the case simply run the below command

chown -R airflow:root /opt/airflow/dags

Broken DAGs can't load config file: Permission Denied

You might need to change the permission on the /opt/airflow/dag_generated_config folder. SSH into the ingestion container and check the permission on the folder running the below commands

ls -l /opt/airflow

ls -l /opt/airflow/dag_generated_config

both the dags folder and the files inside dags/ should have airflow root permission. if this is not the case simply run the below command

chown -R airflow:root /opt/airflow/dag_generated_config