yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-07-12 19:48:26 +00:00
Code Issues Packages Projects Releases Wiki Activity
OpenMetadata/openmetadata-docs/content/v1.3.x/deployment/backup-restore-metadata-sql-dump-tools.md
Imri Paran aade838020
Fixes #15388: Use native backup tools (#15393) 
* feat: use native backup tools

1. added mysqldump 8.3 to the ingestion container.
2. documented how to use native tools to back up and restore.
3. added deprecated message on the cli backup and restore.

* added deprecation notice for 1.3 backup

* removed 1.3.x deprecation notice

* added another backup page in 1.3 introducing SQL dump tools

* added --set-gtid-purged=OFF to the mysql dump process
2024-03-12 06:23:05 +01:00

2.8 KiB
Raw Blame History

title slug
Backup Metadata /deployment/backup-restore-metadata-sql-dump-tools

Backup & Restore Metadata

Introduction

Since version 1.3.0, OpenMetadata's encourages using the builtin-tools for creating logical backups of the metadata:

  • mysqldump for MySQL
  • pg_dump for Postgres

Requirements

  • mysqldump 8.3 or higher (ingestion container is shipped with mysqldump 8.3)
  • pg_dump 13.3 or higher

Example

Start a local instance of OpenMetadata using the docker-compose file provided in the repository. Then, we can use the following commands to backup the metadata:

MySQL

1. Start a local docker deployment

docker/run_local_docker.sh

Ingest some data...

2. Backup and Restore

BACKUP_FILE="backup_$(date +%Y%m%d%H%M).sql"
DOCKER_COMPOSE_FILE="docker/development/docker-compose.yml"
# backup
docker compose -f $DOCKER_COMPOSE_FILE exec ingestion mysqldump --set-gtid-purged=OFF --no-tablespaces -u openmetadata_user -popenmetadata_password -h mysql -P 3306 openmetadata_db > $BACKUP_FILE
# create the restore database
docker compose -f $DOCKER_COMPOSE_FILE exec mysql  mysql -u root -ppassword -e "create database restore;"
docker compose -f $DOCKER_COMPOSE_FILE exec mysql  mysql -u root -ppassword -e "grant all privileges on restore.* to 'openmetadata_user'@'%';"
docker compose -f $DOCKER_COMPOSE_FILE exec mysql  mysql -u root -ppassword -e "flush privileges;"
# restore from the backup
docker compose -f $DOCKER_COMPOSE_FILE exec -T ingestion mysql -u openmetadata_user -popenmetadata_password -h mysql -P 3306 restore < $BACKUP_FILE

3. Restart the docker deployment with the restored database

export OM_DATABASE=restore
docker compose -f $DOCKER_COMPOSE_FILE up -d

PostgreSQL

1. Start a local docker deployment

docker/run_local_docker.sh -d postgres

Ingest some data...

2. Backup and Restore

BACKUP_FILE="backup_$(date +%Y%m%d%H%M).sql"
DOCKER_COMPOSE_FILE="docker/development/docker-compose-postgres.yml"
# backup
docker compose -f $DOCKER_COMPOSE_FILE exec -e PGPASSWORD=openmetadata_password ingestion pg_dump -U openmetadata_user -h postgresql -d openmetadata_db > $BACKUP_FILE
# create the restore database
docker compose -f $DOCKER_COMPOSE_FILE exec -e PGPASSWORD=openmetadata_password postgresql psql -U postgres -c "create database restore;"
docker compose -f $DOCKER_COMPOSE_FILE exec -e PGPASSWORD=openmetadata_password postgresql psql -U postgres -c "ALTER DATABASE restore OWNER TO openmetadata_user;"
# restore from the backup
docker compose -f $DOCKER_COMPOSE_FILE exec -e PGPASSWORD=openmetadata_password -T ingestion psql -U openmetadata_user -h postgresql -d restore < $BACKUP_FILE

3. Restart the docker deployment with the restored database

export OM_DATABASE=restore
docker compose -f $DOCKER_COMPOSE_FILE up -d