156 lines
5.8 KiB
Markdown
Raw Normal View History

---
title: Upgrade 0.9 to 0.10
slug: /deployment/upgrade/versions/090-to-010
---
# Upgrade from 0.9 to 0.10
**The 0.10 Release consists of backward-incompatible changes. We do not support database migration from the 0.9.0 release.
Please follow the steps carefully and backup your database before proceeding. 0.10.0 installations require brand new
installation, and we have a migration tool to transfer all your entity descriptions, tags, owners, etc. to the 0.10.0
release**
Please reach out to us on [Slack](https://slack.open-metadata.org), we can schedule a zoom session to help you upgrade your
production instance.
## Requirements
Update the `openmetadata-ingestion` client to the latest available version.
Make sure that you [backed up](/deployment/upgrade/backup-metadata) your metadata.
<Note>
To migrate the metadata from the 0.9 to the 0.10 version you need to ingest all the connectors in the 0.10 version with
the same service name as done in the 0.9 version. For example, if you have ingested snowflake connector in 0.9 version
with service name `Snowflake_Prod` then in the 0.10 version, you need to ingest the snowflake connector again with the
name `Snowflake_Prod`.
</Note>
## Metadata Migration
All connectors are now defined as JSON Schemas. [Here](https://github.com/open-metadata/OpenMetadata/blob/main/catalog-rest-service/src/main/resources/json/schema/entity/services/connections/metadata/openMetadataConnection.json)
you can find the structure to create a connection to Snowflake. In order to create and run a Metadata Ingestion workflow,
we will follow the steps to create a YAML configuration able to connect to the source, process the Entities if needed,
and reach the OpenMetadata server. The workflow is modeled around the following [JSON Schema](https://github.com/open-metadata/OpenMetadata/blob/main/catalog-rest-service/src/main/resources/json/schema/metadataIngestion/workflow.json).
## 1. Setup your 0.10.1 installation
1. OpenMetadata 0.9, keep it running as it is with its own MySQL and ES
2. Install OpenMetadata 0.10.1, make it completely isolated from your 0.9 install new MySQL DB and other ES indexes
3. Install 0.10.1 connectors
4. Run ingestion of all your entities using the new connector. Keep the service names the same as your 0.9
installations. Once all the entities are ingested into OpenMetadata 0.10.1. For example, if you have ingested
snowflake connector in 0.9 version with service name Snowflake_Prod then in the 0.10 version, you need to ingest the
snowflake connector again with the name `Snowflake_Prod`.
5. Run migration using the newly installed connector
## 2. Define the YAML Config
This is a sample config:
```yaml
source:
type: migrate
serviceName: local_metadata
serviceConnection:
config:
type: OpenMetadata
hostPort: http://<hostport of 0.9.0 Openmetadata Server>/api
authProvider: no-auth
includeTables: true
includeUsers: true
includeTopics: true
includePipelines: true
includeTags: true
includeGlossaryTerms: true
includeMessagingServices: true
includeDatabaseServices: true
includePipelineServices: true
enableVersionValidation: false
limitRecords: 100000
sourceConfig:
config:
enableDataProfiler: false
stage:
type: migrate
config:
dirPath: <Directory Path to store data>
bulkSink:
type: migrate
config:
dirPath: <Directory Path to store data>
workflowConfig:
openMetadataServerConfig:
hostPort: http://<hostport of 0.10.x Openmetadata Server>/api
authProvider: no-auth
```
### Source Configuration - Service Connection
You can find all the definitions and types for the `serviceConnection` [here](https://github.com/open-metadata/OpenMetadata/blob/main/catalog-rest-service/src/main/resources/json/schema/entity/services/connections/database/clickhouseConnection.json).
- `hostPort`: Enter the OpenMetadata 0.9 Server Config. Must include API end point ex: `http://localhost:8585/api`
- `includeTables`: Include tables for migration
- `includeUsers`: Include users for migration
- `includeTopics`: Include topics for migration
- `includePipelines`: Include pipelines for migration
- `includeGlossaryTerms`: include glossary terms for migration
- `includeTags`: Include tags for migration.
- `includeMessagingServices`: Include messaging services for migration.
- `includeDatabaseServices`: Include database services for migration.
- `enableVersionValidation`: Enable the server OpenMetadata and client OpenMetadata version validation. For migrating the metadata this has to be set to `false`
- `limitRecords`: Limit the number records that is fetched by 0.9 OpenMetadata API (1 to 1000000, default = 10).
### Stage Configuration
```yaml
stage:
type: migrate
config:
dirPath: <Directory Path to store data>
```
The data while migrating from the 0.9 to 0.10 version needs to be stored in a file directory. in the dir Path field
enter a valid file directory path to store the metadata.
### BulkSink Configuration
```yaml
bulkSink:
type: migrate
config:
dirPath: <Directory Path to store data>
```
The `dirPath` needs to be set as the same passed in stage config, which contains the file directory path where the
metadata is stored.
### Workflow Configuration
The main property here is the `openMetadataServerConfig`, where you can define the host and security provider of your
OpenMetadata installation.
For a simple, local installation using our docker containers, this looks like:
```yaml
workflowConfig:
openMetadataServerConfig:
hostPort: http://<hostport of 0.10.x Openmetadata Server>/api
authProvider: no-auth
```
<Note>
In `openMetadataServerConfig` you need to pass details about the 0.10 OpenMetadata Server.
</Note>
## 3. Run with the CLI
First, we will need to save the YAML file. Afterward, and with all requirements installed, we can run:
```commandline
metadata ingest -c <path-to-yaml>
```