Pere Miquel Brull 7140b344d0
Docs migration - Deployment and upgrades (#5984)
Docs migration - Deployment and upgrades (#5984)
2022-07-09 20:31:41 +02:00

2.4 KiB

title slug
Upgrade 0.10 to 0.11 /deployment/upgrade/versions/010-to-011

Upgrade from 0.10 to 0.11

Upgrading from 0.10 to 0.11 can be done directly on your instances. This page will list a couple of details that you should take into consideration when running the upgrade.

Highlights

Data Indexing

0.11 comes with changes to the Elasticsearch indexes. In this case, indexes will need to be recreated. You can do so by following this guide: How to fully index OpenMetadata into Elasticsearch.

Ingestion Pipelines

The process of how the Ingestion Pipelines retrieve the service data has been updated. During the upgrade, your services will be automatically updated if their schema changed, and the Ingestion Pipelines will be removed.

From the Services > Ingestion tab, you can then re-create and deploy the Ingestion Pipelines.

Clear the Cache!

Are you having some issues with the UI after upgrading to 0.11? Make sure you clean the browser cache to pick up all the updated components and definitions properly.

Metadata Workflow Updates

In the 0.11.0 release, we moved around a couple of things from the metadata ingestion workflow, specifically in the sourceConfig:

  • generateSampleData: this now happens on the profiler workflow. This option is removed from the metadata pipeline
  • sampleDataQuery is not needed anymore, as in the profiler workflow, we use real automated sampling
  • enableDataProfiler is part of profiler workflow. As running the profiler takes a longer time, we moved the profiler into a separate ingestion workflow. This way, one can schedule the metadata extraction workflows to be more frequent and get the latest metadata. Please refer to the examples here to schedule a profiler workflow.

Service Connection Updates

  • BigQuery
    • Removed: username, projectId, enablePolicyTagImport, database
    • Added: taxonomyLocation, usageLocation
  • Snowflake
    • Added: queryTag
  • Redshift:
    • Added: ingestAllDatabases
  • Oracle, Mysql, MariaDB, Hive, DB2, Clickhouse, SingleStore, Trino, Presto:
    • Renamed from database to databaseSchema
  • Databricks, Athena:
    • Removed: database
  • Dashboards:
    • Removed dbServiceName from the connection information to the sourceConfig