yujunjun/OpenMetadata
1
0
Fork 0
mirror of https://github.com/open-metadata/OpenMetadata.git synced 2025-07-23 09:22:18 +00:00
Code Issues Packages Projects Releases Wiki Activity
OpenMetadata/openmetadata-docs/content/v1.3.x/connectors/database/databricks/index.md
Pere Miquel Brull ce15728327
MINOR - Docs 1.3 BETA & Slack Alert (#14961) 
* MINOR - Docs 1.3 BETA & Slack Alert

* MINOR - Docs 1.3 BETA & Slack Alert
2024-01-31 07:16:22 +01:00

4.0 KiB
Raw Blame History

title slug
Databricks /connectors/database/databricks

Databricks

{% multiTablesWrapper %}

Feature Status
Stage PROD
Metadata {% icon iconName="check" /%}
Query Usage {% icon iconName="check" /%}
Data Profiler {% icon iconName="check" /%}
Data Quality {% icon iconName="check" /%}
Owners {% icon iconName="cross" /%}
Tags {% icon iconName="check" /%}
DBT {% icon iconName="check" /%}
Supported Versions Databricks Runtime Version 9+
Feature Status
Lineage {% icon iconName="check" /%}
Table-level {% icon iconName="check" /%}
Column-level {% icon iconName="check" /%}

{% /multiTablesWrapper %}

{% note %} As per the documentation here, note that we only support metadata tag extraction for databricks version 13.3 version and higher. {% /note %}

In this section, we provide guides and references to use the Databricks connector.

Configure and schedule Databricks metadata and profiler workflows from the OpenMetadata UI:

{% partial file="/v1.3/connectors/ingestion-modes-tiles.md" variables={yamlPath: "/connectors/database/databricks/yaml"} /%}

{% partial file="/v1.3/connectors/external-ingestion-deployment.md" /%}

{%inlineCallout icon="description" bold="OpenMetadata 0.12 or later" href="/deployment"%} To deploy OpenMetadata, check the Deployment guides. {%/inlineCallout%}

Unity Catalog

If you are using unity catalog in Databricks, then checkout the Unity Catalog connector.

Metadata Ingestion

{% partial file="/v1.3/connectors/metadata-ingestion-ui.md" variables={ connector: "Databricks", selectServicePath: "/images/v1.3/connectors/databricks/select-service.png", addNewServicePath: "/images/v1.3/connectors/databricks/add-new-service.png", serviceConnectionPath: "/images/v1.3/connectors/databricks/service-connection.png", } /%}

{% stepsContainer %} {% extraContent parentTagName="stepsContainer" %}

Connection Details

  • Host and Port: Enter the fully qualified hostname and port number for your Databricks deployment in the Host and Port field.
  • Token: Generated Token to connect to Databricks.
  • HTTP Path: Databricks compute resources URL.
  • connectionTimeout: The maximum amount of time (in seconds) to wait for a successful connection to the data source. If the connection attempt takes longer than this timeout period, an error will be returned.
  • Catalog: Catalog of the data source(Example: hive_metastore). This is optional parameter, if you would like to restrict the metadata reading to a single catalog. When left blank, OpenMetadata Ingestion attempts to scan all the catalog.
  • DatabaseSchema: databaseSchema of the data source. This is optional parameter, if you would like to restrict the metadata reading to a single databaseSchema. When left blank, OpenMetadata Ingestion attempts to scan all the databaseSchema.

{% partial file="/v1.3/connectors/database/advanced-configuration.md" /%}

{% /extraContent %}

{% partial file="/v1.3/connectors/test-connection.md" /%}

{% partial file="/v1.3/connectors/database/configure-ingestion.md" /%}

{% partial file="/v1.3/connectors/ingestion-schedule-and-deploy.md" /%}

{% /stepsContainer %}

{% partial file="/v1.3/connectors/troubleshooting.md" /%}

{% partial file="/v1.3/connectors/database/related.md" /%}